Using odds ratios to detect differential item functioning

Kuan Yu JIN, Hui-Fang CHEN, Wen Chung WANG

Research output: Contribution to journalArticlespeer-review

15 Citations (Scopus)

Abstract

Differential item functioning (DIF) makes test scores incomparable and substantially threatens test validity. Although conventional approaches, such as the logistic regression (LR) and the Mantel–Haenszel (MH) methods, have worked well, they are vulnerable to high percentages of DIF items in a test and missing data. This study developed a simple but effective method to detect DIF using the odds ratio (OR) of two groups' responses to a studied item. The OR method uses all available information from examinees' responses, and it can eliminate the potential influence of bias in the total scores. Through a series of simulation studies in which the DIF pattern, impact, sample size (equal/unequal), purification procedure (with/without), percentages of DIF items, and proportions of missing data were manipulated, the performance of the OR method was evaluated and compared with the LR and MH methods. The results showed that the OR method without a purification procedure outperformed the LR and MH methods in controlling false positive rates and yielding high true positive rates when tests had a high percentage of DIF items favoring the same group. In addition, only the OR method was feasible when tests adopted the item matrix sampling design. The effectiveness of the OR method with an empirical example was illustrated. Copyright © 2018 The Author(s).
Original languageEnglish
Pages (from-to)613-629
JournalApplied Psychological Measurement
Volume42
Issue number8
Early online dateMar 2018
DOIs
Publication statusPublished - Nov 2018

Citation

Jin, K.-Y., Chen, H.-F., & Wang, W.-C. (2018). Using odds ratios to detect differential item functioning. Applied Psychological Measurement, 42(8), 613-629. doi: 10.1177/0146621618762738

Keywords

  • Odds ratio
  • Differential item functioning
  • Mantel–Haenszel
  • Logistic regression
  • Scale purification
  • Missing data

Fingerprint

Dive into the research topics of 'Using odds ratios to detect differential item functioning'. Together they form a unique fingerprint.