A scale purification procedure to likelihood ratio test method for assessment of DIF

Theoretical Articles Background: The performance of differential item functioning (DIF) assessment was usually influenced by the percentage of DIF items in the test. Therefore, the scale purification procedure was strongly recommended to implement in DIF assessment methods. Theoretical Articles Aims: The likelihood ratio test (LRT) method has been used to assess differential item functioning (DIF) for decades. In this study, we implemented a scale purification procedure to the standard LRT method (denoted as LRT-ST) and called it the LRT method with scale purification (LRT-SP). Theoretical Articles Arguments: As the percentage of DIF items in the test increased, the false positive (FP) rate was increased, whereas the true positive (TP) rate decreased, for both LRT-ST and LRT-SP methods. At the meanwhile, the differences of both FP and TP rates between these two methods were also increased. The LRT-SP outperformed than LRT-ST, for both FP control and higher TP rates, under all conditions that manipulated in this study. Theoretical Articles RASCH: The findings of this study can be further applied to the data that fit Rasch model. Furthermore, since the raw score is a sufficient statistics of latent trait under Rasch model, the purification procedure and DIF-free-then-DIF (DFTD) strategy can be implemented to the DIF assessment method that taking raw scores as matching variable, such as Mantel-Haenszel and logistic regression. Theoretical Articles Conclusions: The scale purification procedure was proven to be effective for DIF assessment methods, such as logistic regression, Mantel-Haenszel, and MIMIC (multiple indicators, multiple causes). It is recommended that scale purification should be implemented as a standard procedure during DIF analysis. Through Monte Carlo simulations in this study, it was found that the LRT-SP method can control FP rates better than the LRT-ST method. However, when the DIF percentage was high, even the scale purification could loss its control on FP rates. When this is the case, the DIF-free-then-DIF (DFTD) strategy was recommended. The further question is then switch to how to locate a set of DIF-free items to serve as anchors. The DFTD procedure will then be implemented to likelihood ratio test and its performance will be investigated in the further study.