A scale purification procedure to SIBTEST for assessment of DIF

Theoretical Articles Background: The performance of differential item functioning (DIF) assessment was usually influenced by the percentage of DIF items in the test. To overcome such problem, two ways were usually suggested to implement. The first is to add the scale purification procedure onto the DIF method, the other is applying the DIF-free-then-DIF strategy, where a set of DIF-free items should be located first to serve as anchors and then proceed DIF assessment to other items in the test. Theoretical Articles Aims: The SIBTEST (Simultaneous Item Bias TEST) method was developed to assess uniform differential item functioning (DIF) for dichotomous items. Two extensions of SIBTEST were proposed, one for assessing crossing DIF and the other for polytomous items. In this study, we implemented a scale purification procedure to the SIBTEST method (denoted as SIBTEST -ST) and called it the SIBTEST method with scale purification (SIBTEST -SP). Theoretical Articles Arguments: As the percentage of DIF items in the test increased, the false positive (FP) rate was increased for both SIBTEST-ST and SIBTEST-SP methods. At the meanwhile, the difference of FP and TP rates between these two methods was also increased. The SIBTEST-SP showed better FP control and higher TP rates than SIBTEST-ST under all conditions that manipulated in this study. Theoretical Articles Conclusions: The scale purification procedure was recommended to be implemented during DIF analysis. Through Monte Carlo simulations in this study, it was found that the SIBTEST-SP method can control FP rates better than the SIBTEST-ST method. However, when the DIF percentage was equal to or higher than 30%, even the scale purification could loss its control on FP rates. When this is the case, the DIF-free-then-DIF (DFTD) strategy was strongly recommended. The further question is then switch to how to locate a set of DIF-free items to serve as anchors. The DFTD procedure will then be implemented to SIBTEST and its performance will be investigated in the further study.