Abstract
Testlets, which are defined as a set of items linked by a common stimulus, are commonly used in educational and psychological tests. Such a linkage may make items within a testlet locally dependent. There are three major approaches to testlet-based items. First, one can fit standard item response theory (IRT) models and ignore the possible local dependence. Second, one can transform items in a testlet into a super (polytomous) item and then fit polytomous IRT models to the transformed data. Third, one can fit testlet response models that were developed to account for the local dependence. This study compared the performance of these three approaches in recovering person measures and test reliability through simulations. It was found that the polytomous-item approach performed highly satisfactorily when data were generated from testlet response models or when data had chain effects between adjacent items. In contrast, fitting standard item response models tended to result in overestimation of test reliability when data were generated from testlet response models, and underestimation or overestimation of test reliability when the data had chain effects. Likewise, fitting testlet response models might result in underestimation or overestimation of test reliability when the data have chain effects. Thus, if person measures as well as their measurement precision (test reliability) is the major concern, the polytomous-item approach is recommended. Copyright © 2016 Springer Science+Business Media Singapore.
Original language | English |
---|---|
Title of host publication | Pacific Rim Objective Measurement Symposium (PROMS) 2015 conference proceedings |
Editors | Quan ZHANG |
Place of Publication | Singapore |
Publisher | Springer |
Pages | 199-214 |
ISBN (Print) | 9789811016868, 9789811016875 |
DOIs | |
Publication status | Published - 2016 |
Citation
Wang, W.-C., & Jin, K.-Y. (2016). Analyses of testlet data. In Q. Zhang (Ed.), Pacific Rim Objective Measurement Symposium (PROMS) 2015 conference proceedings (pp. 199-214). Singapore: Springer.Keywords
- Testlet response theory
- Item response theory
- Local independence
- Chain effect
- Polytomous-items