Analyses of testlet data

Wen Chung WANG, Kuan Yu JIN

Research output: Chapter in Book/Report/Conference proceedingChapters

Abstract

Testlets, which are defined as a set of items linked by a common stimulus, are commonly used in educational and psychological tests. Such a linkage may make items within a testlet locally dependent. There are three major approaches to testlet-based items. First, one can fit standard item response theory (IRT) models and ignore the possible local dependence. Second, one can transform items in a testlet into a super (polytomous) item and then fit polytomous IRT models to the transformed data. Third, one can fit testlet response models that were developed to account for the local dependence. This study compared the performance of these three approaches in recovering person measures and test reliability through simulations. It was found that the polytomous-item approach performed highly satisfactorily when data were generated from testlet response models or when data had chain effects between adjacent items. In contrast, fitting standard item response models tended to result in overestimation of test reliability when data were generated from testlet response models, and underestimation or overestimation of test reliability when the data had chain effects. Likewise, fitting testlet response models might result in underestimation or overestimation of test reliability when the data have chain effects. Thus, if person measures as well as their measurement precision (test reliability) is the major concern, the polytomous-item approach is recommended. Copyright © 2016 Springer Science+Business Media Singapore.
Original languageEnglish
Title of host publicationPacific Rim Objective Measurement Symposium (PROMS) 2015 conference proceedings
EditorsQuan ZHANG
Place of PublicationSingapore
PublisherSpringer
Pages199-214
ISBN (Print)9789811016868, 9789811016875
DOIs
Publication statusPublished - 2016

Citation

Wang, W.-C., & Jin, K.-Y. (2016). Analyses of testlet data. In Q. Zhang (Ed.), Pacific Rim Objective Measurement Symposium (PROMS) 2015 conference proceedings (pp. 199-214). Singapore: Springer.

Keywords

  • Testlet response theory
  • Item response theory
  • Local independence
  • Chain effect
  • Polytomous-items

Fingerprint

Dive into the research topics of 'Analyses of testlet data'. Together they form a unique fingerprint.