Multidimensional computerized adaptive testing using non-compensatory item response theory models

Chia-Ling HSU, Wen Chung WANG

Research output: Contribution to journalArticle

Abstract

Current multidimensional computerized adaptive testing (MCAT) has been developed in conjunction with compensatory multidimensional item response theory (MIRT) models rather than with non-compensatory ones. In recognition of the usefulness of MCAT and the complications associated with non-compensatory data, this study aimed to develop MCAT algorithms using non-compensatory MIRT models and to evaluate their performance. For the purpose of the study, three item selection methods were adapted and compared, namely, the Fisher information method, the mutual information method, and the Kullback–Leibler information method. The results of a series of simulations showed that the Fisher information and mutual information methods performed similarly, and both outperformed the Kullback–Leibler information method. In addition, it was found that the more stringent the termination criterion and the higher the correlation between the latent traits, the higher the resulting measurement precision and test reliability. Test reliability was very similar across the dimensions, regardless of the correlation between the latent traits and termination criterion. On average, the difficulties of the administered items were found to be at a lower level than the examinees' abilities, which shed light on item bank construction for non-compensatory items. Copyright © 2018 The Author(s).
Original languageEnglish
Pages (from-to)464-480
JournalApplied Psychological Measurement
Volume43
Issue number6
Early online dateOct 2018
DOIs
Publication statusPublished - 01 Sep 2019

Fingerprint

model theory
bank
simulation
ability
performance

Citation

Hsu, C.-L., & Wang, W.-C. (2019). Multidimensional computerized adaptive testing using non-compensatory item response theory models. Applied Psychological Measurement, 43(6), 464-480. doi: 10.1177/0146621618800280

Keywords

  • Item response theory
  • Non-compensatory models
  • Computerized adaptive testing
  • Item selection methods