Using Bayesian decision theory in cognitive diagnosis computerized adaptive testing

Chia Ling HSU, Wen Chung WANG, Shu-Ying CHEN

Research output: Contribution to conferencePaper

Abstract

Cognitive diagnosis computerized adaptive testing (CD-CAT) purports to provide each individual a profile about the strengths and weaknesses of attributes or skills with computerized adaptive testing. In the CD-CAT literature, researchers dedicated to evolving item selection algorithms to improve measurement efficiency, and most algorithms were developed based on information theory. By the discontinuous nature of the latent variables in CD-CAT, this study introduced an alternative for item selection, called the minimum expected cost (MEC) method, which was derived based on Bayesian decision theory. Using simulations, the MEC method was evaluated against the posterior weighted Kullback-Leibler (PWKL) information, the modified PWKL (MPWKL), and the mutual information (MI) methods by manipulating item bank quality, item selection algorithm, and termination rule. Results indicated that, regardless of item quality and termination criterion, the MEC, MPWKL, and MI methods performed very similarly and they all outperformed the PWKL method in classification accuracy and test efficiency, especially in short tests; the MEC method had more efficient item bank usage than the MPWKL and MI methods. Moreover, the MEC method could consider the costs of incorrect decisions and improve classification accuracy and test efficiency when a particular profile was of concern. All the results suggest the practicability of the MEC method in CD-CAT. Copyright © 2017 IACAT.
Original languageEnglish
Publication statusPublished - Aug 2017

Citation

Hsu, C.-L., Wang, W.-C., & Chen, S.-Y. (2017, August). Using Bayesian decision theory in cognitive diagnosis computerized adaptive testing. Paper presented at the annual meeting of International Association for Computerized Adaptive Testing 2017 (IACAT 2017), Niigata Convention Center, Niigata, Japan.

Fingerprint Dive into the research topics of 'Using Bayesian decision theory in cognitive diagnosis computerized adaptive testing'. Together they form a unique fingerprint.