Abstract
Cognitive diagnosis computerized adaptive testing (CD-CAT) purports to provide each individual a profile about the strengths and weaknesses of attributes or skills with computerized adaptive testing. In the CD-CAT literature, researchers dedicated to evolving item selection algorithms to improve measurement efficiency, and most algorithms were developed based on information theory. By the discontinuous nature of the latent variables in CD-CAT, this study introduced an alternative for item selection, called the minimum expected cost (MEC) method, which was derived based on Bayesian decision theory. Using simulations, the MEC method was evaluated against the posterior weighted Kullback-Leibler (PWKL) information, the modified PWKL (MPWKL), and the mutual information (MI) methods by manipulating item bank quality, item selection algorithm, and termination rule. Results indicated that, regardless of item quality and termination criterion, the MEC, MPWKL, and MI methods performed very similarly and they all outperformed the PWKL method in classification accuracy and test efficiency, especially in short tests; the MEC method had more efficient item bank usage than the MPWKL and MI methods. Moreover, the MEC method could consider the costs of incorrect decisions and improve classification accuracy and test efficiency when a particular profile was of concern. All the results suggest the practicability of the MEC method in CD-CAT. Copyright © 2017 IACAT.
Original language | English |
---|---|
Publication status | Published - Aug 2017 |
Event | 2017 conference of the International Association for Computerized Adaptive Testing - Niigata, Japan Duration: 18 Aug 2017 → 21 Aug 2017 |
Conference
Conference | 2017 conference of the International Association for Computerized Adaptive Testing |
---|---|
Abbreviated title | IACAT 2017 |
Country/Territory | Japan |
City | Niigata |
Period | 18/08/17 → 21/08/17 |