Cognitive diagnostic models for rater effects

Xiaomin Nicky LI, Wen Chung WANG, Qin XIE

Research output: Contribution to journalArticles

Abstract

In recent decades, cognitive diagnostic models (CDMs) have been intensively researched and applied to various educational and psychological tests. However, because existing CDMs fail to consider rater effects, the application of CDMs to constructed-response (CR) items that involve human raters is seriously limited. Given the popularity of CR items, it is desirable to develop new CDMs that are capable of describing and estimating rater effects on CR items. In this study, we developed such new CDMs within the frameworks of facets models and hierarchical rater models, using the log-linear cognitive diagnosis model as a template. The parameters of the new models were estimated with the Markov chain Monte Carlo methods implemented in the freeware JAGS. Simulations were conducted to evaluate the parameter recovery of the new models. Results showed that the parameters were recovered fairly well and the more data there were, the better the recovery. Implications and applications of the new models were illustrated with an empirical study that adopted a fine-grained checklist to assess English academic essays. Copyright © 2020 Li, Wang and Xie.
Original languageEnglish
Article number525
JournalFrontiers in Psychology
Volume11
DOIs
Publication statusPublished - Mar 2020

Citation

Li, X., Wang, W.-C., & Xie, Q. (2020). Cognitive diagnostic models for rater effects. Frontiers in Psychology, 11. Retrieved from https://doi.org/10.3389/fpsyg.2020.00525

Keywords

  • Cognitive diagnostic models
  • Facets models
  • Hierarchical rater models
  • Rater effect
  • Item response theory

Fingerprint Dive into the research topics of 'Cognitive diagnostic models for rater effects'. Together they form a unique fingerprint.