Item response models for local dependence among multiple ratings

Wen Chung WANG, Chi-Ming SU, Xuelan QIU

Research output: Contribution to journalArticlespeer-review

16 Citations (Scopus)

Abstract

Ratings given to the same item response may have a stronger correlation than those given to different item responses, especially when raters interact with one another before giving ratings. The rater bundle model was developed to account for such local dependence by forming multiple ratings given to an item response as a bundle and assigning fixed-effect parameters to describe response patterns in the bundle. Unfortunately, this model becomes difficult to manage when a polytomous item is graded by more than two raters. In this study, by adding random-effect parameters to the facets model, we propose a class of generalized rater models to account for the local dependence among multiple ratings and intrarater variation in severity. A series of simulations was conducted with the freeware WinBUGS to evaluate parameter recovery of the new models and consequences of ignoring the local dependence or intrarater variation in severity. The results revealed a good parameter recovery when the data-generating models were fit, and a poor estimation of parameters and test reliability when the local dependence or intrarater variation in severity was ignored. An empirical example is provided. Copyright © 2014 by the National Council on Measurement in Education.
Original languageEnglish
Pages (from-to)260-280
JournalJournal of Educational Measurement
Volume51
Issue number3
Early online dateAug 2014
DOIs
Publication statusPublished - 2014

Citation

Wang, W.-C., Su, C.-M., & Qiu, X.-L., (2014). Item response models for local dependence among multiple ratings. Journal of Educational Measurement, 51(3), 260-280.

Fingerprint

Dive into the research topics of 'Item response models for local dependence among multiple ratings'. Together they form a unique fingerprint.