A new facets model for rater's centrality/extremity response style

Kuan-Yu JIN, Wen Chung WANG

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

The Rasch facets model was developed to account for facet data, such as student essays graded by raters, but it accounts for only one kind of rater effect (severity). In practice, raters may exhibit various tendencies such as using middle or extreme scores in their ratings, which is referred to as the rater centrality/extremity response style. To achieve better measurement quality in rater data, it is desirable to simultaneously consider both rater severity and rater centrality/extremity. A new facets model is thus developed by adding to the Rasch facets model a weight parameter for the item thresholds for each rater. The parameters of the new facets model can be estimated with the JAGS freeware. An empirical example is provided to illustrate the implications and applications of the new model. Two simulation studies were conducted. The first simulation was to evaluate the parameter recovery of the new facets model and the consequences of ignoring the effects of rater centrality/extremity on parameter estimation. The second simulation was to illustrate how rater severity affects the relationship between rater centrality and the standard deviation of raw rating scores. Copyright © 2018 by the National Council on Measurement in Education.
Original languageEnglish
Pages (from-to)543-563
JournalJournal of Educational Measurement
Volume55
Issue number4
Early online dateDec 2018
DOIs
Publication statusPublished - 2018

Fingerprint

Extremities
simulation
rating
Students
Education
Weights and Measures
National Council
education
student

Citation

Jin, K.-Y., & Wang, W.-C. (2018). A new facets model for rater's centrality/extremity response style. Journal of Educational Measurement, 55(4), 543-563. doi: 10.1111/jedm.12191