The facets graded response model

Background: Raters are often recruited to mark constructed-response items. For example, student essays are graded by teachers; or teachers’ performances are judged by students. Linacre (1989) has developed the facets model to account for the joint effects of item, person, and rater on item responses using the conditional-probability formulation. In the statistical literature, the latent-response formulation is used more often. The latent-response formulation is in line with the cumulative-logit models, which incorporate the graded response model (Samejima 1969) and the 2-parameter logistic model (Birnbaum 1968) as special cases. Aims and Keywords: In this study, we aim to extend the two-facet graded response model to more than two facets so that rater effects can be evaluated. Simulations were conducted to evaluate its parameter recovery using the MPlus program. Methods: The item responses were generated under the three-facet graded response model, where 5 raters judged 200 examinees on 5 tasks on a five-point rating scale. A total of 100 replications were made. The true model was used to analyze the data using the Mplus. The bias and root mean square error (RMSE) were computed to evaluate parameter recovery. Results: The bias values were (0.154 ~ 0.186), and the RMSE values were (0.043 ~ 0.215). Apparently, the parameter recovery of the three-facet graded response model was satisfactory. Conclusion: The two-facet graded response model is extended to facets graded response model to examine rater effects on open-ended items. The parameters can be recovered fairly well using the Mplus. Applications of the new model are welcome.