A multidimensional and random-effects approach to subjective judgment in multiple rating scales

Xuelan QIU, Wen Chung WANG

Research output: Contribution to conferencePapers

Abstract

This study developed a multidimensional random-effects rating scale model (MRE-RSM) to take into account both multidimensionality and subjective judgment in multiple rating scales where each test measures a distinct latent trait with a set of items. A real data set from PISA 2006 was analyzed to demonstrate the advantages of the MRE-RSM over the RE-RSM (Wang, Wilson, & Shih, 2006) and the RSM (Andrich, 1978). The result showed that the MRE-RSM and the RE-RSM had a better fit, indicating subjective judgment in rating scales existed; the MRE-RSM, with the correlation among latent traits, yielded a higher test reliability, suggesting the MRE-RSM was more efficient; and the correlation among latent traits can be directly estimated only through the MRE-RSM.
Original languageEnglish
Publication statusPublished - 2010

Citation

Qiu, X., & Wang, W.-C. (2010, May). A multidimensional and random-effects approach to subjective judgment in multiple rating scales. Paper presented at the Annual Meeting of American Educational Research Association: Understanding Complex Ecologies in a Changing World, Denver, CO.