A multidimensional and multilevel extension of a random-effect approach to subjective judgment in rating scales

Wen Chung WANG, Xuelan QIU

Research output: Contribution to journalArticles

5 Citations (Scopus)

Abstract

In responding to rating scale items, respondents may hold different perspectives on the given categories. The random-effect rating scale model (RERSM), developed to account for variations in the category thresholds across respondents, is unidimensional and unilevel. It becomes statistically inefficient when multiple unidimensional tests have to be analyzed and inapplicable when data have a multilevel structure (e.g., respondents nested within organizations, students nested within schools). To resolve these problems, this study develops a multidimensional and multilevel version of the RERSM. The parameters can be estimated with existing computer software. Thus, there is no need to develop estimation procedures or corresponding computer programs. Simulation studies were conducted to evaluate the parameter recovery of the multidimensional RERSM, the multilevel RERSM, and the multidimensional and multilevel RERSM using WinBUGS. The results showed that the parameter recovery was generally satisfactory. An empirical example of the application of the multidimensional and multilevel RERSM to 2006 Program for International Student Assessment inventories about attitudes toward learning sciences is provided. Copyright © 2013 Taylor & Francis Group, LLC.
Original languageEnglish
Pages (from-to)398-427
JournalMultivariate Behavioral Research
Volume48
Issue number3
Early online dateJun 2013
DOIs
Publication statusPublished - 2013

Citation

Wang, W.-C., & Qiu, X.-L. (2013). A multidimensional and multilevel extension of a random-effect approach to subjective judgment in rating scales. Multivariate Behavioral Research, 48(3), 398-427.

Fingerprint Dive into the research topics of 'A multidimensional and multilevel extension of a random-effect approach to subjective judgment in rating scales'. Together they form a unique fingerprint.