A mixture Rasch facets model for raters' illusory halo effects

Kuan-Yu JIN, Ming Ming CHIU

Research output: Contribution to conferencePapers

Abstract

A rater's overall impression of a ratee's essay (or other assessment) can influence ratings on multiple criteria to yield excessively similar ratings (halo effect). Hence, we introduce and test a mixture Rasch facets model for halo effects (MRFM-H) that distinguishes true versus illusory halo effects and classifies normal and halo raters. In a simulation study, when raters assessed enough ratees, MRFM-H accurately identified halo raters. Also, more rating criteria increased classification accuracy. Ignoring halo effects (via a simpler model) biased parameters for evaluation criteria and rater severity but not ratee assessments. MRFM-H application to three empirical datasets showed (a) experienced raters' illusory halo effects, (b) fewer illusory halo effects with more criteria; and (c) more versus less informative survey responses. Copyright © 2021 AERA21.
Original languageEnglish
Publication statusPublished - Apr 2021

Citation

Jin, K.-Y., & Chiu, M. M. (2021, April). A mixture Rasch facets model for raters' illusory halo effects [Virtual]. Paper presented at the 2021 American Educational Research Association Annual Meeting (AERA21): Accepting Educational Responsibility, USA.

Fingerprint

Dive into the research topics of 'A mixture Rasch facets model for raters' illusory halo effects'. Together they form a unique fingerprint.