A rater's overall impression of a ratee's essay (or other assessment) can influence ratings on multiple criteria to yield excessively similar ratings (halo effect). Hence, we introduce and test a mixture Rasch facets model for halo effects (MRFM-H) that distinguishes true versus illusory halo effects and classifies normal and halo raters. In a simulation study, when raters assessed enough ratees, MRFM-H accurately identified halo raters. Also, more rating criteria increased classification accuracy. Ignoring halo effects (via a simpler model) biased parameters for evaluation criteria and rater severity but not ratee assessments. MRFM-H application to three empirical datasets showed (a) experienced raters' illusory halo effects, (b) fewer illusory halo effects with more criteria; and (c) more versus less informative survey responses. Copyright © 2021 AERA21.
|Publication status||Published - Apr 2021|
|Event||2021 Virtual Annual Meeting of American Educational Research Association: "Accepting Educational Responsibility" - , United States|
Duration: 08 Apr 2021 → 12 Apr 2021
|Conference||2021 Virtual Annual Meeting of American Educational Research Association: "Accepting Educational Responsibility"|
|Abbreviated title||AERA 2021|
|Period||08/04/21 → 12/04/21|