Abstract
A rater's overall impression of a ratee's essay (or other assessment) can influence ratings on multiple criteria to yield excessively similar ratings (halo effect). Hence, we introduce and test a mixture Rasch facets model for halo effects (MRFM-H) that distinguishes true versus illusory halo effects and classifies normal and halo raters. In a simulation study, when raters assessed enough ratees, MRFM-H accurately identified halo raters. Also, more rating criteria increased classification accuracy. Ignoring halo effects (via a simpler model) biased parameters for evaluation criteria and rater severity but not ratee assessments. MRFM-H application to three empirical datasets showed (a) experienced raters' illusory halo effects, (b) fewer illusory halo effects with more criteria; and (c) more versus less informative survey responses. Copyright © 2021 AERA21.
| Original language | English |
|---|---|
| Publication status | Published - Apr 2021 |
| Event | 2021 Virtual Annual Meeting of American Educational Research Association: "Accepting Educational Responsibility" - , United States Duration: 08 Apr 2021 → 12 Apr 2021 https://www.aera.net/Events-Meetings/2021-Annual-Meeting |
Conference
| Conference | 2021 Virtual Annual Meeting of American Educational Research Association: "Accepting Educational Responsibility" |
|---|---|
| Abbreviated title | AERA 2021 |
| Country/Territory | United States |
| Period | 08/04/21 → 12/04/21 |
| Internet address |
