Individualized feedback to raters in language assessment: Impacts on rater effects

Jing HUANG, Gaowei CHEN

Research output: Contribution to journalArticlespeer-review

2 Citations (Scopus)

Abstract

This study examined the impacts of individualized feedback on rater effects in performance-based language assessment. The raters were 93 native Chinese speakers without previous rating experience, and they were randomly assigned to one of the three treatment groups. The three groups of raters differed in the way that they received individualized feedback in a given period: (a) a control group receiving no feedback, (b) a single-feedback group receiving the feedback once, and (c) a double-feedback group receiving the feedback twice. The results indicated that individualized feedback significantly reduced rater severity/leniency and rater misfit. Specifically, raters from both the double-feedback and single-feedback groups exhibited lower levels of rater severity/leniency, compared with those from the control group. Raters from the double-feedback group showed lower levels of rater misfit than those from the single-feedback group. With regard to the retention of reduction in rater effects, individualized feedback was found to assist raters from the double-feedback group to maintain the reduction in rater severity/leniency. It also helped raters from the single-feedback group to maintain the reduction in rater misfit. These findings may shed light on the applications of individualized feedback in the designs of face-to-face and online rater training programs. Copyright © 2022 Elsevier Inc. All rights reserved.

Original languageEnglish
Article number100623
JournalAssessing Writing
Volume52
Early online dateMar 2022
DOIs
Publication statusPublished - Apr 2022

Citation

Huang, J., & Chen, G. (2022). Individualized feedback to raters in language assessment: Impacts on rater effects. Assessing Writing, 52. Retrieved from https://doi.org/10.1016/j.asw.2022.100623

Keywords

  • Individualized feedback
  • Rater severity/leniency
  • Rater misfit
  • Rater bias
  • Writing assessment
  • Many-facet Rasch measurement

Fingerprint

Dive into the research topics of 'Individualized feedback to raters in language assessment: Impacts on rater effects'. Together they form a unique fingerprint.