AI-assisted automated scoring of picture-cued writing tasks for language assessment

Ruibin ZHAO, Yipeng ZHUANG, Di ZOU, Qin XIE, Leung Ho Philip YU

Research output: Contribution to journalArticlespeer-review

Abstract

Grading assignments is inherently subjective and time-consuming; automatic scoring tools can greatly reduce teacher workload and shorten the time needed for providing feedback to learners. The purpose of this paper is to propose a novel method for automatically scoring student responses to picture-cued writing tasks. As a popular paradigm for language instruction and assessment, a picture-cued writing task typically requires students to describe a picture or pictures. Correspondingly, the automatic scoring methods must measure the link(s) between visual pictures and their textual descriptions. For this purpose, we first designed a picture-cued writing test and collected nearly 4 k responses from 279 K12 students. Based on these responses, we then developed an AI scoring model by incorporating the emerging cross-modal matching technology and some NLP algorithms. The performance of the model was evaluated carefully with six popular measures and was found to demonstrate accurate scoring results with a small mean absolute error of 0.479 and a high adjacent-agreement rate of 90.64%. We believe this method could reduce the subjective elements inherent in human grading and save teachers’ time from the mundane task of grading to other valuable endeavors such as designing teaching plans based on AI-generated diagnosis of student progress. Copyright © 2022 The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.

Original languageEnglish
JournalEducation and Information Technologies
Early online dateNov 2022
DOIs
Publication statusE-pub ahead of print - Nov 2022

Citation

Zhao, R., Zhuang, Y., Zou, D., Xie, Q., & Yu, P. L. H. (2022). AI-assisted automated scoring of picture-cued writing tasks for language assessment. Education and Information Technologies. Advance online publication. doi: 10.1007/s10639-022-11473-y

Keywords

  • Automated writing assessment
  • Picture-cued writing
  • Cross-modal matching
  • Artificial intelligence
  • PG student publication

Fingerprint

Dive into the research topics of 'AI-assisted automated scoring of picture-cued writing tasks for language assessment'. Together they form a unique fingerprint.