Abstract
Background: Language assessment plays a pivotal role in language education, serving as a bridge between students' understanding and educators' instructional approaches. Recently, advancements in Artificial Intelligence (AI) technologies have introduced transformative possibilities for automating and personalising language assessments.
Objectives: This article aims to explore the design and implementation of AI-enabled assessment tools in language education, filling the research gaps regarding the impact of assessment type, intervention duration, education level, and first language learner/second language learner (L1/L2) on the effectiveness of AI-enabled assessment tools in enhancing students' language learning outcome.
Methods: This study conducted a systematic review and meta-analysis to examine 25 empirical studies from January 2012 to March 2024 from six databases (including EBSCO, ProQuest, Scopus, Web of Science, ACM Digital Library and CNKI).
Results: The predominant design in AI-driven assessment tools is the structural AI architecture. These tools are most frequently deployed in classroom settings for upper primary students within a short duration. A subsequent meta-analysis showed a medium overall effect size (Hedges's g = 0.390, p < 0.001) for the application of AI-enabled assessment tools in enhancing students' language learning, underscoring their significant impact on language learning outcomes. This evidence robustly supports the practical utility of these tools in educational contexts.
Conclusions: The analysis of several moderator variables (i.e., assessment type, intervention duration, educational level and L1/L2 learners) and potential impacts on language learning performance indicates that AI-enabled assessment could be more useful in language education with a proper implementation design. Future research could investigate diverse instructional designs for integrating AI-based assessment tools in language education. Copyright © 2024 John Wiley & Sons Ltd.
Original language | English |
---|---|
Journal | Journal of Computer Assisted Learning |
Early online date | Nov 2024 |
DOIs | |
Publication status | E-pub ahead of print - Nov 2024 |
Citation
Chen, A., Zhang, Y., Jia, J., Liang, M., Cha, Y., & Lim, C. P. (2024). A systematic review and meta-analysis of AI-enabled assessment in language learning: Design, implementation, and effectiveness. Journal of Computer Assisted Learning. Advance online publication. https://doi.org/10.1111/jcal.13064Keywords
- Artificial intelligence
- Assessment
- Language learning
- Meta-analysis
- Systematic review