Evaluating the language resources of chatbots for their potential in English as a second language

Research output: Contribution to journalArticle

19 Citations (Scopus)

Abstract

This paper investigates the linguistic worth of current ‘chatbot’ programs – software programs which attempt to hold a conversation, or interact, in English – as a precursor to their potential as an ESL (English as a second language) learning resource. After some initial background to the development of chatbots, and a discussion of the Loebner Prize Contest for the most ‘human’ chatbot (the ‘Turing Test’), the paper describes an in-depth study evaluating the linguistic accuracy of a number of chatbots available online. Since the ultimate purpose of the current study concerns chatbots' potential with ESL learners, the analysis of language embraces not only an examination of features of language from a native-speaker's perspective (the focus of the Turing Test), but also aspects of language from a second-language-user's perspective. Analyses indicate that while the winner of the 2005 Loebner Prize is the most able chatbot linguistically, it may not necessarily be the chatbot most suited to ESL learners. The paper concludes that while substantial progress has been made in terms of chatbots' language-handling, a robust ESL ‘conversation practice machine’ (Atwell, 1999) is still some way off being a reality. Copyright © 2008 European Association for Computer Assisted Language Learning.
Original languageEnglish
Pages (from-to)98-116
JournalReCALL
Volume20
Issue number1
DOIs
Publication statusPublished - Jan 2008

Fingerprint

Linguistics
language
resources
conversation
linguistics
Resources
Language
English as a Second Language
examination
learning
Turing Test

Citation

Coniam, D. (2008). Evaluating the language resources of chatbots for their potential in English as a second language. ReCALL, 20(1), 98-116. doi: 10.1017/S0958344008000815

Keywords

  • Chatbots
  • ESL
  • Linguistic accuracy
  • CALL