The linguistic accuracy of chatbots: Usability from an ESL perspective


Research output: Contribution to journalArticlespeer-review

66 Citations (Scopus)


This paper reports on the linguistic accuracy of five renowned "chatbots," with an evaluator (an ESL teacher) chatting with each chatbot for about three hours. The chatting consisted of a series of set questions/statements (determined as being in the domain of an ESL learner) - aimed at assessing the accuracy and felicity of the chatbots' answers at the grammatical level. Results indicate that chatbots are generally able to provide grammatically acceptable answers, with three chatbots returning acceptability figures in the 90% range. When meaning is factored in, however, a different picture emerges, with the chatbots often providing meaningless, nonsensical answers, and the accuracy rate for the joint categories of grammar and meaning falling below 60%. The paper concludes on the note that although chatbots as "conversation practice machines" do not yet make robust chatting partners, improvements in chatbot performance bode well for future developments. Copyright © 2014 De Gruyter Mouton.
Original languageEnglish
Pages (from-to)545-567
JournalText & Talk
Issue number5
Early online dateAug 2014
Publication statusPublished - Sept 2014


Coniam, D. (2014). The linguistic accuracy of chatbots: Usability from an ESL perspective. Text & Talk, 34(5), 545-567.


  • Chatbots
  • English language
  • ESL
  • Computer analysis of language
  • Syntax


Dive into the research topics of 'The linguistic accuracy of chatbots: Usability from an ESL perspective'. Together they form a unique fingerprint.