CBET: Design and evaluation of a domain-specific chatbot for mobile learning

Qingtang LIU, Jingxiu HUANG, Linjing WU, Ke ZHU, Shen BA

Research output: Contribution to journalArticlespeer-review

35 Citations (Scopus)

Abstract

The popularity of mobile devices and conversational agents in recent years has seen wide use of chatbots in different educational scenarios. In relation to the advances in mobile devices and conversational agents, there are few research works concerning the design and evaluation of domain-specific chatbots to fulfill the demand of mobile learning. To address this issue, we propose an agent-based conceptual architecture to develop a domain-specific chatbot for mobile learning. We extend the open-domain DeepQA agent to make it sensitive to restricted domain questions by building a domain-specific gate, and employ WeChat as user interface. To evaluate our chatbot, subjective and objective criteria are employed to assess its effectiveness. Additionally, its usability evaluation proceeds with system usability scale questionnaire and net promoter score simultaneously. In total, 18 domain experts participated in the evaluation of effectiveness, and 52 participants were involved in the evaluation of usability. Based on the evaluation results, we conclude that our chatbot can serve as an effective information retrieval tool in a specific domain. The perceived usability of our chatbot tends to be moderate and marginal and has positively affected the promotion of our chatbot for mobile learning. This paper contributes to the educative application of chatbots in specific subject fields. Copyright © 2019 Springer-Verlag GmbH Germany, part of Springer Nature.

Original languageEnglish
Pages (from-to)655-673
JournalUniversal Access in the Information Society
Volume19
Early online dateJul 2019
DOIs
Publication statusPublished - Aug 2020

Citation

Liu, Q., Huang, J., Wu, L., Zhu, K., & Ba, S. (2020). CBET: Design and evaluation of a domain-specific chatbot for mobile learning. Universal Access in the Information Society, 19, 655-673. https://doi.org/10.1007/s10209-019-00666-x

Keywords

  • Chatbots
  • Conversational agents
  • Mobile learning
  • Evaluation of educative chatbots
  • Usability evaluation

Fingerprint

Dive into the research topics of 'CBET: Design and evaluation of a domain-specific chatbot for mobile learning'. Together they form a unique fingerprint.