Automatic identification of ineffective online student questions in computing education

Qiang HAO, April GALYARDT, Bradley BARNES, Robert Maribe BRANCH, Ewan Thomas Mansell WRIGHT

Research output: Chapter in Book/Report/Conference proceedingChapter

Abstract

This Research Full Paper explores automatic identification of ineffective learning questions in the context of large-scale computer science classes. The immediate and accurate identification of ineffective learning questions opens the door to possible automated facilitation on a large scale, such as alerting learners to revise questions and providing adaptive question revision suggestions. To achieve this, 983 questions were collected from a question & answer platform implemented by an introductory programming course over three semesters in a large research university in the Southeastern United States. Questions were firstly manually classified into three hierarchical categories: 1) learning-irrelevant questions, 2) effective learning-relevant questions, 3) ineffective learning-relevant questions. The inter-rater reliability of the manual classification (Cohen's Kappa) was.88. Four different machine learning algorithms were then used to automatically classify the questions, including Naive Bayes Multinomial, Logistic Regression, Support Vector Machines, and Boosted Decision Tree. Both flat and single path strategies were explored, and the most effective algorithms under both strategies were identified and discussed. This study contributes to the automatic determination of learning question quality in computer science, and provides evidence for the feasibility of automated facilitation of online question & answer in large scale computer science classes. Copyright © 2018 IEEE.
Original languageEnglish
Title of host publicationProceedings of 2018 IEEE Frontiers in Education Conference (FIE)
Place of PublicationDanvers, MA
PublisherIEEE
ISBN (Print)9781538611739
DOIs
Publication statusPublished - 2018

    Fingerprint

Citation

Hao, Q., Galyardt, A., Barnes, B., Branch, R. M., & Wright, E. (2018). Automatic identification of ineffective online student questions in computing education. In Proceedings of 2018 IEEE Frontiers in Education Conference (FIE). Retrieved from https://doi.org/10.1109/FIE.2018.8658642