Toward more robust automatic analysis of student program outputs for assessment and learning

Chung Keung POON, Tak Lam WONG, Yuen Tak YU, Victor Chung Sing LEE, Chung Man TANG

Research output: Chapter in Book/Report/Conference proceedingChapter

Abstract

Automated analysis and assessment of students' programs, typically implemented in automated program assessment systems (APASs), are very helpful to both students and instructors in modern day computer programming classes. The mainstream of APASs employs a black-box testing approach which compares students' program outputs with instructor-prepared outputs. A common weakness of existing APASs is their inflexibility and limited capability to deal with admissible output variants, that is, outputs produced by acceptable correct programs that differ from the instructor's. This paper proposes a more robust framework for automatically modelling and analysing student program output variations based on a novel hierarchical program output structure called HiPOS. Our framework assesses student programs by means of a set of matching rules tagged to the HiPOS, which produces a better verdict of correctness. We also demonstrate the capability of our framework by means of a pilot case study using real student programs. Copyright © 2016 by The Institute of Electrical and electronics engineers, inc.
Original languageEnglish
Title of host publicationProceedings: 2016 IEEE 40th Annual Computer Software and Applications Conference
EditorsSheikh Iqbal Ahamed, Carl K. Chang, William Chu, Ivica Crnkovic, Pao-Ann Hsiung, Gang Huang, Jingwei Yang
Place of PublicationLos Alamitos
PublisherIEEE Computer Society
Pages780-785
Volume1
ISBN (Print)9781467388450
DOIs
Publication statusPublished - 2016

Fingerprint

Students
Black-box testing
Computer programming
Electronic equipment
Engineers

Citation

Poon, C. K., Wong, T.-L., Yu, Y. T., Lee, V. C. S., & Tang, C. M. (2016). Toward more robust automatic analysis of student program outputs for assessment and learning. In S. Reisman, S. I. Ahamed, L. Liu, D. Milojicic, W. Claycomb, M. Matskin, H. Sato, et al. (Eds.), Proceedings: 2016 IEEE 40th Annual Computer Software and Applications Conference (Vol.1, pp. 780-785). Los Alamitos: IEEE Computer Society.

Keywords

  • Automated assessment technology
  • Computer science education
  • Learning computer programming
  • Program output variant
  • Student program analysis