Diagnosing and tracking English academic writing: Is cognitive diagnostic modelling the way forward?

Research output: Contribution to conferencePoster

Abstract

To date, only a handful of studies in language testing and assessment utilized Cognitive Diagnostic Modeling (CDM) for diagnostic assessment of L2 academic writing. Even fewer studies tracked the development of L2 academic writing over time. This study adopted CDM to profile the English academic writing performance of first-year university students and tracked the changes of their writing profiles over an academic writing course. This study involved 472 first-year undergraduate students from a university in Hong Kong. Participants were involved in a 13-week English academic writing course, for which they produced three essays for assessment at the beginning (Time 1: individual, timed essay), during (Time 2: pair co-construction, untimed essay), and the end (Time 3: individual and timed essay). At each assessment point, students’ writing was assessed by nine EAP instructors against a fine-grained diagnostic checklist for academic writing. The diagnostic data were analyzed via the reduced RUM to generate for each student a multi-dimensional diagnostic profile on five writing attributes. Salient writing profiles were identified for each assessment point and their changes were tracked over the three observations. The study identified four major performance patterns out of the 25 posterior probability of mastery profiles at each assessment point, and multiple transition trajectories of such profiles from Time 1 to 3 and from Time 2 to 3. Comparing Time 1 and 3, students’ writing performance improved on the two attributes of ‘task fulfillment’ and ‘vocabulary use’, but deteriorated in terms of ‘grammar’ and ‘organization’. Performance on ‘mechanics’ has marginal but insignificant decreases. Besides generating individualized feedback to students, the CDM-based diagnostic assessment has potential to provide useful information for course planning, instruction, and evaluation. Limitations of this study and further directions will be discussed at the end. Copyright © 2017 AILA2017 - World Congress in Rio - Brazil. All Rights Reserved.
Original languageEnglish
Publication statusPublished - Jul 2017
Event The 18th World Congress of Applied Linguistics: "Innovation and Epistemological Challenges in Applied Linguistics" - Rio de Janeiro, Brazil
Duration: 23 Jul 201728 Jul 2017

Conference

Conference The 18th World Congress of Applied Linguistics: "Innovation and Epistemological Challenges in Applied Linguistics"
Abbreviated titleAILA 2017
Country/TerritoryBrazil
CityRio de Janeiro
Period23/07/1728/07/17

Citation

Xie, Q. (2017, July). Diagnosing and tracking English academic writing: Is cognitive diagnostic modelling the way forward? Poster presented at The 18th World Congress of Applied Linguistics: Innovation and Epistemological Challenges in Applied Linguistics, Windsor Barra Hotel, Rio de Janeiro, Brazil.

Fingerprint

Dive into the research topics of 'Diagnosing and tracking English academic writing: Is cognitive diagnostic modelling the way forward?'. Together they form a unique fingerprint.