The development and validation of the Maze test for Chinese reading comprehension: Effects of deletion ratio, distractor, and scoring

Maze tests have been widely used to assess students’ reading comprehension. In Maze tests, some words are omitted from the passage as reading comprehension questions; students need to fill out the blank by identifying the word that best fits the context from two distractors. Previous work found the validity of Maze tests to be dependent on how the tests were designed. The study aimed to clarify how the design of a Maze test affects its validity for measuring Chinese reading comprehension, and, thus, to develop a highly valid Chinese Maze test.

We applied a 2 (deletion ratio: every 7th vs. 15th word was omitted in the passage) × 2 (difficulty of distractors: easy vs. difficult) × 2 (scoring rubric: accuracy rate with vs. without a penalty for errors in the scoring process) within-participant design. We asked 127 Mandarin Chinese-speaking Grade 3 students to complete Chinese Maze tests on 12 passages, with a counterbalanced design with eight versions (the eight experimental conditions) for each passage.

All participants’ performance on Character Naming, Vocabulary Knowledge, and Passage Reading Comprehension were also measured to evaluate the validity of our Maze tests under different conditions. Results suggested that the validity of Chinese Maze tests was affected by deletion ratio and difficulty of distractors, but not the scoring rubric. The most valid Maze test in our study is the one with the deletion ratio of 15 (when every 15th word was omitted in the passage) and difficult distractors. Copyright © 2023 by Association for Reading and Writing in Asia.