Reading comprehension of multimodal text in children vs adults

Pui Lun Alan TAI, Yen Na Cherry YUM, Neil COHN

Research output: Contribution to conferencePoster


Multimodal texts that integrate words and pictures, such as comics and picture books, are increasingly used in education. However, how readers make use of the additional visual information to support their reading comprehension and whether how this ability change across age groups is not well understood. This study investigated how children and adults read text passages presented with different amount of visual information. Participants were 21 children (mean age = 10.2) and 42 young adults (mean age = 24.6) who were native Chinese readers. In the reading phase, participants read three 8-page short passages in Chinese in three picture conditions: (1) plain text (no-picture), (2) text with accompanying pictures on half of the pages (half-picture), and (3) text with accompanying pictures on each page (all-picture) in a self-paced reading paradigm. A Tobii Pro X3-120 eye-tracker was used during the reading phase to track participants’ eye movements. After each reading, participants rated the passage on interest level, ease, and familiarity. Then they completed six short open-ended comprehension questions while the entire text was presented on the screen without pictures.
We measured three aspects of the reading, namely ratings, reading time, and comprehension performance. Data analyses using mixed-effects modelling were conducted for children and adults separately. In terms of reading ratings, children rated the all-picture condition as significantly easier relative to the half-picture and the no-picture conditions. Relative to plain texts, adults perceived the half-picture and all-picture readings as significantly more interesting. In terms of reading times, the children group showed no significant effect of picture condition, suggesting that children focused on the texts regardless of the presence of pictures. In contrast, adults spent significantly longer time on pages with pictures relative to plain texts. In the half-picture condition, adults spent significantly more time reading the words in the picture-less pages compared to the words on the pictured pages. These suggested that under some conditions, adults would preferentially take in the picture information instead of texts. In terms of comprehension questions, presence of pictures produced no significant effect on either accuracy or reaction time in children. For adults, when the questions have picture cues, accuracy rates were higher and response times to correct answers were faster. Critically, response times were slower for questions that had no picture cues in the pictured conditions relative to questions in the no picture condition, suggesting that picture cues facilitated information encoding and retrieval, but at the expense of uncued information.
In sum, the study revealed interesting differences in how pictures affected reading performance in children versus adults. Contrary to some previous findings, it was found that children in upper primary school were text-oriented in reading comprehension, while adults were more picture-oriented. Although children reported that texts with pictures were easier, reading time and reading comprehension performance was unaffected. Adults reported that pictures increased the interest level of the passage and indeed their reading performance reflected additional attention to pictures. Adults were subject to the attention directing and cuing effects of pictures during reading, while children were relatively impervious. Copyright © 2018 ICPEAL 17 - CLDC 9.
Original languageEnglish
Publication statusPublished - Oct 2018
EventThe 17th International Conference on the Processing of East Asian Languages and The 9th Conference on Language, Discourse, and Cognition - National Taiwan University, Taipei, Taiwan, Province of China
Duration: 19 Oct 201821 Oct 2018


ConferenceThe 17th International Conference on the Processing of East Asian Languages and The 9th Conference on Language, Discourse, and Cognition
Abbreviated titleICPEAL 17 – CLDC 9
Country/TerritoryTaiwan, Province of China
Internet address


Tai, P. L. A., Yum, Y. N., & Cohn, N. (2018, October). Reading comprehension of multimodal text in children vs adults. Poster presented at The 17th International Conference on the Processing of East Asian Languages and the 9th Conference on Language, Discourse, and Cognition (ICPEAL 17 – CLDC 9), National Taiwan University, Taipei, Taiwan.


Dive into the research topics of 'Reading comprehension of multimodal text in children vs adults'. Together they form a unique fingerprint.