The more the better? The modality matters: Effects of multimodal learning on connected speech in Chinese ESL learners

Learning with more than a modal benefits perceptual learning in connected speech for English as a Second Language (ESL) learners. Moreover, learning with multimodalities may overload attention and cognitive processing which hinder learning. To examine this hypothesis, a connected speech perceptual dictation is designed and test the effects of learning with bi-modal, listen with either subtitles or shadowing; and multimodal, listen with both subtitles and shadowing. A group of ninety ESL undergraduate and graduate students were recruited from a local university in Hong Kong to participate in the present study. The results challenges the hypothesis that the multiple modalities aid the learning of connected speech. The Wicken’s (2007) Multiple Resources Model and the Cognitive Load Theory (CLT) (e.g. Sweller, 2010) may account for the results. The current study sheds light on the pedagogical implication of connected speech learning with multimodalities in ESL classroom context.