Multifaceted Rasch analysis on perceptual judgments of Chinese-accented speech

Background: The term “foreign accent” might be readily characterized as the subjective impressions of a native listener or a learner of a foreign language. The precise nature of foreign accent remains largely unexplored. As listeners (raters) play an important role in assessing foreign accents, rater effects cannot be ignored. Aims: With the acknowledgement that the observed rating score is exactly a three-way interaction among ratee, rater, and criterion, the multifaceted Rasch model (Linacre, 1989) was employed to analyze the ratings on foreign accent. Methods: An experiment was conducted, in which 16 recordings were divided into four blocks, and each rater was randomly assigned to two blocks and requested to evaluate the accent of recordings according to five criteria. Sample: Forty-eight students with different language backgrounds were invited as raters to provide accent ratings. Results: Among the five criteria for accent ratings, “No foreign accent at all” was the most difficult to attain, while “Very easy to understand” and “This accent is very familiar” were the easiest. The 48 raters exhibited very different degrees of severity (SD = 0.74 logit), suggesting rater effects should be carefully considered. By means of the fit statistics, it was found that two raters, one from Hong Kong (outfit = 1.96) and the other from mainland China (outfit = 1.93), could not maintain stable severity throughout the ratings. A mainland China speaker, who exaggeratedly and purposely imitated the accent of English native speakers, received considerably lower ratings from the non-Chinese speakers than from the Chinese speakers (outfit = 2.05). Conclusions: Fitting the multifaceted Rasch model to rating data in accent studies helps identify aberrant rating behavior and obtain fair measures. Future Directions: Other than accent, the impression of whether a person can speak a language fluently may be influenced by other factors such like exical, grammatical and discourse features. Further studies can focus on judgment in terms of these factors and apply the multifaceted Rasch model to the rating data likewise.