The measurement of civic knowledge and the response process behind: A multidimensional mixture IRT modeling approach

This paper was based on the International Civic and Citizenship Education Study (ICCS) conducted by the IEA, where 14-year-old students responded to more than 120 attitudinal items on civic and citizenship. As a typical outcome of large scale assessment, traditional Rasch analyses produce single scale scores for each participating country for comparison in a league table. In order to make realistic assessments of students’ level of trust, the heterogeneity in the sample should be explored since there is evidence that unobserved heterogeneity may provide multiple classes with unique characteristics (Willse, 2011).This study, therefore, brings in the idea of “sub-populations” within a population. We will demonstrate with empirical datasets to examine the use of the mixture Rasch model as an alternative to the traditional Rasch model, especially when misfit items are observed and/ or when there is a signal of threat to uni-dimensionality of the measurement scale. We analyzed the dataset on students’ trust of political institutions (6 scaled items). By “unmixing” the unobserved sub-populations, the mixture Rasch analysis showed that a majority of students showed relatively more trust to the police but the minority showed less trust; however, these two groups showed similar trust towards other institution. It should be thus noticed that these unobserved groups within a sample may be very important to identify especially when they represent a sizable proportion of the sample. Left unanalyzed, heterogeneity may result in severe distortions and any implications drawn would be misleading.