Assessing dimensionality using Rasch rating scale model

In practice, many educational and psychological tests are multidimensional. Ackerman (2003) highlighted Multidimensional Item Response Theory (MIRT) models for polytomous data and higher dimensional space as one of the key directions for future research. Many methods have been proposed for assessing dimensionality in IRT models, including the use of Principal Components Analysis on standardized residuals and others (see e.g., Hattie, 1985; Karabatsos, 2000; Linacre, 1998; Smith, 2002). Nevertheless, the literature is still equivocal regarding such issues as cut-point of the first eigenvalue used in PCA of standardized residuals, and its relation to sample size, test length, the number of dimensions, and the degree of dimensionality (Chou & Wang, 2010). This study aims to verify methods of assessing dimensionality using a series of Monte Carlo simulation with data generated in the context of Rating Scale Rasch model under systematic combination of conditions of test length (5 or 10 items), difference in test length between dimensions (nil, small, large), sample size (500 or 1000 students), strength of correlation (0.0, 0.3, 0.7, 0.9), number of dimensions (2 or 3 dimensions), and status of dimensionality (between or within items). Combinations of these conditions were manipulated in this study in order to find out their impacts on rate of Type I error and Power of multidimensional detection.