In the social sciences, latent traits often have a hierarchical structure, and data can be sampled from multiple levels. Both hierarchical latent traits and multilevel data can occur simultaneously. In this study, we developed a general class of item response theory models to accommodate both hierarchical latent traits and multilevel data. The freeware WinBUGS was used for parameter estimation. A series of simulations were conducted to evaluate the parameter recovery and the consequence of ignoring the multilevel structure. The results indicated that the parameters were recovered fairly well; ignoring multilevel structures led to poor parameter estimation, overestimation of test reliability for the second-order latent trait, and underestimation of test reliability for the first-order latent traits. The Bayesian deviance information criterion and posterior predictive model checking were helpful for model comparison and model-data fit assessment. Two empirical examples that involve an ability test and a teaching effectiveness assessment are provided. Copyright © 2013 The Author(s).
CitationHuang, H.-Y., & Wang, W.-C. (2014). Multilevel higher-order item response theory models. Educational and Psychological Measurement, 74(3), 495–515.
- Item response theory
- Higher-order latent trait
- Multilevel model
- Markov chain Monte Carlo (MCMC) estimation