The logistic testlet framework for within-item multidimensional testlet-effect

A testlet is a cluster of items that share a common stimulus, and the possible local dependence among items within a testlet is called testlet-effect. Under the framework of Item Response Theory (IRT), various Testlet Response Models (TRM) have been developed to take into account such testlet-effect (e.g., Bradlow, Wainer, & Wang, 1999; Wang & Wilson, 2005; Li, Blot, & Fu, 2006). However, these existing TRM all assume that an item is affected by only one single testlet-effect (Zhan, Wang, Wang, & Li, 2013). Therefore, they are essentially within-item unidimensional testlet-effect models. In practice, multiple testlet effects may simultaneously affect item responses in a testlet. For example, in addition to common stimulus, items can be grouped according to their domains, knowledge units, or item format, such that multiple testlet effects are involved. In essence, an item measures multiple latent traits, in addition to the target latent trait(s) that the test was designed to measure. Existing TRM become inapplicable when multiple testlet effects are involved. To solve this problem, we develop a logistic testlet framework that specifically account for the within-item multidimensional testlet-effect. Results of a series of simulations demonstrated that the parameters of the new models were recovered fairly well by using WinBUGS; and ignoring any one of the multiple testlet effects resulted in a biased estimation of item parameters. Additionally, it did little harm on parameter estimation to fit a more complicated model to data with a simple structure.