This study aims to investigate how a recently developed mixed item selection approach would work with two advanced models in tackling complex non-statistical constraints in computerized adaptive testing. The mixed item selection approach (MX) is developed to capitalize on the strengths of two conventional item selection methods, namely the maximum information method (MI) and the b-matching method (BM). The MI method is good at providing efficient estimation of ability but poor at item security and pool utilization, whereas the BM method is less efficient in ability estimation but good at item security and pool utilization. The mixed approach has demonstrated to be efficient at ability estimation and item pool utilization for computerized adaptive testing by allowing first half of the test items to be selected by BM and then the next half by MI at later stages when ability estimate is close to its true value. In this study, the performance of mixed selection approach is further examined under complex constraints on several factors including content area, cognitive level, answer key position, enemy items and testlet. Two prevalent models, namely the weighted deviation model (WDM) and shadow test assembly (STA), are integrated with the mixed selection approach to tackle complex constraints. Performances of the integrated methods are evaluated on estimation efficiency, item security and item pool utilization. Preliminary results indicate that (a) MX is a better choice than MI and BM for long test (40-item) and (b)MX-STA outperforms MX-WDM in terms of test validity (satisfying constraints) and estimation efficiency.
|Publication status||Published - 2011|