Modeling selection effects in examinee-selected items

Wen Chung WANG, Chen Wei LIU

Research output: Contribution to conferencePaper

Abstract

In an examinee-selected-item (ESI) design, examinees are required to respond to a fixed number of items from a set of given items (e.g., responding to 2 items from 5 given items; leading to 10 selection patterns). The ESI design has the advantages of enhancing students’ learning motivation and reducing their testing anxiety. However, these advantages come at a price: scores from different combinations of items are not directly comparable. This ESI design yields incomplete data, which may be missing not at random so that standard IRT models become inappropriate. Recently, Wang et al. (2012) proposed the examinee-selected-item item response theory (IRT) model by adding an additional latent trait to account for such a selection effect. This latent trait could correlate with the target (intended-to-be-measured) latent trait. Along this research line, we developed a general class of IRT models, which include the examinee-selected-item IRT model as a special case. In the most general case, each selection pattern has one random effect to account for its distinct selection effect. Simulation results indicated a good parameter recovery for the new models. We also conducted an experiment to collect real data, in which 462 fifth graders took five pairs of mathematic (dichotomous) items. In each pair of items, students were first asked to indicate which item they prefer to answer and then answer both items. This is referred to as the “Choose one, Answer all” approach. The new IRT models were fit to the real data and the results were discussed.
Original languageEnglish
Publication statusPublished - Jul 2014

Fingerprint

Model Theory
Latent Trait
Modeling
Missing at Random
Anxiety
Student Learning
Incomplete Data
Random Effects
Correlate
Recovery
Choose
Distinct
Testing
Target
Line
Experiment
Design
Simulation

Citation

Wang, W.-C., & Liu, C.-W. (2014, July). Modeling selection effects in examinee-selected items. Paper presented at the 79th Annual Meeting of the Psychometric Society (IMPS 2014), The Lowell Center, Madison, Wisconsin.