Modeling selection effects in examinee-selected items

Wen Chung WANG, Chen Wei LIU

Research output: Contribution to conferencePapers

Abstract

In an examinee-selected-item (ESI) design, examinees are required to respond to a fixed number of items from a set of given items (e.g., responding to 2 items from 5 given items; leading to 10 selection patterns). The ESI design has the advantages of enhancing students’ learning motivation and reducing their testing anxiety. However, these advantages come at a price: scores from different combinations of items are not directly comparable. This ESI design yields incomplete data, which may be missing not at random so that standard IRT models become inappropriate. Recently, Wang et al. (2012) proposed the examinee-selected-item item response theory (IRT) model by adding an additional latent trait to account for such a selection effect. This latent trait could correlate with the target (intended-to-be-measured) latent trait. Along this research line, we developed a general class of IRT models, which include the examinee-selected-item IRT model as a special case. In the most general case, each selection pattern has one random effect to account for its distinct selection effect. Simulation results indicated a good parameter recovery for the new models. We also conducted an experiment to collect real data, in which 462 fifth graders took five pairs of mathematic (dichotomous) items. In each pair of items, students were first asked to indicate which item they prefer to answer and then answer both items. This is referred to as the “Choose one, Answer all” approach. The new IRT models were fit to the real data and the results were discussed.
Original languageEnglish
Publication statusPublished - Jul 2014

Citation

Wang, W.-C., & Liu, C.-W. (2014, July). Modeling selection effects in examinee-selected items. Paper presented at the 79th Annual Meeting of the Psychometric Society (IMPS 2014), The Lowell Center, Madison, Wisconsin.

Fingerprint Dive into the research topics of 'Modeling selection effects in examinee-selected items'. Together they form a unique fingerprint.