Analysis of test items suitability of examination papers for Yi Jin students of HKIEd

Empirical Research Background: Subjects were from Full time Yi Jin students of HKIEd in the academic year of 2007-08. They took an elective module of Outdoor Physical Education and were assessed by a written test (50%) and a performance test (50%) in the end of the module. It is important to assess the quality of the written test, which consisted of 30 multiple-choice items. Empirical Research Aims: As in general these students had low proficiency, we needed to adjust the difficulty of the written test. It is important to know whether the test was too difficult or too easy, and whether the standard of this examination really was suitable for their levels? Rasch analysis was conducted to investigate these issues. Empirical Research Sample: In the academic year of 2007-08, about 150 Yi Jin students took the module of Outdoor Physical Education and the written test. A sample of 50 students was selected and their item responses were analyzed using Rasch analysis. Empirical Research Method: Rasch analysis was conducted to assess quality of the written test. Empirical Research RASCH: The dichotomous Rasch model was applied to the item responses of 30 multiple-choice items of 50 students using the Winsteps software. Empirical Research Results: In general, there was a good model-data fit. The item infit mean square error of the 30 items ranged from 0.90 to 1.20, with mean 1.00. The item difficulty ranged from -3 to +3 logits, with mean zero. The person measures ranged from .00 to 3.08, with mean 1.28. Obviously, the test was too easy for them. The test reliability was as low as .34, suggesting the test was not able to differentiate the students very well. Empirical Research Conclusions: The 30 items have a good model-data fit, indicating they measure the same latent trait. Rasch analysis is very efficient in assessing the validity and reliability of the test.