Alternatives to on-site evaluations of the performance of student-teachers: Issues in the measurement of inter-rater agreement

The evaluation of student-teacher performance during their field experience remains an important component of studentteachers' professional training. However, increasing pressure on university faculty has prompted a search for alternatives to the practice of on-site visits for this evaluation. This research focused on the measurement issues related to the use of video recordings to evaluate student-teacher performance in a teacher education programme in Hong Kong. Sixteen studentteachers and three university faculty participated in the research, using a 17-item Field Experience Supervision Form as the basis for the evaluation. The first part of the study estimated intra-rater agreement of student-teacher performance in a live context compared with an evaluation based on a video recording. The second part estimated inter-rater agreement of student-teacher performance based solely on video recordings of lessons. Analysis of the responses using Gwet's (2001) first-order agreement coefficient to estimate both intra- and inter-rater agreement of student-teacher performance, and a multi-facet Rasch model to study the interactions between student-teachers, raters, and medium, showed that video-recorded lessons can be an effective alternative to on-site evaluations. Copyright © 2012 De La Salle University, Philippines.