The double marking of Liberal Studies in the Hong Kong public examination system

Background: This article reports a study into the double marking of Liberal Studies in Hong Kong. This is now a compulsory subject in Hong Kong's Years 10-12 curriculum which, when first examined in the new Hong Kong Diploma of Secondary Education in 2012, will increase its candidature from its current 3,300 to 80,000. 

Aims: To examine the reliability of the forthcoming double marking of LS, investigating whether high inter-marker correlations are achieved in the double marking of LS, matching the reliability rate achieved by other examinations administered by the HKEAA. 

Method: Broadly adopting the methodology of an earlier study of marking in the 2007 Year 11 English Language examination, the current study investigates double marking using classical test statistics-inter-, intra- and marker-subject correlations-and the amount of discrepancy between pairs of markers. 

Sample: Seven experienced markers (re)marked 677 scripts from the Hong Kong Studies module and another seven markers (re)marked 654 scripts from the Human Relationships module. 

Results: Moderate to strong correlations emerged between pairs of markers. Discrepancy levels were below 10%-in line with other Hong Kong public examinations. 

Conclusion: With a view to improving marker reliability, the study concludes with the recommendation that the current holistic marking scheme should be re-evaluated, with a view to investigating its replacement by an analytic, domain-based marking scheme. This study highlights the need for public examination bodies to carry out a range of validation, reliability and other studies prior to the implementation of changes to their large-scale examinations.

背景：本文報告一項有關香港通識教育科雙閱卷員制 (double marking)的研究。通識教育現在是一個在香港高中一年級至三年級課程的必修科目，它會在第一屆的香港中學文憑試（即2012年）內首次評核，屆時其考生人數會從目前的3300人增至80000人。    

目的：查證即將實施的通識教育科雙閱卷員制的可靠性，以調查其閱卷員之間能否取得高相關性，以及其可靠性能否相匹配其他由香港考試評核局管理的考試。

方法：廣泛採用的方法是利用較早前在2007年有關高中二年級英國語文考試的研究所採用的，目前的研究使用標準的測驗統計 - 閱卷員及科目的間信度和內信度，以及每對閱卷員之間的差異水平之大小。

樣本：七個富經驗的閱卷員（重新）評核677份來自單元〈香港研究〉的考卷，另七個富經驗的閱卷員（重新）評核654份來自單元〈人際關係〉的考卷。

結果：閱卷員之間取得中度到高度的相關性。閱卷員之間的差異水平低於 10％ - 與其他香港公開考試一致。

結論：為了提高閱卷員的可靠性，這個研究結論建議將目前整體性的評分準則重新評估，以便調查能否以分析性、範圍性的評分準則取代之。這項研究結果強調了公開考試機構在實施應用在大規模考試的改變前是有需要進行一系列的有關有效性、可靠性和其他方面的研究。 Copyright © 2011 New Horizons in Education. All rights reserved.