Multilevel modelling for accountability systems in education

Data collected for accountability purposes often involve nested or hierarchical data structure. For instance, in the development of a value-added indicator system, achievement data are collected at the student-level but inferences are to be made at the school and system levels. Here, students are nested within classes, classes are nested within schools, and schools within regions. Since the late 1980s multilevel modelling has been used as a standard approach to handle such nested data structure. Research has shown that failure to apply multilevel models to nested data structure leads to wrong statistical inferences. Furthermore, much valuable information may be lost if the nested data structure is not modelled explicitly. In the last twenty years, the application of multilevel modelling has been extended to areas other than the handling of nested data structure. Multilevel modelling can now be applied for effective analysis of complicated data structures, including longitudinal data (e.g. growth trajectory), multivariate data (e.g. achievement in the four skills in the learning of English), discrete outcome data (e.g. students’ choice of entry to universities versus work), multiple membership data (e.g. the modelling of students’ mobility across schools in the estimation of the value added by schools to academic achievement), and cross-classified data structure (e.g. secondary schools taking students from a number of ‘feeder’ primary schools). The presentation will provide a non-technical introduction to multilevel modelling. Examples will be drawn from the value-added indicator system of the Hong Kong Special Administrative Region, and research on self-directed learning using the latest version of the MlwiN software package (Goldstein, 2003). Strengths and weaknesses of multilevel modelling will be discussed in the analyses of these real data.