Mixing interview and questionnaire methods: Practical problems in aligning data

Lois Ruth HARRIS, Gavin Thomas Lumsden BROWN

Research output: Contribution to journalArticles

73 Citations (Scopus)

Abstract

Structured questionnaires and semi-structured interviews are often used in mixed method studies to generate confirmatory results despite differences in methods of data collection, analysis, and interpretation. A review of 19 questionnaire-interview comparison studies found that consensus and consistency statistics were generally weak between methods. Problems in aligning data from the two different methods are illustrated in a questionnaire-interview study of teacher conceptions of assessment. Poor alignment appeared attributable to: differences in data collection procedures, the complexity and instability of the construct being investigated, difficulties in making data comparable, lack of variability in participant responses, greater sensitivity to context and seemingly emotive responses within the interview, possible misinterpretation of some questionnaire prompts, and greater control of content exposure in the questionnaire. Results indicated that if ‘confirmatory’ results are being sought, researchers must create tightly aligned and structured instruments; present the construct in a simple, concrete, and highly contextualised manner; collect the two types of data with a minimal time gap; and estimate agreement between methods using consistency statistics. However, the cost of confirmation through strong alignment may lead to the loss of rich complementary data obtained through allowing each method to be analysed in its own right. Copyright is retained by the first or sole author, who grants right of first publication to the Practical Assessment, Research & Evaluation. Copyright © 2010 Practical Assessment, Research & Evaluation.
Original languageEnglish
Pages (from-to)1-19
JournalPractical Assessment Research & Evaluation
Volume15
Issue number1
Publication statusPublished - Jan 2010

Citation

Harris, L. R., & Brown, G. T. L. (2010). Mixing interview and questionnaire methods: Practical problems in aligning data. Practical Assessment Research & Evaluation, 15(1), 1-19.

Fingerprint Dive into the research topics of 'Mixing interview and questionnaire methods: Practical problems in aligning data'. Together they form a unique fingerprint.