Abstract
Purpose: The paper investigates small-scale, qualitative observations of interviewerrespondent interaction in the Organisation for Economic Co-operation and Development’s Programme for the International Assessment of Adult Competencies (PIAAC).
Design/methodology/approach: The paper employs video-ethnographic methods to document talk and gesture in assessment in Slovenian household settings. It presents an indepth case study of interaction in a single testing situation.
Findings: Observing interaction in assessment captures data on assessment performance that is not available in quantitative analysis of assessment response processes. The character of interviewer-respondent interaction and rapport is shaped by the cognitive demands of assessment and the distinctive ecological setting of the household.
Research limitations/implications: Observational data on assessment response processes and interaction in real-life assessments can be integrated into and synthesized with other sources of “process data.”
Practical implications: Assessment programs such as PIAAC should consider the
significance of the household setting on assessment quality and observations of interaction in assessment as a valid source of paradata.
Social implications: There is a place for small-scale observational studies of assessment to inform public understanding of assessment quality and validity.
Originality/value: The paper provides qualitative insights into the significance of interaction and “interviewer effects” in household assessment settings
Design/methodology/approach: The paper employs video-ethnographic methods to document talk and gesture in assessment in Slovenian household settings. It presents an indepth case study of interaction in a single testing situation.
Findings: Observing interaction in assessment captures data on assessment performance that is not available in quantitative analysis of assessment response processes. The character of interviewer-respondent interaction and rapport is shaped by the cognitive demands of assessment and the distinctive ecological setting of the household.
Research limitations/implications: Observational data on assessment response processes and interaction in real-life assessments can be integrated into and synthesized with other sources of “process data.”
Practical implications: Assessment programs such as PIAAC should consider the
significance of the household setting on assessment quality and observations of interaction in assessment as a valid source of paradata.
Social implications: There is a place for small-scale observational studies of assessment to inform public understanding of assessment quality and validity.
Originality/value: The paper provides qualitative insights into the significance of interaction and “interviewer effects” in household assessment settings
| Original language | English |
|---|---|
| Pages (from-to) | 182-195 |
| Number of pages | 14 |
| Journal | Quality Assurance in Education |
| Volume | 26 |
| Issue number | 2 |
| Early online date | 3 Apr 2018 |
| DOIs | |
| Publication status | Published - 2018 |