Process and product in computer-based assessments: Clearing the ground for a holistic validity framework

Bruno D. Zumbo (Lead Author), Bryan Maddox, Naomi M. Care

Research output: Contribution to journalArticlepeer-review

7 Citations (Scopus)
13 Downloads (Pure)

Abstract

There is no consensus among assessment researchers about many of the central problems of response process data, including what is it and what is it comprised of. The Standards for Educational and Psychological Testing (American Educational Research Association et al., 2014) locate process data within their five sources of validity evidence. However, we rarely see a conceptualization of response processes; rather, the focus is on the techniques and methods of assembling response process indices or statistical models. The method often overrides clear definitions, and, as a field, we may therefore conflate method and methodology – much like we have conflated validity and validation (Zumbo, 2007). In this paper, we aim to clear the conceptual ground to explore the scope of a holistic framework for the validation of process and product. We review prominent conceptualizations of response processes and their sources and explore some fundamental questions: Should we make a theoretical and practical distinction between response processes and response data? To what extent do the uses
of process data reflect the principles of deliberate, educational, and psychological measurement? To answer these questions, we consider the case of item response times and the potential for variation associated with disability and neurodiversity.
Original languageEnglish
Pages (from-to)252-262
Number of pages11
JournalEuropean Journal of Psychological Assessment
Volume39
Issue number4
Early online date21 Mar 2023
DOIs
Publication statusPublished - Jul 2023

Keywords

  • process data
  • educational assessment
  • validity
  • digital assessment

Cite this