Reliability in assessment centers depends on general and exercise performance, but not on dimensions

Duncan Jackson, George Michaelides, Chris Dewberry, Jo Nelson, Catherine Stephens

Research output: Contribution to journalArticlepeer-review

Abstract

This study contributes to the literature on assessment centre (AC) measurement structure by evaluating whether dimension, exercise or mixed-model theoretical perspectives are supported by reliability outcomes. In a large-scale study (Ncandidates = 2917) utilizing Bayesian generalizability theory, we tested reliability estimates configured to conform to dimension, exercise or mixed-model perspectives. Our findings reveal that reliability outcomes for AC ratings greatly depend on the measurement intentions of the researcher. When this intent aligned with the traditional dimension perspective, we found evidence that reliability was unacceptably low (mean reliability = .38, SD = .15). However, when the intent aligned with the exercise perspective, we found evidence that reliability exceeded acceptable criteria (mean reliability = .91, SD = .09). The addition of dimension- to exercise-related effects to reflect a mixed-model perspective did not make an appreciable difference to reliability.
Original languageEnglish
JournalJournal of Occupational and Organizational Psychology
Early online date15 Jun 2022
DOIs
Publication statusE-pub ahead of print - 15 Jun 2022

Keywords

  • assessment centers
  • reliability
  • generalizability theory
  • situational bandwidth
  • multitrait-multimethod

Cite this