Reliability in assessment centres depends on general and exercise performance, but not on dimensions

Duncan J. R. Jackson, George Michaelides, Chris Dewberry, Jo Nelson, Catherine Stephens

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)
5 Downloads (Pure)

Abstract

This study contributes to the literature on assessment centre (AC) measurement structure by evaluating whether dimension, exercise or mixed-model theoretical perspectives are supported by reliability outcomes. In a large-scale study (Ncandidates = 2917) utilizing Bayesian generalizability theory, we tested reliability estimates configured to conform to dimension, exercise or mixed-model perspectives. Our findings reveal that reliability outcomes for AC ratings greatly depend on the measurement intentions of the researcher. When this intent aligned with the traditional dimension perspective, we found evidence that reliability was unacceptably low (mean reliability = .38, SD = .15). However, when the intent aligned with the exercise perspective, we found evidence that reliability exceeded acceptable criteria (mean reliability = .91, SD = .09). The addition of dimension- to exercise-related effects to reflect a mixed-model perspective did not make an appreciable difference to reliability.
Original languageEnglish
Pages (from-to)739-757
Number of pages19
JournalJournal of Occupational and Organizational Psychology
Volume95
Issue number4
Early online date15 Jun 2022
DOIs
Publication statusPublished - Dec 2022

Keywords

  • assessment centers
  • reliability
  • generalizability theory
  • situational bandwidth
  • multitrait-multimethod
  • assessment centres

Cite this