Everything that you have ever been told about assessment center ratings is confounded

Duncan J.R. Jackson, George Michaelides, Chris Dewberry, Young Jae Kim

Research output: Contribution to journalArticlepeer-review

32 Citations (Scopus)

Abstract

Despite a substantial research literature on the influence of dimensions and exercises in assessment centers (ACs), the relative impact of these 2 sources of variance continues to raise uncertainties because of confounding. With confounded effects, it is not possible to establish the degree to which any 1 effect, including those related to exercises and dimensions, influences AC ratings. In the current study (N = 698) we used Bayesian generalizability theory to unconfound all of the possible effects contributing to variance in AC ratings. Our results show that ≤1.11% of the variance in AC ratings was directly attributable to behavioral dimensions, suggesting that dimension-related effects have no practical impact on the reliability of ACs. Even when taking aggregation level into consideration, effects related to general performance and exercises accounted for almost all of the reliable variance in AC ratings. The implications of these findings for recent dimension- and exercise-based perspectives on ACs are discussed.

Original languageEnglish
Pages (from-to)976-994
Number of pages19
JournalJournal of Applied Psychology
Volume101
Issue number7
DOIs
Publication statusPublished - 1 Jul 2016
Externally publishedYes

Keywords

  • Assessment centers
  • Bayesian analysis
  • Generalizability theory

Cite this