Clarifying the scope of generalizability theory for multifaceted assessment

Generalizability theory (G theory) continues to be underutilized in applied psychological research, both in New Zealand and internationally, possibly due to uncertainties about the types of questions that it can be used to address. G theory and its associated random effects model basis is often positioned as an approach limited to the study of reliability. In contrast, latent variable theory, and its confirmatory factor analytic (CFA) basis, is used more widely to address issues of validity whilst controlling for reliability. This study clarifies the types of questions to which G theory can be applied by testing whether there is any justification for the difference in interpretation between results based on G theory and latent variable theory. We reanalyzed data from an operational assessment center (N = 214 managerial assessees) and found comparable aggregated effects, generalizability coefficients, and latent scores across the G theory and latent variable theory approaches, suggesting that both can be applied to problems related to reliability and structural validity.