Testing for homogeneity in meta-analysis I. The one parameter case: Standardized mean difference

Meta-analysis seeks to combine the results of several experiments in order to improve the accuracy of decisions. It is common to use a test for homogeneity to determine if the results of the several experiments are sufficiently similar to warrant their combination into an overall result. Cochran'sQstatistic is frequently used for this homogeneity test. It is often assumed thatQfollows a chi-square distribution under the null hypothesis of homogeneity, but it has long been known that this asymptotic distribution forQis not accurate for moderate sample sizes. Here, we present an expansion for the mean ofQunder the null hypothesis that is valid when the effect and the weight for each study depend on a single parameter, but for which neither normality nor independence of the effect and weight estimators is needed. This expansion represents an orderO(1/n)correction to the usual chi-square moment in the one-parameter case. We apply the result to the homogeneity test for meta-analyses in which the effects are measured by the standardized mean difference (Cohen'sd-statistic). In this situation, we recommend approximating the null distribution ofQby a chi-square distribution with fractional degrees of freedom that are estimated from the data using our expansion for the mean ofQ. The resulting homogeneity test is substantially more accurate than the currently used test. We provide a program available at the Paper Information link at theBiometricswebsitefor making the necessary calculations.