TY - JOUR
T1 - On the Q statistic with constant weights for standardized mean difference
AU - Bakbergenuly, Ilyas
AU - Hoaglin, David C.
AU - Kulinskaya, Elena
N1 - Funding Information: The work by E. Kulinskaya was supported by the Economic and Social Research Council [grant number ES/L011859/1].
PY - 2022/11
Y1 - 2022/11
N2 - Cochran's Q statistic is routinely used for testing heterogeneity in meta-analysis. Its expected value is also used in several popular estimators of the between-study variance, (Formula presented.). Those applications generally have not considered the implications of its use of estimated variances in the inverse-variance weights. Importantly, those weights make approximating the distribution of Q (more explicitly, (Formula presented.)) rather complicated. As an alternative, we investigate a new Q statistic, (Formula presented.), whose constant weights use only the studies' effective sample sizes. For the standardized mean difference as the measure of effect, we study, by simulation, approximations to distributions of (Formula presented.) and (Formula presented.), as the basis for tests of heterogeneity and for new point and interval estimators of (Formula presented.). These include new DerSimonian–Kacker-type moment estimators based on the first moment of (Formula presented.), and novel median-unbiased estimators. The results show that: an approximation based on an algorithm of Farebrother follows both the null and the alternative distributions of (Formula presented.) reasonably well, whereas the usual chi-squared approximation for the null distribution of (Formula presented.) and the Biggerstaff–Jackson approximation to its alternative distribution are poor; in estimating (Formula presented.), our moment estimator based on (Formula presented.) is almost unbiased, the Mandel – Paule estimator has some negative bias in some situations, and the DerSimonian–Laird and restricted maximum likelihood estimators have considerable negative bias; and all 95% interval estimators have coverage that is too high when (Formula presented.), but otherwise the Q-profile interval performs very well.
AB - Cochran's Q statistic is routinely used for testing heterogeneity in meta-analysis. Its expected value is also used in several popular estimators of the between-study variance, (Formula presented.). Those applications generally have not considered the implications of its use of estimated variances in the inverse-variance weights. Importantly, those weights make approximating the distribution of Q (more explicitly, (Formula presented.)) rather complicated. As an alternative, we investigate a new Q statistic, (Formula presented.), whose constant weights use only the studies' effective sample sizes. For the standardized mean difference as the measure of effect, we study, by simulation, approximations to distributions of (Formula presented.) and (Formula presented.), as the basis for tests of heterogeneity and for new point and interval estimators of (Formula presented.). These include new DerSimonian–Kacker-type moment estimators based on the first moment of (Formula presented.), and novel median-unbiased estimators. The results show that: an approximation based on an algorithm of Farebrother follows both the null and the alternative distributions of (Formula presented.) reasonably well, whereas the usual chi-squared approximation for the null distribution of (Formula presented.) and the Biggerstaff–Jackson approximation to its alternative distribution are poor; in estimating (Formula presented.), our moment estimator based on (Formula presented.) is almost unbiased, the Mandel – Paule estimator has some negative bias in some situations, and the DerSimonian–Laird and restricted maximum likelihood estimators have considerable negative bias; and all 95% interval estimators have coverage that is too high when (Formula presented.), but otherwise the Q-profile interval performs very well.
UR - http://www.scopus.com/inward/record.url?scp=85123882398&partnerID=8YFLogxK
U2 - 10.1111/bmsp.12263
DO - 10.1111/bmsp.12263
M3 - Article
SN - 0007-1102
VL - 75
SP - 444
EP - 465
JO - British Journal of Mathematical and Statistical Psychology
JF - British Journal of Mathematical and Statistical Psychology
IS - 3
ER -