Combining the evidence using stable weights

In a meta-analysis one seeks to combine the results of several studies in order to improve the accuracy of decisions. Here we compare by simulation four methods for combining estimates of the risk difference, namely the Cochran and Mantel–Haenszel (MH) methods, the inverse-variance weights approach and a recent variance-stabilized weights approach. Both the level and power of corresponding test statistics, as well as the coverage of related confidence intervals are compared over a wide range of sample size and parameter configurations.

We found that the inverse-variance weights methodology is unreliable and is not recommended, while for equal risks, the Cochran test and the associated confidence intervals are the most reliable. Under alternatives of unequal risks, the coverage probabilities of the variance-stabilized confidence intervals are almost uniformly more reliable than those based on other methods except when the average risk is small in which case the MH confidence intervals are preferable.