When performing a meta-analysis, different methods for calculating the summary effect, heterogeneity (between-study) variance and their allied confidence intervals may yield different, or even conflicting, results. The performance of these methods may vary depending on the characteristics of the meta-analysis (e.g., the number and size of the included studies). The Wald-type confidence interval and the DerSimonian and Laird (DL) estimator may be misleading, particularly in small meta-analyses. The Cochrane's Statistical Methods Group recommended the following alternative methods to be implemented in RevMan based on the literature, including statistical simulations and empirical evaluations.
- The restricted maximum likelihood (REML) method to estimate the heterogeneity variance.
- Methods to calculate confidence intervals for the heterogeneity variance using the Q-profile statistics method.
- The Hartung and Knapp, Sidik and Jonkman (HKSJ) method for calculating confidence intervals for the summary effect.
- Methods to calculate prediction intervals (i.e. using the t-distribution or the normal distribution)
Update May 2024: The following addendums to the original proposal have been approved to ensure feasibility and consistency among the proposed methods:
- To use the Q-profile method to calculate the confidence interval for the heterogeneity variance due to implementation issues related to the generalized Q-statistic method.
- To use Tau-square to calculate the I-square statistics in order to reflect the estimated heterogeneity variance under DL or REML.
September 2022: Cochrane endorsed these new statistical methods for fitting random effects models following a recommendation from the Methods Executive, which was informed by the relevant Methods Group, Cochrane's Methods Support Unit, the Head of Methods and Evidence Synthesis/Deputy Editor-in-Chief, a Cochrane Review Group representative and the Methods Executive representative on Cochrane's Editorial Board. Implementation developments are underway.
Key resources:
- Video- lecture presented by Areti Angeliki Veroniki (Co-convenor of the Cochrane Statistical Methods Group). Recorded during the 2022 online training programme for statistical editors and statisticians supporting Cochrane groups.