Random-effects meta-analysis methods for Intervention Reviews

When performing a meta-analysis, different methods for calculating the summary effect, heterogeneity (between-study) variance and their allied confidence intervals may yield different, or even conflicting, results. The performance of these methods may vary depending on the characteristics of the meta-analysis (e.g., the number and size of the included studies). The Wald-type confidence interval and the DerSimonian and Laird (DL) estimator may be misleading, particularly in small meta-analyses. The Cochrane's Statistical Methods Group recommended the following alternative methods to be implemented in RevMan based on the literature, including statistical simulations and empirical evaluations.

  • The restricted maximum likelihood (REML) method to estimate the heterogeneity variance.
  • Methods to calculate confidence intervals for the heterogeneity variance using the Q-profile statistics method.
  • The Hartung and Knapp, Sidik and Jonkman (HKSJ) method for calculating confidence intervals for the summary effect.
  • Methods to calculate prediction intervals (i.e. using the t-distribution or the normal distribution)

 

Update May 2024: The following addendums to the original proposal have been approved to ensure feasibility and consistency among the proposed methods:

  • To use the Q-profile method to calculate the confidence interval for the heterogeneity variance due to implementation issues related to the generalized Q-statistic method.
  •  To use Tau-square to calculate the I-square statistics in order to reflect the estimated heterogeneity variance under DL or REML.

September 2022: Cochrane endorsed these new statistical methods for fitting random effects models following a recommendation from the Methods Executive, which was informed by the relevant Methods Group, Cochrane's Methods Support Unit, the Head of Methods and Evidence Synthesis/Deputy Editor-in-Chief, a Cochrane Review Group representative and the Methods Executive representative on Cochrane's Editorial Board. Implementation developments are underway.

 

Key resources: