Random-effects meta-analysis methods for Intervention Reviews

When performing a meta-analysis, different methods for calculating the summary effect, heterogeneity (between-study) variance and their allied confidence intervals may yield different, or even conflicting, results. The performance of these methods may vary depending on the characteristics of the meta-analysis (e.g., the number and size of the included studies). The Wald-type confidence interval and the DerSimonian and Laird (DL) estimator may be misleading, particularly in small meta-analyses. The Cochrane's Statistical Methods Group recommended the following alternative methods to be implemented in RevMan based on the literature, including statistical simulations and empirical evaluations.

  • The restricted maximum likelihood (REML) method to estimate the heterogeneity variance.
  • Methods to calculate confidence intervals for the heterogeneity variance using the Q-profile statistics method.
  • The Hartung and Knapp, Sidik and Jonkman (HKSJ) method for calculating confidence intervals for the summary effect.
  • Methods to calculate prediction intervals (i.e. using the t-distribution or the normal distribution)

Update May 2025: In addition to the resources available in the Cochrane Handbook for Systematic Reviews of Interventions and from the Methods Support Unit, this brief explainer may be helpful for planning analyses when using the random-effects model in systematic reviews of interventions.  

Update January 2025: The new random-effects methods in RevMan have been implemented. Authors can now use these methods in their protocols, reviews, and updates and Chapter 10 in the Cochrane Handbook provides more information about these methods. To learn more about these methods and view a demonstration on their use in RevMan, please visit the Methods Support Unit Web Clinics page. 

Update May 2024: The following addendums to the original proposal have been approved to ensure feasibility and consistency among the proposed methods:

  • To use the Q-profile method to calculate the confidence interval for the heterogeneity variance due to implementation issues related to the generalized Q-statistic method.
  • To use Tau-square to calculate the I-square statistics in order to reflect the estimated heterogeneity variance under DL or REML.

September 2022: Cochrane endorsed these new statistical methods for fitting random effects models following a recommendation from the Methods Executive, which was informed by the relevant Methods Group, Cochrane's Methods Support Unit, the Head of Methods and Evidence Synthesis/Deputy Editor-in-Chief, a Cochrane Review Group representative and the Methods Executive representative on Cochrane's Editorial Board. Implementation developments are underway.

 

Key resources: