The Comparing Multiple Interventions Methods Group (CMIMG) has received funding from the Cochrane Methods Innovation Fund to address specific issues related to the comparison of multiple interventions in Cochrane reviews. The project involves three meetings to bring together investigators, methodologists, authors, consumers, managing editors and other interested parties to help CMIMG develop consensus guidance for carrying out reviews of multiple intervention.
For a full list of the products of this project please see Comparing Multiple Interventions in Cochrane Reviews
The second meeting was held by Group 2 in Bristol, UK on July 16 and 17, 2013 and addressed statistical issues in network Meta-analysis
The meeting was informed by a A discussion document that describes all relevant methodologies suggested in the scientific literature to date.
Slide Sets from presentations at the meeting:
Session 1: Models for Network Meta-Analysis and evaluation of assumptions
Session 2: Presenting data from the network and results from Network Meta-Analysis
Session 3: Selecting the appropriate effect size for Network Meta-Analysis
Session 4: Evaluation of inconsistency
|See also - Donegan, S., Williamson, P., D’Alessandro, U., & Tudur Smith, C. (2013). Assessing key assumptions of network meta-analysis: a review of methods.Research Synthesis Methods, doi:10.1002/jrsm.1085|
Session 5: What to do with inconsistent networks
Summary of Minutes from the meeting
Conventional Cochrane intervention reviews with pairwise meta-analyses were criticized for not addressing sufficiently the assumptions underlying principled analysis of data, e.g. the study inclusion/exclusion criteria are often unclear. Moreover, several published Cochrane reviews include informal indirect comparisons without stating the assumptions being made. Consequently, enhancing Cochrane reviews to include network meta-analysis (NMA) needs very careful consideration with respect to acknowledging and evaluating the underlying assumptions. The authors should built up their network very carefully and consider the implications of the inclusion criteria for studies and treatments. Cochrane protocols and reviews with multiple interventions should include a marked section where they will defend the transitivity assumption that underlies NMA; this can be reflected on the concepts that treatments are jointly randomizable and that missing arms in studies are missing at random.
Guidance to reviewers is needed about how to deal with inconsistency and there is a wealth of methods that can be recommended. Both local tests (that identify whether a part of the network e.g. a specific comparison or ‘evidence loop’ is associated with inconsistency) and global tests (that infer about inconsistency in the entire network) are useful. Guidance needs to highlight that in the case that only indirect evidence is available, the model assumptions can be still evaluated by comparing the distribution of effect modifiers. Cochrane reviewers are familiar with strategies to address and account for heterogeneity; Cochrane guidance for inconsistency should draw in the analogy with heterogeneity.
Network graphs offer a useful options to present the evidence base but they need to be quite flexible in terms of shape and when adding/excluding treatments. Reviewers can use tables to present the summary relative treatment effects estimated in NMA; if few treatments are compared a ‘league table’ with all treatment comparisons can be presented. Otherwise reviewers can select a reference treatment (e.g placebo) and present the treatment effects compared to that reference. Ranking measures such as mean or median ranks and SUCRAs can be presented optionally along with the relative effects providing also confidence intervals for ranking if possible. The order of presenting treatment results could follow their ranking on the most important outcome.
Written guidance in Cochrane Handbook should be short and does not need to provide statistical details. A ‘good practice’ and/or ‘good reporting’ checklist drawing on ongoing works from various teams (e.g the NICE, ISPOR and PRISMA) would be helpful to guide reviewers while they perform and report NMA.