Methods Innovation Fund 2

Development of guidance for inclusion of adverse effects in systematic reviews - PROJECT COMPLETED

Project is completed and a new Handbook chapter was presented to the Handbook editors for review for version 6.

Lead investigator: Yoon Loke (Co-convenor Adverse Effects MG endorsed by the Non-Randomised Studies MG and the Information Retrieval MG).

This project appraised evolving work into more reliable or useful ways of identifying and summing up the whole body of evidence on harmful effects to produce concise guidance for researchers on the best methods for assessing harm through systematic reviews. 

Interim guidance on the inclusion of clinical study reports and other regulatory documents in Cochrane Reviews - PROJECT COMPLETED

A glossary of terminology and guidance on when to incorporate clinical study reports and other regulatory data not in published peer review journal is available here.

Lead investigator: Tom Jefferson (Supported by the Information Retrieval MG, Bias MG, Adverse Effects MG, and Individual Participant Data Meta-analysis MG )

Jefferson T, Doshi P, Boutron I, et al.  When to include clinical study reports and regulatory documents in systematic reviews. BMJ Evidence-Based Medicine Published Online First: 11 October 2018. doi: 10.1136/bmjebm-2018-110963

Guidance document

Methods for systematic review and meta-analysis of prognostic factors and prediction modelling studies - PROJECT COMPLETED

The team have developed protocol templates and methods for meta-analysis for prognosis studies. They continue to run workshops at major events. More information.

Lead investigator: Karel Moons (Co-convenor Prognosis MG )

This project will support the completion of prognostic exemplar reviews and develop new related training materials. The team will further develop necessary methods for this novel type of Cochrane Review. This will include testing the use of the QUIPS (Quality in Prognostic Studies) and CHARMS (CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies) tools for critical appraisal of prognostic factors and modelling studies, respectively. They will test methods of prognostic factor systematic review literature searching, develop and illustrate meta-analysis methods for summarizing results of prognostic modelling studies.

Assessing the quality of evidence using GRADE and presenting results from non-randomized studies in Cochrane Systematic Reviews - PROJECT COMPLETED

Lead investigator: Holger Schünemann (Co-convenor Applicability and Recommendations MG)

This project builds on previous work from the Applicability and Recommendations Methods Group, and results of work from the MIF 1 project. It will assist Cochrane authors to decide how and when to include, assess, and present evidence obtained from non-randomized studies (NRS) in a systematic review, and to facilitate the GRADE assessment in the ‘Summary of findings’ (SoF) table on a body of evidence from both randomized studies (RS) and NRS. With the launch of the ROBINS-I tool, authors need guidance specific to the use of GRADE and creating SoF tables. Specifically, the project will address solutions for:

when to include NRS in a systematic review and in SoF tables;

how to incorporate the risk of bias for NRS (ROBINS-I) into the GRADE assessments, and how to GRADE evidence from NRS (including more details about criteria to upgrade evidence);

how to draw conclusions based on a consideration of effect and levels of evidence from both RS and NRS combined (e.g. when conflicting results from RS and NRS are present, but they have the same level of evidence); and

how to present both RS and NRS combined data in a SoF table.


  • Cuello CA, Morgan RL, Brozek J, Santesso N, Verbeek J, Thayer K, Guyatt G, Schünemann HJ. A Scoping Review and Survey Provides the Rationale, Perceptions, and Preferences for the Integration of Randomized and Non-Randomized Studies in Evidence Syntheses and GRADE Assessments. J Clin Epidemiol. 2018 Feb 13. pii: S0895-4356(17)30889-2. doi: 10.1016/j.jclinepi.2018.01.010. [Epub ahead of print] PubMed PMID: 29452221.
  • Schünemann HJ, Cuello C, Akl EA, Mustafa RA, Meerpohl JJ, Thayer K, Morgan RL, Gartlehner G, Kunz R, Katikireddi SV, et al. GRADE guidelines: 18. How ROBINS-I and other tools to assess risk of bias in nonrandomized studies should be used to rate the certainty of a body of evidence. J Clin Epidemiol. 2018 Feb 9. pii: S0895-4356(17)31031-4. doi: 10.1016/j.jclinepi.2018.01.012. [Epub ahead of print] PubMed PMID: 29432858.
  • Case studies to explore the optimal use of randomized and non-randomized studies in evidence syntheses that use GRADE. (Submitted to the JCE, under revision).
  • GRADE guidelines. The role of randomized and non-randomized studies in evidence syntheses of health interventions (in preparation)

Developing plain language summaries for DTA reviews - PROJECT COMPLETED

Interim guidance for DTA Plain language summaries available here.

Lead investigator: Penny Whiting (Supported by the Screening and Diagnostic Tests MG )

This project will investigate what information potential users of diagnostic reviews would like to have included in PLS and how they would prefer this information to be presented. Users include the science and health-interested public, health professionals, policy makers and journalists. They will also explore how this information can be best described using plain language. A combination of focus groups, user testing, and a web-based survey methods will be used to develop guidance for authors of reviews of diagnostic tests about the information needed to write an informative PLS for potential users of these reviews.

Statistical methods for updating meta-analyses - PROJECT COMPLETED

Lead investigator: Mark Simmonds (Supported by the Statistics Methods Group )

This project aimed to identify all the proposed statistical methods for updating or repeating meta-analyses and compare their properties by applying them to existing Cochrane Reviews and using simulated meta-analyses. The results will be used to recommend which methods are most appropriate when updating a meta-analysis, and to inform discussions within Cochrane about when an update is needed.

The CERQual approach for assessing how much confidence to place in findings from qualitative evidence syntheses – Development of component 1: Cochrane qualitative Methodological Limitations Tool (CAMELOT) - PROJECT COMPLETED

An overview of the CerQual approach is here. for further information on developments please go here.

Lead investigator: Claire Glenton (Supported by the Qualitative and Implementation Methods Group )

This project will be the first phase of several phases required. This phase will review existing critical appraisal tools for qualitative research and identify common elements across tools. This work and the following phases will result in a Cochrane qualitative Methodological Limitations Tool (CAMELOT) for incorporation in CERQual. The output will be publication of an overview of existing tools and common elements across these tools.