Tools

The Prognosis Methods Group has developed guidance to perform a systematic review of prognosis studies.
Links to articles and tools can be found below, ordered by review step.
This list will be updated once more guidance becomes available.

Templates
The PMG has developed templates for writing a protocol and writing a review.
Peer review templates are available for a protocol and a full review

Introduction to systematic reviews of prognosis studies:
-   Prognosis research: toward evidence-based results and a Cochrane methods group (Riley et al, J Clin Epidemiol 2007).
-   Implementing systematic reviews of prognosis studies in Cochrane (Moons et al, Cochrane Database Syst Rev 2018).

Full description of the review process (including meta-analysis), from A to Z:
-   A guide to systematic review and meta-analysis of prediction model performance (Debray et al, BMJ 2017).
-   A guide to systematic review and meta-analysis of prognostic factor studies (Riley et al, BMJ 2019).

Searching for studies:
-   Search Filters for Finding Prognostic and Diagnostic Prediction Studies in Medline to Enhance Systematic Reviews (Geersing et al, PLOS One 2012).
-   Optimal search strategies for retrieving scientifically strong studies of treatment from Medline: analytical survey (Haynes et al, BMJ 2005).
-   Searching for clinical prediction rules in MEDLINE (Ingui et al, J Am Med Inform Assoc 2001).
-   Development and evaluation of a search filter to identify prognostic factor studies in Ovid MEDLINE (Stallings et al, BMC MRM 2022).

Formulating the review question, data extraction and critical appraisal:
-   Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies: The CHARMS Checklist (Moons et al, PLOS Med 2014).
-   Example data extraction forms are available for scoping reviews of prognostic models and reviews of prognostic model validation studies.

Risk of bias assessment:
-   Assessing Bias in Studies of Prognostic Factors (Hayden et al, Ann Intern Med 2013).
-   Evaluation of the Quality of Prognosis Studies in Systematic Reviews (Hayden et al, Ann Intern Med 2006)
-   PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies (Wolff et al, Ann Intern Med 2019).
-   PROBAST: A Tool to Assess Risk of Bias and Applicability of Prediction Model Studies: Explanation and Elaboration (Moons et al, Ann Intern Med 2019). See www.probast.org for the latest version of the PROBAST tool.
-   A revised tool for assessing risk of bias in randomized trials (Higgins et al, Cochrane Database Syst Rev 2016). Available via riskofbias.info.

Dealing with missing data:
-   Extracting summary statistics to perform meta-analyses of the published literature for survival endpoints (Parmar et al, Stat Med 1998).
-   Practical methods for incorporating summary time-to-event data into meta-analysis (Tierney et al, Trials 2007).
-   Interaction revisited: the difference between two estimates (Altman et al, BMJ 2003).

Meta-analysis:
-   Meta-analysis and aggregation of multiple published prediction models (Debray et al, Stat Med 2014).
-   External validation of clinical prediction models using big datasets from e-health records or IPD meta-analysis: opportunities and challenges (Riley et al, BMJ 2016).
-   Meta-analysis of prediction model performance across multiple studies: Which scale helps ensure between-study normality for the C-statistic and calibration measures? (Snell et al, Stat Methods Med Res 2017).
-   Meta-analysis of a binary outcome using individual participant data and aggregate data (Riley et al, Res Synth Methods 2010).
-   Multivariate meta-analysis of individual participant data helped externally validate the performance and implementation of a prediction model (Snell et al, J Clin Epidemiol 2016).
-   A random-effects regression model for meta-analysis (Berkey et al, Stat Med 1995).
-   A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes (Debray et al, Stat Methods Med Res 2018).
-   An example R script is available for meta-analysis of c-statistics. This script uses an example dataset.

Reporting bias:
-   Detecting small-study effects and funnel plot asymmetry in meta-analysis of survival data: a comparison of new and existing tests (Debray et al, Research Synthesis Methods 2018)

GRADE:
-   Judging the quality of evidence in reviews of prognostic factor research: adapting the GRADE framework (Huguet et al, Syst Rev 2013)
-   Use of GRADE for assessment of evidence about prognosis: rating confidence in estimates of event rates in broad categories of patients (Iorio et al, BMJ 2015)
-   GRADE Guidelines 28: Use of GRADE for the assessment of evidence about prognostic factors: rating certainty in identification of groups of patients with different absolute risks (Foroutan et al, J Clin Epidemiol 2020)
-   GRADE concept paper 2: Concepts for judging certainty on the calibration of prognostic models in a body of validation studies (Foroutan et al, J Clin Epidemiol 2022)

Reporting of systematic reviews:
-   Transparent reporting of multivariable prediction models for individual prognosis or diagnosis: checklist for systematic reviews and meta-analyses (TRIPOD-SRMA) (Snell et al, BMJ 2023)
-   Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement (Moher et al, PLOS Med 2009)
-   Meta-analysis of observational studies in epidemiology: a proposal for reporting (Stroup et al, JAMA 2000)

Different types of primary prognosis studies:
-   Prognosis Research Strategy (PROGRESS) 1: A framework for researching clinical outcomes (Hemingway et al, BMJ 2013).
-   Prognosis Research Strategy (PROGRESS) 2: Prognostic Factor Research (Riley et al, PLOS Med 2013).
-   Prognosis Research Strategy (PROGRESS) 3: Prognostic Model Research (Steyerberg et al, PLOS Med 2013).
-   Prognosis Research Strategy (PROGRESS) 4: Stratified medicine research (Hingorani et al, BMJ 2013).