A tool for assessing Risk Of Bias due to Missing Evidence in a synthesis (ROB-ME)

We are delighted that the tool for assessing Risk Of Bias due to Missing Evidence in systematic reviews with meta-analysis (ROB-ME) has been published in the BMJ.

The reporting of primary studies or results may be influenced by magnitude, direction, and statistical significance of the study results. This can lead to bias in a meta-analysis because the available evidence differs systematically from the missing evidence. While existing methods mostly help users assess whether selective non-publication of studies or selective non-reporting of study results has occurred, ROB-ME presents the first structured approach for assessing the risk of bias that arises when entire studies or particular results within studies are missing from meta-analyses because of the p-value, magnitude, or direction of the study results (i.e., due to non-reporting biases).

The scope, development, and an overview of the tool together with worked examples of applying the tool are published in the BMJ.

The full ROB-ME tool and the guidance documents are available here.

The ROB-ME tool is intended for authors of systematic reviews to assess risk of bias due to missing evidence in meta-analyses of the effects of interventions, in any other type of pairwise synthesis that yields a point estimate of an intervention effect (e.g. calculation of the median effect across studies when meta-analysis is not possible or appropriate), or in syntheses consisting of only one study with available results.

ROB-ME operates in the same manner as the RoB 2 and ROBINS-I tools, whereby responses to signalling questions provide the basis for a judgement about the risk of bias in the specific synthesized result being assessed.

Watch Matthew Page, Julian Higgins, and Jonathan Sterne introduce the tool here.