Members of the Cochrane GRADEing conduct research on the applicability and the presentation of evidence. In addition, the group disseminates relevant research conducted by its members and other investigators.
CURRENT PROJECTS
Comparison between the standard and a new alternative format of the Summary-of-Findings tables in Cochrane review users: study protocol for a randomized controlled trial. Systematic reviews represent one of the most important tools for knowledge translation but users often struggle with understanding and interpreting their results. GRADE Summary of Findings tables have been developed to display results of systematic reviews in a concise and transparent manner. The current format of the Summary of Findings tables for presenting risks and quality of evidence improves understanding and assists users with finding key information from the systematic review. However, it has been suggested that additional methods to present risks and display results in the Summary of Findings tables are needed. We will conduct a non-inferiority parallel-armed randomized controlled trial to determine whether an alternative format to present risks and display Summary of Findings tables is not inferior compared to the current standard format. PDF
Summary of Findings (SoF) tables in diagnostic test accuracy reviews. In this Cochrane MIF project we used an iterative process based on the analysis of data from rounds of discussions, feedback and user testing. More than 150 users participated in initial discussions and provided their formal and informal feedback. 20 users completed one-on-one user testing interviews. Almost all participants preferred summarizing the results of systematic reviews of test accuracy in tabular format rather than plain text. Users generally preferred less complex tables but found presenting sensitivity and specificity estimates only as too simplistic. This work led to the development of the current formats of the SoF tables for test accuracy as suggested by the GRADE approach. These formats can be produced using GRADEpro Guideline Development Tool (www.gradepro.org) and are being further developed into electronic interactive tables that will suit the needs of different users.
DECIDE
The EU funded DECIDE project (www.decide-collaboration.eu) aims to develop and evaluate methods that address the targeted dissemination of guidelines by building on the work of the GRADE Working Group. One of the products that is currently user tested is the interactive SoF table (iSoF). The mean features are layers of information, from simple to complex, and the choice of viewing evidence as test, numbers or graphics. The DECIDE project, initiated and developed by the GRADE working group, which runs from 2011 to 2015 (www.decide-collaboration.eu), attempts to take advantage of technological advances to improve the SoF table. As a result, the interactive SoF table (iSoF) uses electronic presentation of information to reconcile precision and simplicity through a layered approach to information presentation. In addition, the user has control over how many and which outcomes the table displays and in which format (words, numbers, graphics). The iSoF also has a column to express results in a narrative format in addition to statistical formats, or in lieu of these when statistical data are not available. Additional advantages of the iSoF include an interactive presentation and explanation of confidence intervals and other statistical terms, and a responsive format for printing and for displaying on different sizes and types of electronic devices. Future plans include the translation of the iSoF into different languages. The iSoF will be incorporated in GRADEpro and possibly other interfaces.
Development of standardized reporting and wording of results and interpretation. Having assessed the risk of bias of individual studies, pooled and analysed results, and rated the quality of the body of evidence, authors need to interpret the results, draw conclusions and present their findings, and do it consistently across the review. Audits of reviews by the Cochrane Editorial Unit indicate that review authors are struggling with interpretation and reporting, and in particular with wording of their findings in the results and discussion section of the review. This project aims to improve standardized wording to present results and conclusions in the results section (including the risk of bias section), the discussion (including the quality of the evidence), the abstract, the plain language summary, and the Summary of Findings Table, through work with a core group and a Delphi process with key stakeholders in the review process. A major future direction for the presentation formats of PLS and SoF tables is making both of them usable by both audiences and cross-linking them for those who are interested in both.
Assessing the quality of evidence using GRADE and presenting results from non‐randomized studies in Cochrane systematic reviews. Until now, the Cochrane Handbook includes methods for reviews focusing randomized studies (RS). With the increasing use of non-randomized studies (NRS) evidence in Cochrane reviews, the launch of the NRSi ACROBAT tool, authors need guidance specific to the use of GRADE and creating Summary of Findings tables (SoF). This project aims to provide assistance for Cochrane authors on how and when to include, assess, and present evidence obtained from NRS in a systematic review, and to facilitate the GRADE assessment in the Summary of Findings (SoF) Tables on a body of evidence from Randomized Studies (RS) and NRS.
Network Meta-analysis of intervention and Summary of Findings tables. Decisions about optimal care, and the clinical practice guidelines that inform these decisions, rely on evidence-based evaluation of the different treatment options. Systematic reviews and meta-analyses of randomised controlled trials are the main sources of evidence. When multiple interventions have been used and compared for the same disease and outcomes, network meta-analysis offers a set of methods to visualize and interpret the wider picture of the evidence and better understanding of the relative merits of these multiple interventions . In a systematic review of interventions, Summary of Findings (SoF) tables present the main findings of a review in a transparent and simple form. They provide key information concerning the certainty of the evidence , the magnitude of treatment effects, and the sum of the available data on the main outcome. We will develop and evaluate a SoF format that can be understood by users of NMAs; both health professionals and other relevant groups. In this project, we expect that health professionals who want to summarise the main findings from NMAs in a condensed form will use the NMA-SoF tables.
PAST PROJECTS
Formatting modifications in GRADE evidence profiles improved guideline panelists comprehension and accessibility to information. A randomized trial. The objective was to determine the effects of formatting alternatives in Grading of Recommendations Assessment, Development, and Evaluation (GRADE) evidence profiles on guideline panelists' preferences, comprehension, and accessibility. PDF
User testing and stakeholder feedback contributed to the development of understandable and useful summary of findings tables for Cochrane Reviews. The objective of this study was to develop an Summary of Findings (SoF) table for use in Cochrane reviews that is understandable and useful for health professionals, acceptable to Cochrane Collaboration stakeholders, and feasible to implement. We gathered stakeholder feedback on the format and content of an SoF table from an advisory group of more than 50 participants. Stakeholders were concerned about precision in the data representation and about production feasibility. User testing revealed unexpected comprehension problems, mainly confusion about what the different numbers referred to (class reference). User testing led to a table more useful and understandable for clinical audiences. PDF
Summary-of-findings tables in Cochrane reviews improved understanding and rapid retrieval of key information. The objective of this study was to measure the effects of a summary-of-findings (SoF) table on user satisfaction, understanding, and time spent finding key results in a Cochrane review. We randomized participants in an evidence-based practice workshop (randomized controlled trial [RCT] I) and a Cochrane Collaboration entities meeting (RCT II) to receive a Cochrane review with or without an SoF table. In RCT I, we measured user satisfaction. In RCT II, we measured correct comprehension and time spent finding key results. RCT I: Participants with the SoF table were more likely to “agree” or “strongly agree” that it was easy to find results for important outcomes than participants without the SoF table. RCT II: Participants with the SoF table were more likely to correctly answer two questions regarding results than those without the SoF table. In two small trials, we found that inclusion of an SoF table in a review improved understanding and rapid retrieval of key findings compared with reviews with no SoF table. PDF
In connection with the Summary of Findings tables (SoF), research has been conducted on how to communicate the results of systematic reviews to consumers and how these are perceived and understood. This work is currently feeding into the Plain Language Expectations for Authors of Cochrane Summaries (PLEACS) project. The PLEACS group has developed a set of minimum criteria for the content of PLS (www.consumers.cochrane.org/PLEACS). The purpose is to ensure that authors convey the key question and findings of the review in a succinct and clear manner to consumers. The group is also working on improving the narrative reporting of results by testing the inclusion of headings and numbers. The Collaboration is currently exploring how standardized language could contribute to ensuring the quality of translation of PLS and abstracts.
Presenting the Results of Cochrane Systematic Reviews to a Consumer Audience: A Qualitative Study. The objective was to develop and obtain feedback about a summary format for Cochrane reviews that would be accessible to a consumer audience. Using a semi-structured interview guide, three versions of a Plain Language Summary (PLS) format of a Cochrane Systematic Review were tested. Participants preferred results presented as words, supplemented by numbers in a table. They also found it difficult to move between presentations of dichotomous and continuous outcomes. PDF
A summary to communicate evidence from systematic reviews to the public improved understanding and accessibility of information: a randomized controlled trial. The objective was to evaluate a new format of a summary, which presents research from synthesized evidence to patients and the public. We conducted a randomized controlled trial in 143 members of the public from five countries. Participants received either a new summary format (a plain language summary [PLS]) or the current format used in Cochrane systematic reviews. With the new PLS, more participants understood the benefits and harms and quality of evidence; more answered each of the five questions correctly; and they answered more questions correctly. Overall, participants preferred the new PLS. This new PLS format for patients and the public is a promising tool to translate evidence from synthesized research. PDF
The GRADE approach is reproducible in assessing the quality of evidence of quantitative evidence syntheses. We evaluated the inter-rater reliability (IRR) of assessing the quality of evidence (QoE) using the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) approach. On completing two training exercises, participants worked independently as individual raters to assess the QoE of 16 outcomes. After recording their initial impression using a global rating, raters graded the QoE following the GRADE approach. Subsequently, randomly paired raters submitted a consensus rating. Our findings suggest that trained individuals using the GRADE approach improves reliability in comparison to intuitive judgments about the QoE and that two individual raters can reliably assess the QoE using the GRADE system. PDF