Systematic Review Replication

Abstract

Replication is a corner stone of the scientific method yet replication of systematic reviews is too often overlooked, done unnecessarily or done poorly, as identified by Francis Collins, the Director of the National Institutes of Health (NIH). Excessive replication - doing the same study repeatedly - is unethical and a cause of ‘research wastage’. There is a lack of guidance for when or why to replicate systematic reviews.

Overarching Aim: The proposed research will use an evidence-driven, transparent process and implement consensus approaches to develop value-added guidance on: a) when to replicate and b) when not to replicate systematic reviews.

Objectives: We propose three sequential objectives:

1. To assess the extent of replication of systematic review topics and explore reasons for replication as well as issues such as discordance, agreement and bias.

2. To develop consensus-based recommendations on when to replicate systematic reviews of effectiveness (and when not), taking into account the needs and preferences of the various stakeholder groups.

3. To identify strategies for disseminating and implementing these recommendations to key stakeholder groups (e.g. patients, clinicians, journal editors, systematic review authors, systematic review users, systematic review organizations (such as Cochrane, Campbell, Joanna Briggs Institute), guideline developers, research funding agencies, KT researchers).

To achieve these objectives we will: assess evidence about when to replicate systematic reviews, document characteristics of overlapping systematic reviews, survey and interview authors of replicated systematic reviews, consult with global experts on replicability guidelines, prioritize and seek consensus on items for systematic review replication guidance, and develop collaborative knowledge translation plan including practical tools for specific knowledge users.

Proposed Papers and Leads

Paper

Lead author

Co-authors

Target journal

Target deadline

Current status

Guideline paper

Peter, Sathya, Vivian

All

BMJ

Draft in circulation

June 27

Framework and definitions for SR replication

Jeremy 

Jeff Valentine, Mark Petticrew, Brigitte Vachon, Alison Riddle, Elie Akl, Brigitte Vachon, Lara Maxwell, Laura Weeks, Kednapa Thavorn, Zulfi Bhutta, Karla Soares, Bev Shea, Betsy Kristjansson, Sathya, Vivian, Peter T

Research Synthesis Methods, 

J Clin Epi

Fall, 2019


Framework under development 

SR replication for internal validity 

Jeremy 

Jeff, Jeremy, Zulfi, Terri, Bev, Sathya, Larissa, Peter

Discordance checklist

Sathya

Elie, Janet, Lara, Howard, Gabriel, Betsy, Terri, Jeff, Peter T, Julian, Tamara, Larissa, Sathya, Vivian, Peter T

Stakeholder interviews & Survey

Sathya

Peter T, Tamara, Vivian, Lara

Cochrane methods SR

Sathya

Jennifer, Peter T, Zulfi, Vivian, Lara, Jordi

Cochrane & Campbell

December, 2019 

Protocol reviewed; Responding to reviewer comments 

Discussions underway for a possible Lancet commentary

Peter, Vivian & Jocalyn Clark

Janet, George, David, Sathya

To be discussed after main paper is submitted

Evaluation plan

Peter, Sathya, Vivian

David, Betsy, Howard, Kednapa, Brigitte, Sathya, Vivian, Peter T

Expand VOI

Kednapa

Howard, Peter T, Vivian, Lara, Mark P, George, Janet, Peter W, Bev, Sathya, Vivian, Peter T

Series of other papers: editorials and think pieces?

 

 Conferences and Meetings 2019-20

Conference

Meeting Dates & Venue

Lead Presenter 

Presentation title

CADTH

April 14-16, 2019

Edmonton, Canada

Kednapa Thavorn

Using a conceptual value of information approach to decide when to, and when not to replicate systematic reviews

SRSM

July 23-25, 2019

Chicago, USA

1. Vivian Welch

2. Jeff Valentine 

1. When to, and when not to replicate systematic reviews: consensus meeting and checklist

2. Towards a framework to replicate systematic reviews

Sicily evidence based healthcare (EBHC)

November 6-9, 2019

Messina, Italy

Janet Martin

Abstract submitted:

To Replicate, or Not to Replicate: A Value of Information Approach to Decide Whether to Replicate Systematic Reviews

Campbell-what works global summit

October 16-18, 2019

Mexico City, Mexico

Vivian Welch, Jeremy Grimshaw, Howard White, Terri Pigott, Betsy Kristjansson, Peter Tugwell

Panel presentation (abstract submitted):

Developing guidance for when to, and when not to replicate systematic reviews

Cochrane

October 22-25, 2019.

Santiago, Chile

Welch V, Grimshaw J, Rada G, Smith M, Pardo Pardo J, Soares-Weiser K, Tugwell P

Workshop (abstract accepted):

When should systematic reviews be replicated, and when is it wasteful?

REWARD alliance

February, 2020