It was recognized that a deeper understanding of bias was required in order for previously published systematic reviews and meta-analyses to be evaluated and evidence-based guidelines for the conduct of future reviews and meta-analyses be developed. Such guidelines would be important aids for the reviewer in deciding, at the outset of a review, how to frame the review question, how to search the literature and how to interpret the findings of each report identified in the search.
In 1998, an initial meeting was held in Baltimore, MD USA at the 6th Cochrane Colloquium. A further exploratory meeting was held in Rome at the 7th Cochrane Colloquium in 1999. At the 8th Cochrane Colloquium in Cape Town in 2000, a pre-registration meeting was held and the Reporting Bias Methods Group (RBMG) was formally represented at other relevant sessions, including "meet the entities" and the annual general meeting. As a registered methods group, the RBMG met at the 9th Cochrane Colloquium in Lyon in 2001 and again at the systematic review symposium in July 2000, Oxford, U.K. In September 2003, the RBMG officially relocated from Bristol, UK to Ottawa, Canada. Subsequently, in 2005, the RBMG transformed into the Bias Methods Group (BMG) with the movement to incorporate all types of bias. In January 2016, BMG relocated to Odense, Denmark where it is currently located.
Cochrane Bias Methods Group focuses on the "epidemiology of results and outcomes", that is, the characteristics of primary study results and their distribution within and outside the literature. There is a risk of bias being introduced into a systematic review and/or meta-analysis if the characteristics and distribution of results (concerning a particular treatment) are associated with the nature and direction of the results (degree of harm or benefit, statistical significance, precision) themselves, and if the reviewer does not take account of these associations. There is now evidence at a general level that such associations do exist.
BMG investigates a range of different forms of bias, such as "publication bias", "language bias", and "outcome reporting bias", and examines whether, and in which circumstances, these biases may have a substantial impact on systematic reviews and meta-analyses. BMG will do this with regard to the broad categories of quantitative and non-quantitative syntheses, and with regard to experimental and observational research. The focus will be on individual topic areas and Cochrane Review Group interests. BMG conducts methodological systematic reviews, provides advice to Cochrane entities and individuals, hold workshops and publicize its findings through appropriate means.
BMG is funded by Odense University Hospital (OUH).
Providing Policy Advice
BMG has recently contributed two chapters for the Cochrane Handbook for Systematic Reviews of Interventions update. The first chapter (Chapter 8, edited by Julian Higgins, Douglas Altman and Jonathan Sterne) focuses on assessing risk of bias in studies included in systematic reviews, and introduces the Risk of Bias tool for this purpose. The second chapter (Chapter 10, edited by Jonathan Sterne, Matthias Egger and David Moher) focuses on relevant reporting biases, highlighting the evidence supporting the presence of each bias and ways to detect potential biases.
A major initiative that now falls within the remit of BMG is the continued development of new guidance for assessing risk of bias ('quality assessment') of included studies in Cochrane reviews. Dissemination of the Risk of Bias tool began in late 2007 and the tool has recently, after an extensive evaluation funded by the Methods Innovations Fund, been updated and incorporated into the new versions of the Handbook and Review Manager.
BMG conducts in-house methodology research directly and indirectly pertaining to bias in systematic reviews. A "methodology review" is a systematic review of methodological studies. If you wish to register a methodology review on a BMG-related topic, please feel free to contact us for support.
Methodology Review Group
Mike J Clarke
All-Ireland Hub for Trials Methodology Research
Centre for Public Health, Queens University Belfast
Institute of Clinical Sciences, Block B
Royal Victoria Hospital, Grosvenor Road
BMG participates actively in training programs for Cochrane authors, editors and other members of the systematic review community. Since its establishment in 2001, BMG has hosted over 20 workshops, and presented unquantifiable contributions at meetings and events as well as online webinars, developing standardised training materials and leading train-the-trainer sessions.
BMG endeavours to provide training to both Cochrane and non-Cochrane systematic reviewers via formal and informal training opportunities. Training topics reflect the group's remit and include foundational topics, such as investigating and dealing with bias in systematic reviews, and more advanced topics such as identifying and assessing the impact of selective outcome reporting and identifying risk of bias in studies included in a systematic review.
Liaison with Cochrane Groups, Networks & Centers
Bias is now the focus of any existing Methods Group, and is a topic of interest to many Cochrane reviewers.
BMG includes a large proportion of Cochrane Collaborators with an active interest in this area. The BMG also views links with the Cochrane Consumer Network as important. Members of the Network should be encouraged to join BMG or be included on its mailing list. Biases such as selective outcome reporting bias, developed country biases and reviewer bias, (as described above), are likely to be of particular interest to consumers, as the nature of these biases is determined by the behaviour and mindset of health professionals, researchers and editors. BMG, in turn, will benefit greatly from the perspectives of consumers on these biases.
Mechanisms of increasing BMG membership and expanding group activities, primarily training and methodology reviews, are the two overarching priorities for the group.
In 2009 BMG launched a membership survey, contacting all listed members and asking them to list topics which they felt should be of priority to the group. As a result, in addition to the BMG's original remit, following forms of bias and topics are considered to be of focus:
- Publication bias
- Language bias
- Funding bias
- Outcome variable selection bias
- Database bias (a type of retrieval bias)
- Coding bias (a type of retrieval bias)
- Citation bias (a type of retrieval bias)
- Regional bias
- Developed Country biases
- The relationship between mandatory registration of clinical trials and quality of reporting
- Bias due to withdrawal in studies
- Meta-analysis of individual-level data and collaborative analyses
- Bias in prognostic studies
- Risk of Bias and the Cochrane Risk of Bias Tool, evaluation, update and influence
- Reporting guidelines for Randomised Controlled Trials
Our Vision: The vision of Cochrane Bias Methods Group is to ensure that bias in all systematic reviews is identified, assessed and transparently reported to enable the production of high quality, reliable research allowing health care decisions to be made based on the best available evidence.
Our Mission: The mission of BMG is to provide expertise and evidence upon which bias can be reviewed, reduced and transparently considered by all reviewers, ultimately improving the reliability of systematic reviews.
BMG considers the internal threat of bias with regards to both quantitative and non-quantitative syntheses, and with regard to experimental and observational research. BMG also conducts and publishes original methodological research and conducts specific systematic reviews on various forms of bias. Alongside published works which contribute to improve Evidence Informed Decision-Making, the mission of BMG includes Integrated Knowledge Translation via dissemination of methodological research, methods consultation and perhaps most significantly provides training and education vis-à-vis workshops to key stakeholders.
BMG's focus on the development and dissemination of reporting guidelines will help ensure the ongoing quality of Cochrane reviews. In particular, the revision and update of the QUality Of Reporting Of Meta-analyses (QUOROM) Statement, recently renamed Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), was published in July 2009 (Moher and colleagues). The aim of PRISMA is to help authors improve the reporting of systematic reviews and meta-analysis. In collaboration with the EQUATOR network (Enhancing the QUAlity And Transparency Of health Research) BMG will continue to promote PRISMA and other reporting guidelines as a means to facilitate quality Cochrane reviews.
Adequate reporting of the conduct of Randomised Controlled Trials and other epidemiological studies is fundamental to the comprehensive and transparent evaluation of potential biases in systematic reviews. As such, BMG executive is heavily involved with the EQUATOR Network and the development of reporting guidelines which provide support for authors when writing up all aspects of primary studies.
For more information, please visit the EQUATOR Network website.