Evidence based medicine and healthcare

Introduction

Learning objectives:You will learn about evidence based medicine, systematic reviews, the Cochrane Collaboration and grey literature.

Evidence based medicine involves the consistent use of current best evidence, and such evidence is often assessed using systematic reviews. Both the evidence hierarchy for assessing quality of evidence and systematic reviews are explained in this section.

The Cochrane Collaboration prepares, maintains and disseminates systematic reviews of the effect of healthcare interventions. Such reviews must not only consider the published literature but also the grey literature. Both are explained in this section.

Read the resource text below.

Resource text

Evidence based medicine

Evidence based medicine has been defined as the process of systematically finding, appraising, and using contemporaneous research findings as the basis for clinical decisions [1].

The practice of evidence-based medicine means integrating individual expertise with the best available external clinical evidence from systematic research. Best available external clinical evidence means clinically relevant research, often from the basic sciences of medicine, but especially from patient centred clinical research into the accuracy and precision of diagnostic tests, the power of prognostic markers, and the efficacy and safety of therapeutic, rehabilitative, and preventative regimens [2].

One of the driving forces behind the development of evidence based medicine has been the recognition of the gap between research evidence and clinical practice [1] [3]. Research literature is constantly changing and the volume of health information has increased rapidly. The growth of health care information has been particularly rapid in diagnostic and therapeutic technologies.

The hierarchy of evidence

Research evidence in healthcare can be classified according to level of quality. This is known as the hierarchy of evidence. This is illustrated in figure 1.

Figure 1: Hierachy of Evidence.

RCT = Randomised Controlled Trial.

Diagram taken from Akobeng 2005 [4]

Systematic reviews of randomised controlled trials represent the highest quality of evidence from which to draw conclusions. Opinion is at the bottom of the evidence hierarchy; this is due to its subjective nature.

The practice of evidence based medicine.

The practice of evidence based medicine involves five steps [5]:

1. Formulating answerable clinical questions.
2. Systematic retrieval of best evidence available.
3. 'Critically appraising' the evidence (determining the validity and applicability).
4. Applying the evidence (directly in patient care, development of protocols, guidelines).

Systematic Reviews

The Cochrane Collaboration defines a systematic review as a review of a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise relevant research, and to collect and analyse data from studies that are included in the review [6].

Systematic reviews synthesize the results of multiple primary investigations by using strategies that limit bias and random error. These include a comprehensive search of all potentially relevant articles and the use of explicit, reproducible criteria in the selection of articles from review.

Strengths of a systematic review

  • A well conducted systematic review provides a summary of multiple studies that is easily accessible to clinicians, health care providers and policy makers.
  • By critically examining primary studies, systematic reviews can improve understanding of inconsistencies among diverse research evidence [7].
  • The explicit methods applied limit bias in identifying and rejecting studies [7].
  • They provide a more precise estimate of effect because of methods used [7], [8].
  • The results from different studies can be systematically compared to establish generalisability of findings and consistency of results [7], [8].
  • Reasons for any heterogeneity (difference) in results across studies can be identified and new hypotheses generated about specific subgroups [7].
  • They are useful for decision making.
  • They help to define limits of what is known and unknown and help formulate hypotheses for further investigation.

Limitations of systematic reviews

    Knowledge of an average treatment effect may not apply to an individual patient.

Meta-analysis

A meta-analysis is a statistical technique used to combine and summarise the results of several independent studies relating to a specific hypotheses. The aim is to integrate the findings, pool the data and identify the overall trend of results [9]. The accuracy of these overall results depends on the quality of the randomised controlled trials included in the meta-analysis.

Meta-analyses are commonly used to assess the clinical effectiveness of health interventions from randomised controlled trials and are viewed as providing a more effective method of estimating a treatment effect. In a meta-analysis the overall effect of an intervention is calculated using weighted averages of the results from multiple trials. Combining the results of trials can overcome the problem of reduced statistical power in trials with small sample sizes [9]. Meta-analyses are used increasingly to establish clinical policy.

Strengths of a meta-analysis

  • A well conducted meta-analysis can provide an objective evaluation of available evidence.
  • Well conducted meta-analyses allow a more objective appraisal of the evidence than traditional narrative reviews, provides a more precise estimate of a treatment effect, and may explain heterogeneity between the results of individual studies (7).

Limitations of a meta-analysis

  • Poorly conducted meta-analyses may be biased due to the exclusion of relevant studies or inclusion of inadequate studies.
  • May be subject to publication bias - studies with a negative effect may not get published and therefore be excluded, while studies that that report a large treatment effect may be more likely to be published.
  • Bias may be introduced if all relevant studies are not included.
  • Heterogeneity is a major problem in meta-analysis, as not all studies are comparable, but it may be decided to combine them anyway, reducing accuracy.

The Cochrane Collaboration

The Cochrane Collaboration is an international and independent non-profit organisation established in 1993 aimed at producing up-to-date, accurate information about the effects of healthcare available worldwide. The Cochrane Collaboration produces and disseminates systematic reviews of healthcare interventions and promotes the search for evidence in the form of clinical trials and other intervention studies [10].

The Cochrane Collaborations handbook for systematic reviews of interventions is its main working document. The handbook describes in detail the process of creating Cochrane systematic reviews. It is available online at: http://www.cochrane.org/resources/handbook/

The main output of the Collaboration is the Cochrane Database of Systematic Reviews, which is contained as part of the Cochrane Library.

The Cochrane Library

The Cochrane Library is a collection of evidence based healthcare databases.

Databases within the Cochrane Library

  • The Cochrane Database of Systematic Reviews (CDSR).
  • The Cochrane Database of Abstracts of Reviews of Effectiveness (DARE).
  • The Cochrane Central Register of Controlled Trials (CCTR). This contains references to thousands of controlled trials identified by the Cochrane Collaboration.
  • Cochrane database of methodology reviews (CDMR). This contains protocols and reviews of Cochrane methodological studies.

Grey Literature

Grey literature has been defined as: that which is produced on all levels of government, academics, business and industry in print and electronic formats, but which is not controlled by commercial publishers [11].

Grey literature has also been broadly defined to include everything except peer-reviewed books and journals accepted by Medline.

Types of grey literature

Grey literature comprises a wide range of material including, government publications, reports, statistical publications, newsletters, fact sheets, working papers, technical reports, conference proceedings, policy documents and protocols and bibliographies.

Producers of grey literature

A wide range of organizations produce a significant amount of grey literature related to public health, health policy and epidemiology. These include:

  • Government health agencies
  • Non-profit organisations
  • Universities
  • Research centres
  • International agencies such as the World Health Organization (WHO) and UNAIDS.
  • Health institutes
  • Professional organisations
  • Special interest groups.

Grey literature on the internet

The internet has increasingly become a major source of grey literature accessible to a wide international audience. Many databases increasingly cite grey literature in addition to published material.

Use of grey literature

Grey literature may contain useful scientific findings and so it should be considered in the production of systematic reviews [12]. Grey literature is included in Cochrane systematic reviews of health interventions.

References

1. Rosenberg W, Donald A. Evidence based medicine: an approach to clinical problem-solving. BMJ 1995; 310: 1122-1126.
2. Sackett, DL, Rosenberg W, Muir Gray JA et al. Evidence-Based Medicine: What it is and what it isn't. BMJ 1996; 312: 71-72.
3. Dawes M, Summerskill W, Glasziou P et al. Sicily statement on evidence based practice. BMC Medical Education 2005; 5: 1. doi:10.1186/1472-6920-5-1.
4. Akobeng AK. Principles of evidence based medicine. Archives of Disease in Childhood 2005; 90: 837-840.
5. Porzsolt F, Ohletz A, Thim A et al. Evidence-based decision making - the six step approach. Evidence Based Medicine 2003; 8: 165-166.
6. The Cochrane Collaboration.Cochrane handbook for systematic reviews of interventions [online] [Accessed 08/08/2008].
7. Cook DJ, Mulrow CD, Haynes RB. Systematic reviews: synthesis of best evidence for clinical decisions. Ann Intern Med. 1997; 126 (5): 389-91.
8. Greenhalgh, T. How to read a paper: Papers that summarise other papers (systematic reviews and meta-analyses). BMJ 1997; 315: 672-675.
9. Dickerson, K. Berlin, J.A. Meta-analysis: State of the science. Epidemiol Rev 1992; 14: 154-176.
10. The Cochrane Collaboration. The Cochrane Collaboration [online] [Accessed 08/08/2008].
11. Fourth International Conference on Grey Literature: New Frontiers in Grey Literature. GreyNet, Grey Literature Network Service, Washington DC, USA, 4-5 October 1999. [Accessed 08/08/2008].
12. Last, J.M. A dictionary of epidemiology. p78. 4th Edition. Oxford University Press 2001.

Related links

Evidence based medicine on the web

Evidence Based Medicine Journals

The Cochrane Collaboration

Grey Literature