The hierarchy of research evidence - from well conducted meta-analysis down to small case series

 

PLEASE NOTE:

We are currently in the process of updating this chapter and we appreciate your patience whilst this is being completed.

 

Evidence-based medicine has been described as ‘the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients.’1 This involves evaluating the quality of the best available clinical research, by critically assessing techniques reported by researchers in their publications, and integrating this with clinical expertise. Although it has provoked controversy, the hierarchy of evidence lies at the heart of the appraisal process.
 

Ranking of trial designs

The hierarchy indicates the relative weight that can be attributed to a particular study design. Generally, the higher up a methodology is ranked, the more robust it is assumed to be. At the top end lies the meta-analysis – synthesising the results of a number of similar trials to produce a result of higher statistical power. At the other end of the spectrum lie individual case reports, thought to provide the weakest level of evidence.

Several possible methods for ranking study designs have been proposed, but one of the most widely accepted is listed below.2 Information about the individual study designs can be found elsewhere in Section 1A.

  1. Systematic reviews and meta-analyses
  2. Randomised controlled trials
  3. Cohort studies
  4. Case-control studies
  5. Cross-sectional surveys
  6. Case series and case reports
     

Concerns and caveats

The hierarchy is widely accepted in the medical literature, but concerns have been raised about the ranking of evidence, versus that which is most relevant to practice. Particular concerns are highlighted below.

  • Techniques lower down the ranking are not always superfluous. For example, the link between smoking and lung cancer was initially discovered via case-control studies carried out in the 1950s3. Although randomised control trials (RCTs) are considered more robust, it would in many cases be unethical to perform an RCT. For example, if studying a risk factor exposure, you would need a cohort exposed to the risk factor by chance or personal choice.
  • The hierarchy is also not absolute. A well-conducted observational study may provide more compelling evidence about a treatment than a poorly conducted RCT.
  • The hierarchy focuses largely on quantitative methodologies. However, it is again important to choose the most appropriate study design to answer the question. For example, it is often not possible to establish why individuals choose to pursue a course of action without using a qualitative technique, such as interviewing.

Alternatives to the traditional hierarchy of evidence have been suggested. For example, the GRADE system (Grades of Recommendation, Assessment, Development and Evaluation) classifies the quality of evidence not only based on the study design, but also the potential limitations and, conversely, the positive effects found. For example, an observational study would start off as being defined as low-quality evidence. However, they can be downgraded to “very low” quality if there are clear limitations in the study design, or can be upgraded to “moderate” or “high” quality if they show a large magnitude of effect or a dose-response gradient.

The GRADE system is summarised in the following table (reproduced from4):

 

Insert diagram re the GRADE system here:

 

 

 

 

The Oxford Centre for Evidence-Based Medicine have also developed individual levels of evidence depending on the type of clinical question which needs to be answered. For example, to answer questions on how common a problem is, they define the best level of evidence to be a local and current random sample survey, with a systematic review being the second best level of evidence. The complete table of clinical question types considered, and the levels of evidence for each, can be found here.5

 

References

  1. Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't. BMJ 1996: 312:7023
  2. Guyatt GH, Sackett DL, Sinclair JC, Hayward R, Cook DJ, Cook RJ. Users' guides to the medical literature. IX. A method for grading health care recommendations. JAMA 1995; 274:1800-4.
  3. Doll R and Hill AB. Smoking and carcinoma of the lung. BMJ 1950;2:739.
  4. Takada T, Strasberg S, Solomkin J et al. Updated Tokyo Guidelines for the management of acute cholangitis and cholecystitis. Journal of Hepato-Biliary-Pancreatic Sciences 2013;20:1-7.
  5. Oxford Centre for Evidence-Based Medicine. Levels of evidence, 2011
    http://www.cebm.net/wp-content/uploads/2014/06/CEBM-Levels-of-Evidence-2.1.pdf  - Accessed 8/04/17 

 

Further reading

  • Greenhalgh T. How to Read a Paper: The Basics of Evidence Based Medicine. London: BMJ, 2001
  • Guyatt G, Rennie D et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. McGraw-Hill Medical, 2008.

 

 

© Helen Barratt 2009, Saran Shantikumar 2018