Different Levels of Evidence (Critical Reading)

Practising evidence based medicine encourages clinicians to integrate valid and useful evidence with clinical expertise and each patient's unique features, enabling clinicians to apply evidence to the treatment of patients.¹ There are five main steps to practising evidence based medicine:¹

• Identify knowledge gaps and formulate a clear clinical question
• Search the literature to identify relevant articles
• Critically appraise the articles for quality and the usefulness of results; always question whether the available evidence is valid, important and applicable to the individual patient
• Implement clinically useful findings into practice
• Evaluate performance using audit

For issues of therapy or treatment, the highest possible level of evidence is a systematic review or meta-analysis of randomised controlled trials (RCTs) or an individual RCT. For issues of prognosis, the highest possible level of evidence is a cohort study.² Expert opinion must not to be confused with personal experience that is sometimes called eminence-based medicine. Expert opinion is the lowest level of evidence and is quite correctly below experimental evidence but in the absence of experimental evidence may be the best guide available.

Healthcare professionals must always apply their general medical knowledge and clinical judgement not only in assessing the importance of recommendations but also in applying the recommendations which may not be appropriate in all circumstances. The following questions should be asked when deciding on the applicability of evidence to patients:³

• Is my patient so different from those in the study that results cannot be applied?
• Is the treatment feasible in my setting?
• What are my patient's likely benefits and harms from the therapy?
• How will my patient's values influence the decision?

• When looking for appropriate evidence:
  • Search for available guidelines, e.g. NICE, National Library for Health, professional bodies (e.g. relevant specialist site such as the Royal College of Obstetricians and Gynaecologists).
  • If no guidelines are available, search for systematic reviews, e.g. Cochrane database.
  • If no systematic reviews are available, look for primary research, e.g. PubMed.
  • If no research is available, consider general internet search, e.g. Google, or discuss with a local specialist (at this level beware poor quality information from the internet or individual personal bias from even the most respected specialist).
• The National Library for Health provides access to a range of medical search sites, including PubMed, Medline, EMBASE, Bandolier, York Centre for Review and Dissemination and the Cochrane database.
• National guidelines and guidance sites include the National Institute of Clinical Excellence (NICE) and the Scottish Intercollegiate Guidelines Network (SIGN). Guidance on many topics is also available at the Clinical Knowledge Summaries (formerly PRODIGY) website (see below for links).

The hierarchy of studies for obtaining evidence is:

• Systematic reviews of randomised controlled trials
• Randomised controlled trials
• Controlled observational studies - cohort and case control studies
• Uncontrolled observational studies - case reports

• Type of intervention.
• Size of the sample.
• Type of person included and excluded.
• Type of control group used for comparison (ideally placebo).
• How reliable is the methodology? Bad methodology is rarely obvious from reading the paper or the editor would not have published it. It usually comes to light some time later.
• Outcome; how convincing is the result? Are the statistics (e.g. P value, confidence limits) impressive? What is the rate of loss of follow-up during the study? Association and causation are not the same; are there possible alternative explanations for the results?
• Type of outcome; the results of a trial may be relatively simple to express in terms of numbers dying or surviving or may be much harder to quantify. There is an index called QALY (quality adjusted life years) that may be used for such parameters as pain, incontinence and disability.⁵ Other validated tools include the Hospital Anxiety and Depression Scale and the Geriatric Depression Scale.
• Type of study or analysis:
  • Randomised controlled trials (RCTs):
    • Randomised controlled trials, especially those with double blind placebo controls, are regarded as the gold standard of clinical research.
    • These studies work very well for certain interventions, e.g. drug trials, but it is much more difficult for other interventions, such as using sham acupuncture or sham manipulation as the control.
  • Longitudinal or cohort studies:
    • A group of people are followed over many years to ascertain how variables such as smoking habits, exercise, occupation and geography may affect outcome.
    • Prospective studies are more highly rated than retrospective ones although the former obviously take many years to perform. Retrospective studies are more likely to produce bias.
  • Meta-analysis:
    • The more data is pooled the more valid the results but possibly less relevant it becomes to individual patients.⁶ Meta-analysis can therefore be a useful tool but it has some important limitations.
    • A meta-analysis takes perhaps 10 trials of 100 patients and to combine the results as if it were a trial of 1,000 patients.
    • Although this technique rates highly the methodology may not be identical in all studies and further errors may be caused by a bias to certain publications. A good meta-analysis should contain funnel plotting with cut and fill to assess the completeness of a publication.⁷
    • A large, well conducted trial is therefore far more valuable than a meta-analysis.
• Is there a conflict of interest? Just because a pharmaceutical company finances a study of its products does not mean that the paper is corrupt or invalid but the reader may be a little more wary.

• A variety of grading systems for evidence and recommendations are currently in use. The system used is usually defined at the beginning of any guidelines publication.
• The hierarchy of evidence and the recommendation grading's relate to the strength of the literature and not necessarily to clinical importance.

Grading of evidence

• Ia: systematic review or meta-analysis of randomised controlled trials
• Ib: at least one randomised controlled trial
• IIa: at least one well-designed controlled study without randomisation
• IIb: at least one well-designed quasi-experimental study, such as a cohort study
• III: well-designed non-experimental descriptive studies, such as comparative studies, correlation studies, case–control studies and case series
• IV: expert committee reports, opinions and/or clinical experience of respected authorities

Grading of recommendations

• A: based on hierarchy I evidence
• B: based on hierarchy II evidence or extrapolated from hierarchy I evidence
• C: based on hierarchy III evidence or extrapolated from hierarchy I or II evidence
• D: directly based on hierarchy IV evidence or extrapolated from hierarchy I, II or III evidence

A simpler system of A,B or C is recommended by the US Government Agency for Health Care Policy and Research (AHCPR):

• A: requires at least one randomised controlled trial as part of the body of evidence.
• B: requires availability of well-conducted clinical studies but no randomised controlled trials in the body of evidence.
• C: requires evidence from expert committee reports or opinions and/ or clinical experience of respected authorities. Indicates absence of directly applicable studies of good quality.

Guideline recommendation and evidence grading (GREG)

In an attempt to improve the way recommendations and evidence statements are graded, a new grading system has been used (Guideline Recommendation and Evidence Grading - GREG):

• Evidence grade:
  • I (High): the described effect is plausible, precisely quantified and not vulnerable to bias
  • II (Intermediate): the described effect is plausible but is not quantified precisely or may be vulnerable to bias
  • III (Low): concerns about plausibility or vulnerability to bias severely limit the value of the effect being described and quantified
• Recommendation grade:
  • A (Recommendation): there is robust evidence to recommend a pattern of care
  • B (Provisional recommendation): on balance of evidence, a pattern of care is recommended with caution
  • C (Consensus opinion): evidence being inadequate, a pattern of care is recommended by consensus