Critical Appraisal Frameworks

Critical Appraisal Frameworks

Summarise Article in a few lines - ie abstract

Comment on: author, journal, recency
State briefly study design and what done with main conclusion.
e.g. 'an RCT assessing treatment A against B which showed A to be better than B..'

Then consider each section of the paper in turn:

1.Introduction

  • Is objective clear?
  • Is study important?
  • Is study relevant?

2. Method : Assess:

- setting

- subjects

  • choice of source population
  • method of sampling
  • controls
  • cases - who, how, numbers, selection, exclusion..

- design

  • type, appropriate
  • sample size calculation / statistical power

- intervention/ data collection - quality

  • quality control
  • validity
  • reproducibility

- data analysis

  • how is this done and is it appropriate
  • assess statistics
  • any gaps in the data

3. Results

  • Are they clearly presented? (text and tables)
  • is the response rate good?
  • Is there any missing data? - drop outs/non responders/deaths/missing data
  • Are the results interpreted correctly?
  • State statistically significant results and explain Confidence Intervals/sensitivity/specificity/PPV/clinical significance
  • Is the sample size good / sufficient?

4. Internal validity of study

Could results be explained by:
a) Chance

  • multiple testing
  • check CI an p values
  • sample size/power

b) Bias

  • selection bias
  • information bias
    • random misclassification
    • recall
    • observer bias

c) Confounding

  • age, sex, social class, smoking
  • design: matching , stratification
  • analysis: standardisation, stratify

5. External validity of study

  • Is it generalisable to population / to UK
  • Are results relevant

6. Discussion

  • Are the objectives covered?
  • Are limitations of study discussed?
  • Are conclusions justified from results? NB: many studies don't
  • Are the results relevant/ important?

7. Importance of results to public health practice

8. Any other strengths/ weaknesses

Extras: How might it have been better. E.g. RCT might have been better than CC

Critical Appraisal of particular study types.- Key Points in addition to generic

Surveys:

  • who was studied?
  • How was sample obtained?
  • What was response rate?

Cohort studies:

  • who exactly has been studied?
  • Was a control group used? (should one have been used?)
  • How adequate was follow up?

Case control studies:

  • How were cases obtained?
  • Is control group appropriate?
  • Was data collected in the same way for cases and controls?

Clinical trials:

  • Were treatments randomly allocated?
  • Were all patients accounted for?
  • Were outcomes assessed blind?

Qualitative studies:

  • Were the relationships between the researcher and those studied adaquately considered?
  • Was the analysis conducted rigorously to remove researcher bias?
  • To what population is this study generalisable

Review Papers:

  • How were papers identified?
  • How was quality of papers assessed?
  • How were results summarised?