Critical Appraisal

What is critical appraisal?

Critical appraisal is the process of carefully and

systematically examining research to judge its

trustworthiness, value and relevance in a particular

context.

Evidence-based nursing and health care practice

Requires the critical appraisal of evidence so that it can

be used to inform and promote clinically effective care

and decision-making.

Why is it important?

To make the best decisions, health care professionals need to be able to:

Decide whether studies have been undertaken in a way that makes their findings reliable.

Make sense of the results.

Know what these results mean in the context of the decision that needs to be made.

Be confident about applying research to practice.

.

Spotting bias in articles

Selection bias - how were participants selected? What sampling method was used e.g. random, convenient, purposive.

Attrition - did any participants drop out during the course of the study? This is particularly important in qualitative studies where numbers of participants tend to be lower.

Publication bias - studies with positive results are more likely to be published than those with negative or inconclusive findings. It is also easier for established researchers to be published - many good studies may never be published simply because the authors are unknown.

Critiquing quantitative research

Critiquing a quantitative research article can be daunting

but good maths skills are not essential. Look for things that

are not mentioned as these may be deliberately omitted to

disguise gaps or weakness in the research.

Example: If an article mentions relative risk (RR), but not

absolute risk (AR), the author may be trying to put a

positive spin on the results. RR is a proportional rather than

an actual figure and so will always appear to be higher.

Critiquing quantitative research

Likewise, if numbers need to treat (NNT) is not mentioned it

may indicate that the treatment was not effective (i.e. you

would need to treat too many patients before one was cured).

Small confidence intervals indicate the likelihood of the

same results recurring with a larger sample.

TIP! For help critiquing quantitative research read:

Harris, M. and Taylor, G. (2008) Medical statistics made

easy, 2nd edn. Bloxham: Scion Harris.

Critiquing qualitative researchSummary of the CASP tool for qualitative research:

Is there a clear statement of the aims of the research? Is the research design appropriate to address the aims of

the research? Was the sampling strategy appropriate e.g. size,

purposive. Was the data collected in an appropriate way? Was the relationship between researcher and participants

adequately considered? Was the data analysis sufficiently rigorous? Is there a clear statement of the findings? Is the research valuable?

Critiquing other materials

Use this checklist for critiquing general material e.g. web sites, policy, discussion or opinion papers etc.

Why am I reading this? What are the authors trying to do in writing this? What are the authors saying that is relevant to what I want

to find out? How convincing is the author’s viewpoint? Is there any

supporting evidence? Is it consistent with what I already know?

What use can I make of this?

Wallace, M. and Wray, A. (2006) Critical reading and writing for postgraduates. London: Sage.

CASP tools

CASP are the most commonly used checklists http://www.casp-uk.net

Quantitative research tools Randomised controlled trial checklist Systematic review checklist Cohort study checklist Case control study checklist Diagnostic test checklist

Qualitative research tools There is a single checklist for qualitative material.

Other critiquing tools

Other critiquing tools include:

Tools outlined in standard textbooks

Best Bets critical appraisal tools

CEMB critical appraisal sheets

SIGN Checklists

Tip! These are not meant to replace considered thought and judgement when reading a paper – they are simply there as guides and aide memoires.

Textbooks with useful tools

UCS textbooks with worked examples of appraisal tools:

Cluett, E.R. and Bluff, R. (2006) Principles and practice of research in midwifery. 2nd edn. Edinburgh: Churchill Livingstone.

Holland, K. and Rees, C. (2010) Nursing; evidence-based practice skills. Oxford: Oxford University Press. Chapter 7, Table 7.3 (qualitative) p.179.

Newell, R. and Burnard, P. (2010) Research for evidence-based practice in healthcare [E-book]. New edn. Chichester: Wiley-Blackwell.