Audit and SEA Made EasyAudit and SEA Made Easy

GPST Teaching October 2012

Dr Kate LewinGPST Course OrganiserNHS Education for ScotlandNHS Education for Scotland

Definitely not rocket science

Aims of workshop

Refresh your knowledge of, or introduce you to, audit and SEA

Define criteria and standards

Increase confidence in performing Audit/SEA in your own practice

Appreciate the use of audit/SEA in quality assurance, also for revalidation, QOF etc.

Signpost to resources for further guidance

Discuss YOUR audits or audit plans in groups

What is audit?

What is Audit?What is Audit?

Audit is about taking note of what we do, learning from it and changing if necessary

Audit is the improvement in the quality of care through standard-setting, peer review, implementation of change and re-evaluation

Quite simply, audit is a tool that enables you to monitor and then improve the quality of care you provide to your patients.

Systematic critical analysis of the quality of health care

A range of definitions exist:

The Audit CycleThe Audit Cycle

Define Criteria & StandardsDefine Criteria & Standardsoror

““What do you think you should be doing”What do you think you should be doing”

Collect the DataCollect the Datai.e. i.e.

The Information onThe Information onwhat you are doingwhat you are doing

Assess PerformanceAssess Performanceagainst criteria & standardsagainst criteria & standards

How are we doing in relation to How are we doing in relation to what we should be doing ? what we should be doing ?

Identify the ChangesIdentify the Changes Required & Required &

Implement Them !!Implement Them !!

Define Criteria & StandardsDefine Criteria & Standardsoror

What do you think you should be doing What do you think you should be doing

Choose the TopicChoose the Topic

The Audit CycleThe Audit Cycle

Choosing The TopicChoosing The Topic

Eg. Clopidogrel Prescribing

The Audit CycleThe Audit Cycle

Define Criteria & StandardsDefine Criteria & Standards

Criteria - what you want to measure (yard-stick)

eg. Patients should have clopidogrel prescribed only in accordance with NHSGGC guidelines (ie. aspirin contraindicated or postACS/TIA/Stroke/stent insertion)

Standard - how well you should be doing

80% of patients should have clopidogrel prescribed in accordance with NHSGGC guidelines

The Audit CycleThe Audit Cycle

Collect the DataCollect the Data

1. Identify patients on clopidogrel

2. Pharmacist review of notes – identify when started, by whom, indication, whether ever on aspirin +/- PPI

3. Determine whether in accordance with guidelines

The Audit CycleThe Audit Cycle

Assess PerformanceAssess Performance

Compare our results with the

standard previously set

e.g. 21 of 116 patients on clopidogrel (18%) were prescribed according to NHSGGC guidelines – far below standard of 80%

The Audit CycleThe Audit Cycle

Agree & Implement Agree & Implement

Changes RequiredChanges Required

1. Explore reasons for inappropriate use

2. Feedback to colleagues, discuss changes and implement them

Eg. Letters to patients/cardiologists, face-to-face medication review, raising awareness of prescribers

The Audit CycleThe Audit Cycle

Repeat the Audit!!!

Data Collection 2Data Collection 2

Repeat data collection once changes have had

a chance to take hold

The Audit CycleThe Audit Cycle

Re-assess PerformanceRe-assess Performance

Compare the results with the

standards previously set and results

of data collection 1

Has the standard now been met?

e.g. Now find that 48 of 90 (53%) of patients on clopidogrel are prescribed within

guidelines

ie. Significant improvement but still below standard

The Audit CycleThe Audit Cycle

Identify Further Changes RequiredIdentify Further Changes Required

Long term issues:

Determine if further change is required to sustain performance, and decide when

next to audit this topic (annually, every 2 years etc.)

Criteria – what you want to Criteria – what you want to measuremeasure

Simple logical statements, used to describe a measurable item of quality health care

ie. What you want to measure

e.g. Patients with asthma should have their inhaler technique assessed at least once every 12 months.

Describes the ideal level of care to be achieved for each criterion

ie. How well you should be doing

e.g. 80% of patients with asthma should have their inhaler technique assessed at least once every 12 months.

Standard – How well you should Standard – How well you should be doing!be doing!

Criteria Standard

Children under 2 years old shouldbe immunized against tetanus

and polio

90% of the registered 2 year oldsimmunized against tetanus and

polio

The notes of those patientssensitive to penicillin should

be clearly marked.

The notes of all (100%) patientssensitive to penicillin clearly marked.

Patients should wait no longerthan 30 minutes in the surgery

before consultation.

75% of patients should wait nolonger than 30 minutes in thesurgery before consultation.

Examples of Criteria & StandardsExamples of Criteria & Standards

Arriving at StandardsArriving at Standards

Don’t get overly concerned - standard setting is flexible, can be revised upwards or down

Those involved decide on the level of care they find desirable - it is a professional issue/decision.

Guidance can be derived from the literature/textbooks, but ultimately you decide with your practice.

Can be based on your own work and observations, varies between practices

Report format for audit

Report Section

Guidance

Reason for Audit

Explain why the topic was chosenPotential benefits to patients/staff/practice

Criteria Set Try to limit to 1-3, relevant to topicShort simple logical statementsJustify with reference to current evidence

Standards Set

Agree a measureable standard for each criterionSet a realistic timescale

Preparation and Planning

Who you discussed the audit with, and who assisted youHow you collected and analysed the data

Report format cont.

Report Section

Guidance

Data Collection 1

Present using simple descriptive statistics/tables/graphsDo not present irrelevant dataComment on comparison with standard set

Description of Change

Describe changes agreed and implementedAttach example as evidence if possible

Data Collection 2

Compare with standard and with data collection 1If standard not reached speculate as to why

Conclusions What have you learnedHow do you intend to take forward in future?peer review

What topics to audit?

Areas relevant to your practice

Linked to educational event attended

Look at NES Audit Ideas Booklet – online link, lots of ideas, topic areas and suggested criteria!

Group Exercise 1Criteria & Standards – understanding the difference

Group Exercise 2Implementing Change – sometimes the hardest part of doing an audit!!

Time to discuss own audits?

Significant Event Analysis

What is a significant event?

“ Any event thought by anyone in the team to be significant in the care of patients or the conduct of the practice “

(Pringle et al, 1995)

SEA

What happened?

Why did it happen?

What have you learnt?

What have you changed?

What happened?

Record all of the facts relating to the identified significant event (including any relevant dates, times and people or organisations involved)

Data source: those directly and indirectly involved

Establish a clear and full picture of what happened

Impact or potential impact

Why did it happen?

Establish all of the main and underlying reasons why the event actually occurred.

Eg. A written telephone message about an important meeting was not passed to the practice manager because it had been lost.

But…. why was it lost?

Because it was written on a post-it and left on top of a report, which was subsequently filed away by an unsuspecting member of staff.

ie. Internal communication practices not up to scratch!

What have you learned?

Highlight any learning issues you and/or the practice experience.

For example it may be related to a training need or a lack of knowledge concerned with therapeutics, disease management or administrative procedures.

It could also reflect a learning experience (good or not so good) in dealing with patients, colleagues, or other organisations

Ensure that insight into the event has been established by the practice team or the individuals concerned

What have you changed?

Often a change in some aspect of care is required to improve the provision of care and/or minimise the risk that a similar event will occur.

If so, a description of the change actually implemented should be given rather than a “wish list” of thoughts

What have you changed? (cont.)

Sometimes it is not possible to implement change, either because the likelihood of the event happening again is so rare or because change is outwith the control of the individual/practice.

If this is the case, then reasons should be clearly documented.

Regardless of the type of significant event, change should at least be considered, then either implemented or justifiably ruled out

Important points

Doesn’t have to be an bad event Could explore example of excellent practice

Sharing Team activity

Blame-free Constructive learning not finger-pointing

Look beyond the superficial For underlying/systematic causes

Useful links for further information

Guidance on Audit, RCGP Revalidation Toolkit (p28) http://www.rcgp.org.uk/PDF/

Scot_Complete_Revalidation_Toolkit_(Read_Only).pdf

Ideas for Audit, NES www.clinicalgovernance.scot.nhs.uk/.../ideasforauditandSEA.rtf

SEA – NPSA Guide 2008 http://www.npsa.nhs.uk/nrls/improvingpatientsafety/

primarycare/significant-event-audit/

Group Exercise 3Analysis of SEAs

Time to discuss own SEA's?