Lecture 1 – Introduction to the process of

contextualisation of international guidelines

Karen Grimmer, PhD

Karen.Grimmer@unisa.edu.au

DirectorInternational Centre for Allied Health Evidence

University of South Australia

Background

• The Philippines is a third-world nation consisting of 7000 islands and 97,484,000 people (end 2013)

• Health expenditure in 2013 was US$119/year/person• Compared with Aus$6,140/year/person

• Filipinos have variable access to health care, from well-resourced metropolitan tertiary hospitals to remote community health centres staffed by a nurse• Private insurance is rare, and there are significant patient-out-of-pocket

expenses to access care which must be paid at time of consultation• There is no leeway personally, institutionally or nationally for misuse, overuse

or underuse of health expenditure• Unless care is informed by best evidence, it is likely to be ineffective and

wasteful

Impetus for improvement

• The Philippine government launched a Continuing Quality Improvement (CQI) Program in 2009, with limited funding directed to the development of de novo clinical practice guidelines, but not to dissemination or implementation– Hypertension and stroke are 3rd and 4th ranked causes

of death– Spinal pain is a high-ranked cause of physical

disability

• Both diseases impact on national productivity

Workforce

• Stroke and low back pain rehabilitation is generally managed by physiatrists– There are fewer than 200 rehabilitation physicians

(physiatrists) nationwide in 2010• Variable training about, and access to, current best

evidence

– Allied health and nursing providers treat by referrals from physiatrists which dictate prescriptions (what treatment should be provided)

• Even more variable training about, and access to, current best evidence

Barriers to best care

• Variable and generally inadequate skills and knowledge of Filipino health workers

• Hierarchy within healthcare services which makes change difficult

• Prohibitive costs for patients• Limited availability of specialist services• Variable equity of access nationwide• Physical barriers to care (eg travel)

Phil Dept Health 2014

PARM Vision

• Provide wide access to current best evidence CPGs for physiatrists, GPs, other specialists, allied health, nurses– Should de novo guidelines be constructed to

guide best practice rehabilitation of Filipino stroke and low back pain patients?

– Could guidelines from other countries be accessed, and directly implemented?

Challenges

• Constructing de novo evidence-based guidelines is not simple, quick or inexpensive

• There was no standard international approach in 2010 to guideline construction– The comprehensive McMaster guidance was

only released in 2013-14

• There was very limited financial support• The PARM Group had good will, but limited

training and expertise re guidelines

• High quality clinical guidelines support evidence-based care were mostly from Western countries

• Irrespective of the availability of CPGs, the recommendations had to be acceptable in the Philippines, context-specific and able to be implemented in a range of clinical settings by a range of health practitioners

Using other guidelines

Implementation barriers

Factor Potential barrier(s)Patient Patients expectations

EBP process Identification and implementing EBP is a difficult process

Team Issues Multidisciplinary teams, uniformity of approach

Care process Lack of uniformity, range of service delivery models

Management Support

Changes in leadership

Time/facilities/cost Time pressures, cost effectiveness, structural

limitations

Health System All stakeholders having similar expectations

• Could we be certain that by developing de novo guidelines specifically targeted to Filipino contexts, they would be adopted in practice?

• Were the issues of implementation as daunting as the issues of guideline production?

Framework

Research evidence quality dimensions

1. Hierarchy level • study design

2. Study Quality• how good is the study?

3. Statistical precision of results– statistical significance (p value, confidence limits)

4. Effect size • how clinically important are the findings?

5. Relevance • usefulness of results in clinical practice

NH&MRC (1998)

NH&MRC body of evidence assessment

matrixComponent

A B C D

Excellent Good Satisfactory Poor

Evidence baseseveral level I or II studies with low risk of bias

one or two level II studies with low risk of bias or a SR/multiple level III studies with low risk of bias

level III studies with low risk of bias, or level I or II studies with moderate risk of bias

level IV studies, or level I to III studies with high risk of bias

Consistency all studies consistentmost studies consistent and inconsistency may be explained

some inconsistency reflecting genuine uncertainty around clinical question

evidence is inconsistent

Clinical impact very large substantial moderate slight or restricted

Generalisability

population/s studied in body of evidence are the same as the target population for the guideline

population/s studied in the body of evidence are similar to the target population for the guideline

population/s studied in body of evidence different to target population for guideline but it is clinically sensible to apply this evidence to target population*

population/s studied in body of evidence different to target population and hard to judge whether it is sensible to generalise to target population

Applicabilitydirectly applicable to Australian healthcare context

applicable to Australian healthcare context with few caveats

probably applicable to Australian healthcare context with some caveats

not applicable to Australian healthcare context

Hillier et al 2011

Philosophical debate

• Can evidence from well-developed guidelines from other countries be readily taken up for Filipinos with stroke or spinal pain? – Is it the lack of a Filipino evidence base which

is the problem? OR Does it really matter? – Is it the issue of getting evidence (any

evidence) into local practices?

Local evidence-uptake questions

• Is a Filipino patient with stroke the same as a Western world patient with stroke? – Is the pathology and manifestation the same?

• Are Western best-evidence statements acceptable in a developing country? – Do Filipino physiatrists have the same skills as others

in the Western world? – Is the Filipino rehabilitation workforce the same as

found in other Western countries? – Are resources the same?– Is there a local will to improve quality care?

De novo or ‘borrow’ and contextualise?

• There were many current, good quality CPGs internationally for stroke, and low back pain– Was there a need for another CPG specific to Filipino

contexts?

• Could existing CPGs be borrowed, and contextualised, to meet Filipino needs?– Separation of the evidence base from generalisability

and applicability

• What was the best use of PARM resources and energies?

Patient journey

• Our decision hinged on a ‘typical’ patient journey, and the decision-making points along the way– Preliminary scoping of available Western

guidelines identified that no one guideline provided recommendations for each decision-making point

– More than one guideline would be needed to populate the patient journey with best evidence recommendations

What were we doing?

• Adapt (Encarta English dictionary) to change something to suit different conditions or a different purpose, or be changed in this way

• Contextualise (Encarta English dictionary) to place a word, phrase, or idea within a suitable context

• ADAPTE vs our process

Contextualisation• We did not adapt any recommendation

– We developed a standard process of:• Distilling the intent of multiple recommendations • Synthesising the underlying evidence and the way it was

reported

• We ‘endorsed’ existing recommendations at each decision-making point

• We wrote context points which addressed generalisability and applicability (basic and enhanced workforce, training, resources) and appropriate measures of outcome

NH&MRC body of evidence assessment

matrixComponent

A B C D

Excellent Good Satisfactory Poor

Evidence baseseveral level I or II studies with low risk of bias

one or two level II studies with low risk of bias or a SR/multiple level III studies with low risk of bias

level III studies with low risk of bias, or level I or II studies with moderate risk of bias

level IV studies, or level I to III studies with high risk of bias

Consistency all studies consistentmost studies consistent and inconsistency may be explained

some inconsistency reflecting genuine uncertainty around clinical question

evidence is inconsistent

Clinical impact very large substantial moderate slight or restricted

Generalisability

population/s studied in body of evidence are the same as the target population for the guideline

population/s studied in the body of evidence are similar to the target population for the guideline

population/s studied in body of evidence different to target population for guideline but it is clinically sensible to apply this evidence to target population*

population/s studied in body of evidence different to target population and hard to judge whether it is sensible to generalise to target population

Applicabilitydirectly applicable to Filipino healthcare context

applicable to Filipino healthcare context with few caveats

probably applicable to Filipino healthcare context with some caveats

not applicable to Filipino healthcare context

Our barriers

• Limited funding• Lack of a specific mechanism to map

multiple guideline recommendations to single decision-making points in the patient journey– Different wording of recommendations,

different mechanisms for reporting the evidence base, different references OR same references interpreted differently

• No culture of evidence-based practice or quality improvement

• The need for widespread training in EBP and guideline methodologies in PARM

• No information on current national practices

Our enablers

• Dr Gonzalez-Suarez & Dr Dizon• A longstanding international collaboration

(Australia and Philippines) and access to experienced methodologists

• Enormous goodwill by PARM members– Generosity of time commitment– Willingness to learn – Commitment to work for the greater good– Enjoyment of the process (even making mistakes)

Our way forward

• PARM decided to:– Use the existing evidence base for its

contextualised guidelines– Invest effort in understanding how to get the

evidence into practice across the Philippines– Value the spin-offs

• Training opportunities • Engage as many PARM members as possible to

ensure later commitment to evidence uptake• Focus on contextual barriers

• We decided to share our work and experience at the G-I-N 2014 conference with the theme “Creation and Innovation: Guidelines in the Digital Age”

• Our work fits in the context of both “creation and innovation”

• Our main aim is to teach and guide the participants in our process of “contextualisation” to come up with more practical recommendations which are based on current evidence and relevant to the management of specific conditions

• Overall, the workshop objectives are for the participants to:– Have a clear understanding of the process and practicalities of

contextualisation of international guidelines– Have a clear understanding of how to source and appraise

existing international guidelines to be used as basis for contextualisation

– Have knowledge of identifying patient journeys to map international guidelines with local practice

– Have knowledge and skills to apply the contextualisation process in at least one condition relevant to them.