EUROPEAN DRUG PREVENTION QUALITY STANDARDS EQUS conference, Brussels, 15-17 June 2011.

EUROPEAN DRUG PREVENTION QUALITY STANDARDS

EQUS conference, Brussels, 15-17 June 2011

Synergy between two projects

Prevention part of the EQUS standards was developed through an earlier, separate EC co-funded project

Project started 2008, completed in 2010

It was recognised that it was more efficient to adapt the existing prevention standards than to develop a new set of prevention standards synergy between both projects

However: different structure of standards, different methodologies etc.

Outline of the presentation

1. Introduction to the prevention standards project

2. Methodology of standards development

3. Structure/content of the prevention standards (brief!)

4. Adaptation of the standards to the EQUS project

5. Structure/content of the EQUS prevention standards

If you have any questions, pleasefeel free to ask at any time

Introduction to the Standards project

Project beneficiaries

Project partners: Liverpool John Moores University (LJMU), United Kingdom

(Project lead) Azienda Sanitaria Locale della Città di Milano (ASL), Italy Consejeria de Sanidad - Servicio Gallego de Salud (Xunta de

Galicia) (CS-SERGAS), Spain Azienda Sanitaria Locale n. 2 - Savonese (ASL2), Italy Institute for Social Policy and Labour (SZMI-NDI), Hungary National Anti-Drug Agency (NAA), Romania National Bureau for Drug Prevention (NBDP), Poland European Monitoring Centre for Drugs and Drug Addiction

(EMCDDA)

Adaptation of standards for EQUS project carried out by LJMU

Background & Aims

At the time of starting the project: No EU-level guidance on evidence-based drug prevention National or regional guidance available in some countries –

applicable to wider EU? USA standards of evidence – applicable to European

context? Lack of guidance for policy makers and practitioners

Aims: To bridge the gaps between science, policy and practice To produce a set of evidence-based drug prevention

standards for use in the EU To provide a checklist for policy makers and practitioners

Potential Impact of Standards

Policy and practice: Improve drug prevention practice (e.g. increase sustainability of

interventions) Improve efficiency of funding (better outcomes)

Target populations: Reduce likelihood of implementation of ineffective or iatrogenic interventions Increase relevance and acceptability of interventions for target populations

Research and evidence base: More evidence-based and scientifically sound interventions Improve European evidence base for prevention by promoting research

methodology Interested? European Society for Prevention Research (EUSPR)

(http://euspr.org)

Methodology of developing the Standards

Collecting information about available drug prevention guidance

12 EU countries with drug prevention standards or guidance(9 already on EMCDDA website, 3 new)

9 EU countries: no standards (yet) 6 EU countries: no information received

Available guidance – EU countries(spring 2009)

Drug prevention standards and/or guidelines available

No standards No information received

Czech RepublicDenmarkFinlandGermanyIrelandItaly (regionally)LithuaniaPolandPortugalRomaniaSpain (Galicia)United Kingdom

AustriaCyprus (in progress)*FranceGreeceHungary (in progress)LatviaNetherlands (in progress)SloveniaSweden

BelgiumBulgariaEstoniaLuxembourgMaltaSlovakia

* Cyprus: standards completed in late 2009

Methodology

Method Aims Implementation Timeline

Collation and review of existing guidance

To produce a long list of standards; to identify a common structure that will synthesise existing standards

77 documents retrieved, 19 documents selected

March-September 2009

First draft of standards

Methodology

Field testingUsability and feasibility of standards

72 professionals took part

August-September 2010

Method Aims Implementation Timeline

Delphi surveyPerceived priority of standards

423 professionals completed both rounds

January-February 2010

Focus groups(Cultural) relevance of standards

14 focus groups held March-April 2010

First draft of standards

Second draft of standards

Final standards

Participants in consultations

In six countries: Galicia (Spain), Hungary, Italy, Poland, Romania, UK

Sampling frame covered ten professional backgrounds: Regional drug teams or networks Education Health Mental Health Social services/ Children, young people, families Criminal Justice Voluntary/ Community sector Government representatives Prevention consultants Media

Online Delphi survey

Aim: to validate draft structure and to identify priority items Two rounds, with 2-3 weeks per round (January and February 2010)

First round: Participants rated the priority of each component from “High Priority” to

“Not a priority at all” Additionally, possibility to choose whether standards should be made

“mandatory” or not Text comments possible

Second round: Same as above, but – Also information on how all participants had rated the components in the

first round, and examples of text comments from the first round Only results from Round 2 are shown in this presentation

Response rates

EU ES HU IT PL RO UK

Recruitment: Persons contacted

987 56 114 210 319 82 206

Round 1: complete and valid responses

487 53 91 118 95 53 77

% of contacted 49% 95% 80% 56% 30% 65% 37%

Round 2: complete valid responses

423 53 83 94 84 46 63

% of Round 1 responses 87% 100% 91% 80% 88% 87% 82%

Delphi survey findings (excerpt)

Priority

Consensus

High priority

Mandatory yes

Priority Rank

1. Situation Analysis 1.1. Assessing drug use and community needs

2.73 74.6 % 87.7 % 12

1.2. Defining the problem - Justifying the intervention

2.63 66.4 % 87.3 % 16

1.3. Understanding the target population 2.88 88.7 % 92.9 % 12. Resource Assessment 2.1. Knowing policies and legislation 2.36 42.3 % 80.9 % 362.2. Identifying funding opportunities 2.35 < 50 % 43.4 % 72.1 % 372.3. Mapping existing stakeholders and services

2.48 53.7 % 83.4 % 30

2.4. Assessing target population and community resources

2.74 76.1 % 84.9 % 10

2.5. Assessing internal capacities 2.54 55.5 % 81.3 % 243. Programme Formulation 3.1. Defining the target population 2.88 88.8 % 94.5 % 23.2. Defining goals and objectives 2.83 83.5 % 90.0 % 43.3. Defining the setting 2.32 40.8 % 78.4 % 383.4. Summarising the evidence base 2.10 26.9 % 57.5 % 443.5. Illustrating the theoretical framework 1.97 20.1 % 60.0 % 463.6. Developing or selecting an effective intervention

2.75 78.6 % 89.5 % 8

3.7. Determining the timeline 2.28 < 50 % 40.4 % 76.3 % 423.8. Ensuring an ethical intervention 2.61 69.0 % 85.5 % 20

Note: Some UK results not included under "Mandatory", as missing values > 50%.

First draft standardsFirst draft standards

Delphi findings - summary

Across all partner countries, 23 out of 46 components obtained consensus and a higher priority rating

Components with highest / lowest priority ratings:

Components PriorityConsensu

sHigh

priority Mandatory

yesPriority Rank

1.3. Understanding the target population 2.88 88.7 % 92.9 % 1

3.1. Defining the target population 2.88 88.8 % 94.5 % 2

4.2. Adapting the intervention to the target population

2.86 87.9 % 89.7 % 3

3.4. Summarising the evidence base 2.10 26.9 % 57.5 % 44

6.3. Conducting a pilot intervention 2.05 < 50 % 30.1 % 42.4 % 45

3.5. Illustrating the theoretical framework 1.97 20.1 % 60.0 % 46

Low ratings – standards not necessarily for exclusion, but for discussion and modification

Focus groups: Relevance, usefulness,feasibility

Relevance: What is the current practice of professionals working in the area, and how do the draft standards relate to it? Are the draft standards appropriate for these professional groups and for their everyday work/projects?

Usefulness: How would the standards improve current practice? What are potential unintended consequences of the draft standards (e.g. economic, ethical)? Are the standards clear to understand and without ambiguity? Are the draft standards likely to be accepted by intended users?

Feasibility: How can the standards be integrated with current practice? Which standards are already common practice in most projects; which standards are “new” and yet to be achieved? Overlap with existing statutory frameworks? What further resources, workforce development/ training or support might be needed to implement the standards? What are the organisational, professional and other barriers to / opportunities for implementation?

Field testing (further focus groups)

Aims:

To discuss second draft standards Content of introduction and appendix Layout

To discuss possibilities for implementation Practical use of the standards Implementation tools

Consultations: Emerging challenges

Issues identified included: Knowledge on how to use standards, and what for - Promote the use of

quality standards Lack of content guidance (i.e. what to do) - Develop content guidelines Funding (e.g. for outcome evaluations, staff development), knowledge

on how to attract funding - Attract funding Prevention work is given less priority than treatment - Prioritise and

support prevention Lack or weakness of governmental prevention agencies - Establish and

strengthen central prevention agencies Diversity of prevention work - Consider differences in prevention practice Duplication of work - Create synergies Scepticism among professional groups - Develop professional attitudes

and skills

- Will be discussed by Dr Rachele Donini in tomorrow’s session

Using the Standards

Applicability of the standards

The standards are applicable to a wide range of drug prevention activities, for example:universal, selective, indicated or environmental prevention;preventing initiation of drug use, reducing the frequency of use and/or reducing drug-related harms;targeting legal and/or illegal drugs;structured manualised programmes or ongoing participant- and needs-led services;short- or long-term projects;different methods, target populations, settings …

The standards give advice on how to plan, implement, and evaluate interventions, and they can be used to reflect on new, ongoing, or completed activities.

Although the standards refer to programmes, they can be used to reflect on prevention work at several levels of delivery, including:

People: individual staff members or teams. Activities: singular interventions or wider programmes comprising several interventions. Organisations: organisations involved in drug prevention, such as service providers or schools. Strategies: priorities, action plans and tenders set out by government or funding bodies.

Applicability of the standards

Who contributes to drug prevention?

Policy- and decision-making: those who work on a strategic level, for example government representatives, commissioners, regional planning teams, funders.

Service management: those who are in charge of managing drug prevention activities at an organisational level within prevention providers.

Front-line work / work in direct contact with the target population: those who conduct, or contribute to, drug prevention in (direct) contact with the target population, such as psychologists, youth workers, social workers, outreach workers, teachers, pharmacists.

Who contributes to drug prevention?

Training: those who provide training in drug prevention. Training may be directed at practitioners (e.g. social workers), or it may take place in a higher education setting (i.e. university students).

Supervision: those who supervise and support drug prevention professionals (particularly practitioners).

Programme development: those who design and develop drug prevention interventions.

Consultancy / evaluation / academic research: those who provide consultancy on prevention issues, for example with regard to effective drug prevention, prevention policy development, adaptation of programmes, and/or evaluation.

Recommended uses of Standards

Purpose Recommended

Information, education and guidance

Developing or updating quality criteria

Self-reflection checklist

Discussion in group settings

Performance appraisals

Dissemination and evaluation plan

Purpose Recommended

Formal self-assessment Not yet

Funding decisions Not yet

External accreditation Not yet

Follow-on projects: Produce user-friendly implementation tools/manuals Trial standards with real programmes Develop accreditation system for model programmes Provide training/education to relevant target audiences International Standards initiative with CCSA, CICAD,

EMCDDA, UNODC (in early planning stage)

Importance of additional materials

Example scenarios of how the standards could be used General and country-specific conditions for (and potential barriers

to) implementation Implementation considerations for each of the 35 standard

components References to additional literature, materials, websites Bespoke glossary developed for use with the standards (over 100

terms) Self-reflection checklist allowing users to reflect upon their current

position in relation to the standards

The Prevention Standards

The drug prevention project cycle- a model to be adopted and adapted

Content of Standards

Cross-cutting Considerations

A: Sustainability and funding

B: Communication and stakeholder involvement

C: Staff development

D: Ethical drug prevention1 Needs Assessment

1.1 Knowing drug-related policy and legislation

1.2 Assessing drug use and community needs

1.3 Describing the need – Justifying the intervention

1.4 Understanding the target population

2 Resource Assessment

2.1 Assessing target population and community resources

2.2 Assessing internal capacities


3 Programme Formulation

3.1 Defining the target population

3.2 Using a theoretical model

3.3 Defining aims, goals, and objectives

3.4 Defining the setting

3.5 Referring to evidence of effectiveness

3.6 Determining the timeline

4 Intervention Design

4.1 Designing for quality and effectiveness

4.2 If selecting an existing intervention

4.3 Tailoring the intervention to the target population

4.4 If planning final evaluations


5 Management and Mobilisation of Resources

5.1 Planning the programme - Illustrating the project plan

5.2 Planning financial requirements

5.3 Setting up the team

5.4 Recruiting and retaining participants

5.5 Preparing programme materials

5.6 Providing a programme description

6 Delivery and Monitoring

6.1 If conducting a pilot intervention

6.2 Implementing the intervention

6.3 Monitoring the implementation

6.4 Adjusting the implementation


7 Final Evaluations

7.1 If conducting an outcome evaluation

7.2 If conducting a process evaluation

8 Dissemination and Improvement

8.1 Determining whether the programme should be sustained

8.2 Disseminating information about the programme

8.3 If producing a final report

Example layout

Level 2: Component title

Implementationconsiderations

Glossary terms

Level 3: Attributes (basic)

Examples to clarifymeaning

Level 3: Attributes (expert)

Three levels of detail

1. Project stages: major areas of prevention practice covered by the standards

2. Components: actions to be taken at that point in the project

3. Attributes: show how achievement of the standards can be evidenced

Two quality levels

Basic standards: applicable to all drug prevention work, regardless of its particular circumstances.

Additional basic standards (“basic standard if”): reflect the basic level under particular circumstances.

Expert standards: in addition to the basic standards, representing a higher level of quality; desirable but may not always be applicable or feasible.

Adaptation to EQUS project

Adaptation of the standards

1) Description and re-assessment of original standards documents in line with template used for treatment and harm reduction areas

Development, applicability, legal status etc. of original standards

All original standards deemed relevant for EQUS project

2) Coding of individual ‘basic’ standards and components in line with standards structure developed by EQUS project coordinators

i.e. structure/ process/ outcome standards + sub-categories No need for new data extraction


3) Developing prevention standards for EQUS project

Adaptation at component rather than attribute level Reduction of 35 components to 30 EQUS prevention

standards Phrasing changed in order to correspond to treatment and

harm reduction standards, but meaning has been preserved Proposed list of minimal quality standards reflects a

summary of the basic standards submitted for EMCDDA publication


4) Revisiting results from Delphi survey to determine level of consensus for standards in line with methods used in treatment and harm reduction areas

Cut-off levels (> 80%, >50-80%, 50% or less) ‘Acceptability’ defined as percentage of those who thought

that a standard would represent a ‘mandatory’ part of good quality drug prevention work (based on first draft standards)

Remember that Delphi survey conducted with first draft standards – likely that final version would receive higher levels of acceptability

EQUS prevention standards

The standards can consequently be distinguished as follows: standards with high acceptability (rated as mandatory by > 80%

of responding participants) standards with medium acceptability (rated as mandatory by 50-

80% of responding participants consensus) – for discussion standards with low acceptability (rated as mandatory by less

than 50% of responding participants) (already excluded/modified through previous work).

Lists were established for structural standards at service level (5 + 3) outcome standards at system level (4 + 0) process standards at service/intervention level (11 + 7)

High acceptability – for information

Discussion:•Are the proposed lists of minimum quality standards (high consensus in surveys) acceptable?

•For which types of services / interventions are they accepted?


Structural Standards of Services

1. Ethical drug prevention: A code of ethics is defined. Participants’ rights are protected through informed consent. The programme has clear benefits for participants, and will not cause them any harm. Participant data is treated confidentially. The physical safety of participants and staff members is protected.

2. Staff composition: The staff required for successful implementation is defined and (likely to be) available (e.g. type of roles, number of staff). The set-up of the team is appropriate for the programme. Staff selection and management procedures are defined.

3. Policy and legislation: The knowledge of drug-related policy and legislation is sufficient for the implementation of the programme. The programme supports the objectives of local, regional, national, and/or international priorities, strategies, and policies.


4. Staff composition: Internal resources and capacities are assessed (e.g. human, technological, financial resources). The assessment takes into account their current availability as well as their likely future availability for the programme.

5. Financial requirements: A clear and realistic cost estimate for the programme is given. The available budget is specified and adequate for the programme. Costs and available budget are linked. Financial management corresponds to legal requirements.

Outcome Standards at the System Level

6. Goal: It is clear what is being ‘prevented’ (e.g. what types of drug use?). The programme’s aims, goals, and objectives are clear, logically linked, and informed by the identified needs. They are ethical and ‘useful’ for the target population. Goals and objectives are specific and realistic.

7. Outcome evaluation: Evaluation is seen as an integral and important element to ensuring programme quality. It is determined what kind of evaluation is most appropriate for the intervention, and a feasible and useful evaluation is planned. Relevant evaluation indicators are specified, and the data collection process is described.


8. Monitoring the implementation: Monitoring is seen as an integral part of the implementation phase. Outcome and process data are collected during implementation and reviewed systematically. The project plan, resources, etc. are also reviewed. The purpose of monitoring is to determine if the programme will be successful and to identify any necessary adjustments.

9. Outcome evaluation: The sample size on which the outcome evaluation is based is given, and it is appropriate for the data analysis. An appropriate data analysis is conducted, including all participants. All findings are reported in measurable terms. Possible sources of bias and alternative explanations for findings are considered. The success of the programme is assessed.

Process Standards of Interventions

10.Assessment procedures: The needs of the community (or environment in which the programme will be delivered) are assessed. Detailed and diverse information on drug use is gathered. The study utilises existing epidemiological knowledge as possible, and adheres to principles of ethical research.

11.Assessment procedures: Sources of opposition to, and support of, the programme are considered, as well as ways of increasing the level of support. The ability of the target population and other relevant stakeholders to participate in the programme is assessed.

12.Assessment procedures: Justifying the intervention: The need for an intervention is justified. The main needs are described based on the needs assessment, and the potential future development of the situation without an intervention is indicated. Gaps in current service provision are identified.


13.Assessment procedures: The target population is chosen in line with the needs assessment. The chosen target population(s) can be reached. The needs assessment considers the target population’s culture and its perspectives on drug use.

14.Intervention design: The programme is adequate for the specific circumstances of the programme (e.g. target population characteristics), and tailored to those if required. Elements to tailor include: language; activities; messages; timing; number of participants

15.Intervention design: The programme builds on positive relationships with participants by acknowledging their experiences and respecting diversity. Programme completion is defined.


16.Intervention design: If selecting an existing intervention, benefits and disadvantages are considered, as well as the balance between adaptation, fidelity, and feasibility. The interventions’ fit to the needs assessment and other local circumstances are also examined. The chosen intervention is adapted carefully, and changes are made explicit.

17.Implementation: The programme is implemented according to the written project plan. The implementation is adequately documented, including details on failures and deviations from the original plan.

18.Process evaluation: The implementation of the programme is documented and explained. The following aspects are evaluated: target population involvement; activities; programme delivery; use of financial, human, and material resources.


19.Staff development: It is ensured prior to the implementation that staff members have the competencies which are required for a successful programme implementation. If necessary, high quality training based on a training needs analysis is provided. During implementation, staff members are supported in their work as appropriate.

20.Dissemination: The final report documents all major elements of programme planning, implementation, and (where possible) evaluation in a clear, logical, and easy-to-read way. Details on implementation experiences and unintended outcomes are included. Legal aspects of reporting on the programme are included (e.g. copyright).

Moderate acceptability – for discussion

Discussion:•Which standards from the presented additional lists of quality standards (medium consensus in surveys) should be included in the definite lists of minimum standards?

•For which types of services / interventions?



1. Sustainability: The programme promotes a long-term view on drug prevention and is not a fragmented short-term initiative. The programme is coherent in its logic and practical approach.

2. Planning the programme: A systematic programme plan is constructed. A written project plan outlines the main programme elements and procedures. Contingency plans and risk management strategies are developed.

3. Sustainability: A programme is continued on the basis of evidence provided by monitoring and/or final evaluations. If it is to be continued, opportunities for continuation are outlined. The lessons learnt from the implementation are used to inform future activities.


(none – all high acceptability)


4. Communication and stakeholder involvement: The multi-service nature of drug prevention is considered. All stakeholders relevant to the programme (e.g. target population, other agencies) are identified, and they are involved as required for a successful programme implementation. The organisation cooperates with other agencies and institutions.

5. Physical environment: The setting(s) for the activities is (are) described. It matches the aims, goals, and objectives, available resources, and is likely to produce the desired change. Necessary collaborations for implementation of the programme in this setting are identified.

6. Recruiting and retention: It is clear how participants are drawn from the target population, and what mechanisms are used for recruitment. Specific measures are taken to maximise recruitment and retention of participants.


7. Intervention design: The programme is based on an evidence-based theoretical model that allows an understanding of the specific drug-related needs and shows how the behaviour of the target population can be changed. Scientific literature reviews and/or essential publications on the issues relating to the programme are consulted. The reviewed information is of high quality and relevant to the programme. The main findings are used to inform the programme.

8. Intervention design: Materials necessary for implementation of the programme are specified. If intervention materials (e.g. manuals) are used, the information provided therein is factual and of high quality.

9. Intervention design: A written, clear programme description exists and is (at least partly) accessible by relevant groups (e.g. participants). It outlines major elements of the programme, particularly its possible impact on participants.


10.Implementation: Flexibility is possible if required for a successful implementation. The implementation is adjusted in line with the monitoring findings, where possible. Issues and problems are dealt with in a manner that is appropriate for the programme. Adjustments are well-justified, and reasons for adjustments are documented. (Not rated in Delphi survey. Included as a result of target group consultation)

Next steps

For European drug prevention quality standards project Publication of standards by EMCDDA

For EQUS project To determine which of the adapted EQUS prevention

standards are acceptable/feasible as EU minimum quality standards – AT THIS MEETING

Suggestions for better adaptation and integration into the EQUS project? – AT THIS MEETING

LJMU will submit revised list of standards to project coordinator following meeting

EMCDDA Manual

Full list of prevention standards will be available as EMCDDA manual (publication expected with 3-6 months).Full list of prevention standards will be available as EMCDDA manual (publication expected with 3-6 months).

Contacts

Angelina BrotherhoodPublic Health [email protected]

Dr Harry SumnallReader in Substance [email protected]

Centre for Public HealthLiverpool John Moores University, UK

EUROPEAN DRUG PREVENTION QUALITY STANDARDS EQUS conference, Brussels, 15-17 June 2011.

Documents

Transcript of EUROPEAN DRUG PREVENTION QUALITY STANDARDS EQUS conference, Brussels, 15-17 June 2011.