Systematic reviews from a policy perspective

Sara Hayes

Director of Public Health, Abertawe Bro Morgannwg University Health Board

Honorary Senior Lecturer, Swansea University

Background Systematic reviews can be

conducted on policy areas as well as defined health interventions

In many countries, local interagency collaborations have been introduced to improve health outcomes and evidence is needed on their effectiveness

Collaboration between local health and local government agencies for health improvement

S Hayes, M Mann, F Morgan, H, Kitcher, M Kelly, A WeightmanPublished 2011

Aimed to evaluate the effects of interagency collaboration between local health and local government agencies on health outcomes

What were we looking for?Interventions delivered through

interagency collaboration between statutory health and local government agencies, where the level of partnership between collaborators could be clearly determined and where the interventions were aimed at improving health

How did we define collaboration?Two or more parties that pursue an

agreed set of goals and work cooperatively toward a set of shared health outcomes

Local collaboration was judged to have taken place if there was evidence that the partners had agreed local joint working arrangements and shared objectives

Search strategyTwenty-five databases Relevant websites Experts were contacted Reference lists followed up for relevant - randomized controlled trials (RCTs)- controlled clinical trials (CCTs)- controlled before-and-after studies

(CBAs) - interrupted time series (ITS)

ResultsEleven studies were identified,

representing a total of 26,686 participants

Owing to the heterogeneity between studies a narrative synthesis was undertaken

Six studies of mental health initiatives - 1 showed health benefit- 4 showed modest improvement in one or

more of the outcomes measured, but no clear overall health gain

- 1 showed no evidence of health gain

Two studies on lifestyle - 1 failed to show health gains - 1 showed more unhealthy lifestyle

behaviours persisting in the intervention population

Three studies on chronic disease management

- all 3 failed to demonstrate health gains

Six studies of mental health initiatives - 1 showed health benefit- 4 showed modest improvement in one or

more of the outcomes measured, but no clear overall health gain

- 1 showed no evidence of health gain

Two studies on lifestyle - 1 failed to show health gains - 1 showed more unhealthy lifestyle

behaviours persisting in the intervention population

Three studies on chronic disease management

- all 3 failed to demonstrate health gains

Six studies of mental health initiatives - 1 showed health benefit- 4 showed modest improvement in one or

more of the outcomes measured, but no clear overall health gain

- 1 showed no evidence of health gain

Two studies on lifestyle - 1 failed to show health gains - 1 showed more unhealthy lifestyle

behaviours persisting in the intervention population

Three studies on chronic disease management

- all 3 failed to demonstrate health gains

Six studies of mental health initiatives - 1 showed health benefit- 4 showed modest improvement in one or

more of the outcomes measured, but no clear overall health gain

- 1 showed no evidence of health gain

Two studies on lifestyle - 1 failed to show health gains - 1 showed more unhealthy lifestyle

behaviours persisting in the intervention population

Three studies on chronic disease management

- all 3 failed to demonstrate health gains

ConclusionsThe review did not identify reliable evidence that

interagency collaboration, compared to standard services, leads to health improvement

A few studies identified some benefits but these were not reflected in overall outcomes and could have resulted from the use of significant additional resources

Methodological flaws and incomplete implementation of initiatives have prevented the development of a strong evidence base in this field

RecommendationsHealth and local government services

should ensure their standard services work well together

Pilot projects and new interventions should include health outcome measurement and evaluation in the study design from the start

http://summaries.cochrane.org/CD007825/collaboration-between-local-health-and-local-government-agencies-for-health-improvement

 

What did I learn from the results of this review?

I have always worked in multidisciplinary settings. Am I now saying it’s all been a waste of time?

Port health officers, CsCDC and related agencies across Wales meet regularly and run exercises

Consider the limitations of the reviewProjects and new programmes

were compared to mainstream services

No evaluation of mainstream services themselves

Statutory partnerships were not included

What else might we want to know?

It is not yet clear how much resource should be spent supporting partnership working (eg through team meetings, joint training, shared staffing) compared to that used to deliver the required service

Our update will look for evidence on partnership evaluation in our included studies

How do we present these findings to policy makers?

This is a difficult message to give. The findings can be expressed

more positively ..... .......routine services perform as well as one-off projects and consideration needs to be given to maximising their effectiveness

How can we help policy makers?Encourage links between public health advocates and policy makers Help decision makers understand that not all evidence is of equal value

Improve the quality of published papers

Disseminate key findings

Rigour is as important as enthusiasm in any new development

Help is at hand for the policy maker

Conclusions for policy makersJoint working between local services needs to

be supported in line with local priorities and resulting health outcomes should be monitored

New service developments need to be evidence-based, evaluated at implementation and applied to the whole service if successful

Pilot projects should not be funded if there is no prospect of them being adopted across the whole service

The systematic review process can be a useful tool to assess the effectiveness of policy implementation

What might Archie say about this review?

Would he accept that the effectiveness of collaborative partnership working could be tested through the systematic review process?

Would he accept the principle of combining information from such diverse trials?

How should such information be analysed?

What did I learn from conducting this review?It is possible to be a systematic reviewer

while in a service postI aim to be the advocate for the

systematic use of evidence in service development

It is important to negotiate a commitment to the review in the job plan to be able to take part in group work in the working week

However...

...one summer holiday

Introduction

Aim

Background

Methods

Results

Discussion

Conclusion

Thank you for listening.

Any questions?