Researching Processes and System-level Change

Penny HaweMarkin Institute, University of Calgary

phawe@ucalgary.ca

Take home message

• When it comes to school-based health promotion, what’s inside the “black box” is still unclear/contested

• Useless programs tell us important things

• We need to shift from program thinking to system thinking

2

3

4

The Gatehouse Project: changes in health risk behaviour in year 8 students after 2

years

0

20

40

60

80

100

120

Smoking Regularsmoking

Bingedrinking

Cannabis WeeklyCannabis

% o

f gr

oup

Comparison schools

Intervention

All analyses adjusted for previous level of substance use in the school

5

Theoretical Framework of the Gatehouse Project

INDIVIDUAL CHARACTERISTICS

Improved learning outcomes

Emotional well-being

Sense of Belonging

Connectedness

SOCIAL AND LEARNING ENVIRONMENTS

Security Communication Positive regard

Skills OpportunitiesOpportunities

6

Social change processes matter.

Social processes are easy to miss and elusive to track.

But if we want to be more intentional and effective in creating supportive school environments, we’d better get better at

measuring and recognising them.

7

TODAY

• Looking inside the “black box” to understand change processes

• Insights from “useless” programs

• The promise of system thinking

8

Prevention Studies, Implementation Findings

Battish et al (1996) 66% of schools did notimplement the program properly

Rohrbach et al (1993) 79% teachers omitted program components

Taggart et al (1990) 65% teachers did not implement properly

Flannery et al(1993) 67% teachers miss key components

Dulak JA, J Prev & Intervention in the Community 1998;17:5-189

Implementing preventive interventions

Dosage Providing sufficient exposure to the program

Adherence Following methods as outlined

Quality of process Active methods to engage students

Adaptation Modification to developmental or cultural needs

+

teacher attitude and experience

Dusenbury et al Health Ed Res 2004;20(3):308-313

10

Prevention Studies, Implementation Findings

Battish et al (1996) 66% of schools did notimplement the program properly

Rohrbach et al (1993) 79% teachers omitted program components

Taggart et al (1990) 65% teachers did not implement properly

Flannery et al(1993) 67% teachers miss key components

Dulak JA, J Prev & Intervention in the Community 1998;17:5-1811

12

Thinking of my teachers this term, I really like……..

All of them 14%

Most of them 42%

Half of them 16%

One or two 25%

None of them 3%

13

What is formally “taught” or implemented

14

Three perspectives on innovation

Technological perspective

Political perspective

Cultural perspective

House ER. Three perspectives on innovation. In Lehming, R., & Kane, M. Eds Improving Schools. Using What We Know. : Sage 1981

15

School Staff and Teacher Network Survey

Assessed - knowing by name- regular conversations- knowing more personally- advice seeking- socialising

Twice. At the start, and one year later.

16

From the list below, tick of the name of the people

you talk with regularly (more than just saying ‘hi’)

Ahmed AbboudSue SmithKyle HilllaryGideon GlucksternNick QuinnTim BlytheSalim NoosarYvette LemieuxJosee GiscardMitzi Lamarack

17

Ahmed Sue Kyle Gideon Nick Salim Yvette

Ahmed 1 0 0 1 1 0 0Sue 0 1 0 0 1 0 1Kyle 0 0 1 0 1 0 0Gideon 1 0 0 1 0 0 0Nick 0 1 0 0 1 0 0Salim 0 0 0 0 0 1 0Yvette 1 0 0 1 1 1 1

18

Advice seeking a year ago

19

Advice seeking now

20

Socialising with each other a year ago

21

Socialising with each other now

22

Density of relationships, before and after (%)

Before After

Knowing by name 66 95

Knowing more personally 29 39

Regular conversations 26 41

Seeking advice 15 21

Socialising with 6 8

23

TIME 1 TIME 2

2-Step reach 2-Step reach

Knows personally Principal Vice principal

98%98%

100%100%

Reg conversations Principal Vice principal

75%73%

100%98%

Advice seeking Principal Vice principal

85%83%

100%100%

Socialise with Principal Vice principal

27%47%

33%43%

Two step reach for key actors

24

A good result

But it always depends on what type of networks are desirable:

- dense versus sparse- reciprocity- centrality of particular people

..…it depends on your theory and objectives

25

TODAY

• Looking inside the “black box” to understand change processes

• Insights from “useless” programs

• The promise of system thinking

26

DARE Drug Abuse Resistance Education

• 33 million children 1983-1997, no evidence of effectiveness

• Costly. Average of $217-$334 per child per year

• Renovated at cost of $13.7m in 2001, still not known if effective

Example 1

27

“One of the most important benefits and by products is the relationship with the school department…. It really couldn’t be better. If ever I need anything all I have to do pick up the phone.”

Police Chief

Evaluation and Program Planning2002;26:575-603

28

Example 229

• Physician innovator whose actions people want to reward, praise

• Manager wants to inform nurses of how some new thing should get done, so a leaflet with ‘dual’ purpose gets designed

• Committee formed to develop a leaflet develops a solution that reflects everyone’s interests

• Public relations department updates, and makes more attractive, a leaflet designed by rival agency

30

TODAY

• Looking inside the “black box” to understand change processes

• Insights from “useless” programs

• The promise of system thinking

31

You know you have changed a person when……….

You know you have changed a system when…………..

32

System thinking….

Inter-connections, interactions

Local contextual meaning

Feedback loops

Amplification mechanisms

Status of the parts of the system engaged in the intervention

33

Essence of ecological system thinking

Properties of social contexts keep replicating particular health outcomes...regardless of who is cycled through these contexts

…..social roles, work roles, classrooms, schools….

34

Research implications