Update on the Monitoring and Evaluation of RS-10 Brazil/Vida no Trânsito JH-IIRU: Jeffrey C. Lunnen...

Update on the Monitoring and Evaluation of RS-10 Brazil/Vida no

Trânsito

JH-IIRU: Jeffrey C. Lunnen and Adnan A. Hyder UFMG: Waleska Teixeria Caiaffa

PUC-PR: Samuel Moyses and Sandra Ulinksi AguileraUFRGS: Flavio Pechansky and Tanara Sousa

Outline

• JH-IIRU’s approach to M&E• Background information on 5 RS-10/VnT

intervention cities• Summary of data sources• Secondary data• Primary data (by intervention city and risk

factor)• Discussion• Challenges/Barriers/Opportunities• Future plans

JH-IIRU’s approach to M&E

• M&E to be conducted by partners independent from implementing agencies – Design of the global consortium – Enhances credibility of an evaluation

• M&E must not depend only on routinely collected information – Nor can we wait…

JH-IIRU’s role in RS-10 Brazil/VnT

• Defining baseline impact of RTI in selected sites – Based on secondary and primary data

• Monitoring trends in selected indicators over a period of four years – Using repeated (serial) observations

• Assess the impact of interventions in changing health outcomes – Crashes, injuries and deaths

Background: RS-10/VnT

Teresina

Palmas

Belo Horizont

e

Campo Grande

Curitiba

Targeted Risk Factors Drinking & Driving Speeding

JH-IIRU’s role M&E via collaborations with:

UFMG (Belo Horizonte & Campo Grande)

PUC-PR (Curitiba) UFRGS (Palmas &

Teresina)

Interventions Social Marketing Enhanced enforcement (blitz) Engineering

Background: RS-10/VnT Intervention Cities

Site RegionPopulation (2010)

Death per 100,000

population (2010) Motor vehicle

per 1,000

Population

(2010)

Fleet; % motos)

(2012)

Fleet Growth (2010-2012)

place of occurrenc

e

place of residence

Brazil191,481,0

4522,5 339.8

76,137,191

(25.5%)17.5%

Belo Horizont

e

Southeast

2,375,151 26.3 19.3 564.21,519,438(13.4%)

13.4%

Campo Grande

Midwest 786,797 28.9 28.5 498.4453,531 (29.2%)

15.7%

Curitiba South 1,751,907 22.9 17.7 712.41,371,431(10.9%)

9.9%

Palmas North 228,332 45.1 31.5 474.1132,537 (37.4%)

22.4%

TeresinaNorthea

st814,230 57.8 31.7 246.6

348,120(39.0%)

23.4%

Source: IBGE/Ministry of Health/DENATRAN Source: MoH; IBGE; DENATRAN

Data Sources for ongoing M&E

Secondary data: – Publically available sources:

• Police data • Ministry of Health:

– Available only by cooperative effort [with risk factor data]:

• Enforcement blitz & infraction data from police

Primary data: – Drinking & driving: Traffic agents/police

roadblocks• Results from breathalyzer testing• Roadside surveys

– Speeding: observations by handheld speed camera/radar

Secondary Data:

Overview

Outcomes: Rate of crashes with victims, 2008-2012

2008 2009 2010 2011 20120.00

100.00200.00300.00400.00500.00600.00700.00800.00900.00

1,000.00

Belo Horizonte Campo Grande CuritibaPalmas Teresina

Cra

sh r

ate

per

10

0,0

00

p

op

ula

tion

Sources: DETRAN-MG; DETRAN-MS; DETRAN-PR; DETRAN-PI; Transit Secretariat of Palmas

Outcomes: Rate of non-fatal hospitalized RTIs, 2002 – 2012

Source: MoH; SIH

2002 2003 2004 2005 2006 2007 2008 2009 20100.00

50.00

100.00

150.00

200.00

250.00

300.00

350.00

Brazil Belo Horizonte Campo GrandeCuritiba Palmas Teresina

Year

Inju

ry r

ate

per

year

per

10

0,0

00

pop

ula

iton

Outcomes: Non-fatal RTI rate 2002 – 2012

2008 2009 2010 2011 20120.00

200.00

400.00

600.00

800.00

1000.00

1200.00

Belo Horizonte Campo Grande CuritibaPalmas Teresina

Year

Inju

ry r

ate

per

year

(per

10

0,0

00

)

Sources: DETRAN-MG; DETRAN-MS; DETRAN-PR; DETRAN-PI; Transit Secretariat of Palmas

Outcomes: RTI mortality rate 2002 – 2010

2002 2003 2004 2005 2006 2007 2008 2009 20100.00

10.00

20.00

30.00

40.00

50.00

60.00

70.00

Brazil Belo Horizonte Campo Grande Curitiba PalmasTeresina

Year

Mort

ality

rate

per

10

0,0

00

p

op

ula

tion

Source: MoH; SIM

Primary Data:

Overview

Primary data collection: Summary of rounds conducted to date

Belo Horizon

te

Campo Grande Curitiba Palmas Teresina

Drinking & Driving

Obsr - - 3 4 4RSS - - 3 4 4

Speeding

Obsr 1* 1* 3 2 2Obsr=Roadside Observational Study; RSS=Roadside Surveys*Raw data pending

Primary Data:Belo Horizonte and Campo Grande

UFMG

Primary Data:CuritibaPUC-PR

Risk factor: Drinking & driving

Methods

35 (military) police officers

• Data collection – 08:00pm to 01:00am

• Sites and participants randomly selected

• Staff: 11 researchers

• Police conduct checkpoint procedures

Risk Factor: Drinking & driving

• After Round 2 we utilized tablets for data collection

Drinking & driving Roadside observations (2012)

March 2012 September 2012 November 2012

Description N% (95%

CI)N

% (95% CI)

N % (95% CI)

Breathalyzer Offered

15779% (74,

85)212

63% (58, 68)

29 63% (49, 77)

Breathalyzed 7246% (39,

54)99

47% (40, 53)

1138% (20,

56)

Breathalyzer test >0

4 6% (0, 11) 0 0 0 0

Reported any alcohol consumption in last 6 hours

148.9% (4,

13)12 5,7 (2, 8) 2 9% (-8, 26)

March September November0

50

100

150

200

250

300

158

212

30

40

55

16

RefusedAcepted

Drinking & driving Roadside observations

Round 1 Round 2 Round 3 P-value

MonthMarch 2012 July

November -

Sample Size (n.) 157 212 30 -

Male (%) 71 81,6 84,8 0.00Age (20 to 29 years)% 46.4 47.3 48.3 0.53

High School or + (%) 65.6 86.8 80.0 0.00Resides in Curitiba (%) 85,3 74,5 90,0 0.00

Has a License (%) 97.5 99,1 96,7 0,43

Drinking and DrivingRoadside surveySocio-Demographic Data

Round 1 Round 2 Round 3 P-value

Month March July November

Sample Size (n.) 157 212 30 -

Speeding 26.8 15.6 24.1 0.02

Red light 8.3 8.0 6.9 0.96Drinking & driving 0.0 1.9 6.9 0.16

Mobile phone use 8.9 6.6 13.8 0.35

Restraint device 6.4 4.7 0 0.35

Drinking and drivingRoadside SurveySelf-reported having been ticketed

Description n=33 % χ² p ValueGender Male 29 87,9 3,8 0,15 Female 4 12,1 Vehicle Car 23 71,9 0,7 0,69 Motorcycle 9 28,1 Licence Without licence 3 9,1 18,9 0,04

Less than 1 year 2 6,1

1 to 2 years 3 9,1

2 to 5 years 8 24,2

5 to 10 years 5 15,2

More than 10 years 12 36,4 Origin Home 7 21,2 94,4 0

Friend’s house 3 9,1 Work 5 15,2 Bar/restaurant 9 27,3 University 4 12,1 Others 5 15,2

Risk Factor: Drink and driving

Description n=33 % χ² p ValueReason for travel Work 3 9,1 60,5 0.00 Pleasure 24 72,7 Studies 4 12,1 Outher 2 6,1 Frequency of alcohol consumption

5 or more times a week

7 21,2 133,2 0.00

1 - 4 times a week 8 24,2

1 - 3 times a week 15 45,5

Less than 1 time a month

3 9,1

Alcoholic beverages consumed 0,5 – 2 8 24,2 93,3 0.00 3 - 5 6 18,2 6 – 10 13 39,4

More than 11 6 18,2 Alcoholic beverage consumed Beer 18 54,5 367,5 0.00

Beer and liquor/spirits

1 3

Liquor/spirits 3 9,1 Wine 3 9,1

Risk Factor: Drink and driving

• Methods– Selected roads with a high rate of speed-

related infractions and or crashes in 2011

– Excluded crosswalks, sites with poor visibility

– Special assurances to ensure observer safety and parking

– Observations: 09:00-11:00am and 02:00-04:00pm

– Sites and days of week for observation were selected at random

Risk Factor: Speeding

SpeedingRoadside observations (2012)

March 2012 September 2012 October 2012

Sample

% speeding (95% CI)

Sample

% speeding (95% CI)

Sample% speeding (95%

CI)

Car 15641 1.8% (1.6, 2.0) 17090 14% (13.6, 14.6) 14605 31% (30.5, 32)

Motorcycle

1782 9.1% (8, 10) 1320 34% (32, 37) 1161 53% (50, 56)

Bus 427 0.5% (0, 1) 255 6% (3, 8) 217 6% (3, 9)

Truck 548 0.6% (0, 1) 1056 12% (10, 14) 753 13% (11, 16)

Total 18698 2.4% (2.3, 2.5) 19721 15% (14.7, 15.7) 16736 32% (30, 32.3)

Primary Data:Palmas and Teresina

UFRGS

Risk factor: Drinking & driving

• Method– Sample size: based in fleet/population size and

estimated prevalence– Site selection: roadblocks locations, dates and times

were chosen with the police and traffic agents. – Roadblocks location: selected based on the traffic

crash occurrence in 2010/11. – Process

• Traffic agents/police stop cars & motorcycles • Standard roadblock procedures • Invited the drivers to participated on the study

(interview)• Offer breathalyzer

Risk factor: Speeding

• Method– Hand-held static radars with attached camera – 7 sites randomly selected from a composite list

of 30 sites with high frequencies of accidents in 2011

• Data collected in five 1-hour intervals for seven days a week, from 8:00AM to 6:00PM

• Site selection considered the absence of fixed radars, general traffic flow and safety of data collectors

Results from Palmas

Drinking & drivingRoadside observations (2011-2013)

Round 1(Aug 2011)

Round 2 (Jan 2012)

Round 3 (Jun 2012)

Round 4 (Apr 2013)

Description N % N % N % N %

Flow 5356 - 4244 - 6064 - 2526 -

Invited to participate in

study211 3.9 288 6.8 414 6.7 285 11.3

Agreed to participate

147 69 220 74 350 85 229 81


118 56 189 66 275 64 184 77.6

Breathalyzed 95 80.5 138 73.0 23485.1

173 94.0

Breathalyzer Test >0

11 11.6 11 8 17 7.3 10 5.8

Drinking and DrivingRoadside surveySocio-Demographic Data

Round 1

Round 2

Round 3 Round 4

P-value

Month Aug-11 Jan-12 Jun-12 Apr-13 -

Sample Size (n.) 147 220 350 229 -

Male (%) 80.3 87.3 84.0 80.3 0.00

Age (mean) 30.5 31.7 30.9 30.8 0.92

High School or + (%)

78.9 89.5 88.8 93.0 0.00

Resides in Palmas(%)

100.0 95.0 96.5 95.6 0.07

Has a License (%)

- 96.8 91.6 98.7 0.00

Drinking and drivingRoadside SurveyAlcohol use in the last 6 hours Consumo de álcool nas últimas 6 horas

Round 1 Round 2 Round 3 Round 40%

25%

50%

75%

100%

27.5 23.214.1 10.3

Yes NoP-value = 0.00

Drinking and drivingRoadside SurveyAbility to drive compromise by alcohol consumption?Habilidade para dirigir está comprometida por este consumo de álcool?


25%

50%

75%

100%

2.9 9.4 13.223.5


Drinking and drivingRoadside SurveyDrinking and driving in the last yearDirigiu após beber, no último ano


25%

50%

75%

100%

15.931.7 25.4 26.5

46.627.2

24.9 21.1

37.5 41.1 49.7 52.4

at least 1X at month less then1X at monthno drinking driving

Drinking and drivingRoadside SurveyKnowledge of the legal alcohol limit, by LawConhecimento do limite para beber e dirigir (pela legislação)


25%

50%

75%

100%

37.4 32.322.5

43.5



25%

50%

75%

100%

44.5 47.0 45.5 44.5

Yes No

Drinking and drivingRoadside SurveyIn the last year was a passenger of driver under the influenceUsou carona de condutor que bebeu, no último ano

P-value = 0.95


25%

50%

75%

100%

11.7 11.9 12.0 5.7

Yes No

Drinking and drivingRoadside SurveyHave had at least 1 alcohol associated crash (driver)Teve pelo menos um acidente causado por álcool enquanto era condutor, na vida

P-value = 0.07

Round 2 Round 3 Round 40%

25%

50%

75%

100%

59.6 62.6 61.6

Yes No

Drinking and drivingRoadside SurveyIn the last year was stopped at at least one blitz Foi parado em pelo menos uma blitz nesta cidade, no último ano

P-value = 0.78


25%

50%

75%

100%

10.7 9.2 14.2

Yes No

Drinking and drivingRoadside SurveyI invited to take a breathalyzer testFoi convidado a fazer teste do etilômetro

P-value = 0.35

SpeedingRoadside observations (2012-2013)

*Second Round in March/2013

June 2012 March 2013

Sample % speeding (97% CI) Sample % speeding (97%

CI)

Car 8009 50% (48, 51) 7135 49% (47, 50)

Motorcycle 2215 30% (28, 32) 2415 41% (39, 43)

Bus 217 28% (22, 34) 281 14% (10, 18)

Truck 269 32% (27, 40) 318 37% (31, 50)

Total 10710 45% (44, 46) 10149 46% (45, 47)

Speeding (2012-2013)


round 1 round 20%

25%

50%

75%

100%

+50% above limit20-50% above limit5-20% above limit0-5% above limitno speeding

14% 18%

56% 55%

6% 7%

19% 16%

5% 3%

Speeding (2012-2013)


0%

20%

40%

60%

80%

52%

4%

22%

74%

25%

71%

22%

44%59%

14%

34%

73%

34%18%

27%45%

round 1 round 2


25%

50%

75%

100%

9.1 11.2 8.7

Yes No

SpeedingRoadside SurveyHave had at least 1 ticket for speeding in the last yearTeve pelo menos multa por excesso de velocidade, no último ano

P-value = 0.56

Drinking & driving,Roadside observations (2011-2013)

Round 1(Aug 2011)

Round 2 (Feb 2012)

Round 3 (Aug 2012)

Round 4 (Feb 2013)

Description N % N % N % N %

Flow 3958 - 5429 - 3098 - 2676 -

Invited to participate in

study

286 7.2 249 4.6 214 6.8 141 5.3

Agreed to participate

236 82.5 209 81.1 187 82.2 126 89.4


197 68.8 112 45.0 132 61.7 120 85.1

Breathalyzed 153 77.7 110 98 125 94.7 115 95.9

Breathalyzer Test >0

6 4.0 7 6.4 13 10.4 4 3.5

Results from Teresina

Drinking and DrivingRoadside surveySocio-Demographic DataDados Socio-demográficos

Round 1

Round 2

Round 3 Round 4

P-value

Month Aug 11 Feb 12 Aug 12 Feb 13 -

Sample Size (n.) 236 209 187 126 -

Male (%) 80.9 89.7 85.0 83.3 0.03

Age (mean) 35.2 35.9 35.4 35.9 0.72

High School or +(%)

83.9 75.7 76.5 88.9 0.00

Resides in Teresina(%)

100.0 77.2 81.2 69.8 0.07

Has a License (%)

- 95.5 95.2 97.6 0.53

Drinking and drivingRoadside SurveyAlcohol use in the last 6 hours Consumo de álcool nas últimas 6 horas


25%

50%

75%

100%

16.1 16.9 23.8

2.9


Drinking and drivingRoadside SurveyAbility to drive compromise by alcohol consumption?Habilidade para dirigir está comprometida por este consumo de álcool?


25%

50%

75%

100%

6.7

36.247.1

12.5


Drinking and drivingRoadside SurveyKnowledge of the legal alcohol limit, by LawConhecimento do limite para beber e dirigir (pela legislação)


25%

50%

75%

100%

38.926.7 31.6

52



25%

50%

75%

100%

43.8 38.1 34.8 30.4

Yes No

Drinking and drivingRoadside SurveyIn the last year was a passenger of driver under the influenceUsou carona de condutor que bebeu, no último ano

P-value = 0.12


25%

50%

75%

100%

9.4 9.5 11.2 5.7

Yes No

Drinking and drivingRoadside SurveyHave had at least 1 alcohol associated crash (driver)Teve pelo menos um acidente causado por álcool enquanto era condutor, na vida

P-value = 0.43

Speeding, Roadside observations (2012-2013)


August 2012 February 2013

Sample % speeding (95% CI) Sample % speeding

(95% CI)

Car 11756 10.6% (10, 11) 8134 12% (10.8, 12.2)

Motorcycle 5153 8.1% (7, 9) 3862 6.9% (6.1, 7.7)

Bus 536 10% (8, 13) 432 9.4% (7, 12)

Truck 526 7% (5, 8) 726 4.4% (3, 6)

Total 17017 9.7% (9.3, 10.2) 13334 9.6% (9. 10)

Challenges/Barriers/Opportunities

Discussion

Drinking & Driving – Observations conducted in absence of

reliable data– Breathalyzers increasingly offered and

accepted, but drivers can refuse– Small samples due largely to low

cooperation from police + pre-defined days for data collection in some cities

Discussion

Speed– Teresina: low prevalence compared to other

cities• Poor street quality; cars do not reach the speed

limit

– Palmas:• High prevalence• Drivers do not respect crosswalks

– Curitiba: 3 waves using 3 different methods• Changes in radar types, street locations

– Belo Horizonte & Campo Grande:• Different method as well

Challenges & Barriers: In general

• Uniqueness of cities – Size– Infrastructure

• Changing municipal leadership and institutional policies – Inhibits access to secondary data

– Delays primary data collection

Challenges & Barriers

• Data collection did not begin until 2011

• Alcohol data particularly difficult to obtain due to:– Refusals + inconsistent cooperation

from police in some cities

• Methodological differences

Opportunities

• Tougher “Lei Seca”– Every driver will be fined/jailed for

refusal to take a Breathalyzer test– Police empowered!

• Standardize methods across 5 RS-10 Brazil/VnT intervention cities

• Increase the number of rounds of data collection

Future Plans

• Improve quality/frequency of data collection– Insert training component for police

officers– Make linkages with National Highway

Patrol Department– Insert specific social marketing

messages into KAP survey to determine reach

• Bebida no trânsito mata. E nem sempre só você.

• Gather additional secondary information

ACKNOWLEDGEMENTS

• Dr. Aruna Chandran• International Consortium Partners• National Commission VnT• Local VnT partners• Secretariats of Health and

Transportation/Transit• Military Police• Highway Patrol

OBRIGADO! THANK YOU!

Update on the Monitoring and Evaluation of RS-10 Brazil/Vida no Trânsito JH-IIRU: Jeffrey C. Lunnen...

Documents

Transcript of Update on the Monitoring and Evaluation of RS-10 Brazil/Vida no Trânsito JH-IIRU: Jeffrey C. Lunnen...