Intro to SMS

34
U.S. Department of Transportation Federal Motor Carrier Safety Administration CSA 2010 Operational Model Test Introduction to the Safety Measurement System April 2010

description

Introduction to the CSA 2010 Safety Measurement System

Transcript of Intro to SMS

Page 1: Intro to SMS

U.S. Department of TransportationFederal Motor Carrier Safety Administration

CSA 2010 Operational Model Test

Introduction to the Safety Measurement System

April 2010

CSA 2010 Operational Model Test

Introduction to the Safety Measurement System

April 2010

Page 2: Intro to SMS

U.S. Department of TransportationFederal Motor Carrier Safety Administration

Comprehensive Safety Analysis 2010Comprehensive Safety Analysis 2010

What is CSA 2010?CSA 2010 is a pro-active initiative to improve the efficiency and effectiveness of FMCSA’s enforcement and compliance program to achieve the Agency’s mission to reduce commercial motor vehicle (CMV) crashes, fatalities, and injuries.

2

Page 3: Intro to SMS

U.S. Department of TransportationFederal Motor Carrier Safety Administration

Three Core ComponentsThree Core Components

1. New Safety Measurement System (SMS)

Improved ability to identify demonstrated safety problems

2. New interventions process

Employs an array of interventions instead of the single option, labor-intensive compliance review

3. New approach to Safety Fitness Determination (SFD)

SFD would be tied to current safety performance; not limited to results of acute/critical violations from a Compliance Review

3

Page 4: Intro to SMS

U.S. Department of TransportationFederal Motor Carrier Safety Administration

New Safety Measurement SystemNew Safety Measurement System

• Assesses safety of carriers and drivers based on unsafe behaviors that lead to crashes– Calculates safety performance based on 7 Behavior Analysis and Safety

Improvement Categories (BASICs)– Weights time and severity of violations based on relationship to crash risk– Uses crash records and all safety-based violations found roadside.

• Identifies carriers for interventions– Determines what problems need to be addressed by intervention process– Monitors carriers on road performance for improvements throughout the process

• In the future, measurement scores would support future Safety Fitness Determinations– New Safety Fitness Determination methodology is currently in rulemaking; initial

rollout of CSA 2010 is not dependent on rule

4

Page 5: Intro to SMS

U.S. Department of TransportationFederal Motor Carrier Safety Administration

Methodology Overview Methodology Overview

• Obtain on-road safety data (e.g. inspections, crashes) and attribute to carrier to create a safety event history

• Place each carrier’s violations/crashes into a BASIC

• Convert BASIC data to quantifiable measure/rate (in future, Safety Fitness Determination will likely be based on absolute performance)

• Based on each carrier’s BASIC measure, develop rank and percentile for each entity’s BASIC performance

Safety Data

BASIC Data

BASIC Measures Percentile

5

Page 6: Intro to SMS

U.S. Department of TransportationFederal Motor Carrier Safety Administration

Safety DataSafety Data

• Safety Data Attributed to Carrier

• Carrier Safety Measurement System (CSMS)– Includes 24 months of carrier on-road safety performance

• 6.6 Million inspections• 290 Thousand crashes• 690 Thousand carriers

Safety Data

BASIC Data

BASIC Measures Percentile

6

Page 7: Intro to SMS

U.S. Department of TransportationFederal Motor Carrier Safety Administration

BASIC DataBASIC Data

• Safety Data Sorted by BASIC– Unsafe Driving (Parts 392 & 397)– Fatigued Driving (Hours-of-Service) (Parts 392 & 395)– Driver Fitness (Parts 383 & 391)– Controlled Substances/Alcohol (Parts 382 & 392)– Vehicle Maintenance (Parts 393 & 396)– Cargo-Related (Parts 392, 393, 397 & HM)– Crash Indicator

Safety Events

BASIC Data

BASIC Measures Percentile

7

Page 8: Intro to SMS

U.S. Department of TransportationFederal Motor Carrier Safety Administration

BASIC MeasuresBASIC Measures

• Convert BASIC Data into Quantifiable Measure

• Factors used in calculating a BASIC measure:– Time Weighting/Time Frame - More recent events more relevant– Severity Weightings – Increase weighting of violations that have been

shown to create a greater risk of crash involvement– Normalizing – Based on exposure; use of number of inspections and

power units– Violation Cap –Cited section number only counts once per inspection– BASIC Severity cap –Limits the severity weight applied to a BASIC

measure from a single poor inspection.

Safety Events

BASIC Data

BASIC Measures Percentile

8

Page 9: Intro to SMS

U.S. Department of TransportationFederal Motor Carrier Safety Administration

Unsafe Driving MeasureUnsafe Driving Measure

• Operation of Commercial Motor Vehicles (CMVs) in a dangerous or careless manner– Examples – speeding, reckless driving, improper lane change

• Factors used in calculating the measure:– Time Weight – 0-6 Months (3), 6-12 Months (2), 12-24 Months (1)– Violation Severity Weight

• Based on crash risk – Range from 1-10, where 10 is the most severe– Normalized by Average Power Units

(PUs) UnitsPowerofNumber Average

ViolationsWeightedSeverity&TimeofSumMeasure BASIC

9

Page 10: Intro to SMS

U.S. Department of TransportationFederal Motor Carrier Safety Administration

Fatigued Driving (HOS) MeasureFatigued Driving (HOS) Measure

• Operation of CMVs by drivers ill, fatigued or in non-compliance with the hours-of-service (HOS) regulations– Examples: HOS, logbook, and operating CMV while ill or fatigued

• Factors used in calculating the measure:– Time Weight - 0-6 Months (3), 6-12 Months (2), 12-24 Months (1)– Violation Severity Weight

• Based on crash risk – Range from 1-10, where 10 is the most severe• OOS (+2)

– Normalized by Relevant Inspections – Levels 1,2,3,6 and any other inspections resulting in related violations

sInspectionRelevantWeightedTimeofNumber

ViolationsWeightedSeverity&TimeofSumMeasure BASIC

10

Page 11: Intro to SMS

U.S. Department of TransportationFederal Motor Carrier Safety Administration

Driver Fitness MeasureDriver Fitness Measure

• Operation of CMVs by drivers who are unfit to operate a CMV due to lack of training, experience or medical qualifications– Failure to have a valid and appropriate commercial driver’s license,

being medically unqualified to operate a CMV.

• Factors used in calculating the measure :– Time Weight – 0-6 Months (3), 6-12 Months (2), 12-24 Months (1)– Violation Severity Weight

• Based on crash risk – Range from 1-10, where 10 is the most severe• OOS (+2)

– Normalized by Relevant Inspections – Levels 1,2,3,6 and any other inspections resulting in related violations

sInspectionRelevantWeightedTimeofNumber

ViolationsWeightedSeverity&TimeofSum MeasureBASIC

11

Page 12: Intro to SMS

U.S. Department of TransportationFederal Motor Carrier Safety Administration

Controlled Substances/Alcohol MeasureControlled Substances/Alcohol Measure

• Operation of CMVs by drivers who are impaired due to alcohol, illegal drugs, and misuse of prescription or over-the-counter medications– Examples: Use or possession of controlled substances or alcohol

• Factors used in calculating the measure:– Time Weight – 0-6 Months (3), 6-12 Months (2), 12-24 Months (1)– Violation Severity Weight

• Based on crash risk – Range from 1-10, where 10 is the most severe– Normalized by Average Power Units

(PUs) UnitsPowerofNumber Average

ViolationsWeightedSeverity&TimeofSum MeasureBASIC

12

Page 13: Intro to SMS

U.S. Department of TransportationFederal Motor Carrier Safety Administration

Vehicle Maintenance MeasureVehicle Maintenance Measure

• Failure to properly maintain a CMV– Examples: brakes, lights, and other mechanical defects, and failure to

make required changes

• Factors used in calculating the measure:– Time Weight – 0-6 Months (3), 6-12 Months (2), 12-24 Months (1)– Violation Severity Weight

• Based on crash risk – Range from 1-10, where 10 is the most severe• OOS (+2)

– Normalized by Relevant Inspections – Levels 1,2,5,6 and any other inspections resulting in related violations

sInspectionRelevantWeightedTimeofNumber

ViolationsWeightedSeverity&TimeofSumMeasure BASIC

13

Page 14: Intro to SMS

U.S. Department of TransportationFederal Motor Carrier Safety Administration

Cargo-Related MeasureCargo-Related Measure

• Failure to properly prevent shifting loads, spilled or dropped cargo, and unsafe handling of hazardous materials on a CMV– Examples: improper load securement, cargo retention, and hazardous

material handling

• Factors used in calculating the measure:– Time Weight – 0-6 Months (3), 6-12 Months (2), 12-24 Months (1)– Violation Severity Weight

• Based on crash risk – Range from 1-10, where 10 is the most severe• OOS (+2)

– Normalized by Relevant Inspections – Levels 1,2,5,6 and any other inspections resulting in related violation

sInspectionRelevantWeightedTimeofNumber

ViolationsWeightedSeverity&TimeofSum MeasureBASIC

14

Page 15: Intro to SMS

U.S. Department of TransportationFederal Motor Carrier Safety Administration

Crash MeasureCrash Measure

• Histories or patterns of high crash involvement, including frequency and severity– Based on state-reported crash records

• Factors used in calculating the measure:– Time Weight – 0-6 Months (3), 6-12 Months (2), 12-24 Months (1)– Crash Severity Weight

• Range from 1-3 – Crashes involving injury/fatality or HM release have more weight

– Normalized by Average Power Units

(PUs) UnitsPowerofNumber Average

Crashes WeightedSeverity/TimeofSum Measure BASICCrash

15

Page 16: Intro to SMS

U.S. Department of TransportationFederal Motor Carrier Safety Administration

PercentilePercentile

• Based on each BASIC measure, develop percentile indicating carrier’s BASIC performance– Provides a relative assessment of performance– Allows for prioritizing intervention resources by behavior

• Considerations:– Peer Grouping – compare measures of entities with similar levels of exposure– Data Sufficiency standards – define events/exposure necessary to generate a

robust measure– SFD/Intervention standards – define “critical mass” of poor performance

necessary for inclusion of entity in intervention process or detrimental SFD– Current Inspection and Crash Data – assignment of percentile dependent on age

and result of most recent inspection (12 months)

Safety Events

BASIC Data

BASIC Measures Percentile

16

Page 17: Intro to SMS

U.S. Department of TransportationFederal Motor Carrier Safety Administration

Peer GroupingPeer Grouping

Create percentile based on measure for carrier with similar exposure (same peer group)

*PU = Power Unit

Unsafe Driving Controlled Substances/Alcohol Crash

Fatigued Driving (HOS) Driver Fitness Vehicle MaintenanceCargo-Related

1 0 < PU<= 5 5 – 10 Inspections; (3-10 Fatigued)2 5 < PU <= 15 11 – 20 Inspections3 15 < PU <= 50 21 – 100 Inspections4 50 < PU <= 500 101 – 500 Inspections5 500 < PU 501+ Inspections

BASICs

Peer Group

17

Page 18: Intro to SMS

U.S. Department of TransportationFederal Motor Carrier Safety Administration

Data SufficiencyData Sufficiency

• Minimum number of inspections with applicable violations required for percentile to be assigned

• Assists in identification of patterns of carrier behavior – note safety problems across multiple inspections

BASICNumber of Inspections

Unsafe Driving 3Fatigued Driving (HOS) 3

Driver Fitness 5Controlled Substances / Alcohol 1

Vehicle Maintenance 5Cargo-Related 5

Crash 2 Crashes

18

Page 19: Intro to SMS

U.S. Department of TransportationFederal Motor Carrier Safety Administration

BASIC percentiles trigger InterventionsBASIC percentiles trigger Interventions

• Carriers that meet data sufficiency are assigned a percentile– Ex: Driver Fitness BASIC percentile of 85% means the carrier is worse

than 85% of its peers

• Carriers that exceed the BASIC threshold are identified for interventions

BASIC Passenger HMAll

OthersUnsafe Driving, Fatigued Driving (HOS), Crash

50% 67% 72%

Driver Fitness, Controlled Substances/ Alcohol, Vehicle Maintenance, Cargo-Related

55% 72% 77%

19

Page 20: Intro to SMS

U.S. Department of TransportationFederal Motor Carrier Safety Administration

Example of SafeStat vs. SMSExample of SafeStat vs. SMS

Carrier under the Radar with Existing

SafeStat System

20

Page 21: Intro to SMS

U.S. Department of TransportationFederal Motor Carrier Safety Administration

Carrier Measurement: SafeStat ResultsCarrier Measurement: SafeStat Results

21

Page 22: Intro to SMS

U.S. Department of TransportationFederal Motor Carrier Safety Administration

Carrier Measurement: SMS Results Carrier Measurement: SMS Results

22

Page 23: Intro to SMS

U.S. Department of TransportationFederal Motor Carrier Safety Administration

Violation Details Provided in SMSViolation Details Provided in SMS

23

Page 24: Intro to SMS

U.S. Department of TransportationFederal Motor Carrier Safety Administration

Further Drilldown in SMSFurther Drilldown in SMS

24

Page 25: Intro to SMS

U.S. Department of TransportationFederal Motor Carrier Safety Administration

How does a carrier improve and get out of the intervention process?How does a carrier improve and get out of the intervention process?

• “Good” Inspections

• “Get Well” Rules– Unsafe Driving and Controlled Substances/Alcohol

BASICs• No percentile assigned if no inspections with a violation in these

BASICs in the last year

– Crash Indicator• No percentile assigned if no crashes in last year

– Fatigued Driving (HOS), Driver Fitness, Vehicle Maintenance and Cargo-Related BASICs

• No percentile assigned if:– No inspections with a violation in that BASIC within the past year; and– Most recent relevant inspection does not have a violation of that BASIC

25

Page 26: Intro to SMS

U.S. Department of TransportationFederal Motor Carrier Safety Administration

Carrier Access to PercentilesCarrier Access to Percentiles

When will carriers’ SMS results be made available?– Currently, only test state carriers have access to SMS results

by using the Comprehensive Safety Information (CSI) system– FMCSA will provide all carriers an early review of their own

safety data by BASIC starting April 12, 2010– Non-test carriers’ SMS results will be available to carriers in

August• Public will have access to carrier SMS results in the winter of

2010

26

Page 27: Intro to SMS

U.S. Department of TransportationFederal Motor Carrier Safety Administration

New Agency Plans for DriversNew Agency Plans for Drivers

• The new measurement system provides an internal tool to address CMV drivers:– Provides enhanced information on individual drivers to

investigators to identify drivers with safety problems– Allows for prioritizing driver sampling during carrier

investigation– Supports investigator follow up on serious violations

• Under CSA 2010, individual drivers will not be assigned safety ratings or safety fitness determinations

27

Page 28: Intro to SMS

U.S. Department of TransportationFederal Motor Carrier Safety Administration

New Agency Plans for Drivers (cont’d)New Agency Plans for Drivers (cont’d)

• Other Agency initiatives are underway, including the Pre-employment Screening Program (PSP)– PSP was mandated by Congress and is not a part of

CSA 2010– “Driver Profiles” from FMCSA’s Driver Information

Resource (DIR) will be available to carriers through PSP

– Driver Profiles will only be released with driver authorization

– PSP is under development, more information can be found at www.psp.fmcsa.dot.gov

28

Page 29: Intro to SMS

U.S. Department of TransportationFederal Motor Carrier Safety Administration

Roadside Data Uniformity Roadside Data Uniformity

• Data collected at the roadside is the foundation of all data driven traffic safety initiatives

• CSA 2010 relies on roadside data in its SMS Methodology

• The CSA 2010 SFD methodology would use roadside data as a component of safety fitness determinations

29

Page 30: Intro to SMS

U.S. Department of TransportationFederal Motor Carrier Safety Administration

Roadside Uniformity-Background Roadside Uniformity-Background

Effort organized into four core initiatives:

1. Consistent documentation of roadside inspection and violation data

2. Standardized processes for challenging data

3. Increased awareness of high level goals of the inspection program

– Good inspections can support systematic enforcement program– Screening vs. Inspection

4. Uniform inspection selection processes

30

Page 31: Intro to SMS

U.S. Department of TransportationFederal Motor Carrier Safety Administration

SMS and Safety Fitness DeterminationSMS and Safety Fitness Determination

SFD would:– Incorporate on-road safety performance via new SMS

which updates on a monthly basis– Continue to include major safety violations found as

part of CSA 2010 investigations– Produce a Safety Fitness Determination (SFD) of

• Unfit or• Marginal or• Continue Operation

31

Draft rulemaking is currently in review within DOT;

NPRM expected to be published late 2010.

Draft rulemaking is currently in review within DOT;

NPRM expected to be published late 2010.

Page 32: Intro to SMS

U.S. Department of TransportationFederal Motor Carrier Safety Administration

Safety Measurement System vs. SafeStatSafety Measurement System vs. SafeStat

Today’s Measurement System: SafeStat CSA 2010’s SMS

Organized by four broad categories - Safety Evaluation Areas (SEAs): Accident, Driver, Vehicle, and Safety Management

Organized by seven specific Behavior Analysis and Safety Improvement Categories (BASICs)

Identifies carrier for a compliance review (CR) Identifies carriers for an investigation and where to focus the investigation

Uses only out-of-service (OOS) and moving violations from roadside inspections.

Uses all safety-based road-side inspection violations from roadside inspections.

No impact on safety rating Used to propose adverse safety fitness determination based on carriers’ current on-road safety performance (future)

Violations are not weighted based on relationship to crash risk

Violations are weighted based on relationship to crash risk

Assesses carriers only Assesses carriers and drivers – the driver SMS is a tool for investigators to identify drivers with safety problems during carrier investigations

32

Page 33: Intro to SMS

U.S. Department of TransportationFederal Motor Carrier Safety Administration

What Can Carriers Do To Prepare Now?What Can Carriers Do To Prepare Now?

• Educate Yourselves and Your Employees:– Understand the SMS Methodology and the BASICs– Check the website for information and updates (http://csa2010.fmcsa.dot.gov)– Raise awareness that every inspection counts and every violation counts

• Check and update records:– Motor Carrier Census Form (MCS -150)– Routinely monitor and review inspection and crash data– Question potentially incorrect data (DataQs: https://dataqs.fmcsa.dot.gov)

• Ensure compliance: – Review inspections and violation history over the past 2 years – Address safety problems now– Educate drivers about how their performance impacts their own driving record

and the safety assessment of the carrier

33

Page 34: Intro to SMS

U.S. Department of TransportationFederal Motor Carrier Safety Administration

For more information, visit csa2010.fmcsa.dot.gov

For more information, visit csa2010.fmcsa.dot.gov