You Can ’ t Manage What You Don ’ t Measure
description
Transcript of You Can ’ t Manage What You Don ’ t Measure
![Page 1: You Can ’ t Manage What You Don ’ t Measure](https://reader033.fdocuments.net/reader033/viewer/2022051418/56815839550346895dc5964f/html5/thumbnails/1.jpg)
Program Name or Ancillary Text eere.energy.gov
You Can’t Manage What You Don’t Measure
FAST Data Quality Issues: Your VMIS Can Help!
FedFleet 2012
Ron Stewart – INL FAST Web Team
Fleet Data Management Best Practices
![Page 2: You Can ’ t Manage What You Don ’ t Measure](https://reader033.fdocuments.net/reader033/viewer/2022051418/56815839550346895dc5964f/html5/thumbnails/2.jpg)
Federal Energy Management Program femp.energy.gov 2
• DOE, GSA, FAST team– always talking about it…– adding tools to find and fix issues– looking for good ideas on how solve this challenge
• What we see in FAST:– systems + tools + processes + policies + procedures + people– Your VMIS may be part of the problem…– … it definitely can be a big part of the solution!
FAST data quality: a real problem
![Page 3: You Can ’ t Manage What You Don ’ t Measure](https://reader033.fdocuments.net/reader033/viewer/2022051418/56815839550346895dc5964f/html5/thumbnails/3.jpg)
Federal Energy Management Program femp.energy.gov 3
• 2 capabilities your VMIS should provide• 2 strategies for minimizing FAST data quality problems
Today’s take-aways…
![Page 4: You Can ’ t Manage What You Don ’ t Measure](https://reader033.fdocuments.net/reader033/viewer/2022051418/56815839550346895dc5964f/html5/thumbnails/4.jpg)
Federal Energy Management Program femp.energy.gov 4
• Reviewed 45 agency FY2011 submissions– Agency “Data Quality & Consistency Report”– Prior-year OMB A-11 AMVFR– FAST’s “Data Call Summary” report– Basic aggregate fleet metrics:
• Average gross vehicle cost/mile
• Average gross vehicle GGE/mile
– Agency use of DQR late in the data call
Real examples…
![Page 5: You Can ’ t Manage What You Don ’ t Measure](https://reader033.fdocuments.net/reader033/viewer/2022051418/56815839550346895dc5964f/html5/thumbnails/5.jpg)
Federal Energy Management Program femp.energy.gov 5
• Every agency reviewed had at least one issue identified• 33 of 45 agencies flagged on AMVFR• 33 of 45 agencies flagged on DQR• 15 agencies flagged on Data Call Summary• 14 agencies had no/minimal use of DQR
Real examples…
![Page 6: You Can ’ t Manage What You Don ’ t Measure](https://reader033.fdocuments.net/reader033/viewer/2022051418/56815839550346895dc5964f/html5/thumbnails/6.jpg)
Federal Energy Management Program femp.energy.gov 6
• OMB A-11 AMVFR:– Monthly operating costs out of normal range– Acquisition costs out of range– Planned acquisitions w/o costs– Armored or LE vehicles w/o costs– Negative out-year inventory projections– Average age of owned vehicles out of range/negative– Missing one or more narrative sections
Real examples…
![Page 7: You Can ’ t Manage What You Don ’ t Measure](https://reader033.fdocuments.net/reader033/viewer/2022051418/56815839550346895dc5964f/html5/thumbnails/7.jpg)
Federal Energy Management Program femp.energy.gov 7
• Data Quality & Consistency Report– Significant swings in one metric w/o corresponding supporting
increases– Mismatches in data
• Inventory w/o costs
• Inventory w/o fuel
• Inventory w/o mileage
• Operational data w/o inventory
Real examples…
![Page 8: You Can ’ t Manage What You Don ’ t Measure](https://reader033.fdocuments.net/reader033/viewer/2022051418/56815839550346895dc5964f/html5/thumbnails/8.jpg)
Federal Energy Management Program femp.energy.gov 8
• Data Call Summary report– Significant agency swings relative to prior year
• Fuel cost & consumption
• Mileage
• Overall costs
• Inventory
– EISA Section 141 designation of acquisitions
Real examples…
![Page 9: You Can ’ t Manage What You Don ’ t Measure](https://reader033.fdocuments.net/reader033/viewer/2022051418/56815839550346895dc5964f/html5/thumbnails/9.jpg)
Federal Energy Management Program femp.energy.gov 9
• “Gross” … as in visible in agency aggregates
• … but traceable back to– specific groups of vehicles or– specific slices of agency-submitted data
Gross errors
![Page 10: You Can ’ t Manage What You Don ’ t Measure](https://reader033.fdocuments.net/reader033/viewer/2022051418/56815839550346895dc5964f/html5/thumbnails/10.jpg)
Federal Energy Management Program femp.energy.gov 10
• Attack the problems where it makes the most sense…
How do we solve this?
VMIS(s) Reporting
FAST
![Page 11: You Can ’ t Manage What You Don ’ t Measure](https://reader033.fdocuments.net/reader033/viewer/2022051418/56815839550346895dc5964f/html5/thumbnails/11.jpg)
Federal Energy Management Program femp.energy.gov 11
• VMIS should characterize data similar to FAST– Vehicle types & classes– Fleet costs
• Difficult to compare VMIS to FAST w/o similar groupings
VMIS Capability #1
![Page 12: You Can ’ t Manage What You Don ’ t Measure](https://reader033.fdocuments.net/reader033/viewer/2022051418/56815839550346895dc5964f/html5/thumbnails/12.jpg)
Federal Energy Management Program femp.energy.gov 12
• Compare VMIS to FAST– along every major data axis
• inventory, acquisitions, …
• fuel cost & consumption
• mileage
• fleet costs
– variety of reports and tools on the FAST side• DQR, AMVFR, query tool
• Investigate, understand, resolve major discrepancies
Strategy #1: Compare, compare, …
![Page 13: You Can ’ t Manage What You Don ’ t Measure](https://reader033.fdocuments.net/reader033/viewer/2022051418/56815839550346895dc5964f/html5/thumbnails/13.jpg)
Federal Energy Management Program femp.energy.gov 13
• Robust report generation & data querying– Ability to group data similar to FAST– Ability to identify vehicles with anomalies
• stale, incomplete, or inconsistent data?
• compare vehicles to their own history?
• compare vehicles to their peers?
VMIS Capability #2
![Page 14: You Can ’ t Manage What You Don ’ t Measure](https://reader033.fdocuments.net/reader033/viewer/2022051418/56815839550346895dc5964f/html5/thumbnails/14.jpg)
Federal Energy Management Program femp.energy.gov 14
• … early in time:– Use VMIS through the year to make sure vehicle-level data
quality issues are identified and resolved as early as possible• Dashboards
• Data quality & consistency threshold
• Monthly or quarterly data reviews
• … early in the process:– Fix the problems in the reporting process– Fix the problems in the VMIS(s)– FAST reporting
• … complete as early as possible
• … review as early as possible
Strategy #2: Attack the problem early
![Page 15: You Can ’ t Manage What You Don ’ t Measure](https://reader033.fdocuments.net/reader033/viewer/2022051418/56815839550346895dc5964f/html5/thumbnails/15.jpg)
Federal Energy Management Program femp.energy.gov 15
• Key VMIS Capabilities:– Characterize data the same way FAST does– Robust report generation and querying
• Strategies to improve data quality:– Compare, compare, compare– Attack the problem early
Summary