Hybrid OMS in Hurricane Ike
Jeff Lane
Presentation Goals
• About TVEC• Dispatch center setup• Our typical storm• Hurricane IKE response structure• Questions
About Trinity Valley
• Located 30 miles south east of Dallas Texas• 61,000 meters• 6,900 miles of line• 151 employees• SEDC Billing system• DisSPatch OMS (using enhanced Crew Management)• ESRI/WindMilMap• Porche and Twenty First Century Communications IVR• Wireless Matrix AVL integrated with DisSPatch
Service Territory
Dispatch Center Layout
Dispatch Center
Typical “Major” Storm• 50 to 75 concurrent outages affecting 5000 to
7000 meters• 40 troubleshooting/repair personnel• 1 to 2 contract construction and/or ROW crews• 2 Dispatchers• 1 to 2 Supervisors• 1 to 2 Dispatch helpers• Restoration time 10 to 15 hours
Typical Dispatch Structure
Incident Commander
Athens District
Kaufman District
Dispatcher
Crew Coordinator
Dispatcher
Crew Coordinator
DispatchAssistant
20 – 30 Troubleshooting
Personnel
20 – 30 Troubleshooting
Personnel
District line
Hurricane IKE• Approximately 200 concurrent outages (total of 269)• 14,000 out at height of storm (total affected 20,141)• Damage:
– 19 broken poles– 14 damaged transformers– 93 lines down
• 66 troubleshooting/repair personnel (16 from other coops and contractors)
• 6 contract construction crews• 10 contract ROW crews• Had 20 Dispatchers on Saturday, dropped down to 17 on
Sunday• Restoration time 42 hours
Hurricane IKE Event structure
Incident Commander Jeff Lane
Operations Section Chief
Tony Watson
Planning Section Chief
Tim Craig
Logistics Section Chief
Greg Starek
Accounting Section Chief
Frank Skube
Kaufman/Tawakoni Branch
Sam Grimes
Safety Officer Danny Belcher
Public InformationOfficer
Jerry Boze
Support Section Chief
Jerry Woolston
Liason Officer
Customer Service Group
Shirley Murphree
IT Group Jerry Woolston
Mapping
Engineering
Staking
HR
Facilities
Dispatch BranchChase SnufferJerry Williams
Incident Command SystemEvent Name: Hurricane Ike
Event Date: 9/13/2008
East BranchJohn Jackson
West/North BranchChris Durden
South BranchJimmy Splawn
Cedar Creek Branch
Rodney Wesley
Canton Branch Gene
Operations SectionOperations
ChiefTony
Canton/CCRKBranch
Sam
CombineBranchRodney
West/NorthBranch
Chris(14)
EastBranch
Jimmy(13)
SouthBranch
John(14)
Walton DivisionLineman
Cedar DivisionLineman
Bartons ChapelDivisionLineman
WollowSpringsDivisionLineman
AntiochDivisionLineman
New YorkDivisionLineman
CantonDivisionLineman
CoffeeDivisionLineman
Massey LakeDivisionLineman
Carroll SpringsDivisionLineman
MontablaDivisionLineman
TuckerDivisionLineman
MabankDivision
Paul
JacksonDivisionJamie
Glenn PinesDivision
Bo
ScurryDivisionLathem
KempDivision
Virgil
Tool/Seven PointsDivisionEddie
CombineDivisionKenneth
TaltyDivision
Raymond
MarshallDivisionLance
PattonDivision
Jay
RandDivisionFloyd
EnterpriseDivisionShayde
Cobb SwitchDivision
Rudy
Duck CoveDivisionLarry B.
ROW Division(5 Crews)Pete
ROW Division(5 Crews)
Paul
Contractor Division J&L
Jimmy Kirk
Contractor Division J&L
Jason
KaufmanDispatch
Jerry
Tawkoni BranchGene
ROW DivisionKelly Massey
Contractor Division J&L
Jimmy Ray
AthensDispatchChase
Contractor DivisionJ&L
George
ROW DivisionRamirez Tree
Service
Contractor DivisionJ&L
Chad
ROW DivisionRamirez Tree
Service
Contractor DivisionS&H
ROW Division Ramirez Tree
Service
OfficeField
District Dispatchers
• Dispatchers managed the OMS.– Evaluated and corrected predictions when possible– Assigned and printed outage tickets to appropriate branches
• Outage tickets included Name, Map #, Meter #, Address, Phone #, Outage #, and included all calls received for the outage.
– When outages were restored, the tickets were brought back to the Dispatcher to be restored in the OMS.
Operations ChiefTony
KaufmanDispatch
Jerry
AthensDispatchChase
Branches
• Branch Directors– Received outage tickets from Dispatcher and dispatched to appropriate Division leaders– Worked with Dispatcher to modify any incorrect outage predictions– Coordinated Construction/ROW contractor movement between Division Leaders within
their branch– Issued line clearances to Division Leaders– Record restoration date/time, cause, and equipment codes on outage tickets once
restored– Return outage tickets to Dispatcher for closing in the OMS
Operations ChiefTony
Canton/CCRKBranch
Sam
CombineBranchRodney
West/NorthBranch
Chris(14)
EastBranch
Jimmy(13)
SouthBranch
John(14)
KaufmanDispatch
Jerry
Tawkoni BranchGene
AthensDispatchChase
Divisions
• Division Leaders– Received outage information from Branch Directors via cell phone and radio– Coordinated personnel with their divisions to restore power– Communicated resource needs back to Branch Director (i.e., need more or less
crews)– Communicated restoration progress back to Branch Director
Operations ChiefTony
Canton/CCRKBranchSam
CombineBranchRodney
West/NorthBranch
Chris(14)
EastBranch
Jimmy(13)
SouthBranch
John(14)
Walton DivisionLineman
Cedar DivisionLineman
Bartons ChapelDivisionLineman
WollowSpringsDivisionLineman
AntiochDivisionLineman
New YorkDivisionLineman
CantonDivisionLineman
CoffeeDivisionLineman
Massey LakeDivisionLineman
Carroll SpringsDivisionLineman
MontablaDivisionLineman
TuckerDivisionLineman
MabankDivision
Paul
JacksonDivisionJamie
Glenn PinesDivision
Bo
ScurryDivisionLathem
KempDivision
Virgil
Tool/Seven PointsDivisionEddie
CombineDivisionKenneth
TaltyDivision
Raymond
MarshallDivisionLance
PattonDivision
Jay
RandDivisionFloyd
EnterpriseDivisionShayde
Cobb SwitchDivision
Rudy
Duck CoveDivisionLarry B.
ROW Division(5 Crews)Pete
ROW Division(5 Crews)
Paul
Contractor Division J&L
Jimmy Kirk
Contractor Division J&L
Jason
KaufmanDispatch
Jerry
Tawkoni BranchGene
ROW DivisionKelly Massey
Contractor Division J&L
Jimmy Ray
AthensDispatchChase
Contractor DivisionJ&L
George
ROW DivisionRamirez Tree
Service
Contractor DivisionJ&L
Chad
ROW DivisionRamirez Tree
Service
Contractor DivisionS&H
ROW Division Ramirez Tree
Service
How did it actually work???• After the event was over, we conducted post-event reviews with everyone involved,
from the CEO to field personnel. All of the feedback we received was positive.• Typical bottlenecks at the Dispatch level were avoided by dividing the workload into
manageable levels.• Even the linemen who were acting as “Division Leaders” and performing dispatch
functions for the personnel under them felt like we should continue to use this structure in future events.
• The normal Operations department supervisors were asking to begin using this structure more often (i.e., events that may not be as large as a hurricane but that are larger than a typical “major” thunderstorm).
• The normal Dispatchers loved it because they had a manageable workload.• Everyone felt like this provided a safer working environment because they weren’t
being overworked and that reduced the possibility for mistakes.• The general consensus was that restoration was achieved sooner than it would have
been if we had used the normal Dispatch structure.
Did we have issues???• Sure we did.• Some of the division leaders were holding onto restoration information until
they had several outages restored. Their reasoning was they wanted to report them all in one call back to the Branch Director instead of multiple calls. This caused issues in the office with accurate reporting to the PUC, Critical Account restoration status, and planning of the next day’s work.
• Outage re-predictions were a problem. The Dispatcher would give the Branch Director several outages, all on separate outage tickets, then a re-prediction would push those outages upline. We had to manage that paper with effective communications between the Dispatchers and the Branch Directors.
• Reallocating workload dynamically based on restoration status was challenging but worked with effective communication between Branch Directors and the Operations Section Chief.
Will this structure work to manage “The Big One”?
• With minor modifications, I believe it will. That’s just one opinion, though, and I haven’t been through “The Big One.” What is your opinion?
Top Related