55

The Umbra Simulation And Integration Framework AppliedTo Emergency Response Training

Paul Lawrence Hamilton and Robert BrittainORION International Technologies, Inc., Albuquerque, NM, USA

phamilton@orionint.com

Abstract. The Mine Emergency Response Interactive Training Simulation (MERITS) is intended to preparepersonnel to manage an emergency in an underground coal mine. The creation of an effective trainingenvironment required realistic emergent behavior in response to simulation events and trainee interventions,exploratory modification of miner behavior rules, realistic physics, and incorporation of legacy code. It alsorequired the ability to add rich media to the simulation without conflicting with normal desktop securitysettings. Our Umbra Simulation and Integration Framework facilitated agent-based modeling of miners andrescuers and made it possible to work with subject matter experts to quickly adjust behavior through scriptediting, rather than through lengthy programming and recompilation. Integration of Umbra code with theWebKit browser engine allowed the use of JavaScript-enabled local web pages for media support. Thisproject greatly extended the capabilities of Umbra in support of training simulations and has implications forsimulations that combine human behavior, physics, and rich media.

1.0 INTRODUCTION

1.1 Problem and Significance

On January 2, 2006, there was an explosion at theSago mine in Upshur County, West Virginia.Twelve miners were trapped. Eleven miners died.According to one mine rescue expert, in theory,they could have all walked out. The national outcrywas overwhelming as the Sago story dominated allmedia for over a week. The Sago mine disasterdemonstrated that mine accidents today have apotential for political and economic consequencesfar beyond their direct cost in money and lives.

Furthermore, although the US has the lowestfatality rate per ton produced of any major coal­produ"cing nation, there are several factors thathave the potential to make mining less safe in thefuture. The first is that much of the undergroundcoal mining workforce is approaching retirement. [1]As they retire, the training and work habits thathelped build this safety record will retire with them.The second is that underground miners must workunder more challenging conditions as productionexpands. Closed mines have been reopened, likeSago, as the demand for coal has increased.

Coming after a half-year of dramatic increases inpetroleum prices and rising concerns about energyand national security, the Sago disaster focused thenation's attention upon coal mining issues. Withinweeks, the West Virginia Legislature passedsweeping new regulations affecting all undergroundcoal mines in West Virginia and mandating a

number of new technologies for messaging andminer tracking. Dr. R. Larry Grayson, chair of therecently formed National Mining Association (NMA)Committee on Mine Safety, points out thelimitations of this approach. "What miners reallyneed", he says," is a comprehensive system withscenario-based training and multiple options forsurvival"[2]

2.0 METHODS

2.1 Current Methods

"Table top" exercises are a common way to meetthis need. People who could be in the commandcenter role play the events of an emergency, oftenwhile sitting around a table. These exercises do agood job of helping participants visualize some ofthe implications of emergency procedures.However, they also miss much of the complexityinherent in mine emergencies and lack the impactthat builds learning. Because only managers areinvolved in these exercises, they do not help minersunderstand how their actions must harmonize withthose of mine management in order to mitigate theconsequences of an emergency. "Mocks" providemore realistic training situations that do not havethese shortcomings. Mocks are large-scale drillscarried out at the mine itself or at a training andresearch facility. Not only does everyone involvedwith the mine participate, but in some cases,organizations from the nearby town even play rolesin the simulation. Police, fire, and emergency

https://ntrs.nasa.gov/search.jsp?R=20100012867 2020-04-10T02:29:39+00:00Z

56

medical personnel all benefit from the opportunity towork with mine personnel. The mocks are realisticand inclusive, but they interfere with production andmay not be practical at smaller mines.

2.2 Computer Simulation

Computer simulation has the potential to expandthe realism of a "table top" exercise. Just assimulation has come to play a significant role inenhancing aviation safety, one would expect similarbenefits in the field of mine safety. The MineEmergency Response Interactive TrainingSimulation (MERITS), developed by the NationalInstitute for Occupational Safety and Health(NIOSH) is a notable development [3]. In testimonybefore the U.S. Senate Appropriations Subcommit­tee on labor, Health and Human Services, andEducation, David E. Hess, Secretary of the Penn­sylvania Department of Environmental Protection,stated that training, including MERITS, was a keyfactor in the successful rescue after a majoraccident at Quecreek, Pennsylvania. Hess testifiedthat "the rescued miners have said the safetytraining they received helped them in several ways,first to warn the other miners to leave the rapidlyflooding mine, how to share resources and protectthemselves underground and to understand whatrescuers above ground would be doing to rescuethem."

MERITS communicates events stemming from asimulated mine accident to a class through voicesynthesis. The class may give instructions tominers and rescuers and order needed supplies.The class must piece together the nature of theemergency from the voices and execute asuccessful rescue of two trapped miners during amine fire. The class can draw upon a multimedialibrary of information about the mine, including apersonnel roster, emergency plan, and a detailedmap of the mine and its contents. Althoughdesigned for supervisors who would direct a rescueoperation, MERITS clearly gave these miners auseful understanding of the role their individualactions had to play in the rescue process.

Unfortunately, MERITS was designed to run on thecomputer systems of 10 years ago and is no longercompatible with current standard desktop securitysettings. It is a single scenario exercise that wouldrequire rewriting and recompiling portions of thecode in order to change the scenario or reflectchanges in Federal regUlations. Furthermore, muchof the code is written in MODSIM [4], a simulationlanguage that is no longer widely used. Weproposed to create a modernized, improvedMERITS 2 to rectify these limitations.

2.3 Umbra Integration and SimulationFramework

The Umbra Simulation and Integration Framework,was a promising tool for a number of reasons. Itallows program function to be easily changed, evenduring system execution. This is because, in anUmbra simulation, C++ modules call one anotherthrough easily edited Tcl scripts [4]; [5]. In Umbra,it is possible to "fine tune" behaviors without havingto pause to recompile code. Standard interfacesfacilitate the rapid construction of complexsimulations. Scalability is enhanced through theuse of the Worlds concept, by which eachsubprogram in the simulation is "aware" only of theinformation that the real entity would have accessto.

Umbra was originally conceived at Sandia NationalLaboratories for the development of swarming andother cooperative behaviors and tactics for robots,including Unmanned Aerial Systems (UAS).Consequently, it lends itself to agent-basedsimulation and has a large library of routines thatcan be used for either embodied or non-embodiedagents. However, it has not previously been appliedto a training situation for a number of reasons. Oneis that there was no provision for integration withrich media, such as audio, video, and htmldocuments. Another is that the lack of dockablewindows made it impossible to cleanly pagebetween simulation output and web pages. Finally,in the proposed mine rescue simulation, thecomputer would have to play the role of the minetelephone, communicating with the class throughvoice synthesis, a capability that had not beenpreviously incorporated into Umbra.

2.4 Extending Umbra

The first requirement was to integrate themultiplatform Graphical User Interface toolkit, Qt 4.5[8], into Umbra. By doing so, we would be able tonot only quickly build effective user interfaces, butalso to add the WebKit open source browserengine, the same code that powers Apple's Safaribrowser. WebKit, unlike Explorer, provides asecure way to communicate between local andremote web content and local SQl databases usingHTML5 database support. This would make itpossible to replicate the media functionality of theoriginal MERITS while complying with currentsecurity standards for networked desktops, thussolving one of the most important problemsassociated with the older version of MERITS. Inorder to use Qt 4.5, it was first necessary torecompile Umbra using VisualStudio 2008. The C­Space Toolkit [9], which provides detection of

57

vi

... '

_C.....-.cf

Figure 1: Command Menu

lord. ",.... Yarcben-.Ch.td<,-_Orterton, Jon, Gener" Moe ForI!lnoW'lPm, 81,~ OperatorReed, Dan, Freboss_.W_. (,...,)5«'Iders, Tal'I"t'ny, Mne Clerkstuart, Mery, Rod Bob, OpetItor~, .lin, Rod 801:. Operatot

r-·Go-Oll.-C.<>oerat"Tn.ocIle._._CJo<I.

lhoerson, RIch. Sc.cdy Mon

__chedc

Figure 1 shows how the menu selects telling aminer to pass on a commant to make a gas checkat a specified location and send the results back bymine phone.

Rescuer agents are miner agents that are scriptedto establish a Fresh Air Base for communicationsand enter the mine to find stranded miners usingapproved procedures. A third class of agents is themantrip. These are trolley-like vehicles that travelin and out of the mine on command, carryingminers and supplies.

.t'I'(IIIIIIII"I"""l1ll1d ? ~

Check_ <hedc"'-

I..-1"....-'- --=-9 I... k....-....-

_.~

As"",ERPdty"'lWldnrtfy"'''..-c>Ioyeeso..ckao..Enter a freeform C'Cll'l'll'I'II-Move cbjed(s)-­<loon"'do<_"'do< .........

2.7 Path Planner

We used Umbra's existing RoadPathPlanner class,which implements the GBS, or graph-based search,algorithm. This algorithm was created to plan apath using city streets on irregular terrain.However, we found that appropriate reformatting ofthe files describing mine geometry allowed the codeto simultaneously plan paths using both theunderground tunnels and surface road. Also, theability to randomize the speed taken on a givenroute is very useful for naturally separating minerson the same route - otherwise they would all be ontop of each other and it would be impossible todetermine who was there. It also prevents multiple"arrival" events from occurring simultaneously. Inthe event of a group evacuating the mine, the miner

• Check (smoke, gasses, status)• Close (door, regulator, etc.)

• Open (door, regulator, etc.)• Evacuate (section or mine)• Monitor (phone, fan, etc.)• Stop monitoring (phone, fan, etc.)• Move object(s)• Move people (includes self)• Pick up phone• Turn on (belts, etc.)• Shut off/turn off (belts, etc.)

geometric intersection of objects and plays a keyrole in path planning and the OpenSceneGraphoutput graphical libraries also had to be recompiled.

The existing libraries that were implemented intoMERITS were the graphics interface for the visualelements, path planning, route placementinterpolators and related generic utilities, collisiondetection, time-event generation, and eventhandling. New libraries generated were the Qtinterface; GUI design, layout, and implementation;GUI event generation and handling; miner objectbehaviors; the voice interfaces; the wrapper for theexisting NIOSH fire effects algorithm (GFire); andcode to query GFire data based on position andobject type.

2.5 Voice Synthesis

Because the current Umbra libraries had noprovision for voice synthesis, it was necessary toadd this capability. However, one of our constraintswas that the resulting training application had to bedistributable to noncommercial users without royaltypayments. Consequently, we chose to use the opensource code eSpeak [6]. However, the quality ofthe voices generated by this application wassufficiently low as to limit intelligibility.Consequently, we used phonemes generated byMbrola [7], developed by the TCTS Lab of theFaculte Polytechnique de Mons (Belgium). Mbrolaperforms concatenation of diphones. Starting with afile of phonemes, durations, and a piecewise lineardescription of pitch, it uses a phoneme database toproduce synthetic speech. We configured eSpeakto create appropriate phoneme files for Mbrola toconvert to .wav output.

2.6 Agent-Based Simulation

Miners and rescuers were represented byautonomous agents based on preexisting robotsimulation codes. We programmed these agents tocarry out the following commands:

58

agents can be instructed to travel together at thespeed of the slowest member of the group.

3.0 RESULTS

3.1 Extending Umbra

The combination of Qt and WebKit worked well,allowing the full integration of local and remote webmedia into the Umbra environment. JavaScriptcode not only made dynamic web pages possible,but also allowed the incorporation of persistent datastorage. Simulation variables could be easilytransferred to and from web pages, enhancingrealism. One is having each miner tag in and out ofthe mine on a simulated tag board visible to thetrainees. The persistent data storage capabilityallowed users to insert, edit, and delete noteslabeled with the simulation time into the record ofthe session.

Figure 2 shows a JavaScript-enabled graphic thatsimulates a fan pressure chart. It is refreshed everyminute from the simulation. As the simulationprogresses, there is a roof collapse that isaccompanied with a sudden pressure spike at theintake fan. The trainees can quickly page betweena map of the mine and the fan pressure chart asthey try to interpret the available data to determinewhat has happened.

Figure 2: Web Image of Fan Chart

3.2 Voice Synthesis

The combination of eSpeak and Mbrola producedvoices that were highly intelligible. Depending uponthe text supplied to the synthesis code, the resulting

voices vary in naturalness, primarily because thereis no way to code syllable stress. Nevertheless, thevoices were judged suitable for this application. Forcommercial applications, proprietary products suchas AT&T Natural Voices could greatly enhancerealism as they overcome the limitations ofdiaphone concatenation [11].

One problem we encountered was that the voiceshad to run in the same thread as the simulation.This led to the voices breaking up under somecircumstances. The MERITS 2 system employs astream of background "chatter" to simulate theeffect of typical conversations over the mine phonesystem. When an event-driven simulation responseproduced speech during the playing of "chatter", theresult was two voices speaking at the same time.In most circumstances, the voices simply playedtogether. However, there were times when one ofthe voices was blocked and replayed, causing anunnatural "echo" effect. Both of these problemswere mitigated by allowing the Umbra process touse more of the CPU. They might well persist in amore graphics-intensive environment thanMERITS 2, however, and it would be important toinvestigate ways of multithreading such simulations.

3.3 Agent-based Training

We demonstrated that by equipping miner, mantrip,and rescuer agents with a relatively small numberof scripted behaviors that could be elicited by eithersimulation events or user commands, we couldbuild a flexible simulation in which scenarios couldbe easily modified by editing Tel scripts. We plan toextend the behavior by adding some more rulessuch as detection and reporting of smoke and fire,mortality if unprotected in a toxic atmosphere, andincorporation of fire, smoke, and gas concentrationsinto the path planner. We anticipate that theseadditions will further increase the training value ofMERITS scenarios by fostering more complexemergent behavior for the trainees to deal with.

In Figure 3, three miner agents have noticed smokein the passageways. The routine for the calculationof smoke density and gas concentrations is anadaptation of one developed by NIOSH. Thisexample is from a developmental run and showshow a legacy program can be incorporated into anUmbra simulation. During a MERITS 2 exercise,the trainees are not able to see this view of theminers and smoke except when the instructorreplays the exercise with "god mode" turned on.However, the trainees can call the miners on thenearest mine phone and ask them to report backairflow, smoke, and gas data.

59

more general problem of workplace and other civilaccidents and emergencies. These situationsinclude firefighting, ship damage control, andresponse to acts of terrorism. Such trainingsimulations must incorporate accurate physics towork at all yet also deal with fragmentaryknowledge of the situation, a potential for complexemergent behavior leading to unexpectedconsequences, and even a lack of definition as towhat the objective should be (for example, shouldone try a risky plan to rescue all or a safer one thatwill only rescue some). These problems fall on acontinuum stretching from linear "tame" problems toill-defined "wicked" ones. Umbra, with its flexibilityand low computational overhead, has beeneffective even at the "wicked" end of this continuum,and is now suitable in a training context.

5.0 REFERENCES

Figure 3: Miners Discover Smoke and Fire

3.4 Path planner

Once the C-Space Toolkit was incorporated intoMERITS 2, it was found to work well. The originalMERITS had scripted miners who followed aspecified path regardless of whether or not a realminer could use it. With MERITS 2, miner andrescuer agents were able to discover a routebetween locations either on the surface or insidethe mine whenever such a path existed. In fact, itwas discovered that one of the work locationsspecified in the original MERITS was inappropriatewhen the path planner determined that there wasno exit from it.

4.0 DISCUSSION AND CONCLUSIONS

As a result of this work, Umbra became much moreuseful in a scenario-based training environment.The ability to modify agent actions through the useof Tel scripts turned out to be useful in therefinement of miner and rescuer behavior. Theintegration of dynamic web pages and audio intothe Umbra simulation made for a more immersivetraining experience while also allowing users toresearch topics relevant to a better understandingof the simulated mine emergency. The potentialuses of the Qt 4.5 integration include theincorporation of event-triggered video into Umbrasimulations and the triggering of simulation actionsfrom HTML links.

In a broader sense, our work with MERITS 2 showshow the benefits of simulator training, longrecognized in aviation, can be extended into avariety of industrial environments to address the

2

3

4

5

Fotta, B., and Bockosh, G., "The AgingWorkforce: An Emerging Issue in the MiningIndustry". Proceedings of the Thirty-FirstAnnual Institute of Mining Health, Safety, andResearch, 28-30 Aug 2000, VirginiaPolytechnic Institute and State University,Department of Mining and MineralsEngineering, Blacksburg, VA, 2000, 16 Mar2006,http://www.cdc.gov/niosh/mining/pubs/pdfs/tawae.pdf.

Smith, V, "Technology may not offer quickfixes to coal mine dangers", TMC NET, 24Feb 2006, 16 Mar 2006http://news.tmcnet.com/news/2006/02/24/1405329.htm.

Reinke, Dana, Launa Mallett, Michael J.Brnich, and Charles Vaught, "MERITSVolume 2: Trainer's Guide (Draft)", 2002, 16Mar 2006,http://merits.niosh.cdc.gov/merits/misc files/MERITS%20Trainer's%20Guide.pdf.

Mullarney, A. and J. West. (1987) Mullarney,A. and J. West. "ModSim: A Language forObject-Oriented Simulation. DesignSpecification." ACI Products, La Jolla,California. 1987

Gottlieb, E. J, R. Harrigan, M. McDonald, F.Oppel., and P. Xavier (2001) "The UmbraSimulation Framework," SAND2001-1533,

60

Sandia National Laboratories, Albuquerque,NM

6 Gottlieb, E. J., M. J. McDonald, F. J. Oppel, J.B. Rigdon, and P. G. Xavier (2002), "TheUmbra Simulation Framework as Applied toBuilding HLA Federates," Informs.

7 Sourceforge, eSpeak text to speechhttp://espeak.sourceforge.netfdocindex.htmI

8 MBROLA Project Development TeamFaculte Polytechnique de Mons, MULTITEL-

TCTS Lab,Initialis Scientific Park, 1, Copernic Ave, B­7000 Mons, Belgium.http://tcts.fpms.ac.be/synthesis/mbrola.htmI

9 Nokia CorporationQt - A cross-platform application and UI

frameworkhttp://www.qtsoftware.com/

10 Xavier, P.G., "Fast swept-volume distance forrobust collision detection", (1997) IEEEInternational Conference on Robotics andAutomation, 1997. Proceedings., Volume 2,Issue, 20-25 Apr 1997 Page(s):1162 - 1169vol.2 Digital Object Identifier10.1109/ROBOT.1997.614295

11 AT&T (2002) "AT&T Natural Voices Text-To­Speech Engines System Developer's GuideServer, Server-Lite, and Desktop Editions USEnglish, Latin American Spanish, UK English,German, and French, Release 1.4"http://www.naturalvoices.att.com