Energy-AwareDynamicData-DrivenApplicationSystemsforTransportationMonitoringandAnalysis

RichardFujimoto,Aradhya Biswas,MarkJackson,SaBra NealComputationalScience&EngineeringMichaelHunter,Angshuman GuinCivil&EnvironmentalEngineering

GeorgiaInstituteofTechnology

Outline

• MotivationandOverview• GreenRuntimeInfrastructure• EnergyEfficientSynchronizationofDistributedSimulations

• EnergyEfficientDataDistributionManagement

• OtherProgramElements

DDDASSystemOverview

• Roadside,in-vehiclesensorsproviderealtimedata• Predictionusingonlinedata-drivendistributedsimulationandanalytics• Adaptation:travelerguidance,transportationsystemmanagement• Applications

– Reducetravelerdelays,fuelconsumption,emissions– DoDapplicationtoadaptivesurveillancesystems

Roadside Server

In-Vehicle Simulations

SimulationsonHandheld

Devices

Sensors

Mobile platforms

DoDApplication:VehicleTrackingEachmobiledeviceincludes• Sensors• Predictivedatadriven

simulationandanalytics• Wirelesscommunication

Autonomousteamofmobilesensorsmonitoringvehiclesinanurbanenvironment

Sense:determinecurrentlocationoftarget vehicles

Adapt:repositionsensorsinlightofpredictedlocations

Predict:projectlikelyfuturelocationsandpotentialdestinations

DDDASProcessingLoop

Otherpotentialapplicationincludetrackingspreadofforestfires,cloudplumes

Micro-ClusterHardwareJetsonTK1

CPU ARMA15(32-bit, 2.3GHz,4cores+1

lowpowercore)

GPU 192coreKepler,326GF/s(peak)

Memory 2GBLPDDR3

Storage 16GBeMMC

Networking Ethernet

FormFactor Devboard

I/O USB,HDMI,Serial

Micro-ClusterServerHardware• Jetson TK1Boards(TX1underevaluation)• Tunablecore(10x),memory(3x)

frequencies• PowerMon2measurementsystemClientHardware• Quadcore QualcommMSM8974

SnapdragonProcessor(LGNexus5cellularphone)

Communications:802.11n

TransportationSimulationCalibration

CalibrationProcedure

Uncalibratedsimulation(default

parameters)

NGSim DataSet:PeachtreeStreet,AtlantaGA

MonteCarloruns:identifyparameters

Flowratetests

Statisticaltests

TransportationSystemModeling

• Models– CellularAutomata– QueueingNetwork

• SectioninmidtownAtlanta

• NGSim dataset0

10

20

30

40

50

60

70

80

0 1/20 1/10 3/20 1/5 1/4 E

nerg

y C

onsu

mpt

ion

(J)

Interarrival Rate (vehicle/sec)

Embedded SimulatorsEnergy Consumption

DES-Q Model CA Model

Historic Current(e.g.,last15minutes)

+ =(ω) (1-ω)

Bayesiansolution

Speedandaccelerationonalternateroutes

EnergyUseinData-driventrafficsimulation

BayesianPrediction

Outline

• MotivationandOverview• GreenRuntimeInfrastructure• EnergyEfficientSynchronizationofDistributedSimulations

• EnergyEfficientDataDistributionManagement

• OtherProgramElements

9

Network(Internet/Ad-

hoc)FastCGI

(mod_fcgi)

Server

GlobalSync

Module

PushMessageModule

MessageAggregatorModule

CppCMScontroller

• Green Runtime Infrastructure (G-RTI) middlewareØ Distributed simulation integration framework based on DoD High Level Architecture

standard (IEEE 1516) supporting DDDAS simulation, emulation, and deploymentØ Scalable designØ Flexible, supporting wide variety of devices, Internet of ThingsØ Energy-efficient implementation of key services

Ø Time Management (Synchronization)Ø Data Distribution Management (Communications)

Green Runtime Infrastructure Middleware

Example 1: DDDAS Deployment• Real-time sensor,

crowd-sourced data• Traveler assistance:

mobile apps, in-vehicle systems with predictive simulations, analytics customized for individual travelers;

• Back end systems (simulations, analytics, databases) for system-wide analysis

• Communications over wireless networks, Internet

• Runtime Infrastructure (RTI) provides services for interconnecting sensors, mobile devices, and back end computing facilities, databases

• Green RTI (G-RTI): RTI optimized to minimize energy consumption in mobile devices and other computing elements

Example2:DDDASEmulation

• Development of DDDAS systems in the laboratory• Mix of actual software/devices (e.g., mobile apps on cell phones,

databases) and simulated elements (e.g., networks, vehicles, sensors)

• Real-time interaction among system components

Example 3: DDDAS Simulation

• Early stage DDDAS development• Simulated sensors, back end system, mobile applications, vehicles,

networks• As-Fast-As-Possible execution (may not be real-time)

DistributedSimulationEnergyProfiles

• LGNexushardware• CMBsynchronization

algorithm• Queueingnetwork

benchmark• Varymeasurement

parameter(replications)

• Profilingtechniquesdevelopedtoseparateenergyconsumedbydifferentlayersandcomponents

• Energyconsumedbyeachlayercanbesignificant

Outline

• MotivationandOverview• GreenRuntimeInfrastructure• EnergyEfficientSynchronizationofDistributedSimulations

• EnergyEfficientDataDistributionManagement

• OtherProgramElements

TimeManagementServices

• Servicesusedtosynchronizedistributedsimulations

• Timemanagementservicesmustsupport– Differentlocaltimeadvancemechanisms(timestepped,eventdriven)

– Synchronizationapproaches• conservative(asynchronous,synchronous)• optimistic(rollback)

• Goal:developenergyefficientsynchronizationalgorithms

PowerConsumption

Trace

EnergyCostofDistributedSimulation

• QueueingNetworkSimulation• Weakscalingexperiment,powerononedevice• Energyrequiredfordistributedsimulationsignificant

• LGNexushardware,AndroidOS

• Chandy/Misra/Bryantasynchronousnullmessagealgorithm

• YAWNSsynchronousalgorithm

Synchronization:Energy

• Syntheticworkload(onlysynchronizationoperations)• Lookahead parameter• Chandy/Misra/Bryant(asynchronous)

– NULLmessagesusedtosynchronizesimulation– Overheadincreasesasfunctionoflookahead

• YAWNS(synchronous)– Barriersusedtosynchronizecomputation– Moreefficienttimeadvanceatlowlookahead

Lookahead

Energy(J)

Lookahead

Messages

PowerandPerformanceTradeoffs• Powercostutilizingadditionalcores• NS3parallelsimulation• Microcluster platform

– 4coresperboard– Lowpowercore(“0core”datapoint)

• Performancew/powercap• Strongscalingexperiment• Powerawarespeedup:speedup

relativetolowpowercore(lowestpowerapproach)

• Novalueinusing5 or6coresfrompowerperspective

OptimisticEventProcessing(TimeWarp)

• Conservativemethodsincurenergyoverheadstopreventcomputationsfromadvancingtoofaraheadwhichmightresultinsynchronizationerrors– CMB:nullmessages– YAWNS:globalsynchronizations,reductioncomputation

• Optimisticprocessing(e.g.,TimeWarp)avoidssuchoverheads,butincursotherenergycostsnotpresentinconservativeexecution– Computationsthatarelatercanceledbysubsequentrollbacks– Statesavingoverhead– Therollbackcomputationsthemselves(includingsecondaryrollbacks)– ComputingGlobalVirtualTimeandreclaimingmemory

• Weconjecturethatoptimisticapproachesofferthepotentialforlowerenergyoverheadsprovidedrollbackoverheadscanbemanaged

EnergyReductionUsingOptimisticExecution

95

100

105

110

115

120

125

0 2 4 0.6 0.8 1

Energy(J)

FractionofRemoteCommunications

EnergyUsingOptimisticExecution

YAWNS TimeWarp

• Syntheticbenchmark(PHOLD)• Onenodeofmicrocluster (4processors)• Efficientexecution(96-96%ofprocessedeventscommitted)• Suggestssimulationsmaybeabletouseoptimisticexecutionto

achievegreaterenergyefficiencies

Outline

• MotivationandOverview• GreenRuntimeInfrastructure• EnergyEfficientSynchronizationofDistributedSimulations

• EnergyEfficientDataDistributionManagement

• OtherProgramElements

HLADataDistributionManagementServices

• Publish/subscribecommunicationservices• Routingspace• Publicationregions(areaswhereinformationknown)• Subscriptionregions(areasofinterest)• Regionoverlapsdeterminewhoreceivesmessages• Regionsaredynamic

PublicationRegions

(e.g.,sensors)

SubscriptionRegion(e.g.,vehicles)

RoutingSpace

Googlemaps

B

A

C

DDMImplementationApproaches

Region-Based• Multicastgroupper

publicationregion– Messagessenttosinglegroup– Joingroupsbasedon

subscriptionregionoverlaps• Matchingcomputation

Grid-Based• Multicastgrouppergridcell

– Sendtogroupsforcellsoverlappingpublicationregion

– Joingroupsoverlappingsubscriptionregion

• Mayresultinirrelevantandduplicatemessages

Variationshavebeenproposed,e.g.,hybridapproaches

EnergyConsumption:Communications

• Data-driventrafficsimulation(midtownAtlanta)

• Streamoffixedsized(12byte)messages;varyfrequency

• Sendingconsumes~5xenergycomparedtoreceiving

• Aggregatemessages(12byteseach)atsender

• Energyreductionduetoreducednumberofmessagesends

• Largemessagesrequiremultiplepackets

TestScenario:VehicleTraffic• Routingspace:Twodimensional,50x50kilometerarea• Publicationregions:sensorsattrafficintersections,publishingtrafficvolumedataforvehiclestravelinginnorthandsouthdirection;sensorrangeof400meters

• Subscriptionregions:mobilevehiclesreceivingdatafromsensorsapproximately805meters(0.5miles)fromcurrentlocationincurrentdirectionoftravel

• DDMImplementationapproaches– Centralizedregion-based– Distributedregion-based– Grid-basedimplementation– Optimizedgrid-basedimplementation(filteredgrids)

DDMComputationEnergy

• Computationenergyingrid-basedapproachesnegligible• Region-basedefficientif

– Smallnumberofdevices,or– Infrequentchangestopublicationandsubscriptionregions

• Region-based– O(N2)matchingcomputationto

determinegroupcomposition• DistributedRegion-based

– MultipleDDMservers• Grid-based(Rak,VanHook 1996)

– O(1)computationtodetermineoverlappingcells

• GridFilteredRegionBased(Boukerche etal2005)

– Optimizationtogrid-basedusingcoveragethresholdstoreduceDDMoverheads

0.00001

0.0001

0.001

0.01

0.1

100 200 300 400 500 600 700 800 900 1000

Energy(J)

NumberofDevices

DDM:ComputationEnergy

Regions DistributedRegions FixedGrid FilteredGrid

DDMCommunicationsEnergy

Trafficsimulationscenario• FixedGrid:Irrelevantandduplicatemessages• GridFiltered:AvoidsIrrelevantmessages• Regions:Noirrelevantorduplicatemessages

0.01

0.1

1

10

100 200 300 400 500 600 700 800 900 1000

Energy(J)

NumberofDevices

EnergyperStateUpdate

Region/DistributedRegionBased FixedGrid GridFiltered

IntegratedDDMDesign

• ExistingapproachestreatDDMimplementationseparatefromtheapplication

• Integrateddesignenablesreductioninenergyconsumption– Staticvs.dynamicregions– Alignmentofgridreduces/avoidsduplicateandirrelevantmessages

• ApplicationtunableRTIimplementationscanleadtosimplerDDMimplementation,reduceenergyconsumption

0.1

0.12

0.14

0.16

0.18

0.2

60 80 100 120 140

Energy(J)

SubscriptionRegionSize

EnergyperStateUpdate

RegionBased DistributedRegionBased FixedGrid GridFiltered

• Constrainpublicationregionstoasinglecell

• Eliminatesirrelevantandduplicatemessages

29

Energy-Aware Dynamic Data-Driven Application Systems for Transportation Monitoring and Analysis

Richard Fujimoto, Georgia Institute of Technology• Accomplishments

– Calibration procedure for data-driven traffic simulations– Energy consumption for data-driven traffic simulations– Energy consumption for synchronization and data distribution services– Realization of initial G-RTI implementation

• Awards– Article selected to 21st Annual Best of Computing list, ACM Computing Reviews Publications

(“Research Challenges in Parallel and Distributed Simulation,” ACM TOMACS)• Publications

– S. Neal, R. M. Fujimoto, “Energy Consumption of HLA Data Distribution Management Approaches,” Winter Simulation Conference, December 2017.

– R. M. Fujimoto, “Power Consumption in Parallel and Distributed Simulations,” Winter Simulation Conference, December 2017.

– R. M. Fujimoto, M. Hunter, A. Biswas, M. Jackson, S. Neal, “Power Efficient Distributed Simulation,” Principles of Advanced Discrete Simulation, May 2017.

– W. Suh, D. Henclewood, G. Angshuman, R. Guensler, M. Hunter, and R. M. Fujimoto, “Dynamic Data Driven Transportation Systems,” Multimedia Tools and Applications, Feb. 2017.

– D. Henclewood, W. Suh, M. O. Rodgers, R. M. Fujimoto. M. P. Hunter, “A Calibration Procedure for Increasing the Accuracy of Microscopic Traffic Simulation Models,” Transactions of the Society for Modeling and Simulation Intl., Vol. 93, No. 1, pp. 35-47, 2017.

– A. Biswas and R. M. Fujimoto, “Energy Consumption of Synchronization Algorithms in Distributed Simulations,” Journal of Simulation, 11(3), pp. 242-252, 2016.

– S. Neal, R. M. Fujimoto, M. Hunter, “Energy Consumption of Data Driven Traffic Simulations,” Winter Simulation Conference, December 2016.

– M. Hunter, A. Biswas, B. Chilukuri, A. Guin, R. Fujimoto, R. Guensler, J. Laval, H. Liu, S. Neal, P. Pecher, M. Rodgers “Energy-Aware Dynamic Data-Driven Distributed Traffic Simulation for Energy and Emissions Reduction,” InfoSymbiotics/DDDAS Conference August 2016.

30

Energy-Aware Dynamic Data-Driven Application Systems for Transportation Monitoring and Analysis

Richard Fujimoto, Georgia Institute of Technology

• Coordination, Synergy, and Collaborations– Panel, Winter Simulation Conference, Dec. 2016; R. M. Fujimoto, N. Celik, H.

Damgacioglu, M. Hunter, D. Jin, Y-J. Son, J. Xu, “Dynamic Data Driven Application Systems for Smart Cities and Urban Infrastructures,” Winter Simulation Conference, December 2016.

– Panel, Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC), Dec. 2017 [follow up on 2016 Workshop for creating a national research agenda in M&S)

– Panel, Winter Simulation Conference, Dec. 2017; R. M. Fujimoto, C. Carothers, A. Ferscha, D. Jefferson, M. Loper, M. Marathe, S. J. E. Taylor, “Computational Challenges in Modeling and Simulation of Complex Systems,” Winter Simulation Conference, December 2017. [organized with Dong Jin]

– Visit to Illinois Institute of Technology (Dong Jin), Winter 2017– “Network Performance Monitoring and Distributed Simulation to Improve

Transportation Energy Efficiency,” (ARPA-E, R. Guensler, PI)– Assistance in updating IEEE 1516 HLA standard (time management services)

• Exposure/Use by other groups– “Smart Cities” demonstration, Atlanta, Georgia September 2017