[IoT Tech Expo] Smart Cities – Leveraging Messaging from Project to City to Nation
-
Upload
solace -
Category
Technology
-
view
395 -
download
2
Transcript of [IoT Tech Expo] Smart Cities – Leveraging Messaging from Project to City to Nation
Solace Logistics IoT Architecture
Smart Cities Leveraging Messaging from Project to City to Nation
Solace SystemsCONFIDENTIAL
#CONFIDENTIAL1
Talking Points Smart City Various Facets Smart * - Transport, Buildings, Energy, Healthcare, Security and MoreAspects of an IoT Ecosystem Capture, Analyze, Store, React, Predict, Manage and InterconnectSmart City - Big Data in Motion Across Layers Fog, Cloud, CoreImplementing a Smart City Project Methodology and Case StudiesSmart City to Smart Nation - Interconnecting the Various Facets
#CONFIDENTIALThere are various ways to slice and dice the IoT architecture, implementation and management puzzle. I will focus on the data movement aspect. 2
Diversified Blue-Chip Customer BaseDisruptive Products in Rapidly Growing Market Solace Systems Company Overview
BetterFasterCheaper
Solace Systems is a provider of hardware and software end-to-end message routersWidely deployed in various IoT projects smart cities, industrial IoT, mobility and beyondFounded in 2001, HQ in Ottawa, Canada
About Sumeet Sumeet runs the customer facing technology group for Europe and Asia
#CONFIDENTIAL
3
Smart City Various Facets Smart * Frost & Sullivans Categorization
http://www.slideshare.net/FrostandSullivan/global-smart-city-market-a-15-trillion-market-ortunity-by-2020
#CONFIDENTIALFrost and Sullivan, among other analysts have analyzed the Smart City market. Lets focus on the categorization of the market by them, size isnt really the focus of this slide pack. The categorization is as good as any, across building, infra, healthcare, transport, governance, education, security and energy. We see these independent categories a nice way to talk about real projects. But what is most interesting is the power of interconnecting those smart projects to make a smart city, of in Singapore, a Smart Nation!4
Why IOT now?Because Its Needed and Its Possible
Affordable SensorsBetter BatteriesMuch Better NetworksDigital and Mobility InnovationsReal Time TechnologiesMuch Better Messaging and Integration TechnologiesBig Data Technologies
#CONFIDENTIALSmart City various facets Smart *
FogDMZCore
Network
Messaging and APIs
Multi Protocol StandardsAggregationThrottlingConnection ScalingFiltering & RoutingFan in -Burst HandlingPersistenceSlow Consumer HandlingFanout Lambda Stream
Things & AppsFog ComputingLegacy IntegrationReal Time AnalyticsGUIs and DashboardsLegacy GatewaysBusiness Logic & Integration[Big] Data Analytics[Big] Data Store
Management and MonitoringDevice ManagementSecuritySmart Cities are built block by blockEach project can be built by assembling the shown blocksProjects are then integrated to create an interconnected smart cityLets go block by block and explore!
#CONFIDENTIAL
6
Aspects of an IoT Ecosystem Capture, Analyze, Store, React, Predict, Manage and Interconnect3 layers, if we vertically slice the blockFog - between the Thing and the CloudDBZ/Cloud Data center edgeCore traditional business logic and big data destination, where business logic lives3 layers if we horizontally sliceNetworkMessaging and APIsAppsPlus management and security
NetworkFogDMZCore
Messaging and APIs
#CONFIDENTIAL
Aspects of an IoT Ecosystem Capture, Analyze, Store, React, Predict, Manage and InterconnectThing a very broad termAre M2M, Sensor Networks and IoT all the same?Thing is a source of information and sometimes destination tooIt could be as simple as a remote sensor and as sophisticated as a device with appsIt is connected via Network and Messaging
NetworkFogDMZCore
Things & Apps
Messaging and APIs
#CONFIDENTIAL
Aspects of an IoT Ecosystem Capture, Analyze, Store, React, Predict, Manage and Interconnect
NetworkFogDMZCore
Messaging and APIs
Multi Protocol StandardsAggregationConnection ScalingFan in -Burst Handling
Things & AppsFog ComputingLegacy IntegrationLegacy GatewaysCapture and Reply the ability to communicate with the thingEssentially in the fog, but could be in the DMZ tooStandards based capture and reply MQTT, REST, Adapters for legacy integration, such as SCADAFog computing if needed quick calculations, filtering etcMessaging plays a critical role both in the Fog and DMZ
#CONFIDENTIAL
NetworkFog
Aspects of an IoT Ecosystem Capture, Analyze, Store, React, Predict, Manage and interconnectStore the ability to store information where it mattersZero data loss persistence with HA and DRBurst handling fan inSlow consumer isolation and handling no back pressure to the publisherBig data storage is a good idea
DMZCoreMessaging
Multi Protocol StandardsAggregationConnection ScalingFan in -Burst HandlingPersistenceSlow Consumer Handling
Things & AppsFog ComputingLegacy IntegrationLegacy Gateways[Big] Data Store
#CONFIDENTIAL
App 1
App 3
App 2
Aspects of an IoT Ecosystem Capture, Store, Analyze React, Predict, Manage and interconnectAnalyze - this is why IoT is importantLambda & kappa architectureBatch layer and speed layerMake sure that there is no back pressure
NetworkFogDMZCore
Messaging
Multi Protocol StandardsAggregationThrottlingConnection ScalingFan in -Burst HandlingPersistenceSlow Consumer HandlingFanout Lambda Stream
Things & AppsFog ComputingLegacy IntegrationGUIs and DashboardsLegacy GatewaysBusiness Logic & Integration[Big] Data Analytics[Big] Data Store
App 4
App 5
App 6
HDFSReal-Time Analytics
#CONFIDENTIALReact and predict- lambda speed layer/kappa stream layerFeed complex events processing or streaming analytics layersTopic routing based pub sub/wildcard filteringQueuing as neededLow latency 18 micro seconds
NetworkFogDMZCore
Messaging
Multi Protocol StandardsAggregationThrottlingConnection ScalingFiltering & RoutingFan in -Burst HandlingPersistenceSlow Consumer HandlingFanout Lambda Stream
Things & AppsFog ComputingLegacy IntegrationReal Time AnalyticsGUIs and DashboardsLegacy GatewaysBusiness Logic & Integration[Big] Data Analytics[Big] Data StoreAspects of an IoT Ecosystem Capture, Analyze, Store, React, Predict, Manage and Interconnect
#CONFIDENTIAL
Device ManagementAspects of an IoT Ecosystem Capture, Analyze, Store, React, Predict, Manage and Interconnect
FogDMZCore
Network
Security
Messaging
Multi Protocol StandardsAggregationThrottlingConnection ScalingFiltering & RoutingFan in -Burst HandlingPersistenceSlow Consumer HandlingFanout Lambda Stream
Things & AppsFog ComputingLegacy IntegrationReal Time AnalyticsGUIs and DashboardsLegacy GatewaysBusiness Logic & Integration[Big] Data Analytics[Big] Data Store
Management and MonitoringDevice management Firmware managementCertificate managementProvisioning and maintenanceAll layers monitoring and managementSecurityAuthenticationAccess controlEncryptionIntegrate with core systems
#CONFIDENTIALSmart City - Big Data in Motion Across Layers Fog, Cloud, CoreLets focus on the aspects of integrationMessaging required capabilities and relevant case studies
NetworkFogDMZCore
Messaging
Multi Protocol StandardsAggregationThrottlingConnection ScalingFiltering & RoutingFan in -Burst HandlingPersistenceSlow Consumer HandlingFanout Lambda Stream
Things & AppsFog ComputingLegacy IntegrationReal Time AnalyticsGUIs and DashboardsLegacy GatewaysBusiness Logic & Integration[Big] Data Analytics[Big] Data Store
Multi Protocol StandardsAggregationThrottlingConnection ScalingFiltering & RoutingFan in -Burst HandlingPersistenceSlow Consumer HandlingFanout Lambda Stream
#CONFIDENTIAL
Next Generation Data Distribution FabricOpenWirelinesOpen APIs& StandardsStorageTechnologiesMQTTCEPRESTDataWarehouseAnalyticsIn-MemoryData GridsWebsocketsDatabaseBusinessIntelligenceArchitecturesEnterpriseService BusSOA/Web ServicesJEEOpenMamaJMSEvent-DrivenArchitectureNetworksMobileWAN
Easy, Fully IntegratedAdministration & SecurityReal-Time & Very High VolumeReal-time EDA for the Future of Streaming DataMove, share and synchronize information
#CONFIDENTIAL
15
Unified APIJMSCC#JavaObjective CiOSJavaScriptHTML5Silverlight, Flash & .NETMQTTRESTServerApplicationsWebMessagingTechnologiesOpen APIs& Protocols
Java RTO.NET
Robust client access to all capabilities and qualities of service
#CONFIDENTIAL
16
Solace Flexibility in IoTEnd-to-End Data Movement FabricCoreAnalyticsDeviceManagementCEPGW
Edge
FogGWGWMQTTRESTCoAPLWM2MOMA-DMGWGWWebSocketWebRTCVirtualizedSoftware
AnalyticsDevice ManagementCEP
Filtering, Routing, Replication, and DRCANBusZigBee
VMR
VMR
VMRDC1 (Active)DC2(Active)
Messaging
Multi Protocol StandardsAggregationThrottlingConnection ScalingFiltering & RoutingFan in -Burst HandlingPersistenceSlow Consumer HandlingFanout Lambda Stream
#CONFIDENTIAL
17
SAP Solace is the IoT FabricSolace = HCPIWhat is the Use CaseMessage service in the HANA CloudBig-data move and bursty trafficFast-path to HANA platform, JMS-like APIs for developersConnect via REST,MQTT,WebSocket Fully cloud enabled (SLAs, multi-tenant)Why SolaceM2M and M2X demands for ultra scaleConnect thousands to millions of devicesData bursts and big-data in motion: hundred thousands+ messages per secondFlexible publish and subscribe and reliable messaging for event stream data
SAP HANA
EngagementSAP Gateway
SAP BPMSAP Operational Process IntelligenceData Services
Cloud AppsOn Pre- mise Apps
HCP Connected Apps
ColumHadoopIQESP
HCI Messaging
Device
EdgeFogCore
#CONFIDENTIALConnected Solutions
Pub sub:between devices, event processing and db services (HANA
18
Building/Asset MonitoringPreventive Maintenance
Solace BridgeCollect information from building sensors preventive maintenance and optimizationLift open/close, lighting, water, electricity etc50 block pilot, 10,000 blocks in cityFogEdgeCore
EdgeFogCore
Modern Sensors
Aggregator Collect data from legacy sensors at client Blocks.
Client BlockREST/MQTT
ESBCore Business Logic(Event Driven)What is the Use CaseGetting real time/batch data from housing buildings, such as lift data, pipelines, electricals etc. for security and preventive maintenance Why SolaceSimplicity one fabric for Fog, Edge and Core Scalability easy to scale the 50 bloc pilot to 10,000 blocks in SGAggregation per block as needed, every block is fiber connected and has a ruggedized serverLegacy bridging with ESBsFeeding Big Data store for longer term predictive maintenance and cost/contract optimizationReal time security feeds as necessary, such as roof lock compromiseSchedulingERPBig Data Store and Analytics
#CONFIDENTIAL
19
BI, Hadoop, HBASE, Data warehouse, Analytics, Visual Analytics, etcIoT Foundational Streaming Fabric (e2e pub/sub, req/reply)Connected cars, Smart homes
CEP/ESP EnginesInternal:Internal routing, filtering, fan-out, downstream/WAN distribution
LAN/WAN
External:Internet/Edge collection & Streaming, fan-outRESTMQTTWebSociPhoneAndroidetc
Pub/sub, Req/Reply, P2P, persistence HA, DR, filtering, hardware or software, throttling/shock absorbing, scale, multi-site streamingRESTJMSC/C#JAVA
Cloud or on-prem
VMR
Concentrator/Fog Layer
#CONFIDENTIAL
20
City Bus Monitoring
Solace MessagingREST/MQTT
Regulators DashboardCommuters Apps and RIAsAnalyticsDWHESB (if needed)
CEP
Edge
What is the Use CaseGet location, door open/close, loading etc information from 5000 buses in real time to optimize bus movement, fleet management, avoid bunching, forecast arrival time and moreWhy SolaceScale much smaller footprint for 5K vehicles streaming dataReliability and robustnessTibco integration the ESB and CEP stack is TIBCOLow Latency for predictive analyticsMulti protocol support - REST, MQTT, JMS and more, without any bridges all nativeJMSLegacy RESTEdgeCoreFog
Legacy Gateway
Bus Stop DisplayREST/MQTTPush
#CONFIDENTIALLTA started with Solace with the bus monitoring system.
Each bus has about 30 sensors, and generates various kinds events, such as door open/close, location, loading, environment information.
The bus has a bus wide Wifi across which all sensors talk to a linux server called Nobi (New onboard interface). This server has 4g connectivity to Solace, where the bus sends the information collected from various sensors by topic (REST URLs) to Solace.
Solace routes this information to backend servers (TIBOC), which do real time analytics. Examples are solving problems like Bus Bunching (https://publictransportsg.wordpress.com/2013/05/28/bus-bunching/). Driver monitoring, jam prevention etc. are determined by the analytics systems - data visualisation, as well as CEP systems21
Next Generation Electronic Road Pricing
EdgeCoreMHI OBU
DMZ Solace
Core Solace
Routing Server
Payments
Analytics
0 0 0200K Concurrent ConnectionsMQTT TerminationReq Rep for firmware upgradesPush NotificationsPersistent MessagingShock AbsorberHigh ThroughputFanoutSlow Consumer HandlingHigh Availability and Disaster RecoveryMQTTJMSEdgeCoreFogWhat is the Use CaseConnect 1.5M vehicles (200K at a time) in real time for pay per use for road tolls. Suggest better routers, traffic management real time and long term and value added services/analyticsWhy SolaceScale much smaller footprint for 200K vehicles streaming dataReliability and robustnessAdvanced features such as subscription management, MQTT request reply, topic to queue bridging, fanoutAdvanced security and monitoring/event generationCost of competition
#CONFIDENTIALLTA is now rolling out the next generation ERP system. This will eventually replace the current road toll system.
Each vehicle in the city (1.5M in total) will be fitted with a smartphone like device from MHI.
This device will be communicating with Solace over MQTT, and send all sorts of information, especially real time location to Solace. A lambda architecture will be implemented in the core layer.- The toll server gets the real time location of vehicles and charges as per prevailing road prices.- For street side parking, similarly, parking also becomes real time-There is a traffic management server, which in real time manages where the congestion is, and suggests vehicles when drivers start them on routes. Congested routes are put at higher tolls in real time- on the batch side, since all information is captured, historical starts will below better road management, finding our congestion areas, etc- the possibilities are immense - if a driver regularly blocks ambulances - its easy to tell. Similarly, drivers not keeping safe distance are easily picked out. insurance companies will be ab;e to source this data as an example.
http://www.straitstimes.com/singapore/transport/ncs-mhi-to-build-islandwide-satellite-based-erp-for-556m
22
Cloud Core Communication
Common IoT Platform Building BlocksCloud EdgeCommunicationFog LayerThingsBack-end Systems/AppsCloud PaaS (CF, OS, etc)Device addressing, management, SecurityAPI & app lifecycle managementIoT Application Connectivity/CommunicationHealthcare, Banking, Oil&Gas, Transportation, TelematicsMonitoring & ManagementThingsIntelligent DevicesLimitless
OptionalHardware or SoftwareDeployed in fieldAggregator/normalization
DMZ Datecenter EdgeCloud or Enterprise offeringUltimate point of aggregation and messagingDatacentre coreMain integration point for all back-end applicationsMiddleware centric
Cloud Edge
Solace FocusCloud CoreNetwork (IP, wi-fi, satellite, etc)
#CONFIDENTIAL
23
Layer by Layer Analyses of Capability Requirements
Solace appliance when customer prem or large local fan-out requirementSoftware option for lightweight queueingSolace API for app connectivity
Massive scale 200k connections, 24 Million msg/sec fanoutMulti-protocol - MQTT, REST, JMS, Web MessagingSecurity hardened appliance (no OS facing devices), authentication, access controls, SSL, separate management port, management roles, config change loggingMulti-tenant required for cost-effective aaS offeringsSeamless connectivity to core and multi-site federationRobust real-time behavior slow consumers, HA, consistent latency
Solace is proven in large scale, real-time event driven messaging environments20-200x the performance the competition for messaging (JMS, etc)Robustness and stability (built-in HA, DR, slow consumer management, etc)Simplicity. Appliance includes full stack of O/S, server, messaging, management. Seamless connectivity to coreHundreds of enterprise integrated features
SolaceRouter
Solace Router
VMR
#CONFIDENTIAL
24
Interfacing with LegacyLegacy Fog Layer
ProprietarySCADARail sensorsLegacy Gateway Ingest SCADA etc
Protocol Translation (from Legacy to HTTP/TCP/JMS)Takes data and quickly throws it to SolaceSolace in the DCL performs guaranteed messagingScaled horizontally with more hardwareSolace Cloud/Edge/Core
Solace is proven in large scale, real-time event driven messaging environments20-200x the performance the competition for messaging (JMS, etc)Robustness and stability (built-in HA, DR, slow consumer management, etc)Simplicity. Appliance includes full stack of O/S, server, messaging, management. Seamless connectivity to coreHundreds of enterprise integrated features
Solace Router
Legacy GatewayThin Protocol Translation and Handoff Layer, Heavy Lifting in the Data Centre Layer
Legacy Gateway
#CONFIDENTIAL
25
Positive/Negative Train Control
BusinessIntelligenceComplex EventProcessingLocationServicesArchival
Core
#CONFIDENTIALAizu Bridge MonitoringSolace appliances are used as a backbone for sensor networks in transportOne such example is vibration sensors on bridges for earthquakesSolace hardware appliances connect to sensor gateways to reliably route data to analytic servers in the data center
Core
#CONFIDENTIALBurst Handling - Buffering Spikes and Overflow
Capacity & Availability Limits
Big DataShock AbsorberNetworkProcessingStorageOutagesUpgrades
InconsistentAggregateInputStream
#CONFIDENTIALThe Python eating a Pig problem End to end latency problems Sizing for peak load problem Versioning and planned/unplanned outagesInterleaving small data inside big data with chunking
The second challenge we often see in data collection is gracefully handling data spikes. When you have different sources this is inevitable they wont all behave the same way.
Protect data loaders from overwhelming, protect data sources from pushback.
Whatever your method of collection, it must have some way to buffer data and pass it along.28
Replication and Global Federation
Core
Edge
GatewayThingsBack-end Systems/ApplicationsGlobal Site Federation
EdgeMessagingCore MessagingIoT securityDevice addressing & managementAPI & app lifecycle managementRESTJMSMQTTIoT ConnectivityGlobal Messaging Platform FederationHealthcare, Banking, Oil&Gas, Transportation, TelematicsMonitoring & ManagementThings
DMZGlobal Site Federation
EdgeMessagingCore Messaging
Global Site Federation
EdgeMessagingCore Messaging
EMEAAPACNA2
S/W- Gateway / aggregation- IoT Edge- Core Messaging- Global / Inter-datacenterInter-appliance dynamic routing including multi-tenantInter-appliance persistent message routingWAN optimization TCP optimizations, compressionDetailed per-connection visibility
#CONFIDENTIAL
29
[Semiconductor] ManufacturingReal Time Movement of QC Data
MESmes/main
mes/measure
MAINmes/equip
Master Systemmes/masterDB
History Systemmes/historyDB
OIPENGR
Client PCClient MobileSolace APIjavaSolace APICSolace APIC#
Manufacturer
Data At Rest Analytics
Real Time Analytics (Recall)WAN Optimized Lossless Data TransferHead QuartersQC DataDemand DataControl Data
Visual AnalyticsFogEdge and CoreSolaceWhat is the Use CaseReal-time quality control data sourcing and analytics to reduce recall times. Also using the same data bus for bidirectional contract manufacturer communicationSolace APIHTML5Why SolaceLow latency, real-time messaging moving away from a data base driven message bus to Solace real time messaging allows recall time reduction from days to minutesWAN Optimization the ability to more data in a low bandwidth, low latency, loss less manner from 21 factories around the world to HQBig Data ingest was added later without any changes to messagingPerformance, speed and simplicity
EdgeCore
#CONFIDENTIALB2BDMZ
IoT Event Bus
Big Data Lake Repository & Real-Time Analytics
Cross Agency Message BusEnergy
Transport
Security
Environment
Domain Service BusInfrastructure
Bank MachinesBranchesTelephone
DedicatedNetworksCentral ServicesIntegrationServices
DataGovernance
Business BenefitsAgilityLow TCORobustnessInnovationTime to MarketReal-time servicesOrchestration
Realtime 360 degree Customer ViewDR
Building
PublicCloud
PrivateCloud
VMR
VMR
VMR
VMR
VMR
VMR
#CONFIDENTIAL
31
Network
Messaging
Industry ParticipationWeaving Smart * Projects into a Smart City/Smart NationLooking at Smart Cities from an Administrative PerspectiveTypically a few pillars drive the initiativesCore ecosystem provided by the pillarsIndividual projects then plug into the infrastructure and get integratedStandards and Access Control are key
IT MinistryRail
PoliceDefenceBuildingsEnvironmentPort AuthAirRoadBuses
PMOParksTaxEducation
ParksPUBEnergy
Finance Ministry
#CONFIDENTIAL
32
Fix my *everything*Routing by Jurisdiction and Event TypeLocation is determined by phoneUser optionally selects event type
Solace Messaging
Report/amk/vandal/>
Submit
VandalismFireAssault...
Report/amk/fire/>
Report/cbd/vandal/>Report/cbd/fire/>And Mo KioCBD
Topic: >
MHA HQTopic: report/>
IDA
#CONFIDENTIAL
33
Single Enterprise Messaging Platform for All Applications / All EnvironmentsPaaS Private CloudIaaS
BoshProvisioningServiceBrokerMaaSCatalogApp
AppApp
AppAppAppAppCustomAppsApplicationPlatformsOracle OSBWebSphereApp ServerDataPowerJBossJboss FuseMuleSoftAppAppAppAppAppAppAppAppAppAppAppApplication portabilityUbiquitous ConnectivityBest-of-breed Messaging
App
#CONFIDENTIAL
Thank [email protected]?We can also carry the conversation on at the Solace booth. Come have a beer with us at booth 121.
Solace SystemsCONFIDENTIAL
#CONFIDENTIAL35