GeoEvent Server - EsriVisualization. choosing a service type: stream service, feature service, map...

GeoEvent Server

1

2

AgendaProduct Positioning & EvolutionWhat is Real-Time GIS?Applying Real-Time AnalyticsVisualizing Real-Time Data

Road Ahead

3

4

5

1 Product Positioning & Evolution

ArcGIS Enterprise

GeoEvent Server is a “server role” extending the capabilities of your ArcGIS Enterprise …

GeoEvent Server – Real-time and big-data capabilities• Ingest high velocity real-time

data into ArcGIS

• Perform continuous analytics on events as they are received

• Store observations in a spatiotemporal big data store

• Run batch analytics on stored observations

• Visualize high velocity & volume data:- as an aggregation- as discrete features

• Notify those who need to know about patterns of interest

DesktopWeb Device

visualization

live & historicaggregates & features

map & feature servicestream service

live features

ArcGISEnterprise

GeoEventServer

spatiotemporalbig data store

storage

ingestion

analytics

GeoAnalyticsServer

analytics

Product Adoption Landscape

1,210Organizations

87Countries

as of August 2017

Commercial- Financial Services- Insurance- Logistics / Trucking- Manufacturing- Media & Entertainment- Real Estate- Retail

Defense & Intelligence- Intelligence- Military Operations

Public Safety- Emergency / Disaster Mgmt- Fire, Rescue, EMS- Homeland Security- National Security- Law Enforcement- Special Events

Utilities- Electric & Gas- Telco / Cable- Public Works

Water- Water resources- Water / wastewater /

stormwater

Professional Services- AEC- Environmental Mgmt- GIS & IT

Nonprofits & Education- Conservation- Humanitarian- Sustainable Development- Higher Ed- Research/Science Institutions

Natural Resources- Agriculture- Forestry- Mining- Oil & Gas- Pipeline

Transportation- Aviation- DOT- Railways- Maritime & Ports- Public Transit

Government- National- State- Local

Health & Human Services- Hospital & Health Systems- Pharmaceuticals- Public Health

ingest visualize

analyze store

real-time& big data

A Quadrant ApproachKey Goals of Real-Time & Big Data GIS

Web GIS

Web GISwith real-time capabilities

ingest visualize

analyze store

real-time & big data

thousands3K e/s

3K e/s 200 e/s

bring your owndatabase

GeoEventServer

200 e/s

Web GIS


ingest visualize

analyze store


thousands3K e/s

3K e/s 200 e/s

GeoEventServer

200 e/srelationaldata store

Web GIS


IoT

ingest visualize

analyze store


millions4K e/s

4K e/s 10Ks e/s

GeoEventServer

4K e/sspatiotemporalbig data store

ArcGISEnterprise

ArcGIS Enterprisewith real-time capabilities scaled out

ingest visualize

analyze store


millions12K e/s

12K e/s 10Ks e/s

bring your owngateway

IoT12K e/s

GeoEventServer


ArcGIS Enterprisewith real-time capabilities

actuation

actuation

ArcGISEnterprise


GeoEventServer

• compliments IoT providers w/ geospatial reasoning- actuation adjusts behavior of things in our environment

2 What is Real-Time GIS?

Observation datatypes of observation data

movingsomethingthat moves

• planes• vehicles• animals• satellites• storms

stationarystands still but

attributes change

• water gauges• weather stations• traffic sensors• air quality

discretesomething that“just happens”

• crimes• lightning• accidents

you can createyour own

connectors

Ingestionuse an existing input connector

GeoEvent Server

Inpu

ts

Out

puts

GeoEvent Services

Poll an ArcGIS Server for Features

Poll an external website for GeoJSON, JSON, or XML

Receive Features, GeoJSON, JSON, or XML on a REST endpoint

Receive RSS

Receive GeoJSON or JSON on a WebSocket

Receive Text from a TCP or UDP Socket

Subscribe to an external WebSocket for GeoJSON or JSON

Watch a Folder for new CSV, GeoJSON, or JSON Files

Out

of t

he B

ox REST

.csv

WS

WS

HTTP

Esri

Gal

lery

ActiveMQ

CAP

Instagram

Exploitation Support Data

CoT Cursor-on-Target

RabbitMQ

NMEA 0183

MQTT

Sierra Wireless (RAP)

KML

Kafka

Trimble (TAIP)

Twitter

Part

ner G

alle

ry

CompassLDE

enviroCar

GNIP

FAA (ASDI)

exactEarth AIS

Waze

Valarm

Networkfleet

OSIsoft ****

Zonar *

Amazon IoT

Azure IoT

Poll an ArcGIS Server for Features

Watch a Folder for new CSV, GeoJSON, or JSON Files.csv

CoT Cursor-on-Target

you can createyour own

connectors

Disseminationuse an existing output connector

GeoEvent Server

Inpu

ts

Out

puts

GeoEvent Services

Out

of t

he B

oxPublish Text to a UDP Socket

Send an Email

Add or Update a Feature

Push Text to an external TCP Socket

Push GeoJSON or JSON to an external WebSocket

Push GeoJSON or JSON to an external Website

Send a Text Message

Send an Instant Message

Send Features to a Stream Service

Write to a CSV, GeoJSON, or JSON File .csv

WS

im

HTTP

Add a Feature to a spatiotemporal big data store

Update a feature in a spatiotemporal big data store

ActiveMQ

Esri

Gal

lery

Cursor-on-Target

Hadoop

Kafka

MongoDB

MQTT

RabbitMQ

Twitter

Amazon IoT

Azure IoT


Send a Text Message

Send an Email

3 Applying Real-Time Analytics

GeoEvent Services• A GeoEvent Service defines the flow of event records

- The input(s) ingesting event data- The filtering and processing performed on each event record- The output(s) to which event records are sent

Real-Time analyticsgeofencing – notify when construction vehicles become disjoint from a zone vs. all zones

Applying Real-Time AnalyticsConfiguring a GeoEvent Service

33.322336-117.416323

33.322125-117.398225

Processors

Processors generally alter event records they receive …

You can createyour own

processors.

GeoEvent Extension

Inpu

ts

Out

puts

GeoEvent Services

Buffer Creator

Convex Hull Creator

Difference Creator

Envelope Creator

Field Calculator

Field Enricher

Field Mapper

Geotagger

Incident Detector

Intersector

Projector

Simplifier

Symmetric Difference

Track Gap Detector

Field Reducer Union Creator

Out

of t

he B

oxAdd XYZ

Esri

Gal

lery

Bearing

Ellipse

Event Volume Control

Extent Enricher

Field Grouper

GeoNames Lookup

Range Fan

Reverse Geocoder

Service Area Creator

Symbol Lookup

Track Idle Detector

Unit Converter

Visibility

Motion Calculator Query Report

• Use a Processor to calculate new values, enrich events, project geometry, …

Buffer Creator Geotagger

Intersector

Incident Detector

Field Enricher

Motion Calculator

4 Visualizing Real-Time Data

Visualizationchoosing a service type: stream service, feature service, map service

Stream LayerMap Layer

Feature Layer

• Stream layers in apps subscribe to stream services to immediately visualize observations- does not require storage, low latency, no playback

• Map & Features layers in apps periodically poll to visualize most current observations- backed by an enterprise geodatabase (EGDB) or a spatiotemporal big data store (BDS)- history can be retrieved & queried for playback

desktop

device

web

ArcGISEnterprise

Stream Service

Send Features to a Stream Service subscribe (push)

polling (pull)

Map ServiceFeature Service

GeoAnalyticsServer

Add a Feature to a BDSUpdate a Feature in a BDS



EGDBGeoEventServer

On-the-fly aggregationsof observations

• The spatiotemporal big data store enables you to aggregate data on-the-fly- aggregating on-the-fly accommodates for real-time data- you can perform exploratory analysis via any combination of space, time and attribute queries- toggle from aggregation to raw feature view- access feature level attributes in aggregate or feature level view

geohash

square

trianglepointy

triangleflat

hexagonflat

hexagonpointy

Storagespatial indexing to support on-the-fly aggregations

5 Road Ahead

GeoEvent ServerRoad Ahead

•RASP-Reliability-Availability-Scalability-Performance


ArcGISEnterprise


GeoEventServer

IoT

MINIMUM deployment3 machines

1

functional servers & spatiotemporal big data storeSHOULD BE on ISOLATED machines!!!

23


ArcGISEnterprise


GeoEventServer

IoT

RECOMMENDED deployment5 machines

1

2 3 45



ArcGISEnterprise


GeoEventServer

IoT

RESILIENT deployment7 machines

1

2 3 4765


machine 1machine 1data store: relational

Portal for ArcGIS

ArcGIS Server


deployment experience

ArcGIS Server “hosting”

web adaptorfor portal

web adaptorfor server

machine 3

machine 4

machine 2data store

big data storespatiotemporal

data store


data store


machine 5

site-geoevent01

ArcGIS Server

ServerGeoEvent


Portal for ArcGIS

ArcGIS Server






machine 3

machine 4

machine 2data store


data store


data store


machine 5

site-geoevent01

site-geoevent02

machine 6

site-geoevent03

machine 7

ArcGIS Server

ServerGeoEvent

ArcGIS Server

ServerGeoEvent

ArcGIS Server

ServerGeoEvent

RESILIENT


Portal for ArcGIS

ArcGIS Server






machine 3

machine 4

machine 2data store


data store


data store


machine 5

site-geoevent01

site-geoevent02

machine 6

site-geoevent03

machine 7

ArcGIS Server

ServerGeoEvent

ArcGIS Server

ServerGeoEvent

ArcGIS Server

ServerGeoEvent

RESILIENT

bring-your ownmessage broker



machine 1machine 1Portal for ArcGIS

ArcGIS Server “hosting”data store: relational

ArcGIS Enterprisewap was

machine 3

machine 4

machine 2machine 5

site-geoevent

data store


data store


data store


GeoEvent Servergateway

ArcGIS Server






machine 3

machine 4

machine 2machine 5

machine 6

machine 7

data store


data store


data store



ArcGIS Server


ArcGIS Server

RESILIENT

site-geoevent


ArcGIS Server


• At 10.6, GeoEvent Server is resilient by default and includes:- a ‘gateway’: a productized component that enables resilient event distribution to multiple GeoEvent Servers- a ‘GeoEvent Server’:

- product that ingests real-time data from a variety of inputs and produces received events to a gateway- product that consumes from a gateway and performs streaming analytics and disseminates results via outputs

- an ‘ArcGIS Server’: is still required that a user pre-install ArcGIS Server prior to installing GeoEvent Server at 10.6




site-geoevent

machine 2machine 5data store

big data storespatiotemporalGeoEvent Server

gateway

ArcGIS Server

ArcGIS Enterprise

GeoEvent Server is a “server role” extending the capabilities of your ArcGIS Enterprise …

GeoEvent Server – Real-time and big-data capabilities• Ingest high velocity real-time

data into ArcGIS

• Perform continuous analytics on events as they are received

• Store observations in a spatiotemporal big data store

• Run batch analytics on stored observations

• Visualize high velocity & volume data:- as an aggregation- as discrete features

• Notify those who need to know about patterns of interest

DesktopWeb Device

visualization

live & historicaggregates & features

map & feature servicestream service

live features

ArcGISEnterprise

GeoEventServer


storage

ingestion

analytics

GeoAnalyticsServer

analytics

Questions / Feedback?

Josh JoynerGeoEvent Server, Product Manager

[email protected]

mailto:[email protected]

GeoEvent Server - EsriVisualization. choosing a service type: stream service, feature service, map...

Documents

Transcript of GeoEvent Server - EsriVisualization. choosing a service type: stream service, feature service, map...