LOREM IPSUM DOLOR SIT AMET. - SOFIA2sofia2.com/docs/SOFIA2 - SOFIA2 as Interoperability...

62
SOFIA2, BIG DATA INTEROPERABILITY PLATFORM April 2014

Transcript of LOREM IPSUM DOLOR SIT AMET. - SOFIA2sofia2.com/docs/SOFIA2 - SOFIA2 as Interoperability...

SOFIA2, BIG DATA INTEROPERABILITY PLATFORM

April 2014

2

CONTENTS

01 Introduction to SOFIA2 Platform

02 SOFIA2 Platform in detail

03 SOFIA2 Capabilities at a glance

3

Indra already has mature assets to build this platform, to integrate with commercial solutions

and open source components:

iCloudBroker as the Marketplace solution to facilitate the Cloud commercialization of

application, services and data.

iDynamics as the corporate architecture, easing application development (traditional and

mobile) and services that can be deployed in the cloud.

SOFIA2, designed originally in the homonymous R&D project, covering IoT, M2M and Big

Data in real time. G@PaaS to extend the platform with capabilities to support business in Cloud

Mayor información en: http://www.sofia-project.eu/

INTRODUCTION TO SOFIA2 PLATFORM

ZOOM TO SOFIA2 INTO INDRA SMART PLATFORM

3

Product / Solution bundling

MARKETPLACE (iCloudBroker)

DEPLOYMENT &

INFRASTRUCTURE

STORAGE &

DATA MANAGEMENT

APPLICATIONS & SERVICES

TOOLS

INTEGRATION &

SECURITY

COMPONENTS & BUNDLES

IDYNAMICS ARCHITECTURE

BUNDLES

BUSINES

S

COMP ARQ

COMP

IBM

TIBCO

ORACLE

BIZTALK

WSO2

OAuth

ORACLE

SQLSERVER

HADOOP

NoSQL

SOFIA2 & Gdb (BIG DATA, DataGrid)

GPaaS (Provisioning, Multitenant) AZURE

AMAZON

COMPONENTS APPS DATA SERVICES PERSONALIZATION

IDE PLUGINS

REVERSE

ENG

UI GENERAT

MDA

SUPPORT

MASHUP

WIDGETS

WEB ADMIN

REST ADMIN

SDK

CONSOLE

MONITOR

BUSINESS END2END

SOLUTIONS & SERVICES

BUSINESS

APPLICATIONS MIND

SUPPORT

FLEX-IT (IaaS)

3e

r P

art

y I

nte

gra

tio

n F

ac

ilit

ies

MOBILE

COMP

UI

COMP

SMART

COMP

SOFIA2 (IoT, ESB, CEP, BPM)

4

SOFIA (SMART OBJECTS FOR INTELLIGENT APPLICATIONS) emerges as a R&I Artemis project for a three years duration, (concluded in March 2012) with the participation of 19 partners from four EU countries, including Nokia, Philips, Fiat and Acciona

Mayor información en: http://www.sofia-project.eu/

INTRODUCTION TO SOFIA2 PLATFORM

THE ORIGIN OF SOFIA

The project budget was 36,5M euro where 8.9M were funded by partners’ countries and 6.1M euro by the EU.

5

Mayor información en: http://www.sofia-project.eu/

INTRODUCTION TO SOFIA2 PLATFORM

WHAT WAS INTENDED

ENTORNOS

INTELIGENTES

SERVICIOS

ADAPTABLES

INTERFACES

NATURALES

INFORMACIÓN

SEMÁNTICA

SOFIA can be defined as:

A middleware that provides seamless interoperability between

multiple devices and systems.

Offering a semantic interoperability platform, which allows the

exchange of information from the real world between smart

applications.

Contemplates standard definition of concepts (Things) in the

domain of the applications.

This way SOFIA is focused on be an Internet of Things platform.

All of that with an open source, multi-language and

communications agnostic approach.

6

INDRA participated in the work packages: WP3, WP4, WP5, WP6 and WP7

Achieve skills and knowledge in the following fields:

Smart Cities

Interoperability protocols (SSAP)

Semantics and Internet of Things

And obtains some products like:

Several implementations of the architecture solution (OSGI, C++, Ansi-C)

Productivity tools.

INTRODUCTION TO SOFIA2 PLATFORM INDRA IN SOFIA R&I PROJECT

7

Once the R&I project finished, Indra decided to continue reusing and improving the results of the project. This way, SOFIA was included in some commercial offers into the interoperability field.

In this sense stands up Zigamit project.

Zigamit is a project from Gas Natural-Fenosa (A leading gas and electricity utility in Spain) with Indra’s Energy area participation. SOFIA was included into the final solution as gathering of information layer

Mayor información en: http://www.sofia-project.eu/

Indra + SOFIA (Zigamit)

SOFIA acts as gathering of information layer from sensors and actuators, providing the information to the application.

INTRODUCTION TO SOFIA2 PLATFORM

8

Indra + SOFIA (Smart Coruña)

One of the main offers including SOFIA as middleware solution is the proposal for the RFP of the Smart Coruña project

With the objective to apply the innovation in new technologies in order to improve the public governance, the City Council of La Coruña takes position to deploy an Internet of Things (IoT) platform, supporting the development of several pilots as base to develop the public services in the future..

Indra, in consortium with other companies propose a technological offer to develop the platform

The offer includes as gathering, interconnectivity and event correlation middleware a new enterprise view of SOFIA called SOFIA2.

In February 2013, the offer from the consortium leaded by Indra is selected to develop the project.

INTRODUCTION TO SOFIA2 PLATFORM

SOFIA EVOLUTION

+ =

INTRODUCTION TO SOFIA2 PLATFORM

Indra takes advantage of the knowledge acquired during the ARTEMIS project, and creates SOFIA2 as infrastructure to develop Smart Cities.

SOFIA Enterprise Vision by incorporating the following features into its capabilities:

Big Data Interfaces (Hadoop) for storage and data warehouse.

Temporary reasoning capabilities.

Integration with back-ends.

Storage and geographical queries.

Addition of pluggable security mechanism

Multiplatform:

REST interfaces: smart phones, devices,…

Platform simplification:

Increasing SSAP communication protocol with a JSON-based protocol: lighter than XML and suitable for devices.

Simplification in the development of ontologies and query language.

Horizontal scaling:

By adding elements that scale horizontally at every level (DataGrids, Storage, Processing…)

Personalization & Adaptation

EVOLUTION OF SOFIA: SOFIA2

INTRODUCTION TO SOFIA PLATFORM

SOFIA2: NEW STANDARDS I

SOFIA2 includes new standards to extend the capabilities of the platform, making of SOFIA2 an enterprise product ready to use in real smart city environments. These standards include:

JSON (JavaScript Object Notation):

Text format to exchange information between systems. Lightweight and suitable for devices (Arduino, smart phones, …)

Standard in Web 2.0 (used by Google, Facebook, …)

Many Open DATA platforms are JSON-based

REST and RESTful services:

Web APIs as evolution of SOA services (API Manager Vs. Login + ESB)

Standard in Web 2.0 to consume information (Google, Facebook, Twitter, …)

Suitable for browsers, devices...

Hadoop:

Synonymous with Big Data (HDFS + Map Reduce)

Open source product + commercial distributions

Connectors from commercial providers

Used by Facebook, Yahoo!, Nokia, Amazon…

INTRODUCTION TO SOFIA2 PLATFORM

SOFIA KEY CONCEPTS I: Ontologies

INTRODUCTION TO SOFIA2 PLATFORM

Ontology represents the Definition of an Entity into my system. Make a standard classification of the information exchanged between devices/systems.

Can be pre-defined (Templates) or be created ad-hoc for a project.

Defined in JSON-Schema

Ontology instance represents a concrete entity in the system.

Defined in JSON

SOFIA KEY CONCEPTS II

INTRODUCTION TO SOFIA2 PLATFORM

Ontology instance: JSON:

SensorTemperatura

Ontology: Schema JSON:

SensorTemperatura Schema

Smart Space:

Collaborative, virtual environment where devices and applications inter-operate to offer a complex functionality

Knowledge Processor (KP):

Represents each element which communicates with a Smart Space by producing and/or consuming information.

Semantic Information Broker (SIB):

Represents the Smart Space's core. Integrates the semantic information that KP's exchange with each other; and stores information.

SOFIA KEY CONCEPTS III

INTRODUCTION TO SOFIA2 PLATFORM

Smart Space Access Protocol (SSAP): Standard messaging protocol used between the KP's and the SIB

JOIN, LEAVE

INSERT, UPDATE, REMOVE

QUERY, SUBSCRIBE, UNSUBSCRIBE

GET_VERSION, GET_CONFIG

SOFIA KEY CONCEPTS IV

INTRODUCTION TO SOFIA2 PLATFORM

OPERATING OF SOFIA2 PLATFORM

INTRODUCTION TO SOFIA2 PLATFORM

Interoperability Bus

KP

Connectors

Security

KP KP KP

KP

KP

Incidence

JSON

Luminosity

JSON

Biometrics Measurements

JSON

ON MP3

Subscribed to changes

Ontology definition

CDB RTDB HDB

Clients ands Authentication Tokens

KP

Seamless collaboration between any Thing (IoT)

SOFIA2 AS INTEGRATION PLATFORM

SOFIA Ontological Information: The definition of

the information entities is standard for all SmartCity domains.

Collaboration: A system which produces information into the platform, makes it available to other systems which can use it to their own business.

Semantic: Each system will interpret the normalized information according to their business logic.

Systems Integration at information level

To integrate several systems it is enough with develop them like KPs and connect them to the platform :

• No coupling: Systems don’t need to know the other systems they interoperate with.

• Allows to follow an incremental model to deploy system/applications. New systems/applications will be compatible with existing ones because they use the same information structures.

Knowledge Processor (KP) Producer/Consumer of Information: a KP is

any user, device, application or system that produces or consumes any kind of information into the SOFIA platform

SSAP: Makes standard the way to interoperate with the platform and thus with other KPs. It is agnostic from the programming language and communications protocol.

INTRODUCTION TO SOFIA2 PLATFORM

Traffic Management Dashboard reports a traffic accident to the platform.

Other dashboards exploit the information:

Urban Assets Management Dashboard generates an alert to inspect the area to identify damages.

Urban cleaning Dashboard analyzes the information and in case of crash generates an alert to send cleaning stuff to the area.

SIB

TRAFFIC MANAGEMENT DASHBOARD

URBAN ASSETS MANAGEMENT DASHBOARD

URBAN CLEANING DASHBOARD

INTEGRATION USE OF CASE WITH SOFIA2

INTRODUCTION TO SOFIA2 PLATFORM

Traffic incidence notification to the plaform from Traffic Management Dashboard.

SIB

TRAFIC MANAGEMENT DASHBOARD

1. Traffic accident happen

2. Notification of the incidence to the SIB

Ontology TrafficIncidence: • Geographyc Location • Vehicle number • Gravity level

·············

INTEGRATION USE OF CASE WITH SOFIA

INTRODUCTION TO SOFIA2 PLATFORM

SIB

URBAN ASSETS MANAGEMENT

DASHBOARD

URBAN CLEANING DASHBOARD

5. Generate Alert to inspect damages in urban assets

5. Generate Incidence to send cleaning staff

4. Analysis of the information received from SIB

INTEGRATION USE OF CASE WITH SOFIA

Incidence information reception by the other systems, analysis of the information and actuation according to each system logic.

3. SIB notify to Suscriptors the traffic incidence

INTRODUCTION TO SOFIA2 PLATFORM

21

CONTENTS

01 Introduction to SOFIA2 Platform

02 SOFIA2 Platform in detail

03 SOFIA2 Capabilities at a glance

SOFIA2 PLATFORM IN DETAIL

LOGIC ORGANIZATION

De

plo

ym

en

t

(On

Pre

mis

e &

Clo

ud

)

Re

al T

ime

DB

SD

K

Clients (Consumer / Publisher)

Security

Semantic Information Broker

His

tori

cal D

ata

ba

se

Co

nfi

g D

B

Query & SQL Engine

Suscription Engine

Rules&Scripting Engine

CEP Engine

Management & Config UI & API …

Connector Adapters

… …

Plugins

It is in charge of:

Publishing information to the interoperability bus. Information can be provided by a sensor, a smart phone…

Consuming information from the interoperability bus via either specific queries (last value of sensors near my position) or by using subscribe/notify logic (notify me whenever a temperature sensor in the city reports less than 0º C)

In detail:

This concept is called KP (Knowledge Processor)

Communicates with the bus (SIB) using the SSAP protocol language (Smart Space Access Protocol)

Applies configured security mechanism in communication.

Offers API’s for several languages/platforms: Java, Javascript, iOS, Android, Arduino,…

LOGIC ORGANIZATION: CONSUMER/PUBLISHER

SOFIA2 PLATFORM IN DETAIL

It is in charge of:

Establishing authentication/authorization mechanisms between the platform and consumer/publisher

Communication Protocol security and application-layer security.

In detail:

2 Implementations out-of-the-box:

Based on user/password

Based on authentication Token

Extension mechanism via plugins to create/adapt security to any requirement: Token, Oauth, certificates…

Security involves all concepts in the platform (ontologies, KPs,…)

LOGIC ORGANIZATION: SECURITY

SOFIA2 PLATFORM IN DETAIL

It is in charge of:

Providing protocol adaptation mechanisms to communicate from clients:

REST: appropriate for Javascript clients, smartphones…

MQTT: appropriate for bidirectional communications and low resources devices.

Web Services/JMS: appropriate for enterprise applications.

Checking client’s permission to consume or publish data.

Storing data in Persistence Module (RTDB + HDB)

Routing requests to the proper engine depending on its specifics

In detail:

This concept is called SIB (Semantic Information Broker)

Clients (KPs)can:

JOIN, LEAVE: Open/close a logic session with the platform.

INSERT, UPDATE y REMOVE: Produce and modify information in the platform.

QUERY, SUBSCRIBE: Consume information via query or subscription to changes.

GET_VERSION, GET_CONFIG: Get from SIB centralized configuration for KP.

LOGIC ORGANIZATION: INTEROPERABILITY BUS

SOFIA2 PLATFORM IN DETAIL

It is in charge of:

Provide extension mechanisms to the Platform

Via plugins, the platform can be extended with new connectors, new authentication/authorization methods, audit information, store information in other datastores, generate KPIs, monitor the platform,…

In detail:

Allows to pre and post-process any action into the SIB.

Security Plugin: Provides a mechanism to modify or change the authentication and authorization model.

Life-cicle Plugin Allows to execute new actions during the starting and stopping of the SIB.

SSAP Pre/Post-Process: Allows to pre-process an incoming SSAP message to the SIB and post-process the SSAP response from the SIB. The pre/post process operation can include invocation to other Backends.

Engine Plugin: Allows to intercept any action performed by an engine.

LOGIC ORGANIZATION: PLUGINS CONTAINER

SOFIA2 PLATFORM IN DETAIL

They are in charge of:

Processing the requests reaching the Broker.

In detail:

• Query Engine: Makes queries to the RTDB and to the HDB in a SQL-Like language which is independent from the database engine. Provides geospatial query capabilities

• Subscription Engine: Provides capabilities to subscribe to an event in an ontology. Thus, when the event occurs the engine notify it to the subscriber.

• Scripting Rules Engine: Provides a way to define rules in the platform in an easy way via the UI. These rules will be executed when certain conditions happens (arrival of a certain type of data that meets certain rule) and allow to execute a set of preconfigured and expandable actions.

CEP Engine: Provides a way to define CEP Rules to be applied to incoming data and trigger notifications to subscribers. (e.g. The average of temperature measures is bigger 50º C, or a kind event have not occur in 1 day)

LOGIC ORGANIZATION: ENGINES

SOFIA2 PLATFORM IN DETAIL

It is in charge of:

The CDB (Configuration Database) stores all configuration data of the Platform.

In detail

Stores ontologies, ontology templates, CEP Rules, Scripting Rules, Predefined queries, KPs, Assets,...

The CDB can be any relational database with JDBC driver. Currently there are releases on MySQL and Oracle.

LOGIC ORGANIZATION: CDB (Configuration Database)

SOFIA2 PLATFORM IN DETAIL

It is in charge of:

The RTDB (Real Time Database) stores all ontology instances received by the Interoperability Bus from KPs during a time window considered as real time.

In detail:

For each ontology, the time window representing the real time can be configured. Once the real time window past, the data will be discarded or stored as Historic information (e.g.: A Temperature Sensor can have a real time window of 1 minute).

It is based on MongoDB that provides the following features::

Document-oriented storage (JSON)

High performance.

Complex queries, including geospatial queries.

Horizontal scaling, including sharding.

LOGIC ORGANIZATION: RTDB (Real Time Database)

SOFIA2 PLATFORM IN DETAIL

It is in charge of:

The HDB (Historic Database) stores all data that currently are not into the real time window of the ontology but can be exploited for historical purposes.

In detail:

The HDB is based on Hadoop and the datawarehouse HIVE. This provides:

Unlimited storage capabilities.

Process capabilities to exploit the stored information when needed.

Query language based on SQL.

Easy extension with Impala to support on-line queries.

Real time data are moved to Historic data automatically.

Historic data can be requested by clients (KPs) using a SSAP Query message.

LOGIC ORGANIZATION: HDB (Historic Database)

SOFIA2 PLATFORM IN DETAIL

It is in charge of:

Provide an easy way to manage all concepts and configurations in the platform.

In Detail:

UI Web using HTML5 and Spring technologies.

REST API to manage the configuration or create new UIs.

Authorization control in UI and API to restrict the actions that a user/role can manage in the configuration.

Adaptable and extendable UI according to the project requirements.

LOGIC ORGANIZATION: UI + Management API

SOFIA2 PLATFORM IN DETAIL

It is in charge of:

Provide APIs in several programming languages to develop clients.

Provide Productivity Tools to make agile development of clients.

Provide APIs to develop plugins to extend the SIB being deployed in the plugins container.

Provide Guides, Examples and Documentation to make easier the learning of the concepts and APIs.

In detail:

Multilanguage KPs API: Java, Javascript, Arduino, Android, C…

API to extend the Broker developing plugins

Development Environment for local testing: SIB+KPs

Open cloud environment to make integration tests.

Eclipse plugin with wizards to easy create KPs.

User guides + examples.

LOGIC ORGANIZATION: DEVELOPMENT KIT

SOFIA2 PLATFORM IN DETAIL

It is in charge of:

Ensure the compatibility of the platform with the leading Cloud platforms.

Provide to the platform with an elastic providing of resources according to its load at any moment.

In detail:

Deployment in several IassS/PaaS: Amazon, Azure, Jelastic ,…

Deployment in Indra’s PaaS platform: G@PASS

Deployment in private or public cloud.

LOGIC ORGANIZATION: CLOUD

SOFIA2 PLATFORM IN DETAIL

SOFIA2 LICENSES MODEL

SOFIA2 can be used under 2 licenses model

Community License

AGPL open-source license v2.

No cost per use.

Base version of the operative platform.

Not includes Web management console neither advance tools (Scripting Rules and CEP).

Source code available in SOFIA2 Forge: http://sofia2.org

Includes use guides and SDK http://sofia2.com/

Enterprise License

License with no AGPL license restriction.

Commercial support under several levels and SLAs.

Complete Platform including all tools.

Includes use, installation and customization guides.

SDK including Hadoop and BI support.

SOFIA2 PLATFORM IN DETAIL

SOFIA2 LICENSES

SOFIA2 PLATFORM IN DETAIL

SOFIA2 LICENSES

SOFIA2 PLATFORM IN DETAIL

The RI (Reference Implementation) of the SIB is based on these technologies:

Spring 3.X development framework

iCore4TR as Technical Architecture

Apache CXF supporting REST/Web Services

Siddhi CEP (WSO2 CEP) as embedded CEP

Hazelcast as DataGrid supporting HA SIBs

MongoDB as RTDB (Real Time Database)

Hadoop (Cloudera/Pivotal HD/…) as HDB (Historic Database)

Hive as SQL engine in Hadoop

MySQL/BD JDBC as CDB (Configuration database)

MQTT/JMS/Sockets/REST as communication protocols for KP

JMX as monitoring protocol

Spring MVC + Thymeleaf as Web framework for Web console UI

SOFIA2 PLATFORM IN DETAIL Technologies of the SIB RI (Reference Implementation)

SOFIA2 PLATFORM IN DETAIL

config server

mongos

SIB CLUSTER

SIB SIB SIB

MONGO DB Cluster

config server

mongos

config server

mongos

3 x Node MongoDB: • 32 GB of RAM • 1 CPU Intel Xeon E5 (2-2.5GHz 6/8 cores) • 2 TB Raid-10 Ext4 o XFS • Linux OS 64 bits (CentOS 6.x)

Namenode +JobTracker

(Master)

HADOOP

Datanode + TaskTracker

(Slave)

1 x NameNode Hadoop: • 32 GB RAM • 2 quad core CPUs • 1 TB RAID • Linux OS 64 bits

3 x Datanode Hadoop: • 16 Gb RAM • 4 discos x 1 TB (conf JBOD) • 2 quad core CPUs (2-2.5GHz) • Linux OS 64 bits

3 x Node SIB+Grid: • 16 GB of RAM • 1 CPU Intel Xeon E5 (2-2.5GHz 6/8 cores) • 100-200 Gb RAID • Linux OS 64 bits (CentOS 6.x)

DATA GRID

SOFIA2 Desployment proposal BALANCER

Balancer HW o SW: • F5,… • gINX, HAProxy

DN DN

SOFIA2 PLATFORM IN DETAIL

BDC

config server

mongos

SIB RUNTIME

CLUSTER SIB SIB

RUNTIME

SIB RUNTIME

MONGO DB Sharded Cluster

config server

mongos config

server mongos

Namenode +JobTracker

(Master)

HADOOP

Datanode + TaskTracker

(Slave)

Datanode + TaskTracker

(Slave)

DATA GRID

BALANCER

CDB

SIB CONSOLE

WEB

SIB CONSOLE

WEB

CLUSTER CONSOLE

WEB

SIB TOOLS

SIB TOOLS

CLUSTER SIB

TOOLS

SOFIA2 PLATFORM IN DETAIL

WARNING NOTIFIER

···

ENTERPRISE BUS (ESB)

SMART PHONE

KP

1

KP

WARNINGS SYSTEM

Service

Company

KP

···

GR

EE

N H

ER

ITA

GE

UR

BA

N M

OB

ILIT

Y

IRR

IGA

TIO

N

WARNINGS SERVICE

2

3

4

4 4

5

Service

Company

KP

5

SIB SIB SmartSpace

Gathering

BACKEND INT.

MODULE

1. A citizen send a warning alert of a dead tree.

2. The warning is processed by the warnings service. 3. The warnings system notifies to the orchestration system. 4. The orchestration system notifies the warning alert to all systems involved. 5. The service concession company makes the service removing the tree 6. The City Council checks the realization of the works Systems

integrated with SOFIA

Systems NO integrated with SOFIA

EXAMPLE OF URBAN WARNING

SOFIA2 PLATFORM IN DETAIL

DESPLOYMENT IN CITY USING ENTERPRISE BUS

H.A. SIB

TRAFFIC CAMERA

KP

ILLUMINATION

KP

···

SMART PHONE

KP

TABLET

KP ARDUINO/ RASPBERR

Y

KP

IRRIGATION GATEWAY

KP

Sprinkler network

Drip netw

Urban Mobility

Dashboard

KP

RTDB

ontologíes

<<tabla>>

BIGDATA INFRASTRUCTURE

HDFS Map Reduce

DB NoSQL HIVE

HISTORIC MEASURES SIB AREA ADAPTER

KP WARNINGSSERVICE

SERVICE XXX Ontology

x Ontology

y

KP KP

Dispatcher

SYSTEMS DEVELOPED AS

SOFIA KPS

SYSTEMS NOT SOFIA BUT USING

ONTOLOGIES

SYSTEMS NOT INTEGRATED

KP KP

BUSINESS INTELLIGENCE

···

ENTERPRISE BUS (ESB)

MANAGEMENT UI

···

INDICATORS

BDH BDCONF

users

<<tabla>>

SIB

···

PROTOCOLS GATEWAY

BACKEND INT. MODULE

PROTOCOLS GATEWAY

CEP + SCRIPT

Smart Space

SERVICE YYY

KP

OPEN DATA

42

CONTENTS

01 Introduction to SOFIA2 Platform

02 SOFIA2 Platform in detail

03 SOFIA2 Capabilities at a glance

PERSISTENCE > RTDB

The RTDB (Real Time Database) stores all the information into the time window considered as real time for each specific ontology (it is configurable for each ontology).

The RTDB for the Reference Implementation is based on MongoDB:

MongoDB is a NoSQL database, oriented to document storage, and designed to be fast, scalable and easy to use.

The NoSQL databases are developed to support increasing requirements in terms of performance, scalability and fast response.

The NoSQL databases in comparision with relational databases, do not require static data models to store the information.

It is appropiate for applications using high volume of information or complex data

SOFIA2 provides an SQL query engine over MongoDB.

Support to execute geographical queries

SOFIA2 CAPABILITIES AT A GLANCE

PERSISTENCE > RTDB

db.pt_bus.find({"MobileElement.geometry" : {$near : {$geometry : {type : "Point", coordinates : [-8.396402999999964, 43.37016999999999]}}, $maxDistance: 2000}} )

SOFIA2 CAPABILITIES AT A GLANCE

PERSISTENCE > HDB OVER HADOOP

The high volume of information that the platform has to handle makes necessary the use of Big Data approaches, in our case, Hadoop:

Hadoop is used to store and to make further processing (datawarehouse) of all data received by the platform, allowing indeterminate growth.

The features that makes Hadoop synonymous with Big Data are::

Economic: it is deployed in commodity hardware creating clusters.

Scalable: If more processing or storage capabilities are needed, the cluster can be extended adding more nodes.

Efficient: Hadoop distributes the data between all nodes and process them in parallel into the node where they are stored. In this sense, Hadoop moves processing (Tasks) where the information is stored.

Reliable: Hadoop File System (HDFS) is able to maintain multiple copies of the data in several nodes. If one node falls out, the information stored in it can be found in another one.

SOFIA 2 integrates with Hadoop infrastructure in several ways::

HDFS: As filesystem to store information without processing.

HBASE: As NoSQL database to store historic information.

Sqoop: Providing capabilities to import/export data from/to relational databases.

Hive+Impala: Providing capabilities to query information from HDFS and HBASE using SQL language.

SOFIA2 CAPABILITIES AT A GLANCE

HDFS: Is the filesystem of Hadoop.

HDFS is a distributed filesystem that provides an integrated view of all storage resources into the Hadoop cluster, abstracting this logical view from the physical storage.

When HDFS stores a file, it splits the file in several blocks, storing each block in separated nodes, and duplicating it in at least three nodes.

This makes possible to store files bigger than the maximum disk size of any of the nodes in the cluster.

If a node falls out, the cluster will continue working using the duplicated information in other nodes.

Node 1 Node 2 Node n ·····

Hadoop HDFS

····· Block 1 Block 2 Block n

Node X Node Y Node Z

Hadoop HDFS

Block

Partitioning Duplication

PERSISTENCE > HDB OVER HADOOP

SOFIA2 CAPABILITIES AT A GLANCE

MapReduce is a distributed solution to process large volumes of information:

It is based on Google works.

It works in commodity hardware.

HDFS and MapReduce are the core of Hadoop.

PERSISTENCE > HDB OVER HADOOP

SOFIA2 CAPABILITIES AT A GLANCE

DRONE DEPLOYEMENT VIEW

Semantic Information Broker

Connectors

Security

KP

Subscribed to changes

Ontologies definition

CDB RTDB HDB

Clients and Tokens definition

KP

Commands

JSON Commands KP

• DroneCameraCommand • DroneMovementCommand • DroneCameraScriptStream • KP_Drone_Joystick

• KP_Drone_Controller

HTTP MQTT

http://sofia2.com

SOFIA2 CAPABILITIES AT A GLANCE

SOFIA2 CAPABILITIES AT A GLANCE

WORKFLOW

SOFIA2 CAPABILITIES AT A GLANCE

Public Web site providing access to:

Presentations

Guides.

Videotutorials.

SDK.

Forge.

PUBLIC INFORMATION

WEB CONFIGURATION CONSOLE

SOFIA2 CAPABILITIES AT A GLANCE

Allows to manage all conceps of the platform via Web:

Management of Ontologies

Management of KPS

Management of Assets

Databases console

Management of Rules

MANAGEMENT RESTFUL API

SOFIA2 CAPABILITIES AT A GLANCE

CONNECTORS

SOFIA2 CAPABILITIES AT A GLANCE

Connectors

REST

Web Services

MQTT

WebSockets

JMS

SDK

SOFIA2 CAPABILITIES AT A GLANCE

Forge .

Multiplataform SDK

Productivity tools to create KPs

Documentation and guides.

Videotutorials.

WEB CONSOLE KP CONTROL

KP Sensor Gateway connected to a Zigbee Gateway (T, H, Wh)

SOFIA2 CAPABILITIES AT A GLANCE

SOFIA provides tools to automate KPs creation, generating a Web UI control

UI HTML + Javascript

Web Javascript visualization KP (T, H, Wh)

SOFIA2 CAPABILITIES AT A GLANCE

SOFIA provides a Javascript API to interoperate with other KPs using Ajax Push support

Arduino KP publishing luminosity measures

SOFIA2 CAPABILITIES AT A GLANCE

Web Javascript KP showing luminosity from Arduino

SOFIA2 CAPABILITIES AT A GLANCE

ONTOLOGY TEMPLATES

SOFIA2 CAPABILITIES AT A GLANCE

SCRIPTING RULES ENGINE

SOFIA2 CAPABILITIES AT A GLANCE

SIB MONITORING

SOFIA2 CAPABILITIES AT A GLANCE

SOFIA monitors all its components via JMX.

Also Provides a monitoring REST service.

And a HTTP adaptor via JMX.

http://sofia2.com

We are waiting for you!!!