LOREM IPSUM DOLOR SIT AMET. - SOFIA2sofia2.com/docs/SOFIA2 - SOFIA2 as Interoperability...
-
Upload
nguyenhanh -
Category
Documents
-
view
212 -
download
0
Transcript of LOREM IPSUM DOLOR SIT AMET. - SOFIA2sofia2.com/docs/SOFIA2 - SOFIA2 as Interoperability...
2
CONTENTS
01 Introduction to SOFIA2 Platform
02 SOFIA2 Platform in detail
03 SOFIA2 Capabilities at a glance
3
Indra already has mature assets to build this platform, to integrate with commercial solutions
and open source components:
iCloudBroker as the Marketplace solution to facilitate the Cloud commercialization of
application, services and data.
iDynamics as the corporate architecture, easing application development (traditional and
mobile) and services that can be deployed in the cloud.
SOFIA2, designed originally in the homonymous R&D project, covering IoT, M2M and Big
Data in real time. G@PaaS to extend the platform with capabilities to support business in Cloud
Mayor información en: http://www.sofia-project.eu/
INTRODUCTION TO SOFIA2 PLATFORM
ZOOM TO SOFIA2 INTO INDRA SMART PLATFORM
3
Product / Solution bundling
MARKETPLACE (iCloudBroker)
DEPLOYMENT &
INFRASTRUCTURE
STORAGE &
DATA MANAGEMENT
APPLICATIONS & SERVICES
TOOLS
INTEGRATION &
SECURITY
COMPONENTS & BUNDLES
IDYNAMICS ARCHITECTURE
BUNDLES
BUSINES
S
COMP ARQ
COMP
IBM
TIBCO
ORACLE
BIZTALK
WSO2
OAuth
ORACLE
SQLSERVER
HADOOP
NoSQL
SOFIA2 & Gdb (BIG DATA, DataGrid)
GPaaS (Provisioning, Multitenant) AZURE
AMAZON
COMPONENTS APPS DATA SERVICES PERSONALIZATION
IDE PLUGINS
REVERSE
ENG
UI GENERAT
MDA
SUPPORT
MASHUP
WIDGETS
WEB ADMIN
REST ADMIN
SDK
CONSOLE
MONITOR
BUSINESS END2END
SOLUTIONS & SERVICES
BUSINESS
APPLICATIONS MIND
SUPPORT
FLEX-IT (IaaS)
3e
r P
art
y I
nte
gra
tio
n F
ac
ilit
ies
MOBILE
COMP
UI
COMP
SMART
COMP
SOFIA2 (IoT, ESB, CEP, BPM)
4
SOFIA (SMART OBJECTS FOR INTELLIGENT APPLICATIONS) emerges as a R&I Artemis project for a three years duration, (concluded in March 2012) with the participation of 19 partners from four EU countries, including Nokia, Philips, Fiat and Acciona
Mayor información en: http://www.sofia-project.eu/
INTRODUCTION TO SOFIA2 PLATFORM
THE ORIGIN OF SOFIA
The project budget was 36,5M euro where 8.9M were funded by partners’ countries and 6.1M euro by the EU.
5
Mayor información en: http://www.sofia-project.eu/
INTRODUCTION TO SOFIA2 PLATFORM
WHAT WAS INTENDED
ENTORNOS
INTELIGENTES
SERVICIOS
ADAPTABLES
INTERFACES
NATURALES
INFORMACIÓN
SEMÁNTICA
SOFIA can be defined as:
A middleware that provides seamless interoperability between
multiple devices and systems.
Offering a semantic interoperability platform, which allows the
exchange of information from the real world between smart
applications.
Contemplates standard definition of concepts (Things) in the
domain of the applications.
This way SOFIA is focused on be an Internet of Things platform.
All of that with an open source, multi-language and
communications agnostic approach.
6
INDRA participated in the work packages: WP3, WP4, WP5, WP6 and WP7
Achieve skills and knowledge in the following fields:
Smart Cities
Interoperability protocols (SSAP)
Semantics and Internet of Things
And obtains some products like:
Several implementations of the architecture solution (OSGI, C++, Ansi-C)
Productivity tools.
INTRODUCTION TO SOFIA2 PLATFORM INDRA IN SOFIA R&I PROJECT
7
Once the R&I project finished, Indra decided to continue reusing and improving the results of the project. This way, SOFIA was included in some commercial offers into the interoperability field.
In this sense stands up Zigamit project.
Zigamit is a project from Gas Natural-Fenosa (A leading gas and electricity utility in Spain) with Indra’s Energy area participation. SOFIA was included into the final solution as gathering of information layer
Mayor información en: http://www.sofia-project.eu/
Indra + SOFIA (Zigamit)
SOFIA acts as gathering of information layer from sensors and actuators, providing the information to the application.
INTRODUCTION TO SOFIA2 PLATFORM
8
Indra + SOFIA (Smart Coruña)
One of the main offers including SOFIA as middleware solution is the proposal for the RFP of the Smart Coruña project
With the objective to apply the innovation in new technologies in order to improve the public governance, the City Council of La Coruña takes position to deploy an Internet of Things (IoT) platform, supporting the development of several pilots as base to develop the public services in the future..
Indra, in consortium with other companies propose a technological offer to develop the platform
The offer includes as gathering, interconnectivity and event correlation middleware a new enterprise view of SOFIA called SOFIA2.
In February 2013, the offer from the consortium leaded by Indra is selected to develop the project.
INTRODUCTION TO SOFIA2 PLATFORM
Indra takes advantage of the knowledge acquired during the ARTEMIS project, and creates SOFIA2 as infrastructure to develop Smart Cities.
SOFIA Enterprise Vision by incorporating the following features into its capabilities:
Big Data Interfaces (Hadoop) for storage and data warehouse.
Temporary reasoning capabilities.
Integration with back-ends.
Storage and geographical queries.
Addition of pluggable security mechanism
Multiplatform:
REST interfaces: smart phones, devices,…
Platform simplification:
Increasing SSAP communication protocol with a JSON-based protocol: lighter than XML and suitable for devices.
Simplification in the development of ontologies and query language.
Horizontal scaling:
By adding elements that scale horizontally at every level (DataGrids, Storage, Processing…)
Personalization & Adaptation
EVOLUTION OF SOFIA: SOFIA2
INTRODUCTION TO SOFIA PLATFORM
SOFIA2: NEW STANDARDS I
SOFIA2 includes new standards to extend the capabilities of the platform, making of SOFIA2 an enterprise product ready to use in real smart city environments. These standards include:
JSON (JavaScript Object Notation):
Text format to exchange information between systems. Lightweight and suitable for devices (Arduino, smart phones, …)
Standard in Web 2.0 (used by Google, Facebook, …)
Many Open DATA platforms are JSON-based
REST and RESTful services:
Web APIs as evolution of SOA services (API Manager Vs. Login + ESB)
Standard in Web 2.0 to consume information (Google, Facebook, Twitter, …)
Suitable for browsers, devices...
Hadoop:
Synonymous with Big Data (HDFS + Map Reduce)
Open source product + commercial distributions
Connectors from commercial providers
Used by Facebook, Yahoo!, Nokia, Amazon…
INTRODUCTION TO SOFIA2 PLATFORM
SOFIA KEY CONCEPTS I: Ontologies
INTRODUCTION TO SOFIA2 PLATFORM
Ontology represents the Definition of an Entity into my system. Make a standard classification of the information exchanged between devices/systems.
Can be pre-defined (Templates) or be created ad-hoc for a project.
Defined in JSON-Schema
Ontology instance represents a concrete entity in the system.
Defined in JSON
SOFIA KEY CONCEPTS II
INTRODUCTION TO SOFIA2 PLATFORM
Ontology instance: JSON:
SensorTemperatura
Ontology: Schema JSON:
SensorTemperatura Schema
Smart Space:
Collaborative, virtual environment where devices and applications inter-operate to offer a complex functionality
Knowledge Processor (KP):
Represents each element which communicates with a Smart Space by producing and/or consuming information.
Semantic Information Broker (SIB):
Represents the Smart Space's core. Integrates the semantic information that KP's exchange with each other; and stores information.
SOFIA KEY CONCEPTS III
INTRODUCTION TO SOFIA2 PLATFORM
Smart Space Access Protocol (SSAP): Standard messaging protocol used between the KP's and the SIB
JOIN, LEAVE
INSERT, UPDATE, REMOVE
QUERY, SUBSCRIBE, UNSUBSCRIBE
GET_VERSION, GET_CONFIG
SOFIA KEY CONCEPTS IV
INTRODUCTION TO SOFIA2 PLATFORM
OPERATING OF SOFIA2 PLATFORM
INTRODUCTION TO SOFIA2 PLATFORM
Interoperability Bus
KP
Connectors
Security
KP KP KP
KP
KP
Incidence
JSON
Luminosity
JSON
Biometrics Measurements
JSON
ON MP3
Subscribed to changes
Ontology definition
CDB RTDB HDB
Clients ands Authentication Tokens
KP
Seamless collaboration between any Thing (IoT)
SOFIA2 AS INTEGRATION PLATFORM
SOFIA Ontological Information: The definition of
the information entities is standard for all SmartCity domains.
Collaboration: A system which produces information into the platform, makes it available to other systems which can use it to their own business.
Semantic: Each system will interpret the normalized information according to their business logic.
Systems Integration at information level
To integrate several systems it is enough with develop them like KPs and connect them to the platform :
• No coupling: Systems don’t need to know the other systems they interoperate with.
• Allows to follow an incremental model to deploy system/applications. New systems/applications will be compatible with existing ones because they use the same information structures.
Knowledge Processor (KP) Producer/Consumer of Information: a KP is
any user, device, application or system that produces or consumes any kind of information into the SOFIA platform
SSAP: Makes standard the way to interoperate with the platform and thus with other KPs. It is agnostic from the programming language and communications protocol.
INTRODUCTION TO SOFIA2 PLATFORM
Traffic Management Dashboard reports a traffic accident to the platform.
Other dashboards exploit the information:
Urban Assets Management Dashboard generates an alert to inspect the area to identify damages.
Urban cleaning Dashboard analyzes the information and in case of crash generates an alert to send cleaning stuff to the area.
SIB
TRAFFIC MANAGEMENT DASHBOARD
URBAN ASSETS MANAGEMENT DASHBOARD
URBAN CLEANING DASHBOARD
INTEGRATION USE OF CASE WITH SOFIA2
INTRODUCTION TO SOFIA2 PLATFORM
Traffic incidence notification to the plaform from Traffic Management Dashboard.
SIB
TRAFIC MANAGEMENT DASHBOARD
1. Traffic accident happen
2. Notification of the incidence to the SIB
Ontology TrafficIncidence: • Geographyc Location • Vehicle number • Gravity level
·············
INTEGRATION USE OF CASE WITH SOFIA
INTRODUCTION TO SOFIA2 PLATFORM
SIB
URBAN ASSETS MANAGEMENT
DASHBOARD
URBAN CLEANING DASHBOARD
5. Generate Alert to inspect damages in urban assets
5. Generate Incidence to send cleaning staff
4. Analysis of the information received from SIB
INTEGRATION USE OF CASE WITH SOFIA
Incidence information reception by the other systems, analysis of the information and actuation according to each system logic.
3. SIB notify to Suscriptors the traffic incidence
INTRODUCTION TO SOFIA2 PLATFORM
21
CONTENTS
01 Introduction to SOFIA2 Platform
02 SOFIA2 Platform in detail
03 SOFIA2 Capabilities at a glance
SOFIA2 PLATFORM IN DETAIL
LOGIC ORGANIZATION
De
plo
ym
en
t
(On
Pre
mis
e &
Clo
ud
)
Re
al T
ime
DB
SD
K
Clients (Consumer / Publisher)
Security
Semantic Information Broker
His
tori
cal D
ata
ba
se
Co
nfi
g D
B
Query & SQL Engine
Suscription Engine
Rules&Scripting Engine
CEP Engine
Management & Config UI & API …
Connector Adapters
… …
Plugins
It is in charge of:
Publishing information to the interoperability bus. Information can be provided by a sensor, a smart phone…
Consuming information from the interoperability bus via either specific queries (last value of sensors near my position) or by using subscribe/notify logic (notify me whenever a temperature sensor in the city reports less than 0º C)
In detail:
This concept is called KP (Knowledge Processor)
Communicates with the bus (SIB) using the SSAP protocol language (Smart Space Access Protocol)
Applies configured security mechanism in communication.
Offers API’s for several languages/platforms: Java, Javascript, iOS, Android, Arduino,…
LOGIC ORGANIZATION: CONSUMER/PUBLISHER
SOFIA2 PLATFORM IN DETAIL
It is in charge of:
Establishing authentication/authorization mechanisms between the platform and consumer/publisher
Communication Protocol security and application-layer security.
In detail:
2 Implementations out-of-the-box:
Based on user/password
Based on authentication Token
Extension mechanism via plugins to create/adapt security to any requirement: Token, Oauth, certificates…
Security involves all concepts in the platform (ontologies, KPs,…)
LOGIC ORGANIZATION: SECURITY
SOFIA2 PLATFORM IN DETAIL
It is in charge of:
Providing protocol adaptation mechanisms to communicate from clients:
REST: appropriate for Javascript clients, smartphones…
MQTT: appropriate for bidirectional communications and low resources devices.
Web Services/JMS: appropriate for enterprise applications.
Checking client’s permission to consume or publish data.
Storing data in Persistence Module (RTDB + HDB)
Routing requests to the proper engine depending on its specifics
In detail:
This concept is called SIB (Semantic Information Broker)
Clients (KPs)can:
JOIN, LEAVE: Open/close a logic session with the platform.
INSERT, UPDATE y REMOVE: Produce and modify information in the platform.
QUERY, SUBSCRIBE: Consume information via query or subscription to changes.
GET_VERSION, GET_CONFIG: Get from SIB centralized configuration for KP.
LOGIC ORGANIZATION: INTEROPERABILITY BUS
SOFIA2 PLATFORM IN DETAIL
It is in charge of:
Provide extension mechanisms to the Platform
Via plugins, the platform can be extended with new connectors, new authentication/authorization methods, audit information, store information in other datastores, generate KPIs, monitor the platform,…
In detail:
Allows to pre and post-process any action into the SIB.
Security Plugin: Provides a mechanism to modify or change the authentication and authorization model.
Life-cicle Plugin Allows to execute new actions during the starting and stopping of the SIB.
SSAP Pre/Post-Process: Allows to pre-process an incoming SSAP message to the SIB and post-process the SSAP response from the SIB. The pre/post process operation can include invocation to other Backends.
Engine Plugin: Allows to intercept any action performed by an engine.
LOGIC ORGANIZATION: PLUGINS CONTAINER
SOFIA2 PLATFORM IN DETAIL
They are in charge of:
Processing the requests reaching the Broker.
In detail:
• Query Engine: Makes queries to the RTDB and to the HDB in a SQL-Like language which is independent from the database engine. Provides geospatial query capabilities
• Subscription Engine: Provides capabilities to subscribe to an event in an ontology. Thus, when the event occurs the engine notify it to the subscriber.
• Scripting Rules Engine: Provides a way to define rules in the platform in an easy way via the UI. These rules will be executed when certain conditions happens (arrival of a certain type of data that meets certain rule) and allow to execute a set of preconfigured and expandable actions.
CEP Engine: Provides a way to define CEP Rules to be applied to incoming data and trigger notifications to subscribers. (e.g. The average of temperature measures is bigger 50º C, or a kind event have not occur in 1 day)
LOGIC ORGANIZATION: ENGINES
SOFIA2 PLATFORM IN DETAIL
It is in charge of:
The CDB (Configuration Database) stores all configuration data of the Platform.
In detail
Stores ontologies, ontology templates, CEP Rules, Scripting Rules, Predefined queries, KPs, Assets,...
The CDB can be any relational database with JDBC driver. Currently there are releases on MySQL and Oracle.
LOGIC ORGANIZATION: CDB (Configuration Database)
SOFIA2 PLATFORM IN DETAIL
It is in charge of:
The RTDB (Real Time Database) stores all ontology instances received by the Interoperability Bus from KPs during a time window considered as real time.
In detail:
For each ontology, the time window representing the real time can be configured. Once the real time window past, the data will be discarded or stored as Historic information (e.g.: A Temperature Sensor can have a real time window of 1 minute).
It is based on MongoDB that provides the following features::
Document-oriented storage (JSON)
High performance.
Complex queries, including geospatial queries.
Horizontal scaling, including sharding.
LOGIC ORGANIZATION: RTDB (Real Time Database)
SOFIA2 PLATFORM IN DETAIL
It is in charge of:
The HDB (Historic Database) stores all data that currently are not into the real time window of the ontology but can be exploited for historical purposes.
In detail:
The HDB is based on Hadoop and the datawarehouse HIVE. This provides:
Unlimited storage capabilities.
Process capabilities to exploit the stored information when needed.
Query language based on SQL.
Easy extension with Impala to support on-line queries.
Real time data are moved to Historic data automatically.
Historic data can be requested by clients (KPs) using a SSAP Query message.
LOGIC ORGANIZATION: HDB (Historic Database)
SOFIA2 PLATFORM IN DETAIL
It is in charge of:
Provide an easy way to manage all concepts and configurations in the platform.
In Detail:
UI Web using HTML5 and Spring technologies.
REST API to manage the configuration or create new UIs.
Authorization control in UI and API to restrict the actions that a user/role can manage in the configuration.
Adaptable and extendable UI according to the project requirements.
LOGIC ORGANIZATION: UI + Management API
SOFIA2 PLATFORM IN DETAIL
It is in charge of:
Provide APIs in several programming languages to develop clients.
Provide Productivity Tools to make agile development of clients.
Provide APIs to develop plugins to extend the SIB being deployed in the plugins container.
Provide Guides, Examples and Documentation to make easier the learning of the concepts and APIs.
In detail:
Multilanguage KPs API: Java, Javascript, Arduino, Android, C…
API to extend the Broker developing plugins
Development Environment for local testing: SIB+KPs
Open cloud environment to make integration tests.
Eclipse plugin with wizards to easy create KPs.
User guides + examples.
LOGIC ORGANIZATION: DEVELOPMENT KIT
SOFIA2 PLATFORM IN DETAIL
It is in charge of:
Ensure the compatibility of the platform with the leading Cloud platforms.
Provide to the platform with an elastic providing of resources according to its load at any moment.
In detail:
Deployment in several IassS/PaaS: Amazon, Azure, Jelastic ,…
Deployment in Indra’s PaaS platform: G@PASS
Deployment in private or public cloud.
LOGIC ORGANIZATION: CLOUD
SOFIA2 PLATFORM IN DETAIL
SOFIA2 LICENSES MODEL
SOFIA2 can be used under 2 licenses model
Community License
AGPL open-source license v2.
No cost per use.
Base version of the operative platform.
Not includes Web management console neither advance tools (Scripting Rules and CEP).
Source code available in SOFIA2 Forge: http://sofia2.org
Includes use guides and SDK http://sofia2.com/
Enterprise License
License with no AGPL license restriction.
Commercial support under several levels and SLAs.
Complete Platform including all tools.
Includes use, installation and customization guides.
SDK including Hadoop and BI support.
SOFIA2 PLATFORM IN DETAIL
The RI (Reference Implementation) of the SIB is based on these technologies:
Spring 3.X development framework
iCore4TR as Technical Architecture
Apache CXF supporting REST/Web Services
Siddhi CEP (WSO2 CEP) as embedded CEP
Hazelcast as DataGrid supporting HA SIBs
MongoDB as RTDB (Real Time Database)
Hadoop (Cloudera/Pivotal HD/…) as HDB (Historic Database)
Hive as SQL engine in Hadoop
MySQL/BD JDBC as CDB (Configuration database)
MQTT/JMS/Sockets/REST as communication protocols for KP
JMX as monitoring protocol
Spring MVC + Thymeleaf as Web framework for Web console UI
SOFIA2 PLATFORM IN DETAIL Technologies of the SIB RI (Reference Implementation)
SOFIA2 PLATFORM IN DETAIL
config server
mongos
SIB CLUSTER
SIB SIB SIB
MONGO DB Cluster
config server
mongos
config server
mongos
3 x Node MongoDB: • 32 GB of RAM • 1 CPU Intel Xeon E5 (2-2.5GHz 6/8 cores) • 2 TB Raid-10 Ext4 o XFS • Linux OS 64 bits (CentOS 6.x)
Namenode +JobTracker
(Master)
HADOOP
Datanode + TaskTracker
(Slave)
1 x NameNode Hadoop: • 32 GB RAM • 2 quad core CPUs • 1 TB RAID • Linux OS 64 bits
3 x Datanode Hadoop: • 16 Gb RAM • 4 discos x 1 TB (conf JBOD) • 2 quad core CPUs (2-2.5GHz) • Linux OS 64 bits
3 x Node SIB+Grid: • 16 GB of RAM • 1 CPU Intel Xeon E5 (2-2.5GHz 6/8 cores) • 100-200 Gb RAID • Linux OS 64 bits (CentOS 6.x)
DATA GRID
SOFIA2 Desployment proposal BALANCER
Balancer HW o SW: • F5,… • gINX, HAProxy
DN DN
SOFIA2 PLATFORM IN DETAIL
BDC
config server
mongos
SIB RUNTIME
CLUSTER SIB SIB
RUNTIME
SIB RUNTIME
MONGO DB Sharded Cluster
config server
mongos config
server mongos
Namenode +JobTracker
(Master)
HADOOP
Datanode + TaskTracker
(Slave)
Datanode + TaskTracker
(Slave)
DATA GRID
BALANCER
CDB
SIB CONSOLE
WEB
SIB CONSOLE
WEB
CLUSTER CONSOLE
WEB
SIB TOOLS
SIB TOOLS
CLUSTER SIB
TOOLS
SOFIA2 PLATFORM IN DETAIL
WARNING NOTIFIER
···
ENTERPRISE BUS (ESB)
SMART PHONE
KP
1
KP
WARNINGS SYSTEM
Service
Company
KP
···
GR
EE
N H
ER
ITA
GE
UR
BA
N M
OB
ILIT
Y
IRR
IGA
TIO
N
WARNINGS SERVICE
2
3
4
4 4
5
Service
Company
KP
5
SIB SIB SmartSpace
Gathering
BACKEND INT.
MODULE
1. A citizen send a warning alert of a dead tree.
2. The warning is processed by the warnings service. 3. The warnings system notifies to the orchestration system. 4. The orchestration system notifies the warning alert to all systems involved. 5. The service concession company makes the service removing the tree 6. The City Council checks the realization of the works Systems
integrated with SOFIA
Systems NO integrated with SOFIA
EXAMPLE OF URBAN WARNING
SOFIA2 PLATFORM IN DETAIL
DESPLOYMENT IN CITY USING ENTERPRISE BUS
H.A. SIB
TRAFFIC CAMERA
KP
ILLUMINATION
KP
···
SMART PHONE
KP
TABLET
KP ARDUINO/ RASPBERR
Y
KP
IRRIGATION GATEWAY
KP
Sprinkler network
Drip netw
Urban Mobility
Dashboard
KP
RTDB
ontologíes
<<tabla>>
BIGDATA INFRASTRUCTURE
HDFS Map Reduce
DB NoSQL HIVE
HISTORIC MEASURES SIB AREA ADAPTER
KP WARNINGSSERVICE
SERVICE XXX Ontology
x Ontology
y
KP KP
Dispatcher
SYSTEMS DEVELOPED AS
SOFIA KPS
SYSTEMS NOT SOFIA BUT USING
ONTOLOGIES
SYSTEMS NOT INTEGRATED
KP KP
BUSINESS INTELLIGENCE
···
ENTERPRISE BUS (ESB)
MANAGEMENT UI
···
INDICATORS
BDH BDCONF
users
<<tabla>>
SIB
···
PROTOCOLS GATEWAY
BACKEND INT. MODULE
PROTOCOLS GATEWAY
CEP + SCRIPT
Smart Space
SERVICE YYY
KP
OPEN DATA
42
CONTENTS
01 Introduction to SOFIA2 Platform
02 SOFIA2 Platform in detail
03 SOFIA2 Capabilities at a glance
PERSISTENCE > RTDB
The RTDB (Real Time Database) stores all the information into the time window considered as real time for each specific ontology (it is configurable for each ontology).
The RTDB for the Reference Implementation is based on MongoDB:
MongoDB is a NoSQL database, oriented to document storage, and designed to be fast, scalable and easy to use.
The NoSQL databases are developed to support increasing requirements in terms of performance, scalability and fast response.
The NoSQL databases in comparision with relational databases, do not require static data models to store the information.
It is appropiate for applications using high volume of information or complex data
SOFIA2 provides an SQL query engine over MongoDB.
Support to execute geographical queries
SOFIA2 CAPABILITIES AT A GLANCE
PERSISTENCE > RTDB
db.pt_bus.find({"MobileElement.geometry" : {$near : {$geometry : {type : "Point", coordinates : [-8.396402999999964, 43.37016999999999]}}, $maxDistance: 2000}} )
SOFIA2 CAPABILITIES AT A GLANCE
PERSISTENCE > HDB OVER HADOOP
The high volume of information that the platform has to handle makes necessary the use of Big Data approaches, in our case, Hadoop:
Hadoop is used to store and to make further processing (datawarehouse) of all data received by the platform, allowing indeterminate growth.
The features that makes Hadoop synonymous with Big Data are::
Economic: it is deployed in commodity hardware creating clusters.
Scalable: If more processing or storage capabilities are needed, the cluster can be extended adding more nodes.
Efficient: Hadoop distributes the data between all nodes and process them in parallel into the node where they are stored. In this sense, Hadoop moves processing (Tasks) where the information is stored.
Reliable: Hadoop File System (HDFS) is able to maintain multiple copies of the data in several nodes. If one node falls out, the information stored in it can be found in another one.
SOFIA 2 integrates with Hadoop infrastructure in several ways::
HDFS: As filesystem to store information without processing.
HBASE: As NoSQL database to store historic information.
Sqoop: Providing capabilities to import/export data from/to relational databases.
Hive+Impala: Providing capabilities to query information from HDFS and HBASE using SQL language.
SOFIA2 CAPABILITIES AT A GLANCE
HDFS: Is the filesystem of Hadoop.
HDFS is a distributed filesystem that provides an integrated view of all storage resources into the Hadoop cluster, abstracting this logical view from the physical storage.
When HDFS stores a file, it splits the file in several blocks, storing each block in separated nodes, and duplicating it in at least three nodes.
This makes possible to store files bigger than the maximum disk size of any of the nodes in the cluster.
If a node falls out, the cluster will continue working using the duplicated information in other nodes.
Node 1 Node 2 Node n ·····
Hadoop HDFS
····· Block 1 Block 2 Block n
Node X Node Y Node Z
Hadoop HDFS
Block
Partitioning Duplication
PERSISTENCE > HDB OVER HADOOP
SOFIA2 CAPABILITIES AT A GLANCE
MapReduce is a distributed solution to process large volumes of information:
It is based on Google works.
It works in commodity hardware.
HDFS and MapReduce are the core of Hadoop.
PERSISTENCE > HDB OVER HADOOP
SOFIA2 CAPABILITIES AT A GLANCE
DRONE DEPLOYEMENT VIEW
Semantic Information Broker
Connectors
Security
KP
Subscribed to changes
Ontologies definition
CDB RTDB HDB
Clients and Tokens definition
KP
Commands
JSON Commands KP
• DroneCameraCommand • DroneMovementCommand • DroneCameraScriptStream • KP_Drone_Joystick
• KP_Drone_Controller
HTTP MQTT
http://sofia2.com
SOFIA2 CAPABILITIES AT A GLANCE
SOFIA2 CAPABILITIES AT A GLANCE
Public Web site providing access to:
Presentations
Guides.
Videotutorials.
SDK.
Forge.
…
PUBLIC INFORMATION
WEB CONFIGURATION CONSOLE
SOFIA2 CAPABILITIES AT A GLANCE
Allows to manage all conceps of the platform via Web:
Management of Ontologies
Management of KPS
Management of Assets
Databases console
Management of Rules
…
SDK
SOFIA2 CAPABILITIES AT A GLANCE
Forge .
Multiplataform SDK
Productivity tools to create KPs
Documentation and guides.
Videotutorials.
…
WEB CONSOLE KP CONTROL
KP Sensor Gateway connected to a Zigbee Gateway (T, H, Wh)
SOFIA2 CAPABILITIES AT A GLANCE
SOFIA provides tools to automate KPs creation, generating a Web UI control
UI HTML + Javascript
Web Javascript visualization KP (T, H, Wh)
SOFIA2 CAPABILITIES AT A GLANCE
SOFIA provides a Javascript API to interoperate with other KPs using Ajax Push support
SIB MONITORING
SOFIA2 CAPABILITIES AT A GLANCE
SOFIA monitors all its components via JMX.
Also Provides a monitoring REST service.
And a HTTP adaptor via JMX.