Post on 18-Jan-2016
Consulenza, soluzioni e
servizi per l’ICT
©Copyright 2013 Lutech Spa
Matteo RedaelliAnkara 25-03-2015C2Sense Meeting
Task D 3.2
Kafka quick introduction
Demostration Structure
Producer Web Service
Consumer Web Service
Producer Client
Consumer Client
Web Console
Agenda
Task 3.2 Protocol Interoperability
This task’s objective is to achieve the protocol interoperability of emergency applications/systems and sensors
using heterogeneous protocols. In this respect, “Protocol interoperability” addresses the transport level protocols
such as TCP/IP, HTTP, SOAP, REST or SMTP and is in charge of end-to-end delivery of messages. C2-SENSE
will use Web service protocols for this layer by exposing the proprietary services of emergency applications and
organizations as “Web services”. In this task, first the detailed Operational and Core Web Service categories
will be identified under the supervision of SAGEM. In this layer only the abstract definition of these services
will be developed. Their interface schemas describing inputs/outputs in detail will be defined in Task 3.3 and
the services will be implemented in Task 3.4). After that for these services related protocol interoperability
profiles will be developed and based on these profiles corresponding Protocol Adapters will be implemented.
These protocol adapters will be used to access emergency applications/systems and sensors in the C2-SENSE
architecture through Web Service protocols. There will be two types of Protocol Adapters: (1) Sensor Protocol
Adapters (to be developed by AIT) and (2) Emergency Applications/Systems Protocol Adapters (to be developed
by SAGEM).
Furthermore, the communication among the components and services of the C2-SENSE Architecture will be based on an Enterprise Service Bus (ESB), which will be developed by LUTECH.
Task 3.2 Protocol Interoperability
Kafka Introduction
Kafka as ESB:– Fast
Could handle hundreds of messages from different sources.
– Scalable It has an architecture that could evolve and grow
without losing time.– Durable
The possibility of losing messages is reduced.– Distributed by Design
It has possibility of retrieving messages in a second moment.
Introduction – Connection with C2Sense
Kafka Architecture - Glossary (1)
Producer = who publish messagesConsumer = who read messagesBroker = nodeTopic = message categoriesPartition = ordered, immutable sequence of messages
Kafka Architecture(2)
• Queue, in which each message is sent only to one reader, even if there are many
• Publish/Subscribe, in which message is sent to any reader interested in it
Producer Web Service
Producer Web Service:– Connector for sensors– Web Service is connected to the ESB
Producer Web Service
Consumer Web Service
Consumer Web Service:– Connector for retrieving messages
from the ESB– Web Service is connected to the ESB
Consumer Web Service
Producer Client
Producer Client– Very Simple Python script– Sensor like client
Producer Client
tstamp topic from lat lng hydrolevel
StringTimestamp
String“Sensor1”
StringFrom Water Sensor
Stringlatitude
Stringlongitudine
IntRandom number
Consumer Client
Consumer Client– Very Simple Python script– Retrieving messages from Web Service
Consumer Client
maxNumOfMsgs fromTopic fromPartition
Int100
String“Sensor1”
Int 1
Web Console
Web Console– Topic– Brokers– SettingsCould be a consumer
Web Console
Dove siamo
Direzione e Sede OperativaVia W.A. Mozart 47,
20093 Cologno Monzese (MI)Tel. +39-02-25427011
Fax +39-02-25427090
Sede di RomaVia A. Mantegna 4,
Parco Leonardo00054 Fiumicino (RM)
Tel. +39-06-227501Fax +39-06-22771542
Altre sedi aTorino, Modena, Firenze, Bari
Email info@lutech.it Web http://www.lutech.it