Smart Spaces Semantic Interoperability and Complex Event Processing

Seppo Törmä Distributed Systems Group

Department of Computer Science and Engineering School of Science, Aalto University

Contents •  Research background •  Smart spaces

–  Terminology and definitions –  Examples –  Enablers and goals

•  Interoperability –  Volatility and interoperability –  Exising interoperability solutions

•  DIEM approach –  Semantic information broker –  Protocols –  RDF and flexible merging of data –  Event-based systems –  Loose coupling –  Complex events and SPARQL

Distributed Systems Group

•  Heikki Saikkonen, professor, software technology •  Esko Nuutila, researcher, distributed systems •  Seppo Törmä, researcher, distributed systems •  Ville Karavirta, post doc, service adapters •  Abdullah Haris, Ph.D. student, efficient subscription matching •  Jyrki Oraskari, Ph.D. student, learning in event-based systems •  Nam Vu Hoang, Ph.D. student, distributed transactional BIM •  Juho Makkonen, project researcher, Kassi social exchange service •  Antti Virolainen, project researcher, Kassi social exchange service •  Sampo Toiva, M.Sc. student, event-based social networking (ASI) •  Emmi Suhonen, M.Sc. student, motivational issues in Kassi •  Zuheb Hussain, M.Sc. student, mobile clients for event-based services

Distributed Systems Group •  Digital services

–  Service interoperability and composition –  Loose coupling between services –  Non-intrusive interoperation with service adapters

•  Distributed event-based systems –  Content-based publish/subscribe –  Efficient, incremental subscription matching –  Complex event processing –  Smart space applications

•  Semantic interoperability –  Semantic representions (RDF) –  Semantic query languages (SPARQL)

•  Distributed models –  Transactional changes –  Workflow management

DIEM Device

interoperability

OtaSizzle Ubiquitous

social media

DRUM Distributed

transactional BIMs

ServiceCloud Service

interoperability

EIT SSAL Complex event

processing in smart spaces

Terminology •  Smart space

–  Definition: A built environment with embedded services for mobile users

–  Merge physical and digital worlds –  Synonyms: Smart environment, Intelligent environment

•  Related concepts –  Ambient intelligence –  Ubiquitous computing –  Pervasive computing –  Internet of Things

•  Special focus –  Users’ activities –  Useful, interesting applications

Examples •  Home •  Offices and meeting rooms •  Vehicles - cars, busses, trains, … •  Gyms, sport facilities •  Shops, restaurants, shopping malls •  Train stations, seaports, airports •  Hospitals, art museums, market squares •  Entertainment complexes - movies, tivolis, game arcades

Enablers

•  Key technological enablers –  Wireless connectivity –  Device minituarization

•  Resulting trends –  Increasing number of computing devices –  “1000 devices/person by 2015”

•  Influenced by advanced interaction technologies –  Video and audio sensing –  Display technology –  Speech and gesture recognition –  Gaze tracking –  …

Goals

•  Enhance user’s experience –  Adaptiation – sense and learn user preferences –  Interact in a manner that is natural to humans –  Connect different domains – User is in interaction with

functionalities from many domains; for example, at home with •  Building automation (lightning, ventilation, heating) •  Home entertainment (audio and video players) •  Home appliances (fridges, washing machines) •  ICT devices (mobile phones, computers)

•  Maximize energy efficiency •  Provide safe and secure environment

Volatility

•  Smart spaces are volatile systems –  Distributed systems with frequent and unpredictable changes in

users, devices, and software components •  device and communication failures •  changes in bandwidth •  creation and deletion of associations

–  Physical mobility is seen as appearance and disappearance of devices from a smart space

•  Volatility leads to the need of spontaneous interoperation –  preconfiguration is not possible as a general solution

Interoperability

•  Generally: The ability of diverse systems to work together (inter-operate)

•  Technically: The ability of two or more systems –  to exchange information and –  to use the information that has been exchanged

•  Levels –  Technical – communication protocol, message syntax –  Semantic – interpret the information exchanged meaningfully

•  can refer to same entities •  can refer to same real-world properties and relations

•  Simplicity, flexible interaction, common languages

Existing interoperability solutions •  Generic interoperation – insufficient to smart spaces as such

–  CORBA –  Web Services (WSDL, SOAP, UDDI) –  Semantic Web (RDF, SPARQL, …)

•  Specific for smart spaces –  UPnP – Universal Plug and Play (ICT & entertainment at home) –  DLNA – Digital Living Network Alliance (entertainment) –  NoTA – Nokia Terminal Architecture –  OSGi – A dynamic component model for JVM –  Amigo – Ambient Intelligence for the Networked Home –  oBIX – Open Building Information eXchange

•  Why not in widespead use? –  Domain-specific, closed, complex, low level? –  May require coordinated development from multiple parties

DIEM approach

•  Semantic representations –  Graph-based data –  RDF

•  Complex event processing –  Subscriptions to complex situations –  Dynamic SPARQL queries

•  Connect existing services through adapters –  Respecting the legacy solutions

•  Open, standard-compliant, and cross-domain •  Focus on semantics rather than protocols

Semantic information broker

SSAP - Smart Space Access Protocol

RDF CRUD operations

Applications and services

RDF – Resource Description Framework

•  Family of W3C Recommendations –  Nothing fancy but flexible and standardized –  Lots of practical problems solved

•  Data represented as a graph –  consists of arcs represented as triplets

–  arcs can be interpreted as simple statements •  subject, predicate, object

•  Graph is flexible and easily extensible structure –  new properties and relations can be added to existing nodes –  entities, and their properties and relations can come from

different sources

Merging of data

•  Entities are identified with a URI (or actually an IRI) –  The properties and relations concerning same URI can be

merged into same node

•  Types, properties, and relations are defined in an ontology –  If terms used in different sources are based on a same ontology,

they can be interpreted as same –  Ontologies can support the definition of relationships between

terms •  type, subClassOf •  subPropertyOf, inverseFunctionalProperty •  …

Example: Two data sources in RDF

Example: Merging data into one graph

Example: Meeting interoperability

RDF in practice

•  Fragments of an RDF graph can be easily exchanged –  There are multiple serializations for RDF

•  RDF/XML, Turtle, N3, …

•  Locally RDF graphs are manipulated in RDF stores –  Sesame, Jena, rdflib, NEO4J, …

•  There are different ways to specify ontologies –  RDFS – RDF Schema (very basic constructs) –  OWL –Web Ontology Language (based on description logics)

•  There are some widely used special ontologies available –  for time, location, social relations, calendar information

•  Broad, all-encompassing ontologies are less useful •  “A little semantics goes a long way” – Jim Hendler

Event-based systems •  Publish/subscribe interaction style

–  Consumers subscribe to the information they want to have and get notified when information comes available or changes

–  Content-based subscriptions are dynamic queries to the information of the producers

–  Information production, subscription matching, and notification can be distributed to multiple nodes (event processing network)

Event-based systems

•  Taxonomy of interaction styles –  Does the producer or consumer initiate the information transfer –  Is the receiving component directly or indirectly addressed?

–  Does indirect addressing require some kind of middleware functionality?

Initiator Consumer Producer

Addressing Direct Request/reply Callback Indirect Anonymous

request/reply Event-based

Loose coupling

•  Publish/subscribe provides a decoupling of consumers and producers in the following dimensions: –  Anonymous (Space decoupling). Consumers and producers do not

need to know of each other. In particular, the consumer does not make any assumption about the location of the producer.

–  Asynchronous (Time decoupling). Consumers and producers do not have to be simultaneously present during the interaction. The publication of the information may have happened long before consumer becomes active – signs in or regains the network connectivity – and is notified about the event.

–  Non-blocking (Synchronization decoupling). Producers do not block when publishing events and consumers do not block when notified about an event. That is, the production and consumption of events do not happen in their main flow of control.

Example: Request/reply location access •  Request/reply example as a REST call

–  Request: http://www.l.com/getLocation?user=”userXXX” –  Reply: {"latitude": 60.1633, "longitude": 24.8571}

•  Assumptions –  Location of service: www.l.com –  Method call interface: getLocation?user=<string> –  Format of reply: a JSON object as a string with fields

“latitude” having value in -180 – 180 and “longitude” having value in -90 – 90

–  Availability of www.l.com at the time of call –  Sufficiently low response time of www.l.com

•  Any of these assumptions can fail •  The fewer assumptions are made between parties, the more

flexible and robust the interaction style is

Example: Event-based location access

•  Content-based subscription: –  SELECT ?lat ?long {

userXXX hasLocation ?location . ?location latitude ?lat . ?location longitude ?long . }

•  The consumer –  makes a subscription, a dynamic query distributed to all producers –  is notified whenever the result of the query changes

•  Much weaker assumptions about –  the producers location, availability, or response time –  particular method call or reply formats

•  There can be multiple producers or the producer may change over the time

Complex events

•  Notify me if two of my friends are together in my vicinity –  SELECT ?a ?b ?la {

me location ?lm me knows ?a . me knows ?b . ?a location ?la . ?b location ?lb . FILTER (near(?la, ?lb, 5) and near(?lm, ?la, 200)) }

•  How to evaluate such queries efficiently?

Incremental subscription matching

•  Queries can be translated into an incremental matching network

•  The SSAP events can be inserted to the network –  events are propagated if they pass all the filters –  with insert the variables in the queries are bound to the proper

parts of the triples –  when remove, the bindings are deleted

•  Algorithmic variations –  RETE, TREAT, LEAPS

•  Under construction!

Summary

•  Smart spaces need capability for spotaneous interoperation

•  The research challenges are at the level of semantic interoperability –  Techniques from the Semantic Web can be used –  RDF, SPARQL, Ontologies

•  Complex event processing –  Behavior driven by events from sensors or mobile devices –  Content-based publish/subscribe –  Complex situations and complex event patterns