Designing Cross-Domain

Semantic Web of Things

Applications

Amelie Gyrard

Christian Bonnet (Eurecom, Mobile Communication)

Karima Boudaoud (I3S, Security)

Agenda

Introduction & Motivation

State of The Art & Main challenges

Contributions: M3 framework

Components

Use cases

Evaluations

Demonstrations

Conclusion & Future work

2

How to interpret Internet of Things (IoT) data?

Thermometer

Sensor data

Applications to visualize data

Interpretation

by humans How machines can

interpret data?

3

Machine learning?

Reusing domain knowledge?

How to combine and reuse IoT data?

How to get

additional

information?

How to combine data from

different domains?

4

How to

combine

domains?

How to describe data?

How to describe data?

Taking inspiration from the Web

Automatically built

by machines

5

How to get additional information?

Agreeing on common

vocabularies to describe data

on the web:

Semantic search engines

Web sites

They built together Schema.org

6

How to apply semantic web technologies to

Internet of Things?

Machine-understandable data

Describe data with common vocabularies

Reuse domain knowledge

Link to other data

Ease the reasoning

=> How to provide a common description of sensor

data to later reason on it?

7

How to combine IoT data from different domains?

8

Innovative

applications

Interoperability on protocols or data?

Agenda

Introduction & Motivation

State of The Art & Main challenges

Contributions: M3 framework

Conclusion & Future work

9

“Semantic Web of Things: an analysis of the application semantics for the IoT moving

towards the IoT convergence” [Jara et al. 2014]

Semantic Web of Things: Main challenges (1)

Machine-to-Machine (M2M): no human intervention

Global

interoperability

How?

Why? Common description

Common App. Protocol

Device Abstraction

Common Nwk. Protocol

10

Semantic Web of Things: Main challenges (2)

“Semantics for the Internet of Things: early progress and back to the future” [Barnaghi et al.

2012] 11

State of the Art: Semantic Sensor Networks

2008

‘Semantic Sensor Web’

‘Linked Sensor Data’

2013 2014

SemSOS,

‘Semantic

Perception’ ‘Infer high-level

abstraction’

‘Linked Stream Data’

2015

‘SPARQLStream’

• A) How to design

semantic-based

IoT applications?

• B) Interpret data?

Combine domains ?

Reuse domain knowledge?

• C) Security & IoT?

2011

W3C SSN ontology

Real-time? Use

semantic web

technologies?

Interpret data?

12

State of the art: W3C SSN ontology

Limitations of W3C SSN ontology:

Interoperability issues to reuse and combine domain ontologies

Need of a common description to describe sensor measurements

Need of an approach to share and reuse the reasoning approach

Need to integrate semantics to IoT and M2M

=> How to extend the W3C SSN ontology to provide a

common description of sensor data to later reason on it

by reusing domain knowledge?

http://www.w3.org/2005/Incubator/ssn/ssnx/ssn#

http://www.w3.org/2005/Incubator/ssn/XGR-ssn-20110628/

http://www.w3.org/2015/spatial/charter 13

Three main research challenges to address

Challenge A: How to design semantic-based IoT

applications?

Challenge B: How to interpret IoT data?

Challenge B.1: How to reuse and combine IoT data?

Challenge B.2: How to reuse and combine domain knowledge?

Challenge C: How to secure IoT applications?

14

Agenda

Introduction & Motivation

State of The Art & Main challenges

Contributions: M3 framework

Conclusion & Future work

15

Our solution: Machine-to-Machine Measurement

Framework (M3)

Challenge A: Design

semantic based IoT

applications

Challenge B.1 &

B.2: Combine

data and domains

Challenge B:

Interpret IoT

data

Challenge C:

Secure IoT

applications

Challenge B.2:

Reuse domain

knowledge 16

Agenda

Introduction & Motivation

State of The Art & Main challenges

Contributions: M3 framework

Conclusion & Future work

17

SWoT generator

Template used in 3 steps:

1) Designing phase

2) Development phase

3) Running phase

SWoT template

=> Benefits: No need to learn semantic web technologies

IoT

Application

generate

build

use

IoT

developers

18

Designing phase

19

*

* Domain where is deployed the sensor, not the applicative domain

Challenge A: Design

semantic based IoT

applications

Development phase

IoT

developers

SWoT

template

1) Load:

- M3 ontologies

- M3 IoT data

- M3 datasets

4) Get M3 suggestions or

high level abstractions STEPS BEFORE

Get

template

3) Execute M3 SPARQL query +

SPARQL engine

Se

ma

ntic

We

b

Fra

mew

ork

2) Execute M3 rules +

reasoning engine

20

Running phase

21

Agenda

Introduction & Motivation

State of The Art & Main challenges

Contributions: M3 framework

Conclusion & Future work

22

M3 language & ontology

23

Challenge B.1 &

B.2: Combine

data and domains

M3 language & M3 ontology

Data is from heterogeneous projects and domains

Domain (e.g., health, smart building, weather, room, city, etc.)

Measurement type (e.g., t = temp = temperature)

Sensor type (e.g., rainfall sensor = precipitation sensor)

Units (e.g., Celsius, Fahrenheit, Kelvin)

M3 language implemented in the M3 ontology

Describe data in an unified way

Extension of the W3C Semantic Sensor Networks (SSN) ontology

(Observation Value concept)

Provide a basis for reasoning and cross-domain interlinking

24 http://www.sensormeasurement.appspot.com/documentation/Nomenclat

ureSensorData.pdf

M3 language: a hub for cross-domain

ontologies and datasets

25

Sensor-based Linked Open Rules (S-LOR)

26

Challenge B:

Interpret IoT

data

S-LOR: Deducing new knowledge

How to deduce new knowledge?

S-LOR: a dataset of interoperable rules

Rules example:

If Domain == Health && MeasurementType == Temperature

then NewType = BodyTemperature

If BodyTemperature > 38,7°C then “Fever”

BodyTemperature and Fever are already described in

domain ontologies or datasets!

27 Demo paper: Helping IoT application developers with Sensor-based Linked Open

Rules [Gyrard et al., ISWC 2014, SSN workshop]

Linked Open Vocabularies for

Internet of Things (LOV4IoT)

Challenge B.2:

Reuse domain

knowledge 28

A dataset of more than 270 ontology-based projects

relevant for IoT

Ontologies

Datasets

Rules to interpret IoT data

Technologies used

Sensors used

Security mechanisms used

Domains relevant for IoT

LOV4IoT

http://www.sensormeasurement.appspot.com/?p=ontologies 29

A second life for ontologies!

LOV4IoT is used to build the SWoT template

Used to re-design interoperable ontologies, rules, datasets

Limitations: Manually and not automatically

LOV4IoT

http://www.sensormeasurement.appspot.com/?p=ontologies

Collect Classify Interoperability SWoT

template

30

A second life for ontologies!

M3 interoperable domain knowledge

Need to have the set of files generated in the template

compatible with sensor data

Ontologies + datasets + rules + sensor data

Domain knowledge structured in the same way

Domain

ontologies

Domain

datasets

Rules

Interoperable

IoT

Application

Provide

sensor data

SWoT template M3 IoT

data

Produce

31

M3 semantic engine

Enrich data & combine domains

32 Paper: Enrich Machine-to-Machine Data with Semantic Web Technologies for Cross-

Domain Applications [Gyrard et al., WF-IoT 2014]

Agenda

Introduction & Motivation

State of The Art & Main challenges

Contributions: M3 framework

Conclusion & Future work

33

Security Toolbox: Attacks & Countermeasures

(STAC)

Challenge C: Help non-

security experts to secure

IoT applications

34

The STAC ontology

Paper: The STAC (Security Toolbox: Attacks & Countermeasures) ontology

[Gyrard et al., Poster, WWW 2013] 35

STAC Hub

Reusing security knowledge from LOV4IoT

36 Paper: An ontology-based approach for helping to secure the ETSI Machine-to-

Machine Architecture [Gyrard et al., iThings 2014]

Agenda

Introduction & Motivation

State of The Art & Main challenges

Contributions: M3 framework

Use cases

Evaluations

Demonstrations

Conclusion & Future work

37

M3 use cases

3 Mock-ups: Naturopathy, Tourism, Transport

Proof of concept: less user-friendly

Integrating the M3 approach everywhere!

Cloud, Android-powered devices and Gateway

Combine domain-specific sophisticated applications

Not just data visualization

Suggestions or high-level abstractions

38

Use Case: Embedding M3 in smart fridges

M3 suggestions:

Home remedies

Get temperature

measurement

Stop to be sick with M3!

39

Use Case: Embedding M3 in smart luggage

M3 suggestions:

Garments & Activities

Get weather

measurement

Stop to forget things with M3!

Smart Luggage

Destination: Mountain in

winter

Destination: Beach in

summer

40

Use Case: Embedding M3 in smart cars

Avoid accidents with M3!

41

Evaluations: Research hypotheses

Templates help IoT projects build their scenarios

The semantic engine is not too resource consuming

The semantic engine is generic enough to support

various kind of IoT measurement.

The interoperable knowledge bases built follows

semantic web best practices.

Our knowledge bases help non-experts in semantic

web or in security

LOV4IoT is exploited outside of the M3 framework.

42

Evaluating the SWoT generator

Do we have templates covering the most popular IoT

use cases?

http://www.sensormeasurement.appspot.com/?p=m3_scenario

Adding a new template?

Less than 1 day

Depends on whether we already have the interoperable domain

knowledge

43

Evaluating M3 software performances

Goal: The semantic engine is not too resource consuming

Evaluation:

Measuring time consumed

Results:

Encouraging (16 – 31 ms)

Could be embedded on

Android-powered device

44

Demo

Demo

http://sensormeasurement.appspot.com

45

M3 framework at work

Domain

experts

IoT developers

End users

Design

applications

Need new

applications

Standardize

Design new ontology

matching tools +

Automatic extraction of

domain knowledge

Exploit &

Contribute

46

Relevant for different communities

Agenda

Introduction & Motivation

State of The Art & Main challenges

Contributions: M3 framework

Conclusion & Future work

47

Summary of contributions

48

Ch

all

en

ge A

M3: An entire chain from sensor data to build IoT

cross-domain IoT applications

Sensor

data

Interpret data +

Combine

domains

Interoperable

sensor data

descriptions

Reuse domain

knowledge Build IoT

applications Provide

template

Secure

applications

Ch

all

en

ge C

Ch

all

en

ge A

Ch

all

en

ge B

.1

Ch

all

en

ge

B

Ch

all

en

ge B

.2

49

Conclusion & Lessons learnt

M3: an innovative approach to assist users in

designing interoperable cross-domain Semantic Web of

Things applications:

A uniform language for sensor data descriptions

An open-source approach to interpret IoT data

Combine domains

Semantics is hidden to the users

Lessons learnt:

M3 generic enough for other domains than IoT and security

50

Our proposed

approach:

M3 framework

Future work

Sensor Plug & Play

Extract & combine

domain knowledge

Standardizing common descriptions

Merge M3 to existing SWoT projects

Global

interoperability

Common description

Device Abstraction

Common App. Protocol

Common Nwk. Protocol

51

S-LOR with more reasoning

Future work: Merge M3 to existing SWoT projects

Use real datasets & scenarios

+ real-time

Suggest machine learning algorithms

to employ for complicated sensors

Connect

new sensors

52

Rewrite

ontologies

Future work: Extracting and combining domain

knowledge

Extracting popular concepts from domain ontologies

Cloud tag inspired by the W3C SSN validator

Extracting rules from ontologies

OWL 2 RL template, DLEJena

Combining domain knowledge

Design and combine new ontology matching tools

Look at ontology alignment ontology & merging tools

Designing an interoperable domain knowledge

53

Thank you!

gyrard@eurecom.fr

http://sensormeasurement.appspot.com/

54

Relevant Publications

International Conferences:

Enrich Machine-to-Machine Data with Semantic Web Technologies

for Cross-Domain Applications (WF-IoT 2014)

An ontology-based approach for helping to secure the ETSI

Machine-to-Machine Architecture (iThings 2014)

A machine-to-machine architecture to merge semantic sensor

measurements (WWW 2013, DC)

International Workshops:

Standardizing Generic Cross-Domain Applications in Internet of

Things (Globecom , WTS, 2014)

Demo paper: Helping IoT application developers with Sensor-based

Linked Open Rules (ISWC, SSN 2014)

See Google Scholar for more publications

Participation to standardizations:

55