openAAL Training

open source software for Ambient Assisted Living

08.04.2023 1please visit www.openAAL.org

uAAL training

Main concepts of SOPRANO middleware / openAAL

Peter Wolf, FZI



• IP, FP 6 http://www.soprano-ip.org• 25 partners from 7 EU-countries• Start: 01/07 end 10/10• Project-lead: Exodus (Greece)• uAAL-contacts: ProSyst, FZI (, CAS)• Strong interest in – Defining/applying methodology for evaluation– Market-analysis– Middleware platform

SOPRANO


please visit www.openAAL.org

• Support AP in the house ambiently with the help of IT technology– Positive mindset: enhance quality of life– But also: help in critical situations

• SOPRANO Ambient Middleware (SAM)

• openAAL is open-source software based on the SAM architecture and implementation

3

(Technical) Goal of SOPRANO



• Problem: Stove/oven have been left on unintentionally while person was leaving the home

• Solution: Reminder service that helps person to remember turned on devices, open windows etc.

• High-level UC description:

– Identify situation that AP is about to leave the home

– Check for open windows, critical devices that are turned on (stove, oven, kettle etc.)

– Warn AP and wait for confirmation that message has been read

– If not warn informal carer (e.g. son)

4

Example and Motivation:the soprano remembering UC



• Problem: typical problem of AP

• Solution: ambient services that incorporate existing technologies and services and are based on

predefined rules

• High-level UC description:

– Automatically derive situations from sensor information

– Execute predefined workflow as reaction

– Execute services, actuators as needed5

Example and Motivation:Generalisation


please visit www.openAAL.org 6

Architecture



SAM in Action: CM

7

Composer

OSGi Service Middleware

Context ManagerKettle is on

Window is open...

Kettle is onWindow is open

...

Kettlesensor switched statusWindowsensor switched status...

Kettlesensor switched statusWindowsensor switched status...

State:Window open

Actuators Sensors

Procedural Manager

LittleWorld:Sensors:kett-sensor connected-to kettleoven-sensor connected-to ovendoor-sensor connected-to frontdoor...Actuators:message-actuator connected-to touchscreen...



SAM in Action: CM

8

Composer


Context ManagerAP at doorKettle is on

Window is openNo bell

AP at doorKettle is on


Frontdoor sensor switched statusRFID-sensor detects chip...

Frontdoor sensor switched statusRFID-sensor detects chip...

State:Door open

Event:AP about to leave home

Actuators Sensors

Procedural Manager



Context Manager

Sensor Service

Context Storage

Conflict Resolution

Sensor Service Sensor Service

ContextUplifter

provide state information according to low-level context ontology

Translate low-levelcontext into high-level

contextStore all context facts with timestamp and

uncertainty

Provide a consistent view on context at a

specific point in time

can be extended bysoftware developers



• State-based modeling• No reasoning semantic instead more db-like

semantics (abstract-query and constraints)• Low-level ontology

– describes sensors, actuators and their states– Simplifies integration of sensors, actuators

• High-level ontology– Describes context of assisted person– Used in communication between platform components

• Uplifter define reasoning and ensure semantic coherence

Ontology


11please visit www.openAAL.org

• Two conceptual models for easy sensor and service integration

• Service-level ontology

High-level Ontology



• Two conceptual models for easy sensor and service integration

• Sensor-level ontology

Sensor-level Ontology



• RDF-based data model– Triples describing states of an entity– Entity, state-property, state– “lamp, has-power-status, on”

• Tuple representation to capture domain-specific meta-information– History, for complex reasoning considering past

states– Confidence, for statistical reasoning approaches– Validity, Source, ...

Data-model



Tuple representation: (s,p,o,t0,v,a0,src,i)– s is subject URI class– p is predicate(=property) URI property– o is object URI class– t0 transaction time– v validity duration– a0 confidence of statement in t0– Src source of the statement– i is statement URI statement class

• Every URI must be instantiated• Classes and properties are disjoint• Corresponding tuple (class, property, class, statement class)

must correspond to conceptual model

Data-model & Ontology


please visit www.openAAL.org15

Collaboration of Uplifters - Example

Doorsensor sFires

Door d Open

RFID chip cDetected by

reader rAP a wears

chip cReader r is lo-

cated near door d

AP a near door d

Rule-basedRule-based Written tocontextstorage

Written tocontextstorage

s attached to Door d

AP inside

AP a leaving

triggers workflow

Doorbell dbpressed

AP outside

Decision TreeWritten tocontext storage



• Model and context reasoning seperated to allow for more flexibility – Formalism neutral reasoning– Flexibility considering low-level changes

• Constraints on property cardinality – needed for conflict-resolution– Conformity of interfaces

• Abstraction of classes and properties for abstract queries

openAAL Ontology



• Stores all information facts• Provides simple query interface• Manages AP-context and low-level ontology• Implements blackboard architecture• Derives AP-context information from sensor

information via aggregation services (uplifter)• Decouples situations from sensors to detect them• Decouples definition of situations from detection

algorithms• Provides for heuristic-based conflict resolution at

query-time

CM Overview



openAAL Architecture



SAM in Action

19

Composer






Kettlesensor switched statusWindowsensor switched status...Frontdoor sensor switched status


State:Door open


Procedure: AP about to leave homeInform AP about critical DevicesWait for confirmation of APIf no confirmation, send SMS to informal carer


Actuators Sensors

Procedural Manager



PM ArchitectureAdmin Tools

BPEL Engine

Store/retrieve/trigger workflow xml-document

BPEL Parser

Context ManagerComposer

Database

Procedural Manager

Store wf object struct.

Execute trigger element Get wf object struct.

Sync./async. communicationRequest service execution



• Based on Standard: Business Process Execution Language (BPEL)

21

Context-Aware Workflows

Standard Constructs Context-Aware Extensions

Sequence If Loop Assign …

Trigger event starting the wf Query context manager Store context statement Wait for statement (or timeout) Call semantic service



• Abstract workflow description equals easy to specify + reusable

• Context manager can abstract information• Composer can context-aware concretize service requests

Workflow 1

When AP is leaving Is

Stove/Oven/Kettle… turned on?

Reminde Person on Devices

Wait for confirmation until timeout?

Stop

Inform responsible person

Stop

Procedural Manager

Context Manager

Composer



• But workflows can also – operate on sensor-information (bed-sensor)– Request services directly (actuator for concrete light)

Workflow 2

Procedural Manager

Context Manager

Composer

When AP is leavingIs

stove/oven/kettle… turned on?

Remind AP on devices

Wait for confirmation and

timeout?

Stop

Inform responsible person Turn devices off

Stop



• Stores BPEL workflows that define (abstract) reactions to situations

• Executes workflow upon situation detection

• Communicates with CM for context-awareness

• Requests service execution from composer to change AP-context and/or low-level context

PM Overview



SAM in Action

26

Composer








State:Door open


Workflow instantiation:(1) Inform AP about problems

Not-hearing-impaired + near frontdoor = (1a) audio warning(1b) warning on touch screen



Loudspeaker/touch screen “Kettle is still on.”“Window is still open.”

Loudspeaker/touch screen “Kettle is still on.”“Window is still open.”

Actuators Sensors

Procedural Manager



Inner Architecture of Composer

Procedural Manager

OSGi Repository

Query

Service Matchmaker + Invocator

Abstract service goal

Query context

Concrete service goal

Virt

ual S

ervi

ces

ServiceActuator Service

Parameterised invocation

Concrete service goalsStore context

Cont

ext M

anag

er

Invoke



Solution: Virtual Services

Inform AP about medication

Play sound messageon loudspeaker-1

Show text messagevia GUI on tv-1

Send SMSto Number

Show text messagevia GUI on touchscreen-1

Play avatar message on tv-1

Inform APIf not hearing-impaired

Play sound messageon loudspeaker in this

or connected room of AP

Inform APIf not vision-impaired

Show message on GUIon switched-on device

in room of AP



Absicht: Zustandsorientierte Beschreibungen

Diensteffect

Idea: • Describe Service effect by configurable sets of complex states

• Configurable: set depends on input parameters• Complex: states are arbitrary complex graphs formable with the

ontology

• Inform Person<in,1> about Medication<in,2>, where Person<in,1> is watching tv-1

• Turn off Lamp<in,1> , where Lamp<in,1> is in kitchen-1

Set of States

Input

precondition

Set of States

Input



Absicht: Zustandsorientierte Beschreibungen

Service-Requesteffect

Idea: • Describe Service requests by configurable fuzzy sets of complex states

• Configurable: set depends on input parameters• Fuzzy: User can specify evaluation value for elements of set• Complex: states are arbitrary complex graphs formable with the

ontology

• Emit Message<in,1> on MessageActuator, where MessageActuator is in kitchen(0.5) or is in living-room(1)

Set of States

Input



• Finds end executes services upon request• Provides for predefined semantic

concretization if necessary• Decouples service requests from services at

runtime by semantic matchmaking• Decouples specification of requests and

offers by powerful DSD descritpion language• Provides heuristic-independent, real-time

service matching

Composer Overview



• Complex in terms of configuration– LittleWorlds, Uplifter, Procedures, Virtual Services, DSD-statements,

service integration– No clearly defined best practices of how and where to

configure/model certain things– Very limited tool support that is tailored to openAAL requ. or AAL-

domain• Some performance issues

– Partly due to research prototype nature and – Use of open-source software (database + OSGi)

• Still ongoing work:– No output-variables in DSD-requests in workflows– Aging of context facts was taken out in SOPRANO– Not all BPEL-constructs are thoroughly tested (yet)

• Query capability on CM-data needs to be extended

Main Drawbacks



• Modularity– Components can be used stand-alone or with

complementing dummy implementations• Highly configurable• (partly) based on (quasi-)standards (OWL/RDFS-

Syntax, BPEL, DSD)• Still continuing development in different projects

(FZI living-lab AAL, SOPRANO, MOPS, SpeedUp)• Open-source via LGPL-license • 2 running demo installations are available

General Features



• Thank you for your attention

• Questions?

openAAL Training

Technology

Transcript of openAAL Training