LBERTO ONGE OFFARELLO SUPERVISOR: FULVIO CORNO · COLLABORATIVE CONNECTIONS FEATURES EXTRACTION...

END-USER DEVELOPMENT IN THE INTERNET OF THINGS

ALBERTO MONGE ROFFARELLOSUPERVISOR: FULVIO CORNO

e-Litehttps://elite.polito.it

““The Internet of Things is a recognized paradigm that already helps society in many different ways, through applications ranging in scope from the individual to the planetary

Vint Cerf and Max Senges, Google Research

A COMPLEX NETWORK OF PHYSICAL AND VIRTUAL ENTITIES

THAT CAN BE PERSONALIZED ON THE BASIS OF OUR PERSONAL NEEDS

END USERS CAN

PROGRAMTHE IOT THROUGHTRIGGER-ACTION

IF I PUBLISH A POST ONFACEBOOK

THEN SHARE IT ONTWITTER

IF THE SURVEILLANCECAMERA RECOGNIZES ME

THEN TURN ON THESMART THERMOSTAT

ISSUES

LOW LEVEL OF ABSTRACTION

ISSUES

INFORMATION OVERLOAD

ISSUES

INFORMATION OVERLOAD

RUN-TIME PROBLEMS

Assisting end users in easily and efficiently personalizing the functionality of their connected entities.

The main goal is to simplify the definition of IF-THEN rules.

RESEARCH GOAL

1 MOVING TOWARDS A HIGH-LEVEL OF ABSTRACTION

EUPont

2 DISCOVERING IF-THEN RULES AND FUNCTIONALITY

3 DEBUGGING IF-THEN RULES AT DEFINITION TIME

EUDoptmizer RecRules

EUDebug My IoT PuzzleSCPN

EUPont

Higher level of abstractionProgramming by functionalityContext-dependent IF-THEN rules

EUPont

IFI enter an indoor location,

THENset its temperature to 20 Celsius

degree

EUPont

EUPont is available at http://elite.polito.it/ontologies/eupont.owl

EUPont

TRIGGER-ACTION PROGRAMMINGLAYER

CONTEXTUAL INFORMATIONLAYER

INTERNET OF THINGSLAYER

EUPont

15 males

15 females

University studentsDifferent backgrounds

22.23 mean age

USER STUDY

EUPont

USER STUDY

Does the EUPont representation help usersdefine their IF-THEN rulesmore effectively and efficientlycompared with the IFTTT representation?

EUPont

USER STUDY

5 TASKS: Scenario + Goal

2 INTERFACES

IFTTT EUPontvs.

EUPont

ADVANTAGES

The EUPont representationallows end users to avoiderrors and to reduce the timeneeded to define their IF-THEN rules.

0.279incorrect TRIGGERSon average withIFTTT

0.120incorrect TRIGGERSon average withEUPont

0.279incorrect ACTIONSon average withIFTTT

0.120incorrect ACTIONSon average withEUPont

p < 0.05 p < 0.05

ADVANTAGES

EUPont

39.65 saverage time for defining RULESwith IFTTT

25.05average time for defining RULESwith EUPont

p < 0.05

ADVANTAGES

EUPont

CHALLENGES

Moving towards a higher levelof abstraction is promising butposes new challenges

EUPont

Adapting Contexts

How can we decide howto reproduce abstract

behaviors?

Preferred Levelof Abstraction

Which level of abstraction do users prefer?

Security, and Privacy

Can we execute abstract behavior by preserving security

and privacy?

CHALLENGES

EUPont

RecRules

Mixed content & collaborative approachSemantic graph-based settingRecommending by functionality

PERSONALIZING THE TEMPERATURE WHEN I’M GOING TO USE AN INDOOR PLACE

RecRules

IF the kitchen Nest Cam recognizes meTHEN turn on the kitchen Philips Hue lamp

IF the living room Homeboy Cam detects a movementTHEN open the Hunter Douglas blinds

KNOWLEDGE GRAPH

RecRules

KNOWLEDGE GRAPH

RecRules

KNOWLEDGE GRAPH

RecRules

COLLABORATIVECONNECTIONS

FEATURES EXTRACTION

RecRules

TECHNOLOGYCONNECTIONS

FEATURES EXTRACTION

RecRules

FUNCTIONALITYCONNECTIONS

FEATURES EXTRACTION

RecRules

MODEL TRAINING

POINTWISE

Pointwise approaches look at a single instance at a time, and

transform the ranking problem into a regression or a

classification one

PAIRWISE

Pairwise approaches look at a pair of instances at a time and

try to find out their optimal ordering.

LISTWISE

Listwise approaches directly look at the entire list

of instances, and they try to produce the optimal ordering by minimizing a loss function.

RANDOM FOREST RANKBOOST LAMBDAMARTBreiman et al. Freund et al. Wu et al.

RecRules

To what extent the the different types of path-based features influence the recommendation accuracy?

Does RecRules outperform state-of-the-art recommendation systems in suggestingIF-THEN rules?

EVALUATION

FEATURES

CT configuration: Collaborative and Technology featuresCTF configuration: Collaborative, Technology, and Functionality features

IFTTT datasetUr et al. 2016

RecRules

COMPARISON

We compared RecRules with state-of-the-art recommendation algorithms

RecRules

COMPARISON

RecRules

COMPARISON

RecRules

Fulvio Corno, Luigi De Russis, Alberto Monge Roffarello

TAPrec: Supporting the Composition of Trigger-Action Rules ThroughDynamic Recommendations

IUI ’20

RecRules

EUPont

PROBLEMS

Loops InconsistenciesRedundancies

IF I post a photo on FacebookTHEN save it on my iOS library

IF I add a new photo on my iOSTHEN post it on Instagram

IF I post a photo on InstagramTHEN post it on Facebook

IF I play a new song on my AlexaTHEN post a tweet on Twitter

IF I play a new song on my Alexa THEN save the track on Spotify

IF I save a track on Spotify THEN post a tweet on Twitter

REDUNDANCIES

IF I exit homeTHEN turn off the Philips Hue lamp

IF I exit homeTHEN set the Nest to Away mode

IF the Nest is set to Away ModeTHEN turn on the Philips Hue lamp

INCONSISTENCIES

TRANSITION

IF I exit home (T1)THEN turn off the kitchen Philips Hue lamp (A1)

IF I exit home (T2) THEN set the Nest to Away mode (A2)

IF the Nest is set to Away Mode (T3)THEN turn on the kitchen Philips Hue lamp (A3)

UNDER DEFINITION

PETRI NETS

T1 = I exit homeA1 = turn off the kitchen Hue lamp

PETRI NETS

T2 = I exit homeA2 = set the Nest to Away mode

PETRI NETS

T1 = I exit homeT2 = I exit home

PETRI NETS

T1|2 = I exit home

PETRI NETS

T3 = the Nest is set to Away modeA3 = turn on the kitchen Hue lamp

PETRI NETS

A2 = set the Nest to Away T3 = the Nest is set to Away

PETRI NETS

activate

A2 = set the Nest to Away T3 = the Nest is set to Away

PETRI NETS

activate

SEMANTIC

activate

A1 = turn off the kitchen Hue lampA3 = turn on the kitchen Hue lamp

EUPont:Lights on

EUPont:Lights off

EXECUTION

activate

EUPont:Lights on

EUPont:Lights off

T1|2 = I exit home

activate

EUPont:Lights on

EUPont:Lights off

T1 = I exit homeT2 = I exit home

EXECUTION

activate

EUPont:Lights on

EUPont:Lights off

T2 = I exit homeA1 = turn off the kitchen Hue lamp

EXECUTION

activate

EUPont:Lights on

EUPont:Lights off

A1 = turn off the kitchen Hue lampA2 = set the Nest to Away mode

EXECUTION

activate

EUPont:Lights on

EUPont:Lights off

A1 = turn off the kitchen Hue lampT3 = the Nest is set to Away Mode

EXECUTION

activate

EUPont:Lights on

EUPont:Lights off

EXECUTION

activate

EUPont:Lights on

EUPont:Lights off

EXECUTION

STRATEGY 1:Highlightingrule conflicts

STRATEGY 2:Simulatingrule conflicts

STRATEGY 1:Highlightingrule conflicts

STRATEGY 2:Simulatingrule conflicts

9 males

6 females

University students20.34 mean age

No experience in computer science and programming

USER STUDY

Do end users understand problems and why the defined rules generate them?

Is highlighting the detected problemssufficient? Do users need a step-by-stepsimulation of the involved rules?

USER STUDY

12 TRIGGER-ACTION RULES

USER STUDY

12 TRIGGER-ACTION RULES2 Inconsistencies 2 Redundancies 1 Loop

IC1 IC2 RD1 RD2 LP

USER STUDY

IC2 RD2 LPIC1 RD1DIRECT PROBLEMS

USER STUDY

IC1 RD1IC2 RD2 LPINDIRECT PROBLEMS

USER STUDY

IC1 IC2 RD1 RD2 LP

PERCEPTION

End users perceive problems in trigger-action rules differently: loops and inconsistencies are dangerous, redundancies can be even acceptable

PERCEPTION

80 %DISCARDEDloops

PERCEPTION

80 %DISCARDEDloops

97.67 %DISCARDEDinconsistencies

PERCEPTION

80 %DISCARDEDloops

97.67 %DISCARDEDinconsistencies

46.67 %SAVEDredundancies

PERCEPTION ““The important thing is that the lamp is turned on, I do not care its color!

RD1 - P13

UNDERSTANDING

Some problems are difficult to understand: loops and indirectproblems are oftenmisinterpreted

UNDERSTANDING

16,67 %misinterpretationswhen facingan inconsistency

13,34 %misinterpretationswhen facinga redundancy

40 %misinterpretationswhen facinga loop

UNDERSTANDING

26.67 %misinterpretationswhen facingan indirect problem

10 %misinterpretationswhen facinga direct problem

UNDERSTANDING ““I am sure that this problem will never occur with the rules I have defined! Moreover, such rules are useful, because the photo will be saved in 3 places at the same time.

LP - P13

STRATEGIES

Highlighting the detected problem is often not sufficient: a step-by-step simulation of the involved rules helps users understand difficult problems

STRATEGIES

77,81 % of correct interpretationswhen a problem ishighlighted

83,78 %correct interpretations

when a problem issimulated

STRATEGIES

77,81 % of correct interpretationswhen a problem ishighlighted

83,78 %correct interpretations

when a problem issimulated

50 %highlighted

71,43 %simulated

FOR LOOPS

STRATEGIES ““The step-by-step simulation helped me in understanding the loop because I could better see the evolution of the rules. It visually told me what happened!

LP - P6

FIXING

Visual languages and adopted feedback effectively help participants avoiding and finxing run-time problems in IF-THEN rules

EUPont

FUTURE DIRECTIONSIn-the-wild

Studies

Trigger Conditionsand

Multiple actions

Security and Privacy

e-Litehttps://elite.polito.it

END-USER DEVELOPMENT IN THE INTERNET OF THINGS

ALBERTO MONGE ROFFARELLOSUPERVISOR: FULVIO CORNO

ACCEPTED PUBBLICATIONS – FIRST YEAR

F.CORNO, L. DE RUSSIS, A. MONGE ROFFARELLO, «A HEALTHCARE SUPPORT SYSTEM FOR ASSISTED LIVING FACILITIES: AN IOT SOLUTION», 40TH

IEEE COMPUTER SOCIETY INTERNATIONAL CONFERENCE ON COMPUTERS, SOFTWARE & APPLICATIONS (COMPSAC 2016)

F.CORNO, L. DE RUSSIS, A. MONGE ROFFARELLO, «IOT FOR AMBIENT ASSISTED LIVING: CARE4ME - A HEALTHCARE SUPPORT SYSTEM», BOOK

CHAPTER, INTERNET OF THINGS AND ADVANCED APPLICATION IN HEALTHCARE, 2016

L. DE RUSSIS, A. MONGE ROFFARELLO, «ON THE BENEFIT OF ADDING USER PREFERENCES TO NOTIFICATION DELIVERY», CHI 2017: THE 35TH

ANNUAL CHI CONFERENCE ON HUMAN FACTORS IN COMPUTING SYSTEMS

F.CORNO, L. DE RUSSIS, A. MONGE ROFFARELLO, «A HIGH-LEVEL APPROACH TOWARDS END USER DEVELOPMENT IN THE IOT», CHI 2017: THE

35TH ANNUAL CHI CONFERENCE ON HUMAN FACTORS IN COMPUTING SYSTEMS

F.CORNO, L. DE RUSSIS, A. MONGE ROFFARELLO, «A SEMANTIC WEB APPROACH TO SIMPLIFYING TRIGGER-ACTION PROGRAMMING IN THE

IOT», IEEE COMPUTER, 2017

ACCEPTED PUBBLICATIONS – SECOND YEAR

F.CORNO, L. DE RUSSIS, A. MONGE ROFFARELLO, «IOT FOR AMBIENT ASSISTED LIVING: CARE4ME - A HEALTHCARE SUPPORT SYSTEM», BOOK

CHAPTER, WEARABLE TECHNOLOGIES: CONCEPTS, METHODOLOGIES, TOOLS, AND APPLICATIONS, 2018 (REPRINT)

L. DE RUSSIS, A. MONGE ROFFARELLO, «A DEBUGGING APPROACH FOR TRIGGER-ACTION PROGRAMMING», CHI 2018: THE 36TH ANNUAL CHI CONFERENCE ON HUMAN FACTORS IN COMPUTING SYSTEMS

F.CORNO, L. DE RUSSIS, A. MONGE ROFFARELLO, «AWARENOTIFICATIONS: MULTI-DEVICE SEMANTIC NOTIFICATION HANDLING WITH USER-DEFINED PREFERENCES», JOURNAL OF AMBIENT INTELLIGENCE AND SMART ENVIRONMENTS, 2018

A. MONGE ROFFARELLO, «END USER DEVELOPMENT IN THE IOT: A SEMANTIC APPROACH», IE 2018: THE 14TH INTERNATIONAL CONFERENCE

ON INTELLIGENT ENVIRONMENTS

ACCEPTED PUBBLICATIONS – THIRD YEAR

F.CORNO, L. DE RUSSIS, A. MONGE ROFFARELLO, «A HIGH-LEVEL SEMANTIC APPROACH TO END-USER DEVELOPMENT IN THE INTERNET OF

THINGS», INTERNATIONAL JOURNAL OF HUMAN-COMPUTER STUDIES, 2019

F.CORNO, L. DE RUSSIS, A. MONGE ROFFARELLO, «EUDOPTIMIZER: ASSISTING END USERS IN COMPOSING IF-THEN RULES THROUGH

OPTIMIZATION», IEEE ACCESS, 2019

F.CORNO, L. DE RUSSIS, A. MONGE ROFFARELLO, «MY IOT PUZZLE: DEBUGGING IF-THEN RULES THROUGH THE JIGSAW METAPHOR»,IS-EUD: THE 7TH INTERNATIONAL SYMPOSIUM ON END-USER DEVELOPMENT

F.CORNO, L. DE RUSSIS, A. MONGE ROFFARELLO, «EMPOWERING END USERS IN DEBUGGING TRIGGER-ACTION RULES»,CHI 2019: THE 37TH ANNUAL CHI CONFERENCE ON HUMAN FACTORS IN COMPUTING SYSTEMS

A. MONGE ROFFARELLO, L. DE RUSSIS, «THE RACE TOWARDS DIGITAL WELLBEING: ISSUES AND OPPORTUNITIES»,CHI 2019: THE 37TH ANNUAL CHI CONFERENCE ON HUMAN FACTORS IN COMPUTING SYSTEMS

F.CORNO, L. DE RUSSIS, A. MONGE ROFFARELLO, «RECRULES: RECOMMENDING IF-THEN RULES FOR END-USER DEVELOPMENT»,TIST ACM TRANSACTION ON INTELLIGENT SYSTEM AND TECHNOLOGY

A. MONGE ROFFARELLO, L. DE RUSSIS, «TOWARDS DETECTING AND MITIGATING SMARTPHONE HABITS »,UBICOMP 2019: THE 2019 ACM INTERNATIONAL JOINT CONFERENCE ON PERVASIVE AND UBIQUITOUS COMPUTING

LBERTO ONGE OFFARELLO SUPERVISOR: FULVIO CORNO · COLLABORATIVE CONNECTIONS FEATURES EXTRACTION...

Documents

Transcript of LBERTO ONGE OFFARELLO SUPERVISOR: FULVIO CORNO · COLLABORATIVE CONNECTIONS FEATURES EXTRACTION...

Business Connections BUSINESS connections

Extraction of Proteins from Serum - OSCAR Main Pageoscar.iitb.ac.in/onsiteDocumentsDirectory/Extraction of Serum Protein/Extraction of... · Extraction of Proteins from Serum Methodology

Study on Optimization of polyphenols extraction from ... · Compared with the traditional extraction methods, these new extraction methodssuch as , ... extraction and supercritical

microwaveassisted extraction antioxidant extraction using ...

Outsource Data Extraction Services | Data Extraction | Cogneesol

LBERTO UINTANA cro - La Crónica de Hoy en Hidalgo

EXTraCTION INDIVIDUELLE GaMME EXTraCTION INDIVIDUELLE · • Extraction renforcée : permet le raccordement sur conduits longs, • pratique en extraction de cuisine, • Facile à

Automatic extraction of translations from web …ccc.inaoep.mx/~villasen/bib/Automatic extraction of...Automatic extraction of translations 141 such translation extraction system:

Information Extraction Lecture 9 – Multilingual Extraction

EXTRACTION CHROMATOGRAPHY Technical ... › scripts › files › 5ee8...Extraction chromatographic resins EXTRACTION CHROMATOGRAPHY Technical Documentation - All Resins Extraction

s a ños A lberto M. González fue un joven abogado que ... · lberto M. González fue un joven abogado que llegó a ser presidente de la Suprema Corte de Justicia de la Nación.

Armamentarium for simple extraction Extraction forceps

Information Extraction - Relation Extraction - LMU Munichfraser/information...Information Extraction Lecture 8 – Relation Extraction CIS, LMU München Winter Semester 2015-2016 Dr.

Information Extraction Based on Extraction Ontologies: Design ...

A CCIÓN COLECTIVA, VIDA COTIDIANA Y DEMOCRACIA A LBERTO M ELUCCI C APITULO III Angélica Gutiérrez Ana María Jiménez.

T ema 3 a lberto, valva y carmen

K'rlos @lberto Producciones

Exploring Connections Between Active Learning and …...Exploring Connections Between Active Learning and Model Extraction Varun Chandrasekaran1, Kamalika Chaudhuri3, Irene Giacomelli2,

lbeRTo cIAmPI RIvoluzIone In TIPoGRAFIA - traccedizioni.ittraccedizioni.it/libri/1994AlbertoCiampiRIVOLUZIONETIPOGRAFIA.pdf · attorno alla prima guerra mondiale, dove sarà ... Sacchetti,

Relation Extraction and Machine Learning for IE Feiyu Xu feiyu@dfki€¦ · •Topic Extraction •Term Extraction •Named Entity Extraction •Binary Relation Extraction •N-ary