Socially intelligent sensing - Hayley Hung - TU Delft - Behavior Design AMS
-
Upload
behavior-design-ams -
Category
Design
-
view
1.852 -
download
4
description
Transcript of Socially intelligent sensing - Hayley Hung - TU Delft - Behavior Design AMS
1Hayley Hung, TUDelft
Socially Intelligent Sensing
Hayley Hung
Pattern Recognition and Bioinformatics GroupIntelligent Systems Department
2Hayley Hung, TUDelft
What is social behaviour?
3Hayley Hung, TUDelft
What is Human Face-to-face interaction for ?
Makes our lives easier:Relationships TrustCo-operationPersuade/influence othersInformation sharing How could technology help
us to understand/interpret/socially relevant behaviour?How could this help to influence/enhance our experience?
4Hayley Hung, TUDelft
Research Mission Statement
To develop algorithms that can model and understand non-verbal human social behaviour in real life situations. And through this, to understand how to build systems that can enhance people's quality of life by behaving with socially aware intelligence.
Develop algorithms that are perceptive to human social behaviour: Social Signal Processing, Machine LearningEnhancing people's quality of life: Human Machine Interaction, Ambient Intelligent Environments, Design, Architecture.Socially aware intelligence: Social and Behavioural Psychology, Ethnography.
Hayley Hung, TUDelft
What can you say about this picture?
Hayley Hung, TUDelft
What can you say about this picture?
Relaxed postureGestures
Vocal Behaviour
Mutual Gaze
Interpersonal Distance
7Hayley Hung, TUDelft
Current Research Frontier
Person detection
Person tracking
Gaze detection
Body pose estimation
Group detection
Social and Behavioural Pscychology, Ethnography
Activity modelling
Action recognition
Attraction Estimation
Rapport Estimation
Role Recognition
Personality estimation
Dominance Estimation
8Hayley Hung, TUDelft
Current Research Frontier
Relationship intimacy estimation
Conversation quality estimation
Person detection
Person tracking
Gaze detection
Body pose estimation
Group detection
Conversational event estimation
Personality estimation
Relationship quality estimation
Social and Behavioural Pscychology, Ethnography
Activity modelling
Action recognition
New Problem
Definitions
9Hayley Hung, TUDelft
How to model social behaviour
Sensor Data Feature and Cue Extraction
Data AnnotationSocial BehaviourModelling and Classification
Model PerformanceEvaluation
10Hayley Hung, TUDelft
Task 1: Estimating Attraction
Source: http://catinbag.blogspot.nl/2010/07/fatal-attraction.htmlVeenstra and Hung, “Do They Like Me? Using Video Cues to Predict Desires during Speed-dates” in ICCV Workshops 2011
11Hayley Hung, TUDelft
Speed Dating, Non-verbal cues and Attraction
Can proximity-related video cues be used to automatically predict attraction in speed-dates?
12Hayley Hung, TUDelft
Automated Position Extraction
- =
13Hayley Hung, TUDelft
Automated Position Extraction
14Hayley Hung, TUDelft
A Sped up Speed Date:
15Hayley Hung, TUDelft
Speed Dating Results
Predicting attractionVariance in position is best feature predictor for women (70%).Variance in position of the women and synchrony both perform well (70%) for men.
Fusion of all synchrony features
Fusion of all movement features
16Hayley Hung, TUDelft
Speed Date Experiments : Conclusion
The video channel can indeed be a source of valuable information in speed-dates Results differ per gender:
Movement synchrony information is more important for males than females.For females, information on the movement of their male counterpart gives good results
17Hayley Hung, TUDelft
Task 2: Classifiying Social Actions using a Single Wearable Accelerometer
Hung, Englebienne, Kools, “Estimating Social Actions”, Ubicomp 2013
18Hayley Hung, TUDelft
Modelling Human Social Behaviour in Dense Crowds
How can we model instantaneous social behaviour in extremely large crowds?
19Hayley Hung, TUDelft
Our Goal
To develop methods to automatically measure socially relevant behaviour and moods in dense crowds using just a single tri-axial accelerometer
First step: detect socially relevant actions
Speaking; Laughing; Gesturing; Stepping; Drinking
20Hayley Hung, TUDelft
Our Goal
Use insights from Social Psychology:
Speakers move more than listeners (McNeill 2000)
Laughter and joking correlated with sudden bursts of motion (Kendon 1990)
Synchronised motion during conversation (Kendon 1990,Chartrand and Bargh 1999)
21Hayley Hung, TUDelft
Data: The Scenario
32 volunteers (mostly mutual strangers)Experiment Stages: Briefing; Meeting and Mingling; Team formation (groups of 4); Quiz; Award GivingPrizes for top 3 teams5mx6m recording area
22Hayley Hung, TUDelft
Data: The Scenario
32 volunteers (mostly mutual strangers)Experiment Stages: Briefing; Meeting and Mingling; Team formation (groups of 4); Quiz; Award GivingPrizes for top 3 teams5mx6m recording area
23Hayley Hung, TUDelft
Data: The Scenario
32 volunteers (mostly mutual strangers)Experiment Stages: Briefing; Meeting and Mingling; Team formation (groups of 4); Quiz; Award GivingPrizes for top 3 teams5mx6m recording areaEach participant wore a sensing badge
24Hayley Hung, TUDelft
Classifying social actions
Social Actions: speaking, laughing, gesturing, drinking, or stepping
(%) Gesture Step Drink Laugh SpeechPrecision 59 100 100 100 64Recall 24 21 21 38 82F-measure
34 35 35 56 72
25Hayley Hung, TUDelft
Social Action Conclusion
It is possible to detect socially relevant behaviour.Can we detect when people are in the same conversational group?Could we even detect
personality traits quality of people's interaction?Quality of people's relationships?...etc
26Hayley Hung, TUDelft
Looking to the Future...
The way we behave socially can exhibit strong detectable patterns, which are robust to noise.
How simple can the extracted features be?How could socially aware systems benefit better design?
27Hayley Hung, TUDelft
Potential Applications: Human Robot Interaction
28Hayley Hung, TUDelft
Applications: Urban Planning
Behaviour in Public SpacesHow can we measure statistically generaliseable changes as a result of interventions?
29Hayley Hung, TUDelft
Potential Applications: Organisational behaviour
30Hayley Hung, TUDelft
Acknowledgements/Collaborators
Arno Veenstra
Gwenn Englebienne Jeroen Kools
Ben KroseMaarten van Steen
Matt Dobson Claudio Martella
Domenic Vossen
31Hayley Hung, TUDelft
Workshop at ACM Multmedia (acmmm13.org/)Barcelona , October 22
Human Behaviour Understanding for the Interactions in Arts, Creativity, Entertainment and Edutainment
Albert Ali Salah (Bogazici University, Turkey)Oya Aran (Idiap Research Institute, Switzerland)Hatice Gunes (Queen Mary University of London, UK)
32Hayley Hung, TUDelft
Data:Sensors
Each participant wore:indoor positioning deviceProximity and Acceleration sensor
12 participants wore wireless microphone (for annotation)3 fish eye cameras (for annotation)3 accelerometer readings failed (software bug)