iSenseStress: Assessing Stress Through Human-Smartphone Interaction Analysis
-
Upload
katarzyna-wac -
Category
Education
-
view
109 -
download
0
Transcript of iSenseStress: Assessing Stress Through Human-Smartphone Interaction Analysis
iSenseStress: Assessing Stress Through Human-Smartphone
Interaction AnalysisMatteo Ciman1, Katarzyna Wac2 and Ombretta Gaggi1
1 University of Padua Padua, Italy
2 University of Geneva and University of Copenhagen Geneva, Switzerland and Copenhagen, Denmark
PervasiveHealth 2015, Istanbul, Turkey /26
Stress Experience
• Stress is mental condition experienced every day
• Long exposure can lead to anxiety, depression etc. => increase of healthcare costs
• In 2013 American teens reported stress experienced at unhealthy levels (and at increasing lower ages) [http://www.apa.org/news/press/releases/stress/2013/teenstress.aspx]
2
• Early assessment of stress condition can help to provide feedback to improve health state of individuals
PervasiveHealth 2015, Istanbul, Turkey /26
Stress Assessment - State of the Art
• Stress assessment using wearable and ubiquitous devices can increase individuals’s acceptance without interfering with their life
• MouStress: project for stress assessment considering computer mouse movements or keyboard [1]
• Usage smartphone sensors (WiFi, GPS, Bluetooth, calls, SMS) [2]
3
[1] D. Sun, P. Paredes, and J. Canny, “Moustress: Detecting stress from mouse motion,” in SIGCHI Conference on Human Factors in Computing Systems, 2014, pp. 61–70. [2] G. Bauer and P. Lukowicz, “Can smartphones detect stress-related changes in the behaviour of individuals?” in PERCOM Workshops, 2012.
PervasiveHealth 2015, Istanbul, Turkey /26
Our Approach
• No external devices used, just smartphone (less expensive, more usable)
• No privacy-related information (i.e., calls, messages, location etc.)
• Possible to run a phone background service all the day long
• Based on human-smartphone interaction analysis
• Limitation: an interaction with the smartphone is required to make an assessment
5
PervasiveHealth 2015, Istanbul, Turkey /26
Human-Smartphone Interaction
6
Tap
Scroll
Swipe
Text WritingDouble
Tap
Rotate
Zoom
Pinch
Long press
PervasiveHealth 2015, Istanbul, Turkey /26
Tasks Definition
• Search Task:
• Scroll, swipe and tap
• Write Task:
• Tap, Text Writing
7
Tap Scroll Swipe
Tap Text Writing
PervasiveHealth 2015, Istanbul, Turkey /26
Search Task
• Find inside a 21x15 grid the right icon (s)
• Scroll and Swipe to inspect all the icons
• Tap to select the right icon
8
PervasiveHealth 2015, Istanbul, Turkey /26
Search Task Features
• Tap {min, max, average} pressure / length / size
• Scroll, Swipe
• {min, max, average} speed / time length / acceleration / pixels length / pressure
• Linearity
• D(interaction, center), D(interaction, top_left_screen)
9
PervasiveHealth 2015, Istanbul, Turkey /26
Write Task
• Paragraph writing as text message
• Keyboard without autocorrection or word suggestion
• English as text language
10
PervasiveHealth 2015, Istanbul, Turkey /26
Write Task Features
• Tap {min, max, average} pressure, length, size
• Tap movement and duration
• Writing:
• Speed
• # errors
• Back digits
11
PervasiveHealth 2015, Istanbul, Turkey /26
Protocol
12
Initial Relax (5’)
Relaxed Tasks (~30’)
Stressor (5’-10’)
Stressed Tasks (~10’)
Self Assessment
Negative ValenceLow energyNot Stressed
Positive ValenceHigh Energy
Stressed
1 2 3 4 5 1 2 3 4 5 1 2 3 4 5
ESM 1 ESM 2 ESM 3 ESM 4 ESM 5
PervasiveHealth 2015, Istanbul, Turkey /26
Protocol (II)
13
Initial Relax (5’)
Relaxed Tasks (~30’)
Stressor (5’-10’)
Stressed Tasks (~10’)
PervasiveHealth 2015, Istanbul, Turkey /26
How to Stress People
• Most common used stressors for tests
• Mathematical problems
• Timing pressure
• Social evaluation
• Repetition
• Uncontrollability
14
We used these
PervasiveHealth 2015, Istanbul, Turkey /26
Stressor Task: Math (II)
16
Progress bar + tic-tac sound
Random decrease Digits back to 0
every time
Each wrong answer annoying sound and going back
PervasiveHealth 2015, Istanbul, Turkey /26
Search Task - Stressed
17
Progress bar + tic-tac sound
Sound + vibration
PervasiveHealth 2015, Istanbul, Turkey /26
User Study
• 13 Participants (7M, 6F), average age 26,38 (± 2,53)
• Own phone, no constraints for the type to use (Android OS)
• Different English literacy level
• Average protocol duration: 1 hour
• Cover story: New Google interface analysis
19
PervasiveHealth 2015, Istanbul, Turkey /26
Stress Induction Analysis
20
Initial Relax
Relaxed Tasks
Stressor Stressed TasksESM 1 ESM 2 ESM 3 ESM 4 ESM 5
TEST t(13) p-value
ESM 3 VS ESM 4 1.99 0,007 *
ESM 3 VS ESM 5 -2.84 0,009 *
ESM 4 VS ESM 5 -2.74 0.5
Participants were stressed
Different stress level at the end of tasks
Kept stressed during stress tasks
PervasiveHealth 2015, Istanbul, Turkey /26
Features Evaluation
• Statistical analysis for significance evaluation
• Stress prediction model using Decision Tree (DT), k-Nearest Neighbourhood (kNN), Bayes Network (BN), Support Vector Machine (SVM) and Neural Networks (NN)
• User and global model (evaluated using 10-Fold cross validation and leave-one-out)
21
PervasiveHealth 2015, Istanbul, Turkey /26
Search Task - Statistical Correlation• Only weak correlation between our features
• Global Model
• Average swipe pressure (p-value = 0,09)
• Scroll distance from center (p-value = 0,065)
• Scroll distance from top left (p-value = 0,07)
22
• User model
• Scroll interaction length (strong correlation for 61% of users)
• Scroll delta (strong correlation for 40% of users)
• Scroll linearity (strong correlation for 45% of users)
PervasiveHealth 2015, Istanbul, Turkey /26
Search Task - Prediction Model
23
F-measure for Scroll interaction models
MODEL DT KNN SVM NN BN
USER (AVERAGE) 0.79 0.80 0.81 0.80 0.77
GLOBAL (AVERAGE) 0.73 0.71 0.78 0.74 0.67
F-measure for Swipe interaction modelsMODEL DT KNN SVM NN BN
USER (AVERAGE) 0.86 0.86 0.79 0.87 0.85
GLOBAL (AVERAGE) 0.92 0.75 0.81 0.82 0.77
PervasiveHealth 2015, Istanbul, Turkey /26
Write Task - Statistical Correlation
• User Model
• Digits size (64% of users with strong correlation)
• Pressure/Size ratio (55% of users with strong correlation)
• Global Model
• Wrong Words / Total words ratio (p-value = 0,028)
• Digits time distance (p-value = 0,012)
• Digit duration (p-value = 0,08)
24
PervasiveHealth 2015, Istanbul, Turkey /26
Conclusions• Stress assessment using data from non-intrusive
devices can increase people’ acceptance
• Human-smartphone interaction analysis can be leveraged to assess stress state in users
• Scroll and Swipe: F-measure of stress prediction between 79% and 85% for user models, and between 70% and 80% for global model.
• Text writing: several features showed strong correlation
25
PervasiveHealth 2015, Istanbul, Turkey /26
Future works
• Real-time background service for stress assessment
• Behaviour suggestion implementation
• Stress assessment in the wild (ongoing study, 29 participants)
26
• My PhD Thesis :)
iSenseStress: Assessing Stress Through Human-Smartphone
Interaction AnalysisMatteo Ciman, Katarzyna Wac and Ombretta Gaggi
{mciman,gaggi}@math.unipd.it [email protected]