The usability canary in the security coal mine
19
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Background Airline self-service checkin kiosk Model Model validation Conclusions The usability canary in the security coal mine: A cognitive framework for evaluation and design of usable authentication solutions Brian Glass 1 Graeme Jenkinson 2 @gcjenkinson Yuqi Liu 1 M. Angela Sasse 1 Frank Stajano 2 1 University College London (United Kingdon) 2 University of Cambridge (United Kingdon) EuroUSEC 2016-07-18
-
Upload
graeme-jenkinson -
Category
Technology
-
view
146 -
download
1
Transcript of The usability canary in the security coal mine
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Airline self-service checkin kiosk Model Model validation Conclusions
The usability canary in the security coal mine:A cognitive framework for evaluation and design of usable
authentication solutions
Brian Glass1 Graeme Jenkinson2 @gcjenkinson Yuqi Liu1
M. Angela Sasse1 Frank Stajano2
1University College London (United Kingdon)2University of Cambridge (United Kingdon)
EuroUSEC 2016-07-18
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Airline self-service checkin kiosk Model Model validation Conclusions
Usability canary
A usability canary
It’s as dark as a dungeon way down in the mine—Merle Travis
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Airline self-service checkin kiosk Model Model validation Conclusions
Usability canary
The Quest to Replace Passwords Bonneau et. al. 2012
Evaluated two decades ofproposed passwordreplacement schemes
25 usability, deployment andsecurity benefits
No one scheme better in alldimensions
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Airline self-service checkin kiosk Model Model validation Conclusions
Usability canary
Authentication doesn’t exist in a vacuum
The impact of a given security measureis a function of:
When it occurs in the user’sworkflow
What functions of the brain itloads
What the user was meant to bedoing before and after
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Airline self-service checkin kiosk Model Model validation Conclusions
Airline self-service checkin kiosk
Modelling business process
A representation of the setof steps
Tasks that can be performedat each step
Hard constraints thatenforce partial ordering oftasks
Soft constraints that capturethe costs of switching tasks
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Airline self-service checkin kiosk Model Model validation Conclusions
Airline self-service checkin kiosk
Airline self-service checkin kiosk
LANG
AIRL
BKRF
FRBNLIQH
DIMH
STSO
STSR
EXBG
CFRM
PRLT
PRBP
AUTH
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Airline self-service checkin kiosk Model Model validation Conclusions
Congnitive pyschology
Cognitive pyschology
When a person switches from one task to another task,the bain must reorganize and reallocate cognitiveresources to ensure an efficient transition.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Airline self-service checkin kiosk Model Model validation Conclusions
Congnitive pyschology
Operationalizing the check-in task
ToVWM PWM DR SR ER
From
Visual working memory (VWM) 0 0.495 0.495 0.495 0.157Procedural memory (PM) 0.495 0 0.495 0.699 0.699
Declarative recall (DR) 0.495 0.495 0 0.482 0.482Semantic recognition (SR) 0.495 0.842 1.078 0 0.433Episodic recognition (ER) 0.307 0.842 1.078 0.354 0
Table: Costs of switching between tasks utilising different cognitivemechanisms, given as Cohen’s d effect sizes.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Airline self-service checkin kiosk Model Model validation Conclusions
Congnitive pyschology
Operationalizing the check-in task
Code Primary cognitive resource Modality Voluntary? Familiarity ComplexityLANG Semantic recognition Touchscreen No 5 1AIRL Episodic recognition Touchscreen No 5 1BKRF Visual working memory Touchscreen QW-
ERTYNo 3 3
. . . . . . . . . . . . . . . . . .
Table: Properties of the check-in kiosk tasks. Familiarity and complexityare on a scale from 1 (low) to 5 (high).
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Airline self-service checkin kiosk Model Model validation Conclusions
Congnitive pyschology
Constraint satisfaction problems
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Airline self-service checkin kiosk Model Model validation Conclusions
Congnitive pyschology
Constraint satisfaction problems
Select language Select language Select language Select languageSelect airline Select airline Select airline Select airlineCheck liquids Check liquids Check liquids Check liquidsBooking reference Booking reference Booking reference Booking referenceCheck forbidden items Insert payment card Passport info PasswordSelect return seat Buy extra bag Select return seat Check forbidden itemsCheck luggage size Select return seat Check luggage size Select outbound seatPassport scan Check luggage size Check forbidden items Check luggage sizeBuy extra bag Check forbidden items Buy extra bag Buy extra bagConfirm Confirm Confirm ConfirmPrint boarding pass Print boarding pass Print boarding pass Print boarding passSelect outbound seat Select outbound seat Select outbound seat Select return seatPrint luggage tag Print luggage tag Print luggage tag Print luggage tag
Cost 5.53 5.88 8.18 8.42
Table: Optimal task ordering of the self-service check-in using differentauthentication mechanisms.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Airline self-service checkin kiosk Model Model validation Conclusions
User study
Simulated kiosk
Optimal group N = 40(AgeMean = 26.6, AgeSD = 7.2, 28females)
Pessimal group N = 20(AgeMean = 29.1, AgeSD = 13.5,15 females)
On completion participants filledout a subjective satisfactionquestionnaire
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Airline self-service checkin kiosk Model Model validation Conclusions
Self service checkin kiosk results
Completion times
Significant maineffects of Condition(F1,55 = 4.82, p =0.03) and Experience(F1,55 = 5.01, p =0.03)Significant maineffect of Repetition(F2,110 = 81.0, p <0.001)
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Airline self-service checkin kiosk Model Model validation Conclusions
Self service checkin kiosk results
Completion times by experience
Significant interaction ofRepetition and Experience(F2,110 = 5.09, p = 0.01)
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Airline self-service checkin kiosk Model Model validation Conclusions
Self service checkin kiosk results
User satisfaction
Directionally in favor ofthe Optimal ordering, thesatisfaction ratings werenot statisticallysignificantly
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Airline self-service checkin kiosk Model Model validation Conclusions
Self service checkin kiosk results
Ordering task
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Airline self-service checkin kiosk Model Model validation Conclusions
Self service checkin kiosk results
Ordering distance
Expert group (n=17),software/web settings(n=33), or both (n=6),with 5.2 mean years ofexperience (SD = 6.0)
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Airline self-service checkin kiosk Model Model validation Conclusions
Conclusions
The cost of a security measure is a function of its relationshipto the user’s actual goal
The impact of task switching has been operationalized usingresults from experimental psychology
Participants performed better and were more satisfied withthe model’s optimally order interface
The model’s optimal ordering was more similar to suggestedorderings of professional designers
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Airline self-service checkin kiosk Model Model validation Conclusions
Questions?
