Method for gesture based authentication in physical access ...1111553/FULLTEXT01.pdfMaster of...

Method for gesture based authentication in

physical access control

JOAKIM OLSSON

Master of Science Thesis

Stockholm, Sweden 2016

Method for gesture based authentication in physical access control

av

Joakim Olsson

Examensarbete MMK 2016:155 MDA 525

KTH Industriell teknik och management

Maskinkonstruktion

SE-100 44 STOCKHOLM

Examensarbete MMK 2016:155 MDA 525

Metod för gestbaserad autentisering för fysisk passerkontroll

Joakim Olsson

Godkänt

2016-06-16

Examinator

Martin Grimheden

Handledare

Bengt Eriksson

Uppdragsgivare

Assa Abloy

Kontaktperson

Fredrik Einberg

Sammanfattning ASSA Abloy är den största globala leverantören av intelligenta lås och säkerhetslösningar. Företaget strävar ständigt efter att utveckla nya och innovativa lösningar för fysisk passerkontroll. Ett koncept som företaget ville undersöka riktade sig mot att göra det möjligt för användaren att enkelt låsa upp en dörr med hjälp av gester, vilket resulterar i en användarvänlig

upplevelse. Tanken var att använda en wearable som en credential-enhet och identifiera användarens gester med sensorerna som tillhandahålls av denna. Gesten som används i denna avhandling var knackar, vilket innebär att användaren låser upp dörren genom att knacka på den. Huvudsyftet med detta arbete var att utveckla ett system som tillåter knackar att användas som en metod för autentisering och att utvärdera systemet baserat på systemsäkerhet och användarvänlighet. Systemet som har utvecklats består av två accelerometersensorer; en belägen i wearablen och en belägen i låset/dörren. Signalerna från varje sensor bearbetas och analyseras för att detektera knackar. Tidskorrelationen mellan knackar som detekteras av varje sensor analyseras för att kontrollera att de härstammar från samma användare. En teoretisk modell av systemet har utvecklats för att underlätta utvärdering av systemet. Utvärderingen av systemet visade att både systemetsäkerheten och användarvänligheten uppnår tillfredsställande värden. Denna avhandling visar att konceptet har stor potential men det krävs ytterligare arbete. Metoderna som har används för att utvärdera systemet i denna avhandling kan på samma sätt användas för att utvärdera system under fortsatt arbete.

Master of Science Thesis MMK 2016:155 MDA 525

Method for gesture based authentication in physical access control

Joakim Olsson

Approved

2016-06-16

Examiner

Martin Grimheden

Supervisor

Bengt Eriksson

Commissioner

Assa Abloy

Contact person

Fredrik Einberg

Abstract ASSA Abloy is the largest global supplier of intelligent locks and security solutions. The company constantly strives to develop new and innovative solutions for physical access control. One concept the company wanted to investigate aimed to allow the user to effortlessly unlock a door using gestures, resulting in a seamless experience. The idea was to use a wearable as a credential device and identifying the user gestures with the sensors supplied by the wearable. The gesture used in this thesis project were knocks, meaning that the user unlocks the door by knocking on it. The main objective of this thesis project was to develop a system allowing knocks to be used as a method of authentication and evaluate the system based on system security and user convenience. The system developed consists of two accelerometer sensors; one located in the wearable and one located in the lock/door. The signals from each sensor are processed and analyzed to detect knocks. The time correlation between the knocks detected by each sensor are analyzed to verify that they originate from the same user. A theoretical model of the system was developed to facilitate the evaluation of the system. The evaluation of the system showed that both the system security and the user continence attained satisfying values. This thesis shows that the concept has high potential but further work is needed. The framework of methods used to evaluate the system in this thesis can in the same way be used to evaluate systems during any further work.

AcknowledgementAcknowledgementAcknowledgementAcknowledgement

I would like to thank my supervisor Bengt Eriksson, my company supervisor Fredrik Einberg and Håkan Olsson for the incredible help and guidance throughout the thesis project.

NomenclatureNomenclatureNomenclatureNomenclature

NotationsNotationsNotationsNotations

Symbol Description

&'()(*+, The true detect rate of the system.

&'+.. The true detect rate of the event detection algorithm in the door.

&'+./ The true detect rate of the event detection algorithm in the wearable.

&',0 The true detect rate of the matching algorithm.

&1 The size of the matching window.

3.445 The size of total possible error range for the door.

3/+05067+ The size of total possible error range for the wearable.

389 Time error due to sampling frequency.

;( Sampling frequency.

3+.. Time error due to the event detection algorithm in the door.

3+./ Time error due to the event detection algorithm in the wearable.

3*( Time error due to the time synchronization.

31 The offset between two events.

3/ The time error for any event detected in the wearable.

3. The time error for any event detected in the door.

<. The probability that an event detected in the door has the error

value 3..

</ The probability that an event detected in the wearable has the error

value 3/.

<1 The probability that the offset between two events has the value 31.

<10*=> The probability of the wearable detecting an event within the set

matching window.

?@ Event detection rate in the wearable.

'@ Event detection rate in the door.

A0**0=B Time needed to successfully spoof the system.

ℎD Hysteresis.

E The size of each addition to the hysteresis.

ContentsContentsContentsContents

Introduction ................................................................................................................................. 1 1.1 Background ........................................................................................................................................... 1

1.1.1 Concept ........................................................................................................................................... 1 1.2 Purpose ................................................................................................................................................... 2 1.3 Limitations ............................................................................................................................................. 2 1.4 Method .................................................................................................................................................... 2 1.5 Results ..................................................................................................................................................... 3

Second factor authentication design .................................................................................. 5 2.1 The overall system.............................................................................................................................. 5 2.2 Time synchronization........................................................................................................................ 7 2.3 Matching algorithm ............................................................................................................................ 7 2.4 User interactions ................................................................................................................................. 8

2.4.1 Forms of interactions ................................................................................................................ 8 Theoretical model of system characteristics ................................................................. 11

3.1 Convenience ........................................................................................................................................ 11 3.1.1 Matching algorithm true detect .......................................................................................... 11

3.2 Security ................................................................................................................................................. 23 3.3 Conclusion ........................................................................................................................................... 27

Data acquisition ........................................................................................................................ 29 4.1 Test environment setup ................................................................................................................. 29 4.2 Measurements .................................................................................................................................... 31

Data analysis .............................................................................................................................. 35 5.1 Door signals ......................................................................................................................................... 35 5.2 Wearable .............................................................................................................................................. 41 5.3 Conclusion ........................................................................................................................................... 46

Event detection algorithm .................................................................................................... 48 6.1 Signal processing............................................................................................................................... 48

6.1.1 High pass filter .......................................................................................................................... 48 6.1.2 Derivative .................................................................................................................................... 51 6.1.3 Interaction location in the processed signals ................................................................ 51

6.2 Interaction detection ....................................................................................................................... 52 6.2.1 Method 1 – Detecting the slope of the first peak.......................................................... 52 6.2.2 Method 2 – Detecting the top of the first peak .............................................................. 53 6.2.3 Method 3 – Detecting the top of the highest peak ....................................................... 54 6.2.4 Conclusion ................................................................................................................................... 55

6.3 Optimization and simulation ........................................................................................................ 55 6.3.1 Simulation Run .......................................................................................................................... 58

6.4 Results ................................................................................................................................................... 59 System Evaluation .................................................................................................................... 64

7.1 Attack mitigation ............................................................................................................................... 69 Results .......................................................................................................................................... 74

8.1 User convenience .............................................................................................................................. 74 8.2 System security .................................................................................................................................. 74

8.2.1 Avrage case scenario ............................................................................................................... 75 8.2.2 Worst case scenario ................................................................................................................ 75 8.2.3 Environmental aspects ........................................................................................................... 76

8.3 Conclusion ........................................................................................................................................... 76 References ............................................................................................................................................................. 77

1

IntroductionIntroductionIntroductionIntroduction 1.11.11.11.1 BackgroundBackgroundBackgroundBackground ASSA Abloy is the largest global supplier of intelligent locks and security solutions [1]. With smartphones being an essential part of most people’s everyday life, using the phone as a credential device to unlock the door is a natural evolution. ASSA Abloy has a mobile key solution on the market today using mobiles as the credential device. The required credentials is transmitted from the mobile to the lock using Bluetooth Smart Technology triggered by the user pushing a button in the mobile app. With today’s solution the user has to bring out the phone from the pocket. The desired user experience is to be able to open the lock without interacting with the phone resulting in a seamless experience. To achieve this a second factor authentication step has to be implemented where the key problems to solve is as;

1. Determine if the user intends to open a door. 2. Determine which door the user intends to open.

Wearables are emerging strongly on the market and it is obvious that these, just like smartphones, can be used as a credential device. ASSA Abloy wants to investigate the possibilities of using wearables to fulfill the requirements stated above.

1.1.11.1.11.1.11.1.1 ConceptConceptConceptConcept The concept is to develop a movement based sensing system using a wearable and a door sensor. The user unlocks the door by interacting with it while wearing the credential device, i.e. the wearable. By detecting the interaction using the sensor in the wearable the system can verify user’s intent to open a door, fulfilling the first requirement. By detecting the interaction using the door sensor the system verifies that someone intends to open that door. To link these together and verify that a certain user intends to open that specific door the time correlation between the interaction detected in the wearable and the one detected in the door are analyzed to verify that both sensors detected the same user interaction, fulfill the second requirement. The concept is illustrated in Figure 1.

CHAPTER 1. INTRODUCTION

2

Figure 1 – Concept

1.21.21.21.2 PurposePurposePurposePurpose The aim of this thesis is to develop and evaluate the concept of the second factor authentication method described. The main focus is estimating the reliability, security and user convenience. Other aspects like power consumption and implementation complexity will be considered but not focused on. Based on the results this thesis will present key aspects of developing the system and try to answer the question; “Is it possible to develop a user convenient yet secure system?”.

1.31.31.31.3 LLLLimitationsimitationsimitationsimitations A number of limitations were presented to the thesis work. These limitations relates heavily to hardware used and are as follow;

• There was no real time communication between the sensors limiting the ability to test the developed system as a whole. Instead the parts of the system were tested and evaluated individually.

• As a result of not being able to test the system as a whole the accuracy of the theoretical model used to evaluate the system could not be verified.

• The limited sampling frequency of the sensors made it difficult to investigate certain problems mentioned in the report.

1.41.41.41.4 MethodMethodMethodMethod Iterative meetings with the company constituted an important part of the work where design decisions and project focus were discussed and further specified throughout the work. The entire work can be divided into seven main parts and are as follow;


3

Literature survey Literature survey Literature survey Literature survey A theoretical study of relevant topics were conducted to gain knowledge about the subject at the start of the project. Focused literature surveys were conducted based on relevant topics identified either by a previous survey or by the company meetings.

Testing Testing Testing Testing eeeenvironmentnvironmentnvironmentnvironment A testing environment including sensors and software was set up to record and analyze sensor signals.

Data aData aData aData acquisicquisicquisicquisitititition and on and on and on and analysisanalysisanalysisanalysis A series of use cases were constructed, recorded and analyzed. This was to gain knowledge about the signal characteristics and signal behavior to facilitate the algorithm development.

SystemSystemSystemSystem designdesigndesigndesign Concept generation and system design based on the knowledge gained from the previous steps. A theoretical model of the system characteristics were developed to facilitate the evaluation of the system.

Algorithm DevelopmentAlgorithm DevelopmentAlgorithm DevelopmentAlgorithm Development Algorithms and signal processing were developed and evaluated using the data recorded during the data acquisition. EvaluationEvaluationEvaluationEvaluation Evaluating the system based on the theoretical model and the results gained from the previous steps. ConclusionConclusionConclusionConclusion Based on the evaluation a conclusion of the system was made considering convenience and security.

1.51.51.51.5 ResultsResultsResultsResults This thesis will show that by using the described concept it is faceable that a secure and still user convenient system can be attained.


4

5

Second factor authenticationSecond factor authenticationSecond factor authenticationSecond factor authentication designdesigndesigndesign This chapter will further explain the second factor authentication method. The second factor authentication method will be referred to as the system throughout this report.

2.12.12.12.1 The oThe oThe oThe overall systemverall systemverall systemverall system The system is made up by six components; wearable and door sensor, two event detection algorithms (one associated to each sensor), a matching algorithm and time synchronization, illustrated in Figure 2.

Figure 2. User and door with system.

The signals from the two sensors is analyzed by its corresponding event detection algorithm. When a predefined interaction is detected by an algorithm, this will be referred to as an event, the time of the event is sent to the matching algorithm. The matching algorithm analyze the time correlation between two events detected by each event detection algorithm to verify that they originates from the same interaction. Since the matching is done based on the times of the events relative each other, time synchronization is needed.

CHAPTER 2. SECOND FACTOR AUTHENTICATION DESIGN

6

The system is used as a second factor authentication step following a first factor based on existing standard cryptographic protocols. The sequence is as follow. 1st factor1st factor1st factor1st factor

- RF connection is established, e.g. via BLE (Bluetooth Low Energy). - Cryptographic authentication is performed base on a shared secret key. - Exchange of access credential data. Lock verifies if credential is valid.

2nd factor2nd factor2nd factor2nd factor - Sync clocks - User interaction. - Interaction are sensed by wearable and door sensor. - Interactions are matched in time. - If match, unlock.

Figure 3 illustrates the procedure of the first and second authentication in a sequence diagram.

Figure 3. Sequence diagram of first and second factor authentication.


7

2.22.22.22.2 Time synchronization Time synchronization Time synchronization Time synchronization ASSA Abloy already uses BLE technology to communicate with devices such as smart phones and for this thesis; wearables. This thesis is based on using the BLE communication technology and the well-known precision time protocol IEEE 1588 [2] for time synchronization. Due to the queue structure in the BLE communication a possible time synchronization error was considered. This thesis did not do any experiments to determine the exact size of this error but used a value of 10 milliseconds as a maximum time synchronization error.

2.32.32.32.3 Matching algorithm Matching algorithm Matching algorithm Matching algorithm The matching algorithm is to analyze the time correlation between two events to verify that the events occurred at the same time and thus from the same user interaction. The two events detected in the door and the wearable should in theory be detected at the same time given that they originates from the same user interaction. But due to error factors in the system, e.g. the 10 millisecond time synchronization error, any event has the risk of being detected with a time error. This entails that two events originating from the same user interaction has the risk of differ in time, i.e. there is a risk that there is an offset between the two. To still be able to match the two events the offset has to be accounted for. This is done by implementing a matching window which defines an allowed offset between the two events, illustrated in Figure 4.

Figure 4 – Illustration of the matching algorithm. In the left hand side the offset between the two events is

compensated for by the matching window and the two events are considered a match. In the right hand side the offset exceeds the matching window and the events are not considered a match.

The figure shows two scenarios; one where the offset is compensated for by the matching window and one where the offset exceeds the matching window. The size and shape of the matching window will be further explained in Chapter 3.


8

2.42.42.42.4 User interactionsUser interactionsUser interactionsUser interactions The system uses predefined user interactions to verify the user’s intent to open. The possible forms of interactions to use relies directly on the sensors available for implementation, mainly the wearable sensor.

Wearable sensorWearable sensorWearable sensorWearable sensor When choosing a wearable sensor two aspects were considered; what sensors are available if using a commercial wearable and what sensor would be suitable if designing a wearable intended for this system only. Using a commercial wearable on today’s market the sensors available depends on the selected model. The majority of today’s models include an accelerometer and a gyroscope [3-5]. When designing a wearable intended for this system the price and implementation complexity are the largest determining factors. A suitable sensor, taking both aspects into account, was an accelerometer sensor. An accelerometer is cheap and easy to implement if developing a new wearable and it is included in the majority of the commercial wearables on today’s market.

Door sensorDoor sensorDoor sensorDoor sensor The choice of door sensor is not limited in the same way as the wearable sensor in regards that no limitations by commercial products has to be considered. The door sensor can therefore be chosen based on the user interaction.

2.4.12.4.12.4.12.4.1 Forms of interactionsForms of interactionsForms of interactionsForms of interactions Using an accelerometer as the wearable sensor two forms of interactions is considered; user reaches and grabs the door handle and user knocks on the door. Both forms of interaction is to be matched the same way, by the time correlation of the events, but would require different door sensors and event detection algorithms.

User reaches and grabs the door handleUser reaches and grabs the door handleUser reaches and grabs the door handleUser reaches and grabs the door handle A door sensor which is suitable for this concept is a binary touch sensor located on the door handle. The event detection algorithm detects and marks the time of the door event when the sensor is enabled by the user. To detect the interaction in the wearable the characteristic movement of reaching for and grabbing the door handle has to be identified and distinguishable from other forms of user movements. This is done using gesture recognition based on Hidden Markov Models [6]. The


9

event detection algorithm in the wearable detects and marks the time of the wearable event when the user’s hand touches the door handle.

User knocksUser knocksUser knocksUser knocks on dooron dooron dooron door Whit this concept a binary door sensor could be used, e.g. a touch sensor or a simple button. With these types of sensors it is required that the user interacts directly with the sensor. To avoid this an accelerometer is chosen as both the door and wearable sensor. Using an accelerometer as the door sensor allows the sensor to be located inside the lock and for the user to interact with the entire door. Both event detection algorithms detects and marks the time of the events as the moment the user’s hand hits the door. For this thesis the second form of interaction, user knocks on door, is used.


10

11

Theoretical model of Theoretical model of Theoretical model of Theoretical model of systemsystemsystemsystem characteristicscharacteristicscharacteristicscharacteristics The following chapter will define and explain the theoretical model used to determine the characteristics of the system. The purpose of the model is to be used to evaluate the system. The characteristics of the system is divided into two categories; convenience and security. The convenience correlates to the ability to successfully unlocking the door when a valid interaction is introduced by an authorized user. The security correlates to the risk of wrongfully unlocking the door.

3.13.13.13.1 Convenience Convenience Convenience Convenience The convenience is measured by the system true detect rate. This is defined as the probability that a valid interaction introduced by an authorized user is detected by both the event detection algorithms and that the matching algorithm successfully match the two events. A low true detect entails a higher risk of not unlocking the system when supposed to resulting in a lower system convince. Let be the system true detect where &'+.. and &'+./ are the true detect of the event detection algorithm in the door and in the wearable and &',0 is the true detect of the matching algorithm. The true detect of the event detection algorithms is defined as the percentage of valid interactions detected and will be analyzed and discussed in Chapter 6. The true detect of the matching algorithm is defined as the percentage of successful matches given two events originating from the same valid interaction.

&'()(*+, d &'+..&'+./&',0 Eq. 1

3.1.13.1.13.1.13.1.1 Matching Matching Matching Matching aaaalgorithmlgorithmlgorithmlgorithm true detecttrue detecttrue detecttrue detect When matching two events the matching algorithm analyzes the time correlation between the two. To account for any offsets between two events a matching window is used, as explained in Chapter 2.3. To determine the design of the matching window the possible offsets between the events has to be analyzed. The offset is depending on time errors introduced by the system. This thesis considers three forms of time errors;

CHAPTER 3. THEORETICAL MODEL OF SYSTEM CHARACTERISTICS

12

Time error due to the event detection algorithmsTime error due to the event detection algorithmsTime error due to the event detection algorithmsTime error due to the event detection algorithms The event detection algorithms has a risk of detecting an event either before or after the time of the interaction. This error can vary in size for any individual interaction. Let 3+.. be the possible time error introduced by the event detection algorithm in the door and let 3+./ be the possible time error introduced by the event detection algorithm in the wearable.

Time error due to sampling frequencyTime error due to sampling frequencyTime error due to sampling frequencyTime error due to sampling frequency Each sensor is using a sampling frequency. This entails a risk of detecting an event one sample either too fast or too slow making the size of the time error due to sampling frequency

389 d 2;f

where ;( is the sampling frequency in Hz.

Time error due to the time synchronization Time error due to the time synchronization Time error due to the time synchronization Time error due to the time synchronization To be able to use time as a reference between the two sensors their clocks has to be synchronized, explained in Chapter 2.2. Using the door sensor as the master and the wearable sensor as the slave the event detected in the wearable has the risk of attaining a time error relative the door, this error is defined as 3*(. The considered time errors entails that any event detected has the risk of being affected by the time error due to sampling frequency. Events detected in the door has the risk of being effected by an additional time error introduced by the event detection algorithm in the door. Events detected in the wearable has the risk of being affected by additional time errors introduced by the event detection algorithm in the wearable and by the time synchronization. Let

3.445 d 3+.. + 389 be the size of total possible error range for the door and

3/+05067+ d 3+./ + 3*( + 389 be the size of the total possible error range for the wearable. Figure 5 illustrates the individual and the total possible time errors for both the door and the wearable. The time of the interaction is marked by a black line where the event can be detected with a possible error, marked by the gray windows.


13

Figure 5 - Error factors

Let 3. be the time error for any event detected in the door and be defined as

3. d Ah i Aj where Aj is the time of the interaction and Ah is the time the event was detected. 3.445 is divided into k. parts where each part represents a separate value of 3.. The same is done with 3/+05067+ where 3/ is the time error for the events detected in the wearable. Figure 6 illustrates the concept.


14

Figure 6 - Visual representation of 3.445 and 3/+05067+

To compensate for these errors a matching window is used. Using the event detected in the door as a reference the algorithm checks if there is an event detected in the wearable within the set time limit defined by the matching window. Given any two events detected in the door and the wearable the offset between the two events that is to be compensated for by the matching window is describes as

as illustrated in Figure 7.

31mn d 3/o i 3.p

Eq. 2


15

Figure 7 - Visual representation of 31mn

Figure 8 illustrates where &1 is the size of the matching window and 31 is the offset values needed to be compensated for.

Figure 8 – Visual representation of the matching window.

The minimum value of the offset is defined as

and the maximum value of the offset is defined as

31q d 3/q i 3.Dr Eq. 3


16

31Ds d 3/,t i 3.q. Eq. 4

The size of the matching window is defined as

Using Eq. 3 and Eq. 4 in Eq. 5 gives

&1 d 3.445 + 3/+05067+ Eq. 6

ExampleExampleExampleExample 3.13.13.13.1 To illustrate this an example is used. Let

3.445 d 3 milliseconds where i1 ≤ 3. ≤ 2 milliseconds and

3/+05067+ d 5 milliseconds where i2 ≤ 3/ ≤ 3 milliseconds. Eq. 6 gives the matching window

&1 d 3.445 + 3/+05067+ d 8 milliseconds. Eq. 3 and Eq. 4 gives

i4 ≤ 31 ≤ 4 milliseconds. Figure 9 illustrates the example.

&1 d 31Ds i 31q.

Eq. 5


17

Figure 9 – Visual representation of Example 3.1.

For this example the matching window is eight milliseconds and compensates for all possible offsets resulting in a one hundred percent true detect. The matching window compensates for offsets from minus four to plus four milliseconds relative the event detected in the door. Calculating the matching algorithm true detect Calculating the matching algorithm true detect Calculating the matching algorithm true detect Calculating the matching algorithm true detect The true detect of the matching algorithm is defined as the percentage of successful matches given two events originating from the same interaction. By using a matching window

&1 v 3.445 + 3/+05067+ all possible errors considered are compensated for and the true detect attains a value of one hundred percent. In cases where the size of the matching window is reduced, i.e.

&1 w 3.445 + 3/+05067+,


18

the true detect of the algorithm decrease. This part will explain the method used to determine the true detect of the algorithm for those cases. When reducing the size of the matching window there is a risk that the errors from the two events will result in an offset greater than the ones compensated for by the matching window. To determine the risk of this occurring the probability that an event having a certain error has to be considered. Let <.B be the probability that an event detected in the door has the error value 3.B and let </B be the probability that an event detected in the wearable has the error value 3/B. Figure 10 illustrates the concept.

Figure 10 – Visual representation of the variables 3.445 and 3/+05067+ .

In a similar way let <1B be the probability that the offset between two events has the value 31B, illustrated in Figure 11.


19

Figure 11 – Visual representation of the matching window.

The probability that two events would attain a certain offset 31 is defined as

Reducing the size of the matching window can be done by reducing from the left side, the right side or from both sides. The true detect of the matching algorithm with an reduced matching window is defined as the probability that two events would attain an offset compensated for by the reduced matching window, i.e.

where x7 is the number of reductions from the left and x5 is the number of reductions from the right.

<1B d y </o<.po,p:+tn{+m|+s}

Eq. 7

&',0~�~� d y <1o

Ds{5~

o|q�5�

Eq. 8


20

ExampleExampleExampleExample 3.23.23.23.2 This is further illustrated using an example with the same values as in example 3.1 and adding the probability values for each 3. and 3/. Figure 12 illustrates the result of simulating twenty events, ten for each sensor, with randomized error values 3. and 3/.

Figure 12 - Simulation

Using the data from the simulation 3. and <. for each position in 3.445 was attained, see Table 1. The same was be done for 3/+05067+, see Table 2.

Table 1 – Error and probability values for the door.

Position k.q k.� k.� k.�

3. -1 0 1 2

Events detected 1 5 2 2

<. 0.1 0.5 0.2 0.2


21

Table 2 – Error and probability values for the door.

Position k/q k/� k/� k/� k/� k/�

3/ -2 -1 0 1 2 3

Events Detected 2 2 1 1 2 2

</ 0.2 0.2 0.1 0.1 0.2 0.2

Eq. 2 gives the offset 31 for every combination of events detected in the door and wearable. Table 3 shows the result where the combinations with the same offset is marked with the same color.

Table 3 – Error combinations and their resulting offset.

��

�� -1 0 1 2 3 4

�� -2 -1 0 1 2 3

�� -3 -2 -1 0 1 2

�� -4 -3 -2 -1 0 1

The probability of that the combinations of events will occur is calculated using </<., see Table 4.


22

Table 4 – Probability for each error combination.

</q </� </� </� </� </�

<.q 0.02 0.02 0.01 0.01 0.02 0.02

<.� 0.1 0.1 0.05 0.05 0.1 0.1

<.� 0.04 0.04 0.02 0.02 0.04 0.04

<.� 0.04 0.04 0.02 0.02 0.04 0.04 Eq. 7 gives <1 for each offset, see

Table 5.

Table 5 – Probability values for the offsets.

The matching window is reduced to three milliseconds by reducing two milliseconds from the right and three milliseconds from the left, i.e. x5 d 2 and x7 d 3. Eq. 8 gives the true detect

&',0�� d y <1o�

o|�d 0.16 + 0.13 + 0.14 + 0.15 d 0.58

i.e. by reducing the matching window to three milliseconds the true detect of the matching algorithm decreases to 58 percent.

Position k1q k1� k1� k1� k1� k1� k1� k1� k1�

38 -4 -3 -2 -1 0 1 2 3 4

<8 0.04 0.08 0.16 0.16 0.13 0.14 0.15 0.12 0.02


23

3.23.23.23.2 SSSSecurityecurityecurityecurity The security of the system correlates to the risk of matching two events that does not originate from the same interaction, i.e. the risk that an event detected in the wearable is matched with an event detected in the door introduced by a second unauthorized user. When detecting an event in the door the matching algorithm checks if an event is detected in the wearable within the time frame determined by the matching window. This means that a second, unauthorized, user could trigger an event in the door that has a risk of unlocking the system given that the wearable detects an event at the same time. To measure the security of the system two measurements were used. The first one correlates to the scenario where the second unauthorized user accidently triggers an event in the door. This will be referred to as an accidental unlock and is measured by the risk of matching a single event in the door with an event in the wearable. The second measurement correlates to the scenario where the second user intentionally tries to spoof the system by continuously triggering events in the door. This will be referred to as an attack unlock and is measured by the time it takes for the second unauthorized user to successfully spoof the system to unlock. Accidental uAccidental uAccidental uAccidental unlocknlocknlocknlock The probability that an event detected by the door is matched with an event detected by the wearable where these are not originating from the same interaction is depending on two parameters; &1 – The size of the matching window measured in seconds. ?@ – The rate of events being detected by the wearable measured in events per second. Figure 13 illustrates the parameters &1 and ?@.

Figure 13 – Illustration of the parameters &1 and ?@ . The inverse ?@{q is the time in seconds it takes for the

wearable to detect one event, i.e. one event will be detected at some point during this time.


24

In Figure 13, one event is expected to be detected during the time ?@{q. This time is divided into timeslots, each with the size of the matching window &1. The timeslots can be compared to the sides on a dice where the probability that the one event detected by the wearable is detected within the matching window is the same as the probability to roll one certain number on the dice. Given that dice has

k d ?@{q

&1

sides (in the illustration seen in Figure 13 – Illustration of the parameters &1 and ?@. The inverse ?@{q is the time in seconds it takes for the wearable to detect one event, i.e. one event will be detected at some point during this time.Figure 13 there is five sides on the dice). The probability < of rolling one certain number on the dice is defined as

< d 1k d 1

?@{q&1

d &1?@{q d &1?@ .

Given that the door algorithm detects one event; let

<10*=> d min (&1?@ , 1) Eq. 9

be the probability of the wearable detecting an event within the set matching window. Since the product &1?@ can attain values greater than one and p�� is a probability the min-function is included in Eq. X.

AttackAttackAttackAttack To investigate the time it would take to spoof the system, let � be a random variable describing the number of events needed to be detected by the door to successfully match with the wearable for the first time. Then, � can be geometric distributed [7] with the well-known probability function

�(� d k) d <�D{q, � d 1 i <


25

and expected value

�(�) d y k<�D{q d 1<

D|q.

Letting

< d <10*=> gives the expected number of events needed for a successful match for the first time

�(�) d 1<10*=>

d 1&1?@

.

Using

¡ d 1&1?@

as the number of events needed to be detected by the door to successfully spoof the system, the time needed can be described as

A0**0=B d ¡'@

d 1&1?@'@

Eq. 10

where '@ is the rate of events being detected in the door measured in events per second.


26

Example 3.3Example 3.3Example 3.3Example 3.3 To better illustrate this an example is used. Let there be two users; one authorized user wearing the wearable and one unauthorized user. The wearable worn by the authorized user has contact and is communicating with the system. The authorized user is performing an activity where the user’s movement is detected as events. The user movement generates on average one event every minute, i.e.

?@ d 0.0167. In a situation where the unauthorized user accidently interacts with the door, generating one single event in the door the risk of matching the door event with an event detected in the wearable is given by Eq. 9. Let the matching window be 10 milliseconds, i.e.

&1 d 0.001 Eq. 9 gives

<,0*=> d min(&1?@ , 1) d min(0.001 ∗ 0.0167, 1) d 0.000017. This means that there is a probability of 0.000017, approximately 1 in 60000, that the system accidentally unlocks due to the unauthorized user’s accidental interaction. If the unauthorized user intends to attack the system by keep interacting with the door, generating more events, the time it will take to successfully unlock the door is given by Eq. 10. Let the second user generate three events per second in the door, i.e.

'@ d 3 Eq. 10 gives

A0**0=B d 1&1?@'@

d 10.001 ∗ 0.0167 ∗ 3 d 20000.

It would take 20000 seconds, approximately 5.6 hours, for the unauthorized user to successfully spoof the system and unlock the door.


27

3.33.33.33.3 ConclusionConclusionConclusionConclusion To determine the convenience and the security of the system three measurements were used; the true detect of the system

&'()(*+, d &'+..&'+./&',0 , the risk of accidentally unlocking the system

<10*=> d min (&1?@ , 1) and the time it would take to successfully unlock the system by an attack

A0**0=B d 1&1?@'@

.

In order to acquire the values for each of the measurements parameters characteristic to the system and user scenarios was needed. Figure 14 illustrates the relationship between the three measurements and the parameters needed to acquire the measurement values.

Figure 14 - Parameter relationships


28

As seen in the figure all three measurements are depending on the size of the matching window. The remaining parameters is acquired from certain parts of the system. In order to evaluate the system convenience and security of the system used in this thesis all the mentioned parameters has to be attained. Table 6 explains the parameters and the corresponding part in this thesis where they will be acquired.

Table 6 - Table of needed parameters to evaluate the system and their corresponding section in this thesis.

Parameter(s) Description Corresponding part in this thesis

&'+.. and &'+./

The true detect of the event detection algorithms used in the door and in the wearable.

These will be acquired in Section 6.

Error Factors The error factors of the system. These are based on the errors from the event detection algorithms and the time synchronization.

The error from the event detection algorithms will be acquired in Section 6 and the error from the time synchronization was defined as 10 milliseconds in chapter 2.4.

?@ and '@

The event detection rate in the wearable and the door. These are depending on user scenarios.

These will be discussed in Section 7.

&',0 The true detect of the matching

algorithm. This will be acquired in Section 7.

&1 The size of the matching window. This will be discussed and

acquired in Section 7.

29

Data acquisitionData acquisitionData acquisitionData acquisition This chapter will describe the testing environment setup and the sets of use cases constructed and used for the data acquisition.

4.14.14.14.1 TestTestTestTest environmentenvironmentenvironmentenvironment setupsetupsetupsetup To be able to acquire and analyze data a testing environment was set up. The environment consists of one door sensor, one wearable sensor and a computer to analyze the sensor data, Figure 15 illustrates.

Figure 15 - Testing environment

After investigating different sensor options the final sensors used for both the door and the wearable is the EVAL-ADXL345Z-DB, a data logger with an integrated accelerometer developed by Analog Devices. The accelerometer data is recorded and stored on a SD card which is transferred to the computer where the data is analyzed using Matlab [8]. The accelerometer sensor specifications is as follow;

CHAPTER 4. DATA ACQUISITION

30

• Sampling Frequency: 400 Hz

• Range: ±2 g

• Resolution: 10-bit

The choice of sampling frequency of 400 Hz is not optimal as will be seen later in this report. But for a low power system and today’s most common wearables a higher sampling frequency would not be realistic.

The two loggers are each mounted inside an electric box for protection and ease of mounting, see Figure 16. The boxes are each supplied with the necessarily attachments to be mounted correctly on either the door or on the wrist of the user, see Figure 17.

Figure 16 - Logger mounted in electric box


31

Figure 17 - Sensors mounted

4.24.24.24.2 MeasurementsMeasurementsMeasurementsMeasurements

A series of measurements were recorded. Each sensor was interacted with separately during a separate time. The purpose of this is to get a better understanding of the signal behavior for that specific sensor facilitating the development of the event detection algorithms.

InteractionsInteractionsInteractionsInteractions

Three types of interactions are used during the measurements, see Table 7. The measurements are divided into door measurements and wearable measurement, each recorded by the specific sensor.

Table 7 - Forms of interactions


32

Interaction type Description

Knock A standard, audible knock, using the knuckle of the hand.

Tap Comparable to typing on a keyboard or a knock using the tip of the finger.

Touch A light touch, comparable to the touch used when interacting with a smartphone.

Door Door Door Door measurements measurements measurements measurements

For the door measurements, in addition to forms of interaction, the distance from the sensor the interaction took place varies between 0-60 cm. Each type of interaction is recorded 10 times at the specified distances resulting in 18 data sets, see Table 8.

Table 8 – Door data sets

Type/Distance 0 cm 3 cm 10 cm 20 cm 30 cm 60 cm

Knock Door 1 Door 2 Door 3 Door 4 Door 5 Door 6

Tap Door 7 Door 8 Door 9 Door 10 Door 11 Door 12

Touch Door 13 Door 14 Door 15 Door 16 Door 17 Door 18


33

Wearable Wearable Wearable Wearable measurementsmeasurementsmeasurementsmeasurements

For the wearable measurements, in addition to forms of interaction, the user movement before and after the impact between hand and door varies, see Table 9.

Table 9 - User movements

Name of movement Movement before interaction Movement after interaction

Stay at door Users hand is positioned approximately 10 cm from the door.

User lets the hand rest at the door.

Return Users hand is positioned approximately 10 cm from the door.

User returns the hand to the starting position.

From waist User brings the hand up from the waist.

User returns the hand to the waist.

Walk up User walks up to the door from approximately 3 meters.

User lowers the hand to the waist.

Each interaction is recorded 10 times with the specified user movement resulting in 12 data sets, see Table 10.

Table 10 - Use cases wearable

Type/Movement Stay at door Return From waist Walk up

Knock Wearable 1 Wearable 2 Wearable 3 Wearable 4

Tap Wearable 5 Wearable 6 Wearable 7 Wearable 8

Touch Wearable 9 Wearable 10 Wearable 11 Wearable 12


34

35

Data analysis Data analysis Data analysis Data analysis

This chapter will analyze the recorded signals from the data sets defined in Table 9 and Table 10 in chapter 4.2. The purpose of the analysis is to gain knowledge about the signal behavior by identifying the interactions in the signal and defining important signal characteristics. This is done to facilitate the development of the event detection algorithms. The signals from each sensor is analyzed individually and the position of each interaction in the signals are manually defined and stored to be used as the true position of the interaction for further development.

5.15.15.15.1 DoorDoorDoorDoor ssssignalsignalsignalsignals

This chapter will analyze the signals recorded by the door sensor (defined in Table 8) and investigate the signal characteristics for each form of interaction (defined in Table 7) and the impact of the distance between sensor and point of interaction.

AxesAxesAxesAxes

Each recording consists of three signals, one for each of the three sensor axes illustrated in Figure 18. Initial testing showed that the interactions could be identified on all three axes where the most significant appearance can be observed on the axis parallel to the interaction direction, in this case the z-axis. Because of this only the z-axis is used and analyzed where the remaining two axes are discarded.

CHAPTER 5. DATA ANALYSIS

36

Figure 18 - Door sensor axes

KnockKnockKnockKnock InteractionInteractionInteractionInteraction

The first form of interaction analyzed is the knock. Figure 19 illustrates signal from the z-axis from the recording Door 1 where ten interactions in the form of knocks are performed directly on the door sensor.

Figure 19 Signal "Door1"

The figure shows that the signal has no offset and low noise where all ten knocks can easily be identified in the form of a distinctive change in amplitude. The differences in the amplitude of the knocks is explained by two factors. The first factor is the human inconsistency, e.g. the user did not use the same amount of force for each knock. The second


37

factor is due to the sampling frequency. The sampling frequency of the sensor is 400 Hz, this entails that the time between each sample in the signal is 2.5 milliseconds. If the change of the amplitude occurs very fast, there is a risk of not detecting the sample with the highest amplitude, Figure 20 illustrates. This inconsistency can be seen in the majority of the recordings.

Figure 20 - Illustration of how the sampling frequency effects the signal characteristics

To better understand the characteristics of the knock Figure 21 illustrates one of the knocks closer.

Figure 21 - Zoom of one knock. The sensor is at rest (1). The moment the users hand hits the door (2) it displaced the door and the sensor resulting in a positive z-acceleration. The door springs back (3) to the original position resulting in an oscillatory behavior until it settles (4).


38

To define the characteristics of the interactions the maximum amplitude change, the duration for the door to return to its original position and the time of the interaction are used. The following characteristics can be seen in knock illustrated above;

• Amplitude: 2 g.

• Duration: ~15 milliseconds.

• Time of interaction: 4.5735 seconds (Marked (2) in the figure).

The impact of the distance between the sensor and the knock is analyzed using recoding Door 6, illustrated in Figure 22.

Figure 22 - Door6

The knocks are performed 60 centimeters from the sensor and the figure shows that the increased distance results in a decrease of the amplitude change caused by the knocks. The average maximum amplitude of all interactions is reduced to 1 g. The average duration of each knock is ~15 milliseconds, i.e. the same as for recording Door 1. Figure 23 illustrates one knock from Door 6 where the time of interaction is defined to be at 7.694 seconds.


39

Figure 23 - Zoomed Door 6

Tap interaction Tap interaction Tap interaction Tap interaction

The second form of interaction is the tap. Figure 24 illustrates the signal from recording Door 7 where the taps were performed directly on the sensor.

Figure 24 - Door 7

The figure shows that all ten taps are clearly distinguishable from the signal where the average maximum amplitude change of the taps are 1.6 g. Figure 25 illustrates one tap where the same signal behavior as for the knock can be observed. The time for the door to settle is on average ~15 milliseconds, i.e. same as for the knocks.


40

Figure 25 - Closer view of tap

An increased distance between sensor and point of interaction decreased the maximum amplitude change for the knocks, the same behavior can be seen for the taps. Figure 26 illustrates the signal from recording Door 8 where the taps are performed 3 cm from the sensor.

Figure 26 - Door 8

The figure shows that only eight of the ten taps are clearly observable and that the average maximum amplitude change is approximately 0.25 g. A further increase of the distance between sensor and point of interaction decreases the amplitude to the point where none of the taps are clearly observable.

Touch interactionTouch interactionTouch interactionTouch interaction

The third and lightest form of interaction is the touch. Figure 27 shows the signal from recording Door 13 where ten touched are performed directly on the sensor.


41

Figure 27 - Touch interaction

As seen in the figure none of the touches are observable. For the remaining recordings (Door 14-18) the distance between the sensor and point of interaction are increased and none of the interactions are observable.

5.25.25.25.2 WearableWearableWearableWearable

This chapter will analyze the signals recorded by the wearable sensor (see Table 9) and investigate the signal characteristics for each form of interaction and the impact of the user’s movement before and after the interaction.

AxesAxesAxesAxes

The wearable sensor is placed on the wrist of the user. Due to this placement the orientation of the axes are depending on the user’s orientation, illustrated in Figure 28. This entails that the signal behavior on each axis varies depending on the orientation of the sensor. To ensure that no important signal behavior is neglected all three axis signals has to be included for the analysis.


42

Figure 28 - Wearable sensor axes


43

Knock interaction Knock interaction Knock interaction Knock interaction

Recording Wearable 2 recorded ten knocks where the user positions the hand approximately ten centimeters from the door, knocks on the door and then returned to the initial position. Figure 29 illustrates all three axis from the recording.

Figure 29 - Knock interaction

The ten distinguishable behaviors seen in the figure are due to the user’s described movement. To better explain the behavior and to identify the interaction a closer view of one set of the movement is illustrated in Figure 30.


44

Figure 30 - Closer view of user movement

As seen in the figure the user movement surrounding the interaction affects the signal making the interaction more difficult to pinpoint. Focusing on the z-axis three behaviors can be observed. An amplitude decrease can be seen in the beginning of the signal, this corresponds to the user accelerating the hand towards the door. This is followed by a positive z-acceleration due to the user retracting the hand from the door. The moment of impact between hand and door is identified by the spike in the signal at 15.6 seconds.

This behavior is consistent for all the interactions in the recording. The general user movement, leading up to and from the interaction, results in slower changes in the signal where the interaction results in a faster and smaller change of amplitude in the form of a spike. Depending on the orientation of the sensor the significance of these spikes on each axes change. In worst cases the orientation of the sensor results in an amplitude change distributed over all three axes, making it less significant from the rest of the signal and more difficult to identify.

Tap interactionTap interactionTap interactionTap interaction

The same behavior seen for the knocks can be seen for the taps. The interactions can be identified as spikes in the signal faster than the amplitude changes inflicted by the user’s general movement surrounding the interaction. Differences in directions of the accelerations


45

and what appears as offset values for each axis is explained by the change of orientation of the sensor.

Touch interactionTouch interactionTouch interactionTouch interaction

For the touch interaction a slight change of amplitude is noticeable for some interactions but not significant enough to be used in this thesis. This was consistent for all recording using the touch interaction.


46

5.35.35.35.3 Conclusion Conclusion Conclusion Conclusion

The analysis shows major differences between the signals recorded by the sensor in the door and the wearable. The differences is due to the sensor placement where the door sensor is placed on a rigid door making the signal less affected by disturbances and therefore includes less noise. The wearable sensor, placed on the wrist of the user, is highly influenced by the user movement making the signal more inconsistent. In spite of these differences the interaction in both signals can be identified by a significantly fast amplitude change.

DoorDoorDoorDoor

For the door signal the ability to identify the interaction depends on two factors, the form of interaction and the distance between sensor and interaction. Figure 31 illustrates the average maximum amplitude change for each form of interaction depending on the distance between sensor and point of interaction.

Figure 31 – Average amplitude of the interactions observed in the door signals.

As illustrated in the figure all knocks, regardless of the distance, result in a significant maximum amplitude change and can be identified. Only the taps performed very close to the sensor can be identified and none of the touches can be identified. This thesis will only focus on the cases where the interactions result in a significant maximum amplitude change, see Table 11.

Table 11 - Door data sets focused on in this thesis. Discarded sets are marked in gray.


47

Type/Distance 0 cm 3 cm 10 cm 20 cm 30 cm 60 cm

Knock Door 1 Door 2 Door 3 Door 4 Door 5 Door 6

Tap Door 7 Door 8 Door 9 Door 10 Door 11 Door 12

Touch Door 13 Door 14 Door 15 Door 16 Door 17 Door 18

Inside outInside outInside outInside out

In the project it was discussed to determine if the interaction is performed on the inside or outside of the door. Given the used sampling frequency of 400 Hz this was not possible since the first peak could be lost, explained earlier. With the use of a higher sampling frequency it could be possible to determine if the interaction is performed on the inside or outside of the door by analyzing the direction of the change.

WearableWearableWearableWearable

For the wearable the ability to identify the interaction is determined by the form of interaction where both knocks and taps but none of the touches could be identified regardless of the user movement surrounding the interaction. The wearable cases focused on in this thesis will be all cases including knocks and taps, see Table 12.

Table 12 - Wearable data sets focused on in this thesis. Discarded sets are marked in gray.

Type/Movement Stay at door Return From waist Walk up

Knock Wearable 1 Wearable 2 Wearable 3 Wearable 4

Tap Wearable 5 Wearable 6 Wearable 7 Wearable 8

Touch Wearable 9 Wearable 10 Wearable 11 Wearable 12

48

Event Event Event Event detection adetection adetection adetection algorithm lgorithm lgorithm lgorithm

The system contains two separate event detection algorithms, one for each sensor. The algorithms are divided into two parts; a signal processing part followed by an interaction detection part. Chapter 6.1 will analyze two forms of signal processing and chapter 6.2 will analyze three forms of interaction detection methods. To determine the optimal combination of signal processing and interaction detection method simulations and optimizations of each combination will be performed in chapter 6.3.

6.16.16.16.1 Signal Signal Signal Signal pppprocessing rocessing rocessing rocessing

As seen in Chapter 5 the signal of the two sensors have large differences regarding noise and signal consistency. The door sensor have little to no noise where the wearable sensor is subjected to a lot of noise and irregular changes due to the user’s movement. Common for both the sensors is that the interactions generate fast amplitude changes in the signal. In the wearable signal these changes can vary in significance on each of the axes depending on the orientation of the sensor. For the door signal the changes is most significant on the axis parallel to the interaction, in this case the z-axis. This entails that all three axis signals from the wearable sensor and only the z-axis signal from the door sensor is to be included in the signal processing.

The purpose of the signal processing is to facilitate the detection of interactions for the interaction detection part of the algorithm. This is done by reducing the noise and irrelevant data in the form of slow changes from the signal enhancing the appearance of the fast changes due to the interactions. It is also desired to attain a single signal expressing the magnitude of the amplitude change, regardless off change direction.

6.1.16.1.16.1.16.1.1 High pass High pass High pass High pass ffffilterilterilterilter

The filter used in this thesis is a third order Butterworth high pass filter. Since this is to be implemented in a low power system a higher order filter is not investigated. Using a high pass filter the unwanted components in the form of slow changes can be filtered out from the signal. What frequencies to filter out is determined by the cutoff frequency.

CHAPTER 6. EVENT DETECTION ALGORITHM

49

Door Door Door Door SignalsSignalsSignalsSignals

A spectral analysis of one interaction recorded in the door shows that the frequency of the signal change caused by the interaction is above approximately 125 Hz, see Figure 32. This entails that a suitable cutoff frequency is below 125 Hz.

Figure 32 - Signal and normalized frequency scale

The door signal has little to no noise. This entails that a relatively low cutoff frequency can be set without having any problem with disturbances and noise. The optimal cutoff frequency is attained in optimization and simulation part in Chapter 6.

To achieve a signal expressing the magnitude of the amplitude change regardless of the direction, i.e. positive or negative the filtered signal is rectified using

where y is the filtered signal.

© = ª«� Eq. 11

Wearable Wearable Wearable Wearable SignalsSignalsSignalsSignals

The wearable sensor is subjected to a lot of noise and irregular changes due to the user’s movement where these unwanted components made up the majority of the signal. To filter out these a spectral analysis is done of the signal where the unwanted components are included. Each axis is analyzed individually, see Figure 33.


50

Figure 33 - Signals and normalized frequency scale for each axis

The spectral analysis shows that the majority of the frequency content, i.e. the unwanted components, is located below approximately 30 Hz. A suitable cutoff frequency is therefore 30 Hz or above. The optimal cutoff frequency is attained in optimization and simulation part in Chapter 6.

Each axis signal is filtered separately and to attain a single signal that expresses the magnitude of the amplitude regardless of direction for the interaction detection part to analyze the wearable signals is combined. This is done by using the resultant of the filtered signals where the resultant

where ¬, « and is the filtered axis signals.

® = ª¬� + «� + �

Eq. 12


51

6.1.26.1.26.1.26.1.2 Derivative Derivative Derivative Derivative

The second signal processing method is derivation. The derivative of the signal expresses the rate of the change in the signal. Since the interactions appears as fast amplitude changes this method will enhance the interactions and reduce the noise and the unwanted components in forms of slower changes in the signals. This is done using the well-known derivation formula

©WkX = «WkX − «Wk − 1X

where © is the derived signal of «.

For the door signals the z-axis is derived. The derived signal is rectified using Eq. 11. For the wearable the three axis signals are derived separately and combined using Eq. 12.

6.1.36.1.36.1.36.1.3 Interaction location in the processed signalsInteraction location in the processed signalsInteraction location in the processed signalsInteraction location in the processed signals

In the processed signals the interactions can be observed as a series of peaks. For the door signals the series of peaks consist of one dominant peak surrounded by smaller, less significant, peaks. For the wearable signals the significance of the peaks varies where in some cases one dominant peak can be observed and in other cases multiple peaks are equally significant. This inconsistency is explained by the irregular changes due to the user’s movement effecting the wearable signal.

Using the known signal locations of interactions obtained during the data analysis conducted in chapter 5, the locations of the interactions in the processed signals can be determined. All interactions are located within the series of peaks described, Figure 34 illustrates.

Figure 34 - Characteristic locations

The majority of the interactions in the door signals are located at the top of the most significant peak (marked 1 in the figure). The interactions in the wearable signals can be observed at three characteristic locations; at the slope of the first peak (marked 2 in the


52

figure), at the top of the first peak (marked 3 in the figure) and at the top of the most significant, i.e. highest, peak (marked 4 in the figure).

6.26.26.26.2 InteractionInteractionInteractionInteraction ddddetection etection etection etection

The second part of the event detection algorithm is to detect the interactions from the processed signals and store the time of each interaction as an event. As seen in chapter 6.1.3 the interactions can be observed as a series of peaks where the location of the interaction varies between three characteristic locations in the series; the slope of the first peak, the top of the first peak and the top of the highest peak. Due to this inconsistency of interaction location all three locations within the series of peaks has to be located. This is done by using three different event detection methods where the optimal method to use is decided in the simulation and optimization part in Chapter 6.3.

6.2.16.2.16.2.16.2.1 MeMeMeMethod 1 thod 1 thod 1 thod 1 –––– Detecting tDetecting tDetecting tDetecting the he he he slope of the first peakslope of the first peakslope of the first peakslope of the first peak

The first location to be detected in the series of peaks is the slope of the first peak. The slope can be recognized as the first part of the series of peaks reaching an amplitude considerable higher than the rest of the signal.

The method to detect this is to use a threshold where each sample of the signal is analyzed and the samples with an amplitude exceeding the set threshold is labeled as an event. The problem using this method is that other samples within the series of peaks also will have an amplitude exceeding the threshold and be labeled as an event, Figure 35 illustrates.

Figure 35 - Signal and thresholds and events


53

As seen in the figure the amplitude of five samples exceeds the threshold where the algorithm is only detect the first one. To only detect the first sample a wait time is implemented. The implementation entails that after each detection the algorithm waits a set period of time before detecting new events, Figure 36 illustrates. By implementing a wait time of approximately ten milliseconds in the case illustrated only the first event is labeled as an event while the remaining four is discarded due to the wait time.

Figure 36 - Flowchart of threshold method

The method uses two parameters; the threshold and the wait time. The parameter values effects the outcome of the algorithm where a too low threshold results in detecting lower amplitude changes due to noise and where a too high threshold results in not detecting the interactions at all. By using a too small wait time multiple events are detected from one interaction and by using a too large wait time results in not detecting an interaction occurring close after a previous detected one. The optimal parameter values will be acquired in the optimization and simulation part in Chapter 6.3.

6.2.26.2.26.2.26.2.2 Method 2 Method 2 Method 2 Method 2 –––– Detecting tDetecting tDetecting tDetecting the top of the fhe top of the fhe top of the fhe top of the first irst irst irst ppppeakeakeakeak

The second location to be detected is the top of the first peak. This is done in a similar way as method 1 adding an extra step to verify that the sample is located at the top of a peak.

To verify that a sample is located at the top of a peak the amplitude of the sample is compared to the amplitude of the surrounding two samples. If the amplitude is larger than the amplitude of the surrounding two samples it is considered to be located at a top. A threshold is used to verify that the top is located within the series of peaks and a wait time is implemented to ensure that only the first top is detected, Figure 37 illustrates the algorithm.


54

Figure 37 - Illustration of first peak method

The method uses the same two parameters as method 1; threshold and wait time. The optimal parameter values will be acquired in the optimization and simulation part in Chapter 6.3.

6.2.36.2.36.2.36.2.3 Method 3 Method 3 Method 3 Method 3 –––– Detecting tDetecting tDetecting tDetecting the top of the highest phe top of the highest phe top of the highest phe top of the highest peakeakeakeak

The third and final location to be detected is the top of the highest peak. This can be described as the sample with the highest amplitude in the series of peaks. The sample with the highest amplitude is located using a moving window method W9X.

The method uses a fixed sized window stepping through the signal. For each step the sample with the maximum amplitude is detected. If the amplitude exceeds a set threshold the position and amplitude of the sample is stored. If the window moves past the position of the stored sample it is labeled and stored as an event, Figure 38 illustrates the method.

Figure 38 - Illustration of moving window method


55

6.2.46.2.46.2.46.2.4 ConclusionConclusionConclusionConclusion

Three event detection methods are described, each to detect one of the characteristic interaction locations in the processed signal described in Chapter 6.1.3, see Figure 39.

Figure 39 - Each characteristic location detected with all three methods

Each method can be altered by changing certain parameters. All three methods uses a threshold parameter to locate the part of the signal including the interaction. A second parameter is used to ensure that the correct location is detected. The second parameter of method 1 and method 2 (slope and first peak) is the size of the wait time and for method 3 (highest peak) it is the size of the moving window. The optimal method and parameter values for each method will be attained in the following chapter.

6.36.36.36.3 OOOOptimization and simulation ptimization and simulation ptimization and simulation ptimization and simulation

To determine the optimal combination between signal processing technique, event detection method and the optimal parameter settings for these an evaluation script is used. The script evaluates all the combinations to find events in a given data set for each sensor. The events detected by each combination are compared to the correct event locations obtained from the data analysis conducted in Chapter 5. The results for each combination of signal processing technique, event detection method and parameter settings is stored and evaluated. Figure 40 illustrates the simulation method.


56

Figure 40 - Simulation Method

Data setData setData setData set

The data sets used is a selection from the data sets acquired previously mentioned in chapter 4. The data sets were reduced to only include the sets motivated in chapter 5.3, see Table 13.


57

Table 13 - Data sets used for each sensor

Sensor Data set

Wearable 1-8

Door 1-7

Parameter SettingsParameter SettingsParameter SettingsParameter Settings

Each component of the algorithm can be optimized by changing certain parameters mentioned in Chapter 6.1 and Chapter 6.2. The component parameters and the different values used in the script can be seen in Table 14 and Table 15.

Table 14 - Component parameters. ID stands for interaction detection and SP stands for signal processing.

Component Parameter(s)

ID Slope Threshold, Wait Time

ID First Peak Threshold, Wait Time

ID Highest Peak Threshold, Window Size

SP High Pass Filter Cutoff frequency

SP Derivative -


58

Table 15 - Component parameter values used.

Parameter Parameter Values

Threshold 0.01 – 2

Wait Time 1.25-250 milliseconds

Window Size 5-100 milliseconds

Cutoff frequency 5-180 Hz

ResultsResultsResultsResults

Three measurements are used to evaluate the results, see Table 16. The results for each combination and parameter setting were saved in a result matrix.

Table 16 - Measurements used to evaluate each combination.

Measurements Description

True Detect The percentage of correctly detected events, i.e. &'+./ and &'+..

.

False Detect The number of wrongfully events detected.

Error Range The event detection error, i.e. 3+./ and 3+..

.

6.3.16.3.16.3.16.3.1 Simulation RunSimulation RunSimulation RunSimulation Run

Using the values described simulations for each sensor were performed individually. A total of approximately 3.6 million simulations of unique combinations were performed. Figure 41 illustrates the simulation run.


59

6.46.46.46.4 ResultsResultsResultsResults

To determine which combination yields the best results the optimal parameter settings for each combination was determined. This was done using prioritized filters where each filter corresponds to one of the result measurements, see Table 17.

Table 17 - Result measurements and their corresponding priority.

Measurement Priority

High True Detect 1

Low False Detect 2

Low Error Range 3

Figure 41 - Simulation Run


60

The filters are applied to the result matrix in prioritized order. Each filter locates the optimal value for its corresponding measurement and filters out combinations resulting in lower values, illustrated in Figure 42.

.

Figure 42 - Optimization Algorithm


61

Wearable Wearable Wearable Wearable

The results for the wearable sensor is described in Table 18. All combinations achieves a 100 percent true detect where the optimal combination results in zero false detect and the lowest error range of 25 milliseconds is the derivative signaling process and the first peak event detection method.

Table 18 - Wearable results.

Signal Processing Fc

Interaction detection method

Threshold

Algorithm parameter WmillisecondsX

True Detect W%X

False Detect WnX

Error range WmillisecondsX

Filter 20 Slope 0.1 12.5 100 15 12.5

Filter 60 First Peak 0.3 125 100 0 30

Filter 40 Highest Peak

0.2 87.5 100 0 30

Derivative - Slope 0.35 12.5 100 49 12.5

Derivative - First Peak 0.35 82.5 100 0 25

Derivative - Highest Peak

0.3 75 100 0 55

DoorDoorDoorDoor

The results for the door sensor, illustrated in

Table 19, shows that all combinations achieves a 100 percent true detect. Two combinations both results in a zero false detect and a 5 millisecond error range where the chosen combination is the derivative signaling process and the first peak event detection method, the same as for the wearable.


62

Table 19 - Door results.

Signal Processing

Fc WHzX

Interaction detection method

Threshold Algorithm parameter WmillisecondsX

True Detect W%X

False Detect WnX

Error Range WmillisecondsX

Filter 20 Slope 0.25 12.5 100 25 10

Filter 20 First Peak 0.15 60 100 0 5

Filter 20 Highest Peak

0.1 57.5 100 0 10

Derivative - Slope 0.3 12.5 100 17 12.5

Derivative - First Peak 0.2 62.5 100 0 5

Derivative - Highest Peak

0.1 72.5 100 15 10


63

64

System EvaluationSystem EvaluationSystem EvaluationSystem Evaluation

This chapter will evaluate the system based on the system convenience and the system security using the theoretical model in chapter 3. The convenience of the system is measured by system true detect and the security is measured by two measurements; the probability of an accidental unlock, <,0*=> and the time needed for an attack, A0**0=B .

The model in Chapter 3 shows that the values of these measurements depends on three things; the system parameters, the event detection rate in the wearable and door and the size of the matching window.

System ParametersSystem ParametersSystem ParametersSystem Parameters

The evaluation is done using the event detection algorithms motivated in Chapter 6.4 and the parameters attained throughout the report, see Table 20.

Table 20 - Time Error Factors of the System

Parameter description Parameter Value

Error introduced by the event detection algorithm in the door.

3+.. 25 ms

Error introduced by the event detection algorithm in the wearable.

3+./ 5 ms

Error due to the sampling frequency of the wearable sensor. 389t 5 ms

Error due to the sampling frequency of the door sensor. 389r 5 ms

Error due to the time synchronization of the two sensors. 3*( 10 ms

True detect of event detection in the door. &'+.. 100 %

True detect of event detection in the door. &'+./ 100 %

CHAPTER 7. SYSTEM EVALUATION

65

Event detection rateEvent detection rateEvent detection rateEvent detection rate

As demonstrated in Chapter 3, the security of the system depends on the event detection rate in both the wearable and door. The event detection rate in the door, '@ , inflicted by the second, unauthorized, user or attacker is set to three, i.e. the second user introduces three event per second to the door sensor.

The event detection rate in the wearable, ?@, depends on the activity performed by the authorized user wearing the wearable. Three forms of user activities are considered; normal use, high use and extreme use, each resulting in a different value of ?@, see Table 21.

Table 21 - User Scenarios Wearable

User scenario Wearable

Description Event detect rate wearable, ?@

WEvent/sX

Normal Use Everyday normal use, no specific activity.

0.003

High Use User scenario where wrist movement is high, for example typing, cooking etc.

0.2

Extreme Use User scenario where wrist movement is extremely high, for example drumming.

3

Using the parameters described above the three measurements; system true detect, probability of accidental unlock and the time needed for an attack, is attained for any given size of the matching window, Figure 43 illustrates the simulation model.

Figure 43 - Evaluation model


66

Given the size of the matching window, &1, the probability of an accidental unlock, <,0*=>, is given by Eq. 9 and he time needed for an attack, A0**0=B , is given by Eq. 10. The true detect of the matching algorithm, &',0 , is given by Eq. 8 where a breadth first search W10X is used to find the optimal reduction, i.e. the values of x5 and x7. The system true detect is given by Eq. 1. The results are illustrated using ROC (Receiver Operating Characteristic) curves where each user scenario result in a different curve.

Normal useNormal useNormal useNormal use

Figure 44 illustrates the resulting ROC curve for the use case “Normal use”. The figure shows that an increasing system true detect results in a decreasing system security. The two curves showing the security parameters; risk of accidental unlock and time needed for an attack; both appear linear between the system true detect values 40-95 percent. After a 95 percent system true detect rate the curves changes exponentially. This is explained by the error distribution. To achieve a greater than 95 percent system true detect rate the size matching window have to increase to include all the errors. Since a minority of the errors attained a greater absolute value from zero, i.e. a greater error value, the size of the matching window has to be increased resulting in a drastic negative change of the system security.

Figure 44 - ROC Curve Normal Use

The value ranges of the security measurements can be seen in Table 22. Using a one hundred percent system true detect, a probability of 0.00016 to accidentally unlock the system and a needed time for attack of 0.59 hours is achieved.


67

Table 22 - Security measurements for normal use.

Parameter Accidental Unlock Time needed for an attack

Value Range 0.00006 - 0.00016 5556 - 2116 seconds (1.54 – 0.59 hours)

HighHighHighHigh useuseuseuse

The same curve characteristics seen in the normal use case can be seen for the high use case. Figure 45 shows the result for the user case high use.

Figure 45 - ROC curve high use

The value ranges of the security measurements can be seen in Table 23. Comparing the values to the normal use case it is clear that the increase of event detection rate in the wearable results in a significant decrease of system security. Using a system true detect of one hundred percent the probability to accidentally unlock the system increased to 0.01 and the needed time for attack decreased to 32 seconds.

Table 23 - Security measurements for high use


Value Range 0.004-0.01 83 - 32 seconds


68


69

Extreme useExtreme useExtreme useExtreme use

The ROC curve for the extreme use cases, illustrated in Figure 46, demonstrates the security decreasing further using a further increased value of ?@ .

Figure 46 - ROC curve extreme use

The value ranges of the security measurements can be seen in Table 24. Using a system true detect of one hundred percent the probability to unlock increased to 0.16 and the time needed for an attack decreased to only 2 seconds.

Table 24 - Security measurements for extreme use.


Value Range 0.06 – 0.16 6 - 2 seconds

The curves for each user case clearly shows that the event detection rate in the wearable is crucial for the system security. A relatively low detection rate, normal use, has a very high security regardless of the system true detect. With a higher detection rate, high and extreme use, the security of the system decreases to a level where an attacker can trigger an unlock operation in under a minute or just a few seconds in case of the extreme use case, making the system unsecure.

7.17.17.17.1 Attack mitigation Attack mitigation Attack mitigation Attack mitigation

To increase the system security attack mitigation is implemented. As the previous chapter showed, the event detection rate in both the sensors had a major impact on the security. By


70

limiting the maximum event detection rate in either one or both sensors the system security can be improved. This is done using a hysteresis.

A hysteresis defines the maximum rate of which evets can be detected. This means that after a detected event a certain amount of time, defined by the hysteresis, have to pass before the detection of a new event is possible.

The use of the hysteresis affects the user convenience. For example; the user interacts with the door and for some reason the door does not unlock. The user then have to wait the set period of time, defined by the hysteresis, before a new interaction can be detected and unlock the door. As a result of this the size of the hysteresis is set to 2 seconds to not affect the user convenience in a noticeable negative way.

By using the hysteresis of two seconds the maximum event detection rate is limited to 0.5 events per second. Figure 47 shows the ROC curve for extreme use case if using a two second hysteresis in both the wearable and the door.

Figure 47 - ROC curve using 2 second hysteresis

As the figure shows the implementation of a two second hysteresis results in the time needed for an attack to be above one minute and the probability to accidentally unlock the system to be below 0.027.

To further increase the security an additional dynamic hysteresis is implemented in the wearable where the size of the hysteresis increases if the previous event is not successfully matched. The increase of the hysteresis is done using two methods; either a linear incensement where the hysteresis for the k:th unmatched event


71

ℎD = kE

where E is the size of each addition to the hysteresis in seconds, or by an exponential incensement where the hysteresis for the k:th unmatched event

ℎD = °D

where ° is the factor of the addition to the hysteresis in seconds. By using a dynamic hysteresis in the wearable the wearable event detection rate decreases with the increase of unmatched events detected, defined as

?@± =1

ℎD

where k is the number of unmatched events. Figure 48 illustrates the values of ℎD and ?@± for both the linear and the exponential increasing hysteresis given the number of unmatched events with the values of E and ° set to 2 seconds.


72

Figure 48 - Hysteresis

As seen in the figure the event detection rate in the wearable drastically decreases, ergo the probability of an accidental unlock decreases, as the number of unmatched events detected increases, illustrated in Figure 49. The values in the figure is for a true detect rate of one hundred percent and using the initial hysteresis of two seconds.

Figure 49 – Probability of accidental unlock decreasing using a dynamic hysteresis


73

Using these values in Eq. 10 gives time needed to successfully attack the system for the same scenario of approximately 25 minutes using the linear increasing hysteresis and approximately 17433 years using the exponential increasing hysteresis.

As the size of the hysteresis increases so does the noticeably negative effects on the user convenience. This can be solved by implementing a notification function notifying the user when the size of the hysteresis reaches a certain value giving the user the ability to reset the hysteresis via the wearable. The hysteresis can also be reset automatically if no events are detected within a set period of time, preventing the hysteresis to slowly increase over a long time.

74

ResultsResultsResultsResults

The aim of this thesis was to determine if the concept developed can result in a secure yet user convenient system. This chapter will present the results gained throughout the thesis and answer the research question “Is it possible to develop a user convenient yet secure system?”

8.18.18.18.1 User convenience User convenience User convenience User convenience

The user convenience of the system correlates to the effort required by the user to unlock the door. The first part of this is the effort to perform the interaction required to unlock the door. In the developed concept knocks and finger taps are used as interaction methods. Both methods are considered close to effortless and a natural way of interaction. The second part correlated to the responsiveness of the system and is determined by the systems true detect rate.

As seen in this thesis a one hundred percent system true detect rate can be attained meaning that every valid interaction unlocks the door. It is also shown that by decreasing the true detect rate a higher system security could be attained. The lower limit of the true detect rate is determined to be 90 percent, meaning that nine out of ten valid interactions will unlock the door. An even lower true detect rate would result in a negative user experience where the result would not be considered satisfying.

8.28.28.28.2 System securitySystem securitySystem securitySystem security

The system security is evaluated using two measurements; accidental unlock probability and time needed for a successful attack. The security of the system is dependent on the system true detect where the minimum value allowed is defined as 90 percent. It is also dependent on the event detection rate in the wearable defined by the three use cases; normal use, high use and extreme use. This thesis considers two case scenarios when evaluating the system security; an average case scenario which corresponds to the expected everyday use of the system and a worst case scenario which corresponds to rare occasions where the system security is at its lowest.

CHAPTER 8. RESULTS

75

8.2.18.2.18.2.18.2.1 AvrageAvrageAvrageAvrage ccccaseaseasease scenario scenario scenario scenario

The average case scenario is evaluated using the wearable event detection rate defined by the normal use case. Using a one hundred percent true detect rate the security measurements are given by Figure 44 and are as follow

- Probability of accidental unlock: 0.00016 (approximately 1 in 6250).

- Time needed for an attack: 0.59 hours.

Both measurements are considered satisfying meaning that the system is considered secure for the average case scenario.

8.2.28.2.28.2.28.2.2 Worst caseWorst caseWorst caseWorst case scenarioscenarioscenarioscenario

To evaluate the worst case scenario the wearable event detection rate defined by the extreme use case is used. Using a one hundred percent system true detect rate the security measurements are given by Figure 46 and are as follow


- Time needed for an attack: 2 seconds.

None of the measurements are considered satisfying but by implementing the attack mitigation methods described in chapter 7.1 the security can be improved. With the initial hysteresis of two seconds implemented in both sensors the following measurements are given by Figure 47 and are as follow


- Time needed for an attack: 1 minute.

By using the dynamic hysteresis in the wearable security improves further where the probability to accidentally unlock the door is given by Figure 49. The figure illustrates that he probability of an accidental unlock decreases as the number of unmatched events detected in the wearable increases. The time needed for a successful attack is attained by using Eq. 10 and shows that the time increases to at least 25 minutes using the dynamic hysteresis. This gives the following measurements

- Probability of accidental unlock: < 0.027 (approximately 1 in 37).

- Time needed for an attack: > 25 minutes.

Considering that these values drastically improves further as the number of unmatched events increases the values are considered secure.

CHAPTER 8. RESULTS

76

8.2.38.2.38.2.38.2.3 Environmental aspectsEnvironmental aspectsEnvironmental aspectsEnvironmental aspects

The security measurements is been based on that the requirements from the first authentication step is fulfilled. These includes that the wearable has to have an established connection to the system. With BLE and distance measurement based on RSSI (Received Signal Strength Indication) the accuracy of approximately 10 meters can be achieved. This entails that for the system to unlock the authorized user has to be within 10 meters of the door. Taking this into consideration it is highly likely that the authorized user would notice any form of attack before the attacker successfully unlocks the door, adding an extra aspect of security.

8.38.38.38.3 ConclusionConclusionConclusionConclusion

This thesis shows that, based on the theoretical model, satisfying levels of both user convenience and system security can be attained for both the average and worst case scenarios using the methods described on the specified data recorded. To verify the validity of the theoretical model further work and testing is required using a proof of concept implementation of the lock and wearable running the algorithms in real time and exchanging data over BLE.

Other environments and/or implementation limitations effecting the sensors properties and/or the signal behavior could generate different results. Although this has to be further investigated this thesis shows that the concept has high potential and the framework of methods used to evaluate the data sets in this thesis can in the same way be used to evaluate systems with different properties and in a different environment.

77

ReferencesReferencesReferencesReferences

W1X Assa Abloy. Retrieved 2016, from http://www.assaabloy.com/en/com/about-us/

W2X J. C. Eldson “Measurement Control and Communication Using IEEE 1588”, 2006

W3X GSM Arena. Retrieved 2016, from http://www.gsmarena.com/samsung_gear_s2_3g-7585.php

W4X GSM Arena. Retrieved 2016, from http://www.gsmarena.com/huawei_watch-7687.php

W5X Apple. Retrieved 2016, from https://support.apple.com/kb/SP735?locale=sv_SE

W6X L. R. Rabiner and B. H. Juang, "An Introduction to Hidden Markov Models", IEEE ASSP Magazine, January 1986.

W7X P. Olofsson and M. Andersson, “Probability, Statistics, and Stochastic Processes second Edition”, 2012

W8X Mathworks, Matlab R2015b, 2015

W9X S. Lu, D. Qu and Y. He, “Sliding Window Tone Reservation Technique for the Peak-to-Average Power Ratio Reduction of FBMC-OQAM Signals” in IEEE Wireless Communications Letter, Vol. 1, No. 4, August 2012.

W10X S. S. Ray, “Graph Theory with Algorithms and its Applications”, 2013

Method for gesture based authentication in physical access ...1111553/FULLTEXT01.pdfMaster of...

Documents

Transcript of Method for gesture based authentication in physical access ...1111553/FULLTEXT01.pdfMaster of...