Springlet Apps: Applications of On-Skin Shape Memory Alloy · 2020. 3. 1. · by Ajay Kumar Jha...

byAjay Kumar Jha

Springlet Apps:Applications ofOn-Skin ShapeMemory AlloyActuators

Thesissubmitted to theMedia Computing GroupProf. Dr. Jan BorchersComputer Science DepartmentRWTH Aachen University

Thesis advisor:Prof. Dr. Jan Borchers

Second examiner:Prof. Dr. Ulrik Schroeder

Registration date: 08.10.2019Submission date: 26.02.2020

Eidesstattliche Versicherung

___________________________ ___________________________

Name, Vorname Matrikelnummer

Ich versichere hiermit an Eides Statt, dass ich die vorliegende Arbeit/Bachelorarbeit/Masterarbeit* mit dem Titel

__________________________________________________________________________

__________________________________________________________________________

__________________________________________________________________________

selbständig und ohne unzulässige fremde Hilfe erbracht habe. Ich habe keine anderen als die angegebenen Quellen und Hilfsmittel benutzt. Für den Fall, dass die Arbeit zusätzlich aufeinem Datenträger eingereicht wird, erkläre ich, dass die schriftliche und die elektronische Form vollständig übereinstimmen. Die Arbeit hat in gleicher oder ähnlicher Form noch keiner Prüfungsbehörde vorgelegen.

___________________________ ___________________________

Ort, Datum Unterschrift

*Nichtzutreffendes bitte streichen

Belehrung:

§ 156 StGB: Falsche Versicherung an Eides Statt

Wer vor einer zur Abnahme einer Versicherung an Eides Statt zuständigen Behörde eine solche Versicherung falsch abgibt oder unter Berufung auf eine solche Versicherung falsch aussagt, wird mit Freiheitsstrafe bis zu drei Jahren oder mit Geldstrafe bestraft.

§ 161 StGB: Fahrlässiger Falscheid; fahrlässige falsche Versicherung an Eides Statt

(1) Wenn eine der in den §§ 154 bis 156 bezeichneten Handlungen aus Fahrlässigkeit begangen worden ist, so tritt Freiheitsstrafe bis zu einem Jahr oder Geldstrafe ein.

(2) Straflosigkeit tritt ein, wenn der Täter die falsche Angabe rechtzeitig berichtigt. Die Vorschriften des § 158 Abs. 2 und 3 gelten entsprechend.

Die vorstehende Belehrung habe ich zur Kenntnis genommen:

___________________________ ___________________________

Ort, Datum Unterschrift

Jha, Ajay Kumar 384269

Springlet Apps: Applications of On-Skin Shape Memory AlloyActuators

26.02.2020

26.02.2020

v

Contents

Abstract xvii

Uberblick xix

Acknowledgements xxi

Conventions xxiii

1 Introduction 1

1.1 The Sense of Touch . . . . . . . . . . . . . . . 1

1.2 On-Skin Actuators . . . . . . . . . . . . . . . 1

1.3 Applications of On-Skin Shape Memory Al-loy Actuators . . . . . . . . . . . . . . . . . . 2

1.3.1 Motivation . . . . . . . . . . . . . . . . 2

1.3.2 Applications . . . . . . . . . . . . . . . 2

Stroke Rehabilitation . . . . . . . . . . 2

Haptic Feedback in Virtual RealityGames . . . . . . . . . . . . . 3

Haptic Navigation . . . . . . . . . . . 3

vi Contents

Weight Training Guide . . . . . . . . . 3

1.4 Outline of the Thesis . . . . . . . . . . . . . . 3

2 Related Work 5

2.1 Stroke Rehabilitation . . . . . . . . . . . . . . 5

2.2 Haptic Feedback in Virtual Reality Games . . 7

2.3 Haptic Navigation . . . . . . . . . . . . . . . 9

2.4 Weight Training Guide . . . . . . . . . . . . . 11

2.4.1 JARVIS : A Virtual Reality ExerciseCoach . . . . . . . . . . . . . . . . . . 11

2.4.2 Haptic Guidance as an Exercise Guide 12

2.4.3 A Comparison of Various GuidanceTechniques . . . . . . . . . . . . . . . . 15

2.5 Springlets: A Light-Weight Tactile Actuator . 16

2.6 Contributions of this Thesis . . . . . . . . . . 16

3 Evaluation of application areas 19

3.1 Evaluation of Springlets for Stroke Rehabili-tation . . . . . . . . . . . . . . . . . . . . . . . 19

3.1.1 Conclusion of the Evaluation . . . . . 22

3.2 Evaluation of Springlets for Virtual RealityGames . . . . . . . . . . . . . . . . . . . . . . 22


3.3 Evaluation of Springlets for Haptic Navigation 24


Contents vii

3.4 Evaluation of Springlets for Weight Training 25


4 Implementation of Springlets as an Exercise Guide 27

4.1 Design of the Exercise Guide . . . . . . . . . 27

4.1.1 Shape Memory Alloy . . . . . . . . . . 27

4.1.2 Construction of the Pincher Springlet 28

4.1.3 Pincher Designs . . . . . . . . . . . . . 29

Large Pincher . . . . . . . . . . . . . . 29

Wide-Ended Large Pincher . . . . . . 29

Thin-Base With Medium-Sized Pincher 30

Short and Wide Base With Medium-sized Pincher . . . . . . . . . 30

4.1.4 Requirement of Multiple Springlets . 31

4.2 Investigated Placement Positions . . . . . . . 32

4.2.1 Elbow . . . . . . . . . . . . . . . . . . 32

4.2.2 Outer Side of the Elbow . . . . . . . . 33

4.2.3 Upper Forearm . . . . . . . . . . . . . 34

4.2.4 Upper Arm . . . . . . . . . . . . . . . 35

4.2.5 Selected Positions . . . . . . . . . . . . 36

4.3 Operation of the Pincher as an Exercise Guide 36

4.3.1 Electronic Circuit . . . . . . . . . . . . 36

4.3.2 Exercise Guide in Action . . . . . . . . 37

viii Contents

4.3.3 Logging Arm Movements . . . . . . . 38

5 User Study 39

5.1 Research Questions . . . . . . . . . . . . . . . 39

5.2 Hypotheses . . . . . . . . . . . . . . . . . . . 40

5.3 Pilot Study . . . . . . . . . . . . . . . . . . . . 40

5.4 Study Design . . . . . . . . . . . . . . . . . . 40

5.4.1 Independent Variables . . . . . . . . . 41

5.4.2 Dependent Variables . . . . . . . . . . 42

5.4.3 Study Type . . . . . . . . . . . . . . . 42

5.4.4 Latin Square Randomization . . . . . 42

5.4.5 Setup and Surroundings . . . . . . . . 43

5.4.6 Experiment Procedure . . . . . . . . . 43

5.5 Experiment Recordings . . . . . . . . . . . . . 45

5.5.1 Videos . . . . . . . . . . . . . . . . . . 45

5.5.2 Reaction times . . . . . . . . . . . . . . 45

5.5.3 Likert Scale Ratings and Rankings . . 45

5.6 Participants . . . . . . . . . . . . . . . . . . . 46

5.7 Results . . . . . . . . . . . . . . . . . . . . . . 46

5.7.1 Statistical Tests . . . . . . . . . . . . . 46

Repeated Measures ANOVA . . . . . 46

Friedman Test . . . . . . . . . . . . . . 48

Contents ix

Friedman-Nemenyi Test . . . . . . . . 48

5.7.2 Rankings . . . . . . . . . . . . . . . . . 50

5.7.3 Observations . . . . . . . . . . . . . . 50

5.7.4 Discussion . . . . . . . . . . . . . . . . 51

5.8 Conclusion . . . . . . . . . . . . . . . . . . . . 53

6 Summary and Future work 55

6.1 Summary and Contributions . . . . . . . . . 55

6.2 Future Work . . . . . . . . . . . . . . . . . . . 56

6.2.1 More Exercise Movements . . . . . . . 56

6.2.2 Strong and Quick Actuation . . . . . . 57

6.2.3 More Applications . . . . . . . . . . . 57

A Breadboard Layout 59

B Study Consent and Questionnaire 61

B.1 Informed Study Consent . . . . . . . . . . . . 61

B.2 General Questionnaire . . . . . . . . . . . . . 61

B.3 Likert Scale Questionnaire . . . . . . . . . . . 61

C p-values and Graphs 67

C.1 p-values from the Friedman Test . . . . . . . 67

C.2 Graphs for Likert Scale Questions with NoSignificant Differences . . . . . . . . . . . . . 68

x Contents

Bibliography 69

Index 73

xi

List of Figures

2.1 Exoskeleton robotic hand training device de-scribed by Ho et al. [2011] . . . . . . . . . . . 6

2.2 Skin stretch on the fingertip as a directionalguide [Norman et al., 2014] . . . . . . . . . . 6

2.3 Skin-stretch direction guidance device (Left).The eight supported tactile cueing directions(Right) [Norman et al., 2014] . . . . . . . . . . 7

2.4 Skin-stretch device to convey weight, fric-tion and stiffness information while manip-ulating virtual objects [Schorr and Okamura,2017] . . . . . . . . . . . . . . . . . . . . . . . 9

2.5 Skin stretch to deliver superficial hand ex-tension in Virtual Reality games . . . . . . . . 9

2.6 Waist-belt for Navigation with numbersshowing the positions of vibration units[Steltenpohl and Bouwer, 2013] . . . . . . . . 10

2.7 Wristband fitted with navigational tactorsfor Pedestrian Navigation (Left). Positionof the navigational tactors (Right). [Dobbel-stein et al., 2016] . . . . . . . . . . . . . . . . . 11

2.8 JARVIS : A Virtual Reality Weight TrainingCoach [Rabbi et al., 2018] . . . . . . . . . . . . 12

xii List of Figures

2.9 Setup to achieve skin-stretch near the indexfinger joints . . . . . . . . . . . . . . . . . . . 13

2.10 Comparison of Finger stretch intensitieswith and without skin-stretch . . . . . . . . . 13

2.11 Comparison of Elbow stretch intensities withand without skin-stretch . . . . . . . . . . . . 14

2.12 Comparison of Knee stretch intensities withand without skin-stretch . . . . . . . . . . . . 14

2.13 A : A variant of Springlets that produces apinching sensation on the skin. B: The effectof the Springlet on the skin that turns into apinching sensation. [Hamdan et al., 2019] . . 17

3.1 A : Operation of a pincher springlet. B:Operation of directional-stretcher Springlet.[Hamdan et al., 2019] . . . . . . . . . . . . . . 20

3.2 A: Straightened index finger highlightingthe contracted skin pattern on the middlejoint. B: Bent index finger highlighting thestretched skin pattern on the middle joint . . 20

3.3 Attempted Springlet placement positions forStroke Rehabilitation . . . . . . . . . . . . . . 21

3.4 Pincher placements to create a long-arm illu-sion . . . . . . . . . . . . . . . . . . . . . . . . 23

3.5 Pincher placement on the wrist to create aweb-shoot haptic illusion during SpidermanVR game . . . . . . . . . . . . . . . . . . . . . 23

3.6 Two Springlets placed on the wrist to sup-port turn-by-turn Haptic Navigation . . . . . 24

3.7 Left: Skin patterns when the arm is stretchedRight: Skin patterns when the arm is thebent towards the shoulder . . . . . . . . . . . 26

List of Figures xiii

4.1 Stretched and Shrinked Springlets . . . . . . 28

4.2 Pincher Springlet Construction Steps . . . . . 28

4.3 Investigated Pincher Springlet Designs . . . . 30

4.4 The arm movement pattern in a Biceps Mus-cle Exercise. The Springlet on the top (or-ange) guides upwards arm movement andthe bottom Springlet (green) guides down-ward arm movement . . . . . . . . . . . . . . 32

4.5 Springlets positioned on the elbow andabove the elbow . . . . . . . . . . . . . . . . . 33

4.6 Springlets positioned on the outer sides ofthe elbow . . . . . . . . . . . . . . . . . . . . . 34

4.7 Springlets positioned on the upper forearm . 35

4.8 Springlets positioned on the upper arm . . . 36

4.9 An example configuration of two PincherSpringlets to function as an exercise guide . . 37

5.1 Research Questions . . . . . . . . . . . . . . . 39

5.2 The four Springlet configurations for theuser study . . . . . . . . . . . . . . . . . . . . 41

5.3 The user study setup . . . . . . . . . . . . . . 44

5.4 Graph denoting the reaction times in sec-onds (Y-axis) for the four Springlet config-urations (X-axis). The orange bars denotereaction times for upwards arm movementwhile the green bars denote the reactiontimes for downwards arm movement . . . . 47

5.5 Likert-Scale Ratings for the Questions withSignificant Differences . . . . . . . . . . . . . 49

xiv List of Figures

5.6 Graph denoting the rankings from 1 (best)to 4 (worst) along the X-axis and the distri-bution of a rank among different Springletconfigurations by the total number of partic-ipants along the Y-axis. . . . . . . . . . . . . . 50

A.1 Circuit Diagram . . . . . . . . . . . . . . . . . 59

C.1 Graphs for Likert Scale Questions that didnot have significant differences . . . . . . . . 68

xv

List of Tables

5.1 The order in which each participant goesthrough the Springlets configurations. TheSpringlet configurations are represented bydigits 1-4. . . . . . . . . . . . . . . . . . . . . . 43

C.1 p-values from the Friedman test performedon Likert Scale data (Calculated using Real-Statistics). . . . . . . . . . . . . . . . . . . . . 67

xvii

Abstract

The importance of mechano-tactile actuators lies in their ability to generate naturalsensations on the skin such as dragging, pinching and pulling. However, most ofthe mechano-tactile actuators can be inconvenient to wear as they are heavy andnoisy due to hard moving parts in their design. On the other hand, Springlets arelightweight mechano-tactile actuators that are easy to build and have a sticker-likeform factor. Thus, it becomes important to explore the applications that are possiblewith a lightweight mechano-tactile actuator like Springlets.

In this thesis, we evaluated a variety of possible applications using Springletssuch as weight training, stroke rehabilitation, cutaneous feedback in virtual real-ity games and haptic navigation. Among these applications, we found that thelightweight design of Springlets suited well with the application area of learningweight exercises. Hence, we chose to explore the area of weight training in detail.

We conducted a user study focusing on an exercise to train the biceps muscle. Thestudy aimed to find if Springlets served as an intuitive haptic guidance interfaceto learn weight exercises. We studied different placement positions of Springlets tofind out the best position to place Springlets on the user’s arm while performingthe mentioned exercise.

xviii Abstract

xix

Uberblick

Die Bedeutung von mechanisch-taktiler Aktuatoren liegt in ihrer Fahigkeit,naturliche Empfindungen auf der Haut wie Ziehen, Kneifen und Ziehen zu erzeu-gen. Die meisten mechanisch-taktilen Aktuatoren konnen jedoch unbequem zutragen sein, da sie aufgrund hart beweglicher Teile in ihrer Konstruktion schwerund laut sind. Auf der anderen Seite sind Springlets leichte mechanisch-taktile Ak-tuatoren, die einfach zu bauen und auf der Haut aufklebbar sind. Deshalb ist eswichtig, die Anwendungen zu untersuchen, die mit einem leichten mechanisch-taktilen Aktuator wie Springlets moglich sind.

In dieser Arbeit haben wir eine Vielzahl moglicher Anwendungen mithilfe vonSpringlets bewertet, z. B. Krafttraining, Schlaganfallrehabilitation, haptischesFeedback auf der Haut in Virtual-Reality-Spielen und haptische Navigation. Unterdiesen Anwendungen fanden wir, dass das leichte Design von Springlets gut zumAnwendungsbereich des Lernens von Gewichtsubungen passt. Daher haben wiruns entschlossen, den Bereich des Krafttrainings im Detail zu erkunden.

Wir haben eine Anwenderstudie durchgefuhrt, die sich auf eine Ubung zumTrainieren des Bizepsmuskels konzentriert. Ziel der Studie war es herauszufinden,ob Springlets als intuitive Schnittstelle und als haptischen Anleitung zum Erler-nen von Gewichtsubungen dienen kann. Wir haben verschiedene Platzierungspo-sitionen von Springlets untersucht, um herauszufinden, in welcher Position aufdem Arm des Nutzers Springlets beim platziert am effektivsten platziert werdenkonnen.

xxi

Acknowledgements

First of all, I am grateful to my parents for their enormous support throughoutmy studies. I am extremely thankful to my thesis supervisor Anke Brocker forsupporting me and putting me on the right track in all stages of my thesis.

I further thank Adrian Wagner for supporting me in the initial stages of my thesisand providing me with the required technical foundation. I thank Krishna Subra-manian for advising me on the statistical results of this thesis.

I finally thank all the participants for their time spent in the user study of this thesis.

xxiii

Conventions

Throughout this thesis we use the following conventions.

Emphasized words are represented in italics.

Definitions of technical terms or short excursus are set offin coloured boxes.

EXCURSUS:Excursus are detailed discussions of a particular point ina book, usually in an appendix, or digressions in a writ-ten text.

Definition:Excursus

The whole thesis is written in American English.

The third person is identified in the male form.

1

Chapter 1

Introduction

1.1 The Sense of Touch

The sense of touch is a natural and also sensitive feeling Touch is capable oftransferring physicalinformation

for most people. This sensation enables us to feel thingsaround us by establishing physical contact with them. Italso is capable of transferring information such as texture,the temperature of the material and the pressure that is be-ing exerted from the material.

From the haptic feedback that we receive from the physicalproperties of an object, we can infer a variety of informationsuch as:

1. A signal to initiate an action when the physical state Some examples ofphysical informationthat can be inferred

of the object changes.

2. Physical boundaries of the interaction with the object.

3. Inferring the type of object using past experience.

1.2 On-Skin Actuators

Some types of information like physical boundaries of aninteraction, pressure, texture are natural to communicate to

2 1 Introduction

the user only using the sense of touch as they become verydifficult to convey using non-tactile techniques. As modernModern actuators

are heavy, whileshape-memory

alloys are lightweight

digital devices are often missing natural physical interac-tions, tactile actuators bridge this gap in enabling naturalskin interactions such as dragging, pinching, and pressing.However, most of the modern tactile actuators are often dif-ficult to wear and carry due to the heavy mechanical partsthat they employ. Shape-memory alloys (SMA) on the otherhand are light-weight actuators that can be fabricated intoan on-skin actuator [Hamdan et al., 2019] thereby support-ing a light and noise-free operation.

1.3 Applications of On-Skin Shape Mem-ory Alloy Actuators

1.3.1 Motivation

As SMA-based tactile actuators differ from heavy-weightIt is important tounderstand the

capabilities of theSMA actuator

tactile actuators in their operation, it becomes important tounderstand the possible applications that can emerge outof them. The thesis aims to find new ways of interactionwith various applications and also aims to find limitationsof the SMA-based tactile actuator.

1.3.2 Applications

Stroke Rehabilitation

A number of people around the world suffer a stroke eachThis thesis exploresa cost-effective

solution forrehabilitation

year. However, many of them fail to afford robotic rehabili-tation devices. This thesis explores the usage of SMA-basedtactile actuators as a cost-effective solution for Stroke Re-habilitation by stimulating movement receptors using skincontraction.

1.4 Outline of the Thesis 3

Haptic Feedback in Virtual Reality Games

Gaming in a Virtual Reality (VR) environment is visually This thesis aims tostimulate natural skinsensations during aVR game

immersive but lacks the sense of touch when interactingwith virtual objects. The thesis aims to explore the usage ofSMA-based tactile actuators to simulate natural skin sen-sations such as a pinch on the skin during a VR game se-quence.

Haptic Navigation

Navigation systems have become an integral part of our This thesis exploresan eyes-freenavigation method

lives when traveling to new locations. However, Naviga-tion systems are mostly visual systems and are hence noteyes-free which can be distracting during driving. This the-sis explores a new method for eyes-free navigation usingskin contraction near the wrist.

Weight Training Guide

Weight training is a highly adopted method among peo- This thesis exploresexplores ahaptic-feedbackmethod to guideexercise movements

ple to stay fit. However, weight training is highly effectiveonly when the exercise is done in the correct manner. Often,newbies miss details such as timing, pattern, and bound-aries of the exercise movement. This thesis explores meth-ods to tackle this problem by using SMA-based tactile ac-tuators to provide haptic feedback in guiding the timing,pattern, and boundaries of an exercise movement.

1.4 Outline of the Thesis

This chapter is followed by “2. Related Work” which de-scribes the existing research in the applications introducedin this chapter. In the third chapter “3. Evaluation of ap-plication areas”, we describe the applications in detail andevaluate them to finally choose the ’Weight Training Guide’

4 1 Introduction

application to study in detail as we find that it is highlycompatible with SMA-based tactile actuators. In the fourthchapter “4. Implementation of Springlets as an ExerciseGuide”, we describe the design decisions and implemen-tation of the Weight Training Guide in detail. This is fol-lowed by “5. User Study” where we evaluate our imple-mentation by performing a user study and applying statis-tical methods. Finally, the last chapter “6. Summary andFuture work” proposes the possible improvements of thedesigned system.

5

Chapter 2

Related Work

Research on application areas such as stroke rehabilitation,virtual reality, haptic navigation, and weight training hasbeen going on for decades as suggested by Jack et al. [2001],Ertan et al. [1998], and Chang et al. [2007]. In this chapter,we look at a few approaches that are in the direction of tac-tile actuators or give a strong foundation towards tactileactuators.

2.1 Stroke Rehabilitation

Occupational therapy can play a very important role in the Occupational therapyis vital but expensivelives of stroke patients [Steultjens et al., 2003]. This therapy

requires skilled therapists to plan necessary activities forstroke survivors in order to improve their quality of life.However, Norman et al. [2014] pointed out that occupa-tional therapy is expensive and highly dependent on thepresence of a skilled therapist.

As a solution, there has been a lot of research in develop- Exoskeleton robotichand is used to trainstroke patients

ing robotic devices that assist and train stroke patients inmaking body movements [Norman et al., 2014]. Ho et al.[2011] proposed one such exoskeleton robot aimed at train-ing a stroke affected hand as shown in Figure 2.1. This de-vice was driven by surface electromyography (EMG) sig-

6 2 Related Work

nals that are read through surface EMG electrodes placedon the impaired arm. This ensured that the device assistspatients in opening or closing their impaired hand onlywhen they intend to do so.

Figure 2.1: Exoskeleton robotic hand training device de-scribed by Ho et al. [2011]

Norman et al. [2014] suggested that such exoskeletonTactile cueing can bean alternative to

exoskeleton devicesrobotic devices have proven to be beneficial in restoringmotor skills in stroke patients. However, they also arguedupon the fact that stroke patients can be deprived of suchapparatus due to cost and size constraints. They took a stepback from traditional robotic devices that use force to guidebody movements and proposed a low-cost tactile cueingtechnique to guide arm movements as shown in Figure 2.2.

Figure 2.2: Skin stretch on the fingertip as a directionalguide [Norman et al., 2014]

2.2 Haptic Feedback in Virtual Reality Games 7

Figure 2.3: Skin-stretch direction guidance device (Left).The eight supported tactile cueing directions (Right) [Nor-man et al., 2014]

The proposed device (Figure 2.3) looked similar to a com-puter mouse and could guide the arm on a horizontal planein eight directions using skin-stretch feedback on the fin-gertip. User studies concluded that the device was effec- Skin-stretch is

capable of guidingthe arm on ahorizontal plane

tive in guiding users through random paths within the sup-ported angle limits of the device [Norman et al., 2014]. Thissuggests that even though the device was not built for help-ing stroke patients with force feedback, it could leveragetactile cues on the fingertip to train stroke patients in prac-tising certain horizontal motions supporting their speedyrecovery.

2.2 Haptic Feedback in Virtual RealityGames

Haptic feedback becomes an important aspect of virtual Haptic feedbackmakes Virtual Realityimmersive

reality environments during natural interactions such astouching, holding or dragging virtual objects [Schorr andOkamura, 2017]. Aiming to make virtual reality an im-mersive experience, many researchers proposed vibrotactilefeedback as a haptic feedback mechanism when interactingwith virtual objects.

In the context of Augmented Reality, Buchmann et al. [2004]proposed a haptic glove fitted with buzzers on the finger-tips. Vibrations from the buzzers were programmed to be

8 2 Related Work

activated while performing dragging, pressing or holdinga virtual object. These vibrations were a placeholder forGloves deliver haptic

feedback onfingertips using

vibration

the actual haptic feedback that one receives while interact-ing with real-world objects. However, Schorr and Okamura[2017] argued that even though vibrotactile haptic feedbackcould instantly notify surface touch events and also conveysurface roughness information, it was inherently incapableof directly conveying properties such as friction and weightwhen holding a virtual object.

As a solution, Schorr and Okamura [2017] introduced a fin-Haptic feedback onfingertips using

skin-stretch is moreimmersive than

vibration

gertip skin stretch device (Figure 2.4) that supported move-ment in three degrees of freedom in Virtual Reality envi-ronments. Properties such as friction and mass were con-veyed by dragging the device’s fingerpad against the skinand stiffness was conveyed by pressing on the skin. Studiesindicated that users could recognize changes in mass, fric-tion, and stiffness while interacting with various virtual ob-jects. Moreover, the studies interestingly indicated a strongconnection between stiffness and mass as many users per-ceived changes in stiffness as a change in the mass of theobject.

In the context of Virtual Reality gaming, immersiveness ofSkin-stretch gives asense of an

elongated arm in VRGames

the experience highly depends on how naturally the hap-tics are rendered on the body [Yamashita et al., 2018]. Ya-mashita et al. [2018] described the necessity of a skin stretchtechnique on the forearm in situations where one needsto extend arms to a superficial limit in fictional games toreach a distant object. They proposed a device to achievesuch a skin stretch as shown in Figure 2.5. The device em-ployed motors, a pulley and a sliding rail weighing 200grams to achieve skin-stretching between the wrist and theelbow. During an arm stretch, it was necessary to increasethe weight of the wrist to create a natural experience ofan extended arm. This was achieved by the sliding railby shifting the weight from the elbow towards the wristduring an arm stretch sequence. The device was exhib-ited at IVRC2017 and the comments from visitors indicatedthat the users could realistically feel the arm extension [Ya-mashita et al., 2018].

2.3 Haptic Navigation 9

Figure 2.4: Skin-stretch device to convey weight, frictionand stiffness information while manipulating virtual ob-jects [Schorr and Okamura, 2017]

Figure 2.5: Skin stretch to deliver superficial hand extension in Virtual Realitygames [Yamashita et al., 2018]

2.3 Haptic Navigation

The idea of a Haptic Navigation system goes back to 1998where Ertan et al. [1998] described a wearable navigation

10 2 Related Work

system based on vibrotactile feedback. Various researchHaptic Navigationsupports eyes-free

interactionworks have explored the placement possibilities of the tac-tors on various body locations including waist, back andwrists [Dobbelstein et al., 2016] [Ertan et al., 1998]. The mo-tivation for a Haptic Navigation system among researcherswas either to guide blind users [Ertan et al., 1998] or to sup-port eyes-free navigation [Pielot et al., 2012].

Steltenpohl and Bouwer [2013] proposed one such device toVibrating beltfunctions as a

navigational guideconvey a general sense of direction to cyclists using vibro-tactors placed in a circular manner (Figure 2.6). The beltwas designed to hint the distance of the approaching turnusing different vibration rhythms. A variation of pulsedrhythms indicated the distance of approaching turn and thehint to make a turn was delivered using a long and contin-uous rhythm.

Figure 2.6: Waist-belt for Navigation with numbers show-ing the positions of vibration units [Steltenpohl andBouwer, 2013]

Dobbelstein et al. [2016] argued that with the advent ofVibrating wristbandfunctions as a

navigational guidesmartwatches, the wrist area was a better area to place thenavigational tactors. Similar to the idea of Steltenpohl andBouwer [2013], they proposed a vibrotactile navigation sys-tem to give pedestrians a general sense of the direction in-stead of turn-by-turn navigation. The device had a form-factor similar to a wristband (Figure 2.7) and the tactorswere placed on four areas of the wrist.

Studies from both the research works (Steltenpohl andBouwer [2013], Dobbelstein et al. [2016]) proved the effec-tiveness of Haptic Navigation as all the users could reach

2.4 Weight Training Guide 11

the destination using the tactile feedback.

Figure 2.7: Wristband fitted with navigational tactors forPedestrian Navigation (Left). Position of the navigationaltactors (Right). [Dobbelstein et al., 2016]

2.4 Weight Training Guide

Numerous people around the world incorporate Weight Weight training isquite common butinvolves risk of injury

Training in their routine to maintain fitness [Toor et al.,2017]. However, performing weight exercises in an inap-propriate manner often involves the risk of injury [Tooret al., 2017]. This motivated researchers to look for newideas in the direction of guiding exercise movements andthe following sections discusses some of these ideas.

2.4.1 JARVIS : A Virtual Reality Exercise Coach

Rabbi et al. [2018] introduced the concept of a Virtual Real- Virtual Reality can beused to guideexercise movements

ity trainer to guide weight-exercise movements. The mainmotivation behind this device (Figure 2.8) was to give feed-back to beginners regarding the ideal body movements andspeed of the exercise. The device made use of an animatedvirtual character that performed the exercise sequences toguide the users visually. The device could also identify thetype of exercise, provide feedback regarding the exerciseand track the progress of the exercise.

12 2 Related Work

Figure 2.8: JARVIS : A Virtual Reality Weight TrainingCoach [Rabbi et al., 2018]

User studies showed that the right set of muscles were bet-VR trainer is effectiveand enjoyable ter activated during an exercise with Virtual Reality when

compared to the traditional approach of naturally perform-ing the exercise movements without VR [Rabbi et al., 2018].The comments from the users showed that they were mo-tivated to follow the virtual character in doing the exerciseand enjoyed the interactive exercise coaching experience.

2.4.2 Haptic Guidance as an Exercise Guide

As discussed previously in the Stroke Rehabilitation sec-Skin-stretch near theindex finger is known

to trigger fingermovements

tion, Norman et al. [2014] demonstrated that haptic skin-stretch devices are effective in guiding arm movements(Figure 2.3). Before this work, there was already substantialresearch in the context of Neurophysiology to prove thatsensory receptors on the skin can trigger finger movements[Collins et al., 2005]. Collins et al. [2005] extended the exist-ing research work to investigate if skin-stretch near indexfinger, elbow and knee joints trigger movements in the re-spective joints.

Collins et al. [2005]’s experimental setup made use of anelastic band that was connected to the skin through a thread


Figure 2.9: Setup to achieve skin-stretch near the index finger joints (Left). Positionof MCP and PIP joints of the index finger (Right) [Collins et al., 2005]

Figure 2.10: A: Comparison of mean intensities of finger bending movements invarious settings for a group of participants (n=8). B: Average perceived fingermovements with and without skin-stretch from one participant [Collins et al., 2005]

and an adhesive tape pasted onto the skin (Figure 2.9). The experimentalsetup correspondingto the index finger

Skin-stretch was achieved by pulling and releasing the elas-tic band alternatively. A study was conducted with 8 par-ticipants which aimed at comparing the extent of triggeredfinger movements with two levels of skin-stretch inten-sities including and excluding vibration at the MCP andPIP finger joints (Figure 2.9). The results indicated thatskin-stretch with vibration near the finger joint triggereda higher bending movement of the finger than with just vi-bration near the finger joint (Figure 2.10).

14 2 Related Work

Figure 2.11: A: Comparison of mean intensities of elbow bending movements invarious settings for a group of participants (n=10). B: Average perceived elbowmovements with and without skin-stretch from one participant [Collins et al., 2005]

Figure 2.12: A: Comparison of mean intensities of knee bending movements in var-ious settings for a group of participants (n=10). B: Average perceived knee move-ments with and without skin-stretch from one participant [Collins et al., 2005]

A similar setup was used to compare the extent of trig-gered elbow and knee movements resulting from skin-stretch near the elbow and knee joints respectively. The re-Skin-stretch is vital in

triggeringmovements

sults (Figure 2.11 and 2.12) were closely similar to index fin-ger movement triggers indicating that skin-stretch played akey role in triggering movements near the joints.


2.4.3 A Comparison of Various Guidance Tech-niques

Research work described in the previous sections shows A comparison ofVisual and HapticGuidance

that it is possible to guide body movements using visualand haptic methods. Hence, it is important to understandhow both of these methods compare with each other. Fey-gin et al. [2002] performed a study to compare Haptic Guid-ance and Visual Guidance techniques where users had toredraw complex trajectories that were learned by only vi-sual guidance, only haptic guidance, and a combination ofvisual and haptic guidance. To receive haptic guidance, theuser held the actuator while the actuator guided the userthrough the pattern. The redrawing phase was tested us-ing only Haptic guidance and also a combination of Hapticand Visual guidance. The user studies indicated that the Visual Guidance is

better to learn thepattern while HapticGuidance is better tolearn the timing ofthe movement

timing of the movements was better learned from Hapticguidance when compared to Visual guidance. However,Visual guidance was more effective than Haptic guidancein communicating the pattern of the movement. While inthis study, haptic guidance was received by holding the ac-tuator, there also exist other guidance methods such as vi-bration and skin-stretch.

Bark et al. [2008] compared vibrotactile and skin-stretch A comparison ofvibrotactile andskin-stretch guidance

guidance methods by setting up a study where the sub-jects had to move the cursor to a target position from thestart position. Subjects started at a random start positionand were instructed to move a fixed number of steps toleft or right to reach the target position. They had to ac-curately stop at the target cursor location by perceiving theamplitude of the vibration where the lowest vibration am-plitude signified the leftmost end and the highest vibra-tion amplitude signified the rightmost end. Skin-stretch, onthe other hand, was delivered using rotational skin-stretch Skin-stretch is

effective incommunicating thedirection of themovement

where anti-clockwise stretch signified moving to the leftand clockwise stretch signified moving to the right. Theuser studies indicated that skin-stretch was more effectivethan vibration in guiding users to the target position. Barket al. [2008] noted that this was because skin-stretch had thenatural ability to convey position information and hencewas more intuitive to the user.

16 2 Related Work

This concludes that Haptic Guidance has several advan-tages in guiding body movements especially when the tim-ing, direction, and extent of the movement are important.

2.5 Springlets: A Light-Weight Tactile Ac-tuator

As an alternative to complex and heavy mechano-tactileSpringlets are analternative to

complex mechanicalactuators

actuators, Hamdan et al. [2019] introduced the idea of alight-weight tactile actuator called Springlets. The maindriving part of Springlets was the shape-memory alloy(SMA). Spring-shaped SMAs possess the capability of heat-ing and contracting when current passes through it andrestoring its original shape and cooling down when thecurrent flow stops. Springlets made use of spring-shapedshape-memory alloys to achieve various sensations on theskin such as pinching, dragging and pulling. Different sen-sations were achieved by employing an appropriate de-sign while building the Springlet. User studies indicatedSpringlets could flexibly fit in various parts of the bodysuch as arm, neck, and shoulder generating smooth, noise-less and noticeable sensations. Moreover, it was also foundout that the variant designed to generate pinching sensa-tions could be perceived faster (M=1.22 sec, SD=.87) thanother variants.

2.6 Contributions of this Thesis

While the existing research work targets various applica-Using springlets as alight-weight

alternative for thementioned

applications

tion areas such as Stroke Rehabilitation, Virtual Realitygames, Haptic Navigation and Weight Training using com-plex mechanical actuators or vibration units, this thesis em-ploys Springlets as a light-weight, easy-to-build, noiselesstactile interface that can serve as an alternative to com-plex mechano-tactile actuators employed in various do-mains. Especially, the area of learning exercises using Hap-tic Guidance is unexplored and hence this thesis investi-gates this area in detail with a user study making use of a

2.6 Contributions of this Thesis 17

Figure 2.13: A : A variant of Springlets that produces apinching sensation on the skin. B: The effect of the Springleton the skin that turns into a pinching sensation. [Hamdanet al., 2019]

skin-contraction technique unlike a skin-stretch techniquein other studies. While Collins et al. [2005]’s study focuses Learning exercises

using Springlets isexplored in detailwith a user study

on creating an illusion of body movements, this thesis fo-cuses on generating noticeable cues that signifies the bodymovements during an exercise. This thesis also investigatesthe effectiveness of Springlets in harnessing the power ofskin contraction in order to assist or signify finger move-ments in stroke patients, deliver natural skin feedback dur-ing various scenarios in Virtual Reality games, and com-municate noiseless Haptic Navigation signals on the wrist.Lastly, all the mentioned application areas are evaluated inChapter 3 where we choose to explore the area of learningexercises in detail.

19

Chapter 3

Evaluation ofapplication areas

This chapter investigates the suitability of Springlets inthe application areas such as Stroke Rehabilitation, VirtualReality games, Haptic Navigation and Learning exercises.This chapter aims to choose one application area among thementioned application areas to investigate in detail.

3.1 Evaluation of Springlets for Stroke Re-habilitation

Inspired by the work of Collins et al. [2005], we inves- We studied thefeasibility of usingSpringlets tostraighten a bentfinger

tigated the role of skin contraction in driving kinestheticmovements in the body with Springlets. The goal was tocheck if Springlets are strong enough to drive or assist sim-ple finger movements or provide a strong cue towards fin-ger movements. In our feasibility study, we targeted the in-dex finger on the right hand to check if a bent index fingercan be assisted to a straightened position by the Springlet(Figure 3.2).

The next step was to choose an appropriate design of theSpringlet to support the straightening movement of the in-dex finger. Hamdan et al. [2019] discusses two approaches

20 3 Evaluation of application areas

of achieving skin actuation using Springlets - The Direc-tional Stretcher and the Pincher. The Pincher pulls the skinfrom both ends of the springlet towards the center of thespringlet while the Directional Stretcher generates a one-sided skin-stretch (Figure 3.1).

Figure 3.1: A : Operation of a pincher springlet. B: Opera-tion of directional-stretcher Springlet. [Hamdan et al., 2019]

Figure 3.2: A: Straightened index finger highlighting thecontracted skin pattern on the middle joint. B: Bent indexfinger highlighting the stretched skin pattern on the middlejoint

On closely examining the middle joint of the index fin-We examined theskin patterns of the

bending fingerger, we found that the skin on both sides of joint comecloser towards the joint when the finger is straightenedand goes apart on both sides when the finger is bent (Fig-ure 3.2). This behaviour closely resembles the operation of a

3.1 Evaluation of Springlets for Stroke Rehabilitation 21

Figure 3.3: A: Highlighted position of the Springlet in the first attempt. B: High-lighted position of the Springlet in the second attempt. C: Highlighted position ofthe Springlet in the third attempt D: Highlighted position of the Springlet in thelast attempt

Pincher Springlet and hence we decided to go further witha Pincher Springlet. It was now necessary to decide the po-sition of the Springlet to ensure an efficient finger move-ment. The palm-side of the index finger was ruled out forthe positioning as it would hinder the bending movement,cause discomfort on the palm due to heat, and could dam-age the topology of the SMA.

The springlet was initially chosen to be placed directly onto Springlets wereaffected by thetopological changesof the finger

the middle joint of the index finger (Figure 3.3 A). However,it was found that this position was not effective in straight-ening the finger or providing straightening cues due tochange in topology of the index finger while bending andstraightening the finger. In the second attempt, the Pincherwas placed on the bottom joint of the index finger (Fig-ure 3.3 B) but it was still inefficient in generating fingermovement cues because the minor topological change ofthe finger was still nullifying the effect of the Pincher. In thethird attempt, the area between the middle joint and bottomjoint was chosen (Figure 3.3 C) as this area doesn’t sufferfrom topological changes during the bending movement.However, the length of the area was quite low to generatea noticeable pinch. This was mainly because Springlets can Springlets did not

work efficiently on asmall area and onloose skin

only contract until about half of its length and this signifiesthat the amount of contraction would be quite low to benoticeable if the Springlets had a lower length. In the lastattempt, the area below the bottom joint (Figure 3.3 D) wasinvestigated. However, this area suffered from loose skinand contraction of the springlet did not provide any cuesfor movement.


3.1.1 Conclusion of the Evaluation

Springlets mainly suffered from problems such as topolog-Stroke Rehabilitationwas not explored in

detail due to theobserved problems

ical changes on the joints, low space, and loose skin andhence could not offer an efficient operation to trigger move-ment in the fingers. Further, it was observed that while thestrength of the Springlets could be enough to provide a cuefor the finger movement, it was quite low to be able to as-sist finger movements even with the stronger versions ofthe Pincher. Due to these problems, the Stroke Rehabilita-tion application was not chosen to be explored in detail.

3.2 Evaluation of Springlets for VirtualReality Games

We investigated the possibilities of making Virtual Reality(VR) an immersive experience using Springlets as a deviceto deliver natural sensations on the skin during a VR game.The first step was to check if Springlets could offer a simi-We explored the

usage of Springletson the forearm and

the wrist to generateimmersive haptic

experiences in VRgames

lar haptic experience as described by Yamashita et al. [2018](Figure 2.5) during a distant arm-stretch movement in a VRgame. The idea was to arrange multiple Pinchers parallellybelow the elbow region of the arm (Figure 3.4) and acti-vate all the Pinchers during the arm-stretch movement inthe game. This could enable Springlets to handle naturalskin-stretch requirements of a VR game without using anycomplex setup. Further, we also conceptualized the idea ofusing Springlets on the wrist as an immersive experiencefor games such as Spiderman during a web-shooting se-quence (Figure 3.5).

A quite common method to deliver haptic signals of gettingWe explored the ideaof placing Springlets

on multiple bodylocation to simulate

getting shot in VRgames

hit by bullets in games is by using vibration units [Cohenet al., 2017]. But, the feeling of getting hit by bullets couldfeel more real if the sensation produced matched a quick-pinch sensation that stayed for a while before getting backto normal. This closely matched the operation of a PincherSpringlet and the idea was to have Springlets on multiplelocations on the body and activate them during a bullet-shooting sequence in a VR game.

3.2 Evaluation of Springlets for Virtual Reality Games 23

Figure 3.4: Left : Pinchers positioned parallely on the arm to create long-arm illu-sion in VR games. Right: Visual arm-stretch experience in a VR game [Yamashitaet al., 2018]

Figure 3.5: Left : Pincher on the wrist to create a haptic web-shoot illusion.Right: Visual web-shoot experience in the Spiderman game - Image credit: http://createvr.com/spider-man-far-from-home (accessed: 17.11.2019)


Games are usually fast-paced which means that Haptic de- Games arefast-paced butSpringlets are slow

vices need to react fast to create an immersive VR experi-ence. But, we observed that while medium-sized Pincherscould contract almost instantaneously on passing a highamount of current, they required at least 3 seconds to get

http://createvr.com/spider-man-far-from-home

http://createvr.com/spider-man-far-from-home


back to the uncontracted state. This behavior was not com-patible with most fast-paced VR games as Springlets couldbe actuated and released only once in about 3 seconds.Hence, the area of Virtual games was not chosen to be ex-plored in detail.

3.3 Evaluation of Springlets for HapticNavigation

Figure 3.6: Two Springlets placed on the wrist to supportturn-by-turn Haptic Navigation

Inspired by the work of Dobbelstein et al. [2016] and Ham-We exploredturn-by-turn

navigation usingSpringlets

dan et al. [2019], we investigated the possibility of usingSpringlets as a Haptic Navigation device. The main ideawas to place two Pinchers on both sides of the wrist (Fig-ure 3.6) to deliver turn-by-turn navigation signals usingskin-contraction. The obvious advantage over vibrationunits was that Springlets could deliver mild and noiseless

3.4 Evaluation of Springlets for Weight Training 25

haptic signals when compared to harsh vibrations originat-ing from vibration units. We also conceptualized the po-sitions of Springlets on other locations in the body such asupper-arm and shoulders to enable mild Haptic Navigationsignals from Springlets hidden inside clothes.


Navigation devices are used in multiple situations such as Springlets generatemild sensations andcan fail in fast-pacednavigation scenarios

travelling by vehicle or walking by foot. In both situations,Navigation involves mobility that is caused by movementsof body parts. As Springlets deliver mild sensations, thesesensations could go unnoticed frequently as Springlets areless noticeable in mobility situations [Hamdan et al., 2019].Further, as Springlets are not compatible in fast-paced envi-ronments, they would fail in scenarios where there are mul-tiple turns at low distances in the same direction as therewould not be enough time for the Springlets to go back tothe uncontracted state and contract again to signal a turn.Due to these inherent problems, the area of Haptic Naviga-tion was not chosen to be explored in detail.

3.4 Evaluation of Springlets for WeightTraining

Collins et al. [2005] showed that it was possible to create Skin patterns on thearm allow the use ofPincher Springlets toguide arm movement

movement illusions using skin-stretch. Moreover, Feyginet al. [2002] pointed out that Haptic Guidance was a goodchoice over Visual Guidance when the timing of the move-ment was important. This was the motivation to look intothe area of guiding weight exercises as timing plays an im-portant role in the effectiveness of exercises. The initial ideawas to guide arm movements to train the Biceps muscle.The natural skin patterns on the region on the other sideof the elbow (Figure 3.7) showed that the skin is stretchedat this region when the arm is stretched and the skin fromboth sides of the arm shrink at the same region when thearm is bent towards the shoulder. The skin patterns closelyimitated the operation of the Pincher Springlet.


Figure 3.7: Left: Skin patterns when the arm is stretchedRight: Skin patterns when the arm is the bent towards theshoulder

Due to these skin patterns, skin-contraction seemed to bean better alternative when compared to one-sided skin-stretch described by Collins et al. [2005].


During the feasibility study, we evaluated several positionsHaptic cues fromSpringlets are

noticeable andconsistent with the

nature of the weightexercises

of the Pincher Springlets near the region specified in (Fig-ure 3.7) and we found that the haptic cues generated bythe Pinchers were noticeable and could potentially serveas an arm-movement guide. The delay in the operation ofSpringlets was consistent with the fact that weight exercisesare meant to be slow in order to be effective. As these fac-tors pointed towards the effectiveness of Springlets as anexercise guide, we investigated various positions and de-signs of the Pincher Springlet in Chapter 4 to ensure its ef-ficient operation as an exercise guide.

27

Chapter 4

Implementation ofSpringlets as an ExerciseGuide

4.1 Design of the Exercise Guide

As skin patterns above the elbow during an arm move-ment suggested the use of a Pincher Springlet, the design ofthe Springlet to guide weight exercises was inherited fromHamdan et al. [2019].

4.1.1 Shape Memory Alloy

Spring-shaped shape memory alloy (SMA) is the core of a SMA shrinks when acurrent greater thanthe threshold currentis supplied

Pincher Springlet. On passing electricity, the SMA heats upand shrinks to about half its size (Figure 4.1). The shrinkcreates a pulling force at both the ends of the SMA towardsthe center of the SMA. However, to activate the shrinkingprocess, the current passed must be greater than the thresh-old current of the SMA. The threshold current varies withdifferent SMAs. Generally, the coil diameter of an SMA isdirectly proportional to the pulling force, generated heatand the threshold current.

28 4 Implementation of Springlets as an Exercise Guide

Figure 4.1: A : Stretched Springlet measuring 10.5 cm. B: Shrinked Springlet mea-suring 5 cm after passing current.

Figure 4.2: Pincher Springlet construction steps numbered from 1-6

4.1.2 Construction of the Pincher Springlet

Hamdan et al. [2019] described the use of rapid-fabricationWe built Springletsusing two functionallayers with the SMAplaced in the center

technique as a quick way to construct the Pincher Springletusing low-cost materials with three functional layers. Webuilt a simpler version of the Pincher Springlet involv-ing two functional layers namely the silicon layer and thekinesio-tape layer. Figure 4.2 shows the steps that werefollowed to construct the Pincher Springlet. The ends ofthe SMA were first crimped to electrical wires with malejumper pins. A rectangular layer of size 3 x 1 cm is removedfrom the center of a thin rectangular layer of silicon measur-ing 6 x 2.5 cm. Both the male jumper pins connected to theends of the SMA were then pierced into the ends of the rect-angular layer of silicon keeping the SMA in the center of the

4.1 Design of the Exercise Guide 29

silicon layer. Finally, the rectangular silicon layer with theSMA was then pasted to a layer of Kinesio tape of the samesize completing the construction of the Pincher Springlet.

4.1.3 Pincher Designs

As the Pincher Springlet was aimed to guide arm move-ments during Biceps training, we investigated several de-signs of the Pincher Springlet to improve the movementcues generated by the Pincher before finalizing the designmentioned in (Figure 4.3 D).

Large Pincher

Figure 4.3 A shows the initial design of the Pincher Large Pinchergenerates a lot ofheat and it peels offeasily

Springlet. This design was made of a large SMA so that itcould generate a stronger pinch leading to stronger move-ment cues for the exercise movement. However, while thePincher generated a strong pinch, it generated a a lot of heatand was consistently peeling off from both the ends duringthe contraction of the SMA.

Wide-Ended Large Pincher

There were hardly any movement cues from the first de- Wide-Ended Pincherresulted in a pullingsensation at the endsrather than apinching sensation

sign because of the SMA getting peeled off at both the endsmainly due to strong forces of the large SMA. To solvethe peeling issue, we changed the design of the Pincher asshown in Figure 4.3 B. The ends of the Pincher Springletnow were more wider than before resulting in a good gripat the ends of the SMA. However, the result was that the”Pinching” sensation quickly turned into ”Pulling” sensa-tion from both the ends because the ends covered a largerarea of the skin.


Figure 4.3: Investigated Pincher Springlet Designs

Thin-Base With Medium-Sized Pincher

In order to keep pinching sensation intact with a goodThin-Base Pincherresulted in a pinchingsensation but did notcover a wide area on

the skin

grip at the ends, the width of the ends were reduced (Fig-ure 4.3 C) when compared to the Wide-ended Large PincherDesign. This resulted in better grip at the ends when com-pared to the Large Pincher Design and also resulted inmore of a ”Pinching” sensation rather than a ”Pulling sen-sation”. Also, this design incorporated a medium-sizedPincher to solve the heat problem from the Large PincherDesigns. To increase the Pinching feel without overloadingthe medium-sized Pincher, the width of the base was re-duced to 1.7 cm. However, while there was higher amountof Pinch, the Pinch was not strongly felt as the SMA covereda large but not wide region on the skin. Also, there was stillsome amount of pulling sensation at the ends as the endswere still a little wider than the main base of Springlet.

Short and Wide Base With Medium-sized Pincher

As the Pinch was not strongly felt in the Thin-Base de-Changes in skinpatterns are at peak

near the elbow.Hence, a large

Pincher wasunnecessary

sign, the new design (Figure 4.3 D) incorporated a widebase similar to the Large Pincher design. As the changes inskin patterns during an arm movement occur in the regioncloser to the elbow joint, the large length of the Springletwas unnecessary and hence the length was decreased to 6

4.1 Design of the Exercise Guide 31

cm. To solve the minute ”Pulling” sensation near the ends Peeling was solvedby spacing the SMAaway from the ends

of the SMA, the width of the ends were not changed whencompared to the width of the main base of the Springlet. Asthis would have recreated the peeling issue, the SMA wasnow attached to the base with a small amount of spacingfrom both the ends of the SMA. This ensured that forceswere not high at the ends of the Pincher Springlet whichensured that the Springlets were not peeled off easily at theends when they contracted.

There were several advantages of this design over all the Short and Wide Basedesign has severaladvantages overother designs

other designs. First of all, the ”Pinching” sensation washigh as the Pincher’s length was small and hence the SMAwas not overloaded and could efficiently contract. Sec-ondly, the Pincher occupied a wide region on the same areaof the skin where the skin patterns change the most duringan arm movement. Thirdly, the peeling issue was reducedfrom the initial designs while keeping the ”Pinching” sen-sation intact. Hence, this design was finalized for the im-plementation of the Pincher Springlet to function as an ex-ercise guide.

4.1.4 Requirement of Multiple Springlets

Apart from the position of the Springlets, the configura-tion in which they were placed also had an importantrole to play. The initial trials showed that more than onespringlet was required to function as a Biceps training Ex-ercise guide. This was because Biceps training requiresupward and downward movement of the arm and onePincher could only guide the movement in one direction.

When the Pincher shown in (Figure 4.3 D) was placed on One Springlet isinsufficient to guidearm movement

the arm and was activated by passing a current greaterthan the threshold current, it stretched the skin to gener-ate a cue possibly signaling the upward movement of thearm. However, when the current flow was stopped, theSpringlet went back to its unstretched configuration with-out generating any cue signaling the movement of the armin the downward direction. In fact, one could not noticethat the Springlet went back to its normal configuration as


the skin on the arm got loose. This created the need to havea secondary springlet to guide the movement in the oppo-site direction.

Figure 4.4: The arm movement pattern in a Biceps MuscleExercise. The Springlet on the top (orange) guides upwardsarm movement and the bottom Springlet (green) guidesdownward arm movement

4.2 Investigated Placement Positions

It was important to place the Pincher Springlets on the rightpositions on the arm so as to generate an efficient move-ment cue to guide the Biceps training exercise.

4.2.1 Elbow

The initial idea of choosing a position was an area on thearm where the skin stretches and contractions are at themaximum during a Biceps exercise. Hence, the chosen po-sition on the arm was the elbow and the area exactly on theopposite side of the elbow (Figure 4.5). When the Pincher

4.2 Investigated Placement Positions 33

Figure 4.5: Springlets positioned on the elbow and abovethe elbow

activated on the top, it was observed that the SMA contrac-tion did not create any haptic cue. This was because the Springlet contraction

on elbow did notcontract the skin

SMA was not placed on the skin in a flat manner due to thetopology of the arm near this area. As the base was not flat,the contraction of the SMA did not result in contraction ofthe skin (Figure 4.5). The SMA on the elbow was identicallyplaced as the SMA above it. As this area also did not have aflat base, the contraction of the SMA on the elbow was notfruitful.

4.2.2 Outer Side of the Elbow

To solve the problem of an uneven base, another region on The outer side had astable base but failedto produce amovement cue in ameaningful direction

sides of the arm was investigated (Figure 4.6). This regionhad a flat base as well as skin movements during a Bicepsexercise. However, the movement of the arm created a side-ways topological change in the base of the Springlet. Al-though the base was flat now, the contraction of the SMAstill did not result in an efficient cue signaling arm move-


ment in a certain direction.

Figure 4.6: Springlets positioned on the outer sides of theelbow

4.2.3 Upper Forearm

Going further, the area on the upper forearm was investi-Springlet on theupper forearm

generated a strongcue to move upwards

gated (Figure 4.7). The reason to choose this area was be-cause this area had a flat base and was the closest to theelbow where the skin contractions are at the maximum dur-ing a Biceps exercise. When the SMA on the top area wasactivated to contract, it was observed that it generated astrong cue to move the arm upwards. This was mainlybecause the skin on this area is usually tight and whenthe SMA contracts, the pulling force is highly noticeableand can function as a strong cue signaling arm movement.Naturally, the direction of this movement must be upwardbacked by the fact that the skin patterns generated by thepincher imitates the natural skin patterns created duringthe movement of the arm in upward direction.

The secondary Springlet to generate cues for downwardarm movement was identically placed near the elbow onthe opposite side of the top springlet. The skin on this

4.2 Investigated Placement Positions 35

Figure 4.7: Springlets positioned on the upper forearm

area stretches when making an upward arm movement andcontracts while making a downward arm movement. Theplaced on this area imitates the contracting skin patternhence generating a cue for the downward arm movement.However, it was noticed this cue was less stronger than thecue of the top springlet due to lower changes in natural skinpatterns during an upward and downward arm movement.

4.2.4 Upper Arm

As a substitute to the placement of Springlets on the up- On the upper arm,both the Springletswere effective

per forearm, the placement of the Springlets on the upperarm near the elbow joint was investigated (Figure 4.8). Thepinchers were placed in a similar configuration as on theforearm. It was noticed that the Pincher on the top gener-ated strong cues for upward arm movement and the lowerspringlet generated a cue at a similar intensity for down-ward arm movement.


Figure 4.8: Springlets positioned on the upper arm

4.2.5 Selected Positions

From the investigation, it was clear that the PincherSpringlets were not efficient in generating cues on the el-bow and outer side of the elbow. Hence, the Upper Fore-arm and Upper Arm positions (Figure 4.7 and Figure 4.8)were chosen to be studied in detail.

4.3 Operation of the Pincher as an ExerciseGuide

4.3.1 Electronic Circuit

Appendix A shows the Breadboard layout describing theThe electronic circuitis powered by an

Arduino UNO,transistors and

MOSFETS

electronic circuit that powers the Pincher Springlets to func-tion as an exercise guide. The electronic circuit was built ontop of an Arduino UNO microcontroller which is poweredby 5V. The activation of Pincher Springlets was controlledusing electronic signals from the Arduino board that travelthrough a 2N222A transistor and a IRF540N MOSFET con-nected in series. The transistor and MOSFET together act

4.3 Operation of the Pincher as an Exercise Guide 37

as a switch and disable the 6V DC power supply to thePincher Springlets when there is no signal from the micro-controller.

4.3.2 Exercise Guide in Action

In order to guide the Biceps muscle exercise, two Pinchers The top Pincherdrives upwards armmovement and thebottom Pincherdrives downwardsarm movement

were required to be placed on the top and bottom region ofthe upper forearm or upper arm. (Figure 4.9) shows an ex-ample of the required setup with one Pincher placed on topof the upper forearm and the other placed on the bottomof the upper arm. The Pincher on the top was responsi-ble for guiding the arm upwards towards the shoulder andthe Pincher on the bottom was responsible for guiding thearm downwards. The Pinchers together continuously gen-erate cues for an upward and a downward arm movementrequired in a Biceps exercise.

Figure 4.9: An example configuration of two PincherSpringlets to function as an exercise guide


To activate the Pincher on the top, an electronic signal isThe transistors drivethe MOSFETs which

finally activate theSMAs

sent to the transistor from one of the analog pins on the mi-crocontroller. This switches on the transistor and it allowsthe current from the external power source to flow to theMOSFET. This switches on the MOSFET which is capable ofenabling a high current flow to the SMA embedded insidethe top Pincher. This enables the top Pincher to contract theskin of the top of the forearm enabling the top Springlet toguide the upward arm movement.

The microcontroller was programmed to allow the currentflow to the top Springlet for 3 seconds which is sufficienttime for the top Pincher to contract. After this, a delay of4 seconds was programmed following which the Pincheron the bottom is activated in a similar manner. This cyclecontinues with a delay of 4 seconds in between the cyclesto allow the user to complete the arm movement.

4.3.3 Logging Arm Movements

The electronic circuit also connects to an additional ac-Arm movements aredetected through the

accelerometerplaced on the

dumbbell

celerometer placed on the dumbbell. The accelerometerwas placed to track changes in the position of the arm dur-ing the user study. The microcontroller was programmedto read the position information from the accelerometer ev-ery 200 milliseconds and print it on the Serial port. Fur-ther, a simultaneous program running on the host machinecontinuously picked these values from the Serial port andrecorded them into a file on the host machine with times-tamps.

39

Chapter 5

User Study

5.1 Research Questions

Figure 5.1: Research Questions

40 5 User Study

5.2 Hypotheses

HYPOTHESES OF THE STUDY:H0: The haptic sensations delivered by Springlets do notgenerate cues that signal arm movement for the Bicepsexercises.

H1: There is no difference in reaction times whenSpringlets are placed in different positions on the arm.

H2: There is no change in sensation to move thearm when Springlets are placed in different positions onthe arm.

Definition:Hypotheses of the

study

5.3 Pilot Study

We performed a pilot study with two participants beforeThe pilot studyinvolved twoparticipants

starting the actual study. The goal was to find deficits inthe experiment setup. No problems were observed withthe pilot study involving the first participant. In the pi-lot study involving the second participant, it was observedthat the Springlet easily peeled off during the contractionof the SMA. Later, it was discovered that the participanthad used a skin product that made the skin oily. Hence,it was concluded that oily skin could create problems andthis was to be kept in mind while recruiting participants inthe actual study.

5.4 Study Design

From the previous investigations, we found that SpringletsDiagonal placementgave rise to four

Springletconfigurations

were efficient when placed on the upper arm and upperforearm. It was also interesting to see how Springlets per-form when placed diagonally in both of these positions.This gave rise to 4 configurations (Figure 5.2) and the aimwas to find out if there is a feeling to move the arm up-

5.4 Study Design 41

Figure 5.2: The four Springlet configurations for the user study. A: Top forearm,Bottom forearm (TFA, BFA) B: Top upper-arm, Bottom upper-arm (TUA, BUA) C:Top forearm, Bottom upper-arm (TFA, BUA) D: Top upper-arm, Bottom forearm(TUA, BFA)

wards and downwards in any of the configurations. Wealso aimed to find out which configuration among theseconfigurations drives an easy understanding of the armmovement and also has a low reaction time.

5.4.1 Independent Variables

There are two independent variables in this study:

1. The direction of arm movement:

(a) Upwards arm movement(b) Downwards arm movement

2. The configuration in which the Springlets are ar-ranged:

(a) Top forearm, Bottom forearm (TFA, BFA): Theupper Springlet is placed on the top of the fore-arm and the bottom Springlet is placed on thebottom of the forearm (Figure 5.2 A).

(b) Top upper-arm, Bottom upper-arm (TUA,BUA): The upper Springlet is placed on the topof the upper-arm and the bottom Springlet isplaced on the bottom of the upper-arm (Fig-ure 5.2 B).

(c) Top forearm, Bottom upper-arm (TFA, BUA):The upper Springlet is placed on the top of theforearm and the bottom Springlet is placed onthe bottom of the upper-arm (Figure 5.2 C).

42 5 User Study

(d) Top upper-arm, Bottom forearm (TUA, BFA):The upper Springlet is placed on the top of theupper-arm and the bottom Springlet is placed onthe bottom of the forearm (Figure 5.2 D).

5.4.2 Dependent Variables

There are two dependent variables in this study:

1. Time taken by the participant to start an arm move-ment after the Springlet activates.

2. Likert Scale rating from 1-5 for questions regardingunderstandability and comfort for each Springlet con-figuration.

5.4.3 Study Type

The study followed a within-group design. Therefore, eachA participantevaluated all the four

Springletconfigurations on a

Likert Scale

participant completed the tasks for all the 4 Springlet con-figurations. The reaction time to make an arm movementwas measured using the attached accelerometer while theparticipant performed the task for a particular configura-tion. After completing the task for a particular configura-tion, the participant rated the questionnaires related to theconfiguration on a 1-5 Likert Scale.

5.4.4 Latin Square Randomization

To ensure that the test results were not biased with the or-We randomized theorder of the Springlet

configurationsder in which the participants go through the Springlet con-figurations, we randomized the order of the Springlet con-figurations for each participant using the Latin Square tech-nique.

5.4 Study Design 43

Participant 1 1 2 4 3Participant 2 2 3 1 4Participant 3 3 4 2 1Participant 4 4 1 3 2Participant 5 4 3 1 2Participant 6 1 4 2 3Participant 7 2 1 3 4Participant 8 3 2 4 1Participant 9 1 2 4 3

Participant 10 2 3 1 4Participant 11 3 4 2 1Participant 12 4 1 3 2Participant 13 4 3 1 2Participant 14 1 4 2 3

Table 5.1: The order in which each participant goes throughthe Springlets configurations. The Springlet configurationsare represented by digits 1-4.

5.4.5 Setup and Surroundings

The user study was conducted in a quiet room. The study The study took placein a quite room andthe participant madearm movements in astanding position

did not take distractions into account as the primary aimof this user study was to identify if skin contractions usingSpringlets was capable of producing cues for arm move-ment. The setup (Figure 5.3) consists of a power supplyconnected to the electronic circuit. The electronic circuitcontrols the current supplied to the two Pinchers. The par-ticipant executes the study in a standing position, hold-ing the dumbbell and making arm movements in an ap-propriate direction (Up and Down) when the appropriateSpringlet activates. The connected accelerometer in thedumbbell keeps track of the arm movements and sends thedata to the computer via the serial port.

5.4.6 Experiment Procedure

The user study took 60 minutes to complete per partic-ipant. The participant was first presented with an “In-formed Study Consent” explaining the procedure and the

44 5 User Study

Figure 5.3: The user study setup

risks involved in the study. After the participant signed theThe participantstarted with filling out

the Study Consentconsent form, he was presented with a “General Question-naire” which contained questions such as name, age, sex,arm length, and previous gym experience.

Before proceeding with the actual study, the participantsParticipants wentthrough trial rounds

before the actualstudy and were

asked if they sensedany motivation to

move their arm

were given a feel of the actual study by asking them to holdthe dumbbell, attaching the Springlets participant’s rightarm and activating the electronic circuit so that the partici-pant gets a feeling of the Springlets and the minor heat thatthey produce. Then, the participant was asked to imagine ifthe Springlets were trying to guide them to move in a par-ticular direction, What direction would it be according tothem? If the participant responded with the expected verti-cal arm movement pattern of the Biceps exercise, they wereinformed that we are studying the same movement patternin this study. Otherwise, they were shown the vertical armmovement pattern for the Biceps exercise that is being stud-ied as part of this user study.

Then the participant went through 4 rounds of the userstudy testing each Springlet Configuration. In each round,the participant was asked to make an arm movement in

5.5 Experiment Recordings 45

the expected pattern only when he feels the cue. The map- The participant hadto figure out themappings bythemselves

pings of the upper and lower Springlets to the movementdirection was not explained. During each round, the at-tached accelerometer on the dumbbell measured the reac-tion times.

After each round, the participant was presented a question-naire to rate the facts related to the last round involving oneof the Springlet configurations.

5.5 Experiment Recordings

5.5.1 Videos

All the user studies were video recorded in order to drawretrospective observations from the user studies at a laterpoint in time.

5.5.2 Reaction times

Reaction time is the time required to sense the Springlet ac-tuation and accordingly make an arm movement. The reac-tion time was recorded for all the Springlet configurationsin all the user studies.

5.5.3 Likert Scale Ratings and Rankings

The participants were presented with questionnaires after Participantsevaluated theconfigurations usingLikert Scales andRankings

performing tasks for each Springlet configuration. The par-ticipants then rated the questionnaires from 1-5 on a LikertScale. At the end of the study, the participant was asked torate the Springlet configurations from 1 (best) to 4 (worst).

46 5 User Study

5.6 Participants

A total number of 14 participants (13 Male, 1 Female) took14 participants tookpart in the user study part in the study. The average age of the participants was

26 (SD=1.84). The average length of the participant’s rightarm was 75.71 cm (SD=3.38). 9 out of 14 participants hadprevious experience with weight exercises.

5.7 Results

5.7.1 Statistical Tests

Repeated Measures ANOVA

The Repeated Measures ANOVA test is run on the reactionRepeated MeasuresANOVA test is

applied on the datawith reaction times

times data to find out if there is a significant difference inreaction times among the four Springlet configurations andthe test also checks if there is a significant difference in re-action times between the directions of arm movement (Upand Down).

NULL HYPOTHESES FOR REPEATED MEASURES

ANOVA TEST:H01: There is no significant difference in reaction timesamong the four Springlet configurations.

H02: There is no significant difference in reactiontimes between the directions of arm movement.

H03: There is an interaction between the independentvariables (The Direction and the Springlet Configura-tion), i.e. the direction variable affects the reaction timeof a particular Springlet Configuration and vice-versa.

Definition:Null Hypotheses for

Repeated MeasuresANOVA Test

The preconditions to run the Repeated Measures ANOVAtest are that data must follow a normal distribution andsphericity must not be violated. Shapiro–Wilk test for Nor-mality and Mauchly’s Sphericity test show that the in-

5.7 Results 47

put data satisfies these constraints and hence the RepeatedMeasures ANOVA test can be run.

MEANING OF SPHERICITY IN TERMS OF REACTION

TIMES DATA:Sphericity: Sphericity is not violated when the variancesin terms of reaction times between all the possible groupsof Springlet configurations are nearly the same [Yatani].

Definition:Meaning ofSphericity in terms ofReaction Times Data

The Repeated Measures ANOVA runs once for the data No significantdifference is found inreaction timesamong Springletconfigurations

with reaction times in seconds for the four Springlet config-urations in both the directions of the arm movement (Fig-ure 5.4). The level of significance (α) for this test is 0.05. Thetest shows that there is no significant difference (p= 0.86)between the reaction times among the four Springlet con-figurations. There is also no significant difference (p= 0.31)between reaction times with respect to the direction of armmovement. Additionally, no interaction effect (p= 0.66) be-tween the independent variables (Direction and Springletconfiguration) is observed. Hence, H01, H02, and H03 can-not be rejected.

Figure 5.4: Graph denoting the reaction times in seconds(Y-axis) for the four Springlet configurations (X-axis). Theorange bars denote reaction times for upwards arm move-ment while the green bars denote the reaction times fordownwards arm movement

48 5 User Study

Friedman Test

The Friedman test is run on the Likert Scale data containingFriedman test isapplied on the data

with Likert Scalequestions

ratings for various questions regarding noticeability, un-derstandability, comfort, arm-movement motivation for allthe four Springlet configurations. The test checks if thereis a significant difference between the ratings for one ques-tion with respect to different Springlet configurations. Thistest is run 9 times for the 9 questions on the Likert ScaleQuestionnaire shown in B.3.

NULL HYPOTHESES FOR FRIEDMAN TEST:H04: There is no significant difference in Likert Scale rat-ings between all the four Springlet configurations.

Definition:Null Hypotheses for

Friedman Test

The level of significance (α) for this test is 0.05. There wereSignificantdifferences were

foundsignificant differences observed in ratings for 4 questionsout of 9 (Figure 5.5) and hence H04 can be rejected for these4 questions. The resulting p-values from the Friedman testare shown in C.1. There were significant differences be-tween the following Likert Scale ratings:

1. Noticeability of the lower Springlet.

2. Motivation to move the arm when the Springlet acti-vates.

3. Understanding of the arm movement when the lowerSpringlet activates.

4. Effort required to sync arm movements.

Friedman-Nemenyi Test

Friedman-Nemenyi test is run as a post-hoc test to identifyFriedman-Nemenyitest shows significant

differences amongtwo pairs of Springlet

configurations

the pairs of Springlet configurations where the significantdifference in Likert Scale ratings lie. The level of signifi-cance (α) for this test is 0.05. Though the Friedman Testshows significant differences for 4 Likert Scale questions,

5.7 Results 49

Figure 5.5: Graphs showing the visualization of Likert Scale ratings for the 4 ques-tions with significant differences. Each graph denotes the different Springlet con-figurations along the X-axis and the distribution of ratings for a Springlet configura-tion by the total number of participants along the Y-axis. Graphs for other questionsare shown in C.2

the post-hoc tests show significant differences among twopairs of Springlet configurations only for the question re-garding “understanding of the arm movement” when thelower Springlet activates.

50 5 User Study

5.7.2 Rankings

At the end of the study, each participant was asked to rankthe Springlet configuration from 1 (best) to 4 (worst). Theresults of the ranking are:

1. TFA, BUA configuration (best)TFA, BUA is the bestconfiguration based

on the rankings 2. TUA, BUA configuration

3. TFA, BFA configuration

4. TUA, BFA configuration (worst)

Figure 5.6: Graph denoting the rankings from 1 (best) to4 (worst) along the X-axis and the distribution of a rankamong different Springlet configurations by the total num-ber of participants along the Y-axis.

5.7.3 Observations

The following observations are made from the statisticaltests:

1. There is no significant difference in reaction timesNo significantdifference in reactiontimes but significant

differences observedin four Likert Scale

questions

among the different Springlet configurations.

2. There is a significant difference in the ratings of thequestion that asks if it was hard to feel the bottomSpringlet (p= 0.01).

5.7 Results 51

3. There is a significant difference in the ratings of thequestion that asks if there was any motivation tomove the arm when the Springlet activated (p= 0.03).

4. There is a significant difference in the ratings of thequestion that asks if there was any difficulty in sync-ing arm movements (p= 0.02).

5. There is a significant difference in the ratings of thequestion that asks if the participant could understandthat he needs to move the arm downwards when thelower springlet gets activated (p= 0.002). Also, theFriedman-Nemenyi test identified significant differ-ences only for this question.

6. The Friedman-Nemenyi test identified significant dif-ferences in terms of “understandability of the lowerarm movement” in the ratings between:

(a) TFA, BFA configuration and TFA, BUA configu-ration (p= 0.02).

(b) TFA, BFA configuration and TUA, BUA config-uration (p= 0.02).

5.7.4 Discussion

The main goal of the study was to check if Springlets Springlets triggermovement cuescan trigger movement cues. A significant difference in

the mean ratings of the question regarding “motivation tomake an arm movement” and the mean ratings of all theSpringlet configurations hints at the presence of movementcues.

However, the movement cues were perceived differently Participants hadmixed reactions fordifferent Springletconfigurations

among the participants. Many participants expressed dur-ing the study that the bottom Springlet was not quite no-ticeable when placed on the forearm. This was most likelybecause the Springlets were very close to the elbow andit was observed among participants that the elbow areawas less sensitive to pinches than other areas of the arm.On the contrary, many participants also felt that movementcues were especially present in the TFA, BUA configuration

52 5 User Study

(“Cannot ignore the feeling. It is mild but still makes me movemy arm: P7”).

The next goal of this study was to find if there is a signifi-cant difference among the Springlet configurations in termsof multiple performance factors. The result of the Fried-man test on the Likert scale data shows a significant dif-ference in the ratings of the question that asks if the par-ticipant could understand that he needs to move the armdownwards when the lower Springlet gets activated. OutSignificant difference

in Likert Scaleratings were mainly

due to the lowerperformance of

bottom Springlet onthe forearm

of the observations made during the study, this significantdifference was likely due to the lower performance of theSpringlet when placed on the forearm. The significant dif-ferences in other Likert Scale questions are also most likelydue to the same reason which is hinted by lower mean rat-ings for the Springlet configurations that incorporate thebottom Springlet on the forearm (Figure 5.5).

Moreover, the significant differences in the Friedman-Significant differencein post-hoc tests

confirm the lowerperformance of

bottom Springlet onforearm

Nemenyi test and the observations from the study alsopoint that the participants better perceive and understandthe downward arm movement when the bottom Springletis placed on the upper arm rather than the forearm. This isalso supported by the higher mean ratings of the Springletsconfigurations where the bottom Springlet is placed on theupper arm (Figure 5.5).

While there are no significant differences in the Friedman-Forearm is a betterchoice than upper

arm for the topSpringlet

Nemenyi test that compares the upper arm and forearmpositions for the upper Springlet, the user studies and thehigher mean ratings on the Likert scale hint at better per-ception and understandability of the upper arm movementwhen the upper Springlet is placed on the forearm ratherthan the upper arm.

Interestingly, correlation tests revealed a positive correla-Correlation testssuggest higher

reaction time forlonger arm lengths

tion (r= 0.768) between the arm length and the reaction timein the TFA, BFA configuration (p= 0.001). However, thiswas found to be to inline with the theory that people withlonger arms have higher reaction times [Brown et al., 2018].

5.8 Conclusion 53

5.8 Conclusion

As there is no significant difference between reaction times, H1 cannot berejected as reactiontimes do not differsignificantly

H1 cannot be rejected. Hence, the reaction times do not dif-fer a lot among the different Springlet variants. However,variations are observed between the Springlet variants con-cerning other factors.

The result of the Friedman test on the Likert scale datashows a significant difference in 4 questions (noticeability,motivation, understandability, and effort to sync) out of 9questions. From the discussion, it is evident that the signif- H0 and the H2 can be

rejectedicant differences in all questions are mainly due to low sen-sitivity of the bottom Springlet when placed on the forearmand the bottom Springlet performs better when placed onthe upper arm. As the Likert scale ratings have significantdifferences for the questions related to “motivation to makean arm movement” and “understandability of the move-ment”, we reject the H0 and the H2 hypotheses.

Further, the discussion also hints towards the fact the up- TFA, BUA is the bestconfiguration toguide armmovements for theBiceps exercise

per Springlet performs better on the forearm than the up-per arm. Hence combining the results from the statisticaltests, the hints from the study observations and the rank-ings, we conclude that the TFA, BUA configuration worksbest in terms of guiding the arm movement for the Bicepsexercise.

55

Chapter 6

Summary and Futurework

This chapter summarizes the thesis and proposes the pos-sibilities of further work in terms of improving the existingsystem, extending the system and finding new applicationareas.

6.1 Summary and Contributions

This thesis explored the usage of SMA-based tactile actua- We talked aboutvarious applicationsand shifted the focustowards tactileimplementations

tors in application areas such as Stroke Rehabilitation, Hap-tic Feedback in VR, Haptic Navigation, and Weight Train-ing Guide. In the Related Work chapter, the focus shiftedfrom traditional implementations to cutaneous tactile im-plementations in these applications which in turn depictedsome advantages of the tactile implementations in the di-rectional and temporal aspects of the interaction.

After conducting an evaluation of the four application areas We studied theWeight Trainingapplication in detail

and checking their feasibility with Springlets, the “WeightTraining Guide” application was chosen to be explored indetail as the design of Springlets was a good fit in enablingHaptic Guidance of exercise movements while supportingthe temporal aspects of the exercise movement.

56 6 Summary and Future work

After evaluating several iterative designs for the PincherShort andWide-Base Design

on the upper armand forearm

performed betterthan other designs

and positions

Springlet to function as an exercise guide, the “Short andWide Base Design” was chosen as it covered a wide areaon the forearm, also covering the area which is near to theelbow joint. This led to an efficient operation when com-pared to other designs. In the next step, the positions onthe arm where the Pincher Springlets could be placed wereevaluated and the upper-arm and upper-forearm positionswere the best in terms of generating a cue for the verticalarm movement.

The statistical tests conducted after the user study foundNo difference wasfound in terms of

reaction times butdifferences were

found in terms of 4performance factors

no significant difference in reaction times among the fourSpringlet configurations. However, they revealed signifi-cant differences among the four Springlets in the way theywere perceived and understood. Gathering informationfrom further tests and observation during the user stud-ies, we concluded that the forearm region near the elbowwas less sensible and hence was the reason for the signifi-cant differences. The observations and tests also pointed to-wards the fact that the upper Springlet functions best whenplaced on the forearm and the lower Springlet functionswell when placed on the upper arm (TFA, BUA Springletconfiguration).

6.2 Future Work

This thesis lays a foundation on tactile guidance techniquesThe thesis lays afoundation that can

be extendedto train exercise movements. Hence, this work can befurther extended to check the feasibility of more exercisemovements, improve the SMA-based actuator for strongermovement cues, and explore more application areas for theSMA-based actuator.

6.2.1 More Exercise Movements

This thesis chose the Biceps Exercise movement to studyExercise movementsinvolve skin stretch

and contractionthe usage of SMA-based actuators to work as a WeightTraining Guide. The SMA-based actuators are able to

6.2 Future Work 57

guide arm movements because of the skin contraction andstretching that occurs naturally near the elbow joint. How-ever, skin contraction and stretching also typically happensat other joints in the body. Hence, the foundation of thiswork can be used to explore the feasibility of more exercisemovements using Springlets.

6.2.2 Strong and Quick Actuation

Although, the strength and speed of Springlets were suffi- There is a need forstronger and fasteractuation in somescenarios

cient to guide the Biceps exercise movement in most par-ticipants, exploring other exercise movements may requirea faster and stronger skin actuation which is currently notpossible due to the technological limitations of the Shape-memory alloy. Further research can look into improving thetechnology of the shape-memory alloy or exploring similarmethods to achieve a natural but also a strong and fast ac-tuation.

6.2.3 More Applications

This thesis explored the usage of SMA-based actuators for More applicationscan be exploredbased on HapticGuidance

four application areas. Further research could look formore applications that can build upon the Tactile Guidancetechnique explained in this work. The ideas brought for-ward in this thesis can be combined with existing ideas toexplore newer applications that ideally leverage the advan-tages of Springlets over other types of on-skin actuators.

59

Appendix A

Breadboard Layout

Figure A.1: Circuit Diagram (Created using Fritzing)

61

Appendix B

Study Consent andQuestionnaire

B.1 Informed Study Consent

B.2 General Questionnaire

B.3 Likert Scale Questionnaire

Informed Consent Form

Evaluating the relation between skin contraction techniques and body movements.

PRINCIPAL INVESTIGATOR Ajay Kumar Jha

Media Computing Group

RWTH Aachen University

Email: [email protected]

Purpose of the study: The goal of this study is to investigate the relation between skin contraction techniques and body movements. In order to do this, Springlets will be stuck to the arm of the participant and the resulting movements will be observed.

Procedure: Before the study, the participants are asked to fill out a questionnaire with some information about themselves. The study will be conducted in five rounds each round lasting about 2 minutes where in each round, the springlets will be stuck to a different position on the arm. The springlets will be stuck to the participants arm by the investigator and the springlets will be controlled by a microcontroller. The springlets will constantly contract and relax the skin during the study. After the end of each round the participants will be asked to fill a short questionnaire. This study will last about 35 minutes to complete.

Risks/Discomfort: The electrical components draw very less current and are well insulated. The spray that will be used to stick the springlets onto the skin is especially made for skin and is safe. Any discomfort or rash feeling on the skin can be immediately reported to the investigator. Springlets are shape memory alloys and they generate minor heat. If the heat causes any discomfort or becomes unbearable, it can be reported. In any of above cases, if the participant reports a problem, the study will be aborted immediately.

Benefits: The results of this study will help to understand the relation between skin contraction techniques and body movements.

Alternatives to Participation: Participation in this study is voluntary. You are free to withdraw or discontinue the participation.

Cost and Compensation: Participation in this study will involve no cost to you. There will be drinks provided for you during the participation.

Confidentiality: All information collected during the study period will be kept strictly confidential. You will be identified through identification numbers. No publications or reports from this project will include identifying information on any participant. If you agree to join this study, please sign your name below.

_____ I have read and understood the information on this form.

_____ I have had the information on this form explained to me.

Participant’s Name Participant’s Signature Date

Principal Investigator Date

If you have any questions regarding this study, please contact Ajay Kumar Jha at email: [email protected]

1

Experiment Questionnaire

Evaluation of the relation between skin-contraction

techniques and body movements.

Information about you:

1. How old are you? _________________

2. Please tick your sex Male Female N.A.

3. Have you ever been to the Gym or have any experience with weight

exercises? Yes No

4. Do you have any skin abnormalities such as low-sensitive or hypersensitive

skin? Yes No

5. If yes, name the skin abnormality? _____________________

6. Arm Length: _____________________

2

How well do you agree with the following statements? (First

variant)

1. It was hard to feel the springlet on the top:

Don’t agree Fully agree

1 2 3 4 5

2. It was hard to feel the springlet near the elbow:


1 2 3 4 5

3. I did not feel the need to move my arm:


1 2 3 4 5

4. The springlet on the top was comfortable:


1 2 3 4 5

5. The springlet near the elbow was comfortable:


1 2 3 4 5

3

How well do you agree with the following statements? (First

variant)

6. It was hard to understand what the springlets where trying to communicate

to me:


1 2 3 4 5

7. I could easily understand that I need to pull my arm when the upper springlet

got activated:


1 2 3 4 5

8. I could easily understand that I need to release my arm when the lower (near

elbow) springlet got activated:


1 2 3 4 5

9. It was difficult to sync the arm movements with the springlets:


1 2 3 4 5

** These 9 questions are repeated for all the 4 Springlet configurations **

4

Ranking

1. How would you rank the variants?

Variant Rank (1 to 4)

2. Do you have any thoughts about the suitability of Springlets to learn exercise

movements? Also, suggestions for improvement or other comments (if any)

67

Appendix C

p-values and Graphs

C.1 p-values from the Friedman Test

Question p-value

1 It was hard to feel the springlet on the top 0.805

2 It was hard to feel the springlet near the elbow 0.019

3 I did not feel the need to move my arm 0.036

4 The springlet on the top was comfortable 0.614

5 The springlet near the elbow was comfortable 0.790

6 It was hard to understand what the springlets where trying to commu-nicate to me

0.519

7 I could easily understand that I need to pull my arm when the upperspringlet got activated

0.265

8 I could easily understand that I need to release my arm when the lower(near elbow) springlet got activated

0.002

9 It was difficult to sync the arm movements with the springlets 0.021

Table C.1: p-values from the Friedman test performed onLikert Scale data (Calculated using RealStatistics).

68 C p-values and Graphs

C.2 Graphs for Likert Scale Questionswith No Significant Differences

Figure C.1: Graphs showing the visualization of Likert Scale ratings for the ques-tions that did not have significant differences. Each graph denotes the differ-ent Springlet configurations along the X-axis and the distribution of ratings for aSpringlet configuration by the total number of participants along the Y-axis

69

Bibliography

K. Bark, J. W. Wheeler, S. Premakumar, and M. R. Cutkosky.Comparison of skin stretch and vibrotactile stimulationfor feedback of proprioceptive information. In 2008 Sym-posium on Haptic Interfaces for Virtual Environment and Tele-operator Systems, pages 71–78, March 2008. doi: 10.1109/HAPTICS.2008.4479916.

Ato Brown, Leonard Derkyi-Kwarteng, Christian Kobi-naAckom, Queenstar Kwakwa, Rachael Atta, and VeraTakyi. Is simple reaction time influenced by arm span,neck length and arm span to height ratios of individu-als? Academia Anatomica International, 4, 02 2018. doi:10.21276/aanat.2018.4.1.7.

Volkert Buchmann, Stephen Violich, Mark Billinghurst, andAndy Cockburn. Fingartips: gesture based direct manip-ulation in augmented reality. In Proceedings of the 2nd in-ternational conference on Computer graphics and interactivetechniques in Australasia and South East Asia, pages 212–221. ACM, 2004.

Keng-Hao Chang, Mike Y Chen, and John Canny. Trackingfree-weight exercises. In International Conference on Ubiq-uitous Computing, pages 19–37. Springer, 2007.

A. Cohen, L. Goodman, S. Keaveney, C. Keogh, and K. Dil-lenburger. Sustaining a caring relationship at a dis-tance: Can haptics and 3d technologies overcome thedeficits in 2d direct synchronous video based communi-cation? In 2017 23rd International Conference on VirtualSystem Multimedia (VSMM), pages 1–6, Oct 2017. doi:10.1109/VSMM.2017.8346290.

D. F. Collins, K. M. Refshauge, G. Todd, and S. C. Gande-

70 Bibliography

via. Cutaneous receptors contribute to kinesthesia at theindex finger, elbow, and knee. Journal of Neurophysiol-ogy, 94(3):1699–1706, 2005. doi: 10.1152/jn.00191.2005.URL https://doi.org/10.1152/jn.00191.2005.PMID: 15917323.

David Dobbelstein, Philipp Henzler, and Enrico Rukzio.Unconstrained pedestrian navigation based on vibro-tactile feedback around the wristband of a smart-watch. In Proceedings of the 2016 CHI Conference Ex-tended Abstracts on Human Factors in Computing Sys-tems, CHI EA ’16, pages 2439–2445, New York, NY,USA, 2016. ACM. ISBN 978-1-4503-4082-3. doi: 10.1145/2851581.2892292. URL http://doi.acm.org/10.1145/2851581.2892292.

Sevgi Ertan, Clare Lee, Abigail Willets, Hong Tan, and AlexPentland. A wearable haptic navigation guidance sys-tem. In Digest of Papers. Second International Symposiumon Wearable Computers (Cat. No. 98EX215), pages 164–165.IEEE, 1998.

David Feygin, Madeleine Keehner, and R Tendick. Hapticguidance: Experimental evaluation of a haptic trainingmethod for a perceptual motor skill. In Proceedings 10thSymposium on Haptic Interfaces for Virtual Environment andTeleoperator Systems. HAPTICS 2002, pages 40–47. IEEE,2002.

Nur Al-huda Hamdan, Adrian Wagner, Simon Voelker,Jurgen Steimle, and Jan Borchers. Springlets: Expres-sive, flexible and silent on-skin tactile interfaces. In Pro-ceedings of the 2019 CHI Conference on Human Factors inComputing Systems, CHI ’19, pages 488:1–488:14, NewYork, NY, USA, 2019. ACM. ISBN 978-1-4503-5970-2.doi: 10.1145/3290605.3300718. URL http://doi.acm.org/10.1145/3290605.3300718.

NSK Ho, KY Tong, XL Hu, KL Fung, XJ Wei, W Rong, andEA Susanto. An emg-driven exoskeleton hand robotictraining device on chronic stroke subjects: task trainingsystem for stroke rehabilitation. In 2011 IEEE interna-tional conference on rehabilitation robotics, pages 1–5. IEEE,2011.

https://doi.org/10.1152/jn.00191.2005

http://doi.acm.org/10.1145/2851581.2892292

http://doi.acm.org/10.1145/2851581.2892292

http://doi.acm.org/10.1145/3290605.3300718

http://doi.acm.org/10.1145/3290605.3300718

Bibliography 71

David Jack, Rares Boian, Alma S Merians, MarilynTremaine, Grigore C Burdea, Sergei V Adamovich,Michael Recce, and Howard Poizner. Virtual reality-enhanced stroke rehabilitation. IEEE transactions onneural systems and rehabilitation engineering, 9(3):308–318,2001.

Sumner L Norman, Andrew J Doxon, Brian T Gleeson, andWilliam R Provancher. Planar hand motion guidance us-ing fingertip skin-stretch feedback. IEEE transactions onhaptics, 7(2):121–130, 2014.

Martin Pielot, Benjamin Poppinga, Wilko Heuten, and Su-sanne Boll. Pocketnavigator: Studying tactile navigationsystems in-situ. In Proceedings of the SIGCHI Conferenceon Human Factors in Computing Systems, CHI ’12, pages3131–3140, New York, NY, USA, 2012. ACM. ISBN 978-1-4503-1015-4. doi: 10.1145/2207676.2208728. URL http://doi.acm.org/10.1145/2207676.2208728.

Fazlay Rabbi, Taiwoo Park, Biyi Fang, Mi Zhang, andYoungki Lee. When virtual reality meets internet ofthings in the gym: Enabling immersive interactive ma-chine exercises. Proc. ACM Interact. Mob. Wearable Ubiqui-tous Technol., 2(2):78:1–78:21, July 2018. ISSN 2474-9567.doi: 10.1145/3214281. URL http://doi.acm.org/10.1145/3214281.

RealStatistics. The data analysis for this paper was gener-ated using the real statistics resource pack software (re-lease 6.7). copyright (2013 – 2020) charles zaiontz. URLwww.real-statistics.com.

Samuel B Schorr and Allison M Okamura. Fingertip tactiledevices for virtual object manipulation and exploration.In Proceedings of the 2017 CHI Conference on Human Factorsin Computing Systems, pages 3115–3119. ACM, 2017.

Haska Steltenpohl and Anders Bouwer. Vibrobelt: Tac-tile navigation support for cyclists. In Proceedings ofthe 2013 International Conference on Intelligent User In-terfaces, IUI ’13, pages 417–426, New York, NY, USA,2013. ACM. ISBN 978-1-4503-1965-2. doi: 10.1145/2449396.2449450. URL http://doi.acm.org/10.1145/2449396.2449450.

http://doi.acm.org/10.1145/2207676.2208728

http://doi.acm.org/10.1145/2207676.2208728

http://doi.acm.org/10.1145/3214281

http://doi.acm.org/10.1145/3214281

www.real-statistics.com

http://doi.acm.org/10.1145/2449396.2449450

http://doi.acm.org/10.1145/2449396.2449450

72 Bibliography

Esther MJ Steultjens, Joost Dekker, Lex M Bouter, Jos CMvan de Nes, Edith HC Cup, Cornelia HM van den Ende,Francesco Landi, and Roberto Bernabei. Occupationaltherapy for stroke patients: When, where, and how?Stroke, 34(3):676–687, 2003.

Karanbir S. Toor, Ameet S. Toor, Charlton M. Smith, andAlexander G. Orozco. Oli, your weight-training as-sistant. In Proceedings of the 2017 CHI Conference Ex-tended Abstracts on Human Factors in Computing Sys-tems, CHI EA ’17, pages 184–189, New York, NY, USA,2017. ACM. ISBN 978-1-4503-4656-6. doi: 10.1145/3027063.3048429. URL http://doi.acm.org/10.1145/3027063.3048429.

Shunki Yamashita, Ryota Ishida, Arihide Takahashi,Hsueh-Han Wu, Hironori Mitake, and ShoichiHasegawa. Gum-gum shooting: Inducing a senseof arm elongation via forearm skin-stretch and thechange in the center of gravity. In ACM SIGGRAPH 2018Emerging Technologies, SIGGRAPH ’18, pages 8:1–8:2,New York, NY, USA, 2018. ACM. ISBN 978-1-4503-5810-1. doi: 10.1145/3214907.3214909. URL http://doi.acm.org/10.1145/3214907.3214909.

Koji Yatani. Koji yatani’s course webpage. URLhttp://yatani.jp/teaching/doku.php?id=hcistats:anova#dokuwiki__top.

http://doi.acm.org/10.1145/3027063.3048429

http://doi.acm.org/10.1145/3027063.3048429

http://doi.acm.org/10.1145/3214907.3214909

http://doi.acm.org/10.1145/3214907.3214909

http://yatani.jp/teaching/doku.php?id=hcistats:anova#dokuwiki__top

http://yatani.jp/teaching/doku.php?id=hcistats:anova#dokuwiki__top

73

Index

abbrv . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . see abbreviation

evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56–57

Typeset February 28, 2020

Springlet Apps: Applications of On-Skin Shape Memory Alloy · 2020. 3. 1. · by Ajay Kumar Jha...

Documents

Transcript of Springlet Apps: Applications of On-Skin Shape Memory Alloy · 2020. 3. 1. · by Ajay Kumar Jha...