UNDERSTANDING THE EFFECTS OF POSITIVE AND NEGATIVE …

The Pennsylvania State University

The Graduate School

College of Engineering

UNDERSTANDING THE EFFECTS OF POSITIVE AND NEGATIVE AFFECT ON PERCEIVED USABILITY

A Dissertation in

Industrial and Manufacturing Engineering

By

Maria Angelica Velazquez

2010 Maria Angelica Velazquez

Submitted in Partial Fulfillment of the Requirements

for the Degree of

Doctor of Philosophy

December 2010

ii

The dissertation of Maria Angelica Velazquez was reviewed and approved* by the following:

Gül E. Kremer

Associate Professor of Industrial Engineering

Dissertation Co-Adviser

Co-Chair of Committee

Andris Freivalds

Professor of Industrial Engineering

Dissertation Co-Adviser

Co-Chair of Committee

David Nembhard

Associate Professor of Industrial Engineering

Frank E. Ritter

Professor of Information Sciences and Technology

Paul Griffin

Professor, Department Head of the Industrial Engineering Department

*Signatures are on file in the graduate school.

iii

Abstract

Emotions (or affective states) are mental and physiological states associated with a variety of cognitive

activities such as memory, learning, decision making, appraisal, and judgment. This research investigates

the ways in which emotions upon initial interaction with the product affect user perceptions of product

usability. Our hypothesis is that the perceived product usability will be higher in the presence of positive

emotion than negative emotion.

The effect of emotion on perceived usability was tested for different controlled usability scenarios. The

experiments were conducted using smart phones and software applications within the smart phones.

Two smart phones with different input modalities (keypad and touch screen) were used in combination

with two software applications, one of which was designed with good usability and the other with poor

usability.

A total of four experiments were conducted. In the first experiment, the emotion elicitation strategy was

validated. The second and third experiments explored the effect of emotion on perceived usability for a

single product, and provided data on experiment parameters to be used in the fourth experiment. The

fourth experiment explored the effect of emotion on perceived usability of smart phone/software

systems and the software applications in isolation. A randomized complete block design was used.

The results from experiment 1 showed that our emotion elicitation strategy is effective for both positive

and negative emotions. Experiments 2, 3, and 4 proved that there is a statistically significant effect of

user affective states on perceived product usability. Overall, the results suggest the existence of carry-

over effects of affective states on usability appraisals. Thus, for the specific products, scenarios, and

emotions elicited, user affective states prior to product interactions were determinant factors of

perceived product usability. In general, users in positive affective states believed that products were

better. Likewise, users in negative affective states believed that products were worse. No significant

iv

differences were observed for the effect of emotion for different smart phones and software

applications.

The results of this study provide insight into the importance of previous emotional states on product

interactions. This information will help designers to create products that are able to overcome negative

emotions and elicit positive ones that improve product interactions, and by extension, quality of life.

v

Table of Contents

List of Figures ............................................................................................................................................... ix

List of Tables ................................................................................................................................................ xi

Acknowledgements .....................................................................................................................................xiii

Chapter 1 ....................................................................................................................................................... 1

Introduction: Emotion and Perceived Usability ............................................................................................ 1

1.1 Emotion, Cognition, and Usability ...................................................................................................... 1

1.2 Research Questions ............................................................................................................................ 5

1.3 Problem Relevance and Significance .................................................................................................. 8

1.4 Smart Phones and Software as Research Applications ..................................................................... 10

1.5 Document Outline ............................................................................................................................. 11

Chapter 2 ..................................................................................................................................................... 12

Literature Review ........................................................................................................................................ 12

2.1 What Is Usability? ............................................................................................................................. 13

2.1.2 Usability and Emotions .............................................................................................................. 15

2.2 An Overview of Emotion ................................................................................................................... 16

2.2.1 What is Emotion? ....................................................................................................................... 16

2.2.2 Understanding the Nature of Emotion ...................................................................................... 19

2.2.3 Emotions, Moods, and Affective States ..................................................................................... 20

2.2.4 Emotion and Cognitive Activities ............................................................................................... 22

2.2.5 Emotions and Evaluative Judgment ........................................................................................... 23

2.2.6 Evaluative Judgment of Products and Affect ............................................................................. 26

2.2.7 Emotion Transfer and Effect on Product Perception ................................................................. 29

2.2.8 Models of Product Appraisal and Usability ................................................................................ 31

2.2.9 Positive Affect, Negative Affect and Perceived Usability ........................................................... 34

2.2.10 Negative Affect - Frustration ................................................................................................... 35

vi

2.2.11 Positive Affect - Happiness ...................................................................................................... 36

2.3 Self-Report Methods for Emotion ..................................................................................................... 37

2.4 Self-Report Methods for Usability .................................................................................................... 42

2.5 Summary ........................................................................................................................................... 43

Chapter 3 ..................................................................................................................................................... 45

Methodology ............................................................................................................................................... 45

3.1 Experiment 1: Validation of the Emotion Elicitation Strategy .......................................................... 47

3.1.1 Materials for Experiment 1 ........................................................................................................ 49

3.1.2 Experimental Design for Experiment 1 ...................................................................................... 50

3.1.3 Procedures for Experiment 1 ..................................................................................................... 53

3.2 Experiment 2: Testing the Effect of Emotion on Usability ................................................................ 56

3.2.1 Materials for Experiment 2 ........................................................................................................ 58


3.2.3 Procedures for Experiment 2 ..................................................................................................... 62

3.3 Experiment 3: Testing the Effect of Days between Sessions in Perceived Usability ........................ 64

3.4 Experiment 4: Testing the Effect of Emotion on Different Levels of Designed (Controlled) Usability

......................................................................................................................................................... 66

3.4.1 Materials .................................................................................................................................... 70


3.4.3 Procedure for Experiment 4 ....................................................................................................... 76

3.5 Summary ........................................................................................................................................... 77

Chapter 4 ..................................................................................................................................................... 78

Results from Experiments 1, 2 and 3 .......................................................................................................... 78

4.1 Results of Experiment 1 .................................................................................................................... 78

4.1.1 Emotion Elicitation ..................................................................................................................... 79

4.1.2 Stimuli and Other Variables ....................................................................................................... 82

vii

4.1.3 Summary .................................................................................................................................... 86


4.2.1 Emotion Elicitation Verification ................................................................................................. 87

4.2.2 Effect of Emotion on Usability ................................................................................................... 88

4.2.3 Summary .................................................................................................................................... 90


4.3.1 Emotion Elicitation Verification ................................................................................................. 91

4.3.2 Effect of Emotion and the Time between Sessions on Perceived Usability ............................... 92

4.3.3 Summary .................................................................................................................................... 96

4.4 Summary of the Results for Experiments 1, 2, and 3 ........................................................................ 96

Chapter 5 ..................................................................................................................................................... 97

Results and Discussion for Experiment 4 .................................................................................................... 97

5.1 Hypotheses Related to Emotion Elicitation ...................................................................................... 99

5.2 Hypotheses Related to the Effects of Affective States on Usability ............................................... 103

5.3 Hypothesis Related to the Effect of Affective State on Perceived Software Usability ................... 114

5.5 Hypotheses Related to Other Usability Metrics ............................................................................. 121

5.5.1 Perceived Ease of Use .............................................................................................................. 122

5.5.2 Task Completion Rate .............................................................................................................. 123

5.5 Summary ......................................................................................................................................... 125

Chapter 6 ................................................................................................................................................... 127

Conclusions and Future Work ................................................................................................................... 127

References ................................................................................................................................................ 132

Appendix A: Emotion Elicitation Strategy ................................................................................................ 139

Appendix B: Materials Used in Experiment 1 .......................................................................................... 143

B.1 Instruction Sheet Given to the Participants .................................................................................... 143

B.2 Pre-Experiment Questionnaire ....................................................................................................... 145

viii

B.3 Myers-Briggs Type Indicator (MBTI) ............................................................................................... 146

B.4 Positive and Negative Affect Schedule (PANAS): ............................................................................ 147

B.5 Affect Grid (AG) ............................................................................................................................... 147

B.6 Self-Assessment Mannequin (SAM) ................................................................................................ 148

B.7 Post-Experiment Questionnaire ..................................................................................................... 150

Appendix C: Materials Used in Experiment 2 and Experiment 3 ............................................................. 151

C.1 Pre-Experiment Questionnaire ....................................................................................................... 151

C.2 Emotion Questionnaires and Emotion Elicitation Materials .......................................................... 153

C.3 System Usability Scale (SUS) ........................................................................................................... 154

C.4 IBM Post-Study System Usability Questionnaire (PSSUQ) .............................................................. 155

C.5 Post-Experiment questionnaire ...................................................................................................... 157

C.6 Description of Environment and List of Tasks................................................................................. 159

Appendix D: Materials Used in Experiment 4 .......................................................................................... 163

D.1 Pre-Experiment Questionnaire ....................................................................................................... 163

D.1.a About Your Personality ............................................................................................................ 164

D.2 Affective Questionnaire .................................................................................................................. 165

D.3 Emotion Elicitation Materials ......................................................................................................... 165

D.4 Usability Questionnaire .................................................................................................................. 165

D.5 Software Usability Questionnaire – Adapted from the Software Usability Measurement Inventory

(SUMI) ............................................................................................................................................ 165

D.6 Perceived Ease of Use ..................................................................................................................... 166

D.7 List of Tasks ..................................................................................................................................... 167

Appendix E: Consent Forms ..................................................................................................................... 169

Appendix F: Experiment Data ................................................................................................................... 179

ix

List of Figures

Figure 2.1 Neurological structure of emotion (Brave and Nass 2003). ...................................................... 20

Figure 2.2 Conception of the memory-based theories of affect (Clore and Parrott 1991). ....................... 24

Figure 2.3 Conception of the affect-as-information theory (Clore and Parrott 1991). .............................. 25

Figure 2.4 The CUE model for product appraisal (Thüring and Mahlke 2007). .......................................... 31

Figure 2.5 Usability model with EQ and HQ (Hassenzahl, Platz et al. 2000). .............................................. 33

Figure 2.6 Circumplex model of affect (Russell 1980). .............................................................................. 34

Figure 2.7 Affect grid and circumplex model of affect (Russell 1980). ....................................................... 39

Figure 2.8 The Self-Assessment Mannequin (Morris 1995). ....................................................................... 40

Figure 3.2 Sequence of events for experiment 1. ....................................................................................... 48

Figure 3.3 Sampling procedure for experiment 1. ...................................................................................... 51

Figure 3.4 Sequence of tasks for experiment 1. ......................................................................................... 54

Figure 3.5 Events and task sequence for experiment 2. ............................................................................. 57

Figure 3.6 HTC Pure smart phone. .............................................................................................................. 59

Figure 3.7 Adarian Money software program. ........................................................................................... 59

Figure 3.8 Sampling procedure. .................................................................................................................. 61

Figure 3.9 Task sequence for experiment 2. ............................................................................................... 63

Figure 3.10 Assignment of participants to the experimental conditions. .................................................. 67

Figure 3.11 Phones used in experiment 4................................................................................................... 70

Figure 3.13 Block design for experiment 4. ................................................................................................ 74

Figure 3.14 Sequence of tasks for experiment 4. ....................................................................................... 76

Figure 5.1 Histogram of the initial and the final positive affect. ................................................................ 99

Figure 5.2 Histogram of the initial and the final negative affect. ............................................................... 99

Figure 5.3 Interval plot and boxplot of perceived usability scores for positive and negative affect. ....... 107

Figure 5.4 Interval plot and boxplot of perceived usability scores for smart phone 1 and smart phone 2.

.................................................................................................................................................................. 108

Figure 5.5 Interval plot and boxplot of perceived usability scores for software 1 and software 2. ......... 108

Figure 5.6 Residual plots for perceived usability. ..................................................................................... 111

Figure 5.7 Main effects plot for perceived usability. ................................................................................ 113

Figure 5.8 Interactions plot for perceived usability. ................................................................................. 113

Figure 5.9 Guidelines for values of Cronbach’s Alpha. ............................................................................. 115

Figure 5.10 Interval plot and boxplot of perceived software usability based on emotion. ..................... 116

Figure 5.11 Interval plot and boxplot of perceived software usability based on smart phone used. ...... 116

Figure 5.12 Interval plot and boxplot of perceived software usability based on software used. ............ 117

Figure 5.13 Residual plots for perceived usability. ................................................................................... 119

Figure 5.14 Interactions plot for perceived software usability. ................................................................ 120

x

Figure 5.15 Interactions plot for perceived software usability. ................................................................ 121

Figure A.1 Circuit schematic. .................................................................................................................... 139

Figure A.2 Panel showing power in the circuit components. ................................................................... 140

Figure B.1 Affect Grid ................................................................................................................................ 148

Figure B.2 Self-Assessment Mannequin. .................................................................................................. 149

xi

List of Tables

Table 2.1 Multiple Definitions of Usability ................................................................................................. 14

Table 2.1 (Continued) Multiple Definitions of Usability ............................................................................. 15

Table 2.2 Most Relevant Definitions and Views of Emotion....................................................................... 17

Table 3.1 Variables and Their Classifications for Experiment 1 .................................................................. 52


Table 3.3 Treatment Levels and Factor Combinations for Experiment 4 .................................................. 67


Table 4.1 Wilcoxon Signed Rank Test Results for Change in Emotion (Positive Affect) ............................. 80

Table 4.2 Wilcoxon Signed Rank Test Results for Change in Emotion (Negative Affect) ........................... 80

Table 4.3 Descriptive Statistics for the Change in Emotion Score Values .................................................. 83

Table 4.4 P-values for the Wilcoxon Signed Rank Test for Each Measurement Method ........................... 83

Table 4.5 Post-Experiment Questionnaire .................................................................................................. 85

Table 4.6 Descriptive Statistics for the PANAS Scores in Experiment 2 ...................................................... 87

Table 4.7 Mean Values of Reported Perceived Usability ............................................................................ 88

Table 4.8 ANOVA Results for Experiment 2 ................................................................................................ 89

Table 4.9 ANOVA Results for SUS and IBM Usability Questionnaire and Previous Experience .................. 90

Table 4.10 Descriptive Statistics for the PANAS Scores in Experiment 3 .................................................... 92

Table 4.11 Descriptive Statistics for Experiment 3 ..................................................................................... 94

Table 4.12 Regression Results for Experiment 3 (R-Sq. 25.9%) .................................................................. 94

Table 5.1 Summary of Participant Characteristics ...................................................................................... 98

Table 5.2 Descriptive Statistics for the PANAS Scores for Experiment 4 .................................................. 100

Table 5.3 Wilcoxon Signed Rank Test Results for Change in Emotion for Experiment 4 .......................... 101

Table 5.4 Regression Results for Changes in Positive and Negative Affect as Response Variables ......... 102

Table 5.6 Description of Previous Participant Experiences ...................................................................... 104

Table 5.7 Descriptive Statistics for SUS Scores ......................................................................................... 106

Table 5.8 Descriptive statistics for SUS Scores for Each Level of Emotion, Smart Phone and Software .. 106

Table 5.9 SUS Score ANOVA for Experiment 4 .......................................................................................... 110

Table 5.10 Estimates of Power and Cohen’s Coefficients for Emotion, Smart Phone, and Software ...... 112

Table 5.11 Descriptive Statistics for the Software Usability Scale ............................................................ 115

Table 5.12 Descriptive Statistics for the Software Usability Scale, by Emotion ....................................... 115

Table 5.13 ANOVA for Software Usability Score, Experiment 4 ............................................................... 118

Table 5.14 Estimates of Power for Emotion and Software ....................................................................... 120

Table 5.14 Descriptive Statistics for Perceived Ease of Use ..................................................................... 122

Table 5.15 ANOVA for Perceived Ease of Use, Experiment 4 ................................................................... 123

Table 5.16 Descriptive Statistics for Task Completion Rate ...................................................................... 124

Table 5.17 ANOVA for Task Completion Rate, Experiment 4 ................................................................... 124

Table F.1 Data Gathered in Experiment 1 (Participants 1 to 15) .............................................................. 179

Table F.2 Data Gathered in Experiment 1 (Participants 16 to 30) ............................................................ 180

xii

Table F.3 Data Gathered in Experiment 2 (Participants 1 to 14) .............................................................. 181

Table F.4 Data Gathered in Experiment 2 (Participants 15 to 27) ............................................................ 182

Table F.5 Data Gathered in Experiment 3 ................................................................................................. 183

Table F.6 Data Gathered in Experiment 4 (Participants 1-20) .................................................................. 184

Table F.7 Data Gathered in Experiment 4 (Participants 21-40) ................................................................ 185

xiii

Acknowledgements

I would like to express my gratitude to my advisors, Dr. Andris Freivalds and Dr. Gül E. Kremer for all the

support, the ideas and the encouragement to complete this research work. Thank you for your valuable

help in the development of this research and the revisions of this manuscript. Thank you Dr. Freivalds

for your guidance and continuous support. Thank you Dr. Kremer, for believing in me and for your

devotion to make this research work possible.

I would also like to thank my committee members, Dr. Frank Ritter and Dr. David Nembhard for their

time, and for their helpful comments and suggestions. Thank you Dr. Frank Ritter for going beyond your

obligations as an outside member, for your detail review of this work and for being an outstanding

mentor and a role model.

Thanks to Dr. Richard Koubek for his encouragement and guidance during the very early stages of this

research work. I am also very grateful of Dr. Freivalds and the Industrial and Manufacturing Engineering

department for the monetary contribution for the last experiment. Thanks to AT&T Laboratories for

providing the smart phones to be used in my experiments. I will always be grateful of all the participants

for their time and collaboration in the data collection process.

I would like to thank the AT&T Labs Fellowship Program (ALFP) and my mentor Dr. Robert Bell for all his

advices and encouragement. Also, I would like to thank the National Science Foundation Alliance for

Graduate Studies and the Professorate program, the Alfred P. Sloan Foundation, and the Industrial and

Manufacturing Engineering department at Penn State for their support towards completing my research

work and the Ph.D. requirements.

xiv

I also want to thank Dr. Suzanne Adair and the Office of Graduate Educational Equity Program (OGEEP)

for their continuous support and advice. I thank Dr. Luciano Castillo and Dr. Jose Holguin-Veras for their

encouragement and valuable guidance. Last but not least, I would like to thank my husband David

Claudio for his support during this process, his mentorship and faith in me.

xv

Dedication

To God, for being my source of wisdom and my refuge when I needed the most. Thank you my dear

Father, this work is all for you.

To my beloved husband David Claudio, my inspiration and my perfect complement. I am very glad the

universe conspired to have us together during this process, and forever.

To my dear father Mariano and my beloved mother Martha. From you I learned to work with dedication

and love. Thank you for believing in me and for devoting your lives to make me be what I am now.

To my brothers, sisters, nephews, nieces, for bringing true happiness to my life, always.

To my in –laws, and my family. Your help and love made a difference in my steps towards this goal.

To all my friends, the ones that are far and the ones near. Thank you for your encouragement and faith

in me.

To Dr. Frank Ritter for being a professor, a mentor, and a friend when I needed each one of them. God

bless you always.

And last but not least, to my dear son Angel David. Thank you for your unconditional love. You opened

my eyes to what is truly important in life.

1

Chapter 1

Introduction: Emotion and Perceived Usability

1.1 Emotion, Cognition, and Usability

Emotions, or affective states, are mental and physiological states associated with a variety of feelings,

thoughts, and behaviors. They are experienced subjectively by every individual and form an inherent

part of human life. Emotions have been studied since the classical era by philosophers such as Plato and

Aristotle, who offered opposing theories on the role of emotions. Emotion is “one of the most central

and pervasive aspects of human experience” (Eich, Kihlstrom et al. 2000). Although vast research exists,

debate continues over the definition, origin and purpose of emotion and its role in cognition (Lazarus

1991).

Early researchers visualized emotions as mental states that interfere with rational thinking and cognitive

activities, emphasizing a separation between rational and emotional behavior. In the past, emotions

were considered non-logical and irrelevant for cognitive processing. Thus, acting emotionally meant

acting irrationally and with poor judgment (Picard 1997). This stigmatic view of emotions as non-rational

agents dominated psychological thinking until the past few decades (Dalgleish and Power 1999).

However, over the past 30 years the research community has become more interested in studying the

interaction of emotions and cognition and how they influence rationality and human performance.

Several experiments have demonstrated the effects of emotions on memory (Damasio 1994), problem

solving (Simon 1967), human performance and learning (Power and Dalgleish 2008), and perceptions of

ambiguous stimuli and the likelihood of events (Eich, Kihlstrom et al. 2000).

2

In recent decades, psychological research has also studied the way emotions impact human perception.

Several studies have demonstrated that our perception of the world is influenced by minor changes in

our affective states (Schwarz 1997). In general, negative emotions promote negative perceptions, while

positive emotions facilitate positive perceptions and promote tolerance. Happiness and joy are thought

to make tasks easier by providing a more positive perspective of the world, while negative emotions

promote the reverse effects.

When a user interacts with a product he or she is potentially using cognitive skills — especially learning,

decision making, and evaluative judgment. Since these cognitive processes are affected by emotions, it

stands to reason that emotions may influence interactions with products and the way they are

perceived. Consequently, human-product interactions and judgments regarding product usability may

be affected by user emotions. A happy user is more prone to continue using a given product and to rate

the product well. As a user’s negative emotions increase, product desirability decreases, and the effort

required to use the product increases (Hazlett and Benedek 2007).

Recently, the emotional side of human-product interaction has been acknowledged. Researchers in

human factors, psychology, human-computer interaction, product design, and marketing have studied

aesthetics, hedonic quality, and emotion. However, most of these studies focus on emotions generated

while using the product and evoked by product characteristics. While promising, these approaches are

limited to emotions generated as a consequence of human-product interaction. Thus, they neglect

affective states prior to product interaction.

In addition, the effect of affective states on perceived product usability has not been studied. Usability is

the extent to which a product meets user needs. ISO 9241-11 (International Organization for

Standardization (ISO) Webpage) defines usability as the extent to which a product is effective, efficient

and satisfying in a particular context of use. Usability assessments are crucial for the development of

3

successful and satisfying products. Lately, traditional usability has been seen as dehumanizing, as it

simplifies human subjects into cognitive information processors (Green and Jordan 1999). As a result,

some recent definitions of usability have acknowledged the importance of affective states resulting from

product interactions (Hassenzahl 2001; Thuring and Mahlke 2007).

Perceptions of usability may be influenced by emotional states prior to product use. Accordingly, this

research investigates the ways in which affective states upon initial interactions with products affect

usability perceptions. For this purpose, smart phones and related software applications will be used in

the experiment sessions. These technologies are common yet innovative products with usability

improvement opportunities.

In general, the focal research question is as follows: Is a product’s perceived usability better in the

presence of positive emotion than negative emotion? More specifically, we will test the hypothesis that,

for different scenarios of usability, whenever a user experiences positive affect before interacting with a

product, its perceived usability will be higher than in the presence of negative affect.

Usability is the touchstone of human-computer interaction. It is an ever-expanding concept that

continues to evolve as knowledge about design considerations and human-product interaction in

various contexts continues to expand (Carroll and Mentis 2008). A significant part of product evaluation

relies on usability to assess product effectiveness and efficiency. Affect covers a host of attitudinal,

emotional and mood-related elements of an experience. Even though emotions exist in all human

endeavors, they have been overlooked in studies of usability (Dillon 2002). We believe that usability

engineers must understand how emotional states affect perceived usability. The findings of this research

should not only facilitate such understanding, but expand the overall concept of usability.

Currently, studies of emotional states are executed from the perspectives of psychology, marketing, and

consumer research. Thus, research findings are targeted for use in these communities. For instance, a

4

marketing study may explore the effect of emotion on product desirability, or a consumer research

study may focus on product choice and the effect of emotion on product use decisions. Usability differs

greatly from these metrics, as it focuses on effectiveness and efficiency of use. To the best of our

knowledge, the effect of emotion on product effectiveness and efficiency has not been studied.

Most of the studies pertaining to the effect of emotion on product perception and choice are not

executed from engineering and product design perspectives; in general, the experiments lack effective

and reliable experimental design procedures. In addition, such studies distribute and apply their findings

to other areas that are not closely related to usability and human-computer interaction. Thus, most of

the time usability professionals are unaware of such effects. This research should provide insights on

specific design features and the way they enhance or diminish affect.

Finally, there is a whole new school of thought within human factors that argues for the enhancement of

usability and human factors fields by considering user emotions in product interaction (Green and

Jordan 1999; Jordan 2000; Dillon 2002; Khalid and Helander 2006; Desmet and Hekkerd 2007; Helander,

Peng et al. 2007; Hekkert and Schifferstein 2008). Members of this new school of thought, known as

new human factors, argue that ignoring the role of human emotions in product interaction ignores what

makes us human. This research supports this new definition of human factors.

The effect of emotion on perceived usability will be assessed in a laboratory experiment. In this

experiment, participants will be induced with positive and negative affective states, and then will be

asked to interact with a smart phone and related software. The participants will be asked to rate

product usability after their interactions.

Positive and negative affect will be induced through computer interactions. Verbal feedback and praise

will be used to generate happiness, and random intentional delays in the computer program will

generate frustration. Usability assessments for the smart phone/software system will be compared

5

based on positive and negative affect. The results of this study should suggest carry-over effects of

affective states for usability appraisals.

1.2 Research Questions

This research intends to investigate the effect of emotional states prior to interaction with products on

perceptions of usability. The following problem statements summarize the research questions to be

investigated:

1. Is perceived usability affected by user affective states prior to product interactions?

The effect of emotional states on the generation of evaluative judgment has been previously studied.

For instance, mood has been shown to affect consumer behavior in many ways. Prior research studies

on advertising, perceived risk, and new product evaluation support the idea that negative or positive

emotions can color unrelated consumer judgments, such as evaluations of brand extensions and actual

choices (Fedorikhin and Cole 2004); Gorn, Goldberg et al. 1993; Chaudhuri 1990). Moods have also been

suggested to be sources of information in the evaluative judgment of products and choices (White and

McFarland 2009).

However, these studies focus on general perceptions of products and the circumstances under which

they are accepted by users, providing insights from consumer research and marketing perspectives.

Such perspectives focus mostly on product attractiveness and attachment, often neglecting the usability

evaluation of products. It is a common practice to rely on the human factors and human-computer

interaction disciplines to assess product usability.

Usability is a very important part of new product evaluation. Although the effects of emotion on new

product development have been studied, effects on the perceived usability of such products have never

been assessed. Evidence is strong that emotions impact new product evaluation; we believe this can be

6

explained by the effects of emotional states on perceived usability. The human factors community

would benefit greatly from knowing if such an effect exists and to what extent. The results of this study

will provide insight into understanding product evaluation and assessment.

This brings us to our second research question:

2. In the presence of emotional effects, what specific factors of perceived usability are affected, and

how?

It is important to investigate which factors of usability are affected, and how. Is a system’s perceived

usefulness affected by previous emotional states? Are the mental, physical, and temporal demands of

product tasks affected by emotional state? Clarifying such effects would allow us to provide more

specific insights into the particular aspects of a product’s usability that are affected. These insights

would provide guidance for product modification.

Specific products are used in contexts requiring design strategies aimed at controlling or neutralizing

emotional states. For example, an emergency response system should be designed to minimize stress

and promote fast and accurate decision making. In such circumstances, it is important to neutralize

affective states. Affective states may change tolerance levels, bias decision making, and impede optimal

performance. Understanding such effects is critical to obtaining accurate assessments of usability.

3. How do the effects of emotional states on perceived usability vary as a function of software and

hardware features?

In the presence of such effects, we believe it is important to understand which design characteristics

promote or mitigate them. Is the effect of emotions on perceived usability greater for a particular design

feature? Or, are the effects of emotional states mitigated through design improvements? Would a user

in a negative mood perceive better usability for one design feature over another?

7

For this purpose, we plan to use two different smart phones and two versions of data management

software. In the experiment, there will be four smart phone-software combinations. With this

experiment we expect to gather data on the different effects of emotions on perceived usability.

In addition to the main problems presented, we are also interested in understanding the following sub-

problems:

4. How does emotion elicitation vary as a function of experimental parameters and individual

factors?

Often, researchers and designers study emotions and their effects on humans by manipulating

emotional states with elicitation techniques. Many theories on the effects of emotions on human

cognitive activities are derived from these studies. Therefore, it is important to have effective and

reliable methods for emotion elicitation in experimental settings.

Although there are multiple techniques for emotion generation, in general, they do not explain the

degree to which emotions may vary as a function of experimental parameters. Also, they do not explain

how individual factors such as personality type and goal importance may affect the emotions generated.

In the process of selecting an emotion elicitation strategy for our experiment, we discovered that there

is a need to investigate the ways in which individual and experimental factors affect which emotions are

elicited. Thus, it is important to understand how to account for such factors in order to create a reliable

and repeatable emotion elicitation strategy.

5. In which ways do experimental parameters (e.g., goal importance, design features, individual

characteristics) affect usability perceptions?

We believe it is appropriate to understand the ways in which usability assessments are affected by

parameters such as the degree of emotion, design features, and user characteristics. Such knowledge

8

would assist designers in understanding the relationships between emotions and cognitive aspects of

the product experience, which may then provide information for comprehending user-product

interactions.

1.3 Problem Relevance and Significance

Perceived usability refers to the degree to which users think a product would be easily implemented for

their purposes, and how well they think product features would support their tasks. Based on the results

of this research, a new conceptual model of usability will be proposed. In this model, emotions will be

considered as a factor affecting product usability. Such a model will be aligned with the emerging view

of human factors where emotions are considered an important part of human-product interaction.

We believe this research will be beneficial to the product design community, as it will make designers

aware of this phenomenon. The results will help them reflect on the importance of affective states

whenever individual differences are considered. The results of this research will support designers’

efforts to create emotionally robust designs. Thus, despite the initial emotions of users, affective states

supporting performance quality and optimal product usage can be assured.

As mentioned previously, many products require a user’s affective state to be controlled. For example,

software systems used in emergency response, airplane cockpits, and air traffic control need to be

designed to minimize anxiety, stress, and frustration, and support timely and accurate decision making.

Also, many products are designed to generate a pre-determined emotional state based on the product’s

characteristics and the context in which the interface is to be used. In many cases, design effectiveness

is not tested for the range of emotions that may be present prior to a user’s interaction with a product.

Thus, there is a risk that existing moods or emotional states will alter the emotions generated during the

interaction.

9

Moreover, if the task to be executed generates emotions, designers should be aware of possible carry-

over effects from previous emotions. These effects may lower emotional thresholds and propitiate the

generation of new and augmented states. These new emotions may affect user performance and

perceptions of the product. Although smart phones and related software applications may not represent

environments with constant stress and cognitive workload, we believe they are nevertheless

appropriate instruments for establishing a baseline for the effect of emotions on perceived usability.

As a result of this study, designers will be made aware of the importance of previous emotional states

and moods to product interaction. This will help them create products capable of overcoming negative

emotions and eliciting positive affects, thereby improving a user’s quality of life. Designers cannot

control user moods as variables; thus, they need to be aware of these effects, and produce robust

designs capable of overcoming them. The expectation is that designers will want happy and satisfied

users who are likely to continue using their products and recommending them to others.

The findings of this study will have implications for marketing practitioners as well. Often, they attempt

to manipulate affective states by, for example, creating uplifting advertisements or running them after

humorous television programs (White and McFarland 2009). But how effective are these techniques in

the presence of pre-conceived positive or negative emotional states? The answer to this question is

unknown.

Our findings may also shed light on product acceptance. In his Technology Acceptance Model (TAM),

Davis (1993) highlights the importance of perceived usefulness and perceived ease of use in the

acceptance of technology products. The TAM provides an informative representation of the mechanisms

by which design choices affect product acceptance. It also specifies the causal relationships among

design features, perceived usefulness, perceived ease of use, attitudes towards using, and actual system

usage (Davis 1993). Based on the TAM, both perceived ease of use and perceived usefulness affect

10

attitudes toward using the product and subsequent product usage. Both constructs are traditionally

assessed in usability engineering. The results of this study may be used to refine predictions of usage

based on specific circumstances where emotional carry-overs may be experienced.

1.4 Smart Phones and Software as Research Applications

Wireless communication is one of the most vibrant areas in the contemporary communication field.

Although active research in this area has been conducted since the 1960s, we have recently seen a

remarkable surge in related literature (Tse and Viswanath 2005). In past decades, the advancement and

innovation of wireless technology motivated the development of wireless devices such as smart phones.

Consequently, data management applications for smart phones have been developed. Increasingly,

mobile professionals want to check e-mail or have easy access to data while they are outside the

traditional office. These professionals rely on wireless data applications and smart phones to do so.

Despite advancements in smart phone technology and software development, these systems are far

from perfect. People frequently complain about the complex nature of such systems and the cognitive

work entailed in using them. While some dislike navigating smart phone operating systems, others find

the speed and capacity of such phones to make calls and run software applications sorely lacking

(Consumer Search Inc. 2010)

In this research, we have selected smart phones and related software applications as the products for

which usability will be appraised under the induced affective states, given the rapid growth and

development of such devices. Considering user emotional states before interaction will provide insight

into how the systems are evaluated and assessed. Further, in this experiment we are interested in using

a product that is commonly used by a wide range of people yet remains novel, with opportunity areas

for usability improvements. We believe that smart phones fulfill these criteria. It will be interesting to

11

understand the extent to which emotional states improve perceptions of a smart phone’s usability and

user performance when interacting with a software application on the device.

1.5 Document Outline

The next chapter provides a literature review on emotions, usability, and the effects of emotional states

on cognitive activities followed by a summary of the methodology and a description of the procedures

for each experiment. The results for the preliminary experiments and the final experiment are then

presented, followed by conclusions and recommendations in the last chapter.

12

Chapter 2

Literature Review

Usability is the extent to which products and systems are functional and practical to the user. Originally,

usability was developed as a way to assess and monitor employee productivity in the workplace.

Currently, the concept has been extended to evaluate the degree to which products and services of any

kind satisfy user needs.

Emotions are an important element of human life. They have been studied for thousands of years by

philosophers such as Plato, Aristotle, and Descartes (McDonagh, Hekkert et al. 2004). Recently,

emotions began to be acknowledged in the human factors field as motivators behind perception,

cognition and creativity (Picard 1997). Today, emotions are considered essential to human cognition,

behavior, values, judgment and agency (Love 2002).

Even though emotions have an impact on cognitive processes, traditional usability does not account for

the effects of emotional states on interactions with products. There is sufficient evidence proving that

emotions affect human interactions with products and devices and subsequent perceptions of those

products. Therefore, this research proposes a study of human affective states prior to product use and

their effects on usability perceptions.

The following literature review provides background information on the research problem and related

approaches and theories. This review supports the purpose of this research by highlighting opportunity

areas within both current and past research related to usability, human-computer interaction, emotions,

and product experience.

13

2.1 What Is Usability?

Usability is defined by ISO 9241-11 (International Organization for Standardization (ISO) Webpage) as

the extent to which a product is effective, efficient, and satisfying in a particular context of use. Initially,

usability was a term employed to describe interactive systems that are easy to learn and use, simple,

and effective for task execution. The meaning of usability has continued to change over the past 25

years. Today, usability is enriched with knowledge of human development and cognitive processes

(Carroll and Mentis 2008), yielding a variety of approaches for usability assessments.

Numerous definitions of usability have been proposed, all from different perspectives and in a variety of

contexts. Some of the definitions describe usability as a product acceptance criterion, while others focus

on ease of learning and use and the study of user practices with products. Subjective aspects of user

experiences with products such as attitude, satisfaction and pleasure have been incorporated as well.

Some of the most remarkable definitions of usability are summarized in Table 2.1.

Shackel’s (1991) model of usability describes usability as a dimension of product acceptance along with

utility, likeability and costs. Along the same lines, Nielsen (1993) suggests that usability and utility

influence a product’s usefulness. Departing from a more cognitive approach, ISO 9241 part 11

(International Organization for Standardization (ISO) Webpage) defines usability as quality of use. The

three dimensions of usability are effectiveness, efficiency and satisfaction.

Bevan (1995) defines usability from a user experience perspective. Usability is defined as the extent to

which a product satisfies stated and implied needs when used under specific conditions (Bevan 1995).

While Bevan focuses on user experiences with products, the usability model presented by Keinonen

14

(2007) is based on differentiating product types in order to provide different usability criteria for

automatic products and interactive products.

Table 2.1 Multiple Definitions of Usability

Source Definition Variables / Dimensions Focus

Shackel (1986)

The capability, in human functional terms, to be used easily and effectively to fulfill a specific range of tasks within a specific range of environmental scenarios.

Effectiveness, learnability, flexibility, and attitude

In the context of product acceptance, it is a property of a product or system that is relative to the user, training, tasks and environments.

Norman (1990) Describes how easily a user can understand how a product works and how to use it.

Ease of learning and use

One of the four components of good behavioral design, along with function, understandability and physical feel.

Nielsen (1993) Describes how well a product's functionality is leveraged.

Learnability, efficiency, memorability, errors and satisfaction

A concern of product acceptability that applies to all aspects of a system with which humans interact.

Bevan (1995)

Quality in use. The extent to which specific goals can be achieved with effectiveness, efficiency and satisfaction by specific users carrying out specific tasks in specific environments.

Effectiveness, efficiency, productive period, learning, satisfaction and cognitive workload

Focus on the quality of the user-product interaction and the quality of the experience with the product.

ISO 9241-11 (1998)

The extent to which a product is effective, efficient and satisfying in a particular context of use.

Effectiveness, efficiency and satisfaction

Determined by the ease of use and the extent to which functional properties and other quality characteristics meet user needs in a specific context.

Han et al. (2001)

The degree to which users are satisfied with the product with respect to both the performance and the image/impression.

Performance dimensions and image/impression dimensions

Relates to the context of electronic consumer products; emphasizes the subjective aspects, image and impression.

Rosson and Carroll (2002)

The quality of a system with respect to ease of learning, ease of use, and user satisfaction.

Ease of learning, ease of use, and user satisfaction

It emphasizes understanding the activities of the users in the real environment, which is beyond task analysis, involving detailed studies of work practices, roles and concepts.

15

Table 2.1 (Continued) Multiple Definitions of Usability

Source Definition Variables / Dimensions Focus

Keinonen (2007) The product's suitability to its use, defined by important constituents.

Utility, capacity, subjective pleasure, reliability, ease of use, easy to remember, error rate.

Different usability criteria for automatic and interactive products.

Pew and Mavor (2008)

Ease of use and the extent to which functional properties and other quality characteristics meet user needs in a specific context of use.

Usability as a product quality characteristic and as a parameter satisfying user needs.

Determined by product characteristics and the context of use.

Most of the usability models agree on the inclusion of dimensions such as ease of use, ease of learning,

effectiveness, efficiency, and user satisfaction or user attitude towards the product or system. In this

research we will consider overall usability, software usability, task completion rate, and perceived ease

of use as dimensions of usability.

2.1.2 Usability and Emotions

Emotions are an inherent part of human life that can improve or impede the way users interact with

products. However, they have been ignored by most usability approaches. There are many possible

explanations for why emotional states have been overlooked in usability studies. First, emotions have a

stigma of being non-scientific, irrational, inappropriate, and even embarrassing. Introductory psychology

text books have described emotions as responses that are disorganized, visceral and resulting from a

lack of effective adjustment (Picard 1997). This negative bias has prevented many scientists from

investigating the role of emotions in their studies. Consequently, emotions have not been considered

important to scholars who study usability.

16

Second, emotions are fuzzy and subjective by nature. They are difficult to measure because there are

few or no standards on the psychology of emotions; consequently, there are limited scientific methods

for measuring, analyzing, and comparing them. Despite their idiosyncratic nature, the scientific

community should develop methods for the objective measurement of emotions. More importantly, we

need methods for detecting trends in emotional responses (Picard 1997).

Third, usability was originally created to assess employee productivity, adaptation, and performance in

the workplace. Therefore, traditional approaches and methods focus mostly on productivity aspects

such as efficiency and effectiveness. These approaches disregard employees’ emotions, as they are not

thought to be directly related to workplace tasks. Given the fact that currently the concept of usability

has been expanded to include the evaluation of products and systems of common use, scholars and

practitioners should acknowledge and account for the roles emotions play in product interactions.

2.2 An Overview of Emotion

Emotions add meaning and enrich almost all human experiences. Traditionally in Western society,

individuals have been expected to overcome emotional demands and experiences and to think and act

rationally. Recently, many psychologists have argued that it is impossible for a person to have thoughts

or execute actions without emotional involvement (Brave and Nass 2003). As a result, the human-

computer interaction field is evolving towards the inclusion of affective elements. Therefore, we believe

that it is necessary to review the most relevant literature on emotions, presented below.

2.2.1 What is Emotion?

Emotion is a mental, physiological, and idiosyncratic state that may be associated with a variety of

behaviors, moods, temperaments, and personalities. Since the twentieth century, the definition of

emotion has been at the center of psychological debate. Two different schools of thought, the physical

and the cognitive, propose diverse approaches to its definition (Desmet 2003). The physical view of

17

emotions, first stated by William James, affirms that emotions result from “our feeling of body changes”

(Lange and James 1922). The cognitive view suggests that emotions result from decision making and

evaluative processes.

Based on these two approaches, numerous definitions and theories of emotion have emerged,

combining both cognitive and physical aspects. For example, John Dewey (1894) suggested that

emotions result from the evaluation of discrepancies between expectations and visceral events

produced by the state of the world. Along those lines, numerous researchers and psychologists have

suggested a relationship between cognitive processes and emotions. Table 2.2 summarizes some

common psychological definitions of emotion.

Table 2.2 Most Relevant Definitions and Views of Emotion

Source View of Emotion

Simon (1967)

A response to sudden intense stimuli from the environment, stemming from the arousal of the autonomic nervous system, which serves as an interruption mechanism that allows the processor to respond to urgent needs in real time.

Izard (1977) A complex process with neurophysiologic, neuromuscular, and phenomenological aspects that affect all facets of individuals in many different ways.

Averill (1980) Transitory social role that includes an individual's appraisal of a situation, interpreted as passion rather than action.

Fridja (1986) Occurrence of non-instrumental behaviors, physiological changes, and evaluative, subject-related experiences, as evoked by external or mental events, and primarily by the significance of such events.

Lazarus (1991) Organized psycho-physiological reactions to knowledge about ongoing relationships with the environment, the quality and intensity of which depend on subjective evaluations or cognitive appraisals.

Picard (1997) Physical and cognitive elements closely related to rational and intelligent behavior and essential for human communication.

Rolls (2000) States elicited by rewards and punishments, including changes in them. Emotions can be produced by the delivery, omission, or termination of rewarding or punishing stimuli.

18

Simon’s theory of human information processing relates motivation and emotional behavior with

adaptive, prioritization and survival strategies of information processing. When emotion-producing

stimuli are persistent, they may become disruptive, sometimes producing non-adaptive behavior.

Emotions are considered by Izard (1977) to be processes occurring in the brain and the nervous system.

In Izard’s differential emotions theory, emotions constitute the primary motivational system of human

beings.

Considering the social and cultural aspect of emotions, Averill (1980) defines emotions in terms of social

roles. He argues that to understand emotions, it is necessary to execute an analysis of the current

situation at a social level. On the other hand, Frijda (1986) defines emotions as the awareness of

situations as relevant, urgent and meaningful with respect to ways of dealing with current

circumstances. Stated another way, emotions represent a readiness to either change or maintain

relationships with the environment.

Emotions have also been defined as states elicited by reinforcers in the form of rewards and

punishments. Rolls’ (2000) theory of brain development suggests that brains are designed around

reward and punishment evaluation systems. Thus, genes use such systems to produce appropriate

behaviors for increasing fitness (Rolls 2000). Emotions result from appraisal processes, which assess

whether something is rewarding or punishing based on a person’s values.

Lazarus (1991) refers to the appraisal process as one marked by evaluating personal status with respect

to goals. Thus, emotions are constantly changing, given the dynamic nature of person-environment

relationships. His cognitive-relational-motivational theory of emotion suggests that emotions are a

function of what is important to a person and his or her hierarchy of goals.

In her book Affective Computing, Picard (1997) explores the relationship between computers and

emotions and suggests that emotions play a significant role in rational and intelligent behavior.

19

Emotional skills, especially those related to the recognition and expression of emotions, are essential for

natural communication with humans.

There is no definite, formal definition of emotion that is accepted across disciplines and may be

generalized to all cases. “Nobody can claim to know the answer to basic questions in emotion theory

such as what are emotions, what causes them, and why we (sometimes) dislike them” (Picard 1997). The

answers to these questions are openly debated, with limited evidence for each approach. In addition,

theorists and researchers have used the term “emotion” to imply diverse processes and meanings. This

contributes further to theoretical conflicts over the nature of emotions, their functions, their activation

processes, and their roles in daily activities (Izard 2007).

In general, two aspects of emotion are common across almost all definitions. First, emotions are

reactions to events that are relevant to the goals, needs, and concerns of an individual. Second,

emotions encompass physiological, affective, behavioral, and cognitive components (Brave and Nass

2003).

2.2.2 Understanding the Nature of Emotion

There are multiple frameworks that attempt to explain emotion generation in the human brain. From

the available frameworks, we have chosen LeDoux’s neuropsychological model on emotion generation.

Beyond being one of the most scientific and widely accepted frameworks (Brave and Nass 2003), it

provides a simple, understandable and straightforward explanation of the emotion generation process.

In LeDoux’s model, the three brain regions that are critical to emotion generation are the thalamus, the

limbic system, and the cortex. Sensory input from the environment is received by the thalamus, which

processes information and sends it simultaneously to the limbic system and cortex. The limbic system is

composed of the hypothalamus, the hippocampus in the temporal lobe, and the amygdale. This system

20

evaluates the need/goal relevance of the inputs and determines if they are important to the individual.

If the inputs correspond to an important goal/concern, the limbic system sends appropriate signals to

the body’s physiological response system and to the cortex. The cortex then biases attention and other

cognitive processes. Figure 2.1 depicts the different pathways for emotion generation based on

LeDoux’s model.

Figure 2.1 Neurological structure of emotion (Brave and Nass 2003).

The limbic system is believed to be the seat of emotion, memory and attention. It helps to determine

the valence of an emotion and salience of events. It also contributes to the flexibility, unpredictability

and creativity of human behavior (Picard 1997).

2.2.3 Emotions, Moods, and Affective States

Most researchers agree on a distinction between emotions and moods, temperaments, sentiments,

affective traits, and other related affective constructs. Some researchers suggest that the duration of an

experience, differences in antecedent events, and biological processes involved are good differentiators

(Ekman 1994; Kagan 1994; Bower and Forgas 2000).

Emotions have the properties of a reaction; they have an identifiable cause and also usually include a

spasmodic, intense experience of short duration. Usually the person experiencing the emotion is aware

of such an experience (Bower and Forgas 2000). Moods, on the other hand, tend to be more subtle,

Thalamus

Cortex

Limbic

System

Physiological

Responses

21

long-lasting, and less intense. A moods is more like a frame of mind, non-specific and not linked to a

clear-cut and consciously available cognitive representation (Bower and Forgas 2000). A person may not

be aware of his mood until someone brings it to his or her attention. Typically, it is not possible to relate

a mood to a specific person, object, or event.

According to Frijda (1986), emotions are intentional and always related to objects. Moods are

unintentional and not related to particular objects. Lazarus (1991) makes a similar distinction, suggesting

that emotions are immediate strategies for adapting to the environment. Moods, on the other hand,

result from appraising life’s existential background.

Emotions are functional, as they influence action and prepare the body for an appropriate immediate

response. Moods, in contrast, influence cognitive strategies and processing over a longer term (Brave

and Nass 2003). Moods may be seen as a background internal filter through which both internal and

external events are appraised. Moods tend to influence which emotions are experienced, lowering the

activation threshold for mood-related emotions (Brave and Nass 2003).

Among emotions, moods, sentiments and feelings, emotions are the most relevant to product

experience. Emotions imply a one-to-one relationship between an affective state and a particular object

(Desmet and Hekkert 2000). Other expressions of affect (i.e., moods, sentiments and feelings) do not

involve specific objects.

In the context of this research, a distinction among emotions and moods is not relevant. Our purpose is

to provide evidence of the effects of pre-conceived affective states on product perceptions. Thus, we

will refer to conditions of emotions and moods as affective or emotional states interchangeably.

Affective state is a generic label that encompasses both moods and emotions (Stickney 2009). Affect

may be defined as an intensive and relatively short-lasting emotional state (Khalid 2006). It

22

encompasses moods, emotions, and feelings, and represents a consumer’s psychological response to a

product. Affective states are fundamental to the development of beliefs, values and judgment (Khalid

2006). Elicited emotions may influence a user’s judgment, while resultant moods may influence the

cognitive processes related to decision making.

The emotions generated in our study will have an identifiable cause. They are also expected to be

functional and to bias action, preparing the body for an appropriate immediate response. Also, they

should influence cognitive strategies and serve as affective filters for appraising external events -

specifically, product usability.

As stated previously, for the purposes of this study, we refer to emotions and affective states

interchangeably. This will avoid issues related to terminology, which in this case will be irrelevant to the

interpretation of the results.

2.2.4 Emotion and Cognitive Activities

In his experiments in the fields of neurology and psychology, Damasio (1994) studied the ability of

patients with impaired emotional abilities to act rationally and make intelligent decisions. He proposed

that neurological, hormonal, and physiological processes operate simultaneously in a structured

architecture. Such systems generate the cognitive and affective mechanisms essential for human

thought, judgment, creativity, feelings, actions, and decision making (Damasio 1994). Results also show

that patients with synesthesia, a condition that produces heightened perceptual experiences, have high

levels of episodic limbic activity, further indicating that the limbic system, which is the “home of

emotions,” plays a significant role in perception (Damasio 1994). Thus, it is hypothesized that there is no

perception without attachment of valence to memory (Picard 1997; Eich, Kihlstrom et al. 2000). Other

studies show that moods bias the perceptions of both ambiguous stimuli and the likelihood of events.

23

Norman (2004) described emotions as being a necessary part of life that affect the way humans feel,

behave, and think. The author asserted that emotions are “inseparable from cognition” (Norman 2004).

Based on Norman’s theory, cognition provides an interpretation of the world while emotions provide

more information to make quick decisions about the world. The affective system is judgmental and

assigns positive and negative valence to the environment.

Emotions have important effects, including directing a person’s attention and focus to objects and

situations that are critical to satisfying needs and achieving goals (Brave and Nass 2003). Conscious

processing is dominated mostly by thoughts with emotional relevance. The importance of the situation

or object determines the level of arousal and focus (Clore and Gasper 2000). Affect is also thought to

influence information processing strategies. In problem solving situations, positive affect enables a

person to analyze a situation through relational processing. Negative affect promotes more item-specific

processing (Clore and Hudtsinger 2007).

In summary, the effects of positive and negative emotions on cognitive processes such as decision

making, attention and information processing have been studied. Overall, being happy or sad influences

both the content and the style of thoughts (Clore and Hudtsinger 2007), information processing

strategies, attention, decision making, and perceptions of events and targets.

2.2.5 Emotions and Evaluative Judgment

Different theories have been created to explain the effects of emotions on consumer judgment,

thinking, and decision making. Within these theories, two deserve to be highlighted: (1) memory-based

theories, and (2) inferential models (Fedorikhin and Cole 2004).

Memory-based theories state that emotion primes similar or related nodes. In cases when one affective

node is activated, other events that are connected to the affective node will be activated as well. Thus, a

24

person’s affective state will bias the direction of generated cognitive activities, predisposing judgment

(Bower and Forgas 2000). Based on memory theories, emotions and moods tend to bias thoughts in the

direction of the mood. Thus, a person in a good or positive mood will likely judge both product

interactions and the resulting work more positively, disregarding any emotions related to product

experiences (Clore, Wyer et al. 2001).

Other theories, such as mood-congruency theory, state that positively valenced material is more

accessible in memory when individuals are in positive moods rather than negative. Similarly, negative

material is more accessible when people are in negative moods. Thus, target stimuli are considered

more favorably under positive moods than negative moods (Schwarz and Clore 1988). A depiction of the

effect of memory-based theories on cognitive processes is shown in Figure 2.2.

The accessibility of mood-related material theory states that in judgment situations, the material related

to a specific emotion is more accessible to memory when a person is experiencing that emotion (Isen

1987). Thus, when judging an object, a person who is in a positive mood will more quickly access

dimensions of memories that belong to the positive mood category.

Figure 2.2 Conception of the memory-based theories of affect (Clore and Parrott 1991).

25

These approaches have been grouped together and are referred to as affect-priming theories. These

theories view moods as “primers” that directly influence the formation of judgments. When generating

a judgment, mood serves to prime specific dimensions of the target to be judged.

Memory-based theories have been contrasted with feeling-based explanations of mood effects (Clore,

Wyer et al. 2001). Inferential models suggest that current moods are used as information for assessing

and judging targets. An example of this type of theory is the “affect-as-information” hypothesis. This

hypothesis states that affective feelings influence judgments when they are experienced as reactions to

what is being judged (Clore, Wyer et al. 2001). A depiction of the effect of mood-as-information on

cognitive processes is shown in Figure 2.3.

Figure 2.3 Conception of the affect-as-information theory (Clore and Parrott 1991).

An alternative account of the effects of mood on evaluative judgments was proposed by Schwarz and

Clore (1988) as the “How do I feel about it?” theory. Within this framework, the affective state is

assumed to be used as information by the person experiencing it. There is evidence that internally-

generated information is processed just like any other information. Thus, when individuals feel either

positive or negative emotions, they use their perceived affective reactions as relevant information when

making evaluative judgments (Gorn, Goldberg et al. 1993). When people use this heuristic, they mistake

their moods for feelings about the stimulus (Fedorikhin and Cole 2004).

26

Both memory-based and inference-based models agree on the fact that emotion affects human

perception and evaluative judgment. Memory-based approaches relate cognitive activities to affective

states, while inferential models suggest that current moods are used as information for generating

judgments. These theories suggest that moods and emotions conceived prior to interactions with

targets may significantly affect the way such targets are perceived and judged. Although these theories

differ in their explanations of such effects, both theories agree on the effects of emotions on judgment

and the direction of such effects.

2.2.6 Evaluative Judgment of Products and Affect

Emotion has been shown to affect consumer behavior in many ways. Empirical research supports the

idea that negative or positive moods created by one event can change unrelated consumer judgments

(Fedorikhin and Cole 2004). In this section, we summarize some of the most remarkable studies on the

effect of emotion on product judgment.

Another study investigated the effects of mood states on product evaluation (Gorn, Goldberg et al.

1993). In this study, mood states were viewed as potential biasing cues in the evaluation of new

products, consistent with Schwarz and Clore’s “How do I feel about it?” theory. Participants in the study

were asked to evaluate the performance of a new set of stereo speakers while they listened to music

that would make them feel either good or bad. The participants in the positive mood condition

evaluated the speakers more favorably than those in the negative mood condition. The participants

were not aware of their moods; they also did not know that global ratings of the speakers would be

used to assess the products. The results of the study suggest that emotions have an inferential effect on

product appraisal despite limitations related to the use of global ratings (i.e., comparing and rating

products) for product evaluation.

27

From a marketing perspective, research in advertising has found that the information provided in

advertising is processed differently by participants in positive or negative moods. Also, advertisement-

altered consumer moods affect attitudes towards the brands advertised and purchase intentions (Gorn,

Goldberg et al. 1993). The results of these studies point to the prominent role of emotion on advertising

effectiveness and attitudes towards products.

In regard to risk associated with products, studies suggest that emotional factors account for a

significant and substantial portion of the variance in perceived risk, even after the effects of rational

factors are considered (Chaudhuri 1990). Products that are capable of eliciting undesirable feelings are

associated with greater perceived risks. Products that potentially generate pleasant feelings are

associated with lower risk (Chaudhuri 1990). Thus, the perceived risk of products is directly associated

with a user’s affective state.

These studies prove the importance of emotion in product appraisal from the perspective of marketing

and consumer research. The circumstances under which such effects are present have been studied as

well. For instance, White and McFarland (2009) studied the effects of focused attention to moods and

perceptions of mood relevance, finding that they are the strongest cause of mood-congruent product

evaluation (White and McFarland 2009). The results of the study support the mood-as-information

model, allowing the authors to conclude that the effect of emotion on product appraisal is enhanced

when the consumers are aware of their emotions and use that information for product evaluation.

In a separate study, Fedorikhin and Cole (2004) proved that consumer tendencies to bias judgments

based on their moods are stronger when constructive processing requirements are high. When

consumers construct judgments, they are involved in substantially transforming existing cognitive

representations, rather than reproducing such representations (Forgas 2007). Consumers may minimize

effort and construction by retrieving or using previously conceived perceptions from memory (Fazio and

28

Towles-Schwen 1999). They can also use external information about how to form product evaluations.

When processing strategies require more constructive and generative processing, contextual variables

like mood are more prone to affect judgment (Coupey, Irwin et al. 1998). Bias in the form of affect-as-

information occurs when consumers have limited processing resources or cognitive capacity. Affect-

priming biases occur when consumers have unrestricted processing resources (Fedorikhin and Cole

2004).

Other studies focus on the relationship between individual differences and the effect of emotion on

product appraisal. For instance, Pham (1998) suggested that when consumers rely on their feelings to

make product decisions, such feelings must not only represent a target, but also reflect the importance

of those feelings toward the target. In summary, Pham’s experiments proved that the degree to which

people rely on feelings when making decisions depends on factors affecting: (1) the heuristic value of

the feelings, (2) how well the feelings represent the target, and (3) their perceived relevance.

These studies provide vast evidence for the effects of emotional states on multiple cognitive activities.

The theories derived from these experiments provide the basis and rationale for understanding the

mechanisms involved in such effects. However, these studies focus on aspects such as aesthetics,

product attachment, marketing, and product choice. Usability is a product development standard. It

differs greatly from these metrics, as it focuses on effectiveness and efficiency of use. This research

studies the effect of affective states specifically on perceived usability. Thus, the results of this study will

have direct implications on usability, and (by extension) product design, providing practical knowledge

to usability engineers.

29

2.2.7 Emotion Transfer and Effect on Product Perception

In general, most theories of emotion and cognition agree that stimuli are judged through what is known

as “the filter of mood.” The filter of mood suggests that a user in a good mood will make a more positive

judgment of his work and the interface he or she is using (Brave and Nass 2003).

The Appraisal-Tendency Framework (ATF), developed by Han, Lerner et al. (2007), is a theory that

explains and predicts the influence of specific emotions on consumer decision making. It explains how

and why emotions carry over from past situations to influence present judgments and choices (Han,

Lerner et al. 2007). Within ATF, particular emotions provoke specific cognitive and motivational

mechanisms that are responsible for the effects of such emotions on future judgment and decision

making processes. The ATF postulates that the appraisal tendencies of a specific emotion are “goal-

directed processes that affect future judgment by providing a perceptual lens for the interpretation of

future situations, and of which the person is not necessarily aware” (Lerner and Tiedens 2006). Based on

the ATF principles, emotions of the same valence may provoke different effects on judgment, while

emotions with different valences may provoke similar effects (Lerner and Keltner 2001). Thus, two

emotional states that belong to the same valence (e.g., fear and anger – negative valence) can generate

very different judgmental effects. These effects are related to the fact that emotions are comprised of

cognitive and motivational appraisal strategies that are expressed both at the biological and behavioral

levels. Thus, based on the ATF, the effects of emotion on product judgment are not solely based on the

valences of emotions, but also on the cognitive and motivational circumstances in which emotions are

generated and the biological and behavioral changes presented in the individual when emotions are

experienced.

30

The ATF has been used to understand sequential choices and high-stakes decisions, and has been

extended to consider the interaction of multiple emotions generated by sequences of choices

(Cavanaugh, Bettman et al. 2007).

Affect congruence in cognition and judgment is thought by some researchers to be a context-dependent

phenomenon, suggesting that different situations lead to different emotional effects on cognitive

activities. The Affect Infusion Model (AIM) proposed by Forgas (2001) is an integrative theoretical

framework arguing that the key to understanding affect infusion is to recognize individual tendencies to

adopt different processing strategies in response to different contextual requirements (Forgas 2001).

Thus, AIM proposes that the reason why emotional effects on cognitive processing vary is the adoption

of different information processing strategies, which depend on both the context and the situation. This

explains the presence and absence of affect infusion in cognitive processes by assuming that affective

states interact and inform cognition and judgment, influencing processing strategies and the availability

of cognitive constructs (Forgas 2001). The three major assumptions of the AIM are: (1) affect infusion

depends on which processing strategy is adopted; (2) affect infusion is most likely to occur in situations

requiring constructive, immediately accessible and motivated processing strategies rather than

predetermined ones; and (3) people exert a minimum amount of effort when choosing an appropriate

processing strategy.

Along the same lines, Zillmann (2008) proposed the excitation-transfer theory (ETT) to explain the carry-

over effects of emotions. Based on ETT, emotions provoked by any stimulus can be present even after a

stimulus has disappeared, because it takes some time for the autonomic nervous system to return to its

deactivated state (Brave and Nass 2003). If an emotion is generated before the previous emotional state

decays completely, the activated emotions that remain will combine with the newly-activated emotions;

the previous emotion may be perceived as being part of the current emotion, causing “overly intense

31

affective reactions to subsequent stimuli” (Zillmann 2008). Based on this theory, affective reactions may

not relate to the first emotion experienced.

These theories may explain the observed effects of emotion on product appraisal. However, researchers

thus far have not attempted to validate any of these theories. They are summarized for the

informational purposes and to provide awareness of their existence.

2.2.8 Models of Product Appraisal and Usability

In this section, relevant models of product appraisal are presented. These conceptual models represent

the relationships among the variables encompassing human-product interaction.

Thüring and Mahlke (2007) proposed the components of user experience model (CUE-model) to explain

the product appraisal process. In this model, user experience is defined as a combination of emotions

and perceptions of both instrumental and non-instrumental product qualities. Figure 2.4 depicts the CUE

model.

Figure 2.4 The CUE model for product appraisal (Thuring and Mahlke 2007).

32

In the model, instrumental qualities are related to a product’s functionality, controllability, and

effectiveness. Non-instrumental qualities are related to the look and feel of a product (i.e., product

aesthetics). In this model of perceived usability, both qualities influence emotional reactions to a

product, which affect system appraisals. Empirical data from three different experiments demonstrate

the influence of different usability levels on the emotional reactions of users. These experiments show

that the degree of usability may impact user emotions; further, aesthetics affect perceived usability,

thereby impacting product perceptions (Thuring and Mahlke 2007).

While this model illustrates the relationship between emotional reactions to a product and perceived

usability, it limits emotions to those evoked by quality perceptions. Although individual characteristics

are included, the model does not explain the ways in which emotional states affect product interactions

and the generation of affective states. Another limitation of the study is the use of a single type of

product for usability appraisal.

Hassenzahl, Platz et al. (2000) defined usability in the context of two main types of product qualities:

ergonomic qualities (EQ) and hedonic qualities (HQ). EQ refers to aspects of traditional usability such as

efficiency and effectiveness. HQ relates to dimensions that are not obviously related to the task but

rather to a user’s personal interests and to a product’s originality, innovativeness, beauty, etc. A

depiction of the model is shown in Figure 2.5.

33

Figure 2.5 Usability model with EQ and HQ (Hassenzahl, Platz et al. 2000).

The model is based on experimental data showing that users can perceive hedonic and ergonomic

qualities independently. Both types of qualities equally contribute to the appeal of products - in this

case, software prototypes. Thus, the ergonomic qualities of a product alone may not influence the

emotions evoked during interaction. Instead, both types of qualities influence product judgment, or

perceived usability.

While this model of usability acknowledges emotional aspects, it assumes that emotions are evoked

only through interaction with a product and its characteristics. It also assumes that both hedonic and

ergonomic qualities have the same importance to the user and that these qualities are independent. In

reality, it is appropriate to think that the importance given to both qualities would depend on the type

of product and the context of use. In addition, hedonic qualities may impact the perceived ergonomic

qualities. The model does not include individual factors such as affective states and the effect of

emotion on product appraisal.

Although these models acknowledge the effect of emotions on product interaction, most researchers

study the emotions evoked by product characteristics.

34

2.2.9 Positive Affect, Negative Affect and Perceived Usability

In this section, we summarize Russell’s (1980) circumplex model of affect to explain differences in

positive and negative emotions. It is a spatial model that represents concepts of affect in a circle,

following a specific order. The circle is divided into four quadrants by a horizontal axis, which represents

the valence of an emotion (positive or negative) and a vertical line representing the level of arousal

(aroused or sleepy). Moving counter-clockwise beginning at the right horizontal axis, the affective

concepts are organized as follows: pleasure (0°), excitement (45°), arousal (90°), distress (135°),

displeasure (180°), depression (225°), sleepiness (270°), and relaxation (315°). Figure 2.6 depicts the

circumplex model of affect (Russell 1980).

Figure 2.6 Circumplex model of affect (Russell 1980).

The affective processing principle states that positive affect promotes behavior while negative affect

inhibits behavior (Clore and Hudtsinger 2007). Positive affect serves as an incentive to use accessible

information, privileging the expectation-driven or top-down aspects of these processes (Clore, Wyer et

al. 2001). Positive moods may also be associated with the adoption of a global focus involving

integrative processing. In contrast, negative moods may be associated with a reluctance to rely on

35

accessible information and adoption of a local focus, resulting in the selection of analytic processing.

People in negative moods employ strategies that direct attention to details, tending to disregard existing

knowledge (Schwarz 2000). On the other hand, individuals in positive moods are more likely to use top-

down cognitive strategies, relying on heuristics and pre-existing knowledge. In addition, people in

positive moods tend to overestimate the likelihood of positive outcomes and underestimate the

likelihood of negative outcomes (Stickney 2009). Positive affect seems to promote the use of schemas

that are available in memory, while negative affect inhibits their use, leading to more local, stimulus-

bound processing (Clore and Hudtsinger 2007).

2.2.10 Negative Affect - Frustration

Frustration is the most common emotion experienced by users, especially by users of computer

software-related products (Hazlett and Benedek 2007). Although most of the studies related to

frustration and human-computer interaction are based on software products, some studies have

examined frustration with hardware devices (Hoffberg 1991; Luczak 2003).

Studies on frustration related to software and hardware products agree that user frustration may be

generated by poorly designed interfaces, which require high levels of cognitive processing. Although

there are several definitions for the word frustration, most agree that it emerges as a result of an

obstacle preventing a person from satisfying a need (Scheirer, Fernandez et al. 2002).

One of the principal causes of frustration is delayed reinforcement of a conditioned response (Scheirer,

Fernandez et al. 2002). Hazlett and Benedek (2007) found that the main factors influencing frustration

levels are: (1) the importance of the goal to be attained, and (2) the user’s expectation for successfully

achieving the goal. From the perspective of task performance, the factors that influence frustration are:

(1) the severity of the interference or goal obstacle, (2) the length of the delay in goal attainment, and

(3) the amount of effort required to overcome the obstacle.

36

It is known that frustration decreases product desirability (Hazlett and Benedek 2007). Most, if not all of

the efforts in usability are focused on understanding the effects of frustration generated by products.

These efforts ignore previous frustration, how it affects interactions with products and the resulting

usability perceptions. How would pre-conceived frustration lower the threshold of negative emotions

while interacting with a product? In which ways would this propitiate the generation of frustration with

the product, or similar negative emotions? How would a user react to the same frustrating situation

with the product if the user is in a happy mood? The answers to these questions, as well as the effects of

emotional states on usability perceptions are unknown (Brave and Nass 2003).

2.2.11 Positive Affect - Happiness

Based on a summary of articles, evidence shows that positive affect enhances problem solving and

decision making, promoting flexible, innovative, creative, systematic, careful, and efficient cognitive

processing (Isen 2001). Several experiments in the healthcare and consumer research areas suggest that

positive affect facilitates creativity, cognitive flexibility, innovative responsiveness, and openness to

information (Isen 2001). Positive affect changes cognitive processing through elaborating and

incorporating a range of ideas. Medical diagnostic studies suggest that positive affect facilitates the

integration of material needed for decision making, alleviating confusion and promoting efficiency (Isen

2001). As Brave and Nass (2003) note, “Even mild positive affect profoundly affects the flexibility and

efficiency of thinking and problem solving”(Brave and Nass 2003). Positive affect has also been shown to

increase heuristic processing. “Keeping the user happy may not only affect satisfaction, but may also

lead to efficiency and creativity” (Brave and Nass 2003). Based on the filter of mood principle (Clore and

Gasper 2000) a happy user visiting an e-commerce site would be more likely to evaluate products

positively.

37

2.3 Self-Report Methods for Emotion

Self-report methods are those that require an individual to express his or her emotions by verbalization

or identification, or by rating them on a scale. These methods require individuals to reflect on their

feelings and emotional experiences, which may be defined as conscious representations of changes in

the states of body systems that occur in reaction to emotion-eliciting situations. Post-interaction

questionnaires currently serve as the primary methods for ascertaining emotions and moods during

interactions (Brave and Nass 2003). An example of an emotional experience follows:

I have just listened to someone insulting me, I feel my muscles tensing and a hot feeling around my neck and head region, I am gesturing with a clenched fist and gritting my teeth, I know I’m feeling angry (Wallbott and Scherer 1989).

The introspective report of this sort of emotional experience is an effective way to obtain the maximum

and most accurate information about what an individual is experiencing. Currently, self-report is one of

the most accurate and comprehensive ways to obtain information about an individual’s emotional

experience (Wallbott and Scherer 1989). Self-report is based on an individual’s introspection and

analysis of the current state, which is usually more cost effective and makes data easier to collect. It can

be used as a complement to other techniques, such as physiological measurement. Self-report

techniques may assist in the identification of systematic patterns of relationships among situation

evaluation processes, physiological responses, expressive behaviors, action tendencies, and feeling

states, as well as control attempts (Wallbott and Scherer 1989).

Methods such as the Emotions Profile Index (EPI) (Plutchik 1989), the Differential Emotions Scale (DES)

(Izard 1977), Mood Adjective Checklist (Nowlis 1965), and the Multiple Affect Adjective Checklist

(Zuckerman and Lubin 1965) ask participants to rate how much they felt specific emotions in certain

situations using word lists and Likert scales ranging from three to nine points. These methods, which

38

have been previously validated and are commonly used in psychology, focus mostly on the overall

identification of affective states based on pre-determined dimensions of emotions. Most of them

include a dimension to account for happiness or joy, but fail to include frustration.

Questionnaires and adjective checklists are the most common self-report methods, as they can be

designed for specific contexts. Previous research has proven that questionnaires are better suited to

capturing emotional responses than any other self-report technique (Wallbott and Scherer 1989). In this

research, we propose the use of questions related to the user experience at the beginning and at the

end of the designed experiment. The questions will ask the participants how much joy and frustration

they felt using a seven-point Likert scale.

Watson, Clark et al. (1988) developed the Positive Affect and Negative Affect Schedule (PANAS Scale),

which is a method used to estimate values for two major dimensions of affect, positive (PA) and

negative (NA). The participants are provided with a list of 20 randomly-organized items, with 10

referring to positive emotions, and 10 referring to negative emotions. The users are asked to select

values ranging from one to five to describe the extent to which they felt that mood. One represents “not

at all” and five represents “extremely.” The total of these ratings represent the PA and NA scores for

each category, measuring the degree to which each affect is experienced. The PANAS scale can be used

to assess emotional states of seven different timeframes (present, past week, etc.). This scale uses 60

adjectives discovered by Zevon and Tellegen (1982) through factor analysis that purely represent

positive and negative affect. This was verified by including terms with substantial loading on one factor

but with values near zero on the other (Gray and Watson 2007). The 10-item scale showed excellent

internal consistency and convergent and discriminant validity (Watson, Clark et al. 1988). Because the

PANAS method is reliable, brief, and easy to administer, it will be used in this research to assess the

positive and negative emotional states of the participants. This method should provide a quantitative

39

measure of the positive and negative emotions experienced by the subjects. Please refer to Appendix B

for the list of PANAS scale items.

The affect grid (Russell, Weiss et al. 1989) is a single-item scale designed as a quick way of assessing

affective states in the dimensions of valence and arousal. It is based on Russell’s (1980) circumplex

model of affect; thus, the affective descriptors are placed on a grid according to their relationships in the

circumplex model, defining the dimensions of pleasure-displeasure and arousal-sleepiness. Figure 2.7

shows the affective grid and the circumplex model of affect.

Figure 2.7 Affect grid and circumplex model of affect (Russell 1980).

The participants are provided with a single-item 9x9 grid graphic showing affect descriptors on each

corner and at the midpoints of the grid, and are instructed to check the cell in the grid that best

represents their actual emotional state. The values for emotional valence (positive or negative emotion)

and arousal level (degree of emotion) are estimated based on the position of the grid selected by the

participant.

Developed based on the pleasure, arousal, and dominance theory of emotions, the Self-Assessment

Mannequin (SAM) is a pictorial instrument that makes use of animated cartoons representing emotions.

It has been created to overcome cultural comparison issues related to the verbal identification of

40

emotions (Morris 1995). SAM is a non-verbal pictorial assessment technique that provides direct

measures of valence (pleasure), activation (arousal), and dominance associated with a person’s affective

reaction (Bradley 1994). It visually represents Mehrabian and Russell’s (1974) three PAD dimensions,

which are the three dimensions of an emotion: valence, arousal and dominance (Morris 1995). Valence

refers to the positive or negative strength of an emotion; activation is the excitation level or emotion

intensity; and dominance is the strength or weakness of the person. SAM is based on the assumption

that all emotional reactions are a combination of the three universal core human emotions. Figure 2.8

shows a depiction of SAM.

Figure 2.8 The Self-Assessment Mannequin (Morris 1995).

In SAM, a user is asked to select the picture that best represents his or her emotional state in a three-

dimensional emotion space, or the space between pictures. The location selected by the participant

indicates the score obtained for each dimension, which is the metric or indicator of emotion in all three

41

dimensions. Thus, each participant provides feedback on three dimensions of emotion, and a numeric

score ranging from 1 to 9 is assigned for each dimension.

This method allows participants to express how they feel by using visual representations. An advantage

of this method is that it appears to avoid the cognitive processing of the subject’s emotional state,

bypassing rational evaluative processing. The main disadvantage of SAM is that it does not provide

information on specific emotions, but rather general affective states.

Measuring affect using self-report methods presents additional challenges. Questionnaires are capable

of measuring only the conscious experience of emotion and mood, leaving no opportunity for the

recognition of emotions lying in the limbic system and in subconscious processes. In addition, when an

individual is asked to describe an emotional experience from the past, he or she often relies on

imperfect and potentially biased memories. If requests to describe emotional experiences occur during

interactions with situations or products, the interactions themselves may be obstructed or altered.

Furthermore, emotions and moods may be difficult to describe with words, and such words may not

have the same semantic meanings for all humans, adding subjectivity to the reports which are based on

individual interpretations. Self-report measures also rely on an individual’s willingness to communicate

about emotions, which may be affected by personality and nature.

Despite all these disadvantages, self-report measures are still the most direct way to measure

sentiments and serve as a reasonable alternative to direct measures of emotion and mood (Brave and

Nass 2003). Thus, self-report measures remain the only devices for studying the subjective experience of

emotions, and are potentially useful (Izard 1977). In this study, the Positive and Negative Affect

Schedule (PANAS), the Self-Assessment Mannequin, and the Affect Grid will be used.

42

2.4 Self-Report Methods for Usability

Usability has been defined and measured in many ways. Based on ISO’s definition of usability, it

comprises the effectiveness and the efficiency with which a user executes a task in a system and the

satisfaction experienced by the user after system use. In the experiments for this investigation, the IBM

Post-Study System Usability Questionnaire (PSSUQ) and the System Usability Scale (SUS) will be used to

estimate smart phone/software system usability. Both questionnaires are standardized, validated

methods that provide numerical values for usability. This following section provides a brief description

of the IBM PSSUQ and the SUS.

The IBM Post-Study System Usability Questionnaire is a standardized questionnaire for assessing system

usability. It is comprised of approximately 19 questions by which measures of overall satisfaction,

system usefulness, information quality, and system quality may be estimated. This questionnaire has

been validated and widely used in the HCI/usability community. We will use the questionnaire to obtain

an assessment of overall perceived usability.

Similarly, the System Usability Scale (SUS) is a simple, 10-item scale that provides a global view of

usability based on subjective assessment. SUS uses Likert scales to gather participant impressions of

usability aspects, and consists of ten questions where respondents must indicate the degree to which

they agree or disagree with the statements. SUS has proved to be a valuable evaluation instrument as

well as being robust and reliable, as it correlates well with other subjective measures of usability

(Brooke 1996). SUS has been used for a variety of research projects and industrial evaluations. We

propose the use of SUS to estimate a measure of overall usability based on a participant’s subjective

experience with the product.

43

2.5 Summary

In this chapter, a summary of the literature relevant to this research was presented. Multiple definitions

of usability and emotions were discussed. In addition, the nature of emotion and its effect on cognitive

activities was summarized. Numerous studies confirming the effect of affective states on human

thought, judgment, creativity, feelings, actions, and decision making were discussed. Similarly, the

effects of emotion on consumer judgment and product appraisal were summarized. Additionally this

chapter presented models of product appraisal and perceived usability, which show the relationships

between emotions experienced while using a product and perceptions of it.

The definitions of positive and negative affect were summarized and the self-report methods for

emotional state and perceived usability were discussed. In this research, three self-report methods for

emotions will be used: the Positive and Negative Affect Schedule (PANAS), the Affect Grid (AG), and the

Self-Assessment Mannequin (SAM).

To measure perceived usability, we will use the System Usability Scale (SUS) and the IBM Post-Study

System Usability Questionnaire. The SUS consists of ten questions and provides a numerical value for

perceived usability ranging from 0 to 100. The IBM questionnaire provides a numeric estimate for

perceived usability in terms of overall satisfaction.

Usability is the extent to which a product is effective, efficient, and satisfying in a particular context.

Usability is continuously evolving, and it is an important part of product evaluation. Even though new

trends in human factors take into account the importance of emotions in product experiences, emotions

have been overlooked in usability studies (Dillon 2002). Most studies on the relationship between

emotion and perceived usability are limited to emotions experienced while interacting with a product

that are linked to certain product characteristics. Thus, to the best of our knowledge, the effects of

emotions experienced prior to product interactions have not been considered.

44

In addition, most of the research on emotion and product appraisal are executed from a marketing and

product preference perspective, the metrics of which differ significantly from perceived usability.

Research on the effect of emotion on product appraisal suggests that emotions prior to interactions

with products may affect usability perceptions. Therefore, we believe the community of usability

engineers must understand the effects of prior affective states on perceived product usability. The

findings of this research should satisfy such a need, and contribute to the expansion of the concept of

usability. This study will provide practical knowledge to usability engineers and designers on the effects

of emotion on perceived usability, thereby filling a significant gap in the literature.

45

Chapter 3

Methodology

This research presents a set of designed experiments to analyze the effect of preconceived emotional

states on perceptions of product usability. We intend to:

1) Understand the extent to which positive and negative affective states prior to interactions with

products influence usability perceptions.

2) Identify how the usability dimensions of overall usability, software usability, perceived ease of

use, and task completion rate are affected by emotional states prior to interactions.

3) Make recommendations to the product design and human-computer interaction community on

the importance of previous emotional states for product usability perceptions.

This will be accomplished through experimentation completed in two stages. Stage 1 consists of

preliminary data collection and assessing the instruments to be used in experiment 4 (the main

experiment in this research work). The second stage consists of the main experiment (experiment 4), in

which the effect of emotion in different scenarios of controlled usability will be assessed. The use of a

preliminary stage prior to the final experiment will help us validate the instruments to be used in the

main experiment and make necessary corrections in order to minimize biases. Figure 3.1 illustrates the

two stages and their purposes. Each stage of experimentation consists of one or more experiments.

Stage 1 consists of experiments 1, 2 and 3. Experiment 1 will evaluate the effectiveness of the emotion

elicitation strategy to generate emotions in the participants. Other variables, such as exposition time

and the self-report methods used to report emotion will be tested as well. Experiment 2 will verify the

46

effects of emotion on usability assessments, while experiment 3 will assess the effect of variables such

as time between sessions on perceived usability.

The results of experiments 1, 2, and 3 will determine the experimental parameters and instruments to

be used in Stage 2. Stage 2 consists of experiment 4, in which the effect of emotion in different

scenarios of controlled usability will be assessed.

Emotion elicitation

• Validate strategy• Test exposition time• Test 3 self-report methods

Emotion effect in

usability

•Validate effect • Test 2 usability report methods •Understand presence of biases

Exposition timeSelf-report method

Emotion effect with controlled usability

•Explore effect for different smart phones and in different scenarios of software usability

MethodologyUsability report

Compare the effect of the time between sessions in the perceived usability

Experiment 1 Experiment 2 Experiment 4

Experiment 3

Stage 1

Preliminary Experiments

Stage 2

Main Experiment

Figure 3.1 Stages of experiment execution.

In this research, we will elicit emotions related to happiness and frustration, which will be directly

related to the interfaces used and the task environment. We chose to elicit happiness for positive affect

and frustration for negative affect because these two emotions are easily generated with the selected

emotion elicitation method. The user will feel contentment after successfully completing the computer

task, eliciting happiness. For the negative affect condition, intentional delays in the computer task will

provoke frustration, eliciting a negative emotion.

The following sections of this chapter provide detailed descriptions of each experiment.

47

3.1 Experiment 1: Validation of the Emotion Elicitation Strategy

The main purpose of this research is to provide empirical evidence supporting the effect of emotion on

perceived product usability. For this purpose, it will be necessary to use an emotion elicitation technique

that generates emotions in the participants.

The methodology created to elicit emotions will be tested in this experiment. In this methodology, the

participants will be asked to execute a computer task. For the positive affect group, the tasks will be

very simple and easy. For the negative affect group, the tasks will be more difficult. Also, the interface

for the computer task was programmed to create random freezes from 1-4 seconds. A detailed

description of the tasks and the emotion elicitation environment are provided in Appendix A.

This stage will, with an experiment, validate the emotion elicitation strategy. This strategy may be used

for other experiments in which the effect of emotion is explored. The objectives of this experiment will

be as follows:

1) To validate the emotion elicitation strategy and its adequacy in generating positive and negative

emotions in the participants.

2) To understand if elicitation effectiveness changes based on the number of stimulus exposures

(45 stimuli or 65 stimuli); the computer task with 45 stimuli will require less time (approximately

5 minutes) than the one with 65 stimuli (approximately 8 minutes).

3) To gather data on three self-report methods for measuring emotional states, and to understand

which method is more suitable and sensitive to observed changes in emotions.

4) To understand the degree to which the generated emotions are affected by the perceived

importance of the goal by specific individuals and personality types.

5) To test experimental parameters (e.g., time to forget the task) and the biases that may be

present.

48

In this experiment, the subjects’ emotions will be assessed using three self-report methods, the Affect

Grid (AG), the Self-Assessment Mannequin (SAM), and the Positive and Negative Affect Schedule

(PANAS). Goal importance will be assessed by asking a series of questions related to the importance of

maximizing the amount of money earned and minimizing the execution time while maximizing the

accuracy of the responses to the tasks. Personality type will be measured using the Myers-Briggs Type

Indicator (MBTI) personality scale (Myers-Briggs-Foundation 2010). Participants will be asked to interact

with two variations of the computer task on two separate sessions, which will be scheduled seven days

apart. For each participant, the same emotion will be elicited in both sessions. Figure 3.2 shows the

sequence of events for the experiment.

Figure 3.2 Sequence of events for experiment 1.

For this experiment the following hypotheses apply:

Ho1: The mean initial emotion score will be equal to the mean final emotion score.

This hypothesis tests the effectiveness of the emotion elicitation strategy for both positive and negative

emotion. The proposed emotion elicitation strategy is based on appraisal theories of emotion. Such

theories state that emotion generation stems partly from situation appraisal processes and partly from

individual values and goals (Roseman and Smith 2001). When situations in the environment are in

49

congruence with the world, positive emotions are generated. Conversely, when situations in the

environment are not in congruence with the world, negative emotions are generated.

In the proposed emotion elicitation strategy, it is expected that the participant’s goal will be to maximize

monetary compensation by minimizing task time and maximizing task accuracy. In order to elicit a

positive emotion, easy and repetitive tasks and a functional interface will promote accomplishment of

the goal. In order to elicit a negative emotion, random delays between task transitions in the computer

program will impede a participant’s ability to maximize compensation. For the emotion elicitation

strategy to be effective, the differences between the initial and final emotion will have to be statistically

significant.

Ho2: The mean difference in emotion score for the computer game with 45 stimuli will be equal

to the mean difference in emotion score for the computer game with 65 stimuli.

Given that the emotion elicitation strategy will be developed for this experiment, parameters such as

the number of stimuli necessary to elicit an emotion in the participants is unknown. By testing this

hypothesis, we will ascertain the most effective number of stimuli for eliciting positive and negative

emotions.

3.1.1 Materials for Experiment 1

A portable computer with Windows Vista Home Edition will be used to execute the tasks. Director MX

will be used to program the interface for the computer task. The Recoding User Input (RUI) (Kukreja,

Stevenson et al. 2006) program will be used to record task times and the answers provided.

A pre-experiment questionnaire will be used to obtain demographic information. In that questionnaire,

the participants will be asked five questions to assess the importance of the computer task goals.

50

The personalities of the participants will be assessed by using the Myers-Briggs Type Indicator (MBTI).

The MBTI is a psychometric questionnaire that is designed to assess psychological preferences in how

people make decisions and perceive information from the world. The results of the questionnaire yield a

personality profile for each participant, composed of a four-letter code. The questionnaire is based on

Jung’s typological model and sorts differences based on four opposite pairs, resulting in 16 different

psychological types. The four typological pairs are: Extraversion/Introversion, Sensing/Intuition,

Thinking/Feeling, and Judgment/Perception. In this experiment, we are interested in the

Extraversion/Introversion dimension of personality.

Extraversion/Introversion describes the part of the personality that prefers to either be action-oriented

(extraversion) or thought-oriented (introversion). This preference may influence a participant’s

willingness to share information about his or her emotional state. Given that self-report methods for

emotion will be used, it will be appropriate to collect this data. For analysis purposes, this trait will be

treated as a categorical variable with two levels, introvert and extravert.

Diag tasks will be used as task stimuli (Ritter and Bibby 2008). An instruction sheet will be provided to

all the participants at both trials.

To measure emotions, the questionnaires consisting of the Affect Grid, Self-Assessment Mannequin

(SAM) and the Positive and Negative Affect Schedule (PANAS) will be used. A final questionnaire will ask

participants about their experiences in order to ascertain whether they suspected the true purpose of

the task. Please refer to Appendices B.4 – B.6 for the affective questionnaires that will be used.

3.1.2 Experimental Design for Experiment 1

In this experiment, we will test the use of 45 stimuli versus 65 stimuli embedded in a Diag computer task

to elicit emotions. Each participant will be exposed to each condition in separate sessions, although both

51

sessions will involve the same emotional valence (type of emotion). There will be seven days between

sessions to allow for sufficient forgetting of the tasks. The two computer game conditions will be

counterbalanced within subjects.

This experiment is a 2 (positive or negative) x 2 (45 or 65 stimuli) factorial design. Given that the type of

emotion to be assigned will be fixed, and the other factors will be randomized, our model is a mixed

model. Given that each participant will test the two conditions, we will use a within-participant

crossover repeated measures design for our experiment (Montgomery 2009). The study is within

participants, which means that each participant will experience at least two of the conditions to be

tested. It is a crossover repeated measure because the participants will experience the factors more

than once, and the order in which the factors will be applied to the participants will be counterbalanced.

Participants will be randomly divided into two groups of 15 participants each: the positive affect group

and the negative affect group. Within each group, participants will be assigned to one of the two

sequences available (45 stimuli in the first session or 65 stimuli in the first session). Figure 3.3 shows the

sampling procedure.

Figure 3.3 Sampling procedure for experiment 1.

52

All of the conditions in the experiment will be counterbalanced among subjects. Given that we are also

interested in analyzing the effectiveness of the three measurement methods, the order in which the

questionnaires will be administered will be counterbalanced.

Table 3.1 describes the variables of the study and their categorization. The response variable will be the

difference across the emotional scores before and after the computer task. Given that three

measurement methods will be investigated, there will be three response variables. Emotion indicates

the type of emotion elicited. Nstimuli refers to the number of stimuli in the computer task. Sequence

refers to the number of stimuli a participant sees first, either 45 or 65 stimuli. Session indicates when

each observation is made, either the first or second session. Goal importance refers to the degree to

which it is important to a participant to maximize accuracy and minimize task time, thus maximizing the

amount of money earned. Goal importance will be estimated by averaging the answers to five Likert

scale questions. Please refer to Appendix B for the goal importance scale.

Table 3.1 Variables and Their Classifications for Experiment 1

Variable type Variable name Type Levels

Response (dependent

variable)

Difference in emotion score

Numerical NA

Factors (independent

variables)

Emotion Categorical 2 (positive and negative)

Nstimuli Categorical 2 (45 and 65 stimuli)

Sequence Categorical 2 (45 first and 65 first)

Session Categorical 2 (first and last session – seven days apart)

Goal importance Numerical NA

Personality Categorical 2 (extravert and introvert)

53

3.1.3 Procedures for Experiment 1

All participants will arrive in the room (Leonhard 236A) and be debriefed on the requirements of the

study as well as its purpose. The initial introduction to the experiment will convey a cover story to make

participants believe that there is a different purpose for the study. To avoid biases related to suspecting

the purpose of the experiment, they will be told that the purpose of the study is to analyze the effect of

displays on visual cognition and reaction time. If the participants are told that the purpose of the study

is to elicit either positive or negative emotions, then the emotions experienced, if any, might not be

natural or genuine. Thus, we might not be able to see a true emotional effect. All participants will be

informed of the true purpose of the experiment at the end of the second session. They will be reminded

of their rights to withdraw their data from the experiment if they wish.

The participants will be instructed to come back after seven days for their second session. They will be

asked to allocate 45 minutes to one hour of time to execute the tasks for each session. The

experimenter will explain the possibility of increasing the amount of money earned if they minimize the

time required to complete the tasks and maximize the accuracy of their answers. The formula used to

estimate the extra compensation will be explained. The participants will earn a fixed amount of 8 dollars

per session. They will earn 10 cents for each correct answer and they will lose 5 cents for every minute

they take. Refer to Appendix A for more information about the formula used to estimate the monetary

compensation.

To avoid biases in the experiment, the researcher executing the sessions will follow a pre-written script

with detailed instructions. In addition, the participants will be provided with written instructions for the

computer task.

Participants will be provided with consent forms to sign, at which point the study will begin. Figure 3.4

shows the sequence of the tasks to be executed by the participants in the experiment.

54

Personality typeGoal importance

Initial emotionSAM, AG, PANAS

7 days later

Positiveaffect

Initial emotionSAM, AG, PANAS

Final emotionSAM, AG, PANASNegative

Final emotionSAM, AG, PANASNegative

Positiveaffect

Figure 3.4 Sequence of tasks for experiment 1.

A pre-experiment questionnaire will be provided to the participants to obtain information about their

backgrounds, interests, and the levels of importance they attach to maximizing the amount of money

earned. Then, they will be asked to complete the three affect questionnaires in order to ascertain

baseline emotional states. The questionnaires will be provided in random order. Immediately after, the

participants will be provided with instructions on the task and the expectations, and will be provided

with the literature required to execute the tasks. Both groups of participants will be given five minutes

to study the materials and familiarize themselves with the tasks. For the negative affect group, the study

materials will be removed after five minutes. The participants in the negative affect group will be

instructed to memorize the circuit schematic.

The investigator running the session will upload the RUI program to record all user input and the

amount of time it takes to execute the tasks. Each participant will be told the cover story that the Diag

task will measure cognitive style and the ability to interact with technology. They will be instructed to

work as quickly and accurately as possible, and to try to maximize their monetary compensation.

55

The participants will interact with the computer program that serves as the platform for the Diag task.

The Diag task is a fault-finding task in which the participants are asked to find the part of a circuit that is

broken based on the structure of the circuit schematic and the specific control panel that is shown

(Ritter and Bibby 2008). The task of finding the broken part of the circuit is used in the emotion

elicitation strategy.

In the positive affect condition, the participants will be exposed to easy tasks and the tasks will be

repetitive. This will promote learning the task, and the participants will feel happy and satisfied because

they will be able to be fast and accurate in their answers. In the negative affect condition, there will be

random delays in between task transitions in the computer program. The delays will be programmed in

Director MX to appear randomly and they will range from 2 to 4.5 seconds. The frustration elicitation

strategy developed by Picard (2000) will be used to estimate which delays will cause frustration in the

participants (Picard 2000).

At the end of the tasks, participants will be instructed to fill out the three affect measurement

questionnaires again. While each participant completes the questionnaires, the experimenter will

determine his or her extra compensation. Both the task time and the number of correct answers will be

determined based on the information recorded in RUI. After finishing all of the questionnaires, each

participant will be told the amount of extra compensation earned, which will be paid at the end of the

second session.

The second session will follow the same procedure as the first, except: 1) the participants will not be

randomly assigned to an emotion or condition group, but will stay in the same emotion assigned in the

first session, completing the task under the other condition; and 2) the participants will not complete

the pre-experiment questionnaire. At the end of the second run, the participants will be provided with a

post-experiment questionnaire.

56

The total amount of money earned will be estimated based on the formula provided, and the

participants will be paid in cash based on their performance. The investigator will thank them for their

participation, and the participants will be free to leave.

3.2 Experiment 2: Testing the Effect of Emotion on Usability

When users interact with products, they are potentially using cognitive skills. Thus, given the evidence

on the effect of emotion on certain cognitive activities, we believe that usability appraisals may be

influenced by emotional states. Specifically, we are interested in effects on perceived usability based on

the emotional state of a user just prior to interacting with a product. The objectives of this preliminary

experiment will be as follows:

1) To understand whether there is a difference in perceived usability based on the affective state

of the user, and to what extent such a difference may be perceived.

2) To gather data on two different methods of measuring perceived usability: IBM’s Post-Study

System Usability Questionnaire (PSSUQ) (Lewis 1995), and the System Usability Scale (SUS)

(Brooke 1996).

3) To test the effectiveness of the design of experiment (DOE).

4) To detect any potential bias generated during the experiment such as discovering or suspecting

its real purpose, and to ascertain the effectiveness of strategies aimed at mitigating such biases.

In this experiment, we will use only one smart phone/software combination. This experiment will

provide preliminary data to inform and improve the DOE for the main experiment. In the main

experiment, the perceived usability of all possible smart phone/software combinations will be tested for

both emotional states.

57

Immediately after completing the Diag computer task, participants will be asked to fill out the emotion

level questionnaires and interact with the phone and software for about 10-15 minutes. Emotion levels

will be measured using the Positive and Negative Affect Schedule (PANAS). The participants will then be

asked to execute a series of tasks in the software using the smart phone provided. After their

interactions, they will fill in the SUS and the IBM PSSUQ to assess the usability of the phone and

software as a system. In the second session, which will be executed seven days later, a different

emotion will be elicited in the participants. Figure 3.5 shows the sequence of events for the experiment.

Figure 3.5 Events and task sequence for experiment 2.

The following hypotheses apply to this experiment:


This hypothesis will test the effectiveness of the emotion elicitation strategy for both positive and

negative emotions. Experiment 1 will validate the emotion elicitation strategy. In the same fashion, this

hypothesis will validate the fact that emotions are generated after the computer task. We expect the

differences between initial and final emotions to be statistically significant.

Ho4: The differences in mean usability scores will be equal for the positive and negative affect

conditions.

58

We expect the differences in mean usability scores between the positive and the negative emotion

conditions to be statistically significant. Previous studies on product judgment and emotion suggest that

there is a tendency for users to judge a product in the same direction as their current emotions. In

general, positive affect promotes a positive impression of products while negative affect promotes a

negative impression. These studies evaluate the judgment of products by focusing on their performance,

where participants use global ratings to indicate the degree to which they like a product in general.

Metrics such as product preference, purchasing intentions, product attachment, and the perceived risks

of using the products are incorporated. In this research, we intend to study such effects specifically for

perceived usability. Thus, we expect that the mean usability for positive affect will be higher than the

mean for negative affect.

Usability is the touchstone of human-computer interaction. It refers to the effectiveness and efficiency

of a product, and its ability to satisfy user needs. Usability differs from general product judgment in that

it has specific measures that are objective and subjective, and in that it is the standard used in human

factors to assess products.

Ho5: The degree of experience with similar smart phones will have no statistical effect on the

reported perceived usability.

Previous experience may be a factor that influences perceptions of a product’s usability. We expect that

if the participants have used smart phones with similar characteristics prior to the experiment, usability

assessments will be influenced by their previous experiences.

3.2.1 Materials for Experiment 2

A portable computer with Windows Vista Home Edition will be used to execute the computer tasks.

Director MX will be used to program the interface for the computer task. The Recoding User Input (RUI)

59

program (Kukreja, Stevenson et al. 2006) will be used to record the task time and the answers provided.

An HTC Pure smart phone will be used with a Windows Mobile 6.0 operating system. The smart phone is

a touch screen phone, with dimensions of 4.33” x 2.1” x 0.59”, and a weight of 6.28 oz. Figure 3.6 shows

the smart phone used in this experiment.

Figure 3.6 HTC Pure smart phone.

A software program named Adarian Money will be used. This personal financial management software

allows users to verify and edit financial information. Figure 3.7 shows two views of the software

program.

Figure 3.7 Adarian Money software program.

60


the participants will be asked five questions to assess the importance of the computer task goals. They

will be asked about usage, previous experience, and smart phone preference. The Diag tasks will be used

as task stimuli (Ritter and Bibby 2001), as per experiment 1. An instruction sheet will be provided to all

participants in both trials. The same emotion elicitation strategy described in the previous section of this

chapter will be used. Please refer to Appendix A for a detailed description of the computer task, and to

Appendix C for the set of questionnaires used in this experiment.

To measure emotions, the Positive and Negative Affect Schedule (PANAS) will be used. To measure

usability, participants will complete the IBM Post-Study System Usability Questionnaire (PSSUQ) and the

System Usability Scale (SUS). A final questionnaire will be used to ask participants about their

experiences, thereby discovering if they suspected the true purpose of the task. Please refer to

Appendix C for all the questionnaires.


In this experiment, we will test the effect of positive and negative emotions on the perceived usability of

the smart phone software system. Each participant will be exposed to both types of emotions, positive

and negative, in separate sessions. There will be seven days between sessions to allow for sufficient

forgetting of the tasks and the smart phone/software system. The type of emotion experienced in each

session will be counterbalanced among subjects.

The participants will be randomly assigned to one of the two available sequences (positive affect in the

first session or negative affect in the first session). Figure 3.8 shows the sampling procedure.

61

Figure 3.8 Sampling procedure.

All conditions in the experiment will be counterbalanced among subjects. Given that we are also

interested in analyzing differences among the two usability measurement methods, the order in which

the questionnaires will be administered will be counterbalanced as well.

Table 3.2 describes study variables and their categorization. The response variable will be the perceived

usability of the smart phone software system. Emotion indicates which type of emotion will be elicited.

Sequence refers to which emotion the participant experienced first, either positive or negative. Session

corresponds to the session in which the observations will be made, either the first or second session.

Previous experience will be estimated by averaging the answers to five Likert scale questions related to

experience with similar cell phones and software. Please refer to Appendix C for the previous experience

scale.

62


Variable type Variable name Type Levels

Response (dependent variable)

Difference in usability score

Numerical NA

Factors (independent variables)

Emotion Categorical 2 (positive and negative)

Sequence Categorical 2 (positive first and negative first)

Session Categorical 2 (first and last session – seven days apart)

Potential covariate Previous experience

Numerical NA

3.2.3 Procedures for Experiment 2

Upon arrival in Leonhard 236A, participants will be debriefed on the requirements of the study and its

purpose. The same cover story from experiment 1 will be used. Then, participants will complete a

computer task before interacting with the smart phone and the software. All participants will be

informed of the true purpose of the experiment at the end of the second session. They will be reminded

of their rights to withdraw their data from the experiment if they wish.

The participants will be instructed to come back in seven days for their second session. They will be

asked to allocate 45 minutes to one hour to execute the tasks for each session. The experimenter will

explain that more money can be earned by minimizing task time and maximizing accuracy in the

computer task. They will be provided with the consent form and after their agreement, the study will

begin. Figure 3.9 shows the sequence of the main tasks to be executed by the participants in the

experiment.

63

Previous experience

Emotion meas.#1

7 days later

Positive

Negative

Training Interaction

Emotion elicitation

Usability evaluation(SUS, IBM)

Interaction with Smart phone

Training Interaction

Emotion elicitation


Emotion meas.#2

Emotion meas.#1 Usability evaluation

(SUS, IBM)

Emotion meas.#2

Figure 3.9 Task sequence for experiment 2.

As in experiment 1, to avoid biases, the researcher executing the sessions will follow a pre-written script

with detailed instructions. In addition, participants will be provided with written instructions for both

the computer task and the smart phone task.

A pre-experiment questionnaire will be provided to the participants to obtain information about their

backgrounds, interests, and previous experiences with similar phones and software. Then, they will be

asked to complete the affect questionnaires (PANAS) to ascertain baseline emotional states.

Immediately after, the participants will be given standard training on the smart phone and the software,

and will be allowed to interact with the smart phone/software system for 5 minutes. Then, the purpose

of the computer tasks will be disclosed and participants will be provided with instructions. In both trials,

the participants will be given 5 minutes to study the materials and become familiar with the computer

tasks. For the negative affect trial, the study materials will be removed after 5 minutes. The participants

will complete the computer tasks and the emotion assessment questionnaires. Right after completing

the questionnaires, they will be asked to execute tasks using the smart phone software system. The

order of the tasks will be assigned randomly in both sessions. After finishing the tasks, participants will

complete the SUS and PSSUQ, which will be provided in random order.

64

The second session will follow the same procedure as the first, except: 1) participants will not be

randomly assigned to an emotion or condition group, but will complete the condition not completed in

the first session; and 2) participants will not complete the pre-experiment questionnaire. At the end of

the second run, participants will be provided with a post-experiment questionnaire. The total amount of

money earned will be estimated based on the formula provided, and the participants will be paid in cash

based on their performance. The investigator will thank the participants for their participation, and the

participants will be free to leave.

3.3 Experiment 3: Testing the Effect of Days between Sessions in Perceived Usability

In this study, we intend to evaluate whether the number of days elapsed between sessions affects

perceived usability. The essence of the experiment will be very similar to experiment 2. The objectives of

this preliminary experiment are:

1) To understand the degree to which there is a difference in perceived usability based on the

affective state of the user.

2) To understand the extent to which the days between sessions affects the perceived usability of

the smart phone/software system.

In this experiment, the materials and procedures will correspond to the ones used in experiment 2. The

only difference will be the use of five days and 10 days between sessions. We chose to use five and 10

days because they represent a reasonable interval containing the seven days between sessions

proposed in experiments 1 and 2. The results of this experiment will provide information on the effects

of different time intervals between sessions on perceived usability. Please refer to the previous section

of this chapter for information on the experiment materials and procedures.

The following hypotheses apply to this experiment:

65



negative emotion. We expect the differences between the initial and final emotions to be statistically

significant.

Ho7: The mean usability score will be equal for the positive and negative affect conditions.

We expect the mean usability scores for the positive and the negative emotion conditions to be

different. The mean usability for positive affect will be higher than the mean for negative affect.

Ho8: The elapsed time between sessions is not a significant variable affecting perceived usability.

Because the experiment requires participants to interact and assess the same product under different

induced emotions, it is necessary to assess the effects of the number of days elapsed between the first

and second sessions on perceived usability. With this hypothesis we will test differences in perceived

usability reported when there are only five days between sessions compared to when there are 10 days

between sessions.

The only variation in the procedure from the one detailed in Section 3.2 of this document will be as

follows:

1) The participants will be assigned to a condition for the days between the sessions. Half of the

participants will wait five days from the day of their first session to come back and execute the

second session. The other half will wait 10 days between sessions.

2) The variable days between sessions will represent the elapsed time between the first and the

second session. This is a categorical variable with two levels representing five and 10 days.

66

3.4 Experiment 4: Testing the Effect of Emotion on Different Levels of Designed (Controlled) Usability

This study will explore the effects of emotions on perceived usability as a function of product features.

The objectives of this preliminary experiment will be as follows:

1) To understand differences in perceived usability based on the affective states of users.

2) To understand which aspects of usability are affected by the affective states of users.

3) To understand differences in such effects as a function of two main product features: the

modality of data entry on the smart phone, and the usability of the software.

In this experiment, we will use four different smart phone/software combinations. We will use two

different smart phones, one that uses a physical keypad for data input (Blackberry), and another that

uses a virtual keypad (HTC). The Blackberry will be smart phone 1 and the HTC will be smart phone 2.

We will also use two different software versions, one that works seamlessly and is easy to navigate, and

another one that requires more steps to navigate and has intentional errors and delays. The easy to

navigate software will be software 1, while the one with poor usability will be software 2. In this

experiment, the controlled usability will be limited to the software application used. The software will

be created specifically for this research study. It will emulate a software application that manages

personal finances such as credit cards and bank accounts. Typical activities will include verifying account

balances, transferring money and making payments.

To avoid bias in the experiment, the researcher executing the sessions will follow a pre-written script

with detailed instructions. The participants will be provided with written instructions for the computer

task and the smart phone/software tasks.

67

Including two levels for the emotion factor, there are a total of eight possible combinations of

treatments, listed in Table 3.3.

Table 3.3 Treatment Levels and Factor Combinations for Experiment 4

Treatment Level

Description Emotion Smart Phone Software

Positive Affect / Smart Phone 1 / Software 1 + + +

Positive Affect / Smart Phone 1 / Software 2 + + -

Positive Affect / Smart Phone 2 / Software 1 + - +

Positive Affect / Smart Phone 2 / Software 2 + - -

Negative Affect / Smart Phone 1 / Software 1 - + +

Negative Affect / Smart Phone 1 / Software 2 - + -

Negative Affect / Smart Phone 2 / Software 1 - - +

Negative Affect / Smart Phone 2 / Software 2 - - -

Initially, participants will be randomly assigned to one of the eight possible experimental treatments.

Figure 3.10 shows the assignment of participants to the experimental conditions.

Positive Emotion

Smart Phone 1

Software 1

Positive Emotion

Smart Phone 2

Software 1

Negative Emotion

Smart Phone 1

Software 2

Positive Emotion

Smart Phone 2

Software 2

Negative Emotion

Smart Phone 1

Software 1

Positive Emotion

Smart Phone 1

Software 2

Negative Emotion

Smart Phone 2

Software 2

Negative Emotion

Smart Phone 2

Software 1

Figure 3.10 Assignment of participants to the experimental conditions.

Participants will be asked to execute the computer task to elicit positive and negative emotions. Their

emotions will be measured using the selected measure for emotion assessment. The participants will be

asked to execute a series of tasks using the software on the smart phone provided. Twenty tasks will be

executed in random order (refer to Appendix D for the list of tasks). Participants will be assigned to use

one smart phone/software combination in a single session experiment. A tracking sheet for task

accuracy and accomplishment will be provided to the participants. They will be asked to record each

68

time they execute a task and the result for that task, when applicable. After interacting with the smart

phone and software, participants will be instructed to complete the following questionnaires: the

System Usability Scale (SUS), a Software Usability Questionnaire (adapted version of the Software

Usability Measurement Inventory (SUMI)), and the scale for Perceived Ease of Use (adapted from Davis

(1993)).

In this experiment the following hypotheses related to emotion elicitation will be tested:



negative emotions. We expect the differences between the initial and final emotions to be statistically

significant.

Ho10: The degree of participant extraversion will not be a significant factor influencing the

change of affect score.

The personality of the participant is thought to influence the generation of emotion and the willingness

or ability to communicate it using a self-report method. Specifically, the degree of extraversion may

affect the willingness of a participant to report his or her true affective state. A person with a high

degree of extraversion will be expected to share more information about his current emotional state.

Therefore, we are interested in testing the relevance of personality type to the change in emotional

score. The results of this hypothesis will provide insight explaining variances in the effects of the

emotion elicitation strategy and its effectiveness.

In this experiment the following hypotheses related to affective state and usability will be tested:

Ho11: The degree of experience with similar smart phones will not be a significant factor affecting


69

This hypothesis will serve to verify the assumption that using a similar product may bias the usability

assessment given. It will help to determine the degree to which previous experiences with similar

products affect perceived usability.

Ho12: The type of emotion will not be a statistically significant factor affecting perceived

usability.

This hypothesis will test the main purpose of this experiment. Previous studies on product judgment and

emotion support the idea that products are judged more positively in the presence of positive affect

(Gorn, Goldberg et al. 1993). In this research we believe there is a difference in perceived usability given

the type of emotion experienced prior to interacting with the system. Thus, we expect that the mean

usability for positive affect will be higher than the mean for negative affect. If the variable of emotion is

significant in determining perceived usability, then this will validate the fact that emotions prior to

interaction do affect perceived usability.

Ho13: The interaction of emotion and type of smart phone is not a statistically significant factor

affecting perceived usability.

Ho14: The interaction of emotion and the type of software is not a statistically significant factor


We will test interactions between the variables of emotion, smart phone and software, as we are

interested in possible variations in the effects of emotions on usability for different software/smart

phone scenarios. In this research, we will explore the effect of emotion based on different smart phone

input modalities and different levels of controlled usability in the software. Because there are limited or

no experiments on the effect of emotion on perceived usability as a function of product characteristics,

no formal assumption is made in this hypothesis.

70

Ho15-17: The type of emotion is not a statistically significant factor affecting other metrics related

to perceived usability.

This generic hypothesis represents the set of metrics related to usability that will be collected in the

experiment, along with perceived usability. In the case that significant differences in such metrics as a

function of the type of emotion are observed, the effect of emotion prior to interaction in perceived

usability will be supported.

3.4.1 Materials

A portable computer with Windows Vista Home Edition will be used to execute computer tasks. Director

MX will be used to program the interface for the computer task. The Recoding User Input (RUI) program

(Kukreja, Stevenson et al. 2006) will be used to record the task time and answers provided. An HTC Pure

smart phone with a virtual keypad will be used with a Windows Mobile 6.0 operating system. The

Blackberry Bold will be used as the cell phone with a tangible keypad. Figure 3.11 shows the two phones

that will be used.

Figure 3.11 Phones used in experiment 4.

71

These two smart phones represent different modalities of data entry. Our intention is to test the

different smart phone designs, using input modality as the main differentiator between the smart

phones.

Two different versions of a software program for personal finance management will be used. Software 1

follows good usability practices such as the appropriate use of colors, use of legible text, excellent

navigation capabilities, and appropriate organization of the information in context. In addition, visual

aids such as icons are used. In contrast, software 2 has a more complex navigation scheme, using dark

colors, very small letters, no organization of the information and intentional random errors and delays

(1-4 second delay in transitions). Software 2 represents the application with poor usability. Figure 3.12

(a) shows a screen view of software 1, while Figure 3.12 (b) shows the same view for software 2.


participants will be asked three questions to assess the importance of computer task goals. They will

also answer questions regarding usage and preferences for smart phones.

Participants will be assessed using the Big Five factor model of personality. This model of personality

uses five broad domains: Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism.

In this experiment, only the Extraversion dimension of personality will be examined. This dimension

refers to energy level and tendency to be connected with the external world. A numeric/continuous

variable will be created for the participants’ Extraversion scores. The Big Five questionnaire provides a

score for each one of the five categories, while the MBTI provides a classification in a personality type. In

our case, we decided to shift from MBTI to the Big Five personality profile because we want to have a

quantitative variable to assess the personalities of the participants. Please refer to Appendix D for more

details regarding the Big Five personality questionnaire.

72

1

Money Manager 3.5

Transaction Log Budget view Summary Reports

SettingsVisa accountBank of America

(a)

Money Management 3.5

About Money Manager 3.5

Retrieve report

Store new bank info

View all

Purchase full version

Accounts

Exit the program

Financial plan

Summary

Set up

(b)

Figure 3.12 (a) Home screen for software 1, and (b) home screen for software 2.

The Diag tasks will be used as task stimuli (Ritter and Bibby 2001) to generate positive and negative

emotions. To generate frustration, the interface will be programmed to have random freezes ranging

73

from 2 to 4.5 seconds. An instruction sheet will be provided to all participants in both trials. Please refer

to Appendix B.1 for the instruction sheet provided.

To measure emotions, the PANAS questionnaire will be used (Appendix D.3). To measure the usability of

the smart phone/software system, the SUS will be used (Appendix D.4). To measure the usability of the

software itself, the Software Usability Questionnaire will be used. This questionnaire is based on the

Software Usability Measurement Inventory (SUMI). It consists of 10 questions that will be strictly related

to the software’s usability. Please refer to Appendix D.5 for more details on the Software Usability

Questionnaire.

There are other variables such as willingness to use the system that may be related to perceived

usability. In this regard, three measures will be taken: attitude toward using the system, perceived

usefulness and perceived ease of use. Attitude toward using the system refers to “the degree of

evaluative affect that the individual associates with using the system” (Davis 1993). Perceived usefulness

refers to “the expected overall impact on task performance” while perceived ease of use “refers only to

those performance impacts related to the process of using the system” (Davis 1993).

The three questionnaires used to assess these metrics will be adopted from the scales developed by

Davis (1993) to validate his Technology Acceptance Model (TAM). These scales have been validated and

used by many researchers from diverse fields (Davis 1993). Please refer to Appendices D.6-D.8 for the

specific scales.

A task sheet will be provided to each participant to record task accomplishments and answers for tasks,

when required. This information will be used to estimate task completion rates and accuracy for

participant trials.

74


A Randomized Complete Block Design (RCBD) will be used for this experiment. This design divides the

experimental units into blocks, which constitute a single replication in the experiment. Blocking is a

technique commonly used to remove the effects of relevant nuisance variables (Montgomery 2009). A

blocking factor represents a source of variability that is not of primary interest. In this case, variability

between participants is a nuisance factor that we want to remove from the analysis. Thus, we will use a

block design to isolate the effects related to personal individual differences from the true effects of the

treatments themselves. The blocks will be homogeneous groups and within-block variability will be

smaller than between-blocks variability. This design will allow us to have a between-participant

experiment design and to make conclusions about differences in perceived usability.

In this experiment, each block will consist of eight treatments (all possible combinations of experimental

conditions). Figure 3.13 depicts the blocks, in which the order of the treatments will be randomly

assigned. In addition, we assume that the emotion elicited will be statistically the same for all

participants.

Block 1 Block 2 Block 3

Emotion Smart Phone Software Emotion Smart Phone Software Emotion Smart Phone Software

+ + + + + + + - -

+ + - + + - - + +

+ - + - - + + + +

+ - - + - + - - +

- + + - - - - - -

- + - - + + + - +

- - + - + - + + -

- - - + - - - + -

Figure 3.13 Block design for experiment 4.

75

The participants will be assigned to the corresponding blocks based when they participate in the session.

Once participants arrive, they will be randomly assigned to one of the eight possible combinations of

treatments. This experiment is therefore a between-participants design, where every participant

experiences only one of the eight possible conditions or treatments. Thus, in this experiment, different

participants will be used to test each one of the experimental treatments. Although it is a between-

participants design, it will provide the statistical means to isolate the variability associated with having

different participants assessing products under experimental conditions. Each block will replicate all

available experimental conditions. Each block is expected to have similar characteristics.

In summary, each participant will interact with one of the eight experimental conditions available. Thus,

the participants will experience one type of emotion and evaluate one type of smart phone and one

type of software only. The experiment will be conducted in a single session.

Table 3.4 describes the variables of the study and their categorization. The response variable of main

interest is the perceived usability of the smart phone/software system. Emotion indicates which type of

emotion is being elicited. The smart phone and software variables represent the type of system being

assessed. In this experiment, previous experience with similar phones and software applications will be

considered, along with participant extraversion quotients. Previous experience will be analyzed for any

relationship with the usability score provided. The extraversion quotient will be used to inspect for any

relationship with the emotion reported. Although not shown in the table, the interactions of these

variables will be inspected as well.

76


Variable type Variable Name Numerical / Categorical

Levels

Response (dependent variable)

Usability Score Numerical NA

Factors (independent

variables)

Emotion Categorical 2 (positive or negative)

Smart phone Categorical 2 (HTC or Blackberry)

Software Categorical 2 (good usability or poor

usability)

Extraversion Numerical NA

Previous Experience

Numerical NA

The other response variables related to usability will also be related to the factors presented in Table

3.4. The other response variables are: software usability, perceived ease of use, and task completion

rate.

3.4.3 Procedure for Experiment 4

The procedure for experiment 4 is depicted in Figure 3.14. This figure shows the sequence of the main

tasks to be executed by the participants in the experiment.

Consent

Personality questionnairePre-experiment questionnaire

PANAS questionnaire (1)Training on

two systemsEmotion

elicitation

Usability evaluation


PANAS questionnaire (2)

Figure 3.14 Sequence of tasks for experiment 4.

77

The participants will be assigned to a specific block upon arrival. A condition within the block will be

randomly assigned. Participants will be given the consent form and their signatures will be obtained. The

participants will be instructed to complete the personality questionnaire, the pre-experiment

questionnaire, and the PANAS questionnaire. They will be given a few minutes to interact with the smart

phone/software system. Then, the participants will be given the written instructions for the computer

task and allowed to execute the task as specified in Appendix A.

Immediately afterward, the participants will be instructed to complete the PANAS questionnaire again.

Then, they will be given the corresponding smart phone/software system, task sheet, and paper to

record task accomplishments and task answers, as appropriate. The participants will be allowed to

interact with the system for 10 minutes. Then, they will be asked to complete the questionnaires to

assess the system. At the end, the participants will be debriefed about the purpose of the experiment

and will receive their monetary compensation.

3.5 Summary

In summary, this research will be divided in two stages. The first stage, composed of three experiments,

will serve to validate our emotion elicitation strategy and will preview the effects of affective states on

perceived usability. The second stage will test the effects of affective states within different usability

scenarios, using a diverse range of usability metrics. These experiments should provide data

demonstrating the effects of emotion on usability assessments.

78

Chapter 4

Results from Experiments 1, 2 and 3

In this chapter, the results for experiments 1, 2 and 3 are provided. The purpose of this chapter is to

present the results of the exploratory experiments which were executed to gather data for the design of

experiment 4, which is the main experiment in this research work. The results served as the baseline for

the experimental parameters of experiment 4.

4.1 Results of Experiment 1

The purpose of Experiment 1 was to test the effectiveness of the strategy for emotion elicitation and the

number of stimuli needed to generate the emotion. We were also interested in testing three self-report

methods for emotion, the Positive and Negative Affect Schedule (PANAS), the Affect Grid (AG), and the

Self-Assessment Mannequin (SAM).

In this experiment, the participants were asked to complete a Diag task using a computer interface that

would elicit either positive or negative emotion. The participants were assigned to a type of emotion

and were asked to participate in two sessions, seven days apart. One session exposed the participants to

45 stimuli and the other session to 65 stimuli. The participants were asked to fill out the three self-

report measures (PANAS, SAM and AG), before and after the computer task.

Strategy effectiveness was assessed by comparing the initial emotion score to the post-task emotion

score. The number of stimuli was assessed by comparing the change in emotion score (final-initial) for

the conditions with 45 and 65 stimuli, respectively.

For this experiment, the participants were recruited from the Department of Industrial Engineering at

The Penn State University. The sample consisted of 30 participants, including 12 females and 18 males,

79

whose average age was 20.5 years (19-24 range) with junior or senior standing. A total of 18 participants

were classified as Extraverts based on the MBTI personality questionnaire.

4.1.1 Emotion Elicitation

We were interested in understanding the effectiveness of the emotion elicitation strategy at generating

positive and negative emotion. For that purpose, we formulated the following hypothesis:


To test this hypothesis, the emotion scores obtained at the beginning of the session were compared to

the emotion scores obtained after participant interactions with the computer task. The difference

between the initial and the final affective scores expressed the change in emotion after the emotion

elicitation task quantitatively.

The Wilcoxon Signed Rank test was applied to the differences between emotion responses before and

after the computer task. This test is a non-parametric test for repeated measurements on a single

sample (Wilcoxon 1945). It evaluates the equality of two population medians, and calculates the

corresponding point estimate and confidence interval. The Wilcoxon test was used in place of the t-test

because the data did not follow a normal distribution. The equivalent hypotheses are as follows:

Ho1: The median change in emotion will equal 0.

Ha1: The median change in emotion will not equal 0.

Individual analyses were conducted for each measurement method (PANAS, AG, SAM) and for each type

of emotion elicited (positive and negative). This allowed us to understand the sensitivity differences

among the self-report methods based on the type of emotion analyzed. In addition, segregating the

analyses by type of emotion provided detailed data on the validity of the emotion elicitation strategy for

the two emotions.

80

Table 4.1 shows the results of the Wilcoxon Signed Rank test for the positive affect. Table 4.2 shows the

results for the negative affect.

Table 4.1 Wilcoxon Signed Rank Test Results for Change in Emotion (Positive Affect)

Positive Affect

PANAS Affect Grid Self-Assessment Mannequin

PA NA Valence Arousal Valence Dominance Arousal

Mean 6.53 -1.93 0.43 1.17 0.80 0.33 0.77

Median 6.50 -1.50 0.00 1.00 1.00 0.00 0.00

St.Dev. 3.84 2.18 1.10 1.84 1.38 1.98 1.83

W 406.00 5.00 128.00 203.50 42.00 95.00 31.50

P-value p < 0.05 0.990 0.067 0.002 0.011 0.394 0.035

Table 4.2 Wilcoxon Signed Rank Test Results for Change in Emotion (Negative Affect)

Negative Affect



Mean -14.20 3.36 -0.50 -0.33 0.10 -0.37 -0.17

Median -13.50 2.27 0.00 0.00 0.00 0.00 0.00

St.Dev. 5.80 3.50 1.45 1.39 1.49 1.22 1.31

W 0.0 351.0 68.0 99.0 63.0 24.0 49.5

P-value p < 0.05 p < 0.05 0.102 0.242 0.530 0.079 0.570

In this test, we wanted to calculate the statistical difference between the initial emotion and the final

emotion after the computer task. For the positive affect condition, the following changes in the

emotional score were significant at a 95% confidence level: the PA dimension of the Positive and

Negative Affect Schedule (PANAS), the Arousal dimension of the Affect Grid (AG), and the Valence and

Arousal dimensions of the Self-Assessment Mannequin (SAM). Therefore, there was enough statistical

evidence to reject the null hypothesis HO1 which states that the median value of emotion change will be

equal to zero for the measures mentioned above. There was not enough evidence to reject the null

81

hypothesis for the following measures: the NA dimension of PANAS, the Valence dimension of the Affect

Grid, and the Dominance of the Self-Assessment Mannequin.

For negative affect, only the PA and NA dimensions of the PANAS method showed statistical

significance. Therefore, there was enough statistical evidence to reject the null hypothesis HO1 which

states that the median value of the change in emotion will be equal to zero for both the PA and NA

measures of the PANAS method. In the negative affect condition, there was not sufficient evidence to

reject the null hypothesis for the Affect Grid and the SAM methods.

Based on the experiments from which the PANAS method was developed, the positive and negative

affect dimensions are independent, and the correlation between them is invariably low (Watson, Clark

et al. 1988). Thus, a change in one dimension does not imply a change in the other. As a result, we only

focused on the significance of the PA scores in the positive affect condition and the NA scores in the

negative affect condition.

The results corresponding to the PANAS scores suggest that there was a statistically significant change

between the initial and final emotional scores. The only method able to detect such effects was the

PANAS, given the statistical significance of the PA and NA parameters in the positive and negative affect

conditions, respectively. We suspect that this was due to the fact that PANAS is the only method that

asks participants to assign numeric values to their emotions. Because our sample was comprised of

students from the Industrial Engineering Department who tend to be very quantitative, it is quite likely

that they felt more confident using numbers to describe their emotions. We were also interested in

knowing the statistical power of the PANAS method at detecting a maximum difference of 3.5 between

emotion scores. The PA dimension of PANAS had a power of 98%. For the NA dimension, the power was

72%.

82

In summary, there was sufficient evidence to reject the null hypothesis Ho1 presented in this section for

the PANAS method only. This implies that, based on the PANAS measurement method, there was a

significant difference between the initial emotion score reported prior the computer task and the final

emotion after the task. Therefore, based on the results obtained using the PANAS method, the emotion

elicitation strategy was effective at provoking significant changes in the participants’ emotions. This

effect was not observed for the Affect Grid and the SAM methods.

Based on these results, we decided to use the emotion elicitation strategy proposed in Appendix A and

the PANAS method to assess the emotional states of the participants in the next set of experiments.

4.1.2 Stimuli and Other Variables

We were interested in knowing if there was a significant difference in the emotions elicited by the

computer game with 45 stimuli and the one with 65 stimuli. For that purpose, the following hypothesis

was formulated:

Ho2: The mean difference in emotion score for the computer game with 45 stimuli will be equal

to the mean difference in emotion score for the computer game with 65 stimuli.

To test this hypothesis, the change between the emotion responses (emotion score after the computer

task – emotion score before the computer task) was used as the response variable. The change in

emotion represents the intensity of the emotion elicited. Thus, it is used to test the effectiveness of

using the game with 45 stimuli compared to the one with 65 stimuli. The categorical variable Nstimuli

was used to classify the observations in either the 45 stimuli trial or the 65 stimuli trial.

The equivalent hypotheses are as follows:

Ho2: The median of the differences between the change in emotion for 45 stimuli and 65 stimuli

will equal 0.

83

Ha2: The median of the differences between the change in emotion for 45 stimuli and 65 stimuli

will not equal 0.

To test this hypothesis, the Wilcoxon Signed Rank test was applied to the difference between the

emotion score change with 45 stimuli and the emotion score change with 65 stimuli. The Wilcoxon test

is a non-parametric version of the t-test, and it is used in cases such as this where samples do not follow

normal distributions (Wilcoxon 1945). Table 4.3 summarizes the descriptive statistics for the change in

emotion score values by type of stimuli observed. Table 4.4 summarizes the results of the Wilcoxon

Signed Rank test for each measurement method.

Table 4.3 Descriptive Statistics for the Change in Emotion Score Values

Stimuli 45 65 45 65 45 65 45 65 45 65 45 65 45 65

Mean 6.40 6.67 -2.00 -1.87 0.20 0.67 1.53 0.80 -0.67 -0.99 0.20 0.47 -1.00 -0.53

Median 6.00 7.00 -2.00 -1.00 0.00 1.00 1.00 1.00 -1.00 -1.00 0.00 0.00 0.00 0.00

St.Dev. 4.08 3.71 2.42 1.99 1.20 0.98 2.20 1.37 1.54 1.22 2.31 1.68 1.89 1.81

Stimuli 45 65 45 65 45 65 45 65 45 65 45 65 45 65

Mean -14.47 -13.93 3.66 3.07 -0.80 -0.20 -0.11 -0.53 0.60 -0.40 -0.53 -0.20 -0.40 0.07

Median -14.00 -13.00 4.00 3.00 -1.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

St.Dev. 5.32 6.42 1.58 2.81 1.42 1.47 1.30 1.13 0.99 1.76 1.13 1.32 1.29 1.34

NA Valence Arousal Valence Dominance

PA NA Valence Arousal Valence

Positive Affect


PA

Dominance Arousal

Arousal

Negative Affect


Table 4.4 P-values for the Wilcoxon Signed Rank Test for Each Measurement Method



Nstimuli 0.773 0.652 0.107 0.375 0.135 0.451 0.217

Sequence 0.326 0.790 0.756 0.620 0.784 0.438 0.026

Session 0.559 0.888 0.326 0.271 0.971 0.511 0.455

Personality p<0.05 0.993 0.291 0.166 0.230 0.784 0.087

Goal importance p<0.05 p<0.05 0.384 0.797 0.706 0.143 0.109

84

The factor representing the number of stimuli was not significant at 95% confidence for all

measurement methods. The high p-values observed implied that there was not enough evidence to

reject the null hypothesis. Therefore, we could not conclude that the computer task with 45 stimuli

elicited more or less emotion than the one with 65 stimuli. In addition, there was not enough statistical

evidence to say that the experimental factors sequence and session were statistically significant to the

change in emotion score.

Participant personalities are thought to influence emotion generation and the willingness or ability to

self-report them. In studies using functional magnetic resonance imaging (FMRI) technology, brain

responses to positive emotions were positively correlated with the degree of extraversion (Revelle and

Scherer 2009). Conversely, there was no correlation between brain responses to negative emotions and

the degree of extraversion (Revelle and Scherer 2009). Therefore, we tested the relevant scores to see if

personality had a significant effect on positive affect scores. Results of this research suggest that

personality significantly influences the way the positive emotions are generated and experienced. The

results agree with previous research related to the relationships among positive affect and personality,

which found positive affect to be significantly associated with the degree of participant extraversion

(Shiota, Keltner et al. 2006).

Our emotion elicitation strategy is based on the appraisal theories of emotion, which propose that for

an emotion to be elicited, a goal or intended value has to be present. If there are favorable conditions to

achieve the desired goal, the emotion generated should be positive. In contrast, if there are obstacles

for goal attainment, the emotion generated should be negative. According to the theories, the intensity

of the emotion generated should be determined, for the most part, by the relative value and importance

of the given goal. Therefore, we expected the goal importance to be a significant factor affecting the

change in emotion resulting from participation in the computer task. The results suggest that the goal

85

importance factor is indeed significant at a 95% confidence level for the emotion metrics reported in the

PANAS method (p < 0.05).

The post-experiment questionnaire is summarized in Table 4.5. About 64% of participants in the positive

affect condition did not suspect the real purpose of the experiment. About 90% of the participants

thought that 7 days was enough time to forget about the task. All participants reported that they

experienced an emotion in the positive affect condition. For the negative affect, 98% reported that they

felt a negative emotion. The results of this questionnaire helped us to understand the effectiveness of

our emotion elicitation strategy.

Table 4.5 Post-Experiment Questionnaire

Question Positive Affect

Negative Affect

Stimuli preference

45 stimuli 57% 18%

65 stimuli 43% 82%

Suspected

No 64% 60%

A little 36% 27%

Yes 0% 9%

Method

PANAS 73% 30%

AG 18% 40%

SAM 9% 30% 7 days enough to forget and create emotions?

Yes 90% 90%

No 10% 10%

Emotion generated?

Yes 100% 98%

No 0% 2%

86

4.1.3 Summary

In this experiment, the methodology for emotion elicitation was tested and found to be effective. Thus,

the emotion elicitation strategy was validated. In addition, the results suggested that there was no

significant difference between the emotions generated by the computer task version with 45 stimuli and

the one with 65 stimuli.

Additionally, we tested the effectiveness of three self-report methods of emotion. A significant change

in emotion was observed only for the PANAS method. Thus, we adopted this method to assess

participant emotions, as it seemed to be the most sensitive to emotional changes.


The main purpose of experiment 2 was to test the effects of emotion on perceived usability. We were

also interested in testing two methods to assess usability, the System Usability Scale (SUS) and the IBM

Post-Study System Usability Questionnaire (PSSUQ).

The experiment was divided into two sessions, seven days apart. During each session, a different

emotion was elicited in the participants (positive or negative). The participants were assigned to a

specific sequence of emotions (e.g., positive in the first session, negative in the second session). In this

experiment, positive and negative affect were induced in the participants with the computer task and

methodology tested in experiment 1. They were then asked to interact with a software application on a

smart phone to execute a series of tasks. At the end of the interaction, they were asked to complete the

SUS and PSSUQ questionnaires to assess system usability. The effect of emotion on perceived usability

was assessed by comparing the usability scores for the positive and the negative affect conditions.

87

A total of 27 participants were recruited for this experiment from the Department of Industrial

Engineering at The Penn State University. The sample consisted of 13 females and 14 males, with an

average age of 21.2 years (range of 20-24 years) who had either junior or senior standing.

4.2.1 Emotion Elicitation Verification

Before analyzing the effect of emotion on perceived system usability, we needed to validate the fact

that emotions were indeed elicited. Thus, the following hypothesis was tested:


To test this hypothesis, the emotion scores obtained at the beginning of the session were compared to

the emotion scores obtained after completion of the computer task. In the same fashion as experiment

1, the differences in emotion scores represented the effectiveness of the emotion elicitation strategy.

Table 4.6 provides a summary of descriptive statistics for the PANAS scores reported in the experiment.

The Wilcoxon Signed Rank test was applied to the difference between the emotion scores before and

after the computer task. The Wilcoxon test is used in cases such as this when the data do not necessarily

follow a specific distribution (Wilcoxon 1945). The equivalent hypotheses are as follows:



Table 4.6 Descriptive Statistics for the PANAS Scores in Experiment 2

88

Based on the Wilcoxon test, the changes in emotion scores were significant at a 95% confidence level,

for both the PA and NA dimensions (p < 0.05). Thus, emotions changed after the computer task.

4.2.2 Effect of Emotion on Usability

We were interested in knowing if there was a significant difference in perceived usability under the

positive and negative affect conditions. For that purpose, the following hypothesis was formulated:

Ho4: The differences in mean usability scores will be equal for the positive and negative affect

conditions.

The mean usability score was used to assess perceived usability of the system. It was estimated using

the scores obtained from the SUS and the IBM PSSUQ. The mean usability score was found to be lower

for the negative affect condition than for the positive affect condition. Table 4.7 summarizes the values

for perceived usability reported in the positive and negative conditions.

Table 4.7 Mean Values of Reported Perceived Usability

Mean Usability

SUS IBM

Positive emotion 65.65 3.90

Negative emotion 52.96 3.26

An analysis of variance was conducted to test the significance of emotion for usability responses.

Normality of the response variables was confirmed. Other factors such as session and sequence of

stimuli were analyzed as well. Individual analyses of variance were done for the responses from each

usability assessment method (SUS, IBM PSSUQ). Table 4.8 summarizes the results of the analysis of

variance for each measurement method.

89

Table 4.8 ANOVA Results for Experiment 2

The equivalent hypotheses were as follows:

Ho4: µpositive affect - µnegative affect = 0

Ha4: µpositive affect - µnegative affect ≠ 0

At 95% confidence, the factor representing emotional state was significant for both of the usability

measurement methods used. There was sufficient evidence to support that the mean values of usability

were not equal for the two levels of emotion. Thus, we rejected the null hypothesis, implying a

difference in perceived usability based on user emotional states.

Other factors such as the emotion sequence (e.g., positive emotion in the first session, negative in the

second session) and the session were not significant. This implies that there was no statistical evidence

to infer that the changes in usability were due to the order in which the emotions were experienced, or

the session in which the systems were assessed. The statistical power for the SUS method to detect a

maximum difference of 14 was 76%. For the IBM PSSUQ, the power to detect a maximum difference of

0.9 was 79.8%.

We were also interested in understanding the degree to which previous experiences with similar

systems affected perceived usability. For that purpose, the following hypothesis was formulated:

90

Ho5: The degree of experience with similar smart phones will have no statistical effect on


The previous experience with the phone was assessed by asking the participants the input modalities of

their cell phones. They were asked if their phones had keypads or touch screens. This variable should

reflect the degree to which the participants felt familiar with the smart phones used in the study. Refer

to Appendix C.1 for the specific question asked.


Ho5: µusability for previous experience 1 - µusability for previous experience 2 = 0

Ha5: µusability for previous experience 1 - µusability for previous experience 2 ≠ 0

In this case, an ANOVA was conducted to detect any effect of previous experience with the input

modality of the phone on perceived usability. Table 4.9 shows the results of the ANOVA test.

Table 4.9 ANOVA Results for SUS and IBM Usability Questionnaire and Previous Experience

For both usability questionnaires the p-values were greater than 0.05. Thus, there was not enough

evidence to reject Ho5, or to establish significant differences in perceived usability as a function of

previous experience with a similar phone.

4.2.3 Summary

In this experiment, we were able to test for the effect of emotional states on perceived usability for a

smart phone/software system. There was statistical evidence to infer an effect of emotion on perceived

product usability. We were able to capture differences in perceived usability by using self-report

91

methods of usability, the SUS and the IBM PSSUQ. Although both methods were effective a capturing

the effect in perceived usability, the SUS method was more sensitive to changes.


The purpose of this experiment was to inspect for differences in emotion elicitation and perceived

usability as a function of the days elapsed between sessions.

In this experiment, each participant experienced two types of emotions (positive and negative) in

separate sessions. The sessions were five days apart for half of the participants, and 10 days apart for

the other half. For each group, half of the participants experienced the positive emotion condition in the

first session and half of the participants experienced it in the second session. This experiment was

executed following the same procedures as experiment 2. The only variation was to have half of the

participants come for their second session five days after their first session, and the other half come 10

days after their first session.

For this experiment, participants were recruited from the Industrial Engineering department at The

Penn State University. The sample consisted of a total of 16 participants (8 females and 8 males), with

an average age of 22.6 years (range of 20-24 years) with either junior or senior standing. The

participants were different than those in experiments 1 and 2.

4.3.1 Emotion Elicitation Verification

Before analyzing the effect of emotion on perceived usability of the system, we needed to validate the

fact that emotions were elicited. Thus, the following hypothesis was tested:


92

Following the same analysis from experiment 2, the emotion score obtained at the beginning of the

session was compared to the final score reported after the computer task. Table 4.10 provides a

summary of descriptive statistics for the PANAS scores reported in the experiment.

Table 4.10 Descriptive Statistics for the PANAS Scores in Experiment 3

The Wilcoxon Signed Rank test was applied to the difference between emotion scores before and after

the computer task. The Wilcoxon test is an alternative to the t-test for data such as this that do not

follow any specific distribution. The equivalent hypotheses are as follows:



Based on the Wilcoxon test, the change in emotion scores was significant at a 95% confidence level for

both the PA and NA dimensions (p < 0.05). This implies that there was sufficient evidence to reject Ho6.

Thus, we may infer that because the emotion score prior the computer task and the one reported after

the computer task were different, emotion was generated by the computer task.

4.3.2 Effect of Emotion and the Time between Sessions on Perceived Usability

Our primary interest in this experiment was to assess the effect of the time between sessions on the

perceived usability of the smart phone/software system. We intended to gain a better understanding of

the possible effects of the days elapsed between the first and second session as experiment parameters.

Because most similar experiments use a between-participant design (Gardner 1985; Gorn, Goldberg et

al. 1993; Han, Yun et al. 2001), limited data exists on optimal time between sessions. Initially, we used

93

seven days as a baseline for elapsed time between sessions. This experiment was executed to clarify

possible biases in the proposed procedure in experiment 2 related to remembering the smart phone and

the software. In addition, we were concerned about the possibility of learning the system during the

first session and the effects of initial impressions.

Therefore, this experiment served as the baseline for exploring such effects. In this case, the changes in

perceived usability were assessed with respect to the time between sessions. We understood that

comparing differences in perceived usability for two conditions of days between sessions would not

provide information regarding the amount of learning or remembering. However, it provided a sufficient

understanding of possible judgmental biases involved when evaluating the same product at two

different times. The perceived usability scores assessed the presence of such biases.

For this experiment the following hypotheses were formulated:

Ho7: Mean usability scores will be equal for the positive and negative affect conditions.

This hypothesis was created to investigate the expected effect of emotion on perceived usability. The

mean usability score for positive affect was expected to be higher than the mean score for negative

affect. The mean usability score was used to assess perceived system usability. It was estimated by the

scores obtained from the SUS and the IBM PSSUQ.

Ho8: The time between sessions is not a significant variable affecting perceived usability.

To test this set of hypotheses, we executed a regression analysis with the dependent variable usability

score as the response variable, and the variables emotion and days between sessions as factors in the

regression. We used the p-value for the coefficients corresponding to the factors to verify the

hypotheses.

94




Ho8: µ5 days between sessions- µ7 days between sessions = 0

Ha8: µ5 days between sessions- µ7 days between sessions ≠ 0

Table 4.11 shows the descriptive statistics for the change in usability scores. Table 4.12 shows the

results from the regression model with change in usability score as the response variable.

Table 4.11 Descriptive Statistics for Experiment 3

Table 4.12 Regression Results for Experiment 3 (R-Sq. 25.9%)

At 95% confidence, only the variable emotion was statistically significant for perceived usability.

Therefore, we rejected the null hypothesis Ho7, inferring differences in perceived usability due to

emotions experienced prior the interaction with the product.

For the variable days between sessions there was not enough evidence to reject Ho8. Thus, we were

unable to detect any significant difference in usability scores as a function of the days between sessions

at a 95% confidence level.

95

It is relevant to point out that the coefficient of the variable emotion was greater than the standard

error for the variable days between sessions (standard error of 5.40). Thus, the contribution of the

variable emotion was higher than the variance introduced by the variable days between sessions.

In addition, we executed an analysis of variance for the change in usability scores for positive and

negative affect sessions and the days between sessions. This test served to verify the effect of the days

between sessions on the change in usability scores observed between positive and negative affect. The

variable days between sessions was not statistically significant to changes in usability scores at a 95%

confidence level (p > 0.05).

A Cohen’s power analysis was executed to estimate the effect size (Cohen 1988). The Cohen’s d

parameter represents the effect size between the mean changes in usability observed for both days

between sessions conditions. It provides an estimate of the difference between the two changes in

usability estimates and the relationship between the number of days between sessions and changes in

perceived usability. The effect size measures the strength of the relationships between usability scores

and the days between sessions variable. Thus, the parameter can be used to determine if the observed

difference matters.

In this experiment, the effect of the variable days between sessions was not significant. However, it is

appropriate to mention that the value for the Cohen’s kappa was small, which implies that the possible

effect of time between sessions on perceived usability may not be detected with the sample size

selected. Thus, the sample size of 16 was not adequate to capture an effect of the days elapsed between

sessions in the perceived usability. For this particular experiment, this implies that the sample size

selected was insufficient to make any inferences about the effect of the number of days elapsed

between sessions. Thus, it is necessary to collect more data for this experiment to be able to detect any

related significant differences in perceived usability.

96

4.3.3 Summary

In this experiment, we were able to validate the emotion elicitation strategy in the same fashion as in

experiment 2. In addition, we were able to observe the effect of emotion on the perceived usability of

the smart phone/software system. The results show no statistical significance of the variable

corresponding to the number of days between sessions. Thus, there was no statistical evidence of any

effect of the time elapsed between sessions on the perceived usability of the product.

4.4 Summary of the Results for Experiments 1, 2, and 3

The first three experiments provided insights for the design of the fourth experiment. Experiment 1

validated the proposed emotion elicitation strategy and helped us select the appropriate instrument to

measure emotion. Experiments 2 and 3 confirmed the general hypothesis surrounding the effect of

emotion on perceived usability prior to interaction with the smart phone/software combination.

The following outcomes were taken into consideration when designing experiment 4, our main

experiment.

1. The proposed emotion elicitation strategy was validated by significant changes in emotion

observed in experiments 1, 2, and 3.

2. There was no statistical difference between the change in emotion reported for 45 stimuli and

the one reported for 65 stimuli.

3. The Positive and Negative Affect Schedule (PANAS) best measured emotion scores.

4. The System Usability Scale (SUS) best measured perceived usability.

97

Chapter 5

Results and Discussion for Experiment 4

This chapter provides a detailed analysis of the results for Experiment 4 and provides evidence

supporting emotional effects on the perceived usability of products.

The main objective of experiment 4 was to test the effects of prior emotional states on perceived

usability. We were interested in exploring such effects based on different smart phone characteristics

and controlled usability scenarios within a smart phone software application. We explored specific

aspects of usability such as software usability, perceived ease of use and task completion.

In this experiment, four different smart phone/software combinations were used. Two smart phones

had different modalities for data input. Smart phone 1 used a keypad while smart phone 2 used a touch

screen. We also tested two different versions of a software application to be used on the smart phone.

Software 1 was designed following principles of good usability while software 2 was not.

A total of three factors at two levels each were tested in this experiment. The factors were: emotion

(positive or negative), smart phone (1 or 2), and software (1 or 2). This experiment was executed

following a randomized complete block design (RCBD). Each block represented the eight possible

combinations of factor levels, which were randomly assigned to the participants as they arrived.

Participants were the blocked factor, as we wanted to exclude individual differences from the analysis.

In this experiment, the participants were asked to execute a computer task to elicit positive and

negative emotions. Their emotions were measured using the PANAS questionnaire. Then, the

participants were asked to execute a series of tasks using one smart phone/software combination in a

98

single session. After the interaction with the smart phone and software, the participants were instructed

to complete the following questionnaires: the System Usability Scale (SUS), a Software Usability

Questionnaire, a Perceived Usefulness Scale, and the Perceived Ease of Use Scale (adopted from Davis

(1993)).

Although most participants were recruited from the Industrial Engineering department at Penn State

University, some participants were members of the College of Engineering who had not yet declared

majors. A total of 40 participants were used in this study. The sample consisted of 33 male and 7

female students with an average age of 20.8 years. The average score for the degree of extraversion was

40.6. Table 5.1 shows a summary of participant characteristics.

Table 5.1 Summary of Participant Characteristics

Mean Age Familiar with

Blackberry Familiar with HTC Extraversion level

Owned a Smart Phone

20.80 27.50% 27.50% 40.60 57.50%

A majority of participants in this study owned a smart phone. About 27% of the participants were very

familiar with the Blackberry brand (smart phone 1) and 27% were familiar with the HTC phone (smart

phone 2).

The effects of emotion on perceived usability were assessed by inspecting the degree of significance of

the variable emotion in the analysis of variance, where the SUS score represented perceived usability,

our response variable. The results of the experiment and detailed explanations are provided in the

following sections.

99

5.1 Hypotheses Related to Emotion Elicitation

The goal of this analysis was to validate the effectiveness of the emotion elicitation strategy. Please

refer to Appendix A for a detailed explanation of the emotion elicitation strategy.

Initial positive and negative emotions of the participants were compared to the final positive and

negative emotions reported after the computer task. Figure 5.1 shows the histograms for initial and final

positive emotions. Figure 5.2 shows the histograms for initial and final negative emotions.

Figure 5.1 Histogram of the initial and the final positive affect.

Figure 5.2 Histogram of the initial and the final negative affect.

100

From both figures it can be noted that the data do not follow normal distributions. Thus, non-parametric

methods were used to analyze the data.

The ability of the strategy to elicit the desired emotion was assessed by the following hypothesis:

Ho9: The mean initial affect score will be equal to the mean final affect score.

As in the previous experiments, the emotion scores obtained at the beginning of the session were

compared to the emotion scores obtained after the computer task. The Positive and Negative Affect

Schedule (PANAS) was used to assess emotional states before and after the computer task. On the

PANAS scale, the value of PA increases with positive affect; likewise, the value of NA increases with

negative affect.

Table 5.2 contains the positive and negative affect scores reported before and after the computer task.

Table 5.2 Descriptive Statistics for the PANAS Scores for Experiment 4

Positive Affect Negative Affect

Initial

PA Initial

NA Final PA

Final NA

Initial PA

Initial NA

Final PA

Final NA

Mean 21.90 10.50 20.70 10.15 21.90 11.10 19.30 11.20

Median 25.00 9.00 21.50 9.00 23.00 10.00 20.00 10.50

St. Dev. 6.66 3.54 5.48 3.10 6.19 4.35 5.80 3.99

The following observations are based on Table 5.2:

1) For positive affect condition, the mean value of PA increased and the mean value of NA

decreased after the emotion elicitation task.

2) For the negative affect condition, the mean value of NA increased and the mean value of PA

decreased after the emotion elicitation task.

101

The Wilcoxon Signed Rank test was applied to the differences between emotional scores before and

after the computer task to test the following equivalent hypotheses:



The Wilcoxon test was conducted because the data did not follow a normal distribution. This test is the

non-parametric equivalent to the t-test. Individual analyses were done for each type of affect (positive

and negative). Table 5.3 shows the results of the Wilcoxon Signed Rank test for the positive and negative

affect conditions.

Table 5.3 Wilcoxon Signed Rank Test Results for Change in Emotion for Experiment 4

Positive Affect Negative Affect

PA NA PA NA

Mean 4.80 2.65 7.10 4.00

Median 5.00 2.00 6.50 4.00

St. dev. 3.34 2.77 4.35 2.88

W 666.00 595.00

P-value p < 0.05 p < 0.05

Based on the Wilcoxon test, the change in emotion scores were significant at a 95% confidence level for

both the PA and NA dimensions (p < .05). Thus, we rejected Ho9; there was a change in emotion after

the computer task. This result supports the validity and effectiveness of the emotion elicitation

strategy.

In addition to validating the fact that positive and negative affect were elicited, we were interested in

examining participant personality differences and the degree of affect expressed on the positive and

negative affective scale (PANAS). The next hypothesis, Ho10, tested the relevance of personality to

changes in affect scores.

102

Ho10: The degree of participant extraversion is not a significant factor influencing changes in the

affective scores.

We believed that the changes in emotion reported by the participants may have been influenced by a

willingness or disposition to disclose feelings on the PANAS questionnaire.

For analysis purposes, two regressions were conducted: one used the change in positive affect as a

response and the other used the change in negative affect as a response.


Ho10: The population medians are all equal for the change in emotion = 0

Ha10: The population medians are not all equal for the change in emotion ≠ 0

The variable extroversion was tested as a response variable. Table 5.4 shows the regression results for

both response variables.

Table 5.4 Regression Results for Changes in Positive and Negative Affect as Response Variables

Response Predictor Coefficient T value P value

PA_Change Extraversion 0.039 1.44 0.160

NA_Change Extraversion 0.023 1.12 0.271

Based on this table, changes in emotion appear to be unrelated to the degree of extraversion.

103

Based on the p-value obtained, there is not enough statistical evidence to infer that the variable

extraversion is a predictor of reported changes in affect. In this experiment, the extraversion metric was

not a significant predictor of the reported changes in affective state. This suggests that the changes in

affect reported were not determined by levels of participant extraversion.

In the next section, the results of the hypotheses related to the effect of emotional states on perceived

usability of the smart phone/software system are discussed.

5.2 Hypotheses Related to the Effects of Affective States on Usability

We were interested in understanding the effects of emotion on perceived usability as well as variations

in such effects under different scenarios of controlled software usability and for different smart phone

input modalities.

Before investigating the effect of emotion on perceived usability, we were interested in understanding

the extent to which previous experiences with similar products may have affected perceived usability.

The following hypothesis was formulated for that purpose:

Ho11: The degree of experience with similar smart phones is not a significant factor affecting


For this hypothesis, we tested three different measures to account for previous experience with the

smart phones. The following measures were used to account for degree of experience:

1) Degree of familiarity with the HTC and Blackberry phones. A question was asked in the pre-

experiment questionnaire regarding the participant’s familiarity with the HTC and Blackberry

phones. Answers were provided using a Likert scale from 1 to 5, where 1 was extremely

unfamiliar and 5 was extremely familiar. The variable familiarity with phone was assigned

accordingly.

104

2) Duration of ownership. Participants were asked if they owned a smart phone. If so, they were

asked to select one of three alternatives best reflecting the amount of time they had owned the

phone. The variable duration of ownership was constructed by assigning values from 0 to 2

corresponding to the answers provided, as shown below.

Never = 0

6 months to 1 year = 1

More than 1 year = 2

3) Familiarity with input modality. This variable measured the degree of familiarity with touch

screen and keypad-based smart phones. Because we used input modality as a differentiator

between phones, it was important to account for individual differences in familiarity for both

types of input modalities. To estimate familiarity with input modality, the participants were

asked if their smart phones were touch screen or keypad-based. A value of 1 was assigned if a

participant’s smart phone was congruent with the smart phone used in the experiment.

Table 5.6 shows the participant distribution for these three questions.

Table 5.6 Description of Previous Participant Experiences

Own a Smart Phone

Length of Ownership Touch Screen or

Keypad

Yes No Never 6 mos. -

1 yr. > 1 yr. Touch Screen Keypad

23 17 17 10 13 8 15

58% 43% 43% 25% 33% 20% 38%

The three variables were considered covariates to the analysis of variance done for system usability. The

use of variables that account for the degree of experience as a covariate should isolate the effects of

105

previous experience on system usability ratings from the effects of other variables. Thus, this set of

variables was expected to diminish biases associated with individual experiences with similar phones.

In the same fashion, the extraversion score reported in the Big Five personality test was used as a

covariate in the ANOVA test for system usability. The degree of extraversion may affect willingness to

share information regarding experience with the system. Thus, the system usability reported may have

been related to degree of extraversion. Because we were using self-report methods for system usability,

using the degree of extraversion as a covariate should have isolated the effect of participant willingness

to share information from the true effects of the other variables on the system’s perceived usability.

The following two hypotheses tested the significance of the variable emotion on the score for perceived

usability and the specific effects for different hardware/software scenarios. These hypotheses were

relevant to the main objective of this research. The hypotheses were:

Ho12: The type or level of emotion is not a statistically significant factor affecting perceived

usability.

If emotion was a significant factor influencing perceived usability, it would serve as evidence that

emotions prior to product interactions affected the perceived product usability.

Ho13: The interaction of emotion and type of smart phone is not a statistically significant factor


Ho14: The interaction of emotion and type of software is not a statistically significant factor


We tested the interactions among the variables of emotion, smart phone, and software, as we were

interested in possible variations in the effects of emotion on usability for different smart

phone/software scenarios.

106

Table 5.7 summarizes the descriptive statistics for the response variable, perceived usability, which

corresponds to SUS scores. The higher the SUS score, the higher the perceived usability. Table 5.8

summarizes the descriptive statistics for the SUS scores with respect to each level of the three factors:

emotion, smart phone, and software.

Table 5.7 Descriptive Statistics for SUS Scores

Positive Affect

Negative Affect

Mean 63.75 40.63

St. Dev 21.62 20.95

Table 5.8 Descriptive statistics for SUS Scores for Each Level of Emotion, Smart Phone and Software

Positive Emotion Negative Emotion

Smart Phone 1 Smart Phone 2 Smart Phone 1 Smart Phone 2

Software 1 Software 2 Software 1 Software 2 Software 1 Software 2 Software 1 Software 2

Mean 72.50 60.00 56.00 66.50 46.50 31.50 46.00 38.50 St. dev 25.90 11.59 30.10 17.73 21.84 11.81 34.30 10.09

Positive Emotion Negative Emotion

Software 1 Software 2 Software 1 Software 2

Smart

phone 1 Smart

phone 2 Smart

phone 1 Smart

phone 2 Smart

phone 1 Smart

phone 2 Smart

phone 1 Smart

phone 2

Mean 72.50 56.00 60.00 66.50 46.50 46.00 31.50 38.50 St. dev 25.90 30.10 11.59 17.73 21.84 34.30 11.81 10.09

Based on Table 5.7 we observed that the mean value for perceived usability was higher for positive

emotion than for negative emotion. In addition, the perceived usability values for positive emotion at

different levels of software and smart phone were consistently higher than the ones observed for the

equivalent conditions in the negative affect treatment.

The usability of the software was controlled; software 1 was intentionally designed to have better

usability than software 2. In Table 5.8 we also observed that the mean value for the perceived usability

107

of software 2 was higher than the perceived usability for software 1, specifically for the scenarios where

smart phone 2 was used. This result was unexpected and not in congruence with the software design.

However, this may imply that there was an interaction of the perceived usability of the smart phone and

the perceived usability of the software. In other words, the effect of the perceived usability of the smart

phone may have overridden the effect of the perceived usability of the software in the perceived

usability of the overall system.

For both positive and negative emotion scenarios, and when software 1 was being used, the perceived

usability of smart phone 1 was higher than the perceived usability of smart phone 2. In contrast, when

software 2 was being used, the perceived usability of smart phone 2 was higher than perceived usability

of smart phone 1. These results may be due to an interaction between software 2 and smart phone 2.

Such interactions may have promoted perceptions of better usability for software 2 under the positive

emotion condition, and for smart phone 2 for both positive and negative emotion.

Figure 5.3 shows the interval plot and boxplot of perceived usability for positive and negative affect. In

these plots, the differences between perceived usability for positive affect and negative affect are

evidenced by the separation between the two intervals. Perceived usability was noticeably higher in the

positive emotion condition.

Software

SU

S S

co

re

21

100

80

60

40

20

0

Boxplot of SUS Score vs Software

Emotion

SU

S S

co

re

NegativePositive

80

70

60

50

40

30

Interval Plot of SUS Score vs Emotion95% CI for the Mean

Figure 5.3 Interval plot and boxplot of perceived usability scores for positive and negative affect.

108

Figure 5.4 shows the interval plot and boxplot of perceived usability for the two smart phones used.

Based on these plots, there seems to be no difference between perceived usability for smart phone 1

and smart phone 2.

Figure 5.4 Interval plot and boxplot of perceived usability scores for smart phone 1 and smart phone 2.

Figure 5.5 shows the interval plot and boxplot of perceived usability for the two software applications

used. Based on these plots, there seems to be no difference between perceived usability scores for

software 1 and software 2.

Figure 5.5 Interval plot and boxplot of perceived usability scores for software 1 and software 2.

109

To test these hypotheses, the values obtained from the System Usability Scale (SUS) were used to

represent perceived usability. An analysis of variance with the corresponding model for a randomized

complete block design was used. The assumption for normality was verified and met.

The equivalent hypotheses were:



Ho13: µSmart phone*Emotion - µSmart phone*Emotion = 0

Ha13: µSmart phone*Emotion- µSmart phone*Emotion ≠ 0

Ho14: µSoftware*Emotion - µSoftware*Emotion = 0

Ha14: µSoftware*Emotion- µSoftware*Emotion ≠ 0

110

The results of the ANOVA are summarized in Table 5.9.

Table 5.9 SUS Score ANOVA for Experiment 4

Source F P-value

Familiarity 0.38 0.547

Time Owned 0.91 0.354

Input Familiarity 0.73 0.403

Extraversion 2.69 0.120

Emotion 5.64 p < 0.05

Smart Phone 0.47 0.501

Software 0.09 0.767

Emotion*Smart Phone 0.27 0.610

Emotion*Software 1.89 0.187

Smart Phone*Software 2.27 0.150

Emotion*Smart Phone*Software 0.00 0.953

Based on the ANOVA results, the interaction terms were not statistically significant to the perceived

usability scores. Thus, we failed to reject the null hypothesis Ho13.

A t-test was executed for the interaction of the software and emotion variables. The interaction was

significant at a 95% confidence level (p-value <0.05). Thus, there was a statistically significant interaction

between the software applications and emotion. There is enough evidence to say that the effect of

emotion on perceived usability, specifically for the type of software, varied for different scenarios of

emotion.

For the experimental variables, the only factor that was significant to the perceived usability of the

smart phone/software system was the elicited emotion. Thus, we rejected the null hypothesis Ho12 and

concluded that the mean values of perceived usability for the positive affect condition were different

that the ones observed in the negative affect condition.

111

The interaction terms were not statistically significant for the perceived usability scores. Thus, we failed

to reject the null hypotheses Ho13 and Ho14 and concluded that there was not enough evidence to say

that the effect of emotion on perceived usability varied for different smart phone/software scenarios.

Figure 5.6 shows the residual plots for perceived usability, represented by SUS scores. The residuals

exhibit a random pattern and follow a normal distribution, which suggest adequacy of the model.

Residual

Pe

rce

nt

50250-25-50

99

90

50

10

1

Fitted Value

Re

sid

ua

l7060504030

50

25

0

-25

-50

Residual

Fre

qu

en

cy

40200-20-40

8

6

4

2

0

Observation Order

Re

sid

ua

l

4035302520151051

50

25

0

-25

-50

Normal Probability Plot of the Residuals Residuals Versus the Fitted Values

Histogram of the Residuals Residuals Versus the Order of the Data

Residual Plots for SUS Score

Figure 5.6 Residual plots for perceived usability.

The power of the ANOVA test was estimated for the three factors (emotion, smart phone, and

software). In addition, Cohen’s coefficients (α) were estimated to determine the effect size between the

mean usability observed in the positive affect condition and the one observed in the negative affect

condition. The effect size (d’) measures the strength of the relationships between the usability score and

the variable of interest. Table 5.10 summarizes the power estimation and the Cohen’s coefficient for

each factor.

112

Table 5.10 Estimates of Power and Cohen’s Coefficients for Emotion, Smart Phone, and Software

Power was estimated based on the pooled standard deviation of the model and the observed

differences in the mean values of perceived usability for each factor. The power for emotion was 89%,

which is considered very high. The Cohen’s effect size for emotion was 1.085. According to Cohen

(1988), effect sizes of 0.20 are small, 0.50 are medium, and 0.80 are large. This effect is considered large,

which implies that with the actual sample size we were able to detect consistent differences in usability

score changes between the participants who experienced positive affect and the ones who experienced

negative affect.

For the smart phone and software factors, both the power and Cohen’s effects were low. Based on

these results, we may infer that the power of the factors to predict perceived usability was minimal. This

may imply that the assessment instrument used to estimate perceived usability was unable to capture

differences in usability due to the smart phone or the software itself. However, it seems that the

instrument was suitable for capturing an overall impression of product usability, which evidently varied

across different scenarios of affective state. This, in turn, may be the reason why the smart phone and

software factors were not significant.

Initially, we suspected that the effect of emotion on perceived usability would be higher for cases of

poor usability, as the user would become less tolerant to errors and would pay attention to more details

when a negative emotion was induced. To explore these differences, we generated a main effects plot

and an interactions plot.

113

Figure 5.7 Main effects plot for perceived usability.

Emotion

21 21

60

50

40

Smart Phone

60

50

40

Software

Emotion

1

2

Smart

Phone

1

2

Interactions Plot

Figure 5.8 Interactions plot for perceived usability.

In Figure 5.7, the main effects plot for perceived usability shows that the mean usability was higher for

positive emotion than for negative emotion. The following can be observed in Figure 5.8, the

interactions plot:

1) The perceived usability for smart phone 1 (keypad phone) was higher than the perceived

usability of smart phone 2 (touch screen) in the positive emotion condition. For the negative

114

emotion condition, the perceived usability of smart phone 2 was higher than the one observed

for the smart phone 1.

2) The perceived usability of software 1 and software 2 were very similar in the presence of

positive affect. Figure 5.8 illustrates that the differences in perceived software usability were

more noticeable in the presence of negative affect. However, the perceived usability of software

2 was higher when smart phone 2 was used. This may imply that the perceived usability of smart

phone 2 was overriding the perceived usability of software 2.

Although the trends observed in Figures 5.7 and 5.8 support the existence of different effects of

emotion on perceived usability, we did not have enough statistical evidence to conclude different

effects of emotion on perceived usability for different software applications and smart phones.

5.3 Hypothesis Related to the Effect of Affective State on Perceived Software Usability

After examining the effect of emotion on perceived usability overall, we were interested in assessing the

effect of emotion specifically on perceived software usability. For this purpose, we used the values

obtained from the Software Usability Scale (refer to Appendix D) as the response variable. The Software

Usability Scale is an adaptation of the SUMI scale. It consists of 10 questions that assess software

usability. Software usability was obtained by calculating the average value of the 10 responses.

The Cronbach’s alpha estimated for the scale at 95% confidence was 0.897. This value refers to the

coefficient of internal consistency. It measures the validity of the questions used to formulate a measure

from a scale. The closer the alpha value is to 1, the greater the internal consistency of the questions in

the metric (Gliem and Gliem 2003). Figure 5.9 shows the rules that are used to determine the validity of

the scales using Cronbach’s alpha (George and Mallery 2003):

115

α > 0.9 = Excellent, > 0.8 = Good, > 0.7 = Acceptable, > 0.6 = Questionable, > 0.5 =Poor, < 0.05 =Unacceptable

Figure 5.9 Guidelines for values of Cronbach’s Alpha.

Tables 5.11 and 5.12 show the descriptive statistics for the Software Usability Scale.

Table 5.11 Descriptive Statistics for the Software Usability Scale

Table 5.12 Descriptive Statistics for the Software Usability Scale, by Emotion

The software usability score was higher for positive affect overall, and software 1 for both the positive

and negative affect conditions.

Figure 5.10 shows the interval plot and boxplot of perceived software usability based on positive and

negative affect. On these plots, the difference between perceived usability for positive affect and

negative affect is not evident.

116

Emotion

So

ftw

are

Usa

bili

ty

NegativePositive

3.50

3.25

3.00

2.75

2.50

Interval Plot of Software Usability vs Emotion95% CI for the Mean

Emotion

So

ftw

are

Usa

bili

ty

NegativePositive

5

4

3

2

1

Boxplot of Software Usability vs Emotion

Figure 5.10 Interval plot and boxplot of perceived software usability based on emotion.

Figure 5.11 shows the interval plot and boxplot of perceived software usability for the two smart phones

used. Based on these plots, there seems to be no difference between perceived usability for smart

phone 1 and smart phone 2.

Figure 5.11 Interval plot and boxplot of perceived software usability based on smart phone used.

Figure 5.12 shows the interval plot and boxplot of the perceived usability for the two software

applications used. On these plots, there is a difference between perceived usability for positive affect

and negative affect. The perceived usability of the software seems to be higher in the positive emotion

condition.

117

Figure 5.12 Interval plot and boxplot of perceived software usability based on software used.

To test differences in perceived software usability, the following hypothesis was formulated:

Ho15: The type of emotion is not a statistically significant factor affecting perceived software

usability.

The equivalent hypotheses are:



To test these hypotheses, software usability scores were used as a measure of perceived software

usability. A statistical model based on the RCBD was used. Normality assumptions were verified and

met. Table 5.13 shows the results of the ANOVA test with software usability as a response variable.

118

Table 5.13 ANOVA for Software Usability Score, Experiment 4

Source F P-value

Familiarity 0.04 0.850

Time Owned 2.83 0.111

Input Familiarity 2.02 0.173

Extraversion 1.47 0.242

Emotion 0.33 0.571


Software 6.12 p < 0.05




Emotion*Smart Phone*Software 4.11 p < 0.05

In this case, the variables related to previous experience with smart phones were considered covariates.

These variables were related to familiarity with the smart phone and did not relate to previous

experience with the software. However, interactions between the perceived software usability and the

smart phone may have occurred. Because the software applications were created for the experiment,

previous experience with the software was not assessed.

Given that, in principle, the variables related to familiarity with the smart phone should not affect the

perceived usability of the software, no formal hypothesis was defined for this purpose. Moreover, the

ANOVA results show that at 95% confidence there was no significant difference in perceived software

usability as a function of previous experience with the smart phone.

The degree of extraversion was also included as a covariate to reduce variability due to participant

ability or willingness to share information on the software usability questionnaire. Because the p-value

was greater than 0.05, there was not enough evidence to infer that software usability varied as a

function of the degree of participant extraversion.

119

For the experimental variables, the only factor that was significant to the perceived usability of the

software was the type of software used. Thus, the differences in software usability scores varied as a

function of the software. This validated the controlled differences between software 1 and software 2.

The results suggested that there was not sufficient evidence to reject the null hypothesis Ho15.

Therefore, there was no significant difference in perceived software usability as a function of the

emotion experienced upon interaction with the smart phone/software system.

For the interaction terms, the Emotion*Smart Phone*Software interaction was statistically significant at

a 95% significance level to the perceived software usability score. Thus, there was enough statistical

evidence to say that the effect of emotion on perceived software usability varied for different emotion,

smart phone and software combinations.

Figure 5.13 shows the residual plots for perceived software usability. The residuals exhibit a random

pattern and follow normal distribution, suggesting model adequacy.

Residual

Pe

rce

nt

10-1

99

90

50

10

1

Fitted Value

Re

sid

ua

l

4.03.53.02.52.0

1

0

-1

Residual

Fre

qu

en

cy

1.61.20.80.40.0-0.4-0.8-1.2

8

6

4

2

0

Observation Order

Re

sid

ua

l

4035302520151051

1

0

-1

Normal Probability Plot of the Residuals Residuals Versus the Fitted Values

Histogram of the Residuals Residuals Versus the Order of the Data

Residual Plots for Software Usability

Figure 5.13 Residual plots for perceived usability.

120

The power of the ANOVA test was estimated for the factors of emotion and software, shown in Table

5.14.

Table 5.14 Estimates of Power for Emotion and Software

Power was estimated based on the pooled standard deviation of the model and observed differences in

the mean values of perceived software usability for emotion and software. The power for emotion was

14% while the power for software was 86%. This implies that the power of emotion to predict perceived

software usability is minimal.

The main effects plot and the interaction plot shown in Figures 5.14 and 5.15 provide visual

representations of the relationships among the variables in the model.

Me

an

of

So

ftw

are

Usa

bili

ty

NegativePositive

3.2

3.0

2.8

2.6

2.4

Smart_phone2Smart_phone1

Software2Software1

3.2

3.0

2.8

2.6

2.4

Emotion Smart Phone

Software

Main Effects Plot for Software Usability

Figure 5.14 Interactions plot for perceived software usability.

121

Emotion 1 = Positive; Emotion 2 = Negative

Figure 5.15 Interactions plot for perceived software usability.

In Figure 5.14, the main effects plot for perceived software usability shows that the mean usability was

higher for software 1 than for software 2. In Figure 5.15, we may observe that the perceived usability for

the two software applications was consistently higher in the positive affect condition when compared to

the negative affect condition.

5.5 Hypotheses Related to Other Usability Metrics

In this analysis, we considered other metrics related to usability: perceived ease of use and task

completion rate. Perceived ease of use measured how difficult users thought it was to accomplish tasks

in the system, while task completion rate was calculated as the actual percentage of tasks that were

completed.

122

5.5.1 Perceived Ease of Use

The perceived ease of use relates to the degree to which a user felt a product was easy to use to

accomplish specific tasks. To assess the perceived ease of use, the following hypothesis was formulated:

Ho16: The type of emotion is not a statistically significant factor influencing perceived ease of

use.

The perceived ease of use was assessed using an adaptation from the Davis (1993) scale for perceived

ease of use. The metric was estimated by averaging answers across all questions on the scale. The

Cronbach’s alpha for the perceived ease of use scale equals 0.902. Please refer to the Appendix D for the

specific questions asked.




Table 5.14 shows descriptive statistics for perceived ease of use. Table 5.15 shows the results of the

ANOVA test using perceived ease of use as a response variable.

Table 5.14 Descriptive Statistics for Perceived Ease of Use

123

Table 5.15 ANOVA for Perceived Ease of Use, Experiment 4

Source F P-value

Emotion 4.330 p < .05


Software 3.900 p < .05




Emotion*Smart Phone*Software 1.350 0.254

Based on the ANOVA results, emotion was a significant factor affecting perceived ease of use for the

smart phone/software system. Thus, we reject the null hypothesis Ho16 and conclude that there was

enough evidence to say that the mean value of perceived ease of use for the positive affect condition

was different than the one observed in the negative affect condition.

5.5.2 Task Completion Rate

The task completion rate refers to the percentage of tasks that were completed by the participants

during the experiment session. The participants were given a total of 20 tasks in random order. The

tasks were related to financial transactions and management. Please refer to Appendix D.7 for a list of

the 20 tasks given.

The participants were given exactly 10 minutes to interact with the system, and they were instructed to

work as quickly and accurately as they could. They were also provided with recording sheets to track the

tasks that were accomplished. The total number of tasks completed was divided by 20 to obtain the

completion rate. The task completion rate was used as a metric to assess usability in an objective way. It

assessed system efficiency, a fundamental part of usability. In this case, the number of tasks completed

represents an objective and behavioral measure of usability. However, it is important to note that since

the tasks were randomly assigned to the participants, each participant executed a different set of tasks.

124

Thus, the task completion rate may have been affected by the type of tasks executed and their order,

which were randomly assigned to the participants.

To test the significance of emotion on the task completion rate, the following hypothesis was

formulated:

Ho17: The type of emotion is not a statistically significant factor affecting the task completion

rate.

The equivalent hypotheses were:

Ho17: µtask completion rate of positive affect - µtask completion rate of negative affect = 0

Ha17: µtask completion rate of positive affect - µtask completion rate of positive affect ≠ 0

Table 5.16 shows the descriptive statistics for perceived ease of use. Table 5.17 shows the results for the

ANOVA test using the task completion rate as a response variable.

Table 5.16 Descriptive Statistics for Task Completion Rate

Table 5.17 ANOVA for Task Completion Rate, Experiment 4

125

Based on the ANOVA results, familiarity with the phone was a significant factor. This implies that the

type of smart phone used affected the number of tasks that were completed. The software type was

also a significant factor, implying that the type of software affected the number of tasks completed.

Specifically, it was observed that for software 1 the number of tasks completed was higher than the

number of tasks for software 2. The variable of emotion was not statistically significant. Thus, there was

not enough evidence to reject H17.

5.5 Summary

In this experiment, we explored the effects of affective states prior to product interactions on perceived

usability. The experiment studied the presence of such effects for different scenarios of controlled

software usability and for different input modalities of smart phone products.

The effect of emotion on perceived system usability was analyzed using four different response

variables related to usability: perceived system usability, perceived software usability, perceived ease of

use, and task completion rate.

The perceived system usability was measured using the System Usability Scale (SUS). The results

suggested that there was a significant difference in the system’s perceived usability based on the type of

emotion experienced. The perceived usability was higher for positive affect than for negative affect.

There was no statistical evidence to infer a difference in perceived usability as a function of the type of

smart phone and software used.

Software usability was assessed with the Software Usability Scale. No significant changes in perceived

usability were observed for the variable of emotion. However, the variable referring to the type of

software used showed statistical significance affecting perceived software usability. In this experiment,

126

the effects of software usability may have been overriding the effect of the type of emotion in terms of


The emotion factor was also significant for perceived ease of use. This implies that this dimension of

usability was affected by emotions prior to product interactions.

The task completion rate provides data on actual behavior rather than perceptions. For the completion

rate, emotion was not a significant factor. The fact that the tasks were provided randomly to the

participants added noise to the task completion rate measure. In general, we expected emotion to

affect the number of tasks completed. However, for this specific scenario and set of tasks, there was no

statistical evidence to infer an effect of emotion on the task completion rate.

It is relevant to note that although emotions prior to interaction were not significant to the task

completion rate, they were nevertheless a significant factor affecting the perceived usability of the

system. Task completion rate served as a behavioral measure of usability, while perceived usability was

based on product perceptions from questionnaires. Thus, it is possible that the effect observed on

perceived system usability as measured by the SUS score was affected by the bias to rate or perceive a

product as good in the presence of positive emotion, and conversely to perceive it as bad in the

presence of negative emotion. For this specific experiment and set of tasks, even though actual behavior

was not affected, product perceptions were affected by emotion.

127

Chapter 6

Conclusions and Future Work

In this research, the effects of positive and negative emotions on the perceived usability of smart

phones and related software applications were studied. In the past, several studies have provided

evidence of the effect of emotion on cognitive processes such as decision making, evaluative judgment,

perceived risk, consumer behavior, and new product evaluation (Fedorikhin and Cole 2004); Gorn,

Goldberg et al. 1993; Chaudhuri 1990). In general, positive affect is associated with positive evaluations

while negative affect promotes negative perceptions. In this research, we were interested in exploring

the effect of emotion specifically on the way products are perceived as usable. A total of four

experiments were conducted in this research; the first three were preliminary experiments aimed at

obtaining information for the final experiment in which the effect of the emotion on perceived product

usability was assessed. In these experiments, the product system consisted of a smart phone and a

related software application.

Our main question was whether perceived usability was affected by user affective states prior to

interactions with the products. Experiments 2, 3 and 4 proved that there is a statistically significant

effect of user affective states on perceived product usability. Thus, for the specific products, scenarios,

and emotions elicited in the experiments, prior affective states were determinant factors affecting

perceptions of product usability. In general, users in positive emotional states will perceive that a

product is better than those in negative emotional states.

We were also interested in studying the differences among these effects for different smart phone input

modalities and software usability levels. No significant differences were observed for the effect of

emotion when different smart phones and software applications were used. Thus, for this specific set of

128

experiments, the results imply that the effect of emotion on product perceptions is not affected by

product features or software usability. There was not enough statistical evidence to observe significant

differences in the perceived usability of the smart phone, software, or the system as a whole. Based on

the power analyses, this may imply that the sample size was not big enough to capture the differences

between smart phone input modalities and software usability. Thus, it is possible that more data may

need to be collected for further analyses. It may also imply that the assessment instrument used to

estimate perceived usability was unable to capture differences in usability due to the smart phone or

the software itself. This effect may also be because the effect of emotion prior to interaction and the

associated biases in answering the questions on the System Usability Scale (SUS) prevailed over the

actual usability experienced.

For the software usability assessment, the factor of emotion was not significant. Thus, for this particular

software there is no evidence to support that the perceived software usability varied based on the type

of emotion experienced. However, the trends observed in the interactions plot suggest that software

usability was rated higher by those in the positive affect scenario than those in the negative affect

scenario.

Emotion was a significant factor affecting the perceived ease of use. In this experiment, the system was

perceived as easier to use in the presence of positive affect than in the presence of negative affect.

For the task completion rate (specifically for the conditions in experiment 4), emotion was not a

significant factor. This may be related to measurement variability introduced by the randomness of the

tasks. Because different participants were exposed to two different types of smart phones and two

different types of software, the tasks were randomized. The tasks were randomized so as to avoid any

effects from executing the tasks in a certain order. The learning effects could vary by type of phone and

129

software, and thus the tasks were randomized to minimize learning differences based on the system

used.

For the specific scenarios and products used in experiment 4, emotion did not seem to affect real

behavior in terms of the tasks completed, but it did affect usability perceptions. The results suggest that

the usability score measured in SUS reflected a bias to rate or perceive a product as better in the

presence of positive affect and to perceive it as worse in the presence of negative affect. Thus, even

though actual behavior was not affected, product perceptions were affected by emotion.

An emotion elicitation strategy was developed for the purpose of manipulating participant affect states

throughout the experiments. This strategy was proven to be effective at eliciting both positive and

negative affect. The results in experiments 1 through 4 validated the effectiveness of the strategy. This

emotion elicitation strategy may be used by researchers interested in studying the effect of emotion on

any cognitive process. In addition, this represents a contribution to designers who are interested in

emotion, and developers who want to create products that minimize certain emotions. For instance, the

developers of medical devices used to diagnose medical conditions would want to mitigate patient

anxiety. To test the effectiveness of their designs at mitigating existing emotions, designers may have to

execute a series of experiments that elicit emotions from the participants. This community of

professionals will benefit from having an emotion elicitation strategy that has proven to be effective and

reliable.

The results of this research are of particular interest to the community of designers who design products

to generate specific user emotions. As observed, the perceived usability of a product is affected by

emotional state. Thus, designers must consider the way emotions prior to product interactions will

interact with the emotions that will be elicited by the product features. In addition, usability specialists

130

must be aware of the effects of prior emotions on a product’s perceived usability, and take appropriate

steps to block or account for such effects in their usability studies.

This research provided a basis for understanding the effect of emotion on the perceived usability of

smart phones and related software. There are some limitations to the results and conclusions made as

part of this study.

1) The results are limited to the effect of emotion on the perceived usability of the specific phones

and software applications that were used.

2) The results are limited to a population of college undergraduates in the field of Industrial

Engineering (age range of 19-24).

3) The results are based on the System Usability Scale (SUS) to assess product usability and the task

completion rate to assess participant performance while executing the tasks.

4) The experiment was conducted in a laboratory setting, and thus does not replicate natural

circumstances where a person would use a smart phone and the software application for a

specific relevant purpose.

5) Perceived usability is limited to participant impressions of the system after 10 minutes.

6) The tasks executed were not relevant to the participants.

Based on these limitations, we suggest the following research for future work:

1) The effects of emotion on perceived usability may be explored using other types of products

such as computers, mp3 players and cameras. This will provide more insights regarding the

degree to which the effect of emotion on perceived usability may vary as a function of the

product being evaluated. The experiment could be conducted using participants from a different

131

age range with different occupations. This would provide insight on the ways emotional effects

on perceived usability depend on individual characteristics. An older age range of people may

reflect a different effect on perceived usability or even on the emotion elicitation strategy.

2) The effect of emotion may also be assessed using other behavioral metrics, such as user

willingness to use a product, attitudes towards using a product, and the degree of attachment to

a product. This will provide insights on the dimensions of usability and product judgment that

are affected by user emotional states. It will also provide more knowledge on such effects in

other metrics related to product usage.

3) The study may be conducted so as to include participant interests in a naturalistic environment.

The products and the tasks could be adjusted to each individual, and the time to evaluate the

product could be extended. This would provide a more realistic impression of the effect of

emotion on the perceived usability of the products.

4) The effect of prior emotions on the perceived usability of products which were designed

specifically to generate certain emotions could be explored. This may help designers to

understand the effectiveness of their design features at eliciting certain emotions in the

presence of pre-conceived emotions.

5) Currently, the affective design community is using techniques derived from human-to-human

interaction to be implemented in technology. Some of the devices with which we interact are

capable of expressing empathy, providing help and support for our emotional needs while we

interact with the product. It would be relevant to understand the extent to which such

techniques are effective in the presence of positive and negative affect prior to product

interaction. This would provide insights to the design community to help them understand the

effectiveness of such techniques in the presence of positive and negative emotions.

132

References

Averill, J. R. (1980). Constructivist view of emotion. Emotion: Theory, Research and Experience. R.

Plutchik and h. Kellerman. New York, Academic Press. 1.

Bevan, N. (1995). "Measuring usability as quality of use." Software Quality Journal. 4: 115-150.

Bower, G. H. and J. P. Forgas (2000). Chapter 3: Affect, memory, and social cognition. Cognition and Emotion. E. Eich, J. F. Kihlstrom, G. H. Bower, J. P. Forgas and P. M. Niedenthal. New York, Oxford University Press.

Bradley, M. M. (1994). "Measuring emotion: The self-assessment manikin and the semantic differential." Journal of Behavior Therapy and Experimental Psychiatry. 25(1): 49.

Brave, S. and C. Nass (2003). Emotion in Human-Computer Interaction. The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies and Emerging Applications. A. J. Julie and S. Andrew, L. Hillsdale, NJ, Lawrence Erlbaum Associates, Inc.: 1242.

Brooke, J. (1996). SUS: A quick and dirty usability scale. Usability Evaluation in Industry. B. T. P. Jordan, and B. Weerdmeester. London, Taylor and Francis: 198-194.

Carroll, J. M. and H. M. Mentis (2008). The useful interface experience: the role and transformation of usability. Product Experience. H. N. J. Schifferstein and P. Hekkerd. San Diego, CA, Elsevier.

Cavanaugh, L. A., J. R. Bettman, et al. (2007). "Appraising the Appraisal-Tendency Framework." Journal of Consumer Psychology. 17(3): 169-173.

Ceaparu, I., J. Lazar, et al. (2004). "Determining Causes and Severity of End-User Frustration." International Journal of Human-Computer Interaction. 17(3): 333 - 356.

Chaudhuri, A. (1990). Risk. Emotion and Reason in Consumer Behavior. A. Chaudhuri. Burlington, MA, Elsevier.

Clore, G. C. and K. Gasper (2000). Feeling is believing: Some affective influences on belief. Emotions and Beliefs: How Feelings Influence Thoughts. N. H. Frijda, A. S. R. Manstead and S. Bem. Cambridge, Cambridge University Press: 10-44.

Clore, G. L. and J. R. Hudtsinger (2007). "How Emotions Inform Judgment and Regulate Thought." TRENDS in Cognitive Sciences. 11(9): 7.

Clore, G. L. and G. Parrott (1991). Moods and their vicissitudes: Thoughts and feelings as information. Emotion and Social Judgments. J. P. Forgas. Elmsford, NY, Pergamon Press.

Clore, G. L., R. S. Wyer, et al. (2001). Affective Feelings as Feedback: Some Cognitive Consequences Theories of Mood and Cognition: A User's Guidebook. L. L. Martin and G. L. Clore. Hillsdale, NJ. Lawrence Erlbaum Associates.

133

Consumer Search Inc. (2010). Retrieved November 27, 2010, 2010, from http://www.consumersearch.com/cell-phones/best-smartphones

Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd edition). Hillsdale, NJ, Lawrence Erlbaum Associates.

Coupey, E., J. R. Irwin, et al. (1998). "Product category familiarity and preference construction." Journal of Consumer Research. 24: 9.

Dalgleish, T. and M. J. Power (1999). Handbook of Emotion and Cognition. New York, John Wiley & Sons.

Damasio, A. R. (1994). Descartes' Error : Emotion, Reason, and the Human Brain. New York, Avon Books.

Davis, F. D. (1993). "User acceptance of information technology: system characteristics, user perceptions and behavioral impacts." International Journal of Man-Machine Studies. 38(3): 475-487.

Desmet, P. and P. Hekkerd (2007). "Framework of product experience." International Journal of Design. 1(1): 10.

Desmet, P. and P. Hekkert (2000). The basis of product emotions. Pleasure with Products: Beyond Usability. W. S. Green and P. W. Jordan. New York, Taylor & Francis.

Desmet, P. M. A. (2003). Measuring emotions. Funology: From Usability to Enjoyment M. A. Blythe, K. Overbeeke, A. F. Monk and P. C. Wright. Boston, MA, Kluwer Academic Publishers. 3: 293.

Dewey, J. (1894). "The theory of emotion." The Psychological Review. 1(6).

Dillon, A. (2002). "Beyond usability: Process, outcome and affect in human-computer interactions." Canadian Journal of Library and Information Science. 26(4): 12.

Eich, E., J. F. Kihlstrom, et al. (2000). Cognition and Emotion. New York, Oxford University Press, USA.

Fazio, R. H. and T. Towles-Schwen (1999). The MODE Model of Attitude-Behavior Processes. Dual -Process Theories in Social Psychology. S. Chaiken and Y. Trope. New York, The Guilford Press.

Fedorikhin, A. and C. A. Cole (2004). "Mood Effects on Attitudes, Perceived Risk and Choice: Moderators and Mediators." Journal of Consumer Psychology. 14(1-2): 2-12.

Forgas, J. (2001). The Affect Infucion Model (AIM): An Integrative Theory of Mood Effects on Cognition and Judgment. Theories of Mood and Cognition: A User's Guidebook. L. L. Martin and G. L. Clore. Hillsdale, NJ, Lawrence Erlbaum Associates.

Forgas, J. P. (2007). "The role of affect in social thinking and interpersonal behaviour." Retrieved August 12, 2009, from http://www.psychologia-spoleczna.pl/articles-english-158/199-the-role-of-affect-in-social-thinking-and-interpersonal-behaviour.html.

Frijda, N. H. (1986). The Emotions. New York, Cambridge University Press.

Gardner, M. P. (1985). "Mood States and Consumer Behavior: A Critical Review." The Journal of Consumer Research. 12(3): 281-300.

http://www.consumersearch.com/cell-phones/best-smartphones

http://www.psychologia-spoleczna.pl/articles-english-158/199-the-role-of-affect-in-social-thinking-and-interpersonal-behaviour.html

http://www.psychologia-spoleczna.pl/articles-english-158/199-the-role-of-affect-in-social-thinking-and-interpersonal-behaviour.html

134

George, D., and P. Mallery (2003). SPSS for Windows step by step: A Simple Guide and Reference. 11.0

update (4th ed.). Boston, Allyn & Bacon. Gliem, J. A. and R.R. Gliem (2003). Calculating, Interpreting, and Reporting Cronbach’s Alpha Reliability

Coefficient for Likert-Type Scales. 2003 Midwest Research to Practice Conference in Adult, Continuing, and Community Education.

Gorn, G. J., M. E. Goldberg, et al. (1993). "Mood, Awareness, and Product Evaluation." Journal of Consumer Psychology. 2(3): 237-256.

Gray, E. K. and D. Watson (2007). Assessing positive and negative affect via self-report. Handbook of Emotion Elicitation and Assessment. J. A. Coan and J. J. B. Allen. London, Cambridge University Press.

Green, W. S. and P. W. Jordan (1999). Human Factors in Product Design: Current Practice and Future Trends. Philadelphia, Taylor and Francis.

Han, S., J. S. Lerner, et al. (2007). "Feelings and consumer decision making: The appraisal-tendency framework." Journal of Consumer Technology. 17(3): 10.

Han, S. H., M. H. Yun, et al. (2001). "Usability of consumer electronic products." International Journal of Industrial Ergonomics. 28(3-4): 143-151.

Hassenzahl, M. (2001). "The Effect of Perceived Hedonic Quality on Product Appealingness." International Journal of Human-Computer Interaction. 13(4): 481.

Hassenzahl, M., A. Platz, et al. (2000). Hedonic and ergonomic quality aspects determine a software's appeal. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. The Hague, The Netherlands, ACM.

Hazlett, R. L. and J. Benedek (2007). "Measuring emotional valence to understand the user's experience of software." International Journal of Human-Computer Studies. 65(4): 306-314.

Hekkert, P. and H. N. J. Schifferstein (2008). Introducing product experience. Product Experience. P. Hekkert and H. N. J. Schifferstein. Burlington, MA, Elsevier.

Helander, M. G., H. Peng, et al. (2007). Citasara Engineering Model for Affective Design of Vehicles. IEEE International Conference on Industrial Engineering and Engineering Management. Singapore, IEEE: 1282-1286.

Hoffberg, L. I. (1991). "Designing User Interface Guidelines for Time-Shift Programming on a Video Cassette recorder (VCR)." Human Factors and Ergonomics Society Annual Meeting Proceedings. 35: 501-504.

Isen, A. M. (1987). Positive affect, cognitive processes and social behavior. Advances in Experimental Social Psychology. L. Berkowitz. New York, Academic Press: 203-253.

Isen, A. M. (2001). "An Influence of Positive Affect on Decision Making in Complex Situations: Theoretical Issues With Practical Implications." Journal of Consumer Psychology. 11(2): 75-85.

135

International Organization for Standardization (ISO) Webpage. Retreived on January 26, 2009. http://www.iso.org/iso/home.htm

Izard, C. E. (1977). Human Emotions. New York, Plenum Press.

Izard, C. E. (2007). "Basic Emotions, Natural Kinds, Emotion Schemas, and a New Paradigm." Perspectives on Psychological Science. 2(3): 260-280.

Jordan, P. W. (2000). Designing Pleasurable Products: An Introduction to the New Human Factors. Philadelphia, PA, Taylor and Francis.

Kagan, J. (1994). "On the Nature of Emotion." Monographs of the Society for Research in Child Development. 59(2/3): 7-24.

Keinonen, T. (2007). Theory of a design goal: Usbaility of interactive products. Arteology, or the Study of Professional Skills. P. Routio. Helsinki, UIAH publication A21.

Khalid, H. M. (2006). "Embracing diversity in user needs for affective design." Applied Ergonomics. 37(4): 409-418.

Khalid, H. M. and M. G. Helander (2006). "Customer Emotional Needs in Product Design." Concurrent Engineering 14(3): 197-206.

Kukreja, U., W. E. Stevenson, et al. (2006). "RUI—Recording User Input from Interfaces under Windows and Mac OS X." Behavior Research Methods, Instruments, and Computers. 38(4): 4.

Lange, C. G. and W. James (1922). The Emotions. Baltimore, MD, Williams and Wilkins Company.

Lazarus, R. S. (1991). Emotion and Adaptation. New York, Oxford University Press. Lerner, J. S. and Keltner, D. (2001). "Fear, anger, and risk." Journal of Personality and Social Psychology.

81: 146–159.

Lerner, J. S., & Tiedens, L. Z. (2006). "Portrait of the angry decision maker: How appraisal tendencies shape anger’s influence on cognition." Journal of Behavioral Decision Making. 19: 115–137.

Lewis, J. R. (1995). "IBM computer usability satisfaction questionnaires: Psychometric evaluation and

instructions for use." International Journal of Human-Computer Interaction. 7: 57-78.

Love, T. (2002). Beyond emotions in designing and designs: epistemological and practical issues. Design and Emotion. D. McDonagh, P. Hekkert, J. V. Erp and D. Gyi. New York, Taylor & Francis: 387-391.

Luczak, H. (2003). "Letʼs talk: Anthropomorphization as means to cope with stress of interacting with technical devices." Ergonomics. 46(13): 1361.

McCrae, R. R. and P. T. Costa (1987). "Validation of the five-factor model of personality across instruments and observers." Journal of Personality and Social Psychology. 52: 9.

136

McDonagh, D., P. Hekkert, et al. (2004). Design and Emotion: The Experience of Everyday Things. New York, Taylor and Francis.

Mehrabian, A. and J. A. Russell (1974). An Approach to Environmental Psychology. Cambridge, MA, MIT Press.

Montgomery, D. C. (2009). Design and Analysis of Experiments. Hoboken, NJ, John Wiley and Sons, Inc.

Morris, J. D. (1995). "Observations: SAM: The self-assessment manikin — An efficient cross-cultural measurement of emotional response." Journal of Advertising Research. 35(6): 63.

Myers & Briggs Foundation. (2010). Retrieved April 10, 2010 from http://www.myersbriggs.org/.

Nielsen, J. (1993). Usability Engineering. Boston, MA, Academic Press, Inc.

Norman, D. A. (2004). Emotional Design: Why We Love or Hate Everyday Things. New York, Basic Books.

Nowlis, V. (1965). Research with the mood adjective checklist. Affect, Cognition and Personality. S.S. Thompkins, & C.E. Izard. New York, Springer.

Pew, R. W. and A. S. Mavor (2008). Human-System Integration in the System Development Process: A new look. Washington, D.C., National Academies Press.

Pham, M. T. (1998). "Representativeness, Relevance, and the Use of Feelings in Decision Making." The Journal of Consumer Research. 25(2): 144-159.

Picard, R. (1997). Affective Computing. Cambridge, MA, MIT Press.

Picard, R. W. (2000). "Toward computers that recognize and respond to user emotion." IBM Systems Journal. 39(3/4): 14.

Plutchik, R. (1989). Measuring emotions and their derivatives. Emotion: Theory, Research and Experience. R. Plutchik and H. Kellerman. San Diego, CA, Academic Press, Inc. 4.

Power, M. and T. Dalgleish (2008). Cognition and Emotion: From Order to Disorder. New York, Psychology Press.

Revelle, W. and K. R. Scherer (2009). Personality and Emotion. Oxford Companion to Emotion and the Affective Sciences. D. Sander and K. R. Scherer. New York, Oxford University Press.

Ritter, F. E. and P. A. Bibby (2008). "Modeling How, When, and What Is Learned in a Simple Fault-Finding Task." Cognitive Science. 32: 30.

Rolls, E. T. (2000). "Précis of the brain and emotion." Behavioral and Brain Sciences. 23(02): 177-191.

Roseman, I. J. and C. A. Smith (2001). Appraisal Theory: Overview, assumptions, varieties, controversies. Appraisal Processes in Emotion. K. R. Scherer, A. Schorr and T. Johnstone. New York, Oxford University Press.

137

Rosson, M. B. and J. M. Carroll (2002). Usability Engineering: Scenario-Based Development of Human-Computer Interaction. San Diego, Morgan Kaufmann.

Russell, J. A. (1980). "A circumplex model of affect." Journal of Personality and Social Psychology. 39(6): 1161.

Russell, J. A., A. Weiss, et al. (1989). "Affective Grid: A single-item scale of pleasure and arousal." Journal of Personality and Social Psychology. 57(3): 10.

Scheirer, J., R. Fernandez, et al. (2002). "Frustrating the user on purpose: A step toward building an affective computer." Interacting with Computers. 14(2): 93-118.

Schwarz, N. (1997). "Moods and Attitude Judgments: A Comment on Fishbein and Middlestadt." Journal of Consumer Psychology. 6(1): 93-98.

Schwarz, N. (2000). "Emotion, Cognition and Decision Making." Cognition and Emotion. 14: 8.

Schwarz, N. and G. L. Clore (1988). How Do I Feel About It? The Informative Function of Affective States. Affect, Cognition and Social Behaviors. K. Fiedler and J. Forgas. Lewistown, NJ, Hogrefe, Inc.: 44-62.

Shackel, B. (1986). Ergonomics in design for usability. People and Computers, Proceedings from the Second Conference of the BCS HCI Specialist Group. Harrison, M. D. and Monk, A. F. Cambridge, Cambridge University Press: 45-64.

Shackel, B. (1991). Usability - context, framework, definition, design and evaluation. Human Factors for Informatics Usability. Cambridge, Cambridge University Press: 21-37.

Shiota, M. N., D. Keltner, et al. (2006). "Positive emotion dispositions differentially associated with Big Five personality and attachment style." Journal of Positive Psychology. 1(2): 10.

Simon, H. A. (1967). "Motivational and emotional controls of cognition." Psychological Review. 74(1): 29-39.

Spillers, F. (2008). "Design and Emotion." Retrieved January 15, 2009, from

http://www.demystifyingusability.com/design_and_emotion/.

Stickney, L. T. (2009). "Affect and Decision Making." Decision Line. 40: 3.

Thüring, M. and S. Mahlke (2007). "Usability, aesthetics and emotions in human-technology interaction." International Journal of Psychology. 42(4).

Tse, D. and P. Viswanath (2005). Fundamentals of Wireless Communication. Cambridge, Cambridge University Press.

Wallbott, H. G. and K. R. Scherer (1989). Assessing emotions by questionnaires. Emotion: Theory, Research and Experience. R. Plutchik and H. Kellerman. San Diego, CA, Academic Press, Inc. 4.

Watson, D., L. A. Clark, et al. (1988). "Development and validation of brief measures of positive and negative affect: The PANAS scales." Journal of Personality and Social Psychology. 54(6): 1063-1070.

138

White, K. and C. McFarland (2009). "When are moods most likely to influence consumers' product preferences? The role of mood focus and perceived relevance of moods." Journal of Consumer Psychology. 19(3): 526-536.

Wilcoxon, F. (1945). "Individual comparisons by ranking methods." Biometrics. 1: 4. Zevon, M. A. and A. Tellegen (1982). The structure of mood change: An idiographic/nomothetic analysis.

Journal of Personality and Social Psychology. 43: 111–122.

Zillmann, D. (2008). Excitation transfer theory. The International Encyclopedia of Communication. W. Donsbach. Blackwell Publishing.

Zuckerman, M. and B. Lubin (1965). Manual for the Multiple Affect Adjective Check List. San Diego, CA, Educational and Industrial Testing Service.

139

Appendix A: Emotion Elicitation Strategy

To elicit positive and negative emotions in the participants, we decided to have the participants use a

computer to execute tasks. A similar technique was used in the past, but only to elicit negative emotions

such as frustration (Picard 2000). In this case, the same tasks were used to elicit both positive and

negative affect.

We chose the Diag task by Ritter and Bibby (2001) as the computer task. This computer task consisted of

identifying the part of the circuit that was broken, based on the display provided. Figure A.1 in this

document shows the circuit to be used.

Figure A.1 Circuit schematic.

In Figure A.1 the energy runs from the power source (PS) all the way to the laser bank (LB).

Accompanying the circuit is a yellow indicator light that turns on whenever there is energy or power in

the circuit component. The participants were presented with a series of displays and they were asked to

140

type the part that they thought was broken, and then click next. The broken part could be inferred from

the panel display presented. Figure A.2 shows an example of a panel display.

Figure A.2 Panel showing power in the circuit components.

PS refers to the power supply, EB1 to energy booster 1, and so on. The yellow light on the panel

indicates that there is power in the illuminated element. At the bottom of the panel, the direction of

the switches is indicated. For example, in Figure A.2 the yellow light appears only on the power source

(PS), and the eb1/eb2 switch is pointing towards the eb2, indicating that the switch is closed towards

eb2. Thus, the power should flow to eb2, then to the secondary accumulator 1 (SA1) and then to the

laser bank (LB).

A total of 45 panel displays were presented in a computer program. For each stimulus, participants were

asked to type the part of the circuit that they thought was broken. The computer program Director MX

was used to create a movie of the various stimuli.

The negative and positive affect were elicited in the form of frustration and happiness respectively.

Given that emotions are generated when situations are either congruent with a subject’s personal

values or not, we had to make this task important to the participants so as to promote or impede the

accomplishment of tasks and generate emotions in a naturalistic way. Therefore, a small portion of the

monetary compensation received by the participants depended on their performance. The participants

141

received a minimum of $8 for their participation in each session. After that minimum, they could get up

to $12 dollars for the 45 stimuli condition and up to $14 dollars for the 65 stimuli condition, based on

historical data executing this task and perfect accuracy. The following formula was used for the extra

compensation based on performance:

Extra compensation (cents) = (10 cents * # of correct answers) – (5 cents * time to complete all tasks in

minutes)

If the participants exceed the maximum, they were paid based on their performance as per the equation

given. Thus, there was no maximum amount of money to be paid. The minimum compensation of $8 per

session was always given; regardless of the time it took them to complete the tasks. Participants were

informed about how to increase monetary compensation and the formula was explained at the

beginning of each session.

The participants in the positive emotion condition were exposed to a version of the computer game that

contained easy tasks. The easy tasks were the ones in which the direction of the switches were

consistent with the flow of the power. For the tasks that were considered difficult, the direction of at

least one switch was not consistent with the power flow observed. Participants were allowed to keep

the circuit and all the information needed to execute the tasks with them; thus, they did not have to

memorize any information. To elicit positive emotions the investigator used positive feedback on their

performance, verbal praising, and positive feedback at the end of the tasks. We expected the

participants to feel positive emotions based on the verbal praising and positive feedback during the

tasks.

The participants in the negative emotion condition were exposed to a version of the computer game

that contained both easy and difficult tasks. They had 5 minutes to memorize the material required to

execute the task. After those 5 minutes, the study materials were removed and they were asked to

142

execute the tasks, making the task more difficult to accomplish. From the literature it is known that

some of the most important factors affecting frustration (the negative emotion we elicited) are: the

level of commitment to the goal, the severity of interruption to that goal, and the degree of interference

with goal attainment (Ceaparu, Lazar et al. 2004). Thus, we introduced intentional delays in the

transitions between stimuli and intentional situations in which the screen would not advance when

required.

Since delayed reinforcement was the principal contributor to frustration, we made use of intentional

delays in advancing to the next screen, which impeded the natural progress towards achieving the goal

of having the best accuracy score in the lowest time, thereby increasing the amount of money earned.

Frustration emerged as the delays represented impediments to goal attainment. These conditions did

not affect compensation estimates given to the participants. Delays between transitions and the

estimate of maximum compensation were determined during informal trials with four participants

during a preliminary test. Under these conditions, we expected the participants to feel frustrated with

the computer task and with their performance.

The participants in the negative emotions group received both a small present and candy at the end of

their participation in each session, and interacted on a free topic with the researcher conducting the

study for about three minutes. This was intended to mitigate the negative emotions generated,

although the arousal level of the emotion was very low, and thus did not represent any hazard or risk to

the subjects.

143

Appendix B: Materials Used in Experiment 1

B.1 Instruction Sheet Given to the Participants

Instructions for the Diag task:

The Diag task is really simple. All you have to do is to identify the part of the circuit that is broken based

on the display shown. Here is the circuit we will refer to:

Figure 1: Circuit schematic.

The energy runs from the power source (PS) all the way to the laser bank (LB). Accompanying the circuit

is an indicator panel, in which a yellow light is turned on whenever there is energy or power in the

circuit component. Figure 2 shows an example of a panel.

144

Figure 2: Panel showing power in the circuit components.

PS refers to the power supply, EB1 to energy booster 1, and so on. The yellow light on the panel

indicates that there is power in the illuminated element. At the bottom of the panel, the direction of

the switches is indicated.

For example, in Figure 2 the yellow light appears only on the power source (PS), and the eb1/eb2 switch

is pointing towards eb2, indicating that the switch is closed towards eb2. Thus, the power should flow to

eb2, then to the secondary accumulator 1 (SA1), and then to the laser bank (LB).

In this case, given that the eb2 light is not illuminated, and that the switch is pointing towards eb2, it

means that the broken part is eb2. Thus, the correct answer for this part will be eb2.

You will be presented with a series of displays and you will be asked to type the part that you think is

broken and click next. Remember to work as quickly and as accurately as you can.

When you type the name of the broken part, use the abbreviated name, e.g., LB for laser bank, SA1 for

secondary accumulator 1, etc. Do not worry if you do not see a cursor in the space provided for typing

the answer, just click on it and type your answer.

Good luck!

145

B.2 Pre-Experiment Questionnaire

Dear participant:

Thank you for participating in this study. We would like you to fill out a few questionnaires in the course

of the experiment, as indicated by the researcher conducting the session. There is no right or wrong

answers to these questions, so please be as honest as possible and do not hesitate to ask if you have any

questions while filling out the questionnaire. Your name and your answers will be kept confidential.

Thanks!

Section A. Initial Questionnaire

Name ____________________________ E-mail [email protected] Participant #_________

Age _______ Gender _______

Occupation ________________ If student, indicate year __________ Major _______________

Section B. About your participation in this study

Please indicate the alternative that best describes your response to the following statements:

A. Your degree of commitment to this experiment and to do your best performance is:

1 2 3 4 5

Extremely committed Committed Neutral Not committed I do not care

B. For you, to maximize the amount of money earned is:

1 2 3 4 5

Extremely important Important Neutral Not important I do not care

C. For you, to maximize the accuracy of your answers and minimize the time required is:

1 2 3 4 5


D. For you, to win the special prize for the best time and accuracy score is:

1 2 3 4 5


146

B.3 Myers-Briggs Type Indicator (MBTI)

The MBTI provides personality assessments for individuals who answer a set of questions related to

preferences in daily life situations. The questionnaire is based on Jung’s typological model and sorts

differences into four opposite pairs, resulting in 16 different psychological types. The 16 different types

of personality are referred to by an abbreviation of four letters. The four dimensions are the following:

Extraversion or Introversion, Sensing or Intuition, Thinking or Feeling, Judgment or Perception.

Extraversion/introversion describes the part of the personality that prefers to either be action-oriented

(extraversion) or thought-oriented (introversion). Sensing and intuition describe how new information is

understood and interpreted. People with a sensing preference tend to trust tangible and concrete

information, looking for details and facts. In contrast, people with an intuition preference tend to trust

abstract information. The thinking/feeling aspect of personality refers to the preference of the

participant to make rational decisions. Thinkers tend to measure decisions by what seems reasonable,

logical and consistent with existing rules. In contrast, people associated with feeling tend to make

decisions by empathizing with the situation and trying to achieve harmony and consensus.

Judgment/perception describes the part of the personality that describes whether people tend to judge

or perceive. People who like to use judgment tend to use facts to assess situations, while people who

like to use perception tend to base their conclusions on what they perceive as being true.

In this experiment, we classified the participants based on personality traits in the extravert/introvert

dimension. A categorical variable with two levels was created to classify the participants as being either

Extraverts or Introverts.

147

B.4 Positive and Negative Affect Schedule (PANAS):

This scale consists of a number of words that describe different feelings and emotions. Read each item

and then mark the appropriate answer in the space next to that word. Indicate to what extent you feel

this way right now, that is, at the present moment. Use the following scale to record your answers:

1 2 3 4 5

very slightly a little moderately quite a bit extremely

or not at all

___ interested ___ irritable

___ distressed ___ alert

___ excited ___ ashamed

___ upset ___ inspired

___ strong ___ nervous

___ guilty ___ determined

___ scared ___ attentive

___ hostile ___ jittery

___ enthusiastic ___ active

___ proud ___ afraid

B.5 Affect Grid (AG)

The following grid represents a series of affective states. The center of the square (marked by X in the

grid below) represents a neutral, average, everyday feeling. It is neither positive nor negative. The right

half of the grid represents pleasant feelings. The farther to the right the X is, the more pleasant the

feeling. The left half represents unpleasant feelings. The farther to the left the X is, the more unpleasant

the feeling. Please draw an X anywhere on the grid to represent your actual affective state. Please look

over the entire grid to get a feel for the meaning of the various areas.

148

Figure B.1 Affect Grid

B.6 Self-Assessment Mannequin (SAM)

This is the Self-Assessment Mannequin. The first row relates to the type of affect you are experiencing

right now, the second row reflects your degree of dominance and the third row represents the level of

arousal you are experiencing (sleepy – excited). Please make a mark in the mannequin of each row that

best describes your affective state.

149

Figure B.2 Self-Assessment Mannequin.

150

B.7 Post-Experiment Questionnaire

A. About your experience in the sessions:

1. Did you suspect the real purpose of the experiment?

Not at all I suspected a little I completely suspected

2. Which condition do you think was better and more effective?

Shorter version Longer version

3. Which method made it easier for you to report your feelings?

Affect grid (AG) PANAS Self-Assessment Mannequin (SAM)

4. Was 7 days long enough for you to forget about the task and generate emotions again?

________________________

B. Feedback about incentives:

5. Was the extra compensation an incentive for you?

Definitely Neutral Not at all

6. Would you increase the amount of money for the extra compensation? If yes, by how much?

No Yes How much? __________________

C. For participants in the positive affect condition:

7. The praising made you feel: _________________________________

8. The increase in incentive made you feel : ___________________________

9. Did you really experience a positive emotion after the computer task? ___________________

D. For participants in negative affect condition:

10. How did the delays make you feel? __________________

11. Were the delays very severe as goal obstacles? __________________

12. Did you experience negative emotions such as anger, frustration, etc? _____________________

151

Appendix C: Materials Used in Experiment 2 and Experiment 3

C.1 Pre-Experiment Questionnaire

Dear participant:

Thank you for participating in this study. We would like you to fill out a few questionnaires over the

course of the experiment, as indicated by the researcher conducting the session. There is no right or

wrong answers to these questions, so please be as honest as possible and do not hesitate to ask if you

have any questions while filling out the questionnaire. Your name and your answers will be kept

confidential.

Thanks!

Section A. About you

Name ____________________________ E-mail [email protected] Participant #_________

Age _______ Gender _______

Occupation ________________ If student, indicate year __________ Major _______________

Section B. About your participation in this study

Please indicate the alternative that best describes your response to the following statements:

B. Your degree of commitment to this experiment and to do your best performance is:

1 2 3 4 5

I do not care Not committed Neutral Committed Extremely committed

C. For you, to maximize the amount of money earned is:

1 2 3 4 5

I do not care Not important Neutral Important Extremely important

D. For you, to maximize the accuracy of your answers and minimize the time required is:

1 2 3 4 5


E. For you, to win the special prize for the best time and accuracy score is:

1 2 3 4 5


Section C. About your experience with smart phones

F. Do you have a cell phone that may be used as a smart phone?

152

Yes _____ No ______

G. For how many years have you been using a cell phone?

Less than 1 year. 1-2 years 3 years or more

H. How many hours per week you spend on a cell phone?

3 hours or fewer 4-6 hours 7-9 hours more than 9 hours

I. What type of application or program do you use very frequently? (check more than one if necessary)

____E-mail ____Word processing

____Instant messaging ____Spreadsheet program

____Web browsing ____Graphic design

____Presentations ____Programming tools

____Database tools ____Other, please specify ________________________________

J. How frequently do you use your cell phone for texting?

1 2 3 4 5

Never Not frequently Frequently Very frequently Extremely frequently

K. Do you have a touch pad or keypad cell phone?

Touch pad Keypad Does your cell phone has a QWERTY keypad? Yes____ No____

L. How comfortable you feel using the keypad of your cell phone?

1 2 3 4 5

Extremely uncomfortable Very uncomfortable Comfortable Very comfortable Extremely comfortable

M. How comfortable you feel using the touch pad of your cell phone?

1 2 3 4 5

Extremely uncomfortable Very uncomfortable Comfortable Very comfortable Extremely comfortable

N. Please indicate which type of data entry you would prefer for each situation:

A. Texting: Touch pad Keypad

B. Using e-mail application: Touch pad Keypad

C. Web browsing: Touch pad Keypad

D. Making a call: Touch pad Keypad

153

E. Using any Windows application: Touch pad Keypad

O. Indicate your level of knowledge and familiarity with Blackberry phones:

Extremely unfamiliar Very unfamiliar Familiar Very familiar Extremely familiar

P. Indicate your level of knowledge and familiarity with HTC phones:

Extremely unfamiliar Very unfamiliar Familiar Very familiar Extremely familiar

C.2 Emotion Questionnaires and Emotion Elicitation Materials

In this experiment, the instruction sheet used was the same as in experiment 1. The affective state was

measured by the Positive and Negative Affect Schedule (PANAS). Please refer to Appendix A for the

emotion elicitation strategy, to Appendix B.1 for the instruction sheet provided to the participants and

to Appendix B.4 for the PANAS scale used.

154

C.3 System Usability Scale (SUS)

System Usability Scale

© Digital Equipment Corporation, 1986. Strongly Strongly disagree agree 1. I think that I would like to use this system frequently 2. I found the system unnecessarily complex 3. I thought the system was easy to use 4. I think that I would need the support of a technical person to be able to use this system 5. I found the various functions in this system were well integrated 6. I thought there was too much inconsistency in this system 7. I would imagine that most people would learn to use this system very quickly 8. I found the system very cumbersome to use 9. I felt very confident using the system 10. I needed to learn a lot of things before I could get going with this system

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

155

C.4 IBM Post-Study System Usability Questionnaire (PSSUQ)

IBM Post-Study System Usability Questionnaire:

1. Overall, I am satisfied with how easy it is to use this system.

Strongly Agree 1 2 3 4 5 6 7 Strongly Disagree

COMMENTS: _____________________________________________

2. It was simple to use this system.


COMMENTS: _____________________________________________

3. I could effectively complete the tasks and scenarios using this system.


COMMENTS: _____________________________________________

4. I was able to complete the tasks and scenarios quickly using this system.


COMMENTS: _____________________________________________

5. I was able to efficiently complete the tasks and scenarios using this system.


COMMENTS: _____________________________________________

6. I felt comfortable using this system.


COMMENTS: _____________________________________________

7. It was easy to learn to use this system.


COMMENTS: _____________________________________________

8. I believe I could become productive quickly using this system.


156

COMMENTS: _____________________________________________

9. The system gave error messages that clearly told me how to fix problems.


COMMENTS: _____________________________________________

10. Whenever I made a mistake using the system, I could recover easily and quickly.


COMMENTS: _____________________________________________

11. The information (such as on-line help, on-screen messages and other documentation) provided with

this system was clear.


COMMENTS: _____________________________________________

12. It was easy to find the information I needed.


COMMENTS: _____________________________________________

13. The information provided for the system was easy to understand.


COMMENTS: _____________________________________________

14. The information was effective in helping me complete the tasks and scenarios.


COMMENTS: _____________________________________________

15. The organization of information on the system screens was clear.


COMMENTS: _____________________________________________

157

Note: The interface includes those items that you use to interact with the system. For example, some

components of the interface are the keyboard, the mouse, the screens (including their use of graphics

and language).

16. The interface of this system was pleasant.


COMMENTS: _____________________________________________

17. I liked using the interface of this system.


COMMENTS: _____________________________________________

18. This system has all the functions and capabilities I expect it to have.


COMMENTS: _____________________________________________

19. Overall, I am satisfied with this system.


COMMENTS: _____________________________________________

C.5 Post-Experiment questionnaire

Thank you for your participation. Please take a moment to fill out this questionnaire. There is no right or

wrong answers to these questions, so please be as honest as possible and do not hesitate to ask if you

have any questions while filling out the questionnaire. Your name and your answers will be kept

confidential.

Thanks!

A. About your experience in the sessions

13. Did you suspect the real purpose of the experiment?

1 2 3 4 5

Not suspect at all Suspect a little Neutral Suspect Suspect completely

If you suspected,

What made you suspect? ____________________________________________________

158

14. Please indicate the degree to which you feel you learned the computer task from the first

session.

1 2 3 4 5

Not learned at all Learned a little Neutral Learned Learned all

15. Please indicate the degree to which you feel you learned the cell phone and software tasks from

the first session.

1 2 3 4 5

Not learned at all Learned a little Neutral Learned Learned all

16. Please indicate the degree to which you feel having 7 days between sessions helped you to

forget the interfaces and the tasks.

1 2 3 4 5

Not forgot at all Forgot a little Neutral Forgot Forgot it all

17. Indicate the degree to which you feel the tasks were easier in the second session.

1 2 3 4 5

Not easier at all A little easier Neutral Easier Much easier

18. Which method was easier for you to report the perceived usability of the smart phone and the

software:

IBM questionnaire System Usability Scale (SUS) NASA TLX

19. Please indicate the degree to which the 7 day timeframe was sufficient for you to forget about

the task and generate emotions in the second session.

1 2 3 4 5

Not sufficient A little sufficient Neutral Sufficient Very sufficient

B. Feedback about incentives

20. Was the extra compensation an incentive to you?

Definitely Neutral Not at all

21. Would you increase the amount of money for the extra compensation? If yes, by how much?

No Yes How much? __________________

159

C. For participants in the positive affect condition:

22. The praising made you feel: _________________________________

23. The increase in incentive made you feel : ___________________________

24. Did you really experience a positive emotion after the computer task? ___________________

D. For participants in negative affect condition:

25. How did the delays make you feel? __________________

26. Were the delays very severe as goal obstacles? __________________

27. Did you experience negative emotions such as anger, frustration, etc? _____________________

E. About your experience with the cell phone/software:

1. Please mention the two biggest challenges associated with data entry on the smart phone.

______________________________________________________________

______________________________________________________________

2. Please mention the two biggest challenges associated with the project management software:

______________________________________________________________

______________________________________________________________

Thank you for your time!

C.6 Description of Environment and List of Tasks

The tasks on the smart phone consisted of editing and adding a series of transactions in the accounting

software Adarian Money. The software was designed to manage project finances and control budgets

and expenses. It consisted of multiple views from which users could select the accounts to be displayed.

The software allowed users to create transactions such as money withdrawals, deposits, etc. within the

existing accounts. For the purpose of the study, imaginary financial scenarios and projects were created.

These accounts and projects did not exist in reality, but they recreated a real set of accounts. The

160

activities that could be accomplished within the software included typing, creating new accounts and

transferring money, among others.

The following is the list of the tasks provided to the participants in the study. The tasks were given in

random order to each participant. They were given 10 minutes to complete as many tasks as possible.

List of tasks

1. Go to the checking transaction executed on January 15, 2010 and do the following edits:

Add the check number 266 to the transaction.

Also add the following phrase in the notes section “This will cover the month of

February”

2. Now, you need to create a new transaction to reflect some online shopping for miscellaneous

items on February 2, 2010 using your Visa card.

Enter the online shopping transaction for a total of $123.98. Payee is Amazon, and it

is a personal transaction.

3. You just made a single payment (withdrawal) with your Master Card of $855.00 to The Payment

Services Co. on February 12. The payment should cover the following:

o $150.00 for life insurance

o $300.00 for electricity

o $405.00 for rent

Please enter a withdrawal transaction for the total amount paid to the Master Card

account and specify the Payee (The Payment Services Co.).

Also make sure that the payment amount is split among the three expenses to be

covered.

4. You make an investment using E*trade. You want to modify a transaction you did on January 5,

2010 where you bought 2 shares.

Modify the Buy transaction to reflect only 1 share bought.

5. You do a current status report and you realize that you want to keep an eye on the Master Card

account.

From the current status report, select the option to keep an eye on this account.

6. You need to reflect the investment of 2 shares on E*trade on February 5, 2010. You bought 2

shares at $120 with $30 fee. The security should be Maria.

Add the investment transaction in the program.

161

7. You want to make a payment with a check to pay your cell phone bill. Please do the following:

Verify that you have more than $145 available in your checking account.

On February 26, 2010, create a new transaction to reflect the payment of $145 for

your cell phone.

The transaction should be a withdrawal, using the checking account, and with your

name as the payee.

8. You just made a single payment (withdrawal) using your AMEX card in the amount of $1,050.00

to The Payment Services Co. on January 28. The payment should cover the following:

o 150.00 for life insurance

o 300.00 for electricity

o 650.00 for rent

Please enter the withdrawal transaction for the total amount paid to the AMEX Card

account and specify the Payee (The Payment Services Co.).

Also make sure that the payment amount is split among the three expenses to be

covered.

9. You make an investment using E*trade. You want to modify a transaction you made on January

5, 2010 when you did a reinvestment.

Modify the price of the stock to $185.

10. You do a summary report and realize that you want to include the investment information in the

report.

From the summary report, make sure you select the option to include the

investment information.

11. You need to reflect the investment of 3 shares from E*trade on February 17, 2010. You bought 2

shares at $88 with $38 fee. The security should be Maria.

Add the investment transaction into the program.

12. You want to make a payment with a check to pay your cell phone bill. Please do the following:

Verify if you have more than $145 available in your checking account.

On February 26, 2010, create a new transaction to reflect the payment of $145 to

your cell phone provider.

The transaction should be a withdrawal, using the checking account, and with your

name as the payee.

13. Now, you need to create a new transaction to reflect some grocery shopping you did on January

2, 2010 using your Master Card.

Enter the card purchase transaction for a total of $148.73. Payee is Weis, and it is a

personal transaction.

162

14. Go to the cash transaction executed on January 5, 2010 and do the following edits:

Change the money amount to be $90.00

Add the following phrase in the notes section “I added the tip in the total”.

163

Appendix D: Materials Used in Experiment 4

D.1 Pre-Experiment Questionnaire

Name________________________________

e-mail _______________________________

Age ___________________ Gender (optional) _________________

About your participation in this study

Please indicate the alternative that best describes your response to the following statements.

1. Your degree of commitment to this experiment and to do your best is:

I do not care 1 2 3 4 5 Extremely committed

2. For you, to maximize the amount of money is:

3. I do not care 1 2 3 4 5 Extremely committed

4. For you, to maximize the accuracy of your answers and minimize the time required is:

5. I do not care 1 2 3 4 5 Extremely committed

About your experience with smart phones

1. Do you have a cell phone that may be used as a smart phone?

* Yes

* No

2. For how long have you owned the smart phone?

* Never * 6 mo - 1 yr * more than 1 yr

164

3. Do you have a touch screen or a keypad phone?

* Neither * Touch screen * Keypad * Both

4. Please indicate your level of knowledge and familiarity with Blackberry phones:

Extremely unfamiliar 1 2 3 4 5 Extremely familiar

5. Please indicate your level of knowledge and familiarity with HTC phones:

Extremely unfamiliar 1 2 3 4 5 Extremely familiar

D.1.a About Your Personality

The Big Five factor model of personality was used to assess the personalities of the participants. This

model of personality uses five broad domains to describe human personality. The Big Five factors are as

follows: Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism. Openness refers

to inventiveness or curiousness and the degree of caution a person uses. Conscientiousness refers to the

tendency to show self-discipline rather than spontaneous behavior. Extraversion refers to the degree of

energy exhibited by a person and tendency to be connected with the external world. Agreeableness

relates to the tendency to show compassion and act with a cooperative attitude. Neuroticism refers to

the emotional stability of the person (McCrae and Costa 1987).

In this personality test, the participants were asked to answer a series of questions to determine a score

for each one of the dimensions mentioned, creating a personality profile.

In this experiment we examined only the Extraversion dimension of personality. A numerical/continuous

variable was created for the participants’ Extraversion scores.

The Big Five questionnaire does not generate personality types that are predetermined and based on

stereotypical models. Rather, it provides a score for each one of the five categories. In our case, we

165

decided to shift from MBTI to Big Five personality profile because we wanted to have a quantitative

variable to assess the personalities of the participants.

D.2 Affective Questionnaire

In this experiment the Positive and Negative Affect Schedule (PANAS) was used to assess the emotional

states of the participants. Please refer to Appendix B.4 for the PANAS questionnaire.

D.3 Emotion Elicitation Materials

Please refer to Appendix A for the emotion elicitation strategy and to Appendix B.1 for the emotion

elicitation instructions.

D.4 Usability Questionnaire

In this experiment, the System Usability Scale (SUS) was used to assess the perceived usability. Please

refer to Appendix C.3.

D.5 Software Usability Questionnaire – Adapted from the Software Usability Measurement Inventory

(SUMI)

1. This software responds too slowly to inputs.

Strongly Disagree 1 2 3 4 5 Strongly Agree

2. I would recommend this software to my colleagues.


3. I enjoy my sessions with this software.


4. The way the information is presented is clear and understandable.


5. The software seems to disrupt the way I normally like to arrange my work.


6. This software has a very attractive presentation.

166


7. It is relatively easy to move from one part of a task to another.


8. Tasks can be performed in a straightforward manner using this software.


9. Using this software is frustrating.


10. The organization of the menus and information seems logical.


D.6 Perceived Ease of Use

1. I find the system cumbersome to use.


2. Interacting with the system is often frustrating


3. I find it easy to get the system to do what I want to do.


4. Interacting with the system requires a lot of mental effort.


5. My interaction was clear and understandable.


6. Overall, I find the system easy to use.


167

D.7 List of Tasks

Please execute the tasks in the order given by the researcher conducting the study. You do not have to

complete all the tasks.

1. Go to the settings and adjust the following:

a. Select to receive e-mail alerts

b. For user name type: Trial user

c. For password type: psufootball123

2. Go to the transaction log, or where you can see all the transactions and do the following:

a. Edit the amount of the YMCA membership transaction to be $85.00.

3. With respect to the budget, or your financial plan, please answer the following:

a. Are the current expenses for the miscellaneous category OVER or UNDER the budget

amount?

4. Go to the Visa transaction done at Best Buy on 5/14/2010 and:

a. Add the following transaction number: 14630

5. Add a new transaction with the following information:

a. Name: Payment of insurance

b. Date: Sept 15 2010

c. Amount: $125.20

d. Account: Cash

6. From the Summary reports view, indicate the following:

a. Which expense category is the highest expense?

7. From all the Visa transactions, indicate the following:

a. What is the total amount for Debits on the Visa account?

8. Go to the Macy’s Store transaction on 5/17/2010 and do the following:

a. Change the amount to $37.82

9. From the budget, or financial plan view, please indicate the following:

a. The balance of the category – Dining out

b. Is the category over or under budget?

10. Add a new transaction with the following information:

a. Name: Soccer membership

b. Date: Sept 07 2010

c. Amount: $55.00

d. Account: Cash

11. From a budget perspective, and the overall summary of all accounts, please indicate:

a. The total amount of expenses


a. The overall difference between expenses and budget

b. Is the person OVER or UNDER budget overall?


a. Is the person OVER or UNDER budget overall?

168

14. From the summary reports view, please indicate:

a. Which category has the highest expenses?

15. From the summary reports view, please indicate:

a. Is the gasoline category OVER or UNDER budget?

16. From the transaction log, or from where you can see all the transactions, please indicate the

following:

a. What is the net income? For your reference: net income = revenues – expenses)

17. From the transaction log, or from where you can see all the transactions, please indicate the

following:

a. What is the net worth?

18. From the Visa transactions, please indicate:

a. The total credits

b. The total debits

19. From the Visa transactions, please indicate:

a. The net worth

b. The total debits

20. Go to the Bank of America transaction done at Nittany Apartments on 5/11/2010 and:

a. Add the following check number: 148

169

Appendix E: Consent Forms

Informed Consent Form for Social Science Research


Title of Project: IRB#33667

Principal Investigator: Maria A. Velazquez, Graduate Student 214 Leonhard Building University Park, PA 16802 (814) 862-8073; [email protected]

Advisor: Dr. Andris Freivalds 310 Leonhard Building University Park, PA 16802 (814) 865-7601; [email protected]

1. Purpose of the Study: The purpose of this study is to understand and analyze the effect of displays on visual cognition and reaction time. A display type will be presented and the visual cognition effects and reaction times will be analyzed.

2. Procedures to be followed: The experiment is composed of 2 sessions that are to be scheduled 7 days apart. You are required to be present 7 days from now to participate in the second session. In the first session you will agree with the researcher conducting the experiment on a meeting date and time, 7 days after today. Then, you will be asked to complete a set of 4 pre-experiment questionnaires to obtain information about your individual differences and preferences. You will then be debriefed about the purpose of the experiment, the monetary compensation and the steps required in the experiment and you will be provided with an instructions sheet. You will be asked to read and understand the instructions provided and execute a set of tasks in the computer program provided. Then you will complete three of the questionnaires again, and a final questionnaire. You will be compensated with a minimum of $16 total ($8 per session) and a maximum of $24 total at the end of the second session.

3. Discomforts and Risks: There are no risks in participating in this research beyond those experienced in everyday life. Some of the questions are personal and might cause discomfort, but you may choose to refrain yourself from providing an answer to such questions.

4. Benefits: You will have the opportunity to provide valuable feedback about information display and reaction times based on your experience.

5. Duration: It will take about 45 minutes to 1 hour to complete each session.

ORP OFFICE USE ONLY DO NOT REMOVE OR MODIFY

IRB# 33667 Doc. # 1001 The Pennsylvania State University Institutional Review Board Office for Research Protections Approval Date: 04-12-2010 JDM Expiration Date: 04-05-2011 JDM

mailto:[email protected]


170

6. Statement of Confidentiality: Your participation in this research is confidential. I will keep a master list of your information for a minimum of 3 years, which will be separated from your experiment files. This information will be used in the case we need to contact you as well as to get to know your background and your preferences. The data will be stored and secured at the Principal Investigator’s hard drive in a password protected file. The Pennsylvania State University’s Office for Research Protections and Institutional Review Board, and the Office for Human Research Protections in the Department of Health and Human Services may review records related to this project. In the event of a publication or presentation resulting from the research, no personally identifiable information will be shared.

7. Right to Ask Questions: Please contact Maria A. Velazquez at (814) 862-0192 with questions, complaints or concerns about this research. You can also call this number if you feel this study has harmed you. If you have any questions, concerns, problems about your rights as a research participant or would like to offer input, please contact The Pennsylvania State University’s Office for Research Protections (ORP) at (814) 865-1775. The ORP cannot answer questions about research procedures. Questions about research procedures can be answered by the research team conducting the study.

8. Payment for participation: Participants will receive a minimum of $16 for their participation, and a maximum of $24 as a function of their performance in the tasks.

9. Voluntary Participation: Your decision to be in this research is voluntary. You can stop at any time. You do not have to answer any questions you do not want to answer. Refusal to take part in or withdrawing from this study will involve no penalty or loss of benefits you would receive otherwise. You must be 18 years of age or older to take part in this research study.

Completion and return of the questionnaire implies your consent to participate in this research. Please keep a copy of this consent form for your records.

171






1. Purpose of the Study: The purpose of this study is to understand the degree to which visual reaction time and individual preferences affect the execution of tasks in mobile devices such as smart phones and the perceived usability of such products. This study explores the effects of individual preferences and the relationship between individual reaction times and the perceived usability of smart phones and software in such phones. The participants will have the opportunity to interact with new models of smart phones and provide feedback on their usability. Because the results of the study could be affected if the full purpose is known prior to your participation, the purpose of the study cannot be explained to you at this time. You will have an opportunity to receive a complete explanation of the purpose following completion of the study.

2. Procedures to be followed: The experiment is composed of 2 sessions that are to be scheduled 7 days apart. You are required to be present 7 days from now to participate in the second session. In the first session you will agree with the researcher conducting the experiment on a meeting date and time, 7 days after today. Then, you will be asked to complete a set of pre-experiment questionnaires to obtain information about your individual differences and preferences. Then you will be debriefed about the purpose of the experiment, the monetary compensation and the steps required in the experiment and you will be provided with an instructions sheet. You will be asked to read and understand the instructions provided and execute a set of tasks in the computer program provided, to determine your individual reaction time. Then you will be asked to complete a set of tasks using a smart phone and fill out final questionnaires. You will be compensated with a minimum of $16 total ($8 per session) and a maximum of $24 total at the end of the second session, depending on your performance in the reaction time task.

3. Discomforts and Risks: There are no risks in participating in this research beyond those experienced in everyday life. Some of the questions are personal and might cause discomfort, but you may choose to refrain from providing an answer.


IRB# 33677 Doc. # 1001 The Pennsylvania State University Institutional Review Board Office for Research Protections Approval Date: 04-23-10 SJH Expiration Date: 04-04-11 SJH



172

1. Benefits: You will have the opportunity to interact with new software for project management and two different smart phones. You will provide feedback on the cell phones/software usability based on your experience.

2. Duration: It will take about 1 hour to complete each session.

3. Statement of Confidentiality: Your participation in this research is confidential. I will keep a master list of your information for a minimum of 3 years, which will be separated from your experiment files. This information will be used in the case we need to contact you as well as to get to know your background and your preferences. The data will be stored and secured at the Principal Investigator’s hard drive in a password protected file. Penn State’s Office for Research Protections, the Social Science Institutional Review Board and the Office for Human Research Protections in the Department of Health and Human Services may review records related to this research study. In the event of a publication or presentation resulting from the research, no personally identifiable information will be shared. The Pennsylvania State University’s Office for Research Protections and Institutional Review Board, and the Office for Human Research Protections in the Department of Health and Human Services may review records related to this project.

4. Right to Ask Questions: Please contact Maria A. Velazquez at (814) 862-0192 with questions, complaints or concerns about this research. You can also call this number if you feel this study has harmed you. If you have any questions, concerns, problems about your rights as a research participant or would like to offer input, please contact The Pennsylvania State University’s Office for Research Protections (ORP) at (814) 865-1775. The ORP cannot answer questions about research procedures. Questions about research procedures can be answered by the research team.

5. Payment for participation: Participants will receive a minimum of $16 for their participation, and a maximum of $24 as a function of their performance in the computer tasks. The investigators will explain how the amount of compensation is calculated when the study begins.

6. Voluntary Participation: Your decision to be in this research is voluntary. You can stop at any time. You do not have to answer any questions you do not want to answer. Refusal to take part in or withdrawing from this study will involve no penalty or loss of benefits you would receive otherwise. You must be 18 years of age or older to take part in this research study. If you agree to take part in this research study and the information outlined above, please sign your name and indicate the date below. You will be given a copy of this consent form for your records.

______________________________________________ _____________________

Participant Signature Date

______________________________________________ _____________________

Person Obtaining Consent Date

173






1. Purpose of the Study: The purpose of this study is to understand the degree to which visual reaction time and individual preferences affect the execution of tasks in mobile devices such as smart phones and the perceived usability of such products. This study explores the effects of individual preferences and the relationship between individual reaction times and the perceived usability of smart phones and software in such phones. The participants will have the opportunity to interact with new models of smart phones and provide feedback on their usability. Because the results of the study could be affected if the full purpose is known prior to your participation, the purpose of the study cannot be explained to you at this time. You will have an opportunity to receive a complete explanation purpose following completion of the study.

2. Procedures to be followed: The experiment is composed of 2 sessions that are to be scheduled either 5 or 10 days apart. You are required to be present either 5 or 10 days from now to participate in the second session. In the first session you will agree with the researcher conducting the experiment on a meeting date and time, either 5 or 10 days after today. The timeframe between sessions will be assigned randomly once you agree to participate in the study. Then, you will be asked to complete a set of pre-experiment questionnaires to obtain information about your individual differences and preferences. Then you will be debriefed about the purpose of the experiment, the monetary compensation and the steps required in the experiment and you will be provided with an instructions sheet. You will be asked to read and understand the instructions provided and execute a set of tasks in the computer program provided, to determine your individual reaction time. Then you will be asked to complete a set of tasks using a smart phone and fill out final questionnaires. You will be compensated with a minimum of $16 total ($8 per session) and a maximum of $24 total at the end of the second session, depending on your performance in the reaction time task.



IRB# 33677 Doc. # 1001 The Pennsylvania State University Institutional Review Board Office for Research Protections Approval Date: 08-02-2010 JDM Expiration Date: 04-04-11 JDM



174

4. Benefits: You will have the opportunity to interact with new software for project management and two different smart phones. You will provide feedback on the cell phones/software usability based on your experience.

5. Duration: It will take about 1 hour to complete each session.

6. Statement of Confidentiality: Your participation in this research is confidential. I will keep a master list of your information for a minimum of 3 years, which will be separated from your experiment files. This information will be used in the case we need to contact you as well as to get to know your background and your preferences. The data will be stored and secured at the Principal Investigator’s hard drive in a password protected file. Penn State’s Office for Research Protections, the Social Science Institutional Review Board and the Office for Human Research Protections in the Department of Health and Human Services may review records related to this research study. In the event of a publication or presentation resulting from the research, no personally identifiable information will be shared. The Pennsylvania State University’s Office for Research Protections and Institutional Review Board, and the Office for Human Research Protections in the Department of Health and Human Services may review records related to this project.


8. Payment for participation: Participants will receive a minimum of $16 for their participation, and a maximum of $24 as a function of their performance in the computer tasks. The investigators will explain how the amount of compensation is calculated when the study begins.


If you agree to take part in this research study and the information outlined above, please sign your name and indicate the date below. You will be given a copy of this consent form for your records.

______________________________________________ _____________________


______________________________________________ _____________________

Person Obtaining Consent Date

175





Advisor: Dr. Andris Freivalds 310 Leonhard Building University Park, PA 16802 (814) 865-7601; [email protected] 1. Purpose of the Study: The purpose of this study is to understand the degree to which visual

reaction time and individual preferences affect the execution of tasks in mobile devices such as smart phones and the perceived usability of such products. This study explores the effects of individual preferences and the relationship between individual reaction times and the perceived usability of smart phones and software in such phones. The participants will have the opportunity to interact with new models of smart phones and provide feedback on their usability.

2. Procedures to be followed: The experiment is composed of 1 session. During the session you will be asked to complete a set of pre-experiment questionnaires to obtain information about your individual differences and preferences. Then you will be debriefed about the purpose of the experiment, the monetary compensation and the steps required in the experiment and you will be provided with an instructions sheet. You will be asked to read and understand the instructions provided and execute a set of tasks in the computer program provided, to determine your individual reaction time. Then you will be asked to complete a set of tasks using one smart phones and software, and fill out final questionnaires. Because the results of the study could be affected if the full purpose is known prior to your participation, the purpose of the study cannot be explained to you at this time. You will have an opportunity to receive a complete explanation purpose following completion of the study.


4. Benefits: You will have the opportunity to interact with new software for financial management and a smart phone. You will provide feedback on the cell phones/software usability based on your experience.

ORP OFFICE USE ONLY

DO NOT REMOVE OR MODIFY

IRB#34845 Doc. #1001


Office for Research Protections

Approval Date: 10-11-2010 JDM

Expiration Date: 08/31/2011 JDM



176

5. Statement of Confidentiality: Your participation in this research is confidential. I will keep a master list of your information for a minimum of 3 years, which will be separated from your experiment files. This information will be used in the case we need to contact you as well as to get to know your background and your preferences. The data will be stored and secured at the Principal Investigator’s hard drive in a password protected file. Penn State’s Office for Research Protections and the Office for Human Research Protections in the Department of Health and Human Services may review records related to this research study. In the event of a publication or presentation resulting from the research, no personally identifiable information will be shared. The Pennsylvania State University’s Office for Research Protections and Institutional Review Board, and the Office for Human Research Protections in the Department of Health and Human Services may review records related to this project.


7. Payment for participation: You will be compensated with a minimum of $8 and a maximum of $12 total at the end of the session, depending on your performance in the reaction time. The following formula will be used for the any extra compensation provided that would be based on your performance: Extra compensation (cents) = 10 cents (correct answer) – 5 cents (time to complete all tasks in minutes). Those who exceed the maximum shall be paid based on their performance as per the equation. The minimum compensation of $8 per session will always be given; regardless of the time it takes for you to complete the tasks and your accuracy.


If you agree to take part in this research study and the information outlined above, please sign your name and indicate the date below. You will be given a copy of this consent form for your records.

______________________________________________ _____________________


______________________________________________ _____________________

Signature of Person Obtaining Consent Date

177





Advisor: Dr. Andris Freivalds 310 Leonhard Building University Park, PA 16802 (814) 865-7601; [email protected] 1. Purpose of the Study: The purpose of this study is to understand the degree to which visual

reaction time and individual preferences affect the execution of tasks in mobile devices such as smart phones and the perceived usability of such products. This study explores the effects of individual preferences and the relationship between individual reaction times and the perceived usability of smart phones and software in such phones. The participants will have the opportunity to interact with new models of smart phones and provide feedback on their usability.

2. Procedures to be followed: The experiment is composed of 1 session. During the session you will be asked to complete a set of pre-experiment questionnaires to obtain information about your individual differences and preferences. Then you will be debriefed about the purpose of the experiment, the monetary compensation and the steps required in the experiment and you will be provided with an instructions sheet. You will be asked to read and understand the instructions provided and execute a set of tasks in the computer program provided, to determine your individual reaction time. Then you will be asked to complete a set of tasks using one smart phones and software, and fill out final questionnaires. Because the results of the study could be affected if the full purpose is known prior to your participation, the purpose of the study cannot be explained to you at this time. You will have an opportunity to receive a complete explanation purpose following completion of the study.


ORP OFFICE USE ONLY

DO NOT REMOVE OR MODIFY

IRB#34845 Doc. #1001


Office for Research Protections

Approval Date: 10-11-2010 JDM

Expiration Date: 08/31/2011 JDM



178

4. Benefits: You will have the opportunity to interact with new software for financial management and a smart phone. You will provide feedback on the cell phones/software usability based on your experience.

5. Statement of Confidentiality: Your participation in this research is confidential. I will keep a master list of your information for a minimum of 3 years, which will be separated from your experiment files. This information will be used in the case we need to contact you as well as to get to know your background and your preferences. The data will be stored and secured at the Principal Investigator’s hard drive in a password protected file. Penn State’s Office for Research Protections and the Office for Human Research Protections in the Department of Health and Human Services may review records related to this research study. In the event of a publication or presentation resulting from the research, no personally identifiable information will be shared. The Pennsylvania State University’s Office for Research Protections and Institutional Review Board, and the Office for Human Research Protections in the Department of Health and Human Services may review records related to this project.


7. Payment for participation: You will be compensated with a minimum of $8 and a maximum of $12 total at the end of the session, depending on your performance in the reaction time. The following formula will be used for the any extra compensation provided that would be based on your performance: Extra compensation (cents) = 10 cents (correct answer) – 5 cents (time to complete all tasks in minutes). Those who exceed the maximum shall be paid based on their performance as per the equation. The minimum compensation of $8 per session will always be given; regardless of the time it takes for you to complete the tasks and your accuracy.

8. Voluntary Participation: Your decision to be in this research is voluntary. You can stop at any time. You do not have to answer any questions you do not want to answer. Refusal to take part in or withdrawing from this study will involve no penalty or loss of benefits you would receive otherwise.

You must be 18 years of age or older to take part in this research study. If you agree to take part in this research study and the information outlined above, please sign your name and indicate the date below. You will be given a copy of this consent form for your records.

______________________________________________ _____________________


______________________________________________ _____________________

Signature of Person Obtaining Consent Date

179

Appendix F: Experiment Data

Table F.1 Data Gathered in Experiment 1 (Participants 1 to 15)

Participant Emotion Stimuli Initial PA Initial NAInitial

AGV

Initial

AGA

Initial

SAMV

Initial

SAMD

Initial

SAMAFinal PA Final NA

Final

AGV

Final

AGA

Final

SAMV

Final

SAMD

Final

SAMAAGV AGA SAMV SAMD SAMA Personality Importance

1 1 1 36 13 8 7 3 6 4 40 10 9 7 2 6 4 8 7 3 6 4 1 4

1 1 2 35 15 8 7 3 5 4 38 12 8 8 3 5 4 8 7 3 5 4 1 4

2 1 1 41 10 6 9 3 7 7 50 10 8 8 1 9 5 6 9 3 7 7 2 5

2 1 2 35 11 7 7 5 7 5 45 10 8 8 3 9 5 7 7 5 7 5 2 5

3 1 1 27 24 6 4 4 7 5 32 18 5 6 5 7 5 6 4 4 7 5 2 5

3 1 2 33 19 7 7 3 7 5 38 16 8 7 2 7 5 7 7 3 7 5 2 5

4 1 1 27 10 8 2 4 5 6 36 10 8 8 1 6 3 8 2 4 5 6 2 4

4 1 2 18 13 4 7 5 4 5 26 13 7 7 3 5 4 4 7 5 4 5 2 4

5 1 1 34 14 7 5 3 5 3 38 14 7 9 5 3 3 7 5 3 5 3 2 4

5 1 2 31 16 8 7 5 7 5 38 13 8 8 3 7 5 8 7 5 7 5 2 4

6 1 1 35 14 7 7 3 7 7 37 10 5 8 3 7 7 7 7 3 7 7 1 2

6 1 2 32 16 6 7 3 7 5 32 11 6 7 5 7 7 6 7 3 7 5 1 2

7 1 1 32 11 7 6 3 3 5 32 13 8 7 3 5 5 7 6 3 3 5 2 1

7 1 2 36 11 9 8 1 5 3 41 11 9 8 1 7 3 9 8 1 5 3 2 1

8 1 1 32 14 8 8 1 7 3 39 14 8 6 3 5 5 8 8 1 7 3 1 5

8 1 2 28 15 8 7 3 3 5 35 12 9 8 1 5 3 8 7 3 3 5 1 5

9 1 2 41 16 7 8 1 7 5 46 13 9 7 1 5 3 7 8 1 7 5 1 4

9 1 1 30 15 6 3 3 3 5 40 11 6 5 1 7 5 6 3 3 3 5 1 4

10 1 2 36 11 8 6 1 9 5 44 11 7 8 1 9 5 8 6 1 9 5 2 4

10 1 1 28 12 7 7 5 7 7 32 11 7 8 5 7 5 7 7 5 7 7 2 4

11 1 2 37 12 6 7 3 7 7 47 11 7 8 2 7 9 6 7 3 7 7 1 5

11 1 1 35 14 7 7 3 7 5 45 10 9 7 1 7 7 7 7 3 7 5 2 5

12 1 2 28 11 6 3 5 5 7 31 11 6 4 3 5 7 6 3 5 5 7 1 3

12 1 1 34 14 7 4 3 6 5 40 12 8 4 1 5 3 7 4 3 6 5 1 3

13 1 2 17 18 7 1 1 3 7 33 12 7 6 1 7 3 7 1 1 3 7 1 4

13 1 1 17 18 7 1 1 3 7 33 12 7 6 1 7 3 7 1 1 3 7 1 4

14 1 2 26 10 6 7 3 6 6 32 10 7 7 1 3 2 6 7 3 6 6 2 4

14 1 1 28 10 7 6 3 8 5 30 10 5 7 1 3 1 7 6 3 8 5 2 4

15 1 2 29 12 7 7 3 3 2 36 12 8 7 1 4 3 7 7 3 3 2 1 5

15 1 1 28 12 6 5 2 4 3 36 10 7 8 1 4 1 6 5 2 4 3 1 5

180


Participant Emotion Stimuli Initial PA Initial NAInitial

AGV

Initial

AGA

Initial

SAMV

Initial

SAMD

Initial

SAMAFinal PA Final NA

Final

AGV

Final

AGA

Final

SAMV

Final

SAMD

Final

SAMAAGV AGA SAMV SAMD SAMA Personality Importance

16 2 2 35 11 4 4 3 7 5 39 11 4 4 3 7 5 4 4 3 7 5 1 3

16 2 1 29 14 1 9 3 7 7 26 18 2 8 3 7 7 1 9 3 7 7 1 3

17 2 1 23 13 5 8 4 5 5 24 16 5 7 3 5 4 5 8 4 5 5 1 4

17 2 2 31 14 6 6 3 5 4 28 15 7 4 3 5 6 6 6 3 5 4 1 4

18 2 1 32 11 5 6 5 7 5 31 12 5 6 5 5 5 5 6 5 7 5 1 2

18 2 2 32 14 5 6 3 7 5 31 14 6 6 3 7 5 5 6 3 7 5 1 2

19 2 1 31 10 7 6 3 7 3 28 11 7 5 3 5 5 7 6 3 7 3 1 3

19 2 2 33 17 7 7 5 7 5 24 22 7 4 5 5 5 7 7 5 7 5 1 3

20 2 1 31 10 8 5 3 5 5 33 15 7 6 3 5 3 8 5 3 5 5 1 3

20 2 2 29 11 8 3 3 7 5 30 11 7 3 3 7 3 8 3 3 7 5 1 3

21 2 1 33 15 7 5 3 5 3 29 21 3 7 5 5 5 7 5 3 5 3 2 3

21 2 2 35 21 4 8 5 7 3 33 25 5 6 1 9 3 4 8 5 7 3 2 3

22 2 1 38 14 6 6 3 5 5 34 16 5 5 3 5 5 6 6 3 5 5 1 3

22 2 2 42 14 9 8 1 7 3 36 14 5 9 3 7 5 9 8 1 7 3 1 3

23 2 1 35 10 6 7 3 7 5 37 15 6 8 3 5 3 6 7 3 7 5 1 4

23 2 2 25 10 6 6 3 5 5 18 16 4 7 5 5 5 6 6 3 5 5 1 4

24 2 2 26 17 4 4 6 6 6 35 22 6 6 4 7 4 4 4 6 6 6 2 3

24 2 1 26 18 5 6 3 5 5 28 21 7 4 3 5 4 5 6 3 5 5 2 3

25 2 2 30 11 7 6 3 5 5 26 19 6 6 3 3 7 7 6 3 5 5 1 5

25 2 1 27 10 7 7 3 5 5 25 16 6 5 3 3 5 7 7 3 5 5 1 5

26 2 2 28 12 3 5 4 3 5 26 15 3 4 5 3 5 3 5 4 3 5 1 4

26 2 1 26 12 5 6 3 4 3 22 17 4 5 5 5 3 5 6 3 4 3 1 4

27 2 2 30 14 5 6 4 7 5 29 15 4 7 3 7 5 5 6 4 7 5 1 3

27 2 1 30 13 4 3 3 2 3 26 17 3 3 4 2 3 4 3 3 2 3 1 3

28 2 2 29 16 7 7 5 7 5 24 23 7 4 5 5 3 7 7 5 7 5 2 4

28 2 1 27 10 5 4 2 3 4 25 13 3 6 4 3 2 5 4 2 3 4 2 4

29 2 2 35 20 4 8 5 7 3 33 25 5 7 1 9 4 4 8 5 7 3 1 3

29 2 1 34 11 7 7 1 2 2 29 14 5 7 2 3 2 7 7 1 2 2 1 3

30 2 2 34 17 5 7 3 7 5 33 18 5 6 3 5 5 5 7 3 7 5 2 4

30 2 1 37 16 8 3 1 7 5 38 20 6 4 3 5 3 8 3 1 7 5 2 4

181


Participant Emotion Emo Seq Session Importance InitialPA InitialNA FinalPA FinalNA SUS Score IBM Score PrevExp

1 1 1 1 3.00 26.00 13.00 29.00 13.00 42.50 3.89 1

1 2 1 2 3.00 29.00 11.00 28.00 14.00 42.50 3.32 1

2 2 2 1 3.50 32.00 18.00 24.00 24.00 60.00 3.37 2

2 1 2 2 3.50 34.00 14.00 34.00 10.00 65.00 2.68 2

3 1 1 1 5.00 32.00 12.00 36.00 12.00 50.00 3.37 1

3 2 1 2 5.00 28.00 21.00 28.00 20.00 47.50 3.05 1

4 2 2 1 4.00 45.00 11.00 39.00 19.00 87.50 2.00 2

4 1 2 2 4.00 46.00 10.00 46.00 10.00 72.50 2.47 2

5 1 1 1 4.00 34.00 13.00 44.00 13.00 72.50 4.21 1

5 2 1 2 4.00 46.00 16.00 31.00 21.00 62.50 4.89 1

6 2 2 1 3.75 39.00 16.00 38.00 12.00 27.50 3.05 1

6 1 2 2 3.75 24.00 13.00 24.00 10.00 55.00 4.53 1

7 1 1 1 4.25 29.00 19.00 42.00 10.00 62.50 2.79 1

7 2 1 2 4.25 39.00 13.00 41.00 13.00 15.00 5.68 1

8 2 2 1 3.75 37.00 10.00 36.00 14.00 50.00 2.58 2

8 1 2 2 3.75 35.00 12.00 39.00 10.00 72.50 2.58 2

9 1 1 1 3.75 26.00 11.00 21.00 11.00 45.00 4.74 2

9 2 1 2 3.75 23.00 12.00 25.00 10.00 42.50 4.79 2

10 2 2 1 3.75 37.00 20.00 39.00 19.00 95.00 1.21 2

10 1 2 2 3.75 36.00 13.00 32.00 13.00 95.00 5.89 2

11 1 1 1 2.75 39.00 23.00 42.00 13.00 77.50 1.79 2

11 2 1 2 2.75 36.00 14.00 41.00 12.00 85.00 1.42 2

12 2 2 1 2.75 28.00 15.00 18.00 15.00 50.00 2.42 2

12 1 2 2 2.75 33.00 10.00 33.00 10.00 82.50 4.89 2

13 1 1 1 4.00 36.00 14.00 39.00 12.00 55.00 5.32 2

13 2 1 2 4.00 28.00 17.00 26.00 19.00 50.00 4.89 2

14 2 2 1 2.75 21.00 11.00 16.00 15.00 40.00 3.95 2

14 1 2 2 2.75 18.00 12.00 14.00 13.00 37.50 4.05 2

182


Participant Emotion Emo Seq Session Importance InitialPA InitialNA FinalPA FinalNA SUS Score IBM Score PrevExp

15 1 1 1 3.50 35.00 11.00 39.00 11.00 70.00 4.42 2

15 2 1 2 3.50 30.00 10.00 33.00 10.00 32.50 3.37 2

16 2 2 1 4.75 34.00 17.00 32.00 14.00 35.00 4.68 2

16 1 2 2 4.75 33.00 10.00 35.00 10.00 40.00 4.47 2

17 1 1 1 4.75 43.00 12.00 47.00 11.00 87.50 2.21 1

17 2 1 2 4.75 35.00 16.00 41.00 12.00 85.00 1.58 1

18 2 2 1 3.50 30.00 19.00 37.00 18.00 45.00 2.63 2

18 1 2 2 3.50 35.00 24.00 37.00 22.00 65.00 4.05 2

19 1 1 1 3.25 29.00 20.00 33.00 17.00 65.00 3.84 2

19 2 1 2 3.25 31.00 14.00 30.00 15.00 42.50 2.74 2

20 2 2 1 3.75 32.00 17.00 21.00 18.00 37.50 2.84 2

20 1 2 2 3.75 22.00 19.00 25.00 13.00 72.50 4.58 2

21 1 1 1 5.00 37.00 21.00 39.00 18.00 52.50 5.68 2

21 2 1 2 5.00 33.00 24.00 36.00 35.00 30.00 4.63 2

22 2 2 1 4.25 40.00 17.00 35.00 25.00 80.00 1.95 2

22 1 2 2 4.25 35.00 16.00 34.00 13.00 75.00 5.84 2

23 1 1 1 3.00 28.00 12.00 26.00 11.00 55.00 4.16 1

23 2 1 2 3.00 29.00 10.00 26.00 14.00 35.00 3.06 1

24 2 2 1 5.00 38.00 11.00 32.00 13.00 45.00 4.37 1

24 1 2 2 5.00 38.00 10.00 38.00 10.00 65.00 4.37 1

25 1 1 1 3.75 37.00 23.00 37.00 15.00 62.50 3.32 2

25 2 1 2 3.75 37.00 16.00 37.00 17.00 47.50 4.00 2

26 2 2 1 4.00 33.00 11.00 34.00 11.00 87.50 2.11 2

26 1 2 2 4.00 30.00 16.00 28.00 18.00 92.50 2.21 2

27 1 1 1 3.25 25.00 15.00 28.00 17.00 85.00 2.79 2

27 2 1 2 3.25 22.00 13.00 17.00 17.00 72.50 3.37 2

183

Table F.5 Data Gathered in Experiment 3

Participant Emotion Emo Seq Session Days Elapsed Importance InitialPA InitialNA FinalPA FinalNA SUSScore

1 1 1 1 2 4.5 27.0 11.0 23.0 14.0 37.5

1 2 1 2 2 4.5 25.0 14.0 20.0 20.0 30.0

2 2 2 1 2 3.3 32.0 14.0 20.0 21.0 20.0

2 1 2 2 2 3.3 22.0 10.0 33.0 11.0 57.5

3 1 1 1 1 3.3 29.0 13.0 31.0 10.0 70.0

3 2 1 2 1 3.3 22.0 12.0 26.0 12.0 65.0

4 2 2 1 1 2.8 34.0 13.0 23.0 20.0 17.5

4 1 2 2 1 2.8 36.0 17.0 37.0 10.0 65.0

5 1 1 1 2 3.5 31.0 12.0 36.0 11.0 65.0

5 2 1 2 2 3.5 26.0 10.0 19.0 19.0 52.5

6 2 2 1 2 3.8 31.0 19.0 33.0 20.0 35.0

6 1 2 2 2 3.8 34.0 18.0 36.0 13.0 80.0

7 1 1 1 1 4.0 29.0 18.0 29.0 21.0 35.0

7 2 1 2 1 4.0 20.0 16.0 20.0 16.0 32.5

8 2 2 1 1 4.5 47.0 10.0 46.0 14.0 45.0

8 1 2 2 1 4.5 29.0 10.0 36.0 10.0 62.5

9 1 1 1 2 4.3 33.0 11.0 33.0 13.0 67.5

9 2 1 2 2 4.3 33.0 11.0 33.0 15.0 62.5

10 2 2 1 2 3.5 30.0 23.0 26.0 15.0 50.0

10 1 2 2 2 3.5 37.0 11.0 34.0 11.0 70.0

11 1 1 1 1 4.8 27.0 14.0 39.0 13.0 62.5

11 2 1 2 1 4.8 38.0 13.0 31.0 11.0 75.0

12 2 2 1 1 3.8 33.0 13.0 32.0 13.0 55.0

12 1 2 2 1 3.8 28.0 11.0 34.0 10.0 40.0

13 1 1 1 2 3.8 26.0 14.0 25.0 11.0 40.0

13 2 1 2 2 3.8 28.0 10.0 19.0 11.0 27.5

14 2 2 1 2 3.8 36.0 13.0 41.0 15.0 37.5

14 1 2 2 2 3.8 38.0 11.0 39.0 10.0 60.0

15 1 1 1 1 3.3 18.0 11.0 27.0 25.0 57.5

15 2 1 2 1 3.3 22.0 17.0 21.0 25.0 32.5

16 2 2 1 1 4.3 35.0 13.0 23.0 20.0 52.5

16 1 2 2 1 4.3 24.0 11.0 34.0 11.0 55.0

184

Table F.6 Data Gathered in Experiment 4 (Participants 1-20)

Participant Blocks Emotion

Smart

Phone Software

SUS

Score

SW

Usability

Task

Completion

Rate

Perceived

Ease of Use

Familiarity

with

Blackberry

Familiarity

with HTC Familiarity

Time

Owned

Smart

Phone?

Input

Familiarity Inital PA Initial NA Final PA Final NA

1 1 1 2 2 65 2.6 0.13333 3.66667 4 4 4 1 1 1 10 17 16 16

2 1 2 1 1 47.5 3.9 0.4 2.16667 2 2 2 0 0 1 23 10 29 7

3 1 2 2 1 62.5 3 0.33333 3.5 3 1 1 2 1 1 26 7 15 11

4 1 1 1 1 55 3.6 0.26667 2.83333 1 1 1 0 0 0 28 8 24 9

5 1 1 2 1 85 4 0.53333 4.33333 2 2 2 0 0 0 27 7 24 8

6 1 2 2 2 35 2 0.33333 2.33333 2 1 1 2 1 1 27 13 21 14

7 1 2 1 2 20 2.8 0.2 1.83333 2 3 2 1 1 0 33 8 19 8

8 1 1 1 2 50 3.2 0.26667 3 1 2 1 0 0 1 13 14 24 11

9 2 1 1 1 100 4.9 0.4 5 5 4 5 1 1 1 20 14 22 9

10 2 2 2 2 45 1.8 0.06667 2.16667 3 3 3 0 0 1 27 7 27 10

11 2 1 1 2 65 2.6 0.26667 3.66667 2 1 2 0 0 1 13 11 19 13

12 2 2 2 1 5 1.7 0.13333 1.16667 1 1 1 0 0 1 26 8 26 12

13 2 2 1 2 40 2.6 0.26667 2.33333 4 4 4 1 1 1 17 13 11 9

14 2 2 1 1 57.5 2.4 0.4 2.33333 1 1 1 2 1 0 7 13 23 7

15 2 1 2 1 85 4 0.53333 4.33333 2 2 2 0 0 0 26 19 25 7

16 2 1 2 2 92.5 4.9 0.26667 4.83333 1 1 1 1 1 1 25 8 32 7

17 3 2 2 1 62.5 3.6 0.26667 3.5 3 1 1 2 1 1 22 11 23 11

18 3 2 1 2 27.5 2.3 0.26667 2 2 5 2 2 1 1 22 10 15 9

19 3 2 2 2 42.5 2.7 0.2 2.83333 1 1 1 1 1 1 15 24 21 15

20 3 1 2 1 25 2.6 0.66667 2.16667 2 3 3 1 1 1 7 13 10 11

185

Table F.7 Data Gathered in Experiment 4 (Participants 21-40)

Participant Blocks Emotion

Smart

Phone Software

SUS

Score

SW

Usability

Task

Completion

Rate

Perceived

Ease of

Use

Familiarity

with

Blackberry

Familiarity

with HTC Familiarity

Time

Owned

Smart

Phone?

Input

Familiarity Inital PA Initial NA Final PA Final NA

21 3 1 1 1 45 2.4 0.6 2.33333 3 1 3 1 0 1 15 9 13 7

22 3 2 1 1 50 2.4 0.73333 2.33333 5 4 5 2 1 0 23 16 27 8

23 3 1 1 2 50 2 0.26667 2 1 3 1 0 0 1 27 8 17 14

24 3 1 2 2 72.5 2.7 0.13333 3.5 1 3 3 2 1 1 27 9 22 11

25 4 1 2 1 60 3 0.33333 3.66667 2 2 2 2 1 1 29 9 21 9

26 4 1 1 2 57.5 2.2 0.2 2.83333 2 4 2 0 0 0 23 8 13 13

27 4 2 2 1 15 4.3 0.6 3.5 4 4 4 0 0 0 16 8 9 11

28 4 1 2 2 45 2.5 0.13333 2.33333 1 1 1 0 0 0 21 15 15 17

29 4 2 1 2 22.5 1.4 0.06667 1.33333 1 1 1 0 0 1 27 7 23 7

30 4 2 2 2 22.5 2 0.13333 1.66667 4 4 4 0 0 0 10 17 22 9

31 4 2 1 1 67.5 2.9 0.66667 3.33333 5 5 5 1 1 1 23 10 16 15

32 4 1 1 1 100 4.7 1 5 5 2 5 2 1 1 25 8 25 8

33 5 1 2 2 57.5 1.6 0.06667 1.5 4 2 2 1 1 1 28 8 28 8

34 5 2 1 2 47.5 3 0.2 3.33333 1 3 1 0 0 1 22 9 11 14

35 5 1 1 2 77.5 1.7 0.2 1.83333 1 1 1 0 0 0 26 9 20 7

36 5 2 1 1 10 3.4 0.4 2.83333 5 4 5 2 1 0 23 15 17 24

37 5 2 2 2 47.5 3 0.33333 2.33333 1 1 1 1 1 1 23 7 16 10

38 5 1 2 1 25 1.9 0.26667 1.66667 4 2 2 1 1 1 25 7 20 11

39 5 1 1 1 62.5 3.1 0.4 3.33333 3 4 3 2 1 0 23 9 24 7

40 5 2 2 1 85 4.3 0.4 4.16667 2 2 2 1 1 1 26 9 15 13

Vita

Maria Angelica Velazquez, P.E.

Maria Angelica Velazquez obtained her B.S. in Industrial Engineering in 2003 from the

University of Puerto Rico, Mayagüez Campus. She worked in Ethicon for Johnson and

Johnson as a quality assurance engineer. She obtained her M.S. degree in Industrial and

Management Engineering in 2004 from the Rensselaer Polytechnic Institute, located in

Troy, NY. In 2010 Maria Angelica obtained her Ph.D. in Industrial and Manufacturing

Engineering at The Pennsylvania State University. Maria is the author and co-author of

numerous journal and conference articles. She has been the recipient of several

fellowships and scholarships, such as the NSF Alliance for Graduate Education and the

Professorate (AGEP) Fellowship, the AT&T Laboratories ALFP Fellowship, the Graham

Endowed Fellowship, the Keefauver Scholarship, the General Electric Foundation

Fellowship, and the Alfred P. Sloan Dissertation Fellowship. She is a member of the

Institute of Industrial Engineers (IIE), Human Factors and Ergonomics Society (HFES), Tau

Beta Pi Engineering honor society, Alpha Pi Mu Industrial Engineering honor society, and

the Golden Key honor society. She is also a Professional Engineer (PE). Her research

interests include Human Factors, Usability Engineering, and Human-Computer

Interaction.

UNDERSTANDING THE EFFECTS OF POSITIVE AND NEGATIVE …

Documents

Transcript of UNDERSTANDING THE EFFECTS OF POSITIVE AND NEGATIVE …