Enhancing Information Awareness Through Speech Induced Anthropomorphism PhD Examination Nuno Jorge...

Enhancing Information Awareness Through Speech Induced Anthropomorphism

PhD Examination

Nuno Jorge Gonçalves de Magalhães Ribeiro

External examiner: Professor Philip Barker

Internal examiner: Dr. Alistair Edwards

Department of Computer Science

University of York

06 February 2002

06 February 2002 PhD Examination 2

Structure of the Examination(follow presentation, 4 blocks, discussion at the end of each block)

Block Contents Chapters

1 Motivation, main design concepts 1, 2 and 3

Break Questions, discussion

2 Esperimental environment: design and implementation 4 and 5


3 Empirical study: objectives, method, results 6


4 Research achievements and contributions 7



MotivationProblems with current interfaces

for information awareness (lack of humanized interaction)

• Finding visually hidden information is distracting (e.g. information hidden behind opened windows, dynamic information)

• Lack of semantics in alerting cues (e.g. blinking icons, sounds), or visually intrusive displays (e.g. windows that obscure info. being accessed)

• Lack of awareness and co-ordination between high-priority and unimportant alerts to be reported (e.g. junk e-mail, status info.)

• Lack of a unified interruption mechanism for notifications (imminent and just occurred events) and reminders (past, but relevant, events)


How to Humanize Information Awareness?• One way is to exploit speech to deliver alerting messages

(notifications and reminders)

BUT(because listening to speech cues anthropomorphic feelings)

• The reception of speech demands that the interface reflects human social norms

at the delivery level –

design should humanize the delivery of speech-based messages at the content level –

design should humanize the contents of speech-based messages

3 questions: Why speech? How to humanize the delivery? How to humanize the content?


Why Speech?There is an urgent need to find out the situations in which

speech enhances the user interface and the situations in

which it is detrimental to the user interface

This thesis suggests that speech can be used with advantages to:

• enable less disruptive interruptions• convey more meaningful notifications• design of an integrated notification mechanism


How to Humanize Message Delivery?(in order to reinforce social acceptability of the speech interface)

This thesis suggests that there are 3 fundamental design aspects:

1. Attentive interruptions (minimise user disruption)

• When to interrupt: precedence <= prioritiy and urgency,

• How to interrupt: get attention, modulate the audio channel when it is already being used

2. Linguistic variation (avoid irritating repetitive utterances while maintaining the semantics)

• Sources: lexical, syntactical, phonetic

3. Politeness (attenuate irritating effect of interruptions)

• Simple courtesy (e.g. Excuse me, I’m sorry)

• Design based on politeness strategies (e.g. direct, positive, negative)


How to Humanize Message Contents?(in order to enhance the awareness of information that is hidden from view)

This thesis suggests that there are 2 fundamental design aspects:

1. Short alerting messages: just relevant content (because speech is a serial

channel)

2. Context-based cues: extracted from items (because context plays a major role in recalling information)

• Forward Cues: Kind, Time, Quantity, People, Nature, Description, Place

End of block 1

Questions

Discussion


Humanized Experimental EnvironmentIn presenting the design, the thesis illustrates:

• How to split a complex environment into a number of separate software agents that perform specific monitoring tasks.

• How to develop a software architecture that is based on a collection of specialised software agents.

• How to design the internal architecture of each software agent, in such a way that it collects and conveys context information.

• How to communicate collected information between the software agents comprising the environment.

• How to transform collected information into appropriate spoken messages to be delivered to the user.

• How to deliver the spoken information following typical social norms.


Experimental Environment ArchitectureUs er monitor a g e n t

(s e rv ic e a g e n t )

E-m a il agent(s pec ialis ed agent)

Prin te r agent(s pec ialis ed agent)

D ia ry agent(s pec ialis ed agent)

S pe a k in g agent( in terfac e agent)

Us er

s ubs c r ibeus erpres enc e

m es agesenc los ingnotif ic ationsor rem inders

ac know ledge

S peec h output(s poken inform ationpres entation)

S peec h input(us er 's c om m ands )

Launc happlic ations

M ultim edia p layerE-m ail applic ation

D iary applic ationW eb brow s er

W ord proc es s or


Steps in the Humanization Process

1. Deciding when to interrupt (hum. delivery)(specialised agents: I.L. <= urgency, priority, user presence)

2. Extracting context cues (hum. content) (specialised agents: forward cues)

3. Including linguistic variation (hum. content and delivery)(speaking agent: templates for lexical and syntactical variations; prosody markers for phonetic variation)

4. Including politeness (hum. delivery) (speaking agent: attention-grabbers, templates for linguistic style)

5. Deciding how to interrupt (hum. delivery)(speaking agent: attention-grabbing, audio channel modulation)


Experimental Environment Prototype

• Windows environment (familiar, limitations)

• Implements specialised agents:• Environmental awareness, context extraction, message composition• E-mail (MAPI), Diary (TXT), Printer (Default printer queue)

• Implements communication protocol:• Comprises purposefully designed message types and contents• Relies on transport mechanism based on Dynamic Data Exchange (DDE)

• Implements speaking agent:• MS Agent client• Creates spoken reminders and notifications including linguistic variation

and politeness• Interrupts by grabbing the user attention, fading audio channel• Uses TTS synthesiser and Speech Recognition engine

End of block 2

Questions

Discussion


Empirical Study: Objectives (1/3)Assess user´s perceptions of the adequacy of anthropomorphic behaviours

Anthropomorphic characteristicsDependent variables

(adjectives)

InterruptionsAppropriateUsefulDisruptive

Single voiceConfusingUniformAppropriate

Music interruptionNaturalInterfering

Attention-gettersUsefulAnnoyingNatural

Spoken message semantics

Easy to recognise wordsEasy to understand meaningHelpfulAppropriate

PolitenessDegreeAppropriateIrritating

Linguistic variationIrritatingAppropriate


Empirical Study: Objectives (2/3)Assess user´s perceptions of the usefulness and humanization of the system

• Useful: helpful, convenient, comfortable.

• Humanized: pleasant, unobtrusive, satisfying, reliable, efficient, likeable.

Dimensions

1 7

Assistant pleasantness Unpleasant Pleasant

Assistant helpfulness Unhelpful Helpful

Assistant convenience Inconvenient Convenient

Assistant reliability Unreliable Reliable

Assistant comfort Uncomfortable Comfortable

Assistant efficiency Inefficient Efficient

Assistant satisfaction Frustrating Satisfying

Assistant intrusiveness Unobtrusive Intrusive

Assistant likeability Horrible Lovely

Dependent variables (semantic differential)


Empirical Study: Objectives (3/3)• Assess differences in those user’s perceptions according to

psychological (MBTI) and other personal characteristics:

• Heuristic (N, F) versus Analytic (S, T) psychological types

• Intuition (N) / Sensing (S) dimension => preferences for info. Acquisition (awareness - first step in the acquisition of info.)

• Feeling (F) / Thinking (T) dimension => preferences for info. processing

• Gender, O.S. proficiency, familiarity with TTS


Empirical Study: Evaluation Method• Quasi-experimental model

• Extensive laboratory test that simulated closely a real world situation.

• 14 specific research questions

• IV: subject variables; DV: subjective user’s opinions and perceptions

• Participants• 15 participants selected from the population of the C.S. Department

• Measurement of variables• Attitude measurement through the use of questionnaires

• 5-point scales: 1=Strongly Disagree, 3=Neither disagree nor agree, 5=Strongly agree

• 7-point scales: 1=Not at all, 4= Undecided, 7= Very much so

• 7-point semantic differential scales: e.g. 1 = unpleasant, 7 = pleasant


Empirical Study: Experimental Design and Procedure

15 random ly s elec ted tes t par tic ipants

Ans w er es per tis e andpers onality type s elf

as s es s m ent ques tionnaire

P erform S yntheticS peec h T rain ing T as k

P erformw eb s earc h

Ac tiv ity

P erforme-m ail

m anagem entAc tiv ity

P erformw ord

proc es s ingAc tiv ity

P erformprin tingAc tiv ity

Ans w er s ubjec tives ys tem apprec iation

ques tionnaire

S tar tT es t

EndT es t

Res earchTas k

S cenario

S peakingAs s is tant

e-m ail

to do

prin ting

Interruptions(s poken rem indersand notif ic ations )

00:00

00:15

00:30

02:00

02:30


Empirical Study: Summary of Results (1/2)

1. Speech is useful to convey information that is hidden from view (but content must be humanized through context cues)

• The prototype was perceived as supporting the awareness of visually hidden information.

• It was considered particularly useful by heuristic information gatherers – speech-based alerts allowed participants to appreciate the immediate relevance of an informational event, that is the relationship and connection between the alerting information and their personal interests and activities.

2. Information awareness is an arena where using speech is a valid and beneficial approach

• Participants indicated that speech-based interruptions were not overly disruptive and allowed them to decide when to deal with information.


Empirical Study: Summary of Results (2/2)

3. The anthropomorphic behaviours were considered to be important and adequate to reinforce the social acceptability of the speech-based system

• Attentive interurptions, linguistic variation and politeness combine to avoid socially incorrect behaviours, such as respectively unnecessary interruptions, irritating repetitions and rudeness.

• Urgency and priority-based interruptions were perceived as relevant.

• Attention-grabbing and audio modulation were perceived as natural (or expected) behaviours.

• When messages were varied they did not irritate participants. They were found so natural that some people did not even notice the variation.

• Polite behaviour is appropriate to avoid rudeness when getting a person’s attention, but it invokes the perception of a personality and thus must be used in a way so that it suggests a consistent personality.

End of block 3

Questions

Discussion


Summary of the Research Achievements• How to reinforce the social acceptability of a speaking

systemHumanize interruptions, vary language, use politeness

• How to enhance information awarenessSupport speech-based alerts including contextual cues

• Experimental environment: design and prototypeAgent-based approach, protocol, delivery of spoken alerts

• Empirical studyValidated the anthropomorphic speech-based approach for info.

awareness

• Practical guidelines for the design of speech-based systems for information awareness.


Contributions (1/4)• The thesis contributes to understand the

importance of, and how to exploit, speech to enhance the awareness of personal information that is hidden from view in a desktop-based environment:

• How to humanize the delivery of speech-based alerts

• How to humanize the contents of speech-based alerts


Contributions (2/4)• The thesis contributes with a unique experimental

environment that integrates three different anthropomorphic characteristics for the delivery

• Attentive interruptions, linguistic variation, politeness

• They reinforce the social acceptability of the system - it was unexpectedly well accepted by users.


Contributions (3/4)• The thesis contributes to the research community

in that it provides an extensible agent-based architecture for an experimental environment:• Can be extended with additional specialised agents• Can accommodate the incorporation of additional

anthropomorphic behaviours (e.g. gender, personality, emotion)

• Can be used to support controlled experiments with larger and more heterogeneous user groups


Contributions (4/4)• The thesis contributes to expand current

knowledge about how to evaluate user’s perceptions of anthropomorphic systems that employ speech, and suggests a range of appropriate subjective measures• It measured the adequacy of speech-induced

anthropomorphic behaviours through specific questionnaires

• It measured the perceptions of usefulness and humanization through careful designed questionnaires

Enhancing Information Awareness Through Speech Induced Anthropomorphism PhD Examination Nuno Jorge...

Documents

Transcript of Enhancing Information Awareness Through Speech Induced Anthropomorphism PhD Examination Nuno Jorge...