error handling – Higgins / Galatea

1

error handling – Higgins / Galatea

Dialogs on Dialogs Group

July 2005

2

work by …

Gabriel Skantzeph.d. studentKTH, Stockholm

“I am doing research on spoken dialogue systems. More specifically, I am interested in studying miscommunication and error handling, but also in the representation and modelling of utterances and dialogue, as well as conducting experiments with users.“

and co-authors: J. Edlund, D. House, R. Carlson

3

3 papers

HigginsHiggins – a spoken dialogue system for investigating error handling techniques,

Edlund, Skantze, Carlson [2004]

GalateaGALATEA: A Discourse Modeller Supporting Concept-Level Error Handling in

Spoken Dialog Systems, Skantze [2005]

Prosody & ClarificationsThe Effects of Prosodic Features on the Interpretation of Clarification Ellipses, Edlund, House, Skantze [2004]

4

1st paper






5

Higgins

practical goal of Higgins project build a collaborative dialog system in which error handling

ideas can be tested empirically

error handling issues, plus incremental dialogue processing on-line prosodic feature extraction robust interpretation flexible generation and output

6

domain

pedestrian city navigation and guiding user gives system a destination system guides user by giving verbal instructions

complex large variety of error types semantic structures can be quite complex reference resolution

domain can be extended even further

7

architecture

follow-up from Adapt everything is XML

domain objects utterance semantics discourse model database content system output (before surface) 3D city model

8

research issues

early detection and correction late detection incrementality error recovery

9

early detection and correction

KTH LVCSR – output likely to contain errors

robust interpretation Pickering: some syntactic analysis is needed

e.g. relations between objects but handles insertions and non-agreement phrases

humans - good at early detection (woz)

10

late detection and correction

discourse modeller (GALATEA) joins several results from Pickering into a discourse model adds grounding information can be manipulated later

remove concepts which turn out not to be grounded

11

incrementality

end-pointers cause trouble even more so in this domain

better:

12

incrementality [2]

all components support incremental processing

several issues when to barge in? (semantic content and prosody) longer-than-utterance units: interpreter or dialog manager? rapid and unobtrusive feedback: challenge for synthesis

13

error recovery

signaling non-understandings decreased experience of task success slower recovery

ask other task-related question

14

2nd paper






15

GALATEA

a discourse modeller for conversational spoken dialog systems builds a discourse model (what has been said during the

discourse)

resolution of ellipses & anaphora tracks the grounding status

who said what when (plus confidence information) can be used for concept-level error handling

16

should do grounding at concept level

explicit and implicit verification on whole utterance can be tedious and unnatural

45% of clarifications in BNC are fragmentary / elliptical

17

should do grounding at concept level

Traum (1994) – utterance level computational model of grounding

Larsson (2002) – issue-level computational model of grounding in Issue-Based DM

Rieser (2004), Schlangen (2004): systems capable of fragmentary clarification requests, but models do not handle user reactions

systems should keep grounding information at the concept level

like RavenClaw?

18

semantic representation

rooted unordered trees of semantic concepts nodes: attr-value pairs, objects, relations, properties

19

semantic representation

enhanced with “meta”-information confidence communicative acts info is new / given

20

ellipsis resolution

transforms ellipsis into full propositions

rule based ~10 rules domain-specific

21

anaphora resolution

keeps a list of entities (talked about) assigns ids

when given entities are added to the discourse, look up the antecedent

if found, unification (and move to the top of the entity list)

unification also allows entities to be referred to in new ways

how does this fare and compare?

22

grounding status

who added the concept? in which turn? how confident?

may be used by the action manager for instance remove all items with high grounding status

when referring to an entity

23

updating grounding status

24

late error detection

discover inconsistencies in discourse model

look at grounding status to see where error may be concept can be removed

25

future

methods for automatic tuning of strategy selection

extend to track confidence and grounding status at different levels

evaluate how people respond to incorrect

confirmations, and how can that information be used to update grounding status

error recovery after non-understandings other domains

26

3rd paper






27

prosody in clarifications

effects of prosodic features on interpretation of elliptical clarifications U: Further ahead on the right I see a red building… S: Red (?)

vary prosodic features study impact on user’s understanding

of the system’s intention

28

motivation

long (whole utterance) confirmations are not good tedious, unnatural BNC corpus: 45% of clarifications are elliptical

short confirmations make dialog more efficient by focusing on the actual

problematic fragments however

interpretation depends on context and prosody

29

3 readings

U: Further ahead on the right I see a red building… S: Red (?)

Ok, red [all positive]

Do you really mean red? What do you mean by red? [positive perception, negative understanding]

Did you say red? [positive contact, negative perception]

30

stimuli

3 test words [red, blue, yellow] di-phone voice (MBROLA) manipulated

peak position [mid, early, late / 100ms] peak height [130Hz / 160 Hz] vowel duration [normal, long / +100ms]

31

subjects + design

8 speakers: 2f / 6m, 2nn / 6n introduced to Higgins listen to all 42 (only once); random

order 3 options

Okay, X Did you really mean X? Did you say X?

32

results

no effects for color, subject, duration

significant effects for peak position, peak height, & their interaction

33

results

Statement: early, low peak Question: late, high peak Clear division between “did you mean” and “did you say”

34

food for thought

how about English? red red? red!? how many ways can you say it?

35

conclusion

strong relationship between intonation and meaning

statement: early, low peak question: late, high peak clear division between “did you mean”

and “did you say”

36

the end

error handling – Higgins / Galatea

Documents

Transcript of error handling – Higgins / Galatea