menu2dialog

Staffan Larsson, Robin Cooper, Stina Ericsson

Department of linguisticsGöteborgs Universitet

Background• Current projects

– TRINDI: task-oriented dialogue– SIRIDUS: explore action-oriented dialogue (and more)– D’Homme: Talking to applications in the home

• Many applications and services have menu-based interfaces– mobile phone– video– computer– automatic cinema ticket booking via phone

• But menus have certain disadvantages• Dialogue systems can provide more user-friendly

interfaces

the basic idea of this paper:

• conversion menu 2 dialogue– menu-based interfaces can be automatically or

semi-automatically converted into dialogue plans

• flexible dialogue– dialogue plans are used in a way that enables

more flexible interaction than the original menu-based interface

• modularity, plug&play– we isolate general, domain independent

dialogue knowledge from domain-dependent dialogue plans

Overview

• Menu vs. dialogue• GoDiS: a dialogue system• Menu components• Dialogue plans• Converting menus to dialogue plans• Examples

Menus vs. dialogue:Drawbacks of menus

• User must follow predesigned menu structure

• this structure may be complicated and difficult to learn

• menus allow only one activity at a time

Menu vs. dialogue:Advantages of flexible dialogue

• provide information in any order• provide information without pre-

specifying the task • not necessary to learn menu

structure• doing several things at once

GoDiS: a dialogue system

• Originally developed withing the european TRINDI project

• implemented using the information state approach, using the TrindiKit

• based on a theory about dialogue based on ”Questions Under Discussion” (Ginzburg) and dialogue plans

• presented at the previous workshop at IJCAI-99

TRINDIKIT

dialogue theory(IS, rules, moves etc)

domain knowledge(resources)

domain-specificsystem

Building a system using TrindiKit

domain-independentDME

software engineering(basic types, control flow)

inputinter-pret

Information State

lexicon

control

update selectgene-rate

output

deviceinterface

domainDEVICE

PRIVATE = PLAN =

AGENDA = { findout(?return) }

SHARED =

findout(?x.month(x))findout(?x.class(x))

respond(?x.price(x))

COM = dest(paris) transport(plane) task(get_price_info)

QUD = < x.origin(x) >LM = { ask(sys, x.origin(x)) }

BEL = { }

TMP = (same structure as SHARED)

Sample GoDiS information state(travel agency domain)

• integrateAnswer

• Before an answer can be integrated by the system, it must be matched to a question on QUD

pre:

eff:

in(SHARED.LM, answer(usr, A))fst(SHARED.QUD, Q) relevant_answer(Q, A)

pop(SHARED.QUD) reduce(Q, A, P)

add(SHARED.COM, P)

Sample update rule

Typical human-computer dialogin a travel agency

S: Hello, how can I help you?U: I want price information pleaseS: Where do you want to go?U: ParisS: How do you want to travel?U: A flight pleaseS: When do you want to travelU: AprilS: what class did you have in mind?…S: The price is $123

Typical human-human dialogue

S(alesman), C(ustomer)S: hiC: flights to ParisS: when do you want to travel?C: April, as cheap as possible...

Accommodation

• Lewis (1979): If someone says something at t which requires X to be in the conversational scoreboard, and X is not in the scoreboard at t, then (under certain conditions) X will become part of the scoreboard at t

• A: My cat is hungry– B accommodates “A has a cat”

• Has been applied to referents and propositions, as parts of the conversational scoreboard / information state

Question accommodation

• If questions are part of the information state, they too can be accommodated

• If the latest move was an answer, and there is an action in the plan to ask a matching question, put that question on QUD

• Requires that the number of possible matching questions is not too large (or can be narrowed down by asking clarification question)

Update rule for question accommodation

QuAcc

pre:

eff:

in(SHARED.LM, answer(usr, A)) in(PRIVATE.PLAN, findout(Q)) relevant_answer(Q, A)

push(SHARED.QUD, Q)

Task accommodation

• In some cases, the system may not even know what task the user wants the system to perform

• If latest move was an answer, and there is currently no plan, – find (in the domain resource) a task and

corresponding plan containing a matching question; – accommodate the task and load the appropriate plan

• Similar to plan recognition, but no dynamic plan reconstruction or assumption that the full plan is shared

Update rule for task accommodation

taskAcc

- An answer move matches a task if the plan contains a question matching the answer

- More complex version generates clarification question if the number of plans > 1

pre:

eff:

in(SHARED.LM, Move) domain :: match_task(Move, Task, Plan)

add(SHARED.BEL, task(Task)) set(PRIVATE.PLAN, Plan)

Flexible dialogue with GoDiS in the travel agency domain

• Question accommodation– S: when do you want to travel?– U: in April, as cheap as possible

• Task accommodation– S: hello– U: flights to Paris

Work on GoDiS• dialogue types handled by GoDiS

– information seeking– instructional (system instructs user)

• Currently being extended to handle– action-oriented (user instructs system)– multiple simultaneous plans– collaborative negotiation

• GoDiS has been adapted to several domains, including– travel agency (TRINDI)– mobile phone interface (course project)– interface to intellegent house (D’Homme)– cinema ticket booking (MA thesis)– internet search (MA thesis)– telephone receptionist (SIRIDUS)

Menu components

• multiple-choice list– the user selects one of several prespecified

choices

• dialogue window– user types requested information

• tickbox– user ticks box (or not)

• system message– information from system; user clicks ”OK”

Sample menu from a Nokia 3210 mobile phone

• search phonebook– dialogue window to type in name

• Name: _______

– message showing the number• Kalle 031 12 34 56

– tickbox: whether to call this person• Call?

– if a call is made, a message:• Calling 031 12 34 56

Dialogue plans

• domain-specific• partial specifications of how the system

should act in a dialogue– specifies default behaviour– may be overridden by user initiative

• operational semantics of plan constructs – determined by information state update

rules

Dialogue plan constructs

• action A• exec(S)

– S is a task/subplan

• sequence <C1,C2,..., Cn>

• if P then C– P is a proposition and C a construct– If P is (believed to be) true, execute C

• case(<P1, C1>, ...,<Pn-1, Cn-1>, Cn)

Actions

• findout(Q): find the answer to Q• raise(Q): raise Q• respond(Q): provide answer to Q

if there is one in the private beliefs• inform(P): inform of P• (Q is a question, P a proposition)

Questions

• y/n-question: ?P– ” Do you want to call NN?”

• wh-question: ?x.P(x)– ”Who do you want to search for?”

• alt-question: {?P1, ?P2, ..., ?Pn}– ”Do you want to search, add to, erase

from or change the phonebook?”

Converting menus to plans

Menu Planmulti-choice list L1, L2, ..., Ln

findout({?L1,?L2,...,?Ln});

case(<L1,T1>,...,<Ln,Tn>)where T1is the task corresponding to L1

tick-box +/-P findout(?P)

dialogue windowp=___

findout(?x.p(x))

pop-up message M

inform(M)

Converting menus to plans, cont’d

• in addition, we need– if-then statements to check conditions and

pick out propositions– device actions, e.g.

• “call up number N”• “add name N with number M to phonebook”• …

• these are not currently assumed to be part of the menu specification

the menu again

• search phonebook– dialogue window to type in name

• Name: _______

– message showing the number• Kalle 031 12 34 56

– tickbox: whether to call this person• Call?

– if a call is made, a message:• Calling 031 12 34 56

Corresponding plan

findout(?x.name(x));lookup_database(?x.number(x));if number(N) then

respond(?x.number(x) ),findout( ?call ),

if call and name(M) then inform(calling_name(M))))

Another plan: change ringtone

findout(?x.name(x));findout(?x.ringtone(x));if name(N) and ringtone(R) then

save-ringtone( N, R ),inform(ringtone-changed);

Example 1: standard menu navigation

U:Go to the phonebookS: Phonebook. Do you want to search, add, or

remove an entry, or assign a ringing tone? U: Assign ringing toneS: What name?U: JohnS: Which ringtone?U: BossanovaS: The ringtone for John is now Bossanova.

Example 2: question accommodation

U: Assign ringtoneS: What name?U: Bossanova for John

accommodate ”which ringtone?”

S: The ringing tone for John is now Bossanova.

Example 3: task accommodation

U: Bossanova for Johnaccommodate task: assign ringtoneaccommodate ”which ringtone?”accommodate ”what name?”

S: The ringing tone for John is now Bossanova.

Example 4: task clarification

U: 5 o’clock– matches two tasks; ask for

clarification

S: Do you want to set the clock or the alarm clock?

U: alarm clock– accommodate ”what time?”

S: the alarm clock is now set to 5

Example 5 (video): task and question accommodation

U: record from 19:30 to 20:00 today– accommodate task: add program– accommodate ”what starting time?”– accommodate ”what end time?”– accommodate ”what day?”

S: What channel?U: SevenS: OK, I will record channel 7 from 19:30

to 20:00 today.

Example 6: doing several things at once

U: Search the phonebook for JimS: The number is 031 997321U: Assign ringtone “Attraction”

• name(jim) is already a shared proposition

S: The ringtone for Jim is now Attraction. Do you want to make a call?

U: No

• Note: proposition name(jim) is used by both plans

end of talk

Menus in VoiceXML

• Why is VoiceXML interesting?– becoming industry standard – supports plan constructs similar to GoDiS– multiple active grammars allow behaviour

reminiscent of question and task accommodation

• VoiceXML takes form-filling dialogue management to its limit

GoDiS vs. VoiceXML

• VoiceXML: if input matches several fields, the first is chosen– GoDiS can ask clarification question

• VoiceXML: user initiated subdialogues cannot share information with main dialog, or other subdialogues– the default in GoDiS is that all information is

shared between subdialogues

• unclear how to implement several simultaneously active plans

GoDiS vs. VoiceXML

• information state is global and keeps plan separate from accumulated propositions– VoiceXML based on forms, which can be

seen as local information states• VoiceXML mixes dialogue knowledge, domain

knowledge, and language knowledge in a single specification– GoDiS keeps them separate, enabling easier

reconfiguration and plug-and-play – when implement a new menu-driven system, there is

no need to reimplement general dialogue strategies

GoDiS vs. VoiceXML

• Open question whether VoiceXML can extend to more complex dialogue, e.g. negotiation– but the information state approach is ideal

for complex dialogues– GoDiS is currently being extended to handle

collaborative negotiation

• Can GoDiS be combined with VoiceXML?– have GoDiS use VoiceXML specifications,

with some additions