Goteborg University Dialogue Systems Lab Dialogue and grammar Göteborg University Dialogue Systems...
-
Upload
kiley-devere -
Category
Documents
-
view
218 -
download
1
Transcript of Goteborg University Dialogue Systems Lab Dialogue and grammar Göteborg University Dialogue Systems...
Goteborg University Dialogue Systems Lab
Dialogue and grammar
Göteborg University Dialogue Systems Lab
Language Technology in Sweden2004-05-12
Karin Cavallin, Robin CooperDavid Hjelm, Staffan Larsson
Goteborg University Dialogue Systems Lab
Speech recognition and NL interpretation & generation
• Options for SR:– Statistical language model– Grammar model
• Options for NL interpretation:– Word/phrase spotting– Grammar & parsing
• Options for NL generation– Template-based– Grammar-based
• Can be combined in various ways
Goteborg University Dialogue Systems Lab
Are grammars useful for dialogue systems?
• Most working systems don’t use a grammar!
• Instead: – keyword or phrase-spotting techniques for
interpretation– template-based generation
• Also, most working systems are built more or less from scratch– do not fully separate application-specific
knowledge from general and reusable knowledge
Goteborg University Dialogue Systems Lab
GoDiS
• A dialogue system built using TrindiKit and based on the information state approach
• Implements a theory of Issue-based Dialogue Management
• Some goals with GoDiS:– Provide a domain-independent theory covering several
dialogue genres– Modularity and reusability of system components– Enable rapid prototyping of new applications
• Focus on dialogue management• These goals have been achieved on the level of
dialogue management...– ... but not for NL processing
Goteborg University Dialogue Systems Lab
SR and interpretation in GoDiS
• Started out as text-based system– Phrase spotting for interpretation;
strings to dialogue moves– Templates for generation; dialogue
moves to strings• Added SR component
– Did not have a corpus, so needed to write a speech recognition grammar by hand
Goteborg University Dialogue Systems Lab
tvGoDiS
• GoDiS application for programming a VCR over the telephone– schedule recordings– remove planned recordings– TV card used for recording television
programs as video files• Small corpus collected of users calling the
system– approx. 100 sessions (calls) and 1100 utterances– misrecognised utterances not yet transcribed– GUI enables immediate access to corpus
Goteborg University Dialogue Systems Lab
Example from tvGoDIS corpus
• TrindiGui displaying a tvGoDiS session
• Audio files linked with utterances
• Access to information state at different stages of the dialogue
Goteborg University Dialogue Systems Lab
Problems with GoDiS
• Mismatch between SR module and interpretation module– correctly recognised speech was
misinterpreted– Leads to strange and unexpected
behaviour (unexpected interactions with dialogue management)
Goteborg University Dialogue Systems Lab
ASR/interpretation mismatch
S> Välkommen till videon! För att få hjälp, säg hjälp.
S> Låt oss se. Vad kan jag hjälpa dig med?
U> jag vill spela in ett program (0.6)
S> Vill du lägga till en inspelning?
U> ja (0.98)S> Låt oss se. Menar du från
noll ett eller till noll ett?
• jag vill spela in ett program– Covered by SR grammar– Not covered by phrase spotting
lexicon– However spela in and ett are
recognized by interpretation module
• Correct interpretation: – jag vill spela in ett program
request(add_rec_job)• Actual interpretation:
– spela in request(add_rec_job
– ett
answer(time(1:00))• System assumes ett specifies
either when the recording should stop or end – thus the follow-up question
Goteborg University Dialogue Systems Lab
Problems with GoDiS
• Mismatch between interpretation module and generation module– System said something that led the
user to believe they could say X – but the system does not understand X
Goteborg University Dialogue Systems Lab
Interpretation/generation mismatch
S> Du kan lägga till en inspelning, ta bort en planerad inspelning, och lista planerade inspelningar. För att börja om, säg börja om. För att sluta, säg sluta.
S> huvudmenyn. Vad kan jag hjälpa dig med?
U> lista inspelningar (0.75)S> Ockey. gå till inspelningar.S> Låt oss se. Vill du lägga till
en inspelning , radera dina inspelningar eller få information om dina inspelningar?
• Help message suggests that the user can say lista inspelningar
• But help message is not up-to-date, and neither is SR grammar.
• Instead interpretation module finds word inspelningar
• The user is asked to choose between roughly the same alternatives that was presented in the help message
Goteborg University Dialogue Systems Lab
• Bugs can of course be fixed...– would take lots of time– no guarantee that new bugs won’t be
introduced– no guarantee that all bugs have shown up
• These solutions are not reusable• Instead, we want a general and
principled solution– enable reuse and rapid prototyping of
applications, including grammars
Goteborg University Dialogue Systems Lab
Solution, part 1: single grammar for SR, interpretation, generation
• No mismatches between the grammars• Only one grammar needs to be written for each
application• However:
– the different modules may require different kinds of grammars, in different formats
• So:– We need to be able to generate all 3 grammars from a
single grammar• However, we may not want the exact same coverage for
SR, interpretation and generation grammars – May not want to understand everything that can be
generated
Goteborg University Dialogue Systems Lab
Solution, part 2: use GF!
• GF is a powerful tool for mutilingual grammar development
• Can be used to generate grammars in various other formats
• Subgrammars can be extracted from larger grammars– SR, interpretation and generation subgrammars may
have different coverage • In addition, GF offers resource grammars in
several languages– This promises to further decrease the time needed
for writing a new application grammar
Goteborg University Dialogue Systems Lab
GF GoDiS Grammar
• Application grammars– User grammar– System grammar– Shared grammar, used by user grammar
and system grammar• Utility grammars
– Weekday, Time, Channel• Application grammars may use
– Utility grammars– Resource grammars
Goteborg University Dialogue Systems Lab
GF VCR grammar
• One abstract syntax– GoDiS dialogue moves as GF abstract trees
• Three concrete syntaxes– Swedish– English – GoDiS dialogue moves in prolog notation
• Statistics: – Abstract syntax : ~420 lines of code– Concrete syntax :
• Prolog: ~440 lines of code• Swedish: ~440 lines of code
Goteborg University Dialogue Systems Lab
Parsing Swedish to abstract syntax
> p -cat=DMove -lang=Swe "spela in från kvart över tolv" request (vcr_add_rec_job_args1 (startTimeToStoreReq
(startTimeToStore (time hour12 minute15))))
Goteborg University Dialogue Systems Lab
Linearising abstract syntax to Prolog notation
> l -lang=Prolog request (vcr_add_rec_job_args1 (startTimeToStoreReq (startTimeToStore (time hour12 minute15))))
request(add_rec_job , answer(start_time_to_store(12 : 15))
Goteborg University Dialogue Systems Lab
Translating Prolog to Swedish
> t -cat=DMove Prolog Swe "request(add_rec_job , answer(start_time_to_store(12 : 15))"
spela in från tolv femtonspela in från tolv och femtonspela in från femton över tolvspela in från femton minuter över tolvspela in från kvart över tolv
Goteborg University Dialogue Systems Lab
Time Grammar
• A multilingual grammar module• Swedish, English, Prolog, started on
French and Catalan.
Goteborg University Dialogue Systems Lab
Constructing concrete syntaxes for time expressions isn’t always trivial
• Abstract: time hour12 Minute30• Prolog: 12 : 30• English: half past twelve / half twelve• Swedish: halv ett
• klockan två • klockan två och femton• ?klockan fjorton• two o’clock• *two fifteen o’clock• *fourteen o’clock
Corpus studies needed
Goteborg University Dialogue Systems Lab
Another problem
• am or pm?• på morgonen • på eftermiddagen• på kvällen• på natten
Goteborg University Dialogue Systems Lab
Combining GF and statistical methods for robustness
• Use GF to generate a corpus to use as a baseline for SLM training
• Use GF-generated SR grammars in initial system and collect corpus to be used for training SLMs for speech recognition
• For recognized out-of-grammar string, try to find nearest in-grammar string; confirm this with user (Gorrell 2003)
Goteborg University Dialogue Systems Lab
Conclusion
• Keeping track of separate SR, interpretation and generation grammars in a dialogue system is time-consuming
• GF offers the possibility of rapid prototyping of grammars for use in dialogue system applications– Generate other grammar formats from GF– Resource grammars for several languages
• GF can be combined with statistical methods for robustness