UNIVERSITI TEKNOLOGI MARA

SPEECH USER INTERFACES FORAN APPLICATION OF ROUTE NAVIGATION

SYSTEM

HAZRUL SHAZREEN ADD YAZID

Thesis submitted in fulfillment of the requirementsfor the degree of

Master of Science

Faculty of Information Technology and Quantitative Science

JULY 2005

TABLE OF CONTENTS

ACKNOWLEDGEMENTS

TABLE OF CONTENTS

LIST OF TABLES

LIST OF FIGURES

LIST OF ABBREVIATIONS

ABSTRACT

Page No.

ii

iii

ix

xii

xiii

CHAPTERl

1.1

1.2

1.3

1.4

1.5

1.6

1.7

1.8

1.9

CHAPTER 2

2.1

2.2

2.3

2.4

INTRODUCTION

Overview of the Research

Problem Statement

Significance of Study

Research Objectives

Research Scope

Research Approach

Research Questions

Limitations of the Research

Outline of the Thesis

LITERATURE REVIEW

Introduction

Overview of System Technology

Speech Technology

Automatic Speech Recognition Technology (ASR)

2.4.1 Rejection Errors

2.4.2 Natural Language Processing

2.4.3 Dialogue Management

2.4.4 Conversational Context Manager

2.4 .5 Language Generation

iii

10

11

12

12

12

13

13

2.4.6 Speech Generation 13

2.4.7 Computer Telephony Integration 14

2.5 Phone-based Automated Speech Recognition

System 14

2.5.1 Types of Phone-based Speech Application 15

2.5.2 Current Applications 18

2.5.3 Conversation 22

2.5.4 Speech User Interfaces 23

2.6 Issues on Speech User Interfaces 24

2.6.1 Design Constraints 25

2.6.2 Speech Recognition Performance 26

2.6.3 Limitations of Speech Recognition 27

2.6.4 Pause and Tum Transitions 30

2.6.5 Information Loss 32

2.6.6 Related Work of the Design of Speech

User Interfaces (SUI) 35

2.7 Navigational and Wayfinding Knowledge 37

2.8 Navigational Awareness 38

2.8.1 Procedural Route Knowledge 39

2.8.2 Declarative Survey Knowledge 39

2.9 Wayfinding 43

2.10 Spatial Ability 44

2.10.1 Spatial Cognition and Navigation 44

2.11 Biological Navigational Principles 46

2.11.1 Route Navigation and Honeybees 46

2.12 Applying Cognitive Principles to the Route

Navigational Model 50

2.13 Honeybees Metaphors 50

2.14 Spatial Representation with Aspects Maps 52

2.15 Approaches to Human Navigational Behavior 55

2.16 Spatial Ability vs. Navigational Awareness 59

iv

ABSTRACT

Speech Technology has become increasingly important both in personal use and in

industrial. The technology has been improving and an Automatic Speech

Recognition (ASR) has been used widely in recent years. The purpose of this study

is to develop a knowledge base for route navigation system and to construct the

dialogs for supporting route navigation process in speech application. The study

emphasizes the construction of speech dialogs in Malay Language that cover the lack

of capabilities of ASR System in general and tested the ' live system' by speech with

several users. A prototype of a Phone-based Speech Assisted System is developed

from which a knowledge base of rules was constructed using Rapid Application

Developer (RAD Application) as a toolkit for development of Speech Application.

The simulation was done to test the effectiveness of the dialogs design and the

application itself. Assumption has been made, as Malay Speech Recognition already

exists. The data was collected through an observation of Shah Alam area, while

questionnaires were distributed to investigate user behavior in route findings process.

Prior to the development of navigational and wayfinding behavior among Malaysian

community, it is crucial to study the navigational and wayfinding knowledge and the

spatial representation of wayfinding behavior among them.

xiii

CHAPTER 1

INTRODUCTION

1.1 Overview of the Research

The future growth of the computer industry and the acceptance of computer methods

will depend largely on the successful establishment of effective human-machine

communications . Increasingly in the next decade, human will become the prime

focus of system design. The computer is there to serve, to obtain information and to

help them to do their job (Dix et al., 1997).

At the tum of the millennium, there has been significant progress in the development

of ASR System (Automatic Speech Recognition). As a result of technical advances

in speech modeling techniques, recognition search strategies, and other areas,

combined with the increased processing power of workstations and personal

computers, large vocabulary continuous speech recognition is now feasible even

under the constraints and demanding conditions imposed by the Public Switch

Telephone Network (PSTN). These new technical capabilities, along with advances

in Natural Language Processing (NLP), have opened up the possibility of a wide

range of new services and applications, and have made it possible to incorporate

more natural styles of human-computer verbal interactions (Zue & Cole, 2000).

Speech Technology has become increasingly important as it is used to improve the

existing user interfaces and to support new means of human interaction with

computers. It allows hands-free use of computers and supports access to computing

capabilities away from the desk and over the telephone. Furthermore, it is being used

by many enterprises to handle customer calls and internal requests for access to

information, resources and services.

SPEECH USER INTERFACES FOR AN APPLICATION OF ROUTE NAVIGATION SYSTEMACKNOWLEDGEMENTSTABLE OF CONTENTSLIST OF TABLESTable 2.1 Speech Recognition Errors and Possible CausesTable 3.1 Descriptions of LandmarksTable 3.2 Types of Junctions and the LeafTable 3.3 Types of RoundaboutTable 3.4 The Landmarks its DescriptionTable 3.5 Checkpoint along the RoutesTable 3.6 Route Information in Route 1Table 3.7 Route information in Route 2Table 3.8 Route Information in Route 3Table 3.9 Route Information in Route 4Table 4.1 Description of Users

LIST OF FIGURESFigure 1.1 Research ApproachFigure 2.1 Wayfinding KnowledgeFigure 2.2 Navigational KnowledgeFigure 2.3 A Honeybee's Recorded Snapshot and ViewFigure 3.1 Speech User Interfaces DesignFigure 3.2 The Architectural Design of SUI ApplicationFigure 3.3 The Knowledge for Route Navigation ProcessFigure 3.4 The Knowledge of the RoutesFigure 3.5 Shah Alam Area in its RegionFigure 3.6 Route Selected to PKNS BuildingFigure 3.7 Type of JunctionsFigure 3.8 Clockwise Systems in Roundabout SituationFigure 3.9 Checkpoint ConsiderationsFigure 3.10 Walkthrough in Route 1Figure 3.11 Checkpoints from FTMSK to PKNS Building via Tasik Shah AlamFigure 3.12 Walkthrough in Route 2Figure 3.13 Checkpoints via SIRIM BuildingFigure 3.14 Walkthrough in Route 3Figure 3.15 Error Occurred to Intekma ResortsFigure 3.16 Walkthrough in Route 4Figure 3.17 Error Occurred to Hotel ITMFigure 3.18 Rule-based System ArchitectureFigure 4.1 The Dialogs DescriptionFigure 4.2 The Types of DialogFigure 4.3 SUI Flow ChartFigure 4.4 Sequences DiagramFigure 4.5 Generation of Natural DialogsFigure 4.6 RAD Application OverviewFigure 4.7 The Objects in CanvasFigure 4.8 Main Menu Design for Speech Application DesignFigure 4.9 Voices Recording via Generic ObjectsFigure 4.10 Voices Recording Where the Waveform is ObtainedFigure 4.11 Repair DialogFigure 4.12 Subdialog Object for Asking Landmark at Left SideFigure 4.13 Subdialog Object for Asking Landmark at Right SideFigure 4.14 Giving Information About RoutesFigure 4.15 Recognition Preference

LIST OF ABBREVIATIONSABSTRACTCHAPTER 1 INTRODUCTION1.1 Overview of the Research1.2 Problem Statement1.3 Significance of Study1.4 Research Objectives1.5 Research Scope1.6 Research Approach1.7 Research Questions1.8 Limitations of the Research1.9 Outline of the Thesis

CHAPTER 2 LITERATURE REVIEW2.1 Introduction2.2 Overview of System Technology2.3 Speech Technology2.4 Automatic Speech Recognition Technology (ASR)2.4.1 Rejection Errors2.4.2 Natural Language Processing2.4.3 Dialogue Management2.4.4 Conversational Context Manager2.4.5 Language Generation2.4.6 Speech Generation2.4.7 Computer Telephony Integration

2.5 Phone-based Automated Speech Recognition System2.5.1 Types of Phone-based Speech Application2.5.2 Current Applications2.5.3 Conversation2.5.4 Speech User Interfaces

2.6 Issues on Speech User Interfaces2.6.1 Design Constraints2.6.2 Speech Recognition Performance2.6.3 Limitations of Speech Recognition2.6.4 Pause and Turn Transitions2.6.5 Information Loss2.6.6 Related Work of the Design of Speech User Interfaces (SUI)

2.7 Navigational and Wayfinding Knowledge2.8 Navigational Awareness2.8.1 Procedural Route Knowledge2.8.2 Declarative Survey Knowledge

2.9 Wayfinding2.1 Spatial Ability2.10.1 Spatial Cognition and Navigation

2.11 Biological Navigational Principles2.11.1 Route Navigation and Honeybees

2.12 Applying Cognitive Principles to the Route Navigational Model2.13 Honeybees Metaphors2.14 Spatial Representation with Aspects Maps2.15 Approaches to Human Navigational Behavior2.16 Spatial Ability vs. Navigational Awareness2.17 User Interfaces for Route Navigation System2.18 Characteristics of Speech in Route Guiding2.18.1 Speech and Guiding2.18.2 Pros2.18.3 Cons

2.19 Conclusion

CHAPTER 3 PHONE-BASED SPEECH ASSISTED NAVIGATION SYSTEM3.1 Introduction3.2 Speech User Interfaces3.2.1 Knowledge for Route Navigation Process

3.3 Data Collection3.3.1 Observation on Geographical Location3.3.2 Navigation Behavior

3.4 Analysis of Findings3.5 Observation Results3.5.1 Proposed Routes3.5.2 Error Occurrences3.5.3 Existence of Junction and Roundabout3.5.4 Landmarks Selection3.5.5 Duration and Distances

3.6 Questionnaires Results3.6.1 Route Finding Behavior3.6.2 Driving Styles3.6.3 Selecting Landmark3.6.4 Receiving Instruction

3.7 Knowledge Construction3.8 Navigation Knowledge3.9 Knowledge of Route 13.1 Knowledge of Route 23.11 Knowledge of Route 33.12 Knowledge of Route 43.13 Knowledge Base Construction3.14 Representation of the Knowledge3.15 Knowledge Representation in Routes Selected3.16 Rules Construction3.17 Conclusion

CHAPTER 4 DIALOG CONSTRUCTION4.1 Introduction4.2 Speech User Interfaces for an Application of Route Navigation4.3 Usage Scenario4.4 The Design of Dialogs4.5 Ambiguity in Language Understanding4.6 Generation of Natural Language Dialogs4.7 The Organization of Knowledge4.8 Problem Solving Method4.8.1 Deciding How to Say It

4.9 Dialogs Design for Phone-based Speech Assisted Navigation System4.9.1 Capturing Keywords4.9.2 Restricted Dialogs

4.1 The Structures of the Dialogs4.10.1 Greeting4.10.2 Asking User Purpose4.10.3 Declaration Phrase4.10.4 Apology Phrase4.10.5 Asking position4.10.6 Asking Landmarks4.10.7 Follow-up Phrase4.10.8 Giving instruction4.10.9 Asking Condition4.10.10 Silences4. 10.11 System Ends

4.11 Vocabulary4.12 The Conversation4.12.1 Users Response 'No'4.12.2 Users Response 'Yes'4.12.3 Instruction4.12.4 Silences4.12.5 Do Not Recognize4.12.6 Ending Conversation

4.13 Prototype Development4.14 Rapid Application Developer (RAD)4.15 Designing the Prototype4.16 Physical Design4.17 Simulation4.18 Results of Simulation4.19 Conclusion

CHAPTER 5 CONCLUSION5.1 Summary of Findings5.2 Recommendations for Future Works

REFERENCESAPPENDICESAPPENDIX 1 : Journey Times and DistancesAPPENDIX 2: List of QuestionsAPPENDIX 3: Questionnaires Form in MalayAPPENDIX 4: Questionnaires Form in EnglishAPPENDIX 5: Questionnaires Results and AnalysisAPPENDIX 6: IF-THEN Rule-based ApproachAPPENDIX 7: Vocabulary of Malay Words and its PronunciationAPPENDIX 8: Malay Language Translation to English LanguageAPPENDIX 9: Simulation TestsAPPENDIX 10: Analysis of Simulation