Media Manager Mail Access

Barbara Hohlt and Steve Czerwinski

UC BerkeleyEricsson Presentation 2000

Project Overview

• Make messages more accessible– Enable faster browsing of many voicemails– Access from different devices with different

capabilities

• Media Mail services– Universal Inbox infrastructure– Voicemail is email encoded in MIME

• Transcoding services– Enhance voicemail interaction– Includes: skimmed audio, transcript,

text/audio summary, and outline

Related Work

• SpeechSkimmer, MIT’s MultiMedia Lab [Aaron95]

• Auto-Summarization, Microsoft Research

• CueVideo, IBM• Other Universal Inboxes• IBM’s universal inbox type project

Architecture

Transcoder Service•Voicemail->Text Transcript

•Voicemail->Text Summary

•Voicemail->Text Outline

•Email ->Plain Audio

•Voicemail->Audio Summary

•Voicemail->Skimmed Audio

Mail Access Interface

NinjaMail

Mail Access Interface

POP

Mail Access Interface

IMAP

Media Manager Interface

Media Manager Service

Client

Client

Client

Folder Store

Bhaskar’s Cell-Phone

Barbara’s Desktop

Preference Registry

Automatic Path Creation Service

800-MEDIA-MGR UID: mediamgr@cs.berkeley.edu

Naming Service

11

mediamgr: Cluster locn.

22

MediaManager Mail Access Service

33

Bhaskar’s PSTN Phone Universal Inbox

Clients

Messages and Content Objects

MessageMedia Reference IDQueue of Content Objects

Content Object Content IDData

Media Reference IDMessage PartContent Type

MediaManagerServiceIF

• getFolders()• getFoldersAs()• getList()• getListAs()• getMessage()• getMessageContent()

Audio Tools• Speech Recognition/Synthesis

– Transcribe voicemail to text– IBM ViaVoice SDK and custom audio libs

• Natural Language Processing– Directed word spotting by “understanding”

content– ViaVoice SRCL

• Pitch – Detecting important words by emphasized pitch

• Pause– Compression through pause removal

• Spurts– Retrieve sentence structure of voicemail

Results

• Pause detection– Worked well for given applications– Playback speedup by 50-70%

• Pitch detection– Problems due to high pitch sounds and

transitions

• Speech recognition– Performance decrease in conversational

settings

• Natural Language Processing– Performed well with small grammar

Example: Adding GSM

• Adding GSMAudio and GSMSummary• Add ContentObject definition to

MediaManager– Add method createObject()– Add cases to Media Manager

• Add method toGSM() to Transcoder• Edit .config file

– External.transcoder.gsm rungsm

• Edit related transcoders– speechSynthesizer and audioSummary()

Implementing Other Mail Stores• Implement MailAccessIF

– MMgetFolders()– MMgetFoldersAs()– MMgetList()– MMgetListAs()– MMgetMessage()– MMgetMessageContent()

• Add new protocol to FolderStore

Conclusion