2018 IEEE Workshop

on

spokEn LanguagE TEchnoLogy

SLT 2018Proceedings

DEcEmbEr 18–21, 2018aThEns, grEEcE

SponSored by

The InSTITuTe of elecTrIcal and elecTronIcS engIneerS

Ieee SIgnal proceSSIng SocIeTy

IEEE Catalog Number: CFP18SLT-ART ISBN: 978-1-5386-4334-1

2018 IEEE Workshop

on

spokEn LanguagE TEchnoLogy

SLT 2018Proceedings

DEcEmbEr 18–21, 2018aThEns, grEEcE

SponSored by

The InSTITuTe of elecTrIcal and elecTronIcS engIneerS

Ieee SIgnal proceSSIng SocIeTy

ii

Copyright and Reprint Permission: Abstracting is permitted with credit to the source. Libraries are permitted to photocopy beyond the limit of U.S. copyright law, for private use of patrons, those articles in this volume that carry a code at the bottom of the first page, provided that the per-copy fee indicated in the code is paid through the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923. Other copying, reprint, or reproduction requests should be addressed to IEEE Copyrights Manager, IEEE Service Center, 445 Hoes Lane, P.O. Box 1331, Piscataway, NJ 08855-1331. All rights reserved. Copyright © 2018 by the Institute of Electrical and Electronics Engineers, Inc.

The papers in this book comprise the proceedings of the meeting mentioned on the cover and title page. They reflect the authors’ opinions and, in the interests of timely dissemination, are published as presented and without change. Their inclusion in this publication does not necessarily constitute endorsement by the editors, the IEEE Signal Processing Society, or the Institute of Electrical and Electronics Engineers, Inc.

IEEE Catalog Number: CFP18SLT-ARTISBN: 978-1-5386-4334-1

Produced by Conference Management Services, Inc.

iii

General ChairsYannis Stylianou

Apple and University of Crete

Co-ChairsVangelis Karkaletsis

NCSR DemokritosSrinivas Bangalore

Interactions LLC

Technical ChairsAlexandros Papangelis

UberSvetlana Stoyanchev

Ineraction LLCKallirroi Georgila

University of Southern CaliforniaKishore Prahallad

AppleSpyros Matsoukas

AmazonTheodore Giannakopoulos

NCSR DemokritosMilica Gasic

University of CambridgeJerome Bellegarda

AppleDilek Hakkani-Tur

AmazonRichard Rose

GoogleAbdelrahman Mohamed

Amazon

Regional Publicity ChairsReinhold Häb-Umbach

Paderborn UniversityRyuichiro Higashinaka

NTT

Finance ChairMasami Akamine

Toshiba

Organizing CommitteeSponsorship Chairs

Sungjin LeeMicrosoft

Richard SproatGoogle

Andrew RosenbergIDM

Publication ChairsDimitrios Dimitriadis

MicrosoftGerasimos Potamianos

University of Thessaly

Special Session and Tutorial ChairsJohn Hansen

The University of Texas at DallasAlexandros Potamianos

National Technical University of Athens

Demo ChairsTheodore Giannakopoulos

NCSR DemokritosHelen Meng

The Chinese University of Hong Kong (CUHK)

Advisory BoardFrank Soong

MicrosoftNajim Dehak

Johns Hopkins UniversityTatsuya Kawahara

Kyoto University

Scientific Committee

Rutuja Ubale, ETSGilles Adda, LIMSI/CNRSNagaraj Adiga, University of CreteMasami Akamine, Toshiba Corporation, JapanTasos Anastasakos, MicrosoftWalter Andrews, Sierra Nevada Corp.Pongtep Angkititrakul, Robert Bosch LLCEbru Arisoy, MEF UniversityRamon Fernandez Astudillo, INESC-IDKartik Audhkhasi, IBM T. J. Watson Research

CenterMichiel Bacchiani, Google Inc.Gerard Bailly, GIPSA-Lab, CNRS/ Grenoble-

Alpes Univ.

iv

Jonas Borgstrom, Massachusetts Institute of Technology Lincoln Laboratory

Hynek Boril, University of Wisconsin-PlattevilleKonstantinos Bougiatiotis, NCSR DemokritosGilles Boulianne, Centre de recherche

informatique de MontrealNiko Brummer, Nuance Communications, South

AfricaTrung H. Bui, Adobe ResearchLukáš Burget, Brno University of TechnologyAsli Celikyilmaz, Microsoft ResearchAggelina Chatziagapi, Behavioral Signal

Technologies Inc.John Chen, InteractionsNancy Chen, Institute for Infocomm ResearchYun-Nung (Vivian) Chen, National Taiwan

UniversityJen-Tzung Chien, National Chiao Tung UniversityWei Chu, Liulishuo, IncAlistair Conkie, AppleAndre Coy, University of the West IndiesHeriberto Cuayahuitl, University of LincolnRicardo de Cordoba, Universidad Politecnica de

MadridGilles Degottex, ObEN, Inc.; University of

CambridgeNajim Dehak, Johns Hopkins UniversityHuiqun Deng, Deng Audio ResearchNina Dethlefs, University of HullGiuseppe Di Fabbrizio, VUI, Inc.Lukas Drude, Paderborn UniversityThomas Drugman, AmazonJacques Duchateau, NuanceDaniel Erro, Cirrus LogicEngin Erzin, Koc UniversityMaxine Eskenazi, Carnegie Mellon UniversityYannick Esteve, University of Le MansKeelan Evanini, Educational Testing ServiceMauro Falcone, Fondazione Ugo BordoniTiago H. Falk, Institut National de la Recherche

ScientifiqueKevin Farrell, Nuance CommunicationsRaul Fernandez, IBM ResearchEric Fosler-Lussier, The Ohio State UniversityTakashi Fukuda, IBM JapanSADAOKI FURUI, Toyota Technological Institute

at ChicagoDaniel Garcia-Romero, Johns Hopkins UniversityMilica Gasic, Cambridge University

Kallirroi Georgila, University of Southern California - Institute for Creative Technologies

George Giannakopoulos, NCSR DemokritosTheodore Giannakopoulos, NCSR DemokritosOndrej Glembek, Brno University of TechnologyChristian Gollan, Apple Inc.Yifan Gong, Microsoft Corp.Evandro Gouvea, Independent researcherFrantisek Grezl, Brno University of TechnologyDavid Griol, Universidad Carlos III de MadridRaghav Gupta, Google AIReinhold Haeb-Umbach, Paderborn UniversityDilek Hakkani-Tur, Amazon AICemal Hanilci, Bursa Technical UniversityMirko Hannemann, Apple Inc.John Hansen, The University of Texas at Dallas;

Jonsson School of EngineeringVille Hautamäki, University of Eastern FinlandTimothy J. Hazen, MicrosoftJahn Heymann, Paderborn UniversityRyuichiro Higashinaka, NTTKeikichi Hirose, University of TokyoTakaaki Hori, Mitsubishi Electric Research

Laboratories (MERL)Chien-Lin Huang, National Cheng Kung

UniversityShajith Ikbal, IBM Research AILorenzo Trueba Jaime, AmazonDavid Janiszek, University of Paris DescartesMichael Johnson, University of KentuckyKristiina Jokinen, AI Research Center AIST Tokyo

WaterfrontDenis Jouvet, InriaFilip Jurcicek, Charles University in PragueOzlem Kalinli, Apple IncHong-Goo Kang, Yonsei UniversityMartin Karafiat, Brno University of TechnologyNassos Katsamanis, Behavioral Signal

Technologies, Inc.Hideki Kawahara, Wakayama UniversityTatsuya Kawahara, Kyoto UniversityHong Kook Kim, Gwangju Institute of Science

and TechnologyBrian Kingsbury, IBM ResearchTomi Kinnunen, University of Eastern FinlandNorihide Kitaoka, Tokushima UniversityTina Kohler, DoDKazunori Komatani, Osaka UniversityMargarita Kotti, Toshiba Research Europe

v

Antonis Koukourikos, National Centre for Scientific Research “Demokritos”

Lin-shan Lee, National Taiwan UniversitySungbok Lee, University of Southern CaliforniaSungjin Lee, Microsoft ResearchFabrice Lefevre, University of AvignonMichael Levit, MicrosoftHaizhou Li, National University of SingaporeJinyu Li, MicrosoftGang Liu, Alibaba GroupYang Liu, LiulishuoKaren Livescu, Toyota Technological Institute at

ChicagoEduardo Lleida Solano, University of ZaragozaMathew Magimai Doss, Idiap Research InstituteEman Mahmoodi, InteractionsBrian Mak, The Hong Kong University of Science

and TechnologyKazunori Mano, Shibaura Institute of TechnologyAlvin Martin, National Institute of Standards and

Technology (retired)David Martinez, Cirrus LogicTakashi Masuko, Preferred Networks, Inc.Ryo Masumura, NTT CorporationTomoko Matsui, Institute of Statistical

MathematicsYuri Matveev, ITMO UniversityErik McDermott, Google, IncMitchell McLaren, SRI InternationalMichael McTear, Ulster UniversityFlorian Metze, Carnegie Mellon UniversityNobuaki Minematsu, The University of TokyoTeruhisa Misu, Honda Research Institute USA,

Inc.Nikola Mrksic, PolyAISeiichi Nakagawa, Chubu UniversitySatoshi Nakamura, Nara Institute of Science and

TechnologyMahesh Kumar Nandwana, SRI InternationalArun Narayanan, Google Inc.Raymond W. M. Ng, Emotech LabsPatrick Nguyen, Google BrainJuan Nolazco, Tecnológico de MonterreyTetsuji Ogawa, Waseda UniversityAlfonso Ortega, University of ZaragozaDouglas O’Shaughnessy, INRS-EMT (University

of Quebec)Mukund Padmanabhan, Renaissance technologies

llc

Georgios Paraskevopoulos, National Technical University of Athens

Sree Hari Krishnan Parthasarathi, AmazonSarangarajan Parthasarathy, MicrosoftVijayaditya Peddinti, Johns Hopkins UniversityThomas Pellegrini, Université Paul Sabatier

Toulouse III ; Institut de Recherche en Informatique de Toulouse

George Petasis, NCSR DemokritosPetko Petkov, Toshiba Research EuropeMichael Picheny, IBMRoberto Pieraccini, Google ZurichNikiforos Pittaras, NCSR DemokritosOldrich Plchot, Brno University of TechnologyGerasimos Potamianos, University of ThessalyYao Qian, Educational Testing ServicesTuomo Raitio, Aalto UniversityBhuvana Ramabhadran, Google, Inc.Daniel Ramos, Universidad Autonoma de MadridSpyros Raptis, INNOETICSAbhinav Rastogi, Google AIAntoine Raux, b4.aiSteve Renals, University of EdinburghFred Richardson, MIT Lincoln LaboratoryLuis J. Rodriguez-Fuentes, University of the

Basque Country UPV/EHURichard Rose, GoogleAndrew Rosenberg, IBMNick Ruiz, InteractionsDavid Rybach, GoogleMd Sahidullah, InriaSakti Sakriani, Nara Institute of Science and

TechnologyMurat Saraclar, Bogazici UniversityRuhi Sarikaya, AmazonShoei Sato, NHK Science and Technology

Research LaboratoryThomas Schaaf, M*Modal, Inc.Carlos Segura, Telefónica ResearchGregory Senay, xBrainKoichi Shinoda, Tokyo Institute of TechnologyTakahiro Shinozaki, Tokyo Institute of TechnologyMarco Siniscalchi, Università di EnnaOlivier Siohan, GoogleManhung Siu, Raytheon BBN TechnologiesMalcolm Slaney, GoogleHagen Soltau, GoogleAlex Sorin, IBMEvgeny Stepanov, VUI, Inc.Andreas Stolcke, Microsoft Research

vi

Svetlana Stoyanchev, AT&T ResearchBrian Strope, Google, Inc.Ming Sun, Amazon.comXie Sun, NuanceTetsuya Takiguchi, Kobe UniversityHao Tang, Massachusetts Institute of TechnologySamuel Thomas, IBM TJ Watson Research CenterTomoki Toda, Nagoya UniversityPedro A. Torres-Carrasquillo, MIT Lincoln

LaboratoryIsabel Trancoso, INESC-ID / IST, University of

LisbonJan Trmal, Johns Hopkins UniversityYu Tsao, Academia SinicaVassilis Tsiaras, University of CreteGokhan Tur, UberOytun Turk, Sensory Inc.Stefan Ultes, Daimler AGDavid van Leeuwen, Radboud University

NijmegenCarlos Vaquero, CIRRUSDimitra Vergyri, SRI InternationalKarel Vesely, Brno University of TechnologyHsin-Min Wang, Academia SinicaNigel Ward, University of Texas at El PasoShinji Watanabe, Johns Hopkins UniversityLei Xie, Northwestern Polytechnical UniversityJunichi Yamagishi, National Institute of

InformaticsYonghong Yan, Chinese Academy of SciencesKaisheng Yao, Ant FinancialKoichiro Yoshino, Nara Institute of Science and

TechnologyChengzhu Yu, University of Texas at DallasDong Yu, Tencent AI LabKai Yu, Shanghai Jiao Tong UniversityIlija Zeljkovic, Interactions CorporationHeiga Zen, GoogleYunxin Zhao, University of MissouriImed Zitouni, MicrosoftKatharina Zmolikova, Brno UniversityTudor-Catalin Zorila, Toshiba Cambridge

Research Laboratory

vii

TABLE OF CONTENTS

ASR I

#1: HIGH-DEGREE FEATURE FOR DEEP NEURAL NETWORK BASED ACOUSTIC ........................1MODELHoon Chung, Sung Joo Lee, Jeon Gue Park, Electronics and Telecommunications Research Institute, Republic of Korea

#2: DENSENET BLSTM FOR ACOUSTIC MODELING IN ROBUST ASR ................................................6Maximilian Strake, Pascal Behr, Timo Lohrenz, Tim Fingscheidt, Technische Universität Braunschweig, Germany

#3: PHASE-BASED FEATURE REPRESENTATIONS FOR IMPROVING RECOGNITION ................13OF DYSARTHRIC SPEECHSiddharth Sehgal, Stuart Cunningham, Phil Green, University of Sheffield, United Kingdom

#4: EFFICIENT BUILDING STRATEGY WITH KNOWLEDGE DISTILLATION FOR .......................21SMALL-FOOTPRINT ACOUSTIC MODELSTakafumi Moriya, Hiroki Kanagawa, Kiyoaki Matsui, Takaaki Fukutomi, Yusuke Shinohara, Yoshikazu Yamaguchi, Manabu Okamoto, Yushi Aono, NTT Corporation, Japan

#5: ADVANCING MULTI-ACCENTED LSTM-CTC SPEECH RECOGNITION USING A ....................29DOMAIN SPECIFIC STUDENT-TEACHER LEARNING PARADIGMShahram Ghorbani, Ahmet E. Bulut, John H.L. Hansen, University of Texas at Dallas, United States

#6: DYNAMIC EXTENSION OF ASR LEXICON USING WIKIPEDIA DATA .........................................36Badr Abdullah, Irina Illina, Dominique Fohr, LORIA/INRIA, France

#7: IMPROVING LF-MMI USING UNCONSTRAINED SUPERVISIONS FOR ASR ..............................43Hossein Hadian, Sharif University of Technology, Iran; Daniel Povey, Johns Hopkins University, United States; Hossein Sameti, Sharif University of Technology, Iran; Jan Trmal, Sanjeev Khudanpur, Johns Hopkins University, United States

#8: ON TRAINING RECURRENT NETWORKS WITH TRUNCATED ....................................................48BACKPROPAGATION THROUGH TIME IN SPEECH RECOGNITIONHao Tang, James Glass, Massachusetts Institute of Technology, United States

#9: LEARNING NOISE-INVARIANT REPRESENTATIONS FOR ROBUST SPEECH ..........................56RECOGNITIONDavis Liang, Zhiheng Huang, Amazon AI, United States; Zachary Lipton, Carnegie Mellon University, United States

#10: AN EXPLORATION OF DIRECTLY USING WORD AS ACOUSTIC MODELING .......................64UNIT FOR SPEECH RECOGNITIONChunlei Zhang, The University of Texas at Dallas, United States; Chengzhu Yu, Chao Weng, Jia Cui, Dong Yu, Tencent AI Lab, United States

#11: IMPROVED TRAINING OF NEURAL TRANS-DIMENSIONAL RANDOM FIELD .....................70LANGUAGE MODELS WITH DYNAMIC NOISE-CONTRASTIVE ESTIMATIONBin Wang, Zhijian Ou, Tsinghua University, China

viii

#12: IMPROVING VERY DEEP TIME-DELAY NEURAL NETWORK WITH ........................................77VERTICAL-ATTENTION FOR EFFECTIVELY TRAINING CTC-BASED ASR SYSTEMSSheng Li, Xugang Lu, Ryoichi Takashima, Peng Shen, National Institute of Information and Communications Technology, Japan; Tatsuya Kawahara, National Institute of Information and Communications Technology (NICT) / Kyoto University, Japan; Hisashi Kawai, National Institute of Information and Communications Technology, Japan

DETECTION, PARALINGUISTICS AND CODING

#1: EXPLORING END-TO-END ATTENTION-BASED NEURAL NETWORKS FOR ...........................84NATIVE LANGUAGE IDENTIFICATIONRutuja Ubale, Yao Qian, Keelan Evanini, Educational Testing Service Research, United States

#2: ANALYSING THE PREDICTIONS OF A CNN-BASED REPLAY SPOOFING ..................................92DETECTION SYSTEMBhusan Chettri, Saumitra Mishra, Queen Mary University of London, United Kingdom; Bob L. Sturm, KTH Royal Institute of Engineering, Sweden; Emmanouil Benetos, Queen Mary University of London, United Kingdom

#3: IMPROVED CONDITIONAL GENERATIVE ADVERSARIAL NET CLASSIFICATION ..............98FOR SPOKEN LANGUAGE RECOGNITIONXiaoxiao Miao, The University of Kent / Key Laboratory of Speech Acoustics and Content Understanding, Institute of Acoustics / University of Chinese Academy of Sciences, China; Ian McLoughlin, The University of Kent, United Kingdom; Shengyu Yao, Key Laboratory of Speech Acoustics and Content Understanding, Institute of Acoustics / University of Chinese Academy of Sciences, China; Yonghong Yan, Key Laboratory of Speech Acoustics and Content Understanding, Institute of Acoustics / University of Chinese Academy of Sciences / Xinjiang Key Laboratory of Minority Speech and Language Information Processing, Xinjiang Technical Institute of Physics and Chemistry, Chinese Academy of Sciences, China

#4: UNSUPERVISED REPRESENTATION LEARNING OF SPEECH FOR DIALECT .......................105IDENTIFICATIONSuwon Shon, Wei-Ning Hsu, James Glass, Massachusetts Institute of Technology, United States

#5: MULTIMODAL SPEECH EMOTION RECOGNITION USING AUDIO AND TEXT ......................112Seunghyun Yoon, Seokhyun Byun, Kyomin Jung, Seoul National University, Republic of Korea

#6: POSTERIOR CALIBRATION FOR MULTI-CLASS PARALINGUISTIC ........................................119CLASSIFICATIONGábor Gosztolya, MTA-SZTE Research Group on Artificial Intelligence, Hungary; Róbert Busa-Fekete, Yahoo Research Inc., United States

#7: CONTEXT-AWARE ATTENTION MECHANISM FOR SPEECH EMOTION .................................126RECOGNITIONGaetan Ramet, Ecole Polytechnique Federale de Lausanne, Switzerland; Philip N. Garner, Idiap Research Institute, Switzerland; Michael Baeriswyl, Alexandros Lazaridis, Swisscom, Switzerland

#8: AN EXPERIMENTAL STUDY ON AUDIO REPLAY ATTACK DETECTION USING ...................132DEEP NEURAL NETWORKSBekir Bakar, Cemal Hanilci, Bursa Technical University, Turkey

ix

#9: LSTM-BASED WHISPER DETECTION ................................................................................................139Zeynab Raeesy, Kellen Gillespie, Chengyuan Ma, Thomas Drugman, Jiacheng Gu, Roland Maas, Ariya Rastrow, Björn Hoffmeister, Amazon, United States

#10: AMERICAN SIGN LANGUAGE FINGERSPELLING RECOGNITION IN THE WILD ..............145Bowen Shi, Toyota Technological Institute at Chicago, United States; Aurora Martinez Del Rio, Jonathan Keane, University of Chicago, United States; Jonathan Michaux, Toyota Technological Institute at Chicago, United States; Diane Brentari, University of Chicago, United States; Greg Shakhnarovich, Karen Livescu, Toyota Technological Institute at Chicago, United States

#11: WAVENET-BASED ZERO-DELAY LOSSLESS SPEECH CODING .................................................153Takenori Yoshimura, Kei Hashimoto, Keiichiro Oura, Yoshihiko Nankaku, Keiichi Tokuda, Nagoya Institute of Technology, Japan

#12: IMPROVING GENERALIZATION OF VOCAL TRACT FEATURE ..............................................159RECONSTRUCTION: FROM AUGMENTED ACOUSTIC INVERSION TO ARTICULATORY FEATURE RECONSTRUCTION WITHOUT ARTICULATORY DATARosanna Turrisi, Raffaele Tavarone, Leonardo Badino, Istituto Italiano di Tecnologia, Italy

#13: A DEEP LEARNING APPROACH FOR DATA DRIVEN VOCAL TRACT AREA .........................167FUNCTION ESTIMATIONSasan Asadiabadi, Engin Erzin, Koc university, Turkey

ASR II

#1: OCCAM’S ADAPTATION: A COMPARISON OF INTERPOLATION OF BASES ..........................174ADAPTATION METHODS FOR MULTI-DIALECT ACOUSTIC MODELING WITH LSTMSMikaela Grace, Meysam Bastani, Eugene Weinstein, Google, United States

#2: SPEECH CHAIN FOR SEMI-SUPERVISED LEARNING OF JAPANESE-ENGLISH ...................182CODE-SWITCHING ASR AND TTSSahoko Nakayama, Andros Tjandra, Sakriani Sakti, Satoshi Nakamura, Nara Institute of Science and Technology, Japan

#3: IMPROVING ASR ERROR DETECTION WITH RNNLM ADAPTATION ......................................190Rahhal Errattahi, University of Chouaib Doukkali, Morocco; Salil Deena, The University of Sheffield, United Kingdom; Asmaa El Hannani, Hassan Ouahmane, University of Chouaib Doukkali, Morocco; Thomas Hain, The University of Sheffield, United Kingdom

#4: MULTI-OBJECTIVE MULTI-TASK LEARNING ON RNNLM FOR SPEECH ..............................197RECOGNITIONMinguang Song, Yunxin Zhao, University of Missouri, United States; Shaojun Wang, Ping An Technology, China

#5: CONFIDENCE ESTIMATION AND DELETION PREDICTION USING .........................................204BIDIRECTIONAL RECURRENT NEURAL NETWORKSAnton Ragni, Qiujia Li, Mark Gales, Yu Wang, University of Cambridge, United Kingdom

#6: IMPROVING OOV DETECTION AND RESOLUTION WITH EXTERNAL ...................................212LANGUAGE MODELS IN ACOUSTIC-TO-WORD ASRHirofumi Inaguma, Masato Mimura, Shinsuke Sakai, Tatsuya Kawahara, Kyoto University, Japan

x

#7: LSTM LANGUAGE MODEL ADAPTATION WITH IMAGES AND TITLES FOR ........................219MULTIMEDIA AUTOMATIC SPEECH RECOGNITIONYasufumi Moriya, Gareth Jones, Dublin City University, Ireland

#8: FIRST-PASS TECHNIQUES FOR VERY LARGE VOCABULARY SPEECH .................................227RECOGNITION OF MORPHOLOGICALLY RICH LANGUAGESMatti Varjokallio, Sami Virpioja, Mikko Kurimo, Aalto University, Finland

#9: ANALYZING DEEP CNN-BASED UTTERANCE EMBEDDINGS FOR ACOUSTIC .....................235MODEL ADAPTATIONJoanna Rownicka, Peter Bell, Steve Renals, The University of Edinburgh, United Kingdom

#10: MULTI-BAND PROCESSING WITH GABOR FILTERS AND TIME DELAY NEURAL ............242NETS FOR NOISE ROBUST SPEECH RECOGNITIONGyörgy Kovács, MTA-SZTE Research Group on Artificial Intelligence, Hungary; László Tóth, University of Szeged, Hungary; Gábor Gosztolya, MTA-SZTE Research Group on Artificial Intelligence, Hungary

#11: A TEACHER-STUDENT LEARNING APPROACH FOR UNSUPERVISED DOMAIN ................250ADAPTATION OF SEQUENCE-TRAINED ASR MODELSVimal Manohar, Pegah Ghahremani, Daniel Povey, Sanjeev Khudanpur, Johns Hopkins University, United States

#12: DOMAIN ROBUST FEATURE EXTRACTION FOR RAPID LOW RESOURCE ASR ................258DEVELOPMENTSiddharth Dalmia, Xinjian Li, Florian Metze, Alan W Black, Carnegie Mellon University, United States

VOICE CONVERSION AND TTS

#1: STARGAN-VC: NON-PARALLEL MANY-TO-MANY VOICE CONVERSION USING .................266STAR GENERATIVE ADVERSARIAL NETWORKSHirokazu Kameoka, Takuhiro Kaneko, Kou Tanaka, Nobukatsu Hojo, NTT Corporation, Japan

#2: RHYTHM-FLEXIBLE VOICE CONVERSION WITHOUT PARALLEL DATA USING ................274CYCLE-GAN OVER PHONEME POSTERIORGRAM SEQUENCESCheng-chieh Yeh, Po-chun Hsu, Ju-chieh Chou, Hung-yi Lee, Lin-shan Lee, National Taiwan University, Taiwan

#3: ADAPTIVE WAVENET VOCODER FOR RESIDUAL COMPENSATION IN ..................................282GAN-BASED VOICE CONVERSIONBerrak Sisman, Mingyang Zhang, National University of Singapore, Singapore; Sakriani Sakti, Nara Institute of Science and Technology, Japan; Haizhou Li, National University of Singapore, Singapore; Satoshi Nakamura, Nara Institute of Science and Technology, Japan

#4: NEURAL TTS VOICE CONVERSION ....................................................................................................290Zvi Kons, Slava Shechtman, Alex Sorin, Ron Hoory, Carmel Rabinovitz, Edmilson Da Silva Morais, IBM Research, Israel

#5: AN EVALUATION OF DEEP SPECTRAL MAPPINGS AND WAVENET VOCODER ....................297FOR VOICE CONVERSIONPatrick Lumban Tobing, Tomoki Hayashi, Yi-Chiao Wu, Kazuhiro Kobayashi, Tomoki Toda, Nagoya University, Japan

xi

#6: IMPROVING FFTNET VOCODER WITH NOISE SHAPING AND SUBBAND .............................304APPROACHESTakuma Okamoto, National Institute of Information and Communications Technology, Japan; Tomoki Toda, Nagoya University, Japan; Yoshinori Shiga, Hisashi Kawai, National Institute of Information and Communications Technology, Japan

#7: COMPARING PROSODIC FRAMEWORKS: INVESTIGATING THE ............................................312ACOUSTIC-SYMBOLIC RELATIONSHIP IN TOBI AND RAPRaul Fernandez, Andrew Rosenberg, IBM Research, United States

#8: DATA SELECTION FOR IMPROVING NATURALNESS OF TTS VOICES TRAINED ................319ON SMALL FOUND CORPUSESFang-Yu Kuo, Sandesh Aryal, Gilles Degottex, Sam Kang, Pierre Lanchantin, Iris Ouyang, ObEN, Inc., United States

#9: COMPREHENSIVE EVALUATION OF STATISTICAL SPEECH WAVEFORM ............................325SYNTHESISThomas Merritt, Bartosz Putrycz, Adam Nadolski, Tianjun Ye, Daniel Korzekwa, Wiktor Dolecki, Thomas Drugman, Viacheslav Klimkov, Alexis Moinet, Andrew Breen, Rafal Kuklinski, Nikko Strom, Roberto Barra-Chicote, Amazon, United Kingdom

#10: EXAMPLAR-BASED SPEECH WAVEFORM GENERATION FOR TEXT-TO-SPEECH ............332Cassia Valentini-Botinhao, Oliver Watts, Felipe Espic, Simon King, University of Edinburgh, United Kingdom

#11: AN ICELANDIC PRONUNCIATION DICTIONARY FOR TTS ........................................................339Anna Björk Nikulásdóttir, Jón Guðnason, Reykjavik University, Iceland; Eiríkur Rögnvaldsson, University of Iceland, Iceland

#12: MOS NATURALNESS AND THE QUEST FOR HUMAN-LIKE SPEECH ......................................346Sajad Shirali-Shahreza, Gerald Penn, University of Toronto, Canada

ASR III (END-TO-END)

#1: IMPROVING ATTENTION-BASED END-TO-END ASR SYSTEMS WITH ....................................353SEQUENCE-BASED LOSS FUNCTIONSJia Cui, Chao Weng, Guangsen Wang, Jun Wang, Tencent AI Lab, United States; Peidong Wang, The Ohio State University, United States; Chengzhu Yu, Dan Su, Dong Yu, Tencent AI Lab, United States

#2: COMBINING END-TO-END AND ADVERSARIAL TRAINING FOR LOW-RESOURCE ...........361SPEECH RECOGNITIONJennifer Drexler, James Glass, Massachusetts Institute of Technology, United States

#3: A COMPARISON OF TECHNIQUES FOR LANGUAGE MODEL INTEGRATION IN .................369ENCODER-DECODER SPEECH RECOGNITIONShubham Toshniwal, Toyota Technological Institute at Chicago, United States; Anjuli Kannan, Chung-Cheng Chiu, Yonghui Wu, Tara N. Sainath, Google, United States; Karen Livescu, Toyota Technological Institute at Chicago, United States

#4: ON-DEVICE END-TO-END SPEECH RECOGNITION WITH MULTI-STEP ................................376PARALLEL RNNSYoonho Boo, Jinhwan Park, Lukas Lee, Wonyong Sung, Seoul National University, Republic of Korea

xii

#5: DOMAIN ADAPTATION OF END-TO-END SPEECH RECOGNITION IN .....................................382LOW-RESOURCE SETTINGSLahiru Samarakoon, Fano Labs, China; Brian Mak, Hong Kong University of Science and Technology, China; Albert Lam, Fano Labs, China

#6: END-TO-END SPEECH RECOGNITION WITH WORD-BASED RNN LANGUAGE ...................389MODELSTakaaki Hori, Mitsubishi Electric Research Laboratories, United States; Jaejin Cho, Shinji Watanabe, Johns Hopkins University, United States

#7: ACOUSTIC-TO-WORD RECOGNITION WITH SEQUENCE-TO-SEQUENCE ............................397MODELSShruti Palaskar, Florian Metze, Carnegie Mellon University, United States

#8: COMBINING DE-NOISING AUTO-ENCODER AND RECURRENT NEURAL ..............................405NETWORKS IN END-TO-END AUTOMATIC SPEECH RECOGNITION FOR NOISE ROBUSTNESSTzu-Hsuan Ting, Chia-Ping Chen, National Sun Yat-sen University, Taiwan

#9: IMPROVED KNOWLEDGE DISTILLATION FROM BI-DIRECTIONAL TO ...............................411UNI-DIRECTIONAL LSTM CTC FOR END-TO-END SPEECH RECOGNITIONGakuto Kurata, Kartik Audhkhasi, IBM Research, Japan

#10: DEEP CONTEXT: END-TO-END CONTEXTUAL SPEECH RECOGNITION ..............................418Golan Pundak, Tara N. Sainath, Rohit Prabhavalkar, Anjuli Kannan, Ding Zhao, Google, United States

#11: BACK-TRANSLATION-STYLE DATA AUGMENTATION FOR END-TO-END ASR ...................426Tomoki Hayashi, Nagoya University, Japan; Shinji Watanabe, Johns Hopkins University, United States; Yu Zhang, Google, United States; Tomoki Toda, Nagoya University, Japan; Takaaki Hori, Mitsubishi Electric Research Laboratories, United States; Ramon Astudillo, INESC-ID-Lisboa, Portugal; Kazuya Takeda, Nagoya University, Japan

#12: DIALOG-CONTEXT AWARE END-TO-END SPEECH RECOGNITION .......................................434Suyoun Kim, Florian Metze, Carnegie Mellon University, United States

ASR IV

#1: TOWARD DOMAIN-INVARIANT SPEECH RECOGNITION VIA LARGE SCALE .....................441TRAININGArun Narayanan, Ananya Misra, Khe Chai Sim, Golan Pundak, Anshuman Tripathi, Mohamed Elfeky, Parisa Haghani, Trevor Strohman, Michiel Bacchiani, Google, United States

#2: TRANSLITERATION BASED APPROACHES TO IMPROVE CODE-SWITCHED ......................448SPEECH RECOGNITION PERFORMANCEJesse Emond, Bhuvana Ramabhadran, Brian Roark, Pedro Moreno, Min Ma, Google, United States

#3: EXPLORING LAYER TRAJECTORY LSTM WITH DEPTH PROCESSING UNITS ....................456AND ATTENTIONJinyu Li, Liang Lu, Changliang Liu, Yifan Gong, Microsoft, United States

xiii

#4: MULTICHANNEL ASR WITH KNOWLEDGE DISTILLATION AND GENERALIZED ..............463CROSS CORRELATION FEATUREWenjie Li, Yu Zhang, Pengyuan Zhang, Fengpei Ge, Key Laboratory of Speech Acoustics and Content Understanding, Institute of Acoustics, China

#5: OPTIMIZING THE QUALITY OF SYNTHETICALLY GENERATED ............................................470PSEUDOWORDS FOR THE TASK OF MINIMAL-PAIR DISTINCTIONHeiko Holz, Maria Chinkina, University of Tübingen, Germany; Laura Vetter, Ludwig Maximilian University of Munich, Germany

#6: LEVERAGING SEQUENCE-TO-SEQUENCE SPEECH SYNTHESIS FOR ....................................477ENHANCING ACOUSTIC-TO-WORD SPEECH RECOGNITIONMasato Mimura, Sei Ueno, Hirofumi Inaguma, Shinsuke Sakai, Tatsuya Kawahara, Kyoto University, Japan

#7: HIERARCHICAL MULTITASK LEARNING WITH CTC ..................................................................485Ramon Sanabria, Florian Metze, Carnegie Mellon University, United States

#8: A K-NEAREST NEIGHBOURS APPROACH TO UNSUPERVISED SPOKEN TERM ...................491DISCOVERYAlexis Thual, Corentin Dancette, Julien Karadayi, Juan Benjumea, Emmanuel Dupoux, ENS, France

#9: A NEW TIMIT BENCHMARK FOR CONTEXT-INDEPENDENT PHONE .....................................498RECOGNITION USING TURBO FUSIONTimo Lohrenz, Wei Li, Tim Fingscheidt, TU Braunschweig, Germany

#10: EFFICIENT IMPLEMENTATION OF RECURRENT NEURAL NETWORK ...............................506TRANSDUCER IN TENSORFLOWTom Bagby, Kanishka Rao, Khe Chai Sim, Google, United States

#11: AUDIO-VISUAL SPEECH RECOGNITION WITH A HYBRID CTC/ATTENTION .....................513ARCHITECTUREStavros Petridis, Imperial College London, United Kingdom; Themos Stafylakis, University of Nottingham, United Kingdom; Pingchuan Ma, Imperial College London, United Kingdom; Georgios Tzimiropoulos, University of Nottingham, United Kingdom; Maja Pantic, Imperial College London, United Kingdom

#12: MULTILINGUAL SEQUENCE-TO-SEQUENCE SPEECH RECOGNITION: ...............................521ARCHITECTURE, TRANSFER LEARNING, AND LANGUAGE MODELINGJaejin Cho, Johns Hopkins University, United States; Murali Karthick Baskar, Brno university of technology, Czech Republic; Ruizhi Li, Matthew Wiesner, Johns Hopkins University, United States; Sri Harish Mallidi, Amazon, United States; Nelson Yalta, Waseda University, Japan; Martin Karafiat, Brno university of technology, Czech Republic; Shinji Watanabe, Johns Hopkins University, United States; Takaaki Hori, Mitsubishi Electric Research Laboratories, United States

#13: SPEAKER SELECTIVE BEAMFORMER WITH KEYWORD MASK ESTIMATION .................528Yusuke Kida, Dung Tran, Motoi Omachi, Toru Taniguchi, Yuya Fujita, Yahoo Japan Corporation, Japan

#14: SPEAKER ADAPTED BEAMFORMING FOR MULTI-CHANNEL AUTOMATIC ......................535SPEECH RECOGNITIONTobias Menne, Ralf Schlüter, Hermann Ney, RWTH Aachen University, Germany

#15: SPEAKER ADAPTATION FOR END-TO-END CTC MODELS ........................................................542Ke Li, Johns Hopkins University, United States; Jinyu Li, Yong Zhao, Kshitiz Kumar, Yifan Gong, Microsoft AI and Research, United States

xiv

#16: AN EXPLORATION OF MIMIC ARCHITECTURES FOR RESIDUAL NETWORK ..................550BASED SPECTRAL MAPPINGPeter Plantinga, Deblin Bagchi, Eric Fosler-Lussier, The Ohio State University, United States

#17: MULTI-CHANNEL MULTI-SPEAKER OVERLAPPED SPEECH RECOGNITION ....................558WITH LOCATION GUIDED SPEECH EXTRACTION NETWORKZhuo Chen, Xiong Xiao, Takuya Yoshioka, Jinyu Li, Hakan Erdogan, Yifan Gong, Microsoft Cloud & AI, United States

#18: A STUDY ON SPEECH ENHANCEMENT USING EXPONENT-ONLY FLOATING ...................566POINT QUANTIZED NEURAL NETWORK (EOFP-QNN)Yi-Te Hsu, Academia Sinica, Taiwan; Yu-Chen Lin, Szu-Wei Fu, National Taiwan University, Taiwan; Yu Tsao, Academia Sinica, Taiwan; Tei-Wei Kuo, National Taiwan University, Taiwan

#19: RAPID SPEAKER ADAPTATION OF NEURAL NETWORK BASED FILTERBANK .................574LAYER FOR AUTOMATIC SPEECH RECOGNITIONHiroshi Seki, Toyohashi University of Technology, Japan; Kazumasa Yamamoto, Chubu University, Japan; Tomoyosi Akiba, Toyohashi University of Technology, Japan; Seiichi Nakagawa, Chubu University, Japan

#20: FAR-FIELD ASR USING LOW-RANK AND SPARSE SOFT TARGETS FROM ...........................581PARALLEL DATAPranay Dighe, Idiap Research Institute, EPFL, Switzerland; Afsaneh Asaei, Idiap Research Institute, Switzerland; Herve Bourlard, Idiap Research Institute, EPFL, Switzerland

#21: DEEP VIEW2VIEW MAPPING FOR VIEW-INVARIANT LIPREADING ......................................588Alexandros Koumparoulis, National Technical University of Athens, Greece; Gerasimos Potamianos, University of Thessaly, Greece

DEEP LEARNING FOR SPEECH SYNTHESIS

#1: PREDICTING EXPRESSIVE SPEAKING STYLE FROM TEXT IN END-TO-END ......................595SPEECH SYNTHESISDaisy Stanton, Yuxuan Wang, RJ Ryan, Google, United States

#2: A SPECTRALLY WEIGHTED MIXTURE OF LEAST SQUARE ERROR AND .............................603WASSERSTEIN DISCRIMINATOR LOSS FOR GENERATIVE SPSSGilles Degottex, ObEN, Inc. - University of Cambridge, United Kingdom; Mark Gales, University of Cambridge, United Kingdom

#3: SCALING AND BIAS CODES FOR MODELING SPEAKER-ADAPTIVE DNN-BASED ..............610SPEECH SYNTHESIS SYSTEMSHieu-Thi Luong, Junichi Yamagishi, National Institute of Informatics, Japan

#4: HIERARCHICAL RNNS FOR WAVEFORM-LEVEL SPEECH SYNTHESIS ..................................618Qingyun Dou, Moquan Wan, Gilles Degottex, Zhiyi Ma, Mark Gales, University of Cambridge, United Kingdom

#5: PARAMETER GENERATION ALGORITHMS FOR TEXT-TO-SPEECH SYNTHESIS ...............626WITH RECURRENT NEURAL NETWORKSViacheslav Klimkov, Alexis Moinet, Adam Nadolski, Thomas Drugman, Amazon, Poland

#6: SYNTHETIC-TO-NATURAL SPEECH WAVEFORM CONVERSION USING ................................632CYCLE-CONSISTENT ADVERSARIAL NETWORKSKou Tanaka, Takuhiro Kaneko, Nobukatsu Hojo, Hirokazu Kameoka, NTT corporation, Japan

xv

#7: IMPROVING UNSUPERVISED STYLE TRANSFER IN END-TO-END SPEECH .........................640SYNTHESIS WITH END-TO-END SPEECH RECOGNITIONDa-Rong Liu, Chi-Yu Yang, Szu-Lin Wu, Hung-Yi Lee, National Taiwan University, Taiwan

#8: MULTI-SCALE ALIGNMENT AND CONTEXTUAL HISTORY FOR ATTENTION .....................648MECHANISM IN SEQUENCE-TO-SEQUENCE MODELAndros Tjandra, Sakriani Sakti, Satoshi Nakamura, Nara Institute of Science and Technology, Japan

SPOKEN LANGUAGE UNDERSTANDING

#1: LOW-RESOURCE CONTEXTUAL TOPIC IDENTIFICATION ON SPEECH .................................656Chunxi Liu, Matthew Wiesner, Shinji Watanabe, Craig Harman, Jan Trmal, Najim Dehak, Sanjeev Khudanpur, Johns Hopkins University, United States

#2: RANKING APPROACH TO COMPACT TEXT REPRESENTATION FOR PERSONAL ..............664DIGITAL ASSISTANTSIssac Alphonso, Nick Kibre, Tasos Anastasakos, Microsoft, United States

#3: A RE-RANKER SCHEME FOR INTEGRATING LARGE SCALE NLU MODELS .........................670Chengwei Su, Rahul Gupta, Shankar Ananthakrishnan, Spyros Matsoukas, Amazon, United States

#4: PARSING COORDINATION FOR SPOKEN LANGUAGE UNDERSTANDING .............................677Sanchit Agarwal, Rahul Goel, Tagyoung Chung, Abhishek Sethi, Arindam Mandal, Spyros Matsoukas, Amazon, United States

#5: QUATERNION CONVOLUTIONAL NEURAL NETWORKS FOR THEME ...................................685IDENTIFICATION OF TELEPHONE CONVERSATIONSTitouan Parcollet, Mohamed Morchid, Georges Linarès, Université d’Avignon et des pays du Vaucluse, France; Renato De Mori, McGill University, Canada

#6: END-TO-END NAMED ENTITY AND SEMANTIC CONCEPT EXTRACTION FROM ...............692SPEECHSahar Ghannay, Antoine Caubrière, Yannick Estève, Nathalie Camelin, Edwin Simonnet, Antoine Laurent, University of Le Mans, France; Emmanuel Morin, University of Nantes, France

#7: TOWARD MULTI-FEATURES EMPHASIS SPEECH TRANSLATION: ASSESSMENT ..............700OF HUMAN EMPHASIS PRODUCTION AND PERCEPTION WITH SPEECH AND TEXT CLUESQuoc Truong Do, Nara Institute of Science and Technology, Japan; Sakriani Sakti, Satoshi Nakamura, Nara Institute of Science and Technology/AIP, Japan

#8: CORPUS AND ANNOTATION TOWARDS NLU FOR CUSTOMER ORDERING ..........................707DIALOGSJohn Chen, Rashmi Prasad, Svetlana Stoyanchev, Ethan Selfridge, Srinivas Bangalore, Michael Johnston, Interactions, LLC., United States

#9: COUPLED REPRESENTATION LEARNING FOR DOMAINS, INTENTS AND SLOTS ..............714IN SPOKEN LANGUAGE UNDERSTANDINGJihwan Lee, Dongchan Kim, Ruhi Sarikaya, Young-Bum Kim, Amazon, United States

xvi

#10: FROM AUDIO TO SEMANTICS: APPROACHES TO END-TO-END SPOKEN ..........................720LANGUAGE UNDERSTANDINGParisa Haghani, Arun Narayanan, Michiel Bacchiani, Galen Chuang, Neeraj Gaur, Pedro Moreno, Rohit Prabhavalkar, Zhongdi Qu, Austin Waters, Google, United States

#11: INVESTIGATING THE DOWNSTREAM IMPACT OF GRAPHEME-BASED .............................727ACOUSTIC MODELING ON SPOKEN UTTERANCE CLASSIFICATIONRyan Price, Bhargav Srinivas Ch, Surbhi Singhal, Srinivas Bangalore, Interactions, LLC., United States

NATURAL LANGUAGE PROCESSING

#1: ABSTRACTIVE DIALOGUE SUMMARIZATION WITH SENTENCE-GATED ............................735MODELING OPTIMIZED BY DIALOGUE ACTSChih-Wen Goo, Yun-Nung Chen, National Taiwan University, Taiwan

#2: GRAPH-BASED DEEP-TREE RECURSIVE NEURAL NETWORK (DTRNN) FOR .....................743TEXT CLASSIFICATIONFenxiao Chen, Bin Wang, C.-C. Jay Kuo, University of Southern California, United States

#3: EXTENSION OF CONVENTIONAL CO-TRAINING LEARNING STRATEGIES TO ..................750THREE-VIEW AND COMMITTEE-BASED LEARNING STRATEGIES FOR EFFECTIVE AUTOMATIC SENTENCE SEGMENTATIONDogan Dalva, Umit Guz, F.M.V. ISIK University, Turkey; Hakan Gurkan, Bursa Technical University, Turkey

#4: INFORMATION-WEIGHTED NEURAL CACHE LANGUAGE MODELS FOR ASR ....................756Lyan Verwimp, KU Leuven, Belgium; Joris Pelemans, Apple, United States; Hugo Van hamme, Patrick Wambacq, KU Leuven, Belgium

#5: WORD SEGMENTATION FROM PHONEME SEQUENCES BASED ON ......................................763PITMAN-YOR SEMI-MARKOV MODEL EXPLOITING SUBWORD INFORMATIONRyu Takeda, Kazunori Komatani, Osaka University, Japan; Alexander Rudnicky, Carnegie Mellon University, United States

#6: INTELLIGENCE IS ASKING THE RIGHT QUESTION: A STUDY ON JAPANESE .....................771QUESTION GENERATIONLasguido Nio, Koji Murakami, Rakuten Institute of Technology, Japan

#7: INVESTIGATING LINGUISTIC PATTERN ORDERING IN HIERARCHICAL ............................779NATURAL LANGUAGE GENERATIONShang-Yu Su, Yun-Nung Chen, National Taiwan University, Taiwan

#8: GUESS WHO? MULTILINGUAL APPROACH FOR THE AUTOMATED GENERATION ..........787OF AUTHOR-STYLIZED POETRYAlexey Tikhonov, Yandex, Germany; Ivan Yamshchikov, Max Planck Institute for Mathematics in the Sciences, Germany

#9: GENERATING SEMANTIC SIMILARITY ATLAS FOR NATURAL LANGUAGES ......................795Lütfi Kerem Şenel, İhsan Utlu, Veysel Yücesoy, Aykut Koç, ASELSAN, Turkey; Tolga Çukur, Bilkent University, Turkey

#10: RESOLVING REFERRING EXPRESSIONS IN IMAGES WITH LABELED ................................800ELEMENTSNevan Wichers, Dilek Hakkani-Tür, Jindong (JD) Chen, Google AI, United States

xvii

#11: SENTIMENT CLASSIFICATION ON ERRONEOUS ASR TRANSCRIPTS: A MULTI ...............807VIEW LEARNING APPROACHSri Harsha Dumpala, Imran Sheikh, Rupayan Chakraborty, Sunil Kumar Kopparapu, TCS Research and Innovation-Mumbai, India

DIALOGUE

#1: DIRECT OPTIMIZATION OF F-MEASURE FOR RETRIEVAL-BASED PERSONAL .................815QUESTION ANSWERINGRasool Fakoor, Amanjit Kainth, Siamak Shakeri, Christopher Winestock, Abdel-rahman Mohamed, Ruhi Sarikaya, Amazon, United States

#2: EFFICIENT DIALOG POLICY LEARNING VIA POSITIVE MEMORY RETENTION ................823Rui Zhao, Volker Tresp, Siemens & LMU, Germany

#3: TURN-TAKING PREDICTIONS ACROSS LANGUAGES AND GENRES USING AN ..................831LSTM RECURRENT NEURAL NETWORKNigel Ward, Diego Aguirre, Gerardo Cervantes, Olac Fuentes, University of Texas at El Paso, United States

#4: PREDICTION OF DIALOGUE SUCCESS WITH SPECTRAL AND RHYTHM .............................838ACOUSTIC FEATURES USING DNNS AND SVMSAthanasios Lykartsis, Technische Universität Berlin, Germany; Margarita Kotti, Alexandros Papangelis, Yannis Stylianou, Toshiba LTD, United Kingdom

#5: CONTEXT-AWARE DIALOG RE-RANKING FOR TASK-ORIENTED DIALOG ..........................846SYSTEMSJunki Ohmura, Sony Corporation, United States; Maxine Eskenazi, Language Technologies Institute Carnegie Mellon University, United States

#6: OUT-OF-DOMAIN SLOT VALUE DETECTION FOR SPOKEN DIALOGUE ................................854SYSTEMS WITH CONTEXT INFORMATIONYuka Kobayashi, Takami Yoshida, Kenji Iwata, Hiroshi Fujimura, Masami Akamine, Toshiba Corporation, Japan

#7: ACCUMULATING CONVERSATIONAL SKILLS USING CONTINUAL LEARNING ..................862Sungjin Lee, Microsoft Research, United States

#8: LEARNING GOAL-ORIENTED VISUAL DIALOG VIA TEMPERED POLICY ............................868GRADIENTRui Zhao, Volker Tresp, Siemens & LMU, Germany

#9: OPTIMIZING NEURAL RESPONSE GENERATOR WITH EMOTIONAL IMPACT ...................876INFORMATIONNurul Lubis, Sakriani Sakti, Koichiro Yoshino, Satoshi Nakamura, Nara Institute of Science and Technology, Japan

#10: CONVOLUTIONAL NEURAL NETWORKS FOR DIALOGUE STATE TRACKING .................884WITHOUT PRE-TRAINED WORD VECTORS OR SEMANTIC DICTIONARIESMandy Korpusik, James Glass, Massachusetts Institute of Technology, United States

#11: CONTEXTUAL TOPIC MODELING FOR DIALOG SYSTEMS ......................................................892Chandra Khatri, Rahul Goel, Behnam Hedayatnia, Angeliki Metanillou, Anushree Venkatesh, Raefer Gabriel, Arindam Mandal, Amazon Alexa, United States

xviii

#12: USER MODELING FOR TASK ORIENTED DIALOGUES ...............................................................900Izzeddin Gur, University of California Santa Barbara, United States; Dilek Hakkani-Tür, Google, United States; Gokhan Tür, Uber AI Labs, United States; Pararth Shah, Facebook, United States

#13: SCALABLE LANGUAGE MODEL ADAPTATION FOR SPOKEN DIALOGUE SYSTEMS .......907Ankur Gandhe, Ariya Rastrow, Björn Hoffmeister, Amazon, United States

CORPORA AND EVALUATION METHODOLOGIES

#1: DISCOURSE MODELING OF NON-NATIVE SPONTANEOUS SPEECH USING .........................913THE RHETORICAL STRUCTURE THEORY FRAMEWORKXinhao Wang, Binod Gyawali, James V. Bruno, Hillary R. Molloy, Keelan Evanini, Klaus Zechner, Educational Testing Service, United States

#2: TOWARDS FLUENT TRANSLATIONS FROM DISFLUENT SPEECH ............................................921Elizabeth Salesky, Susanne Burger, Carnegie Mellon University, United States; Jan Niehues, Karlsruhe Institute of Technology, Germany; Alex Waibel, Carnegie Mellon University, United States

#3: JSPEECH: A MULTI-LINGUAL CONVERSATIONAL SPEECH CORPUS .....................................927Ali Janalizadeh Choobbasti, Mohammad Erfan Gholamian, Amirkabir University of Technology, Iran; Amir Vaheb, Miras Technologies International, Iran; Saeid Safavi, University of Surrey, Iran

#4: INVESTIGATION OF USERS’ SHORT RESPONSES IN ACTUAL CONVERSATION .................934SYSTEM AND AUTOMATIC RECOGNITION OF THEIR INTENTIONSKatsuya Yokoyama, Hiroaki Takatsu, Waseda University, Japan; Hiroshi Honda, Honda R&D Co.,Ltd, Japan; Shinya Fujie, Chiba Institute of Technology, Japan; Tetsunori Kobayashi, Waseda University, Japan

#5: PHONETIC-AND-SEMANTIC EMBEDDING OF SPOKEN WORDS WITH ..................................941APPLICATIONS IN SPOKEN CONTENT RETRIEVALYi-Chen Chen, Sung-Feng Huang, Chia-Hao Shen, Hung-yi Lee, Lin-shan Lee, National Taiwan University, Taiwan

#6: ODSQA: OPEN-DOMAIN SPOKEN QUESTION ANSWERING DATASET ....................................949Chia-Hsuan Lee, Shang-Ming Wang, Huan-Cheng Chang, Hung-Yi Lee, National Taiwan University, Taiwan

#7: IMPROVED AUTO-MARKING CONFIDENCE FOR SPOKEN LANGUAGE ...............................957ASSESSMENTMarco Del Vecchio, Andrey Malinin, Mark Gales, University of Cambridge, United Kingdom

#8: EVALUATING ON-DEVICE ASR ON FIELD RECORDINGS FROM AN .......................................964INTERACTIVE READING COMPANIONAnastassia Loukina, Nitin Madnani, Beata Beigman Klebanov, Abhinav Misra, Educational Testing Service, United States; Georgi Angelov, Astea Solutions, Bulgaria; Ognjen Todic, Keen Research, United States

#9: DNN-BASED SCORING OF LANGUAGE LEARNERS’ PROFICIENCY USING ..........................971LEARNERS’ SHADOWINGS AND NATIVE LISTENERS’ RESPONSIVE SHADOWINGSSuguru Kabashima, Yuusuke Inoue, Daisuke Saito, Nobuaki Minematsu, The University of Tokyo, Japan

#10: A PROMPT-AWARE NEURAL NETWORK APPROACH TO CONTENT-BASED .......................979SCORING OF NON-NATIVE SPONTANEOUS SPEECHYao Qian, Rutuja Ubale, Matthew Mulholland, Keelan Evanini, Xinhao Wang, Educational Testing Service, United States

xix

#11: QUERYING DEPRESSION VLOGS ......................................................................................................987Joana Correia, Carnegie Mellon University / INESC-ID, Portugal; Isabel Trancoso, INESC-ID / IST, Portugal; Bhiksha Raj, Carnegie Mellon University, Portugal

#12: SEQUENCE TEACHER-STUDENT TRAINING OF ACOUSTIC MODELS FOR ........................994AUTOMATIC FREE SPEAKING LANGUAGE ASSESSMENTYu Wang, Jeremy Wong, Mark Gales, Kate Knill, Anton Ragni, Univerisity of Cambridge, United Kingdom

SPEAKER RECOGNITION/VERIFICATION

#1: TEXT-INDEPENDENT SPEAKER VERIFICATION BASED ON DEEP NEURAL ......................1001NETWORKS AND SEGMENTAL DYNAMIC TIME WARPINGMohamed Adel, Mohamed Afify, Microsoft Advanced Technology Lab, Cairo, Egypt; Akram Gaballah, Microsoft Corporation, United States; Magda Fayek, Cairo University, Egypt

#2: FRAME-LEVEL SPEAKER EMBEDDINGS FOR TEXT-INDEPENDENT SPEAKER ...............1007RECOGNITION AND ANALYSIS OF END-TO-END MODELSuwon Shon, Hao Tang, James Glass, Massachusetts Institute of Technology, United States

#3: GENERATIVE X-VECTORS FOR TEXT-INDEPENDENT SPEAKER VERIFICATION ............1014Longting Xu, Rohan Kumar Das, Emre Yilmaz, Jichen Yang, Haizhou Li, National University of Singapore, Singapore

#4: SPEAKER RECOGNITION FROM RAW WAVEFORM WITH SINCNET .....................................1021Mirco Ravanelli, Yoshua Bengio, Université de Montréal, Canada

#5: INVESTIGATING DEEP NEURAL NETWORKS FOR SPEAKER DIARIZATION IN ...............1029THE DIHARD CHALLENGEIvan Himawan, Md Hafizur Rahman, Sridha Sridharan, Clinton Fookes, Ahilan Kanagasundaram, Queensland University of Technology, Australia

#6: ROLE ANNOTATED SPEECH RECOGNITION FOR CONVERSATIONAL ................................1036INTERACTIONSNikolaos Flemotomos, Zhuohao Chen, University of Southern California, United States; David Atkins, University of Washington, United States; Shrikanth Narayanan, University of Southern California, United States

#7: TEACHER-STUDENT TRAINING FOR TEXT-INDEPENDENT SPEAKER ................................1044RECOGNITIONRaymond W. M. Ng, Xuechen Liu, Emotech Labs, United Kingdom; Pawel Swietojanski, The University of New South Wales, Australia

#8: ATTENTION MECHANISM IN SPEAKER RECOGNITION: WHAT DOES IT LEARN ............1052IN DEEP SPEAKER EMBEDDING?Qiongqiong Wang, Koji Okabe, Kong Aik Lee, Hitoshi Yamamoto, Takafumi Koshinaka, NEC Corporation, Japan

#9: DETECTION AND CALIBRATION OF WHISPER FOR SPEAKER RECOGNITION ................1060Finnian Kelly, John H.L. Hansen, The University of Texas at Dallas, United States

#10: TRAINING SPEAKER RECOGNITION MODELS WITH RECORDING-LEVEL ....................1066LABELSTanel Alumäe, Tallinn University of Technology, Estonia

xx

#11: SHORT UTTERANCE SPEAKER RECOGNITION BY RESERVOIR WITH .............................1073SELF-ORGANIZED MAPPINGNarumitsu Ikeda, The University of Tokyo, Japan; Yoshinao Sato, Fairy Devices Inc., Japan; Hirokazu Takahashi, The University of Tokyo, Japan