Toshiaki Nakazawa Hideya Mino Isao Goto Graham Neubig Sadao Kurohashi Eiichiro Sumita

Overview  of  WAT2015

nakazawa@pa.jst.jp kuro@i.kyoto-u.ac.jp

Participants  List

WAT2015, Kyoto Japan, 16th October 2015

hideya.mino@nict.go.jp goto.i-es@nhk.or.jp eiichiro.sumita@nict.go.jp

Team  ID Organiza8on ASPEC JPC

JE EJ JC CJ CJ KJ NAIST Nara  Ins+tute  of  Science  and  Technology ✓ ✓ ✓ ✓

Kyoto-­‐U Kyoto  University ✓ ✓ ✓ ✓ ✓

WEBLIO_MT Weblio,  Inc. ✓

TMU Tokyo  Metropolitan  University ✓

BJTUNLP Beijing  Jiaotong  University ✓

Sense Saarland  University  &  Nanyang  Technological  University ✓ ✓ ✓

NICT Na+onal  Ins+tute  of  Informa+on  and  Communica+on  Technology ✓ ✓

TOSHIBA Toshiba  Corpora+on ✓ ✓ ✓ ✓ ✓ ✓

WASUIPS Waseda  University ✓

naver NAVER  Corpora+on ✓ ✓

EHR Ehara  NLP  Research  Laboratory ✓ ✓ ✓ ✓

nQ NTT  Communica+on  Science  Laboratories ✓

outside  Japan company

ASPEC  Chronological  Evaluation

Correlations  between  BLEU  (x-‐‑‒axis)  and  Crowd  (y-‐‑‒axis)  Scores

JPO  Adequacy  Evaluation  Results

Evaluation  Summary

neubig@is.naist.jp

The  difference  of  BLEU  (x-­‐axis)  and  Crowd  (y-­‐axis)  scores  between  WAT2014  and  WAT2015.  We  used  first  3  judgements  to  calculate  the  Crowd  score  of  WAT2015  to  fit  to  WAT2014  criteria.