Overview’of’WAT2015Overview’of’WAT2015 [email protected] [email protected]...

1
Toshiaki Nakazawa Hideya Mino Isao Goto Graham Neubig Sadao Kurohashi Eiichiro Sumita Overview of WAT2015 [email protected] [email protected] Participants List WAT2015, Kyoto Japan, 16 th October 2015 [email protected] [email protected] [email protected] Team ID Organiza8on ASPEC JPC JE EJ JC CJ CJ KJ NAIST Nara Ins+tute of Science and Technology KyotoU Kyoto University WEBLIO_MT Weblio, Inc. TMU Tokyo Metropolitan University BJTUNLP Beijing Jiaotong University Sense Saarland University & Nanyang Technological University NICT Na+onal Ins+tute of Informa+on and Communica+on Technology TOSHIBA Toshiba Corpora+on WASUIPS Waseda University naver NAVER Corpora+on EHR Ehara NLP Research Laboratory nQ NTT Communica+on Science Laboratories outside Japan company ASPEC Chronological Evaluation Correlations between BLEU (xaxis) and Crowd (yaxis) Scores JPO Adequacy Evaluation Results Evaluation Summary [email protected] The difference of BLEU (xaxis) and Crowd (yaxis) scores between WAT2014 and WAT2015. We used first 3 judgements to calculate the Crowd score of WAT2015 to fit to WAT2014 criteria.

Transcript of Overview’of’WAT2015Overview’of’WAT2015 [email protected] [email protected]...

Page 1: Overview’of’WAT2015Overview’of’WAT2015 nakazawa@pa.jst.jp kuro@i.kyoto-u.ac.jp ParticipantsList WAT2015, Kyoto Japan, 16th October 2015 hideya.mino@nict.go.jp goto.i-es@nhk.or.jp

Toshiaki Nakazawa Hideya Mino Isao Goto Graham Neubig Sadao Kurohashi Eiichiro Sumita

Overview  of  WAT2015

[email protected] [email protected]

Participants  List

WAT2015, Kyoto Japan, 16th October 2015

[email protected] [email protected] [email protected]

Team  ID Organiza8on ASPEC JPC

JE EJ JC CJ CJ KJ NAIST Nara  Ins+tute  of  Science  and  Technology ✓ ✓ ✓ ✓

Kyoto-­‐U Kyoto  University ✓ ✓ ✓ ✓ ✓

WEBLIO_MT Weblio,  Inc. ✓

TMU Tokyo  Metropolitan  University ✓

BJTUNLP Beijing  Jiaotong  University ✓

Sense Saarland  University  &  Nanyang  Technological  University ✓ ✓ ✓

NICT Na+onal  Ins+tute  of  Informa+on  and  Communica+on  Technology ✓ ✓

TOSHIBA Toshiba  Corpora+on ✓ ✓ ✓ ✓ ✓ ✓

WASUIPS Waseda  University ✓

naver NAVER  Corpora+on ✓ ✓

EHR Ehara  NLP  Research  Laboratory ✓ ✓ ✓ ✓

nQ NTT  Communica+on  Science  Laboratories ✓

outside  Japan company

ASPEC  Chronological  Evaluation

Correlations  between  BLEU  (x-‐‑‒axis)  and  Crowd  (y-‐‑‒axis)  Scores

JPO  Adequacy  Evaluation  Results

Evaluation  Summary

[email protected]

The  difference  of  BLEU  (x-­‐axis)  and  Crowd  (y-­‐axis)  scores  between  WAT2014  and  WAT2015.  We  used  first  3  judgements  to  calculate  the  Crowd  score  of  WAT2015  to  fit  to  WAT2014  criteria.