Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of...
Transcript of Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of...
![Page 1: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/1.jpg)
Reminders
HW10 ON NEURAL MACHINE TRANSLATION OR MILESTONE 2 IS DUE ON WEDNESDAY.
QUIZ ON CHAPTER 18 AND 20 (IE AND SRL) IS DUE TONIGHT AT MIDNIGHT.
![Page 2: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/2.jpg)
Review: Machine Translation
![Page 3: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/3.jpg)
Machine TranslationTranslation from one language to another
I'm giving a talk at University of Pennsylvania
ペンシルベニア大学で講演をしています。
![Page 4: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/4.jpg)
Review: Encoder-Decoder MT
Sutskever et al. 2014
LSTM LSTM LSTM LSTM LSTM
</s>
LSTM LSTM LSTM LSTM
argmax argmax argmax argmax
</s>argmax
I hate this movie
kono eiga ga kirai
I hate this movie
Encoder
Decoder
![Page 5: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/5.jpg)
Review: Encoder-Decoder MTMT is the task of automatically translating sentences from one language into another.
We use bilingual parallel texts to train MT systems – pairs of source-target sentences that are translations of each other.
To extend LMs and autoregressive generation to MT, we will:
1. Add an end-of-sentence marker to each source sentence. Concatenate the target sentence to it.
2. Train an RNN LM based on this combined data.
3. To translate, simply treat the input sentence as a prefix, create a hidden state representation for it (encoding step).
4. Use the hidden state produced by the encoder to then start generating (decoding step)
![Page 6: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/6.jpg)
Evaluating MT Quality
![Page 7: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/7.jpg)
Evaluating MT Quality
• Want to rank systems• Want to evaluate incremental
changes• What to make scientific claims
Why do we want to do it?
• “Back translation”• The vodka is not good
How not to do it
![Page 8: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/8.jpg)
Human Evaluation of MT v. Automatic Evaluation
• Ultimately what we're interested in, but
• Very time consuming• Not re-usable
Human evaluation is
• Cheap and reusable, but• Not necessarily reliable
Automatic evaluation is
![Page 9: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/9.jpg)
Manual Evaluation
![Page 10: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/10.jpg)
Goals for Automatic Evaluation
No cost evaluation for incremental changesAbility to rank systemsAbility to identify which sentences we're doing poorly on, and categorize errorsCorrelation with human judgmentsInterpretability of the score
![Page 11: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/11.jpg)
Methodology
Comparison against reference translations
Intuition: closer we get to human translations, the better we're doing
Could use WER like in speech recognition?
![Page 12: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/12.jpg)
Word Error Rate
Levenshtein Distance (also known as "edit distance")
Minimum number of insertions, substitutions, and deletions needed to transform one string into another
Useful measure in speech recognition
• This shows how easy it is to recognize speech
• This shows how easy it is to wreck a nice beach
![Page 13: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/13.jpg)
Problems with WER
Unlike speech recognition we don't have the
assumption of exact match against the
reference or linearity
In MT there can be many possible (and equally
valid) ways of translating a sentence, and phrases
can be rearranged.
![Page 14: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/14.jpg)
/
Solutions
Compare against lots of test sentences
1Use multiple reference translations for each test sentence
2Look for phrase / n-gram matches, allow movement
3
![Page 15: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/15.jpg)
BLEU
BiLingual Evaluation Understudy
Uses multiple reference translations
Look for n-grams that occur anywhere in the sentence
![Page 16: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/16.jpg)
Multiple references
Ref 1 Orejuela appeared calm as he was led to the American plane which will take him to Miami, Florida.
Ref 2 Orejuela appeared calm while being escorted to the plane that would take him to Miami, Florida.
Ref 3Orejuela appeared calm as he was being led to the American plane that was to carry him to Miami in Florida.
Ref 4Orejuela seemed quite calm as he was being led to the American plane that would take him to Miami in Florida.
![Page 17: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/17.jpg)
n-gram precisionB L E U M O D I F I E S T H I S P R E C I S I O N T O E L I M I N AT E R E P E T I T I O N S T H AT O C C U R A C R O S S S E N T E N C E S .
![Page 18: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/18.jpg)
Multiple references
Ref 1 Orejuela appeared calm as he was led to the American plane which will take him to Miami, Florida.
Ref 2 Orejuela appeared calm while being escorted to the plane that would take him to Miami, Florida.
Ref 3Orejuela appeared calm as he was being led to the American plane that was to carry him to Miami in Florida.
Ref 4Orejuela seemed quite calm as he was being led to the American plane that would take him in Florida. to Miami
“to Miami” can only be counted as correct once
![Page 19: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/19.jpg)
Ref 1 Orejuela appeared calm as he was led to the American plane which will take him to Miami, Florida.
Ref 2 Orejuela appeared calm while being escorted to the plane that would take him to Miami, Florida.
Ref 3 Orejuela appeared calm as he was being led to the American plane that was to carry him to Miami in Florida.
Ref 4 Orejuela seemed quite calm as he was being led to the American plane that would take him to Miami in Florida.
Hyp appeared calm when he was taken to the American plane, which will to Miami, Florida.
![Page 20: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/20.jpg)
American, Florida, Miami, Orejuela, appeared, as, being, calm, carry, escorted, he, him, in, led, plane, quite, seemed, take, that, the, to, to, to, was , was, which, while, will, would, ,, .
Hyp appeared calm when he was taken to the Americanplane , which will to Miami , Florida .
1-gram precision = 15/18
![Page 21: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/21.jpg)
American plane, Florida ., Miami ,, Miami in, Orejuela appeared, Orejuela seemed, appeared calm, as he, being escorted, being led, calm as, calm while, carry him, escorted to, he was, him to, in Florida, led to, plane that, plane which, quite calm, seemed quite, take him, that was, that would, the American, the plane, to Miami, to carry, to the, was being, was led, was to, which will, while being, will take, would take, , Florida
Hyp appeared calm when he was taken to the American plane , which will to Miami , Florida .
2-gram precision = 10/17
![Page 22: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/22.jpg)
2-gram precision = 10/17 = .59 1-gram precision = 15/18 = .83
4-gram precision = 3/15 = .203-gram precision = 5/16 = .31
Hyp appeared calm when he was taken to the American plane, which will to Miami, Florida.
(0.83 * 0.59 * 0.31 * 0.2)^(1/4) = 0.417or equivalently
exp(ln .83 + ln .59 + ln .31 + ln .2/4) = 0.417
• Geometric average
n-gram precision
![Page 23: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/23.jpg)
Ref 1 Orejuela appeared calm as he was led to the American plane which will take him to Miami, Florida.
Ref 2 Orejuela appeared calm while being escorted to the plane that would take him to Miami, Florida.
Ref 3 Orejuela appeared calm as he was being led to the American plane that was to carry him to Miami in Florida.
Ref 4 Orejuela seemed quite calm as he was being led to the American plane that would take him to Miami in Florida.
Hyp to the American plane
![Page 24: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/24.jpg)
2-gram precision = 3/3 = 1.0 1-gram precision = 4/4 = 1.0
4-gram precision = 1/1 = 1.03-gram precision = 2/2 = 1.0
Hyp to the American plane
exp(ln 1 + ln 1 + ln 1 + ln 1) = 1
Is this better?
![Page 25: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/25.jpg)
Brevity Penalty
c is the length of the corpus of hypothesis translationsr is the effective reference corpus lengthThe effective reference corpus length is the sum of the single reference translation from each set that is closest to the hypothesis translation.
![Page 26: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/26.jpg)
0.00
0.25
0.50
0.75
1.00
1.25
-75 -38 0 38 75 113
BP
MT is Shorter
Difference with effective reference length (%)
Brevity Penalty MT is Longer
![Page 27: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/27.jpg)
BP = exp(1-(20/18)) = 0.89
BP = exp(1-(20/4)) = 0.02
Hyp to the American plane
Hyp appeared calm when he was taken to the American plane, which will to Miami, Florida.
Ref 1 Orejuela appeared calm as he was led to the American plane which will take him to Miami, Florida.
Ref 1 Orejuela appeared calm as he was led to the American plane which will take him to Miami, Florida.
r = 20
r = 20
c = 18
c = 4
![Page 28: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/28.jpg)
BLEU Geometric average of the n-gram precisionsOptionally weight them with wMultiplied by the brevity penalty
![Page 29: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/29.jpg)
exp(1-(20/18)) * exp((ln .83 + ln .59 + ln .31 + ln .2)/4) = 0.374
exp(1-(20/4)) * exp((ln 1 + ln 1 + ln 1 + ln 1)/4) = 0.018
Hyp to the American plane
Hyp appeared calm when he was taken to the American plane, which will to Miami, Florida.
BLEU
![Page 30: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/30.jpg)
Problems with BLEU
Synonyms and paraphrases are only handled if they are in the set of multiple reference translations
The scores for words are equally weighted so missing out on content-bearing material brings no additional penalty.
The brevity penalty is a stop-gap measure to compensate for the fairly serious problem of not being able to calculate recall.
![Page 31: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/31.jpg)
More Metrics
WER - word error ratePI-WER - position independent WERMETEOR - Metric for Evaluation of Translation with Explicit ORderingTERp - Translation Edit Rate plus
![Page 32: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/32.jpg)
Cross-lingual Word Representations
![Page 33: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/33.jpg)
GoalLearn the translations of individual words without large bilingual parallel corpora
44
ACL 1995
Egypt 196BC
![Page 34: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/34.jpg)
Translations from monolingual texts
Word embeddings have been shown to be useful for many natural language processing tasks. Can we use these vector space models to learn translations for rare words?
![Page 35: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/35.jpg)
Monolingual Word Embeddings
catdog
Baltimore
MondayFriday
![Page 36: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/36.jpg)
Monolingual Word Embeddings
Baltimore
senen jumat
kucinganjing
catdog
Baltimore
MondayFriday
![Page 37: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/37.jpg)
Bilingual Word Embeddings
cat
dog
Baltimore
Monday
Friday
baltimore
senen
jumat
kucing
anjing
![Page 38: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/38.jpg)
Projecting Vector Space Models
49
crecer
rápidamente
economíasplaneta
empleoextranjero
crecer
rápidamente
economíasplaneta
empleoextranjero
1
crecer
rápidamente
economíasplaneta
empleoextranjero
1
1
crecer
rápidamente
economíasplaneta
empleoextranjero
2
1... este número podría crecer muy rápidamente si no se modifica ...
... nuestras economías a crecer y desarrollarse de forma saludable ...
... que nos permitirá crecer rápidamente cuando el contexto ...
![Page 39: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/39.jpg)
Projecting Vector Space Models
50
7
4
3
1
1
2
5
7
9
crecerexpand
activity
rápidamente
economíasplaneta
empleoextranjero
policy
7
4
37
4
dict.
1
1
2
5
7
9
crecerexpand
activity
quicklypolicy
economic
growth
employment
rápidamente
economíasplaneta
empleoextranjero
policy
crecer(projected)
7
41
1
2
5
7
9
expand
activity
quicklypolicy
economic
growth
employment
policy
crecer(projected)
![Page 40: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/40.jpg)
Word EmbeddingsInstead of high dimensional vector space models used by Rapp and others in the past, we use low-dimensional word embeddings.
51
![Page 41: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/41.jpg)
Learning Bilingual Embeddingsmapping function W
52
![Page 42: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/42.jpg)
Learning Bilingual Embeddingsmatrix of source language embeddings
53
![Page 43: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/43.jpg)
Learning Bilingual Embeddingsmatrix of target language embeddings
54
![Page 44: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/44.jpg)
Learning Bilingual Embeddingsbilingual dictionaries orWikipedia inter-language links
55
![Page 45: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/45.jpg)
Learning Bilingual EmbeddingsApply W to a source language
embedding
56
![Page 46: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/46.jpg)
Learning Bilingual EmbeddingsProject it into the target language space
57
![Page 47: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/47.jpg)
Learning Bilingual EmbeddingsCompare against all target language embeddings
use nearest neighbors as translations
58
![Page 48: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/48.jpg)
Example Translations for Indonesian
59
mediterania pertumbuhannya solusi pagar armada
mediterranean growth solutions fence armada
aegean exponentially solver tail fleet
atlantic germination alternatives fences shipsbaltic rapidly solving info warships
levantine regrowth bootstrapping perimeter freighter
europe thrive solution biography tanker
adriatic /year objective around oiler
pacific growing problem moat lst
marmara steadily enabler embankment frigate
caribbean stunted solvers clothing squadron
![Page 49: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/49.jpg)
Ways to learn WLinear Mapping
Neural Net
Matrix Factorization with Bayesian Personalized Ranking
60
Full details in Wijaya et al (EMNLP 2017)
Derry Wijaya’s postdoc was funded by LORELEI. She is now an assistant professor at Boston University.
![Page 50: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/50.jpg)
We need seed bilingual dictionaries to learn the mapping between source and target language embeddings.
We previously created bilingual dictionaries via crowdsourcing between English and 100 other languages.
Derry tested her models on more than 2 dozen high and low resource languages.
Bilingual Dictionaries
61Full details in Pavlick et al (TACL 2014)
![Page 51: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/51.jpg)
Can we use images instead of bilingual dictionaries?
67
![Page 52: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/52.jpg)
Massively Multilingual Image Dataset (MMID)100 languages, 10,000 words per language, plus 250K English word translations
100 images per word, 35M images, plus text of web pages they appeared on (20TB of data)
68
Hosted by Amazon Public Datasets multilingual-images.org
Full details in Hewitt, Ippolito et al (ACL 2018)
![Page 53: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/53.jpg)
Image-based Translation Previous papers have tried to learn translations based on visual similarity of images.
Bergsma and Van Durme (2011) used SIFT+Histogram featuresKiela et al (2015) used Convolutional Neural Network features
They focused on translating nouns in high resource languages.
69
![Page 54: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/54.jpg)
New multilingual image corpusWe collect images for the 100 bilingual dictionaries created by Pavlick et al (2014)
100 languages, 10,000 words per language, ~263K English word translations (all POS)
We collected images with Google Image Search
100 images per word, 35M images, 21 TB of data
70
![Page 55: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/55.jpg)
Example translations
71
Top 4 English translations for Indonesian word kucing by finding k-NN English images using CNN vectors
![Page 56: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/56.jpg)
Accuracy of image-based translation
72
0
0.125
0.25
0.375
0.5
Span
ish
Dutc
h
Germ
an
Turk
ish
Hung
aria
n
Arab
ic
Thai
Viet
nam
ese
Slov
ak
Latv
ian
Cebu
ano
Wel
sh
Beng
ali
Uzb
ek
Telu
gu
Guja
rati
High resource languages(average = .29)
Low resource languages(average = .16)
![Page 57: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/57.jpg)
When does it work?Nouns and adjectives translate better than verbs and adverbs
Abstract words translate poorly compared to concrete words
73
Most Abstract Words
1. essentialness - 1.042. hope - 1.043. spirituality - 1.074. although - 1.075. possibility - 1.33
Most Concrete Words
1. tulip - 5.02. telescope - 5.03. elephant - 5.04. bedsheet - 5.05. strawberry - 5.0
![Page 58: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/58.jpg)
Concrete example
74
![Page 59: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/59.jpg)
Abstract example
75
![Page 60: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/60.jpg)
Can we predict concreteness?
76
Previous state of the art Our method
![Page 61: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/61.jpg)
We can produce better translations
77
![Page 62: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/62.jpg)
Mitigating Geographic Bias of Image Classifiers with MMID
4XHVWLRQ��:KDW¶V�ZURQJ�ZLWK�WKHVH�SUHGLFWLRQV"
78
Question: What’s wrong with these predictions?
Problem: ~75% of images in ImageNet are from Western countries.
![Page 63: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/63.jpg)
Indian Weddings
79
Find images by translating wedding into Bengali, Bishnupriya-Manipuri, Gujarati, Hindi, Kannada, Malayalam, Marathi, Punjabi, Tamil, and Telugu
![Page 64: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/64.jpg)
80
$WKOHWH &KLOGUHQ )DUPHU
Culturally Divergent Images
Work by Penn students Yoni Nachmany, Nikhil Krishnan, Aditya Kashyap
![Page 65: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/65.jpg)
81
3ROLFH 0LOLWDU\ :HGGLQJ
Work by Penn students Yoni Nachmany, Nikhil Krishnan, Aditya Kashyap
Culturally Divergent Images
![Page 66: Reminders...I hate this movie Encoder Decoder Review: Encoder-Decoder MT MT is the task of automatically translating sentences from one language into another. We use bilingual parallel](https://reader033.fdocuments.net/reader033/viewer/2022042409/5f26a25dd0faa1750e2689d8/html5/thumbnails/66.jpg)
Comparison of Diverse Decoding Methods from Conditional Language Models. Daphne Ippolito, Reno Kriz, Joao Sedoc, Maria Kustikova and Chris Callison-Burch. ACL 2019.
Magnitude: A Fast, Efficient Universal Vector Embedding Utility Package. Ajay Patel, Alex Sands, Marianna Apidianaki and Chris Callison-Burch. EMNLP 2018. Demo papers.
Learning Translations via Images with a Massively Multilingual Image Dataset. John Hewitt, Daphne Ippolito, Brendan Callahan, Reno Kriz, Derry Wijaya and Chris Callison-Burch. ACL 2018.
Learning Translations via Matrix Completion. Derry Wijaya, Brendan Callahan, John Hewitt, Jie Gao, Xiao Ling, Marianna Apidianaki and Chris Callison-Burch. EMNLP 2017.
A Comprehensive Analysis of Bilingual Lexicon Induction. Ann Irvine and Chris Callison-Burch. Computational Linguistics 2016.
The Language Demographics of Amazon Mechanical Turk. Ellie Pavlick, Matt Post, Ann Irvine, Dmitry Kachaev, and Chris Callison-Burch. TACL 2014. 82
References