CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 ·...
Transcript of CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 ·...
![Page 1: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/1.jpg)
CS460/626 : Natural Language Processing/Speech, NLP and the Web
(Lecture 35– Phonetics and phonology; syllabification)
Pushpak BhattacharyyaCSE Dept., IIT Bombay
7th April, 2011
![Page 2: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/2.jpg)
� Task of converting a word from one alphabetic script to another
Motivation for Phonetics and Phonology: Transliteration
script to another
Used for:
� Named entities
� : Gandhiji
� Out of vocabulary words
� : Bank
![Page 3: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/3.jpg)
� Accents
: Thoda or thora?
Linguistic issues
� Mapping of sounds
� Mahaan: Kahaan:
� Back-transliteration
![Page 4: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/4.jpg)
xTransliteration process
Source String Target String
TransliterationUnits
TransliterationUnits
![Page 5: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/5.jpg)
Source String Target String
TransliterationUnits
TransliterationUnits
![Page 6: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/6.jpg)
Phoneme-based approach
Word inSource language
Word inTarget language
P(wt)
Pronunciationin
Source language
PronunciationIn
target language
P( ps | ws)
P ( pt | ps )
P ( wt | pt )
Note: Phoneme is the smallest linguistically distinctive unit of sound.
Wt* = argmax (P (wt). P (wt | pt) . P (pt | ps) . P (ps | ws) )
![Page 7: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/7.jpg)
How phonemes play a role
Transliterating ‘BAPAT’
B A P A T
Source word
to phonemes
Step I :
Consider each character of the word
P /ә//ә/ /a://a://ә//ә/ /a://a:/B T
to phonemes
P /ә//ә/ /a://a://ә//ә/ /a://a:/BT
Source phonemes
to target phonemes
t
t
Step II :Converting to phoneme seq.Step III : Converting to target phoneme seq.
![Page 8: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/8.jpg)
� Unknown pronunciations
Issues in phonetic model
sanhita
� Back-transliteration can be a problem
Johnson� � Jonson
samhita
![Page 9: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/9.jpg)
Back to Phonetics
![Page 10: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/10.jpg)
Ancient 5 x 5 Indian Classification
of Consonants
Group
क वग� क ख ग घ ङ Velar
च वग� च छ ज झ ञ Palatal
ट वग� ट ठ ड ढ ण Alveolar
त वग� त थ द ध न Dental
प वग� प फ ब भ म Labialप वग� प फ ब भ म Labial
![Page 11: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/11.jpg)
Place of Articulation� Labial: Two lips coming together
� [p] as in possum, [b] as in bear
� Dental: Tongue against the teeth
� [th] of thing or the [dh] of though
� Alveolar: Alveolar ridge is the portion of the roof of the mouth just behind the upper teeth; tip of the tongue against the alveolar ridge.
� Phones [s], [z], [t], and [d]
� Palatal: Roof of the mouth; blade of the tongue against this rising back of the alveolar ridge
� sounds [sh] (shrimp), [ch] (china), [zh] (Asian), and [jh] (jar)
� Velar: Movable muscular flap at the back of the roof of the mouth; back of the tongue up against the � Velar: Movable muscular flap at the back of the roof of the mouth; back of the tongue up against the velum
� sounds [k] (cuckoo), [g] (goose), and [N] (kingfisher)
� Glottal: closing the glottis (by bringing the vocal folds together)
� glottal stop [q] (IPA [P]) is made by closing the glotis
![Page 12: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/12.jpg)
Manner of Articulation: Stops and Nasals
� All consonants are produced by restriction of airflow
� Manner of Articulation; how the restriction is produced:
� complete or partial stoppage
� A stop is a consonant in which airflow is completely blocked for a short time
� English has voiced stops like [b], [d], and [g] as well as unvoiced stops like [p], [t], and [k].
� Stops are also called plosives
Nasal sounds [n], [m], and [ng] are made by lowering the velum and allowing air � Nasal sounds [n], [m], and [ng] are made by lowering the velum and allowing air to pass into the nasal cavity
![Page 13: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/13.jpg)
Fricatives� Fricatives, airflow is constricted but not cut off completely. The turbulent airflow that
results from the constriction produces a characteristic “hissing” sound.
� The English labiodental fricatives [f] and [v] are produced by pressing the lower lip against the upper teeth, allowing a restricted airflow between the upper teeth.
� The dental fricatives [th] and [dh] allow air to flow around the tongue between the teeth.
� The alveolar fricatives [s] and [z] are produced with the tongue against the alveolar ridge, forcing air over the edge of the teeth.
� In the palato-alveolar fricatives [sh] and [zh] the tongue is at the back of the alveolar ridge forcing air through a groove formed in the tongue. ridge forcing air through a groove formed in the tongue.
![Page 14: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/14.jpg)
Affricates, Laterals/Liquids and Taps/Flaps
� Affricates are stops followed immediately by fricatives
� English [ch] (chicken); Marathi chaa (e.g., gharaachaa; of the house)
� Lateral or Liquids: tip of the tongue up against the alveolar ridge or the teeth, with one or both sides of the tongue lowered to allow air to flow over it
� [l] (learn)
� Tap or flap: quick motion of the tongue against the alveolar ridge
� [dx] (IPA [R])
The consonant in the middle of the word lotus ([l ow dx ax s]) is a tap in most dialects � The consonant in the middle of the word lotus ([l ow dx ax s]) is a tap in most dialects of American English
� speakers of many UK dialects would use a [t] instead of a tap in this word.
![Page 15: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/15.jpg)
Articulation of consonants: Larynx action/glottis state (1/2)
� Vocal cords are pulled apart. The air passes freely through the glottis. This is called the voicelessness state and sounds produced with this configuration of the vocal cords are called voiceless: p t k f θ s ʃʃʃʃ tʃʃʃʃ
� Vocal cords are pulled close together.
� The air passing through the glottis causes the vocal cords to vibrate. This is called the voicing state and sounds produced with this configuration of the vocal cords are called voiced: b d g v ð z ʒ dʒthe vocal cords are called voiced: b d g v ð z ʒ dʒ
![Page 16: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/16.jpg)
Articulation of consonants: Larynx action/glottis state (2/2)
� Vocal cords are apart at the back and pulled together at the front. This is called the whisper state.
� Vocal cords assume the voicing state but are relaxed. This is called the murmur state.
![Page 17: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/17.jpg)
Vowels (1/2)
![Page 18: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/18.jpg)
Vowels (2/2)
![Page 19: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/19.jpg)
IPA symbol for Vowels
Courtesy: http://www.antimoon.com/misc/phonchart2008.pdf
![Page 20: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/20.jpg)
IPA symbol for consonants
Courtesy: http://www.antimoon.com/misc/phonchart2008.pdf
![Page 21: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/21.jpg)
Vowels and Tongue Positions
![Page 22: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/22.jpg)
Phonology: Syllables
![Page 23: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/23.jpg)
Basic of syllables
“Syllable is a unit of spoken language
consisting of a single uninterrupted soundformed generally by a Vowel and preceded orfollowed by one or more consonants.”
� Vowels are the heart of a syllable (MostSonorous Element) (svayam raajate itisvaraH)
� Consonants act as sounds attached tovowels.
![Page 24: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/24.jpg)
Syllable structure
� A syllable consists of 3 major parts:-� Onset (C)
� Nucleus (V)Nucleus (V)
� Coda (C)
� Vowels sit in the Nucleus of a syllable
� Consonants may get attached as Onset or Coda.
� Basic structure - CV
![Page 25: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/25.jpg)
Possible syllable structures� The Nucleus is always present
� Onset and Coda may be absentmay be absent
� Possible structures
� V
� CV
� VC
� CVC
![Page 26: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/26.jpg)
syllable theories
� Prominence Theory
� E.g. entertaining /entәteɪnɪŋ/
� The peaks of prominence: vowels /e ә eɪɪ/
Number of syllables: 4� Number of syllables: 4
� Chest Pulse Theory
� Based on muscular activities
� Sonority Theory
� Based on relative soundness of segment within words
![Page 27: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/27.jpg)
Introduction to sonority theory
“The Sonority of a sound is its loudness relative to other sounds with the same length, stress and speech.”
� Some sounds are more sonorous� Words in a language can be divided into
syllables� Sonority theory distinguishes syllables on
the basis of sounds.
![Page 28: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/28.jpg)
Sonority hierarchy
� Defined on the basis of amount of sound associated
� The sonority hierarchy is as follows:-Vowels (a, e, i, o, u)� Vowels (a, e, i, o, u)
� Liquids (y, r, l, v)
� Nasals (n, m)� Fricatives (s, z, f,…..sh, th etc.)� Affricates (ch, j)
� Stops (b, d, g, p, t, k)
![Page 29: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/29.jpg)
Sonority scale� Obstruents can be further classified into:-
� Fricatives� Fricatives
� Affricates
� Stops
![Page 30: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/30.jpg)
Sonority theory & syllables
“A Syllable is a cluster of sonority, defined by a sonority peak acting as a structural magnet to the surrounding lower sonority elements.”
� Represented as waves of sonority or Sonority Profile of that syllable
Nucleus
Onset Coda
![Page 31: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/31.jpg)
Sonority sequencing principle
“The Sonority Profile of a syllable must rise until its Peak(Nucleus), and then fall.”
Peak
(Nucleus)
Onset Coda
![Page 32: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/32.jpg)
examples
� ABHIJEET
A JEETProfile-1
A
BHI
JEET
ABHI
JEET
Profile-2
![Page 33: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/33.jpg)
Maximal onset principle
“The Intervocalic consonants are maximallyassigned to the Onsets of syllables inconformity with Universal and Language-Specific Conditions.”Specific Conditions.”
� Determines underlying syllable division
� Example� DIPLOMA
DIP LO MA & DI PLOMA
![Page 34: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/34.jpg)
Syllable Structure: a more detailed look
� Count of no. of syllables in a word is roughly/intuitively the no. of vocalic segments in a word.
� Thus, presence of a vowel is an obligatory element in the structure of a syllable. This vowel is called “nucleus”.
� Basic Configuration: (C)V(C).
Part of syllable preceding the nucleus is called the onset.� Part of syllable preceding the nucleus is called the onset.
� Elements coming after the nucleus are called the coda.
� Nucleus and coda together are referred to as the rhyme.
S ≡ Syllable, O ≡ OnsetR ≡ Rhyme, N ≡ NucleusCo ≡ Coda
![Page 35: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/35.jpg)
Syllable Structure: Examples� ‘word’
� ‘sprint’
![Page 36: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/36.jpg)
Syllable Structure: Examples� ‘may’
� ‘opt’
� No Coda.
� ‘air’
� No Onset.
� No Coda, No Onset.
![Page 37: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/37.jpg)
Syllable Structure� Open Syllable: ends in vowel
� Closed syllable: ends in consonant or consonant cluster
� Light Syllable: A syllable which is open and ends in a short vowel
General Description – CV.� General Description – CV.
� Example, ‘air’.
� Heavy Syllable: Closed syllables or syllables ending in diphthong
� Example: ‘opt’
� Example, ‘may’
![Page 38: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/38.jpg)
Syllabification: Determining Syllable Boundaries
� Given a string of syllables (word), what is the coda of one and the onset of another?
� In a sequence such as VCV, where V is any vowel and C is any consonant, is the medial C the coda of the first syllable (VC.V) or the onset of the second syllable (V.CV)? (VC.V) or the onset of the second syllable (V.CV)?
� To determine the correct groupings, there are some rules, two of them being the most important and significant:
� Maximal Onset Principle,
� Sonority Hierarchy
![Page 39: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/39.jpg)
Discussion on the assignment
![Page 40: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/40.jpg)
Data
� The Carnegie Mellon University Pronouncing Dictionary
� machine-readable pronunciation � machine-readable pronunciation dictionary for North American English that contains over 125,000 words and their transcriptions.
� The current phoneme set contains 39 phonemes
![Page 41: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/41.jpg)
“Parallel” Corpus
Phoneme Example Translation
------- ------- -----------
AA odd AA D
AE at AE T AE at AE T
AH hut HH AH T
AO ought AO T
AW cow K AW
AY hide HH AY D
B be B IY
![Page 42: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/42.jpg)
“Parallel” Corpus cntd
Phoneme Example Translation
------- ------- -----------CH cheese CH IY Z
D dee D IY
DH thee DH IY EH Ed EH D DH thee DH IY EH Ed EH D
ER hurt HH ER T
EY ate EY T
F fee F IY
G green G R IY N
HH he HH IY
IH it IH T
IY eat IY T
JH gee JH IY
![Page 43: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/43.jpg)
The tasks
� First obtain the Carnegie Mellon University's Pronouncing Dictionary
� Train and Test the following Statistical Machine Learning Statistical Machine Learning Algorithms
� HMM - For HMM you can use either Natural Language Toolkit or you can use GIZA++ with MOSES.
![Page 44: CS460/626 : Natural Language Processing/Speech, …cs626-460-2012/cs626-460...2011/04/07 · CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 35– Phonetics](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda7dea89384a50e0790f42/html5/thumbnails/44.jpg)
Tasks (cntd)
� Report all the results using 5-fold cross Validation
� Compare all the results obtained in � Compare all the results obtained in the previous steps in terms of
� Precision
� Recall
� F-Score
� Finally do a detailed error analysis.