Speech Processing Laboratory, Temple University May 5, 2004 1 Structure-Based Speech...

Speech Processing Laboratory, Temple University Speech Processing Laboratory, Temple University

May 5, 2004May 5, 2004

Structure-Based Speech Classification Structure-Based Speech Classification Using Nonlinear Embedding Using Nonlinear Embedding

TechniquesTechniques

Uchechukwu Ofoegbu

AdvisorDr. Robert E. Yantorno

CommitteeDr. Saroj K. Biswas

Dr. Henry M. Sendaula

May 5, 2004May 5, 2004

AcknowledgmentAcknowledgment Dr. Robert YantornoDr. Robert Yantorno Dr. Saroj BiswasDr. Saroj Biswas Dr. Henry SendaulaDr. Henry Sendaula Speech Lab MembersSpeech Lab Members

Air Force Research Laboratory,Air Force Research Laboratory,Rome, NYRome, NY

May 5, 2004May 5, 2004

OverviewOverview Voiced and Unvoiced Speech

Usable and Unusable Speech

Nonlinearities in Speech

Non-Linear Embedding

Research Goal

Proposed Research

May 5, 2004May 5, 2004

Voiced and Unvoiced SpeechVoiced and Unvoiced Speech

May 5, 2004May 5, 2004

Voiced/Unvoiced CharacteristicsVoiced/Unvoiced Characteristics

Voiced

Quasi-periodic excitation

Modulation by vocal tract

Production of vowels, voiced fricatives & plosives

Unvoiced

No periodic vibration of vocal chords

Noise-like nature

Production of unvoiced fricatives and plosives

May 5, 2004May 5, 2004

Usable SpeechUsable Speech

Portions of co-channel speech still usable for applications such as Speaker ID and Speech Recognition.

Low-energy (unvoiced/silence) segments overlap with high-energy (voiced) segments

Target-to-interferer Ratio (TIR) > 20dB

May 5, 2004May 5, 2004

Nonlinearities in SpeechNonlinearities in SpeechGlottal waveform changes

Shape varies with amplitude

Physical observations Flow in vocal tract is non-laminar

Coupling between vocal tract and folds When glottis is open, prominent changes are observed

in formant characteristics

May 5, 2004May 5, 2004

Nonlinear EmbeddingNonlinear Embedding

Nonlinear Systems

Point moving along some trajectory in an abstract state space

Coordinates of the point are independent degrees of freedom of the system

State space could be reconstructed from a scalar signal

May 5, 2004May 5, 2004

Nonlinear Embedding (cont’d)Nonlinear Embedding (cont’d)

Takens’ Method of Delays

A state space representation topologically equivalent to the original state space of a system can be reconstructed from a single observable dimension

Vectors in m-dimensional state space are formed from time-delayed values of a signal

May 5, 2004May 5, 2004

Nonlinear Embedding (cont’d)Nonlinear Embedding (cont’d)

dmisdisdisisix 1,,2,,

m = embedding dimension

d = delay value

May 5, 2004May 5, 2004

Nonlinear Embedding (Cont’d)Nonlinear Embedding (Cont’d)Delay value, d:

Dependent on sampling rate and signal properties

Large enough such that nonlinearities are taken into account by the reconstructed trajectory

Small enough to retain reasonable time resolution

May 5, 2004May 5, 2004

Nonlinear Embedding (Cont’d)Nonlinear Embedding (Cont’d)Dimension, m:

Generation of voiced speech constitutes a low-dimensional system

Generation of unvoiced speech constitutes a relatively high-dimensional system

Using a low dimension (such as m = 3) sufficiently reconstructs voiced but not unvoiced speech

May 5, 2004May 5, 2004

Embedded Voiced and Embedded Voiced and Unvoiced SpeechUnvoiced Speech

-50000

10000-5000

Embedded Voiced Speech

-2000-10000

10002000-2000

Embedded Unvoiced Speech

May 5, 2004May 5, 2004

Embedded Usable and Embedded Usable and Unusable SpeechUnusable Speech

-4000-2000

40006000

5000-4000

Embedded Co-channel Speech of 30dB TIR

-10000-5000

-10000

Embedded Co-channel Speech of 10dB TIR

May 5, 2004May 5, 2004

Research GoalResearch GoalFeature Extraction

Difference-Mean Comparison (DMC) Measure

– Voiced/unvoiced classification

Nodal Density Measure– Voiced/unvoiced classification– Usable/unusable classification

May 5, 2004May 5, 2004

Difference-Mean Difference-Mean Comparison (DMC) MeasureComparison (DMC) Measure

Voiced/Unvoiced ClassificationVoiced/Unvoiced Classification

May 5, 2004May 5, 2004

IntroductionIntroduction 3rd order difference computation along first

non-singleton dimension

Ist order difference of NxN matrix given by

Length(3rd order diff. > mean) observed

(2,1) (1,1) (2, 2) (1, 2) . . . (2, ) (1, )(3,1) (2,1) (3, 2) (2,2) . . . (3, ) (2, )

. . .( ,1) (( 1),1) ( , 2) (( 1),2) . . . ( , ) (( 1), )

X X X X X N X NX X X X X N X N

X N X N X N X N X N N X N N

May 5, 2004May 5, 2004

Embedded Voiced and Embedded Voiced and Unvoiced SpeechUnvoiced Speech

-50000

10000-5000

Embedded Voiced Speech

-2000-10000

10002000-2000

Embedded Unvoiced Speech

May 5, 2004May 5, 2004

Difference-Mean Comparison Difference-Mean Comparison Distribution Distribution

0 20 40 60 80 100 120 140 1600

Difference-Mean Comparison

Clean Speech

VoicedUnvoiced

May 5, 2004May 5, 2004

Difference-Mean Comparison Difference-Mean Comparison DistributionDistribution

0 20 40 60 80 100 120 140 1600

Speech + 15dB Pink Noise

VoicedUnvoiced

May 5, 2004May 5, 2004

Difference-Mean Comparison Difference-Mean Comparison DistributionDistribution

0 50 100 1500

Speech + 15dB White NoiseVoicedUnvoiced

May 5, 2004May 5, 2004

DMC-Based Decisions

200 400 600 800 1000 1200 1400-1

Clean Speech => 1:V; 0:Dont Care; -1:UV

200 400 600 800 1000 1200 1400-1

Sample Number

May 5, 2004May 5, 2004

DMC-Based Decisions

200 400 600 800 1000 1200 1400-1

Speech + 15dB Pink Noise => 1:V; 0:Dont Care; -1:UV

200 400 600 800 1000 1200 1400-1

Sample Number

May 5, 2004May 5, 2004

DMC-Based Decisions

200 400 600 800 1000 1200 1400-1

Speech + 15dB White Noise => 1:V; 0:Dont Care; -1:UV

200 400 600 800 1000 1200 1400-1

Sample Number

May 5, 2004May 5, 2004

DMC-Based Decisions

200 400 600 800 1000 1200 1400-1

Clean Speech => 1:V; 0:Dont Care; -1:UV

200 400 600 800 1000 1200 1400-1

Sample Number

May 5, 2004May 5, 2004

DMC-Based Decisions

200 400 600 800 1000 1200 1400-1

Speech + 15dB Pink Noise => 1:V; 0:Dont Care; -1:UVAm

200 400 600 800 1000 1200 1400-1

Sample Number

May 5, 2004May 5, 2004

DMC-Based Decisions

200 400 600 800 1000 1200 1400-1

Speech + 15dB White Noise => 1:V; 0:Dont Care; -1:UV

200 400 600 800 1000 1200 1400-1

Sample Number

May 5, 2004May 5, 2004

ResultsResultsHits Minus False Alarms for Voiced Speech

Clean 15dB P ink 15dB White

FR/RE E/ZC DMC

May 5, 2004May 5, 2004

Results (Cont’d)Results (Cont’d)Hits Minus False Alarms for Unvoiced Speech

Clean 15dB Pink 15dB White

FR/RE E/ZC DMC

May 5, 2004May 5, 2004

Nodal Density MeasureNodal Density Measure Voiced/Unvoiced ClassificationUsable/Unusable Classification

May 5, 2004May 5, 2004

IntroductionIntroduction Smallest cube which encloses the signal is

determined

This cube is divided into N smaller cubes

Edges of the smaller cubes are defined as nodes

Number of nodes spanned by the signal is determined

Ratio of number of nodes spanned to total number of nodes is defined as nodal density

May 5, 2004May 5, 2004

Voiced/Unvoiced ClassificationVoiced/Unvoiced Classification

May 5, 2004May 5, 2004

Embedded Voiced and Unvoiced Embedded Voiced and Unvoiced Speech Frames with GridsSpeech Frames with Grids

-0.1-0.05

0.10.15

-0.1-0.05

0.10.15-0.1

Voiced

-0.01-0.005

00.005

-0.0050

0.01-0.01

-0.005

Unvoiced

May 5, 2004May 5, 2004

Nodes Spanned by Embedded Voiced and Nodes Spanned by Embedded Voiced and Unvoiced Speech FramesUnvoiced Speech Frames

-0.1-0.05

0.10.15

-0.1-0.05

0.10.15-0.1

Voiced

-0.01-0.005

00.005

-0.005

0.01-0.01

-0.005

Unvoiced

May 5, 2004May 5, 2004

Nodal-Density Distribution Nodal-Density Distribution

0.03 0.04 0.05 0.06 0.070

Nodal-Density

Clean Speech VoicedUnvoiced

May 5, 2004May 5, 2004

0.03 0.04 0.05 0.06 0.070

Nodal-Density

VoicedUnvoiced

May 5, 2004May 5, 2004

0.04 0.045 0.05 0.055 0.06 0.065 0.07 0.0750

Nodal-Density

May 5, 2004May 5, 2004

FilteringFiltering

Moving Average Filter

Order, M = 10

May 5, 2004May 5, 2004

Nodal-Density Distributions after Nodal-Density Distributions after FilteringFiltering

0.03 0.04 0.05 0.06 0.070

Nodal Density

Clean Speech

VoicedUnvoiced

0.03 0.04 0.05 0.06 0.070

Nodal-Density

Clean Speech VoicedUnvoiced

May 5, 2004May 5, 2004

Nodal-Density Distributions after Nodal-Density Distributions after FilteringFiltering

0.03 0.04 0.05 0.06 0.070

Nodal Density

VoicedUnvoiced

0.03 0.04 0.05 0.06 0.070

Nodal-Density

Speech + 15dB Pink NoiseVoicedUnvoiced

May 5, 2004May 5, 2004

Nodal-Density Distributions After Nodal-Density Distributions After FilteringFiltering

0.04 0.05 0.06 0.070

Nodal Density

Speech + 15dB White Noise

VoicedUnvoiced

0.04 0.045 0.05 0.055 0.06 0.065 0.07 0.0750

Nodal-Density

May 5, 2004May 5, 2004

ResultsResultsHits Minus False Alarms for Voiced Speech

010203040506070

Clean 15dB P ink 15dB White

ND ND_Filt

May 5, 2004May 5, 2004

Results (Cont’d)Results (Cont’d)

Hits Minus False Alarms for Unvoiced Speech

010203040506070

Clean 15dB Pink 15dB White

ND ND_Filt

May 5, 2004May 5, 2004

Proposed ResearchProposed Research

Usable/Unusable ClassificationUsable/Unusable Classification

May 5, 2004May 5, 2004

Embedded Usable and Unusable Embedded Usable and Unusable Speech Frames with GridsSpeech Frames with Grids

-10000-5000

-10000

Embedded Co-channel Speech of 10dB TIR with Grids

5000-5000

Embedded Co-channel Speech of 30dB TIR with Grids

May 5, 2004May 5, 2004

Nodes Spanned by Embedded Usable Nodes Spanned by Embedded Usable and Unusable Speech Framesand Unusable Speech Frames

-4000-2000

40006000

5000-4000

Nodes Spanned by Embedded Co-channel Speech of 30dB TIR

-10000

5000-6000

-10000

5000-6000

May 5, 2004May 5, 2004

Preliminary ResultsPreliminary Results

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

1ROC Curve for Usable Speech Detection Using the Nodal Density Measure

False Alarms

May 5, 2004May 5, 2004

SummarySummary

SpeechSpeech Nonlinear Embedding

Difference-Mean

Comparison

Nodal Density Usable/Unusable Usable/Unusable

ClassificationClassification

V/UV ClassificationV/UV Classification

May 5, 2004May 5, 2004

Future Proposed ResearchFuture Proposed Research Determine optimum filter for nodal density-based

voiced/unvoiced classification

Develop nodal density measure for usable/unusable classification

Investigate the presence of complimentary information in between both features (DMC and nodal density) for voiced/unvoiced classification

Perform decision-level fusion of both features

May 5, 2004May 5, 2004

If you understood this If you understood this presentation presentation

……

please askplease ask QUESTIONS !!!QUESTIONS !!!

Speech Processing Laboratory, Temple University May 5, 2004 1 Structure-Based Speech...

Documents

Transcript of Speech Processing Laboratory, Temple University May 5, 2004 1 Structure-Based Speech...

A Speech Classification System detection: the incoming audio signal is classified according to speech/non-speech. The result is a time labelling for each speech signal. Non-speech

Nonlinear feature based classification of speech under ...crss.utdallas.edu/Publications/Zhou2001.pdfNonlinear Feature Based Classification of Speech Under Stress Guojun Zhou, Member,

Politeness theory and the classification of speech acts

Title · 2.3.1 Austin and Speech Act Classification 17 2.3.2 Searl's Approach to Speech Act Classification 19 2.4 Pragmatics, Indirectness and Politeness 20 2.4.1 Leech's Approach

K. Boakye: Qualifying Exam Presentation Speech Detection, Classification, and Processing for Improved Automatic Speech Recognition in Multiparty Meetings.

Speech Recognition Pattern Classification. 4 December 2015Veton Këpuska2 Pattern Classification Introduction Parametric classifiers Semi-parametric.

The classification of speech acts revisited: a dialogical ...webcom.upmf-grenoble.fr/DenisVernant/speechacts.pdf · 1 The classification of speech acts revisited: a dialogical and

Bayesian Predictive Classification With Incremental Learning for Noisy Speech Recognition

Nonlinear feature based classification of speech under ...

A Review on Speech Feature Techniques and Classification Techniques

Emotional Telugu Speech Signals Classification Based on K-nn Classifier

Speech task based automatic classification of ALS and ...

Personality in Speech: Assessment and Automatic Classification

Extensions to the Speech Disorders Classification - Waisman Center

Emotional telugu speech signals classification based on k nn classifier

Emotion Classification Technique in Speech Signal … Classification Technique in Speech ... Section 4 shows the experimental result ... we have proposed a novel emotion classification

Speech balloon contour classification in comics

Project 1 : Eigen-Faces Applied to Speech Style Classification

Hierarchical Phoneme Classification for Improved Speech ...

NON-SPEECH ENVIRONMENTAL SOUND CLASSIFICATION … · NON-SPEECH ENVIRONMENTAL SOUND CLASSIFICATION USING ... Introduction. Over the past ... as well as violent robbery and bag snatching.