HIWIRE MEETING Granada, June 9-10, 2005

HIWIRE MEETINGHIWIRE MEETINGGranada, June 9-10, 2005Granada, June 9-10, 2005

JOSÉ C. SEGURA, LUZ GARCÍAJOSÉ C. SEGURA, LUZ GARCÍAJAVIER RAMÍREZJAVIER RAMÍREZ

GSTC UGRGSTC UGR

2 HIWIRE Meeting – Granada, 9-10 June, 2005

Schedule

Non-linear feature normalization ECDF segmental implementation Progressive equalization 2-class normalization

Non-linear speaker adaptation/independence Non-linear feature normalization Non-linear model adaptation

VAD and technique combination MO-LRT Bi-spectrum based VAD Combined Front-End


Schedule





ECDF-based nonlinear transformation (1)

CDF-matching nonlinear transformation

))((][

))((][)()(

)()()()(

)(])[(][)(

1

11

xCCxTy

yCCyTxyCxC

duupyCduupxC

ypxTpxTyxpx

XY

YXYX

y

YY

x

XX

YYX

In previous works we modeled CDF’s by using histograms


ECDF-based nonlinear transformation (2)

An alternative algorithm based on Order Statistics

Ttr

TttrT

T

trCyCCx

ECDFTtT

tryC

yyyyyyY

XYXt

tY

TrrrT

)(1

,,1)]([

5.0)())(ˆ(ˆ

,,15.0)(

)(ˆ

},,,{

11

)()2()1(21

Is faster, only requires sorting and table indexing Results are almost equal to those obtained with histograms


ECDF Segmental implementation

Based on a sliding window

)12()2()1(},,,,{ TrrrTttTtt yyyyyyY

José C. Segura, M. Carmen Benítez, Ángel de la Torre, Antonio J. Rubio, Javier Ramírez, Cepstral domain segmental nonlinear feature transformations for robust speech recognition, IEEE Signal Processing Letters.,Vol.11, pp. 666-669, 2004


Progressive normalization

As not all MFCC offer equal discrimination And HEQ introduces certain distortion Normalization up to a certain MFCC gives the best performance


ECDF-based normalization results


2-class normalization (1)

A first approach on parametric non-linear equalization PDF’s are modeled as two-Gaussian class mixtures for each MFCC Actually we use speech/noise like classes EM is used on each sentence to obtain the Gaussian classes

C0 C1

Tes

t01

Tes

t02


2-class normalization (2)

22

2

211

1

1 )|2()|1(ˆ xxy

yxx

y

y yyP

yyPx

Equalization of C1 between Test02(Car) and Test01(Clean) of WSJ0 data

Nonlinear parametric transformation


2-class normalization results


Schedule





ECDF Features Normalization

HEQ as a non-linear speaker normalization technique using ECDF

-20 0 20 400

0.5

1

a) Reference (blue) and estimated (red) ECDF-20 -10 0 10 20

-20

0

20

40

b) Transformation

0 200 400 600-20

-10

0

10

20

30

c) Original features0 200 400 600

-20

-10

0

10

20

d) Transformed features


ECDF Norm. for SA

85,50

86,00

86,50

87,00

87,50

88,00

88,50

89,00

89,50

TES

T01

WA

C

MLLR BASELINE AFE ECDF NORM

Test01WER (%)

Test01WAC (%)

MLLR 10,97 89,03

BASELINE 13,22 86,78

AFE 12,74 87,26

ECDF 11,23 88,77


ECDF Models Adaptation

2 APPROACHES Pure Equalization: “HEQ MOD”

new Gaussian Distributions:

- shift on the means: X ->X HEQ

- scale factor on the variances

Equalization mixed with linear transformation: “HEQ PLIN”

LT: XA = M*X + B

M’, B’ such that

D(XA, XHEQ) = || M’X+B’ - XHEQ || 2 = minimum

Speaker Independent Features

Sp

ea

ke

r S

pe

cif

ic F

ea

ture

s


Models Adaptation

85,00

85,50

86,00

86,50

87,00

87,50

88,00

88,50

89,00

89,50

TES

T01

WA

C

MLLR BASELINE HEQ MOD HEQ PLIN

Test01WER (%)

Test01WAC (%)

MLLR 10,97 89,03

BASELINE 13,22 86,78

HEQ MOD 12,95 87,05

HEQ PLIN 13,31 86,52


SA methods. Comparison

85,0085,5086,0086,5087,0087,5088,0088,5089,0089,50

Tes

t 01

WA

C

Baseline MLLR HEQMOD

HEQ PLIN ECDFNORM

Speaker adaptation methods


Future Work 1/2

45,00

50,00

55,00

60,00

65,00

70,00W

AC

MLLR MRC BASELINE AFE ECDF NORM

SA models using MLLR are not robust against noise

Feature Normalization + MLLR


Future Work 2/2

Non linear Feature Normalization and Model Adaptation

Development of further experiments with more complex tasks on WSJ1 database (spoke3 and spoke4)


Schedule





Previous work on VAD

Voice activity detection: Kullback-Leibler divergence

J. Ramírez, J. C. Segura, C. Benítez, A. de la Torre, A. Rubio, “A New Kullback-Leibler VAD for Robust Speech Recognition”,IEEE Signal Processing Letters, Vol.11, No.2, pp. 666-669, Feb. 2004

Long-term spectral divergence J. Ramírez, J. C. Segura, C. Benítez, A. de la Torre, A. Rubio, “Efficient Voice Activity

Detection Algorithms Using Long-Term Speech Information”,Speech Communication, Vol. 42/3-4, pp. 271-287, 2004

Subband SNR estimation using OS filters J. Ramírez, J. C. Segura, C. Benítez, A. de la Torre, A. Rubio, “An Effective Subband

OSF-based VAD with Noise Reduction for Robust Speech Recognition”,To appear in IEEE Transactions on Speech and Audio Processing, 2005/2006.

Multiple observation likelihood ratio test J. Ramírez, J. C. Segura, C. Benítez, L. García, A. Rubio, “Statistical Voice Activity

Detection using a Multiple Observation Likelihood Ratio Test”, To appear in IEEE Signal Processing Letters


Likelihood ratio test

Generalization of the Sohn’s VAD: J. Sohn, N. S. Kim, W. Sung, “A statistical model-based voice activity

detection”, IEEE Signal Processing Letters, vol. 16 (1), pp. 1-3, 1999.

Two hypothesis are considered: H0 : y= n Absence of speech

(Silence) H1 : y= s + n Speech presence

Optimum decision rule (Bayes classifier):

l-frame observation vector:

LRT evaluation Adequate signal model

)( )H(P

)H(P

)H|ˆ(

)H|ˆ()ˆ(

1

0

0H|

1H|

1H

0H0

1

l

ll p

pL

y

yy

y

y

ly

LRT: Likelihood ratio test


Multiple observation likelihood ratio test

MO-LRT (multiple observation LRT): Given a set of N= 2m+1 consecutive observations:

LRT:

Under statistical independence:

Recursive Log-LRT:

ml

mlk k

kmllmlN

k

k

p

pL

)H|ˆ(

)H|ˆ()ˆ,...,ˆ,...,ˆ(

0H|

1H|

0

1

y

yyyy

y

y

)( )H(P

)H(P

)H|ˆ,...,ˆ,...,ˆ(

)H|ˆ,...,ˆ,...,ˆ()ˆ,...,ˆ,...,ˆ(

1

0

0H|,...,,...,

1H|,...,,...,

1H

0H0

1

mllml

mllmlmllmlN

mllml

mllml

p

pL

yyy

yyyyyy

yyy

yyy

ml

mlk k

kmllmlN

k

k

p

p

)H|ˆ(

)H|ˆ(ln)ˆ,...,ˆ,...,ˆ(

0H|

1H|

0

1

y

yyyy

y

y

mllllml yyyyy ˆ ..., ,ˆ ,ˆ ,ˆ ..., ,ˆ 11


Analysis: Optimum delay

-20 0 20 400

0.01

0.02

0.03

0.04m= 1

-20 0 20 400

0.01

0.02

0.03

0.04m= 4

-20 0 20 400

0.01

0.02

0.03

0.04m= 6

-20 0 20 400

0.01

0.02

0.03

0.04m= 8

Non-speech

Speech

Non-speech

Speech

Non-speech

Speech

Non-speech

Speech

Probability distributions Classification errors

Increasing m (number of the observations): Reduction of the overlap between the distributions Misclassification errors:

Reduced for speech vs Moderate increase for non-speech

1 2 3 4 5 6 7 8 9 10 11 120.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

0.22

0.24

0.26

Non-Speech

SpeechTotal error

m


Analysis: Optimum delay

ROC analysis AURORA 3 Spanish (High-Ch1, 5dB)

Sohn’s VAD

MO-LRT


Speech recognition experiments

MO-LRT G.729 AMR1 AMR2 AFE

86.14 70.32 74.29 82.89 83.29

Ref. VAD Woo Li Marzinzik Sohn

86.86 81.09 82.11 85.23 83.80

VAD

Noiseestimation

WienerFiltering

(WF)MFCC HTK

Framedropping

(FD)

Average Wacc (%) for CT and MCT

AURORA 2:


Speech recognition experiments

WACC (%) MO-LRT G.729 AMR1 AMR2 AFE

WM 96.33 88.62 94.65 95.67 95.28

MM 91.61 72.84 80.59 90.91 90.23

HM 87.43 65.50 62.41 85.77 77.53

Average 91.79 75.65 74.33 90.78 87.68

MO-LRT Woo Li Marzinzik Sohn

WM 96.33 95.35 91.82 94.29 96.07

MM 91.61 89.30 77.45 89.81 91.64

HM 87.43 83.64 78.52 79.43 84.03

Average 91.79 89.43 82.60 87.84 90.58

AURORA 3: Spanish SpeechDat-Car


Work in progress

Statistical tests in the bispectrum domain:

J. M. Górriz, et al., “Voice Activity Detection Based on HOS”, 8th International Work-Conference on Artificial Neural Networks (IWANN'2005)

J. M. Górriz, et al., “Statistical Tests for Voice Activity Detection”, Non-linear Speech Processing (NOLISP’2005), 2005.

J. M. Górriz, et al., “Bispectra analysis-based VAD for robust speech recognition”, First International Work-Conference on the Interplay Between Natural and Artificial Computation (IWINAC’2005)

Bispectrum LRT (application of MO-LRT on the bispectra)

J. M. Górriz, et al, “An Improved MO-LRT VAD Based on a Bispectra Gaussian Model”, Submitted to Electronics Letters.


GSTC-UGR speech recognition results

Noisereduction

LTSEVAD

Framedropping

SegmentalECDF

(Gaussian ref.)Progressive

HTK

LTSE VAD: J. Ramírez, et al., “Efficient Voice Activity Detection Algorithms Using Long-

Term Speech Information”, Speech Communication, Vol. 42/3-4, pp. 271-287, 2004

Segmental ECDF: 60 frame delay J. C. Segura, et al., “Cepstral Domain Segmental Nonlinear Feature

Transformations for Robust Speech Recognition”, IEEE Signal Processing Letters, Vol.11, No. 5, pp. 517 - 520, 2004

Progressive: Log-E + Up to the 4th cepstral coefficient



AURORA 2

WACC (%)

SET A SET B SET C Average

Multicondition training

GSTC-UGR 90.58 90.23 89.10 90.14

HIWIRE baseline 88.40 88.96 88.97 88.74

Clean training GSTC-UGR 86.01 86.84 85.00 86.14

HIWIRE baseline 64.00 69.10 64.73 66.18

AURORA 3

WACC (%)

Italian Spanish Average

WM MM HM WM MM HM WM MM HM

GSTC-UGR 96.94 91.89 86.19 96.52 92.03 89.95 96.73 91.96 88.07

HIWIRE baseline 94.40 87.14 46.75 89.30 83.18 65.50 91.85 85.16 56.13

WER Relative Improvements: 12% (MCT) 59% (CT)

WER Relative Improvements: 60% (WM) 46% (MM) 73% (HM)



Test 1 2 3 4 5 6 7 Avg

GSTC-UGR 13.37 19.52 37.53 40.22 39.19 37.16 39.30 32.33

HIWIRE baseline 13.22 24.68 46.00 47.62 52.67 44.79 54.73 40.53

Test 8 9 10 11 12 13 14 Avg.

GSTC-UGR 21.40 30.76 45.49 48.43 50.46 45.30 48.77 41.52

HIWIRE baseline 22.58 36.21 55.40 58.31 65.34 54.11 62.28 50.60

AURORA 4 WER (%) (clean training experiments)

WER Relative Improvements: 20% (Test sets 1:7) 17% (Test sets 8:14)

HIWIRE MEETINGHIWIRE MEETINGGranada, June 9-10, 2005Granada, June 9-10, 2005

JOSÉ C. SEGURA, LUZ GARCÍAJOSÉ C. SEGURA, LUZ GARCÍAJAVIER RAMÍREZJAVIER RAMÍREZ

GSTC UGRGSTC UGR

HIWIRE MEETING Granada, June 9-10, 2005

Documents

Transcript of HIWIRE MEETING Granada, June 9-10, 2005