DYNAMIC COMPONENTS OF LINEAR STABLE MIXTURES FROM FRACTIONAL LOWORDER MOMENTS

ThomasFabricius,PrebenKidmoseandLars Kai Hansen

TechnicalUniversityof DenmarkDepartmentfor MathematicalModelling

RichardPetersensPladsbldg. 3212800Lyngby

Denmark

ABSTRACT

The secondmomentbasedindependentcomponentanaly-sisschemeof Molgedey andSchusteris generalizedto frac-tional low order moments,relevant for linear mixturesofheavy tail stableprocesses.TheMolgedey andSchusteral-gorithmstandsoutbyallowingexplicitly constructionof theindependentcomponents.Surpricingly, this turnsout to bepossiblealsofor decorrelationbasedonfractionallow ordermoments.

Keywords: Blind SourceSeparation(BSS),DynamicCom-ponents,IndependentComponentAnalysis (ICA), Stableprocess,Fractionallowerordermoments,Cocktailpartyprob-lem

1. INTRODUCTION

Reconstructionof independentcomponentsfrom linearmix-turesis an importantsignalprocessingresearchareawithnumerouspracticalapplications[1, 2]. Typically, indepen-dent componentanalysis(ICA) proceedsfrom non-lineartransformations[3, 4] or from temporalcorrelations[5, 6].ThesecondmomentbasedMolgedey-Schusterapproachas-sumesthat the independentsourceshave differentautocor-relationfunctions[5, 7, 8]. Themainvirtueof theapproachis that an explicit solution (reconstructionof sourcesandmixing matrix) is possible.In [8] weappliedtheMolgedey-Schusteralgorithm to imagemixturesand we proposedaminor modificationof the algorithm that relieves a prob-lem of the original approach,namely that it occasionallyproducescomplex mixing coefficientsand sourcesignals.Attias andSchreinerproposeda very rich ICA frameworkbasedon higher order statisticsand decorrelation[9, 10],allowing for completelygeneralandlearnablesourcedistri-

This work is fundedby the DanishResearchCouncils throughtheTHORCenterfor Neuroinformatics.

butions,however at thepriceof significantadditionalcom-putation.

In this contribution we analyzedynamicdecorrelationfor heavy tail stablerandomsignal,andweprovideagener-alizationof theMolgedey-Schusteralgorithmbasedonfrac-tional low ordermomentsratherthansecondmoments.

2. STABLE DISTRIBUTIONS AND SOMEPROPERTIES

Randomvectorsfollowing a stabledistribution are stablewith respectto addition,i.e., if

�and � , arestablethenso

is thesum ��� ��� � . A randomvectorfollowsa strictlysymmetricstabledistribution iff its characteristicfunctionhastheform [11]��� � � ��������� ������ � ����������� "!$## ��&% ##('*) �,+.-/�0� 1�2��32 � (1)

where547698

and ) �,+.-/� is a finite Borel measurecalledthespectralmeasure,

-is theunit sphereand

�is thechar-

acteristicexponent.A randomvariable: thatis distributedby a symmetricstabledistributionwith characteristicexpo-nent

�is denoted:<; -=�>-

. In the classof stabledis-tributions we find the normal distribution

� �?� and theso-calledCauchydistribution

� �A@ . In generalthelow or-der stabledistributionshave heavy tails, andaresupposedto model,e.g.,speechsignals. Becauseof the heavy tail-s,higherordermomentsdo not exists,hence,ICA methodsbasedonsecondmomentsarenotwell-definedfor thesesig-nals. For multivariatestablevariablesthespectralmeasurecarriesinformationaboutthe dependenciesamongthe in-dividual componentsandthis maybeusedfor independentcomponentanalysis[12]. Herewe show that independen-t dynamiccomponentscanbe reconstructedexplicitly a laMolgedey Schuster, even if thesignalsfollow stabledistri-butions.

Our derivation is basedon the generalizedcovariationbetweenrandomvariables: and B jointly

-=�>-asdefined

in [11] C :EDFB9G ' �IH !KJMLON 'QPSRUT ) �,+.-/� (2)

with thedefinition V N '�PWRXT �ZY VOY 'QPSR sign� V � . Notethat

C :5DUB9G ' � 1 is necessarybut not sufficient for : and B tobe independent.The covariationhasa convenientpseudo-linearity property. Let : R , :$[ , B R and B\[ be jointly

-/�-and B R , B\[ independentthenC ] R : R � ] [�:^[_DF` R B R � `a[�Bb[0G ' �] R C : R DUB R G ' ` 'QPSRR � ] [ C :^[_DUB R G ' ` 'QPSRR �] R C : R DUB\[aG ' ` 'QPSR[ � ] [ C :^[_DUB\[aG ' ` 'QPSR[ (3)

From the covariationwe canderive the covariation coeffi-cient cbdfe g � C :EDFBhG 'C BiDFBhG ' (4)

which hastheproperty[11]cbdfe g �kjml :nB N(o PSRUTXpj l BqB N(o PSRUT p (5)

for @hrts 2u� . By equation(5) wecanestimatethecovari-ationcoefficient from data.

3. DECORRELATION BASED ON FRACTIONALLOW ORDER MOMENTS

We arenow equippedto generalizetheMolgedey-Schusterapproach,andstartout by definingthe matrix of observedsignals

�with elements:$v e w ��x7y>z{U| RS} v e { - { e w 1 r�~ 2�� (6)

where } v e { arethemixing coefficientsforming themixingmatrix

�,- { e w is an instanceof the � w,� -/�- sourcesignal

written asanelementin thematrix � and ��� is thenumberof sources.Wenow assumethatthedifferentsourcesignalsareindependent,C - { e w D -S� e w��&� G ' � 1 �������� 4�6�� (7)

The covariationcoefficient between: v e w and : 8 e w��&� canbefoundusingequations(4) and(6)cMd>�>� �Ue d>��� ���_� � � x�y z{U| RW} v

e { - { e w D x7y z� | RO} 8 e � - � e w��W��� '� x7yz� | R } 8e ��-S� e w��W� D x7y>z� | R } 8 e ��-S� e w��&� � '

(8)

Finally, we usethat the sourcesareindependent,hence,e-quation(3) is valid, providingc d �=� � e d ��� ���_� � x7y>z{U| R x7y>z� | RW} v e { C - { e w D - � e w��W� G ' } N '�PWRXT8 e �x�y>z{U| R x7y>z� | RO} 8 e { C - { e w��W� D -S� e w��W� G ' } N '�PWRXT8 e �

(9)

Wedefinematrices�^� ��� with elements  � � �v e 8 � c d �>� � e d ��� �¡�¢� ,£ � ��� with thenominatorof equation(8) aselements¤ � ���v e 8 � y z¥{U| R y z¥ � | R } ve { C - { e w D -S� e w��W� G ' } N 'QPSRUT8 e � (10)

anda diagonalmatrix ¦ with thedenominatorfrom equa-tion (8) aselements§ 8 e 8 � y z¥{U| R y z¥ � | R } 8

e { C - { e w��W� D -S� e w��W� G ' } N 'QPSRUT8 e � (11)

Assumingstationarity, ¦ is independentof � , sothat � � � � �£ � ��� ¦ PSR . Fromequation(10) it is seenthat£ � � � decom-

posesinto £ � ��� � �©¨ � � � � � N 'QPSRUT (12)

where�ª� N 'QPSRUT is the signednorm applied to eachele-

ment of�ª�

.¨ � � � is the matrix with elements« � � �{ e � �C - { e w D -S� e w��&� G ' , andwenoticethat

¨ � � � is diagonalsincethesourcesareindependent.We next form thequotientmatrix¬ � ��� ,¬ � � � � � � � � � �®­ � PSR � £ � ��� £ �®­ � PSR� �©¨ � ��� � � N '�PWRXT � � N '�PWRXT PWR ¨ �®­ � PSR � PSR� �©¨ � ��� ¨ �®­ � PSR � PSR (13)

We cannow constructthemixing matrix�

by solving theeigenvalueproblem, ¬ � ���°¯ � ¯²± (14)

if we identify

¯ � � and

± � ¨ � � � ¨ �®­ � PWR .In short,we have shown our main result, namelythat

it is possibleto recover the mixing matrix from fractionallow order moments. The sourcesignalsare subsequentlyestimatedby �E� � PSR � .

3.1. Sample estimates of Fractional Low Order Moments

Equation(5) providestheestimateof theelementsof �$� ���³ c d �>� � e d ��� ���_� � R� P � x � PSRaP�w | ­ :$v e w : N(o PSRUT8 e w��W�R��x � PSRw | ­ : 8 e w : N´o PWRXT8 e w (15)

from a sampleof�

measurements.We canthenform³¬ � � � � ³� �� � ³� �®­

� PSR� ��¶µ � �ª� �� � N(o PSRUT �n· �²� N(o PWRXT �¹¸ PSR (16)

hence,solve the eigenvalueproblemto obtainthe estimat-ed mixing matrix. In the above equation

� � ��� is the zeropaddedmatrix

�shifted by a lag � . The choiceof � is

a questiononehave to dealwith. The optimal choicewillboth dependon the accuracy on the estimatedelementsin¨ � ��� and

¨ �®­ � andon theactualtruevaluesin¨ � � � ¨ �º­ � PWR .� shouldbeselectedsowehavethelargestdifferencein the

eigenvaluesof¬ � � � i.e. in

¨ � � � ¨ �®­ � PSR comparedto theiraccuracy. In this paperwe will not addressthis further, butsuggestusing �ª�»@ .

4. EXPERIMENTAL RESULTS

We illustrate the performanceof the generalizationof theMolgedey-Schusteralgorithmby analyzingmixing of syn-theticdatafollowing astabledistribution,andby mixing oftwo speechsignals.

4.1. Synthetic data

We performexperimentson mixing of two signalsin twochannels.In thefirst wemix two independentunit-varianceGaussiansignalsby themixing matrix� �½¼ 1Q� ¾_¿�À¢ÀÁ1M� À¢À. �1Q� Ã� @ ÀÁ1M� ¿ � ¾¢ÄÆÅcontainingtwo non-orthogonalnormalizedrows.Thesourcesignalshave differenttemporaldependencies,oneis whitenoise,while the otheris low-passfiltered with a FIR-filterwith two coefficients Ç ­ �È@ and Ç R � 1M� ¾

. We show inFigure1 scatterplotsof the“true signals”andtherecoveredsignalsfrom the estimatedmixing matrix estimatedfromthe conventionalMolgedey-Schusteralgorithm. Next wemix two similarstablesignalsdrawn from distributionswith� �É@ �(Ä , hencewith heavy tails, by the samemixing ma-trix. In Figure2 we show the scatterplotsof the true andestimatedsignals.

The two different scenarioslead to similar estimatedmixing matricesÊmËÍÌ�Î�Ï Ð Î Ñ0ÒÓÎ_Ï Ò"Ô�Õ"ÖÎ�Ï Ô�Õ_×aÔØÎ_Ï Ð Ù�Ð"ÖÆÚfÛ ÊmËÍÌ�Î�Ï Ð Ñ Ü ÖØÎ_Ï Ñ Ù�Î ÑÎ�Ï Ò�Ô Ö�ÔØÎ_Ï Ð�Î�Ò¢×5Údemonstratingthatit is possiblereliablyto reconstructinde-pendentstablesourcesignalsby thegeneralizedMolgedey-Schusterscheme,in caseswherethesecondmomentcorre-lationswould not convergefor largesamples.

4.2. Speech data

Finally, weconsidertheblind separationof amixtureof twospeechsignals.The signalsaretwo sourcesignalsusedin[13]. We usethesamemixing matrix asin theaboveexam-ple. In Figure3 we show thescatterplotof thetrueandes-timatedsignalsusingthe conventionalMolgedey-Schusteralgorithmandin Figure4 usingthenew schemeusing sE�@ � ¾ . The recoveredsignalsarevery similar, indicatingthattheconventionalalgorithmis robustto theunderlyingdistri-bution of thesourcesignals.We conjecturethatthis robust-nessis relatedto the fact that the conventionalMolgedey-Schusteralgorithmproceedsfrom the ratio of two secondordermoments,andthattheratiomight havebetterstatisti-cal propertiesthanthe two individual moments,this is thetopic for on-goingresearch.

5. CONCLUSION

Wehavederivedanew algorithmgeneralizingthatof Molge-dey andSchuster[5] basedonfractionallow ordermoments.We haveprovideda schemefor estimatingtherelevantmo-mentsfrom finite samplesanddemonstratedtheviability ofthenew algorithmin asyntheticdataset.However, wealsofound that the original algorithmis fairly robust to possi-bly diverging secondordermoments,andconjecturedthattheexplanationcouldberelatedto thefact thattheconven-tionalalgorithmis basedontheratiobetweentwo divergingterms,possiblyimproving thestatisticalproperties.

−10 −5 0 5 10−10

−5

0

5

10

Est

imat

ed S

ourc

e S

1

True Source S1

−10 −5 0 5 10−10

−5

0

5

10

Est

imat

ed S

ourc

e S

2

True Source S1

−10 −5 0 5 10−10

−5

0

5

10

Est

imat

ed S

ourc

e S

1

True Source S2

−10 −5 0 5 10−10

−5

0

5

10

Est

imat

ed S

ourc

e S

2

True Source S2

Fig. 1. Original Gaussiansource,i.e.� ��� signalsagainst

recoveredsourcesignalswhenusingtheoriginalMolgedeyandSchusteralgorithm(s3��� ). In this setup

� �Ý@ 1¢1_1¢1andoneof the sourceswaswhite, the otherfiltered by theFIR-filter in thetext. Thealgorithmwasrunat �ª�»@

−100 −50 0 50 100−100

−50

0

50

100

Est

imat

ed S

ourc

e S

1

True Source S1

−100 −50 0 50 100−100

−50

0

50

100

Est

imat

ed S

ourc

e S

2

True Source S1

−100 −50 0 50 100−100

−50

0

50

100

Est

imat

ed S

ourc

e S

1

True Source S2

−100 −50 0 50 100−100

−50

0

50

100

Est

imat

ed S

ourc

e S

2

True Source S2

Fig. 2. Samesetupasin Fig.1but now thesourcesignalsaregeneratedwith

� �Þ@ �(Ä andthealgorithmis runwith s��Þ@ � À¢¿ .

−1 −0.5 0 0.5 1−1

−0.5

0

0.5

1

Est

imat

ed S

ourc

e S

1

True Source S1

−1 −0.5 0 0.5 1−1

−0.5

0

0.5

1

Est

imat

ed S

ourc

e S

2

True Source S1

−1 −0.5 0 0.5 1−1

−0.5

0

0.5

1

Est

imat

ed S

ourc

e S

1

True Source S2

−1 −0.5 0 0.5 1−1

−0.5

0

0.5

1

Est

imat

ed S

ourc

e S

2

True Source S2

Fig. 3. Original speechsource signals against recov-eredsourcesignalswhenusingthe original Molgedey andSchusteralgorithm. The speechsignalshada durationof� ��� â1_1¢1 . Thealgorithmwasrun at �©� Ä .

6. REFERENCES

[1] Te-Won Lee, IndependentComponentAnalysis: TheoryandApplications, Kluwer AcademicPublishers,New York,1998.

[2] M. Girolami (Ed.), Advancesin IndependentComponentAnalysis., Springer-Verlag,New York, 2000.

[3] P. Comon, “Independentcomponentanalysis:A new con-cept?,” SignalProcessing, vol. 36,pp.287–314,1994.

[4] Anthony J.Bell andTerrenceJ.Sejnowski, “An Information-MaximisationApproachto Blind SeparationandBlind De-

−1 −0.5 0 0.5 1−1

−0.5

0

0.5

1

Est

imat

ed S

ourc

e S

1

True Source S1

−1 −0.5 0 0.5 1−1

−0.5

0

0.5

1

Est

imat

ed S

ourc

e S

2

True Source S1

−1 −0.5 0 0.5 1−1

−0.5

0

0.5

1

Est

imat

ed S

ourc

e S

1

True Source S2

−1 −0.5 0 0.5 1−1

−0.5

0

0.5

1

Est

imat

ed S

ourc

e S

2

True Source S2

Fig. 4. Thesetupis thesameasthat in Fig.3 but with sÆ�@ � ¾convolution,” Neural Computation, vol. 7, no. 6, pp. 1129–1159,1995.

[5] L. Molgedey andH. Schuster, “Separationof independentsignalsusing time-delayedcorrelations,” PhysicalReviewLetters, vol. 72,no.23,pp.3634–3637,1994.

[6] A.Belouchrani, K.Abed-Meraim, J.F.Cardoso, andE.Moulines, “Blind source separationtechniqueusingsecond-orderstatistics,” IEEE Transactionson SignalProcessing, vol. 45,no.2, pp.434–444,1997.

[7] L.K.HansenandJ.Larsen, “Sourceseparationin short im-agesequencesusing delayedcorrelation,” ProceedingsoftheIEEE Nordic SignalProcessingSymposium,Vigsø,Den-mark1998.Eds.P. Dalsgaard andS.H.Jensen, pp.253–256,1998.

[8] L.K.Hansen,J.Larsen,and T.Kolenda, “On independen-t componentanalysisfor multimediasignals,” L. Guan,S.Y.KungandJ. Larsen(eds.)MultimediaImage andVideoPro-cessing, CRCPress, 2000.

[9] H. Attias andC.E.Schreiner, “Blind sourceseparationanddeconvolutionby dynamiccomponentanalysis,” Neural Net-works for Signal ProcessingVII: Proceedingsof the 1997IEEE Workshop,456-465(1997, pp.456–465,1997.

[10] H. Attias andC.E.Schreiner, “Blind SourceSeparationandDeconvolution: TheDynamicComponentAnalysisAlgorit-m,” Neural Computation, vol. 10,pp.1373–1424,1998.

[11] ChrysostomosL. Nikias andMin Shao, SignalProcessingwith Alpha-StableDistribution and Applications, Wiley, 1edition,1995.

[12] P.Kidmose, “Independentcomponentsof stablelinearmix-tures,” 2001,Submittedto ICASSP2001.

[13] T.W.Lee, M.S.Lewicki, M.Girolami, and T.J.Sejnowski,“Blind SourceSeparationof More SourcesThan MixturesUsing OvercompleteRepresentations,” IEEE Signal Pro-cessingLetters, vol. 6, pp.87–90,April 1999.