Frequency Domain Causality Analysis Method for Multivariate Systems
15
Frequency Domain Causality Analysis Method for Multivariate Systems in Hypothesis Testing Framework Hao Ye Department of Automation, Tsinghua University
description
Frequency Domain Causality Analysis Method for Multivariate Systems in Hypothesis Testing Framework. Hao Ye Department of Automation, Tsinghua University. Outline. A Brief Introduction to Causal Analysis Two Problems of PDC - PowerPoint PPT Presentation
Transcript of Frequency Domain Causality Analysis Method for Multivariate Systems
Frequency Domain Causality Analysis Method
for Multivariate Systems
in Hypothesis Testing Framework
Hao YeDepartment of Automation, Tsinghua University
OutlineOutline
A Brief Introduction to Causal AnalysisA Brief Introduction to Causal Analysis Two Problems of PDCTwo Problems of PDC Frequency Domain Causal Analysis Frequency Domain Causal Analysis Methods Based on Methods Based on
Two StatisticsTwo Statistics Simulation ExamplesSimulation Examples Concluding RemarksConcluding Remarks
A Brief Introduction to Causal AnalysisA Brief Introduction to Causal Analysis Causal relationship among the time series of an industrial process is of great use Causal relationship among the time series of an industrial process is of great use
for fault detection, alarm management, synthesis design, and modeling, etc.for fault detection, alarm management, synthesis design, and modeling, etc. But it is often complicated and unknown in case of lack of a perfect knowledge But it is often complicated and unknown in case of lack of a perfect knowledge
of system structureof system structure
CausalityAnalysismethods
CausalityAnalysismethods
Time domainTime
domain
Frequency domain
Frequency domain
Wiener(1956): Concept based on dataGranger(1963): Granger causalityGeweke(1982): Conditional Granger causality
Kaminski(1991): Frequency domain Granger causalityBaccala(2001) ： Partial directed coherence (PDC)
Wide application Some issues still need further discussions
, ,
1 11 1 1 1
1 1
ˆ ˆ ˆ( ) ( ) ( ) ... ( ) ( ) ( )
........
ˆ ˆ ˆ( ) ( ) ( ) ... ( ) ( ) ( )
n nr r
n n nn n nr r
x k a r x k r a r x k r e k
x k a r x k r a r x k r e k
2
1
( )PDC (causality :
(
|
)
) |
ijij n
iji
A
A
j i
( ) Threshold is influenced by ij i jxx
2 2
1
normalization constrain( ) follows the 0 | ( ) | 1 and | 1t ) |: (n
ij ij iji
A Brief Introduction to Causal AnalysisA Brief Introduction to Causal Analysis
( ) ( ) ( ) A X E
ˆ ˆ( ) ( ) , ( ) 1 ( )j r j rij ij ii ii
r r
A a r e A a r e
Two Problems of PDCTwo Problems of PDC
, ,
2
1
( )PDC (causality :
(
|
)
) |
ijij n
iji
A
A
j i
2 2
1
0 |normalizat ( ) |ion : 1, | ( ) | 1n
ij iji
Baccala and Sameshima (2001) : PDC has the ability to rank the relative interaction strength with respect to a given signal source because of the normalization.
1 1 3 3 1
2 2 5 2 5 2
3 1 3 5 3 5 3
4 1 2
( ) 0.6 ( 1) 0.4 ( 1) 0.2 ( 2) ( )
( ) 0.6 ( 1) 0.3 ( 1) 0.1 ( 2) 0.4 ( 2) ( )
( ) 0.3 ( 1) 0.45 ( 1) 0.3 ( 1) 0.24 ( 2) 0.2 ( 2) ( )
( ) 0.2 ( 1) 0.3 ( 1) 0.4
x k x k x k x k e k
x k x k x k x k x k e k
x k x k x k x k x k x k e k
x k x k x k x
4 1 5 4
5 2 5 5 5
( 1) 0.2 ( 2)+0.1 ( 1) ( )
( ) 0.4 ( 1) 0.5 ( 1) 0.15 ( 2) ( )
k x k x k e k
x k x k x k x k e k
PDC 1 0( ) | ( ) |dj i ijS x x
，
2
PDC 2 0( ) | ( ) | dj i ijS x x
，
PDC 1 1 3 0.8537( )S x x ， PDC 2 1 3 0.2699( )S x x ，
PDC 1 1 4 0.8598( )S x x ， PDC 2 1 4 0.3490( )S x x ，
1 3=0.2099x xF
1 4=0.1088x xF
Reference: (Barrett and Seth, 2009): Granger causality can measure the strength
Problem 1: PDC cannot correctly rank the causal strength
Two Problems of PDCTwo Problems of PDC
2
1
( )PDC (causality :
(
|
)
) |
ijij n
iji
A
A
j i
2 2
1
0 |normalizat ( ) |ion : 1, | ( ) | 1n
ij iji
Few conclusions about what information the distribution of PDC in frequency domain can further offer. It is natural to guess that it represent how the strength of causal influence changes with ω.
1 1 3 1
2 2 5 2
3 1 3 5 3
4 1 2 4 4
5 2 5 5
( ) 0.6 ( 1) 0.4 ( 1) ( )
( ) 0.6 ( 1) 0.3 ( 1) ( )
( ) 0.3 ( 1) 0.45 ( 1) 0.3 ( 1) ( )
( ) 0.2 ( 1) 0.3 ( 1) 0.4 ( 1) ( )
( ) 0.4 ( 1) 0.5 ( 1) ( )
x k x k x k e k
x k x k x k e k
x k x k x k x k e k
x k x k x k x k e k
x k x k x k e k
1 1 3 1
2 2 5 2
3 1 3 5 3
4 1 2 4 4
5 2 5 5
( ) 0.6 ( 1) 0.4 ( 1) ( )
( ) 0.6 ( 1) 0.3 ( 1) ( )
( ) 0.3 ( 1) 0.45 ( 1) 0.3 ( 1) ( )
( ) 0.2 ( 1) 0.3 ( 1) 0.4 ( 1) ( )
( ) 0.4 ( 1) 0.5 ( 1) ( )
x k x k x k e k
x k x k x k e k
x k x k x k x k e k
x k x k x k x k e k
x k x k x k e k
0
0.5
1
π 0
0.5
1
π 0
0.5
1
π 0
0.5
1
π
0
0.5
1
π 0
0.5
1
π 0
0.5
1
π 0
0.5
1
π
0
0.5
1
π 0
0.5
1
π 0
0.5
1
π 0
0.5
1
π
0
0.5
1
π 0
0.5
1
π 0
0.5
1
π 0
0.5
1
π
0
0.5
1
π 0
0.5
1
π 0
0.5
1
π 0
0.5
1
π
1
2
3
4
5
0
0.5
1
π 0
0.5
1
π 0
0.5
1
π 0
0.5
1
π
0
0.5
1
π 0
0.5
1
π 0
0.5
1
π 0
0.5
1
π
0
0.5
1
π 0
0.5
1
π 0
0.5
1
π 0
0.5
1
π
0
0.5
1
π 0
0.5
1
π 0
0.5
1
π 0
0.5
1
π
0
0.5
1
π 0
0.5
1
π 0
0.5
1
π 0
0.5
1
π
1
2
3
4
5
5( )i
monotonically decreases as ω grows
monotonically increases as ω grows
Can be extended to a general first order multivariate system
Problem 2: PDC cannot describe how the causal strength changes with frequency
Only affects the dynamics of x5
Does not affect the qualitative causal relationship
Frequency domain Causal Analysis Frequency domain Causal Analysis Methods based on Methods based on Two StatisticsTwo Statistics
Schelter et al. (2009): Renormalized PDC (RPDC)Schelter et al. (2009): Renormalized PDC (RPDC)
Schelter et al. (2009): Statistical property of RPDCSchelter et al. (2009): Statistical property of RPDC
Schelter et al. (2009): Detection ruleSchelter et al. (2009): Detection rule
1causality : ˆ ˆ ˆ ˆRPDC ( )= ( ) ' ( ( ) )ij ij ij ijNj i X V X
ˆRe( ( ))ˆ ( )=
ˆIm( ( ))
ij
ij
ij
A
A
X, 1
cos( )cos( ) cos( )sin( )( )= ( , )
sin( )cos( ) sin( )sin( )
p
ij jj iit l
t l t lH t l
t l t l
V
2Under the null hypothe | (sis ) | f 0,o ijA
1~ covariance matrix of , = , ~ covariance matrix of , ~ data lengthe H R R x N
2 22 1 , ( ) or distributionij
0
2ˆ ( ) There exists causal influence from to at (hypothesis test)ij j ix x
How to measure the strength of causality from xj to xi at ω was not discussed
Baccala and Sameshima (2001) : Due to the normalization, PDC(xj→xi) may change if more (or less) signals are influenced by xj, reflects the relative rather than the absolute strength of influence
1causality : ˆ ˆ ˆ ˆRPDC ( )= ( ) ' ( ( ) )ij ij ij ijNj i X V X2Under the null hypothe | (sis ) | f 0,o ijA 2 2
2 1 , ( ) or distributionij
0
2 1
ˆ ( )ij 2ˆ ( )ij
Smaller probability under null hypothesis
Greater probability under null hypothesis
Stronger causal strength
Weaker causal strength
RPDC can be directly used to measure the strength of causality from xj to xi at ω
measure the strength of the causality between each pair of time series
PRDC 0
1 ˆ( )= ( )d2j i ijS x x
Frequency domain Causal Analysis Frequency domain Causal Analysis Methods based on Methods based on Two StatisticsTwo Statistics
Schelter et al. (2005):Schelter et al. (2005):
Statistical propertyStatistical property
2ˆ ˆˆ ( )= | ( ) | / ( )ij ij ijN A C
, 1
( ) [ ( , )(cos( )cos( ) sin( )sin( ))]p
ij ii jjt l
C H t l t l t l
2Under the null hypothe | (sis ) | f 0,o ijA
2 21
ˆ ˆ , | ( ) | / ( ) distributionij ijN A C
Measure the strength of causality from xj to xi at ω
0
1ˆ( )= ( )d
2j i ijS x x
Measure the strength (and the existence) of the causality between each pair of time series
Frequency domain Causal Analysis Frequency domain Causal Analysis Methods based on Methods based on Two StatisticsTwo Statistics (Schelter et al.,2005) : To solve
the over fitting problems in model estimation
Lower computation load compared with RPDC
Simulation ExamplesSimulation Examples
2
1
( )PDC: ( )
| ( )
|
ijij n
iji
A
A
PDC 1 0( ) | ( ) |dj i ijS x x
，
2
PDC 2 0( ) | ( ) | dj i ijS x x
，
1ˆ ˆ ˆ ˆRPDC: ( )= ( ) ) ( ) ' ( ij ij ij ijN X V X PRDC 0
1 ˆ( )= ( )d2j i ijS x x
2ˆ ˆˆ-statistics : ( )= | ( ) | / ( )ij ij ijN A C 0
1ˆ( )= ( )d
2j i ijS x x
Simulation ExamplesSimulation Examples
, ,
1 1 3 3 1
2 2 5 2 5 2
3 1 3 5 3 5 3
4 1 2
( ) 0.6 ( 1) 0.4 ( 1) 0.2 ( 2) ( )
( ) 0.6 ( 1) 0.3 ( 1) 0.1 ( 2) 0.4 ( 2) ( )
( ) 0.3 ( 1) 0.45 ( 1) 0.3 ( 1) 0.24 ( 2) 0.2 ( 2) ( )
( ) 0.2 ( 1) 0.3 ( 1) 0.4
x k x k x k x k e k
x k x k x k x k x k e k
x k x k x k x k x k x k e k
x k x k x k x
4 1 5 4
5 2 5 5 5
( 1) 0.2 ( 2)+0.1 ( 1) ( )
( ) 0.4 ( 1) 0.5 ( 1) 0.15 ( 2) ( )
k x k x k e k
x k x k x k x k e k
PDC 1 1 3 0.8537( )S x x ， PDC 2 1 3 0.2699( )S x x ，
PDC 1 1 4 0.8598( )S x x ， PDC 2 1 4 0.3490( )S x x ，
1 3=0.2099x xF
1 4=0.1088x xF
0.20020.3932
0.2099 0.30560.1088 0.1175 0.0998
0.1951
Granger causality
24.501239.2280
27.7010 32.094216.2684 19.6765 15.7450
22.9304
SPRDC
33.084943.8977
38.8698 39.501122.1023 26.9955 19.9758
32.8947
Sϒ
The calculated strengths based on SPRDC
and Sϒ are consistent with those given by Granger causality
1 1 3 1
2 2 5 2
3 1 3 5 3
4 1 2 4 4
5 2 5 5
( ) 0.6 ( 1) 0.4 ( 1) ( )
( ) 0.6 ( 1) 0.3 ( 1) ( )
( ) 0.3 ( 1) 0.45 ( 1) 0.3 ( 1) ( )
( ) 0.2 ( 1) 0.3 ( 1) 0.4 ( 1) ( )
( ) 0.4 ( 1) 0.5 ( 1) ( )
x k x k x k e k
x k x k x k e k
x k x k x k x k e k
x k x k x k x k e k
x k x k x k e k
1 1 3 1
2 2 5 2
3 1 3 5 3
4 1 2 4 4
5 2 5 5
( ) 0.6 ( 1) 0.4 ( 1) ( )
( ) 0.6 ( 1) 0.3 ( 1) ( )
( ) 0.3 ( 1) 0.45 ( 1) 0.3 ( 1) ( )
( ) 0.2 ( 1) 0.3 ( 1) 0.4 ( 1) ( )
( ) 0.4 ( 1) 0.5 ( 1) ( )
x k x k x k e k
x k x k x k e k
x k x k x k x k e k
x k x k x k x k e k
x k x k x k e k
0
0.5
1
π 0
0.5
1
π 0
0.5
1
π 0
0.5
1
π
0
0.5
1
π 0
0.5
1
π 0
0.5
1
π 0
0.5
1
π
0
0.5
1
π 0
0.5
1
π 0
0.5
1
π 0
0.5
1
π
0
0.5
1
π 0
0.5
1
π 0
0.5
1
π 0
0.5
1
π
0
0.5
1
π 0
0.5
1
π 0
0.5
1
π 0
0.5
1
π
1
2
3
4
5
0
0.5
1
π 0
0.5
1
π 0
0.5
1
π 0
0.5
1
π
0
0.5
1
π 0
0.5
1
π 0
0.5
1
π 0
0.5
1
π
0
0.5
1
π 0
0.5
1
π 0
0.5
1
π 0
0.5
1
π
0
0.5
1
π 0
0.5
1
π 0
0.5
1
π 0
0.5
1
π
0
0.5
1
π 0
0.5
1
π 0
0.5
1
π 0
0.5
1
π
1
2
3
4
5
5( )i
RPDC -statistics
RPDC -statistics
PDC
Simulation ExamplesSimulation Examples
PDC
monotonically decreases as ω grows
monotonically increases as ω grows
0
50
100
π 0
50
100
π 0
50
100
π 0
50
100
π
0
50
100
π 0
50
100
π 0
50
100
π 0
50
100
π
0
50
100
π 0
50
100
π 0
50
100
π 0
50
100
π
0
50
100
π 0
50
100
π 0
50
100
π 0
50
100
π
0
50
100
π 0
50
100
π 0
50
100
π 0
50
100
π
1
2
3
4
5
0
50
100
π 0
50
100
π 0
50
100
π 0
50
100
π
0
50
100
π 0
50
100
π 0
50
100
π 0
50
100
π
0
50
100
π 0
50
100
π 0
50
100
π 0
50
100
π
0
50
100
π 0
50
100
π 0
50
100
π 0
50
100
π
0
50
100
π 0
50
100
π 0
50
100
π 0
50
100
π
1
2
3
4
5The distributions of RPDC or of these two processes are roughly similar as expected
0
50
100
π 0
50
100
π 0
50
100
π 0
50
100
π
0
50
100
π 0
50
100
π 0
50
100
π 0
50
100
π
0
50
100
π 0
50
100
π 0
50
100
π 0
50
100
π
0
50
100
π 0
50
100
π 0
50
100
π 0
50
100
π
0
50
100
π 0
50
100
π 0
50
100
π 0
50
100
π
1
2
3
4
5
0
50
100
π 0
50
100
π 0
50
100
π 0
50
100
π
0
50
100
π 0
50
100
π 0
50
100
π 0
50
100
π
0
50
100
π 0
50
100
π 0
50
100
π 0
50
100
π
0
50
100
π 0
50
100
π 0
50
100
π 0
50
100
π
0
50
100
π 0
50
100
π 0
50
100
π 0
50
100
π
1
2
3
4
5
1 1 1 2 2 1
2 2 2 2
( ) 0.3 ( 1) 0.15 ( 2) 0.4 ( 1) 0.2 ( 2) ( )
( ) 0.2 ( 1) 0.2 ( 2) ( )
x k x k x k x k x k e k
x k x k x k e k
1 1 1 2 2 1
2 2 2 2
( ) 0.3 ( 1) 0.15 ( 2) 0.4 ( 1) 0.2 ( 2) ( )
( ) 0.2 ( 1) 0.2 ( 2) ( ) sin( )
x k x k x k x k x k e k
x k x k x k e k k
0
20
40
60
80
100
π
0
50
100
π
1
2
0
20
40
60
80
100
π
0
20
40
60
80
100
π
1
20.40.40 10 20 30 40 50 60 70 80 90 100
0.16
0.18
0.2
0.22
0.24
0.26
0.28
0.3
0.32
Frequency ω/0.01 π (0 to π )
Granger causality
0.4
The distribution of Granger causality in the frequency domain is consistent with those given by PRDC and ϒ-statistics
Simulation ExamplesSimulation Examples
RPDC -statistics
Concluding RemarksConcluding Remarks
2
1
( )PDC: ( )
| ( )
|
ijij n
iji
A
A
PDC 1 0( ) | ( ) |dj i ijS x x
，
2
PDC 2 0( ) | ( ) | dj i ijS x x
，
1ˆ ˆ ˆ ˆRPDC: ( ) ( ) ' ( ) ( ) ij ij ij ijN X V X PRDC 0
1 ˆ( ) ( )d2j i ijS x x
2ˆ ˆˆ-statistics : ( ) | ( ) | / ( )ij ij ijN A C 0
1ˆ( ) ( )d
2j i ijS x x
Discussion of Problem 1: PDC cannot correctly rank the causal strength (contribution)
Discussion of Problem 2: PDC cannot describe how the causal strength changes with frequency (contribution )
Baccala and Sameshima (2001) : PDC reflects the relative rather than the absolute strength of influence
Solve the two problems respectively (clear physical meaning in the hypothesis testing framework, simulation examples )
Complex computation
(Schelter et al.,2005) : To solve the over fitting problems in model estimation
Similar advantages to PRDC Simpler computation
Concluding RemarksConcluding Remarks
1.1. Zhang J, Yang F, Ye H. Frequency domain causality analysis method for Zhang J, Yang F, Ye H. Frequency domain causality analysis method for multivariate systems in hypothesis testing framework. The 19multivariate systems in hypothesis testing framework. The 19 thth IFAC World IFAC World Progress, 2014Progress, 2014
