Seminar at National Institute of Standards and Technology

ANOMALY DETECTION AND FAILURE MITIGATIONIN COMPLEX DYNAMICAL SYSTEMS

Supported byArmy Research Office

Grant No. DAAD19-01-1-0646

Asok RayThe Pennsylvania State University

University Park, PA 16802Email: axr2@psu.eduTel: (814) 865-6377

November 19, 2004

1 8 5 5

Outline of the Presentation

Anomaly Detection in Complex Dynamical SystemsMicrostate Information Based on Macroscopic Observables

Thermodynamic Formalism of Multi-time-scale NonlinearitiesSymbolic Time Series Analysis of Macroscopic ObservablesPattern Discovery via Information-theoretic Analysis

Real-time Experimental Validation on Laboratory ApparatusesActive Electronic Circuits and Three-phase Electric Induction MotorsMulti-Degree-of-Freedom Mechanical Vibration and Chaotic SystemsFatigue Damage Testing in Polycrystalline Alloys

Discrete Event Supervisory Control for Failure MitigationQuantitative Measure for Language-based Decision and ControlReal-time Identification of Language Measure ParametersRobust and Optimal Control in Language-theoretic Setting

1 8 5 5

Future Collaborative Research in Complex MicrostructuresModeling and Control of Hidden Anomalies and their PropagationExperimentation on Real-time Detection and Mitigation of Malignant

Anomalies on a Hardware-in-the-loop Simulation Test Bed

Intelligent Health Management and Failure Mitigating Control

of Aerial Vehicle Systems

Other Information

Life Extending Control System(including Feedforward Control)

Avionic and StructuralHealth and Usage Monitoring

System (HUMS)

Robust Wide range Gain Scheduling

Feedback Control System

Aircraft Flight and Structural

Dynamics

Conventional and

Special-PurposeSensor Systems

ActuatorDynamics

Analytical Measures(including real-time NDA)

of Damage States and Damage State Derivatives

Anomaly DetectionInformation Fusion

(e.g., FDI, calibration, data fusion,and redundancy management)

..

Mission/Vehicle Management Level(Discrete Event Decision Making)

Flight Management Level(Continuous/Discrete Event Decision Making)

Avionics and Flight Control Level

1 8 5 5

Anomaly Detection and Classification:Symbolic Time Series Analysis1 8 5 5

Multi-Time-Scale Nonlinear DynamicsSlow Time Scale: Anomaly Propagation (Non-stationary Statistics)Fast Time Scale: Process Response (Stationary Statistics)

Model-based Statistical MethodsModeling with Nonlinear Stochastic Differential Equations

Ito Equation:

Fokker Planck Equation:

Uncertainties in Model Identification and Loss of RobustnessStatistical Mechanical Modeling (Canonical Ensemble Approach)

Symbolic Time Series AnalysisSmall perturbation stimulusSelf-excited oscillations

Thermodynamic Formalism and Information Theory

Hidden Markov Modeling (HMM) and Shift Spaces

Notion of Symbolic Dynamics

Discretization of the Dynamical System in Space and Time

Representation of Trajectories as Sequences of Symbols

……φ χ γ η δ α δ χ……Symbol Sequence

Finite State Machine

0

1

φ

2 χ, εγ

3η

δαβ

-1-0.5

00.5

1

-1-0.5

00.5

10

2

4

6

8

10

η

αβ

γ

δχ

φε

Phase Trajectory

1 8 5 5

State Machine ConstructionD-Markov Machine

Ν∈Ν∈ U}0{

110 ( )k D iD

ii w A − −−=∑

kD

kkk wwwW 110 ... −=

kD

kk www 110 .......... −

11

11

10 ........ +

−++ k

Dkk www

11

+−

kDwkw0

11

11

10

1 ... +−

+++ = kD

kkk wwwW

( )1 10 1

k k D kDW w A A w− +

−− +

InOut

State to state transitions from a symbol sequencealphabet size = |A| window size = D kth word (state)=kth word value =

(k+1)th word =(k+1)th word value =

p00 1-p00 0 0

0 0 p01 1-p01

p10 1-p10 0 0

0 0 p11 1-p11

00

10

01

11

0

0

0

0

1

1

1

1

|A|=2; D=2; AD = 4

Example1-D Ising (Spin-1/2) Model

Nearest Neighbor Interactions

1 8 5 5

Computationally efficient for anomaly measure

Fixed depth D and alphabet size A

Only the state transition probabilities to be determinedbased on symbol strings derived from time series data

or wavelet-transformed data

States represented by an equivalence class of stringswhose last D strings are identical

State Space Constructionvia D-Markov Machine1 8 5 5

Anomaly MeasureBased on the D-Markov Machine

State Transition Matrix ConstructionBanded structureSeparation into irreducible subsystemsStationary state probability vectorInformation on the dynamical system characteristics

Chaotic motion, period doubling, and bifurcation

State Probability VectorReference Point: Nominal Condition p(το )Epochs {τk} of Slow Time Scale {p(τk)}

Anomaly Measure at Slow-Time EpochsM(τk; το) = d(p(τk), p(το))

1 8 5 5

Comparison ofEpsilon Machine and D-Markov Machine

Epsilon Machine [Santa Fe Institute]A priori unknown machine structureOptimal prediction of the symbol processMaximization of mutual information

(i.e., minimization of conditional entropy)I[X;Y] = H[X] – H[X|Y]

Analogous to the class of Sofic Shifts in Shift Spaces

D-Markov MachineA priori known machine structure

(Fixed order fixed structure with given |A| and D)Excess states yielding redundant reducible matrices

(Perron-Frobenius Theorem)Suboptimal prediction of the symbol processAnalogous to the class of Finite-type Shifts in Shift Spaces

1 8 5 5

Anomaly Detection Procedure

Forward (Analysis) Problem:Characterization of system dynamical behavior

Parametric and non-parametric anomalies

Evolution of the grammar in the system dynamicsRepresentation of dynamical behavior as formal languagesThermodynamic formalism of anomaly measure

Inverse (Synthesis) ProblemEstimation of feasible ranges of anomalies

Fusion of information generated from responses under several stimuli chosen in the forward problem

1 8 5 5

00011000110101…

Signal Conditioning

Time Series Data Current, Voltage or other Signals

Sampling and Quantization; Denoising, and Decimation

Wavelet Transform

00 01

11 10

10

0

1

0

1 0

1

Partitioning of Wavelet Coefficients

D-Markov Machine HMM Construction

Real-time Analytical Prediction

Dynamical System

Pre-Processing

SymbolGeneration

PatternRepresentation

AnomalyDetection

Summary of Anomaly Detection Procedure1 8 5 5

Anomaly Detection and ClassificationSignal Conditioning and Decimation

DenoisingEmbedding

Symbol Sequence GenerationPhase space partitioningWavelet space partitioning

Markov Modeling of Symbol DynamicsEpsilon machine (sofic shift)D-Markov machine (finite type shift)

Thermodynamic Formalism of Generated Information

Externally Stimulated Duffing Equationwith a single slowly varying parametric anomaly

1 8 5 5

Slowly Varying Damping

Nonlinear Spring

Mass

Externally Applied Force

Ttyty )]()([ &

)(ω tCosA

Random Initial Conditions)0()]()([ δBtyty T

oo ∈&

stt fast time

slow time

Governing Equations:

),[ ∞∈ ott

),[ ∞∈ ott)(

)()()( 3)()(2

2

ω

αθdt

tdys

dttyd

tCosA

tytyt

=

+++

Parameters:1; 0.01; 22; 5.0Aα δ ω= = = =

Externally Stimulated Duffing Equation

Electromechanical Systems LaboratoryAnomaly Detection Apparatus for

Hybrid Electronic Circuits

Computer and Data Acquisition System

A Sin(ωt)

1 8 5 5

Phase-plane Plots under Nominaland Anomalous Conditions

-4.0-3.0-2.0-1.00.01.02.03.04.0

Phas

e V

aria

ble

dy/

dt

-2 -1.5 -1 -0.5 0.0 0.5 1 1.5 2 2.5

Phase Variable y

θ= 0.10

-4.0-3.0-2.0-1.00.01.02.03.04.0

Phas

e V

aria

ble

dy/

dt

-2 -1.5 -1 -0.5 0.0 0.5 1 1.5 2 2.5

Phase Variable y

θ= 0.27

Phas

e V

aria

ble

dy/

dt

Phase Variable y

-4.0

-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

4.0

-2 -1.5 -1 -0.5 0.0 0.5 1 1.5 2 2.5

θ= 0.28

Phase Variable y

-2.5-2.0-1.5-1.0-0.50.00.51.01.52.02.5

Phas

e V

aria

ble

dy/

dt

-1 -0.5 0.0 0.5 1 1.5

θ=0.29

Time0 200 400 600 800 1000 1200 1400 1600 1800 2000-2.0

-1.5-1.0-0.50.00.51.01.52.02.5

Greenθ = 0.27

Redθ = 0.28

Blackθ = 0.29Ph

ase

Varia

ble

Blueθ = 0.10

Electromechanical Systems Laboratory1 8 5 5

Construction of Finite State Machines

0.70790.00300.00350.2856

θ = 0.2800 01

10 11

0/0.9986 1/0.0014

0/0.

251 1/0.749

0/0.5 1/0.

5

0/0.0070

1/0.

9930

S=0.6397

Interpretation usingThermodynamic Formalism

|A|=2; D=2; AD = 4

0/0.0048

00 01

10 11

0/0.9987 1/0.0013

0/0.51/0.5

0/0.51/0.5

1/0.9952

0.78890.00200.00200.2071

θ = 0.10

S=0.5380 0/0.0070

θ = 0.27

0.76590.00250.00250.2291

1/0.9930

00 01

10 11

0/0.9987 1/0.0013

0/0.

33

1/0.67

0/0.50 1/0.

50

S=0.5718

11

1/1.0

θ> = 0.29

0.00.00.01.0

S=0.0

Ideal Monatomic GasConstant V, N

Energy E

Ent

ropy

S

X

X

X

X XEmin

? ?0

θ−1 ~ partialderivative of Swrt E

1 8 5 5

Electronic Circuit Apparatus

0.10 0.15 0.20 0.25 0.30 0.350.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

Parameter θ

Nor

mal

ized

Ano

mal

y M

easu

re

MLPNNRBFNN

D-Markov SFNND-Markov WS

PCA

MLPNNRBFNN

D-Markov SFNND-Markov WS

PCA

Sensitivity of the Detection Algorithmto the Anomalous Parameter θ

1 8 5 5

Electromechanical Systems LaboratoryAnomaly Detection Apparatus for

Mechanical Vibration Systems

0 4 8 12 16 20 24 28 320.00.10.20.30.40.50.60.70.80.91.0

Wavelet Space (WS) Partitioning

Principal Component Analysis (PCA)

Symbolic False Nearest Neighbor (SFNN) Partitioning

Radial Basis Function Neural Network (RBFNN)

Multilayer Perceptron Neural Network (MLPNN)

Nor

mal

ized

Ano

mal

y M

easu

re

Time in minutes

1 8 5 5

1 2 3 4 5 6 7 8

x 10 4

0.00.10.20.30.40.50.60.70.80.91.0

Number of Fatigue Cycles

Nor

mal

ized

Ano

mal

y M

easu

re

D-Markov WSD-Markov SFNNMLPNNRBFNNPCA

First CrackDetection onMicroscope

Electromechanical Systems LaboratoryFatigue Test Apparatus for

Damage Sensing in Ductile Alloys1 8 5 5

Anomaly Detection Symbolic Time Series Analysis

AdvantagesFoundations on fundamental principles of physics and mathematicsQuantitative measure as opposed to qualitative measureRobustness to measurement noise and spurious signal distortionSensitivity to signal distortion due to nonlinearity and nonstationarityAdaptability to low-resolution sensingApplicability to real-time anomaly detection

(Near-term) DisadvantagesRequirement for advanced knowledge to understand the basics Need for much theoretical and experimental researchSeemingly counter-intuitive to inadequately trained technical personnel

1 8 5 5

A. Ray, “Symbolic Dynamic Analysis of Complex Systems for Anomaly Detection,” Signal Processing, Vol. 84, No. 7, July 2004, pp. 1115-1130.

Quantitative Measure for Discrete Event

Supervisory Control

1 8 5 5

Discrete Event Supervisory (DES)Control of Continuous Plants

External CommandsTreated asUncontrollableEvents

Real-TimeContinuously

Varying Plant

Event Generator

(C/D)

Discrete Event Model of the

Continuously Varying Plant(Treated as State Estimator)

Other information

Supervisor

CommandGenerator

(D/C)

Sensor data

DisablingControllable

eventsObservable

EventsGenerated

Events

1 8 5 5

Modeling of Discrete EventSupervisory Control Systems

f(x, u)f(x, u) ∫∫ g(x, u)g(x, u)

h(x, u)h(x, u)

f(x, u)f(x, u) ∫∫ g(x, u)g(x, u)

h(x, u)h(x, u)

fi(x, u)fi(x, u) ∫∫ g(x, u)g(x, u)

h(x, u)h(x, u)

Continuously Varying Systems Discrete Event Systems Hybrid Systems

f(x, u)fi(x, u)

Decoupling: continuous evolutions and discrete transitions

Coupling: continuous evolutions and discrete transitions

1 8 5 5

Behavior-based Robotic System DES Control Architecture

Precision

IntelligenceDES control

Event Detection

Continuous-time operational control

ReferenceGenerationContinuous time

Discrete Event

Pioneer 2 AT Stage Simulator

interchangeable

1 8 5 5

Signed Real Measure of Regular LanguagesSalient Features

Language Metric |µ|Total variation of the signed real measure (Real positive) distance between two languages

Applications of the Language Measure for Failure MitigationRobust and optimal control of discrete-event systemsAnomaly quantification, classification, and mitigation

Vector Space of Formal LanguagesInfinite-dimensional spaceGalois field GF(2)Vector addition operator - Exclusive-OR

Quantitative Measure of *-LanguagesFinite alphabet Σ Σ∗ cardinality N0

Language Measure µ: 2Σ* R

1 8 5 5

State-Based Partitioning

The set Qm of marked states is partitioned as:

By using

Myhill-Nerode Theorem

Hahn Decomposition Theorem

where is the set of good marked states (positive measure)

is the set of bad marked states (negative measure)

∅== −+−+mmmmm QQ;QQQ IU

+mQ−mQ

1 8 5 5

Lemma: The regular expressions

can be expressed by the following set of symbolic equations:( ), {1,2, , }L LG i I ni i≡ ∈ ≡ L

I∈∀+∑= iLL ij

jjii ϑσ where

⎩⎨⎧

∅∈

=otherwise

Qqif mii

εϑ

Theorem: The language measure of the regular expressions is given by the unique solution of the following

set of algebraic equations: I∈∀χ+µ∑ π=µ ijjj

iji

}n,,2,1{i,Li L∈

In vector notation, the system has a unique solution:Χ+µΠ=µ

1]I[ −Π−Remark: exists and is bounded above by ∞Π/1

Main Result:Signed Real Measure of Regular Languages

µ=[Ι−Π] −1 Χ

A. Ray, V. V. Phoha and S. Phoha, QUANTITATIVE MEASURE FORDISCRETE EVENT SUPERVISORY CONTROL: Theory and Applications, Springer, New York, 2004. ISBN 0-387-02108-6

1 8 5 5

Algorithm for the Unconstrained Optimal Control Law

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

=Π

nnnn

n

n

02

01

0

20

220

210

10

120

110

0

πππ

ππππππ

L

MOMM

L

L

( ) χ

µ

µµ

µ10

0

02

01

0 −Π−=

⎟⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜⎜

⎝

⎛

= I

n

M

Given the information on the plant model Gi = (Q, Σ , δ ,qi , Qm)

along with the state transition cost matrix Πand characteristic vector χ , the unconstrained optimal control maximizes the language measure by deleting someof the “bad” strings so that optimality of the supervised plant sublanguage is achieved

1 8 5 5

Supporting Theorems

Theorem #1 (Monotonicity) : Disabling the controllable events leading to states with negative (positive) performance does decrease (increase) supervised plant performance.

Theorem #2 (Monotonicity) : Enabling the controllable event(s) leading to states with non-negative performance does not decrease the performance for any state.

Theorem #3 (Global Performance): The controller at the termination of the algorithm is the global optimal controller in terms of supervised plant performance.

Theorem #4 (Computational Complexity): The optimal control law is solved in at most n steps and each step requires a solution of n linear algebraic equations where n is the number of states of the plant model. Therefore, the computational complexity for synthesis of the optimal control algorithm of a polynomial order in n.

A. Ray, J. Fu and C.M. Lagoa, "Optimal Supervisory Control of Finite State Automata," International Journal of Control, Vol. 77, No. 12, August 2004, pp. 1083-1100.

1 8 5 5

Language-Measure-Theoretic Discrete-Event Supervisory Control

AdvantagesFoundations on principles of automata theory and functional analysisQuantitative measure as opposed to qualitative measure Robustness to measurement noise and spurious signal distortionCapability for emulation of human reasoning in a quantitative wayAdaptability to low-resolution sensingApplicability to real-time decision-making at multiple time scales

DisadvantagesPotential source of instability under switching actionsNeed for much theoretical and experimental researchRequirement for advanced knowledge to understand the basics

1 8 5 5

A. Ray, V. V. Phoha and S. Phoha, QUANTITATIVE MEASURE FORDISCRETE EVENT SUPERVISORY CONTROL: Theory and Applications, Springer, New York, 2004. ISBN 0-387-02108-6

1 8 5 5

Future Collaborative Research in Complex Microstructures

1 8 5 5

Modeling and Control of Hidden Anomalies and their Propagation

Problem DefinitionLet the time series of (macroscopic) measurement(s) θ be available.The first problem is to estimate the unobservable performance parameter(s) β

(e.g., damage states and derivatives). The second problem is to control the microstates via manipulation of macroscopically

controllable inputs u to satisfy desired performance specifications p.

Proposed Solution Construction of a canonical-ensemble model with the state probability vector π(θ, u) of

the unobservable phenomena that are macroscopically controlled by inputs u through usage of the time series data and a microstructural model, such as the OOF of NIST.

Formulation of constitutive equations for the unobservable parameters β(π, u) that areindicator(s) of the internal microstates and control laws u(βd , πd, p) to satisfy desired performance specifications (e.g., remaining service life and reliability)

Experimentation for Real-time Detection and Mitigation of Malignant Anomalies1 8 5 5

Experimental Validation of the Novel Constitutive RelationsSpecial-purpose Fatigue Testing Machine at Penn StateObject Oriented Finite-element (OOF) Modeling Package at NIST

Experimental Validation of the Supervisory Control ConceptSpecial-purpose Multi-degree-of-freedom Machine at Penn StateControl of the unobservable parameters, such as plastic zone size,

that are indicator(s) of the internal microstatesDiscrete Event Supervisory Control at the Upper level

Derived parameter(s) β (e.g., damage states and their derivatives) to provide the input event sequence to the supervisory control module at the upper levelSupervisor command(s) to provide the control inputs u, such as shaft torque,

to the test apparatus to satisfy the desired performance specifications p

1 8 5 5

Questions & Suggestions