Verification and Validation of Adaptive Systems

Research Heaven,West Virginia

Verification and Validation of Adaptive Systems

Bojan Cukic, Eddie Fuller, Marcello Napolitano, Harshinder Singh, Tim Menzies,

Srikanth Gururajan, Yan Liu, Sampath Yeramalla

West Virginia University

3

Research Heaven,West VirginiaIntroduction

• System performance evolves over time.– Evolution needed to address complex interactions

between system and the environment.– Improved performance achieved through learning.

• Off-line and on-line adaptation.– Proper reactions to conditions not

envisioned/identified/analyzed by system designers.• NASA interests.

– Autonomous navigation.– Intelligent flight control.

4


Architectures for Adaptive NN-based Control

Physical Process

Feedback ControlAdaptive NN

Reference Model

Command

Error

LearningRule

•DCS•Sigma-Pi•Multilayer Perceptron (SHL)•Radial Basis Function

Desired Response

ActualResponse+

-

5


R e a l-T im e P ID

e s tim a te dd e r iv a tiv e s

d e r iv a tiv ee rro rs

C o n tro lle rp ilo tin p u ts

S e n s o rs

c o n tro lc o m m a n d s

b a s e lin ed e r iv a tiv e s

B a s e lin eN e u ra l N e tw o rk

O n lin eN e u ra lN e tw o rk

d e r iv a tiv ec o rre c t io n s

a n a lo g

NASA IFCS Architecture(Gen 1) New V&V Techniques

6


NASA IFCS Architecture (Gen II)

pilotinputs

Sensors

ModelFollowing

ControlAllocation

Dynamic InversionController

-

Direct AdaptiveNeural Network

+Feedback

Error

ImplementedInARTS II

ImplementedIn SCE 3

New V&V Techniques

7


Our Approach to V&V

Feedback ControlAdaptive NN

LearningRule

Desired Response

Physical Process

Reference Model

Command

Error

ActualResponse

+-

Monitor Inputs to NN,Novelty Detection

Monitor Stabilityof Learning

EstimateTrustworthinessofOutputs

1.2.

3.

8


The Role of Novelty Detection

• Block the penetration of unreliable or unreasonable training data into the online adaptive component.• “Warn” the system about the incoming uncertainty.

• Provide architectural framework for backward

and forward recovery capabilities.

9


Support Vector Machines (SVM)

• Developed by Vapnik et. al., SVM is designed based on the Structural Risk Minimization Theory.

• In pattern recognition, used for efficient clustering in highly dimensional spaces.

• What SVM does? – Maps input space (by means of a nonlinear

transformation) into a high dimensional (hidden) feature space.

10


SVM finds the maximum margin hyperplane in the feature space. This hyperplane maximizes the minimum distance to the

closest training point.

Support Vector Machine (2)

The maximum margin hyperplane is represented as linear combination of training points with non-zero weights (called support vectors).

11


Support Vector Data Description

• Developed by Tax et. al.,

• Finds a sphere with the minimal volume that contains all data points.

• Basically a one class classifier.

12


Support Vector Data Description (2)

13


5 Failure Modes, 3 Pairs of Parameters.Mode 1: Actuator Failure – stuck left stabilator at current position. ( 0 degree )Mode 2: Actuator Failure – stuck left stabilator at pre-defined deflection. (+3 degree)Mode 3: Actuator Failure – stuck left stabilator at pre-defined deflection. (-3 degree)Mode 4: Actuator Failure – stuck right aileron at pre-defined deflection. (+3 degree)Mode 5: Actuator Failure – stuck right aileron at pre-defined deflection. (-3 degree)

Pair 1: ( the pitch rate, the average of the left stabilator and the right stabilator)Pair 2:( the angle of attack, the average of the left stabilator and the right stabilator)Pair 3: ( the pitch altitude, the average of the left stabilator and the right stabilator)

Experiments

14


Data Collection

15


SVDD and IFCS Simulation

16


Novelty Detection for IFCS

Parameterization of SVDD affects the treatment of novelties

17


Monitoring Stability of Learning

• Traditional V&V techniques for a Neural Network code are not applicable.– The system changes following the deployment.

• The goal of the proposed approach:– Prove stability property of the learner. – Monitor convergence towards the stable state of the

learner in real-time. – Provide a time varying measure of reliance that can

be justifiably placed on the adaptive component.• Smells like dependability, doesn’t it?

18


Lyapunov (like) Stability Analysis

• We emphasize the role of Lyapunov-type self-stabilization analysis for V&V.– Think of NN as dynamical system– Lyapunov’s theorem says: find V such that V>0 and V·0 then

system moves towards given solution (stable).– Perturbations tend back to given solutions so variations in data

will still give learnable states.

• DCS Neural Network algorithm is internally self-stabilizing.

19


Lyapunov Theory and V&V

• Built in to this approach is a real-time measure for self-correction of a neural network based adaptive system

20


A Simple Example

• Learning in DCS neural network.

• Two Spirals data set.– Move neurons to

match data.– Connect regions that

represent similar data.– Grow network.

– Monitor self-correction.

21


Stabilization Within DCS NN

• THEOREM:– During DCS network’s learning and representation

from a fixed input manifold, the evolving state of the network due to neural unit’s positional adjustment is self-stabilizing in a globally asymptotically stable manner.

22


Experimental Evaluation

• An F-15 simulator customized for WVU/NASA research needs.

• Input data sets obtained from simulated flights.

• DCSCz network (Mach, altitude, alpha (angle of attack)

WVU F-15 Flight Simulator

23


Experimental Evaluation (2)

• Input data set (blue)

• Network training (red/green)

• Lyapunov monitoring function

24


Summary

• The project is developing fundamentally new V&V technology applicable to a fundamentally different type of systems.

• Critical for future NASA missions (ASAP).– Autonomy and adaptivity anticipated to be critical.

• Multidisciplinary approach in research is a key to success. • Research solves an engineering problem, which

happens to have software implementation.

• This is a systems engineering exercise.

25


Summary (2)

• Separating nominal flight conditions from failure modes by SVDD appears very promising research direction. – Computationally efficient.

• Lyapunov self-stabilization theory can guarantee that the network actually preserves and learns the input feature data manifold – The central V&V issue of IFCS.

• Effective monitoring of performance aspects of the neural network controllers.

26


Further work

• Investigating real-time application of SVDD.• Extending Lyapunov type analysis to variable

input manifolds.– Stochastic stability analysis.

• Continual maintenance of the flight simulation capability.

• Investigating the “output trustworthiness” issue. • Continuing collaborations with NASA (Ames,

Dryden , IV&V), ISR, contractors…• Keeping up with V&V needs of GEN II and beyond.

Verification and Validation of Adaptive Systems

Documents

Transcript of Verification and Validation of Adaptive Systems