Cyber-Physical V&V Challenges for the Evaluation of State ... · Cyber-Physical V&V Challenges for...
Transcript of Cyber-Physical V&V Challenges for the Evaluation of State ... · Cyber-Physical V&V Challenges for...
![Page 1: Cyber-Physical V&V Challenges for the Evaluation of State ... · Cyber-Physical V&V Challenges for the Evaluation of State of the Art Model Checkers Research in Quantum Enabled V&V](https://reader030.fdocuments.net/reader030/viewer/2022013104/5c847a3309d3f291698d055b/html5/thumbnails/1.jpg)
Copyright 2016, Lockheed Martin Corporation. All rights reserved.
Cyber-Physical V&V Challenges for the Evaluation of State of the Art Model CheckersResearch in Quantum Enabled V&V Technology
July 12-14, 2016
Chris ElliottFlight Controls / Quantum Computing
![Page 2: Cyber-Physical V&V Challenges for the Evaluation of State ... · Cyber-Physical V&V Challenges for the Evaluation of State of the Art Model Checkers Research in Quantum Enabled V&V](https://reader030.fdocuments.net/reader030/viewer/2022013104/5c847a3309d3f291698d055b/html5/thumbnails/2.jpg)
2Copyright 2016, Lockheed Martin Corporation. All rights reserved.
Overview
I. Quantum Enabled V&V Overview
II. Overview of 10 V&V Challenge Problems
III. End to End Analysis Example
IV. Summary
![Page 3: Cyber-Physical V&V Challenges for the Evaluation of State ... · Cyber-Physical V&V Challenges for the Evaluation of State of the Art Model Checkers Research in Quantum Enabled V&V](https://reader030.fdocuments.net/reader030/viewer/2022013104/5c847a3309d3f291698d055b/html5/thumbnails/3.jpg)
3Copyright 2016, Lockheed Martin Corporation. All rights reserved.
What is it? • QVTrace*: This technology is a method for Software Verification &
Validation using Quantum Computer Assisted Formal Methods.
RequirementsAnd Implementation
(Software Code)
Classical Computation
D-Wave Adiabatic Quantum Computer
Defects (Bugs)Req/Code
InconsistencyReport to Designer
Quantum V&V
*Product Developed by QRA Inc.
• Target Users are System/Software Design Teams interested in:- Reducing development costs - Improving final product quality
Who will use it?
Quantum Enabled V&V
![Page 4: Cyber-Physical V&V Challenges for the Evaluation of State ... · Cyber-Physical V&V Challenges for the Evaluation of State of the Art Model Checkers Research in Quantum Enabled V&V](https://reader030.fdocuments.net/reader030/viewer/2022013104/5c847a3309d3f291698d055b/html5/thumbnails/4.jpg)
4Copyright 2016, Lockheed Martin Corporation. All rights reserved.
LM QA Solves a Quadratic Unconstrained Binary Optimization Problem
D-Wave Adiabatic Quantum ComputerCurrent State-of-the-Art, DW-2X: 1152q Washington
![Page 5: Cyber-Physical V&V Challenges for the Evaluation of State ... · Cyber-Physical V&V Challenges for the Evaluation of State of the Art Model Checkers Research in Quantum Enabled V&V](https://reader030.fdocuments.net/reader030/viewer/2022013104/5c847a3309d3f291698d055b/html5/thumbnails/5.jpg)
5Copyright 2016, Lockheed Martin Corporation. All rights reserved.
Quantum Superposition, Entanglement Enable Unique Optimization
Quantum Optimizationwith Superconducting Qubits
![Page 6: Cyber-Physical V&V Challenges for the Evaluation of State ... · Cyber-Physical V&V Challenges for the Evaluation of State of the Art Model Checkers Research in Quantum Enabled V&V](https://reader030.fdocuments.net/reader030/viewer/2022013104/5c847a3309d3f291698d055b/html5/thumbnails/6.jpg)
6Copyright 2016, Lockheed Martin Corporation. All rights reserved.
Early 2010LM ID’s
Quantum asKey Tech
Nov 2010Early Access
To QC
Mar 2011USC/ISI/LMTeam for QC Center
Jan 2012USC-LM
QC Operational
Mar 2013QC Upgrade
March 2016QC Upgrade
128 q DW1 “Rainier”512 q DW2 “Vesuvius”
1152 q DW2X “Washington”
QE-V&V Timeline
![Page 7: Cyber-Physical V&V Challenges for the Evaluation of State ... · Cyber-Physical V&V Challenges for the Evaluation of State of the Art Model Checkers Research in Quantum Enabled V&V](https://reader030.fdocuments.net/reader030/viewer/2022013104/5c847a3309d3f291698d055b/html5/thumbnails/7.jpg)
7Copyright 2016, Lockheed Martin Corporation. All rights reserved.
Overview of Challenge Problems• LM Aero Developed Set of 10 V&V Challenge Problems
• Goal: - Foster Collaboration in S5 Community (Ponder, Present, Publish)- Evaluate & Improve State-of-the-Art Formal Methods Toolsets
• Each Example in Package Includes:- Simulink Model Built in Matlab® R2012B- Parameters, if any, for Simulating Model (.mat)- Documentation Containing Description and Requirements
• Difficult due to Transcendental Functions, Nonlinearities and Discontinuous Math, Vectors, Matrices, States
Challenges Built with Commonly Used Blocks
![Page 8: Cyber-Physical V&V Challenges for the Evaluation of State ... · Cyber-Physical V&V Challenges for the Evaluation of State of the Art Model Checkers Research in Quantum Enabled V&V](https://reader030.fdocuments.net/reader030/viewer/2022013104/5c847a3309d3f291698d055b/html5/thumbnails/8.jpg)
8Copyright 2016, Lockheed Martin Corporation. All rights reserved.
Overview of Challenge Problems1. Triplex Signal Monitor2. Finite State Machine3. Tustin Integrator4. Control Loop Regulators5. Nonlinear Guidance Algorithm6. Feedforward Cascade Connectivity Neural Network7. Abstraction of a Control Allocator (Effector Blender)8. 6DOF with DeHavilland Beaver Autopilot*9. System Safety Monitor10. Euler Transformation
Flight Control and Vehicle Management System Inspired Problems
![Page 9: Cyber-Physical V&V Challenges for the Evaluation of State ... · Cyber-Physical V&V Challenges for the Evaluation of State of the Art Model Checkers Research in Quantum Enabled V&V](https://reader030.fdocuments.net/reader030/viewer/2022013104/5c847a3309d3f291698d055b/html5/thumbnails/9.jpg)
9Copyright 2016, Lockheed Martin Corporation. All rights reserved.
1. Triplex Signal Monitor
Description: this challenge problem involves the verification of a redundancy management system using quantum simulation techniques. The p
Sensor A
Sensor B
Sensor C
Airborne Redundancy Management
OnlineMonitoring
![Page 10: Cyber-Physical V&V Challenges for the Evaluation of State ... · Cyber-Physical V&V Challenges for the Evaluation of State of the Art Model Checkers Research in Quantum Enabled V&V](https://reader030.fdocuments.net/reader030/viewer/2022013104/5c847a3309d3f291698d055b/html5/thumbnails/10.jpg)
10Copyright 2016, Lockheed Martin Corporation. All rights reserved.
2. Finite State Machine
Description: this challenge problem involves the verification of a redundancy management system using quantum simulation techniques. The p
Discrete Interwoven Modes in Integrated Cyber-Physical System
Flight ControlEmbedded System
IntegratedSensor
![Page 11: Cyber-Physical V&V Challenges for the Evaluation of State ... · Cyber-Physical V&V Challenges for the Evaluation of State of the Art Model Checkers Research in Quantum Enabled V&V](https://reader030.fdocuments.net/reader030/viewer/2022013104/5c847a3309d3f291698d055b/html5/thumbnails/11.jpg)
11Copyright 2016, Lockheed Martin Corporation. All rights reserved.
3. Tustin Integrator
Description: this challenge problem involves the verification of a redundancy management system using quantum simulation techniques. The p
Fundamental Modeling and Simulation Component
NumericalIntegration
![Page 12: Cyber-Physical V&V Challenges for the Evaluation of State ... · Cyber-Physical V&V Challenges for the Evaluation of State of the Art Model Checkers Research in Quantum Enabled V&V](https://reader030.fdocuments.net/reader030/viewer/2022013104/5c847a3309d3f291698d055b/html5/thumbnails/12.jpg)
12Copyright 2016, Lockheed Martin Corporation. All rights reserved.
Kp
Gain
Kd
Gain1
Ki
Gain3
1s
Integrator
s
s+1
Transfer Fcn
4. Control Loop Regulators
Description: this challenge problem involves the verification of a redundancy management system using quantum simulation techniques. The p
CommandAuthority?Feedback
ErrorSynthesis
PIDArchitecture
Attributes of Multi-Axis Control Law of Output Commands
![Page 13: Cyber-Physical V&V Challenges for the Evaluation of State ... · Cyber-Physical V&V Challenges for the Evaluation of State of the Art Model Checkers Research in Quantum Enabled V&V](https://reader030.fdocuments.net/reader030/viewer/2022013104/5c847a3309d3f291698d055b/html5/thumbnails/13.jpg)
13Copyright 2016, Lockheed Martin Corporation. All rights reserved.
5. Nonlinear Guidance Algorithm
Description: this challenge problem involves the verification of a redundancy management system using quantum simulation techniques. The p
3D Vector Mathematics for Outer Loop Intercept Guidance
Aim Point Validity?
![Page 14: Cyber-Physical V&V Challenges for the Evaluation of State ... · Cyber-Physical V&V Challenges for the Evaluation of State of the Art Model Checkers Research in Quantum Enabled V&V](https://reader030.fdocuments.net/reader030/viewer/2022013104/5c847a3309d3f291698d055b/html5/thumbnails/14.jpg)
14Copyright 2016, Lockheed Martin Corporation. All rights reserved.
5. …Nonlinear Guidance Algorithm
Block TypesFor NL Guidance
Recent Focus on Import of Common Algorithmic Operators (Primitives)
![Page 15: Cyber-Physical V&V Challenges for the Evaluation of State ... · Cyber-Physical V&V Challenges for the Evaluation of State of the Art Model Checkers Research in Quantum Enabled V&V](https://reader030.fdocuments.net/reader030/viewer/2022013104/5c847a3309d3f291698d055b/html5/thumbnails/15.jpg)
15Copyright 2016, Lockheed Martin Corporation. All rights reserved.
6. Neural Network
Description: this challenge problem involves the verification of a redundancy management system using quantum simulation techniques. The p
2x10x10x1 Feedforward Cascade Connectivity NN
Output Features?
Inputs Layer 1 Layer2 Output
2-y
1-x
Network Topology
Inpu
ts
0.0 - 0.293130.29313 - 0.586260.58626 - 0.879380.87938 - 1.17251.1725 - 1.46561.4656 - 1.75881.7588 - 2.0519
PositiveNegative
-0.2
2
0
0.2
0.4
1
z
0.6
0.8
2
1
y
1.50 10.5
x
0-1 -0.5-1
-1.5-2 -2
Truth Model
![Page 16: Cyber-Physical V&V Challenges for the Evaluation of State ... · Cyber-Physical V&V Challenges for the Evaluation of State of the Art Model Checkers Research in Quantum Enabled V&V](https://reader030.fdocuments.net/reader030/viewer/2022013104/5c847a3309d3f291698d055b/html5/thumbnails/16.jpg)
16Copyright 2016, Lockheed Martin Corporation. All rights reserved.
… and More
Description: this challenge problem involves the verification of a redundancy management system using quantum simulation techniques. The p
Cyber-Physical V&V Challenge ProblemsLM Aeronautics Quantum Information Science Research Team 2015
Copyright © 2015 Lockheed Martin Corporation
AD
ID
AP Eng
HDG Mode
ALT Mode
HDG Ref
Turn Knob
ALT Ref
Pitch Wheel
Aileron Cmd
Elevator Cmd
Rudder Cmd
Autopilot
AC Bus
ID
AD
Sensors
Aileron
Elevator
Rudder
Flap
Throttle
Rudder Trim
Controls
Signal Conditioning
trim_flap
Constant
trim_throttle
Constant1
trim_rudder
Constant2trim_hdgref
Constant3
trim_turnknob
Constant4
trim_altref
Constant5
trim_pitchwheel
Constant6
DeHavilland Beaver Airframe
EnvBus
Environment
DeHavilland Beaver model originally based on work created by
Marc Rauw for Delft University of Technology, http://www.dutchroll.com
and subsequently modified by the Mathworkshttp://www.mathworks.com/matlabcentral/fileexchange/
FLIGHT CONTROL DemonstrationAuthor: elliocmModel Version: 1.80Date: 21-Sep-2015 15:23:12
boolean
Data Type Conversion1
boolean
Data Type Conversion2
boolean
Data Type Conversion3
1
Constant7APeng
HDGmode
HDGref
TurnKnob
![Page 17: Cyber-Physical V&V Challenges for the Evaluation of State ... · Cyber-Physical V&V Challenges for the Evaluation of State of the Art Model Checkers Research in Quantum Enabled V&V](https://reader030.fdocuments.net/reader030/viewer/2022013104/5c847a3309d3f291698d055b/html5/thumbnails/17.jpg)
17Copyright 2016, Lockheed Martin Corporation. All rights reserved.
Description: this challenge problem involves the verification of a redundancy management system using quantum simulation techniques. The p
End to End Analysis (Tustin)Cyber-Physical V&V Challenge Problems
LM Aeronautics Quantum Information Science Research Team 2015Copyright © 2015 Lockheed Martin Corporation
1
yout
2reset
1
xin
3T
xin
T
TL
BL
reset
ic
yout
TustinIntegrator
(Limited, Resettable, States)
4ic
5
TL
6
BL
cmd
Definitions:• Normal operation: the integrator is not in reset mode, and the
output is within the specified limits (TL and BL).• ypv: prior yout value• xinpv: prior xin input value• SP: Saturation Point
Input Signal to Be Integrated
Discrete Time Step
Top Limit
Bottom Limit
Boolean Reset
Initial Condition Upon Reset
Output Signal
Documentation Provides ICD, Definitions, and Requirements
![Page 18: Cyber-Physical V&V Challenges for the Evaluation of State ... · Cyber-Physical V&V Challenges for the Evaluation of State of the Art Model Checkers Research in Quantum Enabled V&V](https://reader030.fdocuments.net/reader030/viewer/2022013104/5c847a3309d3f291698d055b/html5/thumbnails/18.jpg)
18Copyright 2016, Lockheed Martin Corporation. All rights reserved.
Description: this challenge problem involves the verification of a redundancy management system using quantum simulation techniques. The p
End to End Analysis (Tustin)
TUSTIN INTEGRATOR (LIMITED, RESETTABLE, STATES)
Product
~=
Switch
upu
loy
SaturationDynamic
1
yout
.5
Gain
5reset
1xin
2T
4BL
3TL
6ic
z1
Unit Delay
z1
Unit Delay1
[TL]
Goto
TL
BL
TLc
BLc
bounds
[BL]
Goto1
[TL]
From
[BL]
From1
![Page 19: Cyber-Physical V&V Challenges for the Evaluation of State ... · Cyber-Physical V&V Challenges for the Evaluation of State of the Art Model Checkers Research in Quantum Enabled V&V](https://reader030.fdocuments.net/reader030/viewer/2022013104/5c847a3309d3f291698d055b/html5/thumbnails/19.jpg)
19Copyright 2016, Lockheed Martin Corporation. All rights reserved.
Description: this challenge problem involves the verification of a redundancy management system using quantum simulation techniques. The p
End to End Analysis (Tustin)
~=
Switch1
<
Relational Operator
1TL
2BL
1TLc
~=
Switch2
2BLc
![Page 20: Cyber-Physical V&V Challenges for the Evaluation of State ... · Cyber-Physical V&V Challenges for the Evaluation of State of the Art Model Checkers Research in Quantum Enabled V&V](https://reader030.fdocuments.net/reader030/viewer/2022013104/5c847a3309d3f291698d055b/html5/thumbnails/20.jpg)
20Copyright 2016, Lockheed Martin Corporation. All rights reserved.
Requirements: 1. When Reset is True and the Initial Condition (ic) is bounded by the provided Top and
Bottom Limits (BL <= ic <= TL), the Output (yout) shall equal the Initial Condition (ic).2. The Output (yout) shall be bounded by the provided Top and Bottom limits (TL and BL)3. When in normal operation, the output shall be the result of the equation, yout = T/2*(xin
+ xinpv)+ ypv4. The Output of this function shall approximate the integration of the value of the input
signal over time within a specified tolerance, defined in subtests below:a. After 10 seconds of Computation at an execution frequency of 10 hz, the Output
should equal 10 within a +/- 0.1 tolerance, for a Constant Input (xin = 1.0), and the sample delta time T = 0.1 seconds when in normal mode of operation.
b. Over a 10 second computational duration at an execution frequency of 10 hz, the Output should equal the sine of time t, sin(t), where time is defined as a vector from 0 to 10 by increments of 0.1 seconds within a +/- 0.1 tolerance for an input equal to the cosine of time t, cos(t), with the sample delta time T = 0.1 seconds when in normal mode of operation.
End to End Analysis (Tustin)
Requirements Properties (Tests) is At Least Half the Challenge
![Page 21: Cyber-Physical V&V Challenges for the Evaluation of State ... · Cyber-Physical V&V Challenges for the Evaluation of State of the Art Model Checkers Research in Quantum Enabled V&V](https://reader030.fdocuments.net/reader030/viewer/2022013104/5c847a3309d3f291698d055b/html5/thumbnails/21.jpg)
21Copyright 2016, Lockheed Martin Corporation. All rights reserved.
Requirements: 1. When Reset is True and the Initial Condition (ic) is bounded by the provided Top and
Bottom Limits (BL <= ic <= TL), the Output (yout) shall equal the Initial Condition (ic).2. The Output (yout) shall be bounded by the provided Top and Bottom limits (TL and BL)3. When in normal operation, the output shall be the result of the equation, yout = T/2*(xin
+ xinpv)+ ypv4. The Output of this function shall approximate the integration of the value of the input
signal over time within a specified tolerance, defined in subtests below:a. After 10 seconds of Computation at an execution frequency of 10 hz, the Output
should equal 10 within a +/- 0.1 tolerance, for a Constant Input (xin = 1.0), and the sample delta time T = 0.1 seconds when in normal mode of operation.
b. Over a 10 second computational duration at an execution frequency of 10 hz, the Output should equal the sine of time t, sin(t), where time is defined as a vector from 0 to 10 by increments of 0.1 seconds within a +/- 0.1 tolerance for an input equal to the cosine of time t, cos(t), with the sample delta time T = 0.1 seconds when in normal mode of operation.
End to End Analysis (Tustin)
Detailed Formal Property Derivation:# 1. When Reset is True and the Initial Condition (ic) is# bounded by the provided Top and Bottom Limits (BL<=ic<=TL),# the Output (yout) shall equal the Initial Condition (ic).# If the Initial Condition is not bound by the Limits # during a Reset, the Output shall equal the saturation # point (nominally with TL>=BL, ic>=TL impl SP==TL and ic<=BL implSP==BL. # Off-nominally with TL<BL, ic, ic>=BL impl SP==BL and ic<=TL implSP==TL.((reset and ic<=TL and ic>=BL) impl yout == ic); #1a ((reset and ic>=TL and ic>=BL and TL>=BL) impl yout == TL); #1b ((reset and ic<=BL and ic>=BL and TL>=BL) impl yout == BL); #1c((reset and ic>=BL and ic<=TL and TL<BL) impl yout == BL); #1d((reset and ic<=TL and ic>=BL and TL<BL) impl yout == TL); #1e
![Page 22: Cyber-Physical V&V Challenges for the Evaluation of State ... · Cyber-Physical V&V Challenges for the Evaluation of State of the Art Model Checkers Research in Quantum Enabled V&V](https://reader030.fdocuments.net/reader030/viewer/2022013104/5c847a3309d3f291698d055b/html5/thumbnails/22.jpg)
22Copyright 2016, Lockheed Martin Corporation. All rights reserved.
End to End Analysis (Tustin)
Requirements Properties (Tests) is At Least Half the Challenge
Requirements: 1. When Reset is True and the Initial Condition (ic) is bounded by the provided Top and
Bottom Limits (BL <= ic <= TL), the Output (yout) shall equal the Initial Condition (ic).2. The Output (yout) shall be bounded by the provided Top and Bottom limits (TL and BL)3. When in normal operation, the output shall be the result of the equation, yout = T/2*(xin
+ xinpv)+ ypv4. The Output of this function shall approximate the integration of the value of the input
signal over time within a specified tolerance, defined in subtests below:a. After 10 seconds of Computation at an execution frequency of 10 hz, the Output
should equal 10 within a +/- 0.1 tolerance, for a Constant Input (xin = 1.0), and the sample delta time T = 0.1 seconds when in normal mode of operation.
b. Over a 10 second computational duration at an execution frequency of 10 hz, the Output should equal the sine of time t, sin(t), where time is defined as a vector from 0 to 10 by increments of 0.1 seconds within a +/- 0.1 tolerance for an input equal to the cosine of time t, cos(t), with the sample delta time T = 0.1 seconds when in normal mode of operation.
Detailed Formal Property Derivation:# Over a 10 second computational duration at an execution frequency of 10 hz, the Output should equal the sine of time t, sin(t), where time is defined as a vector from 0 to 10 by increments of 0.1 seconds within a +/- 0.1 tolerance for an input equal to the cosine of time t, cos(t), with the sample delta time T = 0.1 seconds when in normal mode of operation(xin{0}==1 and xin{1}==0.995 and … xin{100}==-0.83907 and T{all}==0.1 and reset{never} and (TL{all}>=BL{all}) and (yout{all}>BL{all}) and (yout{all}<TL{all})) impl (abs(yout{0}-0)<=0.1 and abs(yout{1}-0.099833)<=0.1 … and abs(yout{98}--0.36648)<=0.1 and abs(yout{99}--0.45754)<=0.1 and abs(yout{100}--0.54402)<=0.1);
Analytic vs Numerical
|Tustin Error| < .05
10 s
10 s
![Page 23: Cyber-Physical V&V Challenges for the Evaluation of State ... · Cyber-Physical V&V Challenges for the Evaluation of State of the Art Model Checkers Research in Quantum Enabled V&V](https://reader030.fdocuments.net/reader030/viewer/2022013104/5c847a3309d3f291698d055b/html5/thumbnails/23.jpg)
23Copyright 2016, Lockheed Martin Corporation. All rights reserved.
Description: this challenge problem involves the verification of a redundancy management system using quantum simulation techniques. The p
End to End Analysis (Triplex)
Signal A
Signal B
Signal C
Threshold Level
Persistence Limit (Duration Trigger)
Fault Code
Given These Conditions, Prove the Correct Fault Report
FC: 0-nofail, 1-branchC, 2-branchB, 4-branchA# detailed formal property(abs(ia{all}-ib{all})>Tlevel{all} or abs(ia{all}-ic{all})>Tlevel{all} and PC>PClimit and PClimit{all}==1 and Tlevel{all}==1) impl (FC{3}==4);
![Page 24: Cyber-Physical V&V Challenges for the Evaluation of State ... · Cyber-Physical V&V Challenges for the Evaluation of State of the Art Model Checkers Research in Quantum Enabled V&V](https://reader030.fdocuments.net/reader030/viewer/2022013104/5c847a3309d3f291698d055b/html5/thumbnails/24.jpg)
24
![Page 25: Cyber-Physical V&V Challenges for the Evaluation of State ... · Cyber-Physical V&V Challenges for the Evaluation of State of the Art Model Checkers Research in Quantum Enabled V&V](https://reader030.fdocuments.net/reader030/viewer/2022013104/5c847a3309d3f291698d055b/html5/thumbnails/25.jpg)
25
![Page 26: Cyber-Physical V&V Challenges for the Evaluation of State ... · Cyber-Physical V&V Challenges for the Evaluation of State of the Art Model Checkers Research in Quantum Enabled V&V](https://reader030.fdocuments.net/reader030/viewer/2022013104/5c847a3309d3f291698d055b/html5/thumbnails/26.jpg)
26
0 0.5 1 1.5 2 2.5 30
5000
10000
t [sec]
inpu
ts
iaibic
0 0.5 1 1.5 2 2.5 3-1
-0.5
0
0.5
1
t [sec]
FC
Counter Example Data As a Test Harness to Model
![Page 27: Cyber-Physical V&V Challenges for the Evaluation of State ... · Cyber-Physical V&V Challenges for the Evaluation of State of the Art Model Checkers Research in Quantum Enabled V&V](https://reader030.fdocuments.net/reader030/viewer/2022013104/5c847a3309d3f291698d055b/html5/thumbnails/27.jpg)
27
0 1 2 30
5000
10000
t [sec]
inpu
ts
iaibic
0 1 2 3-1
-0.5
0
0.5
1
t [sec]
FC
0 1 2 30
1
2
3
4
t [sec]
inpu
ts
|ia-ib||ia-ic||ib-ic|
This is a ValidDefect DiscoveredBy QVTrace v0.9.1
Closer Inspection Yields a Problem
![Page 28: Cyber-Physical V&V Challenges for the Evaluation of State ... · Cyber-Physical V&V Challenges for the Evaluation of State of the Art Model Checkers Research in Quantum Enabled V&V](https://reader030.fdocuments.net/reader030/viewer/2022013104/5c847a3309d3f291698d055b/html5/thumbnails/28.jpg)
28
0 1 2 3 4 5 6 7 8 9 10-4
-2
0
2
t [sec]
inpu
ts
iaibic
0 1 2 3 4 5 6 7 8 9 100
0.5
1
1.5
2
t [sec]
FC
FC: 0-nofail, 1-branchC, 2-branchB, 4-branchA
Nominal Behavior
![Page 29: Cyber-Physical V&V Challenges for the Evaluation of State ... · Cyber-Physical V&V Challenges for the Evaluation of State of the Art Model Checkers Research in Quantum Enabled V&V](https://reader030.fdocuments.net/reader030/viewer/2022013104/5c847a3309d3f291698d055b/html5/thumbnails/29.jpg)
29
0 1 2 3 4 5 6 7 8 9 10-4
-2
0
2
t [sec]
inpu
ts
iaibic
0 1 2 3 4 5 6 7 8 9 10-1
-0.5
0
0.5
1
t [sec]
FC
FC: 0-nofail, 1-branchC, 2-branchB, 4-branchA
Faulty Behavior
![Page 30: Cyber-Physical V&V Challenges for the Evaluation of State ... · Cyber-Physical V&V Challenges for the Evaluation of State of the Art Model Checkers Research in Quantum Enabled V&V](https://reader030.fdocuments.net/reader030/viewer/2022013104/5c847a3309d3f291698d055b/html5/thumbnails/30.jpg)
30Copyright 2016, Lockheed Martin Corporation. All rights reserved.
Summary and Path Forward• Round 1 V&V Challenge Problems In Use to Develop Novel QE-V&V
• Requirements Formalization is Difficult Alone and Reduces Defects - Requirements Properties (Tests) is At Least Half the Challenge- Beneficial to Front Load Design Process with Formalization- Need Near if Not Equivalent “Primitives” Capability in Properties- Interested in Deploying Challenges Requirements to SPeAR
• Goals:- Publish Results on Current Round of Challenges - Round 2 V&V Challenge Problems To Increase Complexity Further - Transition Formal Methods Analysis Process/Tools to Programs- Interested? Contact: Chris Elliott,
[email protected], 817-935-3054
Thank You
![Page 31: Cyber-Physical V&V Challenges for the Evaluation of State ... · Cyber-Physical V&V Challenges for the Evaluation of State of the Art Model Checkers Research in Quantum Enabled V&V](https://reader030.fdocuments.net/reader030/viewer/2022013104/5c847a3309d3f291698d055b/html5/thumbnails/31.jpg)
31Copyright 2016, Lockheed Martin Corporation. All rights reserved.
![Page 32: Cyber-Physical V&V Challenges for the Evaluation of State ... · Cyber-Physical V&V Challenges for the Evaluation of State of the Art Model Checkers Research in Quantum Enabled V&V](https://reader030.fdocuments.net/reader030/viewer/2022013104/5c847a3309d3f291698d055b/html5/thumbnails/32.jpg)
32Copyright 2016, Lockheed Martin Corporation. All rights reserved.
Dr. Edward H. “Ned” AllenChief Scientist and LM Senior FellowLockheed Martin Corporation
Mr. Greg TallantProgram Manager and LM Fellow
Lockheed Martin AeronauticsSkunk Works
Biography slide
Chris ElliottQuantum Apps Team
Lockheed Martin AeronauticsSkunk Works
Mr. Peter StanfillQuantum Apps TeamLockheed Martin AeronauticsSkunk Works
Dr. Kristen PudenzQuantum Apps Team
Lockheed Martin AeronauticsSkunk Works