Multiple Shooting, CEGAR-based Falsification for Hybrid Systems Aditya Zutshi Sriram...
-
Upload
caitlin-allyson-rogers -
Category
Documents
-
view
220 -
download
4
Transcript of Multiple Shooting, CEGAR-based Falsification for Hybrid Systems Aditya Zutshi Sriram...
1
Multiple Shooting, CEGAR-based Falsification for Hybrid Systems
Aditya Zutshi Sriram Sankaranarayanan
Jyotirmoy DeshmukhJames Kapinski
3
Initial States
ErrorStates
System Description
t
Falsification
Is there a trajectory from an initial state to an error state?
Error?
4
System Description
Hybrid Automaton Model[Alur, Henzinger, Lygeros, Sastry, Tomlin,…]
𝐺21 (𝑥 )=0
𝐺12 (𝑥 )=0
Mode 1 Mode 2
𝑑𝑥𝑑𝑡
= 𝑓 2(𝑥 )𝑑𝑥𝑑𝑡
= 𝑓 1(𝑥)
Most systems do not have
Hybrid Automaton
models!
Simulink/Stateflow
X
t
X’
SIM(X,t)
X, t X’
5
Single Shooting
Initial States
Error States
SIM(X,t)
System Description
Inefficient in the presence of
non-linearities and discrete updates
S-Taliro: [Fainekos, et al.]BREACH: [Donze’]RRT: [Bhatia et al., …]
6
Gaps
Multiple Shooting
Initial States
Error States
• Explore trajectory space
• Narrow gaps iteratively
Proposed Solution
CEGAR
7
ContributionsMultiple Shooting CEGAR
(Counter Example Guided Refinement)
Verification of Hybrid Systems Based on Counterexample-Guided Abstraction Refinement[Clarke, Fehnker, et al.]
𝑥2
𝑥1
Abstract
path
Trajectory segment
RefinementNarrowing
of gaps
A
B
Grid based Abstractions
Scatter and SimulateFundamental question in abstractions:A B ?
8
Scatter & Simulate
• Grid based Abstractions
• Induced by norm
𝑥2
𝑥1
Δ 𝑡
A
BExplicit Abstractions
• Black Box: No system dynamics
• Complex dynamics• Curse of
Dimensionality
9
Multiple Shooting & CEGAR
Assume implicit abstraction
Explore it using scatter & simulate
Enumerate error paths
Assume a finer abstraction
Check for concrete paths
Search Error Paths
• Trade soundness for efficiency.
• Find a subset of paths.
Error Paths
Compute
Compute
Refine abstraction using CEGAR
done
10
CEGAR
Multiple Shooting & CEGAR…
Assume a finer abstraction
Error Paths
Compute
𝐶0
Refine by CEGAR• Examine abstract
error paths• Entire path• Initial cell
done
Assume implicit abstraction
Explore it using scatter & simulate
Enumerate error paths
Compute
Check for concrete paths
Finer grid size
11
Initial States
Error States
Scatter and Simulate
𝜖
𝜖
Δ 𝑡
Δ 𝑡
Δ 𝑡
Compute
Get cell from Q
Sample cell
Simulate for
Identify reached cells
If new, add cell to Q
CellQueue
Enumerate error paths
Error Paths
12
CEGAR
Refinement
𝜖
𝜖
𝜖2
𝜖2
Refine Grid
Scatter & Simulate
Compute Error Paths
New Error Paths
Enumerate error Paths
13
Concretization
• Described procedure can run forever– Only comes up with
segmented trajectories– No termination guarantee
due to numerical errors
• Solution– interleave Concretization:
Use random testing on refined initial cells
Scatter &Simulate
CEGAR
Concretize
Done!!
19
Experiments1. Van Der Pol2. Lorenz3. Brusselator4. Bouncing Ball5. Bouncing Ball + SHM6. Constrained Pendulum7. Navigation 30(mod.)8. Idle Speed Controller9. MPC10. Glucose Insulin11. Quadcopter(mod.)12. Cardiac
Academic Examples
Complex Benchmarks
14 Cont. States625 Modes
20
Comparison1. Van Der Pol2. Lorenz3. Brusselator4. Bouncing Ball5. Bouncing Ball + SHM6. Constrained Pendulum7. Navigation 30(mod.)8. Idle Speed Controller9. MPC10. Glucose Insulin11. Quadcopter(mod.)12. Cardiac
Random Testing
S-Taliro
dReach
S-Taliro: [Fainekos, et. Al.]dReach: [Gao, et. Al. ]
Exhaustive
Light-weight
Scatter and Simulate
21
Experimental Setup
Random Testing S-Taliro Scatter
& Sim.
Times are hard to
compare!
Random Testing• Use random testing to
synthesize safety properties when they don’t exist
• Run 100,000 simulations and find number of violations
¿ 𝑣𝑖𝑜 .100,000
S-Taliro vs Scatter & Sim.• Run 10 times• Run terminates if
• Violation found• Timeout: 1hr
• Tools can restart during a run• Time taken is hard to compare
• S-Taliro has a single threaded impl.
¿𝑣𝑖𝑜 .10
22
Highly non-linear!
Results - Van Der Pol
2 continuous
States
Random Testing S-Taliro Scatter
& Sim.
1010
1010
0100,000
Vs
23
Hybrid!
Results - Bouncing Ball
4 continuous
States
Random Testing S-Taliro Scatter
& Sim.
110
1010
3100,000
1mode
Vs
24
625 Modes!
Results - Navigation30
4 continuous
States
625 modes
Random Testing S-Taliro Scatter
& Sim.
310
1010
1100,000
Vs
Becnhmarks for Hybrid Systems Verification: [Fehnker and Ivancic]
25
Inputs!
Results - Idle Speed Controller
Random Testing S-Taliro Scatter
& Sim.
210
1010
70100,000
9 continuous
States
4 modes
1 input
Vs
A new algorithm for reachability analysis of hybrid automata : [A. Casagrande, et al.]
26
In Summary…
• Falsification technique for Hybrid Systems.• No explicit model required!• Simulations are cheap and parallelizable!• Generalizable in many direction.
But…• Can not find non-robust trajectories• Convergence is not guaranteed• Best effort search– Can provide asymptotic guarantees
28
Falsification Approaches: Shooting
Single Shooting
• Random testing• S-Taliro• BREACH• Systematic Sim.• RRTs• …
Multiple Shooting
• Proposed approach:Scatter & Simulate
29
Single Shooting: Random Testing
Initial States
Error States
SIM(X,T)
System Description
• Naïve: needs guidance
• Curse of dimensionality: Scales poorly with increasing states
30
Single Shooting:Guided Testing
• S-Taliro: [Fainekos, et. Al]
• BREACH: [Donze]
𝜌
Initial States
Error States
Inefficient in the presence of
non-linearities and discrete updates
31
Multiple Shooting
Undesirable Gaps
Solution…?Use mature NLP Solvers
Translate the problem as an optimization problem with equality constraints
Distribute non -
linearity
Initial States
Error States
Proposed Solution
Use Abstractions and CEGAR
A Trajectory Splicing Approach to Concretizing Counterexamples for Hybrid Systems: [Zutshi, et al.]
32
Abstractions and CEGARHow to effectively use Multiple Shooting?
Use Discrete Abstractions and a refinement procedure
CEGAR: Counter Example Guided Refinement
Verification of Hybrid Systems Based on Counterexample-Guided Abstraction Refinement[Clarke, Fehnker, et al.]
𝑥2
𝑥1
Grid Based Implicit Abstraction
• Partitions the state space into rectangular Cells
• Discovers relations using simulation
Induced by norm
33
Grid Based Abstraction
• Discretizes concrete states
• Relations induced by Dynamics
Abstract State: Concrete States:
𝑥1=𝑙1 𝑥1=h1
𝑥2=h2
𝑥2=𝑙2
𝐶0
𝐶1
HSolver: [Ratschan, et al.]
34
Explicit Abstractions
𝑥2
𝑥1
Curse of Dimensionality
Explicit abstraction construction
• Used by verification approaches
• Sound procedure finds relations between adjacent cells
• Enumerate all abstract error paths
Predicate Abstraction for reachability analysis of HS[Alur, Dang, Ivancic]