TRUSTWORTHY DESIGN VALIDATION OF A SATELLITE PLATFORM
Transcript of TRUSTWORTHY DESIGN VALIDATION OF A SATELLITE PLATFORM
TRUSTWORTHY DESIGN VALIDATION OF A SATELLITE PLATFORM
Viet Yen Nguyen [email protected] this research is commissioned by: MDDays 2013, Eindhoven, Netherlands
OVERVIEW
1. Space System Software Design & Validation 2. Formal methods with COMPASS toolset 3. Satellite case study 4. Conclusions
TECHNOLOGY CONTEXT OF THIS TALK
Space Engineering
Formal Methods
Academia Industry
Requirements
Safety & Dependability
Control
Software Hardware
FDIR
Thermal
Systems
Cost
…
…
Logics
Model Checking
Markov Chains
Small-Step Semantics
Timed automata
Hybrid systems
Reachability
SAT-solving
Formal Semantics
…
…
CASE: SATELLITE OF ESA MISSION IN DEVELOPMENT
• Platform keeps satellite in space, like car’s chassis. – control & data unit, – propulsion, – telemetry, tracking & cmd. – power, – attitude & orbit control sys., – reconfiguration module, – etc.
• Fault detection, isolation, recovery (FDIR) software and hardware: – redundancies + recovery, – compensation algorithms, – failure isolation schemes, – omnipresent in satellite.
Note: shown satellite is not the one of the case study.
FDIR
FDIR FDIR
FDIR
FDIR
FDIR
FDIR
FDIR
FDIR
DEMANDING REQUIREMENTS NEED EXTENSIVE V&V
• “Satellite’s in-orbit lifetime shall be at least 5 years.” • “Satellite’s platform reliability shall be equivalent or less than
1000 FIT.” • “The probability that the satellite transits to safe mode shall be
lower than 0.005, considering 40 hours for ground time to repair.”
• “In case of an over-current, a switch to the redundant coil is performed by OBDH SW.”
• “While in nominal mode and an lost uplink, the transponder shall transit within 10 seconds to safe mode.”
• … thousands of similar requirements!
• In presence of space hazards like radiation, extreme heat/cold, radio latencies, gravitational forces, etc.
Note: shown requirements are not from the case study.
EXAMPLE OF V&V IN SPACE SYSTEMS ENGINEERING
ECSSȬMȬSTȬ10CȱRev.ȱ1ȱ6ȱMarchȱ2009ȱ
19ȱ
4.4 Project phasing
4.4.1 Introduction Theȱlifeȱcycleȱofȱspaceȱprojectsȱisȱtypicallyȱdividedȱintoȱ7ȱphases,ȱasȱfollows:ȱ
x Phaseȱ0ȱȬȱMissionȱanalysis/needsȱidentificationȱ
x PhaseȱAȱȬȱFeasibilityȱ
x PhaseȱBȱȬȱPreliminaryȱDefinitionȱ
x PhaseȱCȱȬȱDetailedȱDefinitionȱ
x PhaseȱDȱȬȱQualificationȱandȱProductionȱ
x PhaseȱEȱ–Utilizationȱ
x PhaseȱFȱ–ȱDisposalȱ
AȱtypicalȱprojectȱlifeȱcycleȱisȱillustratedȱinȱFigureȱ4Ȭ3.ȱȱ
Projectȱ phasesȱ areȱ closelyȱ linkedȱ toȱ activitiesȱ onȱ systemȱ andȱ productȱ level.ȱDependingȱ onȱ theȱ specificȱ circumstancesȱ ofȱ aȱ projectȱ andȱ theȱ acceptanceȱ ofȱinvolvedȱrisk,ȱactivitiesȱcanȱoverlapȱprojectȱphases.ȱ
Atȱ theȱ conclusionȱ ofȱ theȱ majorȱ activitiesȱ andȱ theȱ relatedȱ projectȱ reviewsȱconfigurationȱbaselinesȱareȱestablishedȱ(seeȱECSSȬMȬSTȬ40).ȱ
Figureȱ4Ȭ3:ȱTypicalȱprojectȱlifeȱcycleȱ
ȱ
• Requirements analysis. • Fault tree analysis. • Review and inspection.
• HW/SW co-testing. • Early prototyping. • Control simulation.
• End-to-end IS testing. • Operations readiness. • Mission scenario tests.
• In orbit testing.
SPACECRAFT IS BECOMING “FLYING SOFTWARE” Flight Software Complexity
3/5/2009 28
Growth in Code Size for Human and Robotic Missions
1
10
100
1000
10000
100000
1000000
1000000019
68
1970
1972
1974
1976
1978
1980
1982
1984
1986
1988
1990
1992
1994
1996
1998
2000
2002
2004
Year of Mission
NCSL
(log
sca
le)
unmannedmannedExpon. (unmanned)Expon. (manned)
Non
-Com
men
t Sou
rce
Line
s (lo
g sc
ale)
1969 Mariner-6 (30)1975 Viking (5K)1977 Voyager (3K)1989 Galileo (8K)1990 Cassini (120K)1997 Pathf inder (175K)1999 DS1 (349K)2003 SIRTF/Spitzer (554K)2004 MER (555K)2005 MRO (545K)
1968 Apollo (8.5K)1980 Shuttle(470K)1989 ISS (1.5M)
Robotic Human
Figure 3. History of flight software growth in human and robotic missions.
In the realm of human spaceflight, there are only three NASA data points (Apollo, Space Shuttle, and International Space Station), so there is much less confidence in projecting a trend. Orion flight software size estimates exceed 1 million lines of code, which means that it will be more than 100 times larger than Apollo flight software and will run on avionics with thousands of times the RAM and CPU performance of the Apollo avionics.
Not all NASA centers show a growth trend. Figures 4 shows flight software sizes for selected GSFC missions from 1997 through 2009, with NCSL ranging from 28,400 to 149,500. The uncharacteristically large amount of code for the 2001 MAP mission was due to the need for an ‘executive’ for the Remote Serving Nodes (RSN) processors, programmed in assembly language. The lack of a growth trend matched local expectations in that many of Goddard’s science missions are similar in nature, with routine variations in instruments and observation requirements. However, one mission currently in development is expected to have significantly more software, though size estimates are not yet available. Specifically, the Laser Interferometer Space Antenna (LISA) mission comprises three formation-flying spacecraft with distances among them measured to extraordinarily high levels of precision. Flight software will be larger due to a doubling of issues: twice as many control modes, twice as many sensors and actuators, and fault detection on twice as many telemetry points. Section 4.4 examines LISA in more detail as a case study in growth of software complexity.
From NASA Study Flight Software Complexity (2009). Same growth trend in: • military aircrafts, • cars, and • airplanes.
CHALLENGES IN SPACE VERIFICATION & VALIDATION
More demanding requirements.
More FDIR software.
More system behaviors.
More issues during V&V
More pressure on meeting cost and schedule constraints!
Example: once every 175 year launch window for Voyager 2.
OUR SOLUTION: COMPASS Formal-methods based system software co-engineering
HISTORY: COMPASS CONSORTIUM
Formal semantics, performance
evaluation, model checking, etc.
Model checking, safety &
dependability analysis, etc.
Domain experts
Model-based engineering tools
COMPASS
INCREASE FORMALITY IN EARLY PHASE B
ECSSȬMȬSTȬ10CȱRev.ȱ1ȱ6ȱMarchȱ2009ȱ
19ȱ
4.4 Project phasing
4.4.1 Introduction Theȱlifeȱcycleȱofȱspaceȱprojectsȱisȱtypicallyȱdividedȱintoȱ7ȱphases,ȱasȱfollows:ȱ
x Phaseȱ0ȱȬȱMissionȱanalysis/needsȱidentificationȱ
x PhaseȱAȱȬȱFeasibilityȱ
x PhaseȱBȱȬȱPreliminaryȱDefinitionȱ
x PhaseȱCȱȬȱDetailedȱDefinitionȱ
x PhaseȱDȱȬȱQualificationȱandȱProductionȱ
x PhaseȱEȱ–Utilizationȱ
x PhaseȱFȱ–ȱDisposalȱ
AȱtypicalȱprojectȱlifeȱcycleȱisȱillustratedȱinȱFigureȱ4Ȭ3.ȱȱ
Projectȱ phasesȱ areȱ closelyȱ linkedȱ toȱ activitiesȱ onȱ systemȱ andȱ productȱ level.ȱDependingȱ onȱ theȱ specificȱ circumstancesȱ ofȱ aȱ projectȱ andȱ theȱ acceptanceȱ ofȱinvolvedȱrisk,ȱactivitiesȱcanȱoverlapȱprojectȱphases.ȱ
Atȱ theȱ conclusionȱ ofȱ theȱ majorȱ activitiesȱ andȱ theȱ relatedȱ projectȱ reviewsȱconfigurationȱbaselinesȱareȱestablishedȱ(seeȱECSSȬMȬSTȬ40).ȱ
Figureȱ4Ȭ3:ȱTypicalȱprojectȱlifeȱcycleȱ
ȱ
Improve understanding of system software behavior with early V&V.
1. Model-based. 2. Formal methods. 3. Automated analysis tools.
MODEL-BASED ANALYSIS FOR V&V USING COMPASS
Model Nominal Hybrid automata
Error Markov chains
Requirements Property Patterns CTL/LTL
CSL
Automated Analyses
Simulation State exploration
Model Checking Reachability
Fault Tree Analysis Reachability
FMEA Reachability
Probabilistic Risk Assessment
Probabilistic transient
Diagnosis Constraints
TwinPlant reachability
ARCHITECTURE & ANALYSIS DESIGN LANGUAGE: ONE LANGUAGE FOR ALL SYSTEM-LEVEL ASPECTS
• Originated on MetaH from 1989 • Standard by SAE (Society of Automotive Engineers) • Models real-time and performance critical HW/SW architecture • Hierarchical and component-based.
NOMINAL MODELING
system Power ! features! voltage: out data port real; !end Power;!!system implementation Power.Imp ! subcomponents! batt1: device Battery.Imp in modes (primary);! batt2: device Battery.Imp in modes (backup); ! connections! data port batt1.voltage -> voltage in modes (primary);! data port batt2.voltage -> voltage in modes (backup); ! modes! primary: initial mode;! backup: mode; ! transitions! primary -[batt1.empty]-> backup;! backup -[batt2.empty]-> primary; !end Power.Imp;!
Component type
Component implementation
Interface by ports
Behaviour
Dynamic reconfiguration
Modes
ERROR MODELING
error model BatteryFailure ! features!
!ok: initial state; !!dead: error state;!!resetting: error state;!!batteryDied: out error propagation;!
end BatteryFailure;!!error model implementation BatteryFailure.Imp! events!
!fault: error event occurrence poisson 0.001;!works: error event occurrence poisson 0.2;!!fails: error event occurrence poisson 0.8;!
transitions!!ok -[fault]-> dead;!!dead -[batteryDied]-> dead;!!dead -[reset]-> resetting; !!resetting -[works]-> ok; !!resetting -[fails]-> dead; !
end BatteryFailure.Imp;!
Failure modes
Fault propagations
Failure rates (based on FITs)
Fault behaviour
FAULT INJECTION AND MODEL EXTENSION
error model BatteryFailure ! features!
!ok: initial state; !!dead: error state;!!resetting: error state;!!batteryDied: out error propagation;!
end BatteryFailure;!!error model implementation BatteryFailure.Imp! events!
!fault: error event occurrence poisson 0.001;!works: error event occurrence poisson 0.2;!!fails: error event occurrence poisson 0.8;!
transitions!!ok -[fault]-> dead;!!dead -[batteryDied]-> dead;!!dead -[reset]-> resetting; !!resetting -[works]-> ok; !!resetting -[fails]-> dead; !
end BatteryFailure.Imp;!
system Power ! features! voltage: out data port real; !end Power;!!system implementation Power.Imp ! subcomponents! batt1: device Battery.Imp in modes (primary);! batt2: device Battery.Imp in modes (backup); ! connections! data port batt1.voltage -> voltage in modes (primary);! data port batt2.voltage -> voltage in modes (backup); ! modes! primary: initial mode;! backup: mode; ! transitions! primary -[batt1.empty]-> backup;! backup -[batt2.empty]-> primary; !end Power.Imp;!
Dead: voltage := 0!
…! -- nominal transitions!charged -[when _error.state != dead then voltage := 2.0 * energy + 4.0]-> charged;!charged -[empty when energy = 0.2 and _error.state != dead]-> depleted;!depleted -[when _error.state != dead then voltage := 2.0 * energy + 4.0]-> depleted;!-- nominal transitions with fault injection!charged -[when _error.state = dead then voltage := 0]-> charged;!charged -[empty when energy = 0.2 and _error.state = dead then voltage := 0]-> depleted;!depleted -[when _error.state = dead then voltage := 0]-> depleted;!-- error transitions !depleted -[_error.#works when _error.state = resetting]-> depleted;!charged -[_error.#works when _error.state = resetting]-> charged;!-- error transitions with fault injections!depleted -[_error.#fault when _error.state = ok then voltage := 0]-> depleted;!…!
Nominal
Erroneous
Fault injection
Extended model
BEHAVIOUR METAMODEL: FORMAL SEMANTICS
• Operational semantics: – System is tuple of variables.
(mode,alert,battery1.mode,…)!
– System state is a valuation to tuple variables. <primary,false,charged,…>
– Nominal/error transitions change current state to new system state. <primary,false,charged,…> ! ! -[empty]-> <backup,false,charged,…>
• Fundamental to formal methods. – Precise & unambiguous – Automated analyses
FOR REVIEW ONLY 20120903115800
hprimary, [alert 7! ?]i khcharged, [energy 7! 1.0, tryReset 7! ?,voltage 7! 6.0]i khcharged, [energy 7! 1.0, tryReset 7! ?,voltage 7! 6.0]i k
hm0, [voltage 7! 6.0,alert 7! ?]i+ 30.0
hprimary, [alert 7! ?]i khcharged, [energy 7! 0.4, tryReset 7! ?,voltage 7! 6.0]i khcharged, [energy 7! 1.0, tryReset 7! ?,voltage 7! 6.0]i k
hm0, [voltage 7! 6.0,alert 7! ?]i+ ⌧
hprimary, [alert 7! ?]i khcharged, [energy 7! 0.4, tryReset 7! ?,voltage 7! 4.8]i khcharged, [energy 7! 1.0, tryReset 7! ?,voltage 7! 6.0]i k
hm0, [voltage 7! 4.8,alert 7! ?]i+ 10.0
hprimary, [alert 7! ?]i khcharged, [energy 7! 0.2, tryReset 7! ?,voltage 7! 4.8]i khcharged, [energy 7! 1.0, tryReset 7! ?,voltage 7! 6.0]i k
hm0, [voltage 7! 4.8,alert 7! ?]i+ ⌧
hprimary, [alert 7! >]i khcharged, [energy 7! 0.2, tryReset 7! >,voltage 7! 4.4]i khcharged, [energy 7! 1.0, tryReset 7! ?,voltage 7! 6.0]i k
hm0, [voltage 7! 4.4,alert 7! >]i+ empty
hprimary, [alert 7! ?]i khdepleted, [energy 7! 0.2, tryReset 7! >,voltage 7! 4.4]i khcharged, [ 7! 1.0, tryReset 7! ?,voltage 7! 6.0]i k
hm0, [voltage 7! 6.0,alert 7! ?]i+ ...
Figure 3.2: An example trace of the Power system.
3.5 Graphical Notation
As an extension of the COMPASS project, we, together with Ellidiss, developeda graphical notation of SLIM and a graphical editor for it. Given SLIM’s
57
PATTERNS: LOGIC-FREE SPECIFICATION OF REQ’S PROPERTY SPECIFICATION: PATTERNS, NOT FORMULAS!
Patterns
• The system shall have a behavior where 80 voltage 90 globallyholds.
• The system shall have a behavior where with probability higher than0.98 it is the case that voltage � 80 holds continously within time
bound [0, 10] .
|(by automatic transformation)
#
Logic
• ⇤(80 voltage 90) (Linear Temporal Logic)• P
>0.98
�⇤[0,10]
(voltage � 80)
�(Continuous Stochastic Logic)
2012, Viet Yen Nguyen 20/43
CASE STUDY MODEL Detailed version of this is confidential.
STUDY PARALLEL WITH SATELLITE DEVELOPMENT
ECSSȬMȬSTȬ10CȱRev.ȱ1ȱ6ȱMarchȱ2009ȱ
19ȱ
4.4 Project phasing
4.4.1 Introduction Theȱlifeȱcycleȱofȱspaceȱprojectsȱisȱtypicallyȱdividedȱintoȱ7ȱphases,ȱasȱfollows:ȱ
x Phaseȱ0ȱȬȱMissionȱanalysis/needsȱidentificationȱ
x PhaseȱAȱȬȱFeasibilityȱ
x PhaseȱBȱȬȱPreliminaryȱDefinitionȱ
x PhaseȱCȱȬȱDetailedȱDefinitionȱ
x PhaseȱDȱȬȱQualificationȱandȱProductionȱ
x PhaseȱEȱ–Utilizationȱ
x PhaseȱFȱ–ȱDisposalȱ
AȱtypicalȱprojectȱlifeȱcycleȱisȱillustratedȱinȱFigureȱ4Ȭ3.ȱȱ
Projectȱ phasesȱ areȱ closelyȱ linkedȱ toȱ activitiesȱ onȱ systemȱ andȱ productȱ level.ȱDependingȱ onȱ theȱ specificȱ circumstancesȱ ofȱ aȱ projectȱ andȱ theȱ acceptanceȱ ofȱinvolvedȱrisk,ȱactivitiesȱcanȱoverlapȱprojectȱphases.ȱ
Atȱ theȱ conclusionȱ ofȱ theȱ majorȱ activitiesȱ andȱ theȱ relatedȱ projectȱ reviewsȱconfigurationȱbaselinesȱareȱestablishedȱ(seeȱECSSȬMȬSTȬ40).ȱ
Figureȱ4Ȭ3:ȱTypicalȱprojectȱlifeȱcycleȱ
ȱ
Six months. Parallel with satellite development: 1. Nominal design
architecture matured. 2. FDIR architecture volatile
and in development. 3. Many details were
yet to be decided.
CASE: SATELLITE OF ESA MISSION IN DEVELOPMENT
• Platform keeps satellite in space, like car’s chassis. – control & data unit, – propulsion, – telemetry, tracking & cmd. – power, – attitude & orbit control sys., – reconfiguration module, – …
• Fault detection, isolation, recovery (FDIR) software and hardware: – redundancies + recovery, – compensation algorithms, – failure isolation schemes, – omnipresent in satellite.
Note: shown satellite is not the one of the case study.
FDIR
FDIR FDIR
FDIR
FDIR
FDIR
FDIR
FDIR
FDIR
DESCRIPTION: WHAT’S MODELLED - COVERAGE Most particularly: • Voting algorithms on hot
redundant elements. • Off-range checkers. • Initialization sequences. • Recovery sequences,
disabling/enabling system elements.
• Flags: • Status • Commandability
• Failure configurations. • Evolution of physical quantities:
• Time • Pressure • Heat
• Timing constraints.
Platform
AOCS
Gyro
RW
Earth Sensors
Sun Sensors
CDU Processors
TT&C Receivers
Transmitters
EPS Batteries
Solar Panels
LARGEST SYSTEM-LEVEL MODEL EVER DEVELOPED
MODEL AND REQUIREMENTS METRICS
Scope Metric Count
Model
Components 86Ports 937Modes 244Error models 20Recoveries 16Nominal state space 48421100LOC (without comments) 3831
Requirements
Propositional 25Absence 2Universality 1Response 14Probabilistic Invariance 1Probabilistic Existence 1
2011, Viet Yen Nguyen FOR INTERNAL USE ONLY CASE STUDY - 35/45
CASE STUDY ANALYSES All analysis results are in the publication.
INJECTING FAULTS EXPLODE THE STATE SPACE
22
14
10
8
8
7
5
2
1
172
11
1372
10
16
35
11
3
213563
1 10 100 1000 10000 100000 1000000
Reactionwheel + earthsensor failures
Complete earth sensorfailure
Processor modulefailures
Single reactionwheelfailure
All reactionwheel failures
AOCS equipmentsfailure
Propulsion failure
Double earth sensorssignal failure
Single earth sensorsignal failure Multiplication of state space
Number of injections
Explosion correlates with scope of failure.
COMPASS TOOLSET: MAIN WINDOW ANALYSIS WITH COMPASS TOOLSET: MAIN [BOZ+09B, BOZ+09C, BOZ+09D,
BOZ+10A, BOZ+11]
2013, Viet Yen Nguyen 10/14
Loading SLIM models
Adding fault injections
Adding requirements
Simulation states
BDD model checking of CTLformula for discrete case
SMT model checking of LTLformula for hybrid case
Counterexample whenproperty does not hold
Relate faults to higher-levelfailures e.g. “which faults leadto undervoltage?”
Fault tree contains Priority AND,AND, OR-gates and edges rep-resent triggers.
Computes probability of top-level event (e.g. system failure)
Fault tree to interactive Markovchain. Minimise to continuous-time Markov chain. Probabilisticmodel check it.
Determine effects on nominal modelupon combinations of failures.
“what happens when sensor dies?”
Determine system performancewhile under degraded modes.
Transforms full state space toa Markov chain. Probabilisticmodel check it.
Determine whether sufficientalarms are available to deducefaults.
Which alarms aretriggered on failure? Does the system recover from a failure?
COMPASS TOOLSET: SIMULATION ANALYSIS WITH COMPASS TOOLSET: SIMULATION
2013, Viet Yen Nguyen 10/14
Adding fault injections
Choose a transition
Simulation states
BDD model checking of CTLformula for discrete case
SMT model checking of LTLformula for hybrid case
Counterexample whenproperty does not hold
Relate faults to higher-levelfailures e.g. “which faults leadto undervoltage?”
Fault tree contains Priority AND,AND, OR-gates and edges rep-resent triggers.
Computes probability of top-level event (e.g. system failure)
Fault tree to interactive Markovchain. Minimise to continuous-time Markov chain. Probabilisticmodel check it.
Determine effects on nominal modelupon combinations of failures.
“what happens when sensor dies?”
Determine system performancewhile under degraded modes.
Transforms full state space toa Markov chain. Probabilisticmodel check it.
Determine whether sufficientalarms are available to deducefaults.
Which alarms aretriggered on failure? Does the system recover from a failure?
COMPASS TOOLSET: MODEL CHECKING ANALYSIS WITH COMPASS TOOLSET: MODEL CHECKING
2013, Viet Yen Nguyen 10/14
Adding fault injections
Simulation states
Verify a property e.g. “votingscheme on sensor valuesalways compensate doublesensor failures”
BDD model checking of CTLformula for discrete case
SMT model checking of LTLformula for hybrid case
Counterexample whenproperty does not hold
Relate faults to higher-levelfailures e.g. “which faults leadto undervoltage?”
Fault tree contains Priority AND,AND, OR-gates and edges rep-resent triggers.
Computes probability of top-level event (e.g. system failure)
Fault tree to interactive Markovchain. Minimise to continuous-time Markov chain. Probabilisticmodel check it.
Determine effects on nominal modelupon combinations of failures.
“what happens when sensor dies?”
Determine system performancewhile under degraded modes.
Transforms full state space toa Markov chain. Probabilisticmodel check it.
Determine whether sufficientalarms are available to deducefaults.
Which alarms aretriggered on failure? Does the system recover from a failure?
COMPASS TOOLSET: FAULT TREE ANALYSIS ANALYSIS WITH COMPASS TOOLSET: FAULT TREE ANALYSIS
2013, Viet Yen Nguyen 10/14
Adding fault injections
Simulation states
BDD model checking of CTLformula for discrete case
SMT model checking of LTLformula for hybrid case
Counterexample whenproperty does not hold
Relate faults to higher-levelfailures e.g. “which faults leadto undervoltage?”
Reachability analysiswith history variables Fault tree contains Priority AND,
AND, OR-gates and edges rep-resent triggers.
Computes probability of top-level event (e.g. system failure)
Fault tree to interactive Markovchain. Minimise to continuous-time Markov chain. Probabilisticmodel check it.
Determine effects on nominal modelupon combinations of failures.
“what happens when sensor dies?”
Determine system performancewhile under degraded modes.
Transforms full state space toa Markov chain. Probabilisticmodel check it.
Determine whether sufficientalarms are available to deducefaults.
Which alarms aretriggered on failure? Does the system recover from a failure?
COMPASS TOOLSET: PROBABILISTIC RISK ASSESSM. ANALYSIS WITH COMPASS TOOLSET: PROBABILISTIC RISKASSESSMENT
2013, Viet Yen Nguyen 10/14
Adding fault injections
Simulation states
BDD model checking of CTLformula for discrete case
SMT model checking of LTLformula for hybrid case
Counterexample whenproperty does not hold
Relate faults to higher-levelfailures e.g. “which faults leadto undervoltage?”
Fault tree contains Priority AND,AND, OR-gates and edges rep-resent triggers.
Computes probability of top-level event (e.g. system failure)
Fault tree to interactive Markovchain. Minimise to continuous-time Markov chain. Probabilisticmodel check it.
Determine effects on nominal modelupon combinations of failures.
“what happens when sensor dies?”
Determine system performancewhile under degraded modes.
Transforms full state space toa Markov chain. Probabilisticmodel check it.
Determine whether sufficientalarms are available to deducefaults.
Which alarms aretriggered on failure? Does the system recover from a failure?
COMPASS TOOLSET: FAILURE MODES & EFFECTS ANALYSIS WITH COMPASS TOOLSET: FAILURE MODES AND EFFECTSANALYSIS
2013, Viet Yen Nguyen 10/14
Adding fault injections
Simulation states
BDD model checking of CTLformula for discrete case
SMT model checking of LTLformula for hybrid case
Counterexample whenproperty does not hold
Relate faults to higher-levelfailures e.g. “which faults leadto undervoltage?”
Fault tree contains Priority AND,AND, OR-gates and edges rep-resent triggers.
Computes probability of top-level event (e.g. system failure)
Fault tree to interactive Markovchain. Minimise to continuous-time Markov chain. Probabilisticmodel check it.
Determine effects on nominal modelupon combinations of failures.
“what happens when sensor dies?”
“value range out of range!”
Determine system performancewhile under degraded modes.
Transforms full state space toa Markov chain. Probabilisticmodel check it.
Determine whether sufficientalarms are available to deducefaults.
Which alarms aretriggered on failure? Does the system recover from a failure?
COMPASS TOOLSET: PERFORMABILITY ANALYSIS WITH COMPASS TOOLSET: PERFORMABILITY
2013, Viet Yen Nguyen 10/14
Adding fault injections
Simulation states
BDD model checking of CTLformula for discrete case
SMT model checking of LTLformula for hybrid case
Counterexample whenproperty does not hold
Relate faults to higher-levelfailures e.g. “which faults leadto undervoltage?”
Fault tree contains Priority AND,AND, OR-gates and edges rep-resent triggers.
Computes probability of top-level event (e.g. system failure)
Fault tree to interactive Markovchain. Minimise to continuous-time Markov chain. Probabilisticmodel check it.
Determine effects on nominal modelupon combinations of failures.
“what happens when sensor dies?”
Determine system performancewhile under degraded modes.
Transforms full state space toa Markov chain. Probabilisticmodel check it.
Determine whether sufficientalarms are available to deducefaults.
Which alarms aretriggered on failure? Does the system recover from a failure?
COMPASS TOOLSET: DIAGNOSABILITY ANALYSIS WITH COMPASS TOOLSET: DIAGNOSABILITY
2013, Viet Yen Nguyen 10/14
Adding fault injections
Simulation states
BDD model checking of CTLformula for discrete case
SMT model checking of LTLformula for hybrid case
Counterexample whenproperty does not hold
Relate faults to higher-levelfailures e.g. “which faults leadto undervoltage?”
Fault tree contains Priority AND,AND, OR-gates and edges rep-resent triggers.
Computes probability of top-level event (e.g. system failure)
Fault tree to interactive Markovchain. Minimise to continuous-time Markov chain. Probabilisticmodel check it.
Determine effects on nominal modelupon combinations of failures.
“what happens when sensor dies?”
Determine system performancewhile under degraded modes.
Transforms full state space toa Markov chain. Probabilisticmodel check it.
Determine whether sufficientalarms are available to deducefaults.
Which alarms aretriggered on failure? Does the system recover from a failure?
COMPASS TOOLSET: FDIR EFFECTIVENESS ANALYSIS WITH COMPASS TOOLSET: FDIR EFFECTIVENESS
2013, Viet Yen Nguyen 10/14
Adding fault injections
Simulation states
BDD model checking of CTLformula for discrete case
SMT model checking of LTLformula for hybrid case
Counterexample whenproperty does not hold
Relate faults to higher-levelfailures e.g. “which faults leadto undervoltage?”
Fault tree contains Priority AND,AND, OR-gates and edges rep-resent triggers.
Computes probability of top-level event (e.g. system failure)
Fault tree to interactive Markovchain. Minimise to continuous-time Markov chain. Probabilisticmodel check it.
Determine effects on nominal modelupon combinations of failures.
“what happens when sensor dies?”
Determine system performancewhile under degraded modes.
Transforms full state space toa Markov chain. Probabilisticmodel check it.
Determine whether sufficientalarms are available to deducefaults.
Which alarms aretriggered on failure?
Which failures trigger an alarm?
Does the system recover from a failure?
EPILOGUE
FACT SHEET OF THE COMPASS TOOLSET FACTS SHEET OF THE COMPASS TOOLSET [Boz+11]
• Reused core software NuSMV [Cim+02], MRMC [Kat+11], Sigref [Wim+06]
• Subcontractors FBK (Italy), Thales (France), Ellidiss (France)• Budget ⇡ 750,000 Euro• Programmers ⇡ 12 persons• Lines of code ⇡ 100,000 lines of Python• Development time ⇡ 3 years• Reported Trac issues � 500• Used languages Python, C, C++• Licenses
• COMPASS License distribution in ESA member-states only• GNU Public License open source• FBK License binary-only non-distributable• CGM License binary-only non-distributable non-commercial-use• US transfer license for use by a specific US-based organisation
• Website http://compass.informatik.rwth-aachen.de/
2012, Viet Yen Nguyen 26/43
BEYOND TODAY: ESA AND EU FUND FOLLOW-UPS
• ESA projects – AUTOGEF automated synthesis of FDIR logic – FAME methodology for FDIR development lifecycle – HASDEL enhancements for launcher/rocket development
• EU projects – D-MILS security aspects in system-level architectures
These project’s budgets account for over 4 million EUR.
CONCLUSION
• Modeling language – Based on AADL industry standard – Graphical and textual – Formal semantics
• Discrete • Real-time • Hybrid • Probabilistic
• Extensive validation approach – Correctness – Safety & dependability – Performability – FDIR effectiveness
• Engineer-friendly toolset – State of the art (probabilistic) model
checking.
• Spacecraft case studies – Satellite platforms
• Phase B • Phase C
– Satellite subsystems • FDIR mode management • Thermal regulation system
• Follow-up projects – Synthesis of FDIR logic – Methodology and development
lifecycle of FDIR systems – Launcher/rocket development – Security architectures
State-of-the-art of validating spacecraft software designs for correctness, safety, dependability and performance aspects using a single system model.
MORE INFORMATION • M.-A. Esteve, J.-P. Katoen, V.Y. Nguyen, B. Postma and Y. Yushtein.
Formal Correctness, Safety, Dependability and Performance Analysis of a Satellite. In 34th International Conference on Software Engineering (ICSE). ACM & IEEE.
• COMPASS Toolset: compass.informatik.rwth-aachen.de– Freely available for everyone in ESA-member states.