Formal Model-Based Development in Aerospace Systems: Challenges to Adoption Mats P. E. Heimdahl...

Formal Model-Based Development in Aerospace Systems:Challenges to Adoption

Mats P. E. HeimdahlUniversity of Minnesota Software Engineering Center

Critical Systems Research GroupDepartment of Computer Science and Engineering

University of Minnesota

and a Plea for Help

Domain of Concern

How we Develop SoftwareConcept

Formation

Requirements Specification

Design

Implementation

Integration

System

Unit Test

Integration Test

System Test

Object Code

Test

Analysis

Model-Based Development

SpecificationModel

Visualization PrototypingTesting

Code

Analysis

Properties

Model-Based Development Tools

• Commercial Products– Esterel Studio and

SCADE Studio from Esterel Technologies

– Rhapsody from I-Logix– Simulink and Stateflow

from Mathworks Inc.– Rose Real-Time from

Rational– Etc. Etc.

SystemSpecification/Model

How we Will Develop SoftwareConcept

Formation

Requirements

Implementation

Integration

PropertiesAnalysi

s

Integration Test

System

Test

Specification Test

What Does Industry Want?

Better / SaferCheaperFaster

Model-Based Development Examples

Company Product Tools Specified & Autocoded Benefits Claimed

Airbus A340 SCADE With Code Generator

70% Fly-by-wire Controls 70% Automatic Flight Controls 50% Display Computer 40% Warning & Maint Computer

20X Reduction in Errors Reduced Time to Market

Eurocopter EC-155/135 Autopilot

SCADE With Code Generator

90 % of Autopilot

50% Reduction in Cycle Time

GE & Lockheed Martin

FADEDC Engine Controls

ADI Beacon Not Stated

Reduction in Errors 50% Reduction in Cycle Time Decreased Cost

Schneider Electric

Nuclear Power Plant Safety Control


200,000 SLOC Auto Generated from 1,200 Design Views

8X Reduction in Errors while Complexity Increased 4x

US Spaceware

DCX Rocket MATRIXx Not Stated

50-75% Reduction in Cost Reduced Schedule & Risk

PSA Electrical Management System


50% SLOC Auto Generated 60% Reduction in Cycle Time 5X Reduction in Errors

CSEE Transport

Subway Signaling System


80,000 C SLOC Auto Generated Improved Productivity from 20 to 300 SLOC/day

Honeywell Commercial Aviation Systems

Primus Epic Flight Control System

MATLAB Simulink

60% Automatic Flight Controls 5X Increase in Productivity No Coding Errors Received FAA Certification

Problem 1Believing Testing Can be Eliminated

Testing will always be a crucial (and costly) component

How we Develop SoftwareConcept

Formation

Requirements Specification

Design

Implementation

Integration

System

Unit Test

Integration Test

System Test

AnalysisObject Code

Test


Testing Does not go AwayConcept

Formation

Requirements

Implementation

Integration

Properties

Extensive Testing (MC/DC)


It Simply MovesConcept

Formation

Requirements

Implementation

Integration

Properties

Extensive Testing (MC/DC)


Do it the Right WayConcept

Formation

Requirements

Implementation

Integration

PropertiesAnalysi

s

Integration Test

System

Test

Specification Test

Unit Test

Example: ADGS-2100 Adaptive Display & Guidance System

RequirementDrive the Maximum Number of Display Units

Given the Available Graphics Processors

Counterexample Found in 5 Seconds!

Checking 573 PropertiesFound 98 Errors

883 Subsystems

9,772 Simulink Blocks

2.9 x 1052 Reachable States

Remedy

• Be honest about the capabilities of model-based development and formal methods– Done right, provides outstanding requirements,

models, analysis, etc., etc. – May greatly reduce the effort spent in testing

Problem 2Believing the Model is Everything

The model is never enough

Modeling is so much

fun

Properties

Specification/Model

Modeling FrenzyConcept

Formation

Requirements

Implementation

IntegrationHow do we know the model

is “right”?

Headfirst into m

odeling

System


Do it the Right WayConcept

Formation

Requirements

Implementation

Integration

PropertiesAnalysi

s

Integration Test

System

Test

Specification Test

Unit Test

Remedies

• Recognize the Role of Software Requirements– The model is not everything

• Development Methods for Model-Based Development Badly Needed– Model-Based Software Development Process

• Develop Tools and Techniques for Model, Properties, and Requirements Management

• Develop Inspection Checklists and Style Guidelines for Models

Problem 3Trusting Verification

To really mess things up,you need formal verification

Model Checking Process

Does the systemhave property X?

Model

Engineer

SMV

Automatic TranslationSMV Properties

Properties

Automated Check

Yes!

SMVSpec.

Automatic Translation

Model Checking Process


Model

Engineer

SMV


Properties

SMVSpec.


Counter Example

Automated Check

No!

Property or Model: Who is Right?

AG(Onside_FD_On -> Mode_Annunciations_On)

The Mode Annunciations shall be turned onwhen the Flight Director is turned on

AG( (Is_This_Side_Active & Onside_FD_On) -> Mode_Annunciations_On)

If this side is active, the Mode Annunciations shall be turned on when the Flight Director is turned on

If this side is active and the Mode Annunciations are off, the Mode Annunciations shall be turned on when the Flight Director is turned on

AG( ! Mode_Annunciations_On -> AX ((Is_This_Side_Active & Onside_FD_On) -> Mode_Annunciations_On)))

Translated All the “Shalls” into SMV Properties

Analysis Process Steps• All properties verified (!),

or…• Counterexamples found for

some properties • Simulate counterexample in

MBD environment and make corrections to:– model

– properties

– requirements

– assumptions (invariants)

Formal Analysis Model

MBD Model

Shall Statements

CTL Properties

Corrections Corrections

CorrectionsCorrections

Create Model(Manual)

Formalize Properties (Manual)

Merge(Automated)

Translate(Automated)

Simulation / Corrections

Formal Verification

Remedies• Develop techniques to determine adequacy of model and

property set– How do we know they are any “good”

• Techniques for management of invariants– How do we validate the assumptions we make

• Methodology and guidance badly needed – Tools with training wheels– “Verification for Dummies”

All we need is one high-profile verified systemto fail spectacularly to set us back

a decade or more

Model Checking ProcessWhy? Guru


Model

Engineer

SMV


Properties

SMVSpec.


Out to Lunch?

Problem 4Believing One Tool Will Be Enough

To be effective, we need a suite of notations and analysis tools

(and the ability to continually integrate new ones)

Original Tool Chain

RSML-e

NuSMV Model Checker

PVS Theorem Prover

Rockwell Collins/U of Minnesota

SRI International

RSML-e to NuSMVTranslator

RSML-e to PVSTranslator

Conversion to SCADE

DesignVerifier

SCADE

Lustre

NuSMV

PVSSafe StateMachines

Simulink

SimulinkGateway

StateFlow

SPY

Esterel Technologies

MathWorks

University of Minnesota/Rockwell Collins (NASA LaRC Funded)

University of Minnesota (NASA IV&V Funded)

Reactive Systems

Esterel Technologies

MathWorks

SRI International

University of Minnesota/Rockwell Collins (NASA LaRC)

University of Minnesota (NASA IV&V)

Current(?) Tool Status

DesignVerifier

SCADE

Lustre

NuSMV

PVS

Safe StateMachines

SAL

ICS

SymbolicModel Checker

BoundedModel Checker

Infinite Model Checker

Simulink

SimulinkGateway

StateFlow

Reactis

SPY

Three Conjectures

• No one modeling language will be universally accepted, nor universally applicable

• No one verification/validation tool will satisfy the analysis needs of a user

• Languages and tools must be tested on real world problems by practicing engineers– Preferably in commercial tools

Translation – with no ILEffort = m * n High quality translations

Lustre ++

polytables

SCADE

RSML-e

PVS

poly’

SMV

C

m modeling languages n target languages

poly

Translation – with ILEffort = m + n Low quality translations

Lustre IL

Lustre ++

polytables

SCADE

RSML-e

PVS

poly’

SMV

C


poly

A Proposed Framework (Van Wyk)

• Based on techniques from extensible programming languages, specifically attribute grammars extended with forwarding.

• Hypothesis: – An extensible language may serve as a host language for

domain specific extensions (to construct new modeling languages),

– while forwarding enables the feasible construction of high quality translations from source specification languages to target analysis languages.

• Provided to spur discussion only! There may be better solutions.

Translation – with lang. exts.Effort = m + n + Σ t I High quality translations

Lustre Host

Lustre ++

polytables

SCADE

RSML-e

PVS

poly’

SMV

C


forwarding

poly

pvs_trans (t2)

pvs_trans (t1)

c_trans (t3)

forwarding

forwarding

c_trans

smv_trans

pvs_trans

Remedies

• Next generation tools must allow easy extension and modification of notations to meet domain specific needs

• They must allow easy construction of high-quality translations from modeling notations to analysis tools

• They also must enable controlled reuse of tool infrastructure to make tool extensions cost effective

Problem Summary

• Believing Testing Can be Eliminated

• Believing the Model is Everything

• Trusting Verification• Believing One Tool Will

Be Enough

Thank You

• Rockwell Collins– Steven Miller– Michael Whalen– Alan Tribble– Michael Peterson

• NASA Langley– Ricky Butler– Kelly Hayhurst– Celeste Bellcastro

• NASA Ames– Michael Lowry

• NASA IV&V Facility– Kurt Woodham (L3-Titan)

• My Students at Minnesota– Anjali Joshi– Ajitha Rajan– Yunja Choi,– Sanjai Rayadurgam– Devaraj George– Dan O'Brien

Opinions in talk are mine.Do not blame the innocent.

Discussion

For More Information

• Michael W. Whalen et. al., Formal Validation of Avionics Software in a Model-Based Development Process, Formal Methods in Industrial Critical Systems (FMICS’2007), July 2007.

• Steven P. Miller, Alan C. Tribble, Michael W. Whalen, Mats P. E. Heimdahl, Providing the Shalls, International Journal on Software Tools for Technology Transfer (STTT), Feb 2006.

• Michael W. Whalen, John D. Innis, Steven P. Miller, and Lucas G. Wagner, ADGS-2100 Adaptive Display & Guidance System, NASA Contractor Report NASA-2006-CR213952, Feb. 2006. Available at http://hdl.handle.net/2002/16162.

• A lot of good reading at http://shemesh.larc.nasa.gov/fm/fm-collins-intro.html

• Eric Van Wyk and Mats Heimdahl. Flexibility in modeling languages and tools: A Call to Arms. To appear in Software Tools for Technology Transfer.

http://hdl.handle.net/2002/16162

http://shemesh.larc.nasa.gov/fm/fm-collins-intro.html

Formal Model-Based Development in Aerospace Systems: Challenges to Adoption Mats P. E. Heimdahl...

Documents

Transcript of Formal Model-Based Development in Aerospace Systems: Challenges to Adoption Mats P. E. Heimdahl...