Aviation Software Fundamentals course slides#1 › DASP › Docs › Regulations ›...

28/02/2020

1

UNCLASSIFIED

UNCLASSIFIED

Aviation Software Certification - Introduction Directorate of Initial Airworthiness (DIA-DASA)

About this Course

• It’s an introduction only• Assumed knowledge:

– little, if anything, about aviation software– you know that software exists and that it is important to the ADF– it is a bonus if you have undertaken System Safety training

• This course will not make you an expert• Course objective is to provide you with basic knowledge to enable you

to:– Appreciate the peculiarities of Aviation Software to be a smarter

customer – Communicate with software engineers on relevant issues– Know where to find guidance on Aviation Software

2

UNCLASSIFIED

UNCLASSIFIED

UNCLASSIFIED

UNCLASSIFIED

Agenda

• Introduction to Aviation Software• Software Integrity Principles• Software Assurance Techniques and Principles• Software Type Design Approvals• Authority Benchmarks for Approval• Authority LOI• Good Practice Problem Reporting• In-service Software Management• Other Topics

– Aeronautical Data and Mission Planning– Electronic Flight Bags– Unmanned Aerial Systems

3

Admin

• Mobile phones• Evacuation Points• Toilets• Course Attendance List• Course Slides• Course Critiques (Training Reform)

4

UNCLASSIFIED

UNCLASSIFIED

28/02/2020

2

Background

• Our Background• Your Background

– Where do you work?– What is your level of software knowledge?– How much software do you touch in your current job?

5

UNCLASSIFIED

UNCLASSIFIED

Directorate of Initial Airworthiness Roles

• Certification of new acquisition aircraft and Major changes to type design

– Define Safety Benchmarks for Aircraft Design

– Processing Airworthiness Issue Papers (AwIP) and communicate risks to Command

– Authority Endorsement of Statement Of Intent and Usage (SOIU)

– Tender Evaluation Assistance

– Certification Programme (CP) and Type Certification Basis (TCB) Agreement

– Authority Recommendations for Military Type Certificate (MTC), Military Supplemental Type Certificate (MSTC), Military Restricted Type Certificate (MRTC) , Military Permit To Fly (MPTF)

6

UNCLASSIFIED

UNCLASSIFIED


• Design Services

– Design Technologies & Standards (Software Systems Integrity)

• Aviation Software

• Aeronautical Data

• Electronic Flight Bags (EFB)

• Mission Planning Systems (MPS)

– UAS Certification

– Aerodromes & Heliports

– Initial Airworthiness Regulation & Promotion

– Design Organisation Approvals

– Type Certification

7

UNCLASSIFIED

UNCLASSIFIED


• Education and Training

– Sponsor the following DASA Training Courses:

• Type Certification

• Aviation System Safety Engineering / Technical Risk Management

• Aviation Software

• E3 Management for Designers/Maintainers

• ADF Crash Protection

• Ship Aviation Interface Certification and Safety

8

UNCLASSIFIED

UNCLASSIFIED

28/02/2020

3

Software Systems Section

9

Software Specialist 1 (SS1)

Software Specialist 1A

Software Specialist 1B

Software Specialist 1C

UNCLASSIFIED

UNCLASSIFIED

Software Systems Roles

• Interpret, prescribe & apply airworthiness standards– AAP 7001.054 Section 2 Chapter 3 Aviation Software– DO-178C Software Considerations In Airborne Systems and Equipment

Certification– Align with international NAA/NMAA best practice

• Assess the competence of a software development organisation (MDOA)• Review Specs and SOIU• Assist with the Design Certification process, particularly:

– Review and provide agreement on Plans for Software Aspects of Certification or equivalent

– Review Aviation Software aspects of a Certification Programme– Inspection of Compliance Documents– Approval of Major Changes to Type Design involving Aviation Software– Training, Education and Promotion

10

UNCLASSIFIED

UNCLASSIFIED

Software Systems Roles

• Sponsor Training– Aviation Software Certification - Introduction (This course)– Aviation Software Certification – Intermediate – Master of Science in Software Engineering (18 months)

11

UNCLASSIFIED

UNCLASSIFIED

Questions?

12

UNCLASSIFIED

UNCLASSIFIED

28/02/2020

4

UNCLASSIFIED

UNCLASSIFIED

Introduction to Aviation SoftwareDirectorate of Initial Airworthiness (DIA-DASA)

UNCLASSIFIED

UNCLASSIFIED 14

Computer System

https://www.youtube.com/watch?v=xnyFYiK2rSY

UNCLASSIFIED

UNCLASSIFIED 15

Advantages

Flexibility

DurabilityWeight

Software

UNCLASSIFIED

UNCLASSIFIED 16

Source Lines of Code (SLOC)

• Source lines of code (SLOC) is a software metric used to measure the size of a computer program by counting the number of lines in the text of the program's source code.

• How many SLOC in the program below?

int main (){

int i; cout << "Please enter an integer value: "; cin >> i; cout << "The value you entered is " << i; cout << " and its double is " << i*2 << ".\n"; return 0;

}

28/02/2020

5

UNCLASSIFIED

UNCLASSIFIED 17Approx. 200 LOC

UNCLASSIFIED

UNCLASSIFIED

The Large Hadron Collider

18

Approx. 3 Million LOC

UNCLASSIFIED

UNCLASSIFIED

Android Lollipop (2015)

19


UNCLASSIFIED

UNCLASSIFIED 20


28/02/2020

6

UNCLASSIFIED

UNCLASSIFIED 21

If lines of code were printed out on an A4 sheet at 60 lines per page

UNCLASSIFIED

UNCLASSIFIED 22

The C-130J has more than 50 Software Configuration Items and Approx. 600,000 LOC

UNCLASSIFIED

UNCLASSIFIED 23

UNCLASSIFIED

UNCLASSIFIED 24

The Super Hornet has Approx. 1.1 Million LOC

28/02/2020

7

UNCLASSIFIED

UNCLASSIFIED 25

UNCLASSIFIED

UNCLASSIFIED 26

The F-22 Raptor has Approx. 2.2 MLOC

UNCLASSIFIED

UNCLASSIFIED 27

UNCLASSIFIED

UNCLASSIFIED 28

The F-35 Lightning II has Approx. 8 MLOC

28/02/2020

8

UNCLASSIFIED

UNCLASSIFIED 29

There are over 70 applications from 15 different vendors running on the Boeing 787 Dreamliner’s common core computer. It has Approx. 8 MLOC

UNCLASSIFIED

UNCLASSIFIED 30

F-35 & Boeing 787: 8 MLOC 144,000 pages standing at 48ft or 15m tall

This is approx. 4 x the amount of the F-22 Raptor

UNCLASSIFIED

UNCLASSIFIED

5

10

15

0

20

25

30

C-130J‘99

F-18E/F‘99

F-22‘05

B787‘11

F-35’16-18

31

UNCLASSIFIED

UNCLASSIFIED

Don’t forget…

• Aviation Software also includes technologies that are developed using software development paradigms…

• Firmware: “The combination of a hardware device and computer instructions or computer data that reside as read-only software on the hardware device” [12207 & 498]

• Firmware within avionics falls into the definition of Aviation Software and must not be forgotten

• Complex Electronic Hardware: “A hardware item is considered complex if a comprehensive combination of deterministic test and analyses cannot ensure correct functional performance under all foreseeable operating conditions with no anomalous behaviour.” [DO-254]

32

28/02/2020

9

UNCLASSIFIED

UNCLASSIFIED 33

FPGA / EEPROMs

DIS

PLA

YA

RIN

C 4

29Tr

ansm

itter

Line

Driv

er

Har

dwar

e Fa

ults

UNCLASSIFIED

UNCLASSIFIED

Why?

34

• Software is just logic based instruction• Digital Gates do the same thing• Take the following simple example:

A 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1

B 0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1

C 0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1

D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1

Z 0 0 0 1 0 0 0 1 0 0 0 1 1 1 1 1

Z = 0;If A == TRUE {

if B == TRUE {Z = 1;

}}If C == TRUE {

if D == TRUE {Z = 1;

}}

AB

CD

Z

UNCLASSIFIED

UNCLASSIFIED

Why?

• That simple example might be safety logic preventing false CMDS ejection

A 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1

B 0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1

C 0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1

D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1

Z 0 0 0 1 0 0 0 1 0 0 0 1 1 1 1 1

ARM

AUTOOFF

Press for Manual Fire

NO CURRENT THREATS

CMDS_Fire = 0;If ARM == TRUE {

if Manual_Fire == TRUE {CMDS_Fire = 1;

}}If AUTO == TRUE {

if RWR_Detect == TRUE {

CMDS_Fire = 1;}

}

ARMManual_Fire

AUTORWR_Detect

CMDS_Fire

35

UNCLASSIFIED

UNCLASSIFIED 36

Chaos Report 2015

28/02/2020

10

UNCLASSIFIED

UNCLASSIFIED

Why is software difficult

RequirementsComplexity

Continuity

Testability

Flexibility ?!?

37

UNCLASSIFIED

UNCLASSIFIED

Flexibility

Software (in theory)

CHANGE 1 CHANGE

2

Software (in practice)

CHANGE 3

CHANGE 4

CHANGE 5

CHANGE 6

Defects

Time

38

UNCLASSIFIED

UNCLASSIFIED

Complexity

39

UNCLASSIFIED

UNCLASSIFIED

Continuity

Electrical Power

Structural Strength

InputO

utpu

t Test

Predict

Test

Predict

Test

Predict

Software

40

28/02/2020

11

UNCLASSIFIED

UNCLASSIFIED

Testability

• 53 binary options• 2^53 combinations • 9007 trillion combinations!

• If it takes 1 second to write, execute and verify a test case; how long will it take to test this simple program?

• 2.856 Million years

41

UNCLASSIFIED

UNCLASSIFIED

Common causes of Software related accidents

Confusing reliability and safety

Relying on redundancy

Re-using software

Wrong requirements

42

UNCLASSIFIED

UNCLASSIFIED

Wrong requirements

• Lufthansa Flight 2904, A320, Warsaw, 1993– 2 dead, 54 injured, aircraft lost

• Braking Logic– Reverse Thrusters: only when

weight on both wheels (>12T)– Spoilers: only when wheel

speed >72 knots– Brakes: not until wheel speed

has reached a calculated value (variable)

Situation: anticipated cross wind and

standing water on runway

Lesson: The software functioned exactly as specified, but an accident still occurred.

43

UNCLASSIFIED

UNCLASSIFIED

Major causes of Software related accidents



Re-using software

Wrong requirements

44

28/02/2020

12

UNCLASSIFIED

UNCLASSIFIED

Ariane 5 Accident

45

https://www.youtube.com/watch?v=5tJPXYA0Nec

UNCLASSIFIED

UNCLASSIFIED

Software Re-use

• Arianne-5 reused an Inertial Reference System (IRS) from Arianne-4.

4

5 Both at 33s

Horizontal Distance

Altit

ude

46

UNCLASSIFIED

UNCLASSIFIED

Over-reliance on Redundancy

• The Ariane 5 accident report notes that according to the culture of the Ariane program, only random failures are addressed and they are primarily handled with redundancy.

IRS#1

IRS#2

47

UNCLASSIFIED

UNCLASSIFIED

Major causes of Software related accidents



Re-using software

Wrong requirements

48

28/02/2020

13

UNCLASSIFIED

UNCLASSIFIED

Software Configuration

• ECU Software installed / configured incorrectly

• Not tested in simulator due to project delays

49

https://www.youtube.com/watch?v=2DFCJ32ygIE

UNCLASSIFIED

UNCLASSIFIED

C130J & C27 J Software Load Control Issues

50

LRU with Same Part number for 2 different platformsDifferent Software for each Aircraft

Software Part Number not identified in LRU

How do you determine whether the LRU belongs to C130J or C27J?

UNCLASSIFIED

UNCLASSIFIED

Software ‘Silver Bullets’

Formal Methods

Metrics Extensive Testing

BEWARE: SOFTWARE SILVER BULLETS ≠ SAFE DEPENDABLE SOFTWARE

CASE tools

Development Standards

CMMI

Reviews and audits

51

UNCLASSIFIED

UNCLASSIFIED

The Four Software Airworthiness Requirements

Software Dependability Cube

52

28/02/2020

14

UNCLASSIFIED

UNCLASSIFIED 53

Questions?

Software Integrity PrinciplesDirectorate of Initial Airworthiness (DIA-DASA)

UNCLASSIFIED

UNCLASSIFIED

UNCLASSIFIED

UNCLASSIFIED


55


UNCLASSIFIED

UNCLASSIFIED

Available training in System Safety

• Other courses on System safety are provided by DASA:– DASA System Safety Engineering - Introduction (Level 1)– DASA System Safety Engineering Management – Basic

(Level 2)– DASA System Safety Engineering Management – Applied

(Level 3)

• Offerings under review this year

• Enquiries to [email protected]

56

28/02/2020

15

UNCLASSIFIED

UNCLASSIFIED

What is System Safety?

“A systematic, comprehensive evaluation of the implemented system to show that the relevant safety requirements are met”

‐ARP4754A‐

“System safety describes the state of materiel such that any hazards in the design have been identified and evaluated, and associated risks have been eliminated or minimised so far as is reasonably practicable.”

‐SSE Cse‐

57

UNCLASSIFIED

UNCLASSIFIED

What is System Safety?

Systematic approach:• Aircraft Functional Hazard

Assessment (FHA)• Aircraft Fault Tree Analyses (FTAs)• System FHAs• System FTAs• System Failure Modes and Effects

Analyses (FMEAs)• Item FTAs• Item FMEAs

58

UNCLASSIFIED

UNCLASSIFIED

Hazard Probability

• How do we meet Quantitative Safety Criteria given software fails systematically?

59

Hardware

Systematic Failures

Operating Time Statistically Predictable

Failures

Systematic Failures (1 or 0)SoftwareSystem Inputs

System Inputs

UNCLASSIFIED

UNCLASSIFIED

System Safety – Key Requirements

60

28/02/2020

16

UNCLASSIFIED

UNCLASSIFIED

Catastrophic Critical Marginal Negligible

Frequent 1 3 6 10

Probable 2 5 9 14

Occasional 4 8 13 17

Remote 7 12 16 19

Improbable 11 15 18 20

WE CAN’T ASSIGN PROBABILITY TO SOFTWARE HAZARDS, SO WE USE SOFTWARE ASSURANCE

61

Hazard Probability

Rigour applied is dependent on the severity of the failure condition associated with the failure of the software to perform its intended function

Covered shortly!

UNCLASSIFIED

UNCLASSIFIED

System Safety – Key Requirements

• System Safety requirements are detailed in the ADRM (Section 2 Chapter 2)

• 14 Essential requirements• Three key outcomes:

– Design safety objectives are:• Defined• Agreed by the Authority

– Design hazards are identified and controlled– Hazards identified during design and associated controls are

documented

62

UNCLASSIFIED

UNCLASSIFIED



63

UNCLASSIFIED

UNCLASSIFIED

What is Software Assurance?

The planned and systematic actions necessary to provide confidence and evidence that a product or process satisfies given requirements• Objectives• Checks and balances

The IdeaThe more important it is that the software should work, the more rigorous the development must be to make sure it does work.

64

28/02/2020

17

UNCLASSIFIED

UNCLASSIFIED

What is Software Assurance?

The planned and systematic actions necessary to provide confidence and evidence that a product or process satisfies given requirements• Objectives• Checks and balances

The IdeaThe more important it is that the software should work, the more rigorous the development must be to make sure it does work.

65

UNCLASSIFIED

UNCLASSIFIED

Increasing software assurance

Higher software assurance is achieved by increasing the degrees of rigour in:– verification of requirements– test coverage– configuration control– independence of software verification process– robustness testing– verification of compliance with standards– traceability

66

UNCLASSIFIED

UNCLASSIFIED


67


UNCLASSIFIED

UNCLASSIFIED

• Software Assurance provides confidence an item is correct to it’s requirements [Verification]. How do we confirm the requirements are correct to the intended safety goals [Validation].

• Software safety is the in-depth analysis of the software in a system that is conducted in order to identify software safety requirements to treat hazards.

• May result in derived requirements

Software Safety

68

28/02/2020

18

UNCLASSIFIED

UNCLASSIFIED

How do we do software safety?

– Is there a formal approach?– How do we understand software in the system context?– What other requirements could I possibly need?

69

UNCLASSIFIED

UNCLASSIFIED

Software Safety Process

Identify system

functions

Identify system level

functional failure modes

and associated severities

Identify potential software

failure modes

Define software safety

requirements to detect and handle or prevent

software failure modes

1 2 3 4Define generic

software safety

requirements

5

70

UNCLASSIFIED

UNCLASSIFIED

Software Safety Process

71

UNCLASSIFIED

UNCLASSIFIED

Software Safety Requirements

• Derived requirements from in-depth analysis of Software• Feeds back to System Safety• Generic requirements can be imported based on good practice &

experience:– Implement Lessons Learnt– Treat Common Issues– Not Constrained by Functionality– Check on Completeness of Software Hazard Analysis

72

28/02/2020

19

UNCLASSIFIED

UNCLASSIFIED

Generic Software Safety Requirements

• Inadvertent Jumps– The system shall detect inadvertent jumps within or into safety

critical functions; return the system to a safe state, and, if practical, perform diagnostics and fault isolation to determine the cause of the inadvertent jump.

• Decision Statements– Decision statements in safety-critical computing system functions

shall not rely on inputs of all ones or all zeroes, particularly when this information is obtained from external sensors.

73

UNCLASSIFIED

UNCLASSIFIED

Sources of Generic requirements

• JSSSEH – Joint Software System Safety Engineering Handbook

• Good Developers will have their own list (ie. ways to allocate memory, ways of assigning variables, not using certain language functions)

74

UNCLASSIFIED

UNCLASSIFIED

Questions?

75

Software Assurance TechniquesDirectorate of Initial Airworthiness (DIA-DASA)

28/02/2020

20

UNCLASSIFIED

UNCLASSIFIED

Contents

• Software Assurance techniques: – Software requirements– Software Design– Software Implementation– Verification– Configuration Management– Quality Assurance– Software tools– Model Based Development– Formal Methods

77

UNCLASSIFIED

UNCLASSIFIED

Requirements?

• Specification of “what” should be implemented

– How the system should behave

– Application Domain Information

– Constraints on the System’s Operation

– Specification of a system property or attribute

– Constraints on the development process

78

UNCLASSIFIED

UNCLASSIFIED

Requirements Engineering

• The activities involved in discovering, documenting, maintaining a set of requirements for a computer based system

• Systematic and repeatable techniques

79

UNCLASSIFIED

UNCLASSIFIED 80

Inputs and Outputs

MissionNeeds

DomainInformation Regulation

PreviousSystems

Information

Agreed RequirementsSystem Specification

System Models

Requirements Engineering

Process

28/02/2020

21

UNCLASSIFIED

UNCLASSIFIED

Requirements: The “Right” Way

• There is no magical answer or standard

• Some software development standards

• Requires a systematic approach to ensure “goodness” and quality

81

UNCLASSIFIED

UNCLASSIFIED

Example Requirement Qualities

• Atomic• Complete• Concise• Consistent Correct • Implementation free (not solutionised)

82

UNCLASSIFIED

UNCLASSIFIED

Where Do Requirements Hide

• Requirements Evidence:– System Specification– Aircraft Performance Specification– Functional and Performance Specification– Software Requirements Specification– System/Sub-System Design Document– Interface Requirements Specification– Operational Functional Documents– Prime Item Development Specification– Contractor Item Development Specification– DOORS– Requirements Traceability Matrix

83

UNCLASSIFIED

UNCLASSIFIED 84

Requirements Traceability

REQ1REQ2REQ3

.

.

System Specification

REQ1:1REQ1:2REQ1:3

.

.

High LevelRequirements

REQ1:1.1REQ1:1.2REQ1:1.3

.

.

Low LevelRequirements

#REQ1:1.1 CodeDO….LOOP WHILE (x=y)

Code

1 2 3 4TEST 1.1TEST 1.2TEST 1.3TEST 1.4.

5Test Points

28/02/2020

22

UNCLASSIFIED

UNCLASSIFIED

What Is A Software Review?

• A review of the work product, by one or more colleagues of the author, to evaluate the technical content and/or quality of the work

85

UNCLASSIFIED

UNCLASSIFIED

Review Spectrum

86

Ad Hoc

Desk Check

Walkthrough

Inspection

Level of Effort

UNCLASSIFIED

UNCLASSIFIED

Ad Hoc

• Informal reviews may often be unnecessarily expensive (because of time-wasting through lack of focus), and frequently provide a sense of security which is quite unjustified by the relatively small number of real defects found and repaired.

87

UNCLASSIFIED

UNCLASSIFIED

Desk Check

• Only one peer besides the author examines the work product. This is the cheapest review approach, but its usefulness is reliant upon the skills, knowledge and self discipline of the reviewers.

88

28/02/2020

23

UNCLASSIFIED

UNCLASSIFIED

Walkthrough

• The author leads members of the development team through a software product and the participants ask questions and make comments about defects.

89

UNCLASSIFIED

UNCLASSIFIED

Inspection

• Very formal type of peer review where the reviewers are following a well-defined process to find defects.

90

UNCLASSIFIED

UNCLASSIFIED

Why We Do It

91

UNCLASSIFIED

UNCLASSIFIED

What To Look At

92

Object Code

Software Architecture

Software Requirements

Software Design

Source Code

Test Evidence

28/02/2020

24

UNCLASSIFIED

UNCLASSIFIED

What To Look For

• Traceability. Does each higher level element trace to at least one lower level element and does each lower level element trace to at least one higher level element?

• Compliance. Do the requirements/design/code satisfy the required system functional, performance and safety requirements?

• Accuracy/Consistency. Are the requirements/design elements accurate or is there ambiguity? Is there any conflict between requirements or design elements?

• Conformance to Standards. Has the defined requirements or design language been used? Have dangerous design and coding practices been avoided? Does the code comply with the approved language subset?

• Compatible. Are there any conflicts between the software requirements or design and the target hardware (e.g. 16-bit vs. 32-bit).

• Verifiable. Is it possible to prove through test that a requirement has been satisfied or design element correctly implemented?

93

UNCLASSIFIED

UNCLASSIFIED

Evidence of Reviews

• Required: positive evidence from the review (that the reviewed item satisfies each criteria)

• A procedure that outlines the criteria that must be reviewed is usually sufficient

94

Review SheetItem Under Review: Module X Version: 1.1.12

Reviewers: A. SmithM. Jones

Date: 01 Feb 10

Findings:No issues found.

Checklist: The reviewed source code correctly implements the relevant design element

The reviewed source code conforms to the coding standard.

UNCLASSIFIED

UNCLASSIFIED

Software Reviews Summary

• Software reviews are an effective tool for identifying errors as early in the life cycle as possible

• Software reviews are required in order to comply with software assurance standards

• Software reviews must produce positive evidence of item compliance

95

UNCLASSIFIED

UNCLASSIFIED

Software Verification

96

Reviews

Analysis

Verification

Test

28/02/2020

25

UNCLASSIFIED

UNCLASSIFIED

Definition

• The evaluation of software by observing its execution

• “Testing is about choosing elements from the input space of the software being tested”

97

UNCLASSIFIED

UNCLASSIFIED

Software Testing Attitudes

98

No difference between

testing and debugging

12

34

Testing shows that the software

works

Testing shows that the software

doesn’t work

Testing reduces the risk of using the software

5Testing is a mental

discipline that helps everyone develop

higher quality software

BetterBetter

UNCLASSIFIED

UNCLASSIFIED

Good Practice

99

1 2 3 4

Define a test strategy

Carefully design tests for your system

Build a test environment

Track defects to ensure their resolution

5

Adopt tools to automate testing

Manage Testing Resources

UNCLASSIFIED

UNCLASSIFIED

Design Organisation Software CM Goals

• Planned and repeatable process• Software work products are identified and controlled• Changes to identified software work products are controlled• Development and verification is only performed using relevant software

baselines

100

28/02/2020

26

UNCLASSIFIED

UNCLASSIFIED

Benefits of CM

• Links between object code and the development and verification evidence that proves its acceptability.

• Assures that the software that is loaded to the target environment is the same as the software that was verified.

• Assures that developers will work from the correct version of data.• Prevents unintended changes to software or data.• Tracks problems through to resolution.

101

UNCLASSIFIED

UNCLASSIFIED

CM Key Tasks

• Establish a process (software configuration management plan or equivalent)

• Establish a library for repository for software baselines.

• Work products to be placed under CM should be identified.

• Change requests and problem reports are initiated, recorded, reviewed, approved and tracked in accordance with a documented and repeatable procedure.

• Changes to baselines are controlled in accordance with a documented and repeatable procedure.

• Release of software products is controlled in accordance with a documented procedure.

• Status of CIs is recorded IAW a documented procedure.• Keep records/evidence documenting SCM activities and baselines.• Software baseline audits are conducted according to a documented procedure.• Software life cycle environment configuration is controlled.

102

UNCLASSIFIED

UNCLASSIFIED

IBM Clear Case (example)

103

UNCLASSIFIED

UNCLASSIFIED

Software Quality Assurance

• An independent check that the developer has done what they said they would do

• Types of Involvement– Check that particular product complies with requirements.– Check that transition criteria have been satisfied.– Software Conformity Review.– Periodic Audits.

• QA Personnel must be independent enough to be effective (e.g. answer to management).

104

28/02/2020

27

UNCLASSIFIED

UNCLASSIFIED

Final Thoughts on QA

• QA rarely staffed with experienced software developers/coders, but are generally excellent at record keeping

• QA often not capable of negotiating with developers• QA and development need to be in a partnership• Project and senior management often back the developers and not the

QA team• QA people must be proactive, independent and “drivers”• Address QA at the project reviews

105

UNCLASSIFIED

UNCLASSIFIED



106

UNCLASSIFIED

UNCLASSIFIED 107

Software Development

• Your software development process must be sufficiently robust to achieve your software assurance activities.

• Further training on the linkage between Software Development Lifecycles and Software Assurance is available on the Intermediate Software Course.

UNCLASSIFIED

UNCLASSIFIED

Software Life Cycle

108

28/02/2020

28

UNCLASSIFIED

UNCLASSIFIED


109

UNCLASSIFIED

UNCLASSIFIED

System Requirement

s

Concept of Operations

Subsystem Design

System Construction

Subsystem Test

SystemTest

System Requirement

Software Requirement

Software Design

Code and Compile

Integration/ Unit Test

Software/ System Test

Software Engineering

Operational Use

Verify

Verify

Validate

Software Engineering ≠

Software Programming

Software engineering is the set of controls placed around software programming to assure the right product is produced.

110

UNCLASSIFIED

UNCLASSIFIED 111

Example Lifecycles: SpiralUNCLASSIFIED

UNCLASSIFIED

Example Lifecycles: Agile

112

28/02/2020

29

UNCLASSIFIED

UNCLASSIFIED

Example Lifecycles: Agile variants

113

UNCLASSIFIED

UNCLASSIFIED

Software Development Standards

• MIL-STD-498 - Software Development and Documentation

• IEEE/EIA 12207 - Standard for Information Technology -Software Life Cycle Processes

114

UNCLASSIFIED

UNCLASSIFIED

Questions?

115

Software Type Design ApprovalsDirectorate of Initial Airworthiness (DIA-DASA)

UNCLASSIFIED

UNCLASSIFIED

28/02/2020

30

UNCLASSIFIED

UNCLASSIFIED

What is Type Certification

117

“The process through which compliance with the airworthiness design requirements contained in the Type Certification Basis is established through the development of the Type Design to meet the operating roles and environment contained in the Statement of Operating Intent and Usage (SOIU)”

• Outcome: A Type Certificate is issued when sufficient design, test and certification activities necessary to establish compliance with specified airworthiness standards, in order to operate safely within approved roles and environments, have been completed.

UNCLASSIFIED

UNCLASSIFIED

Type Certification Course

• Deputy Director Type Certification• Director Initial Airworthiness

• Detailed overview of Type Certification concepts and DASA application and process.

• Contact: [email protected]

118

UNCLASSIFIED

UNCLASSIFIED

What is an MTC?

“Certificate issued by the Authority … that certifies the aircraft type design complies with the applicable Type Certification Basis when operated within the conditions and limitations specified on the associated Type Certificate Data Sheet”

• Aircraft must be on Defence register

• Defence Registered Aircraft (DRA) must have a valid Type Certificate before being authorised for flight operations (exception: ops under MPTF, some UAS)

• Aircraft must be operated in the roles authorised in OIP and TCDS (and defined in SOIU)

• Applicant is usually the acquisition PO

• An AwB is convened to provide a recommendation to issue (or not to issue) an MTC.

• Issued by DG DASA

• MTC is issued to a Defence organisation

119

UNCLASSIFIED

UNCLASSIFIED

Software Scope in Type Certification

• What aviation software is type certified– Software is regulated in DASRs as a “Part and

Appliance”– Parts and Appliances can be subject to Type

Certification– Scope of Parts and Appliances to which Type

Certification applies is defined in the EMACC– Typically term “Aviation Software” is used to describe

software installed on a Type Certified Product (Aircraft, Propeller, Engine)

120

28/02/2020

31

UNCLASSIFIED

UNCLASSIFIED

Is this Aviation Software?• Picture of in flight entertainment

121

Installed Software. Is it Type Certified?

YES. Subject to Authority prescribed benchmarks.

UNCLASSIFIED

UNCLASSIFIED

Is this Aviation Software?• Picture of Mission Planning System

122

Uninstalled Software. Is it Type Certified?

NO. Good Practice is described in the ADRM.

Note: Some Mission Planning Systems will have installed software components which are subject to Type Certification.

UNCLASSIFIED

UNCLASSIFIED

Is this Aviation Software?• Picture of EFB

123

Electronic Flight Bags (EFBs). Is it Type Certified?

MAYBE. Boundaries are described in the ADRM.

UNCLASSIFIED

UNCLASSIFIED 124

Simulators. Are they Type Certified?

NO. Requirements are described in DASR FSTD.

28/02/2020

32

UNCLASSIFIED

UNCLASSIFIED

Type Certification Process Overview

125125

• Produce/update SOIU

• Identify Applicant and design organisations • Agree TCB Airworthiness codes/standards Tailoring to standards (MCRIs) Command safety/capability position

• Agree Means of Compliance (CCL) Evidence requirements Relief through extant certifications (CRE Δ)

• Agree Authority evidence inspections (LOI)• Agree Declaration of Compliance

• Produce design • Update CPP (if required)

• Produce evidence of compliance Evaluations, inspections, ground tests, etc Flight test

• Authority Inspections• Declaration of Compliance

• Apply for certification or approval• Confirm Continued Airworthiness responsibility

Maintain TC

Start

Define Operating Intent

Authority Agreement (CPP)

Design

Compliance Demonstration

Submission

•Authority assessment (Project DoSA attestation, AwB, etc)

• Issue instrument or approve major change

Certification/Approval

Plan for Software Aspects of Certification

Software Accomplishment Summary, Version Description Document

UNCLASSIFIED

UNCLASSIFIED

Certification Program Plan Contents for Software Projects

• Generic Certification Programme requirements discussed on Type Certification Course

• The TCB, including applicable airworthiness design requirements and standards, exemptions, special conditions, equivalent level of safety, environmental protection, etc.

• Primary Certification Code – FAR 25.1301/1309? MIL-HDBK-516C Section 14 & 15?

• Means of Compliance - DO-178B/C?

• Tailored benchmark?

• MCRIs?

• The Compliance Documents that will be subject to inspection by the Authority or its delegate/s, the proposed Authority Level of Involvement (LOI)

• Software specific elements:

• Plan for Software Aspects of Certification (PSAC)

• Change Impact Analysis (CIA)

126

UNCLASSIFIED

UNCLASSIFIED

Organisation Responsibilities

• An Approved Military Design Organisation – Undertakes the design and ensures safety– Declares compliance to applicable Airworthiness

Requirements– Submits Compliance Documents to the Authority– Requests Approval of the change to type design

• Minor – can be approved if within MDOA scope• Major – submits request for approval to the Authority

127

UNCLASSIFIED

UNCLASSIFIED

Organisation Responsibilities

• The Authority– Assures compliance to applicable Airworthiness

Requirements– Conducts Inspections of Compliance Documents as part

of a defined Authority Level Of Involvement (LOI)– Approves design change requests certifying compliance

requirements have been met for the change to type design

128

28/02/2020

33

UNCLASSIFIED

UNCLASSIFIED

When do Software Certifications take place?

• Initial Certification (Acquisition)• In-Service, Changes to Type Design (DASR 21.A.91)

– Major Change to Type Design– Minor Change to Type Design

• DASR clearly establishes what constitutes a major change to software.

129

UNCLASSIFIED

UNCLASSIFIED

Classification of Software Change to Type Design

Appendix A to GM 21.A.91• Where a change is made to software produced in

accordance with the guidelines of EUROCAE ED12C/RTCA DO–178C 'Software Considerations in Airborne Systems and Equipment Certification', the change is to be classified as major if either of the following apply, and the failure effect is CATASTROPHIC, HAZARDOUS or MAJOR:– the executable code for software, determined to be Level A or Level

B in accordance with the guidelines, is changed unless that change involves only a variation of a parameter value within a range already verified for the previous certification standard; or

– the software is upgraded to or downgraded from Level A, Level B or Level C; or

– the executable code, determined to be Level C, is deeply changed, e.g. after a software re-engineering process accompanying a change of processor.

130

UNCLASSIFIED

UNCLASSIFIED

Questions?

131

Understanding Authority LOI for Aviation Software

In-service Problem Reporting

28/02/2020

34

UNCLASSIFIED

UNCLASSIFIED

Overview

• What is LOI?• Why you need to understand Authority LOI• Terminology• Assessing the requirement for Authority LOI for software Aspects• Conduct of Authority LOI for software aspects• Delegating Software LOI• Role of CVE vs Authority inspector for software audits

133

UNCLASSIFIED

UNCLASSIFIED

What is Level of Involvement

134

• What?– “Selection of the compliance demonstration activities and data

that the [Authority] will investigate and the depth of these investigations during the certification process” – EASA

• ‘Involvement’ in What?– In applicant assessments/Decision making process?– Involved in generating or compiling evidence showing

compliance?– Involvement in the Certification Programme.

• The Certification Programme presented to the Authority must include acceptable provisions to allow the Authority to review required Compliance Documents.

• How:– Negotiated set of evidences and activities in the CP, expanded

in the PSAC

UNCLASSIFIED

UNCLASSIFIED

What is Level of Involvement

135

UNCLASSIFIED

UNCLASSIFIED

Why you need to understand Authority LOI

• Programmatic risks can arise if requirements for Authority LOI are not able to be achieved due to limitations in existing contracts.

• Assist in the ease of development and negotiation of a Certification Programme.

• Prevent misunderstandings arising from role confusion. • Need to understand that the Authority may seek a range of evidences

and time up-front in your software development. • Sourcing software from foreign sub-contractors may often generate a

requirement to send DASA Software SMEs overseas, or require a considered and informed proposal to appoint a delegate.

136

28/02/2020

35

UNCLASSIFIED

UNCLASSIFIED

De-confusing terminology

137

• [TAREG] Compliance Finding: Meant different things to different people, may have referred to:– Authority inspections of selected evidence? – Authority

Inspection delegated from the TAR/DAR– Assessment of CRE deltas?– Involvement in the Design Process?– Supplies Acceptance functions?– Assessment of future Supportability?

• Compliance Demonstration– An activity by the applicant to confirm compliance of a design

to certification requirements. – May involve the creation, compilation, inspection or review of

compliance demonstration evidences.

UNCLASSIFIED

UNCLASSIFIED

De-confusing terminology

138

• Inspection of Compliance Documents: – An [Authority] Inspection of the Compliance Documents

generated by the applicant. – Includes evidence submitted for (or sampled/witnessed by)

Authority review during LOI, or submitted with applications for approval of a Major change (or generated internally for Minor changes).

• Level of Involvement– A review of Compliance Documents by the Authority during the

conduct of the applicant’s Certification Programme. – Identified LOI must be completed prior to applying for approval

of the Major change.

UNCLASSIFIED

UNCLASSIFIED

Assessing LOI

139

• The Authority is not resourced to review all affected compliance evidences for an application.

• A risk based decision made through:– An assessment of the likelihood of an unidentified non-

compliance in an application. Considering:• Novelty,• Complexity,• Organisational Performance

– The criticality (safety affect) of such a non-compliance• The use of a risk based approach, allows the authority to focus it’s

assurance role on the most important aspects of a design change. • The LOI approach is consistent with international convention and

best practice for Airworthiness regulation. • Ensure DASA does not waste reviewing trivial compliance

demonstration activities

UNCLASSIFIED

UNCLASSIFIED

LOI: Likelihood of an unidentified Non-compliance

140

• Step 1: Likelihood– Novel to who? – Complex to who?– Who assesses applicant performance?

28/02/2020

36

UNCLASSIFIED

UNCLASSIFIED

Recall: Oversight vs LOI

141

UNCLASSIFIED

UNCLASSIFIED

Consequence of an unidentified non-compliance

142

• Step 2: Criticality• Intuition DAL A or B Software will usually always require class 3 or

higher

UNCLASSIFIED

UNCLASSIFIED

Assessing LOI: Historical difficulties

143

• Design is already Mature prior to submitting a CPP• Applicant cannot provide key documentation such as the PSAC or

the Change Impact Analysis that is required to undertake the assessment.

• Perception that the PSAC is ‘accepted’ without being provided to the Authority.

• Insufficient data is submitted to assess the CPP: Authority is told requested data is ‘not in the contract’, or other difficulties.

• Not understanding different roles between a CVE Inspection and Authority inspection.

• Data requested by the Authority to complete LOI is ‘not available’.

UNCLASSIFIED

UNCLASSIFIED

So what? What do we look at?

144

• Depending on the project (and certification basis), it may not always make sense to look at the same information.

28/02/2020

37

UNCLASSIFIED

UNCLASSIFIED

LOI Typical Activities

145

• Why is involvement during the software lifecycle preferable:– Authority review may identify a requirement to generate

additional evidence or undertake actions where findings or deficiencies are identified.

– Authority Inspection is improved by a capacity to witness testing, participate in reviews and witness builds or conformity activities.

UNCLASSIFIED

UNCLASSIFIED


146

• As a result of these issues, it is preferable that the Authority is involved upfront and as early as possible, and at key points in the software lifecycle.

UNCLASSIFIED

UNCLASSIFIED


147

• There is no mandated policy on how the Authority is to achieve it’s audit outcomes, however where suitable the Authority will align with good international practice. – An alternative may be sought by the Authority or applicant,

depending on the particular situation, certification requirements, or lifecycle of a software item.

• International Practice: Stage Audits or inspections: “Stage of Involvement” or ‘SOI’– Terminology used by the FAA to describe a staged auditing

process– Originally described in the FAA Job Aid (Now historical)

describes an inspection process that might be used by: ‘Authorities’, ‘Designees’ [DERs], and ‘Applicants’.

– Continues to be used now internationally.

UNCLASSIFIED

UNCLASSIFIED

LOI Typical activities: planning audit (SOI 1)

148

1. Assure plans and standards meet DO-178B objectives and address other applicable software policy, guidance and issue papers.

2. Assure that the processes described in the applicant’s plans meet the objectives of DO-178B and address other applicable software policy, guidance, and issue papers

3. Obtain agreement between FAA and applicant on the plans, standards, and proposed methods of compliance

• Note: For Software Aspects, the Authority will typically seek to achieve some or all of the outcomes prior to approving the CPP.

28/02/2020

38

UNCLASSIFIED

UNCLASSIFIED

LOI Typical Activities – Development Audit (SOI 2)

149

1. Assess implementation of plans and standards for the software requirements, design, and code, and related verification, SQA, and SCM data.

2. Assess and agree to plans and standards changes.3. Assess implementation of new technology and methods to

ensure compliance to plans, standards, and agreements. 4. Assure life cycle data satisfies the certification standards

objectives and other applicable software policy, guidance and issue papers.

UNCLASSIFIED

UNCLASSIFIED

LOI Typical Activities – Verification Audit (SOI 3)

150

1. Assess implementation of verification and test plans and procedures.

2. Assess completion and compliance of all associated SCM and SQA tasks.

3. Ensure software requirements are verified.4. Ensure robustness testing is planned and is being

performed.5. Ensure analyses (including timing, memory, test coverage,

structural coverage and data and control coupling) are being performed as required by the certification standard.

6. Ensure verification activities satisfy the certification standards objectives.

UNCLASSIFIED

UNCLASSIFIED

LOI Typical Activities – Final Audit (SOI 4)

151

1. Assure final software product meets the certification standards objectives and is ready for certification.

2. Address any open items.

UNCLASSIFIED

UNCLASSIFIED

Focus Areas

152

– If there are novel or more critical aspect of the software design, then the Authority will typically focus it’s review on these components. eg:• Thread traces will be done against affected Safety Critical

Requirements. • Planning review may focus on design standards for novel

techniques. • Potentially a focus on the Change Impact Analysis and

regression testing where the modification to the Computer Software Configuration Item may affect existing safety critical requirements / hazards.

28/02/2020

39

UNCLASSIFIED

UNCLASSIFIED

What else may be audited by software inspectors

153

– Other authority sections have differing LOI requirements– Authority software SMEs may seek involvement in

development assurance objectives and focus on requirements validation activities.

– Authority will apply LOI similar to the above for changes to safety critical Complex Electronic Hardware upgrades.

– Minor Changes to type? – Other Authority approvals: i.e. AUSMTSO Authorisations,

UASOPs, ANSP Certificate, may have a similar process in the future.

UNCLASSIFIED

UNCLASSIFIED

Outcomes of Authority LOI

154

• Authority will provide a report at defined stages (perhaps linked to SOI stages) disclosing the outcomes of these involvement activities.

• Authority will provide a report once LOI is complete, advising that Authority LOI (for software aspects) has been closed.

• Applicant cannot submit a declaration of compliance to the Authority until all defined Authority LOI is executed – as the certification programme is not complete.

UNCLASSIFIED

UNCLASSIFIED

Authority LOI – feedback and findings

155

• Authority may pass on Product Certification Findings and Actions that will need to be closed prior to closing LOI. DIFFERENT FROM ORGANISATIONAL FINDINGS AND ACTIONS

• Decision to not action findings and actions will require production of an MCRI (ESF, exception) and DASA must be presented evidence that the elevated risk level has been accepted by command IAW the principles of the DASP.

• Authority may also pass on observations or issues (items related to other areas of the Certification Programme but not precluding closure of LOI)

UNCLASSIFIED

UNCLASSIFIED 156

(What we don’t want to see)

Authority LOI – feedback and findings

28/02/2020

40

UNCLASSIFIED

UNCLASSIFIED

Executing Authority LOI – Historical difficulties

157

• Nested documentation.– DASA will request a document, but the evidence targeted is in

a subordinate document. Seeking the evidence then causes a delay

• Notes taken by auditor become Intellectual Property of the applicant.

• International travel.• Form 31a submitted concurrently with evidence required to

complete Authority LOI.

UNCLASSIFIED

UNCLASSIFIED

Audits one by Compliance Verification Engineers = LOI?

158

• Short answer: No• Many CVEs may elect to follow (or follow a similar process) to the

SOI concept described earlier, to achieve their assessment. • This shared terminology can lead to significant confusion. • DASA may elect to attend a CVE ‘SOI audit’, rather than to run our

own on-site audit, to fulfil our assurance outcomes.

Applicant “SOI” Reports

UNCLASSIFIED

UNCLASSIFIED

Clarification: FAA DERs as CVEs

159

• Projects will sometimes reach back into parent companies (etc) and make use of individuals with privileges and competencies under different regulatory systems. An example of this are individuals with FAA DERs designation, while employed by a design organisation, are able to conduct Authority inspections on behalf of the FAA.

• While a DER (etc) qualified individual may be appointed as a CVE for a DASR MDOA; they are not conducting assessments on behalf of the Authority.

• No one is conducting assessments on behalf of the Authority unless they have been explicitly notified by the DASA that they are fulfilling this role.

• Intuition: When contracted as a CVE, DERs and other highly competent individuals are not exercising an Authority delegation. However, when such an individual is contracted (and this is outlined to the Authority in the CP), this may result in lower Authority LOI.

UNCLASSIFIED

UNCLASSIFIED

Questions

160

28/02/2020

41

Good Practice for Problem ReportingDirectorate of Initial Airworthiness (DIA-DASA)

UNCLASSIFIED

UNCLASSIFIED

UNCLASSIFIED

UNCLASSIFIED 162

Problem Reporting: An overview

• Development Lifecycle: Problem Report Management• Management of Open Problem Reports• Linkages between In-service anomalies, Occurrence Reporting

(potentially unsafe conditions) under DASR 21.A.3A and Problem Reporting

• When does a Problem Report require the Authority to issue an Airworthiness Directive under DASR 21.A.3B(b)

UNCLASSIFIED

UNCLASSIFIED 163

Problem Reporting, defining the problem.

• Problem reporting is one of the most important Software Configuration Management activities. – Multiple stakeholders, terminology (classification systems) and

cross domain. – Usually linked to/generates a Change request and is heavily tied in

to CM activities. • Linkage to problems identified in-service• Impacts Holder obligations and Occurrence Reporting system

UNCLASSIFIED

UNCLASSIFIED

Known and

unknownanomalies

Anomalies Identified and RemovedDuring Test

Anomalies Identified and Resolved During Design

Anomalies w

ithin Software over single developm

ent cycle

Problems / Anomalies

Software Assurance moves this line upwards

Increased Rigour identifies and resolvesanomalies or systematic failures that

result from design errors and faults

Anomaliesresident in

operatingsoftware

164

28/02/2020

42

UNCLASSIFIED

UNCLASSIFIED 165

Development Problem Reporting – States and Classification

• The below terminology is the preferred terminology in upcoming FAA/EASA guidance:– Classification schemes

• Potential Safety: assessed at the product, system, or equipment level [More detail later in this section], a PR having an actual potential Catastrophic, Hazardous or Major safety effect on the aircraft, or affecting compliance with operating rules.

• Functional: A PR recording a process non-compliance, deficiency or deviation that cannot result in a potential safety, nor potential functional, impact.

• Documentary: a PR linked to a deficiency in a life cycle data item but not linked to a process deficiency or deviation. This includes typographical or editorial defects in life cycle documents.

• Other: a PR having no potential safety impact, no potential functional impact, and not linked to a process deficiency or a deviation or to a documentary deficiency

UNCLASSIFIED

UNCLASSIFIED 166

• Different stakeholders for different items

• Complex interfaces between aircraft systems.

• All stakeholders have a role in the In-service Problem Reporting Change

• Stakeholders should consider their view when classifying problem reports (i.e. stakeholders at different levels will have a different position over whether a PR is potential safety)

Development Problem Reporting – Stakeholders

UNCLASSIFIED

UNCLASSIFIED 167

• The applicant (i.e. the top level stakeholder), at the time of certification, should have dispositioned all problem reports. All problem reports with a potential safety impact must be resolved.

Development Problem Reporting – StakeholdersUNCLASSIFIED

UNCLASSIFIED 168

• Due to Capability imperatives, projects may seek to advance with “Open” PRs.

• Good Practice:– Good Practice for Problem Reporting is available in EASA AMC 20-

189– Good practice requires the production of an OPR Summary report

(to higher stakeholders or the cert authority) identifying:• The OPR, the affected configuration item(s), summary of the

OPR (understandable by higher level stakeholders), a description of the problem understandable by higher level stakeholders.

Development Problem Reporting – OPRs and certification

28/02/2020

43

UNCLASSIFIED

UNCLASSIFIED

In-Service Problem Reporting

169

UNCLASSIFIED

UNCLASSIFIED 170


• Anomalies or occurrences encountered in-service may require design organisations to raise and track problem reports

• Per DASR 21.A.3A(a), the Holder must: “establish a system or network of systems to receive information about occurrences”.

• Problem Reports should be provided to applicable design organisations/OEMs for assistance in assessments, and tracking at an appropriate level.

• Recall: All Open Problem reports (i.e. including those captured during in-service operations), with a potential safety impact on the proposed installation must be resolved for a future modification.

UNCLASSIFIED

UNCLASSIFIED

• The MTC Holder must have a system to carry out the following:– Collect

• Establish a system or network of systems to receive information about occurrences

– Assess/Analyse • Determine whether the occurrence is reportable to DASA• Determine what the occurrence means from a type design

Perspective i.e. whether it is or may be an unsafe condition– Report

• Report to DASA in a Form and Manner stipulated by DASA for reportable and/or unsafe occurrences

– Investigate• Investigate and determine the corrective actions required

171

In-service Problem Reporting and Unsafe ConditionsUNCLASSIFIED

UNCLASSIFIED 172

• Following the identification of an occurrence related to a software/AEH item, the MTC Holder in conjunction with any other the stakeholders should:– Record the PR. – Classify the PR (i.e. classify as ‘Potential Safety’)– Assess whether the occurrence, is a reportable occurrence per

DASR 21.A.3A(b), and report if necessary (potentially unsafe condition is identified) to the Authority and relevant stakeholders

– Actions to resolve any unsafe condition• The Authority, will make a determination regarding whether the situation

disclosed in the Occurrence Report necessitates the issuance of an Airworthiness Directive; to mandate additional actions to mitigate / resolve an unsafe condition.

• [INTUITION]: A ‘Potential Safety’ PR, is not automatically an unsafe condition, but is a reportable occurrence.


28/02/2020

44

UNCLASSIFIED

UNCLASSIFIED 173

Questions?

In-Service CM and Load ControlDirectorate of Initial Airworthiness (DIA-DASA)

UNCLASSIFIED

UNCLASSIFIED

UNCLASSIFIED

UNCLASSIFIED

Software Load Control – In-Service

175

UNCLASSIFIED

UNCLASSIFIED

Software Load Control

• Design Organisation, CAMO and Maintenance Organisations – All are responsible

• Software Load Control: Controls must be established to ensure the aviation software configuration is: – approved for installation; – installed in the required configuration; and – verified to be installed correctly.

• DASR 21– 21.A.165 – Obligations of the holder

• CAMO– M.A.201 – Responsibilities

• 145 (AMO)– 145.A.42 – Acceptance of components– 145.A.48 – Performance of Maintenance

176

28/02/2020

45

UNCLASSIFIED

UNCLASSIFIED

C-130J & C-27J Software Load Control Issues

177

LRU with Same Part number for 2 different platformsDifferent Software for each Aircraft

Software Part Number not identified in LRU

How do you determine whether the LRU belongs to C-130J or C-27J?

UNCLASSIFIED

UNCLASSIFIED

On-board interrogation

• “Smart” aircraft may have on-board interrogation capability or a Memory Loader Verifier allowing maintainers to establish current aircraft software configuration during maintenance or when loading software

178

UNCLASSIFIED

UNCLASSIFIED

No interrogation capability

• Legacy aircraft may need to be verified via other means such as an OEM report/attestation from OEM

• There is a need to find out what software is in boxes that return from an OEM to assure that the only the authorised software configuration is installed

179

UNCLASSIFIED

UNCLASSIFIED

Production Approval

• DASR 21.A.307, requires an aviation software item to have an Authorised Release certificate (i.e. DASR Form 1, or equivalent). – This Form 1 certifies that the item was manufactured in conformity

to approved design data and is marked in accordance with DASR 21 Subpart Q.

– DASR 21, Subpart F, and G provide guidance on acquiring a production organisation approval, or producing items without one.

– DASR recognition provides guidance on equivalent instruments.

180

28/02/2020

46

UNCLASSIFIED

UNCLASSIFIED

Aircraft Software Management

• Software may come from– MTC Holder and associated design organisations. – User Modifiable Software (more later)– Vendor supplied. i.e.

• Navigation and Terrain databases• In flight entertainments systems.

• May be loaded on– Removable media– Portable Electronic devices– Ground based servers. – Automatic Test Equipment.

181

UNCLASSIFIED

UNCLASSIFIED

In-Service Software Management:

• Refer to FAA AC 43-216 for the latest guidance.

182

UNCLASSIFIED

UNCLASSIFIED

Special Topics: Field Loadable Software

• Field Loadable software reduces aircraft down-time for maintenance and increases efficiency of maintaining airborne equipment.

• The aircraft system must be designed to allow a software item to be ‘field-loadable’ per DO-178. – This includes in-built protections against corruption or partial

loading at an integrity level appropriate for the FLS. – Protection methods must be implemented to prevent inadvertent

enabling of the field-loading function during cruising or any other safety-critical phase.

– Further guidance is provided in DO-178C section 2.5.5

183

UNCLASSIFIED

UNCLASSIFIED

Special Topics: User Modifiable Software

• User Modifiable software component is that part of the software that may be changed by the user within the modification constraints without certification authority review, If the system requirements provide for user modification.

• User modifiable software must be designed for. Further guidance is available in DO-178C 2.5.2

184

28/02/2020

47

UNCLASSIFIED

UNCLASSIFIED

Special Topics: Cybersecurity (Developing)

• Aircraft Network Security will add additional Software Maintenance management and load control considerations

• Good practice is found in RTCA DO-355• Stay tuned for further developments.

185

UNCLASSIFIED

UNCLASSIFIED

Questions?

186

Aeronautical Data and Mission PlanningDirectorate of Initial Airworthiness (DIA-DASA)

UNCLASSIFIED

UNCLASSIFIED

UNCLASSIFIED

UNCLASSIFIED

What is a mission planning system?

• What is a Mission Planning System:– A suite of software applications and associated

hardware that allow maps, charts, weather, intelligence and aircraft performance data to be used in developing navigation solutions, communication settings, flight/mission calculations, etc.

• Examples:– Portable Flight Planning System– Joint Mission Planning System– F35 ALIS

188

28/02/2020

48

UNCLASSIFIED

UNCLASSIFIED

F-35 ALIS

189

UNCLASSIFIED

UNCLASSIFIED

Is this a good idea?

190

Mission Planning and Google Earth?

UNCLASSIFIED

UNCLASSIFIED

Technical Aspects of MPS

• There are hazards associated with MPS that could propagate to the aircraft safety:– Databases may have errors.– Calculations can be incorrect.– Data could be corrupted during transfer.

• MPS must be assured commensurate with the severity of associated hazards.– No different to any other system with safety implications.

• Knowledge of operational processes is required to determine how severe the hazards are.

191

UNCLASSIFIED

UNCLASSIFIED

Technical Aspects of MPS

• DASA position is that MPS are not directly impacting airworthiness– MPS no longer to be in the TCBs– DASA does not certify MPS– Responsibility lies with MAO, through CASG ( Acq & Sus SPOs)

• Important enough that DASA provides guidance– AeroData managed separately, through DASR.ANSPs– ADRM chapter (Synergies with EFB)– Factsheet

192

28/02/2020

49

UNCLASSIFIED

UNCLASSIFIED

Common MPS Hazards

• Erroneous Weight and Balance Calculations• Erroneous Take Off and Landing Data Calculations• Erroneous Fuel Usage Calculations• Incorrect Translation of Waypoint Map Coordinates• Erroneous Data Packages for Guided Weapons• Introduction of Errors into Aeronautical Data

– Transfer, Formatting or Translation– Digital Maps– Navigation Databases– Airfield Databases– Digital Terrain Elevation Data– Obstacle Data

193

UNCLASSIFIED

UNCLASSIFIED

Aeronautical Data Processing

• For each link, assure that:

– errors are not introduced, or

– errors will be detected.

194

Mission Planning System

Database

Real World Data

Reformat

e.g. DAFIF, DTED, Jeppesen

W&BCalculations

(Generate)

TakeoffData

(Generate)

AirfieldDatabase(Transfer)

ChecksumChecksum

Checksum

Assurance

Assurance

Reversal Check

UNCLASSIFIED

UNCLASSIFIED

The Process

• System Safety – use the 7 step Risk Management Process– Identify the data processed by the MPS.– Identify the data criticality and data assurance level.– Identify each MPS component that processes each data

element.– Identify what function that each MPS component

performs for each data element.• Transfer, Format, Manipulation, Generation

195

UNCLASSIFIED

UNCLASSIFIED

The Process

• System Safety – use the 7 step Risk Management Process– Define and Implement Treatments to assure the

detection and handling or absence of errors.• OpEval (Validation)• Increase assurance• Ensure data comes from an authorised source• Verification Tools• Operational restrictions (Refer SIs/Standards

Manual)

196

28/02/2020

50

UNCLASSIFIED

UNCLASSIFIED

Criticality

197

Severity DAL Data Criticality

Definition Data Assurance

LevelCatastrophic A Critical The data, if erroneous, would prevent continued safe

flight and landing or would reduce the ability to cope with adverse operating conditions to the extent that there is a large reduction in safety margins or functional capabilities. There is a high probability when trying to use corrupted critical data that an aircraft would be placed in a life threatening situation.

1

Hazardous B Critical 1

Major C Essential The data, if erroneous, would reduce the ability to cope with adverse operating conditions to the extent that there is a significant reduction in safety margins. There is a low probability when trying to use corrupted essential data that an aircraft would be placed in a life threatening situation.

2

Minor D Essential 2

No Safety Effect

E Routine The data, if erroneous, would not significantly reduce aircraft safety. There is a very low probability when trying to use corrupted routine data that an aircraft would be placed in a life threatening situation.

3

UNCLASSIFIED

UNCLASSIFIED

Treatment Options

Function Data Assurance

Level

Detection and Handling OR Absence

Transfer Critical Digital Error DetectionORFeedback / Read Back Verify

OR Level A or B

Essential OR Level C or D

Routine No requirements.

Format Critical Feedback / Reversibility CheckORIndependent Redundancy

OR Level A or B



Manipulate/ Generate

Critical Feedback / Reversibility CheckORIndependent Redundancy

OR Level A or B


ANDLogical Consistency Checks

ORSemantic Consistency Checks


198

See notes in Table 3 eADRM S5 C3 for additional information on assurance levels.

UNCLASSIFIED

UNCLASSIFIED

Source of Data Criticality Level examples

• DO-201• DO-276• AERODROME OBSTACLE CHARTS

199

UNCLASSIFIED

UNCLASSIFIED

Other Factors

• There are other, important characteristics of aeronautical data that the MPS may negatively affect:– accuracy– resolution– timeliness– completeness– format

• These considerations may result in additional requirements being placed on the MPS.– e.g. When translating the navigation database, the input

resolution shall be maintained.

200

28/02/2020

51

UNCLASSIFIED

UNCLASSIFIED

Summary

• DASA is not directly involved in the management of MPS– MPS are not considered as directly impacting

airworthiness – Guidance from DASA however exists through ADRM

and (upcoming) factsheet

• System Safety principles are applied, through the 7-step risk management process

201

UNCLASSIFIED

UNCLASSIFIED

Further Reading

• eADRM Section 5 Chapter 3 Mission Planning Systems

• eADRM Section 5 Chapter 4 Electronic Flight Bags• RTCA DO-200B Standards for Processing

Aeronautical Data• RTCA DO-201A Standards for Aeronautical

Information• AC 03/2018 – 7 Step Risk Management Process

202

UNCLASSIFIED

UNCLASSIFIED 203

Questions

Electronic Flight BagsDirectorate of Initial Airworthiness (DIA-DASA)

UNCLASSIFIED

UNCLASSIFIED

28/02/2020

52

UNCLASSIFIED

UNCLASSIFIED

Introduction

• FAA definition:– Electronic display system intended primarily for flight deck use that

includes the hardware and software necessary to support an intended function

• EASA definition:– An information system for flight deck crew members which allows

storing, updating, delivering, displaying, and/or computing digital data to support flight operations or duties.

• EFBs can be either:– Portable or Integrated

205

UNCLASSIFIED

UNCLASSIFIED

ADF use of EFBs

• EFBs are not typically part of the aircraft type design and are usually considered as Aircraft Support Systems

• Authority has prescribed design requirements for EFB as they may have an effect on the safety of flight– ADRM Section 5 Chapter 4

• The above is in-line with the guidance provided by EASA and the FAA– FAA Advisory Circular 120-76D – EASA - ED 2014/001/R: AMC 20-25

206

UNCLASSIFIED

UNCLASSIFIED

Types of EFBs

• Portable EFBs:– Can not have a worst credible failure condition greater than Minor

(2x.1309)• Commercial electronic hardware is not certified for aviation use and so

its reliability is not defined. This means that a given functionality may be lost any moment

– Used primarily for:• Display of Static Information (Publications, Manuals,

Aeronautical Information, Flight Orders, Checklists etc)• Situational awareness tool (Interactive maps, Chart Viewers,

Mission Planning, and Flight Planning)

• Integrated EFBs– Integrated EFBs are part of the aircraft Type Design and require design

approval and certification as per any other flight display or aircraft instrument. See ADRM Section 2 Chapter 3 – Aviation Software.

207

UNCLASSIFIED

UNCLASSIFIED

EFB Design Requirements

• Conduct System Safety Assessment to determine worst credible failure condition– Portable EFBs must not have a worst credible failure

condition greater than Minor (FAA AC 2x.1309)

• EFB Software– Assured commensurate with the worst credible failure

condition (perform intended functions for portable EFBs)

• EFB Hardware – Adhere to role equipment and portable electronic device

design requirements as per ADRM Section 5 Chapter 6

208

28/02/2020

53

UNCLASSIFIED

UNCLASSIFIED

EFB Software

• Portable EFBs:– Evidence must be documented demonstrating that the

Portable EFB operating system and hosted application software can perform the intended functions and do not provide false or misleading information

– FAA Order 8900.1 Vol.4 Ch.15 provides a good checklist to evaluate the EFBs intended functions

• Integrated EFBs:– Operating system and hosted application software must

comply with the requirements of ADRM Section 2 Chapter 3 – Aviation Software

209

UNCLASSIFIED

UNCLASSIFIED

Software Application Considerations

• Verification program should demonstrate that applications meet the functional requirements and are robust enough to prevent the provision of confusing or misleading information.

• Focus on good practice:– Reduce/eliminate misleading data– Readable– Usable– Robust– Accurate, available and timely provision of data– Compatible with OS

• Applications with NAA/MAA approval should be used when available

210

UNCLASSIFIED

UNCLASSIFIED

EFB Hardware Considerations

• Adhere to role equipment and portable electronic device design requirements as per ADRM Section 5 Chapter 6– Use of aircraft power– Batteries– Environmental Considerations– Mounting Devices– Human Machine Interface– Electromagnetic Compatibility – Stowage

211

UNCLASSIFIED

UNCLASSIFIED

Good Practice – Portable EFBs

• Use supported by Safety Case• Perform intended functions • Software robust• Robust backups or procedures in the event of

failure• Hardware compatible with the target environment• Operational procedures are adequate• Use supported by operational evaluation/workload

assessment

212

28/02/2020

54

UNCLASSIFIED

UNCLASSIFIED

Further Information

• ADRM Section 5 Chapter 4• FAA Order 8900.1- Volume 4 Chapter 15

– Electronic Flight Bag Authorization Process • Advisory Circular (AC) 120-76D

– Authorization for Use of Electronic Flight Bags• Advisory Circular (AC) 20-173

– Installation of Electronic Flight Bag Components• EASA Decision (ED) 2014/001/R

– Airworthiness and Operational Considerations for Electronic FlightBags, Annex II – AMC 20-25

• Civil Aviation Advisory Publication (CAAP) 233-1(1) – Electronic FlightBags

• Air Warfare Centre – Command Systems - ADF EFB intranet webpage

213

UNCLASSIFIED

UNCLASSIFIED

Questions?

214

Software in UASDirectorate of Initial Airworthiness (DIA-DASA)

UNCLASSIFIED

UNCLASSIFIED

DIA-UAS

• UAS mailbox - [email protected]

• DASA website -https://www.defence.gov.au/DASP/DefenceAviationSafetyProgram/InitialAirworthiness/UASCertification.asp

• UAS Regulation – self-paced awareness module -https://www.defence.gov.au/DASP/Docs/Regulations/DefenceAviationRegulationSet/SelfPacedDASRUASAwarenessModule.pdf

216

28/02/2020

55

UNCLASSIFIED

UNCLASSIFIED 217

Certified Specific OpenCategory Certified Specific A Specific B Small UAS

<25kgVery Small

<2kgMicro <0.1kg

Is an Airworthiness Instrument required? Yes.

MTC/MRTCYes.

UASOP

No. Command approval with notification to

DASA

No. Command approval

Is production and design organisation approval required?

Yes May be No No

Operational restrictions No Yes Standard Scenario based Standard Operating Conditions

Populous Overfly Yes Subject to SFARP

Dependent on Standard Scenario No

Is continuing airworthiness organisation approval required?

Yes May be No No

Is maintenance organisation approval required?

Yes May be No No

UAS Operator Requirement Mil Pilot Specified in

UASOPCommand discretion Command discretion

SW SME Involvement

Yes Possible

Software Section Interface to UAS UNCLASSIFIED

UNCLASSIFIED

Questions?

218

Aviation Software Fundamentals course slides#1 › DASP › Docs › Regulations ›...

Documents

Transcript of Aviation Software Fundamentals course slides#1 › DASP › Docs › Regulations ›...