Adam Evans & Kristian Cruickshank, Nova Systems - Developing UAV safety cases

http://www.novasystems.com.au Experience Knowledge Independence

Developing UAV Safety Cases

UAV Triple Zero Summit

Mr Adam Evans

Mr Kristian Cruickshank

1


Overview

2

Nova’s Background

What is a Safety Case?

When is a Safety Case Required?

UAV Safety Case Paradigm

Safety Case Process

Levels of Acceptable Risk

Emergency Services Risk Context

UAV Operation Risk Analysis

Treating Unacceptable Risk

UAV Safety Management Systems

Consolidating the Safety Case


Terminology

UAV vs UAS vs RPA vs RP vs RPAS

Throughout this presentation:

UAV = RPA

UAS = RPAS

UAV Controller = RP

3


Origins in Defence T&E

Involved in all significant ADF UAS projects to date:

Heron

Shadow 200

Aerial Targets

Nova contracted by ADF to develop UAV regulatory framework

Specialists in Military and Civil Airworthiness, inclusive of operational and technical risk management

Aeronautical Engineers Australia specialists in Civil Airworthiness and CASRs

Practitioners in various aerospace engineering and operational domains

Nova’s Background

4



Broad Definition:

A structured argument of compiled evidence demonstrating that a system is acceptably safe

No CASA definition for UAV Safety Case

CASA Airworthiness Circular for Aerodromes:

“A documented body of evidence that provides a demonstrated and valid argument that a system is adequately safe for a given application and environment over its lifetime” (AC 139-16(1))

Propose that the definition used in AC 139-16(1) is suitable for UAVs

5



Elements of a UAV Safety Case Adequate Level of Safety. Benchmark is ‘acceptable’* level of risk posed to the general public.

Given Application and Environment. Safety case must define the types of UAV operations and the environmental factors present in those operations

Statement of Operating Intent (SOI) or Concept of Operations (CONOPS) or equivalent

Key environmental factors are – population densities, physical environment, airspace category.

Lifetime. UAV context may lessen the importance of this element – possibly more ‘disposable’ than most aircraft? Still requires consideration.

* ‘Acceptable’ may vary depending on a given emergency services scenario

6



Elements of a UAV Safety Case (cont)

System. Unmanned Aerial System plus the Safety Management System or equivalent implemented.

Demonstrated Argument. Logical, valid, and defensible argument constructed from applicable body of evidence.

No specific CASA guidance on what the argument must consider

Experience with Military UAS provides a reasonable basis for considerations

7



Implied by NPRM1309OS (regulations and guidance not published yet)

Intent of once-off Area Approval is the same as a safety case, but safety case can be enduring

Operation of Large UAV (> 150kg)

Operating outside of Standard Operating Conditions

Over Populous Areas

Beyond Visual Line Of Sight

Greater than 400ft

Other than Class G airspace

Closer than 3NM from aerodrome

8



Put Simply:

UAV OPERATIONS THAT WOULD BE OF MOST BENEFIT TO EMERGENCY SERVICES!

9


Likely Scenarios for Safety Case

Search and Rescue

BVLOS, Over Populous Areas, Above 400ft

Fire Spotting

Restricted Airspace?

Police Tactical Operations

BVLOS, Over Populous Areas, Controlled Airspace

Natural Disasters

BVLOS, Over Populous Areas, Above 400ft, Launching from Aerodromes

Others?

10



Different approach than regular aircraft – Why?

Aircraft Type Certification and Operational Management Regulations established and industry complies

UAV origins – Hobby and Military

No internationally recognised Type Certification Requirements established

‘Risk Management Approach’ instead of a ‘Compliance to Standards’ approach

11



The Future Safety Paradigm – Establishing Compliance with Technical Airworthiness Requirements

Confidence in Integrity of System Design

Confidence in Quality of Manufacture

Design of Maintenance Schemes that maintain aircraft reliability

Same process as normal Aircraft

Challenges with ‘The Future’ Cost

Establishing requirements for different UAV categories (Small, Medium, Large, Commuter?)

Detect and Avoid + more

12



The Current Safety Paradigm – Technical and Operational Risk Management

Defining Acceptable Levels of Risk to Public

Determine worst credible Consequence of UAV accident

Determine Probability of worst credible Consequence occurring

Reliability of UAV (hardware reliability combined with integrity of software) – if possible to determine

Probability of fatality/injury given impact

Population density + more

Technical and Operational risk treatments

Plus ‘normal’ aircraft requirements (maintenance, flight operations system, Safety Management System, etc)

13


UAV Safety Case Process

14

Develop SOI / CONOPS

Define Acceptable

Levels of Risk

System Safety Assessment

Compare Risk to Acceptable

Levels

Risk Acceptable

?

Develop Risk Mitigations

Consolidate Safety Case

Operational, maintenance,

design, SOI change, etc

Yes

No

Evidence

SOI, Acceptable Risk, UAV design,

Maintenance System, Safety Management System, Operators

Manual, OEM Documentation


Statement of Operating Intent

Analogous to Concept of Operations

Derived from Military Context

Defines types of operations and informs risk assessment process

15


Statement of Operating Intent

Key Aspects

Role. Function(s) or purpose(s) assigned to system – SAR, Fire Spotting, Surveillance, etc.

Tasks are a sub-element of Role. Tasks to be conducted under a given role.

Environment. Totality of surroundings/conditions of operations (airspace, areas of operation, physical environment, etc)

Flight Envelope. Defines outermost boundary of flight conditions for UAV to remain airworthy.

Flight Usage Spectrum. Flight Profiles for each task/role, frequency of profiles, Rate of Effort, etc.

16


Safety Case Process

17


Define Acceptable

Levels of Risk



Levels

Risk Acceptable

?





Yes

No

Evidence





Safety Targets – Example Maximum acceptable Individual probability of fatality or serious injury to the General Public: 1 X10-7 per flight hour

Maximum acceptable Collective fatality expectation to the General Public: 1000 X10-6 (1x10-3) per annum OR 5x10-7 per flight hour

Maximum acceptable Individual probability of fatality or serious injury to the Mission Personnel: 1 X10-6 per flight hour

Maximum acceptable Collective fatality expectation to the Mission Personnel: 10000 X10-6 (1 x10-2) per annum OR 1 X10-5 per flight hour

18


Defining Levels of Acceptable Risk

What ‘level of safety’, integrity or reliability do we need to operate a 20kg UAV in an sparsely populated rural environment?

What if the operation is attempting to prevent an assault?

What if the operation is attempting to prevent a homicide?

What if the operation is attempting to prevent multiple homicides

What if the aircraft has sufficient range to fly into densely populated area?

19


Emergency Service Risk Context

May be quite simple to balance risk

When exposing the public to risks, the basis for determining the risk as acceptable must be able to stand up to public scrutiny

20

Public risk benefit

from UAV operation

Public risk exposure

without UAV operation


Example safety target

Homicide Assault

Probable 1x10-3 1x10-4

Likely 1x10-4 1x10-5

Unlikely 1x10-5 1x10-7

Rare 1x10-7 1x10-9


Explanation of Table

UAV operations to prevent a Homicide are reasonable if the risk to the general public is less than 1x10-3 and it is determined that the assailant will Probably commit the crime.

UAV operations to prevent 5 Assaults are reasonable if the risk to the general public is less than 5x10-7 and it is determined that the assailant could, but is Unlikely commit the crime.

22


Safety Case Process

23


Define Acceptable

Levels of Risk



Levels

Risk Acceptable

?





Yes

No

Evidence





24


Unrecoverable Failure Rate

Unrecoverable Failure Rate unknown? Fault tree analysis to identify safety critical systems

Engines/Navigation/Airframe/Autopilot/etc

Various techniques to assess overall reliability / integrity of design

Consider existing Operational Mechanisms

Use Casualty/Fatality Expectation Rate Analysis to quantify risks to personnel

End Product is Unmitigated Risk

Software reliability?


FTA for Military UAS operations – Air Vehicle Escape

25


Casualty Expectation Methodologies

Once Unrecoverable Failure Rates for the vehicle are known

Used to determine Collective and Individual Risks to General Public and Mission Essential Personnel

Based on population densities and Lethal Area of Vehicle

26


Casualty Expectation

CE = λ x PCasualty|Strike x PStrike|Impact x PImpact

CE – Casualty Expectation (collective risk)

λ – Reliability

PImpact – Probability of high energy crash given a failure

PStrike|Impact - Probability of striking an individual

PCasualty|Strike - Probability of killing someone

27



Pimpact

Difficult to determine (e.g. reliability of FTS)

Substantial computational resources

Integrate over all possible crash locations

Pstrike/impact

Exposure time

Population density

Lethal Area of Vehicle

28



Pcasualty/strike

Depends on debris KE and explosive energy in the Air Vehicle

Requires analysis of various materials in Air Vehicle

CASA paper assists


Risk Comparison

30


Define Acceptable

Levels of Risk



Levels

Risk Acceptable

?





Yes

No

Evidence





Risk Comparison

CE = λ x PCasualty|Strike x PStrike|Impact x Pimpact

31

Public risk benefit

from UAV operation

Public risk exposure

without UAV operation

Prevent possible homicide (1x10-4)


Risk Mitigation

32


Define Acceptable

Levels of Risk



Levels

Risk Acceptable

?





Yes

No

Evidence





Risk Mitigation

Develop Risk Treatments if Acceptable threshold exceeded

Operational Treatments:

Restrict range of UAV

Extended VLOS

Operations only up to (X) population density

Etc

Technical Treatments:

Different UAV

OEM redesign (datalink reliability)

33


Minimising Risk (below acceptable?)

Emergency services have a duty of care to minimise the risk to the public

Further work could be done in order to identify risk levels at the front line

Flight plans could be optimised to reduce risk to public

Aircraft type or configuration selected to reduce risk

Possibility for assumed clearance for flight if specific criteria is satisfied (as specified by Operations Manual/Safety Management System)

34



Likely that specific risk mitigating processes or techniques will need to be enacted during operations

If there is an ongoing need to identify and treat risks, or an ongoing Operational Risk Management process – this will form a large portion of the UAV Safety Management System

Best place for these to be documented and enforced (forming part of the safety case) will be in a Safety Management System

35


UAV Safety Management System

Ongoing Risk Assessments may include:

Mission Planning Processes and Tools

Onsite Risk Assessments

Particularly relevant to Emergency Services Risk Context

Risk Assessment and Treatment on repairs or maintenance

More?

36



No specific requirement in CASR 101 (or NPRM1309OS)

However, intent of identifying and managing safety risks associated with UAVs is applicable

SMS Gap Analysis Tool provided by CASA

Operations Manual and other corporate plans/procedures may be sufficient without a dedicated SMS for most operations

Likely that most Operations Manuals would include these considerations, but a dedicated SMS may be advisable for large UAVs and non-standard operating conditions

37



38


Define Acceptable

Levels of Risk



Levels

Risk Acceptable

?





Yes

No

Evidence






Don’t make it too scenario dependent

What if you haven’t thought of all scenarios?

Flexibility where operational functions and risks remain valid

Structured argument

Outline the process

SOI/CONOPS

Justify Risk Acceptability (where did safety targets come from?)

Describe Risk Mitigations (where necessary, show that they’ve been incorporated into design/operations)

39



Various techniques for ‘Structuring’ argument (Goal Structured Notation is a good method)

40

Safe UAV Operation

SO

I/CO

NO

PS

Safety Target(s)

System Reliability

/ Integrity

System Limitations

Op Risk Manage

Ops/Maint Processes

System Safety

Assess



If residual risk is unacceptable – Talk to CASA

In some cases the risk may simply be too high

Back to the drawing board

The UAV you intended to use may not be the answer

UAVs may not be the answer

Finally – Submit the Safety Case!

Hopefully this is not the first time CASA has seen it...

Get them involved from the start for planned ‘high’ risk operations

41


Questions?

42


Developing UAV Safety Cases

UAV Triple Zero Summit

Mr Adam Evans http://www.linkedin.com/pub/adam-evans/12/850/b73

Mr Kristian Cruickshank http://www.linkedin.com/pub/kristian-cruickshank/34/922/ba4

43

Adam Evans & Kristian Cruickshank, Nova Systems - Developing UAV safety cases

Law

Transcript of Adam Evans & Kristian Cruickshank, Nova Systems - Developing UAV safety cases

Nilf2012_Global IT Supply Chain Bryan Cruickshank- KPMG

Cr Kristian

Medicines and Drugs Anti-virals Julia Barnes Anna Cruickshank.

Brown & Cruickshank, 1994

Willie Cruickshank. Dementia Friendly Communties Why and What?

Spencer Shickora, Kristian Real, Connor McNamara, Hassan Latif, Chris Roberts, and Peter Cruickshank.

CRUICKSHANK, SCHNEEBERGER guia de bolsillo gobernanza.pdf

Kristian Teiter

Agama Kristian

Author Kristian Reale Rev. 2011 by Kristian Reale

Librarians Leading Learning Online · Librarians Leading Learning Online Alice Cruickshank – Christchurch City Libraires . Alice Cruickshank •Originally from UK •Masters in

Creating Smarter Cities 2011 - 18 - Peter Cruickshank - CoDesign

Guitar Style of Django (Ian Cruickshank)

S232953 Mark Cruickshank Prac 2 MDOF ENG432.pdf.pdf

'SCIENTIFIC ABSTRACT KRISTIAN, P. - KRISTOF, S.' · Title "SCIENTIFIC ABSTRACT KRISTIAN, P. - KRISTOF, S." Subject "SCIENTIFIC ABSTRACT KRISTIAN, P. - KRISTOF, S." Keywords: KRISTIAN,

kristian sanjaya

Haitham Cruickshank University of Surrey

Ken Cruickshank - Research on Languages in NSW Schools

Kristian Scharling

Kristian Reinfjord