Assessment of Alternate Methodologies for Establishing

Equivalent Satisfaction of the Ec Criterion for Launch Licensing

Terry HardyAST-300/Systems Engineering and Training Division

May 19, 2004

2

Project Description

• P. Birkeland and J. Greason have proposed alternate methods to satisfy the Ec criterion of 30x10-6.

• One alternate methodology uses derived reliability requirements, reliability allocation strategies, and Failure Modes, Effects and Criticality Analysis (FMECA) to demonstrate an equivalent level of safety to Ec criterion.

• AST will examine this alternate methodology and compare it to existing approaches.

3

Plan

• AST will conduct the following in-house tasks:– Review documentation by Birkeland/Greason on the proposed

process.– Examine the FMECA process, and investigate its advantages and

disadvantages – Investigate the use of FMECA within FAA, for aircraft, ELV and

RLV.– Compare the proposed FMECA process to AST’s existing process.– Document findings in a white paper– Present findings to COMSTAC RLV Working Group

• Milestones include:– Status to RLV Working Group May 2004– Draft Report July 2004– Final Report September 2004– Presentation at COMSTAC in October 2004

4

Expected Casualty Analysis

Expected Casualty Analysis:- Is a well-established collective risk measure.- Has been successfully used for decades in the aerospace

community for both launch vehicle risk analyses and explosive safety analyses.

- Quantifies both probability and severity in assessing risk to public safety.

But…- Ec analysis method is complex (inputs include characteristics

for explosive and inert debris, weights/sizes, influence of wind and aerodynamic properties on debris, shelter effects on casualty area, etc.)

5

Alternate Approach

The alternate approach proposed by P. Birkeland can be summarized as follows:

1. Derive an allowable probability of catastrophic failure for RLVs based on commercial aircraft failure rate requirements and historical ground casualty rate.

2. Allocate RLV failure probability to hardware and operator and allocate by phase of flight.

3. Use an FMECA to demonstrate that the allocated failure rate has been obtained.

6

Initial Findings

The proposed approach is appealing because:• Setting reliability goals is consistent with approaches

used by NASA, FAA for commercial aircraft.• Setting reliability requirements for passenger-carrying

RLVs is consistent with approach taken by FAA for commercial aircraft.

• Allocating reliability between hardware and non-hardware systems explicitly recognizes that system safety is more than just hardware failure.

7

Initial Findings

The proposed approach is appealing because:• The methodology directly links the system safety

process to expected reliability objectives.• FMECA is an excellent tool for methodically

identifying safety issues and mitigation measures during design, especially when used in parallel with other hazard analysis/risk assessment tools.

8

Initial Findings

AST has some concerns with this approach:• It may be difficult to justify a top-level failure

probability or reliability allocations based on a comparison to aircraft.– Consequence of a launch vehicle crash is potentially higher

than aircraft due to presence of fuel and oxidizer.– Most RLVs will operate at high velocity with higher kinetic

energy and higher risk of aerodynamic breakup.– Aircraft industry is mature, with years of history and

performance, in comparison to RLV industry.

• The proposed approach does not explicitly incorporate population density or casualty area.

9

Initial Findings

AST has concerns with any approach based only on an FMECA:

• It is likely that the FMECA will miss some failure modes, especially if software and human interactions are not considered.

• FMECA does not consider combinations of failures, and most accidents are the result of a confluence of factors, and often due to a combination of seemingly low-consequence failures.

• FMECA does not include a quantitative assessment of severity.• FMECA does not usually account for uncertainties in the input

data.• FMECA can provide optimistic system reliability estimates.

10

Further Work

• AST will continue its investigation of this proposal, comparing it to the existing Ec methodology and other analytical approaches.

• AST will prepare a white paper documenting its findings.

• AST will present the full report at the next COMSTAC meeting.