National Aeronautics and Space Administration

www.nasa.gov

Launching to the Moon, Mars, and Beyond

Launching to the Moon, Mars, and Beyond

NASA: Engineering Space Exploration

C. Herbert Shivers, PhD, PE, CSP

Deputy Director, Safety and Mission Assurance Directorate

NASA/Marshall Space Flight Center

Presented to the Conference on Quality in the Space and Defense Industries 2007

Cape Canaveral, Florida

March, 2008

National Aeronautics and Space Administration

www.nasa.gov

From Determinism to “Probabilism”

Changing our mindsets, or why

PTC isn’t an easy sell - yet

Deterministic Design Probabilistic Design?

Safety

Factors

PD

DFMR

SDF

SWP

PDF

PRA

DOEPredictive Models & Algorithms

Diagnostics & Prognostics

RAM Design

RedundancyLife Cycle Prediction

FMECA, FTA,

Block Diagrams,Systems of Systems Models

My Muddled Mind

Robust Design

System Safety Tools Feed Risk Informed Decision Making

QualitativeRisk

Assessment

QualitativeRisk

Assessment

ProbabilisticRisk

Assessment

ProbabilisticRisk

AssessmentActuarial/StatisticalAnalyses

Actuarial/StatisticalAnalyses

FMEA.HA,

ESD,ETA,FTA,RBD

FMEA.HA,

ESD,ETA,FTA,RBD

DecisionAnalysis

DecisionAnalysis

Method Technique

TechnicalRisk

and/or

ProgramRisk

TechnicalRisk

and/or

ProgramRisk

Application

ManagementSystem

ManagementSystem

Legend:FMEA - Failure Modes & Effects AnalysisHA - Hazard AnalysisESD - Event Sequence DiagramETA - Event Tree AnalysisFTA - Fault Tree AnalysisRBD - Reliability Block Diagram

Bryan O’Connor, Chief, Safety and Mission Assurance, NASA

SEPT 8, 2007, Huntsville, AL

• Using the Ares quantitative safety and reliability requirement to enforce the “design for reliability and safety” paradigm shift

• Improving system safety by using a functional analysis system approach to model and understand integrated system failures similar to the Shuttle foam problem.

• Improving design reliability by using probabilistic engineering physics-based modeling

• Evaluating and understating design uncertainty and design margins using probabilistic engineering techniques

Excerpted from Chris Cianciola, S&MA, MSFC, CQSDI March 2008

S&MA in Ares Design – Summary

Uncertainties sourcesmanufacturing, storage, aging, use environments, and scenarios

Uncertainties types randomness, lack of knowledge

Model to predict uncertainties in systems

Uncertainties outside acceptable limits?

Uncertainties produce inadequate margins?

Most important contributors to uncertainty (sensitivity analyses)?

Physics model form uncertainty

Sensitivity Analysis Only several contribute most of the uncertainty in system response

Requirements Uncertainty

Uncertainty Quantification and Risk Assessmentrisk-informed decision making

One of many decision variables

Subjective information - scientific and engineering judgment is necessary

Known unknowns and unknown unknowns exist

Resources are limited

Results uncertainty and credibility – challenges decision makers

Social, economic, and political factors exist

Probabilistic Models Limitations

• Is the methods framework credible?

• Address the credibility concern with verification and validation

Credibility

• Start with robustness and margin in design • A 30% design margin is not plausible for space

flight – it’s just too expensive• Our margins are so thin that we really need to

investigate margin• In our case we want to whittle the margin down

as much as possible but still maintain the robustness we need

• Getting those design margin trades into the system analysis is critical

NASA special study data 2006

Special study data

• We still use deterministic design and build margin into designs even though we have the computational capabilities to us PT methods

• We do not have probabilistic design methods in place• Probabilistic methods will have to be an investment by NASA into

the long term technology base and tools• First you need a proper deterministic goal and then you can

successfully apply a probabilistic model • Numerical Propulsion System Simulation is something that must be

implemented to get this country to Mars• The entire thought process needs to change, this is not a

deterministic world – everything is probabilistic• In the deterministic method there is no role for uncertainty or margin• Right now technology is so fast we are dumping technology on

people who are not educated to use these technologies appropriately

NASA special study data 2006

Special study data

• Redundancy is a part of our culture right now because it is much easier than looking for a solution using margin

• When you consider uncertainty in the design phase you are able to examine the trade space more efficiently and thoroughly.

• The nature of the probabilistic skills dealing in design is very different from the traditional PRA.

NASA special study data 2006

Special study data

• Design engineers don’t understand how to account for uncertainty in the design process and how to explore the trades throughout

• In the past we’ve had so much money and time for testing, but we don’t have this anymore

• Apollo did 14,000 tests on the LEM, do you think we could do that now?

• We must trade ability to test versus desired reliability demonstration

Why PTC Now?

NASA special study data 2006

• Probabilistic requirements ultimately relate to risk in achieving a level of performance or safety.

• The most important reason for using a structured approach is to ensure that the potential failure modes of a design are identified early in the design process to better understand the risks.

• An understanding of risks due to incomplete or inaccurate modeling of a design and the operational environment, or due to uncertainty related to the data used in a model is necessary so that a requirement can be verified with confidence.

Surendra N. Singhal, Engineering Directorate,

Marshall Space Flight Center, Sept. 18, 2007

Other Notable Thoughts

• Need a coherent institutionalized effort

• Need a regular training of the entire Center staff (many unaware of PT and its benefits)

• Need a core group where people and projects can go for help

A Way Forward

Robert J. Kuper, Executive for Reliability and Quality,Systems Engineering Conference, 23 October 2002

Utilizes Physics based behavioral model

Considers inherent uncertainties, modeling uncertainties, lack of data, human error, measurement error

Compensates for unknowns using statistical methods

Utilizes past performance data to develop behavioral model

Quantifies safety measures

Qualifies prediction accuracy

PT Selling Points