Life Cycle Risk Management in a Product Tailoring...
Transcript of Life Cycle Risk Management in a Product Tailoring...
Page 2
Life Cycle Risk Management Agenda
• Life Cycle Risk Management
• Product Tailoring Impact
• Architecture Optimization
• Heritage Reuse
• Risk Tracking
• Integration Readiness
• Anomaly Risk Rating
• Cumulative Risk Management
Normalcy Bias: Lack of exposure to failure and small
sample size of operating hours:
• Rejection of proposed failure modes
• Seizing on any ambiguities to infer less credibility
• Interpretation of warnings in the most optimistic way
Bounded Rationality: Decision-making, rationality of
individuals is limited:
• By information
• Cognitive state
• Finite decision times (Herbert A. Simon)
Epistemic Failures due to erroneous technological
assumptions, even thought there were good reasons to
hold that assumption. (John Downer)
Page 3
Lifecycle Risk Management Components
Establish Baseline and Assess Compliance
Residual Risk and Uncertainty Management
Integration Readiness and Anomaly Rating
Cumulative Risk Analysis
Pre-Award!RFP !Selection
Reqmts - Design!SRR!PDR!CDR
Build and Test!MRR! TRR!PER
Sell-Off !PSR !MRR
Operations !FRR !ICR
" Development" Implementation" Risk Profile/Strategy
Architectural Optimization
" Resource" Schedule" Technical
Heritage Reuse Compliance
" Baseline" Realized" "Unknowns Bounding
Risk ID, Tracking, Control
" Next Step Integration" Product Liens" Constraints
Integration Readiness
" Severity" Cause" Corrective Action
Anomaly Risk Rating" Residual Risk" In-Phase Burn Down" Cumulative MS Impact" Process Effectiveness
Cumulative Risk Tracking
Page 5
Process and Product Architectures Risk Exposure
• Type 1: Large risk exposure — Factors: Longest Mission Length, Custom Development
— Overlapping Mitigation Actions
• Type 2: Moderate risk exposure — Factors: Long Mission Length, Heritage Development
— Part Quality/Margins/Redundancy
• Type 3: Low risk exposure — Factors: Short Mission, Heritage Development
— Part Quality/Test Actions
• Type 4: Minimum risk exposure — Factors: Non-flight prototype development
— Safety and Interface Focus
Product Type Risk Management has Demonstrated Equivalent Mission Success
ResidualEngineering PrototypeDevelopment
Residual - Single string- Streamlined Process
ResidualInternal Core BPS STDs
Mitigated - Interfaces- Safety
Mitigated - Part Quality- Test Margins- MA Audits- Supplier Risk- Heritage- Internal MRL
Mitigated - Minor SPFs- Margin- MA- V&V- L2/L2- parts- Supplier risk- Heritage- Internal MRL
Mitigated- No SPFs- Large Margin- MA & MS- Full V&V- L1/L2 parts- Supplier OS- TRL assured- MRL assured
Type 4Type 3Type 2Type 1
Residual Minimal
Mission Success Assurance Activities
Pro
du
ct
typ
e t
ota
l ri
sk e
xp
osu
re
Pro
du
ct
Typ
e
Investm
en
t
Mission Success
T1
T2
T3
T4
Cost
IR&D
FlightProductTypes
Probability of Success
Page 6
Mission Class Risk Surface Legend
Radical Spokes Vector Characteristic
Risk
Management
· People: Risk Process Informal, Sparse documentation
· Programmatic: Risk Process cost/schedule/technical focus
· Mission Success: Programmatic Plus Residual Risk Management
Supplier
Interaction
· COTS: Buying product as catalog item with little data
· Insight: Formal Data available throughout development
· Oversight: Customer/Supplier interactive development
EEE Parts
· COTS: Consumer commercial and Industrial Grade
· L3: Hi-Rel parts with screening but little qualification data
· L2: Military grade parts will full screening and Qual program
· L1: Space grade parts with delta screening and qualification
Complexity · Single String: Simplex assemblies performing mission
· Block Redundant: Parallel active and standby assemblies
· No SPFs: No simplex assemblies beyond DFMR items
Product Type Instantiation Optimizes Development
Architecture • Critical evaluation
— Programmatic Constraints: Acquisition, Requirements, Funding
— Mission Needs: Reliability, Life, Performance
— Acquisition Risk Strategy: Mission Class, Mission Type
Optimal Development Architecture Translates Into Managed Risk and Controlled Cost
SupplierInteraction
EEEParts
Complexity
L2 L3 COTS
BlockRedundant
SingleString
Insight
COTSProgrammatic
People
Risk Management
MissionSuccess
Oversight
NoSPFs
Minimum Practical
Risk L1
Page 7
Category Process
Program
Execution
1 Design Assurance
2 Requirement Analysis and Validation 3 Parts, Materials and Processes 4 Environmental Compatibility 5 Reliability Engineering 6 System Safety 7 Configuration/Change Management 8 Integration, Test and Evaluation
Risk, Oversight
and Assurance
9 Risk Assessment and Management 10 Independent Reviews
11 Hardware Quality Assurance 12 Software Assurance 13 Supplier Quality Assurance
Triage, Information &
Lessons
Learned
14 Failure Review Board 15 Corrective/Preventative Action Board
16 Alerts, Information Bulletins
Product Type Instantiation Optimizes Mission Success
Processes
• Process Application Level: Inductive/Deductive Analysis
• Rigor: Patent Defect Inspections, WCCA Circuits
• Oversight: Independent Internal and Customer Assessments
• Relationships: Test level, Build-in vs. Procedural
Optimal Mission Success Processes Avoid, Eliminate, Precipitate, & Mitigates Risks
Page 8
Early and Continual Assessment of Hardware-Software
Reuse Key to Fulfilling Heritage Promise
Assessment of TRUE Heritage Key to Managing Capability Based Risks
Page 10
Residual Risk Augmented Classical Risk Management
Augmented Risk Management
o ID, Analysis, Plan, Track, Control
o Acquisition, Baseline Constraints, Execution
Systems Engineering
o Risk Interfaces and Relationships
o Product Type Risk Exposure Consequence
Subcontract Management
o Capture Subtle Requirement and Product Changes
Peer, Life-Cycle Gate, & Milestones
o Build Upon the Cumulative Risk Posture
Program Failure Review Board
o Root Cause Complexity and Systemic Risks
Metrics
o Trends and Outliers
Residual Risk Management Ensures Baseline Compliance
Page 11
Managing Uncertainty Across Product Types
Retired Risks No Residual Risk Artifacts
Known-
Knowns
Risk Artifacts
· Test as you Fly Validation
· Demonstrated TPM Performance
· Flight or test-validated analysis,
simulations and models
· Incremental knowledge
Buildup
· Complete verification and
validation
Open Risks
Uncertainty
Uncertainties
Open Residual Risks Risk Handling
Known-
Unknowns
Accepted Risk
· Analysis / test limitations
· Unverified Models/Simulations
· Envelope expansion
· Unverified failure modes
· Evaluate Deltas due to
o Baseline limitations
o Margin gaps
o In-complete V&V
o Analysis thoroughness
Unknown-
Knowns
Execution Risk
· Miscommunicate test/analysis
· Understanding of data/ envir
· Poor documentation combined
with loss of corporate memory
· Program communications
/ data sharing
· Incremental knowledge
build-up w/ trending
Unknown-
Unknowns
Hidden Risk
· Bad assumptions
· Unfinished foundation research
· Untested new environments
· Inadvertent operations outside of
limits
· TRL level 6 by PDR
· Envir analysis/test rigor
· Simulators & test-beds
fidelity, TAYF
· Design Margins
Uncertainty Management Core to an Effective Risk Management Strategy
Page 12
Design/Production Flow-down
- Process Requirements
- Guidelines/ Lessons Learned
- Workmanship
- Part/ Material StandardsFMEA,FTA,PRA
- Redundancy, Cross- strapping
- Graceful Degradation
- SPF CIL Mit igat ion
- Common CauseMargins
- Derating, EOL, LLIs, BudgetsInspection and Screening
- Patent Defect
- Latent Defect
- Infant Mortality Burn- in
- Operational Profi le ConfidenceQualification
- Environmental Tolerance
- Life Characterizat ion
- Radiat ion Characterizat ionReliability Growth
- Categorizat ion, Phase, TrendingProcess Capability
- Control with Randomness
- Coverage, Completeness
- Meets User Expectat ions
Safety Compliance VerificationHazard Analysis, Inhibit Design, FTA, Flight/Operations/Range Safety
System Safety Engineering
Fault Tolerance, SPFs, Lifetime, Deratings, Circuit Margins
Reliability & Maintainability
PMPCB, Qual/Screening, Radiation, GIDEP, Compatibility, Allowables
Parts, Materials, & Processes
Workmanship, Process Control, Non-conformances, Inspection
Hardware Quality Assurance
CMMI L3, Rel iabi l i ty, Safety, Firmware, Peer Review, IV&V
Software Quality Assurance
Fault Tolerance/Avoidance Balance
No Infant Mortality & Wearout
Processes Controlled, No Latents
S/W Maturity,Test Effectiveness
SMA DRIVING ACTIVITIES DISCIPLINE PRODUCT
LEVERAGE
- Equipment & personnel hazards- Pyros, batteries, mechanisms
- Anticipated/unanticipated faults- Random & environmental origins
- Screening/Qual exceptions- Degradation, Contamination
- Process control & capability- Latent and patent defects
- Residual defects: phase, interval- Test efficacy, saturation
Risk Avoidance/Mitigation
Process/Product Risk Exceptions Management
SME Residual Risk Identification, Management, Elimination/Mitigation
Page 14
Next Step Readiness Assurance Manages Risks and
Supports Verification
Integration and Test Readiness Verification (I&TRV): Next level integration readiness
Analyses: complete, Production: current, Non-conformances: addressed, Testing: compliant
Mission Assurance/Systems Engineering: Manage residual risk and requirement verification
Subcontracted Hardware: Integrate I&TRV with the pre-ship review process
I&TRV Manages Integration Risk In-Phase with the Development
- Drawings, Specs, Engineering- Change Proposals- PFS, SOW, ICD's- Waivers, MRB/FRB Results- Previous Unit Risks- Environmental Test Results- Reliability/Parts/Design Analysis- Telemetry/Calibration Data- Mass Data- Operational/Handling Constraints
Review Items
- Adequate Compliance Testing- STE and Documentation Readiness- Waivers/Liens Closure Plan- Receiving Organization Readiness- Pre-integration Critical Items- Operations and Handling Constraints
I&TRV Assessment
Page 15
Failure Effect Rating
(excluding redundancy) Failure Cause/Corrective Action Rating
Severity R R Cause/Corrective Action
Negligible (N) 1 1 Known cause/certainty of corrective action
(No residual risk)
Significant (S) 2 2 Unknown cause/effective corrective action
(No residual risk)
Catastrophic (C) 3 3 Known cause/uncertainty in corrective action
(Some residual risk)
High Priority 4 Unknown cause/uncertainty in corrective action
(Residual risk)
Risk Rating and Tracking of Anomalies
• What: System/Hardware/Software/Test Anomaly and Non-Conformance Reports
• Capture: Anomaly/Failure Effect; Current Active Cause and Corrective Action
• Residual Risk: Post investigating/maturity transferred to program risk
• Ratings programmatically prioritizes anomalies and determine risk to mission success
Risk Rating and Essential Tool in Anomaly Residual Risk Burn-Down
Page 17
0
1
2
3
4
5
6
7
8
9
10
-3
2
7
12
17
22
27
32
37
42
47
Pro
posa
l
Plann
ing
Rqm
ts D
ef.
Prel D
esign
Det
ail D
esign
Pre
-Fab
Fab &
Ass
y I&
T
Del &
Sup
port
Pos
t Deliver
y
Es
ca
pe
s
Sig
nif
ica
nt
Ris
ks
Life Cycle Program Phase
Risk Closure
High Risks
Moderate Risks
Low Risks
Total Risks
Process Escapes
Cumulative Risk Management
• Risk Mitigation Efficacy: Ensuring a “low” risk profile, eliminating and controlling risks
• Risk Triad: Programmatic (cost and schedule), Technical, and Cumulative residual risk
• Metrics: (1) Risk identification, (2) Burn-down efficacy, (3) Orthogonal in-phase detection,
and (4) Risk escapes
Cumulative Product & Process Assessment of Contribution to Mission Success