JOINT SERVICES SOFTWARE JOINT SERVICES SOFTWARE SYSTEMS SAFETY HANDBOOKSYSTEMS SAFETY HANDBOOK

Michael L. Brown, ChairpersonJOINT SOFTWARE SYSTEMS SAFETY COMMITTEE (JSSSC)

March 2001

presented to the

Safety Critical Systems Club

INTRODUCTIONINTRODUCTIONThis presentation provides an overview of the Software Safety Handbook and its Status. The following topics are addressed:

PurposeBackgroundHandbook Layout and ContentsSoftware Systems Safety ProcessesApplicabilityProject StatusAdditional tasksRecommendations

Provide management and engineering “how-to” guidelines to achieve a reasonable level of assurance that software will execute in a system context with an acceptable level of safety risk.

Initial process and methodology based on Independent Software Nuclear Safety Analysis process tailored to conventional systems and experience from many programs.

Process successfully applied to wide range of systems

PURPOSEPURPOSE

BackgroundBackground

Inconsistent processes– Most incomplete– Many good points but not tied together well

Lessons learned– SSSTRP– Took good points from each process– Developed comprehensive systems engineering

based software systems safety process

HandbookSections

Introductionto System

Safety

Introductionto the

Handbook

Executive Summary

4.0

3.0

2.0

1.0

Appendices

SoftwareSystemSafety

Purpose, Scope, Authority, Overview and Handbook Organization

Motivation and Direction for the Program Director and Program Manager

Overview of System Safety and Safety Risk Management

Definition, References, Supplemental Information, Generic Guidelines, Sample Documents, Lessons Learned, Synopsis of Analysis Techniques and Methods, and Agency Specific Information

Planning, Task Implementation, Risk Assessment and Acceptance, Configuration Management and Reusable Software

HANDBOOK LAYOUT & HANDBOOK LAYOUT & CONTENTSCONTENTS

System and Software System and Software EngineersEngineers

Written for all members of safety team

Purpose and scope of handbook

Authority for software safety program requirements

Organization of handbook

Software Safety EngineerSoftware Safety Engineer

How-to guidance on establishing and conducting software systems safety program

Process based on DODI-5000.1 and DOD-5000.2R System Acquisition Process and Requirements, MIL-STD-498/IEEE STD 1498/12207 Software Development Process, MIL-STD-882 Standard Practice for System Safety, and NATO Standardization Agreement requirements

Section 4.0Section 4.0

Main part of the document– “How-to” Guidance

Complete– However, needs expansion in certain areas (e.g., CDI in

safety critical applications) Covers entire software systems safety process

from concept to system retirement Refers reader to examples in appendices and

reference documents

Software Systems Software Systems Safety ProcessSafety Process

Program management– Planning through execution

Requirements derivation– Generic Requirements from Lessons Learned– System Specific Safety Requirements from system

level analyses Requirements verification and validation

– Detailed analyses– Safety Testing

Safety Assessment

SWS ProcessesSWS ProcessesProgram Phases & Milestones

CSUCSCITest

Software RequirementsAnalysis PD DD

Inter-Operability

Test

SystemRequirements & Design

HardwareDesign

PrototypeManufacturing

ConceptExploration

Program Definition& Risk Reduction Engineering and Manufacturing Development

OPEV AL

MS 0 MS 1 MS 2 MS 3

SRR

SDR

PDR CDR TRR

MIL-STD-498

Phase I Phase II Phase IIIPhase 0

Configuration Management

LifecycleSupport Activity

SPRA SPRA SPRASPRA SPRA SPRA

Code &CSUTest

SystemIntegration

Test

DODI 5000.2

ECPPCRSCN

AnalysisPIPs

Safety Processes for Phase 4 Are an Iteration of the Processes Performed for Phases 0-3.

Preliminary Hazard List (PHL) Development

Tailor the Generic Safety Critical Software Requirements List

SSR

MIL-STD882C

Software Safety Program Planning

Software Safety Program Management

Preliminary Hazard Analysis - (PHA)

Derive System Specific SoftwareSafety Critical Requirements

Preliminary Software Design SubsystemHazard Analysis - SSHA

Software Detailed Design Subsystem Hazard Analysis SSHA

SSR PDR TRRCDR

Verify Software Developed IAW Applicable Standards and Criteria

Software Safety Testing & Analysis

SRR SDR

Software Safety Test Planning

System Hazard Analysis (SHA)

Software Safety Assessment

Code Level Analysis

Production, Fielding/Deployment & Operational

Support

Copy Media & DistributionRecovery System Upgrades

MaintenancePIPs

Technial ReviewObsolescence

SWS PROCESSESSWS PROCESSESSoftware Safety Requirements Derivation

Develop Generic Safety Critical Software Guidelines &

Requirements

Obtain Generic Software Safety Requirements Lists

Tailor Generic Software Safety Requirements/Guidelines List for the specific system and/or subsystem

Categorize and Prioritize Generic Software Requirements/Guidelines

Derive Functional Safety Critical Requirements

Develop Safety Critical Functions List

Develop Potential Functional Hazard List

Preliminary Hazard List (PHL)

Preliminary Hazard Analysis (PHA) Categorize & Prioritize System

Functional Hazards Determine System Level H/W,

S/W and HF Causal Factors Execute System Level Trade

Study Begin Determining all of the

Software Specific Causal Factors

Begin Software & Architectural Detailed Design Trade StudyDerive System-Specific Software Safety Critical Requirements

Requirements Hazard Analysis/Software Criteria Requirements Analysis

Tag Safety-Critical Software Requirements Establish Methods for Tracing Software Safety Requirements to

Test Provide Evidence for Each Functional Hazard Mitigated by

Comparing to Requirements Verify Software Developed IAW Applicable Standards and Criteria

SWS PROCESSESSWS PROCESSESSoftware Safety Program Planning - Customer

• Acquisition Policy• OCD/OR/MENS• DOP• Proposal• Safety Policy• Generic Requirements• Lessons Learned• PHL

Inputs (Later Milestones)

• Draft PHA• Draft TEMP• Draft SEMP• Draft ILSP• Draft CRLCMP• Draft SSS• Draft S/SDD• Draft SOW/RFP

Software Safety Program PlanningProcuring Agency - Customer

• Define Acceptable Level of Risk (HRI, Acceptance Authority, Risk Assessment Methodology)

• Establish System Safety Program:– Identify and Establish Interfaces to Other Program Disciplines– Identify Contract Requirements– Develop POA&M– Establish Safety Data Library– Establish Hazard Tracking Process– Resource Requirements Determination:

»Analyses Required»Tests Required (U)»Resources Required

– Develop Safety Statement of Work: (U)»Define Safety Program Requirements (U)»Define Safety Reporting Requirements (U)»Define Safety Review Requirements (U)

– Proposal Evaluation»Develop Evaluation Criteria»Evaluate Proposals

• Input to SOW/SOO• Input to RFP• Safety Proposal Evaluation Plan• Safety POA&M• Review Requirements• System Safety Management Plan• System Safety Program Plan with

SwSPP Appendix• SSWG Charter and Subgroups• Input to SDP• Input to TEMP• Input to SEMP• Input to ILSP • Input to PHL• Input to PHA• Input to CRLCMP

• Program Initiation • Safety Program Management

• Program Initiation

PM, Principal for SafetySE, SWE, SWSE (EVAL)SSWG (Later Milestones)

Identify and Establish System Safety Program Tasks and Requirements

Purpose:

Primary Sub-Processes

Inputs (Suppliers) Outputs (Customers)

Players

Exit ObjectiveEntry Criteria

References in Addition to ¶ 4.1.1

Comments:

Related Sub-Processes

Proceeding Process Next Process

Service Specific GuidanceIEEE 1228FAR’s

• System Safety Program Management Plan

• Proposal Evaluation and Acceptance• Established System Safety Program

• Program Planning and Management• System Safety Management Plan development

SWS PROCESSESSWS PROCESSESPreliminary Hazard Analysis

Preliminary Hazard Analysis - (PHA)

• Identify System Level Causal Factors (I)• Identify Software Level Causal Factors (I)

– Apply Analysis Techniques (for Example SFTA’s) (I)– Develop recommendations to minimize software induced hazards (I)

• Apply HRI and Prioritize Hazards (I)• Apply Risk Assessment Criteria (Categorize) Hazards (I)• Link hazard causal factors to requirements (I)• Develop design recommendations to mitigate hazards (I)

• RHA/SCRA Inputs• PHA Update• Input to S/W Design• Inputs to S/W Development Plan• Input to Preliminary Software Design

Analysis SSHA• Input to Trade-Off Studies• Design Implementation

Recommendations• HARs• Inputs to Software Test Plans

– Test Requirements• Input to SPRA Reviews• Prioritized Hazard List• Input to SSS, SRS, S/SDD, IRS (I)• Input to ECP, SENs, PIPs (I)

• SOW/SOO/RFP

• Risk Assessment Criteria, HRI

• Draft SSS, S/SDD

• Lessons Learned:

– Analyses on Similar Systems

– Incident Reports

– Previous Mishap Causes

• Life Cycle Environment Profile

• PHL • Tailored Generic Software Safety

Requirements List

Inputs Later Milestones)• HARs

• ECPs

• Safety Data Library (SDL) Maintenance• System Level Trade Studies• Software Level Trade Studies

• Tailor The Generic Software Safety Requirements List

• Functional Hazard List Development

• Program Planning/Management

SwSWG, Domain Experts IEEE 1498

• Derived System Specific Software Safety Critical Requirements

• Preliminary Software Design Analysis

• Upon Completion of PHL • Completion of hazard categorization, prioritization, and determination of all causal factors (initial drafts)

• Resolution of identified hazards (completion)

Identification, Classification and Tracking of System Level Software Hazards

Purpose:

Primary Sub-Processes

Inputs (Suppliers) Outputs (Customers)

Players

Exit ObjectiveEntry Criteria

References

Comments

Related Sub-Processes

Proceeding Process Next Process

References in Addition to ¶ 4.1.1

Handbook ChartsHandbook Charts

Preliminary HazardAnalysis (PHA)

Primary Task

Inputs Outputs

Primary Sub-Tasks Critical Interfaces• Identify Sytem-Level Causal Factors• Identify Software Level Causal Factors• Apply HRI and Prioritize Hazards• Apply Risk Assessment Criteria and Categorize Hazards• Link Hazard Causal Factors to Requirements• Develop Design Recommendations

• Software Safety Working Group• Domain Engineers

• SOW/ SOO/ RFP• Risk Assessment Criteria• Draft SSS, S/ SDD• Lessons Learned• Similar Systems Hazard Analyses• Mishap Data• Life Cycle Environmental Profile• PHL• Tailored Rqmts Lists

• Input to RHA/ SCRA• Update PHA• Input to S/ W Design• Input to SDP• Input to Preliminary S/ W Design Analysis • Input to Trade Studies• Safety-specific S/ W Design Requirements• Hazard Action Records• Prioritized Hazard List

Iterative Loop

Developed OutlineDeveloped Outline

Used Process Charts as basisCovered all items in chartAssigned specific sections to primary

authorsAssigned secondary authors to each section

AuthorsAuthors

Selected for particular expertise

Sources sought from each service, industry, and academia

Coordinating author– Selected for expertise in field– Provided “common voice”

through out handbook

AuthorsAuthors Michael Brown,

NAVSEA Dahlgren John Bozarth, EG&G Janet Gill, NAWC AD Brenda Hyland, NAWC

AD Archibald McKinlay

– BAH

Lenny Russo, CECOM

Coordinating Author– Steve Mattern, SEA

Task AssignmentsTask Assignments

Based on author’s area of expertise– Prior experience– Interest in topical area

Secondary author– Prior experience– Area of Expertise

ApplicabilityApplicability

Process applicable to wide range of military and non-military systems– Weapon Systems– Fire Control and Guidance Systems– Operational Flight Control Programs– Any system containing safety critical software

Handbook provides tailoring guidance for wide range of programs

StatusStatus

Submitted draft to community for review and comment October 1997

Received and collated commentsComments adjudicated by committeeDeveloped revisions based on comments

and additional input from authorsForward revised draft to reviewersFinalized handbook in November 1999

Collating commentsCollating comments

Comments are annotated next to applicable paragraph– Source identified– Comment as provided entered– Rationale for acceptance, rejection, or

modification provided.

Handbook with comments placed on web page

4.2.3 TAILORING GENERIC SAFETY-CRITICAL REQUIREMENTS

Figure 4-20 depicts the software engineering process for tailoring the generic safety-related software requirements list. Generic software safety requirements are those design features, development processes, "best practices", coding standards and techniques, and other general requirements that are levied on a system containing safety-critical software, regardless of the functionality of the application. The PHL, as described above, may help determine the disposition or applicability of many individual generic requirements. The software safety analysis must identify the applicable generic software safety requirements necessary to support the development of the Software Requirement Specification (SRS). A tailored list of these requirements should be provided to the developer for inclusion into the software requirements document.

MLB1: Include references to coding standards and guidelines.

Rationale: Provide validity and additional reference material.

Comment: References provided in appendix X. Hyperlinks to appendix to be provided in final document.

JAG3: Include requirements for evidence supporting tailoring of generic safety critical requirements.

Comment: Concur. Guidelines provided in appendix Y. Appropriate hyperlinks provided in final document.

PROJECT STATUS - Mar PROJECT STATUS - Mar 20012001

First Publication - 31 December 1999– Handbook text complete

All topical areas addressed Some areas require additional work

– Appendices approximately 75% complete Need to develop additional examples for guidance

– Based on previous programs and lessons learned

Need to incorporate additional reference documents

Funding Sources to dateFunding Sources to date

Joint Systems Engineering Steering Group– Funding source: Army

Naval Ordnance Center– WSESRB

USAF HQ-AFSC– Lt. Col. Alberico

Naval Facilities Engineering CommandPersonal time by authors

Additional TasksAdditional Tasks

Commercial Developed Items, Government off-the-shelf (legacy) items, and Non-Developmental Items (Hardware and Software) in safety critical applications

– Selection criteria– Design requirements and

guidelines– “How to” guidelines on

influencing system and software architecture, design, and implementation

– “How-to” analysis and testing guidelines

– Configuration Control Requirements

Additional TasksAdditional Tasks

Software Safety Risk Assessment

– Relating software causal factors and control categories to Hazard Risk Indices

– Relating software metrics to hazard risk assessment

– Guidelines for assessing adequacy of safety program efforts

Hazard Risk IndexFor Example Purposes Only

SeverityProbability

I II III IVCatastrophic Critical Marginal Negligible

1 3 7 13

2 5 9 16

4 6 11 18

8 10 14 19

12 15 17 20

(A) FREQUENT 1 IN 100

(B) PROBABLE 1 IN 1,000

(C) OCCASIONAL 1 IN 10,000

(D) REMOTE 1 IN 100,000

(E) IMPROBABLE 1 IN 1,000,000

Unacceptable Risk - Acquisition Executive Resolution Or Risk Acceptance

Marginal Risk - Design To Eliminate, Requires Program Manager Resolution Or Risk Acceptance

Minimum Risk - Design To Minimize, Requires Program Manager Resolution Or Risk Acceptance

Software Hazard Criticality Matrix

For Example Purposes Only

High Risk - Significant Analyses and Testing Resources

Medium Risk - Requirements and Design Analysis and Depth Testing Required

Moderate Risk - High Levels of Analysis and Testing Acceptable With Managing Activity Approval

SeverityControl Category Catastrophic Critical Marginal Negligible

(I ) Software exercises autonomous control over potentially hazardous hardware systems, subsystems or components without the possibility of intervention to preclude the occurrence of a hazard. Failure of the softwareor a failure to prevent an event leads directly to a hazards occurrence.

(I Ia) Software exercises control over potentially hazardous hardware

systems, subsystems, or components allowing time for intervention by independent safety systems to mitigate the hazard. However, these systems by themselves are not considered adequate.

(I Ib) Software item displays information requiring immediate operator

action to mitigate a hazard. Software failure will allow or fail to prevent the hazard’s occurrence.

(I I Ia) Software items issues commands over potentially hazardous

hardware systems, subsystem, or components requiring human action to complete the control function. There are several, redundant, independent safety measures for each hazardous event.

(I I Ib) Software generates information of a safety critical nature used to make

safety critical decisions. There are several, redundant, independent safetymeasures for each hazardous event.

(IV) Software does not control safety critical hardware systems, subsystems,

or components and does not provide safety critical information.

1 1 3 5

1 2 4 5

1 2 4 5

2 3 5 5

2 3 5 5

3 4 5 5

Extracted from Mil-Std 882C

Moderate Risk - High Levels of Analysis and Testing Acceptable With Managing Activity Approval

Low Risk - Acceptable

Additional TasksAdditional Tasks

Guidelines for selection and/or implementation of:– Language– Operating Systems– Operating

Environments– Middleware

– Pros and Cons for each language, OS, OE, etc.

– Selection and implementation guidelines and criteria

– Analysis and testing guidelines

– Caveats and requirements for each language, etc.

– Configuration control

Additional TasksAdditional Tasks

“Software like” hardware: FPGAs, PLDs

Define safety assessment process and develop guidelines

FUTURE OBJECTIVESFUTURE OBJECTIVES

Hypertext CD-ROM Integrate into DoD Acquisition Deskbook Software Safety WWW Home Page Revisions & updates to handbook On-Site Software Systems Safety Training System Safety Handbook Tool Development

ConclusionsConclusions

The JSSSH provides a comprehensive, Systems Engineering based approach to ensuring that software executes safely within the system context.

The process is designed for application to a wide range of systems without the need for highly specialized expertise (e.g., formal methods)

The JSSSH provides a basis against which to evaluate the thoroughness of Software Systems Safety Programs

The JSSSH is a useful guideline for any safety critical system