Integrated Survivability Assessment (ISA) for OT&E

Ron KetchamChairman, JASPO Survivability Assessment SubgroupNaval Air Warfare Center, Weapons DivisionCode 418100D(760) 939-2363 (DSN 437-2363) FAX: (760) 939-2062

Dave HallSURVICE Engineering Company900E N. Heritage Drive, Suite 1Ridgecrest, CA 93555

(760) 446-4624 X106; FAX: (760) 446-2424; Cell: (760) 382-1618

Cleared for Public ReleaseDistribution Unlimited

ISA Project

• Motivation: comprehensive OT&E system survivability evaluations

• Tasking from Joint Aircraft Survivability Program Office (JASPO)– At the request of DOT&E

• Objectives:– Combine the results of LFT&E and OT&E into an overall

assessment of the survivability of a system under test– Adaptable for use in all phases of system development and

fielding from Concept Definition through FOT&E• Make use of appropriate M&S and T&E Resources

2. Crew Protection

5. DefensiveECM/IRCM

10. OffBoard

Jamming 9. SEAD &DEAD

3. DamageResistance

1. DamageTolerance

13. Tactics &Doctrine

12. Policy & ROE

4. Signature Reduction

7. Aircraft Performance

8. Precision & Standoff Weapons

11. MissionPlanning

6. Situational Awareness

Susceptibility

Vulnerability(LFT&E)

Other

Elements of Survivability

Outside Factors

Affecting Susceptibility

Tasking

• Develop a combined survivability OT&E/LFT&E Checklist to evaluate all aspects of the survivability of air vehicles

• Develop metrics to evaluate the checklist

• Develop an ISA process for measuring those metrics using current JASPO and DOT&E methodologies, T&E range capabilities and tools

• Identify deficiencies in the ISA process (M&S, data, T&E, etc.) and develop detailed plans to mitigate them

• Exercise the ISA process for one or more acquisition programs

Integrated Survivability Assessment Process

Mission-Threat

Analysis

EW/ECTargetAcquisition

SignaturePrediction

SensorModels

EW/EC

SAM

AAM

Guns

Lasers

Threat EngagementAssessment

VulnerabilityAssessment

Air-to-AirCombat

Air-to-GroundMissions

MissionSurvivabilityAssessment

RequirementsDefinition

AOA

OT&E/LFT&E

SpecificationCompliance

MissionPlanning

Susceptibility Assessment

Integrated Survivability Assessment

• A process to evaluate all aspects of system survivability in a coordinated fashion– Using both M&S and T&E resources where appropriate– Including LFT&E and survivability OT&E issues

• Perform survivability OT&E and LFT&E in the context of missions and scenarios for the system under test

– By using appropriate “vignettes”

• Vignette Matrix identifies potential OT&E/LFT&E test conditions– Provides a framework for developing LFT&E and survivability OT&E test plans

• Negotiated between PM, Service OT, DOT&E– Provides a roadmap for use of M&S in support of TEMP– Relates to JSF, AIM-9X experience

• Developed example matrices of vignettes representing missions & scenarios for three aircraft types

– Helicopter, Transport, TACAIR

Example Helicopter Vignettes

Urban Forest Desert

AFGHAN

Mountains

SWANEASomalia

Drivingfactors

Close engagement range, hard to find targets

IADS, weather, RF threats, MANPADS, target acq.

Flat terrain, clear weather, CB threat

High altitude,rough terrain

X

X

X

X

X

X X

X

X

X X

X

X

X

X

X

X

X

X

X

X

X

X

Attack Helo

AIR CAV

Takeoff &Landing

CAS

BattlefieldInterdiction

Air MobileEscortCSAREscort**

RECCE*

SCREEN

Target Acq

Air Combat

10

10

5

10

5

30

20

10

100

SampleWeights

MissionsRoles

*Most likely mission**Most stressing mission

Ж = Moststressingscenario

Ж

Ж

Ж

Ж

Ж

Ж

Ж

Ж

Ж

Example Helicopter Vignette

• Close Air Support to troop insertion mission in Operation Anaconda– Combined operation with AH-64, UH-60,

CH47, CAS assets• Identified LZ location, mission

objectives• Rugged terrain • Uncertain threat situation

– Uncertainty in threat means that a number of variations in the vignette should be considered

• To determine “robustness”, or sensitivity of survivability results to threat assumptions

Mission Level Survivability

Engagement Level

Susceptibility

•Threat Envelopes (with and w/o CM)•F-Pole, A-Pole, E-Pole

•Aircraft Pk/h (or damage given a hit or an intercept)

•Threat System Pk Envelopes•Robustness

•Vignettes Accomplished: percentage of vignettes where mission can be

accomplished considering survivability constraints

•Threat Shot Opportunities•Robustness

Engagement Level

Survivability

•Expected # casualties given a hit

Crew Survivability

Vulnerability

Principal Survivability Metrics

Pk, OT&E, M&S and LFT&E(for Survivability Assessment)

PK/E = PA/E*PT/A*PL/T*PI/L*PF/I*PH/F*PK/H

OT&E M&S LFT&E

E = Engagement A = AcquisitionT = TrackL = Launch

I = InterceptF = FuzingH = Hit

K = Kill

Conduct T&E

Characteristics of System under Test

CreateVignettes

Applicable to Program

Assess M&SNeeds

Execute and Collect data from M&S Suite

Conduct AnalysisAssess and combine M&S

and T&E Results

Survivability Metrics

Existing Vulnerability DataCollect

Data from LFT&E

Collect Data from

OT&E

Library of M&S (incl.Credibility Information)

Generic Vignettes

Program Documents(Example: TEMP, ORD)

Data Collected from DT&E, LFT&E(to date and expected)

Data Collected from OT&E (to date and expected)

Existing Susceptibility Data

ISA Process

Process Guidance and Constraints

Develop Test Plans

Select M&S SuiteBest-Suitedfor Program

Mission and System Level Test

Mission

Tasks (UJTL)

Interactions

Actions

Tech Demo

Force Development/ Experimentation

Operational Test

Components

Technology

Interoperability Testing

Subsystem Test

Component Testing

AN

ALY

ZES

YN

THE

SIZ

E

Underlying Figure taken from ITEA Journal, Sept/Oct 2003, “The Test and Evaluation Master Plan: An Operational View”. Daly, Miller, Seglie & Wauer

Test Via

System Vignettes&

Survivability Metrics

Some Known Deficiencies in ISA Process

• General Issues– Model linkages– M&S Validation data availability, coordination with TEMP– Mixed force level assessment (T&E and M&S)

• Modeling and Simulation Deficiencies – DECM/IRCM effectiveness– Near field signatures, threat fuze performance– Fire and Explosion– Human performance– C4ISR

• Test Range Deficiencies– Missile Miss Distance Measurement– Threat System Variability – system to system variations– Signal Density – not representative on ranges– Semi-Active systems – can’t get realistic test data on ECM effectiveness, etc.– Number of platforms, threats in test, test range size – can’t fully test integrated

system– Completeness & fidelity of OT&E data – not generally usable for M&S validation,

sometimes not sufficient for test issue resolution

ISA DemonstrationISA Demonstration

– Demonstrate ISA process for an actual acquisition program

• First candidate: Multi-Mission Maritime Aircraft (MMA)– FY04 Plans:

• Develop focused ISA plan for MMA• Develop and coordinate checklist, metrics and vignettes

with program office, LFT&E and OT&E agencies• Begin developing database requirements to support ISA

assessment

Status & Recommendations• Draft ISA process developed

– Identified checklist, metrics, models, test data needed – Notional examples worked (Transport, TACAIR, Helicopter) – Identified deficiencies in process, models and data– Draft report reviewed by Tri-Service committee

• Developing detailed plans for process improvements– Fleshing out detailed ISA requirements for test data and M&S– Initiating test range capabilities assessment against those

requirements• Correlated with ongoing range capabilities improvements

– Provides roadmap for both M&S and T&E investment

• Recommendations:– Complete and execute ISA process improvement plan

• Coordinate with CTEIP and JASPO funding processes• Investigate POM plus-up opportunities• Investigate Training Range capabilities and limitations to support ISA

– Develop detailed procedures manuals to supplement TEMP– ISA Demonstration

Backups

Metrics and the ChecklistMetrics and the ChecklistLinks in the Threat Kill Chain

ISA Metrics Potential Survivability Enhancement Features Along the Kill Chain

Mission Survivability

Missions Accomplished; robustness All features combine to support mission level survivability

Threat Suppression

Threat Shot opportunities; situational awareness (number, timeliness and accuracy of threats detected)

Tactics, Precision Guided Munitions, mission planning system, low signatures, fighter escort, anti-radiation missiles, self defense weapons

Detection Avoidance

Threat Detection & Acquisition Envelopes Standoff Weapons, Night-time capability, on-board electronic attack (EA) systems, stand-off EA, low signatures, good target acquisition, terrain following (NOE flight), situational awareness, chaff, threat warning, tactics, mission planning system

Engagement Avoidance

Threat Tracking envelopes; F-Pole, A-Pole, E-Pole; ECM effectiveness

Standoff Weapons, onboard EA, off-board EA, low signatures, good target acquisition, situational awareness, chaff and flares, threat warning, speed and altitude, mission planning system

Threat or Hit avoidance

Threat Intercept Envelopes; ECM/IRCM effectiveness

On-board EA, low signatures, chaff and flares, threat warning, speed and altitude, maneuverability, agility (last ditch maneuver)

Threat or hit tolerance

Threat system Pk envelopes; Aircraft Pk/h; Component Pk/h; VA; list of vulnerable components; Expected # casualties given a hit; hit locations on aircraft

Fire/explosion protection, self-repairing flight controls, redundant and separated hydraulics, multiple engines, no fuel adjacent to air inlets, hydrodynamic ram protection, nonflammable hydraulic fluid, rugged structure, armor