25 October 2017

Cyber M&S: Bridging the Gap between

Cyber Effects and Mission Impact

Presented to:

AlaSim International Conference

Keynote Presentation

Presented by:

Tom Barnett

Cyber Technology Principal Investigator

U.S. Army Aviation and Missile Research,

Development, and Engineering Center

DISTRIBUTION STATEMENT A: Approved for public release. Distribution is unlimited.

2

Talking Points

3

To hopefully demystify cyber just a little

To share some observations from the DoD cyber world

To briefly discuss cyber within a SE&I context

To provide the context to discuss how cyber relates to

YOUR day job

To offer a slightly different perspective on this thing that has

everybody so excited

But mostly, to encourage you to ask the most important

question about a cyber attack…

Why am I presenting today?

4

What is Cyber Security?

5

Information Assurance?

PERIMETER, PATCH, PRAY

6

Hacking?

Partial Internet Shutdown

7

Trusted Systems and

Hardware/Software Assurance?

Gotta do it, but remember…

Trust is just a feeling!

8

Network Defense?

Necessary but not Sufficient

9

Protecting Financial Systems

and Critical Infrastructure?

This is where things get Physical!

10

Compliance and Risk Management?

Compliance ≠ Security

11

CyberSPACE

12

• DoDI 5000.02 Operation of the Defense Acquisition System

• DoDD 8500.01 Cybersecurity

• DoDI 8510.01 Risk Management Framework (RMF) for DoD IT

• FM 3-12 Cyberspace Operations, 20 May 2015

• Defense Acquisition Guidebook (DAG)

– Chapter 9, Test and Evaluation

– Chapter 13, Program Protection

• CJCSI 3170.01 JCIDS Manual

• DoD Cybersecurity Test and Evaluation Guidebook

• CNSS 1253 w/overlays.

• DoDI 5200.44 Protection of Mission Critical Functions to Achieve Trusted Systems and

Networks

• Memo for users of the DOT&E TEMP Guidebook, 16 Nov 2015

• FM 3-38 (CEMA)

• Cyber OT&E Gilmore Memo–Aug 2014

• JCIDS Systems Survivability KPP – Feb 2015

• Cyber Security R&D Strategic Plan - Feb 2016

• PM Guidebook for Integrating RMF

• DOD Cyber Strategy

• DOD Cybersecurity Risk Assessment Guide - 2014

• NIST SP 800-160 System Security Engineering

• NIST Cybersecurity Framework

Policy, Guidance and Best Practices

13

But how does that relate to our mission???

Typical Cyber Threat Matrix

14

Cyber Security “State of the Art”

14

95%

80%

60%

85%

100%

15

Defense in Depth

15

16

• Cyber is all the things I just described…and more!

• The cyber lexicon seems out of touch with traditional Systems Engineering

• System Capability is seldom considered

– Cyber “Vulnerability” ≠ System Vulnerability

• We tend to get caught up in the minutia so we feel like we’re doing

something about it

– Activity ≠ Accomplishment

• We adopt abstractions like Risk* because we don’t understand it

– Cyber risk is simply a component of system risk

– System risk must be based on things that can be measured

Observations

It’s not a computer problem…it’s an ENGINEERING problem!

* Risk is a unitless measure used to justify bad decisions

17

Cyber Vulnerabilities- NIST Definitions -

A vulnerability is a weakness in an information system, system security

procedures, internal controls, or implementation that could be exploited by a threat

source. The severity of a vulnerability is an assessment of the relative importance of

mitigating/remediating the vulnerability.

The likelihood of occurrence is a weighted risk factor based on an analysis of the

probability that a given threat is capable of exploiting a given vulnerability (or set of

vulnerabilities).

A predisposing condition is a condition that increases or decreases the likelihood

that threat events will result in adverse impacts. The concept of predisposing

condition is also related to the term susceptibility or exposure.

The level of impact from a threat event is the magnitude of harm that can be

expected to result from the consequences* of unauthorized disclosure of

information, unauthorized modification of information, unauthorized destruction of

information, or loss of information or information system availability.

- NIST SP-800-30

In today’s cyber community, a vulnerability is associated with

a “state of the system” without regard to predicted impact:

It’s about the INFORMATION SYSTEM!

* These are cyber “effects”

18

Vulnerability- An Aircraft Survivability Example -

Aircraft survivability (PS) is defined as the capability of an aircraft to avoid or withstand

hostile environments. It can be measured by the probability the aircraft survives an

encounter with the environment (i.e. threat weapon).

Susceptibility is the inability of an aircraft to avoid the hostile mission environment. The

more likely an aircraft is hit by a threat weapon, the more susceptible is the aircraft, thus:

Susceptibility = PH

[Susceptibility can be reduced via stealth, ECM, chaff/flares, and avoidance tactics]

Vulnerability is the inability of an aircraft to withstand the hostile environment. The more

likely an aircraft is killed by the hit(s) from the threat weapon, the more vulnerable is the

aircraft. Vulnerability can be measured by the conditional probability the aircraft is killed

given that it is hit, thus:

Vulnerability = PK|H

[Vulnerability can be reduced by protecting critical components via armor, placement, or robust design]

In the Defense community, a vulnerability is directly

associated with the predicted impact caused by the threat:

It’s about the MISSION!

19

Today’s approach to Cyber Vulnerabilities

* i.e. Vulnerability Whack-a-Mole

Classical Risk ≈ f (Likelihood, Impact)

Cyber Risk ≈ f (Vulnerability, Threat, Impact)

Without quantitative consideration of Threat and Impact, then at best:

Cyber Risk = Σ(Cyber “Vulnerabilities”)

Which creates a bias toward protecting boundaries and identifying inherent

weaknesses and a virtually exclusive focus on the “Tier 3” aspects of Risk

The consequences of this approach are that :

− A “compliance-focused” paradigm

− There are few tangible discriminators between these intrinsic weaknesses

− There is little basis for prioritization, and

− Each tends to be considered and mitigated independently*

20

Where does cyber fit?

“... For operational plans development, the combination of threats, vulnerabilities, and impacts must be evaluated in order

to identify important trends and decide where effort should be applied to eliminate or reduce threat capabilities; eliminate or

reduce vulnerabilities; and assess, coordinate, and deconflict all cyberspace operations...”-- THE NATIONAL STRATEGY FOR CYBERSPACE OPERATIONS

OFFICE OF THE CHAIRMAN, JOINT CHIEFS OF STAFF

21

First think about WHAT you do…

Now think about WHY you do it!

22

How do we balance Capability and Security?

Completely Functional

but

UNPROTECTED

Completely Secure

but

Non-functioningCAPABILITY SECURITY

23

Evolution of Warfare in the Information Age

PastAirLand Battle

TodayNetwork Enabled

FutureMulti-Domain Battle

(U) Battle fought kinetically

within well defined

geographical boundaries

(U) Battle system now

dependent on network and

communication functions to

support kinetic engagement

and maneuver operations

(U) Battle will now be fought

across physical and virtual

domains simultaneously, the

outcome of each impacting

the other.

RELIANT

24

A Cyber Attack is simply a Threat that attacks Computational Infrastructure!

When you think about it…

25

Cyber Resilience- Basic Tenets -

• Cyber is more than Information Assurance, it’s about Mission Assurance

• Embedded hardware and software must be treated differently than

Enterprise Networks

• We must DEFEND THE CAPABILITY to ensure resiliency in a cyber-

contested environment

• We must translate cyber effects into system terms

– Only then can actual Impacts be determined

• Impact should be estimated at the System/Mission Level

– Must be reflected in measures of performance/effectiveness degradation • i.e. engagement timeline, single shot PK, inventory wastage, etc.

– Cyber Effects (Deny, Degrade, Disrupt, etc.) are NOT Mission Impacts

Cyber must always be viewed through a MISSION lens

26

• We must consider Cyber when developing/assessing Systems– We can no longer assume integrity of information

– Cyber security can no longer be assessed independent of performance

– Cyber solutions must be [re]engineered into the system

– We must reassess critical functions and dependencies

This is where M&S plays a critical role

• The key: Consider cyber as a “non-kinetic threat”

– Then treat it just as you do any other threat that affects performance

– Determine the cyber effects to which the system is Susceptible

– Describe the cyber effect in Quantifiable Terms

– Translate each Quantified Cyber Effect into the associated System Effect

– Perform System Analysis to find the corresponding Mission Impact

Rethinking the Cyber Context

The Goal: Develop Resilient Systems that are capable of

operating in a Contested Cyber Environment

“We cannot solve our problems with the same thinking we used when we created them.”

-- Albert Einstein

27

Traditional SE&I Context- Simulation, Test & Evaluation -

28

Cyber Security SE&I- Cyber Simulation, T&E -

Cyber attacks are threats that target the system’s

infrastructure with impacts realized at the mission level

29

How do you represent Cyber in M&S?

“Look beyond the numbers. Around them. Through them.”- Kevin Costner as Al Harrison in “Hidden Figures”

30

To model cyber, you have to know how to measure it.

How do you measure it?

YOU DON’T! *

* You measure it’s impact on your performance

31

System Simulation- Representing Cyber in Performance Models -

We must first characterize cyber effects in order

to determine the associated performance impact

32

Capability View

Materiel SolutionsTechnical Requirements

System Specifications

Interface Specifications

Warfighting

Capability

Capability GapOperational Need/Request

Initial Capabilities Document

JCIDS

System and Threat Representation

in Performance Simulations

System DesignSubsystems

Algorithms

Functional Flows

System CapabilityPerformance

Timing

InteractionsMission View

Quantified ThreatKinetic (Missile, FW, RW)

Electronic (RF, IR, Comms)

System EffectPerformance Degradation

- Battlespace

- Kill Probability

System ImpactMissed Engagement

Dropped Track

Mission EffectThreat Leaker

Asset Destroyed

Mission ImpactDecreased Effectiveness

Fire Unit Loss

System and Threat

Explicitly Represented

33

Capability View

Materiel SolutionsTechnical Requirements

System Specifications

Interface Specifications

Warfighting

Capability

Capability GapOperational Need/Request

Initial Capabilities Document

JCIDS

Cyber View

Cyber Security

Hardening/Compliance

Perimeter Defense

VulnerabilitiesWeakness/Flaw

Misconfiguration

User Error

Attack Surface

Cyber Effects Injection into

Performance Simulations

IT DesignHardware/Software

Networks

IT InfrastructureHardware/Software

Networks/Protocols

Information Ops

Defensive Cyber Ops

Cyberspace Defense

Network Defense

Mission View

System ImpactDropped Track

Missed Engagement

System EffectTemporary Loss of

Critical Tactical Link

Quantified

Cyber EffectRouter A is out for

90 sec. @ t=xxx

Mission EffectAdditional Threat Leaker

Mission ImpactDecreased Effectiveness

Cyber Effect (CIA)Denial of Service

Degraded

Disrupted

Cyber Attack

Inject System Effects

X XX

34

Conclusion

• No matter your industry, anchor your cyber efforts to your mission– Know thine self

– Know what matters

• The only way to know where you really stand with cyber is to

determine it’s impact to your mission– How bad is it?

– Do you need to do something about it?

• If you can get the cyber guys to QUANTIFY their cyber effects, you

can inject them into your M&S…then just do what you do!

35

What’s your “So What”?

36

Questions?

37

AMRDEC Web Site

www.amrdec.army.mil

Facebook

www.facebook.com/rdecom.amrdec

YouTube

www.youtube.com/user/AMRDEC

Twitter

@usarmyamrdec

Public Affairs

AMRDEC-PAO@amrdec.army.mil