Aggregating T&E Data for CCoD Trust in Operational Implementation

Approved for Public Release: 10-9999. Distribution Unlimited© 2010 The MITRE Corporation. All rights reserved

Aggregating T&E Data for CCoD Trust in Operational Implementation

Suzanne M. Beers, Ph.D.The MITRE Corporation

Operations Research & Systems Analysis Department (E525)Colorado Springs, Colorado

Presented to: 27th International Symposium on Military Operational Research1 September 2010

© 2010 The MITRE Corporation. All rights reserved

Background

Composable Capability on Demand (CCoD)– MITRE research focus area– Provides means to develop agile information architectures– Rapidly meet warfighter mission needs

Implementation challenge: Trust in composed capability

Fuzzy Logic– Multi-valued logic using fuzzy sets– Successfully used in control and analysis applications– Allows humanistic reasoning, gradual transition between states

Decision support system: Aggregate component T&E data

2

sglasser

I’ve seen multiple definitions. The core of CCOD is rapid, customizable, to meet warfighter mission needs.Composable Capability on Demand (CCoD) promises the ability to rapidly customize virtual systems based on the mission and threat of the day.From the MIP strategic plan:Composable Capability on Demand (CCOD) is envisioned to be a set of technical means and constructs that will enable DoD and civilian users to dynamically assemble and employ elements of the C4ISR enterprise to successfully accomplish missions.


Overview

What’s Composable Capability on Demand (CCoD)?– Implementation problem statement– CCoD trust taxonomy

What’s fuzzy logic? Fuzzy logic-based CCoD Decision Support System (DSS)

– Structure– Example DSS and results

Conclusion

3


What’s CCoD?

4

Broker

Discover &

Subscribe

Loosely

Coupled

Services

Producers Consumers

Describe &

Publish

Composable Architectures for Net-centric OperationsRapidly Changeable to Meet Mission Information Needs

Composable Architectures for Net-centric OperationsRapidly Changeable to Meet Mission Information Needs


One instantiation of CCOD: Mashups

Mashup is a web application that brings together several sources of data to form a unique new combination of information

EMML, Enterprise Mashup Markup Language– Language for creating

enterprise mashups– Software applications that

consume and mash data from variety of sources

– Output presented in graphical user interface, widgets, gadgets

5


…for example

6

Mashup: Chicago crimeCCoD: Satellite tracking


Composed Capability Trust Problem

Take the components “off the shelf”

Compose a CCoD C2 capability

Tell the user WHATto give him confidence

to use the system

To know he’ll make theright moves based on

the information

Do I believe this info?Should I act on it?

What should I DO???

7


CCoD Implementation Challenge

Lives could depend upon the answer… How to build trust without slowing down the process?

Use existing component T&E data– Software quality metrics

Aggregate using fuzzy logic– Provide trust measure at composed capability level

8


Software Trust Metrics

Component Quality Models (CQM)– Evaluate the quality of reusable software modules– Typically start with ISO/IEC 9126 standards– Define characteristics, sub-characteristics, attributes/metrics

CCoD Trust Taxonomy developed by MITRE– Trust: to have confidence in; depend on

Formal acquisition: requirements development then T&E process CCoD: collect metrics during private to public state transition

– Categories CCoD environment Component Composition Component Developer & Composer: proficiency

– Conducted decision-maker survey – ID’ed important factors

9


CCoD Trust Taxonomy – Environment

10


CCoD Trust Taxonomy – Component

11


CCoD Trust Taxonomy – Composition

12


T&E Data – to – Composed Capability Trust

13


CCoD Trust Taxonomy – Top Level Component Quality Characteristics

14

Provide required services under specified conditions

Protect info and data; unauthorized cannot read/modify; authorized not denied access

Maintain a specified level of performance

Usable by someone other than developer

Provide appropriate performance relative to amount of resources used

Can be modified


CCoD Trust Taxonomy - Functionality

15


CCoD Trust Taxonomy – Security

16


CCoD Trust Taxonomy - Reliability

17


CCoD Trust Taxonomy - Usability

18


CCoD Trust Taxonomy - Efficiency

19


CCoD Trust Taxonomy - Maintainability

20


Down-sized CCoD Decision Hierarchy

21

Provide correct results w/ needed precision

Provide appropriate set of functions

Interact w/ other components

Prevent information disclosure

Only modifiable by authorized users

Re-establish performance & recover data

Understand if suitable / how used

Provide required services

Protect info & data

Maintain performance

Non-creator usable


Why fuzzy logic?

Multi-valued logic based upon fuzzy set theory– Each element in a fuzzy set has membership value

A(u) [0,1] – Linguistic variables related in if – then rule bases

Ideal for representation and analysis of imprecise dependencies – Lowers the cost of products by simplifying programming

Define fuzzy sets, define rule bases relating the sets– Captures human-reasoning

Wide variety of successful control and analysis applications– Control: vacuum cleaners, video cameras, subway systems– Analysis: optimize manufacturing lines,

predict insurgent network behavior, aging

22


Fuzzy Inference Structure

Crisp Output

Crisp Input

Fuzzy Input

Fuzzy OutputInference Engine

Defuzzification

Fuzzification

23


Fuzzification

Turning a Crisp (Numerical) Value Into a

Degree of Activation of a Fuzzy Set

24


Fuzzy Inference -- Rule Base

IF Height is MED AND Weight is LIGHT THEN Build is SMALL

IF Height is SHORT AND Weight is MED THEN Build is MEDSMALL

MEDmax

25


Center of Area Defuzzification

yCOA = i=1 yi C (yi)

i=1 C (yi)q

q

1 5 6 7 8 9 10 112 3 4

1/3

2/3

yCOA = 1*0 + 2* 1/3 + 3* 2/3 + 4* 2/3 + 5* 2/3 + 6* 2/3 + 7* 2/3 + 8* 1/3 + 9* 1/3 + 10* 1/3 +11*0

0+ 1/3 + 2/3 + 2/3 + 2/3 + 2/3 + 2/3 + 1/3 + 1/3 + 1/3 +0

yCOA = 5.642 26


Strategy-to-Task Operational Test AnalysisReduction

in Probability of Kill

Reduction in Guidance

Reduction in Hits

Increasein Break Lk

Track onJam

ResponseTime

Increase inTrack Error

%RIH => Pk

%RIG => Pk

%IBL => Pk

%TOJ => Pk

%ITE => Pk

Response Time =>Pk

^

RIH Fuzzy Set

RIG Fuzzy Set

IBL Fuzzy Set

TOJ Fuzzy Set

TE Fuzzy Set

Response Time Fuzzy Set

Pk Fuzzy Set

27


Example Fuzzy-Logic CCoD Decision Hierarchy

28

Provide correct results w/ needed precision

Provide appropriate set of functions

Interact w/ other components

Prevent information disclosure

Only modifiable by authorized users

Re-establish performance & recover data

Understand if suitable / how used

Provide required services

Protect info & data

Maintain performance

Non-creator usable


Fuzzy Logic DSS – Test Data to Component Trust: Sample Fuzzy Sets

29


Fuzzy Logic DSS – Test Data to Component Trust: Sample Rule Base (Functionality)

30


Fuzzy Logic DSS – Test Data to Component Trust: Sample Results

31


Fuzzy Logic DSS – Test Data to Component Trust: Sample Results

32

Reliability UsabilityAccuracy Suitability IntInterop Confidentiality Integrity Recoverability Understandability

All Zero 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.17All Bad 0.20 0.20 0.20 0.20 0.20 0.20 0.20 0.20Two Bad 0.20 0.20 0.20 0.20 0.20 0.90 0.90 0.53Two Bad 0.90 0.90 0.90 0.20 0.20 0.20 0.90 0.53Two Bad 0.90 0.90 0.90 0.90 0.90 0.20 0.20 0.53All Medium 0.50 0.50 0.50 0.50 0.50 0.50 0.50 0.50Mixture 0.67 0.75 0.25 0.35 0.85 0.67 0.90 0.75One Bad 0.90 0.90 0.90 0.90 0.90 0.90 0.20 0.70One Bad 0.20 0.20 0.20 0.90 0.90 0.90 0.90 0.70All Good 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.83All Perfect 1.00 1.00 1.00 1.00 1.00 1.00 1.00 0.83

Functionality SecurityComponentTrust


Fuzzy Logic DSS – Component Trust to Composition Trust: Sample Fuzzy Sets

33


Fuzzy Logic DSS – Component Trust to Composition Trust: Sample Rule Base

34


Fuzzy Logic DSS – Component Trust to Composition Trust: Sample Results

35

Description Component1 Component2 Component3 CompositionTrustAll Zero 0.00 0.00 0.00 0.17All Bad 0.25 0.25 0.25 0.22Two Bad 0.25 0.30 0.95 0.47All Medium 0.50 0.50 0.50 0.50Mixture 0.67 0.25 0.85 0.59One Bad 0.25 0.95 0.95 0.67All Good 0.90 0.90 0.90 0.83All Perfect 1.00 1.00 1.00 0.83


Conclusion

CCoD provides a means to rapidly develop agile information architectures

Warfighter needs trust in composed capability

Fuzzy logic provides easily-built decision support

Future research/work focus areas– Incorporate real-time decision-maker risk preferences– Refine metrics, definitize data sources, construct automated data

collection/cataloging – Develop Web 2.0-like tool to operationalize

36

Aggregating T&E Data for CCoD Trust in Operational Implementation

Documents

Transcript of Aggregating T&E Data for CCoD Trust in Operational Implementation