OneSAF Validation & Verification

April 2nd, 2009Ha Ly

Capability Development Process

ImplementationSoftware Development

ImplementationSoftware Development Integration & TestIntegration & TestConceptual Modeling

Knowledge Engineering

Conceptual ModelingKnowledge Engineering

Systems EngineeringSystems Engineering

A&IBaseline

A&IBaseline

ChangeRequest

RIBRequirements

RIBRequirements

Build 1 Build 2 Build 3

Integration & TestIntegration & Test

PLRSPLRS

Version X.Y

Implementation Build Cycle

10 Week Builds

Design2 Weeks

CUT3 Weeks

SWIT3 Weeks

RA2 Weeks

- CDD- DBD-PKAD-RIB

- Update CDD-SRS-PLRS Mapping-RA Review

- Implementation Document-UML Diagrams-GUI Mock-ups-Peer Design Review-TPO Design Review

-Code-Composites-Update ID-CUT Review

-Test Thread-Test Thread Scenarios-SWIT Review-Update Documentation-Built baseline tar-ball

Integration & Test

Conceptual ModelingKnowledge Engineering

Hierarchical Test Strategy

Class / Unit Test

Component / Framework Test

System Integration

Test

Major Release

Class / Unit Test

Component / SW Integration Test

Product / SystemIntegration Test(Informal and Formal)

CapabilityHandover

TestingR

egre

ssio

n T

esti

ngR

egre

ssio

n T

esti

ng

Capability Handover

User AssesmentBaselines

EngineeringSVN Module

I&T SVNModule

I&T Baseline

Reg

ress

ion

Tes

ting

Use

r A

sses

smen

ts

TEST ORGANIZATIONS ROLES AND RESPONSIBILITIES

Developers

A&I Team A&I Team

AAR Team AAR Team

C4I Team C4I Team

ERC Team ERC Team

EDM Team EDM Team

MSDE Team MSDE Team

Models & ToolsTeam

Models & ToolsTeam

Co-DeveloperTeam 1

Co-DeveloperTeam 1

Co-DeveloperTeam N

Co-DeveloperTeam N

Integration and Test

A&I I&T Team A&I I&T Team

Acquisition Agent

User Representatives

TPO OneSAF TPO OneSAF

ACR Domain ACR Domain

RDA Domain RDA Domain

TEMO Domain TEMO Domain

PM OneSAF PM OneSAF

DA Pam 5-11• Verification: the process of determining

that an M&S accurately represents the developer's conceptual description and specifications.

• Validation: the process of determining the extent to which an M&S is an accurate representation of the real world from the perspective of the intended use of the M&S.

• Accreditation: the official determination that a model, simulation, or federation of M&S is acceptable for use for a specific purpose.

Behavior Validation is the process of ensuring that the correct things are modeled.

Behavior Verification determines if a behavior’s implementation performs as intended. For OneSAF, “performs as intended” means runtime performance matches the KAKE artifacts.

OneSAF V&V

OneSAF in BCLSE

• Conducted major, large scale, distributed event using OneSAF as primary simulation.

• Demonstrated reliability of OneSAF in HLA distributed environment.

• Demonstrated viability of OneSAF Brigade level clusters for experimentation.

• Demonstrated ability to net-time, checkpoint and resume major distributed event with OneSAF.

• Provided PM with lessons learned and PTRs for V3.0 release.

OneSAF in Strategic Choice StudyOneSAF in Strategic Choice Study

• TRAC, with the assistance of PM OneSAF, met the required timelines for the study.

• 2 months of significant model changes plus data and scenario revisions.

• TRAC successfully converted four scenarios (2 base case and 2 alternatives) to include new data.

• Processed over 1600 data files including over 200 entity compositions and over 400 unit compositions.

• TRAC was able to collect meaningful output data.

OneSAF in:Stryker Modernization study

OneSAF in:Stryker Modernization study

Purpose: Inform MBL visitors on Planning, Preparations and Execution efforts to conduct the Stryker Modernization Experiment using OneSAF as the main simulation driver

The desired outcome:Results that can be included in Stryker CDD documentation that support specific requirements and are able to withstand AROC and JROC scrutiny.

Weapon System ValidationSensor ValidationBehavior ValidationTerrain Validation

Behavior Artifacts

Domain Analysis Component Development

Capabilities Description Document

Domain Behavior Description

Behavior Data Tables

Implementation Design Description

Design

Source Code

Compositions

Verification Tests

Methodology

TEST EXECUTION AND ANALYSIS

TEST PREPARATION

METHODOLOGY PREPARATIONConduct

Background Research

Conduct User Training

Develop Methodology

Prioritize Behaviors

Select Behavior

Study Behavior Documentation

Develop Criteria

Design Tests

Execute Tests

AnalyzeData

Report Results

Receive Corrected Behavior

Verification Status

Behavior Complete

Pass

Fail

TraceabilityVerification

Courtesy of TRAC-Monterey

Questions ?

Ha LyHa.Ly@us.army.mil

407 384 5375