Nancy MarkusonJoleen Flasher

Adapting Agile Processes for Military Acquisition Programs

1 April 2014

©2014 The MITRE Corporation. All Rights Reserved.Approved for Public Release; Distribution Unlimited Case number 14-0439

| 2 |

Introduction

The current military acquisition process is very complex – Many gates to go through between problem definition and fielding– Lots of documentation required– Can takes years to procure systems

The pace of technology has increased such that warfighters are often left with insufficient and obsolete systems This briefing describes the Agile processes being utilized in the

Air Force Weather programs to ensure frequent releases addressing top priorities– Specific Agile processes and transition paths– Adaptations of Agile for military acquisition– Contractor and stakeholder interactions– Benefits & next steps

| 3 |

Department of Defense Acquisition ProcessOverview

| 4 |

Department of Defense Acquisition ProcessEngineering Development Phase

| 5 |

Agile and the Acquisition World

Agile Manifesto– Individuals and interactions over processes and tools– Working software over comprehensive documentation– Customer collaboration over contract negotiations– Responding to change over following a plan

DoD 5000 doctrine does not require a waterfall process Adapting Agile to work within the confines of the acquisition process can be

an uphill battle

Agile Scrum Process

| 6 |

Air Force Weather Family of Systems (AFWFS) Enterprise

AFWA

Portable Doppler Radar

Services - JMBL Data Files

Collect

Process

Disseminate

Model DevelopmentAnalyze Predict

Weather Data Analysis (WDA)

Store

Strategic/ National Customers

Weather GroupSCA

OWSJoint Environmental Toolkit(JET)

Services - JMBL

Disseminate

Web Products

Product Tailoring

Exploitation

Collect

Forecaster

Sensor Data Weather Products via M2M

ForecasterUser/Forecaster

Operational Weather Squadrons• 6 Worldwide• Theater Support

Air Force Weather Agency

Strategic Support

Weather Groups• 200+ Air Force & Army Sites

Tactical Support

Air Ops

Ground Ops

Subscribe

Deployed Sensors

Mission Planning Weather (MPWx)

Planning and Control Centers

| 7 |

Weather Data Analysis (WDA)

Ingest, stores and disseminates weather products Provides subscription services to internal and external users Issues

– Requirements thrash leading to delayed fielding– Poor customer satisfaction and expensive rework

| 8 |

Joint Environmental Toolkit (JET)

Ingest sensor data for:– Local ATC at Weather Groups– Forecasters back at the OWS

Develop weather products and make them available – WWA and mission impacts– Multiple dissemination

Issues:– Fielded retiring or obsolete capabilities due to slow development

schedules– Low user satisfaction with system performance and process flow

| 9 |

MPWx is a software tool for importing and displaying weather data overlays onto map backgrounds – Decision aid for mission planners

Mission Planning Weather (MPWx)

The small size, and the ease of user feedback from fielded mission planning systems made it a clear choice for moving to Agile in mid-2011.

Tasked Based Contract MPWx was used as a proof-of-

concept, for the larger AF Wx programs

IDEAL Pathfinder Program

| 10 |

Meeting DoD Acquisition Requirements with AgileMilitary Acquisition

Agile Scrum

Acquisition areas addressed to obtain buy-in from leadership

Modified agile scrum process was adopted– Ensured that military acquisition

mandates in all areas were met

• Requirements traceability• Transparency• Design Reviews• Contract wording

• Documentation• Metrics• Risk reduction

| 11 |

Requirements Traceability

As Major Air Force Systems (identified as ACAT III programs), WDA and JET are mandated to have requirements traceabilityBoth programs have operational requirements defined

in a Capabilities Description Document (CDD) and systems level requirements defined in a Technical Requirements Document (TRD)– They are contractually mandated to fulfill the

requirements or work a contract modificationRequirements are written at a high level to allow for a

large solution spaceRequirement prioritization and refinement is the

responsibility of the entire community The Agile process forces the conversation on requirements and functionality to be continual

| 12 |

Transparency via Tools

The Weather Programs use several tools to streamline the management of the development The JIRA and Confluence online tools allow the

stakeholders to – Break down a project into epics and stories – Track the completion of tasks and hours throughout sprints– Track team progress with metrics and visualization tools The DOORS tool is used to map requirements

– Can track the many-to-many relationships between stories and requirements

– Can look at the ripple effect of requirements changes and/or test failures

Key point – The development process is transparent because the tools are available to the stakeholders

| 13 |

One-To-Many Relationship is Tracked in DOORS

Capability

Descripton

Document

Requirements User Story Link Module Acceptance Criteria

DOORS JIRA

1011 MPW shall..1236 MPW shall..3546 MPW shall..5299 MPW shall..6364 MPW shall..5122 MPW shall..9144MPW shall..

……..

……..

User Story 1User Story 2User Story 3 User Story 4User Story 5User Story 6

……..

……..

LinkAttribute

Pass

LinkAttribute

Pass

LinkAttribute

Fail

Criteria 1Criteria 2Criteria 3Criteria 4Criteria 5

Stories

The combination of DOORS and JIRA allows us to easily know when a requirement has been satisfied

| 14 |

Preliminary and Critical Design Reviews

What is the underlying purpose of the Design Reviews?– Educate the stakeholders as to the proposed and implemented

solutions– Gain buy in that the requirements have been satisfied– Identify weaknesses while there is still time to implement changes

Agile solution to these issues– Stakeholders are involved in design process – Stakeholders are able to evaluated a working version of the

solution at the end of each Sprint. – Weaknesses and change requests can be addressed very quickly

through product backlogs Evaluate the actual needs of the program before defining the

review goals and expectations

JET & WDA replaced the large reviews with abbreviated “sanity checks” that verify release deliverables

| 15 |

Contractual Artifacts

Contract Wording– Direction for Agile development in contractual documents varies greatly, from

simply directing the contractor to use Agile, to defining sprint length, tools, govt. interactions, etc.

– We are working on developing templates for programs to start with Documentation = Contract Data Requirements List (CDRL)

– Identify high value documents (Design and Requirements Decomposition) vs. the “check the box” or internal items

– Stakeholders were involved in nominating documents for discontinuation– Items may be dictated by contract type but the format is not. Consider: Must it be a document delivered to the Government or is access to a shared space

sufficient? Can you auto-generate the document with the toolset being used? For each programmatic phase, what level of completion is required?

You may lose the battle to remove CRDLs initially but it should be revisited at each major contract action

| 16 |

Performance Metrics

Current indicators of progress (daily burndown) are more helpful than lagging indicators (post sprint processed information)– DoD Acquisitions prefer Earned Value Management metrics

delivered on a monthly basis at best– Some programs only see metrics during major reviews

With Wx Programs, we take advantage of unexpected burndownsby adding more functionality

| 17 |

Strategic Metrics

Strategic decisions regarding implementation are made by using longer-term velocity metrics which track progress monthly

Sprint Velocity

Commitment Completed

Sprint 1(~ Month 1)

Sprint 2(~ Month 2)

Sprint 3(~ Month 3)

Sprint 4(~ Month 4)

| 18 |

Process Implementation

Monthly sprints, releases every 6 months Active stakeholder community - Contractor, Program

Management/Engineering, Product owner, Testers

Sprint 1(Month 1)

Sprint 2(Month 2)

Sprint 3 ……..(Month 3)

Sprint 6(Hardening)

Requirements Approval

Release Planning

Development

Backlog Grooming

Demo Day/Retrospective

Sprint Planning

Informal Testing(28th/46th)

Contractor/Govt testing

Operational Testing (28th)

Fielding

Pre-DevRelease

Finalization

Group activities Contractor activities Test activities

| 19 |

Benefits Seen

JET Results– 65% reduction in DRs

from Build B– 80% reduction in DRs

from Build A– Retest activity canceled

at the request of the user WDA Results

– Shortened retest MPWx

– Continued user satisfaction and increased user base

Ris

k

TimeDesignReqts. Implementation Testing

Large risks are retired earlier in the process using Agile

| 20 |

Is Agile Right for Your Program?

Take the jump (maybe)– Evaluate your program. Does it fit? What are the goals of moving to Agile?

Will you have the support needed from everyone?– Agile for Agile’s sake will not lead to success

Ask questions– Find out what is truly required (documentation, reviews, etc.) and what has

just “always been done before”– Can a contractual requirement be fulfilled in a non-traditional way?

Get the right people to the table– Is there someone you need to convince? What are their concerns?– Who all is impacted (users, test community, Information Assurance,

contractors) Start off on the right foot

– Training so that Agile means the same thing to everyone involved– Clearly identify roles & responsibilities– Identify process issues as they arise and address them quickly.

Be FLEXIBLE!

| 21 |

Way Forward

Acquisition doctrine is still “streamlined” to the waterfall development process– MITRE is constantly looking for ways to make Agile work within the

acquisitions framework – MITRE is also developing recommendations to amend that framework

in order to enable more diverse development processes

Roundtable sessions to expand knowledge base throughout the community– Programs that have moved to Agile If they failed, why? Success stories and lessons learned.

– Programs looking to move to Agile What challenges are they facing?

Continue to expand resources for other programs– Contracting language examples and recommendations– Cost analysis of Agile vs. traditional development

| 22 |

Summary

The Air Force Weather Agency is successfully executing military acquisition programs using Agile – Small programs that tend to be more services-based– Medium sized programs that are subject to high level govt. regulations

We are working to share our experiences and lessons learned with other programs– We have developed a number of papers to address the perceived

contradictions between DoD Acquisitions and Agile.– We are currently collecting examples of contract language to help

programs as they move to agile If you have examples, we want to talk to you! If you need examples, we can help

Please contact us:– Nancy Markuson (markuson@mitre.org)– Joleen Flasher (jflasher@mitre.org)

| 23 |

Questions?

| 24 |

Acronyms

AFW-WEBS - Air Force Weather Web Services

AFWx – Air Force Weather ATC – Air Traffic Controller CDD: Capability Development

Document. CDR – Critical Design Review CDRL: Contract Data Requirements List. DoD – Department of Defense DR – Deficiency Report EVM – Earned Value Management IT - Information Technology JCIDS - Joint Capabilities Integration and

Development System JET – Joint Environmental Toolkit

JMBL: Joint METOC Broker Language M2M – Machine to Machine MPWx – Mission Planning Weather OWS – Operational Weather Squadron PDR - Preliminary Design Review PPBE – Planning, Programming, Budget,

& Execution SCA – Sensor Collection Appliance SUM – Software User Manual SVD – Software Version Description TRD – Technical Requirements

Document WDA – Weather Data Analysis WF – Weather Flight WWA – Watch, Warnings and Advisories