Stevens-USC SERC:A DoD University Affiliated Research Center (UARC)

Stevens-USC SERC:A DoD University Affiliated Research Center (UARC)

Barry Boehm, USCUSC-CSSE Annual Research Review

March 16, 2009

Outline

• Nature of UARCs• SERC Overview

– SERC organization and vision– SERC research strategy

• Initial tasks– SysE Effectiveness Measures Assessment (EM)– SysE Methods, Processes, Tools Evaluation (MPT)

• Workshop objectives and approach

3/16/2009 2

What is a UARC?What is a UARC?

1. University Affiliated Research Centers are “not-for-profit, private sector organizations affiliated with, or part of, universities or colleges that maintain essential research, development and engineering capabilities needed by sponsoring DoD components.”

2. They maintain long-term, strategic relationships with sponsoring DoD components in specific core areas and operate in the public interest, free from real or perceived conflicts of interest.

3. UARCs are financed through long-term, non-competitive contracts awarded by sponsoring DoD components for specific core work.

33/16/2009

Several Existing UARCsSeveral Existing UARCs

1. Johns Hopkins University APL; 4,300 people; annual funding $680M

2. UC Santa Cruz with NASA Ames – information technology, biotechnology, nanotechnology, computer science, aerospace operations, astrobiology, and fundamental biology. Has a Systems Teaching Institute with San Jose State University and UCSC to teach through hands on experience on research projects.

3. Penn State University Applied Research Laboratory for the Navy with focus on undersea missions and related areas; strategic partner with NAVSEA and ONR; established 1945; has >1000 faculty and staff

4. University of Washington APL – acoustic and oceanographic studies ; established in 1943

5. UC Santa Barbara Institute for Collaborative Biotechnology – Army Research Office; partnered with MIT and Cal Tech – focus on biologically-derived and biologically-inspired materials, sensors, and information processing …

6. University of Texas UARC started in 1945 focuses on sonar, acoustics, software system research, satellite geodesy, active sonar, …; now has 600 people on staff

7. USC Institute for Creative Technology – US Army; focus on virtual reality multimedia applications for training, C4ISR

43/16/2009

• Auburn University• Air Force Institute of Technology• Carnegie Mellon University• Fraunhofer Center at UMD• Massachusetts Institute of

Technology• Missouri University of Science and

Technology (S&T)• Naval Postgraduate School

Lead organizations

• Pennsylvania State University • Southern Methodist University• Texas A&M University • Texas Tech University• University of Alabama in Huntsville • University of California at San Diego • University of Maryland• University of Massachusetts• University of Virginia • Wayne State University

Members

As the DoD Systems Engineering Research-University Affiliated Research Center, SERC will be responsible for systems engineering research that supports the development, integration, testing and sustainability of complex defense systems, enterprises and services. Its members are located in 11 states, near many DoD facilities and all DAU campuses.3/16/2009 5

SERC Organization

Dr. Dinesh Verma Executive Director

Dr. Art PysterDeputy Executive Director

Stevens’ School of Systems and Enterprises will host the SERC at Stevens’ Hoboken, NJ, campus. Stevens’ faculty engagement will be complemented by a critical mass of systems engineering faculty at USC. A fundamental tenet of SERC is its virtual nature – each of its 18 members will be a nexus of research activities. All research projects will be staffed by the best available researchers and graduate students from across the members.

Pool of more than 140 Senior Researchers and hundreds of research faculty and graduate students from across membersPool of more than 140 Senior Researchers and hundreds of

research faculty and graduate students from across members

Julie Norris (acting)Director of Operations

Dr. Barry BoehmDirector of Research

3/16/2009 6

SERC Organization - II

Rough Financial ModelRough Financial Model

• Minimum of $2M/year– Can add more funded tasks within contract

• First year: two specified tasks– 1. SysE Effectiveness Measurement: EM (USC lead; $500K)– 2. SysE Methods, Processes, Tools Assessment : MPT (Stevens lead; $350K)

• Further tasks proposals being submitted– In response to SERC Research Strategy Thrusts

• Other Govt. sponsors can sole-source through UARC– Procedures being worked out– Industry, FFRDC collaboration modes being worked out

73/16/2009

SERC Vision and PerspectiveSERC Vision and Perspective

DoD and IC systems achieving mission outcomes – enabled by research leading to transformational SE methods, processes, and tools.

8

VisionVision

PerspectivePerspectiveThe SERC will be the primary engine for defense and intelligence community SE basic research. In doing so, the SERC will:

1.Transform SE practice throughout the DoD and IC communities by creating innovative methods, processes, and tools that address critical challenges to meeting mission outcomes (what),

2.Become the catalyst for community growth among SE researchers by enabling collaboration among many SE research organizations (who),

3.Accelerate SE competency development through rapid transfer of its research to educators and practitioners (how).

3/16/2009

SERC Research Strategy ThrustsSERC Research Strategy Thrusts

1. Enterprise Responsiveness

2. Systems Science and Complexity

3. Human Capital

4. Program and Systems Engineering Integration

5. Life Cycle Systems Engineering Processes

3/16/2009 9

1. Enterprise Responsiveness Thrust1. Enterprise Responsiveness Thrust

– Collaboration. Research innovative collaborative technologies to dramatically improve systems engineering is performed, especially for teams that are geographically, culturally, and linguistically diverse.

– Modeling and Simulation. Research how to more rapidly and easily develop modeling and simulation tools that help validate concepts of operation, architectures, and other key systems engineering artifacts.

– Resilient Program. Research how to architect and implement a program that is resilient in the face of a wide range of potentially harmful changes, such as budget cuts, requirements turbulence, failure of needed technology maturation, and program redirection

3/16/2009 10

1. Enterprise Responsiveness Thrust - II1. Enterprise Responsiveness Thrust - II

– Resilient System. Research how to architect and implement a system that is resilient in the face of a wide range of threats, both cyber and physical.

– Producibility. Research how to dramatically decrease the amount of effort and schedule required for implementing new software-intensive systems.

– Parsimony. Research how to reduce the amount and size of SE artifacts, e.g., determining minimum-essential architectural views number, detail, rigor, and capture time.

– Strategic Assessment. Research improved process models and decision criteria for better SE support of strategic and enterprise decisions.

3/16/2009 11

2. Systems Science and Complexity2. Systems Science and Complexity

– Composition. Research how to determine key properties of a system (assurance, scalability, availability, producibility, interoperability, lethality, resilience, …) when (a) the properties of its subsystems are known, (b) the system architecture that links those subsystems is known, and (c) the concept of operations is known.

– Emergence. Research how to manage emergence in requirements, technology, and system usage in a way that is beneficial or benign rather than disruptive to development and acquisition programs; and how to anticipate and mitigate emergent behaviors.

– System Conceptualization. Research approaches to better visualize and define system concepts that enable collaboration among multiple stakeholders.

3/16/2009 12

2. Systems Science and Complexity - II2. Systems Science and Complexity - II

– Holistic SE. Investigate systems thinking methodology and its application to DoD enterprise and capabilities planning.

– Validation. Research how to improve the early and continual validation of complex, expensive systems.

– Transformation. Take systems engineering and systems thinking MPTs that are applied to critical contemporary issues outside the DoD domain (such as creating a more cost effective health care system for the U.S. general population) and transform them so as to be directly applicable to problems in the DoD domain.

3/16/2009 13

3. Human Capital3. Human Capital

– Collaboration and Education. Extends Enterprise Collaboration thrust element to include education.

– Acceleration. Research how to accelerate the growth of the systems engineering workforce for both DoD and its contractors.

– Dispersion. Research how to rapidly and effectively grow SE competencies in project managers, contracting officers, and other acquisition personnel besides SEs for whom SE competency could enhance their performance.

– Staffing. Research how to staff a program with the right mix of systems engineers as a function of program type, size, complexity, risk profile, and other system and system acquisition characteristics.

3/16/2009 14

4. Program and Systems Engineering Integration4. Program and Systems Engineering Integration

– SE Effectiveness. Research SE program execution performance, return on investment, and metrics to demonstrate and improve value of SE.

– Value. Determine how much SE is appropriate in specific circumstances to help ensure that the right amount of resources is available with the right skills.

– Economics. Research how to better estimate systems engineering cost and schedule for complex programs and complex systems.

– Assessment. Research improved process models and decision criteria for better SE support of management decisions across all life cycle phases (currently funded, in part, as the EM TTO)

3/16/2009 15

4. Program and Systems Engineering Integration - II4. Program and Systems Engineering Integration - II

– Maturity and Readiness. Research how to quantitatively assess the maturity of SE artifacts. Be able to inform technical, management, governance, and investment decisions using that assessment.

– Teambuilding. Research how to better stimulate and support effective distributed team performance among SEs and other stakeholders.

– System of Systems, Networks, and Enterprises. Research how to manage the acquisition and development of a SoS, a network-centric system, or an enterprise system where there is typically no single “owner” for all the component systems, components emerge and disappear dynamically, and components often have conflicting acquisition strategies.

– Services. Research how to effectively manage the acquisition and development of a service as a function of type, scope and complexity, uncertainty, risk, and other characteristics.

3/16/2009 16

5. Life Cycle Systems Engineering Processes5. Life Cycle Systems Engineering Processes

– Life cycle models. Research the best life cycle models for the acquisition and development of a SoS, a network-centric system, a service, or an enterprise system as in Thrust 4.

– Balance. Research life cycle models for the acquisition and development of a system which begins as a quick response built using agile methods and then evolves into a system that is deployed on a larger-scale with radically different maintainability, availability, and similar attributes.

3/16/2009 17

5. Life Cycle Systems Engineering Processes - II5. Life Cycle Systems Engineering Processes - II

– Architecting. Research how to create architectures that demonstrably and quantifiably support key system properties such as assurance, scalability, availability, producibility, interoperability, lethality, and resilience, and how to derive architectural component characteristics and constraints from desired system properties.

– Landscape. Create a landscape of SE MPTs with respect to key characteristics such as their maturity, the types of systems to which they are best applied (e.g., network-centric vs. embedded or enterprise vs. platform), and their value in actual application. Use insights from this landscape to help guide where SE MPT research is needed. (currently funded, in part, as the MPT TTO)

3/16/2009 18

FY09 Focus

SERC Research Initiation StrategySERC Research Initiation Strategy

Determine SEEffectiveness

And ValueMeasures

Research to Address Specific

SE issues

DetermineModified MPTs

To address gaps

ValidateMPT

Effectiveness

Research to Address Specific

SE issues

DetermineModified MPTs

To address gaps

ValidateMPT

Effectiveness

Research to Address Specific

SE issues

DetermineModified MPTs

To address gaps

ValidateMPT

Effectiveness

Research to Address Specific

SE issues

DetermineModified MPTs

To address gaps

ValidateMPT

Effectiveness

Baseline SEMPTs

Early focus on a solid baseline and quantifiable, observable Measures to enable future demonstration of improvement

SE externalFactors/Context

MissionDrivers

3/16/2009 19

Outline

• Nature of UARCs• SERC Overview

– SERC organization and vision– SERC research strategy

• Initial tasks– SysE Effectiveness Measures Assessment (EM)– SysE Methods, Processes, Tools Evaluation (MPT)

• Workshop objectives and approach

3/16/2009 20

First Two Task Orders

1. “Assessing Systems Engineering Effectiveness in Major Defense Acquisition Programs (MDAPs)”

– Government lead: OUSD(AT&L)/SSE

– Barry Boehm (USC) task lead, with support from Stevens, Fraunhofer Center, University of Alabama at Huntsville

2. “Evaluation of Systems Engineering Methods, Processes, and Tools (MPT) on Department of Defense and Intelligence Community Programs”

– Government lead: DoD

– Mike Pennotti and Rich Turner (Stevens) task leads, with support from USC, University of Alabama at Huntsville, USC, Fraunhofer

3/16/2009 21

Coordinated approachCoordinated approach

• Synergetic contribution to Sponsor’s SysE effectiveness goals

– Best practices + progress monitoring and improvement

• Common context– Domains of interest and levels of

organization

• Coordinated management– Common collaborators, battle rhythm with

regular e-meetings, shared workshops– Research Integrity Team

• Coordinated Technical Approaches– Definitions, evaluation criteria and

methods– Cross-feed/peer review ongoing results

223/16/2009 22

Workshop Objectives and ApproachWorkshop Objectives and Approach• Review, improve on EM and MPT task objectives

– Utility of results to DoD constituencies– Identification of needs-capabilities gaps– Identification of promising research directions

• Review, improve on EM and MPT task approaches– Survey and evaluation approaches, criteria, and instruments– Involvement of key stakeholders

• Contractors, program/portfolio managers, oversight organizations– Coverage of DoD application domains

• Initial EM priority: Weapons platforms• Initial MPT priority: Net-centric services

– Test and evaluation of results• Capture participants’ relevant experience

3/16/2009 23

SERC Research Methodology

3/16/2009 24

Understand and synthesize advances in multi-media technologies and interactive graphics; gaming technologies; real options theory

Example: Mission and StakeholderRequirements and Expectations

Can we develop a transformative, interactive, and graphical environment to bring stakeholders (warfighters and analysts) together with SEs to develop a graphical/visual conops in an extremely agile manner?

Every study on failed projects, refers to inadequate requirements, and understanding of the “real” problem: large projects and small projects; defense or commercial

3/16/2009 25