Governance in Ultra-Large-Scale Systems

Governance in Ultra-Large-Scale (ULS) Systems

Philip Boxer BSc MBA PhD

1 Copyright © Boxer Research Ltd 2015

Contents

• The Policy, Acquisition and Management Research Agenda – What kinds of challenges are these?

• On Orchestration and Synchronisation – What insight can be derived from this experience?

• Practical Consequences – What is it possible to do for clients?

• In Conclusion – This leaves a lot of unresolved issues…


THE POLICY, ACQUISITION AND MANAGEMENT RESEARCH AREA

What kinds of Challenges are these?


The Governance framework is crucial

– Given the scope and scale of ULS systems, technical, organizational, and operational policies will emerge as principal vehicles for ensuring harmonious operations at all levels.

– The size and highly distributed nature of ULS systems will limit global visibility and decentralize system management within an overall framework of organizational, technical, and operational policies.

– Research is required in how to define ULS system policies that specify organizational, technical, and operational constraints for global system integrity and freedoms for flexible adaptation.

– Much of the evolution of ULS systems will occur in situ, thereby imposing requirements to maintain critical operational capabilities while adding or improving other capabilities in place

− What kind of Governance is appropriate to the distributed collaborations across ULS systems (sometimes described as having virtual* characteristics)?

* For example, “Virtual systems of systems lack a central management authority and a centrally agreed upon purpose. Large-scale behavior emerges and may be desirable, but this type of SoS must rely upon relatively invisible mechanisms to maintain it.” Systems Engineering for Systems of Systems, August 2008, Office of the Under Secretary of Defense (Acquisition, Technology and Logistics)


The supporting systems and infrastructures have to be understood differently

– Research is needed to understand how supply-chain organizations could be integrated as first-class operational components into ULS ecosystems…

– In such an environment, supply-chain organizations ranging from established vendors to open-source collaborations could undergo periodic assessment of capabilities, participate in joint training and readiness exercises with the forces, and come and go as needs and capabilities change.

– The supply chains of vendors and integrators that will populate ULS ecosystems must be organized, and incentives must be provided to evolve ULS system capabilities at a rapid pace in response to changing operational needs.

* For example in the need for Through-Life Capability Management (TLCM): “a greater proportion of our overall business is available to industry than in any other major defence nation, and growing expertise in the combination of systems engineering skills, agility and supply chain management required to deliver TLCM gives the UK defence industry a comparative advantage.” UK Defence Industrial Strategy December 2005 p6.

− How are the changing services provided by the operational components of ULS systems to be enabled to achieve levels of cooperation and collaboration that can satisfy requirements for fast system evolution*?


Demand issues are of a different kind

– The pervasive application of ULS systems to support global operations in many simultaneous strategic and tactical situations will generate many requirements for rapid evolution to meet changing threats and environments.

– ULS systems will experience and should create incentives for substantial local adaptation and bottom-up evolution

– ULS systems will be designed to support dynamic coalitions and management of tactical and strategic operations through linkage of field units with command-and-control functions on any scale necessary.

– The overarching requirement for ULS systems is operational readiness at all times under all conditions… ULS systems will be required to adapt to changing missions and unanticipated circumstances encountered by warfighters.

* An ‘edge’ is any particular situation/context where a threat-demand is encountered that the ULS system must respond to. An early formulation of this was in Power to the Edge: command and control in the information age, Alberts & Hayes, DoD CCRP June 2004.

− How are we to enable the distributed collaborations across ULS systems to be edge-driven*?


Collaborative processes are critical

– Policies must support both local and global operations in such a way that people and the computational actions they initiate can achieve cooperative and even competing objectives without impairing the viability of the system as a whole.

– It may often be the case that, to meet immediate needs, local users of ULS systems will be forced to engage in ad hoc acquisition of components whose functionality and quality properties are not well understood or trusted. Because these components address unforeseen problems, an opportunity will arise to improve and generalize their application across similar environments.

– ULS systems engineering development and operational use will generate knowledge that can be preserved and analyzed to guide future evolution.

* “Orchestration is the set of activities needed to make the elements of a ULS system work in reasonable harmony to ensure continuous satisfaction of the mission objectives”. p25 ULS Systems

− How are the processes of orchestration and synchronisation to work in ULS systems?*


Taking the human factor into consideration

– People are key participants in ULS systems. Many problems in complex systems today stem from failures at the individual and organizational level. We therefore need research on user-centered specifications and on modeling users and user communities.

– While some models of human interaction are inspired primarily by economic factors and competitive forces to drive improvements, research is needed to understand other models, such as open source, that involve fostering non-competitive social collaboration.

– We cannot fully anticipate the context within which ULS systems will operate and necessarily evolve, as the socio-cultural practices of many different groups (stakeholders, users) will, in fact, be constructing this real-world computational environment. The challenge is thus to design and support systems using an accurate model (scientific understanding) of this ULS/social-context interaction.

− How are people to be taken into consideration as first-class components of ULS systems?*

Collaborative processes are

critical

Demand issues are of a different kind

The supporting systems and infrastructures have to be

understood differently

The Governance framework is crucial

N-S are limiting E-W are enabling

8

* Boxer, P. J. (2014). "Leading Organisations Without Boundaries: 'Quantum' Organisation and the Work of Making Meaning." Organizational and Social Dynamics 14(1): 130-153.

Copyright © Boxer Research Ltd 2015

Client Situations generating learning about these challenges

Client Instances The problem motivating their interest

• Thales/Bosch

• Unable to take advantage of and manage the opportunities emerging among multiple related customers with differing interests.

• UK NHS/BT • Unaddressed client needs because of limitations to existing business models

• Create/JFSP • Difficulty in managing complexity of modeling frameworks within acceptable timeframes

• NATO/MilSatcom • Systematic identification and structuring of risks to deployment and sustainment of operational capabilities

• MOD NiteWorks/Army Software Blocking

• Unaffordable cost trends plus lack of ability to support rates of operational change, and failure to meet time deadlines supporting deployment

• Raytheon/MoD • The customer changing from an equipment-based to a capability-based approach to acquisition and its effects on how the supplier’s market is defined


ON ORCHESTRATION AND SYNCHRONISATION

What insights can be derived from this experience?


The Supplier is supporting the Client Enterprise managing three diverging tempos…

Customers of the

Client Enterprise Supplier Socio-technical

Client Enterprise

Socio-technical

Ecosystem

Demand

Tempo

The rate at which new forms of demand need to

be satisfied

Supplier 1

Supplier 2

sub-contract

sub-contract

Acquisition

Tempo

The rate at which new requirements can be

met

Client

Enterprise users

users

Readiness

Tempo

orc

hes

trat

ion

synchronization

The rate at which the client enterprise is able to support new

forms of value proposition

Demanded effects in the customer’s context-of-use

The client enterprise aligns to the demand of the customer

The supplier responds to the client enterprise aligning to the demands

of its customers

Projective analysis


Opportunistic (marginal/ incremental costs)

Niche-based (focus where

Positional advantage can be sustained)

Client’s Economies of Alignment

No Yes

Supplier’s Economies of Scale or Scope

Yes

No

Effects-based (focus where

Relational advantage can be sustained)

The challenge for the business is to be able to include effects-based forms of competition

Digitalisation

The processes of digitalisation change the economics of alignment

Over time, innovations in response to demand are imitated and become generally available from suppliers

The Supplier has to support the Client’s effects-basing, adding the need to generate Economies of Alignment for the Client


Defence Enterprise

Acquisition Tempo

Readiness Tempo

Adapted from: “Appropriate Collaboration and Appropriate Competition in C4ISTAR Transformation”, Dr Nicholas Whittall RUSI 2007

Campaign Tempo

Gap = Need Acquisition

Effect

Threat-Demands

Composite Capability

Capability

Capability

Capability

Orchestration

Doctrine Organization

Training Materiel

Leadership Personnel Facilities

Suppliers

Capability

Requirement

Traditional Defense Companies await Requirements expressed

in Programmes.

Competitive advantage to be gained in aligning the Need to the Demand.

Competitive advantage to be gained from dynamic alignment of Composite Capabilities to the Threat-Demands.

Divergence of tempos increases costs of

alignment

Divergence of tempos increases Costs of

Alignment associated with Readiness

The divergence of tempos creates an agility challenge


With effects-basing, Value comes from managing a Double ‘V’

Requirement Solution

Components

Design decomposition

Design integration

The cycle creating Value

Orchestrations

Military Effects

Composite Capabilities

Joint Command

Gaps in Force Command Structure and Composite Capabilities

Scenarios and Campaign Plans

Capability = Solution plus

DOTMLPF

Requirement = Capability gap minus

DOTMLPF

demand-side

supply-side

Boxer, P.J. (2007) Managing the SoS Value Cycle, January 2007, http://www.asymmetricdesign.com/archives/85

Synchronisations


The unanticipated need for agility can be very expensive

Level of short-term savings – the cost of the option to satisfy a significant

part of the UAV III need

The cumulative costs of sourcing UAVs I & II UAV I – over-the-

horizon targeting (Phoenix) UAV II – CCIR ISTAR asset (Watchkeeper)

Level of additional expenditure incurred

through Urgent Operational

Requirements to deliver UAV III

UAV III – extending ‘edge’ capability (+Nimrod, Lydian & Desert Hawk)

How should this option have been valued?

15

Without an ability to analyze cohesion

it is difficult to establish the costs of alignment and

identifying options for increasing agility


a’b’

2. Change in variance in levels of expenditure, based on the difference between the two curves ‘a’ and ‘b’

Probability

Levels of expenditure meeting Customer Demands

The cost of Force Structure ‘b’ across the

variety of demands

b

‘Real Option’ pricing allows a value to be assigned to these changes in variance

a

The cost of Force Structure ‘a’ across the

variety of demands

1. Reduction in average level of expenditure through impact of capability trade.

The value of an incremental investment in some new capability is the impact of both the trade and the change in agility*

* Agility = property of the Force Structure enabling it do more things with the same underlying capability set.

The Value of Agility for the Client Enterprise is to reduce the variance in expenditures

Two kinds of benefit:


Value comes through creating Real Options that increase the agility of force structures

2: Real Option Valuations

(agreed perceptions of likelihood of future scenario mix =>

distribution of demand across options for composite capabilities)

4: Engineering for Flexibility

(maximise mitigation of interoperability risks)

1: Varieties of Geometries of use x

Decisive Points (across all possible scenarios. Assumes technical feasibility)

What is valued politically =>

where to develop agility

Determines the underlying context in which new demands

have to be met

Determines forms of cohesion demanded

Requirement for flexibility in capabilities

3: Type III Agility (defining new capabilities for distributed collaboration and

requisite granularity of supporting capabilities)


PRACTICAL CONSEQUENCES

What is it possible to do for Clients?


Ultra-Large-Scale Socio-Technical Ecosystems

Monolithic Systems

Analysis of Requisite Variety – establishing the variety of

geometries needed

In practice we are always operating between the ultra-large-scale and the monolithic

Invariances resulting in Quality Attributes

Emergent (orchestrations)

Static (architectures)

Effects anticipated on Demand Situation

Unanticipatable (Effects

Ladders)

Planned (Mission Threads)

The client enterprise is operating somewhere between ULS socio-technical

ecosystems in one direction, and monolithic systems in the other

19

QAW/ATAM – establishing the

quality attributes of the architecture

Analysis of stratification and

granularity, identifying

requirements of lower strata

SoS Mission Threads Analysis


Analysis of socio-technical systems needs to include relationships with three new (types of) view…

Constrains what is possible

Shapes granularity and stratification

Example Analyses Functional Architecture Description

Data Architecture Description

Accountability Hierarchies Description

Social Synchronization/

Data Fusion Description

Description of Heterogeneity of

Demand Organization

Stand-alone Systems () - - -

Stand-alone Software () - - - Complex SoS/EA

() () - -

Case Examples

JFSP II: Framework () -

NATO AWACS: SoS ()

Thales: C4ISTAR

Projective Analysis Views

Structure-function

Trace Hierarchy Synchronization Demand

Example Analyses Functional Architecture Description

Data Architecture Description

Stand-alone Systems ()

Stand-alone Software () Complex SoS/EA

()

Case Examples

JFSP II: Framework

NATO AWACS: SoS

Thales: C4ISTAR

Projective Analysis Views

Structure-function

Trace


Stratification and granularity enabling the supply-side to be aligned to the demand-side

4: Organisational Interoperability (shared understanding of organizational processes)

3: Semantic Interoperability (shared understanding of behavioral meaning)

2: Syntactic Interoperability (communication is possible)

1: Machine Level Interoperability (stuff works)

5: Situational Interoperability (the way a situation is engaged with)

6: Effects Environment (the contexts-of-use in which effects are being created)

The layers read into a theatre-of-operations

context

decisive points

mission command

(agile) force structure

force elements

fielded capability

equipment capability

Engineering constraints

‘supply-side’

pragmatic constraints

‘demand-side’

The size of this overlap depends on the

engineering constraints being under-determining


The Governance of Alignment can be approached across eight dimensions of agility

Driven from ‘center’

Driven from ‘the edge’

Doctrine & operational concepts The principles and operational methods underlying the approach to generating effects

Facilities & infrastructure The facilities and infrastructure that are the context within which the enterprise does its work

Leadership & education The ability to lead the enterprise creatively and effectively within the context of the its chosen domain of action

Materiel & technology The tools and technologies that the enterprise needs to be effective within its chosen domain of action

Edge organisation The particular orchestration and synchronisation of capabilities needed to meet a demand

Force composition & collective learning The people needed with the appropriate know-how and ability to work together collaboratively

Situational understanding The way data is able to be fused to provide a composite picture of what is going on in the particualr situation

Personnel & culture The people with the socialisation, background and mutual knowledge and trust to be able to work together


Four diagnostics shape how the need of the Client Enterprise can be engaged with

Alignment of Organization & Infrastructure to Demand

2 How does the client enterprise align itself to its customers’ demands?

White:

how we must

do what we do

Blue:

what we doInternal

External

Internal

ExternalRed:

particular demands

Black:

the contexts

from which the

demands

emerge

The way

things

work

What

determines

shape

The way

things

work

What

determines

shape

Establishing the Key Stakeholders

1 Who are the key stakeholders in the performance of the client enterprise?

Influence Maps and beyond

Internal Alignment of DOTMLPFS

4 How are the internal

processes of the client enterprise aligned?

Alignment of Governance

Processes

Alignment of Service Layers to Demand

3

What strategy and economics are driving how the client enterprise relates to its customers’ demands?

Alignment of Economics

SoS Mission Threads/ Quality Attributes and

interoperability risks

pragmatic constraints

Engineering constraints

1: Machine Level Interoperability (lexis)

2: Syntactic Interoperability (command syntax)

3: Interoperability of Component Behaviors (shared understanding of behavioral semantics)

4: Organizational Interoperability (shared understanding through organizational semantics)

5: Situational Interoperability (pragmatics of the way a situation is engaged with)

6: Effects Environment (the context-of-use in which effects are created)

Alignment of Architectures


This leads to a different kind of analysis of interoperability risks…

Source: Anderson, Boxer & Browsword (2006) An Examination of a Structural Modeling Risk Probe Technique, Special Report, Software Engineering Institute, Carnegie Mellon University, CMU/SEI-2006-SR-017, October 2006. http://www.sei.cmu.edu/publications/documents/06.reports/06sr017.html

Special permission to use PAN in this Technical Probe was granted by Boxer Research Limited.

Identifying Interoperability Gaps in the different strata

Analysis of Granularity

Socio-technical SoS in relation to Demand

Functional/

Data Coupling

Demand

cohesion

Accountability

Hierarchies

Distinguishing three different kinds of pattern

1services

know-how

7 drivers

7b

problem domains

6demand situations

mission situations

5b

5

composition of

orchestrated constituent capabilities

orchestrations of constituent capabilities 4b

constituent capabilities

4

2b

3b

2outcomes 3

1c

super-structure

1b

direct organisation

0processes

events

6bdata fusion platforms

Analyzing alignment of strata to demand


http://www.sei.cmu.edu/publications/documents/06.reports/06sr017.html

Modeling socio-technical systems

Analyzing alignment of strata to demand

5-6

col1

x 5

un

ito

rder

\bo

rder

_isr

_cel

l

un

ito

rder

\bo

rder

_re

aper

_str

ike_

cell

un

ito

rder

\bo

rder

_sf_

cell

un

ito

rder

\afg

han

_bo

rder

_str

ike

un

ito

rder

\bo

rder

_cao

c_at

c_sy

nc

un

ito

rder

\bo

rder

_hal

e_b

m

un

ito

rder

\bo

rder

_mal

e_b

m

trac

even

t\b

ord

er_m

ale_

ou

tpu

ts

trac

even

t\b

ord

er_h

ale_

on

_sta

tio

n

trac

even

t\b

ord

er_m

ale_

on

_sta

tio

n

trac

even

t\b

ord

er_m

ale_

stri

ke

trac

even

t\af

ghan

_rep

ort

trac

even

t\b

ord

er_s

f_o

n_s

tati

on

trac

even

t\in

div

idu

al_i

n_a

fgh

an-p

akis

tan

_bo

rder

chan

nel

\bo

rder

_hal

e_b

m

chan

nel

\bo

rder

_mal

e_b

m

chan

nel

\bo

rder

_isr

_cel

l

chan

nel

\bo

rder

_re

aper

_str

ike_

cell

chan

nel

\bo

rder

_sf_

cell

c_si

tn\i

nd

ivid

ual

_in

_afg

han

-pak

ista

n_b

ord

er

orchn\afghan_border_strike 1 1 1 1 1 1 1 1 1 1 1

outcome\border_hale_on_station 1 1 1 1 1 1

outcome\border_male_on_station 1 1 1 1 1 1

outcome\border_male_strike 1 1 1 1 1 1

outcome\border_sf_on_station 1 1 1 1

khow\border_sf 1 1 1 1 1 1

khow\border_male_strike 1 1 1 1 1 1 1

khow\border_hale_global_hawk 1 1 1

design\border_hale_global_hawk 1 1

design\border_male_operator 1 1 1 1 1 1 1

capy\border_hale_global_hawk 1 1

capy\border_male_reaper 1 1 1 1 1

capy\border_sf 1 1 1 1 1

system\border_hale_global_hawk 1 1

system\border_male_reaper 1 1 1 1 1 1

system\border_sf 1 1

process\border_hale_global_hawk 1 1

process\border_male_reaper 1 1 1 1 1

process\border_sf 1 1 1 1 1

dprocess\border_hale_global_hawk 1

dprocess\border_male_reaper 1 1 1 1

Cohesion-based Costing

Defence Expenditure

Scenario 1

Alternative

Large Scale

Small Scale enduring

Scenario 2

Scenario 3

Medium Scale enduring

Small Scale enduring

Small Scale one-off

Medium Scale enduring

Small Scale limited

Small Scale one-off

Monte Carlo Simulation of impact of Variations in Demand

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0 75 150

225

300

375

450

525

600

675

750

825

900

975

1050

1125

1200

1275

1350

1425

Scaled Cost1 Scaled Cost2 Scaled Difference

The resultant Cost Distributions Value for

Defence: Value of reduced Costs of Alignment

… and to a different kind of analysis of value

* See Boxer, P.J. (2008) What Price Agility? Managing Through-Life Purchaser-Provider Relationships on the Basis of the Ability to Price Agility, Navigator White Paper, Software Engineering Institute, Carnegie Mellon University, September 2008


Case Examples

Case Root Customer Problem Outputs of Analysis Duration

Raytheon Need to create and position new value propositions

Redefined the relationship to the market, making it possible to capture new kinds of opportunity

2-3 day workshop for each market

Thales Unable to cost alternative forms of alignment and establish the value of introducing new system flexibilities.

Established the baseline cost of current levels of agility to support the value propositions of alternative forms of flexibility

Three 1-day workshops + analysis of cost data over 2-3 weeks

NHS Collaboration platform needed to support roles and accountability structure.

Provided a framework for through-life accountability to the customer, changing the basis on which customers can acquire services

2-3 days initial workshop, 3-4 weeks analysis + 1-day feedback workshop

JFSP II/ CREATE

Horizontal and vertical scope of data and scientific model collaboration platform.

Providing dynamic support to collaboration amongst users through analysis of data fusion across chains of scientific models.

Two 1-day workshops + analysis over 2-3 weeks

JFSP I Gaps between profiles of strategic intent, organizational plans and their execution.

Established the ability to determine the lack of alignment between systems and the uses they are ultimately supporting, facilitating investment targeting

‘n’ interviews + interpretation over 2-3 weeks + 1-day feedback workshop.

NATO Gaps in the different layers of alignment between the underlying systems and their ultimate contexts of use.

Identified the risks to sustaining particular organizations of interoperation, creating an explicit focus on mitigating SoS risks

Three or four 2-day workshops over 2-3 weeks

MoD Existing methods ‘blind’ to the required variety of configurations of interoperating capabilities

Extended the definition of capability gaps to include organizations of interoperation, making it possible to direct resources to SoS capabilities

Three 2-day workshops over 2-3 weeks


IN CONCLUSION

This leaves a lot of unresolved issues…


Affordable

Capability

Development Plan

Balance of

Investment

Resource

Constraints

Defence

Priorities

Force

Development

Options

Identify Capability

Mismatches

Capability

Assessment

Current and

Planned Capability

Capability Goals

Scenarios

Defence Priorities Capability

Partitions

Government

Guidance

Future Environment

(Threat, Tech’y etc)

optional

Operational

Concepts

2. Needs addition of

capability goals for

orchestration and

synchronisation

1. Needs partitioning

superstructure made

independent of

stratification of

interoperability

3. Needs stratified analysis of requisite

interoperabilities across DOTMLPFs,

changing basis of assessment,

analysis of gaps and pricing of options

Source: Capability-Based Planning – Developing the Art, 2007

Critique of existing Capability Engineering


Bridging between the medium and the longer term

• The Challenges – What kind of Governance is appropriate

to the distributed collaborations across ULS systems?

– How are the changing services provided by the operational components of ULS systems to be enabled to achieve levels of cooperation and collaboration that can satisfy requirements for fast system evolution?

– How are we to enable the distributed collaborations across ULS systems to be edge-driven?

– How are the processes of orchestration and synchronisation to work in ULS systems?

– How are people to be taken into consideration as first-class components of ULS systems?

• Unresolved Issues – How are the questions of authorization

and ‘in whose interests’ to be approached?

– If stratification is driven as much by the contexts of use as it is by what is computationally feasible, how are these two ‘axes’ of engineering to be held in relation to each other in ULS systems?

– How is the nature of demand on ULS systems to be understood?

– What is the place of accountability, transparency and risk in these processes?

– What assumptions are being made about the relationships between person, enterprise and ULS system?


Balance of

Investment

Capability

Assessment

Capability

Goals

Scenarios

Capability

Partitions

Future Environment

(Threat, Tech’y etc)

optional

Operational

Concepts

Force Development

Options

Stratified

Analysis

Interoperabilities

across all DOTMLPFs

Analysis of

Alignment Costs

Pricing of options

across all DOTMLPFs

Force Geometries

Orchestration &

Synchronisation

Risk Analysis +

Identify Capability

Mismatches

Agility types I, II & III

interoperability risks

Requisite variety of

geometries

Force

Geometries

‘envelope’

Capability Engineering for

Agility


END


UAV II – CCIR ISTAR asset (Watchkeeper)

UAV I – over-the-horizon targeting (Phoenix)

Without an ability to analyze cohesion it is difficult to establish the costs of alignment…

The asset moved from being organic to the MLRS capability to

providing a capability to Divisional Command


… and identifying options for increasing agility

UAV III – extending ‘edge’ capability

(+ Nimrod, Lydian & Desert Hawk)

Afghanistan needs a much greater layering (and fusion)

of feeds and their much greater availability at the

‘edge’ in support of a greater campaign tempo with a commensurately greater

readiness tempo

33

The unanticipated need for agility can be very expensive


Governance in Ultra-Large-Scale Systems

Business

Transcript of Governance in Ultra-Large-Scale Systems