NRECAThe Agile Fractal Grid “The Gathering of Eagles”

Tuesday, February 25, 2014

Arlington, VA

Coffee and Breakfast

Agenda for NRECA Agile Fractal Grid Kickoff Meeting

8:00 Welcome by Martin Lowery; Opening by Chuck Speicher

8:15 Introductions of the Participants – John Reynolds

8:45 Achieving the Common Language through Design Patterns

9:20 The Vision of the Agile Fractal Grid – Craig Miller

9:45 Break

10:00 Discussion of the Managed Services Store and Applications

10:20 Discussion of the Distributed Systems Elements

11:45 Discussion of the Industrial Internet – Security, Resiliency, Efficiency

12:15 Lunch

1:00 Scope of the SmartAmerica Challenge vs. the Production Rollout

2:00 SEIT Process and Phase 1 Planning

2:30 Break

3:00 Phase 2 Implementation and Thread Assignments

3:30 General Discussion, Issues, Action Items, and Understandings

4:00 Wrap Up and Conclusion

Welcome to NRECAMartin Lowrie

Background on the National Rural Electric Cooperative Association

Convening of the Security Fabric AllianceChuck Speicher

Announcement of House SFA RulesJohn Reynolds

It has been said that the greatest engineering achievement of the 20th century

has been the building of the electric grid.

- The National Academy of Engineering

Invited Visitors

Department of Energy

Carol Hawk

Program Manager, Cyber Security for Energy Delivery Systems

IvanpahWorld's largest solar plant started producing electricity today

Ivanpah was supported by $1.6 billion in investments from the Energy Department's Loan Programs Office.

It represents the future of energy –utility-scale renewable power generation

that uses the latest technology to harness the sun where it is most abundant.

Federal CommunicationsCommission

Henning Schulzerinne

Office of the Secretary of Defense

Rollie Wicks

Department of Homeland Security

Lisa KaiserSarah Mahmood

Executive Order (EO) 13636: Improving Critical Infrastructure CybersecurityPresidential Policy Directive (PPD)-21: Critical Infrastructure Security and Resilience,

Critical Infrastructure Cyber Community Voluntary Program

Secretary of Homeland SecurityJeh Johnson

PM – ISEDNI

Pamela Wise-MartinezMike Howell

Kshemendra Paul

Mission Partners

Congress

Diane RinaldoLegislative Director

For Mike RogersMike Rogers – Michigan 8th District

U.S. House of RepresentativesChairman, House Intelligence Committee

Member, Energy and Commerce Committee

United States House Energy Subcommittee on Communications and TechnologyUnited States House Energy Subcommittee on Health

Cyberthreat Data Sharingand Protection

SmartAmericaChallenge

Sokwoo RheePresidential Innovation Fellow

The Challenge

We are planning to support the 940 rural co-ops in the U.S. with hybrid cloud/device services protected by the Security Fabric.

This is also the candidate energy “testbed” for the SmartAmerica demonstration.

What NRECA has asked us to do.

1. Stand up an “app store” for both operational and analytical applications for the cooperatives.

2. Participate in the “Substation of the Future” for the remote equipment and subsystems.

3. Apply the Security Fabric within the emerging Industrial Internet to protect all communications between the elements.

Introduction to the Security Fabric Alliance

• The Security Fabric Alliance is a working association dedicated to practical deployment of the power grid and critical infrastructure complex system solution in the United States:

– Utilities and telecommunications providers

– Systems integrators

– Manufacturers

– Technology partners

– National certification and interoperability entity

• The alliance is intended to give the CEO of a utility the purview of up-to-the moment knowledge of the options available to make wise investment decisions regarding infrastructure deployment for optimal returns.

The variation includes the proper orientation for large, medium, and small utilities.

There are many participants at different levelsin the Security Fabric Alliance.

ComponentsProductsSubsystems

ResearchIntegrationUtility

Customers

• Intel – servers with Quark + TPM • Wind River – Security Connect• Middleware

• RTI – DDS• GridStat • Indra - iSpeed• MultiSpeak

• TeamF1 – Secure Communications• Secure Crossing – Protocol Whitelisting• PsiNaptic – Secure Service Distribution• SNMP Research – SNMP Agent• Freescale – HSM w/Vybrid SoC• Xilinx – CompactRIO SOC• Green Hills Software - INTEGRITY• Altera - tamper proofing• Microsoft – Active Directory• Red Hat – Auth Hub

• General Electric – EMS• Alstom Grid – EMS • Siemens – DMS• Viridity Energy – DR + DER

+ Microgrid • Lemko – LTE systems• Intel Security – SIEM + GTI

• Intel – Encanto+ silicon support

• Sypris – Supply Chain Root of Trust

• TCIPG• EPRI – CIM Standards• MIT – Security &

Privacy Standards

• EPG – Phasor Data Portfolio• GridSense– NAN & Line Sensors• S&C IntelliTeam• SafeNet – Secure Key Management• Heart - Transverter• Freescale One Box• Cisco Cloud-in-a-Box

• Integrated Architectures – SEIT• MACE Fusion - DoD• Kryptos Logic – Red Team Certification• M2M Dynamics• Drummond Group – C&IT• Intel Security - Distribution

...First Stage……• ERCOT• ONCOR• AEP• NRECA• NRTC

Suppliers

• Verizon• Level3• AT&T• Internet2• BT

• ViaSat• Hughes• ARINC• Stratus• Symmetricom

…Second Stage……• APPA• SDG&E• PJM• NYISO• Southern Company• Duke Energy• CAISO

• Pecan Street• Mueller Community• Pike Powers

• PNNL – CyberSecurityTest Center

• Lincoln Labs• OMG SIG• SmartAmerica

Managed Services

• Tazca – Connect• CSG International• Digi International• N-Dimension

• SETI• Lockheed Martin• SAIC• Threat Connect

To establish the security of end-to-end systems using the Security Fabric elements, you need to do all seven… not just some.

4. Audit

– Records noteworthy events for later analysis

5. Confidentiality

– Encrypts sensitive data for matters of privacy.

6. Integrity

– Ensures that messages have not been altered.

7. Availability

– Prevents denial of service attacks

1. Identity Management

– Ensures the device identity is established genuinely

2. Mutual Authentication

– Allows both the Device Node and the Controller to verify the trustworthiness their identity to each other.

3. Authorization

– Manages permission to proceed with specific operations.

These are the seven tenets of security as described in the NIST-IR 7628 Guidelines.

The OMG process is more about establishing marketsas opposed to just setting standards.

SFA ReferenceBuilds

Certification ofConformance &Interoperability

The OMG is planning to standardizethe Security Fabric

for all critical infrastructure.

SFA Participants Introductions

Systems Engineering, Integration, and Test

• President – Energy One Solutions• Secretary General – GO15

Very Large Power Grid Operators• President and CEO

Alstom ESCA Corporation• CTO – PJM Interconnection• Co-founder Viridity EnergyAlain Steven

John ReynoldsAlain StevenRick SchaalMike SistoJim Baller

Bob LamontPaul Carroll

Mike Sisto

• IBM’s Wireless Emerging Business Opportunities• AT&T, Lucent Technologies and Hewlett-Packard• Lemko Distributed Mobile Wireless Network solutions

Rick Schaal

• VP, Engineering – Viridity• Executive Program Director –

Alstom ESCA• Program Director – AREVA• DMS Lead – Puget Sound Energy

• American Public Power Association • Fiber to the Home Council • Google Inc. • National Association of Telecommunications

Officers and Advisors• + several hundred specific broadband projects

Internet2

Christian Todorov

Texas A&M University

Named second in the nation by The Wall Street Journal among all universities,public and private, in a survey of top U.S. corporations, non-profits and government agencies,

based on graduates that recruiters prefer to hire.

“Home of the 12th Man”

Walt Magnussen

Real-Time Innovations

AEGIS Weapons System

Total Ship Computing Environment

NASA Robotic Controls Unmanned Aircraft Systems

Tom O’ConnerMark Hamilton

Indra

Michael PeckEloy Gonzalez Ortega

GridSense

Transformer IQ

Line IQ

Inspection

Brett Sargent• System Capacity• Phase Imbalance• Metering / Theft• Conservation Voltage

Reduction• Four Quadrant True

Power Factor• Line Fault Directionality• PQ – voltage sags/swells,

interruptions, surges, unbalance

• Transformer Asset Management

Lemko

LTE over Satellite

Cellular on Wheels

Utility Industry

WiMax ReplacementSecurity Industry

Transportation Industry

Mike Sisto

Public SafetyHealthcare industry

Subnet Solutions

SEL Substation Controller

Message Management from Substations

Ameen Hamdon

CSGi

Jarrid HallEddie Day

Over 8 trillion revenue event records processed annually for major clients.

InterMediate

Electric Power Group

Direction of Grid Stress

Voltage Stability Monitoring

Vikram BudhrajaVivek Bhaman

BaseN

Anders Viden

BearingPoint Infonova

Concept-to-Cash

Andrew Thomson

The Orchestration of the Ecosystem.

BT America

Milton Keynes

Tom WolfBryan Fite

Milsoft

Engineering Analysis

Geographic Information

Field Engineering

Outage Management

Wayne Carr

TOA

Neil RingersMike Carpenter

Kevin Cantoli

TeMix

Ed Cazalet

Transactive Energy

Netronome

DJ RootJerome Taylor

256 GB

Green Hills Software

Smart Farming

Smart Medical Systems

GE Aircraft Engines

EMC High End Storage Systems

Transportation Systems

Phase Array Radar Systems

Weapons Control Systems

Guidance Systems

Game Systems

Automotive Systems

John WartherJohn Killam

Brad JacksonBrian Riley

LocalGrid Technologies

Power Quality Analyzer

Microgrid controls:• Microgrid and Generation Control• Generation and Asset Monitoring• Storage Integration

Device EnvironmentController Environment

Hypervisor

ApplicationManagement

Agent

Hypervisor

ApplicationManagement

Agent

David Berg

CryoKey

Chris Ho

Identity ManagementMutual AuthenticationAuthorization

Secure Crossing

Randy ReevesRobert Lorente

N-Dimension

Andrew WrightBruce Gordon

Secure Exchange TechnologySETI

Richard MarshallElysa Jones

Threat data collaboration.Community Emergency Management.

National footprint today!

Threat Connect

Leigh Reichel

1300 Communities of Interest already in production!

Cornice Engineering

Gary McNaughton

HereNow

Brian Reynolds

Utilities Telecommunications Council

Eric WagnerBrett Kilbourne

• Advocacy and Training• License Sentry• Frequency Coordination• Microwave Coordination• PLC Database• RF Safety and Training• Regulatory Filing and Tracking

General Electric

Substation Simplification

Transformer Protection System

Energy Management System

Wide Area Monitoring

System Integrity Solution

Joe Salvo

National Rural Telecommunications Cooperative

Kurt Schaubach

National Rural Electrical Cooperative Association

Martin LoweryRobbin Christianson

David PinneyMaurice MartinBarry Lawson

Bridgette BourgeBob LamouthDoug Lambert

Paul CarrolCraig Miller

Building a Common Language

Everyone shapes his or her view of the world through framing, and people with different backgrounds are likely to see their surroundings through vastly different frames.

By themselves, frames are not inherently good or bad…they just are.

The quality that all of us seek in lifethat has no name

includes

Simplicity.

The Laws of Simplicity are very important in complex environments.

1. ReduceThe simplest way to achieve simplicity is through thoughtful reduction.

2. OrganizeOrganization makes a system of many appear fewer.

3. TimeSavings in time feels like simplicity.

4. LearnKnowledge makes everything simpler.

5. DifferencesSimplicity and complexity need each other.

6. ContextWhat lies in the periphery of simplicity is definitely not peripheral.

7. EmotionMore emotions are better than less.

8. TrustIn simplicity we trust.

9. FailureSome things can never be made simple.

10. The OneSimplicity is about subtracting the obvious, and adding the meaningful.

John Maeda leads RISD: the #1 school of design in the country.

Never design a thing without first understanding the next larger context into which it must fit.

• A chair within a room,

• a room within a house,

• a house within a neighborhood,

• a neighborhood within a city.

- Eero Saarinen

There are two more guiding principles in all contemporary architecture.

Cranbrook Academy is the #2 school of design in the country.

Use a system of design patterns to allow local teams to design their own systems

out of interchangeable parts such that integration is assured from the outset…

but such that no two local systems ever look exactly the same.

- Christopher Alexander“The Timeless Way of Building”

Design patterns allow for organic growth.

This treatise on design patterns is now he basis of most object-oriented systems in the world today.

Our assignment here is to build a forest of oak trees.

First start with a vision of the end goal...

… then work backwards in time to determine the first step.

Those who are not students of history ……are doomed to repeat it.

We always seem to build things four times, and in a certain order.

– Make it work

– Make it fast

– Make it small

– Make it pretty

- Jim Ladd

Systems Engineering Strategy:

The Building of the Boeing 777

This iconic building remained the tallest building in the world for over 40 years…

… and yet took only one year to construct…

In their first meeting with the client, Lamb asked Raskob about his vision

for the building. Raskob stood a pencil on end and said,

"How high can you make it so that it won't fall down?“

The initial plans wereready in two weeks.

John Jakob RaskobChief Financial Officer

of bothDupont and General Motors

The building of the Boeing 777drew from the lessons learned.

• The design phase for Boeing's new twinjet was different from the company's previous commercial jetliners. For the first time, eight major airlines had a role in the development of the airliner. This was a departure from industry practice, where manufacturers typically designed aircraft with minimal customer input.

• The 777 was the first commercial aircraft designed entirely on computer. Each design drawing was created on a three-dimensional CAD software system. This let engineers assemble a virtual aircraft, in simulation, to check for interference and verify that the thousands of parts fit properly—thus reducing costly rework.

• The initial 777-200 model was launched with propulsion options from three manufacturers, General Electric, Pratt and Whitney, and Rolls-Royce, giving the airlines their choice of engines from competing firms.

The advice from some of the team is memorablerelevant to creating complex systems.

Chapter 3 on Reliable Deliveryrecounts some of the inspirationfrom the building of theEmpire State Building in 1930.

In the beginning it is always good to come up with at least two or three options for how a capability might be implemented…

… and then defer making a final decision until after they are better understood relative to what the overall system must do.

But how should we start?

• We should first go talk to the least capable of our customers and walk a mile in his moccasins to see what takes place there:

– for that feel of what events go on there – for what their definition of happiness is all about– for that essence of what life itself is all about – for “the quality that has no name”… (more on this later…)

Let’s use a compelling user story to make it easy for all to see the intent.

The Vision“The Agile Fractal Grid”

Achieving

Grid Security, Reliability, and Resiliency

through Advanced Analytics and Control

A Tale of Two Feeders

On a nice day, the

utility is running smoothly.

Everything is “Hot”

A bad storm comes through

and some areas lose power

But backup power,

distributed generation,

and stored energy allow

some areas to island

and continue operation…

Smart control systems allow

the islands to network

This may allow some power to be restored in additional areas as the

utility works to bring dark areas on line

Soon, everyone has power

The smaller grids

reconnect with the utility

Collect Data

Transform and Organize Data

Perform Analysis

Derive Decision

Implement Action

Collect Data

Transform and Organize Data

Perform Analysis

Derive Decision

Implement Action

All grid applications have the same basic structure.

Collect Data

Transform and Organize Data

Perform Analysis

Derive Decision

Implement Action

Data level integration

Abstraction of the data layers opens the grid to better, lower cost development

Abstraction Model for the Agile Grid

Layer Innovations Needed

Action Layer Advanced technology:• Smart feeder switching• CVR• Advanced sectionalization• Direct load control• Storage• Phasor measurement units• Solid state VAR control…High bandwidth, high-reliability communicationsSoftware defined networks

Decision Layer Static and dynamic power flow modelingEvent tree generationTaxonomic system for failure classification

Analysis Layer Predictive modelingStatic and dynamic power flow modelingAI based anomaly detection

Information layer High performance databaseDistributed information storesCloud based computing

Data Layer High performance sensor networksImproved interoperability standardsPassive and active capture technology

Innovations are needed at all layers.

Networking Break<Saturn>

Discussion of the Managed Services Store and Applications

At NRECA the private cloud-based supportneeds to provide applications options for

all the participating utilities.

The Security Fabric must support all the interfaces.

Candidate Applications

• Business Functions External to Distribution Management

– Customer Relationship Management (CRM) (includes CIS)

– Meter Data Management (MDM)

– Customer Billing (CB)

– Payment Processing (PP)

– Prepaid Metering (PPM)

– End Device Testing and Receiving (EDTR)

– Finance and Accounting (FA)

• Distribution Engineering, Planning, Construction, and GIS

– Asset Management (AM)

– Project Manager -Configuration Management

– Engineering Analysis (EA)

– Field Design (Staking)

– Geographic Information Systems (GIS)

– GIS Viewer (GV)

– Inspection (INSP)

Candidate Applications

• Distribution Operations

– Integration Server

– Call Handling (CH)

– Distribution Automation (DA)

– FAST voltage regulation, active power control and load flow control in meshed power systems.

– Interactive Voice Response (IVR),

– Outage Detection (OD)

– Outage Management (OA)

– Supervisory Control and Data Acquisition (SCADA)

– Substation Automation

– Switching Orders (SWO)

– Vehicle Location (AVL)

– Volt/Var Control

– Demand Response (DR)

– Load Management (LM)

– Demand Optimization Systems (DOS)

– DRAS Server

Candidate Applications

• Distribution System Monitoring

– Connect/Disconnect (CD)

– Load Profile (LP)

– Meter Reading (MR) (AMI)

• Work Management

– Work Task Generator (WTG)

– Work Task Originator (WTO)

– Work Task Performer (WTP)

– Work Task Scheduler (WTS)

– Work Task Viewer (WTV)

• System Providers / Aggregators

– DRAS Home/Building Manager

– DRAS Client Aggregator

• Customer Premises

– HAN Communications

– Energy Services Interface

– Meter

Candidate Applications

• G&T Ops

– G&T EMS

– Demand Management

– Load Forecast (Load profiles?)

– Distributed Storage EMS

– Distributed Generation EMS

– Market Services

• (External Market Functions)

– Energy Management Clearinghouse

– ISO EMS

To be relevant as a digital services provider,you must develop new repeatable capabilities

to do the following:

• Bundle service provider services

• Define new services as products

• Automate, monetize, and allocate revenue of the contracted relationships

This is not simply just a walled garden…Plus, it is not just simply an Amazon product shopping cart.

Candidate Functional Architecture

Industry Specific Tenant / Resellers # 1-n

Service Providerse.g. industry specific

service providers

Offerings onService Provider terms

Ecosystem Enablement Platform Aggregator & ICT Service Broker

Service Providerse.g. SaaS, IaaS, PaaS

providers

Service Providerse.g. Telco, Mobile Cloud providers

Offerings onReseller terms

Offerings onICTSB Operator terms

Generalist Tenant / Resellers # 1-n

Offerings onReseller terms

R6 VSO R6 VSO

R6 VSP

“Amazon type” Digital Services Enablement

Infrastructure

Layer

Enabling &

Partnering

Layer

Digital Services

Provider

Products, Content, Applications, Services

Selection of applications to launchwill use a comparatively simple interface.

The repeatable capabilities need to enable

entity to entity partnering and C2C white labeling:

SYSTEM

Ecosystem &

Partnering Enabler /

Cloud Service Broker

on-board Service Provider

on-board Reseller

Define SLA

resolve Incident

publish Retail Offering

on-board End Customer

publish Wholesale Offering

pay Invoice

report Incident

issue Invoice

Monitor SLA

Handle SLA Violation

on-board Service

Service Providers

/ Suppliers

retrieve CatalogInformation

retrieve Quote

place Order

get Order Status

Digital Service Providers

/ Resellers

End Customers

B2B2x Entity Interaction Diagram

Discussion of the Distributed Systems Elements

Electric power distribution and broadband communications are like Siamese twins!

(They can’t go anywhere without each other.)

Digital control is neededat each junction point.

Like electricity,broadband can be used

for multiple things.

The Circulatory System The Nervous System

NRECA

NRTC

We will use a typical distribution path to demonstratethe intelligent combination of power controls and broadband.

CentralControl

Local AreaRelay

NeighborhoodRelay

SubstationRelay

Management

Servers

Substation

Server/Router

NAN

Gateway

HAN

Gateway

Communications / Firewall

FTL (E&LM)

SIEM

E&LM

Communications Communications / Firewall

E&LM

Sensor Mgt

Communications / Firewall

E&LM

Meter App

HomeApp

TE App

SA SA SA

SA

SA

SA

Cell Manager

MasterAgent

Posture Validation

Remediation Server

Jini SP

MA

SA

“Cell Management”

“Local Management”

There are multiple locations in the conceptual network.

Management

Signaling Server

Our communications has redundancybuilt into the control protocols.

LTEMacrocell

andDistribution

Fiber

NANLTE

Picocell

Mobile

HANLTE

Home Gateway

This is the only capability availablethat allows handoff between terrestrialLTE services and satellite services.

This system can simultaneously supportthe public safety 700 MHz frequenciesas well as commercial usage.

Symmetric 2Mbps10Mbps downloading and 786Kbps when uploading.

100 MHz LTE Services!

NationalInterconnect

The telemetry systems occur in front of the analytic applications and feed data to the Big Data stores.

Utility Operations

BroadbandOperations

ServiceProviders

The Fractal Pattern!

Knowledge

Monitor

Record

Analyze

Plan

Execute*

Policy

Sensor Effector

SymptomChange

Plan

Request Change

RequestReview

The policy execution environment is the key to the flexibility.

This is more than just the cybersecurity dimension.

Obtain image of network traffic

Structure data

Interpret events using policies

Compare current state to

baseline

Analyze options

Model power flows (if necessary)

Layer 1: Data

Layer 2: Information

Layer 3: Understanding

Layer 4: Decision

Layer 5: Action

Transmit to collector

ValidateExtract layer

3,4,7 data

1st level anomaly detection

Replicate

Notify anomaly

Update baseline model

Recommend remediation

Notify compromise

Monitor changes in network

performance

Implement network changes

Confirm implementation

Analyze anomalies

Resolve discrepancies

Human review, adjustment, and

approval (if necessary)

Activateworkflow

Compensate for exceptions

Finalize actions

Options: PNNL + Secure Crossing

Investigate options: Carnegie Mellon + InterSystem?

Options: Cigital + CRB

Options: BaseN

Options: ComplexRequest Broker

Activation system + configuration synchronizer

Understanding

Information

Decision

Data in – Action out

But sometimes semi-autonomic policy decisionsare made and executed in the field.

(at the small, the medium, and the large)

The policy logic is actually spread to each major active element.

MultiSpeakInitiative

This effort has been launched with a current research project already underway.

But the ultimate goal is to find COTS products that are fully supported that perform these functions.

“Always find two or three candidatesand explore them fully beforemaking a final selection.”

Power Grid Distributed NodeArchitecture

LegacyApps

SimulationLibs

RT-DB

IECAgents

IEC Drivers

OSGi Container

RT Linux or RTOS

DDS

JVM-RT

DDS API NIO API

ESP/CEP Engine

RDF-DBDistributed

Cache

ReasonerOWL-DL SWRL

APIOWL-DL

Rule EngineSWRL/RuleML

JADE (Bundle)

Publishing Server (HTTP)

Disturbance

OK

The Logical States

We have the process flows currently used for these states.

Trigger Workflow

Affected Workflow

Provider Workflow

Isolated Workflow

Lunch

<Picture Time!><Synchrophasor Visualization>

Discussion of the Industrial Internet –Security, Resiliency, Efficiency

The Security Fabric

Designed with Embedded Systems in Mind

The embedded systems include:

Our strategy is to provide certified interoperability to the key devices controlling the grid.

Our solution would be embedded

at each critical point in the energy infrastructure.

All points must connect to each other

in an end-to-end system.

Management Agents

The Security Fabric follows the guidelines required by the NIST 7628 for the Department of Energy.

xSystem &Network

Management

Controller

Device

Device TheSecurityFabric

The Security Fabric is an implementation of the Tailored Trustworthy Space.

Embedded Device

ApplicationsDevice

Management

Hypervisor

The fundamental concept is that you must first separate the management elements of

a device from the application payload portions.

This is similar to the AMT philosophy, but expanded to the software dimension.

Separation of Protection and Security

• The major hardware approach for security or protection is the use of hierarchical protection domains. Prominent example of this approach is a ring architecture with "supervisor mode" and "user mode“ [Intel].

• This approach adopts capabilities provided by a lower level:(hardware/firmware/kernel).

The Multics Style of Ring Structure

The Security Connected components need to handle the various management agent functions.

The approach also provides for secure persistent storage.

Policy logic goes here.DDS goes here.Industrial ProtocolWhitelisting

The security inner sanctum ring

There are multiple transports planned.

The client/service and peer-to-peer relationships of the applications must be uniformly planned.

The channel stacks facilitate interoperability for individual sessions.

We use an industrial protocol whitelisting technique to avoid false positives in mission critical operations.

Co

nte

ntA

war

e F

irew

all

Pro

toco

l Wh

ite

listi

ng

Laye

rs

4-6

IP C

om

mu

nic

atio

ns

Stac

k –

Laye

rs 2

-3

IPse

c V

PN

UDPv6

UDPv4

Connections

• Kerberos Get Credentials + Tickets• Get Extended Credentials• Kerberos Mutual Authentication• Get Precision Time• Register for Management +

Configuration Synchronization• Service Locator• Service Provider• Multicast Alert• Unicast Command• Event Notification• SNMP Get/Set• Application Event: Send and Receive:

• High Priority• Medium Priority• Low Priority

SessionsIn

terf

ace

A

Inte

rfac

e B

There are servers and agents in the industrial environment.

The next greater context includes collaboration:Cyberthreat collaboration & emergency notification

Sensing Analyze Decision Action

Picocell

UtilitySIEM

Utility

TOPSFA Boundary

Control

Situational Awareness

IncidentManagement

Alert/Notification

Threat Connect

SMS

Email

Phone

Sirens

Social

Media

Radio

• DHS• NSA• DOE• DOD• State&Local• Commercial

CAP

CAP

• Culpepper• Dallas• Chicago• San Joe

DHSFEMAIPAWS

Cell Broadcast

EAS

CMAS

NWEM(HazCollect)

EDXL

All communicationsare protected usingthe Security Fabric

CEF

• 1220 subscribing organizations

• 1300 communities

This will be part of the emergency response system for SmartAmerica.

The cybersecurity threat sharing needs to be performed between multiple communities

to be effective.

Subscriber

Subscriber

Subscriber

Subscriber

Subscriber

Context

Top Secret

Secret

Unclassified

F

F

ThreatConnect

STIX

Custom

Co

mm

un

ity

Co

mm

un

ity

Co

mm

un

ity

1300Communities

1200Subscribers

On-Premises Cloud

UI

API

Private Cloud

Threat ConnectCloud Platform

Amazon EC2

Broker

SourcesFree, $, other

The data arrangements can be hierarchicalto facilitate multi agency awareness.

IODEF

At a higher level of correlation,the Saturn cyberthreat real-time analysis

system will literally run rings around attacks.

Self-organizing Adaptive Technology Underlying Resilient Networks

The “Concierge” Service

• Attention! Some of the co-ops have very tiny IT staffs. (Like one person)

• The coming cyber attacks will be very sophisticated.

• Even the central staff will sometimes be challenged to deal with the complexities associated with cyberdefense.

• Thus, even with collaboration and data sharing, from time to time, a co-op technician would like to have an “OnStar” button to push to get instant help on demand from a specialist.

This Concierge service from ThreatConnect may be very desirable.Saturn comes in here to see the bigger picture.

Scope of the SmartAmerica Challenge vs. the Production Rollout

SmartAmerica Architecture

“Open, secure, high-confidence and collaborative CPS network “

CPSCommunications

Fabric

Healthcare TBs*

Transportation TBs*Energy TBs*

Emergency Response System TBs*

Building TBs*

Smart Manufacturing TBs*

SmartAmerica

Data Center(s)Avionics TBs* Technology TBs*

NRECA + NRTC

FirstNet + AlertSense

TerremarkMIT CICAlphavillePecan Street

IndustrialInternet

SecurityFabric

Our part of the demonstration will feature the initial use of the Industrial Internet.

• Chuck Speicher will be the principal leader for this effort.

• General Electric will be a key member of the effort.

• Internet2 will be a primary contributor for the effort.

• There will be many other partners from the Security Fabric Alliance that will participate.

• We will feature concepts from the NRECA and NRTC inspired movement for the smart grid element.

• Other elements will be added where advantageous.

Let it be said that the Industrial Internet started right here.

As envisioned, Internet2 would eventually provide the requested DWDM-based network for the Industrial Internet.

Ultimately, there would be four regional control centers but all serve as redundant backups.There are 27 core points that connect to feeders.

However, if we are true to our strategy, we should never design a thing without first understanding its next greater context.

And the rest as they say will be called history.

The objective is unusual:Arbitrary results are not the point!

What we do is not so important as how we do it.

Lean development will produce lasting results in half the time and cost half as much.

We are producing the real production product, not conducting a high school science experiment.

We will learn the “other 90%” of what we need to learn using this TPS process.

SEIT Process and Phase 1 Planning

The Nature of the First Iteration

InterMediate

There would be multiple phases to allow for piloting and then production status.

Trans-former

IQSubnet Subnet

InSiteStorage

Platform Gateway

Config

BaseN

OpCenter

SIEM

ActiveDirectory

R6

AuthenticationTime Synchronization

CMDB

SecurityEventManagement

Console

Fault MgtPerformance Mgt

EcoSystemOrchestration

•Validation•Repair•Gap / Dup•Reformat•Correlation•Distribution

ePDC

InterMediate

There would be multiple phases to allow for piloting and then production status.

Trans-former

IQSubnet

InSiteStorage

Platform Gateway

Config

BaseN

OpCenter

SIEM

ActiveDirectory

R6

AuthenticationTime Synchronization

CMDB

SecurityEventManagement

Console

Fault MgtPerformance Mgt

EcoSystemOrchestration

•Validation•Repair•Gap / Dup•Reformat•Correlation•Distribution

RTDMS

PMU

ePDC

Subnet

ePDC

InterMediate

There would be multiple phases to allow for piloting and then production status.

Trans-former

IQSubnet

InSiteStorage

Platform Gateway

Config

BaseN

OpCenter

SIEM

ActiveDirectory

R6

AuthenticationTime Synchronization

CMDB

SecurityEventManagement

Console

Fault MgtPerformance Mgt

EcoSystemOrchestration

•Validation•Repair•Gap / Dup•Reformat•Correlation•Distribution

RTDMS

PMU

ePDC

Subnet

One Box HAN Gateway

TeMixTransactiveEnergy

ePDC

InterMediate

There would be multiple phases to allow for piloting and then production status.

Trans-former

IQSubnet

InSiteStorage

Platform Gateway

Config

BaseN

OpCenter

SIEM

ActiveDirectory

R6

AuthenticationTime Synchronization

CMDB

SecurityEventManagement

Console

Fault MgtPerformance Mgt

EcoSystemOrchestration

•Validation•Repair•Gap / Dup•Reformat•Correlation•Distribution

RTDMS

PMU

ePDC

Subnet

One Box HAN Gateway

TeMixTransactiveEnergy

ThreatConnect

Saturn

Networking Break

In the iterative nature of the agile lean development,the customer recommends the following beta candidates:

• MCNC – North Carolina (distribution)

• Pedernales – Texas (distribution)

• Tri-State – Colorado (generation and transmission)

• Grant County PUD – Washington (muni)

• San Diego Gas & Electric – California (IOU)

• SPEC – Texas (generation and transmission - wind)

• Fort Myers – Florida (smart community – solar)

We should be true to the Agile Manifesto and let the scope float a bitbased on what is learned in each two week sprint.

Agile Lean Development:an integral part of the process

• The frame of reference for this program is the lean mindset.

• Delivery of value to delighted customers is the goal.

• We will use Kanban to manage the flow of learning during the quarterly releases, the two week sprints, and the scrum meetings.

– The backlog

– The in process sprints

– The finished feature support

• Optimization and the elimination of waste and technical debt go on forever.

Methodology for SmartAmerica

Phase 1

Phase 2

Estimated Phase 1 Schedule

Milestone Week ending

• Kickoff meeting 1

• Objectives section complete 1

• Requirements assessment section complete 8

• Strategy section complete 9

• Technology evaluation section complete 9

• Architecture section complete 11

• Leadership presentation 12

• NRECA management presentation 13

Solution provider SEs

Acquisition SEs

At large scale, the effort is divided into parallel threads with cross-thread interactions.

Architecture DevelopmentModeling & Simulation

Interface ManagementData Characterization

Requirements Management

Milestone DocumentationScheduling

Integration & Test

SE Functional Role Focus

Cross-Thread Analysis Focus

Mission Thread SE Focus

Deployment/Baseline Focus

Deployment Team Focus

Thread 1 Thread NThread 2

Presentation/Query Interface

Policies/metadata/knowledge management

Analytic processing

Data ingest processing

End-to-end security The thread leaders are the keyto complex systems engineering.

Phase 2 parallel threads currently include:1. Creation of the Security Fabric Management Agent Simulator

2. Deployment of the systems management servers

3. Creation of the Industrial Internet on the I2 lambda rail

4. Implementation of the Cloud Broker

5. Implementation of the neighborhood LTE picocells

6. Implementation of the GridSense sensors

7. Implementation of the Subnet substation servers

8. Implementation of the takeout node with data ingest server

9. Implementation of the GridSense InSite analytics

10. Implementation of the EPG visualization service

11. Implementation of the SETI data sharing and emergency notification

12. Implementation of the CyberThreat Collaboration Network

13. Implementation of Saturn and the Concierge Service

14. Implementation of the Transactive Energy pilot

15. Followed by an interim demonstration of capability

The entire system will be continuously integrated and tested each day.

The Nature ofGlobal Development Teams

Virtual Organization

• Organizing entities use a structure that resembles a network rather than a hierarchy.

• Organizations are driven to virtual forms to be more:

– Flexible

– Agile

– Responsive

– Inexpensive

Architecture and Task Allocation

Low

High

Good

Bad

Co

up

ling

Co

hes

ion

High

Low

This is easier when the team locations align with the architecture a bit.

The Loss of “Communication Richness”

• Rich “high-touch” communication is defined as two-way interaction involving more than one sensory channel.

• A substantial portion (80%) is non-verbal and implicit. It consists of body language such as:– Gestures

– Facial expressions

– Postures

• Richer media conveys more information and emotion.

We need to decide how we are going to do these things this time:

• Colocation period

• Scrum calls– Electronic kanban boards to plan and follow progress

– Continuous delivery system (integration, testing, & deployment)

– Configuration management and automated build processes

• Regroup huddles

• Estimates

• Funding process

General Discussion –Issues, Action Items, and Understandings

Wrap Up and Conclusion

The Timeless Way of Building

Indeed this ageless character has nothing, in the end, to do with languages.

The language, and the processes which stem from it, merely release the fundamental order which is native to us all.

They do not teach us, they only remind us of what we know already,

and of what we shall discover time and time again,

when we release our ideas and opinions, and do exactly what emerges from ourselves.

Closing

776 B.C.“Let the Threads Begin”

Extras

Background

Focus

Functions

Tasks for Performing EBV in the EMS-DMS-SCADA Platform

Collaboration Scheme among Intelligent Nodessupporting the EBV in the ENERGOS Smart Grid

Osterwalder - Business Model Canvas - page 1

Element Sub-element Semantics

Business

model

Name The name of the business model

Short description Short description of the business model [extension to Osterwalder’s canvas]

Market players/

competition

Who are the relevant players in the market, running this business model? How competitive is the business

model? [extension to Osterwalder’s canvas]

Maturity level What is the level of maturity of the business model within the market? (e.g. innovative, well established)

[extension to Osterwalder’s canvas]

Priority What is the priority (relevance, importance) of the business model from a CSP perspective? [extension to

Osterwalder’s canvas]

Customer

/Market

Customer

Segment

What is the customer segment(s), addressed by the business model?

Customer Segments define the different groups of people or organizations an enterprise aims to reach and serve. A

customer segment is a sub-set of a market made up of people or organizations with one or more characteristics that

cause them to demand similar product and/or services based on qualities of those products such as price or function.

Channel What are the relevant channels (communication as well as sales & distribution channels) to reach the addressed

customers? Channels (communication, distribution & sales channels) comprise a company´s interface with

customers. Channels are customer touch points that play an important role in the customer experience.

Relationship What kind of relationship is expected by the customer/needs to be established for the business model? (e.g. self-

service, communities, personal assistance). Relationships can range from personal to automated.

Product

Offering

Value proposition What kind of products and services are offered to the customer? What value will be delivered through the business

model?

The value proposition is the reason why customers turn to one company over another. It solves a customer problem

or satisfies a customer need. Each value proposition consists of a selected bundle of products and/or services that

caters to the requirements of a specific customer segment. In this sense, the value proposition is an aggregation, or

bundle, of benefits that a company offers customers.

Element Sub-

element

Semantics

C2M

(concept to

market)

Key

Activities

Key activities are the most important things a company must do to make its business model work. What are the key

activities, to launch the offers product/services to the market, to establish the channels, to maintain the customer

relationship and to ensure the revenue streams? (e.g. development of community site, implementation of new

billing concepts).

Key

ResourcesKey resources are the most important assets required to make a business model work. What are the required key

resources, to launch the offers product/services to the market, to establish the channels, to maintain the customer

relationship and to ensure the revenue streams? (e.g. development of community site, implementation of new

billing concepts).

Key

Partners

Who are the key suppliers/partners, what are the key activities and resources, which need to be acquired from

those? (e.g. external call center, supplementary content from a content provider)

Enterprise

Management

Cost

Structure

The Cost Structure describes the most important costs incurred to operate the business model.

Revenue

Streams

Revenue Streams are the cash a company generates from each Customer Segment; A business model can involve

several different types of Revenue Streams:

• Transaction revenues resulting from one-time customer payments;

• Recurring revenues resulting from ongoing payments to either deliver a Value Proposition to customers or provide

post-purchase customer support;

-Shared Revenue – resulting from operations carried out by a B2B partner who uses enterprise’s products or

services to generate its own revenue from its own end customer but does not pay upfront or recurrent for

products/services from the enterprise – just % of actual revenue;

- 3-rd party revenue – revenue from a 3-rd party to include its own services in the enterprise offerings to its

customer segments.

Osterwalder - Business Model Canvas - page 2

Element Sub-element Semantics

Barriers Business

related

What are the business related barriers, to implement the business model? (e.g. high risk regarding

cash flow) [extension to Osterwalder’s canvas]

Technical What are the technical barriers to implementing the business model? (e.g. high risk due to immature

technology) [extension to Osterwalder’s canvas]

Drivers Strategic

portfolio

What are the drivers to implement the business model regarding the strategic portfolio planning? (e.g.

complementary to the existing portfolio strategy) [extension to Osterwalder’s canvas]

Revenue

Streams

What are the drivers to implement the business model regarding the revenue streams? (e.g. new

revenue opportunities to compensate decreasing airtime prices) [extension to Osterwalder’s canvas]

Touchpoints/Use

Cases

Upstream What are the key touchpoint/use cases/interaction patterns to interact with upstream customers? (e.g.

registering app in app store).Upstream customers are B2B partners/enterprise customers (e.g.

retailers, media, advertisers, utilities, finance etc.) [extension to Osterwalder’s canvas]

Downstream What are the key touchpoint/use cases/interaction patterns to interact with downstream customers?

(e.g. end customer billing). Downstream customers are the end users of a product or service, this can

be consumers as well as companies [extension to Osterwalder’s canvas]

Osterwalder - Business Model Canvas - page 3

BT’s Cloud Service Broker

Low Cost

Compute

- for test and dev

Secure SaaS

- e.g. CRM

Private Compute

job

- e.g. pharma,

finance

Extra resources-

- e.g. legacy

Application out of

space/power

CollaborationLow Cost

Compute

Secure SaaS

- e.g. CRM

Private Compute

job

Extra resources-

- e.g. legacy

Application out of

space/power

Collaboration

Low SLA

SaaSBPOS

Provider

High SLA

Compute

Commerce

MPLS

ConnectedVDC

Video

Low SLA

ComputeLow cost

Storage

High SLA

Carrier

PreventingRunaway costs

Security

Avoiding Vendorlock-in

ApplicationPerformance

Compliance and Audit

Transparency

Assurance

Cloud ServiceBroker(EEP)

Monitor & Assure

Compliance and Audit

The “Ecosystem Enablement Platform” (EEP) fulfills the role of the CSB

The NRTC has a variety of broadband services that could be tunneled through the Industrial Internet.

• OTT & TVE services

– Video services

• Walt Disney comprehensive long term distribution sports, news, entertainment

• NBCUniversal comprehensive content affiliation

• 100 HD channels

– NeoNova Cloud Services

– ViaSat Satellite Distribution

– Exede satellite broadband services

– Telispire – MVNO services

– WiMax service

– Planning for First Responder network

• Power services

– OATI demand response options

– Efacec – Advanced Control Services

– Clevest – workforce management and smart grid operations

– Power Secure LED lighting

– Sensus – Flexnet AMI; smart meters; distribution automation; demand response

– GeoNav – nVIEW 360o

– TenKsolar – solar voltaic systems

– Wind farm deployment

– Milsoft and Clevest MultiSpeak certification

“The Prolog”

The Tower of Babel is alive and well in the 21st Century.

In general, it is a good thing to address this reality right up front.

Frames of Reference

• According to cognitive scientists, we all interpret our surroundings through frames – mental constructs that shape our perspectives of the world.

• Frames are sets of beliefs about what elements to pay attention to and how these elements interact with each other.

• Frames place significant limits on our perspective; we can only see what our frames tell us is meaningful, and we usually ignore what lies outside the boundaries.

• Most of us are unaware of the way our background and experience shape the way we frame our decisions and actions; only a few of us consciously adjust our frames as if we were photographers.

• In fact, we seldom even think about the direction in which we are pointing our cameras.

Frames of Reference

• Everyone shapes his or her view of the world through framing, and people with different backgrounds are likely to see their surroundings through vastly different frames.

• By themselves, frames are not inherently good or bad…they just are.

• However, evidence has shown that certain frames are more likely than others to lead to long-term success.

We need to derive a good pattern language to get things properly rolling.

Osterwalder - Business Model Canvas

• …

Ecosystem & Partnering Capabilities ArticulationSuggested Procedure

• Discover “repeatable” functional building blocks from different business models

– Describe business model using Osterwalder’s Business Model Canvas

– Derive “ecosystem” canvas andfind “repeatable” patterns

– Derive reusable “architecture building blocks”

• Refinement of a candidate architecture to a detailed architecture blueprint

• Implementation of prototype scenarios

The Logical States

We have the process flows currently used for these states.

PsiNaptic

Jmatos

Aaron DaganCameron Roe

Change Control Interactions

Ring 2Change Mgt

Change Control

Service Locator

Service ProviderService

Consumer

WorkflowControl

AttestationDirectory

ServiceRepository

TransitionControl

1

2

3

4

5

DynamicBinding

Promote &Store

ActivateService Instance

86

7

KerberosDirectory

1

9

Credentials includeChain of Trust Certificates

Department of Defense

James GalvinDaryl Haegley

Michael AimoneKathleen Ahsing

Booze Allen Hamilton

Aaron Kraus

Change in Level of Trust Between Sites

Trust

Time

Sufficient levelto work togethereffectively

Kick-offmeeting Milestone

meeting

“MBFA”(Management by Flying Around)

High TechRequires

High Touch

But the greatest engineering achievement of the 21st Century is likely to be the reengineering of the electric grid into

secure, resilient, agile fractal patterns.

- Annonymous

But the greatest engineering achievement of the 21st Century is likely to be the reengineering of the electric grid into

secure, resilient, agile fractal patterns.

- Annonymous

(Aye, but there is always a grain of truthin every word spoken in jest.)

- William Shakespeare

Characteristics of Teams: Yesterday vs. Today

Traditional Teams Virtual Teams

Co-located members Distributed members

Face-to-face interaction Electronic communication

Members from the same Members from different

Organization organizations

Hierarchical Networked

Mostly informal communication Continuous structured

communication

Position authority Process and knowledge

authority

Informal distribution (push) Information access (pull)

Information on paper Information electronic

Sharing completed work Continuous sharing of

incomplete work

Knowledge hoarding Knowledge sharing

Transparent process Computer-visible process

Culture learned through osmosis Culture learned through

electronic-based

communications and artifacts

Merit Networks

Joe Adams

MACEMulti Agency

Collaboration Environment

Norman Eaglestone

Freescale

Mike Dow

Vybrid – first SFA compatible silicon