8/2/2019 cdna_cdc99
1/36
Context-Dependent
Network AgentsEPRI/ARO CINS Initiative
CDNA ConsortiumCMU, RPI, TAMU, Wisconsin, UIUC
8/2/2019 cdna_cdc99
2/36
The CDNA Consortium
Carnegie Mellon University
Prof. Pradeep Khosla
Prof. Bruce Krogh
Dr. Eswaran Subrahmanian
Prof. Sarosh Talukdar
Rensselaer Polytechnic Institute
Prof. Joe Chow
Texas A&M University
Prof. Garng Huang
Prof. Mladen Kezunovic
University of Illinois at Urbana-
Champaign
Prof. Lui Sha
University of Minnesota
Prof. Bruce Wollenberg
8/2/2019 cdna_cdc99
3/36
CDNA Objective
s Improve
x agilityand robustness (survivability) of large-scale dynamic
networks
x that face newand unanticipatedoperating conditions.
s Target Networks:
x U.S. Power Grid
x Local networks
8/2/2019 cdna_cdc99
4/36
CDNA Approach
s Improve
x decision-making competenceof components distributed
throughout the network,
x particularlyexisting and future control devices, such as
relays, voltage regulators and FACTS.
8/2/2019 cdna_cdc99
5/36
Why CDNA?
s centralized real-time control is
x infeasible in many situations because of the distribution of
information and growing number of independent decision
makers on the grid
x intractable - robust control algorithms simply dont scale, theproblems are NP hard
x undesirable - we contend that centralized solutions are less
robustagainst major network upsets and less adaptive to
new situations
8/2/2019 cdna_cdc99
6/36
Why CDNA? (contd.)
s control devices are already pre-programmedfor anticipated
situations
BUTone-size fits all strategies are conservative in most
cases, and wrongin some (the most critical!) situations
s necessary communication and computation technology for
CDNA exists today
8/2/2019 cdna_cdc99
7/36
Key Research Issues
s modeling
x operating modes
x contingencies
x impact of restructured power systems
x device capabilities/influence
8/2/2019 cdna_cdc99
8/36
Key Research Issues - 2
s state estimation
x using local information
x network state estimation
x real-time constraints
s hybrid control
x adaptive mode switching
x coverage
8/2/2019 cdna_cdc99
9/36
Key Research Issues - 3
s learning
x distributed learning
x state-space decomposition
s coordination
x collaboration strategies
x moving off-line techniques for asynchronous algorithms on-
line
8/2/2019 cdna_cdc99
10/36
Decentralized Large AreaPower System Control
Bruce WollenbergUniversity of Minnesota
8/2/2019 cdna_cdc99
11/36
Objectivess Research goal is to show how all standard functions built
on a power flow calculation can be accomplished without a
large area (centralized) model and computer system
s Each region of the power system retains its own control
system, models it own power network and communicates
with immediate neighborss Functions that now require central computing
x Security Analysis
x Optimal Power Flow
x
Available Transfer Capability
8/2/2019 cdna_cdc99
12/36
A
C
B
D
E
L A R G E A R E A
C O N T R O L S Y S T E M
R E G I O N A
C O N T R O L
S Y S T E M
R E G I O N C
C O N T R O L
S Y S T E M
R E G I O N B
C O N T R O L
S Y S T E M R E G I O N D
C O N T R O L
S Y S T E M
R E G I O N E
C O N T R O L
S Y S T E M
Typical PowerPool or ISO
Trends:
- Getting larger
- Standard data formats- Less functionality in
regional systemsExamples:
- California ISO- Midwest ISO
8/2/2019 cdna_cdc99
13/36
A
R E G I O N A
C O N T R O L
S Y S T E M
CR E G I O N C
C O N T R O L
S Y S T E M
B
R E G I O N B
C O N T R O L
S Y S T E M
DR E G I O N D
C O N T R O L
S Y S T E M
E
R E G I O N E
C O N T R O L
S Y S T E M
Networked ControlSystems
- Region can be any size
- Can extend to any
number of regions- Aggregate has same
functionality as large
area control system
- Can new functionality
be added that would notbe available in a central
system?
8/2/2019 cdna_cdc99
14/36
Collaborative Nets
Eduardo Camponogara and Sarosh Talukdar
Institute for Complex Engineered Systems
Carnegie Mellon University
8/2/2019 cdna_cdc99
15/36
Controlling Large Networks
Operating goals fall
into categories:
Limitations:
Control Solution:
s Costs & profits
s Safety
s Regulations
s Equipment Limits
s No organization can
cope with alloperating goals
s Need of diverse skills
s Multitudes of agents
s Delegate goals to
separate
organizations
Organization:
Agent:
A network of agents and communication links.
Any entity that makes and implements decisions
such as relays, control devices, and humans.
M l i l O i i i
8/2/2019 cdna_cdc99
16/36
Multiple Organizations inthe Power Grid
Governors, exciters
optimization soft.
Agents
Generator Control Security Systems
Relays
Protection Systems
Simulation & learn.
tools, humans
Goals Keep equipmentunder limits
Reduce costs.t. constraints
Prevent cascadingfailures
Reaction
Time
0.01 to 0.1secs Seconds Hours, days
Low Agent Skills High
Large Number of Agents Small
Fast Agent Speed Slow
O i i D N
8/2/2019 cdna_cdc99
17/36
Organizations Do NotCollaborate
Generator Control Security SystemsProtection Systems
Current Scenario: s Agents in separate organizations do not talk
s
Agents might work at cross-purposes Organizations might interfere with one another
How do we make individual agents more effective?
How do we prevent interference between organizations?
I i O ll
8/2/2019 cdna_cdc99
18/36
Improving OverallPerformance of NetsThe suggested answer is based on:
Generator Control Security SystemsProtection Systems
1.) The use of a common framework to specify agent tasks.
2.) The implementation of a sparse, collaborative net that can cut
across the hierarchic organizations.
3.) The design of collaboration protocols to promote effective
exchange of information.
C-NetC-Net
8/2/2019 cdna_cdc99
19/36
What Is A Collaborative Net?
A flat organization of dissimilar agents that
can integrate hierarchic organization.
Properties:
s Agents are autonomous within the C-Net. They
have initiative, make and implement decisions.
s Agents collaborate with their neighbors.
The collaboration protocol determines:
x what information is exchanged,
x in which way, and
x how agents make use of it.
Advantages: Disadvantages:
s Quick
s Fault Tolerant
s Open
s No structural coordination. if necessary, it can
emerge from the collaboration protocol.
s Unfamiliar.
Th R lli H i
8/2/2019 cdna_cdc99
20/36
The Rolling HorizonFormulation
A framework to solve dynamic control problems
as a series of static optimization problems.
The dynamic control problem
The steps of the rolling horizon formulation:
1.) Choose a horizon [t0,..,tN], I.E. a set of timepoints where t0 is the current time.
2.) Letx(tn) be the state predicted at time tn.
x(t0) is the current state.
3.) Let u(tn) be the planned actions at time tn.4.) Let X=[x(t0),,x(tN)] and U=[u(t0),,u(tN)]
5.) Choose a model to predictx(tn+1) fromx(tn)
and u(tn). Possibly, a discrete approximation
of the dynamic equations (e.g., Eulers
step).
Minimize f(x,dx/dt,u,t)
Subject to h(x,dx/dt,u,t)=0
g(x,dx/dt,u,t)
8/2/2019 cdna_cdc99
21/36
The Rolling HorizonAlgorithm
1.) The current time it t0.
2.) Sense the current statex(t0)
3.) Instantiate the static optimization problem (P).
4.) Solve (P) to obtain the control actions
U=[u(t0),,u(tN)].
5.) Implement the control action u(t0).
6.) Pause and let the physical network progress
in time. The horizon rolls forward.
7.) Repeat from step 1.
s A model is used to predict the future state of the physical network
over a set of discrete points in time (horizon).
s An optimization procedure computes the control actions, over the horizon,
that minimize error.
Steps of the Algorithm:
s The horizon has to be
long enough to avoid
present actions with
poor long-term effects.
s
Accuracy of theprediction model.
Design Issues:
8/2/2019 cdna_cdc99
22/36
t0 t1 t4Time
Control
t2 t3
now
The Rolling Horizon
Plan ahead
model predicted controlimplemented control
8/2/2019 cdna_cdc99
23/36
t0 t1 t4Time
Control
t2 t3
now
plans at t0
plans at t1
The Rolling Horizon
Update plans frequently
A F k f S if i
8/2/2019 cdna_cdc99
24/36
A Framework for SpecifyingAgent Tasks
Break up the static optimization problem, (P),
into
a set of M small, localized subproblems, {(Pm)}.
Assemble M agents into a C-Net, so that each
agent matches one subproblem.
Agent m and its subproblem (Pm)
It has partial perception of,
and limited authority over,
the physical network.
Neighborhood variables (ym)
Variables sensed or set by neighbors.
Proximate variables (xm,um):
It senses the values of a subsetxm ofx.
It sets the values of a subset um ofu.
Remote variables (zm):
All the other variables.
(P)
(P1) (P3)(P2) (P4)
Ag1
C-Net
M t hi A t t
8/2/2019 cdna_cdc99
25/36
Matching Agents toSubproblems
The rolling horizon
formulation of (Pm)
Minimize fm(Xm,Um,Ym,Zm)
Subject to Hm(Xm,Um,Ym,Zm) = 0
Gm(Xm,Um,Ym,Zm)
8/2/2019 cdna_cdc99
26/36
Collaboration Protocols
A protocol prescribes: a) the data exchanged by agents,
b) in which way, and
c) how agents use the data to solve their problems.
Vers
ions
Voting
Proximate Exchange
Each agent broadcasts its plans to nearby agents
which, in turn, take these plans into account.
Semi-synchronous, semi-parallel (mutual
help).
Synchronization between neighbors.
Parallel work if agents are non-neighbors.
In setting the values of its controls, each agent takes
the votes of its neighbors into account.
Asynchronous, parallel.
Two protocols
Equivalence and
8/2/2019 cdna_cdc99
27/36
Equivalence andConvergence
Two Questions:
Equivalence:
When are the solutions to the network of subproblems,{(Pm)}, solutions to (P)?
Sufficient conditions for equivalence and convergence:
The C-Net must provide complete coverage of the network.1.) Coverage:
The matching of agents to subproblems must be exact.2.) Density:
(P) must be convex.3.) Convexity:
(P) must be strictly feasible.4.) Feasibility:
The agents must use an interior-point-method.5.) Int-Pt-Mtd:
The agents run the semi-synchronous, semi-parallel protocol.6.) Serial Work:
Convergence:
When does the effort of the collaborative agents
converge to a solution of {(Pm)}?
Relaxing Sufficient
8/2/2019 cdna_cdc99
28/36
Relaxing SufficientConditions in PracticeWe believe that the following conditions can be relaxed in practice:
Near matching of agents to problems are likely to be adequate.1.) Density:
It is impractical in real-world networks.2.) Convexity:
Serial work within a neighborhood is too slow.3.) Serial work:
A prototypical network: A forest of pendulums.
- One agent at each pend.
- Agents control two forces:
Horizontal & Orthogonal.
- Agents collaborate with
nearest neighbors.
The Dynamic Control
8/2/2019 cdna_cdc99
29/36
The Dynamic ControlProblemProblem: Drive pendulums to the pre-disturbance mode, that is,
minimize cumulative error (from desired trajectory) and
total control-input cost.
dtubdtxxuxf
t
t
t
t
2
00
2~),( =
=
=
=
+=
0),,( =uxxh
Minimize
Subject to
Three ControlSolutions:
C2
C-Net
C1 A centralized, nonlinear optimization
package that solve the stat. opt. prob.
(P).
A centralized, feedback linearization
controller.
A collaborative net, with one agent at each
pendulum, that solves {(Pm)}.
C Net and C1: Experimental
8/2/2019 cdna_cdc99
30/36
C-Net and C1: ExperimentalSet-upGoal:
Scenarios:
2-Pendulum Forest
Evaluate the loss in quality of the Collaborative Net solution.
Set-up: C-Nets and C1s restore synchronous mode of pendulums.
At each sample time t,
1.) solve the network of subproblems, {(Pm)}, with the C-Net,
2.) record the obj-function evaluation of the C-Net, F(C-Net),
3.) solve the static optimization problem, (P), with C1, and
4.) record the obj-function evaluation of C1, F(C1).
Output Data: A list of obj-function-evaluation pairs [F(C-Net),F(C1)].
Place pendulums in a line to form forests of 2 to 9 pendulums.
3-Pendulum Forest
Add 1
Pend.
8/2/2019 cdna_cdc99
31/36
C-Net and C1: Results
C-Net Excess:
F(C-Net) is the obj-function evaluation attained by the C-Net.
F(C1) is the obj-function evaluation attained by controller C1.
The difference in quality between the C-Net and C1 solutions.
C-Net excess = [F(C-Net) F(C1)] / F(C1)
C-Net Penalty: The mean value of the C-Net excess.
C-NetPe
nalty(%)
Number of Pendulums
C-Net penalty is low
C Net and C2: Experimental
8/2/2019 cdna_cdc99
32/36
C-Net and C2: ExperimentalSet-upGoal:
Scenario:
Evaluate the performance of the C-Net and the feedback
linearization controller, C2, a traditional control technique.
Set-up: C-Net and C2 restore synchronous mode of pendulums.
Output Data: The cumulative error and input-cost, f(x,u), for the C-Net & C2.
A forest with 9 pendulums placed in grid.
8/2/2019 cdna_cdc99
33/36
C-Net and C2: Results
dtubdtxxuxf
t
t
t
t
2
00
2~),(
=
=
=
=+=Objective:
Control-Input
Cost (b)
Objective Function Evaluation: f(x,u)
C2 (feedback lin) C-Net
10e-4 9.56 11.89
10e-3 10.49 12.32
10e-2 17.05 16.00
10e-1 82.64 32.07
The lower the f(x,u),
the better the solution
Minimize
C-Net performance
improves
C Net and C2: Trajectory of
8/2/2019 cdna_cdc99
34/36
C-Net and C2: Trajectory ofPendulumsPendulums under control of
C2 (feedback linearization)
Pendulums under control of
the C-Net
C2 immediately drives
pendulums to the
desired trajectory.
The C-Net waits until itbecomes cheaper to
drive pendulums.
8/2/2019 cdna_cdc99
35/36
Conclusion
The experiments show that C-Nets are promising.
Current research effort:
s Development of collaboration protocols that allow agents
to work asynchronously and in parallel, at their own speed.
- Use of safety margins to guarantee feasibility, and
foster effective work between slow and fast agents.
s A taxonomy of collaboration protocols.
What else have we done?
s Employed C-Nets to recover synchronous operation of generators
in power networks IEEE-14, -30, -57.
s Preliminary work on the decomposition of (P) into {(Pm)}:
- Models and algorithms to specify neighborhood perception.
8/2/2019 cdna_cdc99
36/36
Hybrid Control Strategies
PLANT
C1
C1
Cn
M1
M1
Mn
Decision
Module
controllers
performance
monitors
u yu1
u1
un