Distributed Randomized Control for Ancillary Service to the Power Grid
-
Upload
sean-meyn -
Category
Engineering
-
view
560 -
download
2
description
Transcript of Distributed Randomized Control for Ancillary Service to the Power Grid
![Page 1: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/1.jpg)
Distributed Randomized Controlfor Ancillary Service to the Power Grid
With application to rational pools
LIDS Seminar May 6, 2014
Ana Busic and Sean Meyn
Prabir Barooah, Yue Chen, and Jordan Ehren
TREC & Departement d’Informatique
INRIA & ENSElectrical and Computer Engineering
University of Florida
Thanks to NSF, AFOSR, ANR, and DOE / TCIPG
![Page 2: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/2.jpg)
Outline
1 Distributed Control of Loads
2 Design for Intelligent Loads
3 Linearized Dynamics and Passivity
4 One Million Pools
5 Conclusions and Extensions
6 References
0 / 28
![Page 3: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/3.jpg)
Power GridControl Water PumpBatteries
CoalGas Turbine
BP
BP
BP C
BP
BP
Voltage Frequency Phase
HCΣ−
Actuator feedback loop
A
LOAD
Architecture for Distributed Control
![Page 4: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/4.jpg)
Distributed Control of Loads
Power grid as a feedback control systemMassive disturbance rejection problem
One day in Gloucester:
1 / 28
![Page 5: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/5.jpg)
Distributed Control of Loads
Power grid as a feedback control systemMassive disturbance rejection problem
Two typical weeks in the Pacific Northwest (BPA):
0
2
4
6
8
2
4
6
8
0
0.8
-0.8
1
-1
0
0.8
-0.8
1
-1
0
Sun Mon Tue
October 20-25 October 27 - November 1
Hydro
Wed Thur FriSun Mon Tue Wed Thur Fri
Gen
erat
ion
and
Laod
GW
GW
GW
Reg
ulat
ion
GW
Thermal
Wind
Load
Generation
Regulation
1 / 28
![Page 6: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/6.jpg)
Distributed Control of Loads
Control of Deferrable LoadsControl Goals and Architecture
Power GridControl Water PumpBatteries
CoalGas Turbine
BP
BP
BP C
BP
BP
Voltage Frequency Phase
HCΣ−
Actuator feedback loop
A
LOAD
Context: Consider population of similar loads that are deferrable.Two examples of flexible energy hogs:
Chillers in HVAC systems and residential pool pumps
Control Constraints:
Grid operator demands reliable ancillary service;Consumers demand reliable service (and more)
2 / 28
![Page 7: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/7.jpg)
Distributed Control of Loads
Control of Deferrable LoadsControl Goals and Architecture
Power GridControl Water PumpBatteries
CoalGas Turbine
BP
BP
BP C
BP
BP
Voltage Frequency Phase
HCΣ−
Actuator feedback loop
A
LOAD
Context: Consider population of similar loads that are deferrable.Two examples of flexible energy hogs:
Chillers in HVAC systems and residential pool pumps
Control Constraints:
Grid operator demands reliable ancillary service;Consumers demand reliable service (and more)
2 / 28
![Page 8: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/8.jpg)
Distributed Control of Loads
One Million Pools in FloridaPools Service the Grid Today
On Call1: Utility controls residential pool pumps and other loads
Contract for services: no price signals involved
Used only in times of emergency — Activated only 3-4 times a year
Opportunity:FP&L already has their hand on the switch of nearly one million pools!
Surely pools can provide much more service to the grid
Summary (beyond pools):For many household loads, contracts with consumers are already in place
For other loads, such as plug-in electric vehicles (PEVs),consumer risk may be too great.
1Florida Power and Light, Florida’s largest utility.www.fpl.com/residential/energysaving/programs/oncall.shtml
3 / 28
![Page 9: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/9.jpg)
Distributed Control of Loads
One Million Pools in FloridaPools Service the Grid Today
On Call1: Utility controls residential pool pumps and other loads
Contract for services: no price signals involved
Used only in times of emergency — Activated only 3-4 times a year
Opportunity:FP&L already has their hand on the switch of nearly one million pools!
Surely pools can provide much more service to the grid
Summary (beyond pools):For many household loads, contracts with consumers are already in place
For other loads, such as plug-in electric vehicles (PEVs),consumer risk may be too great.
1Florida Power and Light, Florida’s largest utility.www.fpl.com/residential/energysaving/programs/oncall.shtml
3 / 28
![Page 10: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/10.jpg)
Distributed Control of Loads
One Million Pools in FloridaPools Service the Grid Today
On Call1: Utility controls residential pool pumps and other loads
Contract for services: no price signals involved
Used only in times of emergency — Activated only 3-4 times a year
Opportunity:FP&L already has their hand on the switch of nearly one million pools!
Surely pools can provide much more service to the grid
Summary (beyond pools):For many household loads, contracts with consumers are already in place
For other loads, such as plug-in electric vehicles (PEVs),consumer risk may be too great.
1Florida Power and Light, Florida’s largest utility.www.fpl.com/residential/energysaving/programs/oncall.shtml
3 / 28
![Page 11: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/11.jpg)
Distributed Control of Loads
One Million Pools in FloridaPools Service the Grid Today
On Call1: Utility controls residential pool pumps and other loads
Contract for services: no price signals involved
Used only in times of emergency — Activated only 3-4 times a year
Opportunity:FP&L already has their hand on the switch of nearly one million pools!
Surely pools can provide much more service to the grid
Summary (beyond pools):For many household loads, contracts with consumers are already in place
For other loads, such as plug-in electric vehicles (PEVs),consumer risk may be too great.
1Florida Power and Light, Florida’s largest utility.www.fpl.com/residential/energysaving/programs/oncall.shtml
3 / 28
![Page 12: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/12.jpg)
Distributed Control of Loads
One Million Pools in FloridaPools Service the Grid Today
On Call1: Utility controls residential pool pumps and other loads
Contract for services: no price signals involved
Used only in times of emergency — Activated only 3-4 times a year
Opportunity:FP&L already has their hand on the switch of nearly one million pools!
Surely pools can provide much more service to the grid
Summary (beyond pools):For many household loads, contracts with consumers are already in place
For other loads, such as plug-in electric vehicles (PEVs),consumer risk may be too great.
1Florida Power and Light, Florida’s largest utility.www.fpl.com/residential/energysaving/programs/oncall.shtml
3 / 28
![Page 13: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/13.jpg)
Distributed Control of Loads
One Million Pools in FloridaPools Service the Grid Today
On Call1: Utility controls residential pool pumps and other loads
Contract for services: no price signals involved
Used only in times of emergency — Activated only 3-4 times a year
Opportunity:FP&L already has their hand on the switch of nearly one million pools!
Surely pools can provide much more service to the grid
Summary (beyond pools):For many household loads, contracts with consumers are already in place
For other loads, such as plug-in electric vehicles (PEVs),consumer risk may be too great.
1Florida Power and Light, Florida’s largest utility.www.fpl.com/residential/energysaving/programs/oncall.shtml
3 / 28
![Page 14: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/14.jpg)
Distributed Control of Loads
Control of Deferrable LoadsRandomized Control Architecture
Context: Consider population of similar loads that are deferrable.
Constraints: Grid operator demands reliable ancillary service; Consumer demands
reliable service
Control strategy
Requirements:
1.
2.
4 / 28
![Page 15: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/15.jpg)
Distributed Control of Loads
Control of Deferrable LoadsRandomized Control Architecture
Context: Consider population of similar loads that are deferrable.
Constraints: Grid operator demands reliable ancillary service; Consumer demands
reliable service
Control strategy
Requirements:
1. Minimal communication: Each load should know the needs of thegrid, and the status of the service it is intended to provide.
2.
4 / 28
![Page 16: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/16.jpg)
Distributed Control of Loads
Control of Deferrable LoadsRandomized Control Architecture
Context: Consider population of similar loads that are deferrable.
Constraints: Grid operator demands reliable ancillary service; Consumer demands
reliable service
Control strategy
Requirements:
1.
2. Smooth behavior of the aggregate=⇒ Randomization
4 / 28
![Page 17: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/17.jpg)
Distributed Control of Loads
Control of Deferrable LoadsRandomized Control Architecture
Context: Consider population of similar loads that are deferrable.
Constraints: Grid operator demands reliable ancillary service; Consumer demands
reliable service
Control strategy
Requirements:
1. Minimal communication: Each load should know the needs of thegrid, and the status of the service it is intended to provide.
2. Smooth behavior of the aggregate=⇒ Randomization
Need: A practical theory for distributed control based on this architecture
For recent references see thesis of J. Mathieu
4 / 28
![Page 18: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/18.jpg)
Distributed Control of Loads
Example: One Million Pools in FloridaHow Pools Can Help Regulate The Grid
1,5KW 400V
Needs of a single pool
. Filtration system circulates and cleans: Average pool pump uses1.3kW and runs 6-12 hours per day, 7 days per week
. Pool owners are oblivious, until they see frogs and algae
. Pool owners do not trust anyone: Privacy is a big concern
Randomized control strategy is needed.
5 / 28
![Page 19: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/19.jpg)
Distributed Control of Loads
Example: One Million Pools in FloridaHow Pools Can Help Regulate The Grid
1,5KW 400V
Needs of a single pool
. Filtration system circulates and cleans: Average pool pump uses1.3kW and runs 6-12 hours per day, 7 days per week
. Pool owners are oblivious, until they see frogs and algae
. Pool owners do not trust anyone: Privacy is a big concern
Randomized control strategy is needed.
5 / 28
![Page 20: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/20.jpg)
Distributed Control of Loads
Example: One Million Pools in FloridaHow Pools Can Help Regulate The Grid
1,5KW 400V
Needs of a single pool
. Filtration system circulates and cleans: Average pool pump uses1.3kW and runs 6-12 hours per day, 7 days per week
. Pool owners are oblivious, until they see frogs and algae
. Pool owners do not trust anyone: Privacy is a big concern
Randomized control strategy is needed.
5 / 28
![Page 21: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/21.jpg)
Distributed Control of Loads
Example: One Million Pools in FloridaHow Pools Can Help Regulate The Grid
1,5KW 400V
Needs of a single pool
. Filtration system circulates and cleans: Average pool pump uses1.3kW and runs 6-12 hours per day, 7 days per week
. Pool owners are oblivious, until they see frogs and algae
. Pool owners do not trust anyone: Privacy is a big concern
Randomized control strategy is needed.
5 / 28
![Page 22: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/22.jpg)
ytControl @ Utility
Gain
One Million Pools
Disturbance to be rejected
Proportion of pools on
desiredζt
d
dtµt = µtDζt yt = 〈µt,U〉
Intelligent Appliances
![Page 23: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/23.jpg)
Distributed Control of Loads
General ModelControlled Markovian Dynamics
Assumptions
1 Continuous-time model on finite state space X
2 Controlled Markovian generator Dζ(x, y) : x, y ∈ X, ζ ∈ R.
D = D0 models nominal behavior of load.
Each load is subject to common controlled Markovian dynamics.For the ith load,
P{Xi(t+ s) = x+ | Xi(t) = x−} ≈ sDζt(x−, x+) +O(s2)
6 / 28
![Page 24: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/24.jpg)
Distributed Control of Loads
General ModelControlled Markovian Dynamics
Assumptions
1 Continuous-time model on finite state space X
2 Controlled Markovian generator Dζ(x, y) : x, y ∈ X, ζ ∈ R.D = D0 models nominal behavior of load.
Each load is subject to common controlled Markovian dynamics.For the ith load,
P{Xi(t+ s) = x+ | Xi(t) = x−} ≈ sDζt(x−, x+) +O(s2)
6 / 28
![Page 25: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/25.jpg)
Distributed Control of Loads
General ModelControlled Markovian Dynamics
Assumptions
1 Continuous-time model on finite state space X
2 Controlled Markovian generator Dζ(x, y) : x, y ∈ X, ζ ∈ R.D = D0 models nominal behavior of load.
3 Smooth and Lipschitz: For a constant b,∣∣∣ ddζDζ(x, y)
∣∣∣ ≤ b, for x, y ∈ X, ζ ∈ R.
Each load is subject to common controlled Markovian dynamics.For the ith load,
P{Xi(t+ s) = x+ | Xi(t) = x−} ≈ sDζt(x−, x+) +O(s2)
6 / 28
![Page 26: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/26.jpg)
Distributed Control of Loads
General ModelControlled Markovian Dynamics
Assumptions
1 Continuous-time model on finite state space X2 Controlled Markovian generator Dζ(x, y) : x, y ∈ X, ζ ∈ R.
D = D0 models nominal behavior of load.
3 Smooth and Lipschitz: For a constant b,∣∣∣ ddζDζ(x, y)
∣∣∣ ≤ b, for x, y ∈ X, ζ ∈ R.
4 Utility function U : X→ Rrepresents state-dependent power consumption
Each load is subject to common controlled Markovian dynamics.For the ith load,
P{Xi(t+ s) = x+ | Xi(t) = x−} ≈ sDζt(x−, x+) +O(s2)
6 / 28
![Page 27: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/27.jpg)
Distributed Control of Loads
General ModelControlled Markovian Dynamics
Assumptions
1 Continuous-time model on finite state space X
2 Controlled Markovian generator Dζ(x, y) : x, y ∈ X, ζ ∈ R.D = D0 models nominal behavior of load.
Each load is subject to common controlled Markovian dynamics.For the ith load,
P{Xi(t+ s) = x+ | Xi(t) = x−} ≈ sDζt(x−, x+) +O(s2)
6 / 28
![Page 28: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/28.jpg)
Distributed Control of Loads
General ModelMean Field Model
Aggregate model: N loads running independently,each under the command ζ.
Empirical Distributions:
µNt (x) =1N
N∑i=1
I{Xi(t) = x}, x ∈ X
7 / 28
![Page 29: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/29.jpg)
Distributed Control of Loads
General ModelMean Field Model
Aggregate model: N loads running independently,each under the command ζ.
Empirical Distributions:
µNt (x) =1N
N∑i=1
I{Xi(t) = x}, x ∈ X
Limiting model:
d
dtµt(x′) =
∑x∈X
µt(x)Dζt(x, x′), yt :=∑x
µt(x)U(x)
via Law of Large Numbers for martingales
7 / 28
![Page 30: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/30.jpg)
Distributed Control of Loads
General ModelMean Field Model
Aggregate model: N loads running independently,each under the command ζ.
Empirical Distributions:
µNt (x) =1N
N∑i=1
I{Xi(t) = x}, x ∈ X
Mean-field model:
d
dtµt = µtDζt , yt = µt(U)
ζt = ft(µ0, . . . , µt) by design
7 / 28
![Page 31: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/31.jpg)
Distributed Control of Loads
General ModelMean Field Model
Mean-field model:
d
dtµt = µtDζt , yt = µt(U)
ζt = ft(µ0, . . . , µt) by design
ytControl @ Utility
Gain
N Loads
Disturbance to be rejecteddesired
ζt
µt+1 = µtPζtyt = 〈µt,U〉
Questions: 1. How to control this complex nonlinear system? CDC 2013
2. How to design {Dζ} so that control is easy? focus here
8 / 28
![Page 32: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/32.jpg)
Distributed Control of Loads
General ModelMean Field Model
Mean-field model:
d
dtµt = µtDζt , yt = µt(U)
ζt = ft(µ0, . . . , µt) by design
ytControl @ Utility
Gain
N Loads
Disturbance to be rejecteddesired
ζt
µt+1 = µtPζtyt = 〈µt,U〉
Questions: 1. How to control this complex nonlinear system? CDC 2013
2. How to design {Dζ} so that control is easy?
focus here
8 / 28
![Page 33: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/33.jpg)
Distributed Control of Loads
General ModelMean Field Model
Mean-field model:
d
dtµt = µtDζt , yt = µt(U)
ζt = ft(µ0, . . . , µt) by design
ytControl @ Utility
Gain
N Loads
Disturbance to be rejecteddesired
ζt
µt+1 = µtPζtyt = 〈µt,U〉
Questions: 1. How to control this complex nonlinear system? CDC 2013
2. How to design {Dζ} so that control is easy? focus here
8 / 28
![Page 34: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/34.jpg)
Design
![Page 35: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/35.jpg)
Design for Intelligent Loads
Mean Field ModelDesign via “Bellman-Shannon” Optimal Control
Goal: Construct a family of generators {Dζ : ζ ∈ R}.
Explicit solution for finite T :
p∗ζ(xT0 ) ∝ exp
(ζ
∫ T
t=0U(xt) dt
)p0(xT0 )
Markovian, but not time-homogeneous.As T →∞, we obtain generator DζSimple construction via eigenvector problem:
Dζ(x, y) =v(y)v(x)
[ζU(x)− Λ +D(x, y)
]where Dv = Λv, D(x, y) = ζU(x) +D(x, y)
Extension/reinterpretation of [Todorov 2007] ⊕ Kontoyiannis & M 200X
9 / 28
![Page 36: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/36.jpg)
Design for Intelligent Loads
Mean Field ModelDesign via “Bellman-Shannon” Optimal Control
Goal: Construct a family of generators {Dζ : ζ ∈ R}.Design: Consider first the finite-horizon control problem:Choose distribution pζ to maximize
1T
(ζEpζ
[∫ T
t=0U(Xt)
]−D(pζ‖p0)
)D denotes relative entropy.
p0 denotes nominal Markovian model.
Explicit solution for finite T :
p∗ζ(xT0 ) ∝ exp
(ζ
∫ T
t=0U(xt) dt
)p0(xT0 )
Markovian, but not time-homogeneous.As T →∞, we obtain generator DζSimple construction via eigenvector problem:
Dζ(x, y) =v(y)v(x)
[ζU(x)− Λ +D(x, y)
]where Dv = Λv, D(x, y) = ζU(x) +D(x, y)
Extension/reinterpretation of [Todorov 2007] ⊕ Kontoyiannis & M 200X
9 / 28
![Page 37: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/37.jpg)
Design for Intelligent Loads
Mean Field ModelDesign via “Bellman-Shannon” Optimal Control
Goal: Construct a family of generators {Dζ : ζ ∈ R}.Design: Consider first the finite-horizon control problem:Choose distribution pζ to maximize
1T
(ζEpζ
[∫ T
t=0U(Xt)
]−D(pζ‖p0)
)D denotes relative entropy.
p0 denotes nominal Markovian model.
Explicit solution for finite T :
p∗ζ(xT0 ) ∝ exp
(ζ
∫ T
t=0U(xt) dt
)p0(xT0 )
Markovian, but not time-homogeneous.As T →∞, we obtain generator DζSimple construction via eigenvector problem:
Dζ(x, y) =v(y)v(x)
[ζU(x)− Λ +D(x, y)
]where Dv = Λv, D(x, y) = ζU(x) +D(x, y)
Extension/reinterpretation of [Todorov 2007] ⊕ Kontoyiannis & M 200X
9 / 28
![Page 38: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/38.jpg)
Design for Intelligent Loads
Mean Field ModelDesign via “Bellman-Shannon” Optimal Control
Explicit solution for finite T :
p∗ζ(xT0 ) ∝ exp
(ζ
∫ T
t=0U(xt) dt
)p0(xT0 )
Markovian, but not time-homogeneous.As T →∞, we obtain generator DζSimple construction via eigenvector problem:
Dζ(x, y) =v(y)v(x)
[ζU(x)− Λ +D(x, y)
]where Dv = Λv, D(x, y) = ζU(x) +D(x, y)
Extension/reinterpretation of [Todorov 2007] ⊕ Kontoyiannis & M 200X
9 / 28
![Page 39: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/39.jpg)
Design for Intelligent Loads
Mean Field ModelDesign via “Bellman-Shannon” Optimal Control
Explicit solution for finite T :
p∗ζ(xT0 ) ∝ exp
(ζ
∫ T
t=0U(xt) dt
)p0(xT0 )
Markovian, but not time-homogeneous.As T →∞, we obtain generator Dζ
Simple construction via eigenvector problem:
Dζ(x, y) =v(y)v(x)
[ζU(x)− Λ +D(x, y)
]where Dv = Λv, D(x, y) = ζU(x) +D(x, y)
Extension/reinterpretation of [Todorov 2007] ⊕ Kontoyiannis & M 200X
9 / 28
![Page 40: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/40.jpg)
Design for Intelligent Loads
Mean Field ModelDesign via “Bellman-Shannon” Optimal Control
Explicit solution for finite T :
p∗ζ(xT0 ) ∝ exp
(ζ
∫ T
t=0U(xt) dt
)p0(xT0 )
Markovian, but not time-homogeneous.
As T →∞, we obtain generator DζSimple construction via eigenvector problem:
Dζ(x, y) =v(y)v(x)
[ζU(x)− Λ +D(x, y)
]where Dv = Λv, D(x, y) = ζU(x) +D(x, y)
Extension/reinterpretation of [Todorov 2007] ⊕ Kontoyiannis & M 200X
9 / 28
![Page 41: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/41.jpg)
Design for Intelligent Loads
Mean Field ModelDesign via “Bellman-Shannon” Optimal Control
Explicit solution for finite T :
p∗ζ(xT0 ) ∝ exp
(ζ
∫ T
t=0U(xt) dt
)p0(xT0 )
Markovian, but not time-homogeneous.
As T →∞, we obtain generator DζSimple construction via eigenvector problem:
Dζ(x, y) =v(y)v(x)
[ζU(x)− Λ +D(x, y)
]
where Dv = Λv, D(x, y) = ζU(x) +D(x, y)
Extension/reinterpretation of [Todorov 2007] ⊕ Kontoyiannis & M 200X
9 / 28
![Page 42: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/42.jpg)
Design for Intelligent Loads
Mean Field ModelDesign via “Bellman-Shannon” Optimal Control
Explicit solution for finite T :
p∗ζ(xT0 ) ∝ exp
(ζ
∫ T
t=0U(xt) dt
)p0(xT0 )
Markovian, but not time-homogeneous.
As T →∞, we obtain generator DζSimple construction via eigenvector problem:
Dζ(x, y) =v(y)v(x)
[ζU(x)− Λ +D(x, y)
]where Dv = Λv, D(x, y) = ζU(x) +D(x, y)
Extension/reinterpretation of [Todorov 2007] ⊕ Kontoyiannis & M 200X
9 / 28
![Page 43: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/43.jpg)
d
dtµt = µtDζt
yt = 〈µt,U〉
d
dtΦt = AΦt + Bζt
γt = CΦt
Linearized Dynamics
![Page 44: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/44.jpg)
Linearized Dynamics and Passivity
Mean Field ModelLinearized Dynamics
Mean-field model: ddtµt = µtDζt , yt = µt(U)
Linear state space model:Φt+1 = AΦt +Bζt
γt = CΦt
Interpretations: |ζt| is small, and π denotes invariant measure for D.
• Φt ∈ R|X|, a column vector withΦt(x) ≈ µt(x)− π(x), x ∈ X
• γt ≈ yt − y0; deviation from nominal steady-state
• A = DT, Ci = U(xi), and input dynamics linearized:
BT =d
dζπDζ
∣∣∣ζ=0
10 / 28
![Page 45: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/45.jpg)
Linearized Dynamics and Passivity
Mean Field ModelLinearized Dynamics
Mean-field model: ddtµt = µtDζt , yt = µt(U)
Linear state space model:Φt+1 = AΦt +Bζt
γt = CΦt
Interpretations: |ζt| is small, and π denotes invariant measure for D.
• Φt ∈ R|X|, a column vector withΦt(x) ≈ µt(x)− π(x), x ∈ X
• γt ≈ yt − y0; deviation from nominal steady-state
• A = DT, Ci = U(xi), and input dynamics linearized:
BT =d
dζπDζ
∣∣∣ζ=0
10 / 28
![Page 46: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/46.jpg)
Linearized Dynamics and Passivity
Mean Field ModelLinearized Dynamics
Mean-field model: ddtµt = µtDζt , yt = µt(U)
Linear state space model:Φt+1 = AΦt +Bζt
γt = CΦt
Interpretations: |ζt| is small, and π denotes invariant measure for D.
• Φt ∈ R|X|, a column vector withΦt(x) ≈ µt(x)− π(x), x ∈ X
• γt ≈ yt − y0; deviation from nominal steady-state
• A = DT, Ci = U(xi), and input dynamics linearized:
BT =d
dζπDζ
∣∣∣ζ=0
10 / 28
![Page 47: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/47.jpg)
Linearized Dynamics and Passivity
Mean Field ModelLinearized Dynamics
Mean-field model: ddtµt = µtDζt , yt = µt(U)
Linear state space model:Φt+1 = AΦt +Bζt
γt = CΦt
Interpretations: |ζt| is small, and π denotes invariant measure for D.
• Φt ∈ R|X|, a column vector withΦt(x) ≈ µt(x)− π(x), x ∈ X
• γt ≈ yt − y0; deviation from nominal steady-state
• A = DT, Ci = U(xi), and input dynamics linearized:
BT =d
dζπDζ
∣∣∣ζ=0
10 / 28
![Page 48: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/48.jpg)
Linearized Dynamics and Passivity
Mean Field ModelLinearized Dynamics
Mean-field model: ddtµt = µtDζt , yt = µt(U)
Linear state space model:Φt+1 = AΦt +Bζt
γt = CΦt
Interpretations: |ζt| is small, and π denotes invariant measure for D.
• Φt ∈ R|X|, a column vector withΦt(x) ≈ µt(x)− π(x), x ∈ X
• γt ≈ yt − y0; deviation from nominal steady-state
• A = DT, Ci = U(xi), and input dynamics linearized:
BT =d
dζπDζ
∣∣∣ζ=0
10 / 28
![Page 49: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/49.jpg)
Linearized Dynamics and Passivity
Mean Field ModelLinearized Dynamics
Mean-field model: ddtµt = µtDζt , yt = µt(U)
Linear state space model:Φt+1 = AΦt +Bζt
γt = CΦt
Interpretations: |ζt| is small, and π denotes invariant measure for D.
• Φt ∈ R|X|, a column vector withΦt(x) ≈ µt(x)− π(x), x ∈ X
• γt ≈ yt − y0; deviation from nominal steady-state
• A = DT, Ci = U(xi), and input dynamics linearized:
BT =d
dζπDζ
∣∣∣ζ=0
10 / 28
![Page 50: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/50.jpg)
Linearized Dynamics and Passivity
Linearized DynamicsTransfer Function
Linear state space model:
Φt+1 = AΦt +Bζt
γt = CΦt A = DT, Ci = U(xi), BT =d
dζπDζ
∣∣∣ζ=0
Transfer Function:
G(s) = C[Is−A]−1B = C[Is−DT]−1B
hmmmm...
= CRTsB TF for L-MFM � Resolvent for one load
Resolvent Matrix:
Rs =∫ ∞
0e−stetD dt = [Is−D]−1
11 / 28
![Page 51: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/51.jpg)
Linearized Dynamics and Passivity
Linearized DynamicsTransfer Function
Linear state space model:
Φt+1 = AΦt +Bζt
γt = CΦt A = DT, Ci = U(xi), BT =d
dζπDζ
∣∣∣ζ=0
Transfer Function:
G(s) = C[Is−A]−1B = C[Is−DT]−1B hmmmm...
= CRTsB TF for L-MFM � Resolvent for one load
Resolvent Matrix:
Rs =∫ ∞
0e−stetD dt = [Is−D]−1
11 / 28
![Page 52: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/52.jpg)
Linearized Dynamics and Passivity
Linearized DynamicsTransfer Function
Linear state space model:
Φt+1 = AΦt +Bζt
γt = CΦt A = DT, Ci = U(xi), BT =d
dζπDζ
∣∣∣ζ=0
Transfer Function:
G(s) = C[Is−A]−1B = C[Is−DT]−1B hmmmm...
= CRTsB TF for L-MFM � Resolvent for one load
Resolvent Matrix:
Rs =∫ ∞
0e−stetD dt = [Is−D]−1
11 / 28
![Page 53: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/53.jpg)
Linearized Dynamics and Passivity
Linearized DynamicsTransfer Function
Linear state space model:
Φt+1 = AΦt +Bζt
γt = CΦt A = DT, Ci = U(xi), BT =d
dζπDζ
∣∣∣ζ=0
Transfer Function:
G(s) = C[Is−A]−1B = C[Is−DT]−1B hmmmm...
= CRTsB TF for L-MFM � Resolvent for one load
Resolvent Matrix:
Rs =∫ ∞
0e−stetD dt = [Is−D]−1
11 / 28
![Page 54: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/54.jpg)
Linearized Dynamics and Passivity
Linearized DynamicsPassive Pools
Theorem: Reversibility =⇒ Passivity
Suppose that the nominal model is reversible.Then its linearization satisfies,
ReG(jω) = PSDY (ω), ω ∈ R ,
whereG(s) = C[Is−A]−1B for s ∈ C.
PSDY (ω) =∫ ∞−∞
e−jωEπ[U(X0)U(Xt)] dt
Implication for control: G(s) is positive real
12 / 28
![Page 55: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/55.jpg)
Linearized Dynamics and Passivity
Linearized DynamicsPassive Pools
Theorem: Reversibility =⇒ Passivity
Suppose that the nominal model is reversible.Then its linearization satisfies,
ReG(jω) = PSDY (ω), ω ∈ R ,
whereG(s) = C[Is−A]−1B for s ∈ C.
PSDY (ω) =∫ ∞−∞
e−jωEπ[U(X0)U(Xt)] dt
Implication for control: G(s) is positive real
12 / 28
![Page 56: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/56.jpg)
Linearized Dynamics and Passivity
Linearized DynamicsPassive Pools
Theorem: Reversibility =⇒ Passivity
Suppose that the nominal model is reversible.Then its linearization satisfies,
ReG(jω) = PSDY (ω), ω ∈ R ,
whereG(s) = C[Is−A]−1B for s ∈ C.
PSDY (ω) =∫ ∞−∞
e−jωEπ[U(X0)U(Xt)] dt
Implication for control: G(s) is positive real
12 / 28
![Page 57: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/57.jpg)
Linearized Dynamics and Passivity
Linearized DynamicsExample Without Passivity
Example: Eight state model Utility function U(xi) = i.
1 2
3 4
5 6
7 8
a c
c
a
a a
a a
a a
b b b b
Not reversible
13 / 28
![Page 58: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/58.jpg)
Linearized Dynamics and Passivity
Linearized DynamicsExample Without Passivity
Example: Eight state model Utility function U(xi) = i.Generator is the 8× 8 matrix,
D =
−a a 0 0 0 0 0 0a −(a+ b+ c) 0 b c 0 0 0b 0 −(a+ b) a 0 0 0 00 0 a −a 0 0 0 00 0 0 0 −a a 0 00 0 0 0 a −(a+ b) 0 b0 0 0 c b 0 −(a+ b+ c) a0 0 0 0 0 0 a −a
13 / 28
![Page 59: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/59.jpg)
Linearized Dynamics and Passivity
Linearized DynamicsExample Without Passivity
1 2
3 4
5 6
7 8
a c
c
a
a a
a a
a a
b b b b
Example: Eight state model a = c = 10, b = 1
−2
−1
0
1
2
Real Axis−1 0 1 2
Imag
inar
y A
xis
−25 −20 −15 −10 −5 0 5 10−15
−10
−5
0
5
10
15
Real Part
Imag
inar
y Pa
rt
Pole-zero Plot
G(jω), ω >0
Non-minimum phase zero
Nyquist Plot
G(s) = C[Is−A]−1B not positive real13 / 28
![Page 60: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/60.jpg)
ytControl @ Utility
Gain
One Million Pools
Disturbance to be rejected
Proportion of pools on
desiredζt
µt+1 = µtPζtyt = 〈µt,U〉
One Million Pools
![Page 61: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/61.jpg)
One Million Pools
A Single PoolControl Architecture
Transition diagram for a single pool:
1 2 T−1 T. . .
T
On
O�
12T−1
...
Numerics using discrete-time model
1 2 T−1 T. . .
T
On
O�
12T−1
...
Utility:
U(x) = I{
Pool is on}∣∣∣x∝ (Power consumption)
∣∣∣x
Controlled dynamics: As ζ increases, probability of turning on increases:
0 24120
0.5
1
zζ=
- 4
ζ =
-2
ζ = 4
ζ = 2
ζ = 0
(ζ)
14 / 28
![Page 62: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/62.jpg)
One Million Pools
A Single PoolControl Architecture
1 2 T−1 T. . .
T
On
O�
12T−1
...
Utility:
U(x) = I{
Pool is on}∣∣∣x∝ (Power consumption)
∣∣∣x
Controlled dynamics: As ζ increases, probability of turning on increases:
0 24120
0.5
1
zζ=
- 4
ζ =
-2
ζ = 4
ζ = 2
ζ = 0
(ζ)
14 / 28
![Page 63: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/63.jpg)
One Million Pools
A Single PoolControl Architecture
1 2 T−1 T. . .
T
On
O�
12T−1
...
Utility:
U(x) = I{
Pool is on}∣∣∣x∝ (Power consumption)
∣∣∣x
Controlled dynamics: As ζ increases, probability of turning on increases:
0 24120
0.5
1
zζ=
- 4
ζ =
-2
ζ = 4
ζ = 2
ζ = 0
(ζ)
14 / 28
![Page 64: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/64.jpg)
One Million Pools
Mean Field Pool ModelLinearization: Minimum Phase Pole-Zero Plot
−1 0 1
−1
0
1
Real PartIm
agin
ary
Part
Transfer function
Normalized Frequency ( ×π rad/sample)1/24 hrs0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
-180
0
20
Phas
e (d
egre
es)
Mag
nitu
de (d
B)
40
-90
0
15 / 28
![Page 65: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/65.jpg)
One Million Pools
Mean Field Pool ModelStochastic simulation — Filtered regulation signal
−400
−200
0
200
400
MW
July 18 July 20July 19 July 21 July 23July 22 July 24
BPA Balancing Reserves Deployed (July 2013)
−400
−200
0
200
400
MW
July 18 July 20July 19 July 21 July 23July 22 July 24
Filtered BPA reserve signal that can be tracked by pools BPA Balancing Reserves Deployed (July 2013)
∗transmission.bpa.gov/Business/Operations/Wind/reserves.aspx
16 / 28
![Page 66: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/66.jpg)
One Million Pools
Mean Field Pool ModelStochastic simulation — Filtered regulation signal
−400
−200
0
200
400
MW
July 18 July 20July 19 July 21 July 23July 22 July 24
BPA Balancing Reserves Deployed (July 2013)
−400
−200
0
200
400
MW
July 18 July 20July 19 July 21 July 23July 22 July 24
Filtered BPA reserve signal that can be tracked by pools BPA Balancing Reserves Deployed (July 2013)
∗transmission.bpa.gov/Business/Operations/Wind/reserves.aspx
16 / 28
![Page 67: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/67.jpg)
One Million Pools
Mean Field Pool ModelStochastic simulation using N = 105 pools PI control
Reference (from Bonneville Power Authority)
−300
−200
−100
0
100
200
300
Trac
king
BPA
Reg
ulat
ion
Sign
al
(MW
)
Reference (from Bonneville Power Authority)Output deviation y
−300
−200
−100
0
100
200
300
Trac
king
BPA
Reg
ulat
ion
Sign
al
(MW
)
Reference (from Bonneville Power Authority)Output deviation y
−300
−200
−100
0
100
200
300
-2
-1
-3
2
3
0
1
Input ζ
ζ
Trac
king
BPA
Reg
ulat
ion
Sign
al
(MW
)
17 / 28
![Page 68: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/68.jpg)
One Million Pools
Mean Field Pool ModelStochastic simulation using N = 105 pools PI control
Reference (from Bonneville Power Authority)
−300
−200
−100
0
100
200
300
Trac
king
BPA
Reg
ulat
ion
Sign
al
(MW
)
Reference (from Bonneville Power Authority)Output deviation y
−300
−200
−100
0
100
200
300
Trac
king
BPA
Reg
ulat
ion
Sign
al
(MW
)
Reference (from Bonneville Power Authority)Output deviation y
−300
−200
−100
0
100
200
300
-2
-1
-3
2
3
0
1
Input ζ
ζ
Trac
king
BPA
Reg
ulat
ion
Sign
al
(MW
)
17 / 28
![Page 69: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/69.jpg)
One Million Pools
Mean Field Pool ModelStochastic simulation using N = 105 pools PI control
Reference (from Bonneville Power Authority)
−300
−200
−100
0
100
200
300
Trac
king
BPA
Reg
ulat
ion
Sign
al
(MW
)
Reference (from Bonneville Power Authority)Output deviation y
−300
−200
−100
0
100
200
300
Trac
king
BPA
Reg
ulat
ion
Sign
al
(MW
)
Reference (from Bonneville Power Authority)Output deviation y
−300
−200
−100
0
100
200
300
-2
-1
-3
2
3
0
1
Input ζ
ζ
Trac
king
BPA
Reg
ulat
ion
Sign
al
(MW
)
17 / 28
![Page 70: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/70.jpg)
One Million Pools
Mean Field Pool ModelStochastic simulation using N = 105 pools PI control
Two scenarios, using two different reference signals:
0 20 40 60 80 100 120 140 160t/hour
0 20 40 60 80 100 120 140 160 0 20 40 60 80 100 120 140 160t/hour
0 20 40 60 80 100 120 140 160
Refe
renc
eO
utpu
t dev
iatio
n (M
W)
Refe
renc
eO
utpu
t dev
iatio
n (M
W)
Refe
renc
eO
utpu
t dev
iatio
n (M
W)
Refe
renc
eO
utpu
t dev
iatio
n (M
W)
−300
−200
−100
0
100
200
300
−300
−200
−100
0
100
200
300
−600
−400
−200
0
200
400
600
−400
−200
0
200
400
600
800
-2
-1
-3
2
3
0
1
−3
−2
−1
0
1
2
3
−5
0
5
−4
−2
0
2
4
6
Inpu
t
Inpu
t
Inpu
t
Inpu
t
12 Hour Cleaning Cycle 8 Hour Cleaning Cycle
Trac
king
BPA
Reg
ulat
ion
Sign
alBP
A R
egul
atio
n Si
gnal
- Dou
bled
(a) (b)
(c) (d)
18 / 28
![Page 71: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/71.jpg)
One Million Pools
Mean Field Pool ModelStochastic simulation using N = 105 pools PI control
The impact of exceeding capacity
0 20 40 60 80 100 120 140 160
0 20 40 60 80 100 120 140 160
Refe
renc
eO
utpu
t dev
iatio
n (M
W)
Refe
renc
eO
utpu
t dev
iatio
n (M
W)
Refe
renc
eO
utpu
t dev
iatio
n (M
W)
Refe
renc
eO
utpu
t dev
iatio
n (M
W)
0 20 40 60 80 100 120 140 160t/hour
0 20 40 60 80 100 120 140 160
−1000
−500
0
500
1000
−1000
−500
0
500
1000
−1000
−500
0
500
1000
−1000
−500
0
500
1000
−20
−10
0
10
20
−20
−10
0
10
20
−10
0
10
−10
0
10
Inpu
t
Inpu
t
Inpu
t
Inpu
t
12 Hour Cleaning Cycle 8 Hour Cleaning Cycle
Regu
latio
n ex
ceed
s ca
paci
tyIn
tegr
ator
win
d-up
Rec
over
y
(a) (b)
(c) (d)
19 / 28
![Page 72: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/72.jpg)
Power GridControl Water PumpBatteries
CoalGas Turbine
BP
BP
BP C
BP
BP
Voltage Frequency Phase
HCΣ−
Actuator feedback loop
A
LOAD
Conclusions and Extensions
![Page 73: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/73.jpg)
Conclusions and Extensions
ConclusionsRecap
QUIZ: Why intelligence at the loads?
To simplify control at the grid level
A particular approach to distributed control is proposed
The grid level control problem is simple because:
? Mean field model is simple, and a good approximation of finite system
? LTI approximation is positive real, which implies minimum phase
20 / 28
![Page 74: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/74.jpg)
Conclusions and Extensions
ConclusionsRecap
QUIZ: Why intelligence at the loads?To simplify control at the grid level
A particular approach to distributed control is proposed
The grid level control problem is simple because:
? Mean field model is simple, and a good approximation of finite system
? LTI approximation is positive real, which implies minimum phase
20 / 28
![Page 75: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/75.jpg)
Conclusions and Extensions
ConclusionsRecap
QUIZ: Why intelligence at the loads?To simplify control at the grid level
A particular approach to distributed control is proposed
The grid level control problem is simple because:
? Mean field model is simple, and a good approximation of finite system
? LTI approximation is positive real, which implies minimum phase
20 / 28
![Page 76: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/76.jpg)
Conclusions and Extensions
ConclusionsRecap
QUIZ: Why intelligence at the loads?To simplify control at the grid level
A particular approach to distributed control is proposed
The grid level control problem is simple because:
? Mean field model is simple, and a good approximation of finite system
? LTI approximation is positive real, which implies minimum phase
20 / 28
![Page 77: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/77.jpg)
Conclusions and Extensions
ConclusionsRecap
QUIZ: Why intelligence at the loads?To simplify control at the grid level
A particular approach to distributed control is proposed
The grid level control problem is simple because:
? Mean field model is simple, and a good approximation of finite system
? LTI approximation is positive real, which implies minimum phase
20 / 28
![Page 78: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/78.jpg)
Conclusions and Extensions
ConclusionsAre the customers happy?
No time for details, but no ...
There will be “rare events” in which the pool is under- or over-cleaned.
Proposed approach: Additional layer of control at each load, so that hardconstraints on performance can be assured.
This will create some modeling error at grid level.
Preliminary experiments on the pool model:No loss of performance at grid level.
21 / 28
![Page 79: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/79.jpg)
Conclusions and Extensions
ConclusionsAre the customers happy?
No time for details, but no ...
QoS without local control−100 −80 −60 −40 −20 0 20 40 60 80 1000
0.01
0.02
0.03
0.04
0.05
fract
ion
of p
ools
Histogram of pool healthGaussian distribution
There will be “rare events” in which the pool is under- or over-cleaned.
Proposed approach: Additional layer of control at each load, so that hardconstraints on performance can be assured.
This will create some modeling error at grid level.
Preliminary experiments on the pool model:No loss of performance at grid level.
21 / 28
![Page 80: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/80.jpg)
Conclusions and Extensions
ConclusionsAre the customers happy?
No time for details, but no ...
QoS without local control−100 −80 −60 −40 −20 0 20 40 60 80 1000
0.01
0.02
0.03
0.04
0.05
fra
ctio
n of
poo
ls
Histogram of pool healthGaussian distribution
There will be “rare events” in which the pool is under- or over-cleaned.
Proposed approach: Additional layer of control at each load, so that hardconstraints on performance can be assured.
This will create some modeling error at grid level.
Preliminary experiments on the pool model:No loss of performance at grid level.
21 / 28
![Page 81: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/81.jpg)
Conclusions and Extensions
ConclusionsAre the customers happy?
No time for details, but no ...
QoS without local control−100 −80 −60 −40 −20 0 20 40 60 80 1000
0.01
0.02
0.03
0.04
0.05
fra
ctio
n of
poo
ls
Histogram of pool healthGaussian distribution
There will be “rare events” in which the pool is under- or over-cleaned.
Proposed approach: Additional layer of control at each load, so that hardconstraints on performance can be assured.
This will create some modeling error at grid level.
Preliminary experiments on the pool model:No loss of performance at grid level.
21 / 28
![Page 82: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/82.jpg)
Conclusions and Extensions
ConclusionsAre the customers happy?
No time for details, but no ...
QoS without local control−100 −80 −60 −40 −20 0 20 40 60 80 1000
0.01
0.02
0.03
0.04
0.05
fra
ctio
n of
poo
ls
Histogram of pool healthGaussian distribution
There will be “rare events” in which the pool is under- or over-cleaned.
Proposed approach: Additional layer of control at each load, so that hardconstraints on performance can be assured.
This will create some modeling error at grid level.
Preliminary experiments on the pool model:No loss of performance at grid level.
21 / 28
![Page 83: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/83.jpg)
Conclusions and Extensions
ConclusionsThe customers are happy
Proposed approach: Additional layer of control at each load, so that hardconstraints on performance can be assured.
QoS without local control−100 −80 −60 −40 −20 0 20 40 60 80 1000
0.01
0.02
0.03
0.04
0.05
fract
ion
of p
ools
Histogram of pool healthGaussian distribution
QoS without local control QoS with local control−100 −80 −60 −40 −20 0 20 40 60 80 1000
0.01
0.02
0.03
0.04
0.05
fract
ion
of p
ools
Histogram of pool healthConditional Gaussian distribution
Histogram of pool healthGaussian distribution
−100 −80 −60 −40 −20 0 20 40 60 80 100
Load opts-out when its QoS is outside of prescribed bounds
Analysis: LQG approximation for individual load – CDC 2014
22 / 28
![Page 84: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/84.jpg)
Conclusions and Extensions
ConclusionsThe customers are happy
Proposed approach: Additional layer of control at each load, so that hardconstraints on performance can be assured.
QoS without local control−100 −80 −60 −40 −20 0 20 40 60 80 1000
0.01
0.02
0.03
0.04
0.05
fract
ion
of p
ools
Histogram of pool healthGaussian distribution
QoS without local control QoS with local control−100 −80 −60 −40 −20 0 20 40 60 80 1000
0.01
0.02
0.03
0.04
0.05
fract
ion
of p
ools
Histogram of pool healthConditional Gaussian distribution
Histogram of pool healthGaussian distribution
−100 −80 −60 −40 −20 0 20 40 60 80 100
Load opts-out when its QoS is outside of prescribed bounds
Analysis: LQG approximation for individual load – CDC 2014
22 / 28
![Page 85: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/85.jpg)
Conclusions and Extensions
ConclusionsMany issues skipped because they are topics of current research
1 Information at the load: Should these loads simply act as frequencyregulators? Measurement of local frequency deviation will suffice.
Or, more information may be valuable: Two measurements at eachload, the BA command, and local frequency measurements.
2 Information at the BA: Does this macro control view suffice?
Power Grid
Actuation
Control
Water Pump
Batteries
HVAC
Coal
Gas Turbine
BP
BP
BP
BP
BP
Baseline Generation
Disturbancesfrom nature
Measurements: Voltage Frequency Phase
HCΣ−
Does the grid operator need to know the real-time powerconsumption of each population of loads?
23 / 28
![Page 86: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/86.jpg)
Conclusions and Extensions
ConclusionsMany issues skipped because they are topics of current research
1 Information at the load: Should these loads simply act as frequencyregulators? Measurement of local frequency deviation will suffice.
Or, more information may be valuable: Two measurements at eachload, the BA command, and local frequency measurements.
2 Information at the BA: Does this macro control view suffice?
Power Grid
Actuation
Control
Water Pump
Batteries
HVAC
Coal
Gas Turbine
BP
BP
BP
BP
BP
Baseline Generation
Disturbancesfrom nature
Measurements: Voltage Frequency Phase
HCΣ−
Does the grid operator need to know the real-time powerconsumption of each population of loads?
23 / 28
![Page 87: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/87.jpg)
Conclusions and Extensions
ConclusionsMany issues skipped because they are topics of current research
1 Information at the load: Should these loads simply act as frequencyregulators? Measurement of local frequency deviation will suffice.
Or, more information may be valuable: Two measurements at eachload, the BA command, and local frequency measurements.
2 Information at the BA: Does this macro control view suffice?
Power Grid
Actuation
Control
Water Pump
Batteries
HVAC
Coal
Gas Turbine
BP
BP
BP
BP
BP
Baseline Generation
Disturbancesfrom nature
Measurements: Voltage Frequency Phase
HCΣ−
Does the grid operator need to know the real-time powerconsumption of each population of loads?
23 / 28
![Page 88: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/88.jpg)
Conclusions and Extensions
ConclusionsMany issues skipped because they are topics of current research
1 Information at the load: Should these loads simply act as frequencyregulators? Measurement of local frequency deviation will suffice.
Or, more information may be valuable: Two measurements at eachload, the BA command, and local frequency measurements.
2 Information at the BA: Does this macro control view suffice?
Power Grid
Actuation
Control
Water Pump
Batteries
HVAC
Coal
Gas Turbine
BP
BP
BP
BP
BP
Baseline Generation
Disturbancesfrom nature
Measurements: Voltage Frequency Phase
HCΣ−
Does the grid operator need to know the real-time powerconsumption of each population of loads?
23 / 28
![Page 89: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/89.jpg)
Conclusions and Extensions
ConclusionsMany issues skipped because they are topics of current research
3 How can we engage consumers?
The formulation of contracts with customers requires a betterunderstanding of the value of ancillary service, as well as consumerpreferences.
We will not use real timeprices, even via aggregator,if we want responsive, reliableancillary service
Refe
renc
eO
utpu
t dev
iatio
n (M
W)
−300
−200
−100
0
100
200
300
-2
-1
-3
2
3
0
1
Inpu
t
Trac
king
BPA
Reg
ulat
ion
Sign
al
(a)
24 / 28
![Page 90: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/90.jpg)
Conclusions and Extensions
ConclusionsMany issues skipped because they are topics of current research
3 How can we engage consumers?
The formulation of contracts with customers requires a betterunderstanding of the value of ancillary service, as well as consumerpreferences.
We will not use real timeprices, even via aggregator,if we want responsive, reliableancillary service
Refe
renc
eO
utpu
t dev
iatio
n (M
W)
−300
−200
−100
0
100
200
300
-2
-1
-3
2
3
0
1
Inpu
t
Trac
king
BPA
Reg
ulat
ion
Sign
al
(a)
24 / 28
![Page 91: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/91.jpg)
Conclusions and Extensions
ConclusionsMany issues skipped because they are topics of current research
3 How can we engage consumers?
The formulation of contracts with customers requires a betterunderstanding of the value of ancillary service, as well as consumerpreferences.
We will not use real timeprices, even via aggregator,if we want responsive, reliableancillary service
Refe
renc
eO
utpu
t dev
iatio
n (M
W)
−300
−200
−100
0
100
200
300
-2
-1
-3
2
3
0
1
Inpu
t
Trac
king
BPA
Reg
ulat
ion
Sign
al
(a)
24 / 28
![Page 92: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/92.jpg)
Conclusions and Extensions
Conclusions
Thank You!
25 / 28
![Page 93: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/93.jpg)
Control TechniquesFOR
Complex Networks
Sean Meyn
Pre-publication version for on-line viewing. Monograph available for purchase at your favorite retailer More information available at http://www.cambridge.org/us/catalogue/catalogue.asp?isbn=9780521884419
Markov Chainsand
Stochastic Stability
S. P. Meyn and R. L. Tweedie
August 2008 Pre-publication version for on-line viewing. Monograph to appear Februrary 2009
π(f
)<∞
∆V (x) ≤ −f(x) + bIC(x)
‖P n(x, · )− π‖f → 0
supCE
x [Sτ
C(f)]
<∞
References
![Page 94: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/94.jpg)
References
References: Demand Response
S. Meyn, P. Barooah, A. Busic, and J. Ehren. Ancillary service to the grid from deferrableloads: the case for intelligent pool pumps in Florida (Invited). In Proceedings of the 52ndIEEE Conf. on Decision and Control, 2013.
A. Busic and S. Meyn. Passive dynamics in mean field control. ArXiv e-prints:arXiv:1402.4618. Submitted to the 53rd IEEE Conf. on Decision and Control (Invited),2014.
S. Meyn, Y. Chen, and A. Busic. Individual risk in mean-field control models fordecentralized control, with application to automated demand response. Submitted to the53rd IEEE Conf. on Decision and Control (Invited), 2014.
J. L. Mathieu. Modeling, Analysis, and Control of Demand Response Resources. PhDthesis, Berkeley, 2012.
D. Callaway and I. Hiskens, Achieving controllability of electric loads. Proceedings of theIEEE, 99(1):184–199, 2011.
S. Koch, J. Mathieu, and D. Callaway, Modeling and control of aggregated heterogeneousthermostatically controlled loads for ancillary services, in Proc. PSCC, 2011, 1–7.
H. Hao, A. Kowli, Y. Lin, P. Barooah, and S. Meyn Ancillary Service for the Grid ViaControl of Commercial Building HVAC Systems. ACC 2013
(much more on our websites)
26 / 28
![Page 95: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/95.jpg)
References
References: Markov Models
I. Kontoyiannis and S. P. Meyn. Spectral theory and limit theorems for geometricallyergodic Markov processes. Ann. Appl. Probab., 13:304–362, 2003.
I. Kontoyiannis and S. P. Meyn. Large deviations asymptotics and the spectral theory ofmultiplicatively regular Markov processes. Electron. J. Probab., 10(3):61–123 (electronic),2005.
E. Todorov. Linearly-solvable Markov decision problems. In B. Scholkopf, J. Platt, andT. Hoffman, editors, Advances in Neural Information Processing Systems, (19) 1369–1376.MIT Press, Cambridge, MA, 2007.
M. Huang, P. E. Caines, and R. P. Malhame. Large-population cost-coupled LQG problemswith nonuniform agents: Individual-mass behavior and decentralized ε-Nash equilibria.IEEE Trans. Automat. Control, 52(9):1560–1571, 2007.
H. Yin, P. Mehta, S. Meyn, and U. Shanbhag. Synchronization of coupled oscillators is agame. IEEE Transactions on Automatic Control, 57(4):920–935, 2012.
P. Guan, M. Raginsky, and R. Willett. Online Markov decision processes withKullback-Leibler control cost. In American Control Conference (ACC), 2012, 1388–1393,2012.
V.S.Borkar and R.Sundaresan Asympotics of the invariant measure in mean field modelswith jumps. Stochastic Systems, 2(2):322-380, 2012.
27 / 28
![Page 96: Distributed Randomized Control for Ancillary Service to the Power Grid](https://reader033.fdocuments.net/reader033/viewer/2022061209/5486328fb4af9fbf568b48e0/html5/thumbnails/96.jpg)
References
References: Economics
G. Wang, M. Negrete-Pincetic, A. Kowli, E. Shafieepoorfard, S. Meyn, and U. Shanbhag.Dynamic competitive equilibria in electricity markets. In A. Chakrabortty and M. Illic,editors, Control and Optimization Theory for Electric Smart Grids. Springer, 2011.
G. Wang, A. Kowli, M. Negrete-Pincetic, E. Shafieepoorfard, and S. Meyn. A controltheorist’s perspective on dynamic competitive equilibria in electricity markets. In Proc.18th World Congress of the International Federation of Automatic Control (IFAC), Milano,Italy, 2011.
S. Meyn, M. Negrete-Pincetic, G. Wang, A. Kowli, and E. Shafieepoorfard. The value ofvolatile resources in electricity markets. In Proc. of the 10th IEEE Conf. on Dec. andControl, Atlanta, GA, 2010.
I.-K. Cho and S. P. Meyn. Efficiency and marginal cost pricing in dynamic competitivemarkets with friction. Theoretical Economics, 5(2):215–239, 2010.
U.S. Energy Information Administration. Smart grid legislative and regulatory policies andcase studies. December 12 2011.http://www.eia.gov/analysis/studies/electricity/pdf/smartggrid.pdf
28 / 28