Optimal Switching Games for Emissions Trading - msri. · PDF fileOptimal Switching Games for...

1 / 29

Emissions Trading Mathematical Framework Solution Approach Two-Agent Game Setting Conclusion

Optimal Switching Games for Emissions Trading

Mike LudkovskiDepartment of Statistics & Applied Probability

University of California Santa Barbara

MSRI, May 4, 2009

Ludkovski Pricing Energy Storage

2 / 29


Outline

Cap-and-Trade: Producer PerspectiveSingle Agent ProblemMulti Agent ProblemNumerical Challenges


3 / 29


Emissions Trading

Major new initiatives are underway to introduce CO2 cap-and-tradeschemes that will create new commodity markets.AB32 proposal in California; Obama proposal federally.The estimated size of the market is in the hundreds of billions or eventrillions of dollars.Key regulatory details are still unresolved and undergo active publicdebate.Crucial to understand the financial implications of these initiatives onenergy producers.


4 / 29


A New Commodity Market

Compared to existing markets, cap-and-trade is fundamentally different:a finite resource is initially allocated and subject to exhaustion.A well-defined horizon (e.g. 1 year) exists for each allocation.The permit prices converge to deterministic values as horizonapproaches.Price formation is driven by participant strategies: must be endogenousto any model.New stochastic models are needed.

=⇒ The finite resource leads to game-theoretic aspects in the emissionsmarket.


5 / 29


Effect on Producers

The foremost constituency affected by cap-and-trade would be energyproducers.The net profit of energy production would change from the spark-spreadto the clean spread.Commodity prices (input fuel, output fuel) are stochastic.Must take into account (dynamic) strategies of other participants.Feedback between production policy and carbon prices.Spot vs. Forward Trading.


6 / 29


Related Literature

Real Options: Dixit and Pindyck (1994), Dockner et al. (2000).Analysis of Cap-and-Trade: Carmona et al. (2008,2009), Cetin andVerschuere (2008), ...Optimal Switching Problems (single-agent): Zervos (2003), Hamadeneand Jeanblanc (2005), L. and Carmona (2005, 2009), Hu and Tang(2008).Optimal Stopping Games: Ohtsubo (1987,1991), Shmaya and Solan(2004), Ferenstein (2005,2007), Ramsey and Szajowski (2008).


7 / 29


Model Setup

We focus on the timing optionality within a real-options framework.Take the point of view of an energy producer.Sells electricity into a stochastic market at price Pt , competes with cleanelectricity producers.Needs emission permits to produce.Has no initial allocation: must buy CO2 permits on the market at price Xt .Take a reduced-form price-impact model for (Xt ) (do not explicitly modelthe remaining supply of permits).Simplify the strategy set: at each time epoch either produce, or stayoffline, ξt ∈ {0,1}.Model in continuous-time.

First pass: single-agent model.Next: two-agent model with a game aspect.


7 / 29


Model Setup

We focus on the timing optionality within a real-options framework.Take the point of view of an energy producer.Sells electricity into a stochastic market at price Pt , competes with cleanelectricity producers.Needs emission permits to produce.Has no initial allocation: must buy CO2 permits on the market at price Xt .Take a reduced-form price-impact model for (Xt ) (do not explicitly modelthe remaining supply of permits).Simplify the strategy set: at each time epoch either produce, or stayoffline, ξt ∈ {0,1}.Model in continuous-time.First pass: single-agent model.Next: two-agent model with a game aspect.


8 / 29


Basic Ingredients

(Pt ) is exogenously given and is a 1-dimensional jump-diffusion.At each instant t choose an operating regime: ξt .(Xt ) is also a 1-dimensional diffusion, with drift controlled by ξt .Changes in ξt are expensive (fixed switching costs Ki,j ) and induce inertiaand hysteresis.Net profit is ξt (Pt − Xt − g)dt per unit time.Fixed horizon T : expiration date of the permits.As t → T , Xt converges to either 0 (excess supply of permits) or X(penalty price for exceeding permit allowance).Controls are piecewise-constant; can be thought of as a sequence ofstopping times ξ ≡ (τk ) (times of production policy changes).

=⇒ Set of admissible policies ξ ∈ U .


9 / 29


Controlled Probability Measures

Each admissible control ξ induces a probability law Pξ of (Xt ) through theprice impact mechanism.Define Pi , i ∈ {0,1} to be the probability law of X under no-production,full production.Apply a Girsanov transformation to obtain the controlled law Pξ wrt P0.Admissibility condition: Pξ(τk < T ∀k) = 0; number of regime changesis finite a.s.


10 / 29


Single Agent Control Problem

The manager maximizes total revenue on [0,T ].Work with the physical measures; do not consider any no-arbitragerestrictions.Let V (t ,p, x , i) denote maximum expected future profit given the initialconditions.Wish to find

V (0,p, x , i) = supξ∈U,ξ0=i

Eξ∫ T

0ξt (Pt − X (ξ)

t − g) dt −∑t≤T

Kξt−,ξt

.Bellman Principle:

V (0,p, x , i) = supξ:ξ0=i

Eξ∫ t

0ξs(Ps − Xs − g) ds −

∑s≤t

Kξs−,ξs + V (t ,Pt ,X(ξ)t , ξt )

.Ludkovski Pricing Energy Storage

11 / 29


Emission Price Model

Representative model:

dXt = 1{0<Xt<X} ·{

κX√T − t

(Xt − X (1− ξt )) dt + σX dW Xt

}.

If ξt = 1, then the trend is Xtdt , up;If ξt = 0, trend is Xt − X downward.Absorbing boundaries at 0 and X .As t → T the drift term grows without bound.Xt converges to one of the two boundaries, XT ∈ {0, x} a.s.Electricity pricesdPt = κP(θ − log Pt )Pt dt + σPPtdW P

t +∫

R(ez − 1)N(dt ,dz).


12 / 29


Solution Approach

The operational flexibility of the manager is a compound timing option.Under mild assumptions can show that will only make a finite number ofchanges in the optimal policy.

⇒ Recursively define V k (t ,p, x) for k = 0,1, . . ., 0 ≤ t ≤ T , p, x ∈ R:

V 0(t ,p, x) , E1[∫ T

t(Ps − Xs − g) ds

∣∣∣Pt = p,Xt = x], V 0(t ,p, x) = 0

V k (t ,p, x) , supτ∈St

E1[∫ τ


+{−K1,0 + V k−1(τ,Pτ ,Xτ )

}∣∣∣Pt = p,Xt = x].

V k (t ,p, x) , supτ∈St

E0[−K0,1 + V k−1(τ,Pτ ,Xτ )

∣∣∣Pt = p,Xt = x].

Coupled iterative optimal stopping problems.V k : value function when producing and k options left; V k : when offlineand k options left.


13 / 29


Basic Result

Proposition (Carmona-L (2008))1 V k , V k are equal to the value functions for the single-agent emission

production with at most k regime switches allowed.

2 An optimal finite strategy u∗ = (τ∗1 , ξ∗1 , τ∗2 , ξ∗2 , · · · ) for V 2k (t ,p, x) exists

and is: τ∗0 = t , ξ∗0 = 1, and for ` = 1, . . . , k

τ∗2` , inf{

s ≥ τ∗2`−1 : V 2`(s,Ps,Xs(u∗))

=(−K1,0 + V 2`−1(s,Ps,Xs(u∗))

)}∧ T ,

τ∗2`−1 , inf{

s ≥ τ∗2`−2 : V 2`−1(s,Ps,Xs(u∗))

=(−K0,1 + V 2`−2(s,Ps,Xs(u∗))

)}∧ T ,

3 limk→∞ V k (t ,p, x) = V (t ,p, x ,1) pointwise, uniformly on compacts.


13 / 29


Basic Result

Proposition (Carmona-L (2008))1 V k , V k are equal to the value functions for the single-agent emission

production with at most k regime switches allowed.2 An optimal finite strategy u∗ = (τ∗1 , ξ

∗1 , τ∗2 , ξ∗2 , · · · ) for V 2k (t ,p, x) exists

and is: τ∗0 = t , ξ∗0 = 1, and for ` = 1, . . . , k

τ∗2` , inf{

s ≥ τ∗2`−1 : V 2`(s,Ps,Xs(u∗))

=(−K1,0 + V 2`−1(s,Ps,Xs(u∗))

)}∧ T ,

τ∗2`−1 , inf{

s ≥ τ∗2`−2 : V 2`−1(s,Ps,Xs(u∗))

=(−K0,1 + V 2`−2(s,Ps,Xs(u∗))

)}∧ T ,

3 limk→∞ V k (t ,p, x) = V (t ,p, x ,1) pointwise, uniformly on compacts.


14 / 29


Coupled Optimal Stopping Problem

In the limit:

V (t ,p, x) , supτ∈St

E1[∫ τ


+{−K1,0 + V (τ,Pτ ,Xτ )

}∣∣∣Pt = p,Xt = x].

V (t ,p, x) , supτ∈St

E0[−K0,1 + V (τ,Pτ ,Xτ )

∣∣∣Pt = p,Xt = x].


15 / 29


Dynamic Programming by Backward Recursion

Discretize time: decisions are now made only at t = m∆t .Then to make a decision today, assuming currently offline, compute

V (t ,Pt ,Xt ) = max(−K0,1 + (Pt − Xt − g)∆t

+ E1[V (t + 1,Pt+1,Xt+1)| Ft],E0[V (t + 1,Pt+1,Xt+1)| Ft

]).

Similarly if “online” right now.Can apply these equations backward in time once can do the conditionalexpectation against the Markov state (P,X , ξ).


16 / 29


QVI Approach

The classical analytic theory (Øksendal and Sulem, 2005) implies that thevalue function is also the (unique viscosity) solution of the Quasi-VariationalInequality

min{− Vt − L1V (t , p, x)− (p − x − g), V (t , p, x)− (V (t , p, x)− K1,0)

}= 0.

min{− Vt − L0V (t , p, x), V (t , p, x)− (V (t , p, x)− K0,1)

}= 0.

V (T , p, x) = V (T , p, x) = 0.

(1)

Assuming a smooth pair (V , V ) this can then be implemented with a freeboundary pde solver.


17 / 29


Least Squares Monte Carlo

To compute the conditional expectations, a robust algorithm is to useMonte Carlo simulation.Simulate paths of (P,X ) for each of the two possible regimes.Continuation values are approximated through a cross-sectionalregression.If the optimal decision is to switch to another regime, then use theapproximate continuation value; else recursively update the futurepath-value.Extends the Longstaff-Schwartz method for American option pricing (asingle optimal stopping problem).


18 / 29


Simulation Methods

Simulate {pnt }N

n=1.

Simulate {xn,kt } for each regime k .

Will compute path values v(t ,pnt , x

n,kt , k) starting with

v(T ,gnT , x

n,kT , k) = 0.

Repeat for t = T −∆t down to t = ∆t .Given future path values {v(t + 1,pn

t+1, xn,kt+1, k)} regress them onto

{pnt , x

n,kt } to find out the coefficients α(t , k).

Taking into account current regime i , current revenue i · (pnt − xn,j

t − g)∆t ,and switching cost Ki,k , compute continuation values Cn

t (i , k) for eachpath.Find best action k∗ for each path.If k∗ == i , v(t ,pn

t , xn,kt , i) = v(t + 1,pn

t+1, xn,kt+1, i

∗) + i(pnt − xn,k

t − g)∆t .

Else v(t ,gnt , x

n,kt ; i) = Cn

t (i , k∗).

Return 1N

∑n v(0,gn

0 , xn,k0 , k).


19 / 29


Solution Structure

This is an optimal switching problem as studied in Carmona-L. (2008).The price impact leads to significant hysteresis effect.If the price impact is severe enough, will always stay offline (or at leastwith very high probability.Example: production function is (Ps − 0.5Xs − 30);(Pt ) is a jump-diffusion with mean 45 $/MWh.(Xt ) is a trend-persisting OU process with X0 = 20, X = 100.


20 / 29


Effect of Price Impact

Figure: Effect of price impact on optimal policy, T0 = 0.5,T = 1.


21 / 29


Hysteresis Band

Figure: Production Hysteresis in Carbon Permit Consumption, T0 = 0.5,T = 1. Greencurve: regime 0; blue curve: regime 1.


22 / 29


Multiple Agents: Oligopoly


23 / 29


Two-Agent Game

Consider now a market with several energy producers.Due to the price impact, the scheduling decisions of agents affect eachother.

dXt = 1{0<Xt<X} ·{

κ√T − t

(Xt − X (1− (ξ1t + ξ2

t )2/4) dt + σX dW Xt

}.

This leads to a stochastic game.Look at the simplest version: Two agents, each has 2 actions (switch orcontinue).

Bellman’s Principle is replaced by a Game Equilibrium.Pure Nash equilibria might not exist.Existence: Need mixed equilibria.Might also have multiple Nash equilibria.Uniqueness: equilibrium refinement algorithm (or correlationmechanism).


23 / 29


Two-Agent Game


dXt = 1{0<Xt<X} ·{

κ√T − t

(Xt − X (1− (ξ1t + ξ2


}.

This leads to a stochastic game.Look at the simplest version: Two agents, each has 2 actions (switch orcontinue).Bellman’s Principle is replaced by a Game Equilibrium.Pure Nash equilibria might not exist.Existence: Need mixed equilibria.

Might also have multiple Nash equilibria.Uniqueness: equilibrium refinement algorithm (or correlationmechanism).


23 / 29


Two-Agent Game


dXt = 1{0<Xt<X} ·{

κ√T − t

(Xt − X (1− (ξ1t + ξ2


}.

This leads to a stochastic game.Look at the simplest version: Two agents, each has 2 actions (switch orcontinue).Bellman’s Principle is replaced by a Game Equilibrium.Pure Nash equilibria might not exist.Existence: Need mixed equilibria.Might also have multiple Nash equilibria.Uniqueness: equilibrium refinement algorithm (or correlationmechanism).


24 / 29


Classification of 2-state Stochastic Games

Three equivalence classes:A single dominating pure equilibrium (unanimity).A competitive game (essentially zero-sum) which admits a unique mixedNash eqm.A coordination/anti-coordination game which admits two pure eqm’s, amixed one and a continuum of correlated eqm’s.The last case is the most complex one and naturally occurs in ourproblem:If the profit margin is small, the market can support only one producer(the second producer will kill the profits through price impact) –“battle-of-the-sexes” situation where one producer must yield to the other.


25 / 29


Stopping Games

A stopping game: each agent chooses a stopping time τi , i = 1,2.Agent i receives Xi (τ)1{τi<τj} + Yi (τ)1{τi>τj} + Wi (τ)1{τi =τj}, for somesextuple (Xi ,Yi ,Wi ), τ ≡ τ1 ∧ τ2.Solution now relies on Nash equilibria rather than on Dynamicprogramming.Starting with known values at T move back in time; each period yields a2-by-2 game with payoffs corresponding to conditional expectation ofnext-period value.

Let Val(Γt ) be an equilibrium of a 2-by-2 one-period game with payoffs

Γt =

((Z1(t),Z2(t)) (Y1(t),X2(t))(X1(t),Y2(t)) (W1(t),W2(t))

).

Then a stopping game has a value which solves (V1(t),V2(t)) = Val(Γt )as above, with (Z1(t),Z2(t)) ≡ (E[V1(t + 1)|Ft ],E[V2(t + 1)|Ft ]).Once such conditional expectations are computed, can solve all these1-period games iteratively back.


25 / 29


Stopping Games

A stopping game: each agent chooses a stopping time τi , i = 1,2.Agent i receives Xi (τ)1{τi<τj} + Yi (τ)1{τi>τj} + Wi (τ)1{τi =τj}, for somesextuple (Xi ,Yi ,Wi ), τ ≡ τ1 ∧ τ2.Solution now relies on Nash equilibria rather than on Dynamicprogramming.Starting with known values at T move back in time; each period yields a2-by-2 game with payoffs corresponding to conditional expectation ofnext-period value.Let Val(Γt ) be an equilibrium of a 2-by-2 one-period game with payoffs

Γt =

((Z1(t),Z2(t)) (Y1(t),X2(t))(X1(t),Y2(t)) (W1(t),W2(t))

).

Then a stopping game has a value which solves (V1(t),V2(t)) = Val(Γt )as above, with (Z1(t),Z2(t)) ≡ (E[V1(t + 1)|Ft ],E[V2(t + 1)|Ft ]).Once such conditional expectations are computed, can solve all these1-period games iteratively back.


26 / 29


Switching Game

We have a switching game. This is a sequential stopping game: canrepeatedly “stop” to alter production regimes in response to changingelectricity prices, permit prices or other agent’s actions.Player one: value function V (t ,Pt ,Xt , ξ

1t , ξ

2t ); player two: value function

W (t ,Pt ,Xt , ξ1t , ξ

2t ).

By logic similar to above expect that

(Vt ,Wt )(k , `) = Val(

(Et [(Vt+1,Wt+1)(k , `)] (Vt ,Wt − K`, ¯)(k , ¯)

(Vt − Kk,k ,Wt )(k , `) (Vt − Kk,k ,Wt − K`, ¯)(k , ¯)

).

Sketch of proof: Restrict strategy sets so that agents can only use up to(n,m) switches. This is equivalent to an iterative stopping game withpayoffs corresponding to (n − 1,m), (n,m − 1) or (n − 1,m − 1) cases,respectively.Taking the limit n,m→∞ to obtain a coupled pair of value functions asabove.


27 / 29


Numerical Algorithm

Extend the single-agent simulation method.Simulate paths forward, move backward and solve the one-period games.Use cross-sectional regressions to estimate continuation values.Allow randomized strategies by “mixing” different continuation values.Examples coming soon...


28 / 29


Conclusion

Stochastic games naturally occur in studying oligopolies.The emission market would be a new important class of such problems.Investigate the simplest possible scenario where the game is non-trivial:a new model of an optimal switching game.Already the problems of equilibrium-refinement and computationaltractability arise.Many open problems: how to succinctly capture the competition forpermits between different producers? How does the initial allocationaffect outcomes? Collusion between producers?(Xt ) is a traded asset: no-arbitrage implications?Also, producers will be allowed to bank and trade their permits.


29 / 29


References

California Air Resources BoardClimate Change Proposed Scoping Plan.http://www.arb.ca.gov, October 2008.

E. J. Dockner, S. Jorgensen, N.V. Long, and G. Sorger,Differential games in economics and management science.Cambridge University Press, Cambridge, 2000.

R. Carmona, M. Fehr, J. Hinz and A. Porchet.Market Design for Emission Trading Schemes,SIAM Review, forthcoming.

R. Carmona, and M. Ludkovski.Pricing asset scheduling flexibility using optimal switching.Appl. Math. Finance, 15(4):405–447, 2008.

Y. Ohtsubo.A nonzero-sum extension of Dynkin’s stopping problem.Math. Oper. Res., 12(2):277–296, 1987.

D. M. Ramsey and K. Szajowski.Selection of a correlated equilibrium in Markov stopping games.European J. Oper. Res., 184(1):185–206, 2008.


http://www.arb.ca.gov

Optimal Switching Games for Emissions Trading - msri. · PDF fileOptimal Switching Games for...

Documents

Transcript of Optimal Switching Games for Emissions Trading - msri. · PDF fileOptimal Switching Games for...