1. O. TeytaudFull time research fellow.Head of Tao-Uct-Sig, the part of Taodedicated to sequential decision making.Three ph.D. students, two of them full time on energy;one engineer full time on energy.Probably new funding on energy as well.J. Decock, A. Coutoux: two (great :-) )ph.D. students around energy management.

2. MY LAB: LRI (115 permanent staff) Depends on:- CNRS (best research institute inthe world for the SCImago Ranking)- INRIA(more focus on links with real world andeconomy)- Universit Paris-Sud 3. MY TEAM: TAO http://tao.lri.frArtificial intelligence / operations research / optimizationTheory and applicationsMany full-time researchersMany ph.D. students, coming from famous schoolsMany links ==> real-world, industries 4. 1. Energy in France2. Our target problem3. Some ideas 5. France and energy, historically:- decision of independencew.r.t other countries (==> nuclear power)- nuclear power is not stopped after Fukushima- due to strong dirigism, huge companies: - EDF = largest utility company in the world - GDF-Suez = largest energy company - Areva = world leader of nuclear power - Renault-Nissan = leader of electric carsIn particular, very strong electricity companies. 6. French energy company EDF:- worlds largest utility company.- 120,000+ megawatts in Europe,Latin America, Asia,the Middle-East and Africa.- Controls companies in allcontinents.- France wants to increaseenergetic independence (Taiwan also ?) 7. ELECTRICITY GENERATIONin FRANCE France = world champion of nuclearpower plants 8. ELECTRICITY GENERATIONin FRANCE also hydroelectricity 9. A RISK: SOMETHING LIKETHE 2003 BLACK OUT IN USWe had similar stuff (smaller) 10. ANOTHER RISK: LACK OF WATER(DROUGHT)(nuclear power plantscooled by water) E.g. the terrible 2003 heat wave (15 000 died; 6.2 times more than 921 earthquake in Taiwan) (a night with no less than 25C)==> close to electricity breakdown! 11. FUTURE EXTENSIONS:CONNECTIONS TO THE NORTH OF EUROPE ?CONNECTIONS TO AFRICA ?- using North-Europe wind in winter ?- using Africas sun in summer ?MORE NUCLEAR POWER ?- because currently coal / oil kill more peoplethan nuclear power ?- and because no earthquake and no tsunamiin France?OR NO MORE NUCLEAR POWER ? because we never know what could happen (terrorism ?)SMART GRIDS: which kind of smartness ? (a lot of work on heat networks; in Taiwan I guess air conditionning is more relevant) 12. 1. Energy in France2. Our target problem3. Some ideas 13. For all works around energy,we collaborate with other teams(French company Artelys, ANR projects,Ademe projects...)Why ?- because we work on thecomputational intelligence aspects;- other teams have more knowledgeand data specifically on electricity. 14. What we do: optimizing investment policiesFor example: build in 2015-2050 hydroelectric power plants (how many ? where ?), nuclear power plants (how many ? where ?), concentrated solar power plants (how many ? Where ?), photovoltaic units (...), wind farms (...), new connections (...), How to choose all these parameters ?Simulations: based on annual planning tools(big part of the work)Uncertainties: - technological breakthroughs - climate - demography - industrial changes - ecological constraints==> optimization with uncertainties, using scenarios 15. Investments MacroPolicy long-term scenarii (when/where/howare units built)Economy, Provided byindustry,domaindemography,expertstechnologies.(solar concentration, wind energy, future nuclear tools,long-term demography, oilreserves) Architecture Annual model for year yStochasticity ?Annualplanningtool.Space / timeAnnual resultsDecomposition ? for year y 16. Investments MacroPolicy long-term scenarii (when/where/howare units built)Economy,industry, Statisticaldemography,modelizationtechnologies. Architecture Annual model for year yStochasticity ?Annualplanningtool.Space / timeAnnual resultsDecomposition ? for year y 17. Investments MacroPolicy long-term scenarii (when/where/howare units built)Economy,industry, Reinforcement learning,demography, stochastic dynamictechnologies.programming, direct policysearch, MCTS... Architecture Annual model for year yStochasticity ?Annualplanningtool.Space / timeAnnual resultsDecomposition ? for year y 18. Investments Macro Policy long-term scenarii(when/where/how are units built)Economy,industry,demography,technologies. Reinforcement learning, stochastic dynamicprogramming, direct policyArchitecture Annual modelfor year y search,MCTS ...Stochasticity ?Annualplanningtool.Space / time Annual resultsDecomposition ?for year y 19. Investments Macro Policy long-term scenarii(when/where/how are units built)Economy,industry,demography,technologies.Architecture Annual modelfor year yStochasticity ?Annualplanningtool.Space / time Annual resultsDecomposition ?for year y 20. Classical tools for this:- climate modelization (including climate change)- power plant modelization- ecological penalization (what is the cost of 1 ton CO2 ?)- sequential decision making (also known as: reactive planning):On day D, at time T, if stock level in reservoir #i = Li,if climate is in state Ci, then what is the part Pjof the electricity demandthat should be satisfied by power plant #j ?==> classical tool: Bellman decomposition==> our specificity: less constrained models; modelsas close as possible to reality (Bellman solving implies a lot of constrained on the model). 21. So we use: - field experts (for datas), climate, oil, nuclear physics... - statisticians - people from operation research - sequential decision making - noisy constrained non-linear optimization - real-world data- grids==> for the study in France/Europe/Africa we have all this==> Taiwan is simpler (by far smaller) but non trivial==> I try to do something similar in Taiwan/Asia ==> for the moment no data ==> and I wont do anything serious on the Taiwan casewith no Taiwanese people 22. Kind of questions that we plan to analyze: - which connections should we build between Europe and Africa ? (which type of connections ? AC/DC ?) - what is the consequence of massively switching off nuclear power plants in Europe ?- what is the consequence of subventions of PV units (or other) ?(depending on the amount; which amount is best ?)- what is the benefit of demand-reduction contracts ? (which price for these contracts ?)==> importantly, all these questions jointly !==> room for such questions in Taiwan ? 23. Classical tools for this:- climate modelization (including climate change)- power plant modelization- ecological penalization (what is the cost of 1 ton CO2 ?)This is a key part- sequential decision making (alsoour contribution.oflevel in reservoir #i = planning):known as: reactiveOn day D, at time T, if stockLi, if climate is in state Ci, then what is the part Pj of the electricity demand that should be satisfied by power plant #j ?==> classical tool: Bellman decomposition==> our specificity: less constrained models; models as close as possible to reality(Bellman solving implies a lot of constraints on the model). 24. My (maybe bad) ideas for the futureof our optimization platform1. What is Direct Policy Search (DPS)2. Why is consider DPS 25. First, you need to know what isdirect policy search (DPS).Principle of DPS: (1) Define a parametric policy Pi with parameters t1,...,tk. (2) maximize (t1,...,tk) average reward when applying Policy pi(t1,...,tk) on the problem.==> You must define Pi ==> You must choose a noisy optimization algorithm==> There is a Pi by default (an actor neural network),but its only a default solution (overload it) 26. Strengths of DPS:- Good warm start If I have a solution for problem A, and if I switch to problem B close to A, then I quickly get good results.- Benefits from expert knowledge on the structure- No constraint on the structure of the objective function- Anytime (i.e. not that bad in restricted time)Drawbacks:- needs structured direct policy search - not directly applicable to partial observation 27. DPS and MCTS (1player case) are anytime: you start from a strategy, and it improves over time.I like this very much.I want to combine these two good tools. 28. Weakness of DPS:- weak for short term combinatorial effects (compared to tree search methods)- no online learning (i.e. no use of online time for making decisions). Weakness of MCTS:- weak for long term effects- weak offline learningPrinciple: MCTS with DPS-optimized playouts:1) Optimize a policy (offline)2) use the optimized policy as default policy in MCTS==> Compare to Bellman style or LP-style optimization;possibly combine. 29. Last part:join the force !The first time Ive met Energy Management problems,I was very impressed. I believed that all programs around this must be perfect, bug-free, no bad assumption; I believed I wouldnever be clever enough for working on such a crucial problem.Now I have seen codes. Bugs everywhere, wrong assumptions.I want to test really new ideas, and compare them to existing tools.We have access to data, to people, to existing codes;we can test, combine, have results with impact; thats just extremelymotivating. 30. Plans:- I have fundings for working onthese topics in France / Europe- we have fundings for organizing meetings between Europe & Taiwan- maybe future joint work about same methodologies but in Taiwan or Asia 31. Short term plans: - You have ideas that you would like to see tested inMCTS/reinforcement learningfor energy management ?==> tell me. For this problem I have motivationenough for implementing other peoples ideasmyself :-) - Meeting in May:- can you join ? ==> doodle- contacts in industry for this meeting ?- photovoltaic units (present + future) ? - Future:- interested in future collaborations ?- have contacts in industry / companies possiblymotivated by long-term planning in electricityManagement ?