DREAMPlace: Deep Learning Toolkit-Enabled GPU …DREAMPlace: Deep Learning Toolkit-Enabled GPU...

6
DREAMPlace: D eep Lear ning Toolkit-E nabled GPU A cceleration for M odern VLSI Place ment Yibo Lin ECE Department, UT Austin [email protected] Shounak Dhar ECE Department, UT Austin [email protected] Wuxi Li ECE Department, UT Austin [email protected] Haoxing Ren Nvidia, Inc., Austin [email protected] Brucek Khailany Nvidia, Inc., Austin [email protected] David Z. Pan ECE Department, UT Austin [email protected] ABSTRACT Placement for very-large-scale integrated (VLSI) circuits is one of the most important steps for design closure. This paper proposes a novel GPU-accelerated placement framework DREAMPlace, by casting the analytical placement problem equivalently to training a neural net- work. Implemented on top of a widely-adopted deep learning toolkit PyTorch, with customized key kernels for wirelength and density computations, DREAMPlace can achieve over 30× speedup in global placement without quality degradation compared to the state-of-the- art multi-threaded placer RePlAce. We believe this work shall open up new directions for revisiting classical EDA problems with advance- ment in AI hardware and software. 1 INTRODUCTION Placement is a critical but time-consuming step in the VLSI design flow. As it determines the locations of standard cells in the physi- cal layout, its quality has significant impacts on the later stages in the flow such as routing and post-layout optimization. A placement solution also provides relatively accurate estimation to routed wire- length and congestion, which is very valuable in guiding the earlier stages like logic synthesis. The commercial design flows often run core placement engines many times to achieve design closure. As placement involves large-scale numerical optimization, current place- ment usually takes hours for large designs, thus, slowing down design iterations. Therefore, ultra-fast yet high quality placement is always in strong demand. Analytical placement is the current state-of-the-art for VLSI place- ment [17]. It essentially solves a nonlinear optimization problem. Although analytical placement can produce high-quality solutions, it is also known to be relatively slow [811]. Here we provide a brief introduction to the analytical placement problem. Suppose a circuit is described as a hypergraph H = ( V , E), where V denotes the set of vertices (cells) and E denotes the set of hyperedges (nets). Let x, y denote the locations of cells. The objective of analytical placement is to determine the locations of cells with wirelength minimized and no overlap in the layout. To accelerate placement, existing parallelization efforts have mostly targeted multi-threaded CPUs using partitioning [1113]. As the num- ber of threads increases, speedup quickly saturates at around 5× in global placement with typical quality degradation of 2-6%. Cong et al. explored GPU acceleration for analytical placement [14]. They DAC ’19, June 2–6, 2019, Las Vegas, NV, USA © 2019 Association for Computing Machinery. This is the author’s version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in The 56th Annual Design Automation Conference 2019 (DAC ’19), June 2–6, 2019, Las Vegas, NV, USA, https: //doi.org/10.1145/3316781.3317803. combined clustering and declustering with nonlinear placement opti- mization. By parallelizing the nonlinear placement part, an average of 15× speedup in global placement was reported with less than 1% quality degradation. Lin et al. proposed GPU acceleration techniques for wirelength gradient computation and area accumulation [15], but their experiment failed to consider real operations such as density cost computation and it lacked the validation from real analytical placement flows. In addition, current research on placement is facing challenges in the lack of well-maintained public frameworks and the high development overhead, raising the bar to systematically validate new algorithms. In this work, we propose DREAMPlace, a GPU-accelerated analyt- ical placer developed with deep learning toolkit PyTorch by cast- ing an analytical placement problem to training a neural network. DREAMPlace is based on the state-of-the-art analytical placement algorithm ePlace/RePlAce family [4, 6], but the framework is designed in a generic way that is compatible with other analytical placers such as NTUplace [2]. The key contributions are summarized as follows. We take a totally new perspective of making analogy between placement and deep learning, and build an open-source generic analytical placement framework that runs on both CPU and GPU platforms developed with modern deep learning toolkits. We propose efficient GPU implementations of key kernels in analytical placement like wirelength and density computation. We demonstrate over 30× speedup in global placement and le- galization without quality degradation of the entire placement flow over multi-threaded CPU implementations. More specifi- cally, a design with one million cells finishes in one minute. The framework maintains nearly linear scalability with industrial designs up to 10-million cells. The source code is released on Github 1 . To clarify, the casting of place- ment problem to deep learning problems aims at using the toolkit to solve placement, which is orthogonal to using deep learning models for placement. The rest of the paper is organized as follows. Section 2 describes the background and motivation; Section 3 explains the de- tailed implementation; Section 4 demonstrates the results; Section 5 concludes the paper. 2 PRELIMINARIES This section will review the background and motivation. 2.1 Analytical Placement Analytical placement usually consists of three steps: global placement (GP), legalization (LG), and detailed placement (DP). Global placement spreads out cells in the layout with a target cost minimized; legaliza- tion removes the remaining overlaps between cells and aligns cells to 1 https://github.com/limbo018/DREAMPlace

Transcript of DREAMPlace: Deep Learning Toolkit-Enabled GPU …DREAMPlace: Deep Learning Toolkit-Enabled GPU...

Page 1: DREAMPlace: Deep Learning Toolkit-Enabled GPU …DREAMPlace: Deep Learning Toolkit-Enabled GPU Acceleration for Modern VLSI Placement Yibo Lin ECE Department, UT Austin yibolin@utexas.edu

DREAMPlace: Deep Learning Toolkit-Enabled GPU Accelerationfor Modern VLSI Placement

Yibo LinECE Department, UT Austin

[email protected]

Shounak DharECE Department, UT [email protected]

Wuxi LiECE Department, UT Austin

[email protected]

Haoxing RenNvidia, Inc., Austin

[email protected]

Brucek KhailanyNvidia, Inc., Austin

[email protected]

David Z. PanECE Department, UT Austin

[email protected]

ABSTRACTPlacement for very-large-scale integrated (VLSI) circuits is one of themost important steps for design closure. This paper proposes a novelGPU-accelerated placement framework DREAMPlace, by casting theanalytical placement problem equivalently to training a neural net-work. Implemented on top of a widely-adopted deep learning toolkitPyTorch, with customized key kernels for wirelength and densitycomputations, DREAMPlace can achieve over 30× speedup in globalplacement without quality degradation compared to the state-of-the-art multi-threaded placer RePlAce. We believe this work shall openup new directions for revisiting classical EDA problems with advance-ment in AI hardware and software.

1 INTRODUCTIONPlacement is a critical but time-consuming step in the VLSI designflow. As it determines the locations of standard cells in the physi-cal layout, its quality has significant impacts on the later stages inthe flow such as routing and post-layout optimization. A placementsolution also provides relatively accurate estimation to routed wire-length and congestion, which is very valuable in guiding the earlierstages like logic synthesis. The commercial design flows often runcore placement engines many times to achieve design closure. Asplacement involves large-scale numerical optimization, current place-ment usually takes hours for large designs, thus, slowing down designiterations. Therefore, ultra-fast yet high quality placement is alwaysin strong demand.

Analytical placement is the current state-of-the-art for VLSI place-ment [1–7]. It essentially solves a nonlinear optimization problem.Although analytical placement can produce high-quality solutions, itis also known to be relatively slow [8–11]. Here we provide a briefintroduction to the analytical placement problem. Suppose a circuitis described as a hypergraph H = (V ,E), where V denotes the set ofvertices (cells) and E denotes the set of hyperedges (nets). Let x, ydenote the locations of cells. The objective of analytical placement isto determine the locations of cells with wirelength minimized and nooverlap in the layout.

To accelerate placement, existing parallelization efforts have mostlytargeted multi-threaded CPUs using partitioning [11–13]. As the num-ber of threads increases, speedup quickly saturates at around 5× inglobal placement with typical quality degradation of 2-6%. Cong etal. explored GPU acceleration for analytical placement [14]. They

DAC ’19, June 2–6, 2019, Las Vegas, NV, USA© 2019 Association for Computing Machinery.This is the author’s version of the work. It is posted here for your personal use. Notfor redistribution. The definitive Version of Record was published in The 56th AnnualDesign Automation Conference 2019 (DAC ’19), June 2–6, 2019, Las Vegas, NV, USA, https://doi.org/10.1145/3316781.3317803.

combined clustering and declustering with nonlinear placement opti-mization. By parallelizing the nonlinear placement part, an averageof 15× speedup in global placement was reported with less than 1%quality degradation. Lin et al. proposed GPU acceleration techniquesfor wirelength gradient computation and area accumulation [15], buttheir experiment failed to consider real operations such as densitycost computation and it lacked the validation from real analyticalplacement flows. In addition, current research on placement is facingchallenges in the lack of well-maintained public frameworks and thehigh development overhead, raising the bar to systematically validatenew algorithms.

In this work, we propose DREAMPlace, a GPU-accelerated analyt-ical placer developed with deep learning toolkit PyTorch by cast-ing an analytical placement problem to training a neural network.DREAMPlace is based on the state-of-the-art analytical placementalgorithm ePlace/RePlAce family [4, 6], but the framework is designedin a generic way that is compatible with other analytical placers suchas NTUplace [2]. The key contributions are summarized as follows.• We take a totally new perspective of making analogy betweenplacement and deep learning, and build an open-source genericanalytical placement framework that runs on both CPU andGPU platforms developed with modern deep learning toolkits.• We propose efficient GPU implementations of key kernels inanalytical placement like wirelength and density computation.• We demonstrate over 30× speedup in global placement and le-galization without quality degradation of the entire placementflow over multi-threaded CPU implementations. More specifi-cally, a design with one million cells finishes in one minute. Theframework maintains nearly linear scalability with industrialdesigns up to 10-million cells.

The source code is released on Github 1. To clarify, the casting of place-ment problem to deep learning problems aims at using the toolkit tosolve placement, which is orthogonal to using deep learning modelsfor placement. The rest of the paper is organized as follows. Section 2describes the background and motivation; Section 3 explains the de-tailed implementation; Section 4 demonstrates the results; Section 5concludes the paper.

2 PRELIMINARIESThis section will review the background and motivation.

2.1 Analytical PlacementAnalytical placement usually consists of three steps: global placement(GP), legalization (LG), and detailed placement (DP). Global placementspreads out cells in the layout with a target cost minimized; legaliza-tion removes the remaining overlaps between cells and aligns cells to

1https://github.com/limbo018/DREAMPlace

Page 2: DREAMPlace: Deep Learning Toolkit-Enabled GPU …DREAMPlace: Deep Learning Toolkit-Enabled GPU Acceleration for Modern VLSI Placement Yibo Lin ECE Department, UT Austin yibolin@utexas.edu

Data Instance

ErrorFunction

(xi, yi)<latexit sha1_base64="D+UM8CEWLQdzKFU+eSJkb2ayH30=">AAADU3icfZPPatwwEMYVb/+kbtMm7bEXUaeQQlls76E5BnLpMYVuErDMImvHu2Il2UhymsXxa+TavlQPfZZeKjsOZJ00A4aPmfn0G4lxVgpubBj+2fJGT54+e779wn/5auf1m929t6emqDSDKStEoc8zakBwBVPLrYDzUgOVmYCzbHXc1s8uQBteqO92XUIq6ULxnDNqXYrsH1zO+Ge8nvFP+7PdIByHXeD7IupFgPo4me15EZkXrJKgLBPUmCQKS5vWVFvOBDQ+qQyUlK3oAhInFZVg0robusEfXWaO80K7T1ncZe86aiqNWcvMdUpql2ZYa5MP1ZLK5odpzVVZWVDsBpRXAtsCty+A51wDs2LtBGWau1kxW1JNmXXv5Ps+UfCDFVJSNa9JpmmTRGlNQJlKQ0uriYDcEkHVQkAQEc0XS3vVNJvGFdiHjVe3FqK7E4ZGR2SdOX6MislFuaTKFrIO4qafAXc9YxzEd8tR8yjPHa75JYgmmfwfiAfEYNLgW2hL6y6Gg8kG1w2Gh2i3ZNFwpe6L03gcOf0tDo4O+3XbRu/RB3SAIvQFHaGv6ARNEUMlukY/0S/vt/d35P6Sm1Zvq/e8Qxsx2vkHAi8UJA==</latexit><latexit sha1_base64="D+UM8CEWLQdzKFU+eSJkb2ayH30=">AAADU3icfZPPatwwEMYVb/+kbtMm7bEXUaeQQlls76E5BnLpMYVuErDMImvHu2Il2UhymsXxa+TavlQPfZZeKjsOZJ00A4aPmfn0G4lxVgpubBj+2fJGT54+e779wn/5auf1m929t6emqDSDKStEoc8zakBwBVPLrYDzUgOVmYCzbHXc1s8uQBteqO92XUIq6ULxnDNqXYrsH1zO+Ge8nvFP+7PdIByHXeD7IupFgPo4me15EZkXrJKgLBPUmCQKS5vWVFvOBDQ+qQyUlK3oAhInFZVg0robusEfXWaO80K7T1ncZe86aiqNWcvMdUpql2ZYa5MP1ZLK5odpzVVZWVDsBpRXAtsCty+A51wDs2LtBGWau1kxW1JNmXXv5Ps+UfCDFVJSNa9JpmmTRGlNQJlKQ0uriYDcEkHVQkAQEc0XS3vVNJvGFdiHjVe3FqK7E4ZGR2SdOX6MislFuaTKFrIO4qafAXc9YxzEd8tR8yjPHa75JYgmmfwfiAfEYNLgW2hL6y6Gg8kG1w2Gh2i3ZNFwpe6L03gcOf0tDo4O+3XbRu/RB3SAIvQFHaGv6ARNEUMlukY/0S/vt/d35P6Sm1Zvq/e8Qxsx2vkHAi8UJA==</latexit><latexit sha1_base64="D+UM8CEWLQdzKFU+eSJkb2ayH30=">AAADU3icfZPPatwwEMYVb/+kbtMm7bEXUaeQQlls76E5BnLpMYVuErDMImvHu2Il2UhymsXxa+TavlQPfZZeKjsOZJ00A4aPmfn0G4lxVgpubBj+2fJGT54+e779wn/5auf1m929t6emqDSDKStEoc8zakBwBVPLrYDzUgOVmYCzbHXc1s8uQBteqO92XUIq6ULxnDNqXYrsH1zO+Ge8nvFP+7PdIByHXeD7IupFgPo4me15EZkXrJKgLBPUmCQKS5vWVFvOBDQ+qQyUlK3oAhInFZVg0robusEfXWaO80K7T1ncZe86aiqNWcvMdUpql2ZYa5MP1ZLK5odpzVVZWVDsBpRXAtsCty+A51wDs2LtBGWau1kxW1JNmXXv5Ps+UfCDFVJSNa9JpmmTRGlNQJlKQ0uriYDcEkHVQkAQEc0XS3vVNJvGFdiHjVe3FqK7E4ZGR2SdOX6MislFuaTKFrIO4qafAXc9YxzEd8tR8yjPHa75JYgmmfwfiAfEYNLgW2hL6y6Gg8kG1w2Gh2i3ZNFwpe6L03gcOf0tDo4O+3XbRu/RB3SAIvQFHaGv6ARNEUMlukY/0S/vt/d35P6Sm1Zvq/e8Qxsx2vkHAi8UJA==</latexit><latexit sha1_base64="D+UM8CEWLQdzKFU+eSJkb2ayH30=">AAADU3icfZPPatwwEMYVb/+kbtMm7bEXUaeQQlls76E5BnLpMYVuErDMImvHu2Il2UhymsXxa+TavlQPfZZeKjsOZJ00A4aPmfn0G4lxVgpubBj+2fJGT54+e779wn/5auf1m929t6emqDSDKStEoc8zakBwBVPLrYDzUgOVmYCzbHXc1s8uQBteqO92XUIq6ULxnDNqXYrsH1zO+Ge8nvFP+7PdIByHXeD7IupFgPo4me15EZkXrJKgLBPUmCQKS5vWVFvOBDQ+qQyUlK3oAhInFZVg0robusEfXWaO80K7T1ncZe86aiqNWcvMdUpql2ZYa5MP1ZLK5odpzVVZWVDsBpRXAtsCty+A51wDs2LtBGWau1kxW1JNmXXv5Ps+UfCDFVJSNa9JpmmTRGlNQJlKQ0uriYDcEkHVQkAQEc0XS3vVNJvGFdiHjVe3FqK7E4ZGR2SdOX6MislFuaTKFrIO4qafAXc9YxzEd8tR8yjPHa75JYgmmfwfiAfEYNLgW2hL6y6Gg8kG1w2Gh2i3ZNFwpe6L03gcOf0tDo4O+3XbRu/RB3SAIvQFHaGv6ARNEUMlukY/0S/vt/d35P6Sm1Zvq/e8Qxsx2vkHAi8UJA==</latexit>

Neural Network

minw

nX

i

f(�(xi;w), yi) + �R(w)<latexit sha1_base64="9C8W+D2OgNTUNRjy5z6yg/eD+No=">AAAD1HicfZJBb9MwFMfdZsAIbHRw5GIRIXUCVU17YBKXSXDgOBDtJtUlctyX1KrjBNvZWqU5Ia58Py58FpwsaG03ZinS03vv79/fLy/MBNem3//dajt7Dx4+2n/sPnl6cPisc/R8rNNcMRixVKTqIqQaBJcwMtwIuMgU0CQUcB4uPlT180tQmqfyq1llME1oLHnEGTU2FXSWJISYywK+53WmdEnCZVCQhJp5GBVXZYmJzpOAf5M4wl2SzXl3GfD3+Kbj+C1eBfwYv8FEWPKM4i/djSomMjU0dgnI2Q0n6Hj9Xr8++HbgN4GHmnMWHLU/klnK8gSkYYJqPfH7mZkWVBnOBFjjuYaMsgWNYWJDSRPQ06IeUYlf28wMR6mynzS4zm4qCppovUpC21k517u1KnlXbZKb6GRacJnlBiS7BkW5wCbF1bzxjCtgRqxsQJni1itmc6ooM/avbFEMt4ZL13WJhCuWJgm18yKhouXEnxZ2ejpXUBkoiIDI2FnLWIDnE8XjuVmX5bZwAeZu4fqfhKj6hl2hJbJaPLiPisllNqfSpEnhDcrGA657etgbbJb98l6evVzxJYhyMvw/EO8QvaHdywZa0eqHYW+4xbXG8C7a7p2/u2W3g/Gg59v488A7PWk2cB+9RK9QF/noHTpFn9AZGiGG/rT2WgetQ2fsrJ0fzs/r1nar0bxAW8f59RdVQESb</latexit><latexit sha1_base64="9C8W+D2OgNTUNRjy5z6yg/eD+No=">AAAD1HicfZJBb9MwFMfdZsAIbHRw5GIRIXUCVU17YBKXSXDgOBDtJtUlctyX1KrjBNvZWqU5Ia58Py58FpwsaG03ZinS03vv79/fLy/MBNem3//dajt7Dx4+2n/sPnl6cPisc/R8rNNcMRixVKTqIqQaBJcwMtwIuMgU0CQUcB4uPlT180tQmqfyq1llME1oLHnEGTU2FXSWJISYywK+53WmdEnCZVCQhJp5GBVXZYmJzpOAf5M4wl2SzXl3GfD3+Kbj+C1eBfwYv8FEWPKM4i/djSomMjU0dgnI2Q0n6Hj9Xr8++HbgN4GHmnMWHLU/klnK8gSkYYJqPfH7mZkWVBnOBFjjuYaMsgWNYWJDSRPQ06IeUYlf28wMR6mynzS4zm4qCppovUpC21k517u1KnlXbZKb6GRacJnlBiS7BkW5wCbF1bzxjCtgRqxsQJni1itmc6ooM/avbFEMt4ZL13WJhCuWJgm18yKhouXEnxZ2ejpXUBkoiIDI2FnLWIDnE8XjuVmX5bZwAeZu4fqfhKj6hl2hJbJaPLiPisllNqfSpEnhDcrGA657etgbbJb98l6evVzxJYhyMvw/EO8QvaHdywZa0eqHYW+4xbXG8C7a7p2/u2W3g/Gg59v488A7PWk2cB+9RK9QF/noHTpFn9AZGiGG/rT2WgetQ2fsrJ0fzs/r1nar0bxAW8f59RdVQESb</latexit><latexit sha1_base64="9C8W+D2OgNTUNRjy5z6yg/eD+No=">AAAD1HicfZJBb9MwFMfdZsAIbHRw5GIRIXUCVU17YBKXSXDgOBDtJtUlctyX1KrjBNvZWqU5Ia58Py58FpwsaG03ZinS03vv79/fLy/MBNem3//dajt7Dx4+2n/sPnl6cPisc/R8rNNcMRixVKTqIqQaBJcwMtwIuMgU0CQUcB4uPlT180tQmqfyq1llME1oLHnEGTU2FXSWJISYywK+53WmdEnCZVCQhJp5GBVXZYmJzpOAf5M4wl2SzXl3GfD3+Kbj+C1eBfwYv8FEWPKM4i/djSomMjU0dgnI2Q0n6Hj9Xr8++HbgN4GHmnMWHLU/klnK8gSkYYJqPfH7mZkWVBnOBFjjuYaMsgWNYWJDSRPQ06IeUYlf28wMR6mynzS4zm4qCppovUpC21k517u1KnlXbZKb6GRacJnlBiS7BkW5wCbF1bzxjCtgRqxsQJni1itmc6ooM/avbFEMt4ZL13WJhCuWJgm18yKhouXEnxZ2ejpXUBkoiIDI2FnLWIDnE8XjuVmX5bZwAeZu4fqfhKj6hl2hJbJaPLiPisllNqfSpEnhDcrGA657etgbbJb98l6evVzxJYhyMvw/EO8QvaHdywZa0eqHYW+4xbXG8C7a7p2/u2W3g/Gg59v488A7PWk2cB+9RK9QF/noHTpFn9AZGiGG/rT2WgetQ2fsrJ0fzs/r1nar0bxAW8f59RdVQESb</latexit><latexit sha1_base64="9C8W+D2OgNTUNRjy5z6yg/eD+No=">AAAD1HicfZJBb9MwFMfdZsAIbHRw5GIRIXUCVU17YBKXSXDgOBDtJtUlctyX1KrjBNvZWqU5Ia58Py58FpwsaG03ZinS03vv79/fLy/MBNem3//dajt7Dx4+2n/sPnl6cPisc/R8rNNcMRixVKTqIqQaBJcwMtwIuMgU0CQUcB4uPlT180tQmqfyq1llME1oLHnEGTU2FXSWJISYywK+53WmdEnCZVCQhJp5GBVXZYmJzpOAf5M4wl2SzXl3GfD3+Kbj+C1eBfwYv8FEWPKM4i/djSomMjU0dgnI2Q0n6Hj9Xr8++HbgN4GHmnMWHLU/klnK8gSkYYJqPfH7mZkWVBnOBFjjuYaMsgWNYWJDSRPQ06IeUYlf28wMR6mynzS4zm4qCppovUpC21k517u1KnlXbZKb6GRacJnlBiS7BkW5wCbF1bzxjCtgRqxsQJni1itmc6ooM/avbFEMt4ZL13WJhCuWJgm18yKhouXEnxZ2ejpXUBkoiIDI2FnLWIDnE8XjuVmX5bZwAeZu4fqfhKj6hl2hJbJaPLiPisllNqfSpEnhDcrGA657etgbbJb98l6evVzxJYhyMvw/EO8QvaHdywZa0eqHYW+4xbXG8C7a7p2/u2W3g/Gg59v488A7PWk2cB+9RK9QF/noHTpFn9AZGiGG/rT2WgetQ2fsrJ0fzs/r1nar0bxAW8f59RdVQESb</latexit>

f(�(xi;w), yi)<latexit sha1_base64="Bw6B3M5YjcGyF/3cXJQj8eOZs2I=">AAADonicfVJda9swFFXjfXTeR9PtcS9i3iCFEeLkYYW9FDbG2FPLmrZghSAr146ILHuS3Da4+kf7NXsb24+Z7Lq0SbteEBzOvUfn3suNC8G1GQx+b3S8Bw8fPd584j999vzFVnf75ZHOS8VgzHKRq5OYahBcwthwI+CkUECzWMBxvPhU549PQWmey0OzLGCS0VTyhDNqHDXtfiExpFxW8KNsGOsnuEeKOe+dT/lHTDJq5nFSndmd93g55TsYE5kbmvoE5OxaNe0Gg/6gCXwbhC0IUBv70+3OZzLLWZmBNExQraNwUJhJRZXhTID1SamhoGxBU4gclDQDPamagS1+55gZTnLlnjS4YW8qKpppvcxiV1kPoNdzNXlXLipNsjupuCxKA5JdGiWlwCbH9fbwjCtgRiwdoExx1ytmc6ooM27HKy6Gu4at7/tEwhnLs4y6fZFYURuFk8ptT5cK6gYqIiAxRFCZCghCong6NxfWrgoXYO4WXlxJiGp+WBc6R9aIh/e5YnJazKk0eVYFQ9v2gJuaPg6GN9OhvdfPfa74OQgbjf5viNccg5HFV6a1WzMYDkYrvq4xvG7t7i5cv7Lb4GjYDx0+GAZ7u+0FbqLX6A3qoRB9QHvoK9pHY8TQT/QL/UF/vbfeN+/A+35Z2tloNa/QSnjkH45HMuM=</latexit><latexit sha1_base64="Bw6B3M5YjcGyF/3cXJQj8eOZs2I=">AAADonicfVJda9swFFXjfXTeR9PtcS9i3iCFEeLkYYW9FDbG2FPLmrZghSAr146ILHuS3Da4+kf7NXsb24+Z7Lq0SbteEBzOvUfn3suNC8G1GQx+b3S8Bw8fPd584j999vzFVnf75ZHOS8VgzHKRq5OYahBcwthwI+CkUECzWMBxvPhU549PQWmey0OzLGCS0VTyhDNqHDXtfiExpFxW8KNsGOsnuEeKOe+dT/lHTDJq5nFSndmd93g55TsYE5kbmvoE5OxaNe0Gg/6gCXwbhC0IUBv70+3OZzLLWZmBNExQraNwUJhJRZXhTID1SamhoGxBU4gclDQDPamagS1+55gZTnLlnjS4YW8qKpppvcxiV1kPoNdzNXlXLipNsjupuCxKA5JdGiWlwCbH9fbwjCtgRiwdoExx1ytmc6ooM27HKy6Gu4at7/tEwhnLs4y6fZFYURuFk8ptT5cK6gYqIiAxRFCZCghCong6NxfWrgoXYO4WXlxJiGp+WBc6R9aIh/e5YnJazKk0eVYFQ9v2gJuaPg6GN9OhvdfPfa74OQgbjf5viNccg5HFV6a1WzMYDkYrvq4xvG7t7i5cv7Lb4GjYDx0+GAZ7u+0FbqLX6A3qoRB9QHvoK9pHY8TQT/QL/UF/vbfeN+/A+35Z2tloNa/QSnjkH45HMuM=</latexit><latexit sha1_base64="Bw6B3M5YjcGyF/3cXJQj8eOZs2I=">AAADonicfVJda9swFFXjfXTeR9PtcS9i3iCFEeLkYYW9FDbG2FPLmrZghSAr146ILHuS3Da4+kf7NXsb24+Z7Lq0SbteEBzOvUfn3suNC8G1GQx+b3S8Bw8fPd584j999vzFVnf75ZHOS8VgzHKRq5OYahBcwthwI+CkUECzWMBxvPhU549PQWmey0OzLGCS0VTyhDNqHDXtfiExpFxW8KNsGOsnuEeKOe+dT/lHTDJq5nFSndmd93g55TsYE5kbmvoE5OxaNe0Gg/6gCXwbhC0IUBv70+3OZzLLWZmBNExQraNwUJhJRZXhTID1SamhoGxBU4gclDQDPamagS1+55gZTnLlnjS4YW8qKpppvcxiV1kPoNdzNXlXLipNsjupuCxKA5JdGiWlwCbH9fbwjCtgRiwdoExx1ytmc6ooM27HKy6Gu4at7/tEwhnLs4y6fZFYURuFk8ptT5cK6gYqIiAxRFCZCghCong6NxfWrgoXYO4WXlxJiGp+WBc6R9aIh/e5YnJazKk0eVYFQ9v2gJuaPg6GN9OhvdfPfa74OQgbjf5viNccg5HFV6a1WzMYDkYrvq4xvG7t7i5cv7Lb4GjYDx0+GAZ7u+0FbqLX6A3qoRB9QHvoK9pHY8TQT/QL/UF/vbfeN+/A+35Z2tloNa/QSnjkH45HMuM=</latexit><latexit sha1_base64="Bw6B3M5YjcGyF/3cXJQj8eOZs2I=">AAADonicfVJda9swFFXjfXTeR9PtcS9i3iCFEeLkYYW9FDbG2FPLmrZghSAr146ILHuS3Da4+kf7NXsb24+Z7Lq0SbteEBzOvUfn3suNC8G1GQx+b3S8Bw8fPd584j999vzFVnf75ZHOS8VgzHKRq5OYahBcwthwI+CkUECzWMBxvPhU549PQWmey0OzLGCS0VTyhDNqHDXtfiExpFxW8KNsGOsnuEeKOe+dT/lHTDJq5nFSndmd93g55TsYE5kbmvoE5OxaNe0Gg/6gCXwbhC0IUBv70+3OZzLLWZmBNExQraNwUJhJRZXhTID1SamhoGxBU4gclDQDPamagS1+55gZTnLlnjS4YW8qKpppvcxiV1kPoNdzNXlXLipNsjupuCxKA5JdGiWlwCbH9fbwjCtgRiwdoExx1ytmc6ooM27HKy6Gu4at7/tEwhnLs4y6fZFYURuFk8ptT5cK6gYqIiAxRFCZCghCong6NxfWrgoXYO4WXlxJiGp+WBc6R9aIh/e5YnJazKk0eVYFQ9v2gJuaPg6GN9OhvdfPfa74OQgbjf5viNccg5HFV6a1WzMYDkYrvq4xvG7t7i5cv7Lb4GjYDx0+GAZ7u+0FbqLX6A3qoRB9QHvoK9pHY8TQT/QL/UF/vbfeN+/A+35Z2tloNa/QSnjkH45HMuM=</latexit>

�(·;w)<latexit sha1_base64="4HW84KISJFbuhOP23qB1asQocjo=">AAADdXicfZJNaxsxEIYVbz/S7ZeTnkopiK5b0ovrtQ8J9BJoDz2mUCcByxitPGsLa7WLNJvUbER/Ta/t7+kv6bXajQOxk2ZAMMzMq+eVmKRQ0mKv92erFdy7/+Dh9qPw8ZOnz563d3aPbV4aAUORq9ycJtyCkhqGKFHBaWGAZ4mCk2Txqe6fnIGxMtffcFnAOOMzLVMpOPrSpP2yw4q53GNimuNHyjKO8yStzt37zqQd9bq9JujNJF4lEVnF0WSn9ZlNc1FmoFEobu0o7hU4rrhBKRS4kJUWCi4WfAYjn2qegR1XzRscfesrU5rmxh+NtKleV1Q8s3aZJX6yNmk3e3Xxtt6oxPRgXEldlAhaXILSUlHMaf0hdCoNCFRLn3BhpPdKxZwbLtB/2xoFpTfswjBkGs5FnmVcTyuWGO5G8bhioG1poDZQMQUpMsX1TEEUMyNnc7xwbl24ALxdeHElYaa5YVPoiaIR9++iUnZWzLnGPKuivlt5oM1Ml0b96+3Y3cnzlxv5HZQbDf4PpBvEaODoFbSmNQ+j0WCN643RTbTfu3hzy24mx/1u7POv/ejwYLWB2+QVeUP2SEz2ySH5Qo7IkAjyg/wkv8jv1t/gddAJ3l2OtrZWmhdkLYIP/wD/fCFR</latexit><latexit sha1_base64="4HW84KISJFbuhOP23qB1asQocjo=">AAADdXicfZJNaxsxEIYVbz/S7ZeTnkopiK5b0ovrtQ8J9BJoDz2mUCcByxitPGsLa7WLNJvUbER/Ta/t7+kv6bXajQOxk2ZAMMzMq+eVmKRQ0mKv92erFdy7/+Dh9qPw8ZOnz563d3aPbV4aAUORq9ycJtyCkhqGKFHBaWGAZ4mCk2Txqe6fnIGxMtffcFnAOOMzLVMpOPrSpP2yw4q53GNimuNHyjKO8yStzt37zqQd9bq9JujNJF4lEVnF0WSn9ZlNc1FmoFEobu0o7hU4rrhBKRS4kJUWCi4WfAYjn2qegR1XzRscfesrU5rmxh+NtKleV1Q8s3aZJX6yNmk3e3Xxtt6oxPRgXEldlAhaXILSUlHMaf0hdCoNCFRLn3BhpPdKxZwbLtB/2xoFpTfswjBkGs5FnmVcTyuWGO5G8bhioG1poDZQMQUpMsX1TEEUMyNnc7xwbl24ALxdeHElYaa5YVPoiaIR9++iUnZWzLnGPKuivlt5oM1Ml0b96+3Y3cnzlxv5HZQbDf4PpBvEaODoFbSmNQ+j0WCN643RTbTfu3hzy24mx/1u7POv/ejwYLWB2+QVeUP2SEz2ySH5Qo7IkAjyg/wkv8jv1t/gddAJ3l2OtrZWmhdkLYIP/wD/fCFR</latexit><latexit sha1_base64="4HW84KISJFbuhOP23qB1asQocjo=">AAADdXicfZJNaxsxEIYVbz/S7ZeTnkopiK5b0ovrtQ8J9BJoDz2mUCcByxitPGsLa7WLNJvUbER/Ta/t7+kv6bXajQOxk2ZAMMzMq+eVmKRQ0mKv92erFdy7/+Dh9qPw8ZOnz563d3aPbV4aAUORq9ycJtyCkhqGKFHBaWGAZ4mCk2Txqe6fnIGxMtffcFnAOOMzLVMpOPrSpP2yw4q53GNimuNHyjKO8yStzt37zqQd9bq9JujNJF4lEVnF0WSn9ZlNc1FmoFEobu0o7hU4rrhBKRS4kJUWCi4WfAYjn2qegR1XzRscfesrU5rmxh+NtKleV1Q8s3aZJX6yNmk3e3Xxtt6oxPRgXEldlAhaXILSUlHMaf0hdCoNCFRLn3BhpPdKxZwbLtB/2xoFpTfswjBkGs5FnmVcTyuWGO5G8bhioG1poDZQMQUpMsX1TEEUMyNnc7xwbl24ALxdeHElYaa5YVPoiaIR9++iUnZWzLnGPKuivlt5oM1Ml0b96+3Y3cnzlxv5HZQbDf4PpBvEaODoFbSmNQ+j0WCN643RTbTfu3hzy24mx/1u7POv/ejwYLWB2+QVeUP2SEz2ySH5Qo7IkAjyg/wkv8jv1t/gddAJ3l2OtrZWmhdkLYIP/wD/fCFR</latexit><latexit sha1_base64="4HW84KISJFbuhOP23qB1asQocjo=">AAADdXicfZJNaxsxEIYVbz/S7ZeTnkopiK5b0ovrtQ8J9BJoDz2mUCcByxitPGsLa7WLNJvUbER/Ta/t7+kv6bXajQOxk2ZAMMzMq+eVmKRQ0mKv92erFdy7/+Dh9qPw8ZOnz563d3aPbV4aAUORq9ycJtyCkhqGKFHBaWGAZ4mCk2Txqe6fnIGxMtffcFnAOOMzLVMpOPrSpP2yw4q53GNimuNHyjKO8yStzt37zqQd9bq9JujNJF4lEVnF0WSn9ZlNc1FmoFEobu0o7hU4rrhBKRS4kJUWCi4WfAYjn2qegR1XzRscfesrU5rmxh+NtKleV1Q8s3aZJX6yNmk3e3Xxtt6oxPRgXEldlAhaXILSUlHMaf0hdCoNCFRLn3BhpPdKxZwbLtB/2xoFpTfswjBkGs5FnmVcTyuWGO5G8bhioG1poDZQMQUpMsX1TEEUMyNnc7xwbl24ALxdeHElYaa5YVPoiaIR9++iUnZWzLnGPKuivlt5oM1Ml0b96+3Y3cnzlxv5HZQbDf4PpBvEaODoFbSmNQ+j0WCN643RTbTfu3hzy24mx/1u7POv/ejwYLWB2+QVeUP2SEz2ySH5Qo7IkAjyg/wkv8jv1t/gddAJ3l2OtrZWmhdkLYIP/wD/fCFR</latexit>

Forward Propagation(Compute obj)

<latexit sha1_base64="7OB+Bw5IysanGRJyNebQHTtZ2Hk=">AAADpnicfVJNaxsxEFWy/Ui3X0577EV0U3AvxmsfmmMgpZQeigt1HLCMq5XHtmJ9LJI2idnoP/XXFHpqf0q1GxtiJ82A4DEzT+/NMFkuuHXt9u+d3ejBw0eP957ET589f/Gysf/qxOrCMOgzLbQ5zagFwRX0HXcCTnMDVGYCBtniuKoPzsFYrtV3t8xhJOlM8Sln1IXUuPGFZDDjqmSgHBgff9LmgpoJ7hmd01ndREjcPNYyLxzgA+Lg0hlZ6uzMH7yPCajJmjtuJO1Wuw58G6QrkKBV9Mb7ux/JRLNCBj4T1Nph2s7dqKTGcSbAx6SwkFO2oDMYBqioBDsq66E9fhcyEzzVJjzlcJ29ySiptHYps9ApqZvb7VqVvKs2LNz0cFRyVQ2s2LXQtBDYaVxtEE+4AebEMgDKDA9eMZtTQ1lYwqaK48Gwj+OYKLhgWkoatkUyQ/0wHZVhd7YwUBkoiYCpI4KqmYAkJYbP5u7K+03iAtzdxKs1hZj6h21iUGQ1uXOfKibn+Zwqp2WZdPzKA657Wjjp3Cyn/l698LnhlyD8sPt/QbylmHQ9XotWavVgOOlu6AZjeFs63F26fWW3wUmnlQb8rZMcHa4ucA+9QW9RE6XoAzpCn1EP9RFDP9Ev9Af9jZrR16gfDa5bd3dWnNdoI6If/wC2DTVm</latexit><latexit sha1_base64="7OB+Bw5IysanGRJyNebQHTtZ2Hk=">AAADpnicfVJNaxsxEFWy/Ui3X0577EV0U3AvxmsfmmMgpZQeigt1HLCMq5XHtmJ9LJI2idnoP/XXFHpqf0q1GxtiJ82A4DEzT+/NMFkuuHXt9u+d3ejBw0eP957ET589f/Gysf/qxOrCMOgzLbQ5zagFwRX0HXcCTnMDVGYCBtniuKoPzsFYrtV3t8xhJOlM8Sln1IXUuPGFZDDjqmSgHBgff9LmgpoJ7hmd01ndREjcPNYyLxzgA+Lg0hlZ6uzMH7yPCajJmjtuJO1Wuw58G6QrkKBV9Mb7ux/JRLNCBj4T1Nph2s7dqKTGcSbAx6SwkFO2oDMYBqioBDsq66E9fhcyEzzVJjzlcJ29ySiptHYps9ApqZvb7VqVvKs2LNz0cFRyVQ2s2LXQtBDYaVxtEE+4AebEMgDKDA9eMZtTQ1lYwqaK48Gwj+OYKLhgWkoatkUyQ/0wHZVhd7YwUBkoiYCpI4KqmYAkJYbP5u7K+03iAtzdxKs1hZj6h21iUGQ1uXOfKibn+Zwqp2WZdPzKA657Wjjp3Cyn/l698LnhlyD8sPt/QbylmHQ9XotWavVgOOlu6AZjeFs63F26fWW3wUmnlQb8rZMcHa4ucA+9QW9RE6XoAzpCn1EP9RFDP9Ev9Af9jZrR16gfDa5bd3dWnNdoI6If/wC2DTVm</latexit><latexit sha1_base64="7OB+Bw5IysanGRJyNebQHTtZ2Hk=">AAADpnicfVJNaxsxEFWy/Ui3X0577EV0U3AvxmsfmmMgpZQeigt1HLCMq5XHtmJ9LJI2idnoP/XXFHpqf0q1GxtiJ82A4DEzT+/NMFkuuHXt9u+d3ejBw0eP957ET589f/Gysf/qxOrCMOgzLbQ5zagFwRX0HXcCTnMDVGYCBtniuKoPzsFYrtV3t8xhJOlM8Sln1IXUuPGFZDDjqmSgHBgff9LmgpoJ7hmd01ndREjcPNYyLxzgA+Lg0hlZ6uzMH7yPCajJmjtuJO1Wuw58G6QrkKBV9Mb7ux/JRLNCBj4T1Nph2s7dqKTGcSbAx6SwkFO2oDMYBqioBDsq66E9fhcyEzzVJjzlcJ29ySiptHYps9ApqZvb7VqVvKs2LNz0cFRyVQ2s2LXQtBDYaVxtEE+4AebEMgDKDA9eMZtTQ1lYwqaK48Gwj+OYKLhgWkoatkUyQ/0wHZVhd7YwUBkoiYCpI4KqmYAkJYbP5u7K+03iAtzdxKs1hZj6h21iUGQ1uXOfKibn+Zwqp2WZdPzKA657Wjjp3Cyn/l698LnhlyD8sPt/QbylmHQ9XotWavVgOOlu6AZjeFs63F26fWW3wUmnlQb8rZMcHa4ucA+9QW9RE6XoAzpCn1EP9RFDP9Ev9Af9jZrR16gfDa5bd3dWnNdoI6If/wC2DTVm</latexit><latexit sha1_base64="7OB+Bw5IysanGRJyNebQHTtZ2Hk=">AAADpnicfVJNaxsxEFWy/Ui3X0577EV0U3AvxmsfmmMgpZQeigt1HLCMq5XHtmJ9LJI2idnoP/XXFHpqf0q1GxtiJ82A4DEzT+/NMFkuuHXt9u+d3ejBw0eP957ET589f/Gysf/qxOrCMOgzLbQ5zagFwRX0HXcCTnMDVGYCBtniuKoPzsFYrtV3t8xhJOlM8Sln1IXUuPGFZDDjqmSgHBgff9LmgpoJ7hmd01ndREjcPNYyLxzgA+Lg0hlZ6uzMH7yPCajJmjtuJO1Wuw58G6QrkKBV9Mb7ux/JRLNCBj4T1Nph2s7dqKTGcSbAx6SwkFO2oDMYBqioBDsq66E9fhcyEzzVJjzlcJ29ySiptHYps9ApqZvb7VqVvKs2LNz0cFRyVQ2s2LXQtBDYaVxtEE+4AebEMgDKDA9eMZtTQ1lYwqaK48Gwj+OYKLhgWkoatkUyQ/0wHZVhd7YwUBkoiYCpI4KqmYAkJYbP5u7K+03iAtzdxKs1hZj6h21iUGQ1uXOfKibn+Zwqp2WZdPzKA657Wjjp3Cyn/l698LnhlyD8sPt/QbylmHQ9XotWavVgOOlu6AZjeFs63F26fWW3wUmnlQb8rZMcHa4ucA+9QW9RE6XoAzpCn1EP9RFDP9Ev9Af9jZrR16gfDa5bd3dWnNdoI6If/wC2DTVm</latexit>

Backward Propagation(Compute Gradient @obj

@w )<latexit sha1_base64="eHt4RlacyTBkwSNbNglROd3M4wE=">AAAD1XicfZLBb9MwFMbdhcEIsHVw5GKRIY1L1bQHdpwYEhyLRNdJdVU5zktq6tiR46yrMt8QV/4+TvwrOFkrtd2YpUhP773Pvy+fHOWCF6bb/dPa857sP3128Nx/8fLV4VH7+PVloUrNYMiUUPoqogUILmFouBFwlWugWSRgFM0v6vnoGnTBlfxuljlMMppKnnBGjWtN20sSQcplxUAa0Nb/RNl8QXWMB1rlNG22CPFPL1SWlwbwF01j7nbxCUk0ZRXJqTacCkwM3BidVSr6Ye1GO6NmFiXVwtqTDz4BGa9R03bQ7XSbg+8X4aoI0OoMpsd7n0msWJk5PRO0KMZhNzeTqiYxAdYnZQG5809TGLtS0gyKSdVkZPF714lxorT7nP2mu6moaFYUyyxym7XlYndWNx+ajUuTnE0qLut4JLsDJaXARuE6cBxzDcyIpSso09x5xWxGXXYuhG2K4c6w9X2fSFgwlWXUpUUiTe04nFQuu6LUUBuoiIDEEEFlKiAIiebpzNxauy2cg3lYeLuWEN3csCt0RNaIe49RMbnOZ1QalVVBz6484Gang4Pe5ji0j/Lc5ZrfgLDj/v+BeIcY9C1eQ2ta82M46G9xnTG8i3bvLtx9ZfeLy14ndPW3XnB+tnqBB+gteodOUYg+onP0FQ3QEDH0t7XfOmwdeSPPej+9X3ere62V5g3aOt7vfz3aRs4=</latexit><latexit sha1_base64="eHt4RlacyTBkwSNbNglROd3M4wE=">AAAD1XicfZLBb9MwFMbdhcEIsHVw5GKRIY1L1bQHdpwYEhyLRNdJdVU5zktq6tiR46yrMt8QV/4+TvwrOFkrtd2YpUhP773Pvy+fHOWCF6bb/dPa857sP3128Nx/8fLV4VH7+PVloUrNYMiUUPoqogUILmFouBFwlWugWSRgFM0v6vnoGnTBlfxuljlMMppKnnBGjWtN20sSQcplxUAa0Nb/RNl8QXWMB1rlNG22CPFPL1SWlwbwF01j7nbxCUk0ZRXJqTacCkwM3BidVSr6Ye1GO6NmFiXVwtqTDz4BGa9R03bQ7XSbg+8X4aoI0OoMpsd7n0msWJk5PRO0KMZhNzeTqiYxAdYnZQG5809TGLtS0gyKSdVkZPF714lxorT7nP2mu6moaFYUyyxym7XlYndWNx+ajUuTnE0qLut4JLsDJaXARuE6cBxzDcyIpSso09x5xWxGXXYuhG2K4c6w9X2fSFgwlWXUpUUiTe04nFQuu6LUUBuoiIDEEEFlKiAIiebpzNxauy2cg3lYeLuWEN3csCt0RNaIe49RMbnOZ1QalVVBz6484Gang4Pe5ji0j/Lc5ZrfgLDj/v+BeIcY9C1eQ2ta82M46G9xnTG8i3bvLtx9ZfeLy14ndPW3XnB+tnqBB+gteodOUYg+onP0FQ3QEDH0t7XfOmwdeSPPej+9X3ere62V5g3aOt7vfz3aRs4=</latexit><latexit sha1_base64="eHt4RlacyTBkwSNbNglROd3M4wE=">AAAD1XicfZLBb9MwFMbdhcEIsHVw5GKRIY1L1bQHdpwYEhyLRNdJdVU5zktq6tiR46yrMt8QV/4+TvwrOFkrtd2YpUhP773Pvy+fHOWCF6bb/dPa857sP3128Nx/8fLV4VH7+PVloUrNYMiUUPoqogUILmFouBFwlWugWSRgFM0v6vnoGnTBlfxuljlMMppKnnBGjWtN20sSQcplxUAa0Nb/RNl8QXWMB1rlNG22CPFPL1SWlwbwF01j7nbxCUk0ZRXJqTacCkwM3BidVSr6Ye1GO6NmFiXVwtqTDz4BGa9R03bQ7XSbg+8X4aoI0OoMpsd7n0msWJk5PRO0KMZhNzeTqiYxAdYnZQG5809TGLtS0gyKSdVkZPF714lxorT7nP2mu6moaFYUyyxym7XlYndWNx+ajUuTnE0qLut4JLsDJaXARuE6cBxzDcyIpSso09x5xWxGXXYuhG2K4c6w9X2fSFgwlWXUpUUiTe04nFQuu6LUUBuoiIDEEEFlKiAIiebpzNxauy2cg3lYeLuWEN3csCt0RNaIe49RMbnOZ1QalVVBz6484Gang4Pe5ji0j/Lc5ZrfgLDj/v+BeIcY9C1eQ2ta82M46G9xnTG8i3bvLtx9ZfeLy14ndPW3XnB+tnqBB+gteodOUYg+onP0FQ3QEDH0t7XfOmwdeSPPej+9X3ere62V5g3aOt7vfz3aRs4=</latexit><latexit sha1_base64="eHt4RlacyTBkwSNbNglROd3M4wE=">AAAD1XicfZLBb9MwFMbdhcEIsHVw5GKRIY1L1bQHdpwYEhyLRNdJdVU5zktq6tiR46yrMt8QV/4+TvwrOFkrtd2YpUhP773Pvy+fHOWCF6bb/dPa857sP3128Nx/8fLV4VH7+PVloUrNYMiUUPoqogUILmFouBFwlWugWSRgFM0v6vnoGnTBlfxuljlMMppKnnBGjWtN20sSQcplxUAa0Nb/RNl8QXWMB1rlNG22CPFPL1SWlwbwF01j7nbxCUk0ZRXJqTacCkwM3BidVSr6Ye1GO6NmFiXVwtqTDz4BGa9R03bQ7XSbg+8X4aoI0OoMpsd7n0msWJk5PRO0KMZhNzeTqiYxAdYnZQG5809TGLtS0gyKSdVkZPF714lxorT7nP2mu6moaFYUyyxym7XlYndWNx+ajUuTnE0qLut4JLsDJaXARuE6cBxzDcyIpSso09x5xWxGXXYuhG2K4c6w9X2fSFgwlWXUpUUiTe04nFQuu6LUUBuoiIDEEEFlKiAIiebpzNxauy2cg3lYeLuWEN3csCt0RNaIe49RMbnOZ1QalVVBz6484Gang4Pe5ji0j/Lc5ZrfgLDj/v+BeIcY9C1eQ2ta82M46G9xnTG8i3bvLtx9ZfeLy14ndPW3XnB+tnqBB+gteodOUYg+onP0FQ3QEDH0t7XfOmwdeSPPej+9X3ere62V5g3aOt7vfz3aRs4=</latexit>

(a)

Net Instance

ErrorFunction

WL(ei;w)<latexit sha1_base64="elVEo36980ERLdo+mzttF+DlmKE=">AAADoHicfVJNTxsxEDVsP+j2K7THXixWleglyiYHkHpBKgcOlQpSQ1DjKPI6s4kVr3drzwLR4j/UX9Mr/TX1LkElgTKSpaeZefOeR5MUSlrsdK43NoMnT58933oRvnz1+s3b1va7U5uXRkBf5Co3Zwm3oKSGPkpUcFYY4FmiYJDMv9T1wTkYK3P9HRcFjDI+1TKVgqNPjVuHLIGp1BX8LJuMCxnCJZqsGnx1uzCWnynLOM6StLpwnyhlOkc+DRnoyT/OuBV12p0m6H0QL0FElnE83t48ZJNclBloFIpbO4w7BY4qblAKBd5EaaHgYs6nMPRQ8wzsqGq+6+hHn5nQNDf+aaRN9i6j4pm1iyzxnbV1u16rkw/VhiWm+6NK6qJE0OJGKC0VxZzWu6MTaUCgWnjAhZHeKxUzbrhAv+EVFZTesAvDkGm4EHmWcb8vlhjuhvGo8tuzpYHaQMUUpMgU11MFUcyMnM7wyrlV4hzwYeLVLYWZZsI60SuKhtx9TJWy82LGNeZZFXXd0gNteto06t4tx+5RPT/cyEtQbtj7vyBdU4x6jt6K1mrNx2jUW9H1xui6tL+7eP3K7oPTbjv2+KQbHewvL3CLfCA7ZJfEZI8ckCNyTPpEkF/kN7kmf4Kd4Cj4FpzctG5uLDnvyUoEP/4Cz+YzVg==</latexit><latexit sha1_base64="elVEo36980ERLdo+mzttF+DlmKE=">AAADoHicfVJNTxsxEDVsP+j2K7THXixWleglyiYHkHpBKgcOlQpSQ1DjKPI6s4kVr3drzwLR4j/UX9Mr/TX1LkElgTKSpaeZefOeR5MUSlrsdK43NoMnT58933oRvnz1+s3b1va7U5uXRkBf5Co3Zwm3oKSGPkpUcFYY4FmiYJDMv9T1wTkYK3P9HRcFjDI+1TKVgqNPjVuHLIGp1BX8LJuMCxnCJZqsGnx1uzCWnynLOM6StLpwnyhlOkc+DRnoyT/OuBV12p0m6H0QL0FElnE83t48ZJNclBloFIpbO4w7BY4qblAKBd5EaaHgYs6nMPRQ8wzsqGq+6+hHn5nQNDf+aaRN9i6j4pm1iyzxnbV1u16rkw/VhiWm+6NK6qJE0OJGKC0VxZzWu6MTaUCgWnjAhZHeKxUzbrhAv+EVFZTesAvDkGm4EHmWcb8vlhjuhvGo8tuzpYHaQMUUpMgU11MFUcyMnM7wyrlV4hzwYeLVLYWZZsI60SuKhtx9TJWy82LGNeZZFXXd0gNteto06t4tx+5RPT/cyEtQbtj7vyBdU4x6jt6K1mrNx2jUW9H1xui6tL+7eP3K7oPTbjv2+KQbHewvL3CLfCA7ZJfEZI8ckCNyTPpEkF/kN7kmf4Kd4Cj4FpzctG5uLDnvyUoEP/4Cz+YzVg==</latexit><latexit sha1_base64="elVEo36980ERLdo+mzttF+DlmKE=">AAADoHicfVJNTxsxEDVsP+j2K7THXixWleglyiYHkHpBKgcOlQpSQ1DjKPI6s4kVr3drzwLR4j/UX9Mr/TX1LkElgTKSpaeZefOeR5MUSlrsdK43NoMnT58933oRvnz1+s3b1va7U5uXRkBf5Co3Zwm3oKSGPkpUcFYY4FmiYJDMv9T1wTkYK3P9HRcFjDI+1TKVgqNPjVuHLIGp1BX8LJuMCxnCJZqsGnx1uzCWnynLOM6StLpwnyhlOkc+DRnoyT/OuBV12p0m6H0QL0FElnE83t48ZJNclBloFIpbO4w7BY4qblAKBd5EaaHgYs6nMPRQ8wzsqGq+6+hHn5nQNDf+aaRN9i6j4pm1iyzxnbV1u16rkw/VhiWm+6NK6qJE0OJGKC0VxZzWu6MTaUCgWnjAhZHeKxUzbrhAv+EVFZTesAvDkGm4EHmWcb8vlhjuhvGo8tuzpYHaQMUUpMgU11MFUcyMnM7wyrlV4hzwYeLVLYWZZsI60SuKhtx9TJWy82LGNeZZFXXd0gNteto06t4tx+5RPT/cyEtQbtj7vyBdU4x6jt6K1mrNx2jUW9H1xui6tL+7eP3K7oPTbjv2+KQbHewvL3CLfCA7ZJfEZI8ckCNyTPpEkF/kN7kmf4Kd4Cj4FpzctG5uLDnvyUoEP/4Cz+YzVg==</latexit><latexit sha1_base64="elVEo36980ERLdo+mzttF+DlmKE=">AAADoHicfVJNTxsxEDVsP+j2K7THXixWleglyiYHkHpBKgcOlQpSQ1DjKPI6s4kVr3drzwLR4j/UX9Mr/TX1LkElgTKSpaeZefOeR5MUSlrsdK43NoMnT58933oRvnz1+s3b1va7U5uXRkBf5Co3Zwm3oKSGPkpUcFYY4FmiYJDMv9T1wTkYK3P9HRcFjDI+1TKVgqNPjVuHLIGp1BX8LJuMCxnCJZqsGnx1uzCWnynLOM6StLpwnyhlOkc+DRnoyT/OuBV12p0m6H0QL0FElnE83t48ZJNclBloFIpbO4w7BY4qblAKBd5EaaHgYs6nMPRQ8wzsqGq+6+hHn5nQNDf+aaRN9i6j4pm1iyzxnbV1u16rkw/VhiWm+6NK6qJE0OJGKC0VxZzWu6MTaUCgWnjAhZHeKxUzbrhAv+EVFZTesAvDkGm4EHmWcb8vlhjuhvGo8tuzpYHaQMUUpMgU11MFUcyMnM7wyrlV4hzwYeLVLYWZZsI60SuKhtx9TJWy82LGNeZZFXXd0gNteto06t4tx+5RPT/cyEtQbtj7vyBdU4x6jt6K1mrNx2jUW9H1xui6tL+7eP3K7oPTbjv2+KQbHewvL3CLfCA7ZJfEZI8ckCNyTPpEkF/kN7kmf4Kd4Cj4FpzctG5uLDnvyUoEP/4Cz+YzVg==</latexit>

(ei, 0)<latexit sha1_base64="nCOOSvQ+LlWfE+8cVyNCz1vAYOk=">AAADZHicfZLfatswFMbVeH86r1vblV0Nhpg76GAE27loLwvbxS47WNoy2wRZOUlEJNlIctfg6i12u73XXmDPMdl1oXG7Cgwf55xPv0/m5CVn2oThn42B9+jxk6ebz/znWy9ebu/svjrVRaUojGnBC3WeEw2cSRgbZjiclwqIyDmc5ctPTf/sApRmhfxmViVkgswlmzFKjCt93z+ACfuIww/7k50gHIbtwXdF1IkAdedksjv4nE4LWgmQhnKidRKFpclqogyjHKyfVhpKQpdkDomTkgjQWd1Gtvi9q0zxrFDukwa31duOmgitVyJ3k4KYhe73muJ9vaQys6OsZrKsDEh6DZpVHJsCN+/HU6aAGr5yglDFXFZMF0QRatxfWqMY5gJb3/dTCT9oIQSR0zrNFbFJlNUpSF0paALUKYeZSTmRcw5BlCo2X5gra9eNSzD3G69uLKlqb+gbHZG25vghKk4vygWRphB1ENsuA25nhjiIb7cj+yDPXa7YJXCbjP4PxD1iMLL4BtrQ2ofhYLTGdcFwH+32Lupv2V1xGg8jp7/GwfFRt4Gb6A16hw5QhA7RMfqCTtAYUSTRT/QL/R789ba8Pe/19ehgo/PsobXjvf0HPzoZ+A==</latexit><latexit sha1_base64="nCOOSvQ+LlWfE+8cVyNCz1vAYOk=">AAADZHicfZLfatswFMbVeH86r1vblV0Nhpg76GAE27loLwvbxS47WNoy2wRZOUlEJNlIctfg6i12u73XXmDPMdl1oXG7Cgwf55xPv0/m5CVn2oThn42B9+jxk6ebz/znWy9ebu/svjrVRaUojGnBC3WeEw2cSRgbZjiclwqIyDmc5ctPTf/sApRmhfxmViVkgswlmzFKjCt93z+ACfuIww/7k50gHIbtwXdF1IkAdedksjv4nE4LWgmQhnKidRKFpclqogyjHKyfVhpKQpdkDomTkgjQWd1Gtvi9q0zxrFDukwa31duOmgitVyJ3k4KYhe73muJ9vaQys6OsZrKsDEh6DZpVHJsCN+/HU6aAGr5yglDFXFZMF0QRatxfWqMY5gJb3/dTCT9oIQSR0zrNFbFJlNUpSF0paALUKYeZSTmRcw5BlCo2X5gra9eNSzD3G69uLKlqb+gbHZG25vghKk4vygWRphB1ENsuA25nhjiIb7cj+yDPXa7YJXCbjP4PxD1iMLL4BtrQ2ofhYLTGdcFwH+32Lupv2V1xGg8jp7/GwfFRt4Gb6A16hw5QhA7RMfqCTtAYUSTRT/QL/R789ba8Pe/19ehgo/PsobXjvf0HPzoZ+A==</latexit><latexit sha1_base64="nCOOSvQ+LlWfE+8cVyNCz1vAYOk=">AAADZHicfZLfatswFMbVeH86r1vblV0Nhpg76GAE27loLwvbxS47WNoy2wRZOUlEJNlIctfg6i12u73XXmDPMdl1oXG7Cgwf55xPv0/m5CVn2oThn42B9+jxk6ebz/znWy9ebu/svjrVRaUojGnBC3WeEw2cSRgbZjiclwqIyDmc5ctPTf/sApRmhfxmViVkgswlmzFKjCt93z+ACfuIww/7k50gHIbtwXdF1IkAdedksjv4nE4LWgmQhnKidRKFpclqogyjHKyfVhpKQpdkDomTkgjQWd1Gtvi9q0zxrFDukwa31duOmgitVyJ3k4KYhe73muJ9vaQys6OsZrKsDEh6DZpVHJsCN+/HU6aAGr5yglDFXFZMF0QRatxfWqMY5gJb3/dTCT9oIQSR0zrNFbFJlNUpSF0paALUKYeZSTmRcw5BlCo2X5gra9eNSzD3G69uLKlqb+gbHZG25vghKk4vygWRphB1ENsuA25nhjiIb7cj+yDPXa7YJXCbjP4PxD1iMLL4BtrQ2ofhYLTGdcFwH+32Lupv2V1xGg8jp7/GwfFRt4Gb6A16hw5QhA7RMfqCTtAYUSTRT/QL/R789ba8Pe/19ehgo/PsobXjvf0HPzoZ+A==</latexit><latexit sha1_base64="nCOOSvQ+LlWfE+8cVyNCz1vAYOk=">AAADZHicfZLfatswFMbVeH86r1vblV0Nhpg76GAE27loLwvbxS47WNoy2wRZOUlEJNlIctfg6i12u73XXmDPMdl1oXG7Cgwf55xPv0/m5CVn2oThn42B9+jxk6ebz/znWy9ebu/svjrVRaUojGnBC3WeEw2cSRgbZjiclwqIyDmc5ctPTf/sApRmhfxmViVkgswlmzFKjCt93z+ACfuIww/7k50gHIbtwXdF1IkAdedksjv4nE4LWgmQhnKidRKFpclqogyjHKyfVhpKQpdkDomTkgjQWd1Gtvi9q0zxrFDukwa31duOmgitVyJ3k4KYhe73muJ9vaQys6OsZrKsDEh6DZpVHJsCN+/HU6aAGr5yglDFXFZMF0QRatxfWqMY5gJb3/dTCT9oIQSR0zrNFbFJlNUpSF0paALUKYeZSTmRcw5BlCo2X5gra9eNSzD3G69uLKlqb+gbHZG25vghKk4vygWRphB1ENsuA25nhjiIb7cj+yDPXa7YJXCbjP4PxD1iMLL4BtrQ2ofhYLTGdcFwH+32Lupv2V1xGg8jp7/GwfFRt4Gb6A16hw5QhA7RMfqCTtAYUSTRT/QL/R789ba8Pe/19ehgo/PsobXjvf0HPzoZ+A==</latexit>

minw

nX

i

WL(ei;w) + �D(w)<latexit sha1_base64="CS7lVLxi+bEy0kkNJOyOP7cqWJs=">AAAD0nicfZLPb9MwFMfdhh8jG9DBkYtFhFSEVDXtgUlcJrEDBw5DWtdJdYkc96W1ZjvBdrqVLAfElX8QiT8GJwta241ZivT03vv68/XLizPBje33f7fa3oOHjx7vPPF3954+e97Zf3Fq0lwzGLFUpPospgYEVzCy3Ao4yzRQGQsYx+cfq/p4CdrwVJ3YVQZTSeeKJ5xR61JRZ0limHNVwLe8zpQ+kVxFBZHULuKkuChLTEwuI/5VYWLh0mpZjD+XXYj4B3zT9Ra/w0Q47ozio+56nqjU0rlPQM1uKFEn6Pf69cG3g7AJAtSc42i/fURmKcslKMsENWYS9jM7Lai2nAlwtnMDGWXndA4TFyoqwUyLekAlfuMyM5yk2n3K4jq7riioNGYlY9dZOTfbtSp5V22S2+RgWnCV5RYUuwYlucA2xdW08YxrYFasXECZ5s4rZguqKbPun2xQLHeGS9/3iYILlkpJ3bxIrGk5CaeFm57JNVQGCiIgsW7Wai4gCInm84W9KstN4TnYu4VX/yRE1zdsCx2R1eLBfVRMltmCKpvKIhiUjQdc9/RwMFgvh+W9PHe55pcgysnw/0C8RQyGbisbaEWrH4aD4QbXGcPbaLd34faW3Q5OB73QxV8GweFBs4E76BV6jbooRO/RIfqEjtEIMfSn1W7ttva8E++798P7ed3abjWal2jjeL/+AmFoRQA=</latexit><latexit sha1_base64="CS7lVLxi+bEy0kkNJOyOP7cqWJs=">AAAD0nicfZLPb9MwFMfdhh8jG9DBkYtFhFSEVDXtgUlcJrEDBw5DWtdJdYkc96W1ZjvBdrqVLAfElX8QiT8GJwta241ZivT03vv68/XLizPBje33f7fa3oOHjx7vPPF3954+e97Zf3Fq0lwzGLFUpPospgYEVzCy3Ao4yzRQGQsYx+cfq/p4CdrwVJ3YVQZTSeeKJ5xR61JRZ0limHNVwLe8zpQ+kVxFBZHULuKkuChLTEwuI/5VYWLh0mpZjD+XXYj4B3zT9Ra/w0Q47ozio+56nqjU0rlPQM1uKFEn6Pf69cG3g7AJAtSc42i/fURmKcslKMsENWYS9jM7Lai2nAlwtnMDGWXndA4TFyoqwUyLekAlfuMyM5yk2n3K4jq7riioNGYlY9dZOTfbtSp5V22S2+RgWnCV5RYUuwYlucA2xdW08YxrYFasXECZ5s4rZguqKbPun2xQLHeGS9/3iYILlkpJ3bxIrGk5CaeFm57JNVQGCiIgsW7Wai4gCInm84W9KstN4TnYu4VX/yRE1zdsCx2R1eLBfVRMltmCKpvKIhiUjQdc9/RwMFgvh+W9PHe55pcgysnw/0C8RQyGbisbaEWrH4aD4QbXGcPbaLd34faW3Q5OB73QxV8GweFBs4E76BV6jbooRO/RIfqEjtEIMfSn1W7ttva8E++798P7ed3abjWal2jjeL/+AmFoRQA=</latexit><latexit sha1_base64="CS7lVLxi+bEy0kkNJOyOP7cqWJs=">AAAD0nicfZLPb9MwFMfdhh8jG9DBkYtFhFSEVDXtgUlcJrEDBw5DWtdJdYkc96W1ZjvBdrqVLAfElX8QiT8GJwta241ZivT03vv68/XLizPBje33f7fa3oOHjx7vPPF3954+e97Zf3Fq0lwzGLFUpPospgYEVzCy3Ao4yzRQGQsYx+cfq/p4CdrwVJ3YVQZTSeeKJ5xR61JRZ0limHNVwLe8zpQ+kVxFBZHULuKkuChLTEwuI/5VYWLh0mpZjD+XXYj4B3zT9Ra/w0Q47ozio+56nqjU0rlPQM1uKFEn6Pf69cG3g7AJAtSc42i/fURmKcslKMsENWYS9jM7Lai2nAlwtnMDGWXndA4TFyoqwUyLekAlfuMyM5yk2n3K4jq7riioNGYlY9dZOTfbtSp5V22S2+RgWnCV5RYUuwYlucA2xdW08YxrYFasXECZ5s4rZguqKbPun2xQLHeGS9/3iYILlkpJ3bxIrGk5CaeFm57JNVQGCiIgsW7Wai4gCInm84W9KstN4TnYu4VX/yRE1zdsCx2R1eLBfVRMltmCKpvKIhiUjQdc9/RwMFgvh+W9PHe55pcgysnw/0C8RQyGbisbaEWrH4aD4QbXGcPbaLd34faW3Q5OB73QxV8GweFBs4E76BV6jbooRO/RIfqEjtEIMfSn1W7ttva8E++798P7ed3abjWal2jjeL/+AmFoRQA=</latexit><latexit sha1_base64="CS7lVLxi+bEy0kkNJOyOP7cqWJs=">AAAD0nicfZLPb9MwFMfdhh8jG9DBkYtFhFSEVDXtgUlcJrEDBw5DWtdJdYkc96W1ZjvBdrqVLAfElX8QiT8GJwta241ZivT03vv68/XLizPBje33f7fa3oOHjx7vPPF3954+e97Zf3Fq0lwzGLFUpPospgYEVzCy3Ao4yzRQGQsYx+cfq/p4CdrwVJ3YVQZTSeeKJ5xR61JRZ0limHNVwLe8zpQ+kVxFBZHULuKkuChLTEwuI/5VYWLh0mpZjD+XXYj4B3zT9Ra/w0Q47ozio+56nqjU0rlPQM1uKFEn6Pf69cG3g7AJAtSc42i/fURmKcslKMsENWYS9jM7Lai2nAlwtnMDGWXndA4TFyoqwUyLekAlfuMyM5yk2n3K4jq7riioNGYlY9dZOTfbtSp5V22S2+RgWnCV5RYUuwYlucA2xdW08YxrYFasXECZ5s4rZguqKbPun2xQLHeGS9/3iYILlkpJ3bxIrGk5CaeFm57JNVQGCiIgsW7Wai4gCInm84W9KstN4TnYu4VX/yRE1zdsCx2R1eLBfVRMltmCKpvKIhiUjQdc9/RwMFgvh+W9PHe55pcgysnw/0C8RQyGbisbaEWrH4aD4QbXGcPbaLd34faW3Q5OB73QxV8GweFBs4E76BV6jbooRO/RIfqEjtEIMfSn1W7ttva8E++798P7ed3abjWal2jjeL/+AmFoRQA=</latexit>

Neural NetworkWL(·;w)

<latexit sha1_base64="E6V3HS16/idoqA6LIsbuWfFLtL8=">AAADfHicfZJNaxsxEIYVux+p+xGnPfYiui6klAavDW2gl0B76KGHFOo4YBmjlWdtYUm7SLNJzEbX/ppe2//SP1Oq3TgQO2kGBMPMvHpeiUlyJR12u3+2Gs179x883H7Uevzk6bOd9u7zY5cVVsBAZCqzJwl3oKSBAUpUcJJb4DpRMEwWn6r+8BSsk5n5jsscxprPjEyl4BhKkzbtMIRztLocfvV7TEwz/EiZ5jhP0vLMv+lM2lF3v1sHvZnEqyQiqzia7DY+s2kmCg0GheLOjeJujuOSW5RCgW+xwkHOxYLPYBRSwzW4cVk/xdPXoTKlaWbDMUjr6nVFybVzS52Eycqk2+xVxdt6owLTg3EpTV4gGHEJSgtFMaPVv9CptCBQLUPChZXBKxVzbrnA8HtrFJTBsG+1WszAmci05mZassRyP4rHJQPjCguVgZIpSJEpbmYKophZOZvjhffrwgXg7cKLKwmz9Q2bwkAUtbh3F5Wy03zODWa6jHp+5YHWM/s06l1vx/5OXrjcynNQftT/P5BuEKO+p1fQilY/jEb9NW4wRjfRYe/izS27mRz39uOQf+tFhwerDdwmL8krskdi8oEcki/kiAyIID/IT/KL/G78bXaab5vvLkcbWyvNC7IWzff/AD+MJJU=</latexit><latexit sha1_base64="E6V3HS16/idoqA6LIsbuWfFLtL8=">AAADfHicfZJNaxsxEIYVux+p+xGnPfYiui6klAavDW2gl0B76KGHFOo4YBmjlWdtYUm7SLNJzEbX/ppe2//SP1Oq3TgQO2kGBMPMvHpeiUlyJR12u3+2Gs179x883H7Uevzk6bOd9u7zY5cVVsBAZCqzJwl3oKSBAUpUcJJb4DpRMEwWn6r+8BSsk5n5jsscxprPjEyl4BhKkzbtMIRztLocfvV7TEwz/EiZ5jhP0vLMv+lM2lF3v1sHvZnEqyQiqzia7DY+s2kmCg0GheLOjeJujuOSW5RCgW+xwkHOxYLPYBRSwzW4cVk/xdPXoTKlaWbDMUjr6nVFybVzS52Eycqk2+xVxdt6owLTg3EpTV4gGHEJSgtFMaPVv9CptCBQLUPChZXBKxVzbrnA8HtrFJTBsG+1WszAmci05mZassRyP4rHJQPjCguVgZIpSJEpbmYKophZOZvjhffrwgXg7cKLKwmz9Q2bwkAUtbh3F5Wy03zODWa6jHp+5YHWM/s06l1vx/5OXrjcynNQftT/P5BuEKO+p1fQilY/jEb9NW4wRjfRYe/izS27mRz39uOQf+tFhwerDdwmL8krskdi8oEcki/kiAyIID/IT/KL/G78bXaab5vvLkcbWyvNC7IWzff/AD+MJJU=</latexit><latexit sha1_base64="E6V3HS16/idoqA6LIsbuWfFLtL8=">AAADfHicfZJNaxsxEIYVux+p+xGnPfYiui6klAavDW2gl0B76KGHFOo4YBmjlWdtYUm7SLNJzEbX/ppe2//SP1Oq3TgQO2kGBMPMvHpeiUlyJR12u3+2Gs179x883H7Uevzk6bOd9u7zY5cVVsBAZCqzJwl3oKSBAUpUcJJb4DpRMEwWn6r+8BSsk5n5jsscxprPjEyl4BhKkzbtMIRztLocfvV7TEwz/EiZ5jhP0vLMv+lM2lF3v1sHvZnEqyQiqzia7DY+s2kmCg0GheLOjeJujuOSW5RCgW+xwkHOxYLPYBRSwzW4cVk/xdPXoTKlaWbDMUjr6nVFybVzS52Eycqk2+xVxdt6owLTg3EpTV4gGHEJSgtFMaPVv9CptCBQLUPChZXBKxVzbrnA8HtrFJTBsG+1WszAmci05mZassRyP4rHJQPjCguVgZIpSJEpbmYKophZOZvjhffrwgXg7cKLKwmz9Q2bwkAUtbh3F5Wy03zODWa6jHp+5YHWM/s06l1vx/5OXrjcynNQftT/P5BuEKO+p1fQilY/jEb9NW4wRjfRYe/izS27mRz39uOQf+tFhwerDdwmL8krskdi8oEcki/kiAyIID/IT/KL/G78bXaab5vvLkcbWyvNC7IWzff/AD+MJJU=</latexit><latexit sha1_base64="E6V3HS16/idoqA6LIsbuWfFLtL8=">AAADfHicfZJNaxsxEIYVux+p+xGnPfYiui6klAavDW2gl0B76KGHFOo4YBmjlWdtYUm7SLNJzEbX/ppe2//SP1Oq3TgQO2kGBMPMvHpeiUlyJR12u3+2Gs179x883H7Uevzk6bOd9u7zY5cVVsBAZCqzJwl3oKSBAUpUcJJb4DpRMEwWn6r+8BSsk5n5jsscxprPjEyl4BhKkzbtMIRztLocfvV7TEwz/EiZ5jhP0vLMv+lM2lF3v1sHvZnEqyQiqzia7DY+s2kmCg0GheLOjeJujuOSW5RCgW+xwkHOxYLPYBRSwzW4cVk/xdPXoTKlaWbDMUjr6nVFybVzS52Eycqk2+xVxdt6owLTg3EpTV4gGHEJSgtFMaPVv9CptCBQLUPChZXBKxVzbrnA8HtrFJTBsG+1WszAmci05mZassRyP4rHJQPjCguVgZIpSJEpbmYKophZOZvjhffrwgXg7cKLKwmz9Q2bwkAUtbh3F5Wy03zODWa6jHp+5YHWM/s06l1vx/5OXrjcynNQftT/P5BuEKO+p1fQilY/jEb9NW4wRjfRYe/izS27mRz39uOQf+tFhwerDdwmL8krskdi8oEcki/kiAyIID/IT/KL/G78bXaab5vvLkcbWyvNC7IWzff/AD+MJJU=</latexit>

Backward Propagation(Compute Gradient @obj

@w )<latexit sha1_base64="eHt4RlacyTBkwSNbNglROd3M4wE=">AAAD1XicfZLBb9MwFMbdhcEIsHVw5GKRIY1L1bQHdpwYEhyLRNdJdVU5zktq6tiR46yrMt8QV/4+TvwrOFkrtd2YpUhP773Pvy+fHOWCF6bb/dPa857sP3128Nx/8fLV4VH7+PVloUrNYMiUUPoqogUILmFouBFwlWugWSRgFM0v6vnoGnTBlfxuljlMMppKnnBGjWtN20sSQcplxUAa0Nb/RNl8QXWMB1rlNG22CPFPL1SWlwbwF01j7nbxCUk0ZRXJqTacCkwM3BidVSr6Ye1GO6NmFiXVwtqTDz4BGa9R03bQ7XSbg+8X4aoI0OoMpsd7n0msWJk5PRO0KMZhNzeTqiYxAdYnZQG5809TGLtS0gyKSdVkZPF714lxorT7nP2mu6moaFYUyyxym7XlYndWNx+ajUuTnE0qLut4JLsDJaXARuE6cBxzDcyIpSso09x5xWxGXXYuhG2K4c6w9X2fSFgwlWXUpUUiTe04nFQuu6LUUBuoiIDEEEFlKiAIiebpzNxauy2cg3lYeLuWEN3csCt0RNaIe49RMbnOZ1QalVVBz6484Gang4Pe5ji0j/Lc5ZrfgLDj/v+BeIcY9C1eQ2ta82M46G9xnTG8i3bvLtx9ZfeLy14ndPW3XnB+tnqBB+gteodOUYg+onP0FQ3QEDH0t7XfOmwdeSPPej+9X3ere62V5g3aOt7vfz3aRs4=</latexit><latexit sha1_base64="eHt4RlacyTBkwSNbNglROd3M4wE=">AAAD1XicfZLBb9MwFMbdhcEIsHVw5GKRIY1L1bQHdpwYEhyLRNdJdVU5zktq6tiR46yrMt8QV/4+TvwrOFkrtd2YpUhP773Pvy+fHOWCF6bb/dPa857sP3128Nx/8fLV4VH7+PVloUrNYMiUUPoqogUILmFouBFwlWugWSRgFM0v6vnoGnTBlfxuljlMMppKnnBGjWtN20sSQcplxUAa0Nb/RNl8QXWMB1rlNG22CPFPL1SWlwbwF01j7nbxCUk0ZRXJqTacCkwM3BidVSr6Ye1GO6NmFiXVwtqTDz4BGa9R03bQ7XSbg+8X4aoI0OoMpsd7n0msWJk5PRO0KMZhNzeTqiYxAdYnZQG5809TGLtS0gyKSdVkZPF714lxorT7nP2mu6moaFYUyyxym7XlYndWNx+ajUuTnE0qLut4JLsDJaXARuE6cBxzDcyIpSso09x5xWxGXXYuhG2K4c6w9X2fSFgwlWXUpUUiTe04nFQuu6LUUBuoiIDEEEFlKiAIiebpzNxauy2cg3lYeLuWEN3csCt0RNaIe49RMbnOZ1QalVVBz6484Gang4Pe5ji0j/Lc5ZrfgLDj/v+BeIcY9C1eQ2ta82M46G9xnTG8i3bvLtx9ZfeLy14ndPW3XnB+tnqBB+gteodOUYg+onP0FQ3QEDH0t7XfOmwdeSPPej+9X3ere62V5g3aOt7vfz3aRs4=</latexit><latexit sha1_base64="eHt4RlacyTBkwSNbNglROd3M4wE=">AAAD1XicfZLBb9MwFMbdhcEIsHVw5GKRIY1L1bQHdpwYEhyLRNdJdVU5zktq6tiR46yrMt8QV/4+TvwrOFkrtd2YpUhP773Pvy+fHOWCF6bb/dPa857sP3128Nx/8fLV4VH7+PVloUrNYMiUUPoqogUILmFouBFwlWugWSRgFM0v6vnoGnTBlfxuljlMMppKnnBGjWtN20sSQcplxUAa0Nb/RNl8QXWMB1rlNG22CPFPL1SWlwbwF01j7nbxCUk0ZRXJqTacCkwM3BidVSr6Ye1GO6NmFiXVwtqTDz4BGa9R03bQ7XSbg+8X4aoI0OoMpsd7n0msWJk5PRO0KMZhNzeTqiYxAdYnZQG5809TGLtS0gyKSdVkZPF714lxorT7nP2mu6moaFYUyyxym7XlYndWNx+ajUuTnE0qLut4JLsDJaXARuE6cBxzDcyIpSso09x5xWxGXXYuhG2K4c6w9X2fSFgwlWXUpUUiTe04nFQuu6LUUBuoiIDEEEFlKiAIiebpzNxauy2cg3lYeLuWEN3csCt0RNaIe49RMbnOZ1QalVVBz6484Gang4Pe5ji0j/Lc5ZrfgLDj/v+BeIcY9C1eQ2ta82M46G9xnTG8i3bvLtx9ZfeLy14ndPW3XnB+tnqBB+gteodOUYg+onP0FQ3QEDH0t7XfOmwdeSPPej+9X3ere62V5g3aOt7vfz3aRs4=</latexit><latexit sha1_base64="eHt4RlacyTBkwSNbNglROd3M4wE=">AAAD1XicfZLBb9MwFMbdhcEIsHVw5GKRIY1L1bQHdpwYEhyLRNdJdVU5zktq6tiR46yrMt8QV/4+TvwrOFkrtd2YpUhP773Pvy+fHOWCF6bb/dPa857sP3128Nx/8fLV4VH7+PVloUrNYMiUUPoqogUILmFouBFwlWugWSRgFM0v6vnoGnTBlfxuljlMMppKnnBGjWtN20sSQcplxUAa0Nb/RNl8QXWMB1rlNG22CPFPL1SWlwbwF01j7nbxCUk0ZRXJqTacCkwM3BidVSr6Ye1GO6NmFiXVwtqTDz4BGa9R03bQ7XSbg+8X4aoI0OoMpsd7n0msWJk5PRO0KMZhNzeTqiYxAdYnZQG5809TGLtS0gyKSdVkZPF714lxorT7nP2mu6moaFYUyyxym7XlYndWNx+ajUuTnE0qLut4JLsDJaXARuE6cBxzDcyIpSso09x5xWxGXXYuhG2K4c6w9X2fSFgwlWXUpUUiTe04nFQuu6LUUBuoiIDEEEFlKiAIiebpzNxauy2cg3lYeLuWEN3csCt0RNaIe49RMbnOZ1QalVVBz6484Gang4Pe5ji0j/Lc5ZrfgLDj/v+BeIcY9C1eQ2ta82M46G9xnTG8i3bvLtx9ZfeLy14ndPW3XnB+tnqBB+gteodOUYg+onP0FQ3QEDH0t7XfOmwdeSPPej+9X3ere62V5g3aOt7vfz3aRs4=</latexit>

Forward Propagation(Compute obj)

<latexit sha1_base64="7OB+Bw5IysanGRJyNebQHTtZ2Hk=">AAADpnicfVJNaxsxEFWy/Ui3X0577EV0U3AvxmsfmmMgpZQeigt1HLCMq5XHtmJ9LJI2idnoP/XXFHpqf0q1GxtiJ82A4DEzT+/NMFkuuHXt9u+d3ejBw0eP957ET589f/Gysf/qxOrCMOgzLbQ5zagFwRX0HXcCTnMDVGYCBtniuKoPzsFYrtV3t8xhJOlM8Sln1IXUuPGFZDDjqmSgHBgff9LmgpoJ7hmd01ndREjcPNYyLxzgA+Lg0hlZ6uzMH7yPCajJmjtuJO1Wuw58G6QrkKBV9Mb7ux/JRLNCBj4T1Nph2s7dqKTGcSbAx6SwkFO2oDMYBqioBDsq66E9fhcyEzzVJjzlcJ29ySiptHYps9ApqZvb7VqVvKs2LNz0cFRyVQ2s2LXQtBDYaVxtEE+4AebEMgDKDA9eMZtTQ1lYwqaK48Gwj+OYKLhgWkoatkUyQ/0wHZVhd7YwUBkoiYCpI4KqmYAkJYbP5u7K+03iAtzdxKs1hZj6h21iUGQ1uXOfKibn+Zwqp2WZdPzKA657Wjjp3Cyn/l698LnhlyD8sPt/QbylmHQ9XotWavVgOOlu6AZjeFs63F26fWW3wUmnlQb8rZMcHa4ucA+9QW9RE6XoAzpCn1EP9RFDP9Ev9Af9jZrR16gfDa5bd3dWnNdoI6If/wC2DTVm</latexit><latexit sha1_base64="7OB+Bw5IysanGRJyNebQHTtZ2Hk=">AAADpnicfVJNaxsxEFWy/Ui3X0577EV0U3AvxmsfmmMgpZQeigt1HLCMq5XHtmJ9LJI2idnoP/XXFHpqf0q1GxtiJ82A4DEzT+/NMFkuuHXt9u+d3ejBw0eP957ET589f/Gysf/qxOrCMOgzLbQ5zagFwRX0HXcCTnMDVGYCBtniuKoPzsFYrtV3t8xhJOlM8Sln1IXUuPGFZDDjqmSgHBgff9LmgpoJ7hmd01ndREjcPNYyLxzgA+Lg0hlZ6uzMH7yPCajJmjtuJO1Wuw58G6QrkKBV9Mb7ux/JRLNCBj4T1Nph2s7dqKTGcSbAx6SwkFO2oDMYBqioBDsq66E9fhcyEzzVJjzlcJ29ySiptHYps9ApqZvb7VqVvKs2LNz0cFRyVQ2s2LXQtBDYaVxtEE+4AebEMgDKDA9eMZtTQ1lYwqaK48Gwj+OYKLhgWkoatkUyQ/0wHZVhd7YwUBkoiYCpI4KqmYAkJYbP5u7K+03iAtzdxKs1hZj6h21iUGQ1uXOfKibn+Zwqp2WZdPzKA657Wjjp3Cyn/l698LnhlyD8sPt/QbylmHQ9XotWavVgOOlu6AZjeFs63F26fWW3wUmnlQb8rZMcHa4ucA+9QW9RE6XoAzpCn1EP9RFDP9Ev9Af9jZrR16gfDa5bd3dWnNdoI6If/wC2DTVm</latexit><latexit sha1_base64="7OB+Bw5IysanGRJyNebQHTtZ2Hk=">AAADpnicfVJNaxsxEFWy/Ui3X0577EV0U3AvxmsfmmMgpZQeigt1HLCMq5XHtmJ9LJI2idnoP/XXFHpqf0q1GxtiJ82A4DEzT+/NMFkuuHXt9u+d3ejBw0eP957ET589f/Gysf/qxOrCMOgzLbQ5zagFwRX0HXcCTnMDVGYCBtniuKoPzsFYrtV3t8xhJOlM8Sln1IXUuPGFZDDjqmSgHBgff9LmgpoJ7hmd01ndREjcPNYyLxzgA+Lg0hlZ6uzMH7yPCajJmjtuJO1Wuw58G6QrkKBV9Mb7ux/JRLNCBj4T1Nph2s7dqKTGcSbAx6SwkFO2oDMYBqioBDsq66E9fhcyEzzVJjzlcJ29ySiptHYps9ApqZvb7VqVvKs2LNz0cFRyVQ2s2LXQtBDYaVxtEE+4AebEMgDKDA9eMZtTQ1lYwqaK48Gwj+OYKLhgWkoatkUyQ/0wHZVhd7YwUBkoiYCpI4KqmYAkJYbP5u7K+03iAtzdxKs1hZj6h21iUGQ1uXOfKibn+Zwqp2WZdPzKA657Wjjp3Cyn/l698LnhlyD8sPt/QbylmHQ9XotWavVgOOlu6AZjeFs63F26fWW3wUmnlQb8rZMcHa4ucA+9QW9RE6XoAzpCn1EP9RFDP9Ev9Af9jZrR16gfDa5bd3dWnNdoI6If/wC2DTVm</latexit><latexit sha1_base64="7OB+Bw5IysanGRJyNebQHTtZ2Hk=">AAADpnicfVJNaxsxEFWy/Ui3X0577EV0U3AvxmsfmmMgpZQeigt1HLCMq5XHtmJ9LJI2idnoP/XXFHpqf0q1GxtiJ82A4DEzT+/NMFkuuHXt9u+d3ejBw0eP957ET589f/Gysf/qxOrCMOgzLbQ5zagFwRX0HXcCTnMDVGYCBtniuKoPzsFYrtV3t8xhJOlM8Sln1IXUuPGFZDDjqmSgHBgff9LmgpoJ7hmd01ndREjcPNYyLxzgA+Lg0hlZ6uzMH7yPCajJmjtuJO1Wuw58G6QrkKBV9Mb7ux/JRLNCBj4T1Nph2s7dqKTGcSbAx6SwkFO2oDMYBqioBDsq66E9fhcyEzzVJjzlcJ29ySiptHYps9ApqZvb7VqVvKs2LNz0cFRyVQ2s2LXQtBDYaVxtEE+4AebEMgDKDA9eMZtTQ1lYwqaK48Gwj+OYKLhgWkoatkUyQ/0wHZVhd7YwUBkoiYCpI4KqmYAkJYbP5u7K+03iAtzdxKs1hZj6h21iUGQ1uXOfKibn+Zwqp2WZdPzKA657Wjjp3Cyn/l698LnhlyD8sPt/QbylmHQ9XotWavVgOOlu6AZjeFs63F26fWW3wUmnlQb8rZMcHa4ucA+9QW9RE6XoAzpCn1EP9RFDP9Ev9Af9jZrR16gfDa5bd3dWnNdoI6If/wC2DTVm</latexit>

(b)

Figure 1: Analogy between neural network training and ana-lytical placement. (a) Train a network for weights w. (b) Solvea placement for cell locations w = (x, y).

placement sites; detailed placement performs incremental refinementto further improve the quality. Usually, global placement is the mosttime-consuming portion in analytical placement.

Global placement aims at minimizing thewirelength cost subjectingto density constraints. A typical solving approach is to relax thedensity constraints to the objective as a density penalty [2, 4, 16],

min (∑e ∈E

WL(e; x, y)) + λD(x, y), (1)

where WL(·; ·) is the wirelength cost function that takes any netinstance e and returns thewirelength, andD(·) is the density penalty tospread cells out in the layout. The density constraints can be satisfiedby gradually increasing the weight λ.

2.2 Analogy to Deep LearningAs both solving an analytical placement and training a neural networkare essentially solving a nonlinear optimization problem, we investi-gate the underlying similarity between the two problems: the analogyof the wirelength cost to the error of misprediction and that of thedensity cost to the regularization term. Figure 1 shows the objectivefunctions of the two problems. In neural network training, each datainstance with a feature vector xi and a label yi is fed to the networkand the neural network predicts a label ϕ(xi ;w). The task for train-ing is to minimize the overall objective over weights w, where theobjective consists of the prediction errors for all data instances, and aregularization term R(w) [17]. In the analogy of placement to neuralnetwork training, we combine cell locations (x, y) into w for brevity.Each data instance is replaced with a net instance with a feature vec-tor ei and a label zero. The neural network then takes a net instanceand computes the wirelength cost WL(ei ;w). Using the absolute errorfunction f (y,y) = |y − y | and noting that wirelength is non-negative,the minimization of prediction errors becomes

∑ni WL(ei ;w). The

density cost D(w) corresponds to the regularization term R(w), asit is not related to net instances. With this construction, we find aone-to-one mapping of each component in analytical placement toneural network training, which makes it possible to take advantageof recent developments in deep learning toolkits for implementation.Then, we can solve the placement problem following the neural net-work training procedure, with forward propagation to compute theobjective and backward propagation to calculate the gradient.

Deep learning toolkits nowadays consist of three stacks, low-leveloperators (OPs), automatic gradient derivation, and optimizationengines, as shown in Figure 2(a). Toolkits like TensorFlow andPyTorch offer mature and efficient implementation of these threestacks with compatibility to both CPU and GPU acceleration. The

PlacementAPI

NonlinearOptimizer

AutomaticGradient

CONV

ReLU

WL

DensityOPs

C++/CUDA

Python

(a)

Random Initial Placement

WL Grad Density Grad

Optimization & Update

Converge?

Legalization

NTUplace DP

N

Y

GP

LG

DP

(b)

Figure 2: (a) Software architecture for placement implementa-tion using deep learning toolkits. (b) DREAMPlace flow.

toolkits also provide easy APIs to extend the existing set of low-leveloperators. Each custom operator requires well defined forward andbackward functions for cost and gradient computation. To develop ananalytical placement with deep learning toolkits, we only need to im-plement the custom operators for wirelength and density cost in C++and CUDA. Then we can construct a placement framework in Pythonwith very low development overhead and easily incorporate a varietyof optimization engines in the toolkit. The placement framework canrun on both CPU and GPU platforms. Conventional development ofplacement engines takes huge efforts in building the entire softwarestacks with C++. Thus, the bar of designing and validating a newplacement algorithm is very high due to the development overhead.Taking advantage of deep learning toolkits, researchers can concen-trate on the development of critical parts like low-level operators andhigh-level optimization engines.

2.3 The ePlace/RePlAce AlgorithmePlace/RePlAce is a state-of-the-art family of global placement algo-rithms that model the layout and netlist as an electrostatic system[4–6]. It uses weighted-average wirelength (WA) for wirelength cost,

WAe =∑i∈e xie

xiγ∑

i∈e exiγ−

∑i∈e xie

−xiγ∑

i∈e e−xiγ

, (2)

where γ is a parameter to control the smoothness and accuracy of theapproximation to half-perimeter wirelength (HPWL). The smaller γis, the more accurate it is to approximate HPWL, but the less smooth.

Its density penalty is quite different from other analytical placers[2, 16, 18]. With analogy to an electrostatic system, cells are modeledas charges, density penalty is modeled as potential energy, and densitygradient is modeled as the electric field. The numerical solution ofthe electric potential and field distribution after solving a Poisson’sequation can be obtained with spectral methods. Given anM ×M gridof bins and wu =

2πuM and wv = 2πv

M with u = 0, 1, . . . ,M − 1,v =0, 1, . . . ,M − 1, the solution can be computed as follows [4],

au,v =1M2

M−1∑x=0

M−1∑y=0

ρ(x, y) cos (wux ) cos (wvy), (3a)

ψDCT(x, y) =M−1∑u=0

M−1∑v=0

au,vw2u +w2

vcos (wux ) cos (wvy), (3b)

ξXDSCT(x, y) =M−1∑u=0

M−1∑v=0

au,vwu

w2u +w2

vsin (wux ) cos (wvy), (3c)

ξYDCST(x, y) =M−1∑u=0

M−1∑v=0

au,vwv

w2u +w2

vcos (wux ) sin (wvy), (3d)

Page 3: DREAMPlace: Deep Learning Toolkit-Enabled GPU …DREAMPlace: Deep Learning Toolkit-Enabled GPU Acceleration for Modern VLSI Placement Yibo Lin ECE Department, UT Austin yibolin@utexas.edu

Table 1: NotationsNotation Description Notation Description

V Set of cells E Set of netsP Set of pins B Set of binsx+e maxi ∈e xi ,∀e ∈ E x−e mini ∈e xi ,∀e ∈ Ea+i e

xi −x+e

γ ,∀i ∈ e, e ∈ E a−i e−xi −x

−e

γ ,∀i ∈ e, e ∈ Eb+e

∑i ∈e a

+i ,∀e ∈ E b−e

∑i ∈e a

−i ,∀e ∈ E

c+e∑i ∈e xia

+i ,∀e ∈ E c−e

∑i ∈e xia

−i ,∀e ∈ E

x+ {x+e },∀e ∈ E x− {x−e },∀e ∈ Ea+ {a+i },∀i ∈ P a− {a−i },∀i ∈ Pb+ {b+e },∀e ∈ E b− {b−e },∀e ∈ Ec+ {c+e },∀e ∈ E c− {c−e },∀e ∈ E

LG 1.1

DP5.9

GP-IP

21.1

GP-Nonlinear

71.9

(a)

LG 2.2

DP

11.7

GP-IP

21.1

GP-Nonlinear

65.0

(b)

Figure 3: RePlAce [6] runtime breakdown in percentages onbigblue4 (2 million cells). (a) 1 thread; (b) 10 threads.whereψDCT denotes the potential function, and ξXDSCT and ξYDCSTdenote the electric field in horizontal and vertical directions respec-tively. Equation (3) requires Discrete Cosine Transform (DCT) andInverse Discrete Cosine Transform (IDCT) routines to solve efficiently.The detailed computation is explained in Section 3. With the elec-tric field defined for each bin, the density gradient of each cell is theoverall force taken by the cell in the system.

After defining wirelength cost and density penalty, RePlAce adoptsgradient-descent optimizers, such as Nesterov’s method and conjugategradient method, to solve the optimization problem. RePlAce was im-plemented with multi-threading support [6]. The runtime breakdownfor RePlAce [6] is elaborated in Figure 3. GP including initial place-ment (GP-IP) and nonlinear optimization (GP-Nonlinear) takes about90% of the runtime with both single thread and 10 threads. Therefore,accelerating GP is the most effective in reducing the overall runtime.

3 THE DREAMPLACE ALGORITHMSOur overall placement flow is given in Figure 2(b). It is slightly differentfrom the typical one that starts from a bound-to-bound initial place-ment [2, 4]. We observe that starting from a random initial placementalso achieves the same quality (< 0.04% difference) with significantlyless runtime (21.1% in Figure 3). In initial placement, standard cellsare placed in the center of the layout with a small Gaussian noise.The scales of the noise are set to 0.1% of the width and height of theplacement region in the experiment. The kernel global placement it-erations refer to the loop that involves the computation of wirelengthand density gradient, optimization engines, and cell location updating.After the global placement converges, legalization is performed toremove remaining overlaps and align cells to placement sites. The laststep before the output is detailed placement to refine the placementsolutions relying on NTUplace3 [2]. The rest of this section will focuson GPU acceleration to the ePlace/RePlAce algorithm [4, 6].

3.1 Wirelength Forward and BackwardAs RePlAce adopts WA wirelength, we use WA as an example forthe GPU acceleration to wirelength forward and backward. Similarinsights also apply to other wirelength costs like log-sum-exp (LSE)[2], which is also implemented in the framework. For brevity, we only

Forward Backward

x+e

<latexit sha1_base64="qNQy7Y03d3VulGgtSOdQs5/VGLg=">AAADY3icfZLNatwwFIWVcX9Sp21+2l0piDqFQmGwPYtmGUgXXabQSVJsd5A1d2bESLKR5CSDo6fotn2wPkDfo7LjQMZJIzAc7r1H35G5ecmZNmH4Z2PgPXr85OnmM3/r+YuX2zu7eye6qBSFMS14oc5yooEzCWPDDIezUgEROYfTfHnU9E/PQWlWyG9mVUImyFyyGaPEuNL3/csJ/Kg/2v3JThAOw/bguyLqRIC6czzZHXxOpwWtBEhDOdE6icLSZDVRhlEO1k8rDSWhSzKHxElJBOisbhNb/N5VpnhWKPdJg9vqbUdNhNYrkbtJQcxC93tN8b5eUpnZQVYzWVYGJL0GzSqOTYGb5+MpU0ANXzlBqGIuK6YLogg17ietUQxzga3v+6mEC1oIQeS0TnNFbBJldQpSVwqaAHXKYWZSTuScQxClis0X5sradeMSzP3GqxtLqtob+kZHpK05foiK0/NyQaQpRB3EtsuA25khDuLb7cg+yHOXK3YJ3Caj/wNxjxiMLL6BNrT2YTgYrXFdMNxHu72L+lt2V5zEw8jpr3FweNBt4CZ6g96hDyhCn9Ah+oKO0RhRJNBP9Av9Hvz1trw97/X16GCj87xCa8d7+w9cRhqx</latexit><latexit sha1_base64="qNQy7Y03d3VulGgtSOdQs5/VGLg=">AAADY3icfZLNatwwFIWVcX9Sp21+2l0piDqFQmGwPYtmGUgXXabQSVJsd5A1d2bESLKR5CSDo6fotn2wPkDfo7LjQMZJIzAc7r1H35G5ecmZNmH4Z2PgPXr85OnmM3/r+YuX2zu7eye6qBSFMS14oc5yooEzCWPDDIezUgEROYfTfHnU9E/PQWlWyG9mVUImyFyyGaPEuNL3/csJ/Kg/2v3JThAOw/bguyLqRIC6czzZHXxOpwWtBEhDOdE6icLSZDVRhlEO1k8rDSWhSzKHxElJBOisbhNb/N5VpnhWKPdJg9vqbUdNhNYrkbtJQcxC93tN8b5eUpnZQVYzWVYGJL0GzSqOTYGb5+MpU0ANXzlBqGIuK6YLogg17ietUQxzga3v+6mEC1oIQeS0TnNFbBJldQpSVwqaAHXKYWZSTuScQxClis0X5sradeMSzP3GqxtLqtob+kZHpK05foiK0/NyQaQpRB3EtsuA25khDuLb7cg+yHOXK3YJ3Caj/wNxjxiMLL6BNrT2YTgYrXFdMNxHu72L+lt2V5zEw8jpr3FweNBt4CZ6g96hDyhCn9Ah+oKO0RhRJNBP9Av9Hvz1trw97/X16GCj87xCa8d7+w9cRhqx</latexit><latexit sha1_base64="qNQy7Y03d3VulGgtSOdQs5/VGLg=">AAADY3icfZLNatwwFIWVcX9Sp21+2l0piDqFQmGwPYtmGUgXXabQSVJsd5A1d2bESLKR5CSDo6fotn2wPkDfo7LjQMZJIzAc7r1H35G5ecmZNmH4Z2PgPXr85OnmM3/r+YuX2zu7eye6qBSFMS14oc5yooEzCWPDDIezUgEROYfTfHnU9E/PQWlWyG9mVUImyFyyGaPEuNL3/csJ/Kg/2v3JThAOw/bguyLqRIC6czzZHXxOpwWtBEhDOdE6icLSZDVRhlEO1k8rDSWhSzKHxElJBOisbhNb/N5VpnhWKPdJg9vqbUdNhNYrkbtJQcxC93tN8b5eUpnZQVYzWVYGJL0GzSqOTYGb5+MpU0ANXzlBqGIuK6YLogg17ietUQxzga3v+6mEC1oIQeS0TnNFbBJldQpSVwqaAHXKYWZSTuScQxClis0X5sradeMSzP3GqxtLqtob+kZHpK05foiK0/NyQaQpRB3EtsuA25khDuLb7cg+yHOXK3YJ3Caj/wNxjxiMLL6BNrT2YTgYrXFdMNxHu72L+lt2V5zEw8jpr3FweNBt4CZ6g96hDyhCn9Ah+oKO0RhRJNBP9Av9Hvz1trw97/X16GCj87xCa8d7+w9cRhqx</latexit><latexit sha1_base64="qNQy7Y03d3VulGgtSOdQs5/VGLg=">AAADY3icfZLNatwwFIWVcX9Sp21+2l0piDqFQmGwPYtmGUgXXabQSVJsd5A1d2bESLKR5CSDo6fotn2wPkDfo7LjQMZJIzAc7r1H35G5ecmZNmH4Z2PgPXr85OnmM3/r+YuX2zu7eye6qBSFMS14oc5yooEzCWPDDIezUgEROYfTfHnU9E/PQWlWyG9mVUImyFyyGaPEuNL3/csJ/Kg/2v3JThAOw/bguyLqRIC6czzZHXxOpwWtBEhDOdE6icLSZDVRhlEO1k8rDSWhSzKHxElJBOisbhNb/N5VpnhWKPdJg9vqbUdNhNYrkbtJQcxC93tN8b5eUpnZQVYzWVYGJL0GzSqOTYGb5+MpU0ANXzlBqGIuK6YLogg17ietUQxzga3v+6mEC1oIQeS0TnNFbBJldQpSVwqaAHXKYWZSTuScQxClis0X5sradeMSzP3GqxtLqtob+kZHpK05foiK0/NyQaQpRB3EtsuA25khDuLb7cg+yHOXK3YJ3Caj/wNxjxiMLL6BNrT2YTgYrXFdMNxHu72L+lt2V5zEw8jpr3FweNBt4CZ6g96hDyhCn9Ah+oKO0RhRJNBP9Av9Hvz1trw97/X16GCj87xCa8d7+w9cRhqx</latexit>

x�e

<latexit sha1_base64="Ig5++k+wdBCxFPDhL+gg711KKL4=">AAADY3icfZLNbtQwFIXdCT8lBfoDO4RkkSKxYZRkFnRZqSxYFolpi5Iwcjx3Zqyxnch22o5SPwVbeDAegPfASVOpk5ZainR07z3+jqObl5xpE4Z/Ngbeo8dPnm4+87eev3i5vbO7d6KLSlEY04IX6iwnGjiTMDbMcDgrFRCRczjNl0dN//QclGaF/GZWJWSCzCWbMUqMK33fv5zAj/qj3Z/sBOEwbA++K6JOBKg7x5Pdwed0WtBKgDSUE62TKCxNVhNlGOVg/bTSUBK6JHNInJREgM7qNrHF711limeFcp80uK3edtREaL0SuZsUxCx0v9cU7+sllZkdZDWTZWVA0mvQrOLYFLh5Pp4yBdTwlROEKuayYrogilDjftIaxTAX2Pq+n0q4oIUQRE7rNFfEJlFWpyB1paAJUKccZiblRM45BFGq2HxhrqxdNy7B3G+8urGkqr2hb3RE2prjh6g4PS8XRJpC1EFsuwy4nRniIL7djuyDPHe5YpfAbTL6PxD3iMHI4htoQ2sfhoPRGtcFw32027uov2V3xUk8jJz+GgeHB90GbqI36B36gCL0CR2iL+gYjRFFAv1Ev9DvwV9vy9vzXl+PDjY6zyu0dry3/wBiOBqz</latexit><latexit sha1_base64="Ig5++k+wdBCxFPDhL+gg711KKL4=">AAADY3icfZLNbtQwFIXdCT8lBfoDO4RkkSKxYZRkFnRZqSxYFolpi5Iwcjx3Zqyxnch22o5SPwVbeDAegPfASVOpk5ZainR07z3+jqObl5xpE4Z/Ngbeo8dPnm4+87eev3i5vbO7d6KLSlEY04IX6iwnGjiTMDbMcDgrFRCRczjNl0dN//QclGaF/GZWJWSCzCWbMUqMK33fv5zAj/qj3Z/sBOEwbA++K6JOBKg7x5Pdwed0WtBKgDSUE62TKCxNVhNlGOVg/bTSUBK6JHNInJREgM7qNrHF711limeFcp80uK3edtREaL0SuZsUxCx0v9cU7+sllZkdZDWTZWVA0mvQrOLYFLh5Pp4yBdTwlROEKuayYrogilDjftIaxTAX2Pq+n0q4oIUQRE7rNFfEJlFWpyB1paAJUKccZiblRM45BFGq2HxhrqxdNy7B3G+8urGkqr2hb3RE2prjh6g4PS8XRJpC1EFsuwy4nRniIL7djuyDPHe5YpfAbTL6PxD3iMHI4htoQ2sfhoPRGtcFw32027uov2V3xUk8jJz+GgeHB90GbqI36B36gCL0CR2iL+gYjRFFAv1Ev9DvwV9vy9vzXl+PDjY6zyu0dry3/wBiOBqz</latexit><latexit sha1_base64="Ig5++k+wdBCxFPDhL+gg711KKL4=">AAADY3icfZLNbtQwFIXdCT8lBfoDO4RkkSKxYZRkFnRZqSxYFolpi5Iwcjx3Zqyxnch22o5SPwVbeDAegPfASVOpk5ZainR07z3+jqObl5xpE4Z/Ngbeo8dPnm4+87eev3i5vbO7d6KLSlEY04IX6iwnGjiTMDbMcDgrFRCRczjNl0dN//QclGaF/GZWJWSCzCWbMUqMK33fv5zAj/qj3Z/sBOEwbA++K6JOBKg7x5Pdwed0WtBKgDSUE62TKCxNVhNlGOVg/bTSUBK6JHNInJREgM7qNrHF711limeFcp80uK3edtREaL0SuZsUxCx0v9cU7+sllZkdZDWTZWVA0mvQrOLYFLh5Pp4yBdTwlROEKuayYrogilDjftIaxTAX2Pq+n0q4oIUQRE7rNFfEJlFWpyB1paAJUKccZiblRM45BFGq2HxhrqxdNy7B3G+8urGkqr2hb3RE2prjh6g4PS8XRJpC1EFsuwy4nRniIL7djuyDPHe5YpfAbTL6PxD3iMHI4htoQ2sfhoPRGtcFw32027uov2V3xUk8jJz+GgeHB90GbqI36B36gCL0CR2iL+gYjRFFAv1Ev9DvwV9vy9vzXl+PDjY6zyu0dry3/wBiOBqz</latexit><latexit sha1_base64="Ig5++k+wdBCxFPDhL+gg711KKL4=">AAADY3icfZLNbtQwFIXdCT8lBfoDO4RkkSKxYZRkFnRZqSxYFolpi5Iwcjx3Zqyxnch22o5SPwVbeDAegPfASVOpk5ZainR07z3+jqObl5xpE4Z/Ngbeo8dPnm4+87eev3i5vbO7d6KLSlEY04IX6iwnGjiTMDbMcDgrFRCRczjNl0dN//QclGaF/GZWJWSCzCWbMUqMK33fv5zAj/qj3Z/sBOEwbA++K6JOBKg7x5Pdwed0WtBKgDSUE62TKCxNVhNlGOVg/bTSUBK6JHNInJREgM7qNrHF711limeFcp80uK3edtREaL0SuZsUxCx0v9cU7+sllZkdZDWTZWVA0mvQrOLYFLh5Pp4yBdTwlROEKuayYrogilDjftIaxTAX2Pq+n0q4oIUQRE7rNFfEJlFWpyB1paAJUKccZiblRM45BFGq2HxhrqxdNy7B3G+8urGkqr2hb3RE2prjh6g4PS8XRJpC1EFsuwy4nRniIL7djuyDPHe5YpfAbTL6PxD3iMHI4htoQ2sfhoPRGtcFw32027uov2V3xUk8jJz+GgeHB90GbqI36B36gCL0CR2iL+gYjRFFAv1Ev9DvwV9vy9vzXl+PDjY6zyu0dry3/wBiOBqz</latexit>

c+e

<latexit sha1_base64="QmrF2ShEuF3UmMLt15w51iONeEE=">AAADY3icfZLNbtQwFIXdCT8l5act7BCSRYqEhDRKMgu6rAQLlkVi2qIkjBzPzYw1thPZTmGU+inYwoPxALwHTppKnbTUUqSje+/xdxzdvOJMmzD8szXy7t1/8HD7kb/z+MnTZ7t7+ye6rBWFKS15qc5yooEzCVPDDIezSgEROYfTfPWh7Z+eg9KslF/MuoJMkIVkBaPEuNLXAzqDb807ezDbDcJx2B18U0S9CFB/jmd7o4/pvKS1AGkoJ1onUViZrCHKMMrB+mmtoSJ0RRaQOCmJAJ01XWKL37jKHBelcp80uKtedzREaL0WuZsUxCz1sNcWb+sltSkOs4bJqjYg6SWoqDk2JW6fj+dMATV87QShirmsmC6JItS4n7RBMcwFtr7vpxK+01IIIudNmitikyhrUpC6VtAGaFIOhUk5kQsOQZQqtliaC2s3jSswtxsvriyp6m4YGh2Rdub4LipOz6slkaYUTRDbPgPuZsY4iK+3I3snz12u2A/gNpn8H4gHxGBi8RW0pXUPw8Fkg+uC4SHa7V003LKb4iQeR05/joOjw34Dt9FL9Bq9RRF6j47QJ3SMpogigX6iX+j36K+34+17Ly5HR1u95znaON6rfx1wGpw=</latexit><latexit sha1_base64="QmrF2ShEuF3UmMLt15w51iONeEE=">AAADY3icfZLNbtQwFIXdCT8l5act7BCSRYqEhDRKMgu6rAQLlkVi2qIkjBzPzYw1thPZTmGU+inYwoPxALwHTppKnbTUUqSje+/xdxzdvOJMmzD8szXy7t1/8HD7kb/z+MnTZ7t7+ye6rBWFKS15qc5yooEzCVPDDIezSgEROYfTfPWh7Z+eg9KslF/MuoJMkIVkBaPEuNLXAzqDb807ezDbDcJx2B18U0S9CFB/jmd7o4/pvKS1AGkoJ1onUViZrCHKMMrB+mmtoSJ0RRaQOCmJAJ01XWKL37jKHBelcp80uKtedzREaL0WuZsUxCz1sNcWb+sltSkOs4bJqjYg6SWoqDk2JW6fj+dMATV87QShirmsmC6JItS4n7RBMcwFtr7vpxK+01IIIudNmitikyhrUpC6VtAGaFIOhUk5kQsOQZQqtliaC2s3jSswtxsvriyp6m4YGh2Rdub4LipOz6slkaYUTRDbPgPuZsY4iK+3I3snz12u2A/gNpn8H4gHxGBi8RW0pXUPw8Fkg+uC4SHa7V003LKb4iQeR05/joOjw34Dt9FL9Bq9RRF6j47QJ3SMpogigX6iX+j36K+34+17Ly5HR1u95znaON6rfx1wGpw=</latexit><latexit sha1_base64="QmrF2ShEuF3UmMLt15w51iONeEE=">AAADY3icfZLNbtQwFIXdCT8l5act7BCSRYqEhDRKMgu6rAQLlkVi2qIkjBzPzYw1thPZTmGU+inYwoPxALwHTppKnbTUUqSje+/xdxzdvOJMmzD8szXy7t1/8HD7kb/z+MnTZ7t7+ye6rBWFKS15qc5yooEzCVPDDIezSgEROYfTfPWh7Z+eg9KslF/MuoJMkIVkBaPEuNLXAzqDb807ezDbDcJx2B18U0S9CFB/jmd7o4/pvKS1AGkoJ1onUViZrCHKMMrB+mmtoSJ0RRaQOCmJAJ01XWKL37jKHBelcp80uKtedzREaL0WuZsUxCz1sNcWb+sltSkOs4bJqjYg6SWoqDk2JW6fj+dMATV87QShirmsmC6JItS4n7RBMcwFtr7vpxK+01IIIudNmitikyhrUpC6VtAGaFIOhUk5kQsOQZQqtliaC2s3jSswtxsvriyp6m4YGh2Rdub4LipOz6slkaYUTRDbPgPuZsY4iK+3I3snz12u2A/gNpn8H4gHxGBi8RW0pXUPw8Fkg+uC4SHa7V003LKb4iQeR05/joOjw34Dt9FL9Bq9RRF6j47QJ3SMpogigX6iX+j36K+34+17Ly5HR1u95znaON6rfx1wGpw=</latexit><latexit sha1_base64="QmrF2ShEuF3UmMLt15w51iONeEE=">AAADY3icfZLNbtQwFIXdCT8l5act7BCSRYqEhDRKMgu6rAQLlkVi2qIkjBzPzYw1thPZTmGU+inYwoPxALwHTppKnbTUUqSje+/xdxzdvOJMmzD8szXy7t1/8HD7kb/z+MnTZ7t7+ye6rBWFKS15qc5yooEzCVPDDIezSgEROYfTfPWh7Z+eg9KslF/MuoJMkIVkBaPEuNLXAzqDb807ezDbDcJx2B18U0S9CFB/jmd7o4/pvKS1AGkoJ1onUViZrCHKMMrB+mmtoSJ0RRaQOCmJAJ01XWKL37jKHBelcp80uKtedzREaL0WuZsUxCz1sNcWb+sltSkOs4bJqjYg6SWoqDk2JW6fj+dMATV87QShirmsmC6JItS4n7RBMcwFtr7vpxK+01IIIudNmitikyhrUpC6VtAGaFIOhUk5kQsOQZQqtliaC2s3jSswtxsvriyp6m4YGh2Rdub4LipOz6slkaYUTRDbPgPuZsY4iK+3I3snz12u2A/gNpn8H4gHxGBi8RW0pXUPw8Fkg+uC4SHa7V003LKb4iQeR05/joOjw34Dt9FL9Bq9RRF6j47QJ3SMpogigX6iX+j36K+34+17Ly5HR1u95znaON6rfx1wGpw=</latexit>

c�e<latexit sha1_base64="Q2lmKV+jF55GTzlh2vfrmDees1s=">AAADY3icfZLNbtQwFIXdCT8l5act7BCSRYrEhlGSWdBlJViwLBLTFiVh5HhuZqyxnch2CqPUT8EWHowH4D1w0lTqpKWWIh3de4+/4+jmFWfahOGfrZF37/6Dh9uP/J3HT54+293bP9FlrShMaclLdZYTDZxJmBpmOJxVCojIOZzmqw9t//QclGal/GLWFWSCLCQrGCXGlb4e0Bl8a97Zg9luEI7D7uCbIupFgPpzPNsbfUznJa0FSEM50TqJwspkDVGGUQ7WT2sNFaErsoDESUkE6KzpElv8xlXmuCiV+6TBXfW6oyFC67XI3aQgZqmHvbZ4Wy+pTXGYNUxWtQFJL0FFzbEpcft8PGcKqOFrJwhVzGXFdEkUocb9pA2KYS6w9X0/lfCdlkIQOW/SXBGbRFmTgtS1gjZAk3IoTMqJXHAIolSxxdJcWLtpXIG53XhxZUlVd8PQ6Ii0M8d3UXF6Xi2JNKVogtj2GXA3M8ZBfL0d2Tt57nLFfgC3yeT/QDwgBhOLr6AtrXsYDiYbXBcMD9Fu76Lhlt0UJ/E4cvpzHBwd9hu4jV6i1+gtitB7dIQ+oWM0RRQJ9BP9Qr9Hf70db997cTk62uo9z9HG8V79AyNiGp4=</latexit><latexit sha1_base64="Q2lmKV+jF55GTzlh2vfrmDees1s=">AAADY3icfZLNbtQwFIXdCT8l5act7BCSRYrEhlGSWdBlJViwLBLTFiVh5HhuZqyxnch2CqPUT8EWHowH4D1w0lTqpKWWIh3de4+/4+jmFWfahOGfrZF37/6Dh9uP/J3HT54+293bP9FlrShMaclLdZYTDZxJmBpmOJxVCojIOZzmqw9t//QclGal/GLWFWSCLCQrGCXGlb4e0Bl8a97Zg9luEI7D7uCbIupFgPpzPNsbfUznJa0FSEM50TqJwspkDVGGUQ7WT2sNFaErsoDESUkE6KzpElv8xlXmuCiV+6TBXfW6oyFC67XI3aQgZqmHvbZ4Wy+pTXGYNUxWtQFJL0FFzbEpcft8PGcKqOFrJwhVzGXFdEkUocb9pA2KYS6w9X0/lfCdlkIQOW/SXBGbRFmTgtS1gjZAk3IoTMqJXHAIolSxxdJcWLtpXIG53XhxZUlVd8PQ6Ii0M8d3UXF6Xi2JNKVogtj2GXA3M8ZBfL0d2Tt57nLFfgC3yeT/QDwgBhOLr6AtrXsYDiYbXBcMD9Fu76Lhlt0UJ/E4cvpzHBwd9hu4jV6i1+gtitB7dIQ+oWM0RRQJ9BP9Qr9Hf70db997cTk62uo9z9HG8V79AyNiGp4=</latexit><latexit sha1_base64="Q2lmKV+jF55GTzlh2vfrmDees1s=">AAADY3icfZLNbtQwFIXdCT8l5act7BCSRYrEhlGSWdBlJViwLBLTFiVh5HhuZqyxnch2CqPUT8EWHowH4D1w0lTqpKWWIh3de4+/4+jmFWfahOGfrZF37/6Dh9uP/J3HT54+293bP9FlrShMaclLdZYTDZxJmBpmOJxVCojIOZzmqw9t//QclGal/GLWFWSCLCQrGCXGlb4e0Bl8a97Zg9luEI7D7uCbIupFgPpzPNsbfUznJa0FSEM50TqJwspkDVGGUQ7WT2sNFaErsoDESUkE6KzpElv8xlXmuCiV+6TBXfW6oyFC67XI3aQgZqmHvbZ4Wy+pTXGYNUxWtQFJL0FFzbEpcft8PGcKqOFrJwhVzGXFdEkUocb9pA2KYS6w9X0/lfCdlkIQOW/SXBGbRFmTgtS1gjZAk3IoTMqJXHAIolSxxdJcWLtpXIG53XhxZUlVd8PQ6Ii0M8d3UXF6Xi2JNKVogtj2GXA3M8ZBfL0d2Tt57nLFfgC3yeT/QDwgBhOLr6AtrXsYDiYbXBcMD9Fu76Lhlt0UJ/E4cvpzHBwd9hu4jV6i1+gtitB7dIQ+oWM0RRQJ9BP9Qr9Hf70db997cTk62uo9z9HG8V79AyNiGp4=</latexit><latexit sha1_base64="Q2lmKV+jF55GTzlh2vfrmDees1s=">AAADY3icfZLNbtQwFIXdCT8l5act7BCSRYrEhlGSWdBlJViwLBLTFiVh5HhuZqyxnch2CqPUT8EWHowH4D1w0lTqpKWWIh3de4+/4+jmFWfahOGfrZF37/6Dh9uP/J3HT54+293bP9FlrShMaclLdZYTDZxJmBpmOJxVCojIOZzmqw9t//QclGal/GLWFWSCLCQrGCXGlb4e0Bl8a97Zg9luEI7D7uCbIupFgPpzPNsbfUznJa0FSEM50TqJwspkDVGGUQ7WT2sNFaErsoDESUkE6KzpElv8xlXmuCiV+6TBXfW6oyFC67XI3aQgZqmHvbZ4Wy+pTXGYNUxWtQFJL0FFzbEpcft8PGcKqOFrJwhVzGXFdEkUocb9pA2KYS6w9X0/lfCdlkIQOW/SXBGbRFmTgtS1gjZAk3IoTMqJXHAIolSxxdJcWLtpXIG53XhxZUlVd8PQ6Ii0M8d3UXF6Xi2JNKVogtj2GXA3M8ZBfL0d2Tt57nLFfgC3yeT/QDwgBhOLr6AtrXsYDiYbXBcMD9Fu76Lhlt0UJ/E4cvpzHBwd9hu4jV6i1+gtitB7dIQ+oWM0RRQJ9BP9Qr9Hf70db997cTk62uo9z9HG8V79AyNiGp4=</latexit>

b+e

<latexit sha1_base64="Dp9/3HlwayNHHHi9Tvor15Z77H0=">AAADY3icfZLNbtQwFIXdCT8l5act7BCSRYqEhDRKMgu6rAQLlkVi2qIkjBzPzYw1thPZTmGU+inYwoPxALwHTppKnbTUUqSje+/xdxzdvOJMmzD8szXy7t1/8HD7kb/z+MnTZ7t7+ye6rBWFKS15qc5yooEzCVPDDIezSgEROYfTfPWh7Z+eg9KslF/MuoJMkIVkBaPEuNLXg3wG35p39mC2G4TjsDv4poh6EaD+HM/2Rh/TeUlrAdJQTrROorAyWUOUYZSD9dNaQ0XoiiwgcVISATprusQWv3GVOS5K5T5pcFe97miI0HotcjcpiFnqYa8t3tZLalMcZg2TVW1A0ktQUXNsStw+H8+ZAmr42glCFXNZMV0SRahxP2mDYpgLbH3fTyV8p6UQRM6bNFfEJlHWpCB1raAN0KQcCpNyIhccgihVbLE0F9ZuGldgbjdeXFlS1d0wNDoi7czxXVScnldLIk0pmiC2fQbczYxxEF9vR/ZOnrtcsR/AbTL5PxAPiMHE4itoS+sehoPJBtcFw0O027touGU3xUk8jpz+HAdHh/0GbqOX6DV6iyL0Hh2hT+gYTRFFAv1Ev9Dv0V9vx9v3XlyOjrZ6z3O0cbxX/wAachqb</latexit><latexit sha1_base64="Dp9/3HlwayNHHHi9Tvor15Z77H0=">AAADY3icfZLNbtQwFIXdCT8l5act7BCSRYqEhDRKMgu6rAQLlkVi2qIkjBzPzYw1thPZTmGU+inYwoPxALwHTppKnbTUUqSje+/xdxzdvOJMmzD8szXy7t1/8HD7kb/z+MnTZ7t7+ye6rBWFKS15qc5yooEzCVPDDIezSgEROYfTfPWh7Z+eg9KslF/MuoJMkIVkBaPEuNLXg3wG35p39mC2G4TjsDv4poh6EaD+HM/2Rh/TeUlrAdJQTrROorAyWUOUYZSD9dNaQ0XoiiwgcVISATprusQWv3GVOS5K5T5pcFe97miI0HotcjcpiFnqYa8t3tZLalMcZg2TVW1A0ktQUXNsStw+H8+ZAmr42glCFXNZMV0SRahxP2mDYpgLbH3fTyV8p6UQRM6bNFfEJlHWpCB1raAN0KQcCpNyIhccgihVbLE0F9ZuGldgbjdeXFlS1d0wNDoi7czxXVScnldLIk0pmiC2fQbczYxxEF9vR/ZOnrtcsR/AbTL5PxAPiMHE4itoS+sehoPJBtcFw0O027touGU3xUk8jpz+HAdHh/0GbqOX6DV6iyL0Hh2hT+gYTRFFAv1Ev9Dv0V9vx9v3XlyOjrZ6z3O0cbxX/wAachqb</latexit><latexit sha1_base64="Dp9/3HlwayNHHHi9Tvor15Z77H0=">AAADY3icfZLNbtQwFIXdCT8l5act7BCSRYqEhDRKMgu6rAQLlkVi2qIkjBzPzYw1thPZTmGU+inYwoPxALwHTppKnbTUUqSje+/xdxzdvOJMmzD8szXy7t1/8HD7kb/z+MnTZ7t7+ye6rBWFKS15qc5yooEzCVPDDIezSgEROYfTfPWh7Z+eg9KslF/MuoJMkIVkBaPEuNLXg3wG35p39mC2G4TjsDv4poh6EaD+HM/2Rh/TeUlrAdJQTrROorAyWUOUYZSD9dNaQ0XoiiwgcVISATprusQWv3GVOS5K5T5pcFe97miI0HotcjcpiFnqYa8t3tZLalMcZg2TVW1A0ktQUXNsStw+H8+ZAmr42glCFXNZMV0SRahxP2mDYpgLbH3fTyV8p6UQRM6bNFfEJlHWpCB1raAN0KQcCpNyIhccgihVbLE0F9ZuGldgbjdeXFlS1d0wNDoi7czxXVScnldLIk0pmiC2fQbczYxxEF9vR/ZOnrtcsR/AbTL5PxAPiMHE4itoS+sehoPJBtcFw0O027touGU3xUk8jpz+HAdHh/0GbqOX6DV6iyL0Hh2hT+gYTRFFAv1Ev9Dv0V9vx9v3XlyOjrZ6z3O0cbxX/wAachqb</latexit><latexit sha1_base64="Dp9/3HlwayNHHHi9Tvor15Z77H0=">AAADY3icfZLNbtQwFIXdCT8l5act7BCSRYqEhDRKMgu6rAQLlkVi2qIkjBzPzYw1thPZTmGU+inYwoPxALwHTppKnbTUUqSje+/xdxzdvOJMmzD8szXy7t1/8HD7kb/z+MnTZ7t7+ye6rBWFKS15qc5yooEzCVPDDIezSgEROYfTfPWh7Z+eg9KslF/MuoJMkIVkBaPEuNLXg3wG35p39mC2G4TjsDv4poh6EaD+HM/2Rh/TeUlrAdJQTrROorAyWUOUYZSD9dNaQ0XoiiwgcVISATprusQWv3GVOS5K5T5pcFe97miI0HotcjcpiFnqYa8t3tZLalMcZg2TVW1A0ktQUXNsStw+H8+ZAmr42glCFXNZMV0SRahxP2mDYpgLbH3fTyV8p6UQRM6bNFfEJlHWpCB1raAN0KQcCpNyIhccgihVbLE0F9ZuGldgbjdeXFlS1d0wNDoi7czxXVScnldLIk0pmiC2fQbczYxxEF9vR/ZOnrtcsR/AbTL5PxAPiMHE4itoS+sehoPJBtcFw0O027touGU3xUk8jpz+HAdHh/0GbqOX6DV6iyL0Hh2hT+gYTRFFAv1Ev9Dv0V9vx9v3XlyOjrZ6z3O0cbxX/wAachqb</latexit>

b�e<latexit sha1_base64="UBgNQHbVGfop0CJDW85glJJ1Bt0=">AAADY3icfZLNbtQwFIXdCT8l5act7BCSRYrEhlGSWdBlJViwLBLTFiVh5HhuZqyxnch2CqPUT8EWHowH4D1w0lTqpKWWIh3de4+/4+jmFWfahOGfrZF37/6Dh9uP/J3HT54+293bP9FlrShMaclLdZYTDZxJmBpmOJxVCojIOZzmqw9t//QclGal/GLWFWSCLCQrGCXGlb4e5DP41ryzB7PdIByH3cE3RdSLAPXneLY3+pjOS1oLkIZyonUShZXJGqIMoxysn9YaKkJXZAGJk5II0FnTJbb4javMcVEq90mDu+p1R0OE1muRu0lBzFIPe23xtl5Sm+Iwa5isagOSXoKKmmNT4vb5eM4UUMPXThCqmMuK6ZIoQo37SRsUw1xg6/t+KuE7LYUgct6kuSI2ibImBalrBW2AJuVQmJQTueAQRKlii6W5sHbTuAJzu/HiypKq7oah0RFpZ47vouL0vFoSaUrRBLHtM+BuZoyD+Ho7snfy3OWK/QBuk8n/gXhADCYWX0FbWvcwHEw2uC4YHqLd3kXDLbspTuJx5PTnODg67DdwG71Er9FbFKH36Ah9QsdoiigS6Cf6hX6P/no73r734nJ0tNV7nqON4736ByBkGp0=</latexit><latexit sha1_base64="UBgNQHbVGfop0CJDW85glJJ1Bt0=">AAADY3icfZLNbtQwFIXdCT8l5act7BCSRYrEhlGSWdBlJViwLBLTFiVh5HhuZqyxnch2CqPUT8EWHowH4D1w0lTqpKWWIh3de4+/4+jmFWfahOGfrZF37/6Dh9uP/J3HT54+293bP9FlrShMaclLdZYTDZxJmBpmOJxVCojIOZzmqw9t//QclGal/GLWFWSCLCQrGCXGlb4e5DP41ryzB7PdIByH3cE3RdSLAPXneLY3+pjOS1oLkIZyonUShZXJGqIMoxysn9YaKkJXZAGJk5II0FnTJbb4javMcVEq90mDu+p1R0OE1muRu0lBzFIPe23xtl5Sm+Iwa5isagOSXoKKmmNT4vb5eM4UUMPXThCqmMuK6ZIoQo37SRsUw1xg6/t+KuE7LYUgct6kuSI2ibImBalrBW2AJuVQmJQTueAQRKlii6W5sHbTuAJzu/HiypKq7oah0RFpZ47vouL0vFoSaUrRBLHtM+BuZoyD+Ho7snfy3OWK/QBuk8n/gXhADCYWX0FbWvcwHEw2uC4YHqLd3kXDLbspTuJx5PTnODg67DdwG71Er9FbFKH36Ah9QsdoiigS6Cf6hX6P/no73r734nJ0tNV7nqON4736ByBkGp0=</latexit><latexit sha1_base64="UBgNQHbVGfop0CJDW85glJJ1Bt0=">AAADY3icfZLNbtQwFIXdCT8l5act7BCSRYrEhlGSWdBlJViwLBLTFiVh5HhuZqyxnch2CqPUT8EWHowH4D1w0lTqpKWWIh3de4+/4+jmFWfahOGfrZF37/6Dh9uP/J3HT54+293bP9FlrShMaclLdZYTDZxJmBpmOJxVCojIOZzmqw9t//QclGal/GLWFWSCLCQrGCXGlb4e5DP41ryzB7PdIByH3cE3RdSLAPXneLY3+pjOS1oLkIZyonUShZXJGqIMoxysn9YaKkJXZAGJk5II0FnTJbb4javMcVEq90mDu+p1R0OE1muRu0lBzFIPe23xtl5Sm+Iwa5isagOSXoKKmmNT4vb5eM4UUMPXThCqmMuK6ZIoQo37SRsUw1xg6/t+KuE7LYUgct6kuSI2ibImBalrBW2AJuVQmJQTueAQRKlii6W5sHbTuAJzu/HiypKq7oah0RFpZ47vouL0vFoSaUrRBLHtM+BuZoyD+Ho7snfy3OWK/QBuk8n/gXhADCYWX0FbWvcwHEw2uC4YHqLd3kXDLbspTuJx5PTnODg67DdwG71Er9FbFKH36Ah9QsdoiigS6Cf6hX6P/no73r734nJ0tNV7nqON4736ByBkGp0=</latexit><latexit sha1_base64="UBgNQHbVGfop0CJDW85glJJ1Bt0=">AAADY3icfZLNbtQwFIXdCT8l5act7BCSRYrEhlGSWdBlJViwLBLTFiVh5HhuZqyxnch2CqPUT8EWHowH4D1w0lTqpKWWIh3de4+/4+jmFWfahOGfrZF37/6Dh9uP/J3HT54+293bP9FlrShMaclLdZYTDZxJmBpmOJxVCojIOZzmqw9t//QclGal/GLWFWSCLCQrGCXGlb4e5DP41ryzB7PdIByH3cE3RdSLAPXneLY3+pjOS1oLkIZyonUShZXJGqIMoxysn9YaKkJXZAGJk5II0FnTJbb4javMcVEq90mDu+p1R0OE1muRu0lBzFIPe23xtl5Sm+Iwa5isagOSXoKKmmNT4vb5eM4UUMPXThCqmMuK6ZIoQo37SRsUw1xg6/t+KuE7LYUgct6kuSI2ibImBalrBW2AJuVQmJQTueAQRKlii6W5sHbTuAJzu/HiypKq7oah0RFpZ47vouL0vFoSaUrRBLHtM+BuZoyD+Ho7snfy3OWK/QBuk8n/gXhADCYWX0FbWvcwHEw2uC4YHqLd3kXDLbspTuJx5PTnODg67DdwG71Er9FbFKH36Ah9QsdoiigS6Cf6hX6P/no73r734nJ0tNV7nqON4736ByBkGp0=</latexit>

WLe<latexit sha1_base64="vuVCoGg+W/WyeLZWjKhSD11xU5U=">AAADa3icfZLNbhMxFIXdDD8l/DSlO2BhMa3EKspMFnRZCRYsWBSJNJXiKPI4N4kV2zOy75RGUz8JW3goHoJ3wJmmUjMtvdJIR/fe4++M5axQ0mGv92enFT16/OTp7rP28xcvX+119l+fuby0AgYiV7k9z7gDJQ0MUKKC88IC15mCYbb8tJ4PL8A6mZvvuCpgrPncyJkUHENr0tk7ZAiXaHU1/OoncDjpxL1ury56VyQbEZNNnU72W5/ZNBelBoNCcedGSa/AccUtSqHAt1npoOBiyecwCtJwDW5c1ck9PQqdKZ3lNnwGad297ai4dm6ls7CpOS5cc7Zu3jcblTg7HlfSFCWCEdegWako5nR9DXQqLQhUqyC4sDJkpWLBLRcYLmuLgjIE9u12mxn4IXKtuZlWLLPcj5JxxcC40sI6QMUUzJApbuYK4oRZOV/glffbxiXg/carGwuz9QlNYyCK2pw+RKXsolhwg7mu4tRvMtB6p0vj9PY48Q/ywuFWXoLyo/7/gbRBjPue3kDXtPrHaNzf4oZgtIkO7y5pvrK74iztJkF/S+OT480L3CVvyXvygSTkIzkhX8gpGRBBSvKT/CK/W3+jg+hN9O56tbWz8RyQrYqO/gHSBx3A</latexit><latexit sha1_base64="vuVCoGg+W/WyeLZWjKhSD11xU5U=">AAADa3icfZLNbhMxFIXdDD8l/DSlO2BhMa3EKspMFnRZCRYsWBSJNJXiKPI4N4kV2zOy75RGUz8JW3goHoJ3wJmmUjMtvdJIR/fe4++M5axQ0mGv92enFT16/OTp7rP28xcvX+119l+fuby0AgYiV7k9z7gDJQ0MUKKC88IC15mCYbb8tJ4PL8A6mZvvuCpgrPncyJkUHENr0tk7ZAiXaHU1/OoncDjpxL1ury56VyQbEZNNnU72W5/ZNBelBoNCcedGSa/AccUtSqHAt1npoOBiyecwCtJwDW5c1ck9PQqdKZ3lNnwGad297ai4dm6ls7CpOS5cc7Zu3jcblTg7HlfSFCWCEdegWako5nR9DXQqLQhUqyC4sDJkpWLBLRcYLmuLgjIE9u12mxn4IXKtuZlWLLPcj5JxxcC40sI6QMUUzJApbuYK4oRZOV/glffbxiXg/carGwuz9QlNYyCK2pw+RKXsolhwg7mu4tRvMtB6p0vj9PY48Q/ywuFWXoLyo/7/gbRBjPue3kDXtPrHaNzf4oZgtIkO7y5pvrK74iztJkF/S+OT480L3CVvyXvygSTkIzkhX8gpGRBBSvKT/CK/W3+jg+hN9O56tbWz8RyQrYqO/gHSBx3A</latexit><latexit sha1_base64="vuVCoGg+W/WyeLZWjKhSD11xU5U=">AAADa3icfZLNbhMxFIXdDD8l/DSlO2BhMa3EKspMFnRZCRYsWBSJNJXiKPI4N4kV2zOy75RGUz8JW3goHoJ3wJmmUjMtvdJIR/fe4++M5axQ0mGv92enFT16/OTp7rP28xcvX+119l+fuby0AgYiV7k9z7gDJQ0MUKKC88IC15mCYbb8tJ4PL8A6mZvvuCpgrPncyJkUHENr0tk7ZAiXaHU1/OoncDjpxL1ury56VyQbEZNNnU72W5/ZNBelBoNCcedGSa/AccUtSqHAt1npoOBiyecwCtJwDW5c1ck9PQqdKZ3lNnwGad297ai4dm6ls7CpOS5cc7Zu3jcblTg7HlfSFCWCEdegWako5nR9DXQqLQhUqyC4sDJkpWLBLRcYLmuLgjIE9u12mxn4IXKtuZlWLLPcj5JxxcC40sI6QMUUzJApbuYK4oRZOV/glffbxiXg/carGwuz9QlNYyCK2pw+RKXsolhwg7mu4tRvMtB6p0vj9PY48Q/ywuFWXoLyo/7/gbRBjPue3kDXtPrHaNzf4oZgtIkO7y5pvrK74iztJkF/S+OT480L3CVvyXvygSTkIzkhX8gpGRBBSvKT/CK/W3+jg+hN9O56tbWz8RyQrYqO/gHSBx3A</latexit><latexit sha1_base64="vuVCoGg+W/WyeLZWjKhSD11xU5U=">AAADa3icfZLNbhMxFIXdDD8l/DSlO2BhMa3EKspMFnRZCRYsWBSJNJXiKPI4N4kV2zOy75RGUz8JW3goHoJ3wJmmUjMtvdJIR/fe4++M5axQ0mGv92enFT16/OTp7rP28xcvX+119l+fuby0AgYiV7k9z7gDJQ0MUKKC88IC15mCYbb8tJ4PL8A6mZvvuCpgrPncyJkUHENr0tk7ZAiXaHU1/OoncDjpxL1ury56VyQbEZNNnU72W5/ZNBelBoNCcedGSa/AccUtSqHAt1npoOBiyecwCtJwDW5c1ck9PQqdKZ3lNnwGad297ai4dm6ls7CpOS5cc7Zu3jcblTg7HlfSFCWCEdegWako5nR9DXQqLQhUqyC4sDJkpWLBLRcYLmuLgjIE9u12mxn4IXKtuZlWLLPcj5JxxcC40sI6QMUUzJApbuYK4oRZOV/glffbxiXg/carGwuz9QlNYyCK2pw+RKXsolhwg7mu4tRvMtB6p0vj9PY48Q/ywuFWXoLyo/7/gbRBjPue3kDXtPrHaNzf4oZgtIkO7y5pvrK74iztJkF/S+OT480L3CVvyXvygSTkIzkhX8gpGRBBSvKT/CK/W3+jg+hN9O56tbWz8RyQrYqO/gHSBx3A</latexit>

@WLe

@xi<latexit sha1_base64="XtVSII2w9smARWHSrkjFxguyDxY=">AAADiXicfVJNbxMxEHWzfJSUjxSOXCy2SFyIspsDEadK5cCBQ5FIUymOIq8zm1jxelf2bEm09a/g13CFX8G/wbtNRZOWjmTp6c28eePRJIWSFnu9P3ut4MHDR4/3n7QPnj57/qJz+PLM5qURMBS5ys15wi0oqWGIEhWcFwZ4ligYJcuTOj+6AGNlrr/huoBJxudaplJw9NS08/6IpYaLihXcoOSKMoQVmqwafXFTcP/41VS6o2kn7HV7TdDbINqAkGzidHrY+sRmuSgz0CgUt3Yc9QqcVHVTocC1WWmh4GLJ5zD2UPMM7KRq/uXoW8/MaJob/zTShr2pqHhm7TpLfGXGcWF3czV5V25cYjqYVFIXJYIWV0ZpqSjmtF4SnUkDAtXaAy6M9LNSseB+T+hXueWC0g/s2u020/Bd5FnG9axiieFuHE0qBtqWBuoBKqYgRaa4nisII2bkfIGXzm0Ll4B3Cy+vJcw0HXaF3lE04vg+V8ouigXXmGdVGLvNDLSp6dIwvpmO3L1+vrmRK1Bu3P+/Id1xDPuOXpvWbs3HaNjf8vWD0V1rf3fR7pXdBmdxN/L4axweDzYXuE9ekzfkHYnIB3JMPpNTMiSC/CA/yS/yOzgIomAQfLwqbe1tNK/IVgQnfwETtCnn</latexit><latexit sha1_base64="XtVSII2w9smARWHSrkjFxguyDxY=">AAADiXicfVJNbxMxEHWzfJSUjxSOXCy2SFyIspsDEadK5cCBQ5FIUymOIq8zm1jxelf2bEm09a/g13CFX8G/wbtNRZOWjmTp6c28eePRJIWSFnu9P3ut4MHDR4/3n7QPnj57/qJz+PLM5qURMBS5ys15wi0oqWGIEhWcFwZ4ligYJcuTOj+6AGNlrr/huoBJxudaplJw9NS08/6IpYaLihXcoOSKMoQVmqwafXFTcP/41VS6o2kn7HV7TdDbINqAkGzidHrY+sRmuSgz0CgUt3Yc9QqcVHVTocC1WWmh4GLJ5zD2UPMM7KRq/uXoW8/MaJob/zTShr2pqHhm7TpLfGXGcWF3czV5V25cYjqYVFIXJYIWV0ZpqSjmtF4SnUkDAtXaAy6M9LNSseB+T+hXueWC0g/s2u020/Bd5FnG9axiieFuHE0qBtqWBuoBKqYgRaa4nisII2bkfIGXzm0Ll4B3Cy+vJcw0HXaF3lE04vg+V8ouigXXmGdVGLvNDLSp6dIwvpmO3L1+vrmRK1Bu3P+/Id1xDPuOXpvWbs3HaNjf8vWD0V1rf3fR7pXdBmdxN/L4axweDzYXuE9ekzfkHYnIB3JMPpNTMiSC/CA/yS/yOzgIomAQfLwqbe1tNK/IVgQnfwETtCnn</latexit><latexit sha1_base64="XtVSII2w9smARWHSrkjFxguyDxY=">AAADiXicfVJNbxMxEHWzfJSUjxSOXCy2SFyIspsDEadK5cCBQ5FIUymOIq8zm1jxelf2bEm09a/g13CFX8G/wbtNRZOWjmTp6c28eePRJIWSFnu9P3ut4MHDR4/3n7QPnj57/qJz+PLM5qURMBS5ys15wi0oqWGIEhWcFwZ4ligYJcuTOj+6AGNlrr/huoBJxudaplJw9NS08/6IpYaLihXcoOSKMoQVmqwafXFTcP/41VS6o2kn7HV7TdDbINqAkGzidHrY+sRmuSgz0CgUt3Yc9QqcVHVTocC1WWmh4GLJ5zD2UPMM7KRq/uXoW8/MaJob/zTShr2pqHhm7TpLfGXGcWF3czV5V25cYjqYVFIXJYIWV0ZpqSjmtF4SnUkDAtXaAy6M9LNSseB+T+hXueWC0g/s2u020/Bd5FnG9axiieFuHE0qBtqWBuoBKqYgRaa4nisII2bkfIGXzm0Ll4B3Cy+vJcw0HXaF3lE04vg+V8ouigXXmGdVGLvNDLSp6dIwvpmO3L1+vrmRK1Bu3P+/Id1xDPuOXpvWbs3HaNjf8vWD0V1rf3fR7pXdBmdxN/L4axweDzYXuE9ekzfkHYnIB3JMPpNTMiSC/CA/yS/yOzgIomAQfLwqbe1tNK/IVgQnfwETtCnn</latexit><latexit sha1_base64="XtVSII2w9smARWHSrkjFxguyDxY=">AAADiXicfVJNbxMxEHWzfJSUjxSOXCy2SFyIspsDEadK5cCBQ5FIUymOIq8zm1jxelf2bEm09a/g13CFX8G/wbtNRZOWjmTp6c28eePRJIWSFnu9P3ut4MHDR4/3n7QPnj57/qJz+PLM5qURMBS5ys15wi0oqWGIEhWcFwZ4ligYJcuTOj+6AGNlrr/huoBJxudaplJw9NS08/6IpYaLihXcoOSKMoQVmqwafXFTcP/41VS6o2kn7HV7TdDbINqAkGzidHrY+sRmuSgz0CgUt3Yc9QqcVHVTocC1WWmh4GLJ5zD2UPMM7KRq/uXoW8/MaJob/zTShr2pqHhm7TpLfGXGcWF3czV5V25cYjqYVFIXJYIWV0ZpqSjmtF4SnUkDAtXaAy6M9LNSseB+T+hXueWC0g/s2u020/Bd5FnG9axiieFuHE0qBtqWBuoBKqYgRaa4nisII2bkfIGXzm0Ll4B3Cy+vJcw0HXaF3lE04vg+V8ouigXXmGdVGLvNDLSp6dIwvpmO3L1+vrmRK1Bu3P+/Id1xDPuOXpvWbs3HaNjf8vWD0V1rf3fR7pXdBmdxN/L4axweDzYXuE9ekzfkHYnIB3JMPpNTMiSC/CA/yS/yOzgIomAQfLwqbe1tNK/IVgQnfwETtCnn</latexit>

a+i

<latexit sha1_base64="Cdk3iv5p/PNaPe6NEd/cDu/1z10=">AAADY3icfZLNbtQwFIXdCT8l5act7BCSRYqEhDRKMgu6rAQLlkVi2qIkjBzPzYw1thPZTmGU+inYwoPxALwHTppKnbTUUqSje+/xdxzdvOJMmzD8szXy7t1/8HD7kb/z+MnTZ7t7+ye6rBWFKS15qc5yooEzCVPDDIezSgEROYfTfPWh7Z+eg9KslF/MuoJMkIVkBaPEuNLXAzJj35p39mC2G4TjsDv4poh6EaD+HM/2Rh/TeUlrAdJQTrROorAyWUOUYZSD9dNaQ0XoiiwgcVISATprusQWv3GVOS5K5T5pcFe97miI0HotcjcpiFnqYa8t3tZLalMcZg2TVW1A0ktQUXNsStw+H8+ZAmr42glCFXNZMV0SRahxP2mDYpgLbH3fTyV8p6UQRM6bNFfEJlHWpCB1raAN0KQcCpNyIhccgihVbLE0F9ZuGldgbjdeXFlS1d0wNDoi7czxXVScnldLIk0pmiC2fQbczYxxEF9vR/ZOnrtcsR/AbTL5PxAPiMHE4itoS+sehoPJBtcFw0O027touGU3xUk8jpz+HAdHh/0GbqOX6DV6iyL0Hh2hT+gYTRFFAv1Ev9Dv0V9vx9v3XlyOjrZ6z3O0cbxX/wAjZBqe</latexit><latexit sha1_base64="Cdk3iv5p/PNaPe6NEd/cDu/1z10=">AAADY3icfZLNbtQwFIXdCT8l5act7BCSRYqEhDRKMgu6rAQLlkVi2qIkjBzPzYw1thPZTmGU+inYwoPxALwHTppKnbTUUqSje+/xdxzdvOJMmzD8szXy7t1/8HD7kb/z+MnTZ7t7+ye6rBWFKS15qc5yooEzCVPDDIezSgEROYfTfPWh7Z+eg9KslF/MuoJMkIVkBaPEuNLXAzJj35p39mC2G4TjsDv4poh6EaD+HM/2Rh/TeUlrAdJQTrROorAyWUOUYZSD9dNaQ0XoiiwgcVISATprusQWv3GVOS5K5T5pcFe97miI0HotcjcpiFnqYa8t3tZLalMcZg2TVW1A0ktQUXNsStw+H8+ZAmr42glCFXNZMV0SRahxP2mDYpgLbH3fTyV8p6UQRM6bNFfEJlHWpCB1raAN0KQcCpNyIhccgihVbLE0F9ZuGldgbjdeXFlS1d0wNDoi7czxXVScnldLIk0pmiC2fQbczYxxEF9vR/ZOnrtcsR/AbTL5PxAPiMHE4itoS+sehoPJBtcFw0O027touGU3xUk8jpz+HAdHh/0GbqOX6DV6iyL0Hh2hT+gYTRFFAv1Ev9Dv0V9vx9v3XlyOjrZ6z3O0cbxX/wAjZBqe</latexit><latexit sha1_base64="Cdk3iv5p/PNaPe6NEd/cDu/1z10=">AAADY3icfZLNbtQwFIXdCT8l5act7BCSRYqEhDRKMgu6rAQLlkVi2qIkjBzPzYw1thPZTmGU+inYwoPxALwHTppKnbTUUqSje+/xdxzdvOJMmzD8szXy7t1/8HD7kb/z+MnTZ7t7+ye6rBWFKS15qc5yooEzCVPDDIezSgEROYfTfPWh7Z+eg9KslF/MuoJMkIVkBaPEuNLXAzJj35p39mC2G4TjsDv4poh6EaD+HM/2Rh/TeUlrAdJQTrROorAyWUOUYZSD9dNaQ0XoiiwgcVISATprusQWv3GVOS5K5T5pcFe97miI0HotcjcpiFnqYa8t3tZLalMcZg2TVW1A0ktQUXNsStw+H8+ZAmr42glCFXNZMV0SRahxP2mDYpgLbH3fTyV8p6UQRM6bNFfEJlHWpCB1raAN0KQcCpNyIhccgihVbLE0F9ZuGldgbjdeXFlS1d0wNDoi7czxXVScnldLIk0pmiC2fQbczYxxEF9vR/ZOnrtcsR/AbTL5PxAPiMHE4itoS+sehoPJBtcFw0O027touGU3xUk8jpz+HAdHh/0GbqOX6DV6iyL0Hh2hT+gYTRFFAv1Ev9Dv0V9vx9v3XlyOjrZ6z3O0cbxX/wAjZBqe</latexit><latexit sha1_base64="Cdk3iv5p/PNaPe6NEd/cDu/1z10=">AAADY3icfZLNbtQwFIXdCT8l5act7BCSRYqEhDRKMgu6rAQLlkVi2qIkjBzPzYw1thPZTmGU+inYwoPxALwHTppKnbTUUqSje+/xdxzdvOJMmzD8szXy7t1/8HD7kb/z+MnTZ7t7+ye6rBWFKS15qc5yooEzCVPDDIezSgEROYfTfPWh7Z+eg9KslF/MuoJMkIVkBaPEuNLXAzJj35p39mC2G4TjsDv4poh6EaD+HM/2Rh/TeUlrAdJQTrROorAyWUOUYZSD9dNaQ0XoiiwgcVISATprusQWv3GVOS5K5T5pcFe97miI0HotcjcpiFnqYa8t3tZLalMcZg2TVW1A0ktQUXNsStw+H8+ZAmr42glCFXNZMV0SRahxP2mDYpgLbH3fTyV8p6UQRM6bNFfEJlHWpCB1raAN0KQcCpNyIhccgihVbLE0F9ZuGldgbjdeXFlS1d0wNDoi7czxXVScnldLIk0pmiC2fQbczYxxEF9vR/ZOnrtcsR/AbTL5PxAPiMHE4itoS+sehoPJBtcFw0O027touGU3xUk8jpz+HAdHh/0GbqOX6DV6iyL0Hh2hT+gYTRFFAv1Ev9Dv0V9vx9v3XlyOjrZ6z3O0cbxX/wAjZBqe</latexit>

a�i

<latexit sha1_base64="dzooKQJ+/65aIykkkdZy6OTHeDA=">AAADY3icfZLNbtQwFIXdCT8l5act7BCSRYrEhlGSWdBlJViwLBLTFiVh5HhuZqyxnch2CqPUT8EWHowH4D1w0lTqpKWWIh3de4+/4+jmFWfahOGfrZF37/6Dh9uP/J3HT54+293bP9FlrShMaclLdZYTDZxJmBpmOJxVCojIOZzmqw9t//QclGal/GLWFWSCLCQrGCXGlb4ekBn71ryzB7PdIByH3cE3RdSLAPXneLY3+pjOS1oLkIZyonUShZXJGqIMoxysn9YaKkJXZAGJk5II0FnTJbb4javMcVEq90mDu+p1R0OE1muRu0lBzFIPe23xtl5Sm+Iwa5isagOSXoKKmmNT4vb5eM4UUMPXThCqmMuK6ZIoQo37SRsUw1xg6/t+KuE7LYUgct6kuSI2ibImBalrBW2AJuVQmJQTueAQRKlii6W5sHbTuAJzu/HiypKq7oah0RFpZ47vouL0vFoSaUrRBLHtM+BuZoyD+Ho7snfy3OWK/QBuk8n/gXhADCYWX0FbWvcwHEw2uC4YHqLd3kXDLbspTuJx5PTnODg67DdwG71Er9FbFKH36Ah9QsdoiigS6Cf6hX6P/no73r734nJ0tNV7nqON4736BylWGqA=</latexit><latexit sha1_base64="dzooKQJ+/65aIykkkdZy6OTHeDA=">AAADY3icfZLNbtQwFIXdCT8l5act7BCSRYrEhlGSWdBlJViwLBLTFiVh5HhuZqyxnch2CqPUT8EWHowH4D1w0lTqpKWWIh3de4+/4+jmFWfahOGfrZF37/6Dh9uP/J3HT54+293bP9FlrShMaclLdZYTDZxJmBpmOJxVCojIOZzmqw9t//QclGal/GLWFWSCLCQrGCXGlb4ekBn71ryzB7PdIByH3cE3RdSLAPXneLY3+pjOS1oLkIZyonUShZXJGqIMoxysn9YaKkJXZAGJk5II0FnTJbb4javMcVEq90mDu+p1R0OE1muRu0lBzFIPe23xtl5Sm+Iwa5isagOSXoKKmmNT4vb5eM4UUMPXThCqmMuK6ZIoQo37SRsUw1xg6/t+KuE7LYUgct6kuSI2ibImBalrBW2AJuVQmJQTueAQRKlii6W5sHbTuAJzu/HiypKq7oah0RFpZ47vouL0vFoSaUrRBLHtM+BuZoyD+Ho7snfy3OWK/QBuk8n/gXhADCYWX0FbWvcwHEw2uC4YHqLd3kXDLbspTuJx5PTnODg67DdwG71Er9FbFKH36Ah9QsdoiigS6Cf6hX6P/no73r734nJ0tNV7nqON4736BylWGqA=</latexit><latexit sha1_base64="dzooKQJ+/65aIykkkdZy6OTHeDA=">AAADY3icfZLNbtQwFIXdCT8l5act7BCSRYrEhlGSWdBlJViwLBLTFiVh5HhuZqyxnch2CqPUT8EWHowH4D1w0lTqpKWWIh3de4+/4+jmFWfahOGfrZF37/6Dh9uP/J3HT54+293bP9FlrShMaclLdZYTDZxJmBpmOJxVCojIOZzmqw9t//QclGal/GLWFWSCLCQrGCXGlb4ekBn71ryzB7PdIByH3cE3RdSLAPXneLY3+pjOS1oLkIZyonUShZXJGqIMoxysn9YaKkJXZAGJk5II0FnTJbb4javMcVEq90mDu+p1R0OE1muRu0lBzFIPe23xtl5Sm+Iwa5isagOSXoKKmmNT4vb5eM4UUMPXThCqmMuK6ZIoQo37SRsUw1xg6/t+KuE7LYUgct6kuSI2ibImBalrBW2AJuVQmJQTueAQRKlii6W5sHbTuAJzu/HiypKq7oah0RFpZ47vouL0vFoSaUrRBLHtM+BuZoyD+Ho7snfy3OWK/QBuk8n/gXhADCYWX0FbWvcwHEw2uC4YHqLd3kXDLbspTuJx5PTnODg67DdwG71Er9FbFKH36Ah9QsdoiigS6Cf6hX6P/no73r734nJ0tNV7nqON4736BylWGqA=</latexit><latexit sha1_base64="dzooKQJ+/65aIykkkdZy6OTHeDA=">AAADY3icfZLNbtQwFIXdCT8l5act7BCSRYrEhlGSWdBlJViwLBLTFiVh5HhuZqyxnch2CqPUT8EWHowH4D1w0lTqpKWWIh3de4+/4+jmFWfahOGfrZF37/6Dh9uP/J3HT54+293bP9FlrShMaclLdZYTDZxJmBpmOJxVCojIOZzmqw9t//QclGal/GLWFWSCLCQrGCXGlb4ekBn71ryzB7PdIByH3cE3RdSLAPXneLY3+pjOS1oLkIZyonUShZXJGqIMoxysn9YaKkJXZAGJk5II0FnTJbb4javMcVEq90mDu+p1R0OE1muRu0lBzFIPe23xtl5Sm+Iwa5isagOSXoKKmmNT4vb5eM4UUMPXThCqmMuK6ZIoQo37SRsUw1xg6/t+KuE7LYUgct6kuSI2ibImBalrBW2AJuVQmJQTueAQRKlii6W5sHbTuAJzu/HiypKq7oah0RFpZ47vouL0vFoSaUrRBLHtM+BuZoyD+Ho7snfy3OWK/QBuk8n/gXhADCYWX0FbWvcwHEw2uC4YHqLd3kXDLbspTuJx5PTnODg67DdwG71Er9FbFKH36Ah9QsdoiigS6Cf6hX6P/no73r734nJ0tNV7nqON4736BylWGqA=</latexit>

(a)

⇢<latexit sha1_base64="6YAN3FZRtZC1dyqT+YywGjVg5KU=">AAADYHicfZLNbtQwFIXdCT9D+GkHdrCxSJFYjSaZBV1WggXLIjHTSnFUOZ6biRnHjmynMErzDmzhzdjyJDhpKnXSUkuRju69x99xdNNScGNnsz97I+/Bw0ePx0/8p8+ev9g/mLxcGlVpBgumhNJnKTUguISF5VbAWamBFqmA03Tzse2fXoA2XMmvdltCUtC15Bln1LrS8pDoXB2eHwSz6aw7+LYIexGg/pycT0afyEqxqgBpmaDGxOGstElNteVMQOOTykBJ2YauIXZS0gJMUndxG/zOVVY4U9p90uKuetNR08KYbZG6yYLa3Ax7bfGuXlzZ7CipuSwrC5JdgbJKYKtw+3a84hqYFVsnKNPcZcUsp5oy6/7QDsVyF7jxfZ9I+M5UUVC5qkmqaROHSU1AmkpDG6AmAjJLBJVrAUFINF/n9rJpdo0bsHcbL68tRHc3DI2OyDpzdB8Vk4syp9Kqog6ips+Au5kpDqKb7bC5l+cu1/wHiCae/x+IB8Rg3uBraEvrHoaD+Q7XBcNDtNu7cLhlt8UymoZOf4mC46N+A8foDXqL3qMQfUDH6DM6QQvE0Df0E/1Cv0d/vbG3702uRkd7vecV2jne63+jGxl7</latexit><latexit sha1_base64="6YAN3FZRtZC1dyqT+YywGjVg5KU=">AAADYHicfZLNbtQwFIXdCT9D+GkHdrCxSJFYjSaZBV1WggXLIjHTSnFUOZ6biRnHjmynMErzDmzhzdjyJDhpKnXSUkuRju69x99xdNNScGNnsz97I+/Bw0ePx0/8p8+ev9g/mLxcGlVpBgumhNJnKTUguISF5VbAWamBFqmA03Tzse2fXoA2XMmvdltCUtC15Bln1LrS8pDoXB2eHwSz6aw7+LYIexGg/pycT0afyEqxqgBpmaDGxOGstElNteVMQOOTykBJ2YauIXZS0gJMUndxG/zOVVY4U9p90uKuetNR08KYbZG6yYLa3Ax7bfGuXlzZ7CipuSwrC5JdgbJKYKtw+3a84hqYFVsnKNPcZcUsp5oy6/7QDsVyF7jxfZ9I+M5UUVC5qkmqaROHSU1AmkpDG6AmAjJLBJVrAUFINF/n9rJpdo0bsHcbL68tRHc3DI2OyDpzdB8Vk4syp9Kqog6ips+Au5kpDqKb7bC5l+cu1/wHiCae/x+IB8Rg3uBraEvrHoaD+Q7XBcNDtNu7cLhlt8UymoZOf4mC46N+A8foDXqL3qMQfUDH6DM6QQvE0Df0E/1Cv0d/vbG3702uRkd7vecV2jne63+jGxl7</latexit><latexit sha1_base64="6YAN3FZRtZC1dyqT+YywGjVg5KU=">AAADYHicfZLNbtQwFIXdCT9D+GkHdrCxSJFYjSaZBV1WggXLIjHTSnFUOZ6biRnHjmynMErzDmzhzdjyJDhpKnXSUkuRju69x99xdNNScGNnsz97I+/Bw0ePx0/8p8+ev9g/mLxcGlVpBgumhNJnKTUguISF5VbAWamBFqmA03Tzse2fXoA2XMmvdltCUtC15Bln1LrS8pDoXB2eHwSz6aw7+LYIexGg/pycT0afyEqxqgBpmaDGxOGstElNteVMQOOTykBJ2YauIXZS0gJMUndxG/zOVVY4U9p90uKuetNR08KYbZG6yYLa3Ax7bfGuXlzZ7CipuSwrC5JdgbJKYKtw+3a84hqYFVsnKNPcZcUsp5oy6/7QDsVyF7jxfZ9I+M5UUVC5qkmqaROHSU1AmkpDG6AmAjJLBJVrAUFINF/n9rJpdo0bsHcbL68tRHc3DI2OyDpzdB8Vk4syp9Kqog6ips+Au5kpDqKb7bC5l+cu1/wHiCae/x+IB8Rg3uBraEvrHoaD+Q7XBcNDtNu7cLhlt8UymoZOf4mC46N+A8foDXqL3qMQfUDH6DM6QQvE0Df0E/1Cv0d/vbG3702uRkd7vecV2jne63+jGxl7</latexit><latexit sha1_base64="6YAN3FZRtZC1dyqT+YywGjVg5KU=">AAADYHicfZLNbtQwFIXdCT9D+GkHdrCxSJFYjSaZBV1WggXLIjHTSnFUOZ6biRnHjmynMErzDmzhzdjyJDhpKnXSUkuRju69x99xdNNScGNnsz97I+/Bw0ePx0/8p8+ev9g/mLxcGlVpBgumhNJnKTUguISF5VbAWamBFqmA03Tzse2fXoA2XMmvdltCUtC15Bln1LrS8pDoXB2eHwSz6aw7+LYIexGg/pycT0afyEqxqgBpmaDGxOGstElNteVMQOOTykBJ2YauIXZS0gJMUndxG/zOVVY4U9p90uKuetNR08KYbZG6yYLa3Ax7bfGuXlzZ7CipuSwrC5JdgbJKYKtw+3a84hqYFVsnKNPcZcUsp5oy6/7QDsVyF7jxfZ9I+M5UUVC5qkmqaROHSU1AmkpDG6AmAjJLBJVrAUFINF/n9rJpdo0bsHcbL68tRHc3DI2OyDpzdB8Vk4syp9Kqog6ips+Au5kpDqKb7bC5l+cu1/wHiCae/x+IB8Rg3uBraEvrHoaD+Q7XBcNDtNu7cLhlt8UymoZOf4mC46N+A8foDXqL3qMQfUDH6DM6QQvE0Df0E/1Cv0d/vbG3702uRkd7vecV2jne63+jGxl7</latexit>

au,v<latexit sha1_base64="tljCRb10HB37ROlEsKureKiWkcU=">AAADZHicfZLfbtMwFMa9hj8jDLYxcYWELDIkLlCVpBfschJccDkkuk0kUeW4p61V24lsp1uV+S24hffiBXgOnCyT1mzMUqRP55zPv8/RyUvOtAnDP1sD79HjJ0+3n/nPd1683N3bf3Wqi0pRGNOCF+o8Jxo4kzA2zHA4LxUQkXM4y5efm/7ZCpRmhfxu1iVkgswlmzFKjCv9OCSTuvqIV/ZwsheEw7A9+K6IOhGg7pxM9gdf0mlBKwHSUE60TqKwNFlNlGGUg/XTSkNJ6JLMIXFSEgE6q9vIFr93lSmeFcp90uC2ettRE6H1WuRuUhCz0P1eU7yvl1RmdpTVTJaVAUmvQbOKY1Pg5v14yhRQw9dOEKqYy4rpgihCjftLGxTDXGDr+34q4YIWQhA5rdNcEZtEWZ2C1JWCJkCdcpiZlBM55xBEqWLzhbmydtO4BHO/8erGkqr2hr7REWlrjh+i4nRVLog0haiD2HYZcDszxEF8ux3ZB3nucsUugdtk9H8g7hGDkcU30IbWPgwHow2uC4b7aLd3UX/L7orTeBg5/S0Ojo+6DdxGb9A79AFF6BM6Rl/RCRojiiT6iX6h34O/3o534L2+Hh1sdZ4DtHG8t/8AGSwa7Q==</latexit><latexit sha1_base64="tljCRb10HB37ROlEsKureKiWkcU=">AAADZHicfZLfbtMwFMa9hj8jDLYxcYWELDIkLlCVpBfschJccDkkuk0kUeW4p61V24lsp1uV+S24hffiBXgOnCyT1mzMUqRP55zPv8/RyUvOtAnDP1sD79HjJ0+3n/nPd1683N3bf3Wqi0pRGNOCF+o8Jxo4kzA2zHA4LxUQkXM4y5efm/7ZCpRmhfxu1iVkgswlmzFKjCv9OCSTuvqIV/ZwsheEw7A9+K6IOhGg7pxM9gdf0mlBKwHSUE60TqKwNFlNlGGUg/XTSkNJ6JLMIXFSEgE6q9vIFr93lSmeFcp90uC2ettRE6H1WuRuUhCz0P1eU7yvl1RmdpTVTJaVAUmvQbOKY1Pg5v14yhRQw9dOEKqYy4rpgihCjftLGxTDXGDr+34q4YIWQhA5rdNcEZtEWZ2C1JWCJkCdcpiZlBM55xBEqWLzhbmydtO4BHO/8erGkqr2hr7REWlrjh+i4nRVLog0haiD2HYZcDszxEF8ux3ZB3nucsUugdtk9H8g7hGDkcU30IbWPgwHow2uC4b7aLd3UX/L7orTeBg5/S0Ojo+6DdxGb9A79AFF6BM6Rl/RCRojiiT6iX6h34O/3o534L2+Hh1sdZ4DtHG8t/8AGSwa7Q==</latexit><latexit sha1_base64="tljCRb10HB37ROlEsKureKiWkcU=">AAADZHicfZLfbtMwFMa9hj8jDLYxcYWELDIkLlCVpBfschJccDkkuk0kUeW4p61V24lsp1uV+S24hffiBXgOnCyT1mzMUqRP55zPv8/RyUvOtAnDP1sD79HjJ0+3n/nPd1683N3bf3Wqi0pRGNOCF+o8Jxo4kzA2zHA4LxUQkXM4y5efm/7ZCpRmhfxu1iVkgswlmzFKjCv9OCSTuvqIV/ZwsheEw7A9+K6IOhGg7pxM9gdf0mlBKwHSUE60TqKwNFlNlGGUg/XTSkNJ6JLMIXFSEgE6q9vIFr93lSmeFcp90uC2ettRE6H1WuRuUhCz0P1eU7yvl1RmdpTVTJaVAUmvQbOKY1Pg5v14yhRQw9dOEKqYy4rpgihCjftLGxTDXGDr+34q4YIWQhA5rdNcEZtEWZ2C1JWCJkCdcpiZlBM55xBEqWLzhbmydtO4BHO/8erGkqr2hr7REWlrjh+i4nRVLog0haiD2HYZcDszxEF8ux3ZB3nucsUugdtk9H8g7hGDkcU30IbWPgwHow2uC4b7aLd3UX/L7orTeBg5/S0Ojo+6DdxGb9A79AFF6BM6Rl/RCRojiiT6iX6h34O/3o534L2+Hh1sdZ4DtHG8t/8AGSwa7Q==</latexit><latexit sha1_base64="tljCRb10HB37ROlEsKureKiWkcU=">AAADZHicfZLfbtMwFMa9hj8jDLYxcYWELDIkLlCVpBfschJccDkkuk0kUeW4p61V24lsp1uV+S24hffiBXgOnCyT1mzMUqRP55zPv8/RyUvOtAnDP1sD79HjJ0+3n/nPd1683N3bf3Wqi0pRGNOCF+o8Jxo4kzA2zHA4LxUQkXM4y5efm/7ZCpRmhfxu1iVkgswlmzFKjCv9OCSTuvqIV/ZwsheEw7A9+K6IOhGg7pxM9gdf0mlBKwHSUE60TqKwNFlNlGGUg/XTSkNJ6JLMIXFSEgE6q9vIFr93lSmeFcp90uC2ettRE6H1WuRuUhCz0P1eU7yvl1RmdpTVTJaVAUmvQbOKY1Pg5v14yhRQw9dOEKqYy4rpgihCjftLGxTDXGDr+34q4YIWQhA5rdNcEZtEWZ2C1JWCJkCdcpiZlBM55xBEqWLzhbmydtO4BHO/8erGkqr2hr7REWlrjh+i4nRVLog0haiD2HYZcDszxEF8ux3ZB3nucsUugdtk9H8g7hGDkcU30IbWPgwHow2uC4b7aLd3UX/L7orTeBg5/S0Ojo+6DdxGb9A79AFF6BM6Rl/RCRojiiT6iX6h34O/3o534L2+Hh1sdZ4DtHG8t/8AGSwa7Q==</latexit>

<latexit sha1_base64="Itz1Yggp5wSRPairsRuLc+j/gAk=">AAADYHicfZLNbtQwFIXdCT9D+GkHdrCxSJFYjZLMgi4rwYJlkZhppTiqHM/NjBnHiWynMEr9DmzhzdjyJDhpKnXSUkuRju69x99xdLNKcG3C8M/eyHvw8NHj8RP/6bPnL/YPJi8XuqwVgzkrRanOMqpBcAlzw42As0oBLTIBp9nmY9s/vQCleSm/mm0FaUFXkuecUeNKi0NSaX54fhCE07A7+LaIehGg/pycT0afyLJkdQHSMEG1TqKwMmlDleFMgPVJraGibENXkDgpaQE6bbq4Fr9zlSXOS+U+aXBXveloaKH1tsjcZEHNWg97bfGuXlKb/ChtuKxqA5JdgfJaYFPi9u14yRUwI7ZOUKa4y4rZmirKjPtDOxTDXWDr+z6R8J2VRUHlsiGZojaJ0oaA1LWCNkBDBOSGCCpXAoKIKL5am0trd40bMHcbL68tRHU3DI2OyDpzfB8Vk4tqTaUpiyaIbZ8BdzNTHMQ325G9l+cuV/wHCJvM/g/EA2Iws/ga2tK6h+FgtsN1wfAQ7fYuGm7ZbbGIp5HTX+Lg+KjfwDF6g96i9yhCH9Ax+oxO0Bwx9A39RL/Q79Ffb+zte5Or0dFe73mFdo73+h+sChl+</latexit><latexit sha1_base64="Itz1Yggp5wSRPairsRuLc+j/gAk=">AAADYHicfZLNbtQwFIXdCT9D+GkHdrCxSJFYjZLMgi4rwYJlkZhppTiqHM/NjBnHiWynMEr9DmzhzdjyJDhpKnXSUkuRju69x99xdLNKcG3C8M/eyHvw8NHj8RP/6bPnL/YPJi8XuqwVgzkrRanOMqpBcAlzw42As0oBLTIBp9nmY9s/vQCleSm/mm0FaUFXkuecUeNKi0NSaX54fhCE07A7+LaIehGg/pycT0afyLJkdQHSMEG1TqKwMmlDleFMgPVJraGibENXkDgpaQE6bbq4Fr9zlSXOS+U+aXBXveloaKH1tsjcZEHNWg97bfGuXlKb/ChtuKxqA5JdgfJaYFPi9u14yRUwI7ZOUKa4y4rZmirKjPtDOxTDXWDr+z6R8J2VRUHlsiGZojaJ0oaA1LWCNkBDBOSGCCpXAoKIKL5am0trd40bMHcbL68tRHU3DI2OyDpzfB8Vk4tqTaUpiyaIbZ8BdzNTHMQ325G9l+cuV/wHCJvM/g/EA2Iws/ga2tK6h+FgtsN1wfAQ7fYuGm7ZbbGIp5HTX+Lg+KjfwDF6g96i9yhCH9Ax+oxO0Bwx9A39RL/Q79Ffb+zte5Or0dFe73mFdo73+h+sChl+</latexit><latexit sha1_base64="Itz1Yggp5wSRPairsRuLc+j/gAk=">AAADYHicfZLNbtQwFIXdCT9D+GkHdrCxSJFYjZLMgi4rwYJlkZhppTiqHM/NjBnHiWynMEr9DmzhzdjyJDhpKnXSUkuRju69x99xdLNKcG3C8M/eyHvw8NHj8RP/6bPnL/YPJi8XuqwVgzkrRanOMqpBcAlzw42As0oBLTIBp9nmY9s/vQCleSm/mm0FaUFXkuecUeNKi0NSaX54fhCE07A7+LaIehGg/pycT0afyLJkdQHSMEG1TqKwMmlDleFMgPVJraGibENXkDgpaQE6bbq4Fr9zlSXOS+U+aXBXveloaKH1tsjcZEHNWg97bfGuXlKb/ChtuKxqA5JdgfJaYFPi9u14yRUwI7ZOUKa4y4rZmirKjPtDOxTDXWDr+z6R8J2VRUHlsiGZojaJ0oaA1LWCNkBDBOSGCCpXAoKIKL5am0trd40bMHcbL68tRHU3DI2OyDpzfB8Vk4tqTaUpiyaIbZ8BdzNTHMQ325G9l+cuV/wHCJvM/g/EA2Iws/ga2tK6h+FgtsN1wfAQ7fYuGm7ZbbGIp5HTX+Lg+KjfwDF6g96i9yhCH9Ax+oxO0Bwx9A39RL/Q79Ffb+zte5Or0dFe73mFdo73+h+sChl+</latexit><latexit sha1_base64="Itz1Yggp5wSRPairsRuLc+j/gAk=">AAADYHicfZLNbtQwFIXdCT9D+GkHdrCxSJFYjZLMgi4rwYJlkZhppTiqHM/NjBnHiWynMEr9DmzhzdjyJDhpKnXSUkuRju69x99xdLNKcG3C8M/eyHvw8NHj8RP/6bPnL/YPJi8XuqwVgzkrRanOMqpBcAlzw42As0oBLTIBp9nmY9s/vQCleSm/mm0FaUFXkuecUeNKi0NSaX54fhCE07A7+LaIehGg/pycT0afyLJkdQHSMEG1TqKwMmlDleFMgPVJraGibENXkDgpaQE6bbq4Fr9zlSXOS+U+aXBXveloaKH1tsjcZEHNWg97bfGuXlKb/ChtuKxqA5JdgfJaYFPi9u14yRUwI7ZOUKa4y4rZmirKjPtDOxTDXWDr+z6R8J2VRUHlsiGZojaJ0oaA1LWCNkBDBOSGCCpXAoKIKL5am0trd40bMHcbL68tRHU3DI2OyDpzfB8Vk4tqTaUpiyaIbZ8BdzNTHMQ325G9l+cuV/wHCJvM/g/EA2Iws/ga2tK6h+FgtsN1wfAQ7fYuGm7ZbbGIp5HTX+Lg+KjfwDF6g96i9yhCH9Ax+oxO0Bwx9A39RL/Q79Ffb+zte5Or0dFe73mFdo73+h+sChl+</latexit>

⇠<latexit sha1_base64="cwUJ1SLHgHCKRorkH3x6aHqDleg=">AAADX3icfZLBbtQwEIbdDdASoN3CCXGxSJE4rZLsgR4r0QPHIrFtpTiqHO9k11rbiWyndJXmGbiWR+PIm+CkqdRNSy1F+jUzv7/f0WSl4MaG4Z+tkffs+YvtnZf+q9dvdvfG+29PTVFpBjNWiEKfZ9SA4ApmllsB56UGKjMBZ9nqa9s/uwRteKF+2HUJqaQLxXPOqHWl2QG54gcX4yCchN3BD0XUiwD15+Rif3RM5gWrJCjLBDUmicLSpjXVljMBjU8qAyVlK7qAxElFJZi07tI2+JOrzHFeaPcpi7vqfUdNpTFrmblJSe3SDHtt8bFeUtn8MK25KisLit2C8kpgW+D26XjONTAr1k5QprnLitmSasqs+0EbFMtd4Mb3faLgJyukpGpek0zTJonSmoAylYY2QE0E5JYIqhYCgohovlja66bZNK7APm68vrMQ3d0wNDoi68zxU1RMLsslVbaQdRA3fQbczUxwEN9vR82TPHe55lcgmmT6fyAeEINpg++gLa17GA6mG1wXDA/Rbu+i4ZY9FKfxJHL6exwcHfYbuIM+oI/oM4rQF3SEvqETNEMMcfQL3aDfo7/etrfrjW9HR1u95x3aON77f0aZGQk=</latexit><latexit sha1_base64="cwUJ1SLHgHCKRorkH3x6aHqDleg=">AAADX3icfZLBbtQwEIbdDdASoN3CCXGxSJE4rZLsgR4r0QPHIrFtpTiqHO9k11rbiWyndJXmGbiWR+PIm+CkqdRNSy1F+jUzv7/f0WSl4MaG4Z+tkffs+YvtnZf+q9dvdvfG+29PTVFpBjNWiEKfZ9SA4ApmllsB56UGKjMBZ9nqa9s/uwRteKF+2HUJqaQLxXPOqHWl2QG54gcX4yCchN3BD0XUiwD15+Rif3RM5gWrJCjLBDUmicLSpjXVljMBjU8qAyVlK7qAxElFJZi07tI2+JOrzHFeaPcpi7vqfUdNpTFrmblJSe3SDHtt8bFeUtn8MK25KisLit2C8kpgW+D26XjONTAr1k5QprnLitmSasqs+0EbFMtd4Mb3faLgJyukpGpek0zTJonSmoAylYY2QE0E5JYIqhYCgohovlja66bZNK7APm68vrMQ3d0wNDoi68zxU1RMLsslVbaQdRA3fQbczUxwEN9vR82TPHe55lcgmmT6fyAeEINpg++gLa17GA6mG1wXDA/Rbu+i4ZY9FKfxJHL6exwcHfYbuIM+oI/oM4rQF3SEvqETNEMMcfQL3aDfo7/etrfrjW9HR1u95x3aON77f0aZGQk=</latexit><latexit sha1_base64="cwUJ1SLHgHCKRorkH3x6aHqDleg=">AAADX3icfZLBbtQwEIbdDdASoN3CCXGxSJE4rZLsgR4r0QPHIrFtpTiqHO9k11rbiWyndJXmGbiWR+PIm+CkqdRNSy1F+jUzv7/f0WSl4MaG4Z+tkffs+YvtnZf+q9dvdvfG+29PTVFpBjNWiEKfZ9SA4ApmllsB56UGKjMBZ9nqa9s/uwRteKF+2HUJqaQLxXPOqHWl2QG54gcX4yCchN3BD0XUiwD15+Rif3RM5gWrJCjLBDUmicLSpjXVljMBjU8qAyVlK7qAxElFJZi07tI2+JOrzHFeaPcpi7vqfUdNpTFrmblJSe3SDHtt8bFeUtn8MK25KisLit2C8kpgW+D26XjONTAr1k5QprnLitmSasqs+0EbFMtd4Mb3faLgJyukpGpek0zTJonSmoAylYY2QE0E5JYIqhYCgohovlja66bZNK7APm68vrMQ3d0wNDoi68zxU1RMLsslVbaQdRA3fQbczUxwEN9vR82TPHe55lcgmmT6fyAeEINpg++gLa17GA6mG1wXDA/Rbu+i4ZY9FKfxJHL6exwcHfYbuIM+oI/oM4rQF3SEvqETNEMMcfQL3aDfo7/etrfrjW9HR1u95x3aON77f0aZGQk=</latexit><latexit sha1_base64="cwUJ1SLHgHCKRorkH3x6aHqDleg=">AAADX3icfZLBbtQwEIbdDdASoN3CCXGxSJE4rZLsgR4r0QPHIrFtpTiqHO9k11rbiWyndJXmGbiWR+PIm+CkqdRNSy1F+jUzv7/f0WSl4MaG4Z+tkffs+YvtnZf+q9dvdvfG+29PTVFpBjNWiEKfZ9SA4ApmllsB56UGKjMBZ9nqa9s/uwRteKF+2HUJqaQLxXPOqHWl2QG54gcX4yCchN3BD0XUiwD15+Rif3RM5gWrJCjLBDUmicLSpjXVljMBjU8qAyVlK7qAxElFJZi07tI2+JOrzHFeaPcpi7vqfUdNpTFrmblJSe3SDHtt8bFeUtn8MK25KisLit2C8kpgW+D26XjONTAr1k5QprnLitmSasqs+0EbFMtd4Mb3faLgJyukpGpek0zTJonSmoAylYY2QE0E5JYIqhYCgohovlja66bZNK7APm68vrMQ3d0wNDoi68zxU1RMLsslVbaQdRA3fQbczUxwEN9vR82TPHe55lcgmmT6fyAeEINpg++gLa17GA6mG1wXDA/Rbu+i4ZY9FKfxJHL6exwcHfYbuIM+oI/oM4rQF3SEvqETNEMMcfQL3aDfo7/etrfrjW9HR1u95x3aON77f0aZGQk=</latexit>

D<latexit sha1_base64="A8H5dV+coDEJZNoY0MO5pxgmbYE=">AAADZnicfZLfbtMwFMa9hj+jwNhAiAtuLDIkrqomvWCXk9gFl0Oi26Q4mhz3pLVqO5F9MlZlfg1u4bV4Ax4DN+uktRs7UqRP55zPvy+Wi1pJh8Phn61e9Ojxk6fbz/rPX7zcebW79/rEVY0VMBaVquxZwR0oaWCMEhWc1Ra4LhScFvMvy/npBVgnK/MdFzXkmk+NLKXgGFpsnyFcotXtkd8/342Hg2FX9K5IViImqzo+3+sdsUklGg0GheLOZcmwxrzlFqVQ4PuscVBzMedTyII0XIPL2y60px9DZ0LLyobPIO26tx0t184tdBE2NceZ25wtm/fNsgbLg7yVpm4QjLgGlY2iWNHlDdCJtCBQLYLgwsqQlYoZt1xguKc1CsoQ2Pf7fWbgh6i05mbSssJynyV5y8C4xsIyQMsUlMgUN1MFccKsnM7wyvt14xzwfuPVjYXZ7oRNYyCKzpw+RKXsop5xg5Vu49SvMtBuZ0Dj9PY48Q/ywuFWXoLy2ej/QLpBjEee3kCXtO7HaDxa44ZgdBMd3l2y+cruipN0kAT9LY0PD1YvcJu8Jx/IJ5KQz+SQfCXHZEwEqclP8ov87v2NdqK30bvr1d7WyvOGrFVE/wFLwhxO</latexit><latexit sha1_base64="A8H5dV+coDEJZNoY0MO5pxgmbYE=">AAADZnicfZLfbtMwFMa9hj+jwNhAiAtuLDIkrqomvWCXk9gFl0Oi26Q4mhz3pLVqO5F9MlZlfg1u4bV4Ax4DN+uktRs7UqRP55zPvy+Wi1pJh8Phn61e9Ojxk6fbz/rPX7zcebW79/rEVY0VMBaVquxZwR0oaWCMEhWc1Ra4LhScFvMvy/npBVgnK/MdFzXkmk+NLKXgGFpsnyFcotXtkd8/342Hg2FX9K5IViImqzo+3+sdsUklGg0GheLOZcmwxrzlFqVQ4PuscVBzMedTyII0XIPL2y60px9DZ0LLyobPIO26tx0t184tdBE2NceZ25wtm/fNsgbLg7yVpm4QjLgGlY2iWNHlDdCJtCBQLYLgwsqQlYoZt1xguKc1CsoQ2Pf7fWbgh6i05mbSssJynyV5y8C4xsIyQMsUlMgUN1MFccKsnM7wyvt14xzwfuPVjYXZ7oRNYyCKzpw+RKXsop5xg5Vu49SvMtBuZ0Dj9PY48Q/ywuFWXoLy2ej/QLpBjEee3kCXtO7HaDxa44ZgdBMd3l2y+cruipN0kAT9LY0PD1YvcJu8Jx/IJ5KQz+SQfCXHZEwEqclP8ov87v2NdqK30bvr1d7WyvOGrFVE/wFLwhxO</latexit><latexit sha1_base64="A8H5dV+coDEJZNoY0MO5pxgmbYE=">AAADZnicfZLfbtMwFMa9hj+jwNhAiAtuLDIkrqomvWCXk9gFl0Oi26Q4mhz3pLVqO5F9MlZlfg1u4bV4Ax4DN+uktRs7UqRP55zPvy+Wi1pJh8Phn61e9Ojxk6fbz/rPX7zcebW79/rEVY0VMBaVquxZwR0oaWCMEhWc1Ra4LhScFvMvy/npBVgnK/MdFzXkmk+NLKXgGFpsnyFcotXtkd8/342Hg2FX9K5IViImqzo+3+sdsUklGg0GheLOZcmwxrzlFqVQ4PuscVBzMedTyII0XIPL2y60px9DZ0LLyobPIO26tx0t184tdBE2NceZ25wtm/fNsgbLg7yVpm4QjLgGlY2iWNHlDdCJtCBQLYLgwsqQlYoZt1xguKc1CsoQ2Pf7fWbgh6i05mbSssJynyV5y8C4xsIyQMsUlMgUN1MFccKsnM7wyvt14xzwfuPVjYXZ7oRNYyCKzpw+RKXsop5xg5Vu49SvMtBuZ0Dj9PY48Q/ywuFWXoLy2ej/QLpBjEee3kCXtO7HaDxa44ZgdBMd3l2y+cruipN0kAT9LY0PD1YvcJu8Jx/IJ5KQz+SQfCXHZEwEqclP8ov87v2NdqK30bvr1d7WyvOGrFVE/wFLwhxO</latexit><latexit sha1_base64="A8H5dV+coDEJZNoY0MO5pxgmbYE=">AAADZnicfZLfbtMwFMa9hj+jwNhAiAtuLDIkrqomvWCXk9gFl0Oi26Q4mhz3pLVqO5F9MlZlfg1u4bV4Ax4DN+uktRs7UqRP55zPvy+Wi1pJh8Phn61e9Ojxk6fbz/rPX7zcebW79/rEVY0VMBaVquxZwR0oaWCMEhWc1Ra4LhScFvMvy/npBVgnK/MdFzXkmk+NLKXgGFpsnyFcotXtkd8/342Hg2FX9K5IViImqzo+3+sdsUklGg0GheLOZcmwxrzlFqVQ4PuscVBzMedTyII0XIPL2y60px9DZ0LLyobPIO26tx0t184tdBE2NceZ25wtm/fNsgbLg7yVpm4QjLgGlY2iWNHlDdCJtCBQLYLgwsqQlYoZt1xguKc1CsoQ2Pf7fWbgh6i05mbSssJynyV5y8C4xsIyQMsUlMgUN1MFccKsnM7wyvt14xzwfuPVjYXZ7oRNYyCKzpw+RKXsop5xg5Vu49SvMtBuZ0Dj9PY48Q/ywuFWXoLy2ej/QLpBjEee3kCXtO7HaDxa44ZgdBMd3l2y+cruipN0kAT9LY0PD1YvcJu8Jx/IJ5KQz+SQfCXHZEwEqclP8ov87v2NdqK30bvr1d7WyvOGrFVE/wFLwhxO</latexit>

@D@xi

<latexit sha1_base64="fDsvwDFl7MmWox4YfzrEo2MFIuk=">AAADhnicfVJNaxsxEFW8/XDdjzjtsRfRTaH0YLxrSnI0NIceU6iTgGWMVp61hbXaRZpNbTb6D/01vbZ/o/+m2o1DYyfNgODxZt680TBJoaTFfv/PXit49PjJ0/azzvMXL1/tdw9en9m8NAJGIle5uUi4BSU1jFCigovCAM8SBefJ8nOdP78EY2Wuv+G6gEnG51qmUnD01LT78ZClhouKFdyg5IoyhBWarDpx7h+5mkp3OO2G/V6/CXoXRBsQkk2cTg9aJ2yWizIDjUJxa8dRv8BJVTcVClyHlRYKLpZ8DmMPNc/ATqrmU46+98yMprnxTyNt2NuKimfWrrPEV2YcF3Y3V5P35cYlpseTSuqiRNDi2igtFcWc1huiM2lAoFp7wIWRflYqFtwvCf0et1xQ+oFdp9NhGr6LPMu4nlUsMdyNo0nFQNvSQD1AxRSkyBTXcwVhxIycL/DKuW3hEvB+4dWNhJmmw67QO4pGHD/kStllseAa86wKY7eZgTY1PRrGt9ORe9DPNzdyBcqNB/83pDuO4cDRG9ParfkYDQdbvn4wumvt7y7avbK74CzuRR5/jcPh8eYC2+QteUc+kIgckSH5Qk7JiAjyg/wkv8jvoB30gk/B0XVpa2+jeUO2Ihj+BQ5fKKY=</latexit><latexit sha1_base64="fDsvwDFl7MmWox4YfzrEo2MFIuk=">AAADhnicfVJNaxsxEFW8/XDdjzjtsRfRTaH0YLxrSnI0NIceU6iTgGWMVp61hbXaRZpNbTb6D/01vbZ/o/+m2o1DYyfNgODxZt680TBJoaTFfv/PXit49PjJ0/azzvMXL1/tdw9en9m8NAJGIle5uUi4BSU1jFCigovCAM8SBefJ8nOdP78EY2Wuv+G6gEnG51qmUnD01LT78ZClhouKFdyg5IoyhBWarDpx7h+5mkp3OO2G/V6/CXoXRBsQkk2cTg9aJ2yWizIDjUJxa8dRv8BJVTcVClyHlRYKLpZ8DmMPNc/ATqrmU46+98yMprnxTyNt2NuKimfWrrPEV2YcF3Y3V5P35cYlpseTSuqiRNDi2igtFcWc1huiM2lAoFp7wIWRflYqFtwvCf0et1xQ+oFdp9NhGr6LPMu4nlUsMdyNo0nFQNvSQD1AxRSkyBTXcwVhxIycL/DKuW3hEvB+4dWNhJmmw67QO4pGHD/kStllseAa86wKY7eZgTY1PRrGt9ORe9DPNzdyBcqNB/83pDuO4cDRG9ParfkYDQdbvn4wumvt7y7avbK74CzuRR5/jcPh8eYC2+QteUc+kIgckSH5Qk7JiAjyg/wkv8jvoB30gk/B0XVpa2+jeUO2Ihj+BQ5fKKY=</latexit><latexit sha1_base64="fDsvwDFl7MmWox4YfzrEo2MFIuk=">AAADhnicfVJNaxsxEFW8/XDdjzjtsRfRTaH0YLxrSnI0NIceU6iTgGWMVp61hbXaRZpNbTb6D/01vbZ/o/+m2o1DYyfNgODxZt680TBJoaTFfv/PXit49PjJ0/azzvMXL1/tdw9en9m8NAJGIle5uUi4BSU1jFCigovCAM8SBefJ8nOdP78EY2Wuv+G6gEnG51qmUnD01LT78ZClhouKFdyg5IoyhBWarDpx7h+5mkp3OO2G/V6/CXoXRBsQkk2cTg9aJ2yWizIDjUJxa8dRv8BJVTcVClyHlRYKLpZ8DmMPNc/ATqrmU46+98yMprnxTyNt2NuKimfWrrPEV2YcF3Y3V5P35cYlpseTSuqiRNDi2igtFcWc1huiM2lAoFp7wIWRflYqFtwvCf0et1xQ+oFdp9NhGr6LPMu4nlUsMdyNo0nFQNvSQD1AxRSkyBTXcwVhxIycL/DKuW3hEvB+4dWNhJmmw67QO4pGHD/kStllseAa86wKY7eZgTY1PRrGt9ORe9DPNzdyBcqNB/83pDuO4cDRG9ParfkYDQdbvn4wumvt7y7avbK74CzuRR5/jcPh8eYC2+QteUc+kIgckSH5Qk7JiAjyg/wkv8jvoB30gk/B0XVpa2+jeUO2Ihj+BQ5fKKY=</latexit><latexit sha1_base64="fDsvwDFl7MmWox4YfzrEo2MFIuk=">AAADhnicfVJNaxsxEFW8/XDdjzjtsRfRTaH0YLxrSnI0NIceU6iTgGWMVp61hbXaRZpNbTb6D/01vbZ/o/+m2o1DYyfNgODxZt680TBJoaTFfv/PXit49PjJ0/azzvMXL1/tdw9en9m8NAJGIle5uUi4BSU1jFCigovCAM8SBefJ8nOdP78EY2Wuv+G6gEnG51qmUnD01LT78ZClhouKFdyg5IoyhBWarDpx7h+5mkp3OO2G/V6/CXoXRBsQkk2cTg9aJ2yWizIDjUJxa8dRv8BJVTcVClyHlRYKLpZ8DmMPNc/ATqrmU46+98yMprnxTyNt2NuKimfWrrPEV2YcF3Y3V5P35cYlpseTSuqiRNDi2igtFcWc1huiM2lAoFp7wIWRflYqFtwvCf0et1xQ+oFdp9NhGr6LPMu4nlUsMdyNo0nFQNvSQD1AxRSkyBTXcwVhxIycL/DKuW3hEvB+4dWNhJmmw67QO4pGHD/kStllseAa86wKY7eZgTY1PRrGt9ORe9DPNzdyBcqNB/83pDuO4cDRG9ParfkYDQdbvn4wumvt7y7avbK74CzuRR5/jcPh8eYC2+QteUc+kIgckSH5Qk7JiAjyg/wkv8jvoB30gk/B0XVpa2+jeUO2Ihj+BQ5fKKY=</latexit>

Forward Backward(b)

Figure 4: Forward and backward dependency graph for (a)weighted average wirelength and (b) density computation.

discuss the equations in the x direction, as those in the y direction aresimilar. The real implementation will separate the computation for xand y into different GPU streams as they are independent.

Direct implementation of WA wirelength defined in Equation (2)

may result in numerical overflow, so we convert exiγ to e

xi −maxj∈e xjγ ,

and e−xiγ to e

−xi −minj∈e xj

γ in Equation (2), which is an equivalenttransformation. With the notations in Table 1, the gradient of WAwirelength to a pin location can be written as,

∂WLe∂xi

=(1 + xi

γ )b+e −

1γ c+e

(b+e )2· a+i −

(1 − xiγ )b

−e +

1γ c−e

(b−e )2· a−i . (4)

A naive parallelization scheme is to allocate one thread for each net.This scheme has also been discussed in [15], which only demonstratedlimited speedup because the maximum number of threads to allocateis |E | and the workload for each thread is ill-balanced due to theheterogeneity of net degrees.

Noting that the total number of pins |P | is much larger than |E |, weconsider the possibility of pin-level parallelization. The dependencygraph for WA wirelength forward and backward is elaborated in Fig-ure 4(a). The computation can be completed in four steps: 1) computex±; 2) compute and store a±; 3) compute and store b±, c±; 4) com-pute WLe in forward or ∂WLe

∂xiin backward. Algorithm 1 illustrates

the pin-level parallel implementation of WA wirelength forward andbackward functions. We inline all the CUDA kernel functions, whichshould be separate in practice, for brevity. Furthermore, although allthe computation for an array with different ± signs, e.g., x+ and x−,is written together, they are separated into different CUDA streamsin the implementation. In the algorithm, six kernels are needed. Thex± kernel requires atomic maximum and minimum operations, andthe b±, c± kernels require atomic addition. At the end of the forwardfunction, summation reduction is needed to compute the overall wire-length cost, which is provided by the deep learning toolkit.

Although [15] suggests to use sparse matrix-vector multiplicationprovided by CUDA for the b±, c± kernels (line 10-15 in Algorithm 1).we find that the atomic addition is faster, as matrix-vector multipli-cation is a more general task. A detailed comparison will be made inthe experimental results.

3.2 Density Forward and BackwardBackward and forward of density cost is another bulk computation.Figure 4(b) plots the dependency graph for density cost forward andbackward. The computation consists of four steps: 1) density mapcomputation; 2) au,v ; 3)ψ in forward or ξ in backward; 4) D in forwardor ∂D

∂xiin backwrad. Steps 2 and 3 are the bulk computation parts,

since DCT/IDCT is required for potential and field computation.

Page 4: DREAMPlace: Deep Learning Toolkit-Enabled GPU …DREAMPlace: Deep Learning Toolkit-Enabled GPU Acceleration for Modern VLSI Placement Yibo Lin ECE Department, UT Austin yibolin@utexas.edu

Algorithm 1 Wirelength Forward and Backward by PinInput: A set of nets E, a set of pins P , and pin locations x ;Output: Wirelength cost and gradient;1: function Forward(E, P ,x )2: x+ ← −∞, x− ←∞, b± ← 0, c± ← 0;3: for each thread 0 ≤ t < |P | do ▷ x± kernel4: Define e as the net that pin t belongs to;5: x+e

at .←−−− max(x+e ,xt ); ▷ atomic max

6: x−eat .←−−− min(x−e ,xt ); ▷ atomic min

7: for each thread 0 ≤ t < |P | do ▷ a± kernel8: Define e as the net that pin t belongs to;

9: a±t ← e±xt −x±e

γ ;10: for each thread 0 ≤ t < |P | do ▷ b± kernel11: Define e as the net that pin t belongs to;12: b±e

at .←−−− b±e + a

±t ; ▷ atomic add

13: for each thread 0 ≤ t < |P | do ▷ c± kernel14: Define e as the net that pin t belongs to;15: c±e

at .←−−− c±e + xta

±t ; ▷ atomic add

16: for each thread 0 ≤ t < |E | do ▷WLe kernel17: Define e as t th net in E;18: Compute WLe ←

c+eb+e−

c−eb−e

;

19: return reduce(∑e ∈E WLe ), a±, b±, c±;

20: function Backward(E, P ,x , a±, b±, c±)21: for each thread 0 ≤ t < |P | do ▷ ∂WLe

∂xtkernel

22: Define e as the net that pin t belongs to;23: Compute ∂WLe

∂xt;

24: return { ∂WLe∂xi},∀i ∈ P ;

3.2.1 DCT/IDCT for Electric Potential and Field. The electric potentialand field computation in Equation (3) requires fast DCT/IDCT kernelsfor efficient calculation. The standard DCT/IDCT for one-dimensional(1D) length-N sequence x is,

DCT({xn })k =N−1∑n=0

xn cos (πN(n +

12)k ), (5a)

IDCT({xn })k =12x0 +

N−1∑n=1

xn cos (πNn(k +

12)), (5b)

where k = 0, 1, . . . ,N − 1. We further derive IDXST as,

IDXST({xn })k =N−1∑n=0

xn sin (πNn(k +

12)), (6a)

= (−1)k IDCT({xN−n })k , (6b)

where xN = 0. A detailed derivation is omitted due to page limit.Given anM ×M density map ρ, the electric potential and field can becomputed using DCT/IDCT, IDXST routines.

au,v = DCT(DCT(ρ)T )T , (7a)

ψDCT = IDCT(IDCT({au,v })T )T , (7b)

ξXDSCT = IDXST(IDCT({au,vwu

w2u +w2

v})T )T , (7c)

ξYDCST = IDCT(IDXST({au,vwv

w2u +w2

v})T )T , (7d)

where (·)T denotes matrix transposition. The two-dimensional (2D)DCT/IDCT is computed by performing 1D DCT/IDCT to columnsand then rows. We can see all the computations break down to the

Algorithm 2 DCT/IDCT with N -Point FFTInput: An even-length real sequence x ;Output: An even-length transformed real sequence y;1: function DCT(x )2: N ← |x |;3: for each thread 0 ≤ t < N do ▷ Reorder kernel4: if t < N

2 then5: x ′t ← x2t ;6: else7: x ′t ← x2(N−t )−1;8: x ′′ ← RFFT(x ′); ▷ One-sided real FFT kernel9: for each thread 0 ≤ t < N do ▷ e−

jπ t2N kernel

10: if t ≤ N2 then

11: yt ←2Nℜ(x

′′t e−jπ t2N ); ▷ get real part

12: else13: yt ←

2Nℜ(x

′′(N−t )e

−jπ t2N ); ▷ get real part

14: return y;15: function IDCT(x )16: N ← |x |;17: for each thread 0 ≤ t < N

2 + 1 do ▷ Complex kernel18: x ′t ← (xt − jx(N−t ))e

−jπ t2N ; ▷ let xN ← 0

19: x ′′ ← IRFFT(x ′); ▷ One-sided real IFFT kernel;20: for each thread 0 ≤ t < N do ▷ Reverse kernel21: if t mod 2 == 0 then22: yt ←

N4 x′′t2;

23: else24: yt ←

N4 x′′

(N− t+12 );

25: return y;

1D DCT/IDCT kernels with proper transformations. Thus, highlyoptimized DCT/IDCT kernels are critical to the performance.

As the highly optimized fast Fourier transform (FFT) is providedby many deep learning toolkits, we leverage FFT to compute DCT.There are multiple ways to compute DCT using FFT with linear timeadditional processing. For example, TensorFlow adopts the imple-mentation using 2N -point FFT. We choose the N -point FFT imple-mentation [19] and demonstrate better efficiency in the experiments.Algorithm 2 illustrates the FFT implementation with pre-processingand post-processing kernels. Due to the symmetric property of FFT forreal sequences, we utilize one-sided real FFT/IFFT to save almost halfof the sequence. With additional processing kernels like linear-timereordering and multiplication, DCT/IDCT can be computed with anN -point real FFT/IFFT.

3.3 Optimization EngineePlace/RePlAce [4, 6] uses Nesterov’s method as the gradient-descentsolver with a Lipschitz-constant approximation scheme for line search.We implement the same approach in Python leveraging the efficientAPI provided by the deep learning toolkit. The framework is com-patible with other well-known solvers in deep learning toolkits, e.g.,gradient descent with momentum, Adam [20], providing additionalsolver options.

3.4 LegalizationWe also develop legalization as an operator in DREAMPlace. It firstfollows the Tetris-like procedure similar to NTUplace3 [2]. Then it

Page 5: DREAMPlace: Deep Learning Toolkit-Enabled GPU …DREAMPlace: Deep Learning Toolkit-Enabled GPU Acceleration for Modern VLSI Placement Yibo Lin ECE Department, UT Austin yibolin@utexas.edu

performs Abacus row-based legalization [21]. This step copies thecell locations from GPU to CPU and executes legalization purely onCPU because we observe that it only takes several seconds even formillion-size designs with a single CPU thread.

4 EXPERIMENTAL RESULTSThe framework was developed in Python with PyTorch for optimiz-ers and API, and C++/CUDA for low-level operators. The DREAM-Place program runs on a Linux server with 40-core Intel E5-2698 v4 @2.20GHz and 1 NVIDIA Tesla V100 GPU based on Volta architecture.Only a single CPU thread was used by DREAMPlace for launchingwork to the GPU in the experiments. RePlAce [6] runs on a Linuxserver with 24-core 3.0 GHz Intel Xeon machine with 64GB memoryallocated. ISPD 2005 contest benchmarks [22] and large industrial de-signs were adopted. Currently, we only collected experimental resultswith double-precision floating point numbers on GPU, so additionaleasy speedup is possible by switching to single-precision.4.1 Placement AccelerationTable 2 and Table 3 show the HPWL and runtime details on ISPD 2005and industrial benchmarks. With almost the same solution quality(within 0.2% difference on average), DREAMPlace is able to achieve35× and 43× speedup in GP on the two benchmark suites comparedto RePlAce with 40 threads. RePlAce [6] crashed on the 10-million-cell industrial benchmark at the 9th iteration for Nesterov’s opti-mization. The potential reason is that the peak memory usage ofRePlAce exceeded the maximum memory (64GB). Before crashing, ittook 119 seconds for initial placement and on average 9 seconds foreach Nesterov iteration. As this benchmark takes 1000 iterations withDREAMPlace, we made an estimation to the runtime lower bound119+ 1000 ∗ 9 ≈ 9100 seconds. The LG of DREAMPlace is also around10× faster than the NTUplace3 legalizer in the RePlAce flow. As NTU-place3 does the DP for both placers, so the runtime is similar. Thespeedup for the entire placement flow is 5×.

Figure 5 plots the GP runtime comparison between DREAMPlaceandmulti-threaded RePlAce. The speedup of RePlAce saturates quicklyfrom single thread to 40 threads, i.e., maximum around 2.5×. Fig-ure 5(b) and Figure 5(c) present the scaling curves with increasingnumber of cells. DREAMPlace shows both good stability and nearlylinear scalability with problem sizes. Figure 6 draws the runtime break-down of DREAMPlace on a 2-million-cell design bigblue4, whereGP and LG only take 11.7% runtime of the entire flow. The runtimeof GP and LG is even comparable to that of file IO for benchmarkreading and writing. The majority of the runtime (76%) is taken by DP,which still relies on the external placer currently. A previous study[23] has demonstrated around 6× speedup from GPU acceleration forDP over multi-threaded CPU. While DP is not the focus of this paper,there is a potential of 15× speedup for the entire placement by futureincorporation of GPU-accelerated DP, e.g., 2463

43+9+336/6+54 = 15.2 forbigblue4. On the other hand, within each forward and backwardpass of GP, the density-related computation takes longer than wire-length (67.8% v.s. 32.2%). With efficient DCT/IDCT implementation,the electric field computation is no longer the bottleneck for densityforward and backward.4.2 Acceleration of Low-Level OperatorsWe further investigate the efficiency of our implementations of thelow-level operators, e.g., wirelength forward and backward, DCT/IDCTfor density forward and backward. Figure 7 compares three approachesdiscussed in Section 3.1. “Net-by-Net” denotes the net-level paral-lelization; “Sparse” denotes the sparse matrix-vector multiplicationapproach [15]; “Atomic” denotes the pin-level parallelization with

adap

tec1

adap

tec2

adap

tec3

adap

tec4

bigb

lue1

bigb

lue2

bigb

lue3

bigb

lue4

100

102

104

Ru

nti

me

(s)

RePlAce Threads

DREAMPlace

1

V100

10 20 40

(a)

0 5000 10000

#Cells (K)

0

3500

7000

Ru

nti

me

(s)

RePlAce 1 Thread

40 Threads

(b)

0 5000 10000

#Cells (K)

0

150

300

Ru

nti

me

(s)

DREAMPlace V100

(c)

Figure 5: (a) GP runtime for ISPD 2005 benchmarks. (b) GP run-time scaling for RePlAce on ISPD and industrial benchmarks.The points for the 10-million-cell benchmark design6 areomitted due to crash. (c) GP runtime scaling for DREAMPlace.

LG 2.0

DP

76.0

IO

12.2GP

9.7

(a)

WL Forward

WL Backward

Density Map

Electric Field

Density Backward

Density Forward

25.7

19.1

17.6

6.5

31.1

(b)

Figure 6: Runtime breakdown in percentages of DREAMPlace(a) on bigblue4 and (b) one forward and backward pass in GP.

adap

tec1

adap

tec2

adap

tec3

adap

tec4

bigb

lue1

bigb

lue2

bigb

lue3

bigb

lue4

0

100

200

Ru

nti

me

(ms)

Net-By-Net Sparse Atomic

Figure 7: GPU runtime comparison of different implementa-tion of wirelength forward and backward.

512 1024 2048 4096

Map Size (N ×N)

0.1

1

10

100

Ru

nti

me

(ms) DCT-2N DCT-N

(a)

512 1024 2048 4096

Map Size (N ×N)

0.1

1

10

100

Ru

nti

me

(ms) IDCT-2N IDCT-N

(b)

Figure 8: GPU runtime comparison of DCT/IDCT algorithms.

atomic operations in Algorithm 1. The atomic approach achieves 1.9×speedup over the net-by-net one, 1.4× speedup over the sparse one.

Figure 8 compares the 2D DCT/IDCT implementation using 2N -point FFT (“DCT-2N” and “IDCT-2N”) and N -point FFT (“DCT-N” and“IDCT-2N”) [19]. Considering the map sizes in the experiment (from512 × 512 to 4096 × 4096), the N -point implementation is 1.4× faster.

Page 6: DREAMPlace: Deep Learning Toolkit-Enabled GPU …DREAMPlace: Deep Learning Toolkit-Enabled GPU Acceleration for Modern VLSI Placement Yibo Lin ECE Department, UT Austin yibolin@utexas.edu

Table 2: Experimental results on ISPD 2005 benchmarks [22].

Design #cells #nets RePlAce [6] (40 Threads) DREAMPlaceHPWL GP (s) LG (s) DP (s) Total (s) HPWL GP (s) LG (s) DP (s) IO (s) Total (s)

adaptec1 211K 221K 73.26 106 5 24 139 73.30 5 0.5 25 5 37adaptec2 255K 266K 81.87 194 7 30 236 82.19 6 0.5 32 6 47adaptec3 452K 467K 193.20 382 22 53 466 194.12 10 1 58 10 82adaptec4 496K 516K 175.23 434 21 61 524 174.43 11 2 65 11 92bigblue1 278K 284K 89.85 168 3 30 206 89.43 7 0.3 35 6 51bigblue2 558K 577K 138.09 383 22 90 505 136.69 11 9 90 12 125bigblue3 1097K 1123K 304.83 967 46 138 1171 303.99 22 3 145 24 198bigblue4 2177K 2230K 743.73 2037 56 329 2463 743.75 43 9 336 54 446ratio - - 1.002 34.8 10.6 0.9 5.0 1.000 1.0 1.0 1.0 - 1.0

Table 3: Experimental results on industrial benchmarks.

Design #cells #nets RePlAce [6] (40 Threads) DREAMPlaceHPWL GP (s) LG (s) DP (s) Total (s) HPWL GP (s) LG (s) DP (s) IO (s) Total (s)

design1 1345K 1389K 340.42 974 46 155 1216 340.87 24 4 179 34 244design2 1306K 1355K 275.46 1001 44 152 1238 275.76 24 5 180 32 244design3 2265K 2276K 524.35 1767 84 257 2189 522.79 37 14 305 55 414design4 1525K 1528K 455.22 1130 55 187 1424 454.38 26 8 207 37 281design5 1316K 1364K 287.24 955 46 153 1193 288.41 22 4 186 33 248design6 10504K 10747K NA >9100 NA NA NA 2356.88 282 73 1681 276 2323ratio - - 1.000 43.1 9.5 0.9 5.0 1.000 1.0 1.0 1.0 - 1.0

Although the runtime gap is decreasing, the N -point implementationstill shows superior efficiency with typical map sizes for a wide rangeof designs.

5 CONCLUSIONIn this paper, we take a totally new perspective of solving classicalanalytical placement by casting it into a neural network trainingproblem. Leveraging the deep learning toolkit PyTorch, thus withsmall software development effort, we develop a new open-sourceplacement engine, DREAMPlace with GPU acceleration. It achievesover 30× speedup in global placement and legalization without qualitydegradation for academic and industrial benchmarks, compared tothe state-of-the-art RePlAce running on many threads. We exploredifferent implementations of low-level operators for forward andbackward propagation to boost the overall efficiency.

Furthermore, DREAMPlace is highly extensible to incorporate newalgorithms/solvers and new objectives by simply writing high-levelprogramming languages such as Python. We plan to further in-vestigate cell inflation for routability and net weighting for timingoptimization [24–26] as well as GPU-accelerated detailed placement.It can also be extended to leverage multi-GPU platforms for furtherspeedup. As DREAMPlace decouples the high-level algorithmic de-sign and low-level acceleration efforts, it significantly reduces thedevelopment and maintenance overhead. We believe this work shallopen up new directions for revisiting classical EDA problems.

ACKNOWLEDGEThis project is supported in part by Nvidia, Inc.

REFERENCES[1] T. Chan, J. Cong, and K. Sze, “Multilevel generalized force-directed method for circuit

placement,” in Proc. ISPD. ACM, 2005, pp. 185–192.[2] T.-C. Chen, Z.-W. Jiang, T.-C. Hsu, H.-C. Chen, and Y.-W. Chang, “Ntuplace3: An

analytical placer for large-scale mixed-size designs with preplaced blocks and densityconstraints,” IEEE TCAD, vol. 27, no. 7, pp. 1228–1240, 2008.

[3] M.-K. Hsu, Y.-F. Chen, C.-C. Huang, S. Chou, T.-H. Lin, T.-C. Chen, and Y.-W. Chang,“NTUplace4h: A novel routability-driven placement algorithm for hierarchical mixed-size circuit designs,” IEEE TCAD, vol. 33, no. 12, pp. 1914–1927, 2014.

[4] J. Lu, P. Chen, C.-C. Chang, L. Sha, D. J.-H. Huang, C.-C. Teng, and C.-K. Cheng,“ePlace: Electrostatics-based placement using fast fourier transform and nesterov’s

method,” ACM TODAES, vol. 20, no. 2, p. 17, 2015.[5] J. Lu, H. Zhuang, P. Chen, H. Chang, C.-C. Chang, Y.-C. Wong, L. Sha, D. Huang,

Y. Luo, C.-C. Teng et al., “ePlace-MS: Electrostatics-based placement for mixed-sizecircuits,” IEEE TCAD, vol. 34, no. 5, pp. 685–698, 2015.

[6] C.-K. Cheng, A. B. Kahng, I. Kang, and L. Wang, “Replace: Advancing solution qualityand routability validation in global placement,” IEEE TCAD, 2018.

[7] Z. Zhu, J. Chen, Z. Peng, W. Zhu, and Y.-W. Chang, “Generalized augmented la-grangian and its applications to vlsi global placement,” in Proc. DAC. IEEE, 2018,pp. 1–6.

[8] M.-C. Kim, D.-J. Lee, and I. L. Markov, “SimPL: An effective placement algorithm,”IEEE TCAD, vol. 31, no. 1, pp. 50–60, 2012.

[9] X. He, T. Huang, L. Xiao, H. Tian, and E. F. Y. Young, “Ripple: A robust and effectiveroutability-driven placer,” IEEE TCAD, vol. 32, no. 10, pp. 1546–1556, 2013.

[10] T. Lin, C. Chu, J. R. Shinnerl, I. Bustany, and I. Nedelchev, “POLAR: A high perfor-mance mixed-size wirelengh-driven placer with density constraints,” IEEE TCAD,vol. 34, no. 3, pp. 447–459, 2015.

[11] T. Lin, C. Chu, and G. Wu, “Polar 3.0: An ultrafast global placement engine,” inProc. ICCAD. IEEE, 2015, pp. 520–527.

[12] A. Ludwin, V. Betz, and K. Padalia, “High-quality, deterministic parallel placementfor fpgas on commodity hardware,” in Proc. FPGA. ACM, 2008, pp. 14–23.

[13] W. Li, M. Li, J. Wang, and D. Z. Pan, “UTPlaceF 3.0: A parallelization framework formodern FPGA global placement,” in Proc. ICCAD. IEEE, 2017, pp. 922–928.

[14] J. Cong and Y. Zou, “Parallel multi-level analytical global placement on graphicsprocessing units,” in Proc. ICCAD. ACM, 2009, pp. 681–688.

[15] C.-X. Lin and M. D. Wong, “Accelerate analytical placement with gpu: A genericapproach,” in Proc. DATE. IEEE, 2018, pp. 1345–1350.

[16] A. B. Kahng, S. Reda, and Q. Wang, “Architecture and details of a high quality,large-scale analytical placer,” in Proc. ICCAD. IEEE, 2005, pp. 891–898.

[17] I. Goodfellow, Y. Bengio, A. Courville, and Y. Bengio, Deep learning. MIT pressCambridge, 2016, vol. 1.

[18] A. B. Kahng and Q. Wang, “A faster implementation of APlace,” in Proc. ISPD. ACM,2006, pp. 218–220.

[19] J. Makhoul, “A fast cosine transform in one and two dimensions,” IEEE Transactionson Signal Processing, vol. 28, no. 1, pp. 27–34, 1980.

[20] D. Kinga and J. B. Adam, “A method for stochastic optimization,” in Proc. ICLR, 2015.[21] P. Spindler, U. Schlichtmann, and F. M. Johannes, “Abacus: fast legalization of stan-

dard cell circuits with minimal movement,” in Proc. ISPD, 2008, pp. 47–53.[22] G.-J. Nam, C. J. Alpert, P. Villarrubia, B. Winter, and M. Yildiz, “The ispd2005 place-

ment contest and benchmark suite,” in Proc. ISPD. ACM, 2005, pp. 216–220.[23] S. Dhar and D. Z. Pan, “GDP: GPU accelerated detailed placement,” in Proc. HPEC,

Sept 2018.[24] T. F. Chan, K. Sze, J. R. Shinnerl, and M. Xie, “Mpl6: Enhanced multilevel mixed-size

placement with congestion control,” in Modern Circuit Placement. Springer, 2007.[25] A. B. Kahng and Q. Wang, “An analytic placer for mixed-size placement and timing-

driven placement,” in Proc. ICCAD. IEEE, 2004, pp. 565–572.[26] W. C. Naylor, R. Donelly, and L. Sha, “Non-linear optimization system and method

for wire length and delay optimization for an automatic electric circuit placer,” Oct. 92001, US Patent 6,301,693.