Computational Finance with Map-Reduce in Scala
Transcript of Computational Finance with Map-Reduce in Scala
![Page 1: Computational Finance with Map-Reduce in Scala](https://reader035.fdocuments.net/reader035/viewer/2022062705/5565757ad8b42a7b518b519e/html5/thumbnails/1.jpg)
Computational Finance with Map-Reduce in Scala
Jianfeng Zhang
![Page 2: Computational Finance with Map-Reduce in Scala](https://reader035.fdocuments.net/reader035/viewer/2022062705/5565757ad8b42a7b518b519e/html5/thumbnails/2.jpg)
![Page 3: Computational Finance with Map-Reduce in Scala](https://reader035.fdocuments.net/reader035/viewer/2022062705/5565757ad8b42a7b518b519e/html5/thumbnails/3.jpg)
Abstract
This paper presets results of computational finance experiments using map-reduce in Scala. They observe superlinear speedup, super-efficiency, and evidence for a high degree of compute and I/O overlap in the median runtimes using “naïve,” memory-bound, fine-grain, and course-grain parallel algorithms on three different hardware platforms.
![Page 4: Computational Finance with Map-Reduce in Scala](https://reader035.fdocuments.net/reader035/viewer/2022062705/5565757ad8b42a7b518b519e/html5/thumbnails/4.jpg)
Computational finance is a multidisciplinary field at the crossroads of mathematical finance and computer science. The emphasis is on development and utilization of numerically intensive methods for pricing, risk analysis, forecasting, automated trading, and other applications.
![Page 5: Computational Finance with Map-Reduce in Scala](https://reader035.fdocuments.net/reader035/viewer/2022062705/5565757ad8b42a7b518b519e/html5/thumbnails/5.jpg)
http://www.youtube.com/watch?v=pADuSbjzj0k&feature=youtu.be
![Page 6: Computational Finance with Map-Reduce in Scala](https://reader035.fdocuments.net/reader035/viewer/2022062705/5565757ad8b42a7b518b519e/html5/thumbnails/6.jpg)
Map-reduce is a framework generally to speed-up data analysis using distributed computing. While map-reduce has been applied to different problem domains, many of a data-intensive nature, almost no attention has been given to opportunities for computational finance as a mixture of floating point and data-intensive operations.
![Page 7: Computational Finance with Map-Reduce in Scala](https://reader035.fdocuments.net/reader035/viewer/2022062705/5565757ad8b42a7b518b519e/html5/thumbnails/7.jpg)
![Page 8: Computational Finance with Map-Reduce in Scala](https://reader035.fdocuments.net/reader035/viewer/2022062705/5565757ad8b42a7b518b519e/html5/thumbnails/8.jpg)
Scala is a modern, high-level Java Virtual Machine (JVM) language that blends object-oriented and functional programming styles with actors, a shared nothing model of concurrent computation inspired by physics theories. Proponents have argued that Scala language features are suited to solving large-scale computing tasks on inexpensive, commodity multicore and multiprocessor platforms in an expressive manner that avoids the concurrency hazards and runtime inefficiencies of shared, mutable state programs. Indeed, the function-oriented style of Scala would seem to lend itself precisely to coding mathematical expressions which characterize quantitative operations.
![Page 9: Computational Finance with Map-Reduce in Scala](https://reader035.fdocuments.net/reader035/viewer/2022062705/5565757ad8b42a7b518b519e/html5/thumbnails/9.jpg)
http://ontwik.com/scala-a-scalable-language-by-martin-odersky/
![Page 10: Computational Finance with Map-Reduce in Scala](https://reader035.fdocuments.net/reader035/viewer/2022062705/5565757ad8b42a7b518b519e/html5/thumbnails/10.jpg)
Related work
• The literature shows enduring interest in speeding up computational finance algorithms.
• The literature furthermore indicates map-reduce is a widely accepted approach to speeding up computation for various problem classes.
![Page 11: Computational Finance with Map-Reduce in Scala](https://reader035.fdocuments.net/reader035/viewer/2022062705/5565757ad8b42a7b518b519e/html5/thumbnails/11.jpg)
Method
• Bond pricing theory • Bond generation algorithm • IO design • Pricing algorithms • Serial algorithms• Parallel naïve algorithm• Parallel coarse-grain algorithm• Parallel fine-grain algorithm
![Page 12: Computational Finance with Map-Reduce in Scala](https://reader035.fdocuments.net/reader035/viewer/2022062705/5565757ad8b42a7b518b519e/html5/thumbnails/12.jpg)
Experimental design
• Environment• Trials • Speed-up calculations
![Page 13: Computational Finance with Map-Reduce in Scala](https://reader035.fdocuments.net/reader035/viewer/2022062705/5565757ad8b42a7b518b519e/html5/thumbnails/13.jpg)
Environment
![Page 14: Computational Finance with Map-Reduce in Scala](https://reader035.fdocuments.net/reader035/viewer/2022062705/5565757ad8b42a7b518b519e/html5/thumbnails/14.jpg)
Speed-up calculations
![Page 15: Computational Finance with Map-Reduce in Scala](https://reader035.fdocuments.net/reader035/viewer/2022062705/5565757ad8b42a7b518b519e/html5/thumbnails/15.jpg)
Results
• Parallel naïve results • Parallel fine-grain results • Parallel coarse-grain results
![Page 16: Computational Finance with Map-Reduce in Scala](https://reader035.fdocuments.net/reader035/viewer/2022062705/5565757ad8b42a7b518b519e/html5/thumbnails/16.jpg)
Parallel naïve results
![Page 17: Computational Finance with Map-Reduce in Scala](https://reader035.fdocuments.net/reader035/viewer/2022062705/5565757ad8b42a7b518b519e/html5/thumbnails/17.jpg)
Parallel naïve results
![Page 18: Computational Finance with Map-Reduce in Scala](https://reader035.fdocuments.net/reader035/viewer/2022062705/5565757ad8b42a7b518b519e/html5/thumbnails/18.jpg)
Parallel naïve results
![Page 19: Computational Finance with Map-Reduce in Scala](https://reader035.fdocuments.net/reader035/viewer/2022062705/5565757ad8b42a7b518b519e/html5/thumbnails/19.jpg)
Parallel fine-grain results
![Page 20: Computational Finance with Map-Reduce in Scala](https://reader035.fdocuments.net/reader035/viewer/2022062705/5565757ad8b42a7b518b519e/html5/thumbnails/20.jpg)
Parallel coarse-grain results
![Page 21: Computational Finance with Map-Reduce in Scala](https://reader035.fdocuments.net/reader035/viewer/2022062705/5565757ad8b42a7b518b519e/html5/thumbnails/21.jpg)
• The naïve algorithm appears to be the best performing overall end-to-end, achieving super-linearity and superefficiency for levels of u, depending on the processor type. For instance, the more modern processors, the W3540 and i5, realize super-linearity and superefficiency for u as small as 64.
• I/O is broadly sub-linear which, by itself, is not surprising. However, I/O does not appear to be a processing bottleneck since the difference between compute and memory-bound compute plus memorybound I/O over the range of u appears to be insignificant.
![Page 22: Computational Finance with Map-Reduce in Scala](https://reader035.fdocuments.net/reader035/viewer/2022062705/5565757ad8b42a7b518b519e/html5/thumbnails/22.jpg)
Conclusion
• They would like to explore changes to H-S to support multiprocessor parallelism.
• there are open questions on how to “shard” or parallelize the data.
• we had briefly mentioned Scala’s parallel collections.