Flyby: Improved Dense Matrix Multiplication-(Tom Vacek, Thomson Reuters)
-
Upload
spark-summit -
Category
Data & Analytics
-
view
350 -
download
2
Transcript of Flyby: Improved Dense Matrix Multiplication-(Tom Vacek, Thomson Reuters)
![Page 1: Flyby: Improved Dense Matrix Multiplication-(Tom Vacek, Thomson Reuters)](https://reader030.fdocuments.net/reader030/viewer/2022032506/55ccd9dbbb61ebb65f8b48c0/html5/thumbnails/1.jpg)
Flyby: Improved Dense Matrix Multiplication (& other tasks)
Tom Vacek Thomson Reuters R&D
![Page 2: Flyby: Improved Dense Matrix Multiplication-(Tom Vacek, Thomson Reuters)](https://reader030.fdocuments.net/reader030/viewer/2022032506/55ccd9dbbb61ebb65f8b48c0/html5/thumbnails/2.jpg)
Outline • Matching use case • 3 Algorithms • Evaluation
![Page 3: Flyby: Improved Dense Matrix Multiplication-(Tom Vacek, Thomson Reuters)](https://reader030.fdocuments.net/reader030/viewer/2022032506/55ccd9dbbb61ebb65f8b48c0/html5/thumbnails/3.jpg)
Matching application
users cartesian movies map…combineByKey
![Page 4: Flyby: Improved Dense Matrix Multiplication-(Tom Vacek, Thomson Reuters)](https://reader030.fdocuments.net/reader030/viewer/2022032506/55ccd9dbbb61ebb65f8b48c0/html5/thumbnails/4.jpg)
cartesian matrix multiply aRows cartesian(bCols) map (a , b) => a*b
![Page 5: Flyby: Improved Dense Matrix Multiplication-(Tom Vacek, Thomson Reuters)](https://reader030.fdocuments.net/reader030/viewer/2022032506/55ccd9dbbb61ebb65f8b48c0/html5/thumbnails/5.jpg)
shuffleMultiply • Local reduceByKey • Optimal efficiency:
O(n1.5) workers each do O(n1.5) work O(n1.25) wire
i,j
(i,1),j (i,n),j
j,k
(1,k),j (m,k),j
flatmap
join
(i,k),1 (i,k),n
i,k
map ._1 reduceByKey +
![Page 6: Flyby: Improved Dense Matrix Multiplication-(Tom Vacek, Thomson Reuters)](https://reader030.fdocuments.net/reader030/viewer/2022032506/55ccd9dbbb61ebb65f8b48c0/html5/thumbnails/6.jpg)
left flyby right
![Page 7: Flyby: Improved Dense Matrix Multiplication-(Tom Vacek, Thomson Reuters)](https://reader030.fdocuments.net/reader030/viewer/2022032506/55ccd9dbbb61ebb65f8b48c0/html5/thumbnails/7.jpg)
Flyby matrix multiply
![Page 8: Flyby: Improved Dense Matrix Multiplication-(Tom Vacek, Thomson Reuters)](https://reader030.fdocuments.net/reader030/viewer/2022032506/55ccd9dbbb61ebb65f8b48c0/html5/thumbnails/8.jpg)
Flyby matrix multiply
![Page 9: Flyby: Improved Dense Matrix Multiplication-(Tom Vacek, Thomson Reuters)](https://reader030.fdocuments.net/reader030/viewer/2022032506/55ccd9dbbb61ebb65f8b48c0/html5/thumbnails/9.jpg)
Flyby implementation (hack) def flyby[L,R,O] (left: RDD[L], right:RDD[R], fold0: (L,R)=>O, foldFun: (L,R,O)=>O))={ var reduction:RDD[O] = null val leftLocal = getPartitionsAsLocalArray(left) for(part <- leftLocal) { val flyer = right.sparkContext broadcast (part) val nextReduction = if (reduction == null) { right.map { case rr => val init = fold0(flyer.value.head, rr) flyer.value.tail.foldLeft(init) { case (acc, nextflyer) =>
foldFun(nextflyer, rr, acc)} } } else { right.zip(reduction).map { case (rr, red) => flyer.value.foldLeft(red) { case (acc, nextflyer) =>
foldFun(nextflyer, rr, acc)} } }.persist nextReduction.count flyer.unpersist (false) reduction.unpersist(false) reduction = nextReduction
![Page 10: Flyby: Improved Dense Matrix Multiplication-(Tom Vacek, Thomson Reuters)](https://reader030.fdocuments.net/reader030/viewer/2022032506/55ccd9dbbb61ebb65f8b48c0/html5/thumbnails/10.jpg)
Matrix Experiments • Implemented dist dense matrix class using
Breeze • 5 Nodes, quad socket Xeon E5-2690 (32
cores), 250GB. Cloudera 5.3 (Spark 1.2.0) • Evaluate n×n matrices, npartitions • Code: bitbucket.org/twvacek/spark-flyby
![Page 11: Flyby: Improved Dense Matrix Multiplication-(Tom Vacek, Thomson Reuters)](https://reader030.fdocuments.net/reader030/viewer/2022032506/55ccd9dbbb61ebb65f8b48c0/html5/thumbnails/11.jpg)
MLLib • BlockMatrix introduced to mllib at version
1.3.0 (13 Mar 2015) • Not available at time this work was done. • Our matrix class very similar. • Mllib multiply ~= shuffleMultiply
![Page 12: Flyby: Improved Dense Matrix Multiplication-(Tom Vacek, Thomson Reuters)](https://reader030.fdocuments.net/reader030/viewer/2022032506/55ccd9dbbb61ebb65f8b48c0/html5/thumbnails/12.jpg)
Results
0"
500"
1000"
1500"
2000"
2500"
3000"
3500"
4000"
10k (36)" 25k (100)" 30k (144)"
time
(s)"
n (npartitions)"
cart" shuf" flyby"
![Page 13: Flyby: Improved Dense Matrix Multiplication-(Tom Vacek, Thomson Reuters)](https://reader030.fdocuments.net/reader030/viewer/2022032506/55ccd9dbbb61ebb65f8b48c0/html5/thumbnails/13.jpg)
Discussion • Flyby: signif improvement over cartesian.
Gain for matching applications. • Improvement over shuffle probably from
memory caching. Asymptotics? • FlybyRDD ? • Code: bitbucket.org/twvacek/spark-flyby