BDT303 Data Science with Elastic MapReduce - AWS re: Invent 2012
A Modern Framework for Amazon Elastic MapReduce (BDT309) | AWS re:Invent 2013
-
Upload
amazon-web-services -
Category
Technology
-
view
821 -
download
3
description
Transcript of A Modern Framework for Amazon Elastic MapReduce (BDT309) | AWS re:Invent 2013
![Page 1: A Modern Framework for Amazon Elastic MapReduce (BDT309) | AWS re:Invent 2013](https://reader033.fdocuments.net/reader033/viewer/2022051818/54b3a7f14a79599c5c8b477a/html5/thumbnails/1.jpg)
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
Mortar Data: K Young (CEO), Jeremy Karn (Lead Engineer)
November 15, 2013
A Modern Framework for EMR
Friday, November 15, 13
![Page 2: A Modern Framework for Amazon Elastic MapReduce (BDT309) | AWS re:Invent 2013](https://reader033.fdocuments.net/reader033/viewer/2022051818/54b3a7f14a79599c5c8b477a/html5/thumbnails/2.jpg)
K Young Jeremy Karn
Friday, November 15, 13
![Page 3: A Modern Framework for Amazon Elastic MapReduce (BDT309) | AWS re:Invent 2013](https://reader033.fdocuments.net/reader033/viewer/2022051818/54b3a7f14a79599c5c8b477a/html5/thumbnails/3.jpg)
This talk is technical
Friday, November 15, 13
![Page 4: A Modern Framework for Amazon Elastic MapReduce (BDT309) | AWS re:Invent 2013](https://reader033.fdocuments.net/reader033/viewer/2022051818/54b3a7f14a79599c5c8b477a/html5/thumbnails/4.jpg)
Great: HadoopGreater: Amazon EMR
Friday, November 15, 13
![Page 5: A Modern Framework for Amazon Elastic MapReduce (BDT309) | AWS re:Invent 2013](https://reader033.fdocuments.net/reader033/viewer/2022051818/54b3a7f14a79599c5c8b477a/html5/thumbnails/5.jpg)
A Modern Framework
Friday, November 15, 13
![Page 6: A Modern Framework for Amazon Elastic MapReduce (BDT309) | AWS re:Invent 2013](https://reader033.fdocuments.net/reader033/viewer/2022051818/54b3a7f14a79599c5c8b477a/html5/thumbnails/6.jpg)
A Modern Framework
Friday, November 15, 13
![Page 7: A Modern Framework for Amazon Elastic MapReduce (BDT309) | AWS re:Invent 2013](https://reader033.fdocuments.net/reader033/viewer/2022051818/54b3a7f14a79599c5c8b477a/html5/thumbnails/7.jpg)
Goal: 10x productivity• Collaboration• Efficient, Free Development• Testing / Debug• Reproducibility• And More...
Friday, November 15, 13
![Page 8: A Modern Framework for Amazon Elastic MapReduce (BDT309) | AWS re:Invent 2013](https://reader033.fdocuments.net/reader033/viewer/2022051818/54b3a7f14a79599c5c8b477a/html5/thumbnails/8.jpg)
For data science(not a database)
Friday, November 15, 13
![Page 9: A Modern Framework for Amazon Elastic MapReduce (BDT309) | AWS re:Invent 2013](https://reader033.fdocuments.net/reader033/viewer/2022051818/54b3a7f14a79599c5c8b477a/html5/thumbnails/9.jpg)
Iterate Locally
DeployCloud Execution EMR + Mortar
Friday, November 15, 13
![Page 10: A Modern Framework for Amazon Elastic MapReduce (BDT309) | AWS re:Invent 2013](https://reader033.fdocuments.net/reader033/viewer/2022051818/54b3a7f14a79599c5c8b477a/html5/thumbnails/10.jpg)
Demo
Find the most popularprojects on GitHub
Friday, November 15, 13
![Page 11: A Modern Framework for Amazon Elastic MapReduce (BDT309) | AWS re:Invent 2013](https://reader033.fdocuments.net/reader033/viewer/2022051818/54b3a7f14a79599c5c8b477a/html5/thumbnails/11.jpg)
Goal: Collaboration• Easily run code from others• Contribute changes back
Friday, November 15, 13
![Page 12: A Modern Framework for Amazon Elastic MapReduce (BDT309) | AWS re:Invent 2013](https://reader033.fdocuments.net/reader033/viewer/2022051818/54b3a7f14a79599c5c8b477a/html5/thumbnails/12.jpg)
Goal: Efficient, Free Development• Rapid iteration• No cost• Resilient to errors
Friday, November 15, 13
![Page 13: A Modern Framework for Amazon Elastic MapReduce (BDT309) | AWS re:Invent 2013](https://reader033.fdocuments.net/reader033/viewer/2022051818/54b3a7f14a79599c5c8b477a/html5/thumbnails/13.jpg)
Goal: Testing & Debug• Automated testing• See what is happening at runtime
Friday, November 15, 13
![Page 14: A Modern Framework for Amazon Elastic MapReduce (BDT309) | AWS re:Invent 2013](https://reader033.fdocuments.net/reader033/viewer/2022051818/54b3a7f14a79599c5c8b477a/html5/thumbnails/14.jpg)
Goal: Reproducibility• Know what was run• 1-button deploy• Rollback
Friday, November 15, 13
![Page 15: A Modern Framework for Amazon Elastic MapReduce (BDT309) | AWS re:Invent 2013](https://reader033.fdocuments.net/reader033/viewer/2022051818/54b3a7f14a79599c5c8b477a/html5/thumbnails/15.jpg)
Goal: Miscellaneous goals• Easy scheduling• Easily use other technologies: Python, Amazon
DynamoDB, MongoDB• Results easy to locate• Managed cluster lifetime• More granular API
Friday, November 15, 13
![Page 16: A Modern Framework for Amazon Elastic MapReduce (BDT309) | AWS re:Invent 2013](https://reader033.fdocuments.net/reader033/viewer/2022051818/54b3a7f14a79599c5c8b477a/html5/thumbnails/16.jpg)
A Modern Framework for Amazon EMR
• Collaboration• Efficient, Free Development• Testing / Debug• Reproducibility• And More...
What you saw
Friday, November 15, 13
![Page 17: A Modern Framework for Amazon Elastic MapReduce (BDT309) | AWS re:Invent 2013](https://reader033.fdocuments.net/reader033/viewer/2022051818/54b3a7f14a79599c5c8b477a/html5/thumbnails/17.jpg)
Next steps• bit.ly/mortar-reinvent• Documentation: help.mortardata.com• Follow: @mortardata
Friday, November 15, 13
![Page 18: A Modern Framework for Amazon Elastic MapReduce (BDT309) | AWS re:Invent 2013](https://reader033.fdocuments.net/reader033/viewer/2022051818/54b3a7f14a79599c5c8b477a/html5/thumbnails/18.jpg)
Please give us your feedback on this presentation
As a thank you, we will select prize winners daily for completed surveys!
BDT309 Thank You
Friday, November 15, 13