Big data prototyping in AWS cloud
-
Upload
samuel-yee -
Category
Technology
-
view
170 -
download
0
Transcript of Big data prototyping in AWS cloud
![Page 1: Big data prototyping in AWS cloud](https://reader035.fdocuments.net/reader035/viewer/2022070602/5878522f1a28ab68198b680f/html5/thumbnails/1.jpg)
Big Data Prototyping in AWS CloudSamuel YeeAWS Certified Solutions Architect
![Page 2: Big data prototyping in AWS cloud](https://reader035.fdocuments.net/reader035/viewer/2022070602/5878522f1a28ab68198b680f/html5/thumbnails/2.jpg)
Minimum Viable Product (MVP)
![Page 3: Big data prototyping in AWS cloud](https://reader035.fdocuments.net/reader035/viewer/2022070602/5878522f1a28ab68198b680f/html5/thumbnails/3.jpg)
Why Cloud Computing?• Rapid prototyping for Minimum Viable Product (MVP)
• No need to worry about what-to-buy, what-if-buy-wrong etc.• Scale up and out if demands pick up• Tear down at minimal cost if no demand or project failure
• Deploy anytime and everywhere• Worldwide infrastructure• Flexible, elastic, agile
• No high upfront costs• Pay-as-you-use model• Can start instantly without much cost justifications
![Page 4: Big data prototyping in AWS cloud](https://reader035.fdocuments.net/reader035/viewer/2022070602/5878522f1a28ab68198b680f/html5/thumbnails/4.jpg)
Typical Big Data Use Cases
RetrospectiveAnalysis and reporting
Here-and-NowReal-time processing
and dashboards
PredictionsMachine learning and
smart applications
![Page 5: Big data prototyping in AWS cloud](https://reader035.fdocuments.net/reader035/viewer/2022070602/5878522f1a28ab68198b680f/html5/thumbnails/5.jpg)
Typical Big Data Pipeline
Ingest
Process
Store
Analyse (BI)
Data Answer
![Page 6: Big data prototyping in AWS cloud](https://reader035.fdocuments.net/reader035/viewer/2022070602/5878522f1a28ab68198b680f/html5/thumbnails/6.jpg)
Word Count Example
Ingest
Process
Store
Analyse (BI)
Data Answer
![Page 7: Big data prototyping in AWS cloud](https://reader035.fdocuments.net/reader035/viewer/2022070602/5878522f1a28ab68198b680f/html5/thumbnails/7.jpg)
Which one to buy?
![Page 8: Big data prototyping in AWS cloud](https://reader035.fdocuments.net/reader035/viewer/2022070602/5878522f1a28ab68198b680f/html5/thumbnails/8.jpg)
Demo• Spin your new Hadoop cluster in AWS within 15 minutes• Terminate your Hadoop cluster instantly when you don’t need it
![Page 9: Big data prototyping in AWS cloud](https://reader035.fdocuments.net/reader035/viewer/2022070602/5878522f1a28ab68198b680f/html5/thumbnails/9.jpg)
Data Types Ingest Process Store Analyse (BI)
Flat Files
Database Data
StreamingData
Hadoop-as-a-Service
File Objects
Database Objects
NoSQL Data Store
Desktop Apps
Enterprise Apps
Open Source
![Page 10: Big data prototyping in AWS cloud](https://reader035.fdocuments.net/reader035/viewer/2022070602/5878522f1a28ab68198b680f/html5/thumbnails/10.jpg)
What’s Next?• Measure the outcomes of your data projects
• No need – Change yourself or educate your users/bosses• Fail – Try to understand the needs of your users/bosses better. Deliver faster!• Success – Congratulation!
• Should I continue in AWS cloud?• Do I need it 24x7, anywhere?• Can I move my system and network admins to host my Big Data apps?• Do I have the resource, time & expertise to deal with the intricate details of
Hadoop cluster configurations and troubleshooting?• How do I secure my big data infrastructure?
• If answer is still no, go back to the Big Data Landscape chart. But at least you can keep your users/bosses happy and engaged with your MVP in cloud.