1. 1. AWS Lambda @ 2015.08.01 Yuta Imai Solutions Architect, Amazon Data Services Japan
2. 2. 2 ( ) Solutions Architect
3. 3. 3 AWS AWS Lambda AWS Lambda
4. 4. 4
5. 5. 5 () 11 1.US EAST (Virginia) 2.US WEST (N. California) 3.US WEST 2 (Oregon) 4.EU WEST (Ireland) 5.JAPAN (Tokyo) 6.South America (Sao Paulo) 7.ASP 1 (Singapore) 8.ASP 2 (Sydney) 9.GovCloud 10.BJS 1 (Beijing China) limited preview 11.EU (Frankfurt) 28(AZ) 50
6. 6. 6 AWS 2014516 24 48 61 82 159 280 516 2008 2009 2010 2011 2012 2013 2014
7. 7. 7 TECHNICAL & BUSINESS SUPPORT Account Management Support Professional Services Solutions Architects Training & Certification Security & Pricing Reports Partner Ecosystem AWS MARKETPLACE Backup Big Data & HPC Business Apps Databases Development Industry Solutions Security MANAGEMENT TOOLS Queuing Notifications Search Orchestration Email ENTERPRISE APPS Virtual Desktops Storage Gateway Sharing & Collaboration Email & Calendaring Directories HYBRID CLOUD MANAGEMENT Backups Deployment Direct Connect Identity Federation Integrated Management SECURITY & MANAGEMENT Virtual Private Networks Identity & Access Encryption Keys Configuration Monitoring Dedicated INFRASTRUCTURE SERVICES Regions Availability Zones Compute Storage Objects, Blocks, Files Databases SQL, NoSQL, Caching CDNNetworking PLATFORM SERVICES APP Mobile & Web Front-end Functions Identity Data Store Real-time DEVELOPMENT Containers Source Code Build Tools Deployment DevOps MOBILE Sync Identity Push Notifications Mobile Analytics Mobile Backend ANALYTICS Data Warehousing Hadoop Streaming Data Pipelines Machine Learning
8. 8. 8 AWS Lambda Run your code in the cloud.
9. 9. 9 AWS Lambda
10. 10. 10
11. 11. 11 Bring your own code Node.js, Java / /tmpread/write
12. 12. 12 100
13. 13.
14. 14. 14 S3 AWS LambdaAmazon S3 Bucket 1 2 3
15. 15. 15 DynamoDB AWS Lambda Amazon DynamoDB Table and Stream
16. 16.
17. 17. Web S3 Amazon S3 Consumer Game DB Game Servers Kinesis Amazon Redshift Amazon Elastic MapReduce DSV JSON
18. 18. If..Then.. S3 Amazon S3 Consumer Game DB Game Servers Kinesis Amazon Redshift Amazon Elastic MapReduce DSV JSON
19. 19.
20. 20. Worker While true Supervisord
21. 21. If..Then.. with AWS Lambda! S3 Amazon S3 Consumer Game DB Game Servers Kinesis Amazon Redshift Amazon Elastic MapReduce DSV JSON If..Then.. Lambda ( )
22. 22. 22 If..Then.. with AWS Lambda! With your code!
23. 23. AWS
24. 24. 24 DataAnswers Collect Process Analyze Store
25. 25. 25 Collect Process Analyze Store Data Collection and Storage Data Processing Event Processing Data Analysis
26. 26. 26 S3KinesisDynamoDBAWS Lambda KCL AppsEMR EMR Redshi> Machine Learning Collect Process Analyze Store Data Collection and Storage Data Processing Event Processing Data Analysis
27. 27. 27 S3KinesisDynamoDBAWS Lambda KCL AppsEMR EMR Redshi> Machine Learning Collect Process Analyze Store Data Collection and Storage Data Processing Event Processing Data Analysis AWS
28. 28. 28 AWS Lambda as a pipeline glue Collect Process Analyze Store Data Collection and Storage Data Processing Event Processing Data Analysis
29. 29. 29 : media, log les (sets of records) : records (eg: device stats) : DBRead/Write
30. 30. 30 S3 Kinesis DynamoDB
31. 31. S3 Event Notications Kinesis stream DynamoDB Streams AWS Lambda
32. 32. S3 Event Notications Kinesis stream DynamoDB Streams AWS Lambda Consumer Consumer
33. 33. AWS Lambda
34. 34. 34 Expedia Pipeline http://blogs.aws.amazon.com/bigdata/post/Tx1R28PXR3NAO1I/How-Expedia- Implemented-Near-Real-time-Analysis-of-Interdependent-Datasets
35. 35. 35 TempTracker AWS
36. 36. 36 raspberry pi
37. 37. 37 Python (boto) DynamoDBKinesis App Kinesis Lambda SNS TempTracker: IoT
38. 38. 38
39. 39. 39 Scalable & Responsive Big Data Interface with AWS Lambda at FireEye http://blogs.aws.amazon.com/bigdata/post/Tx3KH6BEUL2SGVA/Building-Scalable-and-Responsive-Big-Data-Interfaces- with-AWS-Lambda Lambda+S3 Web UI Lambda function
40. 40. 40 API Gateway Slack Lambda ECS Lambda S3 Aurora Outgoing Webhook - cluster name - # of tasks - commands RunTasks - cluster name - # of tasks - commands as environment variables - output location Output STDOUT as file Spin up containers and run tasks Incoming Webhook - Read file from S3 and emit it to Slack Sysbench Slack+AWS API Gateway+AWS Lambda+Amazon ECS
41. 41.
42. 42. 42 AWS Lambda Amazon EMRAmazon ECS HadoopSpark
43. 43. 43 AWS Lambda enables... AWS Lambda If Then Amazon DynamoDB Amazon Kinesis Amazon S3 Amazon EMR Amazon Redshift Amazon Machine Learning Amazon API Gateway Amazon ECS With your code!