Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling...
Transcript of Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling...
![Page 1: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/1.jpg)
Scaling Magento for the Global Market
Kian Gould Founder & CEO
![Page 2: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/2.jpg)
![Page 3: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/3.jpg)
Angry Birds
![Page 4: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/4.jpg)
Angry Birds
![Page 5: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/5.jpg)
Angry Birds
![Page 6: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/6.jpg)
Angry Birds
![Page 7: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/7.jpg)
DRM Server
Mail Service
Giftcards
Braintree
Warehouse Inventory
3 Storefronts (US, EU, ASIA)
Custom Reporting
Splitshipping
Warehouse / Inventory
Management Fullfillment
Banner Management
Custom Tax, Custom Shipping
Catalog Rules
…
The Angry Birds Magento Universe
![Page 8: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/8.jpg)
High Level Requirements
• Innovative, highly-scalable e-commerce systems
• High availability
• Unusually high peaks (20x and more)
• High Speed
• Backend server independent from frontend servers
• No down-times, ever.
![Page 9: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/9.jpg)
Solution components
![Page 10: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/10.jpg)
Cloud Services
RightScale Services
Amazon AWS Console
Cloud- Front
Elasti-Cache
S3 EC2 RDS ELB Route 53
Management
Deployments
Auto-Scaling
Monitoring
Logging
Scripting API
![Page 11: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/11.jpg)
Magento in the Cloud
![Page 12: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/12.jpg)
Share nothing?
Code
Cache Session
Storage
Assets Product images,
CMS images,
Scaled images
Database
JS/CSS
Bundles
Server
![Page 13: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/13.jpg)
S3 Assets Storage
Varnish Array EC2
Frontend Array
CloudFront CDN for Skin
RDS MySQL
S3 Deployment
ElastiCache Cache Backend
ELB Load Balancer
CloudFront CDN for Assets
Monitoring
Backend Array EC2 EC2
EC2
Worker Array EC2
Braintree
Mail DRM
Giftcards
Logging
EC2
Route 53 DNS-Service
shop.angrybirds.com
![Page 14: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/14.jpg)
S3 Deployment
Route 53 DNS-Service
shop.angrybirds.com
RDS MySQL
Varnish Array EC2
Frontend Array Backend Array EC2 EC2
EC2
Worker Array EC2 EC2
Deployment X
ELB Load Balancer
![Page 15: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/15.jpg)
Deployment X+1 Release Y
S3 Deployment
Route 53 DNS-Service
shop.angrybirds.com
Varnish Array EC2
Frontend Array
ELB Load Balancer
Backend Array EC2 EC2
EC2
Worker Array EC2 EC2
Deployment X
Varnish Array EC2
Frontend Array Backend Array EC2 Worker Array EC2
ELB Load Balancer
EC2
EC2 EC2 EC2
Cache
warming
Direct Access
through
Local Host entry
RDS MySQL
![Page 16: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/16.jpg)
Deployment X+1 Release Y
Varnish Array EC2
Frontend Array
ELB Load Balancer
Backend Array EC2 EC2
EC2
Worker Array EC2 EC2
Deployment X
Varnish Array EC2
Frontend Array Backend Array EC2 Worker Array EC2
ELB Load Balancer
EC2
EC2 EC2 EC2
Deployment / Rollback
Route 53 DNS-Service
shop.angrybirds.com
![Page 17: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/17.jpg)
Downtime during the deployment:
![Page 18: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/18.jpg)
All Basket Content, active Logins and Sessions are carried through each
Deployment / Rollback
![Page 19: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/19.jpg)
Magento, faster!
![Page 20: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/20.jpg)
Magento Frontend Caching
Areas of optimization
![Page 21: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/21.jpg)
Frontend-Optimization
JavaScript / CSS
• Bundling (Magento)
• Minifying (with YUI Compressor)
• Gzip-Compression (Apache)
• Timestamp in filename
• Send Cache-Header
CDN (Cloudfront)
• Product images (+ scaled versions), CMS Images etc.,...
• Skins
• JS/CSS
Aoe_JsCssTstamp
![Page 22: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/22.jpg)
Magento-Optimizations
Reduce Logging as much as possible
Deactivate non-cacheable features wherever possible
Caching
• Asynchronous Cache Clearing Queue
• Delete old Cache-entries
Cookie Storage for Username and Basket content
Ajax-Basket
and... Profiling, Profiling, Profiling,… (xdebug + kcachegrind, Newrelic, xhprof)
Aoe_AsyncCache
Aoe_CacheCleaner
![Page 23: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/23.jpg)
95% of all hits are cacheable!
5% dynamic content
Homepage
Categories
Products
CMS
Basket
Checkout
Profile
Caching
![Page 24: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/24.jpg)
Varnish Reverse Proxy
Magento
Caching
![Page 25: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/25.jpg)
Solutions
Aoe_Static
Caching +
Handling of
dynamic Content
Magneto_Varnish
Purging
![Page 26: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/26.jpg)
Lessons learned
• 404s are very expensive => cache or redirect
• Cloudfront with S3 Backend doesn’t support gzip => deploy time bundeling/compression
• Don’t run reports on live database!
• You can’t plan for everything => but you can be ready to solve anything quickly. How?
=> continuous deployment
![Page 27: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/27.jpg)
Continuous Integration und Deployment
![Page 28: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/28.jpg)
Don’t be afraid of deploying to Live!
![Page 29: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/29.jpg)
Magento Installationpackage
Database
tables
Files*
‣htdocs [Magento Source]
‣.modman [Custom Packages]
Installer
Settings
‣URLs
‣DB-Parameters
‣…
Files
‣media
ChangeLog
System-
storage SVN
GIT
Settings
![Page 30: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/30.jpg)
Performance Tests
Acceptance Tests
Unit-Tests
PHPUnit
Installation on
„Latest“ System
Build
Deployment Pipeline
![Page 31: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/31.jpg)
Cloud Deployment
Production
Integration Tests
AOE + Rovio
Cloud Deployment
Staging Copy to S3
Deployment Pipeline
Installation on
Deployment System
![Page 32: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/32.jpg)
Deployment of new Release
Space game Launch
Array scales up
USA wakes up
Launch!
![Page 33: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/33.jpg)
In real life…
![Page 34: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/34.jpg)
visitors/day
![Page 35: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/35.jpg)
(real) concurrent users!
![Page 36: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/36.jpg)
orders per second at peak!
![Page 37: Scaling Magento for the Global Marketinfo2.magento.com/rs/magentoenterprise/images/Magento...Scaling Magento for the Global Market Kian Gould Founder & CEO Angry Birds Angry Birds](https://reader034.fdocuments.net/reader034/viewer/2022042320/5f0a7c0d7e708231d42bdcfc/html5/thumbnails/37.jpg)
Thank You!
www.aoemedia.com [email protected]
AOE media Inc. 251 Rhode Island Street San Francisco, CA 94103 USA Tel.: +1 415 230-0690 Fax: +1 415 230-0699
@aoemedia