Scaling Systems: Architectures that Grow
-
Upload
gibraltar-software -
Category
Technology
-
view
1.020 -
download
0
description
Transcript of Scaling Systems: Architectures that Grow
![Page 1: Scaling Systems: Architectures that Grow](https://reader033.fdocuments.net/reader033/viewer/2022060111/5565e430d8b42a20638b4793/html5/thumbnails/1.jpg)
Scaling Systems: Architectures that Grow
Fundamental Patterns for scaling you can implement incrementally
![Page 2: Scaling Systems: Architectures that Grow](https://reader033.fdocuments.net/reader033/viewer/2022060111/5565e430d8b42a20638b4793/html5/thumbnails/2.jpg)
Who Am I?
• Kendall Miller• One of the Founders of Gibraltar Software– Small Independent Software Vendor Founded in 2008– Developers of VistaDB and Gibraltar– Engineers, not Sales People
• Enterprise Systems Architect & Developer since 1995• BSE in Computer Engineering, University of Illinois
Urbana-Champaign (UIUC)• Twitter: @KendallMiller
![Page 3: Scaling Systems: Architectures that Grow](https://reader033.fdocuments.net/reader033/viewer/2022060111/5565e430d8b42a20638b4793/html5/thumbnails/3.jpg)
Fair Warning
![Page 4: Scaling Systems: Architectures that Grow](https://reader033.fdocuments.net/reader033/viewer/2022060111/5565e430d8b42a20638b4793/html5/thumbnails/4.jpg)
What is Scale?
Scaling is the ability to cope and perform under an increasing workload.
![Page 5: Scaling Systems: Architectures that Grow](https://reader033.fdocuments.net/reader033/viewer/2022060111/5565e430d8b42a20638b4793/html5/thumbnails/5.jpg)
What is Scale?
Scaling to a load = available sustaining that load
![Page 6: Scaling Systems: Architectures that Grow](https://reader033.fdocuments.net/reader033/viewer/2022060111/5565e430d8b42a20638b4793/html5/thumbnails/6.jpg)
What is Scale?
Being available is really about a request being
completed in a period of time.
![Page 7: Scaling Systems: Architectures that Grow](https://reader033.fdocuments.net/reader033/viewer/2022060111/5565e430d8b42a20638b4793/html5/thumbnails/7.jpg)
0.00E+00 1.00E+07 2.00E+07 3.00E+07 4.00E+07 5.00E+07 6.00E+07 7.00E+07
Microsoft.com
Twitter.com
Amazon.com
Target.com
Slashdot.org
DevExpress.com
Hanselman.com
Gibraltar Software
What’s your Target?
Average daily traffic in Visitors / Day
![Page 8: Scaling Systems: Architectures that Grow](https://reader033.fdocuments.net/reader033/viewer/2022060111/5565e430d8b42a20638b4793/html5/thumbnails/8.jpg)
1.00E+03 1.00E+04 1.00E+05 1.00E+06 1.00E+07 1.00E+08
Microsoft.com
Twitter.com
Amazon.com
Target.com
Slashdot.org
DevExpress.com
Hanselman.com
Gibraltar Software
What’s your Target?
Average daily traffic in Visitors / Day
![Page 9: Scaling Systems: Architectures that Grow](https://reader033.fdocuments.net/reader033/viewer/2022060111/5565e430d8b42a20638b4793/html5/thumbnails/9.jpg)
What’s your Target?
25,000 Visitors/Day = 125,000 Pages/Day
11 High Traffic Hours/Day = 12,000 Pages/Hour
12,000 Pages/Hour = 3.3 Pages/Second
![Page 10: Scaling Systems: Architectures that Grow](https://reader033.fdocuments.net/reader033/viewer/2022060111/5565e430d8b42a20638b4793/html5/thumbnails/10.jpg)
Specific Architectures
• Gossip • Map Reduce• Tree of Responsibility• Stream Processing• Scalable Storage• Publish/Subscribe• Distributed Queues
• Load Balancers + Shared Nothing Units
• Load Balancers + Stateless Nodes + Scalable Storage
• Content Addressable Networks
• General Peer to Peer
![Page 11: Scaling Systems: Architectures that Grow](https://reader033.fdocuments.net/reader033/viewer/2022060111/5565e430d8b42a20638b4793/html5/thumbnails/11.jpg)
ACD/C
• Async – Do the work whenever• Caching – Don’t do any work you don’t have
to• Distribution – Get as many people to do the
work as you can• Consistency – We all agree on these key things
![Page 12: Scaling Systems: Architectures that Grow](https://reader033.fdocuments.net/reader033/viewer/2022060111/5565e430d8b42a20638b4793/html5/thumbnails/12.jpg)
Async
• Decouple operations so you do the minimum amount of work in performance critical paths
• Queue work that can be completed later to smooth out load
• Speculative Execution• Scheduled Requests (Nightly processes)
![Page 13: Scaling Systems: Architectures that Grow](https://reader033.fdocuments.net/reader033/viewer/2022060111/5565e430d8b42a20638b4793/html5/thumbnails/13.jpg)
Caching
• Save results of earlier work nearby where they are handy to use again later
• Apply in front of anything that’s time consuming
• Easiest to apply from the left to the right• Simple strategies can be really effective (EF
Dump all on update)
![Page 14: Scaling Systems: Architectures that Grow](https://reader033.fdocuments.net/reader033/viewer/2022060111/5565e430d8b42a20638b4793/html5/thumbnails/14.jpg)
Why Caching?
• Loading the world is impractical• Apps ask a lot of repeating questions.– Stateless applications even more so
• Answers don’t change often• Authoritative information is expensive
![Page 15: Scaling Systems: Architectures that Grow](https://reader033.fdocuments.net/reader033/viewer/2022060111/5565e430d8b42a20638b4793/html5/thumbnails/15.jpg)
Distribution
• Distribute requests across multiple systems• Classic web “Scale Out” approach• The less state held, the easier to distribute
work. – Distributed database = hard– Distributed static content server = easy
• Request routing for distribution can serve other availability purposes
![Page 16: Scaling Systems: Architectures that Grow](https://reader033.fdocuments.net/reader033/viewer/2022060111/5565e430d8b42a20638b4793/html5/thumbnails/16.jpg)
Consistency
• The degree to which all parties observe the same state of the system at the same time
• Scaling inevitably requires compromise– Forces one source of the truth for absolute
consistency and requires extensive locking to ensure parties agree
– The real world doesn’t require the consistency we tend to demand of our systems
![Page 17: Scaling Systems: Architectures that Grow](https://reader033.fdocuments.net/reader033/viewer/2022060111/5565e430d8b42a20638b4793/html5/thumbnails/17.jpg)
Consistency Challenges
• Singleton Data Structures (Order numbers..)• State held between the endpoints of a process• Consistent results of queries across
partitioned datasets
![Page 18: Scaling Systems: Architectures that Grow](https://reader033.fdocuments.net/reader033/viewer/2022060111/5565e430d8b42a20638b4793/html5/thumbnails/18.jpg)
Typical Application
Client (Web
Browser)
Server(Web
Server)Storage
(Database)
Session StateSSL Session
Log ContentionMemory Allocation/GC
Network SocketsRequest Queue
Transaction IsolationReader/Writer Locks
Singleton Data Structures
![Page 19: Scaling Systems: Architectures that Grow](https://reader033.fdocuments.net/reader033/viewer/2022060111/5565e430d8b42a20638b4793/html5/thumbnails/19.jpg)
Caching
Client (Web
Browser)
Server(Web
Server)Storage
(Database)
Browser Cache
Output Cache
Content Cache
Query Cache
100% 50% 10% 1%
![Page 20: Scaling Systems: Architectures that Grow](https://reader033.fdocuments.net/reader033/viewer/2022060111/5565e430d8b42a20638b4793/html5/thumbnails/20.jpg)
Client (Web
Browser)
Distribution
Server(Web
Server)
Storage(Database)
Client (Web
Browser)Client (Web
Browser)Client (Web
Browser)
Server(Web
Server)
Reverse Proxy
Session State and Identity need to be factored outPartition (Sticky Session)
First, then stateless nodes
![Page 21: Scaling Systems: Architectures that Grow](https://reader033.fdocuments.net/reader033/viewer/2022060111/5565e430d8b42a20638b4793/html5/thumbnails/21.jpg)
Server(Web
Server)Client (Web
Browser)
Partitioned Storage Zones
Server(Web
Server)
Storage(Database)Client
(Web Browser)
Client (Web
Browser)Client (Web
Browser)
Server(Web
Server)
Customer A Server(Web
Server)Storage
(Database)
Customer B
![Page 22: Scaling Systems: Architectures that Grow](https://reader033.fdocuments.net/reader033/viewer/2022060111/5565e430d8b42a20638b4793/html5/thumbnails/22.jpg)
Server(Web
Server)
Client (Web
Browser)
Partitioned Storage Intra-Zone
OrdersClient (Web
Browser)Client (Web
Browser)Client (Web
Browser)
Server(Web
Server)
Customer A
Products
Customer B
Server(Web
Server)Server(Web
Server)
Inventory
![Page 23: Scaling Systems: Architectures that Grow](https://reader033.fdocuments.net/reader033/viewer/2022060111/5565e430d8b42a20638b4793/html5/thumbnails/23.jpg)
Server(Web
Server)
Asynchronous Processing
OrdersServer(Web
Server)
Products
Server(Web
Server)Server(Web
Server)
Inventory
Order Queue
Order Processing
Server
![Page 24: Scaling Systems: Architectures that Grow](https://reader033.fdocuments.net/reader033/viewer/2022060111/5565e430d8b42a20638b4793/html5/thumbnails/24.jpg)
Fallacies of Distributed Computing
• The network is reliable• Latency is zero• Bandwidth is infinite• The network is secure• Topology doesn’t change• There is one administrator• Transport cost is zero• The network is homogeneous
![Page 25: Scaling Systems: Architectures that Grow](https://reader033.fdocuments.net/reader033/viewer/2022060111/5565e430d8b42a20638b4793/html5/thumbnails/25.jpg)
Client (Web
Browser)
Fresh Problems: Partial Failures
Server(Web
Server)
Storage(Database)
Client (Web
Browser)Client (Web
Browser)Client (Web
Browser)
Server(Web
Server)
![Page 26: Scaling Systems: Architectures that Grow](https://reader033.fdocuments.net/reader033/viewer/2022060111/5565e430d8b42a20638b4793/html5/thumbnails/26.jpg)
Fresh Problems: Partial Failures
1. Break system into individual failure zones2. Monitor each instance of each zone for
problems3. Route around bad instances
![Page 27: Scaling Systems: Architectures that Grow](https://reader033.fdocuments.net/reader033/viewer/2022060111/5565e430d8b42a20638b4793/html5/thumbnails/27.jpg)
Without monitoring, redundancy is worthless
![Page 28: Scaling Systems: Architectures that Grow](https://reader033.fdocuments.net/reader033/viewer/2022060111/5565e430d8b42a20638b4793/html5/thumbnails/28.jpg)
Server(Web
Server)Client (Web
Browser)
Fresh Problems: Upgrades
Server(Web
Server)
Storage(Database)Client
(Web Browser)
Client (Web
Browser)Client (Web
Browser)
Server(Web
Server)
Customer A Server(Web
Server)Storage
(Database)
Customer B
![Page 29: Scaling Systems: Architectures that Grow](https://reader033.fdocuments.net/reader033/viewer/2022060111/5565e430d8b42a20638b4793/html5/thumbnails/29.jpg)
Fresh Problems: Upgrades
1. Break system into individual upgrade zones2. Upgrade each zone – Drain & Stop, Upgrade,
Verify.3. Cut traffic over to updated zones
![Page 30: Scaling Systems: Architectures that Grow](https://reader033.fdocuments.net/reader033/viewer/2022060111/5565e430d8b42a20638b4793/html5/thumbnails/30.jpg)
Design for Software Update From the Start
• Don’t forget Data Schemas
![Page 31: Scaling Systems: Architectures that Grow](https://reader033.fdocuments.net/reader033/viewer/2022060111/5565e430d8b42a20638b4793/html5/thumbnails/31.jpg)
Bringing Home the Bacon
TestingTestingTesting
![Page 32: Scaling Systems: Architectures that Grow](https://reader033.fdocuments.net/reader033/viewer/2022060111/5565e430d8b42a20638b4793/html5/thumbnails/32.jpg)
Critical Lessons Learned
• ACD/C
• Clear Consistency Strategy
• Build in monitoring and management
![Page 33: Scaling Systems: Architectures that Grow](https://reader033.fdocuments.net/reader033/viewer/2022060111/5565e430d8b42a20638b4793/html5/thumbnails/33.jpg)
Additional Information:
Websites– www.GibraltarSoftware.com– www.eSymmetrix.com
Follow Up– [email protected]– Twitter: kendallmiller