Evolution of surface micro-structure and moisture sorption ...
Concurrency at Scale: Evolution to Micro-Services
-
Upload
randy-shoup -
Category
Software
-
view
4.249 -
download
0
Transcript of Concurrency at Scale: Evolution to Micro-Services
![Page 1: Concurrency at Scale: Evolution to Micro-Services](https://reader031.fdocuments.net/reader031/viewer/2022032514/55d57829bb61ebb32f8b4597/html5/thumbnails/1.jpg)
Concurrency at Scale: The Evolution to Micro-
Services
Randy Shoup @randyshoup
linkedin.com/in/randyshoup
![Page 2: Concurrency at Scale: Evolution to Micro-Services](https://reader031.fdocuments.net/reader031/viewer/2022032514/55d57829bb61ebb32f8b4597/html5/thumbnails/2.jpg)
Background• CTO at KIXEYE
o Real-time strategy games for web and mobile
• Director of Engineering for Google App Engineo World’s largest Platform-as-a-Serviceo Part of Google Cloud Platform
• Chief Engineer at eBayo Multiple generations of eBay’s real-time search infrastructure
![Page 3: Concurrency at Scale: Evolution to Micro-Services](https://reader031.fdocuments.net/reader031/viewer/2022032514/55d57829bb61ebb32f8b4597/html5/thumbnails/3.jpg)
Evolution in Action• eBay
• 5th generation today• Monolithic Perl Monolithic C++ Java microservices
• Twitter• 3rd generation today• Monolithic Rails JS / Rails / Scala microservices
• Amazon• Nth generation today• Monolithic C++ Perl / C++ Java / Scala microservices
![Page 4: Concurrency at Scale: Evolution to Micro-Services](https://reader031.fdocuments.net/reader031/viewer/2022032514/55d57829bb61ebb32f8b4597/html5/thumbnails/4.jpg)
Evolution to Micro-Services
• The Monolith
• Micro-Services
• Reactive Systems
• Migrating to Micro-Services
![Page 5: Concurrency at Scale: Evolution to Micro-Services](https://reader031.fdocuments.net/reader031/viewer/2022032514/55d57829bb61ebb32f8b4597/html5/thumbnails/5.jpg)
Evolution to Micro-Services
• The Monolith
• Micro-Services
• Reactive Systems
• Migrating to Micro-Services
![Page 6: Concurrency at Scale: Evolution to Micro-Services](https://reader031.fdocuments.net/reader031/viewer/2022032514/55d57829bb61ebb32f8b4597/html5/thumbnails/6.jpg)
The Monolithic Architecture
2-3 monolithic tiers
• {JS, iOS, Android}
• {PHP, Ruby, Python}
• {MySQL, Mongo}
Presentation
Application
Database
![Page 7: Concurrency at Scale: Evolution to Micro-Services](https://reader031.fdocuments.net/reader031/viewer/2022032514/55d57829bb61ebb32f8b4597/html5/thumbnails/7.jpg)
The Monolithic Application
Simple at first
In-process latencies
Single codebase, deploy unit
Resource-efficient at small scale
Pros
Coordination overhead as team grows
Poor enforcement of modularity
Poor scaling (vertical only)
All-or-nothing deploy (downtime, failures)
Long build times
Cons
![Page 8: Concurrency at Scale: Evolution to Micro-Services](https://reader031.fdocuments.net/reader031/viewer/2022032514/55d57829bb61ebb32f8b4597/html5/thumbnails/8.jpg)
The Monolithic Database
Simple at first
Join queries are easy
Single schema, deployment
Resource-efficient at small scale
Pros
Coupling over time
Poor scaling and redundancy (all-or-nothing, vertical only)
Difficult to tune properly
All-or-nothing schema management
Cons
![Page 9: Concurrency at Scale: Evolution to Micro-Services](https://reader031.fdocuments.net/reader031/viewer/2022032514/55d57829bb61ebb32f8b4597/html5/thumbnails/9.jpg)
“If you don’t end up regretting your early technology decisions, you probably over-engineered”
-- me
![Page 10: Concurrency at Scale: Evolution to Micro-Services](https://reader031.fdocuments.net/reader031/viewer/2022032514/55d57829bb61ebb32f8b4597/html5/thumbnails/10.jpg)
Evolution to Micro-Services
• The Monolith
• Micro-Services
• Reactive Systems
• Migrating to Micro-Services
![Page 11: Concurrency at Scale: Evolution to Micro-Services](https://reader031.fdocuments.net/reader031/viewer/2022032514/55d57829bb61ebb32f8b4597/html5/thumbnails/11.jpg)
Micro-Services
“Loosely-coupled service oriented architecture with bounded contexts”
-- Adrian Cockcroft
![Page 12: Concurrency at Scale: Evolution to Micro-Services](https://reader031.fdocuments.net/reader031/viewer/2022032514/55d57829bb61ebb32f8b4597/html5/thumbnails/12.jpg)
Micro-Services
“Loosely-coupled service oriented architecture with bounded contexts”
-- Adrian Cockcroft
![Page 13: Concurrency at Scale: Evolution to Micro-Services](https://reader031.fdocuments.net/reader031/viewer/2022032514/55d57829bb61ebb32f8b4597/html5/thumbnails/13.jpg)
Micro-Services
“Loosely-coupled service oriented architecture with bounded contexts”
-- Adrian Cockcroft
![Page 14: Concurrency at Scale: Evolution to Micro-Services](https://reader031.fdocuments.net/reader031/viewer/2022032514/55d57829bb61ebb32f8b4597/html5/thumbnails/14.jpg)
Micro-Services
“Loosely-coupled service oriented architecture with bounded contexts”
-- Adrian Cockcroft
![Page 15: Concurrency at Scale: Evolution to Micro-Services](https://reader031.fdocuments.net/reader031/viewer/2022032514/55d57829bb61ebb32f8b4597/html5/thumbnails/15.jpg)
Micro-Services
• Single-purpose• Simple, well-defined interface• Modular and independent• More graph of relationships than tiers• Fullest expression of encapsulation and
modularity• Isolated persistence (!) A
C D E
B
![Page 16: Concurrency at Scale: Evolution to Micro-Services](https://reader031.fdocuments.net/reader031/viewer/2022032514/55d57829bb61ebb32f8b4597/html5/thumbnails/16.jpg)
Micro-Services
Each unit is simple
Independent scaling and performance
Independent testing and deployment
Can optimally tune performance (caching, replication, etc.)
Pros
Many cooperating units
Many small repos
Requires more sophisticated tooling and dependency management
Network latencies
Cons
![Page 17: Concurrency at Scale: Evolution to Micro-Services](https://reader031.fdocuments.net/reader031/viewer/2022032514/55d57829bb61ebb32f8b4597/html5/thumbnails/17.jpg)
Google Services• All engineering groups organized into “services”
• Gmail, App Engine, Bigtable, etc.
• Self-sufficient and autonomous
• Layered on one another
Very small teams achieve great things
![Page 18: Concurrency at Scale: Evolution to Micro-Services](https://reader031.fdocuments.net/reader031/viewer/2022032514/55d57829bb61ebb32f8b4597/html5/thumbnails/18.jpg)
Google Cloud Datastore
• Cloud Datastore: NoSQL serviceo Highly scalable and resiliento Strong transactional consistencyo SQL-like rich query capabilities
• Megastore: geo-scale structured databaseo Multi-row transactionso Synchronous cross-datacenter replication
• Bigtable: cluster-level structured storageo (row, column, timestamp) -> cell contents
• Colossus: next-generation clustered file systemo Block distribution and replication
• Cluster management infrastructureo Task scheduling, machine assignment
Cloud Datastor
e
Megastore
Bigtable
Colossus
Cluster manager
![Page 19: Concurrency at Scale: Evolution to Micro-Services](https://reader031.fdocuments.net/reader031/viewer/2022032514/55d57829bb61ebb32f8b4597/html5/thumbnails/19.jpg)
Evolution to Micro-Services
• The Monolith
• Micro-Services
• Reactive Systems
• Migrating to Micro-Services
![Page 20: Concurrency at Scale: Evolution to Micro-Services](https://reader031.fdocuments.net/reader031/viewer/2022032514/55d57829bb61ebb32f8b4597/html5/thumbnails/20.jpg)
ReactiveMicro-Services
• Responsiveo Predictable performance at 99%ile trumps low mean latency (!)o Tail latencies far more important than mean or mediano Client protects itself with asynchronous, non-blocking calls
• Resiliento Redundancy for machine / cluster / data center failureso Load-balancing and flow control for service invocationso Client protects itself with standard failure management patterns:
timeouts, retries, circuit breakers
![Page 21: Concurrency at Scale: Evolution to Micro-Services](https://reader031.fdocuments.net/reader031/viewer/2022032514/55d57829bb61ebb32f8b4597/html5/thumbnails/21.jpg)
ReactiveMicro-Services
• Elastico Scale up and down service instances according to loado Gracefully handle spiky workloadso Predictive and reactive scaling
• Message-Driveno Asynchronous message-passing over synchronous request-responseo Often custom protocol over TCP / UDP or WebSockets over HTTP
![Page 22: Concurrency at Scale: Evolution to Micro-Services](https://reader031.fdocuments.net/reader031/viewer/2022032514/55d57829bb61ebb32f8b4597/html5/thumbnails/22.jpg)
KIXEYEGame Services
• Minimize request latencyo Respond as rapidly as possible to client
• Functional Reactive + Actor modelo Highly asynchronous, never block (!)o Queue events / messages for complex worko Heavy use of Scala / Akka and RxJava at KIXEYE
• Highly Scalable and Productiveo (-) eBay uses threaded synchronous modelo (-) Google uses complicated callback-based asynchronous model
![Page 23: Concurrency at Scale: Evolution to Micro-Services](https://reader031.fdocuments.net/reader031/viewer/2022032514/55d57829bb61ebb32f8b4597/html5/thumbnails/23.jpg)
KIXEYE Service Chassis• Goal: Make it easy to build and deploy micro-
services
• Chassis core• Configuration integration• Registration and Discovery• Monitoring and Metrics• Load-balancing for downstream services• Failure management for downstream services
• Development / Deployment Pipeline• Transport layer over REST / JSON or WebSockets• Service template in Maven• Build pipeline through Puppet -> Packer -> AMI• Red-black deployment via Asgard
![Page 24: Concurrency at Scale: Evolution to Micro-Services](https://reader031.fdocuments.net/reader031/viewer/2022032514/55d57829bb61ebb32f8b4597/html5/thumbnails/24.jpg)
KIXEYE Service Chassis• Heavy use of NetflixOSS
• Asgard• Hystrix• Ribbon + WebSockets Janus• Eureka
Results• 15 minutes from no code to running service in AWS (!)• Open-sourced at https://github.com/Kixeye
![Page 25: Concurrency at Scale: Evolution to Micro-Services](https://reader031.fdocuments.net/reader031/viewer/2022032514/55d57829bb61ebb32f8b4597/html5/thumbnails/25.jpg)
Evolution to Micro-Services
• The Monolith
• Micro-Services
• Reactive Systems
• Migrating to Micro-Services
![Page 26: Concurrency at Scale: Evolution to Micro-Services](https://reader031.fdocuments.net/reader031/viewer/2022032514/55d57829bb61ebb32f8b4597/html5/thumbnails/26.jpg)
Migrating Incrementally
• Find your worst scaling bottleneck
• Wall it off behind an interface
• Replace it
• Rinse and Repeat
![Page 27: Concurrency at Scale: Evolution to Micro-Services](https://reader031.fdocuments.net/reader031/viewer/2022032514/55d57829bb61ebb32f8b4597/html5/thumbnails/27.jpg)
Building Micro-Services
• Common Chassis / Frameworko Make it trivially easy to build and maintain a service
• Define Service Interface (Formally!)o Proposeo Discuss with client(s)o Agree
• Prototype Implementationo Simplest thing that could possibly worko Client can integrate with prototypeo Implementor can learn what works and what does not
![Page 28: Concurrency at Scale: Evolution to Micro-Services](https://reader031.fdocuments.net/reader031/viewer/2022032514/55d57829bb61ebb32f8b4597/html5/thumbnails/28.jpg)
Building Micro-Services
• Real Implementationo Throw away the prototype (!)
• Rinse and Repeat
![Page 29: Concurrency at Scale: Evolution to Micro-Services](https://reader031.fdocuments.net/reader031/viewer/2022032514/55d57829bb61ebb32f8b4597/html5/thumbnails/29.jpg)
“So you are reallyserious about this …”
• Distributed tracing• Trace a request chain through multiple service invocations
• Network visualization• “Weighted” communication paths between microservices / instances• Latency, error rates, connection failures
• Dashboard metrics• Quickly scan operational health of many services• Median, 99%ile, 99.9%ile, etc.• Netflix Hystrix / Turbine
![Page 30: Concurrency at Scale: Evolution to Micro-Services](https://reader031.fdocuments.net/reader031/viewer/2022032514/55d57829bb61ebb32f8b4597/html5/thumbnails/30.jpg)
Micro-Service Organization
• Small, focused teams • Single service or set of related services• Minimal, well-defined “interface”
• Autonomy• Freedom to choose technology, methodology, working environment• Responsibility for the results of those choices
• Accountability• Give a team a goal, not a solution• Let team own the best way to achieve the goal
![Page 31: Concurrency at Scale: Evolution to Micro-Services](https://reader031.fdocuments.net/reader031/viewer/2022032514/55d57829bb61ebb32f8b4597/html5/thumbnails/31.jpg)
Micro-Service Relationships
• Vendor – Customer Relationshipo Friendly and cooperative, but structuredo Clear ownership and division of responsibilityo Customer can choose to use service or not (!)
• Service-Level Agreement (SLA)o Promise of service levels by the providero Customer needs to be able to rely on the service, like a utility
• Charging and Cost Allocationo Charge customers for *usage* of the serviceo Aligns economic incentives of customer and providero Motivates both sides to optimize
![Page 32: Concurrency at Scale: Evolution to Micro-Services](https://reader031.fdocuments.net/reader031/viewer/2022032514/55d57829bb61ebb32f8b4597/html5/thumbnails/32.jpg)
Recap: Evolution to Micro-Services
• The Monolith
• Micro-Services
• Reactive Systems
• Migrating to Micro-Services
![Page 33: Concurrency at Scale: Evolution to Micro-Services](https://reader031.fdocuments.net/reader031/viewer/2022032514/55d57829bb61ebb32f8b4597/html5/thumbnails/33.jpg)
Thank You!• @randyshoup
• linkedin.com/in/randyshoup
• Slides will be at slideshare.net/randyshoup