Microservices Application Simplicity Infrastructure Complexity
-
Upload
centric-consulting -
Category
Technology
-
view
251 -
download
1
Transcript of Microservices Application Simplicity Infrastructure Complexity
MICROSERVICES
TRADING INFRASTRUCTURE COMPLEXITY
FOR APPLICATION SIMPLICITY
INTRODUCTION
ABOUT ME
▸ Senior Architect w/Centric Consulting
▸ National Service Offering Lead for Cloud Computing
▸ @williamklos
INTRODUCTION
AGENDA
▸ A Bit of History
▸ Definition of Microservices
▸ Application Simplicity
▸ Application Complexity
▸ Infrastructure Complexity
▸ Summary
▸ Resources
HISTORY
“ANY ORGANIZATION THAT DESIGNS A SYSTEM (DEFINED BROADLY) - WILL PRODUCE A DESIGN WHOSE STRUCTURE IS A COPY OF THE ORGANIZATION’S COMMUNICATION STRUCTURE.”
Conway’s Law (1967)
HISTORY
HISTORY
MY STORY
▸ First Set of Applications in 1988
▸ Each PC was a “service”
▸ Running DOS, dBASEII, and support scripts - modem
▸ 5.25” floppy-based Sneaker-net
▸ Next Major Set of Applications in 1994
▸ 1 AS/400, 1 database, tons of applications
▸ REST Web Services since 200x
HISTORY
CLASSIC MONOLITH
▸ One cabinet, maybe more if you’re rich.
▸ One database, shared by all applications.
▸ One set of shared libraries.
▸ No need for networking between application modules because they are all right there in the same address space.
▸ May have some logical partitions for other environments.
▸ Probably a monoglot situation.
finance
inventory
report
capacity
ORDER-WOPR
19x9shop
pickship
return
cartbill
queryDB
HISTORY
MODERN MONOLITH
▸ Web services based. Maybe.
▸ Tightly coupled data, tightly coupled network resources, tightly coupled business logic, tightly coupled code.
▸ The monolith is in the shape of a virtual machine.
▸ To put it in the cloud would require a “lift-and-shift”.
▸ But, we are starting to see external resources be a big part of the overall flow.
ship
pick
shop
financereturn
inventory
query
report
notify
bill
review
cart
capacity
ORDER-WOPR
2008DB
DB
DB
HISTORY
WHAT’S CHANGED?
▸ Technology Constraints Removed
▸ Single machines running single core processors,
▸ Slow Networks and Lack of Pervasive Connectivity
▸ Expensive Storage & Expensive RAM
▸ Organizations structured as monoliths
▸ New Technologies
▸ Virtualized Machines, most notably Containers
▸ Wide Variety of Messaging and other Data Movement Options
▸ and of course, the Cloud.
HISTORY
THE PROGRESSION TOWARDS THE SMALL
MONOLITH CLIENT SERVER
WEB & INTERNET
CLOUD & WEB SERVICES
MICRO SERVICES 2020 - 2030
DEFINITION
KETCHUP BOTTLES, JAM TUBS, & SWEETENER PACKETS
DEFINITION
OPEN FIEFDOM
▸ Self-contained, focused purpose society - with a healthy importing and exporting capability.
▸ The ability to open franchises rivals Starbucks and the ability to close them rivals Sears.
▸ Automation is such that the feudal lord can run everything from his throne without the need of pesky serfs.
▸ If the world goes sideways, they can pull up the drawbridge and watch everything outside burn.
WHAT MAKES IT “MICRO”?
IT’S NOT A FUNCTION OF LINES OF CODE
IT’S A FUNCTION OF OPERATIONAL SCOPE
DEFINITION
ONE MICROSERVICE (MINILITH)
shop shop
shop
1 MICROSERVICE 1 MICROSERVICE1 CONTAINER
1 MICROSERVICE1 CONTAINER
1 VM
DEFINITION
RECALL THE LEGACY ORDER MANAGEMENT SYSTEMS
CLASSIC MONOLITH MODERN MONOLITH
finance
inventory
report
capacity
ORDER-WOPR
19x9shop
pickship
return
cartbill
queryDB
ship
pick
shop
financereturn
inventory
query
report
notify
bill
review
cart
capacityORDER-WOPR
2008DB
DB
DB
DEFINITION
A MICROSERVICE-BASED ORDER MANAGEMENT SYSTEM
rest
ship
pick
query
billcart
return
pubsubpubsub
pubsub
rest
rest
rest
reportpubsubpubsub
pubsub
finance
pubsub
shop
rest
review notify
pubsub pubsubpubsub
pubsub
inventory
rest
rest
rest
rest
rest
APPLICATION SIMPLICITY
CHARACTERISTICS
CHARACTERISTICS
SINGLE RESPONSIBILITY
▸ Goal is for it to do one thing, and to do it well. It is responsible for its own behavior and its input contracts.
▸ Has inputs and outputs, and if a data source or state is necessary, it has its own.
▸ Not aware of the outside environment aside from some shared resources in an API Layer or whatever its input data tells it.
▸ All other tenets of microservices spin-off from this core principle.
pick&ship
shippick
CHARACTERISTICS
UNIX’S “SINGLE RESPONSIBILITY PRINCIPLE”
docker ps | grep my_container | wc -l
IN THIS CASE, IT YIELDS A SIMPLE INTEGER
output becomes input to next step
CHARACTERISTICS
ISOLATION
▸ Goal is for it to be a completely self-contained entity with minimal outside dependencies.
▸ This helps minimize impact on other parts of the system.
▸ Has inputs and outputs, and if a data source is necessary, it has its own.
▸ Keeps track of its own state and will only reveal it of asked.
▸ Independently deployable unit.
rest
ship
pick
query
billcart
return
pubsubpubsub
pubsub
rest
rest
rest
reportpubsubpubsub
pubsub
finance
pubsub
shop
rest
review notify
pubsub pubsubpubsub
pubsub
inventory
rest
rest
rest
rest
CHARACTERISTICS
ISOLATED ANATOMY - SIMPLIFIED
▸ At its base level, it only knows what its specific business logic tells it - plus…
▸ …whatever the inputs bring to it.
microservice
CHARACTERISTICS
ISOLATED ANATOMY - COMPLEX (ILL-ADVISED)
▸ The micro service can include all of the extras to be a good application citizen.
▸ Handles tasks that tell it how to operate, scale, communicate, control access, & report its behavior.
▸ Should probably be handled via an API or mediation layer (discussed later).
bill
microservice
security
loggingback pressure
scaling
config versioning
monitoringdiscovery
cachingtransform
CHARACTERISTICS
ISOLATION EXAMPLE
▸ Upgrading the Billing service forces these things to occur:
▸ New Orders and Returns are collected in holding queues
▸ Existing Orders are processed until they reach 0 and the Billing service is stopped.
▸ When new Billing service launches the Order and Returns queues are depleted then processing returns to normal.
▸ No downstream services are affected.
▸ “Safety of Change”
rest
ship
pick
query
billcart
return
pubsubpubsub
pubsub
rest
rest
rest
reportpubsubpubsub
pubsub
finance
pubsub
shop
rest
review notify
pubsub pubsubpubsub
pubsub
inventory
rest
rest
rest
rest
CHARACTERISTICS
COMPARTMENTALIZATION
▸ A structure to help limit the spread of “badness” across a system.
▸ Because microservices should be designed to handle their own issues and have insurance against malcontents, cascading behavior should be minimized.
CHARACTERISTICS
AUTONOMY
▸ Isolation is a pre-requisite for autonomy. It’s hard to automate something that has tendrils of dependency.
▸ When a service can guarantee its own behavior and is not dependent on outside interactions - then it has all it needs to solve its own problems.
▸ Autonomous services allow for more flexibility in dealing with service orchestration, choreography, management, scaling, etc.
shippick reportnotify ship
ship ship
ship ship
ship ship
CHARACTERISTICS
CAN BE STATELESS OR STATEFUL
▸ Stateless allows for auto-scaling without worry since no copy cares about any other.
▸ Strive for immutability
▸ Separate stateful services into their own containers.
▸ A microservice is responsible for its own state and the persistence of that state. There should be no external changes to it.
▸ Don’t share state keeping mechanisms with other microservices.
?
CHARACTERISTICS
ASYNCHRONOUS MESSAGE PASSING
▸ Synchronous operations inside the web service, async outside of it.
▸ Aids in de-coupling, isolation, compartmentalization, resilience, etc.
▸ Helps change mindset from a rigid procedural perspective to a more event-driven one.
▸ Fire and forget. Sometimes.
sync
async
CHARACTERISTICS
ASYNCHRONOUS MESSAGE PASSING EXAMPLE
cart
shop
review
inventorygetProductSummary()
x10@100mseach
getReviewStars()x10@150mseach
getProductList()x10
getCartCount()x1@100mseach
Sync=1000msAsync=100ms
Sync=1500msAsync=150ms
Sync=100msAsync=100ms
TOTALSync=2600msAsync=350ms
CHARACTERISTICS
INTERNAL INTEGRATION - THE INTERSTITIAL
▸ Essentially the spaces in between the microservices.
▸ Part of the negotiated contract between left-side and right-side.
▸ Can be async or sync mechanisms.
▸ Don’t all have to be the same. What makes sense for each interaction?
pick shippubsub
amqp
rest
interstitial
CHARACTERISTICS
EXTERNAL INTEGRATION
▸ Predicting how external systems are going to misbehave is futile. Design for self-defense.
▸ Construct asynchronous calls where possible.
▸ Negotiate expected traffic rates and volume, be able to handle unforeseen events.
▸ Not really that different over internal service integration.
myms
yourms
ExternalAPI
FORMATCSV?JSON?XML?FORM?
METHODREST?SOAP?PUBSUB?QUEUE?
RESPONSES200?201?404?401?50x?
REACTIONSPROCESSQUEUERE-SUBMITTHROTTLESLEEP
CHARACTERISTICS
MOBILITY
▸ Where possible, a micro service should be able to operate anywhere in the enterprise’s operation.
▸ While a microservice should carry what it needs to execute, certain environmental dependencies are more pragmatically shared.
▸ Should be able to run on AWS, Azure, Joe’s Hosting Shack, or in the corporate data center.
▸ The service should also always be addressable across the locational space.
shop
query
query
CHARACTERISTICS
SHARED CAPABILITY > SHARED DATA OR SHARED INSTANCES
▸ While there is less of a DRY (Don’t Repeat Yourself) push in microservices, it’s not a fully WET (Write Everything Twice) architecture either.
▸ Sharing runtime capabilities like databases and security is perfectly acceptable given those capabilities are sufficiently segmented so as to minimize impact of other applications.
▸ i.e. Don’t negatively impact “change safety”.
CHARACTERISTICS
TIME AWARENESS OF DATA
▸ Data in-flight is either in “the past” (has happened) or in “the hopeful future” (will happen)
▸ There is no single notion of “the present”. Only the “local” now.
▸ Inside our microservice we can control everything within our boundary.
▸ Outside the boundary, not our problem.
thelocalnow
thepast
thefuture
API MANAGEMENT
bill
microservice
security
loggingback pressure
scaling
config versioning
monitoringdiscovery
cachingtransform
API MANAGEMENT
OVERVIEW
▸ Similar to an orchestration layer like SOA designs but still need to abstract direct visibility to endpoints.
▸ Allows you to change granularity level without impacting users.
▸ Includes common functionality around routing, security, load-balancing and registration services.
ship
pick
query
billcart returnreport finance
shop review notifyinventory
API LAYER
Web App Mobile App
PartnerApps
ClientApps
EmployeeApps
SECURITYROUTINGDISCOVERYLOGGINGLOAD BALANCEREGISTRATIONAUTHENTICATETRANSFORM
API MANAGEMENT
SERVICE DISCOVERY
▸ Upon startup of a micro-service, it registers itself with a controlling entity.
▸ Access to service can be direct if all operating copies are known or by redirection.
▸ Assures all nodes get traffic when autoscaling.
▸ Essentially a load balancer.
billing
billing 172.16.6.22:3000 2016/07/31 13:31:04
billing 172.16.6.22:3001 2016/07/31 13:45:04
API Layer
50ms
40ms
MODULE IP ADDR:PORT TIME STARTED PING
registerService()
POST https://apilayer/order/:ord_id/bill
POST https://172.16.6.22:3001/order/:ord_id/bill
GET https://apilayer/billings
{“hosts”:[“172.16.6.22:3000”,”172.16.6.22:3001”],”count”:2}
Get List of Active Billing Services
Auto Redirect Service
API MANAGEMENT
SECURITY
▸ Primary security would typically be handled by the API Layer.
▸ Individual services have enough security to know when they are being called directly and redirect to API Layer or reject the request with a 401.
▸ Individual services will also need to provide authorization for services that call it (certs, whitelisted Services, etc.).
shipbillcart
return
shop
bill
microservice
loggingback pressure
config
transform
APPLICATION COMPLEXITY
DETERMINE GRANULARITY
APPLICATION COMPLEXITY
DETERMINING BOUNDARIES
▸ Define a model based upon a specific business perspective.
▸ But it doesn’t need, and maybe it shouldn’t, be based on your standard organizational makeup.
▸ Distributed > Centralized
▸ Different perspectives of the same topic can yield different models
▸ e.g. Uber driver vs. Uber passenger
▸ Domain-Driven Design may help here.
Back-Office Order ContextCustomer Order Context
rest
ship
pick
query
billcart
return
pubsubpubsub
pubsub
rest
rest
rest reportpubsubpubsub
pubsub
finance
pubsub
shop
rest review
notify
pubsub pubsubpubsub
pubsub
inventory
rest
rest
rest
rest
rest
ORDER PROCESSING DOMAIN
APPLICATION COMPLEXITY
DETERMINE GRANULARITY
▸ Need to do one thing very well, but how do you define “one thing”?
▸ Signs of being too small include services being “chatty”, too much dependence on other services, or poor-performance.
▸ Signs of being too big include code changes impacting pseudo-related functionality, compromises in data design, or too much “global cruft”.
shop
shopbill
cart
orderentry
bill
remove
add2cart
viewcart
savecart
emptycart
APPLICATION COMPLEXITY
DETERMINE GRANULARITYshop
findProduct()listProducts()viewProduct()
cart addToCart()removeFromCart()viewCart()emptyCart()saveCart()
bill gatherBillInfo()gatherShipInfo()billOrder()
notify pick report
query ship finance
return review
notifyBilled()notifyRejected()notifyShipped()notifyReturned()
inventory
cancelOrder()makePickList()pickOrder()notifyBO()
logTransaction()
updateStatus()getStatus()
shipOrder()attachTracking()
allocateInv()pickInv()queryInv()replaceInv()relocateInv()returnInv()
generateRA()receiveRA()
createReview()showReviews()blockReviewer()thumbsUp()thumbsDown()
updateLedger()
DATA COUPLING
APPLICATION COMPLEXITY
DATA COUPLING
▸ Data consistency requires coordination.
▸ Coordination can inhibit scalability, throughput, and availability.
▸ There is no global single source of the truth. Truth is local, within the context of the service.
bill
cart
shop
review
inventory
DB
billcart
shop review
inventory
LOW
HIGH
pick
billcartpubsub
rest
shop
rest
inventory
rest
rest
rest
rest
APPLICATION COMPLEXITY
DATA COUPLING - EVENTUAL CONSISTENCY EXAMPLE
1000 0 1000
MODULEINVENTORY (ON HAND)
INVENTORY (ALLOCATED)
INVENTORY (CURRENT)
shop
1000 5 995
995 0 995pick
bill
1000
SHOP INVENTORY (CURRENT)
995
1000
1000 0 1000pick
995
EVENTUAL CONSISTENCY != BATCH
HANDLING FAILURES
CHALLENGES
HANDLING FAILURES - BACKPRESSURE
▸ When the next phase can’t be completed right now, defer it yourself or give it back to the caller.
▸ A way of governing discrepancy between slow and fast systems.
▸ Process it when the issue is cleared.
POST http://apilayer/order/bill billingcartrest
200
publishBillOrder() > 0
backPressureBillQ billPubSubChannel
NO
YES
50x
Relieve Back Pressure When Possible
Apply Back Pressure When Necessary
OK
billQDepth() > max
billQFillRate() > max
CHALLENGES
HANDLING FAILURES - AVAILABILITY
▸ When a service is down, it can be handled in a similar way to back-pressure.
▸ Can also look for additional services of the same type (API Layer should handle).
▸ If severe enough, then stoppages can be cascaded up in a controlled fashion to make sure overall system integrity is maintained.
POST http://apilayer/order/billcart
rest
200
SERVICE IS DOWN
overflowBillQ billPubSubChannel
NO
YES
50x
Pull from Queue When Service Resumes
OK
billing
CHALLENGES
HANDLING FAILURES - CIRCUIT BREAKER
▸ As long as the circuit stays closed, all traffic passes as normal.
▸ If the service is experiencing a backlog or under stress, it can partially “trip” the circuit effectively throttling input.
▸ If the circuit is open, then the service is down and all calling functions need to step back behavior until service resumes.
cartNO
billing
ADD RETRY DELAY
backPressureBillQ
TRANSACTIONS
APPLICATION COMPLEXITY
DISTRIBUTED TRANSACTIONS
▸ Traditionally upholds the illusion that you are the only one using the data you need.
▸ Distributed transactions can cause scalability issues and introduce “brittleness”.
▸ A recommended alternative to distributed transaction is the Saga pattern.
▸ Essentially having an “undo” action along the way executed in reverse in case of failure.
cartshop inventorybill
addToCart() billOrder()removeFromCart()
allocateInv()
compensate()
pick
compensate()
deductInv()commit()
commit()commit() commit()
APPLICATION COMPLEXITY
MORE DISTRIBUTED TRANSACTIONS
▸ Additional ways of handling would include:
▸ Previously discussed methods of applying “back-pressure”, deferred processing, or retries.
▸ Back off of any dogmatic approach to distributed effort in favor of a centralized approach for that particular piece.
▸ e.g. Inventory in our Order Management model.
INFRASTRUCTURE COMPLEXITY
INDIVIDUAL MICROSERVICES ARE RELATIVELY EASY TO DESIGN AND DEPLOY.
THE COMPLEXITY COMES WHEN YOU HAVE MORE THAN A HANDFUL THAT NEED TO BE MANAGED.
INFRASTRUCTURE COMPLEXITY
YOU NEED HOW MANY SERVERS?
▸ Our example has 12 microservices consisting of 36 actual applications services.
▸ If we need a robust production environment, we’ll want 3x servers (36).
▸ If we want additional environments, then we will need more even still.
▸ Or…we could use containers.
rest
ship
pick
query
billcart
return
pubsubpubsub
pubsub
rest
rest
rest
reportpubsubpubsub
pubsub
finance
pubsub
shop
rest
review notify
pubsub pubsubpubsub
pubsub
inventory
rest
rest
rest
rest
INFRASTRUCTURE COMPLEXITY
HOW ABOUT CONTAINERS?
▸ The base unit of measure for a micro service is a container.
▸ Containers are like micro-VMs.
▸ Lightweight virtual machine that has all of the dependencies necessary to execute the contained application.
▸ And no more.
MICROSERVICE
pick
shop shop
cart
VM
CONTAINER
CAPACITY
INFRASTRUCTURE COMPLEXITY
AUTOMATION IS YOUR FRIEND
pick
shop shop
cart cart
review
ship
query
notify
shop ship shop
bill
bill
return
report▸ Service discovery allows containers to be brought into the world and removed from it in an automated fashion.
▸ Products like Kubernetes and Docker Swarm aid in this ability and make the management of you containers fleet much easier to deal with.
INFRASTRUCTURE COMPLEXITY
DATABASE PROLIFERATION
▸ Self-contained means a private datasource.
▸ Allows for the “right-tool for the right job”.
▸ Adds management and monitoring complexity.
▸ Potentially adds licensing and support costs.
INFRASTRUCTURE COMPLEXITY
TECHNOLOGY PROLIFERATION
▸ It’s not just databases that are proliferating. Languages, protocols, connectors - all are up for consideration.
▸ Allows for the “right-tool for the right job”.
▸ Builds silos of knowledge, but they’re small.
▸ Of course, you can “monoglot” the crap out of your architecture if you want.
INFRASTRUCTURE COMPLEXITY
VARIOUS TECH STACKS FOR EACH MICROSERVICEshop
DB: RDBSTACK: .NETVER: 2.50
cart DB: RDBSTACK: JAVAVER: 3.22
bill DB: DocDBSTACK: GOVER: 1.5
notify pick report
query ship finance
return review
DB: RAMSTACK: GOVER: 1.73
inventory
DB: DocDBSTACK: GOVER: 1.01
DB: KeyValueSTACK: ELIXIRVER: 1.7
DB: DocDBSTACK: RUBYVER: 1.21
DB: DocDBSTACK: GOVER: 4.2
DB: RDBSTACK: JAVAVER: 6.3
DB: DocDBSTACK: RUBYVER: 2.00
DB: RDBSTACK: .NETVER: 0.9B
DB: RDBSTACK: JAVAVER: 10.5
THE DEV/OPS PLAY
CHALLENGES
THE DEV/OPS PLAY
▸ Dev/Ops has a huge impact on a successful micro services implementation.
▸ Monitoring, scaling, load-balancing, routing, instantiation, and overall robustness require better levels of automation.
bill
microservice
security
loggingback pressure
scaling
config versioning
monitoringdiscovery
cachingtransform
SUMMARY
SUMMARY
OPERATING PRINCIPLE EXAMPLES
▸ UNIX
▸ Do One Thing Well
▸ Build Afresh
▸ Expect Output to Become Input
▸ Reduce Human Input
▸ Try/Fail Early
▸ Don’t Hesitate to Throw it Away
▸ Become Toolmakers
▸ Netflix
▸ Antifragility
▸ Immutability
▸ Separation of Concerns
SUMMARY
THINGS TO REMEMBER
▸ Very small scope leads to much easier “micro-applications” that provide small pieces of value. The trade-off is overall environmental management.
▸ A solid DevOps practice is vital in managing the seemingly chaotic environment of a micro services implementation.
▸ You decide the appropriate granularity.
▸ Yes, it’s like SOA, but different. Lighter weight and more nimble.
▸ Once you can automate the management of your microservice fleet, having 5 or 50 or 500 microservices, the amount of pain in administering them should be logarithmic rather than linear.
SUMMARY
MORE THINGS TO REMEMBER
▸ The ironic thing about systems that deal in 1s and 0s is that the rules that govern how to implement them is never 1s and 0s.
▸ There is no one true “right way” to do any of this. Everything is a tradeoff between the needs of the business and the realities of your situation.
▸ That being said, strive towards:
▸ services doing “one thing only”
▸ minimizing coupling (data and functionality)
▸ automation
▸ The function, behavior, and citizenship of your microservice is on you.
SUMMARY
THINGS TO TRY
▸ “Antifragility” testing (Netflix does this) will introduce failures into the system to determine amount of drama created. Try removing nodes or introducing bad keys into your process to see how the system reacts. Seriously!
▸ Treat your servers and processes like “cattle” rather than “pets”. Everything is disposable outside of data.
▸ Assign services to teams. Instead of having people assigned to support entire applications, assign smaller teams to individual services where they own the entire stack including design and support.
“ANY ORGANIZATION THAT DESIGNS A SYSTEM (DEFINED BROADLY) - WILL PRODUCE A DESIGN WHOSE STRUCTURE IS A COPY OF THE ORGANIZATION’S COMMUNICATION STRUCTURE.”
Conway’s Law (1967)
HISTORY
RESOURCES
RESOURCES
MORE DETAILED INFORMATION
▸ http://www.reactivemanifesto.org/
▸ http://thenewstack.io/ten-commandments-microservices/
▸ http://goo.gl/4qLLbG (Data Outside vs. Data Inside)
▸ http://goo.gl/zMnHtU (Cattle vs. Pets)
▸ https://msdn.microsoft.com/en-us/library/jj591569.aspx
▸ http://martinfowler.com/bliki/CircuitBreaker.html
▸ Microservice Architecture, Nadareishvili, Mitra, et al.
▸ Microservices vs. Service-Oriented Architecture, Mark Richards
▸ Reactive Microservices Architecture, Jonas Bonér
Q&A