The promises and perils of microservices A map for the adventurous developer

Uwe Friedrichsen – codecentric AG – 2015-2017

@ufried Uwe Friedrichsen | uwe.friedrichsen@codecentric.de | http://slideshare.net/ufried | http://ufried.tumblr.com

Why?

Must! … Do! … Microservices!

No, you don’t!

Microservices are an architectural choice

When do you need microservices?

If you need to go fast

Go fast? What do you mean with “fast”?

A bit of background …

Evolution of economy & markets

Formal part of value creation Solution: machine

Dynamic part of value creation

Solution: man

sluggishness/low dynamic high dynamic high dynamic

The historical course of market dynamics and the recent rise of highly dynamic and complex markets

The dominance of high dynamics and complexity is neither good nor bad. It‘s a historical fact.

t 1970/80 today

Age of crafts manu- facturing

Age of tayloristic industry

Age of global markets

1850/1900

Spacious markets, little competition

Local markets, high customi-zation

Outperformers exercise market pressure over conventional companies

We call the graph shown here the “Taylor Bathtub”. The “bathtub” curve

Source: BetaCodex Network Associates, “Organize for complexity”, BetaCodex Network White Paper 12 & 13

Formal part of value creation Solution: machine

Dynamic part of value creation

Solution: man

sluggishness/low dynamic high dynamic high dynamic

The historical course of market dynamics and the recent rise of highly dynamic and complex markets

The dominance of high dynamics and complexity is neither good nor bad. It‘s a historical fact.

t 1970/80 today

Age of crafts manu- facturing

Age of tayloristic industry

Age of global markets

1850/1900

Spacious markets, little competition

Local markets, high customi-zation

Outperformers exercise market pressure over conventional companies

We call the graph shown here the “Taylor Bathtub”. Pre-industrial era

Source: BetaCodex Network Associates, “Organize for complexity”, BetaCodex Network White Paper 12 & 13

Tailor-made solutions

“Mastery is key to success”

Formal part of value creation Solution: machine

Dynamic part of value creation

Solution: man

sluggishness/low dynamic high dynamic high dynamic

The historical course of market dynamics and the recent rise of highly dynamic and complex markets

The dominance of high dynamics and complexity is neither good nor bad. It‘s a historical fact.

t 1970/80 today

Age of crafts manu- facturing

Age of tayloristic industry

Age of global markets

1850/1900

Spacious markets, little competition

Local markets, high customi-zation

Outperformers exercise market pressure over conventional companies

We call the graph shown here the “Taylor Bathtub”. Industrial era

Source: BetaCodex Network Associates, “Organize for complexity”, BetaCodex Network White Paper 12 & 13

Cost-efficiently scale production

“Get more done with less people is key to success”

Formal part of value creation Solution: machine

Dynamic part of value creation

Solution: man

sluggishness/low dynamic high dynamic high dynamic

The historical course of market dynamics and the recent rise of highly dynamic and complex markets

The dominance of high dynamics and complexity is neither good nor bad. It‘s a historical fact.

t 1970/80 today

Age of crafts manu- facturing

Age of tayloristic industry

Age of global markets

1850/1900

Spacious markets, little competition

Local markets, high customi-zation

Outperformers exercise market pressure over conventional companies

We call the graph shown here the “Taylor Bathtub”. Post-industrial era

Source: BetaCodex Network Associates, “Organize for complexity”, BetaCodex Network White Paper 12 & 13

Continuously respond to changing demands

“Continuous customer communication

is key to success”

Key drivers Industrial era

•  Cost-efficiency •  Scalability •  Repeatability •  Stability

•  Efficiency & scale

Post-industrial era

•  Cycle times •  Adaptability •  Flexibility •  Resilience

•  Effectiveness & speed

Evolution of IT

1960 1970 1980 1990 2000 2010 2020

Complicated

(Business functions)

Complex

(Business processes)

Highly complex

(Business nervous system)

Software crisis

Software engineering

PC

LAN

Internet Business Support

of IT

Selective

Holistic

Complicated

Complex “Moore’s law”

Mobile IoT

1960 1970 1980 1990 2000 2010 2020

Complicated

(Business functions)

Complex

(business processes)

Highly complex

(Business nervous system)

Software crisis

Software engineering

PC

LAN

Internet Business Support

of IT

Selective

Holistic

Complicated

Complex “Moore’s law”

Mobile IoT

We are here …

1960 1970 1980 1990 2000 2010 2020

Complicated

(Business functions)

Complex

(business processes)

Highly complex

(Business nervous system)

Software crisis

Software engineering

PC

LAN

Internet Business Support

of IT

Selective

Holistic

Complicated

Complex “Moore’s law”

Mobile IoT

… but we still base most of our decisions on that

We are here …

Formal part of value creation Solution: machine

Dynamic part of value creation

Solution: man

sluggishness/low dynamic high dynamic high dynamic

The historical course of market dynamics and the recent rise of highly dynamic and complex markets

The dominance of high dynamics and complexity is neither good nor bad. It‘s a historical fact.

t 1970/80 today

Age of crafts manu- facturing

Age of tayloristic industry

Age of global markets

1850/1900

Spacious markets, little competition

Local markets, high customi-zation

Outperformers exercise market pressure over conventional companies

We call the graph shown here the “Taylor Bathtub”. Remember the bathtub curve?

This adds an additional twist …

1960 1970 1980 1990 2000 2010 2020

Complicated

(Business functions)

Complex

(business processes)

Highly complex

(Business nervous system)

Software crisis

Software engineering

PC

LAN

Internet Business Support

of IT

Selective

Holistic

Complicated

Complex “Moore’s law”

Mobile IoT

… but we still base most of our decisions on that

We are here …

Business is very different today …

… than it was back then

Role of IT today

Business

Market

IT today is a …

… Nervous System

… Medium … Product

… Differentiator

Disruptive Technologies

Business Support Systems

Continuous Conversation Digitization

IT today is a key success factor to survive in a post-industrial market

The traditional IT “best practices” are counterproductive because they solve

a completely different problem

We need to rethink IT!

Rethinking IT

What are the new drivers?

IT

Post-Industrialism Highly dynamic markets

Economic Darwinism Lean startup/lean enterprise

Continuous design

Digitization IT as a product

Digital conversation Social media

Contextual computing

Disruption Innovation through disruption Cloud, mobile, IoT, serverless

Big data analytics Data-driven enterprise

force change

on

What are the new goals?

IT

… be quick Short response times

Holistic IT value chain consideration

… be effective Focus on outcome, not output

… improve continuously Improvement as planned activity

needs to …

… be efficient Provide required throughput

… be robust High availability and adaptability

… be flexible Flexible response to changing needs

What are the building blocks?

Adaption DevOps Systemic optimization Inspect and adapt Quick feedback loops Continuous improvement …

Process DevOps Agile Lean Feature Flow (no projects) Design Thinking …

Governance Beyond Budgeting Decentralized control Outcome-driven Lean EAM …

Organization DevOps Autonomous teams Cross-functional teams End-2-end responsibility Routine task automation …

People Craftsmanship T-shaped Responsibility Curiosity Empathy …

Technology Cloud Automation Microservice Heterogeneity Resilience …

(Some) Building Blocks

µService

•  “Microservices are the mapping of organizational autonomy to software architecture”

•  Limited in scope •  Self-dependent •  Loosely coupled

•  Improves •  Autonomy •  Response time (if done right) •  Elasticity

•  Trade-offs •  Higher design effort •  Harder to operate •  Distributed by default

•  Shared nothing •  No shared data •  No cross-service coordination

Still, do I really need microservices?

Take the quick test

Take the real magic triangle …

You may pick two

Good

Fast Cheap

Optimizing for quality and cycle times will result in higher costs

Optimizing for quality and costs will result in long cycle times

Optimizing for cycle times and costs will result in reduced quality

... and pick the two properties that are most important for you

You may pick two

Good

Fast Cheap

Industrial IT

Deliver large batches at minimized costs towards slow markets

Post-industrial IT

Quickly adapt to ever-changing needs of dynamic, fast-moving markets

Startup IT

Test hypotheses and pivot as fast as possible to discover a product-market fit

This is the domain

of microservices

Adaption DevOps Systemic optimization Inspect and adapt Quick feedback loops Continuous improvement …

Process DevOps Agile Lean Feature Flow (no projects) Design Thinking …

Governance Beyond Budgeting Decentralized control Outcome-driven Lean EAM …

Organization DevOps Autonomous teams Cross-functional teams End-2-end responsibility Routine task automation …

People Craftsmanship T-shaped Responsibility Curiosity Empathy …

Technology Cloud Automation Microservice Heterogeneity Resilience …

(Some) Building Blocks Can’t I just go for microservices

without all that other stuff?

Of course you can, but you shouldn’t …

… because you would pay the full price for µservices and hardly gain anything

What is the price for µservices?

Well, ever heard about µservice hell?

A single µservice is easy … … but the complexity of the business functionality remains the same

☛ Complexity is shifted from single µservices to µservice collaboration

µServices are usually self-contained … … i.e., µservices are independent runtime processes

☛ This results in a highly interconnected, distributed system landscape

Consequences •  Design is (a lot!) more challenging •  Implementation is more challenging •  Distributed systems are challenging

•  Lookup •  Liveness •  Partitioning •  Latency •  Consistency •  …

•  New challenges for “monolith developers” à µServices are not easy at all

Only go for microservices if you holistically aim for shorter cycle times

What?

Characteristics 1.  Componentization via services 2.  Organized around business capabilities 3.  Products not projects 4.  Smart endpoints and dumb pipes 5.  Decentralized governance 6.  Decentralized data management 7.  Infrastructure automation 8.  Design for failure 9.  Evolutionary design

J. Lewis, M. Fowler, https://www.martinfowler.com/articles/microservices.html

Componentization via services •  Out-of-process separation of functionality (in contrast to libraries)

•  Independently deployable, replaceable and upgradeable

•  Explicit published service interface

•  Usually coarse-grained service interface due to remote call costs

Organized around business capabilities •  Not organized around technology layers, but business capabilities

•  Broad-stack implementation for a business area

•  Leads to cross-functional teams (Conway’s law)

•  Also possible for monolithic applications, but not the common case

Products not projects •  Avoid the project team/maintenance team responsibility separation

•  Team should own a product over its full lifecycle

•  “You build it, you run it” – full responsibility for the software

•  Ties in with organization around business capabilities

Smart endpoints and dumb pipes •  Avoid centralized business logic in smart communication mechanisms

•  Maximize decoupling by keeping all business logic in services

•  Minimize smartness of communication mechanisms

•  HTTP (& REST) and lightweight messaging are most commonly used

Decentralized governance •  Empower autonomous teams (and give them the responsibility)

•  Enable heterogeneous language and technology choices

•  Use an OSS-inspired approach for shared assets

•  Use consumer-driven contracts

Decentralized data management •  Allow different conceptual data models between services

•  Allow different storage choices per service (“polyglot persistence”)

•  Strive for transactionless coordination between services

•  Embrace eventual consistency

Infrastructure automation •  “Make deployment boring”

•  Automate tests

•  Automate deployments

•  Automate management of services in production

Design for failure •  Design services that they can tolerate the failure of other services

•  Constantly reflect on how service failures affect the user experience

•  Be able to detect failures quickly

•  If possible, restore service automatically

Evolutionary design •  Evolutionary evolve the service landscape

•  Can be used as transition path from a monolith

•  Emphasis on replaceability and upgradeability

•  Make sure service changes don’t break its consumers

How?

How can we avoid µservice hell?

No silver bullet

Topic areas

Design Interfaces

User Interface Frameworks

Datastores Developer Runtime Environment

Deployment Production Resilience

"It seems as if teams are jumping on µservices because they're sexy, but the design thinking and decomposition strategy required to create a good µservices architecture are the same as those needed to create a well structured monolith.

If teams find it hard to create a well structured monolith, I don't rate their chances of creating a well structured µservices architecture.”

- Simon Brown

http://www.codingthearchitecture.com/2014/07/06/distributed_big_balls_of_mud.html

"In theory, programming languages give us all we need to encapsulate state and environment - we just need to use them well.

Maybe we just don’t have the discipline? Maybe we had to explicitly advocate the practice of writing services running in completely different environments using different languages to trigger the sort of encapsulation that we want? If that’s the case, we can either see it as a clever self-hack or something we were forced into by the fact that we programmers are adept at overcoming any sort of self-discipline we try to impose on ourselves.

Perhaps both are true.”

- Michael Feathers

https://michaelfeathers.silvrback.com/microservices-and-the-failure-of-encapsulaton

Design •  Master modularization first •  Strive for low coupling and high cohesion •  Forget about functional decomposition and layered architecture •  Rethink DRY and resusability – avoid deployment dependencies

Foundations of design •  High cohesion, low coupling

•  Separation of concerns

•  Crucial across process boundaries

•  Still poorly understood issue

•  Start with •  Understanding organizational boundaries

•  Understanding use cases and flows

•  Identifying functional domains (à DDD)

•  Finding areas that change independently

•  Do not start with a data model!

Dismiss reusability

•  Reusability increases coupling

•  Reusability leads to bad service design

•  Reusability compromises availability

•  Reusability rarely pays

•  Do not strive for reuse

•  Strive for replaceability instead

”If every service needs to be updated at the same time it’s not loosely coupled”

- Adrian Cockcroft

http://de.slideshare.net/adriancockcroft/dockercon-state-of-the-art-in-microservices

Interfaces •  Plan for interface evolution •  Remember Postel’s law •  Consider BFF services (and API gateways) •  Synchronous vs. asynchronous

µS

Request/Response : Horizontal slicing

Flow / Process

µS µS

µS µS µS

µS

Event-driven : Vertical slicing

µS µS

µS

µS µS

Flow / Process

Order Fulfillment Service

Online Shop

Payment Service

Credit Card Provider

Shipment Service

Warehouse System

<Foreign Service> <Own Service>

Coupon Management

Promotion Campaign

Management Loyalty

Account Service

Payment Provider

PayPal

Loyalty Management

Accounts Receivables

Music Library

E-Book Library

Video Library

E-Mail Server

Coupon

Credit Card

Coordinate

Warehouse

Coordinate

Assets

Notify Cust.

PayPal

Coordinate

Order confirmed

Online Shop

Credit Card Provider

Warehouse System

<Foreign Service>

<Own Service>

Coupon Management

Campaign Management

Account service

Credit Card Service

Loyalty Management

Accounts Receivables

Music Library

E-Book Library

Video Library E-Mail Server

PayPal

PayPal Service

Warehouse Service

Promotion Service

Bonus Card Service

Coupon Service

Music Library Service

Video Library Service

E-Book Library Service

Notification Service

Payment authorized Digital asset provisioned

Payment failed

<Event>

Order fulfillment supervisor

Track flow of events Reschedule events in case of failure

Services are responsible to eventually succeed or fail for good, usually incorporating a supervision/escalation hierarchy for that

User Interface •  Be prepared for multiple UIs for multiple devices, target groups, etc. •  Separate UI and service by change cohesion, not by dogma •  Decouple via client centric BFF service •  Take optimizations for slow and no connectivity into account

Frameworks •  Not the most important issue of µservices •  Should support at least uniform interfaces, observability, resilience •  Nice if also support for uniform configuration and discoverability •  Options are legion – choose the one that fits your needs best

Datastores •  Avoid the “single, big database” – embrace “polyglot persistence” •  Avoid distributed transactions – usually a sign for a poor design •  Try to relax temporal constraints (and make actions idempotent) •  Treat your storage as being “ephemeral”

Development Runtime Environment •  Developers should be able to run (parts of ) the application locally •  Provide automatically deployable “development runtime environment” •  Containers and schedulers are your friends •  Make sure things build and deploy fast locally

Deployment •  Continuous deployment pipeline is a must •  Unify deployment artifact format •  Use either IaC tool deployment … •  … or distributed infrastructure & scheduler

Production readiness

•  You need to solve at least the following issues •  Configuration, Orchestration/Choreography, Discovery, Routing, Observability, Resilience

•  Bad news: No standard solution (yet) in sight •  Good news: Available solutions evolve quickly

Configuration •  Netflix Archaius

Orchestration/Choreography •  Mesosphere •  Nomad •  Kubernetes •  Swarm

Discovery •  Netflix Eureka •  Apache ZooKeeper •  Kubernetes •  Etcd •  Consul

Routing •  Netflix Zuul & Ribbon •  Twitter Finagle

Monitoring •  Hystrix •  Twitter Zipkin (Distributed Tracing)

Measuring •  Dropwizard Metrics

Logging •  ELK •  Graylog2 •  Splunk

Slide is mostly obsolete due to

missing updates for some months

A distributed system is one in which the failure of a computer you didn't even know existed can render your own computer unusable.

Leslie Lamport

Failures in complex, distributed, interconnected systems are not an exceptional case •  They are the normal case

•  They are not predictable

•  They are not avoidable

Microservice-based systems are complex, distributed, interconnected systems

Failures in microservice-based systems are not an exceptional case •  They are the normal case

•  They are not predictable

•  They are not avoidable

Do not try to avoid failures. Embrace them.

resilience (IT) the ability of a system to handle unexpected situations

-  without the user noticing it (best case) -  with a graceful degradation of service (worst case)

Resilience •  Resilient software design is mandatory •  Start with (functional & technical) isolation and latency control •  Add automated error recovery and mitigation •  Separate control and data flow

Event/data flow Event/data flow

Resource access

Error flow Control flow

µS

Isolation

Separation of control/error and data/event flow

W

Flow / Process

W W W W W W W

S S S

S

S

Escalation

Core

Detect Treat

Prevent

Recover

Mitigate Complement

Supporting patterns

Redundancy

Stateless

Idempotency

Escalation

Zero downtime deployment

Location transparency

Relaxed temporal

constraints

Fallback

Shed load Share load

Marked data Queue for resources

Bounded queue

Finish work in progress

Fresh work before stale

Deferrable work Communication paradigm

Isolation

Bulkhead System level

Monitor

Watchdog

Heartbeat

Acknowledgement

Either level

Voting

Synthetic transaction

Leaky bucket Routine

checks

Health check

Fail fast

Let sleeping dogs lie

Small releases

Hotdeployments

Routine maintenance

Backuprequest

Anti-fragility

Diversity Jitter

Error injection

Spread the news

Anti-entropy

Backpressure

Retry

Limit retries

Rollback Roll-forward

Checkpoint Safe point

Failover

Read repair

Error handler

Reset Restart

Reconnect

Fail silently

Default value

Node level

Timeout

Circuit breaker

Complete parameter checking

Checksum

Statically

Dynamically

Confinement

Wrap-up

•  Microservices are no free lunch

•  Use if business responsiveness is crucial •  Complement with additional measures

•  Know the “what” of microservices

•  Reduce stress by especially taking care of •  Good functional design

•  Production readiness (incl. resilience)

•  New challenges for developers (& ops)

So, hope your hell will be more like this …

@ufried Uwe Friedrichsen | uwe.friedrichsen@codecentric.de | http://slideshare.net/ufried | http://ufried.tumblr.com