Advanced Microservices - Greach 2015

THIRDCHANNEL

Advanced Microservice Concerns

Steve Pember CTO, ThirdChannel

@svpember

Greach, 2015

Let’s talk about Microservices

THIRDCHANNEL

* Before I begin, I should probably point out there’s not much in this talk that specifically relates to Groovy* In fact, there’s no code in this talk at all. Mostly just theory.* Still, I believe the information is useful. Microservices are all the rage.* Incidentally, At 3C we love microservices, and Groovy is our most used language in our services

THIRDCHANNEL

Agenda• Microservices: Overview

• Microservices: Advantages

• Service Design

• Organizational Structure

• Dev Ops

The agenda for this presentation first is to briefly cover the microservice architectural pattern, some of it’s advantages,and then get into some of the advanced concerns, where I’ve grouped them by overall topic… Service Design, Organizational Structure, and Dev ops!

THIRDCHANNEL


* To begin:* Who here has heard of ‘Microservices’? Just trying to measure how to talk about this* Who here is currently working with or has worked on a Microservices platform?

THIRDCHANNEL

Microservices• Fight the Monolith

So let’s get started with a quick overview of what Microservices actually are.

I believe microservices are an attempt to fight against a distressing pattern in software architecture: the monolithic application

THIRDCHANNEL

Building a monolith has become, I think, the natural response for many developers or companies when they first start out on a project

THIRDCHANNEL

-A monolithic app is: Single, logical executable unit, often comprised of 3 components - at least for a web app: the View layer (CSS/HTML/JS), the middle tier (contains business logic) and the data storage.- here’s an example of an e-comm app’s middle tier: product browsing, catalog management, order processing, shopping cart- as I add features to my application, I simply tack them on to the existing code base. Feature upon Feature upon feature

Feels Natural

The danger with the monolith I think, is that this feels like the natural thing to do.You start a new project, choose a framework and begin building.

start out easyquick wins as you add new features“Oh, I need a shopping cart now. Great, let’s shove in a Cart model and a Controller. Maybe service”

…in the long term, though, you’ll run into some problems

NEED TO GO BACK AND ADD BITS ABOUT HOW MONOLITHS GROW

complexity grows quickly as you keep adding new features!Even if you’re very, very careful… application will still inevitably become a mess

–Johnny Appleseed

“Type a quote here.”

To the point where it leads to Developer Misery.

Open up your IDE, only to be bogged down by the countless controllers or View objects popping out at you. Quick hacks and other Technical Debt that promised yourself you would go back and fix … and you never do… make the system hard to develop.

When adding new features or fixing bugs, developers will spend huge amounts of time figuring out how something works before they can even start fixing a problem. May require dragging in other people who worked on it before to help.

Microservices to the Rescue!

This frustration - of working on large, monstrous applications - , I think, is why Microservices have become so popular.

THIRDCHANNEL


• Distributed

So, when we use that term “Microservice”, what exactly are we talking about?

Well, fundamentally what it means is that we’re building a distributed application

THIRDCHANNEL

So, instead of the monolith I showed you before, we do this:- describe break up

What About ‘Service Oriented Architecture’?

Now you may be saying:“Wait, isn’t this just Service Oriented Architecture? People have done this before.”

* It’s true, this has been done before. SOA has been around for years now* Microservices, though, are a more focused and disciplined version of SOA.* REASON BEING: Involves technical and organizational changes not defined in SOA. Let’s go over a few of them:

THIRDCHANNEL


• Distributed

• Independent

First: Independence.In a Microservice system, Each service should be completely independent of each other

THIRDCHANNEL

Independently developed. Each with their own separate code repository.Independently deployed.

- allows for independent history- results in isolated code which is great for adhering to design principles like separation of concerns.-

THIRDCHANNEL


• Distributed

• Independent

• Single Context

Next.Each Service contains a single context…. Or rather, each service is responsible for One thing.This one thing could be a particular set of data or a particular process.

This approach naturally leads to smaller, less complicated codebases

THIRDCHANNEL

Let’s say I was building an e-commerce application. I may have… <name services>Each one responsible for a single object or a single process

Each one of them, will be developed separately from each other and will be each far less complicated than if we tried to jam each service together into a single monolith

THIRDCHANNEL


• Distributed

• Independent

• Single Context

• Simple, “Dumb”, Communication Technology

Your services should communicate with each other using very simplistic channels, but be very smart about how they do so.<slow down>

No Enterprise Service Buses

Which basically means “No enterprise Service Buses” or ESBs.You would often find ESBs in Service Oriented Architecture or SOA implementations. They are complicated services that are responsible for figuring out how to route messages to the various services.

In practice, they Would often be very complex with custom logic; another thing to maintain

Instead, Use HTTP or Message Queues

Instead, use direct HTTP calls or Message Queues* Http: Already in the web, so why not use a web technology? HTTP allows for Direct communication between services, but it’s synchronous… so, while

the request is active, both services have resources that locked in the communication* Message Queues on the other hand: Asynchronous, but require some additional code or library for messaging formats

* Regardless, Each approach does require a lightweight program to help with the communication.

THIRDCHANNEL

So with HTTP based comms and a handful of services, you may end up with something like this:

web of inter service communications. Each service needs to know the ‘address’ of the other services, along with when it needs to talk to the others.So, if a user places an order, that code needs to know to talk to the order history service and the billing service in a particular order.

THIRDCHANNEL

* With HTTP, you’ll need a service to help you discover other services, otherwise each service would also have to maintain a huge internal map of the locations of the other services. * A “Service Discovery Service”* Netfix has open sourced one, called “Eureka” which is very useful.* With all of that, HTTP is very commonly used in Microservice Architectures.

THIRDCHANNEL

With a message Oriented approach on the other hand..- services don’t know where each other are, nor do they care where the other services are.- communicate entirely through the the Message Broker- When an event needs to go out, it is simply generated. The services do not need to individually worry about which other service needs this information.- Personally, I prefer this approach for a variety of reasons, which I’d be happy to talk about after.

THIRDCHANNEL

Regardless, With MQ you’ll need a Message Broker to quickly and efficiently route your messages.I recommend either:

THIRDCHANNEL


• Distributed

• Independent

• Single Context


• REST-ful Communications

Your services should communicate with each other using very simplistic channels, but be very smart about how they do so.

REST != CRUD != HTTP

I feel that when many people think of REST or say that they have a “RESTful api”, they’re thinking of performing CRUD operations (create / read / update / delete) over HTTP, using some of the HTTP verbs.

THIRDCHANNEL

There are many more aspects to REST than that. Primarily, I think, no State should be shared across services. Attempting to keep shared state across distributed services WILL drive you crazy.

Caching, Layering, and Uniform Interface

the REST specification also calls for caching your data, layering your services, and ensuring they have a uniform interface.- Layering: when your services ask for or push information, they shouldn’t care whether who they’re talking to is the actual service or some intermediary that’s then passing it on. As long as the data is handled, the services should trust that the handlers know what to do.- Uniform interface implies many things… I’m going to take a shortcut and just say that each of your services should communicate with each other via the same technique. If anyone wants to argue about HATEOAS later, I’d be happy to.

THIRDCHANNEL


• Distributed

• Independent

• Single Context


• REST-ful Communications

• Dedicated, Cross-Purpose Teams

lastly, … from an organizational standpoint, the teams building the service should be cross-purpose.

THIRDCHANNEL

Teams in charge of a service should come from a variety of disciplines. Engineering, QA, Art & Design, Project Management, perhaps a deployment engineer, etc.

THIRDCHANNEL



* Next on the agenda: Microservice architectures have been gaining huge followings in the past few years due to the benefits they bring.

THIRDCHANNEL

Microservices: Advantages• Loose Coupling!

Because our systems are independent of each other, this promotes natural Loose Coupling. Individual teams can develop the services independently, which keeps complexity down.

THIRDCHANNEL


• Right Tool for the Job

* Each service can be built according to the technology best suited to it. * Not locked into one language or framework

Always use Groovy, naturally

But of course, Groovy is always the right tool for the job!

THIRDCHANNEL



• Easy to Scale Services

It’s easy to scale Microservice applications to meet the particular user demand

–Johnny Appleseed


Borrowed this from Martin Fowler’s websiteAs demand on the application grows, I can scale just the components that are needed, rather than needing to replicate the whole monolith

* targeted growth results in your team using fewer resources * which saves $$$ and makes your team more efficient

THIRDCHANNEL




• Easy to Scale Development

Speaking of scaling…

Scaling development in general is very difficult. With a microservices architecture, you have an advantage where it’s relatively easy for developers.Because the code base is smaller, your developers can hold the context of a service in their head at once (something impossible with a monolith). This allows them to find bugs quickly and identify exactly where new features should go.

THIRDCHANNEL





• Fast to Fail / Start Over

Microservices allow us to fail fast.

Find out quickly of a technology or a service isn’t working. Easy to refactor or replace a service

…

THIRDCHANNEL





• Fast to Fail / Start Over

• Allows for Continuous Delivery

Finally, Adopting a Microservices approach allows for your team to Continuously Deliver your code.

THIRDCHANNEL

Which, I think, is the absolute ultimate goal of any development team. I think it’s the main reason to use microservices to begin with.

The ability to basically make a change or implement a feature and then immediately have it running in production? It’s thrilling, and powerful, and liberating for your developers.

Furthermore, It essentially eliminates the notion of a Sprint and dramatically changes Agile Development.

Goodbye, Release Windows

Who here has worked in teams where you have a development release window or a ‘release train’? I have. Who here was worked in companies where you release a new build every 3 or 4 weeks… or longer? I certainlyIt was miserable, it was depressing, to stack up a bunch of features for release. Everyone was a little unsure if the changes they made broke other people’s code. If a bug was detected in the wild, it wouldn’t be fixed until the next release. Madness.

Microservices Have Disadvantages

That all being said, Microservices do have their downsides. Like any new buzzword or new hot thing in tech, people will complain about it or point out any problems it has.

Microservices do have a few complaints, although I don’t agree with all of these.Some of the complaints I’ve seen have been:

THIRDCHANNEL

Microservices: Problems• Distributed Systems are Hard

Working with distributed systems is difficult. Or really, it’s difficult to wrap your mind around a distributed system.

And yeah, it can be hard to worry about multiple small services.

THIRDCHANNEL


• Added Complexity

Splitting your code into services can add additional operational complexity, as now we have more moving parts in our platform.

I disagree with this one, breaking your code down reduces programming & computational complexity

THIRDCHANNEL



• “What if a service goes down?”

You may here someone in your organization say something like “What if service X goes down? It’s central to our business!”

Well, That person is not ready for microservices.

THIRDCHANNEL



• “What if a service goes down?”

• Significant Operations / Dev Ops Work

Microservices will require some additional focus on dev ops related work, like streamlining deployments.

These Can All Be Mitigated

These points, while valid, can be addressed, reduced, or handled pretty well. These next sections cover some of these ‘Advanced’ concerns when working with Microservices. These are things that everyone should be aware of.

THIRDCHANNEL



• Service Design

* First up: Service design!* As we’re building our individual services, what should we be aware of?

THIRDCHANNEL

Service Design• Single Bounded Context

I’ll reiterate here that each service needs to be a single bounded context.It’s a bit tricky, but this point basically states how large a service should be… which is: a service should be small enough that a single developer can hold the entire functionality of it within their head.

THIRDCHANNEL


• Resilient

your services should be resilient

they should embrace and welcome that errors will occur within the platform… and they should know how to handle any potential error.

For example, if our order billing system goes down… instead of shutting down the whole website, we simply accumulate orders, allow our users to go on as normal, and hope the system comes back up eventually.

Independent Things Fail Independently

The overal rule I think, is that your services should maintain their independence… and fail independently.

THIRDCHANNEL


• Resilient

• Fast and Efficient

Design your services to be fast and efficient.

–Johnny Appleseed


You should strive to return a result to your Request as fast as you can. In the US we have an idiom called “hot potato” which means to get read of something as fast as you can.It’s important that your back end services return a result as quickly as possible in order to Speed is king.

Several years ago, an engineer with Amazon named Greg Linden revealed that the company ran experiment where they intentionally slowed down site performance- 100ms delay resulted in 1 percent drop in sales.

THIRDCHANNEL


• Resilient


• Each Service is an Authority

Each service should be responsible for one particular thing within your application. This relates to single bounded context.

Each service should be an Authority on some data… or an authority on a particular process. For example, I may have a ‘User Service’ that is the source of information on Users within my system… or I could have an Email Notification Service which is responsible for sending emails

THIRDCHANNEL


• Resilient



• Embrace Eventual Consistency

Embrace Eventual consistency. Does everyone know about this? basically, in a distributed system, you must accept that an individual client or service may not have the most recent view of the data… but that eventually, every service will have data consistent with the authority service. This is a very large subject and I’d love to talk about some of the nuances about it later.

THIRDCHANNEL


• Resilient




• CAP Theorem

These next two topics are perhaps the most difficult in this talk. At least, very difficult to explain.

First up, the CAP Theorem

THIRDCHANNEL

CAP Theorem• (C)onsistency

• (A)vailability

• (P)artition Tolerance

The CAP Theorem states that there are 3 three attributes or guarantees that a Distributed System can have:- consistency: when an update occurs in our system, all services know about it at the same time (which is the opposite of eventual consistency)- availability: when information is asked of the service, it responds as quickly as it can- partition tolerance: a service is able to handle or deal with failures occurring in other services

THIRDCHANNEL


• (A)vailability


• Pick 2!

Well, the CAP Theorem says that’s great, and in fact you can only have two of these. A distributed system can only meet two of these guarantees.

THIRDCHANNEL


• (A)vailability


• Pick 2!

• Theorem from Paper which proved “Brewer’s Conjecture”

Originally called Brewer’s Conjecture

THIRDCHANNEL

In 2012, this paper contained a formal proof of the Conjecture and it graduated to full-on theorem.

THIRDCHANNEL


• (A)vailability


• Pick 2!

• Theorem from Paper which proved “Brewer’s Conjecture”

• CAP Decisions can be Service-Specific

Web Apps -> A & P Banks / Money -> C

THIRDCHANNEL

Service Design• Single Bounded Design

• Resilient




• CAP Theorem

• Locality of Reference (Data Locality)

Last… let’s talk about Data Locality.

You should be very concerned about Data Locality

THIRDCHANNEL

Data Locality• Spatial: “How far away or how distant is our data?”

There are two aspects of data locality:1. Spatial. How proximal is our data? In a single data base, data can be considered highly spatial if it’s in the same table. Data which is reachable by joins is less spatial and perhaps less efficient to reach

THIRDCHANNEL


• Temporal: “How often is our data accessed?”

2. Temporal. Being highly temporal means that data is read frequently. Highly temporal data - is an excellent candidate for caching.These terms take on slightly different meanings in a distributed system, though.

THIRDCHANNEL

In an ideal distributed environment, each system would be completely separate. -However, there will always be some conceptual overlap between domain objects. For example, let’s say we have an ecommerce system where <describe system>. -Red lines are anchors between data localities - Here, both the user service and the O history service maintain a concept of user. But, users’ uuid is a shared natural key which acts as each

services’ anchor to understanding the concept of a User. - Our user service is the authoritative source on a User - Our order history system also understands the User, but simply maintains the order history along with the user uuid of the user placing the order

THIRDCHANNEL

Same chart from before, but another service, communication via email. This service also conceptualizes the user, but with a different subset of information. Namely, the email address. - Email address is stored in the user information service, but is *also* used extensively by the communication service. Which one should be the

authoritative source of the email address? The communication service uses it more (and thus has higher Temporal locality), but email is a key piece of information which should be managed by the user service thus keeping it Spatial close to the other user data.

- Tough Choice - One Compromise is to (unfortunately) synchronize the email address across multiple services - Or, communication service has no concept of User’s email address, but simply blindly sends emails out. The email addresses involved must be

placed into each message received by the Communication service - Or, third option, perhaps we cache the email address on the communication service and wait for an event or notification from the user service

when it changes (Eventual consistency, eh? When the user changes their email address, the communication service eventually, busts the cache and resets the email address. The user may miss an email until that happens, and that may be ok).

THIRDCHANNEL



• Embrace Eventual Consistency, Do not sync data!

Remember the point from earlier though… embrace eventual consistency

My recommendation is to never sync data. It will likely be inevitable that you’ll eventually have to synchronize something

Do NOT Synchronize

I believe that synchronizing data will eventually always break down and cause no end of problems. Inevitably data will become out of sync; you will have to create some jobs or processes to re-sync any data which has ‘drifted’. We’ve tried synchronizing data at ThirdChannel in the past, and data discrepancies were the the primary cause of bug reports. It was terrible.

If you overhear anyone on your team say “Well, we *could* just synchronize these two things”, roll up a newspaper and whack them on the nose like a dog.

THIRDCHANNEL



• Embrace Eventual Consistency, Do not sync data!

• Goal: Have Highly Spatial Data & Efficiently Handle Temporal Data

With a MS, your goal is to have..

THIRDCHANNEL



• Service Design


* Next,

THIRDCHANNEL

Organizational Changes• Cross-Discipline Teams

THIRDCHANNEL

DBA

Engineers

QA

UX

In many companies, it’s a common pattern to have your staff broken up into massive departments: a QA department, a UX team, your engineers, and the dreaded DBAs…

Instead, with microservices, it is encouraged to break up that thinking. Build teams around your services such that it’s comprised of multiple people of multiple disciplines.

One of the best working experiences I had was with an awesome QA person that sat right next to me… finish story

THIRDCHANNEL


• Dedicated & Responsible

With that, each team is ultimately responsible for their individual service(s).

When I say I responsible, I mean it. If the service malfunctions over the weekend, the people on that team should be receiving notifications about it. There’s not a team of junior engineers or interns around to fix it. If you commit code, you’re responsible for fixing it.

What ends up happening, is that the engineers end up writing a hell of a lot of unit and integration tests in order to avoid the dreaded early am phone call.

THIRDCHANNEL



• Autonomous

With all that, the engineering teams should be autonomous…

–Johnny Appleseed


-But, They also need to be aligned well with the business. Essentially, provide the end goal, and let the team figure out how best to get there.-I borrowed this drawing from a Spotify blog post on their engineering culture, and it shows the concept of a highly Aligned, highly Autonomous team.

Describe the top right

THIRDCHANNEL



• Autonomous

• Microservice Champion(s)

You will need some architects or team leads to champion the micro service vision.

* These people will need to have the vision to direct people. To guide them. This leadership team will need to have a shared vision and design from which they can guide the rest of the company.*If you start down the Microservices path, you will have some people in house who disagree with it. They’ll complain for various reasons I mentioned earlier.*What they’re really saying is that this new approach is too hard and they’re afraid. The leadership team must be there to guide people forward and to educate.* And to encourage communication

THIRDCHANNEL



• Autonomous

• Microservice Champion(s)

• Conway’s Law

which brings us to Conway’s Law. I talked about Conway’s law last year.. I think at Great.

I was at a conference a couple weeks ago, and it was the subject of several talks. So… I just want to tell you all I’m a bit of a trendsetter, leading the pack.

-Melvin Conway, 1968 http://en.wikipedia.org/wiki/Conway's_law

“organizations which design systems ... are constrained to produce designs which are copies of the communication

structures of these organizations”

Somewhat tongue and cheek, but basically says that the structure of any given computer system will begin to reflect the social structures of the company that built it. While yes, teams should be organized around services, these services still need to communicate with one another.

http://en.wikipedia.org/wiki/Conway's_law

If Conway’s Law is not observed, some interesting things can happen.

You may end up with situations where:- intra service APIs not matching up at all- teams expecting data to be in services, that may not be there. A misunderstanding of what a service provides- team A waiting for Team B to finish service without bothering to help them- eventually you may get a point where teams start blaming or attacking each other for problems in the system. There can be arguments. “Man, the team

working on the Shopping Cart Service is terrible. Their code is always broken”

Hard Problem to Solve

Avoiding Conway’s Law is not easy, and there’s no clear way to do it. Try your best to encourage communication and collaboration between teams. Rotate people between service teamsGet everyone on a shared communication channel (Slack is a popular choice)Ensure that each service’s API is published and highly visible to other teams.

THIRDCHANNEL



• Service Design


• Dev Ops

* Next up, Dev ops

Because each team is responsible for their deployments, nearly everyone in your team is going to get some Dev Ops experience.

Whether they like it or not….

However, there are several things you can do to make the Dev Ops experience better, and make it easier to manage your application

THIRDCHANNEL

Dev Ops• Centralized Service Monitoring & Logging

THIRDCHANNEL

Receives Metrics, Logs and Health checks

One absolute requirement of a successful microservices deployment is to have detailed monitoring of your services.

This monitoring service is responsible for performing heartbeat checks on your services and gathering metrics about them. It should also be responsible acting as a centralized logging platform for each of your services.

If you think this may end up being an absolute ton of data… well, you’re right.

THIRDCHANNEL

There’s many applications or paid services available to store and visualize all of this data. However, many people I speak to recommend the Open Source ‘ELK stack’. Which is LogStash for parsing your logs, elastic search for enabling search on your data, and Kibana for visualization

No SSH == You’re doing it right

If you can get data from services and be aware of what they’re doing… and you *never* have to ssh into your services… well, then, you’re doing it right.

I read somewhere that some companies will actually block the ssh ports (22?) on their instances or their Amazon VPC to prove this point

THIRDCHANNEL


• Standardize your Deployment Process

Even if your services use different languages or frameworks, do your best to make sure everyone is using the same general approach to deployment.

THIRDCHANNEL

Core OS’s rocket

THIRDCHANNEL



• Small, Fast Deployments

THIRDCHANNEL

THIRDCHANNEL



• Small, Fast Deployments

• Continuous Deployment!

With continuous Deployment, your engineers should be releasing as soon as things are ready. Push a feature when it’s complete.

Beware “I have to deploy Service A before Service C or else Service B will break”

Wasn’t sure how to diagram this.

The only thing that should concern your team are the moments where a release has dependencies on other services… E.g. I have to deploy Service C before I deploy service A or else service B will break.

However, these cases should be RARE

THIRDCHANNEL

The day to day deployments, though, will be amazing. It may be a lot of work to get to the point where you can continuous deploy, but once you get there and are able to do it… well, it feels great.

In Summary:

So that’s really all I have, but before I stop, here is a brief summary

THIRDCHANNEL

Laws of the Microservice1. Be Independent

2. Communicate with your coworkers

3. Keep Data Close and Cache it Well

4. Do Not Sync

5. Monitor and Visualize

6. Work towards Continuous Deployment

Be Independent, but at the same time, Communicate with your coworkers. A well designed system requires constant communication

Keep your data close and cache it … but do not sync any data

Get some excellent monitoring in place

Reach for Continuous Deployment!

Thank You!

Any questions?Steve Pember @svpember

[email protected]

mailto:[email protected]

• Sad Dog: https://www.youtube.com/watch?v=Xw1C5T-fH2Y • Monolith: http://commons.wikimedia.org/wiki/File:Monolith_swfc_shanghai.jpg • Morpheus: www.desktopexchange.net/movie-pictures/matrix-wallpapers/ • Enterprise Crew: www.desktopexchange.net/movie-pictures/matrix-wallpapers/ • Mountain Climber: http://www.freshstartministries.com/just-one-more-step/ • Simpson Dev Ops: http://devops.com/blogs/devops-lessons-learned-the-devops-engineer/#!prettyPhoto • rocky stairs: http://pixgood.com/rocky-stairs.html • ELK Stack: https://www.elastic.co/products • Docker deployment Example Diagram: http://blog.risingstack.com/content/images/2015/01/codeship_ansible_docker.png

Images

https://www.youtube.com/watch?v=Xw1C5T-fH2Y

http://commons.wikimedia.org/wiki/File:Monolith_swfc_shanghai.jpg

http://www.desktopexchange.net/movie-pictures/matrix-wallpapers/

http://www.desktopexchange.net/movie-pictures/matrix-wallpapers/

http://www.freshstartministries.com/just-one-more-step/

http://devops.com/blogs/devops-lessons-learned-the-devops-engineer/#!prettyPhoto

http://pixgood.com/rocky-stairs.html

https://www.elastic.co/products

http://blog.risingstack.com/content/images/2015/01/codeship_ansible_docker.png

Advanced Microservices - Greach 2015

Software

Transcript of Advanced Microservices - Greach 2015