Micro services or monolith in a reactive low latency java applications

34
Peter Lawrey CEO of Higher Frequency Trading New York 2015 Micro-services or monolith in a reactive low latency Java applications

Transcript of Micro services or monolith in a reactive low latency java applications

Page 1: Micro services or monolith in a reactive low latency java applications

Peter LawreyCEO of Higher Frequency TradingNew York 2015

Micro-services or monolith in a reactive low latency Java applications

Page 2: Micro services or monolith in a reactive low latency java applications

Peter Lawrey

Java Developer/Consultant for investment banks, hedge fund and trading firms for 7 years, 16 years in Java, 23 years in IT.Most answers for Java and JVM on stackoverflow.comFounder of the Performance Java User’s Group.Architect of Chronicle SoftwareJava Champion

Page 3: Micro services or monolith in a reactive low latency java applications

Chronicle SoftwareHelp companies migrate to high performance Java code.Sponsor open source projects https://github.com/OpenHFTLicensed solutions Chronicle-Enterprise and Chronicle-FixOffer one week proof of concept workshops, advanced Java training, consulting and bespoke development.

Page 4: Micro services or monolith in a reactive low latency java applications
Page 5: Micro services or monolith in a reactive low latency java applications

Enterprise support for custom DSLs

We have a micro-services container which has one thread.A service or strategy can;be written in a language which can be turned into Java and compiled in memory.be deployed to an individual container/CPU pinned thread upgraded while running.run a strategy in test mode before switching control.be stopped.have it’s state examined or altered remotely.Micro-second latencies persisting every message.

Page 6: Micro services or monolith in a reactive low latency java applications

AgendaMicro-Services and Reactive, what’s new?When does it make more sense to have micro-services or a monolith?How can we get micro-second response times in a micro-services architecture?How can micro-services help trading systems and real time applications to be easier to deploy and manage?How can reactive streams help and what are the alternatives?How can we orchestrate and monitor these services?

Page 7: Micro services or monolith in a reactive low latency java applications

Micro-Services and Reactive, what’s new?

These are hot topics which attempt to bring together a collection of best practices in different domains. These are not all or nothing. You can implement a sub-set of these best practices, and most likely you are already doing this to some degree.

Page 8: Micro services or monolith in a reactive low latency java applications

UNIX Philosophy

Small is beautiful.Make each program do one thing well.Build a prototype as soon as possible.Choose portability over efficiency.Store data in flat text files.Use software leverage to your advantage.Use shell scripts to increase leverage and portability.Avoid captive user interfaces.Make every program a filter.https://en.wikipedia.org/wiki/Unix_philosophy#Mike_Gancarz:_The_UNIX_Philosophy

Page 9: Micro services or monolith in a reactive low latency java applications

Old Micro-Services

Before there was HTTP, there wasDaytime, uptime.DNSNTPFTPSyslogNFSSMTP

Page 10: Micro services or monolith in a reactive low latency java applications

Monolith vs Micro-Services

The main difference is in deployment. Monolith means deploying everything at once. Micro-Services supports (re)deploying portions of a running system.

Monoliths are simpler up to a point. If your solution is simple enough, you don’t need more than one service.

Micro-Services can scale better, esp across multiple machines. You are more likely to be forced to follow best practices.

Page 11: Micro services or monolith in a reactive low latency java applications

Micro-Services Best practices

Modelled around the business domain.Culture of automation.Hide implementation details.Decentralize all things.Deployed independently, and monitored independently.Isolated failures.

Micro-Services forces you to think about these issues, however in a monolith, this may require more discipline.

Page 12: Micro services or monolith in a reactive low latency java applications

Micro-Services are a …

Way to introduce best practices for distributed systemsreplacement monoliths which should have been a distributed system.if it’s not distributed already, chances are it should continue to be a monolith.

Page 13: Micro services or monolith in a reactive low latency java applications

Monoliths are a …

Easier to unit and functional test.Easier to debug.Easier to run on one machine.

To get the best of both worlds, make sure you micro-services can be run as a monolith. This allows you to configure your service components to run in a single thread, multiple thread, multiple processes or multiple machines which suits your changing requirements.

Page 14: Micro services or monolith in a reactive low latency java applications

Unit Testing High Availability

Create systems which communicate via asynchronous calls.Allow them to be configured as distributed components or in a single thread.Create a unit tests where the listener of your primary service is your secondary for the same service.

This allows you to unit test your redundant system behaves correctly and is easy to debug when it doesn’t.

Page 15: Micro services or monolith in a reactive low latency java applications

Reactive Streams

an initiative to provide a standard for asynchronous stream processing with non-blocking back pressure.follows the Reactive Manifesto in always being ready to handle any requirements.assumes the producer exists to feed the consumers.producer is dependent on the slowest consumer.coupled testing between the producer and consumer.

Designed to handle high workloads which have the potential to overwhelm your system.

Page 16: Micro services or monolith in a reactive low latency java applications

Chronicle Queue

A solution to provide a non blocking asynchronous stream processing without back pressure via an unbounded queue.follows the Reactive Manifesto in always being ready to handle any requirements.assumes the producer is consumer insensitive.producer and consumer are tested independently.assumes you benefit from a record of every message.

Designed to handle high workloads where a slow consumer cannot or should not slow the producer. E.g. an exchange.

Page 17: Micro services or monolith in a reactive low latency java applications

What is low latency?

The term “low latency” can applied to a wide range of situations. A broad definition might be;Low latency means you have a view on how much the response time of a system costs your business.

In this talk I will assume; Low latency means you care about latencies you can only measure as even the worst latencies are too fast to see.

Page 18: Micro services or monolith in a reactive low latency java applications

What is an ultra low GC?

In a generational collector you have multiple generations of objects.Small to medium sized objects are created in the Eden space.When your Eden space fills up you trigger a Minor Collection.

So how big can your Eden space be and still have Compressed OOPS?

Page 19: Micro services or monolith in a reactive low latency java applications

Where does the Eden space fit in

Java 8 memory layout.

Page 20: Micro services or monolith in a reactive low latency java applications

How can we get micro-second response times in a micro-services architecture?

Using small web services can lead to high latencies. They use HTTP/REST with JSON and the more hops between services, the greater the overhead.

An alternative is to use low latency messaging. This can reduce latency between threads and processes on the same machine to as low at 1 micro-second, and between machines to single digit micro-seconds.

Page 21: Micro services or monolith in a reactive low latency java applications

Low latency messaging

Tibco FTL – RDMA and shared memory.Aeron – UDP and shared memory.Chronicle Queue / Map – persisted shared memory and TCP.

Each of these solutions work in the single digit micro-seconds between machines and sub micro second on the same machine.

Page 22: Micro services or monolith in a reactive low latency java applications

Low latency micro-services

Passing a message from a producer to a worker and a worker to a consumer, we get a typical latency of 4.5 micro-seconds on a laptop.

We are looking to improve performance to be less than 1 micro-second per hop for larger messages.

Page 23: Micro services or monolith in a reactive low latency java applications

Chronicle Demo

Page 24: Micro services or monolith in a reactive low latency java applications

Low latency serialization

The cost of serialization/deserialization can be greater than the message itself.SBE – schema based binary protocolChronicle Bytes – binary protocol.Chronicle Wire – meta data binary/text protocol. E.g. JSON.Protobuf, kryo, Cap’n proto – cross platform serialization.

A key advantage of SBE and Chronicle is re-use of objects in deserialization to avoid creating garbage.

Page 25: Micro services or monolith in a reactive low latency java applications

A low latency API which uses Lambdas

Timings are in micro-seconds with JMH.

* Data was read/written to native memory.

Wire Format Bytes 99.9 %tile 99.99 %tile 99.999 %tile worst

JSONWire 100* 3.11 5.56 10.6 36.9

Jackson 100 4.95 8.3 1,400 1,500

Jackson + Chronicle-Bytes 100* 2.87 10.1 1,300 1,400

BSON 96 19.8 1,430 1,400 1,600

BSON + Chronicle-Bytes 96* 7.47 15.1 1,400 11,600

BOON Json 100 20.7 32.5 11,000 69,000

"price":1234,"longInt":1234567890,"smallInt":123,"flag":true,"text":"Hello World!","side":"Sell" 

Page 26: Micro services or monolith in a reactive low latency java applications

A resizable buffer and a Wire format// Bytes which wraps a ByteBuffer which is resized as needed. Bytes<ByteBuffer> bytes = Bytes.elasticByteBuffer();// YAML based wire format Wire wire = new TextWire(bytes);

// or a binary YAML based wire formatBytes<ByteBuffer> bytes2 = Bytes.elasticByteBuffer(); Wire wire2 = new BinaryWire(bytes2);

// or just data, no meta data. Bytes<ByteBuffer> bytes3 = Bytes.elasticByteBuffer(); Wire wire3 = new RawWire(bytes3);

Page 27: Micro services or monolith in a reactive low latency java applications

Low latency API using Lambdas (Wire)

message: Hello World number: 1234567890 code: SECONDS price: 10.5

wire.read(() -> "message").text(this, (o, s) -> o.message = s) .read(() -> "number").int64(this, (o, i) -> o.number = i) .read(() -> "timeUnit").asEnum(TimeUnit.class, this, (o, e) -> o.timeUnit = e) .read(() -> "price").float64(this, (o, d) -> o.price = d);

wire.write(() -> "message").text(message) .write(() -> "number").int64(number) .write(() -> "timeUnit").asEnum(timeUnit) .write(() -> "price").float64(price);

To write a message

To read a message

Page 28: Micro services or monolith in a reactive low latency java applications

A resizable buffer and a Wire format

message: Hello World number: 1234567890 code: SECONDS price: 10.5

In the YAML based TextWire

Binary YAML Wire

message: Hello World number: 1234567890 code: SECONDS price: 10.5

00000000 C7 6D 65 73 73 61 67 65 EB 48 65 6C 6C 6F 20 57 ·message ·Hello W 00000010 6F 72 6C 64 C6 6E 75 6D 62 65 72 A3 D2 02 96 49 orld·num ber····I 00000020 C4 63 6F 64 65 E7 53 45 43 4F 4E 44 53 C5 70 72 ·code·SE CONDS·pr 00000030 69 63 65 90 00 00 28 41 ice···(A

Page 29: Micro services or monolith in a reactive low latency java applications

Lambdas and Junit tests

message: Hello World number: 1234567890 code: SECONDS price: 10.5

To read the data

To check the data without a data structure

wire.read(() -> "message").text(this, (o, s) -> o.message = s) .read(() -> "number").int64(this, (o, i) -> o.number = i) .read(() -> "timeUnit").asEnum(TimeUnit.class, this, (o, e) -> o.timeUnit = e) .read(() -> "price").float64(this, (o, d) -> o.price = d);

wire.read(() -> "message").text("Hello World", Assert::assertEquals) .read(() -> "number").int64(1234567890L, Assert::assertEquals) .read(() -> "timeUnit").asEnum(TimeUnit.class, TimeUnit.SECONDS,Assert::assertEquals) .read(() -> "price").float64(10.5, (o, d) -> assertEquals(o, d, 0));

Page 30: Micro services or monolith in a reactive low latency java applications

Interchanging Enums and Lambdas

message: Hello World number: 1234567890 code: SECONDS price: 10.5

Enums and lambdas can both implement an interface.Wherever you have used a non capturing lambda you can also use an enum.enum Field implements WireKey { message, number, timeUnit, price;}

@Overridepublic void writeMarshallable(WireOut wire) { wire.write(Field.message).text(message) .write(Field.number).int64(number) .write(Field.timeUnit).asEnum(timeUnit) .write(Field.price).float64(price);}

Page 31: Micro services or monolith in a reactive low latency java applications

When to use Enums

message: Hello World number: 1234567890 code: SECONDS price: 10.5

Enums have a number of benefits.They are easier to debug.The serialize much more efficiently.Its easier to manage a class of pre-defined enums to implement your code, than lambdas which could be any where

Under https://github.com/OpenHFT/Chronicle-Engine search for MapFunction and MapUpdater

Page 32: Micro services or monolith in a reactive low latency java applications

When to use Lambdas

message: Hello World number: 1234567890 code: SECONDS price: 10.5

Lambdas have a number of benefits.They are simpler to writeThey support generics betterThey can capture values.

Page 33: Micro services or monolith in a reactive low latency java applications

Where can I try this out?

message: Hello World number: 1234567890 code: SECONDS price: 10.5

The source for these micro-benchmarks are test are availablehttps://github.com/OpenHFT/Chronicle-Queue

Chronicle Engine with live subscriptionshttps://github.com/OpenHFT/Chronicle-Engine

Page 34: Micro services or monolith in a reactive low latency java applications

Q & A

Peter Lawrey@PeterLawrey

http://chronicle.softwarehttp://vanillajava.blogspot.com