Asynchronous programming using CompletableFutures

Agenda

• What is asynchronous programming?

• Why should I care?

• How to use CompletableFuture for asynchronous calls?

• Does it worth it?

What is asynchronous programming?

Latency Numbers

0.5

25

100

0 20 40 60 80 100 120

ns

Main memory reference

Mutex lock/unlock

L1 Cache Reference

Latency Numbers

100

10,000

150,000

0 20000 40000 60000 80000 100000 120000 140000 160000

ns

Read 4K randomly from SSD

Send 1K over 1 Gbps

Main memory reference

Latency Numbers

150,000

250,000

500,000

10,000,000

0 2,000,000 4,000,000 6,000,000 8,000,000 10,000,000 12,000,000

ns

Disk seek

Round trip within samedatacenter

Read 1 MB sequentially frommemory

Read 4K randomly from SSD

The Hollywood Principle

Don’t call us,

we’ll call you

Why should I care?

Why should I care?

• Process scheduling in modern kernels is pretty mature

• Re-implementing it in user-space is an anti-pattern

• Java relies on kernel process scheduling

It’s about

• Controlling thread lifecycle

• The Nr. of threads and context switches

• I/O operation optimization

Non blocking I/O

Why should I map number of threads to number of connections?

It doesn’t make sense.

The Selector

Client 1

Client 2

Selector

Handler

Handler

Anatomy of a typical Java application

? ?

Your code goes here

How to use CompletableFuturefor asynchronous calls?

A Simple Example

Combining Asynchronous Calls

Using Executors Independently

We Have Exception Handling!

Tomcat 8

HelloService

Small I/O

Scenario 1

Tomcat 8

HelloService

125ms

Scenario 2

In-memory

Tomcat 8

HelloService

125ms

Scenario 3

Tomcat 8

HelloService

Small I/O

Scenario 4

4 threads

Pros & Cons

Pros

• Combinable executors

• Exception handling

• Fast enough execution

• Java language support

• Easy

Cons

• You have to know what you’re doing

• Can be blocking

• Can be slower if Executor scheduling is a bottleneck

Design Considerations

• Blocking 1000 threads because of service failure? Not a smart decision

• Queueing 1000 request-timeouts because of same reasons? Bad idea.• Bit better choice: less pressure on downstream services

• Choose threading and scheduling mechanism wisely• ForkJoinPool is most of the time a bad choice

• Monitor and purge request queues in case of a specific failure rate• Hystrix offers similar mechanism

Questions?

References

GitHub exampleshttps://github.com/gitaroktato/completablefuture-examples

CPU Schedulinghttps://www.cs.uic.edu/~jbell/CourseNotes/OperatingSystems/5_CPU_Scheduling.html

CompletableFuture & Spring Examplehttps://spring.io/guides/gs/async-method/

Non-blocking IO in Javahttps://dnhome.wordpress.com/2012/06/28/java-non-block-io-new-io/

References

CompletableFuture API Examples

https://www.slideshare.net/jpaumard/asynchronous-api-in-java8-how-to-use-completablefuture

Grizzly NIO frameworkhttps://grizzly.java.net/documentation.html

Scalable IO in Javahttps://www.slideshare.net/giridhar510/scalable-io-in-java

Non-blocking IO in Javahttps://dnhome.wordpress.com/2012/06/28/java-non-block-io-new-io/

RxJava – Multithreadinghttps://praveer09.github.io/technology/2016/02/29/rxjava-part-3-multithreading/