Download - Devoxx Morocco 2016 - Microservices with Kafka

Max. width

Min. height

Max. height

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51

Microservices with Kafka

László-Róbert ALBERT Dan BALESCU Casablanca • 02 November 2016

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51

Róbert

Dan

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51

No content below the grey line

ING Software Development Center

Bucharest

3

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


ING Software Development Center

Bucharest

4

IT 5,7% of GDP with 100k ppl

Agriculture 4,4% of GDP with 1.8 million ppl

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


5

FAULT

TOLERANCE

PE

RFO

RM

AN

CE

RELIABILITY SECURITY

AVAILABILITY HIGH

SCALABILITY

REPLICATION

CH

AN

GE

SELF HEALING

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


6

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


7

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


8

Account Manager

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


9

Account Manager

Core Banking

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


10

Account Manager

Core Banking

Transfer

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


11

Account Manager

Core Banking

Transfer

12

Source: http://www.slideshare.net/dberkholz/ how-microservices-are-redefining- modern-application-architecture

13

Source: http://www.slideshare.net/dberkholz/ how-microservices-are-redefining- modern-application-architecture

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


14

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


500 million messages / day

15

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


500 million messages / day 700 billion messages / day

16

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


500 million messages / day 700 billion messages / day 1.4 trillion messages / day

17

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


500 million messages / day 700 billion messages / day 1.4 trillion messages / day

18

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


Messaging Brokers Features

19

Kafka ActiveMQ RabbitMQ

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51



20

Support for multiple protocols - Advanced Message Queueing Protocol - Streaming Text Oriented Messaging Protocol - Message Queuing Telemetry Transport - Etc


ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51



21



Java Messaging System

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51



22




Java Authentication and Authorization Service

Simple Authentication and Security Layer

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51



23






Transactions

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51



24






Transactions

Filtering

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51



25






Transactions

Filtering

Persistence

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


26

Pe

rfo

rma

nce

Features

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


27

Messaging Brokers Performance

Message producing Message consumption

LinkedIn – Kafka: a Distributed Messaging System for Log Processing

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


28

Pe

rfo

rma

nce

Features

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


29

Usecases of Kafka

Consumer

Microservice A Microservice B Transport API

Producer

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


30

Usecases of Kafka

Consumer


Producer

Consumer

Producer

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


31

Usecases of Kafka


Producer

Consumer

Microservice C

Consumer

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


32

System Availability & Performance

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


33

System Availability & Performance

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


34

Scalability

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


35

Scalability

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


36

Usecases of Kafka

Microservice A ELK stack Kafka Consumer Kafka Appender

K

A

F

K

A

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


© ING 2016

37

Producer

// build a 'default' factory based on your-implicit-kafka-producer.properties

// - the properties file contains the list of brokers to connect to

ProducerFactory factory = ProducerFactory.builder().build();

// create a message producer with a Payload that sends the messages to topic ‘test’

Producer<Message<Payload>> producer =

factory.createMessageProducer("test", Payload.class);

// send a Message { header: Map<String, String>; payload: Payload }

producer.send(

Message.<Payload> builder()

.header("applicationId", "sample-producer")

.header("timestamp", now())

.payload(new Payload("/some/file/path"))

.build()

);

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


38

Producer

Native Kafka Producer

API

your-implicit-producer.properties

your-security.properties

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


39

Consumer // build a 'default' factory – your-implicit-kafka-consumer.properties

// - the properties contains the list of brokers from which it will receive messages

ConsumerFactory factory = ConsumerFactory.builder().build();

// obtain a consumer instance that can process Message<Payload> messages

Consumer<Message<Payload>> consumer = factory.createMessageConsumer(Payload.class);

// prepare the message (Message { header: Map<String, String>; payload: Payload } ) handler

ReceiverHandler<Message<Payload>> messageReceiver =

new ReceiverHandler<Message<Payload>>() {

@Override

public void onReceive(Message<Payload> message) {

System.out.printf("message received: %s%n", message);

}

};

// register the handler to receive messages from the topic named “test”

consumer.registerHandler("test", messageReceiver);

© ING 2016

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


40

Consumer

Native Kafka Consumer

API

your-implicit-consumer.properties

your-security.properties

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


41

KafkaAppender

ELK

KAFKA

Microservice A

Logback

File Appender

Kafka Appender

Kafka Producer Logstash

Fordwarder

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


43

Mutual TLS - your-security.properties

# The location of the key store file.

kafka.tls.keystore.location=/full/path/to/a/keystore.jks

# The store password for the key store file.

kafka.tls.keystore.password=some-good-password

# The file format of the key store file.

kafka.tls.keystore.type=JKS

# The password of the private key in the key store file

kafka.tls.key.password=the-private-keys-password

# The location of the trust store file.

kafka.tls.truststore.location=/path/to/truststore.p12

# The password for the trust store file.

kafka.tls.truststore.password=some-other-good-password

# The file format of the trust store file

kafka.tls.truststore.type=PKCS12

© ING 2016

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


44

Serialize

Producer

1

Metadata

N

Metadata

1 N

Transformer Serializer

Metadata Metadata

KAFKA 1 2 3

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


45

Filter

Microservice A

Kafka Producer

Microservice B

Kafka Consumer

KAFKA

Topic A

Kafka Consumer Kafka Producer

Metadata

Topic B

Filter

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


Transformer

46

Transformer

Microservice A Kafka

Producer

Microservice B Kafka

Consumer

KAFKA

Topic A Topic B

1

Kafka Consumer Kafka Producer

Metadata

1 2

1 2

2

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


47

Forward & Dispatch

Microservice A Kafka

Producer

Microservice B Kafka

Consumer

KAFKA

Topic A

Topic C

Kafka Consumer

Metadata

Topic B

Topic N

Kafka Producer

1

1

1 1

Microservice D Kafka

Consumer

Microservice Z Kafka

Consumer

Forward & Dispatch

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


48

Replay

0 1 2 3 i N N-1

beginning i end

Microservice A

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


49

Pe

rfo

rma

nce

Features

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


50

Pe

rfo

rma

nce

Features

FIL

TE

RIN

G

DIS

PA

TC

H

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51


51

Pe

rfo

rma

nce

Features FI

LT

ER

ING

D

ISP

AT

CH

Max. width

Min. height

Max. height

ING Orange

RGB= 255, 98, 0

ING Light Grey

RGB= 168, 168, 168

ING Indigo

RGB= 82, 81, 153

ING Sky

RGB= 96, 166, 218

Colour Guidelines

ING Fuchsia

RGB= 171, 0, 102

ING Lime

RGB= 208, 217, 60

ING Leaf

RGB= 52, 150, 81

ING Mid Grey

RGB= 118, 118, 118

Text Colour

RGB= 51, 51, 51

Use microservices with Kafka … … code ahead of the curve

László-Róbert ALBERT Dan BALESCU Casablanca • 02 November 2016

@danbalescu https://www.linkedin.com/in/albertlaszlorobert

http://www.twitter.com/ING_news

http://www.linkedin.com/company/ing