Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox

ConcursusEvent Sourcing Evolved

Introductions

Dominic Fox

Twitter: @dynamic_proxy

Email: [email protected]

Github: http://github.com/poetix

Concursus Github: http://github.com/opencredo/concursus

https://twitter.com/dynamic_proxy

mailto:[email protected]

http://github.com/poetix

http://github.com/poetix

http://github.com/opencredo/concursus

Agenda

1. From Presence to Presents

2. “Write First, Reason Later”

3. Three processing schedules

4. Concursus

5. Future Directions




4. Concursus


Agenda

"I have told people over and over and over again, don't write a

CQRS framework…I can basically guarantee you that it will be

abandonware within one year, like every other one has become.”

- Greg Young, inventor of the term “CQRS”

Why Concursus?

https://youtu.be/LDW0QWie21s

Agenda




4. Concursus


From Presence to Presents


Aggregate services together into a single container or integrate

them through a single enterprise service bus...

Manage all of our data through a single relational database

schema with global constraints...

Use distributed locks and transactions to make a distributed

system behave as if it were one single system with a global

transactional semantics...

From Presence to Presents“Here the Microservice can become an escape route from reality. Within each Microservice, we can

live on a safe island of determinism and strong consistency — an island where we can live happily

under the illusion that time and the present is absolute.

However, as soon as we exit the boundary of the Microservice we enter a wild ocean of non-

determinism—the world of distributed systems, which is a very different world. You have probably

heard that building distributed systems is hard. It is. That being said it is also the world that gives

us solutions for resilience, elasticity, isolation amongst others. At this point, what we need to do is

not to run back to the monolith, but instead learn how to apply and use the right set of principles,

abstractions and tools in order to manage it.”

- Jonas Bonér, Reactive Microservices Architecture, p. 28-29

Agenda




4. Concursus


“Write First, Reason Later”


Scene from Endgame, by Samuel Beckett

Agenda




4. Concursus


Three Processing Schedules

1.Transient

2.Durable

3.Persistent

Three Processing Schedules

1.Transient - what happens

2.Durable - what’s happening

3.Persistent - what happened

Agenda




4. Concursus


Domain Model: Events

aggregateType: lightbulbaggregateId: 69016fb5-1d69-4a34-910b-f8ff5c702ad9

eventTimestamp: 2016-03-31T10:31:17.981Zparameters: { “wattage”: 60 }



eventTimestamp: 2016-03-31T10:36:42.171Zparameters: { “location”: “hallway”}



eventTimestamp: 2016-03-31T10:36:42.171ZprocessingTimestamp: 2016-03-31T10:36:48.3904Zparameters: { “location”: “hallway”}


Domain Model: Summary

Every Event occurs to an Aggregate, identified by its type and id.

Every Event has an eventTimestamp, generated by the source of the event.

An Event History is a log of Events, ordered by eventTimestamp, with an additional processingTimestampwhich records when the Event was captured.

Network

Event sources

Event processors

Events arrive:• Partitioned• Interleaved• Out-of-order

Processing Model: Ordering

Log is:• Partitioned by aggregate id• Ordered by event timestamp

Processing Model: Ordering

CREATE TABLE IF NOT EXISTS concursus.Event (aggregateType text,aggregateId text,eventTimestamp timestamp,streamId text,processingId timeuuid,name text,version text,parameters map<text, text>,characteristics int,PRIMARY KEY((aggregateType, aggregateId), eventTimestamp,

streamId)) WITH CLUSTERING ORDER BY (eventTimestamp DESC);

Cassandra Schema

CassandraEvent Store

RabbitMQ Topic

DownstreamprocessingLog

events

Publish events

Cassandra & AMQP


RabbitMQ Topic

Downstreamprocessing

out-of-order events

ordered query results

Cassandra & AMQP


Kafka Topic

Downstreamprocessing

Event store listener

Publish events

Log events

Cassandra & Kafka

Processing Model: Summary

Events arrive partitioned, interleaved and out-of-order.

Events are sorted into event histories by aggregate type and id.

Events are sorted within event histories by event timestamp, not processing timestamp.

Event consumers need to take into account the possibility that an event history may be incomplete at the time it is read – consider using a watermark to give incoming events time to “settle”.

Programming Model: Core Metaphor

Consumer<Event>

Programming Model: Core Metaphor

You give me a Consumer<Event>, and I send Events to it one at a time:

Emitting Events

I implement Consumer<Event>, and handle Events that are sent to me.

Handling Events

Event-handling middleware is a chain of Consumer<Event>s that transforms, routes, persists and dispatches events. A single event submitted to this chain may be:

■ Serialised to JSON

■ Written to a message queue topic

■ Retrieved from the topic and deserialised

■ Persisted to an event store (e.g. Cassandra)

■ Published to an event handler which maintains a query-optimised view of part of the system

■ Published to an event handler which maintains an index of aggregates by event property values (e.g. lightbulbs by wattage)

Event-Handling Middleware

Java 8 Mapping

Querying and Replaying

Querying and Collecting

Kotlin Mapping

■ Event type and middleware implementing Consumer<Event>provide basic mechanics for creating, processing and handling events.

■ Java 8 mappings provide a convenient, type-safe method-mapping wrapper around these mechanisms.

■ Kotlin mappings provide an alternative wrapper based on immutable classes.

■ Other mappings are possible – Scala, Clojure, Java with POJOs…

Programming Model: Summary

Agenda




4. Concursus


Future Directions

1. Stable Kafka integration

Future Directions


2. Avro integration

Future Directions


2. Avro integration

3. End-to-end async programming model

Questions?

Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox

Technology

Transcript of Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox