CQRS & event sourcing in the wild

CQRS & EVENT SOURCING IN THE WILD

MICHIEL ROOK

➤ Java, PHP & Scala developer

➤ Consultant, trainer, speaker

➤ Dutch Web Alliance

➤ make.io

➤ Maintainer of Phing

➤ @michieltcs

RAISE YOUR HAND

IF YOU HAVE

heard about CQRS / Event Sourcing

RAISE YOUR HAND

IF YOU HAVE


followed a tutorial, built a hobby project

RAISE YOUR HAND

IF YOU HAVE


followed a tutorial, built a hobby project

used it in production

RAISE YOUR HAND

IF YOU HAVE

TOPICS

➤ Quick recap

➤ Replays and rebuilds

➤ Event versioning

➤ Concurrency

➤ Scale

QUICK RECAP

' Event Sourcing ensures that all changes to application state are stored as a sequence of events.

-Martin Fowler

ACTIVE RECORD VS. EVENT SOURCING

Account number

Balance12345678 �€ 50,00

... ...

Money WithdrawnAccount number 12345678

Amount �€ 50,00

Money DepositedAccount number 12345678

Amount �€ 100,00

Account CreatedAccount number 12345678

COMMANDS TO EVENTS

Deposit MoneyAccount number 12345678

Amount �€ 100,00


Amount �€ 100,00

class DepositMoney { public $accountNumber; public $amount; }

class MoneyDeposited { public $accountNumber; public $amount; public $timestamp; }

function depositMoney(DepositMoney $command) { $this->apply(new MoneyDeposited($command->accountNumber, $command->amount, date())); }

AGGREGATES

class BankAccount { public $accountNumber; public $balance; // command handlers... public function applyAccountCreated(AccountCreated $event) { $this->accountNumber = $event->accountNumber; $this->balance = 0; } public function applyMoneyDeposited(MoneyDeposited $event) { $this->balance += $event->amount; } public function applyMoneyWithdrawn(MoneyWithdrawn $event) { $this->balance -= $event->amount; } }

AGGREGATE STATE

Account number

Balance12345678 �€ 0,00

Money WithdrawnAccount number 12345678

Amount �€ 50,00


Amount �€ 100,00

Account CreatedAccount number 56789012

Account number

Balance12345678 �€ 100,00

Account number

Balance12345678 �€ 50,00

CQRS

➤ Command Query Responsibility Segregation

➤ Separate writes (events) and reads (UI)

➤ Optimize for reads!

Domain

Event Bus

Event Handlers

Command

Repository

Data Layer

Database Database

Event Store

commands

events

events

queries DTOs

Aggregates

PROS AND CONS

➤ Domain fit

➤ Testing

➤ Audit trail

➤ Scalability

➤ Complexity

➤ Library support / maturity

DISCLAIMER

REPLAYS AND REBUILDS

PROJECTIONS AND READ MODELS

User Registered

User Registered

User Unregistered Number of active users?


User Registered

User Deactivated

User Reactivated

Number of active users?User Registered

User Unregistered


User Registered Event HandlerNumber of

active users +1

User Unregistered Event HandlerNumber of

active users -1


Events Event Handler(s) Storage

ELASTICSEARCH

class CompanyRegistered { public $companyId; public $name; public $street; public $city; } function handleCompanyRegistered(CompanyRegistered $event) { $this->elasticsearch->index([ 'index' => 'companies', 'type' => 'company', 'id' => $event->companyId, 'body' => [ 'name' => $event->name, 'address' => [ 'street' => $event->street, 'city' => $event->city ] ] ]); }

READ MODEL UPDATES

➤ New type

➤ New structure

➤ Based on existing events

➤ Generate from scratch?

REBUILDING

Stop application

Remove old read model

Loop over events

Apply to read model

Start application

ZERO DOWNTIME

Loop over existing events

Apply to new read

model

Apply queued events

Start using new read

model

New events Queue

ELASTICSEARCH: MAPPING CHANGE

Create index with

new mapping

Reindex existing

documents

Apply queued events

Start using new index

New events Queue

CHALLENGE: LONG RUNNING REBUILDS

➤ Alternatives

➤ Distributed

➤ In memory

➤ Partial

➤ Background

CHALLENGE: SIDE EFFECTS

User RegisteredUser Id 123abc

Email Address [email protected] Handler

Exclude during replays!

CHALLENGE: TRANSACTIONS

Event Handler

Event Handler

Event

Event Handler

Event Handler ?

CHALLENGE: EVENTUAL CONSISTENCY

➤ Asynchronous event handlers

➤ Reads eventually return the same value

➤ Compare with ACID

EVENT VERSIONING

DILEMMA

➤ New business requirements

➤ Refactoring

➤ New view on events

NEW EVENTS / VERSIONS

➤ No longer relevant

➤ Renamed

➤ Additional or renamed field(s)

➤ Too coarse, too fine

SUPPORT YOUR LEGACY?

➤ Events are immutable

➤ Correct (incorrect) old events with new events

UPCASTING

Event Store

UserRegistered_V1

Upcaster

UserRegistered_V2

Event Handler

UPCASTING

class UserRegistered_V1 { public $userId; public $name; public $timestamp; } class UserRegistered_V2 { public $userId; public $name; public $date; } function upcast($event): array { if (!$event instanceof UserRegistered_V1) { return []; } return [ new UserRegistered_V2($event->userId, $event->name, $event->timestamp->format("Y-m-d")) ]; }

REWRITING HISTORY

Load (subset of) events

Deserialize

Modify

Serialize

Save/replace

THINGS TO BE AWARE OF

Upcasting

➤ Performance

➤ Complexity

➤ Existing projections not automatically updated

Rewriting events

➤ Running code that depends on old structure

➤ Breaking serialization

➤ Changing wrong events

CONCURRENCY

CONCURRENT COMMANDS

Withdraw MoneyAccount number 12345678

Amount �€ 50,00


Amount �€ 100,00

?

PESSIMISTIC LOCKING


Amount �€ 50,00


Amount �€ 100,00

Account number

Balance12345678 �€ 100,00

Account number

Balance12345678 �€ 50,00

wait for lock

lock

OPTIMISTIC LOCKING


Amount �€ 50,00version 1


Amount �€ 100,00version 1

Account number

Balance12345678 �€ 100,00

version 2

ConcurrencyException

PERFORMANCE

➤ Server

➤ Database

➤ Framework

➤ Language

STORAGE

➤ #events

➤ #aggregates

➤ #events_per_aggregate

➤ Serializer

➤ Speed

➤ Payload size

➤ Costs

SNAPSHOTS

Events1 Account Created2 Money Deposited3 Money Withdrawn4 Money Deposited

SNAPSHOT5 Money Withdrawn

Events1 Account Created2 Money Deposited3 Money Withdrawn4 Money Deposited5 Money Withdrawn

SHARDING

➤ Aggregate Id, Type, Event Timestamp, ...

➤ Rebalancing

➤ Distribution

ARCHIVING EVENTS

➤ Reduce working set

➤ Inactive / deleted aggregates

➤ Historic / irrelevant events

➤ Cheaper storage

FRAMEWORK COMPARISON

Framework Upcasting Snapshots Replaying

Broadway (PHP) No (PR) No (PR) Not in core

Prooph (PHP) MessageFactory Yes, triggers on

event countExample code,

off line

Axon (Java/Scala)

Upcaster / UpcasterChain

Yes, triggers on event count

Yes, ReplayingCluster

Akka Persistence (Java/Scala)

Event Adapter Yes, decided by actor Yes

BROADWAY VS PROOPH

class TodoItem extends EventSourcedAggregateRoot { private $itemId; public function __construct(PostTodo $command) { $this->apply(new TodoPosted($command->getItemId())); } public function getAggregateRootId() { return $this->itemId; } public function applyTodoPosted(TodoPosted $event) { echo "Posted: " . $event->getItemId() . "\n"; } }

BROADWAY VS PROOPH

class TodoItem extends AggregateRoot { private $itemId; public function __construct(PostTodo $command) { $this->recordThat(new TodoPosted($command->getItemId())); } protected function aggregateId() { return $this->itemId; } public function whenTodoPosted(TodoPosted $event) { echo "Posted: " . $event->getItemId() . "\n"; } }

THANK YOU!

CQRS & event sourcing in the wild

Software

Transcript of CQRS & event sourcing in the wild