JavaZone 2013 - Datomic vs EventStore

Immutable data stores

onsdag 25 september 13

Immutable data stores

Only add, never remove or change

Can retrieve old values

Source code

Version controlled

Keep all versions

Example domain

Rock - Paper - Scissors

http://rock-paper-scissors.com/

The future Facebook of Rock Paper Scissors

Millions of users

Many games per user

Playing the game

Player A

Player B

Server

Playing the game

Player A

Player B

Server

Game 123

Playing the game

Player A

Player B

Serverpaper

Game 123

Playing the game

Player A

Player B

Serverpaper

Player B: paper

Game 123

Playing the game

Player A

Player B

Server

Player B: paper

Player A: rock

Game 123

Playing the game

Player A

Player B

Server

Player B: paper

Player A: rock

Game 123

Game 123winner: Player Bloser: Player A

Playing the game

Player A

Player B

Server

Player B: paper

Player A: rock

Game 123

Game 123winner: Player Bloser: Player A

CREATED WAITING

GAME WON

GAME TIED

any move

other move(victory)

other move(tie)

Entities

Player+ id+ name+ email

Entities

Game+ id+ state+ players+ moves+ winner+ loser

Entities

Game+ id+ state+ players+ moves+ winner+ loser Move

+ move+ player

Actions

Create game

Make move

Get game result

POST /games

POST /games/123 move=rock

GET /games/123

Actions

Create game

Make move

Get game result

POST /games

POST /games/123 move=rock

GET /games/123

1. Form (make move)

2. Waiting

3. Game result

4. Error

Event Sourcingwith

Event Store

Event Sourcing

Events that have happened

Ordered in time

Source of truth

http://geteventstore.com

Runs on .NET and Mono

Free (store) or Commercial (HA)

APICustom TCP (c#)

ATOM over HTTP (xml/json)

FeaturesEvent streams, projections, generate new events (CEP)

€1500/year

Event example

Server EventStore

Event example

Server

CreateGameCommand{:game 123 :creator "player-1"}

EventStore

Event example

Server

GameCreatedEvent{:game 123 :creator "player-1"}

EventStore

Event example

Server

MakeMoveCommand{:game 123 :player "player-1" :move "rock"}

MoveMadeEvent{:game 123 :player "player-1" :move "rock"}

EventStore

Event example

Server

MakeMoveCommand{:game 123 :player "player-2" :move "paper"}

MoveMadeEvent{:game 123 :player "player-1" :move "rock"}

MoveMadeEvent{:game 123 :player "player-2" :move "paper"}GameWonEvent{:game 123 :winner "player-2" :loser "player-1"}

EventStore

GameCreatedEvent{:game 123, :creator "player-1", :time 1} MoveMadeEvent{:game 123, :player "player-1", :move "rock", :time 2}

Current state?

CREATED WAITING

GAME WON

GAME TIED

any move

if victory

other move

if tie

Current state?

CREATED WAITING

GAME WON

GAME TIED

any move

if victory

other move

if tie

GameCreatedEvent{:game 123, :creator "player-1", :time 1} MoveMadeEvent{:game 123, :player "player-1", :move "rock", :time 2}GameCreatedEvent

Current state?

CREATED

CREATED WAITING

GAME WON

GAME TIED

any move

if victory

other move

if tie

Current state?

WAITING

MoveMadeEvent

Aggregates

Scope of consistency when handling a command

Move+ id+ move+ player

Aggregates as Event Streams

event1 GameCreatedEvent{:game 1...}event2 MoveMadeEvent{:game 1....}

stream = game-1version = 2

event1 GameCreatedEvent{:game 2...}event2 MoveMadeEvent{:game 2....}event3 MoveMadeEvent{:game 2....}event4 GameWonEvent{:game 2....}

curl "http://127.0.0.1:2113/streams/game-1" -d @eventdata.json -H "Content-Type:application/json" -H "ES-ExpectedVersion: 2"

Projections

GameCreatedEvent{:game 1...}MoveMadeEvent{:game 1....}GameCreatedEvent{:game 2...}MoveMadeEvent{:game 2....}MoveMadeEvent{:game 2....}GameWonEvent{:game 2....}

Projections

for (Event events : allEvents) { if (event instanceof GameCreatedEvent) { created++; }}return created;

Projections

GameCreatedEvent{:game 1...}MoveMadeEvent{:game 1....}GameCreatedEvent{:game 2...}MoveMadeEvent{:game 2....}MoveMadeEvent{:game 2....}GameWonEvent{:game 2....} created = 2

Projections

GameCreatedEvent{:game 1...}MoveMadeEvent{:game 1....}GameCreatedEvent{:game 2...}MoveMadeEvent{:game 2....}MoveMadeEvent{:game 2....}GameWonEvent{:game 2....} created = 2GameCreatedEvent{:game 3...}

Projections

(state, event) -> { if (event instanceof GameCreatedEvent) { return state++; } rerturn state;}

created = 2GameCreatedEvent{:game 3...}

Projections in JavascriptfromAll().when({ $init: function () { return { created: 0, completed:0 }; },

GameCreatedEvent: function(s, e) { s.created++; return s; },

GameWonEvent: function(s, e) { s.completed++; return s; },

GameTiedEvent: function(s, e) { s.completed++; return s; }

/projections/games-counter{created: 15,completed: 10

External Projections

events

Business logic

events events

Business logic Projections

Consistent

Available

events events

Business logic Projections

Commands Queries

Consistent

Available

events events

System with EventStore

Command Handler

1.command Command Handler

1.command Event Store

Command Handler

2.get events

Command Handler

2.get events

3.add events

Command Handler

2.get events

3.add events

4.updateprojections

Command Handler

2.get events

3.add events

4.updateprojections

5.get eventsor projections

Some service

Command Handler

2.get events

6.update state

3.add events

4.updateprojections

Some service

Command Handler

2.get events

6.update state

3.add events

4.updateprojections

7.query

Some service

Datomic

http://www.datomic.com/

Runs on JVM

Free (3 peers, local storage) or Commercial (HA, distrib)

FeaturesACID transactions, declarative query engine

€2300

Something known to have happened or existed

2004-10-01 the email of Jan was ”jan.kronquist@jayway.se”

2007-01-01 the email of Jan was ”jan.kronquist@jayway.com”

Atomic fact (datom)

Entity

Attribute

Atomic fact (datom)

Entity

Attribute

The person Jan Kronquist

”jan.kronquist@jayway.se”

2004-10-01

Data perception

Add facts

Add more facts

Even more facts

Data perception

Add facts

Add more facts

Even more facts

Data perception

Add facts

Add more facts

Even more facts

Data perception

Add facts

Add more facts

Even more facts

Data perception

Add facts

Add more facts

Even more facts

Data perception

Add facts

Add more facts

Even more facts

Data perception

Add facts

Add more facts

Even more facts

Database deconstructed

TransactorConsistency by serializing transactions

PeerYour code + Query engine

Storage serviceKey/value store

Scalable reads and writes

eg DynamoDB, InfiniSpan, Riak + ZooKeeper, SQL(!)

System with Datomic

Transactor

Distributed Storage Service

Peer A Peer A

System with Datomic

Transactor

Peer A Peer A

1.command

System with Datomic

Transactor

Peer A Peer A

2.read and cache

1.command

System with Datomic

Transactor

Peer A

3.transact

Peer A

2.read and cache

1.command

System with Datomic

Transactor

Peer A

3.transact

Peer A

4.write

2.read and cache

1.command

System with Datomic

Transactor

Peer A

3.transact

Peer A

4.write

2.read and cache

Peer B

5.notify

1.command

Datomic example

Peer Transactor

Datomic example

Transactor

Datomic example

[123 :creator "player-1" T1][123 :state :created T1]

Transactor

Datomic example

Transactor

["player-1" :email "player@gmail.com" T0]

Datomic example

Transactor

Datomic example

Transactor

Datomic example

Transactor

Datomic example

MakeMoveCommand{:game 123 :player "player-1" :move :rock}

[123 :creator "player-1" T1][123 :state :created T1][123_1 :move :rock T2][123_1 :player "player-1" T2][123 :moves 123_1 T2][123 :state :waiting T2]

Transactor

Datomic example

MakeMoveCommand{:game 123 :player "player-2" :move :paper}

[123 :creator "player-1" T1][123 :state :created T1][123_1 :move :rock T2][123_1 :player "player-1" T2][123 :moves 123_1 T2][123 :state :waiting T2][123_2 :move :paper T3][123_2 :player "player-2" T3][123 :moves 123_2 T3][123 :state :won T3][123 :winner "player-2" T3][123 :loser "player-1" T3]

Transactor

Aggregates?

Only by convention

cas (compare and set)

isComponent on relation

Move+ id+ move+ player

What to find

Where clauses [entity attribute value]

Implicit joins

Call arbitrary function

Query language

Query example - basic

[:find ?email :where [_ :email ?email]]

Query example - basic

[:find ?email :where [_ :email ?email]]

[["jan.kronquist@jayway.com] ["player-1@gmail.com"] ["player-2@gmail.com"]]

Query example - aggregation

[:find ?state (?count game) :where [?game :state ?state]]

[[:created 4] [:waiting 1] [:won 7] [:tied 3]]

inProgess = created + waiting

fromAll().when({ $init: function () { return { created: 0, completed:0 }; },

GameTiedEvent: function(s, e) { s.completed++; return s; }});

/projections/games-counter{

created: 15,completed: 10

Query example - join

[:find ?email (count ?game) :where [?player :email ?email] [?game :winner ?player]]

Query at any time

(q ’[:find ?email :where [_ :email ?email]] db)

Query at any time

(q ’[:find ?email :where [_ :email ?email]] db)

(q ’[:find ?email :where [_ :email ?email]] (as-of db #inst "2013-09-02"))

Query over time

(q ’[:find ?e (count ?tx) :where [?e :email _ ?tx]] db)

Query over time

(q ’[:find ?e (count ?tx) :where [?e :email _ ?tx]] db)

(q ’[:find ?e (count ?tx) :where [?e :email _ ?tx]] (history db))

Java API

List<Object> results = Peer.q(someQuery, conn.db());

Future<Map> txResult = conn.transact(data_tx);

Comparison

Data model

Schema evolution

Queries

Recommendation

EventStore Events

✓ Understandable by normal people

✓ Designed for integration

✓ Forces aggregate design (scalability)

✓ Capture intent

๏ Require design effort to get right

๏ Complect transaction and query

Datomic Facts

✓ Support partial information

✓ Normalization support

✓ CRUD out of the box

๏ Static schema

✓ New events

✓ New event attributes

✓ Event attributes rename (change deserialization)

✓ Projected state change (rebuild projection)

๏ Cross event refactoring

EventStore schema evolution

✓ New or removed attributes

✓ New or removed entity types

✓Move attributes between entities

๏ Fact attributes rename not possible

๏ Fact attribute type change not possible

Datomic schema evolution

✓ Projections use JavaScript

✓ Persistent

๏ Require projection

EventStore queries

✓ Declarative

✓ Logic-based

✓ Allows calling out to your own code

๏ New language to learn

Datomic queries

When to use Event Sourcing?

Domains where events seem natural

Different teams with different models

When to use Datomic?

Data can be modelled using entities and attributes

History is or may be interesting

More details

http://www.jayway.com/author/jankronquist/

https://github.com/jankronquist

Questions?

JavaZone 2013 - Datomic vs EventStore

Technology

Transcript of JavaZone 2013 - Datomic vs EventStore

GlassFish v3 at JavaZone 09

javazone 2014

Javazone Beer Analytics with ELK

Datomic R-trees

How To (Not) Open Source - Javazone, Oslo 2014

JavaZone 2015: Mine containere er lettere enn dine.

Practical microservices - javazone 2014

Test-Driven JavaScript Development (JavaZone 2010)

Datomic rtree-pres

Exploring four Datomic superpowers

Smidig Utrulling - JavaZone 2009

Elasticsearch – mye mer enn søk! [JavaZone 2013]

Cuke4Duke JavaZone 2009

Real-time Financials with Microservices and Functional … · with Microservices and ... - Written in Clojure (functional) - Persistence with Datomic. DATOMIC - Immutable, append-only

CQRS EventStore

Driftbarhet - Steg mot smidig drift, JavaZone 2010

Datomic - Lidando com dados de maneira versionada

Cannybots Workshop @ Javazone 2015

JavaZone 2015 : NodeBots - JavaScript Powered Robots with Johnny-Five

Europe Virtual ALT.NET - EventStore v3