Exploring Lightweight Event Sourcing - Scaladays2011.scala-lang.org/sites/days2011/files/50. Event...
51
Transcript of Exploring Lightweight Event Sourcing - Scaladays2011.scala-lang.org/sites/days2011/files/50. Event...
Exploring lightweight event sourcing
Goals
• Understand event sourcing
• Show why Scala is well-suited as implementation language
2
Exploring lightweight event sourcing
Event Sourcing
• Documented by Martin Fowler at http://martinfowler.com/eaaDev/EventSourcing.html (2005)
• Made practical by Greg Young (QCon 2008)
• Applied to complex, high volume systems (financial trading)
• But what about small systems?
3
Exploring lightweight event sourcing
• Domain-Driven Design An approach to software development that suggests that (1) For most software projects, the primary focus should be on the domain and domain logic; and (2) Complex domain designs should be based on a model.
• Domain Expert A member of a software project whose field is the domain of the application, rather than software development. Not just any user of the software, the domain expert has deep knowledge of the subject.
• Ubiquitous Language A language structured around the domain model and used by all team members to connect all the activities of the team with the software.
4
Exploring lightweight event sourcing
Durability
5
Exploring lightweight event sourcing
Durability
• Most applications require durability of data
5
Exploring lightweight event sourcing
Durability
• Most applications require durability of data
• UPDATE invoice WHERE id = 1234 SET total_amount = 230
5
Exploring lightweight event sourcing
Durability
• Most applications require durability of data
• UPDATE invoice WHERE id = 1234 SET total_amount = 230
• What happened to previous order amount?
5
Exploring lightweight event sourcing
Durability
• Most applications require durability of data
• UPDATE invoice WHERE id = 1234 SET total_amount = 230
• What happened to previous order amount?
• What was the reason for the change?
5
Exploring lightweight event sourcing
ORMs
• Query and persist current state in DB
• Tightly couples domain and data model
• Leaky abstraction
• Still “lossy” and the intent of the user is not captured
6
Exploring lightweight event sourcing
Behavior?
7
Exploring lightweight event sourcing
Invoice
Status: "Draft"
Total: 0.00
Behavior?
7
Exploring lightweight event sourcing
Invoice
Status: "Draft"
Total: 0.00 Invoice
Status: "Draft"
Recipient: "Erik"
Total: 0.00
Behavior?
7
Exploring lightweight event sourcing
Invoice
Status: "Draft"
Recipient: "Erik"
Invoice Item
Total: 9.95
Item: "Food"
Amount: 9.95
Invoice
Status: "Draft"
Total: 0.00 Invoice
Status: "Draft"
Recipient: "Erik"
Total: 0.00
Behavior?
7
Exploring lightweight event sourcing
Invoice
Status: "Draft"
Recipient: "Erik"
Invoice Item
Total: 9.95
Item: "Food"
Amount: 9.95
Invoice
Status: "Draft"
Total: 0.00 Invoice
Status: "Draft"
Recipient: "Erik"
Total: 0.00
It’s just data mutation
8
Exploring lightweight event sourcing
Invoice
Status: "Draft"
Recipient: "Erik"
Invoice Item
Total: 9.95
Item: "Food"
Amount: 9.95
It’s just data mutation
8
Exploring lightweight event sourcing
“Inexperienced programmers love magic because it saves their time. Experienced programmers hate magic because it wastes their time.” – @natpryce
9
ORM implementation
Exploring lightweight event sourcing
Event Sourcing
10
• All state changes are explicitly captured using domain events
• Events represent the outcome of behavior
• Capture the intent of the user and the related data
• The domain expert understands and can explain the meaning of the domain events
Exploring lightweight event sourcing
Domain Behavior
11
Exploring lightweight event sourcing
Draft Invoice Created
create
Domain Behavior
11
Exploring lightweight event sourcing
Invoice
Status: "Draft"
Total: 0.00
Draft Invoice Created
create
apply
Domain Behavior
11
Exploring lightweight event sourcing
Invoice
Status: "Draft"
Total: 0.00
Draft Invoice Created
Recipient: "Erik"
Invoice Recipient Changed
create
createapply
Domain Behavior
11
Exploring lightweight event sourcing
Invoice
Status: "Draft"
Total: 0.00 Invoice
Status: "Draft"
Recipient: "Erik"
Total: 0.00
Draft Invoice Created
Recipient: "Erik"
Invoice Recipient Changed
create
createapply apply
Domain Behavior
11
Exploring lightweight event sourcing
Invoice
Status: "Draft"
Total: 0.00 Invoice
Status: "Draft"
Recipient: "Erik"
Total: 0.00
Draft Invoice Created
Recipient: "Erik"
Invoice Recipient Changed
Item: "Food"Item amount: 9.95Total amount: 9.95
Invoice Item Added
create
create createapply apply
Domain Behavior
11
Exploring lightweight event sourcing
Invoice
Status: "Draft"
Recipient: "Erik"
Invoice Item
Total: 9.95
Item: "Food"
Amount: 9.95
Invoice
Status: "Draft"
Total: 0.00 Invoice
Status: "Draft"
Recipient: "Erik"
Total: 0.00
Draft Invoice Created
Recipient: "Erik"
Invoice Recipient Changed
Item: "Food"Item amount: 9.95Total amount: 9.95
Invoice Item Added
create
create createapply apply apply
Domain Behavior
11
Exploring lightweight event sourcing
Invoice
Status: "Draft"
Recipient: "Erik"
Invoice Item
Total: 9.95
Item: "Food"
Amount: 9.95
Invoice
Status: "Draft"
Total: 0.00 Invoice
Status: "Draft"
Recipient: "Erik"
Total: 0.00
Draft Invoice Created
Recipient: "Erik"
Invoice Recipient Changed
Item: "Food"Item amount: 9.95Total amount: 9.95
Invoice Item Added
Only the events are stored on disk
12
Exploring lightweight event sourcing
Reloading from history
13
Exploring lightweight event sourcing
Invoice
Status: "Draft"
Total: 0.00
Draft Invoice Created
apply
Reloading from history
13
Exploring lightweight event sourcing
Invoice
Status: "Draft"
Total: 0.00 Invoice
Status: "Draft"
Recipient: "Erik"
Total: 0.00
Draft Invoice Created
Recipient: "Erik"
Invoice Recipient Changed
apply apply
Reloading from history
13
Exploring lightweight event sourcing
Invoice
Status: "Draft"
Recipient: "Erik"
Invoice Item
Total: 9.95
Item: "Food"
Amount: 9.95
Invoice
Status: "Draft"
Total: 0.00 Invoice
Status: "Draft"
Recipient: "Erik"
Total: 0.00
Draft Invoice Created
Recipient: "Erik"
Invoice Recipient Changed
Item: "Food"Item amount: 9.95Total amount: 9.95
Invoice Item Added
apply apply apply
Reloading from history
13
Exploring lightweight event sourcing
Invoice
Status: "Draft"
Recipient: "Erik"
Invoice Item
Total: 9.95
Item: "Food"
Amount: 9.95
Invoice
Status: "Draft"
Total: 0.00 Invoice
Status: "Draft"
Recipient: "Erik"
Total: 0.00
Draft Invoice Created
Recipient: "Erik"
Invoice Recipient Changed
Item: "Food"Item amount: 9.95Total amount: 9.95
Invoice Item Added
apply apply apply
Reloading from history
13
Exploring lightweight event sourcing
Aggregates
• A cluster of associated objects that are treated as a unit for the purpose of data changes
• Each aggregate has its own stream of events
• Consistency boundary
14
Exploring lightweight event sourcing
Aggregates
Invoice
Status: "Draft"
Recipient: "Erik"
Invoice Item
Total: 9.95
Item: "Food"
Amount: 9.95
15
Exploring lightweight event sourcing
Aggregates
Invoice
Status: "Draft"
Recipient: "Erik"
Invoice Item
Total: 9.95
Item: "Food"
Amount: 9.95
15
Draft Invoice Created
Recipient: "Erik"
Invoice Recipient Changed
Item: "Food"Item amount: 9.95Total amount: 9.95
Invoice Item Added
Exploring lightweight event sourcing
Event Store
16
• Stores sequence of events per aggregate
• Dispatches events when successfully committed
• Replays events on startup
• It’s your application’s transaction log
Exploring lightweight event sourcing
Event Store
• No good for queries!
• Use separate views or reports that use the domain events to capture answers to queries
17
Exploring lightweight event sourcing
Domain Code
18
sealed trait InvoiceEventcase class InvoiceCreated() extends InvoiceEventcase class InvoiceRecipientChanged(recipient: String) extends InvoiceEventcase class InvoiceItemAdded(item: InvoiceItem, totalAmount: BigDecimal) extends InvoiceEventcase class InvoiceSent(sentOn: LocalDate, paymentDueOn: LocalDate) extends InvoiceEvent
Exploring lightweight event sourcing
Domain Code
18
sealed trait InvoiceEventcase class InvoiceCreated() extends InvoiceEventcase class InvoiceRecipientChanged(recipient: String) extends InvoiceEventcase class InvoiceItemAdded(item: InvoiceItem, totalAmount: BigDecimal) extends InvoiceEventcase class InvoiceSent(sentOn: LocalDate, paymentDueOn: LocalDate) extends InvoiceEvent
sealed trait Invoice extends AggregateRoot { protected[this] type Event = InvoiceEvent}
Exploring lightweight event sourcing
Domain Code
19
case class DraftInvoice( recipient: Option[String] = None, nextItemId: Int = 1, items: Map[Int, InvoiceItem] = Map.empty) extends Invoice {
def changeRecipient(recipient: String): Behavior[DraftInvoice] = ...
def addItem(description: String, amount: BigDecimal): Behavior[DraftInvoice] = ...
def send(sentOn: LocalDate): Behavior[Either[String, SentInvoice]] = ...
...
private def recipientChanged = when[InvoiceRecipientChanged] { event => copy(recipient = Some(event.recipient)) } private def itemAdded = when[InvoiceItemAdded] { event => copy(nextItemId = event.item.id + 1, items = items + (event.item.id -> event.item)) } private def sent = when[InvoiceSent] { event => SentInvoice(event.sentOn, event.paymentDueOn) }
}
Exploring lightweight event sourcing
Domain Code
19
case class DraftInvoice( recipient: Option[String] = None, nextItemId: Int = 1, items: Map[Int, InvoiceItem] = Map.empty) extends Invoice {
def changeRecipient(recipient: String): Behavior[DraftInvoice] = ...
def addItem(description: String, amount: BigDecimal): Behavior[DraftInvoice] = ...
def send(sentOn: LocalDate): Behavior[Either[String, SentInvoice]] = ...
...
private def recipientChanged = when[InvoiceRecipientChanged] { event => copy(recipient = Some(event.recipient)) } private def itemAdded = when[InvoiceItemAdded] { event => copy(nextItemId = event.item.id + 1, items = items + (event.item.id -> event.item)) } private def sent = when[InvoiceSent] { event => SentInvoice(event.sentOn, event.paymentDueOn) }
}
The “Brains”
Exploring lightweight event sourcing
Domain Code
19
case class DraftInvoice( recipient: Option[String] = None, nextItemId: Int = 1, items: Map[Int, InvoiceItem] = Map.empty) extends Invoice {
def changeRecipient(recipient: String): Behavior[DraftInvoice] = ...
def addItem(description: String, amount: BigDecimal): Behavior[DraftInvoice] = ...
def send(sentOn: LocalDate): Behavior[Either[String, SentInvoice]] = ...
...
private def recipientChanged = when[InvoiceRecipientChanged] { event => copy(recipient = Some(event.recipient)) } private def itemAdded = when[InvoiceItemAdded] { event => copy(nextItemId = event.item.id + 1, items = items + (event.item.id -> event.item)) } private def sent = when[InvoiceSent] { event => SentInvoice(event.sentOn, event.paymentDueOn) }
}
The “Brains”
The “Muscle”
Exploring lightweight event sourcing
Domain Code
20
case class DraftInvoice( recipient: Option[String] = None, nextItemId: Int = 1, items: Map[Int, InvoiceItem] = Map.empty) extends Invoice {
def changeRecipient(recipient: String): Behavior[DraftInvoice] = recipientChanged(InvoiceRecipientChanged(recipient))
def addItem(description: String, amount: BigDecimal): Behavior[DraftInvoice] = itemAdded(InvoiceItemAdded(InvoiceItem(nextItemId, description, amount), totalAmount + amount))
def send(sentOn: LocalDate): Behavior[Either[String, SentInvoice]] = if (readyToSend_?) for (updated <- sent(InvoiceSent(sentOn, paymentDueOn = sentOn.plusDays(14)))) yield Right(updated) else pure(Left("invoice not ready to send"))
private def totalAmount = items.values.map(_.amount).sum
private def readyToSend_? = recipient.isDefined && items.nonEmpty
...}
Exploring lightweight event sourcing
Domain Code
21
case class DraftInvoice( recipient: Option[String] = None, nextItemId: Int = 1, items: Map[Int, InvoiceItem] = Map.empty) extends Invoice {
...
protected[this] def applyEvent = recipientChanged orElse itemAdded orElse sent
...}
Exploring lightweight event sourcing
Minimal Infrastructure
• Store only the domain events, replay these on application startup to reconstruct current state
• Keep the current state in-memory
• Use immutability data to reduce need for cloning or locking
22
Exploring lightweight event sourcing
Simple functionality example
• “CRUD”: no domain logic, so no aggregate
• So we’ll persist events directly from the UI
• Useful to get started
• ... and even complex applications still contain simple functionality
23
Exploring lightweight event sourcing
Aggregate Example
24
Exploring lightweight event sourcing
Implementation Limitations
• Serializing (“big lock”) event store
• ~ 50 transactions per second
• Number of events to replay on startup
• ~ 30.000 JSON serialized events per second
• Total number of objects to store in main memory
• JVM can run with large heaps
25
Exploring lightweight event sourcing
But ready to scale
• Advanced event store implementations support SQL, MongoDB, Amazon SimpleDB, etc.
• Use persisted view model for high volume objects (fixes startup time and memory usage)
• Easy partitioning of aggregates (consistency boundary with unique identifier)
• Load aggregates on-demand, use snapshotting
26
Exploring lightweight event sourcing
Conclusion
• Event Sourcing captures full historical information
• Fully encapsulated domain code highly decoupled from persistence mechanism
• Simpler to implement and fully understand than traditional ORM based approach
• “Paradigm” shift towards Event-Driven
27
Thanks!
Exploring lightweight event sourcing
References• Example code https://github.com/erikrozendaal/scala-event-sourcing-example
• CQRS http://cqrsinfo.com/
• Greg Young, “Unshackle Your Domain”http://www.infoq.com/presentations/greg-young-unshackle-qcon08
• Pat Helland, “Life Beyond Distributed Transactions: An Apostate's Opinion” http://www.ics.uci.edu/~cs223/papers/cidr07p15.pdf
• Erik Rozendaal, “Towards an immutable domain model” http://blog.zilverline.com/2011/02/10/towards-an-immutable-domain-model-monads-part-5/
• Frameworks: http://www.axonframework.org/ (Java) and http://ncqrs.org/ (.NET)
• Event store: https://github.com/joliver/EventStore (.NET) with over 20 supported data stores
29
