Event Driven Architectures
Click here to load reader
-
Upload
avinash-ramineni -
Category
Technology
-
view
475 -
download
1
Transcript of Event Driven Architectures
![Page 1: Event Driven Architectures](https://reader037.fdocuments.net/reader037/viewer/2022100304/554f79d5b4c9052a518b498d/html5/thumbnails/1.jpg)
Event Driven Architectures
Avinash Ramineni
![Page 2: Event Driven Architectures](https://reader037.fdocuments.net/reader037/viewer/2022100304/554f79d5b4c9052a518b498d/html5/thumbnails/2.jpg)
• What is Event Driven Architecture• How is EDA related to SOA• Event Capture/Generation• Event Processing• Event Delivery Guarantees• Complex Event Processing• Events, Internet of Things and Big Data
Agenda
![Page 3: Event Driven Architectures](https://reader037.fdocuments.net/reader037/viewer/2022100304/554f79d5b4c9052a518b498d/html5/thumbnails/3.jpg)
• Significant change in the state of an Object– Bank Transaction – Students grade gets posted– New Order gets created– Customer Profile gets updated– Change in Heart Beat
• Characteristics of an Event– Events are broadcast– Communications are asynchronous– Events are fine grained
Event
![Page 4: Event Driven Architectures](https://reader037.fdocuments.net/reader037/viewer/2022100304/554f79d5b4c9052a518b498d/html5/thumbnails/4.jpg)
• Handling of the Events as they occur and the transfer of events among the systems in a loosely coupled way
• Enables Situation Awareness and enables sense-and-respond behavior
Event Driven Architecture (EDA)
![Page 5: Event Driven Architectures](https://reader037.fdocuments.net/reader037/viewer/2022100304/554f79d5b4c9052a518b498d/html5/thumbnails/5.jpg)
• Characteristics of EDA – Producer/Consumers are not aware of each other– Little or no statefullness– Distributed– Loosely Coupled– Platform and Language Independent– Reliable– Fast and Highly Scalable
Event Driven Architecture (EDA)
![Page 6: Event Driven Architectures](https://reader037.fdocuments.net/reader037/viewer/2022100304/554f79d5b4c9052a518b498d/html5/thumbnails/6.jpg)
EDA extension of SOA
Computing with event objects
REST, URIs and HTTP
Event Processing
RPC-style SOA
Event-driven SOA
SOA
Web Architecture
Component software done right
Web-oriented architecture (WOA)
Non-SOA EDA and CEP
Simple Web sites
![Page 7: Event Driven Architectures](https://reader037.fdocuments.net/reader037/viewer/2022100304/554f79d5b4c9052a518b498d/html5/thumbnails/7.jpg)
• EDA similar to SOA facilitates Agility by using Modular Design and Separation of Concerns
• EDA fits well into organizations that have some SOA infrastructure in place
• EDA adds another dimension / Possibilities to SOA– Events are generally stateless and do not have much content
• While processing the events that do not have much data in them, the consuming applications call the services exposed to get the actual data
• Publish – Subscribe Model– Similar to Asynchronous Messaging Architectures
EDA extension to SOA
![Page 8: Event Driven Architectures](https://reader037.fdocuments.net/reader037/viewer/2022100304/554f79d5b4c9052a518b498d/html5/thumbnails/8.jpg)
• Application Generated – Application generates an event when a business transaction
completes– Events generated by parsing the access logs/ server logs– Polling web pages – capturing the events from the services
• Database Generated – Database Trigger generated Events
• On insert/update/delete– During database replication
How to generate Events
![Page 9: Event Driven Architectures](https://reader037.fdocuments.net/reader037/viewer/2022100304/554f79d5b4c9052a518b498d/html5/thumbnails/9.jpg)
• The network is reliable• Latency is zero• Bandwidth is infinite• The network is secure• Topology does not change• Transport cost is zero• Network is homogeneous
Fallacies of Distributed Systems
![Page 10: Event Driven Architectures](https://reader037.fdocuments.net/reader037/viewer/2022100304/554f79d5b4c9052a518b498d/html5/thumbnails/10.jpg)
• Send an email when a customers profile gets updated Customer Profile– Assumptions:
• There is an Application A that provides a updateProfileService• There is an Application B that provides a sendEmailService• All the Events are delivered with Once and Only Once SLA and the Events
are processed
Usecase: 1
A Event Channel
DB
Update DB
Profile Updated Event
B
![Page 11: Event Driven Architectures](https://reader037.fdocuments.net/reader037/viewer/2022100304/554f79d5b4c9052a518b498d/html5/thumbnails/11.jpg)
• The overhead involved in capturing events must be as low as possible to avoid impairing the performance of a business transaction.
• Business Transaction and the Event generation needs to be in the same transaction– 1,2 needs to be part of a same transaction
• Both 1 and 2 should succeed or both should fail
Considerations while generating Events
A Event Channel
DB
1. Update DB
2. Profile Updated Event
B
![Page 12: Event Driven Architectures](https://reader037.fdocuments.net/reader037/viewer/2022100304/554f79d5b4c9052a518b498d/html5/thumbnails/12.jpg)
• Successful Update of Profile -- Call a service to publish a Profile update Event
• Can a database transaction and Web service call be made atomic?
Usecase: 1 – Web service
try{ 1. Update Profile on DB 2. Call web service to publish profile updated event} catch(Exception e){ rollback();}commit;
try{1. Update Profile on DB
} catch(Exception e) { rollback(); }commit(); 2. Call web service to publish profile updated event
• Can Compensating Transaction work ?
![Page 13: Event Driven Architectures](https://reader037.fdocuments.net/reader037/viewer/2022100304/554f79d5b4c9052a518b498d/html5/thumbnails/13.jpg)
• Successful Update of Profile -- Put a message on to Message Broker
Usecase: 1 – JMS
A Event Channel
DB
1. Update DB
2. Profile Updated Event (JMS message)
B
• Possible to make 1 and 2 into a single transaction but it requires – Distributed Transaction.
![Page 14: Event Driven Architectures](https://reader037.fdocuments.net/reader037/viewer/2022100304/554f79d5b4c9052a518b498d/html5/thumbnails/14.jpg)
• Follows 2 Phase Commit protocol• Very Chatty protocol and does a lot of logging to be able to
recover from any failure scenario.• Too much overhead to 99.9% of the cases to handle less than
0.1% of cases• Increases the chances for deadlocks on the database. • Lowers the overall performance of the system.• Bane of scalability. Grinds the entire system to halt by adding
overhead to each and every transaction.
Distributed/XA Transactions (1)
![Page 15: Event Driven Architectures](https://reader037.fdocuments.net/reader037/viewer/2022100304/554f79d5b4c9052a518b498d/html5/thumbnails/15.jpg)
• Availability of the Systems goes down.• XA Configuration is complicated • Difficult to test.• Many People tend to believe that using JTA implementation of
transaction manager will take care of a XA transactions• Many a time people end up using JTA Manager even while
dealing with single resource.
Distributed/XA Transactions (2)
![Page 16: Event Driven Architectures](https://reader037.fdocuments.net/reader037/viewer/2022100304/554f79d5b4c9052a518b498d/html5/thumbnails/16.jpg)
• Successful Update of Profile -- Put a message on to Message Broker
Usecase: 1 – Local Database
A Event Channel
DB
1. Update DB
3. Batch process to send events to Event Channel B
• 1 and 2 are in same transaction. 3 can be retried multiple times till they succeed.
2. Store Event
![Page 17: Event Driven Architectures](https://reader037.fdocuments.net/reader037/viewer/2022100304/554f79d5b4c9052a518b498d/html5/thumbnails/17.jpg)
• Successful Update of Profile -- Put a message on to Message Broker
Usecase: 1 – Local JMS
A Event Channel
DB
1. Update DB
3. Publish to Event Channel(JMS message)
B
• Possible to make 1 and 2 into a single transaction without a Distributed Transaction as long as Queue is backed by a same database
queue
Send Event to Local Queue
2
![Page 18: Event Driven Architectures](https://reader037.fdocuments.net/reader037/viewer/2022100304/554f79d5b4c9052a518b498d/html5/thumbnails/18.jpg)
• Successful Update of Profile -- Put a message on to Message Broker / Call a web service to publish an Event
• Have a Reconciliation Process in place to verify that there is an event generated for every Business Transaction– Have a unique Id (may be stored in DB) along with update and use that
Id as Correlation Id with the Events Archive– If it events don’t tally up, recreate those events
• Use the DB triggers to generate events – Write to a different local table – Call a web service to send the event to event channel (have seen
people do that)
Usecase: 1 – Other Ways
![Page 19: Event Driven Architectures](https://reader037.fdocuments.net/reader037/viewer/2022100304/554f79d5b4c9052a518b498d/html5/thumbnails/19.jpg)
• Event Delivery Guarantees– Reliability• At least Once
– Duplicate events can be delivered
• At most Once– Some events can be lost
• Once and only Once– Each is delivered Exactly once
– Order of Delivery• In Order Guaranteed• In Order not Guaranteed
Event Delivery
![Page 20: Event Driven Architectures](https://reader037.fdocuments.net/reader037/viewer/2022100304/554f79d5b4c9052a518b498d/html5/thumbnails/20.jpg)
• Processing an Event and associated Business Transaction needs to be in the same transaction
• 3,2 needs to be part of a same transaction• Both 3 and 2 should succeed or both should fail
Considerations while Processing Events
Event Channel
3. Acknowledge Event Process success
1. Profile Updated Event
B
2. Send Email
![Page 21: Event Driven Architectures](https://reader037.fdocuments.net/reader037/viewer/2022100304/554f79d5b4c9052a518b498d/html5/thumbnails/21.jpg)
• On Profile Update Event -- Call a service to send Email and on success acknowledge process successful
Usecase: 1 – Consumer (1)
onMessagetry { sendEmail(); jms.commit()}catch (Exception e) { jms.rollback()}
• What if jms.commit() fails ??
![Page 22: Event Driven Architectures](https://reader037.fdocuments.net/reader037/viewer/2022100304/554f79d5b4c9052a518b498d/html5/thumbnails/22.jpg)
• If Jms.commit() fails , message gets delivered again
Usecase: 1 – Consumer (2)
onMessagetry { if I have not processed this message successfully before { do some stuff in the database / Idempotency Logic /JMSRelivered flag jdbc.commit; } jms.commit()}catch (Exception e) { jms.rollback()}
![Page 23: Event Driven Architectures](https://reader037.fdocuments.net/reader037/viewer/2022100304/554f79d5b4c9052a518b498d/html5/thumbnails/23.jpg)
• Idempotent Operation– An idempotent operations means that the result of a successful
performed request is independent of the number of times it is executed.
• Design consumer to be an Idempotent Consumer– By Understanding the Business rules it can be achieved relatively easy
• Good Practice to make all the write operations Idempotent !!• Idempotency logic can be built on Event Ids, Business Ids –
orderId, customerId, etc based on the Business scenario
Work Around for At-Least-Once
![Page 24: Event Driven Architectures](https://reader037.fdocuments.net/reader037/viewer/2022100304/554f79d5b4c9052a518b498d/html5/thumbnails/24.jpg)
• In-order delivery of events– Limit the number of event publisher/consumer instance
for these specific events to 1 • Brings up questions like if one of the event is having an issue – all
the events of that event type can come to a stand still
– AVOID the need for In-order Delivery of Events• By limiting the content in the Event , we can avoid a lot of cases
where In-order delivery is required• Using Message Selectors to select and aggregate the events can
solve a few of the cases
What If In-order Delivery is required
![Page 25: Event Driven Architectures](https://reader037.fdocuments.net/reader037/viewer/2022100304/554f79d5b4c9052a518b498d/html5/thumbnails/25.jpg)
• Student Information System (SIS) maintain a list of students in a course roster. The students can be added or dropped from the course roster. SIS sends an event to Learning Management System (LMS) when the roster gets updated.
• Lets us assume that SIS is sending the course roster as part of the event.
• Processing of this Event requires the Roster Change events to be delivered in order to maintain the data from getting corrupted.
Usecase: 2
![Page 26: Event Driven Architectures](https://reader037.fdocuments.net/reader037/viewer/2022100304/554f79d5b4c9052a518b498d/html5/thumbnails/26.jpg)
Usecase: 2 – Requires In-Order
SIS Event Channel
DB
2. Deliver Roster Change Event
LMS
LMS DB
1. Roster Change Event
![Page 27: Event Driven Architectures](https://reader037.fdocuments.net/reader037/viewer/2022100304/554f79d5b4c9052a518b498d/html5/thumbnails/27.jpg)
• Remove the roster data from the event and just pass a roster Id
• On Delivery of the Event , the LMS system would use the rosterId from the event, makes a service call to the source system (SIS) to get the latest Roster data and builds the classes in LMS
• Even if the events are delivered out of order , the roster data on LMS will be always be the latest.
• By making the processor on LMS idempotent, we can even avoid the database update
Usecase: 2 - Solution
![Page 28: Event Driven Architectures](https://reader037.fdocuments.net/reader037/viewer/2022100304/554f79d5b4c9052a518b498d/html5/thumbnails/28.jpg)
Usecase: 2
SIS Event Channel
DB
2. Deliver Roster Change Event
LMS
LMS DB
1. Roster Change Event
3. Get the latest roster for the Id
4. Update the LMS database
![Page 29: Event Driven Architectures](https://reader037.fdocuments.net/reader037/viewer/2022100304/554f79d5b4c9052a518b498d/html5/thumbnails/29.jpg)
• Make Event Publishers responsible for making sure that events are generated for every Business transaction and are published to the event channel
• Make Event Consumers responsible for making sure that they react to the Stimuli from the event
• Dashboards in place to make sure failed Events are detected and acted upon
• Collects Stats to see if the events need to be made finer or coarser
• Maintain Event Catalogs and self service capabilities to subscribe to events
Best Practices
![Page 30: Event Driven Architectures](https://reader037.fdocuments.net/reader037/viewer/2022100304/554f79d5b4c9052a518b498d/html5/thumbnails/30.jpg)
Complex Event Processing (CEP)
Traditional BI finds "needles in haystacks."
Event-driven CEP finds "needles" in continuously arriving streams of "hay."
• CEP tap the events happening with in an organization and provide a situation awareness and enable better and faster decisions– Identify complicated events that are inferred from event pattern
detection, event pattern interpretors and so on– Ex: Monitoring – capture events from ticket system, network load,
performance and send out notification bring up new servers into the pool
![Page 31: Event Driven Architectures](https://reader037.fdocuments.net/reader037/viewer/2022100304/554f79d5b4c9052a518b498d/html5/thumbnails/31.jpg)
• With Big Data hype and with the “capture-it-all“ approach and IoT , EDA and Event Driven programming will get a huge boost in the coming years.
• Events and EDA due to their characteristics will be very well suited for Ubiquitous Computing
• We might start seeing an advent of Complex Event Applications by being used by various gadgets – The concepts of EDA and Events will be a perfect Match for gadgets like Basis ,FIT
Events ,Big Data and Internet of Things