AWSSimple WorkFlow
by Nathan McCourtneyMay 2nd, 2012
@beaknit
"Workflow"?
A workflow management system is a computer system that manages and defines a series of tasks within an organization to produce a final outcome or outcomes. At each stage in the workflow, one individual or group is responsible for a specific task. Once the task is complete, the workflow software ensures that the individuals responsible for the next task are notified and receive the data they need to execute their stage of the process
http://en.wikipedia.org/wiki/Workflow
Amazon's Workflow Model
● A Workflow is the automation of a business process.● A Domain is a collection of related Workflows.● Actions are the individual tasks undertaken to carry out
a Workflow.● Activity Workers are the pieces of code that actually
implement the tasks. Each kind of Worker has its own Activity Type.
● A Decider implements a Workflow's coordination logic. - From Jeff Barr's post
Workers & Deciders = Actors
From wikipedia: In computer science, the Actor model is a mathematical model of concurrent computation that treats "actors" as the universal primitives of concurrent digital computation: in response to a message that it receives, an actor can make local decisions, create more actors, send more messages, and determine how to respond to the next message received.
Essentially, SWF is a framework for abstracting out actors and passing
messages between them via queues
Act!
Actor
Work Queue
Result Queue
Messages
1. Fetches its work2. Drops off its results
AWS Message Passing
● HTTPS● Long-poll interval of 60 seconds● Message format is JSON:
{"decisionType": "ScheduleActivityTask", "scheduleActivityTaskDecisionAttributes": {"activityType": {"name": "activityVerify", "version": "1.0"}, "activityId": "verification-27", "control": "digital music", "input": "5634-0056-4367-0923,12/12,437", "scheduleToCloseTimeout": "900", "taskList": {"name": "specialTaskList"}, "scheduleToStartTimeout": "300", "startToCloseTimeout": "600", "heartbeatTimeout": "120"}}
● One decision queue per Workflow● Many activity queues per Workflow
Decider to SWF Comms
Decider
DTL
SWF
1. Decider polls Decision Task List (DTL) for new decision tasks
2. Decision task returned with entire workflow execution history to provider context for decisions
3. Decider executes conditional logic and returns an activity task back to SimpleWorkFlow which then deposits in the Activity Task List (ATL)
Note: Multiple deciders can process the same DTL
ATL
From AWS Ruby SDK
# poll for decision tasks from 'my-task-list' domain.decision_tasks.poll( 'my-task-list') do |task|
# investigate new events and make decisions task.new_events.each do |event| case event.event_type when 'WorkflowExecutionStarted' task.schedule_activity_task 'do-something', :input => 'abc xyz' when 'ActivityTaskCompleted' task.complete_workflow_execution :result => event.attributes.result end end
end # decision task is completed here
Worker to SWF Comms
1. Worker polls ATL for appropriate tasks - different workers can consume different tasks
2. Task completion returned to SWF which makes the result available to the Decider via the DTL
Note: Multiple workers can process the same ATL
Worker
ATL
SWF
DTL
# poll 'my-task-list' for activities domain.activity_tasks.poll( 'my-task-list') do |activity_task|
case activity_task.activity_type.name when 'do-something' # ... else activity_task.fail! :reason => 'unknown activity task type' end end
From AWS Ruby SDK
AWS example order-processing SWF
Source: SWF Dev Guide
Practical Exercise
Nightly restore of a database to QA nodes combined with automated release migration.
Examples
Further Reading
● Official Announcement● Rightscale's blog on their use● Dev Guide - Basic Concepts● AWS Ruby SDK
Top Related