Resque - files.meetup.com Awesome.pdf · Tangent Alert: Redis! Scalability The Gem load balances...
46
Resque High-performance asynchronous task queuing and processing
Transcript of Resque - files.meetup.com Awesome.pdf · Tangent Alert: Redis! Scalability The Gem load balances...
ResqueHigh-performance asynchronous task queuing and
processing
ResqueHigh-performance asynchronous task queuing and
processing
Experts‘Cause you might build the next GitHub
Three Things
1. BackgroundWhy run background processes at all?
2. Delayed::JobBecause it’s the foundation for Resque
3. ResqueProcess different.
1. Background
Why?
1. Background
It’s all about the ms• Resize an image
• Send an e-mail
• Communicate with a web service
• Geocode a location
1. Background
Do something later than now
1. Background
Webpages.Rails serves webpages.
Memories. You're talkin about memories.
1. Background
1. Receive requests2. Send responses
1. Background
Everything else can go elsewhere
1. Background
Where?• CRON
• Beanstalkd
• Workling
• BackgroundRB
1. Background
• Silent failure
• No status checks
• Unstable processes
WTF my tasks r fail?!1
2. Delayed::Job
To the rescue!
2. Delayed::Job
ActiveRecord::Base’d
• Persistence! At last!
• Ease of use! At last!
2. Delayed::Job
How easy?
2. Delayed::Job
# without delayed_jobNotifier.deliver_signup(@user)
# with delayed_jobNotifier.send_later :deliver_signup, @user
Super easy.
2. Delayed::Job
Where do they go?
2. Delayed::Job
mysql> SELECT * FROM delayed_jobs;*************************** 1. row *************************** id: 1 priority: 0 attempts: 0 handler: --- !ruby/object:WorkerBee <-- Notice the pretty Marshal dumpn: 18
last_error: NULL run_at: 2010-03-10 22:44:30 locked_at: NULL failed_at: NULL locked_by: NULLcreated_at: 2010-03-10 22:44:30updated_at: 2010-03-10 22:44:301 row in set (0.00 sec)
2. Delayed::Job
What do you do with them?
$ script/delayed_job start
2. Delayed::Job
Workers.
• Live in their own thread
• Query the database for jobs
• Perform the long-running taskaway from request/response
The Old Way
The New Way
Delayed::Job
2. Delayed::Job
Totally rad.
3. Resque
Isn’t Delayed::Job enough?
3. Resque
Apparently not.
3. Resque
Resque is some Enterprise Grade shit.
3. Resque
Runs on Redis
Just like Memcached
Redis is a Key-Value Store
Tangent Alert: Redis!
Tangent Alert: Redis!
PersistenceUnlike Memcached
Tangent Alert: Redis!
ScalabilityThe Gem load balances for you
Resque.redis = DistRedis.new( :hosts=> %w{192.168.1.22:6379 192.168.1.23:6379})
Tangent Alert: Redis!
Basically:Fast & Distributed
Also, Rails-independentNo ActiveRecord/ActiveSupport means you can run Resque (on Redis) in Sinatra or any other Ruby app
Tangent Alert: Redis!
3. Resque
• Forked, sandboxed threads
• Failing tasks are tracked
• Kills timeouts
Better Workers
3. Resque
Queueing
fileserver$ QUEUE=zip_files rake resque:work webserver$ QUEUE=cache_images rake resque:work
Separate queues for separate workers
3. Resque
No MarshalingBecause it should be functional
Resque.enqueue(SignupNotifier, @user.id)
3. Resque
No Marshaling
class SignupNotifier def self.perform(user_id) Notifier.deliver_signup(User.find(user_id)) endend
3. Resque
UIThat’s right. It has its own interface.
3. Resque
• Many background tasks
• Distributed task servers
• Stable workers
So... why?
3. Resque
• Complex and expensive
• No scheduling or priority like in D::J
• Asynchronous Redis writes don’t guarantee persistence
Trouble Points
3. Resque
Live coding FTW!Here goes nothing...
ThnaksThanks for taking the time to listen to me blather
about Resque!
Bmore on Rails
http://www.meetup.com/bmore-on-rails/
Meetups - 2nd Tuesday#OSHN - 4th Tuesday
