rush, the Ruby shell and Unix integration library
-
Upload
adam-wiggins -
Category
Technology
-
view
7.526 -
download
0
description
Transcript of rush, the Ruby shell and Unix integration library
![Page 1: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/1.jpg)
Cluster management with
the Ruby shell and Unix integration library
Adam WigginsApril 18, 2008http://rush.heroku.com/
![Page 2: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/2.jpg)
bash+ssh are the cornerstone of unix systems administration
![Page 3: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/3.jpg)
but...
![Page 4: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/4.jpg)
Ruby is its own universe
![Page 5: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/5.jpg)
unix = awesomeruby = awesome
So put them together and you should get awesome x awesome...
![Page 6: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/6.jpg)
unix = awesomeruby = awesome
So put them together and you should get awesome x awesome...
...right?
![Page 7: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/7.jpg)
def start_mongrel # kill any existing mongrels pids = `ssh #{host} "ps -u #{user} | grep mongrel_rails | grep -v grep |
sed -r 's/ *([0-9]+).*$/\\1/'"` `ssh #{host} "kill #{pids}; sleep 1; kill -9 #{pids}"`
# don't try to restart if previous mongrel crashed log_contents = `ssh #{host} "cat log/mongrel.log"` return if log.match(/^\s*from .*\/mongrel_rails:/)
# do it `ssh #{host} "echo 'mongrel_rails start -d' |
sudo -u #{user} -H bash"`end
![Page 8: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/8.jpg)
What are the problems here?
![Page 9: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/9.jpg)
Quoting
Try inserting a single quote into the regexp here. How many backslashes do you need?
pids = `ssh #{host} "ps -u #{user} | grep mongrel_rails | grep -v grep |
sed -r 's/ *([0-9]+).*$/\\1/'"`
![Page 10: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/10.jpg)
Security
What if this value can be entered by the user, and they type in: ";rm -rf /
pids = `ssh #{host} "ps -u #{user} | grep mongrel_rails | grep -v grep |
sed -r 's/ *([0-9]+).*$/\\1/'"`
![Page 11: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/11.jpg)
Output is all text
Why do we need the inverse grep, or for that matter the complex regexp, just to get a simple list of PIDs?
pids = `ssh #{host} "ps -u #{user} | grep mongrel_rails | grep -v grep |
sed -r 's/ *([0-9]+).*$/\\1/'"`
![Page 12: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/12.jpg)
Error Trapping
How can we catch errors, like permission denied or file not found, and log them?
log_contents = `ssh #{host} "cat log/mongrel.log"`
![Page 13: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/13.jpg)
Unit Testability
Um... yeah. Good luck with that.
`ssh #{host} "echo 'mongrel_rails start -d' | sudo -u #{user} -H bash"`
![Page 14: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/14.jpg)
In short, Unix and Ruby do not play well together.
And that’s really a shame.
![Page 15: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/15.jpg)
Introducing...
Bringing Ruby and Unix together.awesome x awesome!
![Page 16: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/16.jpg)
Let’s try that again.
rush-style.
![Page 17: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/17.jpg)
def start_mongrel # kill any existing mongrels pids = `ssh #{host} "ps -u #{user} | grep mongrel_rails | grep -v grep |
sed -r 's/ *([0-9]+).*$/\\1/'"` `ssh #{host} "kill #{pids}; sleep 1; kill -9 #{pids}"`
# don't try to restart if previous mongrel crashed log_contents = `ssh #{host} "cat log/mongrel.log"` return if log.match(/^\s*from .*\/mongrel_rails:/)
# do it `ssh #{host} "echo 'mongrel_rails start -d' |
sudo -u #{user} -H bash"`end
![Page 18: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/18.jpg)
def start_mongrel # kill any existing mongrels box.processes.each do |p| if p.uid == uid and p.cmdline.match /mongrel_rails/ p.kill end end
# don't try to restart if previous mongrel crashed log = box['log/mongrel.log'] return if log.search(/^\s*from .*\/mongrel_rails:/)
# do it box.bash 'mongrel_rails -d', :user => userend
![Page 19: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/19.jpg)
Processes box.processes.each do |p| if p.uid == uid and p.cmdline.match /mongrel_rails/ p.kill end end
![Page 20: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/20.jpg)
Files log = box['log/mongrel.log'] return if log.search(/^\s*from .*\/mongrel_rails:/)
![Page 21: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/21.jpg)
Bash Commands box.bash 'mongrel_rails -d', :user => user
![Page 22: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/22.jpg)
Basic concepts of rush
![Page 23: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/23.jpg)
Boxes box = Rush::Box.new('remote.example.com') box['/var/log/nginx/*.log'].search(/error/)
![Page 24: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/24.jpg)
Files & Dirs dir = home['myapp/'] dir['config/environment.rb'].contents
dir['**/*.rb'].search(/^module /)
dir['README'].copy_to other_dir
![Page 25: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/25.jpg)
Processes mongrels = box.processes.find_all_by_cmdline /mongrel_rails/ mem_hogs = mongrels.select { |p| p.mem > 50.mb } mem_hogs.each { |p| p.kill }
![Page 26: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/26.jpg)
Permissions file.access = { :user_can => :read_and_write, :group_can => :read }
![Page 27: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/27.jpg)
Bash Commands box.bash 'rake db:migrate'
![Page 28: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/28.jpg)
Bash Commands box.bash 'rake db:migrate', :user => 'www', :env => { :RAILS_ENV => 'production' }
![Page 29: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/29.jpg)
Bash Commands begin box.bash 'rake db:migrate' rescue Rush::BashFailed => e log_warning "Failed to migrate: #{e.message}" end
![Page 30: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/30.jpg)
It’s Ruby! spec_lines = myproj['**/*_spec.rb'].line_count total_lines = myproj['**/*.rb'].line_count spec_percent = (spec_lines * 100 / total_lines).round puts "Specs are #{spec_percent}% of the total code." puts (spec_percent >= 30) ? "Nice!" : "Tsk, tsk."
![Page 31: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/31.jpg)
It’s Ruby!
“I can not adequately convey the sheer joy it was to rewrite my Bash deployment functions in Ruby.”
- Jamie Hoover
![Page 32: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/32.jpg)
What’s good about the traditional unix shells?
Lessons to carry forward.
![Page 33: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/33.jpg)
Remote and local are seamless
box1['myproj/'].move_to box2
![Page 34: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/34.jpg)
Small sharp tools, chained together
dir1['**/*.rb'].search(/^class/).copy_to dir2
![Page 35: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/35.jpg)
Why now?
![Page 36: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/36.jpg)
Cloud computing.Why now?
![Page 37: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/37.jpg)
Cloud computing.
Which means automation of systems administration tasks.
Why now?
![Page 38: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/38.jpg)
Pre-cloud computing, sysadmin tasks are occasional and manual
![Page 39: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/39.jpg)
In cloud computing, sysadmin tasks are frequent and automatic
![Page 40: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/40.jpg)
In cloud computing, sysadmin tasks are frequent and automatic
So we need a real programming language for them!
![Page 41: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/41.jpg)
My need came from:
![Page 42: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/42.jpg)
hostnames = %w(host1 host2 host3 host4)boxes = hostnames.map { |h| Rush::Box.new(h) }
arc_host = Rush::Box.new('archive')archive = arch_host['/arc/nginx_logs/']
boxes.each do |box| log = box['/var/log/nginx/access.log'] archive_name = "#{box.host}_#{log.name}" log.copy_to archive[archive_name] log.rename log.name + ".old"end
Clustering (finally)
![Page 43: rush, the Ruby shell and Unix integration library](https://reader033.fdocuments.net/reader033/viewer/2022051612/54bd19534a7959766e8b45b9/html5/thumbnails/43.jpg)
gem install rush