Design Summit - Rails 4 Migration - Aaron Patterson

123
( ͡ ° ͜ ʖ ͡ °) .oO(Hi)

description

ManageIQ currently runs on Ruby on Rails 3. Aaron "tenderlove" Patterson presents his effort to migrate to RoR 4, which entails some changes in the code to take advantage of the latest advances in RoR. For more on ManageIQ, see http://manageiq.org/

Transcript of Design Summit - Rails 4 Migration - Aaron Patterson

Page 1: Design Summit - Rails 4 Migration - Aaron Patterson

( °͡ ʖ͜ °͡) .oO(Hi)

Page 2: Design Summit - Rails 4 Migration - Aaron Patterson

Aaron Patterson

Page 3: Design Summit - Rails 4 Migration - Aaron Patterson
Page 4: Design Summit - Rails 4 Migration - Aaron Patterson
Page 5: Design Summit - Rails 4 Migration - Aaron Patterson
Page 6: Design Summit - Rails 4 Migration - Aaron Patterson

Ruby Core Rails Core

Page 7: Design Summit - Rails 4 Migration - Aaron Patterson

Enterprise Software

Page 8: Design Summit - Rails 4 Migration - Aaron Patterson

Twitter: tenderlove

GitHub: tenderlove

Instagram: tenderlove

Yo: tenderlove

Page 9: Design Summit - Rails 4 Migration - Aaron Patterson
Page 12: Design Summit - Rails 4 Migration - Aaron Patterson

Tender Parents

Page 13: Design Summit - Rails 4 Migration - Aaron Patterson

OMG

INTERNET

POINTS

Page 14: Design Summit - Rails 4 Migration - Aaron Patterson

Revert Commits Count Too!

Page 15: Design Summit - Rails 4 Migration - Aaron Patterson

More mistakes == more points!!!!

Page 16: Design Summit - Rails 4 Migration - Aaron Patterson

Short Stack Engineer

Page 17: Design Summit - Rails 4 Migration - Aaron Patterson
Page 18: Design Summit - Rails 4 Migration - Aaron Patterson
Page 19: Design Summit - Rails 4 Migration - Aaron Patterson

Gorbachev Puff Puff Thunderhorse

Page 20: Design Summit - Rails 4 Migration - Aaron Patterson

SEA-TAC Airport YouTube

Page 21: Design Summit - Rails 4 Migration - Aaron Patterson
Page 22: Design Summit - Rails 4 Migration - Aaron Patterson

I am new!

Page 23: Design Summit - Rails 4 Migration - Aaron Patterson

LostPassport

Page 24: Design Summit - Rails 4 Migration - Aaron Patterson

Very 😴

Page 25: Design Summit - Rails 4 Migration - Aaron Patterson

I am new!

Page 26: Design Summit - Rails 4 Migration - Aaron Patterson

Upgrading to Rails 4.x

Page 27: Design Summit - Rails 4 Migration - Aaron Patterson

Upgrading to Edge Rails

Page 28: Design Summit - Rails 4 Migration - Aaron Patterson

Rails Release Timeline

Page 29: Design Summit - Rails 4 Migration - Aaron Patterson

4.2 is Very Soon™

Page 30: Design Summit - Rails 4 Migration - Aaron Patterson

Next is 5.0

Page 31: Design Summit - Rails 4 Migration - Aaron Patterson

Upgrading Rails

Page 32: Design Summit - Rails 4 Migration - Aaron Patterson

Why?

Page 33: Design Summit - Rails 4 Migration - Aaron Patterson

Performance

Page 34: Design Summit - Rails 4 Migration - Aaron Patterson

Memory Reduction

Page 35: Design Summit - Rails 4 Migration - Aaron Patterson

Cost

Page 36: Design Summit - Rails 4 Migration - Aaron Patterson

How?

Page 37: Design Summit - Rails 4 Migration - Aaron Patterson

Push upstream

Page 38: Design Summit - Rails 4 Migration - Aaron Patterson

Move to gems

Page 39: Design Summit - Rails 4 Migration - Aaron Patterson

Change implementation

Page 40: Design Summit - Rails 4 Migration - Aaron Patterson

Issues Today

Page 41: Design Summit - Rails 4 Migration - Aaron Patterson

Rails Fork

Page 42: Design Summit - Rails 4 Migration - Aaron Patterson

Rails Monkey Patches

Page 43: Design Summit - Rails 4 Migration - Aaron Patterson

Monkey Patches

Page 44: Design Summit - Rails 4 Migration - Aaron Patterson

Rails Fork

Page 45: Design Summit - Rails 4 Migration - Aaron Patterson

Active Record Object Allocation Time

Page 46: Design Summit - Rails 4 Migration - Aaron Patterson

ms = Benchmark.ms do records = rows.map { |model| # … }.uniq end

logger.debug(' %s Inst Including Associations (%.1fms - %drows)' % [join_base.active_record.name || 'SQL', ms, records.length])

Page 47: Design Summit - Rails 4 Migration - Aaron Patterson

We can’t upstream this :-(

Page 48: Design Summit - Rails 4 Migration - Aaron Patterson

ActiveSupport Notifications

Page 49: Design Summit - Rails 4 Migration - Aaron Patterson

ActiveSupport::Notifications.instrument( "some.key", payload ) do records = rows.map { |model| # ... }.uniq end

Page 50: Design Summit - Rails 4 Migration - Aaron Patterson

ActiveSupport::Notifications.subscribe('some.key' ) do |value| # same logger statement end

Page 51: Design Summit - Rails 4 Migration - Aaron Patterson

Truncate table

Page 52: Design Summit - Rails 4 Migration - Aaron Patterson

# Executes the truncate statement. def truncate(table_name, name = nil) execute("TRUNCATE TABLE #{quote_table_name(table_name)}", name) end

Page 53: Design Summit - Rails 4 Migration - Aaron Patterson

Pushed up stream

Page 54: Design Summit - Rails 4 Migration - Aaron Patterson

Added a monkey patch to backport

Page 55: Design Summit - Rails 4 Migration - Aaron Patterson

64 bit primary keys

Page 56: Design Summit - Rails 4 Migration - Aaron Patterson

In PG: "integer" == 32bit

Page 57: Design Summit - Rails 4 Migration - Aaron Patterson

In PG: "bigint" == 64bit

Page 58: Design Summit - Rails 4 Migration - Aaron Patterson

Default is 32bit

Page 59: Design Summit - Rails 4 Migration - Aaron Patterson

Why?

Page 60: Design Summit - Rails 4 Migration - Aaron Patterson

¯\_(ツ)_/¯

Page 61: Design Summit - Rails 4 Migration - Aaron Patterson

Schema.rb

create_table "accounts" do |t| t.string "name" t.integer "acctid" end

No PK

declared

Means32bit

Page 62: Design Summit - Rails 4 Migration - Aaron Patterson

We can upstream this

Page 63: Design Summit - Rails 4 Migration - Aaron Patterson

Determine backwards compat scheme

Page 64: Design Summit - Rails 4 Migration - Aaron Patterson

Default PK sequences

Page 65: Design Summit - Rails 4 Migration - Aaron Patterson

+ return if options[:id] == false + return unless self.respond_to?(:set_pk_sequence!) + + value = ActiveRecord::Base.rails_sequence_start + set_pk_sequence!(table_name, value) unless value == 0

Page 66: Design Summit - Rails 4 Migration - Aaron Patterson

We can’t upstream this

Page 67: Design Summit - Rails 4 Migration - Aaron Patterson

Extract and "super"

Page 68: Design Summit - Rails 4 Migration - Aaron Patterson

Add "extension" points.

Page 69: Design Summit - Rails 4 Migration - Aaron Patterson

Auto-reconnect

Page 70: Design Summit - Rails 4 Migration - Aaron Patterson

Supposedly fixed?

Page 71: Design Summit - Rails 4 Migration - Aaron Patterson

+ def self.did_retry_db_connection(connection,count) + logger.info "CONNECTION RETRY: #{connection.class.name} retry ##{count}." if logger + end

Page 72: Design Summit - Rails 4 Migration - Aaron Patterson

Push Up Better Exceptions

Page 73: Design Summit - Rails 4 Migration - Aaron Patterson

Connection Extensions

Page 74: Design Summit - Rails 4 Migration - Aaron Patterson

Database Statistics

Page 75: Design Summit - Rails 4 Migration - Aaron Patterson

Create a new gem: activerecord-pg-stats

Page 76: Design Summit - Rails 4 Migration - Aaron Patterson

Virtual Columns

Page 77: Design Summit - Rails 4 Migration - Aaron Patterson

Declaration

virtual_column :name, :type => :string

Page 78: Design Summit - Rails 4 Migration - Aaron Patterson

2 Responsibilities

Page 79: Design Summit - Rails 4 Migration - Aaron Patterson

Declare a "column type" for reporting

Page 80: Design Summit - Rails 4 Migration - Aaron Patterson

Declare associations used by the method

Page 81: Design Summit - Rails 4 Migration - Aaron Patterson

Separate these concerns

Page 82: Design Summit - Rails 4 Migration - Aaron Patterson

Extract reporting

Page 83: Design Summit - Rails 4 Migration - Aaron Patterson

Push up relation declaration.

Page 84: Design Summit - Rails 4 Migration - Aaron Patterson

RJS

Page 85: Design Summit - Rails 4 Migration - Aaron Patterson

Land the Angular patch!!!!

Page 86: Design Summit - Rails 4 Migration - Aaron Patterson

jquery-rjs

Page 87: Design Summit - Rails 4 Migration - Aaron Patterson

prototype-rails

Page 88: Design Summit - Rails 4 Migration - Aaron Patterson

Monkey Patches

Page 89: Design Summit - Rails 4 Migration - Aaron Patterson

First Glance

Page 90: Design Summit - Rails 4 Migration - Aaron Patterson

Churn

Page 91: Design Summit - Rails 4 Migration - Aaron Patterson

"How often is stuff changing?"

Page 92: Design Summit - Rails 4 Migration - Aaron Patterson

git log --all -M -C --name-only --format='format:' "$@" | sort | grep -v '^$' | uniq -c | sort -n | awk 'BEGIN {print "count\tfile"} {print $1 "\t" $2}'

Page 93: Design Summit - Rails 4 Migration - Aaron Patterson

445 vmdb/app/models/miq_server.rb 498 vmdb/app/models/miq_provision_workflow.rb 508 vmdb/app/models/miq_provision.rb 590 vmdb/app/models/miq_report.rb 720 vmdb/app/controllers/report_controller.rb 777 vmdb/app/models/vm.rb 802 vmdb/app/models/host.rb 825 vmdb/app/helpers/application_helper.rb 1186 vmdb/app/controllers/application.rb

Page 94: Design Summit - Rails 4 Migration - Aaron Patterson

Cyclomatic Complexity

Page 95: Design Summit - Rails 4 Migration - Aaron Patterson

Quantify Code Complexity

Page 96: Design Summit - Rails 4 Migration - Aaron Patterson

flog

Page 97: Design Summit - Rails 4 Migration - Aaron Patterson

1965.9: vmdb/db/migrate/20100302205959_collapsed_initial_migration.rb:7 1131.5: vmdb/app/controllers/ops_controller/settings/common.rb:621 1024.9: vmdb/app/controllers/application_controller/filter.rb:137 951.9: vmdb/app/controllers/application_controller/performance.rb:220 862.9: vmdb/spec/models/ems_refresh/refreshers/vc_refresher_spec.rb:27 842.8: vmdb/app/controllers/ops_controller/settings/common.rb:881 824.5: vmdb/spec/models/ems_refresh/refreshers/rhevm_refresher_3_1_spec.rb:230 791.5: vmdb/app/helpers/application_helper.rb:648

Page 98: Design Summit - Rails 4 Migration - Aaron Patterson

application_helper $$$$$$

Page 99: Design Summit - Rails 4 Migration - Aaron Patterson

Static Analysis

Page 100: Design Summit - Rails 4 Migration - Aaron Patterson

Find unused methods

Page 101: Design Summit - Rails 4 Migration - Aaron Patterson

Ripper

Page 102: Design Summit - Rails 4 Migration - Aaron Patterson

require 'ripper'

class MyParser < Ripper def on_def name, _, _ # do something end end

parser = MyParser.new "some code" parser.parse

Page 103: Design Summit - Rails 4 Migration - Aaron Patterson

class MyParser < Ripper attr_reader :methods, :calls

def initialize code, file, line super @methods = [] @calls = [] end

def on_def name, *rest @methods << name end alias :on_command :on_def

def on_defs target, dot, name, *args @methods << name end

def on_call target, dot, name @calls << name end

def on_fcall name @calls << name end alias :on_symbol :on_fcall alias :on_vcall :on_fcall end

Page 104: Design Summit - Rails 4 Migration - Aaron Patterson

Find.find(ARGV[0]) do |file| next if File.directory? file ext = file[/(?<=\.)\w+$/] next unless ext case ext when 'rb' parser = MyParser.new(File.read(file), file, 1) parser.parse when 'erb' parser = MyParser.new ERB.new(File.read(file)).src, file, 1 parser.parse end end

Page 105: Design Summit - Rails 4 Migration - Aaron Patterson

Dynamic Dispatch

superclass.send("virtual_#{m}")

Page 106: Design Summit - Rails 4 Migration - Aaron Patterson

Static Analysis + Ruby = 😰

Page 107: Design Summit - Rails 4 Migration - Aaron Patterson

Probably also why Brakeman isn’t perfect

Page 108: Design Summit - Rails 4 Migration - Aaron Patterson

This Summit is GREAT!

Page 109: Design Summit - Rails 4 Migration - Aaron Patterson

Communication

Page 110: Design Summit - Rails 4 Migration - Aaron Patterson

Browser

WebServer

Memcached

Page 111: Design Summit - Rails 4 Migration - Aaron Patterson
Page 112: Design Summit - Rails 4 Migration - Aaron Patterson

Every 500msBrowser

WebServer

Memcached

Page 113: Design Summit - Rails 4 Migration - Aaron Patterson

Every 500msBrowser

WebServer

Memcached

Page 114: Design Summit - Rails 4 Migration - Aaron Patterson

Cache Size

Page 115: Design Summit - Rails 4 Migration - Aaron Patterson

Code Complexity

Page 116: Design Summit - Rails 4 Migration - Aaron Patterson

Bugs

Page 117: Design Summit - Rails 4 Migration - Aaron Patterson

Server Load

Page 118: Design Summit - Rails 4 Migration - Aaron Patterson

Every 500msBrowser

WebServer

Memcached

Page 119: Design Summit - Rails 4 Migration - Aaron Patterson

Always Question Assumptions

Page 120: Design Summit - Rails 4 Migration - Aaron Patterson

Ask Why

Page 121: Design Summit - Rails 4 Migration - Aaron Patterson

Business Requirements aren’t set in stone.

Page 122: Design Summit - Rails 4 Migration - Aaron Patterson

Thanks Everyone!

Page 123: Design Summit - Rails 4 Migration - Aaron Patterson

Questions?