MCollective as an integration layer

Jon Topper The Scale Factory

MCollective

a framework to build server orchestration

or parallel job execution systems.

“

”

MCollective

not simply a fancy SSH “for loop”“

”

MCollective

Servers

MiddlewareClient

MCollective

Servers

MiddlewareClient

Finding Servers

Check Service Status

Check Package Versions

Still looks a bit like a fancy SSH “for loop”

a framework to build server orchestration

or parallel job execution systems.

“

”

Zabbix

• Distributed monitoring system

• Fetches time series data - can alert on anomaly

• Uses MySQL

• Multiple data sources:

• Zabbix agent / sender

• SNMP poll / traps

• JMX

Zabbix

Zabbix

ZCollective

zcollective —zabbix-api-url \! http://localhost/api_jsonrpc.php

ZCollective

include MCollective::RPC!!mc = rpcclient("rpcutil", :debug => true)!!mc.discover.sort.each do |host|!! inventory = mc.custom_request( ! "inventory", {}, host,! { "identity" => host }! ).first!! do_some_zabbix_magic(inventory)!!end

ZCollective

def do_some_zabbix_magic(inventory)!! # If the host isn’t added to Zabbix! # Add the host using the API! # For each Puppet class on this host! # If there’s a Zabbix template by this name! # Assign that template to the host.! # end! # end! # end!! # If the host is already in Zabbix! # Make sure all templates correctly assigned! # end !!end

ZCollective

ZCollective

ZCollective

https://github.com/scalefactory/zcollective

Traditional Deployment

sf-deploy

Latest: Deployed:

aaa1 aaa1

sf-deploy

Latest: Deployed:

aaa1 aaa1

sf-deploy

Latest: Deployed: aaa1

aaa2

sf-deploy

Latest: Deployed:

aaa2 aaa2

Code

class MCollective::Application::Sfdeploy < MCollective::Application!! def main!! action = configuration[:action]!! mc = rpcclient("sfdeploy")!! printrpc mc.send(! action,! :application => configuration[:application],! :branch => configuration[:branch],! :tag => configuration[:tag],! :groups => configuration[:groups],! :options => options! )!! printrpcstats! end

Code

module MCollective! module Agent! class Sfdeploy < RPC::Agent!! all_actions.each do |act|! action act do!! sf_deploy = config.pluginconf["sfdeploy.binary"] || "sf-deploy"! deploy_user = config.pluginconf["sfdeploy.user"]|| nil!! command = "#{sf_deploy} -a #{request[:application]} -v "! command << "-b #{request[:branch]} " if request[:branch]! command << "-t #{request[:tag]} " if request[:tag]! command << "-g #{request[:groups]} " if request[:groups]! command << act!! if deploy_user! to_run = "su #{deploy_user} -c \"#{command}\""! else! to_run = command! end

ZCollective

https://github.com/scalefactory/sf-deploy

Resources

http://docs.puppetlabs.com/mcollective/

http://www.scalefactory.com/

jon@scalefactory.com

@jtopper

scalefactory