Ansible, Vagrant and Riak

Bryan Hunt

Basho Technologies

2

Introduction and overview

• Basho CSE (Client Services Engineer)

• Support Riak and Riak CS for EMEA customers

• Linux since 90’s

• Mac since 2013

• Ansible since March 2013

• Vagrant since December 2013

• Delft aficionado

Who am I ?

Who are Basho?

• The creators and developers of Riak & Riak CS

• Founded in 2008 by ex-Akamai staff

• Experts in distributed systems

• Offices across the USA, EMEA & Japan

• Providing Professional Services, Customer Support

• Ansible user

• Curious about Riak

• Vagrant user

• All or any of the above

Who are you?

• OSX or Linux

• Vagrant

• Ansible

• Virtualbox

Requirements

What is Riak? (in 2 minutes or less)

• Distributed Key-Value store

• Written in Erlang

• 2i, MapReduce, Link Walking, Solr Integration

• 2.0 out soon

What is Riak?Riak is a fast, reliable, distributed and highly

available key-value store.

Bucket

key

key

key

key value

value

value

value

What is Riak?

Alternate models for Riak local deployment

make devrel NODES=3•Compiling on the host machine

•Requires C compiler

•Requires Erlang installation

•No VM required

•Supported by Makefile

•Low overhead

•Fast

Development OS

ClientHAProxyRiak 2

Riak 1

Riak 3

Riak VM per instance•Requires a lot of RAM

•Can be slower to start a cluster

•Fairly realistic

•Build on a Linux box, nicer than OSX toolchain

Host OS

Vagrant

Virtualbox

Linux VM

Riak 1

Linux VM

Riak 2

Linux VM

Riak 3

HAProxy

Client

Riak CS

•Lots more moving parts

•Minimal cluster requires 7 services on 3 machines

•Riak (storage)

•Riak CS (S3 API implementation)

•Stanchion (serialises user/bucket operations)

Host OS

HAProxy

HAProxy

Vagrant

Virtualbox

Linux VM 2

Riak 2

Riak CS

Linux VM 3

Riak CS

Stanchion

Riak 3

Linux VM 1

Riak 1

Riak CS

(Magnus Gjoen 2011)

Deployment

Riak CS deploy (phase 1)• Install package on each machine

• Apply configuration on each machine

• Start each node

Linux VM 3

Riak CS 3

Stanchion

Riak 3

Linux VM 2

Riak 2 Riak CS 2

Linux VM 1

Riak 1

Riak CS 1

Installation Phase

1

2

3

12

1

2ansible-playbook

Riak Cluster Creation

•bin/riak-admin cluster join riak<n>@192.168.1.10

•bin/riak-admin cluster plan

•bin/riak-admin cluster commit

For each node in the ring

Linux VM 3

Riak 3

Linux VM 2

Riak 2

Linux VM 1

Riak 1

2 (join riak2)

1 (join riak1)

What is Ansible

• Open-source

• Orchestration engine

• Configuration manager

• Software deployer

• Started in Feb 2012

What made me interested in Ansible?

• Didn’t want to install a client before provisioning a box

• Didn’t like running a http server as root (http://bit.ly/webmin_goodies)

• Minimal syntax, Ansible uses YAML

• Wanted things to occur in a specific sequence

• Liked the idea of using the SSH protocol

• Liked the idea of pushing changes to servers

Why does Basho use the Ansible/Vagrant combo?

• Simple to configure and provision

• Reduces Amazon expenditure

• Quickly deploy new versions onto local VM

• Reproduce strange customer configurations

• Provision Virtualbox images for training/demo’s

Ansible Roles

•Self contained reusable modules

•Can have dependencies upon other roles

•Similar to Chef recipes or Puppet modules

•Stored at https://galaxy.ansible.com/

Ansible Riak Roles

•Created by James Martin (now at Ansible)

•http://bit.ly/galaxy_basho

•Actively maintained and developed by Basho

•Volunteers?

Ansible Modules

shell mount

yum service

riakapt

Ansible Riak Roles

riakcs

riakcs-packages

riak-common

riak stanchion

depends

depends depends depends

Ansible

Inventory

reads

reads

provisions

PlaybooksPlaybooks

Playbooks

host1host1host1host1

Ansible basic workings

Ansible minimal example

Ansible example using Role

Walkthrough

Building a Riak CS cluster

Structure of example apps

•Vagrantfile

•form_cluster.yml

•hosts

•roles

•roles.txt

•setup_riak.yml

Vagrant file ensuring roles

Future goals

• Create a reusable set of base boxes

• Create a Vagrant script for each release

• Create new roles for Ansible 2.0

Try it out!

• http://bit.ly/ansible-riak-example

• http://bit.ly/ansible-riakcs-example

• https://galaxy.ansible.com

• https://www.vagrantup.com