Multi-Provider Vagrant: AWS, VMware & More - PuppetConf 2013

Living in a Multi-Provider World

Friday, August 23, 13

I’m Mitchell HashimotoAlso known as @mitchellh


I build toolshttp://hashicorp.com


Vagranthttp://www.vagrantup.com


Packerhttp://www.packer.io


http://vagrantup.com


I think you know what this nowadays...


SUCCESS!


You probably should really be using this nowadays to be working with and testing Puppet modules...


State of the Vagrant


Vagrants are still struggling to find work, shelter, and food.


Vagrant!


Mature, stable, proven.Development since Jan 2010.Used by thousands of companies.


Vagrant


Puppet


Create and Manage Virtualized Development Environments

Mitchell Hashimoto

VagrantUp and Running

http://hashi.co/vagrant-book


Releases in 2013: 14


Vagrant 1.1+ was finally released in March. A year in development.


SUCCESS!


Things are good.


Things are getting

BETTER!


Vagrant 1.1+


I hear you like VirtualBox...


Providers.


a.k.a.No more VirtualBox.


(Don’t worry, Vagrant still works with VirtualBox)


BUT NOW IT WORKS WITH:

AWS, DigitalOcean, HP Cloud, Joyent, KVM, libvirt, lxc, OpenStack, Rackspace, VMware.


Mother of God...


Vagrant 1.1: March 2013


We’re now on 1.2.7.(14 releases later)


The same Vagrant you know and love, butso much better.


Backwards Compatible*Just “vagrant up” your 1.0.x Vagrantfiles.

* Plugins from 1.0.x aren’t backwards compatible.


VAGRANT_API_VERSION = “2”

Vagrant.configure(VAGRANT_API_VERSION) do |config| config.vm.box = “precise64”end

A Vagrantfile


Installer only.“gem install” folks living in the past, take note.

http://downloads.vagrantup.com


Path to 2.0Guaranteed Vagrantfile stability, widespread use.


But 1.2.x is quite goodThe core is much stronger, many subtle issues fixed, latest support for various host/guest combos, NFS is way better, etc.


Vagrant Future


Going to get way better.This is the part where I tease you.


Windows VM support“vagrant up” Windows VMs with ease.


Box versioningNo more “am I using the latest box?”


Docker ProvisionerYes, I’ll take a RabbitMQ with a side of Nginx.


Superpowered “package”Powered by Packer, to be discussed.


Automated box buildingPACKER!


Other stuff...I think I’ve said enough. Get excited.


Multi-ProviderWhat does it mean? Why?


A provider manages compute resources* for Vagrant machines.

* But also sets up networking and some basic storage too.


Vagrant 1.0: VirtualBox was the only “provider”


Why?Because you didn’t love VirtualBox hard enough.


VirtualBox isn’t great for every situation.


Honestly: VirtualBox is bad.* Its okay, until you realize almost anything else is way better.


Multiple providers enable Vagrant to do new and awesome things.


People love Vagrant for the workflow. Not for VirtualBox.


Other providers let you have that workflow in an environment that works best for you.


New use cases...Vagrant where Vagrant has never gone before.


Continuous IntegrationAWS, LXC, etc. Fantastic.


Develop locally. Test remotely.Work in VirtualBox. Test in AWS (more prod-like).


Vagrant in VagrantVMware outside. LXC inside. Etcetera.


Corporate EnvironmentsMaximize that VMware investment.


etc...That was just a taste.


Multi-ProviderHow do I use it?


vagrant up --provider=fooWhere foo is “vmware_fusion”, “aws”, etc.


Same Vagrantfile,multiple providers.Pretty magical once you see it.


Vagrant.configure(“2”) do |config| config.vm.box = “precise64”end

A Vagrantfile


$ vagrant box add precise64 \ http://files.vagrantup.com/precise64.box...

$ vagrant box add precise64 \ http://files.vagrantup.com/precise64_vmware_fusion.box...

Some Boxes


http://files.vagrantup.com/precise64_vmware_fusion.box

http://files.vagrantup.com/precise64_vmware_fusion.box

$ vagrant up --provider=virtualbox...

OR$ vagrant up --provider=vmware_fusion...

Up in VirtualBox or VMware


It’s that easy.


Best effort.Not every provider can satisfy every Vagrant abstraction, but it won’t fail if it can’t.

Example: AWS networking doesn’t map well to Vagrant networking.

Vagrant + AWS will just ignore networking configurations. Not error. It’ll make a “best effort” to work.


BoxesThey’re now tied to providers.


$ vagrant box listcentos (virtualbox)precise64 (aws)precise64 (virtualbox)precise64 (vmware_fusion)

Tied to a Provider


You need a box for each provider.Packer to the rescue!


VirtualBox: OVF exportVMware: VMX exportLXC: rootfs tarballAWS: Metadata (AMI info)

What’s in a box?

It varies by provider. Actually, anything can be in a box. The provider is responsible for reading and verifying structure.


Provider-Specific ConfigThe full power of the provider, if you need it.

Abstractions are nice, but sometimes you want to take advantage of specific properties of a provider. For example, AWS can do things VMware can’t, and vice versa.

Provider-specific config lets you do that.


PowerfulA place to expose full capabilities of underlying providers.


PortableEven if the user doesn’t have that provider installed, the Vagrantfile will still work.


Clear PurposeIt makes it clear that that configuration applies ONLY to specific providers.


Totally OptionalJust use a normal Vagrantfile if you don’t care.This is for power users.


That’s it for providers.Same powerful workflow, clean abstractions, uncompromised flexibility.


Real world: People usually choose one provider and stick with it. The important part is you now have choice.


PackerFriday, August 23, 13

Automatically create machine images for multiple platforms from a single source config


Machine image: A single deployable unit that contains a pre-configured OS and software.


AWS: AMIVMware: VMX + disksVirtualBox: OVF + disksDigitalOcean: Snapshots


Useful on its own, without Vagrant. But also very useful with Vagrant.


Production and dev are different providers.

Case One


“vagrant up” takes too long.

Case Two


Multi-provider Vagrant, but want identical images.

Case Three


Packer images solve these three cases (and way more).


Ew... Images.The anti-image stigma


DevOps has historically been against machine images.


Important to understand why DevOps has historically been against machine images.


Golden images used to be the way.


Quarterly, unchanged, blessed image.


Getting any changes (ops or dev) in was slow and frustrating.


Necessary evil while tooling wasn’t as good as it is today.


With modern config management, easy to simply ignore images.


Machine images also have a ton of benefits.


Super fast infrastructure deployment.


Multi-cloud portability


Stability and Testability


A Modern ToolEmbrace best practices.


Packer brings all the benefits of machine images without the downsides.


AutomationNo human interaction. Great for CI.


Config managementUse Puppet to configure the image, run Puppet after it boots still. Lose nothing.


RepeatabilityTemplate goes into version control. Image creation knowledge is now in code.


ExampleDocker-ready AMI


Template{ "variables": { "aws_access_key": "", "aws_secret_key": "" },


Template "builders": [{ "type": "amazon-ebs", "access_key": "{{user `aws_access_key`}}", "secret_key": "{{user `aws_secret_key` }}", "region": "us-east-1", "source_ami": "ami-23d9a94a", "instance_type": "m1.small", "ssh_username": "ubuntu", "ami_name": "packer-docker {{timestamp}}" }],


Template "provisioners": [{ "type": "shell", "scripts": [ "scripts/kernel.sh", "scripts/docker.sh" ] }]}


BUILD!$ packer build \ -var ‘aws_access_key=YOUR KEY’ \ -var ‘aws_secret_key=YOUR SECRET’ \ template.json...


A few minutes later...


Easiest AMI creation ever? Just the beginning.


Thank you.

vagrantup.com packer.io


Multi-Provider Vagrant: AWS, VMware & More - PuppetConf 2013

Technology

Transcript of Multi-Provider Vagrant: AWS, VMware & More - PuppetConf 2013