Habitat Workshop at Velocity London 2017
-
Upload
mandi-walls -
Category
Software
-
view
163 -
download
2
Transcript of Habitat Workshop at Velocity London 2017
Mandi Walls
Technical Community Manager for EMEA
@lnxchk
https://www.habitat.sh/
http://slack.habitat.sh/
Ian Henry @Eeyun___ Habitat Community lead
EVERY business is a software business
We’re going to be a software
company with airplanes.– CIO, Alaska Airlines
We Run InfrastructureTo Run Applications
That the Business Requires
How Do We Run Applications?
• On a computer
• With an OS
• And some libraries
• And some configuration
• And some way to start it and stop it
That isn’t managing complexity.
It is Adding to it.Or at least just moving it
around.
So. Habitat.
• Reduce snowflakeness
• Support microservices
• Manage container creep
https://www.bonanza.com/listings/Premier-Food-Storage-Containers-20-Piece-Set-Grey/443972348
Modern Applications Are Trending Toward
• Immutability
• Platform agnosticism
• Complexity reduction
• Scalability
https://amazingmusthaves.com/products/steel-insulated-food-containers/
Ways of Working with Habitat
• Artifacts run themselves via the supervisor
• Export to Docker
• Link your github to habitat builder and let us do the
work
• Build plans from scratch or leverage scaffolding
Habitat Goals
• Defer some decisions to runtime
• Do clean room builds
• Repeatable builds
• Distro agnostic packaging system
• Service runtime and discovery
• Configuration exposed via API
• Packages are signed by the system
Single Artifact Demo - Habitatize
• Ruby app with HTTP interface, ImageMagick
• Built via scaffolding
• Run locally
• Login to your machine
User: velocity
Pass: London2017
Run the Package
•sudo hab start learn-chef/habitatize -
-channel unstable
• Open http://{your machine IP}:8000 in a web
browser
Upload a Pic!
What’s in This Thing?
• https://github.com/burtlo/habitatize
pkg_name=habitatize
pkg_origin=franklinwebber
pkg_scaffolding="core/scaffolding-ruby"
pkg_version="1.0.0"
pkg_deps=(franklinwebber/imagemagick)
Basic Parts of a Simple Habitat App
• Your application code
• A plan file – can be as short as a few lines
Include dependencies or use a scaffolding package
• Habitat installation for running it
(and build, we’ll do that later)
• That’s it!
• Stop with
sudo hab stop learn-chef/habitatize
Dependencies?
Community
• Teams and individuals contribute packages to
builder
• The “core” packages are maintained by the Habitat
team
• You can use any public package on builder for your
dependencies
• Packages are libraries, runtimes, applications
Habitat and BuilderApplication Automation Build Machine
Lab: Habitat and Builder
• The fastest way to get packages out of the Habitat
workflow is with Builder
• SaaS for automated build and publish
• Code -> Github -> Builder -> Docker Hub
• You’ll need Github and Docker Hub accounts
No worries, you can get them as we go along
Our Sample Application
• https://github.com/habitat-sh/sample-node-app
• Fork this to your github account
Origins
• Are like organizations for grouping habitat
applications
• You can publish to multiple origins as long as you
have the keys – we’ll talk about that later
• For now, create a new unique origin to use
“username-velocitylondon”
• Origins can be public or private
For this lab it doesn’t matter
Make the App Our Own
• In github, edit inline in the browser
• https://github.com/YOU/sample-node-
app/blob/master/habitat/plan.sh
• Change
pkg_origin
To your new origin
• Commit changes, add a comment if you want
Add Habitat Builder
• As an app in github
• https://github.com/apps/habitat-builder
• Install
• Choose repositories
All, or limit to just the sample-node-app repo
Add Docker Hub Integration
• We’ll link Docker Hub to our origin on habitat builder
Docker Hub
• https://hub.docker.com/
• Sign up for an account if you don’t have one
• Each time you create a new origin, you’ll need to
hook it to Docker Hub
Automatic Builds of Our Sample App
• Habitat Builder will watch our repo for changes
• New builds of our app will then initialize
• We connect Builder to github from “Packages” on
the Builder page
Turn on “Publish to Docker Hub”
Build It!
Our App Shows Up In Docker Hub
• Builder retrieved our code from github
• Built the latest version
• Exported to a Docker container
• Published to Docker Hub
Run Our App with Docker
sudo docker pull your-docker-org/your-
docker-repo
sudo run -it -p 8000:8000 your-docker-
org/your-docker-repo
Open {your IP}:8000/ in a browser
(Make sure you stopped habitatize)
sudo hab stop learn-chef/habitatize
Make a New Version
• On github, update your plan.sh file to show a new
version of your app – 1.0.2
• Commit your change
• Builder will pick up your change and start a new
build
Habitat StudioBuild Locally
Application Life
Source
Code
RepoArtifact
Bare Metal
Container
Cloud Instance
VM
Artifact
Repo
Habitat Studio
• Provides a busy box clean
room for your app
• Plus a set of tools for
manipulating and running
harts
Why the Studio?
• Declare explicit dependencies
• Ship exactly what you need
• Sign your packages and store artifacts
Habitat Plans
• Plan files are where you put together your builds
• They are bash
• Plans are versioned and can be shared on the
habitat depot
• Plans have multiple sections for your needs
What’s In A Plan?
pkg_origin=firstnamelastname
pkg_name=national-parks
pkg_description="A sample JavaEE Web app deployed in Tomcat8"
pkg_version=0.1.3
pkg_maintainer="First Last <[email protected]>"
pkg_license=('Apache-2.0')
pkg_source=https://github.com/billmeyer/national-parks/archive/v${pkg_version}.tar.gz
pkg_shasum=56a70d7fc432ad275ba256150bd010ce2581346c848e40ddc0fd0f62c5b17a54
pkg_upstream_url=https://github.com/billmeyer/national-parks
pkg_deps=(core/tomcat8 billmeyer/mongodb)
pkg_build_deps=(core/git core/maven)
pkg_expose=(8080)
pkg_svc_user="root"
pkg_svc_group="root"
Plans Have Callbacks
Shell functions executed when building
artifacts:
• do_begin()
• do_download()
• do_verify()
• do_check()
• do_clean()
• do_unpack()
• do_prepare()
• do_build()
• do_install()
• do_strip()
• do_end()
Variables
• HAB_CACHE_SRC_PATHThe default path where source archives are downloaded, extracted, & compiled.
• ${pkg_dirname}Set to ${pkg_name}-${pkg_version} by default. If a .tar file extracts to a directory that's different from the filename, then you would need to override this value to match the directory name created during extraction.
• ${pkg_filename} Set manually in the plan.sh
• ${pkg_version} Set manually in the plan.sh
Habitat Tools for Dependencies
•hab pkg path team/app
• You can have build deps and runtime deps
• Build deps get into your hab studio when you are
building
• Runtime deps come along into your hart
Keep artifacts small and under
control!
Scaffolding, like for habitatize
• Default core-built dependencies for common
runtimes
• Ruby and Node so far
Go, Python, Java on the way!pkg_name=MY_APP
pkg_origin=MY_ORIGIN
pkg_version=MY_VERSION
pkg_scaffolding=core/scaffolding-ruby
What Gets Built?
• Everything. Sort of.
• Build your own apps from source
• Decide if you want upstream binaries or source for
things like runtime
You don’t have to build Tomcat, but you can
• For COTS, use the binaries and skip steps
Habitat Hooks
• Travels with the app
• Tell the app how to start, and if there is something
that needs to be done before start
• Let’s look at a run hook file
Run Hook for Tomcat#!/bin/bash
exec 2>&1
echo "Starting Apache Tomcat"
export JAVA_HOME=$(hab pkg path core/jdk8)
export TOMCAT_HOME="$(hab pkg path core/tomcat8)/tc"
source {{pkg.svc_config_path}}/catalina-opts.conf
echo "\$CATALINA_OPTS=$CATALINA_OPTS"
cp "$(hab pkg path {{pkg.origin}}/{{pkg.name}})/{{pkg.name}}.war" "$TOMCAT_HOME/webapps/"
exec ${TOMCAT_HOME}/bin/catalina.sh run
Configuration
• Can be manipulated at runtime
• Also travels with the app
• Provides variable substitution and templating
A catalina-opts File
{{~#if bind.database}}{{~#each bind.database.members}}
export CATALINA_OPTS="-DMONGODB_SERVICE_HOST={{address}} -DMONGODB_SERVICE_PORT={{port}}"
{{~/each}}{{~/if}}
Builder
• You can share plans with builder, and other hab
users share theirs
• Has team namespacing – our origin
• The core plans are those built by the Habitat team
• https://app.habitat.sh/
Caveat - Internet
• You can build your own stuff inside your own
network, sort of, when it’s all on one machine
• This isn’t the best way to use Habitat
• You’ll want internet connectivity on your build box
and on the hosts running habitat artifacts
• Builder provides private origins, linked to github
users
Build Output
• By default, it’s a hart – a compressed tarball with
some metadata and a signature
• The hart itself it runnable
• You can export to other formats, like Docker
containers like we saw eariler
Runtime
• The hab runtime includes management, service
discovery, other features
• The habs in your application create a mesh so they
can talk to each other
• You can even update your application via the mesh
without restarting every application manually
Running a Hart
sudo hab start learn-chef/habitatize --
channel unstable
sudo hab start nathenharvey/national-parks --
listen-gossip 0.0.0.0:9639 --listen-http
0.0.0.0:9632 --peer 127.0.0.1 –bind
database:mongodb.default
Setup
• On your remote host
•hab setup
• Set up a default origin? Yes
Enter what you used in builder – username-velocity
• Create a key – Yes
• Github access token!
Github Access Token
• Go to your account on github
• Select “settings” from your account on the top right
• Click “Developer Settings” on the left bar
• Click “Personal access tokens” on the left bar
• “Generate new token” button
• Description “habitat” or whatever
Select read:org and user:email
• Generate Token
Hab Setup continued
• Copy the github access token into the hab setup
HAB_AUTH_TOKEN
• Your github auth token
• Export as an environment variable
• Can be added to ~/.bashrc
•export HAB_AUTH_TOKEN=3db7f……yourtoken
Add Keys
• We generated new keys
• We need to add them to Builder
• From your host, run
hab origin key upload –s {YOUR ORIGIN}
A place to work
git clone
https://github.com/YOU/sample-node-
app.git
cd sample-node-app
• Your apps have a habitat/ subdirectory for hab-
related files
Hab Studio
•hab studio enter
•build
•hab svc start ORIGIN/sample-node-app
Habitat Supervisor
• Keeps an eye on your applications
• Also provides and API for viewing what’s going on
with the apps running in the supervisor
•http://{Your IP}:9631/services
• It’s running now for you as part of the studio, but
doesn’t need the studio to run
When you exit the studio, it will shut down
• Also allows for applications to connect to each other
Two-Tier Application: National Parks
• Java on Tomcat with MongoDB
• https://blog.chef.io/2016/09/07/deploy-java-web-
app-on-tomcat-8-with-habitat/
• You’ll want a couple terminals open to your host
App
Decomposing into Habitat
• Microservice architectures lend themselves well to
Habitat
Every service becomes a hab package
• For generic services like databases, the initial
package should be very basic
• Your application layer or a separate package
initializes the data
MongoDB
• Already built – we’re just going to use it
sudo hab start nathenharvey/mongodb --
channel unstable
https://bldr.habitat.sh/#/pkgs/nathenha
rvey/mongodb/latest
Java App
• https://bldr.habitat.sh/#/pkgs/nathenharvey/national-
parks/latest
• Let’s look at the plan.sh a bit
Where Is the Code Coming From
• This app isn’t hooked into the Builder pipeline
• The source code is downloaded from github into the
studio when the package is built
• A checksum can be included in the plan.sh
pkg_deps and pkg_build_deps
• Build deps are present in the studio or build
environment but don’t make it into the package
• In this case, we’re not taking Maven with us!
• Also notice the dependency on mongodb!
Finding Stuff that Lives in Other Packages
• Check out the setting for JAVA_HOME
•export JAVA_HOME=$(hab pkg path
core/jdk8)
• Java will be included in our package via the
dependencies
• It arrives via another habitat package
• The filesystem locations are different, so we have to
query for where java will live
Let’s Start the App
•sudo hab start nathenharvey/national-
parks --channel unstable
• The log output in this case will go to the MongoDB
screen (STDOUT for the supervisor)hab-sup(MR): Unable to start nathenharvey/national-
parks, hab-
sup(SS)[src/manager/service/spec.rs:208:23]: Missing
required bind(s), database
Supervisor Can Help Us
Service
Discovery
Service
Groups
Service Groups
• Applications running in the supervisor belong to
service groupsmongodb.default(SR): Configuration recompiled
mongodb.default(SR): Initializing
mongodb.default(SV): Starting service as user=hab,
group=hab
• When connecting services, use the service group
• Groups can be 1 or N instances of an application
Uh Oh
national-parks.default(SR): Initializing
national-parks.default hook[init]:(HK): Seeding Mongo Collection
national-parks.default hook[init]:(HK): $MONGOIMPORT_OPTS=--host= --
port=27017
national-parks.default hook[init]:(HK): 2017-10-17T17:52:35.106+0000
connected to: localhost:27017
national-parks.default hook[init]:(HK): 2017-10-17T17:52:35.106+0000
dropping: demo.nationalparks
national-parks.default hook[init]:(HK): 2017-10-17T17:52:35.109+0000 Failed:
not authorized on demo to execute command { drop: "nationalparks" }
national-parks.default hook[init]:(HK): 2017-10-17T17:52:35.109+0000
imported 0 documents
national-parks.default(HK): Initialization failed! 'init' exited with status code 1
What is This?
• Our app brings along its seed data
• It needs permissions to load that data into
MongoDB
• The app isn’t authorized to do that!
YOLO!
• We’re going to open up mongo to the wild. It’s fine.
• Let’s see how MongoDB is currently configured
hab sup config nathenharvey/mongodb
• In the middle are some security settings[mongod.security]
cluster_auth_mode = "keyFile"
javascript_enabled = true
key_file = ""
Tunables
• The supervisor allows us to update application
config at runtime
• Those mongod.security settings can be changed on
the fly
Make a new tunable toml
hab sup config nathenharvey/mongodb >
user.toml
Edit user.toml
• Around line 46 are the mongod.security settings
• Update cluster_auth_mode to “”
46 [mongod.security]
47 authorization = "disabled”
48 cluster_auth_mode = "”
49 javascript_enabled = true
50 key_file = ""
hab config apply
• Load this new config into the running mongodb:
sudo hab config apply mongodb.default 2 user.toml
• The serial number just has to increase; it’s not
otherwise meaningful
• Must be valid configuration
• The app will reload
Now try the Java again
sudo hab start nathenharvey/national-
parks --peer 127.0.0.1 --bind
database:mongodb.default
Check the Output
national-parks.default(O): 17-Oct-2017 18:07:58.322 INFO
[main] org.apache.catalina.startup.Catalina.load
Initialization processed in 525 ms
national-parks.default(O): 17-Oct-2017 18:07:58.348 INFO
[main] org.apache.catalina.core.StandardService.startInternal
Starting service Catalina
national-parks.default(O): 17-Oct-2017 18:07:58.348 INFO
[main] org.apache.catalina.core.StandardEngine.startInternal
Starting Servlet Engine: Apache Tomcat/8.5.9
national-parks.default(O): 17-Oct-2017 18:07:58.367 INFO
[localhost-startStop-1]
org.apache.catalina.startup.HostConfig.deployWAR Deploying
web application archive
/hab/pkgs/core/tomcat8/8.5.9/20170215223900/tc/webapps/nation
al-parks.war
Check the Browser!
• http://{Your IP}:8080/national-parks/
Habitat Info
• http://{Your IP}:9631/services
Other Places to Find Hints
• https://github.com/habitat-sh/core-plans/
• The plan files for the core packages
• Good for “how do I do x?”
Join Us!
• On Slack!
http://slack.habitat.sh
• Online! With Tutorials!
https://www.habitat.sh/
• On Github!
https://github.com/habitat-sh
• This talk
Try it out!
Share your story!
Get some swag!!
goo.gl/WrHQTU
Other References
• Summary on The New Stack
https://thenewstack.io/chef-habitat-addresses-issues-
moving-containers-production
• Our YouTube Channel
https://www.youtube.com/user/getchef