Terraform Modules and Continuous Deployment
-
Upload
zane-williamson -
Category
Engineering
-
view
216 -
download
2
Transcript of Terraform Modules and Continuous Deployment
zzConfidential
Terraform Modules and Continuous DeploymentsSan Francisco Infrastructure as Code Meetup
zzConfidential
Your speakers…
• Zane Williamson • Trulia Sr. DevOps• Twitter @zane_williamson• Github @sepulworld
• Marius Ducea• AWS Consultant• Twitter @mariusducea• Github @mdxp
zzConfidential
Overview
1. The Challenge
2. System overview
3. Why we chose Terraform and Packer for Continuous
Deployment
4. Terraform modules and how we are using them
5. Demo a continuous deployment with Terraform and Packer
zzConfidential
The Challenge
• Deploying an array of micro services• Multiple teams• Repeatable Cloud configuration• Peer reviewed infrastructure changes• Tooling flexibility• As simple as possible for non experts
zzConfidential
System Overview
• Cloud Provider is AWS
• Micro-services APIs with ELB endpoints
• Micro-services acting as worker clusters without ELB endpoints
• A common shared backend
zzConfidential
System OverviewTerraform Managed
• Kafka cluster• SQS• s3• RDS• EMR (Hbase)
• VPC, subnets, routes, etc
zzConfidential
Why We Chose Packer to Build Images
1. Dev and ops personnel were already familiar with Packer
2. Allowed us to reuse Puppet modules
3. Multiple outputs (VirtualBox, Docker, AMI, etc)
4. Able to embed in the micro-service code base
5. Easy to call from Jenkins server
zzConfidential
Why we chose Terraform to Manage Infrastructure
1. CloudFormation was limited to AWS services
2. Libraries like Boto and Fog required programming skills and don’t
manage state as well
3. Some developers and operations personnel were already familiar with
Terraform
4. Easy to integrate into Jenkins which is the preferred build and
deployment platform of most of our teams
5. Ability to put the Terraform directly into the code base for each micro-
service
6. Leverage Terraform Modules and the capability to use a Git repo as a
source
zzConfidential
Challenges we faced early on
Multiple teams working with terraform
• code duplication | code drift• management & maintenance• versioning • testing
zzConfidential
Terraform modules
The solution to all our problems?
?
zzConfidential
Terraform modules basics
• Modules are used to create reusable components in Terraform as well as for basic code organization.
• Modules are very easy to create and use.• A module is technically just a folder with some
terraform templates.
zzConfidential
Terraform modules - example ELB module
elb├── CHANGELOG.md├── README.md├── main.tf├── outputs.tf├── test│ ├── aws.tf│ └── test.tf└── variables.tf
zzConfidential
variables.tf
variable "name" { default = "dev-elb"}
variable "subnet_ids" { description = "comma separated list of subnet IDs"}
variable "security_groups" { description = "comma separated list of security group IDs"}
variable "port" { description = "Instance port" default = 80}
zzConfidential
main.tf
resource "aws_elb" "main" { name = "${var.name}"
cross_zone_load_balancing = true subnets = ["${split(",", var.subnet_ids)}"] security_groups = ["${split(",",var.security_groups)}"]
listener { lb_port = 80 lb_protocol = "http" instance_port = "${var.port}" instance_protocol = "http" }. . .
zzConfidential
outputs.tf
output "elb_name" { value = "${aws_elb.main.name}"}
output "elb_id" { value = "${aws_elb.main.id}"}
output "elb_dns_name" { value = "${aws_elb.main.dns_name}"}
output "elb_zone_id" { value = "${aws_elb.main.zone_id}"}
zzConfidential
Module usage
Using modules in Terraform is very similar to built-in resources:
module "elb" { source = “./tf_elb” name = "myelb" port = "80" health_check_url = "HTTP:80/"}
zzConfidential
Module usage - remote git source
The module can live in the same place with the code using it (subfolder) or it can be in a separate repo (recommended).
module "elb" { source = "github.com/sepulworld/tf_elb.git?ref=v0.0.1" name = "myelb" port = "80" health_check_url = "HTTP:80/"}
zzConfidential
Module usage - multiple instances
We can instantiate a module multiple times:
module "elb-frontend" { source = “./tf_elb” name = "frontend" port = "80" health_check_url = "HTTP:80/"}
module "elb-internal" { source = “./tf_elb” name = "internal" port = "8080" health_check_url = "HTTP:8080/health"}
zzConfidential
Module usage - get sources
Finally, before using the module we need to first download it from the source, using terraform get:
terraform get -updateGet: git::https://github.com/sepulworld/tf_asg.git?ref=v0.0.3 (update)Get: git::https://github.com/sepulworld/tf_elb.git?ref=v0.0.1 (update)
zzConfidential
So are modules helping solve our challenges?
Let’s see…
✓ code duplication | code drift ✓ management & maintenance✓ versioning ✓ testing
zzConfidential
Lessons learned
• Use variables for everything; have sane defaults.• Document your modules; we use a changelog to
have a history of all bug fixes and new features.• Use separate git repositories for your modules and
use tags to release new versions of the module. • Test your modules (ideally automatically); we use
terraform validate on all commits and a test run on new releases.
• Be aware that different versions of terraform might behave differently.
zzConfidential
Lessons learned
• Separate your terraform code to minimize the impact of a failure; we use something like:
global (global resources like IAM, cloudtrail, s3, etc.)└ users
development (dev environment)└ core (base resources like vpc, sg, etc.)└ db (persistent storage, rds, etc.)└ microservice1 (resources for services like asg, elb, route53, etc.)
production (prod environment)└ core (base resources like vpc, sg, etc.)└ db (persistent storage, rds, etc.)└ microservice2 (resources for services like asg, elb, route53, etc.)
zzConfidential
Lessons learned
• Use terraform_remote_state to share the state between different terraform runs.
data "terraform_remote_state" "vcp" { backend = "s3" config { bucket = “terraform-state-s3-bucket" key = "dev-vpc-us-west-2" }}
resource "aws_instance" "foo" { # ... subnet_id = "${data.terraform_remote_state.vpc.subnet_id}"}
zzConfidential
Lessons learned
• Terraform new features and improvements added by a new version might break the run on an older version. Always tag new releases of a module that might break older version runs.
• If a new resource or a new argument to an existing one was introduced that will be seen as an error on older versions.
resource "aws_elasticsearch_domain" "es" { elasticsearch_version = “${var.es_version}” # added in 0.7.1domain_name = "${var.es_domain_name}"
zzConfidential
Terraform version manager
Use different versions of terraform (tfenv): https://github.com/kamatama41/tfenv
tfenv install 0.7.7install Terraform 0.7.7get archive from https://releases.hashicorp.com/terraform/0.7.7/terraform_0.7.7_darwin_amd64.zip % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed100 16.8M 100 16.8M 0 0 7163k 0 0:00:02 0:00:02 --:--:-- 7167kArchive: /tmp/terraform_0.7.7_darwin_amd64.zip inflating: /Users/marius/bin/tfenv/versions/0.7.7/terraformthe installation 0.7.7 was successful!!!
zzConfidential
Future improvements
• Conditionals, conditionals, conditionals…
• Terraform language is very limited and this prevents us
from writing ‘real’ reusable modules; at this time we are
using all sort of count related hacks to overcome
conditional limitations but hopefully we’ll have better
solutions in the future.
• Terraform state locking
• Better handling of multiple versions of Terraform
• Testing improvements
zzConfidential
DemoInfrastructure Timeline
zzConfidential
Challenges and No Silver Bullets
1. Doesn’t work with systems that require code to be in
consistent state. For this a Blue/Green type of deployment is
required.
2. Clean up of old AMIs needs to be handled external to Packer
and Terraform
3. Requires application to be engineered for the Cloud
4. Autoscaling Group failure scenarios don’t automatically
cleanup
zzConfidential
https://github.com/sepulworld/tf_elbhttps://github.com/sepulworld/tf_asghttps://github.com/sepulworld/terraform-examples
Demo links:
Questions!