Automating the CI / CD pipeline of your containerized applications

Automating the CI / CD pipeline of your containerized

applicationsLauri Nevala, co-founder and engineer @KontenaInc

@nevalau

Kontena briefly

© 2016 Kontena, Inc.

What is Kontena?

Open Source container platform built to maximize developer happiness. Works on

any cloud, easy to setup, simple to use.

The Container Platform

All Batteries Included!

Built-In Image RegistrySometimes projects can not use publicly

hosted container image registries like DockerHub. Kontena comes with built-in

container image registry providing private and secure solution.

Built-In VPN AccessAll containers are run inside a virtual private

network by default. Nothing is exposed to Internet unless explicitly defined. With

Kontena’s built-in VPN access developers can securely access those resources.

Built-In Load BalancerKontena comes with built-in load balancer.

Based on Haproxy. It features fully automatic, zero-downtime operation due to deep

integration with Kontena’s service discovery and orchestration technology.

Aggregated Stats & LogsKontena provides real-time log and statistics streams containers. The streams may be grouped and aggregated to produce service level streams. This allows easy viewing of logs and statistics for your application CPU, memory, disk and network usage.

User Management with Audit TrailAll events and actions performed through Kontena CLI or APIs are logged into audit trail. Combined with users and access control, the audit trail support makes Kontena a reliable and secure solution for any enterprise deployments.

Built-In Secrets ManagementWhen your application requires access to APIs or databases, you'll often need to use secrets such as passwords and access tokens for authenticating the access. Kontena Vault is a secure key/value storage that can be used to manage secrets in Kontena.

“Includes all the sh*t you don’t need to implement by yourself”

What is Drone?

•Docker based CI/CD platform•Easy to install & maintain• Isolated builds (Docker containers)•Pluggable (Docker containers)• Integrates to Github / Gitlab and others•Travis like “just works” feeling


Why should I care?



Sometimes size does matter

Why Should I Care?

• Lightweight, low memory footprint•Single binary, written in Go•Docker native•Simple YAML configuration•Supports any language that can run in Docker


Installation

•Single binary, just execute

•Docker image (preferred way)

•Configuration through environment variables

• 0.5 introduced build agents, easy to scale© 2016 Kontena, Inc.

Installation: The Docker Way


$ docker run -d-v /var/lib/drone:/var/lib/drone -v /var/run/docker.sock:/var/run/docker.sock--env-file /etc/drone/dronerc--restart=always --publish=80:8000 --detach=true --name=drone drone/drone:0.4

Installation: The Kontena Way


server:image: drone/drone:0.5instances: 1stateful: truedeploy:strategy: hawait_for_port: 80

ports:- 8080:80

environment:- DRONE_DEBUG=true- DRONE_GITHUB=true- DRONE_SERVER_ADDR=0.0.0.0:80- DRONE_OPEN=true- DATABASE_DRIVER=sqlite3- DATABASE_CONFIG=/var/lib/drone/drone.sqlite

secrets:- secret: DRONE_GITHUB_CLIENT_ID

name: DRONE_GITHUB_CLIENTtype: env

....volumes:

- /var/lib/drone

agent:image: drone/drone:0.5stateful: noinstances: 3depends_on:- server

environment:- DRONE_DEBUG=true- DRONE_SERVER=http://%{project}-server:80

secrets:- secret: DRONE_SHARED_SECRETname: DRONE_SECRETtype: env

volumes:- /var/run/docker.sock:/var/run/docker.sock

command: agent

Configuration

•Remote driver• Bitbucket, Github, Gitlab, Gogs

•Database• SQLite (default), Postgres, MySQL

•Plugins• Filter• Global params


Concepts – the “.drone.yml”

•Plugins•Pipeline•Services•Matrix•Secrets•Constraints


Concepts - Plugins

• Plugin is actually just a Docker image• Container created and executed part of pipeline• Exit code determines success/failure


pipeline: test: image: ruby:2.3 commands:

- bundle install --path=bundler - rspec spec/

Concepts – Pipeline

•Arbitrary set of “plugins” to run as the build process

•Success determined by exit code

•Defined in .drone.yml


Concepts – Services

• Set of services needed in the build process

• Available on the same network namespace as build containers

• Handy for eg. Databases used in testing


Concepts – Matrix

• Build a single commit against many configurations

• Interpolated as variables in yaml parsing


Concepts – Matrix


pipeline:build:image: golang:${GO_VERSION}commands:

- go get- go build- go test

services:database:image: ${DATABASE}

matrix:GO_VERSION:- 1.4- 1.3

DATABASE:- mysql:5.5- mysql:6.5- mariadb:10.1

Concepts – Secrets

• Keep your secrets out of VCS• Injected into build containers• Some plugins use automatically• Available in the env• Available only to specific container in build• Blast radius


Concepts – Secrets


$ drone secret ls nevalla/todo-exampleDOCKER_USERNAME Images: plugins/dockerEvents: push, tag, deployment

DOCKER_PASSWORD Images: plugins/dockerEvents: push, tag, deployment

DOCKER_EMAIL Images: plugins/dockerEvents: push, tag, deployment

KONTENA_TOKEN Images: jnummelin/kontena-cliEvents: push, tag, deployment

KONTENA_URL Images: jnummelin/kontena-cliEvents: push, tag, deployment

KONTENA_GRID Images: jnummelin/kontena-cliEvents: push, tag, deployment

Concepts – Constraints

• Limit execution of build steps at runtime• Branches: [master, develop]• Events: tag, push, pull_request, …• Platform: [ linux/*, windows/amd64 ]

• “Deploy when pushed to master” type of things


Setup Kontena Grid

1. Create Kontena Master


kontena digitalocean master create --token=$DO_TOKENGenerating self-signed SSL certificate...

Creating DigitalOcean droplet kontena-master-rough-butterfly-2 ...

Waiting for kontena-master-rough-butterfly-2 to start...

Kontena Master is now running at https://192.86.22.19Use kontena login --name=rough-butterfly-2 https://192.86.22.19 to complete Kontena Master setup

$

done

done

$

2. Login


kontena login --name devops-dam https://192.86.22.19Email: [email protected]

Password: **************

_ _| | _____ _ __ | |_ ___ _ __ __ _| |/ / _ \| '_ \| __/ _ \ '_ \ / _` || < (_) | | | | || __/ | | | (_| ||_|\_\___/|_| |_|\__\___|_| |_|\__,_|-------------------------------------Copyright (c)2016 Kontena, Inc.

Logged in as [email protected]! See 'kontena --help' to get started.$

$

3. Add Nodes


kontena grid create droneUsing grid: drone

$ kontena digitalocean node create --token=$DO_TOKEN

Creating DigitalOcean droplet proud-waterfall-3 ...

Waiting for node proud-waterfall-3 join to grid drone ...

kontena digitalocean node create --token=$DO_TOKENCreating DigitalOcean droplet cold-dream-44...Waiting for node cold-dream-44 join to grid drone ...kontena digitalocean node create --token=$DO_TOKEN

Creating DigitalOcean droplet falling-resonance-59... Waiting for node falling-resonance-59 join to grid drone...

$

done

donedone

done

done$

done$

$

Deploy Drone

Create Secrets – kontena.yml


version: '2'name: drone

services:server:

image: drone/drone:0.5instances: 1…secrets:- secret: DRONE_GITHUB_CLIENT_ID

name: DRONE_GITHUB_CLIENTtype: env

- secret: DRONE_GITHUB_CLIENT_SECRETname: DRONE_GITHUB_SECRETtype: env

- secret: DRONE_SHARED_SECRETname: DRONE_AGENT_SECRETtype: env

- secret: DRONE_ADMIN_USERNAMESname: DRONE_ADMINtype: env

Create GitHub OAuth Application


Write secrets


$ kontena vault write DRONE_GITHUB_CLIENT_ID 909da55bb031dd5$ kontena vault write DRONE_GITHUB_CLIENT_SECRET xxxyyyzzz$ kontena vault write DRONE_SHARED_SECRET RleCk8iaqnyngszUQQh$ kontena vault write DRONE_ADMIN_USERNAMES nevalla$

Deploy Drone


$ kontena app deploycreating servercreating agentdeploying server ... donedeploying agent ... done$

git push

webhook

docker push

dockerpull

Next Steps

Try Kontena

• Quick Start Guide: https://kontena.io/docs/getting-started/quick-start

• Drone Example: https://github.com/kontena/examples/tree/master/drone/0.5

• Give Feedback, join discussion @ slack, tweet, star on GitHub

$25 Credits for Testinghttps://www.packet.net/promo/kontena/

Thank You!www.kontena.io

We are hiring!kontena.io/jobs

Automating the CI / CD pipeline of your containerized applications

Technology

Transcript of Automating the CI / CD pipeline of your containerized applications