Bluemix Meetup Cambridge July 22, 2014

© 2014 IBM Corporation

Deliver apps faster using Python, Django, Bluemix and DevOps Services

Carlos Ferreira – IBM

22 July 2014

[email protected]

© 2014 IBM Corporation2

Agenda

What is the right delivery solution for your business problem?

Choose a programming stack and PaaS provider

Move your app on to Bluemix

Collaborate and continuously deliver on DevOps Services

Source: If applicable, describe source origin

IBM Presentation Template Full Version

© Copyright IBM Corporation 2014. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, Rational, the Rational logo, Telelogic, the Telelogic logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others.


What Is the right delivery solution for your business problem?

First understand the business problem and business role user requirements

Method Role Needs

Platform as a service Developers Customized - Upload code and reuse services

Software as a service Business Users Generalized - Access commercial function in utility model

Infrastructure as a service IT Managers Transparent - Provision and manage infrastructure



Example requirements to consider

Application Category– Mobile -> Android Google Play or Apple IOS plus PaaS for persistence.– Web App– “Systems of Engagement” -> PaaS– “Systems of Record” Depends PaaS, SaaS or IaaS

Security– Data privacy concerns?

• In country• In company, extranet and internet

– Collaborating with partners and clients?• SSO Authentication• Authorization

Ecosystem - Are the right services available? Is there vendor lock-in?– Public/private, managed/unmanaged or both– PaaS Business advantage – Licensing– Dev time, runtime and operational

Agility / Flexibility / Scalability

Model cost at initial and full build out state and include on going maintenance costs– Including services– Development, test and production environments

Consider the complete lifecycyle for your requirements.– Develop/Build, Run and Manage – Try before you buy



For chosen delivery method chose: – public/private– managed/unmanaged– both

PaaS allows developers to develop applications and upload source code to PaaS where it is complied and run including account and application specific resource requirements

PaaS provides dynamic containers to compile, build, and run applications– Bluemix http://bluemix.net/– CloudFoundry - Open source http://cloudfoundry.org/– Heroku

Selecting a PaaS http://www.slideshare.net/altoros/slides-how-to-select-a-paas


Agenda

Is PaaS the right solution for your business problem?

Choose a programming stack, identify services and PaaS provider



Source: If applicable, describe source origin



Choosing and planning a stack – Use Case Web App

Identify:

Providing Application Services

Consuming Application Services

Application Framework and or Pattern

Manage Services

Runtime Services

DevOps Services

PaaS Provider ( Bluemix )PaaS Provider

Runtime Services

Application Framework

Other Providers

ApplicationServices

DevOps Services

ApplicationServices


Identify - Application Frameworks, components and services

Choose an actively supported Ecosystem

http://www.ohloh.net/ Check out stacks

Language: Python

Application Framework: Django- MVC Web Application framework


Identify - Runtime Boilerplates are known patterns and stacks

Python is not listed as a runtime but does work with Bluemix

Use Buildpacks to use other runtimes not listed under boilerplates

Python works on Bluemix with these Buildpacks: – https://github.com/ephoning/heroku-buildpack-python – https://github.com/cloudfoundry/python-buildpack


Identify Services – Runtime to Application Components

Runtime Process / Service Application Component Provides

Web Container gunicorn==18.0

Django==1.6.2

dj-static==0.0.5

Serves Django pages and static content

PostgreSQL Database django-toolbelt==0.0.1

psycopg2==2.5.2

Data persistence and queries

Rabbit MQ + transactions via separate command-line process

amqp==1.4.5

celery==3.1.11

django-celery==3.1.10

Transactions

Solr* via separate command-line Tomcat process

pysolr==3.2.0 Indexing and search

Via Add On SendGrid Mail services for registration/activation and password reset etc.

*Applications that aren’t available as services Bluemix can be ported as command line process with corresponding buildpacks


Choosing and planning a stack – Use Case Web App

Identify:

Providing Application Services

Consuming Application Services

Application Framework and or Pattern

Manage Services

Runtime Services

DevOps Services

PaaS Provider ( Bluemix )PaaS Provider

Runtime Services

Application Framework

Other Providers

ApplicationServices

DevOps Services

ApplicationServices



Allows for app portability to Bluemix from other PaaS providers– Vendors that support Cloudfoundry– Heroku and Bluemix both use Buildpacks as supporting technology

Take advantage of this commonality for development and deployment and avoiding vendor lock in

Getting Started with Bluemix: https://www.ng.bluemix.net/docs/#starters/index.html

Django pre-requisites still apply:– https://docs.djangoproject.com/en/dev/howto/deployment/wsgi/gunicorn/


Move your app on to Bluemix - Getting Started

Get an account

Access bluemix Web site here:

https://ace.ng.bluemix.net/

Click on the “Sign Up” for Bluemix button.

Read quick start

http://www.ng.bluemix.net/docs/QuickStart.jsp



Installing the cf tool to do “cf” commands instead of “heroku” commands

Before you begin, install the command line tool that is used to upload and manage your application. BlueMix uses a tool called cf command line interface.

Download cf command line interface from https://github.com/cloudfoundry/cli

The cf command is a program that you need to install. Select the appropriate installer for your operating system. After the cf command tool is installed, you can use cf -v command to verify its version as follows:

$ cf -v

cf version 6.0.0-90db382



Login

Login. Be sure to use the right endpoint.

$cf login [-a API_URL] [-u USERNAME] [-p PASSWORD] [-o ORG] [-s SPACE]

To see the end point type:

$ cf –t

API endpoint: https://api.ng.bluemix.net (API version: 2.2.0)

Not logged in. Use 'cf login' to log in.

I had to reset mine to the from the beta site.

$ cf api https://api.ng.bluemix.net



Get help getting answers to your questions

For Bluemix and DevOps services go to IBM.com forums https://developer.ibm.com/answers/questions/10523/python-run-time-environment-availability/?community=bluemix

For Python questions use https://Stackoverflow.com

Cloudfoundry command line help: $cf help



Generate your requirements.txt file

Identifies the services and components required by the application

Python BuildPack expects this file

$pip freeze > requirements.txt

httplib2==0.8kombu==3.0.15oauth2==1.5.211oauthlib==0.6.1pika==0.9.13psycopg2==2.5.2pysolr==3.2.0pystache==0.5.4python-openid==2.2.5python-social-auth==0.1.24python-twitter==1.3.1pytz==2014.3requests==2.3.0requests-oauthlib==0.4.0simplejson==3.5.2six==1.6.1static==1.0.2

Django==1.6.2amqp==1.4.5anyjson==0.3.3autoenv==1.0.0billiard==3.3.0.17celery==3.1.11dj-database-url==0.3.0dj-static==0.0.5django-celery==3.1.10django-pagination==1.0.7django-registration==1.0django-taggit==0.11.2django-toolbelt==0.0.1gunicorn==18.0


Move your app on to Bluemix- manifest.yml

Contains important settings for deploying the application, the host machine, the services that the application uses, number of instances, memory, routes and more.

Learn more about deploying using manifest.yml http://docs.cloudfoundry.org/devguide/deploy-apps/manifest.html#services-block.

Things to look out for– Manifest.yml syntax.

• Use this guide: http://docs.cloudfoundry.org/devguide/deploy-apps/manifest.html#services-block.

• Get syntax right by validating manifest YAML file format:http://yaml-online-parser.appspot.com/

• Use this Bluemix app by Ryan Baxter to generate a manifest file: http://cfmanigen.mybluemix.net

– Precedence Between Manifests, Command line Options, and Most Recent Values • Covered later


Move your app on to Bluemix - Procfile, deployment manifest.yml or -c

Two ways you can run python django

A. Using development server using python manage.py runserver

or script that includes it. Example run.sh command:#!/bin/bash#echo STARTINGD RUN.SH FOR STARTING DJANGO SERVER PORT IS $PORT##echo ENVIRONMENT VAIRABLES IN RUN.SH#printenv if [ -z "$VCAP_APP_PORT" ]; then SERVER_PORT=5000; else SERVER_PORT="$VCAP_APP_PORT";fiecho [$0] port is------------------- $SERVER_PORTpython manage.py syncdb --noinputecho "from django.contrib.auth.models

import User; User.objects.create_superuser('admin', '[email protected]', 'admin')" | python manage.py shell

echo [$0] Starting Django Server...python manage.py runserver 0.0.0.0:$SERVER_PORT --noreload

Not recommended for production environments




Example manifest:applications:- name: firstdjango memory: 256M command: bash ./run.sh buildpack: https://github.com/cloudfoundry/python-buildpack path: . services: - postgresql-django




Example requirements.txt:Django==1.6.2dj-database-url==0.3.0psycopg2==2.5.2



B. Using Gunicorn and wsgi. Using push command python ./wsgi.py

Gunicorn 'Green Unicorn' is a Python WSGI HTTP Server for UNIX http://gunicorn.org/

Example manifest fileapplications:- name: simpledjango memory: 256M command: python ./wsgi.py buildpack: https://github.com/ephoning/heroku-buildpack-python path: "." services: - postgresql-9.1



B. Using Gunicorn and wsgi.

Gunicorn requires you specify a procfile. Like this:

Using push command python ./wsgi

web: gunicorn app_name.wsgi --workers 2



B. Using Gunicorn and wsgi.

Example requirements.txt:Django==1.6.2dj-database-url==0.3.0psycopg2==2.5.2dj-static==0.0.5django-toolbelt==0.0.1gunicorn==18.0static==1.0.2wsgiref==0.1.2


Move your app on to Bluemix – Django example multiple push sequence

1. First create database service and push web app without the db setup command. It deploys successfully.

$cf push app_name -m 256M -b https://github.com/ephoning/heroku-buildpack-python.git

2. Then bind my app to it.


Move your app on to Bluemix – Django example push sequence

3. Now restart the app with command to populate DB. Now that you have a connection to it. Instead stop the app and then redeploy using the cf push command and this time append the -c "setup_db.sh" option.

$cf push app_name -m 256M -b https://github.com/ephoning/heroku-buildpack-python.git -c "bash ./setup_db.sh"

setup_db.sh#!/bin/bashecho "Trying----------------------------- to synch"python manage.py syncdb --noinputecho "Finish manage.py synchdb ----------------------------"echo "from django.contrib.auth.models import User;

User.objects.create_superuser('carlos', '[email protected]', 'carlos')" | python manage.py shell

echo "Finish creating super user ----------------------------"

4. Now restart the app with command to run.sh without create users cf push command and this time append the -c “bash ./run.sh" option.


Move your app on to Bluemix – access console on Bluemix

These haven’t been verified and your mileage may vary depending on runtime / buildpack you are using

– https://developer.ibm.com/answers/questions/11152/how-can-i-run-rake-mirgatedb-with-ruby-on-rail-bluemix/

– https://www.ng.bluemix.net/docs/#cli/index.html#cfcommands Look at the section on Retrieving environment variables and using a WebSocket console driver.

– https://developer.ibm.com/bluemix/2014/07/16/easy-ssh-sessions-bluemix/

Cloudfoundry doesn’t provide a way of accessing the consoleOther options available to synch DB and create super user


Move your app on to Bluemix – Troubleshooting

Files and logs – Only accessible if you successfully deploy

$cf –logs app_name from command line window to see what is happening before then


Move your app on to Bluemix – Things to keep in mind

Using environment variables to access databases and other service

File system is transient

Deployment tasks starting apps order is important

Multi-scaling don’t expect second request to be persisted in memory


Move your app on to Bluemix – Set environment variables

Use environment variables to reflect the differences in running you app from dev, test and production environments.

– For things like database, accounts etc.

$cf set-env app_name name value


Move your app on to Bluemix – Using environment variables to access databases

Django .settings file

Set local client environment variable to do local client testing:

postgres://userid:password@localhost:5432/test

On bluemix to see what your environment variables are set to

$cf logs your_app /logs/env.log


Move your app on to Bluemix – File system is transient

Django .settings file

Local file system storage is short-lived. When an application instance crashes or stops, the resources assigned to that instance are reclaimed by the platform

Instances of the same application do not share a local file system. Each application instance runs in its own isolated container. Thus a file written by one instance is not visible to other instances of the same application.

Current Path where App is installed


Move your app on to Bluemix – Starting apps order is important

1. The -c command-line option, for example: $ cf push my-app -c "node my-app.js“

2. The command attribute in the application manifest, for example: command: node my-app.js

3. The buildpack, which provides a start command appropriate for a particular type of application.

How cf push Determines its Default Start Command

The first time you deploy an application, cf push uses the buildpack start command by default. After that, cf push defaults to whatever start command was used for the previous push.

To override these defaults, provide the -c option, or the command attribute in the manifest. When you provide start commands both at the command line and in the manifest, cf push ignores the command in the manifest.

Forcing cf push to use the Buildpack Start Command

To force cf to use the buildpack start command, specify a start command of null.

You can specify a null start command in one of two ways.

Using the -c command-line option:

$ cf push my-app -c "null" Using the command attribute in the application manifest:

command: null

Source: Pivotal Web Services


Move your app on to Bluemix - Deployment task order is important

Invoke CF push multiple times to deploy your application:

1. Create your DB service before you doing CF push commands.Bind DB service to your application

2. Subsequent db migration

$cf push app_name --no-start -b "https://github.com/ephoning/heroku-buildpack-python.git" -c "python manage.py syncdb --noinput"

Note: After this step the database has been created but the application itself has not started, because the normal start command is not used.

3. To clear out earlier cached commands and start app

$cf push crawl -m 256M -b https://github.com/ephoning/heroku-buildpack-python.git -c "null"

Note: This example assumes that the normal start command for your application is the one provided by the buildpack, which the -c 'null' option forces cf to use.


Move your app on to Bluemix - Mutliscaling

HTTP Sessions Not Persisted or Replicated– Cloud Foundry supports session affinity or sticky sessions for incoming HTTP requests

to applications if a jsessionid cookie is used.

See Bluechatter tutorial on architecturally scaling apps on cloud: https://www.youtube.com/watch?v=i7_dQQy40ZQ

Add transactional support to improve performance of long running requests Celery for Django using RabbitMQ


Demonstration

Using development server approach using python manage.py runserver– http://goo.gl/wTHV7L


Move your app on to Bluemix – Tutorials, Webcast and sample projects

Tutorials on Django– http://nbviewer.ipython.org/github/agconti/Django-IPython-Tutorial/blob/master/

django_tut-official.ipynb

Tutorials and demonstrations used in this presentation– Deploy to Bluemix http://goo.gl/wTHV7L– Setup Eclipse and Using DevOps Services with Bluemix goo.gl/Dn4jdj

Sample projects from this presentation– https://hub.jazz.net/project/carlosf/crawl%202/overview– https://hub.jazz.net/project/carlosf/django/overview


Agenda

Is PaaS the right solution for your business problem?

Choose a programming stack and PaaS services provider





IBM DevOps Services

Develop, build, track, plan and deploy your apps to Bluemix

Software configuration management (SCM)̶YStore and manage the code for your applications in the hosted Git repository or

Jazz source control

Integrated development environment ̶YUse built-in Web IDE (Orion), Eclipse, Visual Studio, Rational Team Concert –

or leverage your tool of choice

Agile planning & tracking, team collaboration̶YSeamlessly adopt Agile practices̶YEasily share work and collaborate with team members with task boards and

Dashboards

Continuous Delivery Pipeline̶YBuild automation̶YAutomated deployment to Bluemix


40

IBM BluemixDevOps Services

(powered byJazzHub)

Configure build process and auto-deploy

Connect Eclipse client to DevOps services and Team Share or load code

IBM DevOps Services and Bluemix is now ready for continuous delivery. Delivered changes will automatically deploy application to Bluemix.

Setting up continuous delivery environment is easy in IBM Bluemix…

Edit code of app source code

Automatically links to Bluemix SQL

DB

WAS Liberty

Create new IBM DevOps Services project 5

2

1

3

4

CPO Lunch and Learn - IBM Confidential

Alternatively simply CF push to deploy from on local client

A


Many ways to get started - Projects already exist in DevOps Services

Projects already exist in DevOps Services

Repository tools use either Git or Jazz SCM:– Working with a Git project http://goo.gl/A6XJo4– Working with a Jazz SCM project http://goo.gl/ckLrRC


DevOps Services – Use Jazz SCM

Follow this YouTube tutorial http://goo.gl/Dn4jdj to:– Set up your Eclipse client to with Rational Team Concert – Create a DevOps Project Area– Push your app to the DevOps Project Area– Deploy your app to Bluemix– Collaborate with others by synchronizing your workspace with the changes from others


DevOps Services – Collaborate with others

Find other projects that are doing similar things

Collaborate and do development with others on your team – Notifications


DevOps Services – Things to keep in mind

Workflow from going from command line push, to deployed app on bluemix, to a project area on JazzHub.

TIP: It doesn’t move your code there you have to push / check in deliver from your client.

Using Eclipse client with RTC pluginTIP: use your jazz.net id with your ibm.com password to connect to DevOps Services on

JazzHub

You have multiple optionsTIP: You can use Git or SCM with DevOps or just use Bluemix by itself

Your client workspace changes you delivered don’t show up in project areaTIP: Make sure you accept changes in your repository workspace in your project area.


Innovate and build something cool !!!

Solve your business problem

Choose a programming stack and Bluemix


Send me your Bluemix and DevOps feedback at [email protected]

Follow me on Twitter: @carlosTweets

Learn more on my blog: bluemixstirred.wordpress.com


https://bluemix.net

Bluemix Meetup Cambridge July 22, 2014

Internet

Transcript of Bluemix Meetup Cambridge July 22, 2014