An Infrastructure for Team Based PHP DevelopmentGaylord AulkeDirector Professional Services DACHZend Technologies

Agenda

• Why am i talking here? (About the Speaker)• Motivation / The early days• About Processes• About Infrastructure• About Tools

|

About me

• Gaylord Aulke, 38, geek and sometimes businessman

• Did some (serious) Assembler Coding in the past (40kb Machinecode)

• Commercial Projects in Pascal (yes i am THAT old)• 4GL Business Application Coding (i was young

and needed the money)• Started my Web-Life with Perl / CGI• Early Adoption of Java and J2EE (i love it)• Started using PHP in 1997 for very small projects

only

|

Motivation: The good old days

|

Developer

SAMBA Share

Change

Apache

Test

QM

Test

Server

Deploy

PM

Control/Start

Handcrafted approach

• Everyone developed on a common Network share• Apache had its Docroot there• PROs:

Very simple to set up FREQUENT integration of all changes Easy to learn, no errors to make

• CONs: Editing a central file may render the application untestable for

everyone Conflict resolution must be done manually Version Mgmt the hard way (.old, .old2, .veryold, .ancient) Copying code base for concurrent codelines

|

Motivation

• We have serveral software projects• Some are under active development• Some in production and need occasional fixes• We have a limited number of Developers• Everybody needs to work on different projects• Context switches are normal and need to be

efficient• We need to keep a high quality level• We want to scale the team and the customer base

|

About Processes

• Development• Integration• QS• Deployment• Maintenance

|

Processes: Development

• Every developer should be able to change and extend whatever he wants without restrictions

• She should be able to test her changes very frequently in a realistic context

• Changes of others must not influence the tests of her new code

• After successfully testing the changes for themselves they need to be integrated with the changes of everybody else

|

Processes: Integration

• Usually, code runs in a certain environment in production

• Developer needs a development Environment that is as close as possible to the production environment

• Integration needs to be done as early as possible in the process

• The „real“ integration is often only possible at the customer‘s site. A simple way to do this without risk is needed

|

Processes: QS

• „Quality“ is a process issue• Do reasonable tests already at development time

(test data, environment)

• Integrate all code on a continuous integration Server

• Test ALL functionality of the application frequently• Before a release do another test / approval

process• Define measures and threshold for acceptable

quality in all process steps

|

Processes: Deployment

• Automate deployment• Record all deployments made• If you have serveral customers: Manage the

Deployment information centrally

|

Processes: Maintenance

• Manage Defects, Feature Requests, open issues and Requirements

• Define a process for quick fixes/changes• Define a Release Process that coordinates Fixes

and new Releases

|

Local Development

|

Developers

Dev. DB.

Dev.Machinetest

change

SVN RepositoryUpdate/commit/revert

Checkout/delete

QS Machine(all projects)

Checkout/delete

QM Team

test

Packager

Deploy!

Project Config DB

Customer Servers

Scp/zip/rsync/ftp

Local development

• Every developer runs his own copy of the application

• A SCM (Source Code Mgmt) manages all the differerent changes and revisions of the Software Lorna will talk about SVN in a few minutes

• Changes are made to the local copy, local Apache has its docroot there

• Developer tests locally until his component works• Then „check in“ to the repository is done

|

About Databases

• A common database for all developers usually does the job well because: Changes are often backwards compatible (new fields, new

tables) Everybody can use shared test data If the DB is corrupted, it can be re-built from scripts

• Have a separate DB with „authentic“ test data for QS

• Generate temporary DBs during runtime of automated tests (we create and drop these for every single test)

• If a separate DB is needed, you can always copy…

|

Issues with local Development

• Everyone needs to build a local environment with all Extensions / PHP Versions etc.

• Some environment details might only be available at a central location in one instance (an SAP system for example)

• No Control about „the right environment“

|

How we cope with it

• Define a common environment for all applications Framework Set of extensions to use

• Define a common local apache and path setup• Put all environment information in config files• Manage local config files and Vhosts for this setup

in the SCM• Build the application for portability (Windows

Development Systems, Linux Servers)

|

More Sophisticated: Development Sandboxes

|

Developers

Priv. Workspace

Priv. VHostPriv. VHost

Priv. VHost

Priv. Workspace

Priv. Workspace

test

change

SVN Repository

Management ApplicationSelect project

Update/commit/revert

Checkout/delete

checkout

QS VHost

Shared Workspace

Checkout/delete

QM Team

test

Select project

PackagerDeploy!

Deploy!

Project Config DB

Customer Servers

Scp/zip/rsync/ftp

Development Sandboxes: File Structure

|

SymLink

BaseDir

Testserver

Sandboxes

Project#1

Project#2

Project#3

Dyn. Data

Core

Custom

Docroot

Björn

Eric

Thomas

Slot#1

Slot#2

Slot#3

Dyn. Data

Core

Custom

Docroot

Development Sandboxes

• One Development Server• All Environment set up there• Everyone has his own area there where he can

Checkout/Test• Every Developer has a number of „Project Slots“

that he can fill with any project (- revision)• Slots are re-used, can be freed after checkin

|

Development Sandboxes

• PROs: Single Point of administration Complete environment can be set up A common set of tools can be used

• CONs: Quite complex in multi-project environments Development / SCM checkout on network share Single Point of Failure

|

Middelway: VMWare

• Build VMWare Image for each project with all dependencies

• Use hgfs (host file system access) to access local code base (outside VM)

• Host OS can mount shares in the network for shared files and test data

• VM can use external or internal DB• Running a project is as easy as copy a VM, start it

|

Packaging and Branching

|

Change #1 Change #3 Change #5

Change #2 Change #4Live Branch

Development

Change #6

Merge Merge

Deploy new development Versionto Customer Server. Old Live Branch is deleted and then Created new fromdevelopment branch.

Create live branch when developmentVersion has been installed on the liveServer.

Each change in the liveBranch is immediatelyMerged to the dev. branch

Packaging

• Always have the current version that is installed on the customer‘s site in your SCM

• Always follow the same process steps: Change locally on your system Test changes Run automatic Deploy

• To do so, Manage configuration on customer‘s site in some tool Build an infrastructure for deployment to different sites with

different transport mechanisms

|

Tools

• IDE• SCM• Automated Testing / Metrics / Code Coverage• Code-Analyzing• Continuous Integration• Packaging / Deployment• Bug Tracking / Project management

|

IDE: Zend Studio

|

Zend Platform

|

SCM

• RCS works cool for config files• CVS can already manage development projects

with many users• SVN is „the better CVS“ (which is already a

question of believe for some)• GIT is extremely cool• Commercial tools are very different in focus and

functionality/quality• We bought 80 Perforce licenses during New

Economy times (600 $ per seat) and didn‘t regret it.

|

Tesing: PHPUnit

|

Selenium IDE

|

PHP Code Sniffer

|

Static Code Analyzer / Metrics

• „Project Mess Detection“ PMD scans source code and looks for potential problems like: Possible bugs Dead code - unused local variables, parameters and private methods Suboptimal code Overcomplicated expressions - unnecessary if statements, for loops etc. Duplicate code - copied/pasted code means copied/pasted bugs

• Coding Style Check• Static Security analysis (www.armorize.com)• Static Code Quality analysis (Zend Code Analyzer)• …

|

Continuous Integration: Cruise Control

|

Packaging/Deployment• Build-Tools:

ANT, PHING, RAKE, Shell Scripts

• They can generate Packages (ZIP or RPM)• Deployment to Staging Server• Then local deployment to Cluster

• Integrated Solution (Ruby): Capistrano (http://capify.org/) Deploy, Rollback Remote Execution via SSH

• Puppet (http://reductivelabs.com/trac/puppet/)

|

Packaging: What we do

• Intranet-App that has a „deploy“ button for each Project

• SCM Sync to Test-Server first (test-deploy-Button)• Run tests there• Then deploy to customer‘s site (deploy-life Button)• Sync to test server is done via SCM checkout• Sync to customer server is done with arbitrary

shell script ANT, PHING or RAKE for building packets FTP, SCP, RSYNC or HTTP Upload to tansfer Postsync Script on Target Machines to unpack, install

|

Bug Tracking, Project Mgmt

• Trac Integrates with SCM, many plugins (example: http://www.agile42.com/cms/pages/download/) Manages Changes, Tickets, Wiki

• ActiveCollab More PM/Collaboration features, no SCM integration http://www.activecollab.com/

• Phprojekt More Groupware oriented http://www.phprojekt.com/

|

phpUnderControl

• Cruise Control• PHPUnit• PHPCodeSniffer• PHPDocumentor• http://www.phpundercontrol.org

|

phpUnderControl

|

Buildix

http://buildix.thoughtworks.com/

Buildix includes:• Subversion for Source Control• Mingle for Agile Project Management• Cruise Control for Continuous Integration• Trac as a wiki and bug-tracker• …plus a little bit of our own ThoughtWorks magic,

to glue it all together

|

Learnings / Ideas

• For Local development: Set up a local DNS server that resolves all project names to 127.0.0.1

• Example: <projectname>.<customername>.local

• For SCM servers, File System can make a big difference. Choose the right one!

|

Reference to tools

• Zend Studio: http://www.zend.com/products/zend_studio• Cruise Control: http://cruisecontrol.sourceforge.net/• Xinc: http://code.google.com/p/xinc/• Apache ANT: http://ant.apache.org/• PHP Alternative to ANT: http://phing.info/• Subversion: http://subversion.tigris.org/• Tortoise SVN Explorer Extension: http://tortoisesvn.tigris.org/• Trac SVN+ Technical Project Management: http://trac.edgewall.org/• Nagios: http://www.nagios.org/• Zend Platform: http://www.zend.com/de/products/zend_platform• PHPUnit: http://www.phpunit.de/• Selenium: http://www.openqa.org/selenium/

|

Books I found useful

• „Projekt-Automatisierung“, Mike Clark, Carl Hanser Verlag, München, 2006

• (english: „Pragmatic Project Automation – How to build …“, The Pragmatic Programmers, LLC. 2004

• Subversion Book: http://svnbook.red-bean.com/

• “Effektive Softwarearchitekturen”, Gernot Starke, Hanser, 2005

• “Organisational Patterns for Agile Software Development”, Coplien, Harrison, Lucent Technologies, Prentice Hall, 2005

• “Test Driven Development. By Example”, Addison-Wesley, Kent Beck, 2002

|