[Agile Thailand 2012] Release planning & estimation the agile way
Agile Change and Release Management at the #1 Online Rental Site in the US
-
Upload
matt-stratton -
Category
Technology
-
view
74 -
download
1
Transcript of Agile Change and Release Management at the #1 Online Rental Site in the US
Agile Change and Release Management at the #1 Online
Rental Site in the US
Matt Stratton Director, Technology Operations [email protected] Twitter: @mattstratton
Agenda
Ø Introduction
Ø Release Automation – “Robots. Lots of Robots”
Ø ITSM and Agile – The Interface
Ø Questions and Answers
Apartments.com Introduction
Ø Owned by Classified Ventures, a strategic joint-venture owned by five large media partners (A. H. Belo Corp., Gannett Company Inc., The McClatchy Company, Tribune Company, and The Washington Post Company),whose objectives are to collectively capitalize on the online revenue growth opportunities in the automotive, rental and real estate advertising categories.
Ø Began in January of 1997 as as the ApartmentsPlus product of Visual Properties in Chicago, and was acquired by Classified Ventures later that year. It was re-branded as Apartments.com in March of 1998.
Ø Average of 6MM plus unique visitors, a month, and considered most visited ILS for the past year (source MediaMetrix)
Apartments.com Introduction
Technology stack is primarily Microsoft
Over 400 servers across all environments
Nine product teams, each of which has their own dev and FQA environments
The Old Way
Developer builds code on laptop Developer copies build
files to dev server
Dev provides TechOps with migration instructions
“It worked on my laptop!” Dev and TechOps try to Figure out why the release failed
Continue, until QA release is stabilized.
Issues
Ø Release to QA could take up to 2 days – not acceptable when moving in 2-week sprints!
Ø Dependent on Tech Ops to release to QA – resource bottleneck
Ø Way too much time spent on “environment issues” – became a catch-all excuse for problems
Ø Inefficient use of expensive sys admin resource – paying a lot of money for someone to basically push files around
Proposed solutions
Ø Home-grown scripting: PowerShell scripts, Msbuild configuration, MSI from Visual Studio
Ø Substantial effort and no resources available to continually maintain
Ø Would require additional skillset, especially on dev resources
Ø Other third-party solutions
Ø Many still required substantial modification (especially for configuration file transformation)
Ø Often required proprietary scripting skills
Ø Serena Release Automation
Ø Identified by Gartner as industry leader in the space
Ø POC proved out “drag and drop” capability
Ø Did not require specific scripting skillset
Ø Almost all requirements met with “out of the box” actions
How do we do it?
Four environments – Dev, FQA, Smoke, Prod
Very dependent upon branching and merging in TFS
Smoke environment is similar to a traditional “stage”, server is in production, but out of traffic
Development branch for dev, Release for FQA and above
TFS Integration
Release Automation parameters are added into the TFS build to specify the Process and Environment
TFS Integration
SRA CLI is called by the TFS build process
There are two different build definitions – one will build and one will build and deploy. This provides the flexibility to build without a release to an environment.
Release Process
Development/FQA • Any developer has the
ability to push to Dev environment from the Development branch
• The “Build Master” has the ability to push to FQA from the Release branch (but this can be done by any developer with the Build Master skillset)
Production • Not deployed via TFS
build • Uses the same Build
Identifier/drop location as FQA (this information is provided to Tech Ops from Dev)
• Released via the SRA GUI by Tech Ops
Advantages
Ø Can now do continuous integration
Ø Democratizes the release process (Tech Ops not specifically needed until production)
Ø Can deliver product to FQA on day one of a sprint (or as soon as viable)
Ø Removed bottleneck of single person - anyone can do it, even if they aren't the build master (skill wise)
Ø Environment issues reduced approximately 70% ("works fine in dev" type issues)
Caveats
Ø Doesn't address test data discrepancies
Ø Any system still outside of the SRA process could cause issues
Ø Does not necessarily prevent manual changes (this is a good and a bad thing)
Ø Dynamic product teams can increase challenges – skillset transition, etc.
Ø Required quite a bit of custom logging and exception handling to implement the CLI successfully
Advice
Ø POC the product first
Ø Get a really good handle on the publishing process in SRA (not necessarily intuitive and will cause issues if not implemented properly)
Ø Document the process completely “on paper” before automating
Ø Have a good handle on your branching and merging strategy
Issues
Ø Agile framework supported “big ideas” (new product development) but support items were unclear
Ø Previous implementation of TeamTrack was built around defect tracking, not ITIL processes
Ø Re-organization split out previous Production Support group responsibilities between Technical Services and Agile delivery teams
Ø Many rapid releases escalated need for transparency and visibility into changes
Ø No single over-arching technical group anymore – specialized delivery teams required new ways to migrate requests and issues.
TS vs. Delivery
Technical Services team Product delivery teams
• Similar to traditional “service desk”
• Act as funnel for all incoming service requests
• Level 1 and 2 incident management
• Provide escalation to Agile delivery teams for issues
• Built according to Agile framework • Dedicated to particular product line • 2-week sprints • Support items get added to backlog as user stories • Take “marching orders” from Product Owner
ITIL to Agile – The Interface
Incoming connection between ITIL process and Agile is “Problem becomes Backlog item”
Technical Services performs at the Incident level. Agile teams work Problems.
End result is that Technical Services is the “funnel” for all customer/end user communication/requests
Service Requests
Same principle applies for Service Requests – all requests come into Technical Services via SRC/SSM
These may be items TS can perform themselves…
…or may be services that get assigned to specific product teams
What’s next for Apartments.com?
Ø Transition from ITIL problem record (SSM) to Agile backlog item (Rally) is manual right now. Will be automating this integration.
Ø Implement Change and Config in SSM. Automate integration from Rally release action to SSM Change ticket
Ø Trace back user stories to Problems for ability to follow up with customers on status
Ø Discovery on a Continuous Delivery framework
Ø Continue to socialize the need for ticketing system among practioners (“We’re not a bank!”)