Latent Code:Lessons Learned

Implementing Feature Bits

Erik SowaLyris Engineering

DevOpsDay US 2010

Outline of Talk

Business contextDevOps: Continuous flow of code to productionLatent code: Introducing feature bitsLessons learned

Business Context

Enable Flow: Avoid Big Bangs

Photo Credit: USDOE (http://commons.wikimedia.org/wiki/File:Castle_romeo2.jpg)

Deployment Pipeline

Photo Credit: Ryan McFarland (http://commons.wikimedia.org/wiki/File:Alaska_Pipeline.jpg )

Continuous Delivery

Documented examples:

Allspaw, John and Hammond, Paul, 10+ Deploys Per Day: Dev and Ops Cooperation at Flickr

http://en.oreilly.com/velocity2009/public/schedule/detail/7641http://www.slideshare.net/jallspaw/10-deploys-per-day-dev-and-ops-cooperation-at-flickrhttp://velocityconference.blip.tv/file/2284377/

Humble, Jez and Farley, David,

Continuous Delivery (O'Reilly Rough Cuts)What about features you can't build in a day?

Feature Bits: Latent Code

Photo Credit: Marcin Wichary (http://commons.wikimedia.org/wiki/File:IMSAI8080_TV11.jpg)

Latent code variants

DigitalRiver and ExactTarget"Feature bits" used by Sean G. Ryan and team

FlickrPreviously cited talk mentions "feature flags" http://en.oreilly.com/velocity2010/public/schedule/detail/14145

Twitter "dynamically controlling features" and "infrastructure"http://nosql.mypopescu.com/post/407159447/cassandra-twitter-an-interview-with-ryan-king

Google ExperimentsGradual rollout of new features

Etc.

Usage at Lyris

Introduced August, 2008~ 80 active bits by December, 200948 active bits in March, 2010

Photo Credit: superde1uxe (http://commons.wikimedia.org/wiki/File:IMSAI8080.jpg)

Lessons Learned

Photo Credit: Underwood & Underwood (http://commons.wikimedia.org/wiki/File:Ecuadoran_Students.jpg)

Design Pressure is Good

Photo Credit: Tokino (http://commons.wikimedia.org/wiki/File:Air_pressure_gauge.jpg)

Manage the Lifecycle

Photo Credit: Pyrothansia (http://commons.wikimedia.org/wiki/File:Life_cycle.jpg)

Maintain Production Quality

Code hidden behind feature bits is subject to the same quality requirements as any code

Photo Credit: Rjgdoy (http://commons.wikimedia.org/wiki/File:Quality_not_quantity.png)

Default state: Decouple code rolls

Photo Credit: Kazumasa Ogawa (http://commons.wikimedia.org/wiki/File:Japanese_Chain_coupler.jpg)

Naming Convention Matters

Do Not Overload

Photo Credit: Jonathan Mallard(http://commons.wikimedia.org/wiki/File:Unsafe_transport_of_a_shipping_container_in_Afghanistan.jpg)

Limit the overhead

Photo Credit: Henning Klevier (http://commons.wikimedia.org/wiki/File:Fibonacci_%28x%2B1%29%5En.png)

Customer-facing releases

Beta- and Split-Testing

Photo Credit: US Navy (http://commons.wikimedia.org/wiki/File:Navy_binoculars.jpg)

For more information

Feature Bits

Video of longer presentation (LSSC10)http://bit.ly/9m3d3r

Paper (LSSC10 proceedings)http://bit.ly/bu8mNo

Thanks Rob Loh (implementer and co-author) Me

Twitter: @eriksowaBlog: http://eriksowa.wordpress.comLinkedIn: http://www.linkedin.com/in/eriksowaemail: erik.sowa@gmail.com, esowa@lyris.com