Hypothesis/Data-Driven Development

using Feature Toggles

@neal4d nealford.com

Agenda

The Trust Engineers

http://www.radiolab.org/story/trust-engineers/

Agile 101

Iteration 0 1 2 3 4

Analysis + Design

Development

Testing + Showcase

Integration + QA Release and operation

Customer

Centralized QA IT Operations

"Agile" team

The "last mile"

Continuous Delivery

Iteration 0 1 2 3 4

Analysis + Design

Development

Testing + Showcase

Integration + QA Release and operation

Customer

Centralized QA IT Operations

"Agile" team

The "last mile"

Customer

Delivery teamConstant flow of new features into production

Model

User interface

Model Model

Model

Model

User interface

Model Model

Model

Continuous Integration

Model

User interface

Model Model

Model

Continuous Integration

local workstation

version control

continuous integration server

develop

version control

continuous integration server

build

version control

continuous integration server

build

version control

continuous integration server

build

version control

continuous integration server

build

everyone commits to trunk at least once a day

Feature Branching

P1 P2

G2G1

B1

B1

B1

P1-2

G1-2

G3 G4

P4

B2

P5

G5 G6

B2

P3-4

G3

B2

P3

Professor Plum

Reverend Green

Mainline

G1-6

P1-5

P1-5

G1-6

merge ambush!

trunk-based development

P1 P2

G2G1

B1

B1

B1

P1-2

G1-2

G3 G4

P4

B2

P5

G5 G6

B2

P3-4

G3

B2

P3

Professor Plum

Reverend Green

Mainline

G1-6

P1-5

P1-5

G1-6

https://trunkbaseddevelopment.com

Paul Hammant

Feature Branching Big Scary Merge

Continuous Integration

1

Feature Branching

Continuous Integration

Big Scary Merge12 Discouraging refactoring

Feature Branching

Continuous Integration

Big Scary Merge12 Discouraging refactoring

3 Hard to combine features

Untrusted Contributors

Cherry Picking

http://martinfowler.com/articles/feature-toggles.html

make toggle dynamic

— enables trunk based development

— in-progress feature verification

— incomplete or untested code paths

— latent code in production

— perform multivariate or A/B testing

— utilizes a toggle router

— typical lifetime of hours or weeks

— highly dynamic

— operational

— short lived

— “kill switches”

— product experience

— “premium” / alpha

— very long lived/permanent

Static versus Dynamic Toggles

Long-lived versus Transient Toggles

Implementation Techniques

Decoupling Decision Points/Logic

Decoupling Decision Points/Logic

Why should the invoice email code need to know that the order cancellation content is part of the next-gen feature set?

our invoice emailer is responsible for asking the feature toggling infrastructure how it should perform

Inversion of Decision

doesn’t know about feature toggling, just externalized configuration

Inversion of Decision Testing

Avoiding Conditionals

Experimental Toggles

removed as soon as feature decision is resolved

Feature toggles are purposeful technical debt added to support

engineering practices like Continuous Delivery.

https://dzone.com/articles/feature-toggles-are-one-worst

timeline

don’t create toggle that depend on other toggles.

works on all platforms & technology stacks

ubiquitous

take advantage of structured config files

basic-rec-algo

Use a simplistic recommendation algorithm. This is fast and produces less load on backend systems, but is way less accurate than our standard algorithm.

Toggle Configuration

Managing Toggle Configuration

Hard-coded Parameterized Configuration File

In App DB Distributed

www.togglz.org

www.togglz.org

Model

User interface

Model Model

Model

Canary Releasing

Canary Releasing

Canary Releasing

reduce risk of release

multi-variant testing

performance testing

Canary Releasing

www.mysite.com

Blue. No, yel…Burn her! Did you dress her up like this? Found them? In Mercia?! The coconut's tropical! Bloody Peasant! I'm not a witch. A newt?

The Knights Who Say Ni demand a sacrifice!

The swallow may fly south with the sun, and the house martin or the plover may seek warmer climes in winter, yet these are not strangers to our land. We found them. The nose?

• What a strange person.• Now, look here, my good man.• We want a shrubbery!!

Old

Boring

All servers have the same software

www.mysite.com

Blue. No, yel…Burn her! Did you dress her up like this? Found them? In Mercia?! The coconut's tropical! Bloody Peasant! I'm not a witch. A newt?

The Knights Who Say Ni demand a sacrifice!

The swallow may fly south with the sun, and the house martin or the plover may seek warmer climes in winter, yet these are not strangers to our land. We found them. The nose?

• What a strange person.• Now, look here, my good man.• We want a shrubbery!!

request page

returns page v1returns page v2

Old

Boring

New & exciting

taxonomy

http://www.beautifulbuilds.com/feature-toggle-frameworks-list/

http://ff4j.org

http://ff4j.org

http://ff4j.org

http://ff4j.org

http://balvig.github.io/chili/

www.npmjs.com/package/feature-toggle

github.com/mjt01/angular-feature-flags

Toggling Databases

Evolutionary Database Design

http://databaserefactoring.com/

Initial Phase

Transition Phase

Ending State

Architectural Impact

layered

Architectural Impact

microkernel

microserviceservice-based

Model

User interface

Model Model

Model

Evolutionary Architecture

An evolutionary architecture supports incremental, guided change as a first principle across multiple dimensions.

Incremental Change

production

Components are deployed.

Features are released.

Applications consist of routing.

Incremental Change

production

Deployment Pipeline

Deployment Pipeline

Model

User interface

Model Model

Model

barryoreilly.com/2013/10/21/how-to-implement-hypothesis-driven-development/

Requirements

Behavior-Driven Development

As A…. <role> I Want… <goal/desire> So That… <receive benefit>

In Order To… <receive benefit> As A… <role> I Want… <goal/desire>

Hypothesis-driven Development

We believe <this capability>

Hypothesis-driven Development

Will result in <this outcome>

Hypothesis-driven Development

We will have confidence to proceed when <we see a measurable signal>

Business Story

We Believe That increasing the size of hotel images on the booking page

Will Result In improved customer engagement and conversion

We Will Have Confidence To Proceed When we see a 5% increase in customers who review hotel images who then proceed to book in 48 hours.

Hypothesis Driven UX

https://medium.com/@mwambach1/hypotheses-driven-ux-design-c75fbf3ce7cc#.gk3dpip81

Hypothesis Driven UX

Three Hypotheses

More Listings If we provide more listings on the screen then we can provide better comparability and offer more diversity on our platform because users like to compare a lot of listings on the result page

Better Structure If we provide more structure to our listings then we achieve a better scanability because the user is able to scan the relevant information quicker

Better Prioritization If we prioritize information according to user needs then we achieve better guidance because the user can see all relevant information at a glance

More Listings

Better Structure

Better Prioritization

Experiments to Perform

More Listings Better Structure Better Prioritization

Five Takeaways 1. Take your assumptions and state them as

explicit hypotheses

2. Let your team buy into stated hypotheses

3. Force yourself to design according to your hypotheses

4. Force yourself to test against your hypotheses

5. Create a culture of learning by showing stakeholders the benefits of validated customer hypotheses

Move Fast

& Fix

Things

https://github.com/github/scientist

▫︎ It decides whether or not to run the try block,

▫︎ Randomizes the order in which use and try blocks are run,

▫︎Measures the durations of all behaviors,

▫︎ Compares the result of try to the result of use,

▫︎ Swallows (but records) any exceptions raised in the try block

▫︎ Publishes all this information.

Bugs Found; Resolution

▫︎ faster conflict return because shell script exited immediately; replicated in library

▫︎ index write was causing O(n) problem; inlined into memory

▫︎ the ancestor had a file with a given filemode, whilst one side of the merge had removed the file and the other side had changed the filemode; bug in git!

▫︎Git incorrectly successfully merged files w/ 768 conflicts; fixed git shell script

▫︎ new library was skipping an entire step; bug found & fixed

Model

User interface

Model Model

Model

Model

User interface

Model Model

Model

Hypothesis-driven Development

vision, strategy, business goals

ideation

portfolio of ideas

selected experiments:

pivot

fold double down

Dynamic Equilibrium

Summary

nealford.com

@neal4d

nealford.com/books

nealford.com/videos

www.oreilly.com/software-architecture-video-training-series.html