An Open Standard for Learnable APIs

CTO, New IronJanelle Klein

Learnable APIs

An Open Standard for

Easy to Learn and Use Hard to Learn and Use

Learnability Index

©2015 New Iron GroupSaturday, August 8, 15

RESET

Why should I care?

Saturday, August 8, 15

RESET

...

The Modern API Decision-Making Process

“Hmm... I need a Contact Management API.”


RESET

Contact Management API

...




RESET


...

Our API directory now includes 63 CRM APIs. The newest is thePingGiant API. The most popular, in terms of mashups, is theSalesforce API. We list 44 Salesforce mashups. Below you'll find some more stats from the directory, including the entire list of CRM APIs.

source: http://programmableweb.com




http://programmableweb.com


http://www.programmableweb.com/apitag/CRM


http://www.programmableweb.com/api/pinggiant


http://www.programmableweb.com/api/salesforce.com


http://www.programmableweb.com/api/salesforce.com/mashups


RESET


...





“Ugh...this is taking too long.”












RESET


...




“Hey George, what API do you use for Contact Management?”










RESET


...





“We made our own Salesforce mashup.I’ll send you a link.”










RESET


...





“We made our own Salesforce mashup.I’ll send you a link.”

“Thanks, George. I’ll use that one I guess.”










RESET

Okay... but this is just a minor annoyance.

source: Domics - “What’s a Shrug?”

The real pain happens later...


RESET

https://www.youtube.com/watch?v=0mUN3RppEHE

Fear and InspirationJosh Corman, DevOpsDays Austin 2015




Infecting

We’re putting software and connectivity in everything...



The security implications are frightening...


100 million lines of code running a car!


RESET

90% of our software is assembled from existing parts...


The Complexity and Risk is in the Supply Chain


The Talent Shortage

Higher Demand for Software

Software Development knowledge is mostly tacit knowledge that’s learned through mentorship (not school)

Broken Education System+

More Incompetent Devs building software


Bad Software is like Environment Pollution

BreakingPoint

Cost&

Risk

Complexity

Cumulative Complexity in the SW Supply Chain(i.e. shared dumping ground of crappy software)

The risk to public safety is on all of us.


RESET

What Can We Possibly Do?

source: Domics - “What’s a Shrug?”


RESET


The Difference? Modern Building Codes

Haiti Quake: 7.0 with 230,000 deaths

Chile Quake: 8.8 with 279 deaths


RESET


We’ve copied everything from Deming’s manufacturing ideas except Quality and Risk Management Practices.


RESET

To conquer the challenges of quality and risk

We have to manage quality in the Supply Chain



The @iamthecavalry community rally call:

It’s not our software, but it is our problem.


Let’s take ownership of the supply chain problemsand design the future we want.


RESET

Deliberate Design of Human Systems

There is always cost. There are always consequences. Open != Good

Private Competition

Survival of the Fittest Self-Optimizing

CreativityInnovation

Open Standards

Define the CompetitionOptimize the Whole

Public DebateEmpathy

Interactionscause

system effects


Let’s Take Ownership of Evolution

Pain Pushes

(Threat of Survival)

Opportunity Pulls

(Path of Least Resistance)

“The System”


Learnable APIs

Open Standard for

What if we had...


Mistakes are primarily caused by a lack of understanding.


Learnability Index

Learnability measures the cost of understanding.Learnability measures the risk of mistakes.


RESET


...


Let’s Try this Again...


RESET


...



Quality decisions create selection pressures for quality in the marketSaturday, August 8, 15

RESET


...



Quality decisions create selection pressures for quality in the market

Cost of Ownership Analysis


What information do we need to make quality supply chain decisions?

What’s the strategy to get there?


RESET

...

I’ve been working on a related challengefor the last ~8 years.

Software Rewrite Cycle

How to Break the

Start%Over%

Unmaintainable%So0ware%


RESET

My software project...

We tried to do all the “right” things

...and brought down production 3x in a row.


Technical Debt Risk of Mistakes

I thought the problem wasTechnical Debt


Risk of Mistakes?

Most of our mistakes were in the most well-written parts of the code.


We made significantly more mistakesin code that we didn’t write ourselves.

LowerFamiliarity

MoreMistakes=

There had to be more to the story...


Complex(So*ware(

...pain...

This is what I knew...

What made development feel painful?


UnexpectedBehavior

ProblemResolved

Tracking Painful Interaction with the Code (Friction)

Troubleshooting

Progress

5 hours and 18 minutes of troubleshooting...

PAINFUL


The Amount of PAIN was Driven By...

Likeliness(of((Unexpected(Behavior(

Cost(to(Troubleshoot(and(Repair(

High(Frequency(Low(Impact(

Low(Frequency(Low(Impact(

Low(Frequency(High(Impact(

PAIN(


What Causes UnexpectedBehavior (likeliness)?

What Makes TroubleshootingTime-Consuming (impact)?

What causes PAIN?

Bad Input Assumptions

Stale Memory Mistakes

Semantic Mistakes

Similarity Mistakes

Tedious Change Mistakes

Copy-Edit Mistakes

Transposition Mistakes

Failed Refactor Mistakes

False Alarm

Non-Deterministic Behavior

Ambiguous Clues

Lots of Code Changes

Noisy Output

Cryptic Output

Long Execution Time

Environment Cleanup

Test Data Generation

Using Debugger

Most of the pain was caused by human factors.Saturday, August 8, 15

What Causes UnexpectedBehavior (likeliness)?

What Makes TroubleshootingTime-Consuming (impact)?

What causes PAIN?

Bad Input Assumptions

Stale Memory Mistakes

Semantic Mistakes

Similarity Mistakes

Tedious Change Mistakes

Copy-Edit Mistakes

Transposition Mistakes

Failed Refactor Mistakes

False Alarm

Most of the pain was caused by human factors.

Non-Deterministic Behavior

Ambiguous Clues

Lots of Code Changes

Noisy Output

Cryptic Output

Long Execution Time

Environment Cleanup

Test Data Generation

Using Debugger


“Idea Flow” is a metaphor for the human interaction in software development

Idea Flow Map


ConfirmedExpectations

Confirmation:Everything feels as it should be...


ViolatedExpectations

Conflict:Something is out of place...


The Idea Flow Process looks like this:


“Friction” occurs in the context of a Process.

Pain is caused by high-friction movement.

friction


Friction occurs in Three Places:

Conflict Confirm

Rework'

Learn'

Validate(

Modify'

Confirming Loop! Conflict Loop!

Troubleshoot'


Troubleshooting Pain

Likelihood)of))Unexpected)Behavior)

Cost)to)Troubleshoot)and)Repair)

High)Frequency)Low)Impact)

Low)Frequency)Low)Impact)

Low)Frequency)High)Impact)

PAIN)


Learning Pain

Likelihood)of))working)with)Unfamiliar)

Code)

Cost)to)Learn)

High)Frequency)Easy)to)Learn)

Low)Frequency)Easy)to)Learn)

Low)Frequency)Hard)to)Learn)

PAIN)


Rework Pain

Likelihood)of))making)a))

Bad)Assump4on)

Cost)to)Correct)Decisions)

High)Uncertainty)Low)Delay)

Low)Uncertainty)Low)Delay)

Low)Uncertainty)High)Delay)

PAIN)


How do we measure Idea Flow in Developer Experience?


Idea Flow Mapping Tools(Open Source)


Troubleshooting

Progress

Learning

Rework



Troubleshooting

Progress

Learning

Rework


Experiment via Unit Testing

Experiment via Running App


Troubleshooting

Progress

Learning

Rework



Troubleshooting

Progress

Learning

Rework


Tags: #problemtag


Input: Task + Constraints

Target: Optimal Idea Flow

Output: Actual Friction

1. Visibility

2. Clarity

3. Awareness

Idea Flow Learning Framework

1. Visibility - Make the Pain Visible2. Clarity - Understand the Causes of Pain3. Awareness - Improve Decision Habits

(data-driven learning)


We can learn to read the Visual Indicators in Idea Flow Maps

Le#$Atrium$

Le#$Ventricle$

Right$Ventricle$

Right$Atrium$

What’s$causing$this$pa7ern?$

Similar to how an EKG helps doctors diagnose heart problems...


...Idea Flow Maps help developers diagnose software problems.

Problem-SolvingMachine


= Solution Strategy

The “Heart” of Software Development(the problem-solving machine)


= Solution Strategy


Evaluate alternatives and decide on a general strategy


= Solution Strategy


Scan through the code and build a conceptual model of how it works


= Solution Strategy


Feedback loop phase involves writing a little code, validating that it works, then refining the details of the strategy.


= Solution Strategy


If we discover our strategy isn’t going towork, we have to go back to the drawing board to find an alternative.


Depending on where the disruptions are in the process,we see a different effect in Idea Flow.


"How did you evaluate the possible options and choose a strategy?"

"What was wrong with the different strategies you tried?"

"What was the discovery that made you choose a different direction?"

Problems with Evaluating Alternatives


"Were you working with something that you were unfamiliar with?"

"Did you run into code that was noisy, ambiguous, or misleading?"

"What do you think made it difficult to learn?"

Problems with Modeling


"Did your task involve changes to complex code or business rules?"

"Were there a lot of details that you had to keep in your head?"

"What was causing the complexity in the validation cycles?"

Problems with Refining


"What experiments did you run to troubleshoot the problem?"

"How many times did you run the experiment?"

"How long did it take to get through each experiment cycle?"

Problems with the Validation Cycle


"Was there something in the code that made these changes especially mistake-prone?"

"How familiar were you with the language and tools you were working with?"

"Were you tired or distracted when you did the work?"

Problems with Execution


The Ten Pains of Software Development


Add up Friction by Pain Type

100 hours

50 hours

Troubleshooting

Learning

Rework

Focused Improvements Visible Feedback


What’s Causing the Software Rewrite Cycle?

Start%Over%

Unmaintainable%So0ware%


Cycle of Escalating Risk

Constant'Urgency'

Time'Pressure'

Ignore'Risk'

Increase'Likeliness'

&'Magnitude'of'Pain'

More'Frequent'Bugs'and'Higher'

Task'Effort'


Case Study 1: Healthy project about 10 months old

Troubleshooting

Progress

Learning

Rework10-20% friction

Effects of Escalating Risk


Case Study 2: Thrashing project about 18 months old

Troubleshooting

Progress

Learning


0:00 28:15

12:230:00



Case Study 3: Post-meltdown project about 12 years old

Troubleshooting

Progress

Learning


7:070:00

0:00 19:52



Case Study 1

Case Study 2

Case Study 3

1 day

2 days

1 day

3 days

1 day

3 days

We can’t see these effects by measuring velocity or task lead-time.



RESET

Remember This?

We’ve copied everything from Deming’s manufacturing ideas except Quality and Risk Management Practices.


Idea Flow Learning Frameworkis a software control system!

Input: Task + Constraints

Target: Optimal Idea Flow

Output: Actual Friction

1. Visibility

2. Clarity

3. Awareness

My background...Statistical Process Control (SPC) and Supply Chain in Manufacturing


How do we measure Learnability of an API?


“Learnability” is the average friction across a specific set of Developer Use Case Experiments


Learnability Index

Open Standard for All APIs and Libraries


RESET


...

“How do we get here?”

The Conversation Starts with a Vision

Cost of Ownership Analysis

(Even if it’s a half-baked vision)


Proposal for a Road Map

Open Mastery (Launching Fall 2015)

Developer peer learning network focused on using data-driven learning to codify open decision models and standards for industry-wide problems.

Community Backlog1. Personal Mastery (Codify Idea Flow-based Software Control decision-making)2. Open Software Mastery Certification

3. Team Mastery (Software Mentorship, Knowledge Integration, Improvement Strategy)4. Open Software Mentorship Certification

5. Supply Chain Mastery (Cost of Ownership Analysis, Component Decision Hub)6. Open API DX Certification


Think about it,Blog about it,Tweet about it!

@janellekz #ideaflow!

Free e-book if you sign up by Aug 30th!

@[email protected]

Twitter:Email:


mailto:[email protected]

mailto:[email protected]

An Open Standard for Learnable APIs

Software

Transcript of An Open Standard for Learnable APIs