FAILURE: ANALYZE IT, DON’T HUMANIZE IT - Jama...

2014 © Jama Software, Inc www.jamasoftware.com | 1.800.679.3058

build greatproducts™

FAILURE: ANALYZE IT, DON’T HUMANIZE IT

How taking the speculation, emotion and anecdotal evidence out of failure improves product ideas, development, delivery and marketplace results.

INTRODUCTION

As a serial entrepreneur and an early Google employee, Alberto Savoia has experienced great market successes and a few inevitable failures. While at Google, he became a serious student of failure in new and innovative products. After analyzing dozens of studies and hundreds of new products, Alberto identified the main reasons why most new products fail in the market, and he developed a set of sought-after techniques, which he calls pretotyping, to minimize such failures.

Jama Software commissioned Alberto to present his research at our 2014 Product Delivery Summit and to pen this paper, broken into four parts. You can also watch a video of his presentation here.

Alberto is an original thinker and a dynamic presenter, and he makes a compelling case for pretotyping as a best practice for all makers. Learn more about him on his website and follow him on Twitter @Pretotyping.

2 2014 © Jama Software, Inc

http://productdeliverysummit.com/

http://productdeliverysummit.com/

http://www.albertosavoia.com/about.html

https://twitter.com/pretotyping

PART I: THE LAW OF MARKET FAILURE

If you’ve been involved in developing and launching new products, you’ve probably experienced a few flops along the way. But beyond the anecdotal evidence, how common is product failure? Two principles lead us to the answer: 1) Data beats opinions, and 2) Say it with numbers.

For example: Nielsen has been analyzing thousands of worldwide product launches for a long time. Here is a sobering summary from one annual report:

ONE YEAR: 24,543 NEW PRODUCTS*

Failed 27%

16%

37%

14%

6%

100%

Disappointed

Cancelled

Success

Star

Total

80%

*Nielsen


Staggering, isn’t it? Nielsen’s “historical new product performance” analysis found that around 80% will fail, no matter how lenient the definitions of failure, or how accepting and open-minded the markets. But beyond the finger pointing, what’s the explanation for all these failures?

Unfortunately, the causes behind market failures—bad design, poor reliability, subpar performance, unlucky timing, etc.—can’t be quantified or categorized as neatly as the number of market failures, but the odds for failure still trump (and trample) previous records of success and competent execution.

WHY DO YOU F.L.O.P.?


2014 © Jama Software, Inc

THE LAW OF MARKET FAILURE:

Most new products will fail in the market,even if they are competently executed.

But the data we do have is so compelling and consistent that I formalized it into a law: The Law of Market Failure. In criminal law, a person is presumed innocent until proven guilty. When it comes to market law, we should presume a potential new product will fail until we’ve collected enough objective evidence to convince us otherwise.


The tough stuff is out of the way. Once we acknowledge and accept The Law of Market Failure we can stop reacting to it emotionally and can start studying it and developing ways to deal with it.

THE GOOD NEWS:


PART II: F.L.O.P. Analysis — Why Most New Products Fail In the MarketBetween 2009 and 2011, while at Google, I became a

student of market failure. It was a great environment

for doing that, because my colleagues—having worked

for hundreds of companies in dozens of markets—

eagerly compared “war stories,” much like Quint and

Hooper compared battle scars in the movie Jaws.

After a while, I noticed a pattern. Most new product failures could be attributed to one or more of three main causes: Failure in Launch, Operations or Premise. This, incidentally, makes for a convenient and appropriate mnemonic: F.L.O.P.

Let’s look at a typical scenario for each of these causes in isolation. For illustrative purposes, let’s assume that the team got two out of three things right in the Launch, Operations or Premise trio. If you’ve been around long enough, some of these scenarios will sound familiar.


FAILURE IN LAUNCHIn this scenario, products were well thought-out, met a market need, had the right features and were properly designed, engineered and manufactured. However, the timing, pricing or positioning was such that there was no buzz or product traction. People didn’t respond to the “launch” (i.e. the concerted marketing, PR and social-media efforts). Recap: We built the right product. We built it right. But we launched it wrong.

FAILURE IN OPERATIONSHere, products were well thought-out, met a market need, had the right features and design and launched successfully. However, operational problems (reliability, usability, performance, etc.) soon followed. Early adopters questioned product

quality, a few bad reviews appeared in the press and failure was cruelly snatched from the jaws of victory. Recap: We built the right product. We launched it right. But we built it wrong.

FAILURE IN PREMISEIn this final scenario, products were built well (solid, reliable, stable) and the teams responsible for the launch did a great job; there was a lot of buzz and even strong initial sales and adoption. But after a short while, even though the product did exactly what it was designed to do, people realized that they didn’t really need or want it after all. Those who’d already bought it stopped using it, and those who were considering it pursued the next new shiny thing. Recap: We built the wrong product to start with.

WHY DO YOU F.L.O.P.?WHY DO YOU F.L.O.P.?WHY DO YOU F.L.O.P.?


SO WHAT CAN BE FIXED? When we look at patterns of new product failures through the F.L.O.P analysis lens, one thing should become clear: Scenarios 1 and 2 can be remedied, and they probably should be. The caveat: The remedy may not be easy, quick or cheap, but the very same product can be relaunched or re-engineered.

WHAT’S A DEAD END? Scenario 3. If the launch was successful and the product worked well, the most likely conclusion is that we built the wrong product.

As I researched, another thing became clear: Scenario 3 was not only the hardest one from which to recover and regroup, but also it was the most common. The message—now my mantra—for new products was loud and clear:

Make sure you are building The Right It before you build It right.



Part III: Don’t Tell. Don’t Ask. Data Beats Opinions.

All new products begin with an idea. But if all you have

is the idea, the most you can do is solicit opinions. When

you do, you’re likely to run into problems that seriously

derail your decision-making process, such as…


An idea is an abstraction, and a subjective one at that; it’s something that you imagine or picture in your head. The challenge is translation. The way you imagine the new product and its uses may be completely different from the way the customers you seek imagine it.

THE LOST IN TRANSLATION PROBLEM


Even if your audience’s abstract understanding of your idea matches your vision, people are notoriously bad at predicting whether they would actually want, like or use something they have not yet experienced. Your idea lives in a place I call Thoughtland, alongside other unrealized and untested new product ideas. In Thoughtland, every idea can be a winner or a loser depending on whom you ask and how you ask, increasing the possibility of two dangerous outcomes…

THE PREDICTION PROBLEM


People like your idea, they think they understand it and can see themselves using it. They give you a thumbs-up and tell you, “Build it! We’ll definitely buy and use it.” Fueled by such positive feedback you implement the idea. But after you launch it, all that enthusiasm and all those plans to buy and use your product are nowhere to be seen. False positives can lead you to believe that your idea is immune to The Law of Market Failure, so you invest too much too soon in a new product that will eventually flop.

FALSE POSITIVE


Your idea baffles people: “Have random people follow you as you write 140 character blurbs? You’ve got to be kidding!” As a result, you drop your idea and move on. And a few months later another company launches a similar product and you read the headline “Twitter Shares Skyrocket.” False negatives can scare you away from giving your idea a chance, and you can end up prematurely scrapping the next Twitter, or Google or Tesla.

FALSE NEGATIVE


You need something more substantial and objective than opinions—especially opinions from people with no skin in the game. And the only way to do that is to transport your idea from Thoughtland to a more concrete environment called Actionland.

In Thoughtland you use abstract ideas to ask hypothetical questions and collect opinions.

In Actionland you use artifacts to prompt actions and collect data.

THOUGHTLAND: IDEAS -> QUESTIONS -> OPINIONS

ACTIONLAND:ARTIFACTS -> ACTIONS -> DATA


Let’s say, for example, that you have an idea for a computer monitor stand with sensors and motors that automatically moves and adjusts for optimum ergonomics. Let’s call it The Last Stand, as in: “You can now last longer working at your computer.”

If all you have is the idea and a drawing, you’re stuck in Thoughtland. You can explain the idea to your colleagues and ask them their opinions. You can reach out to other companies and talk to HR departments. You might learn a few things. But what will you get? Subjective answers from people with no skin in the game.


You need objective evidence. You can get it by building

a costly, time-consuming proper prototype of The Last

Stand with sensors, motors and software. Working

prototypes are great for determining the feasibility of

a new product, but they’re insufficient for determining

if there is a viable market for a potential product.


Wouldn’t it be wise to objectively test and gauge the potential market for The Last Stand before prototyping it? Are there other artifacts (simpler, cheaper, quicker to build) we can use to collect market data before investing in a prototype? Yes, between abstract ideas and proper prototypes are pretotypes:

Pretotype, noun: An artifact used to test hypotheses and collect data about the market appeal and actual usage of a potential new product objectively and with a minimal investment of time and money.

Pretotypes help you to “make sure that you are building The Right It before you build It right.”


Part IV: Pretotype Before You PrototypeWhy coin a new word for this concept? Two examples

led me to realize that between ideas and what most

people think of as prototypes exists a wonderfully

efficient and effective intermediate step.


Dear Mr. Jones,

I’m writing today to inform you...

Dear Mr. Jones,


Dear Mr. Jones,


Dear Mr. Jones,


EXAMPLE 1: THE IBM SPEECH-TO-TEXT PRETOTYPE

Decades ago, IBM was years away from being able to prototype speech-to-text technology because the hardware was significantly underpowered for the task. So, the company utilized a very clever solution to test and validate some of its ideas and hypotheses: They set up a room with a microphone, a computer monitor—and no keyboard! IBM asked participants to try their speech-to-text machine; all they had to do was speak into the microphone and their words would “magically” appear on the monitor.

Fig. 1: What users thought was going on.


EXAMPLE 1: THE IBM SPEECH-TO-TEXT PRETOTYPE

How did IBM pull this off? They hid a fast typist (with a keyboard) in another room. The microphone was fed to a speaker, and the hidden typist translated the speech into keystrokes which appeared as text on the monitor with amazing speed and accuracy.

With this clever solution IBM discovered some fundamental user interaction issues that would have seriously impacted its chances for market success: Participants’ throats got sore after a while, loud working environments made speech-to-text unappealing and the lack of privacy surfaced as an issue.

The first time I heard this story I was left—ahem—speechless. IBM could not build a proper prototype, so they pretended to have one. I thought that something this clever and unique deserved its own name, and the word pretotype was born.

Fig. 2: What was actually going on.

Dear Mr. Jones,


Dear Mr. Jones,


Dear Mr. Jones,


Dear Mr. Jones,



EXAMPLE 2: THE PALM PILOT PRETOTYPE

In the mid-90s, brilliant innovator and entrepreneur Jeff Hawkins had an idea for the personal digital assistant (PDA) that would eventually become the Palm Pilot. But before investing in building an expensive prototype, he wanted to validate some of his assumptions. He knew he could build it, but would he use it? What would he use it for? And how would he use it?

His solution: Cut a block of wood to match the intended size of the device and use paper sleeves to simulate various user screens and functionality. Seeking insights into how he would use it, Hawkins carried the block of wood with him for a few weeks and pretended it was a functional device. If someone asked for a meeting, for example, he’d pull out the block and tap on it to simulate checking his calendar and to schedule a meeting reminder.

Fig. 3: The Palm Pilot Pretotype


EXAMPLE 2: THE PALM PILOT PRETOTYPE

With the help of his pretotype and simple experiment, Hawkins learned that he would actually carry such a device with him, and that he would be using it mostly for four functions (address book, calendar, memo and to-do lists). After pretotyping validated key assumptions about device size and functionality, investing in building a proper prototype was justified. Eventually, the Palm Pilot not only became incredibly successful, it also established the form factor and paved the way for smart phones.

Fig. 4: The Palm Pilot Prototype


THE PRETOTYPING EFFECT

You can’t escape The Law of Market Failure, but not all failures are created equal. If you fail fast, you can make a few changes, re-test your assumptions or move on to the next idea. By allowing you to validate core hypotheses about new products quickly and cheaply, pretotyping lets you evaluate more ideas which, in turn, helps you increase your success rate and minimize the number of missed opportunities. I call this The Pretotyping Effect:

WITHOUTPRETOTYPING

Ideas Tried Slow Failures Fast Failures Successes Missed Opportunities

Ideas Tried Slow Failures Fast Failures Successes Missed Opportunities

WITHPRETOTYPING

The Pretotyping E�ect


Whether you’re a wildly successful innovator or an ordinary person with an extraordinary idea, it’s been a pleasure sharing these concepts of The Law of Market Failure and Pretotyping with you. May you always find The Right It in life, work and every dream you pursue.

THANK YOU


ABOUT JAMA SOFTWARE

From concept to launch, the Jama product delivery platform helps companies bring complex products to market. By involving every person invested in the organization’s success, the Jama platform provides a structured collaboration environment, empowering everyone with instant and comprehensive insight into what they are building and why. Visionary organizations worldwide, including SpaceX, The Department of Defense, VW, Time Warner, GE, United Healthcare and Amazon.com use Jama to accelerate their R&D returns, out-innovate their competition and deliver business value. Jama is one of the fastest-growing enterprise software companies in the United States, having exceeded 100% growth in each of the past five years, during which time both Inc. and Forbes have repeatedly recognized the company as a model of responsible growth and innovation. For more information please visit www.jamasoftware.com.

Jama Software commissioned Alberto Savoia to present his ideas on pretotyping at its 2014 Product Delivery Summit. This ebook, a condensed version of the blog series Savoia penned for Jama, explores why products fail and how companies can be sure they are building the right “it.” When you’re ready to build your “right it,” contact Jama for help at [email protected].

http://www.jamasoftware.com.

mailto:hello%40Jamasoftware.com.%20%20?subject=

FAILURE: ANALYZE IT, DON’T HUMANIZE IT - Jama...

Documents

Transcript of FAILURE: ANALYZE IT, DON’T HUMANIZE IT - Jama...