How to Build Great Products by Dan Olsen

Copyright © 2011 YourVersion

Dan Olsen Olsen Solu:ons LLC August 24, 2011

How to Build Great Products


What's the Formula for a Great Product?

n  A product that: n Meets customers' needs n  Is bePer than other alterna:ves n  Is easy to use n Has a good value/price

n  Also known as product-‐market fit n  Simple, right? n  It's easy to understand at the conceptual level the aPributes a great product must possess

n  Hard part is HOW to achieve a product like that


My Background n  Educa:on

n  BS, Electrical Engineering, Northwestern n  MS, Industrial Engineering, Virginia Tech n  MBA, Stanford n  Web development and UI design

n  20 years of Product Management Experience n  Managed submarine design for 5 years n  5 years at Intuit, led Quicken Product Management n  Led Product Management at Friendster n  PM consultant to startups: Box.net, YouSendIt, Epocrates n  CEO & Cofounder of YourVersion, startup building “Pandora for your real-‐:me web content”

Will post slides to hPp://slideshare.net/dan_o

Quick Poll of Audience n  Func:onal role

n  Product Management n  Marke:ng n  Other business roles n  Designer n  Engineer n  Other technical roles

n  Market n  Consumer n  Enterprise

n  Industry n  Web n  Mobile n  Hardware n  Biotech n  Greentech n  Other

n  Size of company n  Small: < 50 ppl n  Medium: 50 – 500 ppl n  Large: > 500 ppl


6 Copyright © 2011 YourVersion

Understanding Customer Needs

n  Problem Space n  A customer problem, need, or benefit that the product should address

n  A product requirement


n  Russians: pencil

n  NASA: space pen ($1 M R&D cost)

Example: n  Ability to write in space (zero gravity)

Problem Space vs. Solu:on Space n  Solu:on Space

n  A specific implementa:on to address the need or product requirement


Problem Space vs. Solu:on Space: Product Level

Problem Space (user benefit)

Solu:on Space (product)

TurboTax

TaxCut

Pen and paper

Prepare my taxes

File my taxes

Check my taxes

Maximize deductions

Reduce audit risk


Problem vs. Solu:on Space: Feature Level Problem Space

Solu:on Space

Save time filing taxes

Save time preparing taxes

Maximize my tax deductions

Check my return

Reduce my audit risk

Help me prepare taxes

Empowerement/ Confidence

Save Time

Save Money

Tax Interview Wizard

Audit Risk Analyzer

Tax Return Error Checker

Tax Data Downloader

Electronic Tax Return Filing

Tax Deduction Finder


Priori:za:on Part 1: Customer Value: Benefits & Features

n How do you priori:ze: n Which user benefits should you address? n Which product features to build (or improve)?

n  Importance vs. Sa:sfac:on n  Importance of user need (problem space) n Sa:sfac:on with how well a product meets the user's need (solu:on space)


High Importance + Low Sa:sfac:on = Opportunity to Add Customer Value

Impo

rtance of U

ser N

eed

User Sa:sfac:on with Current Alterna:ves

Compe::veMarket Opportunity

Low High

Low

High

Not Worth Going Aner


Importance vs. Sa:sfac:on Ask Users to Rate for Each Feature

98

8784

8679 847055 80

7280

75

4150

55

60

65

70

75

80

85

90

95

100

40 50 60 70 80 90 100

Satisfaction

Importance

Recommended reading: “What Customers Want” by Anthony Ulwick

Bad

Great


Kano Model: User Needs & Sa:sfac:on User Sa:sfac:on

User Dissa:sfac:on

Performance (more is bePer)

Delighter (wow)

Need not met

Need fully met

Must Have

Needs & features migrate over :me


What is Your Value Proposi:on?

n Which user benefits are you providing? n How are you bePer than compe:tors? Compe&tor A Compe&tor B You Must Have Benefit 1 Y Y Y

Performance Benefit 1 High Low Med

Performance Benefit 2 Low High Low

Performance Benefit 3 Med Med High

Delighter Benefit 1 Y -‐ -‐

Delighter Benefit 2 -‐ -‐ Y


Priori:za:on Part 2: Customer Value & Engineering Effort

n  Customer value is only half the equa:on n  How much engineering effort will it take? n  Need to consider value and effort (ROI) n  Ruthlessly priori:ze: rank order (10 Highs = FAIL) n  Be deliberate about scope & keep it small

n  Strategy = deciding what you're NOT doing n  Break features down into smaller chunks n  LAUNCH! n  Smaller scope → faster itera:ons → higher cust value


Priori:zing Product Ideas by ROI

Investment (developer-‐weeks)

Return (V

alue

Created

)

Idea C

Idea B

Idea D

Idea A

Idea F

1

1

2 3 4

2

3

4 ?

Very Important to have a Priori:zed Feature List

n Have only 1 list & only 1 keeper of the list n Should be a living, real-‐:me document

n Always in rank order & always up to date n Update as new ideas come up

n Should be accessible any:me by team n Google Spreadsheets works great for me n APributes of a good tool n Other tools you’ve used?



UI Design and Ease of Use


User Benefits vs. Ease of Use n  Q: If two products equally deliver the exact same user benefits, which product is bePer?

n  A: The product that's easier to use n  “Ease of use” provides benefits

n  Saves :me n  Reduces cogni:ve load & frustra:on n Makes user feel empowered

n  UI Design can be differen:ator n  Olsen's Law: “The less user effort required, the higher the percentage of users who will do it”


The UI Design Iceberg

Visual Design

Interaction Design

Information Architecture

Conceptual Design

Recommended reading: Jesse James GarreP's “Elements of User Experience” chart, free at www.jjg.net

What most people see and react to

What good product people think about


Elements of User Interface Design

Consists of Three Dis:nct Elements: n  Informa:on Architecture

n  Structure and layout at both site and page level n  How site is structured (sitemap) n  How site informa:on is organized (site layout) n  How each page is organized (page layout)

n  Interac:on Design n  How user and product interact with one another n  User flows (e.g., naviga:on across mul:ple pages) n  User input (e.g., controls and form design)

n  Visual Design n  “How it looks” vs. “What it is”, onen called “chrome” n  Fonts, colors, graphical elements


Dan Olsen CEO, YourVersion July 24, 2009

Early Stage Product Management

Advice on UI Design

n People need visual ar:facts to facilitate discussions about UI design

n Sketch! On paper, whiteboard, sonware n 1st sketch will be bad: embrace itera:on n Diverge (explore) then converge (narrow) n Collaborate in person (vs. remotely) n Great rapid wireframing tool: Balsamiq n Get feedback from users



Learning from Customers


Customer Feedback: Problem Space vs. Solu:on Space

n Most customers CAN'T ar:culate problem space to you

n Customers CAN react to solu:on space n But, customers can't give you solu:ons n Customers aren’t designers

n Mul:ple use cases and priori:es n Trade-‐offs and constraints n Don't have PM, design, and technical skills


Itera:ng Your Product Vector Based on User Feedback in Solu:on Space Problem Space

(your mental model) Solu:on Space

(what users can react to)

Help user book travel

Help user plan travel

Customer Feedback

Mockups or Product


What Are You Gezng Feedback & Learnings About?

Problem Space (your mental model)

Solu:on Space (what users can react to)

Customer Understanding

(needs & preferences)

Feature Set

UI Design Messaging


Persevere or Pivot?

Increasing Product-‐Market

Fit

Pivot

Pivot

Product-‐Market Fit = Gezng enough data to validate that you're climbing up the right mountain


“Ramen” User Feedback for Startups

n Anyone can do it! n  Ingredients:

n Solu:on-‐space product/mockup to test n 1 customer (with laptop if tes:ng code) n 1 desk n 1 person to conduct the session n Pen and paper n Op:onal note-‐taker and observers


Typical Format for Customer Session n  5 -‐ 10 min: Ask ques:ons to understand user needs and solu:ons they currently use

n  30 -‐ 50 min: User feedback n  Show user product/mockup n Non-‐directed as much as possible n When necessary, direct user to aPempt to perform a specific task

n  5 -‐ 10 min: Wrap-‐up n  Answer any user ques:ons that came up n  Point out/explain features you want to highlight n  Ask them if they would use the product


Dos & Don’ts of Conduc:ng Usability n  Do

n  Explain to the user: n  Their usability test will help improve the product n  Not to worry about hur:ng your feelings n  “Think Aloud Protocol”

n  Ask user to aPempt the task, then be a fly on the wall n  Ask non-‐leading, open-‐ended ques:ons n  Take notes and review them anerwards for take-‐aways

n  Don't n  Ask leading ques:ons n  “Help” the user or explain the UI (e.g., “click over here”) n  Respond to user frustra:on or ques:ons (un:l test is over) n  Get defensive n  Blame the user


Dan Olsen, CEO, YourVersion O’Reilly Web 2.0 Expo SF May 6, 2010

Lean Product Management for Web 2.0 Products


Case Study on Valida:ng Product-‐Market Fit: Marke:ngReport.com


Product-‐Market Fit Case Study: Marke:ngReport.com

n My consul:ng client, CEO of TrustedID, had an idea for a new product

n Team: me, CEO, head of marke:ng, UI design consultant

n Goal: n Validate product-‐market fit quickly, cheaply without wri:ng a single line of code

n Determine if their was a business opportunity here


n  Product Concept was “marke:ng report” that let consumers control the direct mail that they receive

n  Concept was fuzzy with various components, so I broke it into 2 different “flavors”: n  #1 “Marke:ng Shield”: Service to reduce/stop junk mail n  #2 “Marke:ng Saver”: Opt in & receive money-‐saving offers n  Each product concept consisted of several modules that each mapped to a specific user benefit

n  Worked with UI designer to create paper mockups of pages for each flavor concept (5 pages each)

Product-‐Market Fit Case Study: Developing Product Concept


Clustering Poten:al User Benefits to Create Product Concepts

Reduce Junk Mail

Find out what “they” know about you

Money Saving Offers

Compare Yourself to Others

Social Networking

Marketing Report

Marketing Score

Marketing Profile

Save Trees

“Shield” Concept “Saver” Concept


n  Telephone recruit of prospec:ve customers n Wrote phone-‐screen ques:onnaire to create rough target customer segmenta:on n Wanted users who work full-‐:me & use internet n Fit for opt-‐in concept: use coupons, Costco membership n Fit for an:-‐junk mail concept: use paper shredder, block caller ID

n  Scheduled 3 groups of 2 or 3 people to discuss each product concept for 90 minutes

n Moderated each group through the paper mockups to hear their feedback

Product-‐Market Fit Case Study: Recrui:ng People


Product-‐Market Fit Case Study: Findings on Concepts &User Benefits

Reduce Junk Mail

Find out what “they” know about you

Money Saving Offers

Compare Yourself to Others

Social Networking

Marketing Report

Marketing Score

Marketing Profile

Save Trees

Legend

Strong appeal

Somewhat positive

Low appeal

“Shield” Concept “Saver” Concept


Product-‐Market Fit Case Study: Learnings from Research

n  “Shield” (an:-‐junk mail) concept stronger than “Saver” n  People didn’t like many “Saver” concept components n  Learned concerns/ques:ons about “Shield” concept n  Refined “Shield” concept:

n  Removed irrelevant components n  Improved messaging to address user concerns / ques:ons

n  Tested revised “Shield” concept with quick 2nd round n  No customer concerns n  Clear willingness to pay


Product-‐Market Fit Case Study: Summary

n 4 weeks from 1st mee:ng to validated product concept with zero coding

n Cost $1,500 to talk to 20 users ($75 each) n 1 round of itera:on on product concept n  Iden:fied compelling concept that users are willing to pay $10/month for

n Trimmed away non-‐valuable pieces n You can achieve similar results


Gezng Quan:ta:ve: Op:miza:on Using Metrics


Approaching Your Business as an Op:miza:on Exercise

Given reality as it exists today, op:mize our business results subject to our resource constraints.


Profit = Revenue -‐ Cost

Unique Visitors x Ad Revenue per Visitor

Impressions/Visitor x Effec:ve CPM / 1000

Visits/Visitor x Pageviews/Visit x Impressions/PV

New Visitors + Returning Visitors

Invited Visitors + Uninvited Visitors

# of Users Sending Invites x Invites Sent/User x Invite Conversion Rate

Define the Equa:on of your Business “Peeling the Onion”

Adver:sing Business Model:


How to Track Your Metrics n  Track each metric as daily :me series

n  Create ra:os from primary metrics: X / Y n  Example: How good is your registra:on page? n Okay: # of registered users per day n  BePer: registra:on conversion rate = # registered users / # uniques to reg page

Date

Unique Visitors

Page views

Ad Revenue

New User Sign-‐ups …

4/24/08 10,100 29,600 25 490

4/25/08 10,500 27,100 24 480

…


Registra:on Conversion Rate Data

Daily Signup Page Yield vs. TimeNew Registered Users divided by Unique Visitors to Signup Page

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1/31 2/14 2/28 3/14 3/28 4/11 4/25 5/9 5/23 6/6 6/20 7/4 7/18 8/1 8/15 8/29 9/12 9/26 10/10

Dai

ly S

ignu

p Pa

ge Y

ield

Changedmessaging

Added questionsto signup page

Started requiringregistration


Iden:fying the “Cri:cal Few” Metrics n  What are the metrics for your business? n  Where is current value for each metric? n  How many resources to “move” each metric?

n  Developer-‐hours, :me, money n  Which metrics have highest ROI opportuni:es?

Return

Investment

Return

Investment Re

turn

Investment

Metric A Good ROI

Metric B Bad ROI

Metric C Great ROI


n  Survey results n  Importance & Sa:sfac:on n  Net Promoter Score

n  Survey.io n  “How would you feel if you could no longer use Product X?”

n  Very disappointed, Somewhat disappointed, Not disappointed

n  User behavior n  Prospects sign up (high conversion rate) n  They keep using it (high reten:on rate) n  They use it onen (high frequency of use)

Metrics to Validate Product-‐Market Fit


Con:nuous Improvement


Adding Metrics and Op:miza:on to your Product Process

Plan

Design

Develop

Business Objec:ves

Product Roadmap

Priori:zed Feature List

Scoping

Requirements & Design

Code

Test

Launch

Site Level

Feature Level

Op:mize Metrics & User Feedback


Op:miza:on through Itera:on: Con:nuous Improvement

Measure the metric

Analyze the metric

Iden:fy top opportuni:es to improve

Design & develop the enhancement

Launch the enhancement

Learning

Gaining knowledge:

•  Market

•  Customer

•  Domain

•  Usability


How to Build Great Products Cheat Sheet

n Clarify problem space by itera:ng in the solu:on space & gezng user feedback

n Revise feature set, UI design, and messaging to improve product-‐market fit

n Ruthlessly priori:ze based on ROI n Define equa:on of your business n  Iden:fy and track key metrics n  Launch, learn, and iterate


Best way to stay on top of the topics you care about

Try it out at

hPp://www.yourversion.com

Free apps for iPad, iPhone, Android phones, Android tablets


Ques&ons? @danolsen slideshare.net/dan_o www.yourversion.com

How to Build Great Products by Dan Olsen

Business

Transcript of How to Build Great Products by Dan Olsen