MDW Boulder April '11 | Tim Malbon_How to actually make something

Wednesday, 27 April 2011

Hello


Gareth Kay - Planningness 2009


Let’s smash stuff up


Act like a start-up


We make new stuff out of the internet


We help clients make innovative products &

services out of the Internet


We make things that have long-term value for people

because they are useful, playful & sustainable


paid media model of buying attention“trad advertising-like objects”

earned media model of making stuff useful, playful & sustainable“objects that are more like products & services”

FROM

TO


“Ideas that do”Stop communicating products & start making communication products

Gareth Kay, Goodby Silverstein & Partners


Noah Brier

Invention

Innovation

Diffusion

Creation of a new idea or process

Arranging the economics requirements for implementing an invention

Adoption and imitationA+C-D-

The Agency Innovation Scorecard


[images: twitter, youtube, flickr, napster, facebook, google, myspace, ebay, amazon, paypal, skype, slide,

spotify, apple, drop box]


In the latest turn of its wheel, strategy becomes about how to make existing institutions

as innovative as start-ups

Walter Kiechel III, The Lords of Strategy


Act like a start-up


Simple & visual

Very, very fast

Self-organising teams

Trusting individuals

No siloesWednesday, 27 April 2011

Concept/Seed

Product Development

Alpha/Beta Test

Launch/Ship


Strategy

Functional Spec

Wireframes

Design

Code

small adjustment

Test

Deploy


ClientBrief

Research

Agency Brief

Creative Development

Testing

StrategicDevelopment

Production

small adjustment


A

B

known problem known solution


PROBLEM SPACE

SOLUTIONSPACE

KNOWN UNKNOWN

UN

KN

OW

NK

NO

WN


PROBLEM SPACE

SOLUTIONSPACE

KNOWN UNKNOWN

UN

KN

OW

NK

NO

WN DOES

NOT EXIST


PROBLEM SPACE

SOLUTIONSPACE

KNOWN UNKNOWN

UN

KN

OW

NK

NO

WN DOES

NOT EXIST

AGILE


PROBLEM SPACE

SOLUTIONSPACE

KNOWN UNKNOWN

UN

KN

OW

NK

NO

WN DOES

NOT EXIST

AGILE LEAN


• Rapid prototyping to test hypotheses • Minimum Viable Product (MVP)• Nail it then scale it• Customer obsession (development) • Iterative, metrics-driven & Agile

• Learn fast, don’t fail fast


Startups are not a smaller versions of larger companies

Steve Blank


StartupLarge

Company

The startup exists to search for a scalable business model


The large company exists to execute a business model at scale

StartupLarge

Company


Err yes, but we have *clients*


There is no one-to-one mapping

(But lots to learn)


‘The Lean Agency’Lean Agencies exist to

search for new solutions to their clients’ problems


The characteristics

of a Lean Agency


Iterative


Integrated


Evidence/ Metrics Driven


Minimal Waste


Skype Education ProjectA new service to help more teachers use Skype in the classroom


Skype came to us with a comms brief

| Skype Education Project

Theyʼd noticed that teachers all over the world were using Skype in extraordinary ways. They wanted us to collect these stories and use them to promote the service & inspire other teachers.


We responded with some ideas


And we proposed using an iterative, customer-development focused approach.This was an Agile software development project driven by a continuous feedback loop of qualitative and quantitative data.


Our initial assumptions were that it was all about a lesson planning tool with examples, ideas and tips on integrating Skype into learning


Stage 1 2 3 4

Touchpoint

User Needs

How would you like to find an expert to take part in a

‘Skype visit’? Some type of marketplace?

How would you like to contact them and what would

you need to discuss?

How would you like to involve an expert in planning or structuring the lesson?

How would you like to share the experience with the

community?

Notes

Use case #1: Invite an expert into the classroom


Daniela Callegari

Primary school teacher in ItalyNo experience with Skype in the classroom yetDesperate to connect and find other classes

George Mayo

Middle school language arts teacher in Maryland, USAPioneer in technology and educationEnthusiastic about helping the project

Dan Sutch

FutureLab :: innovation in educationIn-depth knowledge of education & technologyOpen to ongoing involvement in project

Lisa Reid

6th, 7th & 8th grade science teacher in Ohio, USANo experience with Skype in the classroom yetKeen to start using Skype (has used EduSkypers)

Tracy Peterson

Technology teacher in Iowa, USAHas introduced her classes to initial Skype sessionsKeen to collaborate and extend usage of Skype

“Getting out of the building”


We discovered immediately that their biggest obstacle was actually finding other teachers who also used Skype


They also told us that teachers don’t have time to read lots of lesson plans - video clips would be much more helpful


We identified 3 key components for an Alpha release candidate


Ongoing market research provided further evidence that ʻFinding and sharingʼ was core. We pieced together the simplest thing (MVP) that would allow us to carry out lighter testing of our hypotheses at a greater scale on the Internet. We developed the first release in 2 iterations (4 weeks).


Insights gleaned from teachers’ websites


‘How to’ guides

Belong

Connect

Teach

Sign up

Help

Status

Looking for

Willing to mentor

Expert wanting to

help

Suggestions

Skype educator directory

Profile

Location

Name

Age group

Tags / topics

Search

Filter

Video

Questions & answers

Conversation

Planning

Tools Resources

Curriculum

‘How to’ guides

Case studies

API driven Tools

Scheduling

Time zones

Profile search from anywhere

Skype educator ecologyMapping service propositions

Ask a questionLink with

other teachers

The Skype lesson

Feedback

ITFinding experts

School policy issues

Sharing

Parents

Experiences of existing community

Invite pioneers

D. Video/Comment

A. Question & Answer

B. Micro-Profile

C. Search/Filter


We sketched up 4 pages of a directory service which we tested with 7 teachers over Skype


- 55 teachers- 7 countries- 20+ subjects

Expert30%

Intermediate40%

Level of Skype teaching proficiency

Novice29%

Online surveys


“What teachers want”

Find other teachers & share ideas

Find a partner class

Help other teachers

How to use Skype

Overcome obstacles to Skyping

Practice Skype

28%

25%

23%

11.5%

9%

3%


“Teachers who think concept would be ‘very valuable’”

60% -

55% -

50% -

45% -

Search/Filter Video Q&A


We were working within a time and cost-boxed plan


We used various Agile techniques as a way of managing delivery whilst retaining the flexibility to respond to continuous feedback from testing.We created a charter that defined key metrics for success and a shared vision with Skypeʼs project stakeholders.


Creating an MVP - high level beta stories

I want to add a Skype Educator badge to my blog

I want to login to the site

I want to create a profile

I want to view another educators profile

I want to see all the educators in a directory

I want to filter the educator directory

I want to add a resource manually

I want to like a resource

I want to view a resource on it’s own page.

I want to view the resources that another

educator has added / liked.

I want to share a resource I found easily

I would like an admin section for the Skype

educator site.

As an admin, I want to be able to ban people who

misbehave

As an admin, I want to be able to invite a teacher

with a unique url

I want to be able to moderate content, so that bad stuff can be removed

As a member of the site, I want to invite other

teachers to join.

2

2

3

3

3

3

8

8

2

5

2

1

5

1

5

5

I want a directory overview

3

Tools

People Resources AdminOverview


A user journey through the service

Good ideaI want to view the

resources that another educator has added / liked.

2

My profile

New ideas

People like me

Ideas I might like

Mr Mayo

A N Other profile

People like me

Ideas I like

The more people who mark an idea as good the higher the ranking of the idea

I can mark an idea as good on the Skype teacher directory

Ideas I think are good are saved into my profile so I can find them again easily and other teachers like me can find them

I want to like a resource

2


Moving into production: refined wireframes


With the first release, we start getting analytics


The evidence we now start getting from both Skypeʼs analytics package Omniture and Google analytics provide a new stream of feedback and evidence.The data points to 3 important site enhancements currently being implemented, having also been qualitatively validated with teachers in f2f interviews.


Beta hard metrics

‣ 1860 teachers signed up to the site

‣ 60% of users from the USA

‣ 2/3 of visits are new visitors to the site‣ 17% return 2-3 times‣ 18% frequent return visits

‣ High level of contact activity - 1,000 contact requests

‣ 200 resources shared

‣ 105 favourites

‣ 45 comments

Beta soft metrics

‣ Teachers like Skype in the classroom but it could meet their needs better

‣ Teachers want to use the site to connect with other teachers but find it difficult to find suitable matches with the current system

‣ Teachers want more search tools and a wider range of topics to describe themselves by

‣ Teachers want greater granularity on location - probably in part because of the high number of teachers in the USA

‣ Teachers find the resources section a useful source of inspiration and information. This is often what they look at first.

Analytics summary: qualitative and quantitative working together


What the analytics told us


• More than 1 in 2 users are connecting with another

• We saw only c200 resources shared. We researched this further with teachers

• Teachers have started to ʻhack itʼ to create projects

• 60% of the traffic is from the US

• Return visits seemed low, but we realised that we were losing the tracking when they connected via the Skype app


Phase 2 objectives

➡Make connecting with another teacher easier and richer‣ Teachers have more tools to find each other by‣ Teachers can describe themselves in more detail‣ Teachers can use alternative connection methods ‣ Teachers can find good contacts again

➡Show teachers the best and most relevant resources‣ Teachers see more relevant resources in their profile‣ Popular resources are more visible

➡Centre the site around projects rather than profiles‣ Allow teachers to create and publish projects‣ Surface relevant projects to teachers

➡Increase teacher sign ups and number of visits

Phase 2 success metrics

‣ # new sign ups‣ 1/2 teachers return to the site more than once‣ Increase the number of teachers from outside the USA‣ 500 connections per month - either email or Skype‣ # of favourited resources increases‣ # of bookmarked teachers‣ # number of teachers who update their profile with additional information‣ # number of teachers who embed the badge on their website

New objectives and revised success metrics for phase 2


Data and evidence-based design decisions for the new phase


Lessons learned about iterating..


It’s not iterative if you only do it once...


Incremental


Iterative


Speed


If it takes too long any iterative process is indistinguishable

from waterfall


Externalised + Rough


Sketch Prototypes


Jim Glymph, Gehry Technologies

If you freeze an idea too quickly, you fall in love with it

If you freeze an idea too quickly, you fall in love with it... ...refine it too quickly

& you become attached to it

The crudeness of early models in particular is very deliberate

#bdwny


FeedbackWednesday, 27 April 2011

Get out of the building!


Learning

• The whole team learns

• What we learn is communicated to clients

• What we learn is documented

• Metrics are important


ACQUISITION ACTIVATION

RETENTION

REFERRAL REVENUE

Pirate metrics... AARRR

Dave McClure

Actionable metrics not vanity metrics


Incremental execution not incremental thinking


Our goal is always to discover which aspects of

this vision are grounded in reality, & to adapt those

aspects that are not

We always have a vision that is clearly articulated,

big enough to matter, & shared by the whole team

Eric Ries, Start-up Lessons Learned


Intuition Iteration


props to @bokardoWednesday, 27 April 2011

Iterative optimisationmay only reach the peak of

a smaller idea


A better idea/design may exist but requires an

intuitive leap


a smaller idea


A better idea/design may exist but requires an

intuitive leap


a smaller idea

Local Maximaprops to @bokardo


This is an ongoing experiment

(But we can iterate our way to success)


Copyright Made by Many 2011

Tim Malbon@[email protected]

Thank you

http://madebymany.com


http://www.ted.com/index.php/talks/view/id/129








mailto:[email protected]

mailto:[email protected]

http://www.madebymany.co.uk

http://www.madebymany.co.uk

MDW Boulder April '11 | Tim Malbon_How to actually make something

Technology

Transcript of MDW Boulder April '11 | Tim Malbon_How to actually make something