Enterprise 2.0 Anti-Patterns, ROI and metrics …assets.en.oreilly.com/1/event/22/Enterprise Web 2_0...

Enterprise 2.0 Anti-Patterns, ROI and metrics technology • business • peopleAaron KimSenior Managing Consultant, Emerging Technologies Evangelist - IBM Canada

1 © 2009 IBM CorporationNot for further distribution

Photo by Flickr user and IBMer shawdm,used with author permission

1Friday, April 3, 2009

In Twitter:@aaronjuliuskim

#w2e#antip

Now you have only 115 characters left ;-)


Aaron Kim [email protected]

• Senior Managing Consultant, Emerging Technologies & Web 2.0 Evangelistwith IBM Global Business Services – Application Services

• 17 years of experience in IT Services

• 3 years as a Basel II consultant

• Biology degree from Universidade de São Paulo (Brazil)

• MBA from University of Toronto

• Co-chairs the Web 2.0 for Business IBM Community

• Web 2.0 Consulting services to IBM clients and client teams from Canada, France, US, UK, Singapore, South Africa, Spain, Switzerland and Turkey

• Web 2.0 & Social Computing speaker at several conferences in Canada and the US

About Me


mailto:[email protected]


Aaron Kim [email protected]

Tag cloud generated by Wordle, a masterpiece app by IBMer Jonathan Feinberg

About Me




• From Hype to Productivity

• Enterprise Web 2.0 Anti-Patterns

• ROI and Metrics *

Agenda


Part I:The Journey to Productivity


Hype Curve conceived by Gartner

Visibility

TimeTechnology Trigger

Peak ofInflated

Expectations

Trough ofDisillusionment

Slope ofEnlightenment

Plateau ofProductivity

Web 2.0

Going through Gartner’s Hype Curve


Source: Crossing the Chasm, by Geoffrey A Moore

Technology

Enthusiasts

Visionaries

Pragmatists

Conservatives

Skeptics

Chasm

Early Adopters Early Majority

Crossing the Chasm


Based on: IOA CMM Level 5 Journey

From “Hit or Miss” to Process Stability & Improvement

Initial Manage Defined QuantitativelyManaged

Optimized

Hero-based culture Process-based culture


Based on: IOA CMM Level 5 Journey

From “Hit or Miss” to Predictability

Initial Manage Defined QuantitativelyManaged

Optimized

Risk and Waste Productivity & Quality

Dev

iatio

n %

0

20

40

60

-20

-40

-60


The roadmap to the future

Columbus Monument, Santo Domingo, Dominican RepublicPhoto by Aaron Kim


Part II:Enterprise Web 2.0 Anti-Patterns


In software engineering, an anti-pattern is a design pattern that appears obvious but is ineffective or far from optimal in practice.

It’s a pattern that tells you how to go from a problem to a bad solution.

It’s something that looks like a good idea, but which backfires badly when applied.

Sources: Wikipedia (as of 12/Sep/2008)http://c2.com/cgi/wiki?AntiPattern


http://c2.com/cgi/wiki?AntiPattern

http://c2.com/cgi/wiki?AntiPattern

Antipattern:

<pattern name>

Why the bad solution looks attractive

• It becomes a pattern because somehow it looks like the right thing to doWhy it turns out to be bad

• Common pitfallsWhat positive patterns are applicable instead

• Best (Good?) Practices


Antipattern:

Fear 2.0Photo by Flickr user Violator3, licensed under Creative Commons, Attribution 2.0 Generic


• Fear is not a bad thing, but action paralysis is

• Failure comes with a name tag

• Innovating is risky, not innovating may be riskier

• Full control is no longer in your hands

• Fail often, fail quickly, fail gracefully and learn from it *

* Partially based on a presentation by Mike Moran

Antipattern:

Fear 2.017Friday, April 3, 2009

Antipattern:

Hype 2.018Friday, April 3, 2009

Photo: Leopard EM

Antipattern:

New World, Old Habits20Friday, April 3, 2009

Photo by Flickr user dcjohn, licensed under Creative Commons, Attribution 2.0 Generic

Antipattern:


Photo: StockExchange

Antipattern:


Your clients

Your business partners

Other employees in your company

Co-Workers

Friends

YouJimMary

Your managerJim’s manager

Susan

JohnHelen

Roberto

Akira

Chris

Peter

Frequent e-mails

Infrequent e-mails

Web 2.0 Collaboration

People as your competitive advantage


• “It’s just like phone and email”

• A fool with a tool is still a fool

• “Web 2.0 is an attitude, not a technology” (Ian Davis)

• It’s about culture transformation, not a toolset

• “Ultimately, taking full advantage of Web 2.0 may require Management 2.0” (Business Week, June 5, 2006)

Antipattern:


Antipattern:

Build it, and they will come

Photo by Flickr user Sister72, licensed under Creative Commons, Attribution 2.0 Generic


• “If Wikipedia works, my wiki will too”

• People have limited bandwidth 2.0

• The joke, the circus and the soap-opera

• Clay Shirky’s plausible promise, effective tool and acceptable bargain (HCE)

• User Adoption Plan + Balanced Incentives

Antipattern:

Build it, and they will come


Antipattern:

The World Is Flat

Photo: Wikimedia Commons, Public Domain, NASA


Some worlds are flatter than others

28


Antipattern:

Geekness 2.0

Photo by Flickr user pipeapple, licensed under Creative Commons, Attribution 2.0 Generic


• “For it to work, you just need to use Firefox, download and install Greasemonkey, edit a Userscript and install it. Anybody can do it.”

• Second law of thermodynamics:Energy and Entropy

• Laziness 2.0

• Nudge and KISS

Antipattern:

Geekness 2.030Friday, April 3, 2009

Antipattern:

Best of Breed

Photo by Flickr user aturkus, licensed under Creative Commons, Attribution 2.0 Generic


• “Proven solutions”, but in isolation

• Golden hammer

• To a worm in horseradish, the whole world is horseradish (Word vs Excel)

• Integration becomes a nightmare

• Adopt an integrated solution that:

✓ meets your core needs from the outset

✓ can be augmented by adding best of breed

✓ has enterprise grade support

Antipattern:

Best of Breed32Friday, April 3, 2009

• Processes

• Products

• Services

Photo by Flickr user TimWilson, licensed under Creative Commons, Attribution 2.0 Generic

Integration as your competitive advantage


Antipattern:

Search, and thou should not find

Photo: Wikimedia Commons, Public Domain


Flickr: photos tagged with “Cat”

[Flickr Demo]


Flickr: “Interestingness” and “The Wisdom of Crowds”

What is Web 2.0? Seven Principles

[Flickr Demo]


• Your users embraced Web 2.0 and are creating plenty of content

• Most of it is likely to be, err, not very good

• Information overload will quickly overwhelm your users

• UGC needs to be indexed by the main search facility

• Not all UGC is created equal, so make the good float to the top

Antipattern:

Search, and thou should not find37Friday, April 3, 2009

‣ Data discoveryText search is just the beginning

‣ Data visualizationMake it easily consummable

‣ Data filteringMake the good float to the top

‣ Data augmentationInformation provenance and correlation

‣ Data sharingAllow others to mix and match

‣ Data integrationMake difficult for others to copy you

Data as your competitive advantage


Antipattern:

Intangible means unmeasurable

Photo by Flickr user Memotions, licensed under Creative Commons, Attribution 2.0 Generic


• “Nobody asks what’s the ROI for phone and email”

• Business value must discount costs

• Value creation vs. value capture

• Easy to understand business case

• Easy to calculate ROI models

Antipattern:

Intangible means unmeasurable40Friday, April 3, 2009

Antipattern:

Measuring supply, not demand

Photo by Flickr user Memotions, licensed under Creative Commons, Attribution 2.0 Generic


• Number of bloggers, posts, wiki spaces, wiki authors are measures of supply

• Not all UGC has business value

• Not all UGC has business value proportional to its volume

• Find which demand metrics can be associated to business value

Antipattern:

Measuring supply, not demand


Part III:ROI and Metrics


ROI: The need for an “R”

Source: Let's get real or let's not play" by Mahan Khalsa

"If there is no 'R', your 'I' is a 'C'. And the cost is always too high."


ROI is not a question of taste• Companies, especially public companies, always need to

be conscious of ROI. It is a matter of where and how the ROI is used.

• There are concerns about investor lawsuits to question any project that cannot project an ROI, due to regulations like Sarbanes-Oxley.

• Since resources are always limited, you need a yardstick to use in your due diligence to compare outcomes and decide which projects are worth pursuing.

Source: Rethinking Measurement: More than evaluating performance (Interview with Dean Spitzer, IBM Research)

Consultant’s Edge #146 by Peter Andrews, February 21, 2007


Identifying the right metrics

Photo by Flickr user oskay, licensed under Creative Commons, Attribution 2.0 Generic

1. What is your business objective?2. Is your metric stable?3. Can a benchmark be established?4. Do you have the levers to influence it?

Iterative and Interactive!Adapted from: 1) Rethinking Measurement: More than evaluating performance

(Interview with Dean Spitzer, IBM Research) Consultant’s Edge #146 by Peter Andrews, February 21, 2007

2) The Metrics Maze - Am I Lost?, by Srishti GuptaMediaPost’s Online Metrics Insider, Sep 5, 2008


No Magic Metric

Photo by Flickr user bohman, licensed under Creative Commons, Attribution 2.0 Generic

Each measurement system must be developed for what

the organization wants to accomplish




Returns need to be broadly defined• Increased revenues• Increased conversions• Increased social capital• Brand capital• Future value of client loyalty / employee retention• Proxy metrics: Marketing equivalents• Side benefits: new ideas• Cost avoidance

➡ Time saved➡ Replacement cost

(knowledge lost by turn-over / retirement)➡ Opportunity costs:

What is the cost of status quo, or doing nothing?

Photo by Flickr user zieak, licensed under Creative Commons, Attribution 2.0 Generic


...we rather need many of them!

We don’t need a Web 2.0 ROI Model...

Source: ROI 2.0, Part 3: We don’t need a Social Media ROI modelhttp://aaronkim.wordpress.com/2009/02/19/roi-20-part-3-we-dont-need-a-social-media-roi-model/

There is no perfect Web 2.0 ROI model, there are only perfect Web 2.0 ROI models.


http://aaronkim.wordpress.com/2009/02/19/roi-20-part-3-we-dont-need-a-social-media-roi-model/


Qualitative metrics: Surveys / SNA


ROI:How we can start measuring it


Source: CIO 100 and Rawn Shah’s dW blog

Enterprise Tagging System

• Cost items➡ Development and integration of ETS with Enterprise Search cost IBM US$700K

• Benefits ➡ Time savings:

Survey results indicate that ETS has reduced each w3 search by 12 seconds, on average. IBM's enterprise search receives on average of 286,568 search visits per week, yielding a total of 955 hours per week in time reduction.

➡ Cost savings: Using an average hourly rate per employee of US$100/hr, ETS saves IBM US$95,528 each week. Assuming 40 hours per week and 48 weeks per year, ETS brings IBM US$4.6 million in potential productivity gain alone.

➡ Additional cost savings:✓ US$500,000 per year for the dollar value of the information found✓ US$2.4 million in cost avoidance, thanks to the reusability of the ETS widget

• Enterprise Tagging Service allows users to tag pages in the IBM intranet• Advantages: IBMers find useful information when searching w3, it also helps.

➡ Better quality of search results➡ Page viewers can find other people and content relevant to the current page

An IBM example: Tagging at IBM


Other ROI Examples

Source: Groundswell: winning in a world transformed by social technologies, by Charlene Li and Josh Bernoff - A must read!

Check also the Groundswell book for other ROI examples:

• Executive blog• Energizing a community• Community supporting forums• Ratings & Reviews


Food for thought

• Funding is a scarce resource• Bean counters prefer projects where ROI is easier to

calculate• Corporate strategy often entails a decision on

what NOT to invest on• Even when a decision is made to invest on Web 2.0 and

Social Media, a measurement framework is required to decide which alternative to pursue

• It’s important to track progress compared to targets: when should you stop investing in something?


Closing Thoughts


Control in Social Media is like grabbing water: the stronger you grab, the less you hold. There's a right

way to retain water, but not by being forceful.

Pauline Ores, IBM


In the Social Media world, the most powerful person

is the one who shares the most.

Pauline Ores, IBM


Thank [email protected]@aaronjuliuskim

#antip




Supporting Material


Antipatterns Catalog


Antipattern:

Fear 2.0Why the bad solution looks attractive

• Innovation and change are risky and scary

• Failure comes with a name tagWhy it turns out to be bad

• Not changing is riskierWhat positive patterns are applicable instead

• Foster a culture of innovation that balances risk and rewards

• Fail quickly, fail cheaply, fail gracefully and learn from it (inspired by Mike Moran)

• Exit strategies must be an included in innovation initiatives


Antipattern:

Hype 2.0Why the bad solution looks attractive

• Every magazine, consulting firm and their neighbor are saying that adoption of Web 2.0 is a key survival factor

• My boss / my friends / my kids told me it’s cool / the right way to goWhy it turns out to be bad

• Hype creates unreasonable expectations

• The lack of a business case will come back to haunt youWhat positive patterns are applicable instead

• A fit is needed between your Web 2.0 approach and your business model


Antipattern:

New World, Old HabitsWhy the bad solution looks attractive

• “It’s just another technology”

• From distance, using Web 2.0 looks exactly like the introduction of email and instant messaging years ago

Why it turns out to be bad

• A fool with a tool is still a fool

• Most knowledge in a typical corporation is not searchable and remains like that

What positive patterns are applicable instead

• “Web 2.0 is an attitude, not a technology” (Ian Davis)

• It’s about culture transformation, not a toolset

• “Ultimately, taking full advantage of Web 2.0 may require Management 2.0” (Business Week, June 5, 2006)


Antipattern:

Build it, and they will comeWhy the bad solution looks attractive

• Web 2.0 sites occupy most of the top 50 sites in the world

• Our site/service is so cool and useful people won’t resist itWhy it turns out to be bad

• People have limited attention span

• People behave in unexpected waysWhat positive patterns are applicable instead

• Think about Clay Shirky’s plausible promise, effective tool and acceptable bargain (“Here comes everybody”)

• Define and implement a comprehensive User Adoption plan

• Track adoption and balance incentives with desired outcomes


Antipattern:

The World Is FlatWhy the bad solution looks attractive

• In a flat world, solutions should be universally applicable

• Imitation is the cheapest form of flatteryWhy it turns out to be bad

• Localities and companies have unique cultures

• Online ecosystems mimic natural ones: “Two different populations can not occupy the same niche at the same time”


• The joke, the circus and the soap opera

• Think about survival strategies: competition, predation, cooperation, and symbiosis.

Source: Franklin Institute’s Resources for Science Learning


Antipattern:

Geekness 2.0Why the bad solution looks attractive

• The cool ultra-geek star makes everything look so easy

• “If I can do it, anybody can too”

Why it turns out to be bad

• Second law of thermodynamics: everything tends to go to a situation of less energy and more entropy

• People are lazyWhat positive patterns are applicable instead

• Laziness 2.0: Create solutions for the couch potato

• Nudge: Think about Thaler & Sunstein’s choice architecture

• KISS


Antipattern:

Best of BreedWhy the bad solution looks attractive

• Best of breed are proven solutions

• All features I need are already part of the solutionWhy it turns out to be bad

• 2.0 standards are still being developed

• Integration of disparate solutions becomes a nightmareWhat positive patterns are applicable instead

• Adopt an integrated solution that meets your core needs from the outset

• Augment your 2.0 portfolio over time by adding peripheral services


Antipattern:

Search, and thou shall not findWhy the bad solution looks attractive

• Your employees/business partners/clients embraced your Web 2.0 services and started creating plenty of content

• In the first few months, content consumption seems to be manageableWhy it turns out to be bad

• Over time, you’ll feel like drinking from the firehose

• Information overload will overwhelm new usersWhat positive patterns are applicable instead

• UGC needs to be indexed by main search facility

• Wetware: monitor human interactions with content for ranking (ratings, comments, cross-links, click-throughs, social bookmarking)


Antipattern:

Intangible Means UnmeasurableWhy the bad solution looks attractive

• Anecdotal evidence looks compelling

• “Nobody asks what’s the ROI for phone and email”

• Value creation seems obviousWhy it turns out to be bad

• Value capture takes time, effort and creativity, resources in short supply

• In economic downturns, costs are easier to understand and calculate than indirect business benefits


• Easy to understand business case

• Easy to calculate ROI models


Antipattern:

Measuring supply instead of demand Why the bad solution looks attractive

• If User Generated Content (UGC) is a good thing, the more the better

• Supply is easier to measure than demandWhy it turns out to be bad

• Not all UGC has business value

• Not all UGC has business value proportional to its volumeWhat positive patterns are applicable instead

• Find which demand metrics can be associated to business value

• Keep track of supply metrics, but don’t interpret it as business value by default


Supporting materialfor metrics


The Need for Measurement Frameworks

• Organizations need “measurement frameworks,” which show the hypothesized relationships between desired outcomes and the drivers (or leading indicators) of those outcomes.

• Developing the right measures and using measurement data must be iterative and highly interactive

• No "magic metric": each measurement system must be developed for what the organization wants to accomplish

• It’s important to have appropriate measures of profit potential• The best innovation companies are learning to measure (and

manage) in the context of the changing business and regulatory climate without compromising their overall portfolio of innovation.

• Some people feel threatened by measurement




Supply Metrics• # of wiki articles

• # of wiki contributors

• # of wiki registered users

• Contributors per page

• Editing activity per page

• # of blog posts

• # of bloggers

• # of social bookmarks

• # of communities

• # of forum posts

Good, but not the same as business value74Friday, April 3, 2009

Demand Metrics

• Page Content views (includes mouse-over)

• Searches that are satisfied by user-generated content

• Searches that are not fulfilled (helps to understand what content is in high demand and low supply, or just not "discoverable"

• Comments by blog posts

• Replies to forum posts

• Social bookmarks per blog / blog posts

Think conversions!75Friday, April 3, 2009

Qualitative Metrics

• Create a simple questionnaire based on the business objectives you are trying to achieve and measure it on key project milestones: before implementation (baseline), after pilot, after training sessions, after 6-12-24 months.


Social Network Analysis (SNA)

• It's highly recommended that at least 2 rounds of SNA are performed: baseline and after a significant milestone (after pilot, after 12 months) to understand the impact of the Web 2.0 tools in amplifying internal and external social networks.


Recommendations

• Measure awareness and adoption often to identify areas that need to be reinforced or that require a different approach.

• Allow room for adjustments if necessary. Start with as many metrics you can afford, both quantitative and qualitative, so that you understand which ones are meaningful for your specific context.

• Remember that there’s an ROI of calculating ROI, so don’t over-do it. Build a simple model that can be represented in a spreadsheet, and fine-tune it over time.


Measurement frameworks

Photo by Flickr user furryscaly, licensed under Creative Commons, Attribution 2.0 Generic

It needs to show the hypothesized relationships between desired outcomes and

the drivers (or leading indicators) of those outcomes.




...we rather need many of them!1. No ROI

2. Lagging ROI 3. Efficiency gains4. Cost avoidance5. Proxy metrics

6. Product / Service / Process Innovation7. Improved conversions

8. Digitalization of knowledge9. Social Capital / Workforce empowerment

There is no perfect Web 2.0 ROI model, there are only perfect Web 2.0 ROI models.

We don’t need a Web 2.0 ROI Model...

Source: ROI 2.0, Part 3: We don’t need a Social Media ROI modelhttp://aaronkim.wordpress.com/2009/02/19/roi-20-part-3-we-dont-need-a-social-media-roi-model/




To learn more:• IBM: Value 2.0 - Eight new rules for creating and capturing value from innovative technologies,

by Matt Porta, Lisa Buckley, Brian House and Amy Blitz, http://www-935.ibm.com/services/us/index.wss/ibvstudy/gbs/a1029189?cntxt=a1005266#refauthorsc

• Understanding the Psychology of New-Product Adoption, by John T. Gourville, Harvard Business Review, June 2006

• Groundswell: winning in a world transformed by social technologies, by Charlene Li and Josh Bernoff (several ROI examples)

• IT Will Measure Web 2.0 Tools Like Any Other App, by G.Oliver Young, Forrester Research, July 25, 2007

• Building the Web 2.0 Enterprise,McKinsey Global Survey Results, July 2008

• C’est la maturité, stupide! Maslow s’invite à la table du 2.0,http://mediapedia.wordpress.com/2006/07/30/c%E2%80%99est-la-maturite-stupide-maslow-s%E2%80%99invite-a-la-table-du-20/


http://www-935.ibm.com/services/us/index.wss/ibvstudy/gbs/a1029189?cntxt=a1005266#




http://mediapedia.wordpress.com/2006/07/30/c%25E2%2580%2599est-la-maturite-stupide-maslow-s%25E2%2580%2599invite-a-la-table-du-20/




Enterprise 2.0 Anti-Patterns, ROI and metrics …assets.en.oreilly.com/1/event/22/Enterprise Web 2_0...

Documents

Transcript of Enterprise 2.0 Anti-Patterns, ROI and metrics …assets.en.oreilly.com/1/event/22/Enterprise Web 2_0...