Usage Analysis

March 15 (Ides of March), 2001By Kevin Fox, for SIMS 213

Overview

What is usage analysis?Why perform usage analysis?Understanding your usersTools for usage analysis

What is Usage Analysis?

Theory One-way window between the designer and

real-world users performing their own tasks.

Practice Gathering and analysis of logfiles and other

data from the actual day-to-day use of a site.

Why Perform Usage Analysis?

Task AnalysisDemographic ProfilingDesign Evaluation

Understanding the user

UA Task Analysis

Usage statistics and profiles show real users performing their own real tasks.

Logfile analysis can be used to identify these tasks and their relative frequencies.

UA Task Analysis

Advantages Real users Real environments Real tasks Ideal for iterative design Relatively cheap

UA Task Analysis

Disadvantages You have to have a real site Difficult to obtain demographic data No ‘thinking aloud’ protocol Guessing at a users real intent (usually, more

on that later) Useful analysis can be difficult

Demographic Profiling

How do needs change based on who the user is?

More on this later

Design Evaluation

How well does your existing site work?Where do users run into problems?

Between Groups Testing

Divert a portion of real users to a new version of a site (or several possible new versions) and compare the results

Importance of keeping user groups distinct (between groups, as opposed to within groups)

True iterative process, using the real world in the feedback loop.

UA and Marketing

Marketing is not a bad wordPutting your site into the hands of the

users who need/want it

Measures of success

Money?Usage?Brand Identity?

What is Amazon’s measure of success?

Tracking Marketing Statistics

Where do your users come from?What forms of marketing are most

successful?

What is Marketing Success?

Banner advertisement clickthrough rates?Comparing use patterns across origins

Targeted vs untargeted advertising (punch the monkey, tricking the user)

In short: All users are not equal

So let’s get to it!

Kinds of InformationSources of InformationTools for Visualizing Information

Kinds of Information

Usage pathsLatencyRates of abandonmentViral marketing

Usage Paths

Where do your users go, and how do they get there?

The most obvious navigation might not be the one they use.

Example: Berkeley Paths (http://fury.com/berkeleypaths)

Latency

How long does a user spend on a page before going to the next page?

Why might this statistic be misleading?

Rates of Abandonment

Multi-stage tasks run a higher risk of being abandoned before completion

Shopping Cart examplePage-by-page abandonment stats can

help isolate usage problems.True for more complex interactive ‘loops’

as well (eVite example)

Viral Marketing

Viral Marketing is usually in the form of iterative traffic. eVite example

Brief tangent: Analyzing viral marketing

Sources of Information

Web site log filesSpecialized usability appsExit surveys

NetRaker

Visualizing Usage Data

Make data approachableBest pattern recognition computer is still

the human brainRecognition tools exist, but the most

useful tools are still those that organize data so our own brains can find patterns

Tools for Visualizing Data

AnalogRealtime Stat providersWebQuiltVIS VIP

Analog

Provides: Aggregate data for referrers Most popular page data Ttraffic over time Frequent visitors (IP addresses)

Lacks: Ability to filter one criterion by another. WebTV example

Realtime Stat Providers

Instant accessRequires preparation on site (usually a

graphic on each page)Doesn’t always work (outages, Browser

issues)Superstats

WebQuilt

Proxy-based site and usage data acquisition tool

Visualization tool

http://guir.berkeley.edu/projects/webquilt

VISVIP

Proxy-based site and usage data acquisition tool

Visualization tool

http://www.itl.nist.gov/iaui/vvrg/cugini/webmet/visvip/pix.html

Upcoming Assignment

“Between Groups” logfile study