Migrating a Complex Museum Website from Commercial CMS to Open Source (MW2013)
Key Findings from the 2011 E Source Website Study
-
Upload
e-source-companies-llc -
Category
Documents
-
view
510 -
download
0
description
Transcript of Key Findings from the 2011 E Source Website Study
www.esource.com
Key Findings from the 2011
E Source Website Study
Stephanie Spalding
Research Manager, E Source
EUCI Web Self-Service Conference
January 2012
www.esource.com || © 2011 E Source 2
The Good!
“a real pleasure”
www.esource.com || © 2011 E Source 3
“clever and fun”
www.esource.com || © 2011 E Source 4
“detailed and easily
accessible
user-friendly
experience.”
www.esource.com || © 2011 E Source 5
“awesome!
great resources
I would feel prepared and
informed”
www.esource.com || © 2011 E Source 6
The Bad …
“Overall it was a
frustrating
experience.”
www.esource.com || © 2011 E Source 7
“This was just too many
steps and kind of
annoying.”
www.esource.com || © 2011 E Source 8
“The site is poorly
organized and a few years
behind the times.”
www.esource.com || © 2011 E Source 9
… and the Ugly
“lost, confused,
and frustrated. Should
be revamped.”
www.esource.com || © 2011 E Source 10
“What a horrible online
account page.”
www.esource.com || © 2011 E Source 11
“This was a pretty awful
experience.”
www.esource.com || © 2011 E Source 12
Updated in 2011: Mobile Web
www.esource.com || © 2011 E Source 13
New in 2011: Social Media
Source: Baltimore Gas & Electric
Source: Pacific Gas and Electric Co.
www.esource.com || © 2011 E Source 14
Which Features Were Reviewed?
General Information
Home Page
Contact Us
Payment and Billing
Options
Moving
Energy-Saving Tips
Programs and
Services
Search
Secure
Registration and Login
My Account
Account Balance
Account History
Online Bill Viewing
Online Payment
My Energy Use
Outage/Gas Leak
Status
Online Account
Management
Work Order Status
Payment
Arrangements
Mobile
Home Page
Contact Us
Outage/Emergency
Reporting*
Outage/Gas Leak
Status
Registration and Login
My Account
Account Balance
Online Payment
My Energy Use
Account Alerts*
Payment
Arrangements * = Only reviewed on mobile devices
www.esource.com || © 2011 E Source 15
Which Companies Were Included?
The largest electric and gas companies in the U.S.
and Canada plus a representative set of small and
midsize utilities
100 companies met our criteria, including 94 that
were also reviewed in 2009
72 companies provided us with account information
to review secure options
www.esource.com || © 2011 E Source 16
How Were Reviews Conducted?
Between December 2010 and April 2011
By task in four phases
Tasks by company were randomly assigned
Questions reviewers asked:
Was the feature available on the website?
Was it easy to use?
Reviewers commented on their experiences,
provided ratings, and took screenshots of each
feature
www.esource.com || © 2011 E Source 17
Who Were the Reviewers?
Six people recruited from our local Boulder,
Colorado, community
Three men, three women
Internet and mobile-device savvy
Two reviewer segments:
“Transactional” reviewers view and pay their bills online.
“Informational” reviewers use the web but do not view or
pay their bills online.
www.esource.com || © 2011 E Source 18
How Was Quality Controlled?
We provided extensive training
Supervisors scrutinized reviews for consistency
between the comments and the rating given, and to
ensure our processes were followed accurately
Half of the reviews were conducted a second time
by a different reviewer
www.esource.com || © 2011 E Source 19
Methodology Changes
Tasks were not named
Reviewers were given an unnamed scenario describing
the task they should attempt (“Task A,” not “Payment and
Billing Options”)
This change was intended to remove any bias toward
looking for an exact feature name.
Task A
Scenario: You’ve been receiving and paying your bill through
the same methods for years and are curious what your other
options are. You go to the website to investigate.
www.esource.com || © 2011 E Source 20
Methodology Changes (cont.)
Limited review without account access
If we didn’t have account access, we did not attempt to
review several features that we determined would only be
available behind a login:
My Account
My Account (Mobile)
Account Balance
Account Balance (Mobile)
Account History
Online Account Management
Online Bill Viewing
www.esource.com || © 2011 E Source 21
What Is “Usability”?
A measure of the reviewer’s experience using the
web to accomplish their task.
Usability rating Interpretation
5 (very good) Excellent experience; easy to find and/or use; there are
no suggestions for improvement
4 (good) Good experience; reasonably easy to find and/or use;
has few areas that need improvement
3 (average) Average experience; neither easy nor difficult to find
and/or use; has obvious areas that need improvement
2 (poor) Poor experience; difficult to find and/or use; requires
significant improvements
1 (very poor) Very poor experience; the option is present on the
website, but is extremely difficult to find and/or use;
requires significant improvements to be usable © E Source
www.esource.com || © 2011 E Source 22
What Is “Functionality”?
The percentage of the 29 features that were found on
an electric or gas utility’s website.
Example: If we found 20 of the 29 features we
looked for, the functionality score would be 69
percent.
www.esource.com || © 2011 E Source 23
© E Source
1
2
3
4
5
0 20 40 60 80 100
Us
ab
ilit
y
Ave
rag
e r
atin
g fo
r e
ach
we
bsite
Functionality Average percentage of features found
Account Access No Account Access
Average functionality = 75%
Average usability = 3.1
Overall Performance of Utility Websites
www.esource.com || © 2011 E Source 24
Slight Preference for Apple Devices
1
2
3
4
5
BlackBerry Curve (88) iPad (145) iPod Touch (445) Sanyo Zio [Android](103)
Ave
rag
e u
sa
bil
ity
Mobile device (number of features reviewed on device)
Note: Usability was rated on a scale from 1 to
5, where 1 = very poor and 5 = very good.
© E Source
Overall mobile average usability = 3.0
Total features tested on mobile devices = 781
www.esource.com || © 2011 E Source 25
Usability Ratings Lowest for Online
Payment and My Energy Use
© E Source
Feature Average usability
rating
My Energy Use 2.91
Online Payment 2.87
Online Payment (Mobile) 2.83
My Energy Use (Mobile) 2.78
www.esource.com || © 2011 E Source 26
Not Much Difference Between Mobile
and Desktop Usability
1 2 3 4 5
My Energy Use
Online Payment
My Account
Registration and Login
Contact Us
Payment Arrangements
Account Balance
Home Page
Outage/Gas Leak Status
Usability rating
Fe
atu
re
Mobile Desktop
© E Source
www.esource.com || © 2011 E Source 27
1
4
4
5
5
6
8
10
13
0 2 4 6 8 10 12 14
Account Alerts
Registration and Login
Online Payment
Contact Us
My Account
Account Balance
Outage/Gas Leak Status
Home Page
Outage/Emergency Reporting
Number of Sites Supporting
Mo
bil
e-O
pti
miz
ed
Featu
re
Most Common Mobile-Optimized
Features
© E Source
www.esource.com || © 2011 E Source 28
Mobile-Optimized is Best
3.1 vs. 3.7
www.esource.com || © 2011 E Source 29
Top Recommendations from 2011
www.esource.com || © 2011 E Source 30
© E Source
Utility Website Design Center
www.esource.com || © 2011 E Source 31
www.esource.com || © 2011 E Source 32
What’s Next?
E Source Customer Experience Conference
May 7-9, 2012
Green Valley Ranch, Henderson, Nevada (close to Las
Vegas)
Highlights:
Utility Chief Customer Officer Keynote
Outside industry experts (Zappos, FedEx, etc.)
Zappos Insights Tour
Great facility, networking, and dining!
www.esource.com/ucec2012
www.esource.com || © 2011 E Source 33
For More Information
Stephanie Spalding
Research Manager, E Source
303-345-9121 [email protected]
@EsourceSteph
Have a question? Ask our experts: www.esource.com/question
www.esource.com || © 2011 E Source 34
Questions?