Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design
-
Upload
marketingexperiments -
Category
Business
-
view
1.952 -
download
0
description
Transcript of Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design
![Page 1: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/1.jpg)
Responsive Design Tested What a recent experiment reveals about the potential ROI of mobile design
![Page 2: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/2.jpg)
We’re sharing on Twitter!#WebClinic
![Page 3: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/3.jpg)
Today’s speakers
Austin McCraw Senior Editorial Analyst
MECLABS
Jon Powell Senior Manager, Research
and Strategy MECLABS
![Page 4: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/4.jpg)
What is responsive design?
ResponsiveUnresponsive
![Page 5: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/5.jpg)
What is responsive design?
C = 4m + 3v + 2(i - f) - 2a©
The MECLABS Conversion Index
Friction: Psychological resistance to a given element in the conversion process, typically in the form of:
• Unnecessary length
• Unnecessary difficulty
![Page 6: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/6.jpg)
What impact does responsive design have on friction experienced by the customer?
?
![Page 7: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/7.jpg)
Background: A large news media organization trying to determine whether it should invest in responsive mobile design.
Goal: To significantly increase the number of free trial sign-ups.
Research Question: Which design will generate the highest rate of free trial sign-ups: responsive or unresponsive?
Test Design: A/B multifactor split test
Experiment ID: TP1933Record Location: MECLABS Research LibraryResearch Partner: [Protected]
Experiment: Background
![Page 8: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/8.jpg)
Experiment: Control (unresponsive)Desktop Tablet
*Android devices also included in this test
![Page 9: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/9.jpg)
Experiment: Control (unresponsive)Mobile
*Android devices also included in this test
Desktop
![Page 10: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/10.jpg)
Experiment: Treatment (responsive)
*Android devices also included in this test
TabletDesktop
![Page 11: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/11.jpg)
Experiment: Treatment (responsive)
*Android devices also included in this test
MobileDesktop
![Page 12: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/12.jpg)
Experiment: Side-by-side
ResponsiveUnresponsive
*Android devices also included in this test
![Page 13: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/13.jpg)
Experiment: Results
Design Conversion Rate
Control – Unresponsive 3.71%
Treatment – Responsive 5.80%
% Relative Change: 56.41%
Aggregate increase in conversion 56%The responsive design treatment showed an aggregate statistically significant difference in free trial subscriptions when compared to the control.
*Android devices are also part of these results
![Page 14: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/14.jpg)
Why did the responsive design outperform the unresponsive by so much?
?
![Page 15: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/15.jpg)
Experiment: Results
Design Conversion Rate
Control – Unresponsive 3.71%
Treatment – Responsive 5.80%
% Relative Change: 56.41%
Aggregate increase in conversion 56%The responsive design treatment showed an aggregate statistically significant difference in free trial subscriptions when compared to the control.
*Android devices are also part of these results
![Page 16: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/16.jpg)
Experiment: Results
Aggregate Increase in Conversion 56%The responsive design treatment showed an aggregate statistically significant difference in free trial subscriptions when compared to the control
Treatment – Responsive 5.80%
Design Conversion Rate
Control – Unresponsive 3.71%
+ Desktop %
+ Tablet %
+ Mobile %
+ Desktop %
+ Tablet %
+ Mobile %
Which device makes the greatest contribution to the total difference??
![Page 17: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/17.jpg)
Today, we are going to review test data for each device category to determine its total impact on perceived friction.
!
![Page 18: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/18.jpg)
Desktop PerformanceResponsive design tested
![Page 19: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/19.jpg)
Experiment: Desktop control
Unresponsive
• For most screen sizes, all primary content is accessible without action
• The primary content is framed within a faux pop-up style box over a faded background
![Page 20: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/20.jpg)
Experiment: Desktop treatment
Responsive
• Like the control, all primary content is accessible without action
• The faux pop-up frame, however, is completely eliminated
![Page 21: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/21.jpg)
Experiment: Desktop side-by-side
Unresponsive Responsive
![Page 22: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/22.jpg)
Experiment: Desktop results
Design Conversion Rate
Desktop Control – Unresponsive 2.65%
Desktop Treatment – Responsive 5.43%
% Relative Change: 104.57%
Increase in conversion 105%The responsive design desktop treatment showed a statistically significant difference in free trial subscriptions when compared to the control.
![Page 23: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/23.jpg)
What was actually responsible for the difference in performance? ?
![Page 24: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/24.jpg)
Tablet PerformanceResponsive design tested
![Page 25: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/25.jpg)
Experiment: Tablet controlUnresponsive
iOS: • The majority of the content is
accessible without action (supporting image requires scrolling)
• A portion of the bottom of the page shows the pop-up
Android: • All of the content is accessible
without action; may require zoom for certain devices
• Supporting imagery and aesthetic quality is lost in the page load
![Page 26: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/26.jpg)
Experiment: Tablet treatment
iOS: • The supporting image is
immediately visible and the pop-up background is completely eliminated
Android: • Significant improvement in
aesthetic and page load, though the supporting image is still lost
• Only vertical scrolling required for ancillary content
Responsive
![Page 27: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/27.jpg)
Experiment: Tablet side-by-side
Unresponsive Responsive
![Page 28: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/28.jpg)
Experiment: Tablet results
Design Conversion Rate
Tablet Control – Unresponsive 7.24%
Tablet Treatment – Responsive 8.81%
% Relative Change: 21.6%
No significant difference in conversionThe responsive design tablet treatment showed no statistically significant difference in free trial subscriptions when compared to the control.X
![Page 29: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/29.jpg)
Why did the same treatments perform differently on different devices??
![Page 30: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/30.jpg)
Mobile PerformanceResponsive design tested
![Page 31: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/31.jpg)
Experiment: Mobile control
iOS: • Requires scrolling to read complete
sentences
• Also requires scrolling to see all required fields and the primary call-to-action
Android: • Requires one to two pinches to zoom
movements and read the content
• When zoomed in, requires horizontal scrolling to read complete sentences
Unresponsive
![Page 32: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/32.jpg)
Experiment: Mobile treatment
• Requires vertical scrolling to get to all fields as well as the call-to-action
• Complete sentences are available without action
• Generally a consistent experience across major device platforms
Responsive
![Page 33: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/33.jpg)
Experiment: Mobile side-by-sideUnresponsive Responsive
![Page 34: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/34.jpg)
Experiment: Mobile results
Design Conversion Rate
Mobile Control – Unresponsive 5.14%
Mobile Treatment – Responsive 4.34%
% Relative Change: -15.6%
No significant difference in conversionThe responsive design mobile treatment showed no statistically significant difference in free trial subscriptions when compared to the control.X
![Page 35: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/35.jpg)
Experiment: All results
% Relative Change: 56.41%
Treatment – Responsive 5.80% +56.4%
Design Conversion Rate % Rel. Change
Control – Unresponsive 3.71% -
+ Desktop 2.65% -
+ Tablet 7.24% -
+ Mobile 5.14% -
+ Desktop 5.43% +104.6%
+ Tablet 8.81% +21.6%
+ Mobile 4.34% -15.6%
What is the customer wisdom? Why did the treatment win in aggregate?
?
![Page 36: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/36.jpg)
What you need to understand
C = 4m + 3v + 2(i - f) - 2a©
The MECLABS Conversion Index
The mobile and tablet treatments made significant changes in layout; they did not, however, make significant changes in frictionexperienced by the customer
Friction
![Page 37: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/37.jpg)
What you need to understand
C = 4m + 3v + 2(i - f) - 2a©
The MECLABS Conversion Index
The desktop treatment did something slightly different
The move to responsive design coincidentally reduced both friction and anxiety by eliminating the pop-up appearance of the control
Friction
Anxiety
![Page 38: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/38.jpg)
Responsive design tested
FKey Principles
1. Responsive design, like any new technology or technique, does not necessarily increase conversion. This is because we are not optimizing design, we are optimizing thought sequences.
2. Responsive design is only as effective as the degree of change it affects in the mind of the customer.
3. This requires you to consider not only how responsive design will impact the look of the page, but the customers perception on two levels:• Friction: How much will responsive design decrease perceived effort?
• Anxiety: How much will responsive design decrease perceived concern?
![Page 39: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/39.jpg)
EXCLUSIVE SAVINGS
Save an EXTRA $200 on Email Summit Tickets
Use promo code:
515-ST-1010*Offer expires January 24
![Page 40: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/40.jpg)
Background: A large technology company selling software to small businesses.
Goal: To significantly increase the number of software purchases.
Research Question: Which design will generate the highest rate of software purchases?
Test Design: A/B multifactor split test
Experiment ID: TP1933Record Location: MECLABS Research LibraryResearch Partner: [Protected]
Experiment: Background
![Page 41: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/41.jpg)
Experiment: Side-by-side
COLUMN #1 COLUMN #2COLUMN #1
Control Treatment
![Page 42: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/42.jpg)
Which treatment won?
![Page 43: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/43.jpg)
Live: Jan. 22 at 4:00 p.m. EST
• What is the optimal number of columns for a page?
• How should columns be weighted on a page?• Which is better: a right sidebar or left
sidebar?
Watch live or get the full 35-minute video replay
To see the results …
To watch live or view the full video replay when it is available, register at the link below:
MarketingExperiments.com/Columns
![Page 44: Responsive Design Tested: What a recent experiment reveals about the potential ROI of mobile design](https://reader033.fdocuments.net/reader033/viewer/2022060107/554a5435b4c9054b328b49fd/html5/thumbnails/44.jpg)
See how you can conduct research with usMECLABS conducts rigorous experiments in the new science of optimization. We apply our discoveries to help leaders optimize the financial performance of their sales and marketing programs.
Learn more about how you may be a fit for a MECLABS Research Partnership:
• Select Research Partnership Opportunities on the post-webinar survey
• Contact us [email protected]
x