Economic Impact of Mixed Content Warnings on Consumer Behavior

Economic Impact of Mixed Content Warnings on Consumer Behavior

Sponsored by GhosteryIndependently conducted by Ponemon InstituteApril 2015

Ponemon Institute©: Private & Confidential Report

Economic Impact of Mixed Content Warnings on Consumer Behavior Ponemon Institute, April 2015

What is a mixed content warning? When a user visits a page served over HTTPS, his or her connection with the web server is encrypted with TLS and, hence, safeguarded from sniffers and man-in-the-middle attacks. If the HTTPS page includes content retrieved through regular, clear text HTTP, then the connection is only partially encrypted. The unencrypted content is accessible to sniffers and can be modified by man-in-the-middle attackers. Therefore, the connection is no longer safeguarded. When a webpage exhibits this behavior, it is called a mixed content page. Depending on the browser type, this results in a visual icon or pop-up that attempts to warn the visitor. Description of the project: Ghostery engaged Ponemon Institute to independently determine the economic impact of mixed content warnings. Specifically, we designed and fielded an experimental study that tests consumer reactions to mixed content warnings when browsing secure e-commerce sites. We utilized scientific sampling methods to recruit a representative sample of adult-aged consumers (a.k.a. respondents) located in the United States. Table 1 summarizes our survey response. We achieved a final sample of 1,577 qualified respondents or a 3.4 response rate. This experiment was conducted over a two-week period ending in March 2015.1 Table 1. Survey response Freq Total sampling frame (US consumers) 46,559 Total returns 1,732 Rejected or screened surveys 155 Final sample 1,577 Response rate 3.4%

Key takeaways: Most respondents (52 percent) have a basic understanding of what a mixed content warning means.

Respondents who view the standard warning on Internet Explorer have the highest continuance level or

lowest attrition rate. Respondents who view the standard warning on Chrome have the lowest continuance level or highest

attrition rate. Prior to participation in this research, most respondents (69 percent) can recall seeing mixed content

warnings either frequently or very frequently. Only 14 percent say they saw a mixed content warning for the first time.

The main reason for leaving a website after viewing the mixed content warning is concern about the

pop-up message displayed on the checkout page. Consumer attrition resulting from mixed content warnings on web pages is estimated to cost the top 100

Internet retailers in the United States $310 million per annum.

1 Respondents were compensated with a $5 dollar gift certificate or participation in a lottery.

10 East 39th St-8th Floor New York, NY 10016 | 917.262.2530 | ghosteryenterprise.com 1


Sample characteristics: Following are four charts that show the basic characteristics of individuals who participated in this study. Pie Charts 1 and 2 have a sample distribution of 1,577 respondents by gender and age, respectively.

Pie Chart 1: Gender Pie Chart 2: Age Range

Pie Charts 3 and 4 show the sample distribution by household income and education level, respectively.

Pie Chart 3: Household income Pie Chart 4: Education level

808 769

Female Male

80

292

411 320

196

137

141

Below 18 18 to 25 26 to 35 36 to 45

46 to 55 56 to 65 Above 65

190

277

590

328

61 58 40 33

Less than $25,000 $25,000 to $40,000

$40,001 to $60,000 $60,001 to $80,000

$80,001 to $100,000 $100,001 to $150,000

$150,001 to $250,000 More than $250,000

282

341

462

356

121 15

High School Vocational

College (no degree) College (degree)

Post Graduate Doctorate

10 East 39th St-8th Floor New York, NY 10016 | 917.262.2530 | ghosteryenterprise.com10 East 39th St-8th Floor New York, NY 10016 | 917.262.2530 | ghosteryenterprise.com 2


Experimental design Utilizing a survey instrument, we asked respondents to make a decision about continuing or discontinuing an online activity that displayed a mixed content warning. Respondents were randomly assigned to one of two website activities, described as follows: • Booking a car rental on a Hertz registration site (n1 = 781) • Buying a pair of sneakers on a Sneakerhead checkout page (n2 = 796) Task 1: Respondents were asked if they would continue an online activity such as booking a car or buying sneakers after viewing a “clean” webpage – that is, an HTTPS webpage that does not contain a mixed content warning. This reading served as our baseline. Task 2: Respondents were asked if they would continue an online activity such as booking a car or buying sneakers after viewing a “dirty” webpage – that is, an HTTPS webpage that contains one of three mixed content warnings. Here we used the standard warning displayed in Chrome, Internet Explorer or Foxfire. Dependent Variable: Our primary measure is each respondent’s attrition or churn decision after completing Task 2 versus his or her baseline result in Task 2. This aggregated attrition rate is used to extrapolate the total economic impact of mixed content warnings for online merchants (retailers). The following table summarizes our research design

Table 2: Experimental design Context Task1 Task 2 Difference Hertz A C X1 = C – A

Sneakerhead B D X2 = D – B Guiding hypotheses X1 Likelihood of attrition > 0 X2 Likelihood of attrition > 0

Experimental findings Figure 1 summarizes the respondents’ decision to continue or discontinue an online activity. Both Hertz and Sneakerhead results show a very low attrition for the baseline task and a very high attrition rate after seeing mixed content warnings. The aggregated attrition rate is 57 percent. Figure 1. Would you continue to book a car or buy sneakers online? Percentage Yes response

90% 88% 89%

31% 33% 32%

59% 55% 57%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Hertz Sneakerhead Combined

Task 1 (baseline) Task 2 (post-experiment) Attrition rate



Figure 2 summarizes the respondents’ decision to continue an online activity after seeing mixed content warnings in Chrome, Foxfire or Internet Explorer. As shown, respondents who view the standard warning on Internet Explorer have the highest continuance level or lowest attrition rate. In contrast, respondents who view the standard warning on Chrome have the lowest continuance level or highest attrition rate. Figure 2. Would you continue after seeing a mixed content warning? Percentage Yes response

Figure 3 summarizes the respondents’ self-reported level of understanding about mixed content warnings. These results suggest a majority of respondents (52 percent) believe they have a basic understanding about these messages. Figure 3. Do you understand what the mixed content warning means? Percentage Yes response

25%

33%

41%

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

Chrome Foxfire Internet Explorer

52% 48%

0%

10%

20%

30%

40%

50%

60%

Yes No



Figure 4 shows 69 (29+40) percent of respondents say they recall seeing mixed content warnings either frequently or very frequently prior to their participation in this research. Only 14 percent say they never saw a mixed content warning for this experimental study. Figure 4. Do your recall seeing a mixed content warning when browsing a website?

Figure 5 list the reasons why respondents decided to churn after viewing the mixed content warning. These results show an overwhelming majority of respondents (94 percent) were motivated to stop because of the standard warning. Figure 5. Why did you decide to discontinue after viewing the mixed content warning? More than one response permitted

29%

40%

17% 14%

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

Yes, frequently Yes, sometimes Yes, rarely No

1%

3%

6%

8%

94%

0% 20% 40% 60% 80% 100%

Other

I don’t like providing my personal information on websites

I don’t like the form used to capture [reservation or payment and billing] details

I don’t like [booking a car or buying sneakers] online

I’m concerned about the pop-up message displayed on the checkout page



Determining economic impact Based on the above analysis, we extrapolated the economic impact of consumer attrition resulting from mixed content warnings. We assume our guiding hypotheses X1 and X2 are validated and, hence, utilize the calculated attrition rate of 57 percent. Following are key factors utilized in this analysis: Targeted population: Top 100 Internet retailers headquartered in the United States Rate of mixed content messages: Non-secure calls provided by Ghostery Ghostrank data. This rate is the average compiled over three months for 18 top 100 U.S. online retailers. Online revenue and unique visitors: Determined from the Top 500 Internet Retailer Database (FY 2014 data points). Our analysis is contained in the following three tables. Table 3 lists 18 companies, all containing frequencies of non-secure calls (e.g., mixed content). This information is derived from the Ghostery Ghostrank data over a three-month period. All 18 companies are Top 100 U.S. Retailers. The first step is the collection of conversion rates (e.g., percent of visitors who make a purchase) for the list of 18 retailers. Conversion rates ranged from a low of 1.0 percent to a high of 8.9 percent. The second step is the calculation of a percentage represented by those visitors who saw non-secure calls and then churned or discontinued the web session. Hence, we multiply non-secure calls times 57 percent. This calculation is the basis from which we later determine economic impact.

Table 3. Determining consumer attrition after mixed content warning

Retailers Top 100

retailer rank

Percent of visitors who

make a purchase*


saw non-secure calls**


saw non-secure calls

and churned*** amazon.com 1 4.00% 1.96% 1.12% apple.com 2 4.00% 1.86% 1.06% bestbuy.com 15 1.30% 2.51% 1.43% crateandbarrel.com 77 2.12% 13.30% 7.58% etsy.com 30 2.30% 0.84% 0.48% gap.com 19 3.50% 13.93% 7.94% homedepot.com 16 1.30% 6.39% 3.64% lowes.com 36 1.30% 3.53% 2.01% macys.com 8 4.00% 1.16% 0.66% netflix.com 7 NA 1.76% 1.00% nordstrom.com 24 3.20% 4.30% 2.45% overstock.com 31 2.50% 3.50% 1.99% sears.com 5 1.00% 7.66% 4.36% staples.com 3 8.90% 8.01% 4.57% target.com 18 1.60% 14.58% 8.31% toysrus.com 34 3.00% 16.97% 9.67% walgreens.com 43 2.30% 28.03% 15.97% walmart.com 4 3.31% 22.18% 12.64% Average 20.72 2.92% 8.47% 4.83%

* Internet Retailer’s Top 500 Database ** Ghostrank data three-month average ***Derived from the Ponemon experiment NA Conversion rate was not available for Netflix



Drawing from the Internet Retailer’s Top 500 database, we obtain annual web sales for 18 retailers. We simply divide annual sales by 12 to determine monthly sales. Our third step is to calculate monthly web sales under the condition of zero attrition or no mixed content. Following is our gross-up formula: Monthly web sales ÷ (1 – [attrition X conversion]) Zero attrition would happen if mixed content was eliminated and, hence, mixed content warnings are no longer needed. Our fourth and final step is to calculate the difference between monthly web sales and grossed-up web sales for 18 companies. As shown in Table 4, the total monthly difference or “net gain” for 18 Internet retailers is $14,176,781.

Table 4. Calculation of net gain for 18 Internet retailers

Retailers

FY 2014 Annual web

sales ($billions)*

Monthly web sales

($millions)

Monthly web sales

assuming zero attrition

($millions) Net gain ($) amazon.com 79.50 6,625 6,628 2,960,124 apple.com 20.60 1,717 1,717 729,590 bestbuy.com 3.54 295 295 54,925 crateandbarrel.com 0.51 42 42 68,019 etsy.com 1.93 161 161 17,632 gap.com 2.50 208 209 580,399 homedepot.com 3.76 314 314 148,438 lowes.com 1.27 105 105 27,597 macys.com 5.40 450 450 118,947 netflix.com 5.50 458 458 NA nordstrom.com 2.50 208 208 163,367 overstock.com 1.50 125 125 62,229 sears.com 5.70 475 475 207,359 staples.com 11.23 936 940 3,820,383 target.com 2.99 249 249 331,851 toysrus.com 1.20 100 100 291,023 walgreens.com 1.13 94 94 345,724 walmart.com 12.14 1,011 1,016 4,249,173 Total 162.88 13,573.67 13,587.51 $14,176,781

* Internet Retailer’s Top 500 Database ** Ghostrank data three-month average ***Derived from the Ponemon experiment NA Conversion rate was not available for Netflix Table 5 contains the extrapolated economic impact of mixed content warnings on consumers; Internet behaviors. As shown, the estimated annual value for the 18 top retailers is over $170 million. We then gross up this value using a ratio based on total sales for 18 and the top 100. This produces a total annual estimated net gain of $310 million.

Table 5. Key measures of economic impact Monthly net gain for 18 retailers $14,176,781 Annual net gain for 18 retailers $170,121,377 Gross-up ratio* 55% Extrapolated value for Top 100 $309,674,297 *Ratio = Total sales for 18 retailers ÷ total sales for top 100 retailers



Appendix: Experimental Results

The following tables provide the results of our experimental study on mixed content warnings.

Frequencies


Displayed = clean home page 781 796 1577 Displayed = clean checkout page (no popup) 781 796 1577

Q1. Would you continue [to book a car or buy sneakers online] after seeing this page? Hertz Sneakerhead Combined

Yes 699 703 1402 No 82 93 175 Total 781 796 1577

Q2. If yes, please rate the likelihood that you would continue checkout? Use the following 10-point scale from 1 = not likely to 10 = very likely Hertz Sneakerhead Combined

1 or 2 54 68 122 3 or 4 168 156 324 5 or 6 176 185 361 7 or 8 165 175 340 9 or 10 136 119 255 Total 699 703 1402

Q3. If no, why? Hertz Sneakerhead Combined I don’t like [booking a car or buying sneakers]

online 44 56 100 I don’t like the form used to capture [reservation or

payment and billing] details 31 29 60 I don’t like providing my personal information on

websites 40 39 79 Other (please specify) 2 5 7


Displayed = dirty page ( randomly assigned one of three browser/message type) 781 796 1577

Q4. Would you continue to [book a car or buy sneakers] online after seeing this page? Hertz Sneakerhead Combined

Yes 246 263 509 No 535 533 1068 Total 781 796 1577

Q4. Would you continue to [book a car or buy sneakers] online after seeing this page? Hertz Sneakerhead Combined

Yes, Chrome 65 63 128 Yes, Firefox 80 90 170 Yes Internet Explorer 101 110 211 Total 246 263 509



Q5. If yes, please rate the likelihood that you would continue [booking a reservation or buying sneakers]? Use the following 10-point scale from 1 = not likely to 10 = very likely. Hertz Sneakerhead Combined


Q6. If no, why? Hertz Sneakerhead Combined I don’t like [booking a car or buying sneakers]

online 45 39 84 I don’t like the form used to capture [reservation or

payment and billing] details 30 39 69 I don’t like providing my personal information on

websites 17 18 35 I’m concerned about the pop-up message

displayed on the checkout page 506 499 1005 Other (please specify) 3 4 7 Q7. Do you understand what the pop-up message

actually means? Hertz Sneakerhead Combined Yes 405 416 821 No 376 380 756 Total 781 796 1577 Q8. Do your recall seeing a mixed content warning

when browsing a website like the one viewed before? Hertz Sneakerhead Combined

Yes, frequently 225 240 465 Yes, sometimes 309 314 623 Yes, rarely 129 138 267 No 118 104 222 Total 781 796 1577

Q9. How important is security of the websites you browse or shop? Use the following 10-point scale from 1 = not important to 10 = very important. Hertz Sneakerhead Combined


Q10. How important are the privacy commitments of the websites you browse or shop? Use the following 10-point scale from 1 = not important to 10 = very important. Hertz Sneakerhead Combined




Demographics D1. Gender: Hertz Sneakerhead Combined

Female 403 405 808 Male 378 391 769 Total 781 796 1577

D2. Age range: Hertz Sneakerhead Combined Below 18 38 42 80 18 to 25 142 150 292 26 to 35 201 210 411 36 to 45 167 153 320 46 to 55 96 100 196 56 to 65 67 70 137 Above 65 70 71 141 Total 781 796 1577

D3. Highest level of education: Hertz Sneakerhead Combined High School 139 143 282 Vocational 168 173 341 College (attended, no degree) 231 231 462 College (4 year degree) 176 180 356 Post Graduate 59 62 121 Doctorate 8 7 15 Total 781 796 1577

D4. Household income: Hertz Sneakerhead Combined Less than $25,000 93 97 190 $25,000 to $40,000 140 137 277 $40,001 to $60,000 287 303 590 $60,001 to $80,000 168 160 328 $80,001 to $100,000 29 32 61 $100,001 to $150,000 29 29 58 $150,001 to $250,000 19 21 40 More than $250,000 16 17 33 Total 781 796 1577

Please contact [email protected] or call us at 800.877.3118 if you have any questions.

Ponemon Institute

Advancing Responsible Information Management

Ponemon Institute is dedicated to independent research and education that advances responsible information and privacy management practices within business and government. Our mission is to conduct high quality, empirical studies on critical issues affecting the management and security of sensitive information about people and organizations.

As a member of the Council of American Survey Research Organizations (CASRO), we uphold strict data confidentiality, privacy and ethical research standards. We do not collect any personally identifiable information from individuals (or company identifiable information in our business research). Furthermore, we have strict quality standards to ensure that subjects are not asked extraneous, irrelevant or improper questions.


Economic Impact of Mixed Content Warnings on Consumer Behavior

Internet

Transcript of Economic Impact of Mixed Content Warnings on Consumer Behavior