Field Guide for Validating Premium Ad Inventory

43
Field Guide for Validating Premium Ad Inventory 1 Rami Essaid CEO & Co-Founder Andy Hibbler VP, Brand Advertising & Partnerships Dr. Augustine Fou Anti-Fraud Forensic Scientist & Founder MSCG marketing.science consulting group, inc.

Transcript of Field Guide for Validating Premium Ad Inventory

Page 1: Field Guide for Validating Premium Ad Inventory

Field Guide for Validating Premium Ad Inventory

1

Rami EssaidCEO & Co-Founder

Andy HibblerVP, Brand Advertising &

Partnerships

Dr. Augustine FouAnti-Fraud Forensic

Scientist & Founder MSCG

marketing.scienceconsulting group, inc.

Elias Terman
[email protected] charlie, this is still hard to parse. If we are laying out a taxonomy it needs to be organized better visually. In a table? with labels? How do hijacked tags, etc. fit in?
Charlie Minesinger
I edited it, 8:45pm ET
Elias Terman
much better! should I put in two tables with Term and Definition as labels?
Charlie Minesinger
edit as you like at this point ... I am all set with the content...I will rehearse a couple more times in the morning.
Elias Terman
[email protected] I might need you to take me through this one
Elias Terman
[email protected] Andy, are you going to expand these notes into slides?
Elias Terman
[email protected] is it ok if we take this out? You've got a lot of slides and I'm going to have Rami speak to the Bad Bot Landscape report and other data for his slides (which I'll work on next)
Augustine Fou
ok, replace with your data - much better; this was all I had from before
Elias Terman
[email protected] is it ok if we take this out. You've got a lot of slides and I'm going to have Rami speak to the Bad Bot Landscape report and other data for his slides (which I'll work on next)
Elias Terman
[email protected] can you please drop in a color photo of Rami so things are consistent
Charlie Minesinger
[email protected] ... This is NOT consistent with what we see and would not support the Value prop o WHY Whitepages uses Distil... they saw far higher bot rates and in fact we see pretty high rates on premium publishers... though not all the bots on Prem pubs execute .js, so they are not calling ads
Augustine Fou
ok, we can remove the slide if you prefer (or replace with something else)
Charlie Minesinger
[email protected] need to add axes here, it is not clear what we are looking at
Elias Terman
[email protected] can you please make this into an editable table?
Page 2: Field Guide for Validating Premium Ad Inventory

2

● Terminology and Standards● The State of Digital Ad Fraud

○ Landscape and Trends○ Display Fraud and Search Fraud○ Top Issues for Advertisers and Publishers○ Best Practices and Anti-Fraud Platforms

● Distil Networks Approach● Whitepages Premium Ad Inventory Case Study● Q&A

Agenda

Page 3: Field Guide for Validating Premium Ad Inventory

3

Landscape of Compliance and Standards Organizations● MRC (www.mediaratingcouncil.org) - viewability standards

adopted and aligned with Quality Assurance Guideline

● IAB (www.iab.net) - Anti-Fraud Working Group, authored the Ad Fraud Taxonomy

● TAG (www.tagtoday.net) - Formed from IAB, 4As, and ANA to focus exclusively on trustworthy supply chain, addressing fraud, piracy, and Non-Human Traffic (NHT)

Terminology and Standards

Page 4: Field Guide for Validating Premium Ad Inventory

4

Bad Bots in the TaxonomyNon-Human Traffic

Type Notes

Hijacked session/device fully automated browser

Crawler masquerading posing as searchbot or human

Datacenter traffic hosting companies, ISPs, etc.

Non-Traditional Sources of Traffic on AdsType Notes

Adware additional HTML calls without user action

Proxy traffic anonymous or known

Non-browser user agent fake mobile app, invalid UA, etc.

Browswer pre-rendering loading ad before page

Page 5: Field Guide for Validating Premium Ad Inventory

5

Registration Survey Results

N = 138

Page 6: Field Guide for Validating Premium Ad Inventory

6

Registration Survey Results

Page 7: Field Guide for Validating Premium Ad Inventory

7

Registration Survey Results

Charlie Minesinger
[email protected] match the x-axis scales in graphic for Slides 7 and 8 with survey results
Page 8: Field Guide for Validating Premium Ad Inventory

8

Live Survey

Page 9: Field Guide for Validating Premium Ad Inventory

9

State of Digital Ad Fraud

Landscape and Trends

Page 10: Field Guide for Validating Premium Ad Inventory

Fraud goes up as digital ad spend continues upward

Digital ad fraudHigh / Low Estimatesplus best-guess (36%)

Published estimates

Digital ad spendSource: IAB Annual Reports

$ billions

E

10

Page 11: Field Guide for Validating Premium Ad Inventory

Motive & Opportunity – More programmatic, more fraud

As more digital ads are placed entirely programmatically, the opportunity for fraud continues to increase.

Bad guys also fully automate their digital ad fraud operations, using programmatic tools.

11

Page 12: Field Guide for Validating Premium Ad Inventory

Impressions and clicks remain the biggest targets

Impressions(CPM/CPV)

Clicks(CPC)

Search$18.8B

86% digital spend

Display$7.9B

Video$3.5B Mobile

$6.2B$6.2B

Leads(CPL)

Sales(CPA)

Lead Gen$2.0B

Other$5.0B

• classifieds• sponsorship• rich media

estimated fraudnot at risk

12

$42.5B $7B

Page 13: Field Guide for Validating Premium Ad Inventory

Industry associations are leading the charge

“We must create an all-inclusive program that identifies qualified participants, and commits them to good actions, guaranteed by continual monitoring and sanctions for non-compliance”

Randall RothenbergIAB

“Together we can continue to rebuild the trust that is necessary for the interactive marketing industry to thrive”

Nancy HillPresident and CEO, 4A’s

“We’ve invested [heavily] in data and data technology and [have] been very much at the forefront of fraud verification,”

John MontgomeryGroupM

“It is imperative that the ecosystem addresses online fraud and improves media measurement and transparency.”

Bob LiodicePresident and CEO of the ANA

September 30, 2014 - IAB, 4A’s, and ANA announce cross-industry organization to fight ad fraud, malware, and piracy.

13

Page 14: Field Guide for Validating Premium Ad Inventory

14

State of Digital Ad Fraud

Display Fraud and Search Fraud

Page 15: Field Guide for Validating Premium Ad Inventory

Two main types of fraud and how each is generated

Impression (CPM) Fraud

(includes mobile display, video ads)

1. Put up fake websites and load tons of ads on the pages

Search Click (CPC) Fraud

(includes mobile search ads)

2. Use bots to repeatedly load pages to generate fake ad impressions (launder the origins of the ads to avoid detection)

1. Put up fake websites and participate in search networks

2a. Use bots to type keywords to cause search ads to load

2b. Use bots to click on the ad to generate the CPC revenue

15

Page 16: Field Guide for Validating Premium Ad Inventory

Bad bots are the primary tool for stealing ad dollars

Generate fake ad impressions and clicks• Bots repeatedly load pages or ads to make

impressions• Bots can also “type” search keywords and click

on search ads to create fake clicks

Retargeting fraud• Bots come to advertisers site to collect cookie • Bots visit other sites owned by bad guys• Retargeting follows them around to show ads • When ad impression is served on bad guys

sites, they earn the CPM revenue

16

Elias Terman
[email protected] I would kill this case study as we have a lot of content here already (also, I prefer not to give our competitors so much play). I would also divide this slide into three slides and ad imagery.
Augustine Fou
i am worried we wont fill even half the time slot (I think we dont have enough content)
Augustine Fou
but i can remove this slide and spend a bit more time on the following ones
Elias Terman
I'd just take out the white ops ANA case study, not the other bits
Page 17: Field Guide for Validating Premium Ad Inventory

Bad bots are the primary tool for stealing ad dollars

Diverting ad dollars

• CTR manipulation: bots go to premium site to generate pageviews without clicks (lowers the sitewide CTR ); optimizers divert dollars to sites with higher sitewide CTR (bad guy sites)

• Fake viewability: bad guys stack all ads behind each other so they are all viewable; optimizers divert dollars to sites with higher viewability (bad guy sites)

17

Page 18: Field Guide for Validating Premium Ad Inventory

Different kinds of bots generate fake impressions/clicks

Malware (on PCs)Botnets (from datacenters)

Toolbars (in-browser)Javascript (on webpages)

18

Page 20: Field Guide for Validating Premium Ad Inventory

Techniques to generate more impressions

AD

Ad stacking (dozens in one call) Pixel Stuffing (hiding more ads)

Source: DoubleVerify Case Study

Sourced Traffic (more pageviews) Ad Injection (replace ads)

Image Source: BenEdelman.org

The ANA/WhiteOps study also found rampant injection fraud, including one publisher whose site was hit with 500,000 injected ads every day for the duration of the two-month study.

20

Page 21: Field Guide for Validating Premium Ad Inventory

Techniques to hide fraudulent activity

Domain / IP Spoofing Impression Laundering

Source: Whiteops/ANA Study Dec 2014

Stacked Redirects Passing fake variablesKnown blackhat technique to hide real referrer and replace with faked referrer.

Example how-to:http://www.blackhatworld.com/blackhat-seo/cloaking-content-generators/36830-cloaking-redirect-referer.html

Make it appear that the impression came from legit site

http://www.olay.com/skin-care-products/OlayPro-X?utm_source=msn&utm_medium=cpc&utm_campaign=Olay_Search_Desktop_Category+Interest+Product.Phrase&utm_term=eye%20cream&utm_content=TZsrSzFz_eye%20cream_p_2990456911

Easily trick analytics platforms to think the impression came from legit source

21

Page 22: Field Guide for Validating Premium Ad Inventory

22

State of Digital Ad Fraud

Impact on Advertisers and Publishers

Page 23: Field Guide for Validating Premium Ad Inventory

Fraud siphons 1/3 of dollars out of ad ecosystem

Advertisers “ad spend” in

digital is $56B in 2015

Publishers are left with only 2/3

of the dollars

Bad Guyssiphon 1/3 of ad spend OUT of the ecosystem

• Ad dollars are being siphoned OUT of the ecosystem into the pockets of the bad guys• Advertisers have lower ROI due to fraud (fake impressions, non-humans)• Publishers have lower revenues (ad dollars stolen by bad guys)

2/3

1/3

Usersuse ad blocking and

need to protect privacy

23

Page 24: Field Guide for Validating Premium Ad Inventory

Advertisers and Publishers are Harmed DifferentlyAdvertisers

1. Advertisers attempt to increase revenues by increasing digital reach and frequency.

2. But more ad impressions doesn’t mean more ads seen by people

3. Bots causing webpages and ad impressions to load directly cost advertisers -- in the form of wasted ad dollars that never have a chance to convert into paying customers

Publishers

24

1. Publishers attempt to increase revenues by increasing the number of ads served or the CPM they charge

2. Bots coming to pages not only drain web serving resources but they also have the potential to damage legitimate publishers’ reputation when selling ads

3. Bots also steal valuable content and collect cookies, both of which help bad guys divert traffic and ad revenue to fraudulent sites they own, resulting in direct loss of ad revenue for publishers.

Page 25: Field Guide for Validating Premium Ad Inventory

25

State of Digital Ad Fraud

Best Practices and Anti-Fraud Platforms

Page 26: Field Guide for Validating Premium Ad Inventory

Recommendations for reducing fraud

Move beyond just viewability; tackle NHT/bot fraudEven if viewability was solved, if the impression was caused to load by a bot, it is still fraud and money wasted for the advertiser

Allocate budget toward mainstream, premium publishersAllocate ad spend away from long-tail sites in ad exchanges that have the most fraud; the more direct way to reduce bot fraud is to buy mainstream sites

Demand full transparency and independently verifyIf they can’t show you where each ad impression was served, don’t buy it; put in place your own checks and balances

Focus less on impressions served or traffic, more on observable actionsIf we focus on actions that can be observed rather than tonnage of ads served, and incentivize our agencies differently, we avoid the temptation of buying lower cost (lower quality) ad inventory

26

Page 27: Field Guide for Validating Premium Ad Inventory

Comparison matrix of anti-fraud solutions

Example Vendors Description Mitigation

WhiteOpsForensiq

On-page Typically javascript code installed on websites (like analytics). Detects parameters about the user and computing environment

blacklist domains that send traffic that is mostly bots

Integral Ad ScienceIn-ad exchange

Real-time decisioning in the bid-stream using rules and blacklists (e.g. if a site is on a blacklist, don’t serve the ad)

eject users and reject sites in real-time based on blacklists

SolveMediaAreYouAHuman

Challenge based

Using captchas and other challenges which ask humans to verify themselves as humans

identify and whitelist humans for use in RTB decisioning

Distil NetworksReverseProxy

Inspection of all page requests to prevent scraping, content theft, and fraud BEFORE the ad call and pixel fires. Provides real time protection without relying on IP address

blocks all traffic from bad bots, rejects individual browsers, builds real-time whitelists and blacklists

27

Page 28: Field Guide for Validating Premium Ad Inventory

28

Distil Networks Appoach

Distil Networks Bot Detection and Mitigation

Page 29: Field Guide for Validating Premium Ad Inventory

2929

Single Pixel with Ad vs Reverse Proxy

Tech: Pixel for Tracking → Tech: Security & PerformanceSingle js injection → Multiple injectionsPredictable .js every time → Randomized, SpoofproofProbabilistic → Technical tests on browserReactive, post ad call → Proactive, pre-page loadUA & IP = fingerprint → 40+ parameters for fingerprintIP/Network intelligence → Domain specific modeling

Dealing with Unsolicited NHT

Page 30: Field Guide for Validating Premium Ad Inventory

30

Distil’s Accurate Protection

Inline FingerprintingFingerprints stick to the bot even if it attempts to reconnect from random IP addresses or hide behind an anonymous proxy. Known Violators DatabaseReal-time updates from the world’s largest Known Violators Database, which is based on the collective intelligence of all Distil-protected sites.

Browser ValidationThe first solution to disallow browser spoofing by validating each incoming request as self-reported and detects all known browser automation tools.

Behavioral Modeling and Machine LearningMachine-learning algorithms pinpoint behavioral anomalies specific to your site’s unique traffic patterns.

Page 31: Field Guide for Validating Premium Ad Inventory

31

No Impact on Legit UsersWithout Distil

With Distil

Page 32: Field Guide for Validating Premium Ad Inventory

32

DIGIDAY & Distil Networks Survey

Page 33: Field Guide for Validating Premium Ad Inventory

Understand Your Traffic

Page 34: Field Guide for Validating Premium Ad Inventory
Page 35: Field Guide for Validating Premium Ad Inventory

Serve CAPTCHAs to Bots

Page 36: Field Guide for Validating Premium Ad Inventory

Good Bot Report

Page 37: Field Guide for Validating Premium Ad Inventory

Choose Your Bot ResponseMonitorMonitor to inspect requests and record the traffic to Distil and/or your own server logs BlockSet to Block to serve the client an unblock verification form

CAPTCHA Serve a hardened CAPTCHA to test the client for verification

DropDrop them to present them with an access denied page

Page 38: Field Guide for Validating Premium Ad Inventory

38

Digital identity starts here

Page 39: Field Guide for Validating Premium Ad Inventory

39

Whitepages is committed to eliminating fraud

Page 40: Field Guide for Validating Premium Ad Inventory

40

Bots don’t buy cars

Page 41: Field Guide for Validating Premium Ad Inventory

41

Our recipe for delivering honest impressions

Page 42: Field Guide for Validating Premium Ad Inventory

42

Whitepages Results Partnering with Distil Networks

• Stopped “comment spam” and prevented scrapers from stealing data

• Reduced time spent detecting bad bots and eliminated false positives

• Allowed servers to run and load impressions at optimal speeds

• Created a reliable and safe environment for brands (<5% IVT)

Page 43: Field Guide for Validating Premium Ad Inventory

43

Q&A

Rami EssaidCEO & Co-Founder

Andy HibblerVP, Brand Advertising &

Partnerships

Dr. Augustine FouAnti-Fraud Forensic

Scientist & Founder MSCG

marketing.scienceconsulting group, inc.