Introduction to Elemental Technologies: Software Defined Video Processing - Pop-up Loft Tel Aviv
Real Time Bidding on AWS - Pop-up Loft Tel Aviv
-
Upload
amazon-web-services -
Category
Technology
-
view
913 -
download
0
Transcript of Real Time Bidding on AWS - Pop-up Loft Tel Aviv
Real Time Bidding on AWS Dmitri TchikatilovAdTech BD, [email protected]
Agenda
1. Goals and how AWS works with RTB clients2. Observations and latest trends3. Building RTB solution on AWS4. Examples
RTB Definition
Real time bidding (RTB) refers to the practice of buying and selling display ad impressions through ad exchanges in real time and one impression at a time.
RTB Process Simplified1. User goes
to a web page
2. Ad impression is sent to an Ad
Exchange
3. Ad Exchange invites bidders
4. Highest bidder wins
the impression
5. Winning ad creative is
delivered
Programmatic Market - US
RTB $11B, 74% in 2015
Low margin - efficient operators have advantage
Programmatic direct buy is gaining market share too
Global RTB Market
1. Traffic is traded in a few key locationson AWS (e.g. us-east-1) and several large exchanges outside AWS
2. Easy to start buying traffic globally – roll out new AWS Regions (Korea, China, India – TBD)
RTB Elasticity for Traffic Volume Changes
Traffic changes drastically predictable – daily, weekly, seasonalAlso unpredictable – large scale events (LSE) (Kim Kardashian)
Sample Architecture
What is the Desired Degree of Control?
RTB Network Engineer
Where is the Non Differentiated Heavy Lifting?
Customer Manages
All-On Prem RTB
colo with Exchange S3,
Networking EC2
S3, Networking
EC2
S3, Networking
EC2
ASG, ELBDynamoDB
ASG, ELBDynamoDB
ManageAdminister
Algo, ML
Networking
• Days of RTB customers owning AS numbers are gone...• Large “legacy” colo display bidders and exchangers will
remain very relevant• New RTBs and Exchanges (Mobile, Video) pick AWS
and grow successfully• Simplified networking designs• Optimization of EC2 for networking, latest HVM AMIs
single root I/O virtualization (SR-IOV)
Front End Servers • Most people use ELB, sometime run their own ELB (HA Proxy, NginX)• Enable HTTP Persistent Connections (HTTP Keep-Alive)• Developer feedback - older legacy bidders C/C++, Java, Erlang, newer
systems – Node.JS, Python, Scala • Bidders are becoming more modular, dockerized• When traffic grows, use ELBs and ASGs dedicated to individual
exchanges• Single AZ deployments reduces the ELB latency by few ms.• Custom application level metric(s) is/are computed and published to
CloudWatch, ASG works off that custom metric(s)• Separating Bidding fleet from Exchange Listening (and not bidding)
Low Latency Caches
• Popular choices are: Aerospike, DynamoDB, Redis• DynamoDB hash (userId) + range (audience segment)
partitions• DynamoDB schemas considered to be a differentiator,
but we are looking to share some ideas and patters for best practices (user profile store/cookie store - low latency features for Machine Learning)
• Cross-region replication is becoming more common (Streams + Lambda) moving away from Storm
Data Collection + Analytics
Data from bidders is moved to a streaming service Apache Spark
Most popular choices are: Kinesis + Spark & Kafka + Spark
Batching of the data on the bidder to improve performance and economics
Example RTB – Cost Curve
Do I need co-located bidders?
Do I need a dedicated networking team?
Do I need to own networking equipment? 1 2 5 7 10
$0
$100,000
$200,000
$300,000
$400,000
$500,000
$600,000
Monthly RTB Fleet Spend vs. Roundrip Exchange Latency (ms)
Time (ms)
USD
Customer Perspective on RTB SolutionsCustomers have very different perspectives on what their business considers as competitive advantage
What is your competitive advantage?
What do you consider shareable knowledge? Customer 1 Customer 2 Customer 3
More advanced customers are sharing moreand raising the HL bar
HLHL
HL
“We run the RTB platform on more than 2,500 machines, approximately eight hours a day globally, at a cost of less than $0.05 per day per machine...”
“Because we’re running on AWS, we’re able to focus 95 percent of our staff on new product development. Using AWS allows us to focus on innovating our platform and solving customer problems.”
Valentino Volonghi, CTO AdRoll
Example: Enabling Real Time Bidding
Advertiser Solutions
Example: RTB Data Collection
Improved speed (mins to secs), simplicity & cost reductionOver 60 Billion events/day
Reducing data latency to seconds Ingesting approximately 150TB daily
Reducing Costs Everywhere
The company has saved tens of thousands of dollars. That’s between 20 and 30 percent of our total monthly AWS bill.
Gal Aviv Research & Development Group Manager
• 100 RIs + 800 Spot Instances• Partner – Spotinst
Check out EC2 Spot features – Spot Fleet API and Spot Bid Advisor
Solution Example: Ad Exchanges Outside AWS
20
EquinixAdIX
AWSCustomer
Ad ExchangeProvider
Ad ExchangeProvider
Ad ExchangeProvider
EquinixAdIX Ad Exchange
Provider
Ad ExchangeProvider
Ad ExchangeProvider
Ashburn
New York
Partner provides:Channel on NNI (<1GB) orDedicated port (>1GB)
Private IP
Public IP
Public IP
DX Partner
DX Partner
Reduce AWS traffic spend ~25%Predictable latency (vs. Internet)Reduced latency
AWS Regions as Centers of Gravity
Advertiser
Ad Exchange
Ad Network Publisher
Ad Network
Ad Network
Ad Network
Advertiser
Publisher
Reasons: Low latency Lower traffic costs Large scale secure
B2B data sharing
High Growth in Mobile Video
Dmitri TchikatilovDigital Advertising [email protected]