Chief Analytics Officer Fall USA 2017 - Matthew Marolda - Legendary

Applying Analytics in Unique Settings

September 2017

Topics

•Personal Context

•Building Analytics at Legendary

•Applying Analytics at Legendary

•Lessons Learned

Personal Context

1999 2003 2005 2010 2012 2016

Predict Fraud or Failure for tens of thousands of publicly traded companies Founded StratBridge in 1999; Sold in 2003

Price tickets dynamically based on changes in demand Founded StratTix in 2003; Sold in 2013

Evaluate pro players for trade, free agency, draft Founded StratEdge in 2005; Sold in 2011

Inform Creative process and Transform Marketing Founded Legendary Analytics 2012; Legendary Sold 2016; Commercialized Legendary Analytics 2017

Commonalities

Leverage transformation of data collection

Side-bar / Pet Peeve Data is not “created” –

it is captured or measured

Commonalities


Financial Data Shift from paper SEC filings to text filings and databases

Ticketing Data Expansion of meta-data to include more than simple transaction info

Player Data Introduction of play-by-play logs and player movement systems

People Data Explosion of behavioral data and unstructured data sources

Commonalities


Push boundaries of processing power and storage

Distill complex analytics into human-ready outputs

Operate in ambiguous environments

Drive actions that integrate analytics and judgment

Topics

•Personal Context



•Lessons Learned

Building Analytics at Legendary

Focus on the Practical

Driving All Strategic Decisions with Data and Analytics

Inform Creative

Transform Marketing

Legendary Applied Analytics

Legendary Cognitive Platform: Analytics

Custom Primary Surveys • Guided heavily by latest research in social psychology

• Designed to avoid leading questions and other biases

• Incentives and engaging questions including images and video encourage quality responses

• Balance of quantitative and qualitative feedback

• Open-ended responses provide direction and context

• Consistency in design for optimal comparison and benchmarking

68% 65% 59% 58%

54%

13% 12% 12%

0%

40%

80%

TV Word of Mouth

(not online)

Social Networking

Websites

At the Theater

Websites Other Than

Social Networking

Outdoor Advertising (billboards, bus stops)

Radio Print Publications

(newspapers, magazines)

Example: Through which media do you typically find out about new movies being released in theaters? Please select all that apply.

% C

HO

SE

N

Innovative, modern approaches to traditional tools



Find Data Independence

Legendary Cognitive Platform: Data Overview

Entertainment Industry (esoteric)

People (Digital) People (Physical)

Conversation and Zeitgeist

• 200M households with PII and occupants • Demographics • Interests and Behaviors • Financial details and Transactions • Unique set-top box integration

• 1.5B email addresses • 500M Twitter profiles • 200M Facebook profiles • Hundreds of inferred attributes, interests and

behaviors from proprietary Legendary analytic tools

• Extensive details on movies and tv shows • All US TV movie ads and spots run since 2007 • Box Office by theater for all movies since 2007

• Billions of Tweets • All of Reddit • All of Wikipedia • All published news articles back to 2007 • Major blogs and other online publications

Substantial sets of data integrated in unique ways

VIEWER EXPERIENCE MERCHANDISE

COMPARABLE ENTITIES

HASHTAGS/ SHOUTOUTS

CONVERSATION GROUP

# OF PEOPLE IN GROUP

234,205 7,702 85,684 59,472

TOP DISCUSSION WORDS

great sequel watch

DVD eBay

blu-ray

Pacific Rim Jurassic World

Avengers

@GodzillaMovie @Legendary

#Godzilla

Segmentation: Topical Segmentation • Tweets express user interests and

reactions in real-time, creating the “Twitterverse”

• But the volume of data makes manual inspection prohibitive

Should I watch Godzilla

or Pac Rim first?

Godzilla on DVD! <3 Godzilla

#Godzilla

Godzilla was awesome

Legendary leverages statistical techniques to segment users into topically-coherent conversation groups

Image Recognition: via Neural Networks

Recent advancements in neural network training are dramatically accelerating image recognition capabilities

Image Recognition: Age Inference Accuracy

Predicted age

< 18 18-24 25-34 35-44 45-54 55-64 65 +

< 18

18-24

25-34

35-44

45-54

55-64

65 +

Age

Tru

e ag

e

Facial features are highly predictive of age; 60% better than given names or interests

High

Low




Challenge pre-existing Conventions

Hollywood Paradigm: Four Quadrants

Note: Using the US population as an illustrative example.

Traditional Hollywood Four Quadrant

Legendary Applied Analytics Micro Segment

Four Quadrant: 4 groups of 80 million Micro Segment: 80 million groups of 4

Hollywood Paradigm: Spray and Pray

General Population

Hard Core Fans

Market To

p D

ow

n

Audience

Box Office

Legendary Analytics: Invert Audience Building

Extended Reach

Efficiency Gain

Market B

otto

m U

p

Box Office

Audience

Hard Core Fans

General Population





Create the right Data Science Environment

People and Locations

Based in Boston Extension

in LA

Growing in Beijing

Diverse set of over 90 people and growing





Create the right Data Science Environment

Focus on what matters, not what can be measured

Conversion Emphasis

Population

Aware

Interested

Converted

Individuals Interests Demos Audience

Manage a specific Audience based on Awareness and Interest to drive Conversion

Topics

•Personal Context



•Lessons Learned

LEGENDARY MOVIES CHINA CASE STUDY WARCRAFT, THE GREAT WALL, KONG: SKULL ISLAND

$160 million Production $150 million Production $185 million Production

$120 million Advertising $100 million Advertising $120 million Advertising

26% by critics on Rotten Tomatoes 35% by critics on Rotten Tomatoes 77% by critics on Rotten Tomatoes

83% by fans on Rotten Tomatoes 49% by fans on Rotten Tomatoes 73% by fans on Rotten Tomatoes

Three “tentpole” movies that had a range of receptions

APPLIED ANALYTICS SOPHISTICATED TARGETED MARKETING

Apply and adapt established models and tools in China

Drive marketing decision making

Create closed loop to analyze, predict, and drive ticket sales

TARGETED MARKETING BASED ON STATISTICAL MODELING

1) Identify geographies most likely to have strong box office

2) Utilize unique data relationships to analyze billions of data

3) Train a model to predict likelihood of interest

4) Focus on persuadable audience (not “givens” or “nevers”)

5) Optimize creative by micro-segments

6) Deliver large scale ads

7) Use actual ticket pre-sales to calibrate models

8) Drive ticket sales (including chase)

City level Box office analysis

Audience creative reception analysis Model coefficients affect analysis

PLATFORMS

Search: 79M impressions Nuomi: 29M individuals Maps: 17M individuals Music: 16M individuals QQ Tips: 34.3M targeted ads

Search: 41M impressions Nuomi: 143M impressions Mobile: 27M impressions Tieba: 20M impressions OTV: 38M impressions

Search: 24M impressions Mobile: 27M impressions Tieba: 14M impressions

TARGETED INDIVIDUALS

150+ million 140+ million 100+ million

IMPACT 6.5 ROI

3.0x more likely to buy

6.2 ROI


8.0 ROI


TARGETED MARKETING AT SCALE WARCRAFT, THE GREAT WALL, KONG: SKULL ISLAND

MEDIA REACTION WARCRAFT

$300M+ box office in its first week, $430M+ total, with a record setting $230M in China

MEDIA REACTION GREAT WALL

China opening of $67.4M, growing to $171M total China box office, despite mediocre reviews and tepid audience reacion in market

MEDIA REACTION KONG: SKULL ISLAND

China opening of $72M, growing to $170M total China box office, Becoming one of the top ten English releases of all time

Topics

•Personal Context



•Lessons Learned

Lessons Learned

Shed Hubris

Embrace Naivete

Question Everything

Do not Discard the Traditional

Use Many Techniques to Triangulate the Solution

Chief Analytics Officer Fall USA 2017 - Matthew Marolda - Legendary

Data & Analytics

Transcript of Chief Analytics Officer Fall USA 2017 - Matthew Marolda - Legendary