BIg Data is Not About Data

8/10/2019 BIg Data is Not About Data

1/118

Big Data is Not About the Data!

Gary King1

Institute for Quantitative Social ScienceHarvard University

Talk at the History of Evidence class, Harvard Law School, 11/17/2014

1GaryKing.org1/10


2/118

The Value in Big Data: the Analytics

2/10


3/118


Data:

2/10


4/118


Data: easy to come by; often a free byproduct of IT improvements

2/10


5/118


Data: easy to come by; often a free byproduct of IT improvements becoming commoditized

2/10


6/118


Data: easy to come by; often a free byproduct of IT improvements becoming commoditized Ignore it & every institution will have more every year

2/10


7/118



With a bit of effort: huge data production increases

2/10


8/118



With a bit of effort: huge data production increases Where the Value is: the Analytics

2/10


9/118




Output can be highly customized

2/10


10/118




Output can be highly customized Moores Law (doubling speed/power every 18 months)

2/10


11/118





v. Our Students (1000x speed increase in 1 day)

2/10


12/118






$2M computer v. 2 hours of algorithm design

2/10


13/118






$2M computer v. 2 hours of algorithm design Low cost; little infrastructure; mostly human capital needed

2/10


14/118






$2M computer v. 2 hours of algorithm design Low cost; little infrastructure; mostly human capital needed Innovative analytics: enormously better than off-the-shelf

2/10


15/118

Examples of whats now possible

3/10


16/118


Opinions of activists:

3/10


17/118


Opinions of activists: A few thousand interviews

3/10


18/118


Opinions of activists: A few thousand interviews billions of

political opinions in social media posts (1B every 2 Days)

3/10


19/118



political opinions in social media posts (1B every 2 Days) Exercise:

3/10


20/118



political opinions in social media posts (1B every 2 Days) Exercise: A survey: How many times did you exercise last

week?

3/10


21/118




week? 500K people carrying cell phones withaccelerometers

3/10


22/118





Social contacts:

3/10


23/118





Social contacts: A survey: Please tell me your 5 bestfriends

3/10


24/118





Social contacts: A survey: Please tell me your 5 bestfriends continuous record of phone calls, emails, textmessages, bluetooth, social media connections, address books

3/10


25/118






Economic development in developing countries:

3/10


26/118






Economic development in developing countries: Dubious ornonexistent governmental statistics

3/10


27/118






Economic development in developing countries: Dubious ornonexistent governmental statistics satellite images of

human-generated light at night, road networks, otherinfrastructure

3/10


28/118








Many, many, more...

3/10


29/118








Many, many, more...

In each: without new analytics, the data are useless

3/10


30/118

The End of The Quantitative-Qualitative Divide

4/10


31/118


The Quant-Qual divide exists in everyfield.

4/10


32/118



Qualitative researchers: overwhelmed by information; needhelp

4/10


33/118




Quantitative researchers: recognize the huge amounts ofinformation in qualitative analyses, starting to analyze

unstructured text, video, audio as data

4/10

f Q Q


34/118





unstructured text, video, audio as data Expert-vs-analytics contests: Whenever enough information is

quantified, a right answer exists, and good analytics areapplied: analytics wins

4/10

Th E d f Th Q i i Q li i Di id


35/118







Moral of the story:

4/10

Th E d f Th Q i i Q li i Di id


36/118







Moral of the story: Fully human is inadequate

4/10

Th E d f Th Q tit ti Q lit ti Di id


37/118







Moral of the story: Fully human is inadequate Fully automated fails

4/10

Th E d f Th Q tit ti Q lit ti Di id


38/118







Moral of the story: Fully human is inadequate Fully automated fails We needcomputer assisted, human controlledtechnology

4/10

The E d of The Q a titati e Q alitati e Di ide


39/118







Moral of the story: Fully human is inadequate Fully automated fails We needcomputer assisted, human controlledtechnology (Technically correct, & politically much easier)

4/10

How to Read a Billion Blog Posts


40/118

How to Read a Billion Blog Posts& Classify Deaths without Physicians

5/10



41/118


Examples of Bad Analytics:

5/10



42/118


Examples of Bad Analytics: Physicians Verbal Autopsy analysis

5/10



43/118


Examples of Bad Analytics: Physicians Verbal Autopsy analysis Sentiment analysis via word counts

5/10



44/118



Different problems, Same Analytics Solution:

5/10


45/118



46/118



Different problems, Same Analytics Solution: Key to both methods: classifying(deaths, social media posts)

Key to both goals: estimating %s

5/10



47/118




Key to both goals: estimating %s Modern Data Analytics: New method led to:

5/10



48/118





1.

5/10



49/118

g& Classify Deaths without Physicians




1.

2. Worldwide cause-of-death estimates for

5/10

The Solvency of Social Security


50/118

y y

6/10



51/118

y y

Successful: single largest government program; lifted a whole

generation out of poverty; extremely popular

6/10



52/118

y y


generation out of poverty; extremely popular Solvency: depends on mortality forecasts:

6/10



53/118


generation out of poverty; extremely popular Solvency: depends on mortality forecasts: If retirees receive

benefits longer than expected, the Trust Fund runs out

6/10



54/118




SSA data: little change other than updates for 75 years

6/10



55/118





SSA analytics:

6/10



56/118





SSA analytics: Few statistical improvements for 75 years

6/10



57/118





SSA analytics: Few statistical improvements for 75 years Ignore risk factors (smoking, obesity)

6/10



58/118





SSA analytics: Few statistical improvements for 75 years Ignore risk factors (smoking, obesity) Mostly informal (subject to error & political influence)

6/10



59/118





SSA analytics: Few statistical improvements for 75 years Ignore risk factors (smoking, obesity) Mostly informal (subject to error & political influence) Forecasts: inaccurate, inconsistent, overly optimistic

6/10



60/118






New customized analytics we developed:

6/10



61/118






New customized analytics we developed: Logical consistency (e.g., older people have higher mortality)

6/10



62/118






New customized analytics we developed: Logical consistency (e.g., older people have higher mortality) More accurate forecasts

6/10



63/118






New customized analytics we developed: Logical consistency (e.g., older people have higher mortality) More accurate forecasts Trust fund needs $800 billionmore than SSA thought

6/10



64/118






New customized analytics we developed: Logical consistency (e.g., older people have higher mortality) More accurate forecasts Trust fund needs $800 billionmore than SSA thought Other applications to insurance industry, public health, etc.

6/10

Following Conversations that Hide in Plain Sight


65/118

7/10



66/118

Example Substitution 1:

7/10



67/118


7/10



68/118


Freedom

7/10



69/118


Freedom

7/10



70/118


Freedom

7/10



71/118


Freedom Eye field

7/10



72/118


Freedom Eye field (nonsensical)

7/10



73/118

Example Substitution 1: Homograph


7/10



74/118



7/10



75/118




7/10



76/118




7/10



77/118




Harmonious [Society] (official slogan)

7/10


E l S b i i 1 H h


78/118





7/10


E l S b i i 1 H h


79/118





7/10


E l S b tit ti 1 H h


80/118




Harmonious [Society] (official slogan) River crab

7/10


E l S b tit ti 1 H h


81/118




Harmonious [Society] (official slogan) River crab (irrelevant)

7/10




82/118



Example Substitution 2: Homophone (sound like hexie)


7/10




83/118





They cant follow the conversation; Our methods can!

7/10




84/118






The same task:

7/10




85/118






The same task: (1) Government and industry analysts job,

7/10




86/118






The same task: (1) Government and industry analysts job, (2)language drift (#BostonBombings #BostonStrong),

7/10




87/118






The same task: (1) Government and industry analysts job, (2)language drift (#BostonBombings #BostonStrong), (3) Childpornographers,

7/10




88/118






The same task: (1) Government and industry analysts job, (2)language drift (#BostonBombings #BostonStrong), (3) Childpornographers, (4) Look-alike modeling,

7/10




89/118






The same task: (1) Government and industry analysts job, (2)language drift (#BostonBombings #BostonStrong), (3) Childpornographers, (4) Look-alike modeling,(5) Starting point forsophisticated automated text analysis

7/10

Computer-Assisted Reading (Consilience)


90/118

8/10


To understand many documents humans create categories to


91/118

To understand many documents, humanscreate categoriesto

represent conceptualization, insight, etc.

8/10




92/118


represent conceptualization, insight, etc. Most firms: impose fixed categorizations to tally customer

complaints, sort reports, retrieve information

8/10




93/118




Bad Analytics:

8/10


To understand many documents, humans create categories to


94/118




Bad Analytics: Unassisted Human Categorization: time consuming; huge

efforts trying notto innovate!

8/10


To understand many documents, humans create categories to


95/118





efforts trying notto innovate! Fully Automated Cluster Analysis: Many widely available,

but none work (computers dont know what you want!)

8/10




96/118

y , g






Our alternative: Computer-assisted Categorization

8/10




97/118

y , g






Our alternative: Computer-assisted Categorization You decide whats important, but with help

8/10




98/118

y g






Our alternative: Computer-assisted Categorization You decide whats important, but with help Invert effort: you innovate; the computer categorizes

8/10




99/118






Our alternative: Computer-assisted Categorization You decide whats important, but with help Invert effort: you innovate; the computer categorizes Insights: easier, faster, better

8/10


100/118

Example Insights from Computer-Assisted Reading


101/118

9/10


1. What Members of Congress Do


102/118

9/10


1. What Members of Congress Do Data: 64,000 Senators press releases


103/118

9/10




104/118

Categorization: (1) advertising, (2) position taking, (3) creditclaiming

9/10




105/118

Categorization: (1) advertising, (2) position taking, (3) creditclaiming New Insight: partisan taunting

9/10




106/118


Joe Wilson during Obamas State of the Union: You lie!

/




107/118



Senator Lautenberg Blasts Republicans as Chicken Hawks

/




108/118




How common is it?

/




109/118




How common is it? 27% of all Senatorial press releases!




110/118





2. Reverse Engineering Chinese Censorship

9/10




111/118





2. Reverse Engineering Chinese Censorship Previous approach: manual effort to see what is taken down

9/10




112/118





2. Reverse Engineering Chinese Censorship Previous approach: manual effort to see what is taken down Data: We get posts before the Chinese censor them

9/10




113/118





2. Reverse Engineering Chinese Censorship Previous approach: manual effort to see what is taken down Data: We get posts before the Chinese censor them We analyzed 11 million posts, about 13% censored

9/10




114/118





2. Reverse Engineering Chinese Censorship Previous approach: manual effort to see what is taken down Data: We get posts before the Chinese censor them We analyzed 11 million posts, about 13% censored Previous understanding: they censor criticisms of the

government

9/10


1. What Members of Congress Do Data: 64,000 Senators press releases Categorization: (1) advertising (2) position taking (3) credit


115/118


New Insight: partisan taunting





government Results:

9/10


1. What Members of Congress Do Data: 64,000 Senators press releases Categorization: (1) advertising (2) position taking (3) credit


116/118







government Results:

Uncensored: criticism of the government

9/10


1. What Members of Congress Do Data: 64,000 Senators press releases Categorization: (1) advertising, (2) position taking, (3) credit


117/118







government Results:

Uncensored: criticism of the government

Censored: attempts at collective action

9/10

For more information


118/118

GaryKing.org

10/10

BIg Data is Not About Data

Documents

Transcript of BIg Data is Not About Data