Why AI Goes Wrong And How To Avoid It...›Rolled out in 2015 to compete with instant gratification...

© 2018 FORRESTER. REPRODUCTION PROHIBITED.

Why AI Goes Wrong And How To Avoid It

Brandon Purcell

June 18, 2018

2© 2018 Forrester Research, Inc. Reproduction Prohibited

Source: https://twitter.com/jackyalcine/status/615329515909156865

We probably don’t need to worry about this in the near future…


Source: https://twitter.com/jackyalcine/status/615329515909156865

Google’s response:

“There is still clearly a lot

of work to do with

automatic image labeling,

and we're looking at how

we can prevent these

types of mistakes from

happening in the future."

But this is happening today

© 2018 Forrester Research, Inc. Reproduction Prohibited 4

Companies are learning that the road to hell is paved with good intentions


›Reputational erosion

›Revenue loss

›Regulatory fines

And they are paying for it in three ways:

Reputational risk: the erosion of brand equity

Microsoft Deletes ‘Teen Girl' AI After It Became A

Hitler-Loving Sex Robot Within 24 Hours

Amazon Prime And The Racist Algorithms


"I would definitely stop doing business with any company altogether if I found out that they discriminate against anyone!” – 27 year old female consumer


Source: Google Finance

Ethical failures erode shareholder value


› 9(1) Processing of personal data revealing racial or ethnic origin,

political opinions, religious or philosophical beliefs, or trade union

membership, and the processing of genetic data, biometric data for the

purpose of uniquely identifying a natural person, data concerning health or

data concerning a natural person's sex life or sexual orientation shall

be prohibited.

› For breaches against key points of GDPR such as the basic security

principles for processing data, obtaining consent and requirements to

internal transfers, the higher of 4% of annual global revenue or

€20,000,000 can be fined.

Biased AI could result in severe regulatory penalties


Why is this happening?


Source: https://www.forrester.com/report/The+Ethics+Of+AI+How+To+Avoid+Harmful+Bias+And+Discrimination/-/E-RES130023

Models can learn three types of bias



Algorithmic bias


A model is only as good as the

data used to train it


A crash course in machine learning

Supervised learning Unsupervised learning

Purpose To predict / classify To explore / understand

Training data Labelled (knows the

“answer”)

Not labelled (no “right

answer”)

Accuracy Measurable Qualitatively evaluated

Use cases for

marketing

Predict which customers

are likely to respond /

churn / buy

Behavioral customer

segmentation

The birds and the bees of model-making: supervised machine learning

Machine

learning

algorithm

Classification

model

Final output:

Newly

classified data

Labeled

dataTraining data

Unlabeled

data

Validation data


FaceApp demonstrates the problem of algorithmic bias


Bad training data created a racist filter


Algorithmic bias is caused by unrepresentative training data

Training data Total population


Algorithmic bias is caused by unrepresentative training data



Training data should be IID –independent and identically distributed



Training data should be IID –independent and identically distributed

Much better!


What happens when historical

biases are capture in the data?



Human bias


Even with good training data, models can pick up on human biases

Man is to woman as

Computer programmer is to _________

Google’s Word2Vec model for natural

language processing is sexist


They can be sexist…

Man is to woman as

Computer programmer is to homemaker

Google’s Word2Vec model for natural

language processing is sexist


Amazon Prime same day delivery shows the problem of human bias

› Rolled out in 2015 to compete with instant gratification

factor of brick & mortar retailers

› 27 metropolitan areas

› Postal codes with 77 million people

› Excludes predominantly black postal codes in 6 major

cities: Atlanta, Boston, Chicago, Dallas, New York, and

Washington, D.C.


A tale of two cities

Source: https://www.bloomberg.com/graphics/2016-amazon-same-day/

The blue shaded areas got

same day delivery


A tale of two cities

Source: https://www.bloomberg.com/graphics/2016-amazon-same-day/


› “Demographics play no role in it. Zero.” - Craig

Berman, Amazon’s VP, Global Communications

›Model based on concentration of Prime members

› Inherited historical human bias in the form of red-

lining and de facto segregation

› Included variables that are a proxy for race

Amazon did not intend to exclude predominantly black postal codes


Inherited human bias perpetuates that bias in a vicious cycle

Insights

Action

Data

Data with

human bias

Models with

human bias

Discriminatory

action


The perpetuation of human bias in the criminal justice system…

› COMPAS - Correctional Offender Management Profiling

for Alternative Sanctions


Can have devastating consequences

› Black defendants were almost twice as likely as white

ones to be falsely labelled future criminals

› Whites more likely to be mislabeled as low risk


Combatting human bias requires a deep understanding of the problem and the data

›Are you including variables that are proxies for

race, age, or other protected classes?

›Can you exclude these variables?

›Or can you modify the training data to reflect a

more just outcome?



Useful (intentional) bias


But sometimes it is ok to exploit differences between customers

›Who should you market these items to?


Models help you identify and take advantage of different preferences and behaviors

›Good luck selling Waldo’s sweater to Charlie

Brown!


When is it ok to treat different

customers differently…

and when isn’t it?


• Defining “ethical” needs to be an executive-level

conversation

• Business units should be responsible for overseeing

ethical deployment and measurement

• Data scientists should be the first line of defense

against algorithmic bias

Define roles and responsibilities for ensuring the ethics of algorithms


• Employ diverse perspectives at the data scientist, LoB,

and executive levels

• Listen to the Voice of the Customer for their opinions

• Consult experts in algorithmic bias

• Algorithmic Justice League – Joy Buolamwini

• University of Massachusetts at Amherst – Themis

• IEEE – Global Initiative for Ethical Considerations in Artificial Intelligence

and Autonomous Systems

Embrace diversity by soliciting a diverse array of viewpoints


Most importantly, make your models FAIR

Thank you

Brandon Purcell

[email protected]

Why AI Goes Wrong And How To Avoid It...›Rolled out in 2015 to compete with instant gratification...

Documents

Transcript of Why AI Goes Wrong And How To Avoid It...›Rolled out in 2015 to compete with instant gratification...