Post on 15-Oct-2020
Dark Side of AI/MLDevCamp München
Alexander Pospiech
�alexpospiech2018.04.20
Who Am I?
Data Engineer/Scientist @ inovex
� Security and Privacy Apologist
Father of OneÕ Dinghy-Sailor Nerd
Quadrants of the Dark Side
Intended UnintendedInside killer robots racist robotsOutside mislead robots ?
What is trust?
trustnounthe belief that you can trust someone or something
trustverbto believe that someone is good and honest and will not harm you,or that something is safe and reliable 1
1https://dictionary.cambridge.org/dictionary/english/trust
Quiz time
Do you trust Artificial Intelligence?
� �
Agenda
1 How it already has gone wrong - some Examples
2 Let’s here some warnings
3 What now?
Nguyen A, Yosinski J, Clune J. Deep Neural Networks are Easily Fooled: HighConfidence Predictions for Unrecognizable Images. In Computer Vision and PatternRecognition (CVPR ’15), IEEE, 2015.by Evolving AI Lab, University of Wyoming
Image Recognition Manipulation - Not so trippy
Goodfellow, Ian J., Jonathon Shlens, and Christian Szegedy. "Explaining andharnessing adversarial examples." arXiv preprint arXiv:1412.6572 (2014).by OpenAI
Video Recognition Manipulation - Assault Tortoises
Fooling Neural Networks in the Physical World with 3D Adversarial Objects (2017)by Anish Athalye, Logan Engstrom, Andrew Ilyas & Kevin Kwokat LabSix
Public Domain - OpenClipArtoriginal art: Autonomous Trap 001 (2017) by James Bridle
Autonomous Driving - Like in Looney Toons
Robust Physical-World Attacks on Deep Learning Models (2017)by Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, ChaoweiXiao, Atul Prakash, Tadayoshi Kohno, Dawn Song
Image Recognition Bias - Old, White Males
Gender Shades by Joy Buolamwini (2018) and her MIT group
�jessamyn west (2017)
�Perspectives (2017)
Image Recognition Bias - Let’s step back
Ripe Bananas Bananas with spots
Sugar bananas by Maksym Kozlenko
Mass Surveillance
Aktionstag (2017) by Endstation.jetzt
Countermeasures to Adversarial Examples
Accessorize to a Crime: Real and Stealthy Attacks on State-of-the-Art FaceRecognition (2016) by Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, Michael K.Reiter
Predictive Policing
minority-report-omg-02by youflavio
... the predictive models reinforceexisting police practices because
they are based on databases of crimesknown to police.
... tells us about patterns of policerecords, not patterns of crime.
Project: USAby Human Rights Data Analysis Group
Predictive Policing
minority-report-omg-02by youflavio
... a technologically obscuredtautology: the model predicts
approximately where crimes werepreviously known.
The model cannot predict patternsof crime that are different from thepatterns already known to police.
Project: USAby Human Rights Data Analysis Group
Predictive Policing
minority-report-omg-02by youflavio
... the differences in arrest rates byethnic group between predictive
policing and standard patrol practiceswere not statistically significant, ..."
... departments should monitor theethnic impact of these algorithms tocheck whether there is racial bias, ...
Article: Field-data Study Finds No Evidence ofRacial Bias in Predictive Policing (2018)
by Forensic Magazine
Predictive Policing - White Collar Detector
Responses to Critiques on Machine Learning of Criminality Perceptions by Xiaolin Wu,Xi Zhang
Predictive Judgment
3D Judges Gavel by Chris Potter
If you’re flagged, the chances it wasdeserved are equal, regardless of
race.
If you don’t deserve to be flagged,you’re more likely to be erroneously
flagged if you’re black.
Article: How to Fight Bias with PredictivePolicing (2018)
by Eric Siegel in Scientific American
Predictive Judgment - Breaking News
... COMPAS is no more accurate or fair than predictions madeby people with little or no criminal justice expertise.
... despite COMPAS’s collection of 137 features, the sameaccuracy can be achieved with a simple linear classifier with
only two features.
Paper: The accuracy, fairness, and limits of predicting recidivism (2018)by Julia Dressel and Hany Farid in Science Advances
Predictive Criminality - I have no words for this.
Public Domain - OpenClipArt
Faception
...recognizing “High IQ”,“White-Collar Offender”,
“Pedophile”, and “Terrorist” ...
According to Social and LifeScience research personalities
are affected by genes.
Our face is a reflection of ourDNA.
Faception
Agenda
1 How it already has gone wrong - some Examples
2 Let’s here some warnings
3 What now?
Elon Musk at the 2015Tesla Motors AnnualMeetingby Steve Jurvetson
�Elon Musk (2017)
John Giannandreaby TechCrunch
... be transparent about thetraining data that we are using, andare looking for hidden biases in it,...
If someone is trying to sell you a blackbox system for medical decisionsupport, and you don’t know how itworks or what data was used to train
it, then I wouldn’t trust it.
Article Forget Killer Robots—Bias Is the Real AIDanger (2017)
by John Giannandrea in Technology Review
Kate Crawford - PopTech2013 - Camden, MEby PopTech
People worry that computers will get toosmart and take over the world, but thereal problem is that they’re too stupid andthey’ve already taken over the world.
Article: There is a blind spot in AI research (2016)by Kate Crawford in Nature
Isaac AsimovPhilip K. Dick by PeteWelsch
Arthur C. Clark by ITUPictures
Book tips
Weapons of Math Destruction by Cathy O’Neil
QualityLand by Marc-Uwe Kling
Quiz time
Do you trust Artificial Intelligence?
� �
Agenda
1 How it already has gone wrong - some Examples
2 Let’s here some warnings
3 What now?
Quadrants of the Dark Side
Intended UnintendedInside ? Bias in model/data, wrong usageOutside Adversarial use ?
Cost of Misbehaving AI
Legal Consequences
Loss of Reputation
Loss of Opportunities
Loss of Money
Roles
ResearchersDevelopersUsersRegulators
Adversarial Attacks - Robustness
possible on all types of data and models!Find, investigate and train on attack vectors.Tools: cleverhans , DeepFool, deep-pwning, FoolBox, ...
Interpretability ⇒ Verification
Model: no black boxes
Data: available and transparent
Interpretability ⇒ Explainability ⇒ Understanding ⇒ Verification
Interpretability - LIME
Introduction to Local Interpretable Model-Agnostic Explanations (LIME) (2016)by Marco Tulio Ribeiro, Sameer Singh, Carlos Guestrin in O’Reilly
Introduction to Local Interpretable Model-Agnostic Explanations (LIME) (2016)by Marco Tulio Ribeiro, Sameer Singh, Carlos Guestrin in O’Reilly
Reproducibility
Reproducibility ⇒ Testability
In many real-world cases, the researcher won’t have made notes orremember exactly what she did, so even she won’t be able to
reproduce the model.
Article: The Machine Learning Reproducibility Crisis (2018)by Pete Warden
Yet AI researchers say the incentives are still not aligned withreproducibility.
Article: Missing data hinder replication of artificial intelligence studies (2018)by Matthew Hutson in Science
Fairness
Chris Anderson: “with enough data, the numbers speak forthemselves.”
Kate Crawford: "Sadly, they can’t. Data and data sets are notobjective; they are creations of human design."
Confidentiality - Privacy
Privacy + Encryption ⇒ Confidentiality
Differential Privacy
Homomorphic Encryption
Availability
Availability of the processing? Can I DOS a Neural Network?
Availability of predcitions or decisions?
Regulation
GDPR:
"Right to be forgotten"/"Right to erasure""Algorithmic Fairness" and "The Right to Explanation"
White House report: Preparing for the future of ArtificialIntelligenceHouse of Lords report: AI in the UK: ready, willing and able?Bundestag: some talk and a list of experts
Oversight
Human in the Loop?
Accountability
The vendor?
The users?
The AI?
Trust Availability
Testing
Higher LevelTech Problem
Robustness
Ethics
Technical Problem
Reproducibility
Verification
Fairness
Social Problem
Accountability
Privacy
Explainability
Regulation
Confidentiality
Interpretability
A chain of needed properties for trust in AI by Alexander Pospiech
Trust and Agency
Without our trust AI will grow regardlessly.
With the stated advancements AI will have our trust and maywork like expected.
Independent AI Trust Seal
TÜV, BSI, SomeOneNew, whoever
Tools, Standards, Controls, Audits
Transparency Reports
If you provide transparency information about legal requests, whynot about AI?
Physical Security
A neural network is some files on hardware.
Can be copied, stolen, modified, ...
Education
Educate AI basics in school and college
What can you do?
Techies and Non-Techies:
Educate, Warn, Support
Research, Develop
Quiz time
Do you trust Artificial Intelligence?
� �
Thank you for your attention!
Alexander PospiechBig Data Scientist
Data Management & Analytics
inovex GmbH - Office MunichLindberghstraße 3D-80939 München
+49. 173. 31 81 051alexander.pospiech@inovex.de�alexpospiech
Conferences and Meetings
Specific on the Dark Sides:Conference on Fairness, Accountability, and TransparencyFATML - Fairness, Accountability, and Transparency inMachine LearningInterpretable ML Symposium @NIPSNIPS 2017 Tutorial - Fairness in Machine LearningReproducibility in ML Workshop, ICML’18IEEE 1st Deep Learning and Security WorkshopData Ethics workshop, KDD 2014MAKE-Explainable AIAdvances on Explainable Artificial Intelligence
Generic on AI:AI for Good Global Summit
Conferences and Meetings
General on Security:CCCDefConSHABlackHat
Research Groups and Organizations
AI specific:AINow - A research institute examining the social implicationsof artificial intelligenceEvolving AI Lab, University of WyomingOpenAILabSixEFF on Artificial Intelligence & Machine LearningEFF - AI Progress MeasurementEvalAI - Evaluating state of the art in AIEvadeML - Machine Learning in the Presence of AdversariesAdversarial Machine Learning, Università degli Studi diCagliariSunBlaze at UCBDiskriminierung durch KI (Künstliche Intelligenz) (DiKI)Algorithmische Gegenmacht
Research Groups and Organizations
General:Human Rights Data Analysis GroupAlgorithmWatchNetzpolitik on Predictive Policing
Classes
CS 294: Fairness in Machine Learning, UC Berkeley18739 Security and Fairness of Deep Learning, CarnegieMellonAdversarial and Secure Machine LearningIEEE’s Artificial Intelligence and Ethics in Design
Themensammlung
Netzpolitik on Predictive PolicingEFF on Artificial Intelligence & Machine LearningEFF - AI Progress MeasurementEvalAI - Evaluating state of the art in AI
Github with Lists
Machine Learning for Cyber Security AwesomAwesome Adversarial Machine LearningIntroduction to Adversarial Machine LearningAwesome AI SecurityThe Definitive Security Data Science and Machine LearningGuideAwesome Machine Learning for Cyber Securityawesome-ai-privacyMachine Learning Ethics ReferencesFairness in Machine LearningToward ethical, transparent and fair AI/ML: a critical readinglist for engineers, designers, and policy makersAwesome Interpretable Machine Learning Awesome
Github with Code
Interpretability:H20.ai: Machine Learning Interpretability (MLI)Explanation ExplorerInterpretable Machine Learning with Pythoniml: interpretable machine learningML Insights
Fairness:Comparing fairness-aware machine learning techniques.Themis ML - Fairness-aware Machine Learning
Blogs
a blog about security and privacy in machine learningMLSeccovert.io security + big data + machine learningData Driven SecurityAutomating OSINTBigSnarfSecurity of Machine Learning
Videos - general
Youtube: Stephen Fry describing our future with artificialintelligence and robots34c3 - Beeinflussung durch Künstliche Intelligenz34c3 - Deep Learning BlindspotsSHA2017 - The Security and Privacy Implications of AI andMachine LearningYoutube - DEF CON 24 - Clarence Chio - Machine Duping101: Pwning Deep Learning SystemsYoutube: Do You Trust This Computer?TED - The era of blind faith in big data must end
Videos - specialized
[HUML16] 06: Zackary C. Lipton, The mythos of modelinterpretability"Why Should I Trust you?" Explaining the Predictions of AnyClassifier, KDD 2016Interpretable Machine Learning Using LIME Framework -Kasia Kulma (PhD), Data Scientist, Aviva
Adversarial Attack Competitions
MNIST Adversarial Examples Challenge
NIPS 2017 Competition: Non-targeted Adversarial Attack