The Rise of Machine Intelligence
-
Upload
larry-smarr -
Category
Data & Analytics
-
view
140 -
download
0
Transcript of The Rise of Machine Intelligence
“The Rise ofMachine Intelligence”
Big Thought Leaders Colloquium Series – Spring 2017Jackson State University
Jackson, MSApril 11, 2017
Dr. Larry SmarrDirector, California Institute for Telecommunications and Information Technology
Harry E. Gruber Professor, Dept. of Computer Science and Engineering
Jacobs School of Engineering, UCSDhttp://lsmarr.calit2.net
1
Abstract
Over the next decade, we can expect autonomous machine intelligence to become pervasive in our society. To help spur that future, Calit2 has set up a Pattern Recognition Laboratory with a variety of novel low-energy processors that can execute real-time trained neural networks in the exploding mobile environment of drones, robots, and self-driving cars. However, the training of these neural networks requires massive amounts of Big Data and computing time. To support this need the NSF-funded Pacific Research Platform (PRP), which connects two dozen research universities at 100-1000 times the speed of the commodity Internet, is creating a new community of computer science machine learning researchers and proposing using the optical fiber backbone of the PRP to create a distributed Graphics Processing Unit computing “cloud.” Finally, I will speculate on the exponentially growing machine intelligence and how it will increasingly inter-operate with human intelligence.
Things Are About to Get Very Interesting…
Source: Hans Moravecwww.transhumanist.com/volume1/power_075.jpg
Smarr Slide from 2001
The Defining Issue in IT for the Coming Decades
May 5, 2015August 25, 2015
Traffic Control for Drone Air Delivery is Under Development by NASA, Amazon, & Google
Self-Driving Cars Have Appeared on the Market
I Am Living in the Self-Driving Future
Autopilot at 71 MPH
Streaming Data From the Tesla Fleet Trains Self-Driving Algorithms: The “Hive-Mind” Advantage
Note: Google Self-Driving Cars Have Only Driven 1.5 Million Miles
One Major Source of JobsMay Be the First Victim of Driverless Vehicles
DOD: “Perdix Drones Share One Distributed Brain for Decision-Making, Adapting to Each Other Like Swarms in Nature.”
The Planetary-Scale Computer Fed by a Trillion SensorsWill Drive a Global Industrial Internet
www-bsac.eecs.berkeley.edu/frontpagefiles/BSACGrowingMEMS_Markets_%20SEMI.ORG.html
Next Decade
One Trillion “Within the next 20 years the Industrial Internet
will have added to the global economy
an additional $15 trillion.”--General Electric
www.ge.com/docs/chapters/Industrial_Internet.pdf
What is the Cyberinfrastructure NeededFor The World of Big Data Autonomous Machines?
• Massive Multi-Architecture Cloud Computing
• Trained Neural Nets Downloaded onto Robots with NvN ML Accelerators
• Robots Use Neural Nets to Navigate with Real-Time Data Streams
• Swarm Input to Update Training on Neural Nets
For ¾ of a Century, Computing Has Reliedon von Neumann’s Architecture
Is It Time to Radically Expand Our Computer Architectures?
NCSA 1988
Supercomputer Architectures Remain von NeumannShared Memory CPU Plus SIMD Co-Processor
NCSA 2016
The DOE and NSF Petascale SupercomputersAll Are Built with CPU/GPU Nodes
Like Supercomputers, Commercial Cloud Providers Are Adding GPU Accelerators
All Use Double Precision Nvidia Tesla GPUs
But, Commercial Cloud Providers Are Also Introducing NvN Accelerators:Microsoft is Using Field Programmable Gate Arrays (FPGAs)
• Microsoft Installs FPGAs into Bing Servers– FPGAs are a Non von Neumann (NvN) Architecture– Improved the Ops/Sec of a Critical Component of Bing’s Search Engine by Nearly 2x– Many Other Applications and Services Can be Accelerated as Well
www.microsoft.com/en-us/research/project/project-catapult/
The Democratization of Deep Learning:Google’s TensorFlow
https://exponential.singularityu.org/medicine/big-data-machine-learning-with-jeremy-howard/
From Programming Computers Step by Step To Achieve a Goal
To Showing the Computer Some Examples of
What You Want It to Achieve and Then Letting the Computer
Figure It Out On Its Own--Jeremy Howard, Singularity Univ.
2015
Google Designed a NvN Machine Learning Accelerator
AI is Advancing at an Unprecedented Pace:Deep Learning Algorithms Working on Massive Datasets
1.5 Years!
Training on 30M Moves, Then Playing Against Itself
Google Used TPUs to Achieve the Go Victory
Exascale (1000 PetaFLOPs) Will Blend Traditional HPC and Data Analytics:U.S. Committed to Building by 2025
“High Performance Computing Will Evolve Towards a Hybrid Model,
Integrating Emerging Non-von Neumann Architectures, with Huge Potential in Pattern Recognition,
Streaming Data Analysis, and Unpredictable New Applications.”
Horst Simon, Deputy Director, U.S. Department of Energy’s
Lawrence Berkeley National Laboratory
Realtime Simulation of Human Brain Possible Within the Next Ten Years With Exascale Supercomputer
Horst Simon, Deputy Director, Lawrence Berkeley National Laboratory’s
National Energy Research Scientific Computing Center
Fastest Supercomputer
Trend LineTianhe-2
Using Nanotechnology to Read Out the Living BrainIs Accelerating Under the Federal Brain Initiative
www.whitehouse.gov/infographics/brain-initiative
Reverse Engineering of the Brain:Large Scale Microscopy of Mammal Brains Reveals Complex Connectivity
Source: Rat Cerebellum Image, Mark Ellisman, UCSD
NeuronCell Bodies
Neuronal DendriticOverlap Region
The Rise of Brain-Inspired Computers:Left & Right Brain Computing: Arithmetic vs. Pattern Recognition
Adapted from D-Wave
Brain-Inspired ProcessorsAre Accelerating the non-von Neumann Architecture Era
“On the drawing board are collections of 64, 256, 1024, and 4096 chips.
‘It’s only limited by money, not imagination,’ Modha says.”Source: Dr. Dharmendra ModhaFounding Director, IBM Cognitive Computing Group
August 8, 2014
Calit2’s Qualcomm Institute Has Established a Pattern Recognition Lab For Machine Learning on non-von Neumann Processors
“On the drawing board are collections of 64, 256, 1024, and 4096 chips.
‘It’s only limited by money, not imagination,’ Modha says.”Source: Dr. Dharmendra Modha
Founding Director, IBM Cognitive Computing Group
August 8, 2014
UCSD ECE Professor Ken Kreutz-Delgado Brings the IBM TrueNorth Chip
to Start Calit2’s Qualcomm Institute Pattern Recognition Laboratory
September 16, 2015
Our Pattern Recognition Lab is Exploring Mapping Machine Learning Algorithm Families Onto Novel Architectures
• Deep & Recurrent Neural Networks (DNN, RNN)• Graph Theoretic• Reinforcement Learning (RL)• Clustering and Other Neighborhood-Based• Support Vector Machine (SVM)• Sparse Signal Processing and Source Localization• Dimensionality Reduction & Manifold Learning• Latent Variable Analysis (PCA, ICA)• Stochastic Sampling, Variational Approximation• Decision Tree Learning
Source: Prof. Ken Kreutz-Delgado, Director PRL, UCSD
New Brain-Inspired Non-von Neumann Processors Are Emerging:KnuEdge is Essentially a Cloud-on-a-Chip That Scales to 512K Chips
www.tomshardware.com/news/knuedge-announces-knuverse-and-knupath,31981.html
www.calit2.net/newsroom/release.php?id=2704
“KnuEdge and Calit2 have worked together since the early days of
the KnuEdge LambdaFabric processor, when key
personnel and technology from UC San Diego
provided the genesis for the first processor design.”
www.calit2.net/newsroom/release.php?id=2726
June 6, 2016
KnuEdge Has Provided Processor to Calit2’s PRL
Neurobiological Systems are Flexible and Scalable
30
Animal # NeuronsFlatworm 302Medicinal leech 10,000Pond snail 11,000Sea slug 18,000Fruit fly 100,000Lobster 100,000Ant 250,000Honey bee 960,000Cockroach 1,000,000Frog 16,000,000Mouse 75,000,000Bat 110,000,000Octopus 300,000,000Human 100,000,000,000
Elephant 200,000,000,000
Current generation machine learning capabilities
Where KnuEdge wants to be in 2021:MindScale.
Source: Doug Palmer, CTO KnuEdge
Building a Big Data Cyberinfrastructure for Machine Learning
Based on Community Input and on ESnet’s Science DMZ Concept,NSF Has Funded Over 100 Campuses to Build Local Big Data Freeways
2012-2015 CC-NIE / CC*IIE / CC*DNI PROGRAMS
Red 2012 CC-NIE AwardeesYellow 2013 CC-NIE AwardeesGreen 2014 CC*IIE AwardeesBlue 2015 CC*DNI AwardeesPurple Multiple Time Awardees
Source: NSF
Next Step: The Pacific Research Platform Creates a Regional End-to-End Science-Driven “Big Data Superhighway” System
NSF CC*DNI Grant$5M 10/2015-10/2020
PI: Larry Smarr, UC San Diego Calit2Co-Pis:• Camille Crittenden, UC Berkeley CITRIS, • Tom DeFanti, UC San Diego Calit2, • Philip Papadopoulos, UCSD SDSC, • Frank Wuerthwein, UCSD Physics and SDSC
Letters of Commitment from:• 50 Researchers from 15 Campuses• 32 IT/Network Organization Leaders
FIONAs and FIONettes – Flash I/O Network Appliances:Linux PCs Optimized for Big Data Over Distance
FIONAs Are Science DMZ Data Transfer Nodes (DTNs) &
Also Compute/Visualization/ML Nodes
Phil Papadopoulos & Tom DeFantiJoe Keefe & John Graham
FIONAS—40G, $8,000FIONette—1G, $1,000
PRP Continues to Expand Rapidly While Increasing Connectivity:1 1/2 Years of Progress – 12 Sites to 23 Sites
January 29, 2016 March 29, 2016
Connected 23 DMZs at 10G and 40G, demonstrating disk-to-disk GridFTP
at ~7.5G and 12.5G respectively
PRP’s First 1.5 Years: Connecting Campus Application Teams and Devices
Adding a Cognitive Hardware and Software EcosystemTo the Pacific Research Platform
• Working With 30 CSE Machine Learning Researchers– Goal is 320 Game GPUs in 32-40 FIONAs at 10 PRP Campuses– PRP Couples FIONAs with GPUs into a Condor-Managed Cloud
• PRP Access to Emerging Processors– IBM TrueNorth, KnuEdge, FPGA, and Qualcomm Snapdragon
• Software Including a Wide Range of Open ML Algorithms • Metrics for Performance of Processors and Algorithms
Source: Tom DeFanti, Calit2
Multiple Proposals Under Review
FIONA with 8-Game GPUs
Single vs. Double Precision GPUs:Gaming vs. Supercomputing
8 x 1080 Ti: 1 million GPU core-hours every two days, 500 million for $15K in 3yrs
Contextual Robots Need Low Energy Neuromorphic Processors That Can See and Learn Wirelessly Tied Into the Planetary Cloud Computer
Professor Tajana Rosing
Kurzweil’s Theory of Mind: The Human Neocortex is a Self-Organizing Hierarchical System of Pattern Recognizers
“There are ~300M Pattern Recognizers
in the Human Neocortex.”
In the Emerging Synthetic Neocortex, “Why Not a Billion?
Or a Trillion?”
November 13, 2012
Should We Give Robots Autonomy?
This Next Decade’s Computing TransitionWill Not Be Just About Technology
"Those disposed to dismiss an 'AI takeover' as science fiction may think again after reading this original and well-argued book." —Martin Rees, Past President, Royal Society
If our own extinction is a likely, or even
possible, outcome of our technological
development, shouldn't we proceed with great
caution? – Bill Joy
Success in creating AI would be the biggest event in human history. Unfortunately, it might also be the last, unless we learn how to avoid the risks. – Steven Hawking
For Further Information:
http://lsmarr.calit2.net/