Analog computing 62_1988-07_telecommunications - mike schoenbach
Analog Computing: Back to the Future? - ARPA-E Pitch_Haney_Analog... · Analog Computing: Back to...
Transcript of Analog Computing: Back to the Future? - ARPA-E Pitch_Haney_Analog... · Analog Computing: Back to...
![Page 1: Analog Computing: Back to the Future? - ARPA-E Pitch_Haney_Analog... · Analog Computing: Back to the Future? Michael Haney Program Director Bigstock photo 16th Century Italian woodcut](https://reader036.fdocuments.net/reader036/viewer/2022081611/5f0215017e708231d4027b24/html5/thumbnails/1.jpg)
Analog Computing: Back to the Future?
Michael Haney
Program Director
Bigstock photo16th Century Italian woodcut
![Page 2: Analog Computing: Back to the Future? - ARPA-E Pitch_Haney_Analog... · Analog Computing: Back to the Future? Michael Haney Program Director Bigstock photo 16th Century Italian woodcut](https://reader036.fdocuments.net/reader036/viewer/2022081611/5f0215017e708231d4027b24/html5/thumbnails/2.jpg)
Scaling of Si-based digital computing is coming to an end.
Can energy-efficient Analog Computing be transformative?
Motivation
ML/AI
IoT
Edge Computing
1
![Page 3: Analog Computing: Back to the Future? - ARPA-E Pitch_Haney_Analog... · Analog Computing: Back to the Future? Michael Haney Program Director Bigstock photo 16th Century Italian woodcut](https://reader036.fdocuments.net/reader036/viewer/2022081611/5f0215017e708231d4027b24/html5/thumbnails/3.jpg)
Original “Modern” Analog Computers
Mechanical embodiments
(1800’s – 1960’s)
US Tide Predicting Machine No. 2
“Old Brass Brains”
(1910-1965)
Electronic embodiments
(1940’s-1970’s)
X-15 simulator analog computer
(1960’s)
2
![Page 4: Analog Computing: Back to the Future? - ARPA-E Pitch_Haney_Analog... · Analog Computing: Back to the Future? Michael Haney Program Director Bigstock photo 16th Century Italian woodcut](https://reader036.fdocuments.net/reader036/viewer/2022081611/5f0215017e708231d4027b24/html5/thumbnails/4.jpg)
Example: ~1990 RAPID SAR DARPA project:
Real-time Acousto-optic Programmable Imaging and Display for SAR
What happened?
Digital Computing took over….
50 years+ of Moore’s Law scaling: ~2x in operations/J every 1.5 years
In 1990: ~100W analog vs. 10kW digital 100x more efficient!
In 2000: ~100W analog vs. ~100W digital! Not any better!
However, ~20 years later … 3
![Page 5: Analog Computing: Back to the Future? - ARPA-E Pitch_Haney_Analog... · Analog Computing: Back to the Future? Michael Haney Program Director Bigstock photo 16th Century Italian woodcut](https://reader036.fdocuments.net/reader036/viewer/2022081611/5f0215017e708231d4027b24/html5/thumbnails/5.jpg)
Computing Efficiency Reaching Limits
End of "Moore’s Law"
Physical limits (Gate oxide ~ 5 Si atoms thick; electron tunneling;
leakage current).
Source: B. Marr et al. IEEE Trans. Very Large Scale Integr. (VLSI) Syst. 21 (2013)
Source: J. Hasler and B. Marr, Front. Neurosci. 7 (2013) 4
![Page 6: Analog Computing: Back to the Future? - ARPA-E Pitch_Haney_Analog... · Analog Computing: Back to the Future? Michael Haney Program Director Bigstock photo 16th Century Italian woodcut](https://reader036.fdocuments.net/reader036/viewer/2022081611/5f0215017e708231d4027b24/html5/thumbnails/6.jpg)
Opportunity for Analog Computers?
Artificial Neural Networks
Basis of Artificial Intelligence applied to surge in Machine Learning workloads
Similar to the learning and inference processes of a brain
The connections are
analog: each line is
assigned a weight
that can represent a
real number.
Currently based on digital hardware –
What if the hardware was analog?
5
![Page 7: Analog Computing: Back to the Future? - ARPA-E Pitch_Haney_Analog... · Analog Computing: Back to the Future? Michael Haney Program Director Bigstock photo 16th Century Italian woodcut](https://reader036.fdocuments.net/reader036/viewer/2022081611/5f0215017e708231d4027b24/html5/thumbnails/7.jpg)
Analog
3 x 1014 “OPS”/W
Digital
6 x 108 FLOPS/W
Best commercial digital processor are ~1,000,000x less energy
efficient as the biological brain.
Opportunity for Analog Computers?
6
![Page 8: Analog Computing: Back to the Future? - ARPA-E Pitch_Haney_Analog... · Analog Computing: Back to the Future? Michael Haney Program Director Bigstock photo 16th Century Italian woodcut](https://reader036.fdocuments.net/reader036/viewer/2022081611/5f0215017e708231d4027b24/html5/thumbnails/8.jpg)
Increase in AI Workloads
ML: 300,000X
increase in compute
since 2012
Moore’s Law: 12X
increase
“Moore’s law”
Source: www.openAI.com
7
![Page 9: Analog Computing: Back to the Future? - ARPA-E Pitch_Haney_Analog... · Analog Computing: Back to the Future? Michael Haney Program Director Bigstock photo 16th Century Italian woodcut](https://reader036.fdocuments.net/reader036/viewer/2022081611/5f0215017e708231d4027b24/html5/thumbnails/9.jpg)
Does the “end of Moore’s Law” provide an opportunity for
ANN Analog Computing?
Source: Avi Baum, ee-news, July 2018
Source: Institute of Microelectronics, CMC, Fraunhofer, Jeppix
Electronic
• Resurgence in Analog Artificial
NNs computation and
memory circuit research.
Optical/Integrated Photonics
• Photonic ANNs
Feed-Forward
Fully-Connected
8x8 (analog)
Fully-
Connected
(spiking)
8
![Page 10: Analog Computing: Back to the Future? - ARPA-E Pitch_Haney_Analog... · Analog Computing: Back to the Future? Michael Haney Program Director Bigstock photo 16th Century Italian woodcut](https://reader036.fdocuments.net/reader036/viewer/2022081611/5f0215017e708231d4027b24/html5/thumbnails/10.jpg)
Analog ANN Computing – Questions to ponder:
• What are the limits in energy/operation?
• Can we efficiently exploit analog’s lower precision?
How/Where?
• What are the key technical challenges to overcome?
• How much computation can be off-loaded to more
efficient analog accelerators
In HPCs, now at ~10GFLOPS/W? For which
workloads?
In DCs, for AI/ML/DL– but are there also more
general applications?
Is it time for a deep dive into deep learning? 9
![Page 11: Analog Computing: Back to the Future? - ARPA-E Pitch_Haney_Analog... · Analog Computing: Back to the Future? Michael Haney Program Director Bigstock photo 16th Century Italian woodcut](https://reader036.fdocuments.net/reader036/viewer/2022081611/5f0215017e708231d4027b24/html5/thumbnails/11.jpg)
Thank You!
Acknowledgements: John Qi
Geoff Short