Domain-specific Architectures r2 -...

36
Domain-Specific Architectures for AI and Robotics: Opportunities and Challenges Contact Info email: [email protected] website: www.rle.mit.edu/eems Vivienne Sze Massachusetts Institute of Technology

Transcript of Domain-specific Architectures r2 -...

Page 1: Domain-specific Architectures r2 - ict.ac.cncrva.ict.ac.cn/documents/agile-and-open-hardware/sze...1.DPM v5 [Girshick, 2012] 2.Fast R-CNN [Girshick, CVPR 2015] Exponential Linear Video

Domain-Specific Architectures for AI and Robotics: Opportunities and Challenges

Contact Infoemail: [email protected]: www.rle.mit.edu/eems

Vivienne SzeMassachusetts Institute of Technology

Page 2: Domain-specific Architectures r2 - ict.ac.cncrva.ict.ac.cn/documents/agile-and-open-hardware/sze...1.DPM v5 [Girshick, 2012] 2.Fast R-CNN [Girshick, CVPR 2015] Exponential Linear Video

Wide Range of Compute-Intensive Applications2

Video Compression

Robotics:Autonomous Navigation

AI:Deep Neural Networks

Input Layer

Output Layer

Hidden Layer

Weights Weights

• Rapidly growing volume of data to be processed• Increasingly complex algorithms for higher quality of result• Require high throughput/low latency and energy efficiency

On-c

hip B

uffer Spatial

PE Array

4mm

4mm

Slice Engine 0Slice Engine 1Slice Engine 2Slice Engine 3Slice Engine 4Slice Engine 5Slice Engine 6Slice Engine 7

Slice Engine 8Slice Engine 9

Slice Engine 10Slice Engine 11Slice Engine 12Slice Engine 13Slice Engine 14Slice Engine 15

Core

Dom

ain

Inte

rface

Dom

ain

3.8 mm

4.6

mm Eyeriss

[ISSCC 2016]Navion

[VLSI 2018]

Parallel CABAC

[ISSCC 2011]

Need Domain Specific Architectures à 2 to 3 years to design!

Page 3: Domain-specific Architectures r2 - ict.ac.cncrva.ict.ac.cn/documents/agile-and-open-hardware/sze...1.DPM v5 [Girshick, 2012] 2.Fast R-CNN [Girshick, CVPR 2015] Exponential Linear Video

• Exploit properties of workloads– Specialized hardware translates parallelism, data access patterns and

representation into increased throughput and energy efficiency

• Design more efficient workloads– Co-design of algorithms and hardware without affecting quality of result

• Define range of workloads – Balance flexibility and efficiency depending on application requirements

Key Design Considerations3

How can Agile and Open Hardware help accelerate the design process?

Page 4: Domain-specific Architectures r2 - ict.ac.cncrva.ict.ac.cn/documents/agile-and-open-hardware/sze...1.DPM v5 [Girshick, 2012] 2.Fast R-CNN [Girshick, CVPR 2015] Exponential Linear Video

Deep Neural Networks (DNN)

4

Page 5: Domain-specific Architectures r2 - ict.ac.cncrva.ict.ac.cn/documents/agile-and-open-hardware/sze...1.DPM v5 [Girshick, 2012] 2.Fast R-CNN [Girshick, CVPR 2015] Exponential Linear Video

Properties We Can Leverage• Operations exhibit high parallelism

à high throughput possible

• Memory Access is the Bottleneck

ALU

Memory Read Memory WriteMAC*

* multiply-and-accumulate

filter weightimage pixelpartial sum updated

partial sum

200x 1x

DRAM DRAM

5

Page 6: Domain-specific Architectures r2 - ict.ac.cncrva.ict.ac.cn/documents/agile-and-open-hardware/sze...1.DPM v5 [Girshick, 2012] 2.Fast R-CNN [Girshick, CVPR 2015] Exponential Linear Video

Properties We Can Leverage• Operations exhibit high parallelism

à high throughput possible

• Input data reuse opportunities (up to 500x)

ConvolutionalReuse

(pixels, weights)

Filter Image…

… ………

ImageReuse(pixels)

… … …

2

1

Filters

Image

FilterReuse

(weights)

…… … …

… … …

…Filter

Image

2

1

6

Page 7: Domain-specific Architectures r2 - ict.ac.cncrva.ict.ac.cn/documents/agile-and-open-hardware/sze...1.DPM v5 [Girshick, 2012] 2.Fast R-CNN [Girshick, CVPR 2015] Exponential Linear Video

Data Movement is Expensive

DRAM Global Buffer

PE

PE PE

ALU fetch data to run a MAC here

ALU

Buffer ALU

RF ALU

Normalized Energy Cost*

200×

PE ALU 2×

1× 1× (Reference)

DRAM ALU

0.5 – 1.0 kB

100 – 500 kB

NoC: 200 – 1000 PEs

* measured from a commercial 65nm process

Design memory hierarchy and dataflow to exploit data reuse at low cost memories

7

Page 8: Domain-specific Architectures r2 - ict.ac.cncrva.ict.ac.cn/documents/agile-and-open-hardware/sze...1.DPM v5 [Girshick, 2012] 2.Fast R-CNN [Girshick, CVPR 2015] Exponential Linear Video

Eyeriss: Deep Neural Network Accelerator

On-ch

ip Bu

ffer Spatial

PE Array

4mm

4mm

[Chen et al., ISSCC 2016, ISCA 2016]

[Joint work with Joel Emer]

Results for AlexNet

[Chen et al., ISSCC 2016, ISCA 2016]

Exploits data reuse for 100x reduction in memory accesses from global buffer and 1400x reduction in memory accesses from off-chip DRAM

8

Page 9: Domain-specific Architectures r2 - ict.ac.cncrva.ict.ac.cn/documents/agile-and-open-hardware/sze...1.DPM v5 [Girshick, 2012] 2.Fast R-CNN [Girshick, CVPR 2015] Exponential Linear Video

Features: Energy vs. Accuracy

0.1

1

10

100

1000

10000

0 20 40 60 80Accuracy (Average Precision)

Energy/Pixel (nJ)

VGG162

AlexNet2

HOG1

Measured in on VOC 2007 Dataset1. DPM v5 [Girshick, 2012]2. Fast R-CNN [Girshick, CVPR 2015]

Exponential

Linear

Video Compression

[Suleiman et al., ISCAS 2017]

Measured in 65nm*

* Only feature extraction. Does not include data, classification

energy, augmentation and ensemble, etc.

On-c

hip B

uffer Spatial

PE Array

4mm

4mm

4mm

4mm

[Suleiman, VLSI 2016] [Chen, ISSCC 2016]1 2

9

Page 10: Domain-specific Architectures r2 - ict.ac.cncrva.ict.ac.cn/documents/agile-and-open-hardware/sze...1.DPM v5 [Girshick, 2012] 2.Fast R-CNN [Girshick, CVPR 2015] Exponential Linear Video

• Popular efficient DNN algorithm approaches

Design of Efficient DNN Algorithms

pruning neurons

pruning synapses

after pruningbefore pruning

Network Pruning

C1

1S

R

1

R

SC

Compact Network Architectures

Examples: SqueezeNet, MobileNet

... also reduced precision

• Focus on reducing number of MACs and weights• Does it translate to energy savings and reduced latency?

10

Page 11: Domain-specific Architectures r2 - ict.ac.cncrva.ict.ac.cn/documents/agile-and-open-hardware/sze...1.DPM v5 [Girshick, 2012] 2.Fast R-CNN [Girshick, CVPR 2015] Exponential Linear Video

• Number of weights alone is not a good metric for energy

• All data types should be considered

Key Observations

Output Feature Map43%

Input Feature Map25%

Weights22%

Computation10%

Energy Consumption of GoogLeNet

[Yang et al., CVPR 2017]

11

Energy estimation tool available at: https://energyestimation.mit.edu/

Page 12: Domain-specific Architectures r2 - ict.ac.cncrva.ict.ac.cn/documents/agile-and-open-hardware/sze...1.DPM v5 [Girshick, 2012] 2.Fast R-CNN [Girshick, CVPR 2015] Exponential Linear Video

Directly target energy and incorporate it into the

optimization of DNNs to provide greater energy savings

Energy-Aware Pruning

• Sort layers based on energy and prune layers that consume most energy first

• EAP reduces AlexNet energy by 3.7x and outperforms the previous work that uses magnitude-based pruning by 1.7x

0 0.5

1 1.5

2 2.5

3 3.5

4 4.5

Ori. DC EAP

Normalized Energy (AlexNet)

2.1x 3.7x

x109

Magnitude

Based Pruning

Energy Aware

Pruning

Pruned models available at http://eyeriss.mit.edu/energy.html

[Yang et al., CVPR 2017]

12

Page 13: Domain-specific Architectures r2 - ict.ac.cncrva.ict.ac.cn/documents/agile-and-open-hardware/sze...1.DPM v5 [Girshick, 2012] 2.Fast R-CNN [Girshick, CVPR 2015] Exponential Linear Video

Many Efficient DNN Design Approaches

pruning neurons

pruning synapses

after pruningbefore pruning

Network Pruning Compact Network Architectures

10100101000000000101000000000100

01100110

Reduce Precision

32-bit float

8-bit fixed

Binary 0

No guarantee that DNN algorithm designer will use a given approach.

Need flexible hardware!

[Chen et al., SysML 2018]

C1

1S

R

1

R

SC

G

Depth-WiseLayer

Point-WiseLayer

ConvolutionalLayer

…ChannelGroups

13

Page 14: Domain-specific Architectures r2 - ict.ac.cncrva.ict.ac.cn/documents/agile-and-open-hardware/sze...1.DPM v5 [Girshick, 2012] 2.Fast R-CNN [Girshick, CVPR 2015] Exponential Linear Video

• When reuse available, need multicast to exploit spatial data reuse for energy efficiency and high array utilization

• When reuse not available, need unicast for high BW for weights for FC and weights & activations for high PE utilization

• An all-to-all satisfies above but too expensive and not scalable

Need Flexible NoC for Varying ReuseG

loba

l Buf

fer

PE PE PEPE

PE PE PEPE

PE PE PEPE

PEPE PE PE

Glo

bal B

uffe

r

PE PE PEPE

PE PE PEPE

PE PE PEPE

PE PE PEPE

Glo

bal B

uffe

r

PE PE PEPE

PE PE PEPE

PE PE PEPE

PE PE PEPE

Glo

bal B

uffe

r

PE PE PEPE

PE PE PEPE

PE PE PEPE

PE PE PEPE

High Bandwidth, Low Spatial Reuse Low Bandwidth, High Spatial Reuse

Unicast Networks Broadcast Network1D Multicast Networks1D Systolic Networks

14

Page 15: Domain-specific Architectures r2 - ict.ac.cncrva.ict.ac.cn/documents/agile-and-open-hardware/sze...1.DPM v5 [Girshick, 2012] 2.Fast R-CNN [Girshick, CVPR 2015] Exponential Linear Video

Eyeriss v2: Balancing Flexibility and Efficiency

[Chen et al., JETCAS 2019]Over an order of magnitude faster and more energy efficient than Eyeriss v1

Speed up over Eyeriss v1 scales with number of PEs

# of PEs 256 1024 16384

AlexNet 17.9x 71.5x 1086.7x

GoogLeNet 10.4x 37.8x 448.8x

MobileNet 15.7x 57.9x 873.0x

Efficiently supports• Wide range of filter shapes – Large and Compact

• Different Layers – CONV, FC, depth wise, etc.

• Wide range of sparsity – Dense and Sparse

• Scalable architecture

[Joint work with Joel Emer]

5.610.912.6

15

Page 16: Domain-specific Architectures r2 - ict.ac.cncrva.ict.ac.cn/documents/agile-and-open-hardware/sze...1.DPM v5 [Girshick, 2012] 2.Fast R-CNN [Girshick, CVPR 2015] Exponential Linear Video

• Exploit properties of workloads– Efficient memory hierarchy and dataflow for data reuse;

exploit natural sparsity in activation

• Design more efficient workloads– Design efficient DNN models with increased sparsity,

reduced precision, and compact network architectures– Drive design of algorithms with direct metrics (i.e., energy,

latency) rather than indirect metrics (i.e., # of ops, weights)

• Define range of workloads – Flexibility to support a wide range of DNNs, including

different efficient DNN approaches

DNN Design Considerations16

Page 17: Domain-specific Architectures r2 - ict.ac.cncrva.ict.ac.cn/documents/agile-and-open-hardware/sze...1.DPM v5 [Girshick, 2012] 2.Fast R-CNN [Girshick, CVPR 2015] Exponential Linear Video

Autonomous Navigation

17

Page 18: Domain-specific Architectures r2 - ict.ac.cncrva.ict.ac.cn/documents/agile-and-open-hardware/sze...1.DPM v5 [Girshick, 2012] 2.Fast R-CNN [Girshick, CVPR 2015] Exponential Linear Video

[Joint work with Sertac Karaman]

Robot Exploration

Select candidate scan locations

Compute Shannon MI and choose best location

Move to location

and scan

Update Occupancy

Map

Where to scan?

Occupancy map Mutual information map

Mutual Information (MI) Updated Map

Decide where to go by computing Shannon Mutual Information

Exploration with a mini race car using motion capture for localization

Occupancy map with planned path

MI surface

18

Page 19: Domain-specific Architectures r2 - ict.ac.cncrva.ict.ac.cn/documents/agile-and-open-hardware/sze...1.DPM v5 [Girshick, 2012] 2.Fast R-CNN [Girshick, CVPR 2015] Exponential Linear Video

Challenge is Data Delivery to All CoresProcess multiple beams in parallel

Core 1

Core 2

Core 3

Core N

Core N

Core 2

Core 1

Core N

Core 2

Core 1

Data delivery from memory is limited

Read Port 1

Read Port 2

19

Page 20: Domain-specific Architectures r2 - ict.ac.cncrva.ict.ac.cn/documents/agile-and-open-hardware/sze...1.DPM v5 [Girshick, 2012] 2.Fast R-CNN [Girshick, CVPR 2015] Exponential Linear Video

Specialized Memory ArchitectureBreak up map into separate memory banks and novel storage pattern to minimize read conflicts when processing different beams in parallel.

[Li et al., RSS 2019]

XY

X

Y

Memory Access Pattern

8 8 8

7 7 7

6 6 6

5 5 5

4 4 4 8

3 3 3 5 6 7

2 2 3 4

1 2 3 4 5 6 7 8

Diagonal Banking Pattern

X

Y

Bank 0

Bank 1

Bank 4

Bank 3

Bank 5

Bank 7

Bank 2

Bank 6

8 8 8

7 7 7

6 6 6

5 5 5

4 4 4 8

3 3 3 5 6 7

2 2 3 4

1 2 3 4 5 6 7 8

Achieves throughput within 94% of theoretical limit (unlimited bandwidth). Compute entire 20mx20m map in under a second!

20

Page 21: Domain-specific Architectures r2 - ict.ac.cncrva.ict.ac.cn/documents/agile-and-open-hardware/sze...1.DPM v5 [Girshick, 2012] 2.Fast R-CNN [Girshick, CVPR 2015] Exponential Linear Video

Robot Localization in Under 25mW

[Joint work with Sertac Karaman][Zhang et al., RSS 2017],

[Suleiman et al., VLSI 2018]

NavionConsumes 684× and 1582× less energy than mobile and desktop

CPUs, respectivelyhttp://navion.mit.edu

Entire system fully integrated on chip. Use compression/sparsity to reduce

total storage to 854kB!

Localization is a key step in autonomous navigation

(also AR and VR)

21

Page 22: Domain-specific Architectures r2 - ict.ac.cncrva.ict.ac.cn/documents/agile-and-open-hardware/sze...1.DPM v5 [Girshick, 2012] 2.Fast R-CNN [Girshick, CVPR 2015] Exponential Linear Video

Configurable for Different Environments

[Suleiman et al., JSSC 2019]

22

Dark scenes (MH_4) Motion blur (V2_3)

Examples of Easy Sequences

Examples of Difficult Sequences

Adapting to the environment results in a 2 - 3x energy reduction

Navion has over 250 configurable parameters to adapt to different

sensors and environments

MH_1 V1_1

EuRoC dataset is a very challenging, and widely used UAV dataset 11 sequences with three categories: easy, medium & difficult

Page 23: Domain-specific Architectures r2 - ict.ac.cncrva.ict.ac.cn/documents/agile-and-open-hardware/sze...1.DPM v5 [Girshick, 2012] 2.Fast R-CNN [Girshick, CVPR 2015] Exponential Linear Video

• Exploit properties of workloads– Optimized memory banking and mapping to meet

memory bandwidth requirements for high throughput parallel processing

• Design more efficient workloads– Compact data representation to reduce data movement,

storage and accelerate computation

• Define range of workloads– Adapt to changing environment for improved efficiency

Autonomous Navigation Design Considerations23

Page 24: Domain-specific Architectures r2 - ict.ac.cncrva.ict.ac.cn/documents/agile-and-open-hardware/sze...1.DPM v5 [Girshick, 2012] 2.Fast R-CNN [Girshick, CVPR 2015] Exponential Linear Video

Video Compression

24

Page 25: Domain-specific Architectures r2 - ict.ac.cncrva.ict.ac.cn/documents/agile-and-open-hardware/sze...1.DPM v5 [Girshick, 2012] 2.Fast R-CNN [Girshick, CVPR 2015] Exponential Linear Video

• Video codec composed of multiple heterogenous modules– entropy coding, transform, motion comp, intra coding, deblocking, etc.

• Specialized hardware for each module – Hardcode values for parameters defined by video coding standard (e.g.,

weights of interpolation filter and coefficients of transform)– Dedicated optimized memories and dataflow for each module

• Parallel and pipeline across and within modules

Video Compression25

Page 26: Domain-specific Architectures r2 - ict.ac.cncrva.ict.ac.cn/documents/agile-and-open-hardware/sze...1.DPM v5 [Girshick, 2012] 2.Fast R-CNN [Girshick, CVPR 2015] Exponential Linear Video

• Advanced algorithms more difficult to parallelize– Limits throughput due to Amdahl’s law

Parallelism Limited By Algorithm

bits

De-Binarizer(DB)

ArithmeticDecoder (AD)

Context Memory

Context Selection

(CS)

syntax elements

Context Modeling (CM)

bins

probability

Context-Adaptive Binary Arithmetic Coding (CABAC)

[Joint work with Anantha Chandrakasan]

26

Page 27: Domain-specific Architectures r2 - ict.ac.cncrva.ict.ac.cn/documents/agile-and-open-hardware/sze...1.DPM v5 [Girshick, 2012] 2.Fast R-CNN [Girshick, CVPR 2015] Exponential Linear Video

• Advanced algorithms more difficult to parallelize

• Re-design algorithms to be more hardware-friendly

Parallelism Limited By Algorithm

Parallel entropy coding algorithm gives >10x higher throughput than state-of-the-art with minimal impact on coding loss

bits

De-Binarizer(DB)

ArithmeticDecoder (AD)

Context Memory

Context Selection

(CS)

syntax elements

Context Modeling (CM)

bins

probability

bypassContext-Adaptive Binary Arithmetic Coding (CABAC)

Slice Engine 0Slice Engine 1Slice Engine 2Slice Engine 3Slice Engine 4Slice Engine 5Slice Engine 6Slice Engine 7

Slice Engine 8Slice Engine 9

Slice Engine 10Slice Engine 11Slice Engine 12Slice Engine 13Slice Engine 14Slice Engine 15

Core

Dom

ain

Inte

rface

Dom

ain

3.8 mm

4.6

mm

[Sze et al., ISSCC 2011]

[Joint work with Anantha Chandrakasan]

27

Page 28: Domain-specific Architectures r2 - ict.ac.cncrva.ict.ac.cn/documents/agile-and-open-hardware/sze...1.DPM v5 [Girshick, 2012] 2.Fast R-CNN [Girshick, CVPR 2015] Exponential Linear Video

High Efficiency Video Coding (HEVC)

Coding

Efficiency

Efficient

Implementation

Larger and Flexible Coding Block Size X

More Sophisticated Intra Prediction X

Larger Interpolation for Motion

Comp.

X

Larger Transform Size X

Parallel Deblocking Filter X

Sample Adaptive Offset X

High-Throughput CABAC X X

High Level Parallel Tools X

Size Energy

H.265/HEVC

(2013)

MPEG-2(1994)

H.264/AVC

(2003)

4x

1.5x

2x

2x

Co-design of algorithm and hardware to address coding efficiency, throughput and power challenges

• H.265/HEVC is the successor to H.264/AVC

• Achieves 2x higher compression than H.264/AVC• High throughput (Ultra-HD 8K @ 120fps) & low power

Primetime Emmy

28

Page 29: Domain-specific Architectures r2 - ict.ac.cncrva.ict.ac.cn/documents/agile-and-open-hardware/sze...1.DPM v5 [Girshick, 2012] 2.Fast R-CNN [Girshick, CVPR 2015] Exponential Linear Video

Flexibility Needed for Video Compression29

VVCLegacy (2003, 2013) Emerging (2019, 2020)

Pre-Processing EncodingSource

DestinationPost-Processing Decoding

Scope of Standard

While decoder is standardize, encoder allows for product differentiation

(e.g., better motion estimation)

Shared resources (e.g., cache for motion compensation), but modules tend to be hardcoded due to tight power and speed requirements

• Encoder must be flexible

• Support multiple standards

Page 30: Domain-specific Architectures r2 - ict.ac.cncrva.ict.ac.cn/documents/agile-and-open-hardware/sze...1.DPM v5 [Girshick, 2012] 2.Fast R-CNN [Girshick, CVPR 2015] Exponential Linear Video

• Exploit properties of workloads– Specialized hardware for heterogenous set of modules with

hardcoded parameters; exploit parallelism and pipelining

• Design more efficient workloads– Parallel entropy coding algorithm to remove compute

bottleneck– Co-design of algorithms and hardware in HEVC standard

• Define range of workloads– Flexibility to support multiple video standards and

algorithm changes in encoder

Video Compression Design Considerations30

Page 31: Domain-specific Architectures r2 - ict.ac.cncrva.ict.ac.cn/documents/agile-and-open-hardware/sze...1.DPM v5 [Girshick, 2012] 2.Fast R-CNN [Girshick, CVPR 2015] Exponential Linear Video

• Domain-specific hardware can address the rising compute demands for many existing and emerging applications

• Opportunities– Exploit properties of workloads (e.g., parallelism, access patterns, representation)

– Design efficient workloads using co-design of algorithms and hardware without affecting quality of result

• Challenges– Define range of workloads to support based on flexibility versus efficiency tradeoff

– Workloads will evolve over time and across use cases/environments

• Agile design can be used for rapid exploration of workloads and tradeoff

• Open hardware can allows for rapid system development with shared building blocks– May need to configure for given application requirements

– What is the granularity of the blocks?

Summary

Today’s slides available at www.rle.mit.edu/eems For Updates

31

Page 32: Domain-specific Architectures r2 - ict.ac.cncrva.ict.ac.cn/documents/agile-and-open-hardware/sze...1.DPM v5 [Girshick, 2012] 2.Fast R-CNN [Girshick, CVPR 2015] Exponential Linear Video

Acknowledgements

Research conducted in the MIT Energy-Efficient Multimedia Systems Group would not be possible without the support of the following organizations:

For updates on our research

Joel Emer

Sertac Karaman

AnanthaChandrakasan

32

Page 33: Domain-specific Architectures r2 - ict.ac.cncrva.ict.ac.cn/documents/agile-and-open-hardware/sze...1.DPM v5 [Girshick, 2012] 2.Fast R-CNN [Girshick, CVPR 2015] Exponential Linear Video

• Efficient Processing for Deep Neural Networks– Project website: http://eyeriss.mit.edu

– Y.-H. Chen, T.-J Yang, J. Emer, V. Sze, “Eyeriss v2: A Flexible Accelerator for Emerging Deep Neural Networks on Mobile Devices,” IEEE Journal on Emerging and Selected Topics in Circuits and Systems (JETCAS), Vol. 9, No. 2, pp. 292-308, June 2019.

– Y.-H. Chen, T. Krishna, J. Emer, V. Sze, “Eyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks,” IEEE Journal of Solid State Circuits (JSSC), ISSCC Special Issue, Vol. 52, No. 1, pp. 127-138, January 2017.

– Y.-H. Chen, J. Emer, V. Sze, “Eyeriss: A Spatial Architecture for Energy-Efficient Dataflow for Convolutional Neural Networks,” International Symposium on Computer Architecture (ISCA), pp. 367-379, June 2016.

– Y.-H. Chen*, T.-J. Yang*, J. Emer, V. Sze, “Understanding the Limitations of Existing Energy-Efficient Design Approaches for Deep Neural Networks,” SysML Conference, February 2018.

– V. Sze, Y.-H. Chen, T.-J. Yang, J. Emer, “Efficient Processing of Deep Neural Networks: A Tutorial and Survey,” Proceedings of the IEEE, vol. 105, no. 12, pp. 2295-2329, December 2017.

– A. Suleiman*, Y.-H. Chen*, J. Emer, V. Sze, “Towards Closing the Energy Gap Between HOG and CNN Features for Embedded Vision,” IEEE International Symposium of Circuits and Systems (ISCAS), Invited Paper, May 2017.

– Hardware Architecture for Deep Neural Networks: http://eyeriss.mit.edu/tutorial.html

References33

Page 34: Domain-specific Architectures r2 - ict.ac.cncrva.ict.ac.cn/documents/agile-and-open-hardware/sze...1.DPM v5 [Girshick, 2012] 2.Fast R-CNN [Girshick, CVPR 2015] Exponential Linear Video

• Co-Design of Algorithms and Hardware for Deep Neural Networks– T.-J. Yang, Y.-H. Chen, V. Sze, “Designing Energy-Efficient Convolutional Neural Networks using Energy-

Aware Pruning,” IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. – Energy estimation tool: http://eyeriss.mit.edu/energy.html– T.-J. Yang, A. Howard, B. Chen, X. Zhang, A. Go, V. Sze, H. Adam, “NetAdapt: Platform-Aware Neural

Network Adaptation for Mobile Applications,” European Conference on Computer Vision (ECCV), 2018.

References34

Page 35: Domain-specific Architectures r2 - ict.ac.cncrva.ict.ac.cn/documents/agile-and-open-hardware/sze...1.DPM v5 [Girshick, 2012] 2.Fast R-CNN [Girshick, CVPR 2015] Exponential Linear Video

• Fast Shannon Mutual Information for Robot Exploration– Z. Zhang, T. Henderson, V. Sze, S. Karaman, “FSMI: Fast computation of Shannon Mutual Information

for information-theoretic mapping,” IEEE International Conference on Robotics and Automation (ICRA), May 2019.

– P. Li*, Z. Zhang*, S. Karaman, V. Sze, “High-throughput Computation of Shannon Mutual Information on Chip,” Robotics: Science and Systems (RSS), June 2019

– Z. Zhang, T. Henderson, S. Karaman, V. Sze, “FSMI: Fast computation of Shannon Mutual Information for information-theoretic mapping,” extended preprint on arXiv, May 2019 http://arxiv.org/abs/1905.02238

• Energy-Efficient Visual Inertial Localization – Project website: http://navion.mit.edu

– A. Suleiman, Z. Zhang, L. Carlone, S. Karaman, V. Sze, “Navion: A Fully Integrated Energy-Efficient Visual-Inertial Odometry Accelerator for Autonomous Navigation of Nano Drones,” IEEE Symposium on VLSI Circuits (VLSI-Circuits), June 2018.

– Z. Zhang*, A. Suleiman*, L. Carlone, V. Sze, S. Karaman, “Visual-Inertial Odometry on Chip: An Algorithm-and-Hardware Co-design Approach,” Robotics: Science and Systems (RSS), July 2017.

– A. Suleiman, Z. Zhang, L. Carlone, S. Karaman, V. Sze, “Navion: A 2mW Fully Integrated Real-Time Visual-Inertial Odometry Accelerator for Autonomous Navigation of Nano Drones,” IEEE Journal of Solid State Circuits (JSSC), VLSI Symposia Special Issue, Vol. 54, No. 4, pp. 1106-1119, April 2019.

References35

Page 36: Domain-specific Architectures r2 - ict.ac.cncrva.ict.ac.cn/documents/agile-and-open-hardware/sze...1.DPM v5 [Girshick, 2012] 2.Fast R-CNN [Girshick, CVPR 2015] Exponential Linear Video

• Video Compression– V. Sze, A. P. Chandrakasan, “A Highly Parallel and Scalable CABAC Decoder for Next-Generation Video

Coding,” IEEE Journal of Solid-State Circuits (JSSC), ISSCC Special Issue, Vol. 47, No. 1, pp. 8-22, January 2012.

– V. Sze, M. Budagavi, “High Throughput CABAC Entropy Coding in HEVC,” IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), Vol. 22, No. 12, pp. 1778-1791, December 2012.

– V. Sze, A. P. Chandrakasan, “Joint Algorithm-Architecture Optimization of CABAC to Increase Speed and Reduce Area Cost,” IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1577–1580, May 2011.

– V. Sze, A. P. Chandrakasan, “A High Throughput CABAC Algorithm Using Syntax Element Partitioning,” IEEE International Conference on Image Processing (ICIP), pp. 773-776, November 2009.

– V. Sze, M. Budagavi, A. P. Chandrakasan, M. Zhou, “Parallel CABAC for Low Power Video Coding,” IEEE International Conference on Image Processing (ICIP), pp. 2096-2099, October 2008.

– V. Sze, D. F. Finchelstein, M. E. Sinangil, A. P. Chandrakasan, “A 0.7-V 1.8-mW H.264/AVC 720p Video Decoder,” IEEE Journal of Solid State Circuits (JSSC), A-SSCC Special Issue , Vol. 44, No. 11, pp. 2943-2956, November 2009.

– V. Sze, M. Budagavi, G. J. Sullivan (Editors), High Efficiency Video Coding (HEVC): Algorithms and Architectures, Springer, 2014.

References36