Scalable and Dynamically Updatable Lookup Engine for ... · Scalable and Dynamically Updatable...

26
Scalable and Dynamically Updatable Lookup Engine for Decision-trees on FPGA Yun R. Qu, Viktor K. Prasanna Ming Hsieh Dept. of Electrical Engineering University of Southern California Los Angeles, CA 90089 Email: [email protected] [email protected]

Transcript of Scalable and Dynamically Updatable Lookup Engine for ... · Scalable and Dynamically Updatable...

Page 1: Scalable and Dynamically Updatable Lookup Engine for ... · Scalable and Dynamically Updatable Lookup Engine for Decision-trees on FPGA. Yun R. Qu, Viktor K. Prasanna ... – High-performance

Scalable and Dynamically Updatable Lookup Engine for

Decision-trees on FPGA

Yun R. Qu, Viktor K. PrasannaMing Hsieh Dept. of Electrical Engineering

University of Southern CaliforniaLos Angeles, CA 90089

Email: [email protected] [email protected]

Page 2: Scalable and Dynamically Updatable Lookup Engine for ... · Scalable and Dynamically Updatable Lookup Engine for Decision-trees on FPGA. Yun R. Qu, Viktor K. Prasanna ... – High-performance

Outline• Introduction• Background• Data Structures & Algorithms• Architecture• Evaluation• Conclusion

2

Page 3: Scalable and Dynamically Updatable Lookup Engine for ... · Scalable and Dynamically Updatable Lookup Engine for Decision-trees on FPGA. Yun R. Qu, Viktor K. Prasanna ... – High-performance

Outline• Introduction

– High-performance computing

• Background• Data Structures & Algorithms• Architecture• Evaluation• Conclusion

3

Page 4: Scalable and Dynamically Updatable Lookup Engine for ... · Scalable and Dynamically Updatable Lookup Engine for Decision-trees on FPGA. Yun R. Qu, Viktor K. Prasanna ... – High-performance

High-performance Computing (HPC)• HPC

– Superior performance– Image/signal processing– Scientific computing, simulation– Networking

• Performance– High throughput– Fast updates

[1] http://www.bgp.com.cn/Service/Processing.html

[1]

4

Page 5: Scalable and Dynamically Updatable Lookup Engine for ... · Scalable and Dynamically Updatable Lookup Engine for Decision-trees on FPGA. Yun R. Qu, Viktor K. Prasanna ... – High-performance

Emerging Trends in HPC• Software centralization

– Adapt to the change in demand– Dynamically updatable

• Heterogeneous computing– Accelerated by hardware– FPGA, GPU, VLSI chips, etc.

5

Page 6: Scalable and Dynamically Updatable Lookup Engine for ... · Scalable and Dynamically Updatable Lookup Engine for Decision-trees on FPGA. Yun R. Qu, Viktor K. Prasanna ... – High-performance

Outline• Introduction• Background

– Decision-trees– Challenges

• Data Structures & Algorithms• Architecture• Evaluation• Conclusion

6

Page 7: Scalable and Dynamically Updatable Lookup Engine for ... · Scalable and Dynamically Updatable Lookup Engine for Decision-trees on FPGA. Yun R. Qu, Viktor K. Prasanna ... – High-performance

Decision-trees in HPC• Usage

– Data mining, classification

• Problem definition– Given a decision-tree 𝑁𝑁 leaf nodes, 𝑀𝑀 fields

– High-throughput classification– Support dynamic updates

Decision-tree

Our focus

7

Page 8: Scalable and Dynamically Updatable Lookup Engine for ... · Scalable and Dynamically Updatable Lookup Engine for Decision-trees on FPGA. Yun R. Qu, Viktor K. Prasanna ... – High-performance

State-of-the-art Implementation• Straightforward mapping onto hardware [2][3]

– Each level → Processing Element (PE)– No. of levels ↑ → Memory consumption per stage ↑

PE

PE

PEDeci

sion

-tre

e

pipe

line

[2] D. Sheldon and F. Vahid, “Don’t Forget Memories: A Case StudyRedesigning a Pattern Counting ASIC Circuit for FPGAs,” CODES+ISSS, 2008.[3] Y.-H. E. Yang and V. K. Prasanna, “High Throughput and Large CapacityPipelined Dynamic Search Tree on FPGA,” FPGA, 2010.

8

Page 9: Scalable and Dynamically Updatable Lookup Engine for ... · Scalable and Dynamically Updatable Lookup Engine for Decision-trees on FPGA. Yun R. Qu, Viktor K. Prasanna ... – High-performance

Challenges (1)• Scalability

– Balanced: 𝑂𝑂(log𝑁𝑁) lookup time, 𝑂𝑂(𝑁𝑁) wire length– Imbalanced: 𝑂𝑂(1) wire length, 𝑂𝑂(𝑁𝑁) lookup time– 𝑁𝑁 ↑ → performance (clock rate, throughput) ↓– Challenge 1: a scalable implementation

5

2 8

1 3 6 9

Wire length: 𝑂𝑂(𝑁𝑁)

Bina

ry

tree

9

Page 10: Scalable and Dynamically Updatable Lookup Engine for ... · Scalable and Dynamically Updatable Lookup Engine for Decision-trees on FPGA. Yun R. Qu, Viktor K. Prasanna ... – High-performance

Challenges (2)• Dynamic updates

– Deletion/insertion: nodes being pushed around– Modification: as complex as deletion/insertion– Global changes: a single update → too many operations– Challenge 2: a dynamically updatable implementation

5

2 8

1 3 6 9

6

2 8

1 3 9

update

10

Page 11: Scalable and Dynamically Updatable Lookup Engine for ... · Scalable and Dynamically Updatable Lookup Engine for Decision-trees on FPGA. Yun R. Qu, Viktor K. Prasanna ... – High-performance

Field Programmable Gate Array• Logic Cell Matrix

– Programmable– Logic elements Lookup Table (LUT)

• Programmable interconnect– Connect logic cells → complex functions

Q

QSET

CLR

D

Q

QSET

CLR

D

Logic Cell

Interconnect

0

1

.

.

0

1

Long wire

Short wire

k

11

Page 12: Scalable and Dynamically Updatable Lookup Engine for ... · Scalable and Dynamically Updatable Lookup Engine for Decision-trees on FPGA. Yun R. Qu, Viktor K. Prasanna ... – High-performance

Outline• Introduction• Background• Data Structures & Algorithms

– Rule table– Optimizations– Dynamic updates

• Architecture• Evaluation• Conclusion

12

Page 13: Scalable and Dynamically Updatable Lookup Engine for ... · Scalable and Dynamically Updatable Lookup Engine for Decision-trees on FPGA. Yun R. Qu, Viktor K. Prasanna ... – High-performance

Converting Decision-trees• Rule table

– Alternative representation of a decision-tree– Each leaf node → a root-to-leaf path → a rule– Intuition: parallel searching on the rule table

𝑁𝑁 = 5 rules, 𝑀𝑀 = 2 fields, 𝑊𝑊 = 28 data width

13

Page 14: Scalable and Dynamically Updatable Lookup Engine for ... · Scalable and Dynamically Updatable Lookup Engine for Decision-trees on FPGA. Yun R. Qu, Viktor K. Prasanna ... – High-performance

Optimization Techniques• Splitting (vertically) and partitioning (horizontally)

– Multiple 𝑊𝑊𝑚𝑚-bit sub-fields– Multiple sub-tables having 𝑁𝑁𝑛𝑛 rules

Split

Partition

Mapped to a PE

14

Page 15: Scalable and Dynamically Updatable Lookup Engine for ... · Scalable and Dynamically Updatable Lookup Engine for Decision-trees on FPGA. Yun R. Qu, Viktor K. Prasanna ... – High-performance

Dynamic Updates (1)• Modification of a tree node

– Identify the sub-rules to be modified– Change the values stored, or – Change the polarity of the comparison

modify this node

modify these two rules

15

Page 16: Scalable and Dynamically Updatable Lookup Engine for ... · Scalable and Dynamically Updatable Lookup Engine for Decision-trees on FPGA. Yun R. Qu, Viktor K. Prasanna ... – High-performance

Dynamic Updates (2)• Deletion of a tree node

– Modify sub-rule(s) to produce “invalid” sub-rule(s)

delete this node

modify these two rules

16

Page 17: Scalable and Dynamically Updatable Lookup Engine for ... · Scalable and Dynamically Updatable Lookup Engine for Decision-trees on FPGA. Yun R. Qu, Viktor K. Prasanna ... – High-performance

Dynamic Updates (3)• Inserting an internal node

– All the descendants of the new node has to be modified

• Inserting a leaf node– Pre-allocate (𝑁𝑁max − 𝑁𝑁) invalid rules as placeholders– Modifying an invalid rule into the new rule

17

Page 18: Scalable and Dynamically Updatable Lookup Engine for ... · Scalable and Dynamically Updatable Lookup Engine for Decision-trees on FPGA. Yun R. Qu, Viktor K. Prasanna ... – High-performance

Outline• Introduction• Background• Data Structures & Algorithms• Architecture

– 2-dimensional pipelined architecture

• Evaluation• Conclusion

18

Page 19: Scalable and Dynamically Updatable Lookup Engine for ... · Scalable and Dynamically Updatable Lookup Engine for Decision-trees on FPGA. Yun R. Qu, Viktor K. Prasanna ... – High-performance

Architecture• 2-dimensional pipelined architecture

– Each sub-table → a modular PE– Updates: memory write accesses propagate diagonally

Modular PE

19

Page 20: Scalable and Dynamically Updatable Lookup Engine for ... · Scalable and Dynamically Updatable Lookup Engine for Decision-trees on FPGA. Yun R. Qu, Viktor K. Prasanna ... – High-performance

Outline• Introduction• Background• Data Structures & Algorithms• Architecture• Evaluation

– Peak throughput– Resource consumption

• Conclusion

20

Page 21: Scalable and Dynamically Updatable Lookup Engine for ... · Scalable and Dynamically Updatable Lookup Engine for Decision-trees on FPGA. Yun R. Qu, Viktor K. Prasanna ... – High-performance

Experimental Setups• Prototypes on state-of-the-art FPGA

– Dual-port distributed RAM (distRAM)– Empirical optimizations: 𝑊𝑊𝑚𝑚 = 4, 𝑁𝑁𝑛𝑛 = 2

FPGA device Virtex-7 XC7VX1140t FLG1930 -2LNo. of I/O pins 1100

No. of logic slices 218800BRAM / distRAM 67 Mb / up to 17 Mb

Design Suite Xilinx Vivado 2013.4Programming Language Verilog

Timing constraint 250 MHzReports Post-place-and-route

21

Page 22: Scalable and Dynamically Updatable Lookup Engine for ... · Scalable and Dynamically Updatable Lookup Engine for Decision-trees on FPGA. Yun R. Qu, Viktor K. Prasanna ... – High-performance

Peak Throughput• Comparison with state-of-the-art implementation

– 1.8~2.0 × peak throughput (without any updates)– Sustain high lookup throughput as 𝑁𝑁max ↑ and/or 𝑊𝑊 ↑

22

Page 23: Scalable and Dynamically Updatable Lookup Engine for ... · Scalable and Dynamically Updatable Lookup Engine for Decision-trees on FPGA. Yun R. Qu, Viktor K. Prasanna ... – High-performance

Resource Consumption• No. of logic slices, no. of I/O pins

– Largest design: 𝑁𝑁 = 512 leaf nodes, 𝑊𝑊 = 320 bits data

Resources consumed

evenly

23

Page 24: Scalable and Dynamically Updatable Lookup Engine for ... · Scalable and Dynamically Updatable Lookup Engine for Decision-trees on FPGA. Yun R. Qu, Viktor K. Prasanna ... – High-performance

Outline• Introduction• Background• Data Structures & Algorithms• Architecture• Evaluation• Conclusion

24

Page 25: Scalable and Dynamically Updatable Lookup Engine for ... · Scalable and Dynamically Updatable Lookup Engine for Decision-trees on FPGA. Yun R. Qu, Viktor K. Prasanna ... – High-performance

Conclusion• Scalable and dynamically updatable lookup engine

– Mapping decision-trees onto hardware more efficiently– Works for generic decision-trees in HPC

• Future work– Self-learning on this architecture– Investigate performance tradeoffs on various platforms

25

Page 26: Scalable and Dynamically Updatable Lookup Engine for ... · Scalable and Dynamically Updatable Lookup Engine for Decision-trees on FPGA. Yun R. Qu, Viktor K. Prasanna ... – High-performance

Questions?

Thanks

http://ganges.usc.edu