When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering...

132
When Is Graph Reordering An Optimization? A Cross Application and Input Graph Study on the Effectiveness of Lightweight Graph Reordering Vignesh Balaji Brandon Lucia

Transcript of When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering...

Page 1: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

When Is Graph Reordering An Optimization?

A Cross Application and Input Graph Study on the Effectiveness of Lightweight Graph Reordering

Vignesh Balaji Brandon Lucia

Page 2: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Graph Processing Has Many Applications

2

Path Planning Social network analysis Recommender systems

Page 3: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Graph Processing Has Many Applications

3

Path Planning Social network analysis Recommender systems

Page 4: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Graph Applications Are Memory Bound

4Figure from “Optimizing Cache Performance for Graph Analytics” ArXiv v1;

Cycles stalled on DRAM / Total Cycles LLC Miss Rate

Page 5: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Graph Applications Are Memory Bound

Figure from “Optimizing Cache Performance for Graph Analytics” ArXiv v1; 5

Problem: Poor LLC locality ⇒ Many long-latency DRAM accesses

Cycles stalled on DRAM / Total Cycles LLC Miss Rate

Page 6: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Reason For Poor Locality - Irregular Memory Accesses

6

Page 7: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Reason For Poor Locality - Irregular Memory Accesses

7

Typical graph processing kernel

Page 8: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Reason For Poor Locality - Irregular Memory Accesses

8Input Graph

Typical graph processing kernel

Page 9: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Reason For Poor Locality - Irregular Memory Accesses

Compressed Sparse Row (CSR) Representation 9

Typical graph processing kernel

Input Graph

Page 10: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Reason For Poor Locality - Irregular Memory Accesses

10Input Graph

vtxData

Irregular accesses to vtxData array

Compressed Sparse Row (CSR) Representation

Typical graph processing kernel

Input Graph

Page 11: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Irregular Accesses Have Poor Temporal And Spatial Locality

11

Time

LLC

2 lines

2 words/line

Page 12: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

12

Time

LLC

vtxData[3]

Irregular Accesses Have Poor Temporal And Spatial Locality

Page 13: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

13

Time

LLC

2 3

Miss

Irregular Accesses Have Poor Temporal And Spatial Locality

Line granular transfers.2 words / line

vtxData[3]

Page 14: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

14

Time

LLC

2 3

LLC

2 3

0 1

Miss Miss

Irregular Accesses Have Poor Temporal And Spatial Locality

vtxData[3] vtxData[0]

Page 15: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

15

Time

LLC

2 3

LLC

2 3

0 1

Miss Miss

LLC

6 7

0 1

vtxData[7]

Miss

Irregular Accesses Have Poor Temporal And Spatial Locality

Eviction due to capacity miss

vtxData[3] vtxData[0]

Page 16: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

16

Time

LLC

2 3

LLC

2 3

0 1

Miss Miss

LLC

6 7

0 1

Miss

vtxData[3]

Miss

LLC

6 7

2 3

Irregular Accesses Have Poor Temporal And Spatial Locality

vtxData[7]vtxData[3] vtxData[0]

Page 17: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

17

Time

LLC

2 3

LLC

2 3

0 1

Miss Miss

LLC

6 7

0 1

Miss Miss

LLC

6 7

2 3

Irregular Accesses Have Poor Temporal And Spatial Locality

vtxData[3]vtxData[7]vtxData[3] vtxData[0]

Working set size >> LLC capacity↓

Poor Temporal Locality

Page 18: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

18

Time

LLC

2 3

LLC

2 3

0 1

Miss Miss

LLC

6 7

0 1

Miss Miss

LLC

6 7

2 3

Line Size > Access granularity↓

Poor Spatial Locality

Irregular Accesses Have Poor Temporal And Spatial Locality

vtxData[3]vtxData[7]vtxData[3] vtxData[0]

Working set size >> LLC capacity↓

Poor Temporal Locality

Page 19: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Outline

❖ Poor Locality of Graph Processing Applications ✔

❖ Improving locality through Graph Reordering

❖ Graph Reordering Challenge - Application and Input-dependent Speedups

❖ When is Graph Reordering an Optimization?

❖ Selective Graph Reordering

19

Page 20: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Outline

❖ Poor Locality of Graph Processing Applications ✔

❖ Improving locality through Graph Reordering

❖ Graph Reordering Challenge - Application and Input-dependent Speedups

❖ When is Graph Reordering an Optimization?

❖ Selective Graph Reordering

20

Page 21: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Real-world Graphs Offer Opportunities To Improve Locality

21

Power-law Degree Distribution

Air Traffic Network

Page 22: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

22

Hubs

Power-law Degree Distribution

Real-world Graphs Offer Opportunities To Improve LocalityAir Traffic Network

Page 23: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

23Right figure from “Rabbit Order: Just-in-time Parallel Reordering for Fast Graph Analysis” IPDPS 2016

Community StructurePower-law Degree Distribution

Hubs

Real-world Graphs Offer Opportunities To Improve LocalityAir Traffic Network

Facebook friend Graph

Page 24: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

24Right figure from “Rabbit Order: Just-in-time Parallel Reordering for Fast Graph Analysis” IPDPS 2016

Community StructurePower-law Degree Distribution

CommunitiesHubs

Real-world Graphs Offer Opportunities To Improve LocalityAir Traffic Network

Facebook friend Graph

Page 25: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

25Right figure from “Rabbit Order: Just-in-time Parallel Reordering for Fast Graph Analysis” IPDPS 2016

Community StructurePower-law Degree Distribution

Observation: Subset of vertices are accessed together

CommunitiesHubs

Real-world Graphs Offer Opportunities To Improve LocalityAir Traffic Network

Facebook friend Graph

Page 26: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Reordering To Improve Locality of Graph Applications

26

Key Insight: Store commonly accessed vertices contiguously in memory

Page 27: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Reordering To Improve Locality of Graph Applications

27

Power-law graph

Key Insight: Store commonly accessed vertices contiguously in memory

Page 28: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Reordering To Improve Locality of Graph Applications

28

Reorder

Power-law graph

Key Insight: Store commonly accessed vertices contiguously in memory

Page 29: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

29

Time

LLC

Reordering Improves Spatial & Temporal Locality

2 lines

2 words/line

Reordered CSR

Page 30: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

30

Time

LLC

vtxData[0]

0 1

Miss

Reordering Improves Spatial & Temporal Locality

Page 31: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

31

Time

LLC

0 1

LLC

vtxData[2]

0 1

2 3

Miss Miss

vtxData[0]

Reordering Improves Spatial & Temporal Locality

Page 32: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

32

Time

LLC

0 1

LLC

0 1

2 3

Miss Miss

LLC

0 1

2 3

vtxData[1]

Hit

vtxData[2]vtxData[0]

Reordering Improves Spatial & Temporal Locality

Page 33: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

33

Time

Miss Miss

LLC

0 1

2 3

vtxData[0]

LLC

0 1

LLC

0 1

2 3

LLC

0 1

2 3

Hit Hit

vtxData[1]vtxData[2]vtxData[0]

Reordering Improves Spatial & Temporal Locality

Page 34: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

34

Time

Miss Miss

LLC

0 1

2 3

LLC

0 1

LLC

0 1

2 3

LLC

0 1

2 3

Hit Hit

vtxData[0]

LLC

0 1

2 3

Hit

vtxData[0]vtxData[1]vtxData[2]vtxData[0]

Reordering Improves Spatial & Temporal Locality

Page 35: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

35

Time

Miss Miss

LLC

0 1

2 3

LLC

0 1

LLC

0 1

2 3

LLC

0 1

2 3

Hit Hit

LLC

0 1

2 3

Hit

Graph Reordering improved Spatial and Temporal

locality of vtxData accesses

vtxData[0]vtxData[0]vtxData[1]vtxData[2]vtxData[0]

Reordering Improves Spatial & Temporal Locality

Page 36: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Outline

❖ Poor Locality of Graph Processing Applications ✔

❖ Improving locality through Graph Reordering ✔

❖ Graph Reordering Challenge - Application and Input-dependent Speedups

❖ When is Graph Reordering an Optimization?

❖ Selective Graph Reordering

36

Page 37: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Outline

❖ Poor Locality of Graph Processing Applications ✔

❖ Improving locality through Graph Reordering ✔

❖ Graph Reordering Challenge - Application and Input-dependent Speedups

❖ When is Graph Reordering an Optimization?

❖ Selective Graph Reordering

37

Page 38: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Graph Reordering Is Not A Panacea

38

Page 39: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Graph Reordering Is Not A Panacea

39

Locality Benefits

Reordering Overhead

Net Speedup

Page 40: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

40

Graph Reordering Is Not A Panacea

Page 41: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

41

Net speedup from Reordering depends on the Application and Input Graph

Graph Reordering Is Not A Panacea

Page 42: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Question: What are the properties of Applications and Input Graphs that benefit from Reordering?

42

Page 43: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Outline

❖ Poor Locality of Graph Processing Applications ✔

❖ Improving locality through Graph Reordering ✔

❖ Graph Reordering Challenge - Application and Input-dependent Speedups ✔

❖ When is Graph Reordering an Optimization?

❖ Selective Graph Reordering

43

Page 44: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Outline

❖ Poor Locality of Graph Processing Applications ✔

❖ Improving locality through Graph Reordering ✔

❖ Graph Reordering Challenge - Application and Input-dependent Speedups ✔

❖ When is Graph Reordering an Optimization?➢ Characterization Space➢ Which Applications benefit from Reordering?➢ Which Input Graphs benefit from Reordering?

❖ Selective Graph Reordering

44

Page 45: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Outline

❖ Poor Locality of Graph Processing Applications ✔

❖ Improving locality through Graph Reordering ✔

❖ Graph Reordering Challenge - Application and Input-dependent Speedups ✔

❖ When is Graph Reordering an Optimization?➢ Characterization Space➢ Which Applications benefit from Reordering?➢ Which Input Graphs benefit from Reordering?

❖ Selective Graph Reordering

45

Page 46: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Characterization Space

46

3 Graph

Reordering Techniques

15 Applications

(Ligra, GAP)

8 Input Graphs

(M vertices, B edges)

Server-class Processor

(dual-Socket, 28 cores, 35MB LLC, 64GB DRAM)

Page 47: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Lightweight Reordering (LWR) Techniques

➢ Rabbit Ordering [Arai et. al., IPDPS 2016]

➢ Frequency-based Clustering (or “Hub-Sorting”) [Zhang et. al., Big Data

2017]

➢ Hub-Clustering (Our Variation of Hub Sorting)

47

Selection Criteria: Low reordering overhead (Require very few runs/iterations to amortize overheads)

Page 48: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

LWR 1 - Rabbit Ordering

48Figure from “Rabbit Order: Just-in-Time Parallel Reordering for Fast Graph Analysis” IPDPS 2016

Page 49: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

LWR 1 - Rabbit Ordering

49Figure from “Rabbit Order: Just-in-Time Parallel Reordering for Fast Graph Analysis” IPDPS 2016

Page 50: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

50Figure from “Rabbit Order: Just-in-Time Parallel Reordering for Fast Graph Analysis” IPDPS 2016

● Fast community detection using incremental aggregation

● Complexity - O(|E| - ck|V|) where c = clustering coeff.

k = avg. degree

LWR 1 - Rabbit Ordering

Page 51: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

LWR 2 & 3 - HubSorting & HubClustering

51

degrees

Vtx IDs

Page 52: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

LWR 2 & 3 - HubSorting & HubClustering

52

degrees

Vtx IDs

HubSo

rting

Page 53: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

LWR 2 & 3 - HubSorting & HubClustering

53

degrees

Vtx IDs

HubSo

rting HubClustering

Page 54: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

LWR 2 & 3 - HubSorting & HubClustering

54

degrees

Vtx IDs

HubSorting HubClustering

Locality Benefits Temporal AND Spatial ↑↑ Temporal ↑

Complexity O(|V|.logV) ↓↓ O(|V|) ↓

HubSo

rting HubClustering

Page 55: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Outline

❖ Poor Locality of Graph Processing Applications ✔

❖ Improving locality through Graph Reordering ✔

❖ Graph Reordering Challenge - Application and Input-dependent Speedups ✔

❖ When is Graph Reordering an Optimization?➢ Characterization Space ✔➢ Which Applications benefit from Reordering?➢ Which Input Graphs benefit from Reordering?

❖ Selective Graph Reordering

55

Page 56: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Outline

❖ Poor Locality of Graph Processing Applications ✔

❖ Improving locality through Graph Reordering ✔

❖ Graph Reordering Challenge - Application and Input-dependent Speedups ✔

❖ When is Graph Reordering an Optimization?➢ Characterization Space ✔➢ Which Applications benefit from Reordering?➢ Which Input Graphs benefit from Reordering?

❖ Selective Graph Reordering

56

Page 57: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Legend for Results

57

Application

Input GraphSpeedup with respect to original ordering of graph

Reordering Technique

Page 58: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Legend for Results

58

Speedup excluding the overhead of reordering

(Max Speedup)

Page 59: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Legend for Results

59

Speedup including the overhead of reordering

(Net Speedup)

Page 60: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Legend for Results

60

Reordering overhead

Speedup including the overhead of reordering

(Net Speedup)

Page 61: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

15 Applications →5 Categories

• Category 1: Applications processing Large Frontiers are good candidates

• Category 2: Symmetric bipartite graphs require bi-partiteness aware reordering

• Category 3: Applications processing small frontiers offer limited opportunity

• Category 4: Reordering for Push-style applications introduces false-sharing

• Category 5: Reordering affects convergence for applications with ID-dependent

computations

61

Page 62: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

15 Applications →5 Categories

• Category 1: Applications processing Large Frontiers are good candidates

• Category 2: Symmetric bipartite graphs require bi-partiteness aware reordering

• Category 3: Applications processing small frontiers offer limited opportunity

• Category 4: Reordering for Push-style applications introduces false-sharing

• Category 5: Reordering affects convergence for applications with ID-dependent

computations

62

Page 63: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

63

Category I - Applications Processing a Large Fraction Of Edges

❖ PageRank (Ligra & Gap)

❖ Graph Radii Estimation (Ligra)

Page 64: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

64

Category I - Applications Processing a Large Fraction Of Edges

Page 65: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

65

Observation 1: LWR provides end-to-end speedups in some cases

Category I - Applications Processing a Large Fraction Of Edges

Page 66: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

66

Observation 2: Maximum speedups from HubSort > HubCluster

Category I - Applications Processing a Large Fraction Of Edges

Observation 1: LWR provides end-to-end speedups in some cases

Page 67: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

67

Observation 3: Reordering Overhead is HubSort > HubCluster

Category I - Applications Processing a Large Fraction Of Edges

Observation 2: Maximum speedups from HubSort > HubCluster

Observation 1: LWR provides end-to-end speedups in some cases

Page 68: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

68

Observation 4: HubSort strikes a balance between effectiveness and overhead

Category I - Applications Processing a Large Fraction Of Edges

Observation 3: Reordering Overhead is HubSort > HubCluster

Observation 2: Maximum speedups from HubSort > HubCluster

Observation 1: LWR provides end-to-end speedups in some cases

Page 69: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Category II - Executions On Symmetric Bipartite Graphs

69

❖ Collaborative Filtering (Ligra)

Page 70: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Category II - Executions On Symmetric Bipartite Graphs

70

Page 71: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Category II - Executions On Symmetric Bipartite Graphs

71

Surprising trend: HubSort causes net slowdowns

Page 72: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Category II - Reason For Slowdown With HubSort

72

Part1 Part 2

Page 73: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

73

[0, V)

Category II - Reason For Slowdown With HubSort

Part1 IDs Part2 IDs

Original Ordering

Heatmap showing the part that each vertex belongs to

Part1 Part 2

Page 74: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

74

[0, V)

Category II - Reason For Slowdown With HubSort

Original OrderingPart1 Part 2

Part1 IDs Part2 IDs

Heatmap showing the part that each vertex belongs to

Page 75: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

75

[0, V)

Category II - Reason For Slowdown With HubSort

Range of Irregular accesses to vtxData = #nodes in a part

Original OrderingPart1 Part 2

Part1 IDs Part2 IDs

Page 76: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

76

Category II - Reason For Slowdown With HubSort

[0, V)

Vertices from different parts assigned consecutive IDs

↓Increased range of irregular

accesses to vtxData

Original OrderingPart1 Part 2

Part1 IDs Part2 IDs

Page 77: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

77

Category II - Reason For Slowdown With HubSort

[0, V)

Opportunity: Bipartiteness-aware Graph Reordering Techniques

Original OrderingPart1 Part 2

Part1 IDs Part2 IDs

Page 78: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Category III - Applications Processing a Small Fraction of Edges

78

❖ Betweenness Centrality

❖ BFS

❖ K-Core Decomposition

Page 79: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

79

Low speedup even without Reordering overheads

Category III - Applications Processing a Small Fraction of Edges

Page 80: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

80

Category III - Applications Processing a Small Fraction of Edges

Page 81: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

81

Category III - Applications Processing a Small Fraction of Edges

Page 82: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

82

Category III - Applications Processing a Small Fraction of Edges

Page 83: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

83

Limited reuse in vtxData accesses↓

Lower headroom for reordering

Category III - Applications Processing a Small Fraction of Edges

Page 84: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Outline

❖ Poor Locality of Graph Processing Applications ✔

❖ Improving locality through Graph Reordering ✔

❖ Graph Reordering Challenge - Application and Input-dependent Speedups ✔

❖ When is Graph Reordering an Optimization?➢ Characterization Space ✔➢ Which Applications benefit from Reordering? ✔➢ Which Input Graphs benefit from Reordering?

❖ Selective Graph Reordering

84

Page 85: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Outline

❖ Poor Locality of Graph Processing Applications ✔

❖ Improving locality through Graph Reordering ✔

❖ Graph Reordering Challenge - Application and Input-dependent Speedups ✔

❖ When is Graph Reordering an Optimization?➢ Characterization Space ✔➢ Which Applications benefit from Reordering? ✔➢ Which Input Graphs benefit from Reordering?

❖ Selective Graph Reordering

85

Page 86: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

86

Speedup From HubSorting Varies Across Inputs

HubSort

Category I Application

Page 87: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

87

HubSort

✅ ✅ ✅

❌ ❌ ❌

Need to predict speedup from HubSorting AND

selectively perform HubSorting

Speedup From HubSorting Varies Across Inputs

Category I Application

Page 88: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Understanding Performance Improvement From HubSorting

88

Cache line

vtxData

Page 89: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Understanding Performance Improvement From HubSorting

89

Hub Vertices

Layout of hubs in original ordering

Cache line

Page 90: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Understanding Performance Improvement From HubSorting

90

HubSort

Layout of hubs in original ordering

Layout of hubs after reordering vertices

Hub Vertices

Cache line Cache line

Page 91: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Understanding Performance Improvement From HubSorting

91

HubSort

Layout of hubs in original ordering

Layout of hubs after reordering vertices

Hub Vertices

Cache line Cache line

HubSorting will be most effective for Graphs with:

❖ Property #1: Skew in the degree-distribution (Presence of Hubs)❖ Property #2: Sparsely distributed hub vertices (Quality of original ordering)

Page 92: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Packing Factor - A Measure of Hub Density Packing Factor is a measure of how densely the hubs are packed after HubSorting

92

Cache line

Hub Vertices

Layout of the original ordering of vertices

Layout after reordering vertices by HubSorting

HubSort

Cache line

Page 93: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Packing Factor Can Predict Speedup From HubSorting

93

Pearson Correlation = 0.92

Page 94: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

94

Pearson Correlation = 0.92

Speedup from HubSorting for a given pair of (Application, Input Graph)

Packing Factor Can Predict Speedup From HubSorting

Page 95: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

95

Pearson Correlation = 0.92

Packing Factor is a good predictor for Speedup

Packing Factor Can Predict Speedup From HubSorting

Page 96: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Outline

❖ Poor Locality of Graph Processing Applications ✔

❖ Improving locality through Graph Reordering ✔

❖ Graph Reordering Challenge - Application and Input-dependent Speedups ✔

❖ When is Graph Reordering an Optimization?➢ Characterization Space ✔➢ Which Applications benefit from Reordering? ✔➢ Which Input Graphs benefit from Reordering? ✔

❖ Selective Graph Reordering

96

Page 97: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Outline

❖ Poor Locality of Graph Processing Applications ✔

❖ Improving locality through Graph Reordering ✔

❖ Graph Reordering Challenge - Application and Input-dependent Speedups ✔

❖ When is Graph Reordering an Optimization?➢ Characterization Space ✔➢ Which Applications benefit from Reordering? ✔➢ Which Input Graphs benefit from Reordering? ✔

❖ Selective Graph Reordering

97

Page 98: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Selective Graph Reordering

98

G`= HubSort(G)

Process(G`)

Page 99: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Selective Graph Reordering

99

Increasing order of Packing Factor

G`= HubSort(G)

Process(G`)

Page 100: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Selective Graph Reordering

100

G`= HubSort(G)

Process(G`)

PF = ComputePF(G)if (PF > 4): G` = HubSort(G) Process(G`)else: Process(G)

Page 101: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Selective Graph Reordering

101

G`= HubSort(G)

Process(G`)

PF = ComputePF(G)if (PF > 4): G` = HubSort(G) Process(G`)else: Process(G)

Page 102: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Selective Graph Reordering

102

Selective Reordering avoids slowdowns

Page 103: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Selective Graph Reordering

103

Computing Packing Factor does not degrade performance

Page 104: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Selective Graph Reordering

104

Selective Reordering is a viable Optimization

Page 105: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Conclusions

❖ Graph Reordering does not benefit all Application and Input Graphs

❖ Opportunity to design new Reordering techniques for specific applications

❖ Packing Factor enables Selective Graph Reordering

105

Page 106: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Source Code Available

106

● Includes code for:○ Packing Factor○ Lightweight Reordering Techniques○ Selective HubSorting

● Open sourced at -

○ https://github.com/CMUAbstract/Graph-Reordering-IISWC18

Page 107: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Thank You!

107

Page 108: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

When Is Graph Reordering An Optimization?

A Cross Application and Input Graph Study on the Effectiveness of Lightweight Graph Reordering

Vignesh Balaji Brandon Lucia

Page 109: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Backup Slides

109

Page 110: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Use-cases Where Reordering Overhead Cannot be Amortized

Time

Left fig. from “Chronos: A Graph Engine for Temporal Graph Analysis” EuroSys 2014; Right fig. from “Graph Evolution: Densification and Shrinking Diameters” TKDD 2007

Temporal Graph Mining

Sophisticated Reordering Techniques are impractical for cases where graph is processed only a few times

110

Page 111: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Sophisticated Reordering Techniques Impose High Overhead

Assumption: Reordered graph will be processed multiple times

111

Page 112: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Irregular Accesses Have Poor Temporal And Spatial Locality

112

Time

LLC

vData[3]

2 3

LLC

vData[0]

2 3

0 1

Miss Miss

LLC

6 7

0 1

vData[7]

Miss

vData[3]

Miss

LLC

6 7

2 3

vData[1]

LLC

0 1

2 3

Miss

Page 113: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

113

Time

LLC

vData[3]

2 3

LLC

vData[0]

2 3

0 1

Miss Miss

LLC

6 7

0 1

vData[7]

Miss

vData[3]

Miss

LLC

6 7

2 3

vData[1]

LLC

0 1

2 3

Miss

vData[3]

Hit

LLC

0 1

2 3

Irregular Accesses Have Poor Temporal And Spatial Locality

Page 114: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

114

Time

LLC

vData[3]

2 3

LLC

vData[0]

2 3

0 1

Miss Miss

LLC

6 7

0 1

vData[7]

Miss

vData[3]

Miss

LLC

6 7

2 3

vData[1]

LLC

0 1

2 3

Miss

vData[3]

Hit

LLC

0 1

2 3

vData[7]

Miss

LLC

6 7

2 3

Irregular Accesses Have Poor Temporal And Spatial Locality

Page 115: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Graph Applications

115

Ligra➢ Page Rank➢ Page Rank-Delta➢ SSSP – Bellman Ford➢ Collaborative Filtering➢ Radii➢ Betweenness Centrality➢ BFS➢ Kcore➢ Maximal Independent Set➢ Connected Components

GAP➢ Page Rank➢ SSSP – Delta Stetting➢ Betweenness Centrality➢ BFS➢ Connected Components

11 Distinct Algorithms

Page 116: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

HW Platform

❖ Dual-Socket Intel Xeon E5-2660v4 processors

❖ 14 cores per Socket (2HT/core)

❖ 35 MB Last Level Cache per processor

❖ 64 GB of main memory

116

32GB DRAM

35MB Last Level Cache

35MB Last Level Cache

Core1

Core2

Core14

Core1

Core2

Core14

32GB DRAM

…. ….

Socket 1 Socket 2

Page 117: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Input Graphs

117

Page 118: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Input Graphs

118

Irregular working set size >> Aggregate LLC Capacity

Page 119: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Input Graphs

119

Irregular working set size >> Aggregate LLC Capacity

We use the original ordering of Input Graphs

Page 120: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Lightweight Reordering Can Provide End-to-end Speedups

Figure from “Rabbit Order: Just-in-Time Parallel Reordering for Fast Graph Analysis” IPDPS 2016120

Page 121: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Lightweight Reordering Can Provide End-to-end Speedups

Figure from “Rabbit Order: Just-in-Time Parallel Reordering for Fast Graph Analysis” IPDPS 2016

Reordering techniques exploiting power-law distributions and community structure

can have low-overheads

121

Page 122: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Category II - Executions On Symmetric Bipartite Graphs

122

Surprising trends:● HubSort offers the least performance benefits● HubSort causes slowdowns

Page 123: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

123

Assigning vertices from each part of the graph a contiguous range is good for temporal locality

Need a simple mechanism to assign hub vertices from the same part a contiguous range of IDs

Part1 Part 2

Category II - Reason For Slowdown With HubSort

Page 124: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Category IV - Push-based Graph Applications

124

Push-phase Pull-phase

+ Work Efficient execuztion- Overhead of synchronization

+ No synchronization required- Work-inefficient (iterate over all Vertices)

Page 125: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

125

Category IV - Push-based Graph Applications

Reordering causes an increase in false-sharing

LWR favors pull-style graph applications

Page 126: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Category V - LWR can affect convergence

126

Vertex IDs influence amount of work done each iteration

Page 127: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Category V - LWR can affect convergence

127

Increase in Iterations until convergence due to LWR

Opportunity to accelerate convergence by reordering vertices

Page 128: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

The Need For Selective Lightweight Reordering

128

Unconditionally performing LWR causes net slowdowns on some input graphs

Completely avoid LWR misses speedups up to 1.8x

Need to predict speedup from LWR for an input graph and only selectively perform LWR

Page 129: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Using Packing Factor for Selective Reordering

129Selective Reordering avoids slowdowns for graphs with low Packing Factor

Page 130: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Using Packing Factor for Selective Reordering

130

The low overhead of Packing Factor computation does not sacrifice speedup for high Packing Factor graphs

Page 131: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Speedups From HubSorting Are Due To Locality Improvements

131

Page 132: When Is Graph Reordering An Optimization?vigneshb/papers/Reordering-IISWC18.pdf · Graph Reordering Techniques 15 Applications (Ligra, GAP) 8 Input Graphs (M vertices, B edges) Server-class

Computing Packing Factor

132