Parallel DBMS: Competitive in the Graph Analytics Arena?

Wellington Cabrera∗

University of Houstonwcabrera@cs.uh.edu

Carlos Ordonez†

University of Houstonordonez@cs.uh.edu

ABSTRACTGraph Analytics is a task demanding on computing resources,and the common wisdom is that DBMS are slow and cum-bersome, which makes a requirement to use specialized graphsystems to solve graph problems in large graphs. In thiswork, we show how to express important graph algorithms(including PageRank, Single Shortest Paths, Connected Com-ponents, and Topological Sort) with regular queries, to makethem work in parallel database systems. Moreover, we showthat this four algorithms follow a common pattern. Wepresent an experimental evaluation with two DBMS archi-tectures: The well known columnar DBMSs and the emerg-ing array DBMSs. Using challenging social networks andhyper-links data sets from the real-word, we show that ourapproach let DBMS users run in-database graph algorithms,with a performance better than Spark GraphX in our exper-imental set-up.

KeywordsDBMS, Graphs, Queries, PageRank

1. INTRODUCTION

1.1 Background and MotivationGraph analytics is a field that is increasing its importance

every day. Furthermore, as the world becomes more inter-connected than before, graph data sets become larger andmore complex. In graphs analytics, the goal is to obtaininsight and understanding of complex relationships that arepresent in telecommunication, transportation and social net-works.

Relational databases remain the most common technologyto store transactional and analytical databases, due to op-timized I/O, robustness and security control. Even though,

∗First Author†Second Author

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full cita-tion on the first page. Copyrights for components of this work owned by others thanACM must be honored. Abstracting with credit is permitted. To copy otherwise, or re-publish, to post on servers or to redistribute to lists, requires prior specific permissionand/or a fee. Request permissions from permissions@acm.org.

c© 2016 ACM. ISBN 978-1-4503-2138-9. . . $15.00

DOI: 10.1145/1235

the common understanding is that DBMS cannot handle de-manding graph problems, because relational database queriesare not sufficient to express important graphs algorithms,and a poor performance of database engines in the context ofgraph analytics. Consequently, several graph databases andgraphs analytics systems have emerged, targeting large datasets, specially under the Hadoop/MapReduce platform. Inrecent years, Pregel and its open-source version Giraph havesupported the ”think as a vertex” approach. This approachis based on performing computations at the ”vertex” scopeand sending/receiving messages to/from the neighbors. Onthe other hand, the ”algebraic approach” solves graph al-gorithms via a generalized matrix multiplication, generallywith optimized C++ programs running in clusters with largemain memory.

Why Database Management SystemsDBMSs are between the most widespread system in the in-dustry. It is hard to imagine how to run an organizationwithout using DBMS, in any industry. Moreover, most ofthe internet rely on DBMSs for social networking, dynamiccontent or electronic commerce. Therefore, a lot of datais stored in these systems. We believe that efficient graphalgorithms for relational databases are a contribution thatwill support graphs analytics in large data sets stored inDBMS, avoiding wasting time exporting the data or settingup external systems.

Why a comparison with Spark-GraphXSpark is the emerging star for Analytics in the Hadoopworld, specially for its performance and programming envi-ronment. GraphX offers a complete library for graph analyt-ics, including optimized graph representations for Spark, theResilient Distributed Graph. Supporters of GraphX claim aperformance superior than other important graph Systems.

Summary of ContributionsIn this work we demonstrate that several graph al-gorithms can be expressed as an iteration of efficientSPJA queries that are defined over the foundation oflinear algebra operations. Furthermore, our algorithmsare not limited by the system RAM, since they are conceivedas secondary memory algorithms.

We present a simple pattern based in JOIN /AGGRE-GATION to solve PageRank as well as classical graph algo-rithms, such as Bellman Ford, Connected Components, andTopological Sort.

We show experimentally that Graph algorithms using JOIN/AG-

GREGATION shows better performance in Columnar andArray databases, when compared to a state-of-the art sys-tem, namely Spark-GraphX.

2. RELATED WORK”Do we have to reinvent the wheel?” is the question asked

by the authors in [13]. In such work, the authors shows thattheir SQL implementation of shortest path algorithm hasbetter performance that Neo4j, for social networking dataset. In the last years, the problem of the problem of solv-ing graph algorithms with SQL queries has received limitedattention. Jindal et all [8] presented some graph algorithmsusing a columnar database, showing that DBMS are com-petitive with specialized graph systems, namely Giraph andGraphLab. In [12], we proposed optimized recursive queriesto solve two important graph problems with SPJA queries:Transitive closure and All Pairs Shortest Path. This recentworld also shows that columnar DBMS technology performsmuch better than array or row DBMSs. In our recent work[3], we presented a theoretical discussion about solving fourgraph algorithms with relational queries, on the foundationof matrix-vector multiplication. Although Pregelix [2] is notbuild on the top of a relational DBMS, this system showsbetter performance than Graph-X, claiming that this sys-tem brings ”data-parallel query evaluation techniques fromthe database world”

3. DEFINITIONS

3.1 Graph DatasetLet G = (V,E) be a directed graph, where V is a set of

vertices and E is a set of edges, considered as an orderedpairs of vertices. Let n = |V | vertices and m = |E| edges. Agraph is frequently represented by the adjacency matrix, an× n matrix such that the cell i, j holds a 1 when exists anedge between the vertices i and j. The outdegree of a vertexv is the number of outgoing edges of v. The indegree of v isthe number of incoming edges of v.

Sparse GraphsIn general, real-life graphs are sparse. While there are morethan one billion user accounts in Facebook, a typical usermay be connected with just a few hundreds contacts. Otherexamples are roads connecting cities and flights linking air-ports. Therefore, it is reasonable to represent the adjacencymatrix E in a sparse representation, which saves both com-puting, memory and storage resources. Several methods torepresent sparse matrices are well known, and the interestedreader may check [1]. In our work, sparse matrices ( spe-cially E ) are represented as a set of tuples (i, j, v) such thatv 6= 0, where i and j represent row/column, and v repre-sents the value of the entry. Since cells where v = 0 are notstored, the space complexity will be determined by m = |E|.

Degree DistributionConsidering the degree of the vertices is a random variable,it is well known that web graphs and social networks graphshave a skewed distribution: vertices have high probabilityof a low degree, but very low probability of a high degree.Researchers has observed that the distribution of degree inboth world wide web and social network graphs follows thePower Law. [5, 11]

3.2 Parallel Systems

3.2.1 Column StoreColumnar DBMSs have emerged for fast OLAP. While

row DBMS in general store the data by records, columnarDBMS stores data by column.

3.2.2 Array StoreArray store is a technology aimed to provide efficient stor-

age of array-shaped data. Bidimensional or multidimen-sional arrays are supported, as well as vectors. Array storesorganize array data by rectangular data blocks, named chunks.Each cell of the array has a predefined position in the chunk,just as regular arrays are stored in main memory Becausethe access to the data is relative to the initial address of thechunk, indices are not necessary. Data partitioning acrossnodes is done in a chunk basis. Let’s said that we have acluster of 4 nodes, and our data is comprised of 12 chunks.Chunks 1,5,9 will be assigned to node1, chunks 2,6,10 willbe assigned to node2, and so on.

3.2.3 SparkBuild on top of Hadoop,

3.3 Fundamental Graph Algorithms.

3.3.1 Bellman-Ford : Shortest PathBellman-Ford is a classical algorithm for solving the Single

Source Shortest Path problem. A standard implementationcan be found in [4]. The algorithm iterates on every vertex,and execute a relaxation step for each edge of the currentvertex. The algorithm in [4] presents a drawback: verticesthat are not reachable from the source are useless visited,since they cannot be part of the solution set.

Sk = Sk−1∗Ek = S0∗E∗E∗E...(k vector-matrix multiplications)(1)

3.3.2 Weakly Connected ComponentsA weakly connected component of an undirected graph G

is a subgraph such that for any vertices u, v elements of G,exists a path (u, v). Directed graphs are generally analyzedwithout consideration of the edges direction. A recent, butwell known algorithm is HCC, proposed in []. The algorithmis expressed as an special form of matrix multiplication be-tween the adjacency matrix and a vector where the

⊗op-

eration is × and the⊕

operation is min. Each entry of theresulting vector is updated to minimum value between theresult of matrix computation and the current valued of thevector. Intuitively, vertex v receives the ids of all its neigh-bors as a message, The attribute of the vertex is set to theminimum among its current value , and the minimum valueof the incoming messages. .

3.3.3 PageRankPageRank is an algorithm created by the founders of Google,

to rank the web pages in WWW. The output of PageRank isa vector where each value v[i] is the probability of arriving toi, after a random walk. It is well known that PageRank canbe computed as powers of a modified transition matrix [9].Since page ranking is conceived as a Markov process, thecomputation can be performed as a iterative process thatstops when the Markov chain gets stabilized. The transition

matrix T is defined as Ti,j = Ej,i/degj when Ej,i = 1; oth-erwise Ti,j = 0. Notice that if outdegree(j) = 0, then the

jth column of T is a column of zeroes. Let T′

= T + D ,where D is a matrix such that D[i, j] = 1/n if the columnj is a 0 column. To overcome the problem of disconnectedgraphs, PageRank incorporates an artificial low-probabilityjump to any vertex of the graph. Let A be a n × n ma-trix, whose cells contains always 1/n, and p the damping

factor. The power method can be applied on T′′

defined as:

T′′

= (1− p)T′

+ (p/n)A, as presented in Equation ??.

Sk = (T ′′)k ∗ S0 (2)

Although equation 2 seems to be simple, computing withlarge matrices would be unfeasible. While T might be sparse,

the sparsity of T′

is not guarantee. Moreover, since A isdense by definition, T ′′ is dense, too. Equation ?? can beexpressed as:

Sd = (1−p)T ∗Sd−1 + (1−p)D ∗Sd−1 + (p/n)A∗Sd−1 (3)

3.3.4 Topological SortA topological sort of a graph G(V,E) is an ordered se-

quences of vertices in V , such that if vertex u is before vertexv, there is not a path from v to u. The standard Topologicalsort algorithm is based on a depth-first search.

4. A NEW PATTERN FOR EFFICIENT GRAPHALGORITHMS IN PARALLEL DATABASESYSTEMS

As the reader may already know, several graph algorithmscan be expressed as a iteration of matrix-vector multiplica-tions. Specifically, in each iteration the adjacency matrix ismultiplied by a vector Sd−1. The result is stored in a vectorSd. For instance, in PageRanking, each iteration is a betterapproximation of the PageRank vector. Bellman Ford withMatrix Vector multiplication id presented in [6], ConnectedComponents in [] and PageRank in []. In this work, we willelaborate on Topological Sort.

Why this way to compute graph algorithms is importantfor this this work? Because matrix vector multiplication canbe expressed easily with regular relational queries. Follow-ing the intuition, the reader may question the performance ofthis method: A large graph with one million vertices leads toa huge adjacency matrix with one trillion cells; thus the mul-tiplication of such a large matrix times a large vector mightbe unfeasible. Though, since real world graphs are sparse,zeroes are not stored, and the adjacency matrix would needO(n) space. MOreover, When the input matrix and vectorsare stored orderly, the computation of the multiplicationcan be done in nlogn, as the worse case. In a relationalDBMS, the actual operation to compute the matrix mul-tiplication is a regular query with a join between E andV , and a subsequent aggregation. In array databases, theoperation can be implemented either via join+aggregationor via the built in matrix multiplication operator (whichinvokes SCALAPACK), but we demonstrate later that thefirst option presents better performance.

The entries of E might be weighted or not, dependingon the problem: un-weighted entries for PageRank and CC,and weights representing distances for shortest paths.

Prior to the first iteration, the vector S has to be set to aninitial state accordingly to the problem : infinite distances

Table 1: Matrix-Vector Multiplication with Rela-tional Queries

(+ , x ) semiringSELECT R.i, E.j, sum(R.v * E.v)FROM R join E on R.j=E.iGROUP BY i,j(min, +) semiringSELECT E.i, min(E.v +S.v)FROM E join S on E.j=S.iGROUP BY i( g(),

⊗) general semiring

SELECT E.i, g(S.v⊗

E.v)FROM E JOIN S on E.j=S.iGROUP BY i

for Bellman-Ford, isolated components in CC, or a defaulinitial ranking on Pagerank.

In the pth iteration, the vector Spis computed as E ∗Sp−1

The algorithm iterates until convergence, or when it reachesthe max number of iterations.

We use Matrix-vector multiplication 1) as an abstraction,for a better understanding of the algorithm. 2) becauseit has been largely proved that some graph algorithms areequivalent to matrix vector multiplication.

4.1 Graph StorageGraphs are stored in table containing the edges and their

attributes, and an optional table containing vertex attributes.In social network analysis, vertex attributes may be home-town, company, and other personal information. The com-putation of our algorithms relays in two disk data structures:Table E, a projection of the Edges table, and Table S, a pro-jection of the Vertex table, with a numerical value v. TableE(i, j, v), with primary key (i, j), stores the adjacency ma-trix of G in sparse representation; v represents a numericattribute of the edge, representing distances in a road net-work, costs in distribution network, or any required weight.In this representation, entries with a value of zero are notstored. Thus, the space required for table E is O(m), muchsmaller than O(n2).

4.2 Graph PartitioningDue to the Power Low, a skewed data distribution is com-

mon in real life graphs. In parallel graph processing, Graphpartitioning has as an objective to get an even data distri-bution among the computing nodes( workers). Moreover, inthe case of the family of algorithms studied in this work,a second objective is critical for performance: minimizingthe data shuffling. The most demanding operation in thisfamily of algorithms is the query that computes the new vec-tor S, comprised of a join + aggregation. Moreover, this iscomputed in an iterative way. We focus on optimizing thejoin. The parallel join runs efficiently when data shufflingis avoided. In other words, rows in S matching rows in Eshould be found in the same worker node.

4.2.1 Columnar DBMSThe graph data set is stored as a list of edges in table E.

The graph should be partitioned in such a way that unevendata distribution is avoided, and the join is done with localdata.

For the general case, E is partitioned through the N nodesby a hash function on the starting vertex id. Therefore, the

21

311

5

4

6

79

8

10

i j v

2 1 1

2 3 1

3 5 1

4 5 1

5 6 1

5 7 1

5 8 1

6 4 1

7 6 1

9 5 1

10 9 1

11 2 1

G E

Figure 1: The table E for a sample graph G

Node 1 Node 2 Node 3 Node 4

E E E Ei j v i v i j v i v i j v i v i j v i v

2 1 1 2 2 5 6 1 5 5 3 5 1 3 3 7 6 1 7 7

2 3 1 6 6 5 7 1 4 5 1 4 4 9 5 1 9 9

6 4 1 5 8 1 10 9 1 10 10 11 2 1 11 11

S S S S

Figure 2: Partitioning of Table E in four nodes

set of edges having the same starting vertex is stored in onlyone computing node. The table S is partitioned in the samemanner. The benefit of this partition is that any vertex inS has the corresponding matching vertex in E in the samecomputing node, avoiding costly ”data shuffling”.

4.2.2 Array DBMSAs a first step, the graph data set is uploaded to the Array

DBMS as a ”flat” array, where each entry has 3 properties:i, j and v. In the second step, the data set is translatedto an adjacency matrix which is stored in disk as a bidi-mensional array. In general, big-data graphs, are charac-terized by a sparse adjacency matrix. In the case of array

Table 2: Data partitioning and Physical Ordering inColumnar DBMS

Algorithm Join Partition OrderBellman-Ford E ./i=i S hash(E.i);hash(S.i) E.i, S.iConn.Component E ./i=i S hash(E.i);hash(S.i) E.i, S.iPageRank T ./j=i S hash(T.j);hash(S.i) T.j, S.iTopo.Sort E ./j=i S hash(E.j);hash(S.i) E.j, S.i

Figure 3: Adjacency matrix heat map before andafter repartitioning.

Figure 4: Data distribution in cluster before andafter Repartitioning

DBMSs, the adjacency matrix is partitioned in blocks ofthe same size. SciDB assign the chunks in a rigid way;chunks are assigned to the WORKERS just by applyingMOD(chunknumer, Workers). Skewed graph data resultson uneven distribution of the data to the set of chunks; asa consequence, workers also have an uneven computationload. The LivingJournal data set is very skewed; a smallset of vertices located in the first rows of the adjacency ma-trix has hundreds and even thousands links. As a result, afew blocks of the matrix concentrates a vast majority of thedata. To alleviate that problem, we partition the data witha simple strategy: re indexing of the adjacency matrix: i-> CHUNKSIZE* (i MOD numWorkers) + i/numworkers.This strategy works in LivingJournal dataset because dataconcentrated in a small bloack of the adjacency matrix isspread through the whole matrix. Figure ... shows a plotof the density of the adjacency matrix before and after there-indexing. Furthermore, Fig shows the data distributionamong the workers. Clearly, the reindexing is helpful toalleviate the problem of skewed graph data.

The database user have to choose the appropiate chunksize S is partitioned in Cs chuncks, such that Cs mod N =0,where N is the number of processors. On the other hand thearray E is partioned in Ce chuncks and Cs mod N =0.

the join will find the matching data in the local node.

4.2.3 SparkGraphX includes a set of bult-in partitioning function for

the edges collection. Following the vertex-cut approach,edges are never cut. Edges are partitioned by several strate-gies. RandomVertexCut: The graph is partitioned by as-

signing edges to computing node in random way , EdgePar-tition1D: the adjacency matrix is partioned by horizontalcuts. EdgePartition2D: the adjacency matrix is partitionedin a grid manner, both horizontal and vertical cuts. Canon-icalRandomVertexCut: xxxxxxxxxxxxxx

4.3 Algorithm Pattern

Algorithm 1: A Pattern for Graph Algorithms withMatrix-VEctor Multiplication

Data: Table E, S0, R0, source s (optional), ⊕,⊗, f , εResult: Table Rd(j, v)d = 0; ∆ = 1;while ∆ > ε do

d = d+ 1 ;Sd ← (E ⊗ Sd−1)) ;

end

return Sd ( or∑k

d=1 Sd ) ;

4.4 Particularities for Columnar/Array DBMSsand Spark

The particularities for Columnar and array databases aredescribed below.

Columnar DBMSsWe implement simple but efficient SQL queries that per-forms this operation via Join and Aggregation. In parallelcolumnar DBMS, two factors are important for a good per-formance per iteration:

1. Presorted data: The join between E as in S can achievea linear time complexity is these tables are presortedby the columns participating in the join condition.This is critical for very large graphs.

2. Local match: Rows that satisfy the matching conditionare always in the same computing node. This is keyto avoid data shuffling.

Array DBMSWe propose to compute the matrix-vector multiplicationwith a combination of cross-join and aggregation operations,and we compare our approach to the standard way: call theSPGEMM() scidb operator, routine for sparse matrix multi-plication from the LAPACK library. In the array DBMS, acarefully data partition let us to compute the join avoidingshuffling: the matches of the join condition are always in thesame node. The (sparse) array-like data organization makespossible a merge join, since data is stored in order. On theother hand, the data partition needs to consider skewed datadistribution. It is natural to assign the edge (i, j) to the po-sition (i, j) in the disk array. But due to the power low, thisnaive procedure may lead to uneven data distribution. Toalleviate this problem , we allocate the data with a functionthat redistributes the data when the distribution is skewed.

Joins vs ScaLapack in Array DBMSSpark-GraphXSpark-GraphX stores graphs with two main data structures:EdgeRDD and VertexRDD. Both of them are extensions of

the Spark RDD data structure. The fundamental operationto solve graph problems in GraphX is aggregateMessages,which receives as parameters a sendmsg (or map) function,and an aggregate (or reduce) function. As output, aggre-gateMessages returns a RDD which associates every vertexwith the computed value. In [?], Gonzalez et al. state ”Weidentified a simple pattern of join-map-groupby dataflow op-erators that forms the basis of graph-parallel computation.Inspired by this observation, we proposed the GraphX ab-straction ...”

5. DETAILED EXPLANATION OF ALGO-RITHMS

5.1 Computing PageRankAlthough the full equation for PageRank is Equation 3,

several implementations of the algorithm use a simplifiedversion. The simplified version neglects the term (1− p)D ∗Sd−1.

Our first step is to compute T, which requires the compu-tation of the outdegrees. T is carefully partitioned, to pro-mote local computation, and to reduce data shuffling. Thevector S0 is initialized with a uniform probability. There-fore, S[i]= 1/n. PageRank terminates when the algorithmconverges, or when the maximun number of iterations isreached.

5.1.1 Columnar DBMSRecall that T is a modified form of ET . The query ??

does not return a row for T.j when the outdegree of v iszero. A mistake: It seems straightforward to computePageRank, but mistakes are possible. In [7], the authorsshow SQL queries to solve pageRank with unaccurate re-sults. In the first query, no matches are found for a vertexi if outdeg(i)=0; as a result, these vertices are neglected.The second query does not return a row for a vertex j whenindeg(j)=0.

We solve this problem by a three step query: 1) Join +aggregation, which computes the variable part of the pagerank; 2) Join of S with a scalar, which computes the con-stant part for all the vertices (nobody is left behind); 3)Agreggation of the values in 1 and 2.

insert into GRPselect T. i , 0 . 8 5∗sum( S0 . v/T. v ) v , 0 f

from S0 join T on S0 . i=T. jgroup by T. i ;

insert into GRPselect S0 . i , S ca l a r . va v , f f

from S0 ,( select (0 . 85∗sum( S0 . v∗ f )+

0 .15 )/ count (∗ ) vafrom S0 ) Sca l a r ;

insert into S1Mselect i ,sum( v ) v ,sum( f ) f

from GRPgroup by i ;

Notice D and A contains n identical rows. Therefore,their respective products by Sd−1 results on vectors VD andVA, such that every value in VD and VA is constant. Conse-quently, it is sufficient to compute the inner product of any

Table 3: Comparison of four graphs algorithms... SSSP Connected Comp. PageRank Topological Sort

Computed as Sd ← ET · Sd−1 Sd ← ET · Sd−1 Sd ← T · Sd−1 Sd ← E · Sd−1

Semiring op.(⊕,⊗) min,× min,× +,× ∧,⇒Value S[i] distance s to i id of component probability order

S0 defined as S0[s] = 0 S0[s] = s S0[i] = 1/n S0[i]=1 iff outdeg(i)=1|S0| 1 1 n | {i ∈ V | outdeg(i) = 0} |

Output Sk Sk Sk S0 + S1 + ..Sk

Time complexity O(n logn) O(n logn) O(n logn) O(n logn)Scope from source s ∀i ∈ V ∀i ∈ V ∀i ∈ V

row of D by Sd−1, to get VD; VA is computed in a similarway.

5.1.2 Computation in Array DBMSLike in relational DBMS, the base of the computation is

iterative matrix vector multiplication. The input is the ma-trix E stored in ”flat” array array, a uni-dimensional arraywhere i,j,and v are attributes. Whit this input we com-pute a Transition matrix, that is convertes in a sparse bi-dimensional array, and it is partitioned ”on the fly” to avoidunbalances due to skewed data distribution, as explained inthe previous section. As a collateral effect of using sparsearrays, the join between T and S does not return rows forthose vertices where indegree=0. To avoid that problem, wefill with zeroes the diagonal of the transition matrix.

1) Compute the scalar value. 2) A JOin + Aggregationthat computes the new vector S

s t o r e (p r o j e c t (

apply (s u b s t i t u t e (

aggregate (p r o j e c t (

apply (c r o s s j o i n (

T, S ,T. j , S . i ) ,

u , ALPHA∗S . v∗T. v ) ,u ) ,

sum(u) as vs , i ) ,bu i ld (<va l : double>[x =0 : 0 , 1 , 0 ] , 0 ) ,vs ) ,

v , vs+(1−ALPHA)/n + ALPHA∗SCALAR) ,v ) ,

S ) ;

5.1.3 Computation in Spark-GraphXWe explain the algorithm included as part of the GraphX

library. PageRank is solved iteratively; aggregateMessage isthe main operation in each iteration; it is conceptualized asmap function applied to messages sent from neighbor nodes,and a reduce function that performs an aggregation. Specif-ically, the map function is an scalar multiplication of sourceattribute by edge attr, and the aggregation is a summation.Notice that aggregateMessage returns a VertexRDD, whichis similar to the output of the join+aggregation in relationalDBMS and in array DBMS.

5.2 Connected ComponentsThe authors of [10] proposed HCC, an iterative algorithm

to compute Connected Components. The algorithm can be

sketched as follows: Let Vo a vector of where each entryrepresents a graph vertex, and initialize each value of V withthe its own vertex id. V1i = min ( V0i, min( V0n ) )

This algorithm is been applied in MapReduce, Giraph andrecently [7] applied in RDBMS. The authors shows a joinbetween the edges table an two different vertex table.

We base our algorithm in HCC. In contrast with HCC, thelast (external) min is not necessary. While the algorithm isprocessing, an artificial self loop in inserted in every node.

5.2.1 Computation in Columnar DBMSIn contrast with [7], we avoid a three table join by an

artificial self loop in every vertex.

select E. i , min( S0 . v∗1) vfrom S0 005M S0 join

( select i , j from webGoogleunion a l l

select i , i from S00 ) Eon S0 . i = E. j

group by E. i ;

5.2.2 Computation in Array DBMS

s t o r e (p r o j e c t (

s u b s t i t u t e (aggregate (

p r o j e c t ( apply (c r o s s j o i n (

E, S ,E. j , S . i ) ,

u , S . v ∗1) , u ) ,min(u) as v , i ) ,

bu i ld (<va l : int64 >[x =0 : 0 , 1 , 0 ] , 0 ) , v ) ,v ) ,

S )

5.2.3 Computation in Spark-GraphX

5.3 Single Source Shortest Path

5.3.1 Computation in Columnar DBMSRecalling section 3.3.1, SSPS is computed by an iteration

of matrix-vector multiplication: the transposed of the adja-cency matrix multiplied by a vector which holds the ”cur-rent” minimum value. From a relational point of view, thevector Sd is is stored Sd in a database table with schemaSd(j, v), where j is a destination vertex, and v is the min-imum distance known at the current iteration (relaxed dis-tance). Both the standard and the linear algebra algorithms

Table 4: Data SetsData Set h Vertices Edges Descriptionweb-Google 0.5M 5 M Hyperlink Graphsoc-Pokec 1.6M 30M Social Network

soc-LiveJournal 4.8M 69M Social Networkwikipedia-en 12.4M 378M Hyperlink Graph

Webdatacommons 49.0M 620M Hyperlink Graph

require to initialize as ∞ every vertex, but the source. In-stead, in a relational database we only include vertices inSd when an actual path from s has been found. When thealgorithm starts, Sd is sparse; only one non-zero value.

The matrix multiplication under the min-plus semi-ringreproduces the relaxation step: In the dth iteration, theminimum distance is computed considering the relaxed valuefrom the iteration, stored in Sd−1, as well as the value of newedges discovered in the current iteration.

5.3.2 Computation in Array DBMSThe computation of the vector Sp can be done either

by matrix-vector multiplication using SPGEMM() or by ajoin+ aggregation.

5.3.3 Computation in Spark-GraphX

6. EXPERIMENTSWe conduct experiments to compare performance and re-

sults of three algorithms, under three different systems: Colum-nar database (HP Vertica), array database (SciDb) and SparkGraphx. The three systems were installed in the same hard-ware: a four node cluster, each node 4GB RAM and 1TBdisk, running Linux Ubuntu. In the three systems, redun-dant copies were enabled.

6.1 Data SetsIn graph analysis, social network and web networks are

considered challenging, not only because its size, but alsobecause the skewed distribution of the degree of the vertices.In the case of web graphs a popular page can be referencedfor many thousands pages, and a social network account canbe followed for thousand of users, too. We study our algo-rithms with three data sets from the SNAP repository. Sincevery large web graphs are not available in SNAP, we use aweb data set from WebCommons. All the data sets are wellknown, and statistics are available, as maximum degree, av-erage degree, number of triangles, size of the largest weaklyconnected component, and so on.

6.2 Parameters and Initialization

Connected ComponentsNo parameters or specific initialization is required.

Shortest PathFor all the data sets, we run shortest using as source vertexthe vertex id that identifies the largest connected compo-nent. Because all of our data sets have very large connectedcomponents ( more than 90% of the graph), this is the hard-est vertex for the computation of SSPS: the majority of thevertices are reachable.

0

500

1000

1500

2000

2500

3000

3500

4000

5 30 69 378 620

PageRank:Execution Time

Vertica SciDB Spark_GraphX

Figure 5: PageRank Execution Time

Page RankThe initial probabilities are initialized to 1/n. The numberof maximum iterations is set to 1o.

6.3 Performance ComparisonsOur experimental results shows that in general, our al-

gorithm running in Columnar DBMS have superior perfor-mance than the standard algorithm in GraphX. Further-more, the performance of PageRank under our approach is50% better than the Spark implementation, even when thedata sets fits in main memory. For the medium sized dataset( LIveJournal), the difference is even larger, our proposal is80% faster.

6.4 Evaluation of Graph Computation in Par-allel Systems

7. CONCLUSIONS

8. REFERENCES[1] Z. Bai, J. Demmel, J. Dongarra, A. Ruhe, and

H. van der Vorst. Templates for the solution ofalgebraic eigenvalue problems: a practical guide,volume 11. Siam, 2000.

[2] Y. Bu, V. Borkar, J. Jia, M. J. Carey, and T. Condie.Pregelix: Big(ger) graph analytics on a dataflowengine. Proc. VLDB Endow., 8(2):161–172, Oct. 2014.

[3] W. Cabrera and C. Ordonez. Unified algorithm tosolve several graph problems with relational queries.In Proceedings of the 10th Alberto MendelzonInternational Workshop on Foundations of DataManagement, Panama City, Panama, May 8-10,2016, 2016.

[4] T. H. Cormen, C. E. Leiserson, R. L. Rivest, andC. Stein. Introduction to Algorithms, Third Edition.The MIT Press, 3rd edition, 2009.

[5] M. Faloutsos, P. Faloutsos, and C. Faloutsos. Onpower-law relationships of the internet topology. InACM SIGCOMM computer communication review,volume 29, pages 251–262. ACM, 1999.

Table 5: PageRank Execution Time: GraphX vs Array vs ColumnData set M GraphX Array Columnweb-Google 5M 1 1 1soc-pokec 30M 1 1 1LiveJournal1 69M 1 1 1wikipedia-en 378M 1 1 1Webdatacommons 620M 1 1 1

Table 6: SSSP, Connected Components and PageRank: GraphX vs Array vs Column

Data set |E| SSSP Connected Components PageRankColumn Array GraphX Column Array GraphX Column Array GraphX

web-Google 5M 30 145 34 32 175 32 30 143 58soc-pokec 30M 37 1 59 76 1 83 100 1 153LiveJournal1 69M 128 405 166 148 919 451 261 1073 477wikipedia-en 378M 4574 crash 654 5091 crash 695 stop crashWeb WDC 620M stop 4677 stop 1 3359 1 1

[6] J. T. Fineman and E. Robinson. 5. FundamentalGraph Algorithms, chapter 5, pages 45–58.

[7] A. Jindal, S. Madden, M. Castellanos, and M. Hsu.Graph analytics using vertica relational database. InBig Data (Big Data), 2015 IEEE InternationalConference on, pages 1191–1200. IEEE, 2015.

[8] A. Jindal, P. Rawlani, E. Wu, S. Madden,A. Deshpande, and M. Stonebraker. Vertexica: yourrelational friend for graph analytics! Proceedings ofthe VLDB Endowment, 7(13):1669–1672, 2014.

[9] S. D. Kamvar, T. H. Haveliwala, C. D. Manning, andG. H. Golub. Extrapolation methods for acceleratingpagerank computations. In Proceedings of the 12th Int.Conf. on World Wide Web, pages 261–270. ACM,2003.

[10] U. Kang, C. E. Tsourakakis, and C. Faloutsos.Pegasus: A peta-scale graph mining systemimplementation and observations. In Proceedings ofthe 2009 Ninth IEEE International Conference onData Mining, ICDM ’09, pages 229–238, Washington,DC, USA, 2009. IEEE Computer Society.

[11] A. Mahanti, N. Carlsson, A. Mahanti, M. Arlitt, andC. Williamson. A tale of the tails: Power-laws ininternet measurements. IEEE Network, 27(1):59–64,2013.

[12] C. Ordonez, W. Cabrera, and A. Gurram. Comparingcolumnar, row and array DBMSs to process recursivequeries on graphs. Information Systems, pages –, 2016.

[13] A. Welc, R. Raman, Z. Wu, S. Hong, H. Chafi, andJ. Banerjee. Graph analysis: do we have to reinventthe wheel? In First International Workshop on GraphData Management Experiences and Systems, page 7.ACM, 2013.