Budapest University of Technology and Economics Technical University of Darmstadt Real-Time Systems...

Budapest University of Technology and Economics

Technical University of DarmstadtReal-Time Systems Lab

Benchmarking for Graph Transformation

Gergely Varró [email protected]

Andy Schü[email protected]ániel Varró [email protected]

2Budapest University of Technology and Economics


Talk overview

IntroductionGT

overview

Paradigm & tool features

Benchmarking terminology

Measurements



Model transformation in MDD• Model Driven Development (MDD)

– thorough system modeling before implementation– precise and visual SW models

• on a high abstraction level• from different viewpoints

– automatic code generation• Model transformation

– automated translations within and between visual modeling languages

• MDD requirements on model transformation:– Specification: high-level specification language– Execution: efficient algorithms and techniques– Industrialization: extensive tool support

• Upcoming standard: QVT



Specification techniques for MT

• QVT• Main features:

– object based patterns– precondition– unidirectional– bidirectional

• Problems:– immature tool support– lack of experience

• GT• Main features:

– similar to QVT– wide tool support

• Problems:– non-bidirectional

(except for TGGs)

• Pattern matching phase:– most critical issue

from performance viewpoint

– same as in QVT



Benchmarking

• Aim:– systematic and reproducable measurements– on performance– under different and precisely defined

circumstances

• Overall goal: help system engineers in selecting tools

• Popular approach in different fields– AI– relational databases– rule-based expert systems



Benchmarking in model transformation

• Specification examples for MT– Goal: assessing expressiveness

– UML-to-XMI, object-relational mapping, UML-to-EJB, etc.

• No performance benchmarks for MT

• In the paper:Benchmarks for graph transformation tools– terminology for benchmarking

– quantitative comparison of tools

• Ongoing research:– Benchmarks for general model transformation tools



Talk overview

IntroductionGT

overview



Measurements



Metamodel

Metamodeling

h1:held_by

n4:next

n1:next n2:next

n3:next

h2:held_by

h3:held_byh4:held_by

next

blocked

held_by releasetoken

request*

1

*

1*

111

***

*

Class

Association

Multiplicity constraint

At most one

Arbitrary

p1:Process p3:Process

p2:Process

p4:Processr4:Resource

r1:Resource r2:Resource

r3:Resource

Resource

Process

Object

Link

Slot

Inheritance

Attribute

Instance model



Graph transformation rule

r:Resource

p:Process

r:Resource

ReleaseRule

hb:held_by

reqn:requestrel:release

rn:Resourcep:Process

LHS RHS

Negative applicationc

ondition

p1:Process

r:Resource

GiveRule

n:nextp2:Process

rel:release

p1:Process

r:Resource

n:nextp2:Process

t:token

LHS RHS

Left-HandSide

Right-HandSide



Pattern matching phase

h1:held_by

n4:next

n1:next n2:next

n3:next

h2:held_by



p2:Process



r3:Resource

r:Resource

p:Process

r:Resource

ReleaseRule

hb:held_by



LHS RHS



Updating phase

h1:held_by

n4:next

n1:next n2:next

n3:next

h2:held_by



p2:Process



r3:Resource

r:Resource

p:Process

r:Resource

ReleaseRule

hb:held_by



LHS RHS

rl:release



Talk overview

IntroductionGT

overview



Measurements



Paradigm features

• Definition: problem characteristics

• Model size– number of nodes and edges in the model graph

• Pattern size– number of nodes and edges in the LHS graph

• Fan-out (maximum degree of model nodes)– number of edges adjacent to a certain node

• Number of matchings

• Transformation sequence length



Tool features

• Definition: typical optimization strategies supported by different tools

• Multiplicity based optimization– More powerful strategy when traversing edges with ‘at

most one’ (0..1) multiplicity

• Parameter passing

• Parallel rule execution

• ‘As long as possible’ (ALAP) rule application

• Indexes (for key/index attributes)

• Incremental pattern matching

• ...



Parameter passing I.

h1:held_by

n4:next

n1:next n2:next

n3:next

h2:held_by



p2:Process



r3:Resource

r:Resource

p:Process

r:Resource

ReleaseRule

hb:held_by



LHS RHS

rl1:release

•Between consecutive rule applications•No recalculation in later steps



Parameter passing II.

rl1:release

n4:next

n1:next n2:next

n3:next

h2:held_by



p2:Process



r3:Resource

p1:Process

r:Resource

GiveRule

n:nextp2:Process

rel:release

p1:Process

r:Resource

n:nextp2:Process

t:token

LHS RHS

t1:token



Parallel rule execution (PM)

h1:held_by

n4:next

n1:next n2:next

n3:next

h2:held_by



p2:Process



r3:Resource

r:Resource

p:Process

r:Resource

ReleaseRule

hb:held_by



LHS RHS

4

1 2

3

1

43

2

4

3

2

1

•Calculation of all matchings•Independent matchings



Parallel rule execution (Update)

h1:held_by

n4:next

n1:next n2:next

n3:next

h2:held_by



p2:Process



r3:Resource

r:Resource

p:Process

r:Resource

ReleaseRule

hb:held_by



LHS RHS

4

1 2

3

1

43

2

4

3

2

1

rl1:release

rl3:release

rl2:release

rl4:release

•No re-evaluation of valid matchings•Transaction block



Talk overview

IntroductionGT

overview



Measurements



Terminology I.

Me

tam

ode

l

GT

rule

s

Instance m

ode

l

Tra

nsforma

tion

sequ

ence

Pa

radigm

fea

ture

s

To

ol fea

ture

s

Scenario – – – – – –

– Broad application field of graph transformation• model analysis• model transformation• visual languages with dynamic operational

semantics

– Informal characteristics



Terminology II.

Me

tam

ode

l

GT

rule

s

Instance m

ode

l

Tra

nsforma

tion

sequ

ence

Pa

radigm

fea

ture

s

To

ol fea

ture

s

Scenario – – – – – –Benchmark example + + – – – –

next

blocked

held_by releasetoken

request*

1

*

1*

111

**

*

*

Resource

Process

TakeRule

RequestRule

GiveRule

ReleaseRule

LHS RHS• Incarnation of a scenario– fulfills the informal characteristics

– e.g. Mutex for the simulation of visual languages



Terminology III.

Me

tam

ode

l

GT

rule

s

Instance m

ode

l

Tra

nsforma

tion

sequ

ence

Pa

radigm

fea

ture

s

To

ol fea

ture

s

Scenario – – – – – –Benchmark example + + – – – –Test set + + PD PD PD –

h1:held_by

nN:next

n1:next

hN:held_by

p1:Process

p2:Process

pN:ProcessrN:Resource

r1:Resource

Instance model

1. N ReleaseRule

2. N GiveRule

3. N RequestRule

4. N TakeRule

• Parametric and deterministicspecification



Terminology IV.

Me

tam

ode

l

GT

rule

s

Instance m

ode

l

Tra

nsforma

tion

sequ

ence

Pa

radigm

fea

ture

s

To

ol fea

ture

s

Scenario – – – – – –Benchmark example + + – – – –Test set + + PD PD PD –Test case + + PD PD PD +

• Multiplicity based optimization: OFF

• Parameter passing: OFF

• Parallel rule execution: ON



Terminology V.

Me

tam

ode

l

GT

rule

s

Instance m

ode

l

Tra

nsforma

tion

sequ

ence

Pa

radigm

fea

ture

s

To

ol fea

ture

s

Scenario – – – – – –Benchmark example + + – – – –Test set + + PD PD PD –Test case + + PD PD PD +Run + + + + + +

• Complete characterization

• Reproducable measurements

N = 10



Talk overview

IntroductionGT

overview



Measurements



Categorization

• Style– Interpreted: AGG, GTDB

– Compiled: PROGRES, Fujaba

• Base algorithm– Constraint satisfaction: AGG, GTDB

• variables + constraints

– Local searches: Fujaba, PROGRES • step-by-step extension of the matching



Environment

• Hardware and OS– 1500 MHz Pentium

– 768 MB RAM

– Linux kernel of version 2.6.7

• Tool related– Runs without GUI

– Rule applications guided by Java programs• PROGRES: guided by C program

– Standard services of the default distribution• fine-tuning based on developers’ suggestions



Tool specific issues

• AGG– Manual implementation of parameter passing

• Progres– GRAS (default underlying graph-oriented database)– Prolog-style cuts (determinism)– User undo/redo mechanism being switched off

• Fujaba– Additional node for storing processes and resources– Slightly altered models to provide better performance

• GTDB– MySQL (version 4.1.7)– No additional optimization



Tool feature support

Multiplicity based

optimization

Parameter passing

Parallel rule application

AGG – + –

PROGRES + + +

Fujaba + + +

GTDB – + +



Quantitative comparison

Updating phase

0.01

0.1

1

10

100

10 100 1000

N

av

g. t

ime

(m

s)

AGG

PROGRES

FUJABA

DB

• Result set clean-up procedure• Constant overhead distributed over more rule applications

• Compilation step from graphsto categories in each GT step



Effect of multiplicity based optimization

10 100 1000 10 100 1000 10 100 1000

0.1

1

10

100

1000

10 100 1000

GiveRule

10 100 1000 10 100 1000 10 100 1000

0.1

1

10

100

1000

10 100 1000

ReleaseRule

AGG PROGRES FUJABA GTDB



Effect of parameter passing

10 100 1000 10 100 1000 10 100 1000

0.1

1

10

100

1000

10 100 1000


STS test set, Pattern matching phase, ReleaseRule



Effect of parallel rule application

0.1

1

10

100

1000

10 100 1000 10 100 1000 10 100 1000 10 100 1000


ALAP test set, Pattern matching phase, ReleaseRule



Thanks for your kind attention

• Contributions to model transformation benchmarking are welcome– additional benchmark examples, test sets, test

cases

– participation of further tools

– new optimization strategies

• Benchmark example descriptions, specifications, and measurement results– http://www.cs.bme.hu/~gervarro/benchmark/

Budapest University of Technology and Economics Technical University of Darmstadt Real-Time Systems...

Documents

Transcript of Budapest University of Technology and Economics Technical University of Darmstadt Real-Time Systems...