Incremental Data Transformations on Wide-Column Stores with NotaQL
Applications of incremental pattern matching in model transformations
-
Upload
istvan-rath -
Category
Technology
-
view
573 -
download
5
description
Transcript of Applications of incremental pattern matching in model transformations
![Page 1: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/1.jpg)
Budapest University of Technology and EconomicsDepartment of Measurement and Informa<on Systems
Budapest University of Technology and Economics
Applica'ons of incremental pa1ern matching in model transforma'ons
István Ráth ([email protected])
1
![Page 2: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/2.jpg)
Overview Introduc'on
o VIATRA2o Incremental graph pa1ern matching
Part I: The impact on performanceo Early benchmarks and improvementso EMF-‐INCQuery
Part II: Beyond performanceo Event-‐driven and change-‐driven transforma'onso Constraint sa'sfac'on solving over models – CSP(M)o Model simula'on by stochas'c graph transforma'ons
Future research direc'ons
2
![Page 3: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/3.jpg)
About me h1p://mit.bme.hu/~rath
I amo a soRware engineer graduated in 2006o a PhD candidate at BUTE, hope to finish this yearo a fan and supporter of Eclipse
I’ve been workingo with Dr. Dániel Varró since 2004o as a VIATRA2 developer and enthusiast o also as the chief architect and development coordinator in the VIATRA2 project
o In various other industrial projects too
3
![Page 4: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/4.jpg)
INTRODUCTION
Graph Transforma'ons from the VIATRA2 perspec've…
4
![Page 5: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/5.jpg)
VIATRA2 is… A graph transforma'on tool
o h1p://eclipse.org/gmt/VIATRA2o h1p://wiki.eclipse.org/VIATRA2o h1p://viatra.inf.mit.bme.hu
A projecto Lead by Dr. Dániel Varró at BUTE’s Fault Tolerant Systems Research Groupo Started in 1998 with the original VIATRAo Current tech: since 2004o Many PhD, grad and undergrad par'cipants over the yearso Current team: 2 seniors, 2 PhD candidates, 3 PhD students, 7 students
A technologyo Industrial development partner: OptXware Ltd.o Has been experimented with in various (academic and industrial) organiza'ons world-‐wide
5
![Page 6: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/6.jpg)
Applica'ons of VIATRA2
“Stand-‐alone” model transforma'ons
Early analysis (DECOS, SecureChange)
Integrated features for graphical DSMLs (ViatraDSM)
Model execu'on/analysis (stochas'c GT)
(Development) tool integra'on (DECOS, SENSORIA, MOGENTES)
Model op'miza'on / constraint solving (DIANA)
6
![Page 7: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/7.jpg)
7
Metamodel
Metamodeling
tkn3:tokens
a4:outarc
a1:inarc a2:outarc
a3:inarc
inarc
outarc11
* *
p1:Place p3:Place
t2:Transition
t1:Transition
to1:Token
Transition
Place
Instance model
Token
1
*
tokens
7
![Page 8: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/8.jpg)
7
Metamodel
Metamodeling
tkn3:tokens
a4:outarc
a1:inarc a2:outarc
a3:inarc
inarc
outarc11
* *
At most one
Arbitrary
p1:Place p3:Place
t2:Transition
t1:Transition
to1:Token
Transition
Place
Object
Link
Slot Instance model
Class
Association
Multiplicity constraint
Token
1
*
tokens
7
![Page 9: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/9.jpg)
Graph Transforma'on8
Phases of GT matching– Pattern Matching phase– Updating phase: delete+ create
LHS RHS
Place
Token
Tran. Placea1:inarc a2:outarc
Place
Token
Tran. Plana1:inarc a2:outarc
ttn1:tokens tkn2:tokens
8
![Page 10: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/10.jpg)
Graph Transforma'on8
Phases of GT matching– Pattern Matching phase– Updating phase: delete+ create
LHS RHS
Place
Token
Tran. Placea1:inarc a2:outarc
Place
Token
Tran. Plana1:inarc a2:outarc
ttn1:tokens tkn2:tokens
matching
8
![Page 11: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/11.jpg)
Graph Transforma'on8
Phases of GT matching– Pattern Matching phase– Updating phase: delete+ create
LHS RHS
Place
Token
Tran. Placea1:inarc a2:outarc
Place
Token
Tran. Plana1:inarc a2:outarc
ttn1:tokens tkn2:tokens
matching updating
Pattern Matching is the most critical issue from performance viewpoint
8
![Page 12: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/12.jpg)
Incremental model transforma'ons Key usage scenarios for MT:
o Mapping between languageso Intra-‐domain model manipula;on
• Model execu;on• Validity checking (constraint evalua;on)
They work with evolving models.o Users are constantly changing/modifying them.o Users usually work with large models.
Problem: transforma;ons are slowo To execute… (large models)o and to re-‐execute again and again (always star;ng from scratch).
Solu;on: incrementalityo Take the source model, and its mapped counterpart;o Use the informa;on about how the source model was changed;o Map and apply the changes (but ONLY the changes) to the target model.
9
![Page 13: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/13.jpg)
Towards incrementality
How to achieve incrementality?o Incremental updates: avoid re-‐genera'on.
• Don’t recreate what is already there.
• Use reference (correspondence) models.
o Incremental execu+on: avoid re-‐computa'on.• Don’t recalculate what was already computed.
• How?
10
![Page 14: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/14.jpg)
Towards incrementality
How to achieve incrementality?o Incremental updates: avoid re-‐genera'on.
• Don’t recreate what is already there.
• Use reference (correspondence) models.
o Incremental execu+on: avoid re-‐computa'on.• Don’t recalculate what was already computed.
• How?
10
![Page 15: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/15.jpg)
INCREMENTAL GRAPH PATTERN MATCHING
11
![Page 16: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/16.jpg)
Incremental graph pa1ern matching Graph transforma;ons require paQern matching
o Most expensive phase Goal: retrieve the matching set quickly How?
o Store (cache) matcheso Update them as the model changes
• Update precisely
Expected results: good, if…o There is enough memory (*)o Queries are dominanto Model changes are rela;vely sparse (**)o e.g. synchroniza;on, constraint evalua;on, …
12
![Page 17: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/17.jpg)
Opera'onal overview
XForm interpreter
VIATRA Model space
Incremental pa1ernmatcher
model manipula'on
event no'fica'on
pa1ern matching
updates
13
![Page 18: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/18.jpg)
p2
p1
p3
INPUT
Core idea: use RETE nets
RETE networko node: (par;al) matches of a
(sub)paQerno edge: update propaga;on
Demonstra;ng the principleo input: Petri neto paQern: fireable transi;ono Model change: new transi;on
(t3)
: Place
t2
: Transi'on: Token
t1
k2k1
k2k1
p1 p2 t2t1
p2, k2p1, k1
p1, k1, t1, p3
p1p2p3 t2t1
p1, k1, t1
p3
14
![Page 19: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/19.jpg)
p2
p1
p3
INPUT
Core idea: use RETE nets
RETE networko node: (par;al) matches of a
(sub)paQerno edge: update propaga;on
Demonstra;ng the principleo input: Petri neto paQern: fireable transi;ono Model change: new transi;on
(t3)
: Place
t2
: Transi'on: Token
t1
k2k1
k2k1
p1 p2 t2t1
p2, k2p1, k1
p1, k1, t1, p3
p1p2p3 t2t1
p1, k1, t1
p3Input nodes
Intermediate nodes
Model space
Produc;on node
14
![Page 20: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/20.jpg)
p2
p1
p3
INPUT
Core idea: use RETE nets
RETE networko node: (par;al) matches of a
(sub)paQerno edge: update propaga;on
Demonstra;ng the principleo input: Petri neto paQern: fireable transi;ono Model change: new transi;on
(t3)
: Place
t2
: Transi'on: Token
t1
k2k1
k2k1
p1 p2 t2t1
p2, k2p1, k1
p1, k1, t1, p3
p1p2p3 t2t1
p1, k1, t1
p3
14
![Page 21: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/21.jpg)
p2
p1
p3
INPUT
Core idea: use RETE nets
RETE networko node: (par;al) matches of a
(sub)paQerno edge: update propaga;on
Demonstra;ng the principleo input: Petri neto paQern: fireable transi;ono Model change: new transi;on
(t3)
: Place
t2
: Transi'on: Token
t1
k2k1
k2k1
p1 p2 t2t1
p2, k2p1, k1
p1, k1, t1, p3
t3p1p2p3 t2t1
t3
p1, k1, t1
p3
14
![Page 22: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/22.jpg)
p2
p1
p3
INPUT
Core idea: use RETE nets
RETE networko node: (par;al) matches of a
(sub)paQerno edge: update propaga;on
Demonstra;ng the principleo input: Petri neto paQern: fireable transi;ono Model change: new transi;on
(t3)
: Place
t2
: Transi'on: Token
t1
k2k1
k2k1
p1 p2 t2t1
p2, k2p1, k1
p1, k1, t1, p3
t3t3
t3t3 t3
p1p2p3 t2t1
t3
p1, k1, t1
p3
14
![Page 23: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/23.jpg)
p2
p1
p3
INPUT
Core idea: use RETE nets
RETE networko node: (par;al) matches of a
(sub)paQerno edge: update propaga;on
Demonstra;ng the principleo input: Petri neto paQern: fireable transi;ono Model change: new transi;on
(t3)
: Place
t2
: Transi'on: Token
t1
k2k1
k2k1
p1 p2 t2t1
p2, k2p1, k1
p1, k1, t1, p3
t3t3
t3t3 t3
p1p2p3 t2t1
t3
t3
p1, k1, t1
p3
t3
14
![Page 24: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/24.jpg)
p2
p1
p3
INPUT
Core idea: use RETE nets
RETE networko node: (par;al) matches of a
(sub)paQerno edge: update propaga;on
Demonstra;ng the principleo input: Petri neto paQern: fireable transi;ono Model change: new transi;on
(t3)
: Place
t2
: Transi'on: Token
t1
k2k1
k2k1
p1 p2 t2t1
p2, k2p1, k1
p2, k2, t3
p1, k1, t1, p3
t3t3
t3t3 t3
p1p2p3 t2t1
t3
t3
p1, k1, t1
p3
t3
p2, k2, t3
14
![Page 25: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/25.jpg)
p2
p1
p3
INPUT
Core idea: use RETE nets
RETE networko node: (par;al) matches of a
(sub)paQerno edge: update propaga;on
Demonstra;ng the principleo input: Petri neto paQern: fireable transi;ono Model change: new transi;on
(t3)
: Place
t2
: Transi'on: Token
t1
k2k1
k2k1
p1 p2 t2t1
p2, k2p1, k1
p2, k2, t3
p1, k1, t1, p3
t3t3
t3t3 t3
p1p2p3 t2t1
t3
t3
p1, k1, t1
p2, k2, t2, p3
p3
t3
p2, k2, t3
14
![Page 26: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/26.jpg)
RETE network construc'on Key: paQern decomposi;on
o PaQern = set of constraints (defined over paQern variables)o Types of constraints: type, topology (source/target), hierarchy (containment), aQribute value, generics (instanceOf/supertypeOf), injec+vity, [nega;ve] paQern calls, …
Construc;on algorithm (roughly)o 1. Decompose the paQern into elementary constraints (*)
o 2. Process the elementary constraints and connect them with appropriate intermediate nodes (JOIN, MINUS-‐JOIN, UNION, …)
o 3. Create terminator produc;on node
15
![Page 27: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/27.jpg)
Key RETE components
JOIN nodeo ~rela'onal algebra: natural join
MINUS-‐JOINo Nega've existence (NACs)
INPUTINPUT
INPUT
JOIN
JOIN
PRODUCTIONsourcePlace
16
![Page 28: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/28.jpg)
Suppor'ng a rich pa1ern language PaQern calls
o Simply connect the produc;on nodeso PaQern recursion is fully supported
OR-‐paQernso UNION intermediate nodes
Check condi;onso check (value(X) % 5 == 3)o check (length(name(X)) < 4)o check (myFunction(name(X))!=‘myException’)o Filter and term evaluator nodes
Result: full VIATRA transforma;on language support; any paQern can be matched incrementally.
17
![Page 29: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/29.jpg)
Updates Needed when the model space changes
VIATRA no;fica;on mechanism (EMF is also possible)o Transparent: user modifica;on, model imports, results of a transforma;on, external modifica;on, … RETE is always updated!
Input nodes receive elementary modifica;ons and release an update tokeno Represents a change in the par;al matching (+/-‐)
Nodes process updates and propagate them if neededo PRECISE update mechanism
18
![Page 30: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/30.jpg)
PART I: IMPACT ON PERFORMANCE
19
![Page 31: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/31.jpg)
20
Benchmarking in graph transforma;on
Specifica'on examples for GTo Goal: assessing expressivenesso UML-‐to-‐XMI, object-‐rela'onal mapping, UML-‐to-‐EJB, etc.
„Generic” Performance benchmarks for GTo Varró benchmark o R. Geiß and M. Kroll: On Improvements of the Varró Benchmark for Graph Transforma<on Tools
o (Ag<ve Tool Contest, Grabats ’08, …)
Our ICGT’08 paper:Benchmarks for graph transformaNon with incremental paPern matching
20
![Page 32: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/32.jpg)
Ini'al benchmarking: summary
Benchmarking scenarios focused on o “tradi'onal” MT use cases such as model synchroniza'on
o Addi'onal use cases where INC expected to be very good (e.g. model execu'on, constraint evalua'on)
Predictable near-‐linear growtho As long as there is enough memoryo Certain problem classes: constant execu'on 'me
Some benchmark example descrip'ons, specifica'ons, and measurement results available at:o hPp://wiki.eclipse.org/VIATRA2/Benchmarks
21
![Page 33: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/33.jpg)
Further improvements
Parallelismo Update the RETE network in parallel with the transforma'on
o Support parallel transforma'on execu'ono Paper at GT-‐VMT’09: Paralleliza'on of Graph Transforma'onBased on Incremental Pa1ern Matching
Hybrid pa1ern matchingo „mix” pa1ern matching strategies, use each at its besto Paper at ICMT’09: Efficient model transforma'ons by combining pa1ern matching strategies
22
![Page 34: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/34.jpg)
Parallelism: lessons learned
Can be good AND bad for performance
Cannot be easily automated
VERY problem specific!
Ongoing research: true parallel GT execu'ono “High” level specifica'on language
• Supports locking and synchroniza'on
23
![Page 35: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/35.jpg)
Hybrid PM: lessons learned
Paper for a special issue in STTT’09: Experimental assessment of combining pa1ern matching strategies with VIATRA2
In short: you may get a linear decrease in memory for a linear increase in execu'on 'me retains complexity class characteris'cs
24
![Page 36: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/36.jpg)
EMF-‐INCQUERY
Incremental pa1ern matching beyond VIATRA2
25
![Page 37: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/37.jpg)
Mo'va'on VIATRA2 has lacked generic EMF support
o Even though UML2 and some other EMF-‐based languages are supported
o This is expected to change (for the be1er) with Release4
Main reason: large conceptual gaps between model representa'ono N-‐level metamodeling (VPM) vs. MOF-‐style (EMF)o Graph structures (VPM) vs. A1ributed nodes (EMF)o Dynamic typing (VPM) vs. Sta'c (genera've) types (EMF)o Mul'-‐domain modelspace (VPM) vs. domain-‐specific resources (EMF)
26
![Page 38: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/38.jpg)
Goals
EMF lacks a fast query engineoMDT-‐OCL, EMF Query do not scale well
Goal 1: port incremental PM technology to EMFo Fast, effec've model querieso Easy-‐to-‐use declara've API (graph pa1erns)o Support for on-‐the-‐fly constraint evalua'on and other scenarios
Goal 2: generic EMF support for VIATRA2 R3.xo Reflec've and genera've model export-‐import
27
![Page 39: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/39.jpg)
EMF-‐IncQuery: overview At run'me
o Incremental pa1ern matching engine for EMF modelso Uses the EMF No'fica'on API for updateso Works with any transac'on-‐based EMF applica'on
• Minor modifica'on necessary
o Supports generic as well as parameterized model querieso Supports VIATRA2’s rich declara've pa1ern language (recursion, composi'on, a1ribute constraints…)
o No VIATRA2 dependency
As toolingo Genera've: compiles queries into RETE networks (vs. VIATRA2’s dynamic approach)
o Relies on VIATRA2 for query development and debugging
28
![Page 40: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/40.jpg)
Architecture
29
![Page 41: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/41.jpg)
Evalua'on
(To be) Presented at MODELS2010o Gábor Bergmann, Ákos Horváth, István Ráth, Dániel Varró: Incremental Evalua'on of Model Queries over EMF Models
o Case study: AUTOSAR on-‐the-‐fly constraint evalua'on• Simple and complex constraints
o Compared to:• MDT-‐OCL
• EMF Query
• Ad-‐hoc Java implementa'on
30
![Page 42: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/42.jpg)
Performance
31
![Page 43: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/43.jpg)
Summary Several orders of magnitude faster than the state-‐of-‐the-‐arto Constraint re-‐evalua'on: nearly constant execu'on 'mes
Price of performance: memory consump'ono Grows linearly with the model size (even for complex constraints)
o To be addressed in future work
Addi'onal future worko Accept (a subset of) OCL as input for queries
Will be available as open sourceo h1p://viatra.inf.mit.bme.hu/models10
32
![Page 44: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/44.jpg)
Future research targets
Keyword #1: scalabilityoModel sizeoQuery/transforma;on complexityoReal ;me applica;ons
Keyword #2: teamworkoCo-‐opera;ve modeling repositoriesoConflict detec;on & resolu;on
33
![Page 45: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/45.jpg)
Scalability challenges VIATRA2, EMF, … hit the limit at a few (1-‐5) million model elements in-‐memoryo Compiled model transformers do somewhat be1er on synthe+c benchmarks
o But: compiled approaches reduce flexibility (dynamic typing, untyped elements, mul'-‐domain modeling, …)
Increasing demand for handling really large modelso AUTOSAR: 15-‐30+ million elements
Transforma'on complexity is growingo As MT gains more industrial exposureo Problem areas: re-‐use (libraries), design pa1erns, execu'on infrastructure, specifica'on complexity …
34
![Page 46: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/46.jpg)
Teamwork (and other) challenges
Painful problems in DSML prac'ceoMetamodel evolu'onoModel comparison, versioning, conflict resolu'on, merge
o Concurrent teamwork, collabora've modeling, …
EMF lags behindo Even though the EMF ecosystem is huge
Many of these problems are VERY hard (to solve well)o No “silver bullet” solu'on yet
35
![Page 47: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/47.jpg)
Our vision
Model space
Par''on (EMF)
Par''on (UML)
Par''on (RDF)
Na'vepersistence
Na'vepersistence
Na'vepersistence
High performance transforma'on engineINC
Collabora've
client-‐server access
Language adaptersVIATRA ATL QVT
36
![Page 48: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/48.jpg)
Our vision
Model space
Par''on (EMF)
Par''on (UML)
Par''on (RDF)
Na'vepersistence
Na'vepersistence
Na'vepersistence
High performance transforma'on engineINC
Collabora've
client-‐server access
Language adaptersVIATRA ATL QVT
No need for model export-‐
import
36
![Page 49: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/49.jpg)
Our vision
Model space
Par''on (EMF)
Par''on (UML)
Par''on (RDF)
Na'vepersistence
Na'vepersistence
Na'vepersistence
High performance transforma'on engineINC
Collabora've
client-‐server access
Language adaptersVIATRA ATL QVT
No need for model export-‐
import
Out-‐of-‐memory model storage
36
![Page 50: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/50.jpg)
Our vision
Model space
Par''on (EMF)
Par''on (UML)
Par''on (RDF)
Na'vepersistence
Na'vepersistence
Na'vepersistence
High performance transforma'on engineINC
Collabora've
client-‐server access
Language adaptersVIATRA ATL QVT
No need for model export-‐
import
Acts as a “common run'me”
Out-‐of-‐memory model storage
36
![Page 51: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/51.jpg)
Our vision
Model space
Par''on (EMF)
Par''on (UML)
Par''on (RDF)
Na'vepersistence
Na'vepersistence
Na'vepersistence
High performance transforma'on engineINC
Collabora've
client-‐server access
Language adaptersVIATRA ATL QVT
No need for model export-‐
import
Acts as a “common run'me”
Supports distributed models and versioning
Out-‐of-‐memory model storage
36
![Page 52: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/52.jpg)
Our vision
Model space
Par''on (EMF)
Par''on (UML)
Par''on (RDF)
Na'vepersistence
Na'vepersistence
Na'vepersistence
High performance transforma'on engineINC
Collabora've
client-‐server access
Language adaptersVIATRA ATL QVT
No need for model export-‐
import
Acts as a “common run'me”
Supports distributed models and versioning Adap've IPM
engine (lower memory footprint)
Out-‐of-‐memory model storage
36
![Page 53: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/53.jpg)
PART II: BEYOND PERFORMANCE
37
![Page 54: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/54.jpg)
What really is INC?
Graph pa1erns ~ complex constraint setsoMatching set: overlay of “valid” elements
RETE: a complex overlay cache of the model grapho Stores matching sets explicitly
A historical cacheo Can store “previous” overlay states
38
![Page 55: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/55.jpg)
What really is INC?
Graph pa1erns ~ complex constraint setsoMatching set: overlay of “valid” elements
RETE: a complex overlay cache of the model grapho Stores matching sets explicitly
A historical cacheo Can store “previous” overlay states
38
![Page 56: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/56.jpg)
Idea Use the historical overlay to
o Easily compute non-‐trivial change informa'on (beyond elementary changes)
• Detect high-‐level “events”
• Detect (the net effect of) complex change sequences
o Assign ac'ons to these changes• Event-‐Condi'on-‐Ac'on formalism in rule-‐based systems• event-‐driven transforma'ons
o Or, easily track the validity set of constraints as the model is evolving
• constraint sa'sfac'on solving
o Or, easily track enabledness condi'ons for simula'on rules• stochas'c model simula'on
39
![Page 57: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/57.jpg)
Event-‐driven live transforma'ons Represent events as changes in the matching set of a paQern.o More general than previous approaches
Live transforma;onso maintain the context (variable values, global variables, …);
o run as a “daemon”, react whenever necessary;
o as the models change, the system can react instantly, since everything needed is there in the RETE network: no re-‐computa+on is necessary.
Paper at ICMT2008: Live model transforma;ons driven by incremental paQern matching.
40
![Page 58: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/58.jpg)
Change-‐driven transforma'ons Generaliza'on of the live transforma'on approach
New formalism supports more precise change querieso To dis'nguish between various execu'on trajectories that result in the same net change
o A more complete adapta'on of the ECA approach to graph transforma'ons
Applica'on scenarioso Incremental model synchroniza'on for non-‐materialized models (paper at MODELS2009: Change-‐driven transforma'ons)
o Back-‐annota'on of simula'on traces (paper at SEFM2010)
41
![Page 59: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/59.jpg)
Constraint sa'sfac'on over models Mo'va'on
o Very pragma'c problem area: constraint evalua'on, “quick fix” computa'on, design or configura'on-‐space explora'on etc.
o CLP(FD) limited: only finite problems can be formulated• Problema'c: object birth-‐and-‐death
o CLP(FD) difficult to use
GT-‐based approacheso Easier-‐to-‐use formalismo Infinite (dynamic) problemso “Tradi'onal” problem: scalability
Our aim: combine ease-‐of-‐use and flexibility with performance
42
![Page 60: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/60.jpg)
CSP(M)
Described by (M0,C,G,L)oM0 ini+al model (typed graph)o C set of global constraints (graph pa1erns)o G set of goals (graph pa1erns)o L set of labeling rules (GT rules)
Goalo Find a model Ms which sa'sfies all global constraints and goals.
• One model• All model
• Op'mal model
43
![Page 61: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/61.jpg)
Implementa'on with VIATRA2
Incremental constraint evalua'on by incremental pa1ern matchingo Cached matchingso Incrementally updated
Simple state space representa'on
Typed graph comparisono DSMDiFF
Backtrackingo Transac'ons of atomic manipula'on opera'ons
44
![Page 62: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/62.jpg)
Extensions of CSP(M)
Flexible CSPo Strong-‐weak constraintso Quality metrics
Dynamic CSPo Add-‐remove on-‐the-‐fly
• Global constraints
• Goals
• Labeling rules
o Re-‐use already traversed state space
45
![Page 63: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/63.jpg)
Current state of CSP(M)
“Pure” and “Flexible” CSP(M)o Execu'on 'mes significantly faster than KORAT or GROOVE
Dynamic CSP(M)oWe can even beat Prolog CLP(FD) in some cases
In generalo VERY problem specific resultso Lots of poten'al for future research
46
![Page 64: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/64.jpg)
Stochas;c simula;on by graph transforma;on
Joint work with Reiko Heckel’s group (University of Leicester)
Paperso FASE2010: P. Torrini, R. Heckel, I. Ráth: Stochas'c simula'on of graph transforma'on systems
o GT-‐VMT2010: P. Torrini, R. Heckel, I. Ráth, G. Bergmann: Stochas'c graph transforma'on with regions
o ASMTA2010: A. Khan, P. Torrini, R. Heckel and I. Ráth: Model-‐based stochas'c simula'on of P2P VoIP using graph transforma'on
47
![Page 65: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/65.jpg)
Idea
Conceptually similar to the CSP engineo Simula'on rule: precondi'on + transforma'on
o Assign probability distribu'ons to simula'on rules
o Execute as-‐long-‐as-‐possibleo Observe system state changes
48
![Page 66: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/66.jpg)
Applica'ons
VoIP P2P network behavioral simula'on (Skype)
Other simula'on scenarioso Social networkso Traffic networkso Crystal growth
Evalua'ono +: Scales well (comparable to dedicated simulators)o +: GT is an easier-‐to-‐use specifica'on formalism
49
![Page 67: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/67.jpg)
FUTURE
50
![Page 68: Applications of incremental pattern matching in model transformations](https://reader034.fdocuments.net/reader034/viewer/2022051612/54bfd0884a79594a748b4722/html5/thumbnails/68.jpg)
Research topics overview Transforma'on engineering
o Change-‐driven transforma'onso Sta'c program checking for transforma'onso Re-‐factoring and composi'on (transforma'on chains)o Model transforma'on by exampleo Semiautoma'c back-‐annota'on for dynamic languageso From OCL to graph pa1erns and back
New applica'onso Stochas'c simula'ono CSP(M)
Scalabilityo Very large modelso Collabora've model repositorieso Op'miza'on of the IPM engine
51