Gremlin-ATL: a Scalable Model Transformation Framework

Gremlin-ATL: a Scalable Model Transformation Framework

Gwendal DANIELInria, IMT Atlantique, LS2N

[email protected]

Frédéric JOUAULTESEO

[email protected]

Gerson SUNYEInria, IMT Atlantique, LS2N

[email protected]

Jordi CABOTICREA - UOC

[email protected]

Introduction

• Complex and large models• Civil Engineering

• Biology

• Reverse Engineering

• Need to provide solutions to• Store large models

• Efficiently query them

• Compute complex transformations

2G. Daniel - Gremlin-ATL: a Scalable Model Transformation Framework

Model Persistence

• Default serialization mechanism: XMI

• Scalable model persistence frameworks

• Use databases to store models

• Relational: CDO

• NoSQL: Morsa, NeoEMF

• Low memory footprint


Modeling API

ModelingFramework

DatabaseModeler

Model Persistence

API Call1…

API Calln

ModelTransformation

TransformationEngine

ModelingFramework

Database

rule myRule {from pp : In!Personto ee: Out!Entity (

name <- p.name)

}

get(pp1)get(pp1, name)create(ee1, Entity)set(ee1, name)…get(ppn)get(ppn, name)create(een, Entity)set(een, name)

Typical Model Transformation Tool

Under the hood

4

Low-level modeling API→ Not aligned with thedatabase capabilities

Fragmented queries→ Not efficient→ Remote database

Intermediate objects→ Memory consumption→ Execution time overhead

G. Daniel - Gremlin-ATL: a Scalable Model Transformation Framework

Model Persistence

• Not efficient to compute model queries and transformations

• Why can’t we write database queries manually?• Modern persistence frameworks typically rely on NoSQL databases

• Multiple query languages, data representation, etc

• Low-level queries are hard to understand and maintain

• Modeling expertise vs. Database expertise

• Solution: generate them!


Gremlin-ATL• Generate database queries from model transformations

• Bypass modeling framework APIs

• Benefit of all the query capabilities of the backend

6

ATLtoGremlinTransformation

GremlinTraversal

DatabaseATL Transformation


rule myRule {from pp : In!Personto ee: Out!Entity (

name <- pp.name)

}

g.idx(‘’metaclasses’’)[[name:Person]].transform{ee = g.createVertex(Entity);e.addEdge(‘’instanceof’’,

g.idx(‘’metaclasses’’)[[name:Entity]]ee.name = it.name

}.iterate();Database

Gremlin-ATLTransformation Tool

Under the hood

Gremlin-ATL

• Input: ATL

• Well-known in the modelingcommunity

• Hybrid language

• Transformation rules

• Embeds OCL

rule myRule {from pp : In!Person (

not pp.name.isEmpty())to ee : Out!Entity(

name ← pp.name}


Gremlin-ATL

• Output: Gremlin

• Multi-database query language

• Graph traversals

• Easily extensible

• Native Neo4j support, OrientDB …

• NeoEMF/Graph

g.idx(‘metaclasses’)[[name:Person]].inE(‘instanceOf’).outV.filter{!it.name.isEmtpy()}.transform{

ee = g.createVertex();ee.addEdge(‘instanceOf’, g.idx(‘metaclasses’)[[name:Entity]]); ee.name = it.name;

}.iterate()


rule myRule {from pp : In!Person (

not pp.name.isEmpty())to ee : Out!Entity(

name ← pp.name)

}

Gremlin-ATL

• 3 steps process• Map ATL constructs to their Gremlin equivalent

• Merge the created Gremlin constructs into a script

• Send the script to the database

9

g.idx(‘metaclasses’)[[name:Person]].inE(‘instanceOf’).outV

.filter{!it.name.isEmtpy()}

.transform{ee = g.createVertex();ee.addEdge(‘instanceOf’, g.idx(‘metaclasses’)[[name:Entity]]);

ee.name = it.name;}.iterate()


Evaluation

10

0

50000

100000

150000

200000

250000

300000

set1 set2 set3 set4

Java2KDM Execution Time (ms)

ATL Gremlin-ATL

OutOfMemory


Models containing 6000 to 3,5M elementsJava2KDM TransformationNeoEMF/Graph

Evaluation


Models containing 6000 to 3,5M elementsJava2KDM TransformationNeoEMF/Graph

0

100

200

300

400

500

600

set1 set2 set3 set4

Java2KDM Memory Consumption (MB)

ATL Gremlin-ATL

OutOfMemory

Conclusion

• Benefits

• No query fragmentation / intermediate objects

• Positive results on large models

• Extensible architecture (see our article)

• Drawbacks

• Less efficient than existing solutions for small / in-memory models

• Need to be manually integrated

• Tightly coupled to the model persistence framework


Conclusion

• Future Work• Extend our mapping

• Reuse schema inferrence approaches• Ease the integration of other persistence frameworks / data sources

• Gremlin-ATL to express data migration operations• Promising results on the Neo4j panama paper database (see our article)


Question?

Thank you for your attention!

Websites / RepositoriesNeoEMF: neoemf.comGremlin-ATL: github.com/atlanmod/mogwai

https://github.com/SOM-Research

https://github.com/AtlanMod


Gremlin-ATL: a Scalable Model Transformation Framework

Presentations & Public Speaking

Transcript of Gremlin-ATL: a Scalable Model Transformation Framework