Why we built Kundera - The Polyglot Object Mapper for NoSQLs?
-
Upload
vivekshri -
Category
Technology
-
view
191 -
download
0
description
Transcript of Why we built Kundera - The Polyglot Object Mapper for NoSQLs?
![Page 1: Why we built Kundera - The Polyglot Object Mapper for NoSQLs?](https://reader036.fdocuments.net/reader036/viewer/2022062614/545af11fb1af9fc0638b5e4a/html5/thumbnails/1.jpg)
© 2014 Impetus Technologies - Confidential1
Impetus Technologies Inc.
Why we built Kundera - The Polyglot Object Mapper for
NoSQLs?Vivek Shrivastava
software architect
![Page 2: Why we built Kundera - The Polyglot Object Mapper for NoSQLs?](https://reader036.fdocuments.net/reader036/viewer/2022062614/545af11fb1af9fc0638b5e4a/html5/thumbnails/2.jpg)
© 2014 Impetus Technologies - Confidential2
Evolution of Data
![Page 3: Why we built Kundera - The Polyglot Object Mapper for NoSQLs?](https://reader036.fdocuments.net/reader036/viewer/2022062614/545af11fb1af9fc0638b5e4a/html5/thumbnails/3.jpg)
© 2014 Impetus Technologies - Confidential3
10 LET A = 1
20 LET B = 2
30 LET C = A + B
40 PRINT C
50 END
Data
First Introduction To Data
![Page 4: Why we built Kundera - The Polyglot Object Mapper for NoSQLs?](https://reader036.fdocuments.net/reader036/viewer/2022062614/545af11fb1af9fc0638b5e4a/html5/thumbnails/4.jpg)
© 2014 Impetus Technologies - Confidential4
Relational Databases
• Tables
• Relations
• SQL
• Transactions
• Normalization
![Page 5: Why we built Kundera - The Polyglot Object Mapper for NoSQLs?](https://reader036.fdocuments.net/reader036/viewer/2022062614/545af11fb1af9fc0638b5e4a/html5/thumbnails/5.jpg)
© 2014 Impetus Technologies - Confidential5
Current Landscape
http://blogs.the451group.com/information_management/2014/03/18/updated-data-platforms-landscape-map-february-2014/
![Page 6: Why we built Kundera - The Polyglot Object Mapper for NoSQLs?](https://reader036.fdocuments.net/reader036/viewer/2022062614/545af11fb1af9fc0638b5e4a/html5/thumbnails/6.jpg)
© 2014 Impetus Technologies - Confidential6
Infrastructure
Datastores Designed To Support Partitions
![Page 7: Why we built Kundera - The Polyglot Object Mapper for NoSQLs?](https://reader036.fdocuments.net/reader036/viewer/2022062614/545af11fb1af9fc0638b5e4a/html5/thumbnails/7.jpg)
© 2014 Impetus Technologies - Confidential7
Relationships / Transaction
Datastores Designed To Support Aggregates
Item 1
Order
Item 2
![Page 8: Why we built Kundera - The Polyglot Object Mapper for NoSQLs?](https://reader036.fdocuments.net/reader036/viewer/2022062614/545af11fb1af9fc0638b5e4a/html5/thumbnails/8.jpg)
© 2014 Impetus Technologies - Confidential8
Consistency
DC1
N1
DC2
N2
DC3
N3
Transactional Consistency
Replication Consistency
T=0, user1 fetches O
T=1, user2 fetches O
T=2, user1 edits and saves O
T=3, user2 edits and saves O
![Page 9: Why we built Kundera - The Polyglot Object Mapper for NoSQLs?](https://reader036.fdocuments.net/reader036/viewer/2022062614/545af11fb1af9fc0638b5e4a/html5/thumbnails/9.jpg)
© 2014 Impetus Technologies - Confidential9
Consistency
Availability
Partition Tolerance
Not Possibl
e
CAP Theorem
Datastores Designed To Support Eventual Consistency
![Page 10: Why we built Kundera - The Polyglot Object Mapper for NoSQLs?](https://reader036.fdocuments.net/reader036/viewer/2022062614/545af11fb1af9fc0638b5e4a/html5/thumbnails/10.jpg)
© 2014 Impetus Technologies - Confidential10
Shape
Datastores Designed To Support Richer Data Models
1 David 34
2 John 46
3 Lisa 32
4 Steve 46
5 JP 49
Tables
Graph
Columnar
Key-Value
Document
![Page 11: Why we built Kundera - The Polyglot Object Mapper for NoSQLs?](https://reader036.fdocuments.net/reader036/viewer/2022062614/545af11fb1af9fc0638b5e4a/html5/thumbnails/11.jpg)
© 2014 Impetus Technologies - Confidential11
Cluster/Cloud FriendlySchema less Aggregates With
Redefined Transactions
Eventual ConsistencyRicher Data Model With Minimum Impedance
Mismatch
NoSQL
Finally
![Page 12: Why we built Kundera - The Polyglot Object Mapper for NoSQLs?](https://reader036.fdocuments.net/reader036/viewer/2022062614/545af11fb1af9fc0638b5e4a/html5/thumbnails/12.jpg)
© 2014 Impetus Technologies - Confidential12
Emergence of a Polyglot World
![Page 13: Why we built Kundera - The Polyglot Object Mapper for NoSQLs?](https://reader036.fdocuments.net/reader036/viewer/2022062614/545af11fb1af9fc0638b5e4a/html5/thumbnails/13.jpg)
© 2014 Impetus Technologies - Confidential13
A Single Application Using Multiple Data Storage Technologies
Polyglot Persistence
![Page 14: Why we built Kundera - The Polyglot Object Mapper for NoSQLs?](https://reader036.fdocuments.net/reader036/viewer/2022062614/545af11fb1af9fc0638b5e4a/html5/thumbnails/14.jpg)
© 2014 Impetus Technologies - Confidential14
Data streams
Cassandra, HBase
User information
RDBMS
In memory cache/session
Redis
Orders / Articles / Tickets
MongoDB, Couchbase
User Social graph
Neo4J, Titan
Batch processing & analytics
Hadoop, Spark
Polyglot World
![Page 15: Why we built Kundera - The Polyglot Object Mapper for NoSQLs?](https://reader036.fdocuments.net/reader036/viewer/2022062614/545af11fb1af9fc0638b5e4a/html5/thumbnails/15.jpg)
© 2014 Impetus Technologies - Confidential15
Decision Implementation
Is It Easy?
No. It’s A Bumpy Ride.
![Page 16: Why we built Kundera - The Polyglot Object Mapper for NoSQLs?](https://reader036.fdocuments.net/reader036/viewer/2022062614/545af11fb1af9fc0638b5e4a/html5/thumbnails/16.jpg)
© 2014 Impetus Technologies - Confidential16
Decision
Degree of Polyglot? • X% RDBMS• Y% NoSQLComposition of Polyglot? • Datastores?
Type of Polyglot?
• Don’t choose for pride, passion and trends
• Don’t start with more than 1 NoSQL store
• Don’t start with 100% uses cases
![Page 17: Why we built Kundera - The Polyglot Object Mapper for NoSQLs?](https://reader036.fdocuments.net/reader036/viewer/2022062614/545af11fb1af9fc0638b5e4a/html5/thumbnails/17.jpg)
© 2014 Impetus Technologies - Confidential17
Degree of Polyglot
Scale Relational Behavior RDBMS Non-RDBMS
Low Any 100% 0%
High High 90% 10% (New SQL)
High Moderate 75% 25% (NoSQL)
High Low 50% 50-100% (NoSQL)
![Page 18: Why we built Kundera - The Polyglot Object Mapper for NoSQLs?](https://reader036.fdocuments.net/reader036/viewer/2022062614/545af11fb1af9fc0638b5e4a/html5/thumbnails/18.jpg)
© 2014 Impetus Technologies - Confidential18
RDBMS NoSQL NewSQL
Datastore + Cache Local + Global Client + Server
Composition of Polyglot - Forms
![Page 19: Why we built Kundera - The Polyglot Object Mapper for NoSQLs?](https://reader036.fdocuments.net/reader036/viewer/2022062614/545af11fb1af9fc0638b5e4a/html5/thumbnails/19.jpg)
© 2014 Impetus Technologies - Confidential19
Composition of Polyglot - Datastores
Neo4j, Titan, Objectivity
Cassandra, HBase, BigTable
Oraclekv, Redis, Riak
MongoDB, Couch
base
Graph Columnar
Key-value
Document
Data Model / QueriesConsistency / Performance
Architecture / Non-Functional
![Page 20: Why we built Kundera - The Polyglot Object Mapper for NoSQLs?](https://reader036.fdocuments.net/reader036/viewer/2022062614/545af11fb1af9fc0638b5e4a/html5/thumbnails/20.jpg)
© 2014 Impetus Technologies - Confidential20
Operating In The Polyglot World
![Page 21: Why we built Kundera - The Polyglot Object Mapper for NoSQLs?](https://reader036.fdocuments.net/reader036/viewer/2022062614/545af11fb1af9fc0638b5e4a/html5/thumbnails/21.jpg)
© 2014 Impetus Technologies - Confidential21
Weak Polyglot
Data LayerReporting Lane
(MySQL)
Orders Lane(MongoDB)
Log Lane(Cassandra)Se
rvic
e La
yer
Web
Lay
er
![Page 22: Why we built Kundera - The Polyglot Object Mapper for NoSQLs?](https://reader036.fdocuments.net/reader036/viewer/2022062614/545af11fb1af9fc0638b5e4a/html5/thumbnails/22.jpg)
© 2014 Impetus Technologies - Confidential22
Strong Polyglot
Data Layer
Polyglot MapperReports(MySQL)
+Orders
(MongoDB)+
Logs(Cassandra)Se
rvic
e La
yer
Web
Lay
er
![Page 23: Why we built Kundera - The Polyglot Object Mapper for NoSQLs?](https://reader036.fdocuments.net/reader036/viewer/2022062614/545af11fb1af9fc0638b5e4a/html5/thumbnails/23.jpg)
© 2014 Impetus Technologies - Confidential23
Harmonization
Query / Indexing
Operational
Mapping / Interaction
CRUD
Atomicity
Switching / Migrating
![Page 24: Why we built Kundera - The Polyglot Object Mapper for NoSQLs?](https://reader036.fdocuments.net/reader036/viewer/2022062614/545af11fb1af9fc0638b5e4a/html5/thumbnails/24.jpg)
© 2014 Impetus Technologies - Confidential24
And We Built Kundera
![Page 25: Why we built Kundera - The Polyglot Object Mapper for NoSQLs?](https://reader036.fdocuments.net/reader036/viewer/2022062614/545af11fb1af9fc0638b5e4a/html5/thumbnails/25.jpg)
© 2014 Impetus Technologies - Confidential25
What It Is?
?Object Mapper• RDBMS
• NoSQLs (Cassandra, MongoDB, Hbase, Redis, Oracle NoSQL, CouchDB, Neo4j)
JPA Compliant
Polyglot
![Page 26: Why we built Kundera - The Polyglot Object Mapper for NoSQLs?](https://reader036.fdocuments.net/reader036/viewer/2022062614/545af11fb1af9fc0638b5e4a/html5/thumbnails/26.jpg)
© 2014 Impetus Technologies - Confidential26
JPA
Customer(MySQL)
Order 1 (MongoDB)
{Item11, Item12}
Order 2 (MongoDB){Item21, Item22}
One to
many
Aggregate
![Page 27: Why we built Kundera - The Polyglot Object Mapper for NoSQLs?](https://reader036.fdocuments.net/reader036/viewer/2022062614/545af11fb1af9fc0638b5e4a/html5/thumbnails/27.jpg)
© 2014 Impetus Technologies - Confidential27
<!-- Persistence unit for Cassandra persistence --><persistence-unit name=“cassandrapu"> <provider>com.impetus.kundera.KunderaPersistence</provider> <class>com.impetus.kvapps.entities.Order</class> <exclude-unlisted-classes>true</exclude-unlisted-classes> <properties> <property name="kundera.nodes" value="localhost" /> <property name="kundera.port" value="9160" /> <property name="kundera.keyspace" value=“orderstore" /> <property name="kundera.dialect" value="cassandra" /> <property name="kundera.client.lookup.class" value="com.impetus.client.cassandra.thrift.ThriftClientFactory" /> <property name="kundera.ddl.auto.prepare" value="create" /> <property name="index.home.dir" value="lucene"/> </properties></persistence-unit>
<!-- Persistence unit for Mongo persistence --><persistence-unit name=“mongopu"> <provider>com.impetus.kundera.KunderaPersistence</provider> <class>com.impetus.kvapps.entities.Invoice</class><exclude-unlisted-classes>true</exclude-unlisted-classes> <properties> <property name="kundera.nodes" value="localhost" /> <property name="kundera.port" value="27017" /> <property name="kundera.keyspace" value=“invoicestore" /> <property name="kundera.dialect" value="mongodb" /> <property name="kundera.client.lookup.class" value="com.impetus.client.mongodb.MongoDBClientFactory" /> <property name="kundera.ddl.auto.prepare" value="create" /></properties> </persistence-unit>
//create entity manager factory. EntityManagerFactory emf = Persistence.createEntityManagerFactory(“cassandrapu,mongopu”, properties); EntityManager em = emf.createEntityManager(); ….. em.persist(order);
Configuration : Persistence.xml
Persist Data
At Its Simplest
![Page 28: Why we built Kundera - The Polyglot Object Mapper for NoSQLs?](https://reader036.fdocuments.net/reader036/viewer/2022062614/545af11fb1af9fc0638b5e4a/html5/thumbnails/28.jpg)
© 2014 Impetus Technologies - Confidential28
Supports all major datastores and
Extensible
Polyglot CRUD / Query / Object
Mapping
Schema Generation /
Atomicity / Lazy and Eager fetching
REST Based AccessInterceptors /
Events / Caching / Connection Pool
JPQL / Native Query / Fallback
Indexing Flexibility
Additionally
![Page 29: Why we built Kundera - The Polyglot Object Mapper for NoSQLs?](https://reader036.fdocuments.net/reader036/viewer/2022062614/545af11fb1af9fc0638b5e4a/html5/thumbnails/29.jpg)
© 2014 Impetus Technologies - Confidential29
Commitment to the Polyglot
Kundera.js (Polyglot JavaScript Persister)
Polyglot Data Viewer
Support for more datastores
![Page 30: Why we built Kundera - The Polyglot Object Mapper for NoSQLs?](https://reader036.fdocuments.net/reader036/viewer/2022062614/545af11fb1af9fc0638b5e4a/html5/thumbnails/30.jpg)
© 2014 Impetus Technologies - Confidential30
Thank you.Questions??
https://github.com/impetus-opensource/Kundera