Applying Real-time SQL Changes in your Hazelcast Data Grid

Speedment and Hazelcast

For Automatic Scale out of Existing SQL Databases with Hazelcast

Per-Åke Minborg www.speedment.com

The Shared Database Challenge

Database

Hazelcast

Hazelcast Application Hazelcast Application

Hazelcast Application


Database

Hazelcast

Hazelcast Application 3:rd Party Application 3:rd Party Application

3:rd Party Application Hazelcast Application



Database

Speedment

Hazelcast

Hazelcast Application 3:rd Party Application 3:rd Party Application

3:rd Party Application Hazelcast Application


Overview

Database Speedment

Hazelcast Node

Hazelcast Node

Hazelcast Node

Hazelcast Node

INSERT UPDATE DELETE

Reflects all Data with Hazelcast Exising SQL data from start System.out.println(hazelcastMap.size()); -> 300024 Inserts sql> insert into employees (emp_no, birth_date, first_name, last_name, gender, hire_date) values (10001, '1953-09-02', 'Georgi', 'Facello', 'M', '1986-06-26'); Employees emp = hazelcastMap.get("10001"); System.out.println(emp); -> Employees { "emp_no": 10001, "birth_date": "1953-09-02", "first_name": "Georgi", "last_name": "Facello", "gender": "M", "hire_date": "1986-06-26"} Updates sql> update employees set first_name="Georgi2" where emp_no=10001; Employees emp = hazelcastMap.get("10001"); System.out.println(emp); -> Employees { "emp_no": 10001, "birth_date": "1953-09-02", "first_name": "Georgi2", "last_name": "Facello", "gender": "M", "hire_date": "1986-06-26"} Deletes sql> delete from employees where emp_no = 10001; Employees emp = hazelcastMap.get("10001"); System.out.println(emp); -> null

Works with Transactions

SQL> START TRANSACTION; update employees set first_name="Duangkaew"

where emp_no=10010; update salaries set salary=salary+10 where id=12322;

COMMIT; // Backing hazelcast maps are updated atomically Employees emp = employeesMap.get("10001"); Salaries sal = salaryMap.get("12322");

How to Install

•  Download software •  Unzip distribution file •  Start scripts in ”bin” directory

Generated Code

§  Database triggers §  Triggerlog §  Java POJOs for each selected table §  Table manager/DAO §  Hazelcast serialization factories §  Hazelcast config support §  Relational database metadata §  Transaction support wrappers

POJO Representing a Row

public class Employees extends Struct<Employees> implements IdentifiedDataSerializable {

private Integer empNo;

private Date birthDate;

private String firstName;

private String lastName;

private String gender;

private Date hireDate;

// Accessors, serialization, deserialization, etc.

@Override

public int getFactoryId() {

return EmployeesDataSerializableFactory.FACTORY_ID;

}

@Override

public void writeData(ObjectDataOutput out) throws IOException {

int sizeOf = (int) sizeOf();

final ByteBuffer bb = ByteBuffer.allocate(sizeOf + 4);

bb.putInt(sizeOf);

writeObject(bb);

out.write(bb.array());

}

POJO Representing a Row

POJO Representing a Row @Override

public void readData(ObjectDataInput in) throws IOException {

int sizeOf = in.readInt();

final ByteBuffer bb = ByteBuffer.allocate(sizeOf);

in.readFully(bb.array());

try {

readObject(bb);

} catch (final ClassNotFoundException cnfe) {

_logger.error(cnfe.getMessage(), cnfe);

}

}

}

Manager/DAO Representing a Table

public class EmployeesMgr extends Manager<Employees> {

@Override

public Employees getNewObjectInstance() {

return new Employees();

}

// insert, update, delete, finders, size, etc.


@Override

public void add(final Employees employees) {

addHazelcast(employees);

}

@Override

public void remove(final Employees employees) {

removeHazelcast(employees);

}

}


protected void addHazelcast(final Employees employees) {

final String _key = employees._getPrimaryKeyString();

final TransactionalMap<String, Employees> transactionMap = getHazelcastTransactionalMap();

if (transactionMap != null) {

transactionMap.put(_key, employees);

} else {

hcPrimaryKeyMap.put(_key, employees);

}

}

DataSerializableFactory per Table

public class EmployeesDataSerializableFactoryBase implements DataSerializableFactory {

public static final int FACTORY_ID = 1221169916;

public static final int EMPLOYEES_ID = 0;

@Override

public IdentifiedDataSerializable create(int typeId) {

return EmployeesMgr.getInstance().getNewObjectInstance();

}

}

Configuration Support Class (One) public class SpeedmentHazelcastConfig{

public static void addTo(final Config config) {

addTo(config.getSerializationConfig());

}

public static void addTo(final ClientConfig config) {

addTo(config.getSerializationConfig());

}

public static void addTo(final SerializationConfig serializationConfig) {

serializationConfig.addDataSerializableFactory( DepartmentsDataSerializableFactory.FACTORY_ID, new DepartmentsDataSerializableFactory());

serializationConfig.addDataSerializableFactory( DeptManagerDataSerializableFactory.FACTORY_ID, new DeptManagerDataSerializableFactory());

serializationConfig.addDataSerializableFactory( EmployeesDataSerializableFactory.FACTORY_ID, new EmployeesDataSerializableFactory());

}

}

Transaction Support Classes public class HazelcastTransactionHandlerFactory implements

TransactionHandlerFactory {

private final HazelcastInstance hazelcastInstance;

private final TransactionOptions transactionOptions;

public HazelcastTransactionHandlerFactory(HazelcastInstance hazelcastInstance, TransactionOptions transactionOptions) {

this.hazelcastInstance = hazelcastInstance;

this.transactionOptions = transactionOptions;

}

@Override

public TransactionHandler newTransactionHandler() {

final TransactionContext context = hazelcastInstance.newTransactionContext(transactionOptions);;

return new HazelcastTransactionHandler(context);

}

}

Transaction Support Classes

public class HazelcastTransactionHandler implements TransactionHandler {

private final TransactionContext context;

public HazelcastTransactionHandler(TransactionContext context) {

this.context = context;

}

. . .

}

Transaction Support Classes @Override

public void begin() {

context.beginTransaction();

}

@Override

public void commit() {

context.commitTransaction();

}

@Override

public void rollback() {

context.rollbackTransaction();

}

public TransactionContext getContext() {

return context;

}

How do you use it?

“Easy as 1, 2, 3...”

1 Connect to your existing SQL DB…

…and it is Automatically Analyzed

2 Push play and….

Work as you are used to, just add 4 lines of code in your startup class Config hcConfig = new Config();

// Add optimized serialization Factories for Hazelcast that are generated automatically SpeedmentHazelcastConfig.addTo(hcConfig);

HazelcastInstance hcInstance = Hazelcast.newHazelcastInstance(hcConfig);

// Tell Speedment what Hazelcast instance to use

ProjectManager.getInstance().putProperty(HazelcastInstance.class, hcInstance);

// Automatically build all Database metadata (e.g. Schema, Tables and Columns) new TreeBuilder().build();

// Load selected data from the database into the Hazelcast maps and start tracking DB ProjectManager.getInstance().init();

3 Start Coding with Hazelcast

No change in how maps are used

// Get the Hazelcast map for the database table "employees"

IMap<String, Employees> employeesmap = hcInstance.getMap(EmployeesMgr.getHazelcastMapName());

// Use the Hazelcast map just as you are used to

Employees emp = employeesmap.get("10001");

Customization

•  Use your own beans instead •  Use transactions •  Filter events using predicates •  React on SQL events

Performance

Depends mostly on •  The SQL server •  Hazelcast grid configuration and

implementation

Business model

•  OpenSource coming •  Enterprise license

Thank You!

Per-Åke Minborg [email protected] http://minborgsjavapot.blogspot.com/ Twitter: @Pminborg To download: [email protected]

Applying Real-time SQL Changes in your Hazelcast Data Grid

Software

Transcript of Applying Real-time SQL Changes in your Hazelcast Data Grid