Jdd 2016 DROP DATABASE
-
Upload
jarek-ratajski -
Category
Software
-
view
268 -
download
0
Transcript of Jdd 2016 DROP DATABASE
Jarosaw Ratajski
CSS Versicherung
DROP DATABASE
Jarek Ratajski
Developer, wizard, anarchitectLives in Luzern Works in CSS InsuranceC64, 6502, 68000, 8086, C++, Java, JEE, Scala, Akka, JS, [email protected] @jarek000000
Multiversum
MULTIVERSUM
Real life case
Galakpizza
10 000 Planets
3 Variants
3 Sizes
HAWAIIMARGHERITTAVEGETARIAN
MEDIUMLARGEXL
Domain summary
MARGHERITTA MEDIUM
HAWAII LARGE
HAWAII LARGE
VEGERARIAN LARGE
HAWAII XL
MARGHERITTA XL
HAWAII LARGE
HAWAII LARGE
HAWAII LARGE
HAWAII LARGE
3
3
1
1
Delivery model
10 000 Pizzas / sI want to look at number of queued orders 10000
times / sMy wife also!
Requirements
Alien development(in 2 parallel universes)
Normal universe
Other universe
++
In Normal universehttps://github.com/airomem/galakpizza/tree/master/galakpizza_n
Normal domain
public class Order implements Serializable {
private static final long serialVersionUID = 1L;
public final long id;
public final String planet;
public final Variant variant;
public final Size size;
public Order(long id, String planet, Variant variant, Size size)
{
this.id = id;
this.planet = planet;
this.size = size;
this.variant = variant;
}
}
Normal domain 2
public enum Size {
MEDIUM,
LARGE,
XL
}public enum Variant {
HAWAII,
MARGHERITA,
VEGETARIAN
}
Normal Service
public interface GalakPizzaOrders {
long placeOrder(String planet, Variant variant, Size size);
}
Normal Service
public interface GalakPizzaOrders {
long placeOrder(String planet, Variant variant, Size size);
}
public interface GalakPizzaDelivery {
List takeOrdersFromBestPlanet();
long countStandingOrders();
}
Normal Service
public interface GalakPizzaOrders {
long placeOrder(String planet, Variant variant, Size size);
}
public interface GalakPizzaDelivery {
List takeOrdersFromBestPlanet();
long countStandingOrders();
}public interface GalakPizzaService extends GalakPizzaDelivery,
GalakPizzaOrders{
}
Normal service implementation...
public class GalakPizzaCore implements GalakPizzaService,
Serializable, Storable {
public long placeOrder(String planet, Variant variant, Size size) {
...
}
public List takeOrdersFromBestPlanet() {
...
}
public long countStandingOrders() {
...
}
}
Normal service implementation...
public class GalakPizzaCore implements GalakPizzaService,
Serializable, Storable { private final Map orders = new
HashMap();
public long placeOrder(String planet, Variant variant, Size size) {
...
}
public List takeOrdersFromBestPlanet() {
...
}
public long countStandingOrders() {
...
}
}
Normal service implementation...
public class GalakPizzaCore implements GalakPizzaService,
Serializable, Storable { private final Map orders = new
HashMap();
public long placeOrder(String planet, Variant variant, Size size) {
...
}
public List takeOrdersFromBestPlanet() {
...
}
public long countStandingOrders() {
...
}
}
PlanetOrders - helper
class PlanetOrders implements Serializable {
final String name;
private List orders = new ArrayList();
public PlanetOrders(String name) {
this.name = name;
}
...
}
PlanetOrders - helper
class PlanetOrders implements Serializable {
final String name;
private List orders = new ArrayList();
public PlanetOrders(String name) {
this.name = name;
}
void assignOrder(final Order order) {
this.orders.add(order);
}
...
}
PlanetOrders - helper
class PlanetOrders implements Serializable {
final String name;
private List orders = new ArrayList();
public PlanetOrders(String name) {
this.name = name;
}
void assignOrder(final Order order) {
this.orders.add(order);
}
public boolean isEmpty() {
return this.orders.isEmpty();
}...
}
Normal place order
public class GalakPizzaCore implements GalakPizzaService,
Serializable, Storable {
private final Map orders = new HashMap();
private long orderSequence = 1;
public long placeOrder(String planet, Variant variant, Size size)
{
final long id = orderSequence++;
final Order order = new Order(id, planet, variant, size);
assignOrderToPlanet(order);
return id;
}
Normal place order
public class GalakPizzaCore implements GalakPizzaService,
Serializable, Storable {
private final Map orders = new HashMap();
private long orderSequence = 1;
public long placeOrder(String planet, Variant variant, Size size)
{
final long id = orderSequence++;
final Order order = new Order(id, planet, variant, size);
assignOrderToPlanet(order);
return id;
}
private void assignOrderToPlanet(Order order) {
final PlanetOrders po = orders.computeIfAbsent(order.planet,
planetName -> new PlanetOrders(planetName));
po.assignOrder(order);
...
}
Normal place order
public class GalakPizzaCore implements GalakPizzaService,
Serializable, Storable {
private final Map orders = new HashMap();
private long orderSequence = 1;
private AtomicLong ordersTotal = new AtomicLong(0);
public long placeOrder(String planet, Variant variant, Size size)
{
final long id = orderSequence++;
final Order order = new Order(id, planet, variant, size);
assignOrderToPlanet(order);
ordersTotal.incrementAndGet();
return id;
}
private void assignOrderToPlanet(Order order) {
final PlanetOrders po = orders.computeIfAbsent(order.planet,
planetName -> new PlanetOrders(planetName));
po.assignOrder(order);
...
}
Normal place order revisited
public class GalakPizzaCore { ...private final Map orders = new
HashMap();private PriorityQueue bestPlanets = new
PriorityQueue(256);private long orderSequence = 1;private
AtomicLong ordersTotal = new AtomicLong(0);public long
placeOrder(String planet, Variant variant, Size size) {
final long id = orderSequence++;
final Order order = new Order(id, planet, variant, size);
assignOrderToPlanet(order);
ordersTotal.incrementAndGet();
return id;
}private void assignOrderToPlanet(Order order) {
final PlanetOrders po = orders.computeIfAbsent(order.planet,
planetName -> new PlanetOrders(planetName));
po.assignOrder(order);
this.bestPlanets.offer(po);
}
Normal place order revisited
public class GalakPizzaCore { ...private final Map orders = new
HashMap();private PriorityQueue bestPlanets = new
PriorityQueue(256);private long orderSequence = 1;private
AtomicLong ordersTotal = new AtomicLong(0);public long
placeOrder(String planet, Variant variant, Size size) {
final long id = orderSequence++;
final Order order = new Order(id, planet, variant, size);
assignOrderToPlanet(order);
ordersTotal.incrementAndGet();
return id;
}private void assignOrderToPlanet(Order order) {
final PlanetOrders po = orders.computeIfAbsent(order.planet,
planetName -> new PlanetOrders(planetName));
po.assignOrder(order);
this.bestPlanets.offer(po); BAD BAD BAD BAD
}
Normal place order revisited
public class GalakPizzaCore { ...private final Map orders = new
HashMap();private PriorityQueue bestPlanets = new
PriorityQueue(256);private long orderSequence = 1;private
AtomicLong ordersTotal = new AtomicLong(0);public long
placeOrder(String planet, Variant variant, Size size) {
final long id = orderSequence++;
final Order order = new Order(id, planet, variant, size);
assignOrderToPlanet(order);
ordersTotal.incrementAndGet();
return id;
}private void assignOrderToPlanet(Order order) {
final PlanetOrders po = orders.computeIfAbsent(order.planet,
planetName -> new PlanetOrders(planetName));
po.assignOrder(order);
this.bestPlanets.offer(new PlanetOrders.Wrapper(po));
}
Wrapper... yeah
public static class Wrapper implements
Comparable,Serializable{
final PlanetOrders porders;
final int size; //just keep it final
public Wrapper(PlanetOrders porders) {
this.porders = porders;
this.size = porders.orders.size();
}
@Override
public int compareTo(final Wrapper other) {
return other.size - this.size;
}
}
PlanetOrders take orders
class PlanetOrders implements Serializable {
final String name;
private List orders = new ArrayList();
List takeOrders() {
final List result = this.orders;
this.orders = new ArrayList();
return result;
}
public boolean isEmpty() {
return this.orders.isEmpty();
}}
PlanetOrders take orders
class PlanetOrders implements Serializable {
final String name;
private List orders = new ArrayList();
List takeOrders() {
final List result = this.orders;
this.orders = new ArrayList(); //yes we MUTATE!
return result;
}
public boolean isEmpty() {
return this.orders.isEmpty();
}}
CYPS 2(2) ZYPS 1(1) CYPS 2(2) ZYPS (1) ZYPS 2(2)+ Order 1 Hawaii XL to ZYPS
CYPS 2(2) ZYPS 1(0)Take Orders
ZYPS 1(0)Take Orders
IsEmpty => ignore
Normal take orders
public class GalakPizzaCore implements GalakPizzaService,
Serializable, Storable {
private PriorityQueue bestPlanets = new PriorityQueue(256);
private AtomicLong ordersTotal = new AtomicLong(0); public List
takeOrdersFromBestPlanet() {
final Optional planetOpt = takeBestPlanet();
.... }
private Optional takeBestPlanet() {
PlanetOrders.Wrapper planet = this.bestPlanets.poll();
while (planet != null && planet.porders.isEmpty()) {
planet = this.bestPlanets.poll();
}
return Optional.ofNullable(planet).map(p->p.porders);
}
Normal take orders
public class GalakPizzaCore implements GalakPizzaService,
Serializable, Storable {
private PriorityQueue bestPlanets = new PriorityQueue(256);
private AtomicLong ordersTotal = new AtomicLong(0); public List
takeOrdersFromBestPlanet() {
final Optional planetOpt = takeBestPlanet();
if (planetOpt.isPresent()) {
final PlanetOrders planet = planetOpt.get();
List orders = planet.takeOrders();
return orders;
}
return Collections.EMPTY_LIST;
}
private Optional takeBestPlanet() {
PlanetOrders.Wrapper planet = this.bestPlanets.poll();
while (planet != null && planet.porders.isEmpty()) {
planet = this.bestPlanets.poll();
}
return Optional.ofNullable(planet).map(p->p.porders);
}
Normal take orders
public class GalakPizzaCore implements GalakPizzaService,
Serializable, Storable {
private PriorityQueue bestPlanets = new PriorityQueue(256);
private AtomicLong ordersTotal = new AtomicLong(0); public List
takeOrdersFromBestPlanet() {
final Optional planetOpt = takeBestPlanet();
if (planetOpt.isPresent()) {
final PlanetOrders planet = planetOpt.get();
List orders = planet.takeOrders();
ordersTotal.addAndGet(-orders.size());
return orders;
}
return Collections.EMPTY_LIST;
}
private Optional takeBestPlanet() {
PlanetOrders.Wrapper planet = this.bestPlanets.poll();
while (planet != null && planet.porders.isEmpty()) {
planet = this.bestPlanets.poll();
}
return Optional.ofNullable(planet).map(p->p.porders);
}
Normal count orders
private AtomicLong ordersTotal = new AtomicLong(0);public long
countStandingOrders() {
return this.ordersTotal.get();
}
We have service - but do we deal with?
Concurrency
Persistence
Normal - Alien service wrapper
public class GalakPizza implements GalakPizzaService {
final PersistenceController controller;
public GalakPizza() {
controller = PrevaylerBuilder.newBuilder()
.withinUserFolder("pizza")
.useSupplier( () -> new GalakPizzaCore())
.build();
} public long placeOrder(final String planet, final Variant
variant, final Size size) {
return controller.executeAndQuery( core ->
core.placeOrder(planet,variant,size));
}
public List takeOrdersFromBestPlanet() {
return controller.executeAndQuery( core
->core.takeOrdersFromBestPlanet());
}
public long countStandingOrders() {
return controller.query( c->c.countStandingOrders());
}
...
}
Done!
In alternative universehttps://github.com/airomem/galakpizza/tree/master/galakpizza_h
Other Domain
@Entity
@Table(name="T_ORDER")
@NamedQueries( {
@NamedQuery(name = "select best planet", query = "SELECT o.planet,
count(o) FROM Order o " +
" GROUP BY o.planet ORDER BY count(o) desc"),
@NamedQuery(name = "select orders", query = "SELECT o FROM Order o
WHERE o.planet = :planet"),
@NamedQuery(name = "count orders", query = "SELECT count(o) FROM
Order o ")
})
public class Order {
@Id
@GeneratedValue(generator="increment")
@GenericGenerator(name="increment", strategy = "increment")
final long id;
public final String planet;
public final Variant variant;
public final Size size;
protected Order() {
this("this is strange...", null, null);
}
public Order(String planet, Variant variant, Size size) {
this.planet = planet;
this.variant = variant;
this.size = size;
id = 0;
}
public long getId() {
return id;
}
}
Other domain (same as above)
public enum Size {
MEDIUM,
LARGE,
XL
}public enum Variant {
HAWAII,
MARGHERITA,
VEGETARIAN
}
It is in TABLE so it exists
Other service (same)
public interface GalakPizzaOrders {
long placeOrder(String planet, Variant variant, Size size);
}
public interface GalakPizzaDelivery {
List takeOrdersFromBestPlanet();
long countStandingOrders();
}public interface GalakPizzaService extends GalakPizzaDelivery,
GalakPizzaOrders{
}
Select best blanet...
SELECT o.planet, count(o) FROM Order o
GROUP BY o.planet ORDER BY count(o) desc
Other Planet (but almost same story)
@Entity
@Table(name="T_PLANET",
indexes = {@Index(name="cnt_idx", columnList = "count", unique =
false)} )
@NamedQueries( {
@NamedQuery(name = "select best planet from table", query = "SELECT
p FROM Planet p " +
" ORDER BY p.count desc")})
public class Planet {
@Id
public final String name;
private int count;
protected Planet() {
name = "default";
}
public Planet(String name) {
this.name = name;
}
public int getCount() {
return this.count;
}
public void increment() {
this.count++;
}
public void clear() {
this.count = 0;
}
}
Other service implementation
public class GalakPizza implements GalakPizzaService {
final SessionFactory sessionFactory;
public GalakPizza(SessionFactory sessionFactory) {
this.sessionFactory = sessionFactory;
}
public long placeOrder(String planet, Variant variant, Size size)
{
...
}
public List takeOrdersFromBestPlanet() {
...
}
public long countStandingOrders() { ...
}
}
Other - Place order
public long placeOrder(String planet, Variant variant, Size
size) {
return this.runInSession(session -> {
final Order orderEntity = new Order(planet, variant, size);
final Long key = (Long) session.save(orderEntity);
incrementPlanetCounter(planet, session);
return key;
});
}
private void incrementPlanetCounter(String planet, Session session)
{
Planet planetEntity = session.get(Planet.class, planet);
if (planetEntity == null) {
planetEntity = new Planet(planet);
session.save(planetEntity);
}
planetEntity.increment();
}private T runOnSession(Function dbCommand) { ...}
Other - runInSession
private T runInSession(Function dbCommand) {
final Session session = sessionFactory.openSession();
session.beginTransaction();
try {
final T result = dbCommand.apply(session);
session.getTransaction().commit();
session.close();
return result;
} catch (ConstraintViolationException cve) {
session.getTransaction().rollback();
session.close();
return runInSession(dbCommand);
} catch (Throwable t) {
t.printStackTrace();
throw new RuntimeException(t);
}
}
So easy
Other - Take orders
public List takeOrdersFromBestPlanet() {
return this.runInSession(session -> {
final Query bestPlanetQuery = session.getNamedQuery("SELECT p FROM
Planet p " +
" ORDER BY p.count desc");
bestPlanetQuery.setMaxResults(1);
final Iterator bestPlanetsIterator =
bestPlanetQuery.iterate();
if (bestPlanetsIterator.hasNext()) {
final Planet bestOne = bestPlanetsIterator.next();
final Query ordersQuery = session.getNamedQuery("SELECT o FROM
Order o WHERE o.planet = :planet");
ordersQuery.setParameter("planet", bestOne.name);
final List orders = ordersQuery.list();
orders.forEach(o -> session.delete(o));
bestOne.clear();
return orders;
}
return Collections.EMPTY_LIST;
});
}
Other count orders
@Override
public long countStandingOrders() {
return this.runInSession(session -> {
final Query ordersCount = session.getNamedQuery("count
orders");
final Long cnt = (Long) ordersCount.iterate().next();
return cnt;
});
}
Other - others
Hibernate config, init
MySql config
After one day of coding...
In a real galaxy
That we can simulate as
Customers - 4 Threads doing in loop placeOrder (random planet name, variant and size) no pauses
PizzBoy - 1 Thread doing in loop takeBestOrder no pauses
Owners 2 Threads doing in loop countStandingOrders
Galaxy simulation
public static void main(String... args) {
runSimulation(5000);
System.gc();
sleep();
Result result = runSimulation(SECONDS_TOTAL*1000);
System.out.println("Orders processed:" + result.processed);
System.out.println("p/s:" + (double)result.processed/(double)
SECONDS_TOTAL);
System.out.println("Orders seen:" + result.seen);
System.out.println("o/s:" + (double)result.seen/(double)
SECONDS_TOTAL);
}
?
Normal universe
Other universe
~40 000 p/s
~600 p/s
Processed orders
Normal universe
Other universe
~ 12 823 500 p/s
~6362 p/s
Peeping speed
Normal universe
Other universe
Normal universe
Other universe
GODS of SQL HELP! ME
Normal universe
Other universe
~60 000 p/s
~1000 p/s
After few days of operations...
Normal universe
Other universe
Cool, that it runs onlaptop
Reality strickes back
I do not trust PizzaBoyI want to know where exactly were pizzas delivered
Normal universe
Other universe
Cool, we will not delete orders any more, only mark them as deleted!Cool, we will add a list that will be processed during takeOrders,or maybe we will store data in DB Table (as archive)
Reality strickes back
I do not trust PizzaBoyI want to know where were pizzas deliveredYesterday!!!
Normal universe
Other universe
Crap...We will restart system and commands will be reprocessed
Commands once again
public class GalakPizza implements GalakPizzaService {
final PersistenceController controller;
public List takeOrdersFromBestPlanet() {
return controller.executeAndQuery( core
->core.takeOrdersFromBestPlanet());
}
public long countStandingOrders() {
return controller.query( c->c.countStandingOrders());
}
public long placeOrder(final String planet, final Variant variant,
final Size size) {
return controller.executeAndQuery( core ->
core.placeOrder(planet,variant,size));
}
}
Yet another real world case
System evolves
I want to let people compose their pizza/Like: more cheese on one half, and no meat on the next half, And onion on the third half.
Normal universe
Other universe
We will map it with only 20 tablesOr do we add TEXT field?public Order(long id, String planet, Supplier pizzaDescription, Size size) {
public Order(long id, String planet, Variant variant, Size size) {
We will use factory
Difference
+ Fast
+ Code in Java
+ Strongly typed
+ Flexible
+ Testable
- Sophisticated java
- Terribly slow
- Code in Strings and Annotations- Stringly typed
- Limited domain
- Testing expensive
+ Easy java
Why different ?
Long time ago
Sweet spot known as 90's
Computers had mostly one CPU Core
Memory (RAM) was expensive but more as 100MB available in single unit
Disks were slow but huge (like 2Gb-20Gb)
Time for JEE
Around galactic year 2003
Normal universe
RAM is so cheap and huge
Computers have multiple cores
Alien developers start to use full power of hardware with Java
No need to store data in databases anymore
Because RAM is so cheap and huge
Other universe
RAM is cheap and huge
Computers have multiple cores
But alien developers have never realised that
Other universe is a good place
Database vendors earn money and are happy
Application Servers vendors earn money and are happy
Power plants are happy
Hardware vendors are happy
Only business complains but they still pay
Developers have a lot more work to do they are happy..
Gfx By pressfoto / Freepik
DB / ORM vs Prevalence Problems
ProblemPrevalent (Normal)ORMs (Other)
Detached objectsYesYes
Transaction isolationYes in JAVA(concurrency)Yes in DB
TestingTrivial Awful
CacheIt is all cacheLots of configs...
MagicContainedEverywhere
MappingAll serializable goesLimited
If you want to jump to normal universe...
Simple start
Prevayleror
Airomem (Prevayler wrapper for Java 8 )or
Nothing :-)
Airomem quick - maven
pl.setblack
airomem-core
1.0.5
Airomem - init
PersistenceController controller =controller =
PrevaylerBuilder
.newBuilder()
.withinUserFolder("pizza")
.useSupplier( () -> new GalakPizzaCore())
.build();
GalakPizzaCore is main/plain object - Root aggregate!Must be Serializable! (how bad)
Airomem commands
controller.executeAndQuery( core->core.placeOrder(planet,
variant, size )); ----------------------controller.executeAndQuery(
new PlaceOrderCommand(planet,variant,size));
private static final class PlaceOrderCommand implements Command
{
private final String planet;
private final Variant variant;
private final Size size;
public PlaceOrderCommand(String planet, Variant variant, Size
size){
this.planet = planet;
this.variant = variant;
this.size = size;
}
public Long execute(GalakPizzaCore system) {
return system.placeOrder(planet,variant, size);
}
}
Commands are never executed concurrently
Query
controller.query( c->c.countStandingOrders());
Queries are executed concurrently - also with one command!
Airomem snapshots, events
jarek@ubuntu:~$ cd prevayler/jarek@ubuntu:~/prevayler$ lspizzajarek@ubuntu:~/prevayler$ cd pizza/jarek@ubuntu:~/prevayler/pizza$ ls -altotal 188drwxrwxr-x 2 jarek jarek 4096 May 6 14:46 .drwxrwxr-x 3 jarek jarek 4096 Apr 26 13:40 ..-rw-rw-r-- 1 jarek jarek 675 Apr 26 14:17 0000000000000000001.journal-rw-rw-r-- 1 jarek jarek 3375 Apr 26 14:18 0000000000000000002.journal-rw-rw-r-- 1 jarek jarek 3369 May 5 10:51 0000000000000000007.journal-rw-rw-r-- 1 jarek jarek 683 May 5 13:48 0000000000000000012.journal-rw-rw-r-- 1 jarek jarek 2048 May 5 14:12 0000000000000000012.snapshot-rw-rw-r-- 1 jarek jarek 497 May 5 14:12 0000000000000000013.journal-rw-rw-r-- 1 jarek jarek 1849 May 5 14:15 0000000000000000013.snapshot-rw-rw-r-- 1 jarek jarek 497 May 5 14:15 0000000000000000014.journal-rw-rw-r-- 1 jarek jarek 1685 May 5 14:17 0000000000000000014.snapshot-rw-rw-r-- 1 jarek jarek 8086 May 5 14:19 0000000000000000015.journal-rw-rw-r-- 1 jarek jarek 1084 May 6 13:25 0000000000000000028.snapshot-rw-rw-r-- 1 jarek jarek 57977 May 6 14:05 0000000000000000029.journal-rw-rw-r-- 1 jarek jarek 1228 May 6 14:46 0000000000000000132.snapshot-rw-rw-r-- 1 jarek jarek 59021 May 8 04:28 0000000000000000133.journal
Problems?
RAM?
Data migration
Not big deal in fact
Java Serialization (sth to learn)
readResolve, writeReplace
XML export option
Transaction
This is always transactional!!!
public Long execute(GalakPizzaCore system) {
return system.placeOrder(planet,variant, size);
}
Concurrency
Learn java.util.concurrent, synchronized or die!
Replication/ failover
Not in standard... butTrivial to implement - send events to other machines, share folder
Indices
Indices in memory do existsCQEngineQuery query1 = or(endsWith(Car.NAME, "vic"), lessThan(Car.CAR_ID, 2));for (Car car : cars.retrieve(query1)) { System.out.println(car);}
Airomem summary
CQRS + ES simplified - one command one eventWith RAM projection
More enterprisy alternatives
Lagom!
Akka persistence
CQRS generally
Advice
Remember to add Thread.sleep(1000) often
Storing in Prevayler vs DB
model events vs model domain
store facts vs store state
Shared state is a cache vs shared state is a problem
Domain with little magic vs domain polluted
Clean architecture vs Maybe DDD can help...
End
Try on something small first (not bigger then one planet)
Drop DB from core of your system
Drop your application servers
Drop annotations
Drop magic
Learn Java and algorithms
Feel the power of RAM and CPU alltogether!