Con2715 Baumann Oow

8/17/2019 Con2715 Baumann Oow

1/46

Top Five Ways Swiss

Improved Business Value

Database In-Mem

aolo !reth

Head of Data Manage

Thomas Bauman

Head of IT Performance M


2/46

In-Memory Ma"in# $eadlines at Swiss MobiliarNew business insights due to real-time analytics

ANALYTICS ON OLTP DATAE

PORTS AUF OLTP DATEN


3/46

IN 9 VON 10 FÄLLEN KEIN SQL QUERY TUNII

N 9 VON 10 FÄLLEN KEIN SQL QUERY TUNI

In-Memory Ma"in# $eadlines at Swiss MobiliarMassively reduced tuning efforts for !" #uery te$ts

UT OF 10 QUERIES DON‘T NEED SQL

REPORTS AUF OLTP DATEN


4/46

REDUCED ETL FLOW

In-Memory Ma"in# $eadlines at Swiss Mobiliar % &aradigm change' ()ring %nalytics to the Data(

Verz!"# $%& ETL


5/46

%#enda

wiss Mobiliar in a Nutshell

%nalytics on ,"TP data where we are today %nalytics on ,"TP data Po. with Database In-M

Po. Database In-Memory /esults

0urther $&eriences and Ne$t te&s with Databas

In-Memory at wiss Mobiliar ummary


6/46

legal form of a coo&erative ass

com&any4

wit5erland6s number one insuhousehold contents7 business

life insurance4

close to customers throughout

than8s to around 9: general ag

locations4

over 14< million insured &erson

over *7*:: em&loyees and 3=+

Swiss Mobiliarwit5erland>s most &ersonal insurer

&'( continuously )**'-)*&+


7/46

Overall ,on-i.e Insurance Mar"et /rowth in Swit0erland.lose to =?3 of Mar8et @rowth to wiss Mobiliar in =:1*

@rowth Mobiliar Mar8et @rowth

in Mio .H04 ource' chwei5erische


8/46

ystems' i,7 "inu$7 Bindows and 5?,

+::: Noteboo8s7 19:: iPhones7 3:: iPads

D)M' ,racle7 D)=7 IM7 M !" erver7 Neo*C

tandard-B 2%rc@I7 %P7 ),7 iebel7 .,/ "ife etc4

Many home-grown a&&lications

Swiss Mobiliar1s IT Misson Statement2 De&loy and /uninnovative technologies for efficient business &rocesses

S


9/46

The Spea"er Thomas )aumann

f Technology7 Eurich

istics

ata

Management ystems and Performance since 1=

al conferences

obiliar

dedicated to &erformance since 1;3

Fdedicated to &erformance

also &roduces this search


10/46

%#enda


%nalytics on ,"TP data where we are today %nalytics on ,"TP data Po. with Database In-M





11/46

%nalytics on OT Data at Swiss Mobiliar %rchitecture =:1*

,"TP

Decision u&&ort Business

Intelli#ence

Data Mart

Data

Warehouse

3ross

In.ormation Systems

3ore

In.ormation

Systems

%ccessIn.ormation

Systems

%nalytical

2,"%P

,&erational

2,"TP

,"TP %nalytics

co&e

% l ti OT d t t S i M bili


12/46

%nalytics on OT data at Swiss Mobiliar %rchitecture =:=:

,"TP

Decision u&&ort Business

Intelli#ence

Data Mart

Data

Warehouse

3ross

In.ormation Systems

3ore

In.ormation

Systems

%ccessIn.ormation

Systems

%nalytical

2,"%P

,&erational

2,"TP

,"TP %nalytics

co&e

%rchitecture )*&4


13/46

Desi#n atterns

.olumn oriented data storage

Data re&lication close to real time based onlog records7 not transactionally consistent

!uery /e-/outing decided by o&timi5er7

trans&arent for a&&lication

No need for inde$es

Aery high com&ression rate

%rchitecture )*&4ID%% 2I)M D)= %nalytics %ccelerator Aalue Delivery

D)= 5,

2(Mainframe(

I)M

Nete55a

!" !uery

2to D)=

/esult et

%utomatic #uery re-routing o

intensive #ueries1 to data co

a&&liance

1 0or a&&lications which don>t re#uire transactionall

acce&t data delayed by a few minutes

imilar design &atterns for ,racle Database In-Memory

%re there similar results as wellG

Ma5or 6esults

Increased ,"%P #uery

1:: times faster in ave

0aster inserts on D)= a

scalability

Due to elimination of m

hort timeframe betwe

and analysis

More 6eal Time %nalytics $eadlines at Swiss Mobiliar


14/46

( REDUCTION OF MAINFRAME CPU

More 6eal-Time %nalytics $eadlines at Swiss Mobiliar/educed .P consum&tion during &ea8 time

REDUKTION MAINFRAME CPU RESSOURCE

ID%% in more detail

2or what is different to ,racle Database In-Memory

Massive arallel rocessin# %rchitecture

3= nodes on = blades at wiss Mobiliar Ma$ .a&acity 1:=* nodes 29 rac8s

Data rocessed ocally 0P@% 2HB encoded logical arrays'

Decom&ression7 ProCection7 Predicate

%&&lication all done locally within 0P@%

oins &rocessed locally if &ossible7 dataredistribution among nodes otherwise

Data clustering within node to su&&ort s8i&s

for data scanning 25one ma&s

Tables completely replicated 2all

rows?columns

Tunin# !nobs'

Data distributio

Data clustering

)oth are inde&

&artitioning and

Step by Step ID%% 7sa#e


15/46

0irst )usiness %&&lications

%d-hoc re&orts from business end users

Im&roved end-of-month &rocessing

"og analysis based on D)= tables for access &attern analytics

Im&roved T" flow

treamline Mainframe for ,"TP

liminating inde$es used for analytics only

liminate M!T and other au$iliary structures for analytics

/educed demand for reorg More efficient inserts

Step-by-Step ID%% 7sa#e1= month from installation to broad usage

1= months

New )usiness 0unctions

Be e$&ect a similar timeline for ,racle Database In-Memory 2see later

,ew Business 6eports 21 of =


16/46

,ew Business 6eports 21 of =How wiss Mobiliar>s ./M system is usedG

$8

; ;4+ <


17/46

hich car ma"es do also have many other contracts at Swiss Mobiliar9

,ew Business 6eports 2= of = .ross elling

with partner_MFZ (PNR, Make) as(SELECT DISTINCT PAR.C!"#$ AS PNR, MFZ.D%&' as MAE

FRM D*"P+IE.+-ES-PA" PAR, D*"P+IE.+ALL' +ER, D*"P+IE.+PA'$ +ER_*/,

D*"P+IE.+*MFZP$ MFZ

0ERE PAR.C#'"% 1 +ER.C#'"% AND +ER.C#'"%1+ER_*/.C#'"% AND +ER_*/.C#'&%_$1MFZ.C#'&%

AND

+ER_*/.C#'%"1MFZ.C#'%" AND PAR.C!$# 1 2$! AND PAR.C% 1 AND 3EAR(PAR.C")

1

AND 3EAR(PAR.C#) 1 AND 3EAR(+ER_*/.C#)1 AND 3EAR(+ER_*/.C")1

AND

3EAR(MFZ.C#)1 AND 3EAR(MFZ.C")1 AND +ER.C#'&% 1 $ AND +ER.C% 1

AND

3EAR(+ER.C#) 1 AND 3EAR(+ER.C") 1 AND +ER.D#'& IN ("2,"#,"%,"!) ),

partner_a44 (PNR, N567er) as

(SELECT PAR.C!"#$ AS PNR, C8NT(9)

FRM D*"P+IE.+-ES-PA" PAR, D*"P+IE.+ALL' +ER

0ERE PAR.C#'"% 1 +ER.C#'"% AND PAR.C!$# 1 2$! AND PAR.C% 1 AND 3EAR(PAR.C") 1

AND

3EAR(PAR.C#) 1 AND +ER.C#'&% 1 $ AND +ER.C% 1 AND 3EAR(+ER.C#) 1

AND

3EAR(+ER.C") 1 -R8P *3 PAR.C!"#$)

se4e:t 6ake, a;_:=ntra:ts, :=5nt(9) as n567er_=>_:5st=6ers

>r=6 Partner_6>?, partner_a44where partner_6>?.pnr1partner_a44.pnr


18/46

6eal-Time %nalysis2 6is"snvalidated correlations

)oth statistics7 number of stor8s observed7 and number of new

children counted in same area and year7 are correlated with tim

both were decreasing7 inde&endent from each other4

No4 of new-born

children in samearea and year

No4 of stor8s

observed


19/46

%#enda


%nalytics on ,"TP data where we are today

%nalytics on ,"TP data Po. with Database In-




Oracle Database In Memory o3


20/46

Oracle Database In-Memory o3,bCectives

The o3 2)eta-test of ,racle 1=c Database In-Memory shall &rov

com&arable7 if not better7 res&onse times might be achieved for the

and the same #uery com&ared with D)=?Nete55a4 the number of #ueries with demand for manual #uery statement tu

reduce by at least :J4

analytics on OT data will be: after migration D)=,racle7

K very e..icient4

K trans&arent for users and a&&lications4

K without impact on OT &rocessing4


21/46

Oracle Database In-Memory

column-oriented

Memory

data

Memory

row-oriented

data

o far' Data organi5ed in rows: data loaded into memory at .irst usa#e

designed for

transactions

ew2 %dditional' Data organi5ed in columns: permanently stored in memor

desig

ana

automated7 near real-time

re&lication

Oracle Database In-Memory o3


22/46

O..er and 3ontract data

Migration D)=,racle of < tables with No4 of rows between : an

,ffer data 2++:L contracts in &rogress

.ontract data 2+4*M contracts

In other words' very 1:th contract is in &rogress4

This data is not &art of the traditional data warehouse4

%nalytics of this data needs to be close to real time

%ll tests were &erformed by a&&lying &' real user reports with real da

Oracle Database In-Memory o3.onfiguration

Oracle Database In-Memory o3


23/46

14 How e$&ensive is the migration of the D)= data into ,racleG

=4 How much tuning effort was re#uired after activating ,racle Database In-Mem

34 %re the res&onse times better than with D)=G Bhich &art of the im&rovement

Database In-MemoryG

*4 How much administration effort is necessary to de&loy and run Database In-M

+4 Bhich are the &rere#>s to a&&ly Database In-MemoryG

;4 Bhich are the strengths and wea8nesses of Database In-MemoryG

Oracle Database In-Memory o3The Po. should answer the following ; #uestions


24/46

The Spea"er


25/46

pPaolo Lreth

tics at @enoa niversity

19 ,racle 9i ,bCect /elational Database

cal "ead Mobi

ou&

8uestion &2 Database mi#ration .rom DB) to Oracle


26/46

8uestion &2 Database mi#ration .rom DB) to Oracle

K Data was successfully migrated from D)= to ,racle' 3 days effort4

K %dCustments from D)= to ,/%."− TIMT%MP Data Ty&e conversion =*':: =3'+'

2,racle does not recogni5e =*'::

− %dCustments on DD"-definitions for em&ty trings?Numbers?Timestam

CREATE TABLE TEO_TBENPRO

( C43087 CHAR(30) DEFAULT ON N

NOT NULL, C…. NUMBER(30) DEFAULT ON NU

NOT NULL,

…. )

KNo runtime errors4

8uestion )2 $ow much tunin# e..ort was re


27/46

# <

activatin# Database In-Memory9

K No tuning was re#uired4K ,ne single o&timi5ation'

− Be changed the &artitioning layout of a table to o&timi5e it for the re#u

2%mount' 1 hour


28/46

8uestion '2 $W-3omparison between ID%% and Oracle


29/46

8uestion '2 $W-3omparison between ID%% and Oracle

IBM ,ete00a ID%% vs; Oracle &)c Database In-Memory

O in the red co

Oracle &)c mit

K Hitachi nifiedPlatform

K 1; .ore

K 39* @)

K no attache P

K ,' /edHat

K ,racle 1=41

Oracle &)c mit

K Hitachi nifiedPlatform

K 1; .ore

K 39* @)

K no attache P

K ,' /edHat

K ,racle 1=41

O in the blue cornerO4

IBM ,ete00a ID%%2

K PureData ystem for %nalyticsN1::1-::=

K =* .ores

K


30/46

:41:

14::

1:4::

1::4::

ID%%

,racle IM

.olumn tore

8uestion '2 Oracle &)c Database In-Memory vs; IBM ,ete00a

K .om&arison of ela&sed time O

O Nete55a ID%% vs4 ,racle Database In-Memory

Total e

ID%%

,/%.

8uestion 42 %dminstration o. Oracle Database In-Memory


31/46

8uestion 42 %dminstration o. Oracle Database In Memory

K +: @) /%M was reserved for the in-memory column store

−. O*1 @) used 2Memcom&ress for !uery

−. Oon %N-these data occu&ied 1


32/46

8uestion +2 re conditions .or optimal usa#e

Which are the pre conditions .or an optimal usa#e o. Oracle Database In-M

a se of ,/%." built in functionalities to restrict the data set 2Partitioning

−

The o&timal &artitioning method can significantly im&rove the #uery &b 0ind the o&timal degree of &arallelism in 1=c

− In our tests we found that the o&timal degree of &arallelism was 9

2this value de&ends on hardware we tested on the .Ps were unab

data from memory

− More &arallel slaves &roduced only more overhead

− 0or different hardware this value could differ

c Plan tability

− ,f relevant im&ortance is the stability of e$ecution &lans4 Therefore th

statistics have to be u& to date4

8uestion A2 Stren#ths and Wea"nesses o. Database In-Memory


33/46

8uestion A2 Stren#ths and Wea"nesses o. Database In Memory

Stren#ths2

K It scales very good incrementing the degree of &arallelism4 %lso small deg

show big &erformance im&rovements4

K /ead and 0ilter of large data volumes

K asy administration

You can improve the performance of an application within minutes

Wea"nesses2

K orts of large data volumes is still a bottlenec8 2 1:: @) orts still use tem&orary tables&ace

K Pro&osal to O6%3'

Tem& egment in Memory for .olumn store G

0eedbac8 to ,racle has been &rovided4

%#enda


34/46

%#enda





0urther $&eriences and Ne$t te&s with Databa


6esults within the MobiliarFIS ro5ect


35/46

5

What were the straits within the MobiFis ro5ect2

Mobi0is eDBH is still in develo&ment

Be could wor8 only on a small set of #ueries and

re&orts

Solution2

Be did the tests only on !ueries on a fact table and the

corres&onding dimension tables

)eside the small #uery set on to& of the re&orts we got from develo&ment

synthetic #ueries trying to simulate future re&orts by creating'

ums7 averages and other grou& functions on the e$isting tables

@rou&ing of values

Be issued these #ueries on the following data volumes

K 1 0act-Table with14 Mrd4 /ows

K & to1: Dimension Tables between 1:: and =4+ Mio4 /ows

6esults on MobiliarFIS


36/46

6esults on MobiliarFIS

3omparison o. elapsed times

8uery lapsed Time

& &)c

,verview Tb "evel 2),-/e&ort * 193

Numer of contracts for one &roduct in :1?1* 3= =

)uilding bloc8s &er single contract =


37/46

6esume MobiliarFIS

Summary

K The /esults from the Po. on ,racle 1=c

QIn-Memory .olumn toreQ showed an im&rovement of one or two magnitu

K Ty&ical DBH #ueries li8e sums7 averages etc4 can be im&roved dramatical

K The assum&tions made at the start of the Po. could be confirmed4

,e(t Steps

K Mobiliar0is has been migrated on 1=414:4= and we are beginning to develo&

will benefit from ,racle Database In-Memory4

Test on an e(istin# %pplication - 6I3O


38/46

Be tested on the database of our ris8 controlling a&&lication 2/I.,

The /ico Database occu&ies about 1 Tb of Data

The /ico a&&lication is over =: Rears in &roduction

Be collected the most im&ortant #ueries from the user>s &ers&ective

These #ueries were the candidates for im&rovement4

Tests on 6I3O = Be.ore startin#


39/46

Be analy5ed the e$isting a&&lication and investigated &erformance o&timi5ation

before 1=c'

.om&ression

,&timi5ation of P.T0/ &arameter

.hange of the &artitioning schema of some tables

Be started using In-Memory after these changes


40/46

6esults 6I3O


41/46

6esults 6I3O

&eed & ,racle 1=c In-Memory vs4 ,racle 1=c

Parallel Degree !uery 1 !uery = !uery 3 !uery * !uery

1 1:949 =+431 *94


42/46

6esults 6I3O

)oth slides combined' &eed & ,racle 1=c In-Memory vs4 ,racle 11g

Parallel Degree !uery 1 !uery = !uery 3 !uery * !uery

1 39*4:* *;41; 3


43/46

/ico has been migrated on uly on 1=414:4=

% develo&ment environment has been built to test the u&grade and the

Memory

Be are using the !" Performance %naly5er of the /eal %&&lication T

to identify im&rovements and regression

Be are involved in the ,racle 1=c /eference Program and we are wor

with ,racle4

,racle In Memory will be activated after the Mobiliar oftware /elease

,ctober

%#enda


44/46





0urther $&eriences and Ne$t te&s with Databa

In-Memory at wiss Mobiliar

ummary

Oracle Database In-Memory Bene.itsnd ser Perce&tion


45/46

nd ser Perce&tion

"i8e an additional7 fast7 access &ath

0astest way from an 0rancisco to .hicagoG

!" Tuning D) In-Memory

)eam me u&731 hours *h 3:min21h 3:min

Than" you@


46/46

thomas4b

&erf

&a
mailto:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]

Con2715 Baumann Oow

Documents

Transcript of Con2715 Baumann Oow