Showcasing Data Science Lab functionality

Welcome from Kognitiowww.kognitio.com

Today’s Web Seminar -

Presenters HostMichael HiskeyVice PresidentMarketing & Business Development

Format & Agenda

Keynote Presenters

Dr. Sharon KirkhamData ScientistKognitio Analytics Center of Excellence

• Big Data and Complexity– the need for Data Scientists Question Break #1

• Data Manipulation – functional demonstrationQuestion Break #2

• Product forecasting with parallel R  ‐ practical demonstration Question Break # 3

Kognitio

Kognitio is focused on providing the premier high‐performance analytical platform to power business insight 

around the world

• Kognitio invented the in‐memory analytical platform, first taking it to market in 1989

• Privately held• Labs in the UK ‐ HQ in New York, NY 

The Data Science Lab

Data Scientists &

Staff

MathematicAlgorithms

MPP Computing

BIG DATA

11

What do business users want to do?

Find patterns

Track life time

journeys

Predict behavior

Forecast scenarios

Allocate scarce

resources

Model value

Characterize groups

Visualize discovery

Respond, trigger,

manage, promote

I’m a data scientist! Are you?Entry level skills and development - aspiration

Machine Learning

Graduates

I’m a data scientist! Are you?

BusinessExpertise

Machine Learning

Interpretationskills

= Insight

Graduates

Need guidance

Data Scientist

Supporting the data scientistTypical process – traditionally…

Database

Supporting the data scientistTypical process – direct data preparation

Database

SQL processing

Supporting the data scientistTypical process – produces analytical data set

Database

SQL processingData Set

Supporting the data scientistTypical process – run analytics from server

Database

SQL processingData Set

???

Supporting the data scientistTypical process – data samples often used

Database

SQL processingData Set

???

Data Samples Process runiteratively= slow

Supporting the data scientistTypical process – modelling process is honed

Database

SQL processingData Set

???

Data Samples Process runiteratively= slow

Supporting the data scientistTypical process – model is complete

Database

Data Set

???

Supporting the data scientistTypical process – score full data (Ouch!)

Database

Data Set

???

Full data to score

Supporting the data scientistPush processes to DB – still produce analytical data set

Analytical Platform

SQL processingData Set

Supporting the data scientistPush processes to DB – translate specific processes

Analytical Platform

SQL processingData Set

???

Translation

Supporting the data scientistPush processes to DB – results passed back

Analytical Platform

SQL processingData Set

???

Translation

Result Data Set

Supporting the data scientistPush processes to DB– modelling process is honed

Analytical Platform

SQL processingData Set

???

Translation

Result Data Set

Supporting the data scientistPush processes to DB– model scoring done in DB

Analytical Platform

SQL processingData Set

???

Result Data Set

Supporting the data scientistBut we always want more! Complex data structure

Analytical Platform

Data Set

???

Result Data Set

SQL cannot handleData complexity.How do I integrate into my model?

Supporting the data scientistBut we always want more! non-standard processes

Database

SQL processingData Set

???

Data Samples Back where we started

Supporting the data scientistBring Analytics to data – still produce analytical data set

SQL processing

SQL processing

Supporting the data scientistBring Analytics to data – can use other code for data prep

SQL processing

Kognitio scripting

Code executedUsing MPP

Data held in Memory. Fast access to CPUs

Supporting the data scientistBring Analytics to data – run analytics natively in Kognitio

SQL processing

Kognitio scripting

Code executedUsing MPP

Data held in Memory. Fast access to CPUs

One platform flexible workingfrom data prep through analyticalprocess

New! Kognitio version 8: Enabling and extending the Analytical Platform

External Tables

External FunctionsNot Only SQL

Hadoop Connector Other Connectors

Kognitio Storageas an External table

General Availability: June 2013

External Scripting – Data Transformation

Converting structured data into XML format, i.e. furnishing

personalised content

Assembly

Converting XML into structured data

Disassembly

Extracting complex informationfrom URLs

Pulling words from large text fields, i.e. sentiment analysis

Parsing

Converting row based informationinto columns for data mining,

i.e. supporting classification orsegmentation

Transposition

e.g. using perl

Examples where SQL is typically complex and extensive

Data Manipulation Small Demo

Product Forecasting – with parallel R

ForecastingRequirements

Forecast Inputs

R running in an MPP environment

PersistenceLayer

AnalyticalPlatform

Layer

R running in an MPP environment

PersistenceLayer

AnalyticalPlatform

Layer

Kognitioplatform

specification

16 servers462GB Kognitio

RAM128 Cores

This is old kit

2.9 billionrows of

epos

184 day time seriesfor 12K products

R running in an MPP environment

PersistenceLayer

AnalyticalPlatform

Layer

R running in an MPP environment

PersistenceLayer

AnalyticalPlatform

Layer

1 output table in RAM

128 parallel instances of R

R running in an MPP environment

PersistenceLayer

AnalyticalPlatform

Layer

Application &Client Layer

ExcelAll BI Tools

R running in an MPP environment

PersistenceLayer

AnalyticalPlatform

Layer

Application &Client Layer

ExcelAll BI Tools

13 views of different analytical

output

R running in an MPP environment

PersistenceLayer

AnalyticalPlatform

Layer

Application &Client Layer

ExcelAll BI Tools

Result set contained

# rows

12K forecasts andstats calculated

in # seconds

2.9B EPOS items collated into time seriesin # seconds

Product Forecastingusing parallel R Demo

Thank you for your participation today

• More information on today’s topic can be found at: • kognitio.com/mpp_r• kognitio.com/product‐forecasting

• FREE TO USE – perpetual license– www.kognitio.com/free– Contact us for the pre‐release version 8

• Analyst White Papers– EMA Comparative Analysis – In‐memory database platforms– www.kognitio.com/emacompinmem

• Today’s slides (and more): www.slideshare.net/Kognitio

connect

www.kognitio.com

twitter.com/kognitiolinkedin.com/companies/kognitio

tinyurl.com/kognitio youtube.com/kognitio

NA: +1 855  KOGNITIOEMEA: +44 1344 300 770