Obteniendo valor de toda la Información: Big Data
-
Upload
oracle-espana -
Category
Data & Analytics
-
view
68 -
download
2
Transcript of Obteniendo valor de toda la Información: Big Data
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Jordi TrillCore Tech Business Development Manager
Oracle IbéricaVitoria, 17 Mayo 2015
Obteniendo valor de toda la información: Big Data
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | - Oracle Big Data Architecture
EnterpriseProductivity
DisruptiveTechnology
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
“[Facebook] started in the Hadoop world. We are now bringing in relational to enhance that. We're kind of going [in] the other direction ... We've been there, and [we] realized that using the wrong technology for certain kinds of problems can be difficult.”
Ken RudinDirector of AnalyticsFacebook
http://tdwi.org/Articles/2013/05/06/Facebooks-Relational-Platform.aspx?Page=1
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Execution
Innovation
- Oracle Big Data Architecture
4th Generation Data Architecture for Big Data
WarehouseData FactoryReservoir
Data Streaming
Data Platform
Discovery Lab
Analytics
APIs
Enterprise Data
Other Data Sources
Data Streams
BusinessData
Social/LogData
Model FirstAnalytics
• Reporting-oriented• Often enterprise wide
in scope, cross LoB• “you know the
questions to ask”
Reports & Dashboards
Data FirstAnalytics
• Data Exploration• Highly visual and/or
interactive• “you don’t know the
questions to ask”
Discovery
• Telematics• Industry Services• Internet of Things• Sentiment
DataServices
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Execution
Innovation
- Oracle Big Data Architecture
Comprehensive Oracle Solution for Big Data
WarehouseFactoryReservoir
Data Streaming
Data Platform
Discovery Lab
Analytics
APIs
Enterprise Data
Other Data Sources
Data Streams
BusinessData
Social/LogData
Model FirstAnalytics
• Reporting-oriented• Often enterprise wide
in scope, cross LoB• “you know the
questions to ask”
Reports & Dashboards
Data FirstAnalytics
• Data Exploration• Highly visual and/or
interactive• “you don’t know the
questions to ask”
Discovery
• Telematics• Industry Services• Internet of Things• Sentiment
DataServices
ApacheOracleNoSQL
OracleCAF & OEP
Oracle Data Integration & Governance
Oracle Database& Big Data SQL
OracleR
OracleBig Data
Discovery
OracleBusiness
Intelligence
OracleBig Data
Discovery
Apache
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Execution
Innovation
- Oracle Big Data Architecture
Integrated Oracle Engineered Systems for Big Data
Data Streaming
Data Platform
Discovery Lab
Analytics
APIs
Enterprise Data
Other Data Sources
Data Streams
BusinessData
Social/LogData
Model FirstAnalytics
• Reporting-oriented• Often enterprise wide
in scope, cross LoB• “you know the
questions to ask”
Reports & Dashboards
Data FirstAnalytics
• Data Exploration• Highly visual and/or
interactive• “you don’t know the
questions to ask”
Discovery
• Telematics• Industry Services• Internet of Things• Sentiment
DataServices
APIs
Analytics
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Execution
Innovation
- Oracle Big Data Architecture
Visionary Oracle Cloud for Big Data
Data Platform
Discovery Lab
Analytics
APIs
Enterprise Data
Other Data Sources
Data Streams
BusinessData
Social/LogData
Model FirstAnalytics
• Reporting-oriented• Often enterprise wide
in scope, cross LoB• “you know the
questions to ask”
Reports & Dashboards
Data FirstAnalytics
• Data Exploration• Highly visual and/or
interactive• “you don’t know the
questions to ask”
Discovery
• Telematics• Industry Services• Internet of Things• Sentiment
DataServices
Data Streaming
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | - Oracle Big Data Architecture
Oracle Brings Business Value to Big Data
Enterprise-Grade Capabilities
Discover and Predict – Fast
Govern and Secure All Data
Simplify Access to All Data
Performance Integration Availability Scalability Manageability
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Oracle Big Data
Simplify Access to All Data
Discover and Predict, Fast
Govern and Secure All Data
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Build On Your FoundationHow can your team help ?
11
Data Warehouses
Surveys &Reviews
Social Media
3rd party data
Machine Data
Big Data
Enterprise Applications
Websites
Spreadsheets
Skills
Process
Infrastructure
Tools
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | - Oracle Big Data Architecture
Driving Business Value from Technology Innovation
Use the Right Tool for the Job and benefit from the Power of “AND”
RelationalHadoop NoSQL Graph
Run the Business Integrate existing
systems
Mission-critical
tasks
Use existing
investments
Ensure skills
relevance
Change the Business
Disrupt competitors
Disintermediate
supply chains
Leverage new
paradigms
Exploit new analyses
Scale the Business
Serve data
faster
Persist data streams
Meet mobile and
device challenges
Scale-out
economically
Link the Business
Associate complex
business entities
Link to Open Data
Share data sets via
Ontology
Evolve data and
schema together
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | - Oracle Big Data Architecture
Proven, Cost Effective Solution
“Oracle Big Data Appliance is an excellent choice for customers looking to work with the full suite of Cloudera’s leading Hadoop-based technology. It’s more cost-effective and quicker to deploy than a DIY cluster.”
⁻ Mike Olson, Cloudera founder, Chief Strategy Officer, and Chairman of the Board
Source:ESG White Paper
21%Cost Savings
33%Faster Time
to Value
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
SocialMedia
Expert System
Decision Engine
Oracle Event
Processing
Ejemplo Arquitectura Oracle Big Data
SmartSelling
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
SocialMedia
Expert System
Decision Engine
Oracle Event
Processing
Ejemplo Arquitectura Oracle Big Data
SmartSelling
Fast Data Solution:
Oracle Event Procesing
Oracle Real Time Decisions
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Oracle Event ProcessingCompleta dinámicamente el evento y procesa el dato
Una solución completa para construir aplicaciones para filtrar,
correlacionar y procesar eventos en tiempo real
Asegura que las aplicaciones a continuación son de inteligencia en
tiempo real
Ayuda a eliminar, consolidar, correlacionar y filtrar datos antes
que molesten al data warehouse
Analiza Streams de datos masivos en tiempo real
Base para el “Internet of Things” (en dispositivos)
Sólo envia al DataCenter información relevante
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Arquitectura Real-Time DecisionsSistema Experto de Recomendaciones en tiempo Real
Eventos de Negocio candidatos
Acciones de Negocio propuestas Multicanal y Multi-propósito
Motor de Recomendaciones masivas en Tiempo Real
(“RECOMENDADOR”)
SocialSocial
Objetivos de Negocio
Marcados por Analistas
Reglas y autoaprendizaje, para mejor predicción del comportamiento
del cliente
Base Datos Clientes 360 º
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Dell
Improve revenues and operations through personalized predictive analytics
$132M in net new revenue FY2012
20% increase profit margin per call
“The insights are amazing. You can really see customers' buying patterns and interests, how they change over time, and we can take action on that.”
Mark Surcrese, Marketing Director
Powered by Oracle Fast Data Solution
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
SocialMedia
Expert System
Decision Engine
Oracle Event
Processing
Ejemplo Arquitectura Oracle Big Data
SmartSelling
Oracle Data Integrator
Data Quality
Golden Gate
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Oracle Data Integration
Oracle GoldenGate
Captura cambios en los datos y eventos en
tiempo real
Sin impacto en el rendimiento de los sistemas origen de los datos
Oracle Data Integrator
Aplica filtrados y transformaciones a los datos
de los puntos de captura
Transforma y Carga datos estructurados o no-
estructurados para el análisis
Non-invasive real-time feeds
Small batch windows
Aproximación No-Invasiva de Alto Rendimiento
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Business Value of ODI: Low Cost and High Dev Efficiency
21
No ETL engine is required
Separation of Logical and
Physical design
Physical exec on SQL, Hive, Pig, or
Spark
Runtime exec in Oozie or via ODI
Java Agent
Rich set of pre-built operators
User defined functions
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
ETL Tool
Oracle Data Integrator
• Offload long running ETL jobs to Hadoop
• New Sources via Hadoop
• Leave existing ETL in place
22
Staging
Temp
Files
Files
Detail
Fast load
Data Warehouse Sources
MR
MR
SQL
OracleGoldenGate
Oracle Data Integrator
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Cap
ture
Trai
l
Ro
ute
Del
iver
Pu
mp
- Oracle Big Data Architecture
Oracle GoldenGate for Big Data
GoldenGate
Real-timeData Delivery
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
SocialMedia
Expert System
Decision Engine
Oracle Event
Processing
Ejemplo Arquitectura Oracle Big Data
SmartSelling
Big Data
HDFS
NoSQL
Conectores
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Oracle NoSQL Database Escalable, Alta Disponibilidad, Base de Datos Key-Value
Application
Storage NodesDatacenter B
Storage NodesDatacenter A
Application
NoSQL DB Driver
Application
NoSQL DB Driver
Application
• Datos Key-value, JSON y RDF
• Large Object API
• Transacciones ACID
• Load Balancing Transparente
• Software y Soporte Empresarial
• Online Rolling Upgrade
• Online Cluster Management
• Construida con Berkeley DB
• Integrada con Event Processing
• Latencia predecible
Características
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Oracle Big Data Connectors
Data LoadOracle Loader for Hadoop
Data AccessOracle SQL Connector for
HDFS
R AnalyticsOracle R Advanced Analytics
on Hadoop
Oracle Data Integrator Knowledge Modules
XML/XQueryOracle XQuery on Hadoop
XQueryR Client
Optimized for Hadoop
Maximum parallelism
Fast performance
Analyze all your data in-place
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
SocialMedia
Expert System
Decision Engine
Oracle Event
Processing
Ejemplo Arquitectura Oracle Big Data
SmartSelling
DWH Oracle DB
Olap, Spatial,
Avd. Analytics,
Data Mining & R
InMemory Opt.
BigData SQL
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
BOTH row and column
in-memory formats for
same table
Simultaneously active and transactionally
consistent
Analytics & reporting use
New Column format
Billions of rows/sec scan rate per CPU
core
OLTP uses row format
In-Memory Option: Dual Format In-Memory Database
Column
Format
Memory
Row
Format
Memory
AnalyticsOLTPSales Sales
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Oracle In-Memory Columnar Technology
Memory
Pure Columnar
Pure in-memory format with no logging (
no writes to disk )
Near zero overhead on changes – even
for OLTP
New memory-optimized compression
format – 2x to 10x
Data loaded in-memory for active tables or
partitions
On startup or first access
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Oracle Advanced Analytics
In-database data mining algorithms and open source R algorithms
– Statistical Functions
– Data Mining & Predictive Analytics
– Text Search & Text Mining
– Spatial & Graph Analysis
– In-Database MapReduce
SQL, PL/SQL, R languages
Scalable, parallel in-database execution
Workflow GUI and IDEs
Enables enterprise analytical applications
El más rápido despliegue de Análisis Predictivo y Data Mining
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
R Statistical Programming Language
Open source language and environment
Used for statistical computing and graphics
Strength in easily producing publication-quality graphs
Highly extensible
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Oracle Big Data SQL
One fast query, on all your data.
Oracle SQL on Hadoop, NoSQL and beyond, with a Smart Scan service inspired by Exadata and the security and certainty of Oracle Database
Use Existing Skills, Process and Tools to Deliver Big Data Apps and Analytics
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Data Lifecycle Management & Query OffloadMore data on-line and available at a lower cost
Move Partition to BDA
Oracle Big Data SQL
Rolling 13 months
Month 14-n
Big Data Rolling Windows
• Process
• Copy older partition to BDA
• Update views
• Drop older Exadata partition
• Offloaded data can be accessed via Oracle & Hadoop
• No Application changes required
OracleData
PCI FLASH
DRAM
Warm Data
Hottest Data
Active Data
Hadoop Data
Deep Data
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
SocialMedia
Expert System
Decision Engine
Oracle Event
Processing
Ejemplo Arquitectura Oracle Big Data
SmartSelling
Oracle BI
Foundation &
BigData
Discovery
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Oracle Big Data
Simplify Access to All Data
Discover and Predict, Fast
Govern and Secure All Data
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Not Easy to Get Analytic Value at Fast Enough Pace
37
Tool Complexity
80% evaluating and preparing data
Data Uncertainty
Dependent on highly skilled resources
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 38
Oracle Big Data Discovery. The Visual Face of Hadoop
find explore transform discover share
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 39
BigData Discoverybenefits
• Get Started Fast
• Simplify BigData analysis
• Justify project investment and get results
• Spend 80% of your effort in Analytics
• Leverage more of your analytics talent
find explore transform discover share
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Exadata
Oracle Business Intelligence Foundation Suite
BI Server - Modelo Común de Información Corporativa
• Metadatos Comunes sobre todas las fuentes• Servicios Comunes de Seguridad, Control de Acceso, Autorización, Auditoría• Servicios Comunes de Acceso Optimizado a Datos y Generación de Consultas• Servicios Comunes de Clustering, Gestión y Despliegue• Sistemas y Gestión del Ciclo de Vida Comunes
Dashboard –
Cuadros Mando
Publisher –
ReportingAnswers –
Análisis
Delivers –
Alertas
Office Plug-In
IntegraciónScorecard Mobile SimulaciónEssbase
Endeca Information Discovery
Endeca
Server
Bigdata Appliance
Adaptive In-
Memory Tools
In-memory
Software
In-Memory
Hardware
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Toda la Funcionalidad con Sist. Ingeniería ConjuntaSólo Oracle provee Sistemas de Ingeniería completos
Oracle Big Data Appliance
Oracle Exadata
InfiniBand
Acquire Organize Analyze & VisualizeStream
Oracle Exalytics
InfiniBand
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Oracle Big Data
Simplify Access to All Data
Discover and Predict, Fast
Govern and Secure All Data
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | - Oracle Big Data Architecture
Data Governance is Not Easy, Hadoop is No Silver Bullet!
Data Governance
Metadata Management
Business Glossary
Data Profiling
Data Cleansing
Data Archiving
Data Privacy
PEOPLE
PROCESS TECHNOLOGY
…people and process first, …tools and capabilities next, …and, there is no magic!
“…the overall impact of poor-quality data on the whole dataset remains the same. In addition, much of the data that organizations use in a big data context comes from outside, or is of unknown structure and origin. This means that the likelihood of data quality issues is even higher than before. So data quality is actually more important in the world of big data."
- Ted Friedman, Gartnerhttp://www.gartner.com/newsroom/id/2854917
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | - Oracle Big Data Architecture
Comprehensive Governance
FastLoad
Speed Layer
Batch Layer
DataGovernanceFoundation
Enterprise Data Quality(Profile & Cleanse)
Enterprise Metadata Management & Business Glossary(Business Glossary, Data Lineage, Impact Analysis and Data Provenance)
Veridata(Verify)
Data Enrichment(Prepare)
Data Governance– Prepare unstructured data– Profile data with sampling– Clean data in real time or batch– Verify data for consistency– Trace lineage of all data– Define glossary of business terms
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Oracle Enterprise Metadata ManagementTrust your data
Executives Dashboards / Reports
!=
!!!???
•Dashboard does not match report?
•Where did this data come from?
•Can I trust my other dashboards and reports?
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Value of Enterprise Metadata Management
47
ETL
BIDashboards
App
ETL
ETL
How was sales figure calculated?
What will happen if I change this
table?
What reports use the mainframe
data? Sys Admin
Executive
BI Developer
Where did this data
come from?
Application User
Which reports use this
customer data?
CDC
Data Reservoir
Data Steward
Can I trust the sources of this
customer data?
ETL
Developer
Solves a significant pain point for a wide variety of business consumers and technical staff
I want to design an experiment to measure the
success of a signup page. What data do I have?
Data Scientist
GG
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Govern & Secure All your DataHadoop, NoSQL & Relational
• Existing Capability– Authentication through Kerberos
– Authorization through Apache Sentry
– Auditing through Oracle Audit Vault
– Encryption for Data-at-Rest
– Network Encryption
• Big Data SQL adds– Advanced Security on Hadoop & NoSQL• Masking and Redaction
– Virtual Private Database• Fine-grain Access Control
49
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Oracle: Big Data para la Empresa
• Solución Completa– Incluye BigData, FastData y Analytics para todos los datos
• Optimizada para el Extreme Analytics–Data Discovery y Análisis Predictivo con acceso a todos los datos
• Engineered to Work Together–Elimina el riesgo del despliegue y del soporte
• Enterprise Ready–Rendimiento y Escalabilidad Extremos