Oracle OpenWorld 2016 Review - Focus on Data, BigData, Streaming Data, Machine Learning & Database...
-
Upload
lucas-jellema -
Category
Software
-
view
154 -
download
0
Transcript of Oracle OpenWorld 2016 Review - Focus on Data, BigData, Streaming Data, Machine Learning & Database...
The AMIS Team
Oracle OpenWorld 2016, Nieuwegein, 13th October 2016
Oracle OpenWorld 2016 ReviewData - Database Development, BigData, BI
2
Data
• ACID is expensive• OLTP is a niche• Moving data takes long – bring processing to data• SQL is omnipresent – expose all data in an SQL friendly way
– Including NoSQL and data on Hadoop• Data from the past should be able to help us predict the future
– Bring on machine learning (aka AI aka predictive analytics)– Aided by the citizen data scientist in (Big) Data Discovery
• Fast Data (big data at high velocity) should be handled in real time– Enter: Streaming Analytics & Apache Kafka
• Oh and ehm …. a next major release of Oracle Database is available– 12cR2 – only on the cloud for now– Highlights: Sharding, More PDB (“virtual database”),
Approximate Query Processing, Leverage In-Memory even more, JSON document generation and faster JSON processing, Analytic Views
3
Learning
• How do we learn?– Try something (else) => get feedback => learn
• Eventually:– We get it (understanding) so we can predict the outcome
of a certain action in a new situation– Or we have experienced enough situations to predict
the outcome in most situations with high confidence• Through interpolation, extrapolation, etc.
– We remain clueless
4
Machine Learning
• Analyze Historical Data (input and result – training set) to discover Patterns & Models
• Iteratively apply Models to [additional] Input (test set) and compare model outcome with known actual result to improve the model
• Use Model to predict outcome for entirely new data
5
Data Discovery
A B C D E F G1104534 ZTR 0.1 anijs 2 36 T
631148 ESE 132 rivier 0 21 S
-3 WGN 71 appel 0 1 -
1262300 ZTR 56 zes 2 41 T
315529 HVN 1290 hamer 0 11 -
788914 ASM 676 zwaluw 0 26 T
157762 HVN 9482 wie 0 6 -
946681 DHG 42 rond 1 31 T
-31539 WGN 2423 bruin 0 0 -
47338 HVN 54 hamer 0 16 P
6
Scatter PlotAttribute F (Y-axis)vs Attribute A
-500000 0 500000 1000000 150000005
1015202530354045
Y-Values
Y-Values
7
Scatter PlotAttribute F (Y-axis)vs Attribute A
1960 1970 1980 1990 2000 2010 202005
1015202530354045
Age of Lucas Jellema vs Year
Y-Values
8
Data Discovery
Time City - - #Kids Age Level of Education
1104534 ZTR 0.1 anijs 2 36 T
631148 ESE 132 rivier 0 21 S
-3 WGN 71 appel 0 1 -
1262300 ZTR 56 zes 2 41 T
315529 HVN 1290 hamer 0 11 -
788914 ASM 676 zwaluw 0 26 T
157762 HVN 9482 wie 0 6 -
946681 DHG 42 rond 1 31 T
-31539 WGN 2423 bruin 0 0 -
47338 HVN 54 hamer 0 16 P
9
Machine Learning, Data Mining & Predictive Analytics
10
Recent developments
• More compute capacity, smarter algorithms and better analytical tooling– Evolving Machine Learning – Smart text and speech analysis (NLP, ESA)– Real time predictions become a reality– Streaming (event) Analytics– Visualization– Citizen Data Scientist– SQL against Big Data
• More data available & accessible (IoT, Social, Media, IT operations,business processes,…)
• Better/larger/cheaper/faster data storage capabilities
11
Many cloud services around Big Data & Analytics
12
Big Data Integration Reference Architecture
IngestPrepare
Transform, Merge, Enrich
Save
GovernGovern
Serve
Analyze & Act
Present, Leverage & ‘Action’
Extract
Explore
Purge
13
Data Integration Platform
14
Mapping Oracle portfolio to Reference Architecture
Big Data Discovery
Data Visualization
BI CS
IT Analytics
Security Analytics
Log Analytics
15
Tip: OEMM - Oracle Enterprise Metadata Management
16
Citizen Data Scientist
• Data Visualization CS
• Big Data Preparation CS• Big Data Discovery CS
• Machine Learning CS
17
Oracle Machine Learning Cloud Service
18
19
20
https://www.zeppelinhub.com/
21
Relational Data& friends
22
Traditional approach
• All enterprise data is in the Oracle [relational] Database– Except very unstructured documents - and sometimes even those
23
Center of the Data Universe is shifting
24
Variety in data – Data Tiering
• How long relevant (hot vs cold vs dead)?• How fine grained and how accurate?• What is it used for?
– By whom, where, in what way, using which tools• What format is it in/should it be in?• How to be processed?• What confidentiality & integrity is required?• How much of it?
25
Trends around data storage and data processing
• Take processing to data [to reduce data movement]– Exadata SmartScan in Storage Cells (SQL & R processing)– Hadoop MapReduce/Spark– Coherence Processors– Streaming Analytics– Microservices, stand alone data domains
• Distributed data partitions – for scalability and parallelization [and fault tolerance when also replicated]: – Shards (Oracle Database 12cR2) and Partitioned External Tables– TimesTen Velocity Scale – distributed In-Memory OLTP– Hadoop HDFS, Apache Kafka
• New paradigms regarding transactional data– CQRS (for example Oracle Database In Memory (read) / In Flash/On disk
(read/write), Write behind cache)– Event Sourcing, Transaction Log
26
Oracle Database
• How much of your data– Arrives through (business) transactions that require true ACID?– Is involved in current business operations?– Will ever be updated [again]?– Plays a direct role in integrity [of other records]?– Is actively accessed [on a regular basis] ?– Really has to be in the OLTP engine?
• How much of the data currently in your OLTP engine could be off-loaded– If that data remains accessible through SQL (even from within the OLTP engine,
without altering existing queries) with reasonable response times• What if such off-loading
– Improves performance of the OLTP engine for transactions– Shortens batch jobs [by engaging distributed, scale out processing options]– Opens up possibilities for advanced analytics– Potentially lowers the cost [licenses & specialized hardware] for the OLTP engine– Introduces some change and complexity
27
Oracle Big Data SQL
• Big Data SQL: A ‘franchised query engine,’ enables scalable, integrated access in situ to the entire Big Data Management System (BDMS)– Meta data, Query execution, Workload Management, Data Optimization– Primary role for Oracle Database – foundation for BDMS
See Statement of Direction: http://www.oracle.com/technetwork/database/bigdata-appliance/overview/sod-bdms-2015-04-final-2516729.pdf
28
Oracle Database Development
29
Oracle Forms
• Release 12.2.1.1 is available– 12.2.2 (or 12.2.1.2) is planned for late 2016– Support for Forms 12c: Premier Oct 2020, Extended 2023 – (and moving)
• Forms usually runs in browser – using the Java JRE plugin for Applets– Modern browsers have stopped or will stop supporting the Java plugin– Forms will either have to run on outdated browsers (IE, old versions of Firefox or
Chrome) or run outside the browser– The main changes around Forms are around running Forms outside the browser – as
standalone Java Web Start (jnlp) application– Also: Forms Helper – script for customizing post-install environment (simplified
WLST)• On Reports:
– Reports 12c exists – it is the terminal release– From now on, reporting should be done using BI Publisher– BI Publisher has become part of the Developer Suite and will be included in the
WebLogic Suite
30
Forms in the Cloud
31
APEX – 5.1
• Interactive Grid – A new rich, client-side region type that allows editing multiple rows of data in a dynamic, JSON-enabled grid, and supports multiple grids on a single page.
• Master / Detail / Detail – Provide a wizard interface to define declarative master/detail/detail regions.
• New Charting Engine – Include a new JavaScript (Oracle JET) based charting engine developed by Oracle which produces responsive and accessible HTML5 charts.– AnyChart is on the way out
• Ability to have multiple tabs open to the same APEX application and isolate session state
• Improved Wizards - fewer steps and more attributes set by default.• Declarative RTL Support –declarative methods to control user interface direction-
support for both Left-to-Right and Right-To-Left languages.• Packaged Applications – Improved framework and enhancements to the
packaged applications.• Status: EA 2 is available (hosted) as of September 2016
– APEX 5.1 Production – early 2017?
32
New in SQL in 12cR2
• Listagg improvements• Error handling for CAST function & new Validate_Conversion function• Materialized Views
– Real Time Materialized Views (stale plus logs)– Statement Level Refresh
• AL32UTF8 As the Default Database Character Set• New capabilities for generating JSON documents directly from SQL
queries, improved JSON support in In Memory processing• Beyond 12cR2• Approximate Query Processing (using HyperLogLog)• Analytical Views• Band Join- better performance for non-equijoins• Temporary, cached in memory tables for duration of cursor• Partitioned External Tables
33
New in PL/SQL in 12cR2
• Deprecated procedures and functions• Accessible by at procedure or function level• JSON support: generation of JSON documents using PL/SQL API and
Oracle supplied Object Types (somewhat akin to XMLType)– JSON SQL functions available in PL/SQL expressions
• Supplied package dbms_plsql_code_coverage to identify code units not touched in specific [test] scenarios
• PL/Scope enhancements – more fine grained reporting
• Edition Based Redefinition does ‘garbage collection’ –editioned objects no longer in use are cleaned up
34
Other Database Development News
• JS Stored Procedures• SQL Developer GUI Debugger
– One session can have another start debugging– At breakpoint: execute SQL to inspect run context – including PL/SQL state
• SQLcl
• ORDS – Oracle REST Data Services
35
36
Summary of Oracle OpenWorld 2016
• (5 days filled to brim with 1800+ sessions, 12 keynotes, 150+ demo booths, hundreds of vendors and quite a few rumors & hallway tales)
• Infrastructure [as a Service]– Generation 2 Data Center– Network & IOPS (storage, NVMe, Flash)– Exadata SL
• Abdication of the single, central, enterprise Oracle RDBMS – and the limelight for data– PDBs – Sharding– Hadoop & Spark (& SQL & R)– Machine Learning
• Adoption of open source projects, industry trends & community darlings– Node.js, Docker, Microservices, Git(Hub), Python, Slack, …
37
Summary of Oracle OpenWorld 2016 (2)
• Cloud [First] strategy– Migrate & Extend i/o [bidirectional] Lift & Shift– Cloud@Customer– Status and future of On Premises software (and yet Engineered Systems)– Ops in Oracle Data Centers– Subscription Models, Suites (i/o a la carte)– How fast can Oracle move [without spreading itself too thin]?
• SaaS and [Unlimited] Applications– SaaS portfolio quite extensive– UX is important asset of the SaaS applications– Real cloud elements are improving (APIs, extensibility)– Traditional Apps are still evolving [as promised] – and seem to benefit from SaaS and
technological advances across the board• Oracle Public Cloud consistency, architecture and the Dogfood Doctrine
– Fabric and foundational components– Designated capabilities and mutual integration
38
Oracle OpenWorld 2016 Tag Cloud