Oracle Metadata Management v12.2.1.2.0 New Features Overview · Oracle Metadata Management version...

An Oracle White Paper

August 18th, 2017

Oracle Metadata Management v12.2.1.2.0 New Features Overview

Oracle Metadata Management version 12.2.1.2.0 – August 18th, 2017 New Features Overview

1

Disclaimer

This document is for informational purposes. It is not a commitment to deliver any material, code, or

functionality, and should not be relied upon in making purchasing decisions. The development, release,

and timing of any features or functionality described in this document remains at the sole discretion of

Oracle.

This document in any form, software or printed matter, contains proprietary information that is the

exclusive property of Oracle. This document and information contained herein may not be disclosed,

copied, reproduced, or distributed to anyone outside Oracle without prior written consent of Oracle.

This document is not part of your license agreement nor can it be incorporated into any contractual

agreement with Oracle or its subsidiaries or affiliates.


2

Table of Contents Executive Overview ............................................................................ 3

Oracle Metadata Management 12.2.1.2.0 .......................................... 4

HARVESTED MODEL CHANGE DETECTION .............................. 4

SUBSCRIPTION / NOTIFICATION ................................................. 4

CUSTOM ATTRIBUTES ................................................................. 4

BUSINESS GLOSSARY WORKFLOW .......................................... 5

DATA INTEGRATION BROWSING AND LINEAGE ....................... 5

DATA CONNECTION STITCHING MAJOR IMPROVEMENTS ...... 6

BULK DATA MAPPING .......................................................................... 6

LIGHTWEIGHT MODELS ....................................................................... 6

MODEL COMPARE/MERGE ............................................................. 6

DATA MODELING .......................................................................... 7

REST API SDK............................................................................... 7

Metadata Harvesting (Model Bridges) Improvements ......................... 7

METADATA HARVESTING FROM CLOUD DEPLOYED MM/DG SERVERS 7

DATA STORES (Flat files, RDBMS, Cloud Databases, Big Data / Hadoop): 8

DATA INTEGRATION (DI/ETL/ELT): .............................................. 9

BUSINESS INTELLIGENCE (BI/OLAP):......................................... 9

DATA MODELING (DM) and Object Modeling:............................. 10

METADATA MANAGEMENT (MM) & DATA GOVERNANCE (DG):10


3

Executive Overview

The Oracle Metadata Management (OMM) solutions include two products:

• the Oracle Metadata Management for Oracle Business Intelligence (OMM4OBI)

• and the Oracle Enterprise Metadata Management (OEMM)

Oracle Metadata Management for Oracle Business Intelligence is a software package for metadata management of Oracle environments. Oracle Metadata Management for Oracle Business Intelligence includes the following metadata management features:

• Metadata Harvesting from Oracle technologies

• Metadata Configuration and Stitching

• Metadata Browsing, Search and Reporting

• Metadata Collaboration (external URL, tagging, comments and review)

• Data Flow Lineage & Impact Analysis

• Metadata Explorer (simplified metadata user interface for business users)

Oracle Enterprise Metadata Management is a software package for metadata management of multi-vendor environments along with support for data governance. Oracle Enterprise Metadata Management includes all features of Oracle Metadata Management for Oracle Business Intelligence with the following extra metadata management features:

• Metadata Harvesting from multi-vendor technologies

• Metadata Version and Configuration Management (change management)

• Data Model Diagram Visualizer and Navigator

• Business Glossary for Data Governance

• Semantic Lineage & Impact Analysis

• Semantic Mapping Editor

• Data Flow Mapping Specifications Editor

• Data Documenter

This whitepaper describes in detail some of the new features and capabilities offered in Oracle

Metadata Management with the release of version 12.2.1.2.0.


4

Oracle Metadata Management 12.2.1.2.0

HARVESTED MODEL CHANGE DETECTION

This feature avoids systematically creating a new version of the content when scheduling the harvest of large databases, data models, data integration jobs, or business intelligence servers. Change detection is automatically performed during any new harvesting by a systematic and efficient (fast) comparison with the previously harvested metadata. This ability dramatically reduces the needed database space by creating fewer versions, and enables the possibility of reliable notification when a change really occurred. Note that in previous versions, the incremental harvesting already offered systematic and automatic reuse of unchanged models (e.g. BI Reports) from a large a multi-model server (e.g. BI server).

SUBSCRIPTION / NOTIFICATION

OMM has added a new mechanism to operate in response to changes at both the repository model level (e.g. a new version of a data store is harvested) and at the model object level (e.g. a terms changed or ready to approve/review in a Business Glossary under Workflow). Anyone may be assigned stewardship roles and thus will be notified as new imported content is harvested, with links back to the new object and the ability to compare using the newly re-written powerful comparator, and the ability to identify impacts of change for any architecture or configuration of assets.

CUSTOM ATTRIBUTES

Custom attributes may now be applied on any harvested metadata (models) from data stores, data modeling, data integration, and business intelligence tools. Thus allowing to tag any metadata (from Hive tables to BI reports down at the column/field level) for custom properties such as a company confidential level that can be read for external products on security enforcement and compliance. Note that in previous versions, these same custom attributes could already be applied to authored metadata (models) such as terms of Business Glossaries or tables/columns of Data Models.


5

BUSINESS GLOSSARY WORKFLOW

Even with workflow enabled, business glossary terminology may now be edited by the business users behind the Metadata Explorer UI. Note that the in previous version, a (customizable) Workflow was already available in the Metadata Manager.

DATA INTEGRATION BROWSING AND LINEAGE

Allowing business users behind the Metadata Explorer UI to browse data integration jobs (from DI/ETL/ELT tools as well as SQL Scripts) and analyze summary data flow lineage of their execution. Note that in previous versions, the browsing (open) and full detailed data flow lineage of any data integration models (DI/ETL/ELT and SQL scripts jobs) were already available in the Metadata Manager (and still are).


6

DATA CONNECTION STITCHING MAJOR IMPROVEMENTS

OMM is now providing the ability to stitch by column position (needed in SQL insert statements of ETL scripts ), and smart case aware stitching (as some database's name spaces like schema/table/column are case sensitive while others are not).

BULK DATA MAPPING

Data mapping specifications may now be defined as bulk mappings for all columns in a table.

LIGHTWEIGHT MODELS

Model content (such as a harvested databases or data models) can be stored in the repository as a lightweight model (just the XML file), or fully expanded (as both XML file and fine grained repository objects). When retaining many historical versions of a model, using lightweight models saves repository space, and also avoids slowing down the search by not indexing historical repository objects. Lightweight models cannot be directly used in a Configuration or in a Mapping. However, the lightweight model of a data store (such as a RDBMS or Hadoop Hive) can be documented with a reference to its associated Physical Data Model (PDM) for data model diagramming and semantic linking to a Business Glossary (BG). Such a PDM can of course be used in any Mapping or Configuration to be exposed to business users in the Metadata Explorer. Note that lightweight models can immediately (without any loss of performance) be opened in the Metadata Manager (to browse metadata or trace lineage within that model), Compared (with the Model Comparator to analyze the difference between versions), Exported (for example to BI design tool).

MODEL COMPARE/MERGE

The comparison facility has been completely re-written to include comparison for every level of detail for those models with the same profile (e.g., data model from one technology and data model from another). Even entirely different contents (e.g., an ETL and a data model, or a BI Design and Glossary) may be compared, only at a lessor level of granularity (basically at the granularity of stitching, e.g., schema, table, column). Finally, for physical data models (including documentable models based upon harvested database structures) one may use a powerful merge feature again with full control down to any level of granularity


7

DATA MODELING

Major improvements and new features involving the physical data modeling capabilities centered around a data documentation tool enabled through physical data modeling based upon the structure harvested from existing data stores of the of data warehouse and data lake (including traditional RDBMS and Hadoop Hive big data). Also includes new advanced support for physical data model diagram editing, relationships definition, all the way to export to data modeling tools, business intelligence design, etc., and comparison/compliance with live database / big data hive implementations..

REST API SDK

major enhancements with many new features.

Metadata Harvesting (Model Bridges) Improvements

METADATA HARVESTING FROM CLOUD DEPLOYED MM/DG SERVERS

Two possible deployment architectures for the MIMB server:


8

• Local Network: For a Model Bridge Server connected to a local Metadata Management Server via a dedicated open port on the same network where firewall is not an issue. In such case, the Model Bridge server is controlled by the Metadata Management Server in a classic client/server architecture.

• Over the Web (Cloud): For a Model Bridge Server connected to a remote Metadata Management Server over the Web where a secure firewall is in place. In such case, the Model Bridge Server will proactively connect over http on regular basis to a dedicated remote Metadata Management Server for any metadata import/export activities to perform. This is the recommended architecture for Metadata Management Servers hosted on the cloud needing to access local Model Bridge Servers behind a firewall for metadata harvesting purpose.

DATA STORES (Flat files, RDBMS, Cloud Databases, Big Data / Hadoop):

o DATA CATALOGING OF THE DATA LAKE over the cloud (e.g. Amazon S3), on

big data clusters (e.g. Hadoop distributions), or file servers (POSIX file systems)

- with high level data lake directory inventory import bridges such as:

▪ Amazon S3 (Simple Storage Service) import (beta bridge)

▪ Apache Hadoop Distributed File System (HDFS Java API) import or

(WebHDFS REST API) import (beta bridge)

▪ Posix (Windows/Linux) File System import (beta bridge)

- discovering lower level metadata from the database and file import bridges

such as:

▪ Flat File Database (CSV or Excel) import (beta bridge)

▪ JSON (JavaScript Object Notation) Database Files import

▪ Apache Avro Database Files (JSON) import

▪ Apache Parquet Database Files import

o BIG DATA HADOOP HIVE TECHNOLOGIES (Cloudera, Hortonworks, MapR)

▪ Apache Hadoop Hive Database (HCatalog via JDBC) import

▪ Apache Hadoop Hive Database (Web Hcatalog) import

▪ Apache Hadoop Hive Database (Hive Metastore via JDBC) import (beta

bridge)

o NO SQL TECHNOLOGIES

▪ MongoDB Database (JSON) import

▪ Apache CouchDb Database (JSON) import

▪ MarkLogic NoSQL Database (for JSON only) import

▪ Apache Cassandra Database import

▪ DataStax Enterprise (Cassandra) import


9

o SQL New Databases

▪ Actian Vector (Vectorwise) Database Database (via JDBC) import

▪ HP Vertica Database (via JDBC) import

o SQL Stored Procedures and Functions (major SQL parsing improvements for

lineage)

▪ Microsoft SQL Server Database (via JDBC) import

▪ Oracle Database (via JDBC) import

▪ Pivotal GreenPlum Database (via JDBC) import

▪ PostgreSQL Database (via JDBC) import

▪ Teradata Database (via JDBC) import

o SQL Scripts (major SQL parsing improvements for lineage)

▪ Apache (Cloudera, Hortonworks, MapR) Hadoop HiveQL Script import

▪ IBM DB2 Universal Database (UDB) SQL Script import (beta bridge)

▪ Oracle Database SQL Script (PL/SQL) import

▪ Teradata Database SQL Script (BTEQ) import

o SQL DDL import major improvements (instead of live JDBC connections) which is

better for security reasons and for cloud architectures

▪ Apache (Cloudera, Hortonworks, MapR) Hadoop HiveQL DDL import

▪ Oracle Database DDL import

▪ Teradata Database SQL DDL import

▪ Other Database SQL DDL import (beta bridge)

DATA INTEGRATION (DI/ETL/ELT):

o New Bridges:

▪ Alteryx import (beta bridge)

o Improved Bridges:

▪ Informatica PowerCenter import bridge major improvements in performance

and memory usage.

▪ Microsoft SQL Server Integration Services (SSIS) new support for version

2016.

▪ Talend DI import bridge is now file based instead of API based, includes

major improvements for big data, and new support for MDM.

BUSINESS INTELLIGENCE (BI/OLAP):

o New Bridges:


10

▪ IBM Cognos 11 import (for existing Cognos 10 features, plus beta bridge

import of the new Cognos 11 Analytics features of Dynamic Cubes and Data

Modules)

▪ Qlik Sense import

▪ SAP BusinessObjects 4.2 import (for existing BusinessObjects 4.1 features,

however new 4.2 only features are not supported yet)

o Improved Bridges:

▪ Microstrategy import scalability and performance improvements

▪ Tableau import bridge major improvement in (1) reducing the number of

Tableau Data store connections (Upper/Lower case mix, Default vs specified

port, database to various schemas), and (2) parsing the Tableau Init SQL (In

Tableau data prep / data integration).

DATA MODELING (DM) and Object Modeling:

o New Bridges:

▪ erwin Data Modeler 9.7 import bridges

▪ IDERA ER/Studio 2016 import bridges

▪ NoMagic MagicDraw new import bridge

o Improved Bridges:

▪ Minor improvements in all data modeling bridges

METADATA MANAGEMENT (MM) & DATA GOVERNANCE (DG):

o New Bridges:

▪ Apache Atlas (for HortonWorks) import

▪ Cloudera Navigator import (beta bridge)

\ ODI 12c New Features Overview

October 2013

Author: ODI Product Management

Oracle Corporation

World Headquarters

500 Oracle Parkway

Redwood Shores, CA 94065

U.S.A.

Worldwide Inquiries:

Phone: +1.650.506.7000

Fax: +1.650.506.7200

oracle.com

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. This document is provided for information purposes only and the

contents hereof are subject to change without notice. This document is not warranted to be error-free, nor subject to any other

warranties or conditions, whether expressed orally or implied in law, including implied warranties and conditions of merchantability or

fitness for a particular purpose. We specifically disclaim any liability with respect to this document and no contractual obligations are

formed either directly or indirectly by this document. This document may not be reproduced or transmitted in any form or by any

means, electronic or mechanical, for any purpose, without our prior written permission.

Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective

owners.

AMD, Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices. Intel

and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are

trademarks or registered trademarks of SPARC International, Inc. UNIX is a registered trademark licensed through X/Open

Company, Ltd. 0410

Oracle Metadata Management v12.2.1.2.0 New Features Overview · Oracle Metadata Management version...

Documents

Transcript of Oracle Metadata Management v12.2.1.2.0 New Features Overview · Oracle Metadata Management version...