Best Practice for Data Modeling with HANA.pdf

DMM103 New and Best Practices for Data Modeling with SAP HANA

Christoph Morgen / SAP HANA Product Management, SAP SE

Public

Disclaimer

This presentation outlines our general product direction and should not be relied on in making a purchase decision. This presentation is not subject to your license agreement or any other agreement with SAP. SAP has no obligation to pursue any course of business outlined in this presentation or to develop or release any functionality mentioned in this presentation. This presentation and SAP's strategy and possible future developments are subject to change and may be changed by SAP at any time for any reason without notice. This document is provided without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. SAP assumes no responsibility for errors or omissions in this document, except if such damages were caused by SAP intentionally or grossly negligent.

2014 SAP SE or an SAP affiliate company. All rights reserved. Public 2

Agenda

Data Modeling in SAP HANA

New Approaches

Best Practices

Future Look


Data Modeling with SAP HANA

Overview SAP HANA Information Views

SAP HANA Information Views Virtual Data Modeling approach as a key SAP HANA concept

Virtual Data Flow Models

Operational Reporting | Applications | Analytics

SAP HANA PLATFORM

HANA Views

Database Layer

Physical Tables No Aggregations | single atomic copy of detail ata In-Memory Engines | Performance

Multidimensional Reporting Models

Enterprise Application Virtualize Data Models


SAP HANA Information Views Virtual Data Models for Multidimensional Scenarios

Calculation View

Reporting Tools can usually directly consume HANA Calculation Views or Analytic Views.

Multidimensional Tools support Hierarchies for Navigation, Filtering and Aggregation and HANA Prompts (Variables & Input Parameters) for efficient Pre-Filtering of Data.

Calculation Views are usually build upon Analytic-, Attribute- Views, and Column Tables

Analytical View Attribute View Column Table


SAP HANA Information Views Virtual Data Models for Normalized Data Model Scenarios

SAP HANA Calculation Views provide the means to model sophisticate views based on normalized data structures.

Complex Calculation Views demand a more explicit intent and control of the modeled set-based data flow, i.e. slicing, aggregation and filtering of sets as input to joins, unions etc.

See SAP Note 1857202

SAP HANA Calculation Views typically feed data to Business Applications, like SAP HANA XS build Applications


scenarios like

Combined use of Multiple-Fact

SAP HANA Information Views Flavors of View Modeling Approaches

Attribute Views

Compose a dimensional view

Analytic Views

Combines Fact-Tables with

Calculation View

Great flexibility for advanced use

with a series of attributes derived from a collection of tables e.g. Master Data Views

Highly re-used and shared in

Attribute-Views to Star-Schema- Approach to model custom or OLAP Cube-like objects for multidimensional reporting.

Stores no aggregates and mass- Tables/Analytics Views Analytic- and Calculation Views

Used to build Hierarchies

Hierarchies are key elements in use with Analytic View for multidimensional reporting

aggregates on the fly

Hierarchies are key for multi-

dimensional access (navigation, filtering, slicing and aggregation)

Build Models on Normalized Data

Re-Use and stack views

Make use of custom scripted views


SAP HANA Attribute Views Modeling Attribute Information in Data Models

What is an Attribute View?

... are the reusable dimensions objects adding context

to data in the analysis or data flow.

Can be regarded as Master Data-Views

Build a semantic attribute information collection

from various source tables (e.g. join Plant to Material)

Measures cannot be modeled

Re-used as dimensions in multidimensional scenarios (Analytic Views) or re-used to model complex master in normalized data model approach and master data reporting scenarios

Cannot be directly consumed by multidimensional reporting clients

Semantic Attribute Information

Attribute Data Foundation

2014 SAP SE or an SAP affiliate company. All rights reserved. SAP AG 2011 Public

SAP HANA Attribute Views Modeling Attribute Information in Data Models

What are the capabilities of Attribute Views?

Attribute Views support

Data Foundation join model (various joins types)

Calculated Attributes (static or dynamic calculations)

Description Mapping & Text-Join Lookup master data tables

e.g. used for handling of multi-language master data

Time Dimension Attribute Views

Sematic-Type specification

Hierarchies (Level, Parent-Child), hierarchy-use behavior

Define filter values on attributes & columns

incl. use of Input Parameters

Embedded search properties


SAP HANA Analytic Views Modeling Facts and Dimensions as a Multidimensional Data Model

What is an Analytic View?

Can be regarded as Cube-/OLAP Star Schema-like data model

Without storing aggregated data

Analytical View

1

consumed using multi-dimensional clients or re-used in complex data flows

1 Fact Table

Left Outer

Fact data from the Data Foundation is joined N N 1

against modeled Dimensions (Attribute Views)

fact table contain the key figures

Measures

1

N

N

N

1

Dimensions describe the key figures and enrich the data

Cardinality in star schemas is generally N:1 fact to dimension, Left Outer Joins

OLAP models are not designed to handle complex join operations

HANA Analytical Views are highly optimized for aggregating mass data

2014 SAP SE or an SAP affiliate company. All rights reserved.

SAP AG 2011

Public

SAP HANA Analytic Views Modeling Facts and Dimensions as a Multidimensional Data Model

What are the capabilities of Analytic Views?

Data Foundation Model for Star Schema Fact data

Measure facts derive from only one data foundation table

Measure definition from Data Foundation Facts

Distinct Count Measures, Calculated & Restricted Measures,

Currency- and Unit Conversion-Measures

Default Aggregation (sum, max, count, ...)

Re-use of slowly changing dimension data

Temporal join btw Facts and dimension data

Variables and Input Parameters

Dynamic filtering and parameter-driven calculations

UI-prompts in use with multidimensional reporting clients


SAP HANA Calculation Views Modeling custom Data Flow Scenarios

What is a Calculation View?

Calculation Views are composite views and can be used to combine other views

Can consume other Analytical-, Attribute-, other Calculation Views & tables

Approach to model custom scenarios like

Combined use of Multiple-Fact Tables/Analytics Views

Application UI

Build Models on Normalized Data

Re-Use and stack views

Make use of custom scripted views

Great flexibility for advanced use

Modeling Approaches: Graphical Modeler or SQLScript-based Editor

Calculation View


SAP HANA Calculation Views Modeling custom Data Flow Scenarios

What are the capabilities of graphical modeled Calculation Views?

Graphical Calculation Views

Views as client-consumable multidimensional query objects or

internal re-use objects or views without measures

Builds a composite view with facts from multiple sources

Embeds Analytical-, Attribute-, Calculation Views & tables

Calculation View

Data operation nodes for Union, Join, Projection & Aggregation (No SQL or SQL Script knowledge required)

Great flexibility to build complex Virtual Data Models*

and unique capabilities like dynamic joins, .

Calculation View-data flows

Will get optimized and exploit underlying engines

(i.e. OLAP Engine) where possible, prune data flow,

Attribute View

Analytical View

push-down filters

*SAP HANA Live

Calculation View Column Table


SAP HANA Calculation Views Scripting custom Data Flow Scenarios

SQL Scripts-based Calculation Views

Use of custom SQLScript coding to build Calculation Views

Utilize SQLScript CE-functions or standard SQL-statements (do not mix)

Side-effect free structures / READ-ONLY functions

Consume data from raw tables, modeled views, stored procedures, decision tables,

Semantic output structure is graphically

modeled

Graphical modeled View versus CE Functions

Result in equal performance

(e.g. field pruning, parallelization, join omission, .)

Scripted


New Approaches

SAP HANA Information Models

New approaches for SAP HANA Data Modeling

Value Help Views Variable and Input Parameter

Referencing data structures for value help definitions Variable and Input Parameter support external views or tables to generate value help lists-of-values

Supported with Analytic- and Calculation Views

View Variables and Input Parameter can be mapped to variables and input parameters from external views or tables

Allows filtering and customizing value help LOVs from external views e.g. by passing to values of initially selected variables, a LOV for a dependent variable is generated as a dynamically filtered LOV

Out-source Value Help Information to dedicated views

Benefit from faster value help dialogs

Provide consistent LOVs across consuming views

*was introduced with a revision previous to SPS07


Public

17


Modeling Dimension-Calculation Views

Dimension-Type Calculation View Similar capabilities to Attribute Views

cannot be directly consumed by multidimensional reporting clients

Composing Dimension-Attributes and -Hierarchies based on a custom CalcView-data flow

Hierarchies supporting dynamic input structures

Attribute columns used within hierarchies can ne input parameter-driven

Hierarchy properties like Root Node (e.g. Parent-Child Hierarchies) can be input parameter-driven

Usage Scenario

Dynamic hierarchy structures and properties are required

Star-Join Calculation Views (details see following slides)

Dynamically mapped attributes


New Practices for SAP HANA Data Modeling

Modeling Star Join-Calculation Views

Calculation Views with Star Join capabilities Joining fact input data flows with multiple dimension views in a single node

Fact Input flow can be any custom calculation flow of nodes

Other, only Calculation Views of Data Category-Dimension are allowed as input

Dimension Properties

All DIM-View Attributes are automatically part of the StarJoin (incl. Hierarchies)

DIM-Views are added as shared (referenced) dimensions, changes to dimension views are immediately available

Local hierarchies can also be defined

Measures, like Counters use the dimension reference, hence can reference to attributes hidden from the output

UseCase

Make use of special Calculation View Dimension capabilities (e.g. parameterized Hierarchies) or multiple fact-table input

Important Note: Star Join-Views currently cannot regarded a replacement for Analytic View capability, especially as they have not been yet fully optimized for aggregation performance. Further, Star Join-Views cannot be consumed by other Calculation Views.


Public

19


Modeling with Virtual Tables

Consuming remote data in SAP HANA Views Smart Data Access allows

To access remote data like local data

Specific HANA query optimization and execution handles functional SQL compensation, automatic data type translations, filter push-down, in order to push query processing to the underlying data source system

To enable consumption Remote data structures are registered and referenced as virtual tables

Virtual tables can also be consumed as data sources within HANA Calculation Views.

Supported external systems and restrictions are documented in SAP note 1868209


Public

20


Introducing performance analysis capabilities

Performance Analysis Mode in Modeling Environment

Introduction of performance analysis hints and indicators inside the HANA Model Editor

Manually switched on or defaulted switched on

Hints and indicators about table partitioning and number of rows (threshold as preference)

Switching on performance analysis mode

Scenario indicators for partitioned tables (icon) and

exceeded row thresholds

View details pane: indication about partitioning type by icon (hash, range, )

Performance analysis: more partitioning and row count information.



Enhanced Plan Visualization for detailed analysis

SQL Editor > Context Menu Explain Plan

Time Line View root cause analysis

New Operator List View Summarized & Filter

Visualize Plan



Modeling Productivity Error Handling/HANA Answers Integration

Extended Error Handling with SAP HANA Answers (http://answers.saphana.com/) In extension to documentation and help, the SAP HANA Answers-plugin to SAP HANA Studio enables crawling external sources of information e.g. adds information from SCN and others

Displays crowed-sourced information

embedded in HANA Studio or outside

Integrated with HANA Studio views (job log, ), editors, wizards. Called via key from selected text or feature.

Independent feature-plugin to install

F10 as your new friend


Best Practices


SAP HANA Modeling Best Practices

General Modeling Performance Guidelines

Avoid transfer ring large result sets between the HANA Database and client application

Reduce data transfer between views follow the volcano approach

Procedures

Analytical View

Calculation View

Attribute View

Do calculation after aggregation & avoid calculations before aggregation on line item level

Aggregate data records (e.g using GROUP BY, Keep flags, reducing columns, .)

Filter data amount as early as possible in the lower layers ..

Analytical Privileges

Column

Store Tables

use

Constraint filters

WHERE clause / Parameters

Analytical Privileges



Modeling Scenario Multidimensional Model or not?!

Analytical View (including Attribute views) Calculation View

STAR SCHEMA MODEL

AGGREGATION

WITH OR WITHOUT WHERE CLAUSES

Multidimensional, Star Schema Combine multiple Analytical Models using Unions (Similar Dimensions)

Attribute View (independent) Calculation View

MULTIPLE QUERY OJECTS NORMALZED MODEL COMPLEX JOINS WITH

WHERE CLAUSES

Complex Joins Complex Joins including facts



Things to watch our for Data Movement

SelectWhere Matnr = DPC1017 Group By Spart

Filters pushed

down SELECT SPART, MATNR, WERKS, SUM(KWMENG) FROM AV WHERE WERKS = ? AND MATNR = ? GROUP BY

Aggregation

Refrain from moving

large datasets between views & to

the front-end

While re-using Analytic Views in Calculation Views,

carefully select, prune, filter and slice data from a wide and multidimensional detail-

level data structure



Things to watch our for Implicit Filter Push Down

SELECT WERKS, SUM(KWMENGA), SUM(KWMENGB), SUM(TOTAL) FROM CV WHERE MATNR = DPC1017 GROUP BY WERKS

Review the Visualize Plan to make sure filters are

pushed down

Filter applied late Filter applied early

Select Werks, Matnr, Sum(KwMeng) From Sales_Av Select From Sales_Av Where Werks = 100 & Matnr = DPC1017

Select From Sales_Av Where Werks = 1000 & Matnr = DPC1017



Things to watch our for Column & Row Store operators

Review the Explain Plan! Limit the number of records used by

ROW store operators (through Aggregation & Filters)



Things to watch our for Scripted Calculation Views

SELECT FIELDA, SUM(COUNT) FROM MODEL GROUP BY FIELDA WHERE FIELDA IN Apple, Orange, Banana

SQL is static and is always executed as defined and intermediate result sets will be materialized

CE Functions will try to exploit underlying database engines and will push

filters down, prune columns and omit joins where possible


Filter applied & columns pruned late

FIELDA COUNT

Apple 3000000

Banana 4000000

Orange 9000000

Filters applied & columns pruned early. Aggregation performed by OLAP engine

Public

30

FIELDA COUNT

Apple 3000000

Banana 4000000

Orange 9000000


Tips and Tricks Input Parameters (Explicit filter push down)

SELECT CATEGORY, SUM(YEAR1), SUM(YEAR2) FROM MODEL GROUP BY CATEGORY WITH PARAMETERS (

'PLACEHOLDER' = ('$$YEAR_1$$', '2011'), 'PLACEHOLDER' = ('$$YEAR_2$$', '2012'))

Define Input Parameters in the Data Foundation to explicit and

compulsory Filter the data

Constraint Filter $$YEAR_1$$




Tips and Tricks Input Parameters (Explicit filter push down)

SELECT CATEGORY, SUM(YEAR1), SUM(YEAR2) FROM MODEL GROUP BY CATEGORY WITH PARAMETERS (

'PLACEHOLDER' = ('$$YEAR_1$$', '2011'), 'PLACEHOLDER' = ('$$YEAR_2$$', '2012'))

Define Input Parameters in the Data Foundation to explicit and

compulsory Filter the data





Tips and Tricks Union with Constants for Input Source Pruning

SELECT

ORDER

SALESORG

DIVISION AMOUNT QUANTITY

GROUP_CONSTANT

FROM VIEW WHERE

(GROUP_CONSTANT = A AND ORDER IN 1, 2,3) OR

(GROUP_CONSTANT = B AND ORDER IN 6,7,8)

Constant Column

Filtering on Constant Column will prune input sources. (i.e. Model C, D, E, F are NOT executed)



Tips and Tricks Union with Constant Values (Pivot Data)

Standard Union

Pivot table using Union with Constant values


Public

34


Tips and Tricks if joining Analytics Models is required

Join Analytic Models with caution.

Use WHERE clauses/filters to minimize the amount of records used in the join. Slicing / pruning of columns

Variables or Input Parameters for explicit filtering

exposed as UI prompts

Variable (Created On) pushed down using WHERE clause

Optimized join Where clause(s) filters data-set

before Join occurs

Variable (Delivery Date) pushed down using WHERE clauses



Modeling Scenario Normalized Data Models / Virtual Data Models

Building complex relational Model Scenarios

Identify reproducible pattern

Split big models into

smaller parts

Do not build monolithic models

Demands a more explicit intent and control of the modeled set- based data flow, i.e. slicing, aggregation and filtering of sets as input to joins, unions etc.



SAP HANA Live! Virtual Data Models for SAP Business Suite

Open Interfaces https | oData | SQL | MDX

SAP-

delivered Query Views

Reuse Views

Private Views

Physical Tables

SAP HANA Live (VDM)

Completely build on SAP HANA Calculation Views

Re-use of Real-Time

Views

Query Views Top of the SAP HANA Live view

hierarchy (VDM) and provide consumable output fields

Query Views are consumed by reports or analytical applications

Reuse Views are for use in Query Views, not for

direct consumption by reports or analytical applications

Reuse Views represent the actual data model by exposing or translating original SAP Business Suite source tables into Views

Private Views encapsulate SQL transformations on

single or multiple data base tables or Reuse views

Private Views are comparable to subroutines


Personalized Views

Personalized Views

Personalized Views

Customer Extensions


Modeling Scenario Normalized Data Models / Virtual Data Models

Optimized execution for Queries in complex relational Model Scenarios

For stacked Calculation View data models, the Calculation Engine can

generate an optimized SQL statement at runtime if SQL Engine

execution-flag is used

See SAP Note 1857202 for considerations and constraints i.e. Only Tables and Calculation views are supported etc. Improved Join ordering


Note, the optimized execution of SQL Engine- flagged models does allow for implicit OLAP Engine push-down in certain scenarios.

Public

38

Outlook


Future Directions for SAP HANA Data Modeling

Planned Innovations and Future Direction

Planned Innovations

Future Direction

Editor usability

Value help entity reference

Calculation Views

Rank node

Table function as data sources

Script-based CV enhancements

Table function support

Output column structure import

History Views support (time travel support)

Spatial support: spatial joins, spatial expressions in filters

Harmonize Development- and Modeler- Studio perspectives

Harmonize object naming

Supportability

Debugging Views with drill-down analysis, join cardinality

Logging and tracing for modeler plugins (preferences)

Productivity / object re-usability

Replace node / replace node with a data source / ..

Propagate semantics from data sources / extract semantics

WebIDE

Calculation View editor

Analytic Privilege editor

Further unification of Attribute, Analytics, Calculation View into a unified model

Consumption framework for Application Function Library Procedures

Continuous Usability enhancements

Drive easiness, enhance modeler productivity, reduce complexity, separate modeling intent from model optimization


Further Information

SAP Public Web

scn.sap.com http://scn.sap.com/community/developer-center/hana www.sap.com www.saphana.com

SAP Education and Certification Opportunities

www.sap.com/education

Watch SAP TechEd Online

www.sapteched.com/online

Related Workshops/Lectures at SAP TechEd 2014

DMM161 - Introduction to Data Modeling in SAP HANA, Hands-On Workshop

DMM270 Advanced Data Modeling in SAP HANA, Hands-On Workshop 2014 SAP SE or an SAP affiliate company. All rights reserved.

Public

41

Feedback

Please complete your session evaluation for

DMM103.

Thanks for attending this d-code session.

2014 SAP SESEororananSAP affiliate company. AllAllrights reserved. Public 4242

SAP d-code Virtual Hands-on Workshops and SAP d-code Online Continue your SAP d-code education after the event!

SAP d-code Virtual Hands-on Workshops

Access hands-on workshops post-event

Starting January 2015

Complementary with your SAP d-code registration

http://sapdcodehandson.sap.com

SAP d-code Online Access replays of keynotes, Demo Jam, SAP d-code live interviews, select lecture sessions, and more!

Hands-on replays

http://sapdcode.com/online


Thank you

Contact information: Christoph Morgen SAP HANA Product Management SAP SE | Dietmar-Hopp-Allee 16 | 69190 Walldorf | Germany [email protected] | www.sap.com



No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP SE or an

SAP affiliate company.

SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE

(or an SAP affiliate company) in Germany and other countries. Please see http://global12.sap.com/corporate-en/legal/copyright/index.epx for additional trademark information and notices.

Some software products marketed by SAP SE and its distributors contain proprietary software components of other software vendors.

National product specifications may vary.

These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or warranty of any kind, and SAP SE or its

affiliated companies shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP SE or SAP affiliate company products and services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional warranty.

In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or any related presentation, or to develop or

release any functionality mentioned therein. This document, or any related presentation, and SAP SEs or its affiliated companies strategy and possible future developments, products, and/or platform directions and functionality are all subject to change and may be changed by SAP SE or its affiliated companies at any time for any reason without notice. The information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. All forward- looking statements are subject to various risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements, which speak only as of their dates, and they should not be relied upon in making purchasing decisions.

Source: http://www.saphanacentral.com/


Best Practice for Data Modeling with HANA.pdf

Documents

Transcript of Best Practice for Data Modeling with HANA.pdf