Reza Rad

DataflowIntegration Solution for

Power BI

Reza RadConsultant, Trainer

RADACAD

Consultant, Mentor, Trainer, SpeakerMicrosoft Regional DirectorMicrosoft Data Platform MVPAuthor of SQL Server and BI booksAuthor of Power BI from Rookie to Rock Star bookPower BI Trainer for thousands of DevelopersMicrosoft Certified TrainerMicrosoft Certified ProfessionalCo-Leader of NZ BI User Group & Difinity

/rezarad @rad_reza rezaradf

Agenda

• What is Dataflow?

• Scenarios of Use Cases

• Creating Dataflows

• Entity Types

• Licensing

• Common Data Model

• CDM Folder Structure

What is Dataflow?

What is Dataflow?Dataflow is a Power Query process that runs in the cloud independently from any Power BI reports.

Where the Output Stored?Dataflow stores the data in the Azure Data Lake storage.

But I Don’t Have Azure Data Lake Subscription!

• Dataflow manages the Data Lake configurations internally. You won’t need anything except your Power BI accounts and subscriptions.

Power BI Can Do Get Data from Dataflow

What are the benefits of using Dataflow?

Sample Scenarios

Using One Power Query Table in

Multiple Power BI Reports

Using One Power Query Table in Multiple

Power BI Reports

• Re-usable tables or queries across multiple Power BI files, are one of the best candidates for Dataflow.

Different Data Sources with Different Schedule of Refresh

• Dataflow can run extract, transformation, and load (ETL) process on a different schedule for every query (or table).

Centralized Data Warehouse

• Dataflow can be the ETL engine, that fuels the centralized data warehouse in Azure data lake storage.

Versioning Data from a Data Source

Dataflow can be used for versioning the data from the

source into multiple destination tables.

Creating Dataflow

Prerequisites

Developing or Editing Dataflows are possible through Power BI service (not the Desktop)

Dataflow is only available in an app workspace (not in “My workspace”)

Administrative Control

Dataflow Demo

Computed Entities

• Created when you do Reference Query

• The main query should be “Enable Load”

Computed Entities

• Can be good for performance

• Main table is stored in Power BI Dataflow storage, and the new table will query the main table (not the data source)

Linked Entity

• Different from Computed Entity

• No transformation

• No storage

• Read only

• Just a LINK

Premium Requirement

Common Data Model

Silos of Data: Integration Challenge

Shared Data Model

Common Data Model

• Shared data model

• More than 250 entities

• Started with Dynamic 365

Business analysts

Low/no code

Data scientists, Data engineers

Low to high code

CDM folders

Enabling low friction collaboration among Data + AI professionals

Power BI

dataflows

Dynamics 365

CDS for Apps

data

Office

Substrate

Office 365

Adobe

Customer

Experience

Platform

SAP

C/4HANA

S/4HANA

Power BIAzure Data Services

data cataloging, data prep, AI,

machine learning, data warehousing

Azure IoT

ISV partners

Custom

LOB +

Developer

resources

Benefit of CDM

Decoupling Applications from data sources

Industry Accelerators

• Pre-packaged applications working with CDM

ExampleHigher-Education Accelerator

CDM Internals

Data Storage for CDM

• Azure Data lake

• CDM Folders

Folder Structure

What is the structure of CDM Folder?

• Metadata: Model.json

• Data Files: CSV

Model.json

• Root elements: description, last modified time, data culture

• Entity information:

• Reference models

• Relationships

• Annotations

• Pbi:mashup: Transformations

• https://docs.microsoft.com/en-us/common-data-model/model-json

Dataflow and REST API

Licensing?

Summary

• What is Dataflow?

• Scenarios of Use Cases

• Creating Dataflows

• Entity Types

• Licensing

• Common Data Model

• CDM Folder Structure

Power BI Book

• https://www.apress.com/gp/book/9781484240144#otherversion=9781484240151

References to Study MorePower BI from Rookie to Rock Star book: FREE

http://www.radacad.com/online-book-power-bi-from-rookie-to-rockstar

Reza Rad’s series on Dataflow

• What are the Use Cases of Dataflow for You in Power BI?

• Getting Started With Dataflow in Power BI – Part 2 of Dataflow Series

• What is the Common Data Model and Why Should I Care? Part 3 of Dataflow Series in Power BI

• Linked Entities and Computed Entities; Dataflows in Power BI Part 4

Matthew Roche’s series on Dataflow

Matthew written great series and resources for dataflow read this post.

Microsoft Documentation: Dataflow

https://docs.microsoft.com/en-us/power-bi/service-dataflows-overview