Meeting du Swiss User Group

www.swiss-user-group.org

Focus on IoT, Analytics and Power BI

The Future of Microsoft Data Strategy

Speaker : Vincent ThavonekhamMicrosoft Practice Manager @VISEO

Microsoft MVP Azure

V2.2

@vThavo

Speaker : Vincent Thavonekham, Microsoft Practice Manager @VISEOMVP Azure, P-Seller ALM/DevOps

www.thavo.com @vThavo

Microsoft User Group LyonAzure User Group France

Author & co-author DevOps et Azure(France & international)In 2015 :- White paper: Zend / Programmez!- Round table: live streaming

for Electric Cloud, with Gene Kim (The Phoenix project)

@vThavo

AGENDA (60 min ~ 1 slide per min)

• Description of the Data, traditional vs. new

• What are the problems ?• Solutions, with a focus on Power BI• Conclusion

@vThavo

CONTENT

• Description of the Data (traditional vs. new)

• What are the problems ?• Solutions (with a focus on Power BI)• Conclusion

@vThavoCredits : http://blogs.technet.com/b/dataplatforminsider/archive/2013/06/03/sql-server-2014-unlocking-real-time-insights.aspx

INTRODUCTIONThe Data is an “old” topic addressed by SQL Server

@vThavo

VALU

E

DIFFICULTY

INTRODUCTIONKey types of analysis in the Big DataTraditional BI has evolved. Microsoft has grouped them into 2 categories:

Descriptive and Advanced Analytics (part of the « Data platform »

Source :Gartner

Data Scientists (Senior Doctor in Maths,

Astrophysics, …)

Data Platform leaps in innovation in 2015 !2012

SQL Server 2012

2013

HDInsight

“Our industry does not respect tradition – it only respects innovation.” – Satya Nadella

2015

Azure Machine Learning

Azure Document DB

HDInsight on Linux, HBase, Storm

Azure SQL DB with In-memory OLTP

Cortana Analytics Preview

IoT Services

2014

SQL Server 2014

APS

Azure SQL Database

Power BI

+ BI DataZen+ Revolution R

Note : OLAP : On Line Analytic Processing OLTP : On Line Transaction Processing

Source : Microsoft

@vThavo

INTRODUCTIONKey actors in 2015

2015 2015

@vThavo

Traditional data warehouse approach

BI & analyticsETL

Staging

+ Sharepoint + Power BI + O365

@vThavo

Breaking points of the traditional approach

Staging

Increasing data volumes

1

Real-time data

2

New data types

3

Cloud-born data

4 $100B spend on cloud

50% large orgs have hybrid by 2017

40% CRM sold are SaaS

50x Data growth 2010-2020

40ZB Digital Universe 2020

1Trillion Web pages

Source : Microsoft Convergence

@vThavo

INTRODUCTIONKey actors in 2016

2015 2016

Spectacular progress of the TOP 3 Leaders !

@vThavo

CONTENT

• Description of the Data (traditional vs. new)

• What are the problems ?• Solutions (with a focus on Power BI)• Conclusion

@vThavo

What are the problems ? And how to address them ?

• Many innovations in 2015• Too many data• Too many data types• Cold and hot data (i.e. traditional and real-

time)• Need for Agility (don’t know yet how to use

data)• Need really powerful compute to process

@vThavo

(Continued) What are the problems ? And how to address them ?

• EXCEL is no more the key peace of Microsoft Agile BI,

• Reporting Service is less and less used,

• OLAP cubes are no more the automatic solution

Everyone

3rd waveEnd user BI

A new generation of BI Today, BI extends to everyone

Analyst to end user

2nd waveSelf-service BI

IT to end user

1st waveTechnical BI

@vThavo

FRUSTRATIONS : The old Power BI and its « WOW » effect• Beyond the “WOW effect”, you’ll find the « Ouch » effect !

Ouch : EXCEL 250 Mb

limitations!

Ouch : what ?!?PowerQuery, PowerPivot, PowerView, PowerMap

Ouch : Sharepoint ?Ouch : Silverlight ?

Ouch : Excel Plugins !

Ouch: Office licences 2010/2013/365 Pro

Ouch : Need admin rights ?

Ouch : … « and what else ?? »

@vThavo

Event Queuing System

Collection Presentation and action

Event producers Transformation Long-term storage

Event Hubs

Storage adapters

Stream processingCloud gateways

(web APIs)

Field gateways

Applications

Search and query

Data analytics (Excel)

Web/thick client dashboards

Live Dashboards

Apache Storm on

HDInsight

Devices to take action

Kafka /RabbitMQ /ActiveMQ

Web and Social

Devices

Sensors / IoT

Azure Stream Analytics

HDFS

Azure DBs

Azure storage

HBase

CURRENT BIG DATA ECOSYSTEM !

Source : Microsoft

IoT AS AN INPUT TO THE BIG DATAZOOM

@vThavo

CONTENT

• Description of the Data (traditional vs. new)

• What are the problems ?• Solutions (with a focus on

Power BI)• Conclusion

@vThavo

SOLUTIONS (some of them!)

• Microsoft is Opening even more to the world,• Reinventing Microsoft• Many Open Source solutions supported• Following standards

• SaaS + VERY quick Onboarding• Start small with Free offers• “Suites” to simplify usage: Azure IoT Suites, Cortona Analytics Suites,• Focus on PowerBI.com and PowerBI Desktop

MICROSOFT

ANSWERS

Robust Architecture : LambdaStandards + Generic architecture = OPEN to non Microsoft technologies

Storage Cold Path Analysis

Apache Spark

Apache StormHot Path AnalysisPresentation

ApacheKafka

IoT Hub

Apache Storm on HDInsight

Apache HBASEon HDInsight

Hybrid : MS + Open Source

“Lambda Architecture”

MICROSOFT

ANSWERS

Azure IoT SuiteZOOM

« Think big, start small » : Microsoft referenced architecture

• Asset management• Remote monitoring• Preventive maintenanceMICROSOFT

ANSWERS

Cortana Analytics Suite

http://pocavenet-azure-iot-dashboard.azurewebsites.net/

ZOOMMICROSOFT

ANSWERS

Global Vision of the Microsoft IoTMICROSOFT

ANSWERS

@vThavo

Event Queuing System

Collection Presentation and action

Event producers Transformation Long-term storage

Event Hubs

Storage adapters

Stream processingCloud gateways

(web APIs)

Field gateways

Applications

Search and query

Data analytics (Excel)

Web/thick client dashboards

Live Dashboards

Apache Storm on

HDInsight

Devices to take action

Kafka /RabbitMQ /ActiveMQ

Web and Social

Devices

Sensors / IoT

Azure Stream Analytics

HDFS

Azure DBs

Azure storage

HBase

CURRENT BIG DATA ECOSYSTEM !

Source : Microsoft

@vThavo

Tendency with reporting / BI• Reporting Services are “brittle” • and needs IT experts to modify

• No more Reporting Services within TFS/VSTS• Reports not based on OLAP cube anymore• Replaced by a lightweight real-time reports and Power BI.com (1)

• No more Power BI inside EXCEL,• Replaced by a lightweight standalone Power BI.com

(1) https://blogs.msdn.microsoft.com/visualstudioalm/2016/01/11/announcing-reporting-capabilities-for-visual-studio-team-services

@vThavo

Zoom on Power BIStarting with a definition

“Power BI is a suite of business analytics tools to analyze data and share insights.

Monitor your business and get answers quickly with rich dashboards available on every device.”

MICROSOFT

ANSWERS

@vThavo

Zoom on Power BIStarting with a new Portal

===== December 31th 2015 ===== https://powerbi.com

@vThavo

New and easy prices

Version pro

Version gratuite et pro

===== December 31th 2015 =====

@vThavo

IT challenges

Management of Data in the cloud Security, private data, refresh and data updates.

Need to ease the usage

Need an easy tool for both the IT and the business users, to decrease time spent to create reports and visuals.

Keep data up to date Complex to refresh data from various sources, and manage increasing data volumes due to real time.

Integrating with the existing Time and cost.

IT Administration tasks Time spent for high SLA. Cost of the servers and maintenance.

Various Data sources Complex to connect to both On Premise and Cloud (SaaS) Data.

@vThavo

Power BI – Key benefits and differentiatorsPre-built content packs, consisting of

dashboards and reports, for popular SaaS solutions

Integration with familiar Microsoft products, and commitment for scale and availability in Azure

Intuitive data exploration by using natural language query

Real-time dashboard updates

Fast deployment, hybrid configuration, secure, and integrated with existing IT systems

Secure, live connection to your data sources, on-premises and in the cloud

@vThavo

How does Power BI addresses both the IT challenges and the business needs ?

@vThavo

Data connections Version gratuite et pro

@vThavo

Connectorsto popular SaaS applications

Aujourd’hui :

Soon:

Version gratuite et pro

@vThavo

Quick start samples Version gratuite et pro

@vThavo

Create reports

Desktop

Version gratuite et pro

@vThavo

Power BI DesktopPower BI Desktop

Desktop application to explore data and create interactive reports.

Easy usage, with Drag & Drop experience.

Publish directly to PowerBI.com.

Version gratuite et pro

@vThavo

Reports Version gratuite et pro

Azure Stream Analytics

Azure SQL/DW

HDInsight

AS Tabular

PBI Desktop

Excel

@vThavo

New custom visuals: Gallery (subset)

Funnel

Combo Chart

Gauge

Tree Map

Fill Map

Version gratuite et pro

@vThavo

Dashboard

Azure Stream Analytics

Azure SQL/DW

HDInsight

AS Tabular

PBI Desktop

Excel

Version gratuite et pro

@vThavo

« Ready to go » content packs :to be used against popular SaaS applications

Version gratuite et pro

@vThavo

« Ready to go » content packs :to be used against your data sources

Version pro

@vThavo

Live data sources Version pro

Consume live data sources with full interactivity

@vThavo

Power BI Personnal Gateway Version pro

Dataset

Access your on-premise data using the Data Connectivity Gataways

(Personal and Data Management)

@vThavo

Data updates : Refresh strategies

Real time data streaming in your dashboard- Using Power BI REST API- Using Azure Stream Analytics

Live connectivity to Microsoft SSAS - Auto-refresh- The Data are updated when the user accesses a dashboard or a report

Planned or automatic refresh- SaaS connectors (updates every 24h or planned or on-demand)

- Personal Gateway : planned or on demand refresh

Data refresh architecture

Live query

Popular SaaS Solutions

Live dashboards and reports

Dynamics Marketing

Power BI

SQL Server Analysis Services (SSAS)

Secure Credential Store

PRO Version1 million rows/hour

FREE Version10 000 rows/hour

PRO VersionEvery hour

FREE VersionEvery day

https://support.powerbi.com/knowledgebase/articles/474669-refresh-data-in-power-bi

@vThavo

Sharing reports and personal Dashboards

Version gratuite et pro

Share dashboards with others within your orgianization. • Shared dashboards (and related reports) are Read-Only. • They cannot be saved volontarily• The users cannot access the underlaying datasource• The users always have an up to date view

@vThavo

Team working collaboration

Version pro

Team working with Office 365 groups within Power BI

Create, publish and display contents packs of the company

Manage access control and sharing via Active Directory

groupes

@vThavo

Stay connected from any mobile devices

• Support for iPad, iPhone, and Windows

• Get alerts when your data changes•

Share and collaborate with your team. Take collaborative décisions

Version gratuite et pro

@vThavo

Query your data by asking a question in Natural language

Version gratuite et pro

@vThavo

Push data

Push your data directly into Power BI from your apps.

Integration

Add Power BI to the workflows of your apps.

Develop a real-time connection from your data sources, applications, IoT devices, …

Real-time

{ name":{String}, "tables":[ ..." ]}

0110010100001010

“name”:[string],“columns”:[String]“daType”:[String]

{ "name":{String}, "columns":[ { "name":{String}, "dataType":{String} },

Visuals

Create your own visuals for Power BI

Power BI pour les developpeurs

Version gratuite et proDéveloppements spécifiques

@vThavo

Power BI REST APIsPower BI Desktop

Prepare Explore ShareReport

0110

001

0001

0110

Data sources

SaaS solutionse.g. Marketo, Salesforce, GitHub, Google analytics

On-premises datae.g. Analysis Services

Organizational content packsCorporate data sources or external data services

Azure servicesAzure SQL, Stream Analytics…

Excel filesWorkbook data / data models

Power BI Desktop filesData from files, databases, Azure, and other sources

Power BI

Power BI service

Data refresh

Visualizations

Live dashboards

Content packs Sharing & collaborationNatural language query

Reports

Datasets0100110101

Power BI Overview

@vThavo

Very frequent updates• Example in November 2015

@vThavo

Only ONE access point : Powerbi.com(for both Desktop and Cloud Service)

Sign up for a free Preview account 

Take the Power BI Tour

@vThavo

EXAMPLE 1 : Integration with Azure servicesReal-time dashboards with Azure Stream Analytics

Use Azure Stream Analytics to push live, streaming data to Power BI• Enables real-time dashboards at scale, over data from devices and applications• Can stream millions of events per second• Can perform aggregation over time windows

@vThavo

EXAMPLE 2 : Integration with Azure servicesAdvanced analytics

Coalesce Azure services together to drive advanced analytics:

• Azure HDInsight: Big Data processing• Azure Machine Learning:

Predictive analytics• Azure Data Factory:

Orchestration at scale

@vThavo

CONCLUSION• STOP thinking Data the traditional way

• Nadella Satya : « Mobile First, Cloud First »

• “Microsoft: opening the Windows to the world” ((ref Edouard Danel, 2016)

• “From the inside to the outside” & “From the outside to the inside”

• Microsoft is facilitating (again) the adoption of the new technologies• The NEW Microsoft Power BI (Power BI .com and Power BI Desktop)• Azure Machine Learning, Cortona Analytics, …

@vThavo

Next eventWorldwidein Lyon, France,www.Thavo.com

April 16th 2016

MY SLIDES ARE AVAILABLE ON SLIDESHARE

REFERENCES

http://www.slideshare.net/VincentThavonekham/the-future-of-microsoft-data-strategy-iot-analytics-with-a-focus-on-power-bi

April 2015, Office Mix, IoT and Stream Analytics : https://mix.office.com/watch/12fv2cpp112bv

March 2015 http://www.widriksson.com/power-bi-limitations-considerations March 2015 http://sqlmag.com/power-bi/introducing-free-power-bi-designer-desktop-app Sept 2015 http://www.lemondeinformatique.fr/actualites/lire-pourquoi-power-bi-est-l-avenir-d-excel-62259.html

Meeting du Swiss User Group

www.swiss-user-group.org