Beyond Excel: Enhancing Your Data Analysis with … · Risk Analysis Logistics Retail ... portfolio...
Transcript of Beyond Excel: Enhancing Your Data Analysis with … · Risk Analysis Logistics Retail ... portfolio...
1© 2016 The MathWorks, Inc.
Beyond Excel: Enhancing Your Data
Analysis with MATLAB
David Willingham
Senior Application Engineer – Data Analytics
2
“Data is the sword of the 21st century, those who
wield it the samurai.”
Big data — how to create
it, manipulate it, and put it
to good use.
“If you want to work at
Google, make sure you
can use MATLAB.”
Google’s Former SVP - Jonathan Rosenberg
3
Medical Devices
Aeronautics
Off-highway
vehicles
Automotive
Oil & Gas
Industrial Automation
Fleet Analytics
Health Monitoring
Asset Analytics
Process Analytics
Prognostics
Condition
Monitoring
Clean Energy
Retail Analytics
Mfg Process
Analytics
Supply Chain
Operational
Analytics
Healthcare Analytics
Risk Analysis
Logistics
Retail
Finance
Healthcare
Management
Internet
Railway Systems
5
Data Analytics Workflow
Reporting and
Documentation
Outputs for Design
Deployment
ShareExplore & Discover
Data Analysis
& Modeling
Algorithm
Development
Application
Development
Files
Software
Hardware
Access
Code & Applications
Automate
6
ChallengeProvide clients with an industry-first web platform for
portfolio modeling and analytics
SolutionUse MATLAB to develop and test analytics modules,
and use MATLAB Compiler SDK to deploy them into a
production .NET environment
Results Quantitative development decoupled from
interface development
Stable, responsive system deployed
Rapid delivery of new features enabled
Frontier Advisors Develops Web-Based
Platform for Portfolio Analytics
Link to user story
“MATLAB and MATLAB Compiler
SDK enabled us to rapidly deliver
a sophisticated portfolio analytics
web application with confidence
that it will return accurate results
extremely quickly, ensuring a
highly usable and stable platform
for our clients.”
Lee Eriera
Frontier Advisors
7
Today’s Objectives
Introduce you to data analysis with MATLAB
Show how you can overcome common
data analysis challenges with MATLAB
Demonstrate multiple ways of sharing your
analysis and results with others
8
Common Data Analysis Challenges
using Excel
Complex calculations
Messy Data
Speed of Execution
Automation
Batch Processing
Report Generation
Deployment
9
Demo: Solar Radiation EstimationIntroduction to Data Analysis with MATLAB
Goal:
– Estimate daily mean global solar radiation given
low cost and easily obtained measurements
Approach:
– Process historical measurements
– Develop predictive model
– Document analysis in a report
– Apply analysis on multiple files
10
Modeling Global Solar Radiation
𝑅𝑠 = 𝑎 (1 + 𝑏𝐻)(1 − 𝑒−𝑐 ∆𝑇𝑛)
extraterrestrial
radiation
total global
radiation
Solar Ratio (Rs) =Global solar radiation
Extraterrestrial solar radiation
Daily Temperature Difference (∆𝐓) = TDailyMax – TDailyMin
H is Relative Humidity
a,b,c,n are the model coefficients
11
Explore & Discover
Demo SummarySolar Radiation Estimation
Reporting and
Documentation
Outputs for Design
Deployment
Share
Data Analysis
& Modeling
Files
Software
Hardware
Access
Code & Applications
Automate
Algorithm
Development
Application
Development
Products Used MATLAB
Curve Fitting Toolbox
12
Sharing Results from MATLAB
Automatically generate reports
Create and package applications
Deploy to other environments
13
Using MATLAB with Excel
Passing data between MATLAB and Excel
– MATLAB
Accessing MATLAB from an Excel spreadsheet
– MATLAB
– Spreadsheet Link EX
Deploying MATLAB as an Excel add-in
– MATLAB
– MATLAB Compiler
14
MATLAB Application Deployment
Share MATLAB programs
with people who do not
have MATLAB
– Royalty-free distribution
– Encryption to protect
your intellectual property
Create both standalone
applications and
components for integration
Deploy to desktop, web,
and enterprise applications
MATLAB
MATLAB
Compiler SDK
C/C++ExcelAdd-in
JavaHadoop .NET
MATLAB
Compiler
MATLABProduction
Server
StandaloneApplication
15
Application Author
End User
1
2
Sharing Standalone Applications
MATLAB
ExcelAdd-in Hadoop
StandaloneApplication
Toolboxes
MATLAB Compiler
MATLAB
Runtime3
17
Demo: Preparing Late Plane DataHandling Complex and Messy Data
Goal:
– Prepare late plane data
for further analysis
Approach:
– Load mixed data from files
– Filter data and replace missing data
– Merge observations from different
time intervals into a single data set
18
Accessing Data from MATLAB
Files
– Excel, text, or binary
– Audio and video, image
– Scientific formats and XML
Web Services
– JSON, CSV, and image data
Applications and languages
– C/C++, Java, FORTRAN
– COM, .NET, shared libraries
– Databases (Database Toolbox)
Measurement hardware
– Data acquisition hardware (Data Acquisition Toolbox)
– Stand-alone instruments and devices (Instrument Control Toolbox)
Explore & Discover ShareAccess
20
Machine Learning
Machine learning uses data and produces a program to
perform a task
Standard Approach Machine Learning Approach
𝑚𝑜𝑑𝑒𝑙 = <𝑴𝒂𝒄𝒉𝒊𝒏𝒆𝑳𝒆𝒂𝒓𝒏𝒊𝒏𝒈𝑨𝒍𝒈𝒐𝒓𝒊𝒕𝒉𝒎
>(𝑠𝑒𝑛𝑠𝑜𝑟_𝑑𝑎𝑡𝑎, 𝑎𝑐𝑡𝑖𝑣𝑖𝑡𝑦)
Computer
Program
Machine
Learning
𝑚𝑜𝑑𝑒𝑙: Inputs → OutputsHand Written Program Formula or Equation
If X_acc > 0.5
then “SITTING”
If Y_acc < 4 and Z_acc > 5
then “STANDING”
…
𝑌𝑎𝑐𝑡𝑖𝑣𝑖𝑡𝑦= 𝛽1𝑋𝑎𝑐𝑐 + 𝛽2𝑌𝑎𝑐𝑐+ 𝛽3𝑍𝑎𝑐𝑐 +
…
21
Demo: Machine Learning Using Mobile Phone
Data
Machine
Learning
Data:
3-axial Accelerometer data
3-axial Gyroscope data
22
Demo: Portfolio OptimisationComputing the Efficient Frontier
Goal: Compute an Efficient Frontier in:
– Excel Solver
– MATLAB
Compare the 2 approaches in:
– Performance
– Automation of Workflow
23
Workflow Portfolio Optimization
Convert prices to returns.
Expected Returns.
Covariance matrices
Calculate Efficient Frontier
– Optimize to Maximise the return
– Optimize to Minimise the risk
– Optimze multiple times between Min Risk & Max Return
24
MATLAB Central
Community for MATLAB and Simulink users
– Over 70k daily visits
File Exchange
– Access more than 10k free files including
functions, apps, examples, and models
MATLAB Answers
– Ask programming questions or search
– 18k+ community-answered Questions
Blogs
– Read commentary from engineers who
design, build, and support MATLAB and Simulink
25
Expand Your Analysis Capabilities
Machine learning(Statistics and Machine Learning Toolbox,
Neural Networks Toolbox)
– “Learn” from your data
without assuming an equation
as a model
– www.mathworks.com/machine-learning
Parallel programming (Parallel Computing Toolbox)
– Speed up your analysis using
multicore computers, GPUs, and
computer clusters
– http://www.mathworks.com/products/parallel-computing/
26
Today’s Objectives
Introduce you to data analysis with MATLAB
Show how you can overcome common
data analysis challenges with MATLAB
Demonstrate multiple ways of sharing your
analysis and results with others