Improving Data Modeling Workflow

30
Improving the Data Modeling Workflow October 28, 2014

Transcript of Improving Data Modeling Workflow

Page 1: Improving Data Modeling Workflow

Improving the Data

Modeling Workflow

October 28, 2014

Page 2: Improving Data Modeling Workflow

LendingHome is the most advanced

mortgage marketplace platform in the

world

What Is LendingHome?

Page 3: Improving Data Modeling Workflow

What is LendingHome?

Simple, efficient borrower experience

Investors are matched to safe, high-yield loans

World-class mortgage ops process driven by

transparent analytics

Statistical models for credit, underwriting, pricing,

sales, marketing

Page 4: Improving Data Modeling Workflow

LendingHome: Mortgage Marketplace

• Fastest loan funded in 72 hours

• Borrowers prequalify in 3 minutes

Investors are matched to safe, high-yield loans

• Line-item accounting for payout tracking

• Proceeds from loans wired in under a second

Page 5: Improving Data Modeling Workflow

Looker is a data exploration solution

that operates in the database

to enable organizations to explore data

in all its detail.

What Is Looker?

Page 6: Improving Data Modeling Workflow

Some of Looker’s Customers

Page 7: Improving Data Modeling Workflow

LendingHome: Operations

World-class operational process driven by

transparent analytics

• Integrations with over 20 vendors

• Rigorous 96-item checklist

• Looker-driven workflow measures quality and

timing of entire process

• Highly complex reporting, dashboards are

easy in Looker

Page 8: Improving Data Modeling Workflow

LendingHome: Ops Analytics

Page 9: Improving Data Modeling Workflow

LendingHome: Ops Analytics

Page 10: Improving Data Modeling Workflow

The Challenge

Data scientists create value by creating

actionable models

More time spend preparing data and evaluating

results than using heavy data science

techniques

How to speed up analytical cycles

Page 11: Improving Data Modeling Workflow

Focusing on Expertise

Define

Goals

Data Cleaning

and Shaping

Model

Dev

Expertise

“Work”

Model

Eval

Page 12: Improving Data Modeling Workflow

Our Example

38 million flights between 2000 and 2005

Carrier information, departure and arrival

location, manifest data, aircraft data

Modeling on-time rates

Page 13: Improving Data Modeling Workflow

Getting Started

Analytics as the shortcut to variable selection

Page 14: Improving Data Modeling Workflow

Data Cleaning

Analytics can provide a distinct advantage in the

constant struggle for clean data

Page 15: Improving Data Modeling Workflow

Data Cleaning Cont.

Page 16: Improving Data Modeling Workflow

Selecting Variables

Page 17: Improving Data Modeling Workflow

Pulling Data

Analytics allow data scientists to lever analytical

modeling to easily grab reshaped data

- Time-zone functions

- Sub-select functions

- Cleaned/filtered data

Page 18: Improving Data Modeling Workflow

LendingHome: Quant

Modeling

• Credit models learned over 25M loans, 4B

payments, macroeconomic factors

• Scoring transparent to borrowers, investors

• Finds and predict borrower conversion from

180M RE transactions

• Feature extraction, data exploration, train/test

splits, model analysis in Looker

Page 19: Improving Data Modeling Workflow

LendingHome: Factor

Analysis

Page 20: Improving Data Modeling Workflow

LendingHome: Noisy Data

Page 21: Improving Data Modeling Workflow

Building The Data Model

Let’s try a simple model:

Page 22: Improving Data Modeling Workflow

Evaluating The Model

Cleaning

Analytics

Modeling

Page 23: Improving Data Modeling Workflow

Continuing Our Example

Re-using that same analytics process, we can

explore our data more seamlessly

Page 24: Improving Data Modeling Workflow

Examining Residuals

Page 25: Improving Data Modeling Workflow

In-Sample vs Out-of-Sample

Page 26: Improving Data Modeling Workflow

LendingHome: Evaluate Models

Page 27: Improving Data Modeling Workflow

Layering Entirely New

Variables

Page 28: Improving Data Modeling Workflow

Q & A

Page 29: Improving Data Modeling Workflow

Try Looker on your own data.

Looker.com/trial

Page 30: Improving Data Modeling Workflow

More Questions?

[email protected]

[email protected]