TEST-DRIVEN DATA ANALYSIS - eagle.co.nz Driven... · TEST-DRIVEN DATA ANALYSIS Do you believe your...

14
TEST-DRIVEN DATA ANALYSIS Do you believe your analytical results? Thanks to: Nicholas Radcliffe http://tdda.info [email protected] Dept of Mathematics, University of Edinburgh

Transcript of TEST-DRIVEN DATA ANALYSIS - eagle.co.nz Driven... · TEST-DRIVEN DATA ANALYSIS Do you believe your...

Page 1: TEST-DRIVEN DATA ANALYSIS - eagle.co.nz Driven... · TEST-DRIVEN DATA ANALYSIS Do you believe your analytical results? ... Titanic: Machine Learning from Disaster. Build a Model Get

TEST-DRIVEN DATA ANALYSISDo you believe your analytical results?

Thanks to:Nicholas Radcliffehttp://[email protected] of Mathematics, University of Edinburgh

Page 2: TEST-DRIVEN DATA ANALYSIS - eagle.co.nz Driven... · TEST-DRIVEN DATA ANALYSIS Do you believe your analytical results? ... Titanic: Machine Learning from Disaster. Build a Model Get
Page 3: TEST-DRIVEN DATA ANALYSIS - eagle.co.nz Driven... · TEST-DRIVEN DATA ANALYSIS Do you believe your analytical results? ... Titanic: Machine Learning from Disaster. Build a Model Get

The Big Idea

Transfer the ideas of test-driven development

from software development

to data analysis

Page 4: TEST-DRIVEN DATA ANALYSIS - eagle.co.nz Driven... · TEST-DRIVEN DATA ANALYSIS Do you believe your analytical results? ... Titanic: Machine Learning from Disaster. Build a Model Get
Page 5: TEST-DRIVEN DATA ANALYSIS - eagle.co.nz Driven... · TEST-DRIVEN DATA ANALYSIS Do you believe your analytical results? ... Titanic: Machine Learning from Disaster. Build a Model Get

How

models are really developed

Page 6: TEST-DRIVEN DATA ANALYSIS - eagle.co.nz Driven... · TEST-DRIVEN DATA ANALYSIS Do you believe your analytical results? ... Titanic: Machine Learning from Disaster. Build a Model Get
Page 7: TEST-DRIVEN DATA ANALYSIS - eagle.co.nz Driven... · TEST-DRIVEN DATA ANALYSIS Do you believe your analytical results? ... Titanic: Machine Learning from Disaster. Build a Model Get
Page 8: TEST-DRIVEN DATA ANALYSIS - eagle.co.nz Driven... · TEST-DRIVEN DATA ANALYSIS Do you believe your analytical results? ... Titanic: Machine Learning from Disaster. Build a Model Get
Page 9: TEST-DRIVEN DATA ANALYSIS - eagle.co.nz Driven... · TEST-DRIVEN DATA ANALYSIS Do you believe your analytical results? ... Titanic: Machine Learning from Disaster. Build a Model Get
Page 10: TEST-DRIVEN DATA ANALYSIS - eagle.co.nz Driven... · TEST-DRIVEN DATA ANALYSIS Do you believe your analytical results? ... Titanic: Machine Learning from Disaster. Build a Model Get
Page 11: TEST-DRIVEN DATA ANALYSIS - eagle.co.nz Driven... · TEST-DRIVEN DATA ANALYSIS Do you believe your analytical results? ... Titanic: Machine Learning from Disaster. Build a Model Get

tdda level1: CONSTRAINTS

Look before you leap!

Checking the data conforms to your assumptions before you start.

This is tedious to generalise so there are tools to help...

l Not just the obvious input, but also intermediate and output sets

Page 12: TEST-DRIVEN DATA ANALYSIS - eagle.co.nz Driven... · TEST-DRIVEN DATA ANALYSIS Do you believe your analytical results? ... Titanic: Machine Learning from Disaster. Build a Model Get

Tools to automate L1 constraint testsTool Read input Rules Action Flexibility,

Complexity

ArcGIS Input Dialog Existence, schema

Block from starting

Low

RDBMS Table schema

Field constraints, triggers

Rejection Med

TDDA python module

Pandas framework

Regular expression generator

Report Med

FME Attribute Validatortransformer

Choose from built-in rules, custom tests

Report, repair or filter

High

Page 13: TEST-DRIVEN DATA ANALYSIS - eagle.co.nz Driven... · TEST-DRIVEN DATA ANALYSIS Do you believe your analytical results? ... Titanic: Machine Learning from Disaster. Build a Model Get
Page 14: TEST-DRIVEN DATA ANALYSIS - eagle.co.nz Driven... · TEST-DRIVEN DATA ANALYSIS Do you believe your analytical results? ... Titanic: Machine Learning from Disaster. Build a Model Get

TEST-DRIVEN DATA ANALYSIS


Test-Driven Automation: Adopting Test-First Development to ... · Test-driven automation, automation systems, process support, testing, diagnosis. Test-Driven Automation: Adopting

Test-Driven Automation: Adopting Test-First Development to ... · Test-driven automation, automation systems, process support, testing, diagnosis. Test-Driven Automation: Adopting

Applicability of Acceptance Test Driven Development in ... · Introduction department for customer introduction to do their acceptance test. C. Test Driven Development Test Driven

Applicability of Acceptance Test Driven Development in ... · Introduction department for customer introduction to do their acceptance test. C. Test Driven Development Test Driven

Effects of Test-Driven Development: A Comparative …...Effects of Test-Driven Development: A Comparative Analysis of Empirical Studies Simo Mäkinen and Jürgen Münch University

Effects of Test-Driven Development: A Comparative …...Effects of Test-Driven Development: A Comparative Analysis of Empirical Studies Simo Mäkinen and Jürgen Münch University

Test-Driven Development

Test-Driven Development

ATA Engineering, Inc. High Value Test-Driven and Analysis ...

ATA Engineering, Inc. High Value Test-Driven and Analysis ...

Test Driven Developmenthpac.rwth-aachen.de/teaching/sem-lsc-12/TDD.pdfMotivation Test Driven Development Analysis Conclusion Test Driven Development goal: assure testing, avoid defects

Test Driven Developmenthpac.rwth-aachen.de/teaching/sem-lsc-12/TDD.pdfMotivation Test Driven Development Analysis Conclusion Test Driven Development goal: assure testing, avoid defects

Obstacle Driven Development: Extending Test Driven Development

Obstacle Driven Development: Extending Test Driven Development

Test Driven Elephants

Test Driven Elephants

Test driven infrastructure

Test driven infrastructure

Test Driven Product: Applying Test Driven Thinking to the Product World

Test Driven Product: Applying Test Driven Thinking to the Product World

TEST-DRIVEN DATA ANALYSIS - Stochastic SolutionsTest-driven data analysis (TDDA) is an approach to improving the quality, correctness and robustness of analytical processes by transferring

TEST-DRIVEN DATA ANALYSIS - Stochastic SolutionsTest-driven data analysis (TDDA) is an approach to improving the quality, correctness and robustness of analytical processes by transferring

Test-Driven Development - 國立臺灣大學r97002/temp/cellop/eml_tdd.pdf · Test-Driven Development Case Study: Subject Analysis Conclusion References TDD Process Automatic Unit

Test-Driven Development - 國立臺灣大學r97002/temp/cellop/eml_tdd.pdf · Test-Driven Development Case Study: Subject Analysis Conclusion References TDD Process Automatic Unit

TEST-DRIVEN DATA ANALYSIS

TEST-DRIVEN DATA ANALYSIS

Test Driven

Test Driven

Data-Driven and Keyword-Driven Test[1]

Data-Driven and Keyword-Driven Test[1]

Test Driven Data Analysis Why should anyone believe your ... · Test-Driven Data Analysis or: Why should anyone believe your analytical results? or: Data Analysis as if the answers

Test Driven Data Analysis Why should anyone believe your ... · Test-Driven Data Analysis or: Why should anyone believe your analytical results? or: Data Analysis as if the answers

Test driven development_for_php

Test driven development_for_php

Overview on TDD (Test Driven Development) & ATDD (Acceptance Test Driven Development)

Overview on TDD (Test Driven Development) & ATDD (Acceptance Test Driven Development)

Test Driven Development

Test Driven Development

test driven development && analog test driven development

test driven development && analog test driven development