Lynn Winterboer : Test automation
-
Upload
agiledenver -
Category
Software
-
view
3.626 -
download
0
Transcript of Lynn Winterboer : Test automation
![Page 1: Lynn Winterboer : Test automation](https://reader033.fdocuments.net/reader033/viewer/2022042723/586f765d1a28ab10258b6403/html5/thumbnails/1.jpg)
© 2016 by Winterboer Agile Analytics www.WinterboerAgileAnalytics.com
Test Automation: Agile Enablement for Data Warehousing and Business Intelligence TeamsPresented to Mile High Agile (MHA) 2016April 4, 2016Denver, Colorado
Lynn Winterboer, Agile Analytics Educator and Coach, @AgileLynnBrad Ewald, DW/BI Solution Architect, @JBEwald
![Page 2: Lynn Winterboer : Test automation](https://reader033.fdocuments.net/reader033/viewer/2022042723/586f765d1a28ab10258b6403/html5/thumbnails/2.jpg)
© 2016 by Winterboer Agile Analytics www.WinterboerAgileAnalytics.com
Agenda
• Why is test automation important for agile data teams?
• Why aren’t all data teams automating their tests today?
• What is the path to test automation?
• What does simple DW test automation look like?
![Page 3: Lynn Winterboer : Test automation](https://reader033.fdocuments.net/reader033/viewer/2022042723/586f765d1a28ab10258b6403/html5/thumbnails/3.jpg)
© 2016 by Winterboer Agile Analytics www.WinterboerAgileAnalytics.com
Why is test automation important to agile data teams?
![Page 4: Lynn Winterboer : Test automation](https://reader033.fdocuments.net/reader033/viewer/2022042723/586f765d1a28ab10258b6403/html5/thumbnails/4.jpg)
© 2016 by Winterboer Agile Analytics www.WinterboerAgileAnalytics.com© Ken Collier, 2016, All Rights Reserved
Agile Demands Something Different
Agile calls for small increments of “potentially shippable code”.
That means QA is essential on a regular, frequent basis.
How can we be sure that something we do
here
Doesn’t break something we did back
here
Test Suite Test Suite Test
Suite Test Suite
Test Suite
Test Suite Test
Suite
Manual testing quickly becomes infeasible.The test suite grows larger each iteration.
![Page 5: Lynn Winterboer : Test automation](https://reader033.fdocuments.net/reader033/viewer/2022042723/586f765d1a28ab10258b6403/html5/thumbnails/5.jpg)
© 2016 by Winterboer Agile Analytics www.WinterboerAgileAnalytics.com© Ken Collier, 2016, All Rights Reserved
Testing is Central to Agile
Agile BI development is driven by tests
Acceptance criteria are the definition of “done”
Passing tests are the measure of “done”
Regression tests are the measure of “still done”
![Page 6: Lynn Winterboer : Test automation](https://reader033.fdocuments.net/reader033/viewer/2022042723/586f765d1a28ab10258b6403/html5/thumbnails/6.jpg)
© 2016 by Winterboer Agile Analytics www.WinterboerAgileAnalytics.com© Ken Collier, 2016, All Rights Reserved
Agile Teams Understand…
… that test automation is a key technical enabler
to “being agile”
![Page 7: Lynn Winterboer : Test automation](https://reader033.fdocuments.net/reader033/viewer/2022042723/586f765d1a28ab10258b6403/html5/thumbnails/7.jpg)
© 2016 by Winterboer Agile Analytics www.WinterboerAgileAnalytics.com
Why aren’t all data teams automating their tests
today?
![Page 8: Lynn Winterboer : Test automation](https://reader033.fdocuments.net/reader033/viewer/2022042723/586f765d1a28ab10258b6403/html5/thumbnails/8.jpg)
© 2016 by Winterboer Agile Analytics www.WinterboerAgileAnalytics.com
DW/BI Test Automation ChallengesData industry does not focus on testing as much as other disciplines:
Education
Team Members
Skills & Discipline
![Page 9: Lynn Winterboer : Test automation](https://reader033.fdocuments.net/reader033/viewer/2022042723/586f765d1a28ab10258b6403/html5/thumbnails/9.jpg)
© 2016 by Winterboer Agile Analytics www.WinterboerAgileAnalytics.com
DW/BI Test Automation ChallengesExisting tools are not easily adapted to the data world:
Development Languages
Web, Mobile and UI Focus
![Page 10: Lynn Winterboer : Test automation](https://reader033.fdocuments.net/reader033/viewer/2022042723/586f765d1a28ab10258b6403/html5/thumbnails/10.jpg)
© 2016 by Winterboer Agile Analytics www.WinterboerAgileAnalytics.com
DW/BI Test Automation ChallengesGood test data sets are hard to come by:
Large volumes (on small environments)
Shared test environments
Data sensitivity
Small sets need extra planning and design
![Page 11: Lynn Winterboer : Test automation](https://reader033.fdocuments.net/reader033/viewer/2022042723/586f765d1a28ab10258b6403/html5/thumbnails/11.jpg)
© 2016 by Winterboer Agile Analytics www.WinterboerAgileAnalytics.com
What is the path to test automation?
![Page 12: Lynn Winterboer : Test automation](https://reader033.fdocuments.net/reader033/viewer/2022042723/586f765d1a28ab10258b6403/html5/thumbnails/12.jpg)
© 2016 by Winterboer Agile Analytics www.WinterboerAgileAnalytics.com© Ken Collier, 2016, All Rights Reserved
Agile Testing Perspectives
High QualityDW/BI System
3) Tools &Automation
1) Attitudes &Culture
2) Skills &Practices
Testing is a Team SportLearning CultureBuild Quality In
Testing SkillsTest Design, Organization & MaintenanceRepeatable TestsFeedback Loops
Test EnvironmentBuild AutomationTest Data SetsTest Automation Tool
![Page 13: Lynn Winterboer : Test automation](https://reader033.fdocuments.net/reader033/viewer/2022042723/586f765d1a28ab10258b6403/html5/thumbnails/13.jpg)
© 2016 by Winterboer Agile Analytics www.WinterboerAgileAnalytics.com© Ken Collier, 2016, All Rights Reserved
Test Automation Pyramid
AppTests
Acceptance Tests
Unit & Component Tests
Manual Tests
Lisa Crispin & Janet Gregory, Agile Testing: A Practical Guide for Testers and Agile Teams
Push as many tests as possible to this layer
Write these in business domain language
Brittle tests that are affected by UI changes
Subjective user feedback is essential
![Page 14: Lynn Winterboer : Test automation](https://reader033.fdocuments.net/reader033/viewer/2022042723/586f765d1a28ab10258b6403/html5/thumbnails/14.jpg)
© 2016 by Winterboer Agile Analytics www.WinterboerAgileAnalytics.com© Ken Collier, 2016, All Rights Reserved
Good Places to Start Unit tests on new development
Regression tests you wish you had, before building something new
Deployment tests to ensure each build migrated correctly
Painful test setup or test execution prone to error
![Page 15: Lynn Winterboer : Test automation](https://reader033.fdocuments.net/reader033/viewer/2022042723/586f765d1a28ab10258b6403/html5/thumbnails/15.jpg)
© 2016 by Winterboer Agile Analytics www.WinterboerAgileAnalytics.com
What does simple data warehousing test automation
look like?
![Page 16: Lynn Winterboer : Test automation](https://reader033.fdocuments.net/reader033/viewer/2022042723/586f765d1a28ab10258b6403/html5/thumbnails/16.jpg)
© 2016 by Winterboer Agile Analytics www.WinterboerAgileAnalytics.com© Ken Collier, 2016, All Rights Reserved
Basic ApproachComponent under test
SourceSchema
TargetSchema
ActualResults
TestData
Test Runner
leave no trace
leave no trace
Setup Execute Test Report Clean Up
Static Test Data
ExpectedResults
Test Queries
load test data
compare results
executetests
based on specific input data
![Page 17: Lynn Winterboer : Test automation](https://reader033.fdocuments.net/reader033/viewer/2022042723/586f765d1a28ab10258b6403/html5/thumbnails/17.jpg)
© 2016 by Winterboer Agile Analytics www.WinterboerAgileAnalytics.com
Demonstration
![Page 18: Lynn Winterboer : Test automation](https://reader033.fdocuments.net/reader033/viewer/2022042723/586f765d1a28ab10258b6403/html5/thumbnails/18.jpg)
© 2016 by Winterboer Agile Analytics www.WinterboerAgileAnalytics.com
Testing FrameworkFunctionality and Components:
Example data set SQL Server and related model
Form for creating and describing tests Microsoft Access form
Test execution SQL Server stored procedures (individual tests) Power Shell scripts (batch execution)
BI Reports and Dashboards for monitoring MS Power BI
![Page 19: Lynn Winterboer : Test automation](https://reader033.fdocuments.net/reader033/viewer/2022042723/586f765d1a28ab10258b6403/html5/thumbnails/19.jpg)
© 2016 by Winterboer Agile Analytics www.WinterboerAgileAnalytics.com
What Constitutes a Test?
Actual Result(or Dataset)
Expected Result(or Dataset)
Always a Query SQL Query or Static Value
![Page 20: Lynn Winterboer : Test automation](https://reader033.fdocuments.net/reader033/viewer/2022042723/586f765d1a28ab10258b6403/html5/thumbnails/20.jpg)
© 2016 by Winterboer Agile Analytics www.WinterboerAgileAnalytics.com© Ken Collier, 2016, All Rights Reserved
Test Scenarios
1. Record Comparison
2. Sales Revenue Subject Area Test
3. Operational Data Quality Test
![Page 21: Lynn Winterboer : Test automation](https://reader033.fdocuments.net/reader033/viewer/2022042723/586f765d1a28ab10258b6403/html5/thumbnails/21.jpg)
© 2016 by Winterboer Agile Analytics www.WinterboerAgileAnalytics.com
Testing Demo
![Page 22: Lynn Winterboer : Test automation](https://reader033.fdocuments.net/reader033/viewer/2022042723/586f765d1a28ab10258b6403/html5/thumbnails/22.jpg)
© 2016 by Winterboer Agile Analytics www.WinterboerAgileAnalytics.com
Session Intent
1) Why: Test Automation is key to DW/BI agile success
2) How: Automate a solid testing practice; don’t automate chaos
3) Show: DW/BI test automation is not that hard
![Page 23: Lynn Winterboer : Test automation](https://reader033.fdocuments.net/reader033/viewer/2022042723/586f765d1a28ab10258b6403/html5/thumbnails/23.jpg)
© 2016 by Winterboer Agile Analytics www.WinterboerAgileAnalytics.com
DW/BI Test Automation ToolsTool Description
DBFithttp://dbfit.github.io/dbfit/
Open source database testing tool
iCEDQhttp://icedq.com
A unified platform for ETL testing and data migration testing
Analytix Data Serviceshttp://analytixds.com
Enterprise data mapping and ETL code automation, including new automatic generation of reusable test cases associated with Source to Target Mappings
WhereScapehttps://www.wherescape.com
Data warehouse automation software, including a test harness framework with user-configurable profiling methods.
Tricentis Toscahttp://www.tricentis.com/tricentis-tosca-testsuite/
DW-friendly test automation tool
![Page 24: Lynn Winterboer : Test automation](https://reader033.fdocuments.net/reader033/viewer/2022042723/586f765d1a28ab10258b6403/html5/thumbnails/24.jpg)
© 2016 by Winterboer Agile Analytics www.WinterboerAgileAnalytics.com
DW/BI Test Automation ReferencesReference DescriptionAgile Analytics: A Value-Driven Approach to Business Intelligence and Data Warehousingby Dr. Ken Collier
Chapter 7: Test-Driven Data Warehouse Development
Agile Data Warehousing for the Enterprise: A Guide for Solution Architects and Project LeadersBy Ralph Hughes
Part V: Agile EDW Quality Management Planning
Automated Data Warehouse Testing: Beginner’s Step-by-Step Guideby G. Suden
Java-based test automation for simple data warehousing tests
Agile Testing: A Practical Guide for Testers and Agile Teamsby Lisa Crispin and Janet Gregory
Great introduction to agile testing for any type of development
More Agile Testing: Learning Journeys for the Whole Teamby Lisa Crispin and Janet Gregory
Includes several chapters on test automation as well as one dedicated to agile testing for DW/BI
![Page 25: Lynn Winterboer : Test automation](https://reader033.fdocuments.net/reader033/viewer/2022042723/586f765d1a28ab10258b6403/html5/thumbnails/25.jpg)
© 2016 by Winterboer Agile Analytics www.WinterboerAgileAnalytics.com
Thank You!
Lynn WinterboerAgile Analytics Educator &
Coachwww.WinterboerAgileAnalytics.comlynn@WinterboerAgileAnalytics.com
@agilelynn
Brad EwaldDW/BI Solution Architect
Children’s Hospital Colorado
[email protected]@jbewald