Improving Batch-Process Testing Techniques with a Domain-Specific Language
description
Transcript of Improving Batch-Process Testing Techniques with a Domain-Specific Language
![Page 1: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/1.jpg)
Improving Batch-Process Testing Techniqueswith a Domain-Specific Language
Alberto Lemos (Dr. Spock)Senior Software Architect
SpockNET
Danival Taffarel CalegariMATERA Systems Architect
Globalcode Instructor
![Page 2: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/2.jpg)
Agenda
• Principles & Proposals
• Why to use a DSL to test?
• 2 Case Studies
![Page 3: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/3.jpg)
Objective
“Show a simple way to improve batch process tests
with a Domain-Specific Language”
![Page 4: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/4.jpg)
Motivation
• Test automation contributes to the project success
• It is hard and time consuming to write and automate tests
• It is more complex and hard to write tests for batch process
![Page 5: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/5.jpg)
Agenda
• Principles & Proposals
• Why to use a DSL to test?
• 2 Case Studies
![Page 6: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/6.jpg)
Principles for a good Test Automation
• Automated tests must be:
• Easy to write
• Easy to run
• Self-contained
• Run quickly
• Always updated
![Page 7: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/7.jpg)
Challenges to test a Batch Process
• How to test Batch Process?
• How to control when it starts or stops?
• How to prevent concurrent run?
• The preparation and validation steps require lots of complex SQLs
![Page 8: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/8.jpg)
Domain-Specific Language
• Why to use DSL to test?
• A good DSL must be:
• Expressive and natural
• Easy to use and learn
• Hard to misuse
• Extensible
![Page 9: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/9.jpg)
Agenda
• Principles & Proposals
• Why to use a DSL to test?
• 2 Case Studies
![Page 10: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/10.jpg)
Making test writing easier with DSL
• Define a language specifically to write the tests
• The language must be closer to the business than to the technology
• The easier the DSL, the greater the chance to it will be used
![Page 11: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/11.jpg)
Making test writing easier with DSL
• The DSL implementation strategy in 2 steps:
• 1st step: Use Internal (embedded) DSL
• 2nd step: Use external DSL
![Page 12: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/12.jpg)
Internal (embedded) DSL
• It is easier to write
• It has tool support
• It provides a good fluency for those who know the underlining programming language.
![Page 13: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/13.jpg)
External DSL
• It has less technological limitations
• It adds more valueper line of code
• It can be used by a layman
![Page 14: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/14.jpg)
Steps for DSL Setup
Create a basic infrastructure to run the processes and verify the results.
Create a DSL for each functionalityWrite the test for the functionality
![Page 15: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/15.jpg)
Steps for DSL Setup
![Page 16: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/16.jpg)
Steps for DSL Setup• How can I run a batch in an isolated form?
![Page 17: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/17.jpg)
Steps for DSL Setup• How can I run a batch in an isolated form?
• How should the test code to run the SQL to verify the results?
![Page 18: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/18.jpg)
Steps for DSL Setup• How can I run a batch in an isolated form?
• How should the test code to run the SQL to verify the results?
• How to make the DSL open a browser, do login and press buttons?
![Page 19: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/19.jpg)
Steps for DSL Setup
![Page 20: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/20.jpg)
Steps for DSL Setup• Use the builder pattern
![Page 21: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/21.jpg)
Steps for DSL Setup• Use the builder pattern
• Provide default values or they will be automatically generated
![Page 22: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/22.jpg)
Steps for DSL Setup• Use the builder pattern
• Provide default values or they will be automatically generated
• Provide an abstract base class to be extended
![Page 23: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/23.jpg)
Steps for DSL Setup• Use the builder pattern
• Provide default values or they will be automatically generated
• Provide an abstract base class to be extended
• Reuse similar functionalities
![Page 24: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/24.jpg)
![Page 25: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/25.jpg)
Agenda
• Principles & Proposals
• Why to use a DSL to test?
• 2 Case Studies
![Page 26: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/26.jpg)
Case studies
• Application of the techniques in two global bank systems with strong batch processing.
• Test automation for an already existing system.
• Test automation for an system built from scratch.
![Page 27: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/27.jpg)
First case study
• A system responsible for receiving buy and sell orders from international clients, processing those orders and sending them to brazilian stock exchange systems.
• It’s a very critical system.
![Page 28: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/28.jpg)
First case studyInternational Clients
WebLogic 8.1
WLI
App
InboundQueue
Oracle DB
SQL Server
MQSeries
![Page 29: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/29.jpg)
First case study
• Challenges
• Run the system tests isolated.
• Post messages into WebLogic JMS queues.
• To determine when a process finishes.
![Page 30: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/30.jpg)
Run the system isolated
WebLogic 8.1
International Clients
WLI
App
InboundQueue
MQSeries
Oracle DB
SQL Server
![Page 31: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/31.jpg)
Fedora 4 VM
Windows VM
Run the system isolated
WebLogic 8.1
WLI
App
InboundQueue
Oracle DB
SQL Server
MQSeries
![Page 32: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/32.jpg)
Simplifying test setup• A web application was built to run inside
the test server.
• It acts as a “spy” on the server.
• It can access the server database connections
• It can access the server message queues.
![Page 33: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/33.jpg)
Simplifying test setup
WebLogic 8.1
WLI
App
InboundQueue
Fedora 4 VM
Windows VM
Oracle DB
SQL Server
MQSeries
![Page 34: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/34.jpg)
Simplifying test setup
WebLogic 8.1
WLI
App
InboundQueue
Fedora 4 VM
Windows VM
Oracle DB
SQL Server
MQSeriesTest Spy
![Page 35: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/35.jpg)
Simplifying test setup
WebLogic 8.1
WLI
App
Test SpySelenium Server
Selenium RC
Test cases
Test environment
![Page 36: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/36.jpg)
Simplifying test writing
• A DSL has been made using Selenium to access the test application server.
• The completion for each command is checked by issue SQL commands querying control tables until the results appear or a timeout occurs.
![Page 37: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/37.jpg)
Simplifying test writing
![Page 38: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/38.jpg)
Pros & Cons• The test execution is very “visual”.
• Testers with basic programming skills wrote more than 400 test cases.
• It doesn’t leave an open backdoor in the production environment.
• It was very difficult to put in a continuous integration test environment.
![Page 39: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/39.jpg)
Second case study• A system responsible for receiving
allocation commands for contracts in future markets (BM&F).
• It monitors the allocation status and displays alerts in a monitor.
• System failures and delays can cause fines and image damages to the bank.
![Page 40: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/40.jpg)
Second case studyAllocation commands
Tomcat 6
App
Oracle DB
Tables SP
![Page 41: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/41.jpg)
Second case study• Challenges
• Run the system tests isolated.
• To control the execution of the stored procedure.
• How to check in and out parameters for the stored procedure?
![Page 42: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/42.jpg)
Run the system isolatedAllocation commands
Tomcat 6
App
Oracle DB
Tables SP
![Page 43: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/43.jpg)
Ubuntu Server VM
Run the system isolated
Tomcat 6
AppOracle DB
Tables SPRemoteControl
![Page 44: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/44.jpg)
Ubuntu Server VM
SP control
Tomcat 6
App
Oracle DB
Tables SP*
Tables*RemoteControl
![Page 45: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/45.jpg)
Simplifying test setup
Tomcat 6
App
RemoteControl
Test Spy
Test environment
Spring Remoting
Test cases
![Page 46: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/46.jpg)
Simplifying test writing
• A DSL has been made using Spring Remoting to access the test application server.
• The completion for each command is checked by the remote control put inside the application.
![Page 47: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/47.jpg)
Pros & Cons• It’s faster than Selenium solution.
• The batch execution control is more precise.
• Linux based VM is more portable.
• It demanded such effort to prepare the environment (VM, mock SP, semaphores)
![Page 48: Improving Batch-Process Testing Techniques with a Domain-Specific Language](https://reader033.fdocuments.net/reader033/viewer/2022042606/54809a1db4af9fe2158b5dd6/html5/thumbnails/48.jpg)
Thank you!
• Alberto Lemos (Dr. Spock)
@drspockbr
http://about.me/drspockbr
• Danival T. Calegari
@danivaltc