4th ucaat poster automation at system level case TETRA PMR ... · Automation at System Level: Case...
Transcript of 4th ucaat poster automation at system level case TETRA PMR ... · Automation at System Level: Case...
Automation at System Level: Case TETRA PMR Network
UCAAT 2016
• Achievement (at System Level)� Initial feasibility study showed that 50% of test suites to be automatable� Continuous integration (CI) using test automation implemented and used� Night/weekend executions, collection of statistics.� In average 47 % of tests are automated in several test phases:
SW build (”smoke”), regression, upgrade and some load tests
• Lessons Learned• Choose suitable tools: test execution, result analysis, result reporting, etc.• Build test automation portable• Minimize the effor required for re-configuration• Build test automation maintainable• Test automation challenging to apply if product is not designed to be ”test
automation friendly”• Transparency• Not directly comparable with manual testing
•Introduction• System level end-to-end verification• Continuous Integration & Automation from Sub-system to
System level• Test tools set for Agile, CI process combining manual and
automated test
•Target• Improve feedback cycle in all test levels (Unit, component,
sub-system, system level)• Reduce the time of non-functional tests• Increase requirement coverage• Improve quality of products & SW Release• Define test automation process & best practices • Select, Define and Optimize interface between tools (CI,
Automation, Test mngt tools)
2014 2015 2016
automated tests
Test Automation at System Level
SW Release
Product
Test Automation
UnitIntegration
Acceptance (functional)Capacity (and other non-functional)
Manual
Application p
Application 1
Dispatcher
UserTerminal
Network Elements
Gateways
Generation ofTest Cases
MBT
Generation ofTest CasesManually
Execute
Analysis
Report
Test AutomationFramework
Test AutomationFramework
SW DevelEnvironment
Continuous Integration
(@Sub-System)
Continuous Integration (@System)
Automated testsLibrariesTest data
Manual tests
Libraries ofSW Rel of
SUT
SW Repositoryfor test at System level
SW Repositoryfor test at Sub-system level