24/10/2014 1 © Nokia 2014
SW testing in large SW project Presentation for TTY course TIE-21200 Ohjelmistojen testaus
Sanna Järvenpää
NAdC / Leena Rasinaho
24/10/2014 2 © Nokia 2014
Nokia Networks business in Finland
NAdC / Leena Rasinaho
24/10/2014 3 © Nokia 2014
Networks business in Finland
• Head office in Espoo; other offices in Oulu and in Tampere
• One of the main R&D locations for Networks
• Full range of operations including manufacturing of base station products in Oulu
• Key customers: Nationwide operators Elisa and TeliaSonera; Finnish Railway Administration (GSM-R)
24/10/2014 4 © Nokia 2014
Employees
• 6000 employees
•Over 10% of total Nokia employees
• 3700 R&D employees
• Approximately 400 manufacturing employees
• Employees by location:
•Espoo 3 000 •Oulu 2 300 •Tampere 700
24/10/2014 5 © Nokia 2014
Locations
Espoo •Head office functions •End-to-end Mobile Solution Center
Oulu • Radio Research & Design Center •Base Station manufacturing
Tampere •End-to-end Network Operability Center
24/10/2014 6 © Nokia 2014
Ecosystem • Local universities providing access to latest technologies which
utilized via coop projects and thesis workers for our R&D projects
• Lots of user experience and process automation expertise
in local companies. Coop projects with TUT, UTA, TAKK and TAMK
• Close, regular and trustful relationship with Tampere City officials and
mayor to discuss the future plans mutually. Visible e.g. via Tampere
spirit Training where public funding available
Highlights
• End to end Operability, User Experience and Process Automation
• Operations Support Systems
• GSM Base Station Controller and WCDMA Radio Network Controller
• SW maintenance services
• Real-time data processing for CEM via Traffica
• Shared Reference Network lab and Global OSS Lab
Technology Expertise
• System integration competences for OSS and end-to-end operability
• Customer experience management, Traffica, Service assurance
• GSM / WCDMA Controller Specification, SW and I&V
• System Modules, System on Chip development
Tampere – SoC Development, Operations Support Systems and Customer Experience Center
24/10/2014 7 © Nokia 2014
SW testing in Nokia NetAct product
NAdC / Leena Rasinaho
24/10/2014 8 © Nokia 2014
NetAct
General Architecture in NetAct
3RD party tool
Other
vendors
NEs
Alarms
Configurations
Measurements
GSM
WCDMA LTE
MVI
24/10/2014 9 © Nokia 2014
NetAct functionality
• New features
• Regression testing for functionality in old releases
Network element support in NetAct
• Basic support
• Regression testing for functionality in old releases
• New feature support
What to test
24/10/2014 10 © Nokia 2014
Test Planning
FC / US definition into
backlog
FC analyze/design
phase
FC Test Plan definition
FC / US implementation and testing
Development team +
architect(s) +
spec people +
documentation people
iterative
process
FC = Feature Component US = User Story
24/10/2014 11 © Nokia 2014
Component testing
Unit testing – Junits – first level of regression testing
Integration testing
How components work together
System Testing
Functional WHAT – Testing in development teams, Network Verification, System Verification
Non-functional HOW – Performance, Reliability, Load, Sequrity, Usabilility
Acceptance testing
Customer pilots, development teams defined SAT cases
(SAT = System Acceptance Test)
Validation: Building right product - Verification: Building product right
24/10/2014 12 © Nokia 2014
Static testing code review specification review test case review documentation review
Dynamic testing black-box testing ~ specification based, use cases white-box testing ~ implementation based experience based testing ~ regression testing, add-hoc testing
Static – Dynamic testing
24/10/2014 13 © Nokia 2014
NetAct product contains several building blocks that needs to work together
1. Each building block runs there own test 2. System verification tests that all building blocks work together 3. Network verification tests that network elements work together with
NetAct
Building one NetAct product
NetAct
SWM
PM CM
FM
Trace
iSON
(1) (2)
(3)
24/10/2014 14 © Nokia 2014
TestAutomation in NetAct
NAdC / Leena Rasinaho
24/10/2014 15 © Nokia 2014
Test environment installation
NetAct
Pre-tests Reporting
NetAct installation
System component
upgrade
24/10/2014 16 © Nokia 2014
Jenkins-builds
Copy to System build
Upgrade to CI lab
Execute smoke cases
Execute regression cases Automatic promotion
to NetAct product
Fix for the faulty component.
Responsible team
to check and fix
Execute daily regression cases
Execute weekly regression cases
Failing
Success
Test Automation in system component build creation
Responsible team
to check and fix
24/10/2014 17 © Nokia 2014
Building one NetAct product
NetAct
24/10/2014 18 © Nokia 2014
Different Testautomation types
Purpose Number of tests
Execution time
Smoke Can system component build be installed to system component test environments
32 ~ 15 min
Regression Can system component build be included to NetAct build
39 ~ 15 min
Daily regression
Is the basic functionality working with system component build Tested against daily installed newest system component
118 + 87 + 320 +143
4 parallel runs ~ 8 h
Weekly regression
Is the basic functionality working with system component build Tested against weekly installed system component
Several runs during week
24/10/2014 19 © Nokia 2014
Test Automation reporting
24/10/2014 20 © Nokia 2014
Top Related