1. OVERVIEW OF TESTPROCESS IMPROVEMENTFRAMEWORKSNikita Knysh, Ciklum, February 21, 2012

2. Speaker Nikita Knysh, 31 y.o. E-comonic, Ciklum 11 years in IT, 5 years in software test ISTQB Certified Test Manager 3. Agenda The way of improvement Why use process frameworks TMM TPI CTP STEP 4. The Way of Improvement 5. Why Use Process Frameworks Get aware of industry practices Measure process maturity Compare to others Suggest improvements, set goals Evaluate improvement 6. Test Maturity Model (TMM) Complements CMM 5 maturity levels, a number of processes ineach Establish basic processes first, then grow Staged Need to achieve previous level to claim for next Once a level is achieved, no further improvementneeded in its processes Binary You either fulfill or not. A process cannot be 50% fulfilled 7. Test Maturity Model (TMM) The five maturity levels Initial Defined Integrated Managed and measured Optimized Heavy, documentation-intensive. Suitable for contractual test, in regulatedindustries, where auditable process is required. 8. Test Process Improvement(TPI) The four process cornerstones Lifecycle Organization Infrastructure and tools Techniques 20 Processes, 4 levels of maturity 9. Test Process Improvement(TPI) #2 10. Test Process Improvement(TPI) #3 Allows to establish the process and start addvalue quickly, then improve steadily andincrementally Suitable for startups or introducing QA functionto existing companies 11. Critical Testing Processes(CTP) Be good at critical processes! A criticalprocess: Repeatedfrequently, affects team efficiency Involves large number of people, affects cohesionand cooperation Visible to peers and superiors Linked to project success Identifies attributes of a good process andopportunities for improvement Provides metrics and qualitative evaluationsfor each process, allows tailoring 12. Critical Testing Processes(CTP) #2 Testing Test system Establishing context development Quality risk analysis Test release Test estimationmanagement Test execution Test planning Bug reporting Test teamdevelopment Results reporting Change managementAll evaluations are done against businessneeds May be tailored for wide range of the orgs 13. Systematic Test and Evaluation Process(STEP) Assesses three major stages of test: Planning Acquisition(test analysis, design andimplementation) Measurement (test execution and reporting) Qualitative factors considered: Definedtest process utilization Customer satisfaction 14. Systematic Test and Evaluation Process(STEP) #2 Quantitative metrics: Test status over time Test requirements or risk coverage Defect trends, including detection, severity, andclustering Defect density Defect removal effectiveness Defect detection % Defect introduction, detection, and removal phases Cost of testing in terms of time, effort, and money Assumes test then code approach, closecollaboration of dev and test, defects areprevented or detected early or systematicallyanalyzed. 15. Some Other Frameworks Test Organization Maturity (TOM) Test Improvement Model (TIM) Software Quality Rank (SQR) TMap 16. Thank you! Questions