LogicaCMG & Testing: TestFrame & RRBT - SAST · Risk & Requirement Based Testing & TestFrame 3...

© LogicaCMG 2006. All rights reserved

LogicaCMG & Testing: TestFrame & RRBT

Mike Peters

Senior Test Consultant/Delivery Manager

[email protected]

Risk & Requirement Based Testing & TestFrame 2

About LogicaCMG

• > 40,000 employees worldwide

• 41 countries

• > 40 years experience in IT services

• Offers a wide range of IT-solutions

• Specialised in industry-specific solutions

• Aquisition of WM-data in 2006


LogicaCMG & Software Testing

• 2500 Test Professionals world-wide

– Test consultancy

– Managed testing

– Test outsourcing

• Test Research Centre in the Netherlands

– TestFrame method & tooling

– Test management: Risk & Requirement Based Testing

– Business Acceptance Management & - Testing

– ERP & Testing

– Embedded Software Testing

– STAPPIN (Standard infrastructure testing, e.g. Vista, Exchange)


… LogicaCMG’s knowledge & experience in testing is based on more than 1,000 test projects over the world

Testing credentials


Agenda

• Everyday testing challenges

• Risk & Requirement Based Testing (RRBT)

• TestFrame

• Discussion


The later you find faults, the more it costs to solve them

Testing of systems

Source: Boehm

Definition Design Build Use

Costs

Time


Testing is often under pressure

Planning andspecification

Development Test


Testing in parallel to the development activities

Development activities

Testing activities


Agenda



• TestFrame

• Discussion


Position various solutions

Test analysis

and execution

Test Management

Business

objectives

Structure/Fitting/ToolingStandard

RepeatableReliable

Maintainable

InsightControlFocus

ReportingPredictable

Commitment

Risk ControlQuality to MarketTime to Market

Business AlignmentBudget Control

Reputation Management


Risk & Requirement Based Testing


Test Management Model

EVALUATIONEVALUATIONEVALUATIONEVALUATION

ANDANDANDAND

TRANSFERTRANSFERTRANSFERTRANSFER

REPORTINGREPORTINGREPORTINGREPORTING

ANDANDANDAND

IMPLEMENTATIONIMPLEMENTATIONIMPLEMENTATIONIMPLEMENTATION

ADVICEADVICEADVICEADVICE

ISSUEISSUEISSUEISSUE

MANAGEMENTMANAGEMENTMANAGEMENTMANAGEMENT

PROGRESSPROGRESSPROGRESSPROGRESS

MANAGEMENTMANAGEMENTMANAGEMENTMANAGEMENT

TESTTESTTESTTEST

ORGANIZATIONORGANIZATIONORGANIZATIONORGANIZATION

PLANNINGPLANNINGPLANNINGPLANNING

ESTIMATIONESTIMATIONESTIMATIONESTIMATION

RISK ANALYSISRISK ANALYSISRISK ANALYSISRISK ANALYSIS

ANDANDANDAND

TEST STRATEGYTEST STRATEGYTEST STRATEGYTEST STRATEGY

Risk & Requirement Based Testing & TestFrame

How to reduce risk . . .

13

And still test effectively ?


Product risks versus requirements

Productrisks

Require-ments

Matching risks and requirements

Matching requirements and risks

A risk, no requirement:

• Add requirement (find faults earlier)

• Remove risk (don’t test more than necessary)

A requirement, no risk:

• Adjust risk list (better coverage of testing)

• Remove requirement (don’t develop more than necessary, no “frills”)


RRBT & Quality Attributes (ISO 9126)

FunctionalitySuitability

Accuracy

Interoperability

Security

Functionality compliance

15

ReliabilityMaturity

Fault tolerance

Recoverability

Reliability Compliance

PortabilityAdaptability

Installability

Co-existence

Replaceability

Portability compliance

UsabilityUnderstandability

Learnability

Operability

Attractiveness

Usability compliance

EfficiencyTime behaviour

Resource utilisation

Efficiency compliance

MaintainabilityAnalysability

Changeability

Stability

Testability

Maintainability compliance

40%40%40%40% 15%15%15%15%

10%10%10%10%

15%15%15%15%

20%20%20%20%

0%0%0%0%


Should test:• …• …• …• …• …• ...

Won’t test:• …• …

Could test:• …• …• …• …

Testing under pressure

Must test:• …• …• …• …

OUCH!!OUCH!!OUCH!!OUCH!!


Risk & Requirement Based Testing

Added value:

• Product risks: use the same language as stakeholders

• Unambiguous method for prioritizing product risks

• Detect gaps in requirements

• Good base for managing progress of test project


Agenda



• TestFrame

• Discussion


TestFrame


TestFrame

• LogicaCMG’s method for structured testing– Testing plays a role in quality

• Verifying test process and products

• Working in a structured way

– Testing takes effort

• Re-use defined products

• Optimal use of tools

– Testing is not a goal, it is used to reach a goal

• Use consciously and goal-oriented

• Risk analysis

– Testing has to fit with development method and organization

• Flexible and fitting


TestFrame model

Reusable Test ProductsReusable Test ProductsReusable Test ProductsReusable Test ProductsReusable Test ProductsReusable Test ProductsReusable Test ProductsReusable Test Products

QualityQualityQualityQualityQualityQualityQualityQuality--------totototototototo--------marketmarketmarketmarketmarketmarketmarketmarket

& Time& Time& Time& Time& Time& Time& Time& Time--------totototototototo--------marketmarketmarketmarketmarketmarketmarketmarket

SSSSTTTTRRRRUUUUCCCCTTTTUUUURRRRIIIINNNNGG

TTTTOOOOOOOOLLLLIIIINNNNGGGG

FFFFIIIITTTTTTTTIIIINNNNGGGG

Adjust testing to organization

(not vice versa)

Structured processStructured products

Use of tools for test analysis, test execution and test management


TestFrame phases

Preparation

Analysis Navigation

Execution

Preparation

Analysis Navigation

Execution


Assessment

Report

Risk

analysis

Plan of

ApproachTest plan

Technical

test

Engine

Navigation

scripts

Navigation

structure

Test

report

Evaluation

report

Handover

set

Findings

Scoping

Test

Conditions

Test

cases

Clusters

TestFrame products

Analysis

Preparation

Navigation

Execution


Test specification – Breakdown & automation

2424

Test reportsTest tool& Engine

Actionwords

Automated test scripts

System Under Test

Test cases

Test conditions

Test clusters


Èxample setup using Excel


Testing with action words (1)

• Identify the smallest functional test step

• Assign an action word to every step

• Add parameters

• Use the action words when defining test cases and test scenario’s

• When necessary: automate the action words


Test preparation and - specification

Test execution

Test cluster

A B C D…check balance Taylor 0 transfer Houston Taylor 210check balance Taylor 210transfer Smith Taylor 150check balance Taylor 360. . .

A B C D…check balance Taylor 0 transfer Houston Taylor 210check balance Taylor 210transfer Smith Taylor 150check balance Taylor 360. . .

Testing with action words (2)

Automated test execution

Navigation script

…check balancetransfer…


TestFrame: facts & advantages

• Easy accessible and well maintainable test products

• Early start possible and therefore time saving at the end of the project

• Effort “scarce” experts is minimized

• Better steering and control test execution process

• Test automation possible without adjustments to test documentation

• Test automation also possible when application is not 100% correct

• Easy reporting on test execution status and test results


Conclusion

Risk & Requirement Based Testing:Risk & Requirement Based Testing:Risk & Requirement Based Testing:Risk & Requirement Based Testing:

Make the right choicesMake the right choicesMake the right choicesMake the right choices

TestFrameTestFrameTestFrameTestFrame::::

Do it structuredDo it structuredDo it structuredDo it structured

“… for a list of all the ways technology has failed to improve the quality of life, please press 3.”

(Alice Kahn)


Agenda



• TestFrame

• Discussion


ne global team

LogicaCMG & Testing: TestFrame & RRBT - SAST · Risk & Requirement Based Testing & TestFrame 3...

Documents

Transcript of LogicaCMG & Testing: TestFrame & RRBT - SAST · Risk & Requirement Based Testing & TestFrame 3...