UNIVERSITY OF WATERLOO Faculty of Engineering

REQUIREMENTS TRACEABILITY TECHNIQUES IN SOFTWARE DEVELOPMENT

RBC Financial Group Toronto, Ontario

Prepared by Kishore Chittella

ID 20230514 1A Systems Design Engineering

May 8, 2007

Kishore Chittella 5506 Fleur-De-Lis Court Mississauga, Ontario L5R 2Z5 May 1, 2007 Ms. P. Jarvie, Executive Director Co-operative Education & Career Services University of Waterloo Waterloo, Ontario N2L 3G1 Dear Madam, The enclosed report Requirements Traceability in Software Development was prepared as my 1A work term report for RBC Financial Group. This is my first work term report. The purpose of the report is to discuss and compare the requirements traceability techniques used in the development of an online application during the System Design Lifecycle (SDLC) Process. The Quality Assurance team of the Global Technology and Operations, with which I was employed, had begun to trace requirements for an online application being developed. The traceability was done using Mercury Quality Centre, a testing and requirements management software. During the process, I also discovered another method of tracing requirements, traceability using a requirements traceability matrix. Due to the sheer importance of requirements traceability in software development and its impact on time and costs, I found it suitable to compare the two techniques of requirements traceability for my work term report. This report was written entirely by me and has not received any previous credit at this or any other institution. I would like to thank Mr. Jeff Somerville and Ms. Sylvia Both for providing me with valuable reading materials that helped gain understanding on the SDLC process. Sincerely, Kishore Chittella 20230514

Table of Contents

Summary ............................................................................................................................. 1

1.0 Introduction................................................................................................................... 2

2.0 Requirements Traceability ............................................................................................ 3

3.0 Traceability Items ......................................................................................................... 3

3.1 Use Cases and use case scenarios ............................................................................. 3

3.2 Test Cases ................................................................................................................. 4

4.0 Benefits of Requirements Traceability ......................................................................... 4

5.0 Techniques of Requirements Traceability .................................................................... 5

5.1 Manual Coverage using a Traceability Matrix ......................................................... 5

5.2 Requirements Management Software ....................................................................... 6

5.2.1 Test plan module................................................................................................ 6

5.2.2 Requirements Module........................................................................................ 7

5.2.3 Requirements Coverage Functionality............................................................... 8

6.0 Evaluation of Traceability Methods.............................................................................. 9

6.1 Communication between stakeholders ..................................................................... 9

6.2 Security ................................................................................................................... 10

6.3 Usability.................................................................................................................. 11

6.4 Change Management .............................................................................................. 12

6.5 Reporting................................................................................................................. 12

7.0 Conclusion .................................................................................................................. 15

8.0 Recommendations....................................................................................................... 15

9.0 References................................................................................................................... 16

List of Tables and Figures

Table 1: Sample Test case to test the log out scenario (success) of an online application. 4

Table 2: Sample Traceability Matrix .................................................................................. 5

Figure 1: Toolbar for the Test plan module ........................................................................ 6

Figure 2: Toolbar for individual test cases ......................................................................... 7

Figure 3: View of Requirements Module with sample requirements ................................. 8

Figure 5: Login access for users in MQC ......................................................................... 10

Figure 6: History log for requirements ............................................................................. 11

Figure 8: Coverage analysis for a sample requirement..................................................... 13

Summary

The report outlines the techniques of requirements traceability in software development

during the System Development Lifecycle (SDLC) Process. Two important techniques

have been identified to address requirements traceability. These include: manually

mapping the projects traceability items using a traceability matrix and using

requirements management software such as the requirements coverage module in the

Mercury Quality Centre. The purpose of the report is to study and evaluate the techniques

of requirements traceability. The criteria used for comparison are: communication with

stakeholders, security, complexity/usability, change management, learning curve,

reporting, cost and time. Through qualitative analysis, it was determined that Mercury

Quality Centre (requirements management software) is far more superior to a traceability

matrix.

1

1.0 Introduction The System Development Lifecycle (SDLC) is the process used at RBC for Software

Development. Within the process, the Quality Assurance team in the Channels &

Integration Services at RBC is involved in functional, performance and user acceptance

testing of the applications.

The Quality Assurance team of the Global Technology and Operations, with which I was

employed, had begun to trace requirements for an online application being developed.

While working on the requirements traceability project for the online application, two

important techniques have been identified to address requirements coverage and

traceability. These include: manually mapping the projects traceability items using a

traceability matrix designed in Microsoft Excel and using the requirements coverage

module in the Mercury Quality Centre.

In this report, the two techniques will be compared. The criteria for comparison are:

Communication between stakeholders Security Usability Change management Reporting

The report also focuses on traceability, traceability items and its benefits. Due to

confidentiality issues, a generic test case and use case will be discussed instead of a

project specific artifact.

2

2.0 Requirements Traceability

Software applications are developed to meet certain requirements specified by the

Business team. Throughout the SDLC, it is important to ensure that the application being

developed includes the desired functionalities and that they are thoroughly tested before

delivery. This is achieved through establishing linkage between traceability items such as

test cases and use cases (more specifically use case scenarios). Therefore, every time a

test case is executed and passed by the QA team, its corresponding use case scenario is

also passed. This assists the QA team to ensure that all the functionalities (documented as

use case scenarios) have been thoroughly tested before the application is released. This

process is referred to as requirements coverage or requirements traceability. In this report,

the terms use case scenario and requirement are interchangeable since they were the only

type of requirements used in the process.

3.0 Traceability Items

Traceability item is defined as any project element that needs to be explicitly traced from

another textual or model item in order to keep track of the dependencies between them

[1]. Most common types of traceability items include use cases, use case scenarios, and

Test Cases. Traceability items are commonly referred to as project artifacts. I will

specifically focus on use case scenarios and test cases since they were the project artifacts

I encountered during the requirements traceability process.

3.1 Use Cases and use case scenarios

A use case is defined as a description of the possible sequences of interactions between

the system under discussion and its external actors, related to a particular goal [2]. Use

case is a project artifact that describes the functional requirements of a system or sub-

system. Use cases contain use case scenarios that describe how the system must interact

with the users.

3

For example, a use case describing the login/logout functionality of an online application

may contain three scenarios. They are: the initial login screen where the user inputs the

username and password, the landing page where the user is taken to upon logging in and

the exit page where the user is taken to upon logging out from the application. If the use

case is labeled as 100, these use case scenarios are labeled as 100.MS.1, 100.MS.2,

100.MS.3 where MS stands for Main Scenario. There may be Alternate Scenarios where

the system fails to login/logout the user.

3.2 Test Cases

A test case is a series of steps that must be executed to test the behavior of a functionality

of a system. Each step in a test case has a description and an expected result. For instance

a test case to test the log out scenario (success) of an online application may contain two

steps.

Table 1: Sample Test case to test the log out scenario (success) of an online application

Step Description Expected Result

1 User clicks on Logout A kiosk window appears

with the message Do you

want to exit from the

application? Yes/No?

2 User clicks Yes Exit page is displayed

4.0 Benefits of Requirements Traceability

Traceability plays a major role in QA and functional testing. For example, if test cases

are linked to use cases, whenever a test case is executed and passed, the linked use case

scenario will also be passed. This ensures that a particular system scenario has been

tested. Also, if a particular test case fails, the QA team will be aware that there is a defect

associated with the linked functionality. Therefore, traceability ensures that the

4

functionalities of the online application in development are thoroughly tested by the QA

team before its release.

Furthermore, when projects evolve, requirements change and use case scenarios are

modified. To ensure accurate test coverage, the corresponding test cases linked to the

modified use case scenarios must also be changed. Requirements tracing assists the QA

team to determine the test cases that will be affected due to a change in requirements.

Finally, requirements Traceability saves the company resources in terms of money and

time. For instance, if the QA team wanted to test a specific functionality without testing

the entire application, it can determine the associated test cases and execute them, instead

of conducting exhaustive testing (executing all test cases to test the application). This

saves the team much time, money and accelerates the time to market of the product.

.

5.0 Techniques of Requirements Traceability

Two important techniques have been identified to address requirements coverage and

traceability. These include: manually mapping the projects traceability items using a

traceability matrix designed in Microsoft Excel and using the requirements coverage

module in the Mercury Quality Centre.

5.1 Manual Coverage using a Traceability Matrix

In a traceability matrix, a single use-case scenario may be traced to more than one test

case and vice-versa. It allows the QA team to identify the test cases that need to be

executed to test specific use case scenarios (functionalities of the system). It also helps

determine whether the requirements have complete coverage i.e. whether all the use case

scenarios are traced to at least one test case. The traceability matrices are developed and

stored in Microsoft Excel spreadsheets. The following is a sample Traceability Matrix:

Table 2: Sample Traceability Matrix

5

TC1 TC2 TC3 TC4

100.MS.1 X

100.MS.2 X X X

100.MS.3 X X

100.MS.4 X X

According to the above matrix, executing test case TC1 would test the use case scenarios

(functionalities) 100.MS.1, 100.MS.2 and 100.MS.4 of a system. The actual details of the

test cases and use case scenarios must be saved in separate word documents.

5.2 Requirements Management Software

In general, requirements management applications like the Mercury Quality Centre

contain two modules: test plan and requirements modules. The requirements coverage

functionality is embedded within the test plan module. The test plan module allows the

user to create test cases and further provides the user with a detailed view where the test

cases details, design steps (test steps), test script, attachments and requirements coverage

can be modified. The requirements module allows the user to add requirements (use case

scenarios in this case) along with descriptions. The requirements coverage functionality

allows the user to establish traceability between test cases and requirements.

5.2.1 Test plan module

The test plan module allows the user to create new test cases. It is comprised of two

windows. The left window lists all the test cases in the projects database. The right

window lists the detailed view for the selected test case such as design steps and the

requirements coverage functionality.

Figure 1: Toolbar for the Test plan module

6

As illustrated in the test plan module toolbar (Figure 1), the user has an option to

add/delete test cases from the database. Also, the user can search for a specific test case

within the database. This toolbar is present in the left window.

Figure 2: Toolbar for individual test cases

As illustrated in the toolbar for individual test cases (Figure 2), the user has an option to

add/remove test steps from each test case. This toolbar is present in the right window. As

seen, the tool bar also has access to the requirements coverage functionality. Both

toolbars have other features which are not the focus of the report.

5.2.2 Requirements Module

The requirements module allows the user to add new requirements (use case scenarios) to

the database. These requirements can be accessed from the requirements coverage

functionality when tracing requirements to test cases. As seen (Figure 3), the

requirements module logs the date and time when a requirement was created and the user

who created it. Also, the requirements can be organized in the form of collapsible entries

(levels). The direct cover status indicates whether a requirement has been traced to a test

case.

7

Figure 3: View of Requirements Module with sample requirements

5.2.3 Requirements Coverage Functionality

Figure 4 shows the detailed view for an individual test case in the database. As seen, this

contains details, design steps, test script, attachments and requirements coverage tabs. To

establish requirements coverage for a test case, one must first select that test case to view

the corresponding detailed view, shift the tab to requirements coverage in the detailed

view window and then drag and drop a requirement from the right panel to the left panel.

The requirements displayed in the right panel are located in the requirements module

where they can be modified if necessary.

8

Figure 4: Detailed view of a test case showing the Requirements coverage functionality

6.0 Evaluation of Traceability Methods

Two methods for requirements traceability have been identified and described. The

various sections in the report analyze the qualitative aspects of Mercury Quality Centre

and Traceability Matrix.

6.1 Communication between stakeholders

Mercury Quality Centre is an application capable of collaborative computing. In

collaborative computing, different stakeholders of the project have access to the

application at the same time, giving them the permission to access, add and update the

information. Members of the QA team can be given separate login usernames and

9

passwords (Figure 5) so that they can all simultaneously work on tracing requirements to

test cases. This saves the QA team time, money and speeds up the requirements coverage

process. Furthermore, whenever coverage is established for a test case, the software

keeps track of the user who created it by logging the username and also the time.

Figure 5: Login access for users in MQC

However, in the case of traceability matrix created using spreadsheet applications such as

Microsoft Excel, multiple users cannot access the file simultaneously as it is saved on a

certain individuals computer. Also by using a traceability matrix, the project team will

be unable to determine who traced a specific requirement or who created specific test

cases and requirements.

6.2 Security

As illustrated in Figure 5, in Mercury Quality Centre, members of the QA team can be

provided with their own username and password. Furthermore, MQC also allows the

administrators to permission the users. For example, depending on the seniority of the

QA tester and the role in the team, he/she may or may not have access to deleting the

requirements added to test cases in the requirements coverage view. Also, whenever a

requirement is added to the test case in the coverage view, a history log is created (Figure

6). This makes the over all process of requirements coverage more safe. The direct cover

status entries indicate that the requirement has been traced to a test case.

10

Figure 6: History log for requirements

However in the case of requirements traceability using a spreadsheet application such as

Microsoft Excel, no user login feature and history logs exist, making it impossible to

track the user responsible for particular changes in requirements coverage. Also, Excel

does not allow permissions to be setup for specific fields. Therefore, anyone with access

to the entire requirements traceability matrix will be able to add/remove the traceability

between use case scenarios and test cases by modifying the markers (Xs in this case).

6.3 Usability

In Mercury Quality Centre, when tracing requirements to test cases in the requirements

coverage view, the user has an option of searching through the requirements in the

database to find the required requirement. This especially helps in projects with a large

number of requirements. Also, the requirements can be organized in the form of

collapsible entries where major functionalities can form the main level entries (not

indented) and the requirements may be indented and placed under the main level. In

projects with a large number of requirements, this feature will help categorize the

requirements as per the functionalities of the system. The descriptions for both the

requirements and test cases can be viewed by switching between the modules, saving

time for the user.

However, in a traceability matrix created using Excel, the user has to rely on the find

feature in Excel to search the matrix. In a traceability matrix, the test cases and use case

scenarios must follow certain naming conventions such as 100.MS.1 (refers to main

11

scenario 1 in use case 100) and TC1 (test case 1). Hence if the QA tester requires more

information about the use case scenario or the test case, he/she has to find the actual

document that contains the use case or test case. This process can get cumbersome for

large projects.

Furthermore, MQC is not installed on all the computers. Therefore, if the user does not

have access to a MQC installed computer, he/she will not be able to establish the

traceability. However in the case of a traceability matrix, a copy of the matrix can be

printed and manually filled.

6.4 Change Management

As projects evolve, requirements change. This in turn requires the QA tester to modify

the test cases accordingly. In the case of Mercury Quality Centre, the user can determine

the linked test cases with the help of the requirements coverage functionality and modify

them in the test plan module since it contains the detailed view of the test case. This

requires the user to simply switch between the modules.

However in the case of a traceability matrix, the user must first access the matrix to

determine the test cases that are affected due to a change in the requirements. Then the

test case documents must be tracked and the test case must be modified in that particular

document. This procedure involves the user to open multiple files, thereby slowing down

the requirements traceability process.

6.5 Reporting

Mercury Quality Centre is capable of generating test coverage reports and graphs to assist

in requirements traceability. For instance, it can generate a graph (Figure 7) to determine

the number of requirements that have been failed, no run, not completed, not covered or

passed as a result of test case execution.

12

Figure 7: Requirements Coverage Progress Graph

Also it can conduct coverage analysis for each requirement based on its

coverage/traceability.

Figure 8: Coverage analysis for a sample requirement

The requirements traceability module is capable of conducting coverage analysis for each

requirement. In Figure 8, a total of 88 (15+24+33+16) test cases have been traced to the

sample requirement. Out of this, 15 test cases have failed, 24 were not run and 33 were

not completed. This suggests that the requirement is not completely tested.

13

However in a traceability matrix created using Excel, there are no available built-in

features to conduct such analysis. Whenever a test case is executed, the user must

manually update the traceability matrix with its status. This can be achieved by

highlighting or bolding the markers (Xs in this case). This can get unmanageable in large

projects with many requirements.

14

7.0 Conclusion

From the qualitative analysis, it can be concluded that Mercury Quality Centre is far

more superior to Traceability Matrix technique for requirements traceability. MQC

facilitates better communication between different stakeholders of the project in

comparison to a traceability matrix in Microsoft Excel. It has security and logging

features that are not available in traceability matrices in Microsoft Excel. It also proves to

be superior to a traceability matrix in terms of usability and change management. Finally,

Mercury Quality Centre is capable of generating graphs and reports from test execution

results for requirements coverage analysis. In the case of a traceability matrix, this has to

be done manually.

8.0 Recommendations

Based on the analysis in the report, it is suggested that RBC use Mercury Quality Centre

to establish requirements traceability for the online applications in development as it

provides a safe and organized platform for requirements coverage.

Though a traceability matrix does not include all the features of requirements

management software like MQC, it may be utilized when a single user tests the coverage

for less than 100 requirements. Also, in places where there is no access to computers

(Mercury Quality Centre), traceability matrix spreadsheets can be printed, manually filled

and then synchronized with the data in Mercury Quality Centre.

15

9.0 References

[1] IBM. Traceability from Use cases to Test cases. IBM Software Group, 27 August

2003. [Online]. Available: http://www-

128.ibm.com/developerworks/rational/library/04/r-3217/3217_rm14.pdf. [Accessed:

February 27, 2007]

[2] A. Cockburn, Writing Effective Use Cases. Addison-Wesley Pub Co, 2000.

16

Table of Contents List of Tables and Figures Summary 2.0 Requirements Traceability 3.0 Traceability Items 3.1 Use Cases and use case scenarios 3.2 Test Cases 4.0 Benefits of Requirements Traceability 5.0 Techniques of Requirements Traceability 5.1 Manual Coverage using a Traceability Matrix 5.2 Requirements Management Software 5.2.1 Test plan module 5.2.2 Requirements Module 5.2.3 Requirements Coverage Functionality

6.0 Evaluation of Traceability Methods 6.1 Communication between stakeholders 6.2 Security 6.3 Usability 6.4 Change Management 6.5 Reporting

7.0 Conclusion 8.0 Recommendations 9.0 References