Requirements-based Testing

SYST TEXT

1 Testing against the requirements

Extracting test cases from requirements

Use cases and test cases

Grammar types implying test cases

A requirement text

Objects identified in the requiremnt text

Actions on objects in the requirement text

States of objects in the requirement text

Conditions in the requirement text

Action oriented test cases

Conditional test cases

State based test cases

2

3

6

7

5

4

8

9

10

15

Functional test cases (1)

Functional test cases (2)

Test Case Table

Boundary analysis test cases

Non functional requirements

Non functional test cases

Properties of a test case

Test case description (1)

Test case description (2)

Test Database

From Requirements to Test Scripts

Generated Test Script

16

17

20

21

19

18

22

2311

13

14

12 24

Testing against the requirements

SYST TEXT-1

UserInterface

ApplicationSystem

Database

Extracting Test Cases from Requirements

SYST TEXT-2

Test Cases

Functional

Non-Functional

Objects

Actions

States

Conditions

Requirements Text

Use Cases and test Cases

SYST TEXT-3

One Use Case can have one or more Test case

Normally, there are at least 2 Test Cases

Positive outcome

Negative outcome

CashWithdrawal

Not enoughcash

Accountoverdrawn

Cashdelivered

WrongPIN Number

InvalidCard

Use Case =

Test Cases

Grammar Types implying Test Cases

SYST TEXT-4

Actions upon objects:

The customer account should be deleted.

The invoice should be sent within 30 days.

Object states:

The customer account is overdrawn.

The customer is an employee.

ConditionsIf the account is overdrawn a notice should be sent.Whenever someone fails to pay within 30 days, a reminder should be sent.Customer accounts should be deleted, if they are overdrawn more than 3 months.In case of a system crash, the recovery routine should be started.

A Requirements Text

For an Order Entry System

SYST TEXT-5

The order entry system to be developed should process orders coming in from customers.

Whenever an order comes in, the person placing the order should be checked if he is a known

customer and if his credibility is adequate. Orders will only be processed for existing customers with

adequate credit rating.

Each order can contain up to 10 order positions. Each filled order position will contain the number,

name and amount of the article desired. For each order it should first be checked if the article is on

stock. If not, that position will skipped. If the article is on stock, it should be checked if the amount on

stock is sufficient to cover the order. If not, a back order should be created for this position.

Otherwise, the order position should be fulfilled and the amount on stock reduced by the amount

ordered. In this case, both a dispatch order and a billing position are to be generated.

When the amount on stock is reduced, the new amount on stock is to be checked against the

minimum amount on stock. If it is less than the minimum amount, a supply order should be

automatically generated and sent to the supplier with the lowest supplier price for that article.

The billing positions are to be collected at the end of each week to be ordered by customer. The

amounts of the ordered articles are to be multiplied by the current price of the article and the charge

for each article computed. The charges for all articles ordered should then be totaled to give the total

charge to the customer.

Objects Identified in the Order Entry Requirement Text

SYST TEXT-6

1. Order_Entry_System

2. Orders

3. Customers

4. Persons_placing_orders

5. Credibility/Credit_Rating

6. Order_Positions

7. Number_of_Article

8. Name_of_Article

9. Amount_of_Article

10.Amount_on_Stock

11.Back_Order

12.Dispatch_Order

13.Billing_Position

14.Minimum_Amount_on_Stock

15.Supply_Order

16.Supplier

17.Supplier_Price

18.Current_Price_of_Article

19.Total_Charge

20.Value_Added_Tax

21.Article_Type

22.Bills

23.Days

Actions on Objects in the Order Entry Requirement Text

SYST TEXT-7

1. Process orders

2. Check persons placing orders

3. Chack article on stock

4. Check amount on stock

5. Create back order

6. Fulfill order position

7. Reduce amount on stock

8. Generate dispatch order

9. Generate billing position

10.Check new amount on stock

11.Generate supply order

12.Send supply order

13.Collect billing positions

14.Multiply amounts

15.Compute charge

16.Total charges

17.Augment total charge

18.Pay bills

States of Objects in the Order Entry Requirement Text

SYST TEXT-8

1. Orders coming in

2. Customer is known/not known

3. Credibility is adequate/not adequate

4. Customer exists/does not exist

5. Orders with up to 10 order positions

6. Order position fulfilled/not fulfilled

7. Article desired

8. Article is on stock/not on stock

9. Amount on stock is sufficient/not sufficient

10.Supplier with lowest price/not with lowest price

11.Positions ordered by customer

12.Pay within 30 days

Conditions in the Order Entry Requirement Text

SYST TEXT-9

1. Whenever an order comes in

2. If he is a known customer

3. If his credibility is adequate

4. For each order position

5. If the article is on stock

6. If the amount on stock is sufficient

7. If not (sufficient)

8. Otherwise, fulfill order

9. In this case (order fulfilled)

10.When the amount on stock is reduced

11. If it (amount on stock) is less than minimum amount

Action Oriented Test Cases

SYST TEXT-10

1. Process orders ← submit 1:n orders• Check persons placing orders ← submit order from unknown customer• Check article on stock ← order article not on stock• Check amount on stock ← set Order.Position.Amount > Article.Amount• Create back order ← order article with amount > amount on stock• Fulfill order position ← order article with amount < amount on stock• Reduce amount on stock ← order article with amount < amount on stock• Generate dispatch order ← order article with amount < amount on stock• Generate billing position ← order article with amount < amount on stock• Check new amount on stock ← order article with amount < amount on stock• Generate supply order ← amount on stock < minimum amount on stock• Send supply order ← amount on stock < minimum amount on stock

& supplier found with the lowest price• Collect billing position ← set time to trigger billing process• Multiply amounts ← have a customer with >0 billing positions• Compute charges ← have a customer with >0 billing positions• Total charges ← have a customer with >0 billing positions• Augment total charge ← set value added tax rate• Pay bills ← create billing positions for n customers

Conditional Test Cases

SYST TEXT-11

2. Whenever an order comes in ← submit order

submit no order• If he is a known customer ← customer is known

customer is not known• If his credibility is adequate ← customer.credit adequate

customer.credit not adequate• For each order position ← submit order with > 1 positions• If the article is on stock ← Order.Position.Art_Nr = Article.Art_Nr

Order.Position.Art_Nr != Article.Art_Nr

• If the amount on stock is sufficient ← Order.Position.Amount < Article.Amount• If not sufficient ← Order.Position.Amount => Article.Amount• Otherwise, fulfill order• In this case (order fulfilled)• When the amount on stock is reduced ← Order.Position.Amount < Article.Amount• If it (amount on stock) is less than ← Article.Amount < Minimum.Amount

minimum amount

State Based Test Cases

SYST TEXT-12

6. Orders coming in ← submit 1:n order

• Customer is known/not known ← Order.Cust_Nr ! e Set(Customer.Cust_Nr)

exist /not exist Order.Cust_Nr ! є Set(Customer.Cust_Nr)

• Credibility is adequate/not adequate ← Customer.Credit => #Adequate

Customer.Credit < #Adequate

• Order with up to 10 order positions← Order.Position (1:10)

• Order with more than 10 positions ← Order.Position (11)

• Order position is fulfilled/not fulfilled ← Order.Position_Nr > 0

Order.Position_Nr = 0

• Article on stock/not on stock ← Order.Position.Art_Nr є Set(Article.Art_Nr)

Order.Position.Art_Nr ! є Set(Article.Art_Nr)

• Amount on stock is sufficient/not sufficient ← Order.Position.Art_Nr -> Article.Amount >

Order.Amount

Order.Position.Art_Nr -> Article.Amount

<= Order.Amount

• Supplier with lowest price ← Article.Art_Nr є Set(Supplier.Art_Nr) &

Article.Art_Nr -> Supplier.Price =

min(Supplier.Price)

• Position ordered by customer ← Order.Position.Art_Nr = Article.Art.Nr

• Payment within 30 days ← Bill.Due.Date = Bill.Date + 30

The Functional Test Cases (1)

SYST TEXT-13

1. Order in which Order.Cust_Nr !є Set(Customer.Cust_Nr) → unknown customer

2. Order in which Order.Cust_Nr є Set(Customer.Cust_Nr) &

Customer.Credit < #Adequate → credit not

adequate

3. Order in which Order.Cust_Nr є Set(Customer.Cust_Nr) &

Customer.Credit => #Adequate &

Order.Position_Nr = 0 → null

positions

• Order in which Order.Cust_Nr є Set(Customer.Cust_Nr) &

Customer.Credit => #Adequate &

Order.Position_Nr > 0 &

Order.Position.Art_Nr !€ Set(Article.Art_Nr) → article not on

stock

• Order in which Order.Cust_Nr є Set(Customer.Cust_Nr) &

Customer.Credit => #Adequate &

Order.Position_Nr > 0 &

Order.Position.Art_Nr є Set(Article.Art_Nr) &

Order.Position.Amount > Article.Amount → create back order

The Functional Test Cases (2)

SYST TEXT-14

6. Order in which Order.Cust_Nr є Set(Customer.Cust_Nr) &

Customer.Credit => #Adequate &

Order.Position_Nr > 0 &

Order.Position.Art_Nr є Set(Article.Art_Nr) &

Order.Position.Amount < Article.Amount &

(Article.Amount – Order.Position.Amount) > Minimum_Amount

7. Order in which Order.Cust_Nr є Set(Customer.Cust_Nr) &

Customer.Credit => #Adequate & Order.Position_Nr > 0 &

Order.Position.Art_Nr є Set(Article.Art_Nr) & Order.Position.Amount < Article.Amount & (Article.Amount – Order.Position.Amount) < Minimum_Amount

& Art_Nr -> Supplier.Price = min(Supplier.Price)

8. Order in which Order.Position.Art_Nr є Set(Article.Art_Nr) &

Order.Position.Amount < Article.Amount &

(Article.Amount- Order.Position.Amount) < Minimum_Amount &

Art_Nr -> Supplier.Price > min(Supplier.Price)

Type Number Pos Source Use_Case TestObject Precondition Postcondition Arguments Input_ValueResults

Func 101285 1 TextAnal Kundenauftragsbearbeitung Kundenauftrag Kunde ~E(Set)Kunden keine Änderung Kundennr 1014710 Response_MsgFunc 101286 1 TextAnal Kundenauftragsbearbeitung Kundenauftrag Kunde E(Set)Kunden keine Änderung Kundennr 1014711 Response_MsgFunc 101286 2 Kundenauftragsbearbeitung Kundenauftrag Position(1).ArtNr ~E(Set)Artikel.ArtNr Position(1).ArtNr 300 Response_MsgFunc 101287 1 TextAnal Kundenauftragsbearbeitung Kundenauftrag Kunde E(Set)Kunden keine Änderung Kundennr 1014711 Response_MsgFunc 101287 2 Position(1).ArtNr E(Set)Artikel.ArtNr Position(1).ArtNr 305 Response_MsgFunc 101287 3 Position(1).Menge > Artikel.Menge Position(1).Menge 22 Response_MsgFunc 101287 4 Artikel.Menge 20 Artikel.MengeFunc 101288 1 TextAnal Kundenauftragsbearbeitung Kundenauftrag Kunde E(Set)Kunden Artikelmenge < Kundennr 1014711 Response_MsgFunc 101288 2 Position(2)ArtNr E(Set)Artikel.ArtNr Last.Artkelmenge Position(2).ArtNr 305 Artikel.MengeFunc 101288 3 Position(2)Menge < Artikel.Menge Position(2).Menge 5 Func 101288 4 Artikel.Menge 20 Artikel.MengeFunc 101289 1 TextAnal Kundenauftragsbearbeitung Kundenauftrag Kunde E(Set)Kunden Artikelmenge < Kundennr 1014711 Response_MsgFunc 101289 2 Position(3)ArtNr E(Set)Artikel.ArtNr Last.Artikelmenge Position(3).ArtNr 305 Artikel.MengeFunc 101289 3 Position(3)Menge = Artikel.Menge Position(3).Menge 15 Func 101289 4 Artikel.Menge 15 Artikel.Menge

Test Case Table

TEXT-14

TEXT-15SYST

Additional Functional Test Cases

With Boundary Analysis

SYST TEXT-16

1. Order.Position_Nr = 0, 1, 10, 11

2. Order.Position.Amount <, =, > Article.Amount

3. Art_Nr -> Supplier.Price <, =, > min(Supplier.Price)

4. Nr_Billing_Positions <, =, > max(Bill.Position_Nr)

Non-Functional Requirements

SYST TEXT-17

The order entry system must be able to handle at least 1000 customer web pages at one time. A customer order must be processed within 2 seconds. The article database may contain up to 10.000 articles. The system should be able to produce as many as 25.000  bills in one billing run. A billing run should not take more than 2  hours. If the system crashes it should be back up and in operation within 15 minutes. Database backups should take place every 4 hours.  Customers should be checked both by number and by password. Passwords should be rejected after 3 tries, customer number should be rejected after 2 tries.

Non-Functional Test Cases

SYST TEXT-18

1. 999 customer orders at one time

2. >1000 customer orders at one time

3. response time per order < 2 sec.

4. response time per order >2 sec.

5. 9.999 article records

6. > 10.000 article records

7. 24.999 bills in one billing

8. >25.000 bills in one billing

9. run time of billing < 2 hours

10. run time of billing >2 hours

11. If system crash, recovery < 15 minutes

12. If > 4 hours, database secured

13. Submit 3 passwords where 3rd password is correct

14. Submit 3 passwords where all 3 are incorrect

15. Submit 2 customer number where 2nd number is correct

16. Submit 2 customer number where both are incorrect 

Properties of a Test Case

SYST TEXT-19

A Test Case:

serves a purpose refers to a Testobject has a precondition state has a postcondition state has a set of required input parameters has a set of expected results has a unique identifier has a given environment tests a use case has a status has a date of execution

Test Case Description (1)

SYST TEXT-20

Test Case – ID: IC- Order.4Test Case – Purpose: to test article not on stockTest Object: Customer – OrderPrecondition: Order.Position.Art.Nr. ! € set (Article.Art.Nr)Postcondition: Response = „Article not available”

Article.Amounts = Last.AmountsInputs: Order.Cust.Nr = 200100

Customer.Credit = ’A’Order.Position.Nr = 2Order.Position.Art.Nr.= 9999

Results: Text-Msg = „ Article not available”Environment: MS- Internet Windows – XPUsecase – tested: Customer.order.processingStatus: testedDate: 25. 8. 2005.

Test Case Description (2)

SYST TEXT-21

TestCase ID: IC-Order-7Test Case – Purpose: to test a fulfilled orderTest-Object: Customer – OrderPrecondition: Ordered Article is on stock.

Article. Amount is > Order.Position.Amount(Article. Amount - Order. Position.Amount ) > minimum – Amount

Postcondition: Article.Amount = a Pre.Article.Amount – Order.Position.AmountNo supply order

Inputs: Order.Cust-Nr = 200100Customer.Credit = ’A’Order.Position-Nr = 2Order.Position.Art – Nr = 4711Order.Position.Amount = 3Article.Amount = 10Minimum – Amount = 7

Results: Text-Msg = „Article order fulfilled”Environment: MS – Internet, Windows – XPUsecase tested: Customer.Order.ProcessingStatus: testedDate: 29. 8. 2005.

Test Database

Master Table

SYST TEXT-22

Test a withdrawal from an accountAccountWithdrawalData1101704Regr

Test a deposit to an accountAccountDepositReq1101604Tech

Test the opening of an accountAccountOpen_AccountReq1101504Func

Test Case DescriptionTest_ObjectUse_CaseSourcePosNumberType

From Requirements via Test Cases to Test Scripts

SYST TEXT-23

TextAnalysis

TestCases

RepositoryLoader

Test CaseDB

RepositoryUnloader

TestScriptsTest

Testerenhances & refinesTest Cases

TesterexecutesTest Cases

Text AnalyzergeneratesTest Cases

Test Script Generatorgenerates Test Scriptsfor Test Automation

DLG-AUFTR/HOR_Öffnen { { Filename = "\GEOS_EU.200\A_ORDER\DLG\GAOAUFTR.93C"; Funktionsbeschreibung = "H1"; Aktion = "OPEN"; Modul = "GAOAUFTR"; }; Edit EB_EF_ENTRYFIELD { Controlbeschreibung = ""; Aktion = "EING"; MandantSpez = "nein"; Eingabe = "61230551302"; MK = "nein"; Inaktiv = "nein"; ReadOnly = "nein"; Domainentest = ""; Domain = ""; Default = ""; Mussfeld = ""; Parameter = ""; }; ComboBox EB_KB_KEYBOX { Controlbeschreibung = ""; Aktion = "AUSW"; MandantSpez = "nein"; Eingabe = "Derivat"; MK = "nein"; Inaktiv = "nein"; ReadOnly = "nein"; Domainentest = ""; Domain = ""; Default = ""; SVZAnzahl = "0"; Mussfeld = ""; Parameter = ""; };

Edit EB_EF_ENTRYFIELD4 { Controlbeschreibung = ""; Aktion = "EING"; MandantSpez = "nein"; Eingabe = "CODAX5800MÄR02"; MK = "nein"; Inaktiv = "nein"; ReadOnly = "nein"; Domainentest = ""; Domain = ""; Default = ""; Mussfeld = ""; Parameter = ""; }; Funktionenmenü ~Kaufen... { Controlbeschreibung = ""; Aktion = "KLCK"; MandantSpez = "nein"; Eingabe = ""; MK = "nein"; Inaktiv = "nein"; ReadOnly = "nein"; Domainentest = ""; Domain = ""; Default = ""; Mussfeld = ""; Parameter = ""; }; };

Generated Test ScriptTEXT-24SYST

Inputs to a Text Analyzer

SYST REQT-22

TextDocument

Object

List

PreProcessor

Text

ProcessorObject Scanner

Text Parser

FilterParameters-Files- Names

Function of a Text Analyzer

SYST REQT-23

TextAnalText Analyzer

ScanObjects

ParseText

CreateEntity Tables

GenerateOutputs

Text

Criteria

ObjectList

SectionsSentences

Words

Section TabObject TabAction TabState Tab

Condition Tab

Object Model

Test Cases

Rule Violations

Metrics

Repository Entries

Outputs from a Text Analyzer

SYST REQT-24

Text

Processor

TxtAnalTextPre

Processor

TxtScan

Object

List

ObjectModel

TestCases

RuleViolations

SpecMetrics

RepositoryEntries

Objects- Use Cases - Actions - States - Rules

TextExcelXML

Format RulesRupp Rules

SizeQuantityComplexity

Sections- Use Cases - Objects - Actions - States - Rules