Automatic Generation of System Test Cases from Use Case Specifications

.lusoftware verification & validationVVS

Automatic Generation of System Test Cases from Use Case Specifications

Chunhui Wang, Fabrizio Pastore, Arda Goknil, Lionel Briand, Zohaib Iqbal

University of Luxembourg, !Luxembourg!

{chunhui.wang,fabrizio.pastore,arda.goknil,lionel.briand, muhammad.zohaib}@uni.lu

ISSTA’15, Baltimore, July 17th, 2015

Problem

• Verifying the compliance of a system with its requirements in a cost-effective way

• Traceability between requirements and system test cases – This cannot be manual

• E.g., automotive industry which must comply to ISO 26262

2

Objectives

• Automatically generate test cases from requirements

• Capture and create traceability information between test cases and requirements

3

Context Matters

• Requirements are captured through use cases

• Use cases are used to communicate with customers and the system test team

• Complete and precise behavioral models are not an option: too expensive

4

Strategy

• Analyzable use case specifications

• Automatically extract test model from the use case specifications

• Minimize modeling

• No behavioral modeling

5

Use Case Modeling for System Tests Generation (UMTG)

6

Elicit Use Cases 1

THE ACTOR SEND THE SYSTEM VALI THE SYSTEM DIS THE ACTOR SEND



RUCM Use Cases

7

RUCM Use Case Example

8

Use Case Name: Identify Occupancy Status Actors: AirbagControlUnit Precondition: The system has been initialized . . .

"Basic Flow 1. The seat SENDS occupancy status TO the system. 2. INCLUDE USE CASE Classify occupancy status. 3. The system VALIDATES THAT the occupant class for airbag control is valid. 4. The system SENDS the occupant class for airbag control TO AirbagControlUnit.

Specific Alternative Flow RFS 3 1. IF the occupant class for airbag control is not valid THEN 2. The system SENDS the previous occupant class for airbag control TO … 4. ENDIF

Postcondition: The occupant class for airbag control has been sent.

Postcondition: The previous occupant class for airbag control has been sent.

[Yue et al. TOSEM’13]

Elicit Use Cases 1




RUCM Use Cases

Model the Domain 2

Domain Model

9

1

System

AirbagControl Sensor

OccupantStatus

- OccupantClassForAirbagControl - OccupantClassForSeatBeltReminder

1

1 1

1 1

1

1..*

Elicit Use Cases 1

Model the Domain 2




RUCM Use Cases

Generate Scenarios and Inputs

6

ERRORS ARE ABSENT

TEMPERATURE IS LOW

STATUS IS VALID

Identify Constraints 4

Constraint descriptions

Evaluate Consistency

3

Domain Model

Based on Natural Language Processing

10

11

Basic Flow

1. The seat SENDS occupancy status TO the system.

2. INCLUDE USE CASE Classify occupancy status.

3. The system VALIDATES THAT

the occupant class for airbag control is valid and

the occupant class for seat belt reminder is valid.

4. The system SENDS the occupant class for airbag control TO AirbagControlUnit.

5. The system SENDS the occupant class for seat belt reminder TO SeatBeltControlUnit.

6. The System Waits for next execution cycle.

Postcondition: The occupant class for airbag control and the occupant class for seat belt reminder have been sent.

INPUT STEP

INCLUDE STEP

CONDITIONAL STEP

OUTPUT STEP

OUTPUT STEP

INTERNAL STEP

POSTCONDITION

DOMAIN ENTITY

CONSTRAINT

CONSTRAINT

DOMAIN ENTITY

DOMAIN ENTITY

ERRORS ARE ABSENT

TEMPERATURE IS LOW

STATUS IS VALID



6

Elicit Use Cases 1





Model the Domain 2

12

RUCM Use Cases


3

Domain Model

Missing Entities

Evaluate Model Consistency

Occupant Class for Airbag Control

Occupant Class for Seat Belt Reminder

Domain Entities

Airbag Control Classification Filter Sensor

Tagged Use Case

Evaluate Model Consistency

14

Occupant Class for Airbag Control

Occupant Class for Seat Belt Reminder

Domain Entities

AirbagControl

System

Sensor

OccupantStatus

- OccupantClassForAirbagControl - OccupantClassForSeatBeltReminder

Airbag Control Classification Filter Sensor

ClassificationFilter

Tagged Use Case

1 1

1 1

1 1

1

1..*

1 1


6

Elicit Use Cases 1

Missing Entities




Model the Domain 2


3

Domain Model

15

ERRORS ARE ABSENT

TEMPERATURE IS LOW

STATUS IS VALID



RUCM Use Cases

Errors.size() == 0 Status != null

t > 0 && t < 50

Specify Constraints 5

OCL constraints

Identify Constraints

16

The occupant class for airbag control is valid

Voltage error is detected

List of Constraints

The occupant class for seat belt reminder is valid

Tagged Use Case

Voltage error is detected

List of Constraint Descriptions

The occupant class for seat belt reminder is valid

Tagged Use Case

Engineers Specify Constraints

System.allInstances() ->forAll(b | b.occupancyStatus.classForAirbag <> Status::Error)

Software Engineers specify corresponding OCL constraints

The occupant class for airbag control is valid

17


t > 0 && t < 50

Elicit Use Cases 1

Missing Entities





Model the Domain 2


3

Domain Model

ERRORS ARE ABSENT

TEMPERATURE IS LOW

STATUS IS VALID



18

RUCM Use Cases


6

Object Diagrams

Test Scenarios

OCL constraints

19

Basic Flow










INPUT STEP

INCLUDE STEP

CONDITIONAL STEP

OUTPUT STEP

OUTPUT STEP

INTERNAL STEP

POSTCONDITION

DOMAIN ENTITY

CONSTRAINT

CONSTRAINT

DOMAIN ENTITY

DOMAIN ENTITY

INCLUDE STEP

CONDITIONAL STEP

OUTPUT STEP

OUTPUT STEP

INTERNAL STEP

POSTCONDITION 20

Basic Flow










UseCaseStart

INPUT STEP DOMAIN ENTITY

CONSTRAINT

CONSTRAINT

DOMAIN ENTITY

DOMAIN ENTITY

Precondition

CONDITIONAL STEP

OUTPUT STEP

OUTPUT STEP

INTERNAL STEP

POSTCONDITION 21

Basic Flow










UseCaseStart

Input

INCLUDE STEP

DOMAIN ENTITY

CONSTRAINT

CONSTRAINT

DOMAIN ENTITY

DOMAIN ENTITY

Precondition

OccupancyStatus DomainEntity

DOMAIN ENTITY

22

UseCaseStart

Input

Include

Condition

Output

Exit

Condition

Exit Output

Exit


System.initialized = true

OCL Constraint

AirbagControlUnit.AllI…

OCL Constraint

… OCL Constraint

… DomainEntity

OCL Constraint OccupancyStatus.ClassForSeat <> Error AND OccupancyStatus.ClassForAirbag <> Error

23

UseCaseStart

Input

Include

Condition

Output

Exit

Condition

Exit Output

Exit

System.initialized = true AND OccupancyStatus.ClassForSeat <> Error AND OccupancyStatus.ClassForAirbag <> Error

Path condition:

OCL Constraint OccupancyStatus.ClassForSeat <> Error AND OccupancyStatus.ClassForAirbag <> Error OccupancyStatus

DomainEntity


OCL Constraint

AirbagControlUnit.AllI…

OCL Constraint

OCL Path Condition: System.allInstances() -> forAll(b | b.initialized = true ) AND OccupancyStatus.allInstances()

-> forAll(o| o.ClassForSeat <> Status:: Error) AND OccupancyStatus.allInstances()

-> forAll(o| o.ClassForAirbag <> Status::Error)

24

System.initialized = true AND OccupancyStatus.ClassForSeat <> Error AND OccupancyStatus.ClassForAirbag <> Error

Path condition:

OCL Solver

:System Initialized = true Version = 3

:OccupancyStatus ClassForSeat = Adult ClassForAirbag = Adult

Input Values OccupancyStatus.ClassForAirbag = Adult OccupancyStatus.ClassForSeat = Adult System.initialized = true

Object Diagram

1

System

AirbagControl Sensor

OccupantStatus - ClassForAirbag - ClassForSeat 1

1 1

1 1

1

1..*

Domain Model:




ERRORS ARE ABSENT

TEMPERATURE IS LOW

STATUS IS VALID




t > 0 && t < 50


6

Elicit Use Cases 1

Missing Entities


OCL constraints

Model the Domain 2


3

Domain Model RUCM Use Cases

Generate Test Cases

7

Test Cases Object

Diagrams Test

Scenarios 25

26

UseCaseStart

Input

Include

Condition

Output

Exit

Test Case

occupancyStatus.ClassForAirbag = Adult


Scenario Under Test


<INPUT>

<CHECK>

<INPUT>

AirbagControlUnit.AllInstances() -> forAll(a|a.status = a.bodySense.occupancyStatus)


OCL Constraint

Input Values occupancyStatus.ClassForAirbag = Adult occupancyStatus.ClassForSeat = Adult System.initialized = true

occupancyStatus.ClassForSeat = Adult AirbagControlUnit.AllI…

OCL Constraint

Abstract Test Case




ERRORS ARE ABSENT

TEMPERATURE IS LOW

STATUS IS VALID




t > 0 && t < 50


6

Elicit Use Cases 1

Missing Entities


OCL constraints

Model the Domain 2


3


Generate Test Cases

7

Test Cases Object

Diagrams Test

Scenarios 27

Mapping Table

Generation of Executable Test Cases

28

ResetPower (Time = INIT_TIME)

SetBus (Chanel = RELAY_CLASS, Status = CLASS1)

ReadAndCheckBus ( D0 = OCCUPIED, D1 = OCCUPIED)

Abstract test steps Resulting Test Driver Functions

Matching Patterns Function Name Parameters

<INPUT> Classfor* = Adult SetBus

<INPUT> System.initialized = true ResetPower

<CHECK> AirBagControlUnit.allInstant… ReadAndCheckBus

Chanel = RELAY_CLASS Status = CLASS1 Time = INIT_TIME

Mapping Table:

occupancyStatus.ClassForAirbag = Adult System.initialized = true

<INPUT>

<CHECK>

<INPUT>

AirbagControlUnit.AllInstances() -> forAll(a|a.status = a.bodySense.occupancyStatus)

occupancyStatus.ClassForSeat = Adult

Abstract Test Case:

D0=OCCUPIED …

Executable Test Case:




ERRORS ARE ABSENT

TEMPERATURE IS LOW

STATUS IS VALID




t > 0 && t < 50


6

Elicit Use Cases 1

Missing Entities


OCL constraints

Model the Domain 2


3


Generate Test Cases

7

Test Cases Object

Diagrams Test

Scenarios 29

Mapping Table

Case Study • BodySense, embedded system for detecting occupancy

status in a car

• Evaluation:

• Cost of additional modelling

• Effectiveness in terms of covered scenarios compared to current practice at IEE

30

Complexity of the Case Study

31

Use Case Steps Use Case Flows

UC1 50 8 UC2 44 13 UC3 35 8 UC4 59 11 UC5 30 8 UC6 25 6

Costs of Additional Modeling

32

Use Case Steps Use Case Flows

OCL Constraints

UC1 50 8 9 UC2 44 13 7 UC3 35 8 8 UC4 59 11 12 UC5 30 8 5 UC6 25 6 12

5 to 10 minutes to write each constraints

Effectiveness: scenarios covered

33

0

5

10

15

20

25

30

35

40

UC1 UC2 UC3 UC4 UC5 UC6 Scenarios Covered By Engineer Scenarios Covered By UMTG

100%

100%

100%

100% 100% 100%

81%

77% 100%

86%

50% 67%

It is hard for engineers to capture all the possible scenarios involving error conditions.

Problem

• Requirements-driven test automation

• Traceability between requirements and system test cases – This cannot be manual

• E.g., automotive industry which must comply to ISO 26262

34

ERRORS ARE ABSENT

TEMPERATURE IS LOW STATUS IS VALID

Errors.size() == 0 Status �= null ��> 0 && t < 50



6 Generate Test Cases

7


3

Elicit Use Cases 1

Model the Domain 2


Domain Model

Missing Entities

Constraint descriptions OCL constraints

THE ACTOR SEND �HE SYSTEM VALI �HE SYSTEM DIS THE ACTOR SEND



RUCM Use Cases

��

Test Cases Object

Diagrams Test

Scenarios 0

5

10

15

20

25

30

35

40

UC1 UC2 UC3 UC4 UC5 UC6 Scenarios Covered By Engineer Scenarios Covered By UMTG

100%

100%

100%

100% 100% 100%

81%

77% 100%

86%

50% 67%

Acceptable modelling effort Full scenario coverage

35

NLP Pipeline

Tokenizer UMTG Gazetteer

Part of Speech Tagger

UMTG Transducers

The seat SENDS occupancy status TO the system.

36

NLP Pipeline



UMTG Transducers


ACTOR SEND NOUN PHRASE SYSTEM TO

37

NLP Pipeline



UMTG Transducers


ACTOR SEND NOUN PHRASE SYSTEM TO

ACTOR SEND TO NOUN PHRSE SYSTEM

INPUT STEP PATTERN

Automatic Generation of System Test Cases from Use Case Specifications

Software

Transcript of Automatic Generation of System Test Cases from Use Case Specifications