Post on 07-Apr-2018
8/6/2019 Automated Embedded Software Testing Using Task Interaction Scenarios
1/18
Automated Embedded Software Testing using TaskInteraction Scenarios
Nam Hee Lee and Sung Deok Cha
Division of Computer Science and AITrc
Department of Electrical Engineering and Computer Science, KAIST
373-1, Gusung-Dong, Yusung-Gu, Daejon, Korea
{nhlee,cha}@salmosa.kaist.ac.kr
Abstract
Modern embedded software is often quite complex for testing interactions among tasks,
and it is desirable to automate testing process as much as possible for cost effective testing.
In this paper, we demonstrate how to automatically generate test cases from a set of scenarios
and execute them in host environment. We first extract the required interactions from the
design documents and developers knowledge. The internal interactions of each external input
event are described using Message Sequence Charts (MSCs), and test cases are automatically
generated from them. We test a substantial portion of the SEC DTV software written in C
and Java.
Keywords : task interaction testing, Message Sequence Charts, automated software testing
1 Introduction
Usage of software in controlling embedded systems has become popular, especially in home appli-
ance applications. Such embedded software is typically not only large in size, but are also quite
complex in requirements. For example, there are over 800,000 lines of C and Java application code
running on a digital TV, and such figure excludes the size of software implementing a real-time
operating system, library routines, and a Java virtual machine. Other embedded systems that are
heavily software-dependent include cellular phones, web phones, automobiles, etc. Such embedded
software must obviously meet an extremely high-level of quality assurance requirement, and there
Correspondence to: Nam Hee Lee, Division of Computer Science and AITrc Department of Electrical Engi-neering and Computer Science, KAIST 373-1, Gusung-Dong, Yusung-Gu, Daejon, Korea
E-mail:nhlee@salmosa.kaist.ac.krPartially supported by the Advanced Information Technology Research Center (AITrc)
1
8/6/2019 Automated Embedded Software Testing Using Task Interaction Scenarios
2/18
are strong financial motivations for developers to conduct as rigorous, cost-effective, and auto-
mated software quality assurance tasks as possible. It is well known in the software engineering
that testing phase alone consumes about the half of the total development cost and that it is still
practically impossible to deliver defect-free software to customers.
Development and testing of embedded software are especially difficult because it typically
consists of a large number of concurrently executing and interacting tasks. Each task in embedded
software is executed at different intervals, under different conditions, and with different timing
requirements. Furthermore, time available to develop and test embedded software is usually
quite limited due to relatively short lifetime of the products. Therefore, cost-effective testing of
embedded software is of critical concern in maintaining competitive edge. Among different phases
in embedded software testing, were most interested in integration testing where interfaces and
interactions among various subsystems are examined.
Natural language representation of interactions is grossly inadequate when automatically gener-
ating test cases. Statecharts[1], SDL[2], and Message Sequence Charts (MSCs)[3] are formal spec-
ification languages that have been successfully applied on several large-scale industrial projects.
Statecharts and SDL are state-oriented specification languages, and all possible external/internal
interactions are described in transitions. On the other hand, MSCs is an event-oriented specifica-
tion language where required interactions are separately described for each external/internal eventsequences and state information is captured as scenario activation conditions. We choose MSCs
formalism for specifying the interaction behavior since it is a more understandable language to
communicate with developers and various MSCs constructs provide features to express complex
behavior in a hierarchical manner. Moreover, the system behavior for each external input event
can be described separately and the internal interactions are expressed naturally and explicitly.
In this paper, interaction scenarios and various system state information are expressed in basic
MSCs (bMSCs). Since test cases for integration testing must consist of sequences of the external
input events and internal interactions, we obtain a global system behavior and generate test cases
by combining a set of MSCs, each of which captures the required but partial scenario. We define a
state in GFSM as the set of values of all the system state variables, and a transition as an ordered
sequence of messages that must be exchanged. Furthermore, transitions in GFSM are created only
when there are changes in the values of the systems state variables for an external event. Manual
application of integrated testing is obviously impractical when applied to software as large and
complex as digital TV software, and we have implemented a software toolset to automate the test
2
8/6/2019 Automated Embedded Software Testing Using Task Interaction Scenarios
3/18
process and test harnesses to executing the generated test cases.
Our paper is organized as follows. The SEC DTV software is briefly introduced in section 2. All
steps to specifying the interaction scenarios are expressed in section 3, and the proposed approach
to generating test cases from a set of MSCs is also described. Furthermore, test harnesses to
executing the generated test cases in host environment are represented. In section 4, lessons learnt
from the case study and experimental results are described. Some related works are presented in
section 5 and future research directions are discussed in section 6.
2 The SEC DTV Software
Digital Television (DTV) is an important new electric home appliance that enables the seamless
convergence of two currently separate media: broadcast TV, an inherently non-interactive medium,
and the Internet, an inherently interactive medium. Currently, broadcast, entertainment, and
consumer electronics industries are developing DTV at an extremely rapid speed. Our proposed
testing approach has been applied to the DTV developed by Samsung Electronics (SEC) which
implements DASE (Digital television Application Software Environment) standards[4], defined by
ATSC (Advanced Television Systems Committee). The SEC DTV software interacts with receiver
platform by accepting inputs from the broadcast transport stream and the end-user and generating
output on the platforms display and audio rendering systems.
DASE includes software modules that allow decoding and execution of application programs
that deliver interactive data broadcast services. The DASE content is generally organized as
a collection of one or more applications, each of which takes the form of either a declarative
application (HTML application), or a procedural application (Xlet application). The SEC DTV
software consists of Java modules and native applications written in C. The former provides
execution environment for the Xlet applications, and the latter provides execution environment
for the HTML applications and the vendor specific functionality such as e-mail.
Overall structure of the SEC DTV software is shown in Figure 1. The application manager,
implemented as a native task, named AL, and three Java threads, with colon in the prefix of the
name, accepts data input such as HTML files, Java class files, or other contents, and checks whether
the applications should be launched. The transport stream parser decodes the content of specific
protocols, PSIP (Program and System Information Protocol) and DB (Data Broadcast), and it is
implemented as two native tasks and three Java threads. The presentation engine, implemented
as a native task, is a content decoder for HTML applications, and provides application execution
3
8/6/2019 Automated Embedded Software Testing Using Task Interaction Scenarios
4/18
services in a uniform and platform-independent manner. It is integrated with the window manager.
In the case of Java class files, an Xlet viewer handler for each Xlet application is allocated, and it
is integrated with the application execution engine, Java VM.
Figure 1: The module structure of SEC DTV software
3 Automated Task Interaction Testing
The proposed approach for generating test cases is illustrated in Figure 2.
Figure 2: An approach for scenario-based test generation
4
8/6/2019 Automated Embedded Software Testing Using Task Interaction Scenarios
5/18
3.1 Event-Oriented Interaction Specification
MSCs is a graphical formalism which is used to visualize communication behavior (message ex-
changes). MSCs have been standardized by the ITU (International Telecommunications Union) in
Recommendation Z.120[3]. Each system component participating in the communication is called
an instance and is represented by a vertical time line in basic MSCs (bMSCs). Time passes from
the top of the chart to its bottom, and all events along each instance axis are totally ordered.
Events are either a message being sent or received. The only ordering constraint other than the
one imposed by the instance axis is inferred from messages: a message has to be sent before it is
received. Thus, a complete MSCs defines a partial order on the events it contains. Graphically,
messages are represented by arrows pointing in the direction of the message flow, and conditionsare depicted by hexagonal shapes.
Although MSCs is an event-oriented specification language, Z.120 also provides high-level MSCs
(hMSCs) to describe a complete behavior of systems in state-oriented manner[5]. While nodes and
frames denote sequential, selective, or parallel execution of scenarios, conditions are used to capture
the changes in system states. For example, the global behavior of the SEC DTV software can
be described using the constructs of hMSCs like Figure 3. The model represents a part of the
software which is a service for an HTML application. A life-cycle of an HTML application includes
installation, execution, pause, and finally termination.
State-oriented description of system behavior, while straightforward in concept, has disad-
vantages when specifying and revising large and complex industrial systems. Information about
possible states at which each event can occur must be known to be able to specify various require-
ments in state-oriented manner. Furthermore, when a new event is to be added, a large portion
of the model may need to be updated. As an alternative, we propose an event-oriented approach
based on scenario specification where the global behavior is derived by combining a set of partial
specifications. Advantages include enhanced understandability and changeability.
The first step to event-oriented interaction specification includes identification of various sub-
systems (i.e. tasks or threads) and separation of implementation under tests (IUT) from environ-
ments (ENV). IUT subsystems are highlighted as shaded boxes in Figure 1. Other subsystems,
the window manager and DASE API, etc., are excluded in this case study since the integration
testing techniques focus on detecting potential flaws on task interactions and message passing.
Since each subsystem consists of a set of tasks and threads, the whole tasks in each module must
be specified. For example, the transport stream parser consists of three Java threads and a native
5
8/6/2019 Automated Embedded Software Testing Using Task Interaction Scenarios
6/18
Figure 3: State-oriented model for HTML executions
6
8/6/2019 Automated Embedded Software Testing Using Task Interaction Scenarios
7/18
task for IUT tasks.
In the second step, all external input events are identified, and a top-level scenario is developed
as shown in Figure 4. Because each of the external input events may occur at any time and they
need to be properly handled as they occur, alt (selective) in-line construct is used. It should be
noted that the top-level scenario may be organized in a hierarchical manner. In our example,
DTV software receive input from two sources, the remote control unit and the transport stream.
The latter, in turn, consists of four inputs.
Figure 4: External input events for HTML executions
Messages and state variables are identified next. In the case of the native tasks, identifying
messages is simpler than Java threads since explicit and common event functions are used. Gen-
erally, real-time operating systems or the window manager support libraries for sending messages.
For example, q send() in pSOS and SendEventTask() in EsWin are event functions in the DTV
software. However, since each thread in Java maintains at least one message receiving method,
we need to identify them. Since each task or thread in embedded software may have states, we
identify these state variables. That is, a state variable raises a different interaction for the same
message received. For example, the application manager in Figure 1 accepts or ignores some re-
mote control inputs through currently running application which is represented by the variable
currentApp.
Finally, detailed interaction sequences for each external input events are separately described
7
8/6/2019 Automated Embedded Software Testing Using Task Interaction Scenarios
8/18
in bMSCs. Complex interaction patterns are hierarchically organized by referencing other bMSCs
within a bMSC. Accurate interaction sequences may be developed by (or with the participation
and review of) domain experts. Figure 5 represents a part of interactions when the external event
TV Video occurs. The DTV software performs such interactions when the systems state satisfies
the activation condition. In addition to simple message exchanges, the alt inline expression
indicates that after exchanging four messages, either Start HTML Installed (Figure 6) or Start-
HTML Paused (Figure 7) bMSCs is selectively activated. The exact conditions under which
bMSCs are activated are further specified in the referenced bMSCs. State changes occurring as
the result of executing a scenario is also specified as shown in Figure 6 where the HTML application
becomes ACTIVE using the method getState(). Other in-line expressions, opt or loop, may
be used as needed.
Figure 5: Launching HTML applications
3.2 Test Case Generation
When translating a set of MSCs into a global finite state machine (GFSM), one needs to translate
each MSCs into separate behavior automata by assigning monotonically increasing integers to
represent the progress. The detailed translating algorithm is referred in [6]. We encoded the
values of configuration variables in states. Brute-force production of automata would quickly
result in state explosion. That is, each state corresponds to a distinct state, and changes made
to the system configurations (as a result of message exchanges and updates made to the system
variables) are shown as transitions. For reactive and embedded systems, the initial state of the
8
8/6/2019 Automated Embedded Software Testing Using Task Interaction Scenarios
9/18
Figure 6: HTML execution in INSTALLED state
Figure 7: HTML execution in PAUSED state
9
8/6/2019 Automated Embedded Software Testing Using Task Interaction Scenarios
10/18
system is often known and fixed in advance. Therefore, it makes sense to generate a GFSM
containing only the feasible states by conducting forward search starting from the initial state.
Our algorithm constructed a GFSM (Figure 8) which contains 8 states and 22 transitions for the
example.
s0 t2
s1
t0
s2
t1
t2
t8
s6
t5
t3
t2
s3
t0
t8 s7
t6 t4
t2
s4
t5
t4
s5
t6
t4
t7
t2
t8
t7
t2
Figure 8: GFSM for HTML applications
It should be noted that as the DTV software makes a transition from the state s0 to s1, a
sequence of message exchanges is required and monitored. Transitions are specified in two parts
so that test cases for native and Java test driver can be easily separated. As a result, Java
Transition part contains interactions between Java threads of bMSC, but Native Transition part
contains all the other interactions including interactions between native tasks and Java threads.
Each interaction consists of sending tasks (or thread), receiving tasks (or thread), message name,and parameters.
s0: (T_id = 0, currentApp = NONE, dst_version = 0,
getState() = UNINSTALLED, length = 0, minor_channel = 1)
t0:
Java Transition:
AM ALAgent appInfoSent appCommand SDF_REQUEST
ALAgent AM alEventSent id SDF_READY
AM AppInfoViewer stateChange state INSTALLED
10
8/6/2019 Automated Embedded Software Testing Using Task Interaction Scenarios
11/18
Native Transition:
DBTK SDFT q_send msg1 DST_ARRIVED
SDFT DBAgent q_send msg1 DST_ARRIVED
ALAgent BROWSER SendEventTask para1 SDF_REQUEST
SDFT DBAgent q_send msg1 DDL_INFO_ARRIVED
BROWSER AL SendEventTask para2 SDF_READY
AL ALAgent q_send msg1 SDF_READY
s1: (T_id = 1, currentApp = NONE, dst_version = 1,
getState() = INSTALLED, length = 1, minor_channel = 1)
The generation of test sequences from finite state machines is a well understood problem,
and various techniques exist to generate sequences[7]. Since each state of GFSM is encoded by
considering the value of state variables of the target system, state-tour and transition-tour1[8]
coverage criteria are sufficient methods for generating test sequences. The former requires that
each state in the GFSM is visited at least once, and the latter requires that each transition is
visited at least once. For event-oriented specifications, the state-tour coverage criteria means that
all external input events of the system must be covered at least once, and the transition-tour
coverage criteria means that all combination of the activation conditions in the specification must
be covered.
One test sequence, s0t0 s1
t5 s6t6 s7
t8 s2t0 s3
t5 s4t6 s5
t4 s0, is generated to satisfy
the state-tour coverage criteria. For transition-tour coverage criteria, seven test sequences are
generated as follows.
1. s0t0 s1
t2 s1t5 s6
t6 s7t2 s7
t7 s6t8 s2
t0 s3t2 s3
t5 s4t6 s5
t2 s5t4 s0
2. s0t1 s2
t3 s0
3. s0t0 s1
t8 s2t3 s0
4. s0t0 s1
t5 s6t6 s7
t8 s2t3 s0
5. s0t0 s1
t5 s6t6 s7
t8 s2t0 s3
t4 s0
6. s0t0 s1
t5 s6t6 s7
t8 s2t0 s3
t5 s4t4 s0
7. s0t0 s1
t5 s6t6 s7
t8 s2t0 s3
t5 s4t6 s5
t7 s4t4 s0
It must be noted that care was taken in our algorithm to generate test cases so that test
sequences always terminate in the initial state. That is, test sequences merely satisfying the
definition of state- and transition-tour coverage criteria could be shorter if we allow the sequence
to terminate as soon as each state or transition is covered at least once. However, when executing
1A tour means a sequence from/to an initial state, and if the tour reaches the initial state, it is completed.
11
8/6/2019 Automated Embedded Software Testing Using Task Interaction Scenarios
12/18
such test sequences to test embedded software, the system would terminate in a state which is
different from the initial state. One must somehow reset the system state to the initial state
in order to execute the next test sequence. In order to accomplish automated and continuous
execution of test sequences, we have modified the algorithm so that the test sequences always
include sequences that would restore the system state into the initial state.
The generated test cases are realized and split into two test scripts for native and Java test
drivers. Following test script is the native part for the transition s0t0 s1. The condition and
assert scripts mean that the test drivers check the source and destination states of each transition
of the GFSM before and after the transition is tested, respectively. While the monitor script
is used for checking whether the internal interactions are properly occurred, the message script
is used for generating external input events. Finally, the data script is used for selecting a
pre-generated stream data suitable for test cases.
1: condition SDFT DSTversion = 0
2: message DBTK SDFT q_send msg1 DST_ARRIVED
3: monitor SDFT DBAgent q_send msg1 DST_ARRIVED
4: monitor ALAgent BROWSER SendEventTask para1 SDF_REQUEST
5: monitor SDFT DBAgent q_send msg1 DDL_INFO_ARRIVED
6: monitor BROWSER AL SendEventTask para2 SDF_READY
7: monitor AL ALAgent q_send msg1 SDF_READY
8: assert SDFT DSTversion = 1
3.3 Test Harness
An automated task interaction testing environment is developed since economics and development
efficiency provide good reasons to conduct as large a proportion of the software life-cycle in the
host system as possible. Appropriate test scripts are automatically generated in either Java or
C from the GFSM, but the test driver and message monitoring software are manually developed.Figure 9 shows how the native test driver operates.
For testing of sequential programs, it is sufficient to supply inputs and monitor outputs to
determine if a test run was successful or not. For concurrent systems, however, we need observe
the order of task execution and communication. In addition, we need to instrument source code
to insert probes. There are basically three different types of probes, i.e. kernel-probes, inline-
probes, and probe-tasks, depending on where they are implemented. Kernel-probes monitor task
switches, interrupt interference, etc. These types of probes are typically not programmable by the
12
8/6/2019 Automated Embedded Software Testing Using Task Interaction Scenarios
13/18
Figure 9: Native test driver
application designer, but rather given as an infrastructure by the real-time kernel. Inline-probes
are task level probes that add auxiliary outputs to the task they instrument. Probe-tasks are tasks
dedicated to collecting data from kernel-probes, inline-probes and other probe-tasks. A dedicatedprobe-task receives data from a set of tasks. All probe-tasks must be taken into account when
designing and scheduling the system. In our work, a probe-task is implemented in the native
test driver, kernel-probes are inserted in two event functions, q send and SendEventTask, for
intercepting the message interactions, and inline-probes are inserted in the message receiving
methods of Java threads.
The script parser reads the scripts line by line, and if the condition or assert script is met, the
system state checker compares current system state (source state) or resulting state (destination
state), respectively. For the message script the input event generator generates an event which
simulates one of the external input events. When the test driver meets the monitor script, the
monitored event checker operates whether desired interactions are occurred in specified order. The
Java test driver acts almost the same as the native test driver except that it does not generate
environments inputs. For the data script we use the pre-generated stream data suitable for each
test case. For example, we prepare a special stream data for test cases of 4 which require an
HTML application for each channel, and this is used when executing the test cases. The SEC
13
8/6/2019 Automated Embedded Software Testing Using Task Interaction Scenarios
14/18
DTV software was developed and tested using a simulator running on a PC where commands
entered by a user using the remote control unit are simulated as keyboard or mouse inputs.
4 Discussion
ATSC provides conformance test suites for system testing and for DASE APIs, and several com-
mercial testing tools, such as McCabe Test[9], etc., are available for unit testing. However, no
tools exist to automatically generate test cases to validate task interactions of embedded software.
In our work, we newly specify a substantial portion of test interactions from the design documents
(about 500 pages) for each subsystem written in natural language, from developers knowledge us-
ing individual and team interviews (about 24 hours), and from the implementation (about 800,000
lines of code written C and Java).
The generated test cases were executed in our environment, and test results were analyzed by
developers. While the test cases generated by the state-tour coverage criteria failed to uncover
flaws in the DTV software, the test cases based on the transition-tour coverage criteria detected
an implementation error that was previously unknown. It involved a deadlock situation when two
consecutive requests to change the channels are received during processing the first request. In
the DTV software, if TV channel is changed, data information for new channel is requested to
the transport stream parser. If the DTV software accepts new channel change event before the
information parsing completed for the previous request, the software must cancel the previous
one. However, since the developer of the transport stream parser subsystem did not implement
this requirement, the deadlock situation is occurred.
Since all test cases starts with the initial state of the system, a reset function is required in
order to restore the systems state for consecutively executing the next test case. If the GFSM,
however, forms a connected component, the reset function is not necessary because all test cases
generated by state- and transition- tour coverage criteria end with the initial state. In the pro-
posed approach, the GFSM is connected only if scenarios contain all complementary events. For
example, since the complementary event sets of Figure 4 are {TV Video, EXIT}, {Channel Up,
Channel Down}, and {(DST Arrived, Channel Up), (DST Arrived, Channel Down)}, the GFSM
in Figure 8 is connected. As a result, if the input specification contains all events in the com-
plementary event sets, the generated test cases are automatically executable without any user
intervention. Furthermore, the complementary events sets are also used as an unit of incremental
testing.
14
8/6/2019 Automated Embedded Software Testing Using Task Interaction Scenarios
15/18
We have applied the proposed approach to a larger subsystem2 (shown in Figure 10) of DTV
software which consists of 10 remote control unit events out of 25 and 5 stream data inputs out of
42. The required interaction sequences are described in 84 bMSCs. When the GFSM is developed
as explained in this paper, 248 states and 1952 transitions were created. There were 17 and 875 test
cases needed to satisfy the state- and transition-tour coverage criteria, respectively. The number
of states and transitions of the GFSM did not grow rapidly when the number of external input
events increased from 5 to 15. Since many input events such as the entering channel numbers
0 9 are processed only with one or two message exchanges, we described a major portion of
interactions of the SEC DTV software. Consequently, our approach is scalable enough to perform
integration testing of reasonably large and complex system.
17 bMSCs
DTV
RC_Events
DB_Event_1
DB_Event_2
DB_Event_3DB_Events
Process_HTML
Deregister_HTML
Deregister_Xlet
Start_HTML_Installed
Browser_Exit
PE_Exit
DB_Event_4
DB_Event_5
RC_Key_9
RC_Key_7
RC_Key_6
RC_Key_4
RC_Key_3
RC_Key_2
RC_Key_1
App_Exit
menu_Exit
PE_Browser_Exit
EPG_Exit
Xlet_Exit
Start_HTML_Paused
DST_Change
Process_DST
Process_Xlet
DDL_Arrived
RC_Key_5
RC_Key_8
RC_Key_10
13 bMSCs
6 bMSCs
12 bMSCs
2 bMSCs
Figure 10: Specification Structure of the SEC DTV Software
5 Related Works
Several structural testing approaches are proposed to test the task integration of Ada[10, 11]. Since
the points of send and receive in a rendezvous of Ada are explicitly implemented in each task,
interaction information can be extracted automatically from the code. However, the subsystems
2The bMSCs included in the HTML executions are represented with italicized names. Exact names and detailedbMSCs are replaced anonymous names and the number of sub-bMSCs to protect proprietary information of SamsungElectronics Co.
15
8/6/2019 Automated Embedded Software Testing Using Task Interaction Scenarios
16/18
of embedded software written C or Java are often implemented by tasks and threads and the
interactions are mainly performed by message queues. In the message queue-based interactions,
the interaction information cannot be automatically extracted from the code, since the behavior
of tasks or threads is performed as follows: in each execution period of the task or thread, periodic
functions are processed after it checks its own message queue first, and the receiving messages are
handled or messages are sent to other tasks. Consequently, specification-based integration testing
is required.
The idea of describing requirements in a formal notation such as Statecharts or MSCs and
translating it into a finite state machine (FSM) to conduct behavioral analysis is certainly not
new. For example, SDL[12] or Estelle[13] adopt approaches where the required behavior of each
task is expressed in separate finite state machines. Products of individual FSMs capture the
global behavior of the system. In Statecharts-based approach[14], behavior of each task is usually
described using AND state, and unfolding (or combining) of them represent the global behavior.
Unfortunately, when generating test cases to detect flaws in task interactions, it is necessary
to change the Statecharts so that only the transitions related to the task interactions are kept.
Although technically feasible, these approaches have a couple of drawbacks: (1) If the system
consists of a large number of tasks, the number of states and transitions generated grows too
quickly to make the approach impractical; and (2) In Statecharts-based approach, identifying onlythe states and transitions related to task interactions may be difficult and potentially error-prone.
In [15, 16, 17], MSCs is used to describe test purposes in order to select some particular parts
to be tested. There were two different approaches to generate test cases from MSCs. Normally,
a state space exploration is started which simulates both a given MSCs and the SDL system. In
this case, alternative receive events which violate the MSCs but are valid according to the SDL
specification are added to the test case representation with a TTCN inconclusive verdict. If a
state space exploration is not applicable, MSCs have to be translated directly into test cases. In
[18], MSCs is used to validate whether or not the test cases are feasible. Once the SDL models
are developed, they are verified using the SDL simulator. The simulator can produce an MSCs
that records the simulation execution. They check that the resulting MSCs are consistent with
the original requirements MSCs.
16
8/6/2019 Automated Embedded Software Testing Using Task Interaction Scenarios
17/18
6 Conclusion
This paper demonstrated that Message Sequence Charts formalism is useful when describing the
interaction behavior for large and complex embedded software, and a set of MSCs is automatically
translated into a global finite state machine (GFSM). Once a GFSM is constructed, test cases are
generated from the GFSM according to the state or transition tour coverage criteria. We have
applied the proposed approach to specify a substantial portion of complete requirements for an
embedded software running on a digital TV, and reported some lessons learnt from the case study.
Embedded software typically consists of a set of tasks which communicate and synchronize with
each other using shared variables or message exchanges. Since the behavior description of shared
variables is not supported in the current version of MSCs, only message passing is used in our
approach. Since the tasks and threads of our case study are operated in event-driven manner, our
approach is applicable to the event-driven system well. Therefore, the proposed technique could
be further extended for shared variables and time-driven systems. The extension must include
notations of bMSCs, new algorithm for generating GFSM, and execution environment.
References
[1] David Harel. Statecharts: A Visual Formalism for Complex Systems. Sciences of Computer
Programming, 8:231274, 1987.
[2] ITU-T Recommendation Z.100. Specification and Description Language (SDL), 1999.
[3] ITU-T Recommendation Z.120. Message Sequence Chart (MSC), 1999.
[4] ATSC Document CS/100. DTV Application Software Environment Level 1 (DASE-1), 2001.
[5] E. Rudolph, J. Grabowski, and P. Graubmann. Tutorial on Message Sequence Charts
(MSC96). In FORTE/PSTV96, 1996.
[6] N.H. Lee and S.D. Cha. Generating Test Sequences from a set of MSCs. Computer Networks
(to be published), 2003.
[7] D. Lee and M. Yannakakis. Principles and Methods of Testing Finite State MachinesA
Survey. Proceedings of the IEEE, 84(8):10901123, 1996.
[8] S. Naito and M. Tsunoyama. Fault Detection for Sequential Machines by Transition-Tours.
In FTCS (Fault Tolerant Computing Systems), pages 238243, 1981.
17
8/6/2019 Automated Embedded Software Testing Using Task Interaction Scenarios
18/18
[9] McCabe & Associates Inc. McCabe Test, http://www.mccabe.com/mccabe test.php.
[10] R.N. Taylor, D.L. Levine, and C.D. Kelly. Structural Testing of Concurrent Programs. IEEE
Transactions on Software Engineering, 3:206215, 1992.
[11] T. Katayama, E. Itoh, Z. Furukawa, and K. Ushijima. Test-case Generation for Concurrent
Programs with the Testing Criteria Using Interaction Sequences. In Asia-Pacific Software
Engineering Conference (APSEC99), 1999.
[12] L. Bromstrup and D. Hogrefe. TESDL: Experience with Generating Test Cases from SDL
Specifications. In Fourth Proc. SDL Forum, pages 267279, 1989.
[13] O. Henniger and P. Neumann. Test Case Generation based on Formal Specifications in Estelle.
In Proc. of IEEE International Workshop on Factory Communication Systems, 1995.
[14] Y.G. Kim, H.S. Hong, D.H. Bae, and S.D. Cha. Test Cases Generation from UML State
Diagrams. IEE Proceedings - Software, 146:187193, 1999.
[15] J. Grabowski, D. Hogrefe, and R. Nahm. Test Case Generation with Test Purpose Specifica-
tion by MSCs. In 6th SDL Forum, pages 253266, 1993.
[16] J. Grabowski. SDL and MSC Based Test Case Generation - An Overall View of the SAMSTAG
Method. Technical report, University of Berne, IAM-94-0005, 1994.
[17] M. Schmitt, A. Ek, B. Koch, J. Grabowski, and D. Hogrefe. Autolink Putting SDL-based
Test Generation into Practice. In In A. Petrenko et al., editor, IWTCS98, pages 227243.
Kluwer Academic Publishers, 1998.
[18] R.L. Probert, H. Ural, and A.W. Williams. Rapid Generation of Functional Tests using
MSCs, SDL and TTCN. Computer Communications, 24(3-4):374393, 2001.