Automated Scenario Test

8/14/2019 Automated Scenario Test

1/27

Purpose

This document provides guidance and information that is necessary to be able to effectively

implementing the automations of the test-cases.

Audience

The audience of this documentation is the software SDETs1 responsible for the implementation of the

automations of the test-cases.

Implementing the automations involves translating the test-cases into XML documents where the steps

in the test case are spelled out, and implementing custom actions (in the java programming language)

that actually carry out the tasks to in each step. Therefore, knowledge of XML and programming injava are required on the side of the SDET.

Scope

Each XML mentioned above captures a test-case (the intention and the steps) in a program-readableformat. Throughout this documentation we will refer to that XML as scenario xml. This documentation

explains all the elements and attributes expected / allowed in the scenario XML.

1 Software Development Engineer in Test


2/27

Scenario XML

The scenario XML will be explained using several examples, starting with the one below.

eq(mediator("isStarted()"), true)

"lee"

step 1 desc"fu"

action 1.1 desclist("a","b")range(7,8,1)9

action 1.2 desc

{rand = new Random(action.getArgument("randSeed"));}Integer theNextInt = new Integer(rand.nextInt());

action.getStep().setValue("theNextInt", theNextInt);System.out.println(Thread.currentThread() + ": Done! : " + theNextInt);return null;]]>list(10000, null)

2action 2.1 desc

list(3, true, "test", null)switch(ref(x), {true:concat("It's true! Look:", ref(x)),

"zeppelin":"Wow that's a monster.", any:"Well..."})

Insert 1. Scenario XML #1


3/27

First pass through the scenario XML

Thenode is the root of a scenario XML. It has an optional attribute repetition, whose

value can be any integer equals to or greater than 1 (by default it is set to 1). It is used to specify how

many times the scenario will be ran, sequentially. If we set it to 0, the scenario will be repeated overand over, until we kill the process.

Another optional attribute ofismax-thread, that specifies the maximum number of threads

the scenario runner can use to run the actions. The number specified here is not a hard limit; there can

be a little more threads than the number specified. As a general guideline, in order to avoid resorting

into sequential execution of actions, pick the highest multiplicity in the scenario, add a constant to it,and assign the result to themax-threadattribute. In the above example it's 4 (the multiplicity ofstep 2),

and the constant used is 1 (becausestep 2 is one level below the scenario). Therefore the chosen value

formax-threadis 4 + 1 = 5.

Inside thenode we can put anode, in which we write the (high-level) description

of the scenario. We must enclose the description in a CDATA section if the description contains special

characters which otherwise would render the scenario XML unprocessable.

A scenario, as well as step and action, is a configurable node; it can take arguments. Those arguments

are contained in thenode . Each of them is represented by annode , whose value can

be a string literal, integer literal, bigdecimal literal (e.g.: 1.234), boolean literal (true / false), null, XMLrepresentation of a java object, XML tree, or function.

The following table contains shows some examples of how to define an argument.


4/27

Type Example Value received in the java program(action class)

String literal foobar java.lang.String foobar

Int literal 78 java.lang.Integer 78

Boolean literal false java.lang.Boolean false

BigDecimal literal 7.5 java.math.BigDecimal 7.5

Null2 null An instance of com.baktun.bstrd.director.Null

Any any An instance of com.baktun.bstrd.director.Any

XML tree

Some beans

An instance of org.w3c.dom.Node

XML representation ofJava object

Some beans

An instance ofcom.eshop.PurchaseOrder3

Function4 mult(3,2) An instance of BigDecimal 6

Table 1. Examples of argument

The arguments defined in a configurable node are visible to the child configurable-nodes. The usual

scoping rules apply (i.e.: an argument defined in a configurable will override the argument with thesame key defined in the parent node(s)).

2 Assigning nullto an argument is not the same as not defining the argument.

3 The class must contain special (JAXB) annotations that would allow automatic mapping to XML .

4 All the available functions will be explained in separate section.


5/27

A scenario is composed of one or more steps, which will be executed one after another. Those steps

must be contained within thenode . Each of them is represented by anode that

has the following attributes:

name (mandatory): the name of the step. Each step within the same scope5 must have a unique

name.

offset (mandatory): the duration, in seconds, for the execution of the step to be delayed. The

value must be an integer equal to or greater than 1. mult (mandatory): the number of instances of the step to be created and executed. The value

must be an integer equal to or greater than 1.

concurrent (optional): specifies whether the instances of the step will be executed sequentially

or concurrently. The valid value is true (for concurrent) orfalse (for sequential). The defaultvalue isfalse.

Inside the a step we define the actions that belong to the step. Those actions must be contained within

thenode. Each of them is represented by annode that has the following

attributes:

name (mandatory): the name that uniquely identifies the action within the step.

class (optional): the name of the java class that actually performs the task intended by theaction. If we don't specify this, we must write the code directly inside the script node , as

exemplified by action 1.2.

asynch (optional): specifies whether the action is to be executed in the same thread as the

containing step or in separate thread.

Running a scenario

To see a scenario XML in action, a separate program has to be written. This program, which we will

refer to as directorthroughout this documentation, in general performs the following tasks:

Load the scenario XML (into a java object of type ScenarioDefinition).

Create an instance ofmediator.

Create an instance ofScenarioRunner.

Start the scenario runner by calling its run(...) method6, feeding in the scenario definition

created in step #1.

Start the mediator.

The mediator is a generic java object; it can be anything. It is the entity we want to control, manipulate,

or test throughout the execution of scenario, by means of actions. The following diagram depicts the

relationship between scenario, mediator, and action.

5 The meaning of it will be clarified in the section where action-nesting is explained.

6 It has to run in a thread different from the one that runs the mediator.


6/27

Figure 1. The relationship between scenario, mediator, and actions.

The following insert contains an example of such director. In this example, the mediator is an instanceofcom.baktun.bstrd.unittest.Frijol7.

We have something quite similar for the spectel bridge simulator; the director is a java applicationnamed com.baktun.bridgesimulator.BridgeDirector, and the mediator is an instance of

com.baktun.bridgesimulator.Bridge, wrapped inside an instance of

com.baktun.bridgesimulator.BridgeServer.

package com.baktun.bstrd.unittest;

import java.io.BufferedReader;import java.io.FileReader;import org.apache.log4j.Logger;import com.baktun.bstrd.director.ScenarioDefinition;import com.baktun.bstrd.director.ScenarioRunner;

publicclass FrijolDirector {privatestaticfinal Logger logger = Logger.getLogger(FrijolDirector.class);

private FrijolDirector() {}

publicstaticvoidmain(String[] args) throws Exception { if (args == null || !(args.length == 2) ) {

System.out.println("Usage: java FrijolDirector [keep-frijol-up]");System.exit(-1);

}

boolean keepFrijolUp = true; if (args.length == 2) {

keepFrijolUp = Boolean.parseBoolean(args[1]);}

String scenarioDefXml = null;{BufferedReader br = new BufferedReader(new FileReader(args[0]));StringBuffer sb = new StringBuffer();

try {String line = br.readLine();

while (line != null) {sb.append(line).append("\n");line = br.readLine();

7 Spanish word for bean.


7/27

}} finally {

if (br != null) {br.close();

}}

scenarioDefXml = sb.toString();}

final ScenarioDefinition scenarioDef = ScenarioRunner

.loadScenarioDef(scenarioDefXml);

final Frijol frijol = new Frijol();

final ScenarioRunner scenarioRunner = new ScenarioRunner();

Thread scenarioRunningThread = new Thread(new Runnable() { publicvoidrun() { try {

scenarioRunner.run(scenarioDef, frijol);} catch (Exception e) {e.printStackTrace();

}}

});

scenarioRunningThread.setPriority(Thread.MIN_PRIORITY);

scenarioRunningThread.start();

frijol.run(); }

}

Insert 2. An example director

The output of the execution of the director above, in the following insert, will help clarifying a coupleof important points about the execution model of the scenario runner.

2009-09-23 15:42:24,564 [INFO] Thread-2 com.baktun.bstrd.unittest.Frijol - Frijol is goingto sleep for 5 seconds....

2009-09-23 15:42:24,583 [INFO] Thread-1 com.baktun.bstrd.director.ScenarioRunner -Running scenario for the 1 times2009-09-23 15:42:30,640 [INFO] Thread-1 com.baktun.bstrd.director.ScenarioRunner -About toexecute step step 1

Thread[pool-1-thread-2 ,5,main]:{jet=lee, kung=fu, a=b , c=9, b=8 }

Thread[pool-1-thread-1 ,5,main]:{jet=lee, kung=fu, a=a , c=9, b=7 }------2009-09-23 15:42:30,661 [INFO] Thread-1 com.baktun.bstrd.director.ScenarioRunner -About toexecute step step 2

Thread[pool-1-thread-3 ,5,main]: Done! : 1913434059

Thread[pool-1-thread-4 ,5,main]: Done! : -498702880

Thread[Thread-1 ,1,main]:{jet=lee, y=Well..., x=3}---Thread[Thread-1,1,main]:{jet=lee, y=It's true! Look:true, x=true}---Thread[Thread-1,1,main]:{jet=lee, y=Well..., x=test}---Thread[Thread-1,1,main]:{jet=lee, y=Well..., x=null}---

Insert 3. Output of the execution of example director


8/27

Threading in the scenario

Sincestep 1 is declared as concurrent, there mightbe new thread(s) allocated for the execution of the

instance(s) ofstep 1. In the above case, we also declaredstep 1 to have multiplicity 2, which might

cause two instances ofstep 1 to be created, each of them will be running in separate threads8.

In contrast, when a step is not declared as concurrent, as is the case ofstep 2, it will be executed in thesame thread as the one in which the containing node is running . In the above case, the containingnode is the scenario, which is running in the main thread.

The scenario runner will go through all the steps sequentially, in the document order. That means,step

2 will be executed only after all the instances ofstep 1 have finished running.

Action will be running in the same thread as the one in which the containing step is running , except

when it is declared as asynchronous (the case of action 1.2). Asynchronous action will be running in

yet separate thread 9.

Implementions of action are not expected to either create or manage any threads (although currentlythere's nothing that prevents it from doing so). If an action is intended to run in the background (e.g.: as

a listener), we can simply declare it as asynchronous in the scenario XML.

The declarative style of doing things here is aimed at improving the communication among the team-

members, by making the intention spelled out clearly in the scenario XML. Above all, this test

framework is a communication tool that fills the gap between the test-case document, and the

executable that realizes the test-case.

The test-framework takes common, repeated concerns (such as threading) away from us, while still

making it accessible in another way (declarations in the scenario XML). This allows us to have a lean

and compact implementation of actions, which are expected to consist of only a few lines of code thatbasically take the arguments passed-in to it, process them a little bit, and use them during the

interaction with the mediator.

8 There is no guarantee that instances of a step declared as concurrent will be running concurrently. The scenario-runnermakes its best effort to fulfill it, though, but if the maximum number of threads is already reached, the scenario-runner

will resort to executing those steps in the same thread as the one in which the parent node is executing, resulting in

sequential execution of those instances of step.

9 Care has been taken about creation of new threads. The engine utilizes Java's built-in thread-pooling mechanism. This

has consequences such as: action that uses ThreadLocal as storage will likely break.


9/27

Argument's-value resolution

Initially there was a simple mechanism: you declare the argument by simply specifying the key and the

value, what you see (in the XML) is what you get (in the action). Things changed when we introduceddynamism into the scenario XML, by means of a domain-specific expression language.

Some of the functions available in the expression language allows the value of an argument to be basedon the value of other argument, for example: concat(ref(segment_id), "-",

ref(participant_id)).

Now we can make the value of an argument to vary according to the index in the iteration:list("white", "red", "black").

We can also make use of a construct similar to the switch-case-default statement in programming

language like java: switch(ref(hobby), {"music":"give him a guitar",

"literature":"buy him a kindle", null:"give him some cash"}).

By combining that dynamic argument-value and multiplicity of steps, a whole set of combinations ofinputs for the test-case can be packed into a single scenario XML.

There are two functions that particularly deserves a couple of paragraphs (of clarification) here:

The list function, with syntax: list(element_1, element_2, ..., element_n)

The range function, with syntax: range(start_val, end_val, step)

Assigning a list to an argument in the scenario XML does not mean that an instance of

java.util.Listwill be passed-in to the action. Instead, the action will only see a single element

bound to that argument, taken from the list. Example:

list("green", "orange")

list(1980, 1990)

Insert 3. An example scenario definition with list(...)

In the example presented above, there will be two threads. The value of the arguments accessible from

the action running in the first thread: {var_a: green, var_b: 1980} . The second thread, on the other

hand, will see the following values: {var_a: orange, var_b: 1990}.


10/27

The function range behaves in similar fashion. In fact, we can see a range as just anotherlist, only

that we don't have to specify all the elements manually. That is to say:

range(10, 15, 1) is equivalent to list(10, 11, 12, 13, 14, 15)

range(10, 15, 2)10is equivalent to list(10, 12, 14)

The number of elements in the list must be at least equal to multiplicity of the step to which it directly

belongs. In the above example, list var_a and var_b both belong to the stepstep 1 whose multiplicityis set to 2. Therefore the length of those list be at least equal to 2. Extra elements at the end will simply

be ignored / unused.

The following insert contains another example that uses list, this time involves nesting of actions. In

that case, the length of list var_c and var_d must be at least 1, which is the multiplicity ofstep 2 to

which those lists directly belong.

list("green", "orange")

list(1980, 1990)

list(9)

list("bar")

Insert 4. Another example scenario definition with list

10 Probably mod-check is necessary to eliminate misunderstanding / mis-expectations.


11/27


12/27

mult(1.6, 2), for example).

switch: takes the formswitch(selector, map_of_cases), returns the value of the key-value pair

whose key matches the value of the selector. Examples:

switch(ref(x), {"boy": "Rama", "girl": "Shinta", any:

ref(generic_name)}) returns:

the string "Rama" ifx resolves to "boy".

the string "Shinta" ifx resolves to "girl". the value the variable generic_name resolves to ifx resolves to any value other than

"boy" or "girl".

neg: accepts either integer / bigdecimal / boolean, and returns the negated value of its input.Examples:

neg(1) returns Integer(-1)

neg(-2.5) returns BigDecimal(2.5)

neg(ref(x)) returns Boolean(true), assuming x resolves to false.

eq: compares its two inputs, returns true if they're equal. Except for some special cases, the

equality is based on the value of the two inputs (i.e.: the result of calling equals method on

one of the input, passing in the other input as the argument). Examples: eq(1, 1) returns Boolean(true).

eq(1, 1) returns Boolean(false).

eq(1, ref(x)) returns Boolean(true), assuming x resolves to Integer(1).

eq(ref(x), any) returns Boolean(true), assuming x resolves to any non-null value.

eq(mediator(getWatcher()), null) returns Boolean(true), assuming the

invocation of method getWatcher() on the mediator returns null.

eq(mediator(getWatcher()), any) returns Boolean(true), assuming the

invocation of method getWatcher() returns any non-null value.

gt: compares its two inputs, returns Boolean(true) if the first input is greater than the second

input. Will return error if any of the input is neither an instance ofInteger norBigDecimal.Examples:

gt(2, 1) returns Boolean(true).

gt(2.1, 1) returns Boolean(true).

gt(1, 2) returns Boolean(false).

gt(ref(x), 2) returns Boolean(true), assuming x resolves to any Integer or

BigDecimal greater than 2.

gt(2, 1) throws an error.

lt: compares its two inputs, returns Boolean(true) if the first input is smaller than the second

input. Will return error if any of the input is neither an instance ofInteger norBigDecimal.

Examples: lt(1, 2) returns Boolean(true).

lt(1, 2.1) returns Boolean(true).

lt(2, 1) returns Boolean(false).

lt(2, ref(x)) returns Boolean(true), assuming x resolves to any Integer or

BigDecimal greater than 2.

gt(1, 2) throws an error.


13/27

and: performs logical AND operation on all11 of its inputs. All of the inputs must return an

instance ofBoolean. Examples:

and(true, true) returns Boolean(true).

and(true, false) returns Boolean(false).

and(true, true, ref(x)) returns Boolean(false), assuming x resolves toBoolean(false).

and(mediator(isStarted()), mediator(isOkToTest())) returnsBoolean(true) assuming the invocation of both isStarted() and isOkToTest()

method on the mediator returns true.

and(true, mediator(getNumberOfLines())) throws an error, assuming the

invocation ofgetNumberOfLines()method on the mediator returns anything other than

Boolean (object / primitive).

or: similar to the and function, only that the logical operation performed on the inputs is OR.

rand: takes two inputs, both of them must resolve to integers; will generate a random integer

between the range specified by those two inputs. Examples:

rand(3, 7) can return any of the following values: 3, 4, 5, 6, or 7.

rand(2, ref(x)) can return either 2, 3, 4 assuming x resolves to 4. mediator: takes the OGNL12expression that will be applied on the mediator. The simplest way

to see it: it allows us to specify the chain of method invocations (that starts on the mediator).

This function returns the value returned from the invocation(s). Example:

mediator(isStarted()), invokes the isStarted() method on the mediator, returns

the value returned from that invocation.

mediator(getParticipant(#participant_id)), invokes the

getParticipant(...)method on the mediator, passing in the value of the argument

whose key is participant_id. This requires an argument to be defined in the same node

(action / step / scenario) or above as the node that in which the mediator function is used.

Example:

mediator(getParticipant(#participant_id))part_2009_xx_00021

mediator(getParticipant(#participant_id).getName()), invokes the

getName() method on the object returned by the invocation ofgetParticipant(...)

method on the mediator.

mediator(launchRocket()) throws an error if the method launchRocket() is

neither defined nor accessible in the mediator object.

context: takes the form context(path, context_var_key) . Thepath specifies the node in thescenario where the object stored under the key context_var_key13 is to be found. The syntax of

the path is as follows: . (single dot): current node.

... (triple dot): root node (scenario)

.. (double dot): parent

./action_1

11 There is no short-circuiting here; each and every arguments will be evaluated.

12 Object-Graph Navigation Language: http://en.wikipedia.org/wiki/OGNL

13 Storing object in the scenario-execution context will be explained in separate section.
http://en.wikipedia.org/wiki/OGNLhttp://en.wikipedia.org/wiki/OGNL


14/27

../..

../action_1

../../../action_1

error: it is meant to be used only inside thenode, to specify the error that is

expected to occur inside an execution-node. This function has two possible syntaxes:

with one argument: error(exception_class_name), which will return true if any

instance of error of the specified type occured. Examples: error(java.lang.Exception)

error(java.lang.IOException)

with two arguments: error(exception_class_name, ognl_expression), which

goes further than the first version, by doing a chain of method invocation starting at the

instance of exception (using the specified ognl_expression). This functions returns the valuereturned from the method invocation(s). Examples:

error(java.io.FileNotFoundException, getMessage()); returns the

message of the instance of the FileNotFoundException.

This version oferror function is meant to be used together with the eq function. Example:

eq(c:\\theimportantfile.txt can not be found,error(java.io.FileNotFoundException, getMessage()))

stepidx: this function returns the index of the instance of the step. It takes as input the path to

the execution-node where the calculation of the index is based on14. For example, when we have

a step declaring the following, the second instance of the step will see the integer 2 assigned tothe argument the_index.

...step 1 descstepidx(".")

...

...

Caution about the context_path that is used as input for this function: it must resolve to a

node that is a step. Otherwise an error will be thrown. Example:

...step 1 desc

stepidx(".")...

...

...

...

14 See the explanation of function context for some examples of path.


15/27

The following definition of argument the_index, however, will not throw an error. Instead,

the action that is part of the second instance of the step will see the integer 2 assigned to the

argument the_index.

...step 1 desc

stepidx("..")

...

...

...

...

stepmult: just like the stepidx function, it takes as input the path to the execution-node where

the calculation of the multiplicity is based on. The same restriction as in the stepidx applies;the path must resolve to a step. It returns an integer, the multiplicity of the step. Examples:

...step 1 descstepmult(".")

...

...

...step 1 descstepmult(".")...

...

...

...


16/27

...step 1 descstepmult("..")

...

...

...

...

As you can see from the examples above, we can pass a function as argument of another function.

This capability allows to build plethora of interesting combinations. The following grammar gives acomplete picture of the expression language of this test framework.


17/27

stat: expr;expr: STR_LITERAL | INT_LITERAL | BIGDEC_LITERAL | NULL_EXPR | ANY_EXPR | func | bool_expr| mediator_expr | context_expr | rand_func | stepidx_func | stepmult_func | str_func;

bool_expr: BOOL_LITERAL | eq_expr | gt_expr | lt_expr | and_expr | or_expr | error_expr;eq_expr: 'eq' '(' expr ',' expr ')' -> ^('eq' expr expr);gt_expr: 'gt' '(' expr ',' expr ')' -> ^('gt' expr expr);lt_expr: 'lt' '(' expr ',' expr ')' -> ^('lt' expr expr);and_expr: 'and' '(' expr ',' expr (',' expr)* ')' -> ^('and' expr expr+);or_expr: 'or' '(' expr ',' expr (',' expr)* ')' -> ^('or' expr expr+);

mediator_expr: 'mediator' '(' STR_LITERAL ')' -> ^('mediator' STR_LITERAL);context_expr: 'context' '(' STR_LITERAL ',' STR_LITERAL ')' -> ^('context' STR_LITERALSTR_LITERAL);error_expr: 'error' '(' STR_LITERAL ')' -> ^('error' STR_LITERAL) | 'error' '(' STR_LITERAL',' STR_LITERAL ')' -> ^('error' STR_LITERAL STR_LITERAL);func: ref_func | list_func | range_func | concat_func | mult_func | sum_func | neg_func |mod_func | switch_func | div_func | int_func | wheel_func | short_func;str_func: 'str' '(' expr ')' -> ^('str' expr);range_func: 'range' '(' arith_func_param ',' arith_func_param ',' INT_LITERAL ')' ->^('range' arith_func_param arith_func_param INT_LITERAL);ref_func: 'ref' '(' VAR_NAME ')' -> ^('ref' VAR_NAME);

mult_func: 'mult' '(' arith_func_param ',' arith_func_param ')' -> ^('mult'arith_func_param arith_func_param);sum_func: 'sum' '(' arith_func_param (',' arith_func_param)+ ')' -> ^('sum'arith_func_param arith_func_param+);div_func: 'div' '(' arith_func_param ',' arith_func_param ',' INT_LITERAL ')' -> ^('div'arith_func_param arith_func_param INT_LITERAL);

mod_func: 'mod' '(' arith_func_param ',' arith_func_param ')' -> ^('mod' arith_func_paramarith_func_param);neg_func: 'neg' '(' neg_func_param ')' -> ^('neg' neg_func_param);neg_func_param: arith_func_param | bool_expr;int_func: 'int' '(' short_int_func_param ')' -> ^('int' short_int_func_param);short_func: 'short' '(' short_int_func_param ')' -> ^('short' short_int_func_param);short_int_func_param: arith_func_param | bool_expr;rand_func: 'rand' '(' arith_func_param ',' arith_func_param ')' -> ^('rand'arith_func_param arith_func_param);stepidx_func: 'stepidx' '(' STR_LITERAL ')' -> ^('stepidx' STR_LITERAL);stepmult_func: 'stepmult' '(' STR_LITERAL ')' -> ^('stepmult' STR_LITERAL);arith_func_param: INT_LITERAL | BIGDEC_LITERAL | ref_func | mult_func | sum_func | neg_func| list_func | range_func | div_func | int_func | mod_func | switch_func | wheel_func |rand_func | stepidx_func | stepmult_func;list_func: 'list' '(' map_entry_key_value (',' map_entry_key_value)+ ')' -> ^('list'map_entry_key_value map_entry_key_value+);wheel_func: 'wheel' '(' map_entry_key_value (',' map_entry_key_value)+ ')' -> ^('wheel'map_entry_key_value map_entry_key_value+);switch_func: 'switch' '(' switch_func_selector ',' '{' switch_func_case (','switch_func_case)* '}' ')' -> ^('switch' switch_func_case+ switch_func_selector);switch_func_selector: arith_func_param | concat_func;switch_func_case: map_entry_key_value ':' map_entry_key_value -> ^(SWITCH_CASEmap_entry_key_value map_entry_key_value);

map_entry_key_value: STR_LITERAL | NULL_EXPR | bool_expr | ANY_EXPR | concat_func |arith_func_param;concat_func: 'concat' '(' expr (',' expr)+ ')' -> ^('concat' expr expr+);STR_LITERAL: '"' ( options {greedy=false;} : . )* '"';INT_LITERAL: '-'? ('0'..'9')+;BIGDEC_LITERAL: '-'? ('0'..'9')+ '.' ('0'..'9')+;BOOL_LITERAL: 'false' | 'true';

NULL_EXPR: 'null';ANY_EXPR: 'any';VAR_NAME: ('A'..'Z' | 'a'..'z' | '_' | '$') ('A'..'Z' | 'a'..'z' | '0'..'9' | '_' | '$')*;WS: (' ' | '\t' | '\n' | '\r')+ {skip();};

Insert 5. The grammar of the expression language of the test framework


18/27

Implementing action

Once the scenario is completely laid-out and received an approval, we are for the final step that is

implementing the actions that will actually carry out the tasks. There two ways of implementing action:(a) create the action class, or (b) script it directly inside the scenario XML.

Creating an action class

An action class is a (java) class that extends com.baktun.bstrd.director.Action. We

only have to implement one method: callExecute(...). The following insert contains the source

code of an example action class, PrintArgsAction that simply prints out all the arguments visible

from it.

package com.baktun.bstrd.unittest;

import java.util.Map;import com.baktun.bstrd.director.Action;import com.baktun.bstrd.director.ActionDefinition;import com.baktun.bstrd.director.Step;import com.baktun.bstrd.director.ValueProvider;

publicclass PrintArgsAction extends Action {public PrintArgsAction(ActionDefinition definition, Step step,

Map waitValPro,ValueProvider condValPro, ValueProvider expectValPro, Object mediator) {

super(definition, step, valPros, waitValPro, condValPro, expectValPro, mediator);}

public Object execute(Object resultFromPreviousAction) throws Exception {System.out.println(Thread.currentThread() + ":" + getCopyOfArguments());System.out.println("---");

returnnull;}

}

Insert 6. The source code of an example action class

The execute(...) method takes the result of the execution of theprevious action, that is the action

defined right before this action in the scenario XML (it's a way of sharing objects between actions,

beside sharing through context which will be explained in separate section). Consequently, theexecute(...) method returns an object (that will be passed on to the next action by the scenario

runner). If there's nothing to share, simply returns null.

From within the execute method, we can access the containing step, the mediator, and the arguments.

We can also store / retrieve values stored in the context. The following methods let you do theaforementioned things:

Step getStep()

T getMediator()

Object getArgument(String key)

Object getArgument(String key,boolean lookUp)

boolean hasArgument(String key)

boolean hasArgument(String key,boolean lookUp)

LinkedHashMapgetCopyOfArguments()


19/27

LinkedHashMapgetCopyOfArguments(boolean lookUp) boolean hasValue(String key)

boolean hasValue(String key,boolean lookUp)

voidsetValue(String key, Object value)

Map getCopyOfValues()

Map getCopyOfValues(boolean lookUp)

voidunsetValue(String key)

voidunsetAllValues()

ExecutionNode getParent()

V getDefinition()

One thing worth mentioned here is the behavior of the method getArgument(...) and

getValue(...): they will throw an exception if the corresponding argument can not be found (in

the scenario XML) or there's no object associated with the specified key can be found (in the context),

respectively.

This is by design to force the SDET to state its intention in the scenario XML (in the case ofgetArgument(...)); if the value of a variable is intended to be null, don't leave it out in the XML.

Instead specify its value to null, like in the example below.

null

Insert 7. Specifying null in the scenario XML

Inside the action class, we handle it the following way:

...public Object execute(Object resultFromPreviousAction) throws Exception {

...Object theVarA = null;

if (getArgument("var_a") instanceof com.baktun.bstrd.director.Null) {theVarA = calculateDefaultValueOfVarA(...)

} else {theVarA = calculateValueOfVarABasedOn(getArgument("var_a"));

}...

}...

Insert 8. Handling null in the action class

Similarly if we want to state that var_a can take any value, set it to any in the scenario XML, as in the

example below.


20/27

any

Insert 8. Specifying any in the scenario XML

Inside the action class, we handle it the following way:

...public Object execute(Object resultFromPreviousAction) throws Exception {

...Object theVarA = null;

if (getArgument("var_a") instanceof com.baktun.bstrd.director.Null) {theVarA = calculateDefaultValueOfVarA(...)

} elseif (getArgument("var_a") instance of com.baktun.bstrd.director.Any) {theVarA = calculateRandomValueOfVarA(...)

}else {theVarA = calculateValueOfVarABasedOn(getArgument("var_a"));

}...

}...

Insert 8. Handling any in the action class

To use that action class in the scenario XML, simply specify its fully-qualified name as the value of

class attribute of the appropriatenode. Example:

list("green", "orange")list(1980, 1990)

Insert 8. Using action class in the scenario XML


21/27

Scripting the action

Alternatively, when the action is so simple15, we can directly script it inside the scenario XML. In that

case, there is no need to specify a value forclass attribute16in thenode. Instead, we have to

write directly those lines of code inside thenode (and don't forget to wrap guard it inside a

CDATA section). The scripting language is still java. The following insert contains an example of

scenario XML with scripted action.

action 1.2 desc

Random rand = null;if (action.getArgument("randSeed") instanceof Null) {rand = new Random();} else

{rand = new Random(action.getArgument("randSeed"));}

Integer theNextInt = new Integer(rand.nextInt());action.getStep().setValue("theNextInt", theNextInt);

System.out.println(Thread.currentThread() + ": Done! : " + theNextInt);return null;]]>list(10000, null)

Insert 9. Example of scenario XML with scripted action

From inside the script we can use the methods available in the Action class (page 19-20). However, we

have to prepend the invocation with action..

15 No exact definition of simple provided. It can be taken as: "no more than 5 lines of code", for example.

16 In fact, we must omit the class attribute.


22/27

Passing objects through context

Other than passing the return of the execute method from one action to the next one (in the same step),

the framework allows actions to publish objects into the context, such that it can be later retrived andused by subsequent actions. The following code-snippet shows an example of how to use thesetValue and getValue methods for storing object in the context and retrieving object from the

context, respectively,

...public Object execute(Object resultFromPreviousAction)

throws ActionExecutionException {...RedBean redBean = processIt((GreenBean) getValue("theBeanThatIsShared"));setValue("theBeanToBeShared", redBean);...

}...

Insert 10. Using getValue and setValue

The above example, however, is not really useful because it tries to retrieve / store from the sameexecution-node; which translates to sharing object(s) only with itself, which is pretty much

meaningless. Normally we would retrieve / store from the execution node at least one level above. Thefollowing snippet shows how an example of storing an object in the step that the action belongs to, so

that subsequent actions within the same step can retrieve it. The snippet also shows an example of

retrieving an object from the step.


throws ActionExecutionException {...RedBean redBean = processIt((GreenBean) getStep().getValue("theBeanThatIsShared"));

getStep().setValue("theBeanToBeShared", redBean);

...}

...

Insert 11. Store and retrive in / from a step

Finally, the following snippet shows how to store in and retrieve from the scenario.


throws ActionExecutionException {...

RedBean redBean = processIt((GreenBean)getStep().getScenario().getValue("theBeanThatIsShared")); getStep().getScenario().setValue("theBeanToBeShared", redBean);

...}

...

Insert 12. Store and retrive in / from a scenario


23/27

Nesting actions

Some scenarios requires nesting action(s) inside other action. We achieve nesting by, first, adding a

inside the. Further, inside the . Please note that unlike scenario, action can only have

one child step. The structure of a scenario XML (with nesting) can be depicted the following way:

Scenario Steps

Step

Actions

Action

Action

Step

Actions

Action

Action

...

Action

Step

Insert 13. Tree-structure of a scenario XML

Consider the following example which is taken from the real, current, project (LaCross). In the test

scenario we needed to setup 20 conferences. Each conference in turn has to have 5 segments, and eachsegment has to be filled with 100 participants.

In order to realize it, we have a step with multiplicity 20 step Create conference where we

define the BILLING_CONFIGID for each conference that will be created . Inside that step we havean action, a special kind of action that does nothing, com.baktun.bstrd.director.TransitAction

. It is a trick that allows us to go directly from a step to a (child)step without doing anything in

between (because in this case we don't really create the instance of conference before creating thesegments. The conference is only an abstract concept that is created at the time the segment is created).

We already saw a nesting (ofstep Create segment) inside an action (action Transit to segment

creation (and population)). Further down, we nest another step step Create participants

inside the Create segment action.

Another practical thing that can be observed in the example which has been described in the

argument's value resolution section is that we can use the argument defined in the parent node from a

child node, typically to construct a more complex pattern. For example: the value of

PARTICIPANT_ID defined in the Create participant action is based on the value of

SEGMENT_ID defined in the Create segment step. Similarly, the value of PHONE_NUMdefined in the Create participant action is constructed by concatenating the value of

BILLING_CONFID defined at the scenario level and the value of SEGMENT_ID defined in theCreate segment step.


24/27

Scenario QA

eq(mediator("isStarted()"), true)

str(range(10101, 10120, 1)) concat("SpectelEmu_", ref(BILLING_CONFID))

short(sum(mult(stepidx("../.."), stepmult(".")),

stepidx("."), 1))

"OPERATOR_ASSISTED"falsenullnullnull

Map segmentMap = new HashMap();

segmentMap.put(SegmentProperty.BILLING_CONFID,

action.getArgument("BILLING_CONFID"));

segmentMap.put(SegmentProperty.CONF_NAME,action.getArgument("CONF_NAME"));segmentMap.put(SegmentProperty.CONFERENCE_ID,

action.getArgument("SEGMENT_ID"));if ((action.getArgument("IS_QA_ON") instanceof Null) == false) {segmentMap.put(SegmentProperty.IS_QA_ON,

action.getArgument("IS_QA_ON"));}if ((action.getArgument("IS_POLLING_ON") != Null) == false) {segmentMap.put(SegmentProperty.IS_POLLING_ON,

action.getArgument("IS_POLLING_ON"));}if ((action.getArgument("NRP_MODE") != Null) == false) {segmentMap.put(SegmentProperty.NRP_MODE,

Byte.valueOf(action.getArgument("NRP_MODE")));

}//System.out.println(segmentMap);mediator.getBridge().createSegment(segmentMap,

action.getArgument("NOTIFY_EVENTS"));return null;

]]>

rand(5, 10)


25/27

short(sum(mult(sum(ref(SEGMENT_ID), -1),

stepmult("..")), stepidx(".."), 1)) falsefalseconcat("Participant_", ref(CONF_NAME), "_",

ref(SEGMENT_ID), "_", sum(stepidx(".."), 1))"Baktun"concat(ref(BILLING_CONFID), "-",

ref(SEGMENT_ID), "-", sum(stepidx(".."), 1))

Map legProperties = new HashMap();

legProperties.put(LineProperty.CONFERENCE_ID,

action.getArgument("SEGMENT_ID"));legProperties.put(LineProperty.PARTICIPANT_ID,

action.getArgument("PARTICIPANT_ID"));legProperties.put(LineProperty.CALL_TYPE, "DIAL-IN");legProperties.put(LineProperty.CHAN_TYPE,

Line.LineType.NET_USER_CHANTYPE);legProperties.put(LineProperty.IS_MUTED, true);legProperties.put(LineProperty.IS_IN_QA, false);legProperties.put(LineProperty.IS_IN_POLL, false);legProperties.put(LineProperty.PHONE_NUM,

action.getArgument("PHONE_NUM"));if ((action.getArgument("IS_MODERATOR") instanceof Null) == false)

{legProperties.put(LineProperty.IS_MODERATOR,

action.getArgument("IS_MODERATOR"));} else {legProperties.put(LineProperty.IS_MODERATOR, false);

}if((action.getArgument("NAME") instanceof Null) == false){legProperties.put(LineProperty.NAME, action.getArgument("NAME"));

} else {legProperties.put(LineProperty.NAME, "");

}if((action.getArgument("COMPANY") instanceof Null) == false){legProperties.put(LineProperty.COMPANY,

action.getArgument("COMPANY"));} else {legProperties.put(LineProperty.COMPANY, "");

}

mediator.getBridge().dialInParticipant(legProperties,action.getArgument("NOTIFY_EVENTS"));

]]>

action.getArgument("SEGMENT_ID") + " ===========================");mediator.getBridge().startQA(action.getArgument("SEGMENT_ID"));

]]>


26/27


27/27

Specifiying wait

In the above example we can seein action. Anode can be placed in any execution node

(scenario, step, or action). It will cause the execution of the node to be held on, until a specific

condition is met. The condition is the expression that we type inside thenode, which can be

either any function that returns integer, or any function that returns boolean.

In case the wait expression produces an integer (n), the scenario-runner will wait for (n) seconds beforeexecuting the node containing that wait. An example of this is the wait for action Stop QA section .

In case the wait expression is a boolean evaluation, as is the wait of scenario in the above example,the scenario-runner will first evaluate the wait expression. The execution moves on normally if the wait

expression evaluates to true. Otherwise, the thread in which the evaluation takes place will move

suspended state (i.e.: calling the wait() method), and only wakes up when either the specified number

of seconds (timeout attribute) has passed. When it wakes up, the wait expression will be re-evaluated.

Wash. Rinse. Repeat.

There's an advanced mechanism that causes the thread to wake up upon notification (thus avoiding the

polling). For that, we need to have an object an approriate type of object that is registered as alistener for the events that take place inside the mediator. Upon receiving the notification from the

mediator, that listener will have to invoke the notifyAll() method, such that the scenario-runner gets

back to live from the suspended state, and re-evaluates the wait expression. Additionaly, the scenario-

runner must be made aware of the existence of such listener. It is achieved by invoking thesetMediatorListener method from inside the very first action that the scenario executes.

The other important, but optional, property ofis timeout. When it's set to true, the scenario-

runner will evaluate thebefore it evaluates the(seespecifying condition below). By

default it is set to true.

Specifying condition

An execution-node can have anode, where we specify the condition under which the

execution-node will be executed. Thenode accepts boolean expression as its value. Normally

we check the value stored in the context or a property of the mediator against a specific value. See the

description of the context ormediator function to understand how to check the object stored in the

context or mediator.

Specifying expectation

Finally, an execution-node can have anode, where we specify the checkings to be performed

at the end of the execution of the execution-node. Just like the, it accepts any boolean

expression as its value, and normally we would make thecontext ormediator function together with

either the eq, lt, orgt function. Additionaly, only for the, we can also use the error

function, if we expect that the execution would throw an error.

Automated Scenario Test

Documents

Transcript of Automated Scenario Test