
2/61

Keynote SIGOS GmbH 2008, page 2

real time

reporting

alarming

Measurements

Strategy

Test

Executions

Test Execution on

different locations

KPIs

o v e r v i e w - S I T E r e p o r t e n v i r o n m e n t

Base

Measurements

(Input and Output)


3/61


How to create a report?


4/61


Toolbox for Reports, Report Editor

Reports are created with the Report Editor.

You can open the Report Editor with the corresponding button in the toolbar.

Together with the Report Editor the Toolbox for Reports will be opened in the

navigation area...

Edit Report Definition


5/61



You can open the Toolbox for Reports manually with the regarded button in the

navigation bar:

Toolbox for Reports


6/61



The Toolbox for Reports contains all items you need to define your reports:

Folder to build up a hierarchy in your report definitions

Report Definition defines a new report which consists

of one or more charts

Chart is a diagram which shows one or more

measurements

Alarm defines a threshold on a measurement. If the

threshold is reached a defined action will be

executed.

Measurement Databases contain the measurements of the

testruns. The layout is defined by the Measurement

Strategy.

Measurements are results of the testruns. The calculation of

the measurements is defines in the Measurement

Strategy


7/61



You can build up a structure in the Report Editor by dragging items from the

Toolbox for Reports (navigation area) and dropping them in the working area:

Toolbox For Reports Report Editor


8/61


Toolbox For Reports, Report Editor

You can create reports out of these

structures


9/61


Toolbox For Reports, Report Editor

You create a report by selecting the report definition node in the Report Editor

and pressing the Show Current Report button afterwards.

Show Current Report

or by using the context menu


10/61


f i r s t r e p o r t

open the report editor

drag a new folderfrom the toolbox for reports to the structure tree

drag a new report definition from the toolbox beneath the new folder

drag the KPI Rx_level from the toolbox beneath the new chart

show current report by opening the context menu of the new report or

selecting the icon from the toolbar.

Demonstration: Display the Rx_Level of the testruns within the last 2 hours


11/61


Report Definition

When you click on a Report Definition you will find the parameter tree in the

working area:


12/61


Report Definition - size

With the size parameter you can define the initial size of the

charts in the report.When set to automatic the value is taken from

the table ConfigurationParameters (report image width and report

image height) in the database sitecu. Default is 500x300)


13/61


Report definition Synchronize charts

Synchronize the timelines across all

charts in the report.

By setting to 'no' it is possible to

define individual timelines for each

chart.


14/61


Report Definition - Report Interval

The timing parameter defines the time range for the report. Only results of

testruns with a Timestamp value within this range will be shown in the report.

The timestamp value is the time when the final verdict of a testrun is set. So in

most cases the Timestamp will match the end time in the TestcaseStatus view.


15/61


Report Definition Report Interval

Fixed (account KPIs from a certain start time till a certain end time)

start time end time

Rubberband (account KPIs from a certain start time till time of report generation)

start time Report generation

Window (account all KPIs within a certain time windowbefore report generation)

window Report generation

start time end time

start time Report generation

Report generation

Report generation

window Report generation

t

t

t

t

t

t


16/61


Report Definition - Data Aggregation

In the Report Timing dialog you also

setup the Data Aggregation Interval.

In case you use aggregated

measurements, the Report Interval is

split into slots of the selected

duration.

For each time slot the aggregated

measurements will be combined to a

single data point in the report which

is placed at the end of the time slot.

Note:

If you select Window the time range displayed is aligned according to the data

aggregation interval. If you select Until now you will force the system to extend

the end point of the report window to the time of the report generation.


17/61


Report Definition - Data Aggregation

1

2

3

1

2

3

4

Aggregation method: count

min

max

average

t

t


18/61


Report Definition - Report Schedule

You can schedule a report to be generated one or more times. Any time a

scheduled report is generated, the report data is stored in the database.

You can also enter an email address which the generated reports will be sent

to.


19/61


Report Definition - eMail Report Example


20/61


Report Definition - Filter

The filter parameters allow you to define

a filter for all measurements used in the

charts of the report.

The filter is valid for all measurements

which are taken from the filtered table.

(A filter on the Main table will be valid

for all measurements originating from the

same database.)

You can define following filter:

db_name (Measurement Target), db_table (Testcase), column_filter

If include_invalid is set to no measurements marked as 'invalid'will not be

included in the report.


21/61


Report Definition - Column Filter


22/61


Chart Definition

When you click on a Chart Definition you will find the chart parameter tree in

the working area:


23/61


Chart Definition - Chart Type

marker lines lines

areas

marker

bar stack bar group bar percentage

Chart Types:

matrix single

matrix multi

s1


24/61

Slide 23

s1 matrix multi: multiple KPI, just column headers (x-value)

matrix single: just one KPI, column and row headerssigos, 1/17/2008


25/61


Chart Definition Aggregation Method

The aggregation method is

necessary for the matrix

report types


26/61


Chart Definition Timing

If the parameter synchronize_charts

in the report definition is set to no,

the timeline (report interval) can be

specified for each chart in the report,

separately.


27/61


Chart Definition - Chart Title

You can enter a title of a chart. The title will be displayed at the top of the chart.

There are some text macros available (that will be substituted by the actual values):

: Begin of timing

: End of timing

: Report-definition-Path : Report-definition-Name

: Time zone setting of the report's owner


28/61


Chart Definition - Chart Title

Here you can modify the information

contained in the legend within the chart:


29/61


Chart Definition - Chart Filter

Here you can define a filter for all measurements in

the chart(not in the whole report).

Remember: The filter will only be taken into account if

it is set on the same table as the measurements

actually used in the chart. (The main table will be validfor all measurements of the same target.)


30/61


Chart Definition - Chart Filter

You can select a background color for

the chart. Default is white.


31/61


Chart Definition - Bar Labels

If you choose chart type bar stack or

bar percentage you can enable the

display of labels for each bar. You can

choose between labels showing the

absolute value or the percentage

value.


32/61


KPI Definition

When you click on a KPI Definition you will find the KPI parameter tree in the

working area:


33/61


KPI Definition Data Source

The data source of the KPI can

still be changed.

Changing this parameter have the same

effect as dragging a different KPI,

respectively a KPI from a different KPI

database and/or a different KPI table, intothe chart.


34/61


KPI Definition - Type

You can define a column and/or a test

definition filter on measurements.

The column filter is a string pattern filter

on string value or a numerical filter on

numeric columns on a table which is a

join of the Main table and the testcasespecific table from which the

measurement was taken.

The test definition filter is a filter on the

test definition node in the test editor.

Only results of the given test definition

will be taken of the report.


35/61


Difference test definition and column filter on TestDefPath

You can define a test definition filter or you can define a column filter that

filters for certain test definition paths.

Both filters will only allow measurements of a specific test definition.What's

the difference?

Works after moving the test definition

to another position in the structure or

after renaming

Works after export and reimport tothe same location

column filter testdef filter

no yes

yes no


36/61


KPI Definition - Type

Here you can choose the "type" of the

KPI in regard to its role to the chart.

Following types are available:

y1-value The value will be displayed on

the y-axis.

y2-value The value will be displayed onthe y-axis, but a second scale will be

used.

y3-value, y4-value Additional y-scales

like y2-value

x-value-string All other KPIs of the chart

will be aggregated on the different

values of this KPI. These differentvalues build up the x-axis of the chart.

x-value-numeric Similar to x-value-string

but the points on the x-axis will be

scaled according to the value.


37/61


y2-value example


38/61


x-value-string example


39/61


x-value-numeric example


40/61


KPI Definition - Factor And Scale

In the y_scale_factorand scalingsettings you

can adjust scaling setting. You can multiply

the measurement values with a given factor oradjust the range of the y-axis.

You can also define a label for the axis and a

unit description.


41/61


KPI Definition - Aggregation

There are different aggregation methods

available:

raw no aggregation is done

min/max/avg the minimal (maximal, average)

value of all values in each aggregationinterval is calculated

sum all values in the specific aggregation

interval are summarized

count is the number of database entries

matching all filter criteria

Remember: In the chart the aggregatedvalues will be displayed at the end of the

aggregation interval.


42/61


43/61


KPI Definition Alarm Expiry

Here you can define an expiry period.

If there is no KPI data for at least the

duration of the specified period the

severity of a defined alarm will be set to

indeterminate.

The period can be defined in following

formats: sec, min:sec, h:min:sec. If the

expiry is set to 0 a trapped alarm will

not expire.


44/61


Let's have a closer lookat the report...


45/61


Report Properties

If you look at a report you canadjust the timing settings

temporarily.


46/61


Report Properties

It's also possible to change the

color of the measurements


47/61


Interactive Chart Viewer

With the chart viewer you are

able to browse through the

measurements of your report.


48/61


Interactive Chart Viewer

With the cursor you can runthrough the results by using the

arrow keys. You will see the

Timestamp and the Value for

each data point.

(If the cursor is at the exact x

position the information is

displayed with the color of theaccording KPI. Otherwise the

color of the information is grey

which means, that you see the

information of a data point which

is near to the current position of

the cursor). In case

unaggregated values are used inthe chart you will also see the

TestcaseId.

If you see the TestcaseId you are

able to go to the trace of the

testrun directly.


49/61


Interactive Chart Viewer - Drill down


50/61


Measurement Data

From the report dialog or the

interactive chart viewer you can

open the measurement data view.

Here you can see the list of all

values used for the report.


51/61


Measurement Data Drill down

In the Measurement Data view you can open the traces of the testcases

which resulted in the measurements had the chart consists of.

Note: Drill down is only possible if you use raw data.


52/61


Chart Viewer - Toolbar

Change ReportProperties

Open Interactive

Chart Viewer

Show

Measurement

Data

Show

Report Info

Save Chart

as Picture

Store Report

to Database

Enable/Disable

Cyclic Update

Update view of

current Report

Send link to report

graphic via email


53/61


Chart Viewer - Toolbar

Change Report

Properties

Show

Measurement

Data

Save Chart

as Picture

Enable/Disable

Cyclic Update

Update view of

current Report

Open Trace


54/61


Alarming


55/61


Alarm Definition

The first step to define an alarm is to drag an alarm item from the Toolbox For

Reports beneath a KPI Item of a report in the Report Editor via Drag and Drop.

Toolbox For Reports Report Editor


56/61


Alarm Parameter

If you click on an Alarm definition you will find a parameter tree like this in the

working area:


57/61


Alarm Definition - Parameter

An alarm can be activated

or deactivated. Alarm traps

will only be sent when the

alarm is activated.

The alarm threshold can be

shown in the report chart.

The color of the alarm line in the

report chart can be changed.

An alarm has a certain severity.

You can choose between

warning, minor, major, critical


58/61


Alarm Definition - Threshold

The alarm will be trapped if the

measurement will be above, below,

equal or not equal (condition) a

given thresholdfor at least count

times.


59/61


Alarm Definition - Actions

The alarm text is a user definable

string that will be send together

with other information as alarm

trap

There are different ways to send an

alarm trap. You can send an alarmemail, SMS (additional HW required).

SITE can also send traps to a SNMP

trap host.

There is also the option to call a

certain application or shell script once

the alarm was raised.


60/61


alarm email example


61/61


w o r k s h o p

Chart showing the average duration of all testcases run in the last two weeks.

Chart showing the number of testruns for each testcase (TCName) run in the last

two weeks.

Chart showing the number of passed, failed and incomplete testruns for each

testcase (TCName) run in the last two weeks.

1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2

Documents

Transcript of 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2