TPT – Automated Embedded Testing

Automated Testing Control Systems Systematic Testing Continuous Behavior TPT is a test tool used in the testing of embedded control systems. It supports full automation testing in a real-time context and can use continuous as well as discrete signals. Test cases are modelled graphically. Thanks to the built–in platform independence, TPT can be used universally from Model-test (MiL), through Software-test (SiL) to Hardware-in-the-Loop-Test (HiL). TPT currently offers test support for MATLAB/Simulink models, ‘C’ code on the host processor, ‘C’ code on debugger boards via Tessy, HiL test systems as well as some proprietary environments.

Time Partition Testing

IBES INTELLIGENT BUSINESS & EDUCATION SOLUTION JOINT STOCK COMPANY Add. Room 19A , Indochina Building, No. 4 Nguyen Dinh Chieu Street, Dakao Ward, Dist.1, HCMC, Vietnam Tel: 08 37965969 - Fax: 08 37965969 - Cell No. +84 90 3384610

Model-Based Testing in Automotive Industry

with TPT Software

Typical Users of TPT Software

IBES INTELLIGENT BUSINESS & EDUCATION SOLUTION JOINT STOCK COMPANY Add. Room 19A , Indochina Building, No. 4 Nguyen Dinh Chieu Street, Dakao Ward, Dist.1, HCMC, Vietnam Tel: 08 37965969 - Fax: 08 37965969 - Cell No. +84 90 3384610

1. Introduction about MBD Testing for AI with TPT software

- A tool for testing control and feedback control systems

- Testing and verifying in all development phases

- Dynamic testing at all integration levels

Time Partition Testing

IBES INTELLIGENT BUSINESS & EDUCATION SOLUTION JOINT STOCK COMPANY Add. Room 19A , Indochina Building, No. 4 Nguyen Dinh Chieu Street, Dakao Ward, Dist.1, HCMC, Vietnam Tel: 08 37965969 - Fax: 08 37965969 - Cell No. +84 90 3384610

- Supported Test Cases in different environments

- Dashboard – Interactive UI's for the system under test

IBES INTELLIGENT BUSINESS & EDUCATION SOLUTION JOINT STOCK COMPANY Add. Room 19A , Indochina Building, No. 4 Nguyen Dinh Chieu Street, Dakao Ward, Dist.1, HCMC, Vietnam Tel: 08 37965969 - Fax: 08 37965969 - Cell No. +84 90 3384610

2. Main Automotive applications testing by TPT Software

▪ Body controller projects ▪ Interior and exterior lights, seat heating and

ventilation, climate, cluster ▪ Powertrain controller projects

▪ Transmission, HV battery, power electronics, hybrid powertrain, engine

▪ ADAS controller projects ▪ Emergency braking, lane departure warning, blind

spot detection ▪ Autonomous driving

▪ Drive dynamics controller projects ▪ Cross-wind control, roll control

IBES INTELLIGENT BUSINESS & EDUCATION SOLUTION JOINT STOCK COMPANY Add. Room 19A , Indochina Building, No. 4 Nguyen Dinh Chieu Street, Dakao Ward, Dist.1, HCMC, Vietnam Tel: 08 37965969 - Fax: 08 37965969 - Cell No. +84 90 3384610

3. TPT Test Automation

3.1 - Test Modeling Technique

TPT automates all steps from execution to reporting

Testlet Testlet

Testlet Testlet

Variant modeling

Decision at runtime

IBES INTELLIGENT BUSINESS & EDUCATION SOLUTION JOINT STOCK COMPANY Add. Room 19A , Indochina Building, No. 4 Nguyen Dinh Chieu Street, Dakao Ward, Dist.1, HCMC, Vietnam Tel: 08 37965969 - Fax: 08 37965969 - Cell No. +84 90 3384610

Testlet Testlet Testlet

Time partitioning Parallel processes

Loop

Hirarchy

IBES INTELLIGENT BUSINESS & EDUCATION SOLUTION JOINT STOCK COMPANY Add. Room 19A , Indochina Building, No. 4 Nguyen Dinh Chieu Street, Dakao Ward, Dist.1, HCMC, Vietnam Tel: 08 37965969 - Fax: 08 37965969 - Cell No. +84 90 3384610

3.2 – Manual Testing

a) TPT Dashboard

▪ Design your own graphical interface for your system under test ▪ Observe and interact with tests using the Dashboard ▪ Test your system manually with the Dashboard (Simulation) ▪ Automatically record test cases from your interactions ▪ Distribute your simulation environment with

the Dashboard Player (separate tool at no charge)

IBES INTELLIGENT BUSINESS & EDUCATION SOLUTION JOINT STOCK COMPANY Add. Room 19A , Indochina Building, No. 4 Nguyen Dinh Chieu Street, Dakao Ward, Dist.1, HCMC, Vietnam Tel: 08 37965969 - Fax: 08 37965969 - Cell No. +84 90 3384610

b) Macro recorder Dashboard interaction

3.3 – Test case generation methods in TPT

▪ Record dashboard

interaction as test cases

▪ Generate Step Lists

▪ Create new test cases

▪ Integrate recorded tests

into existing step-lists

▪ Post-process step list

manually if required

▪ TASMO for structural coverage tests of Simulink models and C-Code

▪ Combination of Equivalence classes

▪ Path and variant combination of TPT automatons

▪ Generate test cases from value ranges

IBES INTELLIGENT BUSINESS & EDUCATION SOLUTION JOINT STOCK COMPANY Add. Room 19A , Indochina Building, No. 4 Nguyen Dinh Chieu Street, Dakao Ward, Dist.1, HCMC, Vietnam Tel: 08 37965969 - Fax: 08 37965969 - Cell No. +84 90 3384610

Test cases generation from Value Ranges

IBES INTELLIGENT BUSINESS & EDUCATION SOLUTION JOINT STOCK COMPANY Add. Room 19A , Indochina Building, No. 4 Nguyen Dinh Chieu Street, Dakao Ward, Dist.1, HCMC, Vietnam Tel: 08 37965969 - Fax: 08 37965969 - Cell No. +84 90 3384610

▪ TPT test model has many variants of states and transitions

▪ Combination can be built automatically

▪ Tests can be generated

IBES INTELLIGENT BUSINESS & EDUCATION SOLUTION JOINT STOCK COMPANY Add. Room 19A , Indochina Building, No. 4 Nguyen Dinh Chieu Street, Dakao Ward, Dist.1, HCMC, Vietnam Tel: 08 37965969 - Fax: 08 37965969 - Cell No. +84 90 3384610

4. Test Execution : TPT tests in different environments…

…and execute as below process:

IBES INTELLIGENT BUSINESS & EDUCATION SOLUTION JOINT STOCK COMPANY Add. Room 19A , Indochina Building, No. 4 Nguyen Dinh Chieu Street, Dakao Ward, Dist.1, HCMC, Vietnam Tel: 08 37965969 - Fax: 08 37965969 - Cell No. +84 90 3384610

4.1 – TPT Test process for Matlab/Simulink

• Support of Simulink with/without Plant Models

+ Opened Loop Control Testing

+ Closed Loop Control Testing

IBES INTELLIGENT BUSINESS & EDUCATION SOLUTION JOINT STOCK COMPANY Add. Room 19A , Indochina Building, No. 4 Nguyen Dinh Chieu Street, Dakao Ward, Dist.1, HCMC, Vietnam Tel: 08 37965969 - Fax: 08 37965969 - Cell No. +84 90 3384610

• Support of internal signals

+ Feed model-internal signals back to TPT

+ No modification of the SUT necessary

4.2 - TPT Test process for ASCET

IBES INTELLIGENT BUSINESS & EDUCATION SOLUTION JOINT STOCK COMPANY Add. Room 19A , Indochina Building, No. 4 Nguyen Dinh Chieu Street, Dakao Ward, Dist.1, HCMC, Vietnam Tel: 08 37965969 - Fax: 08 37965969 - Cell No. +84 90 3384610

4.3- TPT Test process for C/C++ Code

4.4- TPT Test for FUSION Co-simulation platform

IBES INTELLIGENT BUSINESS & EDUCATION SOLUTION JOINT STOCK COMPANY Add. Room 19A , Indochina Building, No. 4 Nguyen Dinh Chieu Street, Dakao Ward, Dist.1, HCMC, Vietnam Tel: 08 37965969 - Fax: 08 37965969 - Cell No. +84 90 3384610

• FUSION support:

+ FMI : Functional Mock-up Interface

+ FMUs for co-simulation with other FUSION nodes

4.5- QTRONIC Silver Adapter

▪ Testing of closed loop virtual ECUs with QTRONIC Silver ▪ TPT Integration as a Silver Node ▪ Read, Write and Read/Write access to signals and parameters

IBES INTELLIGENT BUSINESS & EDUCATION SOLUTION JOINT STOCK COMPANY Add. Room 19A , Indochina Building, No. 4 Nguyen Dinh Chieu Street, Dakao Ward, Dist.1, HCMC, Vietnam Tel: 08 37965969 - Fax: 08 37965969 - Cell No. +84 90 3384610

4.6- AUTOSAR Support

4.7- SiL and PiL with Debuggers

▪ Test SW-C using TPT on PC ▪ Support of S/R, C/S , IRV, PIMs, Compositions, explicit and implicit communication ▪ AUTOSAR compatibility mode

▪ Support of testing with debuggers on target HW such as ▪ GDB – GNU debugger on PC ▪ PLS-UDE on TPT +UDE on AURIX, Tricore, ARM Cortex etc. ▪ Lauterbach TRACE 32 on ARM, Power Architecture etc.

IBES INTELLIGENT BUSINESS & EDUCATION SOLUTION JOINT STOCK COMPANY Add. Room 19A , Indochina Building, No. 4 Nguyen Dinh Chieu Street, Dakao Ward, Dist.1, HCMC, Vietnam Tel: 08 37965969 - Fax: 08 37965969 - Cell No. +84 90 3384610

4.8- HiL Test Execution

a) dSPACE HiL Fusion Adapter (ASAM XIL)

▪ Test of dSPACE HiL / VeOS / SCALEXIO components via ASAM XIL API

▪ Change signals

▪ Error simulation

▪ Application via CANape or INCA

▪ Measurement via HiL, CANape/Inca

▪ Diagnosis via CANape

▪ Extensible for other ASAM XIL-based platforms

IBES INTELLIGENT BUSINESS & EDUCATION SOLUTION JOINT STOCK COMPANY Add. Room 19A , Indochina Building, No. 4 Nguyen Dinh Chieu Street, Dakao Ward, Dist.1, HCMC, Vietnam Tel: 08 37965969 - Fax: 08 37965969 - Cell No. +84 90 3384610

b) National Instrument – NI : VeriStand

c) VECTOR CANoe Integration

▪ Test National Instrument HiL system via Test Stand ▪ Real-time via test frontloading ▪ Automatic Custom Device Conf iguration from System

Def inition File

▪ Test VECTOR CANoe Simulation Setups

▪ Stimulation an Observation of

bus signals and system variables

▪ Test driver integrated into

CANoe via custom library

▪ Incubation feature

IBES INTELLIGENT BUSINESS & EDUCATION SOLUTION JOINT STOCK COMPANY Add. Room 19A , Indochina Building, No. 4 Nguyen Dinh Chieu Street, Dakao Ward, Dist.1, HCMC, Vietnam Tel: 08 37965969 - Fax: 08 37965969 - Cell No. +84 90 3384610

d) CAN, INCA + CANape platform environment

▪ CAN stimulation: ▪ Map TPT signals to CAN messages

▪ Changing application parameter

▪ via INCA or CANape

▪ Measure

▪ internal signals in controller

▪ Diagnosis via CANape

IBES INTELLIGENT BUSINESS & EDUCATION SOLUTION JOINT STOCK COMPANY Add. Room 19A , Indochina Building, No. 4 Nguyen Dinh Chieu Street, Dakao Ward, Dist.1, HCMC, Vietnam Tel: 08 37965969 - Fax: 08 37965969 - Cell No. +84 90 3384610

e) Driver in The Loop Testing – DiL

▪ Via CAN ▪ Via application tools like INCA ▪ Via on-line driver information ▪ information

▪ Message boxes ▪ Acoustic messages ▪ Step list highlighting11

IBES INTELLIGENT BUSINESS & EDUCATION SOLUTION JOINT STOCK COMPANY Add. Room 19A , Indochina Building, No. 4 Nguyen Dinh Chieu Street, Dakao Ward, Dist.1, HCMC, Vietnam Tel: 08 37965969 - Fax: 08 37965969 - Cell No. +84 90 3384610

5.TEST ASSESSMENT

There are 4 main scopes of TPT Test Assessment process

1

2

3 4

In this Test Assessment process, the special thing that we are not only using Script to write a test cases (customized) but also make it automated

IBES INTELLIGENT BUSINESS & EDUCATION SOLUTION JOINT STOCK COMPANY Add. Room 19A , Indochina Building, No. 4 Nguyen Dinh Chieu Street, Dakao Ward, Dist.1, HCMC, Vietnam Tel: 08 37965969 - Fax: 08 37965969 - Cell No. +84 90 3384610

5.1- Manual evaluation of test result

5.2- Back to Back regression testing in Simulink

• Data Viewer for data display

• Manual setting of test results + Successful + Failed

• Signal Viewer also available as standalone application

• Data formats such as tptbin, mat, mdf, csv

▪ Two test execution configurations -> e.g. MiL and SiL ▪ Automatic code generation ▪ Run both executions automatically ▪ Compare test results automatically including tolerances

IBES INTELLIGENT BUSINESS & EDUCATION SOLUTION JOINT STOCK COMPANY Add. Room 19A , Indochina Building, No. 4 Nguyen Dinh Chieu Street, Dakao Ward, Dist.1, HCMC, Vietnam Tel: 08 37965969 - Fax: 08 37965969 - Cell No. +84 90 3384610

5.3- Step based Assessment in Test Step List

5.4- Standard TPT Assessments in GUI

Reporting:

▪ Min-Max

▪ Signal Comparison

▪ Temporal and Logical Constraints

▪ Interactive Condition-Tree

▪ Mode detection

IBES INTELLIGENT BUSINESS & EDUCATION SOLUTION JOINT STOCK COMPANY Add. Room 19A , Indochina Building, No. 4 Nguyen Dinh Chieu Street, Dakao Ward, Dist.1, HCMC, Vietnam Tel: 08 37965969 - Fax: 08 37965969 - Cell No. +84 90 3384610

Programming support under Test

Scripting:

Function:

▪ Analysis is always performed in context of a well-def ined time interval using temporal regular expression

▪ Every assessment can be analyzed in multiple time intervals:

during [a(t)>17]: TPT.assertAlways(this.getLength()<10.0,“Check for length“)

during [light_switch == LIGHT_ON]: TPT.assertAlways(headlight == true,“Check headlight on if switch on“)

during [pedal > 50 and speed < 100]: cond := TPT.monotony(v(t), TPT.INCREASING);

TPT.assertAlways(cond,“Check for monotony“)

▪ Compare := TPT.hose(orig(t), ref(t), xtol, ytol)

▪ VehicleBoundsCheck := TPT.checkAlways(v(t) < 250.0); ▪ AccelerationCheck := TPT.monotony(a(t),TPT.INCREASING); ▪ Never := TPT.never(rpm(t), 30); ▪ rpm_average := TPT.filterMA(rpm(t), 30);

IBES INTELLIGENT BUSINESS & EDUCATION SOLUTION JOINT STOCK COMPANY Add. Room 19A , Indochina Building, No. 4 Nguyen Dinh Chieu Street, Dakao Ward, Dist.1, HCMC, Vietnam Tel: 08 37965969 - Fax: 08 37965969 - Cell No. +84 90 3384610

6.TEST REPORTING

TPT is fully support to generate test report with high accuracy

6.1- Report for Coverage of Requirement

6.1- Report for Coverage of Requirement

IBES INTELLIGENT BUSINESS & EDUCATION SOLUTION JOINT STOCK COMPANY Add. Room 19A , Indochina Building, No. 4 Nguyen Dinh Chieu Street, Dakao Ward, Dist.1, HCMC, Vietnam Tel: 08 37965969 - Fax: 08 37965969 - Cell No. +84 90 3384610

6.2- Report for Coverage of Model and Code

▪ MATLAB/Simulink ▪ MathWorks V&V toolbox (e.g Simulink Coverage will be used from R2017b) ▪ TargetLink ▪ CTC++ ▪ TASMO

▪ C-Code ▪ CTC++ ▪ GNU gcov

IBES INTELLIGENT BUSINESS & EDUCATION SOLUTION JOINT STOCK COMPANY Add. Room 19A , Indochina Building, No. 4 Nguyen Dinh Chieu Street, Dakao Ward, Dist.1, HCMC, Vietnam Tel: 08 37965969 - Fax: 08 37965969 - Cell No. +84 90 3384610

7. SUMMARY

TPT features

▪ Modeling ▪ Graphical test models ▪ Test step lists or direct def initions ▪ Closed-loop (reactive) tests ▪ Hard real-time enabled (≤100µs cycles) ▪ Signal import ▪ Signal editor ▪ Wizard based signal creation ▪ Variant handling / one model for all tests ▪ Test case generation (combinatory) ▪ Libraries ▪ Parameter Support ▪ Scalars, arrays, curves, and maps ▪ Parameter import ▪ Parameter overloading/calibration ▪ Online parameter calibration

▪ Execution ▪ Support of many execution platforms

(MATLAB, fusion ,…) ▪ Multiple test sets and execution

conf igurations ▪ Debugger for analysis

▪ Assessment ▪ Online and Offline ▪ Back-to-back Analysis ▪ Temporal conditions ▪ General constraints and analysis per scenario ▪ Flexible offline concept with scripting

language ▪ Assessment Libraries

▪ Test Management ▪ TestRail ▪ TestLink ▪ IBM ALM ▪ Jenkins ▪ SVN

▪ Requirements Management ▪ Doors (native DXL and CSV) ▪ Excel ▪ Polarion ▪ Reqtify

▪ Hardware ▪ CAN ▪ CANape ▪ INCA ▪ PLS UDE ▪ NI-HW

▪ File I/O (Test Data I/O) MDF, MD4, DBC, LDF, A2L, DCM, ARXML,

HDR, CSV, TPTBIN, EXCEL, MAT

IBES INTELLIGENT BUSINESS & EDUCATION SOLUTION JOINT STOCK COMPANY Add. Room 19A , Indochina Building, No. 4 Nguyen Dinh Chieu Street, Dakao Ward, Dist.1, HCMC, Vietnam Tel: 08 37965969 - Fax: 08 37965969 - Cell No. +84 90 3384610

Application Examples

• BMS integration Testing (or: BMC – Battery Management Controller)

• GearBox testing on SiL

▪ Integration testing as above ▪ Module testing with or without cell models ▪ 4 years of testing, thousands of test cases

▪ Driving maneuvers in different environments ▪ Error simulation ▪ Failure reaction tests

IBES INTELLIGENT BUSINESS & EDUCATION SOLUTION JOINT STOCK COMPANY Add. Room 19A , Indochina Building, No. 4 Nguyen Dinh Chieu Street, Dakao Ward, Dist.1, HCMC, Vietnam Tel: 08 37965969 - Fax: 08 37965969 - Cell No. +84 90 3384610

Notes:

TPT – Automated Embedded Testing Test modelling with TPT Test cases are modelled graphically in a feature-rich environment. Features such as parallel and hierarchical test ow, conditional branching as well as support for continuous and discrete signals allow the construction of complex test cases. For the testing of complex systems TPT offers a powerful approach for systematic test-case-generation which guarantees easy interaction and readability even with a larger number of tests.

Test execution Test execution with TPT is fully automatic in almost every test environment. The core of the execution is TPT’s virtual machine (VM) which allows tests to execute in real-time, hence making it possible to have cycle times in the order of microseconds. The VM requires only a few kilobytes of memory so that it may also be used in environments with limited resources .

Test evaluation The automatic test evaluation can be done online (i.e. during the test execution) or offline (at the end of the test execution). It can formulate complex quality criteria made up from powerful operations such as comparisons with reference data, signal filters, state sequences and timing conditions. Test Documentation TPT produces a test report of the test execution which contains the most important information regarding the execution and results of the test case. The information in the report can be configured such that it remains a readable summary of the execution even for highly complex tests.

Examples TPT was originally designed for use in a

wide variety of development projects

from initial modelling through to pre-

production verification at Mercedes. It is

now used by many companies within the

automotive industry. Typical projects

using TPT include engine control, power

train control, driver assistant systems

and interior systems control.

Features of TPT - Testing using continuous

and discrete signals - Real-time enabled test execution - Graphical test modelling - Automated execution,

evaluation, documentation and management of tests

- Compatibility with MATLAB/Simulink-

Models, C-Code, HiL systems and others

Integration into development environments To integrate TPT in an existing test or develop- ment environment, it is sufficient to attach the TPT virtual machine. TPT handles this integration itself, the development project’s toolset does not need to be extended or changed for TPT .

Time Partition Testing