Automated Testing Environment

04/21/23 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada

1

Automated Testing Environment

Concepts required for testing embedded systems adopted in this course

(quizzes, assignments and laboratories)


2 / 28

To be tackled today

Why test, and what kinds of tests are there? Difference between

TLD – Test Last Development TDD – Test Driven Development

(Test First Development) How do you add tests? More details on the testing process E-TDD and

the testing toll E-UNIT Other kinds of available tests – time and access Design of custom TESTs and ASSERTS


3 / 28

Why test?

Unit test I AM CURRENTLY BUILDING A NEW FUNCTION –

does it work the way I expect? Testing my and my partner’s understanding of the

Design WE ARE DESIGNING A NEW PRODUCT

-- If we can’t explain (to ourselves via a test) the product ideas – how do we know when product is working?

Regression testing MY PARTNER ADDED A NEW FEATURE to a product –

how do I know it did not “break something” I wrote? System testing

PROOF TO THE CUSTOMER that the product (Lab. or assignment) works the way that was requested.


4 / 28

What are the problems with this sort of test?


5 / 28

VisualDSP development environment(IDDE)

DebuggingInterface

“JTAG” boundary scan operation

Processor

Peripherals

BF533 Evaluation Board

VisualDSP++

program

Debugging Interface

HPPCIce

Lab. station (ICT318 / 320)

Each time we want to send a message (via printf( ) or cout) we must “stop” the processor from doing what we want it to do, and then send messages over the interface – Slow, plus the testing can impact on peripheral performance.

E.g. no sound while printing in Assignment 2 and Lab. 1


6 / 28

More issues with this sort of test!

Tests are mixed up “with production code” Must remove tests before release to customer

Unreliable – may result in test messages at wrong time in released code

Difficult to “add” and “remove tests” Can’t be automated – therefore unlikely to have tests run

often during development – leads to unreliable product


7 / 28

E-TDD Tool

E-TDD – embedded test driven development Build-the-tests-first (during design) ideas for product

development has been made popular by the “Agile” approach

Requires an automated testing approach In this class, we are going to use the E-Unit tool for

its automated behaviour Will use in both “test last” development (what most people

currently do) and in “test first” development. I (the customer) will provide tests so that you can check that

you have developed the “right” stuff at the “high” level You will build additional tests to check your own code at the

“low level”


8 / 28

Test Environment

Can download “LabTestEnvironment.zip” file from the web (Check the September webpages for exact name of file)

LabX directory iswhere you keepyour “final” code

LabXTests is whereyou keep the testsfor LabX

E-UNITIncludes directory contains “everything” needed to make test environment work

E-UNITIncludes


9 / 28

Build a new VDSP project “Assignment1Test” and place in “AssignmentTests” directory

Now add certain standard files to the project – same set of files for all assignments and laboratory

All found in E-UNITIncludes directory

07.h

07.h

07.dlb

07.dlb

Expect minor changes (names and dates) as we “improve” E-UNIT as part of our own going research work on embedded systems

E-UNIT name replaces

E-TDD name


10 / 28

Now add your Assignment1 subroutines files to the Assignment1Test project

Don’t move or copy the files into the test directory – just “add” them to the project

Note the use of “Assignment1.h” which has the prototypes for all “C++” and “assembly code” functions used (from Assignment dir.)

This files are “added” to the“Test” project and NOT copiedfrom the Assignment1 directory

07.dlb

07.dlbE-UNIT name replaces

E-TDD name


11 / 28

Now we need to develop and add “Testmain.cpp”

E-UNIT name replaces

E-TDD name


12 / 28

Now we need to develop and add Assignment1Tests.cpp”

1

2

3

4

5

6

7

8

9

“../E-UNITIncludes/E-UNIT_Tests1Sept07.h”


13 / 28

Tests have a standard format

1

3

4

8

9

TESTGROUP NAME

TESTGROUP NAME, TEST TYPE

TEST TYPETEST NAME

Add further tests at this location



14 / 28

Tests then need to be customized

1

2

3

4

5

6

7

8

9

07h“../E-UNITIncludes/E-UNIT_Tests1Sept07.h”


15 / 28

Test Components

The ACTUAL Test (This is a TEST MACRO)CHECK(result1 == result2) Would provided better test messages if we

“self-documented the code”CHECK(result_CPP == result_ASM);

Test CONTROL statements (ALL MACROS) TEST_CONNECT( test_name) LINK_TEST( test_name, test_type) TEST_LEVEL( which_test_level)


16 / 28

Stages to “automating” the tests

1) Activate the E-TDD Gui (E-TDDIncludes directory)

This causes the GUI to appear, and adds new “menus” to VisualDSP


17 / 28

Stages to “automating” the tests

2) Select E-UNIT Connection | Refresh Connection List

This automatically builds the E-Unit Test Info file


18 / 28

Now “build and run” the testsQuick test failure

visualization

Detailed test failure information

Clicking on the “detailed test info”automatically takes you to the “test” that failed


19 / 28

Test information modes arecontrolled via “ActivateTests.cpp” Show Failures only

Show Failures and Successes

| BTC_CONNECTION

| BTC_CONNECTION


20 / 28

Add a separate test to show that result2(from the ASM code) is equal to 7

Add further tests at this location



21 / 28

Write the tests for “self-documenting” messages(What does “result1 == result2” mean if 200 tests present?)

HAVE

WANT


22 / 28

Write the “CPP” code to get to workbut “Write the ASM for speed”

FAIL

SUCCESS

SUCCESS



23 / 28

Did the ASM code “destroy” or even “use” the CPParray[ ] defined in CPPassignment1.cpp?


24 / 28

Available ASSERTS

LONG INT tests – 32 bit – typical for “C++” CHECK(condition) – Success if passes CHECK_EQUAL(expected, actual) EXPECTED_FALSE(condition) – Success if fails ARRAYS_EQUAL(expected, actual, length) CHECK_VALUES(expectedTemp, condition, actualTemp)

DOUBLES_EQUAL(expected, actual, threshold)


25 / 28

ASSERTS for embedded systems

Short int tests – 16 bit – typical for embedded systems SHORTS_EQUAL(expected, actual) SHORTS_CHECK(expected, condition, actual) USHORTS_EQUAL(expected, actual) USHORTS_CHECK(expected, condition, actual)

Timing of routine – Blackfin Specific MAXTIME_ASSERT(max_time, function) Useful utility

MEASURE_EXECUTION_TIME(timetaken, function)

Examples available in the tests associated with Lab. 1


26 / 28

New ASSERTS can be added to customize the tests or speed the testsASSERT FORMAT

#define CHECK_VALUES(expectedTemp, condition, actualTemp)\if (!((expectedTemp) condition (actualTemp))) \{\

FAILURE1(#expectedTemp##" !"#condition##" "#actualTemp);\}\else {\ SUCCESS1(#expectedTemp##”""#condition##" "#actualTemp);\}


27 / 28

Practice – Assert Design

Design an assert that will determine whether the lower 8 bits of a 32-bit variable is identical to the lower 8 bits of a second 32-bitvariable.

The two variables should remain unchanged by the assert (since they may be reused later)

The upper 24 bits of each variable MUST be ignored in the assert

Does it matter if the variables were defined in the .data section of an assembly code routine or in an C++ code routine?


28 / 28

Handled today – some easy “non-programming” quiz and midterm questions

Why test, and what kinds of tests are there? Difference between

TLD – Test Last Development TDD – Test Driven Development

(Test First Development) How do you add tests for Assignment 1? More details on the testing process E-TDD and

the testing tool E-UNIT Other kinds of available tests – time and access Design of custom TESTs and ASSERTS

Automated Testing Environment

Documents

Transcript of Automated Testing Environment