Agere Systems - Optimay GmbHMay, 2003 - 1 - UMTS terminal testing: A practical perspective Olaf...

Agere Systems - Optimay GmbH May, 2003 - 1 -

UMTS terminal testing: A practical perspective

Olaf Bergengruen

May, 2003


„We will not survive without a proper regression test system“, an Optimay engineer


Overview Virtual testing

• Logical view (what we want to test)

• Why virtual testing

• Requirements to the test system

• 3GPP approach

• Test model

• Test cases

• Optimay‘s implementation

• GSM/GPRS virtual test system

• GSM/GPRS/UMTS virtual test system


Test environment / logical view(conformance testing, FTA)

GSM/GPRS testsGSM 11.10

UMTS tests3GPP 34.123

SAT testsGSM 11.10-4

UTRANsimulation

GERANsimulation

SIMsimulationMS

U m

U u

C u


Test environment / logical view (2)

UTRANsimulation

GERANsimulation

SIMsimulationMS

U m

U u

C u

USB802.11

Bluteoothsimulation

MMIsimulation

WAPTCP/IP

test cases

W APtcp/ip


Why virtual test environment

We need a SW only environment (no specific HW)

• HW is expensive or does not exist

• HW debugging is difficult and time consuming

• SW developers don‘t want to be bothered with HW issues or don‘t have the specific skills

We need to be much faster than real time Developers want to use standard tools (compilers,

editors, debuggers)


Requirements to the virtual test system

Test system shall be much faster than real time

Support the design of test scenarios at a high level of abstraction

All test suites need to run ‚over night‘: GSM/GPRS tests (GSM 11.10), UMTS test (3GPP 34.123), STK tests (GSM 11.10-4), MMI test cases and possibly other suites (TCP/IP, WAP, ...)

Test system shall be available to anybody, at anytime, on any PC


GSM/GPRS test architecture


GSM/GPRS test architecture

SSGMM, MM, CC, etc

Optimay test scripts(GSM/GPRS test cases)

RLC/MAC L2

MMIcontrol

Logical L1

MAC/RLC

MSPhone Apps

Logical L1

L2

DSP emulation

GRR RR

L1 simulation(Radio Block transfer)


Great approach: L1 simulated interface

Basic communication with the Tester via three functions: RxRadioBlock, TxRadioBlock, TimerTick

These functions can be extended as needed to simulate DSP code or other HW related functionality

MS is the master of system ‚ticks‘, about 100 times faster than real time

Tester and MS are tightly coupled which allows debugging using standard tools


Debugging GMM code ...static void GMMDoRoutingAreaUpdate ( GM_IE_UPDATE_TYPE RauType, BOOLEAN FirstAttempt )

{

MMRAUpdPending = FALSE;

GMRauType = RauType;

GMStopT3302( );

GMStopT3311( );

if ( ! FirstAttempt )

{

GMT3330Expiries = 0;

if ( GMActState == GMActDetachInit )

{

GMStopT3321( );

GMDetachPending = TRUE;

}

}

if ( GMTriggerAction == GM_GS_ATTACH_COMP )

{

/* Reinitialise the trigger action variable. */

GMTriggerAction = GM_NO_TRIGGER_ACTION;

/* Confirm the end of the ATTACH procedure to GS */

SEND( GSGMEstCnf ) ( );

}


Highlights of our GSM/GPRS virtual test system

Extremely efficient performance (around 1300 tests in 20 minutes)

Includes testing of DSP codeAbout 30 man-years development effort

(including test case development)


Drawbacks of our GSM/GPRS simulation

Tester uses proprietary (in house) test script languages and encoders which are difficult to maintain

It is difficult to write test cases Tester is not nicely layered as shown in the

previous slide, so it is difficult to extend and maintain

Summarizing: the system can not be extended to implement an UMTS System Simulator


3GPP test model(TS 34.123-3)

RF cable

MAC

PHY (UTRAN)

SS

PHY

MAC

RLC

RRC

NAS (GMM, MM, CC, etc)

TTCN test scenarios

RLC

RRC-adapt.

PDCP

L1 (GERAN)

RLC/MAC L2

LLC

SNDCP

MMI / E-MMIcontrol

L1

MAC/RLCL2

RR / GRR

UEPhone Apps


Why we need a Test Model

Main reason for us: If we comply with the 3GPP Test Model we can use the TTCN test cases developped here at ETSI / STF 160

In general:• Clear and stable interfaces to the SS enables

manufacturers to develop the test equipment

• It reduces ambiguities within test case scenarios

• It enables outsourcing of testing activities and SW re-use

• It provides a common consens or understanding of the complete system to manufacturers and operators


A test case (1/2) ...

Configure SS• Configure PHY / L1

• Configure MAC

• Configure RLC

Schedule and send System Information Blocks Test case preamble

• Bring UE into initial state (e.g. CS and PS registration)

— Perform Location Update procedure

— Perform GPRS Attach procedure


Test case body• Stimulate the UE, e.g.

— Send X message to the UE, or— Change cell power levels, or— Trigger the UE via AT command to initiate a call

• Verify responses— Match messages from the UE to expected values — Assign a verdict (PASS, FAIL, INCONC)

Test case postamble• Complete signalling to bring UE into stable state

A test case (2/2) ...


So, how to design a Virtual Tester which matches the 3GPP model and our requirements ?


Problems designing an overall GSM/GPRS/UMTS Test Env.

The clean 3GPP model is not enough for a complete MS test environment

We need to re-use tools, tracers, and most of all test cases developed for GSM in the new environment

We can not afford to develop the UMTS test suite in house, we need to re-use the 3GPP tests

The L1 and DSP simulation strategies for GSM do not match those of UMTS


Current approach (a compromise)

Multi-threaded system: The proper MS, the GSM SS, the UMTS SS and the SIM simulator

All communicating via shared buffers The tick-master is the MS, when all jobs for

current frame are completed, it issues a ‚tick‘ and all System Simulators prepare data (radio blocks) for next frame.

Each SS will send a ‚tick-ack‘ to the MS when it is ready for next frame

When all SSs are done: the MS L1 engine will fetch or store a radio block


Cu

UuUm

Thread 3

Thread 2Thread 1

.bst, .asc files(Optimay tests)

L1 em ula tion

L2R LC /M AC

GERAN simulation

SN D C P /LLC

TTCN suite(3GPP tests)

PH Y em ula tion

M AC

UTRAN simulation

PD C P

R LC

SIMsimulation

.sat files(Optimay)

Thread 0

MS

GSM/GPRS/UMTS Test Environment

Thread 4

Tracer


Summary

A Virtual Test Environment is an essential tool

• for development

• for quality assurance

3GPP test model

A rough overview of our implementation


Thank you.

Agere Systems - Optimay GmbHMay, 2003 - 1 - UMTS terminal testing: A practical perspective Olaf...

Documents

Transcript of Agere Systems - Optimay GmbHMay, 2003 - 1 - UMTS terminal testing: A practical perspective Olaf...