HW-SW Co-Verification: A Constrained Random Approach

I N V

E N

T I

V E

HW-SW Co-Verification

A Constrained Random Approach

Swaminathan Venkatesan ([email protected])

Ashwin Menon ([email protected])

Trends

• Embedded software complexity continues to increase

• Quality is becoming more important everywhere – Patches and updates take time – Even though it’s “just software”, time is not free

• Metric-Driven Verification, Validation and Test – Automated stimulus generation, functional coverage, and

checking might be a solution

Some Co-Verification issues

• Software development needs to be brought forward and ideally done in parallel with hardware

• Software and Hardware should be debugged together – View software alongside hardware

• Writing many directed software tests is : Tedious, Time consuming, Not Scalable

• Hardware and software stimulus should be coordinated and activity monitored in a unified way to – Hit cross domain corner cases – Check data/temporal behaviour across domains

Finding High-Value Bugs at the Hardware/Software Boundary

• How do we stress the hw/sw boundary? • How do we know we thoroughly stressed it? • How do we know it behaved correctly?

Plogram flow corner cases State based scenarios

Etc..

Interface corner cases State based scenarios

Etc..

SW_STATE == init_driver &

HW_ABORT

MDV MDV

We know how to apply MDV to

hardware interfaces

How do we apply MDV to

software interfaces

Metric Driven Verification

Metric Driven Verification

• Collect Software Metrics – Functional Coverage

• Apply Constrained random verification to Software functions

• Perform HW-SW co-debug

Functional Coverage

6

Functional Coverage

• Monitoring software accesses to variables • Collecting coverage on the software. Coverage is collected

on: – function calls – function call parameter values – function call return values – variable accesses

7

Functional Coverage (Con’t)

8

IP 2

BUS

Memory

Bridge

IP 4 IP 1

IP 3

Bridge Bridge

Driver

RTOS

Application

CPU

snooper

Specman Testbench

Embedded Memory Read

Embedded firmware code

and data in Soc SRAM

Emits event whenever embedded firmware accesses C variables of interest

in the SoC SRAM

Firmware

elf

Extract address of Variables to

be covered

Create Snooper (hdl) to monitor the address

On the bus

Compile and link

Hook the snooper to The bus interface

Coverage goals

Functional Coverage (Con’t) • Variable coverage

– Assign local variables to global variables whose address are deterministic // global variable dtype

enum data_type dtype; // PULL, PUSH, ISO, ASYNC_SIMPLEX sw_test() {

enum data_type dt_local;

set_data_type(dt_local); // API to set the data type

dtype = dt_local; // write to global variable };

– Create the snooper that monitors the writes to the global addresses • Function Coverage

– Wrap the functions for coverage // global variable dtype

sw_test() {

unsigned long pkt_len = 0x20; // packet length

enum pkt_t my_pkt = ENET_802_3; // packet length

enum tag_t my_tag = UNTAGGED; // untagged packet

sn_send_pkt(pkt_len, my_pkt, my_tag);// Function wrapper

};

sn_send_pkt(long pkt_len, enum pkt_t my_pkt, enum tag_t my_tag) {

enet_pkt_len = pkt_len; // assign to global variables enet_pkt_type = my_pkt; // assign to global variables enet_pkt_tag = my_tag; // assign to global variables send_pkt(enet_pkt_len, enet_pkt_type, enet_pkt_tag); // Call the Software API

};

Software Randomization

10

Directed vs Metric-Driven Tests

11

• Direct pre-ordered tests – Write test cases manually

• Random constraint tests – dynamic generation of test cases

Start system

Modify network delay

Set GPS location

Receive call

Start system

Modify network delay

keep delay in [150..500]; keep lattitude in [45..49];

keep longitude in [ -93.. -95];

Set GPS location

Send SMS

Receive Call May miss important

use cases!

SMS received

12

Software Randomization (Con’t)

IP 2

BUS

Memory

Bridge

IP 4 IP 1

IP 3

Bridge Bridge

Driver

RTOS

Application

CPU

snooper

Specman Testbench

Embedded Memory Read

Embedded Memory Write

Randomize variable and write back to

memory (backdoor)

Emits event whenever embedded firmware

accesses C variables of interest in the SoC SRAM

Embedded firmware code

and data in Soc SRAM

13

Software Randomization (Con’t) • Directed Test

sw_test() {

unsigned long pkt_len = 0x20; // - Fixed enum pkt_t my_pkt = ENET_802_3; // - Fixed enum tag_t my_tag = UNTAGGED; // - Fixed sn_send_pkt(pkt_len, my_pkt, my_tag);

};

sn_send_pkt(long pkt_len, enum pkt_t my_pkt, enum

tag_t my_tag) {

enet_pkt_len = pkt_len; // assign to global variable

enet_pkt_type = my_pkt; // assign to global variable

enet_pkt_tag = my_tag; // assign to global variable

send_pkt(enet_pkt_len, enet_pkt_type,enet_pkt_tag); // Call the API

};

• Randomize the packet parameters

extend enet_pkt_u {

rand_pkt() is {

var pkt_len : uint;

var pkt_type : enet_pkt_t;

var pkt_tag : enet_pkt_tag_t;

// Randomize packet length

gen pkt_len keeping

{ it > 50 && it <= 1500};

// random packet type

gen pkt_type;

// random

gen pkt_tag;

// back door write to memory

write_backdoor(pkt_len, pkt_type, pkt_tag); }

}

Hardware/Software co-debug

14

Embedded Software Trace Requirements

• Debug hardware/software corner case issues • Connects waveform viewer and software source code • Trace waveforms from simulation or emulation • Post mortem backward and forward debugging

Extensions for “software debugger like” features

Allows waveform position to be related to software/

firmware position

Step forward and back to the next/previous

address value Step forward and back

to the next/previous line of source

Step forward and back to a specific address

Step forward and back to next/previous function

execution

All the time the source is linked to the waveform cursors

Conclusion

• No single person/team understands a complex embedded system

• Each group is using different tools, languages, methods • Unit testing is not enough • Connectivity and communication tests • Testing speed and accuracy tradeoff

17

Conclusion (con‘t)

• Define constraints and checks for each component • Provide API to use testbench in system context • Combine testbenches for next level • Testing language that supports

– Parallelism – Event-based Communication – Reproducible Constraint Randomization – Coverage – Checks

18

HW-SW Co-Verification: A Constrained Random Approach

Technology

Transcript of HW-SW Co-Verification: A Constrained Random Approach