LZRW3LZRW3 Data Compression CoreData Compression Core

Project part B final presentation

Shahar Zuta Netanel Yamin

Advisor: Moshe porian

December 2013

Contents Project goals & overview Algorithm review Architectures GUI Implementation Test Plan Methods Project Movie

Project OverviewWhy do we need an hardware data

compressor?Reduce storage capacity

Reduce power consumption Reduce amount of data to handle with.

Reduce communication co$tsSpeed

COMPRESSION &SYSTEMS EFFICIENCY

Reduce traffic packets

Reduce comm. resources

Reduce expensesBetter traffic reliability

STORAGE

LZRW3 DECOMPRESSOR

LZRW3 COMPRESSOR LZRW3

DECOMPRESSOR

Less capacity

Less storage discs

Less Area

Less Power

More environmental

Fast lossless compression

Full data recovery

Efficient System

LAN/WAN NETWORK

USER

USER

LZRW3 COMPRESSOR

Project GoalsImplementation of LZRW3 data compression

algorithm High performance- data transfer of 1GbpsAdapted to data templates of 2Kbyte to

32KbyteInternal memory on FPGA only ( Virtex-5 ), no

interface to external memory

Implementing strong debugging capabilities

via GUI

Lossless Data CompressionThe following algorithm is lossless, which

guarantees that the original data could be reconstructed from the compressed file.

Lossy compression could give better compression rates in exchange of data loss (JPEG, MP3)

Known lossless application: ZIP, GZIP and other.

Lossless data compressions are which mean the compressor and decompressor maintain a data structure to help them find repeated strings.

dictionary coding,

mechanism

HASH FUNCTIO

N

INDEX4095

0

INPUT FILE:

Offset

Expression_compress _ion

E x p

Offset value=

0

XXX

ZZZ

YYY

UUU

demonstration

UUU

r e s

3

XXX

Output

Exp

res

L.I

L.I

NOTE: The next 3 byte should be

“x p r” , then “ p r e “ and only then “r e s”, we did’nt demonstrate all the actions

for simplicity.

“L.I“ stands for

“Literal Item “

mechanism

HASH FUNCTIO

N

INDEX4095

0

INPUT FILE:

Expres sion_compress _ion

Offset value=

XXX

ZZZ

YYY

UUU

demonstration

ZZZ

03

6

s i

9

_ o

YYYExp

res

Output

L.I

L.I

sio L.I

n_c L.I

Offset

cn

mechanism

HASH FUNCTIO

N

INDEX4095

0

INPUT FILE:

Expression_c ompress _ion

Offset value=

XXX

ZZZ

YYY

UUU

demonstration

o m p

03

12

6

9

Exp

res

Output

L.I

L.I

sio L.I

n_c L.I

omp L.I

Offset

mechanism

HASH FUNCTIO

N

INDEX4095

0

INPUT FILE:

Express _comp ress _i o

Offset value=

XXX

ZZZ

YYY

UUU

r e s

XXX

03

15

12

96

demonstration

Exp

res

Output

L.I

L.I

sio L.I

n_c L.I

omp L.I

123

C.IXXX

io nn

3+ 012345

Offset

“C.I“ stands for

“Copy Item“

Input file analyze Output file analyze

Input file made of repeating strings Output file made of header + groups

File compression example

Top Architecture

Rx PATH

Tx PATH

INPUT BLOCK memory LZRW3

COMPRESSOR

CORE

COMPRESSED FILE memory

GUI

XILINX VIRTEX 5 ON XUVP505 BOARD

UART

UART

Top Architecture

Rx PATH

Tx PATH

INPUT BLOCKmemory LZRW3

COMPRESSOR

CORE

COMPRESSED FILE memory

GUI

XILINX VIRTEX 5 ON XUVP505 BOARD

UART

UART

Simulations

Frame type 1 Frame type 2

INPUT BLOCK save bytes from

RX PATH

Compression & transffering to output block

OUTPUT BLOCK transffers data to

TX PATH

TX PATHData Encapsulation

&Transmittion

Data out Frame type 0

OUTPUT BLOCK receive compressed

file to FIFO

RAW DATA:

COMPRESSED DATA:

Zoom on compressor

Compressor START, filling the pipeline

“FLY” mode,All stages enabled

EOF arrives,Last 18 bytes

Sending HEADER

SendingDATA

Zoom on compressor

Compressor DONE

PipelineCLEAR

COMPRESSORREADY

COMPRESSOR READY

Test PlanThe tests include different series of data insertion which are supposed to bring the core to it’s extreme cases.

Whenever a change was made in the design all the tests validity was reasserted.

Basic set examplesRandom input (Length, Num Of Vars)

INPUT VAR. LENGTH OUTPUTLENGTH

CASE

RANDOM 10 32K 23K Reasonable compression

RANDOM 10 32K 7407 Each variable repeated 18 times

RANDOM 1 32K 3886 Very high compression ratio ~90 % compression

RANDOM 256 32K 32K Output = input

Input buffer

Test methods:

Hash function

Hash table

comparator

Output block

Compression core

TB

checked

TB TB TB TB

checked

checked

checked

checked

Test methods:

Compression core

Simulation and comparison against golden model

Core periphery

Compression core

Compression core

Core periphery

Simulation adapted to the full chip and comparison against golden model

Verification environment GUI

Simulating full testplan + debug

GUIDirect file path insertion interface

Manually inserted text interface

LZRW3 .Verification environment

GUIConsole

Box

Progress bar

Manually inserted text interface

LZRW3 .Verification environment

GUIRandom data generation characteristics

Random data generationstart button

CLICK

LZRW3 .Verification environment

GUIStart

analyzing buttonCLICK

LZRW3 .Verification environment

Work methodIdentifying bugs such as unreceived data or any

difference from the GM using the verification environment.

Tracing the problem using ChipScope or the simulation environment.

Solving the issue and fixing the code.Asserting the solution validity in the simulation

environment.Resynthesizing the solution and burn it to the FPGA.Verification using the GUI.Reasserting the validity of former tests.Progress.

Resource Utilization

Plan Ahead synthesis result

Implementation ResultsPost -place & route report showed that user

timing constraint is not met. Project goal was 125MHz, the achieved

frequency of the full design is 88MHz.

The critical path found in stage 4.

Input file memorybanks

comparator

Continue

1

0

clk

clkTentative

Next address

clk

counter

offset

TAG

Com

pris

on_v

alid

Compare_success

clk

Offset_tag

Tentative_tag

clk

Tentative_taken

Compare_success_P

Item_length_p

Offs

et_v

alid

Bank 0,1,2addresses

0

1

Addresses

alignment

Older_byte_P

Offset_valid

TENT

B CD

BD

C

C

11

1

0

Input file memorybanks

comparator

Continue

1

0

clk

clkTentative

Next address

clk

counter

offset

TAG

Com

pris

on_v

alid

Compare_success

clk

Offset_tag

Tentative_tag

clk

Tentative_taken

Compare_success_P

Item_length_p

Offs

et_v

alid

Bank 0,1,2addresses

0

1

Addresses

alignment

Older_byte_P

Offset_valid

01

Performance improving actionsAnalyzing critical paths and rewriting their

logic.Checking the extra effort flag (in ISE) and

target to maximum speed.Trying to synthesize with third party

synthesis tools (Precision, Synplify_pro).Swapping the RAM simple blocks

implemented by VHDL code with blocks created by the Core generator (Xilinx tool).

What have we learned?Planning and Specifying a Project.Consider the specifications for micro/macro

architectureRTL coding and “hardware thinking”.Utilizing softwares: Modelsim, ISE, PlanAhead, Synplify, ChipScope, CoreGenerator, Visual Studio (and learning C#)…Testing blocks and determining wanted resultsSynthesizing design and validating it using GUIIncisively explore the FPGA-computer

communicationProtocols: UART, Wishbone

Lab examinationAll we saw this summer

Add picture ofXUP5 BOARD

GPI/O-5: Blinking led, indicates that the DCM

is locked

System reset “push button”

Board JTAG connection

Serial DB-9 connectionXUP5 BOARD

Project MovieLZRW3 Project movie on YouTube