Apr 22, 2023 1

Real Time Image Warper using Digilent Spartan 3 FPGA

By Shane AgnewSupervisor: Dr. Fearghal Morgan

Final Year Project

Apr 22, 2023 2

Contents

Initial aims / SpecificationsIntroductionCameraBitmapsVHDLGraphical User InterfaceFuture WorkSummary

Apr 22, 2023 3

Initial aims / Specifications

Generate image dataIncorporate a communication protocol into the designPerform all required file formattingImplement digitalsignal processingon-board the FPGADevelop a real-time multi-functionalvideo processing application

Apr 22, 2023 4

Introduction

SRAM 0 SRAM 1

DPIMREF

IOCSRBlkDatCtrlr

fileWrite

fileRead

USB 2.0 PeripheralCommunications

Module

Spartan 3 Starter Board

Spartan 3 FPGA

B1

RAW OutputFile

RAW Input File

Camera 24-bit BMP bmpConverter24to8Bit

8-bit BMPbmp2raw

raw2bmp8-bit BMP

DSPBlk MemCtrlr

Empty

3000h

2000h

1000h

0000hSRAM Memory Contents

3FFFFh

AppliedVHDLUSBSimple VB GUI

Apr 22, 2023 5

Camera

Trust 120 SpacecamStandard web camera24-bit BMP OutputSoftware provided

24-bit BMPsSimple to useUncompressed dataMost web-cameras support them1 byte for each of red, green and blue, hence RGB value

Apr 22, 2023 6

Bitmaps

8-bit BMPsLimited SRAM memory available3 times less communication overhead3 times less file accessLess awkward to process and store in SRAMSingle byte represents a single pixel

Apr 22, 2023 7

Bitmaps – Greyscale

Greyscale256 colours availableBlack represents 0, white represented by 25524-bit BMP converted to 8-bit by processing each pixel’s RGB value

Apr 22, 2023 8

Bitmaps – Colours

Colours256 Colours created using RGB values from 0% 0% 0% to 100% 100% 100%, e.g. 20% red 0% green 20% blue24-bit pixels rounded to these boundariesColours sorted by hue to give a natural colour progression

Apr 22, 2023 9

VHDL – AppliedVHDLUSBSimple System Overview

rxDat(7:0)

dPathDone

dspDone

ramDone

csrOut(7:0)

ldIODat2RamReg

IOCSRBlk 4.0

txDatAck

rxDatValidtxDatValid

clkCSR(7:0)

rxDatAck

csrTask

maxCount(15:0)

ldByteCnt

IORamWr

IORamRd

dspActive

addrCnt(15:0)

rst

astb

mclk

dstbRegister0(7:0)

pwait

clk

rst

DPIMREF 1.0

dspActive

datFromRam(31:0)

dspRamWr

dspBlk 3.0

dspAddRange(15:0)

ramDone

dspDat2Ram(31:0)

clk

dspDone

dspRamAdd(17:0)

dspRamRd

rst

dspFunc(7:0)

datFromRam(31:0)

txDat(7:0)

datCtrlr 2.0

clk

rst

txDatValid

rxDatAck

rxDat(7:0)

ldByteCnt

ldIODat2RamReg

csrTask

csrOut(7:0)

dPathDone

IODat2Ram(31:0)

dspActive

dspRamAdd(17:0)dspDat2Ram(31:0)

IORamWr

datFromRam(31:0)

UB1L

memCtrlr 5.0

dspRamWrdspRamRd

CE1L

clk

ramAdd(17:0)

ramDone

ramIO(31:0)

IORamRd

LB1L

WEL

OEL

CE0LUB0L

IODat2Ram(31:0)

IORamAdd(17:0)LB0L

rxDatValid

txDatAck

pdb(7:0)

pwr

Register1(7:0)

CSR7(7:0) : dspFunc(7:0)CSR6(7:0) : dspAddRange(7:0)

CSR5(7:0) : dspAddRange(15:8)CSR4(7:0) : endIOAddr(7:0)

CSR3(7:0) : endIOAddr(15:8)CSR2(7:0) : startIOAddr(7:0)

CSR1(7:0) : startIOAddr(15:8)XXXXXX CSR0(1:0) : quadrant

&

&

&-

&+

dspFunc(7:0)

dspAddRange(15:0)

maxCount(15:0)

maxCount(15:0)

dspAddRange(15:0)

dspFunc(7:0)

txDat(7:0)

dPathDone

txDat(7:0)

dPathDone

dspDone

dspDone

ramDone

ramDone

mclkastbdstbpwr

pdb(7:0)pwait

ramAdd(17:0)ramIO(31:0)

CE1LUB1LLB1L

WELOEL

CE0LUB0LLB0L

88

8 8

8

8

16

8

16

32

8

8

For Reference

8

88

88

888

8

16

16

16

218

16

18

32

32

8

8

16

32

18

32

32

18

32

32

18

addrAndMaxCount 6.0

Apr 22, 2023 10

VHDL – DPIMREF

register1 register1

busEppIn regEppAdr

register0

busEppData

CS

NS

NS & OutputDecode

FSM Comb

FSM Synch

clkMain

ctlEppDstb

ctlEppAstb

ctlEppWrctlEppAwr

ctlEppDwr

dpimrefFSM 1.1

memWrValid

memRdValid

pwr

astb

dstb

mclkrxDatValid

dpimrefRegisterRW 1.2

txDatAck

rxDatValid

pdb

8

8register0

“00000001”

“00000000”

0

1

8

8

8

8

8

8

8 8

8

memWrValid

memRdValid

clk

clk

busEppOut

ctlEppWait

txDatAck

pwait

ctlEppDir

ctlEppWr

memWrVal 1.4

ctlEppDwr

ctlEppAwr

=

memRdVal 1.3

=

Apr 22, 2023 11

VHDL – DPIMREF

Sample provided by DigilentProvides VHDL side USB interface with USB 2.0 Peripheral DeviceAltered to generate rxDatValid and txDatAck signalsThese signals along with the rxDat and txDat input and output data lines allow communication with rest of the appliedVHDLUSBSimple system

Apr 22, 2023 12

VHDL – IOCtrlr

rxDat(3:0)

dPathDone

dspDone

ramDone

enCSRWr

ldIODat2RamReg

ioCtrlrFSM 4.1.1

txDatAck

rxDatValidrxDatValid

txDatValid

allWrDone

allRdDone

clk

regCSRInfo

rxDatAck

enCSRWr

clrCSRInfo

allReadDone 4.1.3

>

allWriteDone 4.1.4

>

maxCount16

16

DinCnt(15:0)

Ld

CEclk

addrCnt 4.1.2

16

16

X”0000"16

16

allRdDone

allWrDone

ldByteCnt

IORamWr

IORamRd

dspActive

incAddrCnt

ldAddrCnt

CS

NSNS & OutputDecode

FSM Comb

FSM Synch dspActive

IORamRd

IORamWr

ldByteCnt

rxDatAck

txDatValid

ldIODat2RamReg

D0(2:0) Q0(2:0)

Ldclr

cmdReg 4.1.5

clk

Q1D1

txDatAck

dPathDone

dspDone

ramDone

rxDat(7:0)4

3

1

csrAdd(2:0)csrTask

addrCnt(15:0)

+116

rst

rst

rst

4

8

rxDat(1) & rxDat(6:4)

rxDat(6:4)rxDat(1)

rxDat(3:0)

3

Apr 22, 2023 13

VHDL – IOCtrlr

Responds to USB transfer signals, rxdatValid and txDatAck, generated by the DPIMREFAllows the system registers (CSRs) to be setup, both DSP and Input/OutputKeeps track of the total number of bytes written or read from SRAM and increments the current SRAM address accordinglyControls DSP activation

Apr 22, 2023 14

VHDL – DSPBlk

dDatFromRam(31:0)

datFromRam(31:0)

-A

BB-A

&

012

3

4

0

1

%2-

A

BB-A

cnt

0

1

dspAddRange(15:0)

dspRamAdd(15:0)

dspRamDat(31:0)

(int)

imageInvDSPWr

intDSPFunc

Conv_std_logic_vector

16

32

16 16

16

16

16

323232

32

32

(int)

32

3232

32

32

32

32

32

32

8888

dDatFromRam(7:0)

dDatFromRam(15:8)

dDatFromRam(23:16)

dDatFromRam(31:24)

This ANDing reverses theorder of bytes -> backwards

Apr 22, 2023 15

VHDL – DSPBlk

Six DSP functions implemented:Delta functionColour inversionFlip image over y-axisDistortionMorphingColour change

Effective demonstrations of the Spartan 3’s power but more complex DSP functions are possible

Apr 22, 2023 16

Graphical User Interface

Apr 22, 2023 17

Graphical User Interface

Allows a user to check the contents of any SRAM locationCan activate the camera software to take picturesImage loading and conversion from 24-bit to 8-bit BMPsDSP options as mentioned previouslyOption to perform all operations in colour or greyscale

Apr 22, 2023 18

Future Work

Implement more complicated DSP, including:Edge detectionObject ExtractionObject warping

Provide networking solution to allow Electronics department to host the project and make it accessible anywhere in the world

Apr 22, 2023 19

Summary

Initial aims / SpecificationsIntroductionCameraBitmapsVHDLGraphical User InterfaceFuture WorkSummary