Real Time Video Capture and Real Time Video Capture and Image Processing System Image Processing System

using FPGAusing FPGA

Jahnvi VaidyaJahnvi Vaidya

Advisors: Dr. Yufeng Lu and Dr. In Soo AhnAdvisors: Dr. Yufeng Lu and Dr. In Soo Ahn

4/30/20094/30/2009

OutlineOutlinenn IntroductionIntroductionnn System developmentSystem development

nn Video captureVideo capturenn Image processingImage processingnn Image processingImage processing

nn ResultsResultsnn ApplicationApplicationnn ConclusionConclusionnn BibliographyBibliography

Embedded systemEmbedded system

EquipmentEquipmentnn 30,816 logic cells30,816 logic cellsnn 138 18138 18--bit multipliersbit multipliersnn 2,448 kB of block RAM2,448 kB of block RAMnn PowerPc405 embedded PowerPc405 embedded processorprocessor

Virtex II Pro FPGA boardVirtex II Pro FPGA board

processorprocessor

EquipmentEquipment

nn Component, Component, composite and composite and SS--video inputsvideo inputs

nn II22C® compatible C® compatible control buscontrol bus

Video decoder Video decoder daughter boarddaughter board

control buscontrol busnn HighHigh--speed speed Hirose FX2 data Hirose FX2 data connectorconnector

Embedded development kit (EDK) flowEmbedded development kit (EDK) flow

EmbeddedDevelopment Kit

Instantiate the ‘System Netlist’

HDL Entry

Simulation/Synthesis

Standard FPGAHW Development Flow

VHDL or Verilog

System NetlistInclude the BSPand Compile the

Code Entry

C/C++ Cross Compiler

Standard EmbeddedSW Development Flow

C Code

Board SupportPackage

Data2MEM

Download CombinedImage to FPGA

Compiled ELF Compiled BIT

RTOS, Board Support Package

‘System Netlist’ and Implement

the FPGA

?

Implementation

Download BitstreamInto FPGA

Chipscope

and Compile theSoftware Image

?

Linker

Load SoftwareInto FLASH

Debugger

12 3 Compiled BITCompiled ELF

Xilinx Workshop

ObjectivesObjectives

What does the project do?What does the project do?What does the project do?What does the project do?

nn Design a realDesign a real--time video capture time video capture and image processing system using and image processing system using FPGAFPGA

nn System developmentSystem development

Project goalsProject goals

System developmentSystem developmentnn Video capture and realVideo capture and real--time displaytime displaynn Video capture and image processingVideo capture and image processing

System developmentSystem development

Video captureVideo capture

Video

Input

Video

OutputDigilent Video decoder

XUP Virtex II Proboard

Hirose socket connecto

XSGA Video Port

Video capture

Console

decoder daughter board

Composite signal

boardconnector

RS-232

nn Video inputVideo inputnn Analog cameraAnalog camera

nn Resolution : 512x492 pixelsResolution : 512x492 pixelsnn Video output format : compositeVideo output format : composite

nn Video outputVideo outputnn PC MonitorPC Monitor

Video Decoder Line Field Decoder

YCrCb to RGB Conversion

4:2:2 to 4:4:4Conversion

Buffer Control

Composite S-Video

Component (YPrPb)

Analog Interlaced Video

VDEC1 Board

Digital Video

ITU-R BT.656 YCrCb Data Format

XUP-V2Pro Development System

Video Timing Generation Logic

Video Timing Extraction

4:2:2 YCrCb

Create the missing Chroma data samples

4:4:4 YCrCb

Color Space Conversion

Hsync Vsync Blanking Pixel

4:4:4 RGB

Select the active line buffer for READS

@27MHz

Select the active line buffer for WRITES

@13.5MHzBuffer Control Logic

Line Buffer (BRAM)

Line Buffer (BRAM)

Video DAC

Monitor

I2C Master

24 Bit RGB Video Data

Select the video source

Blanking Pixel Clock

MultiplexerMultiplexer

XSGA Video Port

De- interlace by Line doubling

Analog Processive Video 858x525 pixels @ 60Hz

SpecificationsSpecifications

nn Developed in VHDL using EDK tools Developed in VHDL using EDK tools

nn C language is used to configure the video C language is used to configure the video decoder decoder

nn Supports NTSC and PAL inputsSupports NTSC and PAL inputs

nn Output format (NTSC) : 858x525 pixels Output format (NTSC) : 858x525 pixels @60 Hz refreshing rate@60 Hz refreshing rate

System developmentSystem development

Image processingImage processing

Video

Input

Video

OutputDigilent Video

Composite

XUP Virtex II Proboard

Hirose socket connect

XSGA Video Port

Image processing

Console

Video decoder daughter board

Composite signal board

connector

Video Port

RS-232

Image processing

1.1. BrighteningBrightening2.2. SharpeningSharpening3.3. BlurringBlurring4.4. Edge detectionEdge detection

Image processingImage processing

4.4. Edge detectionEdge detection1.1. Sobel x filterSobel x filter2.2. Sobel y filterSobel y filter3.3. Sobel xy filterSobel xy filter

nn Designed an embedded system to Designed an embedded system to configure and drive the video decoderconfigure and drive the video decoder

nn Developed video capture functions such as Developed video capture functions such as data buffer and color space conversionsdata buffer and color space conversions

Developed a function to drive video DAC Developed a function to drive video DAC

Work doneWork done

nn Developed a function to drive video DAC Developed a function to drive video DAC and display the image on the monitor and display the image on the monitor through XSGA portthrough XSGA port

nn Implemented image processing algorithms Implemented image processing algorithms

ResultsResults

ResultsResults

Original imageOriginal image Brightened imageBrightened image

ResultsResults

Sharpened imageSharpened image Blurred imageBlurred image

ResultsResults

Edge detected imageEdge detected image

nn Medical imagingMedical imaging

nn Digital cinemaDigital cinema

nn SurveillanceSurveillance

Applications Applications

nn SurveillanceSurveillance

nn Machine intelligenceMachine intelligence

nn High definition TVHigh definition TV

nn Implemented realImplemented real--time video capture systemtime video capture systemnn Successfully applied edge detection and other Successfully applied edge detection and other

image processing algorithmsimage processing algorithmsDisplayed realDisplayed real--time video data on monitortime video data on monitor

Conclusion Conclusion

Completed

nn Displayed realDisplayed real--time video data on monitortime video data on monitor

nn Saving video data in memorySaving video data in memorynn Advanced image processing algorithms could be Advanced image processing algorithms could be

studied on the system for different applications. studied on the system for different applications.

Future Work

BibliographyBibliography[[11]Xilinx]Xilinx IncInc..,, ““XilinxXilinx UniversityUniversity ProgramProgram VirtexVirtex IIII ProProDevelopmentDevelopment SystemSystem HardwareHardware ReferenceReference Manual”Manual”,, MarchMarch20052005..

[[22]Xilinx]Xilinx IncInc..,, ““DigilentDigilent VideoVideo DecoderDecoder BoardBoard (VDEC(VDEC11)) ReferenceReferenceManualManual”,”, AprilApril 20052005..

[[33]Christophe]Christophe DesmouliersDesmouliers.. ““RealReal--TimeTime FPGAFPGA--BasedBased EmbeddedEmbedded[[33]Christophe]Christophe DesmouliersDesmouliers.. ““RealReal--TimeTime FPGAFPGA--BasedBased EmbeddedEmbeddedVideoVideo CapturingCapturing andand VideoVideo ProcessingProcessing”,”, ProjectProject Report,Report,IllinoisIllinois InstituteInstitute ofof Technology,Technology, AugustAugust 20072007..

[[44]] XilinxXilinx IncInc..,, “Xilinx“Xilinx EDKEDK Workshop”,Workshop”, MayMay 20082008

Questions?Questions?

Thank you!Thank you!

nn Dr. Yufeng Lu Dr. Yufeng Lu nn Dr. In Soo AhnDr. In Soo Ahn

Thank youThank you

nn Dr. In Soo AhnDr. In Soo Ahn