Characterization Presentation Characterization Presentation

Neural Network Neural Network Implementation On Implementation On

FPGAFPGASupervisor: Chen KorenSupervisor: Chen Koren

Maria Nemets 309326767Maria Nemets 309326767Maxim Zavodchik 310623772Maxim Zavodchik 310623772

IntroductionIntroduction

Neural Network is a sort of artificial Neural Network is a sort of artificial intelligenceintelligence

NN is an imitation of a human body NN is an imitation of a human body A NN acquires knowledge through learning. A NN acquires knowledge through learning. NN can learn in real timeNN can learn in real time Applications: Applications:

Optical Characters RecognitionOptical Characters Recognition Voice RecognitionVoice Recognition Financial ForecastingFinancial Forecasting Medical DiagnosisMedical Diagnosis

Theoretical BackgroundTheoretical Background

Neuron Model (Perceptron):Neuron Model (Perceptron): The single neuron is a device that implements

the function Activate function – some bounded

continuously rising function (for example, sigmoid).

X is an input vector W is a weight vector

Knowledge is stored within inter-neuron connection strengths

01

_ ( )m

j jj

Activate func w x w

Learning of the neural Learning of the neural networknetwork

Popular implementation is a multilayer feed forward network

Neural network learns from examplesNeural network learns from examples Learning is changing weights to implement the Learning is changing weights to implement the

desired classification functiondesired classification function There are algorithms which change the weights There are algorithms which change the weights

in the networkin the network We will use Back Propagation learning algorithmWe will use Back Propagation learning algorithm

Minimizes the difference between output and training Minimizes the difference between output and training example.example.

Weight changes depend on the units in the layer below.Weight changes depend on the units in the layer below.

Project ObjectivesProject Objectives

Implementing Neural Network on FPGAImplementing Neural Network on FPGA Creating modular designCreating modular design Implementing in software (Matlab)Implementing in software (Matlab) Creating PC InterfaceCreating PC Interface Performance Analyze:Performance Analyze:

Area on chipArea on chip InterconnectionsInterconnections Speed vs. software implementationSpeed vs. software implementation FrequencyFrequency CostCost

Development StagesDevelopment Stages Choosing application: 0-9 Digits RecognitionChoosing application: 0-9 Digits Recognition Implementing a single neuron in VHDLImplementing a single neuron in VHDL Determining network structure Determining network structure Building NN in Matlab, training and testing itBuilding NN in Matlab, training and testing it Determining weights length and resolutionDetermining weights length and resolution Choosing the proper hardware: FPGA and Choosing the proper hardware: FPGA and

MemoryMemory Planning FSM controllerPlanning FSM controller Implementing the NN in VHDL Implementing the NN in VHDL Simulation and SynthesisSimulation and Synthesis Performance analysis: hardware vs. software Performance analysis: hardware vs. software

System InterfaceSystem Interface

InputsInputs Binary image ( 8x8 pixels )Binary image ( 8x8 pixels ) Weights – The size will be determined Weights – The size will be determined Getting inputs by blocksGetting inputs by blocks

OutputsOutputs Vector size of 4 bits coding the digit or Vector size of 4 bits coding the digit or

indicating a failureindicating a failure

Neuron ArchitectureNeuron Architecture

Multipliers for calculatingMultipliers for calculating Adder for calculating theAdder for calculating the ROM implementing Activation functionROM implementing Activation function SRAM saving the weightsSRAM saving the weights SRAM saving the input imageSRAM saving the input image FSM ControllerFSM Controller

j jw x

01

m

j jj

w x w

Neuron ArchitectureNeuron Architecture

Hardware RequirementsHardware Requirements

Memec Development BoardMemec Development Board Virtex II Pro FPGAVirtex II Pro FPGA

44 18x18-bit Multipliers44 18x18-bit Multipliers 88 KB of on-chip memory88 KB of on-chip memory Power PC 4.05 CPU corePower PC 4.05 CPU core

32 MB of SDRAM32 MB of SDRAM 100MHz & 125 MHz clocks100MHz & 125 MHz clocks 2x16 LCD panel2x16 LCD panel Serial port interfaceSerial port interface

Hardware RequirementsHardware Requirements

Load Weight

s

To SDRA

M

Load Image From User

Load Weight

s

To BRAM

Calculate Intermedia

te Sum

Calculate

Output

Return Result

To User

Special ProblemsSpecial Problems Problem:Problem: NN interconnection complexity NN interconnection complexitySolution:Solution: Designing sub-networks for each digit Designing sub-networks for each digit

Problem:Problem: Numbers representation Numbers representationSolution:Solution: Fixed-Point representation avoiding Floating Fixed-Point representation avoiding Floating Point operations Point operations

Problem:Problem: Determining accuracy Determining accuracy Solution:Solution: Choosing proper resolution from Matlab Choosing proper resolution from Matlab simulationssimulations

Problem:Problem: Trade-off between Area and Concurrency Trade-off between Area and Concurrency Solution:Solution: Searching for big FPGA which enables Searching for big FPGA which enables maximal parallelizing maximal parallelizing

Special ProblemsSpecial Problems Problem:Problem: Modulation vs. Speed Modulation vs. Speed

Solution:Solution: Simulating the hardware NN and finding the Simulating the hardware NN and finding the trade-offtrade-off

Problem:Problem: Large Memory size Large Memory size

Solution:Solution: Finding FPGA with big memory or using outside Finding FPGA with big memory or using outside

memory coresmemory cores

Problem:Problem: Sigmoid function is continuous and infinite Sigmoid function is continuous and infinite

Solution:Solution: - Function quantization according - Function quantization according

to our resolution to our resolution

- Saving values for arguments in limited range - Saving values for arguments in limited range

ScheduleSchedule

Date Assignment31.10-

5.11Learning the theoretical backgroundTraining and testing an example NN in

Matlab

7.11-12.11

Implementing a single Neuron in VHDL, synthesizing and checking its area

14.11-19.11

Finding suitable Hardware solution (Area, Finding suitable Hardware solution (Area, Memory)Memory)

17.11 Characterization Presentation

21.11-3.12

Building training setBuilding our NN in Matlab, training and

testingDetermining weight length and needed

resolution

5.12-10.12

Determining NN structureDetermining NN structure

Proposing final architecture Proposing final architecture

ScheduleSchedule

Date: Assignment12.12-17.12

Implementing Sigmoid functionImplementing Sigmoid functionPlanning the FSM ControllerPlanning the FSM Controller

19.12-7.01

Implementing the NN in VHDL

20.1220.12 Midterm presentation

9.01-21.01

Simulating the NN Synthesis

23.01-28.01

Analyzing performanceWriting project book

21.0221.02 Final presentation