VchatEE4214 Project Presentation

OUTLINE

• About the Project

• Design Process

• Demo

• Technical Details

• Future Improvements

• Learning Points

• Team and Task Distribution

• Q & A

Key Features

• Hands-free

• 2-way Audio communication

• Automatic login with facedetection

• Simple and user friendlyinterface

About the Project

Vchat :Audio based public chat terminalwith automated login

Applications

• Public Terminals

• Elderly care

• Hospital Use

• Company Network

• Virtual Window

System Design – Initial Concept

Challenges Faced:• USB Drivers

• Spartan 3 too weak• 8-bit colour

• Slow

Initial Design Concept• Interface two Spartan 3E

• 2-way Video

• USB Webcams

• Keyboard based chatting

• Automatic login

• Interface with USB webcam

System Design - Final

Final Design• Interface two Spartan 6• 2-way Audio• HDMI-in Camera• Automatic login• High resolution video• Hardware Accelerated

• Data manipulation, Logic separation

Demo

Technical Details

HDMI Microblaze 1

Microblaze 0

VDMA

AC97 Audio Codec

Ethernet Lite

Face Detected Audio ReceivedVideo In/Out

Audio In/Out

Data In/Out

- Face Detection/Tracking- Graphic User Interface

- Audio Processing- Communication

VMDA = Video Direct Memory Access

System Architecture

System Architecture

AXI = Advanced eXtensible Interface

Data Corruption:• Microblaze has to sample

a frame for face detection.• HDMI Read/Write speed

>> Microblaze.• HDMI In refreshes frame

at constant rate (60fps), and should not be stopped.

HDMI

Video In/Out (60fps)

Microblaze 1(Face Detection)

RAM

Frame

(5-10fps)

VDMA

HDMI & GUI

VMDA = Video Direct Memory Access

HDMI

Video In/Out (Fast)

RAM

Frame Frame

Frame Frame

Normal

Microblaze 1(Face Detection)

RAM

Frame

(Slow)

Frame

Frame Frame

Sampling

HDMI

Video In/Out (Fast)

HDMI & GUIFrame Sampling with Data Corruption Prevention

RAM128MB

Pixel Extraction

Load at Initialization

bmp

Microblaze 1

HDMI Frame

HDMI & GUI

Face DetectionOriginal Skin Color

Erode Noise

Fill Holes

Face Detected

Nguyen, T. T. (2012). Real-time Face Detection and Tracking. Master of Engineering Thesis, School of Electrical and Computer Engineering, Cornell University.

Result

Audio & Ethernet

AC97 Controller

Mic

Line Out

LM4550AC-97Audio Codec

Microblaze 0

Spartan-6

Ethernet lite

ConnectorMarvell

M88E1111Microblaze 0

Spartan-6

Hardware Software

EthernetEthernet Communication Scheme:• Send: Implicit naming, non-blocking (broadcast message)• Receive: Implicit naming, non-blocking (receive message from everyone), but

check for “valid” address and message.

Ethernet frame, Destination MAC Address = 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF

12

3 4

ACK: Node 1 Online

ACK: Node 1 Online

Transmitting

BufferAudio In

Ethernet Full

Microblaze 0

N samples

1 packet = N samples

Tx Data

44 kHz

44 kHz

Receiving

Buffer

Buffer

Buffer

…Audio Out

Ethernet

At least 1 empty

Microblaze 0

Rx DataFilter

Audio & Ethernet

𝐻 𝑧 =1

4(1 + 2𝑧−1 + 𝑧−2)

Audio Transmitting Receiving

Time

Receiving

Nondeterministic

Time

e.g. Buffer N = 2

Audio Cycle(44kHz)

Transmitting every N Audio Cycles

Transmitting

Deterministic

Audio & Ethernet

• Audio In/Out: Fixed rate at 44kHz. • It is more efficient to send multiple samples in 1 Ethernet packet.• Transmitting: deterministic, the processor know when the buffer is full to

broadcast data away → fixed rate and synchronous with audio.• Receiving: nondeterministic, incoming data can arrive at any time, not

synchronous with audio → multiple buffers. Moreover, data can come from multiple senders.

Audio & Ethernet

Schedule

Time

TimeData Arrive (Interrupt)

Audio Cycle

Audio InAudio Out Data Tx

Data Rx

Real Time Requirements

• Audio & Ethernet

• Audio in, Audio out, Ethernet Tx, and Ethernet Rx scheduling.

• Data buffering.

• Video

• Data corruption prevention.

• DDR2 RAM

• Shared memory access (hardware).

• Dual Microblaze

• Producer/Consumer.

Future Improvements

• Implementation of TCP/IP protocolfor scalable network.

• Face and voice recognition forautomated login with personalinformation

• Machine learning software.

• More powerful hardware.

• 2-way video capabilities

• Faster microprocessor (ARM + FPGA).

• HDL implementation of video codec.

Learning Outcomes

• Complete system implementation

• Application of concepts

• Data and logic separation forhardware optimized processes

• Parallel processing

• Dealing with time-critical IO andreal-time deadlines

• Team work is essential

Team and Task Distribution

Anh Tuan Nguyen A0074465Y System Design & Integration.

Dao Duc Huan A0030591N Face Detection Algorithm.

Huynh Van Vinh A0055979H Audio to Ethernet Communication.

Rahul Rajeev A0077497J Interfacing with peripherals:• HDMI (PLB Bus).• Audio Interfacing.• Keyboard Controller (Spartan3)

User Interface Design:• Image to Pixel-Array

Shambavi Krishnamurthi

A0077496L

Ziyi Wang A0091881X

Thank YouProf. Akash and Chin Hau

Q & A