PCI BASED READ-OUT RECEIVER CARD IN THE

ALICE DAQ SYSTEMW.Carena1, P.Csato2, E.Denes2, R.Divia1,

K.Schossmaier1, C. Soos1, J.Sulyan2, A.Vascotto1, P.Vande Vyvre1

1CERN EP/AID, 2KFKI-RMKI (Budapest)

NEC 2001Varna, Bulgaria

12 – 18 September, 2001

NEC2001, 12-18 September, 2001

2PCI based read-out receiver card

OUTLINEOUTLINE

ALICE DAQ system DDL components DDL requirements PCI-RORC concept PCI-RORC development PCI-RORC performance Summary

NEC2001, 12-18 September, 2001

3PCI based read-out receiver card

ALICE DAQ SYSTEMALICE DAQ SYSTEM

NEC2001, 12-18 September, 2001

4PCI based read-out receiver card

DDL COMPONENTSDDL COMPONENTS

Destination Interface Unit (DIU)

Source Interface Unit (SIU) Physical medium

• Multimode optical cable• Maximum 200m long

Front-EndElectronics(FEE)

Front-End Electronics

(FEE)

Source Interface

Unit (SIU)

Read-OutReceiverCard(RORC)

Forward Channel

Backward Channel

DDL Hardware = Source Interface Unit + Physical Medium + Destination Interface Unit

Physical Medium

Destination Interface

Unit (DIU)

Read-out Receiver

Card (RORC)

NEC2001, 12-18 September, 2001

5PCI based read-out receiver card

DDL REQUIREMENTSDDL REQUIREMENTS

Bi-directional data transfer• 100 MB/s from the detectors• 10 MB/s to the detectors

Detected BER must be less than 10-15

Remote control capability• FEE control• SIU control

Detailed status and error reporting

Built in test capability JTAG BST over the link

• FEE control and test

NEC2001, 12-18 September, 2001

6PCI based read-out receiver card

PCI RORC CONCEPTPCI RORC CONCEPT

Interface between the DIU and PCI local bus• 32bit/33Mhz PCI version, max. throughput 132MB/s

Direct data transfer to the PC memory• No local memory on the board• Small elasticity buffers between different clock domains

Data push architecture, PCI master operation• Data transfer with minimal software interaction• Minimize latency

Memory management• Efficient for scattered memory management• Minimize software overhead

Built-in test capability• Internal pattern generator can produce quasi-realistic

data

NEC2001, 12-18 September, 2001

7PCI based read-out receiver card

PCI RORC DEVELOPMENT 1/2

PCI RORC DEVELOPMENT 1/2

IMB

OMB

PCI to AOL FIFO

AOL to PCI

FIFO

AO

L in

terf

ace

Read DMA controller

Write DMA controller

Memory manager

and command interprete

r DIU

in

terf

ace

Receiver FIFO

Transmitter FIFO

Pattern Generator

PCI DIU

pRORC firmware architecture

AOL – Add-on Logic

Output Mailbox

Input Mailbox

NEC2001, 12-18 September, 2001

8PCI based read-out receiver card

BA BL IDXBA BL IDX

BA BL IDX

PCI RORC DEVELOPMENT 2/2

PCI RORC DEVELOPMENT 2/2

BA BL IDX

RFBA IDX

Free FIFO (128

entry)

PC

ph

ysic

al m

em

ory

BA – Base Address

BL – Block Length

IDX – Ready FIFO Entry Index

RFBA – Ready FIFO Base Address

Ready FIFO

Scattered Memory Management

NEC2001, 12-18 September, 2001

9PCI based read-out receiver card

PCI RORC PERFORMANCE 1/2

PCI RORC PERFORMANCE 1/2

Test firmware developed to test the DMA• Fixed size block transferred (n Word)• Simple test pattern used (without data check)

Test software controls the firmware and measures the performance• Stand alone operation

Base address

Base address + 4

Block Counter

Data

bu

ffer

Base address + 4(n+1)

020406080

100120140

Data length [Byte]

Ban

dwid

th [

MB

/s]

DMA Write

NEC2001, 12-18 September, 2001

10PCI based read-out receiver card

PCI RORC PERFORMANCE 2/2

PCI RORC PERFORMANCE 2/2

Same test firmware and software used Additional software components

• Intensive memory usage (stream_l benchmarking tool)• Intensive network usage (Gigabit Ethernet)• Intensive CPU usage (stream_l + DATE1)

Scenario DMA Rate2

[MB/s]Memory [MB/s]

Networks [MB/s]

Components CPU Usage

1 125 - - pRORC DMA (0%)

2 - 360 - stream_l (100%)

3 - - 40 iperf (100%)

4 113 310 - pRORC DMA (0%) + stream_l (100%)

5 80 - 21 pRORC DMA (0%) + iperf (66%)

6 80 180-210 19-20 pRORC DMA (0%) + DATE (66%) + stream_l (100%)

1 DATE – Data Acquisition and Test Environment 2 400 KB block size

NEC2001, 12-18 September, 2001

11PCI based read-out receiver card

SUMMARYSUMMARY

Results:• 5 boards are available for prototyping• Software drivers for 2.2.x and 2.4.x kernels• Software library for integration with DAQ system

Future plans:• Migrate to PCI 64bit/66MHz• Implement on-board memory and pre-processing

capabilities• Develop WEB-based GUI

NEC2001, 12-18 September, 2001

12PCI based read-out receiver card

NEC2001, 12-18 September, 2001

13PCI based read-out receiver card

PCI RORC BOARDPCI RORC BOARD

NEC2001, 12-18 September, 2001

14PCI based read-out receiver card

DDL DIU BOARDDDL DIU BOARD

NEC2001, 12-18 September, 2001

15PCI based read-out receiver card

DDL SIU BOARDDDL SIU BOARD