Raid-II: A High-Bandwidth Raid-II: A High-Bandwidth Network File ServerNetwork File Server

Ann L. Drapeau, Ken W. Shirriff, John H. Ann L. Drapeau, Ken W. Shirriff, John H. Hartman, Ethan L. Miller, Srinivasan Hartman, Ethan L. Miller, Srinivasan

Seshan, Randy H. Katz, Ken Lutz, David A. Seshan, Randy H. Katz, Ken Lutz, David A. Patterson, Edward K. Lee, Peter M. Chen, Patterson, Edward K. Lee, Peter M. Chen,

Garth A. GibsonGarth A. Gibson

IntroductionIntroduction

• Why do we need RAID-II?Why do we need RAID-II?– Many new applications require larger Many new applications require larger

amounts of data.amounts of data.•Computer Aided DesignComputer Aided Design•DatabasesDatabases•Multi-mediaMulti-media•Scientific SimulationsScientific Simulations

– Raid-I is unable to provide the needed Raid-I is unable to provide the needed I/O bandwidth.I/O bandwidth.

The Need For SpeedThe Need For Speed

• RAID-IRAID-I– Redundant Arrays of Inexpensive DisksRedundant Arrays of Inexpensive Disks– Performs well when processing small random I/O Performs well when processing small random I/O

requests.requests.– Inadequate for high-bandwidth I/O requests.Inadequate for high-bandwidth I/O requests.

• Coping data between kernel DMA buffers and buffers in Coping data between kernel DMA buffers and buffers in user space saturate the memory system when I/O user space saturate the memory system when I/O bandwidth reaches 2.3 mb/s.bandwidth reaches 2.3 mb/s.

• Data must go through the CPU’s virtual addressed Data must go through the CPU’s virtual addressed cache and cache flushes slow the performance.cache and cache flushes slow the performance.

• Bandwidth restricted by the VME system bus (9 mb/s).Bandwidth restricted by the VME system bus (9 mb/s).

Raid-IIRaid-II

• RAID-II contains two data pathsRAID-II contains two data paths– High-bandwidth (HIPPI)High-bandwidth (HIPPI)

•Handles large data transfers.Handles large data transfers.

– Low-bandwidth (Ethernet)Low-bandwidth (Ethernet)•Handles metadata and small data transfers.Handles metadata and small data transfers.

– LFS – Log Structured File SystemLFS – Log Structured File System

High-Bandwidth Data PathHigh-Bandwidth Data Path

• XBUSXBUS– Allows large data requests to bypass the Allows large data requests to bypass the

server workstation.server workstation.– Connects the disks directly to the high Connects the disks directly to the high

speed HIPPI network.speed HIPPI network.•The HIPPI network is the high speed network The HIPPI network is the high speed network

with a transfer rate of 100MB/s.with a transfer rate of 100MB/s.• It takes about 1.1 milliseconds to transfer a It takes about 1.1 milliseconds to transfer a

HIPPI packet. (Due to setup of HIPPI & XBUS HIPPI packet. (Due to setup of HIPPI & XBUS control registers)control registers)

Low-Bandwidth Data PathLow-Bandwidth Data Path

• Uses Ethernet capable of 10 Mb/s = Uses Ethernet capable of 10 Mb/s = 1.25 MB/s.1.25 MB/s.

• Used to transfer the metadata and Used to transfer the metadata and small data requests.small data requests.– It takes about 0.5 milliseconds to It takes about 0.5 milliseconds to

transfer a packet on Ethernet. (HIPPI transfer a packet on Ethernet. (HIPPI takes 1.1 ms)takes 1.1 ms)

RAID-IIRAID-II

XBUS Controller BoardXBUS Controller Board

Raid-II file system Raid-II file system

• LFS = Log Structured File SystemLFS = Log Structured File System– Writes all file data and metadata to an Writes all file data and metadata to an

append only log file.append only log file.– This minimizes the number of small This minimizes the number of small

writes to the disk, by buffering the writes to the disk, by buffering the writes and then writing a single large writes and then writing a single large sequential file to the disk.sequential file to the disk.

– Also provides fast crash recovery.Also provides fast crash recovery.

Raid-II file systemRaid-II file system

• Modified LFSModified LFS– Must be able to separate requests Must be able to separate requests

among the high and low bandwidth data among the high and low bandwidth data paths.paths.

– Manage the memories on the host Manage the memories on the host workstation and the XBUS board.workstation and the XBUS board.•Cache coherencyCache coherency

ConclusionsConclusions

• RAID-II allows data to be transferred RAID-II allows data to be transferred between disks and the HIPPI network between disks and the HIPPI network without going through the low-without going through the low-bandwidth memory by using the bandwidth memory by using the XBUS controller cards.XBUS controller cards.

• RAID-II uses LFS to buffers small RAID-II uses LFS to buffers small writes into large requests to avoid writes into large requests to avoid wasting disk bandwidth.wasting disk bandwidth.

¿¿Questions?Questions?

• What was the purpose of designing What was the purpose of designing the XBUS controller board?the XBUS controller board?

• Describe the two network Describe the two network connections used and their purposes.connections used and their purposes.

• What additional ideas could be What additional ideas could be applied to RAID-II to improve applied to RAID-II to improve performance?performance?