DIS PROPOSAL - Distributed Data Warehouse -

R92725032 蔣孟儒 R92725042 龍秋明

Outline

Motivation and Goals System Architecture diagram Justification Alternatives System Requirement Testing Timeline Job distribution

Motivation Solve the problem of data access on multiple

computers Support virtual file serving

Goals Provide distributed ftp file sharing Dynamic join and leave of service node Provide corresponding mechanism for server

crash.

System Architecture and Diagram

FTP Client FTP ClientFTP ClientFTP ClientR

FC 9

59

RFC

959

FTP Server

Virtual File System

Local File System Remote File System

FTP Server

Virtual File System

Local File System Remote File System

FTP Server

Virtual File System

Local File System Remote File System

Remote Method Invocation

RFC

959

RF

C 9

59

File Service Pool

Server list

Fault tolerance

Description Server side

FTP Server Accept client’s request Invoke remote file system API or directly call local file

system Return files requested by client through FTP

Remote File System Use RMI to communicate with remote file system of

others’ service node. Join service pool:

-Broadcast its own address to all service nodes inside service pool, all nodes,

which maintain a server list, update its own server list and construct RMI connection with the new node.

Description Client Side

Login the system using common ftp client application

Get a global view of file service pool Receive file transited by the system(the same as normal ftp client application)

Justification File Location Transparency

Provide a file system with global view using J2SE

Failure Handling When service crash, system

automatically unmount remote file system and update directory

Scalability Provide service node join or exit our

system dynamically

Alternatives

File Server

Virtual File System

Local File SystemRemote File System

Client ClientClientClient

File Server

Virtual File System

Local File System Remote File System

File Server

Virtual File System

Local File System Remote File System

Remote Method Invocation

File Service Pool

Hash function(SHA-1)Hash function(SHA-1)Hash function(SHA-1)

Alternatives (cont) FTP server

original: Let FTP Server call remote file system of sub-layer, and use sub-layer to handle data transmission

other way: Use upper-layer FTP to connect Service Nodes

original: Use broadcast to let service node dynamically join the service pool

efficient way: Consider to add a coordinator as a entry point ， and manage all of the service nodes

System Requirement and Implementation

Requirement Hardware

4 computers(3 servers and 1 client) Software

Platform: Java VM on Windows or Linux Middleware: Java RMI

Implementation J2SE(JDK1.4) Borland JBuilder, eclipse

Testing

Join & leave testing Service node failure handling

Implementation & Timeline

ID 工作名稱 開始日期 結束日期 期間2003 2004

十一月 十二月 一月

1System analysis

2003/11/1

2003/11/8

1W

2 System design2003/11/

92003/11/

161W

3System development

2003/11/17

2003/12/29

6W

4 System testing2003/12/

302004/01/

203W

Job Distribution

龍秋明： Virtual file system and FTP server daemon

蔣孟儒： Remote file system and service pool management mechanism