Chapter 20 Distributed File Systems Copyright © 2008.

Chapter 20

Distributed File SystemsCopyright © 2008

Operating Systems, by Dhananjay Dhamdhere Copyright © 2008 20.2Operating Systems, by Dhananjay Dhamdhere 2

Introduction

• Design Issues in Distributed File Systems• Transparency• Semantics of File Sharing• Fault Tolerance• DFS Performance• Case Studies


Design Issues in Distributed File Systems


Overview of DFS Operation

• Remote file processing model

• File server agent and client agent are analogous to RPC’s stub processes

• For efficiency, the client agent and the cache manager are typically rolled into a single unit


Transparency

• In a conventional file system, a user identifies a file through a path name– User is aware that file belongs in a specific directory, but

is not aware of its location in the system

• Location info field of the file’s directory entry indicates the file’s location on disk

• Location transparency can be provided in a DFS through a similar mechanism– Location info: (node id, location)

• Location independence requires information in location info field to vary dynamically


Semantics of File Sharing

• Semantics determine manner in which effect of file manipulations performed by concurrent users of a file are visible to one another


Semantics of File Sharing (continued)

• A session consists of some clients of a file that are located in the same node of a system

• Problem with session semantics: poor portability• Session semantics are easy to implement in a DFS

employing file caching– File changes are not visible to clients in other nodes


Fault Tolerance

• File system reliability has several facets:– A file must be robust, recoverable, available

• Robustness is achieved using techniques for reliable storage of data

• Robustness and recoverability depend on how files are stored and backed up, respectively

• Availability depends on how files are opened and accessed

• Only defense against client node crashes is use of transaction semantics in file server


Fault Tolerance (continued)


Availability

• File is available if a copy can be opened and accessed by client– Ability to open file depends on path name resolution

– Access requires functional client and server nodes

• An anomalous situation may arise when path names span many nodes– If a node in path crashes, file operation will fail even if the

node that contains the file has not crashed

• Solution: cached directories

• File replication is transparent to clients– Updating techniques: 2PC, use of primary copies


Client and Server Node Failures

• File server can maintain FCBs and OFT in memory– Stateful design

– Good performance

– Problems in event of client and server crashes

• Solution: client and file server share a virtual circuit– Virtual circuit “owns” the file processing actions and

resources like file server metadata

– Actions and resources become orphans after crash• Actions are rolled back and metadata destroyed

– Client–server protocol implementing transaction semantics may be used to ensure this


Stateless File Servers

• File server does not maintain state information about file processing activity

• Client must:– Keep state information about file processing activity

– Provide all relevant information in a file system call• read (“alpha”, <record/byte id>, <io_area address>);

• Many actions traditionally performed only at file open time are repeated at every file operation

• If file server crashes, time-outs and retransmissions occur in client

• Cannot employ file caching


DFS Performance

• DFS design is scalable if DFS performance doesn’t degrade with increase in size of distributed system


Efficient File Access

• Inherent efficiency of file access depends on how the operation of a file server is structured

• Two server structures that provide efficient file access:– Multithreaded file server

– Hint-based file server• State information is used as a hint• Server operation is stateless if hint is not available


File Caching

• File cache and copy of file on disk in server node form a memory hierarchy– Operation of the file cache and its benefits are analogous

to those of a CPU cache

• Chunks of file data are loaded from the file server into the file cache

• Studies of file size distributions indicate small average file size– Whole-file caching is feasible

• File server may use a separate attributes cache


File Caching (continued)

• Key issues:– Location of the file cache: memory or disk

– File updating policy: write-through or delayed write

– Cache validation policy: client- or server- initiated

– Chunk size: large or small? Fixed or variable?


Scalability

• DFS scalability achieved through techniques that localize most data traffic generated by file processing activities within clusters– Clusters typically represent subnets like high-speed LANs

– An increase in the number of clusters does not lead to degradation of performance

• It does not add much network traffic


Case Studies

• Sun Network File System• Andrew and Coda File Systems• GPFS• Windows


Sun Network File System

• VFS implements mount protocol and creates a system-wide unique vnode for each file

• NFS layer interacts with remote node containing file through NFS protocol


Sun Network File System (continued)

• Several techniques to improve performance– A directory names cache is used in each client node

– A file attributes cache caches inode information• Cached attributes are discarded after 3 seconds for files

and after 30 seconds for directories

– File blocks cache is the conventional file cache• Server uses large (8 Kbytes) data blocks• Cache validation performed through timestamps associated

with each file, and cache block

• File server is stateless• Neither Unix semantics nor session semantics


Andrew and Coda File Systems

• Targeted at gigantic distributed systems• All clients have an identical shared name space

– Is location transparent in nature

– Implemented by dedicated servers (Vice)

• Clusters localize file processing activities– Traffic within cluster reduced by caching entire file on

local disk

• A volume typically contains files of a single user• 64 KB chunks (size adapted on a per-client basis)• User process called Venus performs open/close


Andrew and Coda File Systems (continued)

• Server-initiated cache validation using callbacks• Path name resolution performed on a component-by-

component basis– Venus maintains a mapping cache

• File servers are multithreaded• Client–server communication uses RPCs• Two features to achieve high availability:

– Replication and disconnected operation• Read one, write all policy• Supports hoarding of files


GPFS

• General parallel file system: high-performance shared-disk file system– For large computing clusters operating under Linux

• Uses data striping across all disks in cluster– A large-size block (strip) used to minimize seek overhead

during a file read/write• A smaller subblock is used for small files

– Locking used to maintain consistency of file data• Lock granularity is as coarse as possible, but as fine as

necessary• Centralized lock manager and few distributed lock

managers


GPFS (continued)

• Notion of lock tokens to reduce latency and overhead of locking

• Race conditions may arise over metadata of a file– Solution: one of the nodes is designated as the metanode

for the file; it performs file updates

• Central allocation manager partitions free space map and gives one partition to each node

• Each node writes a separate journal for recovery• If network is partitioned, only nodes in the majority

partition can perform file processing at any time


Windows

• Windows Server 2003 provides two features for data replication and data distribution:– Remote differential compression (RDC)

– DFS namespaces

• Replication organized using notion of a replication group

• DFS namespace is created by a system administrator• Other key concepts: referrals and hot standbys


Summary

• Transparency concerns association between path name of a file and location of the file

• File sharing semantics may differ between DFSs:– Unix semantics

– Session semantics

– Transaction semantics (atomic transactions)

• Stateless server design provides high availability– Notion of a hint used to improve performance

• DFS uses file caching to improve performance– Cache coherence techniques are needed

Chapter 20 Distributed File Systems Copyright © 2008.

Documents

Transcript of Chapter 20 Distributed File Systems Copyright © 2008.