Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

55
Operating Systems Unit 10: Networking – Distributed File Systems Operating Systems

Transcript of Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

Page 1: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

Operating Systems

Unit 10:– Networking– Distributed File Systems

Operating Systems

Page 2: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 2

Networking concepts

• host– single computer– local CPU, memory, disks

• link– connecting media– attributes:

• bandwidth, latency

Page 3: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 3

Network Topologies

Page 4: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 4

Communication Protocols

Data Transport

Application

Page 5: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 5

Data Transport layers

4: Transport layer• End-to-end communication• Relies on network layer to determine proper

path from one end of communication to the other

3: Network layer• Moving data between computers

2: Link layer• Provides an interface between the network layer

and the underlying physical medium of the connection

1: Physical layer

Page 6: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 6

2: Link Layer

• first software layer on top of physical medium

• unit: frame• responsibilities:

– deliver frame – detect errors– correct transmission errors

Page 7: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 7

2: IEEE 802.3 CSMA/CD (Ethernet)• Carrier Sense Multiple Access

with Collision Detection protocol– transceiver tests delivery medium for

availability• due to delays in medium, it is possible that multiple

transceivers begin transmitting simultaneously

– if transceivers detect collision:• continues to transmit bytes for a specific period of

time to ensure that all transceivers become aware of the collision

• waits for a random interval before attempting to retransmit

Page 8: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 8

2: Token Ring

• operates on ring networks • employs tokens to gain access to

the transmission medium

• token is empty frame that is circulated between machines

Page 9: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 9

2: Token Ring example scenario 1/2

Page 10: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 10

2: Token Ring example scenario 2/2

Page 11: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 11

2: Fiber Distributed Data Interface (FDDI)

• operates over fiber-optic cable– support more transfers at greater

speeds over larger distance

• built on two token rings– the second usually being reserved for

backup

Page 12: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 12

2: IEEE 802.11 (Wireless)

• similar to Ethernet:– Carrier Sense Multiple Access

with Collision Avoidance (CSMA/CA)– require each sender broadcast a

Request to Send (RTS) to entire network

– upon receiving an RTS• receiver broadcasts a Clear to Send (CTS)

message to the entire network if the medium is available

Page 13: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 13

3: Network Layer

• routes packet to next host toward destination

• determines next host– address information– network topology– link quality

• strength of signal, error rate and interference– Interference is broadcast throughout networks

using various router protocols, such as Routing Information Protocol (RIP)

Page 14: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 14

3: Internet Protocol (IP or IPv4)

• hosts have unique IP address– 32-bit number written as quad– network

• first n bits of IP number, written as “/n”• 8 - class A, 16 - class B, 24 - class C• more than 24 - class D

– netmask• 32 bit number with first n bits all 1, rest 0

– broadcast• network number (first n bits), rest all 1

– gateway IP

131.94.134.39

/24

255.255.255.0

131.94.134.255

131.94.134.1

Page 15: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 15

3: Internet Protocol (IP or IPv4)

• host names are mapped to IP address via the Domain Name System (DNS)– need IP numbers of DNS servers

• private IP numbers– used locally– Network Address Translation (NAT)

Page 16: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 16

3: IPv4 packet

Page 17: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 17

3: Internet Protocol version 6 (IPv6)

• larger address space: 128 bit number– written as 8 groups of 4 hex digits2001:468:701:3800:206:5bff:fe53:2a87

Page 18: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 18

3: IPv6 new features

• quality of service• authentication and privacy• anycast address

– used to send packet to one of a group

• multicast– used to send packet in all of a group

Page 19: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 19

4: Transport Layer

• Connection-oriented approach• Hosts send each other control information through

handshaking to set up a logical end-to-end connection

• Imposes reliability on unreliable networks• Guarantees that data sent from sender will arrive

at intended receiver undamaged and in correct sequence

• Connectionless approach• Two hosts do not handshake before transmission• No guarantee that sent messages will be received

in their original order, or at all

Page 20: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 20

4: Transmission Control Protocol (TCP)

• connection-oriented transmission protocol– guarantees that segments sent from a

sender will arrive at the intended receiver undamaged and in correct sequence

– handles error control, congestion control, and retransmission

– allows protocols like HTTP and FTP to send information into network as simply and reliably as writing to a file on the local computer

Page 21: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 21

4: User Datagram Protocol (UDP)

• Connectionless User Datagram Protocol– provides minimum overhead necessary

for the transport layer– no guarantee that UDP datagrams will

reach their destination in their original order, or at all

Page 22: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 22

Application Layers

• Application layer protocols– Specify rules that govern remote

interprocess communication– Determine how processes should

interact

• protocols interact with remote resources– via Uniform Resource Identifier (URI)

Page 23: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 23

Application Layer protocols

URL: Uniform Resource Locator– special form of URIsyntax: protocol://host:port/path

• Common protocols– ftp– http– smtp– ldap– sip

Page 24: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 24

Application Layer protocol: RPC

Remote Procedure Callgeneral mechanism to enable function invocation across host boundaries

• extension of local function call model

Page 25: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 25

Application Layer protocol: RPC

Page 26: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 26

RPC portmapper

• common Unix implementation• translates RPC program numbers

– TCP/IP port numbers– UDP/IP port numbers

• common RPC programs– NIS– NFS

Page 27: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 27

Distributed File Systems

• Networked file systems – Allow clients to access files stored on

remote computers

• Distributed file systems– Special examples of networked file

systems that allow transparent access to remote files

Page 28: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 28

Distributed File System Concepts

• location transparency– user is unaware of the physical location of a

file within a distributed file system– user sees only a global file system

• caching & consistency– Clients keep a local copy of a file and flush

modified copies of it to the server from time to time

– Because there are multiple copies of the same file, files can become inconsistent

Page 29: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 29

Distributed File System Concepts

• scalability– Distributed file systems are designed to share

information among large groups of computers– New computers should be able to be added to

the distributed system easily

• security– Ensuring secure communications– Access control

• fault tolerance

Page 30: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 30

Distributed File System Concepts

• server state– can be either stateful or stateless– stateful system

• server keeps state information of the client requests so that subsequent access to the file is easier

– stateless system• client must specify which file to access in

each request

Page 31: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 31

Network File System

• developed by Sun MicroSystems• early 1980s

• dominant for Unix environments• Versions

– NFS-3– NFS-4, IETF standard

Page 32: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 32

NFS Architecture

Page 33: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 33

Network File System (NFS)

• NFS versions 2 and version 3 – assume a stateless server

implementation

– if the server crashes: • client simply retries until the server

responds, or• aborts

– if server resumes, no need to rebuild state

Page 34: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 34

Network File System (NFS)

• NFS-4 – Stateful– enables faster access to files

– if the server crashes• all the state information of the client is lost• client needs to rebuild its state on the

server before retrying

Page 35: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 35

NFS-4 Caching: Delegation

• efficient client-caching scheme:– server temporarily transfers control of

file to client• read delegation: no other client can write• write delegation: no other client can read or

write

– If another client requests a delegated file, the server will revoke the delegation and request that the original client flush the file back to server

Page 36: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 36

Andrew File System

• developed by Carnegie Mellon University– after first names of A. Carnegie & A. Mellon– early 1980s

• design goals:– large scale– secure– efficient

• Versions: AFS-1, AFS-2, AFS-3– basis for DFS of OSF, predecessor to Coda

Page 37: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 37

Andrew File System (AFS)

• location transparency– all files appear as a branch of a

traditional UNIX file system at each workstation

• built on RPC• security:

– uses Kerberos authentication server– access control lists for file/directories

Page 38: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 38

AFS Structure

Page 39: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 39

AFS caching

• files are cached on clients local disk– notifies clients via callback that files

are no longer valid– client must invalidate its file and

request the most recent version

Page 40: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 40

Coda File System

• developed by Andrew team• design goal: fault tolerance• mid 1980s

Page 41: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 41

Coda File System Concepts

• Volumes – logical pieces of the file system – replicated physically across multiple file

servers

• Volume storage group (VSG)– servers that hold the same volume

• Available volume storage group (AVSG)– reachable members of the VSG

• Preferred Server– member of AVSG, currently serving files

Page 42: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 42

Coda volume structure

Page 43: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 43

Coda read/write scenario

• to read, check cache for file– if found, done– else

•get file from PS•get file versions from other members of AVSG• if conflict, then AVSG members agree on new

versions, and update their copies

• to write,– send file to all members of AVSG– track members of AVSG that have written

file

Page 44: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 44

Coda inconsistency scenario

Page 45: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 45

Coda client disconnect scenario

• When connected to Coda, clients cache files so they can be accessed when disconnected (hoarding stage)

• When disconnected, clients enter the emulation stage where all file requests are serviced from the cache, if the file is resident (error otherwise)

• When reconnected, file updates are sent to the server asynchronously (reintegration stage)

Page 46: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 46

Sprite File System

• Part of Sprite distributed OS– Developed at UC Berkeley, mid 1990s

• Sprite file system characteristics– Emulates a UNIX file system

• even remote access to I/O devices

– Every client has the exact same view of the hierarchy

Page 47: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 47

Sprite File System domains

per client

Page 48: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 48

Sprite read scenario

– to open a file, the client first checks its cache, then makes a request to server

– if the server is unable to satisfy the request from its cache, it reads the data from disk

– both caches retain a copy of file

Page 49: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 49

Sprite write scenario

– client writes to its cache– updated pages are flushed to server every

30 seconds

– lazy write-back strategy

Page 50: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 50

Sprite caching protocol

• sequential write sharing– upon opening file for write, client

checks file version with server• might cause reload of cached file from

server

• concurrent write sharing– if 2 clients open file for write, caching

is turned disabled

Page 51: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 51

Common Internet File System

• started by IBM,no propagated by Microsoft

• other name: server message block (smb)

• built on top of NetBIOS– name service– session management– datagram distribution

• also supports printing

Page 52: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 52

CIFS concepts

• Universal Naming Convention (UNC)\\host\path\file

• server publishes shares– workgroup elects browse master

• client maps share to drive letters

Page 53: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 53

CIFS caching

• enabled via locking concept• opportunistic lock (oplock)

– exclusive oplock• client is granted abritrary buffering rights

– batch oplock• client keeps lock despite no local accessor

– level II oplock• multiple clients read, no writer

Page 54: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 54

CIFS features

• user/share authentication– plain password– challenge/response protocol

with password encryption

• replicated virtual volumes– volumes may span servers– volumes can be transparently moved– volumes may be replicated

Page 55: Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems.

COP 5994 - Operating Systems 55

Agenda for next week:

– Chapter 19: Security

– Read ahead !