CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration Slide #1

CIT 470: Advanced Network and System Administration

Filesystems


Topics

1. Filesystems and Namespaces

2. Filesystem Types

3. Inodes and Superblocks

4. Network Filesystems

Filesystems and Namespaces

Filesystems

A filesystem is a method for storing and organizing documents.

– Most filesystems offer a hierarchical tree structure of folders within folders.

– Some filesystems are flat, with no folders.– Some filesystems work like a database, where

files are identified by metadata, such as creator or user-created tags.


Kernel Storage Layers


Filesystem Tree Structure

/

bin boot tmp usr var

ls grub bin lib X11R6

vmlinuzmenu.lst

less

zip bin lib

xclock xterm


UNIX has One Namespace

A single tree-structured namespace which– Provides a single way to identify files by name– Contains multiple filesystems:

• /dev – files represent hardware devices• /media/cdrom – ISO9660 optical media filesystem• /proc – in-memory representation of kernel data

– that are added to the namespace with the mount command: mount /dev/devname /fs/location


Namespace contains many fs


Filesystem Types

Filesystem Types by MediaDisk Filesystems

– Filesystems designed to store files to a fixed or removable permanent storage device.

– examples: ext4fs, FAT, ISO9660, NTFS

Solid State Filesystems– Wear leveling: re-arrange block usage to avoid writing too many

times to any one block on flash.

In-Memory Filesystems– Filesystems that represent kernel data structures, e.g. procfs, devfs.

Network Filesystems– Filesystems where file access operations are performed using

network operations to contact a server where the data is stored on a disk or other physical medium.

10CIT 470: Advanced Network and System Administration

Common Disk-based Filesystems

Extended Filesystems– ext2: first full featured UNIX fs for Linux in 1993

• Recommended use: USB + other solid state drives.– ext3: + journaling; 2TB max file size; 16TB max vol– ext4: faster version of ext3 with larger max file + vol size

Microsoft Filesystems– FAT: inefficient disk usage, slow, 8+3 filenames

• 4GB maximum file size in 32-bit FAT– NTFS: modern filesystem, many versions

• Supports long + old 8+3 filenames for compatibility


Inodes and Superblocks

Ext Filesystem Structure


Superblocks and Block Groups


Inode Block Addressing

Slide #15CIT 470: Advanced Network and System Administration

Journaling Filesystems

Problem: writing to file involves many disk writes1. Modify inode to change file size

2. (potentially) Add new data block to used block map

3. (potentially) Add pointer to new data block

4. Write to new data block

Journaling filesystems perform writes by:5. Write blocks to journal.

6. Wait for write to be committed to journal.

7. Write blocks to filesystem.

8. Discard blocks from journal.


Creating a Filesystem

Select a disk partition to create filesystem onfdisk –l /dev/sda will list partitions on 1st disk

fdisk –l /dev/sdb will list partitions on 2nd disk,

Run mke2fs –v /dev/sda2Creates ext2 filesystem on 2nd partition of 1st disk

Wipes any data already existing on that filesystem

Add a –j option to create an ext3 journaling fs.


Mounting a Filesystem

1. Create a mountpointmkdir -p /stor/video

2. Mount filesystem on chosen directorymount -t ext3 /dev/sda2 /stor/video

3. Use filesystem

4. Unmount filesystem when doneumount /dev/sda2

Happens automatically at reboot or shutdown


Automatic Mounting

Filesystems in /etc/fstab are mounted on boot.

Use mount to see current mounted filesystems.

# /etc/fstab: static file system information.## <device> <mnt pt> <type> <options> <dump> <pass>proc /proc proc defaults 0 0/dev/sda1 / ext3 defaults 0 1/dev/sda2 none swap sw 0 0/dev/sda3 /home ext3 defaults 0 1/dev/sdb1 /backup ext3 defaults 0 0


Checking Filesystem Integrity

fsck utility performs consistency checks– Are used blocks actually used?– Do inodes point to any unused blocks?– Are used inodes pointed to by directory entries?

and repairs inconsistencies if– Sysadmin enters ‘y’ in interactive mode.– Sysadmin uses ‘-y’ argument to do all repairs.

Run fsck with unmounted partition as arg:fsck –y /dev/sda2


Access Control


Read--You can read the file with cat, more, etc.

Write--You can modify the file with vi,

Execute--You can run the file if it’s a program.

POSIX ACLs

Specify individual groups and users.Basic ACL user/group refers to owner.

POSIX ACLs allow specifying users + groups.

To add/modify permissions for a user:setfacl –m u:username:rw- filename

To add/modify permissions for a group:setfacl –m g:groupname:rw- filename


File Attributes

Attributes extend file permissions:a: append-only (only root can set)

i: immutable (read-only, only root can set)

s: safe-delete (overwrite, not supported yet)

Use lsattr to view attributes.Most files do not have any attributes set.

Use chattr to set attributes.chattr +i /boot/vmlinuz*


Network Filesystems

Use filesystem to transparently share files.

Examples:– NFSv3– CIFS– AFS– NFSv4


NFS v3

Network File System– Transparent, behaves like a regular UNIX filesystem.– Uses UNIX UIDs,GIDs,perms but can work on Win.– Since NFS is stateless, file locking and recovery are handled by

rpc.lockd and rpc.statd daemons.

Security– Server only lets certain IP addresses mount filesystems.– Client UIDs have same permissions on server as client.– Client root UID is mapped to nobody, but

– Root can su to any client UID to access any file.

How NFS Works


http://www.cs.ucla.edu/~kohler/class/05f-osp/notes/lec18.html


CIFS

Microsoft Network Filesystem Derived from 1980s IBM SMB net filesystem. Originally ran over NetBIOS, not TCP/IP. \\svr\share\path Universal Naming Convention Auth: NTLM (insecure), NTLMv2, Kerberos

Implementation MS Windows-centric (filenames, ACLs, EOLs) Samba: UNIX client and server software.


AFS

Distributed filesystem– Global namespace: /afs/abc.com/vol_home1– Servers provide one or more volumes.– Volume replication with RO copies on other svrs.

Cells are administrative domains within AFS.– Cells contain multiple servers.– Each server provides multiple volumes.

Security– Kerberos authentication– ACLs with user-controlled groups


NFSv4

New model of NFS– Only one protocol (no separate mount,lock,etc.)– Global namespace.– Security (ACLs, Kerberos, encryption)– Cross platform + internationalized.– Better caching via delegation of files to clients.


Using NFSv3

Client

1. Start portmap

2. …

3. …

4. …

5. Mount filesystems.

Server

1. Start portmap

2. Start NFS services.

3. Configure exports.

4. Export filesystems.


NFSv3 Services

portmap — RPC service for Linuxportmap

nfs — NFS file server processes.rpc.mountdrpc.rquotadnfsd

nfslock — Optional file locking service.rpc.statd


NFSv3 Processes

rpc.mountd — Handles client mount requests.

rpc.nfsd — NFS server processes.

rpc.lockd — Process for optional nfslock service.

rpc.statd — Handles server crashes for nfslock.

rpc.rquotad — Quotas for remote users.


rpcinfo> rpcinfo -pprogram vers proto port 100000 2 tcp 111 portmapper 100000 2 udp 111 portmapper 100021 1 udp 32774 nlockmgr 100021 1 tcp 34437 nlockmgr 100011 1 udp 819 rquotad 100011 2 udp 819 rquotad 100011 1 tcp 822 rquotad 100011 2 tcp 822 rquotad 100003 2 udp 2049 nfs 100003 3 udp 2049 nfs 100003 2 tcp 2049 nfs 100003 3 tcp 2049 nfs 100005 2 udp 836 mountd 100005 2 tcp 839 mountd 100005 3 udp 836 mountd 100005 3 tcp 839 mountd


NFSv4 Processes

nfsd — NFSv4 server processes. Handles mounts.

rpc.idmapd — Maps NFSv4 names (user@domain) and local UIDs and GIDs. Uses /etc/idmapd.conf.

rpc.svcgssd — Server transport Kerberos auth.

rpc.gssd — Client transport Kerberos auth.


NFSv3 Server Configuration

1. Configure /etc/exportsList filesystems to be exported.Specify export options (ro, rw, etc.)Specify hosts/networks to export to.

2. Export filesystems.exportfs

3. Start NFS server (if not already started)service portmap startservice nfs start


/etc/exportsFormat: directory hosts(options)Options

ro, rw Read-only, read-write.async Server replies before write.sync Save before reply (default)all_squash Map all users to anon UID/GID.root_squash Map root to anon UID (default)no_root_squash Don’t map root (insecure.)anon{uid,gid} Set anonymous UID, GID.

Examples:/home *.example.com(rw,sync)/backups 192.168.1.0/24(ro,all_squash)/ex/limited foo.example.com


Client Configuration

Manual mountingmount -t <nfs-type> -o <options>

server:/remote/export /local/directory

Mounting via /etc/fstabserver:/remote/export /local/directory <nfs-type>

<options> 0 0

NFS Type is either nfs or nfs4.


Mount Optionshard or soft — Error handling

hard: NFS requests will uninterruptible wait until server back.soft: NFS requests will timeout and report failure.

intr — NFS requests can be interrupted if server unreachable.nfsvers=2,3— NFS protocol version (not 4)noexec — Prevents execution of binaries.nosuid — Disables setuid for security.rsize,wsize=# — NFS data block size (default 8192) sec=mode — NFS security type.

sys uses local UIDs and GIDs.krb5 uses Kerberos5 authentication.krb5i uses Kerberos5 authentication + integrity checking krb5p uses Kerberos5 auth + integrity checking + encryption.

tcp, udp — Specifies protocol to use for mount.


AutomounterManages NFS mounts

Automounter maps vs /etc/fstab.

Mounts filesystems only when needed:

Makes administering many filesystems easier.

Improves startup speed.

Provides uniform namespaces.

Ex: mounts /home/home7 as /home on login.

/etc/auto.master points to maps/home /etc/auto.home

Maps describe mounts* -fstype=nfs4,soft,intr,nosuid server:/home


Security

Limit which hosts have access to filesystems.– Specify hosts in /etc/exports.– Use iptables to limit which hosts can use NFS.

Limit mount options– Default to ro unless writes are necessary.– Disable suid and execution unless needed.– Map root to nobody.

Block NFS at network firewalls.– Block all protocols, not just port 2049.

Use NFSv4 with Kerberos auth + encryption.


PerformanceMeasuring performance

nfsstat/proc/net/rpc/nfsd

Optimizations– Increase the block size. Problem: fragments?– Set the async option on mounts.– Faster network card.– Faster disk array.– NVRAM cache on array to save NFS writes.


References1. Michael D. Bauer, Linux Server Security, 2nd edition, O’Reilly, 2005.2. Mike Eisler, Ricardo Labiaga, Hal Stern, Managing NFS and NIS, 2nd

edition, O’Reilly, 2001.3. Aeleen Frisch, Essential System Administration, 3rd edition, O’Reilly, 2002.4. Evi Nemeth et al, UNIX System Administration Handbook, 3rd edition,

Prentice Hall, 2001.5. NFS HOWTO, http://nfs.sourceforge.net/nfs-howto6. RedHat, Red Hat Enterprise Linux 4 System Administration Guide,

http://www.redhat.com/docs/manuals/enterprise/RHEL-4-Manual/sysadmin-guide/, 2005.

7. RedHat, Red Hat Enterprise Linux 4 Reference Guide, http://www.redhat.com/docs/manuals/enterprise/RHEL-4-Manual/ref-guide/ch-nfs.html, 2005.

http://www.oreillynet.com/cs/catalog/view/au/690?x-t=book.view



http://www.redhat.com/docs/manuals/enterprise/RHEL-4-Manual/sysadmin-guide/

http://www.redhat.com/docs/manuals/enterprise/RHEL-4-Manual/sysadmin-guide/

http://www.redhat.com/docs/manuals/enterprise/RHEL-4-Manual/ref-guide/ch-nfs.html

http://www.redhat.com/docs/manuals/enterprise/RHEL-4-Manual/ref-guide/ch-nfs.html

CIT 470: Advanced Network and System Administration

Documents

Transcript of CIT 470: Advanced Network and System Administration