Jonathan Wheeler E-Science Centre, RAL Introduction to XFS HEPSYSMAN 5 th December 2005.

Jonathan Wheeler

E-Science Centre, RAL

Introduction to XFS

HEPSYSMAN

5th December 2005

Jonathan Wheeler

e-Science Centre, RAL

What exactly is a filesystem ?

Organization:• Large disk space to store small data items• Use files and directories

In UNIX everything is a stream of bytes:• More complex file definition in other OS’s• Devices are also presented in the filesystem• Linux uses virtual filesystems, e.g. /proc

Jonathan Wheeler


A little bit of background

Three generation of UNIX filesystems:

a) v7 / sysv / coherent / minix

b) ffs / ext2

c) vxfs / ext3 / jfs / xfs / reiserfs

Jonathan Wheeler


Features of 3rd generation UNIX filesystems

a) Intent logging / journaling

b) Flexible metadata structures

c) Dynamic inode allocations

d) Extents

Jonathan Wheeler


History of XFS

Developed by Silicon Graphics (now SGI) for use on IRIX:

• 1993 - original design• 1994 - available for IRIX v 5.3• 2000 - released as Open Source• 2001/2002 – some Linux distributions with

native XFS support• 2003 – XFS support in 2.4 kernel

Jonathan Wheeler


XFS features (1)

Journaling:

• but only metadata updates, not data updates.

Fast Transactions:

• uses efficient B+ tree structures for fast searches and rapid space allocation

Jonathan Wheeler


XFS Features (2)

64-bit file system:a) 64bit variables for global counters and disk addresses

b) Large theoretical maximum filesystem sizei. For Linux 2.4: 2 Terabytes.ii. For Linux 2.6 (when using 64 bit addressing on a

64 bit platform): 9 Petabytes (or the device limits)iii. For Linux 2.6 (on 32 bit platforms): 16Terabytes

(even with 64 bit addressing)

c) Maximum file size is same as filesystem size

Jonathan Wheeler


XFS Features (3)

Partitioned into Allocation Groups:

a) each AG manages its own free space and inodes

b) provides scalability and parallelism within the file system

c) limits the size of the structures needed to track this information

Jonathan Wheeler


XFS Features (3, continued)

d) allows many internal pointers to be 32-bits

e) AGs typically range in size from 0.5 to 4GB

f) files and directories are not limited to a single AG.

Jonathan Wheeler


XFS Features (4)

Sophisticated support utilities:a) fast mkfs (make a file system)b) dump and restore utilities for backupc) xfsrepair to fix corrupt filesystemd) xfs_fsr (XFS defragmenter)e) xfsdb (XFS debug)f) xfscheck (XFS check)g) xfs_growfs (enlarges filesystems online)

Jonathan Wheeler


Why you might want to use XFS

a) Stable, mature code base (oldest journaling filesystem available on Unix)

b) Very good performance for large reads or writes

c) Designed for large systemsd) Supports user and group quotase) Support for Access Control Lists and

Extended Attributes

Jonathan Wheeler


Extended Attributes

An extended attribute is a name/value pair associated with a file. Attributes can be attached to all types of inodes: regular files, directories, symbolic links, device nodes, and so forth. Attribute values can contain up to 64KB of arbitrary binary data.

See man attr for more details

Jonathan Wheeler


Extended attribute namespaces

XFS implements three attribute namespaces:a) user namespace available to all users,

protected by the normal file permissionsb) system namespace, accessible only to

privileged users and used for protected filesystem meta-data such as access control lists (ACLs) and hierarchical storage manager (HSM) file migration status

c) security namespace used by security modules (SELinux)

Jonathan Wheeler


Why you might not want to use XFS

a) No data journaling

b) tar, cpio do not understand Extended Attributes

Jonathan Wheeler


Resources

XFS/Linux homepage: http://oss.sgi.com/projects/xfs/ contains pointers to whitepapers, books, articles, etc.

Public mailing list: [email protected] with searchable list archive

SGI Bugzilla database to report any bugs in XFS for Linux

Jonathan Wheeler


Comparison tests - 1

Tests were performed:• on the same filesystem on the same

hardware (AMD Athlon 1666 Mhz, 500Mb)• using 3 different operating systems:

i. Redhat 7.3 (2.4.20-28.7)

ii. Scientific Linux version 3 (2.4.21-20.EL)

iii. Scientific Linux version 4

(2.6.9-22.0.1.EL and 2.6.9.-11.EL.XFS)

Jonathan Wheeler



Tests used different filesystem types:

• RH 7.3 – ext2, ext3 (xfs not available)

• SL 3 – ext2, ext3 (attempts to create xfs filesystem caused system crash requiring hardware reset)

• SL 4 – ext2, ext3, xfs (default), xfs (custom: mkfs.xfs –d agsize=4g –i version=1 –I size=512 <device>)

Jonathan Wheeler



Tests were performed:• using bonnie++ test program (see

http://www.coker.com.au/bonnie++)• using the same parameters for each test:

/usr/local/sbin/bonnie++ -d /data \ -s 4096M:1024k \-n 192:1048576:128:1024 -m csfnfs30 \-u root 1>> outputfile 2>&1

Jonathan Wheeler



12 tests in total, each reporting filesystem performance and CPU use

The first 6 tests measure I/O intensive work:

• Sequential output for character, blocked and rewrites

• Sequential input for character and blocked

• Random seeks

Jonathan Wheeler



The other 6 tests measure performance for many small files using:

• Sequential create, read and delete

• Random create, read and delete

Jonathan Wheeler


Comparsion of filesystems (Sequential output)

0

5000

10000

15000

20000

25000

30000

35000

73.ext2 73.ext3 sl3.ext2 sl3.ext3 sl4.ext2 sl4.ext3 sl4.xfs sl4.xfs.special

Filesystem type

Per c

hara

cter

(Kb/

sec)

Series1

Jonathan Wheeler


Comparison of filesystems (Sequential output)

0

5000

10000

15000

20000

25000

30000

35000


Filesystem type

Per b

lock

(Kb/

sec)

Series1

Jonathan Wheeler


Comparison of filesystems (Sequential output)

0

5000

10000

15000

20000

25000


Filesystem type

Rew

rite

(Kb/

sec)

Series1

Jonathan Wheeler


Comparsion of filesystems (Sequential input)

0

5000

10000

15000

20000

25000

30000

35000


Filesystem type

Per c

hara

cter

(Kb/

sec)

Series1

Jonathan Wheeler


Comparison of filesystems (Sequential input)

80000

82000

84000

86000

88000

90000

92000

94000

96000

98000

100000


Filesystem type

Per b

lock

(Kb/

sec)

Series1

Jonathan Wheeler


Compariosn of filesystems (Random seeks)

0

5

10

15

20

25


Filesystem type

Num

ber/s

ec

Series1

Jonathan Wheeler


Comparison of filesystems (Sequential create)

0

5

10

15

20

25

30

35

40

45


Filesystem type

Crea

te (N

um/s

ec)

Series1

Jonathan Wheeler


Comparison of filesystems (Sequential create)

0

20

40

60

80

100

120

140

160

180


Filesystem type

Read

(num

ber/s

ec)

Series1

Jonathan Wheeler


Comparison of filesystems

0

50

100

150

200

250

300

350

400


Filesystem type

Dele

te (n

umbe

r/sec

)

Series1

Jonathan Wheeler


Comparison of filesystems (Random create)

0

5

10

15

20

25


Filesystem type

Crea

te (N

umbe

r/sec

)

Series1

Jonathan Wheeler



0

5

10

15

20

25


Filesystem type

Read

(Num

ber/s

ec)

Series1

Jonathan Wheeler



0

5

10

15

20

25

30

35

40

45

50


Filesystem type

Dele

te (N

umbe

r/sec

)

Series1

Jonathan Wheeler E-Science Centre, RAL Introduction to XFS HEPSYSMAN 5 th December 2005.

Documents

Transcript of Jonathan Wheeler E-Science Centre, RAL Introduction to XFS HEPSYSMAN 5 th December 2005.