1

18: Filesystem Examples:NTFS, The Future

Mark Handley

NTFS Filesystem

2

File System API Calls in Windows 2000

Principle Win32 API functions for file I/O

Second column gives nearest UNIX equivalent

Windows 2000:

File System API

Windows API has very many parameters. Eg CreateFile() has 7 parameters:

Pointer to filename to open/create. Flags for read/write/both. Flags for whether multiple processes can simultaneously

open file. Pointer to security descriptor telling who can access the

file. Flags telling what to do if the file exists/doesn’t exist. Flags dealing with attributes such as archiving and

compression. The handle of a file whose attributes should be cloned for

the new file.

3

Windows 2000 File System API:

Copying a File/* Open Files for Input and Output */inhandle = CreateFile(“data”, GENERIC_READ, 0, NULL, OPEN_EXISTING, 0, NULL);outhandle = CreateFile(“new”, GENERIC_WRITE, 0, NULL, CREATE_ALWAYS,

FILE_ATTRIBUTE_NORMAL, NULL);

/* Copy the File */do {

s = ReadFile(inhandle, buffer, BUF_SIZE, &count, NULL);if (s && count > 0)

WriteFile(outhandle, buffer, count, &ocnt, NULL);} while (s > 0 && count > 0)

/* Close the Files */CloseHandle(inhandle);CloseHandle(outhandle);

Windows 2000 File System API:

System Calls for Directory Management

Second column gives nearest UNIX equivalent, when one exists

4

NTFS

NTFS replaces FAT file system in recent Windows releases. Design from scratch: complex and fully featured. Each volume (partition) is a linear sequence of blocks

4KB blocksize is typical 64bit block IDs.

Each volume has a Master File Table (MFT) Sequence of 1KB records. One (or more) record per file or directory.

Somewhat like i-nodes, but more flexible. Each MFT record is a sequence of variable length

(attribute, value) pairs. Long attributes can be stored externally, and a pointer kept

in the MFT record.

NTFS MasterFile Table

First 16 entries arereserved for NTFSmetadata files.

MFT is itself a file.

1st recorddescribes the MFTfile itself (whenthe blocks are ondisk).

5

MFT MetaData Attributes

$LogFile: when many changes to filesystem are made,they’re logged here first. If system goes down,consistency can be recovered by reading the log.

$AttrDef: MFT attributes are defined here, allowingextensibility.

$Bitmap: keeps track of free blocks.

$Boot: points to bootstrap loader for OS booting.

$Upcase: Defines filename case mapping (for non-roman alphabets).

File System Structure (2)

The attributes used in MFT records

6

NTFS File Block Management

NTFS tries to allocate files in runs of consecutive blocks.

Unlike with FAT, files can contain holes.

In an MFT record, blocks are described by a sequence ofDATA attributes - one for each section between holes.

Within each DATA attribute, there are multiple fields eachindicating a run of consecutive disk blocks.

If all the attributes don’t fit into one MFT record, extensionrecords can be use to hold more.

MFT Record for Normal File

An MFT record for a three-run, nine-block file

7

Extension MFT Records

A file that requires three MFT records to store its runs. Typically because file is very fragmented or very large.

MFT Record for a Small Directory

The MFT record for a small directory. Directory Entries stored as a simple list.

Large directories use B+ trees instead.

8

NTFS File Name Lookup

Steps in looking up the file C:\maria\web.htm First prepend \?? to filename, and lookup in \?? directory

NTFS File Compression

API can specify that a file should be compressed by thefilesystem.

OS attempts to compress 16 blocks at a time. If compression reduces to 15 blocks or less, compressed

blocks are written to disk. Otherwise uncompressed blocks are written. Runs of compressed blocks use two DATA runs in MFT,

one for the compressed data blocks, and one for how muchcompression was achieved.

Seeking is not terribly efficient: Must decompress 16 blocks at a time to find the correct

uncompressed block.

9

NTFS File Compression

(a) An example of a 48-block file being compressed to 32 blocks(b) The MTF record for the file after compression

Encrypting File System (EFS) sits above NTFS, below theWin32 API.

K retrieved

user's public key

NTFS File Encryption

10

The Future of FilesystemsWinFS

Apple Spotlight

Microsoft WinFS

Details still sketchy: Integrates SQL database

functionality into the filesystem. Native searching capability. Relationships between files.

Built on top of NTFS? WinFS data can be structured

with an XML schema to explainmeaning/purpose.

Support for notifications whenfile data changes or is deleted.

Windows Future Storage (WinFS), originally scheduled forrelease in Longhorn, but now postponed (2007???)

http://www.c-sharpcorner.com/Longhorn/WinFS/WinFSDataModel.asp

11

MacOS X Spotlight

Part of MacOS 10.4 (Tiger), release in 2005

Not really a new filesystem

Metadata search capability for MacOS.

Automatic indexing system for files, as they are created,modified, or deleted

Plugins for different file types to extract file metadata.

Search capabilities on both metadata and indexed filecontent.