Jonathan MitchellCorey Niepraschk

Sean Moe

After “The Breakup” between IBM and Microsoft, IBM continued work on OS/2 while Microsoft worked on Windows NT.

Microsoft ‘borrowed’ many of the HPFS concepts from the OS/2 project to create the new NTFS.

This may be why HPFS and NTFS share the same partition identification type code (07).

Improved over FAT and HPFS with: Improved support for metadata and the

use of advanced data structures to improve performance, reliability, and disk space utilization

Additional extensions such as security access control lists (ACL) and file system journaling

The exact file system specification is a trade secret, although (since NTFS v3.00) it can be licensed commercially from Microsoft through their Intellectual Property Licensing program.

v1.0 with NT 3.1, released mid-1993

v1.1 with NT 3.5, released autumn 1994

v1.2 written by NT 3.51 (mid-1995) and NT 4 (mid-1996) (occasionally referred to as "NTFS 4.0", because OS version is 4.0)

v3.0 from Windows 2000 (occasionally "NTFS V5.0")

v3.1 from Windows XP (autumn 2001; occasionally "NTFS V5.1"), Windows Server 2003 (spring 2003; occasionally "NTFS V5.2"),Windows Vista (mid-2005) (occasionally "NTFS V6.0") and Windows Server 2008

The NT File System Structure is implemented as a relational database

This database is called the Master File Table (MFT) Rows are file/folder records Columns are the file attributes

Everything is in the MFT! Even the MFT metafiles are contained in the

MFT

Everything is in the MFT! Every File and Every Folder on the drive is

kept track of by a record in the MFT. Each File record or Folder record is 1KB large.

Everything is an attribute! Name Attribute List Security Info The data!

Everything is an attribute!

Resident vs. Extended Records Files < 1KB are actually retained inside the

MFT. Files > 1KB are tracked via logical cluster

extents.

Directory structure Folder records are 1KB Instead of data, they have index information Small Folders can reside within the MFT Larger Folders use B+ tree structures to keep

track of all their files.

The B+ Tree Data Structure

Each file has a long name and a short name. Long names can be up to 255 characters

They can have multiple periods, special chars, etc.

Short names are created when you specify long names.

Short names are restricted to the 8.3 format (for backwards compatibility). Does DOCUME~1 look familiar?

Creating a short file name Removes any special characters, spaces, etc.

. " / \ [ ] : ; | = , * ? If the name is more than 8 characters long,

truncate the name to 6 characters, Add a tilde (~), Add a number (1,2, …) Truncate the file extension to 3 characters.

Journaling

Sparse Files

Cluster Remapping

Disk Quota

Alternate Data Streams (ADT)

The concept of a journaling file system means that it logs all changes in files to a journal as it goes, that way if something gets damaged, it's easier to locate and fix it.

When you create a file, delete a file, modify, rename, edit, save, anything to any file/folder, the file system keeps a log entry of what, where, and when. If damage occurs to the file system (power outage, drive damage), there's a good chance that chkdsk will be able to back-track that log, and reconstruct the files.

Support for sparse files is introduced in the NTFS file system as a way to make the disk space usage more efficient. When the sparse file functionality is enabled, the system does not allocate hard drive space to a file except in regions where it contains nonzero data. When a write operation is attempted where a large amount of the data in the buffer is zeros, the zeros are not written to the file. Instead, the file system creates an internal list containing the locations of the zeros in the file, and this list is consulted during all read operations. When a read operation is performed in areas of the file where zeros were located, the file system returns the appropriate number of zeros in the buffer allocated for the read operation.

The advantage of sparse files is that storage is only allocated when actually needed: disk space is saved, and large files can be created even if there is insufficient free space on the file system.

Disadvantages are that sparse files may become fragmented; file system free space reports may be misleading

When Windows detects a bad-sector, NTFS dynamically remaps the cluster containing the bad sector and allocates a new cluster for the data. If the error occurred during a read, NTFS returns a read error to the calling program, and the data is lost (unless in a raid). If the error occurs during a write, NTFS writes the data to the new cluster, and no data is lost.

NTFS puts the address of the cluster containing the bad sector in its bad cluster file ($badclus and $bitmap) so the bad sector is not reused.

Once errors are detected, the disk should be monitored closely and replaced if the defect list grows. This type of error is displayed in the Event Log.

With NTFS this is automatic with FAT32 chkdsk c: /R will check for bad sectors on Drive C and recovers any readable information

• Allows the administrator to set a threshold of disk space that users may utilize. It also allows administrators to keep track of how much disk space each user is using. An administrator may specify a certain level of disk space that a user may use before they receive a warning, and then deny access to the user once they hit their upper limit of space.

• Disk quotas do not take into account NTFS's file compression. Applications that query the amount of free space will only see the amount of free space left to the user who has a quota applied to them.

In NTFS, a file consists of different data streams. One stream holds the security information (access rights and such things), another one holds the "real data" you expect to be in a file. There may be another stream with link information instead of the real data stream, if the file actually is a link. And there may be alternate data streams, holding data the same way the standard data stream does.

The data fork is for the contents of the document while the resource fork (ADS) is to identify file type and other pertinent details.

For example, a file such as text.txt can have an ADS with the name of text.txt:secret.txt that can only be accessed by knowing the ADS name. Alternate streams are not detectable in the original file's size and will not be affected by compression or limits on disk quotas.

ADS can be useful. There is a lot of non-critical information that alternate streams is the most natural place to store to. Examples are thumbnails for graphical files, parsing information for program sources, spell-check and formatting data for documents, or any other info that can be recovered easily.

There are many potential security problems though.

When a file with ADS is copied to a FAT partition or a network share, the ADS portion is lost. ADS may also be deleted manually.

Demo

Other outdated and mostly read-only solutions exist as well: Linux kernel 2.2: NTFS partitions can be read by the kernel since

version 2.2.0. Linux kernel 2.6: contains a driver written by Anton

Altaparmakov (University of Cambridge) and Richard Russon. It supports file read, overwrite and resize, in some cases.

NTFSMount: A userspace driver with limited file and directory read/write support is available using ntfsmount

NTFS for Linux: A commercial driver with full read/write support available from Paragon.

Captive NTFS: A 'wrapping' driver which uses Windows's own driver, ntfs.sys.

Note that all three userspace drivers, namely NTFSMount, NTFS-3G and Captive NTFS, are built on the Filesystem in Userspace (FUSE), a Linux kernel module tasked with bridging userspace and kernel code to save and retrieve data.

Description Limit

Maximum file size Architecturally: 16 exabytes minus 1 KB (264 bytes minus 1 KB) Implementation: 16 terabytes minus 64 KB (244 bytes minus 64 KB)

Maximum volume size

Architecturally: 264 clusters minus 1 cluster Implementation: 256 terabytes minus 64 KB ( 232 clusters minus 1 cluster)

Files per volume 4,294,967,295 (232 minus 1 file)

http://technet2.microsoft.com/windowsserver/en/library/81cc8a8a-bd32-4786-a849-03245d68d8e41033.mspx?mfr=true

http://en.wikipedia.org/wiki/Ntfs