I/O and Storage · CS 571: Operating Systems (Spring 2020) Lecture 10a Yue Cheng Some material...

I/O and Storage: Flash SSDs

CS 571: Operating Systems (Spring 2020)Lecture 10a

Yue Cheng

Some material taken/derived from: • Wisconsin CS-537 materials created by Remzi Arpaci-Dusseau.Licensed for use under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.

Disk vs. Flash

2Y. Cheng GMU CS571 Spring 2020

Disk Overview

• I/O requires: seek, rotate, transfer

• Inherently:• Not parallel (only one head)• Slow (mechanical)• Poor random I/O (locality around disk head)

• Random requests each taking ~10+ ms

Flash Overview

• Hold charge in cells. No moving (mechanical) parts!

• Inherently parallel!

• No seeks!

Storage Hierarchy Overview

Flash:Smaller capacityFaster accesses

HDD:Larger capacity

Way slower accesses

Y. Cheng GMU CS571 Spring 2020

Disks vs. Flash: Performance

• Throughput• Disk: ~130MB/s (sequential)

• Flash: ~400MB/s

• Latency• Disk: ~10ms (one op)

• Flash:• Read: 10-50us

• Program: 200-500us

• Erase: 2ms

• Flash: ~400MB/s

• Erase: 2ms

Types of write, more later…

Disks vs. Flash: Internal

8* https://www.enterprisestorageforum.com/storage-hardware/ssd-vs-hdd.html

Disks vs. Flash: Summary

10* https://www.enterprisestorageforum.com/storage-hardware/ssd-vs-hdd.html

Flash Architecture

SLC: Single-Level Cell

NAND Cell

MLC: Multi-Level Cell

NAND Cell

Single- vs. Multi-Level Cell

charge

Single- vs. Multi-Level Cell

expensiverobust

1 cell: 1 bit

cheapsensitive

1 cell: multi-bit

Wearout

• Problem: flash cells wear out after being erased too many times

• MLC: ~10K times

• SLC: ~100K times

• Usage strategy: ???

Wearout

• Problem: flash cells wear out after being erased too many times

• MLC: ~10K times

• SLC: ~100K times

• Usage strategy: wear leveling• Prevents some cells from being wornout while others

still fresh

• Flash devices are divided into banks (aka. planes)

• Banks can be accessed in parallel

Bank 0 Bank 1 Bank 2 Bank 3

read read

data data

Flash Writes

• Writing 0’s• Fast, fine-grained

• Writing 1’s• Slow, coarse-grained

Flash Writes

• Writing 0’s• Fast, fine-grained

• called “program”

• Writing 1’s• Slow, coarse-grained

• called “erase”

Flash Writes

• Writing 0’s• Fast, fine-grained [page-level]

• Writing 1’s• Slow, coarse-grained [block-level]

Flash Writes

• Writing 0’s• Fast, fine-grained [page-level]

• Writing 1’s• Slow, coarse-grained [block-level]

• Flash can only “write” (program) into clean pages• “clean”: pages containing all 1’s (pages that have

been erased)

• Flash does not support in-place overwrite!

Banks and Blocks

Bank 0 Bank 2 Bank 3

Each bank contains many “blocks”

Block and Pages

11111111

One block

Block and Pages

11111111

One page

Block and Pages

11111111

All pages are clean (“programmable”)

10111000

11111111

program

10111000

11111111

01101010

11111111

program

10111000

11111111

01101010

11111111

Two pages hold data (cannot be overwritten)

10111000

11111111

01101010

11111111

still want to write data into this page???

Two pages hold data (cannot be overwritten)

10111000

11111111

01101010

11111111

erase(the whole block)

11111111

After erase, again, free state (can write new data in any page)

10110001

11111111

This blue page holds data

Flash vs. Disks: APIs

disk flashread

disk flashre

read sector read page

disk flashre

read sector read page

write sectorprogram page

(0’s)erase block

(1’s)

Flash Architecture

• Bank/plane: 1024 to 4096 blocks• Banks accessed in parallel

• Block: 64 to 256 pages• Unit of erase

• Page: 2 to 8 KB• Unit of read and program

• Flash: ~400MB/s

• Erase: 2ms

Working with File System

Traditional File Systems

File System

Storage Device

Traditional API:• read sector• write sector

Traditional File Systems

File System

Storage Device

Traditional API:• read sector• write sector

Mismatch with flash!

Traditional APIs wrapping around Flash APIsread(addr):

return flash_read(addr)

write(addr, data):

block_copy = flash_read(all pages of block)

modify block_copy with data

flash_erase(block of addr)

flash_program(all pages of block_copy)

Awkward Flash Write

Memory

block 0

block 1

block 2Y. Cheng GMU CS571 Spring 2020

Awkward Flash Write

Memory

block 0

block 1

block 2

File system wantsto write 0001

Awkward Flash Write

Memory

block 0

block 1

block 2Y. Cheng GMU CS571 Spring 2020

Awkward Flash Write

Memory

block 0

block 1

block 2

Read all pages in block

Awkward Flash Write

Memory

block 0

block 1

block 2

Awkward Flash Write

Memory

block 0

block 1

block 2

Modify target pagein memory

Awkward Flash Write

Memory

block 0

block 1

block 2

Awkward Flash Write

Memory

block 0

block 1

block 2

Erase whole block

Awkward Flash Write

Memory

block 0

block 1

block 2

Awkward Flash Write

Memory

block 0

block 1

block 2

Program all pages in block 00

010011

Awkward Flash Write

Memory

block 0

block 1

block 2

Issue: Write Amplification

• Random writes are expensive for flash!

• Writing one 4KB page may cause:• read, erase, and program of the whole 256KB

Flash Translation Layer

Flash Translation Layer (FTL)

• Add an address translation layer between upper-level file system and lower-level flash• Translate logical device addresses to physical

addresses

• Convert in-place write into append-write

• Essentially, a virtualization & optimization layer

Logical

Physical 0011

0 1 2 3 4 5 6 7

block 0 block 1

Logical

Physical 0011

0 1 2 3 4 5 6 7

block 0 block 1

Write 0011

Logical

Physical 0011

0 1 2 3 4 5 6 7

block 0 block 1

Write 0011

Logical

Physical 0011

0 1 2 3 4 5 6 7

block 0 block 1

Write 0011

Change address mapping

Logical

Physical 0011

0 1 2 3 4 5 6 7

block 0 block 1

Write 0011

Change address mapping

Marked as invalidand will be garbage

collected

SSD Architecture with FTL

FTL SRAM:Mapping table

SSD provides disk-like interface

• Usually implemented in flash device’s firmware

• Where to store mapping? • SRAM

• Physical pages can be in three states• valid, invalid, free

State Transition of Physical Pages

Invalid

ProgramValid

Invalid

Program

Relocate(overwrite)

Free Valid

Invalid

Program

Erase Relocate(overwrite)

I/O and Storage · CS 571: Operating Systems (Spring 2020) Lecture 10a Yue Cheng Some material...

Documents

Transcript of I/O and Storage · CS 571: Operating Systems (Spring 2020) Lecture 10a Yue Cheng Some material...

CS 471 Operating Systems Yue Cheng

Transforming Policies into Mechanisms with Infokernel Andrea C. Arpaci-Dusseau, Remzi H. Arpaci-Dusseau, Nathan C. Burnett, Timothy E. Denehy, Thomas J.

Deploying Safe User-Level Network Services with icTCP Haryadi S. Gunawi Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau The ADvanced Systems Laboratory.

Byzantine Generals UNIVERSITY of WISCONSIN-MADISON Computer Sciences Department CS 739 Distributed Systems Andrea C. Arpaci-Dusseau One paper: “The Byzantine.

Journal-guided Resynchronization for Software RAID Timothy E. Denehy, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau University of Wisconsin, Madison.

Midterm Review - GitHub Pages · Midterm Review CS 571: Operating Systems (Spring 2020) Lecture 6b Yue Cheng Some material taken/derived from: •Wisconsin CS-537 materials created

REMZI H. ARPACI-DUSSEAU Research Interests Education ...

Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.

CS 202 Professor Andrea Arpaci-Dusseau Lecture …pages.cs.wisc.edu/~dusseau/Classes/CS202-F10/Lectures/...12/17/10 1 Lecture 41: What problems stretch the limits of computation? UNIVERSITY

Introduction to XPath Bun Yue Professor, CS/CIS UHCL.

1 Storage-Aware Caching: Revisiting Caching for Heterogeneous Storage Systems (Fast’02) Brian Forney Andrea Arpaci-Dusseau Remzi Arpaci-Dusseau Wisconsin.

1 Storage-Aware Caching: Revisiting Caching for Heterogeneous Systems Brian Forney Andrea Arpaci-Dusseau Remzi Arpaci-Dusseau Wisconsin Network Disks University.

CS 202 Professor Andrea Arpaci-Dusseau Fall 2010 …pages.cs.wisc.edu/~dusseau/Classes/CS202-F10/Lectures/lecture-bits.pdfApplication Program Language ... Microarch FORTRAN Organization

Final Reviewnotes.pdfFinal Review CS 571: Operating Systems (Spring 2020) Lecture 12 Yue Cheng Some material taken/derived from: •Wisconsin CS-537 materials created by RemziArpaci-Dusseau.

Exploiting Gray-Box Knowledge of Buffer Cache Management Nathan C. Burnett, John Bent, Andrea C. Arpaci-Dusseau, Remzi H. Arpaci-Dusseau University of.

The CS Program at UHCL UHCL Open House 2005 Oct 1, 2005 Dr. Kwok-Bun Yue Associate Professor, CS Chair, CIS University of Houston-Clear Lake.

A Software Layer for Disk Fault Injection Jake Adriaens Dan Gibson CS 736 Spring 2005 Instructor: Remzi Arpaci-Dusseau.

Intro. to XML & XML DB Bun Yue Professor, CS/CIS UHCL.

Swaminathan Sundararaman, Yupu Zhang, Sriram Subramanian, Andrea C. Arpaci-Dusseau, Remzi H. Arpaci-Dusseau.

Professor Andrea Arpaci-Dusseau How to Solve …pages.cs.wisc.edu/~dusseau/Classes/CS202-F11/Lectures/lecture-AI.pdf · How to Solve Challenging Problems? ... CS 202: Introduction