Physical Limits of ComputingPhysical Limits of ComputingDr. Mike FrankDr. Mike Frank

Slides from a Course Taught at Slides from a Course Taught at the University of Floridathe University of FloridaCollege of EngineeringCollege of Engineering

Department of Computer & Department of Computer & Information Science & EngineeringInformation Science & EngineeringSpring 2000, Spring 2002, Fall 2003Spring 2000, Spring 2002, Fall 2003

Overview of First LectureOverview of First Lecture• Introduction: Moore’s Law vs. Known PhysicsIntroduction: Moore’s Law vs. Known Physics• Mechanics of the course:Mechanics of the course:

– Course websiteCourse website– Books / readingsBooks / readings– Topics & scheduleTopics & schedule– Assignments & grading policiesAssignments & grading policies– misc. other administriviamisc. other administrivia

Physical Limits of ComputingPhysical Limits of ComputingIntroductory LectureIntroductory Lecture

Moore’s Law vs. Known PhysicsMoore’s Law vs. Known Physics

Moore’s LawMoore’s Law• Moore’s Law proper:Moore’s Law proper:

– Trend of doubling of number of transistors per Trend of doubling of number of transistors per integrated circuit every 18 (later 24) monthsintegrated circuit every 18 (later 24) months

• First observed by Gordon Moore in 1965 (see readings)First observed by Gordon Moore in 1965 (see readings)

• ““Generalized Moore’s Law”Generalized Moore’s Law”– Various trends of exponential improvement in many Various trends of exponential improvement in many

aspects of information processing technology (both aspects of information processing technology (both computing & communication):computing & communication):

• Storage capacity/cost, clock frequency, performance/cost, Storage capacity/cost, clock frequency, performance/cost, size/bit, cost/bit, energy/operation, bandwidth/cost …size/bit, cost/bit, energy/operation, bandwidth/cost …

Moore’s Law – Devices per ICMoore’s Law – Devices per IC

1

10

100

1,000

10,000

100,000

1,000,000

10,000,000

100,000,000

1,000,000,000

1950 1960 1970 1980 1990 2000 2010

Avg. increaseof 57%/year

4004

8086286

386486DX Pentium

P2P3

P4Itanium 2

Madison

Early Fairchild

ICs

Intel µpu’s

Microprocessor Performance TrendsMicroprocessor Performance Trends

Raw technologyperformance(gate ops/sec/chip):Up ~55%/year

Source:Hennessy &Patterson,ComputerArchitecture:A QuantitativeApproach.AddedPerformanceanalysis based on datafrom theITRS 1999roadmap.

Super-Exponential Long-Term TrendSuper-Exponential Long-Term Trend

Across Multiple TechnologiesAcross Multiple Technologies

Source: Kurzweil, The Age of Spiritual Machines, pp. 22-25

Mechanical

ElectromechanicalRelays

Vacuum Tubes

DiscreteTransistors

IntegratedCircuits

Ops/second/$1,000

Source: Kurzweil ‘99

Known Physics:Known Physics:• The history of physics has The history of physics has

been a story of:been a story of:– Ever-increasing precision, Ever-increasing precision,

unity, & explanatory powerunity, & explanatory power• Modern physics is veryModern physics is very

nearly perfect!nearly perfect!– All accessible phenomena are All accessible phenomena are

exactly modeled, as far as we exactly modeled, as far as we know, to the limits of know, to the limits of experimental precision, ~11 experimental precision, ~11 decimal places today.decimal places today.

• However, the story is not However, the story is not quite complete yet:quite complete yet:– There is no experimentally There is no experimentally

validated theory unifying GR validated theory unifying GR & QM (yet)& QM (yet) String theory?

M-theory?Loop quantum gravity?

Other?

Fundamental Physical Limits of ComputingFundamental Physical Limits of Computing

Speed-of-LightLimit

Thoroughly Confirmed

Physical Theories

UncertaintyPrinciple

Definitionof Energy

Reversibility

2nd Law ofThermodynamics

Adiabatic Theorem

Gravity

Theory ofRelativity

QuantumTheory

ImpliedUniversal Facts

Affected Quantities in Information Processing

Communications Latency

Information Capacity

Information Bandwidth

Memory Access Times

Processing Rate

Energy Loss per Operation

ITRS Feature Size Projections

0.1

1

10

100

1000

10000

100000

1000000

1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010 2015 2020 2025 2030 2035 2040 2045 2050

Year of First Product Shipment

Feat

ure

Size

(nan

omet

ers)

uP chan L

DRAM 1/2 pmin Tox

max Tox

Atom

We are here

Bacterium

Virus

Proteinmolecule

DNA moleculethickness

Eukaryoticcell

Human hairthickness

ITRS Feature Size Projections

0.1

1

10

100

1000

1995 2000 2005 2010 2015 2020 2025 2030 2035 2040 2045 2050

Year of First Product Shipment

Feat

ure

Size

(nan

omet

ers)

uP chan L

DRAM 1/2 pmin Tox

max Tox

Atom

We are here

Virus

Proteinmolecule

DNA moleculethickness

Bacterium

A Precise Definition of A Precise Definition of NanoscaleNanoscale

10−6 m = 1 µm

10−9 m = 1 nm

10−12 m = 1 pm

10−7.5 m ≈ 31.6 nm

10−10.5 m ≈ 31.6 pm

Nanoscale:Characteristic length scale inNanocomputers

Microscale: Characteristic length scale inMicrocomputers

10−4.5 m ≈ 31.6 µm

Picoscale:Characteristic length scale inPicocomputers (if possible)

Nearnano-scaleFarnano-scale

~Atom size

Trend of minimum transistor switching energy

1

10

100

1000

10000

100000

1000000

1995 2005 2015 2025 2035

Year of First Product Shipment

Min

tran

sist

or s

witc

hing

ene

rgy,

kTs

High

Low

trend

(½CV2 gate energy calculated from ITRS ’99 geometry/voltage data)

What is entropy?What is entropy?• First was characterized by Rudolph Clausius in 1850.First was characterized by Rudolph Clausius in 1850.

– Originally was just defined as Originally was just defined as heat heat ÷÷ temperature temperature..– Noted to never decrease in thermodynamic processes.Noted to never decrease in thermodynamic processes.– Significance and physical meaning were mysterious.Significance and physical meaning were mysterious.

• In ~1880’s, Ludwig Boltzmann proposed that entropy In ~1880’s, Ludwig Boltzmann proposed that entropy SS is the logarithm of the number is the logarithm of the number NN of states, of states, SS = = kk ln ln NN– What we would now call the information capacity of a systemWhat we would now call the information capacity of a system– Holds for systems at equilibrium, in maximum-entropy stateHolds for systems at equilibrium, in maximum-entropy state

• The modern consensus that emerged from 20The modern consensus that emerged from 20thth-century -century physics is that entropy is indeed the amount of physics is that entropy is indeed the amount of unknownunknown or or incompressibleincompressible information in a physical system. information in a physical system.– Important contributions to this understanding were made by Important contributions to this understanding were made by

von Neumann, Shannon, Jaynes, and Zurek.von Neumann, Shannon, Jaynes, and Zurek.

Landauer’s 1961 Principle from basic quantum theoryLandauer’s 1961 Principle from basic quantum theory

…Ndistinct

states

Ndistinct

states

……

2Ndistinctstates

Unitary(1-1)

evolution

Before bit erasure: After bit erasure:

Increase in entropy: S = log 2 = k ln 2. Energy lost to heat: ST = kT ln 2

0s0

0sN−1

…

1s′0

1s′N−1

…

…

0s″0

0s″N−1

0s″N

0s″2N−1

…

Adiabatic Cost-Efficiency BenefitsAdiabatic Cost-Efficiency Benefits

1.00E+22

1.00E+23

1.00E+24

1.00E+25

1.00E+26

1.00E+27

1.00E+28

1.00E+29

1.00E+30

1.00E+31

1.00E+32

1.00E+33

2000 2010 2020 2030 2040 2050 2060

Bit-

oper

atio

ns p

er U

S do

llar

Bit-

oper

atio

ns p

er U

S do

llar

Conventional irreversib

le computing

Worst-case r

eversible c

omputing

Best-ca

se revers

ible computing

Scenario: $1,000/3-years, 100-Watt conventional computer, vs. reversible computers w. same capacity.

All curves would →0 if leakage

not reduced.

~1,000×

~100,000×