Slide Set 1 - for ENCM 501 in Winter Term, 2017

Slide Set 1for ENCM 501 in Winter Term, 2017

Steve Norman, PhD, PEng

Electrical & Computer EngineeringSchulich School of Engineering

University of Calgary

Winter Term, 2017

ENCM 501 W17 Lectures: Slide Set 1 slide 2/49

Contents

About these slides

Review of Computer Organization Basics

What does “Computer Architecture” mean?

ENCM 501 Course Topics

Trends in computer system performance

Classes of computer

Trends in Technology

Preliminaries for energy and power use

Energy and power

Looking Ahead


Outline of Slide Set 1

About these slides





Classes of computer



Energy and power

Looking Ahead


About these slides

This is the first of around ten to fifteen large sets of slides thatwill be used for lectures in ENCM 501 in Winter 2017.

It will usually take 2–3 lectures to get through a single set ofslides.

Reading these slides online is not a good substitute forattending lectures—in most lectures I will do some importanthand-written work using the document camera. Please cometo lectures prepared to take some notes.


Typographical conventions

Either bold text or bright red text will be used for emphasis.

The typewriter font will usually be used for code inassembly language, C, or C++. (I might not use the typewriterfont for code if it makes the code too wide to fit in a slide.)

Text in a box is a general description of what could appearwithin a piece of code.

Example: A C do statement has this syntax . . .do

statement

while ( expression );

(Usually statement is a compound statement that startswith { and ends with } .)


Typographical conventions: Italics

Italics will be used two different ways.

One word or a few words in italics will be used to formally orinformally define a term.

Example: A bit is the basic unit of information in a digitalsystem; the value of a bit is either 0 or 1.

An entire sentence in italics indicates a pause to elaborate aconcept or solve a problem under the document camera.

Example: Let’s translate the C statement into a sequence ofassembly language instructions.



About these slides





Classes of computer



Energy and power

Looking Ahead



The next several slides will review concepts covered early inENCM 369, and make comments about how those conceptsare related to current computer design.


Organization of a Simple Computer

......

Bus

Processor

Main

Memory

I/O

Device

I/O

Device

I What is a bus?

I What is the role of theprocessor?

I What is the role of themain memory?

I What does I/O standfor?

I What are importantcategories ofI/O devices?


Operation of our Simple Computer

Within the processor, there is a special-purpose register calledthe program counter (or PC). The PC holds the memoryaddress of the next instruction to be executed.

When our computer is powered up, some kind of initializationcircuit puts a specific address into the PC. After that, theprocessor repeats two steps, Step 1 and Step 2, over and over,until the computer is powered down.


Operation of our Simple Computer, continued

Step 1: Fetch an instruction and update the PC. Copyone or more bytes, starting at the address in the PC, into theprocessor, then make the PC point at the next byte in memorythat wasn’t part of the instruction.

Remark: Step 1 is simple if all instructions are the same size,but can be quite messy if some instructions occupy more bytesof memory than other instructions.

Step 2: Execute the instruction. Perform whatever tiny,simple task is specified by the instruction.

Let’s make an informal list of the kinds of instructions we’veseen in various computer instruction sets, with one or morespecific examples of each kind of instruction.


Step 1, then Step 2, then Step 1, then Step 2 . . .

This model for computer operation is simultaneously seriouslymisleading regarding current computer systems (exceptperhaps the very “lowest-end” embedded computers) andsupremely important.

Let’s make a list of reasons why the model is seriouslymisleading.

But why then is the model still supremely important?



About these slides





Classes of computer



Energy and power

Looking Ahead



It is surprisingly hard to come up with a simple, shortdefinition of computer architecture. It’s kind of an “umbrella”term that includes a bunch of related ideas and activities.Let’s start at the level of instructions . . .

I What instructions are available to applicationsprogrammers? This is often called instruction setarchitecture, or ISA.

I What additional instructions are provided to operatingsystem kernel programmers? (Examples: Instructions toquery system state when an interrupt occurs, to managevirtual memory hardware, to control I/O devices, and soon.)


Now let’s move a down one or two levels of abstraction . . .

I Given the ISA, how exactly are instructions handled byprocessors—how deep are pipelines; can instructions beexecuted out-of-order?How is the memory system organized to minimize loss ofclock cycles in fetching instructions and reading andwriting data?This category of concern is sometimes calledmicroarchitecture or organization.

I Given a microarchitecture, what are good ways toimplement it at the integrated circuit and printed circuitboard levels? These are hardware design problems.


It’s good to have a broad perspective on

architecture

Obviously, ISA choice dictates much about microarchitecture,and microarchitecture dictates much about hardware.

But the influences also work in the opposite direction, fromlower to higher levels of abstraction.

Costs of hardware design, hardware verification, fabrication(chip production) makes some microarchitectures attractiveand others less attractive.

Aspects of microarchitecture matter when a new ISA isdesigned or an existing ISA is extended. Preference forrelatively simple, clean microarchitecture might rule out someuseful instructions.



About these slides





Classes of computer



Energy and power

Looking Ahead



I introduction to computer system design goals andperformance measurement (textbook, Chapter 1)

I brief overview of ISA principles (parts of Appendix A)

I memory system design and performance assessment(parts of Appendix B and Chapter 2)

I aspects of instruction-level parallelism (parts ofAppendix C and Chapter 3)

I aspects of thread-level parallelism (TLP) (parts ofChapter 5)

I introduction to programming with TLP (not covered intextbook)



About these slides





Classes of computer



Energy and power

Looking Ahead



The next slide shows a plot of “benchmark” performancescores for various computers, showing the years varioussystems were introduced.

“Performance” here means roughly 1 over the time taken tocomplete a collection of processor-intensive tasks. (We’ll lookmuch more carefully at performance measurement in futurelectures.)

(The text on the plot will be pretty much illegible in theclassroom, but we can still make a few important points bylooking at it.)


1

5

9

13

18

24

51

80

117

183

280

481

649

993

1,267

1,779

3,016

4,1956,043 6,681

7,108

11,86514,387

19,48421,871

24,129

1

10

100

1000

10,000

100,000

1978 1980 1982 1984 1986 1988 1990 1992 1994 1996 1998 2000 2002 2004 2006 2008 2010 2012

Perf

orm

ance (

vs. V

AX

-11/7

80)

25%/year

52%/year

22%/year

IBM POWERstation 100, 150 MHz

Digital Alphastation 4/266, 266 MHz



AlphaServer 4000 5/600, 600 MHz 21164

Digital AlphaServer 8400 6/575, 575 MHz 21264Professional Workstation XP1000, 667 MHz 21264A

Intel VC820 motherboard, 1.0 GHz Pentium III processor

IBM Power4, 1.3 GHz

Intel Xeon EE 3.2 GHz AMD Athlon, 2.6 GHz

Intel Core 2 Extreme 2 cores, 2.9 GHz

Intel Core Duo Extreme 2 cores, 3.0 GHz

Intel Core i7 Extreme 4 cores 3.2 GHz (boost to 3.5 GHz) Intel Xeon 4 cores, 3.3 GHz (boost to 3.6 GHz)

Intel Xeon 6 cores, 3.3 GHz (boost to 3.6 GHz)

Intel D850EMVR motherboard (3.06 GHz, Pentium 4 processor with Hyper-Threading Technology)

1.5, VAX-11/785

AMD Athlon 64, 2.8 GHz

Digital 3000 AXP/500, 150 MHz

HP 9000/750, 66 MHz

IBM RS6000/540, 30 MHz

MIPS M2000, 25 MHz

MIPS M/120, 16.7 MHz

Sun-4/260, 16.7 MHz

VAX 8700, 22 MHz

AX-11/780, 5 MHz

Image is Figure 1.1 from Hennessy J. L. and Patterson D. A., ComputerArchitecture: A Quantitative Approach, 5nd ed., c© 2012, Elsevier, Inc.


Performance ratio, 2010 compared to 1978: about 24,000 to 1. Inother words, what took about 7 hours in 1978 took about 1 secondin 2010.

From 1986 to 2003, the average annual performance improvementwas 52% per year.

From 2003 to 2010, the average annual performance improvementwas 22% per year—the pace of improvement has slowed in recentyears.

There have been comparable improvements in telecommunicationbandwidth and data storage capacity.

The result is a pattern that has been seen over and over: Computerapplications go from impossible to practically unaffordable tocheap and commonplace over periods of several years.



About these slides





Classes of computer



Energy and power

Looking Ahead


Classes of computerIn Section 1.2, Hennessy and Patterson divide computersystems into five classes. Knowing what they mean will help infollowing the textbook!

I personal mobile device (PMD): things like smartphonesand tablets.

I desktop: what most of us would call “desktops”, andalso laptops. This is a somewhat unusual definition, butmakes sense as use cases and requirements are broadlysimilar.

I servers

I clusters and warehouse-scale computers: systems largeenough to support operations like Google, Amazon, etc.

I embedded computers: computers built-in to machinessuch as appliances, cars, telecom infrastructure.


Also in Section 1.2, Hennessy and Patterson make somedistinctions between various kinds of parallelism in hardwaredesign.

You can read that material to get a general idea about thediverse forms of parallel computation, but we won’t worryabout the details until much later in the course.

A good “takeaway”: If somebody tells you in a vague way thatan algorithm uses parallel processing, you should ask, Whatkind of parallel processing?



About these slides





Classes of computer



Energy and power

Looking Ahead



Textbook reference: Section 1.4.

Moore’s law is attributed to Gordon E. Moore, a co-founder ofIntel. The idea dates back at least to 1965, and probablyearlier. (There is a lengthy historical discussion on theWikipedia page for Moore’s Law.)

It isn’t really a physical law; it was more of an observation andprediction about integrated circuit (IC) technology. Thegeneral projection was that the number of transistors in atypical state-of-the-art IC chip would double every two years orso.


Moore’s law example

Transistor counts for some famous Intel processors . . .

processor year # transistors clock frequency80386 1985 275 thousand 16–25 MHz80486 1989 1.2 million 25–100 MHzPentium 1993 3.2–4.5 million 60–300 MHzPentium II 1997 7.5 million 233-450 MHz

(Moore’s law didn’t make an explicit forecast about clockspeed, but decreasing transistor size tended to correspond todecreasing transistor switching time.)

Using 1985 as a starting point, let’s estimate the transistorcount for an Intel Core i7 chip in 2010.

Data source: Table 7.7 in Harris D. M. and Harris S. L., Digital Designand Computer Architecture, 2nd ed., c© 2013, Elsevier, Inc.


Limits to Moore’s law

Moore’s law has been reasonably accurate for much longerthan might have been predicted decades ago.

Your instructor is neither an IC design expert nor an IC processexpert, so will refrain from giving a detailed opinion aboutwhen Moore’s law will finally fail.

“Node size” in the latest available Intel processors is10 nanometers, and Intel is projecting a 5-nanometer “node”several years from now, so it seems that Moore’s law hasn’tyet run into a wall.

However, even at 10 nanometers, linear transistor dimensionsare now in the tens of atoms in a silicon crystal lattice, soit’s clear that the number of “shrinks” left must be limited.


Trends in Technology, continued

DRAM (dynamic RAM) is the IC technology used to buildmain memories for the last several decades. Moore’s law hasapplied to DRAM just as it has to processor chips—see the“Mbits/DRAM chip” row in Figure 1.10 in the textbook.

Unfortunately, improvement of DRAM latency hasn’t nearlymatched improvement of DRAM capacity, necessitating thedesign of complex caching systems within memoryhierarchies, and limiting the performance of programs thatneed to access large amounts of memory.

Density of nonvolatile storage—magnetic disks and Flashchips—has also seen decades of exponential growth.


Bandwidth and latency

These terms are very important.

Bandwidth, generally, is the peak rate at which simple taskscan be performed, e.g., arithmetic operations per secondwithin a processor, or bytes transferred per second between aDRAM module and a processor chip. (This definition is relatedto but not exactly the same as the definition of bandwidth inother areas of engineering, such as signal processing.)

Latency is the time delay between the moment a task isstarted and the moment that task is completed.


Bandwidth and latency, continued

Relatively poor latency is a major design issue.

Current DRAM latency is approximately 100 times a processorclock cycle, meaning that to get decent processor performance, thevast majority of instruction fetches, data loads and data storesmust not require actual DRAM access—instead most memoryaccesses need to be handled by caches on the processor chip.

Magnetic disk drive latency is horrible compared to speed of justabout anything implemented entirely with integrated circuits. 2010numbers . . .

kind of time interval duration (ns)

processor clock cycle 0.3DRAM latency 37disk latency 3,600,000



About these slides





Classes of computer



Energy and power

Looking Ahead



Here is a model for a generic CMOS logic gate . . .

pull-up

network

pull-down

network

VDD

C

(load capacitance)

gate outputgate inputs

C is the sum of all wire capacitances and gate inputcapacitances driven by our generic gate.


Models for our CMOS gate trying to output 0 or 1 . . .

VDD

C

gate output

RPU

RPD

VDD

C

gate output

RPU

RPD

Which circuit (left or right) is trying to generate a logic 0output, and which is trying to generate a logic 1?


VDD

C

gate output

RPU

RPD

VDD

C

gate output

RPU

RPD

What are the energy flows when the gate output goes fromlogic 0 to logic 1? What are they when the gate output goesfrom logic 1 to logic 0?


The next two slides try to show why energy losses (heatgeneration) in 1→ 0 and 0→ 1 transitions of CMOS gateoutputs are both 1

2CV 2

DD, regardless of how well or poorly thepull-down and pull-up networks conduct.

My math depends on a crude resistor-and-switch model forpull-down and pull-up networks, but I’m pretty sure the sameresults can be derived without making such rough assumptionsabout NMOS and PMOS transistors.

(This course is Computer Architecture, not Digital CMOSVLSI, so I’m not going to put any more time into this issue!)


1→ 0 transition: Let t = 0 be the instant when the inputswitches to cause a 1→ 0 transition on the output. (In reality,input changes are not instant.)

VDD

C

Vout

RPU

RPD

Vout(t) = VDD exp

(−t

RPDC

)So energy lost in RPD is∫ ∞

t=0

Vout(t)2

RPDdt

=V 2DD

RPD

∫ ∞

t=0

exp

(−2t

RPDC

)dt

=V 2DD

RPD

[−RPDC

2exp

(−2t

RPDC

)]∞t=0

=1

2CV 2

DD


0→ 1 transition: Let t = 0 be the instant when the inputswitches to cause a 0→ 1 transition on the output.

VPU

+

−

VDD

C

Vout

RPU

RPD

Vout(t) = VDD

(1− exp

(−t

RPUC

))VPU(t) = VDD exp

(−t

RPUC

)Energy lost in RPU is∫ ∞

t=0

VPU(t)2

RPUdt

=V 2DD

RPU

[−RPUC

2exp

(−2t

RPUC

)]∞t=0

=1

2CV 2

DD



About these slides





Classes of computer



Energy and power

Looking Ahead


Energy and power

Power is the time rate of energy use. (That should not be anew idea for 4th-year engineering students!)

instantaneous power =d

dtenergy use

average power =energy use over time interval

duration of time interval


Energy and power use of a single logic gate

The energy spent per clock cycle of a gate with an output thatmakes a 0→ 1 or 1→ 0 transition every single clock cycle is

1

2C VDD

2.

If the clock period is T , the frequency is f = 1/T , so thepower use by the gate is

1

2C VDD

2 /T =1

2C VDD

2 f .

The equations are correct but an assumption here is incorrect.Why is this not a good model for power use by a logic gate ina processor circuit?


Energy and power use of a processor chip (1)

A useful concept, unfortunately not mentioned in Section 1.5of your textbook, is a, the activity factor.

Let Ctotal be the sum of all of the capacitive loads for all of thelogic gates in an IC. Then a Ctotal is the average capacitive loadthat actually does a 0→ 1 or 1→ 0 transition in a clock cycle.

Why is a much less than 1 for a modern processor chip?

How could a be greater than 1 for certain small regionswithin a modern processor chip?

Which is a better way to think?

I a is not hard for engineers to estimate, and is pretty muchdetermined by the design of a processor chip.

I a is scarily unpredictable.


Energy and power use of a processor chip (2)

Two formulas, assuming that a varies over time . . .

Energy used and heat that must be dissipated in a single clockcycle, due to switching:

Edynamic =1

2a(t) Ctotal VDD

2.

Power consumption:

Pdynamic =1

2a(t) Ctotal VDD

2f .


Energy and power use of a processor chip (3)An ideal CMOS logic gate does not consume any power whenits output is not switching, because either its pull-up networkor its pull-down network is completely turned off.

In real CMOS ICs, however, there are are various paths forcurrent to leak from VDD to ground:

Pstatic = VDD Ileakage

This is a major concern at both ends of the computingspectrum:

I It gradually drains batteries in battery-powered embeddedsystems.

I It wastes energy in servers that spend significant timeidle, waiting for tasks to arrive.


Both energy and power matter in processor design

Because most processors are idle much of the time, energyspent on a typical task is a good measure of the efficiency of aprocessor.

However, power at maximum load is critical as well . . .

I The power supply must be able to supply the neededcurrent without dropping VDD.

I The cooling system must be capable of removing heat ata rate equal to average power during sustained heavyload.


Energy and power management in processor chips

A simple processor chip is either on or off. When it’s on, thewhole chip is on, and VDD and f are fixed.

More complex processor chips . . .

I turn off idle regions within the chip;

I use DVFS (dynamic voltage-frequency scaling)—VDD andf go up and down with the processor load.

DVFS relies on the fact that a CMOS circuit can operatecorrectly over a wide range of VDD values. Lower VDD is moreenergy-efficient but results in slower switching times, so whenVDD is reduced, f must be reduced as well.



About these slides





Classes of computer



Energy and power

Looking Ahead


Looking Ahead

In ENCM 501, we won’t study the material textbookSection 1.6 (Trends in Cost) and Section 1.7 (Dependability).

We will look in detail at some of the topics in Sections 1.8 and1.9, then move on to Appendix A, which is about instructionset design.

Slide Set 1 - for ENCM 501 in Winter Term, 2017

Documents

Transcript of Slide Set 1 - for ENCM 501 in Winter Term, 2017