GPU and PC System Architecture UC Santa Cruz BSoE – March 2009John Tynefield / NVIDIA Corporation

My Goals

Survey history and direction of GPU/PC system architectureDemonstrate the process of system level architectural problem solvingMotivate some of you to become architects

Disclaimers

I work for NVIDIAPublic InfoAll numbers and dates approximate

Rounding is our friendNo bus/processor is 100% efficient, etc, etc

All examples are meant to be illustrativeNot comprehensive

“ there were >40 gfx companies in 1995”

About Me

I love games and graphicsI love building things

Structure

Intro to PC and GPU ArchitectureA Sampling of Architectures

1996 - Voodoo Graphics / Pentium2000 - GeForce 256 / P32004 - GeForce 6800 / P42008 - Geforce GTX280 / Core2

Ideas for the future of the platform

What do architects do?

Impose structure on complex design problemsMake tradeoffsValidate high risk design betsStructure verification

Why this is a great time to be an Architect

Radical design mobilityI have contributed to 10 completely new processor designs

7 of which shipped in millions of units.Steep competition

Not for everybodyChanging the World…no…really!

Heterogeneous many core computing is here to stay and it has changed the nature of computing

Design Tension

Fixed Function vs. ProgrammableScalar vs. VectorBandwidth vs. LatencyIn Order vs. Out of OrderLimited vs. Unlimited ( virtualized ) resources

Technology Trends

CPUs get fasterGPUs get fasterInterconnects get fasterMemory gets fasterMemory gets denserLatency increasesFeature load increasesPhysics intrudes more and more

All at different rates

1996 2000 2004 20080%

5000%

10000%

15000%

20000%

25000%

30000%

CPU CoresCPU Interconnect BWGPU CoresGPU Interconnect BWSystem Memory BWGPU Memory B/W

The long time horizon

The Awesome ideas of now take 2+ years to reach marketAwesome depreciates rapidly

PredictableSilicon Process RoadmapPC Arch Roadmap3rd Party Component RoadmapYour capabilities and resources

UnpredictableMarket Shifts ( commodity prices, supply shocks )3rd Party Strategic Errors ( os/platform/partner slips )Innovative Competition ( N-way struggle for design initiative )

GPU Memory

GPUCPU

Ultra Simplified PC Anatomy

CPU Core Logic GPU

GPU Memory

System Memory

ProcessorProcessor

Processor

DRAM MGMTDRAM

MGMT

Ultra Simplified GPU Anatomy

Host Logic

DRAM MGMT

Ultra Simplified GPU Anatomy (2) ProcessorProcessor

Processor

DRAM MGMTDRAM

MGMTHost Logic

DRAM MGMT

Geom Gather

GeomProc

TriangleProc

PixelProc Z / Blend

Memory

GPU Prehistory

1960s – 1970sSingle Purpose BIG IRONE&S, GE, Lockheed, …

1980s – 1990sGeneral Purpose BIG IRONCustom ASICs, WorkstationsSGI, Sun, Intergraph, ..

1994Maybe we can fit this on a single consumer add-in card?

Fast consumer CPUs with floating pointTry 3D rendering in fixed point!

PCIVGA and VESAId Software’s DOOMContract Fabrication facilities offering .6 micronASIC design Tools

Enabling Technologies in 1994

1996 3dfx - Voodoo Graphics

PIO Programming ModelPure Pipelined GraphicsPartial Triangle Setup – FP32Fixed Point Integer Texture Mapping and Gouraud ShadingZ Buffer and Full OpenGL BlendingAll at 1 PPC, all the time, with no caches

32-bit PCI - .09 GB/s 128-bit EDO 50 Mhz DRAM - .8 GB/s

Voodoo Graphics System Architecture

Geom Gather

GeomProc

TriangleProc

PixelProc Z / Blend

CPU Core Logic FBI

FB Memory

System Memory

TMU TEX Memory

GPUCPU

Arch Decision – Triangle Setup

Target 3D Triangle with texture and Gouraud shading3 * XYW RGBA ST = 72 bytes/triangle pre setup

32-bit PCI 33Mhz – 90 MB/s 1.25 M triangles / second speed of light ( 1M is magic )

Observe that post setup3 * XY WRGBAST start values + screen space derivatives + Area

76 bytes/triangle – 1.18M Tris ( still magic )Setup can be coded on Pentium in ~100 clocks

1M triangles on P100 ( mktg happy )Data-limited setup on chip - >10% die cost

Typical game scenes <<1000 triangles/frame

2000 Nvidia GeForce 256

Decoupled input queuingHardware Transform & Lighting

FP32 FF TransformFP22 FF Lighting

Complex fixed function pixel shading4 Pipelines

AGP4X – 1.06 GB/s 256 Bit DDR 300 Mhz Memory – 19.2 GB/s

GeForce 256 System Architecture

Geom Gather

GeomProc

TriangleProc

PixelProc Z / Blend

CPU Core Logic GPU

GPU Memory

System Memory

GPUCPU

Architecture Detail – Combiners

Logical fixed function extension of OpenGL MachineSurface Color = Diffuse * Texture + Specular

Diffuse Color Texture

Specular

Multi Texture

If one texture is good, more are betterDiff * ( Tex1 + Tex 2 ) + Spec or Diff * Tex1 * Tex2 or …

Diffuse Color Texture

0.0

Texture

Specular

Diffuse Color Texture

Texture2

1.0

Specular

Combiners

Cascading Mux / SOP / Mux / SOP pipelineVery, flexible, harder to program with deeper nesting

Everything is full speed!

A MUX B MUX

AB Partial

C MUX D MUX

CD Partial

Inputs for Next Stage of Pipeline

Texture Fog Light

Programmable Shading

But the future was obviously Renderman-like shaders

normal surfaceN; color C = { 1.0, 0.5, 0.0 }; normal lightDirection;

Ci = C * dot ( surfaceN, lightDirection );

2004 Nvidia GeForce 6800

Fully general Vertex and Pixel ISA6 Geometry Processors16 Pixel Processors

Deep recirculating pipelines to hide latencyFP32 datapath end to end

AGP8X – 2.11 GB/s256 Bit 700 Mhz GDDR3 – 44 GB/s

GeForce 6800 System Architecture

Geom Gather

GeomProc

TriangleProc

PixelProc

Z / Blend

CPU Core Logic GPU

GPU Memory

System Memory

GPUCPU

Physics and AI

Scene Mgmt

Architecture Decision – Tex/Shader Structure

Problem: Build a general programmable pipelineOptimize for common workloadsTEX – BLEND – FOGCommon Game Shaders ( eg. Doom 3 )

Plan A – Uncoupled

ElegantSmall fundamental unitMany “passes” for common shaders

TBF TEXMTHTEXBLNDBLND R

egis

ters

Texture Math

Less ElegantLarger Fundamental UnitSingle pass for common shadersGood scaling for longer shadersBig perf / area win given workloadsNot forward looking

Plan B - Coupled

Reg

iste

rs

Math

Texture

Math

2008 - GeForce GTX280

Fully unified programmable architecture240 instances of the same processorIEEE FP32 and FP64

Gen2 PCIE – 8GB/s512 bit 1100 Mhz GDDR3 – 144 GB/s

GeForce GTX280 System Architecture

Geom Gather

GeomProc

TriangleProc

PixelProc

Z / Blend

CPU Core Logic GPU

GPU Memory

System Memory

GPUCPU

Physics and AI

Scene Mgmt

Architecture Decision – Heterogeneous Computing Support

Build a bigger ChipRadically improve ability of GPU to share work with the CPU

Thread

Local Memory

Grid 0

. . .

GlobalMemory

. . .

Grid 1

SequentialGridsin Time

Block

SharedMemory

Register File

Computing Support

Add Efficient Thread LaunchingAdd General Load / Store Instructions and DatapathAdd Shared MemoryAdd computational loads to performance design requirements

Future Graphics Directions

Higher densityHigher refreshHigher dynamic rangeUbiquity

Lower PowerShaving off the last burrs

Global IlluminationHigher quality modelingVirtualized resources at interactive rates

Future PC Architecture Directions

Highly Integrated – Low CostRequire a minimum visual feature setWeb/video/run today’s apps

And everyone elseDifferentiated PCsMore bandwidth and more parallel horsepower

More mature unified programming models C on CUDADX11 OpenCL

More resource virtualization

Q & A