Computer Graphics and Image ProcessingIntroductionPart 1 – Lecture 1

1

Christof LutterothChristof LutterothOriginally from Berlin, Germanyg y , yMy research interests:HCI model-based SE CGHCI, model based SE, CG, …Contact details:

l tt th@ kl dlutteroth@cs.auckland.ac.nzPhone 373-7599 84478Office: 303 - 494 (4th level CompSci building)Office: 303 494 (4 level CompSci building)

If you have questions, come to my office☺Particularly available right after each lectureParticularly available right after each lecture

2

P t I O iPart I Overview

Live as if your were to ydie tomorrow. Learn as if you were to live forever.

(Gandhi)(Gandhi)3

Part I OverviewPart I OverviewWeek 1: 2D and 3D Geometryee a d 3 Geo e y

Week 2: Color Theory (Timo)

Week 9: Illumination and Shading

Week 10: Ray TracingWeek 10: Ray Tracing

Assignment 3

4Raytracing image thanks to Tim Babb

g

4

TimetableTimetableWhen What WhereTuesday 3pm Lecture Lib B15Wednesday 3pm Lecture Lib B15Wednesday 3pm Lecture Lib B15Thursday 3pm Lecture Lib B15Monday 3pm Tutorial LgeChemMonday 3pm Tutorial LgeChem

Class Rep ?

2nd June 7pm Assignment 3 (8 1/3 %) ADBW k 7 T t (20%) TBAWeek 7 Test (20%) TBATBA Assignment 1, 2 TBATBA Exam (55%) TBA

5

ReferencesReferencesF.S. Hill. Computer Graphics using OpenGL (2nd or 3rd Edition). Prentice Hall.Nick Efford. Digital Image Processing:A Practical Introduction Using JavaA Practical Introduction Using Java.Addison Wesley.

C/C++:C Reference Manualhtt // b ll l b / h /d /http://www.cs.bell-labs.com/who/dmr/cman.psBruce Eckel - Thinking in C++ http://www ibiblio org/pub/docs/books/eckel/http://www.ibiblio.org/pub/docs/books/eckel/C++ for Java Programmershttp://www.jgcampbell.com/cpp4jp/cpp4jp.pdfhttp://www.jgcampbell.com/cpp4jp/cpp4jp.pdfSee also CS373 resources page

6

IMPORTANT ANNOUNCEMENT

Departmental Policy on Cheating on Assignments1. The Computer Science Department uses many ways to check that the work stu-

dents submit for marking is their own and was not produced by or copied fromdents submit for marking is their own and was not produced by, or copied from, someone else. In particular, for most programming assignments, the department uses a program comparison program to automatically compare all submissions from students Also Turnitin com may be used on essays and reports Thisfrom students. Also Turnitin.com may be used on essays and reports. This detects similarity to online material and submitted works in its own database.

2. All assignments where plagiarism is detected are checked for similarity by the course supervisor or another suitable person associated with the coursecourse supervisor or another suitable person associated with the course.

3. All assignments deemed to be too similar are automatically allocated a zero mark. 4. All students who submitted these assignments are entered in the duplicate

i t i tassignment register. 5. A standard email (see below) is sent to these students. 6. Repeat offenders may be referred to the University Disciplinary Committee.y y y

For more details see http://www.cs.auckland.ac.nz/administration/policies/CheatingPolicy.php 7

3D

2D

3D

4D

DIGITAL IMAGES

8

How to Get Digital Images?How to Get Digital Images?1. 2D real → 2D digital2. 3D real → 2D digital3. 4D real → animated 2D digital3. 4D real → animated 2D digital

4. 3D real → 3D digital

2D slices

5 4D real → 4D digital

2D slicesof 3D volume3D Scanner MRIStereoscopic Camera

5. 4D real → 4D digitalMotion Capturing

6. Synthetic or computer generated9

What to Do with Digital Images?What to Do with Digital Images?Image Processing: process them to get new imagesComputer Vision: analyze them to get information about what is in the imagegComputer Graphics: generate themComputer Vision often seen as part of Image ProcessingComputer Vision often seen as part of Image Processingor Image Analysis

OUTPUTDescriptions Images

OUTPUT

Descriptions Computer Graphics

Images Computer Vision Image ProcessingNPU

T

10

Images Computer Vision Image ProcessingIN

Pixels and ResolutionPixels and ResolutionPixel or pel (picture element)Position (x,y) + signal value v (greyscale or colour)

y

Position: (512, 456)Value: (10 255 8)x Value: (10, 255, 8)

Origin (0,0) of pixel coordinates sometimes in top left corner

Resolution: how many pixels? width × heightResolution: how many pixels? width × heightSpatial resolution: image pixels per cm or inch (in x and y)Can be used to convert pixel coordinates into physical p p ycoordinates

11

Encoding of ColorsEncoding of ColorsBit-depth: number of bits used to represent each pixel's value (typically 1 8 24 or 32)value (typically 1, 8, 24 or 32)Binary image: bit depth is 1;only code values 0 (black) and 1 (white)only code values 0 (black) and 1 (white)Scalar/monochrome/greyscale image:

scalar code values (e.g. just a single number per color)only grey values (from black to white)

d land no colourVector-valued image

t d lvector code values (e.g. several numbers per color)All the colors can be representedAll the colors can be represented

12

Defining Images MathematicallyDefining Images MathematicallyImages can be defined on an M × N arithmetic grid (or lattice)

Pi l di d y

( ){ }NyMxyxNM ≤≤∧≤≤= 11:,,RPixel coordinates x and ywith x = 1,…,M ; y = 1,…,N

y

N

Image as a function f: R→ VV is a set of signal values,e.g. grey levels or colorsExample: 3

4

pixel at position (100, 100)has color 255,i f (100 100) 255

x012

i.e. f (100, 100) = 25513

0 1 2 3 4 M

Image ProcessingImage ProcessingGeometric transformations:resizing, rotation, deformations, …Color transformations:quantization, conversion,color adjustment, …Compositing:combination of two or more imagesMany other operations

14

Computer VisionComputer VisionUse computer to do things similar to human vision, using image processing, artificial intelligence, biology & physicsUsually dealing with 2D images of 3D sceney g gOften real-time and part of a larger system (e.g. robot)E.g. scene reconstruction, image restoration, objectE.g. scene reconstruction, image restoration, object recognition, tracking, motion estimation, event detection, …

15

Human vs Computer VisionHuman vs. Computer VisionHuman Vision

Subjective, unstable,inaccurate in measurementsInvolves active interaction withInvolves active interaction with environmentExploits experience, knowledge,Exploits experience, knowledge, context But unique capabilities to describe

d d dand understand…

Computer VisionpObjective accurate measurements But low capabilities to describe and understand…

16

Generating Digital ImagesGenerating Digital ImagesNeed a 3D scene consisting of:

Objects (3D models)Light sourcesLight sourcesCamera (viewer, eye)Parameters specifying how light interacts with the sceneParameters specifying how light interacts with the scene

Light parameterse.g. type of light (light bulb, spotlight), color, brightnessMaterial parameterse.g. color, texture, transparencyRendering parametersRendering parameterse.g. algorithm, quality, resolution

17

Objects (3D Models)Objects (3D Models)Objects are made up of parts (often also of other objects):

Vertices: 3D points used to define modelEdges: lines between verticesEdges: lines between verticesFaces: polygons bounded by edgesMesh: surface made up of connected polygonsMesh: surface made up of connected polygons

Wireframe modelshowing mesh withshowing mesh withvertices, edges andfaces

18

Smoothed and shaded solid model

The CameraThe CameraView point: where is the camera?View direction: which direction does it point?View orientation: where does the top of the camera point?View orientation: where does the top of the camera point?E.g. is the photo taken upside down?P j ti h t t f l d th h ?Projection: what type of lens does the camera have?

View plane (screen image)Rendering:

Project scene onto view plane in front of the cameraProject scene onto view plane in front of the cameraCompute the color of each pixel on the view plane

19

Image Rendering AlgorithmsImage Rendering Algorithms Raytracing

Trace rays from viewpoint through pixels on the screen into the scene→ high quality but slow

Pixels on screenPolygon RenderingProject polygons representing objects onto the view planeProject polygons representing objects onto the view plane and render pixels covered by the projected objectsby the projected objects→ fast with graphics hardware

Pixels on screenPolygon in scene

20

Preview: Polygon RenderingPreview: Polygon Renderingy y

Modeling ViewScene Primiti es

xz

xz

Transformation TransformationWorld CoordinatesScene PrimitivesMaster Coordinates

vy +1

Projection Ill i tiu

nx

z-1

+1-1 Projection

Transformation IlluminationView CoordinatesNormalized

Device Coordinates

1

ViewportTransformationClipping RasterizationDevice

Coordinates

21

The OpenGL Rendering Pipeline

SUMMARY

22

SummarySummaryThings we can do with images:

Image Processing: process them to get new imagesComputer Vision / Image Analysis:analyze them to get higher-level informationComputer Graphics: generate them

Images:pixels with encoded colors, (spatial) resolution,

b t d ith ti idcan be represented on arithmetic gridFor generating images:3D scene with objects (vertices edges faces meshes) light3D scene with objects (vertices, edges, faces, meshes), light sources, camera

23

QuizQuiz1. What is the spatial resolution of an image?2. What is Computer Vision?3. What are the differences between Computer Vision and3. What are the differences between Computer Vision and

human vision?4 What does a 3D scene consist of?4. What does a 3D scene consist of?5. What are vertices, edges and faces?

24