Post on 04-Feb-2022
𝑰 𝒙, 𝒙′ = 𝒈(𝒙, 𝒙′) 𝝐 𝒙, 𝒙′ + න𝑺
𝝆 𝒙, 𝒙′, 𝒙′′ 𝑰 𝒙′, 𝒙′′ 𝒅𝒙′′
INFOMAGR – Advanced GraphicsJacco Bikker - November 2018 - February 2019
Lecture 1 - “Introduction”
Welcome!
Website
http://www.cs.uu.nl/docs/vakken/magr
▪ Site information overrules official schedule▪ Downloads, news, slides, deadlines, links
Please check regularly.
Slack
▪ Please join! Link on website.
INFOMAGR
Advanced Graphics – Introduction 3
Abstract
In this course, we explore physically based rendering, with a focus on interactivity.
At the end of this course, you will have a solid theoretical understanding of efficient physically based light transport.
You will also have a good understanding of acceleration structures for fast ray/scene intersection for static and dynamic scenes.
You will have hands-on experience with algorithms for efficient realistic rendering of static and dynamic scenes using ray tracing on CPU and GPU.
INFOMAGR
Advanced Graphics – Introduction 4
Concrete / informal:
1. You’ll know how a photo-realistic image is produced
2. You know how to do this quickly / efficient
3. You have built such a renderer
4. You have built an interactive ray tracer
5. You know how to do this on the GPU
6. You got a great score
7. You had fun
Topics
We will cover the following topics:
▪ Ray tracing fundamentals;
▪ Whitted-style ray tracing;
▪ Acceleration structure construction;
▪ Acceleration structure traversal;
▪ Data structures and algorithms for animation;
▪ Stochastic approaches to AA, DOF, soft shadows, … ;
▪ Path tracing;
▪ Variance reduction in path tracing algorithms;
▪ Filtering techniques;
▪ Various forms of parallelism in ray tracing.
INFOMAGR
Advanced Graphics – Introduction 5
Lectures
~17 lectures:Tuesday 11:00 – 12:45, Thursday 13:15 – 15:00
~7 working colleges:Thursday 15:15 – 17:00
Attendance is not mandatory, but of course highly recommended.We move fast; missing a key lecture may be a serious problem.
INFOMAGR
Advanced Graphics – Introduction 6
Literature
Papers and online resources will be supplied during the course.
Slides will be made available after each lecture.
Recommended literature:
Physically Based Rendering – From Theory to Implementation, Pharr & Humphreys. ISBN-10: 9780128006450.
Recently, the 3rd edition was released for free: www.pbr-book.org
INFOMAGR
Advanced Graphics – Introduction 7
Dependencies
It is assumed that you have basic knowledge of rendering (INFOGR) and associated mathematics.
You also should be a decent programmer; this is explicitly not a purely theoretical course. You are expected to verify the theory and experience the good and the bad.
A brief introduction to GPGPU and SIMD will be provided for those that did not take INFOMOV.
INFOMAGR
Advanced Graphics – Introduction 8
Resources
You will develop a ray tracing testbed for assignment 1. As a starting point, several ‘templates’ are available:
Tmpl85.00aA basic C++ framework for graphics programming, which opens a window, provides a 32-bit RGB framebuffer, and some basic helper classes.
gpu_lab (available halfway the course)C++ frameworks for GPGPU via CUDA and OpenCL.
Template_C#A basic C# template which uses OpenTK to provide a 32-bit RGB framebuffer as well as OpenGL functionality, and Cloo for using OpenCL.
INFOMAGR
Advanced Graphics – Introduction 9
Assignments
1. (weight: 1):Ray tracing framework
For this assignment, you prepare a testbed for subsequent assignments.
2. (weight: 1):Acceleration structures
In this assignment, you expand your testbed with efficient acceleration structure construction and traversal. This enables you to run Whitted-style ray tracing in real-time.
3. (weight: 2):Final assignment
In this assignment, you either implement an interactive path tracer, or a rendering algorithm you chose, using CPU and/or GPU rendering.
INFOMAGR
Advanced Graphics – Introduction 10
Exam
One final exam at the end of the block.
Materials to study:
▪ Slides▪ Notes taken during the lectures▪ Provided literature▪ Assignments
INFOMAGR
Advanced Graphics – Introduction 11
Grading & Retake
Final grade for assignments 𝑃 = (𝑃1 + 𝑃2 + 2 ∗ 𝑃3) / 4Final grade for INFOMAGR 𝐺 = (2𝑃 + 𝐸) / 3
Passing criteria:
▪ 𝑃 ≥ 4.50▪ 𝐸 ≥ 4.50▪ 𝐺 ≥ 5.50
Repairing your grade using the retake exam or retake assignment:
▪ only if 5.50 > 𝐺 ≥ 4.00▪ you redo P1 or P2 or P3 or E▪ this replaces the original P1, P2, P3 or E grade.
INFOMAGR
Advanced Graphics – Introduction 12
Ray
A ray is an infinite line with a start point:
𝑃(𝑡) = 𝑂 + 𝑡𝐷, where 𝑡 > 0.
The ray direction 𝐷 is usually normalized.
Recap
Advanced Graphics – Introduction 14
Scene
The scene consists of a number of primitives:
▪ Spheres▪ Planes▪ Triangles
...or anything for which we can calculate the intersection with a ray.
We also need:
▪ A camera (position, direction, FOV, focal distance, aperture size)▪ Light sources
Recap
Advanced Graphics – Introduction 15
Recap
Ray Tracing
World space
▪ Geometry▪ Eye▪ Screen plane▪ Screen pixels▪ Primary rays▪ Intersections▪ Point light▪ Shadow rays
Light transport
▪ Extension rays
Light transport
Advanced Graphics – Introduction 16
Ray setup
A ray is initially shot through a pixel on the screen plane. The screen plane is defined in world space, e.g.:
Camera position: 𝐸 = (0,0,0)
View direction: 𝑉 = (0,0,1)
Screen center: 𝐶 = 𝐸 + 𝑑𝑉Screen corners: 𝑃0 = 𝐶 + −1, −1,0 , 𝑃1 = 𝐶 + 1, −1,0 , 𝑃2 = 𝐶 + (−1,1,0)
From here:
▪ Change FOV by altering 𝑑;▪ Transform camera by multiplying E, 𝑃0, 𝑃1, 𝑃2 with the camera matrix.
Recap
Advanced Graphics – Introduction 17
Ray setup
Point on the screen:
𝑃 𝑢, 𝑣 = 𝑃0 + 𝑢 𝑃1 − 𝑃0 + 𝑣(𝑃2 − 𝑃0)𝑢, 𝑣 ∈ [0,1]
Ray direction (normalized):
𝐷 =𝑃 𝑢, 𝑣 − 𝐸
∥ 𝑃 𝑢, 𝑣 − 𝐸 ∥
Ray origin:
𝑂 = 𝐸
Recap
𝑃0
𝑃1
𝑃2
𝐸
u
v
Advanced Graphics – Introduction 18
Ray setup
Alternatives:
▪ Taking into account HMD lens distortion
Recap
Advanced Graphics – Introduction 19
Ray setup
Alternatives:
▪ Taking into account HMD lens distortion▪ Fisheye lens
Recap
Advanced Graphics – Introduction 20
Ray setup
Alternatives:
▪ Taking into account HMD lens distortion▪ Fisheye lens▪ Complex lens system
Recap
Advanced Graphics – Introduction 21
Ray Intersection
Given a ray P(𝑡) = 𝑂 + 𝑡𝐷, we determine the closest intersection distance 𝑡 by intersecting the ray with each of the primitives in the scene.
Ray / plane intersection:
Plane: 𝑃 ∙ 𝑁 + 𝑑 = 0
Ray: 𝑃(𝑡) = 𝑂 + 𝑡𝐷
Substituting for 𝑃(𝑡), we get
𝑂 + 𝑡𝐷 ∙ 𝑁 + 𝑑 = 0
𝑡 = −(𝑂 ∙ 𝑁 + 𝑑)/(𝐷 ∙ 𝑁)
𝑃 = 𝑂 + 𝑡𝐷
Recap
𝑃0
𝑃1
𝑃2
𝐸
Advanced Graphics – Introduction 22
Math reminder, dot product:
𝐴 ∙ 𝐵 = 𝐴𝑥𝐵𝑥 + 𝐴𝑦𝐵𝑦 + 𝐴𝑧𝐵𝑧
𝐴 ∙ 𝐵 = cos 𝜃𝐴 ∙ 𝐵 is: the length of the
projection of A on B.𝐴 ∙ 𝐵 is a scalar
Math notation:
𝑃 is a point, Ԧ𝑃 is a vector𝑡 is a scalar.
Ray Intersection
Ray / sphere intersection:
Sphere: 𝑃 − 𝐶 ∙ 𝑃 − 𝐶 − 𝑟2 = 0
Substituting for 𝑃(𝑡), we get
𝑂 + 𝑡𝐷 − 𝐶 ∙ 𝑂 + 𝑡𝐷 − 𝐶 − 𝑟2 = 0
𝐷 ∙ 𝐷 𝑡2 + 2𝐷 ∙ 𝑂 − 𝐶 𝑡 + (𝑂 − 𝐶)2−𝑟2 = 0
𝑎𝑡2 + 𝑏𝑡 + 𝑐 = 0 → 𝑡 =−𝑏 ± 𝑏2 − 4𝑎𝑐
2𝑎
𝑎 = 𝐷 ∙ 𝐷
𝑏 = 2𝐷 ∙ (𝑂 − 𝐶)𝑐 = 𝑂 − 𝐶 ∙ 𝑂 − 𝐶 − 𝑟2
Recap
𝑃0
𝑃1
𝑃2
𝐸
Negative: no intersections
Advanced Graphics – Introduction 23
Ray Intersection
Efficient ray / sphere intersection:
void Sphere::IntersectSphere( Ray ray ){
vec3 C = this.pos - ray.O;float t = dot( C, ray.D );vec3 Q = C - t * ray.D;float p2 = dot( Q, Q );if (p2 > sphere.r2) return; // r2 = r * rt -= sqrt( sphere.r2 – p2 );if ((t < ray.t) && (t > 0)) ray.t = t;
}
Note:
This only works for rays that start outside the sphere.
Recap
Advanced Graphics – Introduction 24
O
𝐷
Ԧ𝐶
t
𝑄
𝑝2
Observations
Ray tracing is a point sampling process:
▪ we may miss small details;▪ aliasing will occur.
Ray tracing is a visibility algorithm:
▪ For each pixel, we find the nearest object (which occludes objects farther away).
Recap
Advanced Graphics – Introduction 25
Observations
Note: rasterization (Painter’s or z-buffer) is also a visibility algorithm.
Rasterization:
▪ loop over objects / primitives;▪ per primitive: loop over pixels.
Ray tracing:
▪ loop over pixels;▪ per pixel: loop over objects / primitives.
Recap
Advanced Graphics – Introduction 26
An Improved Illumination Model for Shaded Display
In 1980, “State of the Art” consisted of:▪ Rasterization▪ Shading: either diffuse (N · L) or specular ((N · H)n), both
not taking into account fall-off (Phong)▪ Reflection, using environment maps (Blinn & Newell *)▪ Stencil shadows (Williams **)
Goal: ▪ Solve reflection and refraction
Improved model:▪ Based on classical ray optics
Whitted
** : Williams, L. 1978. Casting curved shadows on curved surfaces. In Computer Graphics (Proceedings of SIGGRAPH 78), vol. 12, 270–274.
* : Blinn, J. and Newell, M. 1976. Texture and Reflection in Computer Generated Images. Communications of the ACM 19:10 (1976), 542—547.
Advanced Graphics – Introduction 28
An Improved Illumination Model for Shaded Display*
Physical basis of Whitted-style ray tracing:
Light paths are generated (backwards) from the camera to the light sources, using rays to simulate optics.
Color Trace( ray r )I, N, mat = NearestIntersection( scene, r )return mat.color * DirectIllumination( I, N )
Whitted
* : T. Whitted. An Improved Illumination Model for Shaded Display. Commun. ACM, 23(6):343–349, 1980.
Advanced Graphics – Introduction 29
An Improved Illumination Model for Shaded Display
Color Trace( ray r )
I, 𝑁, mat = NearestIntersection( scene, r )
return mat.color * DirectIllumination( I, 𝑁 )
Direct illumination:
Summed contribution of unoccluded point light sources, taking into account:
▪ Distance to I
▪ Angle between 𝐿 and 𝑁▪ Intensity of light source
Note that this requires a ray per light source.
Whitted
Advanced Graphics – Introduction 30
An Improved Illumination Model for Shaded Display
Color Trace( ray r )
I, 𝑁, mat = NearestIntersection( scene, r )if (mat == DIFFUSE)
return mat.color * DirectIllumination( I, 𝑁 )if (mat == MIRROR)
return mat.color * Trace( I, reflect( r.𝐷, 𝑁 )
Indirect illumination:
For perfect specular object (mirrors) we extend the primary ray with an extension ray:
▪ We still modulate transport with the material color
▪ We do not apply 𝑁 ∙ 𝐿▪ We do not calculate direct illumination
Whitted
Advanced Graphics – Introduction 31
Reflection
Given a ray direction 𝐷 and a normalized surface normal 𝑁, the
reflected vector 𝑅 = 𝐷 − 2(𝐷 ∙ 𝑁)𝑁.
Derivation:
𝑉 = 𝑁(𝐷 ∙ 𝑁)
𝑈 = 𝐷 − Ԧ𝑣
𝑅 = 𝑈 + (−𝑉)
𝑅 = 𝐷 − 𝑁 𝐷 ∙ 𝑁 − 𝑁(𝐷 ∙ 𝑁)
𝑅 = 𝐷 − 2(𝐷 ∙ 𝑁)𝑁
Whitted
𝑅 𝐷
𝑵𝑉
𝑈
Advanced Graphics – Introduction 32
Question 1: For direct illumination, we take into account:
▪ Material color▪ Distance to light source
▪ 𝑁 ∙ 𝐿
Why?
Question 2: We use the summed contribution of all light sources.
Is this correct?
Question 3: Why do we not sample the light sources for a pure specular surface? (can you cast a shadow on a bathroom mirror?)
Question 4: Why do we not apply 𝑁 ∙ 𝐿 to reflections?
Question 5: Prove geometrically that, for normalized vectors 𝐷 and 𝑁, 𝑅 =
𝐷 − 2 𝐷 ∙ 𝑁 𝑁 yields a normalized vector.
Whitted
Advanced Graphics – Introduction 33
An Improved Illumination Model for Shaded Display
Handling partially reflective materials:
Color Trace( ray r )
I, 𝑁, mat = NearestIntersection( scene, r )s = mat.specularityd = 1 – mat.specularityreturn mat.color * (
s * Trace( ray( I, reflect( r.𝐷, 𝑁 ) ) ) +
d * DirectIllumination( I, 𝑁 ) )
Note: this is not efficient. (why not?)
Whitted
Advanced Graphics – Introduction 34
An Improved Illumination Model for Shaded Display
Dielectrics
Color Trace( ray r )
I, 𝑁, mat = NearestIntersection( scene, r )if (mat == DIFFUSE)
return mat.color * DirectIllumination( I, 𝑁 )if (mat == MIRROR)
return mat.color * Trace( I, reflect( r.𝐷, 𝑁 )if (mat == GLASS)
return mat.color * ?
Whitted
Advanced Graphics – Introduction 35
Dielectrics
The direction of the transmitted vector 𝑇 depends on the refraction indices 𝑛1, 𝑛2 of the media separated by the surface. According to Snell’s Law:
𝑛1𝑠𝑖𝑛𝜃1 = 𝑛2𝑠𝑖𝑛𝜃2
or
𝑛1
𝑛2𝑠𝑖𝑛𝜃1 = 𝑠𝑖𝑛𝜃2
Note: left term may exceed 1, in which case 𝜃2 cannot be computed. Therefore:
𝑛1
𝑛2𝑠𝑖𝑛𝜃1 = 𝑠𝑖𝑛𝜃2 ⟺ 𝑠𝑖𝑛𝜃1 ≤
𝑛2
𝑛1➔ 𝜃𝑐𝑟𝑖𝑡𝑖𝑐𝑎𝑙 = arcsin
𝑛2
𝑛1sin 𝜃2
Whitted
𝑇
𝐷𝑵
𝑛1
𝑛2
𝜃2
𝜃1
Advanced Graphics – Introduction 36
Dielectrics
𝑛1
𝑛2𝑠𝑖𝑛𝜃1 = 𝑠𝑖𝑛𝜃2 ⟺ 𝑠𝑖𝑛𝜃1 ≤
𝑛2
𝑛1
𝑘 = 1 −𝑛1
𝑛2
2
1 − 𝑐𝑜𝑠𝜃12
𝑇 = ൞
𝑇𝐼𝑅, 𝑓𝑜𝑟 𝑘 < 0
𝑛1
𝑛2𝐷 + 𝑁
𝑛1
𝑛2𝑐𝑜𝑠𝜃1 − 𝑘 , 𝑓𝑜𝑟 𝑘 ≥ 0
Note: 𝑐𝑜𝑠𝜃1 = 𝑁 ∙ −𝐷, and 𝑛1
𝑛2should be calculated only once.
* For a full derivation, see http://www.flipcode.com/archives/reflection_transmission.pdf
Whitted
𝑇
𝐷𝑵
𝑛1
𝑛2
𝜃2
𝜃1
Advanced Graphics – Introduction 37
Dielectrics
A typical dielectric transmits and reflects light.
Whitted
𝑇
𝐷𝑵𝑅
Advanced Graphics – Introduction 38
Dielectrics
A typical dielectric transmits and reflects light.
Based on the Fresnel equations, the reflectivity of the surface for non-polarized light is formulated as:
𝐹𝑟 =1
2
𝑛1𝑐𝑜𝑠𝜃𝑖 − 𝑛2 1 −𝑛1𝑛2
𝑠𝑖𝑛𝜃𝑖
2
𝑛1𝑐𝑜𝑠𝜃𝑖 + 𝑛2 1 −𝑛1𝑛2
𝑠𝑖𝑛𝜃𝑖
2
2
+
𝑛1 1 −𝑛1𝑛2
𝑠𝑖𝑛𝜃𝑖
2
− 𝑛2 cos 𝜃𝑖
𝑛1 1 −𝑛1𝑛2
𝑠𝑖𝑛𝜃𝑖
2
+ 𝑛2 cos 𝜃𝑖
2
Whitted
Advanced Graphics – Introduction 39
Reflectance for s-polarized light Reflectance for p-polarized light
Reflectance for unpolarized light
Dielectrics
In practice, we use Schlick’s approximation:
𝐹𝑟 = 𝑅0 + (1 − 𝑅0)(1 − 𝑐𝑜𝑠𝜃)5, where 𝑅0 =𝑛1−𝑛2
𝑛1+𝑛2
2.
Based on the law of conservation of energy:
𝐹𝑡 = 1 − 𝐹𝑟
* An Inexpensive BRDF model for Physically-based Rendering, Schlick, Computer Graphics Forum 13, 1994.
Whitted
𝑇
𝐷𝑵𝑅
Advanced Graphics – Introduction 40
Ray Tracing
World space
▪ Geometry▪ Eye▪ Screen plane▪ Screen pixels▪ Primary rays▪ Intersections▪ Point light▪ Shadow rays
Light transport
▪ Extension rays
Light transport
Advanced Graphics – Introduction 41
Whitted
Ray Tree
Using Whitted-style ray tracing, hitting a surface point may spawn:
▪ a shadow ray for each light source;▪ a reflection ray;▪ a ray transmitted into the material.
The reflected and transmitted rays may hit another object with the same material.
➔A single primary ray may lead to a very large number of ray queries.
Advanced Graphics – Introduction 42
Whitted
Question 6: imagine a scene with several point lights and dielectric materials. Considering the law of conservation of energy, what can you say about the energy transported by each individual ray?
Advanced Graphics – Introduction 43
Whitted
Beer’s Law
Light travelling through a medium loses intensity due to absorption.
The intensity 𝐼(𝑠) that remains after travelling 𝑠 units through a substance with absorption 𝑎 is:
𝐼 𝑠 = 𝐼 0 𝑒− ln 𝑎 𝑠
In pseudocode:
I.r *= exp( -a.r * s );I.g *= exp( -a.g * s );I.b *= exp( -a.b * s );
Advanced Graphics – Introduction 45
Whitted
Whitted - Summary
A Whitted-style ray tracer implements the following optical phenomena:
▪ Direct illumination of multiple light sources, taking into account
VisibilityDistance attenuationA shading model: N dot L for diffuse
▪ Pure specular reflections, with recursion
▪ Dielectrics, with Fresnel, with recursion
▪ Beer’s Law
The ray tracer supports any primitive for which a ray/primitive intersection can be determined.
Advanced Graphics – Introduction 46
Whitted
Ray Tracing Testbed
Implement an experimentation framework for ray tracing. Ingredients:
Scene
Primitives: spheres, planes, trianglesI/O (e.g., obj loader)IntersectionMaterials: diffuse color, diffuse / specular / dielectric, absorption
Camera
Position, target, FOVRay generation
Ray
Renderer
Whitted-style
User interface
Input handlingPresentation
Advanced Graphics – Introduction 48
Assignment 1
Ray Tracing Testbed
Regarding the OBJ file loading requirement:
▪ You may want to start with handcrafted scenes▪ Don’t build your own OBJ loader, that’s a waste of time▪ Use assimp or tinyobjloader (C++) or find a lib if you’re using C#
http://www.stefangordon.com/parsing-wavefront-obj-files-in-c/http://www.rexcardan.com/2014/10/read-obj-file-in-c-in-just-10-lines-of-code/https://github.com/ChrisJansson/ObjLoader
▪ Start with small files; minimize your development cycle.
Advanced Graphics – Introduction 49
Assignment 1
Ray Tracing Testbed
Intersecting triangles:
An easy to implement and quite efficient algorithm is:
Fast, Minimum Storage Ray/Triangle Intersection, Möller & Trumbore, 1997.
…which is explained in elaborate detail by scratchapixel.com:
http://www.scratchapixel.com/lessons/3d-basic-rendering/ray-tracing-rendering-a-triangle/moller-trumbore-ray-triangle-intersection
Advanced Graphics – Introduction 50
Assignment 1