Cryptography Lecture 9 Stefan Dziembowski [email protected].
Scientific VisualizationFall 2015-11-10aht/Vis2015/RenderingTechs_SeSE_Vis2015.pdf · - Scientific...
Transcript of Scientific VisualizationFall 2015-11-10aht/Vis2015/RenderingTechs_SeSE_Vis2015.pdf · - Scientific...
Department of Information Technology
- Scientific Computing
Scientific VisualizationFall 2015-11-10
Stefan Seipel
Ch 7: Transparency
Texture mapping
Direct Volume Rendering (DVR)
Ray Casting
Splatting
Department of Information Technology
- Scientific Computing
Transparency in Visualization
Transparent visualizations used to show structures in spatial context rather than showing detail
Department of Information Technology
- Scientific Computing
Transparency and Alpha Values
Opaque objects reflect, scatter, or absorb light on the surface – no light is transmitted
The level of transparency/opacity of a surface is referred to as alpha in computer graphics
alpha = 0.5 -> object is semitransparent
alpha = 1.0 -> object is fully opaque
alpha = 0.0 -> object id fully transparent
On graphics cards the frame buffer can store the alpha value along with the RGB values
Department of Information Technology
- Scientific Computing
What is alpha compositing?
A method for combining colors of objects in a way that maintains the intervisibility of the scenes
“over” =
“under” =
Department of Information Technology
- Scientific Computing
Alpha compositing: Colors are blended, weighting their alpha values
The color of a pixel is represented by a quadruple (R,G,B,A)
𝑅 = 𝐴𝑆𝑅𝑆+(1-𝐴𝑆) 𝑅𝐵 𝐺 = 𝐴𝑆𝐺𝑆+(1-𝐴𝑆) 𝐺𝐵 𝐵 = 𝐴𝑆𝐵𝑆+(1-𝐴𝑆) 𝐵𝐵 𝐴 = 𝐴𝑆+(1-𝐴𝑆) 𝐴𝐵
0.0, 0.0, 1.0, 1.0 = 𝑜𝑜𝑜𝑜𝑜𝑜 𝑏𝑏𝑜𝑜 0.0, 1.0, 0.0, 0.5 = 𝑠𝑜𝑠𝑠𝑠𝑠𝑜𝑠𝑠𝑜𝑜𝑠𝑜𝑠𝑠 𝑔𝑠𝑜𝑜𝑠 1.0, 0.0, 0.0, 0.0 = 𝑓𝑜𝑏𝑏𝑓 𝑠𝑠𝑜𝑠𝑠𝑜𝑜𝑠𝑜𝑠𝑠 𝑠𝑜𝑟, 𝑠. 𝑜. 𝑠𝑠𝑖𝑠𝑠𝑠𝑏𝑏𝑜
Subscript S indicates current surface to render, and B indicates color behind the object (i.e. what is presently in the frame buffer).
Department of Information Technology
- Scientific Computing
Color of object 2: R = 100 G = 40 B = 200 A = 0.7
Example: Alpha blending Example: Object 1 behind Object 2
Color of object 1: R = 200 G = 100 B = 20 A = 1.0
Color of Background: R = 0 G = 0 B = 0 A = 1.0
1. Object 1 over background: Rout = 200*1.0 + (1.0-1.0)*0 = 200 Gout = 100*1.0 + (1.0-1.0)*0 = 100 Bout = 20*1.0 + (1.0-1.0)*0 = 20
2. Object 2 over background: Rout = 100*0.7 + (1.0-0.7)*0 = 70 Gout = 40*0.7 + (1.0-0.7)*0 = 28 Bout = 200*0.7 + (1.0-0.7)*0 = 140
3. Object 2 over Object 1: Rout = 100*0.7 + (1.0-0.7)*200 = 70+60=130 Gout = 40*0.7 + (1.0-0.7)*100 = 28+30=58 Bout = 200*0.7 + (1.0-0.7)*20 = 140+6=146
Background
Object 2
Object 1
Department of Information Technology
- Scientific Computing
A problem with transparency rendering
• Objects must be drawn in the correct order (back-to-front) for object order rendering algorithms like polygon rendering
• In VTK, this is solved by sorting all objects prior to rendering
• This makes transparency rendering quite slow
• In raycasting/raytracing, samples are always ordered
Department of Information Technology
- Scientific Computing
Texture mapping
2D discrete image 2D polygon (defined in 3D space)
+
Texture-mapped polygon
Applying 2D texture maps to the surface of an object is analogous to pasting a picture. The location of the texture map is specified via texture coordinates.
Department of Information Technology
- Scientific Computing
Texture mapping, cont´d
(0,0) (1,0)
(0,1) (1,1)
Each texel has 2D coordinates assigned to it
Assign texture coordinates to each polygon to establish the mapping
(0,0.5) (0.5,0.5)
(0,0) (0.5,0)
Department of Information Technology
- Scientific Computing
Texture mapping
Purposes: 1. Additional detail is introduced without extensive geometric modeling. 2. Visualization of sampled data
This vertex of the triangle has a coordinate (x,y,z) in a Cartesian coordinate system as well as an associated texture coordinate (u,v) in 2D texture image space.
Department of Information Technology
- Scientific Computing
Textures …
• can be images of the real world (discrete)
• can represent (multi)scalar data sampled on a regular grid
• are often (but do not have to be) 2D discrete images
• can be sampled data on 3D grids and are then referred to as volumetric textures
• can represent computed data on 2D or 3D grids, either with procedural algorithms or by evaluating continuous functions
Department of Information Technology
- Scientific Computing
Alpha value in texture mapping
• By adding alpha values to make an RGBA texture map, parts of the texture become transparent
• Trick in computer graphics
Example: Trees have a complex shape
Render a rectangular RGBA texture map per tree or per tree component leaf/trunks: α = 1 gaps and open space: α = 0
In scientific visualization, volume rendering can be accomplished by compositing a stack of partly transparent 2D textures.
Department of Information Technology
- Scientific Computing
Digital 3D volumetric images
Three-dimensional functions f(x, y, z)
Often sampled medical data from CT, MR, PET, SPECT, …
How can this kind of data be visualized?
N x 2D arrays = 3D array
Department of Information Technology
- Scientific Computing
Example: Visual exploration of Magnetic Resonance Angiography (MRA)
Department of Information Technology
- Scientific Computing
Slice by slice
Department of Information Technology
- Scientific Computing
Slice by slice
Department of Information Technology
- Scientific Computing
Slice by slice
Department of Information Technology
- Scientific Computing
Slice by slice
Issues:
• Spatial context
• Short term visual memory
Department of Information Technology
- Scientific Computing
Multi-Planar Reformatting (MPR)
Department of Information Technology
- Scientific Computing
(MPR) - Example
Department of Information Technology
- Scientific Computing
A direct ”3D” visualization (accomplished by transparent rendering of a stack of 2D slices)
3D imaging of the blood vessel system
Department of Information Technology
- Scientific Computing
3D Volume Rendering Categories of algorithms
3D Object representation Im
age
form
atio
n
Explicit Implicit
Object Order
e.g. Polygon Rasterizer
Volume Splatting
Volume Raytracing
or Raycasting
e.g. Polygon Raytracing
or Raycasting
Image Order
Department of Information Technology
- Scientific Computing
Rendering techniques
Image order rendering techniques: • Image is rendered for each image element
(pixels)
• Ray-casting and ray-tracing most common techniques
• Simulate the interaction of light with objects by following the path of each light ray
• Objects can be polygonal surfaces, parametric surfaces or discrete (3D) images
Department of Information Technology
- Scientific Computing
Ray-casting or ray-tracing (here: explicit geometric models)
Ray-casting Ray-tracing
• image sampling in screen space
• establish ray from eye through pixel
into the scene
• local lighting model assumed at surface
(shading and specular highlights)
• same as ray casting plus:
• iterative back-tracing of reflected and
transmitted rays
• global lighting effects
(shadows, reflections, transparency,
atmospheric effects)
Department of Information Technology
- Scientific Computing
Ray-casting or ray-tracing (here: implicit models – sampled 3D data)
• image sampling in screen space
• 3D data set is sampled along the trajectory of the viewing ray
• profiles of sampled data are recorded for every pixel/ray
• pixel color is determined by compositing function
• there exist many different compositing techniques depending on
the desired visual effect
• more details, see Chapter 7
Department of Information Technology
- Scientific Computing
Ray casting
For each display pixel, cast one or more rays Calculate contribution C for each voxel along the ray
Colour C(x,y,z) determined by gradient (approximate surface normal) lighting (independent of other voxels between the point
and the light)
Opacity α(x,y,z) determined by mapping density values to different types of tissue through transfer functions
Department of Information Technology
- Scientific Computing
Ray casting issues Uniform sampling
Voxel by voxel traversal
What connectivity? Face (6) connected
Edge (18) connceted
Point (26) connected
Department of Information Technology
- Scientific Computing
Various rendering modes
Maximum intensity projection (MIP) Integral projection Distance to a certain threshold value Depth shading Depth gradient shading Grey-level gradient shading Combined multi-modal rendering …
Department of Information Technology
- Scientific Computing
Ray traversal schemes
Depth
Intensity Max
Average
Accumulate First
Study the profile!
Department of Information Technology
- Scientific Computing
Ray traversal - First
Depth
Intensity
First
First: extracts iso-surfaces
Department of Information Technology
- Scientific Computing
Ray traversal - Average
Depth
Intensity
Average
Average: produces basically an X-ray picture, an integral projection
Distance along ray can also be taken into account and used
Department of Information Technology
- Scientific Computing
Ray traversal - MIP
Depth
Intensity Max
Max: Maximum Intensity Projection used commonly for MRA images
Department of Information Technology
- Scientific Computing
Ray traversal - Accumulate
Depth
Intensity
Accumulate
Accumulate opacity while compositing colors: make transparent layers visible
Department of Information Technology
- Scientific Computing
Ray traversal – Sampling frequency
Depth
Intensity
Depth
Intensity
High sample frequency (short sampling intervals) are needed to represent the sampled signal without loss of information.
(->Nyquist-Shannon sampling theorem)
Low sample frequency (long sampling intervals) reduce computational efforts->increase rendering speed.
But the sampled signal is represented with loss of information->aliasing artifacts!
Department of Information Technology
- Scientific Computing
Ray sampling frequency
0.1 unit step size 1.0 unit step size 2.0 unit step size
Smoothness vs. computational cost (linear in time)
Department of Information Technology
- Scientific Computing
Grey-level gradient shading
A threshold is given and the rays are cast into the volume until a voxel above the threshold is encountered
The gradient at the points are determined and taken as a normal vector of an iso-surface at this point
Normal vector together with virtual eye-position and virtual light source are used to render voxel
Cut planes can be used to remove parts of the volume
Department of Information Technology
- Scientific Computing
Gradient/Normal estimation
Estimate gradient vector G per voxel using central differences of the density data D Gx(i,j,k) = D(i+1, j, k) - D(i-1, j, k) Gy(i,j,k) = D(i, j+1, k) - D(i, j-1, k) Gz(i,j,k) = D(i, j, k+1) - D(i, j, k-1)
Gradient g ->derivative of f in
Department of Information Technology
- Scientific Computing
Grey-level gradient shading
Department of Information Technology
- Scientific Computing
Multi-modal rendering
Co-register two data volumes with each other Rays are cast into the first volume For points on cut planes, values from both volumes
are blended to produce a colour that enhances both modalities
MRI +
PET
Department of Information Technology
- Scientific Computing
Rendering techniques
Object order rendering techniques: • Rendering is processed in the order of
graphical objects for every object at a time (points, polygons, or voxels)
• Pixels in the final images are written to many times
• Visibility problem (which object is closest)
• Non-visible objects are processed
• Rasterization and splatting are two typical examples
Department of Information Technology
- Scientific Computing
Polygon rasterization (for explicit geometric models)
• Vertices of polygons are projected from 3D object space onto 2D screen space
• Pixels in the interior area of a polygon on screen is then filled
• Compare 3.7 and slide further down
vp
0 1
2 3
1
2 3
0
3 2
0 1
0
2 3
0 1
1
3
2 3
2
0 1
Screen Space
Object Space
Transform &
Projection
Department of Information Technology
- Scientific Computing
Voxel splatting (for eg. volumetric data sets)
• A discrete voxel space is traversed in voxel order
• Voxel is projected onto image plane (position [x,y])
• A 2D footprint (splat) of the voxel is determined (size and shape)
• The color of all pixels in the image covered by the splat are
updated using compositing functions
• Requires strictly ordered traversal
(back-to-front / front-to-back) to
maintain correct visual results
Department of Information Technology
- Scientific Computing
Splatting (feed-forward) “filling the gaps”
Discrete sampling intervals in object space
Sample displacement in screen-space is transform dependent
Department of Information Technology
- Scientific Computing
Fill the gaps
We need to fill the pixel values between the volume projection samples
To fit a continuous function through the discrete samples Convolution can be used
Department of Information Technology
- Scientific Computing
Convolution
g(x,y)
f(i,j)
The output g is a weighted average of inputs f
g(x,y) = Σ Σ f(x-i,y-j) h (i, j)
i= - j= -
sample value
weight
Department of Information Technology
- Scientific Computing
Convolution, cont´d
Another way of thinking convolution is to deposit each function value to its neighbor pixels
f(i,j)
This weighting function is called kernel
Department of Information Technology
- Scientific Computing
3D kernel for splatting
Need to know the 3D extent of each voxel Project the extent to the image plane
?
Splatting
x This is called a footprint
Department of Information Technology
- Scientific Computing
Splatting
Footprints can be precalculated and saved in a lookup table. Therafter, pasted as needed
Splatting is more difficult for perspective volume rendering; image space extent is not identical for all samples
Considerations: type of kernel radius of kernel resolution of footprint table
Department of Information Technology
- Scientific Computing
Efficient Volume Rendering
Consider 512 x 512 x 512 data
At least one byte per element
134 217 728 bytes = 128 MB
How to render such data interactively?
Department of Information Technology
- Scientific Computing
Big data problem
One way to handle big data problem is to use multiple-resolution data structures (hierarchical volumes)
8x4x4 4x4x2 2x2x2 …
…
Less amount of data required
Department of Information Technology
- Scientific Computing
Adaptive methods: Octrees for volume rendering
• Volume dataset is recursively subdivided on different levels
• Empty nodes on different levels are labeled • Empty nodes are not rendered, thus avoiding the
processing of ”empty space” • Recursion depth of the octree determines
precision with which empty space can be avoided • Trade-off: high resolution octree requires a lot of
memory in itself
The following illustrations are curtsey of Daniel Blomqvist, student at HiG, Spring 2010
Department of Information Technology
- Scientific Computing
Octrees for volume rendering
Department of Information Technology
- Scientific Computing
Octrees for volume rendering
Department of Information Technology
- Scientific Computing
Octrees for volume rendering
Department of Information Technology
- Scientific Computing
Octrees for volume rendering
Department of Information Technology
- Scientific Computing
Octrees for volume rendering
Department of Information Technology
- Scientific Computing
Surface versus Volume Rendering
• Surface rendering + Standard computer graphics software can
be used - simple + Supported by dedicated hardware - fast + A high data reduction can be achieved - Intensity information is lost - Cutting through the data is meaningless - Changes in surface definition criteria
means recalculation of the data
Department of Information Technology
- Scientific Computing
Surface versus Volume Rendering
• Volume rendering + Arbitrary cuts can be made allowing
the user to see inside the data + Allows for display of different aspects MIPs semi-transparent surfaces etc.
+ Rendering parameters can be changed interactively
- Slow
Department of Information Technology
- Scientific Computing
Intermixing Volumes and Geometry