Visual Perception in Realistic Image Synthesis Ann McNamara.
-
Upload
dinah-hart -
Category
Documents
-
view
218 -
download
0
Transcript of Visual Perception in Realistic Image Synthesis Ann McNamara.
Visual Perception in Realistic Image Synthesis
Ann McNamara
Outline
• Introduction• Modeling important characteristics
of the human visual system (HVS)• Perception based rendering• Image quality metrics• Tone reproduction operators• Summary
Realism
• Architecture• Stage lighting• Entertainment• Safety systems• Archaeology
Human Visual System
• Physical structure well established
• Perceptual behaviour is a complex process
Modeling Important Characteristics
of the Human Visual System
Visual Acuity
• How well we can see fine detail • Adaptation level• Rods and cones
Cones
• Number of grating that fall on one degree of the retina
• Dependent on distance
Spatial Frequency
Spatial Frequency
•Spatial mechanisms (channels) which are used to represent the visual information at various scales and orientations as it is believed that primary visual cortex does.
Contrast Sensitivity Function
•Contrast sensitivity function which specifies the detection threshold for a stimulus as a function of its spatial frequencies.
Campbell-Robson contrast sensitivity chartCampbell-Robson contrast sensitivity chart
Contrast Sensitivity
Masking
•Visual masking affecting the detection threshold of a stimulus as a function of the interfering background stimulus which is closely coupled in space and time.
Masking
Colour Appearance
Perceptually Based Rendering
[Mitchell 1987]]
Low Sampling Densities
•Non-uniform sampling is less conspicuous
•Optimise using how the eye perceives noise as a function of contrast and colour
Raytracing -> Point Samples-> Aliasing
Uniform
Non-Uniform
Adaptive
Sampling Schemes[Mitchell
1987]]
[Mitchell 1987]]
Low Sampling Densities
• Contrast
• Colour
IIIIC
minmax
minmax
R 0.4 G 0.3 B 0.6
[Mitchell 1987]]
Low Sampling Densities
Frequency Based Raytracing[Bolin &Meyer 1992]]
• Synthesise directly into frequency domain
• Simple vision model to control •Where to cast rays•How to spawn rays
Frequency Based Raytracing[Bolin &Meyer 1992]]
• Vision model•Contrast sensitivity•Spatial frequency•Masking
Frequency Based Raytracing[Bolin &Meyer 1992]]
• Specific luminance difference at low intensity more important than same luminance difference at high intensity
• Colour spatial frequency variations given fewer samples
• Decrease rays spawned in high frequency regions
Limited Color Acuity[Meyer & Liu1998]]
• Colour Abberation• Limited sampling of receptor• Spatial acuity of opponent
channels
[Meyer & Liu1998]]
Application
• How much computation is enough?• How much reduction is too much?• An objective metric of image quality
which takes into account basic characteristics of the HVS could help to answer these questions without human assistance.
Questions of Appearance Preservation
The Concern Is Not Whether Images Are the Same
Rather the Concern Is Whether Images Appear the Same
Perceptually Based Adaptive Sampling Algorithm
[Bolin &Meyer 1998]]
• Image quality model embedded into image synthesis
• Use statistical information about spatial frequency to determine where to estimate values where samples were yet to be taken
Perceptually Based Adaptive Sampling Algorithm
[Bolin &Meyer 1998]]
JND’s
VDM
= 200s= 200s = 400s= 400s = 800s= 800s = 1600s= 1600s
Deterministic radiosityDeterministic radiosity
Monte Carlo radiosityMonte Carlo radiosity
Convergence Evaluation[Myszkowski 1997]]
vs. referencevs. reference0.50.5 vs. vs.
[Myszkowski 1997]]
Termination Criterion
Physical Based Perceptual Metric[Ramasubramanian et
al1999]]• Threshold model defines a physical error metric
• Handles luminance-dependent and spatially dependent processing independently•Allowing pre-computation of
spatially-dependent component
Physical Based Perceptual Metric[Ramasubramanian et
al1999]]
Image Quality Metrics
Image Quality
• Compare and validate lighting simulations
• Use comparisons to guide rendering more efficiently•Compute less without altering
perception•Pixel by pixel comparison might be >
0, human might not see any difference
RMSE 9.5 RMSE 5.2
Pixel by Pixel ComparisonPrikryl, 1999
Visible Differences PredictorVDPVisible Differences PredictorVDP
Image
2
Image
1
Ps
ych
om
etr
icF
un
cti
on
Pro
bab
ility
Su
mm
atio
n
Vis
ua
lisa
tio
n o
fD
iffe
ren
ce
s
AmplitudeNonlinear.
AmplitudeNonlinear.
ContrastSensitivityFunction
ContrastSensitivityFunction
+
CortexTransform
CortexTransform
MaskingFunction
MaskingFunction
Unidirectionalor MutualMasking
[Daly ‘93, Myszkowski ‘98]
VDP: Results
Standard
Comparison
Pixel differences:Standard - Comparison
Pixel differences
The VDP response:probability ofperceivingthe differences
VDP response
Daly’s VDP: Features
•Predicts local differences between images •Takes into account important visual
characteristics:• Amplitude compression• Advanced CSF model• Masking
•Uses the cortex transform, which is a pyramid-style, invertible & computationally efficient image representation
Daly, 1993
Visible Discrimination Model
• Map of Just Noticeable Differences• Point sample function to model optics• Resample the image according to
foveal eccentricity• Band pass response
•Contrast pyramid•steerable filters
Lubin, 1997
Visible Discrimination Model
• Both images subjected to Identical processing• Distance measure
•Difference in responses for Each channel and summing Them to obtain a JND Map of the two images
input images
optics
sampling
contrastpyramid
transducer
masking
distance
Qnorm
JNDValue
Lubin, 1997
An Experimental Evaluation of Computer Graphics Imagery
• Comparing image to real-world scene
• An approach to image synthesis consisting of•A physical module•A perceptual module
[Meyer et al, 1986]]
An Experimental Evaluation of Computer Graphics Imagery
[Meyer et al, 1986]]
DifferenceSimulated Measured
An Experimental Evaluation of Computer Graphics Imagery
[Meyer et al, 1986]]
An Experimental Evaluation of Computer Graphics Imagery
[Meyer et al, 1986]]
Image Quality Metrics[Rushmeier et al, 1995]]
• Components of perceptually based metrics adapted from image compression•Gervais et al 1984•Mannos et al 1974•Daly 1993
Image Quality Metrics[Rushmeier et al, 1995]]
• Daly tested very well
Real Room Simulated Model of Room
Visual Psychophysics
• Determine the relationship between the physical world and human’s subjective experience of that world
• Measure the response (“psycho”) to a known stimulus (“physics”)
Why Lightness ?[Gilchrist 1977]]
[McNamara et al 1998, 2000]]
A Psychophysical Investigation
• Painted 5-sided cube
• Objects painted with different grey paints
• Complex illumination, with secondary reflections
Graphic Reconstructions[McNamara et al 1998, 2000]]
Experiment
Rendered
Real Scene
[McNamara et al 1998, 2000]]
Results[McNamara et al 1998, 2000]]
Tone Reproduction Operators
~105cd/m2~10-5
cd/m2
Tone Reproduction
~100 cd/m2~1 cd/m2
Same VisualResponse ?
Tone Reproduction for Realistic Images
• Mapping between radiances computed and light energy emitted from CRT
• Psychophysical model of brightness perception
• Observer model• Display model
Tumblin & Rushmeier, 1993
Tone Reproduction
Tone Reproduction for Realistic Images
http://graphics.cs.uni-sb.de/~slusallek/Doc/html/node14.html
Low Medium High
Tumblin & Rushmeier, 1993
A Contrast-based Scalefactor for Luminance Display
http://graphics.cs.uni-sb.de/~slusallek/Doc/html/node14.html
• Linear transformLd = mLW
• Matching contrast between real and image
Ward, 1994
A Contrast-based Scalefactor for Luminance Display
http://graphics.cs.uni-sb.de/~slusallek/Doc/html/node14.html
Min-Max Ward
Ward, 1994
A Model of Visual Adaptation for Realistic Image Synthesis
• Threshold visibility• Changes in colour appearance• Visual acuity• Temporal Sensitivity
Ferwerda et al, 1996
A Model of Visual Adaptation for Realistic Image Synthesis
Ferwerda et al, 1996
Spatially Nonuniform Scaling for High Contrast Images
• Incorrect to apply the same mapping to each pixel
• Spatial position
Chiu et al, 1993
Quantization Techniques for Visualization of High Dynamic
Range Pictures
• Similar to Chiu et al• Rational rather than logarithmic
• Accounts for the non-linearities of both the display device and human perception
• The biggest advantages is speed
Schlick, 1994
A Visibility Matching Tone Reproduction Operator for High
Dynamic Range Scenes
• Preserve visibility of objects• Histogram - adjusted to minimise
the visible contrast distortions• Also includes glare, colour
sensitivity, and acuity
Larson et al, 1997
A Visibility Matching Tone Reproduction Operator for High
Dynamic Range ScenesLarson et al, 1997
Perceptually Driven Radiosity[Gibson & Hubbold. 1997]]
• Steer computation to areas in need of most refinement
• A-priori estimate adaptation luminance•Tone-mapping to transform
luminance to display•Distance between two colors in
uniform colour space = numerical measure of perceived difference
Perceptually Driven Radiosity[Gibson & Hubbold. 1997]]
• Stop patch refinement once the difference between successive levels becomes perceptually unnoticeable
• Determine the perceived importance of any shadow
• Optimise the mesh for faster interactive display and minimise storage
Standard shadowtesting
(19.33 hours)
Perceptually-driven shadow testing
(3.10 hours)
[Gibson & Hubbold. 1997]]
Shadow Testing
Output meshOutput mesh Optimised meshOptimised mesh
[Hedley et al. 1997]]
Discontinuity Meshing
• Throw out discontinuities that are deemed visually unimportant
• Tone mapping• Compare colour differences along
the discontinuity line
Culled discontinuitiesCulled discontinuitiesOriginal sceneOriginal scene
Summary
• Applications of visual perception in computer graphics•Efficient software• Image quality evaluations•Tone reproduction operators
• Knowledge of HVS can be used to greatly benefit the synthesis of realistic images at various stages of production
Conclusion
• Great deal of potential• Perceptually accurate as well as
physically correct• Allow high level of confidence in
computer imagery allowing us to demonstrate to the world that our images are faithful representations !
Extra…
Spatial and Orientation Mechanisms
• The following filter banks are commonly used:
• Gabor functions (Marcelja80), • Steerable pyramid transform
(Simoncelli92), • Discrete Cosine Transform (DCT), • Difference of Gaussians (Laplacian)
pyramids (Burt83,Wilson91), • Cortex transform (Watson87, Daly93).
Cortex Transform: Organization of the Filter Bank
Cortex Transform: Orientation Orientation BandsBands
Input image Input image
Spatiovelocity CSF
• Contrast sensitivity data for traveling gratings of various spatial frequencies were derived in Kelly’s psychophysical experiments (1960).
• Daly (1998) extended Kelly’s model to account for target tracking by the eye movements.
log visual sensitivity
log velocity [deg/sec]
log spatial frequency [cycles/deg]
Temporal frequency [Hz]
Visual Masking•Masking is strongest between stimuli
located in the same perceptual channel, and many vision models are limited to this intra-channel masking.
•The following threshold elevation model is commonly used: