Post on 18-Jan-2021
Gaze-Dependent Simulation of Light Perception
in Virtual Reality
Research Division of Computer Graphics
Institute of Visual Computing & Human-Centered Technology
TU Wien, Austria
ISMAR 2020
Laura R. Luidolt1
Michael Wimmer1
Katharina Krösl2
1TU Wien, 2VRVis Forschungs-GmbH1 2
Motivation ▪
Overview ▫
Methodology ▫
Evaluation ▫
Conclusion ▫
Introduction
L. R. Luidolt2
<brightness range
tone mapping
Motivation ▪
Overview ▫
Methodology ▫
Evaluation ▫
Conclusion ▫
Introduction
L. R. Luidolt3
Motivation ▪
Overview ▫
Methodology ▫
Evaluation ▫
Conclusion ▫
Introduction
▪ → Perceptual algorithms necessary!
▪ Medically based
▪ Account for viewing direction, pupil size
L. R. Luidolt4
Motivation ▪
Overview ▫
Methodology ▫
Evaluation ▫
Conclusion ▫
Contribution
▪ Post-processing workflow
▪ Accurate simulation of light perception in VR/AR
▪ Medically-based, perceptual effects
▪ In real-time VR/AR
▪ Following optometrist advice
▪ Eye tracking for measuring light incidence
▪ Pilot user study, comparison of
▪ Real-world low-light situation
▪ And VR simulation
L. R. Luidolt5
Motivation ▫
Overview ▪
Methodology ▫
Evaluation ▫
Conclusion ▫
L. R. Luidolt6
Visual adjustment to
bright and dark
Adaptation of rods
and cones over time
Temporal Eye Adaptation
Colorful patterns
when viewing bright
light sources
Scattering of light in
the eye
PerceptualGlare
Blurred details in low
light scenes
Rods not present in
fovea (point of
sharpest vision)
Visual Acuity Reduction
Color shift towards
blue in low light
scenes
Rods more sensitive
to longer wavelength
light than cones
Scotopic Color Vision
Based on Krawczyk et al., 2005and Ritschel et al., 2009
Motivation ▫
Overview ▪
Methodology ▫
Evaluation ▫
Conclusion ▫
L. R. Luidolt0 0 2 1,.4 -0.02 0.15 2.2 2.98
Visual adjustment to
bright and dark
Adaptation of rods
and cones over time
Temporal Eye Adaptation
Colorful patterns
when viewing bright
light sources
Scattering of light in
the eye
PerceptualGlare
Blurred details in low
light scenes
Rods not present in
fovea (point of
sharpest vision)
Visual Acuity Reduction
Color shift towards
blue in low light
scenes
Rods more sensitive
to longer wavelength
light than cones
Scotopic Color Vision
Based on Krawczyk et al., 2005and Ritschel et al., 2009
Motivation ▫
Overview ▪
Methodology ▫
Evaluation ▫
Conclusion ▫
L. R. Luidolt10
Visual adjustment to
bright and dark
Adaptation of rods
and cones over time
Temporal Eye Adaptation
Colorful patterns
when viewing bright
light sources
Scattering of light in
the eye
PerceptualGlare
Blurred details in low
light scenes
Rods not present in
fovea (point of
sharpest vision)
Visual Acuity Reduction
Color shift towards
blue in low light
scenes
Rods more sensitive
to longer wavelength
light than cones
Scotopic Color Vision
Based on Krawczyk et al., 2005and Ritschel et al., 2009
Motivation ▫
Overview ▪
Methodology ▫
Evaluation ▫
Conclusion ▫
L. R. Luidolt12
Visual adjustment to
bright and dark
Adaptation of rods
and cones over time
Temporal Eye Adaptation
Colorful patterns
when viewing bright
light sources
Scattering of light in
the eye
PerceptualGlare
Blurred details in low
light scenes
Rods not present in
fovea (point of
sharpest vision)
Visual Acuity Reduction
Color shift towards
blue in low light
scenes
Rods more sensitive
to longer wavelength
light than cones
Scotopic Color Vision
Based on Krawczyk et al., 2005and Ritschel et al., 2009
Motivation ▫
Overview ▪
Methodology ▫
Evaluation ▫
Conclusion ▫
L. R. Luidolt14
Visual adjustment to
bright and dark
Adaptation of rods
and cones over time
Temporal Eye Adaptation
Colorful patterns
when viewing bright
light sources
Scattering of light in
the eye
PerceptualGlare
Blurred details in low
light scenes
Rods not present in
fovea (point of
sharpest vision)
Visual Acuity Reduction
Color shift towards
blue in low light
scenes
Rods more sensitive
to longer wavelength
light than cones
Scotopic Color Vision
Based on Krawczyk et al., 2005and Ritschel et al., 2009
Motivation ▫
Overview ▫
Methodology ▼
Adaptation ▪
Glare ▫
VA reduction ▫
Color shift ▫
Evaluation ▫
Conclusion ▫
Temporal Eye Adaptation
L. R. Luidolt15
▪ 𝐿𝑖 = 𝐿𝑖−1 + (𝑌 − 𝐿𝑖−1) ∙
1 − 𝑒− ൗft 𝜏(𝑌)
▪ Target luminance Y
▪ Temporally filtered
luminance Li of frame i
▪ Photoreceptor adaptation
times τ
L. R. Luidolt16Image adapted fromcommons.wikimedia.org/wiki/File:Eyesection.svg
Motivation ▫
Overview ▫
Methodology ▼
Adaptation ▫
Glare ▪
VA reduction ▫
Color shift ▫
Evaluation ▫
Conclusion ▫
Perceptual Glare
L. R. Luidolt17Image adapted fromcommons.wikimedia.org/wiki/File:Eyesection.svg
Motivation ▫
Overview ▫
Methodology ▼
Adaptation ▫
Glare ▪
VA reduction ▫
Color shift ▫
Evaluation ▫
Conclusion ▫
Perceptual Glare
L. R. Luidolt18Image adapted fromcommons.wikimedia.org/wiki/File:Eyesection.svg
Motivation ▫
Overview ▫
Methodology ▼
Adaptation ▫
Glare ▪
VA reduction ▫
Color shift ▫
Evaluation ▫
Conclusion ▫
Perceptual Glare
𝑀 𝑥, 𝑦 =1
𝜆𝑑 2
1
𝑁∙ ℱ 𝑃(𝑥, 𝑦) ∙ 𝑒
i 𝜋𝜆𝑑
𝑥2+𝑦22
After Ritschel et al., 2009
Diffraction on the retina of a
single wavelength light source
Monochromatic PSF
Combination of multiple
wavelengths to simulate spectral
light
Spectral PSF
L. R. Luidolt19
Motivation ▫
Overview ▫
Methodology ▼
Adaptation ▫
Glare ▪
VA reduction ▫
Color shift ▫
Evaluation ▫
Conclusion ▫
Perceptual Glare
Diffraction on the retina of a
single wavelength light source
Monochromatic PSF
Combination of multiple
wavelengths to simulate spectral
light
Spectral PSF
L. R. Luidolt20
Motivation ▫
Overview ▫
Methodology ▼
Adaptation ▫
Glare ▪
VA reduction ▫
Color shift ▫
Evaluation ▫
Conclusion ▫
Perceptual Glare
Perceptual Glare
L. R. Luidolt21
Motivation ▫
Overview ▫
Methodology ▼
Adaptation ▫
Glare ▪
VA reduction ▫
Color shift ▫
Evaluation ▫
Conclusion ▫
· (1 - ) + ·
=
Perceptual Glare
L. R. Luidolt22
Motivation ▫
Overview ▫
Methodology ▼
Adaptation ▫
Glare ▪
VA reduction ▫
Color shift ▫
Evaluation ▫
Conclusion ▫
Visual Acuity Reduction
L. R. Luidolt23
Motivation ▫
Overview ▫
Methodology ▼
Adaptation ▫
Glare ▫
VA reduction ▪
Color shift ▫
Evaluation ▫
Conclusion ▫
▪ 𝜎 𝐿 = max(1 − 𝐿, 0)
▪ Gaussian variance 𝜎
▪ Pixel’s lightness L
Scotopic Color Vision
L. R. Luidolt24
OnOff
Motivation ▫
Overview ▫
Methodology ▼
Adaptation ▫
Glare ▫
VA reduction ▫
Color shift ▪
Evaluation ▫
Conclusion ▫
Evaluation
Qualitative user study with 5 participants
Motivation ▫
Overview ▫
Methodology ▫
Evaluation ▪
Conclusion ▫
L. R. Luidolt25
Real-time VR/AR post-processing workflow
Using eye tracking
Based on medical research
Pilot user study
ConclusionMotivation ▫
Overview ▫
Methodology ▫
Evaluation ▫
Conclusion ▪
L. R. Luidolt26
▪ temporal eye adaptation▪ perceptual glare▪ visual acuity reduction▪ scotopic color vision
Related article: “CatARact: Simulating Cataracts in Augmented Reality”, Krösl et al., 2020
Laura R. Luidolt, luidolt@cg.tuwien.ac.at
Michael Wimmer, wimmer@cg.tuwien.ac.at
Katharina Krösl, kroesl@vrvis.at
Gaze-Dependent Simulation of Light
Perception in Virtual Reality
Than
k yo
u f
or
you
r at
ten
tio
n!