Post on 29-Jun-2015
description
Estimating natural illumination from a single outdoor image12/8/09
Estimating natural illumination from a single outdoor scene
Sagnik Dhar (107219686)Debaleena Chattopadhay (107224101)
Course Project for CSE – 591Fall ‘09
Based on, “Estimating Natural Illumination from a Single Outdoor Image” by Jean-
Francois Lalonde, Alexei A. Efros, and Srinivasa G. NarasimhanCVPR ‘09
Estimating natural illumination from a single outdoor image12/8/09
Estimation of illumination
Estimating natural illumination from a single outdoor image12/8/09
Why do people care about illumination ?
• Insertion of objects
• Retrieval of objects from a database
Estimating natural illumination from a single outdoor image12/8/09
Approach
Natural Outdoor Daytime Scene
Extract Plausi-ble Cues
Output a Probable Distribution of Sun’s
Zenith angle
Sky Cue
Shadow Cue
Perez Modeling of
Skies
Simulate Normal
Distribution of sun positions
Determining Best Linear
Shadow
Vertical Surface Cue
Intensity Gradi-ents Vote For Pos-
sible Sun positions
Intensity Of The Vertical Surfaces
Weighed By Their Size Vote For Possible
Sun Directions
Compute Size Ratio Of Surfaces
To The Scene
Estimating natural illumination from a single outdoor image12/8/09
Sky Cue
Determine Horizon Line Extract Sky Segment
Classify into Clear, Patchy or Overcast
category and Process(KNN)
Simulate Perez Sky Model to create 360 skies (each
for a sun zenith angle)(Video)
Taking our sky segment as the mean, find a probability
distribution of sun zenith angle
Estimating natural illumination from a single outdoor image12/8/09
KNN Classification of skies
Sky
Clear Patchy Overcast
Binary segmentation to remove clouds Discard Cue
Continue
Estimating natural illumination from a single outdoor image12/8/09
Shadow Cue
Extract ground Segment
Detect Longest Linear Shadow
Shadow pixel intensities vote for
probable sun directions
Convert to CIE L*a*b Color
Space
Detect Shadow Pixels in b channel
Find Intensity orientation of the shadow
pixels
Cluster Using K-means
Top Cluster votes for Sun
Directions
Estimating natural illumination from a single outdoor image12/8/09
Vertical Surface Cue
Extract Vertical Surfaces
Compute surface size proportionality with respect to the scene.
Find intensity of each surfaces
Surface size weighted by intensity vote for their preferred Sun direction
Estimating natural illumination from a single outdoor image12/8/09
Sun Zenith angle
Estimating natural illumination from a single outdoor image12/8/09
Cue Combination
1ST
2ND3RD
4TH
271°Sky
14°Shadow
BehindVertical Surfaces
Estimating natural illumination from a single outdoor image12/8/09
Results- Best Case
Estimating natural illumination from a single outdoor image12/8/09
Results- Best Case
Estimating natural illumination from a single outdoor image12/8/09
Results- Best Case
1ST
2ND3RD
4TH
100°Sky
8°Shadow
RightVertical Surfaces
Estimating natural illumination from a single outdoor image12/8/09
Results- Best Case
Estimating natural illumination from a single outdoor image12/8/09
Results- Best Case
Estimating natural illumination from a single outdoor image12/8/09
Results- Best Case
1ST
2ND3RD
4TH
181°Sky189°
Shadow
LeftVertical Surfaces
Estimating natural illumination from a single outdoor image12/8/09
Results- Sky + Vertical Cue
Estimating natural illumination from a single outdoor image12/8/09
Results- Sky + Vertical Cue
Estimating natural illumination from a single outdoor image12/8/09
Results- Sky + Vertical Cue
1ST
2ND3RD
4TH
271°Sky
BehindVertical Surfaces
Estimating natural illumination from a single outdoor image12/8/09
Results- Shadow + Vertical Cue
Estimating natural illumination from a single outdoor image12/8/09
Results- Shadow + Vertical Cue
Estimating natural illumination from a single outdoor image12/8/09
Results- Shadow + Vertical Cue
1ST
2ND3RD
4TH
14°Shadow
BehindVertical Surfaces
Estimating natural illumination from a single outdoor image12/8/09
Results- Shadow + Vertical Cue
Estimating natural illumination from a single outdoor image12/8/09
Results- Shadow + Vertical Cue
Estimating natural illumination from a single outdoor image12/8/09
Results- Shadow + Vertical Cue
1ST
2ND3RD
4TH
9°Shadow
BehindVertical Surfaces
Estimating natural illumination from a single outdoor image12/8/09
Results- Failure Cases
Estimating natural illumination from a single outdoor image12/8/09
Results- Failure Cases
Estimating natural illumination from a single outdoor image12/8/09
Results- Failure Cases
1ST
2ND3RD
4TH
5°Shadow
BehindVertical Surfaces
292°Sky
Estimating natural illumination from a single outdoor image12/8/09
Results- Failure Cases
Estimating natural illumination from a single outdoor image12/8/09
Results- Failure Cases
Estimating natural illumination from a single outdoor image12/8/09
Results- Failure Cases
1ST
2ND3RD
4TH
16°Shadow
BehindVertical Surfaces
0°Sky
Estimating natural illumination from a single outdoor image12/8/09
Human Validation
No. of Humans: 3
No. of Images: 58
Human Consensus
Human Contradiction
Accuracy
27 31 46%
Match with Human
Mismatch Accuracy
22 36 37%
Estimating natural illumination from a single outdoor image12/8/09
Cue Weightage Investigation
Estimating natural illumination from a single outdoor image12/8/09
Conclusion
• Accurate estimation of sun position – Tough problem for human visual system too.
• The photographic eye doesn’t bother to capture “informative” cues for visual systems to analyze.
• The weak cues are strongly coupled and highly susceptible to incorrect conclusion.
• Shadow detection from a single image is still in the need of better algorithms.
Estimating natural illumination from a single outdoor image12/8/09
Thank You
To end, a quote I came across recently,
“All I see is the sun; And even when it's hiding behind a smoke of cloud, the shadow tells me all that there is...”