Removing Weather Effects fromMonochrome Images

Srinivasa Narasimhan and Shree Nayar

Computer Science DepartmentColumbia University

IEEE CVPR ConferenceDecember 2001, Hawaii, USA

SponsorsDARPA HID, NSF

How does scene contrast degrade in bad weather ?

How can scene contrast be restored from bad weather images ?

Contrast Degradation in Bad Weather

RainFog

Weather Effects are Depth Dependent

Image Processing Does Not Suffice

Histogram Equalized Images

Prior Methods for Contrast Restoration

Yitzhaky, Kopeika [98]

Oakley, Tan, Satherley[98,01]

Nayar, Narasimhan [99]Narasimhan, Nayar [00]

Scene DepthWeather

Information

RequiredPredicted Weather PSF

Required

ComputedWavelengthIndependentScattering

Not Required

MethodClear-day SceneIntensity/Color

Computed

Gaussiandistribution

assumed

Computed(Color Images Required)

OUR GOAL : ComputedNot Required Computed

Scattering Models : Attenuation and Airlight

Object

Observer

d

Attenuation

d

SunlightDiffuseSkylight

DiffuseGround Light

Airlight

E

d

deEE )1( deEE

Contrast Degradation in Bad Weather

Irradiance = Attenuation + Airlight

)(dE += deE )1( deE

HorizonBrightness

Depth Reflectance ScatteringCoefficient

Contrast Decay : Exponential in Scene Depth

)1(2)2()1(

)2()1(

)2()1(

)2()1(

12

deEE

EEC

Contrast between Iso-Depth points , P and P :(1) (2)

Mild Fog Denser Fog

Depth Edge Reflectance Edge

Depth Edges vs. Reflectance Edges

( 2) ( 2)Normalized SSD of

Reflectance Edge NeighborhoodNormalized SSD of

Depth Edge Neighborhood

Edge Classification from Weather Changes

Mild Fog Denser Fog Edge Classification

Reflectance Edge : Depth Edge :

Scene Structure from Weather Changes

Irradiance under versus Irradiance under : Linear 1 21E 2E

2

1

2

1 lnln)(2

112

E

E

EE

EEdScaled Depth :

1E

2E

1E

2E

)1( )()( 12

2

12

1

2 dd eEeE

E

1E2E

All Scene pointsat Depth 1

All Scene pointsat Depth 2

Depth Map from Two Weather Conditions

Mild Fog, 5 PM (Input)

Denser Fog , 5: 30 PM (Input)

Computed Depth Map (Output)

Comparing with Prior Methods:

• Color Images Not Needed

• Works for Wider Range of Weather Conditions

Weather Removal Using Scene Structure

Contrast Restored Image (Output)Dense Fog, 5:30 PM (Input)

Computed Depth Map (Input) Histogram Equalized Image (For comparison)

Different amounts of Fog removed from different Depths

A De-Weathering System

DetectSignificantWeatherChange

Video Frame (Weather 1)

Video Frame(Weather 2)

Scene Structure

System Initialization : Computing Scene Structure

Continuous De-Weathering Using Scene Structure

Scene Structure

ContrastRestored

Video Frame