Physically Based Sky, Atmosphere and Cloud Rendering in Frostbite

empowers game creators to shape the future of gaming

Physically Based Sky, Atmosphere and Cloud Rendering in Frostbite

SEBASTIEN HILLAIRE - ELECTRONIC ARTS / FROSTBITE

3SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice

An EA wide collaborationwithin Frostbite

Sky and atmosphere systems

Master’s thesis: Gustav Bodare & Edvard Sandberg (Ghost)

Bioware (Mass Effect Andromeda), DICE (Mirror’s Edge Catalyst), Ghost (Need for Speed)

Volumetric cloud systems

Master’s thesis: Rurik Högfeldt

Bioware: Marc-Andre Loyer (Programmer), Don Arceta (Lead Environment Art) and Soren Hesse (Tech Environment Art)

The rendering research community


Context

Physically based rendering in Frostbite

Transition done in 2014 [Lagarde&deRousiers14]

Huge increase in visual quality


Context

Previous techniques in Frostbite

Sky

HDRI captures, color gradients

Atmosphere

Depth, height fog, screen space light shafts

Clouds

Panoramic texture + Visual flow [Guerrette14]

Translated planar cloud layers

Cons

Lots of manual work to get components to match

Not dynamic, difficult to achieve time of day


Real-life sky, atmosphere and clouds

With dynamic time of day and evolving weather

Photo by DAVID ILIFF. License: CC-BY-SA 3.0


Scope and motivation

Dynamic time of day and evolving weather at 60 FPS

Physically based rendering

Meaningful material parameters

Decouple material from lighting

Unified interactions

Sky, atmosphere and cloud interact together

Interaction with opaque and transparent objects, global illumination, etc.


Mirror’s Edge Catalyst - DICE[Christin16]


Outline

Sky and atmosphere rendering

Sun rendering

Volumetric cloud rendering



Outline

Sky and atmosphere rendering



Sky and atmosphere

Atmosphere

Rayleigh scattering

Mie scattering

Components


Previous work

Analytical models [CIESky][Hosek12][Preetham99]

Simpler but limited to ground view

Spectral rendering [Elek09][Preetham99]

Sky atmosphere simulation [Bruneton08]

Pre integrated multi-scattering/absorption in 4D LUTs

Scattering occlusion from terrain

Ignore earth shadow 3D LUTs [Elek09]

Improve LUT parameterization [Yusov13][Bruneton08]

[Hosek12]


Sky – algorithm overview

Transmittance LUT

(h, θv)

Single scattering LUT

(h, θv, θl)

L

θl

V

θv

H

h

Final multi ScatteringLUT(Rayleigh, Mie)

Accumulatescattering

order

Scattered light integration

N-order scattering[Bruneton08]

[Bruneton08] [Elek09] [Yusov13]


Atmosphere simulation coefficients[Nishita93][Riley04][Bruneton08][Adams74][Kutz12]

Type Scattering (𝒎−𝟏) Extinction (𝒎−𝟏) distribution

Rayleigh(standard air) σ𝒔

𝒓𝒂𝒚= 𝟓. 𝟖𝑒−6, 𝟏. 𝟑𝟓𝑒−5, 𝟑. 𝟑𝟏𝑒−5 σ𝒔

𝒓𝒂𝒚𝒆

−𝒉𝟖.𝟎 𝒌𝒎

Mie(pollution, water, dust, etc.)

σ𝒔𝒎𝒊𝒆 >= 3𝑒−6 a × σ𝒔

𝒎𝒊𝒆 𝒆−𝒉

𝟏.𝟐 𝒌𝒎

Ozone --- σ𝒂𝒐𝒛𝒐 𝒆

−𝒉𝟖.𝟎 𝒌𝒎

a = 1.11

σ𝒂𝒐𝒛𝒐 = 𝟑. 𝟒𝟐𝟔, 𝟖. 𝟐𝟗𝟖, 𝟎. 𝟑𝟓𝟔 𝒎−𝟏 × 𝟔𝒆−𝟓% [Kutz12]

[Bruneton08]

Rayleigh scattering examples

Mie scattering example


Ozone absorption “Essential […] to reproduce the blue of the zenith sky” [Adams74][Kutz12]

Ozone absorption: “Essential […] to reproduce the blue of the zenith sky”[Adams74]


Sky and aerial perspective rendering

Render the sky using ScatteringLUTs(h,θv,θl)

Camera aerial perspective volumes

32x32 x 16 depth slices

Integrate scattering/transmittance for each slices

Easy to apply on transparents (per vertex evaluation)

Scatteredluminance

Transmittance


Time of day - Moving sun

LUTs + moving sun automatically updates:

Sky

Aerial perspective

Triggers updates of

Sky environment map

Local reflection volumes

Global illumination

Light probes

Sky environment lighting debug


Sky rendering performance

LUT updates

Needed when changing atmospheric properties

Distribute evaluation over several frames

Each scattering order, sub table parts

While updating, interpolate last results

Complete update frame count

Mean frame GPU cost

1 3.50 ms

19 0.22 ms

Component GPU cost

AP camera volume 32x32x16 0.05 ms

Sky/AP rendering 0.42 ms

LUT updates with 4th order scattering measured on XB1

Sky/Atmosphere rendering cost on XB1 720p


Outline

Sun rendering



Sun disk luminance and ground illuminance

In Frostbite, artists specify illuminance E for sun at zenith for sun solid angle 𝜴 [Lagarde&deRousiers14]

E = 100000 Lux(default sun)

L outer space luminance

L outer space luminance

E

current sun position


Atmosphere transmittance on sun luminance = apply transmittance LUT per pixel to outer space luminance


Outline




Previous work

Mesh + HyperTexture [Bouthors08]

High quality but overall expensive

Splatting based rendering [Yusov14]

Particle based, depth aware blending

Specific look did not match our target

Cloud layer [Schneider15]

Completely dynamic solution

Lighting, shadowing, weather

[Yusov14]

[Bouthors08]

[Schneider15]



Similar to [Schneider15]

Cloud layer with weather animation

Dynamic lighting and shadowing

Perlin-Worley noise for realistic cloud details

Beer-Powder

Improvements

Scattering energy conservation

2-lobe phase functions

Time of day and weather interaction with sky and aerial perspective

[Schneider15]


Volumetric cloud lighting components

Transmittance

Ambient

Sun scattering

Cloud self shadow

Phase function

Reality


Scattering integration improvement

Ray marching: improved integration from [Hillaire15]

float3 scattering = float3(0.0, 0.0, 0.0);float transmittance = 1.0;for (s= 0; s< StepCount; ++s){

𝑆 = sample scattered luminance

sampleTransmittance = 𝑒−𝝈𝒕 𝐷 = sample transmittancescattering += 𝑆 * transmittancetransmittance *= sampleTransmittance

}

Wrong?

Analytically integrate lighting w.r.t. transmittance over depth D (ShaderToy demo)

0𝐷𝑒−𝝈𝒕 𝑥 × 𝑆 𝑑𝑥 =

𝑆−𝑆×𝑒−𝝈𝒕 𝐷

𝝈𝒕

𝝈𝒕 : extinction over range𝑆 : scattered luminance𝐷 : integration distance

https://www.shadertoy.com/view/XlBSRz


scattering += 𝑆 * transmittancetransmittance *= sampleTransmittance

Non energy conservative


transmittance *= sampleTransmittance

scattering += 𝑆 * transmittanceOver attenuation dark media


Analytic integration 𝑆−𝑆×𝑒−𝝈𝒕 𝐷

𝝈𝒕

Energy conservative


Default scattering integration, 21 samples

Default scattering integration, 512 samples

Improved scattering integration, 21 samples

Improved scattering integration, 512 samples


Phase function

Two-lobes HG phase function

𝒑𝒉𝒂𝒔𝒆 𝑔0, 𝑔1, α, 𝜇 = 𝑙𝑒𝑟𝑝(𝒉𝒈𝑷𝒉𝒂𝒔𝒆 𝑔0, 𝜇 , 𝒉𝒈𝑷𝒉𝒂𝒔𝒆 𝑔1, 𝜇 , α)

Tried best fit

Back-scattering still not very visible

Would require multi-scattering…

Two-lobes HG phase + artist control

More freedom to control cloud style

Default: 𝑔0 = 0.8, 𝑔1 = −0.5, α = 0.5

[Bouthors08]

CloudHenyey/GreensteinRayleigh

0.0 0.5 1.0 1.5 2.0 2.5 3.010 4

0.01

1

100 Two-lobe HGfitted to cloud phase

Angle

Ph

ase


Phase function0.5 1.0 1.5 2.0 2.5 3.0 3.5

0.30.20.1

0.10.20.3

HG phase with g=0.8


Phase function0.5 0.5 1.0 1.5 2.0

0.3

0.2

0.1

0.1

0.2

0.3

2-lobe HG phase with g0=0.8, g1=-0,5 and lerp=0.5


Phase function0.5 1.0 1.5 2.0 2.5 3.0 3.5

0.30.20.1

0.10.20.3

HG phase with g=0.8


Phase function0.5 0.5 1.0 1.5 2.0

0.3

0.2

0.1

0.1

0.2

0.3

2-lobe HG phase with g0=0.8, g1=-0,5 and lerp=0.5


Aerial perspective and clouds Must look consistent under all time of day and weather conditions

Visual mismatch (missing fog on clouds)

Clouds without aerial perspective


Aerial perspective and clouds

Per sample: expensive to evaluate

Instead sampled once on cloud front interface

Compute mean depth on cloud interface weighted by transmittance


Aerial perspective on cloudClouds with aerial perspective


Cloud coverage and scattering influence

Cloud hemisphere sampling aroundcamera

• Integrate cloud luminance• Mean cloud transmittance

New AP scattered luminance =oldAP x transmittance + luminance


Cloud coverage and scattering influenceRealityClouds with aerial perspective interactions


Volumetric clouds performance

720p 900p

Hemisphere sampling: scattering & coverage 0.090 ms 0.090 ms

Main view (with temporal re-projection) 0.720 ms 1.090 ms

Planar reflection (optional) 0.035 ms 0.048 ms

Total 0.835 ms 1.228 ms

Measured on XB1

Cloud main view resolution = 640x360 (720p/2)

Planar reflection resolution scale = 128x72 (720p/10)

Default quality seen in this presentation

Looking toward horizon


Final result

Sun rise with animated clouds


Final result

Mars blue sunset (eye-balled atmosphere)

[NASA]


Conclusion

Physically based sky, atmosphere and cloud rendering solution

Participating media material definition

Physically based and energy conservative scattering formulation

Supports dynamic time of day and evolving weather

Simulation of interactions between sky, sun, aerial perspective and clouds

Default: earth/sun/moon physical values

Simulation parameters can all be changed to simulate extra-terrestrial worlds

Practical use in Mirror’s Edge Catalyst: [Christin16]


Used widely in production by EA Games

Need for Speed, Ghost

Mass Effect Andromeda, BiowareFIFA 2017


Future work

Aerial perspective scattering shadowing by opaque and translucent clouds

Can’t use epipolar sampling min/max optimisation due to translucent clouds

Directional sky ambient by using cloud local gradient as main direction?

Convolution of sky envmap with phase function for cloud ambient lighting

Dynamic weather texture for cloud animation

GI and reflection volumes update

Still a problem when it comes to delay and performance

A complex problem with different priorities for each games


References[Adams74] The influence of Ozone and Aerosol on the Brightness and Color of the twilight Sky, JAS 1974.

[Bouthors08] Interactive Multiple Anisotropic Scattering in Clouds, I3D08.

[Bruneton08] Precomputed Atmospheric scattering, EGSR 2008.

[Christin16] Lighting the City of Glass, GDC 2016.

[CIESky] Spatial distribution of Daylight

[Elek09] Rendering Parametrizable Planetary Atmospheres with Multiple Scattering in Real-time, CESCG 2009.

[Guerrette14] Moving the heavens, GDC 2014.

[Harris02] Real-Time Cloud Rendering for Games, GDC 2002.

[Hestroffer98] Wavelength dependency of the Solar limb darkening, Astronomy and Physics 1998.

[Hillaire15] Physically Based and Unifier Volumetric Rendering in Frostbite, SIGGRAPH 2015.

[Hosek12] An Analytic Model for Full Spectral Sky-Dome Radiance, SIGGRAPH 2012

[Kutz12] Blog post, 2012.

[Lagarde&deRousiers14] Moving Frostbite to PBR, SIGGRAPH 2014.

[NASA] http://www.nasa.gov/multimedia/imagegallery/image_feature_347.html

[Nishita93] Display of the Earth Taking into Account Atmospheric Scattering, SIGGRAPH 1993.

[PBR] Physically Based Rendering book, www.pbrt.org.

[Preetham99] A Practical Analytic Model for Daylight, SIGGRAPH 99.

[Riley04] Efficient Rendering of Atmospheric Phenomena, EGSR 2004.

[Schneider15] The Real-time Volumetric Cloudscapes of Horizon: Zero Dawn, SIGGRAPH 2015.

[Yusov13] Outdoor Light Scattering, GDC 2013.

[Yusov14] High-Performance Rendering of realistic Cumulus Clouds using Pre-Computed Lighting, HPG14.

[Wenzel07] Real time atmospheric effects in game revisited, GDC 2007.

[Wrenninge10] Oz: The Great and Volumetric, SIGGRAPH 2010.

http://journals.ametsoc.org/doi/pdf/10.1175/1520-0469(1974)031<1662:TIOOAA>2.0.CO;2

http://www-evasion.imag.fr/Publications/2008/BNMBC08/clouds.pdf

http://www-ljk.imag.fr/Publications/Basilic/com.lmc.publi.PUBLI_Article@11e7cdda2f7_f64b69/article.pdf

http://www.slideshare.net/DICEStudio/lighting-the-city-of-glass

http://www.cie.co.at/index.php/index.php?i_ca_id=476

http://www.cescg.org/CESCG-2009/papers/PragueCUNI-Elek-Oskar09.pdf

http://www.gdcvault.com/play/1020146/Moving-the-Heavens-An-Artistic

http://www.markmark.net/PDFs/RTCloudsForGames_HarrisGDC2002.pdf

http://www.physics.hmc.edu/faculty/esin/a101/limbdarkening.pdf

http://www.frostbite.com/2015/08/physically-based-unified-volumetric-rendering-in-frostbite/

http://cgg.mff.cuni.cz/projects/SkylightModelling/HosekWilkie_SkylightModel_SIGGRAPH2012_Preprint_lowres.pdf

http://skyrenderer.blogspot.se/2012/10/ozone-absorption.html

http://skyrenderer.blogspot.se/2012/10/ozone-absorption.html

http://www.frostbite.com/wp-content/uploads/2014/11/course_notes_moving_frostbite_to_pbr.pdf

http://www.nasa.gov/multimedia/imagegallery/image_feature_347.html

http://nishitalab.org/user/nis/cdrom/sig93_nis.pdf

http://www.pbrt.org/

http://www.pbrt.org/

https://www.cs.utah.edu/~shirley/papers/sunsky/sunsky.pdf

http://wwwvis.informatik.uni-stuttgart.de/~kraus/preprints/egsr2004_riley.pdf

http://www.slideshare.net/guerrillagames/the-realtime-volumetric-cloudscapes-of-horizon-zero-dawn

https://software.intel.com/en-us/blogs/2013/09/19/otdoor-light-scattering-sample-update

http://www.highperformancegraphics.org/2014/wp-content/uploads/sites/3/2014/07/Yusov-Clouds.pdf

http://amd-dev.wpengine.netdna-cdn.com/wordpress/media/2012/10/D3DTutorial_Crytek.pdf

http://magnuswrenninge.com/wp-content/uploads/2010/03/Wrenninge-OzTheGreatAndVolumetric.pdf


Battlefield, Mass Effect, Need for Speed, FIFA, Plant vs Zombies, and many more.

We are hiring!

http://www.dice.se/jobs/


Questions?

Thanks!

The Frostbite rendering Team and game teams

Stephen Hill and Stephen McAuley for the course organisation

All the reviewers

Eric Bruneton and Fabrice Neyret

For further discussions, get in touch

[email protected]

https://twitter.com/SebHillaire

Additional notes and Mathematica notebook to follow soon…

mailto:[email protected]

https://twitter.com/SebHillaire


Bonus slides


Sky – how many scattering orders?

Single Scattering 2nd scattering order 4th scattering order


Scattering texture packing

ScatteringLUTs(h, θv, θl) is not storing (RayScat𝒓𝒈𝒃, MieScat𝒓𝒈𝒃)

But (RayScat𝒓𝒈𝒃, MieScat)

MieScat𝒓𝒈𝒃 = RayScat𝒓𝒈𝒃 ×MieScat

RayScat𝒓×σ𝒔𝒓𝒂𝒚

𝒓

σ𝒔𝒓𝒂𝒚

(RayScat𝒓𝒈𝒃, MieScat𝒓𝒈𝒃) (RayScat𝒓𝒈𝒃, MieScat)


Height fog

Uses scattered luminance at horizon

Seamless sky/fog transition

Per pixel coverage

Per pixel phase function

Approximations

Assumes low altitude fog

Ignores opaque shadows and self shadowing

HorizonP

ScatteredLightP ≈ ScatteredLightHorizon


Sun disk limb darkening Attenuate luminance near edge of sun disk (Default = sun model [Hestroffer98])

(Very low exposure) No

lim

b d

ark

en

ing

Su

n m

od

el

Str

on

ger


Night time

Moon and stars rendered as sprite using celestial system

Moon

Angular radius ≈ 0,24 to 0,28 degree

Luminance ≈ 2500 cd/m2

Secondary sky/atmosphere scattering source

Stars

Luminance? Angular radius?


Night timeSun set


Ambient contribution

Ambient component

Capture the sky environment without sun disk

Integrate luminance to scatter

Approximations

Phase function as uniform

Single environment color ignoring cloud self occlusion


Multi-scattering approximation Use [Wrenninge10]: multi-octaves single scattering / extinction / phase

Without multi-scattering


Multi-scattering approximation Use [Wrenninge10]: multi-octaves single scattering / extinction / phase

Withmulti-scattering


Cloud coverage and scattering influenceThick cloud

+ aerial perspective+ hemisphere coverage/scattering

Thick cloud only

Clouds do not fit

Thick cloud + aerial perspective

Aerial perspective too bright


Sunny 16 validation Sun illuminance at ground level: 100000

Camera: ISO 100, aperture f/16 and shutter speed 1/125

Physically Based Sky, Atmosphere and Cloud Rendering in Frostbite

Presentations & Public Speaking

Transcript of Physically Based Sky, Atmosphere and Cloud Rendering in Frostbite