How to Improve Visual - Microsoft · Aliasing 10 — Basics — Mipmapping — Levels of Details...
Transcript of How to Improve Visual - Microsoft · Aliasing 10 — Basics — Mipmapping — Levels of Details...
How to Improve Visual Rendering Quality in VR
Ryan O’Shea
3
4
5
PS4 to Mobile with Inside-out Camera Tracking
6
Current State of VR
7
— VR allows developers to deliver new heights of immersion to
players
— Yet currently VR, especially on standalone headsets, contains
a number of limitations
— These limitations make it an unrealistic, unconvincing and
sometimes an uncomfortable experience
— Therefore these limitations must be either solved or mitigated
Standalone VR
88
2xUplift in standalone VR headset
install base from previous year
38%Projected share of total
2019 VR headset sales*
Data from ABI Research report on 360 Video and Virtual Reality Devices and Services 2019
*Excluding mobile VR headsets
BandingAliasing
Common VR Pitfalls
9
ShadowsDepth
Aliasing
10
— Basics— Mipmapping
— Levels of Details (object and shader)
— Anti-aliasing
— Texture Filtering— Enabling texture filtering for textures is vital to remove aliasing from
mapping texels to pixels
— Colour Space— Linear colour space can reduce specular aliasing by preventing
objects from becoming white as brightness increases
— Alpha Compositing
Aliasing – Alpha Compositing
11
Alpha Test Alpha toCoverage
Aliasing – Alpha Compositing
12
— Alpha test and alpha blend are the two most common
methods of alpha compositing in use today
— Despite their popularity, these methods cause large amounts
of aliasing or a high performance cost
— Alpha to coverage is a method of alpha compositing that
suffers from less aliasing
— Maps the coverage mask from MSAA to the alpha output in
order to minimise aliasing
— If the alpha value is 0.5, only half of the coverage samples are
rendered to
123456789
10111213141516171819202122232425262728293031
AlphaToMask Onstruct frag_in{
float4 pos : SV_POSITION;float2 uv : TEXCOORD0;half3 worldNormal : NORMAL;
};
fixed4 frag(frag_in i, fixed facing : VFACE) : SV_Target{
/* Sample diffuse texture */fixed4 col = tex2D(_MainTex, i.uv);
/* Calculate current mip level */float2 texture_coord = i.uv * _MainTex_TexelSize.zw;float2 dx = ddx(texture_coord);float2 dy = ddy(texture_coord);float MipLevel = max(0.0, 0.5 * log2(max(dot(dx, dx), dot(dy, dy))));col.a *= 1 + max(0, MipLevel) * 0.25;
/* Sharpen texture alpha to the width of a pixel */col.a = (col.a - 0.5) / max(fwidth(col.a), 0.0001) + 0.5;clip(col.a - 0.5);
/* Lighting calculations */half3 worldNormal = normalize(i.worldNormal * facing);fixed ndotl = saturate(dot(worldNormal, normalize(_WorldSpaceLightPos0.xyz)));col.rgb *= ndotl * _LightColor0;
return col;}
13
123456789
10111213141516171819202122232425262728293031
AlphaToMask Onstruct frag_in{
float4 pos : SV_POSITION;float2 uv : TEXCOORD0;half3 worldNormal : NORMAL;
};
fixed4 frag(frag_in i, fixed facing : VFACE) : SV_Target{
/* Sample diffuse texture */fixed4 col = tex2D(_MainTex, i.uv);
/* Sharpen texture alpha to the width of a pixel */col.a = (col.a - 0.5) / max(fwidth(col.a), 0.0001) + 0.5;clip(col.a - 0.5);
/* Lighting calculations */half3 worldNormal = normalize(i.worldNormal * facing);fixed ndotl = saturate(dot(worldNormal, normalize(_WorldSpaceLightPos0.xyz)));col.rgb *= ndotl * _LightColor0;
return col;}
14
Banding
15
— Dithering— Process of introducing noise to the material in order to disrupt the
distinct bands and prevent the undesirable effect
— Tone Mapping— A subset of colour grading that transposes HDR colours to fit within
the LDR range suitable for displaying on screen, this removes sharp
colour gradients
16
No PostProcessing
17
Dithering & Tone Mapping
18
Dithering & Tone Mapping(Cropped)
Depth
19
— Normal Mapping— Provides additional details during lighting calculations producing
higher detail meshes without additional geometry
— The illusion is damaged when viewed through stereoscopic lenses
— Parallax Occlusion Mapping— An advanced method of normal mapping that accounts for the
viewing angle when displacing texture coordinates allowing for a
more realistic illusion of depth
20
Diffuse
21
Normal
22
Parallax Occlusion
23
Shadows
24
— Blob Shadows— Shadow mapping techniques which involve rendering accurate
shadows produce various artefacts
— Dynamic blob shadows can be rendered below objects in order to
provide fast shadows that do not break immersion
— Baked Shadows— Shadow baking doesn’t require any additional render passes, unlike
shadow mapping
— High quality shadows are baked into the textures that they cast
shadows upon
25
Shadow Map Blob Shadow
26
Conclusions
27
— There is a wide range of mitigations for aliasing and
considerate deployment of all of these is vital
— Banding is difficult to completely remove, so mitigations
should be utilised to avoid this pitfall
— Depth simulation is considerably more difficult in VR but
parallax occlusion mapping provides a great solution
— Shadows are vital for immersion and blob shadows provide a
nice middle ground between quality and impact
— All mitigations work best when combined with level design
that is considerate of VR’s weaknesses
Resources
28
— https://bit.ly/2mn7Atw - How to Improve Visual Rendering Quality in VR Guide
— https://community.arm.com/posts/virtual-reality-best-practices-for-mobile - Blog Post
— https://developer.arm.com/unity - Arm Unity Developer Page