1. ROME 11-12 april 2014ROME 11-12 april 2014 Virtual Reality & Sim Racing in Assetto Corsa g.romagnoli@kunos-simulazioni.com Giovanni Romagnoli Kunos Simulazioni

2. VR & SimRacing in Assetto CorsaROME 11-12 april 2014 3. VR & SimRacing in Assetto CorsaROME 11-12 april 2014 4. 1. Assetto Corsa VR & SimRacing in Assetto Corsa Ill be talking about : 1) Assetto Corsa 2) VR & SimRacing 3) Stereo Theory 4) Implementation 5. 1. Assetto Corsa VR & SimRacing in Assetto Corsa 1. Assetto Corsa 6. 1. Assetto Corsa VR & SimRacing in Assetto Corsa Simulated racing is a type of software that attempts to accurately simulate auto racing. Kunos Simulazioni 13 people working 3 Games published 7. 1. Assetto Corsa VR & SimRacing in Assetto Corsa Latest product : Assetto Corsa proprietary technology Graphic, Physics, Sound engine, 8. 1. Assetto Corsa VR & SimRacing in Assetto Corsa 1) Consumer SimRacers & Casual 2 different market categories 9. 1. Assetto Corsa VR & SimRacing in Assetto Corsa 2) Professional Companies interested in vehicle simulation 2 different market categories 10. 2. SimRacing & VR VR & SimRacing in Assetto Corsa 2. SimRacing & VR 11. 2. SimRacing & VR VR & SimRacing in Assetto Corsa On the professional VR isnt the new thing VR is a computer-created environment that can simulate the physical presence of the user in another world Presence VS Immersion VR = Physical + Display 12. 2. SimRacing & VR VR & SimRacing in Assetto Corsa 13. 2. SimRacing & VR VR & SimRacing in Assetto Corsa 14. 2. SimRacing & VR VR & SimRacing in Assetto Corsa 15. 2. SimRacing & VR VR & SimRacing in Assetto Corsa State of the art : multiple screen rendering Lets focus on how we display information Physical feedback is a solved problem 16. 2. SimRacing & VR VR & SimRacing in Assetto Corsa Single screen rendering 17. 2. SimRacing & VR VR & SimRacing in Assetto Corsa Triple screen rendering 18. 2. SimRacing & VR VR & SimRacing in Assetto Corsa 1) Reduce the problem to 2D 2) Make math work: For each Screen : Angle = asin ( dir ); VM = rotate( VM, Angle); Screen = rotate(Screen, -Angle); Proj(Screen, NearPlane); 3) Create the Off-Center Proj-Matrix Its easy to do : dir 19. 2. SimRacing & VR VR & SimRacing in Assetto Corsa Triple screen drawbacks : 1) Expensive rendering 2) Expensive setup 3) Merge & Blending 4) No presence 5) (brute force) 20. 2. SimRacing & VR VR & SimRacing in Assetto Corsa Whats the alternative to multiple rendering? Lets start to talk about optics 21. 3. Stereo Theory VR & SimRacing in Assetto Corsa 3. Stereo Theory 22. 3. Stereo Theory VR & SimRacing in Assetto Corsa 3.1 Optics How we perceive DEPTH 2 signals Focal Plane + 2D info+ 23. 3. Stereo Theory VR & SimRacing in Assetto Corsa Convergence : The meeting of lines of sight through the eyes at a common point 2 signals : 24. 3. Stereo Theory VR & SimRacing in Assetto Corsa No FusionFusionFusion 25. 3. Stereo Theory VR & SimRacing in Assetto Corsa Stereoscopy Stereoscopy is a technique for creating the illusion of depth in an image by means of binocular vision. 2 signals Focal Plane + 2D info+ 26. 3. Stereo Theory VR & SimRacing in Assetto Corsa 27. 3. Stereo Theory VR & SimRacing in Assetto Corsa Safe distance : Closest point > Stereo base * focal length. Other consequences of fixed focal plane : 28. 3. Stereo Theory VR & SimRacing in Assetto Corsa There are several ways to achieve Stereoscopy 29. 3. Stereo Theory VR & SimRacing in Assetto Corsa The device : Oculus Rift 30. 3. Stereo Theory VR & SimRacing in Assetto Corsa Its simple : OLED Display 75 Hz, low persistence Lenses to adjust the image to what the viewer expects 60 Hz Positional tracking 1000 Hz update rate for orientation 31. 3. Stereo Theory VR & SimRacing in Assetto Corsa Why not Before? Fast switching displays Computer Vision algorithms for positional tracking Computing power wasnt there Latency Price 32. 3. Stereo Theory VR & SimRacing in Assetto Corsa DK2 and SDK 2 way more powerful respect to DK1! Higher level of abstraction, tons of utilites e.g.: set low persistence mode sensorDevice->SetDisplayReport(); e.g.: get user info pHMD->GetProfile() How do we develop on it? 33. 3. Stereo Theory VR & SimRacing in Assetto Corsa and theres also a c interface for the library 34. 4. Implementation VR & SimRacing in Assetto Corsa 4. Implementation 35. 4. Implementation VR & SimRacing in Assetto Corsa Problematics to deal with Oculus Rift 1) Rendering in stereo 2) Resolution 3) Latency 4) GUI 5) Motion Sickness 36. 4. Implementation VR & SimRacing in Assetto Corsa 4.1) Rendering Stereo 37. 3. Stereo Theory VR & SimRacing in Assetto Corsa 3D Device User 38. 3. Stereo Theory VR & SimRacing in Assetto Corsa View Matrix Projection Matrix Distortion 39. 4. Implementation VR & SimRacing in Assetto Corsa Rendering in stereo : view matrix We need to render twice the scene with 2 different view matrix with no convergence Just shift the different view matrices by the interpupillary distance V = T V 40. 4. Implementation VR & SimRacing in Assetto Corsa Rendering in stereo : proj matrix a = aspect ratio FOV : 41. 4. Implementation VR & SimRacing in Assetto Corsa The problem is that the FOV in humans is not symmetric! We need to reason in terms of half fov (s) and set our proj matrix accordingly. 42. 4. Implementation VR & SimRacing in Assetto Corsa 43. 4. Implementation VR & SimRacing in Assetto Corsa Rendering in Stereo : distortion New SDK function to do this right (lens-independent) Pincushion Barrel Physical Software 44. 4. Implementation VR & SimRacing in Assetto Corsa The new SDK makes our life easier : It gives you a 3d mesh where to map your render target. (+ corrects for chromatic Ab.) Its easier to use & improve, for example adding a fade effect on the border 45. 4. Implementation VR & SimRacing in Assetto Corsa Trivial shader VS_OUTPUT ksVS(VS_INPUT input) { VS_OUTPUT Output; Output.TexCoord0 =float2(RED.X,RED.Y) ; [] Output.Color = float4(0,0,0,alpha); return Output; } float4 ksPS(VS_OUTPUT In) : SV_TARGET { float ResultR = Texture.Sample(Linear, In.TexCoord0).r; float ResultG = Texture.Sample(Linear, In.TexCoord1).g; float ResultB = Texture.Sample(Linear, In.TexCoord2).b; return float4(ResultR * In.Color.r, ResultG * In.Color.g, ResultB * In.Color.b, 1.0); } 46. 4. Implementation VR & SimRacing in Assetto Corsa 47. 4. Implementation VR & SimRacing in Assetto Corsa 4.2) Resolution 48. 4. Implementation VR & SimRacing in Assetto Corsa Resolution We used : Downsampling MSAA 2X FXAA 2X EYE : 1 pixel per 5/1000 degrees. OCULUS : 1 pixel per 1/6 degree Screen structure : Pixel Mask Grid vs Brick 49. 4. Implementation VR & SimRacing in Assetto Corsa 4.3) Latency 50. 4. Implementation VR & SimRacing in Assetto Corsa Latency & performance motion-to-photon time= time between movement of the users head and the updated image being displayed on the screen