3D Human Motion Capture - GPU Technology...
16
Copyright © 2017 1 3D Human Motion Capture from 2D Video using Cloud-Based CNNs NVIDIA GPU Technology Conference
Transcript of 3D Human Motion Capture - GPU Technology...
Copyright © 2017 1
3D Human Motion Capture
from 2D Video using
Cloud-Based CNNs
NVIDIA GPU Technology Conference
Copyright © 2017 2
Imagine if Cameras Could See People
Computer Vision
bodySLAM™ turns 2D video into 3D humans
2D video 3D human
Copyright © 2017 3
Applications: AR Social Imagery & Filters
Copyright © 2017 4
AR / VR Gaming
Copyright © 2017 5
Health & Wellness
Copyright © 2017 6
Sport Analytics
Copyright © 2017 7
Eyes for Virtual Assistants
Copyright © 2017 8
APIIn: 2D Video
fidelity/performance parameters
Out: Human FeaturesJoints: 21 joints including eyes and earsSkeletons: 2D, 3DSegmentation: background, individuals, limbs
PlatformsCUDAMetal Performance ShadersopenCL
BodySLAM™: Deep Learning API
Copyright © 2017 9
Deep Learning / Computer Vision Secret Sauce100+ layer CNN (Caffe)Synthetic Training dataHuman tracking
Game Tech3D animationJoint FilteringInverse Kinematics
PlatformsNVIDIA GPUAWSMPS iOS
Tech Stack
Copyright © 2017 10
High Level Architecture
Copyright © 2017 11
Training Pipeline
Copyright © 2017 12
BodyFilter™: VFX filters built on top of BodySLAM™
Copyright © 2017 13
Unity Game Engine for Training Pipeline and VFX App
Copyright © 2017 14
Live Demos
Copyright © 2017 15
Working with CustomersOptimizationNative Mobile SupportHigher Fidelity
Future Work
Copyright © 2017 16
Questions? Let’s Start Augmenting Reality Together!
