Tutorial MPEG 3D Graphics
-
Upload
marius-preda -
Category
Technology
-
view
1.869 -
download
2
description
Transcript of Tutorial MPEG 3D Graphics
![Page 1: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/1.jpg)
Marius Preda, PhD Chairman of MPEG 3DG
Institut TELECOM
Web3D 2011, June, 20, Paris
![Page 2: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/2.jpg)
It’s a crazy multimedia world! – Network is everywhere… but very heterogeneous – Terminals are “same same”… but different! – Content must be designed with a priori knowledge of its future use – Applications are platform-centric instead of user-centric
Fragmented value chain
Authoring tools
Capturing devices
HW/SW providers
IPR holders
Production
Providers
Networks
Service scenarios
Transmission
Playback devices Terminal
manufacturers
End users
Consumption
INTEROPERABILITY NEEDED!!!
![Page 3: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/3.jpg)
No common representation – Heterogeneous kind of data
• Different types of geometry, appearance and animation models – Always easier to specify a new data representation format than
learning an existent one
Very different application domains
![Page 4: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/4.jpg)
![Page 5: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/5.jpg)
![Page 6: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/6.jpg)
FAST TRANSPORT
TROUGHT THE
NETWORK
Size doesn’t matter Size is of main
importance Size doesn’t matter
![Page 7: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/7.jpg)
FAST TRANSPORT
TROUGHT THE
NETWORK
Size doesn’t matter
Size is of main importance
Size doesn’t matter
![Page 8: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/8.jpg)
To define a standard format for compressed 3D synthetic content. In other words to be for graphics what MP3 and AAC are for audio, MPEG-2 and MPEG-4 are for video and JPEG is for still images.
Additionally "MPEG 3D Graphics" aims at providing mechanisms such as APIs to enable easy integration and development of applications using its standard representation tools.
![Page 9: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/9.jpg)
MPEG 3D Graphics within the MPEG standards family
Core technologies in MPEG 3D Graphics
Compressing other standards (COLLADA, X3D, …) with MPEG
Virtual Worlds Interoperability for avatars with MPEG
![Page 10: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/10.jpg)
MPEG 3D Graphics within the MPEG standards family
Core technologies in MPEG 3D Graphics
Compressing other standards (COLLADA, X3D, …) with MPEG
Virtual Worlds Interoperability for avatars with MPEG
![Page 11: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/11.jpg)
ISO
JTC1
SC24 Computer graphics and image processing
SC29 Coding of audio, picture, multimedia and hypermedia information
WG1 Coding of still pictures
WG11 Coding of moving pictures and audio
GKS, PHIGS CGM, VRML, X3D
Over 224 technical committees
JPEG
MPEG
Joint Technical Committee with IEC
MPEG has several names • Common: MPEG = Moving Picture Experts Group • Official: MPEG = ISO/IEC JTC1/SC29/WG11
![Page 12: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/12.jpg)
MPEG has several subgroups
MPEG
MPEG-1 ISO/IEC 11172 ’92 Video & audio
(CD-ROM)
MPEG-2 ISO/IEC 13818 ’94 Video & audio (DVD & DVB)
MPEG-4 ISO/IEC 14496 ’99
Multimedia & interactive applications
MPEG-7 ISO/IEC 15938 ’01
Metadata (description of content)
MPEG-21 ISO/IEC 21000 ’02
Terminal & network
specification
Requirements Systems
Video
Audio
3D Graphics
... and produced several successful standards
MPEG-A MPEG-B MPEG-C MPEG-D MPEG-E MPEG-M MPEG-U MPEG-V
![Page 13: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/13.jpg)
Naturals – still images, audio, 2D/3D video
Provide technologies for efficient compression and transmission
Synthetic - audio, 2D/3D objects and scenes
Composition, at end-user side, of natural and synthetic objects, into hybrid and interactive scenes
Compression Compression
Compression
MPEG-4 Features MPEG-4 Objects
MPEG-4 Objectives
MPEG-4 Terminal
![Page 14: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/14.jpg)
Object descriptor
layer
Interactive Scene Description
MPEG-4 stream
Media data layer
Scene layer
MPEG-4 Features System architecture
![Page 15: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/15.jpg)
MPEG-4 3D chronology
![Page 16: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/16.jpg)
MPEG-4 3D chronology
![Page 17: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/17.jpg)
MPEG-4 3D chronology
![Page 18: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/18.jpg)
MPEG 3D Graphics within the MPEG standards family
Core technologies in MPEG 3D Graphics
Compressing other standards (COLLADA, X3D, …) with MPEG
Virtual Worlds Interoperability for avatars with MPEG
![Page 19: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/19.jpg)
IFS surfaces
![Page 20: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/20.jpg)
Approximation of target surface • Method: tesselation with planar facets • Quality: first order (linear) ⇒ no smoothness (C0 continuity)
Mesh definition: IFS (Indexed Face Set) • Connectivity: list of faces {…, Pn = {in0, in1, in2, …}, …}
⇒ arbitrary topology (non-manifold, open, higher genus, etc.) • Geometry: list of vertices {…, Vk = (xk, yk, zk), …}
Mesh coding • Connectivity (lossless): triangle strips, triangle+vertex trees, etc. • Geometry (lossy): coordinate quantisation + prediction from conn.
![Page 21: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/21.jpg)
LOD concept • 1976: Clark introduced the idea • Main interest: rendering efficiency
Taxonomy of LOD extraction techniques • Static vs. dynamic • Global vs. local • Progressive vs. hierarchical LODs
Successful simplification techniques • 1996: Hoppe’s edge collapses • 1997: Garland’s quadrics ⇒ qslim
Progressive 3D mesh coding • 1996: Hoppe’s PM (Progressive Mesh) • 1998: IBM’s PFS (Progressive Forest Split)
![Page 22: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/22.jpg)
29519 Triangles: 69451 vs.
![Page 23: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/23.jpg)
9575 Triangles: 69451 vs.
![Page 24: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/24.jpg)
1627 Triangles: 69451 vs.
![Page 25: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/25.jpg)
238 Triangles: 69451 vs.
![Page 26: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/26.jpg)
MPEG-4 (1999): IFSs • Based on VRML97 • Arbitrary topology meshes • “Properties” (normals, colours and textures)
MPEG-4 Amd.1 (2000): 3DMC (3D Mesh Coding) • 40-50:1 compression of IFSs by IBM’s TS (Topological Surgery) • Incremental transmission and rendering • Progressive coding by IBM’s PFS • Error resilience by SAIT
![Page 27: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/27.jpg)
MPEG-4 (1999): IFSs • Based on VRML97 • Arbitrary topology meshes • “Properties” (normals, colours and textures)
MPEG-4 Amd.1 (2000): 3DMC (3D Mesh Coding) • 40-50:1 compression of IFSs by IBM’s TS (Topological Surgery) • Incremental transmission and rendering • Progressive coding by IBM’s PFS • Error resilience by SAIT
![Page 28: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/28.jpg)
IFS surfaces
![Page 29: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/29.jpg)
IFS surfaces Patches
![Page 30: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/30.jpg)
Approximation of target surface • Method: tesselation with predefined curved patches
• Quality: higher order (polynomic/rational) ⇒ Cn continuity
Mesh definition • Connectivity: regular grid of quads. or triangles
⇒ planar topology • Geometry: list of control points {…, Pk = (xk, yk, zk), …}
Mesh coding • Connectivity (lossless): implicit • Geometry (lossy): coordinate quantisation + prediction from conn.
![Page 31: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/31.jpg)
Tensor product of cubic Bézier curves
Compression
(3547 polygons; 1215 vertices)
vs.
(86 patches; 212 control points)
Single patch (4x4 control points) vs.
two patches (4x7 control points)
![Page 32: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/32.jpg)
MPEG-4 Part 16 (2003): NURBS • Based on VRML97 Amd., originally proposed by blaxxun • Support for NURBS curves and patches
Specific nodes for Bézier’s curves and patches (for increased efficiency)
• Support for free-form deformations
![Page 33: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/33.jpg)
IFS surfaces Patches Subdivision surfaces
![Page 34: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/34.jpg)
SS = limit of recursive refinement of base control mesh
NB: refinement affects both connectivity (of abstract graph) and
geometry (of 3D mapping)
z
y x
Geometry smoothing achieved with stencils particular to each scheme
1/8 1/8
½
½
-1/16 -1/16
-1/16 -1/16
-1/16 -1/16 9/16 9/16
Border/sharp vs. interior edge stencils
of “butterfly” scheme
SS inherently define hierarchically nested LODs
![Page 35: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/35.jpg)
Approximation of target surface • Method: tesselation with curved patches
• Quality: higher order ⇒ Cn continuity
Mesh definition • Connectivity: list of triangles/quads., e.g., {…, Tn = {in0, in1, in2}, …}
⇒ arbitrary (manifold) topology • Geometry: list of control points {…, Pk = (xk, yk, zk), …}
Mesh coding • Connectivity (lossless): as for polygonal (manifold) mesh • Geometry (lossy): as for polygonal (manifold) mesh
![Page 36: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/36.jpg)
Polygons + Are the simplest approach (linear approximation) + Can resolve fine details and handle arbitrary topologies – Lead to unstructured, huge meshes
Patches + Are a more powerful approach (higher order approximation) + Are convenient for coarse and smooth models – Need cumbersome trimming and stitching mechanisms
SSs + Connect and unify the two extremes above ++ Provide multi-resolution handles for hierarchical coding/editing
![Page 37: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/37.jpg)
Dyn++’s “butterfly” (1990): triangular, primal, interpolating, C1
Loop’s (1987): triangular, primal, approximating, C2
Catmull-Clark’s (1978): quadrilateral, primal, approximating, C2
![Page 38: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/38.jpg)
IFS surfaces Patches Subdivision surfaces Wavelet subdivision surfaces
![Page 39: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/39.jpg)
Target surface Base mesh
Subdivide Add details
![Page 40: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/40.jpg)
Price (requirements) • Base mesh extraction • Subdivision scheme = predictor • Details (3D vectors) = prediction errors ⇒ remeshing
Prize (advantages) • Predictive coding ⇒ immediate (if smooth target mesh)
Target surface Base mesh
Subdivide Add details
![Page 41: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/41.jpg)
ZT ZT ZT ZT …
Spatial partitioning Adding details in appearing parts
Removing details in disappearing parts
![Page 42: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/42.jpg)
by courtesy of FranceTelecom
![Page 43: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/43.jpg)
MPEG-4 Part 16 (2003): “plain” + wavelet SSs • “Plain” SSs for mesh smoothing
Considered schemes: Catmull-Clark, [extended] Loop, butterfly No details are added but… normal control achievable through edge/vertex tagging of initial control mesh
• Wavelet/detailed SSs for surface approximation Possibly tagged base mesh Details are added after each subdivision step, which are… wavelet-transformed according to one of several possible schemes Most suited for multi-resolution editing/animation Most suited for view-dependent transmission
![Page 44: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/44.jpg)
IFS surfaces Patches Subdivision surfaces Wavelet subdivision surfaces Mesh Grid
![Page 45: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/45.jpg)
RG (Reference-Grid)
CW (Connectivity-Wireframe)
![Page 46: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/46.jpg)
Vertex offset is a relative value Update vertex position when grid is deformed or animated
![Page 47: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/47.jpg)
![Page 48: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/48.jpg)
IFS surfaces Patches Subdivision surfaces Wavelet subdivision surfaces Mesh Grid Solids
![Page 49: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/49.jpg)
Solid primitives
Solid models: the “arithmetic of forms”
![Page 50: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/50.jpg)
Implicit equation:
Quadrics (2nd order):
Quartics (4th order):
![Page 51: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/51.jpg)
F1
* 0 1 2
F0 0 0 0 0
1 0 1 2
2 0 2 4
Multiplication of two forms
A cube = multiplication of 3 degenerated quadrics
Examples of solid operations
![Page 52: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/52.jpg)
Virtual models
Architecture Mechanics
Biotechnology
![Page 53: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/53.jpg)
Exact geometry
21 Kb 37 Kb 1.1 Mb 407 Kb
Compactness
![Page 54: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/54.jpg)
IFS surfaces Patches Subdivision surf. Wavelet SS Mesh Grid Solids
SC-3DMC
![Page 55: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/55.jpg)
MPEG 3DGC Scalable Complexity 3D Mesh Compression
• Not all 3DG applications have the same needs in compression • Not all the 3DG applications can afford spending extra CPU/GPU for compression
QBCR
SVA
TFAN
Towards the continuum model: enlarge the application domain where MPEG-4 3DG can be used
SC3DMC
Same Quantization and Binarization blocks
![Page 56: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/56.jpg)
MPEG 3DGC Scalable Complexity 3D Mesh Compression
x, y, z (floats) x, y, z (floats) x, y, z (floats)
i, j, k (integers) i, j, k (integers)
i, j, k (integers)
+ normals (floats) + colors (floats) + … (floats)
Connectivity
Attributes
![Page 57: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/57.jpg)
SC-3DMC General Schema
TFAN
SVA
CABAC
AC
BP
FLB
Parallelogram
Delta Quantization
Barycentre
CABAC
AC
BP
FLB
Delta
Connectivity Analysis
Prediction Binarization Entropy Encoding
Connectivity (lossless)
BAC
Attributes (lossy)
5 different paths with different performances
Prediction Binarization Entropy Encoding
x, y, z
i, j, k
![Page 58: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/58.jpg)
SC-3DMC Connectivity Analysis: Empty
Do Nothing
4 3
2
1
123234
![Page 59: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/59.jpg)
SC-3DMC Connectivity Analysis: SVA
Previous Face Current Face Shared Vertex
4 3
2
1
1
2 3
4
5 6
3
2 4
1 5
1
2 3
1
2 3 OR
Mode 0 Mode 2
Mode 1 Mode 3
4 modes for encoding how consecutive faces share vertices
1, 2, 3, 2, 3, 4
1, 2, 3, 3, 4, 5
1, 2, 3, 4, 5, 6
1, 2, 3, 1, 2, 3
12304 123245
1231 5 1233
![Page 60: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/60.jpg)
SC-3DMC Connectivity Analysis: TFAN
Split the mesh in set of triangle fans and encode each fan
1675 62937 2984
Transformed in local indices
![Page 61: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/61.jpg)
SC-3DMC What we measure?
Encoding performances but also complexity for decoder (and encoder)
3DMC
BIFS Complexity
Performances
NCA
SVA-BP SVA-BAC
TFAN SVA-AC
![Page 62: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/62.jpg)
IFS surfaces Patches Subdivision surf. Wavelet SS Mesh Grid Solids
SC-3DMC
![Page 63: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/63.jpg)
• Visual Texture Coding • Synthesized Textures • Procedural Textures • Depth Image-based Representation Less calculation
intensive
Very Compact representation for specific applications
Generic compression tool
![Page 64: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/64.jpg)
2D Texture
3D Mesh
3D object
• MPEG-4 3DMC • MPEG-4 MeshGrid • MPEG-4 Subdivision Surfaces
• MPEG-4 VTC • JPEG 2000
![Page 65: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/65.jpg)
3D mipmaps
WT
Zero-Tree
![Page 66: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/66.jpg)
Without mipmapping With mipmapping
Multiple In Place mapping = multi-resolution textures
![Page 67: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/67.jpg)
WT
Zero-Tree
Computational Graceful Degradation: Quality => processing power
![Page 68: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/68.jpg)
Region of Interest with resolution/quality selection
High Resolution
Low Resolution
Visual importance depends on viewing angle
![Page 69: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/69.jpg)
Packet selection by using error-resilience markers
![Page 70: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/70.jpg)
IFS surfaces Patches Subdivision surf. Wavelet SS Mesh Grid Solids
VTC Synthesized texture
SC-3DMC
![Page 71: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/71.jpg)
Line ( LN ), bounded by 2 Terminal Points ( TP ) Line Segment ( LS ), bounded by 2 Line Points ( LP )
The image cannot be displayed. Line marked as control line (Skeleton)
Line Color Profile ( LC ) Area Color Point ( A C )
The image cannot be displayed. Color Patch ( PA )
Sub - Texture ( ST )
Color profile
Vector Graphics Primitives
![Page 72: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/72.jpg)
65x96 pixels 10 seconds animation 1.35 kB
![Page 73: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/73.jpg)
IFS surfaces Patches Subdivision surf. Wavelet SS Mesh Grid Solids
VTC Synthesized texture Procedural texture
SC-3DMC
![Page 74: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/74.jpg)
DEF Fabric ProceduralTexture { type 2 width 256 height 256 cellWidth 4 cellHeight 4 roughness 1 distortion 0.05 seed 114300 color [ 0.898 0.89418 0.95294, 0.34118 0.29418 0.70196, 0 0 0, 0 0 0 ] aWarpmap [ 0 0, 0.03 1, 0.88 1, 1 0 ] bWarpmap [ 0 0, 0.48 1, 1 0 ] aWeights [ 0, 0, 0, 0, 0, 0, 0.56, 0, 0, 0, 0, 0, 0, 0, 0.20, 0.24 ] bWeights [ 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0 ] }
DEF Marble ProceduralTexture { width 256 height 256 roughness 1 seed 22209 color [ 0.8 0.7098 0.6902, 0.95686 0.8902 0.87451, 0.87451 0.37255 0.23529, 0.95686 0.8902 0.87451 ] aWarpmap [ 0 1, 0.33 0, 1 1 ] bWarpmap [ 0 0, 0.55 0, 0.6 1, 0.65 0, 1 0 ] bWeights [ 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0 ] }
![Page 75: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/75.jpg)
IFS surfaces Patches Subdivision surf. Wavelet SS Mesh Grid Solids
VTC Synthesized texture Procedural texture DIBR
SC-3DMC
![Page 76: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/76.jpg)
Reconstruct 3D representation from projections
Depth image
Projection
![Page 77: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/77.jpg)
VTC Synthesized texture Procedural texture DIBR Point Texture
IFS surfaces Patches Subdivision surf. Wavelet SS Mesh Grid Solids
SC-3DMC
![Page 78: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/78.jpg)
Interpolators VTC Synthesized texture Procedural texture DIBR Point Texture
IFS surfaces Patches Subdivision surf. Wavelet SS Mesh Grid Solids
SC-3DMC
![Page 79: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/79.jpg)
Straightforward animation based on key-frames
Piece-wise linear functions defined by a set of pairs time-value
Standardized by MPEG-4 for: - position - orientation - scale
- coordinate in IFS - normals in IFS
![Page 80: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/80.jpg)
Large amount of data for high quality, smooth animation
![Page 81: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/81.jpg)
Exploit temporal redundancy for the most common interpolators
Re-sampling, sub-sampling: two methods supported by MPEG-4
Path preserving mode Key preserving mode
Three schemes supported by MPEG-4
Coordinate Interpolators Orientation Interpolators
Position Interpolators
![Page 82: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/82.jpg)
Sub-sampling or re-sampling based on minimal distortion
Key(time)
Key value
N-1 key values
Key value
N-2 key values
Key(time)
Key value
N key values
Key(time)
Key value
N-3 key values
Key(time)
Re-sampling, sub-sampling: two methods supported by MPEG-4
Path preserving mode Key preserving mode
Three schemes supported by MPEG-4
Coordinate Interpolators Orientation Interpolators
Position Interpolators
![Page 83: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/83.jpg)
A dedicated elementary stream for IC, multiplexed into an AFX stream
![Page 84: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/84.jpg)
Interpolators Bone-Based
VTC Synthesized texture Procedural texture DIBR Point Texture
IFS surfaces Patches Subdivision surf. Wavelet SS Mesh Grid Solids
SC-3DMC
![Page 85: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/85.jpg)
Two frameworks: human-like (FBA) and generic skeleton (BBA)
Face, MPEG-4 V1, 1999
Skinned Model, MPEG-4 Part 16, 2003
Body, MPEG-4 Amd1, 2000
![Page 86: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/86.jpg)
Seamless mesh affected by a hierarchical skeleton
Geometry: Seamless mesh: shapes sharing the same vertices list
Texture: Image Mapping on vertices sub-set
Hierarchy: Skeleton layer Muscle layer
![Page 87: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/87.jpg)
Right balance between control parameters and influence volume
1D controllers: bone & muscle
for each bone and each muscle - a list of affected vertices - a measure of affectedness are provided
![Page 88: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/88.jpg)
Basic comp.: prediction, freq.transform, quantization and entropy encoding
Frame #n Uncompressed BAP/FAP/BBA
Binary file
Frame P
Prediction
Arithmetic coding
Quantization
Frame I
Arithmetic coding
Quantization
Segment #n Uncompressed BAP/FAP/BBA
DCT
Binary file
Segment P
Prediction
DC Q
Segment I
Huffman coding
DC Q
DC Coeff. AC Coeff.
Huffman coding
AC Q
![Page 89: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/89.jpg)
Very low bit-rate
Body, MP4 Amd1, 5-30 kbps
Face, MP4 V1, 2kbps
Skinned Model, MP4 Part 16, 5-30 kbps for a human like skeleton
![Page 90: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/90.jpg)
Interpolators Bone-Based Morphing
VTC Synthesized texture Procedural texture DIBR Point Texture
IFS surfaces Patches Subdivision surf. Wavelet SS Mesh Grid Solids
SC-3DMC
![Page 91: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/91.jpg)
Local and precise control for shape deformation
Defined as a base mesh and a collection of target meshes
Animation obtained by updating the weights of the target meshes
BBA stream updated to include morph data
Usable for any kind of 3D object
![Page 92: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/92.jpg)
Interpolators Bone-Based Morphing FAMC
VTC Synthesized texture Procedural texture DIBR Point Texture
IFS surfaces Patches Subdivision surf. Wavelet SS Mesh Grid Solids
SC-3DMC
![Page 93: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/93.jpg)
Cluster the vertices with respect to their motion
Encode a cluster motion by an affine transform
Encode the residual error at vertex level by traditional approach (DCT/W, quantization, entropy encoding)
![Page 94: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/94.jpg)
Interpolators Bone-Based Morphing FAMC
VTC Synthesized texture Procedural texture DIBR Point Texture Remote and
programmatic
IFS surfaces Patches Subdivision surf. Wavelet SS Mesh Grid Solids
SC-3DMC
![Page 95: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/95.jpg)
Complex application scenario can be built
Programmatic animation - ECMA Script
part of the scene description - Java Code standardized API for accessing the scene graph
scene Script Node
MPEG-J Stream scene
Bifs-Command scene Remote animation
BIFS-Anim & BIFS Command
Bifs-Anim
![Page 96: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/96.jpg)
Interpolators Bone-Based Morphing FAMC
VTC Synthesized texture Procedural texture DIBR Point Texture Remote and
programmatic
IFS surfaces Patches Subdivision surf. Wavelet SS Mesh Grid Solids SC-3DMC
![Page 97: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/97.jpg)
MPEG 3DG How measuring compression performances?
On-line benchmarking platform: www.MyMultimediaWorld.com
• Allows easy integration of proprietary algorithms by using an API in C++
• No need to disclaim the algorithm source code
• Benchmark automatically updated for new content
• Restricts and refines the benchmark by means of easy-to-control parameters.
![Page 98: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/98.jpg)
MPEG 3DG How measuring compression performances?
On-line benchmarking : www.MyMultimediaWorld.com
MP4
Extended MP7
Filter and Presentation Engine
Web site 3D compression benchmark Benchmark 2 Benchmark 3
Indexed MDB
3D Compression benchmark manager
Algorithm 1 Algorithm n Algorithm 2 …
AP
I
Ben
chm
ark
man
ager
1 B
enchmark m
anager 2
![Page 99: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/99.jpg)
MPEG 3DG How measuring compression performances?
On-line benchmarking : www.MyMultimediaWorld.com
Proprietary Coder library
Proprietary Decoder library
BitStream (1,n)
n
.Compressed_VB (1,n)
Decoder
DumpCompressedVB
Encoder
GetNbBitstream
DumpBitSream
Vertex Buffer
![Page 100: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/100.jpg)
Benchmarking platform, per-object visualization
- object properties (number of vertices/triangles, number of components and files size, - distortion graph (linear or logarithmic), - compression gain with respect to 3DMC. - encoding time - decoding times.
MPEG 3DG How measuring compression performances?
![Page 101: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/101.jpg)
Benchmarking platform, global visualization
Filters
- semantic category, - average distortion, - number of vertices, - number of connected components in the object, - database subset.
MPEG 3DG How measuring compression performances?
![Page 102: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/102.jpg)
Benchmarking platform, global visualization
MPEG 3DG How measuring compression performances?
![Page 103: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/103.jpg)
MPEG 3D Graphics within the MPEG standards family
Core technologies in MPEG 3D Graphics
Compressing other standards (COLLADA, X3D, …) with MPEG
Virtual Worlds Interoperability for avatars with MPEG
![Page 104: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/104.jpg)
![Page 105: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/105.jpg)
![Page 106: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/106.jpg)
Geometry compression: -compression ratio 40:1
Animation compression: -compression ratio 100:1
![Page 107: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/107.jpg)
XML Scene Representation Any XML
(Binary) XML XML Binarisation
3D Graphics Compression
![Page 108: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/108.jpg)
Layer 1:
Textual XML representation of the scene
XML Scene Representation Any XML
(Binary) XML XML Binarisation
3D Graphics Compression
![Page 109: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/109.jpg)
Layer 2:
Binarized XML layer
Contains the unclassified elements of the scene graph
XML Scene Representation Any XML
(Binary) XML XML Binarisation
3D Graphics Compression
![Page 110: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/110.jpg)
Layer 3:
Compressed layer
Contains specific types of media information (geometry, animation, ...)
XML Scene Representation Any XML
(Binary) XML XML Binarisation
3D Graphics Compression
![Page 111: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/111.jpg)
Result:
Multiplexed layers 2 and 3
XML Scene Representation Any XML
(Binary) XML XML Binarisation
3D Graphics Compression
![Page 112: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/112.jpg)
Comp ratio: 40-70:1
MPEG-4 P25 Encoder side: possible implementation (informative)
XMT,
COLLADA,
X3D
MP4
Encoder Side
![Page 113: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/113.jpg)
Lossless for the data structure, lossless or lossy for graphics primitives
MPEG-4 P25 Decoder side (normative)
XMT,
COLLADA,
X3D
MP4
Decoder Side
![Page 114: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/114.jpg)
MPEG 3D Graphics within the MPEG standards family
Core technologies in MPEG 3D Graphics
Compressing other standards (COLLADA, X3D, …) with MPEG
Virtual Worlds Interoperability for avatars with MPEG
![Page 115: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/115.jpg)
Standards for Avatars as visualization support
Standard Generic Features Avatar representation
COLLADA 3D objects/ scenes Generic Object
VRML/ X3D 3D objects /scenes
Application behavior
H- Anim
MPEG-4 2D/3D objects/ scenes
Application behavior
Compression
A set of dedicated nodes
A dedicated compressed stream
![Page 116: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/116.jpg)
Standards for Avatars as interaction support
Representation Features HumanML human physical description, emotion
EmotionML emotion, facial expressions, gestures
BML speech, gesture, gaze
MPML speech VHML facial and body animation, emotional representation
CML character attribute and animation definition
![Page 117: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/117.jpg)
Why none of the existing standards is solving the issue of avatar interoperability in Virtual Worlds?
– The Virtual Worlds are proprietary applications and the 3D assets including the avatars have economical value
– The Virtual Worlds and in general 3D applications have specific data format allowing rendering optimization
However, while maintaining a strict control for economic and technical reasons, Virtual Worlds allow users to personalize the avatars
![Page 118: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/118.jpg)
Attributes that can be modified by the user
Specify the set of Personalization Parameters (PP) that transforms a template of an arbitrary Virtual World into the user designed avatar
Avatar Template
![Page 119: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/119.jpg)
What avatar feature can be personalized?
Mainly the appearance
Analysis of SecondLife, IMVU, Entropia Universe, SonyPlaystation and HumanML
![Page 120: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/120.jpg)
Very heterogeneous set of personalization parameters
Second Life
Entropia Universe
Nintendo Wii
HumanML
PlayStation
![Page 121: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/121.jpg)
“Appearance” element
<Appearance>
<Body> <BodyHeight value=165/> <BodyFat value=15/> </Body>
<Head> <HeadShape value="oval"/> <EggHead value="true"/> </Head>
</Appearance>
![Page 122: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/122.jpg)
Interoperability at the Animation level
<Animation>
<Greeting> <Salute>salut</Salute> <Cheer>cheer</Cheer>
</Greeting>
<Fighting> <shoot>pousse</shoot> <throw>throw</throw> </Fighting>
</Animation>
![Page 123: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/123.jpg)
Motion retargeting
Avatar Template in VW1
“Walk” animation In VW1
Avatar Template in VW2
No “Walk” animation
defined in VW2
![Page 124: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/124.jpg)
“Control” element
<Control> <BodyFeaturesControl > <UpperBodyBones> <LCalvicle>my_LCalvicle</LCalvicle> <RClavicle>my_RCalvicle</RClavicle> </UpperBodyBones> </BodyFeaturesControl>
<FaceFeaturesControl> <HeadOutline> <Left X=0.23 Y=1.25 Z=7.26/> <Right X=0.25 Y=1.25 Z=7.21/> </HeadOutline> </FaceFeaturesControl>
</Control>
Control
Body Control
Face Control
![Page 125: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/125.jpg)
MPEG-V + MPEG-4 The Avatar in
VW1
The Avatar in VW2
The Avatar in an external player
MPEG-V (Personalization
Parameters)
![Page 126: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/126.jpg)
MPEG 3DGC Ongoing work
![Page 127: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/127.jpg)
MPEG 3DGC Multi-Resolution 3DMC
Current 3DG content in VW
Tomorrow’s 3DG content in VW (100 times denser)
© Second Life © Samsung
© OTOY © OTOY
![Page 128: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/128.jpg)
MPEG 3DGC Reconfigurable Graphics Coding
A framework that allows to set up the decoder at run time
![Page 130: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/130.jpg)
MPEG 3DGC Multi-resolution 3DMC
Progressive mesh [Hoppe’96] Progressive Forest Split (PFS) [Taubin’98] Patch coloring [Cohen-Or’99]
Valence-based decimation approach [Alliez’01] Octree-based compression [Peng’05]
Spectral coding [Karni’01]
![Page 131: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/131.jpg)
MPEG 3DGC Multi-resolution 3DMC : two proposed methods
1. Progressive TFAN Original
Interpolated with 1% of the vertices
![Page 132: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/132.jpg)
MPEG 3DGC Multi-resolution 3DMC : two proposed methods
2. KLT encoder
1
1
1
2
2
2
![Page 133: Tutorial MPEG 3D Graphics](https://reader037.fdocuments.net/reader037/viewer/2022102722/5557a5b4d8b42a696c8b4751/html5/thumbnails/133.jpg)
MPEG 3DGC Multi-resolution 3DMC : two proposed methods
1. Work on - document the use case scenarios (object examination, navigation, …) - database - comparison with other codecs - comparison method (PSNR, Resolution) - platform for testing
2. You are invited to contribute