Rate-Distortion Optimization for Geometry Compression of Triangular Meshes
description
Transcript of Rate-Distortion Optimization for Geometry Compression of Triangular Meshes
Rate-Distortion Optimization for Geometry Rate-Distortion Optimization for Geometry Compression of Triangular MeshesCompression of Triangular Meshes
Frédéric PayanFrédéric Payan
I3S laboratory - CReATIVe Research GroupUniversité de Nice - Sophia Antipolis
Sophia Antipolis - FRANCE
PhD Thesis
Supervisor : Marc Antonini
MotivationsMotivations
Goal : Goal :
propose an efficient compression algorithm for highly propose an efficient compression algorithm for highly detailed triangular meshesdetailed triangular meshes
Objectives : Objectives : HighHigh compression ratio compression ratio Rate-Quality OptimizationRate-Quality Optimization Multiresolution Multiresolution approachapproach FastFast algorithm algorithm
SummarySummary
BackgroundBackground
Distortion criterion for multiresolution meshesDistortion criterion for multiresolution meshes
Optimization of the Rate-Distorsion trade-offOptimization of the Rate-Distorsion trade-off
Experimental resultsExperimental results
Conclusions and perpectivesConclusions and perpectives
SummarySummary
BackgroundBackground
Triangular MeshesTriangular Meshes
RemeshingRemeshing
Multiresolution analysis Multiresolution analysis
CompressionCompression
Bit allocationBit allocation
I. Background
Triangular MeshesTriangular Meshes
3D modeling3D modeling
Applications :Applications : Medecine Medecine CADCAD Map modelingMap modeling GamesGames CinemaCinema Etc.Etc.
I. Background
Irregular meshesIrregular meshesvalence different of 6valence different of 6
=> 2 informations :=> 2 informations : GeometryGeometry (vertices) (vertices) ConnectivityConnectivity (edges) (edges)
5 neighbors
9 neighbors
4 neighbors
I. Background
ExamplesExamples
99,732 triangles=> + 1.1 Mb
More than 380 millions of More than 380 millions of triangles triangles =>=> several Gigabytes several Gigabytes ((Michelangelo Project, Michelangelo Project, 1999)1999)
40,000 triangles=> + 0.45 Mb
MultiresolutionMultiresolution AnalysisAnalysis : :
Without connectivity modification => Without connectivity modification => wavelet wavelet transform for irregular meshes (transform for irregular meshes (S.Valette et R.Prost, S.Valette et R.Prost, 20042004))
A mesh is only one instance of the surface A mesh is only one instance of the surface geometry geometry => => RemeshingRemeshing
goal : goal : regular regular and and uniformuniform geometry sampling geometry sampling
=> Considered solution :=> Considered solution :Semi-regular remeshingSemi-regular remeshing
Irregular meshes (2)Irregular meshes (2)
I. Background
SummarySummary
BackgroundBackground
Triangular MeshesTriangular Meshes
RemeshingRemeshing
Multiresolution analysis Multiresolution analysis
CompressionCompression
Bit allocationBit allocation
I. Background
Semi-regular remeshingSemi-regular remeshing
Original meshCoarse meshSubdivised mesh (1)Finest semi-regular version
Simplification
Irregularmesh
Coarsemesh
Subdivision
Semi-regularmesh
I. Background
Semi-regular remesherSemi-regular remesher
MAPSMAPS ((A. Lee et al. A. Lee et al. , 1998, 1998) ) Coarse mesh (geometry+connectivity)Coarse mesh (geometry+connectivity) N sets of 3D details (geometry) => N sets of 3D details (geometry) => 3 floating numbers3 floating numbers
Normal MeshesNormal Meshes ((I. Guskov et al.I. Guskov et al., 2000, 2000)) Coarse mesh (geometry+connectivity)Coarse mesh (geometry+connectivity) N’ sets of 3D details (geometry) => N’ sets of 3D details (geometry) => 1 floating number1 floating number
I. Background
Normal MeshesNormal Meshes
Known direction: Known direction: normal at the surfacenormal at the surface
0v 1v
2v 1M
01,d
3v
4v02 ,d
12 ,d
2M
0M
Surface to remeshSurface to remesh
=>=> More compact representationMore compact representation
I. Background
SummarySummary
BackgroundBackground
Triangular MeshesTriangular Meshes
RemeshingRemeshing
Multiresolution analysis Multiresolution analysis
CompressionCompression
Bit allocationBit allocation
I. Background
Multiresolution analysisMultiresolution analysis
MultiresolutionMultiresolution RepresentationRepresentation:: Low frequencyLow frequency (LF) mesh (LF) mesh
(geometry + topology)(geometry + topology)
N sets of wavelet coefficients (N sets of wavelet coefficients (3D vectors3D vectors) ) (geometry)(geometry)
…
Details Details Details Details
I. Background
SummarySummary
BackgroundBackground
Triangular MeshesTriangular Meshes
RemeshingRemeshing
Multiresolution analysis Multiresolution analysis
CompressionCompression
Bit allocationBit allocation
I. Background
CompressionCompression
Objective : Objective : reduce the information quantity reduce the information quantity useful for useful for
representing numerical datarepresenting numerical data
2 approachs 2 approachs : : Lossy or lossless compressionLossy or lossless compression
High compression ratiiHigh compression ratii=> Lossy compression=> Lossy compression
I. Background
Compression schemeCompression scheme
Q Entropy Coding
Transform 1010…Remeshing
Bit Allocatio
n
Target bitrate
or distortion
Preprocessing
Semi-regular Wavelet coefficients
Optimize the Rate-Distortion (RD)
tradeoff
I. Background
SummarySummary
BackgroundBackground
Triangular MeshesTriangular Meshes
RemeshingRemeshing
Multiresolution analysis Multiresolution analysis
CompressionCompression
Bit allocationBit allocation
I. Background
Bit allocation: goalBit allocation: goal
Optimization of the tradeoff between bitstream size Optimization of the tradeoff between bitstream size and reconstruction quality:and reconstruction quality:
minimize D(R)minimize D(R)
oror minimize R(D)minimize R(D)
R
D
I. Background
Bit allocation and meshesBit allocation and meshes
Related Works (geometry compression):Related Works (geometry compression):
Zerotree Zerotree codingcoding PGCPGC : :
Progressive Geometry Compression Progressive Geometry Compression (A. (A. Khodakovsky et al.,Khodakovsky et al., 2OOO) 2OOO) NMCNMC : :
Normal Mesh Compression Normal Mesh Compression (( A. A. Khodakovsky et I. GuskovKhodakovsky et I. Guskov, , 2002). 2002).
=> => Stop coding when bitstream given size is reached.Stop coding when bitstream given size is reached.
Estimation-quantization (EQ) Estimation-quantization (EQ) codingcoding MSECMSEC : :
Geometry Compression of Normal Meshes Using Rate-Distortion Geometry Compression of Normal Meshes Using Rate-Distortion AlgorithmsAlgorithms (S. (S. Lavu et al.Lavu et al., 2003), 2003)
=> => Local Local RD optimization.RD optimization.
I. Background
Proposed bit allocationProposed bit allocation
1.1. Low computational complexityLow computational complexity
2.2. Improve the quantization processImprove the quantization process
3.3. Maximize the quality of the reconstructed meshMaximize the quality of the reconstructed meshaccording to a given target bitrateaccording to a given target bitrate
=> Which distortion criterion for evaluating the losses?
I. Background
SummarySummary
BackgroundBackground
Distortion criterion for multiresolution meshesDistortion criterion for multiresolution meshes
Optimization of the Rate-Distorsion trade-offOptimization of the Rate-Distorsion trade-off
Experimental resultsExperimental results
Conclusions and perpectivesConclusions and perpectives
Coding/DecodingCoding/Decoding
Q
Entropy Decoding
Transform
Bit Allocatio
n
1010…
Target bitrate or
distortion
Remeshing
Preprocessing
Q*Inverse
transform
Entropy coding
Quantized semi-regular
Semi-regular
II. Distortion criterion for multiresolution meshes
Considered distorsion criterionConsidered distorsion criterionMSE due to quantization of the semi-regular meshMSE due to quantization of the semi-regular mesh
1#
0
2
2ˆ
#
1 SR
jjjSRT vv
SRMSED
semi-regular verticessemi-regular vertices
quantized semi-regular verticesquantized semi-regular vertices
Number of verticesNumber of vertices
Wavelet => ? iSR MSEfunctionMSE
MSE for one subbandMSE for one subband
II. Distortion criterion for multiresolution meshes
Related worksRelated worksK.ParkK.Park and and R.HaddadR.Haddad (1995) (1995) M-channel schemeM-channel scheme quantization model : “noise plus gain”quantization model : “noise plus gain”
B.UsevitchB.Usevitch (1996) (1996) quantization model : “additive noise”quantization model : “additive noise” N decomposition levelsN decomposition levels Sampled on square gridsSampled on square grids
Filter bankFilter bank
Problem : - non adapted for lifting scheme ! - usable for any sampling grid ?
II. Distortion criterion for multiresolution meshes
Lifting scheme for meshesLifting scheme for meshes
3 3 prédiction operators Pprédiction operators P
=> wavelet coefficients=> wavelet coefficients
3 3 update operators U update operators U
=> LF mesh=> LF mesh
Triangular grid => Triangular grid => 4 channels4 channels
II. Distortion criterion for multiresolution meshes
Triangulaire samplingTriangulaire sampling
1 triangular grid => 4 cosets1 triangular grid => 4 cosets
0 0 0
00
0
LF subband (0)
n1
n2
HF subband 1
1 1
1
HF subband 22
2
2
HF subband 3
3
3
3
II. Distortion criterion for multiresolution meshes
4-channel lifting scheme: analysis4-channel lifting scheme: analysis
LF
HF 1
HF 2
HF 3
-P1
+-P2
+-P3
+
U1
+
U2
+
U3
+
split
Semi-regular mesh
II. Distortion criterion for multiresolution meshes
4-channel lifting scheme: synthesis4-channel lifting scheme: synthesis
P
+
P
+
P
+
Merge
LF
HF 1
HF 2
HF 3
Semi-regular mesh
-U
+
-U
+
-U
+
=> Derivation of the MSE of the quantized meshaccording to the quantization error of each 4 subband
II. Distortion criterion for multiresolution meshes
Input signal :Input signal :
Quantization error model : Quantization error model : « additive noise »« additive noise »
S is one realization of a stationar and ergodic S is one realization of a stationar and ergodic random process => random process => deterministic quantitydeterministic quantity
=> MSE of the input signal=> MSE of the input signal
Proposed MethodProposed Method
KRss kk /)(
0RSREQM SR #
1
II. Distortion criterion for multiresolution meshes
ss ˆ
Proposed MethodProposed Method: : HypothesisHypothesis
Uncorrelated error Uncorrelated error in each subbandin each subband
Subband errorsSubband errors mutually uncorrelated mutually uncorrelated
Synthesis filter energyQuantization error
energy
1
0#
1 M
igSR ii
RRSR
MSE 00
II. Distortion criterion for multiresolution meshes
Proposed MethodProposed Method: principle: principle
Synthesis filter energySynthesis filter energy
Polyphase components Polyphase components of the filtersof the filters Cauchy theoremCauchy theorem
Quantization error energyQuantization error energy
Uncorrelated error Uncorrelated error in each subbandin each subband
0ir
0igr
II. Distortion criterion for multiresolution meshes
Proposed MethodProposed Method:: solution solution
withwith
1
0
M
jiiSR MSEwMSE
1
0
2,#
# M
jji
ii
d
GSR
SRw
Zk
k
For 1 decomposition level
Polyphase component
MSE of the subband i
II. Distortion criterion for multiresolution meshes
Weights relative to the non-orthogonal filters
polyphase representationpolyphase representationLifting scheme: Lifting scheme:
=> Polyphase components => Polyphase components depend on only the depend on only the prediction and update prediction and update opératorsopérators
1121111
1222122
1121111
121
1
1
1
1
MMMMM
M
M
M
pupupuu
pupupuu
pupupuu
ppp
G
New formulation : => can be applied easily to lifting scheme
II. Distortion criterion for multiresolution meshes
1,12,11,10,1
1,22,21,20,2
1,12,11,10,1
1,02,01,00,0
MMMMM
M
M
M
GGGG
GGGG
GGGG
GGGG
G
Proposed MethodProposed Method : : solution solution
avecavec
1
0
2,
M
jjii
d
GwZk
k
For N decomposition levels
1
0
1
1,,0,10,1
N
i
M
lliliNN MSEWMSEWMSE
lili
li wwSR
SRW 0
,, #
#
etet
II. Distortion criterion for multiresolution meshes
OutlineOutline
This foThis formulation can be applied to lifting schemermulation can be applied to lifting scheme
Global formulation of the weights for any :Global formulation of the weights for any :
Grid and related subsamplingGrid and related subsampling
number of channels M number of channels M
Number of decomposition levels NNumber of decomposition levels N
II. Distortion criterion for multiresolution meshes
II. Distortion criterion for multiresolution meshes
Experimental ResultsExperimental Results
=> PSNR Gain : up to 3.5 dB=> PSNR Gain : up to 3.5 dB
II. Distortion criterion for multiresolution meshes
Visual impactVisual impact
OriginalWithout the weights With the weights
Coding/DecodingCoding/Decoding
Q
Entropy Decoding
Transform
Bit Allocation
1010…
Target bitrate or
distortion
Remeshing
Preprocessing
Q*Inverse
transform
Entropy coding
Quantized semi-regular
Semi-regular
II. Distortion criterion for multiresolution meshes
MSE and irregular meshMSE and irregular mesh
Quality of the reconstructed mesh :Quality of the reconstructed mesh :
Reference : Reference : irregular meshirregular mesh
Used metric:Used metric:
geometrical distance between two surfaces:geometrical distance between two surfaces:
the «surface-to-surfacethe «surface-to-surface distance distance ((s2ss2s) ») »
II. Distortion criterion for multiresolution meshes
=> Is the MSE suitable to control the quality?
Forward distance:Forward distance:
distance between one point and one surface:distance between one point and one surface:
Quality of the reconstructed meshQuality of the reconstructed mesh
Input mesh Input mesh (irregular)(irregular)
Quantized meshQuantized mesh
(semi-regular)(semi-regular)
12
21( ) ( )
p Md M M d p M dM
M
2M'2'')(Proj'min)',( ppppMpd
Mp
SRqIRdIRSRqdIRSRqss ,,,max,2
II. Distortion criterion for multiresolution meshes
Simplifying approximationsSimplifying approximations
Normal meshesNormal meshes: :
=> => infinitesimal remeshing errorinfinitesimal remeshing error
=>=> uniform and regularuniform and regular geometry sampling geometry sampling
Highly detailed meshes:Highly detailed meshes:
=> densely sampled geometry=> densely sampled geometry
1#
0
2,ˆ#
1,
SR
jj SRvd
SRIRSRqdss
Relation with the quantization error?
II. Distortion criterion for multiresolution meshes
Hypothesis: asymptotical caseHypothesis: asymptotical casePreservation of the LF subbandsPreservation of the LF subbands
=>=> normal orientations slightly modified normal orientations slightly modified=> errors lie in the normal direction (=> errors lie in the normal direction (normal normal
meshesmeshes) )
II. Distortion criterion for multiresolution meshes
n
0v
1v
2v
n’
0v̂
1v̂
2v̂θ
ε(v2
)
nn’
0v
0v̂1v
1v̂
2v̂
2v
θ
ε(v2
)
=>=> 2
ˆ,ˆ jjj vvSRvd
1#
0
2
2ˆ
#
1,
SR
jjjSS vv
SRIRSRqd
SREQM
=> MSE : suitable criterion to controlthe quality of the reconstructed mesh
Proposed heuristicProposed heuristic
1#
0
2,ˆ#
1,2
SR
jj SRvd
SRIRSRqssApproximating formulation:
2
ˆ,ˆ jjj vvSRvd
Asymptotical case+ normal meshes
II. Distortion criterion for multiresolution meshes
SummarySummary
BackgroundBackground
Distortion criterion for multiresolution meshesDistortion criterion for multiresolution meshes
Optimization of the Rate-Distorsion trade-offOptimization of the Rate-Distorsion trade-off
Experimental resultsExperimental results
Conclusions and perpectivesConclusions and perpectives
Optimization of the Rate-Distorsion Optimization of the Rate-Distorsion trade-offtrade-off
Objective : Objective :
find the quantization steps that find the quantization steps that maximize the quality of maximize the quality of the reconstructed meshthe reconstructed mesh
Scalar quantization Scalar quantization (less complex than VQ)(less complex than VQ)
3D Coefficients => 3D Coefficients => data structuring?data structuring?
targetconstraintwith
minimize
RR
MSE
T
SR
III.Optimization of the Rate-Distorsion trade-offIII.Optimization of the Rate-Distorsion trade-off
NormalNormal at the surface: at the surface: z-axis of the local framez-axis of the local frame
=> Coefficient : => Coefficient : Tangential Tangential components (components (x x andand y- y-coordinatescoordinates)) Normal Normal components (components (z-z-coordinatescoordinates))
Local framesLocal frames
zz
x
Global frame
xx
z
Local frame
z
x
III.Optimization of the Rate-Distorsion trade-offIII.Optimization of the Rate-Distorsion trade-off
Histogram of the polar angle
III.Optimization of the Rate-Distorsion trade-offIII.Optimization of the Rate-Distorsion trade-off
Local frame: Local frame:
0° 90° 180°
=> => Most of coefficients have only normal Most of coefficients have only normal componentscomponents
θ
xy
z
=> Components treated separately (2 scalar subbands)
MSE of one subband MSE of one subband ii
2,1,, iSRiSRJj
jii MSEMSEMSEMSEi
MSE relative to the tangential components
MSE relative to the normal components
III.Optimization of the Rate-Distorsion trade-offIII.Optimization of the Rate-Distorsion trade-off
How solving the problem?How solving the problem?
Find the Find the quantization steps and lambdaquantization steps and lambda that minimize the following lagrangian criterion:that minimize the following lagrangian criterion:
Method:Method:=> => first order conditionsfirst order conditions
cible,
0,,,,
0, RqRaqMSEWqJ ji
N
i Jjjiji
JjjijiSR
N
iiji
ii
Distortion Constraint relative to the bitrate
III.Optimization of the Rate-Distorsion trade-offIII.Optimization of the Rate-Distorsion trade-off
SolutionSolution Need to solveNeed to solve
(2N + 4) equations with (2N + 4) unknowns(2N + 4) equations with (2N + 4) unknowns
target0
,,,
,
,,
,,
RqRa
W
a
qR
qMSE
N
i Jjjijiji
i
ji
jiji
jijiSR
i
III.Optimization of the Rate-Distorsion trade-offIII.Optimization of the Rate-Distorsion trade-off
jiq ,
Tji Rq ,
PDF of the component sets:Generalized Gaussian Distribution (GGD)
=> model-based algorithm (C. Parisot, 2003)
Model-based algorithmModel-based algorithm
compute the variance and compute the variance and αα for each subband for each subband
compute the bitratescompute the bitratesfor each subbandfor each subbandλλ
Target bitrateTarget bitratereached?reached?new new λλ
compute the quantizationcompute the quantizationstep of each subbandstep of each subband
jiR ,
jiq ,
Complexity : 12 operations / semi-regularExample : 0.4 second (PIII 512 Mb Ram)
=> Fast process.
III.Optimization of the Rate-Distorsion trade-offIII.Optimization of the Rate-Distorsion trade-off
Look-up tables
SummarySummary
BackgroundBackground
Distortion criterion for multiresolution meshesDistortion criterion for multiresolution meshes
Optimization of the Rate-Distorsion trade-offOptimization of the Rate-Distorsion trade-off
Experimental resultsExperimental results
Conclusions and perpectivesConclusions and perpectives
Compression schemeCompression scheme
SQ 3D-CbAC*
UnliftedButterflyNormal
meshes
BitAllocatio
n
Target BitratePreprocessing
1010…Connectivity coding*
(coarse mesh connectivity)MPX
IV. Experimental resultsExperimental results
* Touma-Gotsman coder* Touma-Gotsman coder
* Context-based Bitplane Arithmetic * Context-based Bitplane Arithmetic Coder (EBCOT-like)Coder (EBCOT-like)
Visual resultsVisual results
Input mesh(irregular)
CR = 832.2 bits/iv
CR = 226 0.82 bits/iv
CR = 9000.2 bits/iv
Compression ratio:
sizebitstream
IRTrIRIRCR
#log3#323# 2
IV. Experimental resultsExperimental results
ComparisonComparison
Quality criterion : Quality criterion :
State-of-the-art methods:State-of-the-art methods: NMCNMC ( (Normal meshesNormal meshes + Butterfly NL + zerotree) + Butterfly NL + zerotree) EQMCEQMC ( (Normal meshesNormal meshes + Butterfly NL + EQ) + Butterfly NL + EQ) PGCPGC (MAPS + Loop) (MAPS + Loop)
s2s between the irregular input s2s between the irregular input mesh and the quantized semi-mesh and the quantized semi-regular oneregular one
Bounding box Bounding box diagonaldiagonal
SRqIRss
bbPSNR
,2log20 10
IV. Experimental resultsExperimental results
PSNR-bitrate curve: PSNR-bitrate curve: RabbitRabbit
IV. Experimental resultsExperimental results
PSNR-bitrate curve: PSNR-bitrate curve: FelineFeline
=> PSNR Gain: up to 7.5 dB=> PSNR Gain: up to 7.5 dB
IV. Experimental resultsExperimental results
PSNR-bitrate curve: PSNR-bitrate curve: HorseHorse
IV. Experimental resultsExperimental results
Geometrical comparisonGeometrical comparison
NMC NMC (62.86 dB)(62.86 dB)
Proposed algorithmProposed algorithm ( (65.35 dB65.35 dB))
Bitrate = 0.71 bits/iv
IV. Experimental resultsExperimental results
SummarySummary
BackgroundBackground
Distortion criterion for multiresolution meshesDistortion criterion for multiresolution meshes
Optimization of the Rate-Distorsion trade-offOptimization of the Rate-Distorsion trade-off
Experimental resultsExperimental results
Conclusions and perpectivesConclusions and perpectives
ConclusionsConclusions New shape compression method:New shape compression method:
Contributions :Contributions :1.1. Weighted MSE : suitable distortion criterionWeighted MSE : suitable distortion criterion
2.2. Original formulation of the weights Original formulation of the weights (suitable in case of lifting scheme)(suitable in case of lifting scheme)
3.3. Bit alllocation of low computational complexity that optimizes Bit alllocation of low computational complexity that optimizes the the quality of a quantized mesh.quality of a quantized mesh.
4.4. An original Context-based Bitplane Arithmetic CoderAn original Context-based Bitplane Arithmetic Coder
V. Conclusions and perspectives
=> Better results than=> Better results thanthe state-of-the-art methods. the state-of-the-art methods.
PerspectivesPerspectives
Take into account some Take into account some visual properties the visual properties the human eye appreciates human eye appreciates (local curvature, volume, (local curvature, volume, smoothness…) smoothness…)
Reference : Z.Reference : Z.Karni and C.GotsmanKarni and C.Gotsman, 2000, 2000
Algorithm for huge meshes: « on the flow » Algorithm for huge meshes: « on the flow » compressioncompression
Reference : Reference : A. Elkefi et al.A. Elkefi et al., 2004, 2004
V. Conclusions and perspectives
EndEnd