Quad-Tree Motion Modeling with Leaf Merging
description
Transcript of Quad-Tree Motion Modeling with Leaf Merging
QUAD-TREE MOTION MODELING WITH LEAF MERGING
Reji Mathew and David S. Taubman
CSVT 2010
Outline Introduction
Quad-tree representation Quad-tree motion modeling
Motion vector prediction strategies Pruning algorithm Merging principle Motion signaling R-D performance results
Hierarchical and polynomial motion modeling Scalable motion modeling Conclusion
Quad-tree Representation Image modeling
Image to be recursively divided into smaller regions, each region represented by a suitable model.
Sub-optimal: dependency between neighboring leaf nodes with different parents is not exploited
Quad-tree Representation Image modeling
Rate-distortion optimization, allowing a Lagrangian cost function(D+λR) to be minimized using tree pruning with leaf merging step.
[1] R. Shukla, P. Dragotti, M. Do, and M. Vetterli, “Rate-distortion optimized tree structure compression algorithms for piecewise polynomial images,” IEEE Trans. Image Process., vol. 14, no. 3, pp. 343–359, Mar. 2005.
Quad-tree Motion Modeling
Motion model forward-only, backward only or bi-directional
motion with two reference frames. Motion vector prediction strategies
Hierarchical motion coding H.264 spatial motion vector prediction
strategy
Motion models
Quad-tree Motion Modeling
Pruning Algorithm Produce a quad-tree structure that minimizes
the Lagrangian cost objective Df + λRf Given a parent node p, the four children ci , 1 ≤ i ≤ 4,
are pruned away if
When pruning occurs, andOtherwise, and
=Rp in hierarchical coding=0 at all times in spatial coding
R-D optimally pruned quad-tree:Tree pruning yields a globally minimal value for Df + λRf for hierarchical coding; while it is somewhat greedy for spatially predictive coding.
Quad-tree Motion Modeling
Merging principle possibility of jointly coding and optimizing
neighboring nodes that belong to different parents. Merge target contains nieghboring node located
at a higher level or at the same level. Merging is allowed to take place only if it
reduces the overall Lagrangian cost.
The same parent
Quad-tree Motion Modeling
Motion signaling Anchor node:
Hierarchical: the only member node of the region that is not signaled as being merged
Spatial: the first node in the region that is encountered during decoding.(the top-left block)
Quad-tree Motion Modeling
R-D performance results35% 25%
45%35%
once merging is included the performance of hierarchical motion representation can be brought close to that achieved by spatial prediction with merging.
Hierarchical and Polynomial Motion Modeling
Further improve the performance of hierarchical motion representation by polynomial motion models. Formation of larger regions during merging process Smoother motion representations
Motion models
The parameters of the motion model are obtained by a weighted least squares fitting procedure.
Pruning phase Merging phase
: mv belonging to node b at level k: motion corresponding to translation, linear and affine flows
Hierarchical and Polynomial Motion Modeling
Motion compensation Generate a set of MVs for each descendants at
level K (4*4 block)
R-D performancewith motion models
depend on the motion model and the central location of block b’
Scalability Motion Modeling
Scalability objective Modified Lagrangian cost function
When terminating decoding at an intermediate resolution level, motion compensation is performed using leaf nodes that may already be available; in those cases where leaf nodes are not available, information contained in branch nodes is utilized.
: The costs for each level k of the quad-tree: The weights assigned to each level,
and
Leaf node b Branch node bContribution to Contribution to
: The total distortion of all nodes for which motion compensation is performedLevel k :
terminate
Scalability Motion Modeling
Scalability performance α0 = α1= α2=0.1, α3=0.7
Scalability Motion Modeling
Residual coding JPEG2000: full resolution motion compensated
residual frames Total rate for coding motion and residual
frames
Scalability Motion Modeling
Wavelet-based video encoding results integrate the quad-tree motion model with the
wavelet-based scalable interactive video (SIV) codec[9]
[9] A. Secker and D. S. Taubman, “Lifting-based invertible motion adaptive transform framework for highly scalable video compression,” IEEE Trans. Image Process., vol. 12, no. 12, pp. 1530–1542, Dec. 2003.
Conclusion The merging step can be incorporated into
quad-tree motion representations for a range of motion modeling contexts.
R-D performance that can be gained by introducing merging for the two cases of hierarchical and spatially predictive motion coding (such as that employed by H.264).
Report on the benefits of polynomial modeling and hierarchical coding, once merging has been incorporated into the conventional quad-tree approach.