Yu Liu and King Ngi Ngan Department of Electronic Engineering, The Chinese University of Hong Kong
description
Transcript of Yu Liu and King Ngi Ngan Department of Electronic Engineering, The Chinese University of Hong Kong
11
Anisotropic Double Cross Search Algorithm Anisotropic Double Cross Search Algorithm using Multiresolution-Spatio-Temporal Context using Multiresolution-Spatio-Temporal Context
for Fast Lossy In-Band Motion Estimationfor Fast Lossy In-Band Motion Estimation
Yu Liu and King Ngi NganYu Liu and King Ngi Ngan
Department of Electronic Engineering,Department of Electronic Engineering,The Chinese University of Hong KongThe Chinese University of Hong Kong
PCS2006, April 24-26, Beijing, ChinaPCS2006, April 24-26, Beijing, China
22
OutlineOutline
IntroductionIntroduction BackgroundBackground Proposed AlgorithmProposed Algorithm Experimental ResultsExperimental Results ConclusionConclusion
33
IntroductionIntroduction
• Motion Estimation in Critically-Sampled Wavelet Domain• Pro: basically free form the blocking effects• Con: inefficient in high bands
• Motion Estimation in Shift-Invariant Wavelet Domain• Pro: perform ME more precisely and efficiently• Con: computational complexity
e.g. low-band-shift (LBS) and complete-to-overcomplete DWT (CODWT)
44
BackgroundBackground
Motion Estimation in Shift-Invariant Wavelet Domain (1)Motion Estimation in Shift-Invariant Wavelet Domain (1)
• Two Level Shift-Invariant Wavelet Decomposition by using Low-Band-Shift (LBS) or Complete-to-Overcomplete DWT (CODWT)
55
BackgroundBackground
Motion Estimation in Shift-Invariant Wavelet Domain (2)Motion Estimation in Shift-Invariant Wavelet Domain (2)
• Generation of Wavelet BlocksGeneration of Wavelet Blocks
jifjiS pkltpklt ,),( ,,,,,,
• The coefficient of the pth wavelet block of current frame t can be represented by
• The v-pixel-shifted or {dx,dy}-pixel-shifted coefficient of the pth wavelet block of reference frame t’ can be represented by
llllpkltvpklt dyjdxidydxfjiS 2/,2/,2%,2%),( ,,,',,,'
66
BackgroundBackground
Motion Estimation in Shift-Invariant Wavelet Domain (3)Motion Estimation in Shift-Invariant Wavelet Domain (3)
• The sum of absolute difference (SAD) of the pth wavelet block for the motion vector v is computed as follows:
• The optimum motion vector v ∗ of the pth wavelet block, which has minimum displacement error, is given by:
77
BackgroundBackground
Anisotropic Motion Model in Wavelet DomainAnisotropic Motion Model in Wavelet Domain
• Traditional 2D ME in spatial domain
• suffers from the aperture problem
• 2D ME in wavelet domain
• the aperture problem can in fact be exploited as an advantage.
Aperture 1 Aperture 2
Aperture 3
Normal flow
(a)
Normal flow
LL LH
HL HH
(b)
(a) Aperture problem in spatial domain, (b) Anisotropic motion model in wavelet domain
88
Proposed AlgorithmProposed Algorithm
Multiresolution-Spatio-Temporal Context (1)Multiresolution-Spatio-Temporal Context (1)
(a) multiresolution context, (b) spatial context, (c) temporal context
• Traditional MRME algorithmsTraditional MRME algorithms• Multiresolution contextMultiresolution context
• Not enough for reducing the risk of getting trapped into a local minimum.Not enough for reducing the risk of getting trapped into a local minimum.
• The proposed algorithmThe proposed algorithm• Multiresolution-spatio-temporal ContextMultiresolution-spatio-temporal Context
• Consists of one multiresolution context, four spatial contexts, and five temConsists of one multiresolution context, four spatial contexts, and five temporal contexts.poral contexts.
c1
c2
X
c3 c4 c7
c5c6 c8
c9
c0
(a) (b) (c)
99
Proposed AlgorithmProposed Algorithm
Multiresolution-Spatio-Temporal Context (2)Multiresolution-Spatio-Temporal Context (2)
• For LL subbandFor LL subband• Initialization: Initialization: spatio-temporal context, plus the spatio-temporal context, plus the
candidate points in shifted LL subband, candidate points in shifted LL subband, where the median predictor is located where the median predictor is located
• Refinement:Refinement: diamond search algorithmdiamond search algorithm
• For other levelsFor other levels• Initialization:Initialization: multiresolution-spatio-temporal context multiresolution-spatio-temporal context
• Refinement: Refinement: anisotropic double cross search algorithmanisotropic double cross search algorithm
1010
Proposed AlgorithmProposed Algorithm
Anisotropic Double Cross Search Algorithm (1)Anisotropic Double Cross Search Algorithm (1)
• Anisotropic motion model suggests that the 2D ME problem in wavelet domain can be approximated by 1D ME along the normal flow direction for the vertical/horizontal subbands.
• During the 1D window searching, only the coefficients in the corresponding subbands and LL subband are computed.
1111
Proposed AlgorithmProposed Algorithm
Anisotropic Double Cross Search Algorithm (2)Anisotropic Double Cross Search Algorithm (2)
(a) (b)
LHyd
LHyd
HLxd
HLxd
LHyd LHyd
HLxd
HLxd
(c)
HLxd
HLxd
LHyd LHyd
Starting point or the center of the searchChecking points for HL subband in the first cross search routeChecking points for LH subband in the first cross search routeChecking points for LH subband in the second cross search routeChecking points for HL subband in the second cross search route
Best matching point obtained from the second cross search routeBest matching point obtained from the first cross search route
1212
Experimental Results (1)Experimental Results (1)
• PSNR
• MAD
• operation number
• speed-up ratio
Simulation results are reported in the following ways:
For performance comparison
• Full Search Algorithm (FSA)
• FMRME [6]
• FIBME [7]
• proposed MR-STC-ADCS
1313
Experimental Results (2)Experimental Results (2)
• Comparison of the Tested Algorithms for QCIF Video Sequences
1414
Experimental Results (3)Experimental Results (3)
• Comparison of the Tested Algorithms for CIF Video Sequences
1515
Experimental Results (4)Experimental Results (4)
• Comparison of the Tested Algorithms for 4CIF Video Sequences
On average, for all sequences examined in the experimental tests:
MR-STC-ADCS is roughly 11.5 and 2.6 times faster whereas its PSNR is approximately 1.46 dB and 0.6 dB higher than FMRME and FIBME; and its MAD is approximately 0.426 and 0.165 lower than FMRME and FIBME.
MR-STC-ADCS is about 271 times faster than FSA for QCIF, 667 times for CIF, and 1313 times for 4CIF, while having an average PSNR loss of only 0.04 dB or an average MAD increase of only 0.018 compared to the FSA.
1616
ConclusionConclusion
Fast Lossy In-Band Motion Estimation Algorithm• Anisotropic property of the motion field in shif
t-invariant wavelet domain• Multiresolution-spatio-temporal Context• Anisotropic Double Cross Search