最新資訊 - Belt and Road Global Forum...一帶一路國際聯盟 新資訊(2018年3月) 2 歡迎辭 歡迎閱覽首期一帶一路國 際聯盟通訊。一帶一路國 際聯盟通訊旨在分享一帶
Feng Liu, JinjunWang,ShenghuoZhu (MM’08) University of Wisconsin-Madison, NEC Laboratories...
-
Upload
beverly-randall -
Category
Documents
-
view
258 -
download
2
Transcript of Feng Liu, JinjunWang,ShenghuoZhu (MM’08) University of Wisconsin-Madison, NEC Laboratories...
Noisy Video Super-Resolution
Feng Liu, JinjunWang,ShenghuoZhu (MM’08)University of Wisconsin-Madison, NEC Laboratories America, Inc.
第一組: 資訊四 B95902105 黃彥達 資訊碩一 R98922046 蔡旻光 網媒碩二 R97944012 鄒志鴻
2
Outline
Introduction Goal File Format
Noise Reduced Image Proposed Approach Motion Estimation & Estimated Super-
Resolution Result Implementation Result Conclusion
3
Introduction
Low-quality videos often not only have limited resolution but also suffer from noise In fact, the requirements of de-noising & super-
resolution is quite similar
This paper present a unified framework which achieves simultaneous video de-noising and super-resolution algorithm by some measurements of visual quality
Goal
Refine low-quality videos from YouTube, and make the video better effects, which has better quality by human eyes.
Input is low-quality and noise-included (block effects or somewhat noise) videos
5
File Format
.3gp file Frame rate: 15(our video) or 25 Frame size: 176(w) * 144(h) MPEG-4 Part 12 It is used on 3G mobile phones
also can be played on2G and 4G phones.
Our video: 867 KB/ 98 sec
Gaussian Time
Fram
e t
- 2Fr
ame
t
- 1Fr
ame
t
Space Gaussian
Time G
au
ssian
Pixel(I,j)
Fram
e
t+1Fram
e
t+2Fram
e t
Shot Detecti
on
11
Proposed Approach – 1 / 4 Consider the visual quality with respect to
the following 3 aspects: Fidelity Preserving▪ To achieve similar high-resolution result
Detail Preserving▪ Enhanced details (edge)
Spatial-Temporal Smoothness▪ Remove undesirable high-frequency contents (e.g. jitter)
12
Proposed Approach – 2 / 4 Fidelity Preserving
Conventional metrics:▪ Measure fidelity by the difference between Ih & Il would
be problematic & waste useful time-space information in video
Proposed metrics:▪ Estimate an approximation of super-resolution results
from space-time neighboring pixels▪ The fidelity measurement:
see next page for details
noised
13
Proposed Approach – 3 / 4 Detail Preserving
Enhanced details (edge)
Contrast preserving▪ Human visual system is more sensitive to contrast
than pixel values▪ Gradient fields of Ih & should be close
,where Wk is one or zero if the patchk with/o edges (canny detector)
14
Proposed Approach – 4 / 4 (Spatial-Temporal) Smoothness
Smooth results are often favored by the human system Encourage to minimize:
A 2-D Laplace filter may be
Spatial-temporal Laplacian
OR
15
An Optimization Problem
Proposed Measurements
A quadratic minimization problem to solve (AX = b):
Contrast
Similarity
Detail Information(edge)
Spatial-Temporal Smoothness
16
Implementation – 1 / 3
Inputlow hI~
hI~
hG~
hI~
)1,()1,(~~
tIwtIw hmv
hmv
= X
I
6 -1 … -1-1 6 -1 … -1 -1 6 -1 … -1
Laplacian
Gradient
-1 0 1 … 1 -1 0 1 … 1 -1 0 1 … 1
EdgeMinimize
Motion Estimation
+
Result (X)
Fidelity
Gaussian filter
mvw
fidv
fidg
dt
sm
17
Implementation – 2 / 3
....,5001111 slowtoosbxA nnnn
hI~
hG~
hI~
)1,()1,(~~
tIwtIw hmv
hmv
= X
I
6 -1 … -1-1 6 -1 … -1 -1 6 -1 … -1
Laplacian
Gradient
-1 0 1 … 1 -1 0 1 … 1 -1 0 1 … 1
EdgeMinimize
Fidelityfidv
fidg
dt
sm
!!!2'1 sbAxAxAA tnnn
t
by sparse least square solver
18
Implementation – 3 / 3
Adjustments for the weight terms The measurement term is more emphasized if
the weight is larger
By iteratively experiments for our test data, we took
However, we found that the best weight set may be different for different videos
3.0,1.0,1,1 smdtfidgfidv
22
Conclusion
The proposed framework formulates noisy video super-resolution as an optimization problem, aiming to maximize the visual quality
By exploiting the space-temporal information, we can estimate a better baseline than conventional fidelity measurement
The properties include fidelity-preserving, detail-preserving and smoothness are considered to achieve the best visual quality results