Cs: compressed sensing Jialin peng. Introduction Exact/Stable Recovery Conditions – -norm based...
-
Upload
charlene-hines -
Category
Documents
-
view
223 -
download
1
Transcript of Cs: compressed sensing Jialin peng. Introduction Exact/Stable Recovery Conditions – -norm based...
![Page 1: Cs: compressed sensing Jialin peng. Introduction Exact/Stable Recovery Conditions – -norm based recovery –OMP based recovery Some related recovery algorithms.](https://reader035.fdocuments.net/reader035/viewer/2022062423/56649e885503460f94b8c5bb/html5/thumbnails/1.jpg)
Cs: Cs: compressecompressed sensingd sensing
Jialin pengJialin peng
![Page 2: Cs: compressed sensing Jialin peng. Introduction Exact/Stable Recovery Conditions – -norm based recovery –OMP based recovery Some related recovery algorithms.](https://reader035.fdocuments.net/reader035/viewer/2022062423/56649e885503460f94b8c5bb/html5/thumbnails/2.jpg)
• Introduction• Exact/Stable Recovery Conditions
– -norm based recovery– OMP based recovery
Some related recovery algorithmsSparse RepresentationApplications
p
OutlineOutline
![Page 3: Cs: compressed sensing Jialin peng. Introduction Exact/Stable Recovery Conditions – -norm based recovery –OMP based recovery Some related recovery algorithms.](https://reader035.fdocuments.net/reader035/viewer/2022062423/56649e885503460f94b8c5bb/html5/thumbnails/3.jpg)
Introduction
high-density sensorhigh speed sampling……A large amount of
sampled data will be discarded
A certain minimum number of samples is required in order to perfectly c
apture an arbitrary bandlimited signal
Data Storage
Receiving & Storage
![Page 4: Cs: compressed sensing Jialin peng. Introduction Exact/Stable Recovery Conditions – -norm based recovery –OMP based recovery Some related recovery algorithms.](https://reader035.fdocuments.net/reader035/viewer/2022062423/56649e885503460f94b8c5bb/html5/thumbnails/4.jpg)
Sparse PropertySparse Property• Important classes of signals have naturally spars
e representations with respect to fixed bases (i.e., Fourier, Wavelet), or concatenations of such bases.
• Audio, images …• Although the images (or their features) are natur
ally very high dimensional, in many applications images belonging to the same class exhibit degenerate structure.
• Low dimensional subspaces, submanifolds• representative samples—sparse representation
![Page 5: Cs: compressed sensing Jialin peng. Introduction Exact/Stable Recovery Conditions – -norm based recovery –OMP based recovery Some related recovery algorithms.](https://reader035.fdocuments.net/reader035/viewer/2022062423/56649e885503460f94b8c5bb/html5/thumbnails/5.jpg)
Transform coding: JPEG, JPEG2000, MPEG, and MP3
![Page 6: Cs: compressed sensing Jialin peng. Introduction Exact/Stable Recovery Conditions – -norm based recovery –OMP based recovery Some related recovery algorithms.](https://reader035.fdocuments.net/reader035/viewer/2022062423/56649e885503460f94b8c5bb/html5/thumbnails/6.jpg)
The GoalThe GoalDevelop an end-to-end system • Sampling• processing • reconstruction• All operations are performed at a low rate:
below the Nyquist-rate of the input (too costly, or even physically impossible)
• Relying on structure in the input
![Page 7: Cs: compressed sensing Jialin peng. Introduction Exact/Stable Recovery Conditions – -norm based recovery –OMP based recovery Some related recovery algorithms.](https://reader035.fdocuments.net/reader035/viewer/2022062423/56649e885503460f94b8c5bb/html5/thumbnails/7.jpg)
Sparse: the simplest choice is the best oneSparse: the simplest choice is the best one
• Signals can often be well approximated as a linear combination of just a few elements from a known basis or dictionary.
• When this representation is exact ,we say that the signal is sparse.
Remark:
In many cases these high-dimensional signals contain relatively little information com
pared to their ambient dimension.
![Page 8: Cs: compressed sensing Jialin peng. Introduction Exact/Stable Recovery Conditions – -norm based recovery –OMP based recovery Some related recovery algorithms.](https://reader035.fdocuments.net/reader035/viewer/2022062423/56649e885503460f94b8c5bb/html5/thumbnails/8.jpg)
Introduction
high-density sensorhigh speed sampling……A large amount of
sampled data will be discarded
A certain minimum number of samples is required in order to perfectly c
apture an arbitrary bandlimited signal
Data Storage
Receiving & Storage
![Page 9: Cs: compressed sensing Jialin peng. Introduction Exact/Stable Recovery Conditions – -norm based recovery –OMP based recovery Some related recovery algorithms.](https://reader035.fdocuments.net/reader035/viewer/2022062423/56649e885503460f94b8c5bb/html5/thumbnails/9.jpg)
IntroductionSparse priors of sig
nalNonuniform sampli
ngImaging algorithm:
optimization
Alleviated sensorReduced data……
modified sensor
Data Storage
Receiving & Storageoptimization
![Page 10: Cs: compressed sensing Jialin peng. Introduction Exact/Stable Recovery Conditions – -norm based recovery –OMP based recovery Some related recovery algorithms.](https://reader035.fdocuments.net/reader035/viewer/2022062423/56649e885503460f94b8c5bb/html5/thumbnails/10.jpg)
Introduction
x×1N ×1N
M N
Φ y• =
×1N
×1MSensing Matrix
×N N
×M N
![Page 11: Cs: compressed sensing Jialin peng. Introduction Exact/Stable Recovery Conditions – -norm based recovery –OMP based recovery Some related recovery algorithms.](https://reader035.fdocuments.net/reader035/viewer/2022062423/56649e885503460f94b8c5bb/html5/thumbnails/11.jpg)
compressioncompressionFind the most concise representation:
Compressed sensing: sparse or compressible representation • A finite-dimensional signal having a sparse or compressible repr
esentation can be recovered from a small set of linear, nonadaptive measurements
• how should we design the sensing matrix A to ensure that it preserves the information in the signal x?.
• how can we recover the original signal x from measurements y?• Nonlinear:1. Unknown nonzero locations results in a nonlinear model:the choice of which dictionary elements are used can change from signal to
signal . 2. Nonlinear recovering algorithmsthe signal is well-approximated by a signal with only k nonzerocoefficients
![Page 12: Cs: compressed sensing Jialin peng. Introduction Exact/Stable Recovery Conditions – -norm based recovery –OMP based recovery Some related recovery algorithms.](https://reader035.fdocuments.net/reader035/viewer/2022062423/56649e885503460f94b8c5bb/html5/thumbnails/12.jpg)
IntroductionLet be a matrix of size with .For a –sparse signal , let b
e the measurement vector.Our goal is to exact/stable recovery the unknow
n signal from measurement.The problem is under-determined.Thanks for the sparsity, we can reconstruct the s
ignal via .
Φ M N M NK Nx M y Φx
How can we recovery the unknown signal:
Exact/Stable Recovery Condition
0min , s.t. x y Φx
![Page 13: Cs: compressed sensing Jialin peng. Introduction Exact/Stable Recovery Conditions – -norm based recovery –OMP based recovery Some related recovery algorithms.](https://reader035.fdocuments.net/reader035/viewer/2022062423/56649e885503460f94b8c5bb/html5/thumbnails/13.jpg)
Exact/stable recovery conditions• The spark of a given matrix A• Null space property (NSP) of order k• The restricted isometry propertyRemark: verifying that a general matrix A satisfi
es any of these properties has a combinatorial computational complexity
![Page 14: Cs: compressed sensing Jialin peng. Introduction Exact/Stable Recovery Conditions – -norm based recovery –OMP based recovery Some related recovery algorithms.](https://reader035.fdocuments.net/reader035/viewer/2022062423/56649e885503460f94b8c5bb/html5/thumbnails/14.jpg)
Exact/stable recovery conditions
The restricted isometry constant (RIC) is defined as the smallest constant which satisfy:
The restricted orthogonality condition (ROC)is the smallest number such that:
2 2 2
2 2 21 1K K x Φx x
K
,K K ,K K
, 2 2, K K Φu Φv u v
Restricted Isometry Property
![Page 15: Cs: compressed sensing Jialin peng. Introduction Exact/Stable Recovery Conditions – -norm based recovery –OMP based recovery Some related recovery algorithms.](https://reader035.fdocuments.net/reader035/viewer/2022062423/56649e885503460f94b8c5bb/html5/thumbnails/15.jpg)
Exact/stable recovery conditions
Solving minimization is NP-hard, we usually relax it to the or minimization.
01 , 0 1p p
![Page 16: Cs: compressed sensing Jialin peng. Introduction Exact/Stable Recovery Conditions – -norm based recovery –OMP based recovery Some related recovery algorithms.](https://reader035.fdocuments.net/reader035/viewer/2022062423/56649e885503460f94b8c5bb/html5/thumbnails/16.jpg)
Exact/stable recovery conditionsFor the inaccurate measurement ,
the stable reconstruction model is
1
1 2min , s. t. x y Φx
y Φx e
![Page 17: Cs: compressed sensing Jialin peng. Introduction Exact/Stable Recovery Conditions – -norm based recovery –OMP based recovery Some related recovery algorithms.](https://reader035.fdocuments.net/reader035/viewer/2022062423/56649e885503460f94b8c5bb/html5/thumbnails/17.jpg)
Exact/stable recovery conditionsSome other Exact/Stable Recovery
Conditions:
![Page 18: Cs: compressed sensing Jialin peng. Introduction Exact/Stable Recovery Conditions – -norm based recovery –OMP based recovery Some related recovery algorithms.](https://reader035.fdocuments.net/reader035/viewer/2022062423/56649e885503460f94b8c5bb/html5/thumbnails/18.jpg)
Exact/stable recovery conditionsBraniuk et al. have proved that for some random
matrices, such as Gaussian, Bernoulli, ……
we can exactly/stably reconstruct unknown signal with overwhelming high probability.
![Page 19: Cs: compressed sensing Jialin peng. Introduction Exact/Stable Recovery Conditions – -norm based recovery –OMP based recovery Some related recovery algorithms.](https://reader035.fdocuments.net/reader035/viewer/2022062423/56649e885503460f94b8c5bb/html5/thumbnails/19.jpg)
Exact/stable recovery conditions
cf: minimization1
![Page 20: Cs: compressed sensing Jialin peng. Introduction Exact/Stable Recovery Conditions – -norm based recovery –OMP based recovery Some related recovery algorithms.](https://reader035.fdocuments.net/reader035/viewer/2022062423/56649e885503460f94b8c5bb/html5/thumbnails/20.jpg)
Exact/stable recovery conditionsSome evidences have indicated that
with , can exactly/stably recovery signal with fewer measurements.
min , s. t.p
x y Φx
0 1p
![Page 21: Cs: compressed sensing Jialin peng. Introduction Exact/Stable Recovery Conditions – -norm based recovery –OMP based recovery Some related recovery algorithms.](https://reader035.fdocuments.net/reader035/viewer/2022062423/56649e885503460f94b8c5bb/html5/thumbnails/21.jpg)
Quicklook InterpretationQuicklook Interpretation• Dimensionality-reducing projection.• Approximately isometric embeddings, i.e., pairwise
Euclidean distances are nearly preserved in the reduced space
RIP
![Page 22: Cs: compressed sensing Jialin peng. Introduction Exact/Stable Recovery Conditions – -norm based recovery –OMP based recovery Some related recovery algorithms.](https://reader035.fdocuments.net/reader035/viewer/2022062423/56649e885503460f94b8c5bb/html5/thumbnails/22.jpg)
Quicklook Interpretation Quicklook Interpretation
![Page 23: Cs: compressed sensing Jialin peng. Introduction Exact/Stable Recovery Conditions – -norm based recovery –OMP based recovery Some related recovery algorithms.](https://reader035.fdocuments.net/reader035/viewer/2022062423/56649e885503460f94b8c5bb/html5/thumbnails/23.jpg)
Quicklook InterpretationQuicklook Interpretation
•the ℓ2 norm penalizes large coefficients heavily, therefore solutions tend to have many smaller coefficients.•In the ℓ1 norm, many small coefficients tend to carry alarger penalty than a few large coefficients.
![Page 24: Cs: compressed sensing Jialin peng. Introduction Exact/Stable Recovery Conditions – -norm based recovery –OMP based recovery Some related recovery algorithms.](https://reader035.fdocuments.net/reader035/viewer/2022062423/56649e885503460f94b8c5bb/html5/thumbnails/24.jpg)
AlgorithmsAlgorithms• L1 minimization algorithms iterative soft thresholding iteratively reweighted least squares …• Greedy algorithms Orthogonal Matching Pursuit iterative thresholding• Combinatorial algorithms
![Page 25: Cs: compressed sensing Jialin peng. Introduction Exact/Stable Recovery Conditions – -norm based recovery –OMP based recovery Some related recovery algorithms.](https://reader035.fdocuments.net/reader035/viewer/2022062423/56649e885503460f94b8c5bb/html5/thumbnails/25.jpg)
CS builds upon the fundamental fact thatCS builds upon the fundamental fact that
• we can represent many signals using only a few non-zero coefficients in a suitable basis or
dictionary.
• Nonlinear optimization can then enable recovery
of such signals from very few measurements.
![Page 26: Cs: compressed sensing Jialin peng. Introduction Exact/Stable Recovery Conditions – -norm based recovery –OMP based recovery Some related recovery algorithms.](https://reader035.fdocuments.net/reader035/viewer/2022062423/56649e885503460f94b8c5bb/html5/thumbnails/26.jpg)
• Sparse property• The basis for representing the data• incoherent->task-specific (often overco
mplete) dictionary or redundant one
![Page 27: Cs: compressed sensing Jialin peng. Introduction Exact/Stable Recovery Conditions – -norm based recovery –OMP based recovery Some related recovery algorithms.](https://reader035.fdocuments.net/reader035/viewer/2022062423/56649e885503460f94b8c5bb/html5/thumbnails/27.jpg)
MRI ReconstructionMRI Reconstruction
MR images are usually sparse in certain transform domains, such as finite difference and wavelet.
![Page 28: Cs: compressed sensing Jialin peng. Introduction Exact/Stable Recovery Conditions – -norm based recovery –OMP based recovery Some related recovery algorithms.](https://reader035.fdocuments.net/reader035/viewer/2022062423/56649e885503460f94b8c5bb/html5/thumbnails/28.jpg)
Sparse RepresentationConsider a family of images, representing natural and typical image content:•Such images are very diverse vectors in•They occupy the entire space?•Spatially smooth images occur much more often than highly non-smooth and disorganized images •L1-norm measure leads to an enforcement of sparsity of the signal/image derivatives.•Sparse representation
![Page 29: Cs: compressed sensing Jialin peng. Introduction Exact/Stable Recovery Conditions – -norm based recovery –OMP based recovery Some related recovery algorithms.](https://reader035.fdocuments.net/reader035/viewer/2022062423/56649e885503460f94b8c5bb/html5/thumbnails/29.jpg)
Matrix completion algorithmsRecovering a unknown (approximate) low-
rank matrix from a sampling set of its entries.
min rank : , ,ij ijX
X X M i j NP-hard
Convex relaxation
*min : , ,ij ijX
X X M i j
*min , ,ij ij FX
X X M i j Unconstraint