Compressed Sensing
description
Transcript of Compressed Sensing
![Page 1: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/1.jpg)
+
Compressed Sensing
![Page 2: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/2.jpg)
+
Compressed Sensing
Mobashir Mohammad Aditya Kulkarni Tobias Bertelsen Malay Singh Hirak Sarkar Nirandika Wanigasekara Yamilet Serrano Llerena Parvathy Sudhir
![Page 3: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/3.jpg)
3
+ Introduction
Mobashir Mohammad
![Page 4: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/4.jpg)
4+The Data Deluge
Sensors: Better… Stronger… Faster… Challenge:
Exponentially increasing amounts of data Audio, Image, Video, Weather, … Global scale acquisition
![Page 5: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/5.jpg)
5+
![Page 6: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/6.jpg)
6+Sensing by Sampling
Sample
N
![Page 7: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/7.jpg)
7+Sensing by Sampling (2)
Sample
N CompressN >> L
JPEG…
L
L DecompressN >> L
N
![Page 8: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/8.jpg)
8+Compression: Toy Example
![Page 9: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/9.jpg)
9+Discrete Cosine Transformation
Transformation
![Page 10: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/10.jpg)
10+Motivation
Why go to so much effort to acquire all the data when most of the what we get will be thrown away?
Cant we just directly measure the part that wont end up being thrown away?
Donoho2004
![Page 11: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/11.jpg)
11
+
Outline
• Compressed Sensing• Constructing Φ• Sparse Signal Recovery• Convex Optimization
Algorithm• Applications• Summary • Future Work
![Page 12: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/12.jpg)
12
+ Compressed Sensing
Aditya Kulkarni
![Page 13: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/13.jpg)
13+What is compressed sensing?
A paradigm shift that allows for the saving of time and space during the process of signal acquisition, while still allowing near perfect signal recovery when the signal is needed
Nyquist rateSampling
AnalogAudioSignal
Compression(e.g. MP3)
High-rate Low-rate
CompressedSensing
![Page 14: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/14.jpg)
14+Sparsity The concept that most signals in our natural world are
sparse
a. Original imagec. Image reconstructed by discarding the zero coefficients
![Page 15: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/15.jpg)
15+How It Works
![Page 16: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/16.jpg)
16+
𝒚=𝚽 𝒙
Dimensionality Reduction Problem
I. Measure II. Construct sensing
matrix III. Reconstruct
![Page 17: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/17.jpg)
17+Sampling
¿
𝑁×𝑁
measurementssparse signal
nonzeroentries
𝑦 𝑥Φ=I
![Page 18: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/18.jpg)
18+
¿
𝑀×𝑁
measurementssparse signal
nonzeroentries
𝑦 𝑥Φ
𝐾 <𝑀≪𝑁
![Page 19: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/19.jpg)
19+
¿
𝑁×𝑁
𝑁×1
nonzeroentries
𝑥 𝛼Ψ
nonzeroentries
𝑁×1
![Page 20: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/20.jpg)
20+Sparsity The concept that most signals in our natural world are
sparse
a. Original imagec. Image reconstructed by discarding the zero coefficients
![Page 21: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/21.jpg)
21+
¿
𝑀×𝑁
𝑦 𝛼Φ Ψ
𝑁×𝑁 𝑁×1𝑀×1
![Page 22: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/22.jpg)
22
+ Constructing Φ
Tobias Bertelsen
![Page 23: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/23.jpg)
23+RIP - Restricted Isometry Property
The distance between two points are approximately the same in the signal-space and measure-space
A matrix satisfies the RIP of order K if there exists a such that:
holds for all -sparse vectors and
Or equally
holds for all 2K-sparse vectors
![Page 24: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/24.jpg)
24+RIP - Restricted Isometry Property RIP ensures that measurement error does not blow up
Image: http://www.brainshark.com/brainshark/brainshark.net/portal/title.aspx?pid=zCgzXgcEKz0z0
![Page 25: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/25.jpg)
25+Randomized algorithm
1. Pick a sufficiently high 2. Fill randomly according to some random
distribution
Which distribution?How to pick ?What is the probability of satisfying RIP?
![Page 26: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/26.jpg)
26+Sub-Gaussian distribution
Defined by Tails decay at least as fast as the Gaussian E.g.: The Gaussian distribution, any bounded distribution
Satisfies the concentration of measure property (not RIP):
For any vector and a matrix with sub-Gaussian entries, there exists a such that
holds with exponentially high probability where is a constant only dependent on
![Page 27: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/27.jpg)
27+Johnson-Lidenstrauss Lemma
Generalization to a discrete set of vectors For any vector the magnitude are preserved with:
For all P vectors the magnitudes are preserved with:
To account for this must grow with
![Page 28: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/28.jpg)
28+Generalizing to RIP
RIP: We want to approximate all -sparse vectors with unit
vectors The space of all -sparse vectors is made up of
-dimensional subspaces – one for each position of non-zero entries in
We sample points on the unit-sphere of each subspace
![Page 29: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/29.jpg)
29+Randomized algorithm
Use sub-Gaussian distributionPick Exponentially high probability of RIP
Formal proofs and specific formulas for constants exists
![Page 30: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/30.jpg)
30+Sparse in another base
We assumed the signal itself was sparse What if the signal is sparse in another base, i.e. is
sparse. must have the RIP As long as is an orthogonal basis, the random
construction works.
![Page 31: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/31.jpg)
31+Characteristics of Random
Stable Robust to noise, since it satisfies RIP
Universal Works with any orthogonal basis
Democratic Any element in has equal importance Robust to data loss
Other Methods Random Fourier submatrix Fast JL transform
![Page 32: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/32.jpg)
32
+ Sparse Signal Recovery
Malay Singh
![Page 33: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/33.jpg)
33+The Hyperplane of
![Page 34: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/34.jpg)
34+ Norms for N dimensional vector x
‖𝑥‖𝑝={ (∑𝑗=1𝑁
|𝑥 𝑗|𝑝)1𝑝 if𝑝>0
|𝑠𝑢𝑝𝑝 (𝑥)| if𝑝=0max
𝑗=1,2 ,… ,𝑁|𝑥 𝑗| if𝑝=∞
Unit Sphere of quasinorm
Unit Sphere of norm
Unit Sphere of norm
Unit Sphere of norm
![Page 35: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/35.jpg)
35+ Balls in higher dimensions
![Page 36: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/36.jpg)
36+How about minimization
But the problem is non-convex and very hard to solve
![Page 37: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/37.jpg)
37+We do the minimization
We are minimizing the Euclidean distance. But the arbitrary angle of hyperplane matters
![Page 38: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/38.jpg)
38+What if we convexify the to
![Page 39: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/39.jpg)
39+Issues with minimization
is non-convex and minimization is potentially very difficult to solve.
We convexify the problem by replacing by . This leads us to Minimization.
Minimizing results in small values in some dimensions but not necessarily zero. provides a better result because in its
solution most of the dimensions are zero.
![Page 40: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/40.jpg)
40
+ Convex Optimization
Hirak Sarkar
![Page 41: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/41.jpg)
41+What it is all about …
Find a sparse representation
Here and Moreover Two ways to solve
(P1) where is a measure of sparseness
(P2)
![Page 42: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/42.jpg)
42+How to chose and
Take the simplest convex function
A simple
Final unconstrained version
![Page 43: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/43.jpg)
43+Versions of the same problem
![Page 44: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/44.jpg)
44+Formalize
Nature of Convex Differentiable
Basic Intuition Take an arbitrary Calculate Use the shrinkage operator Make corrections and iterate
![Page 45: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/45.jpg)
45+
Shrinkage operator We define the shrinkage operator as follows
![Page 46: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/46.jpg)
46+ Algorithm
Input: Matrix ignal measurement parameter sequence
Output: Signal estimate
Initialization:
![Page 47: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/47.jpg)
47+Performance
For closed and convex function any the algorithm converges within finite steps
For and a moderate number of iterations needed is less than 5
![Page 48: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/48.jpg)
48
+ Single Pixel Camera
Nirandika Wanigasekara
![Page 49: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/49.jpg)
49+Single Pixel Camera
What is a single pixel camera An optical computer sequentially measures the Directly acquires random linear measurements without first
collecting the pixel values
![Page 50: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/50.jpg)
50+Single Pixel Camera- Architecture
![Page 51: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/51.jpg)
51+Single Pixel Camera- DMD Array
Digital Micro mirror Device A type of a reflective spatial light modulator Selectively redirects parts of the light beam Consisting of an array of N tiny mirrors Each mirror can be positioned in one of two states(+/-
10 degrees) Orients the light towards or away from the second lens
![Page 52: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/52.jpg)
52+Single Pixel Camera- Architecture
![Page 53: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/53.jpg)
53+Single Pixel Camera- Photodiode Find the focal point of the second lens Place a photodiode at this point Measure the output voltage of the photodiode The voltage equals , which is the inner product
between and the desired image .
![Page 54: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/54.jpg)
54+Single Pixel Camera- Architecture
![Page 55: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/55.jpg)
55+Single Pixel Camera- measurements A random number generator (RNG) sets the mirror
orientations in a pseudorandom 1/0 pattern Repeats the above process for times Obtains the measurement vector and Now we can construct the system in the
𝑦 𝑗 𝑥Φ j
¿
![Page 56: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/56.jpg)
56+Single Pixel Camera- Architecture
![Page 57: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/57.jpg)
+ 57
Sample image reconstructions
256*256 conventional image of black and white ‘R’
Image reconstructed from
How can we improve the reconstruction further?
![Page 58: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/58.jpg)
58+Utility This device is useful when measurements are
expensive Low Light Imager
Conventional Photomultiplier tube/ avalanche photodiode Single Pixel Camera Single photomultiplier
Original 800 1600
65536 pixels from 6600
![Page 59: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/59.jpg)
59+Utility CS Infrared Imager
IR photodiode
CS Hyperspectral Imager
![Page 60: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/60.jpg)
60
+ Compressed Sensing MRI
Yamilet Serrano Llerena
![Page 61: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/61.jpg)
61+Compressed Sensing MRIMagnetic Resonance Imaging (MRI)Essential medical imaging tool with slow data acquisition process.
Applying Compressed Sensing (CS) to MRI offers that:• We can send much less
information reducing the scanned time
• We are still able to reconstruct the image in based on they are compressible
![Page 62: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/62.jpg)
62+Compressed Sensing MRI
Scan Process
![Page 63: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/63.jpg)
63+Scan ProcessSignal Received K-space
Space where MRI data is stored
K-space trajectories:
K-space is 2D Fourier transform of the MR image
![Page 64: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/64.jpg)
64+In the context of CS
Φ :• Is depends on the acquisition device• Is the Fourier Basis• Is an M x N matrix
• Is the measured k-space data from the scanner
y :
y = Φ x
x :
![Page 65: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/65.jpg)
65+Recall ...
The heart of CS is the assumption that x has a sparse representation.
Medical Images are naturally compressible by sparse coding in an appropriate transform domain (e.g. Wavelet Transform)
Not significant
![Page 66: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/66.jpg)
66+Compressed Sensing MRI
Scan Process
![Page 67: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/67.jpg)
67+Image Reconstruction
CS uses only a fraction of the MRI data to reconstruct image.
![Page 68: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/68.jpg)
68+Image Reconstruction
![Page 69: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/69.jpg)
69+Benefits of CS w.r.t Resonance
Allow for faster image acquisition (essential for cardiac/pediatric imaging)
Using same amount of k-space data, CS can obtain higher Resolution Images.
![Page 70: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/70.jpg)
70
+ Summary
Parvathy Sudhir Pillai
![Page 71: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/71.jpg)
71+Summary
Motivation Data deluge Directly acquiring useful part of the signal
Key idea: Reduce the number of samples Implications
Dimensionality reduction Low redundancy and wastage
![Page 72: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/72.jpg)
72+Open Problems
‘Good’ sensing matrices Adaptive? Deterministic?
Nonlinear compressed sensing Numerical algorithms Hardware design
Intensity (x) Phase ()
Coefficients ()
![Page 73: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/73.jpg)
73+Impact
Data generation and storage Conceptual achievements
Exploit minimal complexity efficiently Information theory framework
Numerous application areas
Legacy - Trans disciplinary research Information
SoftwareHardware
Complexity
CS
![Page 74: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/74.jpg)
74+Ongoing Research
New mathematical framework for evaluating CS schemes Spectrum sensing
Not so optimal Data transmission - wireless sensors (EKG) to wired base
stations. 90% power savings
![Page 75: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/75.jpg)
75+In the news
![Page 76: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/76.jpg)
76+References Emmanuel Candes, Compressive Sensing - A 25 Minute Tour, First EU-US
Frontiers of Engineering Symposium, Cambridge, September 2010 David Schneider, Camera Chip Makes Already-Compressed Images, IEEE
Spectrum, Feb 2013 T.Strohmer. Measure what should be measured: Progress and Challenges in
Compressive Sensing. IEEE Signal Processing Letters, vol.19(12): pp.887-893, 2012.
Larry Hardesty, Toward practical compressed sensing, MIT news, Feb 2013 Tao Hu and Mitya Chklovvskii, Reconstruction of Sparse Circuits Using Multi-
neuronal Excitation (RESCUME), Advances in Neural Information Processing Systems, 2009
http://inviewcorp.com/technology/compressive-sensing/ http://ge.geglobalresearch.com/blog/the-beauty-of-compressive-sensing/ http://www.worldindustrialreporter.com/bell-labs-create-lensless-camera-thr
ough-compressive-sensing-technique/ http://www.lablanche-and-co.com/
![Page 77: Compressed Sensing](https://reader035.fdocuments.net/reader035/viewer/2022082209/5681692d550346895de070c7/html5/thumbnails/77.jpg)
77+
THANK YOU