Compressed Sensing - Achuta Kadambi

47
Compressed Sensing: Introduction and Apps Achuta Kadambi Camera Culture, MIT

description

 

Transcript of Compressed Sensing - Achuta Kadambi

Page 1: Compressed Sensing - Achuta Kadambi

Compressed Sensing: Introduction and Apps

Achuta KadambiCamera Culture, MIT

Page 2: Compressed Sensing - Achuta Kadambi

Exploiting Signals• Not all signals are equal! Find a weakness then exploit.

Page 3: Compressed Sensing - Achuta Kadambi

Exploiting Signals• Not all signals are equal! Find a weakness then exploit.

• Shannon-Nyquist Bandlimited signals can be sampled/reconstructed

Page 4: Compressed Sensing - Achuta Kadambi

Exploiting Signals• Not all signals are equal! Find a weakness then exploit.

• Shannon-Nyquist Bandlimited signals can be sampled/reconstructed• Rank-constrained Optimization Low Rank signals can be

interpolated (Netflix Problem)

Page 5: Compressed Sensing - Achuta Kadambi

Exploiting Signals• Not all signals are equal! Find a weakness then exploit.

• Shannon-Nyquist Bandlimited signals can be sampled/reconstructed• Rank-constrained Optimization Low Rank signals can be

interpolated. (Netflix Problem)• Compressed Sensing Sparse signals can be undersampled and

recovered.

Page 6: Compressed Sensing - Achuta Kadambi

Outline of this talk. • Compressed Sensing overview.

• Very brief explanation on the why and how of Compressed Sensing.

• ‘Apps’ that use compressed sensing.

• Practical strategies for implementation (e.g. pseudocode, libraries).

Page 7: Compressed Sensing - Achuta Kadambi

Motivation: JPEG CompressionOur visual system is less sensitive to high (spatial) frequency detail. Can we throw away these frequencies and retain a similar image?

This is the intuition behind JPEG.

Spatial Frequency E.g.:High Hair, Blades of Grass, etc. Low Sky, Skin, etc.

Compressed Sensing: If we are going to throw away stuff … why spend time acquiring it?

Page 8: Compressed Sensing - Achuta Kadambi

E.g.

Page 9: Compressed Sensing - Achuta Kadambi

Wired Magazine: “Fill in the Blanks…”

Page 10: Compressed Sensing - Achuta Kadambi

1D Implementation in L1MagicStep 1: The original signal and its Fourier Transform.

Original Signal (N = 256) Spectrum

Page 11: Compressed Sensing - Achuta Kadambi

Implementation in L1MagicStep 2: The subsampled signal

Red Entries (80 samples) are observed.

Blue Entries (176 samples) must be recovered.

That means we observe only 30% of the original signal.

Page 12: Compressed Sensing - Achuta Kadambi

Implementation in L1MagicStep 3: Exact Recovery of the Signal.

Original Signal (N = 256) Reconstruction (N = 256)

Page 13: Compressed Sensing - Achuta Kadambi

L1Magic for Images

Original Image: 1 million pixels

Reconstruction: from 100,000

random measurements.

Page 14: Compressed Sensing - Achuta Kadambi

Goes back to Fourier

Page 15: Compressed Sensing - Achuta Kadambi

Fourier Transform

Intuition: Projection, or Inner Product, of Signal with Trigonometric Functions.

Page 16: Compressed Sensing - Achuta Kadambi

Sparsity goes back to Fourier (circa 1800)

Superposition of Sinusoids

Original Time Domain Function

Frequency Domain Representation

Page 17: Compressed Sensing - Achuta Kadambi

Discrete Fourier Transform

Example DFT: Time Signal is a Delta. Spectrum is Broadband.

Page 18: Compressed Sensing - Achuta Kadambi

DFT in Matrix Form

Page 19: Compressed Sensing - Achuta Kadambi

Nyquist-Shannon Sampling TheoremIn Shannon’s words:

How to Reconstruct? (Interpolation)

Compressive Sensing: Can we do better?

Page 20: Compressed Sensing - Achuta Kadambi

Inverse Problem𝐷𝑎𝑡𝑎→𝑀𝑜𝑑𝑒𝑙 𝑃𝑎𝑟𝑎𝑚𝑒𝑡𝑒𝑟𝑠

Example 1: Sinc Interpolation. Given the Data (a sufficiently sampled signal), how can we obtain the original signal?

Example 2: Blurry Photos. Given a Blurry Photo, from a Camera, how can we go back to the original, sharp image?

Example 3: Given a discrete time signal, how can we obtain its discrete spectrum? **DFT problem is a Linear Inverse Problem

Page 21: Compressed Sensing - Achuta Kadambi

Solving the DFT Problem𝑦=𝐴𝑥𝐴−1 𝑦=𝑥

Done?

Page 22: Compressed Sensing - Achuta Kadambi

Solving the DFT Problem via Optimization𝑦=𝐴𝑥𝐴−1 𝑦=𝑥

Done?

Loss Function

Page 23: Compressed Sensing - Achuta Kadambi

Solving the DFT Problem via Optimization𝑦=𝐴𝑥𝐴−1 𝑦=𝑥

Done?

PseudoInverse: Minimize MSE

Page 24: Compressed Sensing - Achuta Kadambi

Constraining our Solution via Regularization

Additional Term allows for some prior on original signal. For instance if Tikhonov Matrix is a first order difference, then you are biasing x toward smooth solutions.

Linked to the Lagrange problem, as well as Maximum A Posteriori from probability, and Weiner filter from Sig proc.

We can go beyond loss function, e.g., Tikhonov Regularization

Page 25: Compressed Sensing - Achuta Kadambi

Compressed Sensing

Page 26: Compressed Sensing - Achuta Kadambi

Compressed Sensing Structure• Underdetermined system. y=Ax. • Y is m-dimensional sampled vector• A is mxn matrix• X is n-dimensional original vector. • And m << n y A

x

Page 27: Compressed Sensing - Achuta Kadambi

Simply Solving y=Ax not good enough• This gives you an affine space with many solutions to y=Ax.

• So we must constrain our problem to look for the sparse solution to y=Ax.

Page 28: Compressed Sensing - Achuta Kadambi

Occams RazorOccam's Razor: among otherwise equal explanations,the simplest is best

Page 29: Compressed Sensing - Achuta Kadambi

Occams RazorOccam's Razor: among otherwise equal explanations,the simplest is best

CS Occam's Razor: among otherwise equal solutions, the sparsest is best

Unfortunately, this optimization is not tractable

Page 30: Compressed Sensing - Achuta Kadambi

Geometric Property of Norms

Page 31: Compressed Sensing - Achuta Kadambi
Page 32: Compressed Sensing - Achuta Kadambi

The l1 Optimization Problem

Page 33: Compressed Sensing - Achuta Kadambi

RIP/Spark/Coherence• The sensing matrix A must be carefully chosen.

• For compressed sensing to work, the matrix A must satisfy the Restricted Isometry Property (RIP):

• Calculating RIP is NP-hard. We can work with easier quantities than the RIP, such as spark and mutual incoherence.

Page 34: Compressed Sensing - Achuta Kadambi

App1: Single-Pixel Camera

Page 35: Compressed Sensing - Achuta Kadambi

App1: Single-Pixel Camera

Design Advantage: A MP camera with just a single-pixel.

Page 36: Compressed Sensing - Achuta Kadambi

App2: Single-Pixel THz imaging.

Chan et al. Applied Physics 2008

Design Advantage: CS allows for single-pixel THz sensors, which are much easier to fabricate than pixel array.

In general, you can buy amazing things at single-pixel level, e.g., picosecond detectors, thermal IR sensor, etc.

Page 37: Compressed Sensing - Achuta Kadambi

App2: Monitoring Breathing via Smartphone

Oletic, Skrapec, and Bilas MobiHealth 2012

Design Advantage: Compressed sensing allows for low power acquisition and reduced streaming.

Very similar to OMP, DFT formulation.

Basically finding sparse spectral components that characterize the audio signal of breathing.

Page 38: Compressed Sensing - Achuta Kadambi

App4: Biometrics … Face Recognition

Design Advantage: Using compressed sensing to handle the small sample size problem.

Before, the number of samples in the database Is less than the degrees of freedom of each sample.

Page 39: Compressed Sensing - Achuta Kadambi

App5: Fast MRI

Design Advantage: Less samples means less time for an MRI Scan, which means less time a sick or disabled patient lies in the scanner.

Page 40: Compressed Sensing - Achuta Kadambi

App5: Fast MRI

Page 41: Compressed Sensing - Achuta Kadambi

App6: Compressive Sensing of High Speed Periodic Videos

Design Advantage: Exploit sparsity of Periodic Videos to obtain a high speed video without using a high speed camera.

Veeraraghavan A, Reddy D, Raskar R. IEEE PAMI

Page 42: Compressed Sensing - Achuta Kadambi

App7: Compressive Light Field Photography Marwah, Wetzstein, Bando,

Raskar. ACM SIGGRAPH 2013.

Design Advantage: Obtain High-Resolution Light Field photos by placing a coded mask in front of the sensor.

Page 43: Compressed Sensing - Achuta Kadambi

App8: Sparsity-Induced Time of Flight Cameras

E.g., Light Sweep Movies from Refael’s talk last week. Goal is to obtain a well-conditioned deconvolution problem.

Kadambi et al. ACM SIGGRAPH Asia

Design Advantage: Deconvolve to obtain bounces of light and construct a light sweep video.

Page 44: Compressed Sensing - Achuta Kadambi

App9 YOUR App!

Design Advantage: <insert here>

Page 45: Compressed Sensing - Achuta Kadambi

Practical StrategiesMany libraries are available for C++/Matlab/etc.

• Recommended: L1magic (http://users.ece.gatech.edu/~justin/l1magic/)

• SPGL1 (http://www.cs.ubc.ca/~mpf/spgl1/)

• CVX (http://cvxr.com/cvx/)

• On phone, nothing exists yet, but you can use Efficient Java Matrix Library (EJML) to implement solely in linear algebra.

Page 46: Compressed Sensing - Achuta Kadambi

L1Magic PseudocodeFor 1D signal.

x = original_signal; // read in your original signal. R = randn(m,n); // create a random matrix of dimension mxn A = orth(R’)’; // sensing matrix with orthogonal columns.

y = A*x; // create subsampled signal y of only m entries. X0 = A’*y; // Initial guess by taking matrix inverse. x_hat = l1eq_pd(x0, A, [], y, 1e-3); // run l1 solver.

norm(x_hat – x); // error; should be zero in ideal case

Page 47: Compressed Sensing - Achuta Kadambi

Take-home Messages• Opportunity to integrate cutting-edge mathematical techniques into

your camera apps.

• Compressed Sensing is lightweight in terms of coding. The key is correctly identifying the sparsity in your engineering problem.

• Not all signals are equal… find a weakness, e.g., sparsity/rank, and exploit it.• Exploit in Hardware, Exploit in Software…