Signal Processing with Side InformationA Geometric Approach via Sparsity

João F. C. Mota

Heriot-Watt University, Edinburgh, UK

2/23

Side Information

Medical imaging

MRI PET

Consumer electronics Robotics

multi-modal

prior information

heterogeneous

Signal processing tasks

Denoising

Reconstruction

Demixing (source separation)

Compression

Inpainting, super-resolution, …

Recommender systems

How to represent multi-modal or heterogeneous data ?

How to process it ?

3/23

Outline

Compressed Sensing with Prior Information

Application: Video Background Subtraction

X-ray Image Separation

Conclusions

N DeligiannisVUB-Belgium

M RodriguesUCL

4/23

Compressed Sensing (CS)

How do we integrate in the problem?

Reconstruction guarantees?

Compressed SensingSucess rate (50 trials)

number of measurements

What if we know ? prior information

medical images, video, …

CS boundOur bound

CS performanceCS + PI

sparse

iid Gaussian

Basis pursuit

5/23

Intuition

measurements

Tangent cone of at

solutions of

Our approach

prior information (PI)

model for PI small

random orientation

6/23

Good components

Bad components

7/23

parameter-freeTheorem (L1-L1 minimization) [M, Deligiannis, Rodrigues, 2017]

i.i.d.

L1-L1 minimization

Theorem (BP) [Chandrasekaran, Recht, Parrilo, Willsky, 2012]

sparse

support overestimation

8/23

Experimental Results

Sucess rate (50 trials)

number of measurements

L1-L1 L1-L2

BP BP bound

Mod-CSMod-CS

[Vaswani and Lu, 2010]

Gaussian

9/23

Prior Information can help, but can also hinder

L1-L1 works better than L1-L2 (theory and practice)

(Computable) bounds are tight for L1-L1, but not for L1-L2

Theory predicts optimal ; indicates how to improve

Limitations: Gaussian matrices; bounds depend on unknown parameters

Summarizing

10/23

Outline

Compressed Sensing with Prior Information

Application: Video Background Subtraction

X-ray Image Separation

Conclusions

N DeligiannisVUB-Belgium

M RodriguesUCL

A SankaranarayananCMU-USA

V CevherEPFL-CH

11/23

observed

Compressive Background Subtraction

How to recover from online ?

How many measurements from frame ?

linear operation

CS camera

12/23

Our Approach

Estimate from past frames:

Integrate into BP

Assumption: background is static & known fg measurements

Basis Pursuit

sparse

foregroundCompressive sensing for background subtraction [Cevher, Sankaranarayanan, Duarte, et al, 2008]

Problems

Prior frames are ignored

fixed; depends on foreground area

background

via minimization

13/23

Problem Statement

sparse arbitrary function sparsetime

measurements

Model

Problem

Compute a minimal # of measurements

Reconstruct perfectlyonline algorithm w/ adaptive rate

14/23

Algorithm: computed at iteration

Gaussian

parameters of

L1-L1 minimization

oversampling factor

and repeat ...

# measurements of

Set Estimate

Acquire with

15/23

Estimating a Frame

estimation extrapolation

linear motion

overlap: take average; gaps: fill w/ average of neighbors

state-of-the-art in video coding

16/23

Experimental Results280 frames

Number of measurements

Frame

CS oracle prior state-of-the-art [Warnell et al, 2014]

L1-L1 oracle

Oursreduction of 67%

modified CS (nonadaptive)

17/23

Experimental Results280 frames

Relative error

Frame

reconstruction

estimation

determined by solver

modified CS (nonadaptive)

18/23

Outline

Compressed Sensing with Prior Information

Application: Video Background Subtraction

X-ray Image Separation

Conclusions

N DeligiannisVUB-Belgium

M RodriguesUCL

B CornelisVUB-Belgium

I DaubechiesDuke-USA

19/23

Motivation: X-Ray of Ghent Altarpiece

Mixed X-Ray

Can we use the visual images to separate the x-rays?

20/23

Approach: Coupled Dictionary LearningTraining step VisibleX-Ray

coupling

Demixing step

w/ sparse columns

learn dictionaries by alternating minimization

mixed x-ray visual front visual back

21/23

Results

mixed x-ray

visuals in grayscale

Ours

multiscale MCA w/KSVD

MCA [Bobin et al, 07’]

reconstructed x-rays

22/23

Summary / Conclusionsdata

Low-rank modelobservations

multi-modal features

dictionaries sparse columns

sparse sparse

prior information

measurements

Better models?

Guarantees?

Scalable algorithms?

X-ray separation

Reconstruction w/ PI

Applications

medical imaging (MRI + PET + ECG) SAR + microwave imaging

super-resolution (depth + visual) robotics (laser + sonar)

23/23

References

J. F. C. Mota, N. Deligiannis, M. R. D. Rodrigues Compressed Sensing with Prior Information: Optimal Strategies, Geometries, and BoundsIEEE Transactions on Information Theory, Vol 63, No 7, 2017

J. F. C. Mota, N. Deligiannis, A. C. Sankaranarayanan, V. Cevher, M. R. D. Rodrigues Adaptive-Rate Reconstruction of Time-Varying Signals with Application in Compressive Foreground ExtractionIEEE Transactions on Signal Processing, Vol 64, No 14, 2016

N. Deligiannis, J. F. C. Mota, B. Cornelis, M. R. D. Rodrigues, I. DaubechiesMulti-Modal Dictionary Learning For Image Separation With Application in Art InvestigationIEEE Transactions on Image Processing, Vol 26, No 2, 2017