COMPRESSIVE GHOST IMAGING Andreas Valdmann

1

Traditional imaging

Light source

Matrix sensor (Camera)

Object with reflectivity 𝑂 𝑥,𝑦

Signal 𝑆𝑐𝑐𝑐(𝑥,𝑦) ~ 𝑂(𝑥,𝑦)

Uniform illumination

2

Ghost imaging

Light source

Matrix sensor (Camera) measures 𝐼(𝑥, 𝑦)

Object 𝑂(𝑥,𝑦)

Beam splitter

𝐼(𝑥,𝑦)

𝐼(𝑥,𝑦) 𝑂(𝑥,𝑦)

Single pixel detector measures 𝑆 = ∫ 𝐼 𝑥,𝑦 𝑂 𝑥, 𝑦 𝑑𝑥𝑑𝑦

𝑂(𝑥, 𝑦) = 𝑆 − 𝑆 𝐼(𝑥, 𝑦) − 𝐼(𝑥, 𝑦) 𝑂 𝑥,𝑦 =

1𝑀�𝑆𝑖𝐼𝑖(𝑥,𝑦)

𝑀

𝐼=1

M – number of measurements

Random pattern

3

Image reconstruction

Computational ghost imaging

Object 𝑂(𝑥,𝑦)

𝐼(𝑥,𝑦)

𝐼(𝑥,𝑦) 𝑂(𝑥,𝑦)

Single pixel detector measures 𝑆 = ∫ 𝐼 𝑥,𝑦 𝑂 𝑥, 𝑦 𝑑𝑥𝑑𝑦

𝑂(𝑥, 𝑦) = 𝑆 − 𝑆 𝐼(𝑥, 𝑦) − 𝐼(𝑥, 𝑦) 𝑂 𝑥,𝑦 =

1𝑀�𝑆𝑖𝐼𝑖(𝑥,𝑦)

𝑀

𝐼=1

M – number of measurements

𝐼(𝑥, 𝑦) is determined by user. No camera is needed.

Digital projector

4

Image reconstruction

5

6

7

Multicolor ghost imaging

Object 𝑂𝜇(𝑥,𝑦)

𝐼(𝑥,𝑦)

𝐼(𝑥,𝑦)𝑂𝜇(𝑥,𝑦)

Single pixel detector measures 𝑆𝜇 = ∫ 𝐼 𝑥,𝑦 𝑂𝜇 𝑥,𝑦 𝑑𝑥𝑑𝑦

𝑂𝜇(𝑥, 𝑦) = 𝑆𝜇 − 𝑆𝜇 𝐼(𝑥, 𝑦) − 𝐼(𝑥, 𝑦) 𝑂𝜇 𝑥,𝑦 =

1𝑀�𝑂𝜇𝑖𝐼𝑖(𝑥,𝑦)

𝑀

𝐼=1

M – number of measurements

𝐼(𝑥, 𝑦) is determined by user. No camera is needed.

Digital projector

8

Experimental results

S. Welsh, M. Edgar, R. Bowman, P. Jonathan, B. Sun, and M. Padgett, "Fast full-color computational imaging with single-pixel detectors," Opt. Express 21, 23068-23074 (2013).

1 million measurements!

9

Inversion image reconstruction

𝐈 = [𝐼1, 𝐼2, … , 𝐼𝑀] matrix of all light patterns

𝑆 = [𝑠1, 𝑠2, … , 𝑠𝑀] array of single pixel detector signals

𝐼1, 𝐼2, … , 𝐼𝑀 are flattened light pattern arrays

Pixels (N)

Mea

sure

men

ts (M

) =

𝑆 𝑂 𝐈

M measurements; N pixels

1 2 3 4 1 2 3 4

x

y

𝐈𝑂 = 𝑆

𝑂 flattened image of the object (unknown)

System of linear equations

10

Inversion image reconstruction

M measurements; N pixels

𝐈𝑂 = 𝑆

M = N: perfect image recovery if no noise is present (only in theory). M > N: can be solved with least squares methods. Larger M gives better signal to noise ratio, but measurements and calculations take more time. M < N: Theoretically infinite number of solutions. Additional constraints must be given. Compressive sensing

Pixels (N)

Mea

sure

men

ts (M

)

=

𝑆 𝑂 𝐈

11

Discrete cosine transform

12

JPEG compression DCT frequencies

DCT of a natural image gives a sparse matrix.

13

Compressive sensing Is it necessary to make many measurements if most of the data is omitted during compression anyway? No, if the measured image is natural, i.e. it is sparse in some basis.

Idea of compressive sensing algorithm

• Solve the underdetermined system in some sparse basis.

• Additional constraint: result vector must contain as many zeros as possible: L0 norm minimization.

14

Lp norm

𝑥 𝑝 = �𝑥𝑖𝑝𝑛

𝑖=0

1𝑝

𝑥 0 counts non-zero elements in vector

𝑥 1 sum of absolute values of vector components (Manhattan geometry)

𝑥 2 Euclidian norm

Minimizing L0 norm is NP-hard. Minimizing L1 norm also gives a small L0 norm. Examples of implementation: • Matlab L1-MAGIC toolbox • Python CVXOPT package

15

Comressive ghost imaging 𝐈𝑂 = 𝑆 Initial problem

Discrete cosine transform 𝐈 → 𝐈𝑫𝑫𝑫

𝐈𝑫𝑫𝑫𝑂𝐷𝐷𝐷 = 𝑆 𝑂𝐷𝐷𝐷 is in sparse basis

Minimize 12

𝑆 − 𝐈𝑫𝑫𝑫𝑂𝐷𝐷𝐷 2 + λ� 𝑂𝐷𝐷𝐷

λ is a small regularization parameter

Inverse discrete cosine transform 𝑂𝐷𝐷𝐷 → 𝑂

16

Experimental results

S. Welsh, M. Edgar, R. Bowman, P. Jonathan, B. Sun, and M. Padgett, "Fast full-color computational imaging with single-pixel detectors," Opt. Express 21, 23068-23074 (2013).

17

Potential applications of compressive sensing • Enegry efficient miniature cameras [David Schneider

(March 2013). "New Camera Chip Captures Only What It Needs". IEEE Spectrum.]

• Magnetic resonance imaging (MRI) [Michael Lustig. Compressed sensing MRI resources. http://www.eecs.berkeley.edu/~mlustig/CS.html]

• Hyperspectral imaging [Y. August, C. Vachman, Y. Rivenson,

and A. Stern, "Compressive hyperspectral imaging by random separable projections in both the spatial and the spectral domains," Appl. Opt. 52, D46-D54 (2013).]

18

Summary Ghost imaging • Single pixel detector • Many measurements • Replaces matrix sensor

Compressive sensing • Number of measurements is

less than number of pixels • Uses sparseness of natural

images

Thank you

19