Dense Ray Tracing Based Reconstruction Algorithm for Light Field PIV and Comparative...

18th International Symposium on the Application of Laser and Imaging Techniques to Fluid Mechanics・LISBON | PORTUGAL ・JULY 4 – 7, 2016

Dense Ray Tracing Based Reconstruction Algorithm for Light Field PIV and Comparative Study with Tomo-PIV

Shengxian Shi1*, Junfei Ding1 and T.H. New2 1: Gas Turbine Research Institute, School of Mechanical Engineering, Shanghai Jiao Tong University 200240, Shanghai, China

2: School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore 639798 * Correspondent author: [email protected]

Keywords: 3D-PIV, light field imaging, plenoptic camera, Tomo-PIV, DRT-MART

ABSTRACT

This work presents an in-house developed high resolution light field volumetric PIV system, as well as a new 3D

particle image reconstruction algorithm based on dense ray tracing and multiplicative algebraic reconstruction

technique (DRT-MART). Parametric studies are firstly carried out to access key optical parameters on performance

of the light field volumetric PIV technique, followed by simulation studies that assess the capability of the DRT-

MART algorithm by comparing its reconstruction quality and computational cost with the MART method. In the

last, performance of the new algorithm as well as light field volumetric PIV are further tested with synthetic images

which are generated from a DNS jet flow, and compared with results from Tomo-PIV.

1. Introduction

Many fluid phenomena are inherently complex and three-dimensional, which urges the PIV

technique to progress from planar measurement to fully volumetric velocity measurement. One

of the first efforts was Stereo-PIV, which measures the third velocity component by including

one additional camera to the traditional 2D-PIV system (Prasad et al 1993, Arroyo et al 1996).

Scanning PIV extends such single slice 2D-3C measurement to multiple planes by using a series

of scanning laser sheets and a pair of high speed cameras, however its maximum measurable

velocity is limited by the camera frame rate, laser repetition rate or scanning mirror speed

(Brucker 1996, Hori 2004). Instead of measuring the third velocity component via dual-view

geometry, Defocusing Digital PIV (DDPIV) recovers depth information from defocused particle

images and normally employs a triple-camera arrangement to resolve the flow field with

satisfactory accuracy (Willert et al 1992, Pereira et al 2000). Holographic PIV (HPIV) resolves

volumetric velocity field from particle holograms, which are recorded by in-line or off-axis

holography (Arroyo et al 2008, Katz et al 2010). The application of this technique, however, is

limited by its complex experimental setup. One of widely applied three dimensional velocity


measurement techniques is Tomo-graphic PIV (Tomo-PIV), which employs multiple view

geometry (typical 4-8 views) to capture particle images and calculate three dimensional velocity

from multiplicative reconstruction technique (MART) and volumetric cross correlation (Elsinga

et al 2006, Scarano 2013). Tomo-PIV has advantages in high spatial resolution as well as relative

large measurable volume.

Apart from recording tracer particle’s three dimensional position through multiple view

geometry, other techniques record light field of tracer particles. One of such technique is

synthetic aperture PIV (SAPIV), which uses a large camera array (normally 8 to 15 cameras) to

capture the light field image for seeding particles and reconstructs 3D particle image through

synthetic aperture refocusing method. SAPIV can tolerate much higher particle density than

Tomo-PIV and its measurable range along optical axis can be on the same order as lateral

directions (Belden et al 2010). Instead of using camera array system, light field photography

based PIV (shorted as LF-PIV hereafter) records particle light field image through a compact

plenoptic camera, which is the combination of a high resolution micro-lens array (MLA) and a

high resolution CCD sensor. Studies have demonstrated that LF-PIV can resolve 3D velocity

fields through MART based re-construction and 3D cross-correlation (Ding et al 2015, Fahringer

et al 2015, Shi et al 2016).

In the following sections, systematic studies are firstly performed on how key optical parameters

affect resolution of plenoptic camera. In section 3, methodology of dense ray tracing based

MART (DRT-MART) reconstruction method is outlined and its performance is compared with

MART by ray tracing based simulation. In the last, the LF-PIV technique is evaluated by using

synthetic jet flow light field images, the results are compared with Tomo-PIV measurements.

2. Camera prototyping and ray tracing based light field simulation

Fig. 1. In-house developed plenoptic camera

相机机身

CCD平面

镜头安装筒

MLA及调节支架

Lens mount MLA

CCD

Camera body


An in-house light field camera shown in Fig. 1 was developed according to plenoptic imaging

(Ng 2006), where a customised micro-lens array (MLA) is precisely positioned one focal length

away from the CCD plane (IMPERX B6640). The MLA consists of 458×301 hexagonal lens unit,

which maximises the pixel usage when compared to a square lens unit. Light field image of

tracer particles can be simulated via linear Gaussian optics according to Eqs.1~5 (Georgiev et al

2003).

Particle O Main lens

(

x′y′

θ′φ′

) = (

10

01

So

0

0So

00

00

10

01

) (

xyθφ

) (1)

Through Main lens

(

x′y′

θ′φ′

) = (

10

01

00

00

−1/fm

0

0−1/fm

10

01

) (

xyθφ

) (2)

Main lens MLA

(

x′y′

θ′φ′

) = (

10

01

Si

0

0Si

00

00

10

01

) (

xyθφ

) (3)

Through MLA

(

x′y′

θ′φ′

) = (

10

01

00

00

−1/fl

0

0−1/fl

10

01

) (

xyθφ

) + (

00

Sx/fl

Sy/fl

) (4)

MLA CCD

(

x′y′

θ′φ′

) = (

10

01

fl

0

0fl

00

00

10

01

) (

xyθφ

) (5)

where x,y is the spatial location of particle O. θ, φ is the orientation angle of light ray emitted

from the particle. Geometry of a representative light ray is plotted in Fig. 2. For simulation

studies in the paper, a series of synthetic light field images are generated using ray tracing

method with key parameters listed in Table 1.


Fig. 2. Schematic of ray tracing for plenoptic camera

Table 1 Optical parameters for ray tracing simulation

Symbol Parameter Pixel Microlens Ratio

PMR8 PMR14 PMR28

nlx MLA resolution: X 63 31 15

nly MLA resolution: Y 63 31 15

pl Microlens pitch 44μm 77μm 154μm

fl Microlens focal length 308μm

npx Camera resolution: X 448

npy Camera resolution: Y 448

pp Pixel pitch 5.5μm

fm Main lens focal length 50mm

Pm Main lens aperture 25mm

So Object distance 100mm

Sl Image distance 100mm

M Magnification factor -1

(f/#)m Main lens f number 3.5 2 1

(f/#)l Microlens f number 7 4 2

Ray

pl

pp

Si So

Z

fm

Pm

Focal Plane

dy

dz

Sy

MLA CCD

VB

YCCD

Yl O

Main Lens


According to the studies made by Georgeiv et al (2006), spatial and angular resolution of a

plenoptic camera is determined by the resolution of MLA and number of pixels beneath each

microlens (pixel microlens ration, PMR) respectively. As these spatial and angular resolution

will greatly affect the reconstruction performance of plenoptic camera, detailed ray tracing

analysis is made in this section to study the effect of PMR on y-z and x-y plane resolution. For

illustration purpose, analysis is only made for square microlens, but conclusions can be generally

extent to hexagonal microlens as well.

Fig. 3. Formation of an unresolvable block by back ray tracing

Two spatially separated point light sources are said to be resolved if the location variation results

in any light columns be captured by different microlens. To illustrate the resolution limit, the

outermost light rays (or boundary) of the discretized light columns are plotted in Fig. 3. Take the

top green line as an example, which is plotted by tracing a light ray from the lower edge of the

center microlens through the top portion of the discretized main lens, and back to the object side.

It is clear that any point light source moving across such line will result in the top light column

moving across the center microlens, and hence leads to pixel intensity variations. Based on such

analysis, performing back ray tracing for outermost light rays of the discretized light columns at

the upper edge of the center microlens would form a series of closed blocks. Any point light

sources inside these blocks cannot be distinguished. Extend such analysis to a small region (-

1mm<z<1mm and -0.3mm<y<0.3mm), discretized light columns for PMR=8, 14 and 28 can be

plotted in Figs. 4, 5 and 6, where blank blocks represent unresolvable areas, and separation

between two red lines represents one micro-lens size. An instant observation from Figs. 4, 5 and

6 is that resolution in x-y plane decreases with the increase of PMR. The reason is very

straightforward, for a fixed image sensor size and fixed pixel pitch, increase of PMR will reduce

lenslet number of the correspondent MLA, and hence will reduce planar resolution of plenoptic

camera. On the other hand, higher PMR will generally leads to higher resolution along optical

Focal Plane Main lens MLA CCD


axis, except on the focal plane. As such a very small pixel size as well as densely packed MLA is

preferred. However, too small pixel size will greatly reduce the camera sensitivity and densely

packed microlens array will significantly increase the manufacturing cost or even impossible to

fabricate.

Fig. 4. Resolution variation for PMR=8

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

-0.25

-0.2

-0.15

-0.1

-0.05

0

0.05

0.1

0.15

0.2

0.25

pl / p

p = 8

Z (mm)

Y (

mm

)

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

-0.25

-0.2

-0.15

-0.1

-0.05

0

0.05

0.1

0.15

0.2

0.25

pl / p

p = 8

Z (mm)

Y (

mm

) -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

-0.25

-0.2

-0.15

-0.1

-0.05

0

0.05

0.1

0.15

0.2

0.25

pl / p

p = 8

Z (mm)

Y (

mm

)

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

-0.25

-0.2

-0.15

-0.1

-0.05

0

0.05

0.1

0.15

0.2

0.25

Z (mm)

Y (

mm

)

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

-0.25

-0.2

-0.15

-0.1

-0.05

0

0.05

0.1

0.15

0.2

0.25

Z (mm)

Y (

mm

)

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

-0.25

-0.2

-0.15

-0.1

-0.05

0

0.05

0.1

0.15

0.2

0.25

Z (mm)

Y (

mm

)

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

-0.25

-0.2

-0.15

-0.1

-0.05

0

0.05

0.1

0.15

0.2

0.25

Z (mm)

Y (

mm

)




3. Dense ray tracing based MART reconstruction method (DRT-MART)

Similar to particle reconstruction in Tomo-PIV, the MART reconstruction for light field PIV is

very time consuming, if not worse. As reported by Fahringer et al (2015), weighting matrix of a

300 × 200 × 200 voxels volume takes 350 GB, for only storing non-zero voxel values. The MART

reconstruction took 1.5hrs on a 12 cores work stations. As a matter of fact, tracer particles are

sparsely distributed in the measurement volume, and only a small portion of voxels have non-

zero values. Hence the computational load and storage request can be greatly reduced if only

non-zero voxels are reconstructed. This has been proved to be an efficient reconstruction

method by Atkinson et al (2009), who proposed an MLOS approach to pre-determine the non-

zero voxels.

The proposed dense ray tracing based reconstruction method employs similar idea, but it is

fundamentally different from MLOS on how is implemented. For Tomo-PIV, pixel line of sight

can be easily determined by camera calibration, the line of sight of each pixel is fixed for a

specific experimental set-up. However, this is not the case for light field PIV. As demonstrated in

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

-0.25

-0.2

-0.15

-0.1

-0.05

0

0.05

0.1

0.15

0.2

0.25

pl / p

p = 28

Z (mm)

Y (

mm

)

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

-0.25

-0.2

-0.15

-0.1

-0.05

0

0.05

0.1

0.15

0.2

0.25

pl / p

p = 28

Z (mm)

Y (

mm

)

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

-0.25

-0.2

-0.15

-0.1

-0.05

0

0.05

0.1

0.15

0.2

0.25

pl / p

p = 28

Z (mm)

Y (

mm

)


Fig. 7, pixel line of sight varies with the spatial location of tracer particles. Hence, all affected

pixels must be taken into account for reconstructing a specific voxel.

Fig. 7. Light field images generated by ray tracing simulation

Figure 8 illustrates the principle of the DRT method. To determine the pixel line of sight for the

red voxel, three representative light rays (nine light rays for 3D case) were drawn for each

discretised main lens portion so as to find the affected micro-lens units. Once this is

accomplished, the affected pixels can be easily located according to the analysis made in the

previous section. With the affected pixels available, a simple multiplication of their values would

help picking out the non-zero voxels.

(a) Light source on the focal plane

(b) Light source offset to the focal plane, dz=0.385mm


dz


Fig. 8. Principle of the DRT method

To evaluate the performance for DRT-MART reconstruction algorithm, a small plenoptic camera

(31×31 micro-lens units, 448×448 pixels) was simulated by ray tracing. A small volume measured

2mm×2mm×4mm was discretized into 385×385×57voxels. Note that the voxel size in z direction

is the same as micro-lens pith, whereas the voxel size x and y direction is the same as pixel pitch.

For brevity, weighting coefficients were calculated by the new weighting algorithm for both

DRT-MART and MART reconstruction results. To explore the effect of resolution variation along

optical axis on the proposed new reconstruction algorithm, A series of light field images were

generated by ray tracing simulation for particle location varies from z=-2 ~ 2mm along the

optical axis with a step of 0.077mm. Note that the focal plane locates at z=0. Fig. 9a plots the

accuracy of the reconstructed particle image center for DRT-MART and MART algorithms, and

Fig. 9b shows the reconstruction quality for the two methods.

An instant observation from Fig. 9 is that the accuracy of reconstructed particle image center as

well as reconstruction quality reach to the lowest level near the focal plane (z =-0.308~

0.308mm). This is because that all light rays from particles in this range were all focused onto a

single micro-lens, and subsequently being captured by the same group of pixels, as illustrated in

Fig. 7a. As such, the reconstructed voxel intensity distribution is nearly the same for particles

located in this range.

For particles outside the focal plane region (z <-0.308, z >0.308mm), the reconstructed particle

image centre of the two algorithms matches well with real values. But the DRT-MART algorithm

shows considerably higher reconstruction quality than MART in these regions. The reason is that

DRT-MART could successfully filter out the zero voxels which surround around the affected

voxels. On the other hand, however, these neighboring voxels are also included by MART


Reconstruction area


during its reconstruction iterations. With particle moves away from the focal plane, the voxel

intensity reconstructed by the DRT-MART algorithm starts to expand due to the decrease of

resolution in x and y direction and results in slight decrease in Q value.

Fig. 9. (a) Accuracy of reconstructed particle image centre coordinate; (b) Reconstruction quality;

(c) computational efficiency of the DRT-MART method for various particle density cases

(a)

Real center (Z, mm)

Re

co

ns

tru

cti

on

ce

nte

r(Z

,m

m)

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5

2Real

DRT-MART

MART

Z (mm)

Q

-2 -1.5 -1 -0.5 0 0.5 1 1.5 20

0.2

0.4

0.6

0.8

1 DRT-MART

MART

(b)

Particles per microlens (ppm)

Co

mp

uta

tio

na

lti

me

rati

o(T

MA

RT/T

DR

T-M

AR

T)

1 2 3 4123456789

101112131415

(c)


Further away from the focal plane, either in the far field or near field, the reconstruction quality

decreases for both algorithms. However, there are discernible local minimums for the DRT-

MART algorithm. For instance, at z =-1.078mm the reconstructed intensity expands to even

larger amount of voxels. This is due to the fact that, further away from the focal plane, when

there is a small change in particle’s z location, the affected micro-lens units remain the same with

only a slight change in the distribution of surrounding pixels’ intensity. This fractional pixel

intensity variation is normally difficult to be resolved by reconstruction methods. The

improvement in computation efficiency is very promising, for example, when the particle

density is 1ppm which is the density used for the following simulation studies, DRT-MART is 10

times faster than the MART method as shown in Fig. 9c.

4. Comparison between LF-PIV and Tomo-PIV

In this section, a pair of synthetic light field images is generated by ray tracing simulation. The

simulated plenoptic camera has the same resolution as our in-house developed camera (Shi et al

2016). The first synthetic light field image is generated by randomly scatter tracer particles in a

volume of 36×24×20mm with a particle density of 1 particle per microlens (1ppm). A known

velocity field is imposed on the particles, and new locations of the particles is determined by

giving a short time interval of ∆t=0.5ms. Flow field of a jet issued from a D=6mm circular nozzle

at Re=2500 is simulated by DNS and serves as the exact velocity field. Three dimensional particle

images are reconstructed from these two synthetic light field images by the DRT-MART

algorithm with a reconstruction resolution of 2100×1400×260 voxel. A three dimensional version

of multi-grid cross correlation is then used to calculated the raw instantaneous velocity volume

with 50% overlap and an initial and final interrogation window size of 128×128×32 and

64×64×16, respectively. The raw velocity volume is then further processed by median filter and

linear interpolation to identify and replace any incorrect velocity vectors. Consider very

intensive calculations involved in the reconstruction and cross correlation steps, GPU parallel

processing (Geforce980) is applied to the to improve the computational efficiency.

On the other hand, two sets of synthetic Tomo-PIV particle images are generated for comparison.

To do that, camera calibration matrix is calculated for four Imperx B2041 cameras by using a 110

× 110 mm calibration board and pinhole camera model. A magnification factor of 0.074

mm/pixel is used for capturing the calibration board images to ensure a similar measurement

resolution as LF-PIV. The first four Tomo-PIV images are generated by projecting a group of

randomly scattered particles to four cameras by using the camera calibration matrix with a

particle image density of 0.05ppp. The DNS jet flow is then imposed on the particles in a similar


fashion as LF-PIV to generate the second four Tomo-PIV images. Three dimensional particle

images are reconstructed by the MLOS-MART method and then process by multi-grid cross

correlation with 75% overlap and an initial and final interrogation window size of 64×64×64 and

32×32×32, respectively. Median filter and linear interpolation are also applied to smooth the raw

velocity volume. GPU parallel computation is also applied to Tomo-PIV process to increase the

efficiency. The DNS velocity field and calculation results from LF-PIV and Tomo-PIV is

presented in Fig. 9. The overall flow structure measured by LF-PIV matches generally well with

the DNS data and Tomo-PIV result, which proves the validity of the proposed DRT-MART

method. However, there are discernable differences in velocity contour between the LF-PIV and

Tomo-PIV results, which is due to the inhomogeneous voxel intensity distribution along the z-

axis. Further analysis is underway to normalize the voxel intensity after it is reconstructed by the

DRT-MART method.

5. Conclusion

This paper conducted a detailed simulation of the effect of pixel-microlens-ratio on plenoptic

camera resolution, showing this factor greatly affect the planar and spatial resolution. Based on

such ray tracing analysis method, a dense ray tracing based reconstruction algorithm namely

DRT-MART is proposed to improve the computational efficiency. The performance of DRT-

MART is firstly studied by using a series of synthetic light field particle images, showing that it

is capable of reconstructing particle image at a higher accuracy than MART but with lower

computational cost. Finally, a set of synthetic jet flow light field images is used to evaluate the

overall performance of LF-PIV. Preliminary studies show that the DRT-MART based LF-PIV

technique can provide satisfactory results when compared with the traditional Tomo-PIV

method.

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Grant No.

11472175) and the Shanghai Raising Star Program (Grant No. 15QA1402400).


Fig. 10. Instantaneous velocity volume for (a) DNS results, (b) LF-PIV measurment and (c)

Tomo-PIV measurement

(a)

(c) (b)


References

Arroyo M, Greated C A (1991) Stereoscopic particle image velocimetry. Meas Sci Technol 2:

1181–1186

Atkinson, C.H., Soria, J (2009) An efficient simultaneous reconstruction technique for

tomographic particle image velocimetry, Exp Fluid 47: 553-568.

Brucker C (1996) 3-D Scanning-Particle-Image-Velocimetry: Technique and Application to a

Spherical Cap Wake Flow. Applied Scientific Re-search 56: 157-179

Belden J, Truscott T, Axiak M, Techet A (2010) Three-dimensional synthetic aperture particle

image velocimetry. Meas Sci Technol 21: 1-21

Ding J, Wang J, Liu Y, Shi S (2015) Dense Ray Tracing Based Reconstruction Algorithm for Light

Field Volumetric Particle Image Velocimetry. 7th Australian Conference on Laser Diagnostics in

Fluid Mechanics and Combustion. Melbourne, Australia

Elsinga G, Scarano F, Wieneke B, van Oudheusden B (2006) Tomographic particle image

velocimetry. Exp Fluid 41: 933–947

Fahringer T, Lynch K, Thurow B (2015) Volumetric particle image velocimetry with a single

plenoptic camera. Meas Sci Technol 26: 115201

Georgiev T, Intwala C (2003) Light-field camera design for integral view photography. Adobe

Tech Report.

Georgiev T, Zheng K, Curless B, Salesin D, Nayar S, and Intwala C (2006) Spatio-Angular

Resolution Tradeoff in Integral Photography. Eurographics Symposium on Rendering

Hori T, Sakakibara J (2004) High-speed scanning stereoscopic PIV for 3D vorticity measurement

in liquids. Meas Sci Technol 15: 1067-1078

Hinsch K (2002) Holographic particle image velocimetry. Meas Sci Technol 13: R61–R72

Arroyo M, Hinsch K (2008) Recent developments of PIV towards 3Dmeasurements. In: Particle

Image Velocimetry: New Develop-ments and Recent Applications. Springer, New York

Katz J, Sheng J (2010) Applications of holography in fluid mechanics and particle dynamics.

Annual Review of Fluid Mechanics 42: 531–555

Ng, R. (2006) Digital Light Field Photography, Ph.D. thesis, Stanford University, California

Prasad A, Adrian R (1993) Stereoscopic particle image velocimetry applied to liquid flows. Exp

Fluid 15: 49–60

Pereira F, Gharib M, Dabiri D, Modarress M (2000) Defocusing PIV: a three-component 3-D PIV

measurement technique application to bubbly flows. Exp Fluid 29: S78–84

Scarano F (2013) Tomographic PIV: principles and practice. Meas Sci Technol 24: 1-28


Shi S, Wang J, Ding J, Zhao Z, New TH. Parametric study on light field volumetric particle image

velocimetry. Flow Measurement and Instrumentation, 2016, 49: 70-88

Willert C, Gharib M (1992) Three-dimensional particle imaging with a single camera. Exp Fluid

12: 353-358

Dense Ray Tracing Based Reconstruction Algorithm for Light Field PIV and Comparative...

Documents

Transcript of Dense Ray Tracing Based Reconstruction Algorithm for Light Field PIV and Comparative...