4Pi microscopy deconvolution with a variable point · PDF file4Pi microscopy deconvolution...

4Pi microscopy deconvolution with a variablepoint-spread function

David Baddeley, Christian Carl, and Christoph Cremer

To remove the axial sidelobes from 4Pi images, deconvolution forms an integral part of 4Pi microscopy.As a result of its high axial resolution, the 4Pi point spread function (PSF) is particularly susceptible toimperfect optical conditions within the sample. This is typically observed as a shift in the position of themaxima under the PSF envelope. A significantly varying phase shift renders deconvolution proceduresbased on a spatially invariant PSF essentially useless. We present a technique for computing the forwardtransformation in the case of a varying phase at a computational expense of the same order of magnitudeas that of the shift invariant case, a method for the estimation of PSF phase from an acquired image, anda deconvolution procedure built on these techniques. © 2006 Optical Society of America

OCIS codes: 180.2520, 180.1790, 100.1830, 100.3190.

1. Introduction

4Pi microscopy is a form of confocal light microscopythat uses interference between the foci of two oppos-ing objective lenses to obtain an imaging resolutionsignificantly better than that obtainable with a con-ventional microscope.1–4 The method is slowly reach-ing maturity, and a commercial instrument is nowavailable from Leica Microsystems Incorporated. Thecommercial variant is of type A (only the excitation isbrought to interference) and utilizes two-photon ex-citation. The focus of this paper is thus on two-photontype A 4Pi microscopy. A key feature of 4Pi microscopyis the requirement for postprocessing to remove side-lobes present in the images, and to obtain the fullimaging resolution. Various methods exist for sideloberemoval, ranging from a simple three-point linear fil-ter to fully fledged maximum-likelihood (ML) deconvo-lution algorithms.5 The various methods have theircorresponding pros and cons: the three or five pointfilters are fast, stable, and do not require a highlyaccurate point spread function (PSF) measurement.The filters, however, exhibit a small amount of noiseamplification (or, strictly speaking, a reduction in theeffective signal to noise ratio), and do not offer the

advantages of ML algorithms such as noise reductionor the same degree of resolution enhancement. TheML algorithms in turn suffer from being slow, re-source hungry, susceptible to poor PSF measure-ments, and subject to problems related to stabilityand convergence. It is thus possible to obtain a muchhigher level of noise amplification and artifact gen-eration from the ML algorithms than from the linearfilters. With an accurate PSF and a suitable choice ofregularization, artifact generation in ML algorithmscan be minimized, thus justifying our use of a MLtechnique in this paper. Pure destructive interferenceis also incapable of being solved by using a three- orfive-point deconvolution (the appropriate matrix issingular). ML techniques get around this by provid-ing additional information.

One area in which all current algorithms havedifficulties is that in which the PSF depends on theposition in the sample. The major PSF variationobserved in 4Pi microscopy is a phase shift in theunderlying interference pattern that is caused bya difference in the optical path length from eachobjective.6,7 This is manifested by a shift from con-structive to destructive (or anywhere between) inter-ference at the center of the PSF, as shown in Fig. 1.When constant over the whole specimen, this caneasily be corrected with the phase adjustment piezoor by using a suitable PSF measurement with a con-ventional ML algorithm. When this varies through-out the object, however, it is impossible to correct forby using the existing algorithms. Such a variation iscommonly seen in biological specimens in which pre-cise control of the refractive index inside the cell is

The authors are with the Kirchhoff Institut für Physik,Universität Heidelberg. D. Baddeley’s e-mail address is [email protected].

Received 21 December 2005; accepted 14 March 2006; posted 10May 2006 (Doc. ID 66736).

0003-6935/06/277056-09$15.00/0© 2006 Optical Society of America

7056 APPLIED OPTICS � Vol. 45, No. 27 � 20 September 2006

difficult. Various hardware solutions to this problemhave been presented,7–9 involving continuous com-pensation of the phase during measurements andmonitoring both outputs of the beam-splitter prism.Neither of these options exists for the commerciallyavailable device. In this paper we present a method ofsimulating images with such a spatially varyingphase shift, a method of estimating the phase shiftfrom acquired images, and a technique for the re-construction of experimental images based on thesemethods.

2. Phase Shifted 4Pi Point-Spread Function

Most deconvolution methods require a computation-ally inexpensive method of computing the forwardtransformation, d � Hf (the mapping from the vectorof object voxels, f, to the data voxels, d), and also inseveral cases the conjugate transpose of this map-ping. In the case of a spatially invariant PSF this is aconvolution and can be inexpensively performed byusing Fourier domain multiplication. When the PSFis not spatially invariant, as is the case with a 4PiPSF in nonideal optical conditions, then this ap-proach is not applicable. In the special case of a phaseshift in the interference maxima of the 4Pi PSF, theforward transform, and its conjugate transpose, canbe approximated as the linear combination of a smallnumber of convolutions. Hereafter we will concen-trate on two-photon 4Pi type A, as this the type of 4Pimicroscope that is currently commercially available,and as the inverse problem is not even moderatelywell posed for the one-photon variants. However, theprinciples are applicable to all 4Pi variants.

The 4Pi PSF can be described as an envelope cor-responding to the PSF of a confocal microscope oper-ating at the respective excitation and emissionwavelengths Ienv�x, y, z� � IdetIex or in the case of two-photon excitation IdetIex

2, multiplied by the interfer-ence pattern produced by the coherent addition of

light from both objectives Ifringes�x, y, z� giving

IPSF � Ienv�x, y, z�Ifringes�x, y, z�. (1)

The same general form applies to 4Pi type B whenIfringes is regarded as the projection of the detectedlight interference into the object space, and to type Cwhen the combination of the two interference pat-terns is used.

These interference fringes correspond to the wave-fronts of the excitation beam, which are plane in thefocus of a Gaussian beam and acquire curvature atincreasing distance on either side. If the envelope, Ienv,is sufficiently small, such that the detected intensityis very low before the curvature of the fringes be-comes noticeable, it is possible to approximate thefringes as a plane-wave interference pattern havingonly a z dependence, Ifringes��z�. This approximationforms the basis of the three- and five-point deconvo-lution schemes extensively used in 4Pi microscopy,and is a good approximation in the two-photon case.For plane-wave interference and one-photon excita-tion, the normalized fringe pattern is expected to beIfringes1photon

��z� � cos2��2�nz��ex� � �� where n is therefractive index, �ex the excitation wavelength, and �a phase term. In the case of two-photon excitation, thedetected intensity is proportional to the square of theexcitation giving Ifringes��z� � cos4��2�nz��ex� � ��. Inthe general case, and in samples without ideal opticalconditions, � cannot be thought of as a constant andmust be regarded as a function, ��r�, of the positionr � �x, y, z� in the object space. Substituting Ifringes inEq. (1) gives us an approximate expression for thetwo-photon 4Pi PSF in the presence of an arbitrary,spatially varying, phase shift

IPSF � Ienv�r�cos4�2�nz�ex

� ��r��. (2)

The mapping �d � Hf � responsible for the imagingprocess can be expressed as

d�r� ��

�

f�s�IPSF�r � s, r�dsxdsydsz, (3)

where f(r) is the fluorophore distribution in the sam-ple and d(r) the detected intensity. The dual param-eterization of the PSF IPSF�r � s, r� reflects its spatialvariance. Owing to the dependence of � on the posi-tion in object space this mapping cannot be expressedas a convolution and simply performed. Expandingthe cos4 term in Eq. (2) gives us

IPSF � Ienv

14 �3

2 � 2 cos�2��cos4�nz�ex

�� 2 sin�2��sin4�nz

�ex��

12 cos�4��cos8�nz

�ex�

�12 sin�4��sin8�nz

�ex��. (4)

Fig. 1. (Color online) Schematic representation of a z-axis profilethrough a two photon 4Pi A PSF showing the effect of a varyingphase. This is seen as a shift of the interference fringes withrespect to the maximum of the envelope.

20 September 2006 � Vol. 45, No. 27 � APPLIED OPTICS 7057

Using the linearity property of integration we cannow approximate the imaging through

d �14 3

2 Ienv � f � 2 cos�2��cos4�nz�ex

�Ienv� � f � 2 sin�2��sin4�nz

�ex�Ienv� � f

�12 cos�4��cos8�nz

�ex�Ienv� � f

�12 sin�4��sin8�nz

�ex�Ienv� � f �, (5)

where � represents convolution. The five integralsobtained when substituting Eq. (4) into Eq. (3) allinvolve a spatially invariant PSF term, and can thusbe expressed as convolutions. These convolutions areeasily calculated, for instance by using the Fourierspace multiplication technique. Similar reasoningcan be applied to the conjugate transpose of the trans-formation.

The concept is easily extended to the 4Pi C variantsthrough the introduction of an additional sinusoidalterm in Ifringes� and a second phase map, which isrelated to the original through a scaling and offset.

3. Phase Estimation

To be able to perform a reconstruction based on ourvariable phase PSF model, as outlined in Section 2,we require knowledge of the PSF phase at every pointin the image. It is possible to obtain informationabout the phase during image aquisition by usingmethods such as those presented by Hell et al.8 andBlanca et al.9 Unfortunately, the hardware imple-mentation of the commercial 4Pi microscope does notallow these techniques to be used and one must at-tempt to extract this information from the resultingimages. Given a method for calculating the forwardtransformation that maps the fluorophore densityand PSF phase in the object space to an image it istheoretically possible to invert the problem and si-multaneously estimate the object and phase throughsome form of maximum-likelihood optimization. Asone must estimate two parameters for each datapoint and as the vast majority of voxels contain ab-solutely no information about the phase, being eitherdark or part of a larger structure, this problem is bothunderdetermined and poorly posed, and it is difficultto envisage a regularization technique capable of im-proving the conditioning enough to allow a ML algo-rithm to produce a sensible result. One alternative isto independently estimate the phase from the struc-tures in the image in which the phase information ispresent, such as reference beads or thin membranesaligned perpendicular to the optical axis, and inter-polate this over the whole image before deconvolvingby using this phase estimate.

The technique for phase extraction presented hereis one of several possible methods, but the require-ments for most methods will be similar: structures

that are axially small, separated from other struc-tures in the axial direction and having a good signal-to-noise ratio. Our method simply attempts tomeasure the distance between the central (or high-est) maximum, and the center of the envelope. In thecase of destructive interference, where both maximaare equally high, the choice of which distance to mea-sure becomes arbitrary. To be robust in the presenceof noise, the images are filtered before processing.When only the position of an object is important,optimal filtering is obtained by making the cross cor-relation of the image with the PSF. To extract thecenter of each object, we thus perform a cross corre-lation of the image, d with the envelope of the PSF,Ienv giving us an image denv. The position of the fringesis then extracted by using the cross correlation withcos�4�nz��ex�Ienv giving us dfringes, the cos�4�nz��ex�term from the expansion above being a sufficientlygood approximation for this purpose. The distancefrom the peaks of denv to the correspondingly nearestpeak in dfringes then gives us the phase at that point inthe image. Because cos�4�nz��ex�Ienv averages to zeroon a distance scale greater than a few periods of thecos term, the envelope of dfringes tends to zero overareas of the image with no high frequency compo-nents (corresponding to small structures) in the axialdirection. A threshold on the envelope of dfringes canthus tell us which parts of the image are interesting forphase extraction. However, this is not entirely suffi-cient, as fringes will also be visible at the edges ofextended structures, and where several small struc-tures are in close vicinity or periodically spaced. Whileit should theoretically be possible to estimate thephase from such structures, additional informationabout the structure would be required. To removestructures where the assumptions of negligible axialthickness and isolation from neighboring objects arenot fulfilled, we can examine the height of the maximaon either side of the central peak. In the case of anextended or periodic object, these will be elevated. Fig-ure 2 illustrates the application of phase estimationusing these principles to a line scan through the sim-ulated 2D test image shown in Fig. 5.

The method detailed above is limited in accuracy toone voxel. An alternative method of phase estimation,offering subvoxel phase resolution, extends the aboveby fitting a suitable model function to the data at eachreference point rather than simply looking for themaxima. One could also interpolate denv and dfringes inthe z direction to achieve the same result. In any case,to accurately estimate the phase over the entire imagethe values from the resulting measurement pointsmust be interpolated. As an alternative to the auto-matic selection of reference points, manual selectioncould be desirable in poor signal-to-noise conditions, orwhen speed and�or memory usage is an issue. In thiscase the judgment as to whether a structure is isolatedor not is based on the experience of the user.

In the examples presented here, a smoothly vary-ing phase is assumed. It could be advantageous tofurther develop the phase estimation procedure toallow discontinuities in the first derivative, such as


would occur at a refractive index step. Such a modelwould however, be likely to require a very large num-ber of control points, or additional information suchas could be obtained from differential interferencecontrast (DIC) images. It is in any case unlikely thatthe small errors introduced in the assumption ofsmoothness will be noticeable in the resulting im-ages, although they could be significant for subreso-lution distance measurements.

4. Deconvolution Algorithm

Conventional deconvolution algorithms for light mi-croscopy are optimized for either Gaussian or Poissonnoise, with algorithms such as Richardson–Lucy10 thatare based on Poisson noise being mathematicallysuperior for almost all cases of confocal imaging.Whether this is also the case for 4Pi images remains tobe established. The avalanche photodiodes used in thecommercial 4Pi have a very limited (approximately 3bits) dynamic range. This is due to the fact that theyact in photon-counting mode, combined with a limitedbandwidth and relatively short ��1 �s� per voxel in-tegration time. To overcome the limited dynamicrange, it is thus common practice in 4Pi microscopy tosignificantly oversample (�20 nm voxel size) and tosmooth the resulting images with a Gaussian filter.

This averaging will alter the noise characteristics,with the central limit theorem implying that thenoise should become more Gaussian.

The algorithm we have used solves a Tikhonov reg-ularized, weighted least-squares problem

arg f̂ min ��W(d � Hf̂ )�2 � �2�L(f̂ � fdef)�2� (6)

using a modified conjugate gradient (CG) solver.11–13

This attempts to find an estimate of the object �f̂�which minimizes both a misfit term, �W�d � Hf̂��2,describing how well the object matches the data,and a likelihood term �L�f̂ � fdef��2 that representssome prior knowledge (most often an assumption ofsmoothness) about the form of the object. The param-eter � determines the respective weighting of thedata misfit and likelihood terms. In contrast toordinary least squares, which is the solution inthe presence of uniform Gaussian noise, choosingWii 1��di, Wij|ij � 0 gives a scaled Gaussian noisemodel in which the noise amplitude scales with thesquare root of pixel intensity. This allows us to ap-proximate the Poisson noise, and might indeed bemore appropriate than a pure Poisson model in casesin which the signal is averaged. These weights are inpractice not particularly useful as they give infiniteweight to zero valued voxels in the image. A combi-nation of �n and uniform Gaussian noise, Wii 1�� di�, achieved by putting a constant � in the de-nominator, yields better results. For a likelihoodfunction, L, we use a 3 � 3 � 3 approximation to thesecond derivative, as this yields the qualitatively bestresults on both simulated and real images. The for-ward mapping (H) and its transpose are evaluated asindicated in Section 2. A positivity constraint is alsointroduced by clipping negative values to zero at eachiteration.

This deconvolution approach was chosen due to itsrobustness, fast convergence, the ability to improve theconditioning of the problem through regularization,and the availability of MATLAB code12 for the CGsolver. While the Richardson–Lucy algorithm is tosome extent regularizable by limiting the number ofiterations, or filtering the images between steps,14 it iseasier to guarantee convergence with the Tikhonov–least-squares approach.

To test the proposed methods for image generation,phase estimation, and image reconstruction, we haveapplied them both to a test pattern and to real im-ages.

A. Deconvolution of Simulated Data

The test pattern used was the 2D arrangement ofgeometrical shapes shown in Fig. 3 and its corre-sponding phase map (Fig. 4). Note that the phaseshift applied in this case is fairly extreme—shiftingfrom constructive to destructive interference andback to constructive again as one moves down the zaxis (vertically). A 2D image was used here as thecomputation is quicker, and the results are easier to

Fig. 2. Line scan along z axis through simulated test image show-ing the principles of the phase estimation procedure. A cross cor-relation is performed between (A) the raw data and the PSFenvelope (i.e., the appropriate confocal PSF) giving (B) the profiledenv. From the peaks in this image, the centers of all objects areestimated (as shown by the vertical lines). The raw data are ad-ditionally correlated with Ienv cos�4�nz��ex� to extract the fringeposition [as shown in profile (C)]. To verify that the reference pointbelongs to an isolated object, the relative heights of the peaks in thelow pass filtered image (D) are analyzed. In the displayed case, thesecond object from the left would be rejected. Note that the crosscorrelations�filtering were carried out on the xz image, resulting ina better signal to noise ratio than would be expected from the 1Dfiltering of the data shown.


display and interpret than in the 3D case. The al-gorithm is however, the same and is easily gener-alized to 3D, as will be shown with the biologicalresults. An image, shown in Fig. 5, was obtained byapplying the procedure in Section 2 to the test im-age by using the phase map and an approximationto the theoretical PSF envelope, and passing the resultthrough a Poisson noise process. The PSF envelopewas approximated with a sinc2�9z�5w�sinc4�z�w� axialcomponent, the width of which was chosen so as togive a sidelobe height of approximately 50 percent,multiplied with a Gaussian lateral component. Notethat the sinc2 and sinc4 in the axial component cor-respond to detection and two photon excitation, re-spectively. The use of sinc functions ensures that theaxial PSF remains band limited and avoids any arti-ficial resolution improvement. The width of the com-ponent was chosen so as to give a sidelobe height ofapproximately 50%, multiplied with a Gaussian lat-eral component. The simulation was performed for anexcitation wavelength of 800 nm, a voxel size of 10 nmin the axial direction and 100 nm laterally, and thenoise was applied to give a expected photon count of30 in the brightest voxel of the image. Two counts ofbackground noise were also added to the entire im-age.

Applying the phase extraction procedure describedin Section 3 gives the reconstructed phase shown inFig. 6. The automatically selected control points are

shown as crosses on the image, and it can be clearlyseen that, while not completely perfect, the recon-struction provides a good estimate of the originalphase map. The discrepancies that are seen comefrom a variety of factors including interference fromnearby structures, noise, and the interpolation pro-cedure used (the nearest neighbour interpolation fol-lowed by Gaussian smoothing, the width of which,� � 700 nm, was chosen to ensure a smooth phasechange between the reference points). It can be seenfrom the image reconstruction in Fig. 7 that it is stillpossible to obtain an accurate result despite thesediscrepancies. Figure 7 shows the effective phasecompensation, sidelobe removal, and noise reductionoffered by the restoration procedure. A detailed com-parison is afforded by the line profile taken along thez axis close to the center �x � 0.5 �m� of the imagethrough the test object, simulated data, and the de-convolution result that is shown in Fig. 8. The de-convolution was performed both unweighted (i.e.,uniform Gaussian noise) and with intensity depen-dent weights. As there was no appreciable differencein the quality of the resulting images, and as theexact noise characteristics of the 4Pi images are un-clear, the results presented here use an unweightedsolution, i.e., a uniform Gaussian noise model. Toquantify the improvement obtained through phase

Fig. 4. Simulated phase map.Fig. 6. (Color online) Reconstructed phase map showing automat-ically selected control points (crosses).

Fig. 3. Simulated test pattern. Fig. 5. Two photon 4Pi A image generated from test pattern andphase map with Poisson noise.


compensation, we have compared our test object withthe simulated data and deconvolution results bothwith and without phase compensation by performinga cross correlation between each of the these imagesand our original test pattern. The results of thesecomparisons are shown in Table 1. The correlationvalue shows how similar each image is to the originalobject, with a value of 1 indicating 100% agreement.It is clear that phase-compensating deconvolution of-fers a significantly better match than either the dataor the deconvolution performed without phase com-pensation.

B. Reconstruction of Biological Images

To demonstrate the procedure on biological data, wehave applied the procedure to a 3D image stack ac-quired with the 4Pi microscope from a specimen inwhich one class of potassium channels in the cellmembrane was labeled through the expression ofKir2.1-eYFP (see Appendix A for the details ofthe preparation and acquisition parameters). The de-convolution was performed on the smoothed data

(Gaussian blur, � � 1 voxel), using a theoretical PSFenvelope, and a phase map interpolated from 20 man-ually selected control points on the cell membrane.The PSF simulation was based on vectorial theory andassumed unpolarized light. The parameter selectionwas based on the experimental wavelength, voxel size,and pinhole settings, and a refractive index of 1.46. Toobtain the correct sidelobe height, however, a reducedNA of 1.1 was required. Such a large reduction in NAseems unlikely; one possibility would be a misalign-ment of the two objectives. Interpolation of the phasemap was performed by fitting a linear function in z tothe control points. Figures 9, 10, and 11 show an xzsection through the stacks corresponding to the rawdata, the result of normal ML deconvolution (using ouralgorithm with the phase set to zero over the wholeimage), and the result of ML deconvolution when ap-plying phase correction. In both cases the number ofiterations was 20. Figure 12 shows a line profile takenvertically through the center of the images. The asym-metry in the PSF, and the refractive index mismatchinduced phase change through the image are clearlyvisible. The improvement to the deconvolution ob-

Fig. 8. (Color online) Line profiles through the test object, simu-lated image, and deconvolution result.

Fig. 9. xz slice at y � 2 through raw Kir 2.1 ion channel data afterGaussian blur for noise reduction. Control points for phase esti-mation were manually selected at points in three different y slices.The location of these control points is indicated on the image. Thedynamic range was compressed to improve contrast by saturatingthe bright spot.

Fig. 7. Reconstructed image.

Table 1. Correlation of Simulated Data and Reconstructions With andWithout Phase Compensation to the Test Pattern Used to Generate

the Simulated Dataa

Correlation With Object

Simulated data 0.5937ML - No phase compensation 0.7187ML - With phase compensation 0.9188

aThe simulated data are from Fig. 5 and the reconstructions fromFig. 7. Reconstructions without phase compensation are not shown.The test pattern is from Fig. 3. The values presented are at zeroshift, and the correlation coefficients are normalized so that auto-correlation gives a coefficient of 1.


tained through phase correction is easy to see. Theresidual sidelobes in the deconvolved image can prob-ably be attributed either to overregularization or to aninaccuracy in the estimation of the PSF Envelope. Nev-ertheless, they are certainly low enough to facilitate anunambiguous analysis of the images.

C. Performance of the Algorithm

In its current state, our algorithm is both relativelyslow and relatively resource hungry. On the test ma-chine (AMD AthlonXP 2500�, 1 GB RAM, MATLAB

R14, Windows 2000) the maximum data size that canbe computed in one go is approximately 2 106 voxels.On a stack of this size, 30 iterations take approxi-mately 45 min. The large memory usage can be ex-plained by the fact that floating point representationis used for all the data and that it is necessary to keeptrack of several working variables, including 10 (fiveforward and five transpose) part OTFs being the Fou-rier transforms of the terms in the LHS of each of theconvolutions in Eq. 5, each of which is the full size ofthe image. The memory usage is thus approximatelyfive times that of a similar deconvolution algorithmwith a spatially invariant PSF. The slow speed is dueto the relatively large number of calculations thatmust be performed at each step. This could be im-proved somewhat, at the expense of higher memoryusage, by precalculating the position dependentweights. The relatively large amount of computationinvolved in the forward (and transpose) transformsmakes the fast convergence of a CG solver particu-larly attractive (normally the computation of the for-ward transform is fast compared with the relativelylarge per step time required to calculate the newsearch direction in CG methods, and fast convergenceis partly offset by the fact that another algorithm, e.g.steepest descent, can take many more steps in thesame time).

To overcome the memory related limitations onimage size, it is possible to segment the image anddeconvolve each segment independently. To avoid theedge effect artifacts that would otherwise be present,the segments can be made to overlap and the edgeregions of each segment discarded. In theory, due tothe O�n ln n� complexity of the fast Fourier transformsused to compute the forward–transpose mappings,such segmentation should also offer a performancegain although this is likely to be offset by the extracomputation introduced by the segment overlaps. Itshould also be noted that a segmentation-based tech-nique lends itself well to parallelization, falling intothe class of embarrassingly parallel problems (thosethat can be split into many independent tasks with noneed for data exchange except at the beginning and

Fig. 10. xz slice at y � 2 through Kir 2.1 ion channel imagereconstructed with our ML algorithm and the phase set to zero overthe whole image. This represents what would be obtained from aconventional ML deconvolution algorithm. The dynamic range wascompressed to improve contrast by saturating the bright spot.

Fig. 11. xz slice at y � 2 through Kir 2.1 ion channel imagereconstructed with our ML algorithm and a phase map interpo-lated from 20 control points selected on the cell membrane. Thedynamic range was compressed to improve contrast by saturatingthe bright spot.

Fig. 12. (Color online) Line profiles taken vertically through theraw data and deconvolution results with and without phase correc-tion.


end of computation). We are currently developing analgorithm based on these principles, and preliminarytests show an execution time of �2 h on the machineabove for 10 iterations on a 512 370 256 �5 107 voxel image, and a linear decrease with anincreasing number of nodes (�40 minutes on a three-node cluster).

5. Conclusion

Techniques capable of compensating for phase shiftshould be of considerable advantage in the routinebiological use of 4Pi Microscopy. Despite considerableeffort it is not always possible to achieve a perfectrefractive index match. This is especially so for in vivomeasurements where the options for controlling theintracellular refractive index are particularly limited.An unfortunate characteristic of the current genera-tion of 4Pi microscopy techniques is a rather high rateof photobleaching. In normal measurements one wouldalign the PSF for constructive interference on a refer-ence bead or membrane near the object to be measuredbefore commencing the acquisition. For particularlyphotosensitive samples a “shoot first, ask questionslater” approach, such as is facilitated by an adaptivereconstruction algorithm, could present a decisive ad-vantage.

We have presented a method of image reconstruc-tion for 4Pi microscopy capable of compensating for thecommonly occurring problem of a position dependentphase shift in the 4Pi PSF. This technique is demon-strated, both on synthetic data and on real 4Pi micro-scopic images. The ability to separate the forwardmapping, which owing to its lack of shift invarianceposes a difficult problem for both image generation andreconstruction, into the spatially weighted sum of asmall number of shift invariant mappings represents aconsiderable simplification. Although only the two-photon 4Pi A case is addressed here, the concept caneasily be extended to the other 4Pi variants. The CGsolver used for the results presented in this paper ismost likely less than ideal in terms of performance,regularization method, and the underlying noisemodel. It is, however, more than adequate to demon-strate our concept for phase compensation, and deliv-ers results that are comparable to those of the leadingcommercial offerings in the absence of a phase shift.

Appendix A: Materials and Methods

1. Cell Biology

N-terminal fusion constructs of Kir 2.1 channel sub-units with enhanced yellow fluorescent protein (eYFP)were designed by inserting the respective cDNA inframe into the pEYFP-C1 eukaryotic expression vector(Clontech) by using the EcoRI and BamHI restrictionsites as described in Stockklausner et al.15 Humanembryonic kidney 293 cells were grown in minimumessentialmedium(Invitrogen),supplementedwith10%fetal calf serum and 1% penicillin�streptomycin at37 °C and 5% CO2. The cells were grown on cover-slides provided by Leica Microsystems. At 80% con-fluence, the cells were transfected with the respective

cDNA by using Fugene 6 (Roche) following thesupplier’s instructions.

2. Sample Preparation

The two coverslips specifically designed for use withthe Leica 4Pi Microscope were provided by Leica Mi-crosystems (Mannheim, Germany). Transfected cellswere fixed in 4% paraformaldehyde in phosphate-buffered saline for 20 min at 4 °C on one of thecoverslips. The fixed cells were embedded in 87%(volume) glycerol buffered with 50 mM Tris-HCL toobtain a pH � 7.4. On the other coverslip we immo-bilized green fluorescence beads (diameter 100 nm,G100, Duke Scientific) to implement reference objectsfor evaluation of the surface structure. For sealingthe coverslips we used nail polish and dental glue(Twin-sil A � B).

3. 4Pi Microscopy

The measurements presented here were performedon a commercial prototype 4Pi Microscope at LeicaMicrosystems Incorporated, Mannheim, Germany.This is a two-photon 4Pi microscope of type A (theexcitation foci are brought to interference, but theinterference of the detected light is not used). Two-photon excitation was provided by using a Ti:sap-phire laser producing picosecond pulses and tunablebetween 740 and 990 nm. A matched pair of glycerolimmersion objectives �100 , NA�1.35� were usedwith quartz coverslips and 87% glycerol as a mount-ing and immersion medium. A voxel size of 26.2 nmin x and z and 30.5 nm in y was chosen and an xzyscan performed resulting in a final image size of512 25 512 voxels. The pinhole and beam ex-pander were set to 1 Airy and 3, respectively, a lineaverage of 16 and an accumulation of 2 were used. Theexcitation wavelength was approximately 970 nm.

The authors thank S. M. Tan (Department of Phys-ics, University of Auckland) for the skeleton code of aconjugate-gradient solver, C. Karle and E. Zitron (De-partment of Cardiology, Medical University Hospital,Heidelberg, Germany) for providing the biologicalspecimen, Leica Microsystems Mannheim, wherethe measurements were carried out, and U. Birk forstimulating discussion and help with the proofread-ing. This work was supported by grants from theDeutsche Forschungsgemeinschaft and the state ofBaden-Württemberg.

References1. S. W. Hell, S. Lindek, C. Cremer, and E. H. K. Stelzer, “Mea-

surement of the 4Pi-confocal point spread function proves 75 nmaxial resolution,” Appl. Phys. Lett. 64, 1335–1337 (1994).

2. P. E. Hänninen, S. W. Hell, J. Salo, E. Soini, and C. Cremer,“Two-photon excitation 4Pi confocal microscope: Enhanced ax-ial resolution microscope for biological research,” Appl. Phys.Lett. 66, 1698–1700 (1995).

3. M. Nagorni and S. W. Hell, “Coherent use of opposing lenses foraxial resolution increase in fluorescence microscopy. I. Compar-ative study of concepts,” J. Opt. Soc. Am. A 18, 36–48 (2001).

4. M. Schrader, M. Kozubek, S. W. Hell, and T. Wilson, “Optical


transfer functions of 4Pi confocal microscopes: theory and ex-periment,” Opt. Lett. 22, 436–438 (1997).

5. M. Schrader, S. W. Hell, and H. T. M. van der Voort, “Three-dimensional superresolution with a 4Pi-confocal microscopeusing image restoration,” J. Appl. Phys. 84, 4033–4042 (1998).

6. A. Egner, M. Schrader, and S. W. Hell, “Refractive index mis-match induced intensity and phase variations in fluorescenceconfocal, multiphoton, and 4Pi-microscopy,” Opt. Commun. 153,211–217 (1998).

7. M. Schrader, K. Bahlmann, G. Giese, and S. W. Hell, “4Pi-Confocal imaging in fixed biological specimens,” Biophys. J. 75,1659–1668 (1998).

8. S. W. Hell, C. M. Blanca, and J. Bewersdorf, “Phase determina-tion in interference-based superresolving microscopes throughcritical frequency analysis,” Opt. Lett. 27, 888–890 (2002).

9. C. M. Blanca, J. Bewersdorf, and S. W. Hell, “Determinationof the unknown phase difference in 4Pi-confocal microscopythrough the image intensity,” Opt. Commun. 206, 281–285 (2002).

10. W. H. Richardson, “Bayesian-based iterative method of imagerestoration,” J. Opt. Soc. Am. 62, 55–59 (1972).

11. S. M. Tan, “Aperture synthesis mapping and parameter esti-mation,” Ph.D. dissertation (Mullard Radio Astronomy Obser-vatory, Cavendish Laboratory, Cambridge, 1987).

12. S. M. Tan, C. Fox, and G. K. Nicholls, “Physics 707 InverseProblems (Course Notes),” http://www.math.auckland.ac.nz/�phy707/.

13. R. Barrett, M. Berry, T. F. Chan, J. Demmel, J. Donato, J.Dongarra, V. Eijkhout, R. Pozo, C. Romine, and H. V. derVorst, Templates for the Solution of Linear Systems: BuildingBlocks for Iterative Methods, 2nd ed. (SIAM, 1994).

14. R. Heintzmann, “Resolution Enhancement of Biological LightMicroscopic Data,” Ph.D. dissertation (University of Heidel-berg 1999).

15. C. Stockklausner and N. Klöcker, “Surface expression of in-ward rectifier potassium channels is controlled by selectiveGolgi export,” J. Biol. Chem. 278, 17000–17005 (2003).


4Pi microscopy deconvolution with a variable point · PDF file4Pi microscopy deconvolution...

Documents

Transcript of 4Pi microscopy deconvolution with a variable point · PDF file4Pi microscopy deconvolution...