A review on :3-D reconstruction from confocal scanning microscopeimages: Deterministic and maximum likelihoodreconstructionWrittenby: Jose-AngelConchelloandEricW.Hansenreviewby: SaritShwartz11 IntroductionThepurposeofthisworkwastoimprovethreedimensionalimagesofbiologicalspecimensstainedwithuorescentprobestakenwithconfocal scanningmicroscope. Thearticlede-scribes twoapproachesaniterativedeterministicdeconvolutionandmaximumlikelihoodestimator.1.1 AcquiringtheimagesThree dimensional images are obtained from optical microscope by collecting a series of twodimensional imagesfocusedatdierentplanesthroughthespecimen. Eachoptical sectionshows theplaininfocus plus contributions fromout of focus plains, whichobscures thedesiredimage.Better images canbeobtainedbyusingaconfocal scanningmicroscope. Inconfocalscanning microscope only a small portion of the specimen is illuminated at a given time andlight from other points in the specimen is rejected by a small aperture in front of the detector,butthecontributionsfromoutoffocusplainsarereducedbutnotcompletelyeliminated.21.2 OperatorsandnotationsSymbol meaning(xD, yD, zD) Detector(image)position(xs, ys, zs) Scanposition(x, y, z) CoordinatesforapointinthespecimenfFluorescentlightwavelength Excitationlightwavelengthd Distanceoffocusplanefromthelenssin() Thenumericalapertureofthelenso() Objectintensityi() Recordedimagebrightnessh1() Thepointspreadfunctionofthelensatwavelengthh2() Thepointspreadfunctionofthelensatwavelengthfh() Thepointspreadfunctionofthemicroscopeif() TheuorescentintensitydistributionintheobjectspaceI() Theuorescentimageofif()throughthelensOTP() TheopticaltransferfunctionofthemicroscopeT1() OpticaltransferfunctionofthelensatwavelengthT2() Opticaltransferfunctionofthelensatwavelengthf(vx, vy, u) Opticalcoordinates(s, t, w) NormalizedfrequenciesTn1() OpticaltransferfunctionofthelensatwavelengthinopticalcoordinatesTn2() Opticaltransferfunctionofthelensatwavelengthfinopticalcoordinateshn1() Thepointspreadfunctionofthelensatwavelengthinopticalcoordinateshn2() Thepointspreadfunctionofthelensatwavelengthfinopticalcoordinatesi(k)(x, y, z) Imageofthereconstructedimageatiterationk o(k)(x, y, z) Thereconstructedimageatiterationk Arbitraryconvergencecriterion(x, y, z) Thestepsizeofiterationkforpoint(x,y,z)intheobjectA Halfofthemaximumallowedvalueoftheobjectp() ProbabilitydensityY IncompletedataspaceZ Completedataspacef: Z Y Mappingbetweenincompletedataspaceandcompletedataspace Parametervectortobeestimated Parametersspacehat(k)Parametervectorestimationiniterationk3L LoglikelihoodLidIncompleteloglikelihoodLcdCompleteloglikelihoodQ(o(1), o(2), ...|o(1)(k),o(2)(k), ...) Conditionalexpectationofthecompleteloglikelihood ArbitrarylearningrateconstantXltArbitraryparameterscharacterizingcorrelationofneighboringsourceelementsintheobjectaroundelementl o(l) MeanstrengthvalueofvoxellintheobjectlStandarddeviationofsourceelementlintheobject Convolutionoperator2 Optical transferfunctionandpointspreadfunctionofaconfocalscanninguorescentmicroscope2.1 OpticalsystemTheimagingpropertyof aconfocal scanningmicroscopethatusesinduceduorescenceisdierent fromthat of one that uses transmittedor reectedlight. This is because theuorescenceisincoherent. Thusthebandwidthof itsoptical transferfunctionistwiceaslargeasthatof aconfocal coherentmicroscope.Moreover, themissingconeintheopticaltransferfunctionforthismicroscopeissmaller, andif theexcitationandtheuorescencewavelengthisclose,themissingconedisappear.There are two possible congurations for the optical system. Simplied diagrams of bothareshowninFig. 1. Theoptical transferfunctioncalculationsareidentical forboth, somathematicalanalysiswillbegivenonlyfortheonelensconguration.Excitationlightwithwavelengththatcomesfromapointsourceisreectedbyhalfmirrorandfocusedbythelensatadistancedfromthelens. Thepointinthespecimenspaceisdenotedbycoordinatesx, yandz. Thedirectionof thezaxisisontheopticalaxis. Thedistributionofauorescentobjectmaybeexpressedaso(x xs, y ys, z zs).4Figure1: Confocal optical pathconguration-Up:onelensconguration. Down:twolenscongurationCoordinates xs, ys and zs represents the scan position. The image of the induced uorescenceisformedthroughthefocusinglens, thehalf mirror, andacutlter. CoordinatesxD, yDandzDdenotethepositionoftheimage. Thecutlterisinsertedintotheopticalpathtoblocktheexcitationlight. forsimplicityassumethatonlyuorescencewithwavelengthfis allowed to pass through the lter. and that the a point detector is positioned at the originofthecoordinatesystemoftheimage.52.2 CalculationofOpticaltransferfunctionFollowingShigeharuandChusuke[1], letacomplexamplitudeof thepointsourceimagereectedthroughthefocusinglens at awavelengthof beh1(x, y, z), andlet thesameamplitudeatauorescencewavelengthof fbeh2(x, y, z). Assumethattheinducedu-orescenceatsomeobjectpointisproportional tothedistributiono(x xs, y ys, z zs)and to the incidence intensity of the excitation light at the same point. then the uorescentintensitydistributionif(x, y, z),intheobjectspaceis:if(x, y, z) = |h1(x, y, z)|2o(x xs, y ys, z zs). (1)Thereforetheuorescentimageofthisdistributionthroughthefocusinglensis:I(xD, yD, zD, xs, ys, zs) =___h2_xDM x, yDM y, zDM z_2|h1(x, y, z)|2o(x xs, y ys, z zs) dxdy dz(2)where M is the magnication of the system. the point detector is placed at the origin of theimagecoordinatesystem, andh2isanevenfunctionthereforethedetecteduorescenceisexpressedasI(0, 0, 0, xs, ys, zs) =___|h2(x, y, z)|2|h1(x, y, z)|2o(x xs, y ys, z zs) dxdy dz (3)fromEq.3,thepointspreadfunctionpsf(x, y, z)forthemicroscopeisexpressedaspsf(x, y, z) = h(x, y, z) = |h2(x, y, z)h1(x, y, z)|2(4)The3-Doptical transferfunctioncanbecalculatedwiththe3-Dfouriertransformof thepointspreadfunctionfromEq.4.OTF(fx, fy, fz) =___|h2(x, y, z)|2|h1(x, y, z)|2exp(2i(xfx + yfy + zfz)) dxdy dz=_[T1(fx, fy, z) T2(fx, fy, z)]exp(2izfz) dz(5)6wherefx, fyandfzarespatial frequenciesinthex, yandzdirections. and denotesaconvolution. The functions T1and T2are 2-D optical transfer function with defocus and areexpressedasT1(fx, fy, z) =__|h1(x, y, z)|2exp(2i(xfx + yfy)) dxdy,T2(fx, fy, z) =__|h2(x, y, z)|2exp(2i(xfx + yfy)) dxdy.(6)Byintroducingtheopticalcoordinatesvx=2xsin() 2adx, vy=2y sin() 2ady,u =2z sin2() 2_ad_2z.(7)Whereistheexcitationlightwavelength,sin()isthenumericalaperture,aistheradiusoftheexitpupilofthefocusinglens,assumingtherefractiveindexn=1,andintroducingthenormalizedfrequenciess =sin()fx, t =sin()fy, w =sin2()fz. (8)The2-DopticaltransferfunctionrepresentedbyT1canbereplacedwithTn1(s, t, u) =__|hn1(vx, vy, u)|2exp(i(uxs + vyt)) dvxdvy(9)Wherehn1isthecomplexamplitudeofthepointsourcerepresentedintheoptical coordi-nates. Hopkins [2] calculated Tn1(s, 0, u) analytically. However, his approach involved a seriesofBesselfunctionsconvergesslowlyanditistimeconsuming. LaterStokseth[3]presentedanempiricallyderivedanalyticapproximationoftheoptical transferfunctiontomakethecalculationeasy. Accordingtothisapproximation,Tn1(s, 0, u)cantakethefollowingformTn1(s, 0, u) =_g1(s)_J1[ug2(s)]ug2(s)_0 < s < 20 2 s(10)7whereg1(s) = 2(1 0.69s + 0.0076s2+ 0.043s3),g2(s) = s 0.5s2(11)andJ1isaBesselfunctionoftherstkindoforder1.Insimilar waythe2-Doptical transfer functionat theuorescencewavelengthTn2isrepresentedintheoptical coordinateswithfastheuorescencewavelengthandhn2thecomplexamplitude. expressingf= leadstohn2(vw, vy, u) = hn1_vx , vy , u_(12)HencefromthesimilaritytheoremoftheFouriertransformwegettherelationshipbetweenTn1andTn2.Tn2(s, t, u) = Tn1_s, t, u_(13)ByapplyingEq. 9,12and13toEq. 5wecanwriteitasOTP(s, t, w) = F[hn1(vx, vy, u)|2] F[hn2(vx, vy, u)|2] (14)inwhichthesymbol Fdenotesa3-Dfouriertransformation. Eq. 14showsthatthe3-Doptical transfer function for the confocal uorescent microscope is represented by the convo-lutionofincoherentwideeldmicroscope3-Dopticaltransferfunctionsatthewavelengthsof theexcitationanduorescencelight. theincoherentwideeldmicroscope3-Dopticaltransferfunctionsitselfisaconvolutionofthecoherent3-Dopticaltransferfunctions.2.3 TheopticaltransferfunctionbandwidthFrom Eq. 14 we can see qualitatively how the bandwidth change for the 3-D optical transferfunction. Thebandwidthof wideeldmicroscopeoptical transferfunctionbecomesnar-rowerasthewavelengthoftheuorescencelightbecomeslonger. Whenthewavelengthoftheuorescenceisequaltotheexcitationwavelength,theconfocalopticaltransferfunction8bandwidthis twice as wide as the wide eldmicroscope bandwidth. Whenthe uores-cencewavelengthbecomeslonger,thebandwidthapproachthebandwidthofthewideeldmicroscope. inthatcaseamissingconeregionappears.3 AdditiveDeconvolutionThereconstructionis carriedout byconstrainediterativedeconvolution, apurelydeter-ministicalgorithmwhichisamodicationtotheJansson-van-Cittertmethodofsuccessiveconvolution[4]. Thecorrectionstepsizeforeachvoxelateachiterationisdeterminedbyaquadraticform. Thisquadraticformhasmaximumathalfthemaximumvalueallowedfortheobject(x, y, z)=1ifoold(x, y, z)=A, andminimumatthemaximumvalueallowedfor the object and at zero (x, y, z) = 0 if oold(x, y, z) = 2A or 0. The quadratic form of thestepsizeisdisablingdrasticcorrectionsatthesaturationandcutoregionsof theimage,weresuchcorrectionscouldspoiltheestimation.3.1 Algorithm1.i(k)(x, y, z) = o(k)(x, y, z) h(x, y, z)2. o(k+1)(x, y, z) = o(k)(x, y, z) + (x, y, z)[i(x) i(k)(x)]with(x, y, z) = 1 _ o(k)(x,y,z)AA_23. Applyconstraints.4. k=k+1.5. Repeat14untilmax(|i(x, y, z) i(k)(x, y, z)|) < where o(k)(x, y, z)isthereconstructedobjectofiterationk, i(k)(x, y, z)istheimageofthereconstructedobjectatiterationkandh(x, y, z)isthepointspreadfunction. Aishalfofthe maximum allowed value of the specimen or object,is an arbitrary convergence criterionand(x, y, z)isthestepsize. The3-Dconvolutionatstep1isimplementedbyFFT. Atstep3, priorknowledgeaboutthespecimencanbeincorporatedtothealgorithm. Inthis9articlesimulationtheonlyconstraintthatwasusedwasboundednessi.e. o(k)(x, y, z) ___0 if o(k)(x, y, z) < 02A if o(k)(x, y, z) > 2A o(k)(x, y, z) elseTherecordedimagei(x, y, z)isusedasaninitialguess.3.2 1Dand2DSimulationResultsThe simulations tested several several convergence criterions and several objects smooth,sharpbrightandsharpdim. Therstsimulationcheckedtheconvergenceasfunctionofthestop-ing condition. In Fig. 2 the system parameters (PSF, object) are showed. The results of thesimulationareshowedinFig. 3, 4and5. Wecanseefortheseguresthatthealgorithmisverysensitivetothestopingcondition. Therangeof goodvaluesisverysmall. If wewillchooseavaluetoolarge,theimprovementintheimagewillbeminorwereasifwewillchoose a value too small the algorithm wont converge at all. In addition the smalleris, orthemoreiterationdone, moreartifactswill beintroducedtotheestimation. Theartifactswillbeseenasbrightanddarkspotsorgridonthebackgroundoftheimage. ThiscanbeclearlyseeninFig. 5aslocalminimumsandmaximums.Thesecondsimulationcheckeddierent2DobjectsthatcanbeseeninFig. 6, 8and10,The algorithm results for the dierent objects are in Fig. 7, 9 and 11. for all the objects andfor = 10, 5thealgorithmdidntconvergebutwasstopafter100iterations. Forallobjectsthere is an improvement in the reconstructed image but also some artifacts were introducedto the background. In the sharp and dim object, although the object is less blurred it is stillnoteasilyseparablefromitsbackground.10Figure2: 1DsystemparametersforsharpobjectFigure3: 1Dadditivedeconvolutionresults = 5: convergenceaftertwoiterations.11Figure4: 1Dadditivedeconvolutionresults = 2: convergenceaftersixiterationsFigure5: 1Dadditivedeconvolutionresults=1: didnotconvergeandwasstoppedafter1000iterations12Figure6: 2DsystemparametersforsmoothobjectFigure 7: 2D additive deconvolution results for smooth object: did not converge for= 10, 5andwasstoppedafter100iterations13Figure8: 2DsystemparametersforsharpandbrightobjectFigure9: 2Dadditivedeconvolutionresultsforsharpandbrightobject: didnotconvergefor = 10, 5andwasstoppedafter100iterations14Figure10: 2DsystemparametersforsharpanddimobjectFigure 11: 2D additive deconvolution results for sharp and dim object: did not converge for = 10, 5andwasstoppedafter100iterations153.3 ArticleSimulationResultsThesimulationwasdoneonthreedierentobjects. acrossandacircle200nmapartandwith equal brightness, a cross and a circle 50 nm apart and with equal brightness and a crossand a circle 200 nm apart with the cross strength of 1/10 from the circle strength. The rstsimulationwasappliedtothecasewherethecrossandthecirclehaveequal strength, are200nmapartandwithoutnoise. TheresultofthissimulationareinFig. 12(left)fromthisgurewecanseethattheoutoffocuscontributionaregreatlyreduced,after10iterationsthe objects are bright in the slice where they are in focus, and dim only in the slices or planesimmediatelyaboveandbelow. Someringingartifactsareseenintheplanesimmediatelyaboveandbelowwheretheobjectisinfocus. inthesamegure(right)theresultsforthecasewhenthecrossandcircleare50nmapartareshown. wecanseethattheoutoffocuscontributionfromthetwoobjectsarereducedinonly10iterationtoonesliceaboveandbelowthelocationof thecircle. Thecontributiontothesetwoslicesdoesnottotallygoawayevenafter40iterations. Thecross,ontheotherhand,canonlybeseenatthesecondandthirdplanes.Figure13(left)showsthecasewherethecrosshave1/10theintensityofthecircle. intherecorded image, the contributions of the out of focus circle to the second panel outweigh theinfocuscontributionfromthecross. Astheiterationprogresses, therelativestrengthofthecrossincreases. Oneplaneabovethecross,thecontributionsfromthecirclearequicklyoutweighedbythoseofthecross. Althoughthealgorithmisnotabletoresolveclearlythetwo objects, it is capable of bringing out dim detail that could barely be seen in the recordedimage.Thealgorithmwastestedtheninthepresenceofsignal dependentpoissonnoise. thecasewherethe twoobjectsarewithequalintensity and200nm and50nm apart. forbothcasesthe SNR at the brightest region was set to 3.16(10 corresponding to the mean intensity of10pixelspervoxel). TheresultsfortherstcaseareonFig. 13(right). Thealgorithmwasabletoreducetheoutof focuscontributionstoonlytheoptical slicesimmediatelyaboveand below the two objects. However, at the same time, the noise is amplied as in evidencedbythesaltandpeppernoiseinthereconstructedimage.16TheresultsforthesecondcaseareshowninFig. 14Inonly10iterations,theoutoffocuscontributionsofthecirclearereducestotheslicesaboveandbelowtheplanewhereitisinfocus. Unlike the noiselesscase,the out offocuscontributions from the crossto the bottompanelstillshowsevenafter40iterations. Asinthepreviouscase,thehighfrequencynoisegrowsasiterationprogress.17Figure12: Additivealgorithmresults- withoutnoise. Left: simulatedcrossandcircleofequal intensity 200 nm apart, the optical slices where the cross and circle lays marked X andO respectively. the columns from left to right are the recorded image and the reconstructionafter 10,20,40 iterations. Right: simulated cross and circle of equal intensity 50 nm apart, theopticalslicewherethecirclelaysmarkedO.thecolumnsfromlefttorightaretherecordedimageandthereconstructionafter10,20,40iterations18Figure 13: Additive algorithmresults - withandwithout noise. Left: without noise -simulatedcrossandcircleof 200nmapartthecrossintensityis1/10thecircleintensity,theopticalsliceswherethecrossandcirclelaysmarkedXandOrespectively. thecolumnsfromlefttorightaretherecordedimageandthereconstructionafter10,20,40iterations.Right: withpoissonnoisemax. SNR=3.16simulatedcrossandcircleofequalintensity200nmapart, theoptical slicewherethecirclelaysmarkedO. thecolumnsfromlefttorightaretherecordedimageandthereconstructionafter10,20,40iterations19Figure 14: Additive algorithmresults - withandpoissonnoise max. SNR=3.16. Left:simulatedcrossandcircleofequal intensity50nmapart, theoptical slicewherethecirclelays marked O. the columns from left to right are the recorded image and the reconstructionafter10,20,40iterations.204 Likelihoodapproach4.1 LikelihoodfunctionWe assume that the specimen density is a poisson process with intensity o(x, y, z) because ofthe uorescence emission. The output image is also poisson process with intensity o(x, y, z)h(x, y, z). The problem is to estimate the intensity of the specimen density given the outputimage.Apoissondensitydenedasp(n, (x, y, z)) =n(x, y, z)e(x,y,z)n!(15)By dividing the specimen and image to voxelsl=1..N with las the poissonprocess densityinvoxell. Theprobabilityofobservingi(1), i(2), ..., i(N)intheimageisp(i(1), i(2), .., i(N) | o(1), o(2), ..., o(N)) =N

l=1_1i(l)!(o h)i(l)(l) e(oh)(l)_(16)Theaposterioridatalikelihoodprobabilityln[p(o(1), o(2), ...|i(1), i(2), ...)]isbyBayesruleL(o(1), o(2), ...)=ln[p(i(1), i(2), ...|o(1), o(2), ...)] + ln[p(o(1), o(2), ...)] ln[p(i(1), i(2), ...)].byomittingthetermsthatdontdependono(1), o(2), ...wegetL(o(1), o(2), ...) = Nl=1i(l)ln[(o h)(l)] Nl=1(o h)(l) + ln[p(o(1), o(2), ...)] (17)Po(1),o(2),...(o(1), o(2), ..., o(N)) the probability density of the object intensity is unknown butthis term enable us to insert constraints to the algorithm. this article dont specify the con-straintstheyusedbutapreviousarticle[5]usedapproximationforpo(1),o(2),...(o(1), o(2), ...)baseonneighboringpixelcorrelation.If the source element strengths o(l) are quantized hypothetically into indistinguishablestrengthunits (photons) andthe total number M=lo(l) of the strengthunits is as-sumed to be approximately xed, the source (object) distribution may be characterized as arandom process in which M indistinguishable strength units distribute randomly over the N21voxels,andeachvoxelhasaprobabilitypl(o(1), o(2), ...)ofhavingo(l)strengthunits. Thatmeanstheapriorisourceinformationfunctioncanbeexpressedasp(o(1), o(2), ...) =M!

lo(l)!N

l=1[pl(o(1), o(2), ...)]o(l)(18)pl(o(1), o(2), ...) is assumedtobe afunctionof o(l) andthe neighboringsource elementcorrelation. Thislocal correlationreectsthesourcestrengthcontinuitybehaviorandthephysicalinteractioncorrelationfromanelementtoitsneighboringelements.Let Xltbetheparameterscharacterizingthecorrelationofneighboringsourceelementsaroundelementl. Thelocalcorrelationfunctionpl(o(1), o(2), ...)canbeexpressedaspl(o(1), o(2), ...) =

texp_12Xltlt(o(l) o(l))(o(t) o(t))_(19)wheret {l }andcoverstheneighboringelementsof elementl. landtarethestandarddeviationof sourceelementsl andtrespectively, and o(l)isthemeanstrengthvalueofvoxell.The gaussian local correlation function in Eq. 19 incorporated the nearby source element con-tinuity information and the physical interaction correlation information into the parameters{Xlt} and the mean values { o(l)}. The gaussian parameters Xlt, o(l) and lneed to be chosenarbitrary,apossiblechoicecanbe2l= o(l), Xl,l2= 0.1, Xl,l1= 0.5, Xl,l= 1, Xl,l+1= 0.5,Xl,l+2= 0.1andthemeanvaluecanbeupdatedby o(t) = o(k)(t) +2(o(k)to(k1)t)whereisalearningrateconstant.For the minimization of the log likelihood we need to calculate the derivative of ln[p(o(1), o(2), ...)]sofromsubstitutingEq. 19toEq. 18andderivatingitwegetln[p(o(1), o(2), ...)]o(l)=(2o(l) o(l))2l

t12Xltt(o(t) o(t)) + Xll22l(o(l) o(l))(3o(l) o(l)) + ln[o(l)] +12l+ 1(20).224.2 ExpectationMaximizationThe expectation maximization algorithm is based on the concept of an incomplete data spaceY,acompletedataspace Zandamappingf: Z Y betweenthem[6].The incomplete data space is the space Yin which measured data takes its values. in thisproblem Yistheimageiandtheincompletedatalog-likelihoodisthelikelihoodfunctionwecalculatedintheprevioussubsection.The complete data space Zis a hypothetical space that is contrived to accomplish two goals:1) make the expectation and maximization steps of the algorithm analytically tractable. and2)maketheresultingcomputationsrequiredfortheEMalgorithmfeasiblefornumericallyproducingestimates. Thecompletedataspaceforagivenproblemisnotunique, andtheEMalgorithmcanbemoreorlesscomplicateddependingonthechoicemade.The complete data space is larger than the incomplete data space in the sense that completedatamust determineincompletedata. Theremust beaknownfunctionf() that mapscompletedatatoincompletedata. Thismappingfunctionandtheincompletedatayplaceaconstraintdeterminedbyf(z) = y,onthevaluesthatthecompletedatazmayhave.denotethecollectionofparameterstobeestimatedby,andassumethat Rn.Theparametersmaybedeterministicorrandom. Weusemaximumlikelihoodestimationfordeterministicparametersandmaximumaposteriori probabilityestimationforrandomparameters. for the optical sectioning problem the parameters to be estimated are the objectintensityo. andaccordingtotheprevioussectiontheyarerandomvariable, soweusethemaximumaposterioriestimation.Two log likelihood functions are important for estimating the parameters from measureddatay YwhentheEMalgorithmisused. Theincompletedataloglikelihood Lid()=ln[py(Y |)] +ln[p()]. The maximum a posteriori probability estimate (y) is the vectoroftheincompletedatasety Ythatmaximizestheincompletedataloglikelihood Lid().For theoptical sectioningproblemtheincompletelikelihoodfunctionis theonethat wecalculatedintheprevioussubsection.Thesecondloglikelihoodthatisimportantisthecompletedataloglikelihoodandit23isdenedby Lcd() =ln[pz(Z|)] + ln[p()].isnot amaximizer of thecompletedataloglikelihood. Nevertheless, Lcd() is important intheEMalgorithmfor determiningnumerically.TheEMalgorithmisiterative. Startingfromaninitialestimate(0)oftheparameters,asequenceof estimates(0), (1), (2), ... isproducedforwhichthecorrespondingsequenceof incompletedataloglikelihoodisnondecreasing Lid((0)) Lid((1)) Lid((2)).... Twostepsarerequiredateachstageoftheiterationtoreachthenextstage,anexpectation(E)stepandamaximization(N)step.E-step. determinestheconditionalexpectationofthecompletedataloglikelihood,Q(|(k)) = E[Lcd()|y, |(k)] (21)M-step. determinesthestagek + 1parameterestimateasthemaximizerofQ(|(k)),(k+1)= arg max[Q(|(k))] (22)Thesequence {(k): k = 1, 2, ...}thatisdenedbytheEandMstepscorrespondingtothesequenceof incompletedataloglikelihood {Lid((k)): k=1, 2, ...}isnondecreasing. Theproofcanbefoundattheappendix.Back to the optical sectioning problem. Let Lid(o(1), o(2), ..., o(N)) = L(o(1), o(2), ..., o(N))and the complete data set be the concatenation of the object probability density and the im-age intensity. The only terms of the log likelihood are those who depends on o(1), o(2), ..., o(N)because others dont aect the M-step. so we simply get the log likelihood of the object prob-abilitydensity.Lcd= Nl=1o(l) + Nl=1ln[o(l)]nl + ln[p(o(1), o(2), ...)] (23)wherenlisthetruedensityoftheobjectatpixell.E{Lcd} = Nl=1o(l) + Nl=1ln[o(l)]E{nl| ok(l), i(1), i(2), ..., i(N)} + ln[p(o(1), o(2), ...)] (24)24theexpectationexpressionE{nl|ok(l), i(1), i(2), ..., i(N)}isexpressedby(seedevelopmentintheappendix)E{nl| ok(l), i(1), i(2), ..., i(N)} = Nm=1h(ml) ok(l)(h ok)(m)i(m) (25)andtheE-stepisQ(o(1), o(2), ...|o(1)(k),o(2)(k), ...) =Nl=1o(l) + Nl=1ln[o(l)]E{nl| ok(l), i(1), i(2), ..., i(N)} + ln[p(o(1), o(2), ...)] =Nl=1o(l) + Nl=1ln[o(l)]Nm=1h(ml) ok(l)(h ok)(m)i(m) + ln[p(o(1), o(2), ...)](26)ThefunctionQ(o(1), o(2), ...|o(1)(k),o(2)(k), ...) must nowbemaximizedover o(1), o(2), ...fortheM-step. UsingQ/o(l)=0weget0= 1 +1o(l)E{nl| ok(l), i(1), i(2), ..., i(N)} +ln[p(o(1),o(2),...)]o(l). By isolating o(l) and substituting it with ok+1(l) we get the next step estimate ok+1(l) = ok(l)Nm=1h(ml)(h ok)(m)i(m)1 +ln[p(o(1),o(2),...)]o(l)(27)whereln[p(o(1),o(2),...)]o(l)iscalculatedatEq. 20intheprevioussubsection.4.3 AlgorithmTheapriori datatermisanestimateof thetruedensityandisnotprecise. moreoveritisestimatediterativelywiththealgorithmprogress. Thatiswaywearenotinterestedtoenforcethisapriori datacompletelyfromtherstiterations. Forthispurposeweuseaweightfunction(k)thatchangeswithiterationnumber. Thisfunctionenableustoenforcetheaprioridatagraduallyontheestimation.1. (k)=C1kC2+k2. o

(l) = ok(l) + ( ok(l) ok1(l))253. o(l) = o(k)(l) + 2(o(k)(l) o(k1)(l))4. Z(k)(l) =(2o

(l) o(l))2l

t12Xltt(o

(t) o(t))+ Xll22l(o

(l) o(l))(3o

(l) o(l))+ln[o

(l)]+12l+15. o(k+1)(l) = o(k)(l)Nm=1h(ml)(h ok)(m)i(m)1+(k)Z(k)(l)6. k=k+1.7. Repeat16until|L(k+1)(o(1),o(2),...)L(k)(o(1),o(2),...)|L(k+1)(o(1),o(2),...)< whereo

(l)istheaprioridataestimateatiterationk, 1istheaprioridataestimationlearningrateconstant. For(k)C1,C2andaretheweightfunctionparametersthatneedtobechosenarbitrarily. Optional choicecouldbeC1=0.1, c2=100and =1. o(l)istheestimatedmeanoftheapriori dataprobabilitydensityatvoxel l anditerationk, andZ(k)(l)istheaprioridataderivativethatwascalculatedatEq. 20forvoxellanditerationk.4.4 1Dand2DSimulationResultsSeveral simulations of dierent objects (smooth,sharp bright and sharp dim) were done bothfor MaximumLikelihoodestimationandfor Maximumaposteriori estimation. Therstsimulationwasdoneon1DobjectssharpandsmooththatcanbeseeninFig. 15and16.The Maximum likelihood algorithm results and the Maximum a posteriori algorithm resultsarepresentedinFig. 17, 19, 18and20.fromthesegureswecanclearlyseethatalthoughtheMaximumLikelihoodestimationissharperandmoreaccurateforsharpspecimens, italsointroducesartifactstotheimage. Sincethedierenceintheslopeangleisminor, themaximumaposteriori is better for suchobjects especiallyinmedical applications whereartifacts cancauseanunnecessarysurgeryoperation. For thesmoothobject thereis nodierence in the slope angle but there is more artifacts in the ML estimation, and the objectno longer looks smooth. We can derive another interesting information from the error gures18and20. WecanseethattheMAPestimatorisbiassedandtheerrorisalwayspositivewhiletheMLestimatorisnotbiassed. ThereasonforthisbiasintheMAPestimationisthefactthattheaprioriknowledgeweusedfortheestimationisnottrulycorrectanditismoresmoothingconstraintthantruedata. Becausewereallydontknowwhattheobject26intensity is (we want to nd it), we dont know what its probability density. We are claimingthatitissmoothandestimateitfromneighboringvoxels.In all the cases Both ML estimation and MAP estimation are much better than the additivedeconvolutionalgorithmresults. ItismoresharpandalthoughthereisartifactsintheMLestimatetheyarenotworsethantheartifactsintheadditivedeconvolutionestimate.Thesameconclusionarederivedfromthesecondsimulationthatwasdoneon2Dobjects,bothsmooth, sharpdimandsharpbright. InFig. 21, 22and23thedierentobjectsandsystem parameters are shown, and in Fig. 24, 25 and 26 the ML and MAP algorithm resultsareshown. TheartifactsintheMLestimateareveryobviousinthesmoothobjectFig.24, whereagridof darkandbrightlinesisseenall overthepicture. ForthesharpandbrightobjectitisobviousfromFig. 25thattheMLestimateachievesbetterresultsthantheMAPestimate. Theobjectedgesareclearerandsharperalthoughtheartifactsarestillpresent inthepicture. for thesharpanddimobject Fig. 26, neither algorithmachievegoodresults. TheMAPestimateisnotsharpenoughandtheobjectisblurredintothebackground. WhileintheMLestimatetheobjectissharpbutitisswallowedupintheartifactsatthebackground.27Figure15: systemparametersforsmoothobjectFigure16: systemparametersforsharpobject28Figure17: MLandMAPalgorithmresultsforsmoothobjectFigure18: MLandMAPalgorithmresultsforsharpobject29Figure19: MLandMAPalgorithmresultsforsmoothobjectFigure20: MLandMAPalgorithmresultsforsmoothobject30Figure21: systemparametersofsmoothobjectFigure22: systemparametersofsharpbrightobject31Figure23: systemparametersofdimbrightobject32Figure24: MLandMAPalgorithmresultsforsmoothobject33Figure25: MLandMAPalgorithmresultsforsharpbrightobject34Figure26: MLandMAPalgorithmresultsforsharpdimobject354.5 ArticleResultsThe likelihoodalgorithmwas testedonbothnoiseless andnoisyimages. The rst casewasthecrossandthecirclewithequal strengthand200nmapart. Theresultsfor thiscaseareshowninFig. 27. Aswiththeadditivedeconvolution, thelikelihoodalgorithmreduces the out of focus contributions to the planes immediately above and below the planewheretheobjectis. Whencomparedwiththeadditivedeconvolutionresults, wecanseethat20iterationsofthelikelihoodalgorithmresultsinbetterreductionoftheoutoffocuscontributionsthan40iterationsof theadditivealgorithm. Wecanalsoobservethattheringingintheoutoffocusplanesislessmarkedforthelikelihoodalgorithm.The algorithmwas thentestedwiththe same image corruptedwithasignal dependentpoissonnoisewithSNRof 3.16and1. TheresultsareshowninFig. 28(left)and(right)respectively. WhentheSNRwas3.16, thelikelihoodgavebetterreconstructionthantheadditive deconvolutionalgorithm. For 20iterations of the likelihoodalgorithmthe outof focuscontributionsaredimmer thanthoseobtainedafter 40iterationsof theadditivealgorithm. WhentheSNRwas1,thealgorithmreducestheoutoffocuscontributionsbut,thenoisewas ampliedat thesametimeandafter 20iterations it is not clear that thereconstructedimageisbetterthantherecordeddata.The maximum likelihood was then applied to the case of a dim cross 200 nm above the circle.TwoimagingconditionsweresimulatednoiselessandwithpoissonnoisewithSNR=3.16.TheresultsareshowninFig. 29(left)and(right)respectively. these resultsareshowingusagainthatthelikelihoodisperformingbetterthantheadditivealgorithmandalsothattheartifactsarelesssevere.36Figure 27: MAPalgorithmresults - without noise. simulatedcross andcircle of equalintensity200nmapart, theoptical sliceswherethecrossandcirclelaysmarkedXandOrespectively. thecolumnsfromlefttorightaretherecordedimageandthereconstructionafter5,10,20iterations.37Figure28: MAPalgorithmresults-withpoissonnoise. simulatedcrossandcircleofequalintensity200nmapart, theoptical sliceswherethecrossandcirclelaysmarkedXandOrespectively. thecolumnsfromlefttorightaretherecordedimageandthereconstructionafter5,10,20iterations. Left: maxSNR=3.16Right: maxSNR=1.38Figure29: MAPalgorithmresults-withandwithoutnoise. simulatedcrossandcircle200nm apart, the optical slices where the cross and circle lays marked X and O respectively thecrossstrengthis1/10ofthecirclestrength. thecolumnsfromlefttorightaretherecordedimageandthereconstructionafter 5,10,20iterations. Left: without noise. Right: maxSNR=3.16395 DiscussionThe Maximum likelihood and Maximum a posteriori algorithms has several advantages overadditive deconvolution. Even though each iteration of the formers takes about three times asmuch as the latter,fewer iterations are required to achieve about the same resolution. Bothlikelihoodalgorithmswereobservedtobemoretolerantof noiseandtheMAPalgorithmproduceslesssevereartifacts.Furthermore,thearticlehasconsideredonlytheidealizedcaseofaninnitesimaldetectoraperture. Suchaconguration,whileprovidingthehighestpriorresolution,alsoyieldsthelowest prior SNR in practical situations. While opening the detector aperture to admit morelightimprovestheSNR,itisalsoadmitsmorelightfromoutoffocusplanesanddegradesthe resolution. The detector aperture may be adjusted to achieve an optimum balance, fromtheperspectiveofthepostprocessor,thustuningthemicroscopeandthepostprocessorasasystem.Finally, sinceall themethods discussedinthis paper arenonlinear, theycanpotentiallyrecovercomponentsofthespecimeninthenullspaceoftheimagingoperator.6 FutureworkThenextstepinreconstructionofvolumetricdatafromopticalsectioningbywideeldorconfocal microscopeisblinddeconvolution. thatissimultaneousreconstructionof objectand the point spread function. In this article it is assumed that the optical transfer functionof themicroscopeisknown. Theconventional methodstoacquiretheinformationabouttheoptical transferfunctionare: useapointsourceastheobjectandmeasurethepointspreadfunction, or useananalytical formulaliketheonewedevelopedinsection2andgettherequiredlensparametersfromthemanufacturer. Thelimitationoftherstmethodisthatitisveryhardtoproduceasmall enoughsourceof lightforthisproblem, becausethedistancesinthisproblemareverysmall tobeginwith. Thelimitationof thesecondmethodisthefactthatnotalwaystherequiredparametersareaccurateenoughandtherearevariationsbetweenthemanufacturerdataandtheactuallens. Moreoverifyougotthe40volumetric data from anunknown source youdont have the informationon the microscopeatall.There was some work done on blind deconvolution of microscopic data by several researchers.Holmes [7] usedtheEMalgorithmfor theestimationof boththepoint spreadfunctionandtheobject. MarkhamandConchello[8] suggestedanintegratedalgorithmbasedonparametricmodel forthepointspreadfunctionandEMalgorithmfortheobject. Carasso[9]suggestedanothermethodthatdontassumesmoothnessoftheobjectandsummarizedsomeothermethodsaswell.7 AppendixA- proof for theEMalgorithmconver-genceTheorem1(EMalgorithmconvergence). Thesequence {(k): k= 1, 2, ...}thatdenedbythe E and M steps corresponding to the sequence of incomplete data log likelihood {Lid((k)) :k = 1, 2, ...}isnondecreasing.Proof. Fortheproofofthisproblemnotfrom... and... thatforZ Z(Y )thereholdsln[py(Y |)] = ln[pz(Z|)] ln[pz|y(Z|Y, )] (28)The left size of this equationis the incomplete datalog likelihood, andthe rst termontheright isthecompletedataloglikelihood. Multiplyingbothsidesbypz|y(Z|Y, (k)), andintegratingwithrespecttoZyieldsLid= Q(|(k)) _Z(Y )pz|y(Z|Y, (k))ln[pz|y(Z|Y, )] dZ (29)EvaluatingEq. 30forboth =(k+1)and =(k),andthensubtracting,resultsinLid((k+1)) Lid((k)) = Q(|(k+1)) Q(|(k))_Z(Y )pz|y(Z|Y, (k))ln_pz|y(Z|Y, (k+1))pz|y(Z|Y, (k))_dZ(30)41Applicationof theinequalityln(x) x 1, forwhichequalityholdsif andonlyif x=1,thenyieldsLid((k+1)) Lid((k)) Q(|(k+1)) Q(|(k))_Z(Y )pz|y(Z|Y, (k))_pz|y(Z|Y, (k+1))pz|y(Z|Y, (k))1_dZ= Q(|(k+1)) Q(|(k))(31)withequalityif andonlyif pz|y(Z|Y, (k+1)) =pz|y(Z|Y, (k)), whichholdsif(k+1)=(k).TheM-step... impliesthattherightsideof... isnonnegativeingeneral andequal tozeroif(k+1)=(k). Hence, Lid((k+1)) Lid((k)). whichestablishesthetheorem.

8 AppendixB-calculatingE{nl|ok(l), i(1), i(2), ..., i(N)}Theorem2(conditional meanfor histogramdata). Let {N(A) : A X}beapoissonprocess withanintegrableintensityfunction {(x) : x X}. Points of this process aretranslated to the output space Yto form the output point process {N(B) : B Y}, where eachpointisindependentlytranslatedaccordingtothetransitiondensityp(y|x). Let {Y1, Y2, ...}be disjoint sets in Ysuch that Y=

k=1Yk. Dene M(Yk) to be the number of points in Yk,andlet M= {M(Y1), M(Y2), ...}denotehistogramdataderivedfrompointsintheoutputspace. Theniftherearenoinsertionsanddeletions,theconditionalexpectationE[N(A)|M]ofthenumberofpointsinasubsetAoftheinputspace XgiventhehistogramdataMisE[N(A)|M] = k=1__Yk_Ap(y|x)(x) dxdy_Yk_X p(y|x)(x) dxdy_M(Yk) (32)Proof. LetM(A; Yk)bethenumberofpointsin YkthataretranslatedfromA.pointsintheoutputspacethataretranslatedfromthesubsetAoftheinputspaceformaPoissonprocesson Ywithintensity_Ap(y|x)(x) dx. N(A) = k=1M(A; Yk),fromwhichitfollowsthatE[N(A)|M(Y1), M(Y2)...] = k=1E[N(A, Yk)|M(Y1), M(Y2)...] (33)42E[N(A, Yk)|M(Y1), M(Y2)...] = E[N(A, Yk)|M(Yk)] ==(A; Yk)(A; Yk) + (Ac; Yk)M(Yk)(34)where(A; Yk)=E[M(A; Yk)]=_Yk_Ap(y|x)(x) dxdyistheaveragenumberof pointsinYkthatweretranslatedfromA,and(Ac; Yk)=E[M(Ac; Yk)]=_Yk_Ac p(y|x)(x) dxdyistheaveragenumberof pointsin YkthatweretranslatedfromthecomplementAc= X AofA.SubstitutingtheseexpressionsintoEq. 33yieldsE[N(A)|M(Y1), M(Y2)...] = k=1_(A; Yk)(A; Yk) + (Ac; Yk)_M(Yk) (35)whichisEq. 32

forourreconstructionproblem: substituteN(A)withn(l), M(Yl)witho(k)(l), (l)witho(l),andp(y|x)withh(ml). Then(A; Yk)equalsh(ml)o(l)andEq. 35denominatorisequalto(h o)(m)sowegetEq. 27.43References[1] S. Kimura and C. Munakata, Calculation of three-dimensional optical transfer functionforaconfocalscanninguorescentmicroscope,J.Opt.Soc.Am.A,vol.6,pp.10151019,July1989.[2] H. H. Hopkins, The frequency response of a defocused optical system,Proc.R.Soc.LondonSer.A,vol.231,pp.91103,1955.[3] P. Stokseth, Properties of a defocused optical system, J.Opt.Soc.Am, vol. 59, pp. 13141321,1969.[4] D. A. Agard, Optical sectioning microscopy: cellular architecture in three dimensions,Ann.Rev.Biophys.Bioeng.,vol.13,pp.191219,1984.[5] Z. Liang, Statistical modelsof apriori informationforimageprocessing: neighboringcorrelationconstraints,J.Opt.Soc.Am.A,vol.5,pp.20262031,December1988.[6] D. L. SnyderandM. I. Miller, Randompoint processesintimeandspace. NewYork:Springer-Verlag,1991.[7] H. Timothy J, Blind deconvolution of quantum limited incoherent imagery: maximum-likelihoodapproach,J.Opt.Soc.Am.A,vol.9,pp.10521061,July1992.[8] J. Markham and J.-A. Conchello, Parametric blind deconvolution: a robust method forthe simultaneous estimation of image and blur, J.Opt.Soc.Am.A, vol. 16, pp. 23772391,October1999.[9] C. AlfredS, Linear andnonlinear imagedeblurringadocumentedstudy,SIAMJ,Numer.Anal.,vol.36,no.6,pp.16591689,1999.44