AoM

BJRCTE

1

Iirdamavdatarofittfafra

1

J. Micro/Nanolith. MEMS MOEMS 7�2�, 021009 �Apr–Jun 2008�

J

daptive scanning optical microscope: large fieldf view and high-resolution imaging using aEMS deformable mirror

enjamin Potsaidohn Ting-Yung Wenensselaer Polytechnic Instituteenter for Automation Technologies and Systemsroy, New York 12180-mail: ben.potsaid@gmail.com

Abstract. For a wide range of applications in biology, medicine, andmanufacturing, the small field of view associated with high-resolutionmicroscope systems poses a significant challenge in practice. To ad-dress this limitation, a novel optical microscope uses a micromachinedMEMS deformable mirror working with a specially designed scan lens toachieve a two-order-of-magnitude increase in the field of view area.Called the adaptive scanning optical microscope �ASOM�, the deform-able mirror in the ASOM is an integral component of the optical systemand the static �glass� optical elements are specifically designed to matchthe shape correcting capabilities of the deformable mirror itself. Afterdescribing the design and operating principle of the ASOM, experimentalresults from a low-cost prototype are presented. It is shown how animage-based optimization method can be used to first calibrate the elec-trical voltages to the MEMS deformable mirror. And once calibrated, weshow how the deformable mirror can be used in an open loop controlapproach for very fast operation during run time. The methods for cali-bration of a MEMS deformable mirror and basic control structures dem-onstrated form the basis for a range of emerging adaptive-optics-enabled technologies and instrumentation. © 2008 Society of Photo-OpticalInstrumentation Engineers. �DOI: 10.1117/1.2909451�

Subject terms: microscopy; microelectromechanical-system deformable mirror;adaptive optics; imaging; optimization.

Paper 07053SSR received Jul. 31, 2007; revised manuscript received Oct. 17,2007; accepted for publication Oct. 23, 2007; published online Jun. 25, 2008. Thispaper is a revision of a paper presented at the SPIE conference on MEMS Adap-tive Optics, January 2007, San Jose, California. The paper presented there ap-pears �unrefereed� in SPIE Proceedings Vol. 6467.

Introduction

n a traditional microscope design, the field of view is lim-ted for any given resolution and decreases in size as theesolution is increased. Restrictions on the angles of inci-ence of the light on the lens surfaces, off-axis aberrations,nd manufacturability of the opto-mechanical assembliesake it extremely difficult to design fixed �static� optical

ssemblies that simultaneously achieve a wide field ofiew, high numerical aperture, and flat image field. In ad-ition to these optical design challenges, there is a second-ry effect related to signal processing theory requiring thathe image be spatially sampled by the camera pixels at orbove the Nyquist frequency to avoid aliasing and loss ofesolution due to undersampling effects. Thus, the numberf pixels in the camera determines an absolute maximumeld size for a given resolution, regardless of the quality of

he optics. The relatively small field sizes at high resolutionhat result from the optical design and image sampling ef-ects working together are becoming a significant hindrances optical microscopes are being used in automated systemsor advanced biotechnology research, medical diagnostics,obotic micromanipulation, and industrial inspection. Thesepplications increasingly require high throughput or chal-

537-1646/2008/$25.00 © 2008 SPIE

. Micro/Nanolith. MEMS MOEMS 021009-

lenging spatial-temporal observations where the small fieldsize is often the source of a bottleneck in the process orprevents observation of the event of interest altogether.

Equipping the microscope with multiple parfocal objec-tives on a rotating turret, motorized moving stages, zoommicroscope designs, and line scanning techniques are allexamples of approaches to cope with the small field of viewand are quite common. However, each method suffers fromdeficiencies of its own in practice. For example, a widefield of view and high resolution cannot be obtained simul-taneously when using multiple parfocal objectives or azoom microscope design. Relatively slow dynamic perfor-mance and agitation to the specimen are unavoidable whenusing a moving stage. The line scanning approach uses tra-ditional optics combined with an image sensor consisting ofa single row of pixels. By translating the object relative tothe imaging system, a line across the object is imaged witheach camera exposure. A continuous image of the objectcan then be formed by properly assembling the imagestripes. However, because the object is in constant motionrelative to the camera, line scanning techniques require ex-tremely bright illumination for short exposure times, whichcan be damaging to living biological specimens, or requireslow scanning speeds when bright illumination is unavail-able to avoid blurring the image.

To address some of the major limitations of existing

Apr–Jun 2008/Vol. 7�2�1

maMsertsAtmrrroopcomt

2

Ttmi1mffisbadr1dtttrsa

Potsaid and Wen: Adaptive scanning optical microscope…

J

icroscope technologies, a new microscope called thedaptive scanning optical microscope �ASOM� uses aEMS deformable mirror in coordination with a high

peed steering mirror and specially designed scanner lens toffectively enlarge the field of view without compromisingesolution. The basic operating theory and system layout ofhe ASOM have been described in a previously publishedimulation-based paper.1 Demonstrations using a low-costSOM prototype for microrobotic2 and biological3 applica-

ions illustrate the advantages of the ASOM for trackingultiple moving objects and observing large areas at high

esolution, without agitating the sensitive workspace envi-onment. This work focuses on the MEMS deformable mir-or shape optimization and real-time control. An overviewf the ASOM concept and the deformable mirror technol-gy is presented in Sec. 2. A low-cost but fully functionalrototype that demonstrates all the essential operating prin-iples is described in Sec. 3. The deformable mirror shapeptimization and control is described in Sec. 4, with experi-ental results presented using a USAF 1951 calibration

arget. Conclusions and future work are presented in Sec. 5.

Overview of the Adaptive Scanning OpticalMicroscope

he ASOM was developed to facilitate challenging spatial-emporal observations, and offers a unique set of perfor-

ance capabilities compared with the existing wide-fieldmaging technologies discussed in Sec. 1. Referring to Fig.�a�, the underlying concept of the ASOM is to use a low-ass steering mirror located after the scanner lens and be-

ore the imaging optics �i.e., a postobjective scanning con-guration�. This configuration allows a complete k�k pixelub-field-of-view �not a point as in confocal microscopy� toe quickly scanned throughout the workspace, with an im-ge acquired at each scan position. In this way, multipleisjoint or overlapping regions of the workspace can beapidly visited and imaged sequentially, as shown in Fig.�b�. Additionally, through image warping to remove lensistortion and image mosaic construction, a large and con-inuous virtual image of the object can be constructed fromhe individual image tiles. This configuration has advan-ages in generating a large effective field of view at highesolution with no disturbance to the sample. And the high-peed scanning operation allows both the ability to imagerbitrary regions of the sample and high throughput opera-

Fig. 1 �a� ASOM postobjective scann

. Micro/Nanolith. MEMS MOEMS 021009-

tion. However, compared to a traditional microscope designor a moving-stage-based approach, there is extensive off-axis imaging in the ASOM �i.e., images are obtained bylooking diagonally through the scan lens�, which introducesimage distortion in addition to contrast degrading and res-olution reducing aberrations �e.g., coma, astigmatism, fieldcurvature, higher order aberrations, etc.�. We overcome thischallenge by using a highly dynamic and active opticaldesign that addresses the off-axis aberrations by thefollowing.

1. Explicitly incorporating field curvature into the scan-ner subsystem design to reduce the complexity andlens count of the scanner lens.

2. Using an actuated MEMS deformable mirror �DM� inthe optical path to correct for residual aberrations toboth expand the diffraction-limited field of view andto further simplify the scanner lens design.

3. Image processing to remove image distortion.

Figure 2�a� shows a realistic ASOM layout based onreadily available glass types5 and manufacturable lens ge-ometries. Light from a source illuminates the object �speci-men�. The role of the scanner lens assembly is to collectlight from the object and direct the light to the center of thesteering mirror. In this design, a single mirror with twodegrees of steering freedom is used to reduce cross-axisscan error that is inherent in using separate mirrors for eachaxis6 and to manage the large aperture size. The specificangle of the steering mirror defines the location of the cen-ter of the field of view on the object, as shown in Fig. 2�b�.However, especially for the off-axis field positions, andeven to a certain extent on-axis, the scanner lens introducesconsiderable wavefront aberrations, which blur the image.To manage this blurring, the light from the steering mirrorenters a pupil imaging stage to be directed to the deform-able mirror, which adapts its surface shape to correct for thewavefront errors introduced by the scanner lens assembly.Light exiting the deformable mirror is now well correctedand nearly aberration free. After passing through the secondstage of pupil relay optics and being projected onto thescience camera, the image quality has been corrected tobeyond the diffraction limit. Exceeding the diffraction limit

sign. �b� ASOM modes of operation.

ing de

Apr–Jun 2008/Vol. 7�2�2

ifi

acsa1tauatssnabvstotua

Potsaid and Wen: Adaptive scanning optical microscope…

J

ndicates imaging quality that is nearly indistinguishablerom perfect and is the goal of most practical optical imag-ng system design.

The ASOM design shown in Fig. 2�a� was designedround the �DM100 deformable mirror from Boston Mi-romachines �Watertown, Massachusetts� with 140 electro-tatic actuators, a 3.3-mm round aperture, a 2-�m stroke,nd a 1-kHz update rate.7 The actuators are arranged in a2�12 actuator grid pattern, minus the four corner actua-ors. Differing from most other MEMS membrane deform-ble mirror technologies commercially available today, anique characteristic of the Boston Micromachines deform-ble mirror technology is the inclusion of a third layer inhe design, as shown in Fig. 3. This third layer is a thinilicon beam that is doubly cantilevered at both ends anduspended over an electrostatic actuator pad located under-eath the actuator beam’s bottom surface. Each actuator isttached to the deformable mirror’s continuous face sheety a thin post. By holding the actuator beam at a constantoltage and charging the pad below, an attractive electro-tatic force pulls down on the actuator beam. Conversely, ifhe voltage is removed from the actuator pad, the stiffnessf the doubly cantilevered beam acts as a restoring forcehat can push upward on the face sheet surface. A typicalsage scenario of this type of deformable mirror is to biasll of the actuators to a voltage that deflects the mirror to

Fig. 2 �a� Simulated ASOM design using custom optics scan lens

Fig. 3 MEMS deformable mirror.

. Micro/Nanolith. MEMS MOEMS 021009-

approximately half of the total stroke capability. In thisway, reducing the voltage to an actuator results in a local-ized positive surface deflection, and increasing the voltageto an actuator results in a localized negative surface deflec-tion. A wide range of surface shapes can be generated inthis manner.8

Figure 2�b� shows how the DM corrects for the specificwavefront aberration associated with each field position.Over the entire field and for all field positions, the Strehlratio is much greater than the diffraction limit of 0.8, re-sulting in near perfect imaging. Based on high-fidelitysimulations using ZEMAX to simulate the optics and a fi-nite element model of the deformable mirror to producerealistic deformable mirror shapes, typical values of theStrehl ratio were found to be approximately 0.96, even forthe extreme off-axis positions. Experimental measurementsof the Strehl are yet to be performed and constitute futurework. Image distortion over all field positions is less than1% as obtained from simulation. Given that algorithmicimage distortion removal has been shown for endoscopes9

and wide angle lenses �fish eye lenses�,10 the distortion inthe ASOM is well within the proven distortion compensa-tion capabilities of algorithmic distortion removal tech-niques, which will allow for the construction of seamlessimage mosaics. More detailed information about the opticallayout and operating principle of the ASOM may be foundin a previously published paper.1

Table 1 lists performance specifications for the simu-lated ASOM design shown in Fig. 2�a�, and Fig. 4 com-pares the large 40-mm observable field of view of theASOM to a traditional microscope design using a 1024�1024 and 4096�4096 pixel camera. The 0.38-mm sizeof the ASOM’s subfield of view is also shown, resultingfrom the use of a 512�512 camera. For this design, a512�512 pixel camera was chosen to achieve the highframe rates needed for tracking rapid motile organisms. TheASOM offers diffraction limited �Strehl ratio �0.8� for allfield positions based on high-fidelity simulation. The fieldsizes for the fixed microscope designs in Fig. 4 assumeperfect imaging and were calculated based on the Nyquist

eformable mirror corrects for optical aberrations in the scan lens.

. �b� D

Apr–Jun 2008/Vol. 7�2�3

s=e

itmbscps1wcobtets

absmp

Potsaid and Wen: Adaptive scanning optical microscope…

J

ampling theory using a 0.21 numerical aperture with �0.510 �m for the wavelength of light for each of the cam-ra pixel counts shown.

Under normal operating conditions, the steering mirrorn the ASOM changes angle to observe different regions ofhe specimen. To avoid motion-induced blur, the steering

irror should settle completely before the camera exposureegins. Thus, each steering mirror movement should con-ist of a dynamic portion in which the steering mirror anglehanges and a dwell portion during which the camera ex-osure occurs. Based on our previous work with high-speedteering mirrors,11 we estimate that we can achieve at least00 movements per second by using a combined feed for-ard and feedback control approach. Faster frame rates

ould be achieved if camera exposures were obtained with-ut waiting for the mirror to completely settle, which woulde useful for performing a rapid background scan to iden-ify object locations or detect the onset of a biologicalvent, but at compromised image quality. The system couldhen image the region of interest at high image quality bytopping the mirror for each camera exposure.

In the ASOM, the static optical components are deliber-tely designed to be imperfect �i.e., nondiffraction-limited�,ut in such a way that the wavefront error is within thetroke and shape forming limits of the MEMS deformableirror. As was used in the novel ASOM design, this ap-

roach enables a decrease in the complexity of the static

Fig. 4 Field size comparison.

Fig. 5 Low-co

. Micro/Nanolith. MEMS MOEMS 021009-

optical elements while simultaneously increasing the capa-bilities of the instrument to rapidly observe over an en-larged field of view. However, the intimate relationship be-tween the lens geometry and the adaptive optics wavefrontshape correcting capabilities require that the coupling be-tween the adaptive optics components and the static opticalcomponents be explicitly considered during the design andoptimization process. During the development of theASOM, we have developed systematic design techniques12

for effectively synthesizing such systems. These designtechniques are based on the field of multidisciplinary de-sign optimization �MDO�, which is used to combine opticalsimulations with high-fidelity deformable mirror modelsduring the design process.

3 Experimental Adaptive Scanning OpticalMicroscope Prototype

Shown in Fig. 5, the purpose of the experimental ASOMapparatus was to demonstrate all essential optical aspectsof the ASOM design, but at low cost and with a shortdevelopment time. As such, off-the-shelf optics were used

Table 1 Simulated ASOM performance specifications.

Specifications

Field of view diameter 40 mm

Observable field area 1257 mm2

Numerical aperture 0.21

Operating wavelength 510 nm

Resolution 1.5 �m

Magnification 15.2

Camera pixel count 512�512

Camera pixel size 10 �m

M prototype.

st ASO

Apr–Jun 2008/Vol. 7�2�4

ecidnasooeolpoA

sns5m5j1asdPacmatTtdpapttWtec

4

AkrTfitanaaaBdt

Potsaid and Wen: Adaptive scanning optical microscope…

J

xclusively to avoid the considerable cost of custom fabri-ated optics and to take advantage of available catalogtems that ship within days. However, most stock lenses areesigned to be used in a particular manner �e.g., with infi-ite conjugates� for generic applications and are offered incoarse range of focal distances, lens diameters, and glass

elections. Considering the atypical imaging characteristicsf the scanner lens, the experimental ASOM design usingff-the-shelf optics only is far from optimal, and as such,xhibits a noticeably high lens count to achieve 0.1 NAver a nominal 8-mm field size. However, even with theimitation of using off-the-shelf optics exclusively, this ex-erimental apparatus has been carefully designed to dem-nstrate all of the critical optical characteristics of theSOM.This initial prototype utilizes a transmitted lighting

cheme with a scanning Köhler illumination stage under-eath the object. And because the current design is veryensitive to chromatic aberration using only BK-7 glass, a10-nm wavelength bandpass filter is included in the illu-ination stage to eliminate much of the light below

00 nm and above 520 nm. Light transmits through the ob-ect contrast pattern and is then collected by the telecentric2 element scanner lens assembly. An electromagneticallyctuated fast steering mirror �FSM� with a flexure suspen-ion �Optics in Motion OIM101� has two degrees of free-om to steer the subfield of view within the workspace.upil relay optics project the light onto the MEMS deform-ble mirror �Boston Micromachines Mini-DM�. By pre-isely controlling the shape of the reflective surface of theirror to be opposite the shape of the wavefront error �but

t half the amplitude�, the deformable mirror can correct forhe wavefront aberrations to within the diffraction limit.his mirror has 32 electrostatic actuators with 400-�m ac-

uator spacing, a 2.5-�m actuator stroke, and a 2.0 mmiameter actively controlled area. The 2.5 �m stroke is ca-able of correcting up to several waves of aberration, whichllows for high image quality even for the off-axis fieldositions, and enables the greatly expanded field of view inhe ASOM. Additional optics project the final image ontohe CCD camera �Pulnix TM200�. A computer running the

indows operating system serves as the human interfaceo the instrument and communicates with the camera,lectromechanical subsystems, and real-time hardwareomponents.

Deformable Mirror Shape Optimizationand Control

llowing for significant aberration in the scanner lens is aey idea in the ASOM to achieve a wide field of view at aelatively low cost for the static optical elements �lenses�.he specific shape and size of the aberration depends on theeld position, but it is important that the rate of change in

he aberration between field positions be sufficiently smalls to allow a diffraction-limited image over one instanta-eous field of view �image exposure�. In other words, forny one particular field position, there must be a deform-ble mirror shape that can correct for the aberrations overn entire individual image tile without blurring at the edges.ut, when the steering mirror changes angle, a differenteformable mirror shape is required for the new field posi-ion associated with using a different region of the scanner

. Micro/Nanolith. MEMS MOEMS 021009-

lens. For any given field position, however, the aberrationsare static as the glass causing the aberration is stationary.And because the electrostatic MEMS deformable mirrorsdo not exhibit hysteresis, an open loop strategy can be uti-lized for the deformable mirror control to achieve highspeeds during operation. First, the appropriate deformablemirror shape must be determined through a calibration pro-cedure and the results stored. Then, during operation, theactuator voltages to the deformable mirror V are recon-structed as a function of the current field position as deter-mined by the steering mirror angles �x and �y:

V = f��x,�y� . �1�

In simulation, perfect knowledge of the wavefront shapeis readily available and it is straightforward to numericallydetermine the required mirror shape to correct for the spe-cific aberrations associated with each field position. How-ever, in practice, manufacturing tolerances and assemblyerrors introduce unmodeled aberrations into the optical sys-tem. Furthermore, internal residual stresses combined withmanufacturing variations result in surface bowing in theMEMS deformable mirror, as well as actuator performancevariation.13 Thus, experimental use of the deformable mir-ror requires a method for the in-situ shape optimization ofthe deformable mirror surface. If we consider the inputs ofthe optimization to be the deformable mirror actuator volt-ages V, and Q to be some metric of the image quality orresidual aberration, then the experimental deformable mir-ror shape optimization requires:

• a metric to represent the image quality or residual ab-erration Q�V�

• an optimization algorithm to minimize Q�V�.

The metric Q�V� is in general a nonlinear function of theactuator voltages V, and Q�V� is defined to decrease withimproving image quality. The resulting optimization prob-lem is also subject to upper and lower bounds on the actua-tor voltages. Methods and approaches for optimization con-stitute an active field of research on its own, and there aremany possible image quality metrics and optimizationmethods that can potentially be combined for the ASOMdeformable mirror shape optimization.

In practice, the wavefront cannot be measured directly,but must be inferred from a related measurement of lightintensity. Often a wavefront sensor �e.g., Shack Hartmannor pyramid� is used. Other common methods include inter-ferometry or algorithmic techniques such as phase diversityusing two or more cameras at different focal planes. For theASOM, it is advantageous from a cost and complexity per-spective to use a technique based on the image qualityalone, not requiring any additional hardware or layout re-configuration. Various metrics have been proposed for au-tomatically assessing the quality of an image, including im-age entropy14 and image sharpness.15

The image quality metric used for the optimization inthis research is based on maximizing the energy of thehigh-frequency image content, and the specific formulationof the metric was obtained from Ref. 16. First, a high-passfilter in the form of a Laplacian filter K was convolved withthe acquired image. Often used for edge detection in image

Apr–Jun 2008/Vol. 7�2�5

prissotflrrrTamiq

Q

w

E

a

wr

Potsaid and Wen: Adaptive scanning optical microscope…

J

rocessing, a Laplacian filter computes second-order de-ivative information from the values contained within themage. Thus, the Laplacian filtered data represents theharpness in the knee between dark and bright region tran-itions �as opposed to the steepness of intensity change asbtained from a first order filter�. For a black and whitearget, as was used in this research, a sharp and aberration-ree image will have regions of constant brightness witharge second order derivative information at the transitionegions. The summation of the absolute value of the filteredesult indicates the degree of sharpness �and similarly alsoepresents the high-frequency energy content of the image�.o account for light variations, the result is normalized byn image brightness term E, which is computed as the sum-ation of all pixel intensity values. Given an i� j matrix of

ntensity values to represent the image I, the specific imageuality metric Q was:

= −1

E�

i�

j

��I * K�i,j� , �2�

here

= �i

�j

Ii,j, K = � 0 − 1 0

− 1 4 − 1

0 − 1 0� ,

nd * indicates convolution.Optimization of the image quality metric was performed

ith the parallel stochastic-gradient-descent �PSGD� algo-ithm, which is based on estimating the gradient � Q�V� by

Fig. 6 Deformable mirro

V

. Micro/Nanolith. MEMS MOEMS 021009-

applying random perturbations to the input signals.17 It isapplicable to systems where there are many input signalsand a performance metric that can be quantified, but nomodel information is available to relate the performancemetric to the inputs. A derivation and justification of theapproach can be found in Ref. 17, and the algorithm foundin Ref. 18 is as follows.

Given an initial set of actuator signals Vn at time step n,and a corresponding objective function value Q�Vn�:

• generate a set of equal magnitude, but random signperturbations, �V, to apply to the actuator controlsignals

• apply the perturbations to the system and measure theobjective function: Qn

+=Q�Vn+�V�• Reverse the polarity of the perturbations, apply the

negative perturbations to the system, and measure theobjective function: Qn

−=Q�Vn−�V�.• Update the actuator signals with a gain on the step size

� as:

Vn+1 = Vn + ��Qn+ − Qn

−��V . �3�

Figure 6 shows the evolution of the 32 deformable mir-ror actuator voltages for each optimization iteration usingthe image quality metric in Eq. �2� and update law in Eq.�3�. All actuators start at 85 V, which is slightly less thanhalf the voltage supply. The resulting 32 actuator voltagesare shown in a 6�6 grid pattern missing the corner pixels,as is the configuration of the actual deformable mirror. Re-quiring two frames per iteration �one positive perturbation

tor voltage optimization.

r actua

Apr–Jun 2008/Vol. 7�2�6

aapBbqosabr3aihabwoo

fit

Potsaid and Wen: Adaptive scanning optical microscope…

J

nd one negative perturbation� and with the camera runningt 30 fps, the optimization of each field position takes ap-roximately 33 min to complete the 30,000 iterations.ased on the image of the USAF 1951 calibration targetefore and after optimization, it is clear that the imageuality improves as the shape of the deformable mirror isptimized. The smallest bar pattern that can be clearly re-olved in the optimized image represents a resolution ofbout 4 �m, as can be seen in the zoomed in andrightness/contrast adjusted image in Fig. 6. The theoreticalesolution of the low-cost ASOM apparatus is about.5 �m �0.1 NA at �=510 �m�, which indicates that theberrations have been reduced to being close to, or exceed-ng, the diffraction limit. Note that in related research, weave found that an image sharpness metric15 based on im-ge intensity shows more robustness than the frequency-ased method used in this research, especially for largeavefront errors. Further exploration and characterizationf image-based optimization methods is an active area ofur research.

Because it is not practical to optimize for every possibleeld position, we optimize a grid sampling of points over

he entire observable field of view and use interpolation of

Fig. 7 �a� Deformable mirror is optimized for a grid sampling of th

Fig. 8 Interpolation results

. Micro/Nanolith. MEMS MOEMS 021009-

the deformable mirror actuator voltages for field positionsthat lie between data points, as shown in Fig. 7. Due to theconstraint of using off-the-shelf optics in the low-costASOM prototype, the magnitude and variation of the aber-rations introduced by the scanner lens are relatively small,but significant enough to noticeably degrade the imagequality. We are therefore able to use a 5�5 grid of cali-brated points to cover the entire observable field of view�approximately 5.0�3.5 mm as limited by the steeringmirror maximum deflection�, which is shown in Fig. 7.During ASOM operation, bilinear interpolation is used toestimate the optimal set of actuator voltages in between thecalibrated data points, V00, V01, V10, V11 that include thecurrent operating position:

Vi = V00�1 − x��1 − y� + V10x�1 − y� + V01�1 − x�y + V11xy ,

�4�

where x=x1 / �x1+x2�, y=y1 / �y1+y2� as defined in Fig. 7,and Vi is the set of interpolated voltages for the commandposition of the steering mirror.

Figures 8�a�–8�c� show the image located at �0.0 mm,

rvable region. �b� Deformable mirror voltage interpolation scheme.

intermediate field position.

e obse

for an

Apr–Jun 2008/Vol. 7�2�7

0t�av1r1tban1crbptcj

UtAfisboaaptpssag

Potsaid and Wen: Adaptive scanning optical microscope…

J

.9 mm� with zero volts on all actuators, 85 V on all actua-ors, and the set of voltages optimized for field position0.0 mm, 0.9 mm�, respectively. Figure 8�d� shows the im-ge located at the same position, but using a set of actuatoroltages optimized for a different field position at �1.2 mm,.8 mm�, which is the upper right data point in this gridegion. Applying the voltages optimized for �1.2 mm,.8 mm� at field position �0.0 mm, 1.9 mm� results in no-iceable image degradation �as can be seen in the smallestar pattern, especially in the vertical bars�, which indicatesberration variation between these nearby field positions. Aew image was obtained at a field position of �0.6 mm,.3 mm� using the interpolation scheme of Eq. �4�, and it islear from looking at Fig. 8�e� that the interpolated voltagesesult in nearly the same image quality for a point lying inetween sample points as for a point located at a sampleoint that was explicitly optimized, such as Fig. 8�c�. Notehat the three zoomed-in images of the smallest bars in thealibration target shown in Figs. 8�c�–8�e� have been ad-usted for contrast and brightness.

Figure 9 shows an 11�11 tile image mosaic of theSAF 1951 calibration target obtained using the interpola-

ion scheme at each field position. Note that in the low-costSOM, there are approximately 3.6 frame heights and 3.4

rame widths between sample points, requiring significantnterpolation at most field positions. The image was con-tructed sequentially, moving the center of the field of viewy one frame height between image exposures. At the endf each vertical scan pattern, the horizontal field locationdvances by one field width. The deformable mirror volt-ges were automatically calculated in real time using inter-olation of the previously stored calibration data as a func-ion of the current field position. Image registration is noterfect due to image distortion and a nonlinearity in theteering mirror feedback sensor. These issues will be re-olved by using distortion compensation algorithms9,10 andnonlinear compensation in the steering mirror control al-

orithms for calibrated pointing.In this low-cost prototype, an Optics in Motion OIM101

Fig. 9 11�11 tile image mosa

. Micro/Nanolith. MEMS MOEMS 021009-

steering mirror was used. This system has separate x and yaxis control of a single 1-in. steering mirror suspended on aflexure mount. We found that the dynamic response of thissystem using the stock analog controller was inadequatebecause of long settling times and significant steady-stateerror. For this reason, we replaced the analog controllerwith our own digital controller, using XPC target fromMathWorks as the real-time operating system andMATLAB/Simulink as the development environment. First,we identified the spring constant of the flexure mount andused feedback to cancel out this disturbance. We then iden-tified the dynamic system model and designed a propor-tional lead controller. As is often the case, the performancewith a feedback controller alone is not satisfactory, consid-ering the need to maintain sufficient stability margins �gainmargin and phase margin�. For this reason, we augmentedthe feedback controller with a feed-forward controllerbased on an FIR filter to approximately invert the closedloop dynamics for improved tracking control. The coeffi-cients of the FIR filter were obtained using a fit with resultsfrom iterative learning control �ILC� using methods devel-oped for high-speed mirror control11 of semiconductor pro-cessing equipment. All real-time control runs at a 0.15-mssample rate. The time to complete a move equal to oneframe height is approximately 2.5 msec, and the time tocomplete a move of six frame heights is approximately6 msec. If the system were to run at 100 frame per second,this leaves between 4.0 to 7.5 ms for image exposure forthese particular move lengths. However, our current imple-mentation uses a camera system limited to 30 frames persecond. Thus, a complete 11�11 tile image mosaic �suchas that shown in Fig. 9� can be acquired in approximately 8seconds by grabbing every other frame on this free runningcamera. We are currently in the process of integrating ahigh-speed camera using the Camera Link data transfer in-terface to achieve 100 frames per second operation. Thisimproved high-speed capability will be essential for imag-

USAF 1951 calibration target.

ic of a

Apr–Jun 2008/Vol. 7�2�8

itw

5Bncsdfos

acipolrsfico

sgococtncfogfhmWfss

ATN0aYRbaC

R

Potsaid and Wen: Adaptive scanning optical microscope…

J

ng of dynamic events, such as the microrobotic gripperracking2 and multiple live animal imaging3 research thate are currently pursuing with the ASOM.

Summaryy replacing optical complexity with active optical compo-ents, we develop a new microscope with unique imagingapabilities, called the adaptive scanning optical micro-cope �ASOM�. Applications in biology, medicine, and in-ustry will benefit from the rapid scanning capabilities thatacilitate observation of challenging spatial-temporal eventsver a large field of view, without any agitation to thepecimen.

In the ASOM, a MEMS deformable mirror is an en-bling technology that works with a scanning mirror andustom-designed scanner lens to achieve diffraction-limitedmaging over a greatly expanded field of view. A low-costroof of concept prototype demonstrates all of the criticalptical aspects of the design, but at a compromised reso-ution and field size. After calibrating the deformable mir-or using an image-based optimization metric for a gridampling of operating points over the entire observableeld of view, we successfully demonstrate an open loopontrol of the MEMS deformable mirror using interpolationf the previously optimized neighboring data points.

The lessons learned and experiences gained serve as atrong foundation for the development of the next-eneration ASOM prototype, as well as an emerging classf optical systems19,20 that use adaptive optics as an integralomponent of the optical design. With the current transitionf MEMS deformable mirrors from a specialized wavefrontontrol device used by only a few select research organiza-ions to a low cost and high-performance product that canow be considered for inclusion in certain commercial ma-hines and instrumentation, there will be an increased needor wavefront sensorless calibration and control of adaptiveptics systems. Future research includes realizing the next-eneration ASOM, which will incorporate custom manu-actured optics to achieve the full 40-mm field of view, aigh-performance steering mirror, and MEMS deformableirror with longer stroke and a larger number of actuators.e will then use the next-generation ASOM as a platform

or further investigations into effective methods for the de-ign and use of MEMS deformable mirrors in novel opticalystems and applications.

cknowledgmentshis material is based in part on work supported by theational Science Foundation under grant number CMS-301827 and by the Center for Automation Technologiesnd Systems �CATS� under a block grant from the Nework State Office of Science, Technology, and Academicesearch �NYSTAR�. Author Wen is also supported in party the NSFC two-bases project �number 60440420130�,nd the Outstanding Overseas Chinese Scholars Fund ofhinese Academy of Sciences �number 2005-1-11�, China.

eferences

1. B. Potsaid, Y. Bellouard, and J. T. Wen, “Adaptive scanning opticalmicroscope �ASOM�: a multidisciplinary optical microscope designfor large field of view and high resolution imaging,” Opt. Express

. Micro/Nanolith. MEMS MOEMS 021009-

13�17�, 6504–6518 �2005�.2. B. Potsaid, Y. Bellouard, and J. T. Wen, “Adaptive scanning optical

microscope �ASOM� for large workspace micro-robotic applica-tions,” in Proc. IEEE Intl. Conf. Robotics Automation (ICRA),pp. 1024–1029 �2006�.

3. B. Potsaid, F. P. Finger, and J. T. Wen, “Automation of challengingspatial-temporal biomedical observations with the adaptive scanningoptical microscope �ASOM�,” in Auto. Sci. Eng. 2005 IEEE Intl.Conf., pp. 37–42 �2006�.

4. B. Potsaid, L. I. Rivera, and J. T. Y. Wen, “Adaptive scanning opticalmicroscope �ASOM�: large field of view and high resolution imagingusing a MEMS deformable mirror,” Proc. SPIE 6467, 646706 �2007�.

5. J. Tesar, “Using small glass catalogs,” Opt. Eng. 39�7�, 1816–1821�2000�.

6. J. I. Montagu, “Two-axis beam-steering systems: TABS,” Proc. SPIE1920�1�, 162–173 �1993�.

7. T. Bifano, J. Perreault, P. Bierden, and C. Dimas, “Micromachineddeformable mirrors for adaptive optics,” Proc. SPIE 4825, 10–13�2002�.

8. Y. Zhou and T. Bifano, “Characterization of contour shapes achiev-able with a MEMS deformable mirror,” Proc. SPIE 6113, 61130H�2006�.

9. R. Shahidi, M. R. Bax, C. R. Maurer, J. A. Johnson, E. P. Wilkinson,B. Wang, J. B. West, M. J. Citardi, K. H. Manwaring, and R. Kha-dem, “Implementation, calibration and accuracy testing of an image-enhanced endoscopy system,” IEEE Trans. Med. Imaging 21�12�,1524–1535 �2002�.

10. S. Shah and J. Aggarwal, “A simple calibration procedure for fish-eye�high distortion� lens camera,” in Robotics Auto. 1994 Proc. IEEEIntl. Conf., 4, 3422–3427 �1994�.

11. B. Potsaid and J. T. Wen, “High performance motion tracking con-trol,” in Proc. IEEE Conf. Control Appl., 718–723 �2004�.

12. B. Potsaid, Y. Bellouard, and J. T. Y. Wen, “A multidisciplinary de-sign and optimization methodology for the adaptive scanning opticalmicroscope �ASOM�,” Proc. SPIE 6289, 62890L �2006�.

13. J. W. Evans, K. Morzinski, S. Severson, L. Poyneer, B. Macintosh, D.Dillon, L. Reza, D. Gavel, D. Palmer, S. Olivier, and P. Bierden,“Extreme adaptive optics testbed: performance and characterizationof a 1024-MEMS deformable mirror,” Proc. SPIE 6113, 61130I�2006�.

14. M. B. Garvin, M. T. Gruneisen, R. C. Dymale, and J. R. Rotge,“Image entropy as a metric for iterative optimization of large aberra-tions,” Proc. SPIE 5894, 58940Y �2005�.

15. L. P. Murray, J. C. Dainty, J. Coignus, and F. Felberer, “Wavefrontcorrection of extended objects through image sharpness maximisa-tion,” Proc. SPIE 6018, 60181A �2005�.

16. M. Cohen, G. Cauwenberghs, and M. A. Vorontsov, “Image sharp-ness and beam focus VLSI sensors for adaptive optics,” IEEE Sens. J.2�6�, 680–690 �2002�.

17. M. A. Vorontsov and V. P. Sivokon, “Stochastic parallel-gradient-descent techniques for high-resolution wave-front phase-distortioncorrection,” J. Opt. Soc. Am. A 15, 2745–2758 �1998�.

18. G. Reimann, J. Perreault, P. Bierden, and T. Bifano, “Compact adap-tive optical compensation systems using continuous silicon deform-able mirrors,” Proc. SPIE 4493, 35–40 �2001�.

19. M. T. Gruneisen, M. B. Garvin, R. C. Dymale, and J. R. Rotge,“Agile-field telescope system with diffractive wavefront control,”Proc. SPIE 5894, 589414 �2005�.

20. F. Shevlin, N. Descharmes, and Y. Xue, “Dynamic optics for digitalprojection applications,” 6289, 62890U �2006�.

Benjamin Potsaid received his BS, MS,and PhD in mechanical engineering fromRensselaer Polytechnic Institute in 1999,2002, and 2005, respectively. He is cur-rently a research scientist at the Center forAutomation Technologies and Systems�CATS� at Rensselaer. At the CATS, heoversees the Smart Optics Lab �SOL�,which pursues research in novel active andadaptive optical systems and devices withbiomedical, industrial, and defense applica-

tions. His research interests include optomechatronic system de-sign, robotics and motion control, as well as math, science, andengineering education. He is a member of SPIE and IEEE.

Apr–Jun 2008/Vol. 7�2�9

J

Potsaid and Wen: Adaptive scanning optical microscope…

J

John Ting-Yung Wen received his BEngfrom McGill University in 1979, MS fromUniversity of Illinois in 1981, and PhD fromRensselaer Polytechnic Institute in 1985, allin electrical engineering. Since 1988, hehas been with Rensselaer Polytechnic Insti-tute, where he is currently a professor in theDepartment of Electrical, Computer, andSystems Engineering with a joint appoint-ment in the Department of Mechanical,Aerospace, and Nuclear Engineering. Since

uly 2005, he has been the Director of a New York State sponsored

. Micro/Nanolith. MEMS MOEMS 021009-1

interdisciplinary center, Center for Automation Technologies andSystems �CATS�. He has more than 150 technical publications inleading journals and conferences. His research interest lies in thegeneral area of dynamical systems modeling, control, and planningwith applications to vibration suppression, robot manipulation, bio-medical systems, advanced material design, and network flow andpower control. He is a Fellow of IEEE.

Apr–Jun 2008/Vol. 7�2�0