Application of digital image inpainting in Electrochemical Scanning ProbeMicroscopy

E. Kochkina,1, a) K. McKelvey,2, b) and P. Uniwin2, c)

1)Centre for Complexity Science, University of Warwick, Gibbet Hill Road, Coventry CV4 7AL,UK2)Department of Chemistry,University of Warwick, Gibbet Hill Road, Coventry CV4 7AL,UK

(Dated: 25 December 2014)

Electrochemical scanning probe microscopy is a physical sciences technique that uses a nanoscale physicalprobe with an electrochemical sensor to map the local chemical activity of interesting substrates, such ascarbon nanotubes, graphene and single living cells. These techniques produce measurements of local chemicalproperties that are unevenly distributed over a surface, then which maps of the local chemical activity can beconstructed. Image reconstruction methods, namely PDE-based inpainting, are usually used for restorationof lost or deteriorated parts of photos and videos. We show how to reconstruct local chemical activities usinginpainting methods. We compare different techniques, and conclude that heat equation based inpainting suitsthe purpose best. We also investigate a method to reconstruct the progress of an experiment by applying 3Dheat equation inpainting.

I. INTRODUCTION

The acquisition of spatially resolved functional andstructural information on surfaces and interfaces is a ma-jor theme in contemporary microscopy, with applicationsspanning materials science [1] and technology [2], biol-ogy [3], medicine [4] and nanotechnology [5] generally.To expand the capabilities of microscopy, significant ef-forts have been invested in the development of scanningprobe microscopy (SPM) techniques [6] that facilitate di-rect measurements of various types of processes at a widerange of interfaces. Scanning probe microscopy (SPM)is a branch of microscopy that forms images of surfacesusing a physical probe that scans the specimen. Someprobe techniques reach an impressive atomic resolution.Area of our interest is Electrochemical Scanning ProbeMicroscopy (EC-SPM). It involves the use of a tiny, mo-bile, probe containing an electrode, which can be used tomonitor and/or perturb the chemical environment adja-cent to a sample. Typically the probe is scanned later-ally over a sample and maps of the local chemical envi-ronment are constructed. The data is usually obtainedas a two-dimensional grid of data points, visualized infalse color as a computer image. During the project weworked with two different techniques: Scanning Ion Con-ductance Microscopy (SICM)[7] and Scanning Electro-chemical Cell Microscopy (SECCM)[8], which, togetherwith overall setup, briefly described in Section II. In SPMtip can be moved across the surface in numbers of differ-ent ways: lines, spirals [9]-[12], cycloid [13], Lissajoustrajectories [14]. All of them have their advantages andlimitations. Some of the scanning modes produce uni-formly distributed data, but most of them do not, what

a)Electronic mail: E.Kochkina@warwick.ac.ukb)Electronic mail: Kim.McKelvey@warwick.ac.ukc)Electronic mail: P.R.Unwin@warwick.ac.uk

impedes imaging. We are looking at different data pat-terns together with corresponding missing data patternsand trying to restore the complete picture of the ex-periment using tool from the field of digital inpainting.Digital inpainting is process There are different purpose-dependent types of inpainting, structural(geometric) in-painting, texture synthesis, restoration of videos. We areparticulary interested in geometric inpainting. There-fore the aim of the project is to reconstruct an imageof the whole surface on basis of the available data, thusimprove quality of the images of scans using digital in-painting, and develop protocols and code to automati-cally generate 2D maps and performing 3D inpaintingrestore the process of the experiment as a set of frames,video. The application of inpainting techniques in mi-croscopy is a new approach which recently started to gainpopularity. Ziegler et al. [15] were concerned about thespeed of scanning and introduced spiral scanning patternas a solution to increase scanning rate. They success-fully used heat equation inpainting to deal with unevenlyspaced data, obtained from Atomic Force Microscopy(AFM) [16] scans. Stanciu et al. [17] ‘experimentedwith ‘curvature-preserving’ partial differential equationsas a solution to inpainting regions in images collectedby several optical and scanning probe microscopy tech-niques’, namely to CSLM, DIC, FRET, AFM, STM, ands-SNOM[17]. Our approach was inspired by Ziegler et al.paper. Here we looked at the application of various PDE-based inpainting algorithms to scans obtained via Scan-ning Ion Conductance Microscopy (SICM) and Scan-ning Electrochemical Cell Microscopy (SECCM) tech-niques(described below), considered utilization for differ-ent scanning patterns, i.e hopping mode scans, line scansand spiral scans. We compared the performance of differ-ent inpainting methods, such as Laplace equation-basedapproach, algorithms proposed by Bertalmio et al.[18]and Perona-Malik[19], on experimental data and chosethe most suitable for practical purposes. In the next

2

section we describe methods we used to collect, prepro-cess the data, and to inpaint regions left empty afterredistributing information to the grid. In section III wepresent results, obtained after processing experimentaldata; compare performance of different inpainting tech-niques on SICM and SECCM(described in Section II)scans with different scanning patterns and grid sizes, andexplore their limitations. After we move on to conclu-sions and discussion of further work.

II. METHODS

In this section we describe methods we used for dataacquisition, reduction and preprocessing, filtering and in-painting.

A. Data acquisition

We worked with two types of EC-SPM, namelySECCM and SICM.

Scanning ion conductance microscopy (SICM) is ascanning probe technique, which utilizes a nanopipette toscan samples bathed in electrolytic solution. The opera-tion of SICM relies on an ion current that flows betweenan electrode inside a nanopipette and another electrodein an external bath solution. This ion current, which ishighly dependent on the tip-sample separation, is utilizedas a feedback signal to maintain the tip-sample separa-tion and to allow the pipette to follow surface contours,which generates topographic information [7].

Scanning electrochemical cell microscopy (SECCM) isa new pipette-based imaging technique purposely de-signed to allow simultaneous electrochemical, conduc-tance, and topographical mapping of surfaces. SECCMuses a tiny meniscus or droplet, at the end of a double-barreled (theta) pipette, for high-resolution nanoscaleelectrochemical measurements.[8]

1. Experimental setup

Here we briefly describe the experimental setup forthe scanning techniques used to obtain the data (seeFigures 1,2).

On the Figure 1 we can see configuration of probe,piezoelectric positioners and sample. Probe((1)in Figure1) is fixed in the probe holder((4) in Figure 1) and con-nected to z piezoelectric positioner, which controls ver-tical movement towards or away from the surface of thetip during the scan. Sample ((2)in Figure 1) and sampleholder((3)in Figure 1) are located on x-y piezoelectric po-sitioners((5)in Figure 1) for lateral motion of the samplerelative to the nanopipette.

Figure 2 represents overview of the whole setup. Theamplifiers/servos ((10) in Figure 2) are responsible for

FIG. 1: Probe, positioners and sample holders.Reproduced from[20]

FIG. 2: Overview of full setup

piezoelectric positioners. A bipotentiostat ((12) in Fig-ure 2) measures electrochemical signals at the probe. AnFPGA card ((13) in Figure 2) collects all the data andcontrols the instrument. The use of an FPGA card allowscomplex calculations, such as data filtering and probe po-sition control logic, to be completed quickly. LabVIEWis used ((14) in Figure 2) to control the FPGA card onthe PC.

Electrical, acoustic and vibrational isolation are essen-tial for high resolution current measurements and posi-tional control. All instruments are mounted on vibrationisolation tables ((8) in Figure 2), within a Faraday cage,and with acoustic foam to reduce vibrations ((9) in Fig-ure 2)[20].

2. Scanning modes

Images of a surface are constructed in 3 differentmodes: hopping mode, line scanning, spiral scanning.

In the hopping mode ((a) in Figure 3), the probe ap-proaches the surface at a predetermined x,y position andcollects data. After that the tip retreats from the sam-ple surface to move to the next point. This makes a

3

FIG. 3: (a) Hopping mode; (b) Line scan; (c) Spiralscan.

series of evenly-spaced discrete measurements providinga lattice with uniformly distributed data, which allowsconstruction of a 2D map. This mode is the slowest oneof the three described in this section, because we haveto stop the scanner many times when moving betweenpoints. The advantage is that since data is evenly spacedit does not require redistribution on a grid. In order tospeed up the scanning process or scan larger areas we canmake sparse measurements and use inpainting as a toolto reconstruct the image (see Section III).

In the line scanning mode ((b) in Figure 3) maps of thesurface properties are constructed from a series of par-allel lines scans. The probe is moved forward and thenback over the same line, before being moved laterally tothe start of the next line. Data is collected during boththe forward and reverse movements, therefore two maps(a forward and a reverse map) of the surface propertiescan be constructed from every scan. Since the line scan-ning mode has a so-called ‘fast scan direction’ with manymeasurements on a line and a ‘slow scan direction’ withquite a few lines, data is distributed unevenly and thereare gaps in the ‘slow scan direction’. Inpainting can fillthe gaps to create smooth maps.

Spiral techniques are common to many data storagetechniques on spinning mediums (vinyl records, harddrives, compact disks and DVDs). However the spiralscan concept only recently found appearance in scanningprobe microscopy, where spirals [9-12], cycloid [13], Lis-sajous trajectories [14], and various other scan patternshave been demonstrated. Spiral scanning has been shownto be useful for fast scanning, because they do not requireto stop the tip at any time and cover wider areas in thesame time, comparable to other patterns [9-12]. As itis obvious from the name, during spiral scans, the tipis moved over the surface spirally, x,y positioners followthe system of parametric equations for Archimedean spi-ral with constant speed:

x = α√tsin(β

√t)

y = α√tcos(β

√t)

Acquired data is unevenly spaced and to construct a 2Dmap the process of rendering to a grid and inpaintingbecome crucial.

B. Data description and preprocessing

LabVIEW code was used for management of the ex-periment process and data collection . As a result we gettwo files: one with a description of the experiment pa-rameters and how the data is stored (in .set format), andthe actual results (in .tsv format). The latter consists ofrows of time series of the following variables: x,y,z coordi-nates, tip current, surface current, voltage, line numberand other unused variables. We wish to construct 2Dmaps, which show the current (tip or surface depend-ing on experiment) distribution on xy plane. In orderto bring the data to a usable form we need to do somepreprocessing. We use MATLAB for the implementationof all the algorithms described below.

1. Data reduction

Data measurements are taken continuously during anexperiment, even if it is paused, therefore multiple mea-surements at the same point in space can occur. First,these measurements are averaged, what also reduces sizeof the data. An optional step, for line scans, is to only useinformation from the forward scan. This is not necessarybecause the regridding algorithm (described below) dealswith a mismatch between forward and backward scans orunevenly spaced data.

2. Filtering

Even with number of measures taken to minimize in-terference during the experiment (electrical, acoustic andvibrational isolation), there is still noise present in thedata. In order to reduce noise we use digital filtering.Even though data represents the dependency betweencurrent and a point in space, considering how measure-ments were taken, we treat it as a time series and apply1D Gaussian low pass filtering [21]. The Gaussian filtermodifies the input signal by convolution with a Gaussianfunction,

f(x) = exp(−x2

2σ2),

where standart deviation σ and kernel x size depend onthe experiment.

3. Regridding

To redistribute the non-gridded data ((a) in Figure 4)back to the grid of the desired image we use linear bino-mial interpolation. The height information for each datapoint is spread to the four nearest points on the grid ((b)in Figure 4). Furthermore, we attribute a weighting fac-tor to each point(shown in subsection C), which describes

4

FIG. 4: (a) Non-gridded position sensor data with thecolor of each square representing height values. (b) Todistribute the non-gridded data to the grid, the heightinformation of each data point is spread to the fournearest neighbors. Close proximity of the data point tothe pixel position leads to higher weights shown as sizeof the squares. Original data positions shown as dottedsquares. (c) Heat equation inpainting diffuses theexisting weighted data out to the entire grid fillingempty data points while denoising. (d) Final renderedimage. Reproduced from [15].

the confidence of the data, and is given by the distancefrom the data point to the grid. When only one datapoint contributes to the pixel the height value is simplycopied and the weighting saved for use in the inpaintingalgorithm. When more than one data point contributesto the same pixel, the weights are used to linearly in-terpolate height information from the contributing datapoints to determine the value and a composite weightingvalue is saved for the inpainting algorithm ((c),(d) in Fig-ure 4). Hence, for large data sets and coarse grids, thisfirst step might be sufficient to attribute a value to eachpixel and thereby generate a full image. However pixelsmight remain empty when sparse data sets are projectedon a fine grid. In this case, inpainting must be applied[15].

C. Inpainting

The digital inpainting process can be looked upon asa linear or non-linear transformation, where I0 is theoriginal image and I is the transformed image (i.e. thedigitally inpainted image). The image processor can belooked upon as a function f as follows:

f : I0 → I

FIG. 5: The image I, the region Ω to be inpainted andits boundary ∂Ω

Let Ω denote the set of pixels of the image I to be in-painted; this can be also called a mask. Let ∂Ω denotethe one pixel wide boundary of Ω (see Figure 5). Digitalinpainting regarding 3D objects resembles digital inpaint-ing of 2D images. Geometric partial differential equations(PDEs) are used to inpaint surface holes. However, in-stead of only working in two dimensions, the geometricPDEs may be used to inpaint surface holes in n dimen-sions [22], [23].

1. Linear interpolation

The easiest way to fill the region to be inpainted is touse a similar method to the regridding described above,i.e. linearly interpolate available information using in-verse distance weighting, not being bounded by just fournearest neighbors, but by a suitable radius R or theboundarie of the image. The mask is filled in the fol-lowing way:

Y =

∑I(X/Dp)∑I(1/Dp)

,

where D is the distance (in pixels) from the mask node

Y ∈ Ω to nodes outside the mask X ∈ Ω, all or withinradius R, and p is a positive real number, called the powerparameter. Greater values of p assign greater influenceto values closest to the interpolated point. Values insideΩ farther from ∂Ω will tend toward the average of all thevalues.

2. Heat equation inpainting

This algorithm is in widespread use [24]. We are us-ing the Laplace equation, which is the steady-state heatequation in 2 and 3 dimensions respectively:

• ∂2Ω∂x2 + ∂2Ω

∂y2 = 0

• ∂2Ω∂x2 + ∂2Ω

∂y2 + ∂2Ω∂z2 = 0

5

To reconstruct the image we solve this equation on Ωwith Dirichlet boundary conditions ∂Ω. To do so, wediscretize the PDE using finite differences for the partialderivatives. In other words, we replace any node in thegrid inside Ω by the average of its four neighbors. Wethen solve obtained linear system, which is typically wellconditioned, via LU decomposition.

3. Bertalmio inpainting

Bertalmio et al.[18] pioneered a digital image inpaint-ing algorithm based on partial differential equations(PDEs). A user-provided mask specifies the portionsof the input image to be retouched and the algorithmtreats the input image as three separate channels (R, Gand B). For each channel, it fills in the areas to be in-painted by propagating information from the outside ofthe masked region along level lines (isophotes). Isophotedirections are obtained by computing at each pixel alongthe inpainting contour a discretized gradient vector (itgives the direction of largest spatial change) and by ro-tating the resulting vector by 90 degrees. This intendsto propagate information while preserving edges. A 2-DLaplacian is used to locally estimate the variation in colorsmoothness and such variation is propagated along theisophote direction. After every few steps of the inpaint-ing process, the algorithm runs a number of diffusion iter-ations to smooth the inpainted region. Anisotropic diffu-sion is used in order to preserve boundaries across the in-painted region. Steady state is achieved if the smoothnessof the image (its second derivative) is constant along theisophotes. The assumption of constant smoothness alongisophotes is in general not justified. Since edges are con-tinued straightly into the inpainting domain, round ob-jects tend to develop straight segments meeting at acuteangles, thus producing kinks and neglecting the principleof continuation of direction.

Inpainting is viewed as an iterative process with thefollowing evolution equation, which runs only inside theregion to be inpainted:

In+1 = In(i, j)+ M tInt (i, j),∀(i, j) ∈ Ω,

where n is the current iteration, (i, j) are pixel coordi-nates, M t is the rate of improvement.

Int (i, j) =−−−−−−→∂Ln(i, j) ·

−−−−−→Nn(i, j),

where−−−−−−→∂Ln(i, j) is the measure of the change in the infor-

mation Ln(i, j),−−−−−→Nn(i, j) is the direction of propagation,

chosen to be isophote direction O⊥I(i, j). The steadystate is achieved, when Int (i, j) = 0. The discrete schemeis the following:

Int = (−−−−−−→∂Ln(i, j) ·

−−−−−→Nn(i, j)

|−−−−−→Nn(i, j)|

)|OIn(i, j)|,

−−→∂Ln(i, j) := (Ln(i+1, j)−Ln(i−1, j), Ln(i, j+1)−Ln(i, j−1))

Ln(i, j) = Inxx(i, j) + Inyy(i, j)

−→N (i, j, n)

|−→N (i, j, n)|

:=(−Iny (i, j), Inx (i, j))√

(Inx (i, j))2 + (Iny (i, j))2

βn(i, j) =−−−−−−→∂Ln(i, j) ·

−−−−−→Nn(i, j)

|−−−−−→Nn(i, j)|

,

and

|OIn(i, j)| =

√(Inxbm)2 + (InxfM )2 + (Inybm)2 + (InyfM )2,

whenβn > 0√(InxbM )2 + (Inxfm)2 + (InybM )2 + (Inyfm)2,

whenβn < 0

4. Anisotropic diffusion inpainting

The implementation of discrete anisotropic diffusionis based on the formulas shown in Perona and Malik’sliterature [19]. The underlying equation is the following:

It = div(c(x, y, t)OI) = c(x, y, t) M I + Oc · OI,

where c(x, y, t) = g(‖OI(x, y, t)‖). This equation can bediscretized on a square lattice in the following way, witha 4-nearest-neighbors discretization of the Laplacian op-erator:

In+1i,j = Ini,j +λ[cN ·ONI+cS ·OSI+cE ·OE +cW ·OW ]ni,j ,

where n is the current iteration, with 0 ≤ λ ≤ 1/4 forthe numerical scheme to be stable.

ONIi,j ≡ Ii−1,j − Ii, j

OSIi,j ≡ Ii+1,j − Ii, j

OEIi,j ≡ Ii,j+1 − Ii, j

OW Ii,j ≡ Ii,j−1 − Ii, j

cnXi,j= g(|OXI

ni,j |),

where X = N,S,E,W and g(OI) = e−(‖OI‖/K)2 org(OI) = 1

1+(‖OI‖K )2

. The first option for g privileges high-

contrast edges over low-contrast ones, whereas the secondprivileges wide regions over smaller ones [19].

6

III. RESULTS

In this section, we apply and compare the techniquesdescribed in the previous section and choose the mostsuitable one for the aim of the project.

A. Experiment in hopping mode

Here we test different image reconstruction methods ondata obtained from the experiment in the hopping mode,as described in previous section. In Figure 6 we see aSICM (see Section I) scan of polystyrene (partial filmwith holes) on a glass. The map reflects relative phaseof the ion current which corresponds to surface charge.The original image ((a) in Figure 6) was corrupted ((b)in Figure 6) to create the effect of sparse measurements.This allows a comparison between the original image andthe different inpainting methods introduced in Section II((c)-(f) in Figure 6).

All schemes produce a good approximation of the ini-tial image, considering that about 50 percent of the datawas removed, and we can clearly distinguish the borderof the hole in the polystyrene. In this case, heat equa-tion based inpainting gives the best result: it producesthe image closest to original (in terms of least-squared er-ror).The reason for this might be due to the relationshipbetween the heat equation and the underlying diffusionprocesses during the experiment which is described byFick’s law of diffusion [25]. It is also more practical whencompared with the Bertalmio et. al. algorithm and thePerona-Malik algorithm, because the equation doesn’thave a time term, i.e. does not need to take multiplesteps to converge, therefore it works faster; and unlikethe former two algorithms it does not have parameterson which its performance strongly depends. Bertalmioet. al. algorithm showed itself to be numerically unsta-ble as Bertalmio et. al. discussed [18]. The outcome ofthis experiment gives us an opportunity to speed up thescanning process and to cover a wider area by makingsparse measurements in hopping mode and then restor-ing the rest of the image.

B. Experiment in line scanning mode

Figure 7 shows data from the experiment usingSECCM (see Section II) in line scanning mode: patternin which the tip makes dense measurements in the Xdirection, but leaves large gaps in the Y direction (seeSection II). This scan is of a single-wall carbon nanotube[26],[27]. Carbon nanotubes (CNTs) are allotropes ofcarbon with a cylindrical nanostructure. Their nameis derived from their long, hollow structure with thewalls formed by one-atom-thick sheets of carbon, calledgraphene. The map shows the growth of tip currentwhere the reaction occurs between tip and nanotube,in contrast with an inert substrate. In Figure 7 we see

(a) (b)

(c) (d)

(e) (f)

FIG. 6: (a) Original image; (b) Corrupted image; (c)Inverse distance weighting inpainting; (d) Heat equationinpainting; (e) Bertalmio inpainting; (f) Perona - Malikinpainting.

the original image (a), on which all 20 lines stack to-gether without any explicit interpolation, and results oflinear interpolation (b), Bertalmio et al. inpainting (c),anisotropic diffusion based algorithm (d) and several heatequation inpainting images (f), (h), (j). In this case lin-ear interpolation does not give any benefit when com-pared to the original image. The Bertalmio et al. algo-rithm reveals difficulties in tuning parameters and doesnot produce a smooth result. The anisotropic diffusionand heat equation algorithms work comparably well, butthe heat equation algorithm is preferable due to its ad-vantages in speed and simplicity. Figure 7 (c)-(j) showsresults for various grid sizes, i.e. different gaps betweenlines in slow scan direction. Grid size 200×80 gives goodresults, whereas grid size 200×120 shows artifacts of theinpainting process, therefore limitations in choice of gridsize are always case-dependent. Overall each method hasdifficulties in preserving sharp/smooth edges across largegaps, which is still one of the main research directions for

7

the field of inpainting [28].

C. Experiment in spiral scanning mode

In the experiment using SICM (see Section II), fastspiral scans are used to detect deposition of Plat-inum nanoparticles on HOPG (Highly Ordered PyrolyticGraphite). Figure 8 shows Field Emission Scanning Elec-tron Microscopy (FESEM), discussed in [29], scan of thissurface.

Platinum oxidizes hydrazine, which changes the localconductivity. This process can be detected using SICM.Use of an Archimedean spiral results in a slight rotationin the scan direction. Red color on Figure 9 (a)-(f) rep-resents the highest intensity of tip current, defining theregion with Platinum nanoparticles.

Figure 9 (a), (c), (e) shows images of experimentaldata redistributed on square grids of size 400, 800, 1200respectively. Figure 9 (b), (d), (f) shows images recon-structed using heat equation. Note that, inpainting dif-fuses data to the edges of the square grid circumscribingthe collected data and the pixels outside the scan regiondo not accurately depict sample properties. Heat equa-tion restoration gives impressive results for different gridsizes in this case.

D. 3D inpainting

Dynamic evolution of surface processes can be ob-served using a combination of the fast scanning techniqueand inpainting in 3 dimensions. This is the same exper-iment as for the SICM spiral scan detecting platinumnanoparticles from the previous subsection, however weare now looking at the experiment as a process ratherthan just one spiral. The experiment consists of a seriesof spiral scans with the voltage growing linearly, which al-lows us to see variation in reaction rate and finally, whenvoltage is high enough, to detect platinum nanoparticles.Thus, one spiral represents the experiment at differentvoltage values, in other words, one map does not repre-sent the surface at one constant voltage value. But it ispossible, using digital inpainting in 3D, to reconstruct theprogress of the experiment as a video, where each frame isa surface at one constant voltage value. For this purpose,methods B and C from Section II should be extended tothe 3D case. Data reduction and filtering are done in thesame way, as data acquired as time series. Speaking ofregridding, we divide each spiral into 10 parts, which willbecome 10 frames, assuming that voltage is constant ateach piece, and then redistribute each part to a 2D grid.This was dictated by limitations of computational powerand memory, but it is possible to create an analogue in3D for the 2D regridding algorithm from Section II. Theresult, which depends on the size of the grid, is expectedto be similar.To fill large gaps on each frame we solve the heat equa-

(a) (b)

(c) (d)

(e) (f)

(g) (h)

(i) (j)

FIG. 7: (a) Original image; (b) Inverse distanceweighting inpainting; (c) Bertalmio inpainting; (d)Perona - Malik inpainting; (e) Image with gaps, gridsize 200 × 80; (f) Heat equation inpainting for (e); (g)Image with gaps, grid size 200 × 100; (h) Heat equationinpainting for (g); (i) Image with gaps, grid size 200 ×120; (j) Heat equation inpainting for (i).

8

(a) (b)

(c) (d)

(e) (f)

FIG. 8: (a)Image with gaps, grid size 400 ×400;(b)Heat equation inpainting for (a);(c)Image withgaps, grid size 800 × 800;(d)Heat equation inpaintingfor (c); (e)Image with gaps, grid size 1200 ×1200;(f)Heat equation inpainting for (e).

tion in 3D, using information from all of the other framesas well.Results of 3D inpainting are publicly available, links pro-vided in Section VI Supporting materials.

IV. CONCLUSIONS

We investigated the scope for the application of dig-ital inpainting to the field of Electrochemical ScanningProbe Microscopy imaging. We examined different PDE-based image restoration methods on images collected byseveral electrochemical scanning probe microscopy tech-niques and chose the most suitable one. We found this tobe heat equation inpainting, based on accuracy and over-all performance, high speed and absence of parameters.Protocols and code were developed to automatically gen-erate 2D maps. Along with the advantages and the good

quality results provided by the tested method, we havenoted the limitations of this technique. The quality ofthe results depends on choosing an optimal value for thegrid size. We also note that the ability to preserve edgesover large gaps is another constraint and needs to be im-proved through further work. Overall, our experimentsshow that inpainting techniques can represent a solu-tion in several scenarios common for microscopy imaging,such as the presence of artifacts, and acquisition impos-sibility for particular sample regions. This approach willfacilitate further work in the area of microscopy imaging.

V. ACKNOWLEDGMENTS

This research was funded by the EACEA. Many thanksfor experimental data and support provided by DmitryMomotenko, Joshua Byers, Changhui Chen, Leon Ja-cobse and David Perry from Electrochemistry and In-terfaces group, University of Warwick.

VI. SUPPORTING MATERIALS

Results of 3D inpainting are available to downloadfrom the following links in .gif format:

a) Before inpaintinghttps://www.dropbox.com/s/r3saagpoi3b9ez9

/3dNaN200.gif;b) After inpaintinghttps://www.dropbox.com/s/0v8igfdaes3689n

/3dinp.gif

9

VII. REFERENCES

[1] Edington J. W. Practical electron microscopy inmaterials science. Van Nostrand Reinhold Com-pany, 1976. . 207.

[2] Hansma P. K. et al. Scanning tunneling microscopyand atomic force microscopy: application to biol-ogy and technology //Science. 1988. . 242. .4876. . 209-216.

[3] Evans C. L., Xie X. S. Coherent anti-Stokes Ra-man scattering microscopy: chemical imaging forbiology and medicine //Annu. Rev. Anal. Chem.2008. . 1. . 883-909.

[4] Parot P. et al. Past, present and future of atomicforce microscopy in life sciences and medicine//Journal of Molecular Recognition. 2007. . 20.. 6. . 418-431.

[5] Zhou W., Wang Z. L. Scanning microscopyfor nanotechnology: techniques and applications.springer, 2007.

[6] Bhushan B., Fuchs H., Tomitori M. Applied scan-ning probe methods VIII: Scanning probe mi-croscopy techniques. Springer, 2007. . 8.

[7] Chen C. C., Zhou Y., Baker L. A. Scanning ionconductance microscopy //Annual Review of Ana-lytical Chemistry. 2012. . 5. . 207-228.

[8] Ebejer N. et al. Scanning electrochemical cell mi-croscopy: a versatile technique for nanoscale elec-trochemistry and functional imaging //Annual Re-view of Analytical Chemistry. 2013. . 6. .329-351.

[9] Wieczorowski M. Spiral sampling as a fast way ofdata acquisition in surface topography //Interna-tional Journal of Machine Tools and Manufacture.2001. . 41. . 13. . 2017-2022.

[10] Kotsopoulos A. G., Antonakopoulos T. A. Nanopo-sitioning using the spiral of archimedes: The probe-based storage case //Mechatronics. 2010. . 20. .2. . 273-280.

[11] Hung S. K. Spiral scanning method for atomic forcemicroscopy //Journal of nanoscience and nanotech-nology. 2010. . 10. . 7. . 4511-4516.

[12] Mahmood I. A., Moheimani S. O. R. Fast spiral-scan atomic force microscopy //Nanotechnology.2009. . 20. . 36. . 365503.

[13] Yong Y. K., Moheimani S. O. R., Petersen I. R.High-speed cycloid-scan atomic force microscopy//Nanotechnology. 2010. . 21. . 36. . 365503.

[14] Tuma T. et al. High-speed multiresolution scan-ning probe microscopy based on Lissajous scan tra-jectories //Nanotechnology. 2012. . 23. . 18. .185501.

[15] Ziegler D. et al. Improved accuracy and speed inscanning probe microscopy by image reconstructionfrom non-gridded position sensor data //Nanotech-nology. 2013. . 24. . 33. . 335703.

[16] Binnig G., Quate C. F., Gerber C. Atomic forcemicroscope //Physical review letters. 1986. . 56.. 9. . 930.

[17] Stanciu S. G., Hristu R., Stanciu G. A. Digitalimage inpainting and microscopy imaging //Mi-croscopy research and technique. 2011. . 74.. 11. . 1049-1057.

[18] Bertalmio M. et al. Image inpainting //Pro-ceedings of the 27th annual conference on Com-puter graphics and interactive techniques. ACMPress/Addison-Wesley Publishing Co., 2000. .417-424.

[19] Perona P., Shiota T., Malik J. Anisotropic dif-fusion //Geometry-driven diffusion in computervision. Springer Netherlands, 1994. . 73-92.][REFERENCE Perona P., Malik J. Scale-spaceand edge detection using anisotropic diffusion//Pattern Analysis and Machine Intelligence, IEEETransactions on. 1990. . 12. . 7. . 629-639.

[20] in preparation , Unwin P.[21] Antoniou A. Digital signal processing. Toronto,

Canada: : McGraw-Hill, 2006.[22] Verdera J. et al. Inpainting surface holes. 2003.[23] Davis J. et al. Filling holes in complex surfaces us-

ing volumetric diffusion //3D Data Processing Vi-sualization and Transmission, 2002. Proceedings.First International Symposium on. IEEE, 2002. .428-441.

[24] Aubert G., Kornprobst P. Mathematical problemsin image processing: partial differential equationsand the calculus of variations. Springer, 2006. .147.

[25] Fick A. V. On liquid diffusion //The London, Ed-inburgh, and Dublin Philosophical Magazine andJournal of Science. 1855. . 10. . 63. . 30-39.

[26] Ajayan P. M., Zhou O. Z. Applications of carbonnanotubes //Carbon nanotubes. Springer BerlinHeidelberg, 2001. . 391-425.

[27] Gell A. G. et al. Quantitative nanoscale visual-ization of heterogeneous electron transfer rates in2D carbon nanotube networks //Proceedings of theNational Academy of Sciences. 2012. . 109. . 29.. 11487-11492.

[28] Alvarez L., Lions P. L., Morel J. M. Image selec-tive smoothing and edge detection by nonlinear dif-fusion. II //SIAM Journal on numerical analysis.1992. . 29. . 3. . 845-866.

[29] Huggett J. M., Shaw H. F. Field emission scanningelectron microscopy; a high-resolution technique forthe study of clay minerals in sediments //Clay Min-erals. 1997. . 32. . 2. . 197-203.