Journal of Neuroscience Methods 174 (2008) 106–115

Contents lists available at ScienceDirect

Journal of Neuroscience Methods

journa l homepage: www.e lsev ier .com/ locate / jneumeth

Localization of neurosurgically implanted electrodes viaphotograph–MRI–radiograph coregistration

Sarang S. Dalal a,b, Erik Edwardsa,e, Heidi E. Kirschd, Nicholas M. Barbaroc,d,Robert T. Knighte, Srikantan S. Nagarajana,b,∗

a Biomagnetic Imaging Laboratory, Department of Radiology & Biomedical Imaging, University of California, San Francisco, CA 94143-0628, USAb Joint Graduate Group in Bioengineering, University of California, San Francisco and University of California, Berkeley, USAc Department of Neurological Surgery, University of California, San Francisco, CA 94143-0112, USAd Department of Neurology, University of California, San Francisco, CA 94143-0138, USAe Helen Wills Neuroscience Institute and Department of Psychology, University of California, Berkeley, CA 94720-3190, USA

a r t i c l e i n f o

Article history:Received 8 April 2008Received in revised form 18 June 2008Accepted 18 June 2008

Keywords:Subdural electrodesIntracranial electroencephalographyElectrocorticographyEpilepsy surgery3D localizationCoregistrationSurface renderingX-rayCT

a b s t r a c t

Intracranial electroencephalography (iEEG) is clinically indicated for medically refractory epilepsy andis a promising approach for developing neural prosthetics. These recordings also provide valuable datafor cognitive neuroscience research. Accurate localization of iEEG electrodes is essential for evaluatingspecific brain regions underlying the electrodes that indicate normal or pathological activity, as well asfor relating research findings to neuroimaging and lesion studies. However, electrodes are frequentlytucked underneath the edge of a craniotomy, inserted via a burr hole, or placed deep within the brain,where their locations cannot be verified visually or with neuronavigational systems. We show that oneexisting method, registration of postimplant computed tomography (CT) with preoperative magneticresonance imaging (MRI), can result in errors exceeding 1 cm. We present a novel method for localizingiEEG electrodes using routinely acquired surgical photographs, X-ray radiographs, and magnetic resonanceimaging scans. Known control points are used to compute projective transforms that link the differentimage sets, ultimately allowing hidden electrodes to be localized, in addition to refining the locationof manually registered visible electrodes. As the technique does not require any calibration betweenthe different image modalities, it can be applied to existing image databases. The final result is a set of

MRI electrode positions on the patient’s rendered MRI yielding locations relative to sulcal and gyral landmarkson individual anatomy, as well as MNI coordinates. We demonstrate the results of our method in eight

ed wi

1

pnpirdrhtp

a(2

rnp(e

0d

epilepsy patients implant

. Introduction

Intracranial electroencephalography (iEEG) is indicated inotential neurosurgical patients when noninvasive diagnostic tech-iques prove inconclusive (Bancaud et al., 1965). Most of theseatients have medically intractable epilepsy, requiring long-term

nvasive monitoring to localize seizure foci as well as to preventesection of critical brain areas that would result in cognitiveeficits or paralysis (Wyler et al., 1988; Lesser et al., 1991). These

ecordings also provide rare but highly valuable data to test basicypotheses in neurophysiology and cognitive neuroscience. Fur-hermore, iEEG is a promising avenue for the development of neuralrostheses designed to aid patients with brain or spinal cord dam-

∗ Corresponding author. Tel.: +1 415 476 4982; fax: +1 415 502 4302.E-mail address: sri@radiology.ucsf.edu (S.S. Nagarajan).

moskeh

m

165-0270/$ – see front matter © 2008 Elsevier B.V. All rights reserved.oi:10.1016/j.jneumeth.2008.06.028

th electrode grids spanning the left hemisphere.© 2008 Elsevier B.V. All rights reserved.

ge resulting from trauma, stroke, or neurodegenerative diseasesLeuthardt et al., 2006; Santhanam et al., 2006; Hochberg et al.,006).

Accurate localization of iEEG electrodes is critical for planningesective surgery as well as for relating iEEG findings to the largereuroimaging literature. Direct visual observation of electrodeositions, perhaps supplemented by neuronavigational systemsBarnett et al., 1993) or recorded with digital photography (Wellmert al., 2002; Mahvash et al., 2007), provides the most reliable infor-ation. However, many electrodes are tucked underneath the edge

f a craniotomy or guided into locations deep within the brain, ando their final locations relative to brain anatomy are not precisely

nown since they cannot be visually verified. In some procedures,lectrodes may be inserted into the subdural space via a small burrole.

Postimplant radiographs (X-rays) are routinely ordered atost epilepsy centers, and while they are high-resolution, they

scienc

autae2ip

pHasCnoNCt(ndfmds

aefBi1etp

iarrAarir

2

2

wncCStcRwirm

es

2

Fag0a

acTc

2

wfwfB

2

t(gcbcatrgjoarfhboa

ftFrmuba

u

S.S. Dalal et al. / Journal of Neuro

re nevertheless flat 2D projections that give little indication ofnderlying brain anatomy. They are typically used to visualizehe approximate extent and curvature of the implanted electroderrays. Simple measurements can be performed to estimate roughlectrode positioning on anatomy (Fox et al., 1985; Miller et al.,007). However, variable magnification and distortion as well as

nherent variability in anatomy and X-ray configuration betweenatients may reduce the accuracy of these measurements.

Many epilepsy centers obtain postimplant computed tomogra-hy (CT) scans to visualize the 3D configuration of the electrodes.owever, CT has poor soft tissue contrast and brain structuresre very difficult to discern, especially in the presence of severetreaking artifacts caused by the electrodes themselves. Therefore,T scans must be coregistered to preoperative magnetic reso-ance imaging (MRI) scans in order to visualize electrode positionsn brain anatomy (Grzeszczuk et al., 1992; Winkler et al., 2000;elles et al., 2004; Hunter et al., 2005). However, the resolution ofT is often poor, resulting in partial volume effects. Additionally,he brain may shift considerably following the electrode implantHastreiter et al., 2004; Miyagi et al., 2007), contributing to sig-ificant errors in the coregistered locations of electrodes. Theseisadvantages are compounded by the fact that CT can be uncom-ortable and sometimes risky for implanted epilepsy patients, and

onitoring must be stopped during the exam. Finally, head CTelivers approximately 200 times the radiation dose of a lateralkull radiograph (Wall and Hart, 1997).

Some groups acquire postimplant MRI scans, which can visu-lize brain anatomy with high fidelity. Unfortunately, manylectrodes can be difficult to see on them (Bootsveld et al., 1994);urthermore, from MRI scans published in the literature (Schulze-onhage et al., 2002; Kovalev et al., 2005; Soch�urková et al., 2006),

t seems likely that electrodes on arrays with spacing tighter thancm would be indistinguishable. Susceptibility artifacts from thelectrode material may also distort the images. MRI is also rela-ively expensive, and, as with CT, many epilepsy centers considerostimplant MRIs to be unjustifiably risky.

We present a procedure and algorithm for localizing electrodesmplanted into the brains of neurosurgery patients and for inter-ctively linking a 3D preoperative MRI with a 2D postimplantadiograph and surgical photographs. The method provides a moreobust estimate of electrode positions than previously possible.dditionally, we use the transformation matrices to create an inter-ctive viewer that links coordinates on the MRI, photograph, andadiograph; this both verifies goodness of fit as well as assistsdentification of other structural features on the radiograph by cor-esponding them to 3D locations on the patient’s MRI.

. Methods

.1. Patients and electrodes

Eight intractable epilepsy patients (referred to as GP1–GP8)ere implanted with subdural electrodes for the purpose of plan-ing resective surgery. Electrode implants were guided strictly bylinical indications and research recordings were approved by theommittees on Human Research at the University of California,an Francisco and the University of California, Berkeley. The elec-rodes were 4 mm diameter platinum–iridium disks with a 2.3 mmontact width, embedded within a Silastic sheet (Ad-Tech Medical,

acine, WI, USA). The center-to-center spacing between electrodesas 10 mm. Grid electrodes consisted of 64 electrodes arranged

n an 8 × 8 square and placed on the surface of left frontotempo-al cortex in all patients. Electrode grids were sewn to the dura toinimize movement after placement. Supplemental 4 × 1 or 6 × 1

t

P

e Methods 174 (2008) 106–115 107

lectrode strips (10 mm spacing) were also placed on the corticalurface in some patients.

.2. Medical images

High-resolution T1-weighted MRI scans were acquired with aSPGR sequence at a resolution of 0.43 mm × 1.5 mm × 0.43 mm on3 T scanner (GE, Milwaukee, WI, USA). Lateral skull film radio-

raphs were obtained in 6 patients, and digitized at approximately.15 mm × 0.15 mm. Postimplant CT scans were acquired on GP1nd GP2 at a resolution of 0.43 mm × 0.43 × 3.75 mm.

Digital photographs were taken of the craniotomy both prior tond following placement of the electrodes using a consumer-gradeamera (except GP1, who only had a post-placement photograph).he camera was handheld roughly 50 cm away from the exposedortex and oriented approximately orthogonal to it.

.3. MRI processing

Skull and scalp tissue were stripped from the structural MRIsith the Brain Extraction Tool (Jenkinson et al., 2005, http://www.

mrib.ox.ac.uk/analysis/research/bet/). This segmented volumeas used to determine the coordinates of points on the brain sur-

ace as well as create rendered brains with MRIcro (Rorden andrett, 2000, http://www.sph.sc.edu/comd/rorden/mricro.html).

.4. Registration of surgery photographs

Two photographs are typically taken during surgical implan-ation of the electrode array—one showing the exposed brainanatomical photograph, Fig. 1B) and one showing the electroderid on top of the brain (grid photograph, Fig. 1A). The plastic andabling of the electrode grid partially occludes the view of therain anatomy underneath it; therefore, it is useful to transfer theoordinates of the electrodes seen in the grid photograph to thenatomical photograph. Fortunately, matching two photographs ofhe same scene taken with an uncalibrated camera is a 2D homog-aphy problem that is well-studied in the field of computer vision;iven at least four matched point pairs called control points, a pro-ective transform may be computed to map any given point fromne photograph to the other (Abdel-Azziz and Karara, 1971; Hartleynd Zisserman, 2004). The projective transform automatically cor-ects for the inherent change in perspective and distance expectedrom two photographs taken at different moments with a hand-eld camera. In the case at hand, several control points can easilye chosen on the two photographs, including prominent featuresf blood vessels, distinct corners of the craniotomy, fixed sutures,nd other stationary surgical hardware.

Given N pairs of control points with coordinates ai ≡ (xi, yi)rom the first photograph and bi ≡ (ui, vi) from the second pho-ograph, the projective transform may be computed as follows:irst, the coordinates are normalized with transforms T1 and T2,espectively, such that the centroid for each group is (0,0) and theean distance from the origin is

√2 for each set of points. Let

s define ai ≡ T1[ai1]T and bi ≡ T2[bi1]T, with ai = (xi, yi, 1) and˜

i = (ui, vi, 1). The coordinate vectors are augmented with unity asscaling factor, which will become important later.

Then, the discrete linear transformation (DLT) algorithm can besed to determine the 3 × 3 projective transform P that best maps a

o b (Abdel-Azziz and Karara, 1971; Hartley and Zisserman, 2004):

˜ =[

p11 p12 p13p21 p22 p23p31 p32 1

](1)

108 S.S. Dalal et al. / Journal of Neuroscience Methods 174 (2008) 106–115

Fig. 1. Complete procedure for registration of electrodes from photographs, MRI, and X-ray. (A) Photograph showing position of grid implant. Cables entering the bottomleft and bottom right corners of the craniotomy connect to electrode strips. (B) Photograph taken immediately before grid placement clearly showing anatomical features.A projective transform with the photograph in (A) is computed with several manually selected point correspondences, such as blood vessel features, craniotomy edges, andstationary surgical hardware. The transform were then applied to electrode positions from (A) and superimposed. (C) MRI-based brain rendering with electrode positionstranscribed from the photographs. (D) Lateral radiograph showing electrode positions (dark dots) along with electrode cabling. The green circles outline the electrodes with3D locations known from the previous photograph–MRI registration step; these two sets of coordinates were used as control pairs to generate the MRI-radiograph projectivetransform. (E) The projective transform can be used to compute the location of the X-ray source and trace the path of X-rays that generated the image for each electrodelocation. Surface electrodes are known to be at the intersection of these rays with the cortical surface. (F) The final rendering showing electrode positions confirmed by thephotograph (in white) and those determined solely by backprojection (grid electrodes in blue, strip electrodes in yellow). Note that the strip electrodes A1–A4 and B1–B4are difficult to see at the contrast level chosen for the shown radiograph. (For interpretation of the references to color in this figure legend, the reader is referred to the webversion of the article.)

scienc

w2

⎡⎢⎢⎢⎢⎣

iit

P

fis[

eyit

bvwtpTett

2

boeats

2

t“aftsabFra

tpdag

tpKwtyutapbct

appantbfca

(twdw2a

ta

P

w2

⎡⎢⎢⎣

A

S.S. Dalal et al. / Journal of Neuro

here the entries in P can be determined by solving the followingN × 8 system:

x1 y1 1 0 0 0 −u1x1 −u1y10 0 0 x1 y1 1 −v1x1 −v1y1...

......

......

......

...xN yN 1 0 0 0 −uN xN −uN yN

0 0 0 xN yN 1 −vN xN −vN yN

⎤⎥⎥⎥⎥⎦

⎡⎢⎢⎢⎢⎢⎢⎢⎢⎣

p11p12p13p21p22p23p31p32

⎤⎥⎥⎥⎥⎥⎥⎥⎥⎦

=

⎡⎢⎢⎢⎢⎢⎢⎢⎣

u1v1u2v2...

uN

vN

⎤⎥⎥⎥⎥⎥⎥⎥⎦

(2)

When N > 4, the system is overdetermined, but may be solvedn a least squares sense using singular value decomposition. Then,n order to apply the transform to the original coordinate spaces,he normalization transforms must be applied to P:

≡ T−12 PT1 (3)

Finally, P may be applied to any point m ≡ (mx, my) from therst image to estimate the corresponding point n ≡ (nu, nv) on theecond image:

knu

knv

k

]= P

[mx

my

1

](4)

Note that after applying the transform to (mx, my), the first twolements of the output vector must be divided by the third, k, toield (nu, nv). This effectively removes any perspective distortion;n fact, if P defines an affine transform, then the mapping is distor-ionless by definition and k = 1.

The transform requires matching of fixed landmarks (fiducials)etween the two photographs; in this application, features onisible portions of the brain, blood vessels, and fixed surgical hard-are may be used as landmarks for control points. The computed

ransform can then estimate the coordinates on the anatomicalhotograph corresponding to any point from the grid photograph.herefore, the transform is applied to the coordinates of all visiblelectrodes in the grid photograph. This method does not requirehat the two photographs be taken with identical parameters (e.g.,he angle, distance, and zoom/focus settings need not be the same).

.5. Photograph–MRI registration

The locations of visible electrodes from the photograph can thene manually annotated on a high-resolution MRI surface renderingf the patients brain. This yields a set of 3D coordinates of knownlectrode positions in MRI space (Fig. 1C). This process may bessisted with the use of a 3D–2D projective transform (see Sec-ion 2.6), or a feature matching algorithm may help automate thistep (e.g., Yuan et al., 2004).

.6. MRI–radiograph registration

The projective transform used above to register two pho-ographs of the same scene may be extended to a more generalcamera” problem (Abdel-Azziz and Karara, 1971). A photograph isprojection of 3D “world” points onto a 2D image plane; light rays

rom the photographed scene converge onto a focal point behindhe image plane (film or digital sensor). Note that radiography is apecial case of the camera problem in which the focal point is actu-

lly the X-ray source; the X-rays then diverge before being absorbedy the imaged object or the film/sensor (image plane) behind it.ortunately, most camera geometry principles and formulae areeadily generalizable to the case of radiography with the properssignment of polarities to various parameters. Thus, in our case,

sst

P

e Methods 174 (2008) 106–115 109

he 3D world data is represented by the MRI and the photographicrojection is represented by the X-ray image. It should be noted thatue to the fan beam configuration of typical clinical X-ray sources,n affine transform – which assumes parallel rays – would result inreater registration errors.

Given at least six control points matching 3D world locationso a 2D image plane, a 3 × 4 projective transform may be com-uted to map any 3D location to the image (Abdel-Azziz andarara, 1971; Hartley and Zisserman, 2004). In practice, ten or moreell-distributed control points are needed to generate a reliable

ransform. It is difficult to find that many uniquely identifiableet nonplanar point correspondences given a standard MRI andncalibrated radiograph. Other approaches require calibration ofhe X-ray system (including precise measurement of the positionnd orientation of the X-ray source relative to the patient) and/orlacement of additional external markers that can be observed inoth sets of images (e.g., Gutiérrez et al., 2005). These methods areumbersome to implement in clinical practice and cannot work forypical existing data.

However, electrodes are clearly visible on radiographs (Fig. 1D)nd many of their 3D positions are known from the abovehotograph–MRI registration. Therefore, several known 3D–2Doint correspondences exist from the visible electrodes. With theddition of some potentially clear anatomical landmarks (e.g.,asion, auditory canals, teeth, and upper vertebrae), enough con-rol points can be defined to create a viable projective transformetween the MRI and the radiograph. As a sanity check, the trans-orm can confirm the location of the X-ray source, which in a typicallinical setting is 100 cm from the head. The transform can then bepplied to the X-ray positions of the electrodes.

We start with N pairs of control points with coordinates ai ≡xi, yi, zi) from the MRI and bi ≡ (ui, vi) from the radiograph. As withhe 2D projective transform, the coordinates should be normalizedith transforms T1 and T2, respectively, such that each set of coor-inates has a centroid that falls on the origin and is distributedith a mean distance of

√3 for the 3D coordinates and

√2 for the

D coordinates. Let us define ai ≡ T1[ai1]T and bi ≡ T2[bi1]T, with˜ i = (xi, yi, zi) and bi = (ui, vi).

An expanded form of the DLT algorithm can be used to determinehe 3 × 4 projective transform P that best maps a to b (Abdel-Azziznd Karara, 1971; Hartley and Zisserman, 2004):

˜ ≡[

p11 p12 p13 p14p21 p22 p23 p24p31 p32 p33 1

](5)

here the entries in P can be determined by solving the followingN × 11 system:

x1 y1 z1 1 0 0 0 0 −u1 x1 −u1y1 −u1 z1

0 0 0 0 x1 y1 z1 1 −v1 x1 −v1y1 −v1 z1

.

.

....

.

.

....

.

.

....

.

.

....

.

.

....

.

.

.xN yN zN 1 0 0 0 0 −uN xN −uN yN −vN zN

0 0 0 0 xN yN zN 1 −vN xN −vN yN −vN zN

⎤⎥⎥⎦

⎡⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣

p11

p12

p13

p14

p21

p22

p23

p24

p31

p32

p32

p33

⎤⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦

=

⎡⎢⎢⎢⎢⎢⎣

u1

v1

u2

v2

.

.

.uN

vN

⎤⎥⎥⎥⎥⎥⎦ (6)

gain, in practice the system will be overdetermined (given N > 6),

o singular value decomposition can be used to find the leastquares solution. Next, P is denormalized so that it may be appliedo the original coordinate spaces:

≡ T−12 PT1 (7)

1 scienc

e

[

ta

edaX

M

a

p

s

f

a(

stawt1pa

tgidpagpZ

l

waj

crc

aptsap

smcbX

2

3sIWnMdMiDmtmici

2

Mettraroama

2

2

usbttarst4npfu

10 S.S. Dalal et al. / Journal of Neuro

Finally, P may be applied to any MRI point m ≡ (mx, my, mz) tostimate the corresponding point on the radiograph r ≡ (ru, rv):

kru

krv

k

]= P

⎡⎢⎣

mx

my

mz

1

⎤⎥⎦ (8)

As with the 2D–2D case, the first two elements of the output vec-or must be divided by the third, k, to remove projective distortionnd obtain (ru, rv).

An interesting property of the projective transform P is that thestimated position of the focal point f (i.e., the X-ray source) may beirectly calculated from it. For the present application, this servess a sanity check to ensure that the transform is consistent with the-ray acquisition specifications. Let us define:

≡[

p11 p12 p13p21 p22 p23p31 p32 p33

](9)

nd

4 ≡ [p14 p24 p34]T (10)

uch that P = [M p4]. Then, the focal point is known to be

= −M−1p4 (11)

Indeed, if M is singular, ||f|| = ∞ and therefore P would define anffine transform, which assumes parallel rays with no convergencei.e., an infinite focal length).

In practice, the DLT method for 3D–2D registration is somewhatensitive to noise and inaccuracy in the given control points, andhe computed X-ray source location is subject to some uncertaintylong the axis perpendicular to the X-ray image plane. However,ith only knowledge of the distance and approximate direction of

he X-ray source, Levenberg–Marquardt optimization (Levenberg,944; Marquardt, 1963) may be used to adjust the displacementarameters in the normalization transform T1 to ensure a reason-ble estimate of f and, by extension, P.

Applying the projective transform P maps every 3D MRI pointo a 2D radiograph point. We are interested in mapping the radio-raph points to the MRI as well, but since P is a 3 × 4 matrix, P−1

s not defined. However, a form of the inverse projective transformoes exist! Intuitively, all points on a line passing through the focaloint f must map to a single point on the radiograph. Therefore,pplying the inverse projective transform to a point on the radio-raph should backproject to a line in MRI space. For a non-affinerojective transform, the line equation is known to be (Hartley andisserman, 2004):

(r, �) = M−1

(�

[rT

1

]− p4

)(12)

here r ≡ (ru, rv) is the radiograph coordinate to be backprojected,nd � is a parameter that determines position along the backpro-ected line.

If the electrodes are known to be on the brain surface, then theiroordinates are easily found by backprojecting each electrode’s X-ay location and finding the intersection with the set of MRI pointsomprising the brain outline (Fig. 1E and F).

It should be noted that large backprojection errors may arise inreas where the brain curves sharply away from the X-ray image

lane. For example, when backprojecting to the bottom of theemporal lobe from a lateral radiograph, small errors along theuperior–inferior axis of the brain may result in large displacementlong the left–right axis. Additionally, electrodes are sometimeslaced at depth within the brain. In these cases, known electrode

2

ora

e Methods 174 (2008) 106–115

pacing can be used to constrain the solution. Alternatively, ifultiple X-ray views are available, projective transforms can be

omputed for each of them; these electrodes can then be localizedy computing the intersection of their backprojections from each-ray view.

.7. Transformation to canonical coordinate system

Given a set of electrode positions on a patient’s individualD MRI space, it is quite straightforward to transform them to atandard coordinate system such as the Montreal Neurologicalnstitute (MNI) template (Evans et al., 1993; Mazziotta et al., 2001).

e used SPM2 (http://www.fil.ion.ac.uk/spm) to generate theonlinear spatial transformation function using the preoperativeRI. The transform was then applied to the electrode coordinates

etermined from X-ray backprojection to obtain correspondingNI coordinates. After conversion to Talairach coordinates (http://

maging.mrc-cbu.cam.ac.uk/imaging/MniTalairach), the Talairachaemon (http://ric.uthscsa.edu/projects/talairachdaemon.html)ay be used to determine corresponding anatomical labels from

he Talairach atlas (Talairach and Tournoux, 1988). Note that sinceany epilepsy patients have unusual brain anatomy, it is especially

mportant to use spatial normalization only to obtain standardizedoordinates; functional activations should be rendered on anndividual’s MRI rather than a standard template.

.8. Interactive navigation

Another 3D–2D projective transform can be synthesized to mapRI surface points onto the anatomical photograph, using the

lectrode positions from Section 2.5 and/or distinct anatomical fea-ures as control points. We have created an interactive navigationool that uses this MRI-photograph transform along with the MRI-adiograph transform to allow a user to select any point on the MRInd immediately see that point’s corresponding projection on theadiograph and photograph (Fig. 2). Conversely, selecting a pointn the radiograph defines a corresponding point on the photographnd a line (i.e., the X-ray path) on the MRI. Finally, if MNI spatial nor-alization has been performed, corresponding MNI coordinates

nd anatomical labels can be displayed.

.9. Performance evaluation

.9.1. Accuracy of MRI–radiograph projective transformTo evaluate the performance of the proposed technique, partic-

larly for electrode locations that are not used as control points, weelected one patient (GP6) for further analysis. Since it is not possi-le to definitively validate the computed coordinates for electrodeshat are located beyond extent of the craniotomy, another projec-ive transform was generated using six of the 47 visible electrodes,long with four anatomical fiducials (superior orbital ridge, left andight auditory canals, and the vertex). This effectively simulates acenario with a smaller craniotomy that exposes only these six elec-rodes, providing a means to assess the accuracy of the remaining1 locations computed using the proposed technique. The coordi-ates found from X-ray backprojection were compared with thehotograph-derived coordinates not used to compute the trans-orm, as well as the set of backprojected coordinates resulting fromsing all 47 visible electrodes plus the 4 anatomical fiducials.

.9.2. CT–MRI coregistrationGP1 and GP2 had CT scans in addition to high-resolution pre-

perative MRI scans; GP1 additionally had a postimplant lateraladiograph. CT–MRI coregistration was performed with SPM2 usingnormalized mutual information algorithm. (Simple fiducial-based

S.S. Dalal et al. / Journal of Neuroscience Methods 174 (2008) 106–115 111

F linkel

cTgtciMCh

3

dip

3

tchwpjmp

ig. 2. Interactive navigation is possible with the photograph, MRI, and radiographocation, in this case, the inferior tip of a large parietal encephalomalacia in GP4.

oregistration was first attempted with unsatisfactory results.)he quality of coregistration was visually verified by ensuringood agreement of skull shape between the CT and MRI. Elec-rode coordinates were found using our proposed method andompared with the electrode positions derived from the coreg-stered CT scan. 3D renderings of CT electrode positions on the

RI were created with MRIcro. Figures showing coregisteredT/MRI slices were generated with OsiriX (Rosset et al., 2004,ttp://homepage.mac.com/rossetantoine/osirix/).

. Results

The photographs, radiographs, and final annotated brain ren-erings for all eight patients are shown in Fig. 3. The total time for

mage registration, annotation, and rendering was typically 5 h peratient as performed by a trained researcher/technician.

3

t

Fig. 3. The photographs, radiographs, and final re

d via projective transforms. The crosshairs on each image indicate the same brain

.1. Performance evaluation

One patient’s image (GP6) was selected for further analysiso validate algorithm performance. Electrode locations found byomputing projective transforms based on all visible electrodesad a mean discrepancy of 1.5 mm (S.D. 0.5 mm, max 2.7 mm)hen compared to the photograph-derived coordinates (Fig. 4, blueoints). Using only a subset of six electrodes to compute the pro-

ective transform, the output electrode locations yielded only aean discrepancy of 2.0 mm (S.D. 1.0 mm, max 4.6 mm) from the

hotograph-based positions (Fig. 4, yellow points).

.2. CT–MRI coregistration

As shown in Fig. 5, coregistration of the postimplant CT withhe preimplant MRI did not yield a satisfactory solution for either

nderings for all 8 of the included patients

112 S.S. Dalal et al. / Journal of Neuroscience Methods 174 (2008) 106–115

Fig. 3. (Conti

Fig. 4. Evaluation of registration for the same patient shown in Fig. 1 (GP6). Knownelectrode positions from the photograph are shown as red dots, and the results fromusing the projective transform with all known electrode positions are depicted inblue. Since it is not possible to validate the location of hidden electrodes, a subset ofsix visible electrodes (boxed in yellow) was chosen to compute another projectivetransform. This transform was then applied to all X-ray locations, with the backpro-jection solution shown in yellow. The results from just these six electrodes agreeclosely with the known photograph-based positions as well as the results from thecomplete projective transform (blue dots). (For interpretation of the references tocolor in this figure legend, the reader is referred to the web version of the article.)

pwm(ssca

tpFtwtettpes(

4

pava

nued ).

atient. Even though the skull and intact right hemisphere wereell-registered, several surface electrodes coregistered to locationsore than 1 cm deep into brain tissue on the MRI in both patients

Fig. 5, column 1). Another CT–MRI overlay (Fig. 5, column 4) clearlyhows that the ventricles, midline, and exposed cortical surfacehifted considerably. These displacements are likely due to theraniotomy, associated swelling, and the thickness of the cablesnd electrodes themselves.

Therefore, it was necessary to project electrode locations fromhe CT to the brain surface. Fig. 6 compares the results of thisrocedure to electrode locations from the surgical photograph.or GP1, the discrepancy of CT-based electrode positions withhe photograph-based positions averaged 7.5 mm (S.D. 2.5 mm),ith a maximum error of 14.5 mm. For the other patient (GP2),

he discrepancy averaged 4.3 mm (S.D. 2.1 mm), with a maximumrror of 7.6 mm. In both cases, upon removal of the grid implant,he surgeon (NMB) confirmed that the grid had not shifted fromhe implant photograph. Additionally, iEEG recordings from theseatients showed patterns consistent with the photograph-basedlectrode locations, such as phase reversals across the Sylvian fis-ure as well as auditory and motor responses in expected areasCanolty et al., 2006, 2007; Dalal et al., 2008).

. Discussion

We have presented a method for registration of electrodeositions from standard medical images. In summary, preimplantnd postimplant digital photographs are registered with each otheria a projective transform. Next, the preimplant photograph, nownnotated with the visible electrode positions, is used to manually

S.S. Dalal et al. / Journal of Neuroscience Methods 174 (2008) 106–115 113

Fig. 5. Comparison of preoperative MRI and postimplant CT in two patients. First column: Coregistration of postimplant CT (in color) with preoperative MRI (grayscale); eventhough registration of the skull, scalp, and right hemisphere is good, many surface electrodes on the coregistered CT land more than 10 mm deep on the MRI in both patients.S ntricles ding( red to

ataFo

littiPtnabau

iMunsadtawtowscftt

fbeltesltit

MpmStdphiMotptaamsf

econd column: Another slice of the CT scan adjusted for contrast to visualize the velice overlaid on the CT (grayscale); note the displacement of the whole brain, incluFor interpretation of the references to color in this figure legend, the reader is refer

nnotate the corresponding anatomy on an MRI rendering ofhe brain. These visible electrodes are then used to computenother projective transform between the MRI and the radiograph.inally, backprojection using this transform reveals the locationsf occluded electrodes.

We have demonstrated that our procedure outputs an accurateocalization of electrode positions and does not require additionalmaging procedures that add risk, radiation exposure, and expenseo standard practice. Our technique makes use of digital pho-ographs and high-resolution MRIs that have only recently beenncorporated into the standard of care for epilepsy surgery patients.recise knowledge of the X-ray configuration, placement of addi-ional fiducial markers on the patient, and special calibration areot required. Most importantly, as the input images required arelready collected by most neurosurgery centers, the technique cane quickly implemented and introduced to clinical use as wells applied to archived images from past patients for researchse.

We have also shown that a popular technique for electrode local-zation, coregistration of postimplant CT scans with preoperative

RI scans, may not provide accurate results. The thicker slicessually obtained with standard clinical CT scanners result in sig-ificant partial volume averaging and coarse resolution along theuperior–inferior axis. Furthermore, coregistrations may be unreli-ble since simply performing a craniotomy is known to significantlyeform brain tissue (Hill et al., 1998; Roberts et al., 1998); par-icularly for patients implanted with chronic electrodes, swellingnd the thickness of the electrodes along with associated hard-are can contribute to further brain tissue displacement relative

o the preoperative MRI. While the same issues may arise withur technique since we are also registering postimplant imagesith preoperative structure, we expect their impact to be less

ignificant since the algorithm fits only the portion of the brainovered by electrodes and incorporates highly reliable informationrom the surgical photographs. Therefore, the various projectiveransforms involved may effectively absorb some brain deforma-ion.

eoes

s. Third column: The corresponding MRI slice. Fourth column: The same MR (color)a clear shift of the ventricle and midline, indicating significant brain deformation.the web version of the article.)

We have applied our method to register superficial electrodes,or which the estimated solution is the intersection of the X-rayackprojection with the cortical surface. The localization of deeplectrode arrays can also be accomplished if the depth of the shal-owest electrode contact is used to choose the displacement alonghe backprojected line from the cortical surface. The remaininglectrodes can then be computed similarly using known electrodepacing. If additional X-ray views are available, depth electrodeocations may instead be localized by computing separate projec-ive transforms for each X-ray. The solution would then be thentersection of the multiple backprojected lines for a given elec-rode.

While having a full set of high-quality image data (photographs,RI, and radiograph) increases confidence in the results, the

resented technique is still useful if some images are poor orissing altogether. As anatomy is still identifiable through the

ilastic electrode sheet in a good-quality postimplant photograph,he preimplant photograph is not strictly necessary, though itoes ease the procedure to have it. Without a radiograph, MRI-hotograph registration can be performed, and the position ofidden electrodes can be estimated from known electrode spac-

ng and brain curvature (e.g., see results for GP7 shown in Fig. 3b).RI-radiograph coregistration can still be performed in the absence

f surgical photographs with an estimate of a few electrode posi-ions on the MRI and the use of other anatomical features as controloints. (This last scenario likely would not provide results betterhan CT–MRI registration, but provides an option in the event of anrchived patient dataset in which no photographs or CT are avail-ble.) Finally, at any point in the process, electrode locations can beanually adjusted to account for information from other sources

uch as known electrode spacing, CT coregistration, or coordinatesrom surgical navigation devices.

One metric to evaluate the quality of the solution for hiddenlectrodes in individual patients would be to examine the spacingf estimated electrode locations. Since electrode spacing is gen-rally known, a large disagreement would suggest a suboptimalolution. In this event, however, the electrode spacing can then

114 S.S. Dalal et al. / Journal of Neuroscience Methods 174 (2008) 106–115

Fig. 6. (a) Comparison of electrode registration with the proposed technique and from CT–MRI coregistration for GP1. The red splotches are the CT-derived positions; theirs hite di iderabp tationv

bec

cslodiWp

m(wfwioa

nr

5

iJctdWhbN

hape is determined from intensity-based segmentation of the CT. The translucent wndicate positions backprojected from the X-ray. The CT-derived positions are conshotograph for this patient. (c) and (d) show similar images for GP2. (For interpreersion of the article.)

e used to constrain the solution and provide a satisfactory result,ven in regions of high brain convexity where a small error in X-rayoordinates can result in large backprojection errors.

Much of the time required for our procedure was spent simplyonverting between different image formats and manually tran-cribing coordinates; manual annotation of visible electrodes andandmarks on the various images also consumed a large proportionf time. However, with further development, all required proce-ures can be implemented as an integrated software package, and

t should be possible to automate the image annotation procedures.e expect this could reduce overall rendering time to less than 2 h

er patient.The algorithm presented is a valuable complement to existing

edical imaging software that fuses different imaging modalitiese.g., existing software coregisters MRI and CT or MRI and PET). Itould be useful as part of a suite of programs specifically intended

or the diagnostic imaging and monitoring of neurosurgery patientsith implanted electrodes. We have found that our interactive nav-

gation tool to be clinically useful to aid in the positive identificationf structures of interest on the X-ray and photograph; it couldlso be suitable in an educational setting to help students connect

R

A

isks indicate positions known from the surgical photographs, while the yellow disksly displaced posteriorly relative to the photograph-verified positions. (b) Surgicalof the references to color in this figure legend, the reader is referred to the web

euroanatomy with radiographs, photographs, MRI slices, and MRIenderings.

. Acknowledgments

The authors would like to thank Paul A. Garcia and David S. Fil-ppi for acquisition of surgical photographs, Robert G. Gould andim Buescher for helpful information on X-ray acquisition proto-ols, Kenneth E. Hild II for insightful discussions on algorithmicopics, and Susanne M. Honma for critical assistance in trackingown the medical images and personnel necessary for this project.e also thank Johanna M. Zumer and Edward F. Chang for their

elpful comments on this manuscript. SSD was supported in party NIH Grant F31 DC006762. This work was supported in part byIH grants R01DC004855 and R01DC006435 to SSN.

eferences

bdel-Azziz Y, Karara H. Direct linear transformation from com-parator coordinatesinto object space coordinates in close-range photogrammetry. In: Papers fromthe 1971 ASP symposium on close-range photogrammetry; 1971. p. 1–18.

scienc

B

B

B

C

C

D

E

F

G

G

H

H

H

H

H

J

K

L

L

L

M

M

M

M

M

N

R

R

R

S

S

S

T

W

W

W

S.S. Dalal et al. / Journal of Neuro

ancaud J, Talairach J, Bonis A. La stéréoélectroencéphalographie dans l’epilepsie.Paris: Masson; 1965.

arnett GH, Kormos DW, Steiner CP, Morris H. Registration of EEG electrodes withthree-dimensional neuroimaging using a frameless, armless stereotactic wand.Stereotact Funct Neurosurg 1993;61:32–8.

ootsveld K, Tr’aber F, Kaiser WA, Layer G, Elger CE, Hufnagel A, et al. Localisation ofintracranial EEG electrodes using three dimensional surface reconstructions ofthe brain. Eur Radiol 1994;4:52–6.

anolty RT, Edwards E, Dalal SS, Soltani M, Nagarajan SS, Kirsch, et al. Highgamma power is phase-locked to theta oscillations in human neocortex. Science2006;313:1626–8.

anolty RT, Soltani M, Dalal SS, Edwards E, Dronkers NF, Nagarajan, et al. Spa-tiotemporal dynamics of word processing in the human brain. Front Neurosci2007;1(1):185–96.

alal SS, Guggisberg AG, Edwards E, Sekihara K, Findlay AM, Canolty, et al. Five-dimensional neuroimaging: localization of the time-frequency dynamics ofcortical activity. NeuroImage 2008:44.

vans AC, Collins DL, Mills SR, Brown ED, Kelly RL, Peters TM. In: Proceedings ofIEEE nuclear science symposium and medical imaging conference, vol. 3; 1993.p. 1813–7.

ox PT, Perlmutter JS, Raichle ME. A stereotactic method of anatomical localizationfor positron emission tomography. J Comput Assist Tomogr 1985;9(1):141–53.

rzeszczuk R, Tan KK, Levin DN, Pelizzari CA, Hu X, Chen, et al. Retrospective fusionof radiographic and MR data for localization of subdural electrodes. J ComputAssist Tomogr 1992;16(5):764–73.

utiérrez LF, Shechter G, Lederman RJ, McVeigh ER, Ozturk C. Distortion correction,calibration, and registration: towards an integrated MR and X-ray interven-tional suite. In: Galloway RL, Cleary KR, editors. Proceedings of the SPIE medicalimaging conference, vol. 5744; 2005. p. 146–56.

artley RI, Zisserman A. Multiple view geometry in computer vision. 2nd ed. Cam-bridge University Press; 2004.

astreiter P, Rezk-Salama C, Soza G, Bauer M, Greiner G, Fahlbusch R, et al. Strategiesfor brain shift evaluation. Med Image Anal 2004;8:447–64.

ill DL, Maurer CR, Maciunas RJ, Barwise JA, Fitzpatrick JM, Wang MY. Measurementof intraoperative brain surface deformation under a craniotomy. Neurosurgery1998;43:514–26.

ochberg LR, Serruya MD, Friehs GM, Mukand JA, Saleh M, Caplan, et al. Neuronalensemble control of prosthetic devices by a human with tetraplegia. Nature2006;442:164–71.

unter JD, Hanan DM, Singer BF, Shaikh S, Brubaker KA, Hecox, et al. Locatingchronically implanted subdural electrodes using surface reconstruction. ClinNeurophysiol 2005;116(8):1984–7.

enkinson M, Pechaud M, Smith S. BET2: MR-based estimation of brain, skull andscalp surfaces. In: Eleventh annual meeting of the organization for human brainmapping; 2005.

ovalev D, Spreer J, Honegger J, Zentner J, Schulze-Bonhage A, Huppertz H-J. Rapidand fully automated visualization of subdural electrodes in the presurgical eval-

uation of epilepsy patients. AJNR Am J Neuroradiol 2005;26:1078–83.

esser RP, Gordon B, Fisher RS, Hart J, Uematsu S. Subdural grid electrodes in surgeryof epilepsy. In: L’uders HO, editor. Epilepsy surgery. New York: Raven; 1991.

euthardt EC, Miller KJ, Schalk G, Rao RPN, Ojemann JG. Electrocorticography-basedbrain computer interface-the Seattle experience. IEEE Trans Neural Syst RehabilEng 2006;14:194–8.

W

Y

e Methods 174 (2008) 106–115 115

evenberg K. A method for the solution of certain nonlinear problems in leastsquares. Quart Appl Math 1944;2:164–8.

ahvash M, K’onig R, Wellmer J, Urbach H, Meyer B, Schaller K. Coregistration ofdigital photography of the human cortex and cranial magnetic resonance imag-ing for visualization of subdural electrodes in epilepsy surgery. Neurosurgery2007;61:340–4.

arquardt DW. An algorithm for least-squares estimation of nonlinear parameters.SIAM J Appl Math 1963;11:431–44.

azziotta J, Toga A, Evans A, Fox P, Lancaster J, Zilles, et al. A probabilisticatlas and reference system for the human brain: international Consortiumfor Brain Mapping (ICBM). Philos Trans R Soc Lond B Biol Sci 2001;356:1293–322.

iller KJ, Makeig S, Hebb AO, Rao RPN, denNijs M, Ojemann JG. Cortical elec-trode localization from X-rays and simple mapping for electro-corticographicresearch: The “Location on Cortex” (LOC) package for MATLAB. J Neurosci Meth-ods 2007;162:303–8.

iyagi Y, Shima F, Sasaki T. Brain shift: an error factor during implantation of deepbrain stimulation electrodes. J Neurosurg 2007;107:989–97.

elles M, K’onig R, Kandyba J, Schaller C, Urbach Zentralbl Neurochir H.Ko-registrierung von mrt vor und CT nach der implantation subduraler git-terelektroden. Zentralbl Neurochir 2004;65(4):174–9.

oberts DW, Hartov A, Kennedy FE, Miga MI, Paulsen KD. Intraoperative brain shiftand deformation: a quantitative analysis of cortical displacement in 28 cases.Neurosurgery 1998;3:749–58.

orden C, Brett M. Stereotaxic display of brain lesions. Behav Neurol2000;12:191–200.

osset A, Spadola L, Ratib O. OsiriX: an open-source software for navigating in mul-tidimensional DICOM images. J Digit Imag 2004;17:205–16.

anthanam G, Ryu SI, Yu BM, Afshar A, Shenoy KV. A high-performance brain-computer interface. Nature 2006;442:195–8.

chulze-Bonhage AH-J, Huppertz HJ, Comeau RM, Honegger JB, Spreer JM, Zent-ner JK. Visualization of subdural strip and grid electrodes using curvilinearreformatting of 3D MR imaging data sets. AJNR Am J Neuroradiol 2002;23:400–3.

ochurková D, Rektor I, Jurák P, Stanffcák A. Intracerebral recording of cortical activ-ity related to self-paced voluntary movements: a Bereitschaftspotential andevent-related desynchronization/synchronization. SEEG study. Exp Brain Res2006;173:637–49.

alairach J, Tournoux P. Co-planar stereotaxic atlas of the human brain. Germany:Thieme, Stuttgart; 1988.

all BF, Hart D. Revised radiation doses for typical X-ray examinations: report on arecent review of doses to patients from medical X-ray examinations in the UKby NRPB. Br J Radiol 1997;70:437–9.

ellmer J, von Oertzen J, Schaller C, Urbach H, K”onig R, Widman G, et al. Digital pho-tography and 3D MRI-based multimodal imaging for individualized planning ofresective neocortical epilepsy surgery. Epilepsia 2002;43:1543–50.

inkler PA, Vollmar C, Krishnan KG, Pffuger T, Br’uckmann H, Noachtar S. Useful-

ness of 3D reconstructed images of the human cerebral cortex for localizationof subdural electrodes in epilepsy surgery. Epilepsy Res 2000;41:169–78.

yler AR, Walker G, Richey ET, Hermann BP. Chronic subdural strip electrode record-ings for difficult epileptic problems. J Epilepsy 1988;1:71–8.

uan X, Zhang J, Buckles BP. Evolution strategies based image registration via featurematching. Inf Fusion 2004;5(4):269–82.