c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 8 3 ( 2 0 0 6 ) 63–72

journa l homepage: www. int l .e lsev ierhea l th .com/ journa ls /cmpb

Robotic surgery setup simulation with the integration ofinverse-kinematics computation and medical imaging

Mitsuhiro Hayashibea,∗, Naoki Suzukia, Makoto Hashizumeb,Kozo Konishib, Asaki Hattoria

a Institute for High Dimensional Medical Imaging, The Jikei University School of Medicine,4-11-1, Izumihoncho, Komae-shi, Tokyo 201-8601, Japanb Center for Integration of Advanced Medicine and Innovative Technology, Kyushu University Hospital,3-1-1, Maidashi, Higashi-ku, Fukuoka 812-8582, Japan

a r t i c l e i n f o a b s t r a c t

Article history:

Received 16 June 2005

Received in revised form 3 April

2006

Accepted 18 April 2006

Keywords:

Robotic surgery

Robot setup simulation

Inverse-kinematics

Preoperative planning

DICOM

Volume rendering

At present, there are representative robot operation systems such as da Vinci and ZEUS

which have realized minimally invasive surgery by the use of dexterous manipulators. In

the operating room, medical staff must prepare and set up an environment in which the

robot has optimal freedom of motion and its functions can be fully demonstrated for every

case. The range of motion in which the robot can reach and be maneuvered is restricted by

the fixed point of the trocar site. We have developed a preoperative planning system with

the function of volume rendering of medical images and automatic positioning by applying

an inverse-kinematics computation of surgical robot. The motion of a surgical robot can

be simulated in advance with the intuitive interface and kinematics computation program

running in the background of the system. If robotic surgery planning with volume rendering

of DICOM images is possible, the discussion of a surgical plan can be directly made just after

the diagnosis considering the patient-specific structure. This kind of setup platform would

be essential for the future introduction of surgical robotics into an operating room.

© 2006 Elsevier Ireland Ltd. All rights reserved.

1. Introduction

Present technology advancements have led to the world-widerapid development of laparoscopic surgery. A number of opensurgery procedures are being replaced by minimally invasiveinterventions. In particular, telerobotic and computer-assistedsystems are currently making exceptional progress in the fieldof endoscopic surgery. Two competing systems, the da Vincisystem [1] (Intuitive Surgical Inc., CA) and the ZEUS system [2](originally Computer Motion Inc., CA), have targeted cardiacsurgery for their introduction in the surgical field [4,3]. Cadiereet al. [5], Marescaux et al. [6] and Hashizume et al. [7] havereported the use of these systems in general surgery cases.Good and safe operative results have been verified through

∗ Corresponding author.

those experiences. The merits of introducing surgical robotsinto operating rooms include the facts that safe and fine-scaleoperations can be conducted, trauma is reduced, recovery timeis shortened, and there is little doctor fatigue from prolongedoperations. In particular, surgical instruments located at thetip of the manipulator provide a full range of motion and abil-ity, allowing the instruments to rotate more than 360◦ throughtiny incisions. Forceps can open and shut in a gripping modeand have two wrist joints in the abdominal cavity. Their highlevel of dexterity has obtained a terrific reputation all over theworld.

In the operating room, surgeons must prepare and set upan environment where the robot can have adequate degreesof freedom of motion so that the robot’s functions can be

0169-2607/$ – see front matter © 2006 Elsevier Ireland Ltd. All rights reserved.doi:10.1016/j.cmpb.2006.04.010

64 c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 8 3 ( 2 0 0 6 ) 63–72

fully performed for each clinical case. As with general laparo-scopic surgery, surgeons decide the trocar site placement forapproaching the abdominal cavity and insert surgical instru-ments through a hole of approximately 1 cm diameter in theabdominal wall. In general, two trocar sites for forceps inaddition to one for the endoscope and one auxiliary port arechosen and set. The optimal set-up will vary with the typeof intervention and the equipment used in the operation.In addition, frequent collisions between each robot arm canbecome a problem, and setup time tends to be longer due towidening of the common accessible area for three manipula-tors in the case of ZEUS surgery. Here the actual placementof the surgical robot and the procedure for approaching thetumor in a geometrical sense will be discussed. If the geo-metrical structure of the individual patient and the surgicalrobot are integrated in virtual space, surgeons and assistantscan confirm the movability of the robot in the operating room.Moreover, the region in which the robot can reach and canbe maneuvered is constrained by the fixed points of the tro-car sites. Therefore, in each case these sites must be carefullychosen for the particular patient. If a preoperative planningsystem is available, robot positioning which maintains a largedistance from all surrounding obstacles is realized, and theburden to the patient by the reopening of trocar sites can beavoided.

In order to better utilize patient structure informationnot only for diagnosis but also for medical treatment, some

kinematics computation and DICOM images. The motion ofthe surgical robot is simulated and rehearsed with compu-tation of collisions between the robot arms, constraints atthe trocar site, and the inverse-kinematics of the robot. Beingintegrated with a haptic interface and kinematics computa-tion, surgeons can interactively push and drag the arms ofthe virtual surgical robot, which has the same kinematics asthe real one. In the interactive planning system, medical staffcan easily discuss and validate the planned ports simultane-ously observing the correspondent DICOM images. Addition-ally, they can gain a common perception ahead of the actualintervention procedure, and as an educational tool surgeonscan learn and experience how to set up surgical robots forlaparoscopy. In this manuscript, detailed kinematics compu-tation method for ZEUS system is described. The ZEUS systemis no longer commercially produced. However, as a model forclinical telemanipulation which has been definitely used, it isstill applied at many places. The ZEUS system is composed ofthree independent arms. Therefore, there is more need for pre-operative planning and simulation than the da Vinci. In addi-tion, the important thing is that the simple style manipulatorof ZEUS system can be considered as a generic model of sur-gical robot. The planning software should be used in a genericway to simulate setup of any present or upcoming robotic plat-form. We believe that the detailed description of kinematicscomputation for ZEUS system is useful for the setup algo-rithm of upcoming robotic platform. Automatic guidance of

ushu

research on surgical simulations for operational planning andtraining has been conducted using virtual reality techniques[8–11]. As for applications to robotic surgery, some researchgroups have developed a surgical simulation system to learnthe operational procedure [12–14]. In particular to roboticssetup, the concept of port-placement planning for da Vincicardiac surgery was proposed and addressed by Coste-Maniereet al. [15,16], which includes intraoperative registration of thepatient. Preoperative planning system for da Vinci abdominalsurgery using a haptics interface was implemented and veri-fied in [17].

In this paper, we describe the development of a surgi-cal robot setup simulation with the integration of inverse-

Fig. 1 – Actual scene of ZEUS surgery being performed at Kymanipulator.

robotics setup and optimization of robotics setup were alsoimplemented. Finally, integration of DICOM images for clinicaluse was executed and preliminarily verified for the effective-ness of the system.

2. Method

2.1. Kinematics and geometric modeling of ZEUS

The ZEUS system is composed of two forceps-holding manip-ulators and one laparoscope-holding arm controlled bymaster–slave configuration. An actual scene of ZEUS surgery

University Hospital and the link mechanism of the ZEUS

c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 8 3 ( 2 0 0 6 ) 63–72 65

Table 1 – Link parameters of ZEUS

Link Variable ˛ a d cos ˛ sin ˛

1 d1 0 0 d1 1 02 �2 0 a2 = 0.384 0 1 03 �3 �/2 a3 = 0.08 0 0 14 �4 �/2 a4 = 0.05 0 0 15 �5 −�/2 0 d5 = 0.25 0 −16 �6 −�/2 a6 = 0.015 d6 = 0.0 0 −17 �7 0 0 d7 = 0.25 1 0

being performed at the Kyushu University Hospital is shownin the picture in the left side of Fig. 1. Each of the three armscan be independently placed and set on the operating table,as shown in the picture. Due to this independent structure,the manipulator of the ZEUS system is designed as a compactshape compared with that of the da Vinci system. Further,this configuration allows high degrees of freedom for the setupand placement of the surgical robots. However, it is difficult toplace the three arms in an appropriate manner so as to avoidthe preconceptual collision during the operation. In addition,surgeons must visualize the common movable range for allthree manipulators, and in actual practice this leads to a pro-longed setup time for robotic surgery in the operating room.

There is always the possibility that the trocar site isreplaced due to the misprediction of the movable range of thesurgical robot. The reopening of the trocar site is against theconcept of minimally invasive surgery. Preventing and mini-mizing the repositioning of the surgical robot is a critical issue.A computer-assisted system therefore plays an important rolein robotic surgery planning. In this system, surgeons check thepositioning of the three surgical robots in advance, therebyreducing the setup time for robotic surgery and minimizingthe necessity for physical trial and error in the operating room.

The link structure of the ZEUS system is depicted in theright side of Fig. 1. Each link shape and the distances betweenthe links of the ZEUS system were measured in detail, and ageometric model was then created and reconstructed using 3DChjeittaafmit

ni

rf

Fig. 2 – Local frames on the ZEUS manipulator.

iAi+1(i = 0, . . ., 6) are described as below using a link parametertable (Table 1).

AD software. The featured mechanism of the ZEUS systemas the following configuration: three active joints, one fixed

oint, and two passive joints. Table 1 shows the link param-ters of the ZEUS manipulator. Each link is described withts mechanical information, including the child–parent rela-ionship based on Denavit–Hartenberg notation. Link 1 is aranslatory joint, and links 2 and 3 are revolute joints. Link 4 isfixed revolute joint, links 5 and 6 are passive revolute joints

nd link 7 is a revolute joint around the axial direction of theorceps. Therefore, the forceps held by this arm are able to

ake a pivoting motion around the fixed point on the abdom-nal wall. The two passive joint parameters are determined byhe constraint.

Local frames are aligned based on Denavit–Hartenbergotation, as shown in Fig. 2. The transformation matrix

Ai+1(4 × 4) from frame i to frame i + 1 can be defined withotation matrix iRi+1(3 × 3) and translation vector iPi+1(3 × 1) asollows:

(1)

(2)

where ci is cos �i and si is sin �i if iAi+1 is known, the transfor-mation matrix 0AN from frame 0 to frame N (N = 2, 3, . . ., 7) is

66 c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 8 3 ( 2 0 0 6 ) 63–72

solved as

0AN = 0A11A2

2A3· · ·N−1AN (3)

The velocity of each link should be calculated from the baselink to the top link in order. The velocity of link i + 1 is a com-bination of the velocity of link i and the added velocity by jointi + 1.

If the velocity of the origin of frame i against frame 0 isexpressed as vi, and the angular velocity of frame i is writtenas wi, the velocity of link 1 sliding at the speed of d1 becomes

w1 = [ 0 0 0 ]T

v1 = [ 0 0 d1 ]T (4)

The velocity of link 2 rotating at the speed of �2 becomes

w2 = w1 + Z1�2

v2 = w2 × 1P∗2 + v1

(5)

where Zi is a unit vector at the direction of the z axis of frame iin the coordinate system of frame 0, and iP∗

i+1 is the translationvector from the origin of frame i to frame i + 1 in the coordinatesystem of frame 0. Joint 3–joint 7 are revolute joints at thespeeds of �3, �4, �5, �6 and �7. The velocity and angular velocityare listed as follows:

We define new frame st at the position of the fixed point alongthe axial direction of the forceps. The velocity of this framecan be written as shown below:1

vst = −w6 × 0A6(1 : 3, 3)L + v6 (11)

If frame 6′ is defined as the frame with its origin located at theorigin of frame 0, and the direction is same as that of frame 6,then the velocity of frame st in the coordinate system of frame6, can be written as shown below, and its x and y factors equalzero due to the constraint at the fixed point2.

6′vst = −A−1

6 (1 : 3, 1 : 3)w6 × 0A6(1 : 3, 3)L

= +0A−16 (1 : 3, 1 : 3)v6 = [0 0 ∗]T (12)

�5 and �6 can be represented as a linear sum of d1, �2 and �3 bysolving Eq. (12) algebraically. However, it is difficult to linearizefor the complexity of this equation. Here, we use the Jaco-bian matrix [18] from the base frame to frame st to linearizeand solve the unknown factor of �5 and �6. The transformationmatrix from frame 6 to frame st is

w3 = w2 + Z2�3

v3 = w3 × 2P∗3 + v2

(6)

w4 = w3 + Z3�4

v4 = w4 × 3P∗4 + v3

(7)

w5 = w4 + Z4 �5

v5 = w5 × 4P∗5 + v4

(8)

w6 = w5 + Z5�6

v6 = w6 × 5P∗6 + v5

(9)

w7 = w6 + Z6�7

v7 = w7 × 6P∗7 + v6

(10)

The velocity and angular velocity of the tip of forceps v7 andw7 can be solved using equations from (4) to (10) in order. v7

and w7 are composed of seven variables, d1 and �2–�7, where�4 is zero due to it being a fixed joint during the operation.

2.2. Constraint at the fixed point of the abdominal wall

The fifth and sixth joints of the ZEUS manipulator cannot beactively controlled. These joint parameters are determinedunder the conditions of constraint at the fixed point on theabdominal wall. This mechanism is designed for avoidingexcessive force against the incised part of abdomen. To simu-late the motion of a surgical robot with passive joints, we mustcalculate the angle of joints 5 and 6 depending on the anglesof joints 1–4 in each frame.

When the distance from the forceps-holding point to thefixed point on the abdominal wall is L, the fixed point is locatedat the position of [0 0 − L]T in the coordinate system of frame 6.

(13)

The velocity and angular velocity of frame st is expressedusing Jacobian and link parameters of joint 1–6 as indicatedin Eq. (14).

[vst

wst

]= J

⎡⎢⎢⎢⎣

d1

�2

�3

�5

�6

⎤⎥⎥⎥⎦ =

[J1v J2v J3v J5v J6v

J1w J2w J3w J5w J6w

]⎡⎢⎢⎢⎣

d1

�2

�3

�5

�6

⎤⎥⎥⎥⎦ (14)

If the constraint condition is concerned with vst, then we cal-culate only Jiv (i = 1, . . . , 6). The Jacobian of translatory joint 1is

J1v = 0A1(1 : 3, 3)d1 (15)

The Jacobian of revolute joint 2–joint 6 is

Jiv = 0Ai−1(1 : 3, 3) × (0Ast(1 : 3, 4) − 0Ai−1(1 : 3, 4))�i (16)

1 0A6(1:3,3) implies the longitudinal vector drawn from the firstto third row factors in the third column of 0A6(4 × 4).

2 0A6(1:3,1:3) implies a 3 × 3 matrix drawn from the first to thirdrow factors in the first to third columns of 0A6(4 × 4).

c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 8 3 ( 2 0 0 6 ) 63–72 67

6′vst can be rewritten using the Jacobian as shown below:

6′vst = 0A−1

6 (1 : 3, 1 : 3)[J1v· · ·J6v]

⎡⎢⎣

d1

...�6

⎤⎥⎦ = 6Jv

⎡⎢⎣

d1

...�6

⎤⎥⎦ = [0 0 ∗]T

(17)

where symbol ‘*’ implies any real number. �5 and �6 comes tobe expressed as a linear sum of d1, �2 and �3 as

[�5

�6

]= 6J−1

v (1 : 2, 4 : 5)6Jv(1 : 2, 1 : 3)

⎡⎣ d1

�2

�3

⎤⎦ = CJ

⎡⎣ d1

�2

�3

⎤⎦ (18)

2.3. Calculation of Inverse-kinematics

In this system, the user can interactively input and edit thetip position of the forceps held by the surgical robot. To real-ize the desired position and attitude of the surgical robot,the computation of inverse-kinematics automatically solvesthe time-varying joint angles of each link. Considering theJacobian matrix from the base frame to tip frame 7, the rela-tionship between the velocity of the tip of the forceps and thejoint angle velocities is

[ ] ⎡d1

⎤ ⎡d1

⎤

J

J

Jtta

[

Ifv

3

TfWs

Fig. 3 – Process overview.

the VR environment. An intuitive surgical robot setup simu-lation was enabled with haptic sensation. In order to realizethe surgical robotics platform, this system consisted of threeparts: haptics, kinematics and graphic modules. Graphics aregenerated using OpenGL library. In this research, kinematicscomputation engine for surgical robots is a key function, andits computation is always running in the background of theprogram. These three modules are integrated inseparably inan intuitive manner. Therefore, users can change the robotposition, the trocar position and the target position by thehaptic interface at any time. Surgical robot motion is automat-ically generated to meet the input command, and the graphicsloop also updates the simulated scene. The configuration ofthis system is described below in detail.

The main process is divided into graphics and hapticspipelines as illustrated in Fig. 3. Callback functions allowthe synchronization of object information contained in thehaptic and graphic loops. When the application is ready torender a new scene it issues an update graphics call thatqueries the state nodes with graphics callbacks. The corre-sponding motion of connected links is generated by inverse-kinematics computation when the tip of the forceps is guidedand dragged in 3D space. With this interactive interface, sur-geons can rehearse the intervention for preoperative planningto enable an optimal device setup in the operating room. Theprocedure for setup planning in this system is summarizedas follows:

v

w= J

⎢⎣ ...�7

⎥⎦ = [J1· · ·J7]⎢⎣ ...

�7

⎥⎦ (19)

1 =[

0A1(1 : 3, 3)0

](20)

i (i = 0, . . . , 6) =[

0Ai−1(1 : 3, 3) × (0Ai(1 : 3, 4) − 0Ai−1(1 : 3, 4)0Ai−1(1 : 3, 3)

]

(21)

is the matrix of six rows and seven columns. Using Eq. (18)hat was introduced from the constraint condition, J can beransformed, decreasing the dimension of these parameterss

v

w

]=

⎡⎢⎢⎢⎣

JT1 + CJ(1, 1)JT

5 + CJ(2, 1)JT6

JT2 + CJ(1, 2)JT

5 + CJ(2, 2)JT6

JT3 + CJ(1, 3)JT

5 + CJ(2, 3)JT6

JT7

⎤⎥⎥⎥⎦

T ⎡⎢⎣

d1

�2

�3

�7

⎤⎥⎦ (22)

n each frame, each joint angle velocity could be computedrom the inverse-transformation of Eq. (22) and the targetedelocity of the tip of the forceps.

. Implementation

he purpose of this system is to develop a platform that per-orms the setup simulation for robotic surgery in virtual space.

e adapted the PHANTOM interface device [19] to handle andet the robotic arms against the patient structure model in

(1) Roughly position three ZEUS models along the operatingtable against the patient model.

(2) Roughly set the position of the three trocar sites touchingand confirming the body surface of the patient model by ahaptics interface.

(3) Set the position of the target on the objective organ.Automatically generate the surgical robot motion to meetthe trocar position and the target position. The target isviewed by the laparoscope-holding arm, and the right andleft forceps-holding arms are navigated and set for beingviewed by the laparoscope (Section 4.1).

(4) Reposition and edit the base position of the three arms andthe trocar sites so as not to collide with anything withinthe moveable range of the manipulator and to make spacefor the surgical assistant (Section 4.1).

(5) Optimize the attitude of the surgical robot if necessary(Section 4.2).

68 c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 8 3 ( 2 0 0 6 ) 63–72

Fig. 4 – Appearance of preoperative setup planning forZEUS surgery.

Once the trocar site has been determined, the manipula-tor makes a pivoting motion around the incision point. Setupplanning can be confirmed following the same procedure asthe actual way in which an endoscope is first inserted andthen forceps are guided with reference to the laparoscopicimage. Fig. 4 depicts the appearance of preoperative setupplanning for ZEUS surgery. The surgeon can easily start thesetup program using a laptop computer with an IEEE 1394interface. Haptics signals are transferred through the fire-wire.This portability helps the surgical robot setup in the operatingroom while referring to the simulated result in this program.

The flowchart for kinematic computation algorithm ofkinematics module in this system is shown in Fig. 5 anddescribed as follows:

(1) Initialize the surgical robot kinematics, such as each linklength and the limits of the joint angles.

(2) Read each current joint angle.(3) Calculate forward kinematics and pass these parameters

to the graphics module to visualize the current attitudeand position of the surgical robots.

(4) Check the target command from a haptics loop and inputthe target position to be reached by the robot.

(5) If there is a difference between the present simulated tipposition and the target position, try to activate the robotsto reach the target by three arms.

(6) Calculate the distance to the fixed point on the abdomen.(7) Linearize the passive joint parameters based on the con-

strained condition (Section 2.2).(8) Calculate inverse-kinematics (Section 2.3).(9) Update joint angles.

4. Results

4.1. Automatic guidance of robotics setup

Laparoscopic cholecystectomy is an operation for gallbladdercalculosis or polyp in which the gallbladder is excised andremoved under a laparoscope without making a big incisionin the abdomen, as has been done previously. This is consid-ered a typical laparoscopic operative procedure. The initial

Fig. 5 – Flowchart for kinematic computation algorithm.

pose of the surgical robot and the incision site of the robotarm in laparoscopic cholecystectomy have been thoroughlydiscussed and successfully planned in this developed system.The arm holding the forceps and the laparoscope camera armof the ZEUS system can be guided by referring to the entireand laparoscopic camera views of the patient model. A wholebody structure of a typical Japanese female was reconstructedfrom 400 slices of MRI scans at 4 mm intervals. For gastroen-terological surgery, there are patient models of the stomach,liver, intestines and kidney along with texture information.This typical model can be used for general planning and ineducational trials.

In order to support the surgeon’s usability for the roboticssetup, we developed the function of automatic guidance ofthe surgical robot by inverse-kinematics computation. Thecommand input for the 3D trocar position and target point ismade using the PHANTOM interface. Keeping the constraintat the trocar position, the motions of the three surgical robotsare generated for converging the tips of the forceps and thecamera scope on the target point. Fig. 6 shows the gener-ated motion from the automatic guidance to a gallbladder asa target organ. In the left-bottom corner, a simulated laparo-scopic image is shown. The laparoscope-holding arm has beencontrolled for viewing the target in the center, and the twoforceps-holding arms have been navigated for being viewedby the laparoscope. Fig. 7 is the 3D trajectory of the guidedmanipulator to the target position.

As the background process, the kinematics module pro-gram is always running. At any time, the user can changeand edit the robot base positions, the trocar positions andthe target position in any order. If the constraint condition ischanged, the kinematics module tries to meet the conditionimmediately, and the feedback loop is always checking the dif-

c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 8 3 ( 2 0 0 6 ) 63–72 69

Fig. 6 – Generated motion of an automatically guided ZEUSrobot to a gallbladder.

ference between the current position and the target position.When the robot base is moved and slid by the haptic inter-face, the surgical robot motion is automatically generated asshown in Fig. 8. In this case, the base position of the centeredcamera-holding arm is dragged to the left direction from 1 to3. Note that the trocar position and target point are kept in

Fig. 7 – Trajectory of guided manipulator to the targetposition by inverse-kinematics computation.

Fig. 8 – Time-sequentially generated motion while slidingthe base position of the centered camera-holding arm.

the present condition during the transition of base position.The setup simulation has been enabled in a short time by theintuitive interface.

4.2. Optimization of robotics setup

In the paragraph above, how to set up and edit the robot posi-tioning is described. The conclusive appropriateness of the

70 c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 8 3 ( 2 0 0 6 ) 63–72

locations should be judged and decided by surgeons consider-ing the complex clinical conditions. However, the indicationsfrom the simulation program regarding how suitable the cur-rent setting is would further help the operator. Although itis difficult to take into account the clinical conditions, theoptimization of the robotics setup was implemented from theperspective of robot kinematics. In order to avoid the singu-larity of mechanism and to maximize the movable range ofthe robot from the initial position, the fact that the initial linkparameter of joint 3 should be a right angle is known. Thismanipulator comes to singularity when the angle of joint 3 is

Fig. 10 – Integration of DICOM volume rendering andsurgical robotics planning.

either 0◦ or 180◦. The robot can keep its wide range of motionif the initial starting setting is approximately a right angle.The optimization of these robotics setup functions is shown inFig. 9. Preserving the position of the trocar site and the target,the base position of the three manipulators can be automat-

Fig. 9 – Time-sequentially generated motion whileoptimizing the arm configuration of the three manipulatorsfor better surgical movability.

ically adjusted and slid until the angle of joint 3 meets theright angle. This optimization process should serve the sur-geon as a useful reference. The surgeons can stay focused onlyon the clinical conditions needed for modifying this kinemat-ically optimized result to actually feasible robotic settings.

Fig. 11 – The evaluation result of this system.

c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 8 3 ( 2 0 0 6 ) 63–72 71

4.3. Integration of DICOM images for clinical use andits evaluation

In clinical use, it is very important to reduce time when rou-tinely acquired DICOM images are imported into the planningsystem. As usual process, the segmentation stage for surfacemodeling is required for this kind of virtual reality simulation.It will take several hours to make segmentation and recon-struct 3D surface model. Therefore, we aimed at the roboticsurgery planning without preprocessing to introduce preoper-ative medical images quickly and smoothly. If robotic surgeryplanning with volume rendering of DICOM images is possible,the discussion of a surgical plan can be directly made just afterthe diagnosis considering the patient-specific structure.

Fig. 10 shows the integration result of volume rendering ofDICOM images into robotic surgery planning. In this case, mag-netic resonance cholan-giopancreatography (MRCP) imageswere used. In MRCP, cholecyst, bile duct and hepatic duct canbe visualized without contrast enhancement. In the program,volume rendering with 3D texture was executed in more than10 fps while inverse-kinematics computation of three manip-ulators was being processed with a dual 2.8 GHz CPU. Theresolution of volume data was 256 × 256 × 256. In the volumerendering mode, the setup planning could be started just afterCT scanning of the patient.

Surgeons proficient in robotic surgery have tested and eval-uated this system. The general opinion was that the robotmasargTamtswmaas

5

T

(

(

intuitive interface for modifying the attitude of the surgi-cal robot.

(3) Optimization of the robotics setup has been implementedfor avoiding the singularity of mechanism and maximizingthe movable range of the robot.

(4) Integration of DICOM volume rendering and surgicalrobotics simulation has been realized for clinical use. Thiskind of planning platform would be beneficial as a novelradiological utility for upcoming robotic surgery.

In this paper, we described the algorithm and the functionof the setup platform for robotic surgery. Especially, the auto-matic guidance of robot manipulators is executed consideringthe optimal kinematics of all three arms. This system providesthe interface in which surgeons can drag and edit the armsof the virtual surgical robot in a manner that has consistentkinematics with the real robot. The surgeon does not need tobe conscious about the operability of arm configuration. How-ever, the surgeon has to take care of external constraints. Theexternal constraints such as anesthesia monitoring, positionof an assistant and the nurse, as well as the presence of addi-tional equipment at and next to the OR table affect the actualset up of the robotic arms.

In addition, for better clinical use, further validation of thissystem should be necessary. In this stage, we applied and testi-fied the system within a limited group. The evaluation was still

r

otion was smooth and very similar to that of actual robots,nd that this system is very useful for confirming the expectedurgical robot settings. There was also the indication thatlthough to some extent precise structure data is used, theendering delay is not a cause for concern. The number of poly-ons of the 3D surface patient model is approximately 110,000.he frame rate of surface rendering was 36 fps using a PC withdual 2.8 GHz CPU. The frame rate in case of volume renderingode was 12 fps. We interviewed five doctors about usability of

he system. Fig. 11 shows the result of five-grade evaluation forurface rendering mode and volume rendering mode. Thereere many opinions stating that volume rendering mode wasore preferable and effective in clinical use. Because they

re used to observing and diagnosing with DICOM imagesnd volume rendering includes all information of innertructure.

. Discussion and conclusions

he conclusions of this paper are summarized as follows:

1) A simulation system of surgical robot setup for ZEUSsurgery has been developed with the function of inverse-kinematics computation. The motion of the surgical robotcould be preoperatively simulated and rehearsed with theconstraint condition at the trocar site, and automatic guid-ance of the surgical robot has been enabled by inverse-kinematics.

2) Being integrated with a haptic interface, surgeons coulddrag and edit the arms of the virtual surgical robot in amanner that has consistent kinematics with the real robot.The background running kinematics process enabled an

qualitative approach. A statistical evaluation of this systemwould be desired by increasing the applied clinical examplesfor the later stage.

e f e r e n c e s

[1] Gary S. Guthart, The intuitive telesurgery system, in:Proceedings of the 2000 IEEE International Conference onRobotics and Automation, ICRA 2000, San Francisco, CA,USA, April 24–28, 2000, pp. 618–621.

[2] M. Ghodoussi, S.E. Butner, Y. Wang, Robotic surgery—thetransatlantic case, in: Proceedings of the 2002 IEEEInternational Conference on Robotics and Automation,ICRA 2002, Washington, DC, USA, May 11–15, 2002, pp.1882–1888.

[3] H. Shennib, A. Bastawisy, M.J. Mack, F.H. Moll,Computer-assisted telemanipulation: an enablingtechnology for endoscopic coronary artery bypass, Ann.Thorac. Surg. 66 (1998) 1060–1063.

[4] H. Reichenspurner, R. Damiano, M. Mack, D. Boehm, B.Meiser, R. Elgass, B. Reichart, Use of the voice-controlledand computer-assisted surgical system zeus forendoscopic coronary artery bypass grafting, J. Thorac.Cardiovasc. Surg. 118 (1) (1999) 11–16.

[5] G.B. Cadiere, J. Himpens, O. Germay, R. Izizaw, M.Degueldre, J. Vandromme, E. Capelluto, J. Bruyns,Feasibility of robotic laparoscopic surgery: 146 cases,World. J. Surg. 25 (11) (2001) 1467–1477.

[6] J. Marescaux, M.K. Smith, D. Folscher, F. Jamali, B.Malassagne, Telerobotic laparoscopic cholecystectomy:initial clinical experience with 25 patients, Ann. Surg. 234(1) (2001) 1–7.

[7] M. Hashizume, M. Shimada, M. Tomikawa, Y. Ikeda, I.Takahashi, R. Abe, F. Koga, N. Gotoh, H. Konishi, S.Maehara, K. Sugimachi, Early experiences of endoscopicprocedures in general surgery assisted by a

72 c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 8 3 ( 2 0 0 6 ) 63–72

computer-enhanced surgical system, Surg. Endosc. 16 (8)(2002) 1187–1191.

[8] R.A. Robb, D.P. Hanson, J.J. Camp, Computer-aided surgeryplanning and rehearsal at Mayo Clinic, Computer 29 (1996)39–47.

[9] J. Berkley, G. Turkiyyah, D. Berg, M. Ganter, S. Weghorst,Real-time finite element modeling for surgery simulation:an application to virtual suturing, IEEE Trans. VisualizationComput. Graph. 10 (3) (2004) 314–325.

[10] S. Cotin, H. Delingette, N. Ayache, Real-time elasticdeformations of soft tissues for surgery simulation, IEEETrans. Visualization Comput. Graph. 5 (1) (1999) 62–73.

[11] S. Suzuki, N. Suzuki, A. Hattori, A. Uchiyama, S.Kobayashi, Sphere-filled organ model for virtual surgerysystem, IEEE Trans. Med. Imaging 23 (6) (2004) 714–722.

[12] The LapSim System, http://www.surgical-science.com.[13] The SimSurgery Educational Platform (SEP) for Robotic

Surgery, http://www.simsurgery.no.[14] S. Suzuki, N. Suzuki, M. Hayashibe, A. Hattori, K. Konishi,

Y. Kakeji, M. Hashizume, Tele-surgery simulation toperform surgical training of abdominal da Vinci surgery,

in: ICS 1281: Computer Assisted Radiology and Surgery,Elsevier, 2005, pp. 531–536.

[15] E. Coste-Maniere, L. Adhami, F. Mourgues, O. Bantiche,Optimal planning of robotically assisted heart surgery:first results on the transfer precision in the operatingroom, Int. J. Rob. Res. 23 (4–5) (2004) 539–548.

[16] V. Falk, F. Mourgues, L. Adhami, S. Jacobs, H. Thiele, S.Nitzsche, F.W. Mohr, E. Coste-Maniere, Cardio navigation:planning, simulation, and augmented reality in roboticassisted endoscopic bypass grafting, Ann. Thorac. Surg. 79(6) (2005) 2040–2047.

[17] M. Hayashibe, N. Suzuki, M. Hashizume, Y. Kakeji, K.Konishi, S. Suzuki, A. Hattori, Preoperative planningsystem for surgical robotics setup with kinematics andhaptics, Int. J. Med. Rob. Comput. Assist. Surg. 1 (2) (2005)76–85.

[18] Y. Nakamura, Advanced Robotics: Redundancy andOptimization, Addison-Wesley Longman Publishing Co.,Inc., 1990.

[19] PHANTOM Omni, SensAble Technologies, Inc.,http://www.sensable.com.