3D Visualization Method for Maxillofacial Surgical Planning by Using X-ray and Photo Images

Young-In Kim*, Jung-Hyun Park**, Chang-Hun Kim*

Dept. of Computer Science & Engineering, Korea University*,Dept. of Oral&Maxillofacial Surgery, Dental College, Yonsei University**

{yikim,chkim}@cgvr.korea.ac.kr*, goodputt@hotmail.com**

Abstract

This paper describes a prototype system for predicting of human facial shape and visualizing of realistic 3D images after maxillofacial surgery for patient with facial deformities from three input images consisting of lateral Xray, lateral photo and frontal photo. The basic idea is to join a accurate predicting variation of facial shape with a 3D visualizing postsurgical facial shape. Firstly, we draw out the important parts of face, contours and control points in the profile of a patient from Xray image. Secondly, when a surgeon designates beforehand the change of the direction and magnitude of a maxillary bones (the lower and upper jawbone) at plan step, the system forms the postsurgical image using warping method and predicting function which calculate the change of movement of a soft tissue. The predicting function is based on the clinical study obtained from 100 patients and the warping method is implemented considering anatomic relation of facial soft tissue. Finally we generate the three dimensional images of postsurgical facial shape using the deformation estimating method which predict the 3dimensional movement of vertices on face.

Keywords: Digital Surgery, Medical Visualization, Surgical Planning System

1 Introduction

Digital dental surgery planning system has many advantages in that of predicting the postsurgical morphology and appearance of human face, or solving the problem of a fair facial surface before the actual surgery is carried out. Early works [4,6,7] need expensive equipment and high computation cost because it require the complex and laborious processing steps of mapping the soft tissue structure to the 3D skull model which is constructed from 2D CT or MRI data. More over these systems are not adequate for maxillofacial surgery because subtle malformation can strongly affect the appearance of a face and these systems is hard to handle to surgeon. Thus typically, the facial operation surgeon draws the patients predicted profile using X-ray to give at least a 2D appearance of the future face. In this thesis, we propose the facial surgery planning system, which computes accurate 2D lateral facial contour and realistic 3D picture of the post surgical shape.

Our system consists of two modules, one is about the virtual surgery step and the other is 3D visualization step. In virtual surgery step, we use lateral X-ray of patient and predicting function for computing predicted soft tissue profile after maxillofacial surgery. In 3D visualization step, we use two photos, which are frontal and lateral images of patient, and deformation estimating method for generating 3D appearance of post surgical facial shape. Figure1 shows the simple procedure of our surgery predicting system.

a) b) c)Figure 1: Illustration of system flow for maxillofacial surgery planning.a) Lateral photo and X-ray image of the presurgical face. b) 2D postsurgical image after maxillofacial surgical simulation. c) 3D postsurgical appearance.

The remainder of this paper is structured as follow: In section 2 we describe the motivation of our research. And we give an overview of the system in section 3. Section 4 we explain the principles of the predicting function, which is used for virtual surgery. Section5 describes the deformation estimating method of soft tissue for generating 3D appearance. Finally, we demonstrate and discuss the proposed system with experimental results obtained from the X-ray image and frontal and lateral facial photos in section 6.

2 Motivation

There has been much research in the field of facial surgery simulation. Most facial surgery simulation systems used today in 3D medical image data like CT or MRI. These 3D medical image data requires high cost and complex preprocessing for surgical simulation. So we choose the planar medical image data, X-ray image, traditionally it has been used in clinical area. Early works such as [9] restricted themselves to 2D lateral image generation of the malformed patients. So we propose the system which requires a simple and cheap X-ray image data and generates 3D appearance of postsurgical facial shape.

3 System Overview

This section explains on different procedures, data acquisition and preprocessing steps, virtual surgery step, and 3D visualization step in the chart of Figure 2. Our data sources consist of X-ray image, frontal and lateral facial photos.

First of all, an initial facial contour is extracted from the X-ray image and control points is created on a lateral contour.

Secondly, we simulate the maxillofacial surgery system, which enable to compute the variations between the presurgical profile and postsurgical profile. As this result postsurgical profile contour is created and it is warped with the lateral facial photo. Then we warp the lateral photo to do the virtual surgery result.

Finally, we execute the third pass procedure, 3D postsurgical image generation. In our system, we individualize the generic facial model and predict the postsurgical facial images on frontal view. Figure2 shows the overall flow structure of our dental

surgery planning system.

Figure 2: System overview.

3.1 Data Acquisition and Preprocessing

The first pass procedure of our virtual surgery system is the data acquisition and preprocessing step. In this step, we prepare two 2D wire frame templates composed of feature points with predefined relation for front and side views and draw out the facial profile on X-ray image and align facial lateral photo with it.

Because X-ray image maintains the 1:1 ratio of the surgery area, the lateral photo is transformed by fixed profile on X-ray. Figure 3 shows the result after malformed profile contours extraction and adaptation of them. These images can be aligned by rotation and translation and scaling.

a) b)Figure 3. Facial contour extraction. a) extracted facial profile on X-ray image. b) Adaptation profile to X-ray image and photo.

3.2 Virtual Surgery Operation

This section explains the process of virtual operation necessary to accomplish

facial surgery simulations. First, we cut the malformed mandible contour and move it to correct it. Next, the variations of the control points on facial soft tissue contours between presurgery and postsurgery are computed by predicting function. Finally, non-control points consisting of soft tissue contours are interpolated by linear interpolation method.

Figure 4 shows the process of virtual operation. Figure 4a represents the cut and moved malformed mandible contour. Figure 4b is the result from calculating movement of the control points on soft tissue. Figure 4c shows the linear interpolating movements of non-control points on soft-tissue contours.

a) b) c)Figure 4. Maxillofacial surgery on the hard tissue contour. a) cutting and moving the mandible. b) calculating the movements of control point on soft tissue. c) linear interpolating the movements of non-control point on soft tissue contours.

This system makes synthetic postoperative image of post-operation by image warping technique. Image warping process consists of triangulation and color interpolation. Delaunay triangulation used on this system defines local areas on lateral photo, each area is reorganized by virtual surgery, and color information in area is interpolated.

Figure 5 shows this process about generating warped postsurgical image. Figure 5b shows the divided regions by triangulation and Figure 5d shows the warped result of the modified model by virtual surgery.

a) b) c) d)Figure 5. Virtual postsurgical facial image generation. a) Input data : lateral facial

photo. b) Triangulation of input image. c) Virtual operation d) Warped image of the modified model by virtual surgery

3.3 3D postsurgical image generation

In this section, we explain the process for visualizing 3D postsurgical image generation. First of all, we make individual face to visualize 3D images. After preprocessing, we get two wire templates on front and side views and use them to modify a generic facial model. In this paper, generic facial model with 915 vertices is modified to make an individualized smooth facial surface. Figure 6 shows the process of individualizing a generic facial model.

Figure 6. Individualization of generic facial model

4 Computation of the Soft-Tissue Variation

In this paper, we use the predicting function to calculate the variation of postsurgical soft tissue profile. As the variation of human soft tissue can’t be generally computed by linear equations, we suppose that it can be computed by non-linear equations. To formulate our predicting function, we define 8 control points on hard tissue and 10 control points on soft tissue. Figure 7 shows these control points. Control points on hard tissue consist of A(ANS), B(A), C(MxI), D(MxM), E(MnI), F(B), G(Pg), H(Me) and control points on soft tissue consist of A’(Pn), B’(Sn),

C’(A), D’(Ls), E’(Stms), F’(Stmi), G’(Li), H’(B’), I(Pg’), J’(Me’). Figure 7. Facial control points on hard tissue and soft tissue.

This predicting function is based on the clinical study obtained from 100 patients. For measuring the variation between presurgical profile and postsurgical profile, we define eq. 1 as following.

, Eq. 1

The horizontal variation on soft tissue A’ is formulated as following.

Hard control points

Soft control points

Eq. 2

As the preceding equation 2 , other horizontal variation on control points from B’ to H’ is formulated. Equation 3 explains the linear regression for finding values of

.

Eq. 3

:estimated value, :input value , :parameter , :the number of data, : the number of

parameter

Using hard-tissue location changed from virtual surgery produce the soft-tissue change by soft-tissue movement prediction functions. Each partial function for a feature point on soft-tissue makes the movement based on multiple feature points on hard-tissue. Table 1 shows the example of two partial functions.

Table1 Two functions about hard and soft tissue relation

Soft-tissue Hard-tissue

VPn 0.44409 * ANS – 0.27588 * vA + 0.05905

VB’ 0.65773 * vMe – 0.13109 * hMe + 0.21406 * hMnM – 1.31458

11 soft tissue control points predicted by that of hard tissue control point is as small as this system represent the smooth facial outline. So, additional points of soft tissue is defined as follows:

Eq. 4

, where p is the control point coordinate, w is the maximum length effected by p, and p’ is the point on soft-tissue. [Figure 4] shows that this system determines coordinate of the feature point on soft-tissue by prediction function, and interpolate the additional points by equation 4.

Maxillofacial surgeon generally says that an error tolerance on maxillofacial surgery is less than 2mm. In figure 8, we represent the comparison of result on actual surgery with estimated surgery. You can see that the differences between this two graphs are not greater than 2 mm.

Figure 8. The comparison of result between actual surgery and estimated surgery.

5 Deformation Estimating Method

Our system makes frontal postsurgical facial image by the deformation estimating method. 3D visualization process consists of the individualization of a generic facial model and the calculation of 3dimensional movement of soft tissue on face. For calculating the movement of control points, we define the deformation estimating method. First, we measure 3 dimensional variations of control points between soft tissue and hard tissue from CT data. Figure 8 shows the 3 dimensional control points of the soft tissue. The number of 3 dimensional control points is seven, Pn, Sn, A, Ls, Li, B’, Pg’ , except Stms, Stmi, Me.

a) b) c)

Figure 9. 3 dimensional control points from CT data. a) soft tissue control points on lateral view b) control points on frontal view. c) control points from –90° to +90° on sectional view.

Secondly, we calculate the 3 dimensional variations on facial soft tissue. For this computation, an axis of coordinates is established on individual facial model. We use the following equation to compute the postsurgical soft tissue control points. In this equation 5, P(x,y,z) is a control point on soft tissue contour before virtual maxillofacial surgery and P’(x’,y’,z’) is computed position of it after virtual maxillofacial surgery. P’’(x’’,y’’,z’’) in figure 10 is a temporary point projected on xy-plane a point P(x,y,z).

, , . Eq. 5.

a) b)

Figure 10. Computing variation of 3D difference before and after maxillofacial

Z

surgery. a) lateral comutation model. b) sectional computation model.

6 Results

The goal was to predict the facial shape after procedure in maxillofacial surgery. Figure 11-12 show the shape of soft tissue after maxillofacial surgery. The calculations were carried out on a Intel Pentium-III 700 processor, 256MB RAM, Microsoft Windows NTTM Workstation 4.0 and our system was implemented by using Microsoft Visual C++TM 6.0 and SGI OpenGL. We can compare a estimated postsurgical image with a actual postsurgical image. (See Figure11). We can find the high similarity of two images.

a) b) c)

Figure 11. Comparison simulated results with actual surgery. a) Input data: presurgical images. b)

Estimated postsurgical images. c) Actual postsurgical image

a) b) c) d) f)

f)

Figure 12. 3D images after maxillofacial surgery. a,b) presurgical frontal and lateral pictures of patient. c) Estimated postsurgical facial image. d,e) Actual postsurgical frontal and lateral photo. f) 3d visualization of estimated postsurgical result.

7 Conclusion

We present a system which enables us to predict the deformations of the facial shape after surgical procedures. Our system can generate a high realistic and accurate postsurgical images by predicting function which obtained from 100 clinical studies. In our system, maxillofacial surgeon can easily extract the outline profile and predict the postsurgical facial shape.

8 Reference

[1] Haider, A. Md., Takahashi, E. and Kaneko, T., Automatic Reconstruction of 3D Human Face from CT and Color Photographs, IEICE Trans. Inf. & Syst.., vol.E81-D, No.9, pp.1287-1293, Sep. 1999.

[2] Haider, A. Md., Takahashi, E. and Kaneko, T., A 3D face reconstruction method from CT image and color photographs, IEICE Trans. Inf. & Syst.., Vol.E81-E, No.10, pp.1095-1102, Oct. 1998.

[3] Hounsfield, G.N., and Ambrose, J.A. Computerized transverse axial scanning tomography. British Journal of Radiology, pp.1016-1022, 1973

[4] Koch, R.M., Gross, M.H., Carls, F.R., von Büren, D.F., Fankhauser, G. and Parish, Y.I.H., Simulating facial surgery using finite element models, Computer Graphics(SIGGRAPH’96 Proceedings), pp.421-429, Aug. 1996

[5] Lee, W-S., Thalmann, N.M., Fast head modeling for animation, Image and

Vision Computing, pp.355-364, 2000.[6] Xia, J., Wang, D., Samman, N., Yeung, R.W.K., Tideman, H., Computer-

assisted three-dimensional surgical planning and simulation: 3D color facial model generation, International journal of oral & maxillofacial surgery, Vol.29, No.1, pp.20-10, Feb. 2000.

[7] Xia, J., Ip, H.H.S., Samman, N., Wang, D., Kot, C.S.B., Yeung, R.W.K., Tideman, H., Computer-assisted three-dimensional surgical planning and simulation: 3D virtual osteotomy, International journal of oral & maxillofacial

surgery, Vol.29, No.1, pp.11-17, Feb. 2000.[8] Dr.Pss,(http://www.bit.co.kr/medical- i nfo/html/ drpss.htm )[9] QuickCeph Systems, (http://www.quickceph.com/)