2D Cartoon Character Deformation by Sketch Skeleton

Xin Feng Computer School,

Communication University of China ,Beijing, 100024, China

e-mail: fengxin@cuc.edu.cn

Minyong ShiAnimation School,

Communication University of China,Beijing, 100024, China,

e-mail : myshi@cuc.edu.cn

Abstract—Because setting character key poses in animation is time-consuming , most work today are done in solving this problem. Representing the character pose by skeleton that has the joints and bone information is very intuitive , the sketch operation is very simple in interface and interactive mode . So, we present a novel 2D Cartoon character deformation method combining skeleton with sketch: First user sketch several key poses using of skeleton that indicate the moving sequence. Then sketch the object character’s skeleton corresponding to the origin skeleton that means they are topological isomorphism. Finally, we get the characters animation sequence by using deformation algorithm in terms of the least squares. Most important thing is deformation algorithm, the algorithm aims to preserve shape properties ,that is deformations should be as rigid as possible. We use least squares to solve the deformation function. From the laboratory experiment, the system is not only easy to operate but also can implement character exaggerative deformations, which can be performed in real-time. Specially, the key pose implying from skeleton can retarget different objects .

Keywords-animation; character pose; deformation algorithm;least squares

I. INTRODUCTION

2D cartoon character deformation is an essential thing in animation. A good shape deformation system aims to produce visually pleasing results with simple operations and to provide interactive feedback to users. Many techniques have been proposed to achieve a balance between these two objectives.

There has been much previous work on shape deformation, and we discuss here only those works most related to ours.

The best known method for shape deformation may be free form deformation (FFD) [1,2,3]. In FFD, a shape is embedded in a lattice, then is deformed by moving the control points of the lattice. While FFD is simple and easy to use, it does not take into account the natural way in which shapes features are controlled.

To achieve physically plausible deformation, physicallybased simulations can be employed [4,5,6]. Among thesemethods, the most popular is mass-spring models [5].However, it is too slow to converge and needs careful tuning of various parameters.

Gradient domain techniques [7,8,9,10,11,12] castdeformation as an energy minimization problem. The energy

function contains both a term for a detail-preservingconstraint and a term for a position constraint.

Our paper builds primarily on a recent paper by T.Igarashi[10] and S. Schaefer[13]. Igarashi proposes a point-based image deformation technique for cartoon-like images in which the resulting deformations are as “rigid-as-possible”. Such deformation has the property that amount of local scaling and shearing is minimized. In contrast, Schaeferpresent image deformation method based on Moving LeastSquares , solve a small linear system (2×2) at each point in a uniform grid.

Since, many cartoon characters have a skeleton. Skeleton-based deformation can provides an intuitive approach to control deformation of animal-like shapes. Skeleton-based algorithms define the position of a point as a weighted linear combination of the initial state of the point projected into several moving coordinate frames, corresponding to the bones.

So, we combine skeleton with sketch , implement the cartoon character deformation as rigid as possible by using least squares, aiming to visually pleasing results with simpleoperations. In the following section we will introduce the system flow, skeleton description, shape deformation,implementation and conclusion.

II. OVERVIEW

The input of our system is a 2D sketch skeleton represented as a simple character pose. The skeleton is a line set organized by multi-tree structure. The animating information from skeleton apply a specific character , we obtain that special character’s pose, that is a frame in animation sequence.

In essence, setting sketch pose is setting basic movement of character , such as the animator sketch the draft, it has the following features : The sketch skeleton is simple and direct , it has sufficient information that including character movement and joint structure and relative position. The structure of the skeleton is uniform , it consists of sketch lines that easily stored and organized ,it is very convenient when user interact the system.

First, We input a set of sketch skeleton as key pose. Second , resample and interpolate the origin input lines. Third, import special character, sketch the character skeleton, set up the corresponding relationship between the source skeleton and target characters. Last, the system make the

2010 Third International Joint Conference on Computational Science and Optimization

978-0-7695-4030-6/10 $26.00 © 2010 IEEE

DOI 10.1109/CSO.2010.32

447

character deformate by using the source skeleton information, so we get the deformation sequence. Seeing Figure 1

Figure 1. System process flow

III. SKELETON DESCRIPTION

We represent the sketch skeleton using the uniform sample interval lines. Now we introduce the sampling methods , the interpolation ways, and the data organization.

The input from a mouse is discrete points. These points are dense because of machine sampling rate and they are unmeaning owing to user sketching speed. So, we will resample the sketch lines. We obtain discrete points from the input device , then calculate the line segment length according to the orderly discrete points, set the proper sampling interval in proportion to length, last form the lines after uniform resampling.

The system use the similar rigid interpolation method, that is interpolate the length and the angel. Suppose

1 2, nP P P is the origin point set in lines,1 2, nQ Q Q is the

target point set, the algorithm step is : As the first vertex 1( )V t , we use linear interpolation

1 1 1( ) (1 )V t t P tQ (1)As the second vertex 2 ( )V t , use the length interpolation

and the angel interpolation (see the Figure2) .2 1 1 2( ) ( ) ( ) ( ) ( )V t V t V t V t e t (2)

where 1 2 1 2 1 2

1 2

( ) ( ) (1 )( ) (1 )

( )( )( )

V t V t t PP t Q Q

v t t e tev te tv t

Figure 2. The 1V vertex and

2V vertex interpolation

As the other vertexes ( ), (2 )iV t i n , use the linear interpolation for length and relative rotation angel with parent skeleton.

1 1( ) ( ) ( ) ( ) co s( ( )),sin ( ( ))i i i iV t V t V t V t t t (3)

Figure 3. TheiV vertex interpolation(i>2)

The system organize the data as the tree structure that is corresponding to the joint structure. So first we traversal the nodes from the root of the tree , if the joint is the root node, we obtain the new root node by interpolating the position ,else use the similar rigid interpolation method, get the not root node.

IV. SHAPE DEFORMATIONS

A. Moving Least Squares DeformationFirst consider image deformations based on collections of

points with which the user controls the deformation. Let p bea set of control points and q the deformed positions of the control point p. We construct a deformation function f using Moving Least Squares [14]. Given a point v in the image, we solve for the best affine transformation lv(x) that minimizes

2( )i v i ii

w l p q (4)

where pi and qi are row vectors and the weights wi have the form

2

1i

i

wp v

Because the weights wi in this least squares problem are dependent on the point of evaluation v, we call this a Moving Least Squares minimization. Therefore, we obtain a different transformation lv(x) for each v.

Now we define deformation function f to be f (v) = lv(v).Observe that as v approaches pi, wi approaches infinity and the function f interpolates, (i.e: f (pi) = qi). Furthermore, if qi

= pi, then each lv(x) = x for all x and, therefore, f is the identity transformation f (v) = v. Finally, this deformation function f has the property that it is smooth everywhere.

Now since lv(x) is an affine transformation, lv(x) consists of two parts: a linear transformation matrix M and a translation T.

( )vl x xM T (5)We can actually remove the translation T from this

minimization problem further simplifying these equations. Equation 4 is quadratic in T. Since the minimum value is where the partial derivative with respect to each of the free variables in lv(x) are zero, we can solve directly for T in terms of the matrix M. Taking the partial derivative with respect to the free variables in T produces a linear system of equations. Solving for T yields that

* *T q p M

where p and q are weighted centroids.

1.origin input

2.resample and interpolation

3.deformation result

448

*

i ii

ii

w pp

w,

*

i ii

ii

w qq

w

With this observation we can substitute T into equation 5 and rewrite lv(x) in terms of the linear matrix M.

* *( ) ( )vl x x p M q (6)Based on this insight, the least squares problem of

equation 4 can be rewritten as 2ˆˆi i i

i

w p M q (7)

where, *ˆ i ip p p , *ˆi iq q q , minimizes equation 7 ,we get the transformation matrix M. We are interested in the case where M is a rigid transformation. So we find closed-form solutions to Moving Least Square deformations with rigid transformations.

B. Rigid Transformations By PointsOur deformation purpose is as rigid as possible, that is,

the space of deformations should not even include uniform scaling. We use the Iterated Closest Point community [15]. Horn shows that the optimal rigid transformation can be found in terms of eigenvalues and eigenvectors of a covariance matrix involving the points pi and qi. We showthat these rigid deformations are related to the deformationsfrom above.

We find that the rigid transformation need a different

constant in the solution so that TM M I . Letˆ( )r i i

i

f v q A

where Ai is defined in equation 8,

*

*

ˆˆ ( )

Ti

i ii

p v pA w

p v p (8)

which may be precomputed. This vector ( )rf v is a rotated and scaled version of the vector v-p* . To compute

fr(v) we normalize ( )rf v , scale by the length of v-p* (whichalso can be precomputed), and translate by q*.

* *

ˆ ( )( ) | || ( ) |

rr

r

f vf v v p qf v (9)

This deformation is quite realistic and almost feels as if the user is manipulating a real object. Due to the use of rigid transformations, the deformation maintains rigidity and scale locally so that the body except the right arm of the Ginger Man retain their relative shape(Figure 1).

C. Rigid Transformation By LinesNow, we describe the rigid transformation by lines

(skeleton) in the plane. First, assume pi(t) is the ith control curve and qi(t) is the deformed curve corresponding to pi(t). We generalize the quadratic function in equation 4 by

integrating over each control curve pi(t) where 0,1t .21

0( ) ( ) ( )i i i

i

w t p t M T q t (10)

where iw is2

| ( ) |( )( )

ii

i

p tw t

p t v

and ( )ip t is the derivative of pi(t). Similar with the above part , again we can conclude the weighted centroids p* and

q*, ˆ ( )ip t and ˆ ( )iq t ,therefore ,rewrite the equation 10 in terms of M as

21

0ˆˆ( ) ( ) ( )i i i

i

w t p t M q t (11)

Since ˆ ( )ip t , ˆ ( )iq t are line segments whose end-points

are ˆia ,ib and ic , ˆ

id ,we can represent these curves as matrix products ,

ˆˆ ( ) 1 ˆ

ii

i

ap t t t

b, ˆ

ˆ ( ) 1 ˆi

ii

cq t t t

d

Equation 11 is then written as

21

0

ˆˆ(1 ) ˆˆ

i i

i i i

a ct t M

b d (12)

the minimize is1

ˆˆˆˆˆˆˆˆ

TTj ji i

i ji j j ji i

a ca aM W W

b db b (13)

where iW is a weight matrix given by 00 01

01 11i i

ii i

W

and the i

are integrals of the weight function ( )iw tmultiplied by the different quadratic polynomials.

100 2

0101

0111 2

0

( )(1 )

( )(1 )

( )

i i

i i

i i

w t t dt

w t t tdt

w t t dt

Similar with deducing above , the deformation function ( )af v as

*

ˆ( ) ˆ

j

a jj j

cf v A q

d

where jA is a matrix of the form1

*

ˆˆˆ( ) ˆˆˆ

TTji i

j i ji ji i

aa aA v p w W

bb b

During the deformation, the end-points ai and bi of the line segment pi(t) are fixed while the user manipulates the end-points ci and di of the line segments qi(t). Since Aj is independent of ci and di, Aj can be precomputed.

For rigid-body transformations , the transformation matrix is similar with equation 13 except we choose a different scaling constant so that TM M I .

This deformation is non-linear ,but we can compute it in a simple fashion using equation 9 . This equation uses the rotated vector ( )rf v ,scales the vector so that its length is

449

*v p and translates by *q .For this deformation using line segments , the rotated vector is given by

ˆˆ( )r j j jj

f v c d A

where jA is a 4×2 matrix.

*

*

ˆˆˆ ( )

jT

j

j jj

j

aa v p

A Wv pb

b

(14)

V. IMPLEMENTATION

To implement these deformations, we precompute as much as possible for the deformation functions f(v). When we apply the deformation to an image, we typically do not apply f(v) to every pixel in the image. We approximate the image with a grid and apply the deformation function to each vertex in the grid. We then fill the resulting quads using bilinear interpolation. In practice, this approximation technique produces deformations indistinguishable from the more expensive process of applying the deformation to every pixel in the image.

We have implemented the deformation algorithm on a 3.2GHz Pentium 4 workstation with 1GB memory. For the deformation of Ginger man whose skeleton has eight lines (seeing Figure 1) , the system costs 3.6 ms with the rigid least square deformation and uses a grid of size 100×100.The solution time is mainly spent for solving the square root in the deformation function, but the transformations are still quite fast.

We have tested our deformation method on various kinds of 2D cartoon characters. Figure 4 shows the deformation of a flower. The stem of the flower is deformed naturally, andthe shape of the flower is preserved well. The deformation information can be used by another object such as a maple leaf, then we obtain the leaf similar poses. Figure 5 shows the deformation of a cartoon man , (a)(c)(e) are the key poses by skeleton, (b) and (d) are the interpolation results ,and figure 5 shows a large scale deformation of the legs of the cartoon man.

Figure 4. a) sketch three key skeletons b) target flower and realize

deformation c) the deformation can be reused to a maple leaf

Figure5. Deformatin of a cartoon man. (a)(c)(e) are the original shape ,(b)(d) are the deformation results.

VI. CONCLUSION

We have described a real-time 2D cartoon character deformation method , the deformation algorithm is implemented in terms of least squares optimization. The system has visually pleasing results with simple operations.

In future work, it might be interesting to experimentwith some methods improving the system. Our system demand sketch the skeleton first , but this process is little laborious . If we obtain the skeleton automatically , the system is more convenient. So, we plan to consult the methods [16] for solving this problem. On the other hand , the deformation information can be retargeted (seeing the Figure 4), but the skeletons must be topological isomorphism that largely prevent the application. Finally, in the future we would like to explore generalizing these deformation methods to 3D surfaces. Such a generalization has potential applications in the motion capture field where animation data can take the form of skeleton lines in space for each frame ofanimation.

ACKNOWLEDGMENT

This work was received support from the State Administration of Radio Film and Television of China( Project No.2007-07) and School youth research projects.

REFERENCES

[1] R.MacCracken, K.Joy, “Free-form deformations with lattices of arbitrary topology,” SIGGRAPH 96 Conference Proceedings, pp. 181–188 ,1996.

[2] T.Milliron, R.Jensen, R.Barzel, A.Finkelstein, “A framework for geometric warps and deformations,” ACM Trans. Graphics 21(1), 20–51 ,2002.

[3] T.Sederberg, S.Parry, “Free-form deformation of solid geometric models, ” SIGGRAPH 86 Conference Proceedings 20(4), 151–160 ,1986

[4] G.Celniker, D.Gossard, “Deformable curve and surface finite-elements for free-form shape design, ” SIGGRAPH 91 Conference Proceedings, pp. 257–266 ,1991.

[5] S.F.F.Gibson, B.Mirtich, “A Survey of Deformable Modeling in Computer Graphics,” Technical report TR-97-19, Mitsubishi Electric Research Laboratories,1997.

[6] D.L.James, D.K.Pai, “Artdefo Accurate real time deformableobjects. ” SIGGRAPH 99 Conference Proceedings, pp. 65–72 ,1999.

450

[7] M.Alexa, D.Cohen-Or and D.Levin, “As-rigid-as-possible shape interpolation,” SIGGRAPH 2000 Conference Proceedings, pp. 157–164,2000.

[8] Y.Yu, K.Zhou, D.Xu, X.Shi, H.Bao, B.Guo, H.Y.Shum, “Mesh editing with poisson-based gradient field manipulation, ” ACM Trans. on Graphics 23(3), 644–651 ,2004.

[9] O.Sorkine, Y.Lipman, D.Cohen-Or, M.Alexa, C.R¨ossl, H.P. Seidel,“Laplacian surface editing, ” Symposium on Geometry Processing, pp. 179–188. ACM SIGGRAPH / Eurographics ,2004.

[10] T.Igarashi, T.Moscovich, J.F.Hughes, “As-rigid-aspossible shape manipulation, ” ACM Trans. Graphics 24(3), 1134–1141,2005.

[11] Y.Lipman, O.Sorkine, D.Levin, D.Cohen-Or, “Linear rotation-invariant coordinates for meshes,” ACM Trans. Graphics 24(3), 479–487 ,2005.

[12] K.Zhou, J.Huang, J. Snyder, X.Liu, H.Bao, B.Guo, H.Y.Shum, “Large mesh deformation using the volumetric graph laplacian, ”ACM Trans. Graphics 24(3),496–503 ,2005.

[13] S.Schaefer, T.McPhail, J.Warren. “Image deformation using moving least squares,” ACM Transaction on Graphics(TOG), 25(3): 533–540,2006.

[14] D.Levin ,“The approximation power of moving least squares,”Mathematics of Computation 67, 224, 1517–1531,1998.

[15] B.Horn, “Closed-form solution of absolute orientation using unit quaternions, ” Journal of the Optical Society of America A 4,4 (April), 629–642, 1987.

[16] I.Baran,J. Popovi´c, “Automatic rigging and animation of 3d characters, ” ACM Transactions on Graphics 26, 3 (July), 72:1–72:8,2007.

451