Pose Synthesis Using the Inverse of Jacobian Matrix...

Pose Synthesis Using the Inverse of Jacobian Matrix

Learned from Examples

Chunpeng Li∗

Institute of Computing Technology,Chinese Academy of Sciences

Graduate School,Chinese Academy of Sciences

Shihong Xia†


Zhaoqi Wang‡


ABSTRACT

This paper presents a method of pose synthesis based on a low-dimensional space and a set of characteristics of motion learnedfrom examples. This method consists of two phases: learning andsynthesis. In the learning phase, a low-dimensional and discreterepresentation of the space of natural poses is constructed by us-ing a Self Organizing Map (SOM). Meanwhile, a set of matricesis extracted from the motion data. These matrices describe howthe poses change with the end-effectors’ positions, and play a keyrole in synthesizing natural looking results. In the synthesis phase,a lightweight algorithm based on the learned parameters is used.The synthesis process is very efficient because there is no time-consuming calculation, like numeric optimization or matrix invert-ing. Compared with other methods, our method not only can pro-duce natural looking poses in real-time, but also works well withconstraints positioned in a larger range. We apply our method inapplications of interactive pose editing, real-time motion modifica-tion, and pose reconstruction from image. The results have proventhe robustness and effectiveness of our method.

Keywords: Inverse Kinematics, Self Organizing Map, JacobianMatrix, Character Animation.

Index Terms: I.3.7 [Computer Graphics]: Three-DimensionalGraphics and Realism—Animations; I.2.6 [Artificial Intelligence]:Learning—Connectionism and neural nets

1 INTRODUCTION

In the past decade virtual reality applications have emerged in manyareas, such as entertainment, training, communication and so on.In these applications, virtual human, as the digital representation ofhuman, is one of the most important elements to improve the inter-active capability and immersive experience. However, it remains tobe a challenging issue faced by modeling virtual human to synthe-size natural motions including interactive motions with virtual en-vironment and with the human user. In this paper, a new approachto synthesizing and controlling virtual character poses from a set ofconstraints is presented.

In virtual environment, a virtual character is often modeled as aset of rigid segments connected by joints. Given a set of joint an-gles, we can calculate the posture of the character, which is calledforward kinematics. On the contrary, the problem of computing aset of joint angles to position the end-effector at a desired locationis called inverse kinematics. Inverse kinematics (IK) is more intu-itive and simpler because it operates the positions of few segmentsin 3D space rather than the angles of each joint in angular space.

∗e-mail: [email protected]†e-mail: [email protected]‡e-mail: [email protected]

Figure 1: A simple demonstration of our method. One end of themodel is fixed at the original point. Top row: One example motionis given, shown in the left column. The poses in the right columnare the outputs by our method. Bottom row: A different example isgiven. Our method generates a set of new poses.

The solver of inverse kinematics problem is one of the most im-portant tools for interactive control of characters in virtual realityapplications, such as interactive games, workspace computing, andvirtual space teleconferences.

Given a vector p of known joint variables, also named config-uration or pose of the model, the forward kinematics problem ofcomputing the position vector g of the end-effector is a simple mat-ter of matrix concatenation and the process is denoted as:

f (p) = g (1)

But the IK problem is to place the end-effector at a specifiedposition g, named geometrical constraint. Determining the appro-priate joint variable vector p to achieve the goal requires a solutionto the inverse of equation (1):

p = f−1(g) (2)

Solving the IK problem is not easy because that the functionf (·) is nonlinear and there may be many p’s for a particular g for aredundant structure. This paper focuses on how to search for a posefor a virtual character satisfying constraints in the space of naturalposes.

Inverse kinematics originated from robotics field. The manipu-lation of robot arms with 6 up to 9 degrees of freedom (DOF) waswell solved. However, skeletons of virtual human are much morecomplex which usually have more than 40 DOFs. Although manymethods have been proposed to solve inverse kinematics problemsin the field of character animation, little research have been downto try to extract characteristics out of real motion data for inversekinematics control.

99

IEEE Virtual Reality Conference 2007March 10 - 14, Charlotte, North Carolina, USA1-4244-0906-3/07/$20.00 ©2007 IEEE

We propose a new method to solve the inverse kinematics prob-lem by extracting characteristics from real motion data. The mainidea of our approach is to learn the inverse of Jacobian matrix overposes. The Jacobian matrix of a pose indicates how the end-effectorposition varies with respect to each joint angle near the pose, whichis calculated numerically in many traditional solvers. A simpledemonstration is shown in Figure 1.

Our approach consists of two phases: learning and synthesis.In the learning phase, we project each pose of examples into two-dimensional nodes using Self Organizing Map (SOM). The weightsof each node represent a cluster of human poses. At the same time,a local Jacobian matrix inverse is learned for each node. In the syn-thesis phase, given a desired position of end-effector, a referencepose is obtained by searching these nodes whose end-effector po-sition is nearest to the desired position. Then, an iterative processusing the learned matrices is executed until the goal is reached. Thefeatures of our method are outlined as follows.

• Automation. This method can learn from examples automat-ically without guidance. And there is no need to preprocessthe examples, such as choosing an active set [3] or creatingparameterized motions [7].

• Natural results. The poses produced by this method are nat-ural, because the characteristics extracted from real motionexamples are used in the synthesis process.

• High efficiency. This method can generate required poses inreal-time, because it does not contain time-consuming pro-cesses such as nonlinear optimization or matrix inverting.

• Robustness. Previous methods based on blending or inter-polation usually can not create natural poses when the con-straints are much different from examples. However, thismethod works effectively with a larger range of positioningconstraints compared with these methods.

• Repeatability. If a given constraint always generates thesame pose, the system is said to be repeatable. This methodis repeatable because of having an initializing step in the syn-thesis phase.

2 RELATED WORK

The inverse kinematics for controlling robot manipulators has beenstudied extensively in robotics field and many analytical methodshave been proposed [2]. These methods are very efficient, but theycan not deal with the models with complicated structures.

Tolani [13] used a combination of analytical and numericalmethods to solve the IK problem for anthropomorphic limbs. Thismethod can solve problem in real-time and allow the user to exploreall possible solutions. But the main limitation of this method is thatit is only suitable for models with up to 7 DOFs.

Iterative methods are more general, which can deal with anymodel. These methods can be divided into three subclasses further,namely Cyclic Coordinate Descent (CCD) method, Jacobian matrixbased method and optimization based method. CCD method [6] is aheuristic iterative method, which optimizes only one joint in everysingle step to minimize the total objective function. This methodis relatively fast in computation and immune to singularities. How-ever, because of the heuristic strategy adopted by CCD, rotationsare not equally distributed throughout the kinematical tree whichwill make the result unnatural. Jacobian matrix based method dif-ferentiates the nonlinear objective function by constructing Jaco-bian matrix and inverting it. Differential equations are then solvedby numerical integration [17]. In order to overcome the shortcom-ing of being highly time-consuming and the singularities caused byinverting Jacobian matrix, Jacobian transpose based method [19]

and Damped Least Square method [15] were proposed. The for-mer uses Jacobian matrix transpose instead of Jacobian matrix in-verse and the latter adds a damping term to avoid singularities.Yamane [20] proposed a method which can add and remove con-straints at will. Nonlinear optimization techniques are also used tosolve inverse kinematics problems. Zhao’s work [21] is one of themost general methods. They define the distance between the currentposition of the end-effector and the goal as the objective function,and the joint angles as the variables. Then, modified BFGS is usedto solve the minimization problem. This approach is applicable toany tree structured system and can be added with other kinds ofconstraints easily. But it will slow down rapidly as the number ofvariables increases.

In recent years, with the prevalence of the motion capture de-vices, techniques based on examples have been developed. Thesemethods take a step in improving the naturalness of the results.

Interpolation is one of the most effective ways to use examples.Rose et al. proposed a system blending example motions by com-puting a best fit map between blending weights and motion parame-ters [10]. Kovar’s approach [7] can synthesis new motion sequencesby interpolating examples based on automated analysis and param-eterization of motions in large data sets. These methods have theadvantage of high efficiency, but lack an accurate inverse mappingfrom constraints to blending weights. Therefore, they have to gen-erate a dense sampling of parameter space to get an approximateresult. Recently, a novel motion interpolation technique [14] hasbeen proposed based on geostatistics, which treats motion inter-polations as statistical predictions of missing data in an arbitrarilydefinable parametric space. This method can produce accurate mo-tions with given control parameters as well as give an estimationof reliability. However, the accuracy of result relies on sufficientnumber of example motions. It requires many samples distributedaround the center of the control space to get a good result, which isalso a drawback of interpolation based methods.

Traditional methods are extended with the utilization of motioncapture data. The key idea of Komura’s work [6] is to extracta weight matrix from real motion data based on Jacobian matrixmethod. The weight matrix determines the motion of the redun-dant joints and the transformation functions that define the motionof the end effectors. Though this idea is similar to our method inextracting the characteristics of the motion from examples, the au-thors do not analyze the space of the natural pose and do not con-nect it with weighting matrix, which limits the range to positionconstraints. Example guided inverse kinematics (EGIK) systemproposed in [12] is an extension of Zhao’s work [21]. They takeinto account the distances to examples when constructing the ob-jective function. This method can solve IK problem with humanmodels having different segment lengths, but it is difficult to satisfythe requirement of real-time application because of using nonlinearprogramming techniques.

Approaches based on a learned low-dimensional space are pop-ular recently. Style-based inverse kinematics [3] describes a novelmethod which synthesizes new poses in a low-dimensional prob-abilistic space. Given training poses, a scaled Gaussian processlatent variable model (SGPLVM) is used to represent the proba-bility distribution function over poses. In the synthesis phase, anobjective function derived from the model is optimized to searchfor a new pose which not only satisfies constraints but also hasthe maximum likelihood. Another work, called mesh-based inversekinematics [11], was proposed to manipulate meshes sharing thesame idea. Their approach can find meaningful mesh deformationsthat meet specified vertex constraints efficiently. The algorithm firstconstructs a space of natural deformations using feature vectors ex-tracted from example meshes, and then an optimization techniqueis employed to search for a result in the space as close as possible tothe training examples. Both of the methods have a similarity to ours

100

Figure 2: A demonstration of our method. Left: The original motion sequence. Each yellow marker represents the position of the right hand ineach frame. Middle: Learning results. Each yellow marker represents a reference pose learned by SOM. Right: A set of poses synthesized inreal-time by dragging the right hand with mouse. The positions of the right hand form two letters ”VR” which lie on a plane.

in solving the IK problem within low-dimensional spaces which de-scribe the space of natural poses or appropriate deformations. Butthey are different from ours in the ways of modeling and synthesiz-ing. Our method uses a discrete space and a lightweight iterativeprocedure instead of continuous spaces and nonlinear optimizationprocedures for modeling and synthesizing separately, which cangain an improvement in efficiency.

Many algorithms based on machine learning for control of an in-dustrial robot have been developed in robotic field, among which isRitter’s work for visuo-motor control [8]. The objective is to teachthe robot arm to position its end-effector using information gainedsolely from the pair of cameras. Neither an external teacher norany prior information about the geometry of the robot arm or thecameras was employed. Our work is based on this work and ex-tended from it. Firstly, structure for human model is much morecomplicated than that of robot arm and multiple constraints prob-lem is considered. Secondly, the meaning of the SOM weights isdifferent. Thirdly, we extend the SOM learning algorithm by usingquaternion which is more suitable for joint angles.

3 OVERVIEW

As mentioned above, motion rate control is one of the most com-mon methods solving the inverse kinematics problem. This methodlinearizes the problem about the current configuration, which canbe denoted by the following equation:

∆g = J(p) ·∆p (3)

where

J =∂ f

∂ p(4)

Inverting the relationship of equation (3) provides the basis forresolved motion rate control.

∆p = J−1(p) ·∆g (5)

If the inverse of J is known, we can compute incrementalchanges in the joint variables that produce a desired incrementalchange in the end-effector position. Note that equation (5) standsonly when the Jacobian is both square and non-singular. Whena structure is redundant or when its pose passes through a singu-lar configuration, the Jacobian matrix cannot be inverted. In bothcases, some generalized inverses of Jacobian J+ are used to finda useful solution [15]. In the following of this paper, the matrixlearned by our method is also a kind of generalized inverse of Jaco-bian, but we drop the distinction between them for notational sim-plicity.

A simple iterative scheme for solving the inverse kinematicsproblem can be based on equation (5). In each step, a desired ∆g canbe computed from the current and desired end-effector positions.The joint incremental change ∆p can then be computed using theinverse Jacobian and integrated once to find a new joint state vectorp. The procedure is repeated until the end-effector has reached thedesired goal. Note that since the linear relationship represented byJ is only valid for small perturbations in the manipulator configura-tion, J−1(p) must be recomputed in each step [16].

The main idea of our approach is to learn the inverse of Jaco-bian matrix J−1(p) over poses from motion capture data instead ofcomputing numerically. Synthesizing new poses using the learnedmatrices can get more natural results and avoid computing the Jaco-bian matrix and its inverse. Our approach consists of the followingtwo phases.

Learning Examples. Because of the complexity of inputspace, it is difficult obtain the continuous Jacobian matrix inverseover the entire space. However, SOM is an effective technique toanalyze the complex space, which can reduce the dimension of thespace as well as discrete it. The output of the SOM defines a low-dimensional representation of the input data. The weight vector ofeach neural unit, called reference pose, can be treated as a clustercenter of input data. Besides the weight, each neural unit has a ma-trix used to store the inverse of Jacobian matrix nearby the referencepose. The details of the learning process are described in the nextsection.

Pose Synthesis. In order to generate new poses that reach thegoal, we first search for a pose in training results as an initial con-figuration. Then with the local Jacobian matrix associated with theselected pose, the residual deviation from the desired target loca-tion is transformed into a correctional fine movement. Repeat thefine movement iteratively until the target is reached or no furtherreduction of the distance could be made. Figure 2 demonstrates theprocess of our approach.

4 LEARNING PHASE

4.1 Learning the Reference Poses

SOM is a data visualization technique proposed by Teuvo Ko-honen [5] which can reduce the dimensions of data using self-organizing neural networks. SOM can transform the input of ar-bitrary dimension into a low-dimensional discrete map subject to aneighborhood preserving constraint.

A SOM consists of two layers of neurons: an input layer andan output layer. Each neuron of the input layers is connected with

101

each neuron of the output layer using a weight. The weights con-nected with a output neuron are interpreted as a reference vectorin the input space. The output layer is a one-dimensional or two-dimensional discrete grid. The learning process is competitive andunsupervised. When an input pattern is presented to the network,one neuron in the output layer is determined whose reference vectoris the closest to the input pattern. This neuron is called the winnerneuron and the weights of itself and neurons around it are movedcloser to the input pattern. Readers can refer to [4] for more details.We summarize the features of SOM here briefly.

• From the dimension reduction point of view, SOM is an ef-fective nonlinear dimension reduction method. Data from ahigh-dimensional input space is mapped onto a usually one-dimensional or two-dimensional output space, trying to findthe principal curves or principal surface of the input data [9].

• At the same time, the projection from the continuous inputspace to the discrete output space leads to an automatic clus-tering of the data, i.e. similar data samples are stored in prox-imity thereby forming clusters. This feature makes the learn-ing of Jacobian matrix inverse easy.

• The output of SOM can preserve the topological order of theinput samples. Topological order means that similar inputsamples are close to each other while dissimilar inputs arefar from each other.

The standard learning algorithm uses the Euclidian distance tomeasure the difference of input vectors and to modify the weights.However, we usually use quaternions to represent joint rotations,which can not be measured by Euclidian distances. Refer to [1] forthe basic definitions and operations on quaternion. For this reason,the learning algorithm is modified as following.

Suppose that the map consists of a set of N neuron units la-beled by indices i (i = 1,2, ...N). The weights of neuron i are de-noted as wi = {wi1,wi2, ...wiM}, where wim is the rotation of jointm (m = 1,2, ...M). These weights are randomly initialized in unitquaternion space at the beginning. Each learning step mainly con-tains 4 steps:

1. Given a pose x = {x1,x2, ...xM} of the original data as theinput of the map, compute the distance between the pose andthe reference pose of each neuron unit. The distance functionis defined as:

dist(x,wi) =

√

√

√

√

M

∑m=1

‖λm · log(x∗m ·wim)‖2 (6)

where λm is a weight indicating the importance of the joint mand x∗ is the conjugate of x.

2. Find the winner or excited neuron unit (competitive process):

c = argmini{dist(x,wi)} (7)

3. Determine the neighbor function (cooperative process):

Λc,i(n) =

{

1−dc,i

R(n)dc,i ≤ R(n)

0 dc,i > R(n)(8)

where dc,i is the distance between the unit c and the unit i inthe output layer, and R(n) is the radius of the updating range.

4. Update the weights:

wim(n+1) = Slerp(wim(n),x(n),η(n) ·Λc,i(n)) (9)

where Slerp(p,q, t) = (q · p∗)t · p and η is the learning rate.In the learning process, the learning rate η(n) and the radiusR(n) decrease with the increasing of the iteration step n.

4.2 Learning the Jacobian Matrix Inverse

We assume that the poses belong to the same neuron share the sameJacobian matrix inverse. The assumption is based on the fact thatthese poses are similar and the Jacobian matrix is a continuous func-tion over a space of poses. This assumption can be represented asthe minimization of the following error function:

E =1

2e2 =

1

2∑(∆p− J+ ·∆g)2 (10)

where ∆p means the difference between the reference pose of aneuron and a pose belongs to it, and ∆g denotes the displacementof the end-effector’s position between them. The form of ∆p couldbe a set of Euler angles or quaternions, and we use the former onein the following paragraphs for simplicity.

In order to learn the Jacobian matrix in the process of construc-tion of the SOM, the extended SOM is employed. Each neuron ofthe output layer is associated with a matrix structure which is usedto store J+. The update rule of J+ is given by an error correctionrule of Widrow-Hoff type [18], which is based on the use of instan-taneous values for the cost function. At the nth step, the inverseof Jacobian matrix is denoted as J+(n). The object of learning istrying to minimize the following equation instead of equation (10):

E(n) =1

2∑(∆p− J+(n) ·∆g)2 (11)

The updating rule is given by:

J+i (n+1) = J+i (n)+η(n) ·Λc,i(n) ·∆J(n) (12)

where∆J(n) = (∆p− J+i (n) ·∆g)∆g

T ‖∆g‖−2 (13)

is the partial derivative of E(n) at J+(n).As explained in equation (5), methods directly based on Jacobian

matrix inverse usually suffer difficulties when the model is redun-dant or its configuration is close to singularities. Our method canavoid the first problem and generate stable results as long as theexamples near the singularities are given. Because we can use thelearned matrix to calculate the increment of each joint angle cor-responding to the constraints directly, there is no need to calculatethe inverse or pseudoinverse of Jacobian matrix. Moreover, becauseof the ”cooperative” property in the learning process of SOM, therelationship between the joints and end-effector’s position near thesingular configurations can also be extracted from the examples.That is, the stability of our method is based on a priori knowledgeand on the definition of preferred direction of motion near singular-ities. However, our method does not work in some extreme situa-tions where all the poses in given training motions have the sameend-effectors’ positions. That means ‖∆g‖ equals to zero for anytwo poses. In this situation, equation (10) is degenerated, so theredoes not exist a proper J+. But in practice, these situations rarelyoccur.

5 POSE SYNTHESIS

5.1 Synthesis

Two kinds of parameters are obtained in the learning phase, a set ofreference poses wi and a set of Jacobian matrices inverses J

+i asso-

ciated with each pose. wi are the low-dimensional representationsof the space of natural poses, which are used to initialize the con-figuration. J+i describe how the joints vary with the end-effector’sposition near the pose wi, which are used to refine the configura-tion. Using these parameters in the synthesis phase not only accel-erates the solving process, but also improves the naturalness of theresults. The synthesis of a new pose given the desired position ofthe end-effector consists of two parts according to the parametersused, namely an initializing step and a refinement process.

102

Initialization. This step is used to initialize the configurationby searching for a pose wc, whose end-effector’s position is theclosest to the desired position. We denote the desired position ofthe end-effector as utgt . This step can be presented as:

p0 = wc (14)

where

c = argmini‖utgt − f (wi)‖ (15)

The subscript of p0 indicates the number of iteration.

Refinement. This process is iterative. In each step, it takesa small step towards the goal. The position of the end-effector uican be computed by forward kinematics ui = f (pi). The distancebetween it and the goal is ∆u = utgt − ui. The refinement processcan be represented as the following equation.

pi = pi−1 + J+c ·∆ui−1 (16)

Because the learned matrices are local, which are not very reli-able for large values of ∆u, we will take smaller steps by adding aparameter α(0 ≤ α ≤ 1) as following.

pi = pi−1 +α · J+c ·∆ui−1 (17)

We can use a linear search algorithm to determine the value ofα in each step. Repeat this procedure until ∆ui is small enough orthere is no further steps could be taken. In the latter situation, theobjective function reaches a local minimal which usually occurs insolving multi-constraints problems.

However, it will generate discontinuous results even when weposition the constraint continuously because of the initializing stepin each synthesis process. This problem can be overcome byweighting poses around wc.

p0 = ∑i

hc(i) ·wi (18)

pi = pi−1 +α · (∑i

hc(i) · J+i ) ·∆ui−1 (19)

where hc(i) is the weight function which has a maximum value atpoint c and decrease to zero as the distance between c and i in-creases.

5.2 Multiple constraints

In many applications, we need to set multiple constraints on the po-sitions of different end-effectors of a model. For example, duringpose editing, it would be useful to be able to specify some partsof the body fixed. Many situations can be viewed as synthesisproblems with multiple constraints. Though our method is not re-stricted to solo-constraint situations, it usually can not converge toa right answer if we use it to solve multi-constraints problems di-rectly. That is, the iteration process stops while the end-effectors donot reach the goals. These situations are caused due to the trendscaused by different goals are conflict. For example, if the motionlearned by SOM only with one hand moving rightward while onefoot moving upward, then confliction occurs when we are trying tomove the hand rightward and move the foot downward. The incre-mental change by trying to move the hand rightward causes the footmoved upward and vice versa. When the two forces are equal, theobjective function can not be reduced.

Here we propose a method to overcome this problem which isbased on the observation of the dependencey of the joint angles.Obviously, there is less dependency between joint angles on dif-ferent kinematic chains. We can fix one arm while move one legarbitrarily in a large range. Secondly, on the same kinematic chain,

Figure 3: Poses synthesized by our method. The constraints of thefirst column are near to the body, while the constraints of the secondcolumn are far from the body. Each figure contains the front view andthe side view.

the joint closer to the end-effector changes more easily. For multi-ple constraints problems, we introduce a weight matrix to indicatethe relationships. The effect of using weight matrix is the sameas the traditional method essentially. The elements of the Jacobianmatrix are zeros or small when the joints are not on the chain of thecorresponding constraints. In this way, the multiple constraints canbe satisfied simultaneously.

Given a set of geometry constraints, the target position of con-

straint k is denoted as uktgt . In the learning phase, we learn the in-verses of Jacobian matrices of each constraint separately, denoted

as Jk+. The weight matrix of constraint k is Wk. Then the synthesisprocess can be represented as:

pi = pi−1 +α ·K

∑k=1

Wk · Jk+c · (u

ktgt −∆u

ki−1) (20)

6 EXPERIMENTS

We have implemented a prototype pose synthesis system in C++ ontop of Windows XP and OpenGL.

Figure 1 demonstrates the principle of our method. Two exam-ples generated by computer are used to train a linkage model with 4bone segments and 12 DOFs. Two sets of poses with different con-straints are generated based on the examples separately. Note thatthe end-effector moves along the green axis in a small range in bothexamples, but our method is able to generate new configurationseven when the constraints are positioned under the red axis. Theseexperiments also demonstrate the stability of our method near sin-

103

Figure 4: Reference poses outputted by SOM. The poses on the corners are zoomed in from another viewpoint.

Figure 5: Comparison of outputs synthesized by our method and DLSmethod. Characters colored cyan (left column) show the synthesizedposes of our method while blue characters (right column) representthe output with DLS method. The positions of constraints in each roware the same.

gular configuration. The first pose of output column in each row isa result when we drag the end-effector moving near a singular con-figuration. Different examples cause different actions of the modelnear the same singularity. However, if we put the two examplestogether to SOM, our method can not produce a correct result be-cause that the assumption mentioned in section 4.2 is not satisfiedany more.

Other experiments are performed on a human model with 19bone segments and 60 DOFs. Figure 4 demonstrates the referenceposes learned from a bend-over motion using SOM without anypreprocessing. This example clip contains 2,235 frames sampledat 120Hz, which is obtained from CMU motion capture database.The output of the SOM is a 10×10 array. The learning rate changesfrom 1.0 to 0.02 in the learning phase. We train the map with 30,000steps.

The training output (Figure 4) shows the low-dimensional struc-

Figure 6: Comparison of convergence rate between our method andDLS method.

ture of the motion. We can find out that the position of the righthand varies from high to low along the X axis and the orientation ofthe body changes from right to left along the Y axis. These trendscan be viewed more clearly in the middle of Figure 1. Obviously,after the learning phase, the unorganized motion data became orga-nized.

Some synthesized poses with different constraints based on thebend-over motion are shown in Figure 3. Although some con-straints are far away from that of the original data, our method canstill produce plausible results. This demonstration shows that thelearned matrices can describe the characteristic of the original mo-tion effectively. Figure 5 shows some snapshots comparing the out-puts of our method with that of DLS method. The results demon-strate the superiority of our method in naturalness of results.

In order to demonstrate the efficiency of our method, we com-

104

Figure 7: A kick motion is modified based on the trajectory of the right foot. Left: The original motion and the trajectory of the right foot showedin yellow line. Right: The result synthesized by our method based on the modified trajectory showed in green line.

pare it with the DLS method. Three groups of trials with differentconstraints are executed using both methods. Figure 6 shows theposition error versus steps of iterations. From this figure, one cansee that our method requires more iterations than DLS. The maxiteration steps of our method is 90 while that of DLS method isonly 26 when errors reach to 10−5. However, our method is fasterthan DLS method because of avoiding the calculation of matrix in-verse. We execute these trials 1000 times and the average time foreach execution is presented in Table 1. These experiments are per-formed on a PC with Core Duo T2300E CPU and the memory of1G. The time spent of DLS method is about 20 times than that ofour method.

Table 1: Comparison of time spent.(Unit:ms)

Test 1 Test 2 Test 3 Average

DLS Method 5.313 5.609 5.188 5.370

Our Method 0.296 0.235 0.250 0.260

7 APPLICATIONS

7.1 Interactive Pose Editing

Interactive pose editing is one of the basic applications of our sys-tem. The experiments above show the application of positioningthe right hand of a bend-over motion. We have implemented it witha friendly interface, where we can use mouse to drag the right handdirectly (Figure 2).

7.2 Real-time Motion Modification

Because of the high efficiency of our method, it can be used to mod-ify motion data in real-time. Figure 7 is a demonstration of modi-fying a kick motion. A new kick motion is generated according tothe modified trajectory of the right foot. Note that there is only oneexample motion fed to SOM without any preprocessing. Having nostrict requirements on the quantity and quality of the examples is anadvantage of our method, but it will also have some influence on theresults. In this experiment, our method can not generate plausibleresult when the new trajectory is far away from that of the trainingexample. The reason for the problem is that the Jacobian is a localparameter, which is only valid for limited range of some configura-tion. Given more examples will improve the results and enlarge therange to position constraints.

Additionally, combining our method with techniques such as tra-jectory displacement and motion interpolation can produce moresmooth motion sequences, which is very useful to make various

interaction motions of virtual character according to the virtual en-vironment.

7.3 Pose Reconstruction from Image

Our method can also be used to reconstruct the pose from a 2D im-age of a person. Given a photograph associated with its projectionmatrix, we can reconstruct the pose through a few positions of end-effectors specified by the user interactively. In order to realize thisapplication, some modifications are needed.

We denote the projection matrix of the photograph as P, whichhas two rows and three columns. During the learning of SOM,the results of the forward kinematics need to be projected to a 2Dplane by P. Here, the inverse of Jacobian matrix learned shows therelationships between the positions of 2D plane and the joint angles.And in the synthesis phase, the objective function is modified tominimize the distance between the end effectors’ positions in thephoto and that of the projection of the pose.

8 DISCUSSION AND FUTURE WORK

The problem of synthesizing natural poses that satisfy a set of givengeometry constraints has been studied in this paper. A methodbased on a set of reference poses and a set of matrices learned fromexamples is presented. Our method can produce natural looking re-sults because of two aspects. Firstly, the reference poses with two-dimensional indices represents the space of natural poses. Theyprovide a good initializing configuration for a given problem. Sec-ondly, the learned matrices describe the characteristics of motionseffectively. Because there is no necessity to calculate the inverse ofJacobian matrix in the synthesis phase, the time cost by computinghas been reduced further, which can satisfy the need of time-criticalapplications. Experiments have proven the effectiveness and ro-bustness of this method.

However, there are some limitations in applying our approach.Firstly, compared with the work [3], our method has to specify theconstrained limbs at the learning phase. Although the relationshipfor each end-effector could be learned in advance, it requires vastcomputation and storage. Secondly, in solving multi-constraintsproblem, the weight matrix calculated based on a heuristic strategywill slow down the convergence rate. How to combine the con-straints, that is, calculating the weight matrix automatically fromexamples should be addressed in the future work.

For future work, there are some improvements and extensionsthat need to be considered. In the learning phase, a two-dimensionalstructure learned from examples might not exist in complicated mo-tions. If the dimension of SOM increases, the number of necessarysamples increases exponentially. However, the hierarchical SOMmight be a promising solution to this problem: first, the high-levelSOM can be constructed as a classifier which divides the space of

105

natural pose into different regions; then, a set of low-level SOMsare constructed as the low-dimensional structure of each region. Inthis way, we can enlarge the range of positioning constraints. Prob-ability SOM is also an alternative, in which the probability can beused to evaluate the result.

Inverse dynamics problem can also be considered on the basis ofour method. We can use the force or moment instead of joint anglesin the learning phase, and then the Jacobian matrices will describethe relationships between the forces and the configurations. Addi-tionally, more applications based on our method should be exploredin future work.

ACKNOWLEDGEMENTS

We would like to thank Yanyan Yuan, Rongrong Wang, the mem-bers of the CAS ICT virtual reality laboratory, and the anony-mous reviewers for their advice and assistance. This workwas supported in part by a grant from KBRPC-2002CB312104,NSFC-60403042, NSFC-60573162, NSFC-60533070, NSFC-60603082, CISTC-2005DFA11060, BSTPC-Z0004024040231,BNSFC-4051004, BNSFC-4062032. The data used in this projectwas obtained from mocap.cs.cmu.edu. The database was createdwith funding from NSF EIA-0196217.

REFERENCES

[1] E. B. Dam, M. Koch, and M. Lillholm. Quaternions, interpola-

tion and animation. Technical report, Department of Computer Sci-

ence,University of Copenhagen, July 17 1998.

[2] R. B. Daniel and I. Charles W. Wampler. On the inverse kinematics of

redundant manipulators. International Journal of Robotics Research,

7(2):3–21, 1988.

[3] K. Grochow, S. L. Martin, A. Hertzmann, and Z. Popović. Style-based

inverse kinematics. ACM Transactions on Graphics, 23(3):522–531,

2004.

[4] S. Haykin. Neural Networks: A Comprehensive Foundation Second

Edition. Prentice Hall, 1999.

[5] T. Kohonen. Self-organized formation of topologically correct feature

maps. Biological Cybernetics, 43:59–69, 1982.

[6] T. Komura, A. Kuroda, S. Kudoh, L. Tai Chiew, and Y. Shinagawa.

An inverse kinematics method for 3d figures with motion data. In

Computer Graphics International, pages 266–271, 2003.

[7] L. Kovar and M. Gleicher. Automated extraction and parameteriza-

tion of motions in large data sets. ACM Transactions on Graphics,

23(3):559–568, 2004.

[8] H. Ritter, T. Martinetz, and K. Schulten. Topology-conserving maps

for learning visuo-motor-coordination. Neural Networks, 2:159–168,

1989.

[9] H. Ritter, T. Martinetz, and K. Schulten. Neural Computation and

Self-Organizing Maps; An Introduction. Addison-Wesley Longman

Publishing Co., Inc., Boston, MA, USA, 1992.

[10] C. F. Rose, P.-P. J. Sloan, and M. F. Cohen. Artist-directed inverse-

kinematics using radial basis function interpolation. Computer Graph-

ics Forum, 20(3), 2001.

[11] R. W. Sumner, M. Zwicker, C. Gotsman, and J. Popović. Mesh-based

inverse kinematics. ACM Transactions on Graphics, 24(3):488–495,

2005.

[12] S. Tak and H. Ko. Example guided inverse kinematics. In Interna-

tional Conference on Computer Graphics and Imaging, pages 19–23,

2000.

[13] D. Tolani, A. Goswami, and N. I. Badler. Real-time inverse kinematics

techniques for anthropomorphic limbs. Graphical models, 62(5):353–

388, 2000.

[14] M. Tomohiko and K. Shigeru. Geostatistical motion interpolation.

ACM Transactions on Graphics, 24(3):1062–1070, 2005.

[15] C. W. Wampler. Manipulator inverse kinematic solutions based on

vector formulations and damped least squares methods. IEEE Trans-

actions on Systems, Man, and Cybernetics, SMC-16:93–101, 1986.

[16] C. Welman. Inverse kinematics and geometric constraints for articu-

lated figure manipulation. Master’s thesis, Simon Fraser University,

1989.

[17] D. E. Whitney. Resolved motion rate control of manipulators and

human prostheses. IEEE Transactions on Man-Machine Systems,

10(2):47–53, 1969.

[18] B. Widrow and M. Hoff. Adaptive switching circuits. Western Electric

Show and Convention Record, Part 4, pages 96–104, 1960.

[19] W. A. Wolovich and H. Elliot. A computational technique for inverse

kinematics. In Proceedings of 23rd Conference on Decision and Con-

trol, pages 1359–1363, Las Vegas, 1984.

[20] K. Yamane and Y. Nakamura. Natural motion animation through con-

straining and deconstraining at will. IEEE Transactions on Visualiza-

tion and Computer Graphics, 9(3):352–360, 2003.

[21] J. Zhao and N. I. Badler. Inverse kinematics positioning using nonlin-

ear programming for highly articulated figures. ACM Transactions on

Graphics, 13(4):313–336, 1994.

106

Pose Synthesis Using the Inverse of Jacobian Matrix...

Documents

Transcript of Pose Synthesis Using the Inverse of Jacobian Matrix...