Biped Robot Walking using Particle SwarmOptimization

Behzad Nikbin, Mohammad Reza Ranjbar, Behrooz Shafiee Sarjaz, and HamedShah-Hosseini

Abstract maintaining stability of a high degree of freedom biped robot is a complex prob-lem. One of the efficient methods to address this complexity is to utilize power of computerhardware in order to find optimized values. This paper describes a novel walking algorithmbased on inverse kinematics for a biped robot. The proposed algorithm uses the PSO (ParticleSwarm Optimization) algorithm in order to find optimized values for the five walking algo-rithm parameters. The proposed approach is tested on a Simulated NAO robot in the RoboCupsoccer simulation environment. Simulation results reveal that using the PSO algorithm withan efficient fitness function can significantly reduce learning time. Moreover, a considerablefast walking speed is achieved.

1 Introduction

Humanoid robot locomotion is an exciting challenge for many researchers. In the recent years,a significant body of research and development was performed in this area. The two main clas-sifications of biped robot walking are static and dynamic balance. In Static balance approach,the center of mass (COM) of the robot is kept within the support area of the feet. Similarly inDynamic balance methods, Zero Momentum Point (ZMP) should be kept inside the supportarea. Early biped walking of robots involved static walking with a very low walking speed[1,2]. Hereby, the projected point of COM onto the ground always falls within the supportingpolygon that is made by two feet. During the static walking, the robot can stop the walkingmotion any time without falling down. The disadvantage of static walking is that the motionis too slow and wide for shifting the COM. Researchers thus began to focus on dynamicwalking of biped robots [3-5]. This approach is much faster than static balance. However,if the inertial forces generated from the acceleration of the robot body are not suitably con-trolled, a biped robot easily falls down. In addition, during dynamic walking, a biped robotmay fall down from disturbances and cannot stop the walking motion suddenly. Hence, thenotion of ZMP (Zero Moment Point) was introduced in order to control inertial forces [6,7].The proposed algorithm in this paper, belongs to the dynamic balance methods, and utilizes

Behzad Nikbin, Mohammad Reza Ranjbar, Behrooz Shafiee Sarjaz, and Hamed Shah-HosseiniFaculty of Electrical and Computer Engineering, Shahid Beheshti University, G.C., Tehran, Iran e-mail:{behzad.nikbin, ranjbar.m1, shafiee01}@gmail.com, tasom2002@yahoo.com

1

2 Behzad Nikbin, Mohammad Reza Ranjbar, Behrooz Shafiee Sarjaz, and Hamed Shah-Hosseini

ZMP criteria in order to maintain the stability of a biped robot. In addition, unlike most ofthe previous works, which use fixed classical complex equation to find the best values fortheir algorithms, we have used the PSO algorithm in order to maximize the efficiency of ouralgorithm. The outcomes of our simulations indicate that using the PSO to adjust walkingparameters leads to a considerable fast walking speed with a negligible learning time. Theproceeding parts of this paper are classified as follows: in section 2, the proposed walkingalgorithm is presented.In section 3, optimization of walking parameters is discussed. Experi-mental results are demonstrated in section 4. Finally, in section 5 the conclusion is presented.

2 Proposed Walking Algorithm

Biped-Walking is a periodic process with its period termed half-gait or half-loop. Therefore,through building one half-loop, the walking can be constructed as a result of repetition of onehalf-loop after another. A walking loop or gait is started by a half-loop whether by left orright leg, and is finished by another half-loop by the opposite leg.

Hal f Loop1right = Hal f Loop2le f t , Hal f Loop1le f t = Hal f Loop2right

As a result of this symmetry, the entire walk loop can be easily constructed with a Half-Loop for each leg. In the following, first, HalfLoop1right and then, HalfLoop1left are dis-cussed. We imagine the world coordinate system as depicted in Fig.1.

Fig. 1: World Coordinates

The noticeable point in Fig.1 is that all the movements across the Y-axis are ignored. Thisrestriction leads to using five distinct parameters and merely three constants to model thewalking process of NAO. The second assumption is that at the beginning of the HalfLoop1,the right leg is in front of the left leg. Finally, after defining all the parameters, the Parti-cle Swarm Optimization (PSO) algorithm is utilized to find best values for them. Moreover,constants are determined based on the physical specification of the robot. In the proceedingsections, calculation of each parameter is explained in detail.

Parameter1: HipHeight

HipHeight is the height of the hip joint during the walk process. The hips vertical movement(across the Z-axis) is ignored. Thus, hips height will be fixed during a specific walk pattern;conversely, it can vary in different walk patterns. HipHeight is illustrated in Fig.2.

Biped Robot Walking using Particle Swarm Optimization 3

Parameter2: HipOffset

HipOffset is the distance along X-axis between left hip and left ankle.

HipO f f set = X0 HipLe f t X0 AnkleLe f t (1)

X0HipLeft and X0AnkleLeft are the X-values of the positions of the left hip and left ankle at cyclezero, respectively. HipOffset is demonstrated in Fig.2.

Parameter3: MaxAnkleHeight

MaxAnkleHeight indicates the height of ankle joint at the highest point. We assume this pointin the middle of start and end points of HalfLoop1.

MaxAnkleHeight = Max(Zi ankle) = height(X0 ankle +Xn1 ankle

2) (2)

MaxAnkleHeight is shown in Fig.2.

Fig. 2: HipHeight, HipOffset, and MaxAnkleHeight

Parameter4: NumOfSteps

NumOfSteps defines the number of cycles (hardwares clock cycle) necessary to complete aHalf-Loop.

Parameter5: GaitLength

This parameter demonstrates the distance traversed during a Half-Loop. GaitLength is shownin Fig.3.

GaitLength = Xn1 AnkleLe f t X0 AnkleLe f t (3)

4 Behzad Nikbin, Mohammad Reza Ranjbar, Behrooz Shafiee Sarjaz, and Hamed Shah-Hosseini

Fig. 3: GaitLength and NumberOfSteps

Constants

All the constants in the walking process are determined according to the robot physical spec-ification, and will be fixed during any walk pattern.

Table 1: Constants for NAO

Constant ValueThigh-Length 14cmShank-Length 11cm

Time of 1 Step(clock time of the robot) 0.02s

Fig. 4: Constant values of walking algorithm

3 Half-Loop

In this section the relevant equations for a Half-Loop are presented. First, calculation of eachjoint position during HalfLoop1 is explained. Fig.5 demonstrates the HalfLoop1 for the leftleg.

Biped Robot Walking using Particle Swarm Optimization 5

Fig. 5: Positions of left leg joints during HalfLoop1

3.1 Left Leg

All the variables for the left hipand ankle are defined in the Appendix. A. In addition, the kneeposition is calculated by intersection of Ci1 and Ci2 as illustrated in Fig.6. The intersection ofthese two circles generates two result points. The point with greater X-value will be answerbecause the knee angle of a human or robot is always positive.Ci1(x) = a circle centered to LeftHipi and radius of Thigh-LengthCi2(x) = a circle centered to LeftAnklei and radius of Shank-Length

Fig. 6: Calculating knee position

3.2 Right Leg

Fig.7 demonstrates the HalfLoop1 for the right leg. The position of right hip is always equalto that of the left hip because both hips are connected to each other and move along eachother.

(X ,Z)i hip right = (X ,Z)i hip le f t 0 i n1 (4)

The position of right ankle is fixed during HalfLoop1 and its position can be easily calculated.

Xi ankle right =GaitLength

2HipO f f set (5)

Zi ankle right = 0 0 i n1 (6)

The knee position of right leg is calculated similar to left knee.

6 Behzad Nikbin, Mohammad Reza Ranjbar, Behrooz Shafiee Sarjaz, and Hamed Shah-Hosseini

Fig. 7: Positions of right leg joints during HalfLoop1

3.3 Joints Angle Calculation

After calculating all joints positions, each joint angle can be simply calculated during eachcycle of walk loop. Fig.8 demonstrates the correspondent angles of hip, knee, and ankle joints.

Fig. 8: Calculating angles of hip, knee, and ankle

In Fig.8, i demonstrates the angle of hip joint, i indicates the angle of knee joint, and irepresents the angle of ankle joint at cycle i. These parameters can be easily calculated withrespect to the line slopes.

i =2 arctan(

Zi hipZi kneeXi hipXi knee

) (7)

i = arctan(Zi kneeZi ankleXi kneeXi ankle

) 2

(8)

i =i i (9)

To this point, all of the joints angles at each cycle are calculated; consequently, each jointangular velocity at each cycle can be determined as well. These angular velocities are thefinal output of our algorithm and are sent directly to the robot motors. If the clock cycle timeof robot be T; then, the angular velocities can be calculated as follows:

i, j =i, ji1, j

T(10)

Biped Robot Walking using Particle Swarm Optimization 7

where i,j stands for the angle of joint j, and i,j shows angular velocity of joint j at cycle i.

4 Optimization of Parameters

In order to find the best values for the walking parameters, we used the PSO algorithm.Each iteration must be run to complete the experiment. At each run, a robot in a simulatedworld is run, and the program waits for the walk results, which is a time consuming process.Therefore, experimenting with a great number of populations is not practical. In general,there are two main solutions to reduce the convergence time: first, to use efficient initialvalues; second, to reduce the number of populations. In most of the optimization algorithms,an efficient initial value for parameters plays a crucial role in the reduction of convergencetime. In order to easily manipulate these parameters, we developed a visual toolkit as shownin Fig.9. This toolkit helped us reach proper initial values for the PSO algorithm. As a result,the convergence time significantly was reduced.

Fig. 9: A toolkit for manipulating parameters and running the PSO algorithm

The PSO algorithm was selected because it can solve problems with less population densityand less iteration in comparison with similar algorithms like Genetic algorithm. To make PSOconvergence faster, we limit the parameters as presented in Table.2:

Table 2: Parameters limitation

Joint name Lower bound Upper boundHipHeight 11cm 25cmHipOffset 1cm 10cm

MaxAnkleHeight 1mm 4cmSteps 3 14

GaitLength 1cm 30cm

8 Behzad Nikbin, Mohammad Reza Ranjbar, Behrooz Shafiee Sarjaz, and Hamed Shah-Hosseini

4.1 Fitness Function

Another critical factor in optimization algorithms is fitness function. In order to reach aneffective fitness function, the support area and zero momentum point were utilized. By meansof these concepts, the robots stability can be measured effectively.

4.1.1 Support Area

The support area is the area between the two legs [8]. When both legs touch the ground,support area is the area between these two foots, and when just one foot touches the ground,support area is the area under that foot. The areas in both cases are demonstrated in Fig.10.

Fig. 10: (a) support area in double support phase (b) support area in single support phase

4.1.2 Zero Momentum Point

Zero Momentum Point (ZMP) is originally a point at which sum of all momentums around itis zero [9]. If the projection of this point on the ground is inside the support area, the robotwill be stable; otherwise the robot may fall down. ZMP formulas are presented in appendixB.

4.1.3 Fitness function

In order to achieve the minimum deviation of ZMP and center of support area, the followingfunction was used as the fitness function:

n

i=1|Zi

Ci |2 (11)

Where Zi is ZMP, Ci stands for the center of support area at cycle i, and n shows the numberof simulation cycles. On the other hand, it is clear that the best case is when the robot doesnot move (fitness=0). Therefore, another fitness function to combine with this function isrequired.

n1

i=1|Ci

Ci+1|2 (12)

Biped Robot Walking using Particle Swarm Optimization 9

With maximizing this function, the robot will traverse a longer path in a specific time (ncycles) period. In summary, the final formula which tries to minimize Eq. 11 and maximizethe Eq. 12 becomes as follows:

f = wn

i=1|Zi

Ci |2 +

1wn1i=1 |

Ci

Ci+1|2

(13)

In this equation w can be manipulated to choose between stability and speed.

5 Experimental Results

In order to experiment the proposed algorithm in action, we used the RoboCup Soccer Sim-ulation Server, Rcssserver3d. Rcssserver3d implements a simple internal event model thatimmediately executes every action received from an agent. A screenshot of Rcssserver3d andthe developed toolkit are both presented in Fig.11. The platform used to run the proposed

Fig. 11: Rcssserver3d and the developed toolkit to find the initial parameters

algorithm is a Pentium IV core i7 2.4GHz with 8GB of physical memory. Moreover, the ex-periments are executed in Ubuntu Linux 10.04 with OpenJDK 6.In order to find the appropriate value for W parameter of the fitness function in Eq. 13 andproper initial values for PSO algorithm, we used our visual toolkit before running PSO algo-rithm. We tried different values for the W with initial values for PSO algorithm. We observedthat by choosing W=0.6 the robot can achieve a relatively fast and stable walking, and thisvalue was used during PSO runs. Since the PSO is a stochastic algorithm, we performed 10different runs. Table 3 shows the PSO algorithms parameters. The JSwarm library [10] wasused to implement PSO algorithm. JSwarm is an open source implementation of PSO algo-rithm in java. After spending nearly 130 minutes in each run, the deviations in the output of

Table 3: PSO algorithm parameters

No. of Iterations No. of Particles Inertia Particle Increment Global Increment30 20 0.98 0.01 0.01

10 Behzad Nikbin, Mohammad Reza Ranjbar, Behrooz Shafiee Sarjaz, and Hamed Shah-Hosseini

fitness function were insignificant. Therefore, the algorithm was ceased. This relatively shortconvergence time was the result of using the PSO algorithm and an accurate fitness function.Fig.12 shows the average fitness value after each iteration for 10 runs, and the best fitnessvalue in 10 runs. In addition, the optimized values of parameters are presented in Table4.

Fig. 12: Fitness values in 30 iterations

Table 4: Optimized parameters (best particle of PSO)

GaitLength No. of steps MaxAnkleHeight HipOffset HipHeight19.3cm 8 3.0cm 8.3cm 17.9cm

The NAO robot of the best particle could traverse 6.2m during 10 seconds, which is equalto 0.62m/s on average. In a similar work, which was done during four hours of learning ina simulated environment with a genetic algorithm [11], an average speed of 0.44m/s wasachieved. We believe that this increase in the average speed and significant time reductionis the result of using an accurate fitness function and the PSO algorithm both. In [12] theauthors use PSO algorithm to find optimized value for their TFS (Truncated Fourier Series)based gait generation method. Using PSO with the TFS, they could achieve 0.57m/s; however,they claim that by arm swing and their increasing speed technique they could improve walk-ing speed to 0.77m/s. On the other hand, they did not report about the stability of Robot andthe mechanism which measure their estimation of robots stability. We believe that throughmanipulating the W parameter higher speed can achieve, but there is a tradeoff between sta-bility and speed. In Fig.13, joints positions are shown, and Fig.14 presents the correspondentgenerated angle of each joint.

6 Conclusion and Future works

In this paper, we presented a dynamic balance walking algorithm using the PSO algorithm.First, an inverse kinematic approach was used to estimate the position of each robot joint

Biped Robot Walking using Particle Swarm Optimization 11

Fig. 13: Positions of left and right leg during the walk

Fig. 14: Angles of each joint

and the correspondent angles. Then, the PSO algorithm was utilized to find the best valuesfor parameters which generate the least deviation of robot ZMP from the robot support area.We have two main goals to follow as the future works. First, the proposed algorithm shouldbe altered to a dynamic approach which can maintain the robot stability more effectively. Inaddition, the proposed algorithm should consider the non-straight trajectories; therefore, anOmni-directional approach will be considered.

References

1. W. T. Miller III: Real-time neural network control of a biped walking robot. IEEE Control SystemsMagazine. 14(1): 41-48 (1994)

2. C. L. Shih: Ascending and descending stairs for a biped robot. IEEE Transactions on Systems, Man,andCybernetics. 29(3): 255-268 (1999)

3. J. Yamaguchi, E. Soga, S. Inoue, A. Takanishi: Development of a bipedal humanoid robot controlmethod of whole body cooperative dynamic biped walking. Paper presented at the IEEE internationalconference on robotics and automation, Detroit, Michigan, 10-15 May 1999

4. K. Hirai, M. Hirose, T. Takenaka: The development of Honda humanoid robot. Paper presented at theIEEE international conference on robotics and automation, Leuven, Belgium, 16-20 May 1998

5. S. Kajita, F. Kanehiro, K. Kaneko, K. Fujiwara, K. Harada, K. Yokoi, H. Hirukawa: Biped walkingpattern generation by using preview control of Zero-Moment Point. Paper presented at the IEEE inter-national conference on robotics and automation, Taipei, Taiwan, 14-19 Sep. 2003

6. M.Vukobratovic, B. Borovac, D. Surla, D. Stokic: Biped locomotion. Springer-Verlag (1990)7. K. Nishiwaki, S. Kagami, Y. Kuniyoshi, M. Inaba, H. Inoue: Online generation of humanoid walking

motion based on fast generation method of motion pattern that follows desired ZMP. Paper presented

12 Behzad Nikbin, Mohammad Reza Ranjbar, Behrooz Shafiee Sarjaz, and Hamed Shah-Hosseini

at the IEEE/RSJ international conference on intelligent robots and systems, Lausanne, Switzerland, 30Sep. 5 Oct. 2002

8. J. Mrozowski, J. Awrejcewicz, P. Bamberski: Analysis of stability of the human gait, Journal of Theo-retical and Applied Mechanics, vol. 45, no. 1, pp. 91-98, 2007.

9. L. Yang, C. Chew, T. Zielinska, A. Poo:A uniform biped gait generator with offline optimization andonline adjustable parameters, Robotica (2007) volume 25, pp. 549565, March 19, 2007

10. JSwarm, http://jswarm-pso.sourceforge.net, Last Access Oct. 2011.11. N. Shafii, S. Aslani, S.Mohammad, H.S.Javadi,V. Azizi, O. M. Nezami: Robust Humanoid walking using

Truncated Fourier Series gait generator,Iran Open Symposium, April. 2009.12. N. Shafii, L.P. Reis, N. Lau: Biped Walking using Coronal and Sagittal Movements based on Truncated

Fourier Series, RoboCup-2010: Robot Soccer World Cup XIII, Springer LNAI / LNCS, Vol. 6556, pp.324-335, Berlin, 2011.

Appendix A

Left hip

X0 hip le f t = HipO f f set

Xi hip le f t = X0hiple f t +iGaitLength

2 (n1)Zi hip le f t = HipHeight 0 i n1

Left ankle

X0 ankle le f t = 0Xn1 ankle le f t = X0 ankle le f t +GaitLength

Z0 ankle le f t = 0 Zn1 ankle le f t = 0

Xi ankle le f t = X0 ankle le f t +iGaitLength

n1Zi ankle le f t =C(Xi ankle le f t) 1 i n2

C(X): The circle which passes through the (X0 ankle left, Z0 ankle left), (Xn-1 ankle left, Zn-1 ankle left),and ( X0 ankle left+Xn-1 ankle left2 , MaxAnkleHeight) points.

Appendix B

xzmp =ni=1 mi(Zi +g)xini=1 mixizini=1 mi Iiyiy

ni=1 mi(Zi +g)

yzmp =ni=1 mi(Zi +g)yini=1 miyizini=1 mi Iixix

ni=1 mi(Zi +g)zzmp = 0