Hybrid Control and Motion Planning of Dynamical Legged Locomotion

HYBRID CONTROL ANDMOTION PLANNINGOF DYNAMICAL LEGGEDLOCOMOTION

IEEE Press445 Hoes Lane

Piscataway, NJ 08854

IEEE Press Editorial BoardJohn B. Anderson, Editor in Chief

R. Abhari G. W. Arnold F. CanaveroD. Goldgof B-M. Haemmerli D. JacobsonM. Lanzerotti O. P. Malik S. NahavandiT. Samad G. Zobrist

Kenneth Moore, Director of IEEE Book and Information Services (BIS)

HYBRID CONTROL ANDMOTION PLANNINGOF DYNAMICAL LEGGEDLOCOMOTION

Nasser SadatiGuy A. DumontKaveh Akbari HamedWilliam A. Gruver

Cover Image: Courtesy of the authors

Copyright © 2012 by the Institute of Electrical and Electronics Engineers, Inc.

Published by John Wiley & Sons, Inc., Hoboken, New JerseyPublished simultaneously in Canada

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form orby any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except aspermitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the priorwritten permission of the Publisher, or authorization through payment of the appropriate per-copy fee tothe Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax(978) 750-4470, or on the web at www.copyright.com. Requests to the Publisher for permission shouldbe addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley.com/go/permission.

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts inpreparing this book, they make no representations or warranties with respect to the accuracy orcompleteness of the contents of this book and specifically disclaim any implied warranties ofmerchantability or fitness for a particular purpose. No warranty may be created or extended by salesrepresentatives or written sales materials. The advice and strategies contained herein may not be suitablefor your situation. You should consult with a professional where appropriate. Neither the publisher norauthor shall be liable for any loss of profit or any other commercial damages, including but not limited tospecial, incidental, consequential, or other damages.

For general information on our other products and services or for technical support, please contact ourCustomer Care Department within the United States at (800) 762-2974, outside the United States at (317)572-3993 or fax (317) 572-4002.

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print maynot be available in electronic formats. For more information about Wiley products, visit our web site atwww.wiley.com.

Library of Congress Cataloging-in-Publication Data:

Hybrid control and motion planning of dynamical legged locomotion / Nasser Sadati. . . [et al.].p. cm.

ISBN 978-1-118-31707-5 (hardback)1. Mobile robots. 2. Robots–Motion. 3. Walking. I. Sadati, Nasser.TJ211.415.H93 2012629.8′932–dc23

2012002035

ISBN: 978-1-118-31707-5

Printed in the United States of America

10 9 8 7 6 5 4 3 2 1

CONTENTS

Preface ix

1. Introduction 1

1.1 Objectives of Legged Locomotion and Challenges in ControllingDynamic Walking and Running 1

1.2 Literature Overview 41.2.1 Tracking of Time Trajectories 41.2.2 Poincare Return Map and Hybrid Zero Dynamics 5

1.3 The Objective of the Book 71.3.1 Hybrid Zero Dynamics in Walking with Double Support

Phase 71.3.2 Hybrid Zero Dynamics in Running with an Online Motion

Planning Algorithm 81.3.3 Online Motion Planning Algorithms for Flight Phases

of Running 91.3.4 Hybrid Zero Dynamics in 3D Running 101.3.5 Hybrid Zero Dynamics in Walking with Passive Knees 111.3.6 Hybrid Zero Dynamics with Continuous-Time Update Laws 12

2. Preliminaries in Hybrid Systems 13

2.1 Basic Definitions 132.2 Poincare Return Map for Hybrid Systems 162.3 Low-Dimensional Stability Analysis 232.4 Stabilization Problem 28

3. Asymptotic Stabilization of Periodic Orbits for Walking with DoubleSupport Phase 35

3.1 Introduction 353.2 Mechanical Model of a Biped Walker 37

3.2.1 The Biped Robot 373.2.2 Dynamics of the Flight Phase 373.2.3 Dynamics of the Single Support Phase 393.2.4 Dynamics of the Double Support Phase 403.2.5 Impact Model 43

v

vi CONTENTS

3.2.6 Transition from the Double Support Phase to the SingleSupport Phase 45

3.2.7 Hybrid Model of Walking 453.3 Control Laws for the Single and Double Support Phases 46

3.3.1 Single Support Phase Control Law 463.3.2 Double Support Phase Control Law 49

3.4 Hybrid Zero Dynamics (HZD) 543.4.1 Analysis of HZD in the Single Support Phase 553.4.2 Analysis of HZD in the Double Support Phase 573.4.3 Restricted Poincare Return Map 58

3.5 Design of an HZD Containing a Prespecified Periodic Solution 603.5.1 Design of the Output Functions 603.5.2 Design of u1d and u2d 62

3.6 Stabilization of the Periodic Orbit 673.7 Motion Planning Algorithm 71

3.7.1 Motion Planning Algorithm for the Single Support Phase 723.7.2 Motion Planning Algorithm for the Double Support Phase 733.7.3 Constructing a Period-One Orbit for the Open-Loop Hybrid

Model of Walking 763.8 Numerical Example for the Motion Planning Algorithm 773.9 Simulation Results of the Closed-Loop Hybrid System 82

3.9.1 Effect of Double Support Phase on Angular MomentumTransfer and Stabilization 82

3.9.2 Effect of Event-Based Update Laws on Momentum Transferand Stabilization 92

4. Asymptotic Stabilization of Periodic Orbits for Planar MonopedalRunning 95

4.1 Introduction 954.2 Mechanical Model of a Monopedal Runner 97

4.2.1 The Monopedal Runner 974.2.2 Dynamics of the Flight Phase 974.2.3 Dynamics of the Stance Phase 984.2.4 Open-Loop Hybrid Model of Running 99

4.3 Reconfiguration Algorithm for the Flight Phase 994.3.1 Determination of the Reachable Set 103

4.4 Control Laws for Stance and Flight Phases 1204.4.1 Stance Phase Control Law 1214.4.2 Flight Phase Control Law 1224.4.3 Event-Based Update Law 124

4.5 Hybrid Zero Dynamics and Stabilization 1254.6 Numerical Results 127

CONTENTS vii

5. Online Generation of Joint Motions During Flight Phases of PlanarRunning 137

5.1 Introduction 1375.2 Mechanical Model of a Planar Open Kinematic Chain 1385.3 Motion Planning Algorithm to Generate Continuous Joint Motions 140

5.3.1 Determining the Reachable Set from the Origin 1435.3.2 Motion Planning Algorithm 150

5.4 Motion Planning Algorithm to Generate ContinuouslyDifferentiable Joint Motions 152

6. Stabilization of Periodic Orbits for 3D Monopedal Running 159

6.1 Introduction 1596.2 Open-Loop Hybrid Model of a 3D Running 160

6.2.1 Dynamics of the Flight Phase 1626.2.2 Dynamics of the Stance Phase 1636.2.3 Transition Maps 1646.2.4 Hybrid Model 166

6.3 Design of a Period-One Solution for the Open-LoopModel of Running 167

6.4 Numerical Example 1726.5 Within-Stride Controllers 175

6.5.1 Stance Phase Control Law 1756.5.2 Flight Phase Control Law 178

6.6 Event-Based Update Laws for Hybrid Invariance 1816.6.1 Takeoff Update Laws 1846.6.2 Impact Update Laws 185

6.7 Stabilization Problem 1866.8 Simulation Results 189

7. Stabilization of Periodic Orbits for Walking with Passive Knees 193

7.1 Introduction 1937.2 Open-Loop Model of Walking 194

7.2.1 Mechanical Model of the Planar Bipedal Robot 1947.2.2 Dynamics of the Single Support Phase 1957.2.3 Impact Map 1957.2.4 Open-Loop Impulsive Model of Walking 196

7.3 Motion Planning Algorithm 1977.4 Numerical Example 2007.5 Continuous-Times Controllers 2027.6 Event-Based Controllers 209

7.6.1 Hybrid Invariance 2097.6.2 Continuity of the Continuous-Time Controllers During

the Within-Stride Transitions 212

viii CONTENTS

7.7 Stabilization Problem 2137.8 Simulation of the Closed-Loop Hybrid System 217

8. Continuous-Time Update Laws During Continuous Phasesof Locomotion 221

8.1 Introduction 2218.2 Invariance of the Exponential Stability Behavior for a Class

of Impulsive Systems 2228.3 Outline of the Proof of Theorem 8.1 2248.4 Application to Legged Locomotion 227

A. Proofs Associated with Chapter 3 229

A.1 Proof of Lemma 3.3 229A.2 Proof of Lemma 3.4 230A.3 Proof of Lemma 3.7 230

B. Proofs Associated with Chapter 4 233

B.1 Proof of Lemma 4.2 233B.2 Proof of Theorem 4.2 234

C. Proofs Associated with Chapter 6 237

C.1 Proof of Lemma 6.1 237C.2 Proof of Lemma 6.2 238C.3 Invertibility of the Stance Phase Decoupling Matrix on the

Periodic Orbit 240

Bibliography 241

Index 249

PREFACE

During the last three decades, enormous advances have occurred in robot control ofdynamical legged locomotion. The desire to study legged locomotion has been mo-tivated by the desire to assist people with disabilities to walk and replace humansin hazardous environments. The control of dynamical locomotion is complicated by(i) limb coordination, (ii) hybrid nature of walking and running due to presence ofimpact and takeoff, (iii) underactuation, (iv) overactuation, (v) inability to apply theZero Moment Point criterion during dynamic walking and running, (vi) lack of algo-rithms to achieve feasible period-one orbits, and (vii) conservation of angular momen-tum about the robot’s center of mass during flight phases of running. New applicationsof complex legged robots also require the use of system engineering approaches toresolve these issues that are beyond any single traditional engineering discipline. Asnew problems in legged locomotion require multidisciplinary methodologies, there isa critical need for a comprehensive book covering motion planning algorithms and hy-brid control. This book fills that gap for researchers, professionals, and students whoare versed in robotics and control theory. This book serves as a reference and essentialguide for researchers and engineers to perform future research and development inorder to advance various topics of hybrid control of legged locomotion. This volumealso provides a comprehensive overview of hybrid models describing the evolutionof planar and 3D legged robots during dynamical legged locomotion, and hybridcontrol schemes to asymptotically stabilize periodic orbits for the resulting closed-loop systems. The major topics of this book include hybrid systems, systems withimpulse effects, offline and online motion planning algorithms to generate periodicwalking and running motions and two-level control schemes including within-stridefeedback laws to reduce the dimension of hybrid systems, continuous-time updatelaws for online minimization of a general cost function, and event-based update lawsto asymptotically stabilize the generated desired orbits. This volume can be viewedas a handbook in this important field, as well as a reference book for researchers andpracticing engineers.

Chapter 2 introduces basic ideas, definitions, and results from the literature ofhybrid systems. Chapter 3 shows how to design a continuous-time-invariant feedbacklaw that asymptotically stabilizes a feasible periodic trajectory using an extension ofhybrid zero dynamics for a hybrid model of walking. The main objective is to developa continuous-time-invariant control law for walking of a planar biped robot duringthe double support phase.

A number of control problems for reconfiguration of a planar multilink robotduring flight phases have been considered in the literature. However, these methods

ix

x PREFACE

cannot be employed online to solve the reconfiguration problem for monopedal run-ning. For this reason, Chapters 4 and 5 present online reconfiguration algorithms thatprovide a solution to this latter problem for given flight times and angular momenta.The algorithms proposed in this book are expressed using the methodology of reach-ability and optimal control for time-varying linear systems with input and stateconstraints. In addition, a two-level control scheme based on the online reconfigura-tion algorithms and hybrid zero dynamics is proposed in Chapter 4 to asymptoticallystabilize a desired period-one orbit for a hybrid model describing running by planarmonopedal robots. Chapter 6 presents a time-invariant control scheme to asymptoti-cally stabilize a desired feasible periodic orbit for running by a 3D legged robot alonga straight line. A systematic algorithm to generate desired feasible periodic orbits for3D running is also presented. Chapter 6 extends the results of Chapters 4 and 5 to 3Drunning robots.

In order to reduce the number of actuated joints for walking on a flat surface andrestore walking motion for persons with disabilities, a motion planning algorithm isdeveloped in Chapter 7 for walking with passive knees. In addition, a time-invarianttwo-level control scheme is presented to stabilize the desired motions that are gen-erated. In Chapter 8, an analytical approach for designing a class of continuous-timeupdate laws to update the parameters of stabilizing controllers during continuousphases is proposed such that (i) a general cost function, such as the energy of thecontrol input over single support, can be minimized online, and (ii) the exponentialstability behavior of the limit cycle for the closed-loop system is not affected.

Book Webpage: Supplemental materials are available at the following URL:

http://booksupport.wiley.com.

This webpage includes MATLAB codes for motion planning algorithms and hybridcontrol schemes of several legged robots studied in this book, an erratum, and a linkto submit errors found in this book.

Nasser SadatiGuy A. Dumont

Kaveh Akbari HamedWilliam A. Gruver

November 14, 2011

Figure 2.1 Geometric description of Theorem 2.1. The Poincare return map of theautonomous hybrid system �(X1,X2,S2

1,S12, �

21, �

12, f1, f2), P : S1

2 → S12, is also the

Poincare return map for the autonomous system with impulse effects �ie(X2,S, �, f2), whereS := S1

2 and �(x2) := �21 ◦ P1(x2).

Figure 2.2 (See text for full caption.)

Hybrid Control and Motion Planning of Dynamical Legged Locomotion, Nasser Sadati, Guy A. Dumont,Kaveh Akbari Hamed, and William A. Gruver.© 2012 by the Institute of Electrical and Electronics Engineers, Inc. Published 2012 by John Wiley & Sons, Inc.

Figure 2.3 Geometric description of the restricted Poincare return map ρ : S12 ∩ Z2 →

S12 ∩ Z2. By hypotheses H2–H5 and the construction of �|Z, ρ(z2) = P |Z(z2), where P |Z

is the restriction of the Poincare return map of the full-dimensional hybrid system � to Z. Byapplying Theorem 2.1, it follows that ρ is also the Poincare return map for the reduced-ordersystem with impulse effects �ie|Z2 (Z2,S ∩ Z2, δ, f2|Z2 ), where δ(z2) := δ2

1 ◦ ρ1(z2).

2( )pq

2( )pull q

d tqd tq

d tq

Figure 3.3 Geometrical description of the motion planning algorithm during double support.In equation (3.66), it is assumed that rank( ∂p2

∂q(qd(t))) = 2 and qd(t) can be expressed as qd(t) =

q‖d(t) + q⊥

d (t).

0 0.5 1 1.5 2 2.50.2

0.4

0.6

0.8

1

(s)

Sta

nce

knee

(ra

d)

0 0.5 1 1.5 2 2.5

2.6

2.8

3

(s)

Sta

nce

hip

(rad

)

0 0.5 1 1.5 2 2.50.2

0.3

0.4

0.5

0.6

(s)

Sw

ing

knee

(ra

d)

0 0.5 1 1.5 2 2.5

2.6

2.8

3

(s)

Sw

ing

hip

(rad

)

0 0.5 1 1.5 2 2.51.3

1.4

1.5

1.6

(s)

Sor

so (

rad)

DS

SS

Figure 3.6 Angular positions of the knee, hip, and torso joints during two consecutive stepsof the optimal motion. The discontinuities are due to the coordinate relabling for swapping therole of the legs.

0 0.5 1 1.5 2 2.5−2

−1

0

1

2

(s)

Sta

nce

knee

(ra

d/s)

0 0.5 1 1.5 2 2.5

−1

0

1

(s)

Sta

nce

hip

(rad

/s)

0 0.5 1 1.5 2 2.5−2

−1

0

1

2

(s)

Sw

ing

knee

(ra

d/s)

0 0.5 1 1.5 2 2.5−1.5

−1

−0.5

0

0.5

1

(s)

Sw

ing

hip

(rad

/s)

0 0.5 1 1.5 2 2.5−1

−0.5

0

0.5

1

(s)

Tor

so (

rad/

s)

DS

SS

Figure 3.7 Angular velocities of the knee, hip and torso joints during two consecutive stepsof the optimal motion. The discontinuities are due to the coordinate relabling for swapping therole of the legs.

0 0.5 1 1.5 2 2.5−90

−80

−70

−60

−50

−40

−30

−20

−10

(s)

Sta

nce

knee

(N

m)

0 0.5 1 1.5 2 2.5

−20

0

20

40

60

(s)

Sta

nce

hip

(Nm

)

0 0.5 1 1.5 2 2.5−15

−10

−5

0

5

10

(s)

Sw

ing

knee

(N

m)

0 0.5 1 1.5 2 2.5−50

−40

−30

−20

−10

0

10

20

30

(s)

Sw

ing

hip

(Nm

)

DS

SS


0 0.5 1 1.5 2 2.5−20

−10

0

10

20

30

40

(s) Sta

nce

leg

end

horiz

onta

l for

ce (

N)

0 0.5 1 1.5 2 2.50

100

200

300

400

500

600

(s)

Sta

nce

leg

end

vert

ical

forc

e (N

)

0 0.5 1 1.5 2 2.5−0.2

−0.15

−0.1

−0.05

0

0.05

0.1

0.15

(s)

Sw

ing

leg

end

horiz

onta

l for

ce (

N)

0 0.5 1 1.5 2 2.50

50

100

150

200

250

(s)

Sw

ing

leg

end

vert

ical

forc

e (N

)

DS

SS


0 0.5 1 1.5−0.2

−0.1

0

0.1

0.2 H

oriz

onta

l pos

ition

of C

OM

(m

)

Time (s) 0 0.5 1 1.5

0.64

0.66

0.68

0.7

0.72

0.74

Ver

tical

pos

ition

of C

OM

(m

)

Time (s)

0 0.5 1 1.5−0.2

0

0.2

0.4

0.6

Hor

izon

tal v

eloc

ity o

f CO

M (

m/s

)

Time (s) 0 0.5 1 1.5

−0.3

−0.2

−0.1

0

0.1

Ver

tical

vel

ocity

of C

OM

(m

/s)

Time (s)

−0.1 0 0.1 0.20.64

0.66

0.68

0.7

0.72

0.74

Ver

tical

pos

ition

of C

OM

(m

)

Horizontal position of COM (m) −0.1 0 0.1 0.2

−0.3

−0.2

−0.1

0

0.1

Ver

tical

vel

ocity

of C

OM

(m

/s)

Horizontal position of COM (m)

DS

SS

Figure 3.11 Plot of the vertical height and velocity of the COM versus time and the horizontalposition of the COM on O. At the impact, the velocity of the COM is not pointed downward.

0 2 4 6

0.4

0.6

0.8

(s)

Sta

nce

knee

(ra

d)

0 2 4 6

2.6

2.8

3

(s)

Sta

nce

hip

(rad

)

0 2 4 60.2

0.3

0.4

0.5

0.6

(s)

Sw

ing

knee

(ra

d)

0 2 4 6

2.6

2.8

3

(s)

Sw

ing

hip

(rad

)

0 2 4 61.3

1.4

1.5

1.6

(s)

Tor

so (

rad)

DS SS

Figure 3.12 Configuration variables during five steps of the closed-loop simulation. Discon-tinuities in the graphs are due to coordinate relabling.

0 2 4 6−2

−1

0

1

2

(s)

Sta

nce

knee

(ra

d/s)

0 2 4 6

−1

0

1

(s)

Sta

nce

hip

(rad

/s)

0 2 4 6−2

−1

0

1

2

(s)

Sw

ing

knee

(ra

d/s)

0 2 4 6−2

−1

0

1

(s)

Sw

ing

hip

(rad

/s)

0 2 4 6−1

−0.5

0

0.5

1

(s)

Tor

so (

rad/

s)

DS SS

Figure 3.13 Velocity variables during five steps of the closed-loop simulation. Discontinuitiesin the graphs are due to coordinate relabling.

0 2 4 6

−80

−60

−40

−20

0

(s)

Sta

nce

knee

(N

m)

0 2 4 6−10

0

10

20

30

40

50

60

70

(s)

Sta

nce

hip

(Nm

)

0 2 4 6−25

−20

−15

−10

−5

0

5

10

(s)

Sw

ing

knee

(N

m)

0 2 4 6−50

−40

−30

−20

−10

0

10

(s)

Sw

ing

hip

(Nm

)

DS

SS

Figure 3.14 Control inputs (i.e., joint torques) during five steps of the closed-loop simulation.Discontinuities in the graphs are due to transition between the continuous phases.

0 2 4 6−20

−10

0

10

20

30

40

(s) Sta

nce

leg

end

horiz

onta

l for

ce (

N)

0 2 4 6

100

200

300

400

500

(s)

Sta

nce

leg

end

vert

ical

forc

e (N

)

0 2 4 6−40

−30

−20

−10

0

(s)

Sw

ing

leg

end

horiz

onta

l for

ce (

N)

0 2 4 60

50

100

150

200

250

(s)

Sw

ing

leg

end

vert

ical

forc

e (N

) DS

SS



*1q σ

σ

*2q

* *1t

12

2

Projection onto b1

****1 2

2

: ,: ,b

b

t t t tt t t t

* *2t

*cm

cm

1


*1q

*2q

* *1t

12

2

Projection onto b

****1 2: ,b t t t t

* *2t

*cm

cm

1


max2fxmin

2fx

2fx

2fx

2fx

1fx

2fx

A

B

Solution of the minimization problem

E

1fx

CD

O

1 20 0

, , , 3 4,m M L L x x

Solution of the maximization problem


min3 1 1 2; ,f fx z x x min

4 1 1 2; ,f fx z x x min1 1 2; ,f fv z x x

max1 1 2; ,f fv z x xmax

4 1 1 2; ,f fx z x xmax3 1 1 2; ,f fx z x x

3 1 1 2; ,f fx z x x 4 1 1 2; ,f fx z x x 1 1 2; ,f fv z x x

1z 2z 3z

1 1

cm

1t2t

* .d t Joint

torques 1z

1fx

2fx

1 2, , ,m M L L

Modified reference trajectory

Lookup table


0 0.2 0.4 0.6 0.8

0.8

1

φ 1 (ra

d)

0 0.2 0.4 0.6 0.8

1.8

2

2.2

φ 2 (ra

d)

0 0.2 0.4 0.6 0.80.8

0.9

1

1.1

θ (r

ad)

0 0.2 0.4 0.6 0.8−10

−5

0

5

10

dφ1/d

t (ra

d/s)

0 0.2 0.4 0.6 0.8−10

−5

0

5

10

Time (s)

dφ2/d

t (ra

d/s)

0 0.2 0.4 0.6 0.8

−5

0

5

Time (s)

dθ/d

t (ra

d/s)

Flight Stance

Figure 4.6 Plot of the state trajectories corresponding to two consecutive steps of the desiredperiodic orbit. The discontinuities in velocity are due to the impact.

0 0.2 0.4 0.6 0.8−100

−50

0

50

100

u 1 (N

m)

0 0.2 0.4 0.6 0.8−100

−50

0

50

100

150

200

u 2 (N

m)

0 0.2 0.4 0.6 0.8−200

−100

0

100

200

300

Time (s)

F1h (

N)

0 0.2 0.4 0.6 0.80

200

400

600

800

1000

Time (s)

F1v (

N)

Flight

Stance

Figure 4.7 Plot of commanded control inputs and ground reaction force during two conse-cutive steps of the desired periodic orbit. The discontinuities are due to the transitions betweenthe stance and flight phases.

−22 −20 −18 −16 −14 −12 −10 −8 −6−25

−20

−15

−10

−5

0

σs− (kgm2 /s)

(kgm

2 /s)

ρcl (σs−)

ρol (σs−)

σs−

σs−*

Figure 4.8 Plot of the open-loop and closed-loop restricted Poincare return maps ρol, ρcl. Theplot is truncated at −7.3227(kgm2/s) because this point is an upper bound for the domain ofdefinition of ρcl. For |σ−

s | sufficiently large, the ground reaction force at the leg end will not bein the static friction cone. The mapping ρcl has two fixed points. One fixed point (σ−

s = σ−∗s =

−13.7227(kgm2/s)) is asymptotically stable and corresponds to the desired periodic trajectory,while the other fixed point is unstable and occurs at approximately σ−

s = −7.4964(kgm2/s).

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6

0.7

0.8

0.9

1

1.1

φ 1 (ra

d)

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.61.7

1.8

1.9

2

2.1

2.2

φ 2 (ra

d)

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.60.8

0.9

1

1.1

θ (r

ad)

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6

−20

−10

0

10

dφ1/d

t (ra

d/s)

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6

−20

−10

0

10

Time (s)

dφ2/d

t (ra

d/s)

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6−20

−15

−10

−5

0

5

Time (s)

dθ/d

t (ra

d/s)

Flight

Stance

Figure 4.9 Plot of the state trajectories corresponding to four consecutive steps of themonoped robot. The discontinuities in velocity are due to the impact.

0.7 0.8 0.9 1 1.1−25

−20

−15

−10

−5

0

5

10

15

φ1 (rad)

dφ1/d

t (ra

d/s)

1.8 1.9 2 2.1 2.2−25

−20

−15

−10

−5

0

5

10

φ2 (rad)

dφ2/d

t (ra

d/s)

0.8 0.9 1 1.1−20

−15

−10

−5

0

5

10

θ (rad)

dθ/d

t (ra

d/s)

0.7 0.8 0.9 1 1.1

1.8

2

2.2

0.9

1

1.1

φ1 (rad)φ

2 (rad)

θ (r

ad)

Flight phase

Stance phase

Flight phase

Stance phase

Flight phase

Stance phase

Impact Impact

Impact Stance phase

Flight phase

Impact

Figure 4.10 Phase–plane plots and projection of the state trajectories during 10 consecutivesteps onto (ϕ1, ϕ2, θ). The convergence to the desired periodic trajectory can be seen.

0 0.5 1 1.5−100

−50

0

50

100

Time (s)

u 1 (N

m)

0 0.5 1 1.5−200

0

200

400

600

Time (s)

u 2 (N

m)

5 10 15 203000

3500

4000

4500

Step

(N2 m

s)

5 10 15 204.4

4.6

4.8

5

5.2

Step

(N)

J1(i)

J1*

J1av

(i)

J2(i)

J2av

(i)

Stance

Flight

Figure 4.11 Plot of commanded control inputs during four consecutive steps of running (topgraphs). The discontinuities in the control inputs are due to the transitions between the stanceand flight phases. The bottom graphs present the plot of the cost function J1(i), J2(i), J1,av(i)and J2,av(i) for i = 1, 2, . . . , 20. The periodic orbit O is designed to minimize the cost function(4.45). On this trajectory, J = J∗

1 = 3.2836 × 103(N2ms). From the figure, the value of J1 aftera short transient period (four steps) is approximately equal to J∗

1 , which, in turn, illustrates theefficiency of the algorithm in the sense of electric motor energy per distance traveled.

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0.7

0.8

0.9

1

1.1

φ 1d (ra

d)

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

1.8

2

2.2

φ 2d (ra

d)

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0.8

0.9

1

1.1

Normalized flight time

θ (r

ad)


2fx

1fx

2fx

A

B

R

E

1fx

C

D

O

max2 ;m,Mfxb

min 2 ;m,Mfxb

m,M

maxml

maxMl


maxMW1fx

2fx

OmaxmW

maxmin cc

minmax cc

minmax cc

minmax cc

maxmin cc

minmax cc

maxc

minc

M

M

m

mm

M

12

mMmaxMl

maxml

max1

(M2

m)l

maxmin cc

minmax cc

M

m

M

m M

m


Figure 5.4 Block diagram of the online motion planning algorithm for generation ofcontinuous joint motion ϕ(t) = ϕ∗(τ(t)), t1 ≤ t ≤ t2 to solve configuration determinism.

Figure 5.5 Plot of the desired trajectories for the joint angles (i.e., q1 and q2) generated by theonline motion planning algorithm of Theorem 5.2, the absolute orientation (q3) versus normal-ized time during the flight phases of four consecutive steps (solid curves) and the projection ofthe state variables onto the configuration space. The nominal trajectory is depicted by dashedcurves. The circles at the both ends represent the initial and final predetermined configurations.

0 1 2−0.2

0

0.2

φ 1 (ra

d)

0 1 20

0.5

1

1.5

φ 2 (ra

d)

0 1 20

1

2

φ 3 (ra

d)

0 1 2−0.2

0

0.2

θ 1 (ra

d)

0 1 20

0.2

0.4

θ 2 (ra

d)

0 1 2−0.1

0

0.1

θ 3 (ra

d)

0 1 2−4

−2

0

2

4

d/d

tφ1 (

rad/

s)

0 1 2−10

0

10

20

d/d

tφ2 (

rad/

s)

0 1 2−10

0

10

d/d

tφ3 (

rad/

s)

0 1 2−2

0

2

d/d

tθ1 (

rad/

s)

Time (s) 0 1 2

−2

0

2

4

6

d/d

tθ2 (

rad/

s)

Time (s) 0 1 2

−1

0

1

d/d

tθ3 (

rad/

s)

Time (s)


0 0.5 1 1.5 2−10

−5

0

5

10

u 1 (N

m)

Time (s) 0 0.5 1 1.5 2

−60

−40

−20

0

20

40

u 2 (N

m)

Time (s)

0 0.5 1 1.5 2−100

−50

0

u 3 (N

m)

Time (s) 0 0.5 1 1.5 2

−20

−10

0

10

20

Fx (

N)

Time (s)

0 0.5 1 1.5 2

−5

0

5

Fy (

N)

Time (s) 0 0.5 1 1.5 2

0

100

200

300

400

Fz (

N)

Time (s)


Step 2

Step 1C

Figure 6.4 Geometric description of hybrid invariance. The plot depicts that under the 4-tuple event-based update law (π1→1

s→f , π1→2f→s, π

2→2s→f , π2→1

f→s), the family of the zero dynamics man-ifolds for the first stance phase Z1

s is hybrid invariant, that is, �(x1−s ; ψ1, ξ2, ψ2) ∈ Z1

s,ξ1 , where

�(x1−s ; ψ1, ξ2, ψ2) := �s

f (x2−f ) is the two-step reset map. In addition, a

j

Ns−1 = aj

Ns= 03×1 re-

sults in the common intersection Sfs ∩ Zj

s . Plot also illustrates the five-dimensional restrictedPoincare return map P(x1−

s ; ξ1S, ψ

1S, ξ

2S, ψ

2S) and the HZD.

−0.2 −0.1 0 0.1 0.2

−2

0

2

φ1 (rad)

d/dt

φ 1 (ra

d/s)

0 0.5 1 1.5−10

0

10

20

φ2 (rad)

d/dt

φ 2 (ra

d/s)

0.5 1 1.5

−10

0

10

φ3 (rad)

d/dt

φ 3 (ra

d/s)

−0.1 −0.05 0 0.05 0.1 0.15−2

−1

0

1

2

θ1 (rad)

d/dt

θ 1 (ra

d/s)

0 0.2 0.4 0.6−4

−2

0

2

4

6

θ2 (rad)

d/dt

θ 2 (ra

d/s)

−0.06 −0.04 −0.02 0 0.02 0.04−0.5

0

0.5

θ3 (rad)

d/dt

θ 3 (ra

d/s)

Figure 6.5 Phase portraits of the state trajectories during 40 consecutive steps of running.The stance and flight phases are shown by bold and light curves, respectively. In the figure, theeffect of the impact with the ground is illustrated by jumps in the velocity. The convergence tothe desired limit cycle O can be seen.

0 5 10 15 20−10

−5

0

5

10

15

time (s)

u 1 (N

m)

0 5 10 15 20−60

−40

−20

0

20

40

time (s)

u 2 (N

m)

0 5 10 15 20−150

−100

−50

0

time (s)

u 3 (N

m)

0 5 10 15 20−30

−20

−10

0

10

20

time (s)

Fx (

N)

0 5 10 15 20

−5

0

5

time (s)

Fy (

N)

0 5 10 15 200

100

200

300

400

500

time (s)

Fz (

N)


−0.5 −0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.4 0.50

0.5

1

1.5

x (m)

y (m

)

Figure 7.2 Stick animation of the bipedal robot during one step of the optimal motion.

CHAPTER 1

Introduction

1.1 OBJECTIVES OF LEGGED LOCOMOTION AND CHALLENGESIN CONTROLLING DYNAMIC WALKING AND RUNNING

The most effective type of locomotion in rough terrains is legged locomotion. Duringthe past three decades enormous advances have occurred in robot control and mo-tion planning of dynamic walking and running locomotion. In particular, hundreds ofwalking mechanisms have been built in research laboratories and companies through-out the world. The desire to study legged locomotion has been motivated by the needto assist people with disabilities to walk and replace humans in hazardous environ-ments. Underactuation, impulsive nature of the impact between the lower limbs andthe environment, the existence of foot structure and the large number of degrees offreedom (DOF) are basic problems in controlling legged robots. Underactuation isnaturally associated with dexterity. For example, headstands are considered dexterous[1]. In this case, the contact point between the body and the ground is acting as a pivotwithout actuation. The nature of the impact between the lower limbs of legged robotsand the environment causes the dynamics of the system to be hybrid and impulsive.The impact between the foot and the ground is one of the main difficulties in design-ing control laws for walking and running robots. Unlike robotic manipulators, leggedrobots are always free to detach from the walking surface, thereby leading to varioustypes of motions. Finally, the existence of many degrees of freedom in the mecha-nism of legged robots causes the coordination of the links to be difficult. As a result ofthese latter issues, the design of practical controllers for legged robots remains to bea challenging problem. Also, these features complicate the application of traditionalstability margins. Consequently, the major issues in the control of dynamic walkingand running are as follows:

1. Limb coordination. Legged robots are high degree of freedom mechanisms,and consequently, coordination of their links to achieve dynamic walking andrunning locomotion is complex.


1

2 INTRODUCTION

2. Hybrid nature of locomotion due to presence of impact and liftoff. The presenceof impact, foot touchdown, and liftoff leads to models with impulse effects andhybrid systems consisting of multiple continuous and discrete phases. In par-ticular, mathematical models describing the evolution of legged robots duringwalking and running include both discrete and continuous phenomena. Instan-taneous discrete phases arise when feet impact the ground or feet liftoff theground, whereas ordinary differential equations based on classical Lagrangianmechanics describe the evolution of legged robots during continuous phases oflocomotion.

3. Underactuation. During certain phases of walking and running such as singlesupport (one leg on the ground) in walking and flight (no leg on the ground) inrunning, legged robots have fewer actuators than degrees of freedom.

4. Overactuation. During double support phase of bipedal walking (both legs onthe ground), biped robots have fewer degrees of freedom than actuators. Dueto overactuation, the control input corresponding to a specific trajectory in thestate space is not unique.

5. Inability to apply the Zero Moment Point criterion. Most past work in theliterature of legged robots emphasizes the quasi-static stability criteria andflat-footed walking based on the Zero Moment Point (ZMP) [2–16] and the FootRotation Indicator (FRI) point [17]. The ZMP is defined as the point on theground where the net moment generated from ground reaction forces has zeromoment about two axes that lie in the plane of ground [3]. The ZMP is containedin the robot’s support polygon, where the support polygon is defined as theconvex hull formed by all contact points with the ground. The ZMP criterionstates that when the ZMP is contained within the interior of the support polygon,the robot is stable so that it will not topple. Thus, in this kind of stability, as longas the ZMP lies strictly inside the support polygon of the foot the trajectoriesare feasible. If the ZMP lies on the edge of the support polygon, then thetrajectories may not be feasible. The center of pressure (COP) is a standardnotion in mechanics which was renamed as the ZMP by Vukobratovic [3]. TheFRI point is a concept defined when the foot is in rotation with respect to theground [17]. The FRI is the point on the ground where the net ground reactionforce would have to act to keep the foot stationary. Thus, if FRI is within theconvex hull of the stance foot, the robot can walk and it does not roll over itsextremities, such as the heel or the toe. This type of walking is called as fullyactuated walking. If the FRI is not in the projection of the foot on the ground,the stance foot rotates about the extremities. Such an event is also known asunderactuated walking. As long as the foot does not rotate about its extremities,the ZMP, COP, and FRI points are equivalent [15] (see Fig. 1.1). In the literatureof legged locomotion, a statically stable gait is a periodic locomotion in whichthe robot’s center of mass (COM) does not leave the support polygon. A quasi-statically stable gait is a periodic locomotion in which the COP of the robotis within the interior of the support polygon. Moreover, a dynamically stablegait is a periodic locomotion where the robot’s COP is on the boundary of

OBJECTIVES AND CHALLENGES 3

Figure 1.1 Two planar bipedal models to compare the COP, ZMP, and FRI points. The FRIpoint is a point on the ground contact surface, within or outside the convex hull of the footsupport area, at which the resultant moment of the force/torque impressed on the foot is normalto the surface [17]. In the left figure, the foot does not rotate about its extremities, thus, theZMP, COP, and FRI points are equivalent. At the right figure, the foot is starting to rotate sincethe FRI point is outside the convex hull of the stance foot. We note that the COP is at the tip ofthe stance foot about which the foot rotates.

the support polygon for at least part of the walking cycle [18]. Thus, duringdynamic walking and running cycles, the location of the robot’s COP is on theboundary of the support polygon and, as a result, this will prohibit the use ofthe ZMP criterion. To make this notion more precise, the ZMP criterion is asufficient and necessary condition for the stance foot not to rotate. However,this does not imply that the walking motion is asymptotically stable in the senseof Lyapunov [18, Chapter 11].

6. Lack of algorithms to achieve feasible period-one orbits and limit cycles. Themain problem in control of legged locomotion is how to design a feedbacklaw that guarantees the existence of a stable limit cycle for the closed-loopsystem. Underactuation and unilateral constraints must be included in order todesign a feasible periodic orbit for legged locomotion. Unilateral constraintsare constraints on the state and control inputs of the mechanical system thatrepresent feasible contact conditions between the leg ends and the ground.In particular, leg ends, whether they are terminated with feet or points, arenot attached to the ground. Hence, the ground reaction forces must lie in thefriction cone to prevent slippage and foot liftoff. Thus, normal forces at the legends can only act in one direction, and are unilateral. In addition, if the footis to remain flat on the ground and not rotate about its extremities, then theFRI must be between the heel and toe, a condition that can be expressed as apair of unilateral constraints. These facts combined with underactuation duringthe single support and flight phases complicate the design of motion planningalgorithms to generate feasible periodic locomotion.

4 INTRODUCTION

7. Conservation of angular momentum about the robot’s COM during flightphases. During flight phases of running, conservation of angular momentumabout the robot’s COM is a nonintegrable Pfaffian constraint which compli-cates the path planning and control of the robot’s configuration during landing(flight to stance phase).

1.2 LITERATURE OVERVIEW

1.2.1 Tracking of Time Trajectories

Most existing control algorithms in the literature of legged robots are time-dependentapproaches based on tracking of predetermined time trajectories generated by theZMP criterion, inverted pendulum model and nonlinear oscillators as central patterngenerators of the spinal cord. The ZMP criterion [2–16] has been used for trajectorytracking in ASIMO [2] and WABIAN [5, 6]. The Linear Inverted Pendulum Model(LIPM) [19, 20] and ZMP criterion-based approaches for stable walking referencegeneration have been reported in the literature. In these techniques, generally, theZMP reference during a stepping motion is kept fixed in the middle of the supportingfoot. Erbatur and Kurt [9] proposed a reference generation algorithm based on theLIPM and moving support foot ZMP. In addition, they made use of a simple inversekinematics-based joint space controller to test the reference trajectory for simulationof a 3D, 12-DOF biped robot model. By allowing a variation of ZMP over the convexhull of foot polygon, Bum-Joo et al. [10] proposed an algorithm to modify the walk-ing period and the step length in both the sagittal and lateral planes of the humanoidrobot HanSaRam-VII. Motoi et al. [11] presented a real-time gait planning algorithmbased on ZMP for pushing motion of humanoid robots to deal with an object withunknown mass. Kajita et al. [12] presented a ZMP-based running pattern generationalgorithm for running of the humanoid robot HRP-2LR. ZMP-based online jumpingpattern generation for running of a monopedal robot has also been reported in Ref.[13]. Sato et al. [14] proposed walking trajectory planning on stairs for biped robotsusing the method of virtual slope and the ZMP criterion. Sardain and Bessonnet [15]proved the coincidence of COP and ZMP and they examined related control aspects.In this latter reference, the virtual COP–ZMP was also defined to extend the conceptfor walking on uneven terrain. Ref. [21] approximated the biped model as an invertedpendulum and made use of trajectory tracking to control dynamic walking locomo-tion in Biper-3. Katoh and Mori used PID controllers to track reference trajectoriesgenerated by a van der Pol oscillator in the control of BIPMAN [22]. To controlwalking in Kenkyaku, Furusho and Masubuchi applied PID controllers for trackingjoint reference trajectories [23]. Furusho and Sano also applied a decoupled controlapproach for control of motions in the frontal and sagittal planes during walking ofthe three-dimensional bipedal prototype BLR-G2 [24, 25]. PID controllers were em-ployed to track the time trajectories generated by a length-varying inverted pendulumduring walking of Meltran II by Kajita et al. [26, 27]. In these latter references, tomaintain the biped’s COM at a constant height, the pendulum’s length is assumed to

LITERATURE OVERVIEW 5

vary in a proper manner. In Ref. [28], PID control was also used to track predeter-mined trajectories to improve the stability behavior. A computed torque method wasused to control a planar, 5-DOF bipedal robot in Ref. [29]. The performance of threecontrol techniques including PID, computed torque, and sliding mode control in thetracking of joint trajectories during walking by a planar, 5-DOF biped was comparedby Raibert et al. [30]. Tracking of time trajectories, based on computed torque withgravity compensation and a length-varying inverted pendulum model, has also beenapplied during walking by a three-dimensional bipedal robot in Refs. [8, 31, 32]. Tra-jectories generated by an inverted pendulum was also used by Kajita et al., to controlwalking of HRP-2 [33, 34].

The ZMP criterion has become a very powerful tool for trajectory generation inwalking of legged robots. However, it needs a stiff joint control of the prerecorded tra-jectories that leads to poor robustness in unknown rough terrains whereas humans andanimals show exceptional robustness when walking on irregular terrains. It is wellknown in biology that there are Central Pattern Generators (CPG) in spinal cordcoupling with the musculoskeletal system [35–37]. The CPG and feedback networkscan coordinate the body links of the vertebrates during locomotion. There are severalmathematical models that have been proposed for a CPG. Among them, Matsuoka[38–41] has studied an approach in which a CPG is modeled by a Neural Oscilla-tor consisting of two mutually inhibiting neurons. Each neuron is represented by anonlinear differential equation. Matsuoka’s approach has been used by Taga [36, 37]and Miyakoshi et al. [42] for biped robots. Kimura has also used this approach atthe hip joints of quadruped robots [43, 44]. Ref. [45] presented a hybrid CPG–ZMPcontroller for the real-time balance of a simulated flexible spine humanoid robot. TheCPG component of the controller allows the mechanical spine and feet to exhibitrhythmic motions using two control parameters. By monitoring the measured ZMPlocation, the control scheme modulates the neural activity of the CPG to allow therobot to maintain balance on the sagittal and frontal planes in real time.

1.2.2 Poincare Return Map and Hybrid Zero Dynamics

As stated in Section 1.1, the main problem in controlling legged locomotion is howto design a controller that guarantees the existence of an asymptotically stable limitcycle for the closed-loop mechanical system. A classical technique for analyzingstability of periodic orbits for time-invariant dynamical systems described by or-dinary differential equations is the method of Poincare sections. This method es-tablishes an equivalence between the stability analysis of the periodic orbit for annth-order continuous-time system and that of the corresponding equilibrium pointfor an (n − 1)th-order discrete-time system. Grizzle et al. [46] extended the methodof Poincare sections to systems with impulse effects. A system with impulse effectsconsists of a continuous phase described by an ordinary differential equation and adiscrete phase described by an instantaneous reinitialization rule for the differentialequation. To simplify the application of the Poincare sections method in the designof time-invariant controllers for walking by an underactuated three-link biped robotand instantaneous double support phase, Grizzle et al. [46] created zero dynamics

6 INTRODUCTION

manifolds that are forward invariant under the flow of the continuous phase of walk-ing (i.e., single support phase). However, since the zero dynamics manifolds are notinvariant under the flow of the discrete phase (i.e., impact model), the approach ofRef. [46] resulted in a restricted Poincare return map (i.e., the Poincare return maprestricted to the zero dynamics manifolds) that cannot be expressed in a closed-form.The outputs in Ref. [46], corresponding to the zero dynamics manifolds, were se-lected as holonomic functions referred to as virtual constraints. Virtual constraintsare a set of holonomic output functions defined on the configuration space of the me-chanical system. They are forced to be zero by within-stride feedback laws to reducethe dimension of the Poincare return map and to coordinate the links of biped robotsduring walking [47]. The method of virtual constraints for designing time-invariantcontrollers in walking of planar biped robots with one degree of underactuation, pointfeet and instantaneous double support has been studied in Refs. [18, Chapter 6, 46–52]. Westervelt et al. [52] created virtual constraints to ensure that the correspondingzero dynamics manifold is hybrid invariant under the closed-loop hybrid model ofwalking and introduced the notion of hybrid zero dynamics (HZD). The zero dy-namics manifolds are said to be hybrid invariant if they are both forward invariant(i.e., invariant under the flow of the continuous phase) and impact invariant (i.e.,invariant under the flow of the discrete phase). During walking of a planar bipedrobot with an underactuated cyclic variable [1, 53], HZD results in a two-dimensionalzero dynamics manifold, and consequently, a one-dimensional restricted Poincarereturn map that can be expressed in closed form. This approach was also extended byChoi and Grizzle [54] for creating a two-dimensional zero dynamics manifold duringwalking of a planar fully actuated biped robot in fully actuated and underactuatedphases. To reduce the dimension of the full-order hybrid model of running, whichin turn simplifies the stabilization problem of the desired orbit, Ref. [55] proposedthat the configuration of the mechanical system should be transferred from a speci-fied initial pose (immediately after the takeoff) to a specified final pose (immediatelybefore the landing) during flight phases. This problem is referred to as landing in afixed configuration or configuration determinism at landing [18, p. 252]. By usingthe virtual constraints approach and the configuration determinism at landing, Ref.[55] obtained a closed-form expression for the one-dimensional restricted Poincarereturn map of running by the five-link, four-actuator planar bipedal robot, RABBIT[47]. Moreover, to ensure that the stance phase zero dynamics manifold is hybridinvariant under the closed-loop hybrid model of running, an additional constraint wasimposed on the vector of generalized velocities at the end of flight phases. To satisfythe configuration determinism at landing and hybrid invariance, Ref. [55] utilized theapproach of parameterized HZD. In particular, using the Implicit Function Theoremand a numerical nonlinear optimization problem with an equality constraint, the pa-rameters of the virtual constraints of the flight phase were updated in a step-by-stepfashion during the discrete transition from stance to flight (i.e., takeoff). However,the stance phase controller was assumed to be fixed. For running of RABBIT, analternative parameterized control law was proposed by Morris et al. [56]. However,their approach did not create HZD. The use of event-based control laws to update theparameters of time-invariant controllers for stabilization of periodic orbits in systems

THE OBJECTIVE OF THE BOOK 7

with impulse effects was presented in Refs. [57–59]. When the amount of underac-tuation during locomotion of biped robots is increased, it becomes difficult to createhybrid invariant manifolds. Morris and Grizzle [60] proposed a method to generatean open-loop augmented system with impulse effects, a new holonomic output func-tion for the resultant system and an event-based update law for the parameters ofthe output such that the zero dynamics manifold associated with this output is hybridinvariant under the closed-loop augmented system. This latter approach has been usedin design of time-invariant controllers for walking of a 3D biped robot in Refs. [61,62] and also for walking and running of planar bipedal robots with springs, MABEL[63–65] and ATRIAS [66]. Hurmuzlu also applied the method of Poincare sectionsto a planar, five-link bipedal robot and imposed a mix of holonomic and nonholo-nomic constraints on the mechanical system to obtain a closed-form expression forthe robot’s trajectory [67].

1.3 THE OBJECTIVE OF THE BOOK

In this book we provide a comprehensive overview of hybrid models describing theevolution of planar and 3D legged robots during dynamical legged locomotion andalso propose hybrid control schemes to asymptotically stabilize desired periodic orbitsfor the closed-loop systems. The topics include (i) hybrid systems, (ii) systems withimpulse effects, (iii) offline and online motion planning algorithms to generate de-sired feasible periodic walking and running motions, (iv) two-level control schemes,including within-stride feedback laws to reduce the dimension of the hybrid systems,(v) continuous-time update laws to minimize a general cost function online, and(vi) event-based update laws to asymptotically stabilize the desired periodic orbits.This book also provides a comprehensive presentation of issues and challenges facedby researchers and practicing engineers in motion planning and hybrid control of dy-namical legged locomotion. Furthermore, we describe the current state of the art andfuture directions across all domains of dynamical legged locomotion so that readerscan extend the proposed motion planning algorithms and hybrid control methodolo-gies to other planar and 3D legged robots. The main objectives of this book are asfollows.

1.3.1 Hybrid Zero Dynamics in Walking with Double Support Phase

There has been little attention given to control of biped robots during the doublesupport phase with unilateral constraints. Such constraints present challenges for thedesign of controllers. The objective of Chapter 3 is to develop an analytical approachfor designing a continuous feedback law that realizes a desired period-one trajectoryas an asymptotically stable orbit for a planar biped robot. The robot is assumed to bea five-link, four-actuator planar mechanism in the sagittal plane with point feet. Thefundamental assumption is that the double support phase is not instantaneous. Hence,bipedal walking can be represented by a hybrid model with two continuous phases,including a single support phase and a double support phase, and discrete transitions

8 INTRODUCTION

between the continuous phases. In the single support phase, the mechanical systemhas one degree of underactuation, whereas it is overactuated in the double supportphase. Chapter 3 shows how to design a continuous time-invariant feedback law thatasymptotically stabilizes a feasible periodic trajectory using an extension of HZD fora hybrid model of walking [68, 69]. The main contribution is to develop a continuoustime-invariant control law for walking of a planar biped robot during the doublesupport phase. Since the mechanical system in the double support phase has threedegrees of freedom and four actuators, a constrained dynamics approach [70, p.157] is used to describe the reduced-order dynamics of the system. Then, two virtualconstraints are proposed as holonomic outputs for the constrained system and anoutput zeroing problem with two control inputs is solved. This results in a nontrivialtwo-dimensional zero dynamics manifold corresponding to the virtual constraintsin the state manifold of the constrained system. Moreover, the corresponding zerodynamics has two control inputs that are not employed for output zeroing. Instead, theyare used to satisfy the unilateral constraints. Furthermore, these inputs are obtainedsuch that the control has minimum norm on the desired periodic trajectory. It canbe shown that the constrained dynamics of the double support phase is completelyfeedback linearizable on an open subset of the state manifold. However, since ourobjective is to design a continuous time-invariant controller based on nontrivial HZD,in contrast to Ref. [71] we do not use input-state linearization nor a discontinuous timeoptimal control for tracking trajectories. An analogous approach is used in Refs. [54,72] for creating a two-dimensional zero dynamics manifold in the state space of a fullyactuated phase of walking where the fully actuated dynamics is completely feedbacklinearizable. The control strategy is presented at the following two levels. At the firstlevel, we employ within-stride controllers including single and double support phasecontrollers. These are continuous time-invariant and parameterized feedback laws thatcreate a family of two-dimensional finite-time attractive and invariant submanifoldson which the dynamics of the mechanical system is restricted. At the second level, theparameters of the within-stride controllers are updated at the end of the single supportphase (in a stride-to-stride manner) by an event-based update law to achieve hybridinvariance and stabilization. As a consequence, the stability properties of the desiredperiodic orbit can be analyzed using a one-dimensional restricted Poincare return map.

1.3.2 Hybrid Zero Dynamics in Running with an Online MotionPlanning Algorithm

Chapter 4 presents an analytical approach for designing a two-level control law toasymptotically stabilize a desired period-one orbit during running by a planar monope-dal robot. The monopedal robot is a three-link, two-actuator planar mechanism in thesagittal plane with point foot. The desired periodic orbit is generated by the methoddeveloped in Ref. [73]. It is assumed that the model of monopedal running can beexpressed by a hybrid system with two continuous phases, including stance phase andflight phase, and discrete transitions between the continuous phases, including takeoffand landing (impact). The configuration of the mechanical system is specified by theabsolute orientation with respect to an inertial world frame and by the joint angles


determining the shape of the robot. During the flight phase, the angular momentum ofthe mechanical system about its COM is conserved. To reduce the dimension of thefull-order hybrid model of running, which in turn simplifies the stabilization problemof the desired orbit, as proposed in Ref. [55], the configuration determinism at landingshould be solved. However, the flight time and angular momentum about the COMmay differ during consecutive steps. Consequently, the reconfiguration problem mustbe solved online. A number of control problems for reconfiguration of a planar mul-tilink robot with zero angular momentum have been considered in the literature, forexample, Refs. [74–78]. For the case that the angular momentum is not necessarilyzero, a method based on the Averaging Theorem [79, Theorem 2.1] was presented inRef. [80] such that for any value of the angular momentum, joint motions can reorientthe multilink arbitrarily over an arbitrary time interval. However, when the angularmomentum is not zero, this method cannot be employed online for solving the re-configuration problem for monopedal running. For this reason, Chapter 4 presents anonline reconfiguration algorithm that solves this problem for given flight times andangular momenta [81, 82]. The algorithm proposed in Chapter 4 is expressed usingthe methodology of reachability and optimal control for time-varying linear systemswith input and state constraints. The main contribution of this chapter is to present ananalytical approach for online generation of twice continuously differentiable (C2)modified reference trajectories during flight phases of running to satisfy the configura-tion determinism at landing [81]. Moreover, by relaxing the constraint of Ref. [55] onthe vector of generalized velocities at the end of the flight phases, Chapter 4 presentsa two-level control scheme based on the reconfiguration algorithm to asymptoticallystabilize a desired periodic orbit. In this scheme, within-stride controllers, includingstance and flight phase controllers, are employed at the first level. The stance phasecontroller is chosen as a time-invariant and parameterized feedback law to generatea family of finite-time attractive zero dynamics manifolds. An alternative approachbased on continuous feedback law is employed here to track the modified referencetrajectories generated by the reconfiguration algorithm during the flight phases. Togenerate a family of hybrid invariant manifolds, an event-based controller updates theparameters of the stance phase controller during the transition from flight to stance(i.e., impact) [81]. Consequently, the stability properties of the desired periodic orbitcan be analyzed and modified by a one-dimensional discrete-time system defined onthe basis of a restricted Poincare return map.

1.3.3 Online Motion Planning Algorithms for Flight Phasesof Running

Following the results of Chapter 4, to asymptotically stabilize the desired periodicorbit for the hybrid model of running using a one-dimensional restricted Poincarereturn map and HZD approach, the configuration of the mechanical system should betransferred from a predetermined initial pose (immediately after takeoff) to a prede-termined final pose (immediately before landing) during the flight phases of running.The objective of Chapter 5 is to present modified online motion planning algorithmsfor generation of continuous (C0) and continuously differentiable (C1) open-loop

10 INTRODUCTION

trajectories in the body configuration space of the mechanical system such that thereconfiguration problem is solved [82, 83]. The algorithms presented in Chapter 5are extensions of that presented in Chapter 4. In particular, the generated trajecto-ries in Chapter 4 were C2 while the reachable sets associated with the algorithms ofChapter 5 are larger than that of Chapter 4. We address the motion planning problemfor general planar open kinematic chains composed of N ≥ 3 rigid links intercon-nected with frictionless and rotational joints. The main contribution of Chapter 5 isto present online motion planning algorithms based on virtual time for generation ofjoint motions to satisfy configuration determinism at transitions. In particular, it isassumed that the time trajectory of a desired joint motion, precomputed offline, solvesthe reconfiguration problem. By replacing the time argument of the desired motionby a strictly increasing function of time called the virtual time, Chapter 5 shows howto determine continuous and continuously differentiable joint motions in an onlinemanner during consecutive steps of running so that they solve the reconfigurationproblem.

1.3.4 Hybrid Zero Dynamics in 3D Running

Chapter 6 presents a motion planning algorithm to generate periodic time trajectoriesfor running by a 3D monopedal robot. In order to obtain a symmetric gait along astraight line, the overall open-loop model of running can be expressed as a hybridsystem with four continuous phases consisting of two stance phases and two flightphases and discrete transitions among them (takeoff and impact). The robot is assumedto be a 3D, three-link, three-actuator, monopedal mechanism with a point foot. Duringthe stance phases, the robot has three degrees of underactuation, whereas it has sixdegrees of underactuation in the flight phases. The motion planning algorithm isdeveloped on the basis of a finite-dimensional nonlinear optimization problem withequality and inequality constraints and extends the results of Refs. [73, 84] for planarbipedal robots. The main objective of Chapter 6 is to develop time-invariant feedbackscheme to exponentially stabilize a desired periodic orbit generated by the motionplanning algorithm for the hybrid model of running.

Chapter 6 shows how to create hybrid invariant manifolds during 3D running [85].By assuming that the control inputs of the mechanical system have discontinuitiesduring discrete transitions between continuous phases, the takeoff switching hyper-surface can be expressed as a zero level set of a scalar holonomic function. In otherwords, takeoff occurs when a scalar quantity, a strictly increasing function of timeon the desired gait, passes through a threshold value. The virtual constraints duringstance phases are defined as the summation of two terms including a nominal holo-nomic output function vanishing on the periodic orbit and an additive parameterizedBezier polynomial, both in terms of the latter strictly increasing scalar. By propertiesof Bezier polynomials, an update law for the parameters of the stance phase virtualconstraints is developed, which in turn results in a common intersection of the parame-terized stance phase zero dynamics manifolds and the takeoff switching hypersurface.By this approach, creation of hybrid invariance can be easily achieved by updating theother parameters of the Bezier polynomial. Consequently, a parameterized restricted


Poincare return map can be defined on the common intersection for studying thestabilization problem. Thus, the overall feedback scheme can be considered at twolevels. At the first level, within-stride controllers including stance and flight phasecontrollers, which are continuous time-invariant and parameterized feedback laws,are employed to create a family of attractive zero dynamics manifolds in each of thecontinuous phases. At the second level, the parameters of the within-stride controllersare updated by event-based update laws during discrete transitions between continu-ous phases to achieve hybrid invariance and stabilization. By this means, the stabilityanalysis of the periodic orbit for the full-order hybrid system can be treated in termsof a reduced-order hybrid system with a five-dimensional Poincare return map.

1.3.5 Hybrid Zero Dynamics in Walking with Passive Knees

In Chapter 7, a motion planning algorithm to generate time trajectories of a periodicwalking motion by a five-link, two-actuator planar bipedal robot is presented. In orderto reduce the number of actuated joints for walking on a flat ground and restore thewalking motion in the disabled, it is assumed that the robot has passive point feet andunactuated knee joints. In other words, only the hip joints of the robot are assumedto be actuated. The motion planning algorithm is developed on the basis of a finite-dimensional nonlinear optimization problem with equality and inequality constraints.The equality constraints are necessary and sufficient conditions by which the impul-sive model of walking has a period-one orbit. Whereas the inequality constraints areintroduced to guarantee (i) the feasibility of the periodic motion and (ii) capability ofapplying the proposed two-level control scheme for stabilization of the orbit. The mainobjective of Chapter 7 is to present a time-invariant two-level feedback law based onthe notion of virtual constraints and HZD to exponentially stabilize a desired periodicmotion generated by the motion planning algorithm [86]. The studied mechanicalsystem has three degrees of underactuation during single support. Chapter 7 presentsa control methodology for creation of hybrid invariant manifolds and stabilization ofa desired periodic orbit for the impulsive model of walking. In particular, for a giveninteger number M ≥ 2, we introduce M − 1 within-stride switching hypersurfacesand thereby split the single support phase into M within-stride phases. The within-stride switching hypersurfaces are defined as level sets of a scaler holonomic quantitythat is strictly increasing function of time on the desired walking motion. To stabilizethe desired orbit, the overall controller is chosen as a two-level feedback law. At thefirst level, during a within-stride phase, a parameterized holonomic output function isdefined for the dynamical system and imposed to be zero by using a continuous-timefeedback law. The output function is expressed as the difference between the actualvalues of the angle of hip joints and their desired evolutions, in terms of the latter in-creasing holonomic quantity. At the second level, the parameters of continuous-timefeedback laws are updated during within-stride transitions by event-based updatelaws. The purpose of updating the parameters is (i) achieving hybrid invariance, (ii)continuity of continuous-time feedback laws during within-stride transitions, and (iii)stabilization of the desired orbit. From the construction procedure of the parameter-ized output functions and event-based update laws, it is shown that the intersections

12 INTRODUCTION

of the corresponding zero dynamics manifolds and within-stride switching hyper-surfaces are independent of the parameters. Consequently, by choosing one of thesecommon intersections as the Poincare section, stabilization can be addressed on thebasis of a five-dimensional restricted Poincare return map.

1.3.6 Hybrid Zero Dynamics with Continuous-Time Update Laws

To improve the convergence rate, the idea of updating the parameters of time-invariantstabilizing controllers by event-based update laws has been described in Refs. [57–59]. The contribution of Chapter 8 is to develop a method for designing a class ofcontinuous-time update laws to update the parameters of stabilizing controllers duringcontinuous phases of locomotion such that (i) a general cost function (such as theenergy of the control input over single support) can be minimized in an online manner,and (ii) the exponential stability behavior of the limit cycle for the closed-loop systemis not affected [87]. In addition, Chapter 8 introduces a class of continuous-time updatelaws with radial basis step length to minimize a desired cost function in terms of thecontroller parameters and initial states.

CHAPTER 2

Preliminaries in Hybrid Systems

In this chapter, we will introduce some of the basic ideas, definitions, and results fromthe literature of hybrid systems. We will concentrate on the ideas used most often forgeneralization of the Poincare return map. Our treatment is primarily intended as areview for the reader’s convenience, with some additional focus on the geometricaspects of the subject. This chapter may be skipped in the first reading, or by thereader familiar with the results. We will refer to texts such as Ref. [18] for morecomplete developments and details.

2.1 BASIC DEFINITIONS

To define a hybrid model with two continuous phases, consider the differential equa-tions x1 = f1(x1) and x2 = f2(x2) which are defined on the state spaces X1 and X2,respectively. It is assumed thatX1 andX2 are embedded submanifolds ofRn1 andRn2

for some n1, n2 ∈ N. Let S21 and S1

2 be switching hypersurfaces in the state spacesX1 and X2 on which the transition from X1 to X2 and the transition from X2 to X1occur, according to the continuously differentiable transition maps �2

1 : S21 → X2

and �12 : S1

2 → X1, respectively. It is assumed that S21 and S1

2 can be expressed as

S21 = {x1 ∈ X1|H2

1 (x1) = 0}S1

2 = {x2 ∈ X2|H12 (x2) = 0},

where H21 : X1 → R and H1

2 : X2 → R are C1 such that ∀x1 ∈ S21,

∂H21

∂x1(x1) /= 0 and

∀x2 ∈ S12,

∂H12

∂x2(x2) /= 0. By these assumptions, S2

1 and S12 are embedded submani-

folds of X1 and X2. Moreover, suppose that S21 ∩ �1

2(S12) = φ and S1

2 ∩ �21(S2

1) = φ.The autonomous hybrid model with two continuous phases denoted by the 8-tuple


13

14 PRELIMINARIES IN HYBRID SYSTEMS

�(X1,X2,S21,S1

2, �21, �

12, f1, f2) is defined as follows

�1 :

⎧⎪⎪⎪⎨⎪⎪⎪⎩

X1 ⊂ Rn1

F1 : x1 = f1(x1)

S21 = {x1 ∈ X1 | H2

1 (x1) = 0}T 2

1 : x+2 = �2

1(x−1 )

�2 :

⎧⎪⎪⎪⎨⎪⎪⎪⎩

X2 ⊂ Rn2

F2 : x2 = f2(x2)

S12 = {x2 ∈ X2 | H1

2 (x2) = 0}T 1

2 : x+1 = �1

2(x−2 ),

(2.1)

where for every i ∈ {1, 2}, Fi represents the flow of the autonomous differentialequation xi = fi(xi). Moreover, x−

i (t) := limτ↗t xi(τ) and x+i (t) := limτ↘t xi(τ) are

the left and right limits of the trajectory xi(t) ∈ Xi, respectively.

Definition 2.1 (Continuously Differentiable Hybrid Model) The autonomous hy-brid model �(X1,X2,S2

1,S12, �

21, �

12, f1, f2) is said to be continuously differentiable

or C1 if for every i ∈ {1, 2}, fi : Xi → TXi is C1.

As in Ref. [18, p. 92], a solution of the hybrid model (2.1) is constructed bypiecing together the trajectories of the flows Fi, i = 1, 2 such that the transitions takeplace when these flows intersect the switching hypersurfaces Sj

i , i, j = 1, 2, i /= j.The new initial conditions for the differential equations xj = fj(xj), j = 1, 2 are

also determined by the transition maps �ji , i, j = 1, 2, i /= j (i.e., x+

j = �ji (x

−i )). To

make this notion precise, we need to define exactly what we mean by a “solution” ofthe hybrid model.

Definition 2.2 (Solutions of the Hybrid Model) Let X := X1 ∪ X2 be the union ofthe state spaces. Assume that (t0, x0) ∈ R× X and tf ∈ R ∪ {∞} are a given initialpair and final time, respectively. Suppose that there exists a closed discrete subsetT := {t0 < t1 < · · · < tj < · · · } ⊂ [t0, tf ) representing the switching times, and afunction i : T → {1, 2} determining the continuous phases of the hybrid model suchthat i(j) /= i(j + 1) for all j ≥ 0, where i(j) := i(tj). A function ϕ : [t0, tf ) × {x0} →X is said to be a solution of the hybrid model (2.1) if

1. for all j ≥ 0,

(a) ϕ(t, x0) restricted to the interval [tj, tj+1) is right continuous, andϕ(t, x0) ∈Xi(j) for every t ∈ [tj, tj+1);

(b) for every point t ∈ [tj, tj+1), the left limit ϕ−(t, x0) := limτ↗t ϕ(τ, x0) ex-

ists and is finite. Furthermore, ϕ−(t, x0) /∈ Si(j+1)i(j) ;

BASIC DEFINITIONS 15

(c) for every point t ∈ (tj, tj+1), ∂∂t

ϕ(t, x0) = fi(j)(ϕ(t, x0)); and

2. for all j ≥ 1 and tj < ∞, ϕ+(tj, x0) = �i(j+1)i(j) (ϕ−(tj, x0)).

Next let the autonomous hybrid model �(X1,X2,S21,S1

2, �21, �

12, f1, f2) satisfy

the following hypothesis:

(H1) The vector fields fi : Xi → TXi, i = 1, 2 are continuous and the solutionsof the differential equations xi = fi(xi), i = 1, 2 for a given initial conditionin the state spaces Xi, i = 1, 2 are unique and depend continuously on theinitial conditions.

Then, the solutions of � are unique. By definition, the solution is also right continuous,and there does not exist a value of t such thatϕ(t, x0) ∈ Sj

i for i, j = 1, 2, i /= j. Conse-

quently, if x0 ∈ Sji for some i, j = 1, 2, i /= j, then, ϕ(t, x0) is defined as ϕ(t, �j

i (x0))[18, p. 86]. A solution ϕ(t, x0) of the hybrid model (2.1) defined on the interval [t0, ∞)is said to be periodic if there exists a finite T > 0 such that ϕ(t + T, x0) = ϕ(t, x0) forall t ≥ t0.

Now we will direct our attention to the definitions of orbital stability from theliterature of hybrid systems. For this purpose, let O be a periodic orbit of the hybridmodel corresponding to the periodic solution ϕ(t, x0), that is, O = {ϕ(t, x0)|t ≥ t0}.The following definitions adopted from Ref. [18, p. 86] will be used for orbitalstability throughout this book.

Definition 2.3 (Stability) The periodic orbit O of the hybrid model (2.1) is said tobe stable if for every ε > 0, there is an open neighborhood N of O such that for everyx ∈ N(O) there exists a solution ϕ(t, x) of the hybrid model defined on the interval[0, ∞) such that

dist(ϕ(t, x),O) < ε

for all t ≥ 0.

Definition 2.4 (Asymptotic Stability) The periodic orbit O of the hybrid model(2.1) is said to be asymptotically stable if it is stable, and there is an open neighbor-hood N(O) such that for every x ∈ N(O) there exists a solution ϕ(t, x) of the hybridmodel defined on the interval [0, ∞) where

limt→∞ dist(ϕ(t, x),O) = 0.

Definition 2.5 (Exponential Stability) The periodic orbit O of the hybrid model(2.1) is said to be exponentially stable if there exist an open neighborhood N(O) andpositive scalar numbers N and γ such that for every x ∈ N(O) there exists a solution


ϕ(t, x) of the hybrid model defined on the interval [0, ∞) where

dist(ϕ(t, x),O) ≤ N dist(x,O) exp(−γt)

for all t ≥ 0.

2.2 POINCARE RETURN MAP FOR HYBRID SYSTEMS

This section reviews some of the key concepts that will be useful in the existence andstability analysis of periodic solutions of closed-loop hybrid systems. A classical ap-proach for analyzing dynamical systems is due to Poincare. In the method of Poincaresections, the flow of annth-order autonomous continuous-time system is replaced withan (n − 1)th-order autonomous discrete-time system. Thus, the Poincare return mapreduces the order of the dynamical system. Moreover, it establishes an equivalencebetween the existence and stability analysis of limit cycles of continuous-time sys-tems and those of corresponding equilibrium points of discrete-time systems. In thissection, the method of the Poincare sections for autonomous hybrid systems with twocontinuous phases is introduced. To do this, assume that hypothesis H1 is satisfied.Let ϕi(t, x0) for i ∈ {1, 2} represent the unique integral curve of the differential equa-tion xi = fi(xi) with the initial condition x0, that is, there exists tf,i(x0) > 0 suchthat ∂

∂tϕi(t, x0) = fi(ϕi(t, x0)) for all t ∈ [0, tf,i(x0)) and furthermore, ϕi(0, x0) =

x0. Define the function T2 : X2 → R ∪ {∞} as the first time at which the flowϕ2(t, x0) intersects the switching manifold S1

2. This is made precise in the followingdefinition:

T2(x0) :={

inf{t ≥ 0|ϕ2(t, x0) ∈ S12} if∃t such that ϕ2(t, x0) ∈ S1

2

∞ otherwise.(2.2)

Next, introduce S21 as the set of all x1 ∈ S2

1 for which the flow ϕ2(t, �21(x1)) intersects

the manifoldS12 transversally in a positive finite time. In a similar manner, the function

T1 : X1 → R ∪ {∞} is defined as follows:

T1(x0) :={

inf{t ≥ 0|ϕ1(t, x0) ∈ S21} if∃t such that ϕ1(t, x0) ∈ S2

1

∞ otherwise.(2.3)

Moreover, define S12 as the set of all x2 ∈ S1

2 for which the flow ϕ1(t, �12(x2)) intersects

the manifold S21 transversally in a positive finite time. We refer the reader to Ref. [18, p.

94] for more details. A formal definition of the Poincare return map can be expressedas follows. Let the maps P2 : S2

1 → S12 and P1 : S1

2 → S21 can be introduced in the

POINCARE RETURN MAP FOR HYBRID SYSTEMS 17

following forms:

P2(x1) := ϕ2(T2 ◦ �21(x1), �2

1(x1))

P1(x2) := ϕ1(T1 ◦ �12(x2), �1

2(x2)).(2.4)

Then, the Poincare return map P : S12 → S1

2 is defined as

P(x2) := P2 ◦ P1(x2). (2.5)

The following theorem is an important result which allows us to consider P as thePoincare return map for a system with impulse effects. Consequently, the resultsdeveloped for the existence and stability analysis of systems with impulse effects canbe applied to hybrid systems with two continuous phases.

Theorem 2.1 (P as the Poincare Map for an Impulsive System) [18, p. 95]Assume that hypothesis H1 is satisfied and P denotes the Poincare return map forthe autonomous hybrid model �. Then, P is well-defined and continuous. If � isalso C1, P is continuously differentiable. Moreover, P is the Poincare return map forthe autonomous system with impulse effects �ie(X2,S, �, f2), where S := S1

2 and�(x2) := �2

1 ◦ P1(x2).

It is remarkable that the system with impulse effects �ie(X2,S, �, f2) can beexpressed as

�ie :

{x2 = f2(x2) x−

2 /∈ Sx+

2 = �(x−2 ) x−

2 ∈ S.(2.6)

The definition of solutions for the system �ie is similar to that presented for thehybrid system �. Following the results of Ref. [18, p. 88], it can be shown that underhypothesis H1 the switching map � is continuous. Moreover, if the hybrid system �

is C1, � is also C1. The geometric description of Theorem 2.1 is illustrated in Fig. 2.1.Let O be a periodic orbit of the hybrid system � transversal to S2

1 and S12. To define

what we mean by the notion of “transversality,” letO1 := O ∩ X1 andO2 := O ∩ X2.In addition, for the later purposes, define {x∗

1} := O1 ∩ S21, {x∗

2} := O2 ∩ S12, T ∗

1 :=T1 ◦ �1

2(x∗2), and T ∗

2 := T2 ◦ �21(x∗

1), where O1 and O2 represent the closure sets ofO1 and O2, respectively. The periodic orbit O = O1 ∪ O2 is said to be transversalto S2

1 and S12 if (i) {x∗

1} and {x∗2} are singletons and (ii) Lf1H

21 (x∗

1), Lf2H12 (x∗

2) /= 0,

where LfiHji (x∗

i ) := ∂Hj

i

∂xi(x∗

i )fi(x∗i ) for i, j ∈ {1, 2} and i /= j.

By the construction of the system �ie, O2 is a periodic orbit of the system �ie.Conversely, if O2 is the periodic orbit of the system �ie, hypothesis H1 implies thatthere exists a unique solution of the differential equation x1 = f1(x1) with the initialcondition �1

2(x∗2) and, as a consequence, O is the only periodic orbit of the system �

such that O ∩ X2 = O2.


Figure 2.1 Geometric description of Theorem 2.1. The Poincare return map of theautonomous hybrid system �(X1,X2,S2

1,S12, �

21, �

12, f1, f2), P : S1

2 → S12, is also the

Poincare return map for the autonomous system with impulse effects �ie(X2,S, �, f2), whereS := S1

2 and �(x2) := �21 ◦ P1(x2). (See the color version of this figure in color plates section.)

Now we are in a position to introduce one of the fundamental theorems of thischapter. This theorem is basically a generalization of the method of Poincare sectionsfor systems with impulse effects. It also establishes an equivalence between thestability analysis of periodic orbits of �ie and corresponding equilibrium points ofthe discrete-time system x2[k + 1] = P(x2[k]) in the state space S.

Theorem 2.2 (Method of Poincare Sections) [18, p. 89]1 Assume that hypothesisH1 is satisfied. Then, the following statements are true:

1. x∗2 is a stable (asymptotically stable) equilibrium point of x2[k + 1] = P(x2[k])

if and only if the orbit O2 is stable (asymptotically stable) for the impulsivesystem �ie.

1 This theorem is a restatement of Theorem 4.1 of Ref. [18, p. 89].


2. If the hybrid system � is also continuously differentiable, then, x∗2 is an expo-

nentially equilibrium point of x2[k + 1] = P(x2[k]) if and only if the orbit O2is exponentially stable for the impulsive system �ie.

Using Theorem 2.2, the stability analysis of the transversal periodic orbit O2 forthe system with impulse effects �ie can be translated into the stability analysis ofthe equilibrium point x∗

2 for the discrete-time system x2[k + 1] = P(x2[k]) definedon the state space S. Next, suppose that the periodic orbit O2 is stable for the systemwith impulse effects �ie. What can we say about the stability behavior of the periodicorbit O for the hybrid system �? To investigate the stability properties of the periodicorbit O2 on the basis of stability of the periodic orbit O, we present the followingtheorem.

Theorem 2.3 (Equivalence of the Stability Behavior) Assume that hypothesis H1is satisfied. Then, the following statements are true:

1. O is stable (asymptotically stable) for the hybrid system � if and only if O2 isstable (asymptotically stable) for the system with impulse effects �ie.

2. If the hybrid system� is also continuously differentiable, thenO is exponentiallystable for the hybrid system � if and only if O2 is exponentially stable for thesystem with impulse effects �ie.

Proof. The first statement is immediate, so only the second statement is proved. More-over, since the necessity of the second statement is straightforward, the sufficiency isproved.

From Definition 2.5, if the periodic orbit O is exponentially stable,

dist(ϕ(t, x),O) ≤ N dist(x,O) exp(−γt) (2.7)

for every t ≥ 0 and x ∈ N(O). Also from the definition of the distance function on X[18, p. 93], inequality (2.7) can be expressed as

{dist(ϕ(t, x),O2) ≤ Nd exp(−γt), ϕ(t, x) ∈ X2

dist(ϕ(t, x),O1) ≤ Nd exp(−γt), ϕ(t, x) ∈ X1(2.8)

for all t ≥ 0, where d := dist(x,O). Without loss of generality, assume that x ∈ X2 ⊂X2, where X2 is the set of all points x2 ∈ X2 for which there exists a closed discretesubset T(x2) := {0 = t0 < t1 < ... < tj < ...} ⊂ [0, ∞) determining the switchingtimes corresponding to the initial condition x2, and a solution ϕ(., x2) : [0, ∞) → Xsuch that ϕ(t, x2) ∈ X2 for all t ∈ [t2k, t2k+1), k = 0, 1, 2, · · · and ϕ(t, x2) ∈ X1 for allt ∈ [t2k+1, t2k+2), k = 0, 1, 2, · · · . Hence, inequality (2.8) can be rewritten as follows

{dist(ϕ(t, x),O2) ≤ Nd exp(−γt), t ∈ [t2k, t2k+1)

dist(ϕ(t, x),O1) ≤ Nd exp(−γt), t ∈ [t2k+1, t2k+2).(2.9)


Since in the construction of the system with impulse effects �ie, the time duration ofphase 1 is omitted, the set of switching times for the corresponding solution of thesystem �ie can be expressed as Tie(x) := {tk}∞k=0, where

t0 := t0 = 0

t1 := t1

t2 := t1 + t3 − t2 = t1 + t3 − t2

t3 := t2 + t5 − t4 = t1 + t3 + t5 − t2 − t4

...

tk := tk−1 + t2k−1 − t2k−2 = t1 +k−1∑i=1

(t2i+1 − t2i).

In addition, let us define

ψ(t, x) := ϕ(t − tk + t2k, x), tk ≤ t < tk+1 (2.10)

for k = 0, 1, 2, · · · . By construction, if ϕ(t, x), t ≥ 0 is the solution of the hybridsystem �, then ψ(t, x), t ≥ 0 is the corresponding solution of the impulsive system�ie. Due to the fact that O2 is an exponentially stable periodic orbit of �ie, there existpositive scalers N2 and γ2 such that

dist(ψ(t, x),O2) ≤ N2d exp(−γ2 t) (2.11)

for every t ≥ 0. Inequality (2.11) in combination with equation (2.10) results in

dist(ϕ(t, x),O2) ≤ N2d exp(−γ2(t + tk − t2k))

= N2d exp(−γ2t) exp

(γ2

k∑i=1

(t2i − t2i−1)

)

≤ N2d exp(−γ2t) exp(kγ2T1,max)

(2.12)

for any t ∈ [t2k, t2k+1), k = 0, 1, 2, · · · , where

T1,max(x) := supi≥1

(t2i − t2i−1)

is the supremum of all time durations of phase 1 corresponding to the trajectory ϕ(t, x).Furthermore, t2k ≥ kTmin for k = 0, 1, 2, · · · , where

Tmin(x) := infi≥1

(t2i − t2i−2)


is the infimum of all time durations corresponding to the steps of ϕ(t, x). Thus, forevery t ∈ [t2k, t2k+1), k ≤ t

Tminand consequently, t − kT1,max ≥ (1 − T1,max

Tmin)t. From

Lemma C.1 of Ref. [18, p. 439], hypothesis H1 implies that the functions T1 and T2are continuous on the sets �1

2(S12) and �2

1(S21), respectively. This fact in combination

with the following inequality which holds on the periodic orbit O,

T1,max = T ∗1 < T ∗ = Tmin

implies that there exists ε > 0 such that for any 0 < ε < ε and x ∈ Nε(O) ∩ X2,T1,max(x) < Tmin(x), where Nε(O) is an ε-neighborhood of O. Thus, for every 0 <

ε < ε and x ∈ Nε(O) ∩ X2, (2.12) can be expressed as

dist(ϕ(t, x),O2) ≤ N2d exp(−γ2(t − kT1,max))

≤ N2d exp(−γ2t)(2.13)

for all t ∈ [t2k, t2k+1), k = 0, 1, 2, · · · , where N2 := N2 and γ2 := γ2(1 − T1,maxTmin

).Next, let us define {x2[k]}∞k=1 and {x1[k]}∞k=1 by

x2[k] := ϕ(t2k−1, x) ∩ S12

x1[k] := ϕ(t2k, x) ∩ S21,

(2.14)

whereϕ is the set closure ofϕ. Since the hybrid system� isC1,f1 and�12 are Lipschitz

continuous with Lipschitz constants L1 and L12 on some convex subsets X1 ⊂ X1 and

S12 ⊂ S1

2, respectively such that O1 ⊂ X1 and x∗2 ∈ S1

2 [88, Lemma 3.1, p. 89]. Thus,using the standard results for continuous dependence on initial states of the solutionsof x1 = f1(x1) [88, Theorem 3.4, p. 96], for every t ∈ [t2k+1, t2k+2), k = 0, 1, 2, · · · ,

dist(ϕ(t, x),O1) ≤∥∥∥ϕ1

(t, �1

2(x2[k + 1]))

− ϕ1

(t, �1

2(x∗2)

)∥∥∥≤ ∥∥�1

2(x2[k + 1]) − �12(x∗

2)∥∥ exp(L1(t − t2k+1))

≤ L12

∥∥x2[k + 1] − x∗2

∥∥ exp(L1(t2k+2 − t2k+1))

≤ L12

∥∥x2[k + 1] − x∗2

∥∥ exp(L1T1,max).

(2.15)

Furthermore, since � is C1 and O2 is an exponentially stable periodic orbit of �ie

transversal to S, Theorem 2.2 implies that there exist scalers N2 > 0, 0 < γ2 < 1 andr > 0 such that for any x2[1] ∈ Br(x∗

2) ∩ S,

‖x2[k + 1] − x∗2‖ ≤ N2 ‖x2[1] − x∗

2‖ (γ2)k, k = 1, 2, · · · , (2.16)


where Br(x∗2) := {x2 ∈ Rn2 |‖x2 − x∗

2‖ ≤ r}. Next, let Tmax := T2,max + T1,max,where

T2,max := supi≥1

(t2i−1 − t2i−2)

is the supremum of all time durations of phase 2 corresponding to ϕ(t, x). Then,t2k+1 ≤ t < t2k+2 ≤ (k + 1)Tmax and consequently from inequality (2.16),

‖x2[k + 1] − x∗2‖ ≤ N2 ‖x2[1] − x∗

2‖ exp(

ln γ2

( t

Tmax− 1

)). (2.17)

This fact in combination with inequality (2.15) also results in

dist(ϕ(t, x),O1) ≤ N1d exp(−γ1t)

for all t ∈ [t2k+1, t2k+2), k = 0, 1, 2, · · · , where γ1 := − ln γ2Tmax

and N1 is such that

N1 ≥ ηN2 L1

2 exp(L1T1,max)

γ2,

where η > 0 is an arbitrary scalar. Choosing N = maxx∈N(O2)(N1, N2), γ =minx∈N(O2)(γ1, γ2) and N(O2) as the following compact set

N(O2) = {x ∈ X2| ‖x2[1] − x∗2‖ ≤ η dist(x,O2)}

= {x ∈ X2| ‖ϕ2(T2(x), x) − x∗2‖ ≤ η dist(x,O2)}

completes the proof. �

Theorems 2.2 and 2.3 establish an analytical approach to investigate the stabilitybehavior of the transversal periodic orbit O. However, this approach is particularlyuseful when we have a closed-form expression for the Poincare return map P . Forexample, it is often convenient to check the exponential stability in terms of eigenval-ues of the Jacobian matrix DP(x∗

2). Under hypothesis H1, if the hybrid system � isC1, the Poincare return map is continuously differentiable. To obtain a closed-formexpression for the Jacobian matrix DP(x∗

2), define

�1(t, x) := Dxϕ1(t, x)

�2(t, x) := Dxϕ2(t, x)

as the trajectory sensitivity matrices. From Ref. [89, p. 316], since the periodic orbitO is transversal to S2

1 and S12, the functions T1 and T2 are differentiable at the points

LOW-DIMENSIONAL STABILITY ANALYSIS 23

�12(x∗

2) and �21(x∗

1), respectively, and

DT1(�1

2(x∗2)

) = −1

Lf1H21 (x∗

1)

∂H21

∂x1(x∗

1) �1(T ∗

1 , �12(x∗

2))

DT2(�2

1(x∗1)

) = −1

Lf2H12 (x∗

2)

∂H12

∂x2(x∗

2) �2(T ∗

2 , �21(x∗

1)),

(2.18)

which in turn result in

DP(x∗2) =

⎡⎣In2×n2 − f2(x∗

2)∂H1

2∂x2

(x∗2)

Lf2H12 (x∗

2)

⎤⎦ �2

(T ∗

2 , �21(x∗

1)) ∂�

∂x2(x∗

2), (2.19)

where

∂�

∂x2(x∗

2) = ∂�21

∂x1(x∗

1)

⎡⎣In1×n1 − f1(x∗

1)∂H2

1∂x1

(x∗1)

Lf1H21 (x∗

1)

⎤⎦ �1

(T ∗

1 , �12(x∗

2)) ∂�1

2

∂x2(x∗

2).

We observe that the domain of the Jacobian matrix is the (n2 − 1)-dimensional tangentspace Tx∗

2S. Therefore, the proper notation for the Jacobian matrix is DP |Tx∗

2S(x∗

2).

However, for clarity, we will not use it. From equation (2.19), we require thetrajectory sensitivity matrices �1(T ∗

1 , �12(x∗

2)) and �2(T ∗2 , �2

1(x∗1)) to compute

the Jacobian DP(x∗2). To obtain the trajectory sensitivity matrix �i(T ∗

i , �ij(x

∗j )),

i, j = 1, 2, j /= i, the well-known variational equation [89, p. 305] is appended tothe original differential equation during the continuous phase, that is,

[xi

�i

]=

[fi(xi)

Dxifi(xi)�i

]

which is integrated over the time interval [0, T ∗i ] with the initial condition

[�i

j(x∗j )

Ini×ni

].

Thus, the computations required to check the stability are complex. This situationmotivates us to look for ways to simplify the stability analysis.

2.3 LOW-DIMENSIONAL STABILITY ANALYSIS

Theorems 2.2 and 2.3 present a natural and analytical approach to investigate thestability behavior of the transversal periodic orbitO for the hybrid system �. As men-tioned in the previous section, when the Poincare return map can be written down in a


closed-form expression, this approach is useful. However, determining the Poincarereturn map requires the solutions of the differential equations xi = fi(xi), i = 1, 2which in general can only be computed by applying numerical integration algo-rithms. Therefore, in the general case, the Poincare return map cannot be expressedin a closed-form expression. In order to simplify the stability analysis, this sectionpresents special circumstances where the stability behavior of the periodic orbitO canbe tested by low-dimensional tools such as the restricted Poincare return maps. Themain ideas and results in developing the notion of restricted Poincare return maps,which are employed for inducing asymptotically stable periodic solutions in walkingand running of the biped robots, are due to Grizzle et al. [18, 46, 52]. We first presentthe following definitions.

Definition 2.6 (Hybrid Invariance for Impulsive Systems) The set Z2 ⊂ X2 issaid to be hybrid invariant for the system with impulse effects �ie(X2,S, �, f2) if

1. Z2 is forward invariant under the flow of the differential equation x2 = f2(x2),that is, for every x2 ∈ Z2, there exists tf (x2) > 0 such that ϕ2(t, x2) ∈ Z2 fort ∈ [0, tf ); and

2. Z2 is impact invariant, that is, S ∩ Z2 /= φ and �(S ∩ Z2) ⊂ Z2.

Definition 2.7 (Hybrid Invariance for Hybrid Systems) The sets Z1 ⊂ X1 andZ2 ⊂ X2 are said to be hybrid invariant for the hybrid system

�(X1,X2,S2

1,S12, �

21, �

12, f1, f2

)

if

1. for every i ∈ {1, 2}, Zi is forward invariant under the flow of the differentialequation xi = fi(xi) and

2. for every i, j ∈ {1, 2} and i /= j, Sji ∩ Zi /= φ and �

ji (S

ji ∩ Zi) ⊂ Zj .

Definition 2.8 (Finite-Time Attractiveness) For i ∈ {1, 2}, the settling time to theset Zi, TZi : Xi → R ∪ {∞}, is defined to be the infimum of all times for which thetrajectory ϕi(t, xi) arrives at the set Zi and remain there until the maximal time ofexistence. Note that if there is not such a time, TZi (xi) is defined to be ∞. Moreover,the set Zi is said to be locally continuously finite-time attractive if (i) Zi is forwardinvariant under the flow of the differential equation xi = fi(xi) and (ii) there existsan open set Ni containing Zi such that for every xi ∈ Ni, the function TZi at xi iswell-defined (i.e., finite) and continuous [18, p. 96].

Now assume that the sets Zi ∈ Xi, i = 1, 2 are embedded submanifolds of the statespaces Xi, i = 1, 2 satisfying the following hypotheses:


(H2) For every i, j ∈ {1, 2} and i /= j, Sji ∩ Zi is an embedded submanifold of Xi

such that dim(Sji ∩ Zi) = dim(Zi) − 1;

(H3) For every i ∈ {1, 2}, Zi is locally continuously finite-time attractive;

(H4) Z1 and Z2 are hybrid invariant for the autonomous hybrid system �; and

(H5) For every i ∈ {1, 2}, Oi ⊂ Zi, i.e., Oi is an integral curve of the restrictiondynamics xi = fi|Zi (xi).

Hypothesis H3 implies finite-time convergence to the manifolds Z1 and Z2. Due tothe fact that finite-time convergence implies nonuniqueness of solutions in reversetime, if the set Zi, i = 1, 2 is finite-time attractive, it is not possible that the vectorfield fi : Xi → TXi, i = 1, 2 is Lipschitz continuous. By hypothesis H4, solutions ofthe hybrid system � initialized in Z := Z1 ∪ Z2 remain in Z until the maximal timeof existence. Therefore, we can construct the following reduced-order hybrid modelto study the stability behavior of the transversal periodic orbit O

�1|Z1 :

⎧⎪⎪⎪⎨⎪⎪⎪⎩

Z1 ⊂ X1

F1|Z1 : z1 = f1|Z1 (z1)

S21 ∩ Z1 = {

z1 ∈ Z1 | H21 |Z1 (z1) = 0

}T2

1|Z1 : z+2 = δ2

1(z−1 )

�2|Z2 :

⎧⎪⎪⎪⎨⎪⎪⎪⎩

Z2 ⊂ X2

F2|Z2 : z2 = f2|Z2 (z2)

S12 ∩ Z2 = {

z2 ∈ Z2 | H12 |Z2 (z2) = 0

}T1

2|Z2 : z+1 = δ1

2(z−2 ),

(2.20)

where for i, j ∈ {1, 2} and i /= j, fi|Zi and Hji |Zi denote the restrictions of fi and

Hji to the manifold Zi, respectively. Also, Fi|Zi represents the flow of the restric-

tion dynamics zi = fi|Zi (zi). Moreover, Tji |Zi denotes the restricted switching maps

δji : Sj

i ∩ Zi → Zj defined by the continuously differentiable laws δji := �

ji |Sj

i∩Zi

.

For simplicity, the reduced-order hybrid model (2.20), which is called the hybridrestriction dynamics, is denoted by the 8-tuple

�|Z(Z1, Z2,S21 ∩ Z1,S1

2 ∩ Z2, δ21, δ

12, f1|Z1 , f2|Z2 ).

If the hybrid system � satisfies hypothesis H1, so does the hybrid restriction dynamics�|Z, that is, the vector fields fi|Zi : Zi → TZi, i = 1, 2 are continuous. Also, the so-lutions of the restriction dynamics zi = fi|Zi (zi), i = 1, 2 for every initial condition inthe state spaceZi, i = 1, 2 exist and are unique. Moreover, these solutions depend con-tinuously on the initial conditions. Thus, for every initial condition in the state space Z,there exists a unique solution of the hybrid restriction dynamics �|Z. From hypothesis


H2, for every i, j ∈ {1, 2} and i /= j, Sji ∩ Zi is an embedded submanifold of Zi with

dimension one less than the dimension ofZi. In addition,Sij ∩ δ

ji (S

ji ∩ Zi) = φ, which

in turn implies that a switching does not occur immediately after another switching.Figure 2.2 illustrates the geometry of the hybrid restriction dynamics �|Z.

The solution of �|Z initialized from z ∈ Z can be expressed as ϕ|Z(t, z), whereϕ(t, z) represents the corresponding solution of �. Thus, the restricted Poincare returnmap for �|Z can be defined as ρ : S1

2 ∩ Z2 → S12 ∩ Z2 by

ρ(z2) := ρ2 ◦ ρ1(z2), (2.21)

where ρ2 : S21 ∩ Z1 → S1

2 ∩ Z2 and ρ1 : S12 ∩ Z2 → S2

1 ∩ Z1 are given by

ρ2(z1) := ϕ2|Z2

(T2|Z2 ◦ δ2

1(z1), δ21(z1)

)ρ1(z2) := ϕ1|Z1

(T1|Z1 ◦ δ1

2(z2), δ12(z2)

).

(2.22)

By the construction of �|Z, ρ(z2) = P |Z(z2), where P |Z is the restriction of thePoincare return map of the full-dimensional hybrid system � to Z. ApplyingTheorem 2.1 implies that ρ is also the Poincare return map for the reduced-ordersystem with impulse effects �ie|Z2 (Z2,S ∩ Z2, δ, f2|Z2 ), where δ(z2) := δ2

1 ◦ ρ1(z2).By hypothesis H5 and the construction procedure, the transversal periodic orbit Oof the hybrid system � is the periodic orbit of the hybrid restriction dynamics �|Zwhich is also transversal to S2

1 ∩ Z1 and S12 ∩ Z2 (see Fig. 2.2). Now we are in a

position to present the fundamental theorem of this section. This theorem estab-lishes an equivalence between the stability analysis of the transversal periodic orbit

Figure 2.2 Geometric description of the hybrid restriction dynamics. By hypotheses H2–H5and the construction of the hybrid restriction dynamics, the transversal periodic orbit O of thehybrid system � is also the periodic orbit of the hybrid restriction dynamics �|Z which istransversal to S2

1 ∩ Z1 and S12 ∩ Z2. (See the color version of this figure in color plates section.)


Figure 2.3 Geometric description of the restricted Poincare return map ρ : S12 ∩ Z2 →

S12 ∩ Z2. By hypotheses H2–H5 and the construction of �|Z, ρ(z2) = P |Z(z2), where P |Z

is the restriction of the Poincare return map of the full-dimensional hybrid system � to Z. Byapplying Theorem 2.1, it follows that ρ is also the Poincare return map for the reduced-ordersystem with impulse effects �ie|Z2 (Z2,S ∩ Z2, δ, f2|Z2 ), where δ(z2) := δ2

1 ◦ ρ1(z2). (See thecolor version of this figure in color plates section.)

O of the full-dimensional hybrid system � satisfying hypotheses H1–H5 and thestability analysis of the equilibrium point x∗

2 of the reduced-order discrete-time sys-tem z2[k + 1] = ρ(z2[k]) with the state space S ∩ Z2 (see Fig. 2.3). Consequently,the stability behavior of the periodic orbit of the full-order hybrid model can bedetermined by low-dimensional tools which require less computation.

Theorem 2.4 (Low-Dimensional Stability Analysis) [18, p. 99]2 Assume that thefull-dimensional autonomous hybrid system �(X1,X2,S2

1,S12, �

21, �

12, f1 f2) satis-

fies hypothesis H1. Furthermore, suppose that there exist embedded submanifoldsZi, i = 1, 2 of the state spaces Xi, i = 1, 2 satisfying hypotheses H2–H5. Then, thefollowing statements are true:

2 This theorem is a restatement of Theorem 4.5 of Ref. [18, p. 99].


1. x∗2 is a stable (asymptotically stable) equilibrium point of z2[k + 1] = ρ(z2[k])

if and only if the orbitO2 is stable (asymptotically stable) for the correspondingfull-dimensional impulsive system �ie(X2,S, �, f2).

2. If both fi|Zi, i = 1, 2 are also continuously differentiable, then x∗2 is an ex-

ponentially stable equilibrium point of z2[k + 1] = ρ(z2[k]) if and only if theorbit O2 is exponentially stable for the corresponding full-dimensional impul-sive system �ie(X2,S, �, f2).

Theorem 2.4 in combination with Theorem 2.3 immediately implies the followingresult.

Corollary 2.1 Assume that the hybrid system �(X1,X2,S21,S1

2, �21, �

12, f1, f2)

satisfies hypothesis H1. Furthermore, suppose that there exist embedded submani-folds Zi, i = 1, 2 of the state spaces Xi, i = 1, 2 satisfying hypotheses H2–H5. Then,the following statements are true:

1. x∗2 is a stable (asymptotically stable) equilibrium point of z2[k + 1] = ρ(z2[k])

if and only if the orbitO is stable (asymptotically stable) for the full-dimensionalhybrid system �.

2. If both fi|Zi, i = 1, 2 are also continuously differentiable, then x∗2 is an

exponentially stable equilibrium point of z2[k + 1] = ρ(z2[k]) if and only ifthe orbit O is exponentially stable for the full-dimensional hybrid system �.

2.4 STABILIZATION PROBLEM

Consider the open-loop hybrid system �ol taking the following form

�1,ol :

⎧⎪⎪⎪⎨⎪⎪⎪⎩

X1 ⊂ Rn1

F1 : x1 = f1(x1) + g1(x1) u

S21 = {

x1 ∈ X1 | H21 (x1) = 0

}T2

1 : x+2 = �2

1(x−1 )

�2,ol :

⎧⎪⎪⎪⎨⎪⎪⎪⎩

X2 ⊂ Rn2

F2 : x2 = f2(x2) + g2(x2) u

S12 = {

x2 ∈ X2 | H12 (x2) = 0

}T1

2 : x+1 = �1

2(x−2 ),

(2.23)

where u ∈ U is the control input vector. Moreover, U ⊂ Rm called the admissiblecontrol input region is defined to be the set of all piecewise continuous functions

STABILIZATION PROBLEM 29

t �→ u(t) with the property ‖u‖L∞ := supt≥0 ‖u(t)‖ < umax, where umax is a positivescalar. Suppose that O = O1 ∪ O2 is an orbit corresponding to a period-one solutionof the open-loop hybrid system �ol. This section addresses the problem of asymptoticstabilization of O for the system �ol. The main idea of this section has been takenfrom Ref. [57–59].

To asymptotically stabilize the orbit O for �ol, a two-level control scheme ispresented. At the first level of the control scheme, parameterized and time-invariantcontinuous feedback laws are employed during continuous phases i = 1, 2 to createa family of parameterized, finite-time attractive and forward invariant manifolds onwhich the differential equation xi = fi(xi) + gi(xi)u is restricted. As mentioned inSection 2.3, this will reduce the complexity of the calculations required for obtainingthe Poincare return map. Let Z1,α and Z2,β represent the parameterized manifoldscreated in the phases 1 and 2, respectively. Also, α and β denote the parameters ofthe controllers during phases 1 and 2 which takes values in the open sets A and B.To show explicitly the dependence on the parameters α and β, the time-invariantfeedback laws in phases 1 and 2 are denoted by u1(x1; α) and u2(x2; β), respectively.With these control laws, the closed-loop dynamics of phases 1 and 2 can be given byx1 = f1,cl(x1; α) and x2 = f2,cl(x2; β), where

f1,cl(x1; α) := f1(x1) + g1(x1) u1(x1; α)

f2,cl(x2; β) := f2(x2) + g2(x2) u2(x2; β).

Now assume that the following hypotheses are satisfied:

(H6) For every α ∈ A and β ∈ B,

(a) the sets S21 ∩ Z1,α and S1

2 ∩ Z2,β are independent of α and β, respec-tively. The common intersections are also denoted by S2

1 ∩ Z1 andS1

2 ∩ Z2. Furthermore, S21 ∩ Z1 and S1

2 ∩ Z2 are embedded submani-folds of X1 and X2 with the properties dim(S2

1 ∩ Z1) = dim(Z1,α) − 1and dim(S1

2 ∩ Z2) = dim(Z2,β) − 1;

(b) �21(S2

1 ∩ Z1) ⊂ Z2,β and �12(S1

2 ∩ Z2) ⊂ Z1,α;

(c) Z1,α and Z2,β are locally continuously finite-time attractive for theclosed-loop dynamics x1 = f1,cl(x1; α) and x2 = f2,cl(x2; β), respec-tively; and

(H7) There existα∗ ∈ A andβ∗ ∈ B such thatO1 ⊂ Z1,α∗ andO2 ⊂ Z2,β∗ , that is,O1 andO2 are integral curves of the differential equations x1 = f1,cl(x1; α∗)and x2 = f2,cl(x2; β∗), respectively.

Hypothesis H6 motivates us to define the parameterized restricted Poincare returnmap for the closed-loop hybrid system as ρα,β : S1

2 ∩ Z2 → S12 ∩ Z2 by ρα,β(z2) :=

ρ2,β ◦ ρ1,α(z2), where ρ1,α : S12 ∩ Z2 → S2

1 ∩ Z1 and ρ2,β : S21 ∩ Z1 → S1

2 ∩ Z2 arethe parameterized versions of the maps defined in Section 2.3. Thus, to study the


stabilization problem, we can define the following discrete-time system

z2[k + 1] = ρ(z2[k]; α[k], β[k]), (2.24)

with the state spaceS12 ∩ Z2 and the control inputs α[k] and β[k], where ρ(z2; α, β) :=

ρα,β(z2). Let us continue the problem of stabilizing the periodic orbit O for the open-loop hybrid system (2.23). To do this, assume that there exist continuous functionsαcl : S1

2 ∩ Z2 → A and βcl : S12 ∩ Z2 → B such that αcl(x∗

2) = α∗ and βcl(x∗2) = β∗.

Moreover, suppose that the equilibrium pointx∗2 is asymptotically stable for the closed-

loop discrete-time system

z2[k + 1] = ρcl(z2[k]), (2.25)

where ρcl(z2) := ρ(z2; αcl(z2), βcl(z2)). Then, at the second level of the controlscheme, the parameters of the feedback laws of phases 1 and 2 can be updated atthe end of phase 2 by an event-based update law3 in a step-to-step fashion. To makethis notion more precise, the parameters α and β for the next step are updated by thefollowing static laws

α[k + 1] = αcl(x2[k]), k = 1, 2, · · ·β[k + 1] = βcl(x2[k]), k = 1, 2, · · · ,

where k denotes the step number and x2[k] was defined in equation (2.14). We ob-serve that the parameters α and β are held constant during continuous phases, andconsequently, the two-level control strategy will result in the following closed-loophybrid system (see Fig. 2.4)

�α1,cl :

⎧⎪⎪⎪⎨⎪⎪⎪⎩

X1 ⊂ Rn1

F1 : x1 = f1,cl(x1; α)

S21 = {

x1 ∈ X1 | H21 (x1) = 0

}T2

1 : x+2 = �2

1(x−1 )

�β2,cl :

⎧⎪⎪⎪⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎪⎪⎪⎩

X2 ⊂ Rn2

F2 : x2 = f2,cl(x2; β)

S12 = {

x2 ∈ X2 | H12 (x2) = 0

}

T12 :

⎡⎢⎣

x+1

α+

β+

⎤⎥⎦ =

⎡⎢⎣

�12(x−

2 )

αcl(x−2 )

βcl(x−2 )

⎤⎥⎦ .

(2.26)

3 The terminology of an event-based update law is taken from Ref. [18, p. 199].


Figure 2.4 Geometry of the closed-loop hybrid system (2.26) which is achieved by employ-ing the two-level control strategy. At the first level, the parameterized, time-invariant, andcontinuous feedback laws u1(x1; α) and u2(x2; β) are employed to create the parameterized,finite-time attractive, and invariant manifolds Z1,α and Z2,β during phases 1 and 2, respec-tively. This will result in the restriction dynamics of phases 1 and 2 as x1 = f1,cl|Z1,α

(x1; α)and x2 = f2,cl|Z2,β

(x2; β). At the second level of the control strategy, the parameters of thefeedback laws in phases 1 and 2 are updated by an event-based update law at the end of phase 2(i.e., in a step-to-step fashion). In particular, α[k + 1] = αcl(x2[k]) and β[k + 1] = βcl(x2[k]),where k denotes the step number. Consequently, the restricted Poincare return map re-places the flow of the closed-loop hybrid system with the closed-loop discrete-time systemx2[k + 1] = ρcl(x2[k]). The graphs depict the three steps of the closed-loop hybrid system. Itis assumed that the state of the system is initiated at a point off the manifold Z1,α[1]. Due to thefact that Z1,α is finite-time attractive, the state of the system enters onto Z1,α in a finite-time andremains in it throughout phase 1. When the state enters S2

1 (at x1[1]), a discrete event occurs,according to the transition map �2

1. Hypothesis H6 implies that �21(S2

1 ∩ Z1,α[k]) ⊂ Z2,β[k] andconsequently, during phase 2, the state of the system evolves in Z2,β[1] until it enters S1

2 (atx2[1]). At this point, a discrete event occurs, according to the transition map �1

2. Moreover,event-based update law updates the parameters α and β for the next step as α[2] = αcl(x2[1]) andβ[2] = βcl(x2[1]). These parameters are held constant during the second step and the processrepeats. We observe that by hypothesis H6, �1

2(S12 ∩ Z2,β[k]) ⊂ Z1,α[k+1].

The following theorem is a precise statement concerning the equivalence between thestability behavior of the periodic orbit O for the closed-loop hybrid system (2.26) andthe stability behavior of the equilibrium point x∗

2 for the closed-loop discrete-timesystem (2.25).

Theorem 2.5 (Stabilization Policy) Assume that the time-invariant continuousfeedback laws u1 : X1 × A → U and u2 : X2 × B → U are chosen such thatthe families of the manifolds {Z1,α|α ∈ A} and {Z2,β|β ∈ B} satisfy hypothesesH6–H7. Furthermore, suppose that the static update laws αcl : S1

2 ∩ Z2 → A and


βcl : S12 ∩ Z2 → B are continuous maps and the following additional hypothesis is

satisfied:

(H8) For every α ∈ A and β ∈ B, the vector fields f1,cl : X1 → TX1 and f2,cl :X2 → TX2 are continuous. In addition, the solutions of the augmented dif-ferential equations

[x1

α

]=

[f1,cl(x1, α)

0

][x2

β

]=

[f2,cl(x2, β)

0

]

for every initial conditions in X1 × A and X2 × B are unique and dependcontinuously on the initial conditions.

Then, the following statements are true:

1. x∗2 is a stable (asymptotically stable) equilibrium point of z2[k + 1] = ρcl(z2[k])

if and only if the orbit O is stable (asymptotically stable) for the closed-loophybrid system (2.26).

2. If the static update laws αcl : S12 ∩ Z2 → A and βcl : S1

2 ∩ Z2 → B andthe restricted vector fields f1,cl|Z1,α

: Z1,α → TZ1,α and f2,cl|Z2,β: Z2,β →

TZ2,β are continuously differentiable for every α ∈ A and β ∈ B, thenx∗

2 is an exponentially stable equilibrium point of z2[k + 1] = ρcl(z2[k]) ifand only if the orbit O is exponentially stable for the closed-loop hybridsystem (2.26).

Proof. Define the augmented state spaces and switching manifolds as X1,a := X1 ×A × B, X2,a := X2 × A × B, S2

1,a := S21 × A × B, and S1

2,a := S12 × A × B. The

vector fields and switching maps corresponding to the augmented state spaces andswitching manifolds can also be defined in the following forms

f1,a(x1, α, β) :=

⎡⎢⎣

f1,cl(x1; α)

0

0

⎤⎥⎦ f2,a(x2, α, β) :=

⎡⎢⎣

f2,cl(x2; β)

0

0

⎤⎥⎦

�21,a(x1, α, β) :=

⎡⎢⎣

�21(x1)

α

β

⎤⎥⎦ �1

2,a(x2, α, β) :=

⎡⎢⎣

�12(x2)

αcl(x2)

βcl(x2)

⎤⎥⎦ .


Next consider the following augmented hybrid system

�1,a :

⎧⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎩

X1,a ⊂ Rn1 × A × B

F1,a :

⎡⎢⎣

x1

α

β

⎤⎥⎦ = f1,a(x1, α, β)

S21,a = {

(x1, α, β) ∈ X1,a | H21 (x1) = 0

}

T21,a :

⎡⎢⎣

x+2

α+

β+

⎤⎥⎦ = �2

1,a(x−1 , α−, β−)

�2,a :

⎧⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎩

X2,a ⊂ Rn2 × A × B

F2,a :

⎡⎢⎣

x2

α

β

⎤⎥⎦ = f2,a(x2, α, β)

S12,a = {

(x2, α, β) ∈ X2,a | H12 (x2) = 0

}

T12,a :

⎡⎢⎣

x+1

α+

β+

⎤⎥⎦ = �1

2,a(x−2 , α−, β−).

(2.27)

From the hypotheses of Theorem 2.5 and by introducing the augmented manifolds

Z1,a := {(x1, α, β) ∈ X1,a|x1 ∈ Z1,α}Z2,a := {(x2, α, β) ∈ X2,a|x2 ∈ Z2,β},

all of the hypotheses of Corollary 2.1 are satisfied, and consequently the stabilitybehavior of the augmented periodic orbit Oa := O × {α∗} × {β∗} for the augmentedhybrid model (2.27) can be studied based on the stability analysis of the equilibriumpoint (x∗

2, α∗, β∗) for the following discrete-time system

z2[k + 1] = ρcl(z2[k])

α[k + 1] = αcl(x2[k])

β[k + 1] = βcl(x2[k]).

(2.28)

In addition, the stability properties of the equilibrium point (x∗2, α

∗, β∗) for the system(2.28) are equivalent to those of the equilibrium point x∗

2 for the discrete-time systemz2[k + 1] = ρcl(z2[k]). This fact in combination with the equivalence between thestability properties of Oa for the augmented hybrid system (2.27) and O for theclosed-loop system (2.26) completes the proof. �

CHAPTER 3

Asymptotic Stabilization of PeriodicOrbits for Walking with DoubleSupport Phase

3.1 INTRODUCTION

The objective of this chapter is to develop an analytical approach for designing acontinuous feedback law that realizes a desired period-one trajectory as an asymp-totically stable orbit for a planar biped robot. The robot is assumed to be a five-link,four-actuator planar mechanism in the sagittal plane with point feet. The fundamen-tal assumption is that the double support phase is not instantaneous. Hence, bipedalwalking can be represented by a hybrid model with two continuous phases, includinga single support phase (one leg on the ground) and a double support phase (two legson the ground), and discrete transitions between the continuous phases. In the singlesupport phase, the mechanical system has one degree of underactuation, whereas itis overactuated in the double support phase.

Recently, the method of virtual constraints has been used to design time-invariantfeedback laws for bipedal locomotion with one degree of underactuation [46–52].Virtual constraints are a set of holonomic outputs in the configuration space of themechanical system that coordinate the links of biped robots during walking [47]. Inthe case that the zero dynamics manifold corresponding to the virtual constraints isinvariant under the impact map of walking, the notion of hybrid zero dynamics (HZD)was introduced in Ref. [52]. Moreover, a constructive method, based on parameteri-zation of the virtual constraints and updating their parameters in a stride-to-stride ba-sis, was presented in Ref. [60] for creating an augmented HZD during bipedal walkingwith more than one degree of underactuation. This method was used in Refs. [61, 62]to induce asymptotically stable walking by an underactuated spatial biped robot. Also,


35

36 WALKING WITH DOUBLE SUPPORT PHASE

in [55], parameterized virtual constraints have been utilized to create an HZD duringrunning of a planar biped robot.

There has been little attention given to control of biped robots during the doublesupport phase with unilateral constraints, which are constraints on the state and controlinputs of the mechanical system that represent feasible contact conditions betweenthe leg ends and the ground. Such constraints present challenges for the design ofcontrollers. Moreover, due to overactuation, the control input corresponding to aspecific trajectory in the state space is not unique.

In this chapter, we show how to design a continuous time-invariant feedback lawthat asymptotically stabilizes a feasible periodic trajectory using an extension ofHZD for a hybrid model of walking [68, 69]. The main contribution is to developa continuous time-invariant control law for walking of a planar biped robot duringthe double support phase. Since the mechanical system in the double support phasehas three DOF and four actuators, a constrained dynamics approach [70, p. 157]is used to describe the reduced-order dynamics of the system. Then, we proposetwo virtual constraints as holonomic outputs for the constrained system and solvean output zeroing problem with two control inputs. This results in a nontrivial two-dimensional zero dynamics manifold corresponding to the virtual constraints in thestate manifold of the constrained system. Moreover, the corresponding zero dynamicshas two control inputs that are not employed for output zeroing. Instead, they are usedto satisfy the unilateral constraints. Furthermore, these inputs are obtained such thatthe control has minimum norm on the desired periodic trajectory. It can be shownthat the constrained dynamics of the double support phase is completely feedbacklinearizable on an open subset of the state manifold. However, since our objectiveis to design a continuous time-invariant controller based on HZD, in contrast toRef. [71], we do not use input-state linearization nor a discontinuous time optimalcontrol for tracking trajectories. An analogous approach is used in Refs. [54, 72]for creating a two-dimensional zero dynamics manifold in the state space of a fullyactuated phase of walking where the fully actuated dynamics is completely feedbacklinearizable.

We present the control strategy at the following two levels. At the first level,we employ within-stride controllers including single and double support phase con-trollers. These are continuous time-invariant and parameterized feedback laws thatcreate a family of two-dimensional finite-time attractive and invariant submanifoldson which the dynamics of the mechanical system is restricted. At the second level,the parameters of the within-stride controllers are updated at the end of the singlesupport phase (in a stride-to-stride manner) by an event-based update law to achievehybrid invariance and stabilization. Hybrid invariance yields a reduced-order hy-brid model for walking that is referred to as HZD. As a consequence, the stabi-lity properties of the desired periodic orbit can be analyzed using low-dimensionaltools developed for systems with impulse effects, such as restricted Poincare re-turn maps (see Section 2.3). The idea of updating parameters of the within-stridecontroller in an event-based manner to achieve stabilization has been described inSection 2.4.

MECHANICAL MODEL OF A BIPED WALKER 37

1q

2q

3q

4q

5q

s1 1,x y

2 2,x y

,H Hx yx

y

Figure 3.1 Three different phases of the biped walker motion: flight phase (left), singlesupport phase (middle), and double support phase (right). In the single support phase, thevirtual leg is depicted by the dashed line. All of the angles increase in the clockwise direction,whereas the absolute angle decreases.

3.2 MECHANICAL MODEL OF A BIPED WALKER

3.2.1 The Biped Robot

Throughout this chapter, we consider a planar biped robot, composed of five rigidlinks with distributed masses, walking on a flat surface. The links are connected byfour revolute body joints: two knee joints and two hip joints. This robot has alsopoint feet (see Fig. 3.1). There is an actuator located at each knee and hip joint. It isassumed that the robot cannot apply torques at the end of its legs. We also assumethat a coordinate frame is attached to the flat ground called the world frame.

3.2.2 Dynamics of the Flight Phase

For representing the configuration of the walking robot, a convenient choice of theconfiguration variables consists of the body angles, the absolute orientation, andthe absolute position of the robot with respect to the world frame. Unless other-wise stated, we follow the notation of Ref. [55]. The body angles are the relativeangles qb := (q1, q2, q3, q4)′ describing the shape of the biped robot. The abso-lute orientation of the biped robot is represented by q5, whereas the absolute po-sition is represented by the Cartesian coordinates of its COM, pcm := (xcm, ycm)′.Hence, the generalized coordinates for the flight phase (no leg is in contact withthe ground) can be defined as qf := (q′, p′

cm)′, where q := (q′b, q5)′ and prime repre-sents matrix transpose. We remark that q5 andpcm are unactuated variables during theflight phase.

Following the notation of Ref. [55], the dynamical model during the flight phasecan be expressed as

Df (qb) qf + Cf (qb, qf ) qf +Gf (qf ) = Bf u, (3.1)


where Df (qb) is a (7 × 7) positive-definite mass-inertia matrix with the followingform:

Df (qb) :=[A(qb) 05×2

02×5 mtot I2×2

](3.2)

and mtot represents the total mass of the biped walker. Cf is a (7 × 7) matrix con-taining the Coriolis and centrifugal terms, Gf is a (7 × 1) gravity vector, u :=(u1, u2, u3, u4)′ is a vector of actuator torques, and Bf := [I4×4 04×3]′. Using theblock diagonal form of the mass-inertia matrix in equation (3.2) and equations (7.60)and (7.62) of Ref. [90, p. 256], Cf can be expressed as1

Cf (qb, qf ) =[C(qb, q) 05×2

02×5 02×2

]. (3.3)

Moreover, Gf can be given by

Gf (q) := mtotg0[

0 0 0 0 0 0 1]′, (3.4)

in which g0 represents the gravitational constant. Consequently, the dynamical equa-tion of the flight phase (3.1) can be decomposed as follows:

A(qb) q+ C(qb, q) q = Bu (3.5)

mtot pcm +mtot

[0

g0

]= 02×1, (3.6)

where B := [I4×4 04×1]′.

Remark 3.1 (Cyclic Variables of the Flight Phase) During the flight phase, q5

and pcm are the cyclic variables [1, 53] in the sense that∂Kf∂q5

= ∂Kf∂xcm

= ∂Kf∂ycm

= 0,

where Kf (qf , qf ) := 12 q

′fDf (qb)qf denotes the total kinetic energy of the mechani-

cal system. Thus, the matricesDf andCf in equations (3.2) and (3.3) are independentof q5 and pcm.

Remark 3.2 (Conservation of Angular Momentum in the Flight Phase) FromProposition B.9 of Ref. [18, p. 427], the last row of equation (3.5) can be expressed asσcm = 0, in which σcm := A5(qb)q denotes the angular momentum of the mechanicalsystem about the COM and A5 is the last row of the matrix A.

1 From equation (7.64) of Ref. [90, p. 257], the (k, j)th element of the matrixCf (qb, qf ) can be expressedas

Cf,kj =∑7

i=112

(∂Df,kj∂qf,i

+ ∂Df,ki∂qf,j

− ∂Df,ij∂qf,k

)qf,i.


3.2.3 Dynamics of the Single Support Phase

During the single support phase, the contacting leg is called the stance leg and theother is called the swing leg. By definition, the virtual leg is defined by the lineconnecting the end of the stance leg and the hip joints [61] (see Fig. 3.1). Let (x1, y1)′ =p1(qf ) := pcm − f1(q) ∈ R2 and (x2, y2)′ = p2(qf ) := pcm − f2(q) ∈ R2 representthe Cartesian coordinates of the end of leg-1 and leg-2, respectively. Also, f1 and f2are two smooth functions of the configuration variables. Since both legs of the robotare identical, without loss of generality, let leg-1 be the stance leg. In addition, assumethat the end of leg-1 is on the origin of the world frame, that is, pcm = f1(q). Then,using Remark B.14 of Ref. [18, p. 433], a reduced-order model for describing theevolution of the mechanical system during the single support phase can be expressedas2

D(qb) q+ C(qb, q) q+G(q) = Bu, (3.7)

where

D(qb) := A+mtot∂f1

∂q

′ ∂f1

∂q

C(qb, q) := C +mtot∂f1

∂q

′ ∂∂q

(∂f1

∂qq

)

G(q) := mtot∂f1

∂q

′ [ 0

g0

].

By defining xs := (q′s, q′s)′ as the state vector during the single support phase, whereqs := q and qs := q, equation (3.7) can be expressed in the state space form xs =fs(xs) + gs(xs)u. Moreover, during single support, the state manifold is assumed tobe3

Xs := TQs := {xs := (q′s, q′s)

′|qs ∈ Qs, qs ∈ R5},

where Qs denotes the configuration space of the single support phase and it is chosenas a simply connected and open subset of [(0, π2 ) × (0, 3π

2 )]2 × (−π2 ,π2 ).

2 Equation (3.7) can also be obtained by applying the principle of virtual work in equation (3.5), similarto that presented in proof of Theorem 3.1 during the double support phase.

3 In our notation, the tangent bundle of the manifold M is denoted by TM := ∪x∈MTxM, in whichTxM is the tangent space at the point x ∈ M.


Remark 3.3 (Validity of the Single Support Phase Model) The single support

phase model is valid when Fv1 > 0 and |Fh1Fv1

| < μs, where

F1 :=[Fh1

Fv1

]

=mtot∂f1

∂qq+mtot

∂

∂q

(∂f1

∂qq

)q+mtot

[0

g0

]

=mtot∂f1

∂qD−1(Bu− Cq−G) +mtot

∂

∂q

(∂f1

∂qq

)q+mtot

[0

g0

](3.8)

and μs denotes the ground reaction force at the end of leg-1 and the static frictioncoefficient, respectively [52]. Also q5 is the cyclic variable during single support, thatis, ∂Ks

∂q5= 0, in which Ks(qs, qs) := 1

2 q′sD(qb)qs denotes the total kinetic energy of the

mechanical system in the single support phase.

Remark 3.4 (Angular Momentum Balance in Single Support) From PropositionB.9 of Ref. [18, p. 427], the last row of equation (3.7) can be expressed as4

σs = −G5(q) = −mtot g0 fh1 (q), (3.9)

in which σs := D5(qb)q represents the angular momentum of the mechanical systemabout the end of leg-1, D5 is the last row of the matrix D, G5 is the last row of thegravity vector G, and fh1 denotes the horizontal components of f1.

3.2.4 Dynamics of the Double Support Phase

During double support, the mechanical system has three DOF. We decompose qas q = (q′d, q

′i)

′, where qd and qi denote dependent and independent configurationvariables, respectively. Next, we define

Q :={q = (q′d, q

′i)

′ ∈ Qs |p2(q) = (Ls, 0)′, rank∂p2

∂qd(q) = 2

}, (3.10)

wherep2(q) := f1(q) − f2(q) represents the Cartesian coordinates of the end of leg-2with respect to the end of leg-1 and Ls is a constant denoting the step length. If Q is anonempty set, by the Implicit Function Theorem, there exists a unique function� suchthat qd = �(qi) for any q = (q′d, q

′i)

′ ∈ Q. Without loss of generality, we assume thatqd := (q1, q2)′ and qi := (q3, q4, q5)′ (see Fig. 3.1). For the later purposes, define Qi

4 Since q5 is measured so that it decreases in the clockwise direction, we do not make use of PropositionB.11 of Ref. [18, p. 430].


as a simply connected and open subset of (0, π2 ) × (0, 3π2 ) × (−π

2 ,π2 ) and T : Qi →

R5 by

T (qi) :=[�(qi)

qi

].

The following theorem presents a reduced-order model with three DOF and fourcontrol inputs, called the constrained dynamics,5 for the evolution of the mechanicalsystem during double support.

Theorem 3.1 (Constrained Dynamics of Double Support) Let Q be a nonemptyset. Then, the constrained dynamics for the double support phase is given by

Dψ(qi) qi + Cψ(qi, qi) qi +Gψ(qi) = Bψ(qi) u, (3.11)

where

Dψ(qi) := ∂T

∂qi

′D ◦ T ∂T

∂qi

Cψ(qi, qi) := ∂T

∂qi

′C

(T,∂T

∂qiqi

)∂T

∂qi+ ∂T

∂qi

′D ◦ T ∂

∂qi

(∂T

∂qiqi

)

Gψ(qi) := ∂T

∂qi

′G ◦ T

Bψ(qi) := ∂T

∂qi

′B

and “◦" represents function composition.

Proof. During the double support phase, applying the principal of virtual work inequation (3.1) yields

A(qb) q+ C(qb, q) q = Bu− ∂f1

∂q

′(q)F1 − ∂f2

∂q

′(q)F2

mtot pcm +mtot

[0

g0

]= F1 + F2,

(3.12)

where F1 := (Fh1 , Fv1 )′ ∈ R2 and F2 := (Fh2 , F

v2 )′ ∈ R2 denote the ground reaction

forces at the end of leg-1 and leg-2, respectively. By assuming that the end of leg-1 ison the origin of the world frame, pcm(q) = f1(q), and as a consequence, the last two

5 The terminology of a constrained dynamics is taken from Ref. [70, p. 157].


rows of equation (3.12) yield

F1 = mtot∂f1

∂qq+mtot

∂

∂q

(∂f1

∂qq

)q+mtot

[0

g0

]. (3.13)

Substituting equation (3.13) into the first five rows of matrix equation (3.12) resultsin

D(qb) q+ C(qb, q) q+G(q) = Bu+ ∂p′2

∂q(q)F2. (3.14)

In the case of walking on the flat ground during the double support phase, p2(q) =(Ls, 0)′, and hence, the second time derivative of p2 is zero,

p2(q, q, q) = ∂p2

∂q(q) q+ ∂

∂q

(∂p2

∂q(q)q

)q = 02×1. (3.15)

By definition of Q, q ∈ Q implies that rank ∂p2∂q

(q) = 2. Thus, solving equations (3.14)and (3.15) simultaneously for q and F2 yields

F2 =(∂p2

∂qD−1 ∂p2

∂q

′)−1(∂p2

∂qD−1(Cq+G− Bu) − ∂

∂q

(∂p2

∂qq

)q

)(3.16)

and

D(qb) q+ Cd(q, q) q+Gd(q) = Bd(q) u, (3.17)

where

Cd(q, q) := �C + ∂p2

∂q

′(∂p2

∂qD−1 ∂p2

∂q

′)−1∂

∂q

(∂p2

∂qq

)Gd(q) := �GBd(q) := �B

�(q) := I5×5 − ∂p2

∂q

′(∂p2

∂qD−1 ∂p2

∂q

′)−1∂p2

∂qD−1.

We remark that matrix� is a projection matrix because ∂p2∂qD−1� = 02×5 and�2 =

� (idempotent). Due to the fact that for any (q′, q′)′ ∈ T Q,

∂p2

∂q(q) q = 02×1,


it follows that

∂p2

∂q◦ T (qi)

∂T

∂qi(qi) = 02×3

and consequently,

∂T ′

∂qi(qi)� ◦ T (qi) = ∂T ′

∂qi(qi)

for every qi ∈ Qi. Multiplying equation (3.17) by ∂T′

∂qi(qi) from the left and substituting

the relations q = ∂T∂qiqi and q = ∂T

∂qiqi + ∂

∂qi( ∂T∂qiqi)qi completes the proof. �

For reasons that will be discussed in Sections 3.3 and 3.4 (see Lemma 3.3 andRemark 3.10), we assume that u1 and u2 are two predetermined continuously diff-erentiable functions of independent configuration variables, that is, u1 = u1(qi) andu2 = u2(qi). Then, the dynamics of the double support phase in equation (3.11) canbe rewritten in the following form:

Dψ qi + Cψ qi +Gψ − ∂�′

∂qi

[u1(qi)

u2(qi)

]= βψ

[u3

u4

], (3.18)

where βψ := [I2×2 02×1]′. The functions u1(qi) and u2(qi) will be determined inSection 3.5. Furthermore, by introducing xd := (q′i, q′i)′ as the state vector for thedouble support phase, a state equation for equation (3.18) is

xd = fd(xd, u1(xd), u2(xd)) + gd(xd)[u3

u4

].

The state space is also taken as

Xd := T Qi := {xd := (q′i, q′i)

′|qi ∈ Qi, qi ∈ R3},

where Qi := {qi ∈ Qi | T (qi) ∈ Q}.

Remark 3.5 (Validity of the Double Support Phase Model) The double support

phase model is valid when Fv1 > 0, |Fh1Fv1

| < μs, Fv2 > 0, and |Fh2Fv2

| < μs.

3.2.5 Impact Model

In this section, an impact model is obtained for describing the state of the mechanicalsystem at the beginning of the double support phase (i.e., after impact) in termsof the state at the end of the single support phase (i.e., before impact). We shall


assume that the impact is inelastic and instantaneous. Also, it is assumed that thestance leg does not leave the ground after impact. Let IR1 := (IhR1, I

vR1)′ ∈ R2 and

IR2 := (IhR2, IvR2)′ ∈ R2 represent the intensity of the impulsive ground reaction forces

at the end of leg-1 and leg-2, respectively. For development of the impact map, wemake use of the flight phase model. Let q−f and q+f be the generalized velocity of themechanical system just before and after impact, respectively. From the impact modelof Ref. [91], integration of equation (3.12) over the infinitesimal time interval of theimpact yields

[A 05×2

02×5 mtot I2×2

](q+f − q−f ) =

⎡⎣− ∂f1

∂q

′

I2×2

⎤⎦ IR1 +

⎡⎣− ∂f2

∂q

′

I2×2

⎤⎦ IR2. (3.19)

Since the robot is in single support before impact, p−cm = ∂f1

∂q(q)q−. Moreover, the fact

that the stance leg does not leave the ground after impact implies that p+cm = ∂f1

∂q(q)q+.

Consequently, the first five rows of matrix equation (3.19) in combination with thelast two rows imply that IR1 = 12(q)IR2, where

12(q) := −(I2×2 +mtot

∂f1

∂qA−1 ∂f1

∂q

′)−1(I2×2 +mtot

∂f1

∂qA−1 ∂f2

∂q

′).

Since the impact is assumed inelastic, ∂p2∂q

(q)q+ = 02×1 and thus,

∂p2

∂q

(q− − A−1

(∂f1

∂q

′12 + ∂f2

∂q

′)IR2

)= 02×1. (3.20)

Definition 3.1 (Nonsingular Impact) The impact model is nonsingular if

det

(∂p2

∂qA−1

(∂f ′

1

∂q12 + ∂f ′

2

∂q

))/= 0.

Let the impact model be nonsingular. Then, equation (3.20) implies that

IR2 =(∂p2

∂qA−1

(∂f1

∂q

′12 + ∂f2

∂q

′))−1∂p2

∂qq−

=: 2(q) q−

IR1 = 12(q)2(q) q− =: 1(q) q−.

(3.21)


Finally, q+ is given by q+ = dq,s(q)q−, where

dq,s(q) := I5×5 − A−1(∂f1

∂q

′1 + ∂f2

∂q

′2

).

By defining πi := [03×2 I3×3], the transition map from the single support phase tothe double support phase can be expressed as x+d = ds (x−s ), where

ds (x−s ) = ds (q−s , q−s ) :=

[dqi,s(q

−s )

dqi,s(q−s ) q−s

].

Moreover,dqi,s(q−s ) := πiq−s ,dqi,s(q

−s ) := πidq,s(q−s ) and the superscripts “−" and

“+" denote the state of the mechanical system just before and after the discretetransitions.

Remark 3.6 (Validity of the Impact Model) Following the double impact condi-tions presented in Refs. [67] and [92] (see equations (32)–(35)), the impact model is

valid when IvR1 > 0, | IhR1IvR1

| < μs, | IhR2IvR2

| < μs, and p2+1 ≤ 0.

3.2.6 Transition from the Double Support Phase to the SingleSupport Phase

For simplifying the analysis of the hybrid model of walking in Section 3.4, it isassumed that the transition from the double support phase to the single support phaseoccurs at a predetermined point in the configuration space of the double supportphase. During this transition, the position and velocity remain continuous. Hence, thetransition map can be expressed as x+s = sd(x−d ), in which

sd(x−d ) = sd(q−i , q−i ) :=

[sq,d(q

−i )

sq,d(q−i ) q−i

].

In addition,sq,d(q−i ) := RT (q−i ),sq,d(q

−i ) := R ∂T

∂qi(q−i ) and R is a relabling matrix

to swap the role of the legs, with the propertyRR = I5×5. The validity of this transitioncondition is confirmed by designing the control law in the single support phase sothat it leads to y2 > 0 at the beginning of the single support phase. For this purpose,as in Ref. [55], it is assumed that discontinuities of the control inputs in transitionsare allowed.

3.2.7 Hybrid Model of Walking

The overall model of walking can be expressed as a nonlinear hybrid system consistingof two state manifolds that correspond to the single and double support phases as


follows:

s :

⎧⎪⎨⎪⎩xs = fs(xs) + gs(xs) u, x−s /∈ Sdsx+d = ds (x−s ), x−s ∈ SdsSds := {xs ∈ Xs |Hds (xs) = 0}

d :

⎧⎪⎪⎪⎪⎨⎪⎪⎪⎪⎩xd = fd(xd, u1, u2) + gd(xd)

[u3

u4

], x−d /∈ Ssd

x+s = sd(x−d ), x−d ∈ SsdSsd := {xd ∈ Xd |Hsd(xd) = 0}.

(3.22)

In this model, transition from the single support phase to the double support phaseoccurs when the height of the swing leg end becomes zero. Thus, Hds (xs) := y2(qs)(see Fig. 3.1). Following the assumption of Section 3.2.6, we define the switch-ing hypersurface Ssd as the zero level set of the smooth function Hsd : Xd → R byHsd(xd) := xH ◦ T (qi) − x−H,d , where xH (q) is the horizontal position of the hip jointand x−H,d is a constant threshold to be determined.

3.3 CONTROL LAWS FOR THE SINGLE AND DOUBLESUPPORT PHASES

In order to reduce the dimension of the hybrid model of walking to simplify thestabilization problem for the desired periodic orbit in each of the continuous phases,a finite-time attractive and invariant submanifold is created by a continuous controllaw. Specifically, the control laws in the single and double support phases are chosenas time-invariant feedback based on zeroing holonomic output functions with theuniform vector relative degree 2. This control strategy will result in a two-dimensionalzero dynamics manifold on each of the state manifolds, that is, holonomic quantitiesthat are to be controlled are dependent on a holonomic quantity that is a strictlymonotonic function of time on a typical walking gait.

3.3.1 Single Support Phase Control Law

As in Ref. [52], consider the following holonomic output function for the dynamicsof the mechanical system in the single support phase

ys := hs(qs) := qb − hd,s ◦ θs(qs), (3.23)

CONTROL LAWS FOR THE SINGLE AND DOUBLE SUPPORT PHASES 47

where6 θs(qs) := q12 + q2 − q5 =: Coqs is the angle of the virtual leg with respect

to the world frame and hd,s : R→ R4 is at least a twice continuously differentiable

function that specifies the desired evolution of the body angles in terms of θs. InSection 3.5, the function hd,s will be chosen such that the holonomic output functionin equation (3.23) vanishes on the single support phase of a desired periodic solutionof the open-loop hybrid model of walking in equation (3.22).

It is assumed that there exists an open set Qs ⊂ Qs such that for any qs ∈ Qs thedecoupling matrix

LgsLfshs(qs) = ∂hs

∂qs(qs)D

−1(qs)B

is invertible. For the later uses, let O = Os ∪ Od denote a desired feasible period-onesolution of the open-loop hybrid model of walking in equation (3.22) that is transversalto Sds and Ssd , where Os := O ∩ Xs and Od := O ∩ Xd . Furthermore, suppose thatOs ⊂ T Qs. By Lemma 1 of Ref. [52], since the decoupling matrix LgsLfshs(qs) isinvertible on Qs and the holonomic output function hs(qs) vanishes on Os, the set

Zs := {xs ∈ T Qs|hs(xs) = 04×1, Lfshs(xs) = 04×1}

is an embedded two-dimensional submanifold of TQs. Moreover, suppose that Sds ∩Zs is an embedded one-dimensional submanifold ofTQs.7 Then, on the manifoldSds ∩Zs, the configuration variables are determined [52]. In particular, let πq : (q, q) → q

be a canonical projection map. Then, πq(Sds ∩ Zs) = {q−s }, and for the later purposes,let θ−s := θs(q−s ).

Next, let vs : R4 × R4 → R4 be a continuous function such that the origin for the

closed-loop system ys = vs(ys, ys) is globally finite-time stable.8 Then, the continu-ous time-invariant feedback law

us(xs) := −(LgsLfshs(xs))−1(L2fshs(xs) − vs(hs(xs), Lfshs(xs))

)= −

( ∂hs∂qsD−1B︸︷︷︸

LgsLfshs

)−1( ∂

∂qs

(∂hs∂qsqs

)qs − ∂hs

∂qsD−1(Cqs +G)︸︷︷︸

L2fshs

−vs)

(3.24)

6 In this chapter, it is assumed that the femur and tibia links are of equal length.7 See hypothesis HH5 of Ref. [18, p. 126].8 References [46, 93] describe a method for designing the continuous function vs. In particular, by applying

the continuous feedback law v = 1ε2ψ(y, εy), in which

ψ(y, εy) := −sign(εy) |εy|α − sign(φ(y, εy)) |φ(y, εy)| α2−α ,

0 < α < 1, ε > 0 and

φ(y, εy) := y + 12−α sign(εy) |εy|2−α,

the origin for the scalar double integrator y = v is globally finite-time stable.


renders Zs locally finite-time attractive (see Definition 2.8, Section 2.3) and forwardinvariant9 under the closed-loop dynamics of the single support phase [52]. By defi-nition, Zs is the single support phase zero dynamics manifold and zs = fzero,s(zs) isthe single support phase zero dynamics, where10

fzero,s(zs) := f ∗s |Zs (zs)

f ∗s (xs) := fs(xs) + gs(xs) u∗

s (xs)

u∗s (xs) := −(LgsLfshs(xs))

−1L2fshs(xs).

Following Ref. [52], (θs, σs) is a valid set of local coordinates forZs. Furthermore,the single support phase zero dynamics can be expressed by Ref. [52],

θs = κ1(θs) σsσs = κ2(θs),

(3.25)

in which

κ1(θs) := ∂θs

∂qsλs

∣∣∣Z

= Coλs∣∣∣Z

κ2(θs) := −G5

∣∣∣Z

λs(qs) :=[∂hs∂qs

(qs)

D5(qs)

]−1 [04×1

1

].

It is also shown that the zero dynamics of the single support phase has the LagrangianLzero,s := Kzero,s − Vzero,s, where

Kzero,s(σs) := 1

2(σs)

2

Vzero,s(θs) := −∫ θs

θ+s

κ2(ξ)

κ1(ξ)dξ

and θ+s is a constant value that will be determined later. Moreover, Sds ∩ Zs can beexpressed as

Sds ∩ Zs = {(q′, q′)′|q = q−s , q = λs(q−s ) σ−s , σ

−s ∈ R}. (3.26)

9 A set Z is said to be forward invariant under the dynamics x = f (x) if for every x0 ∈ Z, there existst1 > 0 such that ϕf (t; x0) ∈ Z for t ∈ [0, t1), where ϕf (t; x0) represents the maximal solution of thedifferential equation x = f (x) with the initial condition x0. Furthermore, from Proposition B.1 of Ref.[18, p. 384], if Z is forward invariant, then for all x ∈ Z, f (x) ∈ TxZ.

10 In our notation, f |M represents the restriction of the function f to the set M.


Remark 3.7 (Invertibility of the Decoupling Matrix on Os) From Proposition6.1 of Ref. [18, p. 158] and equation (3.25), if on the orbit Os, the time evolution of θsis an increasing function of time (i.e., θs > 0), invertibility of the decoupling matrixLgsLfshs(qs) on Os is equivalent to the angular momentum about the stance leg end(i.e., σs) being nonzero during the single support phase, because it can be shown that1/κ1(θs) is the determinant of the decoupling matrix in the coordinates (qb, θs).

3.3.2 Double Support Phase Control Law

Analogous to the development for the single support phase, a holonomic output func-tion hd(qi) with dimension two is chosen for the constrained dynamics of the doublesupport phase. The output function is chosen as a vector with relative degree (2, 2)′with respect to the control inputs (u3, u4)′ on an open subset of the configuration spaceQi. Solution of the output zeroing problem by the control inputs (u3, u4)′ results in atwo-dimensional zero dynamics manifold. However, the control inputs u1 and u2 arenot employed in the output zeroing problem. We will employ them to ensure validityof the double support phase model and minimization of the norm of the control inputon Od in Section 3.5. To make this notion precise, we define the following holonomicoutput function:

yd := hd(qi) := ϕ(qi) − hd,d ◦ xH (qi), (3.27)

where ϕ(qi) represents the quantities that are to be controlled. In particular, it consistsof the vertical displacement of the hip joint and trunk angle,

ϕ(qi) :=[yH (qi)

e′3qi

],

where yH (qi) = yH ◦ T (qi) is the vertical displacement of the hip joint and e′3qi = q5is the trunk angle.11 Moreover, xH (qi) = xH ◦ T (qi) is the horizontal displacementof the hip joint and the function hd,d(xH ) is at least a C2 function that specifiesthe desired evolution of ϕ(qi) in terms of xH . The function hd,d(xH ) will be con-structed such that the holonomic output function in equation (3.27) vanishes on Od .Forcing yd to be zero will result in the evolution of the vertical displacement ofthe hip joint and trunk angle to be constrained to the horizontal displacement of thehip joint.

To introduce a valid coordinate transformation on T Qi, we first present the fol-lowing lemma by which a coordinate transformation will be used to obtain the zerodynamics corresponding to the output function (3.27) during the double supportphase.

11 Throughout this book, ei ∈ Rn is defined by ei := [0 · · · 1︸︷︷︸ith

· · · 0]′.


Lemma 3.1 (Coordinate Transformation on Qi) The mapping �d : Qi → R3

by

�d(qi) :=[hd(qi)

xH (qi)

]

is a diffeomorphism to its image.

Proof. It is sufficient to show that the Jacobian matrix ∂∂qi�d(qi) has full rank in Qi.

Since the rank of a matrix does not change by adding a multiple of a row to anotherrow,

rank∂

∂qi�d(qi) = rank

[∂ϕ∂qi

(qi)∂xH∂qi

(qi)

]. (3.28)

Next, define the mapping � : Qs → R5 by

�(q) :=

⎡⎢⎢⎢⎢⎢⎢⎣

x2(q)

y2(q)

xH (q)

yH (q)

e′5q

⎤⎥⎥⎥⎥⎥⎥⎦.

For the biped robot described previously, it can be shown that

det∂�

∂q(q) = l2t l2f sin(q1) sin(q3),

where lt and lf represent the length of the tibia and femur links, respectively. SinceQs is a simply connected and open subset of [(0, π2 ) × (0, 3π

2 )]2 × (−π2 ,π2 ), for every

q ∈ Qs, det ∂�∂q

(q) /= 0. Hence, � is diffeomorphism to its image. Next, define � :

Qi → R5 as the restriction of � to Qi, that is, �(qi) := � ◦ T (qi). The facts that

for any q ∈ Qs, rank ∂�∂q

(q) = 5, and for any qi ∈ Qi, T (qi) ∈ Qs imply that for any

qi ∈ Qi,

rank∂�

∂qi(qi) = rank

(∂�

∂q◦ T (qi)

∂T

∂qi(qi)

)

= rank∂T

∂qi(qi) = 3.


Furthermore,

�(qi) =

⎡⎢⎢⎢⎢⎢⎢⎣

Ls

0

xH (qi)

yH (qi)

e′3qi

⎤⎥⎥⎥⎥⎥⎥⎦,

which, in turn, in combination with equation (3.28) implies that rank ∂�∂qi

(qi) =rank ∂�d

∂qi(qi) = 3 for every qi ∈ Qi. �

Now assume that there exists an open set Qi ⊂ Qi such that for every qi ∈ Qi, thedecoupling matrix

LgdLfdhd(qi) = ∂hd

∂qi(qi)D

−1ψ (qi)βψ

is invertible. Furthermore, suppose that Od ⊂ T Qi. Since the decoupling matrixLgdLfdhd(qi) is invertible on Qi and hd vanishes on Od , the set

Zd := {xd ∈ T Qi|hd(xd) = 02×1, Lfdhd(xd) = 02×1}

is an embedded two-dimensional submanifold of T Qi. We remark that Zd is inde-pendent of u1(qi) and u2(qi) because hd is a holonomic output function.

Lemma 3.2 Let Ssd ∩ Zd /= φ. Then, Ssd ∩ Zd is an embedded one-dimensional sub-manifold of T Qi.

Proof. Ssd ∩ Zd can be expressed as Ssd ∩ Zd = {xd ∈ T Qi|Fd(xd) = 05×1}, where

Fd(xd) :=

⎡⎢⎣

hd(xd)

xH (xd) − x−H,dLfdhd(xd)

⎤⎥⎦ .

Since Ssd ∩ Zd /= φ and by Lemma 3.1, the mapping �d : Qi → R3 is a diffeomor-

phism to its image, there exists a unique point q−id ∈ Qi such that

�d(q−id) =

[hd(q

−id)

xH (q−id)

]=[

02×1

x−H,d

].

In other words, on Ssd ∩ Zd , the configuration variables are determined. In particular,let πqi : (qi, qi) → qi be the canonical projection map. Then, πqi (Ssd ∩ Zd) = {q−id}.


In addition,

rank∂Fd

∂xd(xd) = rank

⎡⎢⎢⎢⎣

∂hd∂qi

(qi) 02×3

∂xH∂qi

(qi) 01×3

∂∂qi

(∂hd∂qi

(qi)qi)∂hd∂qi

(qi)

⎤⎥⎥⎥⎦ ,

which in combination with Lemma 3.1 implies that for every xd ∈ Ssd ∩ Zd ,rank ∂Fd

∂xd(xd) = 5. �

Analogous to the derivation for the single support phase, the feedback law for thedouble support phase is chosen to be a continuous time-invariant feedback law havingthe following form:

[u3d (xd )

u4d (xd )

]= − (Lgd Lfd hd (xd ))

−1

(L2fdhd (xd, u1d (xd ), u2d (xd )) − vd (hd (xd ), Lfd hd (xd )

)

= −(∂hd

∂qiD−1ψβψ︸︷︷︸

Lgd Lfd hd

)−1

(∂

∂qi

(∂hd

∂qiqi

)qi − ∂hd

∂qiD−1ψ

(Cψqi +Gψ − ∂�′

∂qi

[u1d

u2d

])︸︷︷︸

L2fdhd

− vd

),

(3.29)

where vd : R2 × R2 → R2 is a continuous function such that the origin for the

closed-loop system yd = vd(yd, yd) is globally finite-time stable. The feedback law inequation (3.29) renders Zd locally finite-time attractive and forward invariant underthe closed-loop dynamics of the double support phase. By definition,Zd is the doublesupport phase zero dynamics manifold and zd = fzero,d(zd, u1d(zd), u2d(zd)) is thedouble support phase zero dynamics, where

fzero,d(zd, u1d(zd), u2d(zd)) := f ∗d |Zd (zd, u1d(zd), u2d(zd))

f ∗d (xd, u1d(xd), u2d(xd)) := fd(xd, u1d(xd), u2d(xd)) + gd(xd)

[u∗

3d(xd)

u∗4d(xd)

][u∗

3d(xd)

u∗4d(xd)

]:= −(LgdLfdhd(xd))

−1L2fdhd(xd, u1d(xd), u2d(xd)).

(3.30)


From Lemma 3.1,�d(qi) = [h′d(qi), xH (qi)]′ is a valid coordinate transformation

on Qi and thus,

⎡⎢⎢⎢⎣η1

η2

ϑ1

ϑ2

⎤⎥⎥⎥⎦ =

⎡⎢⎢⎢⎣

hd(qi)

Lfdhd(qi, qi)

xH (qi)

Lfd xH (qi, qi)

⎤⎥⎥⎥⎦

is a valid coordinate transformation on T Qi. Consequently, on the manifold Zd , qiand qi can be given by

qi = �−1d

([02×1

ϑ1

])

qi = ∂�d

∂qi(qi)

−1

[02×1

1

]ϑ2 =: λd(qi)ϑ2.

Now we are able to present the main result of this section that is expressed as thefollowing lemma. This lemma proposes a closed form for the zero dynamics of thedouble support phase in the local coordinates (xH , vxH ), where vxH denotes the hor-izontal velocity of the hip joint.

Lemma 3.3 (Double Support Phase Zero Dynamics) Assume that u1 = u1d(qi)and u2 = u2d(qi). Then, the double support phase zero dynamics can be expressed as

˙xH = vxH˙vxH = ω1(xH ) + ω2(xH ) v2

xH,(3.31)

in which

ω1(xH ) :=∂xH∂qi

D−1ψ �

(∂�′

∂qi

[u1d

u2d

]−G�

)

ω2(xH ) := − ∂xH

∂qiD−1ψ �Cψλd

− ∂xH

∂qiD−1ψ βψ

(∂hd

∂qiD−1ψ βψ

)−1∂

∂qi

(∂hd

∂qiλd

)λd

+ λ′d

∂2xH

∂q2i

λd,

�(qi) := I3×3 − βψ( ∂hd∂qiD−1ψ βψ)−1 ∂hd

∂qiD−1ψ and Cψ(qi) := Cψ(qi, λd(qi)).


The proof is given in Appendix A.1. Our aim is to provide closed-form expressionfor the solutions of the double support phase zero dynamics. To achieve this goal,note that if on the zero dynamics manifold, vxH /= 0, equation (3.31) can be rewrittenas follows:

dvxH

dxH= ω2(xH ) vxH + ω1(xH )

vxH,

which is a type of Bernoulli’s equation. Substituting zxH := (vxH )2 reduces theBernoulli’s equation to a first order nonhomogeneous linear equation with thefollowing form:

dzxH

dxH− 2ω2(xH ) zxH = 2ω1(xH ), (3.32)

for which the solutions can be expressed in closed form. To show this, let x+H,d be a con-stant scalar such that x+H,d < x

−H,d . Then, equation (3.32) over the interval [x+H,d, x

−H,d]

with the initial condition zxH (x+H,d) = z+xH := (v+xH )2 has the following solution:

zxH (xH ) = �2(xH )(−Wzero,d(xH ) + z+xH ), (3.33)

where

�2(xH ) := exp

(2∫ xH

x+H,d

ω2(ξ)dξ

)

Wzero,d(xH ) := −2∫ xH

x+H,d

ω1(ξ)

�2(ξ)dξ.

Note that since zxH = (vxH )2 > 0 and �2(xH ) > 0, this solution is valid as long asz+xH >Wmax

zero,d , where

Wmaxzero,d := max

x+H,d

≤xH≤x−H,d

Wzero,d(xH ).

Moreover, Ssd ∩ Zd can be expressed by

Ssd ∩ Zd = {(q′i, q′i)′|qi = q−id, qi = λd(q−id) v−xH, v−xH ∈ R}.

3.4 HYBRID ZERO DYNAMICS (HZD)

The concept of HZD was introduced in Ref. [52]. In order to reduce the dimension ofthe hybrid model of walking, by assumption of hybrid invariance, the zero dynamicsmanifolds of the single and double support phases can be assembled into a hybrid

HYBRID ZERO DYNAMICS (HZD) 55

restricted dynamics called HZD. The hybrid restricted dynamics will result in a low-dimensional test to investigate the stability properties of a periodic orbit of the open-loop hybrid model of walking that is also an integral curve of HZD. This sectionpresents the HZD for the walking model. To achieve this result, letds (Sds ∩ Zs) ⊂ Zdand sd(Ssd ∩ Zd) ⊂ Zs. Then, HZD for the hybrid system in equation (3.22) can bedefined as follows:

zero :

⎧⎪⎪⎪⎪⎨⎪⎪⎪⎪⎩

zs = fzero,s(zs) z−s /∈ Sds ∩ Zsz+d = dzero,s(z

−s ) z−s ∈ Sds ∩ Zs

zd = fzero,d(zd) z−d /∈ Ssd ∩ Zdz+s = szero,d(z

−d ) z−d ∈ Ssd ∩ Zd,

(3.34)

where dzero,s and szero,d are restrictions of the switching maps ds and sd to themanifolds Zs and Zd , respectively.

3.4.1 Analysis of HZD in the Single Support Phase

Let sd(Ssd ∩ Zd) ⊂ Zs. In the local coordinates (θs, σs) for the manifold Zs, thevalues of the quantities θs and σs at the beginning of the single support phase can beexpressed as

θ+s := θs(q+s ) = θs ◦sq,d(q−id)σ+s = D5(q+s ) q+s

= D5 ◦sq,d(q−id)sq,d(q−id) λd(q−id) v−xH=: δsd(q

−id) v

−xH .

Consequently, the restricted transition map szero,d : Ssd ∩ Zd → Zs can be given by

szero,d(x−H,d, v

−xH ) :=

[θ+s

δsd(q−id) v

−xH

], (3.35)

where q−id = �−1d ([01×2, x

−H,d]

′). Following the results in Ref. [52], when the robottakes a step, the angular momentum about the stance leg end is nonzero. Thus, ζs :=12 (σs)2 is a valid coordinate transformation. Furthermore, since the single supportphase zero dynamics is Lagrangian, Ezero,s := Kzero,s + Vzero,s is stationary on Zsand consequently,

1

2(σ−s )2 − 1

2(σ+s )2 = ζ−s − ζ+s = −Vzero,s(θ

−s ).


By introducing z−xH := (v−xH )2, the restricted generalized Poincare map of thesingle support phase can be defined as ρs : Ssd ∩ Zd → Sds ∩ Zs by

ρs(z−xH ) :=ζ+s − Vzero,s(θ

−s )

=1

2(δsd)

2 z−xH − Vzero,s(θ−s ).

Due to the fact that ζ−s = 12 (σ−

s )2 > 0, the domain of definition of ρs can also begiven by

Dρs :={z−xH > 0 | 1

2(δsd)

2 z−xH > Vmaxzero,s

},

where

Vmaxzero,s := max

θ+s ≤θs≤θ−sVzero,s(θs).

Remark 3.8 (Upper Bound in Dρs ) As in Ref. [52], there exists an upper boundin the domain of definition Dρs . This upper bound is the largest value of z−xH such thatthe ground reaction force at the stance leg end is admissible (see Remark 3.3).

The following lemma determines the set of all points in Ssd ∩ Zd for which thetransition from the double support phase to the single support phase can occur onHZD. Using this lemma, the domain of definition of ρs (i.e., Dρs ) can be modified.

Lemma 3.4 (Transition Condition from DS to SS on HZD)12 Letsd(Ssd ∩ Zd) ⊂Zs. Then, there exist functions ω1, ω2 : R→ R such that the transition condition fromthe double support phase to the single support phase on HZD can be expressed as

ω1(x−H,d) + ω2(x−H,d) z−xH > 0.

The proof is given in Appendix A.2. From Lemma 3.4, the domain of definitionDρs is also modified as follows:

Dρs :={z−xH > 0 | 1

2(δsd)

2 z−xH > Vmaxzero,s, ω1 + ω2 z

−xH > 0

}.

12 DS and SS represent the double support and single support phases, respectively.


3.4.2 Analysis of HZD in the Double Support Phase

Let ds (Sds ∩ Zs) ⊂ Zd . In the local coordinates (xH , vxH ) for the manifold Zd , theinitial values of xH and vxH can be expressed as

x+H,d := xH (q+i ) = xH ◦dqi,s(q−s )

v+xH = ∂xH

∂qi(q+i ) q+i

= ∂xH

∂qi◦dqi,s(q−s )dqi,s(q

−s ) λs(q

−s ) σ−

s

=: δds (q−s ) σ−

s .

Hence, the restricted transition map dzero,s : Sds ∩ Zs → Zd can be given by

dzero,s(θ−s , σ

−s ) :=

[x+H,d

δds (q−s ) σ−

s

], (3.36)

where q−s = �−1s ([01×4, θ

−s ]′) and

�s(qs) :=[hs(qs)

θs(qs)

].

From equation (3.33),

z−xH = �2(x−H,d)(z+xH − Wzero,d(x

−H,d)) (3.37)

and thus, the restricted generalized Poincare map of the double support phase can beexpressed as ρd : Sds ∩ Zs → Ssd ∩ Zd by

ρd(ζ−s ) := �2(x−H,d)((v

+xH )2 − Wzero,d(x

−H,d))

= �2(x−H,d)(2(δds )2 ζ−s − Wzero,d(x

−H,d)).

The domain of definition of ρd is also given by

Dρd :={ζ−s > 0 | 2(δsd)

2 ζ−s >Wmaxzero,d

}.

Remark 3.9 (Upper Bound in Dρd) There exists an upper bound in the domain

of definition Dρd due to admissibility of the ground reaction forces at the end of thelegs during the double support phase.


3.4.3 Restricted Poincare Return Map

This section deals with a procedure for constructing a restricted Poincare return mapfor the hybrid model of walking (see Section 2.3). Moreover, the fundamental resultsof this section that are developed to test the stability behavior of a periodic orbit ofHZD are summarized. First, we shall define the restricted Poincare return map. Letds (Sds ∩ Zs) ⊂ Zd andsd(Ssd ∩ Zd) ⊂ Zs. Then, in the local coordinates (θs, ζs) forthe manifold Zs, the restricted Poincare return map is defined as ρ : Sds ∩ Zs :→Sds ∩ Zs by

ρ(ζ−s ) := ρs ◦ ρd(ζ−s )

= (δsd)2 (δds )

2�2(x−H,d) ζ−s − 1

2(δsd)

2�2(x−H,d)Wzero,d(x−H,d) − Vzero,s(θ

−s ).

Moreover, the domain of definition of ρ can be given by

Dρ := {ζ−s ∈ Dρd |ρd(ζ−s ) ∈ Dρs}.

The following lemma is an important result that enables ρ to be considered as thePoincare return map for a system with impulse effects. Consequently, the resultsdeveloped for the existence and stability analysis of the periodic orbits in systemswith impulse effects, see Theorem 2.1, Section 2.2, can be applied to the hybridmodel of walking.

Lemma 3.5 (HZD as a System with Impulse Effects) Let ds (Sds ∩ Zs) ⊂ Zdand sd(Ssd ∩ Zd) ⊂ Zs. Assume that walking occurs from left to right. Then, in thecoordinates (θs, σs) for the manifoldZs, ρ is also a Poincare return map for the systemwith impulse effects

zero,s :

{zs = fzero,s(zs), z−s /∈ Sds ∩ Zsz+s = zero,s(z−s ), z−s ∈ Sds ∩ Zs,

(3.38)

where zero,s(θ−s , σ−s ) := [θ+s ,�(σ−

s )]′ and

�(σ−s ) := δsd

√�2(x−H,d)

((δds )2(σ−

s )2 − Wzero,d(x−H,d)

). (3.39)

Proof. From the procedure for constructing zero,s, zero,s = szero,d ◦ ρd . More-over, Theorem 2.1 of Chapter 2 immediately implies that ρ is a Poincare return mapfor zero,s. Since walking is from left to right (i.e., vxH > 0), equations (3.36) and(3.35) in combination with equation (3.37) yield equation (3.39). �


Definition 3.2 (Continuously Differentiable HZD) HZD is said to be continu-ously differentiable if fzero,s : Zs → TZs, fzero,d : Zd → TZd , dzero,s : Sds ∩ Zs →Zd and szero,d : Ssd ∩ Zd → Zs are C1.

To investigate the stability behavior of the periodic orbits of HZD, we prove thefollowing theorem that is the main result of this section. This theorem establishes anequivalence between the stability of the periodic orbits of HZD and the equilibriumpoints of the discrete-time system ζ−s [k + 1] = ρ(ζ−s [k]) with the state spaceSds ∩ Zs.

Theorem 3.2 (Exponentially Stable Periodic Orbits of HZD) Assume that Sds ∩Zs and Ssd ∩ Zd are embedded one-dimensional submanifolds of TQs and T Qi,respectively. Moreover, suppose that ds (Sds ∩ Zs) ⊂ Zd , sd(Ssd ∩ Zd) ⊂ Zs, andHZD is C1. By defining

ζ∗s := −12 (δsd)

2�2(x−H,d)Wzero,d(x−H,d) + Vzero,s(θ−s )

1 − μ ,

in which

μ := (δsd)2 (δds )

2�2(x−H,d), (3.40)

the following statements are true.

1. If ζ∗s ∈ Dρ, then ζ∗s is the fixed point of ρ.

2. If ζ∗s ∈ Dρ, then ζ∗s is a locally exponentially stable equilibrium point of ζ−s [k +1] = ρ(ζ−s [k]) if and only if μ < 1.

3. HZD has a nontrivial periodic orbit transversal to Sds ∩ Zs and Ssd ∩ Zd if andonly if μ /= 1 and ζ∗s ∈ Dρ.

4. HZD has an exponentially stable periodic orbit transversal to Sds ∩ Zs andSsd ∩ Zd if and only if ζ∗s ∈ Dρ and μ < 1.

Proof. By considering the fact that μ is nonnegative, all of the statements are imme-diate consequences of Theorems 2.2 and 2.4 of Chapter 2 and Lemma 3.5. �

Remark 3.10 (Effect of u1d(qi) and u2d(qi) on Stability of Periodic Orbits) Sinceduring the double support phase, u1 = u1d(qi) and u2 = u2d(qi), from Lemma 3.3,ω2(xH ) and thereby �2(x−H,d) and μ are independent of these control inputs. Thus,the functions u1d(qi) and u2d(qi) do not affect the stability of the fixed point. However,the existence of a limit cycle and the value of ζ∗s are affected by the choice of u1d(qi)and u2d(qi).

Remark 3.11 (Using u1 and u2 to Zero the Output Function) By using u1 and u2to zero the holonomic output function hd(qi), the decoupling matrix can be expressedas LgdLfdhd(qi) = ∂hd

∂qiD−1ψ∂�∂qi

′. It can be shown that the (2 × 2) upper submatrix of


∂�∂qi

′is full rank on Qi and its determinant is equal to sin q1

sin q3. Thus, from Lemma 3.1,

1 ≤ rankLgdLfdhd(qi) ≤ 2, for every qi ∈ Qi. However, small numerical values ofq1 and q3 (as can be observed on a typical gait) may result in significant errors whilecomputing u1 and u2, due to the term sin q1

sin q3in (LgdLfdhd(qi))

−1.

3.5 DESIGN OF AN HZD CONTAINING A PRESPECIFIEDPERIODIC SOLUTION

The objective of this section is to design an HZD containing a desired feasible period-one solution of the open-loop hybrid model. For this purpose, the sample-based virtualconstraints method introduced in Ref. [94] is used. However, since in the doublesupport phase, u1 and u2 are not employed for the output zeroing problem and theopen-loop control input corresponding to a trajectory is not unique, the sample-basedvirtual constraints method is not sufficient to achieve the objective. This sectionpresents a design method that ensures that a desired feasible periodic solution isan integral curve of HZD and the control input associated with this solution in thedouble support phase has minimum norm. The required conditions will be specifiedas we proceed.

Let O := Os ∪ Od be a period-one solution of the open-loop hybrid model inequation (3.22). Suppose that qs(t), 0 ≤ t < Ts and qd(t), Ts ≤ t < Ts + Td =: T rep-resent the time evolutions of the configuration variables on Os and Od , respectively.Moreover, Ts and Td are the time durations of the single and double support phaseson O. For the later purposes, let qid(t) denote the set of independent configurationvariables of qd(t).

3.5.1 Design of the Output Functions

Assume that the following hypotheses of periodic orbit are satisfied:13

(HPO1) qs(t) and qd(t) are at least three-times continuously differentiable on[0, Ts) and [Ts, T ), respectively.

(HPO2) Os is transversal to Sds and Od is transversal to Ssd .(HPO3) �s(t), the time evolution of θs in the single support phase, is a strictly

increasing function of time (i.e., inf0≤t<Ts �s(t) > 0).

(HPO4) xHd(t), the time evolution of xH in the double support phase, is a strictlyincreasing function of time (i.e., infTs≤t<T xHd(t) > 0).

(HPO5) The angular momentum about the stance leg end during the single supportphase is nonzero.14

13 Hypotheses HPO1, HPO2, HPO3, and HPO5 are taken from Ref. [18, p. 162].14 Hypotheses HPO3 and HPO5, together with Remark 3.7, imply the invertibility of the decoupling matrixLgsLfshs(qs) on the orbit Os.

DESIGN OF AN HZD CONTAINING A PRESPECIFIED PERIODIC SOLUTION 61

Next define

hd,s(θs) := qbs(t)∣∣∣t=�−1

s (θs)(3.41)

and

hd,d(xH ) :=[

yHd(t)

q5d(t)

] ∣∣∣∣t=x−1

Hd(xH )

, (3.42)

where qbs(t) is the time evolution of the body configuration variables onOs, and yHd(t)and q5d(t) are the time evolutions of the vertical displacement of the hip joint andtorso angle on Od , respectively. Furthermore, suppose that O satisfies the followingadditional hypothesis.

(HPO6) For every t ∈ [Ts, T ), the decoupling matrix LgdLfdhd(qid(t)) is inver-tible.

Hypotheses HPO3, HPO5, and HPO6 in combination with Remark 3.7 implythat there exist open sets Qs ⊂ Qs and Qi ⊂ Qi such that the decoupling matri-ces LgsLfshs(qs) and LggLfdhd(qi) are invertible on them, and thus the single anddouble support phase zero dynamics (Zs and Zd) exist. Moreover, by HPO2, Theo-rem 6.2 of Ref. [18, p. 163], and Lemma 3.2, Sds ∩ Zs and Ssd ∩ Zd are embeddedone-dimensional submanifolds of TQs and T Qi, respectively. The following lemmapresents the main result of this section that utilizes the previously mentioned con-struction procedure for holonomic output functions in the single and double supportphases to establish that the zero dynamics manifolds Zs and Zd are hybrid invariantfor the hybrid model of walking.

Lemma 3.6 (Hybrid Invariance) Let O be a periodic orbit of the hybrid model inequation (3.22) satisfying hypotheses HPO1–HPO6. Then, for the output functionsin equations (3.23) and (3.27) in combination with equations (3.41) and (3.42), thecorresponding zero dynamics manifolds (Zs and Zd) are hybrid invariant, that is,ds (Sds ∩ Zs) ⊂ Zd and sd(Ssd ∩ Zd) ⊂ Zs. Moreover, HZD exists.

Proof. Let (θs, σs) and (xH , vxH ) be the local coordinates forZs andZd , respectively.By HPO2, σ∗

s /= 0 and v∗xH /= 0, where σ∗s and v∗xH are the values of the quantities

σs and vxH in the points x∗s := Os ∩ Sds and x∗d := Od ∩ Ssd , respectively. Since Sds ∩Zs is an embedded one-dimensional submanifold of TQs, equation (3.26) impliesthat on Sds ∩ Zs, q = q−s and q = λs(q−s )σ−

s , where σ−s ∈ R. Furthermore, from the


construction procedure for hd,d ,

hd ◦dqi,s(q−s ) = 02×1

Lfdhd ◦ds (q−s , λs(q−s )σ∗s ) = ∂hd

∂qi◦dqi,s(q−s )dqi,s(q

−s ) λs(q

−s ) σ∗

s

= 02×1.

(3.43)

Since Lfdhd ◦ds (q−s , λs(q−s )σ−s ) is linear with respect to σ−

s and σ∗s /= 0, equation

(3.43) implies that for every σ−s ∈ R, Lfdhd ◦ds (q−s , λs(q−s )σ−

s ) = 02×1 and hence,ds (Sds ∩ Zs) ⊂ Zd . In a similar manner, it can be shown that sd(Ssd ∩ Zd) ⊂ Zs.The existence of HZD is immediate. �

3.5.2 Design of u1d and u2d

In the single support phase, if qs(t) is known, the corresponding open-loop controlinput us(t) is determined uniquely. In particular, let H0 := [I4×4 04×1], then,

us(t) = H0(D qs(t) + C qs(t) +G). (3.44)

However, in the double support phase, if qd(t) is known, the corresponding open-loop control input is not unique because the mechanical system is overactuated. Thefollowing lemma presents the set of C1 open-loop control inputs that correspond toOd .

Lemma 3.7 (C1 Open-Loop Control Inputs Corresponding to Od) Let O bea periodic orbit of the open-loop hybrid system in equation (3.22) satisfying HPO1.Furthermore, assume that the step length is nonzero (i.e., Ls /= 0) and rank ∂p2

∂q= 2

on Od . Then, C1 open-loop control input vector corresponding to Od is not uniqueand belongs to the set Ud(Od), where15

Ud(Od) :={ud : [Ts, T ) → R

4 | ud(t) = u0d(t) − ∂x′2

∂qbϒh(t), ϒh ∈ C1([Ts, T ),R)

}

and for every t ∈ [Ts, T ),

u0d(t) :=

(H0 − 1

Ls

∂y′2

∂qbe′5

)(D qd(t) + C qd(t) +G). (3.45)

Moreover, there exists a unique F2(t) for every ud(t) ∈ Ud(Od) such that equation(3.14) is satisfied.

15 In our notation, Ck(I,R) denotes the set of all Ck functions f : I → R.


The proof is given in Appendix A.3. Define

u := {u ∈ R4 | |ui| < umax, i = 1, . . . , 4} (3.46)

and

f := {F := (Fh, Fv)′ ∈ R2 |Fv > 0, |Fh| < μs|Fv|} (3.47)

as the admissible regions for the control input and ground reaction forces, respec-tively, where umax is a positive scalar.

Definition 3.3 (Feasible Trajectory of the Hybrid Model of Walking) The tra-jectory O of the open-loop hybrid model of walking is feasible if

1. the constraints on the joint angles and angular velocities are satisfied on O;

2. the open-loop control input and ground reaction force at the stance leg endcorresponding to qs(t) are admissible;

3. there exists at least one admissible control input ud(t) ∈ Ud(Od) such that theground reaction forces at the leg ends corresponding to qd(t) and ud(t) areadmissible;

4. the impact model on O is nonsingular and the impulsive ground reaction forcesare admissible (see Remark 3.6). Moreover, y2(0) > 0, where y2(t) is the timeevolution of the vertical displacement of the swing leg end with respect to theworld frame on O.

For the later purposes, assume that the periodic orbit O satisfies the followingadditional hypothesis.

(HPO7) The step length of the periodic orbit is positive (i.e.,Ls > 0), rank ∂p2∂qd

= 2

on Od , and O is feasible.16

Now let O be a periodic orbit of the hybrid system in equation (3.22) satisfyingHPO1–HPO7. Since all of the open-loop control inputs corresponding to Od are notadmissible, it is difficult to design a time-invariant controller for the double supportphase such that Od is an integral curve of the closed-loop system. To overcome thisdifficulty, we make use of u1d and u2d to ensure admissibility of the double supportphase controller. Assume that u1d = u1d(xH ) and u2d = u2d(xH ). On the manifoldZd , xH lies in [x+H,d, x

−H,d]. Thus, for any xH ∈ [x+H,d, x

−H,d], let u1 and u2 be the values

of the feedback laws u1d(xH ) and u2d(xH ), respectively. From equation (3.29), u3d

16 As discussed previously, rank ∂p2∂qd

= 2 on Od , together with the Implicit Function Theorem, implies the

existence of a unique function � such that qd = �(qi) for every q = (q′d, q′i)

′ ∈ Qi.


and u4d on Od are affine functions with respect to (u1, u2)′,

[u3d(t, u1, u2)

u4d(t, u1, u2)

]= U0(t) + U1(t)

[u1

u2

], (3.48)

where from HPO4, the quantity t can be expressed in terms of xH , in particular,t = x−1

Hd(xH ). Equation (3.48) in combination with equations (3.16) and (3.13) yieldthe following affine relations for the ground reaction forces on Od :

F1(t, u1, u2) = F01(t) + F11(t)

[u1

u2

]

F2(t, u1, u2) = F02(t) + F12(t)

[u1

u2

].

Next, for every xH ∈ [x+H,d, x−H,d], evaluate t = x−1

Hd(xH ), U0(t), U1(t), F01(t), F11(t),F02(t), and F22(t) on Od and define the following nonlinear optimization problem fordetermining u1 and u2:

minu1,u2

1

2‖ud(t, u1, u2)‖2

2

s.t. ud(t, u1, u2) ∈ Ud(Od)(u1, u2)′ ∈ a(Od, xH ),

(3.49)

where for every xH ∈ [x+H,d, x−H,d], a(Od, xH ) is defined to be the set of all points

(u1, u2)′ ∈ R2 for which the control input vector and the ground reaction forces atthe leg ends are admissible on the trajectory Od ,

a(Od, xH ) := {(u1, u2)′ ∈ R2|ud(t, u1, u2) ∈ u, F1(t, u1, u2), F2(t, u1, u2) ∈ f }.

By the constraint ud ∈ Ud(Od) in the optimization problem (3.49), Od is an integralcurve of the closed-loop system in the double support phase. Furthermore, the con-straint (u1, u2)′ ∈ a(Od, xH ) for every xH ∈ [x+H,d, x

−H,d] imposes the admissibility

of ud on Od . Note that from equation (3.48), ud on Od can be given by

ud(t, u1, u2) =[

02×1

U0(t)

]+[I2×2

U1(t)

][u1

u2

]

=: U0(t) + U1(t)

[u1

u2

].

(3.50)


The following lemma implies that for every xH ∈ [x+H,d, x−H,d], the constraint

ud(t, u1, u2) ∈ Ud(Od) can be expressed as an affine equality constraint with respectto (u1, u2)′.

Lemma 3.8 Let O be a periodic orbit of the open-loop hybrid model inequation (3.22) satisfying HPO1–HPO7. Then, there exist functions V0(t) ∈ R and01×2 /= V1(t) ∈ R1×2 such that on the periodic orbit O, the constraint ud(t, u1, u2) ∈Ud(Od) can be expressed as

V0(t) + V1(t)

[u1

u2

]= 0.

Proof. By definition of the set Ud(Od) and equation (3.50), the constraintud(t, u1, u2) ∈ Ud(Od) is equivalent to

U0(t) + U1(t)

[u1

u2

]= u0

d(t) − ∂x′2∂qb

(t)ϒh(t) (3.51)

for some ϒh(t) ∈ C1([Ts, T ),R). Since U ′1(t)U1(t) = I2×2 + U ′

1(t)U1(t) is positivedefinite, equation (3.51) results in

[u1

u2

]= (U ′

1(t)U1(t))−1

U ′1(t)

(u0d(t) − U0(t) − ∂x′2

∂qb(t)ϒh(t)

). (3.52)

Next, define

V2(t) := (U ′1(t)U1(t)

)−1U ′

1(t)∂x′2∂qb

(t)

P(t) := {y := (y1, y2) ∈ R1×2| yV2(t) = 0}.

Choosing an arbitrary nonzero V1(t) ∈ P(t) and V0(t) in the following form:

V0(t) := −V1(t)(U ′

1(t)U1(t))−1

U ′1(t)(u0

d(t) − U0(t)) (3.53)

completes the proof. �

By HPO7, the trajectory O is feasible and as a consequence, the solution space ofthe optimization problem (3.49) is nonempty. Since the set a(Od, xH ) is open forevery xH ∈ [x+H,d, x

−H,d], if the optimal solution of (3.49) exists, it is also the solution


of the following optimization problem:

minu1,u2

1

2

[u1u2

]U ′

1(t)U1(t)

[u1

u2

]+ U ′

0(t)U1(t)

[u1

u2

]

s.t. V0(t) + V1(t)

[u1

u2

]= 0.

(3.54)

Because Q := U ′1U1 = I2×2 + U ′

1U1 is positive definite and the cost function isquadratic, by applying Lagrange multipliers, the global minimum of the constrainedoptimization problem can be obtained online as follows:

[u∗

1(xH )

u∗2(xH )

]= −Q−1

(U ′

1U0 + (V1Q−1V ′

1)−1(V0 − V1Q−1U ′

1U0)V ′1

). (3.55)

Remark 3.12 It can be easily shown that the optimal solution (3.55) is independentof the choice 01×2 /= V1(t) ∈ P(t).

Next assume that the periodic orbit O fulfills hypotheses HPO1–HPO7 and theadditional following hypothesis.

(HPO8) For every xH ∈ [x+H,d, x−H,d], U ′

1(t)∂x′2∂qb

(t) /= 02×1 and(u∗

1(xH ), u∗2(xH ))′ ∈ a(Od, xH ).

Then, choose u1d(xH ) = u∗1(xH ) and u2d(xH ) = u∗

2(xH ). By this choice, the closed-loop control input on an open neighborhood of Od is also admissible. In addition,Od is an integral curve of the closed-loop system in the double support phase.However, on Od the control input vector is not necessarily identical to ud(t) fromDefinition 3.3.

If the condition (u∗1(xH ), u∗

2(xH ))′ ∈ a(Od, xH ) is not satisfied for some xH ∈[x+H,d, x

−H,d], we can define the following optimization problem for determining u1

and u2:

minu1,u2

1

2

[u1 u2

]U ′

1(t)U1(t)

[u1

u2

]+ U ′

0(t)U1(t)

[u1

u2

]

s.t. V0(t) + V1(t)

[u1

u2

]= 0

− umax + ε ≤ ui(t, u1, u2) ≤ umax − ε, i = 1, . . . , 4

Fhi (t, u1, u2) − μsFvi (t, u1, u2) ≤ −ε, i = 1, 2

− Fhi (t, u1, u2) − μsFvi (t, u1, u2) ≤ −ε, i = 1, 2,

(3.56)

STABILIZATION OF THE PERIODIC ORBIT 67

where ε is a positive scalar. We remark that the set of admissible ground reactionforces can also be expressed as

f = {F := (Fh, Fv)′ ∈ R2|Fh − μsFv < 0,−Fh − μsFv < 0}.

The solution of this latter optimization problem can be determined for a finite numberof points in the interval [x+H,d, x

−H,d] that are interpolated, for example, by cubic splines.

Remark 3.13 (Cubic Splines to Compute hd,s(θs) and hd,d(xH )) FollowingPropositions 6.2 and 6.3 of Ref. [18, pp. 163–164], since O is obtained from a motionplanning algorithm (see Section 3.7), we can produce the desired functions hd,s(θs)and hd,d(xH ) by sampling the orbits Os and Od and applying cubic spline interpola-tions between the samples.

Remark 3.14 (Cubic Splines to Compute u1d(xH ) and u2d(xH )) In general, itis difficult to obtain a closed-form expression for t = x−1

Hd(xH ). Moreover, qd(t) isknown at a finite number of samples. Thus, cubic spline interpolation can be used tocompute u1d(xH ) and u2d(xH ) on the interval [x+H,d, x

−H,d].

Remark 3.15 (Validity of the Transition Model from DS to SS) By HPO7, thevertical acceleration of the end of leg-2 is positive at the beginning of the single supportphase on the periodic orbit (i.e., y2(0) > 0). Sincesd : Ssd → Xs and us : Xs → R

4

are continuous, there exists an open set Ssd ⊂ Ssd such that for every x−d ∈ Ssd , y2 > 0at the beginning of the single support phase. Hence, the transition from the doublesupport phase to the single support phase is valid on an open neighborhood of O.

3.6 STABILIZATION OF THE PERIODIC ORBIT

This section addresses the problem of stabilization for a desired periodic orbit Osatisfying hypotheses HPO1–HPO7 or a set of weaker hypotheses as described inSection 2.4. If μ ≥ 1 in Theorem 3.2, the periodic orbit is not asymptotically sta-ble. For stabilizing O, the holonomic outputs in equations (3.23) and (3.27) will bemodified. The modification procedure consists of (i) adding augmentation functionsthat are finitely parameterized functions [18, p. 164], such as Bezier polynomials tothe holonomic output functions, and (ii) updating the parameters of the augmentationfunctions on a stride-to-stride basis. The idea of updating the parameters of HZD andoutput functions was introduced in Refs. [57–59] for stabilization, thereby improvingthe convergence rate and regulation of the average walking rate. In this chapter, wemake use of an event-based controller to update the parameters of the augmenta-tion functions at the end of each single support phase. The purpose of updating theparameters is to achieve hybrid invariance and stabilization.


Now define the modified outputs in the following forms:

hs(qs;α) := hs(qs) − fcn1(ss(qs);α)

hd(qi;β) := hd(qi) − fcn2(sd(qi);β),(3.57)

where α := [α0 α1 ... αMs−1 αMs ] ∈ A and β := [β0 β1 ... βMd−1 βMd ] ∈ B are theparameter matrices for the single and double support phases, respectively. Fur-thermore, A ⊂ R4×(Ms+1) and B ⊂ R2×(Md+1) are also open sets. The functionsfcn1 : [0, 1] × A → R

4 and fcn2 : [0, 1] × B → R2 are defined as Bezier polyno-

mials of degreeMs andMd , that is,17

fcn1(ss;α) :=Ms∑k=0

Ms!

k!(Ms − k)!αksks (1 − ss)Ms−k

fcn2(sd ;β) :=Md∑k=0

Md!

k!(Md − k)!βkskd(1 − sd)Md−k,

where αi ∈ R4, i = 0, 1, . . . ,Ms and βj ∈ R2, j = 0, 1, . . . ,Md denote the ith andjth columns of α and β, respectively. Also ss(qs) and sd(qi) are given by

ss(qs) := θs(qs) − θ+sθ−s − θ+s

sd(qi) := xH (qi) − x+H,dx−H,d − x+H,d

.

The control laws in the single and double support phases (i.e., within-stride con-trollers) are modified by replacing hs(qs) by hs(qs;α) in equation (3.24) and replacinghd(qi) by hd(qi;β) in equation (3.29), respectively. By this method, the single sup-port phase control law is parameterized by α and denoted us(xs;α). Also, the doublesupport phase control law is denoted (u3d(xd ;β), u4d(xd ;β))′. It is worth mentioningthat u1d(xd) and u2d(xd) do not depend on β and are identical to those developed inSection 3.5. Next, let Zs,α and Zd,β represent the zero dynamics manifolds cor-responding to hs(qs;α) and hd(qi;β), respectively. Define α∗ := 04×(Ms+1) andβ∗ := 02×(Md+1). Then, Zs,α∗ = Zs and Zd,β∗ = Zd .

Remark 3.16 (Fundamental Properties of Bezier Polynomials) For anycoefficient matrix α = col{αi}Mi=0 := [α0, . . . , αM], the Bezier polynomial

B(s, α) :=M∑i=0

M!

i!(M − i)! αi si (1 − s)M−i

17 The idea of using Bezier polynomials as augmentation functions is taken from Ref. [94].

STABILIZATION OF THE PERIODIC ORBIT 69

has the following properties [18, p. 139]:

(i) B(0, α) = α0

(ii) B(1, α) = αM(iii)

∂

∂sB(0, α) = M(α1 − α0)

(iv)∂

∂sB(1, α) = M(αM − αM−1).

In addition,

(v)∂2

∂s2B(0, α) = M(M − 1)(α2 − 2α1 + α0)

(vi)∂2

∂s2B(1, α) = M(M − 1)(αM − 2αM−1 + αM−2).

Lemma 3.9 (Hybrid Invariance for Parameterized Manifolds) Let O be aperiodic orbit of the hybrid model (3.22) satisfying hypotheses HPO1–HPO7. As-sume that the desired functions hd,s(θs) and hd,d(xH ) are defined as those ofequations (3.41) and (3.42), respectively. Moreover, suppose that Ms ≥ 3, Md ≥ 3,α0 = α1 = αMs−1 = αMs = 04×1, and β0 = β1 = βMd−1 = βMd = 02×1. Then, thefollowing statements are true:

1. Sds ∩ Zs,α = Sds ∩ Zs2. Ssd ∩ Zd,β = Ssd ∩ Zd3. sd(Ssd ∩ Zd,β) ⊂ Zs,α4. ds (Sds ∩ Zs,α) ⊂ Zd,β.

Proof. By properties of Bezier polynomials, see Remark 3.16, the proof is straight-forward. �

HZD exists for α = α∗ = 04×(Ms+1) and β = β∗ = 02×(Md+1). Furthermore,Lemma 3.9 and the continuity of the modified outputs with respect to α and β im-ply that there exists ε > 0 such that for any α and β with the property ‖α‖ < ε and‖β‖ < ε, the corresponding HZD exists. For any Ms ≥ 3 and Md ≥ 3, the matricesα ∈ A and β ∈ B are said to be regular18 if α0 = α1 = αMs−1 = αMs = 04×1, β0 =β1 = βMd−1 = βMd = 02×1, and the HZD corresponding to hs(qs;α) and hd(qi;β)exists. Figure 3.2 illustrates the geometry of the HZD for some regular α and β.

18 The terminology of a regular parameter follows Definition 6.1 of [18, p. 140].


Zs

Zs,α

Z

d

Zd,β

Ssd

Sds

Δds

Δsd

Δsd(S

sd∩Z

s)

Sds∩Z

d

Ssd∩Z

s

Δds(S

ds∩Z

d)

Figure 3.2 The geometry of the HZD for some regularα andβ. From Lemma 3.9,Sds ∩ Zs,α =Sds ∩ Zs , Ssd ∩ Zd,β = Ssd ∩ Zd , sd(Ssd ∩ Zd,β) ⊂ Zs,α and ds (Sds ∩ Zs,α) ⊂ Zd,β.

For the regular matrices α and β, the parametric restricted Poincare return mapρα,β : Sds ∩ Zs → Sds ∩ Zs is defined by ρα,β(ζ−s ) := ρs,α ◦ ρd,β(ζ−s ), where ρs,α :Ssd ∩ Zd → Sds ∩ Zs and ρd,β : Sds ∩ Zs → Ssd ∩ Zd are the parametric restrictedgeneralized Poincare maps for the single and double support phases, respectively.Let ρ(ζ−s ;α, β) := ρα,β(ζ−s ). Then, the following discrete-time system can be definedto study the stabilization problem:

ζ−s [k + 1] = ρ(ζ−s [k];α[k], β[k]), (3.58)

where Sds ∩ Zs is the one-dimensional state space of equation (3.58) and αij[k], i =1, ..., 4, j = 2, ...,Ms − 2 and βij[k], i = 1, 2, j = 2, ...,Md − 2 are the control in-puts. Linearization of the discrete-time system in equation (3.58) about (ζ−s , α, β) =(ζ∗s , α∗, β∗) results in

δζ−s [k + 1] = aδζ−s [k] +4∑i=1

Ms−2∑j=2

bαij δαij[k] +2∑i=1

Md−2∑j=2

bβij δβij[k], (3.59)

MOTION PLANNING ALGORITHM 71

where

a := ∂ρ

∂ζ−s(ζ∗s ;α

∗, β∗) = μ

bαij := ∂ρ

∂αij(ζ∗s ;α

∗, β∗)

bβij := ∂ρ

∂βij(ζ∗s ;α

∗, β∗).

Moreover, δζ−s [k] := ζ−s [k] − ζ∗s , δαij[k] := αij[k] − α∗ij = αij[k] and δβij[k] :=

βij[k] − β∗ij = βij[k].

Theorem 3.3 (Static Event-Based Update Laws) Let O be a periodic orbit of thehybrid model (3.22) satisfying hypotheses HPO1–HPO7. Moreover, suppose thatMs,Md ≥ 3, α0 = α1 = αMs−1 = αMs = 04×1, and β0 = β1 = βMd−1 = βMd =02×1. Assume that b /= 01×p, where b := (bα, bβ), bα := (bα12 , . . . , bα4Ms−2 ), bβ :=(bβ12 , . . . , bβ2Md−2 ), and p := 4(Ms − 3) + 2(Md − 3). Then, there exist scalarsKαij , i = 1, . . . , 4, j = 2, . . . ,Ms − 2, and Kβij , i = 1, 2, j = 2, . . . ,Md − 2 suchthat by using the static event-based update laws

αij(ζ−s ) = −Kαij (ζ−s − ζ∗s )

βij(ζ−s ) = −Kβij (ζ−s − ζ∗s ),

(3.60)

ζ∗s is a locally exponentially stable equilibrium point for the closed-loop discrete-timesystem ζ−s [k + 1] = ρcl(ζ−s [k]), where ρcl(ζ−s ) := ρ(ζ−s ;α(ζ−s ), β(ζ−s )).

Proof. b /= 01×p implies the controllability of (a, b). Controllability of (a, b) impliesthe existence ofKα := (Kα12 , ..., Kα4Ms−2 )′ andKβ := (Kβ12 , ..., Kβ2Md−2 )′ such that

|μcl| < 1, where μcl := a− bαKα − bβKβ. Since μcl = ∂ρcl∂ζ−s

(ζ−s )|ζ−s =ζ∗s , |μcl| < 1

follows that ζ∗s is a locally exponentially stable equilibrium point for the closed-loopdiscrete-time system that completes the proof. �

Remark 3.17 (Asymptotic Stability by Static Event-Based Update Laws) Theo-rem 2.5 of Chapter 2 in combination with Theorem 3.3 guarantee that O is an asymp-totically stable periodic orbit for the closed-loop hybrid model of walking.

3.7 MOTION PLANNING ALGORITHM

The objective of this section is to present an algorithm for designing a period-one orbitO of the open-loop hybrid model of walking in equation (3.22) satisfying hypothesesHPO1–HPO7. Like many papers in the literature of the bipedal gait design (e.g.,[61, 73, 84]), the algorithm developed in this chapter is based on a finite-dimensionalnonlinear optimization problem with equality and inequality constraints.


3.7.1 Motion Planning Algorithm for the Single Support Phase

The motion planning algorithm for the single support phase is based on the Spongnormal form [1, 95]. To make this notion precise, let�(qs, qs) := C(qs, qs)qs +G(qs)and partition the dynamical equation (3.7) as follows:

Dbb(qb) qb +Db5(qb) q5 +�b(qs, qs) = uD5b(qb) qb +D55(qb) q5 +�5(qs, qs) = 0,

in which Dbb ∈ R4×4 and Db5 ∈ R4×1 denote the upper left and right submatricesofD, andD5b ∈ R1×4 andD55 ∈ R represent the lower left and right submatrices ofD, respectively. Furthermore, �b ∈ R4×1 and �5 ∈ R are the first four rows and lastrow of the vector �. Applying the static feedback law

u = Dbb(qb) vb + �b(qs, qs),

where

Dbb(qb) := Dbb −Db5D−155 D5b

�b(qs, qs) := �b −Db5D−155 �5,

yields the following partially feedback linearized result that is known as the Spongnormal form:

qb = vbq5 = −D−1

55 (qb)D5b(qb) vb − �5(qs, qs),(3.61)

in which �5(qs, qs) := D−155 �5. The motion planning algorithm during the single

support phase is an extension of that developed in Ref. [73]. From the first four rowsof the partially feedback linearized equation (3.61), body angles can be controlledindependently. Hence, we choose the following polynomial evolution of time for thebody angles during single support:

qbs(t) =ms∑i=0

aiti, 0 ≤ t ≤ Ts, (3.62)

where ms is an integer with the property ms ≥ 4. It is obvious that the coefficientsai ∈ R4 for i = 0, 1, . . . , ms can be obtained uniquely if qbs(t) fulfills the following


boundary conditions:

qbs(0) = qibsqbs(0) = qibs

qbs(Ts) = qfbsqbs(Ts) = qfbsqbs(t

js ) = qjbs, j = 1, . . . , ms − 3,

(3.63)

where the superscripts “i” and “f ” will designate the initial and final conditions,respectively. Moreover, in equation (3.63), tjs := j

ms−2Ts, j = 1, . . . , ms − 3 denotea set of intermediate times during the single support phase at which the body anglesare identical to qjbs. For the later purposes, define the intermediate body angles vectoras

qintbs := (q

′1bs, . . . , q

′ms−3bs )′.

Next, to determine the time evolution of the torso angle during single support, werestrict our attention to the Spong normal form. The last row of matrix equation(3.61) implies that the evolution of the torso angle (i.e., q5s(t)) can be described bythe following differential equation:

q5s(t) = −D−155 (qbs(t))D5b(qbs(t)) qbs(t) − �5(qs(t), qs(t)). (3.64)

Hence, by assuming that q5s(t) satisfies the following boundary conditions

q5s(Ts) = qf5sq5s(Ts) = qf5s,

differential equation (3.64) can be integrated numerically on the time interval [0, Ts].Also, from equations (3.44) and (3.8), us(t) and thereby F1s(t) can be determined

uniquely. Next, let us define qfs := (q′fbs , q

f5s)

′ and qfs := (q′fbs , q

f5s)

′ as the positionand velocity vectors of the mechanical system at the end of the single support phase.Due to the fact that the swing leg contacts the ground at the end of single support, qfssatisfies the following equality constraint:

y2(qfs ) = 0. (3.65)

3.7.2 Motion Planning Algorithm for the Double Support Phase

Assume that qd(t) (i.e., the time evolution of the configuration variables during thedouble support phase) is such that rank ∂p2

∂q(qd(t)) = 2 for every t ∈ [Ts, T ]. Using


2( )pq

2( )pull q

d tqd tq

d tq

Figure 3.3 Geometrical description of the motion planning algorithm during double support.In equation (3.66), it is assumed that rank( ∂p2

∂q(qd(t))) = 2 and qd(t) can be expressed as qd(t) =

q‖d(t) + q⊥

d (t). (See the color version of this figure in color plates section.)

this assumption and equation (3.15), qd(t) fulfills the following differential equation:

qd(t) = −(∂p2

∂q

)+∂

∂q

(∂p2

∂qqd(t)

)qd(t) +

(∂p2

∂q

)−λ(t), (3.66)

where ( ∂p2∂q

)+ and ( ∂p2∂q

)− are the pseudo inverse and projection matrices due to ∂p2∂q

,respectively, that is,

(∂p2

∂q

)+:= ∂p′

2

∂q

(∂p2

∂q

∂p′2

∂q

)−1

(∂p2

∂q

)−:= I5×5 − ∂p′

2

∂q

(∂p2

∂q

∂p′2

∂q

)−1∂p2

∂q,

and λ : [Ts, T ] → R5 is an arbitrary continuously differentiable function. Figure 3.3

illustrates a geometric description for differential kinematic inversion problem (3.66).In this equation, qd(t) can be expressed as

qd(t) = q‖d(t) + q⊥

d (t),

where

q‖d(t) := −

(∂p2

∂q

)+∂

∂q

(∂p2

∂qqd(t)

)qd(t)

q⊥d (t) :=

(∂p2

∂q

)−λ(t).

From the properties of the pseudo inverse and projection matrices, for every t ∈[Ts, T ], q‖

d(t) is perpendicular to q⊥d (t), that is, q‖

d(t)⊥q⊥d (t). Furthermore, q‖

d(t) ∈R(

∂p′2

∂q) and q⊥

d (t) ∈ Null( ∂p2∂q

), whereR andNull represent the range and null spaces


of a matrix, respectively. Next, let us define

λ(t) :=md∑i=0

λi(t − Ts)i, Ts ≤ t ≤ T

for some md ∈ Z+ and λi ∈ R5, i = 0, 1, . . . , md . λi is a vector of five componentsbut its projection by ∂p2

∂qhas only three independent components.19 Next, assume that

qd(t) satisfies the boundary conditions

qd(T ) = qfdqd(T ) = qfd ,

where qfd and qfd denote the position and velocity vectors of the mechanical systemat the end of double support. Then, the equation of motion (3.66) can be integratednumerically on the interval [Ts, T ]. The fact that the end of leg-2 is stationary duringthe double support phase also implies that qfd and qfd satisfy the following equalityconstraints:

p2(qfd ) = p2(qfs )

∂p2

∂q(qfd ) qfd = 02×1.

(3.67)

Moreover, suppose that the step length of the biped robot is nonzero (i.e., Ls =x2(qfs ) = x2(qfd ) /= 0) and choose the following polynomial evolution of time forϒh(t) defined in Lemma 3.7:

ϒh(t) :=md∑i=0

ϒi(t − Ts)i, Ts ≤ t ≤ T,

where ϒi ∈ R, i = 0, 1, . . . , md . In this case, from the proof of Lemma 3.7, the timeevolutions of F2d(t) and ud(t) can be given by

F2d(t) =[

01Ls

]e′5(D qd(t) + C qd(t) +G) +

[ϒh(t)

0

]

ud(t) = u0d(t) − ∂x′2

∂qbϒh(t).

Finally, equation (3.13) determines F1d(t) uniquely.

19 It is worth noting that instead of solving the kinematic inversion problem of p2(q) = (Ls, 0)′, during thedouble support phase, we make use of differential kinematic inversion in equation (3.66) that, in turn,simplifies the kinematic inversion problem but increases the number of variables during the optimizationprocess of the motion planning algorithm.


3.7.3 Constructing a Period-One Orbit for the Open-Loop HybridModel of Walking

By considering Definition 3.3, a solution of the open-loop hybrid model of walkinggiven in equation (3.22) is constructed by piecing together the trajectories of the singleand double support phases, according to the transition maps. Thus, the necessary andsufficient conditions by which the open-loop hybrid model of walking has a period-one solution can be expressed as the following equality constraints:

qd(Ts) = qfsqd(Ts) = dq,s(qfs ) qfs[qibs

q5s(0)

]= Rqfd[

qibs

q5s(0)

]= R qfd .

(3.68)

We remark that from equation (3.68), qibs = H0Rqfd and qibs = H0Rq

fd . Thus, the

evolution of the mechanical system during walking with non-instantaneous doublesupport phase can be completely determined by the following vector of parameters:

x := (q′fs , q

′fs , q

′intbs , q

′fd , q

′fd , λ

′, ϒ′, Ts, Td)′,

where λ := (λ′0, . . . , λ

′md

)′ and ϒ := (ϒ0, . . . , ϒmd )′. Next, to determine an admis-

sible value for the vector of parameters x, we set up an optimization problem. Theconstraints of the optimization problem are composed of equality and inequality con-straints.

3.7.3.1 Equality Constraints The equality constraints are expressed as equa-tions (3.65), (3.67), and (3.68).

3.7.3.2 Inequality Constraints The inequality constraints can be expressed ashypotheses HPO2–HPO7. The constraints associated to the double impact model,studied here, are based on those presented in Refs. [67, 92].

3.7.3.3 Cost Function A two-stage strategy is used to solve the motion planningalgorithm. In the first stage, the cost function is chosen as 1 and by using thefminconfunction of MATLAB’s Optimization Toolbox, we search for a feasible periodicsolution of the open-loop hybrid model of walking (3.22), which will be used inthe next stage as an initial guess. To simplify the search procedure for a feasibleperiodic solution, the constraints can be added in a step-by-step manner. Followingthe results of Refs. [61, 73, 84], by using the fmincon function, the motion planningalgorithm during the second stage is continued to minimize the following desired cost

NUMERICAL EXAMPLE FOR THE MOTION PLANNING ALGORITHM 77

function:

J(x) := 1

Ls

(∫ Ts

0‖us(t)‖2

2 dt +∫ T

Ts

‖ud(t)‖22 dt

).

3.8 NUMERICAL EXAMPLE FOR THE MOTIONPLANNING ALGORITHM

The physical parameters of the walking robot are those of the planar biped robot,RABBIT (see Refs. [47, 96] for more details). On the trajectory O that is obtained byapplying the motion planning algorithm,

qbs(t) =6∑i=0

aiti, 0 ≤ t < Ts

ϒh(t) =6∑i=0

ϒi(t − Ts)i, Ts ≤ t < T.

Moreover, from equation (3.15), qd(t), Ts ≤ t < T is the solution of the followingdifferential equation:

qd(t) = −(∂p2

∂q

)+∂

∂q

(∂p2

∂qqd(t)

)qd(t) +

(∂p2

∂q

)−λ(t)

with the initial condition

qd(Ts) = [0.8414, 2.8239, 0.2344, 2.6577, 1.6134]′(rad)

qd(Ts) = [−0.8496, 0.4765, 0.7515,−0.3317,−0.4291]′(rad/s),

where ( ∂p2∂q

)+ and ( ∂p2∂q

)− denote the pseudo inverse and projection matrices, respec-tively, and

λ(t) =6∑i=0

λi(t − Ts)i, Ts ≤ t < T.

The coefficients ai,λi andϒi for i = 0, . . . , 6 are given in Tables 3.1–3.3, respectively.We remark that for this orbit, the vectors of generalized coordinates and velocitiesimmediately before the impact can be expressed as

qs(Ts) = [0.8414, 2.8239, 0.2344, 2.6577, 1.6134]′(rad)

qs(Ts) = [−0.8698, 0.4794, 0.8155,−0.3471,−0.4325]′(rad/s).


TABLE 3.1 Coefficients ai, i = 0, . . . , 6

a′0 0.6635 2.4577 0.4308 3.0458a′

1 0.7483 −0.3624 −0.5762 0.3019a′

2(101) −2.7279 0.1109 0.9938 0.6606a′

3(102) 1.3010 0.1956 −0.1455 −0.9645a′

4(102) −2.5902 −0.4555 −0.2394 2.6304a′

5(102) 2.3831 0.3001 0.5347 −2.6856a′

6(102) −0.8283 −0.0432 −0.2451 0.9470

TABLE 3.2 Coefficients λi, i = 0, . . . , 6

λ′0 0.6318 0.1778 0.1418 −0.1685 −0.2914λ′

1 0.2220 −0.3819 −0.7544 0.2448 0.1614λ′

2 0.9925 −0.4739 −0.7128 −0.3508 −0.3843λ′

3 −0.1777 0.8382 0.3223 0.8882 0.5165λ′

4 −0.4506 −0.1765 0.3838 −0.8357 −0.0285λ′

5 0.8627 0.9618 0.7079 0.9734 0.8718λ′

6 −0.1908 0.9414 0.1576 0.7550 0.8007

TABLE 3.3 Coefficientsϒi, i = 0, . . . , 6

ϒ0 −0.1980ϒ1 0.2652ϒ2 0.7318ϒ3 −0.3610ϒ4 0.2502ϒ5 −0.9358ϒ6 −0.2292

In addition, by using the impact model developed in Section 3.2.5,

∂p2

∂qA−1

(∂f ′

1

∂q12 + ∂f ′

2

∂q

)=[−0.3882 −0.1121

−0.1121 −0.0739

]

IR1 =[

0.0085

0.0220

]

IR2 =[

0.0832

−0.1244

],

and consequently, the impact model is nonsingular. The feasibility conditions of thedouble impact model, presented in Refs. [67, 92] for a five-link planar bipedal robotwith point feet, imply that the impact model is feasible, that is, IvR1 = 0.0221 > 0,


| IhR1IvR1

| = 0.3847 < μs, | IhR2IvR2

| = 0.6667 < μs, and p2+1 = 0, where μs = 2

3 . Following

the results of Refs. [67, 92] (see equations (3.32)–(3.35)), the condition IvR2 > 0 is notincluded in the feasibility conditions of the double impact mode. However, IvR1 > 0should be satisfied because the stance leg end is assumed to remain on the groundduring and after the impact. It is worth mentioning that the coordinates relabelingto swap the roles of the legs occurs immediately after the double support phase (notafter the impact). In addition, an implicit condition is that before the impact the legis above the ground (see Fig. 3.4) and the velocity of the foot is directed downward.Specially, ∂p2

∂q(qs(Ts))qs(Ts) = [−0.0184,−0.0001]′ and sudden changes in angular

velocities during double impact are

qd(Ts) − qs(Ts) = [0.0202,−0.0029,−0.0640, 0.0154, 0.0034]′.

Sudden changes in the absolute angular velocities⎡⎢⎢⎢⎢⎢⎢⎣

θ1

θ2

θ3

θ4

θ5

⎤⎥⎥⎥⎥⎥⎥⎦

=

⎡⎢⎢⎢⎢⎢⎢⎣

1 1 0 0 −1

0 1 0 0 −1

0 0 1 1 −1

0 0 0 1 −1

0 0 0 0 1

⎤⎥⎥⎥⎥⎥⎥⎦

⎡⎢⎢⎢⎢⎢⎢⎣

q1

q2

q3

q4

q5

⎤⎥⎥⎥⎥⎥⎥⎦

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

−0.4

−0.2

0

0.2

0.4

0.6

Time (s)

x 2 (m

)

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.90

0.05

0.1

0.15

0.2

Time (s)

y 2 (m

)

Figure 3.4 Evolution of the coordinates of the end of leg-2 during the single support phaseof the periodic orbit O.


0 0.2 0.4 0.6 0.81.4

1.45

1.5

1.55

1.6

1.65

(s)

θs (

rad)

0 0.2 0.4 0.6 0.80.1

0.2

0.3

0.4

0.5

0.6

(s)

dθ s/d

t (ra

d/s)

0 0.2 0.4 0.6 0.80

2

4

6

8

(s)

det

(Lgs

L fsh s(q

s))

1 1.1 1.2 1.3 1.40.05

0.1

0.15

0.2

0.25

(s)

xH

d (m

)1 1.1 1.2 1.3 1.4

0.3

0.35

0.4

(s) d

x Hd/d

t (m

/s)

1 1.1 1.2 1.3 1.4−0.16

−0.14

−0.12

−0.1

−0.08

(s)

det

(Lgd

L fdh d(q

i))

Figure 3.5 The validity of hypotheses HPO3–HPO6 for the optimal periodic motion. HPO1is trivially satisfied. From Remark 3.7, if on the orbitOs the time evolution of θs is an increasingfunction of time, invertibility of the decoupling matrix LgsLfshs(qs) on Os is equivalent to theangular momentum about the stance leg end being nonzero during the single support phase(HPO5).

during impact can also be given by [0.0139,−0.0063,−0.0521, 0.0120, 0.0034]′,which are in the acceptable range reported in Ref. [97] (using both the Integrationand Newtonian methods) for a five-link bipedal model with point feet and parametersclose to those of RABBIT. The desired periodic motion O also has a period of T =Ts + Td = 0.9443 + 0.5576 = 1.5018(s), a step length of Ls = 0.3602(m), and anaverage walking speed of 0.2398(m

s ). We remark that in solving the motion planningalgorithm by using the fmincon function of MATLAB’s Optimization Toolbox,the average walking speed of the robot was not fixed; in fact, when the averagewalking speed was chosen outside the interval [0.21, 0.34], the fmincon functioncould not converge to a feasible periodic solution satisfying hypotheses HPO3–HPO7(see Figs. 3.4 and 3.5). From equation (3.40), μ = 1.2522 and, consequently, theperiodic orbit is not stable without applying event-based update laws. Table 3.4 alsorepresents the desired gait statistics.

Figures 3.6 and 3.7 show the angular position and velocity of the knee, hip, and torsojoints during two consecutive steps of the optimal motion, respectively. In plotting the


TABLE 3.4 Desired Gait Statistics

ζ∗s z∗xH �2(x−H,d) Vzero,s(θ−

s ) Wzero,d(x−H,d)

40.8587 0.0971 1.1799 −14.0342 0.0671Vmax

zero,s Wmaxzero,d δds δsd μ

28.4940 0.0671 −0.0397 −25.9528 1.2522

results, the stance and swing legs are generalized to the double support phase. In thedouble support phase (DS), the stance leg is defined to be the leg that was the stanceleg in the previous single support phase (SS). The definition of the swing leg in thedouble support phase is analogous. The control inputs during two consecutive steps ofthe optimal motion are also depicted in Fig. 3.8. From Fig. 3.8, ‖u‖L∞ < umax and atthe transitions between the continuous phases, the control inputs have discontinuity.Figure 3.9 shows the horizontal and vertical components of the ground reaction forcesat the end of the legs during two consecutive steps.

0 0.5 1 1.5 2 2.50.2

0.4

0.6

0.8

1

(s)

Sta

nce

knee

(ra

d)

0 0.5 1 1.5 2 2.5

2.6

2.8

3

(s)

Sta

nce

hip

(rad

)

0 0.5 1 1.5 2 2.50.2

0.3

0.4

0.5

0.6

(s)

Sw

ing

knee

(ra

d)

0 0.5 1 1.5 2 2.5

2.6

2.8

3

(s)

Sw

ing

hip

(rad

)

0 0.5 1 1.5 2 2.51.3

1.4

1.5

1.6

(s)

Tor

so (

rad)

DS

SS

Figure 3.6 Angular positions of the knee, hip, and torso joints during two consecutive stepsof the optimal motion. The discontinuities are due to the coordinate relabling for swapping therole of the legs. (See the color version of this figure in color plates section.)


0 0.5 1 1.5 2 2.5−2

−1

0

1

2

(s)

Sta

nce

knee

(ra

d/s)

0 0.5 1 1.5 2 2.5

−1

0

1

(s)

Sta

nce

hip

(rad

/s)

0 0.5 1 1.5 2 2.5−2

−1

0

1

2

(s)

Sw

ing

knee

(ra

d/s)

0 0.5 1 1.5 2 2.5

−1.5

−1

−0.5

0

0.5

1

(s) S

win

g hi

p (r

ad/s

)

0 0.5 1 1.5 2 2.5−1

−0.5

0

0.5

1

(s)

Tor

so (

rad/

s)

DS

SS

Figure 3.7 Angular velocities of the knee, hip and torso joints during two consecutive stepsof the optimal motion. The discontinuities are due to the coordinate relabling for swapping therole of the legs. (See the color version of this figure in color plates section.)

3.9 SIMULATION RESULTS OF THE CLOSED-LOOP HYBRID SYSTEM

This section presents a numerical example for the proposed control strategy to asymp-totically stabilize the desired period-one orbit O for the hybrid model of walking inequation (3.22). Figure 3.10 depicts the plot of the functions hd,s(θs), Vzero,s(θs),hd,d(xH ), u1d(xH ), u2d(xH ), �2(xH ), and Wzero,d(xH ).

3.9.1 Effect of Double Support Phase on Angular MomentumTransfer and Stabilization

Let σ1 and σ2 be the angular momenta of the robot about the end of leg-1 and leg-2, respectively. From the angular momentum balance theorem20 with the clockwise

20 From the angular momentum balance theorem, the time derivative of the angular momentum about afixed point is equal to the sum of moments applied by the external forces about that point.

SIMULATION RESULTS OF THE CLOSED-LOOP HYBRID SYSTEM 83

0 0.5 1 1.5 2 2.5−90

−80

−70

−60

−50

−40

−30

−20

−10

(s)

Sta

nce

knee

(N

m)

0 0.5 1 1.5 2 2.5

−20

0

20

40

60

(s)

Sta

nce

hip

(Nm

)

0 0.5 1 1.5 2 2.5−15

−10

−5

0

5

10

(s)

Sw

ing

knee

(N

m)

0 0.5 1 1.5 2 2.5−50

−40

−30

−20

−10

0

10

20

30

(s)

Sw

ing

hip

(Nm

) DS

SS

Figure 3.8 Open-loop control inputs corresponding to the knee and hip joints during twoconsecutive steps of the optimal motion. Two types of discontinuity due to the transitionsbetween the continuous phases are shown in the graphs. (See the color version of this figure incolor plates section.)

convention, the time derivatives of σ1 and σ2 in double support can be expressed as

σ1 = mtotg0xcm − LsFv2σ2 = −mtotg0(Ls − xcm) + LsFv1 ,

(3.69)

where Fvi , i = 1, 2 denotes the vertical component of the ground reaction force at theend of leg-i. Next, assume that the impact occurs at time t = 0 and the subscripts“s” and “d” represent the single and double support phases, respectively. Then thevariation of σ2 during double support can be expressed as

σ−2,d − σ+

2,d := σ2(t−d ) − σ2(0+)

=∫ t−

d

0+−mtotg0(Ls − xcm) + LsFv1 dt,

(3.70)


0 0.5 1 1.5 2 2.5−20

−10

0

10

20

30

40

(s) Sta

nce

leg

end

horiz

onta

l for

ce (

N)

0 0.5 1 1.5 2 2.50

100

200

300

400

500

600

(s)

Sta

nce

leg

end

vert

ical

forc

e (N

)

0 0.5 1 1.5 2 2.5−0.2

−0.15

−0.1

−0.05

0

0.05

0.1

0.15

(s)

Sw

ing

leg

end

horiz

onta

l for

ce (

N)

0 0.5 1 1.5 2 2.50

50

100

150

200

250

(s)

Sw

ing

leg

end

vert

ical

forc

e (N

)

DS

SS

Figure 3.9 Horizontal and vertical components of the ground reaction forces experienced atthe end of the legs during two consecutive steps of the optimal motion. (See the color versionof this figure in color plates section.)

where σ−2,d := σ2(t−d ), σ+

2,d := σ2(0+), and t = 0+ and t = t−d denote the time in-stances just after the impact and just before the takeoff, respectively. In addition, tdrepresents the time duration of the double support phase. At impact, σ2 is not affectedby the impulsive reaction force IR2 because IR2 acts at the end of leg-2. Hence,

σ+2,d − σ−

2,s = IvR1Ls, (3.71)

in which σ−2,s := σ2(0−), and t = 0− represents the time instance just before the

impact. Furthermore, according to the principle of angular momentum transfer[18, pp. 421,430], σ−

2,s can be expressed in terms of σ−1,s, that is,

σ−2,s = σ−

1,s +mtotLsy−cm,s, (3.72)

where σ−1,s := σ1(0−) and y−

cm,s is the vertical component of the velocity of the COMimmediately before impact. After relabeling, the roles of the legs are swapped and,


1.45 1.5 1.55 1.60 0.51 1.52 2.53 3.5

θs (rad)

hd,

s

θs+ θ

s−

0.1 0.15 0.20.4

0.6

0.8

1

1.2

1.4

1.6

xH

(m)

hd,

d

xH,d+ x

H,d− 0.1 0.15 0.2

−30

−20

−10

0

10

xH

(m) u

1d, u

2d

xH,d+ x

H,d−

1.45 1.5 1.55 1.6−20

−10

0

10

20

30

θs (rad)

Vze

ro,s

θs+ θ

s−

0.1 0.15 0.21

1.05

1.1

1.15

1.2

xH

(m)

Ω2

xH,d+ x

H,d− 0.1 0.15 0.2

0

0.02

0.04

0.06

0.08

0.1

xH

(m)

Wze

ro,d

xH,d+ x

H,d−

h1d,d

h2d,d

h1d,s

h2d,s

h3d,s

h4d,s

u1d

u2d

Figure 3.10 Graphs of the functions hd,s(θs), Vzero,s(θs), hd,d(xH ), u1d(xH ), u2d(xH ), �2(xH ),and Wzero,d(xH ).

therefore,

σ+1,s := σ1(t+d ) = σ−

2,d, (3.73)

in which σ+1,s := σ1(t+d ) is the value of σ1 at the beginning of the next single support

phase. Finally, equations (3.70)–(3.73) result in

σ+1,s = σ−

1,s +mtotLsy−cm,s + IvR1Ls

+∫ t−

d

0+−mtotg0(Ls − xcm) + LsFv1 dt.

(3.74)

In addition, equation (3.26) implies that on the manifold Sds ∩ Zs, q = q−s and q =λs(q−s )σ−

1,s which together with equation (3.21) and pcm = f1(q) result in

y−cm,s = ∂f v1

∂q(q−s ) q = ∂f v1

∂q(q−s ) λs(q

−s ) σ−

1,s =: λvcm,s(q−s ) σ−

1,s

IvR1 = v1(q−s ) q = v1(q−s ) λs(q−s ) σ−

1,s =: λvI,1(q−s ) σ−1,s,

(3.75)


where f v1 andv1 represent the second rows of f1 and1, respectively. Consequently,equation (3.74) can be rewritten as follows:

σ+1,s = (1 +mtotLsλ

vcm,s(q

−s ) + LsλvI,1(q−s )

)σ−

1,s

+∫ t−

d

0+−mtotg0(Ls − xcm) + LsFv1 dt.

(3.76)

By defining

δzero := 1 +mtotLsλvcm,s(q

−s ) + LsλvI,1(q−s )

χ :=∫ t−

d

0+−mtotg0(Ls − xcm) + LsFv1 dt

and using equation (3.39), equation (3.76) can be rewritten as follows:

σ+1,s = �(σ−

1,s) = δzero σ−1,s + χ(σ−

1,s). (3.77)

It is worth mentioning that the effect of the double support phase on momentumtransfer is given by the term χ that can also be expressed in terms of σ−

1,s. To makethis notion precise, we observe that

χ =∫ t−

d

0+−mtotg0(Ls − xcm) + LsFv1 dt

=∫ x−

H,d

x+H,d

−mtotg0(Ls − xcm(xH )) + LsFv1 (xH , vxH )

vxHdxH ,

where from equation (3.33) and v+xH = δds (q−s )σ−1,s, vxH can be expressed in terms of

σ−1,s. Finally, we are able to represent effect of u1 and u2 on χ and stabilization.

Lemma 3.10 Let O = Os ∪ Od be a periodic orbit of the hybrid model (3.22) sat-isfying hypotheses HPO1–HPO7. Assume that σ+∗

1,s and σ−∗1,s are the initial and final

values of σ1,s on Os, respectively. Then,

dχ

dσ−1,s

(σ−1,s)|σ−

1,s=σ−∗1,s

is independent of the choice of the continuous functions u1 = u1d(xH ) and u2 =u2d(xH ).

Proof. As discussed previously, the single support phase zero dynamics is Lagrangianand therefore, Ezero = 1

2 (σ1,s)2 + Vzero,s(θs) is stationary. Let us consider the


0 0.5 1 1.5−0.2

−0.1

0

0.1

0.2

Hor

izon

tal p

ositi

on o

f CO

M (

m)

Time (s) 0 0.5 1 1.5

0.64

0.66

0.68

0.7

0.72

0.74

Ver

tical

pos

ition

of C

OM

(m

)

Time (s)

0 0.5 1 1.5−0.2

0

0.2

0.4

0.6

Hor

izon

tal v

eloc

ity o

f CO

M (

m/s

)

Time (s) 0 0.5 1 1.5

−0.3

−0.2

−0.1

0

0.1

Ver

tical

vel

ocity

of C

OM

(m

/s)

Time (s)

−0.1 0 0.1 0.20.64

0.66

0.68

0.7

0.72

0.74

Ver

tical

pos

ition

of C

OM

(m

)

Horizontal position of COM (m) −0.1 0 0.1 0.2

−0.3

−0.2

−0.1

0

0.1

Ver

tical

vel

ocity

of C

OM

(m

/s)

Horizontal position of COM (m)

DS

SS

Figure 3.11 Plot of the vertical height and velocity of the COM versus time and the horizontalposition of the COM on O. At the impact, the velocity of the COM is not pointed downward.(See the color version of this figure in color plates section.)

restricted Poincare return map in the coordinates (θs, σ1,s) by σ−1,s → P(σ−

1,s) and

P(σ−1,s) :=

√(σ+

1,s)2 − 2Vzero,s(θ

−s )

=√

(�(σ−1,s))

2 − 2Vzero,s(θ−s ).

(3.78)

Then the derivative of P with respect to σ−1,s evaluated at σ−

1,s = σ−∗1,s can be expressed

as

dP

dσ−1,s

(σ−1,s)|σ−

1,s=σ−∗1,s

=�(σ−∗

1,s )d�

dσ−1,s

(σ−∗1,s )√

(�(σ−∗1,s ))

2 − 2Vzero,s(θ−s )

= σ+∗1,s

σ−∗1,s

(δzero + dχ

dσ−1,s

(σ−1,s)|σ−

1,s=σ−∗1,s

).

(3.79)


0 2 4 6

0.4

0.6

0.8

(s)

Sta

nce

knee

(ra

d)

0 2 4 6

2.6

2.8

3

(s)

Sta

nce

hip

(rad

)

0 2 4 60.2

0.3

0.4

0.5

0.6

(s)

Sw

ing

knee

(ra

d)

0 2 4 6

2.6

2.8

3

(s) S

win

g hi

p (r

ad)

0 2 4 61.3

1.4

1.5

1.6

(s)

Tor

so (

rad)

DS SS

Figure 3.12 Configuration variables during five steps of the closed-loop simulation. Discon-tinuities in the graphs are due to coordinate relabling. (See the color version of this figure incolor plates section.)

On the other hand, the restricted Poincare return map in the coordinates (θs, ζs)for Zs was expressed as ρ(ζ−s ) in Section 3.4.3. In addition from Remark 4,μ = dρ

dζ−s(ζ−s )|ζ−s =ζ−∗

sis independent of the continuous functions u1 = u1d(qi) and

u2 = u2d(qi). Finally, the fact that δzero is dependent only on the orbit Os and

P(σ−1,s) =

√2ρ( 1

2 (σ−1,s)

2) completes the proof. �

Corollary 3.1 The effects of the double support phase on angular momentum transferand stabilization are given byχ and dχ

dσ−1,s

(σ−1,s)|σ−

1,s=σ−∗1,s

, respectively. In addition, from

Lemma 3.10, dχ

dσ−1,s

(σ−1,s)|σ−

1,s=σ−∗1,s

is independent of continuous functions u1d(qi) and

u2d(qi).

In this chapter, u1d and u2d are designed as the solution of a nonlinear optimizationproblem (3.56). By this means,


0 2 4 6−2

−1

0

1

2

(s)

Sta

nce

knee

(ra

d/s)

0 2 4 6

−1

0

1

(s)

Sta

nce

hip

(rad

/s)

0 2 4 6−2

−1

0

1

2

(s)

Sw

ing

knee

(ra

d/s)

0 2 4 6

−2

−1

0

1

(s) S

win

g hi

p (r

ad/s

)

0 2 4 6−1

−0.5

0

0.5

1

(s)

Tor

so (

rad/

s)

DS SS

Figure 3.13 Velocity variables during five steps of the closed-loop simulation. Discontinuitiesin the graphs are due to coordinate relabling. (See the color version of this figure in color platessection.)

1. Od is an integral curve of the double support phase dynamics (according to theequality constraint);

2. the control input and ground reaction forces corresponding to Od are feasible(according to the inequality constraints);

3. the control input corresponding to Od is minimum norm.

For the periodic orbit O obtained from the motion planning algorithm, the verticalcomponent of the velocity of the COM just before the impact is positive (see Fig. 3.11),and hence, δzero ∼= 1.1199 > 1. In addition, dχ

dσ−1,s

(σ−1,s)|σ−

1,s=σ−∗1,s

= −0.0018. Hence,

σ+∗1,s

σ−∗1,s

(δzero + dχ

dσ−1,s

(σ−1,s)|σ−

1,s=σ−∗1,s

)= 1.2522 > 1.

Since the Jacobian of the restricted Poincare return map without the double supportphase is δ2zero = 1.2541 > μ, the double support phase controller does not destabilizeO. However, it does not have sufficient strength to overcome the terms δzero > 1 nor


0 2 4 6

−80

−60

−40

−20

0

(s)

Sta

nce

knee

(N

m)

0 2 4 6−10

0

10

20

30

40

50

60

70

(s)

Sta

nce

hip

(Nm

)

0 2 4 6−25

−20

−15

−10

−5

0

5

10

(s)

Sw

ing

knee

(N

m)

0 2 4 6−50

−40

−30

−20

−10

0

10

(s)

Sw

ing

hip

(Nm

)

DS

SS

Figure 3.14 Control inputs (i.e., joint torques) during five steps of the closed-loop simulation.Discontinuities in the graphs are due to transition between the continuous phases. (See the colorversion of this figure in color plates section.)

stabilize O without applying the event-based update laws. We remark that in contrastto the approach proposed in Ref. [71], the proposed within-stride control law duringthe double support phase is continuous and utilizes input–output linearization to obtaina nontrivial HZD.21 Since the periodic orbit is not stable without applying the secondlevel of the control scheme, the output functions are modified as in equation (3.57).The degrees of the Bezier polynomials are chosen asMs = 4 andMd = 4. Then, thecoefficients bα and bβ can be calculated numerically as follows:

bα = [25.2776, 49.7273,−5.4290,−13.8925]

bβ = [298.2165,−25.2154],

21 Since the approach of this chapter is based on Theorem 4.1 of Ref. [18, p. 89], the closed loop hybridsystem has continuous vector fields during the single and double support phases to satisfy hypothesesHSH1 and HSH2 of Ref. [18, p. 83]. We also remark that while solving the motion planning algorithmby using the fmincon function, the inequality constraint y−

cm,s < 0 was not imposed on the motionplanning algorithm.


0 2 4 6−20

−10

0

10

20

30

40

(s) Sta

nce

leg

end

horiz

onta

l for

ce (

N)

0 2 4 6

100

200

300

400

500

(s)

Sta

nce

leg

end

vert

ical

forc

e (N

)

0 2 4 6−40

−30

−20

−10

0

(s)

Sw

ing

leg

end

horiz

onta

l for

ce (

N)

0 2 4 60

50

100

150

200

250

(s)

Sw

ing

leg

end

vert

ical

forc

e (N

)

DS

SS

Figure 3.15 Horizontal and vertical components of the ground reaction forces at the stanceand swing leg ends during five steps of the closed-loop simulation. The four graphs depict theadmissibility of the ground reaction forces. (See the color version of this figure in color platessection.)

and consequently (a, b) is controllable. The gains of the static update laws Kα andKβ can be calculated via DLQR subject to the linearized system in equation (3.59).22

Calculation forQ = 1 andR = 100 × I6×6 by thedlqr function of MATLAB yields

Kα = 10−3 × [0.3371, 0.6631,−0.0724,−0.1853]′

Kβ = 10−2 × [0.3977,−0.0336]′.

In this case, the Jacobian of the closed-loop restricted Poincare map ρcl evaluated atζ∗s can be calculated as μcl = a− bαKα − bβKβ = 0.0013 < 1. Thus, by using theevent-based controller, the periodic orbit O is an asymptotically stable limit cycle forthe closed-loop hybrid model of walking.

22 In Ref. [61], the DLQR design method has been used in control of walking of an underactuated 3Dbiped robot.


Figure 3.16 Stick diagram of the five-link, four-actuator biped robot taking three steps fromleft to right. (See the color version of this figure in color plates section.)

3.9.2 Effect of Event-Based Update Laws on Momentum Transferand Stabilization

To investigate the effect of event-based update laws on angular momentum transferand stabilization, let us assume that only the parameters of the double support phasecontroller (i.e., β) are updated and the parameters of the single support phase con-troller are held constant, that is, α = α∗. Then, χ can be expressed as χ(σ−

1,s;β) andthereby,

dP

dσ−1,s

(σ−1,s)|σ−

1,s=σ−∗1,s

= σ+∗1,s

σ−∗1,s

(δzero + ∂χ

∂σ−1,s

(σ−∗1,s ;β

∗) − ∂χ

∂β(σ−∗

1,s ;β∗)Kβ

).

According to this latter equation, − ∂χ∂β

(σ−∗1,s ;β

∗)Kβ is an auxiliary term introduced bythe event-based update laws to stabilize O.

To confirm the analytical results obtained in this chapter, the simulation of theclosed-loop hybrid model of walking is started at the end of single support phase.The initial condition of the configuration vector is assumed to be q−s . However, theinitial condition for the velocity vector is chosen as the value of the velocity vectorat the end of single support phase on the periodic orbit with an error of +2◦s−1

on each of its component. Figures 3.12–3.16 depict results of the simulation of theclosed-loop system. Figures 3.12 and 3.13 represent the angular position and velocity


of the knee, hip, and torso joints during five consecutive steps of the closed-loopsystem, respectively. The control inputs during five steps of walking are depicted inFig. 3.14. Figure 3.15 also displays the horizontal and vertical components of theground reaction forces at the end of the legs during five consecutive steps. Finally,the stick diagram of the five-link, four-actuator biped robot taking three steps fromleft to right is presented in Fig. 3.16.

CHAPTER 4

Asymptotic Stabilization of PeriodicOrbits for Planar Monopedal Running

4.1 INTRODUCTION

This chapter presents an analytical approach for designing a two-level control lawto asymptotically stabilize a desired period-one orbit during running by a planarmonopedal robot. The monoped robot is a three-link, two-actuator planar mecha-nism in the sagittal plane with point foot. The desired periodic orbit is generatedby the method developed in Ref. [73]. It is assumed that the model of monopedalrunning can be expressed by a hybrid system with two continuous phases, includ-ing stance phase (one leg on the ground) and flight phase (no leg on the ground),and discrete transitions between the continuous phases, including takeoff and landing(impact).

The configuration of the mechanical system is specified by the absolute orientationwith respect to an inertial world frame and by the joint angles determining the shapeof the robot. During the flight phase, the angular momentum of the mechanical systemabout its COM is conserved. To reduce the dimension of the full-order hybrid modelof running, which, in turn, simplifies the stabilization problem of the desired orbit,as proposed in Ref. [55], we desire that the configuration of the mechanical systemcan be transferred from a specified initial pose (immediately after the takeoff) to aspecified final pose (immediately before the landing) during flight phases. This prob-lem is referred to as landing in a fixed configuration or configuration determinism atlanding [18, p. 252]. However, the flight time and angular momentum about the COMmay differ during consecutive steps. Consequently, the reconfiguration problem mustbe solved online. A number of control problems for reconfiguration of a planar mul-tilink robot with zero angular momentum have been considered in the literature, forexample, [74–78]. For the case that the angular momentum is not necessarily zero,a method based on the Averaging Theorem [79, Theorem 2.1] was presented in [80]


95

96 ASYMPTOTIC STABILIZATION OF PERIODIC ORBITS FOR PLANAR MONOPEDAL RUNNING

such that for any value of the angular momentum, joint motions can reorient the multi-link arbitrarily over an arbitrary time interval. However, when the angular momentumis not zero, this method cannot be employed online for solving the reconfigurationproblem for monopedal running. For this reason, we present an online reconfigura-tion algorithm that solves this problem for given flight times and angular momenta[81, 82]. The algorithm proposed in this chapter is expressed using the methodologyof reachability and optimal control for time-varying linear systems with input andstate constraints.

Probably the most basic tool for analyzing the stability of periodic orbits oftime-invariant dynamical systems described by ordinary differential equations is thePoincare first return map that establishes an equivalence between the stability analysisof the periodic orbit for an nth-order continuous-time system and that of the corre-sponding equilibrium point for an (n − 1)th-order discrete-time system. Grizzle et al.[46] showed that the Poincare return map can be applied to systems with impulseeffects for analyzing the stability of periodic orbits. To reduce the dimension of thePoincare return map during bipedal walking with one degree of underactuation, thestrategy of using virtual constraints has been developed in Refs. [46–51]. For coor-dination of robot links, a set of holonomic output functions, referred to as virtualconstraints, are defined and imposed to be zero by a feedback law [47]. For the casethat the corresponding zero dynamics manifold is impact invariant, the concept ofHZD was introduced in Ref. [52] that, in turn, results in a one-dimensional restrictedPoincare return map with a closed-form expression. To create impact invariance dur-ing bipedal walking with more than one degree of underactuation, a new approachbased on parameterization of the outputs and updating their parameters in a stride-to-stride manner was presented in [60]. Following this strategy, asymptotically stablewalking for an underactuated spatial biped robot is described in Ref. [61]. By using thevirtual constraints approach, the configuration determinism at landing was proposedin [55] to obtain a closed-form expression for the one-dimensional restricted Poincarereturn map of running by a five-link, four-actuator planar bipedal robot. Moreover,to ensure that the stance phase zero dynamics manifold is hybrid invariant under theclosed-loop hybrid model of running, an additional constraint was imposed on thevector of generalized velocities at the end of flight phases. To satisfy the configurationdeterminism at landing and hybrid invariance, Ref. [55] utilized the approach of pa-rameterized HZD. Specifically, on the basis of the Implicit Function Theorem and anumerical nonlinear optimization problem with an equality constraint, the parametersof the virtual constraints of the flight phase were updated in a step-by-step fashionduring the discrete transition from stance to flight (i.e., takeoff). However, the stancephase controller was assumed to be fixed.

The main contribution of this chapter is to present an analytical approach for onlinegeneration of twice continuously differentiable (C2) modified reference trajectoriesduring flight phases of running to satisfy the configuration determinism at landing[81]. Moreover, by relaxing the constraint of Ref. [55] on the vector of generalizedvelocities at the end of the flight phases, we present a two-level control scheme basedon the reconfiguration algorithm to asymptotically stabilize a desired periodic orbit.

MECHANICAL MODEL OF A MONOPEDAL RUNNER 97

In this scheme, within-stride controllers, including stance and flight phase controllers,are employed at the first level. The stance phase controller is chosen as a time-invariantand parameterized feedback law to generate a family of finite-time attractive, zerodynamics manifolds. An alternative approach based on continuous feedback law isemployed here to track the modified reference trajectories generated by the recon-figuration algorithm during the flight phase. To generate a family of hybrid invariantmanifolds, an event-based controller updates the parameters of the stance phase con-troller during the transition from flight to stance (i.e., impact) [81]. Consequently,the stability properties of the desired periodic orbit can be analyzed and modified bya one-dimensional discrete-time system defined on the basis of a restricted Poincarereturn map (see Section 2.4).

4.2 MECHANICAL MODEL OF A MONOPEDAL RUNNER

4.2.1 The Monopedal Runner

A planar three-link monopedal robot with two ideal revolute joints and point foot (seeFig. 4.1) is considered throughout this chapter. The joints are controlled by internalactuators. Also, it is assumed that torques cannot be applied at the leg end. For thelater uses, a coordinate frame is assumed to be attached to the ground called the worldframe.


A convenient choice of the configuration variables consists of the body angles, theabsolute orientation, and the absolute position of the monoped with respect to the

1

2

s

1 1,x y

Figure 4.1 Flight (left) and stance phases (right) during running of the monopedal robot. Thevirtual leg is depicted by the dashed line connecting the stance leg end and the hip joint.


world frame. The body angles represented by ϕ := (ϕ1, ϕ2)′ describe the shape of therobot, where prime denotes matrix transpose. The absolute orientation of the robot isrepresented by θ, and the absolute position is represented by the Cartesian coordinatesof its COM, pcm := (xcm, ycm)′. Consequently, the generalized coordinates during theflight phase are defined as qf := (ϕ′, θ, p′

cm)′ = (q′, p′cm)′, where q := (ϕ′, θ)′.

Following the notation of Ref. [18, Chapter 3], the dynamical model during theflight phase can be expressed as the following second-order equation:

A(ϕ) q + C(ϕ, q) q = B u (4.1)

mtot xcm = 0 (4.2)

mtot ycm + mtot g0 = 0, (4.3)

in which A is a (3 × 3) mass-inertia matrix, C is a (3 × 3) matrix containing theCoriolis and centrifugal terms, mtot is the total mass of the robot, u := (u1, u2)′ is avector of actuator torques, g0 is the gravitational constant, and

B :=[I2×2

01×2

].

The configuration space for the flight phase, Qf , is also chosen to be a simply-connected and open subset of (0, π

2 ) × (0, 3π2 ) × (−π

2 , π2 ) × R2. By introducing

xf := (q′f , q′

f )′ as the state vector, the evolution of the mechanical system duringthe flight phase can be expressed in the state space form xf = ff (xf ) + gf (xf )u.Moreover, the state manifold for the flight phase is chosen as the tangent bundle ofQf , that is, Xf := TQf .

4.2.3 Dynamics of the Stance Phase

Using the principle of virtual work, a reduced-order model for describing the evolutionof the mechanical system during the stance phase can be obtained as follows:

D(ϕ) q + C(ϕ, q) q + G(q) = B u, (4.4)

where

D(ϕ) := A + mtot∂f1

∂q

′ ∂f1

∂q

C(ϕ, q) := C + mtot∂f1

∂q

′ ∂

∂q

(∂f1

∂qq)

G(q) := mtot∂f1

∂q

′ [ 0

g0

]

RECONFIGURATION ALGORITHM FOR THE FLIGHT PHASE 99

and f1 is a smooth function of the configuration variables such that (x1, y1)′ =p1(qf ) := pcm − f1(q) ∈ R2 denotes the Cartesian coordinates of the leg end (seeFig. 4.1). By defining the state vector of the stance phase as xs := (q′

s, q′s)

′, whereqs := q and qs := q, equation (4.4) can be represented in state space form byxs = fs(xs) + gs(xs)u. The state manifold is chosen as Xs := TQs, in which Qs isthe configuration space of the stance phase and assumed to be a simply connectedand open subset of (0, π

2 ) × (0, 3π2 ) × (−π

2 , π2 ).

4.2.4 Open-Loop Hybrid Model of Running

Following the modeling method presented in [55], the open-loop model of monopedalrunning can be expressed by a nonlinear hybrid system consisting of the followingstance and flight phase state manifolds:

�s :

⎧⎪⎪⎨⎪⎪⎩

xs = fs(xs) + gs(xs) u, x−s /∈ Sf

s

x+f = �

fs (x−

s ), x−s ∈ Sf

s

Sfs := {xs ∈ Xs | Hf

s (xs) = 0}

�f :

⎧⎪⎨⎪⎩

xf = ff (xf ) + gf (xf ) u, x−f /∈ Ss

f

x+s = �s

f (x−f ), x−

f ∈ Ssf

Ssf := {xf ∈ Xf | Hs

f (xf ) = 0}.

(4.5)

In equation (4.5), the superscripts “−” and “+” denote the state of the hybrid systemimmediately before and after the switching between the state manifolds, respectively.We assume that the takeoff switching hypersurface Sf

s can be defined as the zero levelset of the smooth function H

fs (xs) := γs(qs) − γ−

s , where γs(qs) is the angle of thevirtual leg with respect to the world frame (see Fig. 4.1) and γ−

s is a threshold value.Moreover, the impact switching hypersurface Ss

f is defined as the zero level set ofHs

f (xf ) := y1(qf ) (see Fig. 4.1). �fs : Sf

s → Xf and �sf : Ss

f → Xs also representthe takeoff and impact maps, respectively.

4.3 RECONFIGURATION ALGORITHM FOR THE FLIGHT PHASE

The conservation of angular momentum about the COM of the monopedal robotduring the flight phase is expressed in the third row of matrix equation (4.1) that can


be rewritten as follows1:

θ = σcm

A3,3(ϕ)−

2∑i=1

A3,i(ϕ)

A3,3(ϕ)ϕi

= σcm

A3,3(ϕ)− J(ϕ) ϕ,

(4.6)

where σcm is a constant representing the angular momentum of the mechanical systemabout its COM and

J(ϕ) := 1

A3,3(ϕ)[A3,1(ϕ) A3,2(ϕ)] ∈ R1×2.

Also Qb is the body configuration space of the mechanical system and is assumed tobe a simply connected and open subset of (0, π

2 ) × (0, 3π2 ).

Remark 4.1 (ϕ as the Control Input for the Dynamical System (4.6)) Since θ is acyclic variable [1] for the mechanical system during the flight phase, the mass-inertiaand Coriolis matrices in equation (4.1) are independent of θ. Hence, the right-handside of equation (4.6) is expressed as a function of ϕ and ϕ, and we can study thefollowing dynamical system:

ϕ = ν

θ = σcm

A3,3(ϕ)− J(ϕ) ν

(4.7)

with the state space Q := Qb × S1 and the control ν, where S1 := [0, 2π) denotesthe unit circle.

Fundamental Assumption We will assume that the takeoff and landing occur infixed configurations. In particular, we present the following problem.

Problem 4.1 (Boundary Conditions on Configuration Variables) Assume that atwice continuously differentiable (i.e., C2) nominal trajectory ϕ∗ : [t∗1 , t∗2 ] → Qb (theevolution of body angles on a nominal trajectory) can transfer the configuration of themonoped robot during the flight phase from the initial condition q∗

1 := [ϕ′∗(t∗1 ), θ1]′to the final condition q∗

2 := [ϕ′∗(t∗2 ), θ2]′ when the angular momentum about its COMis identically equal to σ∗

cm, that is,

θ2 = θ1 +∫ t∗2

t∗1

( σ∗cm

A3,3(ϕ∗(s))− J(ϕ∗(s)) ϕ∗(s)

)ds.

1 Because matrix A(ϕ) is positive definite, A3,3(ϕ) > 0 for any ϕ ∈ Qb.


*1q σ

σ

*2q

* *1t

12

2

Projection onto b1

****1 2

2

: ,: ,b

b

t t t tt t t t

* *2t

*cm

cm

1

Figure 4.2 Geometric description of Problem 4.1 in the state space of the dynamical systemgiven in equation (4.7). (See the color version of this figure in color plates section.)

Next, let the angular momentum about the COM be σcm, where σcm /= σ∗cm. The

objective of this section is to present an online algorithm for generating the trajectoryϕ : [t1, t2] → Qb based on the nominal trajectory ϕ∗ such that the configuration ofthe mechanical system can be transferred from the initial condition q∗

1 to the finalcondition q∗

2 , where t1 /= t∗1 and t2 /= t∗2 . In other words, we look for a C2 function ϕ

such that (i) ϕ(t1) = ϕ∗(t∗1 ), (ii) ϕ(t2) = ϕ∗(t∗2 ), and (iii)

θ2 = θ1 +∫ t2

t1

( σcm

A3,3(ϕ(t))− J(ϕ(t)) ϕ(t)

)dt.

Figure 4.2 represents a geometric description for Problem 4.1 in the state space ofsystem (4.7). In this figure, the nominal C1 input ν∗(t) := ϕ∗(t), t∗1 ≤ t ≤ t∗2 transfersthe state of the system from the initial point q∗

1 to the final point q∗2 when the angular

momentum about the COM is equal to σ∗cm. The objective is to generate the C1

input ν(t) := ϕ(t), t1 ≤ t ≤ t2 transferring the state of the system from q∗1 to q∗

2 whenσcm /= σ∗

cm. In addition,

C∗ := {ψ ∈ Qb|ψ = ϕ∗(t), t∗1 ≤ t ≤ t∗2 }C := {ψ ∈ Qb|ψ = ϕ(t), t1 ≤ t ≤ t2}

denote the projections of the nominal and generated trajectories onto the body config-uration space Qb. The results of this section will be utilized in Section 4.4 to reducethe dimension of the full-order hybrid model of the monopedal robot and therebysimplify the stabilization problem in Section 4.5.

Integrating equation (4.6) over the time interval [t1, t2] results in

θ(t2) = θ1 +∫ t2

t1

σcm

A3,3(ϕ(t))dt −

∫CJ(ϕ) dϕ. (4.8)


*1q

*2q

* *1t

12

2

Projection onto b

****1 2: ,b t t t t

* *2t

*cm

cm

1

Figure 4.3 Geometric description of the proposed reconfiguration algorithm. In this algo-rithm, it is assumed thatϕ(t) = ϕ∗(τ(t)), where τ : [t1, t2] → [t∗1 , t

∗2 ] is the virtual time satisfying

the constraints given in equation (4.9). Thus, C = C∗ and the geometric phases correspondingto the trajectories ϕ and ϕ∗ are equal. (See the color version of this figure in color platessection.)

The second and third terms in the right-hand side of equation (4.8) are called thedynamic and geometric phases, respectively [80, 98]. The dynamic term is nonzeroif and only if the angular momentum σcm is nonzero because A3,3(ϕ) > 0 for everyϕ ∈ Qb. In addition, the geometric phase is expressed as a line integral along thepath of joint angles (i.e., C) and consequently, it depends only on the path C [80]. Tosimplify the analysis, we look for ways in which the geometric phase generated by theproposed reconfiguration algorithm is equal to that of the nominal trajectory ϕ∗ (seeFig. 4.3). Toward that end, by assuming ϕ(t) := ϕ∗(τ(t)), where τ : [t1, t2] → [t∗1 , t∗2 ]is the virtual time fulfilling the following constraints

(i) τ(t1) = t∗1(ii) τ(t2) = t∗2

(iii) inft1≤t≤t2

τ(t) > 0,

(4.9)

C = C∗ and equation (4.8) can be rewritten as follows:

θ(t2) = θ1 +∫ t∗2

t∗1

σcm

A3,3(ϕ∗(s))

ds

τ ◦ τ−1(s)−∫C∗

J(ϕ∗) dϕ∗,

and consequently,

θ(t2) − θ2 =∫ t∗2

t∗1

1

A3,3(ϕ∗(s))

( σcm

τ ◦ τ−1(s)− σ∗

cm

)ds.


By defining μ(s) := 1τ◦τ−1(s)

> 0 and w(s) := 1A3,3(ϕ∗(s)) > 0 for s ∈ [t∗1 , t∗2 ], and as-

suming σcm /= 0, the condition θ(t2) = θ2 can be expressed as the following equalityconstraint:

∫ t∗2

t∗1w(s) μ(s) ds = σ∗

cm

σcm

∫ t∗2

t∗1w(s) ds. (4.10)

Furthermore, from the definition of μ(s), τ(t) = 1μ(τ(t)) , t1 ≤ t ≤ t2, and hence,

∫ t∗2

t∗1μ(s) ds = t2 − t1. (4.11)

By using the virtual time approach, we can present an alternative problem equiv-alent to Problem 4.1 in which the reconfiguration can be solved on the basis ofreachability and optimal control formulations of a linear time-varying system withinput constraints.

Problem 4.2 Determination of μ(τ) > 0, t∗1 ≤ τ ≤ t∗2 such that the equality con-straints in equations (4.10) and (4.11) are met is equivalent to determining the open-loop control input μ : [t∗1 , t∗2 ] → R

>0 that transfers the state of the following lineartime-varying system in the virtual time domain:

� :x1 = w(τ) μ

x2 = μ(4.12)

from (x1(t∗1 ), x2(t∗1 ))′ = (0, 0)′ to (x1(t∗2 ), x2(t∗2 ))′ = (xf1 , x

f2 )′, where xi := d

dτxi for

i = 1, 2 and

xf1 := σ∗

cm

σcm

∫ t∗2

t∗1w(s) ds

xf2 := t2 − t1.

(4.13)

4.3.1 Determination of the Reachable Set

The purpose of this section is to determine the reachable set from the origin (at t∗1 )at time t∗2 for the system �. Since ϕ(t) = ϕ∗(τ(t)), the following relations can beobtained for the first and second time derivatives of ϕ(t):

ϕ(t) = ∂ϕ∗

∂τ(τ(t)) τ(t)

ϕ(t) = ∂ϕ∗

∂τ(τ(t)) τ(t) + ∂2ϕ∗

∂τ2 (τ(t)) τ2(t),


and, hence, a discontinuity of μ (or equivalently, a discontinuity of τ) may result inan impulsive nature of ϕ(t). In view of the actuator limitations, this latter fact impliesthat ϕ(t) cannot be used as a reference trajectory for the joint angles. Thus, we presentthe following definition.

Definition 4.1 (Admissible Open-Loop Control Inputs for System �) The set ofadmissible open-loop control inputs for system � is denoted by Um,M and defined tobe the set of all continuously differentiable functions τ �→ μ(τ) ∈ [m, M] defined onthe interval [t∗1 , t∗2 ], where 0 < m < M.

We present a design method for obtaining an admissible open-loop control μ ∈C1([t∗1 , t∗2 ], [m, M]). For this purpose, we consider μ to be the output of a doubleintegrator and study the following augmented system:

�a :

x1 = w(τ) x3

x2 = x3

x3 = x4

x4 = v,

which can be viewed as a cascade connection of two components. The first componentis the system � in equation (4.12) with x3 as input and the second component is thedouble integrator with a piecewise continuous function v as input. The admissibilityof μ can be expressed as m ≤ x3 ≤ M that is a constraint on the state of the system�a.

Definition 4.2 (Admissible Open-Loop Control Inputs for System �a) The setof admissible open-loop control inputs for system �a is denoted byVL1,L2 and definedto be the set of all piecewise continuous functions τ �→ v(τ) ∈ [L1, L2] defined onthe interval [t∗1 , t∗2 ], where L1 < 0 < L2.

Definition 4.3 (Reachable Set from the Origin) For any 0 < m < M, L1 < 0 <

L2 and (x03, x

04)′ ∈ R2, defineAm,M,L1,L2 (x0

3, x04) as the set of all points (xf

1 , xf2 )′ ∈ R2

for which there exists an open-loop control v ∈ VL1,L2 such that the state of thesystem �a is transferred from the initial point (0, 0, x0

3, x04)′ at t∗1 to the final point

(xf1 , x

f2 , x

f3 , x

f4 )′ at t∗2 with the constraint m ≤ x3(τ) ≤ M, t∗1 ≤ τ ≤ t∗2 , where x

f3 and

xf4 are free.

It is clear that for every x03 /∈ [m, M], Am,M,L1,L2 (x0

3, x04) = φ. To determine

Am,M,L1,L2 , we study two optimal control problems. From these problems, the opti-mal admissible open-loop control inputs, vmax(τ), vmin(τ) ∈ VL1,L2 , t

∗1 ≤ τ ≤ t∗2 , are

determined such that the state of the augmented system �a is to be transferred from theinitial point x0 := (0, 0, x0

3, x04)′ at t∗1 to the final point (x1(t∗2 ), x2(t∗2 ), x3(t∗2 ), x4(t∗2 ))′


max2fxmin

2fx

2fx

2fx

2fx

1fx

2fx

A

B

Solution of the minimization problem

E

1fx

CD

O

1 20 0

, , , 3 4,m M L L x x

Solution of the maximization problem

Figure 4.4 The reachable set Am,M,L1,L2 (x03, x

04). The solutions of the minimization and max-

imization problems for a given xf

2 are denoted by C and D, respectively. (See the color versionof this figure in color plates section.)

at t∗2 with the property m ≤ x3(τ) ≤ M, t∗1 ≤ τ ≤ t∗2 , while the performance measure

Ia(v) := x1(t∗2 )

is maximized (see point D in Fig. 4.4) and minimized (see point C in Fig. 4.4). Notethat in these two optimal control problems, x2(t∗2 ) = x

f2 is specified, whereas x3(t∗2 )

and x4(t∗2 ) are free.The constraint m ≤ x3(τ) ≤ M can be rewritten as the following inequality con-

straints:

S1(x) := m − x3 ≤ 0

S2(x) := x3 − M ≤ 0.

Next, we take successive virtual time derivatives of S1(x) and S2(x) until obtainingan expression that is explicitly dependent on v [99, p. 118]. This process will resultin S1 = −v and S2 = v. Now, define the following Hamiltonian function:

H(x, p, λ, v, τ) :=p1w(τ) x3 + p2 x3 + p3 x4 + p4 v + λ1 S1 + λ2 S2

=p1w(τ) x3 + p2 x3 + p3 x4 + (p4 − λ1 + λ2) v,(4.14)

where x := (x1, x2, x3, x4)′, p := (p1, p2, p3, p4)′, and λ := (λ1, λ2)′ are the state,costate, and multiplier vectors, respectively. Furthermore, in equation (4.14),

Si = 0, on the constraint boundary (i.e., Si = 0)

λi = 0, off the constraint boundary (i.e., Si < 0),


for i = 1, 2, which can also be expressed as

v = 0, on the constraint boundary (i.e., Si = 0)

λi = 0, off the constraint boundary (i.e., Si < 0).(4.15)

Necessary conditions for multipliers λi(τ), i = 1, 2 to minimize the performance mea-sures2 are

λi(τ) ≥ 0, on the constraint boundary (i.e., Si = 0). (4.16)

The costates satisfy the following differential equations:

p1 = 0

p2 = 0

p3 = −p1w(τ) − p2

p4 = −p3,

in which pi := ddτ

pi, i = 1, · · · , 4. From here on, the superscripts “max” and “min”will denote the solutions of the maximization and minimization problems, respec-tively. We first study the maximization problem. Since the final values xmax

1 (t∗2 ),xmax

3 (t∗2 ), and xmax4 (t∗2 ) are free, from Table 5.1 of [100, p. 200], pmax

1 (t∗2 ) = −1 andpmax

3 (t∗2 ) = pmax4 (t∗2 ) = 0. These boundary conditions in combination with the costate

equations yield

pmax3 (τ; pmax

2 ) = −∫ t∗2

τ

w(s) ds − pmax2 (τ − t∗2 )

pmax4 (τ; pmax

2 ) = −∫ t∗2

τ

∫ t∗2

s

w(η) dη ds + pmax2

2(τ − t∗2 )2.

From Pontryagin’s Minimum Principle [101], vmax(τ) is given by

vmax(τ) =

⎧⎪⎨⎪⎩

L1 pmax4 − λmax

1 + λmax2 > 0

L2 pmax4 − λmax

1 + λmax2 < 0

undetermined pmax4 − λmax

1 + λmax2 = 0.

(4.17)

Note that if pmax4 (τ; pmax

2 ) − λmax1 (τ) + λmax

2 (τ) passes through the zero, a switchingof the optimal control input vmax(τ) occurs. Assume that w(τ) satisfies the followinghypothesis:

2 Note that the maximization of the performance measure Ia(v) := x1(t∗2 ) can be expressed as the mini-mization of −Ia(v).


(H1) w(τ) := ddτ

w(τ) is not zero on the open set (t∗1 , t∗2 ).

It will be shown that by hypothesis H1 the optimal control inputs vmax(τ) and vmin(τ)can switch at most once, and the singular condition does not occur. For this purpose,we present the following result.

Lemma 4.1 (Behavior of the Solutions for the Optimization Problems) Letm < x0

3 < M and L1 < 0 < L2. Assume that hypothesis H1 holds and x04 is such

that the optimal trajectories xmax(τ) and xmin(τ) of the system �a exist. Then, thefollowing statements are true.

(a) The optimal trajectories do not enter onto the boundaries S1 = 0 and S2 = 0.

(b) The optimal control inputs vmax(τ) and vmin(τ) can switch at most once.

(c) The singular condition does not occur. In other words, the sets

T max0 := {

τ ∈ [t∗1 , t∗2]∣∣pmax

4 (τ) − λmax1 (τ) + λmax

2 (τ) = 0}

T min0 :=

{τ ∈ [t∗1 , t∗2

]∣∣pmin4 (τ) − λmin

1 (τ) + λmin2 (τ) = 0

}

are Lebesgue negligible.

Proof. To prove the statements (a), (b), and (c) of Lemma 4.1, the maximizationproblem will be studied. Similar reasonings can also be presented for the minimizationproblem.

If the optimal trajectory enters onto the constraint boundary S1 = 0, S2 will benegative and consequently from condition (4.15), vmax = 0 and λmax

2 = 0. Since L1 <

0 < L2, from the open-loop control input vmax(τ) in equation (4.17), vmax = 0 resultsin λmax

1 = pmax4 that, in combination with the necessary conditions given in equation

(4.16), yields pmax4 ≥ 0. Similarly, if the optimal trajectory enters onto the constraint

boundaryS2 = 0, thenvmax = 0 andλmax1 = 0. Moreover,λmax

2 = −pmax4 that implies

that pmax4 ≤ 0.

Next, we study the roots of the nonlinear equationpmax4 (τ; pmax

2 ) = 0 on the interval[t∗1 , t∗2 ]. This equation can also be expressed as follows:

W(τ) = pmax2

2

(τ − t∗2

)2, (4.18)

where

W(τ) :=∫ t∗2

τ

∫ t∗2

s

w(η) dη ds.


For any pmax2 ∈ R, t∗2 is the solution of equation (4.18). We claim that the equation

pmax4 (τ; pmax

2 ) = 0 has at most one root in the interval [t∗1 , t∗2 ). To show this, let τ ∈[t∗1 , t∗2 ) exist such that pmax

4 (τ; pmax2 ) = 0. Then, pmax

2 is unique and can be given by

pmax2 = 2(

τ − t∗2)2W(τ). (4.19)

We remark that pmax2 is positive. If there exists τ ∈ [t∗1 , t∗2 ) such that τ /= τ and

pmax4 (τ; pmax

2 ) = 0, then equation (4.19) implies that

W(τ)(τ − t∗2

)2 = W(τ)(τ − t∗2

)2 .

Hence, it is sufficient to show that the function κ : [t∗1 , t∗2 ) → R by

κ(τ) := W(τ)(τ − t∗2

)2 .

is strictly monotonic. The first derivative of κ(τ) can be obtained as follows:

κ(τ) = W(τ)(τ − t∗2

)− 2W(τ)(τ − t∗2

)3 =:F (τ)(

τ − t∗2)3 . (4.20)

Assume that there exists η1 ∈ (t∗1 , t∗2 ) such that F (η1) = 0. Since F (η1) = F (t∗2 ) = 0,the Rolle’s Theorem implies that there is η2 ∈ (η1, t

∗2 ) such that

F (η2) = W(η2)(η2 − t∗2

)− W(η2) = 0.

Furthermore, F (t∗2 ) is also zero. Hence, from the Rolle’s Theorem, there exists η3 ∈(η2, t

∗2 ) such that

F (η3) = w(η3)(η3 − t∗2

) = 0,

which follows that w(η3) = 0, and this contradicts hypothesis H1. Therefore, κ(τ) isstrictly monotonic and, as a consequence, the equation pmax

4 (τ; pmax2 ) = 0 has at most

one root in the interval [t∗1 , t∗2 ).


Let τ ∈ (t∗1 , t∗2 ) be the root of the equationpmax4 (τ; pmax

2 ) = 0. Substituting equation(4.19) into pmax

4 (τ; pmax2 ) yields

pmax4

(τ; pmax

2

) = −W(τ)(τ − t∗2

)+ 2W(τ)

τ − t∗2

= F (τ)

t∗2 − τ/= 0.

Therefore, the condition pmax4 (τ; pmax

2 ) < 0 that can be expressed as

2

t∗2 − τ

∫ t∗2

τ

∫ t∗2

s

w(η) dη ds >

∫ t∗2

τ

w(s) ds (4.21)

implies that pmax4 (τ; pmax

2 ) > 0 for any τ ∈ [t∗1 , τ) and pmax4 (τ; pmax

2 ) < 0 for any τ ∈(τ, t∗2 ]. If condition (4.21) is not satisfied, then pmax

4 (τ; pmax2 ) < 0 for any τ ∈ [t∗1 , τ)

and pmax4 (τ; pmax

2 ) > 0 for any τ ∈ (τ, t∗2 ].Since m < x0

3 < M and xmax(τ) is the optimal trajectory, S1(x0) and S2(x0) arenegative that result in λmax

1 (t∗1 ) = λmax2 (t∗1 ) = 0. Without loss of generality, assume

that pmax4 (t∗1 ; pmax

2 ) /= 0.3 Then there are four possible cases.

Case 1: Assume that there exists τ ∈ (t∗1 , t∗2 ) such that pmax4 (τ; pmax

2 ) = 0 andinequality (4.21) holds. In this case, on the interval [t∗1 , τ), vmax(τ) = L1 andconsequently,

xmax4 (τ) = x0

4 + L1(τ − t∗1

)xmax

3 (τ) = x03 + x0

4

(τ − t∗1

)+ L1

2

(τ − t∗1

)2

xmax2 (τ) = x0

3

(τ − t∗1

)+ x04

2

(τ − t∗1

)2 + L1

6

(τ − t∗1

)3.

Because pmax4 (τ; pmax

2 ) > 0 on the interval [t∗1 , τ), the trajectory xmax(τ), t∗1 ≤ τ < τ

cannot enter onto the boundary S2 = 0. From Ref. [99, p. 118], since the control ofS1 is obtained only by changing S1, no finite control can keep the optimal trajectoryof the system �a on the constraint boundary S1 = 0, unless the following tangencyconstraints4 hold:

N1(xmax(τ)) :=[S1(xmax(τ))

S1(xmax(τ))

]=[m − xmax

3 (τ)

−xmax4 (τ)

]=[

0

0

]. (4.22)

3 If pmax4 (t∗1 ; pmax

2 ) = 0, then τ = t∗1 . Since the nonlinear equation pmax4 (τ; pmax

2 ) = 0 has at most one rootin the interval [t∗1 , t∗2 ), on the open set (t∗1 , t∗2 ), pmax

4 (τ; pmax2 ) /= 0. In addition, vmax(t∗1 ) is finite and as a

consequence, without loss of generality, we can assume that pmax4 (t∗1 ; pmax

2 ) /= 0.4 The terminology of a tangency constraint is taken from Ref. [99, p. 118].


Because L1 < 0, xmax3 (τ), t∗1 ≤ τ < τ is a quadratic and concave function with respect

to τ. Thus, the condition m < x03 < M implies that the tangency constraints (4.22)

cannot be satisfied on the interval [t∗1 , τ). Next, define

x4 := xmax4 (τ) = x0

4 + L1(τ − t∗1

)x3 := xmax

3 (τ) = x03 + x0

4

(τ − t∗1

)+ L1

2

(τ − t∗1

)2

x2 := xmax2 (τ) = x0

3

(τ − t∗1

)+ x04

2

(τ − t∗1

)2 + L1

6

(τ − t∗1

)3.

Also, on the interval (τ, t∗2 ], vmax(τ) = L2 and thus,

xmax4 (τ) = x4 + L2(τ − τ)

xmax3 (τ) = x3 + x4(τ − τ) + L2

2(τ − τ)2

xmax2 (τ) = x2 + x3(τ − τ) + x4

2(τ − τ)2 + L2

6(τ − τ)3.

Since pmax4 (τ; pmax

2 ) < 0 on (τ, t∗2 ], the optimal trajectory xmax(τ), τ < τ ≤ t∗2 cannotenter onto the boundary S1 = 0. Moreover, the tangency constraints to remain on theboundary S2 = 0 can be expressed as

N2(xmax(τ)) :=[S2(xmax(τ))

S2(xmax(τ))

]=[xmax

3 (τ) − M

xmax4 (τ)

]=[

0

0

]. (4.23)

The fact that xmax3 (τ), τ < τ ≤ t∗2 is a quadratic and convex function with respect to

τ in combination with the condition5 m < x3 < M implies that the tangency con-straints in equation (4.23) cannot be satisfied on the interval (τ, t∗2 ]. Consequently,S1(xmax(τ)), S2(xmax(τ)) < 0 for any τ ∈ [t∗1 , t∗2 ].

The final constraint xmax2 (t∗2 ) = x

f2 can also be expressed as the following third-

degree equation:

L1 − L2

6(τ − t∗2 )3 + x0

3 lmax + x04

2l2max + L1

6l3max = x

f2 , (4.24)

5 Since the optimal trajectory is feasible (i.e., m ≤ xmax3 (τ) ≤ M) and L1 < 0, x3 = m will result in

x4 < 0, which, in turn, implies the existence of τ ∈ (τ, t∗2 ] such that xmax3 (τ) < m for any τ ∈ (τ, τ).

This contradicts the feasibility of the optimal trajectory xmax(τ). In a similar manner, it can be shownthat x3 /= M.


where lmax := t∗2 − t∗1 . From equation (4.24), τ ∈ R is unique and can be calculatedas follows:

τ = t∗2 + 3

√6

L1 − L2

(xf2 − x0

3 lmax − x04

2l2max − L1

6l3max

).

If t∗1 < τ < t∗2 and inequality (4.21) is satisfied, then τ is a feasible solution and as aconsequence, the validity of Case 1 is confirmed.

Case 2: There exists τ ∈ (t∗1 , t∗2 ) such that pmax4 (τ; pmax

2 ) = 0 and

2

t∗2 − τ

∫ t∗2

τ

∫ t∗2

s

w(η) dη ds <

∫ t∗2

τ

w(s) ds. (4.25)

An analysis similar to that presented for Case 1 can be performed. However, thethird-degree equation in (4.24) is given by

L2 − L1

6

(τ − t∗2

)3 + x03 lmax + x0

4

2l2max + L2

6l3max = x

f2 . (4.26)

Equation (4.26) has the following real and unique root:

τ = t∗2 + 3

√6

L2 − L1

(xf2 − x0

3 lmax − x04

2l2max − L2

6l3max

)

that is feasible if t∗1 < τ < t∗2 and inequality (4.25) is satisfied.

Case 3: The function pmax4 (τ; pmax

2 ) is positive on the interval [t∗1 , t∗2 ]. Thisimplies that pmax

2 is not unique. In addition, vmax(τ) ≡ L1 and similar to theanalysis performed for the interval [t∗1 , τ) in Case 1, it can be shown that

S1(xmax(τ)), S2(xmax(τ)) < 0. Also, the final condition xmax2 (t∗2 ) = x

f2 can be satisfied

only for the following specific value of xf2 :

xf2 = x0

3 lmax + x04

2l2max + L1

6l3max.

Case 4: If the function pmax4 (τ; pmax

2 ) is negative on the interval [t∗1 , t∗2 ],pmax

2 is not unique and, moreover, pmax4 (τ; pmax

2 ) < 0 and L2 > 0 imply that

S1(xmax(τ)), S2(xmax(τ)) < 0, t∗1 ≤ τ ≤ t∗2 . The final condition xmax2 (t∗2 ) = x

f2 can be


satisfied only for the following specific value of xf2 :

xf2 = x0

3 lmax + x04

2l2max + L2

6l3max.

The proof of Lemma 4.1 follows from the results obtained in Cases 1–4. �

Remark 4.2 (Solutions of the Minimization Problem) In the minimization prob-lem,

pmin3 (τ; pmin

2 ) =∫ t∗2

τ

w(s) ds − pmin2 (τ − t∗2 )

pmin4 (τ; pmin

2 ) =∫ t∗2

τ

∫ t∗2

s

w(η) dη ds + pmin2

2(τ − t∗2 )2.

Moreover, the nonlinear equation pmin4 (τ; pmin

2 ) = 0 has at most one root in the in-terval [t∗1 , t∗2 ). Let τ ∈ [t∗1 , t∗2 ) be such that pmin

4 (τ, pmin2 ) = 0. The validity of Cases 1

and 2 in the minimization problem are confirmed by

2

t∗2 − τ

∫ t∗2

τ

∫ t∗2

s

w(η) dη ds <

∫ t∗2

τ

w(s) ds

and

2

t∗2 − τ

∫ t∗2

τ

∫ t∗2

s

w(η) dη ds >

∫ t∗2

τ

w(s) ds,

respectively. Cases 3 and 4 of the minimization problem are similar to the onespresented in the maximization problem.

Remark 4.3 (Infeasible Cases of the Optimization Problems) As discussed pre-viously, by hypothesis H1, the function F (τ) defined in equation (4.20) is nonzero onthe interval (t∗1 , t∗2 ). Thus, without loss of generality, we will assume that F (τ) < 0 onthe interval (t∗1 , t∗2 ). This assumption imposes that Case 2 of the maximization problemand Case 1 of the minimization problem are not feasible.

Now let m < x03 < M and x0

4 ∈ R. Moreover, define �maxm,M,L1,L2

(x03, x

04) and

�minm,M,L1,L2

(x03, x

04) to be the sets of all x

f2 ∈ R for which the optimal solutions

of the maximization and minimization problems starting from the initial point(0, 0, x0

3, x04)′ exist. Denote the solutions of the maximization and minimization prob-

lems corresponding to xf2 by xmax(τ; xf

2 ) and xmin(τ; xf2 ), respectively. The functions

πmax : �maxm,M,L1,L2

(x03, x

04) → R and πmin : �min

m,M,L1,L2(x0

3, x04) → R are introduced


by

πmax(xf2 ) :=

∫ t∗2

t∗1w(s) xmax

3

(s; xf

2

)ds = xmax

1

(t∗2 ; xf

2

)

πmin(xf2 ) :=

∫ t∗2

t∗1w(s) xmin

3

(s; xf

2

)ds = xmin

1

(t∗2 , x

f2

) (4.27)

(see Fig. 4.4). We claim that if the sets �maxm,M,L1,L2

(x03, x

04) and �min

m,M,L1,L2(x0

3, x04) are

nonempty, they are connected sets. For this purpose, we present the following lemmafor which it is shown that �max

m,M,L1,L2(x0

3, x04) is a connected set. A similar result can

also be obtained for the set �minm,M,L1,L2

(x03, x

04).

Lemma 4.2 Let m < x03 < M and x0

4 ∈ R. Assume that α < β are two scalars suchthat α, β ∈ �max

m,M,L1,L2(x0

3, x04). Then, for any γ ∈ (α, β), γ ∈ �max

m,M,L1,L2(x0

3, x04).

The proof is given in Appendix B.1. Now we are in a position to present the mainresult of this section. This result is expressed as the following theorem that determinesthe C1 open-loop control input μ transferring the state of the system � from the originat t∗1 to the final point (xf

1 , xf2 )′ ∈ Am,M,L1,L2 (x0

3, x04) at t∗2 .

Theorem 4.1 (Reachable Set from the Origin) Let m < x03 < M and x0

4 ∈ R.Assume that L1 < 0 and L2 > 0 are such that

min

(x0

3, x03 + x0

4 lmax + L1

2l2max

)> m

max

(x0

3, x03 + x0

4 lmax + L2

2l2max

)< M.

(4.28)

Then, the set Am,M,L1,L2 (x03, x

04) is given by

Am,M,L1,L2

(x0

3, x04

)={(

xf1 , x

f2

)′ ∈ R2|xf2 ≤ x

f2 ≤ x

f2 , πmin

(xf2

)≤ x

f1 ≤ πmax(xf

2 )}

,

where

xf2 := x0

3 lmax + x04

2l2max + L1

6l3max

xf2 := x0

3 lmax + x04

2l2max + L2

6l3max.


Proof. From Remark 4.3, define

vmax(τ; xf

2

):=

⎧⎨⎩

L1 t∗1 ≤ τ < τ(xf2

)L2 τ

(xf2

)< τ ≤ t∗2

(4.29)

and

vmin(τ; xf

2

):=

⎧⎨⎩

L2 t∗1 ≤ τ < τ(xf2

)L1 τ

(xf2

)< τ ≤ t∗2 ,

(4.30)

where

τ(xf2

):= t∗2 + 3

√6

L1 − L2

(xf2 − x

f2

)

τ(xf2

):= t∗2 + 3

√6

L2 − L1

(xf2 − x

f2

).

Since from part (c) of Lemma 4.1, the sets Tmax0 and Tmin

0 are Lebesgue negligi-ble, we shall leave the functions vmax and vmin undefined on them. For any τ ∈[t∗1 , t∗2 ], vmax(τ; xf

2 ) = vmin(τ; xf2 ) ≡ L1 (see point A in Fig. 4.4) and vmax(τ; xf

2 ) =vmin(τ; xf

2 ) ≡ L2 (see point B in Fig. 4.4), which, in turn, imply that

xmax3

(τ; xf

2

)= xmin

3

(τ; xf

2

)= x0

3 + x04

(τ − t∗1

)+ L1

2

(τ − t∗1

)2

xmax3

(τ; xf

2

)= xmin

3

(τ; xf

2

)= x0

3 + x04

(τ − t∗1

)+ L2

2

(τ − t∗1

)2,

and, consequently, xmax3 (τ; xf

2 ) ≤ xmax3 (τ; xf

2 ). This fact in combination withinequality (4.28), while considering L1 < 0 < L2, implies that for every τ ∈ [t∗1 , t∗2 ],

m < xmax3

(τ; xf

2

)≤ xmax

3

(τ; xf

2

)< M.

Therefore, xf2 , x

f2 ∈ �max

m,M,L1,L2(x0

3, x04), �min

m,M,L1,L2(x0

3, x04). Moreover, from

Lemma 4.2,

�maxm,M,L1,L2

(x0

3, x04

)= �min

m,M,L1,L2

(x0

3, x04

)=[xf2 , x

f2

].

Also, πmax(xf2 ) = πmin(xf

2 ) and πmax(xf2 ) = πmin(xf

2 ).


Next we claim that the functions πmax an πmin are strictly increasing functionswith respect to x

f2 . To show this, from the definition of πmax(xf

2 ) in equation (4.27),

πmax(xf2 ) can be rewritten as follows:

πmax(xf2

)=∫ τ

(xf

2

)t∗1

w(s) xmax3

(s; xf

2

)ds +

∫ t∗2

τ

(xf

2

)w(s) xmax3

(s; xf

2

)ds. (4.31)

Differentiating equation (4.31) with respect to xf2 results in

∂πmax

∂xf2

(xf2

)=∫ t∗2

τ

(xf

2

)w(s)∂xmax

3

∂τ

(s; xf

2

)ds

∂τ

∂xf2

(xf2

)

= 2(τ(xf2

)− t∗2

)2

∫ t∗2

τ

(xf

2

)w(s)(s − τ

(xf2

))ds

= 2(τ(xf2

)− t∗2

)2

∫ t∗2

τ

(xf

2

)∫ t∗2

s

w(η) dη ds

= 2(τ(xf2

)− t∗2

)2W(τ(xf2

))> 0.

In the above derivation, we have made use of the integration by parts for the third

equality. In a similar way, ∂πmin

∂xf

2

(xf2 ) = 2

(τ(xf

2 )−t∗2 )2W(τ(xf

2 )) > 0. Also,6

∂2πmax

∂xf 2

2

(xf2

)= ∂

∂τ

2(τ − t∗2

)2W(τ)∂τ

∂xf2

(xf2

)

=4F(τ(xf2

))(L1 − L2)

(τ(xf2

)− t∗2

)5 < 0

and

∂2πmin

∂xf 2

2

(xf2

)=

4F(τ(xf2

))(L2 − L1)

(τ(xf2

)− t∗2

)5 > 0.

6 From Remark 4.3, it is assumed that F (τ) < 0 for any τ ∈ [t∗1 , t∗2 ).


Next we show that πmin(xf2 ) < πmax(xf

2 ) for any xf2 ∈ (xf

2 , xf2 ). Introduce the error

function

e(xf2

):= πmax

(xf2

)− πmin

(xf2

)

and assume that there exists xf2 ∈ (xf

2 , xf2 ) such that e(xf

2 ) = 0. Since e(xf2 ) =

e(xf2 ) = e(xf

2 ) = 0, by the Rolle’s Theorem there exist ξ1 ∈ (xf2 , x

f2 ) and ξ2 ∈

(xf2 , x

f2 ) such that d

dxf

2

e(ξ1) = d

dxf

2

e(ξ2) = 0. However, d2

dxf2

2

e(xf2 ) = ∂2πmax

∂xf2

2

(xf2 ) −

∂2πmin

∂xf2

2

(xf2 ) < 0, and hence d

dxf

2

e(xf2 ) is strictly monotonic that contradicts the assumed

existence of xf2 . This result in combination with

∂2πmax

∂xf 2

2

(xf2

)< 0 ,

∂2πmin

∂xf 2

2

(xf2

)> 0

implies that πmin(xf2 ) < πmax(xf

2 ) for any xf2 ∈ (xf

2 , xf2 ).

We show that for any (xf1 , x

f2 )′ ∈ Am,M,L1,L2 (x0

3, x04), there exists an admissible

open-loop control input v ∈ VL1,L2 that transfers the state of the system �a from the

initial point (0, 0, x03, x

04)′ (at t∗1 ) to the point (xf

1 , xf2 , x

f3 , x

f4 )′ (at t∗2 ). For this purpose,

choose ϑ ∈ [0, 1] such that

xf1 = ϑ πmin

(xf2

)+ (1 − ϑ) πmax

(xf2

),

(see point E in Fig. 4.4) and define the following open-loop control input:

v(τ; xf

1 , xf2

):= ϑ vmin

(τ; xf

2

)+ (1 − ϑ) vmax

(τ; xf

2

).

Since7 ϑ ∈ [0, 1], v(τ; xf1 , x

f2 ) ∈ [L1, L2]. Moreover, due to the fact that8

x3(τ) = ϑ xmin3

(τ; xf

2

)+ (1 − ϑ) xmax

3

(τ; xf

2

)∈ (m, M),

the open-loop control v(τ; xf1 , x

f2 ) transfers the state of the system �a from the initial

point (0, 0, x03, x

04)′ (at t∗1 ) to the point (xf

1 , xf2 , x

f3 , x

f4 )′ (at t∗2 ) such that m < x3(τ) <

M, t∗1 ≤ τ ≤ t∗2 .Finally, it can be shown that the Hamilton–Jacobi–Bellman Equation is satisfied

along optimal trajectories of the system �a. The proof of this statement will not bepresented here (a detailed proof of a similar rechability problem is to be presented in

7 We remark that from the assumptions of Theorem 4.1, the set Am,M,L1,L2 (x03, x

04) is convex.

8 Note that the system �a is linear. In addition, from part (a) of Lemma 4.1, xmin3 (τ; xf

2 ), xmax3 (τ; xf

2 ) ∈(m, M) for every τ ∈ [t∗1 , t∗2 ].


Chapter 5, Lemma 5.2). Thus, Theorem 5.12 of Ref. [101, p. 357] implies that thesufficient conditions for optimality are satisfied, and hence, the developed open-loopcontrol laws (i.e., vmax and vmin) are indeed optimal. �

Using the constructive proof of Theorem 4.1, we develop an online algorithm forreconfiguration. For this purpose, suppose that the assumptions of Theorem 4.1 aresatisfied.

Algorithm 4.1 Online reconfiguration algorithm

Step 1: Define xf1 := σ∗

cmσcm

∫ t∗2t∗1

w(s)ds and xf2 := t2 − t1, and suppose that

(xf1 , x

f2 )′ ∈ Am,M,L1,L2 (x0

3, x04). By assuming that the values of the functions

πmax(xf2 ) and πmin(xf

2 ) on the interval [xf2 , x

f2 ] are precomputed and stored in

a lookup table, choose ϑ such that xf1 = ϑπmin(xf

2 ) + (1 − ϑ)πmax(xf2 ) and for

any τ ∈ [t∗1 , t∗2 ], let

v(τ; xf

1 , xf2

):= ϑ vmin

(τ; xf

2

)+ (1 − ϑ) vmax

(τ; xf

2

)x3

(τ; xf

1 , xf2

):= ϑ xmin

3

(τ; xf

2

)+ (1 − ϑ) xmax

3

(τ; xf

2

)x4

(τ; xf

1 , xf2

):= ϑ xmin

4

(τ; xf

2

)+ (1 − ϑ) xmax

4

(τ; xf

2

).

(4.32)

Note that the functions v(τ; xf1 , x

f2 ), x3(τ; xf

1 , xf2 ), and x4(τ; xf

1 , xf2 ) can be

computed in an online manner.

Step 2: Introduce the state vector

z(t) :=

⎡⎢⎣

z1(t)

z2(t)

z3(t)

⎤⎥⎦ :=

⎡⎢⎣

τ(t)

τ(t)

τ(t)

⎤⎥⎦

for t1 ≤ t ≤ t2 to augment the state of the mechanical system. The augmentedsystem over the time interval [t1, t2] is given by

xf = ff (xf ) + gf (xf )u

z1 = z2

z2 = z3

z3 =−v(z1; xf

1 , xf2

)x3

(z1; xf

1 , xf2

)+ 3x2

4

(z1; xf

1 , xf2

)x5

3

(z1; xf

1 , xf2

) .

(4.33)

The initial condition for z(t) is also defined as z(t1) = (t∗1 , 1x0

3, − x0

4

x033

)′.


Step 3: Define

ϕ(t) := ϕ∗(z1(t)), t1 ≤ t ≤ t2

as the reference trajectory for the joint angles.

Remark 4.4 From the definition of μ, τ(t) = 1μ(τ(t)) , and thus,

d

dtτ(t) = −

∂μ∂τ

(τ(t))

μ2(τ(t))τ(t) = −

∂μ∂τ

(τ(t))

μ3(τ(t))

d

dtτ(t) =

− ∂2μ

∂τ2 (τ(t)) μ3(τ(t)) + 3μ2(τ(t))(

∂μ∂τ

(τ(t)))2

μ6(τ(t))τ(t)

=− ∂2μ

∂τ2 (τ(t)) μ(τ(t)) + 3(

∂μ∂τ

(τ(t)))2

μ5(τ(t)),

which follows the system introduced in equation (4.33).

Remark 4.5 (The First and Second Time Derivatives of ϕ) Since we will make useof ϕ(t) as the reference trajectory for the body angles, the states z2 and z3 are intro-duced so that the first and second time derivatives of ϕ can be calculated as follows:

ϕ(t) = ϕ∗(z1) z2

ϕ(t) = ϕ∗(z1) z22 + ϕ∗(z1) z3.

Remark 4.6 (Planar Multilink Systems Composed of N ≥ 3 Links) The pro-posed online reconfiguration algorithm can be used for motion planning problemof planar multilink systems composed of N ≥ 3 rigid links which conserve angularmomentum.

Figure 4.5 illustrates a block diagram for the online reconfiguration algorithmover the time interval [t1, t2] during flight phases of monopedal running. The follow-ing theorem is an extended version of the latter algorithm by which the problem ofmonopedal running can be treated in Section 4.4.

Theorem 4.2 (Online Reconfiguration Algorithm for Landing in a Fixed Con-figuration During Flight Phases of Monopedal Running) Let t∗f be a positivereal number. Assume that 0 < t∗1 < t∗2 < t∗f is a partition of the interval [0, t∗f ]. Let

ϕ∗ : [0, t∗f ] → Qb be a nominal C2 trajectory in the configuration space satisfyinghypothesis H1 such that

ϕ∗(t∗1) = ϕ∗(t∗1) = 0

ϕ∗(t∗2) = ϕ∗(t∗2) = 0.(4.34)


min3 1 1 2; ,f fx z x x min

4 1 1 2; ,f fx z x x min1 1 2; ,f fv z x x

max1 1 2; ,f fv z x xmax

4 1 1 2; ,f fx z x xmax3 1 1 2; ,f fx z x x

3 1 1 2; ,f fx z x x 4 1 1 2; ,f fx z x x 1 1 2; ,f fv z x x

1z 2z 3z

1 1

cm

1t2t

* .d t Joint

torques 1z

1fx

2fx

1 2, , ,m M L L

Modified reference trajectory

Lookup table

Figure 4.5 Block diagram of the online reconfiguration algorithm over the time interval[t1, t2] during flight phases of monopedal running. In this diagram, for a given t1 < t2 and

σcm, the quantities xf

1 := σ∗cm

σcm

∫ t∗2t∗1

w(s)ds and xf

2 := t2 − t1 are computed. Then, by assuming

that (i) (xf

1 , xf

2 )′ ∈ Am,M,L1,L2 (x03, x

04) and (ii) the values of the functions πmax(xf

2 ) and πmin(xf

2 )on the interval [xf

2 , xf

2 ] are precomputed and stored in a lookup table, we choose ϑ such thatx

f

1 = ϑπmin(xf

2 ) + (1 − ϑ)πmax(xf

2 ). Using equation (4.32), ϑ is used to construct the functionsv(z1; x

f

1 , xf

2 ), x3(z1; xf

1 , xf

2 ), and x4(z1; xf

1 , xf

2 ). Finally, on the basis of equation (4.33), theaugmented states z1, z2, and z3 are introduced to construct the desired trajectory ϕd(t) =ϕ∗(z1(t)), t1 ≤ t ≤ t2 and its derivatives up to the second order. (See the color version of thisfigure in color plates section.)

Furthermore, suppose that when the angular momentum of the mechanical systemabout its COM is identical to σ∗

cm, the nominal absolute orientation, θ∗(t), satisfiesthe boundary conditions θ∗(0) = θ0, θ∗(t∗1 ) = θ1, θ∗(t∗2 ) = θ2 and θ∗(t∗f ) = θf . For

a given tf > 0 and σcm with the property σcmσ∗cm > 0, let x

f1 := σ∗

cmσcm

∫ t∗2t∗1

w(s)ds and

xf2 := t2 − t1, where

t1 := σ∗cm

σcmt∗1

t2 := tf − σ∗cm

σcm

(t∗f − t∗2

).

Choose x03 ∈ (m, M), x0

4 ∈ R, L1 < 0, and L2 > 0 such that the assumptions of Theo-

rem 4.1 hold. If (xf1 , x

f2 )′ ∈ Am,M,L1,L2 (x0

3, x04), then the following statements hold:


(a) The trajectory ϕd : [0, tf ] → Qb by

ϕd(t) :=

⎧⎪⎪⎪⎨⎪⎪⎪⎩

ϕ∗(

σcmσ∗

cmt)

0 ≤ t ≤ t1

ϕ∗(z1(t)) t1 ≤ t ≤ t2

ϕ∗(

σcmσ∗

cm(t − tf ) + t∗f

)t2 ≤ t ≤ tf

is C2, where z1(t) was introduced in equation (4.33), and fulfills the followingboundary conditions

ϕd(0) = ϕ∗(0)

ϕd(tf ) = ϕ∗(t∗f )

ϕd(0) = ϕ∗(0)σcm

σ∗cm

ϕd(tf ) = ϕ∗(t∗f )σcm

σ∗cm

.

(4.35)

(b) The trajectory ϕd(t) satisfies the boundary conditions θ(t1) = θ1, θ(t2) = θ2and θ(tf ) = θf when the initial condition of the absolute orientation and theangular momentum about the COM are θ(0) = θ0 and σcm, respectively.

(c) If x03 = 1, x0

4 = 0, 0 < m < 1 < M, and L2 = −L1 with the following prop-erty

− 2

l2max(1 − m) < L1 < 0 < L2 <

2

l2max(M − 1),

then for σcm = σ∗cm and tf = t∗f , ϕd(t) = ϕ∗(t) for every t ∈ [0, tf ].

The proof is given in Appendix B.2.

4.4 CONTROL LAWS FOR STANCE AND FLIGHT PHASES

This section presents a design method for determining the control laws during thestance and flight phases to realize a desired periodic trajectory as an asymptoticallystable orbit. Let O := Os ∪ Of denote a desired period-one orbit of the open-loophybrid model of running in equation (4.5), in which Os and Of are the stance andflight phases of the periodic orbit, respectively. Reference [73] proposes a method

CONTROL LAWS FOR STANCE AND FLIGHT PHASES 121

based on a finite-dimensional optimization problem for generating the time trajectoryof O.9

4.4.1 Stance Phase Control Law

Following the ideas of Refs. [46, 52, 55], the stance phase controller is assumed to be acontinuous time-invariant feedback law based on zeroing a parameterized holonomicoutput function with the uniform vector relative degree 2. This control law creates aparameterized finite-time attractive two-dimensional zero dynamics manifold in thecorresponding state manifold (i.e., Xs). To make this notion precise, introduce thefollowing holonomic output function for the dynamical system of equation (4.4):

ys(xs; α) := hs(qs; α) := hOs (qs) +B(s(qs); α), (4.36)

where hOs : Qs → R2 is at least a C2 function vanishing on the orbit Os. For deter-

mining hOs , the sample-based virtual constraints method introduced in Ref. [94] canbe used. In addition, the functionB : [0, 1] × A → R

2 is an augmentation functionthat is expressed as a Bezier polynomial of degree N

B(s(qs); α) :=N∑

k=0

N!

k!(N − k)!αks

k(1 − s)N−k,

where α := [α0 α1 ... αN−1 αN ] ∈ A, and A ⊂ R2×(N+1) is an open set. Also, s(qs)is defined as the normalized value of the angle of the virtual leg, that is, s(qs) :=γs(qs)−γ+

s

γ−s −γ+

s, in which γ+

s and γ−s are the initial and final values of the angle of the

virtual leg on Os, respectively (see Fig. 4.1). It is assumed that the set

Zs,α := {xs ∈ TQs|hs(xs; α) = 02×1, Lfshs(xs; α) = 02×1}

is an embedded two-dimensional submanifold of TQs. Moreover, suppose that Sfs ∩

Zs,α is an embedded one-dimensional submanifold of TQs. By properties of Bezier

polynomials, see Remark 3.16, for αN−1 = αN = 02×1, the manifold Sfs ∩ Zs,α is

independent of α. Moreover, following the results of Ref. [18, p. 125], it can beexpressed as

Sfs ∩ Zs,α =

{(q′, q′)′|q = q−∗

s , q = q−∗s

σ−s

σ−∗s

, σ−s ∈ R

}, (4.37)

9 In this chapter, the design method introduced in Ref. [73] for generating O is modified such thathypothesis H1, condition (4.34), and HO3–HO4 of [18, p. 162] are satisfied. Specifically, for thispurpose, some constraints are added to the proposed nonlinear optimization problem in Ref. [73].


where q−∗s and q−∗

s are the final configuration and velocity of the robot on Os, re-spectively. σ−

s represents the final value of the angular momentum of the mechanicalsystem about the leg end. In addition, σ−∗

s denotes the value of σ−s on the orbit Os.10

In the coordinates (γs, σs) for Zs,α, the stance phase zero dynamics can be given by

γs = κ1(γs; α) σs

σs = κ2(γs; α),(4.38)

where σs is the angular momentum of the monoped robot about the leg end that canbe obtained as σs = D3(qs) qs [18, Proposition B.11, p. 430]. Furthermore, the stancephase feedback law is chosen as the parameterized version of the finite-time controllerproposed in Refs. [18, p. 134, 46].

Remark 4.7 In contrast to the approach of Ref. [55], the stance phase controller ofour strategy is parameterized. The main reasons for this difference can be expressedas follows.

(1) In Ref. [55], a parameterized flight phase controller is used to achieve hy-brid invariance and configuration determinism at landing. Specifically, due tohybrid invariance, an additional constraint is imposed on the vector of gen-eralized velocities at the end of flight phases. Thus, by the Implicit FunctionTheorem and a numerical constrained optimization problem, the parametersof the flight phase controller are updated (at the beginning of the flight phase)to satisfy hybrid invariance and configuration determinism at landing.

(2) In our approach, this latter constraint on the final velocity is relaxed andan analytical reconfiguration algorithm is proposed. Instead, for creation ofhybrid invariance, the parameter α1 of the stance phase controller should beupdated at the beginning of the stance phase (see equation (4.41)).

4.4.2 Flight Phase Control Law

In this chapter, the flight phase control law is designed as a continuous feedback lawto track the modified reference trajectories generated by Theorem 4.2 (i.e., ϕd(t)).Define σ∗

cm, ϕ∗(t), and θ∗(t) as the angular momentum of the mechanical systemabout its COM, the time evolution of the joint angles, and the time evolution of theabsolute orientation on the orbit Of , respectively. Now assume that x−

s ∈ Sfs ∩ Zs,α

is the state of the closed-loop hybrid system immediately before the takeoff. From

equation (4.37), x−s = (q

′−∗s , q

′−s )′, where q−

s = q−∗s

σ−s

σ−∗s

. This latter result in com-

bination with the fact that the position and velocity remain continuous during thetakeoff implies that the joint angles and velocities at the beginning of the flight phase

10 Equations (6.72) and (6.73) and Proposition 6.1 of Ref. [18, p. 158] in combination with hypothesesHO3 and HO4 of Ref. [18, p. 162] imply that σ−∗

s /= 0.

CONTROL LAWS FOR STANCE AND FLIGHT PHASES 123

can be given by ϕ(0) = ϕ∗(0) and ϕ(0) = ϕ∗(0) σ−s

σ−∗s

. We remark that

ϕ∗(0) = [I2×2 02×1] q−∗s

ϕ∗(0) = [I2×2 02×1] q−∗s .

Moreover, following the notation of Ref. [55], on Sfs ∩ Zs,α, the position and velocity

of the COM are given by p−cm,s = f1(q−∗

s )

p−cm,s =

[λx(q−∗

s )

λy(q−∗s )

]σ−

s ,

where [λx(q−∗s ) λy(q−∗

s )]′ := ∂f1∂q

(q−∗s ) q−∗

s

σ−∗s

. Continuity of the position and velocity

during the takeoff and conservation of angular momentum about the COM during theflight phase in combination with equation (C.57) of Ref. [18, p. 454] imply that theangular momentum about the COM in the flight phase can be expressed asσcm = χσ−

s ,

where χ := 1 + mtoty−cm,sλx(q−∗

s ) − mtotx−cm,sλy(q−∗

s ). If χ /= 0,11 σcmσ∗

cm= σ−

s

σ−∗s

, and,

consequently, from part (a) of Theorem 4.2, ϕ(0) = ϕd(0) and ϕ(0) = ϕd(0).Next, let �(ϕ, q) := C(ϕ, q)q. Then, the static feedback law

u = Aϕϕ(ϕ) u + �ϕ(ϕ, q), (4.39)

where Aϕϕ(ϕ) := Aϕϕ − Aϕ3A−133 A3ϕ and �ϕ(ϕ, q) := �ϕ − Aϕ3A

−133 �3, yields the

following partially feedback linearized result that is known as the Spong normal form[95]:

ϕ = u

θ = σcm

A33(ϕ)− J(ϕ) ϕ

xcm = 0

ycm = −g0.

Furthermore, since ϕ(0) = ϕd(0) and ϕ(0) = ϕd(0), the feedback law

u := ϕd(t) − K1(ϕ − ϕd(t)) − K0(ϕ − ϕd(t)),

11 It is assumed that on the desired periodic trajectory O, χ /= 0. The condition χ /= 0 can be imposedthrough an inequality constraint in the nonlinear optimization problem of Ref. [73].


where K1, K0 ∈ R2×2 are diagonal and positive definite matrices, imposes thatϕ(t) = ϕd(t), 0 ≤ t ≤ tf , which, in turn, from part (b) of Theorem 4.2, implies thatθ(tf ) = θ∗(t∗f ) = θf . Hence, at the end of the flight phase, the values of the jointangles and absolute orientation are specified and equal to the desired values, that is,[ϕ

′(tf ) θ(tf )]′ = q+∗

s , where q+∗s is the initial configuration on Os (configuration de-

terminism at landing). Since the impact map preserves positions during the transitionfrom flight to stance, q+∗

s will be the initial configuration of the mechanical system atthe beginning of the stance phase. Thus, to achieve hybrid invariance, it is necessaryto choose α0 = 02×1 as in this case,

hs(q+∗s ; α) = hOs (q

+∗s ) +B(0; α) = 02×1.

As in Ref. [55], the configuration determinism at landing implies that the heightof the COM at the beginning of the stance phase , y+

cm,s, is predetermined. Therefore,the flight time tf satisfies the following quadratic equation:

y+cm,s = y−

cm,s + λy(q−∗s ) σ−

s tf − 1

2g0 t2f (4.40)

from which, tf can be computed as a function of σ−s .

Remark 4.8 (Configuration Determinism at Landing) Since (i) the modi-fied reference trajectories generated by Theorem 4.2 are C2 and (ii) for everyx−s ∈ Sf

s ∩ Zs,α, the projection of the trajectory onto TQb at the beginning of theflight phase is identical to that of the orbit Of (i.e., ϕ(0) = ϕd(0) and ϕ(0) = ϕd(0)),the feedback law of equation (4.39) together with parts (b) and (c) of Theorem 4.2result in configuration determinism at landing. Moreover, since the sets Zs,α are lo-cally continuously finite-time attractive (see Definition 2.8, Section 2.3), it can beconcluded that there exist open sets Vs,α containing Zs,α such that for every initialcondition x0

s ∈ Vs,α, the solution of the closed-loop hybrid model of running throughx0s at time t = 0 satisfies configuration determinism at landing.

4.4.3 Event-Based Update Law

The event-based update law updates the coefficients of the augmentation functionB(s; α) at each impact event (i.e., transition from flight to stance) to achieve hybridinvariance and asymptotic stabilization of the desired periodic orbit O as describedin Section 2.4. We remark that these coefficients are held constant during the stancephase. As mentioned previously, αN−1 = αN = 02×1 implies that Sf

s ∩ Zs,α is in-dependent of α. The parameter α0 was also chosen as zero in the previous section.Here, we obtain an update law for α1 in terms of σ−

s (i.e., the value of the angularmomentum about the leg end at the end of previous stance phase) to render the familyof manifolds Zs := {Zs,α|α ∈ A} invariant under the transition map � : Sf

s → Xs

defined by �(xs) := �sf ◦ Ff ◦ �

fs (xs), where Ff represents the flow map of the

HYBRID ZERO DYNAMICS AND STABILIZATION 125

flight phase. In particular, the parameter α1 is updated in a stride-to-stride mannersuch that �(Sf

s ∩ Zs,α) ⊂ Zs. The update laws of α2, · · · , αN−2 that stabilize thedesired periodic trajectory will be addressed in Section 4.5.

At the end of the flight phase, according to the definition of the flight time tfas a function of σ−

s in equation (4.40), the generalized velocity of the mechanicalsystem at the end of the flight phase, q−

f , can be obtained as a function of σ−s

12.Moreover, the impact map in equation (4.5) yields immediately the initial velocityin the stance phase, q+

s , in terms of σ−s , that is, q+

s (σ−s ). As discussed previously,

α0 = 02×1 implies that hs(q+s ; α) = 02×1. Let us define x+

s (σ−s ) := (q

′+∗s , q

′+s (σ−

s ))′.To create hybrid invariance, at the beginning of the stance phase, the event-based lawshould update α such that x+

s (σ−s ) ∈ Zs,α. To achieve this goal, α is updated so that

Lfshs(x+s (σ−

s ); α) = ∂hs

∂qs

(q+∗s ; α) q+

s (σ−s ) = 02×1.

In particular,

Lfshs(x+s (σ−

s ); α) = ∂hOs

∂qs

(q+∗s ) q+

s (σ−s ) + N(α1 − α0)

γ−s − γ+

s

∂γs

∂qs

(q+∗s ) q+

s (σ−s )

= 02×1,

and as a consequence, α1 should be updated by the following law13:

α1(σ−s ) = −γ−

s − γ+s

N

∂hOs

∂qs

(q+∗s ) q+

s (σ−s )

(∂γs

∂qs

(q+∗s ) q+

s (σ−s )

)−1

. (4.41)

4.5 HYBRID ZERO DYNAMICS AND STABILIZATION

To obtain HZD for the closed-loop hybrid model of monopedal running, let N ≥ 3be an integer number and α ∈ A. Assume that α0 = αN−1 = αN = 02×1 and α1 isupdated as in equation (4.41) at the beginning of the stance phase. Then, the angularmomentum of the mechanical system about the leg end at the beginning of the stancephase is given by σ+

s = ω(σ−s ), where

ω(σ−s ) := D3(q+∗

s ) q+s (σ−

s ).

12 Note that since the flight phase controller is based on the reconfiguration algorithm of Theorem 4.2,q−

f(σ−

s ) is different from that presented in equation (9.51) of Ref. [18, p. 271].13 Hypothesis HO3 of Ref. [18, p. 162] implies that on the stance phase of the periodic orbit, γs /= 0. Hence,

there exists an open neighborhood N(σ−∗s ) such that for every σ−

s ∈ N(σ−∗s ), ∂γs

∂qs(q+∗

s )q+s (σ−

s ) /= 0.


Thus, hybrid invariance reduces the analysis of the full-order model to the analysisof the following reduced-order system:

�zero :

⎧⎪⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎪⎩

[γs

σs

]=[κ1(γs; α) σs

κ2(γs; α)

]γs /= γ−

s

[γ+s

σ+s

]=[

γ+s

ω(σ−s )

]γs = γ−

s ,

(4.42)

which is referred to as HZD.By extending the results of Ref. [52] to HZD of equation (4.42) and also assuming

that σs is negative during the stance phase, we define the restricted Poincare returnmap as ρα : Sf

s ∩ Zs,α → Sfs ∩ Zs,α by

ρα(σ−s ) := −

(ω2(σ−

s ) + 2∫ γ−

s

γ+s

κ2(γs; α, α1(σ−s ))

κ1(γs; α, α1(σ−s ))

dγs

) 12

,

where α := [α′2 α′

3 · · · α′N−2]′ ∈ Rp×1 and p := 2(N − 3).

Remark 4.9 Since in this chapter, (i) the constraint of Ref. [55] on the final velocitiesof the robot at the end of flight phases is relaxed, and (ii) instead, hybrid invariance iscreated by the parameter update law given in equation (4.41), the resultant restrictedPoincare return map is different from that of Ref. [55]. Specifically, the first derivativeof ρα with respect to σ−

s should be calculated numerically.

Let ρ(σ−s ; α) := ρα(σ−

s ). Then, the following discrete-time system

σ−s [k + 1] = ρ(σ−

s [k]; α[k]) (4.43)

with the one-dimensional state space Sfs ∩ Zs,α and input α can be considered to

analyze stabilization. From the constructive procedure of hOs (qs) and determinationof the parameters of the stance and flight phase controllers on the basis of the periodicorbit O, σ−∗

s is an equilibrium point of the discrete-time system in equation (4.43)when the input α is zero (i.e., α = α∗ = 0p×1).

Theorem 4.3 (Asymptotic Stability of the Periodic Orbit) Suppose that theassumptions of part (c) of Theorem 4.2 hold. Define a := ∂ρ

∂σ−s

(σ−∗s ; α∗) and b :=

∂ρ∂α

(σ−∗s ; α∗). If b /= 01×p, then there exists a gain matrix K ∈ Rp×1 such that using

the within-stride controllers and the following static update law

α(σ−s ) = −K(σ−

s − σ−∗s ), (4.44)

NUMERICAL RESULTS 127

O is an asymptotically stable periodic orbit for the closed-loop hybrid modelof running.

Proof. If b /= 01×p, the pair (a, b) is controllable, which, in turn, implies the exis-tence of K ∈ Rp×1 such that |acl| < 1, where acl := a − bK. Hence, σ−∗

s is a locallyexponentially stable equilibrium point for the closed-loop discrete-time system14

σ−s [k + 1] = ρcl(σ−

s [k]), where ρcl(σ−s ) := ρ(σ−

s ; α(σ−s )). Next, denote the right-

hand side of the closed-loop augmented system of equation (4.33) by fa(xf , z), whichis discontinuous at the following hypersurfaces:

Zmax1 := {

xa := (x′f , z′)′ ∈ Xf × R3

∣∣z1 = τ}

Zmin1 := {

xa := (x′f , z′)′ ∈ Xf × R3

∣∣z1 = τ}.

Since the vector field fa is transversal to Zmax1 and Zmin

1 at every point in

Xa := {xa := (

x′f , z′)′ ∈ Xf × R3

∣∣z2 /= 0},

it follows from [102, Lemma 2, p. 107] that there exists an open set Xa ⊂ Xa suchthat the closed-loop ordinary differential equation of equation (4.33) for every initialcondition in Xa has a unique solution in forward time. Moreover, from [102, Corollary,p. 93], this solution depends continuously on the parameters of fa(xf , z), that is, σcmand tf . From σcm = χσ−

s , equation (4.40) and |acl| < 1, this latter fact in combinationwith Theorem 2.5 of Chapter 2 and part (c) of Theorem 4.2 guarantees that theproposed control scheme realizes O as an asymptotically stable periodic orbit for theclosed-loop hybrid system. �

4.6 NUMERICAL RESULTS

This section presents a numerical example for the proposed online reconfigurationalgorithm and control scheme. It is assumed that all masses of the three-link monopedrobot are lumped. The torques u1 and u2 are applied between the femur and tibia,and the torso and femur, respectively. The physical parameters of the monoped robotare given in Table 4.1.15 Similar to the motion planning algorithm presented in Ref.[73], a modified algorithm can be developed for designing a feasible period-onetrajectoryO satisfying hypothesis H1, condition (4.34), and hypotheses HO3–HO4 of

14 Since acl = ∂ρcl∂σ−

s

(σ−s )|σ−

s =σ−∗s

, |acl| < 1 implies that σ−∗s is locally exponentially stable equilibrium point

for the closed-loop discrete-time system.15 The fourth row of Table 4.1 represents the distance between the COM of the links and joints. Note that

for the torso and tibia links, the position of the COM is measured with respect to the hip and knee joints,respectively. Furthermore, the position of the COM of the femur is measured with respect to the hipjoint.


TABLE 4.1 Physical Parameters of the Monoped Robot

Femur Tibia Torso

Length in m 0.5 0.5 0.5Mass in kg 2 2 10Mass center in m 0.1667 0.2500 0.3333

Ref. [18, p. 162] and also to minimize the cost function

J := 1

Ls

∫ T

0‖u(t)‖2

2 dt, (4.45)

in which T and Ls denote the period of O and the step length, respectively. Thecost function (4.45) roughly represents electrical motor energy in the body jointsper distance traveled. Using the fmincon function of the MATLAB’s OptimizationToolbox on the optimal trajectory, the joint paths during the stance and flight phasesare defined by polynomial evolutions (i.e., ϕ∗

s and ϕ∗f ) with respect to t, specifically,

ϕ∗s (t) :=

6∑i=0

aist

i, 0 ≤ t < t∗s

ϕ∗f (t) :=

10∑i=0

aif ti, 0 ≤ t < t∗f ,

where t∗s and t∗f are the stance and flight times, respectively. The coefficients

ais, i = 0, · · · , 6 and ai

f , i = 0, · · · , 10 are given in Tables 4.2 and 4.3, respectively.Furthermore, the initial position and velocity for the absolute orientation at the begin-ning of the stance phase are θ∗

s (0) = 0.8376(rad) and θ∗s (0) = 0.5555(rad/s), which,

in turn, in combination with ϕ∗s (t), ϕ∗

f (t) and the open-loop hybrid system in equation(4.5) completely determine the reference trajectory O. This trajectory has a period ofT = t∗f + t∗s = 0.2073 + 0.2356 = 0.4429(s), a step length of Ls = 0.4429(m), and

TABLE 4.2 Coefficients ais, i = 0, . . . , 6 for

the Joint Paths During the Stance Phase

a0′s 0.6315 1.7704

a1′s (101) 1.0000 −0.3770

a2′s (102) −2.6992 1.7708

a3′s (103) 2.7612 −2.5305

a4′s (104) −1.1920 1.9725

a5′s (104) 1.9346 −7.8882

a6′s (105) −0.0379 1.2286


TABLE 4.3 Coefficients aif , i = 0, . . . , 10 for

the Joint Paths During the Flight Phase

a0′f 0.7840 2.1446

a1′f 0.5988 9.9929

a2′f (103) 1.3096 −0.5331

a3′f (104) −8.1640 1.7127

a4′f (106) 2.1984 −0.3909

a5′f (107) −3.1680 0.6625

a6′f (108) 2.6560 −0.7794

a7′f (109) −1.3335 0.5823

a8′f (109) 3.9338 −2.5843

a9′f (109) −6.2405 6.1852

a10′f (109) 4.0564 −6.1401

an average running speed of 1(m/s). On the trajectory, the robot will not slip for acoefficient of friction greater than 0.55. Table 4.4 presents the reference trajectorystatistics that will be used in the control law. Desired state trajectories corresponding totwo steps of the mechanical system are depicted in Fig. 4.6, where the discontinuities

0 0.2 0.4 0.6 0.8

0.8

1

φ 1 (ra

d)

0 0.2 0.4 0.6 0.8

1.8

2

2.2

φ 2 (ra

d)

0 0.2 0.4 0.6 0.80.8

0.9

1

1.1

θ (r

ad)

0 0.2 0.4 0.6 0.8−10

−5

0

5

10

dφ1/d

t (ra

d/s)

0 0.2 0.4 0.6 0.8−10

−5

0

5

10

Time (s)

dφ2/d

t (ra

d/s)

0 0.2 0.4 0.6 0.8

−5

0

5

Time (s)

dθ/d

t (ra

d/s)

Flight Stance

Figure 4.6 Plot of the state trajectories corresponding to two consecutive steps of the desiredperiodic orbit. The discontinuities in velocity are due to the impact. (See the color version ofthis figure in color plates section.)


TABLE 4.4 Periodic Trajectory Statistics

γ+s γ−

s σ−∗s σ∗

cm x−cm,s

1.2486 1.5342 −13.7227 0.9021 0.1215y−

cm,s y+cm,s λx(q−∗

s ) λy(q−∗s ) χ

1.0041 0.9663 −0.0832 −0.0608 −0.0657

0 0.2 0.4 0.6 0.8−100

−50

0

50

100

u 1 (N

m)

0 0.2 0.4 0.6 0.8−100

−50

0

50

100

150

200

u 2 (N

m)

0 0.2 0.4 0.6 0.8−200

−100

0

100

200

300

Time (s)

F1h (

N)

0 0.2 0.4 0.6 0.80

200

400

600

800

1000

Time (s)

F1v (

N)

Flight

Stance

Figure 4.7 Plot of commanded control inputs and ground reaction force during two conse-cutive steps of the desired periodic orbit. The discontinuities are due to the transitions betweenthe stance and flight phases. (See the color version of this figure in color plates section.)

in velocity are due to impact. The control signals and components of the ground reac-tion force at the leg end during two steps of the desired periodic orbit are also shownin Fig. 4.7. The discontinuities of the open-loop control signals are due to transitionsbetween the stance and flight phases.

The results of the stability analysis performed for the desired trajectory with afourth-degree Bezier polynomial as an augmentation function are given in Table 4.5.

TABLE 4.5 Stability Analysis of the Desired Periodic Trajectory

a b K′ acl

0.9576 [0.1194 0.1838] [1.1372 1.7556] 0.4981


−22 −20 −18 −16 −14 −12 −10 −8 −6−25

−20

−15

−10

−5

0

σs− (kgm2 /s)

(kgm

2 /s)

ρcl (σs−)

ρol (σs−)

σs−

σs−*

Figure 4.8 Plot of the open-loop and closed-loop restricted Poincare return maps ρol, ρcl. Theplot is truncated at −7.3227(kgm2/s) because this point is an upper bound for the domain ofdefinition of ρcl. For |σ−

s | sufficiently large, the ground reaction force at the leg end will not bein the static friction cone. The mapping ρcl has two fixed points. One fixed point (σ−

s = σ−∗s =

−13.7227(kgm2/s)) is asymptotically stable and corresponds to the desired periodic trajectory,while the other fixed point is unstable and occurs at approximately σ−

s = −7.4964(kgm2/s).(See the color version of this figure in color plates section.)

Since a = 0.9576 ∈ (−1, 1), this trajectory is asymptotically stable orbit for theclosed-loop system. However, to improve the convergence rate, we will make useof the static update law in equation (4.44). Moreover, the gain of this update law canbe calculated via DLQR.16 In this design method, the gain K is obtained such that bythe static update law in equation (4.44), the cost function

J := 1

2

∞∑k=0

{q(δσ−

s [k])2 + rδα′[k]δα[k]

}

subject to the linearization of the system in equation (4.43) about (σ−∗s , α∗) is

minimized, where q ≥ 0 and r > 0. Calculation for q = 10 and r = 1 by the

16 In Ref. [61], the DLQR design method has been used in control of walking of an underactuated 3Dbiped robot.


TABLE 4.6 Parameters of the Online Reconfiguration Algorithm

m M L1 L2 t∗1 t∗2 lmax

0.001 1000 −104 104 0.0345 0.1727 0.1382

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6

0.7

0.8

0.9

1

1.1

φ 1 (ra

d)

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.61.7

1.8

1.9

2

2.1

2.2

φ 2 (ra

d)

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.60.8

0.9

1

1.1

θ (r

ad)

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6

−20

−10

0

10 dφ

1/dt (

rad/

s)

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6

−20

−10

0

10

Time (s)

dφ2/d

t (ra

d/s)

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6−20

−15

−10

−5

0

5

Time (s)

dθ/d

t (ra

d/s)

Flight

Stance

Figure 4.9 Plot of the state trajectories corresponding to four consecutive steps of themonoped robot. The discontinuities in velocity are due to the impact. (See the color version ofthis figure in color plates section.)

dlqr function of MATLAB yields K = [1.1372 1.7556

]′ and, as a consequence,acl = a − bK = 0.4981. The open-loop and closed-loop restricted Poincare returnmaps (i.e., ρol and ρcl) are shown in Fig. 4.8.17 From part (c) of Theorem 4.2, wechoose x0

3 = 1, x04 = 0, m = 0.001, M = 1000, and L2 = −L1 = 104. Moreover,

lmax = t∗2 − t∗1 = 0.1382(s) (see Table 4.6).To illustrate the convergence to the desired periodic orbit, the simulation of the

closed-loop hybrid model of running is started at the end of the stance phase with aninitial velocity 4% higher than the value onO. State trajectories corresponding to foursteps of the mechanical system are depicted in Fig. 4.9. Discontinuities in velocity are

17 ρol(σ−s ) is identical to ρα(σ−

s ) when α = α∗ = 0p×1.


0.7 0.8 0.9 1 1.1−25

−20

−15

−10

−5

0

5

10

15

φ1 (rad)

dφ1/d

t (ra

d/s)

1.8 1.9 2 2.1 2.2−25

−20

−15

−10

−5

0

5

10

φ2 (rad)

dφ2/d

t (ra

d/s)

0.8 0.9 1 1.1−20

−15

−10

−5

0

5

10

θ (rad)

dθ/d

t (ra

d/s)

0.7 0.8 0.9 1 1.1

1.8

2

2.2

0.9

1

1.1

φ1 (rad)φ

2 (rad)

θ (r

ad)

Flight phase

Stance phase

Flight phase

Stance phase

Flight phase

Stance phase

Impact Impact

Impact Stance phase

Flight phase

Impact

Figure 4.10 Phase–plane plots and projection of the state trajectories during 10 consecutivesteps onto (ϕ1, ϕ2, θ). The convergence to the desired periodic trajectory can be seen. (See thecolor version of this figure in color plates section.)

due to the impact. Figure 4.10 represents the phase portraits and projection of the statetrajectories onto (ϕ1, ϕ2, θ). The effect of the impact with the ground is illustrated byjumps of the velocity in the phase portraits. Commanded control inputs during fourconsecutive steps of running are also shown in Fig. 4.11. The discontinuities in thecontrol inputs are due to the transitions between the stance and flight phases. Finally,Fig. 4.12 depicts the desired trajectories for the joint angles (i.e., ϕd(t)) generated bythe algorithm of Theorem 4.2 and absolute orientation during the flight phases of 10consecutive steps. As mentioned in Chapter 1, most of the past work in the literatureof legged locomotion is based on quasistatic stability criteria and flat-footed walkingand running motions such as ZMP criterion and its extensions [2–16]. The monopedalmodel studied in this chapter has a point foot, and hence, the ZMP criterion cannotbe applied. By using the approach of this chapter, the periodic orbit is asymptoticallystable in the sense of Lyapunov. In addition, the configuration of the mechanicalsystem during discrete transitions (i.e., impact and takeoff) are predetermined, andhence, the step length of the robot during consecutive steps of running is equal to thaton the periodic orbit O. In the literature of monopedal and bipedal gait control, the


0 0.5 1 1.5−100

−50

0

50

100

Time (s)

u 1 (N

m)

0 0.5 1 1.5−200

0

200

400

600

Time (s)

u 2 (N

m)

5 10 15 203000

3500

4000

4500

Step

(N2 m

s)

5 10 15 204.4

4.6

4.8

5

5.2

Step

(N)

J

1(i)

J1*

J1av

(i)

J2(i)

J2av

(i)

Stance

Flight

Figure 4.11 Plot of commanded control inputs during four consecutive steps of running (topgraphs). The discontinuities in the control inputs are due to the transitions between the stanceand flight phases. The bottom graphs present the plot of the cost function J1(i), J2(i), J1,av(i)and J2,av(i) for i = 1, 2, . . . , 20. The periodic orbit O is designed to minimize the cost function(4.45). On this trajectory, J = J∗

1 = 3.2836 × 103(N2ms). From the figure, the value of J1 aftera short transient period (four steps) is approximately equal to J∗

1 , which, in turn, illustrates theefficiency of the algorithm in the sense of electric motor energy per distance traveled. (See thecolor version of this figure in color plates section.)

two most popular cost functions are [18, 52, 73]

J1(i) := 1

Ls

∫step (i)

‖u(t)‖22 dt, i = 1, 2, . . .

J2(i) := 1

Ls

∫step (i)

〈q, B u〉 dt, i = 1, 2, . . . ,

(4.46)

in which Ls is the common step length, B is the input matrix, 〈x, y〉 := x′y, and step (i)represents the ith step, i = 1, 2, · · · . In equation (4.46), J1(i) and J2(i) denote theelectric motor energy and the integral of instantaneous mechanical power, per distancetraveled during the ith step, respectively. Figure 4.11 (bottom graphs) illustrates thevalue ofJ1(i),J2(i),J1,av(i), andJ2,av(i) during 20 consecutive steps of running, whereJ1,av and J2,av are the average values of J1 and J2, respectively, that is, J1,av(i) :=1i

∑ij=1 J1(j) and J2,av(i) := 1

i

∑ij=1 J2(j). As mentioned previously, the periodic

orbit O is designed to minimize the cost function (4.45). From Fig. 4.11, it can beconcluded that by applying the proposed feedback scheme, the value of J1 after ashort transient period (four steps) is approximately equal to that on O, which, in turn,illustrates the efficiency of the algorithm in the sense of control effort (electric motor


0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0.7

0.8

0.9

1

1.1φ 1d (

rad)

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

1.8

2

2.2

φ 2d (ra

d)

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0.8

0.9

1

1.1

Normalized flight time

θ (r

ad)

Figure 4.12 Plot of the desired trajectories for the joint angles (i.e., ϕd(t)) generated by thealgorithm of Theorem 4.2 (two top graphs) and absolute orientation (bottom graph) versusnormalized time during the flight phases of 10 consecutive steps. In three graphs, the nominaltrajectory is depicted by dashed curves. The circles represent the initial and final configurations.(See the color version of this figure in color plates section.)

energy) per distance traveled. In addition, by defining dimensionless cost functionJ3(i) := J2(i)

mtotg0= 1

mtotg0Ls

∫step (i)〈q, B u〉 dt, as the mechanical energy per unit weight

per unit distance traveled during the ith step, where mtot is the total mass of the robot,it can be observed that J3(i) ∈ [0.0322, 0.0367], i = 1, 2, . . ., and limi→∞ J3(i) =0.0366. Comparing these dimensionless results with those reported in Ref. [73], whichdescribe desired reference trajectories during running by another robot, demonstratesthat the resultant running locomotion on the desired trajectory O and the closed-looptrajectory is energy efficient.

CHAPTER 5

Online Generation of Joint MotionsDuring Flight Phases of Planar Running

5.1 INTRODUCTION

The motion of a planar bipedal robot during running can be described by a hybridsystem with two continuous phases, a stance phase (one leg on the ground) and aflight phase (no leg on the ground), and discrete transitions between the continuousphases for take-off and landing. An offline motion planning algorithm, based on afinite-dimensional nonlinear optimization problem with equality and inequality con-straints, has been proposed in Ref. [73] to generate time trajectories of a desiredperiodic orbit for the hybrid model of bipedal running. Following the results ofChapter 4, to asymptotically stabilize the desired periodic orbit for the hybrid model ofrunning using a one-dimensional restricted Poincare return map and HZD approach,the configuration of the mechanical system should be transferred from a predeterminedinitial pose (immediately after take-off) to a predetermined final pose (immediatelybefore landing) during the flight phases of running. As mentioned in Chapter 4, thisproblem is referred to as configuration determinism at transitions. The objective ofthis chapter is to present modified online motion planning algorithms for generationof continuous (C0) and continuously differentiable (C1) open-loop trajectories inthe body configuration space of the mechanical system such that the reconfigurationproblem is solved [82, 83]. The algorithms presented here are extensions of that pre-sented in Chapter 4. In particular, the generated trajectories in Chapter 4 were twicecontinuously differentiable (C2) while the reachable sets associated with the algo-rithms of the present chapter are larger than that of Chapter 4. We address the motionplanning problem for general planar open kinematic chains composed of N ≥ 3 rigidlinks interconnected with frictionless and rotational joints. The configuration of thismultilink system is specified by the absolute position and orientation of the mechani-cal system with respect to an inertial world frame, and by the joint angles determiningthe shape of the robot. It is assumed that the angular momentum of the mechanicalsystem about its COM is conserved.


137

138 ONLINE GENERATION OF JOINT MOTIONS DURING FLIGHT PHASES OF PLANAR RUNNING

The main contribution of this chapter is to present online motion planningalgorithms based on virtual time for generation of joint motions to satisfy configu-ration determinism at transitions. Since the flight time and angular momentum aboutthe COM may differ during consecutive steps, the reconfiguration problem must besolved online. Control problems for reconfiguring multilink mechanisms with zeroangular momentum have been treated in the literature, for example, Refs. [74–78].Reference [80] proposed a method based on the averaging theorem [79, Theorem 2.1]such that for any value of the angular momentum, joint motions can reorient a multi-link mechanism over an arbitrary time interval. Since this latter approach is based ondetermining roots of nonlinear equations, it cannot be employed online. We assumethat the time trajectory of a desired joint motion, precomputed offline, solves thereconfiguration problem. By replacing the time argument of the desired motion by astrictly increasing function of time called the virtual time, we show how to determinecontinuous and continuously differentiable joint motions online during consecutivesteps of running so that they also solve the reconfiguration problem. In this chapter, itis shown that the reconfiguration problem can be viewed in terms of reachability andan optimal control model for a linear time-varying system with input constraints.

5.2 MECHANICAL MODEL OF A PLANAR OPEN KINEMATIC CHAIN

Throughout this chapter, we treat a planar multilink system comprised of N ≥ 3rigid links that are connected by frictionless rotational joints and constituting an openkinematic chain conserving angular momentum about its COM (see Fig. 5.1). Thejoints have internal actuators such as dc motors. Assume that a coordinate framecalled the world frame is attached to the ground. To represent the configurationof the multilink system, a convenient choice of coordinates consists of the bodyangles, the absolute orientation, and the absolute position of the mechanical sys-tem with respect to the world frame. The body angles consist of the relative an-gles ϕ := (ϕ1, . . . , ϕN−1)′ ∈ Qb describing the shape of the multilink system, where

COM

ϕ1

θϕ2

ϕN–1

Figure 5.1 A planar multilink system composed of N ≥ 3 rigid links interconnected withfrictionless rotational joints in the form of an open kinematic chain. The configuration of themechanical system is completely determined by the body angles ϕ := (ϕ1, . . . , ϕN−1)′ ∈ Qb,the absolute orientation θ ∈ S1, and the absolute position pcm := (xcm, ycm)′ ∈ R2.

MECHANICAL MODEL OF A PLANAR OPEN KINEMATIC CHAIN 139

prime denotes matrix transpose. Furthermore, Qb is a simply connected and opensubset of

SN−1 := S1 × · · · × S1︸︷︷︸

N−1

referred to as the body configuration space, in which S1 := [0, 2π) denotes the unitcircle. The absolute orientation of the system is represented by θ, whereas the absoluteposition is represented by the Cartesian coordinates of its COM, pcm := (xcm, ycm)′ ∈R

2. Hence, the generalized coordinates for the mechanical system can be expressedas qe := (ϕ′, θ, p′

cm)′ = (q′, p′cm)′, where q := (ϕ′, θ)′. Moreover, the configuration

space, Qe, is chosen as Qe := Qb × S1 × R2.By introducing the Lagrangian of the mechanical system as the real-valued function

Le : TQe → R by

Le(qe, qe) := Ke(ϕ, qe) − Ve(qe),

where Ke and Ve represent the total kinetic and potential energy, respectively, adynamical model for describing the motions of the multilink system can be obtained.To this end, the potential energy can be expressed as Ve(qe) := mtotg0ycm, in whichmtot is the total mass of the multilink system and g0 is the gravitational constant.Moreover, the total kinetic energy of the system can be represented by the positivedefinite quadratic function Ke(ϕ, qe) := 1

2 q′eDe(ϕ)qe, where

De(ϕ) :=[

A(ϕ) 0N×2

02×N mtot I2×2

]∈ R(N+2)×(N+2) (5.1)

is a block-diagonal mass-inertia matrix. By applying Lagrange’s equation, thedynamical model of the multilink mechanism can be expressed as a nonlinear,ordinary differential equation

De(ϕ) qe + Ce(ϕ, qe) qe + Ge(qe) = Be u, (5.2)

where Ce ∈ R(N+2)×(N+2) is a matrix containing Coriolis and centrifugal terms, Ge ∈R

N+2 is a gravity vector, u := (u1, . . . , uN−1)′ ∈ RN−1 is a vector of actuator torques,and

Be :=[I(N−1)×(N−1)

03×(N−1)

]

is the input matrix. Using the block diagonal form of the mass-inertia matrix in equa-tion (5.1) and equations (7.60) and (7.62) of Ref. [90, p. 256], Ce can be expressed as

Ce(ϕ, qe) =[C(ϕ, q) 0N×2

02×N 02×2

], (5.3)


and consequently, the equation of motions in equation (5.2) can be decomposed asfollows:

A(ϕ) q + C(ϕ, q) q = B u (5.4)

xcm = 0 (5.5)

ycm + g0 = 0, (5.6)

where B := [I(N−1)×(N−1) 0(N−1)×1]′. By introducing xe := (q′e, q

′e)′ as the state

vector of the mechanical system, equation (5.2) can be expressed in following statespace form:

xe = fe(xe) + ge(xe) u.

Moreover, the state manifold is taken as the tangent bundle of Qe, that is,

Xe := TQe :={

xe := (q′e, q

′e)′|qe ∈ Qe, qe ∈ RN+2

}.

5.3 MOTION PLANNING ALGORITHM TO GENERATECONTINUOUS JOINT MOTIONS

Following the results of Section 4.3, the configuration of the mechanical systemshould be transferred from a specified initial pose to a specified final pose duringthe flight phases of running. In other words, we desire that the take-off and landingoccur in fixed configurations. Regulating the robot’s shape and absolute orientationduring flight phases is referred to as the reconfiguration problem. As mentioned inChapter 4, during running of the robot, the angular momentum about the COM andflight time may differ during consecutive steps. Consequently, the reconfigurationproblem must be solved online. The conservation of angular momentum about theCOM of the mechanical system studied here is expressed in the Nth row of matrixequation (5.4) that can be rewritten as1

θ = σcm

AN,N (ϕ)−

N−1∑i=1

AN,i(ϕ)

AN,N (ϕ)ϕi

= σcm

AN,N (ϕ)− J(ϕ) ϕ,

(5.7)

where σcm is a constant representing the angular momentum of the mechanical systemabout its COM and

J(ϕ) := 1

AN,N (ϕ)[AN,1(ϕ), . . . , AN,N−1(ϕ)] ∈ R1×N−1.

1 Since the matrix A(ϕ) is positive definite, AN,N (ϕ) > 0 for any ϕ ∈ Qb.

MOTION PLANNING ALGORITHM TO GENERATE CONTINUOUS JOINT MOTIONS 141

Remark 5.1 Since θ is a cyclic variable [1], for the LagrangianLe, the mass-inertiaand Coriolis matrices in equations (5.1) and (5.3) are independent of θ. Hence, theright-hand side of equation (5.7) is expressed as a function of ϕ and ϕ.

Assume that the twice continuously differentiable nominal trajectory ϕ∗ :[t∗1 , t∗2 ] → Qb can transfer the state of the system given in equation (5.7) from theinitial condition θ1 to the final condition θ2 when the angular momentum about theCOM is identical to σ∗

cm, that is,

θ2 = θ1 +∫ t∗2

t∗1

(σ∗

cm

AN,N (ϕ∗(s))− J(ϕ∗(s)) ϕ∗(s)

)ds.

Next we let the angular momentum about the COM be σcm, where σcm /= σ∗cm. The

objective of this section is to develop an online algorithm for generating the trajectoryϕ : [t1, t2] → Qb such that (i) ϕ(t1) = ϕ∗(t∗1 ), (ii) ϕ(t2) = ϕ∗(t∗2 ), and (iii)

θ2 = θ1 +∫ t2

t1

(σcm

AN,N (ϕ(s))− J(ϕ(s)) ϕ(s)

)ds,

where t1 /= t∗1 and t2 /= t∗2 . Let

C∗ := {ψ ∈ Qb|ψ = ϕ∗(t), t∗1 ≤ t ≤ t∗2 }C := {ψ ∈ Qb|ψ = ϕ(t), t1 ≤ t ≤ t2}

be the projections of the nominal and generated trajectories onto the bodyconfiguration space Qb. Attention is now turned to online generation of the trajectoryϕ(t), t1 ≤ t ≤ t2. For this purpose, integration of equation (5.7) over the time interval[t1, t2] implies that

θ(t2) = θ1 +∫ t2

t1

σcm

AN,N (ϕ(t))dt −

∫CJ(ϕ) dϕ. (5.8)

Analogous to the approach of Chapter 4, by assuming ϕ(t) := ϕ∗(τ(t)), where τ :[t1, t2] → [t∗1 , t∗2 ] fulfills the following constraints:

(i) τ(t1) = t∗1(ii) τ(t2) = t∗2

(iii) inft1≤t≤t2

τ(t) > 0,

(5.9)


C = C∗, and equation (5.8) can be rewritten as follows:

θ(t2) = θ1 +∫ t2

t1

σcm

AN,N (ϕ∗(τ(t)))dt −

∫C∗

J(ϕ∗) dϕ∗

= θ1 +∫ t∗2

t∗1

σcm

AN,N (ϕ∗(s))

ds

τ ◦ τ−1(s)−∫C∗

J(ϕ∗) dϕ∗.

Hence,

θ(t2) − θ2 =∫ t∗2

t∗1

1

AN,N (ϕ∗(s))

(σcm

τ ◦ τ−1(s)− σ∗

cm

)ds.

Since τ can be viewed as the argument of ϕ∗(.), it is called the virtual time. Bydefining μ(s) := 1

τ◦τ−1(s)> 0 and w(s) := 1

AN,N (ϕ∗(s)) > 0 for s ∈ [t∗1 , t∗2 ], and assum-ing σcm /= 0, the condition θ(t2) = θ2 can be expressed as the following equalityconstraint:

∫ t∗2

t∗1w(s) μ(s) ds = σ∗

cm

σcm

∫ t∗2

t∗1w(s) ds. (5.10)

Moreover, from the definition of μ(s), τ(t) = 1μ(τ(t)) , t1 ≤ t ≤ t2, and hence,

∫ t∗2

t∗1μ(s) ds = t2 − t1. (5.11)

Determination of the piecewise continuous function μ(τ) > 0, t∗1 ≤ τ ≤ t∗2 such thatequality constraints in equations (5.10) and (5.11) are satisfied is equivalent to solvingfor the open-loop control μ : [t∗1 , t∗2 ] → R

>0 transferring the state of the followingsystem in the virtual time domain:

� :x1 = w(τ) μ

x2 = μ(5.12)

from the initial condition (x1(t∗1 ), x2(t∗1 ))′ = (0, 0)′ to the final condition

(x1(t∗2 ), x2(t∗2 ))′ = (xf1 , x

f2 )′, where ˙(.) := d

dτ(.) and

xf1 := σ∗

cm

σcm

∫ t∗2

t∗1w(s) ds

xf2 := t2 − t1.

(5.13)

Due to the fact that w(τ) is continuous on the compact set [t∗1 , t∗2 ], mw :=mint∗1≤τ≤t∗2 w(τ) and Mw := maxt∗1≤τ≤t∗2 w(τ) exist and are positive scalars.


Moreover, since μ(τ) > 0, the following inequality holds:

mw

∫ t∗2

t∗1μ(s) ds ≤

∫ t∗2

t∗1w(s) μ(s) ds ≤ Mw

∫ t∗2

t∗1μ(s) ds,

which, in turn, follows 0 < mwxf2 ≤ x

f1 ≤ Mwx

f2 . Thus, the state of the system �

cannot be transferred to any arbitrary final point (xf1 , x

f2 )′ ∈ R2 by positive open-loop

control μ. In the following, we assume that σ∗cm

σcm> 0.

5.3.1 Determining of the Reachable Set from the Origin

The purpose of this subsection is to determine the reachable set from the origin at t∗1at time t∗2 for the system �. For this purpose, we present the following definitions.

Definition 5.1 (The Admissible Open-Loop Control Inputs) The set of admissibleopen-loop control inputs for system � is denoted by Um,M and defined to be the setof all piecewise continuous functions τ → μ(τ) ∈ [m, M] defined on the interval[t∗1 , t∗2 ], where 0 < m < M.

Definition 5.2 (The Reachable Set from the Origin) The reachable set from theorigin (at t∗1 ) with respect to Um,M at time t∗2 is denoted byRm,M and defined to be the

set of all points (xf1 , x

f2 )′ ∈ R2 for which there exists an admissible open-loop control

μ (i.e., μ ∈ Um,M) such that there is a trajectory of the system � with the property

(x1(t∗1 ), x2(t∗1 ))′ = (0, 0)′ and (x1(t∗2 ), x2(t∗2 ))′ = (xf1 , x

f2 )′.

To determine the set Rm,M , we first formulate an optimal control problem, inwhich the optimal admissible open-loop control input, μmax(τ) ∈ Um,M, t∗1 ≤ τ ≤ t∗2transfers the system � from the origin to the final point (x1(t∗2 ), x2(t∗2 ))′, while theperformance measure

I(μ) := x1(t∗2 ) (5.14)

is maximized.2 In this problem, we remark that x2(t∗2 ) = xf2 is specified (see point D

in Fig. 5.2). By introducing the Hamiltonian

H(x1, x2, p1, p2, μ, τ) := p1 w(τ) μ + p2 μ,

where p1 and p2 denote the costate variables, the costate equations are

pmax1 (τ) = − ∂H

∂x1

(xmax

1 , xmax2 , pmax

1 , pmax2 , μmax, τ

) = 0

pmax2 (τ) = − ∂H

∂x2

(xmax

1 , xmax2 , pmax

1 , pmax2 , μmax, τ

) = 0.

2 This problem is equivalent to the minimization of the performance −I(μ) = −x1(t∗2 ).


2fx

1fx

2fx

A

B

R

E

1fx

C

D

O

max2; m, Mfxb

min 2; m, Mfxb

m,M

maxml

maxMl

Figure 5.2 The reachable set Rm,M . Solutions of the minimization and maximization prob-lems for a given x

f

2 are represented by points C and D, respectively. The values of the costfunction I for the minimization and maximization problems are also denoted by bmin(xf

2 ; m, M)and bmax(xf

2 ; m, M). (See color version of this figure in color plates section.)

Thus, the costate variables pmax1 (τ) and pmax

2 (τ) are constant valued functions of thevirtual time. Note that the superscript “max” denotes the solutions of the maximizationproblem. Since the final value of x1 (i.e., x1(t∗2 )) is free, from Table 5.1 of Ref. [100,p. 200], pmax

1 (τ) = pmax1 (t∗2 ) = −1. Moreover, by Pontryagin’s Minimum Principle

[101] and defining c := pmax2 , the optimal open-loop control input is given by

μmax(τ) :=

⎧⎪⎨⎪⎩

m w(τ) < c

M w(τ) > c

undetermined w(τ) = c.

(5.15)

Let

T−(c) := {τ ∈ [t∗1 , t∗2] ∣∣w(τ) ≤ c

}T+(c) := {τ ∈ [t∗1 , t∗2

] ∣∣w(τ) ≥ c}

,

and assume that w(τ) satisfies the following hypothesis:

(H1) For any c ∈ [mw, Mw], the set T0(c) := T+(c) ∩ T−(c) is Lebesguenegligible.

If the function w(τ) passes through c, a switching of the optimal control input μmax(τ)is indicated. However, hypothesis H1 implies that for any c ∈ [mw, Mw], w(τ) is notequal to c on any finite time interval, and thus, the singular condition does not occur.For the later purposes, define W−(c) and W+(c) as follows:

W−(c) :=∫ t∗2

t∗1w(s) 1(c − w(s)) ds

W+(c) :=∫ t∗2

t∗1w(s) 1(w(s) − c) ds,


where 1(.) is the unit Heaviside step function. From hypothesis H1, W−(c) + W+(c) =Wmax, where Wmax := ∫ t∗2

t∗1w(s)ds because Lebesgue integration over an interval with

zero measure results in zero. Furthermore, define the functions l−, l+ : R→ R≥0 by

l−(c) :=∫ t∗2

t∗11(c − w(s)) ds

l+(c) :=∫ t∗2

t∗11(w(s) − c) ds,

which can be viewed as the Lebesgue measures of the sets T−(c) and T+(c), respec-tively. Hypothesis H1 also implies that l−(c) + l+(c) = lmax, where lmax := t∗2 − t∗1 .

Therefore, the final constraint xmax2 (t∗2 ) = x

f2 can be expressed as

xmax2 (t∗2 ) =

∫ t∗2

t∗1μ(s) ds

= ml−(c) + M(lmax − l−(c))

= xf2 ,

and consequently, c is the solution of the equation

l−(c) = Mlmax − xf2

M − m.

Since 0 ≤ l−(c) ≤ lmax, xf2 satisfies the constraint mlmax ≤ x

f2 ≤ Mlmax (see

Fig. 5.2).

Lemma 5.1 (Existence and Uniqueness of Optimal Solutions) Let ϕ∗ : [t∗1 , t∗2 ] →Qb be a C2 nominal trajectory such that hypothesis H1 is met. Then, for every 0 <

l < lmax, the equation l−(c) = l has a unique solution.

Proof. l−(c) is a well-defined function for any c ∈ R. It can be shown that3

dl−(c)

dc=∑

τ∈T0(c)

1

|w(τ)| .

Thus, for every mw ≤ c ≤ Mw for which w(τ) /= 0 on the set T0(c), l−(c) is differen-tiable at c and dl−(c)

dc> 0. From hypothesis H1, for any c ≤ mw, l−(c) = 0. Moreover,

for any c ≥ Mw, l−(c) = lmax. Hence, l−(c) is a strictly increasing function of c forevery mw < c < Mw that completes the proof. �

3 We remark that since the functions F (s, c) := 1(c − w(s)) and ∂F∂c

(s, c) are discontinuous in c, the Leibniz

integral rule cannot be applied to obtain dl−(c)dc

.


From Lemma 5.1, for every mlmax < xf2 < Mlmax, there exists a unique solution

for the equation l−(c) = Mlmax−xf

2M−m

, and hence, the final value of xmax1 can be expressed

as follows:

xmax1 (t∗2 ) =

∫ t∗2

t∗1w(s) μmax(s) ds

= MWmax − (M − m) W− ◦ l−1−

(Mlmax − x

f2

M − m

)

=: bmax(xf2 ; m, M

).

Moreover, the optimal open-loop control input can be described in terms of xf2 , m

and M as

μmax(τ; xf

2 , m, M)

:=⎧⎨⎩

m w(τ) < cmax(xf2

)M w(τ) > cmax

(xf2

),

(5.16)

where

cmax(xf2

):= l−1

−

(Mlmax − x

f2

M − m

).

We remark that the optimal open-loop control input μmax is not defined on the discon-tinuity points τ ∈ T0(cmax(xf

2 )). In other words, since the setT0(cmax(xf2 )) is Lebesgue

negligible, it therefore makes sense for the function μmax(τ; xf2 , m, M) to be undefined

on the points of discontinuity.If the optimal control problem is defined as the minimization of the performance

measure in equation (5.14) (see point C in Fig. 5.2), an analogous analysis can beperformed and it can be shown that

xmin1 (t∗2 ) = mWmax + (M − m) W− ◦ l−1

−

(xf2 − mlmax

M − m

)

=: bmin(xf2 ; m, M

).

In addition, the optimal open-loop control input is given by

μmin(τ; xf

2 , m, M)

:=⎧⎨⎩

M w(τ) < cmin(xf2

)m w(τ) > cmin

(xf2

),

(5.17)

where

cmin(xf2

):= l−1

−

(xf2 − mlmax

M − m

).


Next, we show that sufficient conditions for optimality are satisfied along theoptimal trajectories of the system �. For this purpose, the following result is presented.

Lemma 5.2 (Sufficient Conditions for Optimality) Let hypothesis H1 hold.Then, for every x

f2 ∈ [mlmax, Mlmax], the functions μmax(τ; xf

2 , m, M) and

μmin(τ; xf2 , m, M), t∗1 ≤ τ ≤ t∗2 given in equations (5.16) and (5.17) are optimal open-

loop control inputs for the maximization and minimization problems, respectively.

Proof. To verify sufficiency, we remark that the minimization and maximizationproblems for system � are equivalent to the minimization and maximization of theperformance index

J(x0, t0) :=∫ t∗2

t0

w(s) μ(s) ds

subject to the system �e : x = μ, μ ∈ Um,M transfers the state of the system �e from

the initial pair (x0, t0)′ = (0, t∗1 )′ to the final pair (xf2 , t∗2 )′. We shall verify that the

Hamilton–Jacobi–Bellman Equation is satisfied along the optimal trajectories of thesystem �e. In the proof of Lemma 5.2, sufficient conditions for the minimizationproblem are verified. A similar reasoning can also be presented for the maximizationproblem. Introduce the Hamiltonian

H(x, p, μ, τ) := (w(τ) + p) μ,

where p denotes the costate variable. From Definition 5.12 of Ref. [101, p. 357], H isnormal relative to X := X × [t∗1 , t∗2 ], where X ⊂ R is a connected set containing the

points 0 and xf2 . Hypothesis H1 implies that the control

μmin(τ) :={

m w(τ) > c

M w(τ) < c

is the H-minimal control relative to X, where c = −p. By defining

l−(c, t0) :=∫ t∗2

t0

1(c − w(s)) ds

W−(c, t0) :=∫ t∗2

t0

w(s)1(c − w(s)) ds

and also considering hypothesis H1, the final constraint xmin(t∗2 ) = xf2 can be

expressed as

(M − m)l−(c, t0) + m(t∗2 − t0) = xf2 − x0. (5.18)

Moreover, the performance index along the optimal trajectory of the minimizationproblem is given by

J min(x0, t0) = m

∫ t∗2

t0

w(s) ds + (M − m)W−(c, t0).


From equation (5.18),

(M − m)∂l−∂c

(c, t0)∂c

∂x0(x0, t0) = −1

(M − m)∂l−∂c

(c, t0)∂c

∂t0(x0, t0) + (M − m)

∂l−∂t0

(c, t0) − m = 0,

which, in turn, in combination with ∂W−∂c

(c, t0) = c∂l−∂c

(c, t0), yield

∂J min

∂x0(x0, t0) = − c

∂J min

∂t0(x0, t0) = − mw(t0) + cm + (M − m) c 1(c − w(t0))

− (M − m) w(t0) 1(c − w(t0)).

From the definition of the H-minimal control μmin(τ), we deduce that the Hamilton–Jacobi–Bellman Equation is satisfied along the optimal trajectory of the system �e,that is,

∂J min

∂t0(x0, t0) + H

(xmin(t0),

∂J min

∂x0(x0, t0), μmin(t0), t0

)= 0,

for all (x0, t0)′ ∈ X. This fact together with Theorem 5.12 of Ref. [101, p. 357]implies that sufficient conditions for optimality are satisfied, and hence, the controllaw μmin(τ; xf

2 , m, M) is optimal. �

Now we can present the main result of this section as follows.

Theorem 5.1 (Reachable Set Rm,M) Let hypothesis H1 hold. Then, for anyarbitrary m, M with the property 0 < m < M, the reachable set at t∗2 (from the originat t∗1 ) can be represented by

Rm,M ={(

xf1 , x

f2

)′ ∈ R2|mlmax ≤ xf2 ≤ Mlmax,

bmin(xf2 ; m, M

)≤ x

f1 ≤ bmax

(xf2 ; m, M

)}.

Proof. If μ(τ) ≡ m, the trajectory of the system � is transferred from the origin at t∗1to the final point (mWmax, mlmax)′ at t∗2 (see point A in Fig. 5.2). Moreover, μ(τ) ≡ M

transfers the trajectory of the system from the origin to the point (MWmax, Mlmax)′(see point B in Fig. 5.2). Thus,

bmax(mlmax; m, M) = bmin(mlmax; m, M) = mWmax

bmax(Mlmax; m, M) = bmin(Mlmax; m, M) = MWmax.


The fact that dW−(c)dc

= cdl−(c)

dcfor any mw ≤ c ≤ Mw implies that for every x

f2 ∈

(mlmax, Mlmax),

∂bmax

∂xf2

(xf2 ; m, M

)= −(M − m)

dW−dc

(cmax)∂cmax

∂xf2

(xf2

)

= cmax(xf2

)> 0,

and in a similar manner, ∂bmin

∂xf

2

(xf2 ; m, M) = cmin(xf

2 ) > 0. Moreover, from the proof

of Lemma 5.1, the derivative of the function l−1− (.) with respect to its argument ispositive, and hence,

∂2bmax

∂xf 2

2

(xf2 ; m, M

)= ∂cmax

∂xf2

(xf2

)< 0

∂2bmin

∂xf 2

2

(xf2 ; m, M

)= ∂cmin

∂xf2

(xf2

)> 0.

(5.19)

We show that bmin(xf2 ; m, M) < bmax(xf

2 ; m, M) for every xf2 ∈ (mlmax, Mlmax). For

this purpose, introduce the error function e : [mlmax, Mlmax] → R by

e(xf2

):= bmax

(xf2 ; m, M

)− bmin

(xf2 ; m, M

).

Assume that there exists xf2 ∈ (mlmax, Mlmax) such that e(xf

2 ) = 0. Since e(mlmax) =e(xf

2 ) = e(Mlmax) = 0, Rolle’s Theorem implies that there exist ξ1 ∈ (mlmax, xf2 ) and

ξ2 ∈ (xf2 , Mlmax) such that

d

dxf2

e(ξ1) = d

dxf2

e(ξ2) = 0.

However,

d2

dxf22

e(xf2

)= ∂2bmax

∂xf 2

2

(xf2 ; m, M

)− ∂2bmin

∂xf 2

2

(xf2 ; m, M

)< 0

for any xf2 ∈ (mlmax, Mlmax). Therefore, d

dxf

2

e(xf2 ) is strictly monotonic, which, in

turn, contradicts the assumed existence of xf2 . This result in combination with

equation (5.19) follows that

bmin(xf2 ; m, M

)< bmax

(xf2 ; m, M

)

for every xf2 ∈ (mlmax, Mlmax).


Next, we claim that for any (xf1 , x

f2 )′ ∈ Rm,M there exists an admissible open-loop

control input that transfers the state of the system � from the origin at t∗1 to the final

point (xf1 , x

f2 )′ at t∗2 . To show this, choose4 ϑ ∈ [0, 1] such that

xf1 = ϑ bmin

(xf2 ; m, M

)+ (1 − ϑ) bmax

(xf2 ; m, M

)(see point E in Fig. 5.2) and define the following open-loop control input:

μ

(τ; xf

1 , xf

2 , m, M

):= ϑ μmin

(τ; xf

2 , m, M

)+(

1 − ϑ) μmax(τ; xf

2 , m, M

). (5.20)

Since μmin, μmax ∈ {m, M}, μ is also admissible. Furthermore,

x1(t∗2 ) = ϑ

∫ t∗2

t∗1

μmin(s; xf

2 , m, M

)w(s) ds + (1 − ϑ)

∫ t∗2

t∗1

μmax(s; xf

2 , m, M

)w(s) ds

= ϑ bmin(x

f

2 ; m, M

)+ (1 − ϑ) bmax

(x

f

2 ; m, M

)= x

f

1

and

x2(t∗2 ) = ϑ

∫ t∗2

t∗1μmin

(s; xf

2 , m, M)

ds + (1 − ϑ)∫ t∗2

t∗1μmax

(s; xf

2 , m, M)

ds

= ϑ xf2 + (1 − ϑ) x

f2

= xf2 .

Finally, use of the sufficient conditions for optimality, as verified in Lemma 5.2,completes the proof. �

5.3.2 Motion Planning Algorithm

From the constructive proof of Theorem 5.1, an online reconfiguration algorithm tosatisfy configuration determinism at transitions can be expressed as follows.

Assume that 0 < m < M and hypothesis H1 holds.

Step 1: For the given σcm, t1, and t2, calculate xf1 and x

f2 as follows:

xf1 := σ∗

cm

σcm

∫ t∗2

t∗1w(s) ds

xf2 := t2 − t1.

4 We remark that the set Rm,M is convex.


maxMW1fx

2fx

OmaxmW

maxmin cc

minmax cc

minmax cc

minmax cc

maxmin cc

minmax cc

maxc

minc

M

M

m

mm

M

12

mMmaxMl

maxml

max1

(M2

m)l

maxmin cc

minmax cc

M

m

M

m M

m

Figure 5.3 Plot of the open-loop control μ(τ; xf

1 , xf

2 , m, M) versus w(τ) to transfer the stateof the system � from the initial condition (x1(t∗1 ), x2(t∗1 ))′ = (0, 0)′ to the final condition(x1(t∗2 ), x2(t∗2 ))′ = (xf

1 , xf

2 )′ ∈ Rm,M for 13 different typical cases. It is clear that the controlμ can switch at most twice. (See color version of this figure in color plates section.)

Step 2: Suppose that (xf1 , x

f2 )′ ∈ Rm,M . By assuming that the values of functions

bmin(xf2 ; m, M) and bmax(xf

2 ; m, M) on the interval [mlmax, Mlmax] are precom-puted and stored in a lookup table, choose ϑ ∈ [0, 1] such that

xf1 = ϑ bmin

(xf2 ; m, M

)+ (1 − ϑ) bmax

(xf2 ; m, M

).

For every τ ∈ [t∗1 , t∗2 ], let (see Fig. 5.3)

μ(τ; xf

1 , xf2 , m, M

):= ϑ μmin

(τ; xf

2 , m, M)

+ (1 − ϑ) μmax(τ; xf

2 , m, M)

.

Step 3: Over the time interval [t1, t2], integrate the following ordinary differentialequation

τ(t) = 1

μ(τ(t); xf

1 , xf2 , m, M

) (5.21)

with the initial condition τ(t1) = t∗1 .


Figure 5.4 Block diagram of the online motion planning algorithm for generation ofcontinuous joint motion ϕ(t) = ϕ∗(τ(t)), t1 ≤ t ≤ t2 to solve configuration determinism.(See color version of this figure in color plates section.)

Step 4: Define

ϕ(t) := ϕ∗(τ(t)), t1 ≤ t ≤ t2

as the continuous open-loop joint motion that transfers the configuration of themechanical system from q∗

1 to q∗2.

Figure 5.4 represents a block diagram for the proposed motion planning algorithm togenerate continuous joint motions.

5.4 MOTION PLANNING ALGORITHM TO GENERATE CONTINUOUSLYDIFFERENTIABLE JOINT MOTIONS

Based on the results of Section 5.3, this section presents an alternative online modifi-cation of the nominal trajectory ϕ∗ : [t∗1 , t∗2 ] → Qb to construct a continuously differ-entiable (C1) trajectory ϕ : [t1, t2] → Qb transferring θ from θ(t1) = θ1 to θ(t2) = θ2.Without loss of generality, we shall assume that t∗1 = 0, t∗2 = T ∗, t1 = 0, and t2 = T .To present the main results, as given in preceding sections of Chapters 4 and 5,let ϕ(t) = ϕ∗(τ(t)), where τ : [0, T ] → [0, T ∗] satisfies the constraints (i) τ(0) = 0,

MOTION PLANNING ALGORITHM TO GENERATE CONTINUOUSLY DIFFERENTIABLE 153

(ii) τ(T ) = T ∗, and (iii) ddt

τ(t) > 0 for all t ∈ [0, T ]. By defining

w(ξ) := 1

AN,N (ϕ∗(ξ))> 0

μ(ξ) := 1

τ ◦ τ−1(ξ)> 0

for ξ ∈ [0, T ∗], and also considering the fact that

∫ T

0J(ϕ) ϕ dt =

∫ T ∗

0J(ϕ∗) ϕ∗ dt,

the condition θ(T ) = θ2 can be expressed as the following equality constraint:

∫ T ∗

0w(ξ) μ(ξ) dξ = σ∗

cm

σcm

∫ T ∗

0w(ξ) dξ. (5.22)

Moreover, condition (ii) can be rewritten as

∫ T ∗

0μ(ξ) dξ = T. (5.23)

Since we desire that the generated trajectory ϕ is continuously differentiable withrespect to t, τ(t) and equivalently μ(ξ) should be continuous. Consequently, we let

μ(ξ) = 1 +∫ ξ

0v(ζ) dζ, ξ ∈ [0, T ∗], (5.24)

in which v is a piecewise continuous function. The number 1 is added to the right-handside of equation (5.24) to guarantee that for v(ξ) ≡ 0, μ(ξ) ≡ 1, which, in turn, resultsin τ(t) ≡ t (for σcm = σ∗

cm and T = T ∗). For the later purposes, assume that VL is theset of all piecewise continuous functions v : [0, T ∗] → [−L, L]. Determination ofv ∈ VL such that the constraints (5.22)–(5.24) are satisfied is equivalent to determiningthe admissible control input v ∈ VL such that the states of the following linear system

dz1

dξ(ξ) = w(ξ) z3(ξ), 0 ≤ ξ ≤ T ∗

dz2

dξ(ξ) = z3(ξ)

dz3

dξ(ξ) = v(ξ)

(5.25)

are transferred from (z1(0), z2(0), z3(0))′ = (0, 0, 1)′ to

(z1(T ∗), z2(T ∗), z3(T ∗))′ = (z1,f , z2,f , z3,f )′,


in which

z1,f := σ∗cm

σcm

∫ T ∗

0w(ξ) dξ

z2,f := T

and z3,f is free. For a given L > 0, the reachable set RL is defined to be theset of all points (z1,f , z2,f )′ ∈ R2 for which there exists an admissible controlv ∈ VL transferring the state of system (5.25) from z0 := (0, 0, 1)′ (at ξ = 0) tozf := (z1,f , z2,f , z3,f )′ (at ξ = T ∗), where z

f3 is assumed to be free. To determine

the set RL, we study the following optimal control problems for a given z2,f :

⎧⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎩

maxv∈VL z1(T ∗)

s.t. system (5.25)

z(0) = z0

z2(T ∗) = z2,f

⎧⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎩

minv∈VL z1(T ∗)

s.t. system (5.25)

z(0) = z0

z2(T ∗) = z2,f .

(5.26)

By introducing the Hamiltonian function

H(ξ, z, p) := (p1w(ξ) + p2) z3 + p3 v,

where p := (p1, p2, p3)′ is the costate vector, and also using Table 5.1 of Ref.[100, p. 200], it can be concluded that the costate variables for the maximizationproblem are

p1(ξ) ≡ −1

p2(ξ) = 1

T ∗

∫ T ∗

0w(ζ) dζ

p3(ξ) =∫ ξ

0w(ζ) dζ −

(1

T ∗

∫ T ∗

0w(ζ) dζ

)ξ.

Lemma 5.3 Assume that the equation ddξ

w(ξ) = 0 has at most one root in the openinterval (0, T ∗). Then, the nonlinear equation p3(ξ) = 0 has at most one root in(0, T ∗).

Proof. Assume that there exist ξ1, ξ2 ∈ (0, T ∗) such that ξ1 /= ξ2 (ξ1 < ξ2) andp3(ξ1) = p3(ξ2) = 0. Then, the Rolle’s Theorem implies that there is ξ3 ∈ (ξ1, ξ2)such that dp3

dξ(ξ3) = 0, or equivalently

w(ξ3) = 1

T ∗

∫ T ∗

0w(ζ) dζ.


In addition, p3(0) and p3(T ∗) are also zero. Thus, there exist ξ4 ∈ (0, ξ1) and ξ5 ∈(ξ2, T

∗) to satisfy dp3dξ

(ξ4) = dp3dξ

(ξ5) = 0, which, in turn, results in

w(ξ4) = w(ξ5) = 1

T ∗

∫ T ∗

0w(ζ) dζ.

Since w(ξ4) = w(ξ3) = w(ξ5), applying the Rolle’s Theorem follows that there existξ6 ∈ (ξ4, ξ3) and ξ7 ∈ (ξ3, ξ5) such that dw

dξ(ξ6) = dw

dξ(ξ7) = 0, which contradicts the

assumption of Lemma 5.3. �

The Pontryagin’s Minimum Principle [101] results in

v = −L sign(p3).

Following Lemma 5.3 without loss of generality, we shall assume that dwdξ

(ξ = 0+) <

0. Next, let ξmax ∈ (0, T ∗) be the root of equation pmax3 (ξ) = 0, in which the super-

script “max” denotes the solution of the maximization problem. Then

vmax(ξ) = L sign(ξ − ξmax)

zmax3 (ξ) = 1 + L σ1(ξ; ξmax)

zmax2 (ξ) = ξ + L σ2(ξ; ξmax),

where

σ1(ξ; ξmax) :=∫ ξ

0sign(ζ − ξmax) dζ

σ2(ξ; ξmax) :=∫ ξ

0σ1(ζ; ξmax) dζ.

The boundary condition zmax2 (T ∗) = z2,f also implies that

ξmax(z2,f ) = T ∗ −√

1

L

(z2,f − T ∗ + L

2T ∗2

). (5.27)

An analogous analysis for the minimization problem results in

pmin1 (ξ) ≡ 1

pmin2 (ξ) = − 1

T ∗

∫ T ∗

0w(ζ) dζ

pmin3 (ξ) = −

∫ ξ

0w(ζ) dζ +

(1

T ∗

∫ T ∗

0w(ζ) dζ

)ξ


(see Table 5.1 of Ref. [100, p. 200]) that together with the assumption dwdξ

(ξ = 0+) < 0yields

vmin(ξ) = −L sign(ξ − ξmin)

zmin3 (ξ) = 1 − L σ1(ξ; ξmin)

zmin2 (ξ) = ξ − L σ2(ξ; ξmin),

where

ξmin(z2,f ) = T ∗ −√

1

L

(L

2T ∗2 + T ∗ − z2,f

). (5.28)

From equations (5.27) and (5.28), it can be concluded that

z2,f ∈[T ∗ − L

2T ∗2, T ∗ + L

2T ∗2].

Furthermore, we define

Zmax1 (z2,f ) :=

∫ T ∗

0w(ζ) zmax

3 (ζ) dζ

and

Zmin1 (z2,f ) :=

∫ T ∗

0w(ζ) zmin

3 (ζ) dζ

as the values of the cost functions in the maximization and minimization problems,respectively. Since the system (5.25) is linear,RL is convex. In addition, Theorem 5.12of Ref. [101, p. 357] implies that the sufficient conditions for optimality are satisfiedalong the optimal solutions (see Section 5.3, Lemma 5.2 for a similar optimizationproblem). Thus, the reachable set can be expressed as

RL = {(z1,f , z2,f )′ ∈ R2|Zmin1 (z2,f ) ≤ z1,f ≤ Zmax

1 (z2,f ), z2,f ≤ z2,f ≤ z2,f },

in which z2,f := T ∗ − L2 T ∗2 and z2,f := T ∗ + L

2 T ∗2. The following theorempresents the main result of this section.

Theorem 5.2 (Motion Planning Algorithm) Assume that the functions Zmax1 and

Zmin1 on the interval [z2,f , z2,f ] are precomputed and stored in a lookup table. For a

given σcm and T , calculate z1,f := σ∗cm

σcm

∫ T ∗0 w(ξ) dξ and z2,f := T . If (z1,f , z2,f )′ ∈

RL, let ϑ ∈ [0, 1] be such that

z1,f = ϑ Zmin1 (z2,f ) + (1 − ϑ) Zmax

1 (z2,f ).


Then, the trajectory

ϕ(t) = ϕ∗(s1(t)), 0 ≤ t ≤ T

can transfer the state of system (5.7) form θ(0) = θ1 to θ(T ) = θ2, where s1 is thestate of the following system with the initial condition (s1(0), s2(0))′ = (0, 1)′

s1 = s2

s2 = − ϑ vmin(s1) + (1 − ϑ) vmax(s1)(ϑzmin

3 (s1) + (1 − ϑ)zmax3 (s1)

)3 .(5.29)

Proof. Let us define s1(t) := τ(t) = ξ and s2(t) := τ(t). Since system (5.25) is linear,

v(s1) = ϑ vmin(s1) + (1 − ϑ) vmax(s1) (5.30)

implies that

μ(s1) = z3(s1) = ϑ zmin3 (s1) + (1 − ϑ) zmax

3 (s1). (5.31)

We also remark that

z2(T ∗) = ϑ z2,f + (1 − ϑ) z2,f = z2,f

z1(T ∗) = ϑ Zmin1 (z2,f ) + (1 − ϑ) Zmax

1 (z2,f ) = z1,f .

By considering the fact that τ(t) = 1μ(τ(t)) , it can be concluded that

s2(t) = −∂μ/∂τ(τ(t))

μ2(τ(t))τ(t) = −∂μ/∂τ(τ(t))

μ3(τ(t)), (5.32)

which, in turn, results in equation (5.29). In addition, the right-hand side ofsystem (5.29) is discontinuous with respect to s1. However, from Lemma 2 ofRef. [102, p. 107], there exists a unique solution for system (5.29) through(s1(0), s2(0))′ = (0, 1)′ (at t = 0). �

Figure 5.5 depicts the trajectories for the joint angles (q1 and q2) generated bythe algorithm of Theorem 5.2 and the absolute orientation q3 during the flight phasesof four consecutive steps for the monopedal robot investigated in Section 4.6. It isclearly observed that configuration determinism at landing is satisfied.

Figure 5.5 Plot of the desired trajectories for the joint angles (i.e., q1 and q2) generated by the online motion planning algorithm of Theorem 5.2,the absolute orientation (q3) versus normalized time during the ßight phases of four consecutive steps (solid curves) and the projection of the statevariables onto the conÞguration space. The nominal trajectory is depicted by dashed curves. The circles at the both ends represent the initial and Þnalpredetermined conÞgurations. (See color version of this Þgure in color plates section.)

158

CHAPTER 6

Stabilization of Periodic Orbits for 3DMonopedal Running

6.1 INTRODUCTION

This chapter presents a motion planning algorithm to generate periodic time trajecto-ries for running by a 3D monopedal robot. In order to obtain a symmetric gait alonga straight line, the overall open-loop model of running can be expressed as a hybridsystem with four continuous phases consisting of two stance phases (the leg is onthe ground) and two flight phases (the leg is off the ground), and discrete transitionsbetween them (takeoff and impact). The robot studied here is a 3D, three-link, three-actuator, monopedal mechanism with a point foot. During the stance phases, the robothas three degrees of underactuation, whereas it has six degrees of underactuation inthe flight phases. The motion planning algorithm is developed on the basis of a finite-dimensional nonlinear optimization problem with equality and inequality constraints.The main objective of this chapter is to develop time-invariant feedback scheme toexponentially stabilize a desired periodic orbit generated by the motion planning al-gorithm for the hybrid model of running. When the amount of underactuation duringlocomotion of legged robots is increased, it becomes difficult to create hybrid invariantmanifolds. Reference [60] proposed a method to generate an open-loop augmentedsystem with impulse effects, a parameterized holonomic output function for the resul-tant system, and an event-based update law for the parameters of the output such thatthe zero dynamics manifold associated with this output is hybrid invariant under theclosed-loop augmented system. Recently, this approach has been used in the designof time-invariant controllers for walking of a 3D biped robot in Refs. [61, 62]. In thischapter, we show how to create hybrid invariant manifolds during 3D running [85]. Byassuming that the control inputs of the mechanical system have discontinuities duringdiscrete transitions between continuous phases, the takeoff switching hypersurfacecan be expressed as a zero-level set of a scalar holonomic function. In other words,takeoff occurs when a scalar quantity, a strictly increasing function of time on thedesired gait, passes through a threshold value. The virtual constraints during stance


159

160 STABILIZATION OF PERIODIC ORBITS FOR 3D MONOPEDAL RUNNING

phases are defined as the summation of two terms including a nominal holonomicoutput function vanishing on the periodic orbit and an additive parameterized Bezierpolynomial, both in terms of the latter strictly increasing scalar. By properties of Bezierpolynomials, an update law for the parameters of the stance phase virtual constraintsis developed, which, in turn, results in a common intersection of the parameterizedstance phase zero dynamics manifolds and the takeoff switching hypersurface. Bythis approach, creation of hybrid invariance can be easily achieved by updating theother parameters of the Bezier polynomial. Consequently, a parameterized restrictedPoincare return map can be defined on the common intersection for studying thestabilization problem. Thus, the overall feedback scheme can be considered at twolevels. At the first level, within-stride controllers including stance and flight phasecontrollers, which are continuous time-invariant and parameterized feedback laws,are employed to create a family of attractive zero dynamics manifolds in each of thecontinuous phases. At the second level, the parameters of the within-stride controllersare updated by event-based update laws during discrete transitions between continu-ous phases to achieve hybrid invariance and stabilization. By this means, the stabilityanalysis of the periodic orbit for the full-order hybrid system can be treated in termsof a reduced-order hybrid system with a five-dimensional Poincare return map.

6.2 OPEN-LOOP HYBRID MODEL OF A 3D RUNNING

In this section, we present an open-loop hybrid model for describing running by a3D monopedal robot as illustrated in Fig. 6.1. The robot consists of three rigid links:a torso link and a leg with tibia and shin links. The links are connected by a set ofactuated body joints including a two degree of freedom revolute hip joint and a onedegree of freedom revolute knee joint. It is assumed that the robot has a point footand cannot apply torques at the end of its leg. The body angles that determine theshape of the monoped are denoted by ϕ := (ϕ1, ϕ2, ϕ3)′ as shown in Fig. 6.1, whereprime represents matrix transpose. Assume that o0x0y0z0 is an inertial world frameattached to the ground. Now attach the torso frame otxtytzt rigidly to the torso linkwith the origin on its COM. The orientation of the torso frame relative to the worldframe can be represented by the rotation matrix

R(θ) = Rz(θ3) Rx(θ1) Ry(θ2),

in which θ := (θ1, θ2, θ3)′ and Rx, Ry, and Rz are basic rotations about the x, y,and z axes, respectively. We assume that the COM frame ocxcyczc is a frame withthe same orientation matrix R(θ) relative to the world frame such that its origin ison the COM of the robot. Let pcm := (xcm, ycm, zcm)′ denote the position of theCOM relative to the world frame. For describing the configuration of the mechanicalsystem during the flight phase, the vector of generalized coordinates can be definedas qf := (ϕ′, θ′, p′

cm)′ = (q′, p′cm)′, where q := (ϕ′, θ′)′ and the subscript “f ” will

designate the flight phase. The configuration space for the flight phase is also denotedby Qf .

OPEN-LOOP HYBRID MODEL OF A 3D RUNNING 161

Figure 6.1 A three-link, three-actuator 3D monopedal robot with point foot. The body jointsare denoted by ϕ := (ϕ1, ϕ2, ϕ3)′. The orientation of the torso and COM frames relative tothe world frame can be expressed by the rotation matrix R(θ) = Rz(θ3)Rx(θ1)Ry(θ2), whereθ := (θ1, θ2, θ3)′. R can be considered as the following sequence of basic rotations in the orderspecified: (i) a rotation of θ3 about the fixed z-axis, (ii) a rotation of θ1 about the current x-axis,and (iii) a rotation of θ2 about the current y-axis. The virtual leg is depicted as a dashed lineconnecting the end of the leg and the hip joint. The angle of the virtual leg in the sagittal planecan be expressed as γ = −ϕ2 + ϕ3

2 + θ3 + π

2 .

Let pi ∈ R3 for i = 1, 2, 4 represent the position of the COM of the tibia, shin,and torso links relative to the world frame, respectively. The masses of these linksare also given by mi for i = 1, 2, 4. Furthermore, p3 ∈ R3 denotes the position ofthe lumped mass m3 located at the hip joint. Let �i(q), i = 1, 2, 3, 4 represent theCartesian coordinates of the COM with respect to the mass mi. Then, pi can beexpressed as

pi = pcm − �i(q) = pcm − R(θ) ϒi(ϕ), i = 1, 2, 3, 4, (6.1)

where ϒi ∈ R3, i = 1, 2, 3, 4 are smooth functions with respect to ϕ with the pro-perty

∑4i=1 miϒi(ϕ) = 0. Since

∑4i=1 mipi/

∑4i=1 mi = pcm, it can be concluded

that∑4

i=1 mi�i(q) = 0 and∑4

i=1 miϒi(ϕ) = 0. In an analogous manner, the positionof the leg end with respect to the world frame can be expressed as pl = pcm − �l(q) =pcm − R(θ)ϒl(ϕ), where the subscript “l” denotes the leg end. Now suppose thatduring the stance phase, the leg end is on the origin of the world frame, then pcmcan be expressed as pcm = �l(q). Consequently, the generalized coordinates for thestance phase is given by qs := q = (ϕ′, θ′)′, where the subscript “s” denotes the stancephase.



Throughout this chapter, all of the link masses except the mass of the torso link areassumed to be lumped. Let I4 denote the inertia tensor expressed in the torso frame.During the flight phase, the angular velocity vector of the torso link relative to theworld frame is given by ω = (ωx, ωy, ωz)′ = (θ)θ, where

(θ) :=

⎡⎢⎣

cos(θ3) − sin(θ3) cos(θ1) 0

sin(θ3) cos(θ3) cos(θ1) 0

0 sin(θ1) 1

⎤⎥⎦ . (6.2)

Considering the fact that∑4

i=1 mi�i(q) = 0, the positive definite quadratic kineticenergy of the mechanical system during the flight phase, Kf : TQf → R, can beexpressed as

Kf (qf , qf ) = 1

2

4∑i=1

mi p′i pi + 1

2ω′ R I4 R′ ω = 1

2q′

f Df (q) qf ,

where Df (q) := block diag{A(q), mtotI3×3} ∈ R9×9, mtot := ∑4i=1 mi, and

A(q) :=4∑

i=1

mi

∂�i

∂q

′ ∂�i

∂q+[

03×3 03×3

03×3 ′ R I4 R′

]∈ R6×6. (6.3)

By introducing the Lagrangian of the flight phase as Lf (qf , qf ) := Kf (qf , qf ) −Vf (qf ), where Vf : Qf → R by

Vf (qf ) := mtot g0 zcm

is the total potential energy of the robot and g0 is the gravitational constant, thesecond-order dynamical equation of motions during the flight phase can be expressedas

Df (q) qf + Cf (q, qf ) qf + Gf (qf ) = Bf u, (6.4)

in which, Cf (q, qf ) is a (9 × 9) Coriolis and centrifugal matrix, Gf (qf ) is a (9 × 1)gravity vector, and u := (u1, u2, u3)′ ∈ R3×1 is the vector of applied torques at thebody joints. Moreover,

Bf :=[

B

03×3

]:=

⎡⎢⎢⎣[I3×3

03×3

]

03×3

⎤⎥⎥⎦ ∈ R9×3.


Using the Christoffel symbols [90, p. 256] and the block-diagonal form of the mass-inertia matrix, it can be shown that the Coriolis matrix Cf also has the block diagonalform as

Cf (q, qf ) = block diag{C(q, q), 03×3}.

Due to the block diagonal form of the mass-inertia matrix (i.e., Df ) and Coriolis andcentrifugal (i.e., Cf ) matrix, qf = (q′, p′

cm)′ and

Gf (qf ) = ∂Vf

∂qf

′=mtot g0

[08×1

1

],

the equation of motion in (6.4) can be decomposed as follows:

A(q) q + C(q, q) q = B u

mtot pcm = mtot

⎡⎢⎣

0

0

−g0

⎤⎥⎦ .

(6.5)

Remark 6.1 (Cyclic Variables of the Flight Phase) During the flight phase, θ3(the orientation about the z-axis), xcm, ycm, and zcm are cyclic variables [1] in the

sense that∂Kf

∂θ3= ∂Kf

∂xcm= ∂Kf

∂ycm= ∂Kf

∂zcm= 0. Thus, Df and Cf in equation (6.4) are

independent of them. By introducing the state vector xf := (q′f , q′

f )′ for the flightphase, equation (6.4) can be expressed in a state space form xf = ff (xf ) + gf (xf )u.Moreover, the state space for the flight phase is taken as the tangent bundle of Qf ,that is,

Xf := TQf := {(q′f , q′

f )′|qf ∈ Qf , qf ∈ R9}.

6.2.2 Dynamics of the Stance Phase

During the stance phase, we assume that the position of the leg end is on the originof the world frame (i.e., pcm = �l(q)). Let F := (Fx, Fy, Fz)′ ∈ R3×1 represent theground reaction force at the leg end. Then, by applying the principle of virtual work,1

equation (6.5) can be reduced as follows:

Ds(qs) qs + Cs(qs, qs) qs + Gs(qs) = B u

mtot∂�l

∂qs

(qs) qs + mtot∂

∂qs

(∂�l

∂qs

(qs)qs

)qs + mtot

⎡⎢⎣

0

0

g0

⎤⎥⎦ = F,

(6.6)

1 Applying the principle of the virtual work in equation (6.5) results in Aq + Cq = Bu − ∂�l/∂qF andmtotpcm = mtot[0 0 − g0]′ + F , which together with pcm = �l(q) yield equation (6.6).


where

Ds := A + mtot∂�l

∂qs

′ ∂�l

∂qs

∈ R6×6

Cs := C + mtot∂�l

∂qs

′ ∂

∂qs

(∂�l

∂qs

qs

)∈ R6×6

Gs := mtot∂�l

∂qs

′⎡⎢⎣

0

0

g0

⎤⎥⎦∈ R6×1.

Remark 6.2 (The Cyclic Variable of the Stance Variable) The configurationmanifold of the stance phase can be expressed as the submanifold Qs := {qf ∈Qf |pl(qf ) = 03×1} and the stance phase Lagrangian can be given by Ls := Lf |TQs .Moreover, θ3 is the cyclic variable during the stance phase in the sense that ∂Ks

∂θ3= 0.

Thus, Ds and Cs in equation (6.6) are independent of θ3.

Remark 6.3 (Validity of the Stance Phase) The stance phase model is valid if theground reaction force, F , satisfies the constrains (i) Fz > 0 (to avoid takeoff) and

(ii) |√

F2x + F2

y /Fz| < μs (to avoid slipping), where μs denotes the static friction

coefficient between the leg end and the ground. By defining the state vector xs :=(q′

s, q′s)

′ ∈ Xs, where

Xs := TQs := {(q′s, q

′s)

′|qs ∈ Qs, qs ∈ R6},

a state equation for describing the evolution of the mechanical system during thestance phase can be expressed as xs = fs(xs) + gs(xs)u.

6.2.3 Transition Maps

The transition from flight to stance (impact) takes place when the height of the legend becomes zero. Thus, define the impact switching hypersurface as

Ssf := {xf = (q′

f , q′f )′ ∈ Xf |pv

l (qf ) := zcm − �vl (q) = 0},

where the superscript “v” denotes the vertical component of the position vector. Thistransition can be modeled as x+

s = �sf (x−

f ), where �sf : Ss

f → Xs denotes the impactmap and the superscripts “−” and “+” represent the state of the mechanical systemjust before and after the transition, respectively. By extending the planar impact modelpresented in Ref. [18, pp. 74–75] for the mechanical system of Fig. 6.1, �s

f can beexpressed as

�sf (x−

f ) :=[

[I6×6 06×3] q−f

�sf (q−

f ) q−f

], (6.7)


in which

�sf (q−

f ) :=(

A + mtot∂�l

∂q

′ ∂�l

∂q

)−1 [A mtot

∂�l

∂q

′].

Remark 6.4 (Validity of the Impact Model) The impact model is valid if the inten-sity of the impulsive ground reaction force, F := (Fx, Fy, Fz)′, satisfies the constraints

(i) Fz > 0 and (ii) |√

F2x + F2

y/Fz| < μs. In addition, F can be given by

F = mtot(p+cm − p−

cm) = mtot

(∂�l

∂q(q−) �s

f (q−f ) q−

f − p−cm

). (6.8)

The transition from stance to flight (takeoff) occurs when the vertical compo-nent of the leg end acceleration becomes positive. However for simplicity, it is as-sumed that this transition takes place when the angle of the virtual leg in the sagittalplane that is denoted by γ (see Fig. 6.1) passes through the threshold value γ−. Weremark that for the mechanical system of Fig. 6.1,2 γ can be expressed as

γ(qs) = −ϕ2 + ϕ3

2+ θ3 + π

2.

Consequently, the takeoff switching hypersurface can be defined as

Sfs := {xs = (q′

s, q′s)

′ ∈ Xs|γ(qs) − γ− = 0}.

In addition, by assuming that the position and velocity remain continuous during thistransition, the takeoff map �

fs : Sf

s → Xf can be expressed as

�fs (x−

s ) :=

⎡⎢⎢⎢⎢⎣

[q−s

�l(q−s )

][

q−s

∂�l

∂q(q−

s ) q−s

]⎤⎥⎥⎥⎥⎦ . (6.9)

Remark 6.5 (Validity of the Takeoff Model) The transition model from stance toflight is valid if the feedback law for the flight phase is designed such that the verticalcomponent of the leg end acceleration is positive at the beginning of the flight phase(i.e., pv

l > 0). To achieve this objective, it is assumed that control inputs may havediscontinuities during transitions between continuous phases.

2 It is assumed that the lengths of the tibia and shin links are equal, that is, L1 = L2.


6.2.4 Hybrid Model

A symmetric monopedal running motion along the x-axis of the world frame can beconsidered as a periodic orbit composed of two consecutive steps

O = O1s ∪ O1

f︸︷︷︸step 1

∪O2s ∪ O2

f︸︷︷︸step 2

in the overall state space of the mechanical system, in which Oji for i ∈ {s, f } and

j ∈ {1, 2} denotes the intersection ofO and the state space of the jth continuous phaseof type i. This is resulted from the property that the leg would be in the left-hand sideof the x-axis during a step if it was on the right-hand side of the same axis in theprevious step. In other words, if we assume that T ∗

s and T ∗f are the time durations

of the stance and flight phases on a step of O and define T ∗ := T ∗s + T ∗

f as the timeduration of a step, then symmetry implies that for every t ≥ 0 on the periodic orbit

ϕ(t + T ∗) = S1 ϕ(t)

θ(t + T ∗) = S2 θ(t)

pcm(t + T ∗) = S3 pcm(t),

(6.10)

where S1 := diag{−1, 1, 1}, S2 := diag{−1, 1, −1}, and S3 := diag{1, −1, 1} (seeFig. 6.2 as a typical periodic motion with T ∗ = 0.5730(s)). This fact motivates us tostudy the period-one solutions and stabilization for the following open-loop hybridsystem composed of four continuous phases:

�1s :

{x1s = fs

(x1s

)+ gs

(x1s

)u x1−

s /∈ Sfs

x1+f = �

fs

(x1−s

)x1−s ∈ Sf

s

�1f :

⎧⎨⎩

x1f = ff

(x1f

)+ gf

(x1f

)u x1−

f /∈ Ssf

x2+s = �s

f

(x1−f

)x1−f ∈ Ss

f

�2s :

{x2s = fs

(x2s

)+ gs

(x2s

)u x2−

s /∈ Sfs

x2+f = �

fs

(x2−s

)x2−s ∈ Sf

s

�2f :

⎧⎨⎩

x2f = ff

(x2f

)+ gf

(x2f

)u x2−

f /∈ Ssf

x1+s = �s

f

(x2−f

)x2−f ∈ Ss

f ,

(6.11)

where the notation (.)ji for i ∈ {s, f } and j ∈ {1, 2} corresponds to the jth continuousphase of type i. In addition, the superscripts “−” and “+” denote the state just beforeand after the discrete transitions.

DESIGN OF A PERIOD-ONE SOLUTION FOR THE OPEN-LOOP MODEL OF RUNNING 167

0 1 2−0.2

0

0.2

φ 1 (ra

d)

0 1 20

0.5

1

1.5

φ 2 (ra

d)

0 1 20

1

2

φ 3 (ra

d)

0 1 2−0.2

0

0.2

θ 1 (ra

d)

0 1 20

0.2

0.4

θ 2 (ra

d)

0 1 2−0.1

0

0.1

θ 3 (ra

d)

0 1 2−4

−2

0

2

4

d/d

tφ1 (

rad/

s)

0 1 2−10

0

10

20

d/d

tφ2 (

rad/

s)

0 1 2−10

0

10

d/d

tφ3 (

rad/

s)

0 1 2−2

0

2

d/d

tθ1 (

rad/

s)

Time (s) 0 1 2

−2

0

2

4

6

d/d

tθ2 (

rad/

s)

Time (s) 0 1 2

−1

0

1

d/d

tθ3 (

rad/

s)

Time (s)

Figure 6.2 Angular positions and velocities during four consecutive steps of the optimalmotion (T ∗ = 0.5730(s)). Bold and light curves correspond to stance and flight phases, respec-tively. Discontinuities are due to impacts. (See the color version of this figure in the color platessection.)

6.3 DESIGN OF A PERIOD-ONE SOLUTION FOR THEOPEN-LOOP MODEL OF RUNNING

The objective of this section is to present a motion planning algorithm based on a finite-dimensional nonlinear optimization problem with equality and inequality constraintsto generate a feasible period-one solution for the open-loop hybrid model of runningin equation (6.11). We first present the following definition.

Definition 6.1 (Feasible Periodic Orbit) The periodic orbit O of the open-loophybrid model in equation (6.11) is said to be feasible if

1. the constraints on the joint angles representing an anthropomorphic gait andthe constraints on the angular velocities representing the actuation limits aresatisfied on O;

2. the open-loop control input corresponding to the trajectory O is admissible inthe sense that ‖u‖L∞ := supt≥0 ‖u(t)‖ is less than a physically realizable valueumax > 0;


3. the ground reaction forces during stance phases of the orbit O are admissibleas stated in Remark 6.3;

4. the intensity of impulsive ground reaction forces during impacts are admissibleas stated in Remark 6.4;

5. the vertical component of the leg end acceleration at the beginning of flightphases is positive;

6. the height of the leg end during flight phases is positive.

Now let N ≥ 1 and M ≥ 0 be two integer numbers. For any matrix α =[α0 · · · αM] = col{αi}Mi=0 ∈ RN×(M+1), the Bezier polynomial with the parameter α

denoted byB(., α) : R→ RN is defined as follows:

B(s, α) :=M∑i=0

M!

i!(M − i)!αis

i(1 − s)M−i. (6.12)

To develop the motion planning algorithm, assume that

x1−∗s :=

[q1−∗s

q1−∗s

]∈ Sf

s , x1−∗f :=

[q1−∗f

q1−∗f

]∈ Ss

f (6.13)

represent the states of the mechanical system at the end of the first stance and flightphases of the periodic orbitO (just before the corresponding transitions), respectively,where the superscript “∗” denotes the quantities corresponding to the periodic orbit.By using equation (6.10) to generate a symmetric periodic gait along the x-axis, thefinal states of the second stance and flight phases (i.e., x2−∗

s and x2−∗f ) are defined as

follows:

x2−∗s := Ss x1−∗

s

x2−∗f := Sf x1−∗

f ,(6.14)

where

Ss := block diag{S1, S2, S1, S2} ∈ R12×12

Sf := block diag{S1, S2, S3, S1, S2, S3} ∈ R18×18.

We will assume that phases are executed in a fixed order

stance1 → flight1 → stance2 → flight2 → stance1. (6.15)

According to the order in equation (6.15) and transition maps in equations (6.7)and (6.9), the states just after the transitions on each of the continuous phases


of O can be given by

x1+∗s = �s

f

(x2−∗f

)x2+∗s = �s

f

(x1−∗f

)x1+∗f = �f

s

(x1−∗s

)x2+∗f = �f

s

(x2−∗s

).

(6.16)

Since the body joints are independently actuated, we choose a Bezier polynomialevolution of time for the body angles ϕ during the stance and flight phases. For thispurpose, define the scaled times

ss := t

T ∗s

, sf := t − T ∗s

T ∗f

.

Suppose that Ms ≥ 3 and Mf ≥ 3 are the degrees of the Bezier polynomials duringthe stance and flight phases, respectively. In addition, let the matrices α1∗, α2∗ ∈R

3×(Ms+1) and β1∗, β2∗ ∈ R3×(Mf +1) be the parameters of the Bezier polynomialsduring the first and second stance and flight phases of the periodic orbit O. Weremark that symmetric gait conditions in equation (6.10) imply that α2∗ = S1α

1∗ andβ2∗ = S1β

1∗. Next, let

ϕ(t) = B(ss, α

1∗)

, 0 ≤ t ≤ T ∗s

ϕ(t) = B(sf , β1∗

), T ∗

s ≤ t ≤ T ∗s + T ∗

f = T ∗.(6.17)

Following properties of Bezier polynomials given in Remark 3.16, the initial and finalconditions on the angular position and velocity of the body joints can be taken intoaccount in the following manner:

α1∗0 =

(q1+∗s

)ϕ

β1∗0 =

(q1+∗f

)ϕ

α1∗1 = α1∗

0 + T ∗s

Ms

(q1+∗

s

)ϕ

β1∗1 = β1∗

0 + T ∗f

Mf

(q1+∗

f

)ϕ

α1∗Ms

=(q1−∗s

)ϕ

β1∗Mf

=(q1−∗f

)ϕ

α1∗Ms−1 = α1∗

Ms− T ∗

s

Ms

(q1−∗

s

)ϕ

β1∗Mf −1 = β1∗

Mf− T ∗

f

Mf

(q1−∗

f

)ϕ,

(6.18)

where the subscript “ϕ” denotes the components corresponding to the body joints.Moreover, if we define H(q, q) := Cs(q, q)q + Gs(q) and decompose dynamicalequation (6.6) into ϕ and θ components, the evolution of θ during the stance phase isgiven by

θ = −D−1s,θθ(q) Ds,θϕ(q) ϕ − D−1

s,θθ(q) Hθ(q, q), (6.19)

where Ds,θθ and Ds,θϕ are the (3 × 3) lower right and left submatrices of Ds, respec-tively. Furthermore, Hθ represents the last three rows of H . We remark that in this


latter equation, ϕ can be considered as an input. A similar procedure during the flightphase (see equation (6.5)) yields

θ = −A−1θθ (q) Aθϕ(q) ϕ − A−1

θθ (q) Jθ(q, q)

mtot pcm = mtot

⎡⎢⎣

0

0

−g0

⎤⎥⎦ ,

(6.20)

in which J(q, q) := C(q, q)q. Now we are in a position to present the following lemmathat states that for the motion planning algorithm, it is sufficient to generate and studythe first step of O.

Lemma 6.1 (Symmetric Gait) For the mechanical system of Fig. 6.1, the followingstatements are true.

1. Assume that θ∗(t), 0 ≤ t ≤ T ∗s is the unique solution of equation (6.19) when

ϕ(t) = ϕ∗(t). Then, for the initial condition

θ(0) = S2 θ∗(0)

θ(0) = S2 θ∗(0)

and the input

ϕ(t) = S1 ϕ∗(t), 0 ≤ t ≤ T ∗s ,

the trajectory

θ(t) = S2 θ∗(t)

is the unique solution for (6.19).

2. Assume that (θ∗(t), p∗cm(t)) for T ∗

s ≤ t ≤ T ∗ is the unique solution of equation(6.20) when ϕ(t) = ϕ∗(t). Then, for the initial condition

(θ(T ∗s ), pcm(T ∗

s )) = (S2 θ∗(T ∗s ), S3 p∗

cm(T ∗s ))

(θ(T ∗s ), pcm(T ∗

s )) = (S2 θ∗(T ∗s ), S3 p∗

cm(T ∗s ))

and the input

ϕ(t) = S1 ϕ∗(t), T ∗s ≤ t ≤ T ∗,

the trajectory

(θ(t), pcm(t)) = (S2 θ∗(t), S3 p∗cm(t))

is the unique solution of (6.20).


The proof is given in Appendix C.1. On the basis of Lemma 6.1, we consider themotion planning algorithm only during the first stance and flight phases (O1

s and O1f ).

By this approach, O2s and O2

f can be obtained by applying the linear maps Ss and Sf

on O1s and O1

f , respectively, that is,

O2s = Ss O1

s , O2f = Sf O1

f .

Consequently, the evolution of the mechanical system on the periodic orbit O can becompletely determined by the following vector of parameters:

�∗ :=(q1−∗′s , q1−∗′

s , q1−∗′f , q1−∗′

f , T ∗s , T ∗

f , α1∗′2 , · · · α1∗′

Ms−2, β1∗′2 , . . . , β1∗′

Mf −2

)′,

(6.21)which are utilized in the first step. Next, we present the following motion planningalgorithm.

Algorithm 6.1 Motion Planning Algorithm for Generating a Periodic Orbit

1. Choose the degrees of the Bezier polynomials as Ms ≥ 3 and Mf ≥ 3.

2. Select �∗ and using equations (6.13), (6.14), and (6.16) calculate the initialconditions for the first stance and flight phases. From equation (6.18), calculatethe first and last two columns of the parameter matrices α1∗ = [α1∗

0 · · · α1∗Ms

] =col{α1∗

i }Ms

i=0 and β1∗ = [β1∗0 · · · β1∗

Mf] = col{β1∗

i }Mf

i=0.

3. Integrate equation (6.19) on the interval [0, T ∗s ] with the initial condition ob-

tained in Step 2 for the first stance phase. Calculate the open-loop control inputand the ground reaction force by applying equation (6.6).

4. Integrate equation (6.20) on the interval [T ∗s , T ∗] with the initial condition

obtained in Step 2 for the first flight phase. Calculate the open-loop controlinput by applying equation (6.5). Obtain the vertical acceleration of the leg endat the beginning of the flight phase as follows:

pvl = − g0 − ∂�v

l

∂qq − ∂

∂q

(∂�v

l

∂qq

)q.

5. Since θ and (θ, pcm) are unactuated degrees of freedom during the stance andflight phases, to have a periodic orbit for the open-loop hybrid model of run-ning (6.11), �∗ should be designed such that the final values of the positionand velocity vectors corresponding to these DOF are equal to their predeter-mined values that are given in �∗. To achieve this goal, introduce the equalityconstraint vector ce(�∗) and evaluate its components as follows:

ce,1(�∗) := θ(T ∗s ) − (

q1−∗s

)θ

ce,5(�∗) := pcm(T ∗) − (q1−∗

f

)pcm

ce,2(�∗) := θ(T ∗s ) − (

q1−∗s

)θ

ce,6(�∗) := pcm(T ∗) − (q1−∗

f

)pcm

ce,3(�∗) := θ(T ∗) − (q1−∗

f

)θ

ce,7(�∗) := pvl

(q1−∗

f

),

ce,4(�∗) := θ(T ∗) − (q1−∗

f

)θ

(6.22)


where the subscripts “θ” and “pcm” denote those components corresponding toθ and pcm, respectively.

6. Evaluate a cost function and an inequality constraint vector cie(�∗) such thatcie(�∗) ≤ 0 guarantees the feasibility of the optimal trajectory (as stated inDefinition 6.1) and invertibility of the decoupling matrix during the stancephase (to be stated in Remark 6.7).

7. Repeat Steps 2–6 until ce(�∗) = 0, cie(�∗) ≤ 0 and the cost function (6.23) isminimized.

Remark 6.6 (Interpretation of the Equality Constraints) The equality con-straints ce,j(�∗) = 0, j = 1, . . . , 6 are necessary and sufficient conditions by whichthe open-loop hybrid model of running (6.11) has a period-one solution. Moreover,ce,7(�∗) = pv

l (q1−∗f ) = 0 in equation (6.22) implies that the height of the leg end

at the end of the flight phase is zero, that is, x1−∗f ∈ Ss

f . We also observe that the

threshold γ− and thereby Sfs are determined on the basis of the periodic orbit O.

6.4 NUMERICAL EXAMPLE

This section presents a numerical example for the motion planning algorithm de-veloped in Section 6.3. The physical parameters of the monopedal robot in Fig. 6.1are given in Table 6.1.3 We assume that the inertia tensor in the torso frame canbe given by I4 = diag{I4,xx, I4,yy, I4,zz}. In the optimization problem, it is assumedthat the body angles ϕ1, ϕ2, and ϕ3 lie in the intervals [−10◦, 10◦], [−80◦, 80◦],and [1◦, 120◦], respectively. Also, the orientations θ1, θ2, and θ3 lie in [−10◦, 10◦],[0◦, 60◦], and [−30◦, 30◦], respectively. Moreover, suppose that T ∗

s , T ∗f ∈ [0.1, 1](s)

and α1∗i , β1∗

j ∈ [−1, 1]3 for i = 2, . . . , Ms − 2, j = 2, . . . , and Mf − 2. The static

friction coefficient between the end of the leg and the ground is equal to μ = 23 .

Furthermore, according to the actuation limits and considering the gear reduction atthe body joints, we choose umax = 300 (Nm) and the maximum absolute value of thejoint angular velocities as 20 (rad/s). A two-stage strategy is used to solve the motionplanning algorithm. In the first stage, the cost function is chosen as 1 and by using thefmincon function of MATLAB’s Optimization Toolbox, we search for a feasibleperiodic solution of the open-loop hybrid model of equation (6.11) that will be usedin the next stage as an initial guess. By using the fmincon function, the motionplanning algorithm during the second stage is continued to minimize the followingdesired cost function:

I(�∗) := 1

Ls

∫ T ∗

0‖u(t)‖2

2 dt, (6.23)

3 In Table 6.1, d1 and d2 represent the distances between the lumped masses m1 and m2 and the hip andknee joints, respectively.

NUMERICAL EXAMPLE 173

TABLE 6.1 Physical Parameters of the Monopedal Robot

m1(kg) m2(kg) m3(kg) m4(kg) d1(m) d2(m)

1 1 5 0.5 0.25 0.25

L1(m) L2(m) L3(m) I4,xx(kgm2) I4,yy(kgm2) I4,zz(kgm2)

0.5 0.5 0.5 0.2 0.2 0.2

TABLE 6.2 Components of q1−∗s and q1−∗

s

ϕ1(rad) 0.1534 ϕ1( rads ) −0.8932

ϕ2(rad) 0.1142 ϕ2( rads ) −0.9604

ϕ3(rad) 0.3738 ϕ3( rads ) 0.9428

θ1(rad) −0.1169 θ1( rads ) 0.8591

θ2(rad) 0.0427 θ2( rads ) −0.8987

θ3(rad) −0.0404 θ3( rads ) −0.2346

where Ls represents the step length. By choosing Ms = 4 and Mf = 4, a local mini-mum is obtained with the components given in Tables 6.2–6.5. The value of the costfunction evaluated in the optimal point is also equal to I(�∗) = 3.9150 × 103(N2ms).The optimal running motion has a period of 2T ∗ = 2(T ∗

s + T ∗f ) = 2(0.3846 +

0.1884) = 1.1459(s), a step length of Ls = 0.2292(m), and an average running speed

TABLE 6.3 Components of q1−∗f and q1−∗

f

ϕ1(rad) −0.1313 ϕ1( rads ) −2.6961

ϕ2(rad) 1.3450 ϕ2( rads ) −0.2557

ϕ3(rad) 1.4494 ϕ3( rads ) −9.9276

θ1(rad) 0.0808 θ1( rads ) 1.2216

θ2(rad) 0.4886 θ2( rads ) 0.8391

θ3(rad) −0.0192 θ3( rads ) 0.1276

xcm(m) 0.1946 xcm( ms ) 0.4489

ycm(m) −0.0374 ycm( ms ) −0.0282

zcm(m) 0.6779 zcm( ms ) −1.9809

TABLE 6.4 Stance andFlight Phase Times

T ∗s (s) 0.3846

T ∗f (s) 0.1884


TABLE 6.5 Third Columns of the ParameterMatrices α1∗ and β1∗

α1∗′3 0.0247 0.4540 0.9027

β1∗′3 0.1473 −0.0547 1.0000

of 0.4(ms ). In Figs. 6.2 and 6.3, the bold and light curves correspond to stance and

flight phases, respectively. Figure 6.2 depicts the angular position and velocity of therobot during four consecutive steps (i.e., two periods) of the optimal motion, respec-tively. Discontinuities in Fig. 6.2 are due to impacts. Figure 6.3 shows open-loopcontrol inputs and horizontal and vertical components of the ground reaction force atthe end of the leg during four consecutive steps of the optimal motion. Two types ofdiscontinuity due to transitions between continuous phases are shown. On the optimalperiodic orbit, pv

l = 13.8115( ms2 ) > 0 at the beginning of the flight phases and, hence,

the takeoff model is valid.

0 0.5 1 1.5 2−10

−5

0

5

10

u 1 (N

m)

Time (s) 0 0.5 1 1.5 2

−60

−40

−20

0

20

40

u 2 (N

m)

Time (s)

0 0.5 1 1.5 2−100

−50

0

u 3 (N

m)

Time (s) 0 0.5 1 1.5 2

−20

−10

0

10

20

Fx (

N)

Time (s)

0 0.5 1 1.5 2

−5

0

5

Fy (

N)

Time (s) 0 0.5 1 1.5 2

0

100

200

300

400

Fz (

N)

Time (s)

Figure 6.3 Open-loop control inputs and horizontal and vertical components of the groundreaction force at the leg end during four consecutive steps of the optimal motion. Bold andlight curves correspond to stance and flight phases, respectively. Discontinuities are due totransitions between continuous phases. Impulsive ground reaction forces are not presented.(See the color version of this figure in the color plates section.)

WITHIN-STRIDE CONTROLLERS 175

6.5 WITHIN-STRIDE CONTROLLERS

This section presents a design method to obtain time-invariant control laws during thestance and flight phases to realize the desired period-one orbit O as an exponentiallystable orbit. Assume that O is a periodic orbit for the time-invariant closed-loophybrid model of running. On the basis of extended method of Poincare sectionsfor hybrid systems (Theorem 2.2 of Chapter 2), an equivalence can be establishedbetween the stabilization problem of O for the closed-loop hybrid model and that ofthe corresponding equilibrium point for a discrete-time system defined on the basis ofa two-step Poincare return map. The stabilization issue will be studied in Section 6.7.In order to reduce the dimension of the stabilization problem, by extending the ideasdeveloped in Refs. [56, 59], a two-level control action is proposed. At the first level ofcontrol action, within-stride controllers including stance and flight phase controllersare employed to create a family of attractive forward invariant manifolds in each of thecontinuous phases. At the second level, the parameters of within-stride controllers areupdated by event-based update laws during discrete transitions between continuousphases to achieve hybrid invariance and stabilization.

6.5.1 Stance Phase Control Law

Based on Refs. [52, 56], by assuming that the angle γ of the virtual leg is a strictlyincreasing function of time during the stance phases of the desired periodic orbit O,we choose the nominal holonomic output functions

hjs,O : Qs → R

3, j = 1, 2

such that they vanish on the orbits Ojs for j = 1, 2. In our notation, “s” and “j” denote

the jth stance phase. Moreover, “h” and the subscript “O” correspond to the holonomicfunction vanishing on the periodic orbit. To make this notion precise, according to thesymmetry, let �(t), 0 ≤ t ≤ T ∗

s be the time evolution of γ during the first and second

stance phases of the periodic orbitO. Then, we define hjs,O(qs) := ϕ − ϕ

js,d(γ), where

ϕjs,d(γ) := B

(�−1(γ)

T ∗s

, αj∗)

j = 1, 2 (6.24)

is the desired evolution of body anglesϕ onOjs in terms of γ and t = �−1(γ) represents

the inverse of function γ = �(t). We remark that αj∗ and T ∗s are obtained from the

motion planning algorithm in Section 6.3. Since γ is a strictly increasing function oftime on Oj

s , the desired evolutions of ϕ can be expressed as a function of γ insteadof the time. We observe that by zeroing the nominal output function h

js,O(qs), the

evolution of the body angles can be expressed in terms of γ . To define a modifiedholonomic output function for the system during the stance phases, a corrective term


as a Bezier polynomial is also added to hjs,O in the following manner:

yjs

(xjs ; ξ

j)

:= hjs

(qs; ξ

j)

:= hjs,O(qs) +B

(γ − γj+

γ− − γj+ , aj

)

= ϕ −(

ϕjs,d(γ) −B

(γ − γj+

γ− − γj+ , aj

))

=: ϕ − �js,d

(γ; ξj

), j = 1, 2.

(6.25)

The additive term is introduced to create hybrid invariance and stabilization and itvanishes as t → ∞. In equation (6.25), the additive Bezier polynomial is assumedto be degree Ns ≥ 3. In addition, aj := col{aj

i }Ns

i=0 ∈ R3×(Ns+1) represents the coeffi-cients of the additive Bezier polynomial. Note that γ− is the fixed threshold value usedin Remark 6.6 to define the takeoff switching hypersurface Sf

s . Also, γj+, j = 1, 2denotes the initial value of γ during the jth stance phase. The values of γj+ and aj forj = 1, 2 may differ during consecutive steps. Due to this fact, we form the followingvector:

ξj :=(aj′0 , a

j′1 , . . . , a

j′Ns−1, a

j′Ns

, γj+)′

, j = 1, 2 (6.26)

as the parameter vector of the jth stance phase controller. The parameter vector ξj ∈R

3(Ns+1)+1 is held constant during stance phases, that is, ξj = 0, and updated duringtransitions from flight to stance by an event-based control law. In addition, �j

s,d is themodified desired evolution of the body angles ϕ in terms of γ .

By using the input–output linearization [103], it can be formally shown that

yjs

(xjs ; ξ

j) = LgsLfsy

js

(xjs ; ξ

j)

u + L2fs

yjs

(xjs ; ξ

j), j = 1, 2,

where

LgsLfsyjs

(xjs ; ξ

j) = ∂h

js

∂qs

D−1s B ∈ R3×3

L2fs

yjs

(xjs ; ξ

j) = ∂

∂qs

(∂h

js

∂qs

qs

)qs − ∂h

js

∂qs

D−1s (Csqs + Gs) ∈ R3×1.

Let ξj∗ be the nominal value of ξj on the periodic orbit Ojs , that is,

ξj∗ :=[

03(Ns+1)×1

γj+∗

].


If the decoupling matrix LgsLfsyjs (xj

s ; ξj∗) is invertible on the orbit Ojs , there exists

an open neighborhood Nj of Ojs × ξj∗ such that for every (xj

s , ξj) ∈ Nj(Oj

s × ξj∗),the feedback law

u(xjs ; ξ

j) = − (

LgsLfsyjs

(xjs ; ξ

j))−1

(L2

fsyjs

(xjs ; ξ

j)− vj

s

(yjs , y

js

))(6.27)

is well defined and results in yjs = v

js (y

js , y

js ). We will assume that the continuous laws

vjs (y

js , y

js ) are designed such that the origin of the closed-loop system y

js = v

js (y

js , y

js )

is globally finite-time stable for j = 1 and globally exponentially quickly stable forj = 2. To achieve this result, the methods developed in Refs. [46, 93] can be used forthe design of v1

s , and v2s is also defined as follows:

v2s

(y2s , y

2s

):= −1

εKD y2

s − 1

ε2 KP y2s ,

where KP, KD ∈ R3×3 are positive definite diagonal matrices and ε is a suf-ficiently small positive scalar. For the later purposes, define Zj

s,ξj as the

parametric stance phase zero dynamics manifold associated with the output yjs ,

Zj

s,ξj := {xjs ∈ Xs

∣∣yjs

(xjs ; ξ

j) = 03×1, Lfsy

js

(xjs ; ξ

j) = 03×1

}.

Furthermore, let

zjs = f j

zero,s

(zjs ; ξ

j)

denote the corresponding parametric stance phase zero dynamics, where fjzero,s is the

restriction of the stance phase closed-loop vector field to the zero dynamics manifoldZj

s,ξj . The following lemma presents proper local coordinates for the zero dynamicsmanifold and a closed-form expression for the stance phase zero dynamics.

Lemma 6.2 (Stance Phase Zero Dynamics) Let θ := (θ1, γ, θ3)′ and q := (ϕ′, θ′)′.Then,

q = T (q) := T0 q + T1.

Moreover, define the mass-inertia matrix in the new coordinates as

D(q) := (T−10 )′ Ds(T

−1(q)) T−10

and the conjugate momenta as

σs := (σs,1, σs,2, σs,3)′ := ∂Ls

∂ ˙θ

′= E2 D(q) ˙q,


where E2 := [03×3 I3×3]. Then, in the local coordinates (θ, σs) for the manifoldZj

s,ξj , the parametric stance phase zero dynamics can be given by

˙θ =(Dθθ(q) + Dθϕ(q)

∂�js,d

∂γ(γ; ξj) e′

2

)−1

σs

σs,1 = 1

2σ′

s λj′(θ; ξj)

∂D∂θ1

(q) λj(θ; ξj) σs + mtot g0 sin(θ3) xcm(q)

− mtot g0 cos(θ3) ycm(q)

σs,2 = 1

2σ′

s λj′(θ; ξj)

∂D∂γ

(q) λj(θ; ξj) σs + mtot g0 cos(θ3) cos(θ1) xcm(q)

+ mtot g0 sin(θ3) cos(θ1) ycm(q)

σs,3 = 0,

where at the right-hand side q can be expressed in terms of θ (i.e., q = q(θ)), e2 :=[0 1 0]′ and

λj(θ; ξj) :=[

∂∂qs

hjs (T−1(q(θ)); ξj) T−1

0

E2 D(q(θ))

]−1 [03×3

I3×3

].

The proof is given in Appendix C.2.

Remark 6.7 (Invertibility of the Decoupling Matrix on O) Appendix C.3 showsthat LgsLfsy

js (xj

s ; ξj∗) is invertible on the orbit Ojs if and only if the scalar function

κj(θ) := 1 + e′2 D−1

θθDθϕ

∂ϕjs,d

∂γ

is nonzero on Ojs .

6.5.2 Flight Phase Control Law

Analogous to the development for the stance phase, assume that xcm is a strictlyincreasing function of time on the orbits Oj

f , j = 1, 2, and, by considering symme-try, denote its time evolution by Xcm(t), T ∗

s ≤ t ≤ T ∗. Then, we define the nominalholonomic output functions

hjf,O : Qf → R

3, j = 1, 2


such that they vanish on the the orbits Ojf . In our notation, “f ” and “j” represent the

jth flight phase. The nominal holonomic function can be expressed as hjf,O(qf ) :=

ϕ − ϕjf,d(xcm), in which

ϕjf,d(xcm) := B

(X−1

cm(xcm) − T ∗s

T ∗f

, βj∗)

, j = 1, 2 (6.28)

denotes the desired evolution of body angles ϕ in terms of xcm and t = X−1cm(xcm)

represents the inverse of function xcm = Xcm(t). We remark that T ∗s , T ∗

f , and βj∗are obtained from the motion planning algorithm in Section 6.3. Zeroing the outputfunction (6.28) forces the desired evolution of the body angles to be constrained toxcm. By adding a corrective Bezier polynomial to the output function (6.28) for hybridinvariance and stabilization, define the following modified holonomic output for thesystem during flight phases:

yjf (xj

f ; ψj) := hjf (qf ; ψj)

:= hjf,O(qf ) +B

(xcm − x

j+cm

xj+cmT

jf

, bj

)

= ϕ −(

ϕjf,d(xcm) −B

(xcm − x

j+cm

xj+cmT

jf

, bj

))

=: ϕ − �jf,d(xcm; ψj), j = 1, 2,

(6.29)

where, Nf ≥ 1 and bj := col{bji }

Nf

i=0 ∈ R3×(Nf +1) are the degree and coefficients of

the additive Bezier polynomial, respectively. Moreover, xj+cm and x

j+cm represent the

first components of the position and velocity of the COM at the beginning of the jthflight phase. T

jf is also an estimate of the flight phase time duration. Consequently,

the parameter vector of the jth flight phase controller can be defined as

ψj :=(bj′0 , b

j′1 , . . . , b

j′Nf −1, b

j′Nf

, xj+cm, xj+

cm, Tjf

)′, j = 1, 2. (6.30)

In addition, �jf,d is the modified desired evolution of the body angles ϕ in terms of

xcm. The parameter vector ψj ∈ R3(Nf +1)+3 is held constant during flight phases,that is, ψj = 0, and updated during transitions from stance to flight by an event-basedcontrol law. Using the input–output linearization, it can be shown that

yjf = Lff

Lgfy

jf

(xjf ; ψj

)u + L2

ffyf

(xjf ; ψj

), j = 1, 2, (6.31)


where

LgfLff

yjf

(xjf ; ψj

)= ∂h

jf

∂qf

D−1f Bf ∈ R3×3

L2ff

yjf

(xjf ; ψj

)= ∂

∂qf

(∂h

jf

∂qf

qf

)qf − ∂h

jf

∂qf

D−1f (Cf qf + Gf ) ∈ R3×1.

(6.32)

The decoupling matrix, associated with the output yjf , is invertible on Xf ×

R3(Nf +1)+3 because

LgfLff

yjf

(xjf ; ψj

)= ∂h

jf

∂qf

D−1f Bf = E1 A−1

and A is positive definite, where E1 := [I3×3 03×3]. Thus, the feedback law

u

(x

j

f ; ψj)

= −(Lgf

Lffy

j

f

(x

j

f ; ψj))−1 (

L2ff

yj

f

(x

j

f ; ψj)

− vj

f

(y

j

f , yj

f

))(6.33)

is well defined for every (xjf , ψj) ∈ Xf × R3(Nf +1)+3 and results in y

jf = v

jf (yj

f , yjf ).

Furthermore, by choosing

vjf

(y

jf , y

jf

):= −1

εKD y

jf − 1

ε2 KP yjf , j = 1, 2,

the origin is globally exponentially quickly stable for the closed-loop system yjf =

vjf (yj

f , yjf ). The parametric flight phase zero dynamics manifold associated with the

output yjf is defined as follows:

Zj

f,ψj :={

xjf ∈ Xf

∣∣yjf

(xjf ; ψj

)= 03×1, Lff

yjf

(xjf ; ψj

)= 03×1

}.

The corresponding parametric flight phase zero dynamics can also be expressed as

zjf = f

jzero,f

(zjf ; ψj

),

in which fjzero,f is the restriction of the flight phase closed-loop vector field to the

zero dynamics manifold Zj

f,ψj .

Lemma 6.3 (Flight Phase Zero Dynamics) Define the conjugate momenta

σf := (σf,1, σf,2, σf,3)′ := ∂Lf

∂θ

′= E2 A(q) q,

EVENT-BASED UPDATE LAWS FOR HYBRID INVARIANCE 181

where E2 := [03×3 I3×3]. Then, in the global coordinates (θ, pcm, σf , pcm) for the

manifold Zj

f,ψj , the parametric flight phase zero dynamics can be given by

θ = A−1θθ (q) σf − A−1

θθ (q) Aθϕ(q)∂�

jf,d

∂xcm(xcm; ψj) xcm

σf,1 = 1

2

[σ′

f xcm]μj′

(θ, xcm; ψj)∂A

∂θ1(q) μj(θ, xcm; ψj)

[σf

xcm

]

σf,2 = 1

2

[σ′

f xcm]μj′

(θ, xcm; ψj)∂A

∂θ2(q) μj(θ, xcm; ψj)

[σf

xcm

]

σf,3 = 0

mtot pcm = mtot

⎡⎢⎣

0

0

−g0

⎤⎥⎦ ,

where q can be expressed as a function of θ and xcm (i.e., q = q(θ, xcm)) and

μj(θ, xcm; ψj) :=

⎡⎢⎣ 03×3

∂�j

f,d

∂xcm(xcm; ψj)

A−1θθ (q) −A−1

θθ (q) Aθϕ(q)∂�

j

f,d

∂xcm(xcm; ψj)

⎤⎥⎦ .

The proof is similar to that presented for Lemma 6.2.

6.6 EVENT-BASED UPDATE LAWS FOR HYBRID INVARIANCE

In order to render stance and flight phase zero dynamics manifolds hybrid invariant forthe closed-loop hybrid model of running, this section presents a policy for takeoff andimpact update laws. These update laws will result in a reduced-order hybrid modelfor which stabilization will be studied in Section 6.7.

Definition 6.2 (Regular Parameter Vector of the Stance Phase) The parametervector of the jth stance phase controller ξj, j = 1, 2 is said to be regular if

1. there exists an open neighborhood Vj of Ojs such that for every x

js ∈

Vj(Ojs ) ∩ Zj

s,ξj , the decoupling matrix LgsLfsyjs (xj

s ; ξj) is invertible,

2. γj+ /= γ− (see equation (6.25)), and

3. ajNs−1 = a

jNs

= 03×1.


Definition 6.3 (Regular Parameter Vector of the Flight Phase) The param-eter vector of the jth flight phase controller ψj, j = 1, 2 is said to be regular ifxj+cm , T

jf /= 0 (see equation (6.29)).

Next, let ξj and ψj for j = 1, 2 be regular vector of parameters. Then, by the con-tinuous within-stride feedback laws developed in Section 6.3, Zj

s,ξj and Zj

f,ψj are for-ward invariant under the stance and flight phases closed-loop dynamics, respectively.It can be shown that for a regular vector of parameters ξj , Zj

s,ξj is a six-dimensionalembedded submanifold of Xs. This fact in combination with the definition of theswitching hypersurface Sf

s as a level set of the virtual leg angle γ , that is,

Sfs = {xs ∈ Xs|γ(qs) − γ− = 0}

implies that Sfs ∩ Zj

s,ξj is a five-dimensional submanifold of Xs. In addition, by Re-

mark 3.16, ajNs−1 = a

jNs

= 03×1 implies that

B(1, aj) = ajNs

= 03×1

∂

∂sB(1, aj) = Ns

(ajNs

− ajNs−1

)= 03×1,

and consequently Sfs ∩ Zj

s,ξj is independent of the coefficient matrix of the additive

Bezier polynomial during the jth stance phase (i.e., aj) and, for simplicity, this com-mon intersection is denoted by Sf

s ∩ Zjs . It can also be easily shown that for every

regular ψj , Zj

f,ψj is a 12-dimensional embedded submanifold of Xf .According to the order stance1 → flight1 → stance2 → flight2 → stance1, in

which the phases are executed, we will denote the event-based update laws by the4-tuple

(π1→1

s→f , π1→2f→s , π2→2

s→f , π2→1f→s

).

Let xjs = f

jcl,s(x

js ; ξj) and x

jf = f

jcl,f (xj

f ; ψj) be the closed-loop dynamics of the jthstance and flight phases, respectively, where j ∈ {1, 2}. For every initial conditionxjs,0 ∈ Xs and parameter vector ξj ∈ R3(Ns+1)+1, the flow of the jth stance phase

is denoted by Fjs (x

js,0; ξj) and defined as the solution of the initial-value problem

xjs = f

jcl,s(x

js ; ξj), xj

s (0) = xjs,0 evaluated at the takeoff time. In an analogous manner,

for every initial condition xjf,0 ∈ Xf and parameter vector ψj ∈ R3(Nf +1)+3, the flow

of the jth flight phase is denoted by Fjf (xj

f,0; ψj) and defined as the solution of

xjf = f

jcl,f (xj

f ; ψj), xjf (0) = x

jf,0 evaluated at the impact time.


Next, we present the following definition.

Definition 6.4 (Hybrid Invariance) Let Fji for i ∈ {s, f } and j ∈ {1, 2} represent

the flow of the jth closed-loop phase of type i. Define the family of the zero dynamicsmanifolds for the first stance phase as

Z1s :=

{Z1

s,ξ1 : ξ1 is regular}

. (6.34)

Z1s is said to be hybrid invariant for the closed-loop hybrid model of running under the

4-tuple event-based update law (π1→1s→f , π1→2

f→s, π2→2s→f , π2→1

f→s) if there exists an open

neighborhoodN of x1−∗s such that for every x1−

s ∈ N(x1−∗s ) ∩ Sf

s ∩ Z1s , the following

update sequence (see Fig. 6.4):

ψ1 := π1→1s→f

(x1−s

)x1−f := F1

f

(�f

s

(x1−s

); ψ1

)ξ2 := π1→2

f→s

(x1−f

)x2−s := F2

s

(�s

f

(x1−f

); ξ2

)ψ2 := π2→2

s→f

(x2−s

)x2−f := F2

f

(�f

s

(x2−s

); ψ2

)ξ1 := π2→1

f→s

(x2−f

)

Figure 6.4 Geometric description of hybrid invariance. The plot depicts that under the 4-tuple event-based update law (π1→1

s→f , π1→2f→s, π

2→2s→f , π2→1

f→s), the family of the zero dynamics man-ifolds for the first stance phase Z1

s is hybrid invariant, that is, �(x1−s ; ψ1, ξ2, ψ2) ∈ Z1

s,ξ1 , where

�(x1−s ; ψ1, ξ2, ψ2) := �s

f (x2−f ) is the two-step reset map. In addition, a

j

Ns−1 = aj

Ns= 03×1 re-

sults in the common intersection Sfs ∩ Zj

s . Plot also illustrates the five-dimensional restrictedPoincare return map P(x1−

s ; ξ1S, ψ

1S, ξ

2S, ψ

2S) and the HZD. (See the color version of this figure

in the color plates section.)


results in (i) regular ξj, ψj for j = 1, 2, and (ii) two-step reset invariance, that is,

�(x1−s ; ψ1, ξ2, ψ2

)∈ Z1

s,ξ1 , (6.35)

where

� :(Sf

s ∩ Z1s

)× R3(Nf +1)+3 × R3(Ns+1)+1 × R3(Nf +1)+3 → Xs

is the two-step reset map given by

�(x1−s ; ψ1, ξ2, ψ2

):= �s

f

(x2−f

). (6.36)

Figure 6.4 represents a geometric description for hybrid invariance. In other words,hybrid invariance means that for every final state at the first stance phase of the currentstep x1−

s ∈ Sfs ∩ Z1

s , the initial state at the first stance phase of two-step ahead belongsto the family Z1

s . By definition, the static event-based laws

ψ1 = π1→1s→f

(x1−s

), ψ2 = π2→2

s→f

(x2−s

)are called takeoff update laws and the static event-based laws

ξ2 = π1→2f→s

(x1−f

), ξ1 = π2→1

f→s

(x2−f

)are called impact update laws.

6.6.1 Takeoff Update Laws

According to the the takeoff map �fs , at the end of the jth stance phase, calculate the

initial state of the jth flight phase as xj+f = �

fs (xj−

s ). On the basis of xj+f , obtain ϕ+,

ϕ+, x+cm, and x+

cm and then, update the parameters of the modified output function inequation (6.29) as

xj+cm = x+

cm , xj+cm = x+

cm.

The other components of the vector of parameters ψj are updated by the followingpolicy:

Tjf = T ∗

f

bj0 = −ϕ+ + ϕ

jf,d

(xj+

cm

)bj1 = b

j0 − T

jf

Nf

(ϕ+ − ∂ϕ

jf,d

∂xcm

(xj+

cm

)xj+

cm

).

(6.37)


By Remark 3.16, equation (6.37) results in

yjf

(xj+f ; ψj

)= 03×1

Lffy

jf

(xj+f ; ψj

)= 03×1,

and consequently xj+f = �

fs (xj−

s ) ∈ Zj

f,ψj (see Fig. 6.4), where ψj := πj→js→f (xj−

s ).

Therefore, the vector of parameters ψj can be split into two vectors ψjHI ∈ R9 and

ψjS ∈ R3(Nf −1) that are employed for hybrid invariance and stabilization, respectively.

In particular,

ψjHI :=

(bj′0 , b

j′1 , xj+

cm, xj+cm, T

jf

)′

ψjS :=

(bj′2 , . . . , b

j′Nf

)′.

The update policy for ψjS will be presented in Section 6.7.

6.6.2 Impact Update Laws

By considering the impact map �sf and using the state of the system at the end of kth

flight phase, calculate the state of the mechanical system at the beginning of the jthstance phase as x

j+s = �s

f (xk−f ), where for k = 1, j(k) = 2 and for k = 2, j(k) = 1.

On the basis of xj+s , calculate ϕ+, ϕ+, γ+, and γ+, and update the parameters of the

modified output function in equation (6.25) as γj+ = γ+ and

aj0 = −ϕ+ + ϕ

js,d(γj+)

aj1 = a

j0 − γ− − γj+

Nsγ+

(ϕ+ − ∂ϕ

js,d

∂γ(γj+) γ+

)

ajNs−1 = 03×1

ajNs

= 03×1.

(6.38)

By Remark 3.16, equation (6.38) results in

yjs

(xj+s ; ξj

) = 03×1

Lfsyjs

(xj+s ; ξj

) = 03×1,

and consequently xj+s = �s

f (xk−f ) ∈ Zj

s,ξj (see Fig. 6.4), where ξj := πk→jf→s(x

k−f ).

Analogous to the development for the takeoff update law, the vector of parameters


ξj can also be split into ξjHI ∈ R13 and ξ

jS ∈ R3(Ns−3) according to hybrid invariance

and stabilization, respectively. In particular,

ξjHI :=

(aj′0 , a

j′1 , a

j′Ns−1, a

j′Ns

, γj+)′

ξjS :=

(aj′2 , . . . , a

j′Ns−2

)′.

The update policy for ξjS will be presented in Section 6.7. The following lemma

presents the main result of this section.

Lemma 6.4 (Hybrid Invariance) Assume that the stabilizing update laws forψ

jS and ξ

jS, j = 1, 2 are continuous functions with respect to x1−

s and vanishon the periodic orbit O. Then, the family of the zero dynamics manifolds Z1

s

in equation (6.34) is hybrid invariant under the 4-tuple event-based update law(π1→1

s→f , π1→2f→s, π

2→2s→f , π2→1

f→s) for which ψjHI and ξ

jHI are given in equations (6.37)

and (6.38), respectively.

Proof. Continuity of the stabilizing update laws ψjS(x1−

s ) and ξjS(x1−

s ) for j = 1, 2and the fact that on the periodic orbitO, aj = 03×(Ns+1), bj = 03×(Nf +1) (the additiveBezier polynomials of the modified outputs in equations (6.25) and (6.29) are zeroon O), κj(θ) /= 0 for j = 1, 2 (the decoupling matrices corresponding to the outputs(6.25) on the stance phases of the periodic orbit are invertible as stated in Remark 6.7),γ+∗ /= γ−, γ+∗ /= 0, x+∗

cm /= 0, and T ∗f /= 0 imply that there exists an open neighbor-

hoodN of x1−∗s such that for every x1−

s ∈ N(x1−∗s ) ∩ Sf

s ∩ Z1s the update laws of ψ

jHI

and ξjHI in equations (6.37) and (6.38) are well defined and result in hybrid invariance,

that is, �sf (x2−

f ) ∈ Z1s . �


In this section, static event-based update laws for stabilizing parameters ψjS ∈

R3(Nf −1) and ξ

jS ∈ R3(Ns−3), j = 1, 2 in terms of x1−

s are presented such that theperiodic orbit O is an exponentially stable limit cycle for the closed-loop hybridmodel of running. Under the assumptions of Lemma 6.4, by applying the 4-tupleevent-based update law (π1→1

s→f , π1→2f→s, π

2→2s→f , π2→1

f→s), the family of the manifolds Z1s

is hybrid invariant. In addition, as mentioned previously, ajNs−1 = a

jNs

= 03×1 results

in the common intersection Sfs ∩ Zj

s . For simplicity, the switching map � can bedenoted by

�(x1−s ; ψ1

S, ξ2S, ψ2

S

)


(see Fig. 6.4). Thus, to study the stabilization problem, we consider the followingreduced-order system with impulse effects:

�s,cl :

{z1s = f 1

zero,s

(z1s ; ξ1

S

)z1−s /∈ Sf

s ∩ Z1s

z1+s = �

(z1−s ; ψ1

S, ξ2S, ψ2

S

)z1−s ∈ Sf

s ∩ Z1s ,

(6.39)

which is called HZD. From Lemma 6.2,

z1s := (θ1, γ, θ3, σs,1, σs,2, σs,3)′

is a valid coordinates transformation for the parametric zero dynamics manifoldZ1s,ξ1 .

In addition by considering the fact that on Sfs , γ = γ−, then

z1−s := (θ1, θ3, σs,1, σs,2, σs,3)′

is a valid coordinates transformation for Sfs ∩ Z1

s,ξ1 = Sfs ∩ Z1

s . By defining the five-dimensional Poincare return map for the HZD in equation (6.39) as

P :(Sf

s ∩ Z1s

)× R3(Ns−3) × R3(Nf −1) × R3(Ns−3) × R3(Nf −1) → Sf

s ∩ Z1s

by

P(z1−s ; ξ1

S, ψ1S, ξ2

S, ψ2S

):= F1

s

(�(z1−

s ; ψ1S, ξ2

S, ψ2S); ξ1

S

), (6.40)

the following discrete-time system with the state space Sfs ∩ Z1

s can be introduced tostudy the stabilization problem

z1−s [k + 1] = P

(z1−s [k]; ξ1

S[k], ψ1S[k], ξ2

S[k], ψ2S[k]

), k = 1, 2, . . . . (6.41)

Figure 6.4 presents a geometric description for the Poincare return map and stabiliza-tion problem.

Theorem 6.1 (Exponential Stability) Consider the open-loop hybrid model ofrunning (6.11) by the within-stride controllers (6.27) and (6.33), and the event-basedupdate laws with ψ

jHI and ξ

jHI given in equations (6.37) and (6.38). Let

Aol := ∂P∂z1−

s

(z1−s ; ξ1

S, ψ1S, ξ2

S, ψ2S

) ∣∣∣z1−s =z1−∗

s ,ξj

S=ξ

j∗S

,ψj

S=ψ

j∗S

Bjs := ∂P

∂ξjS

(z1−s ; ξ1

S, ψ1S, ξ2

S, ψ2S

) ∣∣∣z1−s =z1−∗

s ,ξj

S=ξ

j∗S

,ψj

S=ψ

j∗S

Bjf := ∂P

∂ψjS

(z1−s ; ξ1

S, ψ1S, ξ2

S, ψ2S

) ∣∣∣z1−s =z1−∗

s ,ξj

S=ξ

j∗S

,ψj

S=ψ

j∗S

(6.42)


and

Bol :=[B1

s B1f B2

s B2f

],

where z1−∗s is the intersection of the orbit O1

s and Sfs , and ξ

j∗S := 03(Ns−3)×1 and

ψj∗S := 03(Nf −1)×1 for j = 1, 2 are the nominal stabilizing parameters. Then, if

(Aol, Bol) is controllable, there exists the gain matrix

K :=[K1′

s K1′f K2′

s K2′f

]′

such that by using the static update laws

ξjS = −Kj

s

(z1−s − z1−∗

s

)

ψjS = −K

jf

(z1−s − z1−∗

s

)

for j = 1, 2, the periodic orbit O is exponentially stable for the closed-loop hybridmodel of running.

Proof. Since (i) the event-based update laws ψjHI and ξ

jHI for j = 1, 2 are continu-

ously differentiable (i.e., C1) and (ii) vjf , j = 1, 2 and v2

s are C1, �(·; ψ1S, ξ2

S, ψ2S) is

C1, which, in combination with the fact that the continuous law v1s [46, 93] vanishes

on Z1s,ξ1 , follows that P(·, ; ξ1

S, ψ1S, ξ2

S, ψ2S) is C1. Moreover, similar to the proof of

Lemma C.6 of Ref. [18, p. 450], it can be shown that P is also C1 with respect to(ξ1

S, ψ1S, ξ2

S, ψ2S) in an open neighborhood of (ξ1∗

S , ψ1∗S , ξ2∗

S , ψ2∗S ). Hence, Aol and Bol

are well defined. In addition, controllability of (Aol, Bol) implies the existence of thegain matrix K such that |eig(Aol − BolK)| < 1, where eig(·) denotes the eigenvalues,which, in turn, implies that z1−∗

s is locally exponentially stable for the closed-loopsystem z1−

s [k + 1] = Pcl(z1−s [k]), where

Pcl

(z1−s

):= P

(z1−s ; ξ1

S

(z1−s

), ψ1

S

(z1−s

), ξ2

S

(z1−s

), ψ2

S

(z1−s

)).

Finally, applying the extended method of Poincare sections for hybrid systems (The-orems 2.2 and 2.5, Chapter 2) completes the proof. �

SIMULATION RESULTS 189

6.8 SIMULATION RESULTS

This section presents a numerical example for the control scheme proposed in thischapter to exponentially stabilize the desired periodic orbitO generated by the motionplanning algorithm in Section 6.4. The additive Bezier polynomials are chosen withdegree Ns = Nf = 9. The state matrix of the linearized open-loop restricted Poincarereturn map can be obtained numerically as follows:4

Aol =

⎡⎢⎢⎢⎢⎢⎢⎣

13.1446 −3.2632 1.4352 −1.7912 1.8410

−2.2285 −0.9782 0.0489 −0.0864 0.6449

262.4331 −62.9310 26.4986 −31.6193 29.5938

−0.5343 −1.3510 0.7338 −1.4694 2.5789

5.0767 −2.1732 1.0688 −1.4461 2.3939

⎤⎥⎥⎥⎥⎥⎥⎦

,

which has the eigenvalues eig(Aol) = {40.8797, −0.8783 ±0.0570i, 0.4726, −0.0061}. We remark that without stabilizing parameters, theperiodic orbit O is not stable in the sense of Lyapunov for the closed-loop hybridmodel of running. Due to space limitations, the matrices B

js , B

jf for j = 1, 2 are

not presented here. By using the DLQR design method, the matrix gain of thestatic stabilizing update laws (i.e., K

js , K

jf for j = 1, 2) can be calculated. In this

method, the gain matrix K is obtained such that by the static stabilizing feedbacklaw δu = −Kδz1−

s , where δz1−s := z1−

s − z1−∗s , the cost function

1

2

∞∑k=0

{δz1−′

s [k] Q δz1−s [k] + δu′[k]R δu[k]

}

subject to the linearized system

δz1−s [k + 1] = Aol δz

1−s [k] + Bol δu[k]

is minimized, where Q = Q′ 0 and R = R′ � 0. For Q = diag{qi}5i=1 and R =

diag{ri}84i=1, where qi = 1, i = 1, . . . , 5, and ri = 10, i = 1, . . . , 84, the state matrix

4 For a given smooth function f : Rn → Rn, the element (i, j) of the Jacobian matrix evaluated at x∗ can

be calculated numerically as follows:

Df(i,j)(x∗) ∼= fi(x∗1 ,...,x∗

j+δxj ,...,x

∗n)−fi(x∗

1 ,...,x∗j−δxj ,...,x

∗n)

2δxj,

where δxj is a scalar perturbation. In this chapter, the perturbations used for calculation of Aol and Bol

are assumed to be 10−6 and 10−5, respectively.


−0.2 −0.1 0 0.1 0.2

−2

0

2

φ1 (rad)

d/dt

φ 1 (ra

d/s)

0 0.5 1 1.5−10

0

10

20

φ2 (rad)

d/dt

φ 2 (ra

d/s)

0.5 1 1.5

−10

0

10

φ3 (rad)

d/dt

φ 3 (ra

d/s)

−0.1 −0.05 0 0.05 0.1 0.15−2

−1

0

1

2

θ1 (rad)

d/dt

θ 1 (ra

d/s)

0 0.2 0.4 0.6−4

−2

0

2

4

6

θ2 (rad)

d/dt

θ 2 (ra

d/s)

−0.06 −0.04 −0.02 0 0.02 0.04−0.5

0

0.5

θ3 (rad)

d/dt

θ 3 (ra

d/s)

Figure 6.5 Phase portraits of the state trajectories during 40 consecutive steps of running.The stance and flight phases are shown by bold and light curves, respectively. In the figure, theeffect of the impact with the ground is illustrated by jumps in the velocity. The convergence tothe desired limit cycle O can be seen.

of the linearized closed-loop Poincare return map is obtained as follows5:

Acl =

⎡⎢⎢⎢⎢⎢⎢⎣

−0.5044 0.0333 0.0107 −0.0449 0.1458

−2.3334 0.0639 −0.0302 −0.0647 0.4142

0.2268 −0.1153 −0.0474 0.1162 −0.2056

−3.1307 −0.0024 −0.0190 −0.1131 0.6323

−3.6323 0.0440 0.0660 −0.1370 0.9969

⎤⎥⎥⎥⎥⎥⎥⎦

with the eigenvalues eig(Acl) = {0.4059, 0.0816 ± 0.1240i, −0.1735, 0.0004}. To il-lustrate the convergence to the desired periodic orbit O, the simulation of the closed-loop hybrid model of running is started at the end of the stance phase with an initialcondition off of this trajectory. Figures 6.5 and 6.6 depict the results of the closed-loopsimulation. In these figures, the stance and flight phases are shown by bold and light

5 Acl = Aol − BolK.

SIMULATION RESULTS 191

0 5 10 15 20−10

−5

0

5

10

15

Time (s)

u 1 (N

m)

0 5 10 15 20−60

−40

−20

0

20

40

Time (s)

u 2 (N

m)

0 5 10 15 20−150

−100

−50

0

Time (s)

u 3 (N

m)

0 5 10 15 20−30

−20

−10

0

10

20

Time (s)F

x (N

)

0 5 10 15 20

−5

0

5

Time (s)

Fy (

N)

0 5 10 15 200

100

200

300

400

500

Time (s)

Fz (

N)

Figure 6.6 Closed-loop control inputs and horizontal and vertical components of the groundreaction force at the leg end during 40 consecutive steps of the monopedal robot. Bold and lightcurves correspond to stance and flight phases, respectively. Discontinuities are due to discretetransitions between continuous phases. Impulsive ground reaction forces are not presented.

curves, respectively. The phase portraits of the state trajectories during 40 consecutivesteps of running are presented in Fig. 6.5 in which the effect of the impact with theground is illustrated by jumps in the velocity. Figure 6.6 shows closed-loop controlinputs and horizontal and vertical components of the ground reaction force during 40consecutive steps of running.

CHAPTER 7

Stabilization of Periodic Orbits forWalking with Passive Knees

7.1 INTRODUCTION

In this chapter, a motion planning algorithm to generate time trajectories of a periodicwalking motion by a five-link, two-actuator planar bipedal robot is presented. In orderto reduce the number of actuated joints for walking on a flat ground and restore thewalking motion in people with disabilities, we assume that the robot has passive pointfeet and unactuated knee joints. In other words, only the hip joints of the robot areassumed to be actuated. The motion planning algorithm is developed in terms of afinite-dimensional nonlinear optimization problem with equality and inequality con-straints. The equality constraints are necessary and sufficient conditions by which theimpulsive model of walking has a period-one orbit, whereas the inequality constraintsare introduced to guarantee (i) the feasibility of the periodic motion and (ii) capabilityof applying the proposed two-level control scheme for stabilization of the orbit. Thisalgorithm is an extension of results developed in the previous chapters.

The main objective of this chapter is to present a time-invariant two-level feed-back law based on the concept of virtual constraints and HZD to exponentially sta-bilize a desired periodic motion generated by the motion planning algorithm. Themechanical system studied in this chapter has three degrees of underactuation dur-ing single support. We present a control methodology for creation of hybrid invari-ant manifolds and stabilization of a desired periodic orbit for the impulsive modelof walking. In particular, for a given integer number M ≥ 2, we introduce M − 1within-stride switching hypersurfaces and thereby split the single support phase intoM within-stride phases. The within-stride switching hypersurfaces are defined aslevel sets of a scalar holonomic quantity that is a strictly increasing function of timeon the desired walking motion. To stabilize the desired orbit, the overall controller ischosen as a two-level feedback law. At the first level, during a within-stride phase,a parameterized holonomic output function is defined for the dynamical system andimposed to be zero by using a continuous-time feedback law. The output function is


193

194 STABILIZATION OF PERIODIC ORBITS FOR WALKING WITH PASSIVE KNEES

expressed as the difference between the actual values of the angle of hip joints and theirdesired evolutions, in terms of the latter increasing holonomic quantity. At the secondlevel, the parameters of continuous-time feedback laws are updated during within-stride transitions by event-based update laws. The purpose of updating the parametersis (i) to achieve hybrid invariance, (ii) continuity of continuous-time feedback lawsduring within-stride transitions, and (iii) stabilization of the desired orbit. From theconstruction procedure of the parameterized output functions and event-based updatelaws, intersections of the corresponding zero dynamics manifolds and within-strideswitching hypersurfaces are independent of the parameters. Consequently, by choos-ing one of these common intersections as the Poincare section, stabilization can beaddressed on the basis of a five-dimensional restricted Poincare return map.

7.2 OPEN-LOOP MODEL OF WALKING

7.2.1 Mechanical Model of the Planar Bipedal Robot

In this chapter, a five-link, two-actuator planar bipedal mechanism with point feet isstudied (see Fig. 7.1). The mechanical system consists of a torso link and two identicallegs with tibia and femur links. The links are rigid and have mass. They are connectedthrough a set of body joints including two actuated revolute hip joints (q1 and q2) andtwo unactuated revolute knee joints (q3 and q4). The body joints are relative anglesdetermining the shape of the robot and denoted by qb := (q1, q2, q3, q4)′, where thesubscript “b” represents the body joints and prime denotes matrix transpose. Theabsolute orientation of the torso link with respect to an inertial world frame is givenby q5. Assume that the control inputs applied at the hip joints are represented by u :=(u1, u2)′ ∈ U, where U is a simply connected and open subset of R2 containing theorigin u = 02×1. It is assumed that bipedal walking can be modeled by a hybrid system

3q

θ

4q

1q2q 5q

3d

1d

2d

Figure 7.1 A five-link, two-actuator planar bipedal mechanism with point feet during singlesupport. The virtual leg is depicted by the dashed line connecting the stance leg end and thehip joints.

OPEN-LOOP MODEL OF WALKING 195

with a continuous single support phase (one leg on the ground) and an instantaneousdouble support phase (two legs on the ground). During single support, the contactingleg is called the stance leg and the other is called the swing leg. The virtual leg isdefined by a line connecting the stance leg end and the hip joint. The angle of thevirtual leg with respect to the world frame is given by θ, as shown in Fig. 7.1. Animpact occurs when the swing leg end contacts the ground. We assume that duringthe impact, the swing leg neither slip nor rebound. The instantaneous double supportphase is modeled using a coordinate relabeling to swap the role of the legs immediatelyafter impact.

7.2.2 Dynamics of the Single Support Phase

During single support, the mechanical system has five DOF and three degrees ofunderactuation. Let q := (q′

b, q5)′ ∈ Q denote the generalized coordinates vector ofthe mechanical system, where Q is the configuration space. The evolution of themechanical system during single support can be expressed by the following secondorder equation:

D(qb) q + C(qb, q) q + G(q) = B u, (7.1)

in which D is a (5 × 5) mass-inertia matrix, C is a (5 × 5) matrix containing theCoriolis and centrifugal terms, G is a (5 × 1) gravity vector, and

B :=[I2×2

03×2

]

is the input matrix. By introducing the state vector x := (q′, q′)′ ∈ X, equation (7.1)can be expressed as x = f (x) + g(x)u, where X is the single support phase statemanifold taken as

X := TQ := {x = (q′, q′)′|q ∈ Q, q ∈ R5}.Remark 7.1 (Validity of Single Support) The single support phase model is valid ifthe ground reaction force at the stance leg end F1 := (Fh

1 , Fv1 )′ satisfies the unilateral

constraints (i) Fv1 > 0 and (ii) |Fh

1Fv

1| < μs, where μs represents the coefficient of static

friction between the leg end and the ground.

7.2.3 Impact Map

Define the impact switching hypersurface as

S := {x = (q′, q′)′ ∈ X|pv2(q) = 0},

in which pv2 denotes the vertical Cartesian position of the swing leg end with respect

to the ground. Following the results of Ref. [18, p. 75], the impact can be modeled by


the discrete transition x+ = �(x−), in which x− = (q−′, q−′

)′ and x+ = (q+′, q+′

)′represent the state of the mechanical system immediately before and after the impact,respectively. Moreover, � : S → X is the impact map given by

�(x−) :=[

R q−

R �(q−) q−

],

where R is a nonsingular matrix to swap the role of the legs.

Remark 7.2 (Validity of the Impact Model) The impact model is valid if (1) theprevious stance leg end lifts from the ground without interaction immediately after theimpact, that is, pv

2(q+, q+) := ∂p2∂q

(q+) q+ ≥ 0 and (2) the intensity of the impulsive

ground reaction force δF := (δFh, δFv) ∈ R2 satisfies the unilateral constraints (i)

δFv > 0 and (ii) | δFh

δFv | < μs.

7.2.4 Open-Loop Impulsive Model of Walking

By assembling the single support phase model and impact map, the overall open-loopmodel of walking can be expressed as the impulsive system�(X,S, �, f, g) defined as

� :

{x = f (x) + g(x) u x− /∈ S

x+ = �(x−) x− ∈ S.(7.2)

We shall assume that for a given initial condition x0 ∈ X\S (at time t0 ∈ R) and theopen-loop control input u ∈ U, the single support phase solution at time t ≥ t0 isrepresented by ϕ(t; t0, x0, u). Moreover, as in Ref. [46], the time to impact functionTI : R× X × U → R is defined by

TI (t0, x0, u) := inf{t > t0|ϕ(t; t0, x0, u) ∈ S}.

Definition 7.1 (Feasible Periodic Orbit) Assume that there exist (t0, x−∗) ∈R× S and the open-loop control u∗ ∈ U such that (i) �(x−∗) ∈ X\S, (ii) T ∗

I :=TI (t0, �(x−∗), u∗) < ∞, and (iii) ϕ(T ∗

I ; t0, �(x−∗), u∗) = x−∗. Then, the set

O := {x = ϕ(t; t0, �(x−∗), u∗)|t0 ≤ t < T ∗I }

is a period-one orbit for the open-loop impulsive system given in equation (7.2).1

Moreover, the orbit O is said to be feasible if

1. the constraints due to the joint angles and velocities are satisfied on O;

1 From Definition 2.2 of Chapter 2, it is assumed that the solutions of the impulsive system �(X,S, �, f, g)are right continuous.


2. the open-loop control input is feasible in the sense that ‖u∗(t)‖L∞ < umax,where umax is a positive scalar;

3. the ground reaction force experienced at the end of leg-1 satisfies the unilateralconstraints of Remark 7.1;

4. the impact model satisfies the unilateral constraints of Remark 7.2;

5. for every t ∈ (t0, T ∗I ), pv

2(t) > 0. In addition, the transversality condition issatisfied, that is, the swing leg end contacts the ground with nonzero verticalvelocity, pv

2(T ∗I ) < 0;

6. on the orbit O, θ is a strictly increasing function of time, that is,

mint0≤t≤T ∗

I

θ(t) > 0.

7.3 MOTION PLANNING ALGORITHM

In this section, a motion planning algorithm to generate a feasible period-one orbitO for the open-loop impulsive system of equation (7.2) is presented. The algorithmis developed in terms of a finite-dimensional nonlinear optimization problem withequality and inequality constraints. To describe the motion planning algorithm, wefirst introduce the generalized coordinates vector

q :=[qa

qu

]:= T q,

in which qa := (q1, q2)′, qu := (q3, q4, θ)′ and the subscripts “a” and “u” denote theactuated and unactuated components, respectively. The matrix T is also defined as

T :=[H0

�0

]∈ R5×5,

where H0 := [I4×4 04×1] and2 �0 := [1 0 − 12 0 − 1]. In coordinates (q, ˙q), the

equation of motions during single support can be expressed as

D(qb) ¨q + H(q, ˙q) = B u, (7.3)

where

D(qb) := (T′)−1 D(qb) T−1

H(q, ˙q) := (T′)−1(C(qb, T−1 ˙q) T−1 ˙q + G(T−1q)

).

2 In this chapter, it is assumed that the lengths of the tibia and femur links are identical. Thus, θ can begiven by θ = �0 q = q1 + q3

2 − q5.


Next, assume that

x−∗ :=[q−∗

˙q−∗

]

is the state of the mechanical system immediately before the impact on the orbit O.We remark that

q−∗ :=[q−∗a

q−∗u

], ˙q−∗ :=

[q−∗a

q−∗u

].

According to the impact map, we obtain the state of the mechanical system at thebeginning of the orbit O (i.e., x+∗ := (q+∗′

, ˙q+∗′)′) as follows:

q+∗ :=[q+∗a

q+∗u

]= T R T−1q−∗

˙q+∗ :=[q+∗a

q+∗u

]= T R �(T−1q−∗) T−1 ˙q−∗.

(7.4)

Since the hip joints are independently actuated, we shall assume that the evolutionof the hip joints on the orbit O can be described by a Bezier polynomial of time.For this purpose, let N ≥ 3 and w∗ := col{w∗

i }Ni=0 ∈ R2×(N+1) denote the degree andcoefficient matrix of the Bezier polynomial on O, respectively. In particular, let

q∗a(t) := B

(t

T ∗I

, w∗)

, 0 ≤ t ≤ T ∗I ,

where q∗a(t), 0 ≤ t ≤ T ∗

I represents the evolution of the hip joints on the single supportphase of O. By properties of the Bezier polynomials in Remark 3.16, the coefficientmatrix w∗ can be easily adjusted such that the states (qa, qa) are transferred from theinitial conditions (q+∗

a , q+∗a ) at time t = 0 to the final conditions (q−∗

a , q−∗a ) at time

t = T ∗I . In particular, let

w∗0 = q+∗

a w∗N = q−∗

a

w∗1 = w∗

0 + T ∗I

Nq+∗

a w∗N−1 = w∗

N − T ∗I

Nq−∗

a .

In addition, from the last three rows of equation (7.3), the evolution of q∗u(t), 0 ≤ t ≤

T ∗I can be described by the following ordinary differential equation (ODE):

q∗u = −D−1

uu

(Dua q∗

a + Hu

), 0 ≤ t ≤ T ∗

I

q∗u(0) = q+∗

u

q∗u(0) = q+∗

u ,

(7.5)


in which Duu and Dua are the (3 × 3) and (3 × 2) lower right and left submatricesof D, respectively. In addition, Hu consists of the last three rows of H. Thus, for agiven N ≥ 3, the evolution of the mechanical system on the orbitO can be completelydetermined by the following vector of parameters:

ξ∗ := (q−∗′a , q−∗′

u , q−∗′a , q−∗′

u , T ∗I , w∗′

2 , . . . , w∗′N−2)′ ∈ R2N+5.

Now we are able to present the following motion planning algorithm.

Algorithm 7.1 Motion Planning Algorithm

For a given N ≥ 3, the motion planning algorithm is expressed as a nonlinearminimization problem in the finite-dimensional parameter space ⊂ R2N+5 withthe following constraints.

Equality Constraints: The equality constraints are defined as Ce(ξ∗) = 07×1 byCe(ξ∗) := (C1′

e (ξ∗), C2′e (ξ∗), C3

e (ξ∗))′, in which

C1e (ξ∗) := q∗

u(T ∗I ; ξ∗) − q−∗

u

C2e (ξ∗) := q∗

u(T ∗I ; ξ∗) − q−∗

u

C3e (ξ∗) := pv

2(T ∗I ; ξ∗).

To emphasize the dependence of solutions on the vector of parameters ξ∗ in this latterset of equations, we make use of the notations q∗

u(.; ξ∗) and pv2(.; ξ∗). In addition, the

equality constraints C1e (ξ∗) = 03×1 and C2

e (ξ∗) = 03×1 are necessary and sufficientconditions for the existence of a period-one orbit O for the impulsive system � andthe constraint C3

e (ξ∗) = 0 implies that x−∗ ∈ S, where x−∗ := (q−∗′, q−∗′

)′, q−∗ :=T−1q−∗, and q−∗ := T−1 ˙q−∗.

Inequality Constraints: To guarantee the feasibility of O, we can define the vectorCie(ξ∗) := (C1

ie(ξ∗), . . . , Cp

ie(ξ∗))′ for some positive integer p such that the inequal-

ity constraints Cjie(ξ

∗) ≤ 0 for j = 1, . . . , p − 1 satisfy items 1–6 of Definition 7.1.Moreover, the last component of the vector Cie(ξ∗) is introduced such that Cp

ie(ξ∗) ≤ 0

implies that

κ(t; ξ∗) := 1 + e′3 D−1

uu (q∗b(t; ξ∗))Dua(q∗

b(t; ξ∗))q∗

a(t; ξ∗)

θ∗(t; ξ∗)/= 0, ∀t ∈ [0, T ∗

I ],(7.6)

where e′3 := [0 0 1]. In Section 7.5, it will be shown that the condition of

equation (7.6) referred to as invertibility of the decoupling matrix on the orbit O isa necessary condition by which, our control methodology for stabilization of O canbe applied.


Cost Function: The cost function during the motion planning algorithm is definedas

J(ξ∗) := 1

Ls(ξ∗)

∫ T ∗I

0‖u∗(t; ξ∗)‖2

2 dt,

where Ls(ξ∗) represents the step length. To solve the minimization problem by usingthe fmincon function of the MATLAB’s Optimization Toolbox, we employ a two-stage strategy. In the first stage, the cost function is chosen as 1 and we search for afeasible period-one solution of the impulsive system of equation (7.2). To simplifythe search process, the components of the equality and inequality constraints can beadded in a step-by-step fashion. The solution of the first stage will be used as an initialguess to minimize the cost function J(ξ∗) in the second stage.

7.4 NUMERICAL EXAMPLE

In this section, a numerical example for the proposed motion planning algorithm ispresented. The physical parameters of the biped robot are given in Table 7.1. To obtainan anthropomorphic gait, it is assumed that q1, q2(deg) ∈ [110◦, 200◦], q3, q4(deg) ∈[1◦, 90◦], q5(deg) ∈ [45◦, 90◦], and θ(deg) ∈ [45◦, 135◦]. Moreover, due to actua-tion limits, we assume that umax = 100(Nm) and the maximum absolute value ofangular velocities at the hip joints is 10( rad

s ). Also, suppose that T ∗I (s) ∈ [0.1, 1]

and w∗i ∈ [−5, 5]2 for i = 2, . . . , N − 2. For N = 5, a local optimal solution for the

motion planning algorithm is obtained. Tables 7.2 and 7.3 show the components ofξ∗. At this point, the optimal motion of the robot has a period of T ∗

I = 0.8586(s),a step length of Ls = 0.3857(m), and an average walking speed of 0.4492(m

s ).

TABLE 7.1 Physical Parameters of the Biped Robot

Femur Tibia Torso

Length in m 0.5 0.5 0.5Mass in kg 2 1 4Mass center in m 0.15 0.2 0.25Inertia in kgm2 0.2 0.2 0.5

TABLE 7.2 Components of q−∗a , q−∗

u , q−∗a , and q−∗

u

q−∗1 (rad) 2.8281 q−∗

1 ( rads ) 6.7814

q−∗2 (rad) 2.6048 q−∗

2 ( rads ) 6.7929

q−∗3 (rad) 0.4554 q−∗

3 ( rads ) −1.9315

q−∗4 (rad) 0.1160 q−∗

4 ( rads ) −3.9019

θ−∗(rad) 1.7059 θ−∗( rads ) 1.7035

NUMERICAL EXAMPLE 201

TABLE 7.3 Third andFourth Columns of theCoefficient Matrix w∗

w∗2 w∗

3

1.3161 4.64580.4362 2.6536

The value of the cost function at this point is also equal to J(ξ∗) = 501.7779(N2ms).On the optimal trajectory, the robot will not slip for a coefficient of friction greaterthan 0.45. A stick animation of the biped robot taking one step of the optimal motion isdepicted in Fig. 7.2. Angular positions of the mechanical system during three steps ofthe optimal motion are presented in Fig. 7.3. The discontinuities are due to coordinaterelabeling for swapping the role of the legs. Figure 7.4 shows the angular velocitiesof the robot during three steps of walking. The discontinuities are due to impacts andcoordinate relabeling. The open-loop control inputs, and the horizonal and verticalcomponents of the ground reaction force at the stance leg end are depicted in Fig. 7.5.Moreover, the absolute ratio of the horizontal component to the vertical componentand the path of the swing leg end in the sagittal plane are presented in Fig. 7.5.

−0.5 −0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.4 0.50

0.5

1

1.5

x (m)

y (m

)

Figure 7.2 Stick animation of the bipedal robot during one step of the optimal motion. (Seethe color version of this figure in the color plates section.)


0 0.5 1 1.5 2 2.52.5

2.6

2.7

2.8

2.9

q1 (

rad)

Time (s)

0 0.5 1 1.5 2 2.51.5

2

2.5

3

3.5

q2 (

rad)

Time (s)

0 0.5 1 1.5 2 2.50

0.1

0.2

0.3

0.4

q3 (

rad)

Time (s)

0 0.5 1 1.5 2 2.50

0.5

1

1.5

2

q4 (

rad)

Time (s)

0 0.5 1 1.5 2 2.5

1.2

1.4

1.6

q5 (

rad)

Time (s)

0 0.5 1 1.5 2 2.51.3

1.4

1.5

1.6

1.7

θ (

rad)

Time (s)

Figure 7.3 Angular positions during three steps of the optimal motion. The discontinuitiesare due to coordinate relabeling for swapping the role of the legs.

7.5 CONTINUOUS-TIMES CONTROLLERS

To exponentially stabilize a desired period-one orbit O generated by the motion plan-ning algorithm for the impulsive model of walking, this section presents a time-invariant control scheme that is applied at two levels. In this approach, for a giveninteger number M ≥ 2, the state space of the system is split into M subspaces by defin-ing M − 1 within-stride switching hypersurfaces. To reduce the dimension of the sta-bilization problem at the first level of the control scheme, parameterized continuous-time controllers are employed to create finite-time attractive and forward invariantmanifolds in the corresponding internal phase. The event-based controllers, whichare applied at the second level, update the parameters of the continuous-time con-trollers during the transitions among the internal phases. This section presents thecontinuous-time controllers. The event-based update laws will be treated in Sections7.6 and 7.7.

Assume that θ+∗ and θ−∗ are the initial and final values of θ on the orbit O,respectively. From Definition 7.1, we remark that θ+∗ < θ−∗. Let M ≥ 2 be an in-teger number and θ+∗ < θ∗

1 < θ∗2 < · · · < θ∗

M−1 < θ−∗ be a partition of the interval

CONTINUOUS-TIMES CONTROLLERS 203

0 0.5 1 1.5 2 2.5−2

0

2

4

6

8

d/d

tq1 (

rad/

s)

Time (s)

0 0.5 1 1.5 2 2.5−5

0

5

10

d/d

tq2 (

rad/

s)

Time (s)

0 0.5 1 1.5 2 2.5

−2

0

2

d/d

tq3 (

rad/

s)

Time (s)

0 0.5 1 1.5 2 2.5−5

0

5

d/d

tq4 (

rad/

s)

Time (s)

0 0.5 1 1.5 2 2.5−2

0

2

4

6

d/d

tq5 (

rad/

s)

Time (s)

0 0.5 1 1.5 2 2.50

0.5

1

1.5

2

d/d

tθ (

rad/

s)

Time (s)

Figure 7.4 Angular velocities during three steps of the optimal motion. The discontinuitiesare due to impacts and coordinate relabeling for swapping the role of the legs.

[θ+∗, θ−∗]. During a step, split the single support phase into M internal phases. De-note the index of these phases by j that takes values in the discrete set {1, . . . , M}.Next introduce the following switching hypersurfaces among the internal phases:

S21 := {x = (q′, q′)′ ∈ X|θ = θ∗

1}...

SMM−1 := {x = (q′, q′)′ ∈ X|θ = θ∗

M−1}S1

M := S.

(7.7)

For simplicity, we define the index of next phase function as i : {1, . . . , M} →{1, . . . , M} by

i(j) := j + 1, j = 1, . . . , M − 1

i(M) := 1.


0 0.5 1 1.5 2 2.5

0

20

40

u1 (

Nm

)

Time (s)

0 0.5 1 1.5 2 2.5

−20

−10

0

u2 (

Nm

)

Time (s)

0 0.5 1 1.5 2 2.5−20

0

20

40

60

F1h (

N)

Time (s)

0 0.5 1 1.5 2 2.50

50

100

150

F1v (

N)

Time (s)

0 0.5 1 1.5 2 2.50

0.1

0.2

0.3

0.4

|F1h /F

1v |

Time (s)

−0.4 −0.2 0 0.2 0.40

0.1

0.2

0.3

0.4

p 2v (m

)

p2h (m)

Figure 7.5 Plots of the open-loop control inputs (two top graphs) during three steps of theoptimal motion. Two middle graphs represent the horizontal and vertical components of theground reaction force experienced at the end of leg-1 during three steps. The bottom graphsdepict the absolute ratio of the horizontal component to the vertical component of the groundreaction force (during three steps) and the path of the swing leg end in the sagittal plane (duringone step).

In our notation, Si(j)j represents the switching hypersurface from the jth phase

into the i(j)th phase. Subsequently, the transitions j → i(j) for j = 1, . . . , M − 1are called within-stride transitions, whereas the transition M → 1 is called the

impact transition. The switching maps �i(j)j : Si(j)

j → X are also defined as

�i(j)j (x) := Id

Si(j)j

(x), j = 1, . . . , M − 1

�1M(x) := �(x),

in which IdSi(j)

j

denotes the identity map. In other words, the position and velocity

are assumed to be continuous during the within-stride transitions. The switching


hypersurfaces in equation (7.7) motivate us to study the stabilization problem of thedesired orbit O for the hybrid system �H(�1, . . . , �M) with the following form:

�1 :

{x1 = f (x1) + g(x1) u x1− /∈ S2

1

x2+ = �21(x1−) x1− ∈ S2

1

...

�M :

{xM = f (xM) + g(xM) u xM− /∈ S1

M

x1+ = �1M(xM−) xM− ∈ S1

M.

(7.8)

In this latter set of equations, xj, j = 1, . . . , M denotes the state of the mechanicalsystem during the jth internal phase. By the construction procedure of �H, O is alsoa period-one orbit for the hybrid system �H and it can be expressed as O = ∪M

j=1Oj ,

where Oi ∩ Oj = φ for every i /= j ∈ {1, . . . , M}. Moreover, since θ is a strictlyincreasing function of time on O (see item 6 of Definition 7.1), the intersections

Oj ∩ Si(j)j , j = 1, . . . , M are singletons, in which Oj

denotes the closure of Oj . In

particular, let {xj−∗} := Oj ∩ Si(j)j for j = 1, . . . , M. We observe that xM−∗ = x−∗.

Next, assume that hd,O : R→ R2 is at least a C2 function such that the

nominal holonomic output function hO : Q → R2 by

hO(q) := qa − hd,O(θ)

vanishes on the single support phase of the periodic orbit O. In particular, hd,O repre-sents the desired evolution of the hip joints onO in terms of θ. Now for a given M ≥ 2and the sequence {θ∗

j }M−1j=1 with the property θ+∗ < θ∗

1 < θ∗2 < · · · < θ∗

M−1 < θ−∗,

let ψj ∈ �j, j = 1, . . . , M represent the parameter vector of the jth phase controllerduring a step. In addition,�j denotes the corresponding parameter space. By adding anaugmentation function as a Bezier polynomial to the nominal holonomic output func-tion hO, we define the parameterized holonomic output function yj : X × �j → R

2

for the jth phase as follows:

yj(xj; ψj) := hj(q; ψj)

:= qa − hd,O(θ) +B(sj; αj)

= qa − (hd,O(θ) −B(sj; αj)

)=: qa − h

jd(θ; ψj).

(7.9)

In equation (7.9), the superscript “j” stands for the jth phase. It is also assumedthat the additive Bezier polynomial is of the degree n ≥ 5 and the coefficient matrix


αj := col{αji }ni=0 ∈ R2×(n+1). Furthermore, the arguments of the Bezier polynomial

are defined as

sj(θ) :=

⎧⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎩

(θ − θ+)/(θ∗1 − θ+), j = 1

(θ − θ∗j−1)/(θ∗

j − θ∗j−1), j = 2, . . . , M − 1

(θ − θ∗M−1)/(θ− − θ∗

M−1), j = M,

(7.10)

where θ+ and θ− are the parameters of the first and Mth phase controller to be deter-mined in Section 7.6. Consequently, the parameter vector of the jth phase controllercan be expressed as

ψj :=

⎧⎪⎪⎪⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎪⎪⎪⎩

(α1′

0 , α1′1 , . . . , α1′

n , θ+)′

, j = 1

(α

j′0 , α

j′1 , . . . , α

j′n

)′, j = 2, . . . , M − 1

(αM′

0 , αM′1 , . . . , αM′

n , θ−)′

, j = M.

By applying the input–output linearization [103] during the jth phase, it can be shownthat

yj(xj; ψj) = LgLf yj(xj; ψj) u + L2f yj(xj; ψj),

where

LgLf yj(xj; ψj) := ∂hj

∂q(q; ψj) D−1(qb) B

L2f yj(xj; ψj) := ∂

∂q

(∂hj

∂q(q; ψj)q

)q − ∂hj

∂q(q; ψj) D−1(qb) (C(qb, q)q + G(q)) .

For the later purposes, LgLf yj(xj; ψj) is called the decoupling matrix. The followinglemma studies the invertibility of the decoupling matrices on the orbit O.

Lemma 7.1 (Invertibility of the Decoupling Matrices on O) Assume that Ois a feasible period-one orbit for the impulsive system of equation (7.2) gen-erated by the motion planning algorithm. Let ψj∗, j = 1, . . . , M represent thenominal parameter vector of the single support phase controller during the jthphase, that is, ψj∗ := 02(n+1)×1 for j = 2, . . . , M − 1, ψ1∗ := (01×2(n+1), θ

+∗)′and ψM∗ := (01×2(n+1), θ

−∗)′. Then, the invertibility of the decoupling matrices


LgLf yj(xj; ψj∗), j = 1, . . . , M on the orbit O is equivalent to condition ofequation (7.6).

Proof. Since D−1 = TD−1T′ and (T′)−1B = B, the decoupling matrix can berewritten as follows:

LgLf hj(q; ψj) = ∂hj

∂qD−1 B = ∂hj

∂q

∂q

∂qD−1 B

=[I2×2 02×2 − ∂h

j

d

∂θ

]D−1 B.

By defining

� := D−1 =[�aa �au

�′au �uu

]

and also considering B = [I2×2 02×3]′, the decoupling matrix can be expressed as

LgLf hj(q; ψj) = �aa − ∂hjd

∂θe′

3 �′au,

together with

�aa =(Daa − Dau D−1

uu D′au

)−1

�au = −�aa Dau D−1uu

results in

LgLf hj(q; ψj) =(

I2×2 + ∂hjd

∂θe′

3 D−1uu Dua

)(Daa − Dau D−1

uu D′au

)−1.

Consequently,3

det(LgLf hj(q; ψj)) = 1 + e′3 D−1

uu Dua∂h

j

d

∂θ

det(Daa − Dau D−1uu D′

au).

3 Note that the matrix D is positive definite. In addition, we make use of the identity

det(In×n + A B) = det(Im×m + B A)

for every A ∈ Rn×m and B ∈ Rm×n.


Finally, the fact that on the periodic orbit O, ψj = ψj∗ and thereby∂h

j

d

∂θ= q∗

a

θ∗ , com-pletes the proof. �

If the decoupling matrix LgLf yj(xj; ψj∗) is invertible on the orbit Oj ,there exists an open neighborhood Nj(Oj × ψj∗) ⊂ X × �j such that for every(xj; ψj) ∈ Nj , the control input

ujcl(x

j; ψj) := − (LgLf yj(xj; ψj))−1(L2

f yj(xj; ψj) − vj(yj, yj))

(7.11)

is well defined and results in the closed-loop dynamics yj = vj(yj, yj). Assumethat vj : R2 × R2 → R

2 is a continuous function such that the origin for theclosed-loop dynamics yj = vj(yj, yj) is globally finite-time stable. For this pur-pose, the approaches of Refs. [46, 93] can be applied. Next we introduce theparameterized zero dynamics manifold of the jth phase as follows:

Zj

ψj := {xj ∈ X|yj(xj; ψj) = 02×1, Lf yj(xj; ψj) = 02×1}.

It can be shown that Njx ∩ Zj

ψj is a six-dimensional embedded submanifold of X,

where Njx is the projection of Nj onto X. Moreover, from the definition of switching

hypersurfacesSi(j)j in equation (7.7),Nj

x ∩ Zj

ψj ∩ Si(j)j for j = 1, . . . , M − 1 is a five-

dimensional embedded submanifold of X. We also assume that this is true for j = M.The following lemma presents a valid coordinates transformation for the manifoldsNj

x ∩ Zj

ψj , j = 1, . . . , M.

Lemma 7.2 (Zero Dynamics) Define the following conjugate momenta vector

σu := ∂L∂qu

′= Dua qa + Duu qu,

in which L represents the Lagrangian of the single support phase. Then, (qu, σu) isa valid local coordinates transformation for Nj

x ∩ Zj

ψj , j = 1, . . . , M. Moreover, inthese coordinates, the zero dynamics of the jth phase is given by

qu =⎛⎝I3×3 − D−1

uu Dua∂h

j

d

∂θe′

3

1 + e′3 D−1

uu Dua∂h

j

d

∂θ

⎞⎠D−1

uu σu

σu = Hu.

(7.12)

Proof. Since (i) the distribution generated by the columns of the matrix g (i.e.,span{g1, g2}) is involutive and (ii) Lgiσu = 0 for i = 1, 2 (this is a consequenceof unactuation of qu), by Ref. [103, p. 222], (qu, σu) is a valid coordinates transfor-mation on Nj

x ∩ Zj

ψj for j = 1, . . . , M. In addition, since the components of qu are

EVENT-BASED CONTROLLERS 209

unactuated, the Euler–Lagrange equation immediately implies that σu = Hu. Finally,on the manifold Nj

x ∩ Zj

ψj ,

qu =(Duu + Dua

∂hjd

∂θe′

3

)−1

σu

together with the Matrix Inversion Lemma4 completes the proof. �

Remark 7.3 (Valid Coordinates for Njx ∩ Zj

ψj ∩ Si(j)j ) From the definition of the

within-stride switching hypersurfaces Si(j)j in equation (7.7), it can be concluded that

(q3, q4, σu) is a valid local coordinates transformation for Njx ∩ Zj

ψj ∩ Si(j)j , j =

1, . . . , M − 1.

7.6 EVENT-BASED CONTROLLERS

During the transition j → i(j), j = 1, . . . , M, the event-based controller πi(j)j up-

dates the parameters of the i(j)th phase continuous-time controller. The parametervector ψi(j) remains constant during the i(j)th phase, that is, ψi(j) = 0. The purpose ofupdating the parameters in an event-based manner is (i) to achieve hybrid invariance,(ii) continuity of the continuous-time controllers during the within-stride transitions,and (iii) exponential stabilization of the orbit O for the system �H.

7.6.1 Hybrid Invariance

By applying the continuous-time controllers ujcl, j = 1, . . . , M, zero dynamics man-

ifolds Zj

ψj are forward invariant.

Definition 7.2 (Hybrid Invariance) Under the event-based update laws(π2

1, π32, . . . , π

1M), the manifolds {Zj

ψj }Mj=1 are said to be hybrid invariant for the

hybrid system �H if there exist open neighborhoods Vj of xj−∗ such that for every

xj− ∈ Vj(xj−∗) ∩ Si(j)j ∩ Zj

ψj and j = 1, . . . , M,

�i(j)j (xj−) ∈ Zi(j)

ψi(j),

where ψi(j) := πi(j)j (xj−).

4 The Matrix Inversion Lemma states that for every A ∈ Rn×m and B ∈ Rm×n, if the matrix (In×n + AB)is invertible, then

(In×n + AB)−1 = In×n − A(Im×m + BA)−1B.


Lemma 7.3 (Hybrid Invariance) Let ZO be the zero dynamics manifold corre-sponding to the nominal holonomic output function hO, that is,

ZO :={

x = (q′, q′)′ ∈ X|hO(q) = 02×1,∂hO∂q

(q) q = 02×1

}.

Assume that the event-based update laws (π21, π

32, . . . , π

1M) are such that for every

j = 2, . . . , M − 1,

αj0 = α

j1 = α

jn−1 = αj

n = 02×1 (7.13)

and

α1n−1 = α1

n = 02×1

αM0 = αM

1 = 02×1.(7.14)

Then, the following statements are true.

(a) The intersections Si(j)j ∩ Zj

ψj for j = 1, . . . , M − 1 are independent of ψj . In

addition, these common intersections are equal to Si(j)j ∩ ZO.

(b) The sets �i(j)j (Si(j)

j ∩ Zj

ψj ) for j = 1, . . . , M − 1 are independent of ψj and

ψi(j), and equal to �i(j)j (Si(j)

j ∩ ZO). Also, for every xj− ∈ Si(j)j ∩ Zj

ψj , j =1, . . . , M − 1,

�i(j)j (xj−) ∈ Zi(j)

ψi(j),

where ψi(j) := πi(j)j (xj−).

(c) Let xM− = (q−′, q−′

)′ ∈ S1M represent the state of the mechanical system im-

mediately before the impact. Next define

θ− := θ−∗

q+a := E1 q+ = E1 R q−

q+a := E1 q+ = E1 R �(q−) q−

θ+ := �0 q+ = �0 R q−

θ+ := �0 q+ = �0 R �(q−) q−,

(7.15)

EVENT-BASED CONTROLLERS 211

in which E1 := [I2×2 02×3], and update the parameters α10 and α1

1 as follows:

α10 = −q+

a + hd,O(θ+)

α11 = α1

0 − θ∗1 − θ+

n θ+

(q+

a − ∂hd,O∂θ

(θ+) θ+)

.(7.16)

Then, there exists open neighborhood VM of xM−∗ such that for every xM− ∈VM(xM−∗) ∩ S1

M ∩ ZMψM ,

�1M(xM−) ∈ Z1

ψ1 ,

where ψ1 := π1M(xM−).

Proof. By definition of sj in equation (7.10), at the end of the jth phase, it followsthat

sj = 1, j = 1, . . . , M − 1.

From properties of the Bezier polynomials given in Remark 3.16, this latter fact

together with αjn−1 = α

jn = 02×1 implies part (a). In addition, �

i(j)j (Si(j)

j ∩ Zj

ψj ) is

also independent of ψj . By an analogous reasoning, it can be shown that αj0 = α

j1 =

02×1 for j = 2, . . . , M together with sj = 0 at the beginning of the jth phase, j =2, . . . , M, results in the first statement of part (b). By part (a), this latter fact andcontinuity of the position and velocity during the transition j → i(j), j = 1, . . . ,

M − 1 complete the proof of part (b).Since the orbit O is feasible, from item 6 of Definition 7.1 and continuity of the

impact map �1M : S1

M → X, it can be concluded that there exists an open neighbor-hood VM of xM−∗ such that for every xM− = (q−′

, q−′)′ ∈ S1

M ∩ VM , θ+ defined inequation (7.15) is positive. Thus, α1

1 in equation (7.16) is well defined. Moreover, theupdate laws for α1

0 and α11 immediately imply that

h1(q+; ψ1) = 02×1

∂h1

∂q(q+; ψ1) q+ = 02×1,

which, in turn, completes the proof of part (c). �

The geometry derived from the event-based update laws of Lemma 7.3 andcontinuous-time controllers of Section 7.5 for M = 5 is illustrated in Fig. 7.6.


Figure 7.6 Geometry derived from the event-based update laws of Lemma 7.3 andcontinuous-time controllers of Section 7.5 for M = 5. It is seen that the manifolds {Zj

ψj }5j=1

are hybrid invariant for the closed-loop hybrid system �H under the event-based update laws

(π21, π

32, π

43, π

54, π

15). The intersectionsSi(j)

j ∩ Zj

ψj are independent of ψj and equal toSi(j)j ∩ ZO

for j = 1, 2, 3, 4. This is not true for the case j = 5.

7.6.2 Continuity of the Continuous-Time Controllers During theWithin-Stride Transitions

Assume that all assumptions of Lemma 7.3 hold. Then, the event-based updatelaws (π2

1, π32, . . . , π

1M) given in equations (7.13)–(7.16) result in hybrid invari-

ance. However, we desire that during the within-stride transitions (i.e., j → i(j) forj = 1, . . . , M − 1), the hip torques remain continuous. For this purpose, we presentthe following lemma by which the acceleration is also imposed to be continuous.

Lemma 7.4 (Continuity of the Continuous-Time Controllers) Under the assump-tions of Lemma 7.3 and the event-based update laws (π2

1, π32, . . . , π

1M), as given in

equations (7.13)–(7.16), the additional conditions

αi(j)2 = α

jn−2, j = 1, . . . , M − 1 (7.17)

result in continuity of the continuous-time controllers during the within-stride tran-sitions.

Proof. Since the continuous function vj vanishes on the zero dynamics mani-fold Zj

ψj , j = 1, . . . , M, the restriction of the continuous-time controller ujcl to the


manifold Zj

ψj can be expressed as

ujcl =

(∂hj

∂qD−1 B

)−1 (∂

∂q

(∂hj

∂qq

)q − ∂hj

∂qD−1(Cq + G)

). (7.18)

Equations (7.13) and (7.14) and the fact that the position and velocity remain con-tinuous during the within-stride transition j → i(j), j = 1, . . . , M − 1, imply thecontinuity of the terms ∂hj

∂qD−1B and ∂hj

∂qD−1(Cq + G) during this transition. It can

also be easily shown that

∂

∂q

(∂hj

∂q(q; ψj)q

)q = −∂2h

jd

∂θ2 (θ; ψj)(�0q)2.

Consequently, the following condition

∂2hjd

∂θ2

(θ∗j ; ψj

)= ∂2h

i(j)d

∂θ2

(θ∗j ; ψi(j)

)

for j = 1, . . . , M − 1 results in

ujcl(x

j−; ψj) = ui(j)cl

(�

i(j)j (x−

j ), πi(j)j (x−

j ))

,

which, in turn, together with properties of Bezier polynomials given in Remark 3.16completes the proof. �

Remark 7.4 (Simplification of the Stabilization Problem) To simplify the stabili-zation problem in Section 7.7, we shall assume that

α12 = α2

2 = · · · = αM2 = 02×1

α1n−2 = α2

n−2 = · · · = αM−1n−2 = 02×1.

(7.19)

From Lemma 7.4, equation (7.19) results in the continuity of the control inputs duringthe within-stride transitions. In addition, by equation (7.19), the event-based update

laws πi(j)j , j = 1, . . . , M − 1 can be expressed as a function of xj− and not αj (see

equation (7.17)).


In this section, the event-based update laws proposed by Lemmas 7.3 and 7.4 aremodified such thatO is an exponentially stable orbit for the closed-loop hybrid model


of walking. Our approach for stabilization of O is based on the Poincare sectionsmethod developed for systems with impulse effects given in Chapter 2. However, inthis chapter, the Poincare section is chosen to be one of the within-stride switchinghypersurfaces. To make this notion precise, we first present some definitions. Sincethe closed-loop system is time invariant, the solution of the jth internal phase startingfrom the initial condition xj (at t0 = 0) can be denoted by

ϕcl(t; xj, ψj) := ϕ(t; 0, xj, u

jcl)

at time t ≥ 0. Next for j = 1, . . . , M, define the function T j : X × �j → R by

T j(xj; ψj) := inf{

t > 0|ϕcl(t; xj, ψj) ∈ Si(j)

j

}.

Let the generalized Poincare map of the jth phase, j = 1, . . . , M, given by Pj :

Si(j)j × �i(j) → Si◦i(j)

i(j)be expressed as

Pj(xj−; ψi(j)) := ϕcl

(T i(j)

(�

i(j)j (xj−); ψi(j)

); �i(j)

j (xj−), ψi(j))

.

Next by taking the Poincare section as S21, the Poincare return map can be expressed

as P : S21 × �2 × · · · × �M × �1 by

P := PM ◦ · · · ◦ P1. (7.20)

By considering Lemmas 7.3 and 7.4, we denote the remaining parameters of thejth phase controller by ψ

jrem, where

ψjrem :=

⎧⎪⎪⎪⎨⎪⎪⎪⎩

(α

j′3 , . . . , α

j′n−3

)′, j = 1, . . . , M − 1

(αM′

3 , . . . , αM′n−2, α

M′n−1, α

M′n

)′, j = M.

Slightly abusing the notation, we assume that for a given x1− ∈ S21,

P(x1−; ψs)

represents the parameterized Poincare return map corresponding to x1−, where

ψs :=(ψ2′

rem, . . . , ψM′rem, ψ1′

rem

)′ ∈ R2(n−5)M+6

denotes the stabilizing parameters vector.


The continuous-time controllers developed in Section 7.5 in combination with theevent-based update laws of Lemmas 7.3 and 7.4 reduce the stability analysis of theorbit O for the full-order hybrid model �H to that of the following reduced-orderhybrid model:

�1|Z1ψ1

:

{z1 = f 1

zero

(z1; ψ1

rem

)z1− /∈ S2

1 ∩ Z1ψ1

z2+ = �21(z1−) z1− ∈ S2

1 ∩ Z1ψ1

...

�M |ZM

ψM:

{zM = fM

zero

(zM ; ψM

rem

)zM− /∈ S1

M ∩ ZMψM

z1+ = �1M(zM−) zM− ∈ S1

M ∩ ZMψM ,

(7.21)

which is referred to as the HZD. The Poincare return map for the HZD can be expressedas ρ(z1−; ψs), where

ρ := ρM ◦ · · · ◦ ρ1

and ρj is the restriction of Pj to Zj

ψj , that is, ρj := Pj|Zj

ψj

. Thus, we can consider

the following discrete-time system for stabilization of O:

z1−[k + 1] = ρ(z1−[k]; ψs[k]). (7.22)

In equation (7.22), k ∈ {1, 2, . . .} represents the step number and ψs[k] is consideredas a control input to be updated in a step-by-step manner on the hypersurface S2

1.Moreover, by part (a) of Lemma 7.3, the state space for equation (7.22) is taken asthe five-dimensional manifold S2

1 ∩ ZO that is independent of the control input ψs.Let z1−∗ be the projection of x1−∗ onto S2

1 ∩ ZO. From the construction procedure,z1−∗ is an equilibrium point of equation (7.22) when ψs is replaced by the nominalparameter ψ∗

s := 0(2(n−5)M+6)×1. The following theorem presents the main result ofthis section.

Theorem 7.1 (Exponential Stabilization of the Orbit O) Define the Jacobianmatrices A and B as follows:

A := ∂ρ

∂z1− (z1−; ψs)∣∣∣z1−=z1−∗,ψs=ψ∗

s

∈ R5×5

B := ∂ρ

∂ψs

(z1−; ψs)∣∣∣z1−=z1−∗,ψs=ψ∗

s

∈ R5×(2(n−5)M+6).


If the pair (A, B) is controllable, then there exists the matrix gain

K ∈ R(2(n−5)M+6)×5

such that by using the continuous-time controllers given in equation (7.11) and theevent-based update laws (π2

1, π32, . . . , π

1M) given in equations (7.13)–(7.17) and

ψs[k] = −K (z1−[k] − z1−∗), (7.23)

O is an exponentially stable period-one orbit for the hybrid systems �H and �.

Proof. Linearization of the state equation (7.22) about (z1−∗, ψ∗s ) results in

δz1−[k + 1] = A δz1−[k] + B δψs[k], (7.24)

in which δz1− := z1− − z1−∗ and δψs := ψs − ψ∗s = ψs. Controllability of (A, B)

implies the existence of the matrix gain K such that |eig(Acl)| < 1, where Acl :=A − BK. This latter fact implies that z1−∗ is an exponentially stable equilibriumpoint for the closed-loop discrete-time system z1−[k + 1] = ρcl(z1−[k]), in which

ρcl(z1−) := ρ(z1−, −K(z1− − z1−∗)).

Finally, applying Theorems 2.2 and 2.5 of Chapter 2 completes the proof. �

Remark 7.5 (Event-Based Updating Policy) We observe that unlike the event-based update laws given in equations (7.13)–(7.17), which are updated during thecorresponding within-stride transitions, the stabilizing parameters vector ψs is up-dated only on the switching hypersurface S2

1 in a step-by-step fashion (see equation(7.23)).

Remark 7.6 (Assignment of Stabilizing Parameters to Steps) From equations(7.14) and (7.12), it can be concluded that for a given z1− := (q3, q4, σ

′u)′ ∈ S2

1 ∩ ZO,qa and qa at the end of the first internal phase are expressed as

qa = h1d(θ∗

1 ; ψ1) = hd,O(θ∗1)

qa = ∂h1d

∂θ(θ∗

1 ; ψ1) θ = ∂hd,O∂θ

(θ∗1) θ

= ∂hd,O∂θ

(θ∗1) e′

3

(I3×3 − D−1

uu Dua∂hd,O

∂θe′

3

1 + e′3D−1

uu Dua∂hd,O

∂θ

)D−1

uu σu

=: λ(q3, q4) σu.

SIMULATION OF THE CLOSED-LOOP HYBRID SYSTEM 217

Thus, the state of the mechanical system at the beginning of the second internal phaseis independent of α1. In addition, from equation (7.19), α2

2 = 02×1 and α1n−2 = 02×1

(which result in continuity of control inputs during the within-stride transition 1 → 2)imply that α2 is also independent of α1. Consequently, to calculate the restrictedPoincare return map ρ(z1−; ψs), we do not need to know the parameters that havebeen used by the first internal phase controller to reach the point z1−. We note thatthe components of ψ1

rem will be used during the first internal phase of the next step,

while the components of ψjrem, j = 2, . . . , M are employed during the jth internal

phase of the current step.

7.8 SIMULATION OF THE CLOSED-LOOP HYBRID SYSTEM

In order to confirm the analytical results developed to exponentially stabilize thedesired periodic orbit O generated by the motion planning algorithm, this sec-tion presents a numerical example. In this example, the threshold values θ∗

j , j =1, . . . , M − 1 are generated as follows:

θ∗j = θ+∗ + j

θ−∗ − θ+∗

M.

Furthermore, the gain matrix K in equation (7.23) is obtained by using the DLQRdesign method subject to the linearized system given in equation (7.24). For this goal,we make use of the dlqr function of MATLAB with Q = I5×5 and R = 10Ip×p,where p := 10(n − 5)M + 30. Tables 7.4 and 7.5 present the spectral radius of thematrices A and Acl for M ∈ {2, . . . , 5} and n ∈ {5, . . . , 10}, respectively. From thesetables, it can be concluded that for a given M ∈ {2, . . . , 5}, by increasing the degreeof the Bezier polynomial n, the values of max |eig(A)| and max |eig(Acl)| decreaseand increase, respectively.

In order to present a numerical example confirming the analytical results obtainedin this chapter, the simulation of the closed-loop impulsive model of walking is startedat the end of single support for M = 3 and n = 6. The initial condition of the positionvector is assumed to be q−∗. However, the initial condition for the velocity vector is

TABLE 7.4 max |eig(A)| for M ∈ {2, . . . , 5} and n ∈ {5, . . . , 10}M = 2 M = 3 M = 4 M = 5

n = 5 461.2236 445.7609 441.6970 440.0632n = 6 453.9132 443.5014 440.5908 439.4399n = 7 449.1912 441.9656 439.8695 439.0794n = 8 446.1504 440.8708 439.4388 438.8435n = 9 444.1028 440.2748 439.1323 438.6916n = 10 442.8971 439.8698 438.9407 438.6029


TABLE 7.5 max |eig(Acl)| for M ∈ {2, . . . , 5} and n ∈ {5, . . . , 10}M = 2 M = 3 M = 4 M = 5

n = 5 0.2216 0.2415 0.2470 0.2466n = 6 0.2246 0.2420 0.2474 0.2467n = 7 0.2260 0.2430 0.2474 0.2467n = 8 0.2276 0.2429 0.2474 0.2469n = 9 0.2271 0.2421 0.2476 0.2470n = 10 0.2283 0.2404 0.2477 0.2471

chosen as the value of the velocity vector at the end of single support on O with anerror of +2( deg

s ) on each of its components. Results of the simulation of the closed-loop system are illustrated in Figs. 7.7 and 7.8. Figure 7.7 presents the phase-planeplots of the state trajectories during four consecutive steps. Discontinuities are due

2.6 2.7 2.8 2.9−2

0

2

4

6

q1 (rad)

d/d

tq1 (

rad/

s)

2 2.2 2.4 2.6 2.8 3−5

0

5

q2 (rad)

d/d

tq2 (

rad/

s)

0 0.1 0.2 0.3 0.4 0.5

−2

−1

0

1

2

q3 (rad)

d/d

tq3 (

rad/

s)

0.5 1 1.5−5

0

5

q4 (rad)

d/d

tq4 (

rad/

s)

1.2 1.3 1.4 1.5

0

2

4

q5 (rad)

d/d

tq5 (

rad/

s)

1.3 1.4 1.5 1.6 1.7

0.5

1

1.5

θ (rad)

d/d

tθ (

rad/

s)

Figure 7.7 Phase-plane plots of the state trajectories during four consecutive steps.

SIMULATION OF THE CLOSED-LOOP HYBRID SYSTEM 219

0 1 2 3

0

20

40

u1 (

Nm

)

0 1 2 3

−20

−10

0

u2 (

Nm

) 0 1 2 3

−20

0

20

40

60

F1h (

N)

Time (s)

0 1 2 30

50

100

150

F1v (

N)

Time (s)

Figure 7.8 Plots of the closed-loop control inputs (two top graphs) during four consecutivesteps. Two bottom graphs represent the horizontal and vertical components of the groundreaction force experienced at the end of leg-1 during four steps.

to the impact and coordinate relabeling to swap the role of the legs. The closed-loopcontrol inputs during four consecutive steps are shown in Fig. 7.8. Furthermore, thisfigure represents the horizontal and vertical components of the ground reaction forceexperienced at the stance leg end during four steps.

CHAPTER 8

Continuous-Time Update Laws DuringContinuous Phases of Locomotion

8.1 INTRODUCTION

As studied in the previous chapters, the evolution of a bipedal robot during walkingcan be described by an impulsive system composed of a single support model and animpact map. The main problem in control of legged locomotion is how to design afeedback law that guarantees the existence of a stable limit cycle for the closed-loopsystem. The basic tool for analyzing the existence and stability of limit cycles forsmooth autonomous dynamical systems is the Poincare return map. Grizzle et al.extended the method of the Poincare return map to autonomous systems with impulseeffects [46] and made use of virtual constraints to reduce the dimension of the Poincarereturn map during bipedal walking. In addition, to improve the convergence rate, theidea of updating the parameters of time-invariant stabilizing controllers by event-based update laws has been described in references [57–59]. The contribution of thischapter is to develop a novel method for designing a class of continuous-time updatelaws to update the parameters of stabilizing controllers during continuous phases oflocomotion such that

(i) a general cost function (such as the energy of the control input over singlesupport) can be minimized in an online manner, and

(ii) the exponential stability behavior of the limit cycle for the closed-loop systemis not affected.

In addition, this chapter introduces a class of continuous-time update laws with aradial basis step length to minimize a desired cost function in terms of the controllerparameters and initial states.


221

222 CONTINUOUS-TIME UPDATE LAWS DURING CONTINUOUS PHASES OF LOCOMOTION

8.2 INVARIANCE OF THE EXPONENTIAL STABILITY BEHAVIORFOR A CLASS OF IMPULSIVE SYSTEMS

Let us consider the following autonomous impulsive system

� :

{x = f (x, α∗) x− /∈ S

x+ = �(x−) x− ∈ S,(8.1)

where f and α∗ are a C1 vector field defined on the state space X ⊂ Rn and a nominalparameter vector in the parameter space A ⊂ Rp, respectively. In addition, X is asimply connected open subset ofRn and S is assumed to be a switching hypersurfacetaking the form

S := {x ∈ X|H(x) = 0},

where H : X → R is a smooth function. � : S → X also denotes a C1 reset map.Assume that the impulsive system � has a period-one orbit O ⊂ X that is transversalto S, that is, {x∗} := O ∩ S is a singleton and

Lf H(x∗, α∗) := ∂H

∂x(x∗) f (x∗, α∗) /= 0,

where O denotes the set closure of O. For the later purposes, let T ∗ represent theminimum period of the orbit O. Denote the solution of x = f (x, α∗) with the initialcondition x(0) = x0 by ϕ(t; x0, α

∗) for every t ≥ 0 in its maximal domain of existence.Throughout this chapter, we assume that the periodic orbit O satisfies the followingkey hypothesis.

(H1) The state vector x can be decomposed as

x =[ξ

θ

],

in which ξ ∈ Rn−1 and θ ∈ R is a strictly increasing function of time on O.

Under hypothesis H1, the desired evolution of ξ on the orbit O can be expressed asa function of θ instead of time, that is, ξ = ξd(θ) for all x ∈ O. By assuming thaton O, θ takes values in the closed interval [θmin, θmax] and solutions of � are rightcontinuous (see Definition 2.2, Chapter 2), a description for O can be expressed as

O = {x = (ξ′, θ)′ ∈ X|ξ = ξd(θ), θmin ≤ θ < θmax}.

INVARIANCE OF THE EXPONENTIAL STABILITY BEHAVIOR FOR A CLASS 223

By extrapolating the function ξd outside the interval [θmin, θmax], we define the func-tion da : Xa → R by

da(x, α) := 1

2‖ξ − ξd(θ)‖2

2 + 1

2‖α − α∗‖2

2, (8.2)

where xa := (x′, α′)′ and Xa := X × A denote the augmented state and augmentedstate space, respectively. To present the main result of this chapter, we consider thesystems

�Ia :

⎧⎪⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎪⎩

[x

α

]=

[f (x, α)

g(x, α)

]x− /∈ S

[x+

α+

]=

[�(x−)

π(x−, α−)

]x− ∈ S

(8.3)

�IIa :

⎧⎪⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎪⎩

[x

α

]=

[f (x, α)

0p×1

]x− /∈ S

[x+

α+

]=

[�(x−)

π(x−, α−)

]x− ∈ S,

(8.4)

with the following structural hypotheses.

(H2) It is assumed that α = g(x, α) is a C1 continuous-time update law with aradial basis step length, that is,

g(x, α) := da(x, α) g(x, α),

where g is C1 with respect to xa.

(H3) The function π : Sa → A is a C1 event-based update law with the propertyπ(x∗, α∗) = α∗, where Sa := S × A represents the augmented switchinghypersurface.

For the later purposes, we define

x∗a :=

[x∗

α∗

]∈ Sa.


Hypotheses H2 and H3 imply that the augmented orbit Oa := O × {α∗} is a periodicorbit for the systems �I

a and �IIa with the minimum period T ∗. In addition, by defining

f Ia (xa) :=

[f (x, α)

g(x, α)

], f II

a (xa) :=[f (x, α)

0p×1

],

and also considering the fact that

Sa = {xa ∈ Xa|Ha(xa) = 0},

where Ha(xa) := H(x), it can be concluded that Oa is transversal to Sa.

Theorem 8.1 (Invariance of the Exponential Stability Behavior) Under hypo-theses H1–H3, Oa is exponentially stable for the system �I

a if and only if it isexponentially stable for the system �II

a .

8.3 OUTLINE OF THE PROOF OF THEOREM 8.1

In this section, Theorem 8.1 is proved through a sequence of lemmas. Subsequently,the solution of the augmented system xa = f

ja (xa), j ∈ {I, II} with the initial condi-

tion xa0 := (x′0, α

′0)′ ∈ Xa is denoted by

ϕja(t; x0, α0) :=

[ϕ

jx(t; x0, a0)

ϕjα(t; x0, a0)

],

where the subscripts “x" and “α" represent the components corresponding to x andα, respectively. In addition,

ja(t; x0, α0) :=

⎡⎣

jxx(t; x0, α0)

jxα(t; x0, α0)

jαx(t; x0, α0)

jαα(t; x0, α0)

⎤⎦

denotes the trajectory sensitivity matrix of xja = f

ja (xa), j ∈ {I, II}. Under hypotheses

H2 and H3, for the initial condition (x′0, α

′0)′ = (�′(x∗), α∗′

)′, the solution of xa =f

ja (xa), j ∈ {I, II} is denoted by

ϕ∗a(t) := ϕj

a(t; �(x∗), α∗) :=[ϕ∗(t)

α∗

],

where ϕ∗(t) := ϕ(t; �(x∗), α∗).

OUTLINE OF THE PROOF OF THEOREM 8.1 225

Lemma 8.1 Under the assumptions of Theorem 8.1, the components of the trajectorysensitivity matrices

ja(t; �(x∗), α∗), 0 ≤ t ≤ T ∗ for j ∈ {I, II} satisfy the following

equations:

jxx(t; �(x∗), α∗) = A∗(t) j

xx(t; �(x∗), α∗)

jxx(0; �(x∗), α∗) = In×n

jxα(t; �(x∗), α∗) = A∗(t) j

xα(t; �(x∗), α∗) + B∗(t)

jxα(0; �(x∗), α∗) = 0n×p

jαx(t; �(x∗), α∗) = 0p×n

jαα(t; �(x∗), α∗) = Ip×p,

where

A∗(t) := Dxf (ϕ∗(t), α∗)

B∗(t) := Dαf (ϕ∗(t), α∗).

Proof. From hypothesis H2 and equation (8.2), it can be concluded that

∂g

∂x(ϕ∗(t), α∗) = 0p×n

and

∂g

∂α(ϕ∗(t), α∗) = 0p×p,

which, in turn, together with the variational equation [89, Appendix B] completes theproof. �

Following the results of Ref. [46], the time-to-impact function for the systemxa = f

ja (xa), j ∈ {I, II} is defined as T j : Xa → R by

T j(x0, α0) := inf{t ≥ 0|ϕjx(t; x0, α0) ∈ S}.

Under H2 and H3 and results of Ref. [18, pp. 83–84], T j for j ∈ {I, II} is differentiableat the point (�′(x∗), α∗′

)′. Furthermore, equation (C.25) of reference [18, p. 445]together with Lemma 8.1 and

∂Ha

∂xa

(xa) = [∂H∂x

(x) 01×p

]


yields

D1Tj(�(x∗), α∗) = − 1

Lf H(x∗, α∗)

∂H

∂x(x∗) j

xx(T ∗; �(x∗), α∗)

D2Tj(�(x∗), α∗) = − 1

Lf H(x∗, α∗)

∂H

∂x(x∗) j

xα(T ∗; �(x∗), α∗).(8.5)

By taking Sa as the Poincare section, the augmented Poincare return map for�

ja, j ∈ {I, II} can be expressed as Pj

a : Sa → Sa by

Pja(xa) =

[Pj

x(x, α)

Pjα(x, α)

]:=

[ϕ

jx

(T j(�(x), π(x, α)); �(x), π(x, α)

)ϕ

jα

(T j(�(x), π(x, α)); �(x), π(x, α)

)]

.

Lemma 8.2 Suppose that the assumptions of Theorem 8.1 are satisfied. Then,DPI

a(x∗a) = DPII

a (x∗a).

Proof. Under the assumptions of Theorem 8.1, the Poincare return map Pja, j ∈

{I, II} is differentiable at the point x∗a = (x∗′

, α∗′)′. Furthermore, using the chain

rule, the components of the Jacobian matrix of Pja evaluated at x∗

a can be expressedas

DxPjp(x∗

a) =(D1ϕ

jp(T ∗; �(x∗), α∗) D1T

j(�(x∗), α∗)

+ D2ϕjp(T ∗; �(x∗), α∗)

)D�(x∗)

+(D1ϕ

jp(T ∗; �(x∗), α∗) D2T

j(�(x∗), α∗)

+ D3ϕjp(T ∗; �(x∗), α∗)

)D1π(x∗, α∗)

DαPjp(x∗

a) =(D1ϕ

jp(T ∗; �(x∗), α∗) D2T

j(�(x∗), α∗)

+ D3ϕjp(T ∗; �(x∗), α∗)

)D2π(x∗, α∗),

where p ∈ {x, α}. Since (i) from hypothesis H2,

D1ϕjα(T ∗; �(x∗), α∗) = 0p×1,

and (ii) from Lemma 8.1,

D2ϕjα(T ∗; �(x∗), α∗) = j

αx(T ∗; �(x∗), α∗) = 0p×n

APPLICATION TO LEGGED LOCOMOTION 227

and

D3ϕjα(T ∗; �(x∗), α∗) = j

αα(T ∗; �(x∗), α∗) = Ip×p,

equation (8.5) in combination with straightforward calculations implies that

DxPjx(x∗

a) = Jj(x∗, α∗) + Sj(x∗, α∗) D1π(x∗, α∗)

DαPjx(x∗

a) = Sj(x∗, α∗) D2π(x∗, α∗)

DxPjα(x∗

a) = D1π(x∗, α∗)

DαPjα(x∗

a) = D2π(x∗, α∗),

in which

Jj(x∗, α∗) := � jxx(T ∗; �(x∗), α∗) D�(x∗)

Sj(x∗, α∗) := � jxα(T ∗; �(x∗), α∗)

and

� := In×n − f (x∗, α∗) ∂H∂x

(x∗)

Lf H(x∗, α∗).

Finally, from Lemma 8.1,

Ixx(T ∗; �(x∗), α∗) = II

xx(T ∗; �(x∗), α∗)

Ixα(T ∗; �(x∗), α∗) = II

xα(T ∗; �(x∗), α∗),

which, in turn, results in DPIa(x∗

a) = DPIIa (x∗

a). �

The proof of Theorem 8.1 is an immediate consequence of Lemma 8.2 andTheorems 2.2 and 2.5.

8.4 APPLICATION TO LEGGED LOCOMOTION

To give an application of Theorem 8.1, assume that �(x, α) represents a continuous-time stabilizing controller inducing an exponentially stable periodic walking for abipedal mechanism, where x and α denote the states of the mechanical system and theparameters of the controller, respectively. Moreover, let J(x0, α) represent a generalcost function to be minimized online, in terms of the initial states of the mechanical


system (i.e., x0) and the controller parameters (i.e., α), such as the energy of controller�(x, α) over single support,

J(x0, α) :=∫ T II (x0,α)

0‖�(x, α)‖2

2 dt.

Theorem 8.1 states that, under hypotheses H1–H3, the C1 function g does not affectthe exponential stability behavior of the period orbit Oa for the system �I

a. As aconsequence, in order to minimize the cost function J, we can choose g(x, α) =−γ ∂J

∂α

′(x0, α) or, equivalently,

α = −γ da(x, α)∂J∂α

′(x0, α), (8.6)

where γ > 0 is a scalar representing the continuous-time update gain. We remark thatequation (8.6) introduces a gradient-based update law in the parameter space A forwhich the step length is assumed to change radially.

APPENDIX A

Proofs Associated with Chapter 3

A.1 PROOF OF LEMMA 3.3

Proof. ¨xH can be given by

¨xH = ∂xH

∂qi

qi + q′i

∂2xH

∂q2i

qi.

On the manifold Zd , the dynamics of double support phase is expressed asequation (3.18), in which (u3, u4) is replaced by (u∗

3d, u∗4d). Also from equation (3.30),

(u∗3d, u

∗4d)′ can be expressed as

[u∗

3d

u∗4d

]= −

(∂hd

∂qi

D−1ψ βψ

)−1

(∂

∂qi

(∂hd

∂qi

qi

)qi − ∂hd

∂qi

D−1ψ

(Cψqi + Gψ − ∂�′

∂qi

[u1d

u2d

])), (A.1)

which is a quadratic function with respect to qi. Since the Coriolis matrix Cψ(qi, qi)is linear with respect to qi, equations (3.18) and (A.1) together with

qi = �−1d ([01×2, xH ]′)

qi = λd(qi) vxH

u1 = u1d(qi)

u2 = u2d(qi)

yield equation (3.31). �


229

230 PROOFS ASSOCIATED WITH CHAPTER 3


Proof. The vertical acceleration of the end of leg-2 in the single support phase, y2,can be given by

y2 = ∂y2

∂qs

qs + q′s

∂2y2

∂q2s

qs.

On the manifold Zs, the dynamics of the single support phase can be expressed asDqs + Cqs + G = Bu∗

s , where

u∗s = −

(∂hs

∂qs

D−1 B

)−1 (∂

∂qs

(∂hs

∂qs

qs

)qs − ∂hs

∂qs

D−1(Cqs + G)

)

is a quadratic function with respect to qs. Since

q+s = �s

q,d(q−id) q−

i = �sq,d(q−

id) λd(q−id) v−

xH

and the Coriolis matrix C(qs, qs) is linear with respect to qs, it follows that

y+2 = ω1(x−

H,d) + ω2(x−H,d) z−

xH,

which completes the proof. �


Proof. Equation (3.14) can be decomposed as

H0(D qd(t) + C qd(t) + G) = ud(t) + ∂p′2

∂qb

F2(t)

e′5(D qd(t) + C qd(t) + G) = ∂p′

2

∂q5F2(t).

(A.2)

If rank∂p′

2∂q5

= 1, the last row of matrix equation (A.2) yields

F2(t) =(

∂p′2

∂q5

)+e′

5(D qd(t) + C qd(t) + G) +(

∂p′2

∂q5

)−ϒ(t), (A.3)

where ϒ(t) := (ϒh(t), ϒv(t))′ ∈ R2 is an arbitrary continuously differentiable func-

tion. Also (∂p′

2∂q5

)+ and (∂p′

2∂q5

)− are the pseudo inverse and projection matrices,

PROOF OF LEMMA 3.7 231

respectively. For the bipedal robot, since q5 decreases in the clockwise direction,using Proposition B.8 of Ref. [18, p. 424],

∂p2

∂q5= ∂

∂q5

[x2

y2

]=

[−y2

x2

]=

[0

Ls

]. (A.4)

Therefore, rank∂p′

2∂q5

= rank[0, Ls]′ = 1 for every t ∈ [Ts, T ). Furthermore, fromequations (A.3) and (A.4),

F2(t) =[

01Ls

]e′

5(D qd(t) + C qd(t) + G) +[

1 0

0 0

]ϒ(t). (A.5)

Substituting equation (A.5) into the first four rows of equation (A.2) implies that

ud(t) = u0d(t) − ∂x′

2

∂qb

ϒh(t).

Since qd(t) is such that for every t ∈ [Ts, T ),

∂p2

∂q(qd(t)) qd(t) + ∂

∂q

(∂p2

∂q(qd(t))qd(t)

)qd(t) = 02×1

and rank ∂p2∂q

(qd(t)) = 2, from equation (3.16), knowledge of ud(t) results in a uniqueF2(t). Moreover, ud(t) satisfies equation (3.17) because equations (3.14) and (3.16)result in equation (3.17), which completes the proof. �

APPENDIX B


B.1 PROOF OF LEMMA 4.2

Proof. Since from Remark 4.3,

xmax3

(τ; xf

2

)=

⎧⎪⎨⎪⎩

x03 + x0

4(τ − t∗1 ) + L12 (τ − t∗1 )2, t∗1 ≤ τ ≤ τ

(xf2

)x3 + x4

(τ − t

(xf2

))+ L2

2

(τ − t

(xf2

))2, τ

(xf2

)≤ τ ≤ t∗2 ,

the sensitivity function ∂xmax3 (τ; xf

2 )/∂τ can be expressed as follows:

∂xmax3∂τ

(τ; xf

2

)=

⎧⎨⎩

0, t∗1 ≤ τ ≤ τ(xf2

)(L1 − L2)

(τ − τ

(xf2

)), τ

(xf2

)≤ τ ≤ t∗2 .

(B.1)

Moreover, from equation (4.24),

∂τ

∂xf2

(xf2

)= 2

(L1 − L2)(τ − t∗2 )2 ,

together with equation (B.1) yields

∂xmax3

∂xf

2

(τ; xf

2

)=

⎧⎪⎪⎨⎪⎪⎩

0, t∗1 ≤ τ ≤ τ(xf2

)2(τ−τ

(xf

2

))(τ(xf

2

)−t∗2

)2 , τ(xf2

)≤ τ ≤ t∗2 .

(B.2)


233


Therefore,

∂xmax3

∂xf2

(τ; xf

2

)≥ 0

for any (τ, xf2 )′ ∈ [t∗1 , t∗2 ] × �max

m,M,L1,L2(x0

3, x04), which, in turn, results in

xmax3 (τ; α) ≤ xmax

3 (τ; γ) ≤ xmax3 (τ; β)

for any τ ∈ [t∗1 , t∗2 ]. Moreover, from part (a) of Lemma 4.1, α, β ∈ �maxm,M,L1,L2

(x03, x

04)

implies that for every τ ∈ [t∗1 , t∗2 ],

m < xmax3 (τ; α) ≤ xmax

3 (τ; γ) ≤ xmax3 (τ; β) < M,

which completes the proof. �

B.2 PROOF OF THEOREM 4.2

Proof. The construction procedure for ϕd and condition (4.34) immediately implypart (a). Next, we show that the trajectory ϕd(t), 0 ≤ t ≤ t1 with the initial conditionθ(0) = θ0 results in θ(t1) = θ1. For this purpose, define

C1 := {ψ ∈ Qb|ψ = ϕd(t), 0 ≤ t ≤ t1}.

Then, by introducing the variable s = σcmσ∗

cmt,

θ(t1) = θ0 +∫ t1

0

σcm

A3,3(ϕd(t))dt −

∫C1

J(ϕd) dϕd

= θ0 +∫ t∗1

0

σ∗cm

A3,3(ϕ∗(s))ds −

∫C∗

1

J(ϕ∗) dϕ∗

= θ1,

where

C∗1 := {ψ ∈ Qb|ψ = ϕ∗(t), 0 ≤ t ≤ t∗1 } = C1.

PROOF OF THEOREM 4.2 235

Using the proposed reconfiguration algorithm, θ(t1) = θ1 and ϕd(t) = ϕ∗(z1(t)), t1 ≤t ≤ t2 imply that θ(t2) = θ2. In addition, the change of variable s = σcm

σ∗cm

(t − tf ) + t∗fyields

θ(tf ) = θ2 +∫ tf

t2

σcm

A3,3(ϕd(t))dt −

∫C3

J(ϕd) dϕd

= θ2 +∫ t∗

f

t∗2

σ∗cm

A3,3(ϕ∗(s))ds −

∫C∗

3

J(ϕ∗) dϕ∗

= θf ,

where

C3 := {ψ ∈ Qb|ψ = ϕd(t), t2 ≤ t ≤ tf }C∗

3 := {ψ ∈ Qb|ψ = ϕ∗(t), t∗2 ≤ t ≤ t∗f } = C3.

If x3(τ) ≡ 1, t∗1 ≤ τ ≤ t∗2 , the states x1 and x2 of the system �a are transferred from

the origin at t∗1 to xf∗1 := ∫ t∗2

t∗1w(s)ds and x

f∗2 := lmax = t∗2 − t∗1 at t∗2 , respectively. On

the other hand, x03 = 1, x0

4 = 0, L2 = −L1, and 0 < m < 1 < M imply that

τ(xf∗2

)= τ

(xf∗2

)= t∗2 − 3

√1

2lmax

and ϑ = 12 , which, in turn, follows that x3(τ; xf∗

1 , xf∗2 ) ≡ 1. This completes the proof

of part (c). �

APPENDIX C


C.1 PROOF OF LEMMA 6.1

Proof. Define Ss := block diag{S1, S2}. By the symmetrical structure of the monope-dal robot, for every (q′

s, q′s)

′ ∈ TQs,

Ls(qs, qs) = Ls(Ssqs, Ssqs),

which, in turn, results in

Ks(qs, qs) = Ks(Ssqs, Ssqs)

Vs(qs) = Vs(Ssqs).

Consequently, for every (q′s, q

′s)

′ ∈ TQs,

Ds(qs) = S′s Ds(Ssqs) Ss

Cs(qs, qs) = S′s Cs(Ssqs, Ssqs) Ss

Gs(qs) = S′s Gs(Ssqs).

Next, let q∗s (t), 0 ≤ t ≤ T ∗

s together with the open-loop control input u∗s (t) satisfy the

differential equation Ds(q∗s )q∗

s + Cs(q∗s , q

∗s )q∗

s + Gs(q∗s ) = Bu∗

s . This equation canalso be expressed as

S′s Ds(Ssq

∗s ) Ss q∗

s + S′s Cs(Ssq

∗s , Ss q∗

s ) Ssq∗s + S′

s Gs(Ssq∗s ) = B u∗

s . (C.1)


237


Premultiplying equation (C.1) by the matrix (S′s)

−1 and decomposing the result yields

Ds,ϕϕ(Ssq∗s ) S1 ϕ∗ + Ds,ϕθ(Ssq

∗s ) S2 θ∗ + Hϕ(Ssq

∗s , Ssq

∗s ) = S1 u∗

s

Ds,θϕ(Ssq∗s ) S1 ϕ∗ + Ds,θθ(Ssq

∗s ) S2 θ∗ + Hθ(Ssq

∗s , Ssq

∗s ) = 03×1,

which completes the proof of part (1). The proof of part (2) is similar. �

C.2 PROOF OF LEMMA 6.2

Proof. The first part, that is, q = T (q) := T0q + T1 is immediate. For the later pur-poses, we remark that T0 is invertible. Since the kinetic energy of the mechanicalsystem is invariant under coordinates transformations, that is, Ks(q, q) = Ks(q, ˙q), itcan be concluded that

D(q) = (T−10 )′ Ds(T

−1(q)) T−10 .

By assuming that the decoupling matrix LgsLfsyjs (xj

s ; ξ∗j) is invertible on the or-

bit Ojs for j = 1, 2, there exists an open neighborhood Nj of Oj

s × ξj∗ such thatfor every (xj

s , ξj) ∈ Nj(Oj

s × ξj∗), the output function yjs (xj

s ; ξj) has vector relativedegree (2, 2, 2). Next, define the new state variable x := (q′, ˙q′)′ and introduce thestate equation in the new coordinates as ˙x = f s(x) + gs(x)u, where

gs(xs) = gs(q) :=[

06×3

D−1(q) B

].

Since (i) the change of coordinates q = T (q) does not change the vector relativedegree of the output y

js and (ii) the distribution generated by the columns of the

matrix gs is involutive, by Ref. [103, p. 127], (θ, σs) is valid local coordinates for thezero dynamics manifold Zj

s,ξj because

σs(q, ˙q) = E2 D(q) ˙q

and

Lgsσs(q, ˙q) =[

∂σs

∂q(q, ˙q) ∂σs

∂ ˙q(q, ˙q)

] [06×3

D−1(q)B

]= E2 B = 03×3.

PROOF OF LEMMA 6.2 239

In the coordinates (θ, σs) for the stance phase zero dynamics, ϕ = �js,d(γ; ξj) and

hence, σs can be expressed as

σs = E2 D(q) ˙q = Dθϕ(q) ϕ + Dθθ(q) ˙θ

=(Dθθ(q) + Dθϕ(q)

∂�js,d

∂γ(γ; ξj) e′

2

)˙θ,

where e2 = [0 1 0]′. In addition, since

σs = ∂Ls

∂ ˙θ= E2 D(q) ˙q,

the vector θ is unactuated and θ3 is the cyclic variable for the stance phase (i.e.,∂Ks

∂θ3= 0), the Euler–Lagrange equations imply that

σs,1 = ∂Ls

∂θ1= ∂Ks

∂θ1− ∂Vs

∂θ1

σs,2 = ∂Ls

∂γ= ∂Ks

∂γ− ∂Vs

∂γ

σs,3 = ∂Ls

∂θ3= 0.

(C.2)

We remark that θ3 is the orientation about the z-axis and hence, ∂Vs

∂θ3= 0. Since (i)

during stance phases,

pcm = �l(q) = R(θ) ϒl(ϕ)

and (ii)

Vs(q) = mtotg0�vl (q) = mtotg0R3(θ) ϒl(ϕ),

where R3 denotes the third row of the rotation matrix R, we obtain

∂Vs

∂θ1= ∂Vs

∂q

∂q

∂q

∂q

∂θ1= mtot g0

∂R3

∂θ1R−1pcm

= −mtot g0 sin(θ3) xcm + mtot g0 cos(θ3) ycm

∂Vs

∂γ= ∂Vs

∂q

∂q

∂q

∂q

∂γ= ∂Vs

∂θ2= mtot g0

∂R3

∂θ2R−1pcm

= −mtot g0 cos(θ3) cos(θ1) xcm − mtot g0 sin(θ3) cos(θ1) ycm.

(C.3)


Moreover, on the zero dynamics manifold Zj

s,ξj ,

∂hjs

∂qq = ∂h

js

∂qT−1

0˙q = 03×1,

which together with σs = E2D(q) ˙q imply that ˙q = λj(θ; ξj)σs. Substituting equation(C.3) and ˙q = λj(θ; ξj)σs into equation (C.2) completes the proof. �

C.3 INVERTIBILITY OF THE STANCE PHASE DECOUPLING MATRIXON THE PERIODIC ORBIT

Following the proof of Lemma 6.2 in Appendix C.2, the decoupling matrix in the newcoordinates (q, ˙q) can be expressed as

LgsLf shj

s (q; ξj) = ∂hjs

∂qD−1B =

(E1 − ∂�

js,d

∂γe′

2 E2

)D−1

[I3×3

03×3

],

where E1 := [I3×3 03×3], E2 := [03×3 I3×3], and e2 := [0 1 0]′. By defining thesymmetric matrix

� := D−1 =[�ϕϕ �ϕθ

�θϕ �θθ

]

and considering the fact that

�θϕ = −D−1θθ

Dθϕ �ϕϕ,

it can be concluded that

LgsLf shj

s (q; ξj) = �ϕϕ

(I3×3 + ∂�

js,d

∂γe′

2 D−1θθ

Dθϕ

).

Since � is positive definite (det �ϕϕ /= 0) and on the periodic orbit �js,d = ϕ

js,d , it

can be concluded that det LgsLf sh

js (q; ξj∗) /= 0 if and only if1

κj(θ) = 1 + e′2 D−1

θθDθϕ

∂ϕjs,d

∂γ= det

(I3×3 + ∂ϕ

js,d

∂γe′

2 D−1θθ

Dθϕ

)/= 0.

1 We remark that for every A ∈ Rm×n and B ∈ Rn×m, det(Im×m + AB) = det(In×n + BA).

BIBLIOGRAPHY

[1] J. W. Grizzle, C. H. Moog, and C. Chevallereau, “Nonlinear control of mechanicalsystems with an unactuated cyclic variable,” IEEE Transactions on Automatic Control,vol. 30, no. 5, pp. 559–576, May 2005.

[2] K. Hirai, M. Hirose, Y. Haikawa, and T. Takenake, “The development of Honda hu-manoid robot,” Proceedings of the 1998 IEEE International Conference on Roboticsand Automation, Leuven, Belgium, pp. 1321–1326, May 1998.

[3] M. Vukobratovic and B. Borovac, “Zero-moment point-thirty five years of its life,”International Journal of Humanoid Robotics, vol. 1, no. 1, pp. 157–173, 2004.

[4] M. Vukobratovic, B. Borovac, D. Surla, and D. Stokic, Biped Locomotion, Springer-Verlag, Berlin, 1990.

[5] H. Lim, Y. Yamamoto, and A. Takanishi, “Control to realize human-like walking of abiped humanoid robot,” Proceedings of the IEEE International Conference on Systems,Man and Cybernetics, Nashville, TN, pp. 3271–3276, June 2000.

[6] J. Yamaguchi, E. Soga, S. Inoue, and A. Takanishi, “Development of a bipedial humanoidrobot-control method of whole body cooperative dynamic biped walking,” Proceedingsof the 1999 IEEE International Conference on Robotics and Automation, Detroit, MI,pp. 368–374, May 1999.

[7] K. Loffler, M. Gienger, F. Pfeiffer, and H. Ulbrich, “Sensors and control concept of abiped robot,” IEEE Transactions on Industrial Electronics, vol. 51, no. 5, pp. 972–980,2004.

[8] J. H. Park and K. D. Kim, “Biped robot walking using gravity-compensated invertedpendulum mode and computed torque control,” Proceedings of the 1998 IEEE Interna-tional Conference on Robotics and Automation, Leuven, Belgium, pp. 3528–3533, May1998.

[9] K. Erbatur and O. Kurt, “Natural ZMP trajectories for biped robot reference generation,”IEEE Transactions on Industrial Electronics, vol. 56, no. 3, pp. 835–845, March 2009.

[10] L. Bum-Joo, D. Stonier, K. Yong-Duk, Y. Jeong-Ki, and K. Jong-Hwan, “Modifiablewalking pattern of a humanoid robot by using allowable ZMP variation,” IEEE Trans-actions on Robotics, vol. 24, no. 4, pp. 917–925, August 2008.

[11] N. Motoi, M. Ikebe, and K. Ohnishi, “Real-time gait planning for pushing mo-tion of humanoid robot,” IEEE Transactions on Industrial Informatics, vol. 3, no. 2,pp. 154–163, May 2007.


241

242 BIBLIOGRAPHY

[12] S. Kajita, T. Nagasaki, K. Kaneko, and H. Hirukawa, “ZMP-based biped running control,”IEEE Robotics and Automation Magazine, vol. 14, no. 2, pp. 63–72, June 2007.

[13] B. Ugurlu and A. Kawamura, “ZMP-based online jumping pattern generation fora one-legged robot,” IEEE Transactions on Industrial Electronics, vol. 57, no. 5,pp. 1701–1709, May 2010.

[14] T. Sato, S. Sakaino, E. Ohashi, and K. Ohnishi, “Walking trajectory planning on stairsusing virtual slope for biped robots,” IEEE Transactions on Industrial Electronics, vol.58, no. 4, pp. 1385–1396, April 2011.

[15] P. Sardain and G. Bessonnet, “Forces acting on a biped robot. Center of pressure-zeromoment point,” IEEE Transactions on Systems, Man and Cybernetics, Part A: Systemsand Humans, vol. 34, no. 5, pp. 630–637, September 2004.

[16] E. Kim, T. Kim, and J. W. Kim, “Three-dimensional modelling of a humanoid in threeplanes and a motion scheme of biped turning in standing,” IET (IEE) Control Theoryand Applications, vol. 3, no. 9, pp. 1155–1166, September 2009.

[17] A. Goswami, “Postural stability of biped robots and the foot-rotation indicator (FRI)point,” International Journal of Robotics Research, vol. 18, no. 6, pp. 523–533,June 1999.

[18] E. R. Westervelt, J. W. Grizzle, C. Chevallereau, J. H. Choi, and B. Morris, FeedbackControl of Dynamic Bipedal Robot Locomotion, Boca Raton, CRC Press, June 2007.

[19] S. Kajita and K. Tani, “Study of dynamic biped locomotion on rugged terrain-derivationand application of the linear inverted pendulum mode,” Proceedings of the 1991IEEE International Conference on Robotics and Automation, pp. 1405–1411, April1991.

[20] K. Erbatur and U. Seven, “An inverted pendulum based approach to biped trajectorygeneration with swing leg dynamics,” Proceedings of the 2007 IEEE-RAS InternationalConference on Humanoid Robots, pp. 216–221, December 2007.

[21] H. Miura and I. Shimoyama, “Dynamic walk of a biped,” International Journal ofRobotics Research, vol. 3, no. 2, pp. 60–74, June 1984.

[22] R. Katoh and M. Mori, “Control method of biped locomotion giving asymptotic stabilityof trajectory,” Automatica, vol. 20, no. 4, pp. 405–414, July 1984.

[23] J. Furusho and M. Masubuchi, “Control of a dynamical biped locomotion system forsteady walking,” Journal of Dynamic Systems, Measurement, and Control, vol. 108,no. 2, pp. 111–118, 1986.

[24] J. Furusho and A. Sano, “Sensor-based control of a nine-link biped,” International Jour-nal of Robotics Research, vol. 9, no. 2, pp. 83–98, 1990.

[25] A. Sano and J. Furusho, “Realization of natural dynamic walking using the angularmomentum information,” Proceedings of the 1990 IEEE International Conference onRobotics and Automation, Cincinnati, OH, pp. 1476–1481, May 1990.

[26] S. Kajita and K. Tani, “Experimental study of biped dynamic walking,” IEEE ControlSystems Magazine, vol. 16, no. 1, pp. 13–19, February 1996.

[27] S. Kajita, T. Yamaura, and A. Kobayashi, “Dynamic walking control of a biped robotalong a potential energy conserving orbit,” IEEE Transactions on Robotics and Automa-tion, vol. 8, no. 4, pp. 431–438, August 1992.

[28] A. A. Grishin, A. M. Formal’sky, A. V. Lensky, and S. V. Zhitomirsky, “Dynamicalwalking of a vehicle with two telescopic legs controlled by two drives,” InternationalJournal of Robotics Research, vol. 13, no. 2, pp.137–147, 1994.

BIBLIOGRAPHY 243

[29] K. Mitobe, N. Mori, K. Aida, and Y. Nasu, “Nonlinear feedback control of a bipedwalking robot,” Proceedings of the 1995 IEEE International Conference on Roboticsand Automation, Nagoya, Japan, pp. 2865–2870, May 1995.

[30] M. H. Raibert, S. Tzafestas, and C. Tzafestas, “Comparative simulation study of threecontrol techniques applied to a biped robot,” Proceedings of the IEEE InternationalConference on Systems, Man and Cybernetics Systems Engineering in the Service ofHumans, Le Touquet, France, pp. 494–502, October 1993.

[31] Y. Fujimoto and A. Kawamura, “Simulation of an autonomous biped walking robotincluding environmental force interaction,” IEEE Robotics and Automation Magazine,vol. 5, no. 2, pp. 33–42, June 1998.

[32] Y. Fujimoto, S. Obata, and A. Kawamura, “Robust biped walking with active in-teraction control between foot and ground,” Proceedings of the 1998 IEEE Interna-tional Conference on Robotics and Automation, Leuven, Belgium, pp. 2030–2035,May 1998.

[33] S. Kajita, F. Kanehiro, K. Kaneko, K. Fujiwara, K. Yokoi, and H. Hirukawa, “A real-time pattern generator for biped walking,” Proceedings of the 2002 IEEE InternationalConference on Robotics and Automation, Washington, D.C., pp. 31–37, 2002.

[34] S. Kajita, F. Kanehiro, K. Kaneko, K. Yokoi, and H. Hirukawa, “The 3D linear invertedpendulum mode: A simple modeling for a biped walking pattern generation,” Proceed-ings of the 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems,Maui, HI, pp. 239–246, November 2001.

[35] S. Grillner, “Control of locomotion in bipeds, tetrapods and fish,” Handbook ofPhysiology II, Bethesda, MD, American Physiological Society, pp. 1179–1236,1981.

[36] G. Taga, Y. Yamaguchi, and H. Shimizu, “Self-organized control of bipedal locomotionby neural oscillators,” Biological Cybernetics, vol. 65, no. 3, pp. 147–159, 1991.

[37] G. Taga, “A model of the neuro-musculo-skeletal system for human locomotion II:Realtime adaptability under various constraints,” Biological Cybernetics, vol. 73, no. 2,pp. 113–121, 1995.

[38] K. Matsuoka, “Mechanism of frequency and pattern control in the neural rhythm gener-ators,” Biological Cybernetics, vol. 56, nos. 5–6, pp. 345–353, 1987.

[39] K. Matsuoka, “Sustained oscillations generated by mutually inhibiting neurons withadaptation,” Biological Cybernetics, vol. 52, no. 6, pp. 367–376, 1985.

[40] M. M. Williamson, “Neural control of rhythmic arm movements,” Neural Networks, vol.11, nos. 7–8, pp. 1379–1394, 1998.

[41] N. Sadati, G. A. Dumont, and K. A. Hamed, Design of a Neural Controller for Walkingof a 5-Link Planar Biped Robot via Optimization, Human-Robot Interaction. D. Chugo(ed.), Rijeka, Croatia, InTech, pp. 267–288, February 2010.

[42] S. Miyakoshi, G. Taga, Y. Kuniyoshi, and A. Nagakubo, “Three dimensional bipedalstepping motion using neural oscillators-towards humanoid motion in the real word,”Proceedings of the 1998 IEEE/RSJ International Conference on Intelligent Robots andSystems, Victoria, BC, Canada, pp. 84–89, May 1998.

[43] H. Kimura, Y. Fukuoka, Y. Hada, and K. Takase, “Three-dimensional adaptive dynamicwalking of a quadruped-rolling motion feedback to CPGs controlling pitching motion,”Proceedings of the IEEE International Conference on Robotics and Automation,pp. 2228–2233, May 2002.

244 BIBLIOGRAPHY

[44] H. Kimura and Y. Fukuoka, “Adaptive dynamic walking of the quadruped on irregularterrain-autonomous adaptation using neural system model,” Proceedings of the IEEEInternational Conference on Robotics and Automation, pp. 436–443, 2000.

[45] J. Or, “A hybrid CPG-ZMP controller for the real-time balance of a simulated flexiblespine humanoid robot,” IEEE Transactions on Systems, Man, and Cybernetics, Part C:Applications and Reviews, vol. 39, no. 5, pp. 547–561, September 2009.

[46] J. W. Grizzle, G. Abba, and F. Plestan, “Asymptotically stable walking for biped robots:Analysis via systems with impulse effects,” IEEE Transactions on Automatic Control,vol. 46, no. 1, pp. 51–64, January 2001.

[47] C. Chevallereau, G. Abba, Y. Aoustin, F. Plestan, E. R. Westervelt, C. Canudas-DeWit,and J. W. Grizzle, “RABBIT: A testbed for advanced control theory,” IEEE ControlSystems Magazine, vol. 23, no. 5, pp. 57–79, October 2003.

[48] J. W. Grizzle, F. Plestan, and G. Abba, “Poincare’s method for systems with impulseeffects: Application to the mechanical biped locomotion,” Proceedings of the 1999 IEEEInternational Conference on Decision and Control, Phoenix, AZ, pp. 3869–3876, De-cember 1999.

[49] J. W. Grrizle, G. Abba, and F. Plestan, “Proving asymptotic stability of a walking cyclefor a five DOF biped robot model,” Proceedings of the 1999 International Conferenceon Climbing and Walking Robots, pp. 69–81, September 1999.

[50] E. R. Westervelt, J. W. Grizzle, and D. E. Koditschek, “Zero dynamics of underactuatedplanar biped walkers,” Proceedings of the 15th IFAC World Congress, Barcelona, Spain,pp. 1–6, July 2002.

[51] F. Plestan, J. W. Grizzle, E. R. Westervelt, and G. Abba, “Stable walking of a 7-DOF biped robot,” IEEE Transactions on Robotics and Automation, vol. 19, no. 4,pp. 653–668, August 2003.

[52] E. R. Westervelt, J. W. Grizzle, and D. E. Koditschek, “Hybrid zero dynamics of planarbiped walkers,” IEEE Transactions on Automatic Control, vol. 48, no. 1, pp. 42–56,January 2003.

[53] C. Chevallereau, J. W. Grizzle, and C. H. Moog, “Nonlinear control of mechanicalsystems with one degree of underactuation,” Proceedings of the 2004 IEEE InternationalConference on Robotics and Automation, New Orleans, LA, pp. 2222–2228, April 2004.

[54] J. H. Choi and J. W. Grizzle, “Planar bipedal walking with foot rotation,” Proceedingsof the 2005 American Control Conference, Portland, OR, pp. 4909–4916, June 2005.

[55] C. Chevallereau, E. R. Westervelt, and J. W. Grizzle, “Asymptotically stable runningfor a five-link, four-actuator, planar bipedal robot,” International Journal of RoboticsResearch, vol. 24, no. 6, pp. 431–464, June 2005.

[56] B. Morris, E. R. Westervelt, C. Chevallereau, G. Buche, and J. W. Grizzle, “Achievingbipedal running with RABBIT: Six steps toward infinity,” Fast Motions Symposiumon Biomechanics and Robotics, Lecture Notes in Control and Information Sciences,Springer-Verlag, Heidelberg, Germany, pp. 277–297, 2006.

[57] J. W. Grizzle, E. R. Westervelt, and C. Canudas, “Event-based PI control of an underactu-ated biped walker,” Proceedings of the 2003 IEEE International Conference on Decisionand Control, Maui, HI, pp. 3091–3096, December 2003.

[58] E. R. Westervelt, J. W. Grizzle, and C. Canudas, “Switching and PI control of walkingmotions of planar biped walkers,” IEEE Transactions on Automatic Control, vol. 48,no. 2, pp. 308–312, February 2003.

BIBLIOGRAPHY 245

[59] J. W. Grizzle, “Remarks on event-based stabilization of periodic orbits in systems withimpulse effects,” Second International Symposium on Communications, Control andSignal Processing, 2006.

[60] B. Morris and J. W. Grizzle, “Hybrid invariant manifolds in systems with impulse ef-fects with application to periodic locomotion in bipedal robots,” IEEE Transactions onAutomatic Control, vol. 54, no. 8, pp. 1751–1764, August 2009.

[61] C. Chevallereau, J. W. Grizzle, and C. L. Shih, “Asymptotically stable walking of afive-link underactuated 3-D bipedal robot,” IEEE Transactions on Robotics, vol. 25,no. 1, pp. 37–50, February 2009.

[62] C. L. Shih, J. W. Grizzle, and C. Chevallereau, “From stable walking to steering of a 3Dbipedal robot with passive point feet,” submitted to Robotica, January 2010.

[63] H. W. Park, K. Sreenath, J. W. Hurst, and J. W. Grizzle, “Identification of a bipedalrobot with a compliant drivetrain,” Control Systems Magazine, vol. 31, no. 2, pp. 63–88,April 2011.

[64] K. Sreenath, H. W. Park, I. Poulakakis, and J. W. Grizzle, “A compliant hybrid zerodynamics controller for stable, efficient and fast bipedal walking on MABEL,” In-ternatioanl Journal of Robotics Research, vol. 30, no. 9, pp. 1170–1193, August2011.

[65] J. W. Grizzle, J. Hurst, B. Morris, H. W. Park, and K. Sreenath, “MABEL, a new roboticbipedal walker and runner,” Proceedings of the American Control Conference, St. Louis,MO, pp. 2030–2036, June 2009.

[66] http://mime.oregonstate.edu/research/drl/

[67] Y. Hurmuzlu, “Dynamics of bipedal gait-Part 1: Objective functions and the con-tact event of a planar five-link biped,” Journal of Applied Mechanics, vol. 60, no. 2,pp. 331–336, 1998.

[68] K. Akbari Hamed, N. Sadati, W. A. Gruver, and G. A. Dumont, “Stabilization of periodicorbits for planar walking with noninstantaneous double-support phase,” IEEE Transac-tions on Systems, Man, and Cybernetics, Part A: Systems and Humans, vol. 42, no. 3,pp. 685–706, May 2012.

[69] N. Sadati, K. Akbari Hamed, W. A. Gruver, and G. A. Dumont, “Radial basis functionnetwork for exponential stabilisation of periodic orbits for planar bipedal walking,” IET(IEE) Electronics Letters, vol. 47, no. 12, pp. 692–694 June 2011.

[70] C. Canudas, B. Siciliano, and G. Bastin, Theory of Robot Control, Springer-Verlag,London, 1996.

[71] S. Miossec and Y. Aoustin, “A simplified stability study for a biped walk with under-actuated and overactuated phases,” International Journal of Robotics Research, vol. 24,no. 7, pp. 537–551, June 2005.

[72] D. Djoudi and C. Chevallereau, “Feet can improve the stability property ofa control law for a walking robot,” Proceedings of the 2006 IEEE Interna-tional Conference on Robotics and Automation, Orlando, FL, pp. 1206–1212, May2006.

[73] C. Chevallereau and Y. Aoustin, “Optimal reference trajectories for walking and runningof a biped robot,” Robotica, vol. 19, no. 5, pp. 557–569, September 2001.

[74] S. Dubowsky and E. Papadopoulos, “The kinematics, dynamics and control of free-flyingand free-floating space robotic systems,” IEEE Transactions on Robotic and Automation,vol. 9, no. 5, pp. 531–543, October 1993.

246 BIBLIOGRAPHY

[75] C. Fernandes, L. Gurvits, and Z. Li, “Near-optimal nonholonomic motion planning fora system of coupled rigid bodies,” IEEE Transactions on Automatic Control, vol. 39,no. 3, pp. 450–463, March 1994.

[76] C. Fernandes, L. Gurvits, and Z. Li, “Attitude control of space paltform/manipulatorsystem using internal motion,” Proceedings of the 1992 IEEE International Conferenceon Robotics and Automation, Nice, France, pp. 893–898, March 1994.

[77] P. S. Krishnaprasad, “Geometric phases and optimal reconfiguration for multi-body systems,” Proceedings of the American Control Conference, pp. 2440–2444,May 1990.

[78] G. C. Walsh and S. S. Sastry, “On reorienting linked rigid bodies using internal mo-tions,” IEEE Transactions on Robotics and Automation, vol. 11, no. 1, pp. 139–146,February 1995.

[79] L. Gurvits and Z. X. Li, “Smooth time-periodic feedback solutions for nonholonomicmotion planning,” Nonholonomic Motion Planning. Z. Li and J. F. Canny (eds.), Kluwer,pp. 53–108, 1993.

[80] I. Kolmanovsky, N. McClamroch, and V. Coppola, “New results on control of multi-body systems which conserve angular momentum,” Journal of Dynamical and ControlSystems, vol. 1, no. 4, pp. 447–462, 1995.

[81] N. Sadati, G. A. Dumont, K. Akbari Hamed, and W. A. Gruver, “Two-level controlscheme for stabilisation of periodic orbits for planar monopedal running,” IET (IEE)Control Theory and Applications, vol. 5, no. 13, pp. 1528–1543, August 2011.

[82] N. Sadati, K. Akbari Hamed, G. A. Dumont, and W. A. Gruver, “Nonholonomic motionplanning based on optimal control for flight phases of planar bipedal running,” IET (IEE)Electronics Letters, vol. 47, no. 20, pp. 1120–1122, September 2011.

[83] N. Sadati, W. A. Gruver, K. Akbari Hamed, and G. A. Dumont, “Online generation ofjoint motions during flight phases in running of planar bipedal robots,” IET (IEE) ControlTheory and Applications, under review, January 2012.

[84] C. Chevallereau, Y. Aoustin, and A. Formal’sky, “Optimal walking trajectories for abiped,” Proceedings of the First Workshop on Robot Motion and Control, Kiekrz, Poland,pp. 171–176, June 1999.

[85] K. Akbari Hamed, N. Sadati, W. A. Gruver, and G. A. Dumont, “Exponen-tial stabilisation of periodic orbits for running of a three-dimensional monopedalrobot,” IET (IEE) Control Theory and Applications, vol. 5, no. 11, pp. 1304–1320,August 2011.

[86] K. Akbari Hamed, N. Sadati, G. A. Dumont, and W. A. Gruver, “Exponential stabilizationof periodic motions for a five-link, two-actuator planar bipedal robot,” InternationalJournal of Robotics and Automation, submitted 15 September 2010, revised 14 April2011.

[87] K. Akbari Hamed, N. Sadati, W. A. Gruver, and G. A. Dumont, “Continuous-time up-date laws with radial basis step length for control of bipedal locomotion,” IET (IEE)Electronics Letters, vol. 46, no. 21, pp. 1431–1433, October 2010.

[88] H. K. Khalil. Nonlinear Systems - 3rd Edition, Upper Saddle River, NJ, 2002.

[89] T. S. Parker and L. O. Chua, Practical Numerical Algorithms for Chaotic Systems,Springer-Verlag, New York, 1989.

[90] M. W. Spong, S. Hutchinson, and M. Vidyasagar, Robot Modeling and Control, Wiley,New York, 2006.

BIBLIOGRAPHY 247

[91] Y. Hurmuzlu and D. B. Marghitu, “Rigid body collisions of planar kinematic chainswith multiple contact points,” International Journal of Robotics Research, vol. 13,no. 1, pp. 82–92, 1994.

[92] X. Mu and Q. Wu, “On impact dynamics and contact events for biped robots via impacteffects,” IEEE Transactions on Systems, Man, and Cybernetics-Part B: Cybernetics,vol. 36, no. 6, pp. 1364–1372, December 2006.

[93] S. P. Bhat and D. S. Bernstein, “Continuous finite-time stabilization of the translationaland rotational double integrators,” IEEE Transactions on Automatic Control, vol. 43,no. 5, pp. 678–682, May 1998.

[94] E. R. Westervelt, B. Morris, and K. D. Farrell, “Sample-based HZD control for ro-bustness and slope invariance of planar passive bipedal gaits,” Proceedings of the 14thMediterranean Conference on Control and Automation, pp. 1–6, June 2006.

[95] M. W. Spong, “Energy based control of a class of underactuated mechanical systems,”Proceedings of IFAC World Congress, San Francisco, CA, pp. 431–435, 1996.

[96] E. R. Westervelt, G. Buche, and J. W. Grizzle, “Experimental validation of a frameworkfor the design of controllers that induce stable walking in planar bipeds,” InternationalJournal of Robotics Research, vol. 23, no. 6, pp. 559–582, 2004.

[97] X. Mu and Q. Wu, “A complete dynamic model of five-link bipedal walking,” Proceed-ings of the American Control Conference, pp. 4926–4931, June 2003.

[98] N. Sreenath, “Nonlinear control of planar multibody systems in shape space,”Mathematics of Control Signals System vol. 5, no. 4, pp. 343–363, 1992.

[99] A. E. Bryson and Y. C. Ho, Applied Optimal Control, Hemisphere, New York, 1975.

[100] D. E. Kirk, Optimal Control Theory: An Introduction, Dover, New York, 1970.

[101] M. Athans and P. L. Falb, Optimal Control: An Introduction to the Theory and ItsApplications, McGraw-Hill, New York, 1966.

[102] A. F. Filippov, Differential Equations with Discontinuous Righthand Sides, Kluwer Aca-demic, Dordrecht, 1988.

[103] A. Isidori, Nonlinear Control Systems-3rd Edition, Springer-Verlag, Berlin, 1995.

INDEX

Absolute orientation, 8Acceleration

leg end, 165, 168vertical, 67, 171, 230

Actuators, 2, 6, 8, 10, 11, 36, 92, 194internal, joints controlled by, 97, 138

Algorithms, 3based on virtual time for, 138lack of, 3online motion planning, 9online reconfiguration, 9

parameters, 132reconfiguration, for flight phase, 99–120

Anglesbody, 37, 72, 73, 97, 98, 100, 118, 138,

160, 169, 175, 179of hip joints, 11increase in clockwise direction, 37joint, 8, 9, 63, 122, 137, 157, 158, 196relative, 138torso, 61, 73trunk, 49virtual leg with respect to world frame,

47, 161Angular momentum, 4, 82, 95

conservation of, 99of mechanical system about COM, 95transfer, 82, 83, 88, 92

effects of double support phase, 88Applications

to legged locomotion, 227, 228Poincare sections method, 5

ATRIAS, 7Augmented hybrid system, 33

Bezier polynomial, 12, 168, 169, 176, 179,182, 189

coefficient matrix, 182linearized open-loop restriction,

189of time, 171, 198, 205

Bipedal robot, 7, 8, 10, 35–37, 50, 75, 78,92, 96, 193

mechanical model, 194, 195physical parameters, 200stick animation, 201

Body configuration space, 139

Cartesian coordinates, 161Center of pressure (COP), 2–4Central pattern generators (CPG), 5Centrifugal matrix, 162Christoffel symbols, 163Closed-loop

control inputs, 219hybrid system, 29–31

simulation, 82, 217–219impulsive model of walking, 217

COM frames, 160, 161Complexity, 29Configuration

determinism at landing, 95determinism at transitions, 137

Constraints, 76equality, interpretation, 172unilateral, 3, 7, 8, 36, 196, 197virtual, 96

concept, 193two-level feedback law, 193


249

250 INDEX

Continuous-time controllers, 202–209, 212closed-loop dynamics, 208decoupling matrices, invertibility, 206Euler–Lagrange equation, 209event-based update laws, 202index of next phase, 203input–output linearization, 206nominal holonomic output function,

205parameterized holonomic output function,

205within-stride switching hypersurfaces,

209within-stride transitions, 204zero dynamics, 208

Continuous-time feedback law, 193Continuous-time update laws, 221

impulsive systems, class, 222event-based update law, 223exponential stability behavior, 222,

224periodic orbit, 222proof, 224radial basis step length, 223time-to-impact function, 225trajectory sensitivity matrices, 225

legged locomotion, application,227, 228

locomotion, continuous phases, 221radial basis step length, 221

Controlevent-based controllers, 209inputs in transitions, 45inputs of mechanical system, 10input vector, 28laws (See Control laws)open-loop, 237

control inputs, 83, 104, 116, 151two-level control strategy, 30, 31

Control laws, 46double support phase, 49

for biped robot, 50configuration variables, 51coordinate transformation, 50feedback law, 52Jacobian matrix, 50valid coordinate transformation, 53zero dynamics, 52–54

single support phase, 46–48

continuous time-invariant feedbacklaw, 47

decoupling matrix, 47invertibility of decoupling matrix, 49zero dynamics, 48

for stance and flight phases, 120, 175event-based update law, 124, 125flight phase control law, 122–124stance phase control law, 121, 122

Controllers, 206event-based, 202jth phase, 206open-loop, 204

COP (See Center of pressure (COP))Coriolis matrix, 141, 162, 163, 195, 229, 230CPG (See Central pattern generators (CPG))

Decoupling matrix, 180, 206–208, 240invertibility, 178, 199

Degrees of freedom (DOF), 1Design

cubic splines to compute, 67feasible trajectory of hybrid model of

walking, 63–67hybrid invariance, 61, 62of HZD containing prespecified periodic

solution, 60open-loop control input, 62, 63of output functions, 60, 61validity of transition model, 67

Directionangles in clockwise, 37, 231normal forces at leg ends, 3

Discrete-time system, 33, 126DLQR design method, 189Double support phase, 2Dynamics

double support phase, 40–43applying principal of virtual work, 41constrained, 41validity of model, 43

of flight phase, 37, 38conservation of angular momentum, 38cyclic variables, 38

hybrid zero, 12of single support phase, 39, 40

angular momentum balance, 40validity of model, 40

two-dimensional zero, 36

INDEX 251

Effectevent-based update laws on angular

momentum, 92of impact with ground, 133impulse, 5, 7, 18, 19, 58, 96, 159, 187,

190, 191Energy

control input over single support, 12,221

potential, 139, 162total kinetic, 38, 40, 139, 238

EquationBernoulli, 54closed-loop augmented system, 127constraints in, 142Coriolis matrices in, 100decompose dynamical, 169discrete-time system, 126Euler–Lagrange, 209, 239feedback law, 52, 124flight phase, 38Hamilton–Jacobi–Bellman, 116, 147,

148holonomic outputs, 67impulsive system, 200, 206Lagrange, 139linearization of discrete-time system, 70,

131, 216, 217matrix, 73, 99, 140nonlinear, 107, 109, 112, 138open-loop hybrid model of walking, 47,

71, 128, 196, 197partially feedback linearized, 72premultiplying equation, 238principal of virtual work, 41quadratic, 124second-order, 98, 162static update law, 131symmetric gait conditions, 169third-degree, 111transition maps in, 168variational, 23

Error function, 116, 149Event-based control law, 176, 179Event-based controllers, 209

continuous-time controllers, continuity,212, 213

hybrid invariance, 209–212stabilization problem, simplification, 213

Event-based update laws, 175, 183, 188,212, 213

Exponential stability, 187 (See also Stability)behavior, 228

invariance, 224orbit, 215

Feasible periodic orbit, 167, 196Feedback law, 180Finite-dimensional nonlinear optimization

problem, 197Finite-time attractiveness, 24–27Flight phase, 96

controllerparameter vector, 179

reconfiguration algorithmadmissible open-loop control inputs for

system, 104angular momentum of mechanical

system, 100behavior of solutions for optimization

problems, 107–112boundary conditions on configuration

variables, 100–103control input for dynamical system, 100determination of reachable set, 103, 104first and second time derivatives of, 118fundamental assumption, 100infeasible cases of optimization

problems, 112, 113matrix equation, 99, 100online reconfiguration algorithm, 117,

118for landing, 118–120

planar multilink systems composed oflinks, 118

reachable set from origin, 104–107,113–117

solutions of minimization problem, 112zero dynamics, 180

Footflat-footed walking, 133monopedal mechanism, 10rotation, 3

indicator point, 2three-link, three-actuator 3D monopedal

robot with point, 161two-actuator planar mechanism, 8velocity of, 79

252 INDEX

Forceground reaction, 40, 41, 44, 56, 57, 81,

163, 168, 174, 219horizontal, 84, 91vertical, 84, 91

Friction, 10, 129, 195cone, 3static, 40, 131

Gait, 200anthropomorphic, 167, 200bipedal, 71, 133desired gait statistics, 80dynamically stable, 2real-time, 4statically stable, 2statistics, 80symmetric, 10, 169, 170

periodic, 168walking, 46

Gear reduction, 172Gradient-based update law, 228

Height, 4, 164, 172of COM at beginning, 124leg end during flight phases, 168of swing leg end, 46vertical, 46

Hybrid invariance, 11, 183, 209, 210geometric description, 183, 184for hybrid systems, 24for impulsive systems, 24stabilizing update laws, 186

Hybrid model, 13asymptotic stability, 15autonomous, 13continuously differentiable, 14defined, 13, 14exponential stability, 15, 16solutions of, 14, 15stability, 15of walking, 45, 46

Hybrid restriction dynamics, 25, 26Hybrid zero dynamics (HZD), 5–8, 35, 54

analysisin double support phase, 57in single support phase, 55, 56

continuously differentiable, 59with continuous-time update laws, 12

exponentially stable periodic orbits, 59with impulse effects, 58nontrivial, 90Poincare return map, 187in running with online motion planning

algorithm, 8, 9stability of periodic orbits, 59stabilization, 125–127transition condition, 56upper bound, 56, 57in walking

with double support phase, 7, 8with passive knees, 11, 12

zero output function, 59, 60HZD (See Hybrid zero dynamics (HZD))

Impact model, 43, 78nonsingular impact, 44, 45validity, 45, 165, 196

Impact switching hypersurface, 164, 195Impact update laws, 184Implicit function theorem, 96Impulse effects, 159Impulsive ground reaction forces, 174Inequality constraints, 199Inertia tensor, 162Input–output linearization, 176

Jacobian matrices, 22, 23, 25, 215, 226Joints

angles, 8body, 172hip, 5, 11, 37, 81–83, 194, 198, 200, 205knee, 10, 37, 81–83motions, 96rotational, 10, 137, 138torso, 80–82, 93

Kinematicsbased joint space controller, 4differential kinematic inversion problem,

74planar open kinematic chains (See Planar

open kinematic chain)

Lagrangianflight phase, 162mechanical system, 139phase zero dynamics, 86

INDEX 253

single support phase, 48, 55, 208stance phase, 164

Landingin fixed configuration, 95, 96, 100, 118,

122, 124, 157takeoff and, 8

Legangle of virtual, 161end acceleration, 165ground reaction forces, 91robots (See Robots)during single support phase, 79stance, 39, 44, 49, 55, 56, 60, 63, 79, 80,

91, 97, 195, 196, 201, 204, 219swing, 39, 46, 73, 91, 195, 197

Cartesian position of, 195virtual, 37, 47, 97, 121, 161, 162, 194, 195

Limb coordination, 1Links

to achieve dynamic walking, 1body, 5femur, 50, 194five-link bipedal robot, 7, 78rigid, 10, 118, 127, 138, 160three-link biped robot, 5, 8three-link monoped robot, 127, 161three-link, three-actuator, 10tibia, 47, 160torso, 160–162, 194

Lipschitz constants, 21Locomotion

bipedal, 35hybrid nature of, 2legged, application to, 227, 228open-loop control inputs, 83phases of biped walker motion, 37

MABEL, 7Mass-inertia matrix, 38, 98, 139, 163, 177,

195MATLAB, 76, 80, 172, 200

dlqr function, 91, 132, 217fmincon function, 76, 80, 90, 128, 172,

200Matrix

centrifugal, 162Coriolis, 141, 162, 163, 195, 229, 230Jacobian, 22, 23, 50, 189, 215, 226mass-inertia, 38, 98, 139, 163, 177, 195

rotation, 160, 161, 239symmetric, 240

Mechanical modelbiped walker, 37monopedal runner, 97planar bipedal robot, 194planar open kinematic chain, 138

Modelclosed-loop hybrid, 91closed-loop impulsive model of walking,

217mechanical (See Mechanical model)planar impact, 164

Momentumangular, 38, 40, 49, 81, 95, 96, 100, 102,

119, 120, 122, 123, 125, 138, 140,141

conjugate, 180, 208transfer, 82, 84, 88, 92

effect of event-based update laws, 92zero moment point, 2

Monopedal runner, 97dynamics of flight phase, 97, 98dynamics of stance phase, 98, 99mechanical model, 97–99open-loop hybrid model of running, 99

3D Monopedal running, 159event-based update laws, for hybrid

invariance, 181–186flight phase, regular parameter vector,

182hybrid invariance, 183, 184impact update laws, 185, 186stance phase, regular parameter vector,

181takeoff update laws, 184, 185

numerical example, 172–174open-loop hybrid model, 159, 160

Coriolis and centrifugal matrix, 163flight phase, dynamics, 162, 163hybrid model, 166, 167impact model, validity, 165mass-inertia matrix, 163stance phase, validity, 164stance variable, cyclic variable, 164takeoff model, validity, 165transition maps, 164, 165

period-one solution designopen-loop model, 167–172

254 INDEX

3D Monopedal running (Continued)simulation results, 189–191stabilization problem, 186–188

exponential stability, 187, 188time-invariant controllers for walking,

159within-stride controllers, 175

flight phase control law, 178–181stance phase control law, 175–178

Monoped robot, 95, 97, 99, 100, 128, 159,229

angular momentum of, 122physical parameters, 128state trajectories corresponding to,

132three-link, 127

Motion planning algorithm, 71, 150–152,159, 167, 170, 179, 197–200, 199,206

Bezier polynomial of time, 198cost function, 200for double support phase, 73–75equality constraints, 199finite-dimensional nonlinear optimization

problem, 197to generate

continuous joint motions, 140–143continuously differentiable joint

motions, 152–158determining reachable set from origin,

143–150hybrid model of running, 159inequality constraints, 199numerical example, 77–82, 172, 200–202for single support phase, 72, 73

Nominal holonomic function, 178, 179Numerical nonlinear optimization problem,

96

Open-loop augmented system, 7Open-loop control, 174, 237Ordinary differential equation (ODE),

198Overactuation, 2, 36

Parameterized feedback law, 97Parameterized holonomic output function,

205

Parameter matricesfor the single and double support phases,

68third columns, 174

Parametric flight phase zero dynamics, 180,181

Parametric stance phase zero dynamics, 177,178

Pathof joint angles, 102joint, during stance, and flight phases, 128of swing leg end in sagittal plane, 201,

204Periodic orbits

feasible, 167mechanical system, 171motion planning algorithm, 171nominal value, 176open-loop hybrid model, 171stabilization, 193

walking with passive knees, 193Pfaffian constraint, 4Phase decoupling matrix

invertibility of, 240Planar bipedal robot (See Bipedal robot)Planar open kinematic chain, 138

block diagonal form of mass-inertiamatrix, 139

body configuration space, 139equation, 140mechanical model, 138–140planar multilink system, 138

Planeslateral, 4phase–plane plots, 133, 218sagittal, 4, 8, 95, 181, 201, 204

Poincare return map, 5–7, 226for hybrid systems, 16, 17for impulsive system, 17linearized closed-loop, 190method of, 18restricted, 58, 87, 89, 97, 126, 160

defined, 58geometric description of, 27

Poincare section, hybrid systems, 175Point

COP, ZMP, and FRI, 2, 3equilibrium, 18, 19, 28, 33, 59, 71, 96,

126, 127, 175

INDEX 255

fixed, 131passive, 11, 193zero moment, 2

RABBIT, 6, 77, 80Reconfiguration problem, 9Reduced-order hybrid model, 215Robots, 35

bipedal, 7, 8, 10, 35–37, 50, 75, 78, 92,96, 193, 231

center of mass (COM), 2manipulators, 1monopedal, 95, 97, 99, 128, 159,

229physical parameters, 172

point foot, three-actuator 3D monopedal,161

Rotation matrix, 160, 161, 239

Scalar function, 178Scalar holonomic quantity, 193Simulation

of closed-loop hybrid system, 217–219results, 81–93

trajectory for, 4Stability

analysisof desired periodic trajectory, 130using fourth-degree Bezier polynomial,

130asymptotic, 29, 82behavior of transversal periodic orbit,

23of desired orbit, 11equivalence of stability behavior, 19event-based update laws, 92low-dimensional, analysis, 23–28periodic orbit, 67–71Poincare section, 12policy, 31–33problem, 28–33

of desired orbit, 95properties of desired periodic orbit, 97

Stabilization problem, 213–217closed-loop discrete-time system, 216continuous-time controllers, 215event-based updating policy, 216exponential stabilization of orbit, 215parameterized Poincare return map, 214

Poincare section, 214simplification, 213stabilizing parameters assignment, 216,

217Stance phase, 161

controller, 9, 97model, 164regular parameter vector, 181zero dynamics, 6, 10, 177

Static event-based laws, 184Symmetric matrix, 240

Takeoff model, validity, 165Takeoff switching hypersurface, 165Takeoff update law, 184, 185Terrains

irregular, 5rough, 1, 5uneven, 4

Time-invariant feedback scheme, 159Time-to-impact function, 225Time-varying linear systems, 96Torques, 127, 160, 162

actuator, 38, 139computed, 5hip, 212joint, 90vector of actuator, 38, 98, 139

Torso frames, 160, 162Torso link, angular velocity vector, 162Trajectories, 14, 20, 23, 24, 36, 129

modified reference, 9nominal, 100–102optimal, 107, 109, 110, 172periodic, 36, 120, 125, 128, 133

statistics, 130phase–plane plots and projection, 133,

135sensitivity matrices, 22, 224, 225state, phase portraits, 191tracking of time, 4, 5

Transitionsdiscrete, 95double support phase to single support

phase, 45Transversality, 17, 197Two-actuator planar bipedal mechanism, 194

Unilateral constraints, 3, 7, 8, 36, 196, 197

256 INDEX

Validitydouble support phase model, 43of hypotheses HPO3–HPO6 for optimal

periodic motion, 80impact model, 45, 165, 196single support, 195

phase model, 40stance phase, 164transition model from DS to SS, 67

Variablesconfiguration, 88, 97, 99, 100costate, 143, 144, 154cyclic, 38, 163velocity, 89

Vectors, 77of actuator torques, 38coordinates, 195fields, 15, 25, 32flight phase closed-loop vector, 180of generalized velocities, 6, 9gravity, 40inequality constraint, 172multiplier, 105stance phase closed-loop, 177state, 99velocity, 73, 79, 92, 218

Velocitiesabsolute angular, 79angular position and, 80, 174

angular velocity vector, 162continuous, 213discontinuities in, 129, 132during double impact, 79generalized, 44horizontal, 53swing, 81during transition, 211variables, 89vertical, 87

Virtual time, 10, 102, 103, 105, 138, 142,144

Walking, open-loop model, 194impact map, 195, 196impulsive model, 196, 197planar bipedal robot, mechanical model,

194, 195single support phase, dynamics, 195

Within-stride controllers, 9, 11, 36, 68, 97,126, 160, 175–181

Within-stride feedback laws, 6, 7,182

Within-stride switching hypersurfaces, 11,12, 193, 202, 209, 214

Within-stride transitions, 11, 204, 209, 212,213, 216, 217

Zero moment point (ZMP), 2–5, 133

IEEE PRESS SERIES ON SYSTEMS SCIENCE AND ENGINEERING

Editor:MengChu Zhou, New Jersey Institute of Technology and Tongji University

Co-Editors:Han-Xiong Li, City University of Hong-KongMargot Weijnen, Delft University of Technology

The focus of this series is to introduce the advances in theory and applications ofsystems science and engineering to industrial practitioners, researchers, andstudents. This series seeks to foster system-of-systems multidisciplinary theory andtools to satisfy the needs of the industrial and academic areas to model, analyze,design, optimize and operate increasingly complex man-made systems rangingfrom control systems, computer systems, discrete event systems, informationsystems, networked systems, production systems, robotic systems, service systems,and transportation systems to Internet, sensor networks, smart grid, social network,sustainable infrastructure, and systems biology.

1. Reinforcement and Systemic Machine Learning for Decision MakingParag Kulkarni

2. Remote Sensing and Actuation Using Unmanned VehiclesHaiyang Chao, YangQuan Chen

3. Hybrid Control and Motion Planning of Dynamical Legged LocomotionNasser Sadati, Guy A. Dumont, Kaveh Akbari Hamed, and William A. Gruver

Forthcoming Titles:

Operator-based Nonlinear Control Systems Design and ApplicationsMingcong Deng

Contemporary Issues in Systems Science and EngineeringMengchu Zhou, Han-Xiong Li and Margot Weijnen

Design of Business and Scientific Workflows: A Web Service-Oriented ApproachMengchu Zhou and Wei Tan

ieee systems science and engineering_ieee digital and mobile-cp@2011-05-06T15;32;42.qxd 3/19/2012 5:48 PM Page 1


Hybrid Control and Motion Planning of Dynamical Legged Locomotion

Documents

Transcript of Hybrid Control and Motion Planning of Dynamical Legged Locomotion