A Robust Pushing Skill For Object Delivery Between Obstacles

Senka Krivic 1, Emre Ugur1,2 and Justus Piater 1

Abstract— Nonprehensile manipulation can play a sig-nificant role in complex robotic scenarios, especially formaneuvering non-graspable objects. A big challenge is toconstruct a robust skill for pushing highly-diverse objects.We present a strategy for pushing unknown objects thatdiffer widely in their properties. For this purpose weintroduce the concept of a pushing corridor for clutteredenvironments that leaves the robot sufficient space forcorrective motions. We propose a reactive manipulationskill for pushing objects along this collision-free corridor.The motion of the robot is generated and adapted on the flybased on the observed reactions of the object in responseto pushing actions. Our results show that the robot is ableto successfully push objects in various complex scenarios.

I. INTRODUCTION

Pushing is a commonly used activity by humansin different everyday activities. We push things outof the way, close doors, bring and relocate objects bypushing them. Introducing a similar skill for robotswould enable new capabilities and increase dexterityof a mobile robot in complex scenarios. For example, ifa mobile robot is not equipped with an arm, or alreadyholding an object, the ability to push with the base canbe used for clearing paths or conveying objects. In thispaper we address the problem of delivering an objectto a goal position in cluttered environments.

In various robotic scenarios a robot has to operate inenvironments with scattered objects and tangled heapson the floor, limiting the robot’s freedom to move.Therefore, we introduce a collision-free corridor whichis defined by a path and its surrounding area with afixed width as shown in Fig. 1. To push along such acorridor, we also introduce the concept of a target pointtowards which the robot pushes the object at everyinstant. This point is determined by a cost functionwhich we define in Section IV. This allows cuttingcorners as long as the corridor is not violated by therobot or by the object.

In complex scenarios objects may have diverse, evenanisotropic properties under pushing. For example,while some objects slide freely, other may even flip

*The research leading to these results has received fundingfrom the European Community’s Seventh Framework ProgrammeFP7/2007-2013 (Specific Programme Cooperation, Theme 3, Informa-tion and Communication Technologies) under grant agreement no.610532, Squirrel

1Authors are with the Institute of Computer Science, Universityof Innsbruck, 6020 Innsbruck, Austria. Contact information:senka.krivic@uibk.ac.at,emre.ugur@uibk.ac.at,justus.piater@uibk.ac.at

2 The author is with the Department of Computer Engineering,Bogazici University, Istanbul, Turkey.

Fig. 1. Pushing in a complex environment. The collision-free path(red line) is obtained from a path planner, and defines the corridor(yellow). The current target point is marked with green dot on thepath. (This and other figures are best viewed in color).

over. Generally, it is not straightforward to design aphysical model describing the behavior of an object [1],[2]. Visual features can be non-informative, or even mis-leading, for the description of object behaviors whilepushing [2]. Thus, a robot should react adaptively todifferent robot-object-environment dynamics. Humansare able to push various objects in a desired way bymoving them approximately into designated directionsand changing the hand motion based on the observedbehavior of the object. We aim to incorporate a similarconcept into the push-manipulation skill by adaptingrobot motion based on observations of the behavior ofthe object.

Most of the time, particularly along smooth trajecto-ries, the robot should be able to push the object usingsmooth corrective motions while keeping contact withthe object. Sometimes, however, especially at sharpturns, the robot will need to relocate by leaving theobject and moving towards a new angle of attack.Thus, at every time step, we define robot movementdirections both for smooth pushing and for relocating.The final motion of the robot is a result of blendingthese two motion vectors weighted by their respectiveactivation functions. These functions are adapted overtime based on observations from the robot-object inter-action. At onset of the pushing manipulation, the robotassumes it can push the object if it is behind it withrespect to the target point. Since properties of the objectand its behaviour are unknown, we also introduce anadditional feedback component to react to unexpectedmovements of the object.

II. RELATED WORK

Nonprehensile manipulation raised interest in vari-ous scientific areas. Early work on pushing by Masonand Lynch [3] presented a theoretical model of thedynamics of pushing with a robotic manipulator. Oneof the first systems that took into account vision feed-back for building a forward model of a planar objectwas created by Salganicoff et al. [4]. In later work, theaction of pushing was often used for learning dynamicmodels, behaviors and object specific properties byobserving results of the push [5], [2].

Ruiz-Ugalde et al. [2], [6] built a mathematical modelfor planar sliding motion. They executed a pushingbehavior by determining the static and kinetic frictioncoefficients between the robot hand, the table anda rectangular object. Scholz and Stilman [7] learnedobject-specific dynamic models for a set of objectsthrough experience. A robot learns Gaussian models ofthe pose displacement which are then used for the pushselection. In a similar tabletop scenario Omrcen et al. [8]used a neural network to learn the pushing dynamicsfor flat objects. The learned models were specific toindividual objects, and generalization to similar objectswas not examined.

Several methods considered the problem of learningcontact locations for successful pushes [9]. Most ofthem focus on extracting the shape of the object anddetermining contact locations that are good for push-ing. Early work by Salganicoff et al. [4] presents an un-supervised online method for pushing in an obstacle-free environment by controlling only one degree offreedom. Their method selects each pushing directionusing a nearest-neighbor approach. The point of contactis restricted to a single rotational contact point. With asimilar approach, Walker and Salisbury [10] learn thesupport friction by mapping pushes to motion. Theyperform experiments by pushing the object from sev-eral contact points. Using local regression they modelthe support friction within a two-dimensional envi-ronment. Hermans et al. [11] develop an offline, data-driven method for predicting effective contact locationsfor pushing previously-unseen objects. They also de-velop a method for extracting the 2D shape of an objectand generate points for performing pushing tests.

For the problem of delivery of the object from oneposition to another some scientists [12] focused ondetermining appropriate pushing actions and devel-oping a push planner which will complete the taskusing these actions. Hermans at al. [11] were pushingtowards the centroid of the object from a given pointusing the feedback controller. Input signals includedthe object-target displacement and the displacementof the robot from the pushing direction through theobject’s centroid.

Igarashi et al. [13] presented a novel pushing al-gorithm for circularly-shaped objects. Their controller

consists of components for pushing and orbiting. Theyassigned two weighting functions to these movementswhere the vector of the robot movement resembles adipole field (in analogy to physics). Although this is anelegant solution, it has many disadvantages in practicesuch as long trajectories while moving to correct po-sitions, and excessive orbiting caused by even smalldisplacements of the robot from the pushing direction.

Inspired by this approach, we create a feedback con-troller that is also based on the displacement angle ofthe robot from the desired object movement direction.However, in our approach, the robot learns when torelocate and when to push. In addition, we introduce acomponent to react to the changes in object movementbehavior to overcome the strong assumptions aboutideal properties of the object.

The state-of-the-art method presented by Mericli etal. [14] addresses the problem of pushing complex,massive objects on caster wheels with a mobile base incluttered environments. Based on experience from self-exploration or demonstration via joy-sticking, they cre-ate experimental models used for planning the pushingaction using an RRT-based method.

Among the many existing approaches and problemsfor push-manipulation in the literature, ours differs inseveral aspects:• We address single-contact pushing of objects of

various sizes, weight, mass and friction distribu-tions.

• We introduce the notion of a pushing corridorwithin which the robot can interact with objectswithout having to take care of obstacle avoidance.

• The presented method is reactive and requiresneither time-expensive learning phases nor priorknowledge.

• Our strategy does not make strong assumptionsabout the way the object reacts to pushes; it adaptsthe pushing motion based on current observationson the fly.

III. PROBLEM DESCRIPTION

Our case study involves an untidy room of a childwhich has to be tidied up. A robot needs to be ableto push various objects like toys, boxes and smallpieces of furniture to designated locations. In orderto create a robust skill that can be applied directlyin different scenarios, we decompose the problem intotwo subproblems:• Define a pushing corridor and choose a target

point for pushing the object at each time stepallowing collision-free pushes.

• Determine a robust method for pushing objectstowards a target point.

We assign a two-dimensional static frame g to theenvironment and a local frame r to the robot. Weassume that the global position R of the robot and

the global position O of an object are available at eachtime step t and are given by R(t) =

[xgR(t) ygR(t)

]ᵀand O(t) =

[xgO(t) ygO(t)

]ᵀ, respectively. In addition,we assume that a map of the environment is availablefor path planing. The size of the robot is determinedby its radius rR, and the system is able to approximatethe radius rO of the bounding circle around the pushedobject.

IV. PUSHING CORRIDOR

We define a pushing corridor C (Q, w) as an areaaround the collision-free path Q from the start to thegoal object pose given by a set of waypoints

Q = {Q1, Q2, Q3, ..., QN},

with the width w of the corridor which is defined astwice the distance from the path to the closest obstacle.A robot is allowed to move and push objects withinthis corridor ensuring collision avoidance.

At each time step, the robot pushes along the cor-ridor by choosing the next target point. This allowsstraight-line pushes while constraints of the corridorare not violated. This procedure can be compared tothe concept of the lookahead distance in navigation[15] which tells the robot how far along the pathit should look from the current location to computethe velocity commands. In a pushing scenario, shortlookahead distance ensures better path tracking butmight lead to oscillations in object movements, andto long robot trajectories. By contrast, large lookaheaddistances enable short object trajectories but raise thepossibility of corridor violations. Thus, to determine atarget point we define a cost function that ensures longlookahead distances for pushing but also penalizesapproaching corridor margins.

Let J(O,Q) be a cost function defined as

J(O(t), Q) =1

d(O(t), Q)+ ζdmax

where (as shown in the Fig. 3):• O(t) is the current object position, Q is the way-

point and t is the time.• d(·, ·) denotes the Euclidean distance between two

points,• dmax is the maximum distance between the path

Q and the current push-line OT , and

ygwTxg

OR

dmaxpush-lineP

Q

Fig. 3. Determining the next target point for pushing. R and Ocorrespond to robot and object positions, the red line is the path, Tis the current target point, and P denotes the desired position of therobot. Q is the path given by set of waypoints.

• ζ is the inflexibility factor of a pushing corridorgiven by the ratio of the corridor width and theminimum width needed for both robot and objectto fit between the path Q and the closest obstacle:

ζ =

{w

4(rR+rO) , if w4(rR+rO) ≤ 1

1, otherwise.

Then a target point T (t) can be determined at each timestep as

T (t) = argminQ∈Q

J(O(t), Q) (1)

s.t . w2 − ζ (dmax + 2rR) ≥ 0 (2)w2 − ζ(dmin + rR) ≥ 0

where dmin is the minimal distance between the pathQ and an ideal position of the robot for the currentpushing direction denoted by P . This is a point on thepush-line with distance rR + rO to the object.

The conditions (2) ensure that neither the robot northe object violate the corridor in case of high path cur-vatures or the relocation of the robot. It can be noticedthat ζ < 1 defines a corridor with less flexibility for therobot movement since the robot cannot fit between thecorridor edge and the object. This factor allows mildshortcuts even in case of corridors so narrow that itsnon-violation cannot be guaranteed (Fig. 2).

Observe that our method can be extended to environ-ments with moving obstacles by checking the validityof a corridor at each time step and recalculating it ifrequired. Such a dynamic change of the corridor doesnot affect the pushing procedure since the target pointis continually updated.

ζ= 1.0

w

w = 1.6 ζ= 0.9091w = 1.2 ζ= 0.7576w = 1.0 ζ= 1.0w = 1.0Fig. 2. Examples of the procedure for determining target points. Parameters were set as following: rR = 0.23, rO = 0.10. The path isdefined as y = sin (x) for x = [0, 2π]. Two examples on the right side are given for the same corridor with and without effect of ζ.

T(t -Δt) O(t -Δt)push-lineO(t)T

OR push-linexOyO

α γ

Fig. 4. The angle α between the robot, the object and the currentpush-target is used to determine the robot’s movement direction(left). The angle of object movement displacement γ (right).

V. REACTIVE PUSHING

In this section we introduce a reactive controllerfor pushing the object towards a target point. Thetarget point is calculated at each time step based onthe optimization routine introduced in the previoussection. Therefore it is necessary that the robot adaptsits movements to new pushing directions as well as tothe behavior of an object. Thus two robot behaviors areneeded: pushing and relocating.

Inspired by the dipole field method by Igarashi etal. [13] we define directions of the robot movement forboth behaviors. The final robot movement direction isdetermined by blending these two. They are definedby the angle of robot displacement from the desiredobject movement direction denoted by α = ]TOR inFig. 4. To simplify the description, let us assign a localframe to the object o where the x axis coincides with thedesired object motion defined by line OT . The pushingdirection is determined in the object local frame as

vopush = sgn(cosα)

[cosαsinα

](3)

If the robot is in a position from which it cannot di-rectly push the object, it should relocate to the positionfrom which the push can be achieved. This relocationdirection is given by

vorelocate = sgn(sinα)

[− sinαcosα

](4)

Examples for such movement directions are shown inthe Fig. 5.

A. Activating the movement actionsTo achieve smooth movements of the robot we

blend these two behaviors. However, they have to beweighted to achieve successful object deliveries. If therobot is behind the object (α ≈ π), it should be ableto push it towards the target. Thus, we build in thisassumption by assigning the activation function ψpush

as a weight for the pushing behavior. The robot needsto learn which values of α are suitable for pushing. Forthat purpose, we define the push activation functionas a Gaussian kernel over the angles α from whichpushing towards the target point is achievable,

ψpush(α) = exp

(− (α− µα)

2

2σ2α

),

vpushvrelocate

vrelocatevrelocatevpush

vpushFig. 5. Examples of push and relocate non-weighted directionsdefined in (3) and (4) for different locations of the robot, the objectand the target point.

where µα is the expected value of the robot displace-ment α for which the object moves towards the target,and σα is its variance. At the onset of a pushing action,the initial values are set to µα = π and σα = π

3 .These values are then incrementally updated duringthe execution of pushing based on observations of theobject behavior.

Desired values of the α for the pushing behaviorare those that result in the object moving along thepush-line OT . Thus, we define the error of the objectmovement

γ (t) = ]T (t−∆t)O (t−∆t)O (t) (5)

as the angle between the desired push-line and the re-sulting position of the object. Therefore the distributionof robot displacements α resulting in successful pushescan be described by the conditional probability densityp(α|γ̇ < 0) ∝ ψpush(α), where γ̇ is the change of theerror.

The relocation activation function is simply de-fined in terms of the push activation function,ψrelocate(α) = 1− ψpush(α). While relocation is active,the pushing component attracts the robot to the object,not allowing large distances between them.

B. Compensating displacements of the object movementWhen the robot is pushing an object, it assumes

the object moves on the push-line (γ ≈ 0). However,this is not necessarily the case due to different massdistribution and the dynamics of objects. Thus, therobot tries to capture the model of the object behaviourby learning the expected value of the object movementerror γ. We introduce an additional feedback term tocompensate for constant deviations of the observedfrom the expected object movement:

vocompensate =

[− sin γ− cos γ

](µγ − γ) , (6)

where µγ is the mean value of experienced displace-ments, and all angles are expressed in radians. Largervalues of µγ may indicate objects which do not havequasi-static properties or uniform mass/friction distri-butions. To produce more cautious movements of therobot in such cases, we modulate the velocity of therobot movement by this value:

V =Vmax

1 + |µγ |. (7)

Fig. 6. A pushing trial in the real environment. Parameters were rR = 0.23, rO = 0.12, w = 1.6. The path was given as y = sinx forx ∈ [0, 2π]. The values are given in [m]. Time of delivery 58.70s

C. Robot motionThe direction of the robot motion is now defined as

vomotion = ψpush

(vopush + vocompensate

)+ ψrelocatev

orelocate

The control signal urR is defined by a scalar term V (7)representing the desired velocity magnitude, and a unitvector in the desired robot movement direction vomotion:

urR = RorVvomotion

‖vomotion‖where Ror is the rotation matrix from the object localframe o to the frame r of the robot.

VI. EXPERIMENTAL RESULTS

To evaluate the effectiveness of the proposed method,several tests were done both in simulation and ina real environment. The tests were performed withsynthetic corridors of straight-line and sine-wave shape(5 times with each object for a particular path), aswell as with corridors obtained automatically from anavigation system to demonstrate the robustness of thestrategy. We used a holonomic robot with a circularbase, the Robotino1 by Festo, both in simulation andreal environments. The simulator used is Gazebo, andthe scenario is the one shown in Fig. 1. We performedtests with objects of different properties as shown inFig. 7, and in environments with surfaces of differentfriction properties (carpet as in Fig. 6 and tiles as inFig. 7).

Table I shows success rates of pushing tests per-formed by the real robot. The results show that methodis robust to the width change of the corridor and theobject shape and size. However, in most cases recordedas failures for violating the corridor, the robot stilldelivered the object to the target. In straight-line testsmost of the failures occurred at the end of pushingprocedures where pushing actions were biased towardspresented initial values which did not change muchdue to a simple path. In sine-wave shape tests, robotwas loosing the object the most at the places of high

1http://www.festo-didactic.com/int-en/services/robotino

0 1 2 3 4 5 60

0.2

0.4

0.6

0.8

1t1t2t3t4

Push activation function

θ[rad]Fig. 7. Left: The set of objects used for pushing experiments containsobjects of various shapes, sizes, friction properties, mass distributionsand quasi-static properties. Right: Example of the change of the pushactivation function over time (t1 < t2 < t3 < t4). It can be noticedthat the variance of α was decreasing and increasing again.

TABLE ISUCCESS RATES OF PUSHING TESTS IN REAL ENVIRONMENT

path w = 1.8 [m] w = 1.6 [m] w = 1.4 [m]straight line

length: 3.14 [m] 100% 92% 88%sine wave

length: 6.28 [m] 96% 84% 76%

curvatures where it was not able to push object towardsthe push line due to unexpected object behavior. Linepushes lasted between 16 [s] and 22 [s], and sine pushesbetween 41 [s] and 67 [s] where the maximum speed ofthe robot was set to 0.2 [m/s].

Examination of the robot trajectories reveals that therobot simplifies the pushing task by cutting corners asallowed by the corridor. One such sample execution isprovided in Fig. 6. We analysed this effect with differ-ent corridor widths w. In the case of wide corridors,the robot tries to push towards the final goal at onset(illustrated in Fig. 8) but becomes more conservative assoon as deviations from desired direction are observed.When pushing towards the final goal is possible again,the robot relocates.

In the case of narrow corridors, the ζ parameterrelaxes the corridor non-violation condition of thepushing strategy (2). The robot is allowed to violate thecorridor at most for the distance (1−ζ)

ζ

w

2(see Eq.(2)).

We did multiple tests also with narrow corridors withw = 1.0. The object delivery was achieved with 68%success.An example is shown in Fig. 8.

0 1 2 3-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

0 1 2 3-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

Fig. 8. Left: Result of pushing a small object of radius rO = 0.06 ina very wide corridor of width w = 3.4. Right: Result of pushing thesame object of in a very narrow corridor of width w = 1.0, ζ = 0.862.Both results were produced by the real robot.

0 1 2 3-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

0 1 2 3-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

0 20 40 60 800

0.05

0.1

0.15

t[s]

v[m/s]

robotobject

intermediate target-point

Fig. 9. Results of pushing with (top left) and without (top right) thecompensation term (6). The bottom graph demonstrates the changeof velocity (7); Vmax = 0.15[m/s].

The details of the pushing behavior depend on theobservations of the object behavior, especially at thebeginning of a push. The parameters µα, σα and µγ canbe updated with incorrect/noisy perception or insuffi-cient experience, which is the principal cause of thefailures observed. During the pushing procedure therobot might become increasingly conservative (smallervalues of σα) which causes increased relocating behav-ior, but due to new experience it increases the varianceof α, generally settling at smooth and robust pushingbehavior over time as can be seen in Fig. 7 (right).

We analyzed the effect of the compensation termand velocity modulation by performing tests with andwithout them. The results are shown in Fig. 9. It canbe noticed that as soon as deviation of the objectmovement direction occurs, the robot slows down andcorrects its movement direction by the compensationterm.

VII. CONCLUSION

We proposed a strategy for object delivery by single-contact pushing with a robot base in cluttered environ-ments. For this purpose we introduced the notion of apushing corridor which enables following a collision-free path with constraints but in a flexible way. Insteadof tuning control parameters with respect to the ob-ject class or the shape, we proposed identifying theobject behavior by learning. The presented algorithmis computationally cheap and enables real-time pushmanipulation. It is purely reactive and does not requireprior models or a training phase. Our results show itis robust for our set of objects of various shapes indifferent environments.

REFERENCES

[1] Matthew T. Mason. Mechanics and planning of manipulatorpushing operations. The International Journal of Robotics Research,5(3):53–71, 1986.

[2] F. Ruiz-Ugalde, G. Cheng, and M. Beetz. Fast adaptation foreffect-aware pushing. In Humanoid Robots (Humanoids), 2011 11thIEEE-RAS International Conference on, pages 614–621, Oct 2011.

[3] Kevin M. Lynch and Matthew T. Mason. Stable pushing: Me-chanics, controllability, and planning. The International Journalof Robotics Research, 15(6):533–556, 1996.

[4] Marcos Salganicoff, Giorgio Metta, Andrea Oddera, and GiulioSandini. A vision-based learning method for pushing manip-ulation. In In AAAI Fall Symposium Series: Machine Learning inVision: What Why and, 1993.

[5] D. Katz and O. Brock. Manipulating articulated objects withinteractive perception. In Robotics and Automation, 2008. ICRA2008. IEEE International Conference on, pages 272–277, May 2008.

[6] F. Ruiz-Ugalde, G. Cheng, and M. Beetz. Prediction of actionoutcomes using an object model. In Intelligent Robots and Systems(IROS), 2010 IEEE/RSJ International Conference on, pages 1708–1713, Oct 2010.

[7] J. Scholz and M. Stilman. Combining motion planning andoptimization for flexible robot manipulation. In Humanoid Robots(Humanoids), 2010 10th IEEE-RAS International Conference on,pages 80–85, Dec 2010.

[8] D. Omrcen, C. Boge, T. Asfour, A. Ude, and R. Dillmann.Autonomous acquisition of pushing actions to support objectgrasping with a humanoid robot. In Humanoid Robots, 2009.Humanoids 2009. 9th IEEE-RAS International Conference on, pages277–283, Dec 2009.

[9] M. Kopicki, S. Zurek, R. Stolkin, T. Morwald, and J. Wyatt.Learning to predict how rigid objects behave under simplemanipulation. In Robotics and Automation (ICRA), 2011 IEEEInternational Conference on, pages 5722–5729, May 2011.

[10] S. Walker and J.K. Salisbury. Pushing using learned manipula-tion maps. In Robotics and Automation, 2008. ICRA 2008. IEEEInternational Conference on, pages 3808–3813, May 2008.

[11] T. Hermans, Fuxin Li, J.M. Rehg, and A.F. Bobick. Learningcontact locations for pushing and orienting unknown objects. InHumanoid Robots (Humanoids), 2013 13th IEEE-RAS InternationalConference on, pages 435–442, Oct 2013.

[12] Qingguo Li and Shahram Payandeh. Manipulation of convexobjects via two-agent point-contact push. Int. J. Rob. Res.,26(4):377–403, April 2007.

[13] Takeo Igarashi, Yoichi Kamiyama, and Masahiko Inami. Adipole field for object delivery by pushing on a flat surface.In ICRA, pages 5114–5119, 2010.

[14] Tekin A. Mericli, Manuela Veloso, and H. Levent Akin. Achiev-able push-manipulation for complex passive mobile objects us-ing past experience. In Proceedings of the 2013 International Con-ference on Autonomous Agents and Multi-agent Systems, AAMAS’13, pages 71–78, Richland, SC, 2013. International Foundationfor Autonomous Agents and Multiagent Systems.

[15] R Craig Coulter. Implementation of the pure pursuit pathtracking algorithm. Technical report, DTIC Document, 1992.