Vision-Based 3D Motion Reconstruction of Octopus Arm ...

6
Vision-based 3D motion reconstruction of octopus arm swimming and comparison with an 8-arm underwater robot Asimina Kazakidi, Xenophon Zabulis, Dimitris P. Tsakiris Abstract— The octopus uses the arm-swimming behavior pri- marily for escape, defense, or foraging. This mode of locomotion is comprised of two strokes, with the arms opening slowly and closing rapidly, and generally results in considerable propulsive acceleration. In light of the recent development by our group of an octopus-like eight-arm underwater robot, we are interested to analyze the details of the biological arm swimming motion, in order to understand its kinematics. In this paper, we address methodological aspects of the 3D reconstruction process of octopus arm trajectories, based on computer vision, and present the resulting arm swimming movement of a benthic common octopus. The 3D trajectories of all eight octopus arms were tracked and analyzed, providing information about speed, acceleration and arm elongation. The animal’s performance is then used for a direct comparison with our 8-arm robotic swimmer. The data obtained provide new kinematic information about this, relatively unknown, propulsive mode, which can be exploited for multi-functional underwater robots. Index Terms— Biologically-Inspired Robots, Soft Robots, Un- derwater Robots, Octopus, Computer Vision, Kinematics. I. INTRODUCTION Biologically-inspired robots are a class of robots aiming at incorporating features and mechanisms observed in nature, that inspire designs with advanced performances and func- tionalities. Specifically, the design of flexible, multi-function manipulator arms could be used underwater both for object manipulation and for propulsion [1], [2]. The employment of such manipulators on sophisticated underwater robotic vehicles, able to approach the efficiency and adaptability of biological organisms in an aquatic environment, could greatly enhance robotic applications like the inspection of underwa- ter structures and submerged industrial infrastructures, search and rescue operations in the ocean or the exploration and monitoring of marine ecosystems [3], [4]. The octopus is a unique marine animal with outstanding locomotion abilities in unstructured environments and ex- traordinary dexterity and intelligence [5]–[8]. Arm swimming is a particular mode of locomotion for the benthic common octopus (Octopus vulgaris) whenever the animal needs to accelerate rapidly, e.g. at instances of escaping or defense. Although jet swimming is more frequently observed, relying primarily on the siphon, arm swimming has been encountered This work was supported in part by the European Commission and the Hellenic Ministry of Education and Religious Affairs via the ESF- GSRT HYDRO-ROB Project [PE7(281)], the EC/ERDF-GSRT BIOSYS- KRIPIS project [MIS-448301 (2013SE01380036)] and the NSRF-THALES BioLegRob Project [Mis: 379424]. A. Kazakidi, X. Zabulis and D.P. Tsakiris are with the Institute of Computer Science, Foundation for Research and Technology – Hellas (FORTH), N. Plastira 100, Vassilika Vouton, GR-70013, Heraklion, Greece. {kazakidi, zabulis, tsakiris}@ics.forth.gr. extensively both in the wild [9] and in captivity [10], but has never been analyzed in detail. Visually, the motion can be described as a combination of two distinct parts (strokes), one where all 8 arms, initially trailing behind the animal, open outwards at a relatively slow speed (recovery stroke) and another where the arms close inwards at a much higher speed (power stroke). For robotic design purposes, the motion has been approximated by a sculling-like profile [11]. Although this captures the basic motion components, a more quantified kinematic description would reveal new aspects of this unique propulsion mode if implemented in robotic models. We have recently presented a multi-arm underwater robot [11]–[14] that mimics the morphology of the octopus, pos- sessing 8 compliant arms and a passively-compliant web. The robotic model has included detailed information of hydrodynamic results [15]–[18] and is in accordance with relevant elastodynamic investigations of arm muscle acti- vation [19], [20]. The robot has demonstrated forward and turning propulsion with sculling-only gaits, achieving speeds of up to 0.5 body lengths per sec with the web, and maximum propulsive forces of 10.5 N, with a cost of transport as low as 0.62 [12]; it has also been tested for grasping capabilities and suitability in a marine environment [21]. Our aim is to extend these investigations by incorporating more realistic locomotion schemes and advanced soft-robot control strategies. To this end, we describe, in Section II, a methodologi- cal approach for accurately reconstructing 3D octopus arm trajectories, during the arm swimming motion, by utilizing computer vision methods. The process is broadly based on an earlier innovative approach by Yekutieli et al. [22], who addressed this complex problem by exploiting advanced computer vision techniques, leading to a semi-automatic arm segmentation algorithm, which is robust and user-friendly. This algorithm has been extensively modified with new methods being developed for camera calibration and 3D reconstruction, and the addition of several methodological features for further accuracy and robustness [23]. Further- more, the work of Yekutieli et al. [22] pertains to a single arm imaged frontoparallelly, while in our work we extended this approach to 8 arms imaged in 3D. Section III presents a complete series of 3D-reconstructed trajectories of all eight octopus arms during arm swimming motion. Detailed information about individual arms, as well as of the speed and acceleration of the whole body during the motion are calculated and demonstrated. This work gives a good appre- ciation of the complexity of the movement of the biological 2015 IEEE International Conference on Robotics and Automation (ICRA) Washington State Convention Center Seattle, Washington, May 26-30, 2015 978-1-4799-6922-7/15/$31.00 ©2015 IEEE 1178

Transcript of Vision-Based 3D Motion Reconstruction of Octopus Arm ...

Page 1: Vision-Based 3D Motion Reconstruction of Octopus Arm ...

Vision-based 3D motion reconstruction of octopus arm swimmingand comparison with an 8-arm underwater robot

Asimina Kazakidi, Xenophon Zabulis, Dimitris P. Tsakiris

Abstract— The octopus uses the arm-swimming behavior pri-marily for escape, defense, or foraging. This mode of locomotionis comprised of two strokes, with the arms opening slowly andclosing rapidly, and generally results in considerable propulsiveacceleration. In light of the recent development by our group ofan octopus-like eight-arm underwater robot, we are interestedto analyze the details of the biological arm swimming motion,in order to understand its kinematics. In this paper, we addressmethodological aspects of the 3D reconstruction process ofoctopus arm trajectories, based on computer vision, and presentthe resulting arm swimming movement of a benthic commonoctopus. The 3D trajectories of all eight octopus arms weretracked and analyzed, providing information about speed,acceleration and arm elongation. The animal’s performanceis then used for a direct comparison with our 8-arm roboticswimmer. The data obtained provide new kinematic informationabout this, relatively unknown, propulsive mode, which can beexploited for multi-functional underwater robots.

Index Terms— Biologically-Inspired Robots, Soft Robots, Un-derwater Robots, Octopus, Computer Vision, Kinematics.

I. INTRODUCTION

Biologically-inspired robots are a class of robots aiming atincorporating features and mechanisms observed in nature,that inspire designs with advanced performances and func-tionalities. Specifically, the design of flexible, multi-functionmanipulator arms could be used underwater both for objectmanipulation and for propulsion [1], [2]. The employmentof such manipulators on sophisticated underwater roboticvehicles, able to approach the efficiency and adaptability ofbiological organisms in an aquatic environment, could greatlyenhance robotic applications like the inspection of underwa-ter structures and submerged industrial infrastructures, searchand rescue operations in the ocean or the exploration andmonitoring of marine ecosystems [3], [4].

The octopus is a unique marine animal with outstandinglocomotion abilities in unstructured environments and ex-traordinary dexterity and intelligence [5]–[8]. Arm swimmingis a particular mode of locomotion for the benthic commonoctopus (Octopus vulgaris) whenever the animal needs toaccelerate rapidly, e.g. at instances of escaping or defense.Although jet swimming is more frequently observed, relyingprimarily on the siphon, arm swimming has been encountered

This work was supported in part by the European Commission andthe Hellenic Ministry of Education and Religious Affairs via the ESF-GSRT HYDRO-ROB Project [PE7(281)], the EC/ERDF-GSRT BIOSYS-KRIPIS project [MIS-448301 (2013SE01380036)] and the NSRF-THALESBioLegRob Project [Mis: 379424].

A. Kazakidi, X. Zabulis and D.P. Tsakiris are with the Institute ofComputer Science, Foundation for Research and Technology – Hellas(FORTH), N. Plastira 100, Vassilika Vouton, GR-70013, Heraklion, Greece.{kazakidi, zabulis, tsakiris}@ics.forth.gr.

extensively both in the wild [9] and in captivity [10], buthas never been analyzed in detail. Visually, the motioncan be described as a combination of two distinct parts(strokes), one where all 8 arms, initially trailing behind theanimal, open outwards at a relatively slow speed (recoverystroke) and another where the arms close inwards at a muchhigher speed (power stroke). For robotic design purposes,the motion has been approximated by a sculling-like profile[11]. Although this captures the basic motion components,a more quantified kinematic description would reveal newaspects of this unique propulsion mode if implemented inrobotic models.

We have recently presented a multi-arm underwater robot[11]–[14] that mimics the morphology of the octopus, pos-sessing 8 compliant arms and a passively-compliant web.The robotic model has included detailed information ofhydrodynamic results [15]–[18] and is in accordance withrelevant elastodynamic investigations of arm muscle acti-vation [19], [20]. The robot has demonstrated forward andturning propulsion with sculling-only gaits, achieving speedsof up to 0.5 body lengths per sec with the web, and maximumpropulsive forces of 10.5 N, with a cost of transport aslow as 0.62 [12]; it has also been tested for graspingcapabilities and suitability in a marine environment [21]. Ouraim is to extend these investigations by incorporating morerealistic locomotion schemes and advanced soft-robot controlstrategies.

To this end, we describe, in Section II, a methodologi-cal approach for accurately reconstructing 3D octopus armtrajectories, during the arm swimming motion, by utilizingcomputer vision methods. The process is broadly basedon an earlier innovative approach by Yekutieli et al. [22],who addressed this complex problem by exploiting advancedcomputer vision techniques, leading to a semi-automatic armsegmentation algorithm, which is robust and user-friendly.This algorithm has been extensively modified with newmethods being developed for camera calibration and 3Dreconstruction, and the addition of several methodologicalfeatures for further accuracy and robustness [23]. Further-more, the work of Yekutieli et al. [22] pertains to a singlearm imaged frontoparallelly, while in our work we extendedthis approach to 8 arms imaged in 3D. Section III presentsa complete series of 3D-reconstructed trajectories of alleight octopus arms during arm swimming motion. Detailedinformation about individual arms, as well as of the speedand acceleration of the whole body during the motion arecalculated and demonstrated. This work gives a good appre-ciation of the complexity of the movement of the biological

2015 IEEE International Conference on Robotics and Automation (ICRA)Washington State Convention CenterSeattle, Washington, May 26-30, 2015

978-1-4799-6922-7/15/$31.00 ©2015 IEEE 1178

Page 2: Vision-Based 3D Motion Reconstruction of Octopus Arm ...

arms, and allows for a direct comparison with the underwaterrobot, in terms of velocity patterns and achieved acceleration(Section IV).

II. COMPUTER VISION METHODS

The motion reconstruction process involves the followingsequential steps.A. Video acquisition and frame extraction

Video recordings of live swimming octopuses in the aquar-ium were acquired at the Department of Neurobiology, atthe Hebrew University of Jerusalem [10], and were focusedon capturing in detail the octopus arm swimming motion.Octopuses of 150-400 grams were recorded by 3 cameras(Sony PMW-EX1R), at 25 Hz at the resolution of 1080×1920pixels, positioned ≈ 1 m away from the front side of thewater tank and at an extent of ≈ 35 cm (their 3D posturesare illustrated in Figure 1c). Two 150 W daylight emissionhalogen bulbs and a day light emission T5 Neon lampwere used for illumination. A camera flash was used tosynchronize the cameras.

More than 30 occasions of interesting footage were iden-tified. Footage was not always clear (e.g. when the octopusarms were touching the glass of the tank, the sand, or thenet) and, thus, interesting parts were selected after carefulobservation. To facilitate manual segmentation issues, due todistal arms and occluded parts, the selection included videosequences with the mostly visible arms. Synchronized se-quences of a grid target were acquired for camera calibration.

Video frames were deinterlaced, as interlaced imaging wasselected to image at the greatest possible time frequency. Se-quences from the cameras were later synchronized manually,co-registering the snapshots of the camera flash.B. Camera calibration

Camera calibration is essential in the triangulation of 3Dlocations on the arms. Grid based calibration [24] was usedto estimate the intrinsic and extrinsic camera parameters,and the lens distortion parameters. The reprojection errorwas in the sub-pixel range, ensuring that we have a verygood estimate of the projection of an arbitrary world pointon the image sensor. Using the estimated lens distortionparameters, the images are rectified for lens distortion, suchthat the mapping from world objects to image pixels can beconsidered as a linear projective transformation and straightlines in the world (e.g. the tank’s edges) project as straightlines in the 2D image (Figure 1a). It is noted that the pinholecamera model and the corresponding calibration methodsassume a homogeneous medium. This is not the case here,as the calibration rig is in one medium (water), and thecamera is in another (air). We have tested that, in the currentconfiguration of distance and obliqueness, the distortion dueto refraction is negligible.C. Arm segmentation and midline extraction

Arm segmentation requires user input, to determine theoutlining contour of the arm in the 3 acquired images, foreach frame of the sequence. A contour is marked pointwisely,starting from a chosen characteristic octopus feature (e.g. the

eye or the mantle tip), towards the arm base and continuingtowards the arm tip (half-contour); then returning backwardstowards the initial position (Figure 1b). The user can move,delete and add points through an interactive process [22].During this process, a piecewise linear curve interpolatesall marked 2D points and the user interface facilitates themarking of any intersections of looping or occluding arms.To increase automation, a prior estimate of the contour isprovided based on the result of the previous frame so thatthe user needs only to modify this prior contour.

The approach extracts the “midline” from image contours,which is assumed to approximate (the projection of) themorphological skeleton of the 3D arm in the image. Midlineextraction requires the user to annotate two points: the tipof the arm and the eye or mantle tip. This annotation isperformed in temporally-corresponding image triplets. Thesystem assists the user as follows: When two points areannotated, their stereo triangulation (see Section II-D) iscomputed and the reconstructed point is reprojected on thethird image. If annotation was accurate, then the reprojec-tions occur at the image location where the actual armtip or eye/mantle is, and, if otherwise, further away. Thesystem automatically visualizes this reprojection, when twoor more annotations of a physical point are available for animage triplet. In this way, the user receives online feedbackupon annotation accuracy for the correction of inaccurateannotations. At the same time, the automation of the task isincreased: usually the two annotations provide a good priorfor the location of the third.

The interactive approach in [22] is followed to semi-automatically mark arm contours and extract their midlines,through a simple “point and click” task. A prior of the armtip is available as the contour point of maximal curvature.Interactive zoom-in/zoom-out abilities on each frame helpdiscern the fine image pixel differences and, thus, a traineduser can study in detail the shape of the arm and its orien-tation in space, as viewed from 3 different cameras, therebyresolving issues of poor contrast, shadows, and occlusions.Using the method in [22], the midline is automaticallyextracted as a contour skeletonization result. In cases of alooping octopus arm, the contour intersection is split intoparts and the midline is extracted independently for eachpart. On each midline, a cubic spline interpolation algorithmis subsequently applied to assign a known number of curvepoints (n = 200), while preserving the exact curvature of themidline. No smoothing is applied here, as it was found toalter drastically the midline curvature. For the evaluation ofthe process, observer agreement tests were performed forself- and observer-repetition of independent contours; theextracted midlines were found to be in adequately goodagreement, particularly towards the tip of the arm. Inter-uservariations were related to user decision-making difficulties todiscern the position of the eye with respect to the arm base;intra-user repetitions, however, had less variations, verifyingthat a trained user can generate recurrent results.

The selection of the eye or the mantle-tip aims to providea landmark on the octopus body that is common for all arms

1179

Page 3: Vision-Based 3D Motion Reconstruction of Octopus Arm ...

(a) (b) (c)

Fig. 1: Left to right: (a) Confirmation of lens distortion compensation: straight lines in the world (i.e. tank edges) appear as straightlines in the 2D image. (b) Automatic midline extraction. (c) 3D reconstruction showing stereo for annotated points, contour midlines(superimposed in red) and arm reconstruction.

and often visible to all cameras. The octopus mouth wouldbe an alternative, perhaps more ideal, feature, for defining thearms’ bases, however it is not visible in most of the acquiredmovements. In the results of Section III, the eye was chosenas the starting point for arm contour segmentation.

D. Stereo reconstruction

Stereo reconstruction of the octopus arms is based on thecorrespondence of the extracted midlines, across the threeimages, for each frame. For each image triplet, the annota-tions of the eye/mantle-tip and the arm-tip are established ascorresponding. Moreover the n ordered, spline-interpolatedpoints in each image are, also, considered as corresponding.This is a reasonable approximation as differences in perspec-tive distortion across frames are considered to be negligible.

Each triplet of corresponding image points is employedto reconstruct a 3D point (using the trifocal tensor [25]),which is assumed to occur on the morphological skeletonof the 3D arm (Figure 1c). A reprojection process, from thereconstructed 3D points back to the 2D cameras views (red-colored lines in Figure 2), allowed evaluation and verificationof the resulting 3D trajectories, through comparison with theoriginally extracted midlines (green lines in Figure 2).

III. RESULTS

A. Whole body analysis

In the following, we present and analyze a sequenceof a female common octopus, the weight of which wasmeasured at 180 gr, performing arm swimming motion. Themotion is reconstructed from a series of 68 frames, foreach of the 3 cameras. Figure 2 shows 10 frames from themiddle camera. In each subfigure (2a–2j), the first columndisplays the original image captured by the middle camera;the second column shows the same image overlaid by thegreen-colored 2D midlines, extracted for each of the 8 armsduring the contour segmentation process (Section II-C), andthe red-colored reprojected trajectories, calculated after the3D triangulation process (Section II-D); the third columndepicts the 3D-reconstructed arm trajectories that correspondto the same snapshots as the previous two columns. Armsnotation is taken almost clockwise, as labeled in the top-left-most figure (Figure 2a). Each subfigure (2a–2j) correspondsto a constant time interval of 100 ms. Arm colors in the thirdcolumn are according to the arm’s number, from red (Arm1) to magenta (Arm 8), as labeled in the legend of 2a.

The original images (first columns in the subfigures ofFigure 2) already provide a first visual understanding of themotion, with the apparent “opening” and “closing” parts.In 2a–2e, the octopus moves the arms outwards up to amaximum position, before it moves them inwards (in 2f–2j).Although there are differences among the arms movements,they appear to act in a rather synchronous manner. Aconcomitant characteristic, which is visible in these figures,is the extension of the web in-between the arms. Based onlyon the visual result, the web appears to be passive duringthe opening stroke (2a–2e), but it may be taking active partin the movement during the closing (power) stroke (2f–2j); verification of this observation would require differentmethodologies, which is out of the scope of the current paper.

The images shown in the second columns of Figure 2,provide a clear confirmation of the suggested reconstructionprocess (cf. Section II). Red-colored 3D reconstructed lines,reprojected on the 2D images, are found to be in very closeproximity with the actual segmented 2D midlines (before 3Dtriangulation). Correlation with the original images on theleft columns shows that both lines are accurately positionedalong the medial axes of the arms. The calibration andtriangulation processes can, hence, generate realistic andaccurate estimations of the 3D arm trajectories.

The last columns of Figure 2 depict the resulting 3Darm trajectories. These are the raw, unsmoothed data, re-constructed following the vision-based process of Section II.Appropriate camera view rotation allows for a direct corre-lation of the reconstructed result with the original images(shown only for the middle camera). The segmentation wasperformed in such a way so that all arms have a commonstarting point (here the eye); the segment between the eye andthe “base” of each arm, which is in general difficult to definerobustly, was included in the segmentation and is visible inthe 3D plots as the relatively more straightened part at thebeginning of each trajectory. Therefore, the origin of all 8arm trajectories corresponds to the eye of the octopus.

By plotting the first reconstructed 3D point (coincidinghere with the eye), the trajectory of the whole octopus bodycan be traced as it locomotes within the 3D space of thewater tank (Figure 3a). In the plot, the octopus moves fromleft to right, with the initial position being marked with a redcircle; the positions colored in magenta correspond to thetime instances displayed in Figure 2 (therefore, the right-

1180

Page 4: Vision-Based 3D Motion Reconstruction of Octopus Arm ...

a) f)

b) g)

c) h)

d) i)

e) j)

Fig. 2: Complete sequence of octopus arm swimming motion (see accompanying video). Arms notation is taken clockwise, as labeledin (a). (First columns) Original images by the middle camera. (Second columns) The same images overlaid by segmentation-extractedmidlines (colored in green) and the reprojected trajectories (colored in red). (Third columns) 3D-reconstructed arm trajectories, colored bythe arm’s number from red (Arm 1) to magenta (Arm 8). Frames in subfigures (a)–(j) are shown for a constant time interval of 100ms.

most magenta-colored point corresponds to subfigure 2j).The octopus velocity magnitude (Figure 3b) and acceleration(Figure 3c) can then be calculated, taking into account theframe rate of the de-interlaced image sequence (dt = 0.02 s).

The total duration of the specific movement is 1.32 sec.These plots show that at initiation of the opening stroke,this octopus exhibits little momentum, being drifted slowlydownwards (X axis in Figure 3a), with a roughly con-stant sinking velocity magnitude of V ≈ 51 mm/s andan almost zero net acceleration (up to time t = 0.52 s).Between t = 0.52 s and 0.66 s, the animal decelerates toa minimum velocity of Vmin ≈ 14 mm/s, marking theend of the “arm opening” (recovery) stroke (first verticalline in Figures 3b, 3c). The “arm closing” (power) strokestarts immediately after, with accelerations as high as ap-proximately 1300 mm/s2, achieving a maximum speed ofVmax ≈ 250 mm/s (second vertical line in Figures 3b, 3c).The power stroke lasts 0.44 s, giving a ratio of recovery topower stroke duration of 1.5; the Vmax to Vmin ratio is foundapproximately equal to 18, whereas the Vmax to V ratio isapproximately 5. During the last part of the motion (for times

greater than t = 1.1sec), the octopus decelerates at a ratherconstant rate, acquiring a drifting velocity that decreases overtime. Figure 3a shows the maximum distance covered by theanimal during this motion, primarily during the power stroke.B. Individual arm analysis

An individual arm trajectory over the duration of themotion is depicted in Figure 4, for arm 2; as annotated inFigure 2a. The trajectory has been plotted with a zero startingpoint, in order to eliminate the motion of the octopus body.This way, the time-history of an arm’s trajectories is isolatedand can be examined individually. The trajectories have beensmoothed in the Figure. Initial arm positions are colored withdark blue colors and final positions with red colors.

The trajectory pattern is highly three-dimensional andcomplex, particularly at the initial stages of the movement(blue- to green-colored lines), with large rotations and twists.During the power stroke (green- to red-colored lines), thearm appears to close in a wave-like propagation pattern, ina somewhat similar aspect to that described by Yekutieli etal. [26] for the octopus reaching movement. At the end ofthe arm swimming motion, the arm is almost straightened.

1181

Page 5: Vision-Based 3D Motion Reconstruction of Octopus Arm ...

a)

b)

c)

Fig. 3: Whole body motion of the octopus: (a) 3D trajectory, (b)velocity and (c) acceleration.

Figure 5 shows mean arm length values over time, calcu-lated from 7 arms (with the exclusion of arm 3 which needsfurther processing), and error bars showing the standarddeviation of the mean values. The plot is overlaid by thesignificantly-smoothed mean (in red) to extract the generaltrend of arm elongation and shortening. All arms appear toelongate for most of the opening and closing parts of thearm-swimming motion (up until around t = 0.94 s and thenshorten for larger times.

IV. DISCUSSION AND CONCLUSIONS

The results of the previous Section demonstrate that allarms move with a similar speed during the arm swimmingmotion, taking on positions from almost perpendicular (Fig-ure 2a) to nearly parallel (Figure 2j), with respect to thedirection of movement. Taking into account the shape of theoctopus mantle and that of the web, these arm positions letthe animal take on, respectively, blunt- and streamlined-bodyshapes with regard to the oncoming flow [15], [18], [27]. Asa blunt-shaped body, the octopus experiences an increasedfluid drag force at the initial (opened) arms positions, whileas a streamlined-body, at the later (closed) positions, itreduces the acting fluid force. The transition from blunt- tostreamlined-body shapes occurs during the power stroke, inless than 0.5 s, resulting in a fast drag reduction.

Another interesting aspect of the movement is that duringthe opening part, the animal has little momentum and appearsto sink. By closing the arms, the octopus produces a burst offorward thrust (Figure 3a), possibly in an effort to regulate itsbuoyancy and avoid sinking [28]. It is, therefore, during thepower stroke part of this motion that the animal is propelledforward. In addition, although the movement can be broadlysplit into recovery and power strokes, the acceleration profileof Figure 3c indicates that the pattern is more complex, with

Fig. 4: Individual arm trajectories during the arm swimming motion,for a zero starting point. Arm 2, as annotated in Figure 2a. Darkblue color refers to initial (open) positions while red colors to thefinal (closed) positions.

Fig. 5: Mean arm length calculated from 7 arms and standarddeviation, overlaid by significantly smoothed mean arm length (arm3 was excluded from the calculation of the mean values).

several fluctuations in the acceleration.A comparison of the movements of the 8-arm, web-

equipped robotic swimmer presented in [12], with those ofthe real octopus, is shown in Figure 6, which captures thepower stroke of one period of the motion. The similarity inthe movement of the robotic swimmer and of its biologicalcounterpart is evident. Moreover, the normalized velocityover normalized time duration based on the maximum valuesfor the closing part (power stroke) of the arm swimmingmotion was calculated, and is shown in Figure 7. Thedata for the 8-arm swimmer were obtained via computervision methods described in [12], whereas those for the realoctopus via the motion reconstruction process described here(Section II). The plot indicates significant similarities in thenormalized velocity profile between the robotic swimmer andthe real octopus.

In conclusion, the 3D trajectories of the medial axes ofoctopus arms during arm swimming motion were robustlyextracted from a video sequence of a live octopus. This isthe first demonstration of a complete 3D swimming motionsequence involving all eight individual arms. The end goalis to extract a kinematic description of the motion, which isaddressed in ongoing work. Future studies will also includeinvestigations of active and passive parts of the arms duringthis propulsive mode. Future work will also include the use ofmore sophisticated vision algorithms to increase automationand more accurately reconstruct the arms of the octopus, bytaking into account its spatial extent rather than its midline.

1182

Page 6: Vision-Based 3D Motion Reconstruction of Octopus Arm ...

a)

b)

c)Fig. 6: (a) 8-arm octopus robot performing arm swimming motion(Frames time interval: 125 ms). (b) Real octopus arm swimming(Time interval: 120 ms). (c) Different view of the 3D-reconstructedarm trajectories (Time interval: 120 ms).

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

Normalized Time

Nor

mal

ized

Vel

ocity

real octopus8−arm swimmer with web

Fig. 7: Direct correlation between the real octopus analyses and the8-arm swimmer with the web of Sfakiotakis et al. [12], in terms ofnormalized velocity over normalized time duration for the closingpart (power stroke) of the arm swimming motion.

The automatic treatment of occlusions is also warranted, asit requires increased user input, through the introduction ofadditional views that disambiguate these occlusions.

ACKNOWLEDGMENTS

The authors would like to thank B. Hochner and T. Flash for insightfuldiscussions and for sharing the image analysis code of [22]. Also, M.Kuba, A. Botvinnik, S. Hanassy, and T. Gutnick for their help with theacquisition of the video recordings and S. Stefanou, M. Paraskevopoulouand C. Kogeraki for their assistance with the manual segmentation process.Finally, M. Sfakiotakis, A. Chatzidaki, Th. Evdaimon, P. Pandeleris and V.Koussanakis, for their assistance with robotic experiments.

REFERENCES

[1] D. Trivedi, C. D. Rahn, W. M. Kier, and I. D. Walker, “Soft robotics:Biological inspiration, state of the art, and future research,” Appl.Bionics Biomech., vol. 5, no. 3, pp. 99–117, September 2008.

[2] A. Villanueva, K. Marut, T. Michael, and S. Priya, “Biomimeticautonomous robot inspired by the Cyanea capillata (Cyro),” Bioinspir.Biomim., vol. 8, p. 046005 (18pp), 2013.

[3] S. Hirose, Biologically inspired robots: snake-like locomotors andmanipulators. Oxford University Press, 1993.

[4] F. Delcomyn, Biologically inspired robots. In: Bioinspiration androbotics: walking and climbing robots, M. Habib, Ed. I-Tech, 2007.

[5] W. Kier and K. Smith, “Tongues, tentacles and trunks: the biomechan-ics of movement in muscular-hydrostats,” Zool. J. Linn. Soc., vol. 83,pp. 307–324, 1985.

[6] Y. Gutfreund, T. Flash, Y. Yarom, G. Fiorito, I. Segev, and B. Hochner,“Organization of octopus arm movements: a model system for studyingthe control of flexible arms,” J. Neurosc., vol. 16, pp. 7297–7307,1996.

[7] J. Mather, “How do octopuses use their arms?” J. ComparativePsychology, vol. 112, no. 3, pp. 306–316, 1998.

[8] G. Sumbre, Y. Gutfreund, G. Fiorito, T. Flash, and B. Hochner,“Control of octopus arm extension by a peripheral motor program,”Science, vol. 293, pp. 1845–1848, 2001.

[9] C. Huffard, “Locomotion by abdopus aculeatus (cephalopoda: Octopo-didae): Walking the line between primary and secondary defenses,” J.Exp. Biol., vol. 209, pp. 3697–3707, 2006.

[10] A. Kazakidi, M. Kuba, A. Botvinnik, M. Sfakiotakis, T. Gutnick,S. Hanassy, G. Levy, J. A. Ekaterinaris, T. Flash, B. Hochner, andD. P. Tsakiris, “Swimming patterns of the octopus vulgaris,” in 22ndAnnual Meeting NCM Society, Venice, Italy, April 23-29, 2012.

[11] M. Sfakiotakis, A. Kazakidi, N. Pateromichelakis, and D.P. Tsakiris,“Octopus-inspired 8-arm robotic swimming by sculling movements,”in ICRA’13, Karlsruhe, Germany, May 6-10, 2013, pp. 5135–5141.

[12] M. Sfakiotakis, A. Kazakidi, A. Chatzidaki, T. Evdaimon, and D. P.Tsakiris, “Multi-arm robotic swimming with octopus-inspired compli-ant web,” in IEEE/RSJ Int. Conf. on Int. Rob. Syst. (IROS’14), Chicago,Illinois, USA, September 14–18 2014, pp. 302–308.

[13] M. Sfakiotakis, A. Kazakidi, and D. P. Tsakiris, “Turning maneuversof an octopus-inspired multi-arm robotic swimmer,” in 21st Med. Conf.Control Autom. (MED’13), Greece, June 25-28, 2013, pp. 1343–1349.

[14] M. Sfakiotakis, A. Kazakidi, N. Pateromichelakis, J.A. Ekaterinaris,and D.P. Tsakiris, “Robotic underwater propulsion inspired by theoctopus multi-arm swimming,” in IEEE Int. Conf. Rob. Autom.(ICRA’12), St. Paul, USA, May 14-18, 2012, pp. 3833–3839.

[15] A. Kazakidi, V. Vavourakis, D. P. Tsakiris, and J. Ekaterinaris,“A numerical investigation of flow around octopus-like arms: near-wake vortex patterns and force development,” Comp. Meth. Biomech.Biomed. Eng., vol. 18, no. 12, pp. 1321–1339, 2015.

[16] A. Kazakidi, D. P. Tsakiris, D. Angelidis, F. Sotiropoulos, and J. A.Ekaterinaris, “CFD study of aquatic thrust generation by an octopus-like arm under prescribed deformations,” Computers and Fluids (con-ditionally accepted), 2015.

[17] A. Kazakidi, D.P. Tsakiris, F. Sotiropoulos, and J.A. Ekaterinaris,“Numerical investigation of aquatic locomotion with cephalopod-likeappendages,” in 19th Congress of the European Society of Bioengi-neering (ESB), Patra, Greece, August 25-28 2013.

[18] A. Kazakidi, V. Vavourakis, N. Pateromichelakis, J.A. Ekaterinaris,and D.P. Tsakiris, “Hydrodynamic analysis of octopus-like roboticarms,” in ICRA’12, St. Paul, USA, May 14-18, 2012, pp. 5295–5300.

[19] V. Vavourakis, A. Kazakidi, D.P. Tsakiris, and J.A. Ekaterinaris, “Anonlinear dynamic finite element approach for simulating muscularhydrostats,” Comput. Methods Biomech. Biomed. Engin., vol. 17, no. 8,pp. 917–931, 2014.

[20] V. Vavourakis, D. Bampasakis, A. Kazakidi, N. Pateromichelakis,J.A. Ekaterinaris, and D.P. Tsakiris, “Generation of primitive behav-iors for non-linear hyperelastic octopus-inspired robotic arm,” in IEEEInt Conf Biomed Rob Biomech, Roma, Italy, 2012, pp. 725–730.

[21] M. Sfakiotakis, A. Kazakidi, and D. P. Tsakiris, “Octopus-inspiredmulti-arm robotic swimming,” Bioinsp. Biomim. (in press), 2014.

[22] Y. Yekutieli, R. Mitelman, B. Hochner, and T. Flash, “Analyzingoctopus movements using three-dimensional reconstruction,” J. Neu-rophysiol., vol. 98, pp. 1775–1790, 2007.

[23] A. Kazakidi, S. Stefanou, X. Zabulis, M. Kuba, J. A. Ekaterinaris,T. Flash, B. Hochner, and D. P. Tsakiris, “3d reconstruction of octopusarm swimming motion,” in 7th World Congress of Biomechanics(WCB), Boston, MA, USA, July 6-11 2014.

[24] R. Y. Tsai, “A versatile camera calibration technique for high-accuracy3d machine vision metrology using off-the-shelf TV cameras andlenses,” IEEE Journal of Robotics and Automation, vol. 3, no. 4, pp.323–344, 1987.

[25] R. I. Hartley and A. Zisserman, Multiple View Geometry in ComputerVision, 2nd ed. Cambridge University Press, 2004.

[26] Y. Yekutieli, R. Sagiv-Zohar, R. Aharonov, Y. Engel, B. Hochner, andT. Flash, “Dynamic model of the octopus arm. i. biomechanics ofthe octopus reaching movement,” J. Neurophysiol., vol. 94, no. 2, pp.1443–1458, 2005.

[27] A. Kazakidi, D. P. Tsakiris, F. Sotiropoulos, and J. A. Ekaterinaris,“A computational fluid dynamic study of intense cephalopod-likemotions,” in 44th AIAA Fluid Dynamics Conference, Atlanta, USA,16 – 20 June 2014, pp. 1–15.

[28] DM. Webber, JP. Aitken, and RK. O’Dor, “Costs of locomotion andvertic dynamics of cephalopods and fish,” Physiol Biochem Zool,vol. 73, no. 6, pp. 651–662, 2000.

1183