ROBOTICS IN HORTICULTURAL FIELD …€¦ · 2. Overview Background The Agricultural Research...

1

Autonomous and Human-Robot Collaborative Systems

Avital Bechar

for Field Operations in Orchards, Greenhouses and Field Crops

Institute of Agricultural Engineering, ARO, Volcani Center, Israel

2

Overview Background The Agricultural Research Organization Agricultural productivity and production (robotics

perspectives) Characteristics of the agricultural domain (robotics

perspectives) Basic principles (AgRobots) ARL activity Conclusions

AgRA TC Webinar, March 2015 ,24

3

Agricultural Research Organization


4 AgRA TC Webinar, March 2015 ,24

• Founded in 1921.• 1000 people: including 200 research

scientists and 220 graduate students.• 6 Institutes: Soil water and

environmental sciences; Plant protection; Animal Sciences; Plant sciences; Food sciences; and, Agricultural Engineering.

Agricultural Research Organization


Institute of Agricultural Engineering The only research organization in Israel whose activities

encompass a wide range of engineering and technological topics relating to all aspects of agriculture.

About 60 people, including 14 research scientists


Institute of Agricultural Engineering Two departments:

Sensing, information, and mechanization engineering Production, growing and environmental engineering

a

7

Agricultural production Cultivation and production processes in agriculture. Affecting factors: crop characteristics and requirements, the geographical/geological environments, climatic conditions, market demands the farmer’s capabilities and means.

Farm sizes increase and the number of farmers and agricultural workers decreases.

Human labor intensive and labor cost of 25-40%.



(http://www.thadw.us/agricultural-employment-since- 1870/ )


0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

0.55

MetalBolts

Metalscrew nuts

Metalnails

MetalDiscs

Plasticparts

Rubberparts

Woodparts

Flowercuttings

CV

CV of different materials

CV=σ/µ

CV2 > CV1

12

Unstructured Environments

• Unknown a-priori• Unpredictable• Dynamic


13

The terrain, vegetation, landscape, visibility, illumination and other atmospheric conditions are not well defined; vary, have inherent uncertainty, and generate unpredictable and dynamic situations.


Unstructured Environments


Unstructured Objects

Variable and non-uniform:size

shape color

texturelocation


--++Objects

-+-+Env.

Agr.MedicalSpaceUnder-water

Military

Industry

16

Basic principles Main task: pruning, picking, harvesting, weeding... Supporting tasks: localization, detection, navigation…

Mobility and steering Sensing Path planning and navigation Manipulators and end effectors Control Autonomy and human-robot collaboration


17

Autonomy/Human-Robot collaboration


Autonomous robot are lack the capability to respond to ill-defined, unknown, changing, and unpredicted events, such as occur in unstructured environments.

Pareto principle: roughly 80% of a task is easy to adapt to robotics and automation and 20% is difficult (Stentz et al., 2002).

18

Hybrid Human-Robot Systems


Main Task

SupportingTask

1

SupportingTask

3

SupportingTask

4

SupportingTask

2

Subsystem1

Subsystem2


Lab members (current)

2 PhD students (IE, CE-AgEng) 3 MSc students (ME, IE) Mechanical Engineer Electrical Engineer Postdoc Agronomist


Projects (current) Autonomous greenhouse sprayer for specialty

crops (with BGU). A human-robot collaborative system for deciduous

tree selective pruning. a human-robot system for selective melon

collection (with Technion). an autonomous system for monitoring of diseases

in greenhouses (with BGU). Robotic sonar for yield estimation (with TAU). Characterization of Agricultural Tasks for the

Design of a Minimalistic Robot (with Technion).

22

Autonomous greenhouse sprayer Avital Bechar, Itamar Dar, Victor Bloch, Yael Edan,

Roee Finkelshtein, Guy Lidor, Ron Berenstein


23

The motivation


24

Plot geometry

100

m

115

170



R R∆

Sensing (Features)

26

FeaturesFeature Formula Feature Formula

R Red h H/(H+S+V)

G Green s S/(H+S+V)

B Blue v V/(H+S+V)

r R/(R+G+B) deltaH (H-S)+(S-V)

g G/(R+G+B) deltaS (S-H)+(S-V)

b B/(R+G+B) deltaV (V-S)+(V-H)

deltaR (R-G)+(R-B) C1 R-G

deltaG (G-R)+(G-B) C2 R-B

deltaB (B-G)+(B-R) C3 G-B

H Hue Real_ModHue

S Saturation imag_ModHue

V Value

)(cos)(2

21222

{GBRBRGBGR

BGR−−−++

−−−

27

For all featuresFind feature threshold value that maximizes the "splitting criterion“

Among all featuresChoose the one thatmaximizes the"splitting criterion“

Decision Tree - CART Breiman et al., 1984


28

Total success

Movie 1 LevelTS

2 LevelTS

3 LevelTS

Movie 1 0.834 0.834 0.886Movie 2 0.943 0.941 0.940Movie 3 0.617 0.834 0.848Movie 4 0.818 0.874 0.889Movie 5 0.922 0.927 0.920Movie 6 0.892 0.899 0.899Movie 7 0.932 0.925 0.930Average 0.851 0.891 0.902

Number of nodes 1 3 7


Decision tree

29

Judges Vote (~ Majority rule)

A customized CART variation, developed in this research

A “Judge” is single level CART (root node only)

Classification rule:

JudgesofNumberVoteJudges__

_

54321432132121Vote (M)55555444433322Judges (N)


30

Test set – "Judges Vote"Variation 2/2 2/3 3/4 3/5 4/5 2 Level (3 features)

Average 0.903 0.914 0.920 0.915 0.905 0.890

std 0.041 0.021 0.016 0.020 0.022 0.044


31

Algorithm Evaluation Platform

ArdunoCMP-03 Compass

AX3500 - Dual 60A

lifeCam NX-6000

180⁰

Lenovo R400

Servo SC-1256T

Motor DL-30Encoder Optical

E5

45

DATA

PWM

123


33

TXT1 Ein Yahav 261109 1st exp-fast.wmv


34

Modification of a commercial sprayer

An electric motor was installed on the steering wheel controlled by a Roboteq controller.

Installation of encoders on the steering pivot/axle and the front wheels.

PID control system. Control system inputs: platform

steering angle; desired direction from the adaptive algorithm and bearing.

Pure pursuit, carrot point 2m


35

The ‘autonomous unit’

Installed on the platform Connected to sensors and

actuators


36

Commercial Sprayer II


38

A H-R collaborative system for selective pruning

Avital Bechar, Victor Bloch, Roee Finkelshtain, Sivan Levi


39

Objective

Develop a human-robot integrated system for tree pruning and shaping Design of a cutting tool Develop a modelling technique Development of human robot interface and

methodology


40

Cutting tool alternatives

Chain saw Pruning shears Laser Water jet Disc saw Jigsaw


41

Cutting tool design

The cutting tool must be adapted to: Tree dimensions, branch diameter and strength Robot carrying ability, precision, energy source Pruning technique: cutting angle, velocity Tree structure: branch angles, depth inside the crown, obstacle

density, reaching ability

Agronomical requirements: Cutting angle 45° Reduce risk of wounds Cut disinfection (burned by high cutting speed)


42

Cutting tool selection & modification for a robotic arm Energy source, type and consumption Safety Weight Dimensions Precision and accuracy …


43

High accuracy requirements

Pruning shears: 3 directional dim. and 2 angular dim. Total required accuracy in 5D.

Disk saw: 1 directional dim. and 2 angular dim. Total required accuracy in 3D


44

Cutting tool design

0

10

20

30

40

50

60

70

80

3 6 9 12 15 18 21 24 27 30 33 36

Num

ber

of b

ranc

hes

Branch Diameter [mm]

80%82%84%86%88%90%92%94%96%98%

100%

3 6 9 12 15 18 21 24 27 30 33 36

Bra

nch

acc

umul

ativ

e pe

rcen

tage

Branch diameter [mm]

AgRA TC Webinar, March 2015, 24

45

Cutting tool design

AgRA TC Webinar, March 2015, 24

46

Tree Modeling

Simple and reliable method – mechanical


47

Tree Modeling

(Linker et al., 2014)


48

Path planningAssuming cutting points given: Find optimal reaching orientation Solve robot navigation problem in 6 dimensional C-space Find optimal order of cutting points


49

Primal collaboration 3D modelling of a tree a-priori HO marks cutting point on model Trajectory planning Branch pruning

Drawbacks The need for a-priori 3D modelling Long set up time Computation power (time, cost) Inaccurate Lack of information Not up to date


50

Concept and task description

Stage

SenseReason

PlanAct

Sub-Task

Images / modelCutting point detectionTrajectory planning and controlBranch pruning

Control

RH+ R

RR



Brunch orientation (Two methods)

Movements (joints and linear)

53

21 subjects Age: 24 – 69 20 branches Two types of marking (1 click and 2 clicks) Two types of movements

Experiment


54

Results

0

1

2

3

4

5

6

2 clicks 1 click

Tim

e [s

ec]

click 1click 2

aa

bα<<0.01


55

Results

0

5

10

15

20

25

Cut sign move toscan

scan move tocut

return HR cycletime

Robotcycle time

Aver

age

time

[s] Joints

Linearic


56

Results Cutting point accuracy: 8-22 mm Branch orientation accuracy: mean: 9.4º, med: 5.75º

0%10%20%30%40%50%60%70%80%90%

100%

0

1

2

3

4

5

6

7

8

2 4 6 8 10 12 14 16 18 20 22 More

Num

ber

of b

ranc

hes

Error [degree]

Frequency

Cumulative %


57

Avital Bechar, Noa Schor, Sigal Berman, Aviv Dombrovsky, Yigal Elad and Timea Ignat


A Diseases Monitoring Robot

58

A cart roves between crop rows. Manipulator mounted on the cart is maneuvering into

a set of positions for sensing and detection. Sensors acquire data and fuse them to achieve high

precision. Early detection of two diseases: powdery mildew and

tomato spotted wilt virus (TSWV).


Scenario

59

No known algorithms for TSWV detection. Detection of more than one threat has not been

attempted thus far. Development based on a holistic approach integrating

the design of both motion and perception.


Challenges

60

A robotic manipulator (MH5L, Motoman). A custom-made end-effector. Sensory apparatus.


Apparatus

Manipulator

End-effector

Laser sensor

Multispectral camera

Color camera

61

TSWV Powdery mildew


Disease detection


Motion planning and execution


TSWV detection


Disease detection


Melons collecting robotAvital Bechar, Moshe Karagoden, Ariel Weinstock, Moshe Mann, Sasha Katzman, Victor Bloch, Guy Lidor, Roee Finkelshtein, Itzhak Shmulevich

66

A Cartesian robot. Two cylindrical rails, toothed belt axis and end limit

switches; Two Stepper Motors; Motor Controllers; Programmable Logic Controller (PLC); Frame (600 mm x1500mm).

Vacuum operated Gripper


Apparatus

6767

Melons Picking Up Simulator


68

20 trials In each trial, 3-7 objects Density of 2-5 objects/m2

total collecting area of 4m X 0.5m Cart velocity: 51 mm/sec manipulator velocity: 800 mm/sec


Dynamic state experiment

69

Total success rate of 84% Due to technical and

communication problems Position error 7-10 mm at

reach location Collection pace: 7-8

objects per minute


Results

70

Conclusions There has been considerable progress. Technical feasibility was shown. Agricultural modifications or human integration.


71

THANK YOU

Avital [email protected]

ROBOTICS IN HORTICULTURAL FIELD …€¦ · 2. Overview Background The Agricultural Research...

Documents

Transcript of ROBOTICS IN HORTICULTURAL FIELD …€¦ · 2. Overview Background The Agricultural Research...