A Breakthrough in Oil Palm Precision Agriculture: Smart Management of Oil Palm Plantations with...

Redmond Ramin Shamshiri, PhD

2016

Next

Next Prev

Precision Agriculture (Satellite Farming) has faced a breakthrough shift since risings of UAVs

Conventional Remote Sensing platforms are being replaced by integrated UAVs

Precision Agriculture

UAV

Wireless

Sensor

Networks

Robotics

Automation

Control

GPS

GIS

VRT

Remote

Sensing

As a flexible remote sensing platform

Crop/ Tree Scouting

Health/growth assessment

Inventory management

Yield estimation / Monitoring

Weed and disease detection

Mapping (2D, 3D, GIS, NDVI)

Risk/Hazard/Safety management

Soil condition assessment

VRT

Robotics harvesting

Academic and Research application

Typical Applications in Agriculture

RGB

Infrared

NIR

Hyperspectral

Multispectral

Thermal

Mapping software, Mobile Apps

Integrated with Commercial

sensors/Cameras/Software

UAVs & Robotics are the future of Precision Ag

Precision Agriculture of Oil Palm is one of the largest

market in Malaysia to be hit by UAVs

Redmond Ramin Shamshiri, 2016. UAV for Oil Palm Precision Agriculture. https://florida.academia.edu/RaminShamshiri

https://florida.academia.edu/RaminShamshiri


Next Prev

Precision Agriculture is about optimizing returns on inputs while preserving resources

PA is a farming Management concept based on Sensing, Measuring and Assessment

Sensing Components

Sensors , Cameras Platform

Ground-based

Handheld Vehicle mounted

Airborne

Satellite UAV Piloted Airplanes

o RGB

o Hyperspectral

o Multispectral

o Thermal

o NDVI

o Infrared

o NIR

Fixed wing Multi-rotor




Next Prev

Traditional Scouting o Traditional scouting requires spending hours and hours of visualizing

o Involves manual operations, ineffective techniques

o Limitation of human/labor resources (Not on demand)

o Repeated dull tasks (i.e., Palm census) o Not a pleasant environment to work (hot, humid)

o Hazard/ Safety (Falling from trees, bugs, snakes, etc)

o Generates Inaccurate/biased statistics

o Requires expert knowledge/Post processing (i.e., lab analysis of data)

o Generates limited information (Does not provide comprehensive result)

o Ignored parameters due to measurement’s difficulties (i.e., tree height,

canopy diameter, tasks that involves climbing trees)

Satellite remote sensing

o Cost

o Low resolution

o Difficulties of access (Not on-demand)

Ground sensing

o Time consuming

o Limited field of view




Next Prev

Plant densities are an important and limiting factor for growth, Nutritional status,

fruiting and hence for a plantation’s yield.

Oil palm with high mortality Oil palm with good plant density Optimal plant densities depend on different

factors, such as cultivars, climate, soil

characteristics, land preparation…

Refilling of canopy gaps and correction of

non-optimal plant densities are of high

priority for a good plantation management

Yield reduction due to high density palm areas that causes Etiolation*.

Need for Automatic identification of potentially etiolated palms or high density areas

Conventional method, solely based on visual observation, inaccurate, particularly when

coverage is large and dominant topography is hillocky. * Etiolation is a process in flowering plants

grown in partial or complete absence of light.




Next Prev

Main Objective

To develop/adapt a flexible remote sensing platform by integrating UAV, Sensors,

and robust machine vision system for Smart Management of Oil Palm Plantations

Smart Management involves:

A: Smart Inventory Management

B: Smart Growth/Health Assessment

“Smart” refers to:

Autonomous monitoring, data processing and

decision making




Next Prev

Specific Objectives

1. To setup and operate a multi-rotor UAV remote sensing system for Oil Palm plantations

2. To produce 2D and 3D visual maps of the fields under study

(including Blocks, rivers, roads, boundaries)

3. To produce NDVI (Vegetation stress) and GIS maps for health and growth assessment

4. To develop a robust, real-time machine vision system for the following tasks

(4.1) Inventory management

Palm census and density, track and record

Crown diameter estimation, Canopy size

Palm height measurements

Plantable, Unplantable, Overplanted areas

Palms distance

(4.2) Yield mapping system

Detection/Quantification of fresh fruit bunches from UAV images

Yield per Palm and Yield per ha




Next Prev

Consideration for a flexible design

Small size

Light-weight

Affordable

Autonomous

Stable (against wind and other disturbances)

Shifting between multiple sensors

Payload

Flight time

Safety (For operator, environment, and platform)

Low-altitude flight

On-Demand flight (Ease of access operation)

Repair and Maintenance costs




Next Prev




Next Prev

0

2

4

6

8

10

12

14

16

18

20

Billio

n U

S D

ollar Palm Oil Export ($)

0

10

20

30

40

50

60

% o

f th

e t

ota

l w

orl

d

Palm Oil Export (% of the total)

Oil Palm contribute to 12 billion USD of Malaysian economy

Potential for monitoring oil-palm plantations in such a

great detail has been never possible




Next Prev

Automatic palm detection, counting, size measurements, etc

Calculation of planted areas (for replanting or thinning)

Analyzing Palm status based on orthomosaics and digital elevation models

Generating valuable information based on each and every individual palm

Classification of palms based on crown size, height, vegetation indices, etc

Such information can be used for appropriate management decisions (maximize yields)

Yield Estimation Model development Correlation between palm height (𝒙𝟏) , crown size (𝒙𝟐), age (𝒙𝟑), vegetation index (𝒙𝟒) , …, and yield

𝒀𝒊𝒆𝒍𝒅 = 𝒇𝒖𝒏𝒄(𝒙𝟏, 𝒙𝟐, 𝒙𝟑, 𝒙𝟒…)

This research will provide growers/managers with a tool for:

A model that is based on a comprehensive information of each palm location, size, and health, will

provide managers with an estimation of yield, and make decisions for sustainable practices methods for

production increase without necessary needs for expanding the plantation into natural forests




Next Prev

Accurate planted area

Creation of Palm trees inventory database for specific plot

Total land use

Palm distances to specific areas

Canopy diameter estimation

Tree height measurements

Calculating palm density for specific plot

Creation of 2D, 3D, GIS, NDVI maps for plantation

Monitoring Healthy/Unhealthy palms (Stress assessment)

Monitoring exposed soil (VRT application)

Quantification of FFB, Estimation of mature fruits

Calculating yields for each palm from the acquired images

Yield monitoring

Creation of yield maps

Chlorophyll analysis

Drought assessment

Biomass indication

Leaf area index

Growth monitoring

Weed detection

Inventory management decision support systems

Yield Model Development

Academic and Research application




Next Prev

Research phases

1. Platform setup, integrating UAV, sensors and software

2. Creating high quality 2D and 3D maps of the area

3. Developing custom-built programs/algorithms for smart inventory management

4. Developing custom-built programs/algorithms for smart Health/growth assessment

UAV Setup

Flight Preparation

Test and trials

Mission planning

Image acquisition

Video streaming

Camera/Sensor

setups

Calibration

Image processing

Creating 2D, 3D,

GIS, NDVI Maps

(Pix4Dmapper,

Agisoft)

Image/Map

interpretation

GIS analysis

Custom software

Correlation

analysis

Management

strategies

Decision makings

Reports

generation

UAV images need to be photogrammetrically processed and translated into accurate 2D

orthomosaics and maps, 3D models and surface models, and other GIS datasets

Orthomosaics, 3D Models and Digital Surface Models, 3D Flythrough Videos,

Multispectral Image Mosaics, Index Maps, (i.e., NDVI) needs to be processed/interpreted

DATA

COLLECTION

Data

Processing

Mapping

Modeling

Data analyzing

Results /

Reports

Methodology steps




Next Prev

UAV Platform Sensors

(Cameras) Application

Software

Programming

(Processing)

Multi-rotor Fixed wing

Multi-rotor UAVs

launch and land vertically

are favored where space is tight

Fixed-wing UAV

Requires suitable space to launch and land

Can provide longer flight duration

Can carry a heavier payload.

In-flight stability, flight-time duration

and payloads are major paramount

concerns

RGB

NIR

NDVI

Thermal

Multispectral

Hyperspectral

LiDAR

Onboard

GPS Auto flight

Controller




The quadrocopter UAV,

model md4-1000

DJI Phantom, a small

hobbyist quadcopter

MōVI M10 - Digital 3-Axis Gyro-

Stabilized , Price: $4,995

Fixed Wing, ebee Low-cost: affordable by oil palm growers

Modular sensor system (shifting different sensors)

Higher flight time, more ha coverage

Payload

Mobile applications

Data sharing

Wireless networks

Commercial UAVs are expensive,

Not designed for operation inside oil-palm plantations

Might need protection frames

Factors to be considered

Built from scratch UAV

Quadcopter micro UAV

'microdrones md4-200' with

the prototype MSMS

multispectral sensor

The camera system on

board the Oktokopter

Setting up a UAV

Purchase, Adapt, Or Build from Scratch?

Next Prev Redmond Ramin Shamshiri, 2016. UAV for Oil Palm Precision Agriculture. https://florida.academia.edu/RaminShamshiri



Next Prev

Sony A6000 Visible Sensor Focus Points: 179 True Resolution: 24.3Mp Pixel Size: 15.1 µm² ISO: 1347 ISO Image Quality: 82 Dynamic Range: 13.1 EV Colour Depth:24.1 bits Resolution (GSD):1.5cm – 4.5cm

Lumix LX7 Sensor Details True Resolution:10.1Mp Aperture Range:F1.4 - F2.3 Max Shutter Speed: 1 second ISO:80 - 6400 ISO Image Quality: 82 Resolution (GSD): 5cm @400ft Format:RAW, JPG Image Stabilisation:roll & anti-shake

Visible (RGB)

Lumix LX7 Infrared Sensor True Resolution:10.1Mp Aperture Range:F1.4 - F2.3 Max Shutter Speed: 1 second ISO:80 - 6400 ISO Image Quality: 82 Resolution (GSD): 5cm @400ft Format:RAW, JPG Image Stabilisation:roll and anti-shake

Canon S110-NIR, 12 MP, adapted to be controlled by drones autopilot Acquires image data in the NIR band Resolution: 12 MP Ground resolution at 100 m: 3.5 cm/px Sensor size: 7.44 x 5.58 mm Pixel pitch: 1.86 um Image format: JPEG and/or RAW

Healthy plants have a

strong near infrared

reflectivity, called the

"Red Edge".

Infrared\ Near Infrared




Next Prev

−1 ≤ 𝑁𝐷𝑉𝐼 =𝑋%− 𝑌%

𝑋%+ 𝑌%≤ +1

SUN

Y%

X%

Normalized Difference Vegetation Index (NDVI): measurement of the amount of live vegetation in an area

RGB NIR NDVI

NDVI<0 : Dead plants NDVI< 𝟎. 𝟑𝟑 Unhealthy plants 0.33< NDVI< 0.66: Healthy plant NDVI>0.66 Very healthy plants

NDVI - TWIN LUMIX LX7 low cost, rugged, high resolution imaging solution for NDVI, agricultural and archeological data analysis. provides true R and IR information from different sensors, providing the clean photogrammetric information required for vegetation anaysis processing. Combined with a 5cm resolution at 400ft this sensor provides a host of benefits for the agronomist and archeaologist . True Resolution:10.1Mp, Aperture Range:F1.4 - F2.3 Max Shutter Speed: 1 second, ISO:80 - 6400 ISO Image Quality: 82, Resolution (GSD): 5cm @400ft Format:RAW, JPG, Image Stabilisation:roll and anti-shake

Canon PowerShot SX260 12.1 MegaPixel Digital Camera Models: XNiteCanonSX260: UV+Visible+IR XNiteCanonSX260: IR- Only XNiteCanonSX260NDVI: 3-Band Vegetation Stress Remote Sensing Camera




Next Prev

High Resolution Uncooled Thermal

Camera: Tau 640

Optris PI450 Temperature range: -20°C to 900°C Spectral range: 7,5 bis 13 µm Optical resolution: 382 x 288px Frame rate 80 Hz capture single images at a rate one per 2 seconds. Each pixel from each image has an exact temperature associated with it.

High Resolution thermal imaging can assist

disease detection and water stress in Oil Palms,

or for scouting at nights, fire hazard alarm

Thermal Sensors Multispectral Sensor

To identify Oil Palm stress factors, soil types, fertilizers, or

insecticides

To identify differentiate plant species or recognize other plant

(weeds, etc),

To identify soil or chemical conditions that are, in each case, able

to be identified by their unique spectral signature.

To graphically illustrate vegetation indices such as NDVI that are

defined by relationships of specific narrow-band wavelengths.

With each exposure, 4 or 6 separate bands of visible or near-

infrared radiation move through each camera's lens and filter to

form a separate monochromatic image on the camera's sensor.

Tetracam MCA6 contains 4 or 6 factory-aligned multi-spectral cameras Each sensor captures a 5.2 Megapixel image on the MCA4 and 7.8 Megapixel images on the MCA6 User selectable filters PixelWrench2 software allows for viewing and analysing the images. contains 4 or 6 factory-aligned multi-spectral cameras.




Next Prev

Topographic Survey and Mapping

Contour Mapping

Cross Section / Longitudinal Analysis

3D Mapping

Floodplain Mapping

Vegetation Mapping

Shoreline Analysis

Corridor / Route Studies

Slope Analysis

3D Point Cloud of LiDAR

Ortho-mosaic of Aerial Photo

Light Detection and Ranging (LiDAR)

Potential Application

Palm height measurements




Next Prev

Routescene LidarPod




Orthomosaics 3D Models Vegetation, Health, growth

analysis

Convert thousands of UAV images into

Geo-referenced 2D maps

Orthomosaics models

3D surface models

point clouds

SampleVideoPIX4D

8700 USD unlimited use in time

3500 USD /year (rent)

350 USD /month (rent)

Price


https://www.youtube.com/watch?list=UUyFK7-Y8ZyPF-5Ws42Jt6DQ&v=RHyH-EOYlKA



Process thousands of aerial images

Suitable for a non-specialist operator

Generate high- resolution Geo-Ref

orthophotos

Exceptionally detailed Geo-Ref DEMs*

Fully automated workflow

Easy integration to the Q-Pods system

Create projects using more than one

camera and process imagery together

(i.e., NIR and RGB) * DEM: Digital Elevation Model

SampleVideoAgisoft

Price: £400.00


https://www.youtube.com/watch?v=ETq9xoVYeZY



Next Prev

Creating smart data for decision support

systems

Images source: Adapted from Terracentra

GIS tasks

Essentials of GIS & Aerial Image Interpretation

Map Generation

Creating Workflows

Management of Spatial Data

Natural Color Images

Multi-spectral Images

Digital Elevation Models

Multi-temporal Images




Next Prev





Plantation Infrastructure Inventory, road mapping,

inventory and monitoring is very important for

efficient plantation management.

Fertility Mapping, detecting and mapping

of oil palm fertility or palm vigorous

growth level

Identifying unhealthy palms




Next Prev

Estimated/Precise Palm tree counts in a selected area of interest

Plot size: (ha)

Palm counts: 2500

Palm density: 100 trees/ha




Finding average

distance of a plot to

the river/road




Identification of poor spots examples




Calculating total unplanted area from Geo-referenced maps

S1 S2

S3

S4

S5

S6

Total Unplanted Area = 𝑺𝒊𝒏𝒊=𝟏





Next Prev

PLOT B

Tree Counting methods

Training rules Commercial Software Other techniques




Next Prev

Training Area = 0.9 Sqr.Inch

Training Count = 4

Study Area=10.3305 Sqr.Inch

Estimated count= ?

Estimated count = Factor × Training count

= 45.9

1

2

3

4

5

7

19

21 10

11

12

13

14

15

16

17

18

25 20

46

22

23

24

9

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

8

6

0.5 0.4

= ---------- = 11.47 Factor = Study Area

Training

Area

______

0.9

10.33

Accuracy: 97.6%

Manual count ≅ 47




Next Prev

open-source software QGIS The Leading Open Source Desktop GIS

Banana plantation in Indonesia

Pineapple plantation in the Philippines

The software allows plant counting, density

calculations and the generation of mortality maps by

visual inspection of the image products.


http://qgis.org/



Next Prev

ERDAS IMAGINE's Spatial Modeler (NIR thresholding)

RGB, NIR NIR LPF Image

EDF Image Threshold Image




Next Prev

The overall performance (detection rate)

between 0.916 to 0.998.




Next Prev

photogrammetric point clouds

Digital surface model

Local maximum display tree position

Photogrammetric point clouds Technique




The mapping accuracy amounts for 86.1% for the entire

study area and 98.2% for dense growing palm stands.

Photogrammetric point clouds Technique




Next Prev

Correlating between palm heights, ages, and yield




Next Prev

Crown (Canopy) Volume

Crown diameter




Next Prev

Correlating between image and mass of FFB




Next Prev

X=% of Indexed pixels, No. of FFB, etc

Y=

We

igh

t (K

g)

FFB detection/quantification Geo-referenced




Next Prev

Autopilot and mission control

Visual servo control mechanism for FFB detection

Breakthrough innovative ideas (i.e., Night-mission flights)

Making smarter UAV platforms that can learn while flying

Techniques for Improving accuracy and resolution

Development of customized sensors with built-in algorithms for specific task

Improvement of low altitude flight mission

Improvement of flight control over actuator limits and noise (i.e., Controller

Design for Stabilizing of an Autonomous fixed wing Crop Surveillance

Osprey Drone with Actuator Limits and Sensor Noise)




Next Prev

Reducing labor force on field

Developing GIS dabase, 2D, 3D, NDVI, and thermal maps

Reducing labor hazards

Reducing management time

Palm tree tagging

Monitoring fungal disease with different sensors

Potential to be extended to other fields in Malaysia, i.e., rice and rubber

Further contribution

Smart pesticide control

Enhanced pollination

Constant track and record of growth condition

Drastically help growers in decision making

Early warnings for disease

A ground for autonomous robotic harvesting

Academic application

Rubber plantation




Next Prev

FFB quantification is the first step toward building cost-effective robotic

harvesting system for existing palm trees

UAV can contribute to mechanization of Oil Palm Agriculture

Re-design

Building the

prototype Simulation

Field

experiments

Evaluation

Improvement

Preliminary

design

Remote

API

Serial

port

Plugins

ROS

nodes

Control

Mechanisms

Embedded

Child Script




Next Prev

RGB

NIR NDVI

Palm heights, density,

Crown Diameter, Volume,

FFB quantification, etc

Predicted Yield

Disease detection

Management decisions

Nutrient contents, N, P, K, Mg, B




Next Prev

Link to

draft

Link to

draft


https://www.academia.edu/27922304/Smart_Sensing_Technique_for_Rapid_Estimation_of_Nutrient_Concentration_in_Oil_Palm_Leaf_via_Wavelet_Based_analysis_of_Hyperspectral_data

https://www.academia.edu/27921199/Development_of_a_low-cost_smart_UAV_imagery_with_robust_machine_vision_system_for_Oil_Palm_plantation_Management



Next Prev

PrecisionHawk will presented its drone

platform for early disease detection on

February 4, 2016 at Dubai Internet City.




Next Prev

Picture of an 8 year Oil Palm that produced

535kgs of FFB in one year




(UPM)

(UPM)

(Univ of Florida)

(Univ of Florida)

(Univ of Florida)

(Wageningen UR)

For their insightful suggestions and ideas




A Breakthrough in Oil Palm Precision Agriculture: Smart Management of Oil Palm Plantations with...

Education

Transcript of A Breakthrough in Oil Palm Precision Agriculture: Smart Management of Oil Palm Plantations with...