Amit PuntambekarAmit Puntambekar

Advisor: Dr. Arun LakhotiaAdvisor: Dr. Arun Lakhotia

Team CajunBotTeam CajunBot

www.cajunbot.comwww.cajunbot.com

Terrain Mapping and Obstacle Detection Terrain Mapping and Obstacle Detection for Unmanned Autonomous Ground for Unmanned Autonomous Ground Vehicle Without Sensor StabilizationVehicle Without Sensor Stabilization

October 20, 2006October 20, 2006

Presentation OverviewPresentation Overview

► Introduction and motivation Introduction and motivation ►Related workRelated work►Terrain mapping and obstacle Terrain mapping and obstacle

detection algorithmdetection algorithm►Sensor error handlingSensor error handling►Algorithm evaluationAlgorithm evaluation►Conclusion and future workConclusion and future work

Estimated presentation time: 50 minutes

DARPA Grand ChallengeDARPA Grand Challenge

►HistoryHistory► Autonomous Autonomous

Ground RobotsGround Robots► Application of AGV’sApplication of AGV’s► ExamplesExamples

Top: Mars Rover by NASA, Bottom: iGator by iRobot

DARPA Grand ChallengeDARPA Grand Challenge

►Grand Challenge Grand Challenge 20042004

►Grand Challenge Grand Challenge 20052005

Components of Autonomous Components of Autonomous SystemSystem

►Hardware – Sensors, Electronics, etc.Hardware – Sensors, Electronics, etc.►SoftwareSoftware

Obstacle Detection

Path Planning

Steering

ObstaclesObstacles

►Man MadeMan Made

►NaturalNatural

Types of ObstaclesTypes of Obstacles

► Static - Rocks, Cones, Static - Rocks, Cones, Steep Slopes, etc.Steep Slopes, etc.

► Dynamic – Moving Dynamic – Moving Cars, Gate, etc. Cars, Gate, etc.

► Negative Obstacles – Negative Obstacles – Ditches, Potholes, Ditches, Potholes, etc.etc.

MotivationMotivation

►Timely Obstacle DetectionTimely Obstacle Detection-Top speed of vehicle: 25 mi/hr (11.17 m/s).-Top speed of vehicle: 25 mi/hr (11.17 m/s).

-Even a second delay in detecting obstacle might -Even a second delay in detecting obstacle might be fatal.be fatal.

►Static and Dynamic ObstaclesStatic and Dynamic Obstacles

►Negative ObstaclesNegative Obstacles

SensorsSensors

►GPSGPS Position informationPosition information

► INSINS OrientationOrientation

►LIDARLIDAR RangeRange

GPS/INSGPS/INS

► Principle of Principle of OperationOperation

►Data Format Data Format ► Erroneous Erroneous

ConditionsConditions

100 Hz

Position

5 Hz

Position +

Orientation

GPS INS

LIDARLIDAR

0 degree0 degree

180 degree180 degree

LIDAR TerminologiesLIDAR TerminologiesLaser Beams

RangeRange► LIDARLIDAR BeamsBeams

ScanScan

• 1 Scan = 180 1 Scan = 180 beamsbeams

• 75 Scans per Sec75 Scans per Sec

Time Time StampStamp

LIDAR MountingLIDAR Mounting

Parallel to GroundParallel to Ground

Used by CMUUsed by CMU

Minimum obstacle Minimum obstacle sizesize

Slopes as obstaclesSlopes as obstacles

Sensitive to Sensitive to

VibrationsVibrations

MountingsMountings

Air PressureAir Pressure

LIDAR MountingLIDAR Mounting

► Sweeping the Sweeping the terrainterrain Scans sweep terrainScans sweep terrain Successive scans are Successive scans are

geographically closegeographically close

Consecutive scans on flat Consecutive scans on flat groundground

LIDAR MountingLIDAR Mounting

►Vertical MountingVertical Mounting Team GRAYTeam GRAY Data Data

DiscontinuityDiscontinuity

►Combination MountingCombination Mounting

Algorithms for Sweeping Algorithms for Sweeping LIDARsLIDARs

► Consecutive scan Consecutive scan analysisanalysis The laser scans incrementally sweep the The laser scans incrementally sweep the

surfacesurface Analyzing the consecutive scans to Analyzing the consecutive scans to

determine change in geometry of the terraindetermine change in geometry of the terrain

► Data DiscontinuityData Discontinuity Detect Discontinuity in dataDetect Discontinuity in data Team GRAY, METIORTeam GRAY, METIOR

► Plane FittingPlane Fitting Best fitting plane computationBest fitting plane computation Virginia TechVirginia Tech

► Slope ComputationSlope Computation Change in slope of the scans is computedChange in slope of the scans is computed CMU, Team ENSCOCMU, Team ENSCO

Prior Work ReviewPrior Work Review

►Dependent on Dependent on Incremental scan sweepingIncremental scan sweeping

►Flat terrainFlat terrain

Sensor mountingsSensor mountings►Will not work if sensor mounting is changedWill not work if sensor mounting is changed

Off-Road Conditions- BumpsOff-Road Conditions- Bumps

► Indoor Vs. Off-Indoor Vs. Off-road road EnvironmentsEnvironments

Effect of BumpsEffect of Bumps

Scattered scans due to bumpsScattered scans due to bumps

11

2233

44

Sensor StabilizationSensor Stabilization

► Specific Sensor Specific Sensor StabilizersStabilizers

► Vehicle SuspensionsVehicle Suspensions

Top: Sandstorm from CMU; Bottom: IRV from Indiana Robotics

22 out of the 23 2005 Grand Challenge finalist team had vehicle suspensions or hardware sensor stabilizers to mitigate bumps.

Teams like CMU, IRV had both

CajunBot was the only entry without sensor stabilizer and suspensions

Sensor StabilizersSensor Stabilizers

► CostCost - The cost of the CMU Gimbal - The cost of the CMU Gimbal

is approximately $70,000.is approximately $70,000.

► Single Point of Single Point of FailureFailure

Research ContributionResearch Contribution

►Off-Road Obstacle Detection SystemOff-Road Obstacle Detection System Without sensor stabilizationWithout sensor stabilization Not sensitive to sensor mountingsNot sensitive to sensor mountings Accounts for GPS errorsAccounts for GPS errors Scales well with number of sensorsScales well with number of sensors

Core AlgorithmCore Algorithm

►Obstacle Detection AlgorithmObstacle Detection Algorithm TheoryTheory Implementation for a real time system – Implementation for a real time system –

CajunBotCajunBot Error HandlingError Handling

Algorithm TheoryAlgorithm Theory

PointsPoints Triangle FormationTriangle Formation Slope Slope ComputationComputation

Algorithm Theory Algorithm Theory (Continued)(Continued)

High Absolute SlopeHigh Absolute Slope

High Relative SlopeHigh Relative Slope

Height DiscontinuityHeight Discontinuity

ObstacleObstacle

Triangle AnalysisTriangle Analysis

Obstacle DetectionObstacle Detection

► High Absolute SlopeHigh Absolute Slope- - Large surfaces where triangles Large surfaces where triangles

can can be formed be formed Eg: Wall, cars, etc.Eg: Wall, cars, etc.

► High Relative SlopeHigh Relative Slope- - Obstacles on slopeObstacles on slope- When obstacles are not large - When obstacles are not large

enough to register enough to register three LIDAR three LIDAR beams to form beams to form trianglestriangles

- Negative obstacles- Negative obstacles

► High Elevation ChangeHigh Elevation Change- - Narrow obstacles like polesNarrow obstacles like poles

- - Negative ObstaclesNegative Obstacles

angle

Top: Virtual Triangle on a wall like obstacle

Bottom: Obstacle on a slope

Real Time SystemReal Time System

+Data Fusion

(Range, Angle),75

Position

. . . .. . . .

. . . .. . . .

. . . .. . . .

. . . .. . . .

Slope ComputationSlope Computation

Pos +Pos +

OrientatioOrientationn

Effect of BumpsEffect of Bumps

Scans Scattered due to Scans Scattered due to bumpsbumps

Consecutive scans might be Consecutive scans might be geographically far apartgeographically far apart

11

2233

44

Spatial GridingSpatial Griding

+

.. . .. ....

…… .. .. .

.. .. .... .

. .. .. ..Slope Grid

Data Fusion

(Range, Angle),75

Position, Location

Position

. . .. . .

. . . . ..

. . .. . .

. . .. . .

Slope computationSlope computation

Data ConsistencyData Consistency Temporal Temporal

accuracy of GPS accuracy of GPS GPS DriftGPS Drift

11

22

33

44

Sensor Error - GPS DriftSensor Error - GPS Drift

►What is GPS What is GPS Drift ?Drift ? Gradual drift in the GPS dataGradual drift in the GPS data

►Effects of GPS Effects of GPS Drift?Drift? Only temporally close data Only temporally close data

can be comparedcan be compared

►Factors causing Factors causing GPS DriftGPS Drift Hardware and connectivity Hardware and connectivity

with satelliteswith satellitesGraph: GPS Z Vs. Time

Stationary

Moving

Data Collected on a flat parking lot. Vehicle traveling at 3m/s

X Axis: Time (s)

Y Axis: Height (m)

0.130.13

0.200.20

Handling GPS DriftHandling GPS Drift

►Temporal Temporal Data OrderingData Ordering

►GPS stable for GPS stable for 3-4 seconds3-4 seconds

Obstacle DetectionObstacle DetectionObstacle Cell AnalysisObstacle Cell Analysis

Absolute SlopeAbsolute Slope

Relative SlopeRelative Slope

Height Discontinuity Height Discontinuity

Terrain Obstacle Map (TOM) GridTerrain Obstacle Map (TOM) Grid

Obstacle Detection – TOM Obstacle Detection – TOM Grid AnalysisGrid Analysis

…… .... …… ....

.. .... ……....

.... .. .. ..

.. …… ....

Terrain Obstacle Map

► High Absolute SlopeHigh Absolute SlopeAbsolute Orientation > Absolute Orientation >

ThresholdThreshold

► High Relative SlopeHigh Relative SlopeDifference in Orientation > Difference in Orientation >

Threshold Threshold &&Difference in Height > ThresholdDifference in Height > Threshold

► High Elevation High Elevation ChangeChange

Difference in Height > ThresholdDifference in Height > Threshold

(Max Orientation,

Min Orientation,

Max Height,

Min Height,

Num of Centroids,

Num of Hits)

Potential Obstacle Determination

Confidence Factors

Dynamic ObstaclesDynamic Obstacles

►Dynamic ObstaclesDynamic Obstacles registered as registered as

obstacle at every obstacle at every locationlocation

Refresh GridRefresh Grid►Grid RefreshingGrid Refreshing

- TOM in a spatio-- TOM in a spatio-temporal temporal grid grid

- Refresh TOM Cells if - Refresh TOM Cells if existing data existing data

and new data and new data are not are not temporally closetemporally close

- Aging based on - Aging based on access time access time stamp stamp

…… .... …… ....

.. .... ……....

.... .. .. ..

.. …… ....

Last Access Time

Stamp,

T_old

New Data, T_new

Terrain Obstacle Map (TOM)

Sensor ErrorSensor ErrorGPS SpikeGPS Spike

►What is GPS Spike ?What is GPS Spike ?-Sudden change in -Sudden change in

the GPS data in a very the GPS data in a very short time interval.short time interval.

-The elevation data -The elevation data is more prone to GPS is more prone to GPS SpikesSpikes

► Causes for GPS Causes for GPS SpikeSpike

-Weak Signal-Weak Signal

-After ‘Dead -After ‘Dead Reckoning’Reckoning’

X-axis : Time(s)

Y-axis: Height(m)

Graph: GPS (Z) Vs. Time

NQE 2005 Data

3030mm

Effect of GPS Spike Effect of GPS Spike

GPS Spike Data PlaybackGraph: GPS Z Vs. Time

GPS Spike: Reason for False GPS Spike: Reason for False ObstaclesObstacles

Corrupted data enters systemCorrupted data enters system

Slope computation gets Slope computation gets erroneouserroneous

Data FilterData Filter

Detecting GPS SpikeDetecting GPS Spike

Median filter monitors Median filter monitors INS DataINS Data

Erroneous data is Erroneous data is discardeddiscarded

Core Algorithm- RevisedCore Algorithm- Revised

► Terrain ModelingTerrain Modeling►Obstacle Obstacle

DetectionDetection

Effect of Bumps - IIEffect of Bumps - II

► INS, LIDAR data fusionINS, LIDAR data fusion-Mounting INS on -Mounting INS on LIDAR LIDAR-Good Suspensions-Good Suspensions-Sensor Stabilizers-Sensor Stabilizers-Rigid Platform-Rigid Platform

LIDARS

GPS

INS

Rigid Platform

Effects of BumpsEffects of Bumps

► INS, LIDAR Data Rate MismatchINS, LIDAR Data Rate Mismatch

t1 t3

Time

AngleLIDAR

INS00

00 XX

t2t2

Sensor FusionSensor Fusion

► CBWare - Data CBWare - Data interpolation interpolation supportsupport

► Robots with Robots with sensor stabilizers sensor stabilizers can fuse the most can fuse the most recent data from recent data from sensorssensors

► In CajunBot data In CajunBot data is interpolated is interpolated based on time of based on time of productionproduction

t1 t2

Time

Angle

LIDAR

INS

XX

InterpolatioInterpolationn

Algorithm EvaluationAlgorithm Evaluation

►Ability to utilize bumps to see furtherAbility to utilize bumps to see further►Accuracy of resultsAccuracy of results►Algorithm complexityAlgorithm complexity►ScalabilityScalability►Different obstacle typesDifferent obstacle types►Sensor orientation independenceSensor orientation independence

Data SetsData Sets

►Logged data from 2005 GCLogged data from 2005 GC►Testing in a controlled environment Testing in a controlled environment

with CajunBot-IIwith CajunBot-II►Testing in a simulated environment - Testing in a simulated environment -

CBSimCBSim

Evaluation – Effects of BumpsEvaluation – Effects of Bumps► Obstacle detection Obstacle detection

distance increases linearly distance increases linearly with severity of bumps with severity of bumps experiencedexperienced

Testing in a controlled Testing in a controlled environment with CajunBot-II environment with CajunBot-II

Effect of bumps Effect of bumps

With With BumpsBumps

Without Without BumpsBumps

DistanceDistance 42.6 m42.6 m 28.5 m28.5 m

False False ObstaclesObstacles

NILNIL NILNIL

Experimental Setup Obstacle detection without Bumps

Obstacle detection with bumpsComparison table

ScalabilityScalability

Sensor Specific Computation

Data Specific Computation

Results based on analyzing CajunBot-II logged data on a Dell machine with 3.2 GHz Intel

Processor and 1 GB RAM with full load (all other CajunBot software modules running) on Fedora

Core 2 operating system

Scalability and Bumps Scalability and Bumps

Results based on analyzing the 2005 GC final run logged data on a Dell machine with 1.6 GHz Intel Processor and 1 GB RAM with full load (all other CajunBot software modules running) on Fedora Core 2 operating system

Sensor Orientation Sensor Orientation IndependenceIndependence

CajunBot with two distinct sensor orientations

Run 1Run 1

Top sensor Top sensor

Orientation (r, p, h)=(0, -1.5, Orientation (r, p, h)=(0, -1.5, 1)1)

Offsets (X, Y, Z)=(0.25, 1, Offsets (X, Y, Z)=(0.25, 1, 0.25)0.25)

Bottom Sensor Bottom Sensor

Orientation (r, p, h)=(0, -3, Orientation (r, p, h)=(0, -3, 1.5)1.5)

Offsets (X, Y, Z)=(0.25, 1.5, -Offsets (X, Y, Z)=(0.25, 1.5, -0.5)0.5)

Run 2Run 2

Top sensor Top sensor

Orientation (r, p, h)=(2, -4.5, Orientation (r, p, h)=(2, -4.5, 0)0)

Offsets (X, Y, Z)=(0, 1, 0.5)Offsets (X, Y, Z)=(0, 1, 0.5)

Bottom Sensor Bottom Sensor

Orientation (r, p, h)=(-2, -4, Orientation (r, p, h)=(-2, -4, 2)2)

Offsets (X, Y, Z)=(0, 2, 0)Offsets (X, Y, Z)=(0, 2, 0)

Comparison with different Comparison with different obstacle shapes at varying obstacle shapes at varying

speedspeed

HD: Height Discontinuity, AS: Absolute Slope, RS: Relative Slope

Considerable increase in speed, negligible decrease in efficiency

4%, 3.2%, 3.9% decrease in detection distance among the three shapes when the speed increases by 150 %

Limitations of the algorithmLimitations of the algorithm

►High precision INS requiredHigh precision INS required►Sensitive to Boresight MisalignmentSensitive to Boresight Misalignment

Angle at which LIDAR is mounted w.r.t INSAngle at which LIDAR is mounted w.r.t INS Fusion data from multiple LIDARsFusion data from multiple LIDARs

►Limited LIDAR reflectivityLimited LIDAR reflectivity WaterWater Black surfaces – tar roads, etc.Black surfaces – tar roads, etc.

ConclusionConclusion

An Obstacle Detection system An Obstacle Detection system Without sensor stabilizationWithout sensor stabilization

►Takes advantage of bumps to see furtherTakes advantage of bumps to see further Not sensitive to sensor mountingsNot sensitive to sensor mountings Accounts for GPS errorsAccounts for GPS errors Scales well with number of sensorsScales well with number of sensors Handles dynamic obstaclesHandles dynamic obstacles

Efficient Spatio-Temporal Grid Efficient Spatio-Temporal Grid Evaluation of systemEvaluation of system

On logged data, live on CajunBot-II, and On logged data, live on CajunBot-II, and simulatorsimulator

Future WorkFuture Work

►Dynamic obstacle detectionDynamic obstacle detection Detecting trajectory and speedDetecting trajectory and speed

►Obstacle ClassificationObstacle Classification Vegetation, mesh, etc Vegetation, mesh, etc

Questions ?Questions ?

Thank youThank you

Backup SlidesBackup Slides

Boresight Angles / OffsetsBoresight Angles / Offsets

H

Hs

GPS

x

p

n

y

INS

Body on which the sensor is mounted

Laser sensor

Laser beam

Actual Ground Line

Computed Ground Line

R

P

Limitations of the algorithmLimitations of the algorithm

►High precision INS requiredHigh precision INS required►Sensitive to Boresight MisalignmentSensitive to Boresight Misalignment

Angle at which LIDAR is mounted w.r.t INSAngle at which LIDAR is mounted w.r.t INS Fusion data from multiple LIDARsFusion data from multiple LIDARs

►Water, black surfaces do not reflect Water, black surfaces do not reflect LIDARLIDAR

LIDAR ErrorsLIDAR Errors

► Boresight Boresight MisalignmentMisalignment

L1

L2

L3

L4

L5

L7

L6

Effect of Mounting Angles on Sensor Data

Sensors UsedSensors Used

► LIDARsLIDARs

►GPSGPS

► INSINS

GPS/INSGPS/INS

► Principle of Principle of OperationOperation

►Data Format Data Format ► Erroneous Erroneous

ConditionsConditions

100 Hz

Position

5 Hz

Position +

Orientation

GPS INS

LIDARSLIDARS► Principle of Operation Principle of Operation

– Time of Flight– Time of Flight►Data Format – 75 Hz Data Format – 75 Hz

at 0.25 degree offsetat 0.25 degree offset► Erroneous ConditionsErroneous Conditions

Laser Beams

Data HandlingData Handling

► To keep multiple To keep multiple copies of 2 minute copies of 2 minute worth of data in worth of data in memory would memory would require 550 MB of require 550 MB of RAMRAM

► Updating multiple Updating multiple copies is expensivecopies is expensive

Data HandlingData Handling

LIDAR TerminologiesLIDAR Terminologies

LIDAR BeamsLIDAR Beams

. . . . . . . . . . ..

Laser ScanLaser Scan

1 scan 1 scan == 180 beams180 beams

75 scans per sec75 scans per sec