Institute of Photogrammetry and Remote Sensing … · Institute of Photogrammetry and Remote...
Transcript of Institute of Photogrammetry and Remote Sensing … · Institute of Photogrammetry and Remote...
Geomatics PhD school, Finland, 2011 1
Vienna University of Technology Institute of Photogrammetry
and Remote
Sensing
Norbert Pfeifer
Geomatics PhD school, Finland, 2011 2
Geomatics on the move
• to moving
measurement
platforms
• from
single
to multi‐sensor
systems
• to higher
autonomy
and automation
• to new
fields
of applications
Geomatics PhD school, Finland, 2011 3
Today‘s program
• 9:00 ‐
9:45
Experiences from assembling and operating a UAV
• 9:45 ‐
10:15
A UAV application in ecology
• 10:15 ‐
10:30
Discussion
• 10:30 ‐
11:00
Break
• 11:00 ‐
12:30
Lidar
introduction
and beyond
• 12:30 ‐
13:30
Lunch break
• 13:30 ‐
14:00
Lidar
on UAVs
survey
(only
15‘)
• 14:00 ‐
15:00
Lidar
point clouds
and their
use
in vegetation
mapping
• 15:00 ‐
15:30
Break
• 15:30 ‐
16:45
Lidar
point clouds
for
geomorphology
• 16:45 ‐
17:00
Discussion
but with morebreaks
Geomatics PhD school, Finland, 2011 4
TU Wien
• Vienna University of Technology• Mission “technology for
people”
• 23,000 students, 5000 academics• Faculties
– Mathematics
and Geoinformation– Physics– Informatics– Electrotechnical
Engineering– Civil Engineering– Architecture
and Spatial
Planning– Mechanical
Engineering– Chemistry
• Geoinformation institutes– Photogrammetry
and Remote
Sensing– Geodesy
and Geophysics– Geoinformation and Cartography
Geomatics PhD school, Finland, 2011 5
Institute of Photogrammetry and Remote
Sensing
–
I.P.F.
Prof. W. Wagner (Head), Professor in Remote Sensing• Radar Remote
Sensing
• Physical
aspects
of Laser ScanningProf. N. Pfeifer, Professor in Photogrammetry• Photogrammetry• Geometrical
aspects
of Laser Scanning
Assoc. Prof. J. Jansa• Optical
Remote
Sensing
• Digital Image ProcessingStaff: ~50 (incl. ~12 faculty staff)• Geodesists, geographers,
environmental
engineers, physicists,
computer
scientists
5
Geomatics PhD school, Finland, 2011 6
Assembling and operating a UAV
Christian BriesePhilipp Glira
Norbert Pfeifer
Institute of Photogrammetry and Remote
Sensing
Vienna University of Technology
Geomatics PhD school, Finland, 2011 7
Assembling and operating
a lightweight
UAV
• Overall goal Direct
georeferencing
of UAV imagery
using
on‐board
components
• Secondary
goal−
learn
about
UAVs
−
enter
geomatic
UAV research
• Why? / Motivation−
Georeferencing
−
UAV
−
Specific
field
of applications
• How?−
Photogrammetric
Theory
−
Navigation
−
Electronics etc.
• How
precisely? / Combating
problems
Geomatics PhD school, Finland, 2011 8
Motivation: direct georeferencing
Georeferencing• Each
sensor
measures
in its
own
sensor
coordinate
system. • Transformation between
sensor
coordiante
system
and superior coordinate
(reference) system
has
6 independent parameters: exterior
orienation
(pose, position
and angular
attitude, …)• Indirect
georeferencing: Using
control
points
(reference
frame) and the
observation
equations
from
the
sensor
(collinearity, range, …) the
exterior
orientation
of the
sensor
coordinate
system can
be
estimated. Kraus, 2004. Photogrammetrie, Dümmler.
Geomatics PhD school, Finland, 2011 9
Georeferencing
Georeferencing
• Direct
georeferencing: using observations
from
other
(additional) measurement devices, the
exterior
orientation
of the
sensor
can be
„directly“
observed.
ikv
working
group
at the
Institut für Geodäsie der
Universität der Bundeswehr, Munich
Heine, 2004. Hydrografische Vermessung von
Gewässersohlen von kleinen und mittelgroßen
…
Geomatics PhD school, Finland, 2011 10
Direct georeferencing: pros+cons
• Independent observation– not
affected
by
insufficient
sensor
model
(calibration)
– not
affected
by
gross
errors
in sensor
measurements
• Precision
not
necessarily
fits
to measurement precision
of sensor
• Reliability: redundancy may be low (GNSS/INS)
• Mounting
and possibly
other as additional unknowns
(constant?)
Kraus, 2004. Photogrammetrie, Dümmler.
Geomatics PhD school, Finland, 2011 11
Direct georeferencing: pros+cons
Figure: Optech
• Indirect
georeferencing
not
always possible
or
feasible
– too
low
redundancy
(no control
points)– dynamic
aquisition
= individual
exterior
orientation
for
each
sensor
measurement– Singularity
in system
of observation
equations
(image taken
over
flat
scene, …)
• Direct
georeferencing
method
needs
to be
adapted
to environment
(indoor
positioning, etc.)
• Additional costs
and complexity
because
of additional sensors
• Bundling
sensor
and direct
georeferencing
on one platform
can
increase
efficiency
(ease
of deployment,
speed
of delivery, …)
Geomatics PhD school, Finland, 2011 12
Georeferencing
• Integrated
georeferencing
is
optimal (provides
exterior
orientation
with
best
accuracy
and reliability)
• One step
after
the
other
…
• Direct
georeferencing
of UAV
imagery
Geomatics PhD school, Finland, 2011 13
Motivation: UAVs• Get
airborne
yourself
(small
costs, little
expertise, …)
• High resolution: low
flying
height
• Deployment: easy
and fast
• Access areas
forbidden
or dangerous
for
humans
• Drawback: missing
legal framework
• Drawback: reduced
human reasoning
in crash situations
• Drawback: limited
payload
for
small
UAVs
• Question: which
UAV ?
www.nanokopter.at
Geomatics PhD school, Finland, 2011 14
UAV: balloons
• Simple to build, cheap, long
endurance
• Affected
by
wind, bulky, cumbersome
to maneuver, number of people
required
(Vierling, 2006)
(Sanswire, 2007)
(Cheng, 2006)
(USCG, 2008)
Geomatics PhD school, Finland, 2011 15
UAV: micro‐planes
• Up to 500m a.g.l., very
small, low
noise
level
(sound)
• Small payload, high speed
required, short
flight
time, expensive
(Cyberflight, 2006 )
(Us Air Force, 2006))
(Tamblyn, 2003 )
(Israel Aircraft, 2003 )
Geomatics PhD school, Finland, 2011 16
UAV: large span planes
• Own
weight: ~0.5kg, windspan>0.5m, long
endurance, autonomous
flying, larger payload, low
noise
• Forbidden
in most
countries, take‐off
and landing
(Cropcam)
(Aerosonde)
(Skylarc)
Geomatics PhD school, Finland, 2011 17
UAV: unmanned helicopters
• Available
in all sizes, easy
to fly
and to transport, payload according
to size, simple start and landing
• vibrations
(Yamaha) (Aeroscout)
(Schiebel)
(Micro Flying Robot, 2003)
Geomatics PhD school, Finland, 2011 18
UAV: motorized parachutes
• Up to 5000m a.g.l., very
long
endurace
(2 days), high payload, slow
but
stable
flight
• Compicated
take‐off, strongly
affected
by
wind
(Yamaha)(Atair) (Atair)
(Thamm, 2006)
Geomatics PhD school, Finland, 2011 19
UAV: HALE
• Flying height: some
km a.g.l., uses
kerosin
or
solar power, large coverage, very
long
endurance
(staying
airborne
over
night!), high payload
• Very
costly
(Global Hawk)
(Luna)
(Luna)
(Luna)
(Aerovironment
Helios
/ NASA)
Geomatics PhD school, Finland, 2011 20
UAV in use at I.P.F./TU Wienwww.mikrokopter.de• Modular system
– Flight
control
(s/w on main
board)– controllers
for
brushless
motors
– Motors– Propellors– Receiver (feeds
flight
control)
– Transmitter– Frame– Battery
• Active
project
(fast help, etc.)• Single parts
can
be
replaced
• Software can
be
adapted
to own
needs• Supports different additional sensors• Electronics expert(ise) needed
www.mikrocontroller.com
Images not
to scale, if
not
stated
otherwise, images
from
www.mikrokopter.de
Geomatics PhD school, Finland, 2011 21
UAV properties
• 4 rotors• 250g payload
/ 650g total mass
• Electronic stabilization
of attitude
by
gyros and accelerometers
• GPS, barometric
elevation
measurement, magnetic
compass
• Camera mount
• Canon IXUS• Cost: €
1600
Geomatics PhD school, Finland, 2011 22
Controlling and flying
Flying a quadro‐kopter
• neighboring
rotors
have
different rotation sense
• Additional thrust
for
one
rotor: tilts
UAV and moves
forward
• Opposing
pairs
with
different speed: yawing
Transmitter
operation
modes
(alternatives)
1.
Access to controllers
of rotors: not
used
for
multi‐copters
2.
Prescribe: Roll, Nick, Yawing
(3 orthog. axes), Thrust, Hold FC gives
corresponding
signal
to controllers
3.
Prescribe: Up, Forward, Sideways, Yawing NaviControl
gives
via FC corresponding
signal
to controllers
Geomatics PhD school, Finland, 2011 23
Controlling and flyingTransmitter
operation
modes
(alternatives)
1.
Prescribe: Roll, Nick, Yawing
(3 orthog. axes), Thrust, Hold FC gives
corresponding
signal
to controllers
2.
Prescribe: Up, Forward, Sideways, Yawing NaviControl
gives
via FC corresponding
signal
to controllers
Degree
of Automation (switched
by
transmitter
signal)
• Prescribe
way‐points, executed
by
NaviControl
via FC
• Manual control
Geomatics PhD school, Finland, 2011 24
Motivation: applicationsDetailed
studies
of natural
states
and processes
• Geomorphology
• Spatial
ecology, spatial
biology, geo‐bio‐sciences, …
• Archeology
Fields relying
often
on
• low‐tech
• experts
in the
field, therefore
detailed
analysis
• (cheap) satellite
imagery
in the
best case
• undergoing
quantification
processes
w.r.t. applied
methods
Therefore, a demand
exists
for
detailed
quantified
geo‐spatial
data.
Geomatics PhD school, Finland, 2011 25
• Flight control (s/w on main board)• controllers for brushless motors• Motors• Propellors• Receiver (feeds flight control)• Transmitter• Frame• Battery
Assembling a UAV
frame +landing
skid
+motors +controllers +cables
to flight
control +flight
control
+rotors+GPS unit +camera
mountImage of first
flight
…
Geomatics PhD school, Finland, 2011 26
First flight …
Geomatics PhD school, Finland, 2011 27
Adaptations
• Camera mount
for
fixel
vertical
view
and less
vibrations
• Landing
rack
adaptation
• Camera– repetetive
exposures
with
constant
time interval
– no zoom, no focus
(time delay!)
Geomatics PhD school, Finland, 2011 28
Camera
Canon IXUS 80• 8 Mpixel, ~6-18mm principle distance• Interior orientation for fixed zoom and focus found
constant within one flight mission (still a hypothesis)• Radial and tangential low order distortion improves
camera model• Short exposure time for sharp images (vibrations)
Canon Hack Development Kit• firmware „update“ of camera• manual control over focus and exposure time• allows remote exposure signals• allows repetetive exposure in constant time interval
Geomatics PhD school, Finland, 2011 29
Second flight …
• Follows
predefined
way (parts
of the
track)
Geomatics PhD school, Finland, 2011 30
Test in a quarry
Model from
images
alone
1. find correspondence hypotheses
(SIFT et al.)
2. evaluate
hypotheses (RANSAC et al.)
3. estimate
orientation
for entire
block of images
4. pairwise
matching
of surfaces
5. triangulation
of point cloud
Unresolved
• datum
(including
scale)
• no safe‐guard
against model
deformations
Geomatics PhD school, Finland, 2011 31Agisoft Photoscan
Geomatics PhD school, Finland, 2011 32
Direct
georeferencing
of UAV imagery
Datastreams
(3)
• Images are taken (stored on camera)
• GPS positions
are
recorded (stored
independently
on miniSD
on Navi
control)
• Acc.+Gyros
sent
from
FC to ground
laptop
(WiFi)
• Synchronization– PPS not
available
– Navi
control
data
stream
recordes
exposure
events
– Acc./Gyro
+ GPS synchronized
via logging
of identical
variable
– Camera exposure
lag constant
Images: Canon, Transcend, Mikrokopter
Geomatics PhD school, Finland, 2011 33
Observation of Position
• GPS data
processed
onboard
by
NaviControl
• EGNOS correction
signal
exploited
by NaviControl
• European Geostationary
Navigation Overlay
Service
• Elevation from barometer/pressure
sensor
(initialization
on the
ground)
Geomatics PhD school, Finland, 2011 34
Observation of angular attitude
• Gyros: observation
of angular
speed … strong drift behavior
(~ degrees/minute)
• Accelerometers: observation
of acceleration …
measure
dynamic
accelerations
+
gravitational
forces
• Drift behaviour
assessed
in lab experiments
g 180°
0°
270°25°
Acceleration in nick angle: g*sin(25°)
Geomatics PhD school, Finland, 2011 35
Gyro+Acc. lab experiments
• Different test series• Complex
test (e.g.)
shifts, rotations
(10°
each), shift, rotations, shift, rotation
Acc.
Gyro
Geomatics PhD school, Finland, 2011 36
Gyro behavior
• Zero level
of gyro
estimated
from
first
100 measurements
(image)• Angle integral strongly
affected
by
initial
zero
position
due
to drift.
Therefore, zero
level
estimates
necessary
during
flight.
Accelerometer
Gyro (angular speed)
Angle (Integral)
Geomatics PhD school, Finland, 2011 37
Modes for observing angles
• Stop
and Go for
each
exposure: only
accelerometer
measurements
in hold position
used
for
angle measurement• Go and some
stops
– Estimate
gyro
drift from
accelerometer– Take exposures
during
flight
– Some
holds
in between
to estimate
zero
level (pull gyro
to acc)
• Optimal solution
quite
obvious, work
in progress
…
Geomatics PhD school, Finland, 2011 38
Direct geo‐referencing
• Fly
over
test field
• Directly
determine
exterior orientation
by
GPS+EGNOS,
gyros, accelerometers, barometer
• Compute
bundle
block from
control
points (in this
case
independent spatial
resections)
Geomatics PhD school, Finland, 2011 39
Test flight
Geomatics PhD school, Finland, 2011 40
Coverage
of photos
for
geo‐referencing
test
Geomatics PhD school, Finland, 2011 41
Geomatics PhD school, Finland, 2011 42
Detail of previous
image.Note sharpness
of images. Note also image abberations.
Geomatics PhD school, Finland, 2011 43
Direct geo‐referencing quality
• Compare
bundle
block from
control
points
to direct
geo‐referencing
• In position: 60cm, in elevation
100cm (1 sigma!)
shift
~ 50cm
• More
experiments
for
validation
of quality figures
missing
• Verification
with
OP missing
Geomatics PhD school, Finland, 2011 44
Problems encountered
• 1st Mikrokopter
sensors
not
sufficiently
accurate• Crash during
landing
caused
defect
in FC
• Defect
motor
controllers
prohibited
experiments• Rotor broke
during
flight, leading
to crash
(2nd model)
• Faulty
bluetooth
module
connecting
UAV and PC• Not all measurement
written
on data
streams
• Functioning
bluetooth
module
replaced
by
WiFi
for
distance• Measurement
rates
deviate
from
prescribed
values
• GPS modules
sometimes
refused
work
mid‐air• Problems putting
new
firmware
on FC
• …
Geomatics PhD school, Finland, 2011 45
I.P.F. Mikrokopter
GPSIMUBarometric sensor Magnetic compass
PilotCompact Camera
Notebook(Wi232 connection)
Technical data
• 4 Propeller
• Weight: 650g
• Payload: 250g
• Flight time: 15min
• Height: up to ~100m
• Range: range of sight
• Costs: ca. 1600€
Uplink: WaypointsDownlink:Sensor data
Roll and Nick compensation
Geomatics PhD school, Finland, 2011 46
Conclusions
• Direct
georeferencing
of UAV imagery
bridges across
a number
of geomatic
fields
• UAV assembly
requires
flying
and electronic expertise
• Direct
georeferencing
of UAV imagery
using on‐board
components
is
feasibile
• Practical
problems
not
to be
underestimated