Slide 1

TKK 12.4.2007 STRS system combining LiDAR and multiple images: Multi-scale template matching and LS- adjustment of a parametric crown model with lidar data in 3D tree top positioning and estimation of the crown shape Species recognition in aerial images Ilkka Korpela (Morten Larsen) Slide 2 Contents (Demos included?) Single-tree remote sensing (STRS) Photogrammetric 3D reconstruction and object recognition Airborne lidar-based 3D reconstruction and object reconstruction Coupling allometric constraints to the STRS problems 3D treetop positioning with template matching (TM) multi-scale TM Species recognition LS-adjustment of crown models with lidar points Conclusions and outlook Slide 3 STRS Single-Tree Remote Sensing Air- or spaceborne; active and/or passive sensing 2D or 3D = with or without tree height Direct estimation of tree/crown position and species; indirect model-based estimation of height, dbh Restrictions: tree discernibility due to scale (detectable object size), occlusion and shading. Alternative or complement to A) field inventory, B) area-based remote sensing Slide 4 STRS Single-Tree Remote Sensing Accuracy restricted by allometric noise alike in volume functions tree and stand level bias, tree level impresicion. dbh ~ 10-12 %. Measurements subject to bias: DTM-errors, lidar does not hit apexes, Dcr underestimation Nothing can be known about quality, only quantity Unsolved issues: species recognition, regeneration stands, calibration and validation in the field, aggregated crowns result in fused trees. Slide 5 Photogrammetric 3D reconstruction and object recognition Usually from N>1 images (multiscopic) Correspondence problem - ill-posed, perspective errors, reflectance, occlusion, scene complexity Texture in the image functions needed Relies on accurate geometry (camera interior, exterior), ray- intersection Slide 6 Photogrammetric 3D reconstruction and object recognition Digital revolution (2000) Aerial images upto 2GB, manageable (I/O, analysis, storage, transfer) Automatic methods in orientation, incl. DSO with GPS/INS, reduction in GCPs. Digital cameras with MS images 2005 multiple images per target, better radiometry and geometry Automatic DSM production Slide 7 Photogrammetric 3D reconstruction and object recognition Automation Laborious orientation tasks solved DSMs, DTMs using image matching e.g. Building extraction - semiautomatic 3913, 3914, 3915 (triplet matching) 1946, 1962 Demo 1 Slide 8 Photogrammetric 3D reconstruction and object recognition Photogrammetric STRS scene and object complexity occlusion & shading scale: h = 0..40 m, Dcr 0..10 m BDRF-effects automation challenging Slide 9 Photogrammetric 3D reconstruction and object recognition Demo manual STRS - NLS 04403 (19) treetop 3D, height, Dcr, Sp dbh = f(Sp, h, Dcr) + epsilon Image matching fails for treetop positioning unless we use a feature detector for treetops. Slide 10 Airborne lidar-based 3D reconstruction and object reconstruction A pulse of short duration (~ 3 m, 1064 nm) Observing returned signal. Discrete data. Upto 128 samples?. Signal is reconstructed into points or samples for later waveform analysis. Intensity of the return/echo. Pros: No texture needed, active no shading, real ease of 3D Cos: discrete sampling, high sampling rates costly, difficult to reconstruct high- frequency relief. Slide 11 Airborne lidar-based 3D reconstruction and object reconstruction Automation DTMs manual assistance needed high accuracy even under canopy Volume estimation of trees automatic e.g. using regression between lidar features and field observations (e.g. Naesset 2004. Suvanto et al. 2005) Lidar and STRS Algorithms that process point clouds directly or interpolated DSMs (CHMs) Underestimation of heights (footprint size, density, crown shape, equipment sensitivity) Species not obtained (so far) Slide 12 Coupling allometric constraints to the STRS tasks If we know dbh, we have an idea of the height If we know dbh, species and age, we have better idea of height If we know dbh, species, and height we have a good idea of volume If we know dbh, species, height and height of crown, we have a better idea of volume. If we know height, species and crown width we can estimate dbh and volume If we know species we have an idea of the shape of the crown envelope. If we know species and height, can we set limits for the variation of crown width f(sp, h) => [Min, Max] of Dcrm, and assume a basic shape? Slide 13 Coupling allometric constraints to the STRS tasks If we know species can we have an idea of the shape of the crown envelope? Timo Melkas Slide 14 Dcrm (min,max) / Shape | (Sp, height) - Consistency of measurements (rule out impossible observations) - Initial approximations for iterative approaches in finding true Dcrm & crown shape E.g Short trees have small crowns (adjust search space accordingly, or look for small crowns from a low height) Coupling allometric constraints to the STRS tasks Slide 15 3D treetop positioning with template matching (TM) Demo 04402 - 3D treetop positioning using TM 1) Use, for each of the N views, a model image (template) of a crown. 2) Compute N normalized cross- correlation images (template matching). 3) Form a Cartesian 3D grid in the canopy in the search space. 4) Aggregate 3D correlation to the grid points. 5) Process the 3D correlation into hot-spots 3D treetop positions. Fine, but not invariant to object variation. Slide 16 Slide 17 Slide 18 Slide 19 Cartesian grid / Search space in the upper canopy Slide 20 Correlation > threshold Slide 21 Slide 22 Multi-scale TM Treetop positioning Can we assume that the optical properties and the relative shape of trees are invariant to their size? I.e. small trees appear as scaled versions of large trees in the images? (inside one species and within a restricted area) Slide 23 Multi-scale TM Treetop positioning Maxima at different scales, take global (X,Y,Z) Slide 24 Multi-scale TM Crown size Demo 04403_19, 18, 20, 06214_3900, 3901 Near-nadir views are best for manual measurement of Dcr (crown width) Slide 25 Species recognition Spectral values Texture Variation: - Phenology - Tree age and vigour - Image-object-sun geometry => reliable automation problematic => bottleneck Slide 26 LS-adjustment of a crown model with lidar points Assume that 1) Photogrammetric Multi-scale TM 3D treetop position is highly accurate 2) Trees have only moderate slant 3) Crowns are rotation symmetric 4) We know tree height and species which give a reasonable approximaion of the crown size and shape LiDAR hits are observations of crown radius at a certain height below the apex Assume a rather large crown and collect lidar hits in the visinity of the 3D treetop position, down to relative height of apprx. 60 %. Use LS-adjustment to find best set of parameters for the crown model. Slide 27 LiDAR hits are observations of crown radius at a certain height below the apex? Slide 28 LiDAR hits are observations of crown radius at a certain height below the apex what if crowns are interlaced? Slide 29 Example - a 19-m high spruce: Solution in three iterations. Final RMSE 0.31 m Note apex! LiDAR did not hit the apex and the crown width at treetop (constant term) is negative. Slide 30 Example - a 22-m high birch: Solution in six iterations. Final RMSE 0.47 m For some reason RMSEs are larger for birch in comparison to pine and spruce. Convergence? Slide 31 Conclusions and outlook A - Multi-scale TM works in a manual semi-automatic way for treetop positioning Possible to automate? Computation costs? (NCC now tried everywhere) - Multi-scale TM in crown width estimation needs comprehensive testing (Image scales, required overlaps) -Species recognition was overlooked here, still I think that good 3D treetop positions can be used for the purpose. - Matching LiDAR points using LS-adjustment works only if the exact treetop position is known. Aggregated crowns are problematic, but these cases are known from the tree map Slide 32 Conclusions and outlook B - If we have a system that can be operated so that a tree measurement takes 2-3 seconds and the measurement inaccuracies are: h ~ 0.6 m Dcr ~ 10% d13 ~ 10-12 % XY-position ~ 0.3 m Sp ~ 95% Is this fast and accurate enough for sample-plot based STRS? Can we afford the images and LiDAR? Slide 33 Calibration and validation of results? Slide 34 Presenting method and early results in ISPRS workshop in Hannover, June 1, 2007 Presentation at Silvalaser, Espoo September 2007? Article about Multi-Scale TM in 3D treetop positioning (ISPRS JPRS?) Article about Multi-Scale TM combined with crown modeling using LS-adjustment and allometric constraints (Silva Fennica 100-yr issue?) Species recognition study remains of the Academy 3-yr resrach project to be completed before VIII/2008