Semi-automatic 3D building model generation from lidar and high resolution imagery

Sohel Syed, Paul Dare and Simon Jones. Semi-automatic 3D building model generation from lidar and high resolution imagery.Proceedings of SSC2005 Spatial Intelligence, Innovation and Praxis: The national biennial Conference of the Spatial Sciences Institute,

September, 2005. Melbourne: Spatial Sciences Institute. ISBN 0-9581366-2-9

SEMI-AUTOMATIC 3D BUILDING MODEL GENERATION FROM LIDAR AND HIGH RESOLUTION IMAGERY

Sohel Syed1, Paul Dare2, Simon Jones3

1,2 Airborne Research Australia, Flinders University, PO box 335, Salisbury South, SA 5106 Tel: + 61 8 8182 4000, Fax: +61 8 8285 6710

3Mathematical & Geospatial Sciences, RMIT University, GPO Box 2476V, Melbourne, VIC 3001, Tel: +61 3 9925 2419,

syed0009@flinders.edu.au

KEYWORDS: 3D building reconstruction, object-oriented classification, lidar, high resolution imagery ABSTRACT Using lidar point cloud data, the orientation and height of building roof faces can be estimated accurately but the outline of a roof face is more difficult to determine. To improve this part of the 3D building model reconstruction we make use of building footprints that are extracted from object-oriented classification of coincident high resolution imagery. In this process, the lidar derived DSM (Digital Surface Model) is used with high-resolution imagery for initial segmentation and subsequent classification. Spectral reflectance values and the mean difference of the DSM value play a vital roll in identifying the building footprints. The height jump edges within the building footprints are selected semi-automatically from the classified image in conjunction with height data. The building roof faces are generated from bi-linear surface interpolation. A polyhedral building model was prepared which combined the roof and ground plans. Experiments show that this method accurately extracts the building plans and generate simple 3D models. BIOGRAPHY OF PRESENTER Sohel Syed received his masters degree in GIS from Curtin University of Technology. He began his PhD with a ARC funded scholarship to the School of Geography, Population and Environmental Management, Flinders University, South Australia. He also began his career as a research assistant in Airborne Research Australia (ARA), where he investigates feature extraction algorithms from high resolution imagery. His PhD research topic is “the integration of multi-spectral imagery and laser scanner data for land cover classifications”. Sohel's expertise lies at the interface of GIS and remote sensing for landcover mapping and feature extraction.

1421

Semi-automatic 3D building model generation from lidar and high resolution imagery

INTRODUCTION Lidar (Light detection and ranging) is a recent development in remote sensing with great potential for creating high resolution and accurate digital elevation models (DEMs). It has proven a mature state of art technology acquires a three-dimensional point cloud that describes the Earth’s surface. The standard features of recent airborne lidar systems are their ability to discriminate between first and last pulse reflections. A laser pulse that is fired over an object usually has multiple reflections. Some of the laser pulse may be reflected by the top of the object and therefore represents the first returning pulse. The remainder is likely to be reflected by the ground and hence generates the last returning pulse. In this way, lidar produces a fast and highly accurate three-dimensional surface (Rottensteiner et al., 2003). Lidar provides very accurate position and height information, but less direct information on the object’s geometrical shape, while high-resolution imagery offers very detailed information on object attributes, such as spectral signature, texture, shape etc. Combining these two kinds of complementary datasets is quite promising for building extraction, 3D city modelling etc. (Tao and Yasuoka, 2002). While the orientation and height of a roof face can be estimated accurately, the outline of a roof face is more difficult to determine (Vosselman and Dijkman, 2001). The building footprints not only provide the accurate location of the outer roof face edges often they reveal information on the structure of a building (Haala and Anders, 1997; Haala and Brenner, 1997). When building models are constructed with solid geometry, buildings can be regarded as compositions of a few components with simple roof shapes (such as flat roofs, gable roofs and hip roofs). The 2D corners of the building outlines often give an indication of the 3D position of these building components within the building footprint. With the above views in mind, this paper describes the procedure for extracting building footprints and subsequent 3D building models by combining lidar and high resolution imagery in an object-oriented perspective. The following section reviews the use of high resolution imagery and lidar data in context of 3D building modelling. The third section describes the method and techniques and the fourth section discusses the main findings and advantages of the integrated approach. BACKGROUND Building reconstruction from images Using aerial images and existing 2D building information Suveg and Vosselman (2004) developed an automated building reconstruction procedure. In this process, an approximate height is obtained by projection of the given ground plan to the images. Then, ground plans are partitioned into rectangles and 3D corners are extracted from image 2D corners. Using those corners and the 2D ground plan, an initial estimation for 3D volumetric primitives is calculated. This estimation is then refined using a gradient-weighted least squares fitting in the images. Haala and Anders (1997) proposed an automatic building extraction procedure from stereo imagery. In this procedure, using conventional image matching a DSM is obtained which is used to generate a disparity map between the left and right image of the stereo pair. Using this disparity map, two-dimensional line segments of both images are grouped to three-dimensional segments and they are then grouped in object space. Using the arrangement of rectangles in object space, approximate parameters for a building primitive are obtained. Building reconstruction from lidar data In lidar acquired data, the building detection starts with the generation of a coarse digital terrain model (DTM) from the digital surface model (DSM) by morphologic filtering. Then, a further classification must separate points in buildings from points on trees an other objects by evaluating the local surface roughness and other cues (Rottensteiner,Trinder,Clode and Kubik, 2004). Vosselman (1999) extracts roof faces from non-regularised laser scan data using a Hough transform, followed by connected component analysis. In this method, no ground plans are used but from lidar data a ‘main building orientation’ is derived and used as a constraint for the edge orientation. Brenner (2000) proposes a bottom-up approach, which tries to extract planar faces from a regularized DSM using random sampling consensus (RANSAC). Faces are accepted or rejected based on a set of rules, which express possible relationships between faces and ground plan edges

1422

Semi-automatic 3D building model generation from lidar and high resolution imagery

Building reconstruction from multiple data fusion The use of different data sources, such as panchromatic images, colour images, surface models, laser scan data or existing GIS data often increases the reliability of extraction process (Brenner, 2000). Rottensteiner and Briese (2003) suggest a roof face extraction process using seed regions and region growing in regularised lidar derived DSM. In this process, intersection and step edges are detected and a polyhedral model is derived. It is proposed to detect regular structures automatically and to introduce them as constrains into a global adjustment. The authors also suggest the use of additional aerial images to detect small planar segments and to fit the wire frame models obtained from the lidar data to aerial images in order to improve the geometric accuracy. Rottensteiner et al. (2003) integrated lidar and high-resolution multi-spectral data for the automatic detection of buildings with heterogeneous appearances; a hierarchic integration technique was adopted to detect buildings in urban settings. Hofmann (2001) used an object-based classification scheme to detect buildings and roads in IKONOS data using additional elevation information. In light of these reviews, this paper describes the use of high resolution imagery combined with a lidar derived DSM for initial building planar roof reconstruction and subsequent 3D modelling. This approach has been adopted to make more accurate 3D building models with less complicated machine processes. IMPLEMENTATION The work flow for the proposed building extraction method is presented in Figure 1. First, building regions are detected in high resolution imagery and lidar data by an object-oriented classification procedure. As a result, 2D building regions are extracted for further geometric reconstruction. In the subsequent processes, these building regions are treated individually and initial height jumps for each edge are collected manually from lidar data. Then, initial planar patches are reconstructed in the object space through bi-linear surface interpolation.

Figure 1: Steps for building extraction. Initial polyhedral building models are fitted into the image by 3D modeling. Different perspective views of that fitted model give an indication about the accuracy of their geometric parameters.

Lidar

Segmentation & classification

Building region detection Height

measure

2D Building Plan

DSM

3D Building edge detection

Surface interpolation

3D Building reconstruction

3D view

High res. image

1423

Semi-automatic 3D building model generation from lidar and high resolution imagery

Test site The test area covers an area of 80x120m over Mathoura, a small town in southern New South Wales. This area contains most patterns typical of semi-urban landscapes. Buildings in this area have very distinct shape and their size range from very large to medium. The distribution of buildings is very close (i.e. is compact).

Datasets The image data, collected by AEROmetrex, was acquired using a Zeiss LMK 152 camera with a calibrated focal length of 152.261mm. It was captured at an approximate flying height of 850m above the ground level equating to an average scale 1:5500, which was later scanned at 15µm to provide a pixel size of 8.25 cm. The photographic images were orthorectified with the help of accompanying exterior orientation parameters (Xo, Yo, Zo, ω,φ,κ), which were captured using onboard GPS and IMU sensors. Figure 2(a) shows the photographic dataset of the study area. Lidar data was acquired in July 2001 by AAMGeoScan (now AAMHatch). The lidar system used was the ALTM 1225, which operates with a sampling intensity of 11000 Hz at a wavelength of 1.047 µm. Approximate flying height of this sensor was 1100m and the laser swath width was 800m. Vertical accuracy was 0.15m (1σ), the internal precision was 0.05m, and the original laser footprint was 22cm in diameter. The data was subsequently resampled to a 1m grid. More details of the lidar data and its processing information can be found in Molloy and Bruce (2004). The processed lidar image is shown in figure 2(b).

Figure 2: Photographic and lidar data sets of the study area.

2D ground plan extraction As shown in figure 3, several building planar faces were reconstructed from object-oriented classification which utilised the high resolution imagery and lidar derived DSM. The basic processing units of object-oriented image analysis were segments or image objects, and for each image object a meaningful statistic was calculated in an increased uncorrelated feature space using shape, texture and topological features. Image-object primitives are created through multi-resolution segmentation using eCognition (http://www.definiens.com). These objects are polygons of roughly equal size exhibiting interior homogeneity.

(a) Photographic image (R1,G2,B3)

(b) normalized Digital Surface model (nDSM)

1424

Semi-automatic 3D building model generation from lidar and high resolution imagery

Figure 3. Object oriented segmentation of building regions. In this research, building regions were extracted by discriminating them from open space through the mean difference of the nDSM layer. An image object was represented as building region if its mean difference of nDSM was greater or equal to 1.2m. After finding the 2D building objects, the planar faces were reconstructed through merging the segments into a meaningful region. The final plane parameters were determined from a least squares adjustment using all lidar points within a plane. Detection of height jump points The detection of height jump edges is the most difficult part of the building reconstruction (Vosselman and Dijkman, 2001). The accurate location of a height jump edge requires a high density 3D point cloud. In this study, it was assumed that the height jump edge is parallel to the 2D ground plan segment. For each planar face, the location of height jump edges were generated based on the orientations of the segment edges and the extent of the planar face. Creation of initial building model After extracting the height jump edge points with their planar faces, a bi-linear surface interpolation was applied to reconstruct initial building model. In this process, a relatively coarse 3D building model was derived by fitting shape primitives to the original segments of the ground region.

Figure 4. Reconstructed 3D building model.

1425

Semi-automatic 3D building model generation from lidar and high resolution imagery

All detected roof faces of the buildings are shown in figure 4. Only the smaller roofs (silos) were not found due to the discontinuities of the planar roof. In this case, these objects were manually selected and processed. RESULTS AND DISCUSSION The model resulting from the dataset of figure 2 is shown in figure 5. For comparison, the aerial photograph was wrapped with the 3D model. For this result, it can be concluded that most building planes are detected properly. In this experiment, trees or other objects were not considered due to their complexity in the planar surface. This simple model also missed out some details, such as chimney on the coloured roof. However, small differences in height and slope of the adjacent roof faces were mainly responsible for that. Using high density lidar data with refined extraction procedures will help in overcoming this problem. To analyse the efficiency of the building modelling from the combination of lidar data and aerial photography a part of 3D model was produced (figure 5). The procedure for building reconstruction was based on an object-oriented segmentation of the building regions using several tools for splitting the planar faces along height jump edges as observed by the operator in views like those shown in figure 4. With this 3D model a virtual landscape of the study area was reconstructed with other ancillary data (figure 5). The top slide (figure 5a) shows the perspective view. In general it was noted that the building mass is well matched with the building region. Figure 5(b) shows the virtual perspective from the opposite side of figure 5(a). The left and right hand sides of the front view were represented into figure 5(c) and 5(d). This virtual view suggests that this integrated approach is well suited for extracting 3D building models from a lidar derived DSM and aerial photography.

Figure 5. Reconstructed building model wrapped with aerial photograph.

Perspective from front (a) Perspective from opposite (b)

Perspective from left side (c) Perspective from right side (d)

Plate 1

Plate 2

Plate 3

1426

Semi-automatic 3D building model generation from lidar and high resolution imagery

CONCLUSION The experiments to automatically detect 3D building models showed that more efforts are required to precisely define both the rules for selection of objects and the rules for generating height jump edges. In particular, the rules to select objects require a high level understanding of the scene. Low density lidar data is insufficient to precisely recognise a building and will cause many false detections of new buildings. In this study, the criteria for selecting rules were not precisely developed therefore open to multiple interpretations. These rules would need to be formulated in a more exact fashion in order to allow correct human-computer interaction. The original lidar density was ~22cm with an irregular sample spacing and it was finally supplied as 1 per metre squared which suggested small details (less than 1m2) could not be extracted from it. This point density needs to be increased to be able to robustly identify small changes as required in the application. With regard to 3D modelling, integration of lidar and aerial imagery can produce very reliable and detailed surface descriptions. These high quality data allow the easy reconstruction of building roof planes. Automatic building reconstruction procedures need to be fine tuned to optimise these results. More efforts in formalising and using a knowledge-based approach could further improve the efficiency of 3D mapping using lidar and aerial imagery. ACKNOWLEDGEMENT This work was supported by an Australian Research Council (ARC) Discovery Project (DP0450889). REFERENCES Brenner, C. (2000) Towards fully automatic genetation of city models, International Archives of Photogrammetry and

Remote Sensing, Vol. 32, No. 3, pp. 85-92. Haala, N. and Anders, K.H. (1997) Acquisition of 3D urban models by analysis of aerial images, digital surface models

and existing 2D building inforamtion, Proceedings of SPIE Conference on Integrating Photogrammetric Techniques with Scene Analysis and Machine Vision III, pp. 212-222.

Haala, N. and Brenner, C. (1997) Generation of 3D city models from airbrone laser scanning data, Proceedings of

EARSEL workshop on LIDAR remotesensing on land adn sea, Tallin, Estonia, pp. 105-112. Hofmann, P. (2001) Detecting buildings and roads from IKONOS data using additional elevation information,

Proceedings of GIS – Zeitschrift für Geoinformationssysteme 6/2001, Hüthig GmbH & Co. KG, Heidelberg, pp. 28-33.

Molloy, N. and Bruce, D. (2004) Application and analysis of three-dimensional object reconstruction using two

dimensional imagery in a tactical environment, Proceedings of The 12th Australasian Remote Sensing and Photogrammetry Conference, Smith, R. and Dawbin, K. eds., Fremantle, Western Australia, 18-22 October 2004.

Rottensteiner, F. and Briese, C. (2003) Automatic Generation of Building Models from LIDAR Data and the Integration

of Aerial Images, Proceedings of ISPRS Working group III/3 Workshop on '3-D Reconstrction from Airborne Laserscanner and InSAR Data', Mass, H. G., Vosselman, G. and Streilein, A. eds., International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, Dresden, Germany, pp. 174-180.

Rottensteiner, F., Trinder, J., Clode, S. and Kubik, K. (2003) Building detection using LiDAR data and multi-spectral

images, Proceedings of VIIth Digital Image Computing: Techniques and Applications, Sun, C., Talbot, H., Ourselin, S. and Adriaansen, T. eds., Sydney, 10-12 Dec., pp. 673-682.

Rottensteiner, F., Trinder, J., Clode, S. and Kubik, K. (2004) Fusing Airborne Laser Scanner Data and Aerial Imagery

for The Automatic Extraction of Buildings in Densely Built-up Areas, Proceedings of XXth ISPRS Congress, Commission 3, Istanbul, Turkey, p. 512 ff.

Suveg, I. and Vosselman, G. (2004) Reconstruction of 3D building models from aerial images and maps, ISPRS Journal

of Photogrammetry & Remote Sensing, Vol. 58, pp. 202-224.

1427

Semi-automatic 3D building model generation from lidar and high resolution imagery

Tao, G. and Yasuoka, Y. (2002) Combining High Resolution Satellite Imagery and Airborne Laser Scanning Data for Generating bareland DEM in Urban Areas, Proceedings of International Workshop on Visualization and Animation of Landscape, International Archives of Photogrammetry, Remote Sensing and Spatial Information Science, Kunming, China, 26 - 28 February 2002.

Vosselman, G. (1999) Building reconstruction using planar faces in very high density height data, Proceedings of

ISPRS Conference "Automatic Extraction of GIS Objects from Digital Imagery", Munich, 8-9-1999, pp. 87-92. Vosselman, G. and Dijkman, S. (2001) 3D Building Model Reconstruction from Point Clouds and Ground Plans,

Proceedings of ISPRS workshop on Land Surface Mapping en Characterization Using Laser Altimetry,Michelle, H. A. ed., The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Annapolis, Maryland, 22nd to 24th october 2001, pp. 37- 44.

1428