Department of Science and Technology Institutionen för teknik och naturvetenskapLinköpings Universitet Linköpings UniversitetSE-601 74 Norrköping, Sweden 601 74 Norrköping

ExamensarbeteLITH-ITN-MT-EX--04/033--SE

Digitizing the Parthenonusing 3D Scanning –

Managing Huge Datasets

Therese Lundgren

2004-04-07

LITH-ITN-MT-EX--04/033--SE

Digitizing the Parthenonusing 3D Scanning -

Managing Huge Datasets

Examensarbete utfört i Medieteknikvid Linköpings Tekniska Högskola, Campus

Norrköping

Therese Lundgren

Handledare: Paul DebevecExaminator: Mark Ollila

Norrköping 2004-04-07

RapporttypReport category

Examensarbete B-uppsats C-uppsats D-uppsats

_ ________________

SpråkLanguage

Svenska/Swedish Engelska/English

_ ________________

TitelTitle

Digitizing the Parthenon using 3D Scanning - Managing Huge Datasets

FörfattareAuthor

Therese Lundgren

SammanfattningAbstract

Digitizing objects and environments from real world has become an important part of creating realistic computergraphics. Through the use of structured lighting and laser time-of-flight measurements the capturing of geometricmodels is now a common process. The result are visualizations where viewers gain new possibilities for both visualand intellectual experiences.

This thesis presents the reconstruction of the Parthenon temple and its environment in Athens, Greece by using a 3Dlaser-scanning technique.

In order to reconstruct a realistic model using 3D scanning techniques there are various phases in which the acquireddatasets have to be processed. The data has to be organized, registered and integrated in addition to pre and postprocessing. This thesis describes the development of a suitable and efficient data processing pipeline for the givendata.

The approach differs from previous scanning projects considering digitizing this large scale object at very highresolution. In particular the issue managing and processing huge datasets is described.

Finally, the processing of the datasets in the different phases and the resulting 3D model of the Parthenon is presentedand evaluated.

ISBN_____________________________________________________ISRN LITH-ITN-MT-EX--04/033--SE_________________________________________________________________Serietitel och serienummer ISSNTitle of series, numbering ___________________________________

NyckelordKeyword

3D scanning, range image, huge datasets, mesh generation, data processing pipeline, filtering, cultural heritage

DatumDate

2004-04-07

URL för elektronisk version

http://www.ep.liu.se/exjobb/itn/2004/mt/033/

Avdelning, InstitutionDivision, Department

Institutionen för teknik och naturvetenskap

Department of Science and Technology

Digitizing the Parthenon using 3D Scanning -Managing Huge Datasets

Therese Lundgren

USC Institute for Creative Technologies

and Department of Science and Technology, Linköping University

7th April 2004

Abstract

Digitizing objects and environments from real world has become an important partof creating realistic computer graphics. Through the use of structured lighting andlaser time-of-flight measurements the capturing of geometric models is now a commonprocess. The result are visualizations where viewers gain new possibilities for bothvisual and intellectual experiences.

This thesis presents the reconstruction of the Parthenon temple and its environmentin Athens, Greece by using a 3D laser-scanning technique.

In order to reconstruct a realistic model using 3D scanning techniques there arevarious phases in which the acquired datasets have to be processed. The data has to beorganized, registered and integrated in addition to pre and post processing. This thesisdescribes the development of a suitable and efficient data processing pipeline for thegiven data.

The approach differs from previous scanning projects considering digitizing thislarge scale object at very high resolution. In particular the issue managing and process-ing huge datasets is described.

Finally, the processing of the datasets in the different phases and the resulting 3Dmodel of the Parthenon is presented and evaluated.

Keywords

3D scanning, range image, huge datasets, mesh generation, data processing pipeline,filtering, cultural heritage.

Acknowledgments

This thesis is part of the Master of Science degree in Media Technology and Engineer-ing at the Department of Science and Technology at Linköping University, Sweden. Itpresents research and implementation done at the Institute for Creative Technologies(ICT), University of Southern California, Marina del Rey, CA, USA from September2003 until February 2004.

I would like to express my special thanks to my examiner Dr. Mark Ollila at theInstitute of Science and Technology, Linköping University, for making this thesis workpossible. Also, many thanks to my supervisor Dr. Paul Debevec at the Institute forCreative Technologies for giving me the opportunity the be a part of a very interestingresearch project in addition to support and help.

Moreover, thanks to Andrew Gardner and Nathan Yun and all the other staff at theICT for ideas, help and support, with both research issues and time consuming manualwork. Finally, thanks to Paolo Cignoni at CNR in Pisa, Italy, for software support.

1

Contents

1 Introduction 71.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71.2 Aim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71.3 Problem Description . . . . . . . . . . . . . . . . . . . . . . . . . . 8

1.3.1 Central Issues . . . . . . . . . . . . . . . . . . . . . . . . . . 91.3.2 Restrictions . . . . . . . . . . . . . . . . . . . . . . . . . . . 91.3.3 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . 10

1.4 Outline of Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2 Background and Related Work 112.1 History of the Parthenon . . . . . . . . . . . . . . . . . . . . . . . . 112.2 Data Acquisition . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.2.1 Range Images . . . . . . . . . . . . . . . . . . . . . . . . . . 122.3 Registration Phase . . . . . . . . . . . . . . . . . . . . . . . . . . . 132.4 Integration Phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132.5 Post-Processing Phase . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.5.1 Hole filling . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.5.2 Mesh Simplification . . . . . . . . . . . . . . . . . . . . . . 14

3 Data Processing Pipeline 153.1 Data acquisition and 3D Scanning . . . . . . . . . . . . . . . . . . . 153.2 Pre-processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163.3 Registration and the Iterative Closest Point Algorithm . . . . . . . . . 17

3.3.1 MeshAlign . . . . . . . . . . . . . . . . . . . . . . . . . . . 183.3.2 Selection of Regions . . . . . . . . . . . . . . . . . . . . . . 183.3.3 Global Registration . . . . . . . . . . . . . . . . . . . . . . . 193.3.4 Transformation . . . . . . . . . . . . . . . . . . . . . . . . . 20

3.4 Handling large Datasets . . . . . . . . . . . . . . . . . . . . . . . . . 203.4.1 Partitioning into Voxels . . . . . . . . . . . . . . . . . . . . . 203.4.2 The Triangulation Issue . . . . . . . . . . . . . . . . . . . . 213.4.3 The PLY Polygon File Format . . . . . . . . . . . . . . . . . 23

3.5 Integration by Volumetric Merging . . . . . . . . . . . . . . . . . . . 233.5.1 Bounding Box Parameters . . . . . . . . . . . . . . . . . . . 24

3.6 Post-Processing Procedures . . . . . . . . . . . . . . . . . . . . . . . 243.6.1 Redundant Triangle Representation . . . . . . . . . . . . . . 253.6.2 Mesh Editing . . . . . . . . . . . . . . . . . . . . . . . . . . 263.6.3 Decimation . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

2

4 Implementation and Software 284.1 Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284.2 Registration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294.3 Voxelization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

4.3.1 Adjustment of the Coordinate System . . . . . . . . . . . . . 314.3.2 Partitioning . . . . . . . . . . . . . . . . . . . . . . . . . . . 324.3.3 Triangulation . . . . . . . . . . . . . . . . . . . . . . . . . . 33

4.4 Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354.5 Post-Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

4.5.1 Zipper used on the Voxels . . . . . . . . . . . . . . . . . . . 354.5.2 Mesh Editing in GSI Studio . . . . . . . . . . . . . . . . . . 36

4.6 Automatization of the Pipeline Process . . . . . . . . . . . . . . . . . 374.7 The Surrounding Environment . . . . . . . . . . . . . . . . . . . . . 37

5 Results 395.1 Processing Pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . 395.2 Reconstructed Model . . . . . . . . . . . . . . . . . . . . . . . . . . 40

5.2.1 Partial Results . . . . . . . . . . . . . . . . . . . . . . . . . 415.2.2 Final Results . . . . . . . . . . . . . . . . . . . . . . . . . . 42

5.3 Project Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

6 Discussion 466.1 Process discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . 466.2 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476.3 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

A List of Abbreviations 52

B The PLY Polygon File Format 53

3

List of Figures

1.1 Registered point data(left), pieces of data in the registration phase(topright) and integrated geometry (bottom right). . . . . . . . . . . . . . 9

2.1 The west facade of the Parthenon in Athens, Greece. . . . . . . . . . 11

3.1 The Quantapoint laser scanner. . . . . . . . . . . . . . . . . . . . . . 153.2 The acquired scan data, bottom figure visualizes the depth differences. 163.3 Gaussian blur appliedn times. . . . . . . . . . . . . . . . . . . . . . 173.4 Categories of the range scans, 1(top), 2 (middle)and 3(bottom). . . . 193.5 Pair alignment; less accumulated error in theright than in theleft graph. 203.6 Voxel overlap without seams, the right figure with one voxel shaded to

emphasize the overlap. . . . . . . . . . . . . . . . . . . . . . . . . . 213.7 A range image and its (i,j) coordinate system. . . . . . . . . . . . . . 223.8 Thresholds control the triangulation. . . . . . . . . . . . . . . . . . . 223.9 The result when triangulating one single range image. . . . . . . . . . 233.10 Grid after merging; original(left) and using new parameters(right). . 253.11 Boundary edge case. . . . . . . . . . . . . . . . . . . . . . . . . . . 253.12 Black areas will create holes in the geometry . . . . . . . . . . . . . 263.13 Decimation of the surrounding environment; 5, 10 and 20 percent re-

spectively of the original amount of polygons. . . . . . . . . . . . . . 27

4.1 Pre-processing of the range images. . . . . . . . . . . . . . . . . . . 284.2 Selection of regions(left) and corresponding new PFM:s(right). . . . 294.3 The manual alignment method inMeshAlign. . . . . . . . . . . . . . 304.4 Registration phase. . . . . . . . . . . . . . . . . . . . . . . . . . . . 304.5 Voxelization of the facades; east(top left), north(top right), west(bot-

tom left)and south(bottom right). . . . . . . . . . . . . . . . . . . . 314.6 Voxelization phase. . . . . . . . . . . . . . . . . . . . . . . . . . . . 334.7 Triangulation scheme. . . . . . . . . . . . . . . . . . . . . . . . . . . 334.8 The voxel array. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344.9 Integration phase. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354.10 Removal of redundant triangles. . . . . . . . . . . . . . . . . . . . . 354.11 Hole filling; before(left) and after(right). . . . . . . . . . . . . . . . 364.12 Mesh editing and decimation. . . . . . . . . . . . . . . . . . . . . . . 364.13 Graphical user interface; voxelization(left) and merging(right). . . . 374.14 The reconstructed surrounding environment of the Parthenon. . . . . . 38

5.1 Flowchart of the 3D data processing pipeline for producing the model. 395.2 Flowchart of the post-processing phase. . . . . . . . . . . . . . . . . 40

4

5.3 The east facade assembled from four range images(left) compared tothe same part from the completely processed final model(right). . . . 41

5.4 A rendering of the whole model before completion of the post-processingphase. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

5.5 Scanner location map (seen from above). . . . . . . . . . . . . . . . . 435.6 The final model and environment. The white area indicates a voxel and

the blue regions show its neighboring voxels. . . . . . . . . . . . . . 435.7 A real photograph(left) compared to a rendering under the same illu-

mination(right). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445.8 Rendering from the larger research project; the east facade. . . . . . . 445.9 Rendering from the larger research project; the west facade. . . . . . . 45

6.1 Future work involving producing a model of the Erecthion. . . . . . . 48

5

List of Tables

4.1 Number of points to be voxelized. . . . . . . . . . . . . . . . . . . . 324.2 Connectivity thresholds. . . . . . . . . . . . . . . . . . . . . . . . . 34

5.1 Description of the phases in the processing pipeline. . . . . . . . . . . 40

A.1 List of Abbreviations . . . . . . . . . . . . . . . . . . . . . . . . . . 52

B.1 The PLY file format . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

6

Chapter 1

Introduction

This chapter gives an introduction to this thesis work by presenting the motivation tothe work, a problem description and finally an overview of the structure of this report.

1.1 Motivation

Digitizing objects and environments from real world has become an important part ofcreating realistic computer graphics. Capturing geometric models through the use ofstructured lighting, laser triangulation, and laser time-of-flight measurements is now acommon process. The scene digitized in this thesis is the Parthenon and its environ-ment in Athens, Greece, done in collaboration with the ongoing Acropolis Restorationproject.

What makes this thesis and the scanning of the Parthenon different from previouslarger scanning projects [18] [20] is mainly the data acquisition of a structure having thesize of the Parthenon (75m× 35m) at this high resolution resulting in huge datasets.The reproduction of the model can not simply follow the traditional data processingpipeline. Instead the central issue is to develop a data processing pipeline suitable forthe acquired data consisting of both traditional and new phases. By using the developedpipeline an accurate 3D model of the Parthenon is to be constructed.

Moreover, this thesis probably differ from traditional ones by being a part of alarger research project1 with several people involved, and where the digitizing of theParthenon is a very significant component. The final model have to fulfill the require-ments for the larger research project in order to contribute to good results. The continu-ous integration into the larger project is important as well as meeting internal deadlinesduring the time for this thesis.

1.2 Aim

The aim of this thesis is firstly to find an efficient, accurate and not too time consumingdata processing pipeline for the acquired 3D scan data. This by both implementingown algorithms and by using existing software.

1Estimating Reflectance Properties of a Complex Scene under Captured Natural Illumination.

7

Secondly, the aim is to assemble all the acquired data into an accurate geometrymodel of the Parthenon by using the developed data processing pipeline. The recon-struction of the final geometry model is further divided into three partial goals:

• A model of the east facade.

• A model of the west facade.

• A complete 3D representation of the whole structure.

Finally, the purpose of the complete model is to be integrated into the larger re-search project at the ICT, with the aim to publish a scientific research paper as well asfilm for the electronic theater at the SIGGRAPH 20042 conference.

1.3 Problem Description

In order to produce the final model of the Parthenon a suitable processing pipelinehas to be developed. The problem of finding an efficient 3D data processing pipelineinvolves following phases (see also figure 1.1):

Data acquisition First the data has to be automatically collected by the use of a scan-ning sensor. Depending on how the data is acquired the need for certain phasesvaries.

Pre-processingPre-processing of the acquired data involves converting the data to anappropriate format, size and quality. There is a major need for pre-processing ofthe acquired data in terms of filtering since the laser scan measurements includenoise.

Registration In the registration phase pieces of data is registered to each other in orderto form a complete and continuous data set. This means transformations betweenthe coordinate systems of the different pieces of data.

Integration The integration phase addresses the problem of combining and mergingthe different pieces of data from the registration phase into one consistent piece.

Post-processingData acquired from 3D scanning is not always complete, e.g. oc-cluded geometry create holes, and the resulting mesh might include too manypolygons. Therefore a need for various post-processing methods are necessary.

The approaches in the phases have to be able to solve the processing of the acquiredlarge datasets. That implies adjustments of both datasets and implementations.

2http://www.siggraph.org/s2004

8

Figure 1.1: Registered point data(left), pieces of data in the registration phase(topright) and integrated geometry (bottom right).

1.3.1 Central Issues

The central issues in the various phases presented in this thesis work involves follow-ing:

• The large amount of noise in the acquired data, i.e. the range scans have to bedecreased. Depending on the strength of the return, the depth accuracy variesbetween 0.5cm and 3cm.

• To register the range scans without a pre-alignment done on site.

• To merge a large scale model with the size of the Parthenon (75m×35m×20m)at high resolution as well as its surrounding terrain (200m×200m) at low reso-lution.

• How to efficiently handle the large number of scans and their huge amount ofdata. One single full resolution range scan contains 58 million points and thereare in total 120 scans.

• The processing of the data pipeline has to integrate into other processes simulta-neously and the file formats have to be compatible since this thesis is a part of alarger project.

• The processing pipeline have to be partially automatized to be used efficientlywithout supervision, e.g. processes being run over night.

1.3.2 Restrictions

Restrictions in the data acquisition have impact on this thesis work and its possibleresults. When performing the 3D scanning of the Parthenon temple the focus wason capturing the two facades of the east and west side of the temple. Scaffoldingand equipment around the structure and in the middle section of the temple made thescanning more difficult, especially on the north and south sides.

Additionally, most photos were taken of the east facade. Since photos are an es-sential requirement for further work in the larger research project, this affected theordering of the partial goals for the reconstruction of the final model (see section 1.2).

9

1.3.3 Requirements

In this thesis work there are requirements in the data processing which have to befulfilled. These involves quality, resolution and efficiency.

Firstly, due to the varying strength of the return of the scanner’s laser, noise wascreated in the range measurements. The existing noise have to be filtered intelligentlyin order to obtain smooth geometry without loosing details.

Since the purpose of the reconstructed model of the Parthenon is to be integratedinto the larger research project the requirements of that project apply. Texturing themodel is not part of this thesis but one of the central issues in the larger research project.To perform a photo realistic texturing, i.e. to estimate the reflectance properties for thesurface of the reconstructed model, the geometry has to be accurate enough. Thus, theresolution of the model has to be high, with the geometry in a rendering of the recon-structed model being comparable to a real photograph taken from the same locationas the rendering. Furthermore, the final model has to be available in various resolu-tions. This since a complete full resolution model will not be possible to load into thememory of an ordinary workstation at once.

For some of the software used in the process there are restrictions regarding a max-imum amount of polygons allowed in the files to be processed. This requires that theoriginal data cannot simply be processed in the existing form.

There are also requirements regarding the time aspect. The processing times inthe various phases of the pipeline have to be optimized since there are many timeconsuming processes and much manual work involved with data acquired from 3Dscanning.

1.4 Outline of Thesis

This report proceeds as follows. Chapter two presents the background and previouswork in the 3D scanning area. Chapter three describes the methods and algorithmsin the developed data processing pipeline. In chapter four the implementation and thesoftware are discussed. Chapter five presents the results and evaluates them. Finally,chapter six summarizes and discusses the report and examples of future work are pre-sented.

Target group

The reader of this thesis preferably has a good knowledge in computer graphics anda general knowledge about math.

10

Chapter 2

Background and Related Work

This chapter will introduce some important theories and previous work in the area ofresearch within digitizing real world objects as well as the history of the Parthenon. Italso explains the necessary phases when processing data acquired from 3D scanning.

2.1 History of the Parthenon

The Parthenon was erected during the 5th Century B.C., replacing an earlier projectthat was ransacked by invaders before it was finished. The temple was dedicated tothe goddess Athena and came to be her most magnificent monumental offering [17].Sculptures of historical events and Greek myths decorates the Parthenon, not all ofthem there today. Strategies for digitally capturing a large number of sculptures haverecently been described [25].

Figure 2.1: The west facade of the Parthenon in Athens, Greece.

11

Over the centuries, the Parthenon has changed greatly [26]. During the 3rd centuryB.C., the temple was destroyed by fire and then minimally restored. In the 6th centuryA.D., the temple was turned into a cathedral and high windows were carved throughthe walls.

Later on when Athens was a small city in the Turkish Empire the Parthenon wasturned into a mosque. In 1687 a Turkish garrison used it for storing gunpowder when itwas bombed by the Venetian fleet. A cannonball ignited the gunpowder and significantportions of the building were destroyed in the catastrophic explosion [17].

2.2 Data Acquisition

Various people, such as Levoy et al. [18], Ikeuchi [16] and Rushmeier [23] have beeninvolved in 3D scanning and have shown that accurate and detailed geometric modelscan be acquired of real-world objects using these techniques. Various non contactrangefinding techniques can be used and different scanning devices exist, includingboth academic prototypes and commercial systems. A common classification dividesthem intopassiveandactivesystems [4].

The passive systems includes for example photogrammetry1 which is based on thereconstruction of the geometry from a silhouette approach and returns a nearly com-plete model of the object but suffers from scarce accuracy. The active systems project alaser or a structured light pattern on the object and measure the geometry of the pointsthey hit by for example a time-of-flight technique. In this technique the time it takesfor the laser or light is measured and converted into a distance. A problem in the activesystems is to capture a complete model without any missing data.

A novel technique combining passive and active systems has recently been pre-sented by Dias and Vaz [11]. The technique demonstrates the possibility to registerand combine the advantages from both range and intensity based data.

One of the most recent scanning project is the Digital Buddha Project at the Uni-versity of Tokyo [20]. As with all scanning projects the purpose is to accurate measure3D coordinates of a huge amount of points over a short period. The points will then beused to reconstruct and model the scene.

2.2.1 Range Images

The data obtained in the data acquisition is a set ofrange imagesor point cloudscon-sisting of a collection of single (x,y,z) points, which represent the distance between thescanning sensor and the geometry. A range image is a two-dimensionali×j grid ofthese range points sampled from the part of the surface of the target object that thescanning sensor was able to reach during each session.

Range images are usually very large, storing millions of points, and the file size ofthe datasets tend to be huge. One significant challenge that the 3D scanning projectthe Digital Michelangelo Project[18] encountered was the handling of large datasets.Their model of Michelangelo’s David was two orders of magnitude larger than any ex-isting scanned model. For that purpose thePLY polygon file format [30] was inventedin 1998 in order to try to create a single simple file format that is flexible enough for

1Photogrammetry; a science or art of obtaining reliable measurements or information from photographsor other sensing systems.

12

future needs and usable for a wide range of models. The format stores graphical ob-jects described as a collection of polygons, and it has both an ascii representation anda binary version for compact storage and rapid saving and loading (see appendix B).

2.3 Registration Phase

The first step in creating a single model from multiple range images is the registrationor aligningphase. Normally the range image geometry is relative to the current sensorlocation,a local coordinate system, and therefore has to be transformed into a commoncoordinate space where all the range images are well aligned in 3D space,a globalcoordinate system. After the alignment, the 2D surfaces in the range images that arepartially overlapping also will be geometrically overlapping. The aligning procedureis usually a manual and time consuming phase in the 3D scanning pipeline.

The research area of aligning range images involves the Iterative Closest Point(ICP) algorithm proposed by Besl and McKay [2] which have showed to be the stan-dard solution to the alignment problem. Given two 3D point sets,P andQ, the task isto find the rigid body transformation, which means the rotations and translations, thatbringsP into the best possible alignment withQ. The ICP algorithm requires an initialestimate for which points that should have the same 3D position. Different approachesfor efficient [24], stable [13] and trimmed [5] versions of the ICP algorithm have beenproposed.

In a recent publication, Allen et al. [1] focus on an automatically algorithm toassemble data acquired from a time-of-flight laser scanner. One of their test beds fortheir model-building methods is the Cathedral Saint-Pierre in Beauvais, France. Therange images obtained there provides up to one million points, the range images of theParthenon contains originally up to 58 million points each.

2.4 Integration Phase

Integration ormerging follows when multiple range images are aligned in order tobuild a consistent non-redundant mesh. Within this research area many reconstructionmethods have been proposed during the last few years. They are being divided intovolumetricand non-volumetricmethods. Volumetric approaches immerse the rangemap in a discrete voxel space and reduce the integration problem to that for each voxelcell reconstruct that part of the surface included in that particular cell. Most of theseapproaches requires discrete distance functions. Curless and Levoy [8] presented avolumetric method consisting of a cumulative weighted signed distance function wherethey added one range image at a time.

In a paper of Turk and Levoy [27] a non-volumetric method calledzippering isdescribed. The method is incremental and works with pairs of meshes. In the first stepall redundant triangles are discarded by simultaneously eating back the boundaries ofeach mesh that lie directly on top of the other mesh. This is followed by the zipperingwhere the triangles of one mesh are clipped to the other mesh and the vertices on theboundary are shared.

A volumetric merging method which uses aMarching Intersection(MI ) algorithmderived from theMarching Cubes2 [19] method is described, implemented and im-proved in publications from theVisual Computing Groupin Pisa [21] [22]. The al-

2Marching Cubes; a pre-processing operation of surface extraction

13

gorithm is based on dividing the volume to be merged into a discrete voxel grid andindividually extract a surface from each voxel .

2.5 Post-Processing Phase

Depending on the methods used for aligning and merging in the 3D scanning pipelinethere are different needs for post-processing. The post-processing phase involves sev-eral research areas and the main areas in this thesis are hole filling and mesh simplifi-cation.

2.5.1 Hole filling

Models created from 3D scanning tend to include holes due to either occluded geom-etry or dark areas of the surface which the scanner cannot register. Hole filling can beperformed either as post-processing operation or integrated in the integration phase.

Cureless and Levoy [8] integrates hole filling into their volumetric merging algo-rithm which is based on meshes. Another mesh based reconstruction that includes holefilling have been proposed [27] [29] treating each range image as an ordered 2D array,which is triangulated to form a polygonal mesh. Hoppe et al. [15] describes a regis-tration method based on all range images treated as a large point cloud. With no initialconnectivity between the points holes are filled during the surface reconstruction.

One widely used approach to fill simple holes is to triangulate the surroundingcomponents of the boundary by creating a surface with the topology of a disk. Forholes with more complicated topology this may result in self-intersecting geometryand the problem is harder to solve. Davis et al. [9] address the hole filling problemby using volumetric diffusion. The volume is assigned a distance function with thesurface having zero value and with positive values on the inside and negative values onthe outside. The goal is to extend the distance function over the holes so that finally thefunction is zero over the entire surface implying the desired hole-free geometry.

2.5.2 Mesh Simplification

Mesh simplification ordecimationis mandatory when managing meshes produced with3D scanning methods. The high sampling rate, up to 10 points per square millimeter ofthe scanner devices produces meshes consisting of a huge amount of polygons, about20M-100M faces per mesh are common [4]. To be able to import the meshes into otherapplications the number of faces has to be significantly reduced.

A comparative publication is presented by Cignoni et al. [6] introducing variousapproaches of mesh simplification algorithms. Different implementations are testedand the computational complexity and approximation accuracy of the resulting meshesare compared.

Previous work on simplifying meshes while preserving surface properties such astexture coordinates and color is described in a paper by Garland and Heckbert [12].Their mesh simplification algorithm uses iterative contractions of vertex pairs to sim-plify the geometry. The error between old and new vertices is measured using quadricmatrices, which is a symmetric4×4 matrix associated with each vertex. For a givenvertex the sum of squared distances to an associated set of planes is measured. Thisresults in a fast algorithm with high quality approximations which both preserves ge-ometry and topology.

14

Chapter 3

Data Processing Pipeline

This chapter describes the different phases in the chosen 3D data processing pipelinein order to produce the final model of the Parthenon.

3.1 Data acquisition and 3D Scanning

During the summer 2003 the Graphics Lab at the ICT acquired in total 120 rangeimages in and around the Parthenon site over five days. The chosen time-of-flightpanoramic range scanner is manufactured by Quantapoint, Inc1 and uses a 950nm in-frared laser measurement component [14]. It can be classified as an active system,as described in section 2.2. In high resolution mode, the scanner acquires scans of18,000×3200 (x,y,z) points in eight minutes, with a maximum scanning range of 40meters and a field of view of 360 degrees horizontal and 74.5 degrees vertical. Themain reason for choosing this scanner is the very high speed of measuring a largeamount of points at extremely high resolution. Some scans from within the structureor at a very short distance were scanned in low-resolution, acquiring one-quarter thenumber of points.

Figure 3.1: The Quantapoint laser scanner.

The laser system scans a volume of space where millions of digital measurementsdefine every shape and dimension. The massive dataset produced by the scanner makes

1http://www.quantapoint.com

15

it possible to obtain models with photo quality, which, as mentioned in section 1.3.3is necessary for the purpose of the model. The output data from the scanner is in theQuantapoint format (QPG). Besides the range image a 16-bit monochrome image ofthe infrared intensity returned to the sensor for each measurement is produced by thescanner.

During this scanning procedure, depending on the strength of the return, the depthaccuracy varied between 0.5cm and 3cm. This varying depth accuracy resulted in verynoticeably noise in the geometry which had to be reduced, see section 3.2.

Figure 3.2: The acquired scan data, bottom figure visualizes the depth differences.

Scans from outside the structure were initially aligned during the scanning processthrough the use of checkerboard fiducial markers placed within the scene. The pre-alignment procedure was a relatively time consuming process and was therefore notpossible to perform for all the scanning sequences. Additionally, the scanner was con-strained to scan horizontally, no tilting possible, and four markers were necessary foreach range image.

The pre-alignment proceeds as follows. The first scanning session in the sequenceis set as the reference scan. For each new scan in the sequence four markers have to bethe same as for the previous scanning session. The heads of the markers are allowed tobe rotated between the sessions to face the scanning sensor.

The registration on site consisted of marking corresponding fiducial markers in theavailable interface from Quantapoint directly after each session. Thereafter the scan istransformed into the local coordinate space for the previous scan. In this way the firststep in the registration phase is achieved.

3.2 Pre-processing

Initially, in the pre-processing phase, the obtained point clouds in the Quantapointformat are converted into range images in theportable float map2 format (PFM). Eachpixel in a PFM range image has ar, g andb value which is equal to that correspondingpoint’s (x,y,z) location in 3D space. Thus, hereafter when referring to the range images’(r,g,b) values the notation (x,y,z) will be used.

Due to the varying strength of the return of the laser, noise was created in therange measurements. Therefore, in the pre-processing phase the noise has to be filteredin order to produce smooth geometry. For determining the amount of noise and the

2Portable float map; a high dynamic range format storing 32 bits per color channel

16

accuracy of the range images they are triangulated (see section 3.4.2 for a descriptionhow it is implemented).

A few attempts using various smoothing algorithms were tested and the one lever-aging the best result for the test geometry, considering smoothing originally flat sur-faces is theGaussian blurfilter.

A filter kernel consisting of a3×3 matrix with weights as in equation 3.1 is appliedon the range images. To sequentially applying small kernels instead of larger onesreduces the execution time [28]. In order to preserve edges and small details a thresholdis used and added to the Gaussian blur filter. If the difference between a pixel and itsneighbor (either thex, y, or z value) is greater than this threshold the weight is setto zero implying that the neighbor will not affect the current value of the pixel. Thethreshold helps preserving edges and details in the geometry. Following, for each valuethe weight is multiplied and then the average value over the3x3 kernel replaces thecurrent value.

Filter kernel and its weights=

1 2 12 4 21 2 1

(3.1)

Figure 3.3: Gaussian blur appliedn times.

The filtering can be applied multiple times over the same image resulting in differ-ent levels of smoothness. The best result for the range images in this thesis showed tobe applying the filter five times. A comparison is shown in figure 3.3. As visible in thefigure the filtering also results in small holes being filled. This advantage is obviousalready after applying the filter once.

Since the filtering also implies that original information is lost a subsampling algo-rithm could be implemented after the filtering without any noticeable visual difference.Subsampling is performed by, in the image plane ignoring every other pixel in both thei andj dimension. Running the algorithm once decreases the file size by four withoutjeopardizing the accuracy and makes the pipeline faster and the data easier to handle.

3.3 Registration and the Iterative Closest Point Algo-rithm

The registration of the range images is solved in a partially manual and a partiallyautomatic manner. The aligning process is one of the most time consuming phases inthe whole 3D scanning pipeline since it demands user assistance.

17

3.3.1 MeshAlign

To register the range images the Italian softwareMeshAlign v.2developed by the VisualComputing Group at CNR-Pisa [4] is partially used.MeshAlignprovides an interfaceto estimate an initial alignment and has an aligning procedure based on the ICP algo-rithm [2] introduced in section 2.3.

The ICP algorithm includes three basic steps:(P andQ are two 3D point sets.)

1. Pair each point ofP to the closest point inQ.

2. Compute the rigid motion that minimizes themean square error3 (MSE) betweenthe paired points.

3. Apply the rigid motion toP and update the MSE.

The three steps are iterated and the iterations have been proven to converge in termsof the MSE. Since two range images never include the same data but are overlappingthe ICP algorithm cannot be run using all their points. Instead a group of paired pointsis used, i.e. thesample points.

The registration algorithm inMeshAlign is implemented by using the standardsemi-automatic range map alignment approach [4] which starts with following twoprocedures.

Initial Estimate: The first step is to pairwise locate all range images in a single com-mon global coordinate system. Each pair consists of two adjacent and over-lapping range images. This will provide the algorithm with a first rough initialestimate for which points should have the same position after the alignment.

Refinement: After the first step, the initial alignment is refined using the iterative ICPalgorithm which minimizes the alignment error between each pair of meshes.

The initial estimate procedure is user dependent and is performed by either markingpoint pairs or by interactive manual manipulation. For the pre-aligned range imagesthe initial estimate is already solved for, resulting in only the refinement step beingcomputed. The second step, the refinement, is the automatic part of the semi-automaticalignment approach.

3.3.2 Selection of Regions

MeshAligncannot handle meshes having the size of the meshes obtained from theParthenon range images, it has a limit of a file size about 30Mb, corresponding toabout 1.5M polygons. The meshes representing the Parthenon are, after the filteringand included size reduction of a size up to 140Mb each.

In order to be able to take advantage of the already implemented ICP algorithm inMeshAlign, a modified aligning method is developed. The method is based on selectingand cropping out small regions with obvious features in the image plane. The featuresselected from one range image shall easily be found in the corresponding range imageit will be aligned to. This is a manual and user assisted procedure where four regionsare selected (see figure 4.2).

3Mean square error; minimizes the sum of squared distances

18

Thereafter, the new range image is triangulated into a mesh which can be importedinto MeshAlign. The selection of regions is performed carefully since the aim is tochoose easily recognisably features and to select regions as far away from each otherin terms of position in 3D space. Thus, obtaining a rigid motion as robust and correctas possible from the ICP algorithm.

3.3.3 Global Registration

Organizing the global registration, which is what range images should be aligned toeach other and treated as pairs, is a complex task. How complex this task actuallyis showed to be very dependent on how the scanning on the site is performed. In theParthenon case there are basically three categories of range images (see also figure 3.4).

1. The category with the most data consists of range images acquired from thescanner when located at about 40 meters from the structure. The scanner wasplaced at this distance all around the structure, digitizing all its facades. Pre-aligning of the range images was performed.

2. Next group consists of data produced from the scanner when being tilted andfacing upwards. This in order to scan the ceiling and the backsides of columns.Pre-aligning was not possible to perform.

3. The last category are range images acquired when placing the scanner very closeto certain parts of interest of the Parthenon. Due to time restrictions pre-aligningwas only performed for some of the scanning sequences.

Figure 3.4: Categories of the range scans, 1(top), 2 (middle)and 3(bottom).

The global registration is already solved for considering the pre-aligned range im-ages. Pre-aligned datasets have one range image which is theidentity range image, andits coordinate system will be considered the global coordinate system. The identityrange images will have no translation in the (x,y,z) directions. The other range imagesin the sequence have their translations relatively to the identity range image, stored as

19

vectors in a separate file. Rotation of the scanning sensor was not allowed within the se-quence, consequently only the translation is being stored. Having pre-aligned datasetsmakes the global registration much more efficient since this otherwise is a very timeconsuming phase.

When choosing pair of meshes for the not pre-aligned range images there are somethings to consider. Since the pairwise registration sequentially performs the error min-imization, the error tends to accumulate [4]. This may result in artifacts after a numberof pairwise steps. Thus, if possible, a range image in the middle of a sequence is cho-sen as the identity range image in order to decrease the number of pairwise steps wherethe error might accumulate. Figure 3.5 shows how the sequence of pairwise alignmentcan be decreased by choosing another identity range image.

IDID

Figure 3.5: Pair alignment; less accumulated error in theright than in theleft graph.

The result from the global registration is a4×4 matrix describing the translationsand the rotations, i.e. the rigid motion of the aligned mesh, stored in a alignment file(ALN file).

3.3.4 Transformation

Next step is to actually apply the matrices in the correspondingALN files on the rangeimages. This proceeds in the image plane simply by multiplying every non-zero (x,y,z)value with the matrix. Noteworthy is that the pre-alignment showed to leverage veryaccurate results, the additional transformations for these range images were very smallin most cases.

3.4 Handling large Datasets

Most previous 3D scanning projects [18] [2] [25] have involved statues or other smallerthings in the range of a couple of meters. The Parthenon temple measures a surface of75 meters× 35 meters and the surrounding terrain about 40000 square meters. Having120 range images, each of a size up to 18000× 3200 pixels, and each storing threefloat point numbers the initial data set is extremely large. Following section describehow the problem of handling large datasets is solved.

3.4.1 Partitioning into Voxels

Partitioning into voxels means to divide the data into smaller three-dimensional vol-umes, e.g. voxels, which hereafter will be referred to as voxelization. The main reasonfor this is that the merging phase cannot process too large meshes. Another reason isthat the final model has to be manageable in other implementations outside this thesis.

20

The aim is to sequentially merge the voxels separately, resulting in a consistentmodel when viewing them all together afterwards. Different approaches for voxeliza-tion were investigated, e.g. octree representation4 used in [10] or a regular grid. Inthe chosen approach a regular grid is most suitable to use. The space containing theParthenon is divided into equally sized voxels by splitting thex, yandz axis an appro-priate number of times. The determination of a required voxel size in order to obtain asufficient resolution when merging was carried out by testing various cases of differentsize.

To avoid holes being created during the merging process, the voxels are createdpartially overlapping but perfectly connected (see figure 3.6). Thus, neighboring vox-els share several points in the boundary regions. These points are triangulated in allneighboring voxels preventing holes to occur. A resulting fallout is redundant poly-gons being created, section 3.6.1 will describe this in detail.

Figure 3.6: Voxel overlap without seams, the right figure with one voxel shaded toemphasize the overlap.

The result of the voxelization for each range images is, a (large) number of equallysized, partially overlapping voxels all containing a different amount of polygons. Whena range image has been voxelized the triangulation of its points follows.

3.4.2 The Triangulation Issue

The alignedi×j range images are in the portable float format where each pixel in theimage has a (x,y,z) value corresponding to the pixels’ location in 3D space. The trian-gulation issue is solved by triangulating a range image based on the (i,j) coordinates inthe image plane (see figure 3.7). Similar approaches are presented in [27] [29].

By connecting neighbors fulfilling requirements based on different thresholds im-plemented in a connectivity function triangles are created (see section 4.3.3 for furtherdetails). Connecting neighbors means that four points in adjacent rows and columns inthe range image possibly creates zero, one or two triangles. Figure 3.8 shows how thethresholds control and avoid connecting portions of the surface that are separated bydepth discontinuities.

4Octree; subdivision of space recursively into eight cubes, each of which may be subdivided into eightsubcubes, and so on.

21

i

j

Figure 3.7: A range image and its (i,j) coordinate system.

distance > threshold,

no connection made

scanned geometry

scanner location

Figure 3.8: Thresholds control the triangulation.

Triangulating a 2D range image based on its (i,j) coordinates is elementary sincethe width and height of the image are known and can be used for finding the rightneighbors to compute connectivity on. The implementation of the voxelization impliedthat only the non-zero (i,j) coordinates of the points are saved for each voxel. The (i,j)coordinates are stored together with the voxel index in a two-dimensional voxel arraycontaining all voxels.

In order to use the same triangulation method as for a complete range image thevoxel array first is being searched for its minimum and maximumi-values to obtainthe corresponding width of the voxel in the image plane. Knowing the width makes itpossible for the triangulation algorithm to still be based on the ordering of the points inthe original range image. The result of triangulating one single range image is showedin figure 3.9 and implementation details are described in section 4.3.3. When all voxelshave been triangulated they together create one large mesh representing the input rangeimage.

22

Figure 3.9: The result when triangulating one single range image.

3.4.3 The PLY Polygon File Format

One of the main issues with the data pipeline when handling huge amount of large filesis memory and speed. Moreover, the files have to be in a format that other softwarefurther on in the pipeline are able to read. Previous research [18] has shown that thePLY polygon file format introduced in section 2.2.1 is very suitable for this purpose.Therefore, the range images are triangulated into binary meshes in this format. Acomparison between the ascii file formatOBJ and the binary PLY format showed thatthe last one requires almost half the space, e.g. a piece of geometry represented as anOBJ file demands 14390kB compared to 7600kB as a PLY file.

3.5 Integration by Volumetric Merging

For integration this thesis will describe a volumetric merging method which uses aMarching Intersection(MI ) algorithm [22] derived from theMarching Cubes5 method[19] introduced in section 2.4.

A standard volumetric method assigns a discrete voxel grid consisting of tiny voxelsto the input meshes. These voxels are not to confuse with the larger voxels beingdescribed in the voxelization algorithm in section 3.4.1. Each tiny voxel is assigneda distance function,D(x) and a weightW(x). The weight for each voxel is normallydependent on the direction of vertex normals and viewing direction.W(x) is increasingthe more similar the different directions are. The integration of the different meshesinto one consistent mesh is done by extracting an isosurface atD(x)=0.

The MI algorithm [21] [22] was developed to have good performance but with lessmemory occupancy. This is obtained by only storing the intersections of the mesheswith the grid instead of a 3D distance field over the whole surface as in the standardvolumetric method described above.

The algorithm is implemented in the toolPlyMC [4] which is used for the mergingof the Parthenon geometry. There are different parameters available, the most impor-

5Marching Cubes; a pre-processing operation of surface extraction

23

tant are voxel size which determines the resolution of the finally merged mesh. Thesmaller voxel size the more time and memory consuming merging process why it isimportant to determine the largest size without loosing resolution. To avoid loosingdetails in the current meshes the size of the tiny voxels has to be 1.2cm× 1.2cm.

The most important steps in PlyMC are the following:

For each mesh to be produced:

1. Compute the bounding box for the different input meshes.

2. In the bounding box, make a voxel grid of tiny voxels with the chosen size.

3. Perform the volumetric merging by using theMI algorithm.

4. Create and save an output mesh.

3.5.1 Bounding Box Parameters

The larger voxels are obtained by regularly partition the Parthenon space (described insection 3.4.1) and thus have the same size but contain different geometry, which notalways extends all the way to the boundaries of the voxels. If using the ordinary PlyMCthe computed bounding boxes for the large voxels will not have the same size. Whenadjacent voxels are processed in PlyMC and divided into the tinier voxels, the resultinggrid intersections in the overlap regions do not correspond to each other. Consequently,the same vertex but in overlapping adjacent voxels are not likely to have the same 3Dposition in both voxels after merging since the voxel grids might be slightly shifted.

The MI algorithm, as mentioned earlier, is based on the intersections in the voxelgrid. It stores the intersections of the geometry and the grid and creates new verticesat every intersection. In order to have the final voxels connecting without seams thevoxel grid used for merging also have to line up perfectly, i.e. the bounding boxes forthe voxels have to be of the same size and divisible with the tiny voxel size. This isobtained by changing PlyMC not to compute the bounding box itself but to use giveninput parameters for it.

Providing PlyMC with the correct bounding box parameters based on the positionof the large voxels in the Parthenon grid, the voxel grid used in the merging phase alsoconnects seamlessly. This results in vertices in the overlapping regions having the samepositions before as well as after merging (figure 3.10). An exception, though, is if thevertices are too close to a boundary edge of a voxel. Close to an edge the MI algorithmgives different results for the same vertex since it does not have the same neighboringdata as input (figure 3.11).

The existing redundant triangles in the overlapping regions are considered and pro-cessed in the next step in the pipeline (see section 3.6.1). Moreover, the changes inPlyMC and the input bounding box parameters are explained further in section 4.4.

3.6 Post-Processing Procedures

The post-processing procedures of the final voxels in the pipeline involve removingredundant polygons, mesh editing and mesh simplification. First redundant trianglesare removed followed by clean up and hole filling. Finally, mesh simplification isperformed.

24

Figure 3.10: Grid after merging; original(left) and using new parameters(right).

Figure 3.11: Boundary edge case.

3.6.1 Redundant Triangle Representation

Turk and Levoy [27] have developed a program calledzipperwhich central algorithmremoves overlapping portions of meshes and then clips one mesh against another. Tosolve the problem with the redundant triangle representation in the overlap regions inthe voxels zipper is modified.

First, between all adjacent voxels a boundary halfway between the two correspond-ing meshes,A andB, is determined. Next, redundant triangles are discarded by tessel-lating all triangles on meshA. Triangles before and on the boundary are preserved,the ones positioned past the boundary are discarded. Continuing with meshB, all tri-angles in its overlapping region that are on the boundary or redundant with the onespreserved in meshA are removed. These operations result in all redundant trianglesbeing discarded.

In some cases this procedure for removing redundant triangles will leave tiny,hardly visible, holes where the two meshes meet. Therefore, in some places addi-tional vertices are added and triangulated. Triangulation is easily solved since the twovertices next to the new vertex are known as well as the one these two vertices hadin common in the already existing triangle. These four vertices construct two newtriangles that make the two meshes join smoothly.

25

3.6.2 Mesh Editing

A clean up procedure is necessary since there during the scanning were scaffoldinginside and at some locations around the Parthenon. Digitizing items like scaffoldingis difficult since the geometry is very fine. Additionally, the scaffolding inside theParthenon were not completely captured when most scans are from the outside. There-fore and also because of esthetic reasons all scaffolding geometry are removed.

The following necessary mesh editing step is hole filling. Models acquired from3D scanning usually include holes since the scanning sensor usually cannot observethe entire surface of the object due to occlusion. In addition, there are black parts onsome locations of the Parthenon surface (see figure 3.12) that could not be registered,as well as plants growing on the structure preventing the sensor from registering theseregions.

Figure 3.12: Black areas will create holes in the geometry

The mesh editing is done in a semi-automatic process by using the commercialGSIStudio softwarefrom Geometry System Inc.6. It is claimed to be a complete softwarefor creating polygon meshes from range data but that was not the case. Firstly, thedatasets of the Parthenon are too large and could not be loaded into the memory of thesoftware. Secondly, their registration implementation was not fully developed. It wasnot possible to manually change the view, i.e. the 3D position of the camera of a meshin order to align it to another mesh.

Fortunately the data in the developed processing pipeline has been voxelized intosmaller pieces resulting in GSI Studio being convenient for mesh editing purposes.Important is also that loading and saving of PLY files are possible. The approach inGSI Studio uses the method mentioned in section 2.5.1 where a hole is being filled bycreating a surface with the topology of a simple disk.

Due to noise or non-accurate alignment there are tiny parts of geometry floatingaround in space in the merged voxels. In future processing (outside this thesis) of themodel it is essential that these polygons are removed. In GSI Studio it is possible tomanually select the geometry to be deleted.

A hole filling approach is integrated in the volumetric surface reconstruction algo-rithm in PlyMC, i.e. the software used for merging, but the result is not sufficient. Inthe software from GSI, holes are filled using a hole filling tool where the user selects

6Geometry Systems Inc, http://www.geometrysystems.net

26

the boundary of a hole followed by an automatic triangulation of that hole (furtherdescribed in section 4.5.2).

3.6.3 Decimation

The choice of decimation procedure has to suit the processes and requirements for thelarger project the model is used in. In that project an important feature is the abilityto preserve edges and texture coordinates of the meshes when decimated. The require-ments include low-resolution and high-resolution voxels connecting without seams andthe texture map for a voxel only computed once. Preserving the texture mapping spaceallows the same texture map to be used for either high or low-resolution geometry. Byusing theQSlim7 software based on the quadric error technique [12] introduced in sec-tion 2.5.2 these requirements are achieved. The model is decimated into three differentresolutions as shown in figure 3.13.

Figure 3.13: Decimation of the surrounding environment; 5, 10 and 20 percent respec-tively of the original amount of polygons.

7http://graphics.cs.uiuc.edu/ garland/software/qslim.html

27

Chapter 4

Implementation and Software

This chapter describes the various implementations further as well as it goes into detailabout how the software in the 3D data processing pipeline is used.

4.1 Filtering

The implementation of the Gaussian blur filtering program have to consider memoryissues. When performing the filtering algorithm a temporary portable float image withthe same size as the input portable float image is stored. Two full size range imagesin full resolution are not possible to store into the working memory on an ordinarywork station (1 GB RAM). Thus, an out-of-core technique had to be implemented. Bydividing the filtering process into two temporary parts, filtering first the left half of theimage and saving it on the hard drive and then repeating the same with the right halfthe problem is solved. The edge in between them is treated with one pixels overlapresulting in an continuous filtering. For the resulting image the left and right halves areimported one by one and the values are copied. This implies in worst case 1.5 full sizerange image being stored into memory.

To filter one single range image is rather time consuming. For efficiency reasonsthe filtering algorithm is implemented taking multiple (no upper limit) range images asinput. The size reduction, i.e. keeping every other point in both thei and j directionin the range images is performed in the same implementation. The result of this stepin the pipeline is that all range images are saved as new filtered range images having afourth of the original size (see figure 4.1).

Figure 4.1: Pre-processing of the range images.

28

4.2 Registration

The implementation of making the selection of regions possible provides an user inter-face. It is very important to carefully perform the selection of regions since this is alldata the actual aligning algorithm will be based on. In the optimal case there are fourpairs of regions where each pair includes the same part of geometry.

The complex task to perform the global registration, i.e. to determine which rangeimages to be pairwise aligned to each other is solved for regarding the pre-alignedscans. Range images without pre-alignment have no initial estimate and it is thereforeeven more important to select appropriate regions. Thus, manual work demanding timeand effort was spend finding the best solutions.

From the pre-processed range images the user selects the regions by dragging themouse over the image, the first region sets the size limit since a new PFM with a sizefour times the first selection’s size is created. When four regions (see figure 4.2) areselected the new range image is triangulated and ready to be imported intoMeshAlign.

Figure 4.2: Selection of regions(left) and corresponding new PFM:s(right).

The manual part in the semi-automatic aligning process is the initial estimate whichhas two different approaches inMeshAlign. These are either a point based method or amanual method.

The point based method allows the user to point out four pairs of points betweenthe mesh to be registered and its reference mesh. This method is not very efficient forthe geometry of the Parthenon since it showed to be very difficult to point out specificfeature points. Sculptures with more obvious details are more suitable for this approach[25].

The better but more manual method gives the user the ability to move and rotatethe mesh freely until it is placed in the correct position. In both methods, the processimplementing the ICP algorithm is performed. By comparison, the last method gives amore accurate alignment but is more time consuming than the point based method.

For some cases a solution with a two-step process had to be invented sinceMe-shAlignhas some restrictions. If the mesh which is to be registered is too far off fromthe reference mesh it is very hard for the user to perform the initial estimate in anaccurate way due to the way the 3D graphical user interface inMeshAlignis imple-mented. The total bounding box containing the geometry of the mesh pair is too largeand prevents the user to zoom in on details.

In the first step in the two-step process the range images are down sampled to a size

29

Figure 4.3: The manual alignment method inMeshAlign

possible to import the whole corresponding mesh intoMeshAlign(see figure 4.3). Afirst initial estimate is performed and the original range image is transformed by usingthe obtained ALN file. In the second step, regions are selected from this transformedrange image. Thereafter, when the triangulated range image is imported intoMeshAlignagain an accurate alignment is much easier to perform since zooming is possible, andICP can be run on high resolution geometry resulting in a second ALN file. The varioussteps in the registration phase are illustrated in figure 4.4.

Figure 4.4: Registration phase.

30

4.3 Voxelization

The voxelization constitutes one of the main problems in this thesis. It makes theprocessing and management of large datasets possible and efficient, why this sectionreceives much focus.

4.3.1 Adjustment of the Coordinate System

The coordinate system in which the scanner stores the data is not of standard typehaving itsz-axis pointing up. Additionally, both thexy-plane and thexz-plane are notperpendicular to the Parthenon structure. This results in the voxels not being alignedto the actual structure. Some voxels can only include a small part of a column whenother hold a lot of geometry. As a consequence a voxelization is not as efficient as ittheoretically can be. Thus, the voxels needs to be adjusted to the Parthenon in terms oflocation in 3D space.

In order to change coordinate system the actual orientation of the Parthenon have tobe found. First a plan of the desired voxelization is made, and for convenience the gridis chosen to align with the principal architectural features of the structure (see figure4.5).

Figure 4.5: Voxelization of the facades; east(top left), north(top right), west(bottomleft) and south(bottom right).

Thereafter, a voxel size of 4.2 meters is determined by averaging virtually per-formed measurements. The axis direction vectors are determined, and the cross prod-ucts calculated to prove that the basis is perfectly orthonormal. The rotation matrix isbuild by inserting thex, y andz vectors. To be able to transform the original coordi-nate system to the new one an anchor point has to be measured. Finally, a function isimplemented that rotates and translates a point in the original coordinate system intothe new. The result is a coordinate system having its origo at an edge of a voxel, andwhich is adjusted to have all points on the Parthenon having positive (x,y,z) values.

The adjustment of the coordinate system changes the partitioning of space so that itbecomes static, i.e. the grid containing the Parthenon structure is now always dividedinto 8×5×17 equally sized voxels. Previously, the grid was dynamic and the numberof voxels could be changed. Even though, that in order to obtain a reasonable sizeof the voxels, the total volume (with the environment included) was partitioned into

31

40×50×20 voxels. Additionally, besides the adjustment of the coordinate system allpoints belonging to the surrounding terrain are neglected and processed separately.

This change in the voxelization phase made the following integration phase sig-nificantly more efficient. Since the surrounding environment is not voxelized and thenumber of voxels including geometry is less, the time needed for the merging decreasedfrom 46 hours to 5.5 hours.

4.3.2 Partitioning

The idea behind the voxelization is to traverse through each range image and organizeall non-zero (x,y,z) points into corresponding voxels (see figure 4.6). An overlappingregion is used in order to get a continuous surface with connecting points over theboundaries. Consequently, a point possibly belongs to one up to eight voxels. The sizeof the overlapping region is a dynamic parameter, thus the overlapping region can beadjusted to suit different resolutions of a range image. The index for a voxel consistsof a value representing how far away in the (x,y,z) directions from origo the voxel islocated. In the Parthenon case the indices start atvoxel_0_0_0and continues up tovoxel_7_4_16.

The most time consuming phase is the traversal of the huge range images why eachrange image only is traversed once. The algorithm follows this procedure:

For each point in the range image:

1. Get the (x,y,z) value if non-zero.

2. Transform the point into the new coordinate system.

3. Calculate the 3D position in the current voxel grid to obtain a voxel index for thepoint.

4. Check if the point lies within a boundary region and should be treated as an over-lap point. If so, determine additional voxel index (up to seven possible additionalcases).

5. Store the original (i,j) position from the 2D range image for the current point andthe determined voxel index/indices in the voxel array.

The voxel array storing the data consisting of voxel indices and the (i,j) positions ofthe points in the voxels tend to be very large (see table 4.1). For efficiency and memoryissues a resize function of the array is implemented.

Phase Number of points (maximum)

Original range image 18000×3200 = 57,600,000

Pre-processed range image57,600,000÷4 = 14,400,000

Table 4.1: Number of points to be voxelized.

The way the voxel grid is determined was implemented in two different ways. Thefirst version have a dynamic voxel grid depending on the input range images. The

32

maximum and minimum (x,y,z) extents considering all range images is determined andset as the final voxel grid. The user then adjusts the parameters deciding how manytimes each direction is divided. During the voxelization all range images are traversedtwice. This version is preferred when the extents of the geometry is unknown.

The second implementation has a voxel grid adjusted to known measurements ofthe Parthenon coordinates (described in section 4.3.1). Thus, the time consuming gridcalculation is avoided and the range images are only traversed once resulting in a moreefficient voxelization. This is the implementation used for the final model.

Figure 4.6: Voxelization phase.

4.3.3 Triangulation

The kernel in the implementation of the triangulation algorithm is the connectivityfunction. Neighboring points from the two-dimensional range image are sent to thefunction and tested if they are to connect or not (figure 4.7). The connectivity is basedon two thresholds based on the euclidian distance between points in 3D space.

p1

p4p3

p2

Figure 4.7: Triangulation scheme.

Firstly, thedistance thresholdis simply the euclidian distance between the twopoints. Secondly, thescaled distance thresholdis the quota between thedistancethresholdand the average distance from the two points to the location of the scan-ner. The location of the scanner for each scan is obtained from a separate file whichincludes all scanner locations for each session. Consequently, points that are furtheraway from the scanner are connected even though the distance between the two pointsis greater than thedistance threshold. This is necessary due to a sparser distributionof points the further away from the scanner location. By using suitable thresholds,accurate geometry without unnecessary holes and incorrect connections is obtained.Equations 4.1 and 4.2 show the thresholds deciding whether the pointsp1 andp2 willconnect or not.

distance_threshold=√

(p1x−p2x)2 +(p1y−p2y)2 +(p1z−p2z)2 (4.1)

33

scaled_distance_threshold=distance_thresholdscanner_distance

(4.2)

The thresholds (see table 4.2) in the connectivity function are easily adjusted to suitdifferent resolutions of the range images. This is convenient since subsampling showedto be necessary for certain cases in the alignment phase (mentioned in section 4.2).

Range image Distance threshold Scaled distance thresholdOriginal (100%) 3.0 cm 0.0002

Pre-processed (25%) 6.0 cm 0.0004

Table 4.2: Connectivity thresholds.

The triangulation problem, i.e. to triangulate based on the image plane but frompoints stored in a voxel array (introduced in section 3.4.2 and illustrated in figure 4.8)is solved and implemented in following algorithm. The algorithm is possible since thepoints in the array appear in scanline order.

(i-difference andj-difference are the differences between adjacent points’ differenti-andj-values)

For all voxels in each range image:

1. Get the extent in thei-direction. This indicates maximum number of searchesallowed in order to findp3andp4.

2. Loop through the voxel array, when between two adjacent points thei-difference=1andj-difference=0 the pointsp1andp2are found. Then perform a search forp3(i-difference=0 andj-difference=1 betweenp1 andp3) and finally a search forp4 (i-difference=1 andj-difference=0 betweenp3 andp4). The connectivity forthe found points are tested by using the connectivity function.

3. Let p2bep1and go to step 2.

4. Repeat until the end of the voxel array.

5. Goto step 1 and compute the next voxel.

p1 p3 p4p2

extent in i-direction

Figure 4.8: The voxel array.

After all voxels have been processed by the algorithm the vertices and faces (i.e.the indices to the vertices) are saved in the PLY file format and each voxel mesh gets aspecific filename revealing which original range image it cames from and which voxelit is. The writing of the PLY file is not described in the algorithm above, only thesolution to the triangulation problem.

34

4.4 Integration

In the integration phase the voxels are merged one by one using the software PlyMC.Every voxel in the Parthenon grid contains various numbers of meshes which all aregiven as input to the program (see figure 4.9).

To solve the overlapping issue to obtain connecting voxels without seams evenafter merging a bounding box function is implemented giving PlyMC extra parameters.The bounding box in PlyMC is the volume used in the volumetric merging algorithmdivided into the tiny voxels. The way PlyMC is implemented the bounding box has tofulfill some requirements. Firstly, it has to be divisible by the tiny voxel size (0.04feet).Secondly, the number of tiny voxels has to be divisible by eight. This results in abounding box function computing values for where in 3D space the bounding box isgoing be located depending on the original position of the voxels in the Parthenon grid.The result from the function are six values, i.e. minimum and maximum values for thex, y andz directions.

In the merging of every voxel the input parameters to PlyMC are the PLY files, thetiny voxel size, the six computed bounding box values and finally the output file name.When the merging of a voxel is finished a new PLY file is saved, still containing theindices referring to where in the Parthenon grid the voxel is located.

Figure 4.9: Integration phase.

4.5 Post-Processing

The post-processing is performed both by using and by modifying existing software. Itis a time consuming phase since it often demands user assistance.

4.5.1 Zipper used on the Voxels

To discard redundant triangles in the different voxels the modified zipper program de-scribed in section 3.6.1 is automatized in an intelligent, but relatively time consumingway. Starting withvoxel_0_0_0this voxel is processed together with the three adjacentvoxels in the positivex, yandzdirections (see figure 4.10). Next, the neighboring voxelin thex-direction is processed with its three adjacent voxels. This is repeated until thelast voxel and results in a final mesh without redundant triangles.

Figure 4.10: Removal of redundant triangles.

35

4.5.2 Mesh Editing in GSI Studio

GSI Studio has an user interface very much like other common 3D software1 and pro-vides tools for mesh editing.

First, floating polygons and scaffolding geometry are removed. The methods usedfor removing geometry are a painting technique in three dimensions to indicate whatto delete, and also by making bounding boxes around the polygons to be removed.Additionally, it is possible to let the program automatically select all floating verticesand single polygons and then remove them.

Regarding hole filling, holes can be selected both in an automatic and manual way.When selecting manually the boarder around the hole is selected. It is then filled byall connecting components of the surface boundary being triangulated using a patchhaving the topology of a disc.

Figure 4.11: Hole filling; before(left) and after(right).

For holes with more complicated topology one or several bridges are manuallymade across the hole at appropriate locations. The bridges are both straight and cur-vature based. Since the hole filling phase is very manual it is time consuming, a nicefeature is that multi selection and processing of holes is possible. Figure 4.11 visualizesa hole being filled and figures 4.12 illustrates the mesh editing and decimation process.

Figure 4.12: Mesh editing and decimation.

13D software; Mayar (Alias), 3D Studio Maxr (Discreet)

36

4.6 Automatization of the Pipeline Process

The implementations in the 3D data processing pipeline are developed to process onepiece at a time. This since most processes in the implementations are time consumingand handles huge datasets. A central issue in the pipeline is to make the processingefficient and as independent of user assistance as possible. In order to see if imple-mentations perform as expected various tests are made continuously. Even many smalltests tend to be time consuming and therefore an automatization of certain processes inthe pipeline was implemented at an early stage.

Firstly, a graphical user interface (see figure 4.13) for selecting which data, i.e.which range images, to voxelize makes that phase more convenient. The application isindependent from the actual voxelization implementation which is called from the userinterface application on demand of the user.

Figure 4.13: Graphical user interface; voxelization(left) and merging(right).

Secondly, and more important, an automatization of the merging procedure is addedto the application. The data to be merged are represented as a large number of PLY files.For each range image it is theoretically possible to exist up to 680 (8×17×5) differentvoxel meshes, i.e. PLY files. This is never the case since no range image containsdata in all voxels. Anyway, having 54 range images in the final dataset, the possiblenumber of files to be merged is very large. The automatization interface has the samestructure as for the voxelization but differs regarding parameters and is more dynamicin selecting the input data.

The main purpose of this automatization implementation is to easily manage themerging of the voxels, e.g. to find and start the merging of all voxels in the gridobtained from all range images in the data set. This is solved by iteratively parsingthe names of the files, and possible since they are named in an intelligent way.

An automatic merging procedure of a whole dataset is not always necessary, there-fore it is possible to merge only one single set of voxels by using the same application.This is done by the user selecting the appropriate files manually.

4.7 The Surrounding Environment

The reconstruction of the surrounding environment around the Parthenon (see figure4.14) does not follow exactly the same procedure as the structure itself. The main rea-

37

son for this is that the requirements for the resolution and accuracy are not as high forthe terrain as for the Parthenon. Also contributing is that the number of range imagescontaining the surrounding environment is less since the focus of the data acquisitionwas on the structure itself. The differences in the processing pipeline are following:

• The range images in the chosen dataset are subsampled one more time for the sur-rounding environment than previously. This is done after the registration phaseand reduces the file size with a factor of 16 compared to the original range im-ages.

• The implementation of the voxelization is extended to suit the new purpose. Aninverse function keeping only the data not belonging to the Parthenon structureis implemented. As before, all points are transformed into the new coordinatesystem. Since the file size now is relatively small the partitioning is changed toonly compute one single mesh.

• In the integration phase, the tiny voxel size used in the volumetric merging algo-rithm is set to a higher value suiting the new and lower resolution of the rangeimages. By having less input data and lower resolution it is possible to mergethe entire ground (200m×200m) in one piece.

• The post-processing of the surrounding environment differs since there are otherthings that affects the clean up procedure. People walking by are captured insome of the range images and the surrounding environment contains trees thatare not completely scanned, all this are being removed.

Figure 4.14: The reconstructed surrounding environment of the Parthenon.

38

Chapter 5

Results

This chapter presents the obtained results, both regarding the results of the chosen dataprocessing pipeline and the final model of the Parthenon.

5.1 Processing Pipeline

The main part of the chosen processing pipeline to produce the final model is visualizedin figure 5.1, and all phases are described in table 5.1. The table describes whether thephase involves processing data one piece at a time or if it is automatized. The timeaspect is also commented (all processing times are measured on a modern workstationwith 1GB RAM).

QPG PFM

conversion

PFMp

pre-processing

aligning

x ... ... ...... y ... ...... ... z ...0 0 0 1

ALN

PFMt

transformation

PFMp

merging

voxelization

Figure 5.1: Flowchart of the 3D data processing pipeline for producing the model.

The pipeline starts with the conversion of the original data (QPG) produced fromthe scanner into the portable float image format (PFM), each range image processedindividually. After the conversion the data can be pre-processed, which is time con-suming and for efficiency this is automatized avoiding user assistance. From the pre-

39

processed data the registration phase starts with the pairwise selection of regions re-sulting in ALN-files. Each range image is then transformed individually. Thereafter,the voxelization where all range images are processed simultaneously follows. Finally,the voxelized data is merged one piece of geometry at a time. The automatizationapplication makes it possible to process the voxels continuously.

Phase Input CommentConversion Single range image 1-2 minutes per range imagePre-processing Multiple range images Time consumingRegistration Pairwise range images Manual, very time consumingVoxelization All range images Relatively fastIntegration All voxels (one at time) 5.5 hours for the final datasetRedundancy All voxels (four at a time) Very time consumingPost-processing Single voxel Manual, very time consumingDecimation All voxels Fast

Table 5.1: Description of the phases in the processing pipeline.

The last part of the processing pipeline is the post-processing involving removal ofredundant polygons, clean up, hole filling and decimation. Most phases, shown in aflowchart in figure 5.2, during the post-processing are very time consuming comparedto the first part. Firstly, when discarding redundant triangles four voxels have to be pro-cessed simultaneously in order to complete one single voxel. Clean up and hole fillingare user assisted and therefore extremely time demanding. The decimation phase, onthe other hand is automatized, processing all the voxels efficiently.

discard redundancy clean up, hole filling

decimation

Figure 5.2: Flowchart of the post-processing phase.

5.2 Reconstructed Model

The presentation of the results of the reconstructed model is divided into partial andfinal results. The results of the integration with the larger research project are demon-strated in section 5.3.

40

5.2.1 Partial Results

The first goal, to obtain a 3D model of the east facade was achieved early during thisthesis work. By using only a few range images it is possible to construct a rough modelof a part of the geometry. The complex task is to manage all data and to process andintegrate it simultaneously. Additionally, the post-processing phase is time consumingand demands a great deal of manual work.

Metopes

Figure 5.3: The east facade assembled from four range images(left) compared to thesame part from the completely processed final model(right).

The most visible difference between the two models in figure 5.3 are that geometryare missing on the stairs and on the floor in the left model. This is due to the angleon the scanning sensor producing the range images used in this first model. The angleprevented the scanner from acquiring that data and this is the reason for having differentcategories of range images (see section 3.3.3) in order to obtain as much geometry aspossible. Also, the columns are not complete in the left model since the backsides aremissing. The holes on the metopes and on the floor are filled during the post-processingphase resulting in the rightmost model.

Figure 5.4 shows a rendering of the geometry of the whole Parthenon on a planesurface. The model is past the integration phase but only the east and west facades havebeen post-processed, i.e. unwanted geometry have been removed and holes have beenfilled. The middle section still contains scaffolding geometry and holes. Having themodel partitioned into voxels makes it convenient to conform to the goal, to focus onthe two facades during the whole processing pipeline but on the same time include allparts of the model.

41

Figure 5.4: A rendering of the whole model before completion of the post-processingphase.

5.2.2 Final Results

By assembling data from 54 different range images obtained from various locationsaround and inside the structure, a final model of the Parthenon is produced (see figure5.6). The locations of the scanner producing the range images are visualized in figure5.5. This shows that the distribution is more dense on the east and west facades thanalong the north and south sides. As mentioned earlier scaffolding and other equipmentprevented placing the scanner on these sides.

The number of non-empty voxels in the final model of the Parthenon is 442. Thecomplete non-decimated version contains 87 million polygons and the lowest-resolutionmodel uses 1.8 million polygons. The surrounding environment contains 366 000 poly-gons.

42

Figure 5.5: Scanner location map (seen from above).

Figure 5.6: The final model and environment. The white area indicates a voxel and theblue regions show its neighboring voxels.

5.3 Project Integration

The integration into the larger research project is performed by importing voxels ofdifferent resolutions. The algorithm is iterative and in each iteration one voxel of highresolution and surrounding voxels in medium and low resolution are processed.

Figure 5.7 compares the geometry in a resulting rendering of the reconstructedmodel from the larger project with a real photograph. By comparing the two imagesin this figure it is possible to evaluate if the model achieved the aim of this thesis. The

43

texturing of the geometry in the rendering is very much alike the real photograph, evensmall details have the same texture properties showing that the geometry of the modelis accurate compared to the real Parthenon. Noticeable are the differences regardingthe scaffolding in the two images. Since the scaffolding were removed during the post-processing phase they had to be manually modeled and added to the scene in order tocompute the algorithms included the larger project in an correct way.

Figure 5.7: A real photograph(left) compared to a rendering under the same illumina-tion (right).

Finally, the larger research project resulted in many interesting renderings and an-imations. Figure 5.8 shows a synthetic rendering of the east facade under sunset andfigure 5.9 how artificial lighting illuminates the west facade of the Parthenon.

Figure 5.8: Rendering from the larger research project; the east facade.

44

Figure 5.9: Rendering from the larger research project; the west facade.

45

Chapter 6

Discussion

The following section discusses the process and its results. A conclusion to the thesiswork is presented followed by comments on possible future work.

6.1 Process discussion

The implementation of thevoxelization, i.e. the partitioning into voxels, is definitelynecessary for the processing pipeline. The resulting high resolution model would havebeen impossible to merge otherwise. Moreover, it made the handling of the hugedatasets more convenient, e.g. remerging of just one or a few voxels instead of thewhole dataset is possible. The efficiency of the voxelization implementation made itconvenient to add more data to the current dataset during the pipeline processing.

The voxelization implied that the otherwise unnecessaryzipperprogram had to bemodified and used. Additional phases in the pipeline demand more time but since theresulting quality of the final model would not have been achieved otherwise this wasnecessary. As described in chapter 1 the aim is to use the final model in a larger researchproject. This would also not have been achieved without the voxelization since the sizeof one single mesh would have been impossible to process.

Regarding3D scanning techniquesthere are some recommendations to be done.The choice of target object showed not to be optimal, there are too much scaffold-ing and other equipment preventing the scanning to be smoothly performed. It alsoincreases the manual work in the post-processing phase.

Performingpre-aligningon the site showed to be very helpful in the registrationphase. Since this phase is not automatic, significantly more time to obtain the ini-tial estimate used in the ICP algorithm would have been demanded without the pre-alignment.

A drawback of the pre-alignment procedure is that since tilting is not allowed thedistance between the scanner and the target object have to be long in order to capturethe whole height of the Parthenon. Scanning from a long distance showed to increasethe amount of noise in the resulting range images.

The result of the adjustment of thecoordinate systemhad great impact on the effi-ciency of the pipeline. After the adjustment the number of voxels containing geometryis less, as well as the geometry more evenly distributed which results in fewer voxelsbeing processed during the following phases.

In the modification of the programzippera boundary in the overlap region halfway

46

between adjacent voxels is determined. This boundary can be determined in a smarterway, e.g. finding the flattest regions or where the polygons in the different meshes aremost similar. The flatter or more similar the adjacent meshes are the less is the need foradding additional vertices, resulting in a more efficient removal of redundant polygons.

Since thepost-processingphase is significantly more time consuming than the otherphases the efficiency of the total pipeline would increase if a more automatic procedurecould be developed. As mentioned in [9] diffusion can be used to complete a volumetricrepresentation of the surface.

Regarding themanagement of huge datasetsa relatively great part of the time dur-ing this thesis was spent on arranging large numbers of files using several computers.

Before the final pipeline could be developed many tests were performed workingwith smaller datasets. The PLY polygon file format [30] described in section 3.4.3showed to be very useful and efficient during the data processing for managing hugedatasets.

6.2 Conclusion

This thesis describes a 3D data processing pipeline for digitizing the shape of a largebuilding, more specifically the Parthenon. Existing algorithms and software are usedas well as new methods being invented in order to make the production of a final modelpossible. A complete 3D model is reconstructed by using the developed pipeline and isavailable in different resolutions and file formats ready to be texture mapped and usedfor photorealistic viewing.

The aim of the 3D model is to be integrated into a larger research project at theICT resulting in a publication of scientific paper, and a film for the electronic theaterat SIGGRAPH 2004. Figure 5.7 proves, when comparing the rendering with the realphotograph that the aim is achieved. By digitizing the famous and historic valuableParthenon temple viewers gain new possibilities for visual and intellectual experiences.

To conclude, the decreasing cost of 3D range scanning and more tools being devel-oped and automatized make the digitizing of large-scale models possible. The devel-oped data processing pipeline in this thesis can be used for recreating other large scaleobjects from real world.

6.3 Future Work

In the area ofdata acquisitionand future work, capturing more data of the Parthenoncould be done. Range images acquired from a short distances showed to give accurateand high resolution geometry why it would be interesting to see how more data in thisform can improve the parts of the final model not completely captured today.

Currently, the low-resolutionsurrounding environmentis obtained by subsamplingin the image plane. Instead, a more intelligent mesh simplification approach could beimplemented. Simply by considering a point’s distance to the scanner location, thedense distribution of points very close to the scanner could be subsampled more thanthe sparser representation further away. Atolerance distance[27] between points indense areas could be used in order to remove a point’s all neighboring points withinthis distance.

This can in the future result in a more realistic model of the surrounding environ-ment having higher resolution in areas including details. The developed3D processing

47

pipelineto reconstruct a model from data acquired from range scanning can be used infuture work to produce other models. During the scanning in Athens range images ofthe smaller building, the Erecthion on the same site was captured. By using the sametechnique as for the Parthenon this work has already been started (see figure 6.1).

Finally, the extremely large size of the datasets needed to build a complete largescale model is still one of the major challenges in this area of research and future work.

Figure 6.1: Future work involving producing a model of the Erecthion.

48

Bibliography

[1] Allen, P., Troccoli, A., Smith, B., Murray, S., Ioannis, S., Leordeanu, M. 2003.New methods for digital modeling of historic sites.Proceedings IEEE Computergraphics and Applications. pp. 32-40.

[2] Besl, P., McKay, A. 1992. A Method for Registration of 3D Shapes.IEEE Trans-action on Pattern Analysis and Mashine Intelligence. vol 14, pp. 239-256.

[3] Y.Chen, Medioni, G. 1992. Object modeling from multiple range images.Imageand Vision Compting 10. pp. 145-155.

[4] Calleri, M., Cignoni, P., Ganovelli, F., Montani, C., Pingi, P., Scopigno, R. 2003.Vclab’s tools for 3d range data processing.VAST 2003 and EG Symposium onGraphics and Cultural Heritage.

[5] Chetverikov, D., Stepanov, D., Kresk, P. 2002. The trimmed iterative closest pointalgorithm.Proceedings ICPR.

[6] Cignoni, P., Montani, C., Scopigno, R. 1998. A comparison of mesh simplifica-tion algorithms.Computer Graphics. pp. 37-54.

[7] Cignoni, P., Rocchini, C., Montani, C., Scopigno, R. 2003. External memorymanagement and simplification of huge meshes.IEEE Trans. On Visualizationand Computer Graphics.

[8] Curless, B., Levoy, M. 1996. A Volumetric Method for Building Complex Modelsfrom Range Images.Proceedings of SIGGRAPH 96. ACM, pp. 303-312.

[9] Davis, J., Marschener, M., Levoy, M. 2002. Filling holes in complex surfaces us-ing volumetric diffusion.First International Symposium on 3D Data Processing,Visualization, Transmission.

[10] Dey, T.K., Hudson, J. 2002. PMR: Point to mesh rendering, a feature-based ap-proach.IEEE Proceedings of the conference on Visualization. pp. 155-162.

[11] Dias, P., Sequeira, V., Vaz, F., Gonçalves, J.G.M. 2003. Registration and fusionof intensity and range data for 3D modelling.Proceedings Fourth InternationalConference on 3D Digital Imaging and Modeling.

[12] Garland, M., Heckbert, P.S. 1998. Simplifying surfaces with color and textureusing quadric error metrics.IEEE Visualization 98. pp. 263-270.

[13] Gelfand, N., Rusinkiewicz, S., Ikemoto, L., Levoy, M. 2003. Geometrically stablesampling for the ICP Algorithm.Proceedings Fourth International Conference on3D Digital Imaging and Modeling.

49

[14] Hancock, J., Langer, D., Hebert, M., Sullivan, R., Ingimarson, D., Hoffman, E.,Mettenleiter, M., Froehlich, C. 1998. Active laser radar for high-performancemeasurements.Proceedings IEEE International Conference on Robotics and Au-tomation.

[15] Hoppe, H., DeRose, T., Duchamp, T, McDonald, J, Stuetzle, W. 1992. Surfacereconstruction from unorganized points.Proceedings of SIGGRAPH 92. ACM,pp. 71-78.

[16] Ikeuchi, K. 2001. Modeling from reality.Proceedings Third International Con-ference on 3D Digital Imaging and Modeling. pp. 117-124.

[17] Korres, M. 2003. The Parthenon until 1687.Antiken Museum Berlin.

[18] Levoy, M., Pulli, K., Curless, B., Rusinkiewicz, S., Koller, D., Pereira, L., Ginz-ton, M., Anderson, S., Davis, J., Ginsberg, J., Shade, J., Fulk, D. 2000. The digitalmichelangelo project: 3d scanning of large statues.Proceedings of SIGGRAPH2000. pp. 131-144.

[19] Lorenson, W.E., Cline, H.E. 1987. Marching cubes: a high resolution 3D surfaceconstruction algorithm.Computer Graphics. pp. 163-9.

[20] Miyazaki, D. et al. 2000. The great Buddha project: modelling cultural heritagethrough observation.Proceedings of the Sixth International Conference on VirtualSystems and MultiMedia. pp. 138-145.

[21] Rocchini, C., Cignoni, C., Montani, C., Scopigno, R. 2000. The marching inter-sections algorithm for merging range images.Technical Report B4-61-00. I.E.I -C.N.R..

[22] Rocchini, C., Cignoni, P., Ganovelli, F., Montani, C., Pingi, P., Scopigno, R.2001. Marching intersections: an efficient resampling algorithm for surface man-agemant.International Cconf. on Shape Modeling and Applications. pp. 296-305.

[23] Rushmeier, H., Bernardini, F., Mittleman, J., Taubin, G. 1998. Acuiring inputfor rendering at appropriate levels of detail: digitizing a Pietà.Proceedings 8thEurographics Rendering Workshop. Springer-Verlag, pp. 35-44.

[24] Rusinkiewicz, S., Levoy, M. 2001. Efficient Variants of the ICP Algorithm.Pro-ceedings Third International Conference on 3D Digital Imaging and Modeling.

[25] Stumpfel, J., Tschou, C., Yun, N., Martinez, P., Hawkins, T., Jones, A., Emerson,B., Debevec, P. 2003. Digital reunification of the Parthenon and its sculptures.The Eurographics Association 2003.

[26] Tournikiotis, P. 1994. The Parthenon and its impact in modern times.MelissaPublishing House. Athens.

[27] Turk, G., Levoy, M. 1994. Zippered polygon meshes from range images.Pro-ceedings of SIGGRAPH 94. ACM SIGGRAPH / ACM Press, pp. 311-318.

[28] Waltz, F, Miller, J. 1998. An efficient algorithm for Gaussian blur using finite-state machines.SPIE Conference on Machine Vision Systems for Inspection andMetrology.

50

[29] Wheeler, M.D., Sato, Y., Ikeuchi, K. 1998. Consensus surfaces for modeling 3Dobjects from multiple range images.Proceedings ICCV 98.

Web addresses

[30] Turk, G. 1998. The PLY Polygon File Format.http://www.ics.uci.edu/ graph-ics/teaching/ply.html. (Acc. 2004-03-25)

51

Appendix A

List of Abbreviations

Abbreviation Explanation

ALN Alignment FileGSI Geometry System Inc.ICP Iterative Closest PointICT Institute for Creative TechnologiesMI Marching IntersectionsMSE Mean Square ErrorPFM Portable Float MapPLY Polygon File FormatQPG Quantapoint Format

Table A.1: List of Abbreviations

52

Appendix B

The PLY Polygon File Format

Developed by Turk in 1998 [30].

A PLY file contains the description of exactly one object and includes only that ob-ject’s elementsand possible also the element’sproperties. In a typical PLY file thecollection of (x,y,z) triples for vertices and a list of faces that are described by indicesinto the list of faces are the core elements. Each element has a fixed number of proper-ties that are specified for each element. Applications can create new properties that areattached to elements of an object. The most basic example is to associate the propertiesred, greenandbluewith the vertex elements.

Structure of a typical PLY file:HeaderVertex listFace list(List of other elements)

The text in the header describes the remainder of the file. It begins by telling whetherthe file is binary or ascii followed by lines of possible comments. Thereafter comes adescription of each element type, including the element’s name (e.g.vertex), how manyof these that are contained in the object and finally a list of the properties associatedwith the element.

Table B.1 contains a complete ascii description for a cube. The only difference inmaking it a binary file is to replaceasciiwith binary_little_endianorbinary_big_endian.The comments in brackets are not part of the PLY file, they are explanations to the ex-ample. PLY file comments are lines that begin with the wordcomment.

The propertyvertex_index, as described in the header contains an integer tellinghow many indices to vertices the property has in the list followed by the indices them-selves.

53

PLY file Explanation

ply format ascii 1.0 ascii/binary, format version numbercomment Format made by Greg Turkcommentskeyword specifiedelement vertex 8 definevertexelement (eight in this file)property float x vertex contains float x coordinateproperty float y y coordinate is also a vertex propertyproperty float z z coordinate tooelement face 6 there are sixfaceelements in this fileproperty list uint8 int32 vertex_index vertex_indicesis a list of intsend_header delimits the end of the header0 0 0 start of vertex list0 0 10 1 10 1 01 0 01 0 11 1 11 1 04 0 1 2 3 start of face list4 7 6 5 44 0 4 5 14 1 5 6 24 2 6 7 34 3 7 4 0

Table B.1: The PLY file format

54