MeshLab and Arc3D: Photo-Reconstruction

and Processing of 3D meshesP. Cignoni, M Corsini,

M. Dellepiane, G. Ranzuglia,(Visual Computing Lab, ISTI - CNR, Italy)

M. Vergauven, L. Van Gool(K.U.Leuven ESAT-PSI - ETH Zürich D-ITET-BIWI)

Intro• Arc3d + MeshLab

– a complete free software pipeline for the 3D digital acquisition

– based on standard photographic equipment

– Arc3D• A free web based 3D reconstruction service, you

upload photos and you get sequences of aligned depth maps

– MeshLab• An open source mesh processing system for

cleaning, aligning and merging meshes and range maps.

Arc3D: Architecture1. Record a sequence of

images of a scene or object

2. Upload the images tothe ARC server

3. The server computesthe 3D reconstruction

4. Download the resultsfrom the ARC website

5. Process and Visualize the results with MeshLab

Arc3D: How it works

• The entire process is based on finding matches between images

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Arc3D: the output• For each submitted image a

depth image is reconstructed– an image with the distance from the

camera for each pixel– a quality estimation for each pixel

• All these depth images must be – cleaned and filtered– integrated into a full model.

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.QuickTime™ and a

TIFF (Uncompressed) decompressorare needed to see this picture.

A typical Epoch - CI Tool• MeshLab - an epoch result

– Developed at ISTI - CNR

• Just an example of the kind of toolsthat we are targeting for building the pieces a common infrastructure tool chain

• Domain– Digital representations of 3D shapes

• Objectives– Linking together input and output

of different CH apps– Adapting data from one CH app to

another (mesh processing!)

MeshLab

• Generic mesh processing tool• Open source GPL avail for win, mac and linux

(scheduled for inclusion inside debian-science)

– The system relies on a large GPL library for mesh processing

• Aimed to the (almost) automatic processing of large unstructured 3d models– acquired 3D models are different from human built

models • No structure, no particular need of a scene graph• Large datasets : millions of primitives

MeshLab

• Able to manage many different kind of mesh formats

• Input format PLY, STL, OFF, OBJ, 3DS, COLLADA, PTX, X3D, VRML

• Output format PLY, STL, OFF, OBJ, 3DS, COLLADA, X3D, VRML, DXF

MeshLab dissemination• License GPL• Code size: 35k + 65k LOC (>20 py estimated )• V1.0.0 (march 2007)

– Downloaded >30k times, – Web site served more than 400k pages– more then 2000 faithful users (users that opened

more than 50 meshes), hundreds of university and research centers using it all over the world

• V1.1.0 (feb 2008)Downloaded 5k times in three weeks

MeshLab typical tasks• Filtering

– Removal of outliers according to variousheuristics

• Smoothing out noise– Various fairing algorithms

• Editing – Selection of parts and removal of non

interesting portions of the acquired data

Remeshing• Subdivision Surface (loop and butterfly) • Reducing complexity through

simplification– Both fast approximate and slower high

quality algorithms – 500k 50k 5k

Checking

• Visual inspection– See through filters

• Quality evaluation– Curvature dependent, …

• Automatic marking of mesh inconsistencies– topological: non manifoldness– Geometric: self intersection

Repairing

• Hole Filling– Automatically interpolate small missing

mesh portions filling small gaps

MeshLab: Aligning and Merging

• 3D Scanning Pipeline Tools– ICP based alignment – Including global alignment for

error redistribution– Surface reconstruction

algorithms for rangemap merging

MeshLab Features

• Aligning functionalities needed to bring together models reconstructed from different sequences:

MeshLab Features• Merging tools can be used for blending all the

reconstructed range maps into a single (eventually watertight) mesh

• Various surface reconstruction algorithms– Poisson Surface Reconstruction

• Watertight mesh, sometimes can invent too much

– Volumetric distance field• Preserve and blend color information • Do not fill large holes

MeshLab Recon. Sample• Arc du triomphe• >100 photos• ~1M triagles

reconstructed model

• No texture here to better show the geom quality.

• Reconstructed with the distance field approach

MeshLab Recon. Sample• One of the two lions

in the Loggia dellaSignoria in Florence

• >30 photos• 200k triangles

reconstructed model• Hi quality color

reconstructed• Reconstructed with

the distance field approach

MeshLab Recon. Sample

• A Lion statuefrom thePortalada in Ripoll,a large romanic portal, near Barcelona

• >60 photos• 600k triangles

reconstructed model• No color but watertight• Reconstructed with the

Poisson surface reconstruction alg.

PhotoCloud Prototype

• A new way to use the results of Arc3D– Do not aim to a clean 3D mesh model, but retain

and exploit all the original photos information

PhotoCloud Prototype

Basic ideas– Mix a rough 3D with (many)

High Res photos – Allow free navigation in 3D space

and in the space of the images– Allow to to take measure on photos– Work in remote, everything (3D and photos) is

stored on a standard web server • You can show your reconstruction results without

requiring that people download giga sized datasets

Conclusions• Download and play by yourself!• Both tools are ready to use and

tested.– http://www.arc3d.be– http://meshlab.sourceforge.net

• Thanks for your attention!