A System for High-Volume Acquisition and Matching of Fresco Fragments Reassembling Theran Wall...
-
Upload
kathlyn-tate -
Category
Documents
-
view
217 -
download
0
Transcript of A System for High-Volume Acquisition and Matching of Fresco Fragments Reassembling Theran Wall...
![Page 1: A System for High-Volume Acquisition and Matching of Fresco Fragments Reassembling Theran Wall Paintings Benedict Brown 1,2, Corey Toler-Franklin 1, Diego.](https://reader035.fdocuments.net/reader035/viewer/2022070415/5697bf991a28abf838c91e41/html5/thumbnails/1.jpg)
A System forHigh-Volume Acquisition and Matching of Fresco Fragments
Reassembling Theran Wall Paintings
Benedict Brown1,2, Corey Toler-Franklin1, Diego Nehab1,3,Michael Burns1, Andreas Vlachopoulos4, Christos Doumas4,5,
David Dobkin1, Szymon Rusinkiewicz1, Tim Weyrich1,6
1Princeton University3Microsoft Research5National University of Athens2Katholieke Universiteit Leuven
4Akrotiri Excavations, Thera6University College, London
![Page 2: A System for High-Volume Acquisition and Matching of Fresco Fragments Reassembling Theran Wall Paintings Benedict Brown 1,2, Corey Toler-Franklin 1, Diego.](https://reader035.fdocuments.net/reader035/viewer/2022070415/5697bf991a28abf838c91e41/html5/thumbnails/2.jpg)
Bronze Age Thera• Modern day Santorini• Aegean civilization: c. 1700 BC• Traded with other
Mediterranean civilizations • Evidence of fishing, agriculture,
and livestock
• Volcanic eruption c. 1650 BC
NASA Visible Earth
![Page 3: A System for High-Volume Acquisition and Matching of Fresco Fragments Reassembling Theran Wall Paintings Benedict Brown 1,2, Corey Toler-Franklin 1, Diego.](https://reader035.fdocuments.net/reader035/viewer/2022070415/5697bf991a28abf838c91e41/html5/thumbnails/3.jpg)
Akrotiri• Major archaeological
excavation since 1967• Well-preserved by ash• Most significant find:
plaster wall paintings– Pigments excellently
preserved
Thera Foundation
![Page 4: A System for High-Volume Acquisition and Matching of Fresco Fragments Reassembling Theran Wall Paintings Benedict Brown 1,2, Corey Toler-Franklin 1, Diego.](https://reader035.fdocuments.net/reader035/viewer/2022070415/5697bf991a28abf838c91e41/html5/thumbnails/4.jpg)
Akrotiri• Major archaeological
excavation since 1967• Well-preserved by ash• Most significant find:
plaster wall paintings– Pigments excellently
preserved
– But shattered in pieces by earthquake
![Page 5: A System for High-Volume Acquisition and Matching of Fresco Fragments Reassembling Theran Wall Paintings Benedict Brown 1,2, Corey Toler-Franklin 1, Diego.](https://reader035.fdocuments.net/reader035/viewer/2022070415/5697bf991a28abf838c91e41/html5/thumbnails/5.jpg)
The Akrotiri Jigsaw• Current assembly
process is laborious
![Page 6: A System for High-Volume Acquisition and Matching of Fresco Fragments Reassembling Theran Wall Paintings Benedict Brown 1,2, Corey Toler-Franklin 1, Diego.](https://reader035.fdocuments.net/reader035/viewer/2022070415/5697bf991a28abf838c91e41/html5/thumbnails/6.jpg)
The Akrotiri Jigsaw• Current assembly
process is laborious• Enough work for
another century
![Page 7: A System for High-Volume Acquisition and Matching of Fresco Fragments Reassembling Theran Wall Paintings Benedict Brown 1,2, Corey Toler-Franklin 1, Diego.](https://reader035.fdocuments.net/reader035/viewer/2022070415/5697bf991a28abf838c91e41/html5/thumbnails/7.jpg)
Fragment Characteristics• Conservators consider:
– size, thickness
– level of erosion
– discoloration and fading
– set of pigments
– curvature / flatness
– texture of the back
– string impressions
![Page 8: A System for High-Volume Acquisition and Matching of Fresco Fragments Reassembling Theran Wall Paintings Benedict Brown 1,2, Corey Toler-Franklin 1, Diego.](https://reader035.fdocuments.net/reader035/viewer/2022070415/5697bf991a28abf838c91e41/html5/thumbnails/8.jpg)
Constrained 3-D Acquisition Protocol• Automatic turntable control• Acquire scans at 45°• Two 360° scan sequences
– Face-down: front face at known plane
– Face-up: front face visible
![Page 9: A System for High-Volume Acquisition and Matching of Fresco Fragments Reassembling Theran Wall Paintings Benedict Brown 1,2, Corey Toler-Franklin 1, Diego.](https://reader035.fdocuments.net/reader035/viewer/2022070415/5697bf991a28abf838c91e41/html5/thumbnails/9.jpg)
Color and Normals: 2-D Acquisition• Custom scan software
– One-click acquisition
– Preview scan locates fragment
• Five scans– Four front orientations
(photometric normals)
– One back orientation
![Page 10: A System for High-Volume Acquisition and Matching of Fresco Fragments Reassembling Theran Wall Paintings Benedict Brown 1,2, Corey Toler-Franklin 1, Diego.](https://reader035.fdocuments.net/reader035/viewer/2022070415/5697bf991a28abf838c91e41/html5/thumbnails/10.jpg)
Scan Alignment with Multi-Way ICP• Align fragments scanned on turntable
– Axis of rotation gives initial guess
– Standard algorithm to improve alignments:Iterative Closest Points [BESL 1992], [CHEN 1992]
• Flat front surfaces lead to instability• Improved algorithm: Multi-way ICP
– Constrain all scan-to-scantransformations to be identical
– Equivalent to solving fora single rotation axis
![Page 11: A System for High-Volume Acquisition and Matching of Fresco Fragments Reassembling Theran Wall Paintings Benedict Brown 1,2, Corey Toler-Franklin 1, Diego.](https://reader035.fdocuments.net/reader035/viewer/2022070415/5697bf991a28abf838c91e41/html5/thumbnails/11.jpg)
Front/Back Alignment• Flipping fragment is
uncalibrated• Little overlap between
front and back scans• Front/back alignment is
vertically unstable
![Page 12: A System for High-Volume Acquisition and Matching of Fresco Fragments Reassembling Theran Wall Paintings Benedict Brown 1,2, Corey Toler-Franklin 1, Diego.](https://reader035.fdocuments.net/reader035/viewer/2022070415/5697bf991a28abf838c91e41/html5/thumbnails/12.jpg)
Front/Back Alignment• Use front face to determine
vertical alignment– Visible in front scans
– On (calibrated) turntablesurface in back scans
• Initial guess and ICP forwithin-plane alignment
![Page 13: A System for High-Volume Acquisition and Matching of Fresco Fragments Reassembling Theran Wall Paintings Benedict Brown 1,2, Corey Toler-Franklin 1, Diego.](https://reader035.fdocuments.net/reader035/viewer/2022070415/5697bf991a28abf838c91e41/html5/thumbnails/13.jpg)
2-D/3-D Alignment• Flatbed scanner has superior color• Can’t use calibration [LEVOY 2000], reliable silhouette
[LENSCH 2000], or features [LIU 2006] [CHEN 2007]
• Use image alignment: PCA + downhill simplex
Projected 3-D Color Flatbed Scan
![Page 14: A System for High-Volume Acquisition and Matching of Fresco Fragments Reassembling Theran Wall Paintings Benedict Brown 1,2, Corey Toler-Franklin 1, Diego.](https://reader035.fdocuments.net/reader035/viewer/2022070415/5697bf991a28abf838c91e41/html5/thumbnails/14.jpg)
Ribbon Matching• Try all possible alignments• Update alignment
incrementally• Regular edge
parameterization:similar to image correlation
![Page 15: A System for High-Volume Acquisition and Matching of Fresco Fragments Reassembling Theran Wall Paintings Benedict Brown 1,2, Corey Toler-Franklin 1, Diego.](https://reader035.fdocuments.net/reader035/viewer/2022070415/5697bf991a28abf838c91e41/html5/thumbnails/15.jpg)
Ribbon Matching• Try all possible alignments• Update alignment
incrementally• Regular edge
parameterization:similar to image correlation
![Page 16: A System for High-Volume Acquisition and Matching of Fresco Fragments Reassembling Theran Wall Paintings Benedict Brown 1,2, Corey Toler-Franklin 1, Diego.](https://reader035.fdocuments.net/reader035/viewer/2022070415/5697bf991a28abf838c91e41/html5/thumbnails/16.jpg)
Ribbon Matching• Try all possible alignments• Update alignment
incrementally• Regular edge
parameterization:similar to image correlation
![Page 17: A System for High-Volume Acquisition and Matching of Fresco Fragments Reassembling Theran Wall Paintings Benedict Brown 1,2, Corey Toler-Franklin 1, Diego.](https://reader035.fdocuments.net/reader035/viewer/2022070415/5697bf991a28abf838c91e41/html5/thumbnails/17.jpg)
Ribbon Matching• Try all possible alignments• Update alignment
incrementally• Regular edge
parameterization:similar to image correlation
![Page 18: A System for High-Volume Acquisition and Matching of Fresco Fragments Reassembling Theran Wall Paintings Benedict Brown 1,2, Corey Toler-Franklin 1, Diego.](https://reader035.fdocuments.net/reader035/viewer/2022070415/5697bf991a28abf838c91e41/html5/thumbnails/18.jpg)
Ribbon Matching• Try all possible alignments• Update alignment
incrementally• Regular edge
parameterization:similar to image correlation
![Page 19: A System for High-Volume Acquisition and Matching of Fresco Fragments Reassembling Theran Wall Paintings Benedict Brown 1,2, Corey Toler-Franklin 1, Diego.](https://reader035.fdocuments.net/reader035/viewer/2022070415/5697bf991a28abf838c91e41/html5/thumbnails/19.jpg)
Ribbon Matching• Try all possible alignments• Update alignment
incrementally• Regular edge
parameterization:similar to image correlation
![Page 20: A System for High-Volume Acquisition and Matching of Fresco Fragments Reassembling Theran Wall Paintings Benedict Brown 1,2, Corey Toler-Franklin 1, Diego.](https://reader035.fdocuments.net/reader035/viewer/2022070415/5697bf991a28abf838c91e41/html5/thumbnails/20.jpg)
Ribbon Matching• Try all possible alignments• Update alignment
incrementally• Regular edge
parameterization:similar to image correlation
![Page 21: A System for High-Volume Acquisition and Matching of Fresco Fragments Reassembling Theran Wall Paintings Benedict Brown 1,2, Corey Toler-Franklin 1, Diego.](https://reader035.fdocuments.net/reader035/viewer/2022070415/5697bf991a28abf838c91e41/html5/thumbnails/21.jpg)
Ribbon Matching• Try all possible alignments• Update alignment
incrementally• Regular edge
parameterization:similar to image correlation
![Page 22: A System for High-Volume Acquisition and Matching of Fresco Fragments Reassembling Theran Wall Paintings Benedict Brown 1,2, Corey Toler-Franklin 1, Diego.](https://reader035.fdocuments.net/reader035/viewer/2022070415/5697bf991a28abf838c91e41/html5/thumbnails/22.jpg)
Fragment MatchingICP Matching
– Nearest neighbor correspondence search
– Iterate to find matches
– 45 seconds per fragment pair
Ribbon Matching
– Regular edge sampling for correspondences
– Exhaustive search with incremental update
– 2 seconds per pair
Original (irregular) mesh Resampled ribbon
![Page 23: A System for High-Volume Acquisition and Matching of Fresco Fragments Reassembling Theran Wall Paintings Benedict Brown 1,2, Corey Toler-Franklin 1, Diego.](https://reader035.fdocuments.net/reader035/viewer/2022070415/5697bf991a28abf838c91e41/html5/thumbnails/23.jpg)
Erosion Detection• Erosion causes incorrect alignments• Detected on ribbons with normal constraint
Fragment Back
Fragment Front
No Erosion Detection
![Page 24: A System for High-Volume Acquisition and Matching of Fresco Fragments Reassembling Theran Wall Paintings Benedict Brown 1,2, Corey Toler-Franklin 1, Diego.](https://reader035.fdocuments.net/reader035/viewer/2022070415/5697bf991a28abf838c91e41/html5/thumbnails/24.jpg)
Erosion Detection• Erosion causes incorrect alignments• Detected on ribbons with normal constraint
Fragment Back
Fragment Front
No Erosion Detection With Erosion Detection
![Page 25: A System for High-Volume Acquisition and Matching of Fresco Fragments Reassembling Theran Wall Paintings Benedict Brown 1,2, Corey Toler-Franklin 1, Diego.](https://reader035.fdocuments.net/reader035/viewer/2022070415/5697bf991a28abf838c91e41/html5/thumbnails/25.jpg)
Outline• System design• Processing pipeline• Matching• Results
![Page 26: A System for High-Volume Acquisition and Matching of Fresco Fragments Reassembling Theran Wall Paintings Benedict Brown 1,2, Corey Toler-Franklin 1, Diego.](https://reader035.fdocuments.net/reader035/viewer/2022070415/5697bf991a28abf838c91e41/html5/thumbnails/26.jpg)
Ribbon Matching Results
![Page 27: A System for High-Volume Acquisition and Matching of Fresco Fragments Reassembling Theran Wall Paintings Benedict Brown 1,2, Corey Toler-Franklin 1, Diego.](https://reader035.fdocuments.net/reader035/viewer/2022070415/5697bf991a28abf838c91e41/html5/thumbnails/27.jpg)
Synthetic Fresco25 mm strip width12.5 mm strip width50 mm strip width
![Page 28: A System for High-Volume Acquisition and Matching of Fresco Fragments Reassembling Theran Wall Paintings Benedict Brown 1,2, Corey Toler-Franklin 1, Diego.](https://reader035.fdocuments.net/reader035/viewer/2022070415/5697bf991a28abf838c91e41/html5/thumbnails/28.jpg)
Future Work (Matching)• Multi-cue matching• Improved ribbon matching/
Handling gaps– Dynamic programming can probably
handle gaps
– Record all possible alignments instead of only best candidates to do saliency analysis
• Global matching– Fuse matched fragments and re-match
– Do global consistency checks on networks of matches
![Page 29: A System for High-Volume Acquisition and Matching of Fresco Fragments Reassembling Theran Wall Paintings Benedict Brown 1,2, Corey Toler-Franklin 1, Diego.](https://reader035.fdocuments.net/reader035/viewer/2022070415/5697bf991a28abf838c91e41/html5/thumbnails/29.jpg)
Future Work (Scanners)We want to scan:• large fragments• assembled edges?• edge and back normals
Approach:• Hand-held scanner• Two cameras and a projector/fixed pattern• Alignment similar to in-hand scanner• Should be able to get normals from mutiple views
![Page 30: A System for High-Volume Acquisition and Matching of Fresco Fragments Reassembling Theran Wall Paintings Benedict Brown 1,2, Corey Toler-Franklin 1, Diego.](https://reader035.fdocuments.net/reader035/viewer/2022070415/5697bf991a28abf838c91e41/html5/thumbnails/30.jpg)
Future Work (Scanners)We want to scan:• large fragments• assembled edges?• edge and back normals
Approach:• Hand-held scanner• Two cameras and a projector/fixed pattern• Alignment similar to in-hand scanner• Should be able to get normals from mutiple views
![Page 31: A System for High-Volume Acquisition and Matching of Fresco Fragments Reassembling Theran Wall Paintings Benedict Brown 1,2, Corey Toler-Franklin 1, Diego.](https://reader035.fdocuments.net/reader035/viewer/2022070415/5697bf991a28abf838c91e41/html5/thumbnails/31.jpg)
Acknowledgments• Princeton University: Tom Funkhouser, Dimitris Gondicas,
Matt Plough, Phil Shilane, Xiaojuan Ma• Akrotiri Excavation, Laboratory of Wall Paintings:
Manolis Hamaoui, Litsa Kalambouki, Marina Papapetrou, Panagiotis Vlachos, Alexandros Zokos, Iakovos Michailidis, Fragoula Georma, Niki Spanou
• Special thanks to David Koller (University of Viriginia),Misha Kazhdan (Johns Hopkins University), and Peter Nomikos Jr.
• Funding: Thera Foundation, Kress Foundation,Seeger Foundation, Cotsen Family Foundation, andNSF Grants CCF-0347427 and CCF-0702580