Point Cloud Segmentation for 3D Reconstruction
-
Upload
pirouz-nourian -
Category
Engineering
-
view
1.171 -
download
0
Transcript of Point Cloud Segmentation for 3D Reconstruction
![Page 1: Point Cloud Segmentation for 3D Reconstruction](https://reader033.fdocuments.net/reader033/viewer/2022050614/588462941a28abbd308b517b/html5/thumbnails/1.jpg)
1 Challenge the future
Point Cloud Segmentation &
Surface Reconstruction
An overview of methods
Ir. Pirouz Nourian PhD candidate & Instructor, chair of Design Informatics, since 2010 MSc in Architecture 2009
BSc in Control Engineering 2005
MSc Geomatics, GEO1004, Directed by Dr. Sisi Zlatanova
![Page 2: Point Cloud Segmentation for 3D Reconstruction](https://reader033.fdocuments.net/reader033/viewer/2022050614/588462941a28abbd308b517b/html5/thumbnails/2.jpg)
2 Challenge the future
Topics
A very short introduction to segmentation and surface reconstruction
• Big Picture, Problem Statement • Goal is to make a [simple] Brep, with few faces
• General Approaches to this problem (surface
reconstruction):
• Volumetric, Implicit, using voxels and iso-surfaces • Alpha Shapes (Ball-Pivoting)
• Delaunay or Voronoi Based? • Poisson? • General Approaches to this problem (surface
reconstruction):
![Page 3: Point Cloud Segmentation for 3D Reconstruction](https://reader033.fdocuments.net/reader033/viewer/2022050614/588462941a28abbd308b517b/html5/thumbnails/3.jpg)
3 Challenge the future
Topics
A very short introduction to segmentation and surface reconstruction
• Big Picture, Problem Statement • Goal is to make a [simple] Brep, with few faces
Images courtesy of Ajith Sampath
?
![Page 4: Point Cloud Segmentation for 3D Reconstruction](https://reader033.fdocuments.net/reader033/viewer/2022050614/588462941a28abbd308b517b/html5/thumbnails/4.jpg)
4 Challenge the future
A few approaches to surface reconstruction
• Voxels Field of Signed-Distance Iso-Surface
• Ball-Pivoting (alpha shapes) triangulation
• Planar Segments Neighbourhood Matrix Edge List Simple Mesh
• Poisson Surface Reconstruction
• Implicit Function
• [ Hoppe et al. 92 ]
• Volumetric Reconstruction
• [ Curless and Levoy 96 ]
• Alpha Shapes
• [ Edelsbrunner and Mucke 94 ]
• 3D Voronoi-Based Reconstruction
• [ Amenta , Bern & Kamvysselis 98 ]
![Page 5: Point Cloud Segmentation for 3D Reconstruction](https://reader033.fdocuments.net/reader033/viewer/2022050614/588462941a28abbd308b517b/html5/thumbnails/5.jpg)
5 Challenge the future
Segmentation for Detecting Features
Points belonging to a set of surfaces considered as Brep faces
Images courtesy of Dr. Shi Pu, Faculty of Feo Information Science and Earth Observations, University of Twente (ITC)
![Page 6: Point Cloud Segmentation for 3D Reconstruction](https://reader033.fdocuments.net/reader033/viewer/2022050614/588462941a28abbd308b517b/html5/thumbnails/6.jpg)
6 Challenge the future
Segmentation for Detecting Features
Points belonging to a set of surfaces considered as Brep faces
3D reconstruction from AHN pointcloud: Carl Chen, Rusne Sileryte, Kaixuan Zhou; Ed. Pirouz Nourian, Sisi Zlatanova
![Page 7: Point Cloud Segmentation for 3D Reconstruction](https://reader033.fdocuments.net/reader033/viewer/2022050614/588462941a28abbd308b517b/html5/thumbnails/7.jpg)
7 Challenge the future
Segmentation for Detecting Features
Points belonging to a set of surfaces considered as Brep faces
Images courtesy of Ajith Sampath
![Page 8: Point Cloud Segmentation for 3D Reconstruction](https://reader033.fdocuments.net/reader033/viewer/2022050614/588462941a28abbd308b517b/html5/thumbnails/8.jpg)
8 Challenge the future
Segmentation for Detecting Features
Points belonging to a set of surfaces considered as Brep faces
Images courtesy of Ajith Sampath
Building Reconstruction The distance between two planar segments (P & Q) in a roof is defined as:
A neighborhood Matrix is then generated. The matrix shows all mutually intersecting
planes. Any two intersecting planes are selected, and all planes that intersect both of them are
enumerated, and solved. For instance Planes {1,4,10} and {1,4,13} are solved to get the breakline (A-B)as shown.
![Page 9: Point Cloud Segmentation for 3D Reconstruction](https://reader033.fdocuments.net/reader033/viewer/2022050614/588462941a28abbd308b517b/html5/thumbnails/9.jpg)
9 Challenge the future
Problem
• Estimated Normal Vectors For Each Point
• Estimated Curvature For Each Point
Preliminaries…
• Estimate Normal Vectors For Each Point Using the Covariance Matrix
• (Fit a Plane to a Bunch of Points)
• Cross Product of the Two Eigen Vectors Corresponding to the Two Largest Eigen Values
![Page 10: Point Cloud Segmentation for 3D Reconstruction](https://reader033.fdocuments.net/reader033/viewer/2022050614/588462941a28abbd308b517b/html5/thumbnails/10.jpg)
10 Challenge the future
Covariance Matrix Computation Check the attached code for the latest version!
Dim Neighbors = Pts.FindAll(Function(V) V.distanceto(point) < D) Dim Centroid As New Point3d For Each neighbor As point3d In Neighbors
Centroid = Centroid + neighbor Next For k As Integer=0 To Neighbors.count - 1 Dim CiCBar As New Matrix(3, Neighbors.count) Dim Diff As point3d = Neighbors(k) - Centroid
CiCBar(0, k) = Diff.X CiCBar(1, k) = Diff.y CiCBar(2, k) = Diff.z Dim CiCBarOld As New Matrix(3, Neighbors.count) CiCBarOld = CiCBar.Duplicate()
CiCBar.Transpose() CovM = CiCBarOld * CiCBar ‘Why is this a matrix of correct size? Next CovM.Scale(1 / Neighbors.count) CovMatrices.Add(CovM)
![Page 11: Point Cloud Segmentation for 3D Reconstruction](https://reader033.fdocuments.net/reader033/viewer/2022050614/588462941a28abbd308b517b/html5/thumbnails/11.jpg)
11 Challenge the future
Efficient Eigen Value Computation
• Find the roots of this characteristic function (why? & how?)
• Using a Trigonometric Method
• (The following code is my version of it, test it first on a cubic function)
A special case in which a 3x3 Matrix is dealt with
Function CubicRoots(a, b, c, d) Dim t As Double
Dim p,q As Double p = (3 * a * c - b ̂ 2) / (3 * a ^ 2)
If p < 0 Then q = (2 * b ^ 3 - 9 * a * b * c + 27 * a ^ 2 * d) / (27 * a ^ 3) Dim roots As New list(Of Double)
For k As Integer = 0 To 2 t = 2 * Math.Sqrt(-p / 3) * Math.Cos((1 / 3) * Math.Acos(((3 * q) / (2 * p)) * Math.Sqrt(-3 / p)) - k * ((2 * Math.PI) / 3))
Dim x As Double = t - b / (3 * a) roots.add(x) Next
Return roots Else
Return Nothing End If End Function
![Page 12: Point Cloud Segmentation for 3D Reconstruction](https://reader033.fdocuments.net/reader033/viewer/2022050614/588462941a28abbd308b517b/html5/thumbnails/12.jpg)
12 Challenge the future
Ready for Segmentation! We do an iterative segmentation called Region Growing
Mesh Segmentation in Action
low angle threshold
high angle threshold
![Page 13: Point Cloud Segmentation for 3D Reconstruction](https://reader033.fdocuments.net/reader033/viewer/2022050614/588462941a28abbd308b517b/html5/thumbnails/13.jpg)
13 Challenge the future
Mesh Segmentation
• Connected Faces
• Start Making Regions out of Neighbours of Similar Aspects
(normal vectors/orientations)
• Recursively Grow Regions with an Angle Tolerance (how?)
How to detect faces of a Brep given a Mesh?
Define Faces=M.Faces Define Regions As New list(Of List(Of Integer)) For i in range M.Faces.Count
If clusters.count = 0 Or There is no region containing(i) Then Define region As New list(Of Integer) region.Append(i) For j in range adjacentfaces(of i) If isSimilar(M.FaceNormals(i), M.FaceNormals(j), t) Then region.Append(j)
End For region = RecursivelyGrowRegion(M, region, angle threshold) Regions.Append(region) End If End For
![Page 14: Point Cloud Segmentation for 3D Reconstruction](https://reader033.fdocuments.net/reader033/viewer/2022050614/588462941a28abbd308b517b/html5/thumbnails/14.jpg)
14 Challenge the future
Point Cloud Segmentation Region Growing Segmentation Pseudocode
![Page 15: Point Cloud Segmentation for 3D Reconstruction](https://reader033.fdocuments.net/reader033/viewer/2022050614/588462941a28abbd308b517b/html5/thumbnails/15.jpg)
15 Challenge the future
A few approaches to surface reconstruction
• 3d Hough Transform [Vosselman et al.]
• Implicit Function [ Hoppe et al. 92 ]
• Alpha Shapes [ Edelsbrunner and Mucke 94 ]
• Many more:
Berger, Matthew, et al. "State of the art in surface reconstruction from point clouds." EUROGRAPHICS star reports. Vol. 1. No. 1. 2014.
(e.g. Ball-Pivoting algorithm) The space generated by point pairs that can be touched by an empty disc of radius alpha.
Using Voxels Field of Signed-Distance Iso-Surface
![Page 16: Point Cloud Segmentation for 3D Reconstruction](https://reader033.fdocuments.net/reader033/viewer/2022050614/588462941a28abbd308b517b/html5/thumbnails/16.jpg)
16 Challenge the future
2D Hough Transform
![Page 17: Point Cloud Segmentation for 3D Reconstruction](https://reader033.fdocuments.net/reader033/viewer/2022050614/588462941a28abbd308b517b/html5/thumbnails/17.jpg)
17 Challenge the future
2d Hough
Transform
b = -ax + y
![Page 18: Point Cloud Segmentation for 3D Reconstruction](https://reader033.fdocuments.net/reader033/viewer/2022050614/588462941a28abbd308b517b/html5/thumbnails/18.jpg)
18 Challenge the future
2d Hough
Transform
R=x cos θ+ y sin θ
![Page 19: Point Cloud Segmentation for 3D Reconstruction](https://reader033.fdocuments.net/reader033/viewer/2022050614/588462941a28abbd308b517b/html5/thumbnails/19.jpg)
19 Challenge the future
3d Hough transform
c = -ax -by + z
R = x cos θ cos Φ + y sin θ cos Φ+ z sin Φ
![Page 20: Point Cloud Segmentation for 3D Reconstruction](https://reader033.fdocuments.net/reader033/viewer/2022050614/588462941a28abbd308b517b/html5/thumbnails/20.jpg)
20 Challenge the future
20
Restricted 3d Hough Transform
Plane should contain (0,0,0) → z = ax + by
b = -ax/y + z/y (y != 0)
![Page 21: Point Cloud Segmentation for 3D Reconstruction](https://reader033.fdocuments.net/reader033/viewer/2022050614/588462941a28abbd308b517b/html5/thumbnails/21.jpg)
21 Challenge the future
21
![Page 22: Point Cloud Segmentation for 3D Reconstruction](https://reader033.fdocuments.net/reader033/viewer/2022050614/588462941a28abbd308b517b/html5/thumbnails/22.jpg)
22 Challenge the future
22
![Page 23: Point Cloud Segmentation for 3D Reconstruction](https://reader033.fdocuments.net/reader033/viewer/2022050614/588462941a28abbd308b517b/html5/thumbnails/23.jpg)
23 Challenge the future
23
![Page 24: Point Cloud Segmentation for 3D Reconstruction](https://reader033.fdocuments.net/reader033/viewer/2022050614/588462941a28abbd308b517b/html5/thumbnails/24.jpg)
24 Challenge the future
24
Hough Filtering
![Page 25: Point Cloud Segmentation for 3D Reconstruction](https://reader033.fdocuments.net/reader033/viewer/2022050614/588462941a28abbd308b517b/html5/thumbnails/25.jpg)
25 Challenge the future
25
Final result