Shadow Mapping and Shadow Volumes: Recent Developments in Real
A Big Data Approach for Comprehensive Urban Shadow ... · Anh Vu Vo and Debra Laefer | NYU CUSP...
Transcript of A Big Data Approach for Comprehensive Urban Shadow ... · Anh Vu Vo and Debra Laefer | NYU CUSP...
![Page 1: A Big Data Approach for Comprehensive Urban Shadow ... · Anh Vu Vo and Debra Laefer | NYU CUSP Comprehensive Urban Shadow Analysis Background Urban shadow analysis Sunlight is critical](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc922e8891d7d410b5ae5a4/html5/thumbnails/1.jpg)
A Big Data Approachfor Comprehensive Urban Shadow Analysisfrom Airborne Laser Scanning Point Clouds
Anh Vu Vo and Debra F. Laefer
Center for Urban Science and Progress | New York University
![Page 2: A Big Data Approach for Comprehensive Urban Shadow ... · Anh Vu Vo and Debra Laefer | NYU CUSP Comprehensive Urban Shadow Analysis Background Urban shadow analysis Sunlight is critical](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc922e8891d7d410b5ae5a4/html5/thumbnails/2.jpg)
Anh Vu Vo and Debra Laefer | NYU CUSP Comprehensive Urban Shadow Analysis
Background
![Page 3: A Big Data Approach for Comprehensive Urban Shadow ... · Anh Vu Vo and Debra Laefer | NYU CUSP Comprehensive Urban Shadow Analysis Background Urban shadow analysis Sunlight is critical](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc922e8891d7d410b5ae5a4/html5/thumbnails/3.jpg)
Anh Vu Vo and Debra Laefer | NYU CUSP Comprehensive Urban Shadow Analysis
Background
Urban shadow analysis
Sunlight is critical to all aspects of human life
Ensuring access to sunlight can be a challenge in urban environments
Shadow analysis is often a part of the urban planning process
Shadow analysis is mathematically simple but can be computationally intensive
1
![Page 4: A Big Data Approach for Comprehensive Urban Shadow ... · Anh Vu Vo and Debra Laefer | NYU CUSP Comprehensive Urban Shadow Analysis Background Urban shadow analysis Sunlight is critical](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc922e8891d7d410b5ae5a4/html5/thumbnails/4.jpg)
Anh Vu Vo and Debra Laefer | NYU CUSP Comprehensive Urban Shadow Analysis
Background
High resolution airborne laser scanning point cloud
2
1.4 billion data points
Density: 300 points/m2 (hor.) 35 points/m2 (ver.)
DOI:10.17609/N8MQ0N (open dataset)
![Page 5: A Big Data Approach for Comprehensive Urban Shadow ... · Anh Vu Vo and Debra Laefer | NYU CUSP Comprehensive Urban Shadow Analysis Background Urban shadow analysis Sunlight is critical](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc922e8891d7d410b5ae5a4/html5/thumbnails/5.jpg)
Anh Vu Vo and Debra Laefer | NYU CUSP Comprehensive Urban Shadow Analysis
Background
3
Distributed hardware and Hadoop software
Scale beyond the capacity of one single machine
Distributed memory cluster
Hadoop software stack: YARN (resource manager and scheduling) Hadoop Distributed File System (storage) Apache Spark (data analytics) Apache HBase (data management)
![Page 6: A Big Data Approach for Comprehensive Urban Shadow ... · Anh Vu Vo and Debra Laefer | NYU CUSP Comprehensive Urban Shadow Analysis Background Urban shadow analysis Sunlight is critical](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc922e8891d7d410b5ae5a4/html5/thumbnails/6.jpg)
Anh Vu Vo and Debra Laefer | NYU CUSP Comprehensive Urban Shadow Analysis
A Big Data Approachfor Comprehensive Urban Shadow Analysisfrom Airborne Laser Scanning Point Clouds
![Page 7: A Big Data Approach for Comprehensive Urban Shadow ... · Anh Vu Vo and Debra Laefer | NYU CUSP Comprehensive Urban Shadow Analysis Background Urban shadow analysis Sunlight is critical](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc922e8891d7d410b5ae5a4/html5/thumbnails/7.jpg)
Anh Vu Vo and Debra Laefer | NYU CUSP Comprehensive Urban Shadow Analysis
Comprehensive Urban Shadow (CUS) Analysis with Distributed Computing
4
Distributed CUS algorithm
A point p in a point cloud can only cast shadow on and receive shadows from the other points along the same sun ray passing p
To group points into point beams:
Transform the input point cloud to UQP, which has axis P parallel to the Sun direction
Rasterize the point data by a regular 2D grid on the UQ plane
Points in each raster cell resembles a point beam
Fig 1 – Distributed shadow computation algorithm
![Page 8: A Big Data Approach for Comprehensive Urban Shadow ... · Anh Vu Vo and Debra Laefer | NYU CUSP Comprehensive Urban Shadow Analysis Background Urban shadow analysis Sunlight is critical](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc922e8891d7d410b5ae5a4/html5/thumbnails/8.jpg)
Anh Vu Vo and Debra Laefer | NYU CUSP Comprehensive Urban Shadow Analysis
Comprehensive Urban Shadow (CUS) Analysis with Distributed Computing
4
Distributed CUS algorithm
To group points into point beams: Transform the input point cloud to
UQP, which has axis P parallel to the Sun direction
Rasterize the point data by a regular 2D grid on the UQ plane
Points in each raster cell resembles a point beam
Fig 1 – Distributed shadow computation algorithm
![Page 9: A Big Data Approach for Comprehensive Urban Shadow ... · Anh Vu Vo and Debra Laefer | NYU CUSP Comprehensive Urban Shadow Analysis Background Urban shadow analysis Sunlight is critical](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc922e8891d7d410b5ae5a4/html5/thumbnails/9.jpg)
Anh Vu Vo and Debra Laefer | NYU CUSP Comprehensive Urban Shadow Analysis
Comprehensive Urban Shadow (CUS) Analysis with Distributed Computing
4
Distributed CUS algorithm
Each point beam can be processed independently to identity shaded/illuminated points: Density based clustering Points in the cluster with highest p-
values are considered as illuminated. The remaining points in the point
beam are considered as shaded
Fig 1 – Distributed shadow computation algorithm
![Page 10: A Big Data Approach for Comprehensive Urban Shadow ... · Anh Vu Vo and Debra Laefer | NYU CUSP Comprehensive Urban Shadow Analysis Background Urban shadow analysis Sunlight is critical](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc922e8891d7d410b5ae5a4/html5/thumbnails/10.jpg)
Anh Vu Vo and Debra Laefer | NYU CUSP Comprehensive Urban Shadow Analysis
Comprehensive Urban Shadow Analysis with Distributed Computing
5
Fig 2 – Acyclic data flow diagram for the Spark implementation
Spark implementation Spark’s core concepts
Spark: general-purpose, distributed computing framework for distributed data processing
RDD: immutable, resilient, distributed dataset
Except for T2, all other transformations are performed independently on each RDD
![Page 11: A Big Data Approach for Comprehensive Urban Shadow ... · Anh Vu Vo and Debra Laefer | NYU CUSP Comprehensive Urban Shadow Analysis Background Urban shadow analysis Sunlight is critical](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc922e8891d7d410b5ae5a4/html5/thumbnails/11.jpg)
Anh Vu Vo and Debra Laefer | NYU CUSP Comprehensive Urban Shadow Analysis
Results
![Page 12: A Big Data Approach for Comprehensive Urban Shadow ... · Anh Vu Vo and Debra Laefer | NYU CUSP Comprehensive Urban Shadow Analysis Background Urban shadow analysis Sunlight is critical](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc922e8891d7d410b5ae5a4/html5/thumbnails/12.jpg)
Anh Vu Vo and Debra Laefer | NYU CUSP Comprehensive Urban Shadow Analysis
Results
6
Instantaneous shadow map
• Computation time for 1.4 billion points: 1.5 – 3.0 minutes per timestep• 50 nodes, 5 GB of memory per node
(a) Computed 3D shadow map (b) Ground truth
Fig 3 – Shadows of the North-West corner of St. Stephen’s Green computed for 15:02:19 26/03/2015
![Page 13: A Big Data Approach for Comprehensive Urban Shadow ... · Anh Vu Vo and Debra Laefer | NYU CUSP Comprehensive Urban Shadow Analysis Background Urban shadow analysis Sunlight is critical](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc922e8891d7d410b5ae5a4/html5/thumbnails/13.jpg)
Anh Vu Vo and Debra Laefer | NYU CUSP Comprehensive Urban Shadow Analysis
Results
(a) Computed 3D shadow map (b) Ground truth
Instantaneous shadow map
Fig 4 – Shadows of the Liberty Hall and the surrounding computed for 12:50:47 26/03/2015
(a) Computed 3D shadow map
7
![Page 14: A Big Data Approach for Comprehensive Urban Shadow ... · Anh Vu Vo and Debra Laefer | NYU CUSP Comprehensive Urban Shadow Analysis Background Urban shadow analysis Sunlight is critical](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc922e8891d7d410b5ae5a4/html5/thumbnails/14.jpg)
Anh Vu Vo and Debra Laefer | NYU CUSP Comprehensive Urban Shadow Analysis
Results
8
Shadow accrual maps
Fig 4 – Summer Solstice
Fig 5 – Winter Solstice
![Page 15: A Big Data Approach for Comprehensive Urban Shadow ... · Anh Vu Vo and Debra Laefer | NYU CUSP Comprehensive Urban Shadow Analysis Background Urban shadow analysis Sunlight is critical](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc922e8891d7d410b5ae5a4/html5/thumbnails/15.jpg)
Anh Vu Vo and Debra Laefer | NYU CUSP Comprehensive Urban Shadow Analysis
Conclusions
![Page 16: A Big Data Approach for Comprehensive Urban Shadow ... · Anh Vu Vo and Debra Laefer | NYU CUSP Comprehensive Urban Shadow Analysis Background Urban shadow analysis Sunlight is critical](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc922e8891d7d410b5ae5a4/html5/thumbnails/16.jpg)
Anh Vu Vo and Debra Laefer | NYU CUSP Comprehensive Urban Shadow Analysis
Conclusions
• Distributed approach for computing detailed urban shadows in 3D directly from high-resolution airborne laser scanning point clouds
• Highly scalable algorithm suitable for implementation in a distributed memory computing cluster
• Computing an instantaneous shadow map for a 1.4 billion point dataset took 1.5 – 3.0 minutes with 50 cores and 5 GB RAM per core. More cores can be added, if a shorter processing time is desired or more data needs to be processed
• Multiple output formats can be derived, including instantaneous 3D shadow models, shadow accrual maps, shadow timelapse videos
11
![Page 17: A Big Data Approach for Comprehensive Urban Shadow ... · Anh Vu Vo and Debra Laefer | NYU CUSP Comprehensive Urban Shadow Analysis Background Urban shadow analysis Sunlight is critical](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc922e8891d7d410b5ae5a4/html5/thumbnails/17.jpg)
Anh Vu Vo and Debra Laefer | NYU CUSP Comprehensive Urban Shadow Analysis
Authors
Anh Vu Vo
Acknowledgement
Debra Laefer