Object extraction from satellite imagery using deep learning
Transcript of Object extraction from satellite imagery using deep learning
![Page 1: Object extraction from satellite imagery using deep learning](https://reader034.fdocuments.net/reader034/viewer/2022050613/5a64b7267f8b9ac21c8b4c7f/html5/thumbnails/1.jpg)
Object Extraction from Satellite Imagery using Deep Learning
Aly Osama
![Page 2: Object extraction from satellite imagery using deep learning](https://reader034.fdocuments.net/reader034/viewer/2022050613/5a64b7267f8b9ac21c8b4c7f/html5/thumbnails/2.jpg)
Presenter
• Aly Osama
• Research Software Development Engineer at Microsoft
• Contact:
• Email: [email protected]
• https://www.linkedin.com/in/alyosama/
• https://github.com/alyosama
![Page 3: Object extraction from satellite imagery using deep learning](https://reader034.fdocuments.net/reader034/viewer/2022050613/5a64b7267f8b9ac21c8b4c7f/html5/thumbnails/3.jpg)
Agenda
1. Needed data, its size,
2. How training will go,
3. How evaluation should be carried,
4. What learning tools would you see and why?
5. Literature survey on the subject
6. Code sample/preliminary results
![Page 4: Object extraction from satellite imagery using deep learning](https://reader034.fdocuments.net/reader034/viewer/2022050613/5a64b7267f8b9ac21c8b4c7f/html5/thumbnails/4.jpg)
Overview
Key Words:-
1. Satellite Imagery 2. Object Extraction3. Deep Learning4. Python
![Page 5: Object extraction from satellite imagery using deep learning](https://reader034.fdocuments.net/reader034/viewer/2022050613/5a64b7267f8b9ac21c8b4c7f/html5/thumbnails/5.jpg)
1. Satellite Imagery
• Objects are often very small (~20 pixels in size ) as example 0.5m/pixel
• Input images are enormous (often hundreds of megapixels).
• Image has more than 3 channels (RGB) called bands.
• Image Format:• Images (GeoTiff , … etc )• Labels ( GeoJSON , WTK )
• On the positive side, • The physical and pixel scale of objects
are usually known in advance• There’s a low variation in observation
angle.
![Page 6: Object extraction from satellite imagery using deep learning](https://reader034.fdocuments.net/reader034/viewer/2022050613/5a64b7267f8b9ac21c8b4c7f/html5/thumbnails/6.jpg)
Datasets • SpaceNet
• Source : https://aws.amazon.com/public-datasets/spacenet/
• Tutorial https://medium.com/the-downlinq/building-extraction-with-yolt2-and-spacenet-data-a926f9ffac4f
• 5 Solutions: https://github.com/SpaceNetChallenge/BuildingDetectors
• DSTL Satellite Imagery (Kaggle Competition )• https://www.kaggle.com/c/dstl-satellite-imagery-
feature-detection• Tutorial : https://www.kaggle.com/torrinos/dstl-
satellite-imagery-feature-detection/exploration-and-plotting/run/553107
• DeepSat
• UC Merced land
• Satellite Websites• https://modis.gsfc.nasa.gov/• https://sentinel.esa.int/web/sentinel/home• https://landsat.usgs.gov/
![Page 7: Object extraction from satellite imagery using deep learning](https://reader034.fdocuments.net/reader034/viewer/2022050613/5a64b7267f8b9ac21c8b4c7f/html5/thumbnails/7.jpg)
2. Training
• Experiments
1. VGGNet. - (Baseline)• Tune the pretrained model – Transfer
Learning – based on available data
• Data Augmentation like • Random Crops / Scales
• Color Jitter
2. Faster RCNN or YOLO• For Detection and Localization
![Page 8: Object extraction from satellite imagery using deep learning](https://reader034.fdocuments.net/reader034/viewer/2022050613/5a64b7267f8b9ac21c8b4c7f/html5/thumbnails/8.jpg)
General Tips
![Page 9: Object extraction from satellite imagery using deep learning](https://reader034.fdocuments.net/reader034/viewer/2022050613/5a64b7267f8b9ac21c8b4c7f/html5/thumbnails/9.jpg)
3.Evaluation
• Accuracy :D
• Precession and Recall
• Jaccard index ( Intersection over Union)
![Page 10: Object extraction from satellite imagery using deep learning](https://reader034.fdocuments.net/reader034/viewer/2022050613/5a64b7267f8b9ac21c8b4c7f/html5/thumbnails/10.jpg)
4. tools
• Hardware • Deep Learning AMI Amazon Linux Version
• https://aws.amazon.com/marketplace/pp/B01M0AXXQB• Powerful Machine with GPUs
• NVIDIA GTX Titan X
• Software• Caffe + Python
• Pretrained models • Lasagne / Keras Framework
• High level Python• Backend independent “Tensorflow or Theano” • Multi-GPU
• Caffe2 ( 18-4-2017 )
• Utility• QGIS is an open-source tool for managing and editing GeoTIFFs and
geoJSON files
![Page 11: Object extraction from satellite imagery using deep learning](https://reader034.fdocuments.net/reader034/viewer/2022050613/5a64b7267f8b9ac21c8b4c7f/html5/thumbnails/11.jpg)
Tools Comparison
![Page 12: Object extraction from satellite imagery using deep learning](https://reader034.fdocuments.net/reader034/viewer/2022050613/5a64b7267f8b9ac21c8b4c7f/html5/thumbnails/12.jpg)
5. Literature survey on the subject
![Page 13: Object extraction from satellite imagery using deep learning](https://reader034.fdocuments.net/reader034/viewer/2022050613/5a64b7267f8b9ac21c8b4c7f/html5/thumbnails/13.jpg)
1- A Survey on Object Detection in Optical Remote Sensing Images 2016Gong Cheng, Junwei Han*
![Page 14: Object extraction from satellite imagery using deep learning](https://reader034.fdocuments.net/reader034/viewer/2022050613/5a64b7267f8b9ac21c8b4c7f/html5/thumbnails/14.jpg)
Cheng et al. 2016
![Page 15: Object extraction from satellite imagery using deep learning](https://reader034.fdocuments.net/reader034/viewer/2022050613/5a64b7267f8b9ac21c8b4c7f/html5/thumbnails/15.jpg)
Cheng et al. 2016
• Deep Learning Papers • (Han et al., 2015;Tang et al.,
2015; Wang et al., 2015; Zhou et al., 2015b)
• Datasets• NWPU VHR-10 dataset (Cheng et
al., 2014a)1. • SZTAKI-INRIA building detection
dataset (Benedek et al., 2012)2. • TAS aerial car detection dataset
(Heitz and Koller, 2008)3.• Overhead imagery research
dataset (OIRDS) (Tanner et al., 2009)
• IITM road extraction dataset (Das et al., 2011)5.
![Page 16: Object extraction from satellite imagery using deep learning](https://reader034.fdocuments.net/reader034/viewer/2022050613/5a64b7267f8b9ac21c8b4c7f/html5/thumbnails/16.jpg)
FAST AIRCRAFT DETECTION IN SATELLITE IMAGES BASED ON CONVOLUTIONAL NEURAL NETWORKS (Wu et. Al.) 2015
![Page 17: Object extraction from satellite imagery using deep learning](https://reader034.fdocuments.net/reader034/viewer/2022050613/5a64b7267f8b9ac21c8b4c7f/html5/thumbnails/17.jpg)
Region Proposals
TODO : Search on R-CNN
![Page 18: Object extraction from satellite imagery using deep learning](https://reader034.fdocuments.net/reader034/viewer/2022050613/5a64b7267f8b9ac21c8b4c7f/html5/thumbnails/18.jpg)
FULLY CONVOLUTIONAL NETWORKS FOR BUILDING AND ROAD EXTRACTION: PRELIMINARY RESULTS 2016 Zilong Zhong
• Dataset• Massachusetts’ road dataset and building dataset • each image consists of 3×1500×1500 pixels• contains 1,711 aerial images,
• the FCN’s computation consumption could be much higher than that of the ordinary object recognition models.
![Page 19: Object extraction from satellite imagery using deep learning](https://reader034.fdocuments.net/reader034/viewer/2022050613/5a64b7267f8b9ac21c8b4c7f/html5/thumbnails/19.jpg)
Building detection in very high resolution multispectral data with deep learning features 2016• AlexNet -> Features + SVM ( Last layer )
![Page 20: Object extraction from satellite imagery using deep learning](https://reader034.fdocuments.net/reader034/viewer/2022050613/5a64b7267f8b9ac21c8b4c7f/html5/thumbnails/20.jpg)
Road network extraction a neural-dynamic framework based on deep learning and a finite state machine wang2015
CNN + FSM ( for sequence )
![Page 21: Object extraction from satellite imagery using deep learning](https://reader034.fdocuments.net/reader034/viewer/2022050613/5a64b7267f8b9ac21c8b4c7f/html5/thumbnails/21.jpg)
Do Deep Features Generalize from Everyday Objectsto Remote Sensing and Aerial Scenes Domains? 2016Ot´avio A. B. Penatti
• ConvNet using Caffe and OverFeat
![Page 22: Object extraction from satellite imagery using deep learning](https://reader034.fdocuments.net/reader034/viewer/2022050613/5a64b7267f8b9ac21c8b4c7f/html5/thumbnails/22.jpg)
Using convolutional networks and satellite imagery to identify pa.erns inurban environments at a large scaleADRIAN ALBERT*, Massachuse.s Institute of Technology2017
• Dataset :
• UC Merced land use dataset [25] (of 2100 images spanning 21 classes)
• DeepSat land use benchmark dataset ( 4 channels
• (VGGNet and ResNet)
![Page 23: Object extraction from satellite imagery using deep learning](https://reader034.fdocuments.net/reader034/viewer/2022050613/5a64b7267f8b9ac21c8b4c7f/html5/thumbnails/23.jpg)
6. Code sample/prelim
inary results
• (Test) DSTL
• https://www.kaggle.com/alyosama/dstl-satellite-imagery-feature-detection/convnet-baseline/
![Page 24: Object extraction from satellite imagery using deep learning](https://reader034.fdocuments.net/reader034/viewer/2022050613/5a64b7267f8b9ac21c8b4c7f/html5/thumbnails/24.jpg)
YOLT2
• The actual F1 score of 0.21
• Jaccard index between 0.4 and 0.5
https://medium.com/the-downlinq/building-extraction-with-yolt2-and-spacenet-data-a926f9ffac4fT
![Page 25: Object extraction from satellite imagery using deep learning](https://reader034.fdocuments.net/reader034/viewer/2022050613/5a64b7267f8b9ac21c8b4c7f/html5/thumbnails/25.jpg)
CosmiQNet
• Blackbox Fully Convolution Neural Network: CosmiQNet. The inputs are at two resolutions and the output distance transform matches the lower of the input resolutions. The resolution of the 8-band GeoTIFF is roughly one quarter (in each dimension) the resolution of the the 3-band GeoTIFF; the difference in resolution is depicted by the scale of the GeoTIFFs.
https://medium.com/the-downlinq/object-detection-on-spacenet-5e691961d257
![Page 26: Object extraction from satellite imagery using deep learning](https://reader034.fdocuments.net/reader034/viewer/2022050613/5a64b7267f8b9ac21c8b4c7f/html5/thumbnails/26.jpg)
Resources
1. DSTL Satellite Imagery Competiton
2. https://medium.com/@avanetten
3. https://www.kernix.com/blog/image-classification-with-a-pre-trained-deep-neural-network_p11