PointNet for The Automatic Classification of Aerial Point Clouds
1Dept. of Materials Engineering, Applied Mechanics and Construction, University of Vigo, Spain; 2Dept. of Geoscience and Remote Sensing, TU Delft, The Netherlands
During the last couple of years, there has been an increased interest to develop new deep learning networks specifically for processing 3D point cloud data. In that context, this work intends to expand the applicability of one of these networks, PointNet, from the semantic segmentation of indoor scenes, to outdoor point clouds acquired with Airborne Laser Scanning (ALS) systems. Our goal is to of assist the classification of future iterations of a national wide dataset such as the Actueel Hoogtebestand Nederland (AHN), using a classification model trained with a previous iteration. First, a simple application such as ground classification is proposed in order to prove the capabilities of the proposed deep learning architecture to perform an efficient point-wise classification with aerial point clouds. Then, two different models based on PointNet are defined to classify the most relevant elements in the case study data: Ground, vegetation and buildings. While the model for ground classification performs with a F-score metric above 96%, motivating the second part of the work, the overall accuracy of the remaining models is around 87%, showing consistency across different versions of AHN but with improvable false positive and false negative rates. Therefore, this work concludes that the proposed classification of future AHN iterations is feasible but needs more experimentation.
Feature Relevance Analysis for 3D Point Cloud Classification Using Deep Learning
1Institute of Industrial Science, The University of Tokyo, Komaba, Japan; 2School of Engineering, The University of Tokyo, Hongo, Japan; 33D Geospatial Data Processing Research Group (3DGeo), Institute of Geography, Heidelberg University, Heidelberg, Germany; 4Interdisciplinary Center for Scientific Computing (IWR), Heidelberg University, Heidelberg, Germany
3D point clouds acquired by laser scanning and other techniques are difficult to interpret because of their irregular structure. To make sense of this data and to allow for the derivation of useful information, a segmentation of the points in groups, units, or classes fit for the specific use case is required. In this paper, we present a non-end-to-end deep learning classifier for 3D point clouds using multiple sets of input features and compare it with an implementation of the state-of-the-art deep learning framework PointNet++. We first start by extracting features derived from the local normal vector (normal vectors, eigenvalues, and eigenvectors) from the point cloud, and study the result of classification for different local search radii. We extract additional features related to spatial point distribution and use them together with the normal vector-based features. We find that the classification accuracy improves by up to 33% as we include normal vector features with multiple search radii and features related to spatial point distribution. Our method achieves a mean Intersection over Union (mIoU) of 94% outperforming PointNet++’s Multi Scale Grouping by up to 12%. The study presents the importance of multiple search radii for different point cloud features for classification in an urban 3D point cloud scene acquired by terrestrial laser scanning.
Classification of Aerial Laser Scanning Point Clouds using Machine Learning: A Comparison Between Random Forest and Tensorflow
1CIRGEO Interdepartmental Research Center of Geomatics, University of Padova, Italy; 2TESAF Department, University of Padova, Italy
In this investigation a comparison between two machine learning (ML) models for semantic classification of an aerial laser scanner point cloud is presented. One model is Random Forest (RF), the other is a multi-layer neural network, TensorFlow (TF). Accuracy results were compared over a growing set of training data, using a stratified independent sampling over classes from 5% to 50% of the total dataset. Results show RF to have average F1=0.823 for the 9 classes considered, whereas TF had average F1=0.450. F1 values where higher for RF than TF, due to complexity in the determination of a suitable composition of the hidden layers of the neural network in TF, and this can likely be improved to reach higher accuracy values. Further study in this sense is planned.
Joint Classification of ALS and DIM Point Clouds
Leibniz Univerversity Hannover, Institute of Cartography and Geoinformatics, Germany
Over the past years, the algorithms for dense image matching (DIM) to obtain point clouds from aerial images improved significantly. Consequently, DIM point clouds became a good alternative to the established Airborne Laser Scanning (ALS) point clouds for remote sensing applications. However, due to the different characteristics of DIM and ALS point clouds, a joint classification of both point cloud types is very difficult and mostly replaced by one trained classifier for each type. In this work, we explore the classification of ALS and DIM point clouds in a fully convolutional encoder-decoder network using transfer learning. We project the point clouds onto a 2D image raster plane and utilize the minimal, mean and maximal height values within each raster cell as input for the network. We train several transfer learning setups with different ratios of DIM data for fine-tuning as well as two baselines, which are purely trained on either ALS or DIM data. We show qualitative and quantitative results on a DIM test set for all setups. We show that even 10% of DIM data are sufficient to increase the classification results significantly. These findings have practical relevance for national survey departments, which acquire both ALS and DIM data and want to exploit the potential of all their data for classification without providing unnecessary amounts of new training samples for learning.
Extraction and Shape Reconstruction of Guardrails Mobile Mapping Data
The University of Electro-Communications, Japan
The mobile mapping system (MMS) can acquire dense point-clouds of roads and roadside features. Roadways and walkways are located side-by-side in many urban areas, and they are often separated by guardrails. Therefore, it is important to detect guardrails from point-clouds and reconstruct their 3D models for generating 3D street maps. Since there are a large variety of designs for guardrails in Japan, flexible methods are required for extraction and shape reconstruction of guardrails. In this paper, we propose a new method for extracting guardrails with various shape patterns from point-clouds, and reconstructing their 3D models. We suppose that the MMS captures point-clouds and camera images synchronously. In our method, point-clouds are segmented into small parts, and corresponding camera images are cropped as rectangle images. Then cropped images are classified into guardrails and others using the convolutional neural network. When guardrail points are obtained, 3D models of guardrails are reconstructed. However, point-clouds are too sparse to reconstruct complete 3D shapes when guardrails consist of thin pipes. Since the same shape repeatedly appears on a guardrail, we create dense point-clouds by superimposing points on the same shapes. We observe that guardrail shapes consist of vertical pipes, pipes, and beams. We reconstruct these shapes from superimposed point-clouds by extracting the center axes of pipes and section curves of beams.