Ph.D. IN GEOMATICS SCIENCES

Innovative deep learning solutions leveraging unlabeled and multi-source annotated datasets to segment and monitor change in large scale 3D point clouds for Digital Twin Cities

Université Laval, Québec, Canada

Description of the research project

A Digital Twin City (DTC) is a virtual representation of physical assets across their respective life cycles in a city. This is an emerging concept that has become the centre of attention for academia in more recent years. The rapid development of mobile LiDAR system (MLS) provides large-scale, accurate and affordable 3D point clouds (PCs) at a city-scale. These data offer high-quality geometric measurements but limited non-geometric semantics, such as object type and topological relationships. Therefore, semantic segmentation is a fundamental perception task for the massive datasets used in DTC creation.

In recent years, the development of deep neural networks (DNNs) has led to great success in 3D PC semantic segmentation while focusing on designing fully supervised networks for densely annotated data. However, such massive 3D point-wise annotation is time-consuming, labour-intensive, and error-prone. As a result, current effort is in developing efficient methods for creating more labels or learning from cheap supervision (i.e. little annotation or simulated data).

DTC involves 4D dynamic objects. Since LiDAR provides a snapshot of the DTC at a certain timestamp, there is a need to detect and understand changes in multi-temporal LiDAR datasets. Performing change detection in street-level PC streams is often a significantly complex task. One must expect a large variety of object shapes and appearances, numerous occlusion artifacts between the different objects, and a lack of accurate registration between the compared datasets. There is a crucial lack of studies using DNNs to address this problem. Furthermore, DTC creation should be addressed with a focus on automatic updating because a city is constantly changing. Existing urban 3D modelling methods are limited in this respect.

By extending our on-going research, the grand goal for this research is to design a deep learning-based framework for large-scale outdoor point clouds to accommodate the need for Digital Twin City creation and monitoring over time. The intent is: 1) to develop innovative deep learning approaches to be less dependent on annotated data and better able to leverage datasets with different characteristics, even unlabelled or simulated ones; 2) to propose solutions to detect and monitor changes in urban environments at the point level; 3) to design new approaches able to learn the spatial and dynamic structures of urban environments in order to propose suitable and consistent 3D reconstructions.

Anticipated impact and significance of the project

This research program will be instrumental towards the advancement of knowledge on deep learning applied to large-scale outdoor PCs. Furthermore, it will contribute to 3D PCs contexts that are overlooked in the literature which is often dedicated to autonomous vehicles. More specifically, the proposed research program aims at tackling the challenges and problems related to learning in an unsupervised context, generalizable unsupervised approaches, change detection at the point level as well as learning 3D spatial representations from generative models. Even though there has recently been a strong interest and increased research work on URL applied to 3D PCs, this domain still lags far behind its counterparts in NLP and 2D computer vision tasks.
DTC is an emerging concept that has become the centre of attention for industry and academia. There is an increasing demand for 3D city models to support smart cities, autonomous driving, outdoor augmented reality, urban planning, construction 4.0 applications, to name a few. This research could lead to significant advancements in these fields by providing large-scale, accurate, semantically rich and affordable data at a city-scale as well as tools to track city assets over time.

Profile
Ph.D.: current open position for a full-time graduate student for 9 university terms over a period of 4 years (a year consists of 3 university terms)

Fundings

Fully funded with a grant from the Canadian Research Council NSERC.

Study place
This PhD’s project will take place in Quebec, on the Université Laval campus. The student will be a member of the Research Centre on Geospatial Data and Intelligence (CRDIG) and will have access to computing resources from its laboratories.

Starting date
as soon as possible

To apply, send now to Sylvie Daniel (sylvie.daniel@scg.ulaval.ca)
• A cover letter related to the proposed project,
• A CV,
• Your most recent transcript,
• The contact details of 2 references.