The main objective of the RepreSent project is to capitalize on the potential of artificial intelligence (AI) and Earth observation (EO) by exploiting the non-supervised learning paradigms. In this context, it is essential to come up with non-supervised learning-based solutions for impactful use cases that use unlabeled EO data.
The project started on April 1st, 2022 with the project kick-off on April 6th, 2022. The project reached successfully the milestone MS-1 in July 11th, 2022 and the milestone MS-2 in October 12th, 2022. The final review of the project took place successfully on May 11th, 2023 at ESRIN.
Objectives
Firstly, the consortium will investigate the non-supervised learning-based methods for EO, as one of project technical objective. Towards harnessing on the effectiveness of non-supervised learning and exploiting the multitude of unlabeled data, successful feature extractors will be built based on self-supervised learning-based pre-training and transfer learning from pre-trained networks.
These techniques are grouped into three main categories:
- Self-supervised learning for uni-temporal tasks (e.g., Deep Clustering, Contrastive Learning, Bootstrap Your Own Latent, Meta-Learning),
- Change detection (e.g., Deep change vector analysis, Deep multi-temporal segmentation, Temporal contrastive learning),
- Time-series anomaly detection (e.g., LSTM based unsupervised approach, Graph neural networks based on a self‑supervised approach).
DLR and EPFL will use their AI4EO network and they will establish contacts with other research groups addressing similar or related thematic areas in the AI/EO/NS community.
Secondly, the validation of the technical objectives is fulfilled by the consortium by defining five use cases. These use cases are focus on the challenges of EO where either labelled data are scarce or the successful application of supervised methods requires many labels that are tedious or costly to collect. The validation was done on five use cases related to:
- UC1. Forest disturbance monitoring
- UC2. Automated Land Cover mapping
- UC3. Anomaly detection in long time series of PS-P InSAR
- UC4. Cloud detection, and removal
- UC5. Forest biomass estimation
For these use cases, the required EO data is exploit the openly available Sentinel-1, Sentinel-2, and Landsat archives.
The RepreSent EO use cases are tackling core business and scientific questions since they are tightly connected to e-GEOS and VTT core business areas and active projects (InSAR/ground motion and forestry, respectively). This is also ensuring a close link to users and stakeholders in the different sectors where the two partners are very well known.
Finally, the non-supervised learning methods developed by the consortium were evaluated based on usual quantitative performance metrics along with qualitative analyses commenting on their generalization capability and versatility on different EO cases.
AI4EOcommunity of players
Each use case proposed within the project had a user target group. For example, for the use case on forestry, VTT used a wide community of academic and forestry users including forest owners via the Forestry TEP. On May 4th, 2023, an online workshop on Self-supervised learning (SSL) in Earth Observation based forest inventory was conducted by VTT and the ESA RepreSent project consortium.
For the use case on anomaly detection, e-GEOS is in contact with several customers interested in the detection of anomalous points.
The project partners were involved in the organization of special sessions at different conferences related to the topic of the RepreSent project. In June 2022 it was organized by EPFL and DLR the first special session at Living Planet Symposium (LPS’22) in Bonn, Germany entitled “Representation learning in remote sensing: from unsupervised to self- and meta-learning”, while in July 2022 an invited session was organized by EPFL and VTT at the ISPRS congress 2022 in Nice, France entitled “Unsupervised and weakly supervised deep learning for EO”. For 2023, the RepreSent team has organised a community invited session at IGARSS 2023 in Pasadena, USA. The session is titled “Representation learning in remote sensing” and will be composed of two sub sections (out of which of the accepted papers, two belong to the members of the consortium). Members of the team will chair the sessions in Pasadena. EPFL together with ESA are co-organiser of the EarthVision workshop at CVPR in 2022 and 2023.
Finally, during the final review of the project at ESA in ESRIN, two public sessions were organised by the RepreSent consortium together with ESA under the topic AI4EO on SSL.
Conference papers
- R. Kuzu, Y. Wang, O. Dumitru, L. Bagaglini, G. Pasquali, F. Santarelli, F. Trillo, S. Saha, and X. Zhu, An Unsupervised Anomaly Detection Problem in Urban InSAR-PSP Long Time-series, EGU 2023, Vienna, 23-28 April 2023.
- O. Antropov, M. Molinier, R. Kuzu, L. Hughes, M. Rußwurm, D. Tuia, O. Dumitru, S. Ge, S. Saha, X. Zhu, Semi-Supervised Deep Learning Representations in Earthe Observation Based Forest Management, IGRASS 2023, Pasadena, 16-21 July 2023.
- M. Rußwurm, L. Hughes, G. Pasquali, O. Dumitru, D. Tuia, Detection of Settlements in Tanzania and Mozambique by Many Regional Few-Shot Models, IGRASS 2023, Pasadena, 16-21 July 2023.
- C.O. Dumitru, R. Kuzu, L. Hughes, M. Russwurm, D. Tuia, O. Antropov, M. Molinier, G. Pasquali, L. Bagaglini, A. Rösel, and X.X. Zhu, RepreSent: Non-supervised Representation Learning for Sentinels, Neue Perspektiven der Erdbeobachtung Symposium, Bonn, Germany, 26-28 June 2023 (poster).
Additional achievements:
