Fostering Advancements in Foundation Models via Unsupervised and Self-Supervised Learning for Downstream Tasks in Earth ObservationThe objectives will be achieved through the validation of the proposed FMs (Multimodal Foundation Models) on suitable downstream applications that address global societal challenges. More specifically, the validation of the proposed FM architectures will be conducted on six main use cases (UCs), which are as follows:
- UC1: Weather & Climate Disaster Analysis
- UC2: Detection of Methane Leaks
- UC3: Observation of Changes in Forest Above-Ground Biomass
- UC4: Estimation of Soil Properties
- UC5: Detection of Semantic Land Cover Changes
- UC6: Monitoring Expansion of Mining Fields into Farmlands
The resulting FMs tailored to Earth observation (EO), as Geospatial Foundation Models (Geo-FMs), accompanying downstream applications for UCs, and the corresponding dataset and code repositories will be made freely available to the AI4EO community.
This FAST-EO project primarily addresses the needs of the public and private sectors involved in Earth Observation (EO) and environmental analysis. Customers in these sectors are seeking advanced, AI-powered toolkits that can be easily adapted to different use cases, requiring minimal expert knowledge or effort for accurate monitoring and analysis of various environmental and geographical phenomena. In this regard, the project is not only in line with the customer needs but also aligns with the high-level business objectives of each partner
The FAST-EO project is designed to cater to a global customer base, with a particular focus on countries and regions facing significant environmental challenges and those with strong commitments to environmental sustainability. It targets both developed nations with advanced Earth Observation capabilities and developing countries seeking to enhance their environmental monitoring infrastructure. The project’s advanced, AI-powered toolkits are also ideally suited for international organizations and NGOs dedicated to climate change mitigation, environmental protection, and disaster management. This wide-ranging appeal is due to the project’s potential capacity to provide accurate, efficient, and adaptable solutions for monitoring and analyzing diverse environmental and geographical phenomena, making it a valuable asset for countries and entities at various stages of EO and environmental analysis development
The objective of the FAUST-EO project is to enhance the accessibility and democratization of FMs within the EO community. This includes:
- Tailoring to Data Domains: Adapting FMs specifically for synthetic aperture radar (SAR), multispectral, and hyperspectral sensor characteristics, and considering the multi-temporal nature of EO data, with a focus on prioritizing existing European missions such as Sentinels, while also aligning with upcoming initiatives such as the Copernicus Hyperspectral Imaging Mission of ESA (CHIME).
- Enhanced Multimodality: Incorporating text-based, semantic masking-based or geometrical prompts to enhance multimodality beyond just sensor data.
- Overcoming Computational Barriers: Addressing computational limitations to facilitate the optimal reconfiguration and refinement of FMs for various EO tasks, thereby encouraging their widespread adoption in practical applications.
- Affordable Fine-Tuning: Design and train FM’s on world-wide samples, with the objective to reduce the number of labelled data for novel down-stream tasks, which can be deployed at a global scale.
- Operational and Accessible Implementation: Devising strategies for effectively scaling up and moving towards the operationalization of these FMs. Additionally, ensuring their accessibility for non-technical audiences, facilitated by both our project consortium members (IBM, FZJ, DLR, and KP Labs) and our project endorsers/stakeholders.
Since each use case will be examined under various modality, sensor, and temporality conditions, it can be said that some UCs have multiple sub-UCs to examine the provided Geo-FMs from different perspectives
The FAST-EO project will release the model weights, configuration for reproducibility, and datasets as open-source, following a free-of-charge and permissive licensing (Apache-2). These resources will be accessible through the SpatioTemporal Asset Catalog (STAC) or via the OpenEO API for efficient geospatial data retrieval in cloud-based systems. Additionally, the source code will be hosted on GitHub, and pretrained Geo-FMs will be available via HuggingFace, ensuring complete transparency, accessibility, and a “plug-and-play” mindset for easy investigation, similar to what we achieved with the Prithvi Model.Within the FAUST-EO project, we will enhance this software stack to provide a more versatile solution that encompasses both language and vision modalities, while also accommodating additional temporal and sensor variations. This expanded software stack will be made available under a free-of-charge and permissive licensing.
The project started on February 1st, 2024 with the project kick-off on February 5th, 2024. On September 26th-27th, 2024, the project reached successfully the first milestone, MS-1. The updates regarding the project activities and project events can be found the FAST-EO webpage https://www.fast-eo.eu/
