AI4FOOD

An unprecedented richness of data is captured by satellites every day, resulting in an ever-growing time series of EO data. Despite the extensive availability of data, there are still many challenges when defining procedures for extracting relevant information from long time series. The goal of the AI4FOOD project is to tackle the challenges that arise when requiring data fusion or advanced time series analytics.

The AI4FOOD project investigates advanced Artificial Intelligence (AI) and Machine Learning (ML) techniques to develop new algorithms for the creation of fused (with a focus on Sentinel-1 SAR and Sentinel-2 optical) continuous data streams, and evaluate aspects such as time series predictability over different land environments. This is done by a consortium of industry experts on data fusion and time series techniques (University of Valencia and VITO, e.g., CROPSAR, SPIRITS, FAO-ASIS, ESA WorldCover and WorldCereal, ERC-SENTIFLEX, Cost-SENSECO network), and open-source implementation and operational service provision to users (Sinergise and VITO; i.e., Sentinel Hub, Terrascope, Euro Data Cube, openEO Platform, S2 Global Mosaic, eo-learn, s2cloudless, EO Browser, WatchItGrow platform, ESA-MEP and other initiatives by the partners).

Within AI4FOOD, the consortium strives to create an open-source, modular, extensible, and reusable toolbox, called fuseTS. To support the fusion of complementary EO data streams and time series analytics, relevant algorithms will be integrated into the toolbox as a service.

One of the objectives of the project is to develop a data fusion framework for Sentinel-1 SAR and Sentinel-2 multi-spectral data, taking advantage of state-of-the-art ML approaches and allowing the creation of fused data streams for various target variables.

Data fusion can become particularly interesting in the context of land monitoring, where a temporal dense time series is crucial to detect various. For example, in areas affected by frequent cloud cover, optical satellite imagery is often insufficient to reconstruct detailed time series. By integrating algorithms such as Multi Output Gaussian Process Regression (MOGPR), the Whittaker smoother or Deep Learning based methods (e.g., GANs), the framework will provide several options for users to fuse and enhance different EO data streams.

Time Series Analytics
While data fusion can lead to temporally continuous data streams, actual information extraction from these fused data streams requires a time series modelling and analytics framework. The project aims at integrating several algorithms for time series fitting and characterization from which information about land surface dynamics can be extracted. This includes change detection for gradual trends, abrupt changes, and periodical behavior e.g. related to growing seasons.

On-Demand Service
Next to the FuseTS Python library, which can be installed locally, the AI4FOOD toolbox will also be made available as a cloud-based on-demand service. For the actual deployment, existing operational services such OpenEO platform (VITO, Sinergise) and Euro Data Cube (Sinergise) will be used. This cloud-based implementation will lead to a scalable software-as-a-service (SAAS) approach ensuring that a wider group of users will be able to implement data fusion and time series analytics techniques to develop EO-based services.

Use case driven
The algorithms that are included in the toolbox are demonstrated through 3 real-life use cases:

1. Subtle Land Cover Change Monitoring (FAO)
2. Cropland Phenology Indicators (ITACyL)
3. Agriculture and Land Management Activities Identification (Agency for Agricultural Markets and Rural Development, Slovenia)

Code repositories:
https://github.com/Open-EO/FuseTS
https://open-eo.github.io/FuseTS/

Notebooks:
https://github.com/Open-EO/FuseTS/tree/main/notebooks