SYNTHETIC DATA FOR EARTH OBSERVATION (SD4EO)

SD4EO is a study project that aims to demonstrate the benefit of using physically-based simulation data and Artificial Intelligence-based data generation tools in key thematic applications with Earth Observation data and Artificial Intelligence (AI) analytics. The project started in October 2023 and is being led by GMV NSL Ltd (UK), in partnership with the Spanish-based company GMV SGI and the University of Valencia, Spain.

In the Earth Observation domain, Artificial Intelligence techniques have the potential to automate complex analyses without human intervention, enhancing sensitivity to specific patterns of interest. However, achieving significant performance with AI-driven analytics necessitates a training phase involving the processing of a large, diverse, balanced, and curated set of images, cross-checked against matching reference labels. In practice, acquiring and meticulously annotating such a vast number of images for all required observation conditions is challenging and typically requires substantial resources.

Simulation data, capable of realistically replicating physical conditions, can serve as a valuable and complementary data source in this context. This data can indeed provide synthetic images that mimic real imagery acquisitions, thereby enabling the reproduction of the sensing performance of different types of sensors.

Simulated data can be particularly beneficial in Earth Observation thematic domains related to target categorisation. The large number of images required to train any analytic module is difficult to obtain through real sensors, limiting the ability to characterise targets of interest through relevant and accurate image-related attributes. Furthermore, target categorisation is often hampered by a lack of useful, accurate, and verified reference labels. The resources required are considerable, as specialised in-field campaigns typically need to be organised, involving the deployment of advanced equipment and trained data collectors.

The present study seeks to establish foundational elements for the ambitious goal of incorporating simulation data into the AI-driven Earth Observation analytic pipelines (AI4EO), and to explore whether this additional data source can supplement the real measurements obtained by EO sensors. With sufficiently realistic simulations, AI analytics could be designed, developed, and implemented with performance levels that match or exceed those that could be achieved with a vast quantity of real images.

To achieve this, the study makes use of the most recent technical and technological advancements in the domain of real scene simulation and analyses two main theoretical rationales:

• Physically-based simulation, which employs configurable mathematical models to characterise a scenario.
• AI-driven simulation, which utilises training datasets to learn about certain predefined characteristics and applies this knowledge to synthesise simulated data.

The project aims to demonstrate the benefit of using synthetic data and simulation-to-reality techniques with AI4EO by targeting the following 3 use cases:

• Categorisation of crop fields for resource monitoring and management.
• Categorisation of human settlements for energy consumption monitoring and management.
• Monitoring of photovoltaic panels status to evaluate the level of self-consumption power generation.

During the project, the team will also engage the AI and EO communities to raise awareness about the use of physically-based synthetic data for Earth Observation applications and to disseminate the results.