Deep Learning for Hyperspectral (BEETLES)

Hyperspectral imaging can capture hundreds of images acquired for narrow and continuous spectral bands across the electromagnetic spectrum, hence can allow us to precisely analyze the materials that are present in a scene of interest. However, the large volume of hyperspectral images (HSIs) makes their manual analysis and transfer very costly and time-expensive, especially when they are acquired on-board imaging satellites. Therefore, deploying automated algorithms for the efficient HSI processing on-board satellites is an important scientific and engineering topic, and on-board artificial intelligence – employed both in the context of hyperspectral data reduction through band selection or feature extraction, and HSI analysis aiming at extracting the value from raw data – has a potential to speed up adoption of hyperspectral analysis in emerging use cases through bringing “the brain” just next to “the eye”.

The objectives of BEETLES focuses on enabling effective adoption of deep learning in the field of remote sensing and hyperspectral image analysis, where – in most use cases – the availability of ground-truth hyperspectral data is extremely limited or non-existent. Thus, making deep learning algorithms ready-to-use on-board imaging satellites in hardware- and energy-constrained execution environments is of utmost importance. This activity allows to provide the evidence that the designed deep neural networks are applicable in real-life Earth observation use cases. Approaches were designed for quantitatively, qualitatively, and statistically proving their robustness against noise of various distributions that can reflect sensor failures or thermal noise. Finally, we can build better understanding of the underlying materials captured by HSIs through incorporating hyperspectral unmixing into the analysis pipeline.

The algorithms and approaches developed in BEETLES are application-agnostic and can be effectively deployed in orbit and on the ground in a variety of applications. The missions that aim at benefiting from on-board machine and deep learning in Earth observation, space debris monitoring, risk management, deep space missions, and many more, would be the target of BEETLES.

All algorithms were thoroughly verified using multi-fold analysis (quantitatively, qualitatively, and statistically) which helped us understand their operational and non-functional capabilities. The outcomes of the project were presented at the most important conferences in the field (IEEE IGARSS 2020, ɸ-week 2020, IAC 2020, OBPDC 2020) and papers published in top-tier journals.