Showing 1–18 of 18 results for author: Ehret, T

Exploring Robust Features for Few-Shot Object Detection in Satellite Imagery

Authors: Xavier Bou, Gabriele Facciolo, Rafael Grompone von Gioi, Jean-Michel Morel, Thibaud Ehret

Abstract: The goal of this paper is to perform object detection in satellite imagery with only a few examples, thus enabling users to specify any object class with minimal annotation. To this end, we explore recent methods and ideas from open-vocabulary detection for the remote sensing domain. We develop a few-shot object detector based on a traditional two-stage architecture, where the classification block… ▽ More The goal of this paper is to perform object detection in satellite imagery with only a few examples, thus enabling users to specify any object class with minimal annotation. To this end, we explore recent methods and ideas from open-vocabulary detection for the remote sensing domain. We develop a few-shot object detector based on a traditional two-stage architecture, where the classification block is replaced by a prototype-based classifier. A large-scale pre-trained model is used to build class-reference embeddings or prototypes, which are compared to region proposal contents for label prediction. In addition, we propose to fine-tune prototypes on available training images to boost performance and learn differences between similar classes, such as aircraft types. We perform extensive evaluations on two remote sensing datasets containing challenging and rare objects. Moreover, we study the performance of both visual and image-text features, namely DINOv2 and CLIP, including two CLIP models specifically tailored for remote sensing applications. Results indicate that visual features are largely superior to vision-language models, as the latter lack the necessary domain-specific vocabulary. Lastly, the developed detector outperforms fully supervised and few-shot methods evaluated on the SIMD and DIOR datasets, despite minimal training parameters. △ Less

Submitted 8 March, 2024; originally announced March 2024.

Portraying the Need for Temporal Data in Flood Detection via Sentinel-1

Authors: Xavier Bou, Thibaud Ehret, Rafael Grompone von Gioi, Jeremy Anger

Abstract: Identifying flood affected areas in remote sensing data is a critical problem in earth observation to analyze flood impact and drive responses. While a number of methods have been proposed in the literature, there are two main limitations in available flood detection datasets: (1) a lack of region variability is commonly observed and/or (2) they require to distinguish permanent water bodies from f… ▽ More Identifying flood affected areas in remote sensing data is a critical problem in earth observation to analyze flood impact and drive responses. While a number of methods have been proposed in the literature, there are two main limitations in available flood detection datasets: (1) a lack of region variability is commonly observed and/or (2) they require to distinguish permanent water bodies from flooded areas from a single image, which becomes an ill-posed setup. Consequently, we extend the globally diverse MMFlood dataset to multi-date by providing one year of Sentinel-1 observations around each flood event. To our surprise, we notice that the definition of flooded pixels in MMFlood is inconsistent when observing the entire image sequence. Hence, we re-frame the flood detection task as a temporal anomaly detection problem, where anomalous water bodies are segmented from a Sentinel-1 temporal sequence. From this definition, we provide a simple method inspired by the popular video change detector ViBe, results of which quantitatively align with the SAR image time series, providing a reasonable baseline for future works. △ Less

Submitted 6 March, 2024; originally announced March 2024.

Radar Fields: An Extension of Radiance Fields to SAR

Authors: Thibaud Ehret, Roger Marí, Dawa Derksen, Nicolas Gasnier, Gabriele Facciolo

Abstract: Radiance fields have been a major breakthrough in the field of inverse rendering, novel view synthesis and 3D modeling of complex scenes from multi-view image collections. Since their introduction, it was shown that they could be extended to other modalities such as LiDAR, radio frequencies, X-ray or ultrasound. In this paper, we show that, despite the important difference between optical and synt… ▽ More Radiance fields have been a major breakthrough in the field of inverse rendering, novel view synthesis and 3D modeling of complex scenes from multi-view image collections. Since their introduction, it was shown that they could be extended to other modalities such as LiDAR, radio frequencies, X-ray or ultrasound. In this paper, we show that, despite the important difference between optical and synthetic aperture radar (SAR) image formation models, it is possible to extend radiance fields to radar images thus presenting the first "radar fields". This allows us to learn surface models using only collections of radar images, similar to how regular radiance fields are learned and with the same computational complexity on average. Thanks to similarities in how both fields are defined, this work also shows a potential for hybrid methods combining both optical and SAR images. △ Less

Submitted 20 December, 2023; originally announced December 2023.

Shortcut-to-Adiabatic Controlled-Phase Gate in Rydberg Atoms

Authors: Luis S. Yagüe Bosch, Tim Ehret, Francesco Petiziol, Ennio Arimondo, Sandro Wimberger

Abstract: A shortcut-to-adiabatic protocol for the realization of a fast and high-fidelity controlled-phase gate in Rydberg atoms is developed. The adiabatic state transfer, driven in the high-blockade limit, is sped up by compensating nonadiabatic transitions via oscillating fields that mimic a counterdiabatic Hamiltonian. High fidelities are obtained in wide parameter regions. The implementation of the ba… ▽ More A shortcut-to-adiabatic protocol for the realization of a fast and high-fidelity controlled-phase gate in Rydberg atoms is developed. The adiabatic state transfer, driven in the high-blockade limit, is sped up by compensating nonadiabatic transitions via oscillating fields that mimic a counterdiabatic Hamiltonian. High fidelities are obtained in wide parameter regions. The implementation of the bare effective counterdiabatic field, without original adiabatic pulses, enables to bypass gate errors produced by the accumulation of blockade-dependent dynamical phases, making the protocol efficient also at low blockade values. As an application toward quantum algorithms, how the fidelity of the gate impacts the efficiency of a minimal quantum-error correction circuit is analyzed. △ Less

Submitted 18 December, 2023; originally announced December 2023.

Comments: Cover story of Annalen der Physik, Volume 535, Issue 12

Journal ref: Annalen der Physik, 535 (Issue 12), 2300275 (2023)

Reducing False Alarms in Video Surveillance by Deep Feature Statistical Modeling

Authors: Xavier Bou, Aitor Artola, Thibaud Ehret, Gabriele Facciolo, Jean-Michel Morel, Rafael Grompone von Gioi

Abstract: Detecting relevant changes is a fundamental problem of video surveillance. Because of the high variability of data and the difficulty of properly annotating changes, unsupervised methods dominate the field. Arguably one of the most critical issues to make them practical is to reduce their false alarm rate. In this work, we develop a method-agnostic weakly supervised a-contrario validation process,… ▽ More Detecting relevant changes is a fundamental problem of video surveillance. Because of the high variability of data and the difficulty of properly annotating changes, unsupervised methods dominate the field. Arguably one of the most critical issues to make them practical is to reduce their false alarm rate. In this work, we develop a method-agnostic weakly supervised a-contrario validation process, based on high dimensional statistical modeling of deep features, to reduce the number of false alarms of any change detection algorithm. We also raise the insufficiency of the conventionally used pixel-wise evaluation, as it fails to precisely capture the performance needs of most real applications. For this reason, we complement pixel-wise metrics with object-wise metrics and evaluate the impact of our approach at both pixel and object levels, on six methods and several sequences from different datasets. Experimental results reveal that the proposed a-contrario validation is able to largely reduce the number of false alarms at both pixel and object levels. △ Less

Submitted 9 July, 2023; originally announced July 2023.

Detecting Methane Plumes using PRISMA: Deep Learning Model and Data Augmentation

Authors: Alexis Groshenry, Clement Giron, Thomas Lauvaux, Alexandre d'Aspremont, Thibaud Ehret

Abstract: The new generation of hyperspectral imagers, such as PRISMA, has improved significantly our detection capability of methane (CH4) plumes from space at high spatial resolution (30m). We present here a complete framework to identify CH4 plumes using images from the PRISMA satellite mission and a deep learning model able to detect plumes over large areas. To compensate for the relative scarcity of PR… ▽ More The new generation of hyperspectral imagers, such as PRISMA, has improved significantly our detection capability of methane (CH4) plumes from space at high spatial resolution (30m). We present here a complete framework to identify CH4 plumes using images from the PRISMA satellite mission and a deep learning model able to detect plumes over large areas. To compensate for the relative scarcity of PRISMA images, we trained our model by transposing high resolution plumes from Sentinel-2 to PRISMA. Our methodology thus avoids computationally expensive synthetic plume generation from Large Eddy Simulations by generating a broad and realistic training database, and paves the way for large-scale detection of methane plumes using future hyperspectral sensors (EnMAP, EMIT, CarbonMapper). △ Less

Submitted 17 November, 2022; originally announced November 2022.

Regularization of NeRFs using differential geometry

Authors: Thibaud Ehret, Roger Marí, Gabriele Facciolo

Abstract: Neural radiance fields, or NeRF, represent a breakthrough in the field of novel view synthesis and 3D modeling of complex scenes from multi-view image collections. Numerous recent works have shown the importance of making NeRF models more robust, by means of regularization, in order to train with possibly inconsistent and/or very sparse data. In this work, we explore how differential geometry can… ▽ More Neural radiance fields, or NeRF, represent a breakthrough in the field of novel view synthesis and 3D modeling of complex scenes from multi-view image collections. Numerous recent works have shown the importance of making NeRF models more robust, by means of regularization, in order to train with possibly inconsistent and/or very sparse data. In this work, we explore how differential geometry can provide elegant regularization tools for robustly training NeRF-like models, which are modified so as to represent continuous and infinitely differentiable functions. In particular, we present a generic framework for regularizing different types of NeRFs observations to improve the performance in challenging conditions. We also show how the same formalism can also be used to natively encourage the regularity of surfaces by means of Gaussian or mean curvatures. △ Less

Submitted 30 November, 2022; v1 submitted 29 June, 2022; originally announced June 2022.

Sat-NeRF: Learning Multi-View Satellite Photogrammetry With Transient Objects and Shadow Modeling Using RPC Cameras

Authors: Roger Marí, Gabriele Facciolo, Thibaud Ehret

Abstract: We introduce the Satellite Neural Radiance Field (Sat-NeRF), a new end-to-end model for learning multi-view satellite photogrammetry in the wild. Sat-NeRF combines some of the latest trends in neural rendering with native satellite camera models, represented by rational polynomial coefficient (RPC) functions. The proposed method renders new views and infers surface models of similar quality to tho… ▽ More We introduce the Satellite Neural Radiance Field (Sat-NeRF), a new end-to-end model for learning multi-view satellite photogrammetry in the wild. Sat-NeRF combines some of the latest trends in neural rendering with native satellite camera models, represented by rational polynomial coefficient (RPC) functions. The proposed method renders new views and infers surface models of similar quality to those obtained with traditional state-of-the-art stereo pipelines. Multi-date images exhibit significant changes in appearance, mainly due to varying shadows and transient objects (cars, vegetation). Robustness to these challenges is achieved by a shadow-aware irradiance model and uncertainty weighting to deal with transient phenomena that cannot be explained by the position of the sun. We evaluate Sat-NeRF using WorldView-3 images from different locations and stress the advantages of applying a bundle adjustment to the satellite camera models prior to training. This boosts the network performance and can optionally be used to extract additional cues for depth supervision. △ Less

Submitted 21 April, 2022; v1 submitted 16 March, 2022; originally announced March 2022.

Comments: Accepted at CVPR EarthVision Workshop 2022

Global Tracking and Quantification of Oil and Gas Methane Emissions from Recurrent Sentinel-2 Imagery

Authors: Thibaud Ehret, Aurélien De Truchis, Matthieu Mazzolini, Jean-Michel Morel, Alexandre d'Aspremont, Thomas Lauvaux, Riley Duren, Daniel Cusworth, Gabriele Facciolo

Abstract: Methane (CH4) emissions estimates from top-down studies over oil and gas basins have revealed systematic under-estimation of CH4 emissions in current national inventories. Sparse but extremely large amounts of CH4 from oil and gas production activities have been detected across the globe, resulting in a significant increase of the overall O&G contribution. However, attribution to specific faciliti… ▽ More Methane (CH4) emissions estimates from top-down studies over oil and gas basins have revealed systematic under-estimation of CH4 emissions in current national inventories. Sparse but extremely large amounts of CH4 from oil and gas production activities have been detected across the globe, resulting in a significant increase of the overall O&G contribution. However, attribution to specific facilities remains a major challenge unless high-resolution images provide the sufficient granularity within O&G basin. In this paper, we monitor known oil-and-gas infrastructures across the globe using recurrent Sentinel-2 imagery to detect and quantify more than 800 CH4 emissions. In combination with emissions estimates from airborne and Sentinel-5P measurements, we demonstrate the robustness of the fit to a power law from 0.1 tCH4/hr to 600 tCH4/hr. We conclude here that the prevalence of ultra-emitters (> 25tCH4/hr) detected globally by Sentinel-5P directly relates to emission occurrences below its detection threshold. Similar power law coefficients arise from several major oil and gas producers but noticeable differences in emissions magnitudes suggest large differences in maintenance practices and infrastructures across countries. △ Less

Submitted 30 November, 2022; v1 submitted 22 October, 2021; originally announced October 2021.

Comments: Preprint version of https://pubs.acs.org/doi/abs/10.1021/acs.est.1c08575

Parallax estimation for push-frame satellite imagery: application to super-resolution and 3D surface modeling from Skysat products

Authors: Jérémy Anger, Thibaud Ehret, Gabriele Facciolo

Abstract: Recent constellations of satellites, including the Skysat constellation, are able to acquire bursts of images. This new acquisition mode allows for modern image restoration techniques, including multi-frame super-resolution. As the satellite moves during the acquisition of the burst, elevation changes in the scene translate into noticeable parallax. This parallax hinders the results of the restora… ▽ More Recent constellations of satellites, including the Skysat constellation, are able to acquire bursts of images. This new acquisition mode allows for modern image restoration techniques, including multi-frame super-resolution. As the satellite moves during the acquisition of the burst, elevation changes in the scene translate into noticeable parallax. This parallax hinders the results of the restoration. To cope with this issue, we propose a novel parallax estimation method. The method is composed of a linear Plane+Parallax decomposition of the apparent motion and a multi-frame optical flow algorithm that exploits all frames simultaneously. Using SkySat L1A images, we show that the estimated per-pixel displacements are important for applying multi-frame super-resolution on scenes containing elevation changes and that can also be used to estimate a coarse 3D surface model. △ Less

Submitted 3 February, 2021; originally announced February 2021.

Self-Supervised training for blind multi-frame video denoising

Authors: Valéry Dewil, Jérémy Anger, Axel Davy, Thibaud Ehret, Pablo Arias, Gabriele Facciolo

Abstract: We propose a self-supervised approach for training multi-frame video denoising networks. These networks predict frame t from a window of frames around t. Our self-supervised approach benefits from the video temporal consistency by penalizing a loss between the predicted frame t and a neighboring target frame, which are aligned using an optical flow. We use the proposed strategy for online internal… ▽ More We propose a self-supervised approach for training multi-frame video denoising networks. These networks predict frame t from a window of frames around t. Our self-supervised approach benefits from the video temporal consistency by penalizing a loss between the predicted frame t and a neighboring target frame, which are aligned using an optical flow. We use the proposed strategy for online internal learning, where a pre-trained network is fine-tuned to denoise a new unknown noise type from a single video. After a few frames, the proposed fine-tuning reaches and sometimes surpasses the performance of a state-of-the-art network trained with supervision. In addition, for a wide range of noise types, it can be applied blindly without knowing the noise distribution. We demonstrate this by showing results on blind denoising of different synthetic and realistic noises. △ Less

Submitted 20 April, 2021; v1 submitted 15 April, 2020; originally announced April 2020.

Comments: 14 pages

Implementation of the VBM3D Video Denoising Method and Some Variants

Authors: Thibaud Ehret, Pablo Arias

Abstract: VBM3D is an extension to video of the well known image denoising algorithm BM3D, which takes advantage of the sparse representation of stacks of similar patches in a transform domain. The extension is rather straightforward: the similar 2D patches are taken from a spatio-temporal neighborhood which includes neighboring frames. In spite of its simplicity, the algorithm offers a good trade-off betwe… ▽ More VBM3D is an extension to video of the well known image denoising algorithm BM3D, which takes advantage of the sparse representation of stacks of similar patches in a transform domain. The extension is rather straightforward: the similar 2D patches are taken from a spatio-temporal neighborhood which includes neighboring frames. In spite of its simplicity, the algorithm offers a good trade-off between denoising performance and computational complexity. In this work we revisit this method, providing an open-source C++ implementation reproducing the results. A detailed description is given and the choice of parameters is thoroughly discussed. Furthermore, we discuss several extensions of the original algorithm: (1) a multi-scale implementation, (2) the use of 3D patches, (3) the use of optical flow to guide the patch search. These extensions allow to obtain results which are competitive with even the most recent state of the art. △ Less

Submitted 6 January, 2020; originally announced January 2020.

Comments: 18 pages, 7 figures, 5 tables

Robust copy-move forgery detection by false alarms control

Authors: Thibaud Ehret

Abstract: Detecting reliably copy-move forgeries is difficult because images do contain similar objects. The question is: how to discard natural image self-similarities while still detecting copy-moved parts as being "unnaturally similar"? Copy-move may have been performed after a rotation, a change of scale and followed by JPEG compression or the addition of noise. For this reason, we base our method on SI… ▽ More Detecting reliably copy-move forgeries is difficult because images do contain similar objects. The question is: how to discard natural image self-similarities while still detecting copy-moved parts as being "unnaturally similar"? Copy-move may have been performed after a rotation, a change of scale and followed by JPEG compression or the addition of noise. For this reason, we base our method on SIFT, which provides sparse keypoints with scale, rotation and illumination invariant descriptors. To discriminate natural descriptor matches from artificial ones, we introduce an a contrario method which gives theoretical guarantees on the number of false alarms. We validate our method on several databases. Being fully unsupervised it can be integrated into any generic automated image tampering detection pipeline. △ Less

Submitted 3 June, 2019; originally announced June 2019.

Joint Demosaicking and Denoising by Fine-Tuning of Bursts of Raw Images

Authors: Thibaud Ehret, Axel Davy, Pablo Arias, Gabriele Facciolo

Abstract: Demosaicking and denoising are the first steps of any camera image processing pipeline and are key for obtaining high quality RGB images. A promising current research trend aims at solving these two problems jointly using convolutional neural networks. Due to the unavailability of ground truth data these networks cannot be currently trained using real RAW images. Instead, they resort to simulated… ▽ More Demosaicking and denoising are the first steps of any camera image processing pipeline and are key for obtaining high quality RGB images. A promising current research trend aims at solving these two problems jointly using convolutional neural networks. Due to the unavailability of ground truth data these networks cannot be currently trained using real RAW images. Instead, they resort to simulated data. In this paper we present a method to learn demosaicking directly from mosaicked images, without requiring ground truth RGB data. We apply this to learn joint demosaicking and denoising only from RAW images, thus enabling the use of real data. In addition we show that for this application fine-tuning a network to a specific burst improves the quality of restoration for both demosaicking and denoising. △ Less

Submitted 10 September, 2019; v1 submitted 13 May, 2019; originally announced May 2019.

Comments: ICCV 2019

Reducing Anomaly Detection in Images to Detection in Noise

Authors: Axel Davy, Thibaud Ehret, Jean-Michel Morel, Mauricio Delbracio

Abstract: Anomaly detectors address the difficult problem of detecting automatically exceptions in an arbitrary background image. Detection methods have been proposed by the thousands because each problem requires a different background model. By analyzing the existing approaches, we show that the problem can be reduced to detecting anomalies in residual images (extracted from the target image) in which noi… ▽ More Anomaly detectors address the difficult problem of detecting automatically exceptions in an arbitrary background image. Detection methods have been proposed by the thousands because each problem requires a different background model. By analyzing the existing approaches, we show that the problem can be reduced to detecting anomalies in residual images (extracted from the target image) in which noise and anomalies prevail. Hence, the general and impossible background modeling problem is replaced by simpler noise modeling, and allows the calculation of rigorous thresholds based on the a contrario detection theory. Our approach is therefore unsupervised and works on arbitrary images. △ Less

Submitted 25 April, 2019; originally announced April 2019.

Comments: ICIP 2018

Journal ref: 25th IEEE International Conference on Image Processing (2018) 1058-1062

Model-blind Video Denoising Via Frame-to-frame Training

Authors: Thibaud Ehret, Axel Davy, Jean-Michel Morel, Gabriele Facciolo, Pablo Arias

Abstract: Modeling the processing chain that has produced a video is a difficult reverse engineering task, even when the camera is available. This makes model based video processing a still more complex task. In this paper we propose a fully blind video denoising method, with two versions off-line and on-line. This is achieved by fine-tuning a pre-trained AWGN denoising network to the video with a novel fra… ▽ More Modeling the processing chain that has produced a video is a difficult reverse engineering task, even when the camera is available. This makes model based video processing a still more complex task. In this paper we propose a fully blind video denoising method, with two versions off-line and on-line. This is achieved by fine-tuning a pre-trained AWGN denoising network to the video with a novel frame-to-frame training strategy. Our denoiser can be used without knowledge of the origin of the video or burst and the post processing steps applied from the camera sensor. The on-line process only requires a couple of frames before achieving visually-pleasing results for a wide range of perturbations. It nonetheless reaches state of the art performance for standard Gaussian noise, and can be used off-line with still better performance. △ Less

Submitted 25 February, 2020; v1 submitted 30 November, 2018; originally announced November 2018.

Comments: CVPR 2019

Non-Local Video Denoising by CNN

Authors: Axel Davy, Thibaud Ehret, Jean-Michel Morel, Pablo Arias, Gabriele Facciolo

Abstract: Non-local patch based methods were until recently state-of-the-art for image denoising but are now outperformed by CNNs. Yet they are still the state-of-the-art for video denoising, as video redundancy is a key factor to attain high denoising performance. The problem is that CNN architectures are hardly compatible with the search for self-similarities. In this work we propose a new and efficient w… ▽ More Non-local patch based methods were until recently state-of-the-art for image denoising but are now outperformed by CNNs. Yet they are still the state-of-the-art for video denoising, as video redundancy is a key factor to attain high denoising performance. The problem is that CNN architectures are hardly compatible with the search for self-similarities. In this work we propose a new and efficient way to feed video self-similarities to a CNN. The non-locality is incorporated into the network via a first non-trainable layer which finds for each patch in the input image its most similar patches in a search region. The central values of these patches are then gathered in a feature vector which is assigned to each image pixel. This information is presented to a CNN which is trained to predict the clean image. We apply the proposed architecture to image and video denoising. For the latter patches are searched for in a 3D spatio-temporal volume. The proposed architecture achieves state-of-the-art results. To the best of our knowledge, this is the first successful application of a CNN to video denoising. △ Less

Submitted 2 July, 2019; v1 submitted 30 November, 2018; originally announced November 2018.

Comments: A shorter version of this work has been accepted at ICIP 2019 (A NON-LOCAL CNN FOR VIDEO DENOISING). The results of v2 were improved compared to v1 and the code was updated accordingly. Code is available at: https://github.com/axeldavy/vnlnet

Image Anomalies: a Review and Synthesis of Detection Methods

Authors: Thibaud Ehret, Axel Davy, Jean-Michel Morel, Mauricio Delbracio

Abstract: We review the broad variety of methods that have been proposed for anomaly detection in images. Most methods found in the literature have in mind a particular application. Yet we show that the methods can be classified mainly by the structural assumption they make on the "normal" image. Five different structural assumptions emerge. Our analysis leads us to reformulate the best representative algor… ▽ More We review the broad variety of methods that have been proposed for anomaly detection in images. Most methods found in the literature have in mind a particular application. Yet we show that the methods can be classified mainly by the structural assumption they make on the "normal" image. Five different structural assumptions emerge. Our analysis leads us to reformulate the best representative algorithms by attaching to them an a contrario detection that controls the number of false positives and thus derive universal detection thresholds. By combining the most general structural assumptions expressing the background's normality with the best proposed statistical detection tools, we end up proposing generic algorithms that seem to generalize or reconcile most methods. We compare the six best representatives of our proposed classes of algorithms on anomalous images taken from classic papers on the subject, and on a synthetic database. Our conclusion is that it is possible to perform automatic anomaly detection on a single image. △ Less

Submitted 3 June, 2019; v1 submitted 7 August, 2018; originally announced August 2018.

Comments: Thibaud Ehret and Axel Davy contributed equally to this work