-
NIRPS first light and early science: breaking the 1 m/s RV precision barrier at infrared wavelengths
Authors:
Étienne Artigau,
François Bouchy,
René Doyon,
Frédérique Baron,
Lison Malo,
François Wildi,
Franceso Pepe,
Neil J. Cook,
Simon Thibault,
Vladimir Reshetov,
Xavier Dumusque,
Christophe Lovis,
Danuta Sosnowska,
Bruno L. Canto Martins,
Jose Renan De Medeiros,
Xavier Delfosse,
Nuno Santos,
Rafael Rebolo,
Manuel Abreu,
Guillaume Allain,
Romain Allart,
Hugues Auger,
Susana Barros,
Luc Bazinet,
Nicolas Blind
, et al. (89 additional authors not shown)
Abstract:
The Near-InfraRed Planet Searcher or NIRPS is a precision radial velocity spectrograph developed through collaborative efforts among laboratories in Switzerland, Canada, Brazil, France, Portugal and Spain. NIRPS extends to the 0.98-1.8 $μ$m domain of the pioneering HARPS instrument at the La Silla 3.6-m telescope in Chile and it has achieved unparalleled precision, measuring stellar radial velocit…
▽ More
The Near-InfraRed Planet Searcher or NIRPS is a precision radial velocity spectrograph developed through collaborative efforts among laboratories in Switzerland, Canada, Brazil, France, Portugal and Spain. NIRPS extends to the 0.98-1.8 $μ$m domain of the pioneering HARPS instrument at the La Silla 3.6-m telescope in Chile and it has achieved unparalleled precision, measuring stellar radial velocities in the infrared with accuracy better than 1 m/s. NIRPS can be used either stand-alone or simultaneously with HARPS. Commissioned in late 2022 and early 2023, NIRPS embarked on a 5-year Guaranteed Time Observation (GTO) program in April 2023, spanning 720 observing nights. This program focuses on planetary systems around M dwarfs, encompassing both the immediate solar vicinity and transit follow-ups, alongside transit and emission spectroscopy observations. We highlight NIRPS's current performances and the insights gained during its deployment at the telescope. The lessons learned and successes achieved contribute to the ongoing advancement of precision radial velocity measurements and high spectral fidelity, further solidifying NIRPS' role in the forefront of the field of exoplanets.
△ Less
Submitted 13 June, 2024; v1 submitted 12 June, 2024;
originally announced June 2024.
-
GTA-HDR: A Large-Scale Synthetic Dataset for HDR Image Reconstruction
Authors:
Hrishav Bakul Barua,
Kalin Stefanov,
KokSheik Wong,
Abhinav Dhall,
Ganesh Krishnasamy
Abstract:
High Dynamic Range (HDR) content (i.e., images and videos) has a broad range of applications. However, capturing HDR content from real-world scenes is expensive and time-consuming. Therefore, the challenging task of reconstructing visually accurate HDR images from their Low Dynamic Range (LDR) counterparts is gaining attention in the vision research community. A major challenge in this research pr…
▽ More
High Dynamic Range (HDR) content (i.e., images and videos) has a broad range of applications. However, capturing HDR content from real-world scenes is expensive and time-consuming. Therefore, the challenging task of reconstructing visually accurate HDR images from their Low Dynamic Range (LDR) counterparts is gaining attention in the vision research community. A major challenge in this research problem is the lack of datasets, which capture diverse scene conditions (e.g., lighting, shadows, weather, locations, landscapes, objects, humans, buildings) and various image features (e.g., color, contrast, saturation, hue, luminance, brightness, radiance). To address this gap, in this paper, we introduce GTA-HDR, a large-scale synthetic dataset of photo-realistic HDR images sampled from the GTA-V video game. We perform thorough evaluation of the proposed dataset, which demonstrates significant qualitative and quantitative improvements of the state-of-the-art HDR image reconstruction methods. Furthermore, we demonstrate the effectiveness of the proposed dataset and its impact on additional computer vision tasks including 3D human pose estimation, human body part segmentation, and holistic scene segmentation. The dataset, data collection pipeline, and evaluation code are available at: https://github.com/HrishavBakulBarua/GTA-HDR.
△ Less
Submitted 26 March, 2024;
originally announced March 2024.
-
Human Brain Exhibits Distinct Patterns When Listening to Fake Versus Real Audio: Preliminary Evidence
Authors:
Mahsa Salehi,
Kalin Stefanov,
Ehsan Shareghi
Abstract:
In this paper we study the variations in human brain activity when listening to real and fake audio. Our preliminary results suggest that the representations learned by a state-of-the-art deepfake audio detection algorithm, do not exhibit clear distinct patterns between real and fake audio. In contrast, human brain activity, as measured by EEG, displays distinct patterns when individuals are expos…
▽ More
In this paper we study the variations in human brain activity when listening to real and fake audio. Our preliminary results suggest that the representations learned by a state-of-the-art deepfake audio detection algorithm, do not exhibit clear distinct patterns between real and fake audio. In contrast, human brain activity, as measured by EEG, displays distinct patterns when individuals are exposed to fake versus real audio. This preliminary evidence enables future research directions in areas such as deepfake audio detection.
△ Less
Submitted 14 March, 2024; v1 submitted 22 February, 2024;
originally announced February 2024.
-
HistoHDR-Net: Histogram Equalization for Single LDR to HDR Image Translation
Authors:
Hrishav Bakul Barua,
Ganesh Krishnasamy,
KokSheik Wong,
Abhinav Dhall,
Kalin Stefanov
Abstract:
High Dynamic Range (HDR) imaging aims to replicate the high visual quality and clarity of real-world scenes. Due to the high costs associated with HDR imaging, the literature offers various data-driven methods for HDR image reconstruction from Low Dynamic Range (LDR) counterparts. A common limitation of these approaches is missing details in regions of the reconstructed HDR images, which are over-…
▽ More
High Dynamic Range (HDR) imaging aims to replicate the high visual quality and clarity of real-world scenes. Due to the high costs associated with HDR imaging, the literature offers various data-driven methods for HDR image reconstruction from Low Dynamic Range (LDR) counterparts. A common limitation of these approaches is missing details in regions of the reconstructed HDR images, which are over- or under-exposed in the input LDR images. To this end, we propose a simple and effective method, HistoHDR-Net, to recover the fine details (e.g., color, contrast, saturation, and brightness) of HDR images via a fusion-based approach utilizing histogram-equalized LDR images along with self-attention guidance. Our experiments demonstrate the efficacy of the proposed approach over the state-of-art methods.
△ Less
Submitted 8 February, 2024;
originally announced February 2024.
-
TESS and ESPRESSO discover a super-Earth and a mini-Neptune orbiting the K-dwarf TOI-238
Authors:
A. Suárez Mascareño,
V. M. Passegger,
J. I. González Hernández,
D. J. Armstrong,
L. D. Nielsen,
C. Lovis,
B. Lavie,
S. G. Sousa,
A. M. Silva,
R. Allart,
R. Rebolo,
F. Pepe,
N. C. Santos,
S. Cristiani,
A. Sozzetti,
M. R. Zapatero Osorio,
H. M. Tabernero,
X. Dumusque,
S. Udry,
V. Adibekyan,
C. Allende Prieto,
Y. Alibert,
S. C. C. Barros,
F. Bouchy,
A. Castro-González
, et al. (31 additional authors not shown)
Abstract:
The number of super-Earth and mini-Neptune planet discoveries has increased significantly in the last two decades thanks to transit and radial velocity surveys. When it is possible to apply both techniques, we can characterise the internal composition of exoplanets, which in turn provides unique insights on their architecture, formation and evolution.
We performed a combined photometric and radi…
▽ More
The number of super-Earth and mini-Neptune planet discoveries has increased significantly in the last two decades thanks to transit and radial velocity surveys. When it is possible to apply both techniques, we can characterise the internal composition of exoplanets, which in turn provides unique insights on their architecture, formation and evolution.
We performed a combined photometric and radial velocity analysis of TOI-238 (TYC 6398-132-1), which has one short-orbit super-Earth planet candidate announced by NASA's TESS team. We aim to confirm its planetary nature using radial velocities taken with the ESPRESSO and HARPS spectrographs, to measure its mass and to detect the presence of other possible planetary companions. We carried out a joint analysis by including Gaussian processes and Keplerian orbits to account for the stellar activity and planetary signals simultaneously.
We detected the signal induced by TOI-238 b in the radial velocity time-series, and the presence of a second transiting planet, TOI-238 c, whose signal appears in RV and TESS data. TOI-238 b is a planet with a radius of 1.402$^{+0.084}_{-0.086}$ R$_{\oplus}$ and a mass of 3.40$^{+0.46}_{-0.45}$ M$_{\oplus}$. It orbits at a separation of 0.02118 $\pm$ 0.00038 AU of its host star, with an orbital period of 1.2730988 $\pm$ 0.0000029 days, and has an equilibrium temperature of 1311 $\pm$ 28 K. TOI-238 c has a radius of 2.18$\pm$ 0.18 R$_{\oplus}$ and a mass of 6.7 $\pm$ 1.1 M$_{\oplus}$. It orbits at a separation of 0.0749 $\pm$ 0.0013 AU of its host star, with an orbital period of 8.465652 $\pm$ 0.000031 days, and has an equilibrium temperature of 696 $\pm$ 15 K. The mass and radius of planet b are fully consistent with an Earth-like composition, making it likely a rocky super-Earth. Planet c could be a water-rich planet or a rocky planet with a small H-He atmosphere.
△ Less
Submitted 6 February, 2024;
originally announced February 2024.
-
AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Dataset
Authors:
Zhixi Cai,
Shreya Ghosh,
Aman Pankaj Adatia,
Munawar Hayat,
Abhinav Dhall,
Kalin Stefanov
Abstract:
The detection and localization of highly realistic deepfake audio-visual content are challenging even for the most advanced state-of-the-art methods. While most of the research efforts in this domain are focused on detecting high-quality deepfake images and videos, only a few works address the problem of the localization of small segments of audio-visual manipulations embedded in real videos. In t…
▽ More
The detection and localization of highly realistic deepfake audio-visual content are challenging even for the most advanced state-of-the-art methods. While most of the research efforts in this domain are focused on detecting high-quality deepfake images and videos, only a few works address the problem of the localization of small segments of audio-visual manipulations embedded in real videos. In this research, we emulate the process of such content generation and propose the AV-Deepfake1M dataset. The dataset contains content-driven (i) video manipulations, (ii) audio manipulations, and (iii) audio-visual manipulations for more than 2K subjects resulting in a total of more than 1M videos. The paper provides a thorough description of the proposed data generation pipeline accompanied by a rigorous analysis of the quality of the generated data. The comprehensive benchmark of the proposed dataset utilizing state-of-the-art deepfake detection and localization methods indicates a significant drop in performance compared to previous datasets. The proposed dataset will play a vital role in building the next-generation deepfake localization methods. The dataset and associated code are available at https://github.com/ControlNet/AV-Deepfake1M .
△ Less
Submitted 26 November, 2023;
originally announced November 2023.
-
ArtHDR-Net: Perceptually Realistic and Accurate HDR Content Creation
Authors:
Hrishav Bakul Barua,
Ganesh Krishnasamy,
KokSheik Wong,
Kalin Stefanov,
Abhinav Dhall
Abstract:
High Dynamic Range (HDR) content creation has become an important topic for modern media and entertainment sectors, gaming and Augmented/Virtual Reality industries. Many methods have been proposed to recreate the HDR counterparts of input Low Dynamic Range (LDR) images/videos given a single exposure or multi-exposure LDRs. The state-of-the-art methods focus primarily on the preservation of the rec…
▽ More
High Dynamic Range (HDR) content creation has become an important topic for modern media and entertainment sectors, gaming and Augmented/Virtual Reality industries. Many methods have been proposed to recreate the HDR counterparts of input Low Dynamic Range (LDR) images/videos given a single exposure or multi-exposure LDRs. The state-of-the-art methods focus primarily on the preservation of the reconstruction's structural similarity and the pixel-wise accuracy. However, these conventional approaches do not emphasize preserving the artistic intent of the images in terms of human visual perception, which is an essential element in media, entertainment and gaming. In this paper, we attempt to study and fill this gap. We propose an architecture called ArtHDR-Net based on a Convolutional Neural Network that uses multi-exposed LDR features as input. Experimental results show that ArtHDR-Net can achieve state-of-the-art performance in terms of the HDR-VDP-2 score (i.e., mean opinion score index) while reaching competitive performance in terms of PSNR and SSIM.
△ Less
Submitted 7 September, 2023;
originally announced September 2023.
-
S-HR-VQVAE: Sequential Hierarchical Residual Learning Vector Quantized Variational Autoencoder for Video Prediction
Authors:
Mohammad Adiban,
Kalin Stefanov,
Sabato Marco Siniscalchi,
Giampiero Salvi
Abstract:
We address the video prediction task by putting forth a novel model that combines (i) our recently proposed hierarchical residual vector quantized variational autoencoder (HR-VQVAE), and (ii) a novel spatiotemporal PixelCNN (ST-PixelCNN). We refer to this approach as a sequential hierarchical residual learning vector quantized variational autoencoder (S-HR-VQVAE). By leveraging the intrinsic capab…
▽ More
We address the video prediction task by putting forth a novel model that combines (i) our recently proposed hierarchical residual vector quantized variational autoencoder (HR-VQVAE), and (ii) a novel spatiotemporal PixelCNN (ST-PixelCNN). We refer to this approach as a sequential hierarchical residual learning vector quantized variational autoencoder (S-HR-VQVAE). By leveraging the intrinsic capabilities of HR-VQVAE at modeling still images with a parsimonious representation, combined with the ST-PixelCNN's ability at handling spatiotemporal information, S-HR-VQVAE can better deal with chief challenges in video prediction. These include learning spatiotemporal information, handling high dimensional data, combating blurry prediction, and implicit modeling of physical characteristics. Extensive experimental results on the KTH Human Action and Moving-MNIST tasks demonstrate that our model compares favorably against top video prediction techniques both in quantitative and qualitative evaluations despite a much smaller model size. Finally, we boost S-HR-VQVAE by proposing a novel training method to jointly estimate the HR-VQVAE and ST-PixelCNN parameters.
△ Less
Submitted 11 June, 2024; v1 submitted 13 July, 2023;
originally announced July 2023.
-
Glitch in the Matrix: A Large Scale Benchmark for Content Driven Audio-Visual Forgery Detection and Localization
Authors:
Zhixi Cai,
Shreya Ghosh,
Abhinav Dhall,
Tom Gedeon,
Kalin Stefanov,
Munawar Hayat
Abstract:
Most deepfake detection methods focus on detecting spatial and/or spatio-temporal changes in facial attributes and are centered around the binary classification task of detecting whether a video is real or fake. This is because available benchmark datasets contain mostly visual-only modifications present in the entirety of the video. However, a sophisticated deepfake may include small segments of…
▽ More
Most deepfake detection methods focus on detecting spatial and/or spatio-temporal changes in facial attributes and are centered around the binary classification task of detecting whether a video is real or fake. This is because available benchmark datasets contain mostly visual-only modifications present in the entirety of the video. However, a sophisticated deepfake may include small segments of audio or audio-visual manipulations that can completely change the meaning of the video content. To addresses this gap, we propose and benchmark a new dataset, Localized Audio Visual DeepFake (LAV-DF), consisting of strategic content-driven audio, visual and audio-visual manipulations. The proposed baseline method, Boundary Aware Temporal Forgery Detection (BA-TFD), is a 3D Convolutional Neural Network-based architecture which effectively captures multimodal manipulations. We further improve (i.e. BA-TFD+) the baseline method by replacing the backbone with a Multiscale Vision Transformer and guide the training process with contrastive, frame classification, boundary matching and multimodal boundary matching loss functions. The quantitative analysis demonstrates the superiority of BA-TFD+ on temporal forgery localization and deepfake detection tasks using several benchmark datasets including our newly proposed dataset. The dataset, models and code are available at https://github.com/ControlNet/LAV-DF.
△ Less
Submitted 16 July, 2023; v1 submitted 3 May, 2023;
originally announced May 2023.
-
Tilted discs in six poorly studied cataclysmic variables
Authors:
Stefan Y. Stefanov,
Atanas K. Stefanov
Abstract:
In this work, we search for negative superhumps (nSHs) in poorly studied cataclysmic variables using TESS data. We find three eclipsing binaries with nSH signatures: HBHA 4204-09, Gaia DR3 5931071148325476992, and SDSS J090113.51+144704.6. The last one exhibits IW And-like behaviour in archival ZTF data, and appears to have shallow, grazing eclipses. In addition, we detect nSH signatures in two no…
▽ More
In this work, we search for negative superhumps (nSHs) in poorly studied cataclysmic variables using TESS data. We find three eclipsing binaries with nSH signatures: HBHA 4204-09, Gaia DR3 5931071148325476992, and SDSS J090113.51+144704.6. The last one exhibits IW And-like behaviour in archival ZTF data, and appears to have shallow, grazing eclipses. In addition, we detect nSH signatures in two non-eclipsing systems: KQ Mon and Gaia DR3 4684361817175293440, by identifying the orbital period from the superorbital-dependent irradiation of the secondary. We discover nSH signatures in one more system, [PK2008] HalphaJ103959, by using an orbital period from another work. An improved mass ratio - nSH deficit relation $q(\varepsilon_-)$ is suggested by us, which agrees with independent measurements on nova-like variables. With this relation, we estimate the mass ratios of all systems in our sample, and determine the orbital inclinations for the three that are eclipsing. All systems with discovered nSHs in this work are excellent targets for follow-up spectroscopic studies.
△ Less
Submitted 20 January, 2023;
originally announced January 2023.
-
MARLIN: Masked Autoencoder for facial video Representation LearnINg
Authors:
Zhixi Cai,
Shreya Ghosh,
Kalin Stefanov,
Abhinav Dhall,
Jianfei Cai,
Hamid Rezatofighi,
Reza Haffari,
Munawar Hayat
Abstract:
This paper proposes a self-supervised approach to learn universal facial representations from videos, that can transfer across a variety of facial analysis tasks such as Facial Attribute Recognition (FAR), Facial Expression Recognition (FER), DeepFake Detection (DFD), and Lip Synchronization (LS). Our proposed framework, named MARLIN, is a facial video masked autoencoder, that learns highly robust…
▽ More
This paper proposes a self-supervised approach to learn universal facial representations from videos, that can transfer across a variety of facial analysis tasks such as Facial Attribute Recognition (FAR), Facial Expression Recognition (FER), DeepFake Detection (DFD), and Lip Synchronization (LS). Our proposed framework, named MARLIN, is a facial video masked autoencoder, that learns highly robust and generic facial embeddings from abundantly available non-annotated web crawled facial videos. As a challenging auxiliary task, MARLIN reconstructs the spatio-temporal details of the face from the densely masked facial regions which mainly include eyes, nose, mouth, lips, and skin to capture local and global aspects that in turn help in encoding generic and transferable features. Through a variety of experiments on diverse downstream tasks, we demonstrate MARLIN to be an excellent facial video encoder as well as feature extractor, that performs consistently well across a variety of downstream tasks including FAR (1.13% gain over supervised benchmark), FER (2.64% gain over unsupervised benchmark), DFD (1.86% gain over unsupervised benchmark), LS (29.36% gain for Frechet Inception Distance), and even in low data regime. Our code and models are available at https://github.com/ControlNet/MARLIN .
△ Less
Submitted 22 March, 2023; v1 submitted 12 November, 2022;
originally announced November 2022.
-
A Method to Achieve High Dynamic Range in a CMOS Image Sensor Using Interleaved Row Readout
Authors:
Thomas Wocial,
Konstantin D. Stefanov,
William E. Martin,
John R. Barnes,
Hugh R. A. Jones
Abstract:
We present a readout scheme for CMOS image sensors that can be used to achieve arbitrarily high dynamic range (HDR) in principle. The linear full well capacity (LFWC) in high signal regions was extended 50 times from 20 ke$^{-}$ to 984 ke$^{-}$ via an interlaced row-wise readout order, whilst the noise floor remained unchanged in low signal regions, resulting in a 34 dB increase in DR. The peak si…
▽ More
We present a readout scheme for CMOS image sensors that can be used to achieve arbitrarily high dynamic range (HDR) in principle. The linear full well capacity (LFWC) in high signal regions was extended 50 times from 20 ke$^{-}$ to 984 ke$^{-}$ via an interlaced row-wise readout order, whilst the noise floor remained unchanged in low signal regions, resulting in a 34 dB increase in DR. The peak signal-to-noise ratio (PSNR) is increased in a continuous fashion from 43 dB to 60 dB. This was achieved by summing user-selected rows which were read out multiple times. Centroiding uncertainties were lowered when template-fitting a projected pattern, compared to the standard readout scheme. Example applications are aimed at scientific imaging due to the linearity and PSNR increase.
△ Less
Submitted 10 October, 2022;
originally announced October 2022.
-
Hierarchical Residual Learning Based Vector Quantized Variational Autoencoder for Image Reconstruction and Generation
Authors:
Mohammad Adiban,
Kalin Stefanov,
Sabato Marco Siniscalchi,
Giampiero Salvi
Abstract:
We propose a multi-layer variational autoencoder method, we call HR-VQVAE, that learns hierarchical discrete representations of the data. By utilizing a novel objective function, each layer in HR-VQVAE learns a discrete representation of the residual from previous layers through a vector quantized encoder. Furthermore, the representations at each layer are hierarchically linked to those at previou…
▽ More
We propose a multi-layer variational autoencoder method, we call HR-VQVAE, that learns hierarchical discrete representations of the data. By utilizing a novel objective function, each layer in HR-VQVAE learns a discrete representation of the residual from previous layers through a vector quantized encoder. Furthermore, the representations at each layer are hierarchically linked to those at previous layers. We evaluate our method on the tasks of image reconstruction and generation. Experimental results demonstrate that the discrete representations learned by HR-VQVAE enable the decoder to reconstruct high-quality images with less distortion than the baseline methods, namely VQVAE and VQVAE-2. HR-VQVAE can also generate high-quality and diverse images that outperform state-of-the-art generative models, providing further verification of the efficiency of the learned representations. The hierarchical nature of HR-VQVAE i) reduces the decoding search time, making the method particularly suitable for high-load tasks and ii) allows to increase the codebook size without incurring the codebook collapse problem.
△ Less
Submitted 9 August, 2022;
originally announced August 2022.
-
Visual Representations of Physiological Signals for Fake Video Detection
Authors:
Kalin Stefanov,
Bhawna Paliwal,
Abhinav Dhall
Abstract:
Realistic fake videos are a potential tool for spreading harmful misinformation given our increasing online presence and information intake. This paper presents a multimodal learning-based method for detection of real and fake videos. The method combines information from three modalities - audio, video, and physiology. We investigate two strategies for combining the video and physiology modalities…
▽ More
Realistic fake videos are a potential tool for spreading harmful misinformation given our increasing online presence and information intake. This paper presents a multimodal learning-based method for detection of real and fake videos. The method combines information from three modalities - audio, video, and physiology. We investigate two strategies for combining the video and physiology modalities, either by augmenting the video with information from the physiology or by novelly learning the fusion of those two modalities with a proposed Graph Convolutional Network architecture. Both strategies for combining the two modalities rely on a novel method for generation of visual representations of physiological signals. The detection of real and fake videos is then based on the dissimilarity between the audio and modified video modalities. The proposed method is evaluated on two benchmark datasets and the results show significant increase in detection performance compared to previous methods.
△ Less
Submitted 18 July, 2022;
originally announced July 2022.
-
Do You Really Mean That? Content Driven Audio-Visual Deepfake Dataset and Multimodal Method for Temporal Forgery Localization
Authors:
Zhixi Cai,
Kalin Stefanov,
Abhinav Dhall,
Munawar Hayat
Abstract:
Due to its high societal impact, deepfake detection is getting active attention in the computer vision community. Most deepfake detection methods rely on identity, facial attributes, and adversarial perturbation-based spatio-temporal modifications at the whole video or random locations while kee** the meaning of the content intact. However, a sophisticated deepfake may contain only a small segme…
▽ More
Due to its high societal impact, deepfake detection is getting active attention in the computer vision community. Most deepfake detection methods rely on identity, facial attributes, and adversarial perturbation-based spatio-temporal modifications at the whole video or random locations while kee** the meaning of the content intact. However, a sophisticated deepfake may contain only a small segment of video/audio manipulation, through which the meaning of the content can be, for example, completely inverted from a sentiment perspective. We introduce a content-driven audio-visual deepfake dataset, termed Localized Audio Visual DeepFake (LAV-DF), explicitly designed for the task of learning temporal forgery localization. Specifically, the content-driven audio-visual manipulations are performed strategically to change the sentiment polarity of the whole video. Our baseline method for benchmarking the proposed dataset is a 3DCNN model, termed as Boundary Aware Temporal Forgery Detection (BA-TFD), which is guided via contrastive, boundary matching, and frame classification loss functions. Our extensive quantitative and qualitative analysis demonstrates the proposed method's strong performance for temporal forgery localization and deepfake detection tasks.
△ Less
Submitted 3 May, 2023; v1 submitted 13 April, 2022;
originally announced April 2022.
-
Simulations and Design of a Single-Photon CMOS Imaging Pixel Using Multiple Non-Destructive Signal Sampling
Authors:
Konstantin D. Stefanov,
Martin Prest,
Mark Downing,
Elizabeth George,
Naidu Bezawada,
Andrew D. Holland
Abstract:
A single-photon CMOS image sensor design based on pinned photodiode (PPD) with multiple charge transfers and sampling is described. In the proposed pixel architecture, the photogenerated signal is sampled non-destructively multiple times and the results are averaged. Each signal measurement is statistically independent and by averaging the electronic readout noise is reduced to a level where singl…
▽ More
A single-photon CMOS image sensor design based on pinned photodiode (PPD) with multiple charge transfers and sampling is described. In the proposed pixel architecture, the photogenerated signal is sampled non-destructively multiple times and the results are averaged. Each signal measurement is statistically independent and by averaging the electronic readout noise is reduced to a level where single photons can be distinguished reliably. A pixel design using this method has been simulated in TCAD and several layouts have been generated for a 180 nm CMOS image sensor process. Using simulations, the noise performance of the pixel has been determined as a function of the number of samples, sense node capacitance, sampling rate, and transistor characteristics. The strengths and the limitations of the proposed design are discussed in detail, including the trade-off between noise performance and readout rate and the impact of charge transfer inefficiency. The projected performance of our first prototype device indicates that single-photon imaging is within reach and could enable ground-breaking performance in many scientific and industrial imaging applications.
△ Less
Submitted 6 April, 2020;
originally announced April 2020.
-
GravityCam: Wide-field Imaging Surveys in the Visible from the Ground
Authors:
C. Mackay,
M. Dominik,
I. A. Steele,
C. Snodgrass,
U. G. Jørgensen,
J. Skottfelt,
K. Stefanov,
B. Carry,
F. Braga-Ribas,
A. Doressoundiram,
V. D. Ivanov,
P. Gandhi,
D. F. Evans,
M. Hundertmark,
S. Serjeant,
S. Ortolani
Abstract:
GravityCam is a new concept of ground-based imaging instrument capable of delivering significantly sharper images from the ground than is normally possible without adaptive optics. Advances in optical and near infrared imaging technologies allow images to be acquired at high speed without significant noise penalty. Aligning these images before they are combined can yield a 3-5 fold improvement in…
▽ More
GravityCam is a new concept of ground-based imaging instrument capable of delivering significantly sharper images from the ground than is normally possible without adaptive optics. Advances in optical and near infrared imaging technologies allow images to be acquired at high speed without significant noise penalty. Aligning these images before they are combined can yield a 3-5 fold improvement in image resolution. By using arrays of such detectors, survey fields may be as wide as the telescope optics allows. We describe the instrument and detail its application to accelerate greatly the rate of detection of Earth size planets by gravitational microlensing. GravityCam will improve substantially the quality of weak shear studies of dark matter distribution in distant clusters of galaxies. An extensive microlensing survey will also provide a vast dataset for asteroseismology studies, and GravityCam promises to generate a unique data set on the population of the Kuiper belt and possibly the Oort cloud.
△ Less
Submitted 18 November, 2019;
originally announced November 2019.
-
Webcam-based Eye Gaze Tracking under Natural Head Movement
Authors:
Kalin Stefanov
Abstract:
This manuscript investigates and proposes a visual gaze tracker that tackles the problem using only an ordinary web camera and no prior knowledge in any sense (scene set-up, camera intrinsic and/or extrinsic parameters). The tracker we propose is based on the observation that our desire to grant the freedom of natural head movement to the user requires 3D modeling of the scene set-up. Although, us…
▽ More
This manuscript investigates and proposes a visual gaze tracker that tackles the problem using only an ordinary web camera and no prior knowledge in any sense (scene set-up, camera intrinsic and/or extrinsic parameters). The tracker we propose is based on the observation that our desire to grant the freedom of natural head movement to the user requires 3D modeling of the scene set-up. Although, using a single low resolution web camera bounds us in dimensions (no depth can be recovered), we propose ways to cope with this drawback and model the scene in front of the user. We tackle this three-dimensional problem by realizing that it can be viewed as series of two-dimensional special cases. Then, we propose a procedure that treats each movement of the user's head as a special two-dimensional case, hence reducing the complexity of the problem back to two dimensions. Furthermore, the proposed tracker is calibration free and discards this tedious part of all previously mentioned trackers.
Experimental results show that the proposed tracker achieves good results, given the restrictions on it. We can report that the tracker commits a mean error of (56.95, 70.82) pixels in x and y direction, respectively, when the user's head is as static as possible (no chin-rests are used). Furthermore, we can report that the proposed tracker commits a mean error of (87.18, 103.86) pixels in x and y direction, respectively, under natural head movement.
△ Less
Submitted 29 March, 2018;
originally announced March 2018.
-
Self-Supervised Vision-Based Detection of the Active Speaker as Support for Socially-Aware Language Acquisition
Authors:
Kalin Stefanov,
Jonas Beskow,
Giampiero Salvi
Abstract:
This paper presents a self-supervised method for visual detection of the active speaker in a multi-person spoken interaction scenario. Active speaker detection is a fundamental prerequisite for any artificial cognitive system attempting to acquire language in social settings. The proposed method is intended to complement the acoustic detection of the active speaker, thus improving the system robus…
▽ More
This paper presents a self-supervised method for visual detection of the active speaker in a multi-person spoken interaction scenario. Active speaker detection is a fundamental prerequisite for any artificial cognitive system attempting to acquire language in social settings. The proposed method is intended to complement the acoustic detection of the active speaker, thus improving the system robustness in noisy conditions. The method can detect an arbitrary number of possibly overlap** active speakers based exclusively on visual information about their face. Furthermore, the method does not rely on external annotations, thus complying with cognitive development. Instead, the method uses information from the auditory modality to support learning in the visual domain. This paper reports an extensive evaluation of the proposed method using a large multi-person face-to-face interaction dataset. The results show good performance in a speaker dependent setting. However, in a speaker independent setting the proposed method yields a significantly lower performance. We believe that the proposed method represents an essential component of any artificial cognitive system or robotic platform engaging in social interactions.
△ Less
Submitted 18 July, 2019; v1 submitted 24 November, 2017;
originally announced November 2017.
-
GravityCam: Wide-Field High-Resolution High-Cadence Imaging Surveys in the Visible from the Ground
Authors:
C. Mackay,
M. Dominik,
I. A. Steele,
C. Snodgrass,
U. G. Jørgensen,
J. Skottfelt,
K. Stefanov,
B. Carry,
F. Braga-Ribas,
A. Doressoundiram,
V. D. Ivanov,
P. Gandhi,
D. F. Evans,
M. Hundertmark,
S. Serjeant,
S. Ortolani
Abstract:
GravityCam is a new concept of ground-based imaging instrument capable of delivering significantly sharper images from the ground than is normally possible without adaptive optics. Advances in optical and near infrared imaging technologies allow images to be acquired at high speed without significant noise penalty. Aligning these images before they are combined can yield a 2.5 to 3 fold improvemen…
▽ More
GravityCam is a new concept of ground-based imaging instrument capable of delivering significantly sharper images from the ground than is normally possible without adaptive optics. Advances in optical and near infrared imaging technologies allow images to be acquired at high speed without significant noise penalty. Aligning these images before they are combined can yield a 2.5 to 3 fold improvement in image resolution. By using arrays of such detectors, survey fields may be as wide as the telescope optics allows. Consequently, GravityCam enables both wide-field high-resolution imaging and high-speed photometry. We describe the instrument and detail its application to provide demographics of planets and satellites down to Lunar mass (or even below) across the Milky Way. GravityCam is also suited to improve the quality of weak shear studies of dark matter distribution in distant clusters of galaxies and multiwavelength follow-ups of background sources that are strongly lensed by galaxy clusters. The photometric data arising from an extensive microlensing survey will also be useful for asteroseismology studies, while GravityCam can be used to monitor fast multiwavelength flaring in accreting compact objects, and promises to generate a unique data set on the population of the Kuiper belt and possibly the Oort cloud.
△ Less
Submitted 25 September, 2018; v1 submitted 1 September, 2017;
originally announced September 2017.
-
Design and performance of a CMOS study sensor for a binary readout electromagnetic calorimeter
Authors:
J. A. Ballin,
R. Coath,
J. P. Crooks,
P. D. Dauncey,
A. -M. Magnan,
Y. Mikami,
O. D. Miller,
M. Noy,
V. Rajovic,
M. Stanitzki,
K. D. Stefanov,
R. Turchetta,
M. Tyndel,
E. G. Villani,
N. K. Watson,
J. A. Wilson,
Z. Zhang
Abstract:
We present a study of a CMOS test sensor which has been designed, fabricated and characterised to investigate the parameters required for a binary readout electromagnetic calorimeter. The sensors were fabricated with several enhancements in addition to standard CMOS processing. Detailed simulations and experimental results of the performance of the sensor are presented. The sensor and pixels are s…
▽ More
We present a study of a CMOS test sensor which has been designed, fabricated and characterised to investigate the parameters required for a binary readout electromagnetic calorimeter. The sensors were fabricated with several enhancements in addition to standard CMOS processing. Detailed simulations and experimental results of the performance of the sensor are presented. The sensor and pixels are shown to behave in accordance with expectations and the processing enhancements are found to be essential to achieve the performance required.
△ Less
Submitted 4 May, 2011; v1 submitted 22 March, 2011;
originally announced March 2011.
-
ISIS2: Pixel Sensor with Local Charge Storage for ILC Vertex Detector
Authors:
Yiming Li,
Chris Damerell,
Rui Gao,
Rhorry Gauld,
Jaya John John,
Peter Murray,
Andrei Nomerotski,
Konstantin Stefanov,
Steve Thomas,
Helena Wilding,
Zhige Zhang
Abstract:
ISIS (In-situ Storage Imaging Sensor) is a novel CMOS sensor with multiple charge storage capability developed for the ILC vertex detector by the Linear Collider Flavour Identification (LCFI) collaboration. This paper reports test results for ISIS2, the second generation of ISIS sensors implemented in a 0.18 micron CMOS process. The local charge storage and charge transfer were unambiguously demon…
▽ More
ISIS (In-situ Storage Imaging Sensor) is a novel CMOS sensor with multiple charge storage capability developed for the ILC vertex detector by the Linear Collider Flavour Identification (LCFI) collaboration. This paper reports test results for ISIS2, the second generation of ISIS sensors implemented in a 0.18 micron CMOS process. The local charge storage and charge transfer were unambiguously demonstrated.
△ Less
Submitted 14 July, 2010; v1 submitted 18 June, 2010;
originally announced June 2010.
-
Comparison of Measurements of Charge Transfer Inefficiencies in a CCD with High-Speed Column Parallel Readout
Authors:
Andre Sopczak,
Salim Aoulmit,
Khaled Bekhouche,
Chris Bowdery,
Craig Buttar,
Chris Damerell,
Dahmane Djendaoui,
Lakhdar Dehimi,
Rui Gao,
Tim Greenshaw,
Michal Koziel,
Dzmitry Maneuski,
Andrei Nomerotski,
Nouredine Sengouga,
Konstantin Stefanov,
Tuomo Tikkanen,
Tim Woolliscroft,
Steve Worm,
Zhige Zhang
Abstract:
Charge Coupled Devices (CCDs) have been successfully used in several high energy physics experiments over the past two decades. Their high spatial resolution and thin sensitive layers make them an excellent tool for studying short-lived particles. The Linear Collider Flavour Identification (LCFI) Collaboration has been develo** Column-Parallel CCDs for the vertex detector of a future Linear Co…
▽ More
Charge Coupled Devices (CCDs) have been successfully used in several high energy physics experiments over the past two decades. Their high spatial resolution and thin sensitive layers make them an excellent tool for studying short-lived particles. The Linear Collider Flavour Identification (LCFI) Collaboration has been develo** Column-Parallel CCDs for the vertex detector of a future Linear Collider which can be read out many times faster than standard CCDs. The most recent studies are of devices designed to reduce both the CCD's intergate capacitance and the clock voltages necessary to drive it. A comparative study of measured Charge Transfer Inefficiency values between our previous and new results for a range of operating temperatures is presented.
△ Less
Submitted 30 November, 2009;
originally announced November 2009.
-
The LCFIVertex package: vertexing, flavour tagging and vertex charge reconstruction with an ILC vertex detector
Authors:
LCFI Collaboration,
David Bailey,
Erik Devetak,
Mark Grimes,
Kristian Harder,
Sonja Hillert,
David Jackson,
Talini Pinto Jayawardena,
Ben Jeffery,
Tomas Lastovicka,
Clare Lynch,
Victoria Martin,
Roberval Walsh,
Phil Allport,
Yambazi Banda,
Craig Buttar,
Alexandre Cheplakov,
David Cussans,
Chris Damerell,
Nicolo de Groot,
Johan Fopma,
Brian Foster,
Senerath Galagedera,
Rui Gao,
Anthony Gillman
, et al. (36 additional authors not shown)
Abstract:
The precision measurements envisaged at the International Linear Collider (ILC) depend on excellent instrumentation and reconstruction software. The correct identification of heavy flavour jets, placing unprecedented requirements on the quality of the vertex detector, will be central for the ILC programme. This paper describes the LCFIVertex software, which provides tools for vertex finding and…
▽ More
The precision measurements envisaged at the International Linear Collider (ILC) depend on excellent instrumentation and reconstruction software. The correct identification of heavy flavour jets, placing unprecedented requirements on the quality of the vertex detector, will be central for the ILC programme. This paper describes the LCFIVertex software, which provides tools for vertex finding and for identification of the flavour and charge of the leading hadron in heavy flavour jets. These tools are essential for the ongoing optimisation of the vertex detector design for linear colliders such as the ILC. The paper describes the algorithms implemented in the LCFIVertex package, as well as the scope of the code and its performance for a typical vertex detector design.
△ Less
Submitted 20 August, 2009;
originally announced August 2009.
-
A digital ECAL based on MAPS
Authors:
J. A. Ballin,
P. D. Dauncey,
A. -M. Magnan,
M. Noy,
Y. Mikami,
O. Miller,
V. Rajovic,
N. K. Watson,
J. A. Wilson,
J. P. Crooks,
M. Stanitzki,
K. D. Stefanov,
R. Turchetta,
M. Tyndel,
E. G. Villani
Abstract:
Progress is reported on the development and testing of Monolithic Active Pixel Sensors (MAPS) for a Si-W ECAL for the ILC. Using laser and source setups, a first version of the sensor has been characterised through measurements of the absolute gain calibration, noise and pedestal. The pixel-to-pixel gain spread is 10%. Charge diffusion has been measured and found to be compatible with simulation…
▽ More
Progress is reported on the development and testing of Monolithic Active Pixel Sensors (MAPS) for a Si-W ECAL for the ILC. Using laser and source setups, a first version of the sensor has been characterised through measurements of the absolute gain calibration, noise and pedestal. The pixel-to-pixel gain spread is 10%. Charge diffusion has been measured and found to be compatible with simulation results. The charge collected by a single pixel varies from 50% to 20% depending on where it is generated. After adding detector effects to the Geant4 simulation of an ILC-like ECAL, using the measured parameters, the energy resolution is found to be 35% higher than the ideal resolution, but is still lower than the resolution obtained for an equivalent analogue ECAL.
△ Less
Submitted 28 January, 2009;
originally announced January 2009.
-
Modeling of Charge Transfer Inefficiency in a CCD with High Speed Column Parallel Readout
Authors:
Andre Sopczak,
Salim Aoulmit,
Khaled Bekhouche,
Chris Bowdery,
Craig Buttar,
Chris Damerell,
Dahmane Djendaoui,
Lakhdar Dehimi,
Tim Greenshaw,
Michal Koziel,
Dzmitry Maneuski,
Andrei Nomerotski,
Konstantin Stefanov,
Tuomo Tikkanen,
Tim Woolliscroft,
Steve Worm
Abstract:
Charge Coupled Devices (CCDs) have been successfully used in several high energy physics experiments over the past two decades. Their high spatial resolution and thin sensitive layers make them an excellent tool for studying short-lived particles. The Linear Collider Flavour Identification (LCFI) collaboration is develo** Column-Parallel CCDs (CPCCDs) for the vertex detector of a future Linear…
▽ More
Charge Coupled Devices (CCDs) have been successfully used in several high energy physics experiments over the past two decades. Their high spatial resolution and thin sensitive layers make them an excellent tool for studying short-lived particles. The Linear Collider Flavour Identification (LCFI) collaboration is develo** Column-Parallel CCDs (CPCCDs) for the vertex detector of a future Linear Collider. The CPCCDs can be read out many times faster than standard CCDs, significantly increasing their operating speed. An Analytic Model has been developed for the determination of the charge transfer inefficiency (CTI) of a CPCCD. The CTI values determined with the Analytic Model agree largely with those from a full TCAD simulation. The Analytic Model allows efficient study of the variation of the CTI on parameters like readout frequency, operating temperature and occupancy.
△ Less
Submitted 17 November, 2008;
originally announced November 2008.
-
Measurements of Charge Transfer Inefficiency in a CCD with High-Speed Column Parallel Readout
Authors:
Andre Sopczak,
Khaled Bekhouche,
Chris Damerell,
Tim Greenshaw,
Michal Koziel,
Konstantin Stefanov,
Tuomo Tikkanen,
Tim Woolliscroft,
Steve Worm
Abstract:
Charge Coupled Devices (CCDs) have been successfully used in several high energy physics experiments over the past two decades. Their high spatial resolution and thin sensitive layers make them an excellent tool for studying short-lived particles. The Linear Collider Flavour Identification (LCFI) collaboration is develo** Column-Parallel CCDs (CPCCDs) for the vertex detector of a future Linear…
▽ More
Charge Coupled Devices (CCDs) have been successfully used in several high energy physics experiments over the past two decades. Their high spatial resolution and thin sensitive layers make them an excellent tool for studying short-lived particles. The Linear Collider Flavour Identification (LCFI) collaboration is develo** Column-Parallel CCDs (CPCCDs) for the vertex detector of a future Linear Collider. The CPCCDs can be read out many times faster than standard CCDs, significantly increasing their operating speed. A test stand for measuring the charge transfer inefficiency (CTI) of a prototype CPCCD has been set up. Studies of the CTI have been performed at a range of readout frequencies and operating temperatures.
△ Less
Submitted 15 November, 2008;
originally announced November 2008.
-
Monolithic Active Pixel Sensors (MAPS) in a quadruple well technology for nearly 100% fill factor and full CMOS pixels
Authors:
J. A. Ballin,
J. P. Crooks,
P. D. Dauncey,
A. -M. Magnan,
Y. Mikami,
O. D. Miller,
M. Noy,
V. Rajovic,
M. M. Stanitzki,
K. D. Stefanov,
R. Turchetta,
M. Tyndel,
E. G. Villani,
N. K. Watson,
J. A. Wilson
Abstract:
In this paper we present a novel, quadruple well process developed in a modern 0.18mu CMOS technology called INMAPS. On top of the standard process, we have added a deep P implant that can be used to form a deep P-well and provide screening of N-wells from the P-doped epitaxial layer. This prevents the collection of radiation-induced charge by unrelated N-wells, typically ones where PMOS transis…
▽ More
In this paper we present a novel, quadruple well process developed in a modern 0.18mu CMOS technology called INMAPS. On top of the standard process, we have added a deep P implant that can be used to form a deep P-well and provide screening of N-wells from the P-doped epitaxial layer. This prevents the collection of radiation-induced charge by unrelated N-wells, typically ones where PMOS transistors are integrated. The design of a sensor specifically tailored to a particle physics experiment is presented, where each 50mu pixel has over 150 PMOS and NMOS transistors. The sensor has been fabricated in the INMAPS process and first experimental evidence of the effectiveness of this process on charge collection is presented, showing a significant improvement in efficiency.
△ Less
Submitted 18 July, 2008;
originally announced July 2008.
-
A MAPS-based Digital Electromagnetic Calorimeter for the ILC
Authors:
J. A. Ballin,
P. D. Dauncey,
A. -M. Magnan,
M. Noy,
Y. Mikami,
O. Miller,
V. Rajović,
N. K. Watson,
J. A. Wilson,
J. P. Crooks,
M. Stanitzki,
K. D. Stefanov,
R. Turchetta,
M. Tyndel,
E. G. Villani
Abstract:
A novel design for a silicon-tungsten electromagnetic calorimeter is described, based on Monolithic Active Pixel Sensors (MAPS). A test sensor with a pixel size of 50x50 um2 has been fabricated in July 2007. The simulation of the physical sensor is done using a detailed three-dimensional charge spread algorithm. Physics studies of the sensor are done including a digitisation algorithm taking int…
▽ More
A novel design for a silicon-tungsten electromagnetic calorimeter is described, based on Monolithic Active Pixel Sensors (MAPS). A test sensor with a pixel size of 50x50 um2 has been fabricated in July 2007. The simulation of the physical sensor is done using a detailed three-dimensional charge spread algorithm. Physics studies of the sensor are done including a digitisation algorithm taking into account the charge sharing, charge collection efficiency, noise, and dead areas. The influence of the charge sharing effect is found to be important and hence needs to be measured precisely.
△ Less
Submitted 10 September, 2007;
originally announced September 2007.
-
Radiation Hardness of CCD Vertex Detectors for the ILC
Authors:
Andre Sopczak,
Khaled Bekhouche,
Chris Bowdery,
Chris Damerell,
Gavin Davies,
Lakhdar Dehimi,
Tim Greenshaw,
Michal Koziel,
Konstantin Stefanov,
Tim Woolliscroft,
Steve Worm
Abstract:
Results of detailed simulations of the charge transfer inefficiency of a prototype CCD chip are reported. The effect of radiation damage in a particle detector operating at a future accelerator is studied by examining two electron trap levels, 0.17 eV and 0.44 eV below the bottom of the conduction band. Good agreement is found between simulations using the ISE-TCAD DESSIS program and an analytic…
▽ More
Results of detailed simulations of the charge transfer inefficiency of a prototype CCD chip are reported. The effect of radiation damage in a particle detector operating at a future accelerator is studied by examining two electron trap levels, 0.17 eV and 0.44 eV below the bottom of the conduction band. Good agreement is found between simulations using the ISE-TCAD DESSIS program and an analytical model for the 0.17 eV level. Optimum operation is predicted to be at about 250 K where the effect of the traps is minimal which is approximately independent of readout frequency. This work has been carried out within the Linear Collider Flavour Identification (LCFI) collaboration in the context of the International Linear Collider (ILC) project.
△ Less
Submitted 28 November, 2006;
originally announced November 2006.