Showing 1–8 of 8 results for author: Kviatkovsky, I

Distilling the Knowledge in Data Pruning

Authors: Emanuel Ben-Baruch, Adam Botach, Igor Kviatkovsky, Manoj Aggarwal, Gérard Medioni

Abstract: With the increasing size of datasets used for training neural networks, data pruning becomes an attractive field of research. However, most current data pruning algorithms are limited in their ability to preserve accuracy compared to models trained on the full data, especially in high pruning regimes. In this paper we explore the application of data pruning while incorporating knowledge distillati… ▽ More With the increasing size of datasets used for training neural networks, data pruning becomes an attractive field of research. However, most current data pruning algorithms are limited in their ability to preserve accuracy compared to models trained on the full data, especially in high pruning regimes. In this paper we explore the application of data pruning while incorporating knowledge distillation (KD) when training on a pruned subset. That is, rather than relying solely on ground-truth labels, we also use the soft predictions from a teacher network pre-trained on the complete data. By integrating KD into training, we demonstrate significant improvement across datasets, pruning methods, and on all pruning fractions. We first establish a theoretical motivation for employing self-distillation to improve training on pruned data. Then, we empirically make a compelling and highly practical observation: using KD, simple random pruning is comparable or superior to sophisticated pruning methods across all pruning regimes. On ImageNet for example, we achieve superior accuracy despite training on a random subset of only 50% of the data. Additionally, we demonstrate a crucial connection between the pruning factor and the optimal knowledge distillation weight. This helps mitigate the impact of samples with noisy labels and low-quality images retained by typical pruning algorithms. Finally, we make an intriguing observation: when using lower pruning fractions, larger teachers lead to accuracy degradation, while surprisingly, employing teachers with a smaller capacity than the student's may improve results. Our code will be made available. △ Less

Submitted 12 March, 2024; originally announced March 2024.

FPGAN-Control: A Controllable Fingerprint Generator for Training with Synthetic Data

Authors: Alon Shoshan, Nadav Bhonker, Emanuel Ben Baruch, Ori Nizan, Igor Kviatkovsky, Joshua Engelsma, Manoj Aggarwal, Gerard Medioni

Abstract: Training fingerprint recognition models using synthetic data has recently gained increased attention in the biometric community as it alleviates the dependency on sensitive personal data. Existing approaches for fingerprint generation are limited in their ability to generate diverse impressions of the same finger, a key property for providing effective data for training recognition models. To addr… ▽ More Training fingerprint recognition models using synthetic data has recently gained increased attention in the biometric community as it alleviates the dependency on sensitive personal data. Existing approaches for fingerprint generation are limited in their ability to generate diverse impressions of the same finger, a key property for providing effective data for training recognition models. To address this gap, we present FPGAN-Control, an identity preserving image generation framework which enables control over the fingerprint's image appearance (e.g., fingerprint type, acquisition device, pressure level) of generated fingerprints. We introduce a novel appearance loss that encourages disentanglement between the fingerprint's identity and appearance properties. In our experiments, we used the publicly available NIST SD302 (N2N) dataset for training the FPGAN-Control model. We demonstrate the merits of FPGAN-Control, both quantitatively and qualitatively, in terms of identity preservation level, degree of appearance control, and low synthetic-to-real domain gap. Finally, training recognition models using only synthetic datasets generated by FPGAN-Control lead to recognition accuracies that are on par or even surpass models trained using real data. To the best of our knowledge, this is the first work to demonstrate this. △ Less

Submitted 29 October, 2023; originally announced October 2023.

Asymmetric Image Retrieval with Cross Model Compatible Ensembles

Authors: Ori Linial, Alon Shoshan, Nadav Bhonker, Elad Hirsch, Lior Zamir, Igor Kviatkovsky, Gerard Medioni

Abstract: The asymmetrical retrieval setting is a well suited solution for resource constrained applications such as face recognition and image retrieval. In this setting, a large model is used for indexing the gallery while a lightweight model is used for querying. The key principle in such systems is ensuring that both models share the same embedding space. Most methods in this domain are based on knowled… ▽ More The asymmetrical retrieval setting is a well suited solution for resource constrained applications such as face recognition and image retrieval. In this setting, a large model is used for indexing the gallery while a lightweight model is used for querying. The key principle in such systems is ensuring that both models share the same embedding space. Most methods in this domain are based on knowledge distillation. While useful, they suffer from several drawbacks: they are upper-bounded by the performance of the single best model found and cannot be extended to use an ensemble of models in a straightforward manner. In this paper we present an approach that does not rely on knowledge distillation, rather it utilizes embedding transformation models. This allows the use of N independently trained and diverse gallery models (e.g., trained on different datasets or having a different architecture) and a single query model. As a result, we improve the overall accuracy beyond that of any single model while maintaining a low computational budget for querying. Additionally, we propose a gallery image rejection method that utilizes the diversity between multiple transformed embeddings to estimate the uncertainty of gallery images. △ Less

Submitted 29 October, 2023; v1 submitted 30 March, 2023; originally announced March 2023.

Mid-Infrared Microscopy via Position Correlations of Undetected Photons

Authors: Inna Kviatkovsky, Helen M. Chrzanowski, Sven Ramelow

Abstract: Quantum imaging with undetected photons (QIUP) has recently emerged as a new powerful imaging tool. Exploiting the spatial entanglement of photon pairs, it allows decoupling of the sensing and detection wavelengths, facilitating imaging in otherwise challenging spectral regions with mature silicon-based detection technology. All existing implementations of QIUP have so far utilised the momentum co… ▽ More Quantum imaging with undetected photons (QIUP) has recently emerged as a new powerful imaging tool. Exploiting the spatial entanglement of photon pairs, it allows decoupling of the sensing and detection wavelengths, facilitating imaging in otherwise challenging spectral regions with mature silicon-based detection technology. All existing implementations of QIUP have so far utilised the momentum correlations within the biphoton state. Here, for the first time, we implement and examine theoretically and numerically the complementary scenario - utilising the tight position correlations formed within photon pair at birth. This image plane arrangement facilitates high resolution imaging with comparative experimental ease, and we experimentally show resolutions below 10 $μ$m at a sensing wavelength of 3.7 $μ$m. Moreover, imaging a slice of mouse heart tissue at the mid-IR to reveal morphological features on the cellular level, we further demonstrate the viability of the technique for the life sciences. These results offer new perspectives on the capabilities of QIUP for label-free wide-field microscopy, enabling new real-world applications in biomedical as well as industrial imaging at inaccessible wavelengths. △ Less

Submitted 11 June, 2021; originally announced June 2021.

Synthetic Data for Model Selection

Authors: Alon Shoshan, Nadav Bhonker, Igor Kviatkovsky, Matan Fintz, Gerard Medioni

Abstract: Recent breakthroughs in synthetic data generation approaches made it possible to produce highly photorealistic images which are hardly distinguishable from real ones. Furthermore, synthetic generation pipelines have the potential to generate an unlimited number of images. The combination of high photorealism and scale turn synthetic data into a promising candidate for improving various machine lea… ▽ More Recent breakthroughs in synthetic data generation approaches made it possible to produce highly photorealistic images which are hardly distinguishable from real ones. Furthermore, synthetic generation pipelines have the potential to generate an unlimited number of images. The combination of high photorealism and scale turn synthetic data into a promising candidate for improving various machine learning (ML) pipelines. Thus far, a large body of research in this field has focused on using synthetic images for training, by augmenting and enlarging training data. In contrast to using synthetic data for training, in this work we explore whether synthetic data can be beneficial for model selection. Considering the task of image classification, we demonstrate that when data is scarce, synthetic data can be used to replace the held out validation set, thus allowing to train on a larger dataset. We also introduce a novel method to calibrate the synthetic error estimation to fit that of the real domain. We show that such calibration significantly improves the usefulness of synthetic data for model selection. △ Less

Submitted 5 July, 2023; v1 submitted 3 May, 2021; originally announced May 2021.

GAN-Control: Explicitly Controllable GANs

Authors: Alon Shoshan, Nadav Bhonker, Igor Kviatkovsky, Gerard Medioni

Abstract: We present a framework for training GANs with explicit control over generated images. We are able to control the generated image by settings exact attributes such as age, pose, expression, etc. Most approaches for editing GAN-generated images achieve partial control by leveraging the latent space disentanglement properties, obtained implicitly after standard GAN training. Such methods are able to… ▽ More We present a framework for training GANs with explicit control over generated images. We are able to control the generated image by settings exact attributes such as age, pose, expression, etc. Most approaches for editing GAN-generated images achieve partial control by leveraging the latent space disentanglement properties, obtained implicitly after standard GAN training. Such methods are able to change the relative intensity of certain attributes, but not explicitly set their values. Recently proposed methods, designed for explicit control over human faces, harness morphable 3D face models to allow fine-grained control capabilities in GANs. Unlike these methods, our control is not constrained to morphable 3D face model parameters and is extendable beyond the domain of human faces. Using contrastive learning, we obtain GANs with an explicitly disentangled latent space. This disentanglement is utilized to train control-encoders map** human-interpretable inputs to suitable latent vectors, thus allowing explicit control. In the domain of human faces we demonstrate control over identity, age, pose, expression, hair color and illumination. We also demonstrate control capabilities of our framework in the domains of painted portraits and dog image generation. We demonstrate that our approach achieves state-of-the-art performance both qualitatively and quantitatively. △ Less

Submitted 3 October, 2021; v1 submitted 7 January, 2021; originally announced January 2021.

From Real to Synthetic and Back: Synthesizing Training Data for Multi-Person Scene Understanding

Authors: Igor Kviatkovsky, Nadav Bhonker, Gerard Medioni

Abstract: We present a method for synthesizing naturally looking images of multiple people interacting in a specific scenario. These images benefit from the advantages of synthetic data: being fully controllable and fully annotated with any type of standard or custom-defined ground truth. To reduce the synthetic-to-real domain gap, we introduce a pipeline consisting of the following steps: 1) we render scen… ▽ More We present a method for synthesizing naturally looking images of multiple people interacting in a specific scenario. These images benefit from the advantages of synthetic data: being fully controllable and fully annotated with any type of standard or custom-defined ground truth. To reduce the synthetic-to-real domain gap, we introduce a pipeline consisting of the following steps: 1) we render scenes in a context modeled after the real world, 2) we train a human parsing model on the synthetic images, 3) we use the model to estimate segmentation maps for real images, 4) we train a conditional generative adversarial network (cGAN) to learn the inverse map** -- from a segmentation map to a real image, and 5) given new synthetic segmentation maps, we use the cGAN to generate realistic images. An illustration of our pipeline is presented in Figure 2. We use the generated data to train a multi-task model on the challenging tasks of UV map** and dense depth estimation. We demonstrate the value of the data generation and the trained model, both quantitatively and qualitatively on the CMU Panoptic Dataset. △ Less

Submitted 3 June, 2020; originally announced June 2020.

Microscopy with undetected photons in the mid-infrared

Authors: Inna Kviatkovsky, Helen M. Chrzanowski, Ellen G. Avery, Hendrik Bartolomaeus, Sven Ramelow

Abstract: Owing to its capacity for unique (bio)-chemical specificity, microscopy withmid-IR illumination holds tremendous promise for a wide range of biomedical and industrial applications. The primary limitation, however, remains detection; with current mid-IR detection technology often marrying inferior technical capabilities with prohibitive costs. This has lead to approaches that shift detection towave… ▽ More Owing to its capacity for unique (bio)-chemical specificity, microscopy withmid-IR illumination holds tremendous promise for a wide range of biomedical and industrial applications. The primary limitation, however, remains detection; with current mid-IR detection technology often marrying inferior technical capabilities with prohibitive costs. This has lead to approaches that shift detection towavelengths into the visible regime, where vastly superior silicon-based cameratechnology is available. Here, we experimentally show how nonlinear interferometry with entangled light can provide a powerful tool for mid-IR microscopy, while only requiring near-infrared detection with a standard CMOS camera. In this proof-of-principle implementation, we demonstrate intensity imaging overa broad wavelength range covering 3.4-4.3um and demonstrate a spatial resolution of 35um for images containing 650 resolved elements. Moreover, we demonstrate our technique is fit for purpose, acquiring microscopic images of biological tissue samples in the mid-IR. These results open a new perspective for potential relevance of quantum imaging techniques in the life sciences. △ Less

Submitted 29 May, 2020; v1 submitted 14 February, 2020; originally announced February 2020.

Comments: back-to-back submission with arXiv:2002.05956, Anna V. Paterova, Sivakumar M. Maniam, Hongzhi Yang, Gianluca Grenci, and Leonid A. Krivitsky, "Hyperspectral Infrared Microscopy With Visible Light "