-
Integrating Visual and Semantic Similarity Using Hierarchies for Image Retrieval
Authors:
Aishwarya Venkataramanan,
Martin Laviale,
Cédric Pradalier
Abstract:
Most of the research in content-based image retrieval (CBIR) focus on develo** robust feature representations that can effectively retrieve instances from a database of images that are visually similar to a query. However, the retrieved images sometimes contain results that are not semantically related to the query. To address this, we propose a method for CBIR that captures both visual and sema…
▽ More
Most of the research in content-based image retrieval (CBIR) focus on develo** robust feature representations that can effectively retrieve instances from a database of images that are visually similar to a query. However, the retrieved images sometimes contain results that are not semantically related to the query. To address this, we propose a method for CBIR that captures both visual and semantic similarity using a visual hierarchy. The hierarchy is constructed by merging classes with overlap** features in the latent space of a deep neural network trained for classification, assuming that overlap** classes share high visual and semantic similarities. Finally, the constructed hierarchy is integrated into the distance calculation metric for similarity search. Experiments on standard datasets: CUB-200-2011 and CIFAR100, and a real-life use case using diatom microscopy images show that our method achieves superior performance compared to the existing methods on image retrieval.
△ Less
Submitted 16 August, 2023;
originally announced August 2023.
-
Gaussian Latent Representations for Uncertainty Estimation using Mahalanobis Distance in Deep Classifiers
Authors:
Aishwarya Venkataramanan,
Assia Benbihi,
Martin Laviale,
Cedric Pradalier
Abstract:
Recent works show that the data distribution in a network's latent space is useful for estimating classification uncertainty and detecting Out-of-distribution (OOD) samples. To obtain a well-regularized latent space that is conducive for uncertainty estimation, existing methods bring in significant changes to model architectures and training procedures. In this paper, we present a lightweight, fas…
▽ More
Recent works show that the data distribution in a network's latent space is useful for estimating classification uncertainty and detecting Out-of-distribution (OOD) samples. To obtain a well-regularized latent space that is conducive for uncertainty estimation, existing methods bring in significant changes to model architectures and training procedures. In this paper, we present a lightweight, fast, and high-performance regularization method for Mahalanobis distance-based uncertainty prediction, and that requires minimal changes to the network's architecture. To derive Gaussian latent representation favourable for Mahalanobis Distance calculation, we introduce a self-supervised representation learning method that separates in-class representations into multiple Gaussians. Classes with non-Gaussian representations are automatically identified and dynamically clustered into multiple new classes that are approximately Gaussian. Evaluation on standard OOD benchmarks shows that our method achieves state-of-the-art results on OOD detection with minimal inference time, and is very competitive on predictive probability calibration. Finally, we show the applicability of our method to a real-life computer vision use case on microorganism classification.
△ Less
Submitted 29 September, 2023; v1 submitted 23 May, 2023;
originally announced May 2023.
-
Tackling Inter-Class Similarity and Intra-Class Variance for Microscopic Image-based Classification
Authors:
Aishwarya Venkataramanan,
Martin Laviale,
Cécile Figus,
Philippe Usseglio-Polatera,
Cédric Pradalier
Abstract:
Automatic classification of aquatic microorganisms is based on the morphological features extracted from individual images. The current works on their classification do not consider the inter-class similarity and intra-class variance that causes misclassification. We are particularly interested in the case where variance within a class occurs due to discrete visual changes in microscopic images. I…
▽ More
Automatic classification of aquatic microorganisms is based on the morphological features extracted from individual images. The current works on their classification do not consider the inter-class similarity and intra-class variance that causes misclassification. We are particularly interested in the case where variance within a class occurs due to discrete visual changes in microscopic images. In this paper, we propose to account for it by partitioning the classes with high variance based on the visual features. Our algorithm automatically decides the optimal number of sub-classes to be created and consider each of them as a separate class for training. This way, the network learns finer-grained visual features. Our experiments on two databases of freshwater benthic diatoms and marine plankton show that our method can outperform the state-of-the-art approaches for classification of these aquatic microorganisms.
△ Less
Submitted 24 September, 2021;
originally announced September 2021.
