-
OMH: Structured Sparsity via Optimally Matched Hierarchy for Unsupervised Semantic Segmentation
Authors:
Baran Ozaydin,
Tong Zhang,
Deblina Bhattacharjee,
Sabine Süsstrunk,
Mathieu Salzmann
Abstract:
Unsupervised Semantic Segmentation (USS) involves segmenting images without relying on predefined labels, aiming to alleviate the burden of extensive human labeling. Existing methods utilize features generated by self-supervised models and specific priors for clustering. However, their clustering objectives are not involved in the optimization of the features during training. Additionally, due to…
▽ More
Unsupervised Semantic Segmentation (USS) involves segmenting images without relying on predefined labels, aiming to alleviate the burden of extensive human labeling. Existing methods utilize features generated by self-supervised models and specific priors for clustering. However, their clustering objectives are not involved in the optimization of the features during training. Additionally, due to the lack of clear class definitions in USS, the resulting segments may not align well with the clustering objective. In this paper, we introduce a novel approach called Optimally Matched Hierarchy (OMH) to simultaneously address the above issues. The core of our method lies in imposing structured sparsity on the feature space, which allows the features to encode information with different levels of granularity. The structure of this sparsity stems from our hierarchy (OMH). To achieve this, we learn a soft but sparse hierarchy among parallel clusters through Optimal Transport. Our OMH yields better unsupervised segmentation performance compared to existing USS methods. Our extensive experiments demonstrate the benefits of OMH when utilizing our differentiable paradigm. We will make our code publicly available.
△ Less
Submitted 5 April, 2024; v1 submitted 11 March, 2024;
originally announced March 2024.
-
DSI2I: Dense Style for Unpaired Image-to-Image Translation
Authors:
Baran Ozaydin,
Tong Zhang,
Sabine Süsstrunk,
Mathieu Salzmann
Abstract:
Unpaired exemplar-based image-to-image (UEI2I) translation aims to translate a source image to a target image domain with the style of a target image exemplar, without ground-truth input-translation pairs. Existing UEI2I methods represent style using one vector per image or rely on semantic supervision to define one style vector per object. Here, in contrast, we propose to represent style as a den…
▽ More
Unpaired exemplar-based image-to-image (UEI2I) translation aims to translate a source image to a target image domain with the style of a target image exemplar, without ground-truth input-translation pairs. Existing UEI2I methods represent style using one vector per image or rely on semantic supervision to define one style vector per object. Here, in contrast, we propose to represent style as a dense feature map, allowing for a finer-grained transfer to the source image without requiring any external semantic information. We then rely on perceptual and adversarial losses to disentangle our dense style and content representations. To stylize the source content with the exemplar style, we extract unsupervised cross-domain semantic correspondences and warp the exemplar style to the source content. We demonstrate the effectiveness of our method on four datasets using standard metrics together with a localized style metric we propose, which measures style similarity in a class-wise manner. Our results show that the translations produced by our approach are more diverse, preserve the source content better, and are closer to the exemplars when compared to the state-of-the-art methods. Project page: https://github.com/IVRL/dsi2i
△ Less
Submitted 1 May, 2024; v1 submitted 26 December, 2022;
originally announced December 2022.
-
GRAND-assisted Optimal Modulation
Authors:
Basak Ozaydin,
Muriel Médard,
Ken Duffy
Abstract:
Optimal modulation (OM) schemes for Gaussian channels with peak and average power constraints are known to require nonuniform probability distributions over signal points, which presents practical challenges. An established way to map uniform binary sources to non-uniform symbol distributions is to assign a different number of bits to different constellation points. Doing so, however, means that e…
▽ More
Optimal modulation (OM) schemes for Gaussian channels with peak and average power constraints are known to require nonuniform probability distributions over signal points, which presents practical challenges. An established way to map uniform binary sources to non-uniform symbol distributions is to assign a different number of bits to different constellation points. Doing so, however, means that erroneous demodulation at the receiver can lead to bit insertions or deletions that result in significant binary error propagation. In this paper, we introduce a light-weight variant of Guessing Random Additive Noise Decoding (GRAND) to resolve insertion and deletion errors at the receiver by using a simple padding scheme. Performance evaluation demonstrates that our approach results in an overall gain in demodulated bit-error-rate of over 2 dB Eb/N0 when compared to 128-Quadrature Amplitude Modulation (QAM). The GRAND-aided OM scheme outperforms coding with a low-density parity check code of the same average rate as that induced by our simple padding.
△ Less
Submitted 31 October, 2022; v1 submitted 28 October, 2022;
originally announced October 2022.
-
DSR: Towards Drone Image Super-Resolution
Authors:
Xiaoyu Lin,
Baran Ozaydin,
Vidit Vidit,
Majed El Helou,
Sabine Süsstrunk
Abstract:
Despite achieving remarkable progress in recent years, single-image super-resolution methods are developed with several limitations. Specifically, they are trained on fixed content domains with certain degradations (whether synthetic or real). The priors they learn are prone to overfitting the training configuration. Therefore, the generalization to novel domains such as drone top view data, and a…
▽ More
Despite achieving remarkable progress in recent years, single-image super-resolution methods are developed with several limitations. Specifically, they are trained on fixed content domains with certain degradations (whether synthetic or real). The priors they learn are prone to overfitting the training configuration. Therefore, the generalization to novel domains such as drone top view data, and across altitudes, is currently unknown. Nonetheless, pairing drones with proper image super-resolution is of great value. It would enable drones to fly higher covering larger fields of view, while maintaining a high image quality.
To answer these questions and pave the way towards drone image super-resolution, we explore this application with particular focus on the single-image case. We propose a novel drone image dataset, with scenes captured at low and high resolutions, and across a span of altitudes. Our results show that off-the-shelf state-of-the-art networks witness a significant drop in performance on this different domain. We additionally show that simple fine-tuning, and incorporating altitude awareness into the network's architecture, both improve the reconstruction performance.
△ Less
Submitted 25 August, 2022;
originally announced August 2022.
