-
Un-Mixing Test-Time Normalization Statistics: Combatting Label Temporal Correlation
Authors:
Devavrat Tomar,
Guillaume Vray,
Jean-Philippe Thiran,
Behzad Bozorgtabar
Abstract:
Recent test-time adaptation methods heavily rely on nuanced adjustments of batch normalization (BN) parameters. However, one critical assumption often goes overlooked: that of independently and identically distributed (i.i.d.) test batches with respect to unknown labels. This oversight leads to skewed BN statistics and undermines the reliability of the model under non-i.i.d. scenarios. To tackle t…
▽ More
Recent test-time adaptation methods heavily rely on nuanced adjustments of batch normalization (BN) parameters. However, one critical assumption often goes overlooked: that of independently and identically distributed (i.i.d.) test batches with respect to unknown labels. This oversight leads to skewed BN statistics and undermines the reliability of the model under non-i.i.d. scenarios. To tackle this challenge, this paper presents a novel method termed 'Un-Mixing Test-Time Normalization Statistics' (UnMix-TNS). Our method re-calibrates the statistics for each instance within a test batch by mixing it with multiple distinct statistics components, thus inherently simulating the i.i.d. scenario. The core of this method hinges on a distinctive online unmixing procedure that continuously updates these statistics components by incorporating the most similar instances from new test batches. Remarkably generic in its design, UnMix-TNS seamlessly integrates with a wide range of leading test-time adaptation methods and pre-trained architectures equipped with BN layers. Empirical evaluations corroborate the robustness of UnMix-TNS under varied scenarios-ranging from single to continual and mixed domain shifts, particularly excelling with temporally correlated test data and corrupted non-i.i.d. real-world streams. This adaptability is maintained even with very small batch sizes or single instances. Our results highlight UnMix-TNS's capacity to markedly enhance stability and performance across various benchmarks. Our code is publicly available at https://github.com/devavratTomar/unmixtns.
△ Less
Submitted 14 March, 2024; v1 submitted 16 January, 2024;
originally announced January 2024.
-
Distill-SODA: Distilling Self-Supervised Vision Transformer for Source-Free Open-Set Domain Adaptation in Computational Pathology
Authors:
Guillaume Vray,
Devavrat Tomar,
Jean-Philippe Thiran,
Behzad Bozorgtabar
Abstract:
Develo** computational pathology models is essential for reducing manual tissue ty** from whole slide images, transferring knowledge from the source domain to an unlabeled, shifted target domain, and identifying unseen categories. We propose a practical setting by addressing the above-mentioned challenges in one fell swoop, i.e., source-free open-set domain adaptation. Our methodology focuses…
▽ More
Develo** computational pathology models is essential for reducing manual tissue ty** from whole slide images, transferring knowledge from the source domain to an unlabeled, shifted target domain, and identifying unseen categories. We propose a practical setting by addressing the above-mentioned challenges in one fell swoop, i.e., source-free open-set domain adaptation. Our methodology focuses on adapting a pre-trained source model to an unlabeled target dataset and encompasses both closed-set and open-set classes. Beyond addressing the semantic shift of unknown classes, our framework also deals with a covariate shift, which manifests as variations in color appearance between source and target tissue samples. Our method hinges on distilling knowledge from a self-supervised vision transformer (ViT), drawing guidance from either robustly pre-trained transformer models or histopathology datasets, including those from the target domain. In pursuit of this, we introduce a novel style-based adversarial data augmentation, serving as hard positives for self-training a ViT, resulting in highly contextualized embeddings. Following this, we cluster semantically akin target images, with the source model offering weak pseudo-labels, albeit with uncertain confidence. To enhance this process, we present the closed-set affinity score (CSAS), aiming to correct the confidence levels of these pseudo-labels and to calculate weighted class prototypes within the contextualized embedding space. Our approach establishes itself as state-of-the-art across three public histopathological datasets for colorectal cancer assessment. Notably, our self-training method seamlessly integrates with open-set detection methods, resulting in enhanced performance in both closed-set and open-set recognition tasks.
△ Less
Submitted 16 January, 2024; v1 submitted 10 July, 2023;
originally announced July 2023.
-
TeSLA: Test-Time Self-Learning With Automatic Adversarial Augmentation
Authors:
Devavrat Tomar,
Guillaume Vray,
Behzad Bozorgtabar,
Jean-Philippe Thiran
Abstract:
Most recent test-time adaptation methods focus on only classification tasks, use specialized network architectures, destroy model calibration or rely on lightweight information from the source domain. To tackle these issues, this paper proposes a novel Test-time Self-Learning method with automatic Adversarial augmentation dubbed TeSLA for adapting a pre-trained source model to the unlabeled stream…
▽ More
Most recent test-time adaptation methods focus on only classification tasks, use specialized network architectures, destroy model calibration or rely on lightweight information from the source domain. To tackle these issues, this paper proposes a novel Test-time Self-Learning method with automatic Adversarial augmentation dubbed TeSLA for adapting a pre-trained source model to the unlabeled streaming test data. In contrast to conventional self-learning methods based on cross-entropy, we introduce a new test-time loss function through an implicitly tight connection with the mutual information and online knowledge distillation. Furthermore, we propose a learnable efficient adversarial augmentation module that further enhances online knowledge distillation by simulating high entropy augmented images. Our method achieves state-of-the-art classification and segmentation results on several benchmarks and types of domain shifts, particularly on challenging measurement shifts of medical images. TeSLA also benefits from several desirable properties compared to competing methods in terms of calibration, uncertainty metrics, insensitivity to model architectures, and source training strategies, all supported by extensive ablations. Our code and models are available on GitHub.
△ Less
Submitted 17 March, 2023;
originally announced March 2023.
-
Self-Supervised Generative Style Transfer for One-Shot Medical Image Segmentation
Authors:
Devavrat Tomar,
Behzad Bozorgtabar,
Manana Lortkipanidze,
Guillaume Vray,
Mohammad Saeed Rad,
Jean-Philippe Thiran
Abstract:
In medical image segmentation, supervised deep networks' success comes at the cost of requiring abundant labeled data. While asking domain experts to annotate only one or a few of the cohort's images is feasible, annotating all available images is impractical. This issue is further exacerbated when pre-trained deep networks are exposed to a new image dataset from an unfamiliar distribution. Using…
▽ More
In medical image segmentation, supervised deep networks' success comes at the cost of requiring abundant labeled data. While asking domain experts to annotate only one or a few of the cohort's images is feasible, annotating all available images is impractical. This issue is further exacerbated when pre-trained deep networks are exposed to a new image dataset from an unfamiliar distribution. Using available open-source data for ad-hoc transfer learning or hand-tuned techniques for data augmentation only provides suboptimal solutions. Motivated by atlas-based segmentation, we propose a novel volumetric self-supervised learning for data augmentation capable of synthesizing volumetric image-segmentation pairs via learning transformations from a single labeled atlas to the unlabeled data. Our work's central tenet benefits from a combined view of one-shot generative learning and the proposed self-supervised training strategy that cluster unlabeled volumetric images with similar styles together. Unlike previous methods, our method does not require input volumes at inference time to synthesize new images. Instead, it can generate diversified volumetric image-segmentation pairs from a prior distribution given a single or multi-site dataset. Augmented data generated by our method used to train the segmentation network provide significant improvements over state-of-the-art deep one-shot learning methods on the task of brain MRI segmentation. Ablation studies further exemplified that the proposed appearance model and joint training are crucial to synthesize realistic examples compared to existing medical registration methods. The code, data, and models are available at https://github.com/devavratTomar/SST.
△ Less
Submitted 5 October, 2021;
originally announced October 2021.
-
Content-Preserving Unpaired Translation from Simulated to Realistic Ultrasound Images
Authors:
Devavrat Tomar,
Lin Zhang,
Tiziano Portenier,
Orcun Goksel
Abstract:
Interactive simulation of ultrasound imaging greatly facilitates sonography training. Although ray-tracing based methods have shown promising results, obtaining realistic images requires substantial modeling effort and manual parameter tuning. In addition, current techniques still result in a significant appearance gap between simulated images and real clinical scans. Herein we introduce a novel c…
▽ More
Interactive simulation of ultrasound imaging greatly facilitates sonography training. Although ray-tracing based methods have shown promising results, obtaining realistic images requires substantial modeling effort and manual parameter tuning. In addition, current techniques still result in a significant appearance gap between simulated images and real clinical scans. Herein we introduce a novel content-preserving image translation framework (ConPres) to bridge this appearance gap, while maintaining the simulated anatomical layout. We achieve this goal by leveraging both simulated images with semantic segmentations and unpaired in-vivo ultrasound scans. Our framework is based on recent contrastive unpaired translation techniques and we propose a regularization approach by learning an auxiliary segmentation-to-real image translation task, which encourages the disentanglement of content and style. In addition, we extend the generator to be class-conditional, which enables the incorporation of additional losses, in particular a cyclic consistency loss, to further improve the translation quality. Qualitative and quantitative comparisons against state-of-the-art unpaired translation methods demonstrate the superiority of our proposed framework.
△ Less
Submitted 30 September, 2021; v1 submitted 9 March, 2021;
originally announced March 2021.
-
Self-Attentive Spatial Adaptive Normalization for Cross-Modality Domain Adaptation
Authors:
Devavrat Tomar,
Manana Lortkipanidze,
Guillaume Vray,
Behzad Bozorgtabar,
Jean-Philippe Thiran
Abstract:
Despite the successes of deep neural networks on many challenging vision tasks, they often fail to generalize to new test domains that are not distributed identically to the training data. The domain adaptation becomes more challenging for cross-modality medical data with a notable domain shift. Given that specific annotated imaging modalities may not be accessible nor complete. Our proposed solut…
▽ More
Despite the successes of deep neural networks on many challenging vision tasks, they often fail to generalize to new test domains that are not distributed identically to the training data. The domain adaptation becomes more challenging for cross-modality medical data with a notable domain shift. Given that specific annotated imaging modalities may not be accessible nor complete. Our proposed solution is based on the cross-modality synthesis of medical images to reduce the costly annotation burden by radiologists and bridge the domain gap in radiological images. We present a novel approach for image-to-image translation in medical images, capable of supervised or unsupervised (unpaired image data) setups. Built upon adversarial training, we propose a learnable self-attentive spatial normalization of the deep convolutional generator network's intermediate activations. Unlike previous attention-based image-to-image translation approaches, which are either domain-specific or require distortion of the source domain's structures, we unearth the importance of the auxiliary semantic information to handle the geometric changes and preserve anatomical structures during image translation. We achieve superior results for cross-modality segmentation between unpaired MRI and CT data for multi-modality whole heart and multi-modal brain tumor MRI (T1/T2) datasets compared to the state-of-the-art methods. We also observe encouraging results in cross-modality conversion for paired MRI and CT images on a brain dataset. Furthermore, a detailed analysis of the cross-modality image translation, thorough ablation studies confirm our proposed method's efficacy.
△ Less
Submitted 5 March, 2021;
originally announced March 2021.
-
Approaches for user profile Investigation in Orkut Social Network
Authors:
Rajni Ranjan Singh,
Deepak Singh Tomar
Abstract:
Internet becomes a large and rich repository of information about us as individually. Any thing form user profile information to friends links the user subscribes to are reflection of social interactions as user has in real worlds. Social networking has created new ways to communicate and share information. Social networking websites are being used regularly by millions of people, and it now see…
▽ More
Internet becomes a large and rich repository of information about us as individually. Any thing form user profile information to friends links the user subscribes to are reflection of social interactions as user has in real worlds. Social networking has created new ways to communicate and share information. Social networking websites are being used regularly by millions of people, and it now seems that social networking will be an enduring part of everyday life. Social networks such as Orkut, Bebo, MySpace, Flickr, Facebook, Friendster and LinkedIn, have attracted millions of internet user who are involved in bogging, participatory book reviewing, personal networking and photo sharing. Social network services are increasingly being used in legal and criminal investigations. Information posted on sites such as Orkut and Facebook has been used by police, probation, and university officials to prosecute users of said sites. In some situations, content posted on web social network has been used in court. In the proposed work degree of closeness is identified by link weight approaches and information matrices are generated and matched on the basis of similarity in user profile information. The proposed technique is useful to investigate a user profile and calculate closeness or interaction between users.
△ Less
Submitted 5 December, 2009;
originally announced December 2009.
-
Effective Focused Crawling Based on Content and Link Structure Analysis
Authors:
Anshika Pal,
Deepak Singh Tomar,
S. C. Shrivastava
Abstract:
A focused crawler traverses the web selecting out relevant pages to a predefined topic and neglecting those out of concern. While surfing the internet it is difficult to deal with irrelevant pages and to predict which links lead to quality pages. In this paper a technique of effective focused crawling is implemented to improve the quality of web navigation. To check the similarity of web pages w…
▽ More
A focused crawler traverses the web selecting out relevant pages to a predefined topic and neglecting those out of concern. While surfing the internet it is difficult to deal with irrelevant pages and to predict which links lead to quality pages. In this paper a technique of effective focused crawling is implemented to improve the quality of web navigation. To check the similarity of web pages w.r.t. topic keywords a similarity function is used and the priorities of extracted out links are also calculated based on meta data and resultant pages generated from focused crawler. The proposed work also uses a method for traversing the irrelevant pages that met during crawling to improve the coverage of a specific topic.
△ Less
Submitted 26 June, 2009;
originally announced June 2009.
