-
Un-Mixing Test-Time Normalization Statistics: Combatting Label Temporal Correlation
Authors:
Devavrat Tomar,
Guillaume Vray,
Jean-Philippe Thiran,
Behzad Bozorgtabar
Abstract:
Recent test-time adaptation methods heavily rely on nuanced adjustments of batch normalization (BN) parameters. However, one critical assumption often goes overlooked: that of independently and identically distributed (i.i.d.) test batches with respect to unknown labels. This oversight leads to skewed BN statistics and undermines the reliability of the model under non-i.i.d. scenarios. To tackle t…
▽ More
Recent test-time adaptation methods heavily rely on nuanced adjustments of batch normalization (BN) parameters. However, one critical assumption often goes overlooked: that of independently and identically distributed (i.i.d.) test batches with respect to unknown labels. This oversight leads to skewed BN statistics and undermines the reliability of the model under non-i.i.d. scenarios. To tackle this challenge, this paper presents a novel method termed 'Un-Mixing Test-Time Normalization Statistics' (UnMix-TNS). Our method re-calibrates the statistics for each instance within a test batch by mixing it with multiple distinct statistics components, thus inherently simulating the i.i.d. scenario. The core of this method hinges on a distinctive online unmixing procedure that continuously updates these statistics components by incorporating the most similar instances from new test batches. Remarkably generic in its design, UnMix-TNS seamlessly integrates with a wide range of leading test-time adaptation methods and pre-trained architectures equipped with BN layers. Empirical evaluations corroborate the robustness of UnMix-TNS under varied scenarios-ranging from single to continual and mixed domain shifts, particularly excelling with temporally correlated test data and corrupted non-i.i.d. real-world streams. This adaptability is maintained even with very small batch sizes or single instances. Our results highlight UnMix-TNS's capacity to markedly enhance stability and performance across various benchmarks. Our code is publicly available at https://github.com/devavratTomar/unmixtns.
△ Less
Submitted 14 March, 2024; v1 submitted 16 January, 2024;
originally announced January 2024.
-
Distill-SODA: Distilling Self-Supervised Vision Transformer for Source-Free Open-Set Domain Adaptation in Computational Pathology
Authors:
Guillaume Vray,
Devavrat Tomar,
Jean-Philippe Thiran,
Behzad Bozorgtabar
Abstract:
Develo** computational pathology models is essential for reducing manual tissue ty** from whole slide images, transferring knowledge from the source domain to an unlabeled, shifted target domain, and identifying unseen categories. We propose a practical setting by addressing the above-mentioned challenges in one fell swoop, i.e., source-free open-set domain adaptation. Our methodology focuses…
▽ More
Develo** computational pathology models is essential for reducing manual tissue ty** from whole slide images, transferring knowledge from the source domain to an unlabeled, shifted target domain, and identifying unseen categories. We propose a practical setting by addressing the above-mentioned challenges in one fell swoop, i.e., source-free open-set domain adaptation. Our methodology focuses on adapting a pre-trained source model to an unlabeled target dataset and encompasses both closed-set and open-set classes. Beyond addressing the semantic shift of unknown classes, our framework also deals with a covariate shift, which manifests as variations in color appearance between source and target tissue samples. Our method hinges on distilling knowledge from a self-supervised vision transformer (ViT), drawing guidance from either robustly pre-trained transformer models or histopathology datasets, including those from the target domain. In pursuit of this, we introduce a novel style-based adversarial data augmentation, serving as hard positives for self-training a ViT, resulting in highly contextualized embeddings. Following this, we cluster semantically akin target images, with the source model offering weak pseudo-labels, albeit with uncertain confidence. To enhance this process, we present the closed-set affinity score (CSAS), aiming to correct the confidence levels of these pseudo-labels and to calculate weighted class prototypes within the contextualized embedding space. Our approach establishes itself as state-of-the-art across three public histopathological datasets for colorectal cancer assessment. Notably, our self-training method seamlessly integrates with open-set detection methods, resulting in enhanced performance in both closed-set and open-set recognition tasks.
△ Less
Submitted 16 January, 2024; v1 submitted 10 July, 2023;
originally announced July 2023.
-
TeSLA: Test-Time Self-Learning With Automatic Adversarial Augmentation
Authors:
Devavrat Tomar,
Guillaume Vray,
Behzad Bozorgtabar,
Jean-Philippe Thiran
Abstract:
Most recent test-time adaptation methods focus on only classification tasks, use specialized network architectures, destroy model calibration or rely on lightweight information from the source domain. To tackle these issues, this paper proposes a novel Test-time Self-Learning method with automatic Adversarial augmentation dubbed TeSLA for adapting a pre-trained source model to the unlabeled stream…
▽ More
Most recent test-time adaptation methods focus on only classification tasks, use specialized network architectures, destroy model calibration or rely on lightweight information from the source domain. To tackle these issues, this paper proposes a novel Test-time Self-Learning method with automatic Adversarial augmentation dubbed TeSLA for adapting a pre-trained source model to the unlabeled streaming test data. In contrast to conventional self-learning methods based on cross-entropy, we introduce a new test-time loss function through an implicitly tight connection with the mutual information and online knowledge distillation. Furthermore, we propose a learnable efficient adversarial augmentation module that further enhances online knowledge distillation by simulating high entropy augmented images. Our method achieves state-of-the-art classification and segmentation results on several benchmarks and types of domain shifts, particularly on challenging measurement shifts of medical images. TeSLA also benefits from several desirable properties compared to competing methods in terms of calibration, uncertainty metrics, insensitivity to model architectures, and source training strategies, all supported by extensive ablations. Our code and models are available on GitHub.
△ Less
Submitted 17 March, 2023;
originally announced March 2023.
-
Self-Supervised Generative Style Transfer for One-Shot Medical Image Segmentation
Authors:
Devavrat Tomar,
Behzad Bozorgtabar,
Manana Lortkipanidze,
Guillaume Vray,
Mohammad Saeed Rad,
Jean-Philippe Thiran
Abstract:
In medical image segmentation, supervised deep networks' success comes at the cost of requiring abundant labeled data. While asking domain experts to annotate only one or a few of the cohort's images is feasible, annotating all available images is impractical. This issue is further exacerbated when pre-trained deep networks are exposed to a new image dataset from an unfamiliar distribution. Using…
▽ More
In medical image segmentation, supervised deep networks' success comes at the cost of requiring abundant labeled data. While asking domain experts to annotate only one or a few of the cohort's images is feasible, annotating all available images is impractical. This issue is further exacerbated when pre-trained deep networks are exposed to a new image dataset from an unfamiliar distribution. Using available open-source data for ad-hoc transfer learning or hand-tuned techniques for data augmentation only provides suboptimal solutions. Motivated by atlas-based segmentation, we propose a novel volumetric self-supervised learning for data augmentation capable of synthesizing volumetric image-segmentation pairs via learning transformations from a single labeled atlas to the unlabeled data. Our work's central tenet benefits from a combined view of one-shot generative learning and the proposed self-supervised training strategy that cluster unlabeled volumetric images with similar styles together. Unlike previous methods, our method does not require input volumes at inference time to synthesize new images. Instead, it can generate diversified volumetric image-segmentation pairs from a prior distribution given a single or multi-site dataset. Augmented data generated by our method used to train the segmentation network provide significant improvements over state-of-the-art deep one-shot learning methods on the task of brain MRI segmentation. Ablation studies further exemplified that the proposed appearance model and joint training are crucial to synthesize realistic examples compared to existing medical registration methods. The code, data, and models are available at https://github.com/devavratTomar/SST.
△ Less
Submitted 5 October, 2021;
originally announced October 2021.
-
Content-Preserving Unpaired Translation from Simulated to Realistic Ultrasound Images
Authors:
Devavrat Tomar,
Lin Zhang,
Tiziano Portenier,
Orcun Goksel
Abstract:
Interactive simulation of ultrasound imaging greatly facilitates sonography training. Although ray-tracing based methods have shown promising results, obtaining realistic images requires substantial modeling effort and manual parameter tuning. In addition, current techniques still result in a significant appearance gap between simulated images and real clinical scans. Herein we introduce a novel c…
▽ More
Interactive simulation of ultrasound imaging greatly facilitates sonography training. Although ray-tracing based methods have shown promising results, obtaining realistic images requires substantial modeling effort and manual parameter tuning. In addition, current techniques still result in a significant appearance gap between simulated images and real clinical scans. Herein we introduce a novel content-preserving image translation framework (ConPres) to bridge this appearance gap, while maintaining the simulated anatomical layout. We achieve this goal by leveraging both simulated images with semantic segmentations and unpaired in-vivo ultrasound scans. Our framework is based on recent contrastive unpaired translation techniques and we propose a regularization approach by learning an auxiliary segmentation-to-real image translation task, which encourages the disentanglement of content and style. In addition, we extend the generator to be class-conditional, which enables the incorporation of additional losses, in particular a cyclic consistency loss, to further improve the translation quality. Qualitative and quantitative comparisons against state-of-the-art unpaired translation methods demonstrate the superiority of our proposed framework.
△ Less
Submitted 30 September, 2021; v1 submitted 9 March, 2021;
originally announced March 2021.
-
Self-Attentive Spatial Adaptive Normalization for Cross-Modality Domain Adaptation
Authors:
Devavrat Tomar,
Manana Lortkipanidze,
Guillaume Vray,
Behzad Bozorgtabar,
Jean-Philippe Thiran
Abstract:
Despite the successes of deep neural networks on many challenging vision tasks, they often fail to generalize to new test domains that are not distributed identically to the training data. The domain adaptation becomes more challenging for cross-modality medical data with a notable domain shift. Given that specific annotated imaging modalities may not be accessible nor complete. Our proposed solut…
▽ More
Despite the successes of deep neural networks on many challenging vision tasks, they often fail to generalize to new test domains that are not distributed identically to the training data. The domain adaptation becomes more challenging for cross-modality medical data with a notable domain shift. Given that specific annotated imaging modalities may not be accessible nor complete. Our proposed solution is based on the cross-modality synthesis of medical images to reduce the costly annotation burden by radiologists and bridge the domain gap in radiological images. We present a novel approach for image-to-image translation in medical images, capable of supervised or unsupervised (unpaired image data) setups. Built upon adversarial training, we propose a learnable self-attentive spatial normalization of the deep convolutional generator network's intermediate activations. Unlike previous attention-based image-to-image translation approaches, which are either domain-specific or require distortion of the source domain's structures, we unearth the importance of the auxiliary semantic information to handle the geometric changes and preserve anatomical structures during image translation. We achieve superior results for cross-modality segmentation between unpaired MRI and CT data for multi-modality whole heart and multi-modal brain tumor MRI (T1/T2) datasets compared to the state-of-the-art methods. We also observe encouraging results in cross-modality conversion for paired MRI and CT images on a brain dataset. Furthermore, a detailed analysis of the cross-modality image translation, thorough ablation studies confirm our proposed method's efficacy.
△ Less
Submitted 5 March, 2021;
originally announced March 2021.
-
System size dependence of hydration shell occupancy
Authors:
D. Asthagiri,
Dheeraj Singh Tomar
Abstract:
The free energies to evacuate the first hydration shell around a solute and a cavity defined by the first hydration shell depend on the system size. This observation interpreted within the quasichemical theory shows that both the hydrophilic and the hydrophobic contributions to hydration depend on the system size, decreasing with increasing system size. Although the net hydration free energy benef…
▽ More
The free energies to evacuate the first hydration shell around a solute and a cavity defined by the first hydration shell depend on the system size. This observation interpreted within the quasichemical theory shows that both the hydrophilic and the hydrophobic contributions to hydration depend on the system size, decreasing with increasing system size. Although the net hydration free energy benefits somewhat from the balancing of hydrophilic and hydrophobic contributions, a large system still appears necessary to describe the effect of the solvent on the macromolecule.
△ Less
Submitted 3 July, 2019; v1 submitted 29 June, 2019;
originally announced July 2019.
-
Long-Range Interactions Dominate the Inverse-Temperature Dependence of Polypeptide Hydration Free Energies
Authors:
Dheeraj S. Tomar,
Michael E. Paulaitis,
Lawrence R. Pratt,
D. Asthagiri
Abstract:
Direct, all-atom calculations of the free energy of hydration of aqueous deca-alanine structures --- holistically including backbone and side-chain interactions together --- show that attractive interactions and the thermal expansion of the solvent explain the inverse temperature signatures that have been interpreted traditionally in favor of hydrophobic mechanisms for stabilizing the structure an…
▽ More
Direct, all-atom calculations of the free energy of hydration of aqueous deca-alanine structures --- holistically including backbone and side-chain interactions together --- show that attractive interactions and the thermal expansion of the solvent explain the inverse temperature signatures that have been interpreted traditionally in favor of hydrophobic mechanisms for stabilizing the structure and function of soluble proteins.
△ Less
Submitted 31 December, 2018; v1 submitted 17 December, 2018;
originally announced December 2018.
-
Solvophobic and solvophilic contributions in the water-to-aqueous guanidinium chloride transfer free energy of model peptides
Authors:
Dheeraj S. Tomar,
Niral Ramesh,
D. Asthagiri
Abstract:
We study the solvation free energy of two different conformations (helix and extended) of two different peptides (deca-alanine and deca-glycine) in two different solvents (water and aqueous guanidinium chloride, GdmCl). The free energies are obtained using the quasichemical organization of the potential distribution theorem, an approach that naturally provides the repulsive (solvophobic or cavity)…
▽ More
We study the solvation free energy of two different conformations (helix and extended) of two different peptides (deca-alanine and deca-glycine) in two different solvents (water and aqueous guanidinium chloride, GdmCl). The free energies are obtained using the quasichemical organization of the potential distribution theorem, an approach that naturally provides the repulsive (solvophobic or cavity) and attractive (solvophilic) contributions to solvation. The solvophilic contribution is further parsed into a chemistry contribution arising from solute interaction with the solvent in the first solvation shell and a long-range contribution arising from non-specific interactions between the solute and the solvent beyond the first solvation shell. The cavity contribution is obtained for two different envelopes, $Σ_{SE}$ which theory identifies as the solvent excluded volume and a larger envelope ($Σ_G$) beyond which solute-solvent interactions are Gaussian. For both envelopes, the cavity contribution in water is proportional to the surface area of the envelope. The same does not hold for GdmCl(aq), revealing limitations of using molecular area to assess solvation energetics, especially in mixed solvents. The $Σ_G$-cavity contribution predicts that GdmCl(aq) should favor the more compact state, contrary to the role of GdmCl in unfolding proteins. The chemistry contribution attenuates this effect, but still the net local (chemistry plus $Σ_G$-packing) contribution is inadequate in capturing the role of GdmCl. With the inclusion of the long-range contribution, which is dominated by van~der~Waals interaction, aqueous GdmCl favors the extended conformation over the compact conformation. Our finding emphasizes the importance of weak, but attractive, long-range dispersion interactions in protein solution thermodynamics.
△ Less
Submitted 14 January, 2018;
originally announced January 2018.
-
Role of hydration and intramolecular interactions in the helix-coil transition and helix-helix assembly in a deca-alanine peptide
Authors:
Dheeraj S. Tomar,
Valery Weber,
B. M. Pettitt,
D. Asthagiri
Abstract:
For a model deca-alanine peptide the cavity (ideal hydrophobic) contribution to hydration favors the helix state in the coil-to-helix transition and the paired helix bundle in the assembly of two helices. The energetic contributions of attractive protein-solvent interactions are separated into a short-range part arising from interactions with solvent in the first hydration shell and the remaining…
▽ More
For a model deca-alanine peptide the cavity (ideal hydrophobic) contribution to hydration favors the helix state in the coil-to-helix transition and the paired helix bundle in the assembly of two helices. The energetic contributions of attractive protein-solvent interactions are separated into a short-range part arising from interactions with solvent in the first hydration shell and the remaining long-range part. In the helix-coil transition, short-range attractive protein-solvent interactions outweigh hydrophobic hydration and favor the unfolded coil states. Analysis of enthalpic effects shows that it is the favorable hydration of the peptide backbone that favors the unfolded state. Protein intramolecular interactions favor the helix state and are decisive in folding. In the pairing of two helices, the cavity contribution outweighs short-range attractive protein-water interactions. However, long-range, protein-solvent attractive interactions can either enhance or reverse this trend depending on the mutual orientation of the helices. In helix-helix assembly, change in enthalpy arising from change in attractive protein-solvent interactions favors disassembly. In helix pairing as well, favorable protein intramolecular interactions are found to be as important as hydration effects.
△ Less
Submitted 22 August, 2015;
originally announced August 2015.
-
Conditional solvation of isoleucine in model extended and helical peptides: context dependence of hydrophobic hydration and the failure of the group-transfer model
Authors:
Dheeraj Tomar,
Valéry Weber,
B. M. Pettitt,
D. Asthagiri
Abstract:
The hydration thermodynamics of the GXG tripeptide relative to the reference GGG is often used to define the conditional hydration contribution of X. This quantity or the hydration thermodynamics of a small molecule analog of the side-chain or some combination of such estimates, have anchored the interpretation of seminal experiments on protein stability and folding. We show that such procedures t…
▽ More
The hydration thermodynamics of the GXG tripeptide relative to the reference GGG is often used to define the conditional hydration contribution of X. This quantity or the hydration thermodynamics of a small molecule analog of the side-chain or some combination of such estimates, have anchored the interpretation of seminal experiments on protein stability and folding. We show that such procedures to model protein hydration have significant limitations. We study the conditional hydration thermodynamics of the isoleucine side-chain in an extended pentapeptide and in helical deca-peptides, using as appropriate an extended penta-glycine or appropriate helical deca-peptides as reference. Hydration of butane in the gauche conformation provides a small molecule reference for the side-chain. We use the quasichemical theory to parse the hydration thermodynamics into chemical, packing, and long-range interaction contributions. The chemical contribution reflects the contribution of solvent clustering within the defined inner-shell of the solute; the chemical contribution of g-butane is substantially more negative than the conditional chemical contribution of isoleucine. The packing contribution gives the work required to create a cavity in the solvent, a quantity of interest in understanding hydrophobic hydration. The packing contribution for g-butane substantially overestimates the conditional packing of isoleucine. The net of such compensating contributions still disagrees with the conditional free energy of isoleucine but by a lesser magnitude. The excess enthalpy and entropy of hydration of g-butane model are also more negative than the corresponding conditional quantities for the side-chain. The conditional solvation of isoleucine in GGIGG also proves unsatisfactory in describing the conditional solvation of isoleucine in the helical peptides.
△ Less
Submitted 5 November, 2013; v1 submitted 27 October, 2013;
originally announced October 2013.
-
Examining the meaning of the peptide transfer free energy obtained from blocked (Gly)_n and cyclic-diglycine model compounds
Authors:
D. Asthagiri,
D. S. Tomar,
V. Weber
Abstract:
In experiments, the free energy of transferring the peptide group from water to an osmolyte solution is obtained using the transfer free energy of (Gly)_n with the added assumption that a constant incremental change in free energy with n implies that each additional unit makes an independent contribution to the free energy. Here we test this assumption and uncover its limitations. Together with re…
▽ More
In experiments, the free energy of transferring the peptide group from water to an osmolyte solution is obtained using the transfer free energy of (Gly)_n with the added assumption that a constant incremental change in free energy with n implies that each additional unit makes an independent contribution to the free energy. Here we test this assumption and uncover its limitations. Together with results for cyclic-diglycine, we show that, in principle, it is not possible to obtain a peptide group transfer free energy that is independent of the model system. We calculate the hydration free energy of acetyl-(Gly)_n-methyl amide (n=1..7) peptides modeled in the extended conformation in water and osmolyte solutions and find that the hydration free energy is linear in n, suggestive of independent, additive group-contributions. To probe the observed linearity further, we study the hydration of the solute bereft of water molecules in the first hydration shell. This conditioned solute arises naturally in the theoretical formulation and helps us focus on hydration effects uncluttered by the complexities of short-range solute-water interactions. We subdivide the conditioned solute into n+1 peptide groups and a methyl end group. The binding energy of each of these groups with the solvent is Gaussian distributed, but the near neighbor binding energies are themselves correlated: the (i,i+1) correlation is the strongest and tends to lower the free energy over the independent group case. We show that the observed linearity can be explained by the similarity of near neighbor correlations. Implications for group additive transfer free energy models are indicated.
△ Less
Submitted 27 January, 2013;
originally announced January 2013.
-
Approaches for user profile Investigation in Orkut Social Network
Authors:
Rajni Ranjan Singh,
Deepak Singh Tomar
Abstract:
Internet becomes a large and rich repository of information about us as individually. Any thing form user profile information to friends links the user subscribes to are reflection of social interactions as user has in real worlds. Social networking has created new ways to communicate and share information. Social networking websites are being used regularly by millions of people, and it now see…
▽ More
Internet becomes a large and rich repository of information about us as individually. Any thing form user profile information to friends links the user subscribes to are reflection of social interactions as user has in real worlds. Social networking has created new ways to communicate and share information. Social networking websites are being used regularly by millions of people, and it now seems that social networking will be an enduring part of everyday life. Social networks such as Orkut, Bebo, MySpace, Flickr, Facebook, Friendster and LinkedIn, have attracted millions of internet user who are involved in bogging, participatory book reviewing, personal networking and photo sharing. Social network services are increasingly being used in legal and criminal investigations. Information posted on sites such as Orkut and Facebook has been used by police, probation, and university officials to prosecute users of said sites. In some situations, content posted on web social network has been used in court. In the proposed work degree of closeness is identified by link weight approaches and information matrices are generated and matched on the basis of similarity in user profile information. The proposed technique is useful to investigate a user profile and calculate closeness or interaction between users.
△ Less
Submitted 5 December, 2009;
originally announced December 2009.
-
Effective Focused Crawling Based on Content and Link Structure Analysis
Authors:
Anshika Pal,
Deepak Singh Tomar,
S. C. Shrivastava
Abstract:
A focused crawler traverses the web selecting out relevant pages to a predefined topic and neglecting those out of concern. While surfing the internet it is difficult to deal with irrelevant pages and to predict which links lead to quality pages. In this paper a technique of effective focused crawling is implemented to improve the quality of web navigation. To check the similarity of web pages w…
▽ More
A focused crawler traverses the web selecting out relevant pages to a predefined topic and neglecting those out of concern. While surfing the internet it is difficult to deal with irrelevant pages and to predict which links lead to quality pages. In this paper a technique of effective focused crawling is implemented to improve the quality of web navigation. To check the similarity of web pages w.r.t. topic keywords a similarity function is used and the priorities of extracted out links are also calculated based on meta data and resultant pages generated from focused crawler. The proposed work also uses a method for traversing the irrelevant pages that met during crawling to improve the coverage of a specific topic.
△ Less
Submitted 26 June, 2009;
originally announced June 2009.
