-
Task-driven Prompt Evolution for Foundation Models
Authors:
Rachana Sathish,
Rahul Venkataramani,
K S Shriram,
Prasad Sudhakar
Abstract:
Promptable foundation models, particularly Segment Anything Model (SAM), have emerged as a promising alternative to the traditional task-specific supervised learning for image segmentation. However, many evaluation studies have found that their performance on medical imaging modalities to be underwhelming compared to conventional deep learning methods. In the world of large pre-trained language an…
▽ More
Promptable foundation models, particularly Segment Anything Model (SAM), have emerged as a promising alternative to the traditional task-specific supervised learning for image segmentation. However, many evaluation studies have found that their performance on medical imaging modalities to be underwhelming compared to conventional deep learning methods. In the world of large pre-trained language and vision-language models, learning prompt from downstream tasks has achieved considerable success in improving performance. In this work, we propose a plug-and-play Prompt Optimization Technique for foundation models like SAM (SAMPOT) that utilizes the downstream segmentation task to optimize the human-provided prompt to obtain improved performance. We demonstrate the utility of SAMPOT on lung segmentation in chest X-ray images and obtain an improvement on a significant number of cases ($\sim75\%$) over human-provided initial prompts. We hope this work will lead to further investigations in the nascent field of automatic visual prompt-tuning.
△ Less
Submitted 26 October, 2023;
originally announced October 2023.
-
Towards Continuous Domain adaptation for Healthcare
Authors:
Rahul Venkataramani,
Hariharan Ravishankar,
Saihareesh Anamandra
Abstract:
Deep learning algorithms have demonstrated tremendous success on challenging medical imaging problems. However, post-deployment, these algorithms are susceptible to data distribution variations owing to \emph{limited data issues} and \emph{diversity} in medical images. In this paper, we propose \emph{ContextNets}, a generic memory-augmented neural network framework for semantic segmentation to ach…
▽ More
Deep learning algorithms have demonstrated tremendous success on challenging medical imaging problems. However, post-deployment, these algorithms are susceptible to data distribution variations owing to \emph{limited data issues} and \emph{diversity} in medical images. In this paper, we propose \emph{ContextNets}, a generic memory-augmented neural network framework for semantic segmentation to achieve continuous domain adaptation without the necessity of retraining. Unlike existing methods which require access to entire source and target domain images, our algorithm can adapt to a target domain with a few similar images. We condition the inference on any new input with features computed on its support set of images (and masks, if available) through contextual embeddings to achieve site-specific adaptation. We demonstrate state-of-the-art domain adaptation performance on the X-ray lung segmentation problem from three independent cohorts that differ in disease type, gender, contrast and intensity variations.
△ Less
Submitted 4 December, 2018;
originally announced December 2018.
-
Understanding the Mechanisms of Deep Transfer Learning for Medical Images
Authors:
Hariharan Ravishankar,
Prasad Sudhakar,
Rahul Venkataramani,
Sheshadri Thiruvenkadam,
Pavan Annangi,
Narayanan Babu,
Vivek Vaidya
Abstract:
The ability to automatically learn task specific feature representations has led to a huge success of deep learning methods. When large training data is scarce, such as in medical imaging problems, transfer learning has been very effective. In this paper, we systematically investigate the process of transferring a Convolutional Neural Network, trained on ImageNet images to perform image classifica…
▽ More
The ability to automatically learn task specific feature representations has led to a huge success of deep learning methods. When large training data is scarce, such as in medical imaging problems, transfer learning has been very effective. In this paper, we systematically investigate the process of transferring a Convolutional Neural Network, trained on ImageNet images to perform image classification, to kidney detection problem in ultrasound images. We study how the detection performance depends on the extent of transfer. We show that a transferred and tuned CNN can outperform a state-of-the-art feature engineered pipeline and a hybridization of these two techniques achieves 20\% higher performance. We also investigate how the evolution of intermediate response images from our network. Finally, we compare these responses to state-of-the-art image processing filters in order to gain greater insight into how transfer learning is able to effectively manage widely varying imaging regimes.
△ Less
Submitted 20 April, 2017;
originally announced April 2017.
-
Filter sharing: Efficient learning of parameters for volumetric convolutions
Authors:
Rahul Venkataramani,
Sheshadri Thiruvenkadam,
Prasad Sudhakar,
Hariharan Ravishankar,
Vivek Vaidya
Abstract:
Typical convolutional neural networks (CNNs) have several millions of parameters and require a large amount of annotated data to train them. In medical applications where training data is hard to come by, these sophisticated machine learning models are difficult to train. In this paper, we propose a method to reduce the inherent complexity of CNNs during training by exploiting the significant redu…
▽ More
Typical convolutional neural networks (CNNs) have several millions of parameters and require a large amount of annotated data to train them. In medical applications where training data is hard to come by, these sophisticated machine learning models are difficult to train. In this paper, we propose a method to reduce the inherent complexity of CNNs during training by exploiting the significant redundancy that is noticed in the learnt CNN filters. Our method relies on finding a small set of filters and mixing coefficients to derive every filter in each convolutional layer at the time of training itself, thereby reducing the number of parameters to be trained. We consider the problem of 3D lung nodule segmentation in CT images and demonstrate the effectiveness of our method in achieving good results with only few training examples.
△ Less
Submitted 8 December, 2016;
originally announced December 2016.
-
A Posteriori Equivalence: A New Perspective for Design of Optimal Channel Shortening Equalizers
Authors:
Raman Venkataramani,
M. Fatih Erden
Abstract:
The problem of channel shortening equalization for optimal detection in ISI channels is considered. The problem is to choose a linear equalizer and a partial response target filter such that the combination produces the best detection performance. Instead of using the traditional approach of MMSE equalization, we directly seek all equalizer and target pairs that yield optimal detection performan…
▽ More
The problem of channel shortening equalization for optimal detection in ISI channels is considered. The problem is to choose a linear equalizer and a partial response target filter such that the combination produces the best detection performance. Instead of using the traditional approach of MMSE equalization, we directly seek all equalizer and target pairs that yield optimal detection performance in terms of the sequence or symbol error rate. This leads to a new notion of a posteriori equivalence between the equalized and target channels with a simple characterization in terms of their underlying probability distributions. Using this characterization we show the surprising existence an infinite family of equalizer and target pairs for which any maximum a posteriori (MAP) based detector designed for the target channel is simultaneously MAP optimal for the equalized channel. For channels whose input symbols have equal energy, such as q-PSK, the MMSE equalizer designed with a monic target constraint yields a solution belonging to this optimal family of designs. Although, these designs produce IIR target filters, the ideas are extended to design good FIR targets. For an arbitrary choice of target and equalizer, we derive an expression for the probability of sequence detection error. This expression is used to design optimal FIR targets and IIR equalizers and to quantify the FIR approximation penalty.
△ Less
Submitted 19 October, 2007;
originally announced October 2007.
