-
Vision Mamba for Classification of Breast Ultrasound Images
Authors:
Ali Nasiri-Sarvi,
Mahdi S. Hosseini,
Hassan Rivaz
Abstract:
Mamba-based models, VMamba and Vim, are a recent family of vision encoders that offer promising performance improvements in many computer vision tasks. This paper compares Mamba-based models with traditional Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) using the breast ultrasound BUSI and B datasets. Our evaluation, which includes multiple runs of experiments and statistical…
▽ More
Mamba-based models, VMamba and Vim, are a recent family of vision encoders that offer promising performance improvements in many computer vision tasks. This paper compares Mamba-based models with traditional Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) using the breast ultrasound BUSI and B datasets. Our evaluation, which includes multiple runs of experiments and statistical significance analysis, demonstrates that Mamba-based architectures frequently outperform CNN and ViT models with statistically significant results. These Mamba-based models effectively capture long-range dependencies while maintaining inductive biases, making them suitable for applications with limited data.
△ Less
Submitted 3 July, 2024;
originally announced July 2024.
-
AdaFisher: Adaptive Second Order Optimization via Fisher Information
Authors:
Damien Martins Gomes,
Yanlei Zhang,
Eugene Belilovsky,
Guy Wolf,
Mahdi S. Hosseini
Abstract:
First-order optimization methods are currently the mainstream in training deep neural networks (DNNs). Optimizers like Adam incorporate limited curvature information by employing the diagonal matrix preconditioning of the stochastic gradient during the training. Despite their widespread, second-order optimization algorithms exhibit superior convergence properties compared to their first-order coun…
▽ More
First-order optimization methods are currently the mainstream in training deep neural networks (DNNs). Optimizers like Adam incorporate limited curvature information by employing the diagonal matrix preconditioning of the stochastic gradient during the training. Despite their widespread, second-order optimization algorithms exhibit superior convergence properties compared to their first-order counterparts e.g. Adam and SGD. However, their practicality in training DNNs are still limited due to increased per-iteration computations and suboptimal accuracy compared to the first order methods. We present AdaFisher--an adaptive second-order optimizer that leverages a block-diagonal approximation to the Fisher information matrix for adaptive gradient preconditioning. AdaFisher aims to bridge the gap between enhanced convergence capabilities and computational efficiency in second-order optimization framework for training DNNs. Despite the slow pace of second-order optimizers, we showcase that AdaFisher can be reliably adopted for image classification, language modelling and stand out for its stability and robustness in hyperparameter tuning. We demonstrate that AdaFisher outperforms the SOTA optimizers in terms of both accuracy and convergence speed. Code available from \href{https://github.com/AtlasAnalyticsLab/AdaFisher}{https://github.com/AtlasAnalyticsLab/AdaFisher}
△ Less
Submitted 25 May, 2024;
originally announced May 2024.
-
Vim4Path: Self-Supervised Vision Mamba for Histopathology Images
Authors:
Ali Nasiri-Sarvi,
Vincent Quoc-Huy Trinh,
Hassan Rivaz,
Mahdi S. Hosseini
Abstract:
Representation learning from Gigapixel Whole Slide Images (WSI) poses a significant challenge in computational pathology due to the complicated nature of tissue structures and the scarcity of labeled data. Multi-instance learning methods have addressed this challenge, leveraging image patches to classify slides utilizing pretrained models using Self-Supervised Learning (SSL) approaches. The perfor…
▽ More
Representation learning from Gigapixel Whole Slide Images (WSI) poses a significant challenge in computational pathology due to the complicated nature of tissue structures and the scarcity of labeled data. Multi-instance learning methods have addressed this challenge, leveraging image patches to classify slides utilizing pretrained models using Self-Supervised Learning (SSL) approaches. The performance of both SSL and MIL methods relies on the architecture of the feature encoder. This paper proposes leveraging the Vision Mamba (Vim) architecture, inspired by state space models, within the DINO framework for representation learning in computational pathology. We evaluate the performance of Vim against Vision Transformers (ViT) on the Camelyon16 dataset for both patch-level and slide-level classification. Our findings highlight Vim's enhanced performance compared to ViT, particularly at smaller scales, where Vim achieves an 8.21 increase in ROC AUC for models of similar size. An explainability analysis further highlights Vim's capabilities, which reveals that Vim uniquely emulates the pathologist workflow-unlike ViT. This alignment with human expert analysis highlights Vim's potential in practical diagnostic settings and contributes significantly to develo** effective representation-learning algorithms in computational pathology. We release the codes and pretrained weights at \url{https://github.com/AtlasAnalyticsLab/Vim4Path}.
△ Less
Submitted 25 May, 2024; v1 submitted 19 April, 2024;
originally announced April 2024.
-
End-to-End Supervised Multilabel Contrastive Learning
Authors:
Ahmad Sajedi,
Samir Khaki,
Konstantinos N. Plataniotis,
Mahdi S. Hosseini
Abstract:
Multilabel representation learning is recognized as a challenging problem that can be associated with either label dependencies between object categories or data-related issues such as the inherent imbalance of positive/negative samples. Recent advances address these challenges from model- and data-centric viewpoints. In model-centric, the label correlation is obtained by an external model designs…
▽ More
Multilabel representation learning is recognized as a challenging problem that can be associated with either label dependencies between object categories or data-related issues such as the inherent imbalance of positive/negative samples. Recent advances address these challenges from model- and data-centric viewpoints. In model-centric, the label correlation is obtained by an external model designs (e.g., graph CNN) to incorporate an inductive bias for training. However, they fail to design an end-to-end training framework, leading to high computational complexity. On the contrary, in data-centric, the realistic nature of the dataset is considered for improving the classification while ignoring the label dependencies. In this paper, we propose a new end-to-end training framework -- dubbed KMCL (Kernel-based Mutlilabel Contrastive Learning) -- to address the shortcomings of both model- and data-centric designs. The KMCL first transforms the embedded features into a mixture of exponential kernels in Gaussian RKHS. It is then followed by encoding an objective loss that is comprised of (a) reconstruction loss to reconstruct kernel representation, (b) asymmetric classification loss to address the inherent imbalance problem, and (c) contrastive loss to capture label correlation. The KMCL models the uncertainty of the feature encoder while maintaining a low computational footprint. Extensive experiments are conducted on image classification tasks to showcase the consistent improvements of KMCL over the SOTA methods. PyTorch implementation is provided in \url{https://github.com/mahdihosseini/KMCL}.
△ Less
Submitted 8 July, 2023;
originally announced July 2023.
-
Computational Pathology: A Survey Review and The Way Forward
Authors:
Mahdi S. Hosseini,
Babak Ehteshami Bejnordi,
Vincent Quoc-Huy Trinh,
Danial Hasan,
Xingwen Li,
Taehyo Kim,
Haochen Zhang,
Theodore Wu,
Kajanan Chinniah,
Sina Maghsoudlou,
Ryan Zhang,
Stephen Yang,
Jiadai Zhu,
Lyndon Chan,
Samir Khaki,
Andrei Buin,
Fatemeh Chaji,
Ala Salehi,
Bich Ngoc Nguyen,
Dimitris Samaras,
Konstantinos N. Plataniotis
Abstract:
Computational Pathology CPath is an interdisciplinary science that augments developments of computational approaches to analyze and model medical histopathology images. The main objective for CPath is to develop infrastructure and workflows of digital diagnostics as an assistive CAD system for clinical pathology, facilitating transformational changes in the diagnosis and treatment of cancer that a…
▽ More
Computational Pathology CPath is an interdisciplinary science that augments developments of computational approaches to analyze and model medical histopathology images. The main objective for CPath is to develop infrastructure and workflows of digital diagnostics as an assistive CAD system for clinical pathology, facilitating transformational changes in the diagnosis and treatment of cancer that are mainly address by CPath tools. With evergrowing developments in deep learning and computer vision algorithms, and the ease of the data flow from digital pathology, currently CPath is witnessing a paradigm shift. Despite the sheer volume of engineering and scientific works being introduced for cancer image analysis, there is still a considerable gap of adopting and integrating these algorithms in clinical practice. This raises a significant question regarding the direction and trends that are undertaken in CPath. In this article we provide a comprehensive review of more than 800 papers to address the challenges faced in problem design all-the-way to the application and implementation viewpoints. We have catalogued each paper into a model-card by examining the key works and challenges faced to layout the current landscape in CPath. We hope this helps the community to locate relevant works and facilitate understanding of the field's future directions. In a nutshell, we oversee the CPath developments in cycle of stages which are required to be cohesively linked together to address the challenges associated with such multidisciplinary science. We overview this cycle from different perspectives of data-centric, model-centric, and application-centric problems. We finally sketch remaining challenges and provide directions for future technical developments and clinical integration of CPath (https://github.com/AtlasAnalyticsLab/CPath_Survey).
△ Less
Submitted 27 January, 2024; v1 submitted 11 April, 2023;
originally announced April 2023.
-
Pseudo-Inverted Bottleneck Convolution for DARTS Search Space
Authors:
Arash Ahmadian,
Louis S. P. Liu,
Yue Fei,
Konstantinos N. Plataniotis,
Mahdi S. Hosseini
Abstract:
Differentiable Architecture Search (DARTS) has attracted considerable attention as a gradient-based neural architecture search method. Since the introduction of DARTS, there has been little work done on adapting the action space based on state-of-art architecture design principles for CNNs. In this work, we aim to address this gap by incrementally augmenting the DARTS search space with micro-desig…
▽ More
Differentiable Architecture Search (DARTS) has attracted considerable attention as a gradient-based neural architecture search method. Since the introduction of DARTS, there has been little work done on adapting the action space based on state-of-art architecture design principles for CNNs. In this work, we aim to address this gap by incrementally augmenting the DARTS search space with micro-design changes inspired by ConvNeXt and studying the trade-off between accuracy, evaluation layer count, and computational cost. We introduce the Pseudo-Inverted Bottleneck Conv (PIBConv) block intending to reduce the computational footprint of the inverted bottleneck block proposed in ConvNeXt. Our proposed architecture is much less sensitive to evaluation layer count and outperforms a DARTS network with similar size significantly, at layer counts as small as 2. Furthermore, with less layers, not only does it achieve higher accuracy with lower computational footprint (measured in GMACs) and parameter count, GradCAM comparisons show that our network can better detect distinctive features of target objects compared to DARTS. Code is available from https://github.com/mahdihosseini/PIBConv.
△ Less
Submitted 18 March, 2023; v1 submitted 31 December, 2022;
originally announced January 2023.
-
Exploiting Explainable Metrics for Augmented SGD
Authors:
Mahdi S. Hosseini,
Mathieu Tuli,
Konstantinos N. Plataniotis
Abstract:
Explaining the generalization characteristics of deep learning is an emerging topic in advanced machine learning. There are several unanswered questions about how learning under stochastic optimization really works and why certain strategies are better than others. In this paper, we address the following question: \textit{can we probe intermediate layers of a deep neural network to identify and qu…
▽ More
Explaining the generalization characteristics of deep learning is an emerging topic in advanced machine learning. There are several unanswered questions about how learning under stochastic optimization really works and why certain strategies are better than others. In this paper, we address the following question: \textit{can we probe intermediate layers of a deep neural network to identify and quantify the learning quality of each layer?} With this question in mind, we propose new explainability metrics that measure the redundant information in a network's layers using a low-rank factorization framework and quantify a complexity measure that is highly correlated with the generalization performance of a given optimizer, network, and dataset. We subsequently exploit these metrics to augment the Stochastic Gradient Descent (SGD) optimizer by adaptively adjusting the learning rate in each layer to improve in generalization performance. Our augmented SGD -- dubbed RMSGD -- introduces minimal computational overhead compared to SOTA methods and outperforms them by exhibiting strong generalization characteristics across application, architecture, and dataset.
△ Less
Submitted 30 March, 2022;
originally announced March 2022.
-
HistoKT: Cross Knowledge Transfer in Computational Pathology
Authors:
Ryan Zhang,
Jiadai Zhu,
Stephen Yang,
Mahdi S. Hosseini,
Angelo Genovese,
Lina Chen,
Corwyn Rowsell,
Savvas Damaskinos,
Sonal Varma,
Konstantinos N. Plataniotis
Abstract:
The lack of well-annotated datasets in computational pathology (CPath) obstructs the application of deep learning techniques for classifying medical images. %Since pathologist time is expensive, dataset curation is intrinsically difficult. Many CPath workflows involve transferring learned knowledge between various image domains through transfer learning. Currently, most transfer learning research…
▽ More
The lack of well-annotated datasets in computational pathology (CPath) obstructs the application of deep learning techniques for classifying medical images. %Since pathologist time is expensive, dataset curation is intrinsically difficult. Many CPath workflows involve transferring learned knowledge between various image domains through transfer learning. Currently, most transfer learning research follows a model-centric approach, tuning network parameters to improve transfer results over few datasets. In this paper, we take a data-centric approach to the transfer learning problem and examine the existence of generalizable knowledge between histopathological datasets. First, we create a standardization workflow for aggregating existing histopathological data. We then measure inter-domain knowledge by training ResNet18 models across multiple histopathological datasets, and cross-transferring between them to determine the quantity and quality of innate shared knowledge. Additionally, we use weight distillation to share knowledge between models without additional training. We find that hard to learn, multi-class datasets benefit most from pretraining, and a two stage learning framework incorporating a large source domain such as ImageNet allows for better utilization of smaller datasets. Furthermore, we find that weight distillation enables models trained on purely histopathological features to outperform models using external natural image data.
△ Less
Submitted 26 January, 2022;
originally announced January 2022.
-
P4AI: Approaching AI Ethics through Principlism
Authors:
Andre Fu,
Elisa Ding,
Mahdi S. Hosseini,
Konstantinos N. Plataniotis
Abstract:
The field of computer vision is rapidly evolving, particularly in the context of new methods of neural architecture design. These models contribute to (1) the Climate Crisis - increased CO2 emissions and (2) the Privacy Crisis - data leakage concerns. To address the often overlooked impact the Computer Vision (CV) community has on these crises, we outline a novel ethical framework, \textit{P4AI}:…
▽ More
The field of computer vision is rapidly evolving, particularly in the context of new methods of neural architecture design. These models contribute to (1) the Climate Crisis - increased CO2 emissions and (2) the Privacy Crisis - data leakage concerns. To address the often overlooked impact the Computer Vision (CV) community has on these crises, we outline a novel ethical framework, \textit{P4AI}: Principlism for AI, an augmented principlistic view of ethical dilemmas within AI. We then suggest using P4AI to make concrete recommendations to the community to mitigate the climate and privacy crises.
△ Less
Submitted 28 November, 2021;
originally announced November 2021.
-
NoFADE: Analyzing Diminishing Returns on CO2 Investment
Authors:
Andre Fu,
Justin Tran,
Andy Xie,
Jonathan Spraggett,
Elisa Ding,
Chang-Won Lee,
Kanav Singla,
Mahdi S. Hosseini,
Konstantinos N. Plataniotis
Abstract:
Climate change continues to be a pressing issue that currently affects society at-large. It is important that we as a society, including the Computer Vision (CV) community take steps to limit our impact on the environment. In this paper, we (a) analyze the effect of diminishing returns on CV methods, and (b) propose a \textit{``NoFADE''}: a novel entropy-based metric to quantify model--dataset--co…
▽ More
Climate change continues to be a pressing issue that currently affects society at-large. It is important that we as a society, including the Computer Vision (CV) community take steps to limit our impact on the environment. In this paper, we (a) analyze the effect of diminishing returns on CV methods, and (b) propose a \textit{``NoFADE''}: a novel entropy-based metric to quantify model--dataset--complexity relationships. We show that some CV tasks are reaching saturation, while others are almost fully saturated. In this light, NoFADE allows the CV community to compare models and datasets on a similar basis, establishing an agnostic platform.
△ Less
Submitted 28 November, 2021;
originally announced November 2021.
-
Towards Robust and Automatic Hyper-Parameter Tunning
Authors:
Mathieu Tuli,
Mahdi S. Hosseini,
Konstantinos N. Plataniotis
Abstract:
The task of hyper-parameter optimization (HPO) is burdened with heavy computational costs due to the intractability of optimizing both a model's weights and its hyper-parameters simultaneously. In this work, we introduce a new class of HPO method and explore how the low-rank factorization of the convolutional weights of intermediate layers of a convolutional neural network can be used to define an…
▽ More
The task of hyper-parameter optimization (HPO) is burdened with heavy computational costs due to the intractability of optimizing both a model's weights and its hyper-parameters simultaneously. In this work, we introduce a new class of HPO method and explore how the low-rank factorization of the convolutional weights of intermediate layers of a convolutional neural network can be used to define an analytical response surface for optimizing hyper-parameters, using only training data. We quantify how this surface behaves as a surrogate to model performance and can be solved using a trust-region search algorithm, which we call autoHyper. The algorithm outperforms state-of-the-art such as Bayesian Optimization and generalizes across model, optimizer, and dataset selection. Our code can be found at \url{https://github.com/MathieuTuli/autoHyper}.
△ Less
Submitted 12 December, 2021; v1 submitted 28 November, 2021;
originally announced November 2021.
-
In Search of Probeable Generalization Measures
Authors:
Jonathan Jaegerman,
Khalil Damouni,
Mahdi S. Hosseini,
Konstantinos N. Plataniotis
Abstract:
Understanding the generalization behaviour of deep neural networks is a topic of recent interest that has driven the production of many studies, notably the development and evaluation of generalization "explainability" measures that quantify model generalization ability. Generalization measures have also proven useful in the development of powerful layer-wise model tuning and optimization algorith…
▽ More
Understanding the generalization behaviour of deep neural networks is a topic of recent interest that has driven the production of many studies, notably the development and evaluation of generalization "explainability" measures that quantify model generalization ability. Generalization measures have also proven useful in the development of powerful layer-wise model tuning and optimization algorithms, though these algorithms require specific kinds of generalization measures which can probe individual layers. The purpose of this paper is to explore the neglected subtopic of probeable generalization measures; to establish firm ground for further investigations, and to inspire and guide the development of novel model tuning and optimization algorithms. We evaluate and compare measures, demonstrating effectiveness and robustness across model variations, dataset complexities, training hyperparameters, and training stages. We also introduce a new dataset of trained models and performance metrics, GenProb, for testing generalization measures, model tuning algorithms and optimization algorithms.
△ Less
Submitted 23 October, 2021;
originally announced October 2021.
-
CONetV2: Efficient Auto-Channel Size Optimization for CNNs
Authors:
Yi Ru Wang,
Samir Khaki,
Weihang Zheng,
Mahdi S. Hosseini,
Konstantinos N. Plataniotis
Abstract:
Neural Architecture Search (NAS) has been pivotal in finding optimal network configurations for Convolution Neural Networks (CNNs). While many methods explore NAS from a global search-space perspective, the employed optimization schemes typically require heavy computational resources. This work introduces a method that is efficient in computationally constrained environments by examining the micro…
▽ More
Neural Architecture Search (NAS) has been pivotal in finding optimal network configurations for Convolution Neural Networks (CNNs). While many methods explore NAS from a global search-space perspective, the employed optimization schemes typically require heavy computational resources. This work introduces a method that is efficient in computationally constrained environments by examining the micro-search space of channel size. In tackling channel-size optimization, we design an automated algorithm to extract the dependencies within different connected layers of the network. In addition, we introduce the idea of knowledge distillation, which enables preservation of trained weights, admist trials where the channel sizes are changing. Further, since the standard performance indicators (accuracy, loss) fail to capture the performance of individual network components (providing an overall network evaluation), we introduce a novel metric that highly correlates with test accuracy and enables analysis of individual network layers. Combining dependency extraction, metrics, and knowledge distillation, we introduce an efficient searching algorithm, with simulated annealing inspired stochasticity, and demonstrate its effectiveness in finding optimal architectures that outperform baselines by a large margin.
△ Less
Submitted 13 October, 2021;
originally announced October 2021.
-
Probeable DARTS with Application to Computational Pathology
Authors:
Sheyang Tang,
Mahdi S. Hosseini,
Lina Chen,
Sonal Varma,
Corwyn Rowsell,
Savvas Damaskinos,
Konstantinos N. Plataniotis,
Zhou Wang
Abstract:
AI technology has made remarkable achievements in computational pathology (CPath), especially with the help of deep neural networks. However, the network performance is highly related to architecture design, which commonly requires human experts with domain knowledge. In this paper, we combat this challenge with the recent advance in neural architecture search (NAS) to find an optimal network for…
▽ More
AI technology has made remarkable achievements in computational pathology (CPath), especially with the help of deep neural networks. However, the network performance is highly related to architecture design, which commonly requires human experts with domain knowledge. In this paper, we combat this challenge with the recent advance in neural architecture search (NAS) to find an optimal network for CPath applications. In particular, we use differentiable architecture search (DARTS) for its efficiency. We first adopt a probing metric to show that the original DARTS lacks proper hyperparameter tuning on the CIFAR dataset, and how the generalization issue can be addressed using an adaptive optimization strategy. We then apply our searching framework on CPath applications by searching for the optimum network architecture on a histological tissue type dataset (ADP). Results show that the searched network outperforms state-of-the-art networks in terms of prediction accuracy and computation complexity. We further conduct extensive experiments to demonstrate the transferability of the searched network to new CPath applications, the robustness against downscaled inputs, as well as the reliability of predictions.
△ Less
Submitted 15 August, 2021;
originally announced August 2021.
-
CONet: Channel Optimization for Convolutional Neural Networks
Authors:
Mahdi S. Hosseini,
Jia Shu Zhang,
Zhe Liu,
Andre Fu,
**gxuan Su,
Mathieu Tuli,
Sepehr Hosseini,
Arsh Kadakia,
Haoran Wang,
Konstantinos N. Plataniotis
Abstract:
Neural Architecture Search (NAS) has shifted network design from using human intuition to leveraging search algorithms guided by evaluation metrics. We study channel size optimization in convolutional neural networks (CNN) and identify the role it plays in model accuracy and complexity. Current channel size selection methods are generally limited by discrete sample spaces while suffering from manu…
▽ More
Neural Architecture Search (NAS) has shifted network design from using human intuition to leveraging search algorithms guided by evaluation metrics. We study channel size optimization in convolutional neural networks (CNN) and identify the role it plays in model accuracy and complexity. Current channel size selection methods are generally limited by discrete sample spaces while suffering from manual iteration and simple heuristics. To solve this, we introduce an efficient dynamic scaling algorithm -- CONet -- that automatically optimizes channel sizes across network layers for a given CNN. Two metrics -- "\textit{Rank}" and "\textit{Rank Average Slope}" -- are introduced to identify the information accumulated in training. The algorithm dynamically scales channel sizes up or down over a fixed searching phase. We conduct experiments on CIFAR10/100 and ImageNet datasets and show that CONet can find efficient and accurate architectures searched in ResNet, DARTS, and DARTS+ spaces that outperform their baseline models.
This document supersedes previously published paper in ICCV2021-NeurArch workshop. An additional section is included on manual scaling of channel size in CNNs to numerically validate of the metrics used in searching optimum channel configurations in CNNs.
△ Less
Submitted 7 April, 2022; v1 submitted 15 August, 2021;
originally announced August 2021.
-
Reconsidering CO2 emissions from Computer Vision
Authors:
Andre Fu,
Mahdi S. Hosseini,
Konstantinos N. Plataniotis
Abstract:
Climate change is a pressing issue that is currently affecting and will affect every part of our lives. It's becoming incredibly vital we, as a society, address the climate crisis as a universal effort, including those in the Computer Vision (CV) community. In this work, we analyze the total cost of CO2 emissions by breaking it into (1) the architecture creation cost and (2) the life-time evaluati…
▽ More
Climate change is a pressing issue that is currently affecting and will affect every part of our lives. It's becoming incredibly vital we, as a society, address the climate crisis as a universal effort, including those in the Computer Vision (CV) community. In this work, we analyze the total cost of CO2 emissions by breaking it into (1) the architecture creation cost and (2) the life-time evaluation cost. We show that over time, these costs are non-negligible and are having a direct impact on our future. Importantly, we conduct an ethical analysis of how the CV-community is unintentionally overlooking its own ethical AI principles by emitting this level of CO2. To address these concerns, we propose adding "enforcement" as a pillar of ethical AI and provide some recommendations for how architecture designers and broader CV community can curb the climate crisis.
△ Less
Submitted 18 April, 2021;
originally announced April 2021.
-
On Reversing Operator Choi-Davis-Jensen inequality
Authors:
Seyyed Saeid Hashemi Karouei,
Mohammad Sadegh Asgari,
Mohsen Shah Hosseini
Abstract:
In this paper, we first provide a better estimate of the second inequality in Hermite-Hadamard inequality. Next, we study the reverse of the celebrated Davis-Choi-Jensen's inequality. Our results are employed to establish a new bound for the operator Kantorovich inequality.
In this paper, we first provide a better estimate of the second inequality in Hermite-Hadamard inequality. Next, we study the reverse of the celebrated Davis-Choi-Jensen's inequality. Our results are employed to establish a new bound for the operator Kantorovich inequality.
△ Less
Submitted 6 April, 2021;
originally announced April 2021.
-
FocusLiteNN: High Efficiency Focus Quality Assessment for Digital Pathology
Authors:
Zhongling Wang,
Mahdi S. Hosseini,
Adyn Miles,
Konstantinos N. Plataniotis,
Zhou Wang
Abstract:
Out-of-focus microscopy lens in digital pathology is a critical bottleneck in high-throughput Whole Slide Image (WSI) scanning platforms, for which pixel-level automated Focus Quality Assessment (FQA) methods are highly desirable to help significantly accelerate the clinical workflows. Existing FQA methods include both knowledge-driven and data-driven approaches. While data-driven approaches such…
▽ More
Out-of-focus microscopy lens in digital pathology is a critical bottleneck in high-throughput Whole Slide Image (WSI) scanning platforms, for which pixel-level automated Focus Quality Assessment (FQA) methods are highly desirable to help significantly accelerate the clinical workflows. Existing FQA methods include both knowledge-driven and data-driven approaches. While data-driven approaches such as Convolutional Neural Network (CNN) based methods have shown great promises, they are difficult to use in practice due to their high computational complexity and lack of transferability. Here, we propose a highly efficient CNN-based model that maintains fast computations similar to the knowledge-driven methods without excessive hardware requirements such as GPUs. We create a training dataset using FocusPath which encompasses diverse tissue slides across nine different stain colors, where the stain diversity greatly helps the model to learn diverse color spectrum and tissue structures. In our attempt to reduce the CNN complexity, we find with surprise that even trimming down the CNN to the minimal level, it still achieves a highly competitive performance. We introduce a novel comprehensive evaluation dataset, the largest of its kind, annotated and compiled from TCGA repository for model assessment and comparison, for which the proposed method exhibits superior precision-speed trade-off when compared with existing knowledge-driven and data-driven FQA approaches.
△ Less
Submitted 1 October, 2020; v1 submitted 11 July, 2020;
originally announced July 2020.
-
AdaS: Adaptive Scheduling of Stochastic Gradients
Authors:
Mahdi S. Hosseini,
Konstantinos N. Plataniotis
Abstract:
The choice of step-size used in Stochastic Gradient Descent (SGD) optimization is empirically selected in most training procedures. Moreover, the use of scheduled learning techniques such as Step-Decaying, Cyclical-Learning, and Warmup to tune the step-size requires extensive practical experience--offering limited insight into how the parameters update--and is not consistent across applications. T…
▽ More
The choice of step-size used in Stochastic Gradient Descent (SGD) optimization is empirically selected in most training procedures. Moreover, the use of scheduled learning techniques such as Step-Decaying, Cyclical-Learning, and Warmup to tune the step-size requires extensive practical experience--offering limited insight into how the parameters update--and is not consistent across applications. This work attempts to answer a question of interest to both researchers and practitioners, namely \textit{"how much knowledge is gained in iterative training of deep neural networks?"} Answering this question introduces two useful metrics derived from the singular values of the low-rank factorization of convolution layers in deep neural networks. We introduce the notions of \textit{"knowledge gain"} and \textit{"map** condition"} and propose a new algorithm called Adaptive Scheduling (AdaS) that utilizes these derived metrics to adapt the SGD learning rate proportionally to the rate of change in knowledge gain over successive iterations. Experimentation reveals that, using the derived metrics, AdaS exhibits: (a) faster convergence and superior generalization over existing adaptive learning methods; and (b) lack of dependence on a validation set to determine when to stop training. Code is available at \url{https://github.com/mahdihosseini/AdaS}.
△ Less
Submitted 11 June, 2020;
originally announced June 2020.
-
A Comprehensive Analysis of Weakly-Supervised Semantic Segmentation in Different Image Domains
Authors:
Lyndon Chan,
Mahdi S. Hosseini,
Konstantinos N. Plataniotis
Abstract:
Recently proposed methods for weakly-supervised semantic segmentation have achieved impressive performance in predicting pixel classes despite being trained with only image labels which lack positional information. Because image annotations are cheaper and quicker to generate, weak supervision is more practical than full supervision for training segmentation algorithms. These methods have been pre…
▽ More
Recently proposed methods for weakly-supervised semantic segmentation have achieved impressive performance in predicting pixel classes despite being trained with only image labels which lack positional information. Because image annotations are cheaper and quicker to generate, weak supervision is more practical than full supervision for training segmentation algorithms. These methods have been predominantly developed to solve the background separation and partial segmentation problems presented by natural scene images and it is unclear whether they can be simply transferred to other domains with different characteristics, such as histopathology and satellite images, and still perform well. This paper evaluates state-of-the-art weakly-supervised semantic segmentation methods on natural scene, histopathology, and satellite image datasets and analyzes how to determine which method is most suitable for a given dataset. Our experiments indicate that histopathology and satellite images present a different set of problems for weakly-supervised semantic segmentation than natural scene images, such as ambiguous boundaries and class co-occurrence. Methods perform well for datasets they were developed on, but tend to perform poorly on other datasets. We present some practical techniques for these methods on unseen datasets and argue that more work is needed for a generalizable approach to weakly-supervised semantic segmentation. Our full code implementation is available on GitHub: https://github.com/lyndonchan/wsss-analysis.
△ Less
Submitted 17 October, 2020; v1 submitted 23 December, 2019;
originally announced December 2019.
-
Considering pricing and uncertainty in designing a reverse logistics network
Authors:
Mohsen Zamani,
Mahdi Abolghasemi,
Seyed Mohammad Seyed Hosseini,
Mir Saman Pishvaee
Abstract:
Companies try to maximize their profits by recovering returned products of highly uncertain quality and quantity. In this paper, a reverse logistics network for an Original Equipment Manufacturer (OEM) is presented. Returned products are selected for remanufacturing or scrap**, based on their quality and proportional prices are offered to customers. A Mixed Integer Non-linear Programming (MINLP)…
▽ More
Companies try to maximize their profits by recovering returned products of highly uncertain quality and quantity. In this paper, a reverse logistics network for an Original Equipment Manufacturer (OEM) is presented. Returned products are selected for remanufacturing or scrap**, based on their quality and proportional prices are offered to customers. A Mixed Integer Non-linear Programming (MINLP) model is proposed to determine the location of collection centers, the optimum price of returned products and the sorting policy. The risk in the objective function is measured using the Conditional Value at Risk (CVaR) metric. CVaR measures the risk of an investment in a conservative way by considering the maximum lost. The results are analyzed for various values of the risk parameters (α, and λ). These parameters indicate that considering risk affects prices, the classification of returned products, the location of collection centers and, consequently, the objective function. The model performs more conservatively when the weight of the CVaR part (λ) and the value of the confidence level α are increased. The results show that better profits are obtained when we take CVaR into account.
△ Less
Submitted 17 September, 2019;
originally announced September 2019.
-
On the Closed Form Expression of Elementary Symmetric Polynomials and the Inverse of Vandermonde Matrix
Authors:
Mahdi S. Hosseini,
Alfred Chen,
Konstantinos N. Plataniotis
Abstract:
Inverse Vandermonde matrix calculation is a long-standing problem to solve nonsingular linear system $Vc=b$ where the rows of a square matrix $V$ are constructed by progression of the power polynomials. It has many applications in scientific computing including interpolation, super-resolution, and construction of special matrices applied in cryptography. Despite its numerous applications, the matr…
▽ More
Inverse Vandermonde matrix calculation is a long-standing problem to solve nonsingular linear system $Vc=b$ where the rows of a square matrix $V$ are constructed by progression of the power polynomials. It has many applications in scientific computing including interpolation, super-resolution, and construction of special matrices applied in cryptography. Despite its numerous applications, the matrix is highly ill-conditioned where specialized treatments are considered for approximation such as conversion to Cauchy matrix, spectral decomposition, and algorithmic tailoring of the numerical solutions. In this paper, we propose a generalized algorithm that takes arbitrary pairwise (non-repetitive) sample nodes for solving inverse Vandermonde matrix. This is done in two steps: first, a highly balanced recursive algorithm is introduced with $\mathcal{O}(N)$ complexity to solve the combinatorics summation of the elementary symmetric polynomials; and second, a closed-form solution is tailored for inverse Vandermonde where the matrix' elements utilize this recursive summation for the inverse calculations. The numerical stability and accuracy of the proposed inverse method is analyzed through the spectral decomposition of the Frobenius companion matrix that associates with the corresponding Vandermonde matrix. The results show significant improvement over the state-of-the-art solutions using specific nodes such as $N$th roots of unity defined on the complex plane. A basic application in one dimensional interpolation problem is considered to demonstrate the utility of the proposed method for super-resolved signals.
△ Less
Submitted 17 September, 2019;
originally announced September 2019.
-
On the Operator Jensen Inequality for Convex Functions
Authors:
M. Shah Hosseini,
H. R. Moradi,
B. Moosavi
Abstract:
This paper is mainly devoted to studying operator Jensen inequality. More precisely, a new generalization of Jensen inequality and its reverse version for convex (not necessary operator convex) functions have been proved. Several special cases are discussed as well.
This paper is mainly devoted to studying operator Jensen inequality. More precisely, a new generalization of Jensen inequality and its reverse version for convex (not necessary operator convex) functions have been proved. Several special cases are discussed as well.
△ Less
Submitted 7 June, 2019; v1 submitted 30 May, 2019;
originally announced May 2019.
-
Focus Quality Assessment of High-Throughput Whole Slide Imaging in Digital Pathology
Authors:
Mahdi S. Hosseini,
Yueyang Zhang,
Lyndon Chan,
Konstantinos N. Plataniotis,
Jasper A. Z. Brawley-Hayes,
Savvas Damaskinos
Abstract:
One of the challenges facing the adoption of digital pathology workflows for clinical use is the need for automated quality control. As the scanners sometimes determine focus inaccurately, the resultant image blur deteriorates the scanned slide to the point of being unusable. Also, the scanned slide images tend to be extremely large when scanned at greater or equal 20X image resolution. Hence, for…
▽ More
One of the challenges facing the adoption of digital pathology workflows for clinical use is the need for automated quality control. As the scanners sometimes determine focus inaccurately, the resultant image blur deteriorates the scanned slide to the point of being unusable. Also, the scanned slide images tend to be extremely large when scanned at greater or equal 20X image resolution. Hence, for digital pathology to be clinically useful, it is necessary to use computational tools to quickly and accurately quantify the image focus quality and determine whether an image needs to be re-scanned. We propose a no-reference focus quality assessment metric specifically for digital pathology images, that operates by using a sum of even-derivative filter bases to synthesize a human visual system-like kernel, which is modeled as the inverse of the lens' point spread function. This kernel is then applied to a digital pathology image to modify high-frequency image information deteriorated by the scanner's optics and quantify the focus quality at the patch level. We show in several experiments that our method correlates better with ground-truth $z$-level data than other methods, and is more computationally efficient. We also extend our method to generate a local slide-level focus quality heatmap, which can be used for automated slide quality control, and demonstrate the utility of our method for clinical scan quality control by comparison with subjective slide quality scores.
△ Less
Submitted 14 November, 2018;
originally announced November 2018.
-
Convolutional Deblurring for Natural Imaging
Authors:
Mahdi S. Hosseini,
Konstantinos N. Plataniotis
Abstract:
In this paper, we propose a novel design of image deblurring in the form of one-shot convolution filtering that can directly convolve with naturally blurred images for restoration. The problem of optical blurring is a common disadvantage to many imaging applications that suffer from optical imperfections. Despite numerous deconvolution methods that blindly estimate blurring in either inclusive or…
▽ More
In this paper, we propose a novel design of image deblurring in the form of one-shot convolution filtering that can directly convolve with naturally blurred images for restoration. The problem of optical blurring is a common disadvantage to many imaging applications that suffer from optical imperfections. Despite numerous deconvolution methods that blindly estimate blurring in either inclusive or exclusive forms, they are practically challenging due to high computational cost and low image reconstruction quality. Both conditions of high accuracy and high speed are prerequisites for high-throughput imaging platforms in digital archiving. In such platforms, deblurring is required after image acquisition before being stored, previewed, or processed for high-level interpretation. Therefore, on-the-fly correction of such images is important to avoid possible time delays, mitigate computational expenses, and increase image perception quality. We bridge this gap by synthesizing a deconvolution kernel as a linear combination of Finite Impulse Response (FIR) even-derivative filters that can be directly convolved with blurry input images to boost the frequency fall-off of the Point Spread Function (PSF) associated with the optical blur. We employ a Gaussian low-pass filter to decouple the image denoising problem for image edge deblurring. Furthermore, we propose a blind approach to estimate the PSF statistics for two Gaussian and Laplacian models that are common in many imaging pipelines. Thorough experiments are designed to test and validate the efficiency of the proposed method using 2054 naturally blurred images across six imaging applications and seven state-of-the-art deconvolution methods.
△ Less
Submitted 19 July, 2019; v1 submitted 25 October, 2018;
originally announced October 2018.
-
Encoding Visual Sensitivity by MaxPol Convolution Filters for Image Sharpness Assessment
Authors:
Mahdi S. Hosseini,
Yueyang Zhang,
Konstantinos N. Plataniotis
Abstract:
In this paper, we propose a novel design of Human Visual System (HVS) response in a convolution filter form to decompose meaningful features that are closely tied with image sharpness level. No-reference (NR) Image sharpness assessment (ISA) techniques have emerged as the standard of image quality assessment in diverse imaging applications. Despite their high correlation with subjective scoring, t…
▽ More
In this paper, we propose a novel design of Human Visual System (HVS) response in a convolution filter form to decompose meaningful features that are closely tied with image sharpness level. No-reference (NR) Image sharpness assessment (ISA) techniques have emerged as the standard of image quality assessment in diverse imaging applications. Despite their high correlation with subjective scoring, they are challenging for practical considerations due to high computational cost and lack of scalability across different image blurs. We bridge this gap by synthesizing the HVS response as a linear combination of Finite Impulse Response (FIR) derivative filters to boost the falloff of high band frequency magnitudes in natural imaging paradigm. The numerical implementation of the HVS filter is carried out with MaxPol filter library that can be arbitrarily set for any differential orders and cutoff frequencies to balance out the estimation of informative features and noise sensitivities. We then design an innovative NR-ISA metric called `HVS-MaxPol' that (a) requires minimal computational cost, (b) produce high correlation accuracy with image blurriness, and (c) scales to assess synthetic and natural image blur. Specifically, the synthetic blur images are constructed by blurring the raw images using Gaussian filter, while natural blur is observed from real-life application such as motion, out-of-focus, etc. Furthermore, we create a natural benchmark database in digital pathology for validation of image focus quality in whole slide imaging systems called `FocusPath' consisting of 864 blurred images. Thorough experiments are designed to test and validate the efficiency of HVS-MaxPol across different blur databases and state-of-the-art NR-ISA metrics. The experiment result indicates that our metric has the best overall performance with respect to speed, accuracy and scalability.
△ Less
Submitted 18 March, 2019; v1 submitted 1 August, 2018;
originally announced August 2018.
-
An Alternative Estimate for the Numerical Radius of Hilbert Space Operators
Authors:
M. Shah Hosseini,
B. Moosavi,
H. R. Moradi
Abstract:
We give an alternative lower bound for the numerical radii of Hilbert space operators. As a by-product, we find conditions such that
\begin{equation*} ω\left(\left[\begin{array}{cc} 0 & R \\ S & 0 \end{array}\right]\right)=\frac{\Vert R \Vert +\Vert S\Vert }{2} \end{equation*} where $R, S \in \mathbb{B}(\mathcal{H})$.
We give an alternative lower bound for the numerical radii of Hilbert space operators. As a by-product, we find conditions such that
\begin{equation*} ω\left(\left[\begin{array}{cc} 0 & R \\ S & 0 \end{array}\right]\right)=\frac{\Vert R \Vert +\Vert S\Vert }{2} \end{equation*} where $R, S \in \mathbb{B}(\mathcal{H})$.
△ Less
Submitted 27 March, 2019; v1 submitted 24 May, 2018;
originally announced May 2018.
-
Finite Differences in Forward and Inverse Imaging Problems--MaxPol Design
Authors:
Mahdi S. Hosseini,
Konstantinos N. Plataniotis
Abstract:
A systematic and comprehensive framework for finite impulse response (FIR) lowpass/fullband derivative kernels is introduced in this paper. Closed form solutions of a number of derivative filters are obtained using the maximally flat technique to regulate the Fourier response of undetermined coefficients. The framework includes arbitrary parameter control methods that afford solutions for numerous…
▽ More
A systematic and comprehensive framework for finite impulse response (FIR) lowpass/fullband derivative kernels is introduced in this paper. Closed form solutions of a number of derivative filters are obtained using the maximally flat technique to regulate the Fourier response of undetermined coefficients. The framework includes arbitrary parameter control methods that afford solutions for numerous differential orders, variable polynomial accuracy, centralized/staggered schemes, and arbitrary side-shift nodes for boundary formulation. Using the proposed framework four different derivative matrix operators are introduced and their numerical stability is analyzed by studying their eigenvalues distribution in the complex plane. Their utility is studied by considering two important image processing problems, namely gradient surface reconstruction and image stitching. Experimentation indicates that the new derivative matrices not only outperform commonly used method but provide useful insights to the numerical issues in these two applications.
△ Less
Submitted 25 September, 2017;
originally announced September 2017.
-
Breit-Wigner distribution, quantum beats and GSI Anomaly
Authors:
M. S. Hosseini,
S. A. Alavi
Abstract:
The relationship between Breit-Wigner distribution as an underlying basis for decaying unstable quantum systems and GSI experiment (anomaly) has not been addressed properly in the literatures. We show that quantum beats can be obtained using a superposition of two Breit-Wigner distributions. This modified distribution can explain the GSI time anomaly with quantum beats resulting from the existence…
▽ More
The relationship between Breit-Wigner distribution as an underlying basis for decaying unstable quantum systems and GSI experiment (anomaly) has not been addressed properly in the literatures. We show that quantum beats can be obtained using a superposition of two Breit-Wigner distributions. This modified distribution can explain the GSI time anomaly with quantum beats resulting from the existence of two energy levels of the decaying ion.
△ Less
Submitted 3 October, 2019; v1 submitted 19 April, 2017;
originally announced April 2017.
-
Performance Enhancement of the Golden Code by Utilizing ORIOL Antenna
Authors:
Vahid Amiri,
Mohammadali Sadat Hosseini,
Ali Lotfi-Rezaabad,
Siamak Talebi
Abstract:
In this paper, a novel method is exposed to improve the performance of the Golden code, by using octagonal reconfigurable isolated orthogonal element (ORIOL) antennas, instead of a conventional microstrip patch antenna. The aforementioned antenna, should be employed in both the transmitter and the receiver sides, to approach the mentioned improvement. As a matter fact, in this paper, we recommend…
▽ More
In this paper, a novel method is exposed to improve the performance of the Golden code, by using octagonal reconfigurable isolated orthogonal element (ORIOL) antennas, instead of a conventional microstrip patch antenna. The aforementioned antenna, should be employed in both the transmitter and the receiver sides, to approach the mentioned improvement. As a matter fact, in this paper, we recommend space-time-polarization diversity instead of space-time singly; therefore it is obvious that by employing the aforementioned technique, the system obtains more strength against destructive fading. The simulations for different rates have confirmed that, utilizing ORIOL antenna outperforms patch microstrip one, which is roughly about $2$ to $3$ dB, according to the rates.
△ Less
Submitted 26 May, 2016;
originally announced May 2016.
-
High-Accuracy Total Variation for Compressed Video Sensing
Authors:
Mahdi S. Hosseini,
Konstantinos N. Plataniotis
Abstract:
Numerous total variation (TV) regularizers, engaged in image restoration problem, encode the gradients by means of simple $[-1,1]$ FIR filter. Despite its low computational processing, this filter severely deviates signal's high frequency components pertinent to edge/discontinuous information and cause several deficiency issues known as texture and geometric loss. This paper addresses this problem…
▽ More
Numerous total variation (TV) regularizers, engaged in image restoration problem, encode the gradients by means of simple $[-1,1]$ FIR filter. Despite its low computational processing, this filter severely deviates signal's high frequency components pertinent to edge/discontinuous information and cause several deficiency issues known as texture and geometric loss. This paper addresses this problem by proposing an alternative model to the TV regularization problem via high order accuracy differential FIR filters to preserve rapid transitions in signal recovery. A numerical encoding scheme is designed to extend the TV model into multidimensional representation (tensorial decomposition). We adopt this design to regulate the spatial and temporal redundancy in compressed video sensing problem to jointly recover frames from under-sampled measurements. We then seek the solution via alternating direction methods of multipliers and find a unique solution to quadratic minimization step with capability of handling different boundary conditions. The resulting algorithm uses much lower sampling rate and highly outperforms alternative state-of-the-art methods. This is evaluated both in terms of restoration accuracy and visual quality of the recovered frames.
△ Less
Submitted 4 March, 2014; v1 submitted 1 September, 2013;
originally announced September 2013.
-
Compressed Sensing Recoverability In Imaging Modalities
Authors:
Mahdi S. Hosseini,
Konstantinos N. Plataniotis
Abstract:
The paper introduces a framework for the recoverability analysis in compressive sensing for imaging applications such as CI cameras, rapid MRI and coded apertures. This is done using the fact that the Spherical Section Property (SSP) of a sensing matrix provides a lower bound for unique sparse recovery condition. The lower bound is evaluated for different sampling paradigms adopted from the aforem…
▽ More
The paper introduces a framework for the recoverability analysis in compressive sensing for imaging applications such as CI cameras, rapid MRI and coded apertures. This is done using the fact that the Spherical Section Property (SSP) of a sensing matrix provides a lower bound for unique sparse recovery condition. The lower bound is evaluated for different sampling paradigms adopted from the aforementioned imaging modalities. In particular, a platform is provided to analyze the well-posedness of sub-sampling patterns commonly used in practical scenarios. The effectiveness of the various designed patterns for sparse image recovery is studied through numerical experiments.
△ Less
Submitted 5 December, 2012;
originally announced December 2012.
-
Pigment Melanin: Pattern for Iris Recognition
Authors:
Mahdi S. Hosseini,
Babak N. Araabi,
Hamid Soltanian-Zadeh
Abstract:
Recognition of iris based on Visible Light (VL) imaging is a difficult problem because of the light reflection from the cornea. Nonetheless, pigment melanin provides a rich feature source in VL, unavailable in Near-Infrared (NIR) imaging. This is due to biological spectroscopy of eumelanin, a chemical not stimulated in NIR. In this case, a plausible solution to observe such patterns may be provi…
▽ More
Recognition of iris based on Visible Light (VL) imaging is a difficult problem because of the light reflection from the cornea. Nonetheless, pigment melanin provides a rich feature source in VL, unavailable in Near-Infrared (NIR) imaging. This is due to biological spectroscopy of eumelanin, a chemical not stimulated in NIR. In this case, a plausible solution to observe such patterns may be provided by an adaptive procedure using a variational technique on the image histogram. To describe the patterns, a shape analysis method is used to derive feature-code for each subject. An important question is how much the melanin patterns, extracted from VL, are independent of iris texture in NIR. With this question in mind, the present investigation proposes fusion of features extracted from NIR and VL to boost the recognition performance. We have collected our own database (UTIRIS) consisting of both NIR and VL images of 158 eyes of 79 individuals. This investigation demonstrates that the proposed algorithm is highly sensitive to the patterns of cromophores and improves the iris recognition rate.
△ Less
Submitted 29 November, 2009;
originally announced November 2009.