Showing 1–26 of 26 results for author: Shaban, A

Resonate-and-Fire Spiking Neurons for Target Detection and Hand Gesture Recognition: A Hybrid Approach

Authors: Ahmed Shaaban, Zeineb Chaabouni, Maximilian Strobel, Wolfgang Furtner, Robert Weigel, Fabian Lurz

Abstract: Hand gesture recognition using radar often relies on computationally expensive fast Fourier transforms. This paper proposes an alternative approach that bypasses fast Fourier transforms using resonate-and-fire neurons. These neurons directly detect the hand in the time-domain signal, eliminating the need for fast Fourier transforms to retrieve range information. Following detection, a simple Goert… ▽ More Hand gesture recognition using radar often relies on computationally expensive fast Fourier transforms. This paper proposes an alternative approach that bypasses fast Fourier transforms using resonate-and-fire neurons. These neurons directly detect the hand in the time-domain signal, eliminating the need for fast Fourier transforms to retrieve range information. Following detection, a simple Goertzel algorithm is employed to extract five key features, eliminating the need for a second fast Fourier transform. These features are then fed into a recurrent neural network, achieving an accuracy of 98.21% for classifying five gestures. The proposed approach demonstrates competitive performance with reduced complexity compared to traditional methods △ Less

Submitted 22 May, 2024; originally announced May 2024.

Journal ref: 2024 Smart Systems Integration Conference and Exhibition (SSI)

MedPromptX: Grounded Multimodal Prompting for Chest X-ray Diagnosis

Authors: Mai A. Shaaban, Adnan Khan, Mohammad Yaqub

Abstract: Chest X-ray images are commonly used for predicting acute and chronic cardiopulmonary conditions, but efforts to integrate them with structured clinical data face challenges due to incomplete electronic health records (EHR). This paper introduces MedPromptX, the first model to integrate multimodal large language models (MLLMs), few-shot prompting (FP) and visual grounding (VG) to combine imagery w… ▽ More Chest X-ray images are commonly used for predicting acute and chronic cardiopulmonary conditions, but efforts to integrate them with structured clinical data face challenges due to incomplete electronic health records (EHR). This paper introduces MedPromptX, the first model to integrate multimodal large language models (MLLMs), few-shot prompting (FP) and visual grounding (VG) to combine imagery with EHR data for chest X-ray diagnosis. A pre-trained MLLM is utilized to complement the missing EHR information, providing a comprehensive understanding of patients' medical history. Additionally, FP reduces the necessity for extensive training of MLLMs while effectively tackling the issue of hallucination. Nevertheless, the process of determining the optimal number of few-shot examples and selecting high-quality candidates can be burdensome, yet it profoundly influences model performance. Hence, we propose a new technique that dynamically refines few-shot data for real-time adjustment to new patient scenarios. Moreover, VG aids in focusing the model's attention on relevant regions of interest in X-ray images, enhancing the identification of abnormalities. We release MedPromptX-VQA, a new in-context visual question answering dataset encompassing interleaved image and EHR data derived from MIMIC-IV and MIMIC-CXR databases. Results demonstrate the SOTA performance of MedPromptX, achieving an 11% improvement in F1-score compared to the baselines. Code and data are available at https://github.com/BioMedIA-MBZUAI/MedPromptX △ Less

Submitted 28 March, 2024; v1 submitted 22 March, 2024; originally announced March 2024.

Truncated Polynomial Expansion-Based Detection in Massive MIMO: A Model-Driven Deep Learning Approach

Authors: Kazem Izadinasab, Ahmed Wagdy Shaban, Oussama Damen

Abstract: In this paper, we propose a deep learning (DL)-based approach for efficiently computing the inverse of Hermitian matrices using truncated polynomial expansion (TPE). Our model-driven approach involves optimizing the coefficients of the TPE during an offline training procedure for a given number of TPE terms. We apply this method to signal detection in uplink massive multiple-input multiple-output… ▽ More In this paper, we propose a deep learning (DL)-based approach for efficiently computing the inverse of Hermitian matrices using truncated polynomial expansion (TPE). Our model-driven approach involves optimizing the coefficients of the TPE during an offline training procedure for a given number of TPE terms. We apply this method to signal detection in uplink massive multiple-input multiple-output (MIMO) systems, where the matrix inverse operation required by linear detectors, such as zero-forcing (ZF) and minimum mean square error (MMSE), is approximated using TPE. Our simulation results demonstrate that the proposed learned TPE-based method outperforms the conventional TPE method with optimal coefficients in terms of asymptotic convergence speed and reduces the computational complexity of the online detection stage, albeit at the expense of the offline training stage. However, the limited number of trainable parameters leads to a swift offline training process. △ Less

Submitted 19 February, 2024; originally announced February 2024.

Comments: 5 pages, 2 figures, 2 tables

Fine-Tuned Large Language Models for Symptom Recognition from Spanish Clinical Text

Authors: Mai A. Shaaban, Abbas Akkasi, Adnan Khan, Majid Komeili, Mohammad Yaqub

Abstract: The accurate recognition of symptoms in clinical reports is significantly important in the fields of healthcare and biomedical natural language processing. These entities serve as essential building blocks for clinical information extraction, enabling retrieval of critical medical insights from vast amounts of textual data. Furthermore, the ability to identify and categorize these entities is fund… ▽ More The accurate recognition of symptoms in clinical reports is significantly important in the fields of healthcare and biomedical natural language processing. These entities serve as essential building blocks for clinical information extraction, enabling retrieval of critical medical insights from vast amounts of textual data. Furthermore, the ability to identify and categorize these entities is fundamental for develo** advanced clinical decision support systems, aiding healthcare professionals in diagnosis and treatment planning. In this study, we participated in SympTEMIST, a shared task on the detection of symptoms, signs and findings in Spanish medical documents. We combine a set of large language models fine-tuned with the data released by the organizers. △ Less

Submitted 28 January, 2024; originally announced January 2024.

Improving Pseudo-labelling and Enhancing Robustness for Semi-Supervised Domain Generalization

Authors: Adnan Khan, Mai A. Shaaban, Muhammad Haris Khan

Abstract: Beyond attaining domain generalization (DG), visual recognition models should also be data-efficient during learning by leveraging limited labels. We study the problem of Semi-Supervised Domain Generalization (SSDG) which is crucial for real-world applications like automated healthcare. SSDG requires learning a cross-domain generalizable model when the given training data is only partially labelle… ▽ More Beyond attaining domain generalization (DG), visual recognition models should also be data-efficient during learning by leveraging limited labels. We study the problem of Semi-Supervised Domain Generalization (SSDG) which is crucial for real-world applications like automated healthcare. SSDG requires learning a cross-domain generalizable model when the given training data is only partially labelled. Empirical investigations reveal that the DG methods tend to underperform in SSDG settings, likely because they are unable to exploit the unlabelled data. Semi-supervised learning (SSL) shows improved but still inferior results compared to fully-supervised learning. A key challenge, faced by the best-performing SSL-based SSDG methods, is selecting accurate pseudo-labels under multiple domain shifts and reducing overfitting to source domains under limited labels. In this work, we propose new SSDG approach, which utilizes a novel uncertainty-guided pseudo-labelling with model averaging (UPLM). Our uncertainty-guided pseudo-labelling (UPL) uses model uncertainty to improve pseudo-labelling selection, addressing poor model calibration under multi-source unlabelled data. The UPL technique, enhanced by our novel model averaging (MA) strategy, mitigates overfitting to source domains with limited labels. Extensive experiments on key representative DG datasets suggest that our method demonstrates effectiveness against existing methods. Our code and chosen labelled data seeds are available on GitHub: https://github.com/Adnan-Khan7/UPLM △ Less

Submitted 25 January, 2024; originally announced January 2024.

LiDAR-UDA: Self-ensembling Through Time for Unsupervised LiDAR Domain Adaptation

Authors: Amirreza Shaban, JoonHo Lee, Sanghun Jung, Xiangyun Meng, Byron Boots

Abstract: We introduce LiDAR-UDA, a novel two-stage self-training-based Unsupervised Domain Adaptation (UDA) method for LiDAR segmentation. Existing self-training methods use a model trained on labeled source data to generate pseudo labels for target data and refine the predictions via fine-tuning the network on the pseudo labels. These methods suffer from domain shifts caused by different LiDAR sensor conf… ▽ More We introduce LiDAR-UDA, a novel two-stage self-training-based Unsupervised Domain Adaptation (UDA) method for LiDAR segmentation. Existing self-training methods use a model trained on labeled source data to generate pseudo labels for target data and refine the predictions via fine-tuning the network on the pseudo labels. These methods suffer from domain shifts caused by different LiDAR sensor configurations in the source and target domains. We propose two techniques to reduce sensor discrepancy and improve pseudo label quality: 1) LiDAR beam subsampling, which simulates different LiDAR scanning patterns by randomly drop** beams; 2) cross-frame ensembling, which exploits temporal consistency of consecutive frames to generate more reliable pseudo labels. Our method is simple, generalizable, and does not incur any extra inference cost. We evaluate our method on several public LiDAR datasets and show that it outperforms the state-of-the-art methods by more than $3.9\%$ mIoU on average for all scenarios. Code will be available at https://github.com/JHLee0513/LiDARUDA. △ Less

Submitted 23 September, 2023; originally announced September 2023.

Comments: Accepted ICCV 2023 (Oral)

PECon: Contrastive Pretraining to Enhance Feature Alignment between CT and EHR Data for Improved Pulmonary Embolism Diagnosis

Authors: Santosh Sanjeev, Salwa K. Al Khatib, Mai A. Shaaban, Ibrahim Almakky, Vijay Ram Papineni, Mohammad Yaqub

Abstract: Previous deep learning efforts have focused on improving the performance of Pulmonary Embolism(PE) diagnosis from Computed Tomography (CT) scans using Convolutional Neural Networks (CNN). However, the features from CT scans alone are not always sufficient for the diagnosis of PE. CT scans along with electronic heath records (EHR) can provide a better insight into the patients condition and can lea… ▽ More Previous deep learning efforts have focused on improving the performance of Pulmonary Embolism(PE) diagnosis from Computed Tomography (CT) scans using Convolutional Neural Networks (CNN). However, the features from CT scans alone are not always sufficient for the diagnosis of PE. CT scans along with electronic heath records (EHR) can provide a better insight into the patients condition and can lead to more accurate PE diagnosis. In this paper, we propose Pulmonary Embolism Detection using Contrastive Learning (PECon), a supervised contrastive pretraining strategy that employs both the patients CT scans as well as the EHR data, aiming to enhance the alignment of feature representations between the two modalities and leverage information to improve the PE diagnosis. In order to achieve this, we make use of the class labels and pull the sample features of the same class together, while pushing away those of the other class. Results show that the proposed work outperforms the existing techniques and achieves state-of-the-art performance on the RadFusion dataset with an F1-score of 0.913, accuracy of 0.90 and an AUROC of 0.943. Furthermore, we also explore the explainability of our approach in comparison to other methods. Our code is publicly available at https://github.com/BioMedIA-MBZUAI/PECon. △ Less

Submitted 27 August, 2023; originally announced August 2023.

TerrainNet: Visual Modeling of Complex Terrain for High-speed, Off-road Navigation

Authors: Xiangyun Meng, Nathan Hatch, Alexander Lambert, Anqi Li, Nolan Wagener, Matthew Schmittle, JoonHo Lee, Wentao Yuan, Zoey Chen, Samuel Deng, Greg Okopal, Dieter Fox, Byron Boots, Amirreza Shaban

Abstract: Effective use of camera-based vision systems is essential for robust performance in autonomous off-road driving, particularly in the high-speed regime. Despite success in structured, on-road settings, current end-to-end approaches for scene prediction have yet to be successfully adapted for complex outdoor terrain. To this end, we present TerrainNet, a vision-based terrain perception system for se… ▽ More Effective use of camera-based vision systems is essential for robust performance in autonomous off-road driving, particularly in the high-speed regime. Despite success in structured, on-road settings, current end-to-end approaches for scene prediction have yet to be successfully adapted for complex outdoor terrain. To this end, we present TerrainNet, a vision-based terrain perception system for semantic and geometric terrain prediction for aggressive, off-road navigation. The approach relies on several key insights and practical considerations for achieving reliable terrain modeling. The network includes a multi-headed output representation to capture fine- and coarse-grained terrain features necessary for estimating traversability. Accurate depth estimation is achieved using self-supervised depth completion with multi-view RGB and stereo inputs. Requirements for real-time performance and fast inference speeds are met using efficient, learned image feature projections. Furthermore, the model is trained on a large-scale, real-world off-road dataset collected across a variety of diverse outdoor environments. We show how TerrainNet can also be used for costmap prediction and provide a detailed framework for integration into a planning module. We demonstrate the performance of TerrainNet through extensive comparison to current state-of-the-art baselines for camera-only scene prediction. Finally, we showcase the effectiveness of integrating TerrainNet within a complete autonomous-driving stack by conducting a real-world vehicle test in a challenging off-road scenario. △ Less

Submitted 29 May, 2023; v1 submitted 28 March, 2023; originally announced March 2023.

OptBA: Optimizing Hyperparameters with the Bees Algorithm for Improved Medical Text Classification

Authors: Mai A. Shaaban, Mariam Kashkash, Maryam Alghfeli, Adham Ibrahim

Abstract: One of the main challenges in the field of deep learning is obtaining the optimal model hyperparameters. The search for optimal hyperparameters usually hinders the progress of solutions to real-world problems such as healthcare. Previous solutions have been proposed, but they can still get stuck in local optima. To overcome this hurdle, we propose OptBA to automatically fine-tune the hyperparamete… ▽ More One of the main challenges in the field of deep learning is obtaining the optimal model hyperparameters. The search for optimal hyperparameters usually hinders the progress of solutions to real-world problems such as healthcare. Previous solutions have been proposed, but they can still get stuck in local optima. To overcome this hurdle, we propose OptBA to automatically fine-tune the hyperparameters of deep learning models by leveraging the Bees Algorithm, which is a recent promising swarm intelligence algorithm. In this paper, the optimization problem of OptBA is to maximize the accuracy in classifying ailments using medical text, where initial hyperparameters are iteratively adjusted by specific criteria. Experimental results demonstrate a noteworthy enhancement in accuracy with approximately 1.4%. This outcome highlights the effectiveness of the proposed mechanism in addressing the critical issue of hyperparameter optimization and its potential impact on advancing solutions for healthcare. The code is available publicly at \url{https://github.com/Mai-CS/OptBA}. △ Less

Submitted 29 June, 2024; v1 submitted 14 March, 2023; originally announced March 2023.

Adaptive shape optimization with NURBS designs and PHT-splines for solution approximation in time-harmonic acoustics

Authors: Javier Videla, Ahmed Mostafa Shaaban, Elena Atroshchenko

Abstract: Geometry Independent Field approximaTion (GIFT) was proposed as a generalization of Isogeometric analysis (IGA), where different types of splines are used for the parameterization of the computational domain and approximation of the unknown solution. GIFT with Non-Uniform Rational B-Splines (NUBRS) for the geometry and PHT-splines for the solution approximation were successfully applied to problem… ▽ More Geometry Independent Field approximaTion (GIFT) was proposed as a generalization of Isogeometric analysis (IGA), where different types of splines are used for the parameterization of the computational domain and approximation of the unknown solution. GIFT with Non-Uniform Rational B-Splines (NUBRS) for the geometry and PHT-splines for the solution approximation were successfully applied to problems of time-harmonic acoustics, where it was shown that in some cases, adaptive PHT-spline mesh yields highly accurate solutions at lower computational cost than methods with uniform refinement. Therefore, it is of interest to investigate performance of GIFT for shape optimization problems, where NURBS are used to model the boundary with their control points being the design variables and PHT-splines are used to approximate the solution adaptively to the boundary changes during the optimization process. In this work we demonstrate the application of GIFT for 2D acoustic shape optimization problems and, using three benchmark examples, we show that the method yields accurate solutions with significant computational savings in terms of the number of degrees of freedom and computational time. △ Less

Submitted 10 October, 2022; originally announced October 2022.

Low Power Neuromorphic EMG Gesture Classification

Authors: Sai Sukruth Bezugam, Ahmed Shaban, Manan Suri

Abstract: EMG (Electromyograph) signal based gesture recognition can prove vital for applications such as smart wearables and bio-medical neuro-prosthetic control. Spiking Neural Networks (SNNs) are promising for low-power, real-time EMG gesture recognition, owing to their inherent spike/event driven spatio-temporal dynamics. In literature, there are limited demonstrations of neuromorphic hardware implement… ▽ More EMG (Electromyograph) signal based gesture recognition can prove vital for applications such as smart wearables and bio-medical neuro-prosthetic control. Spiking Neural Networks (SNNs) are promising for low-power, real-time EMG gesture recognition, owing to their inherent spike/event driven spatio-temporal dynamics. In literature, there are limited demonstrations of neuromorphic hardware implementation (at full chip/board/system scale) for EMG gesture classification. Moreover, most literature attempts exploit primitive SNNs based on LIF (Leaky Integrate and Fire) neurons. In this work, we address the aforementioned gaps with following key contributions: (1) Low-power, high accuracy demonstration of EMG-signal based gesture recognition using neuromorphic Recurrent Spiking Neural Networks (RSNN). In particular, we propose a multi-time scale recurrent neuromorphic system based on special double-exponential adaptive threshold (DEXAT) neurons. Our network achieves state-of-the-art classification accuracy (90%) while using ~53% lesser neurons than best reported prior art on Roshambo EMG dataset. (2) A new multi-channel spike encoder scheme for efficient processing of real-valued EMG data on neuromorphic systems. (3) Unique multi-compartment methodology to implement complex adaptive neurons on Intel's dedicated neuromorphic Loihi chip is shown. (4) RSNN implementation on Loihi (Nahuku 32) achieves significant energy/latency benefits of ~983X/19X compared to GPU for batch size as 50. △ Less

Submitted 4 June, 2022; originally announced June 2022.

Comments: 3 Pages, 5 figures, 1 table

CAFA: Class-Aware Feature Alignment for Test-Time Adaptation

Authors: Sanghun Jung, Jungsoo Lee, Nanhee Kim, Amirreza Shaban, Byron Boots, Jaegul Choo

Abstract: Despite recent advancements in deep learning, deep neural networks continue to suffer from performance degradation when applied to new data that differs from training data. Test-time adaptation (TTA) aims to address this challenge by adapting a model to unlabeled data at test time. TTA can be applied to pretrained networks without modifying their training procedures, enabling them to utilize a wel… ▽ More Despite recent advancements in deep learning, deep neural networks continue to suffer from performance degradation when applied to new data that differs from training data. Test-time adaptation (TTA) aims to address this challenge by adapting a model to unlabeled data at test time. TTA can be applied to pretrained networks without modifying their training procedures, enabling them to utilize a well-formed source distribution for adaptation. One possible approach is to align the representation space of test samples to the source distribution (\textit{i.e.,} feature alignment). However, performing feature alignment in TTA is especially challenging in that access to labeled source data is restricted during adaptation. That is, a model does not have a chance to learn test data in a class-discriminative manner, which was feasible in other adaptation tasks (\textit{e.g.,} unsupervised domain adaptation) via supervised losses on the source data. Based on this observation, we propose a simple yet effective feature alignment loss, termed as Class-Aware Feature Alignment (CAFA), which simultaneously 1) encourages a model to learn target representations in a class-discriminative manner and 2) effectively mitigates the distribution shifts at test time. Our method does not require any hyper-parameters or additional losses, which are required in previous approaches. We conduct extensive experiments on 6 different datasets and show our proposed method consistently outperforms existing baselines. △ Less

Submitted 3 September, 2023; v1 submitted 31 May, 2022; originally announced June 2022.

Deep convolutional forest: a dynamic deep ensemble approach for spam detection in text

Authors: Mai A. Shaaban, Yasser F. Hassan, Shawkat K. Guirguis

Abstract: The increase in people's use of mobile messaging services has led to the spread of social engineering attacks like phishing, considering that spam text is one of the main factors in the dissemination of phishing attacks to steal sensitive data such as credit cards and passwords. In addition, rumors and incorrect medical information regarding the COVID-19 pandemic are widely shared on social media… ▽ More The increase in people's use of mobile messaging services has led to the spread of social engineering attacks like phishing, considering that spam text is one of the main factors in the dissemination of phishing attacks to steal sensitive data such as credit cards and passwords. In addition, rumors and incorrect medical information regarding the COVID-19 pandemic are widely shared on social media leading to people's fear and confusion. Thus, filtering spam content is vital to reduce risks and threats. Previous studies relied on machine learning and deep learning approaches for spam classification, but these approaches have two limitations. Machine learning models require manual feature engineering, whereas deep neural networks require a high computational cost. This paper introduces a dynamic deep ensemble model for spam detection that adjusts its complexity and extracts features automatically. The proposed model utilizes convolutional and pooling layers for feature extraction along with base classifiers such as random forests and extremely randomized trees for classifying texts into spam or legitimate ones. Moreover, the model employs ensemble learning procedures like boosting and bagging. As a result, the model achieved high precision, recall, f1-score and accuracy of 98.38\%. △ Less

Submitted 29 April, 2022; v1 submitted 10 October, 2021; originally announced October 2021.

Few-shot Weakly-Supervised Object Detection via Directional Statistics

Authors: Amirreza Shaban, Amir Rahimi, Thalaiyasingam Ajanthan, Byron Boots, Richard Hartley

Abstract: Detecting novel objects from few examples has become an emerging topic in computer vision recently. However, these methods need fully annotated training images to learn new object categories which limits their applicability in real world scenarios such as field robotics. In this work, we propose a probabilistic multiple instance learning approach for few-shot Common Object Localization (COL) and f… ▽ More Detecting novel objects from few examples has become an emerging topic in computer vision recently. However, these methods need fully annotated training images to learn new object categories which limits their applicability in real world scenarios such as field robotics. In this work, we propose a probabilistic multiple instance learning approach for few-shot Common Object Localization (COL) and few-shot Weakly Supervised Object Detection (WSOD). In these tasks, only image-level labels, which are much cheaper to acquire, are available. We find that operating on features extracted from the last layer of a pre-trained Faster-RCNN is more effective compared to previous episodic learning based few-shot COL methods. Our model simultaneously learns the distribution of the novel objects and localizes them via expectation-maximization steps. As a probabilistic model, we employ von Mises-Fisher (vMF) distribution which captures the semantic information better than Gaussian distribution when applied to the pre-trained embedding space. When the novel objects are localized, we utilize them to learn a linear appearance model to detect novel classes in new images. Our extensive experiments show that the proposed method, despite being simple, outperforms strong baselines in few-shot COL and WSOD, as well as large-scale WSOD tasks. △ Less

Submitted 25 March, 2021; originally announced March 2021.

A Robotic 3D Perception System for Operating Room Environment Awareness

Authors: Zhaoshuo Li, Amirreza Shaban, Jean-Gabriel Simard, Dinesh Rabindran, Simon DiMaio, Omid Mohareri

Abstract: Purpose: We describe a 3D multi-view perception system for the da Vinci surgical system to enable Operating room (OR) scene understanding and context awareness. Methods: Our proposed system is comprised of four Time-of-Flight (ToF) cameras rigidly attached to strategic locations on the daVinci Xi patient side cart (PSC). The cameras are registered to the robot's kinematic chain by performing a o… ▽ More Purpose: We describe a 3D multi-view perception system for the da Vinci surgical system to enable Operating room (OR) scene understanding and context awareness. Methods: Our proposed system is comprised of four Time-of-Flight (ToF) cameras rigidly attached to strategic locations on the daVinci Xi patient side cart (PSC). The cameras are registered to the robot's kinematic chain by performing a one-time calibration routine and therefore, information from all cameras can be fused and represented in one common coordinate frame. Based on this architecture, a multi-view 3D scene semantic segmentation algorithm is created to enable recognition of common and salient objects/equipment and surgical activities in a da Vinci OR. Our proposed 3D semantic segmentation method has been trained and validated on a novel densely annotated dataset that has been captured from clinical scenarios. Results: The results show that our proposed architecture has acceptable registration error ($3.3\%\pm1.4\%$ of object-camera distance) and can robustly improve scene segmentation performance (mean Intersection Over Union - mIOU) for less frequently appearing classes ($\ge 0.013$) compared to a single-view method. Conclusion: We present the first dynamic multi-view perception system with a novel segmentation architecture, which can be used as a building block technology for applications such as surgical workflow analysis, automation of surgical sub-tasks and advanced guidance systems. △ Less

Submitted 30 March, 2020; v1 submitted 20 March, 2020; originally announced March 2020.

Comments: Accepted in IPCAI 2020

MSC Class: 65D19

Pairwise Similarity Knowledge Transfer for Weakly Supervised Object Localization

Authors: Amir Rahimi, Amirreza Shaban, Thalaiyasingam Ajanthan, Richard Hartley, Byron Boots

Abstract: Weakly Supervised Object Localization (WSOL) methods only require image level labels as opposed to expensive bounding box annotations required by fully supervised algorithms. We study the problem of learning localization model on target classes with weakly supervised image labels, helped by a fully annotated source dataset. Typically, a WSOL model is first trained to predict class generic objectne… ▽ More Weakly Supervised Object Localization (WSOL) methods only require image level labels as opposed to expensive bounding box annotations required by fully supervised algorithms. We study the problem of learning localization model on target classes with weakly supervised image labels, helped by a fully annotated source dataset. Typically, a WSOL model is first trained to predict class generic objectness scores on an off-the-shelf fully supervised source dataset and then it is progressively adapted to learn the objects in the weakly supervised target dataset. In this work, we argue that learning only an objectness function is a weak form of knowledge transfer and propose to learn a classwise pairwise similarity function that directly compares two input proposals as well. The combined localization model and the estimated object annotations are jointly learned in an alternating optimization paradigm as is typically done in standard WSOL methods. In contrast to the existing work that learns pairwise similarities, our approach optimizes a unified objective with convergence guarantee and it is computationally efficient for large-scale applications. Experiments on the COCO and ILSVRC 2013 detection datasets show that the performance of the localization model improves significantly with the inclusion of pairwise similarity function. For instance, in the ILSVRC dataset, the Correct Localization (CorLoc) performance improves from 72.8% to 78.2% which is a new state-of-the-art for WSOL task in the context of knowledge transfer. △ Less

Submitted 19 July, 2020; v1 submitted 18 March, 2020; originally announced March 2020.

Comments: ECCV 2020. formerly "In Defense of Graph Inference Algorithms for Weakly Supervised Object Localization"

Intra Order-preserving Functions for Calibration of Multi-Class Neural Networks

Authors: Amir Rahimi, Amirreza Shaban, Ching-An Cheng, Richard Hartley, Byron Boots

Abstract: Predicting calibrated confidence scores for multi-class deep networks is important for avoiding rare but costly mistakes. A common approach is to learn a post-hoc calibration function that transforms the output of the original network into calibrated confidence scores while maintaining the network's accuracy. However, previous post-hoc calibration techniques work only with simple calibration funct… ▽ More Predicting calibrated confidence scores for multi-class deep networks is important for avoiding rare but costly mistakes. A common approach is to learn a post-hoc calibration function that transforms the output of the original network into calibrated confidence scores while maintaining the network's accuracy. However, previous post-hoc calibration techniques work only with simple calibration functions, potentially lacking sufficient representation to calibrate the complex function landscape of deep networks. In this work, we aim to learn general post-hoc calibration functions that can preserve the top-k predictions of any deep network. We call this family of functions intra order-preserving functions. We propose a new neural network architecture that represents a class of intra order-preserving functions by combining common neural network components. Additionally, we introduce order-invariant and diagonal sub-families, which can act as regularization for better generalization when the training data size is small. We show the effectiveness of the proposed method across a wide range of datasets and classifiers. Our method outperforms state-of-the-art post-hoc calibration methods, namely temperature scaling and Dirichlet calibration, in several evaluation metrics for the task. △ Less

Submitted 23 October, 2020; v1 submitted 15 March, 2020; originally announced March 2020.

Comments: NeurIPS 2020

MMTM: Multimodal Transfer Module for CNN Fusion

Authors: Hamid Reza Vaezi Joze, Amirreza Shaban, Michael L. Iuzzolino, Kazuhito Koishida

Abstract: In late fusion, each modality is processed in a separate unimodal Convolutional Neural Network (CNN) stream and the scores of each modality are fused at the end. Due to its simplicity late fusion is still the predominant approach in many state-of-the-art multimodal applications. In this paper, we present a simple neural network module for leveraging the knowledge from multiple modalities in convol… ▽ More In late fusion, each modality is processed in a separate unimodal Convolutional Neural Network (CNN) stream and the scores of each modality are fused at the end. Due to its simplicity late fusion is still the predominant approach in many state-of-the-art multimodal applications. In this paper, we present a simple neural network module for leveraging the knowledge from multiple modalities in convolutional neural networks. The propose unit, named Multimodal Transfer Module (MMTM), can be added at different levels of the feature hierarchy, enabling slow modality fusion. Using squeeze and excitation operations, MMTM utilizes the knowledge of multiple modalities to recalibrate the channel-wise features in each CNN stream. Despite other intermediate fusion methods, the proposed module could be used for feature modality fusion in convolution layers with different spatial dimensions. Another advantage of the proposed method is that it could be added among unimodal branches with minimum changes in the their network architectures, allowing each branch to be initialized with existing pretrained weights. Experimental results show that our framework improves the recognition accuracy of well-known multimodal networks. We demonstrate state-of-the-art or competitive performance on four datasets that span the task domains of dynamic hand gesture recognition, speech enhancement, and action recognition with RGB and body joints. △ Less

Submitted 30 March, 2020; v1 submitted 19 November, 2019; originally announced November 2019.

Journal ref: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020

Learning to Find Common Objects Across Few Image Collections

Authors: Amirreza Shaban, Amir Rahimi, Shray Bansal, Stephen Gould, Byron Boots, Richard Hartley

Abstract: Given a collection of bags where each bag is a set of images, our goal is to select one image from each bag such that the selected images are from the same object class. We model the selection as an energy minimization problem with unary and pairwise potential functions. Inspired by recent few-shot learning algorithms, we propose an approach to learn the potential functions directly from the data.… ▽ More Given a collection of bags where each bag is a set of images, our goal is to select one image from each bag such that the selected images are from the same object class. We model the selection as an energy minimization problem with unary and pairwise potential functions. Inspired by recent few-shot learning algorithms, we propose an approach to learn the potential functions directly from the data. Furthermore, we propose a fast greedy inference algorithm for energy minimization. We evaluate our approach on few-shot common object recognition as well as object co-localization tasks. Our experiments show that learning the pairwise and unary terms greatly improves the performance of the model over several well-known methods for these tasks. The proposed greedy optimization algorithm achieves performance comparable to state-of-the-art structured inference algorithms while being ~10 times faster. The code is publicly available on https://github.com/haamoon/finding_common_object. △ Less

Submitted 16 August, 2019; v1 submitted 29 April, 2019; originally announced April 2019.

Comments: ICCV 2019

Truncated Back-propagation for Bilevel Optimization

Authors: Amirreza Shaban, Ching-An Cheng, Nathan Hatch, Byron Boots

Abstract: Bilevel optimization has been recently revisited for designing and analyzing algorithms in hyperparameter tuning and meta learning tasks. However, due to its nested structure, evaluating exact gradients for high-dimensional problems is computationally challenging. One heuristic to circumvent this difficulty is to use the approximate gradient given by performing truncated back-propagation through t… ▽ More Bilevel optimization has been recently revisited for designing and analyzing algorithms in hyperparameter tuning and meta learning tasks. However, due to its nested structure, evaluating exact gradients for high-dimensional problems is computationally challenging. One heuristic to circumvent this difficulty is to use the approximate gradient given by performing truncated back-propagation through the iterative optimization procedure that solves the lower-level problem. Although promising empirical performance has been reported, its theoretical properties are still unclear. In this paper, we analyze the properties of this family of approximate gradients and establish sufficient conditions for convergence. We validate this on several hyperparameter tuning and meta learning tasks. We find that optimization with the approximate gradient computed using few-step back-propagation often performs comparably to optimization with the exact gradient, while requiring far less memory and half the computation time. △ Less

Submitted 5 April, 2019; v1 submitted 24 October, 2018; originally announced October 2018.

Journal ref: The International Conference on Artificial Intelligence and Statistics (AISTATS) 2019

Deep Forward and Inverse Perceptual Models for Tracking and Prediction

Authors: Alexander Lambert, Amirreza Shaban, Amit Raj, Zhen Liu, Byron Boots

Abstract: We consider the problems of learning forward models that map state to high-dimensional images and inverse models that map high-dimensional images to state in robotics. Specifically, we present a perceptual model for generating video frames from state with deep networks, and provide a framework for its use in tracking and prediction tasks. We show that our proposed model greatly outperforms standar… ▽ More We consider the problems of learning forward models that map state to high-dimensional images and inverse models that map high-dimensional images to state in robotics. Specifically, we present a perceptual model for generating video frames from state with deep networks, and provide a framework for its use in tracking and prediction tasks. We show that our proposed model greatly outperforms standard deconvolutional methods and GANs for image generation, producing clear, photo-realistic images. We also develop a convolutional neural network model for state estimation and compare the result to an Extended Kalman Filter to estimate robot trajectories. We validate all models on a real robotic system. △ Less

Submitted 19 May, 2018; v1 submitted 30 October, 2017; originally announced October 2017.

Comments: 8 pages, International Conference on Robotics and Automation (ICRA) 2018

One-Shot Learning for Semantic Segmentation

Authors: Amirreza Shaban, Shray Bansal, Zhen Liu, Irfan Essa, Byron Boots

Abstract: Low-shot learning methods for image classification support learning from sparse data. We extend these techniques to support dense semantic image segmentation. Specifically, we train a network that, given a small set of annotated images, produces parameters for a Fully Convolutional Network (FCN). We use this FCN to perform dense pixel-level prediction on a test image for the new semantic class. Ou… ▽ More Low-shot learning methods for image classification support learning from sparse data. We extend these techniques to support dense semantic image segmentation. Specifically, we train a network that, given a small set of annotated images, produces parameters for a Fully Convolutional Network (FCN). We use this FCN to perform dense pixel-level prediction on a test image for the new semantic class. Our architecture shows a 25% relative meanIoU improvement compared to the best baseline methods for one-shot segmentation on unseen classes in the PASCAL VOC 2012 dataset and is at least 3 times faster. △ Less

Submitted 11 September, 2017; originally announced September 2017.

Comments: To appear in the proceedings of the British Machine Vision Conference (BMVC) 2017. The code is available at https://github.com/lzzcd001/OSLSM

On the Degrees of Freedom of SISO X-networks with Synergistic Alternating Channel State Information at transmitters

Authors: Ahmed Wagdy Shaban, Amr El-Keyi, Tamer Khattab, Mohammed Nafie

Abstract: In this paper, we consider the two-user single-input single-output (SISO) X-channel and $K$-user SISO X-network in fast fading environment. It is assumed that the transmitters have access to synergistic alternating channel state information (CSI). Specifically, the CSIT alternates between three states, namely, perfect, delayed and no-CSIT, in a certain manner to enable these states to work togethe… ▽ More In this paper, we consider the two-user single-input single-output (SISO) X-channel and $K$-user SISO X-network in fast fading environment. It is assumed that the transmitters have access to synergistic alternating channel state information (CSI). Specifically, the CSIT alternates between three states, namely, perfect, delayed and no-CSIT, in a certain manner to enable these states to work together cooperatively. These states are associated with fractions of time denoted by $λ_P$, $λ_D$ \text{and} $λ_N$, respectively. For the two-user $X$-channel, simple upper bound is developed to prove the tightness of the achievability result of $4/3$ DoF under a certain distribution of the availability of three CSIT states for $Λ(λ_P=1/3, λ_D= 1/3, λ_N=1/3)$. For the $K$-user $X$-network, it is shown that the sum Degrees of freedom (DoF) is at least $2K/(K + 1)$, using two-phase transmission schemes over finite symbols channel extension and under the same distribution of the availability of $Λ(λ_P=1/3, λ_D= 1/3, λ_N=1/3)$.This achievability result, can be considered as a tight lower bound, coincides with the best lower bound known for the same network but with partial output feedback in stead of alternating CSIT. Hence, we show that the role of synergistic alternating CSIT with distribution $Λ(1/3,1/3,1/3)$ is equivalent to the partial output feedback. Also, this lower bound is strictly better than the best lower bound known for the case of delayed CSI assumption for all values of $K$. All the proposed transmission schemes are based on two phases transmission strategy, namely, interference creation and interference resurrection, which exploit the synergy of instantaneous CSI and delay CSIT to retrospectively align interference in the subsequent channel uses. △ Less

Submitted 23 May, 2016; originally announced May 2016.

Comments: 30 pages, 5 Figures, To be submitted soon isA. A part of this paper was published on IEEE ISIT 2014 and available, arXiv:1404.6348

Confidential Algorithm for Golden Cryptography Using Haar Wavelet

Authors: Marghny H. Mohamed, Yousef B. Mahdy, Wafaa Abd El-Wahed Shaban

Abstract: One of the most important consideration techniques when one want to solve the protecting of digital signal is the golden matrix. The golden matrices can be used for creation of a new kind of cryptography called the golden cryptography. Many research papers have proved that the method is very fast and simple for technical realization and can be used for cryptographic protection of digital signals.… ▽ More One of the most important consideration techniques when one want to solve the protecting of digital signal is the golden matrix. The golden matrices can be used for creation of a new kind of cryptography called the golden cryptography. Many research papers have proved that the method is very fast and simple for technical realization and can be used for cryptographic protection of digital signals. In this paper, we introduce a technique of encryption based on combination of haar wavelet and golden matrix. These combinations carry out after compression data by adaptive Huffman code to reduce data size and remove redundant data. This process will provide multisecurity services. In addition Message Authentication Code (MAC) technique can be used to provide authentication and the integrity of this scheme. The proposed scheme is accomplished through five stages, the compression data, key generation, encryption stage, the decryption stage and decompression at communication ends. △ Less

Submitted 15 January, 2015; originally announced January 2015.

Cascading Randomized Weighted Majority: A New Online Ensemble Learning Algorithm

Authors: Mohammadzaman Zamani, Hamid Beigy, Amirreza Shaban

Abstract: With the increasing volume of data in the world, the best approach for learning from this data is to exploit an online learning algorithm. Online ensemble methods are online algorithms which take advantage of an ensemble of classifiers to predict labels of data. Prediction with expert advice is a well-studied problem in the online ensemble learning literature. The Weighted Majority algorithm and t… ▽ More With the increasing volume of data in the world, the best approach for learning from this data is to exploit an online learning algorithm. Online ensemble methods are online algorithms which take advantage of an ensemble of classifiers to predict labels of data. Prediction with expert advice is a well-studied problem in the online ensemble learning literature. The Weighted Majority algorithm and the randomized weighted majority (RWM) are the most well-known solutions to this problem, aiming to converge to the best expert. Since among some expert, the best one does not necessarily have the minimum error in all regions of data space, defining specific regions and converging to the best expert in each of these regions will lead to a better result. In this paper, we aim to resolve this defect of RWM algorithms by proposing a novel online ensemble algorithm to the problem of prediction with expert advice. We propose a cascading version of RWM to achieve not only better experimental results but also a better error bound for sufficiently large datasets. △ Less

Submitted 2 February, 2015; v1 submitted 3 March, 2014; originally announced March 2014.

Comments: 15 pages, 3 figures

Local Similarities, Global Coding: An Algorithm for Feature Coding and its Applications

Authors: Amirreza Shaban, Hamid R. Rabiee, Mahyar Najibi

Abstract: Data coding as a building block of several image processing algorithms has been received great attention recently. Indeed, the importance of the locality assumption in coding approaches is studied in numerous works and several methods are proposed based on this concept. We probe this assumption and claim that taking the similarity between a data point and a more global set of anchor points does no… ▽ More Data coding as a building block of several image processing algorithms has been received great attention recently. Indeed, the importance of the locality assumption in coding approaches is studied in numerous works and several methods are proposed based on this concept. We probe this assumption and claim that taking the similarity between a data point and a more global set of anchor points does not necessarily weaken the coding method as long as the underlying structure of the anchor points are taken into account. Based on this fact, we propose to capture this underlying structure by assuming a random walker over the anchor points. We show that our method is a fast approximate learning algorithm based on the diffusion map kernel. The experiments on various datasets show that making different state-of-the-art coding algorithms aware of this structure boosts them in different learning tasks. △ Less

Submitted 5 March, 2014; v1 submitted 23 November, 2013; originally announced November 2013.