-
Attention versus Contrastive Learning of Tabular Data -- A Data-centric Benchmarking
Authors:
Shourav B. Rabbani,
Ivan V. Medri,
Manar D. Samad
Abstract:
Despite groundbreaking success in image and text learning, deep learning has not achieved significant improvements against traditional machine learning (ML) when it comes to tabular data. This performance gap underscores the need for data-centric treatment and benchmarking of learning algorithms. Recently, attention and contrastive learning breakthroughs have shifted computer vision and natural la…
▽ More
Despite groundbreaking success in image and text learning, deep learning has not achieved significant improvements against traditional machine learning (ML) when it comes to tabular data. This performance gap underscores the need for data-centric treatment and benchmarking of learning algorithms. Recently, attention and contrastive learning breakthroughs have shifted computer vision and natural language processing paradigms. However, the effectiveness of these advanced deep models on tabular data is sparsely studied using a few data sets with very large sample sizes, reporting mixed findings after benchmarking against a limited number of baselines. We argue that the heterogeneity of tabular data sets and selective baselines in the literature can bias the benchmarking outcomes. This article extensively evaluates state-of-the-art attention and contrastive learning methods on a wide selection of 28 tabular data sets (14 easy and 14 hard-to-classify) against traditional deep and machine learning. Our data-centric benchmarking demonstrates when traditional ML is preferred over deep learning and vice versa because no best learning method exists for all tabular data sets. Combining between-sample and between-feature attentions conquers the invincible traditional ML on tabular data sets by a significant margin but fails on high dimensional data, where contrastive learning takes a robust lead. While a hybrid attention-contrastive learning strategy mostly wins on hard-to-classify data sets, traditional methods are frequently superior on easy-to-classify data sets with presumably simpler decision boundaries. To the best of our knowledge, this is the first benchmarking paper with statistical analyses of attention and contrastive learning performances on a diverse selection of tabular data sets against traditional deep and machine learning baselines to facilitate further advances in this field.
△ Less
Submitted 8 January, 2024;
originally announced January 2024.
-
Between-Sample Relationship in Learning Tabular Data Using Graph and Attention Networks
Authors:
Shourav B. Rabbani,
Manar D. Samad
Abstract:
Traditional machine learning assumes samples in tabular data to be independent and identically distributed (i.i.d). This assumption may miss useful information within and between sample relationships in representation learning. This paper relaxes the i.i.d assumption to learn tabular data representations by incorporating between-sample relationships for the first time using graph neural networks (…
▽ More
Traditional machine learning assumes samples in tabular data to be independent and identically distributed (i.i.d). This assumption may miss useful information within and between sample relationships in representation learning. This paper relaxes the i.i.d assumption to learn tabular data representations by incorporating between-sample relationships for the first time using graph neural networks (GNN). We investigate our hypothesis using several GNNs and state-of-the-art (SOTA) deep attention models to learn the between-sample relationship on ten tabular data sets by comparing them to traditional machine learning methods. GNN methods show the best performance on tabular data with large feature-to-sample ratios. Our results reveal that attention-based GNN methods outperform traditional machine learning on five data sets and SOTA deep tabular learning methods on three data sets. Between-sample learning via GNN and deep attention methods yield the best classification accuracy on seven of the ten data sets. This suggests that the i.i.d assumption may not always hold for most tabular data sets.
△ Less
Submitted 11 June, 2023;
originally announced June 2023.
-
Interplay of viscosity and wettability controls fluid displacement in porous media
Authors:
Saideep Pavuluri,
Ran Holtzman,
Luqman Kazeem,
Malyah Mohammed,
Thomas Daniel Seers,
Harris Sajjad Rabbani
Abstract:
Direct numerical simulations are used to elucidate the interplay of wettability and fluid viscosities on immiscible fluid displacements in a heterogeneous porous medium.We classify the flow regimes based using qualitative and quantitative analysis into viscous fingering (low $M$), compact displacement (high $M$), and an intermediate transition regime ($M \approx 1$). We use stability analysis to o…
▽ More
Direct numerical simulations are used to elucidate the interplay of wettability and fluid viscosities on immiscible fluid displacements in a heterogeneous porous medium.We classify the flow regimes based using qualitative and quantitative analysis into viscous fingering (low $M$), compact displacement (high $M$), and an intermediate transition regime ($M \approx 1$). We use stability analysis to obtain theoretical phase boundaries between these regimes, which agree well with our analyses. At the macroscopic (sample) scale, we find that wettability strongly controls the threshold $M$ (at which the regimes change). At the pore scale, wettability alters the dominant pore-filling mechanism. At very small $M$ (viscous fingering regime), smaller pore spaces are preferentially invaded during imbibition, with flow of films of invading fluid along the pore walls. In contrast, during drainage, bursts result in filling of pores irrespective of their size. As $M$ increases, the effect of wettability decreases as cooperative filling becomes the dominant mechanism regardless of wettability. This suggest that for imbibition at a given contact angle, decreasing $M$ is associated with change in effective wetting from neutral-wet (cooperative filling) to strong-wet (film flow).
△ Less
Submitted 23 April, 2023;
originally announced April 2023.
-
Comparative analysis of five NH$_3$/air oxidation mechanisms
Authors:
Shahid Rabbani,
Dimitris M. Manias,
Dimitrios C. Kyritsis,
Dimitris A. Goussis
Abstract:
Five recently developed chemical kinetics mechanisms for ammonia oxidation are analysed and compared, in the context of homogeneous adiabatic autoignition. The analysis focuses on the ignition delay and is based on the explosive mode that is shown to drive the process. Using algorithmic tools based on the Computational Singular Perturbation algorithm, the reactions responsible for the generation o…
▽ More
Five recently developed chemical kinetics mechanisms for ammonia oxidation are analysed and compared, in the context of homogeneous adiabatic autoignition. The analysis focuses on the ignition delay and is based on the explosive mode that is shown to drive the process. Using algorithmic tools based on the Computational Singular Perturbation algorithm, the reactions responsible for the generation of the explosive mode are identified, along with the variables (species mass fractions and temperature) that associate the most to this mode. Comparison of these sets of reactions and variables, obtained for each mechanism, allows to correlate the differences in the predictive outcomes from the mechanisms with specific reactions. The major differences identified, which lead to different ignition delay times, relate to (i) the relative duration of chemical and thermal runaways (a sizeable chemical runaway develops only in some mechanisms) and (ii) the dominant chemistry during the chemical runaway (chemistry involving species with two nitrogen atoms is active only in some mechanisms). The major similarities identified refer to the thermal runaway and in particular to (i) the chemical activity, which is supported mainly by OH-producing reactions and by reactions producing their reactants and (ii) the thermal activity, which is dominated by strongly exothermic OH-consuming reactions.
△ Less
Submitted 7 April, 2023;
originally announced April 2023.
-
Deep Clustering of Tabular Data by Weighted Gaussian Distribution Learning
Authors:
Shourav B. Rabbani,
Ivan V. Medri,
Manar D. Samad
Abstract:
Deep learning methods are primarily proposed for supervised learning of images or text with limited applications to clustering problems. In contrast, tabular data with heterogeneous features pose unique challenges in representation learning, where deep learning has yet to replace traditional machine learning. This paper addresses these challenges in develo** one of the first deep clustering meth…
▽ More
Deep learning methods are primarily proposed for supervised learning of images or text with limited applications to clustering problems. In contrast, tabular data with heterogeneous features pose unique challenges in representation learning, where deep learning has yet to replace traditional machine learning. This paper addresses these challenges in develo** one of the first deep clustering methods for tabular data: Gaussian Cluster Embedding in Autoencoder Latent Space (G-CEALS). G-CEALS is an unsupervised deep clustering framework for learning the parameters of multivariate Gaussian cluster distributions by iteratively updating individual cluster weights. The G-CEALS method presents average rank orderings of 2.9(1.7) and 2.8(1.7) based on clustering accuracy and adjusted Rand index (ARI) scores on sixteen tabular data sets, respectively, and outperforms nine state-of-the-art clustering methods. G-CEALS substantially improves clustering performance compared to traditional K-means and GMM, which are still de facto methods for clustering tabular data. Similar computationally efficient and high-performing deep clustering frameworks are imperative to reap the myriad benefits of deep learning on tabular data over traditional machine learning.
△ Less
Submitted 17 May, 2024; v1 submitted 2 January, 2023;
originally announced January 2023.
-
Exploratory Data Analysis of Urdu Poetry
Authors:
Shahid Rabbani,
Zahid Ahmed Qureshi
Abstract:
The study presented here provides numerical insight into ghazal -- the most appreciated genre in Urdu poetry. Using 48,761 poetic works from 4,754 poets produced over a period of 800 years, this study explores the main features of Urdu ghazal that make it popular and admired more than other forms. A detailed explanation is provided as to the types of words used for expressing love, nature, birds,…
▽ More
The study presented here provides numerical insight into ghazal -- the most appreciated genre in Urdu poetry. Using 48,761 poetic works from 4,754 poets produced over a period of 800 years, this study explores the main features of Urdu ghazal that make it popular and admired more than other forms. A detailed explanation is provided as to the types of words used for expressing love, nature, birds, and flowers etc. Also considered is the way in which the poets addressed their loved ones in their poetry. The style of poetry is numerically analyzed using Multi Dimensional Scaling to reveal the lexical diversity and similarities/differences between the different poetic works that have drawn the attention of critics, such as Iqbal and Ghalib, Mir Taqi Mir and Mir Dard. The analysis produced here is particularly helpful for research in computational stylistics, neurocognitive poetics, and sentiment analysis.
△ Less
Submitted 3 December, 2021;
originally announced December 2021.
-
Enhancing Autoignition Characteristics: A Framework to Discover Fuel Additives and Making Predictions Using Machine Learning
Authors:
Shahid Rabbani
Abstract:
Combustion process can become more energy efficient and environment friendly if used with appropriate fuel additive. Discovery of fuel additive can be accelerated by applying hybrid approach of using of chemical kinetics and Machine Learning (ML). In this work, we present a framework that takes the robustness of Machine Learning and accuracy of chemical kinetics to predict the effect of fuel addit…
▽ More
Combustion process can become more energy efficient and environment friendly if used with appropriate fuel additive. Discovery of fuel additive can be accelerated by applying hybrid approach of using of chemical kinetics and Machine Learning (ML). In this work, we present a framework that takes the robustness of Machine Learning and accuracy of chemical kinetics to predict the effect of fuel additive on autoignition process. We present a case of making predictions for Ignition Delay Time (IDT) of biofuel n-butanol ($C_4H_9OH$) with several fuel additives. The proposed framework was able to predict IDT of autoignition with high accuracy when used with unseen additives. This framework highlights the potential of ML to exploit chemical mechanisms in exploring and develo** the fuel additives to obtain the desirable autoignition characteristics.
△ Less
Submitted 11 November, 2021;
originally announced November 2021.
-
Multi-level Stress Assessment from ECG in a Virtual Reality Environment using Multimodal Fusion
Authors:
Zeeshan Ahmad,
Suha Rabbani,
Muhammad Rehman Zafar,
Syem Ishaque,
Sridhar Krishnan,
Naimul Khan
Abstract:
ECG is an attractive option to assess stress in serious Virtual Reality (VR) applications due to its non-invasive nature. However, the existing Machine Learning (ML) models perform poorly. Moreover, existing studies only perform a binary stress assessment, while to develop a more engaging biofeedback-based application, multi-level assessment is necessary. Existing studies annotate and classify a s…
▽ More
ECG is an attractive option to assess stress in serious Virtual Reality (VR) applications due to its non-invasive nature. However, the existing Machine Learning (ML) models perform poorly. Moreover, existing studies only perform a binary stress assessment, while to develop a more engaging biofeedback-based application, multi-level assessment is necessary. Existing studies annotate and classify a single experience (e.g. watching a VR video) to a single stress level, which again prevents design of dynamic experiences where real-time in-game stress assessment can be utilized. In this paper, we report our findings on a new study on VR stress assessment, where three stress levels are assessed. ECG data was collected from 9 users experiencing a VR roller coaster. The VR experience was then manually labeled in 10-seconds segments to three stress levels by three raters. We then propose a novel multimodal deep fusion model utilizing spectrogram and 1D ECG that can provide a stress prediction from just a 1-second window. Experimental results demonstrate that the proposed model outperforms the classical HRV-based ML models (9% increase in accuracy) and baseline deep learning models (2.5% increase in accuracy). We also report results on the benchmark WESAD dataset to show the supremacy of the model.
△ Less
Submitted 9 July, 2021;
originally announced July 2021.
-
Analysis of evoked EMG using wavelet transformation
Authors:
Zaid Bin Mahbub,
J H Karami,
K Siddique-e Rabbani
Abstract:
Evoked EMG M-responses obtained from the thenar muscle in the palm by electrical stimulation of the median nerve demonstrate a well-established smooth bipolar shape for normal healthy subjects while kinks are observed in certain neurological disorders, particularly in cervical spondylotic neuropathy. A first differentiation failed to identify these kinks because of comparable values obtained for n…
▽ More
Evoked EMG M-responses obtained from the thenar muscle in the palm by electrical stimulation of the median nerve demonstrate a well-established smooth bipolar shape for normal healthy subjects while kinks are observed in certain neurological disorders, particularly in cervical spondylotic neuropathy. A first differentiation failed to identify these kinks because of comparable values obtained for normally rising and falling segments of the smooth regions, and due to noise. In this study, the usefulness of the wavelet transform (WT), that provides localized measures of non-stationary signals is investigated. The Haar WT was used to analyze a total of 36 M-responses recorded from the median nerves of 6 normal subjects (having smooth shape) and 12 subjects with assumed neurological disorders (having kinks), for two points of stimulation on the same nerve. Features in the time-scale representation of the M-responses were studied using WT to distinguish smooth M-responses from ones with kinks. Variations in the coefficient line of the WT were also studied to allow visualization of WT at different scales (inverse of frequency). The high and low frequency regions in the WT came out distinctively which helped identifications of kinks even of very subtle ones in the M-responses which were difficult to obtain using the differentiated signal. In conclusion, the wavelet analysis may be a technique of choice in identifying kinks in M-responses in relation to time, thus enhancing the accuracy of neurological diagnosis.
△ Less
Submitted 29 May, 2019;
originally announced May 2019.
