-
kNN Classification of Malware Data Dependency Graph Features
Authors:
John Musgrave,
Anca Ralescu
Abstract:
Feature resolution impacts the ability of classifiers to make explainable inferences when applied to malware classification. We explore classification based on features constructed from data dependency graphs, and present results from k-Nearest Neighbors (kNN) classifiers. Our study demonstrates that classification based on a novel feature representation not only yields high accuracy, but also inc…
▽ More
Feature resolution impacts the ability of classifiers to make explainable inferences when applied to malware classification. We explore classification based on features constructed from data dependency graphs, and present results from k-Nearest Neighbors (kNN) classifiers. Our study demonstrates that classification based on a novel feature representation not only yields high accuracy, but also increases explainability in inference, as features of data dependency are directly representative of program behavior. We present classification results using the Microsoft Kaggle 2015 malware dataset which was processed with a novel approach to feature extraction and representation. We show that non-parametric approaches to classification in the metric space are able to obtain classification accuracy of 87.5\% when applied to multi-class classification in the Kaggle malware dataset. Additionally, similarity in the metric space can be calculated directly without prior training. Our results provide evidence that data dependency graphs accurately capture both semantic and structural information.
△ Less
Submitted 4 June, 2024;
originally announced June 2024.
-
Joint Self-Supervised and Supervised Contrastive Learning for Multimodal MRI Data: Towards Predicting Abnormal Neurodevelopment
Authors:
Zhiyuan Li,
Hailong Li,
Anca L. Ralescu,
Jonathan R. Dillman,
Mekibib Altaye,
Kim M. Cecil,
Nehal A. Parikh,
Lili He
Abstract:
The integration of different imaging modalities, such as structural, diffusion tensor, and functional magnetic resonance imaging, with deep learning models has yielded promising outcomes in discerning phenotypic characteristics and enhancing disease diagnosis. The development of such a technique hinges on the efficient fusion of heterogeneous multimodal features, which initially reside within dist…
▽ More
The integration of different imaging modalities, such as structural, diffusion tensor, and functional magnetic resonance imaging, with deep learning models has yielded promising outcomes in discerning phenotypic characteristics and enhancing disease diagnosis. The development of such a technique hinges on the efficient fusion of heterogeneous multimodal features, which initially reside within distinct representation spaces. Naively fusing the multimodal features does not adequately capture the complementary information and could even produce redundancy. In this work, we present a novel joint self-supervised and supervised contrastive learning method to learn the robust latent feature representation from multimodal MRI data, allowing the projection of heterogeneous features into a shared common space, and thereby amalgamating both complementary and analogous information across various modalities and among similar subjects. We performed a comparative analysis between our proposed method and alternative deep multimodal learning approaches. Through extensive experiments on two independent datasets, the results demonstrated that our method is significantly superior to several other deep multimodal learning methods in predicting abnormal neurodevelopment. Our method has the capability to facilitate computer-aided diagnosis within clinical practice, harnessing the power of multimodal data.
△ Less
Submitted 22 December, 2023;
originally announced December 2023.
-
On the ability of CNNs to extract color invariant intensity based features for image classification
Authors:
Pradyumna Elavarthi,
James Lee,
Anca Ralescu
Abstract:
Convolutional neural networks (CNNs) have demonstrated remarkable success in vision-related tasks. However, their susceptibility to failing when inputs deviate from the training distribution is well-documented. Recent studies suggest that CNNs exhibit a bias toward texture instead of object shape in image classification tasks, and that background information may affect predictions. This paper inve…
▽ More
Convolutional neural networks (CNNs) have demonstrated remarkable success in vision-related tasks. However, their susceptibility to failing when inputs deviate from the training distribution is well-documented. Recent studies suggest that CNNs exhibit a bias toward texture instead of object shape in image classification tasks, and that background information may affect predictions. This paper investigates the ability of CNNs to adapt to different color distributions in an image while maintaining context and background. The results of our experiments on modified MNIST and FashionMNIST data demonstrate that changes in color can substantially affect classification accuracy. The paper explores the effects of various regularization techniques on generalization error across datasets and proposes a minor architectural modification utilizing the dropout regularization in a novel way that enhances model reliance on color-invariant intensity-based features for improved classification accuracy. Overall, this work contributes to ongoing efforts to understand the limitations and challenges of CNNs in image classification tasks and offers potential solutions to enhance their performance.
△ Less
Submitted 12 July, 2023;
originally announced July 2023.
-
Semantic Segmentation of Porosity in 4D Spatio-Temporal X-ray μCT of Titanium Coated Ni wires using Deep Learning
Authors:
Pradyumna Elavarthi,
Arun Bhattacharjee,
Ashley Paz y Puente,
Anca Ralescu
Abstract:
A fully convolutional neural network was used to measure the evolution of the volume fraction of two different Kirkendall pores during the homogenization of Ti coated Ni wires. Traditional methods like Otsus thresholding and the largest connected component analysis were used to obtain the masks for training the segmentation model. Once trained, the model was used to semantically segment the two ty…
▽ More
A fully convolutional neural network was used to measure the evolution of the volume fraction of two different Kirkendall pores during the homogenization of Ti coated Ni wires. Traditional methods like Otsus thresholding and the largest connected component analysis were used to obtain the masks for training the segmentation model. Once trained, the model was used to semantically segment the two types of pores at different stages in their evolution. Masks of the pores predicted by the network were then used to measure the volume fraction of porosity at 0 mins, 240 mins, and 480 mins of homogenization. The model predicted an increase in porosity for one type of pore and a decrease in porosity for another type of pore due to pore sintering, and it achieved an F1 Score of 0.95.
△ Less
Submitted 24 June, 2023;
originally announced June 2023.
-
Learning Empirical Bregman Divergence for Uncertain Distance Representation
Authors:
Zhiyuan Li,
Ziru Liu,
Anna Zou,
Anca L. Ralescu
Abstract:
Deep metric learning techniques have been used for visual representation in various supervised and unsupervised learning tasks through learning embeddings of samples with deep networks. However, classic approaches, which employ a fixed distance metric as a similarity function between two embeddings, may lead to suboptimal performance for capturing the complex data distribution. The Bregman diverge…
▽ More
Deep metric learning techniques have been used for visual representation in various supervised and unsupervised learning tasks through learning embeddings of samples with deep networks. However, classic approaches, which employ a fixed distance metric as a similarity function between two embeddings, may lead to suboptimal performance for capturing the complex data distribution. The Bregman divergence generalizes measures of various distance metrics and arises throughout many fields of deep metric learning. In this paper, we first show how deep metric learning loss can arise from the Bregman divergence. We then introduce a novel method for learning empirical Bregman divergence directly from data based on parameterizing the convex function underlying the Bregman divergence with a deep learning setting. We further experimentally show that our approach performs effectively on five popular public datasets compared to other SOTA deep metric learning methods, particularly for pattern recognition problems.
△ Less
Submitted 15 May, 2023; v1 submitted 16 April, 2023;
originally announced April 2023.
-
A Novel Collaborative Self-Supervised Learning Method for Radiomic Data
Authors:
Zhiyuan Li,
Hailong Li,
Anca L. Ralescu,
Jonathan R. Dillman,
Nehal A. Parikh,
Lili He
Abstract:
The computer-aided disease diagnosis from radiomic data is important in many medical applications. However, develo** such a technique relies on annotating radiological images, which is a time-consuming, labor-intensive, and expensive process. In this work, we present the first novel collaborative self-supervised learning method to solve the challenge of insufficient labeled radiomic data, whose…
▽ More
The computer-aided disease diagnosis from radiomic data is important in many medical applications. However, develo** such a technique relies on annotating radiological images, which is a time-consuming, labor-intensive, and expensive process. In this work, we present the first novel collaborative self-supervised learning method to solve the challenge of insufficient labeled radiomic data, whose characteristics are different from text and image data. To achieve this, we present two collaborative pretext tasks that explore the latent pathological or biological relationships between regions of interest and the similarity and dissimilarity information between subjects. Our method collaboratively learns the robust latent feature representations from radiomic data in a self-supervised manner to reduce human annotation efforts, which benefits the disease diagnosis. We compared our proposed method with other state-of-the-art self-supervised learning methods on a simulation study and two independent datasets. Extensive experimental results demonstrated that our method outperforms other self-supervised learning methods on both classification and regression tasks. With further refinement, our method shows the potential advantage in automatic disease diagnosis with large-scale unlabeled data available.
△ Less
Submitted 20 February, 2023;
originally announced February 2023.
-
Learning Generalized Hybrid Proximity Representation for Image Recognition
Authors:
Zhiyuan Li,
Anca Ralescu
Abstract:
Recently, deep metric learning techniques received attention, as the learned distance representations are useful to capture the similarity relationship among samples and further improve the performance of various of supervised or unsupervised learning tasks. We propose a novel supervised metric learning method that can learn the distance metrics in both geometric and probabilistic space for image…
▽ More
Recently, deep metric learning techniques received attention, as the learned distance representations are useful to capture the similarity relationship among samples and further improve the performance of various of supervised or unsupervised learning tasks. We propose a novel supervised metric learning method that can learn the distance metrics in both geometric and probabilistic space for image recognition. In contrast to the previous metric learning methods which usually focus on learning the distance metrics in Euclidean space, our proposed method is able to learn better distance representation in a hybrid approach. To achieve this, we proposed a Generalized Hybrid Metric Loss (GHM-Loss) to learn the general hybrid proximity features from the image data by controlling the trade-off between geometric proximity and probabilistic proximity. To evaluate the effectiveness of our method, we first provide theoretical derivations and proofs of the proposed loss function, then we perform extensive experiments on two public datasets to show the advantage of our method compared to other state-of-the-art metric learning methods.
△ Less
Submitted 15 April, 2023; v1 submitted 31 January, 2023;
originally announced January 2023.
-
Latent Semantic Structure in Malicious Programs
Authors:
John Musgrave,
Temesguen Messay-Kebede,
David Kapp,
Anca Ralescu
Abstract:
Latent Semantic Analysis is a method of matrix decomposition used for discovering topics and topic weights in natural language documents. This study uses Latent Semantic Analysis to analyze the composition of binaries of malicious programs. The semantic representation of the term frequency vector representation yields a set of topics, each topic being a composition of terms. The vectors and topics…
▽ More
Latent Semantic Analysis is a method of matrix decomposition used for discovering topics and topic weights in natural language documents. This study uses Latent Semantic Analysis to analyze the composition of binaries of malicious programs. The semantic representation of the term frequency vector representation yields a set of topics, each topic being a composition of terms. The vectors and topics were evaluated quantitatively using a spatial representation. This semantic analysis provides a more abstract representation of the program derived from its term frequency analysis. We use a metric space to represent a program as a collection of vectors, and a distance metric to evaluate their similarity within a topic. The segmentation of the vectors in this dataset provides increased resolution into the program structure.
△ Less
Submitted 31 October, 2022;
originally announced October 2022.
-
A Novel Feature Representation for Malware Classification
Authors:
John Musgrave,
Temesguen Messay-Kebede,
David Kapp,
Anca Ralescu
Abstract:
In this study we have presented a novel feature representation for malicious programs that can be used for malware classification. We have shown how to construct the features in a bottom-up approach, and analyzed the overlap of malicious and benign programs in terms of their components. We have shown that our method of analysis offers an increase in feature resolution that is descriptive of data m…
▽ More
In this study we have presented a novel feature representation for malicious programs that can be used for malware classification. We have shown how to construct the features in a bottom-up approach, and analyzed the overlap of malicious and benign programs in terms of their components. We have shown that our method of analysis offers an increase in feature resolution that is descriptive of data movement in comparison to tf-idf features.
△ Less
Submitted 18 October, 2022;
originally announced October 2022.
-
Empirical Network Structure of Malicious Programs
Authors:
John Musgrave,
Alina Campan,
Temesguen Messay-Kebede,
David Kapp,
Anca Ralescu
Abstract:
A modern binary executable is a composition of various networks. Control flow graphs are commonly used to represent an executable program in labeled datasets used for classification tasks. Control flow and term representations are widely adopted, but provide only a partial view of program semantics. This study is an empirical analysis of the networks composing malicious binaries in order to provid…
▽ More
A modern binary executable is a composition of various networks. Control flow graphs are commonly used to represent an executable program in labeled datasets used for classification tasks. Control flow and term representations are widely adopted, but provide only a partial view of program semantics. This study is an empirical analysis of the networks composing malicious binaries in order to provide a complete representation of the structural properties of a program. This is accomplished by the measurement of structural properties of program networks in a malicious binary executable dataset. We demonstrate the presence of Scale-Free properties of network structure for program data dependency and control flow graphs, and show that data dependency graphs also have Small-World structural properties. We show that program data dependency graphs have a degree correlation that is structurally disassortative, and that control flow graphs have a neutral degree assortativity, indicating the use of random graphs to model the structural properties of program control flow graphs would show increased accuracy. By providing an increase in feature resolution within labeled datasets of executable programs we provide a quantitative basis to interpret the results of classifiers trained on CFG graph features. An increase in feature resolution allows for the structural properties of program classes to be analyzed for patterns as well as their component parts. By capturing a complete picture of program graphs we can enable theoretical solutions for the map** a program's operational semantics to its structure.
△ Less
Submitted 14 October, 2022;
originally announced October 2022.
-
On the Combinality of Evidence in the Dempster-Shafer Theory
Authors:
Lotfi Zadeh,
Anca Ralescu
Abstract:
In the current versions of the Dempster-Shafer theory, the only essential restriction on the validity of the rule of combination is that the sources of evidence must be statistically independent. Under this assumption, it is permissible to apply the Dempster-Shafer rule to two or mere distinct probability distributions.
In the current versions of the Dempster-Shafer theory, the only essential restriction on the validity of the rule of combination is that the sources of evidence must be statistically independent. Under this assumption, it is permissible to apply the Dempster-Shafer rule to two or mere distinct probability distributions.
△ Less
Submitted 27 March, 2013;
originally announced April 2013.
