-
Algebraic Tools for Computing Polynomial Loop Invariants
Authors:
Erdenebayar Bayarmagnai,
Fatemeh Mohammadi,
Rémi Prébet
Abstract:
Loop invariants are properties of a program loop that hold before and after each iteration of the loop. They are often employed to verify programs and ensure that algorithms consistently produce correct results during execution. Consequently, the generation of invariants becomes a crucial task for loops. We specifically focus on polynomial loops, where both the loop conditions and assignments with…
▽ More
Loop invariants are properties of a program loop that hold before and after each iteration of the loop. They are often employed to verify programs and ensure that algorithms consistently produce correct results during execution. Consequently, the generation of invariants becomes a crucial task for loops. We specifically focus on polynomial loops, where both the loop conditions and assignments within the loop are expressed as polynomials. Although computing polynomial invariants for general loops is undecidable, efficient algorithms have been developed for certain classes of loops. For instance, when all assignments within a while loop involve linear polynomials, the loop becomes solvable. In this work, we study the more general case where the polynomials exhibit arbitrary degrees.
Applying tools from algebraic geometry, we present two algorithms designed to generate all polynomial invariants for a while loop, up to a specified degree. These algorithms differ based on whether the initial values of the loop variables are given or treated as parameters. Furthermore, we introduce various methods to address cases where the algebraic problem exceeds the computational capabilities of our methods. In such instances, we identify alternative approaches to generate specific polynomial invariants.
△ Less
Submitted 15 May, 2024;
originally announced May 2024.
-
GSC: Generalizable Service Coordination
Authors:
Farzad Mohammadi,
Vahid Shah-Mansouri
Abstract:
Services with distributed and interdependent components are becoming a popular option for harnessing dispersed resources available on cloud and edge networks. However, effective deployment and management of these services, namely service coordination, is a challenging task. Service coordination comprises the placement and scalability of components and scheduling incoming traffic requesting for ser…
▽ More
Services with distributed and interdependent components are becoming a popular option for harnessing dispersed resources available on cloud and edge networks. However, effective deployment and management of these services, namely service coordination, is a challenging task. Service coordination comprises the placement and scalability of components and scheduling incoming traffic requesting for services between deployed instances. Due to the online nature of the problem and the success of Deep Reinforcement Learning (DRL) methods, previous works considered DRL agents for solving service coordination problems, yet these solutions have to be retrained for every unseen scenario. Other works have tried to tackle this shortcoming by incorporating Graph Neural Networks (GNN) into their solutions, but they often focus on specific aspects (and disregard others) or cannot operate in dynamic and practical situations where there is no labeled dataset and feedback from the network might be delayed. In response to these challenges, we present GSC, a generalizable service coordinator that jointly considers service placement, scaling, and traffic scheduling. GSC can operate in unseen situations without significant performance degradation and outperforms existing state-of-the-art solutions by 40%, as determined by simulating real-world network situations.
△ Less
Submitted 5 November, 2023;
originally announced November 2023.
-
The Deep Arbitrary Polynomial Chaos Neural Network or how Deep Artificial Neural Networks could benefit from Data-Driven Homogeneous Chaos Theory
Authors:
Sergey Oladyshkin,
Timothy Praditia,
Ilja Kröker,
Farid Mohammadi,
Wolfgang Nowak,
Sebastian Otte
Abstract:
Artificial Intelligence and Machine learning have been widely used in various fields of mathematical computing, physical modeling, computational science, communication science, and stochastic analysis. Approaches based on Deep Artificial Neural Networks (DANN) are very popular in our days. Depending on the learning task, the exact form of DANNs is determined via their multi-layer architecture, act…
▽ More
Artificial Intelligence and Machine learning have been widely used in various fields of mathematical computing, physical modeling, computational science, communication science, and stochastic analysis. Approaches based on Deep Artificial Neural Networks (DANN) are very popular in our days. Depending on the learning task, the exact form of DANNs is determined via their multi-layer architecture, activation functions and the so-called loss function. However, for a majority of deep learning approaches based on DANNs, the kernel structure of neural signal processing remains the same, where the node response is encoded as a linear superposition of neural activity, while the non-linearity is triggered by the activation functions. In the current paper, we suggest to analyze the neural signal processing in DANNs from the point of view of homogeneous chaos theory as known from polynomial chaos expansion (PCE). From the PCE perspective, the (linear) response on each node of a DANN could be seen as a $1^{st}$ degree multi-variate polynomial of single neurons from the previous layer, i.e. linear weighted sum of monomials. From this point of view, the conventional DANN structure relies implicitly (but erroneously) on a Gaussian distribution of neural signals. Additionally, this view revels that by design DANNs do not necessarily fulfill any orthogonality or orthonormality condition for a majority of data-driven applications. Therefore, the prevailing handling of neural signals in DANNs could lead to redundant representation as any neural signal could contain some partial information from other neural signals. To tackle that challenge, we suggest to employ the data-driven generalization of PCE theory known as arbitrary polynomial chaos (aPC) to construct a corresponding multi-variate orthonormal representations on each node of a DANN to obtain Deep arbitrary polynomial chaos neural networks.
△ Less
Submitted 26 June, 2023;
originally announced June 2023.
-
Identifiability of Points and Rigidity of Hypergraphs under Algebraic Constraints
Authors:
James Cruickshank,
Fatemeh Mohammadi,
Anthony Nixon,
Shin-ichi Tanigawa
Abstract:
The identifiability problem arises naturally in a number of contexts in mathematics and computer science. Specific instances include local or global rigidity of graphs and unique completability of partially-filled tensors subject to rank conditions. The identifiability of points on secant varieties has also been a topic of much research in algebraic geometry. It is often formulated as the problem…
▽ More
The identifiability problem arises naturally in a number of contexts in mathematics and computer science. Specific instances include local or global rigidity of graphs and unique completability of partially-filled tensors subject to rank conditions. The identifiability of points on secant varieties has also been a topic of much research in algebraic geometry. It is often formulated as the problem of identifying a set of points satisfying a given set of algebraic relations. A key question then is to prove sufficient conditions for relations to guarantee the identifiability of the points.
This paper proposes a new general framework for capturing the identifiability problem when a set of algebraic relations has a combinatorial structure and develops tools to analyse the impact of the underlying combinatorics on the local or global identifiability of points. Our framework is built on the language of graph rigidity, where the measurements are Euclidean distances between two points, but applicable in the generality of hypergraphs with arbitrary algebraic measurements. We establish necessary and sufficient (hyper)graph theoretical conditions for identifiability by exploiting techniques from graph rigidity theory and algebraic geometry of secant varieties. In particular our work analyses combinatorially the effect of non-generic projections of secant varieties.
△ Less
Submitted 23 January, 2024; v1 submitted 30 May, 2023;
originally announced May 2023.
-
A Novel Plagiarism Detection Approach Combining BERT-based Word Embedding, Attention-based LSTMs and an Improved Differential Evolution Algorithm
Authors:
Seyed Vahid Moravvej,
Seyed Jalaleddin Mousavirad,
Diego Oliva,
Fardin Mohammadi
Abstract:
Detecting plagiarism involves finding similar items in two different sources. In this article, we propose a novel method for detecting plagiarism that is based on attention mechanism-based long short-term memory (LSTM) and bidirectional encoder representations from transformers (BERT) word embedding, enhanced with optimized differential evolution (DE) method for pre-training and a focal loss funct…
▽ More
Detecting plagiarism involves finding similar items in two different sources. In this article, we propose a novel method for detecting plagiarism that is based on attention mechanism-based long short-term memory (LSTM) and bidirectional encoder representations from transformers (BERT) word embedding, enhanced with optimized differential evolution (DE) method for pre-training and a focal loss function for training. BERT could be included in a downstream task and fine-tuned as a task-specific BERT can be included in a downstream task and fine-tuned as a task-specific structure, while the trained BERT model is capable of detecting various linguistic characteristics. Unbalanced classification is one of the primary issues with plagiarism detection. We suggest a focal loss-based training technique that carefully learns minority class instances to solve this. Another issue that we tackle is the training phase itself, which typically employs gradient-based methods like back-propagation for the learning process and thus suffers from some drawbacks, including sensitivity to initialization. To initiate the BP process, we suggest a novel DE algorithm that makes use of a clustering-based mutation operator. Here, a winning cluster is identified for the current DE population, and a fresh updating method is used to produce potential answers. We evaluate our proposed approach on three benchmark datasets ( MSRP, SNLI, and SemEval2014) and demonstrate that it performs well when compared to both conventional and population-based methods.
△ Less
Submitted 3 May, 2023;
originally announced May 2023.
-
Deep Learning in Healthcare: An In-Depth Analysis
Authors:
Farzan Shenavarmasouleh,
Farid Ghareh Mohammadi,
Khaled M. Rasheed,
Hamid R. Arabnia
Abstract:
Deep learning (DL) along with never-ending advancements in computational processing and cloud technologies have bestowed us powerful analyzing tools and techniques in the past decade and enabled us to use and apply them in various fields of study. Health informatics is not an exception, and conversely, is the discipline that generates the most amount of data in today's era and can benefit from DL…
▽ More
Deep learning (DL) along with never-ending advancements in computational processing and cloud technologies have bestowed us powerful analyzing tools and techniques in the past decade and enabled us to use and apply them in various fields of study. Health informatics is not an exception, and conversely, is the discipline that generates the most amount of data in today's era and can benefit from DL the most. Extracting features and finding complex patterns from a huge amount of raw data and transforming them into knowledge is a challenging task. Besides, various DL architectures have been proposed by researchers throughout the years to tackle different problems. In this paper, we provide a review of DL models and their broad application in bioinformatics and healthcare categorized by their architecture. In addition, we also go over some of the key challenges that still exist and can show up while conducting DL research.
△ Less
Submitted 12 February, 2023;
originally announced February 2023.
-
Template-based Program Synthesis using Stellensätze
Authors:
Amir Kafshdar Goharshady,
S. Hitarth,
Fatemeh Mohammadi,
Harshit J Motwani
Abstract:
Template-based synthesis, also known as sketching, is a localized approach to program synthesis in which the programmer provides not only a specification, but also a high-level ``sketch'' of the program. The sketch is basically a partial program that models the general intuition of the programmer, while leaving the low-level details as unimplemented ``holes''. The role of the synthesis engine is t…
▽ More
Template-based synthesis, also known as sketching, is a localized approach to program synthesis in which the programmer provides not only a specification, but also a high-level ``sketch'' of the program. The sketch is basically a partial program that models the general intuition of the programmer, while leaving the low-level details as unimplemented ``holes''. The role of the synthesis engine is then to fill in these holes such that the completed program satisfies the desired specification. In this work, we focus on template-based synthesis of polynomial imperative programs with real variables, i.e.~imperative programs in which all expressions appearing in assignments, conditions and guards are polynomials over program variables. While this problem can be solved in a sound and complete manner by a reduction to the first-order theory of the reals, the resulting formulas will contain a quantifier alternation and are extremely hard for modern SMT solvers, even when considering toy programs with a handful of lines. Moreover, the classical algorithms for quantifier elimination are notoriously unscalable and not at all applicable to this use-case.
In contrast, our main contribution is an algorithm, based on several well-known theorems in polyhedral and real algebraic geometry, namely Putinar's Positivstellensatz, the Real Nullstellensatz, Handelman's Theorem and Farkas' Lemma, which sidesteps the quantifier elimination difficulty and reduces the problem directly to Quadratic Programming (QP). Alternatively, one can view our algorithm as an efficient way of eliminating quantifiers in the particular formulas that appear in the synthesis problem. The resulting QP instances can then be handled quite easily by SMT solvers. Notably, our reduction to QP is sound and semi-complete, i.e.~it is complete if polynomials of a sufficiently high degree are used in the templates...
△ Less
Submitted 8 September, 2022;
originally announced September 2022.
-
3D-model ShapeNet Core Classification using Meta-Semantic Learning
Authors:
Farid Ghareh Mohammadi,
Cheng Chen,
Farzan Shenavarmasouleh,
M. Hadi Amini,
Beshoy Morkos,
Hamid R. Arabnia
Abstract:
Understanding 3D point cloud models for learning purposes has become an imperative challenge for real-world identification such as autonomous driving systems. A wide variety of solutions using deep learning have been proposed for point cloud segmentation, object detection, and classification. These methods, however, often require a considerable number of model parameters and are computationally ex…
▽ More
Understanding 3D point cloud models for learning purposes has become an imperative challenge for real-world identification such as autonomous driving systems. A wide variety of solutions using deep learning have been proposed for point cloud segmentation, object detection, and classification. These methods, however, often require a considerable number of model parameters and are computationally expensive. We study a semantic dimension of given 3D data points and propose an efficient method called Meta-Semantic Learning (Meta-SeL). Meta-SeL is an integrated framework that leverages two input 3D local points (input 3D models and part-segmentation labels), providing a time and cost-efficient, and precise projection model for a number of 3D recognition tasks. The results indicate that Meta-SeL yields competitive performance in comparison with other complex state-of-the-art work. Moreover, being random shuffle invariant, Meta-SeL is resilient to translation as well as jittering noise.
△ Less
Submitted 28 May, 2022;
originally announced May 2022.
-
The application of Evolutionary and Nature Inspired Algorithms in Data Science and Data Analytics
Authors:
Farid Ghareh Mohammadi,
Farzan Shenavarmasouleh,
Khaled Rasheed,
Thiab Taha,
M. Hadi Amini,
Hamid R. Arabnia
Abstract:
In the past 30 years, scientists have searched nature, including animals and insects, and biology in order to discover, understand, and model solutions for solving large-scale science challenges. The study of bionics reveals that how the biological structures, functions found in nature have improved our modern technologies. In this study, we present our discovery of evolutionary and nature-inspire…
▽ More
In the past 30 years, scientists have searched nature, including animals and insects, and biology in order to discover, understand, and model solutions for solving large-scale science challenges. The study of bionics reveals that how the biological structures, functions found in nature have improved our modern technologies. In this study, we present our discovery of evolutionary and nature-inspired algorithms applications in Data Science and Data Analytics in three main topics of pre-processing, supervised algorithms, and unsupervised algorithms. Among all applications, in this study, we aim to investigate four optimization algorithms that have been performed using the evolutionary and nature-inspired algorithms within data science and analytics. Feature selection optimization in pre-processing section, Hyper-parameter tuning optimization, and knowledge discovery optimization in supervised algorithms, and clustering optimization in the unsupervised algorithms.
△ Less
Submitted 6 February, 2022;
originally announced February 2022.
-
Applications of Machine Learning in Healthcare and Internet of Things (IOT): A Comprehensive Review
Authors:
Farid Ghareh Mohammadi,
Farzan Shenavarmasouleh,
Hamid R. Arabnia
Abstract:
In recent years, smart healthcare IoT devices have become ubiquitous, but they work in isolated networks due to their policy. Having these devices connected in a network enables us to perform medical distributed data analysis. However, the presence of diverse IoT devices in terms of technology, structure, and network policy, makes it a challenging issue while applying traditional centralized learn…
▽ More
In recent years, smart healthcare IoT devices have become ubiquitous, but they work in isolated networks due to their policy. Having these devices connected in a network enables us to perform medical distributed data analysis. However, the presence of diverse IoT devices in terms of technology, structure, and network policy, makes it a challenging issue while applying traditional centralized learning algorithms on decentralized data collected from the IoT devices. In this study, we present an extensive review of the state-of-the-art machine learning applications particularly in healthcare, challenging issues in IoT, and corresponding promising solutions. Finally, we highlight some open-ended issues of IoT in healthcare that leaves further research studies and investigation for scientists.
△ Less
Submitted 6 February, 2022;
originally announced February 2022.
-
OptABC: an Optimal Hyperparameter Tuning Approach for Machine Learning Algorithms
Authors:
Leila Zahedi,
Farid Ghareh Mohammadi,
M. Hadi Amini
Abstract:
Hyperparameter tuning in machine learning algorithms is a computationally challenging task due to the large-scale nature of the problem. In order to develop an efficient strategy for hyper-parameter tuning, one promising solution is to use swarm intelligence algorithms. Artificial Bee Colony (ABC) optimization lends itself as a promising and efficient optimization algorithm for this purpose. Howev…
▽ More
Hyperparameter tuning in machine learning algorithms is a computationally challenging task due to the large-scale nature of the problem. In order to develop an efficient strategy for hyper-parameter tuning, one promising solution is to use swarm intelligence algorithms. Artificial Bee Colony (ABC) optimization lends itself as a promising and efficient optimization algorithm for this purpose. However, in some cases, ABC can suffer from a slow convergence rate or execution time due to the poor initial population of solutions and expensive objective functions. To address these concerns, a novel algorithm, OptABC, is proposed to help ABC algorithm in faster convergence toward a near-optimum solution. OptABC integrates artificial bee colony algorithm, K-Means clustering, greedy algorithm, and opposition-based learning strategy for tuning the hyper-parameters of different machine learning models. OptABC employs these techniques in an attempt to diversify the initial population, and hence enhance the convergence ability without significantly decreasing the accuracy. In order to validate the performance of the proposed method, we compare the results with previous state-of-the-art approaches. Experimental results demonstrate the effectiveness of the OptABC compared to existing approaches in the literature.
△ Less
Submitted 15 December, 2021;
originally announced December 2021.
-
Data Analytics for Smart cities: Challenges and Promises
Authors:
Farid Ghareh Mohammadi,
Farzan Shenavarmasouleh,
M. Hadi Amini,
Hamid R. Arabnia
Abstract:
The explosion of advancements in artificial intelligence, sensor technologies, and wireless communication activates ubiquitous sensing through distributed sensors. These sensors are various domains of networks that lead us to smart systems in healthcare, transportation, environment, and other relevant branches/networks. Having collaborative interaction among the smart systems connects end-user dev…
▽ More
The explosion of advancements in artificial intelligence, sensor technologies, and wireless communication activates ubiquitous sensing through distributed sensors. These sensors are various domains of networks that lead us to smart systems in healthcare, transportation, environment, and other relevant branches/networks. Having collaborative interaction among the smart systems connects end-user devices to each other which enables achieving a new integrated entity called Smart Cities. The goal of this study is to provide a comprehensive survey of data analytics in smart cities. In this paper, we aim to focus on one of the smart cities important branches, namely Smart Mobility, and its positive ample impact on the smart cities decision-making process. Intelligent decision-making systems in smart mobility offer many advantages such as saving energy, relaying city traffic, and more importantly, reducing air pollution by offering real-time useful information and imperative knowledge. Making a decision in smart cities in time is challenging due to various and high dimensional factors and parameters, which are not frequently collected. In this paper, we first address current challenges in smart cities and provide an overview of potential solutions to these challenges. Then, we offer a framework of these solutions, called universal smart cities decision making, with three main sections of data capturing, data analysis, and decision making to optimize the smart mobility within smart cities. With this framework, we elaborate on fundamental concepts of big data, machine learning, and deep leaning algorithms that have been applied to smart cities and discuss the role of these algorithms in decision making for smart mobility in smart cities.
△ Less
Submitted 12 September, 2021;
originally announced September 2021.
-
HyP-ABC: A Novel Automated Hyper-Parameter Tuning Algorithm Using Evolutionary Optimization
Authors:
Leila Zahedi,
Farid Ghareh Mohammadi,
M. Hadi Amini
Abstract:
Machine learning techniques lend themselves as promising decision-making and analytic tools in a wide range of applications. Different ML algorithms have various hyper-parameters. In order to tailor an ML model towards a specific application, a large number of hyper-parameters should be tuned. Tuning the hyper-parameters directly affects the performance (accuracy and run-time). However, for large-…
▽ More
Machine learning techniques lend themselves as promising decision-making and analytic tools in a wide range of applications. Different ML algorithms have various hyper-parameters. In order to tailor an ML model towards a specific application, a large number of hyper-parameters should be tuned. Tuning the hyper-parameters directly affects the performance (accuracy and run-time). However, for large-scale search spaces, efficiently exploring the ample number of combinations of hyper-parameters is computationally challenging. Existing automated hyper-parameter tuning techniques suffer from high time complexity. In this paper, we propose HyP-ABC, an automatic innovative hybrid hyper-parameter optimization algorithm using the modified artificial bee colony approach, to measure the classification accuracy of three ML algorithms, namely random forest, extreme gradient boosting, and support vector machine. Compared to the state-of-the-art techniques, HyP-ABC is more efficient and has a limited number of parameters to be tuned, making it worthwhile for real-world hyper-parameter optimization problems. We further compare our proposed HyP-ABC algorithm with state-of-the-art techniques. In order to ensure the robustness of the proposed method, the algorithm takes a wide range of feasible hyper-parameter values, and is tested using a real-world educational dataset.
△ Less
Submitted 11 September, 2021;
originally announced September 2021.
-
Embodied AI-Driven Operation of Smart Cities: A Concise Review
Authors:
Farzan Shenavarmasouleh,
Farid Ghareh Mohammadi,
M. Hadi Amini,
Hamid R. Arabnia
Abstract:
A smart city can be seen as a framework, comprised of Information and Communication Technologies (ICT). An intelligent network of connected devices that collect data with their sensors and transmit them using cloud technologies in order to communicate with other assets in the ecosystem plays a pivotal role in this framework. Maximizing the quality of life of citizens, making better use of resource…
▽ More
A smart city can be seen as a framework, comprised of Information and Communication Technologies (ICT). An intelligent network of connected devices that collect data with their sensors and transmit them using cloud technologies in order to communicate with other assets in the ecosystem plays a pivotal role in this framework. Maximizing the quality of life of citizens, making better use of resources, cutting costs, and improving sustainability are the ultimate goals that a smart city is after. Hence, data collected from connected devices will continuously get thoroughly analyzed to gain better insights into the services that are being offered across the city; with this goal in mind that they can be used to make the whole system more efficient. Robots and physical machines are inseparable parts of a smart city. Embodied AI is the field of study that takes a deeper look into these and explores how they can fit into real-world environments. It focuses on learning through interaction with the surrounding environment, as opposed to Internet AI which tries to learn from static datasets. Embodied AI aims to train an agent that can See (Computer Vision), Talk (NLP), Navigate and Interact with its environment (Reinforcement Learning), and Reason (General Intelligence), all at the same time. Autonomous driving cars and personal companions are some of the examples that benefit from Embodied AI nowadays. In this paper, we attempt to do a concise review of this field. We will go through its definitions, its characteristics, and its current achievements along with different algorithms, approaches, and solutions that are being used in different components of it (e.g. Vision, NLP, RL). We will then explore all the available simulators and 3D interactable databases that will make the research in this area feasible. Finally, we will address its challenges and identify its potentials for future research.
△ Less
Submitted 22 August, 2021;
originally announced August 2021.
-
DRDrV3: Complete Lesion Detection in Fundus Images Using Mask R-CNN, Transfer Learning, and LSTM
Authors:
Farzan Shenavarmasouleh,
Farid Ghareh Mohammadi,
M. Hadi Amini,
Thiab Taha,
Khaled Rasheed,
Hamid R. Arabnia
Abstract:
Medical Imaging is one of the growing fields in the world of computer vision. In this study, we aim to address the Diabetic Retinopathy (DR) problem as one of the open challenges in medical imaging. In this research, we propose a new lesion detection architecture, comprising of two sub-modules, which is an optimal solution to detect and find not only the type of lesions caused by DR, their corresp…
▽ More
Medical Imaging is one of the growing fields in the world of computer vision. In this study, we aim to address the Diabetic Retinopathy (DR) problem as one of the open challenges in medical imaging. In this research, we propose a new lesion detection architecture, comprising of two sub-modules, which is an optimal solution to detect and find not only the type of lesions caused by DR, their corresponding bounding boxes, and their masks; but also the severity level of the overall case. Aside from traditional accuracy, we also use two popular evaluation criteria to evaluate the outputs of our models, which are intersection over union (IOU) and mean average precision (mAP). We hypothesize that this new solution enables specialists to detect lesions with high confidence and estimate the severity of the damage with high accuracy.
△ Less
Submitted 18 August, 2021;
originally announced August 2021.
-
Search Algorithms for Automated Hyper-Parameter Tuning
Authors:
Leila Zahedi,
Farid Ghareh Mohammadi,
Shabnam Rezapour,
Matthew W. Ohland,
M. Hadi Amini
Abstract:
Machine learning is a powerful method for modeling in different fields such as education. Its capability to accurately predict students' success makes it an ideal tool for decision-making tasks related to higher education. The accuracy of machine learning models depends on selecting the proper hyper-parameters. However, it is not an easy task because it requires time and expertise to tune the hype…
▽ More
Machine learning is a powerful method for modeling in different fields such as education. Its capability to accurately predict students' success makes it an ideal tool for decision-making tasks related to higher education. The accuracy of machine learning models depends on selecting the proper hyper-parameters. However, it is not an easy task because it requires time and expertise to tune the hyper-parameters to fit the machine learning model. In this paper, we examine the effectiveness of automated hyper-parameter tuning techniques to the realm of students' success. Therefore, we develop two automated Hyper-Parameter Optimization methods, namely grid search and random search, to assess and improve a previous study's performance. The experiment results show that applying random search and grid search on machine learning algorithms improves accuracy. We empirically show automated methods' superiority on real-world educational data (MIDFIELD) for tuning HPs of conventional machine learning classifiers. This work emphasizes the effectiveness of automated hyper-parameter optimization while applying machine learning in the education field to aid faculties, directors', or non-expert users' decisions to improve students' success.
△ Less
Submitted 29 April, 2021;
originally announced April 2021.
-
Malware Detection using Artificial Bee Colony Algorithm
Authors:
Farid Ghareh Mohammadi,
Farzan Shenavarmasouleh,
M. Hadi Amini,
Hamid R. Arabnia
Abstract:
Malware detection has become a challenging task due to the increase in the number of malware families. Universal malware detection algorithms that can detect all the malware families are needed to make the whole process feasible. However, the more universal an algorithm is, the higher number of feature dimensions it needs to work with, and that inevitably causes the emerging problem of Curse of Di…
▽ More
Malware detection has become a challenging task due to the increase in the number of malware families. Universal malware detection algorithms that can detect all the malware families are needed to make the whole process feasible. However, the more universal an algorithm is, the higher number of feature dimensions it needs to work with, and that inevitably causes the emerging problem of Curse of Dimensionality (CoD). Besides, it is also difficult to make this solution work due to the real-time behavior of malware analysis. In this paper, we address this problem and aim to propose a feature selection based malware detection algorithm using an evolutionary algorithm that is referred to as Artificial Bee Colony (ABC). The proposed algorithm enables researchers to decrease the feature dimension and as a result, boost the process of malware detection. The experimental results reveal that the proposed method outperforms the state-of-the-art.
△ Less
Submitted 1 December, 2020;
originally announced December 2020.
-
DRDr II: Detecting the Severity Level of Diabetic Retinopathy Using Mask RCNN and Transfer Learning
Authors:
Farzan Shenavarmasouleh,
Farid Ghareh Mohammadi,
M. Hadi Amini,
Hamid R. Arabnia
Abstract:
DRDr II is a hybrid of machine learning and deep learning worlds. It builds on the successes of its antecedent, namely, DRDr, that was trained to detect, locate, and create segmentation masks for two types of lesions (exudates and microaneurysms) that can be found in the eyes of the Diabetic Retinopathy (DR) patients; and uses the entire model as a solid feature extractor in the core of its pipeli…
▽ More
DRDr II is a hybrid of machine learning and deep learning worlds. It builds on the successes of its antecedent, namely, DRDr, that was trained to detect, locate, and create segmentation masks for two types of lesions (exudates and microaneurysms) that can be found in the eyes of the Diabetic Retinopathy (DR) patients; and uses the entire model as a solid feature extractor in the core of its pipeline to detect the severity level of the DR cases. We employ a big dataset with over 35 thousand fundus images collected from around the globe and after 2 phases of preprocessing alongside feature extraction, we succeed in predicting the correct severity levels with over 92% accuracy.
△ Less
Submitted 30 November, 2020;
originally announced November 2020.
-
DeepMSRF: A novel Deep Multimodal Speaker Recognition framework with Feature selection
Authors:
Ehsan Asali,
Farzan Shenavarmasouleh,
Farid Ghareh Mohammadi,
Prasanth Sengadu Suresh,
Hamid R. Arabnia
Abstract:
For recognizing speakers in video streams, significant research studies have been made to obtain a rich machine learning model by extracting high-level speaker's features such as facial expression, emotion, and gender. However, generating such a model is not feasible by using only single modality feature extractors that exploit either audio signals or image frames, extracted from video streams. In…
▽ More
For recognizing speakers in video streams, significant research studies have been made to obtain a rich machine learning model by extracting high-level speaker's features such as facial expression, emotion, and gender. However, generating such a model is not feasible by using only single modality feature extractors that exploit either audio signals or image frames, extracted from video streams. In this paper, we address this problem from a different perspective and propose an unprecedented multimodality data fusion framework called DeepMSRF, Deep Multimodal Speaker Recognition with Feature selection. We execute DeepMSRF by feeding features of the two modalities, namely speakers' audios and face images. DeepMSRF uses a two-stream VGGNET to train on both modalities to reach a comprehensive model capable of accurately recognizing the speaker's identity. We apply DeepMSRF on a subset of VoxCeleb2 dataset with its metadata merged with VGGFace2 dataset. The goal of DeepMSRF is to identify the gender of the speaker first, and further to recognize his or her name for any given video stream. The experimental results illustrate that DeepMSRF outperforms single modality speaker recognition methods with at least 3 percent accuracy.
△ Less
Submitted 21 July, 2020; v1 submitted 14 July, 2020;
originally announced July 2020.
-
A Unified Architecture for Data-Driven Metadata Tagging of Building Automation Systems
Authors:
Sakshi Mishra,
Andrew Glaws,
Dylan Cutler,
Stephen Frank,
Muhammad Azam,
Farzam Mohammadi,
Jean-Simon Venne
Abstract:
This article presents a Unified Architecture for automated point tagging of Building Automation System data, based on a combination of data-driven approaches. Advanced energy analytics applications-including fault detection and diagnostics and supervisory control-have emerged as a significant opportunity for improving the performance of our built environment. Effective application of these analyti…
▽ More
This article presents a Unified Architecture for automated point tagging of Building Automation System data, based on a combination of data-driven approaches. Advanced energy analytics applications-including fault detection and diagnostics and supervisory control-have emerged as a significant opportunity for improving the performance of our built environment. Effective application of these analytics depends on harnessing structured data from the various building control and monitoring systems, but typical Building Automation System implementations do not employ any standardized metadata schema. While standards such as Project Haystack and Brick Schema have been developed to address this issue, the process of structuring the data, i.e., tagging the points to apply a standard metadata schema, has, to date, been a manual process. This process is typically costly, labor-intensive, and error-prone. In this work we address this gap by proposing a UA that automates the process of point tagging by leveraging the data accessible through connection to the BAS, including time series data and the raw point names. The UA intertwines supervised classification and unsupervised clustering techniques from machine learning and leverages both their deterministic and probabilistic outputs to inform the point tagging process. Furthermore, we extend the UA to embed additional input and output data-processing modules that are designed to address the challenges associated with the real-time deployment of this automation solution. We test the UA on two datasets for real-life buildings: 1. commercial retail buildings and 2. office buildings from the National Renewable Energy Laboratory campus. The proposed methodology correctly applied 85-90 percent and 70-75 percent of the tags in each of these test scenarios, respectively.
△ Less
Submitted 11 September, 2020; v1 submitted 26 February, 2020;
originally announced March 2020.
-
On Parameter Tuning in Meta-learning for Computer Vision
Authors:
Farid Ghareh Mohammadi,
M. Hadi Amini,
Hamid R. Arabnia
Abstract:
Learning to learn plays a pivotal role in meta-learning (MTL) to obtain an optimal learning model. In this paper, we investigate mage recognition for unseen categories of a given dataset with limited training information. We deploy a zero-shot learning (ZSL) algorithm to achieve this goal. We also explore the effect of parameter tuning on performance of semantic auto-encoder (SAE). We further addr…
▽ More
Learning to learn plays a pivotal role in meta-learning (MTL) to obtain an optimal learning model. In this paper, we investigate mage recognition for unseen categories of a given dataset with limited training information. We deploy a zero-shot learning (ZSL) algorithm to achieve this goal. We also explore the effect of parameter tuning on performance of semantic auto-encoder (SAE). We further address the parameter tuning problem for meta-learning, especially focusing on zero-shot learning. By combining different embedded parameters, we improved the accuracy of tuned-SAE. Advantages and disadvantages of parameter tuning and its application in image classification are also explored.
△ Less
Submitted 11 February, 2020;
originally announced March 2020.
-
DuMu$^\text{x}$ 3 -- an open-source simulator for solving flow and transport problems in porous media with a focus on model coupling
Authors:
Timo Koch,
Dennis Gläser,
Kilian Weishaupt,
Sina Ackermann,
Martin Beck,
Beatrix Becker,
Samuel Burbulla,
Holger Class,
Edward Coltman,
Simon Emmert,
Thomas Fetzer,
Christoph Grüninger,
Katharina Heck,
Johannes Hommel,
Theresa Kurz,
Melanie Lipp,
Farid Mohammadi,
Samuel Scherrer,
Martin Schneider,
Gabriele Seitz,
Leopold Stadler,
Martin Utz,
Felix Weinhardt,
Bernd Flemisch
Abstract:
We present version 3 of the open-source simulator for flow and transport processes in porous media DuMu$^\text{x}$. DuMu$^\text{x}$ is based on the modular C++ framework Dune (Distributed and Unified Numerics Environment) and is developed as a research code with a focus on modularity and reusability. We describe recent efforts in improving the transparency and efficiency of the development process…
▽ More
We present version 3 of the open-source simulator for flow and transport processes in porous media DuMu$^\text{x}$. DuMu$^\text{x}$ is based on the modular C++ framework Dune (Distributed and Unified Numerics Environment) and is developed as a research code with a focus on modularity and reusability. We describe recent efforts in improving the transparency and efficiency of the development process and community-building, as well as efforts towards quality assurance and reproducible research. In addition to a major redesign of many simulation components in order to facilitate setting up complex simulations in DuMu$^\text{x}$, version 3 introduces a more consistent abstraction of finite volume schemes. Finally, the new framework for multi-domain simulations is described, and three numerical examples demonstrate its flexibility.
△ Less
Submitted 11 September, 2019;
originally announced September 2019.
-
An Introduction to Advanced Machine Learning : Meta Learning Algorithms, Applications and Promises
Authors:
Farid Ghareh Mohammadi,
M. Hadi Amini,
Hamid R. Arabnia
Abstract:
In [1, 2], we have explored the theoretical aspects of feature extraction optimization processes for solving largescale problems and overcoming machine learning limitations. Majority of optimization algorithms that have been introduced in [1, 2] guarantee the optimal performance of supervised learning, given offline and discrete data, to deal with curse of dimensionality (CoD) problem. These algor…
▽ More
In [1, 2], we have explored the theoretical aspects of feature extraction optimization processes for solving largescale problems and overcoming machine learning limitations. Majority of optimization algorithms that have been introduced in [1, 2] guarantee the optimal performance of supervised learning, given offline and discrete data, to deal with curse of dimensionality (CoD) problem. These algorithms, however, are not tailored for solving emerging learning problems. One of the important issues caused by online data is lack of sufficient samples per class. Further, traditional machine learning algorithms cannot achieve accurate training based on limited distributed data, as data has proliferated and dispersed significantly. Machine learning employs a strict model or embedded engine to train and predict which still fails to learn unseen classes and sufficiently use online data. In this chapter, we introduce these challenges elaborately. We further investigate Meta-Learning (MTL) algorithm, and their application and promises to solve the emerging problems by answering how autonomous agents can learn to learn?.
△ Less
Submitted 26 August, 2019;
originally announced August 2019.
-
Applications of Nature-Inspired Algorithms for Dimension Reduction: Enabling Efficient Data Analytics
Authors:
Farid Ghareh Mohammadi,
M. Hadi Amini,
Hamid R. Arabnia
Abstract:
In [1], we have explored the theoretical aspects of feature selection and evolutionary algorithms. In this chapter, we focus on optimization algorithms for enhancing data analytic process, i.e., we propose to explore applications of nature-inspired algorithms in data science. Feature selection optimization is a hybrid approach leveraging feature selection techniques and evolutionary algorithms pro…
▽ More
In [1], we have explored the theoretical aspects of feature selection and evolutionary algorithms. In this chapter, we focus on optimization algorithms for enhancing data analytic process, i.e., we propose to explore applications of nature-inspired algorithms in data science. Feature selection optimization is a hybrid approach leveraging feature selection techniques and evolutionary algorithms process to optimize the selected features. Prior works solve this problem iteratively to converge to an optimal feature subset. Feature selection optimization is a non-specific domain approach. Data scientists mainly attempt to find an advanced way to analyze data n with high computational efficiency and low time complexity, leading to efficient data analytics. Thus, by increasing generated/measured/sensed data from various sources, analysis, manipulation and illustration of data grow exponentially. Due to the large scale data sets, Curse of dimensionality (CoD) is one of the NP-hard problems in data science. Hence, several efforts have been focused on leveraging evolutionary algorithms (EAs) to address the complex issues in large scale data analytics problems. Dimension reduction, together with EAs, lends itself to solve CoD and solve complex problems, in terms of time complexity, efficiently. In this chapter, we first provide a brief overview of previous studies that focused on solving CoD using feature extraction optimization process. We then discuss practical examples of research studies are successfully tackled some application domains, such as image processing, sentiment analysis, network traffics / anomalies analysis, credit score analysis and other benchmark functions/data sets analysis.
△ Less
Submitted 22 August, 2019;
originally announced August 2019.
-
Evolutionary Computation, Optimization and Learning Algorithms for Data Science
Authors:
Farid Ghareh Mohammadi,
M. Hadi Amini,
Hamid R. Arabnia
Abstract:
A large number of engineering, science and computational problems have yet to be solved in a computationally efficient way. One of the emerging challenges is how evolving technologies grow towards autonomy and intelligent decision making. This leads to collection of large amounts of data from various sensing and measurement technologies, e.g., cameras, smart phones, health sensors, smart electrici…
▽ More
A large number of engineering, science and computational problems have yet to be solved in a computationally efficient way. One of the emerging challenges is how evolving technologies grow towards autonomy and intelligent decision making. This leads to collection of large amounts of data from various sensing and measurement technologies, e.g., cameras, smart phones, health sensors, smart electricity meters, and environment sensors. Hence, it is imperative to develop efficient algorithms for generation, analysis, classification, and illustration of data. Meanwhile, data is structured purposefully through different representations, such as large-scale networks and graphs. We focus on data science as a crucial area, specifically focusing on a curse of dimensionality (CoD) which is due to the large amount of generated/sensed/collected data. This motivates researchers to think about optimization and to apply nature-inspired algorithms, such as evolutionary algorithms (EAs) to solve optimization problems. Although these algorithms look un-deterministic, they are robust enough to reach an optimal solution. Researchers do not adopt evolutionary algorithms unless they face a problem which is suffering from placement in local optimal solution, rather than global optimal solution. In this chapter, we first develop a clear and formal definition of the CoD problem, next we focus on feature extraction techniques and categories, then we provide a general overview of meta-heuristic algorithms, its terminology, and desirable properties of evolutionary algorithms.
△ Less
Submitted 16 August, 2019;
originally announced August 2019.
-
Evolutionary Algorithms and Efficient Data Analytics for Image Processing
Authors:
Farid Ghareh Mohammadi,
Farzan Shenavarmasouleh,
M. Hadi Amini,
Hamid R. Arabnia
Abstract:
Steganography algorithms facilitate communication between a source and a destination in a secret manner. This is done by embedding messages/text/data into images without impacting the appearance of the resultant images/videos. Steganalysis is the science of determining if an image has secret messages embedded/hidden in it. Because there are numerous steganography algorithms, and since each one of…
▽ More
Steganography algorithms facilitate communication between a source and a destination in a secret manner. This is done by embedding messages/text/data into images without impacting the appearance of the resultant images/videos. Steganalysis is the science of determining if an image has secret messages embedded/hidden in it. Because there are numerous steganography algorithms, and since each one of them requires a different type of steganalysis, the steganalysis process is extremely challenging. Thus, researchers aim to develop one universal steganalysis to detect all known and unknown steganography algorithms, ideally in real-time. Universal steganalysis extracts a large number of features to distinguish stego images from cover images. However, the increase in features leads to the problem of the curse of dimensionality (CoD), which is considered to be an NP-hard problem. This COD problem additionally makes real-time steganalysis hard. A large number of features generates large datasets for which machine learning cannot generate an optimal model. Generating a machine learning based model also takes a long time which makes real-time processing appear impossible in any optimization for time-intensive fields such as visual computing. Possible solutions for CoD are deep learning and evolutionary algorithms that overcome the machine learning limitations. In this study, we investigate previously developed evolutionary algorithms for boosting real-time image processing and argue that they provide the most promising solutions for the CoD problem.
△ Less
Submitted 19 October, 2020; v1 submitted 23 July, 2019;
originally announced July 2019.
-
An Efficient Algorithm for Computing Network Reliability in Small Treewidth
Authors:
Amir Kafshdar Goharshady,
Fatemeh Mohammadi
Abstract:
We consider the classic problem of Network Reliability. A network is given together with a source vertex, one or more target vertices, and probabilities assigned to each of the edges. Each edge appears in the network with its associated probability and the problem is to determine the probability of having at least one source-to-target path. This problem is known to be NP-hard.
We present a linea…
▽ More
We consider the classic problem of Network Reliability. A network is given together with a source vertex, one or more target vertices, and probabilities assigned to each of the edges. Each edge appears in the network with its associated probability and the problem is to determine the probability of having at least one source-to-target path. This problem is known to be NP-hard.
We present a linear-time fixed-parameter algorithm based on a parameter called treewidth, which is a measure of tree-likeness of graphs. Network Reliability was already known to be solvable in polynomial time for bounded treewidth, but there were no concrete algorithms and the known methods used complicated structures and were not easy to implement. We provide a significantly simpler and more intuitive algorithm that is much easier to implement.
We also report on an implementation of our algorithm and establish the applicability of our approach by providing experimental results on the graphs of subway and transit systems of several major cities, such as London and Tokyo. To the best of our knowledge, this is the first exact algorithm for Network Reliability that can scale to handle real-world instances of the problem.
△ Less
Submitted 19 March, 2019; v1 submitted 27 December, 2017;
originally announced December 2017.