-
FFCL: Forward-Forward Net with Cortical Loops, Training and Inference on Edge Without Backpropagation
Authors:
Ali Karkehabadi,
Houman Homayoun,
Avesta Sasan
Abstract:
The Forward-Forward Learning (FFL) algorithm is a recently proposed solution for training neural networks without needing memory-intensive backpropagation. During training, labels accompany input data, classifying them as positive or negative inputs. Each layer learns its response to these inputs independently. In this study, we enhance the FFL with the following contributions: 1) We optimize labe…
▽ More
The Forward-Forward Learning (FFL) algorithm is a recently proposed solution for training neural networks without needing memory-intensive backpropagation. During training, labels accompany input data, classifying them as positive or negative inputs. Each layer learns its response to these inputs independently. In this study, we enhance the FFL with the following contributions: 1) We optimize label processing by segregating label and feature forwarding between layers, enhancing learning performance. 2) By revising label integration, we enhance the inference process, reduce computational complexity, and improve performance. 3) We introduce feedback loops akin to cortical loops in the brain, where information cycles through and returns to earlier neurons, enabling layers to combine complex features from previous layers with lower-level features, enhancing learning efficiency.
△ Less
Submitted 20 May, 2024;
originally announced May 2024.
-
Optimizing Underwater IoT Routing with Multi-Criteria Decision Making and Uncertainty Weights
Authors:
Ali Karkehabadi,
Mitra Bakhshi,
Seyed Behnam Razavian
Abstract:
Effective data routing is vital in the Internet of Things (IoT) paradigm, especially in underwater mobile sensor networks where inefficiency can lead to significant resource consumption. This article presents an innovative method designed to enhance network performance and reduce resource usage, while also accurately determining component weights in these networks, ensuring quality service. Buildi…
▽ More
Effective data routing is vital in the Internet of Things (IoT) paradigm, especially in underwater mobile sensor networks where inefficiency can lead to significant resource consumption. This article presents an innovative method designed to enhance network performance and reduce resource usage, while also accurately determining component weights in these networks, ensuring quality service. Building upon previous research on multi-criteria decision-making systems in coastal RPL networks, our method involves key adaptations for underwater environments. It integrates comprehensive network features to identify the optimal parent node for each sensor, employing the fuzzy SWARA decision-making approach under uncertain conditions. This method takes into account various factors including hops, energy, ARSSI rate, delay, ETX, link delivery rate, and depth to determine the most effective parent node assignment. Through simulation, our approach demonstrates marked improvements in network performance compared to existing solutions. These advancements are significant, offering a new direction in enhancing underwater IoT communications and suggesting wider applications for IoT systems facing similar challenges.
△ Less
Submitted 19 May, 2024;
originally announced May 2024.
-
SMOOT: Saliency Guided Mask Optimized Online Training
Authors:
Ali Karkehabadi,
Houman Homayoun,
Avesta Sasan
Abstract:
Deep Neural Networks are powerful tools for understanding complex patterns and making decisions. However, their black-box nature impedes a complete understanding of their inner workings. Saliency-Guided Training (SGT) methods try to highlight the prominent features in the model's training based on the output to alleviate this problem. These methods use back-propagation and modified gradients to gu…
▽ More
Deep Neural Networks are powerful tools for understanding complex patterns and making decisions. However, their black-box nature impedes a complete understanding of their inner workings. Saliency-Guided Training (SGT) methods try to highlight the prominent features in the model's training based on the output to alleviate this problem. These methods use back-propagation and modified gradients to guide the model toward the most relevant features while kee** the impact on the prediction accuracy negligible. SGT makes the model's final result more interpretable by masking input partially. In this way, considering the model's output, we can infer how each segment of the input affects the output. In the particular case of image as the input, masking is applied to the input pixels. However, the masking strategy and number of pixels which we mask, are considered as a hyperparameter. Appropriate setting of masking strategy can directly affect the model's training. In this paper, we focus on this issue and present our contribution. We propose a novel method to determine the optimal number of masked images based on input, accuracy, and model loss during the training. The strategy prevents information loss which leads to better accuracy values. Also, by integrating the model's performance in the strategy formula, we show that our model represents the salient features more meaningful. Our experimental results demonstrate a substantial improvement in both model accuracy and the prominence of saliency, thereby affirming the effectiveness of our proposed solution.
△ Less
Submitted 10 October, 2023; v1 submitted 1 October, 2023;
originally announced October 2023.
-
Does Saliency-Based Training bring Robustness for Deep Neural Networks in Image Classification?
Authors:
Ali Karkehabadi
Abstract:
Deep Neural Networks are powerful tools to understand complex patterns and making decisions. However, their black-box nature impedes a complete understanding of their inner workings. While online saliency-guided training methods try to highlight the prominent features in the model's output to alleviate this problem, it is still ambiguous if the visually explainable features align with robustness o…
▽ More
Deep Neural Networks are powerful tools to understand complex patterns and making decisions. However, their black-box nature impedes a complete understanding of their inner workings. While online saliency-guided training methods try to highlight the prominent features in the model's output to alleviate this problem, it is still ambiguous if the visually explainable features align with robustness of the model against adversarial examples. In this paper, we investigate the saliency trained model's vulnerability to adversarial examples methods. Models are trained using an online saliency-guided training method and evaluated against popular algorithms of adversarial examples. We quantify the robustness and conclude that despite the well-explained visualizations in the model's output, the salient models suffer from the lower performance against adversarial examples attacks.
△ Less
Submitted 28 June, 2023;
originally announced June 2023.
-
TM-vector: A Novel Forecasting Approach for Market stock movement with a Rich Representation of Twitter and Market data
Authors:
Faraz Sasani,
Ramin Mousa,
Ali Karkehabadi,
Samin Dehbashi,
Ali Mohammadi
Abstract:
Stock market forecasting has been a challenging part for many analysts and researchers. Trend analysis, statistical techniques, and movement indicators have traditionally been used to predict stock price movements, but text extraction has emerged as a promising method in recent years. The use of neural networks, especially recurrent neural networks, is abundant in the literature. In most studies,…
▽ More
Stock market forecasting has been a challenging part for many analysts and researchers. Trend analysis, statistical techniques, and movement indicators have traditionally been used to predict stock price movements, but text extraction has emerged as a promising method in recent years. The use of neural networks, especially recurrent neural networks, is abundant in the literature. In most studies, the impact of different users was considered equal or ignored, whereas users can have other effects. In the current study, we will introduce TM-vector and then use this vector to train an IndRNN and ultimately model the market users' behaviour. In the proposed model, TM-vector is simultaneously trained with both the extracted Twitter features and market information. Various factors have been used for the effectiveness of the proposed forecasting approach, including the characteristics of each individual user, their impact on each other, and their impact on the market, to predict market direction more accurately. Dow Jones 30 index has been used in current work. The accuracy obtained for predicting daily stock changes of Apple is based on various models, closed to over 95\% and for the other stocks is significant. Our results indicate the effectiveness of TM-vector in predicting stock market direction.
△ Less
Submitted 13 March, 2023;
originally announced April 2023.
