-
Fusing Depthwise and Pointwise Convolutions for Efficient Inference on GPUs
Authors:
Fareed Qararyah,
Muhammad Waqar Azhar,
Mohammad Ali Maleki,
Pedro Trancoso
Abstract:
Depthwise and pointwise convolutions have fewer parameters and perform fewer operations than standard convolutions. As a result, they have become increasingly used in various compact DNNs, including convolutional neural networks (CNNs) and vision transformers (ViTs). However, they have a lower compute-to-memory-access ratio than standard convolutions, making their memory accesses often the perform…
▽ More
Depthwise and pointwise convolutions have fewer parameters and perform fewer operations than standard convolutions. As a result, they have become increasingly used in various compact DNNs, including convolutional neural networks (CNNs) and vision transformers (ViTs). However, they have a lower compute-to-memory-access ratio than standard convolutions, making their memory accesses often the performance bottleneck. This paper explores fusing depthwise and pointwise convolutions to overcome the memory access bottleneck. The focus is on fusing these operators on GPUs. The prior art on GPU-based fusion suffers from one or more of the following: (1) fusing either a convolution with an element-wise or multiple non-convolutional operators, (2) not explicitly optimizing for memory accesses, (3) not supporting depthwise convolutions. This paper proposes Fused Convolutional Modules (FCMs), a set of novel fused depthwise and pointwise GPU kernels. FCMs significantly reduce pointwise and depthwise convolutions memory accesses, improving execution time and energy efficiency. To evaluate the trade-offs associated with fusion and determine which convolutions are beneficial to fuse and the optimal FCM parameters, we propose FusePlanner. FusePlanner consists of cost models to estimate the memory accesses of depthwise, pointwise, and FCM kernels given GPU characteristics. Our experiments on three GPUs using representative CNNs and ViTs demonstrate that FCMs save up to 83% of the memory accesses and achieve speedups of up to 3.7x compared to cuDNN. Complete model implementations of various CNNs using our modules outperform TVMs' achieving speedups of up to 1.8x and saving up to two-thirds of the energy.
△ Less
Submitted 30 April, 2024;
originally announced April 2024.
-
Forecasting Tech Sector Market Downturns based on Macroeconomic Indicators
Authors:
Morteza Maleki
Abstract:
Predicting stock price movements is a pivotal element of investment strategy, providing insights into potential trends and market volatility. This study specifically examines the predictive capacity of historical stock prices and technical indicators within the Global Industry Classification Standard (GICS) Information Technology Sector, focusing on companies established before 1980. We aim to ide…
▽ More
Predicting stock price movements is a pivotal element of investment strategy, providing insights into potential trends and market volatility. This study specifically examines the predictive capacity of historical stock prices and technical indicators within the Global Industry Classification Standard (GICS) Information Technology Sector, focusing on companies established before 1980. We aim to identify patterns that precede significant, non-transient downturns - defined as declines exceeding 10% from peak values. Utilizing a combination of machine learning techniques, including multiple regression analysis, logistic regression, we analyze an enriched dataset comprising both macroeconomic indicators and market data. Our findings suggest that certain clusters of technical indicators, when combined with broader economic signals, offer predictive insights into forthcoming sector-specific downturns. This research not only enhances our understanding of the factors driving market dynamics in the tech sector but also provides portfolio managers and investors with a sophisticated tool for anticipating and mitigating potential losses from market downturns. Through a rigorous validation process, we demonstrate the robustness of our models, contributing to the field of financial analytics by offering a novel approach to predicting market downturns with significant implications for investment strategies and economic policy planning.
△ Less
Submitted 15 April, 2024;
originally announced April 2024.
-
Identification of cardiovascular diseases through ECG classification using wavelet transformation
Authors:
Morteza Maleki
Abstract:
Cardiovascular diseases are the leading cause of mortality globally, necessitating advancements in diagnostic techniques. This study explores the application of wavelet transformation for classifying electrocardiogram (ECG) signals to identify various cardiovascular conditions. Utilizing the MIT-BIH Arrhythmia Database, we employed both continuous and discrete wavelet transforms to decompose ECG s…
▽ More
Cardiovascular diseases are the leading cause of mortality globally, necessitating advancements in diagnostic techniques. This study explores the application of wavelet transformation for classifying electrocardiogram (ECG) signals to identify various cardiovascular conditions. Utilizing the MIT-BIH Arrhythmia Database, we employed both continuous and discrete wavelet transforms to decompose ECG signals into frequency sub-bands, from which we extracted eight statistical features per band. These features were then used to train and test various classifiers, including K-Nearest Neighbors and Support Vector Machines, among others. The classifiers demonstrated high efficacy, with some achieving an accuracy of up to 96% on test data, suggesting that wavelet-based feature extraction significantly enhances the prediction of cardiovascular abnormalities in ECG data. The findings advocate for further exploration of wavelet transforms in medical diagnostics to improve automation and accuracy in disease detection. Future work will focus on optimizing feature selection and classifier parameters to refine predictive performance further.
△ Less
Submitted 14 April, 2024;
originally announced April 2024.
-
Clustering Analysis of US COVID-19 Rates, Vaccine Participation, and Socioeconomic Factors
Authors:
Morteza Maleki
Abstract:
The COVID-19 pandemic has presented unprecedented challenges worldwide, with its impact varying significantly across different geographic and socioeconomic contexts. This study employs a clustering analysis to examine the diversity of responses to the pandemic within the United States, aiming to provide nuanced insights into the effectiveness of various strategies. We utilize an unsupervised machi…
▽ More
The COVID-19 pandemic has presented unprecedented challenges worldwide, with its impact varying significantly across different geographic and socioeconomic contexts. This study employs a clustering analysis to examine the diversity of responses to the pandemic within the United States, aiming to provide nuanced insights into the effectiveness of various strategies. We utilize an unsupervised machine learning approach, specifically K-Means clustering, to analyze county-level data that includes variables such as infection rates, death rates, demographic profiles, and socio-economic factors. Our analysis identifies distinct clusters of counties based on their pandemic responses and outcomes, facilitating a detailed examination of "high-performing" and "lower-performing" groups. These classifications are informed by a combination of COVID-specific datasets and broader socio-economic data, allowing for a comprehensive understanding of the factors that contribute to differing levels of pandemic impact. The findings underscore the importance of tailored public health responses that consider local conditions and capabilities. Additionally, this study introduces an innovative visualization tool that aids in hypothesis testing and further research, enhancing the ability of policymakers and public health officials to deploy more effective and targeted interventions in future health crises.
△ Less
Submitted 11 April, 2024;
originally announced April 2024.
-
Clinical Trials Protocol Authoring using LLMs
Authors:
Morteza Maleki
Abstract:
This report embarks on a mission to revolutionize clinical trial protocol development through the integration of advanced AI technologies. With a focus on leveraging the capabilities of generative AI, specifically GPT-4, this initiative aimed to streamline and enhance the efficiency and accuracy of clinical trial protocols. The methodology encompassed a detailed analysis and preparation of compreh…
▽ More
This report embarks on a mission to revolutionize clinical trial protocol development through the integration of advanced AI technologies. With a focus on leveraging the capabilities of generative AI, specifically GPT-4, this initiative aimed to streamline and enhance the efficiency and accuracy of clinical trial protocols. The methodology encompassed a detailed analysis and preparation of comprehensive drug and study level metadata, followed by the deployment of GPT-4 for automated protocol section generation. Results demonstrated a significant improvement in protocol authoring, highlighted by increases in efficiency, accuracy, and the customization of protocols to specific trial requirements. Challenges encountered during model selection and prompt engineering were systematically addressed, leading to refined methodologies that capitalized on the advanced text generation capabilities of GPT-4. This project not only showcases the practical applications and benefits of generative AI in clinical trial design but also sets a foundation for future innovations in the field.
△ Less
Submitted 7 April, 2024;
originally announced April 2024.
-
Heterogeneous Multi-core Array-based DNN Accelerator
Authors:
Mohammad Ali Maleki,
Mehdi Kamal,
Ali Afzali-Kusha
Abstract:
In this article, we investigate the impact of architectural parameters of array-based DNN accelerators on accelerator's energy consumption and performance in a wide variety of network topologies. For this purpose, we have developed a tool that simulates the execution of neural networks on array-based accelerators and has the capability of testing different configurations for the estimation of ener…
▽ More
In this article, we investigate the impact of architectural parameters of array-based DNN accelerators on accelerator's energy consumption and performance in a wide variety of network topologies. For this purpose, we have developed a tool that simulates the execution of neural networks on array-based accelerators and has the capability of testing different configurations for the estimation of energy consumption and processing latency. Based on our analysis of the behavior of benchmark networks under different architectural parameters, we offer a few recommendations for having an efficient yet high performance accelerator design. Next, we propose a heterogeneous multi-core chip scheme for deep neural network execution. The evaluations of a selective small search space indicate that the execution of neural networks on their near-optimal core configuration can save up to 36% and 67% of energy consumption and energy-delay product respectively. Also, we suggest an algorithm to distribute the processing of network's layers across multiple cores of the same type in order to speed up the computations through model parallelism. Evaluations on different networks and with the different number of cores verify the effectiveness of the proposed algorithm in speeding up the processing to near-optimal values.
△ Less
Submitted 25 June, 2022;
originally announced June 2022.
-
HeunNet: Extending ResNet using Heun's Methods
Authors:
Mehrdad Maleki,
Mansura Habiba,
Barak A. Pearlmutter
Abstract:
There is an analogy between the ResNet (Residual Network) architecture for deep neural networks and an Euler solver for an ODE. The transformation performed by each layer resembles an Euler step in solving an ODE. We consider the Heun Method, which involves a single predictor-corrector cycle, and complete the analogy, building a predictor-corrector variant of ResNet, which we call a HeunNet. Just…
▽ More
There is an analogy between the ResNet (Residual Network) architecture for deep neural networks and an Euler solver for an ODE. The transformation performed by each layer resembles an Euler step in solving an ODE. We consider the Heun Method, which involves a single predictor-corrector cycle, and complete the analogy, building a predictor-corrector variant of ResNet, which we call a HeunNet. Just as Heun's method is more accurate than Euler's, experiments show that HeunNet achieves high accuracy with low computational (both training and test) time compared to both vanilla recurrent neural networks and other ResNet variants.
△ Less
Submitted 14 May, 2021; v1 submitted 13 May, 2021;
originally announced May 2021.
-
Using an Epidemiological Model to Study the Spread of Misinformation during the Black Lives Matter Movement
Authors:
Maryam Maleki,
Esther Mead,
Mohammad Arani,
Nitin Agarwal
Abstract:
The proliferation of social media platforms like Twitter has heightened the consequences of the spread of misinformation. To understand and model the spread of misinformation, in this paper, we leveraged the SEIZ (Susceptible, Exposed, Infected, Skeptics) epidemiological model to describe the underlying process that delineates the spread of misinformation on Twitter. Compared to the other epidemio…
▽ More
The proliferation of social media platforms like Twitter has heightened the consequences of the spread of misinformation. To understand and model the spread of misinformation, in this paper, we leveraged the SEIZ (Susceptible, Exposed, Infected, Skeptics) epidemiological model to describe the underlying process that delineates the spread of misinformation on Twitter. Compared to the other epidemiological models, this model produces broader results because it includes the additional Skeptics (Z) compartment, wherein a user may be exposed to an item of misinformation but not engage in any reaction to it, and the additional Exposed (E) compartment, wherein the user may need some time before deciding to spread a misinformation item. We analyzed misinformation regarding the unrest in Washington, D.C. in the month of March 2020 which was propagated by the use of the #DCblackout hashtag by different users across the U.S. on Twitter. Our analysis shows that misinformation can be modeled using the concept of epidemiology. To the best of our knowledge, this research is the first to attempt to apply the SEIZ epidemiological model to the spread of a specific item of misinformation, which is a category distinct from that of rumor, and a hoax on online social media platforms. Applying a mathematical model can help to understand the trends and dynamics of the spread of misinformation on Twitter and ultimately help to develop techniques to quickly identify and control it.
△ Less
Submitted 22 March, 2021;
originally announced March 2021.
-
Diagnosis of COVID-19 and Non-COVID-19 Patients by Classifying Only a Single Cough Sound
Authors:
Masoud Maleki
Abstract:
In this study, we proposed a machine learning-based system to distinguish patients with COVID-19 from non-COVID-19 patients by analyzing only a single cough sound. Two different data sets were used, one accessible for the public and the other available on request. After combining the data sets, the features were obtained from the cough sounds using the mel-frequency cepstral coefficients (MFCCs) m…
▽ More
In this study, we proposed a machine learning-based system to distinguish patients with COVID-19 from non-COVID-19 patients by analyzing only a single cough sound. Two different data sets were used, one accessible for the public and the other available on request. After combining the data sets, the features were obtained from the cough sounds using the mel-frequency cepstral coefficients (MFCCs) method, and then they were classified with seven different machine learning classifiers. To determine the optimum values of hyperparameters for MFCCs and classifiers, the leave-one-out cross-validation (LOO-CV) strategy was implemented. Based on the results, the k-nearest neighbors classifier based on the Euclidean distance (k-NN Euclidean) with the accuracy rate, sensitivity of COVID-19, sensitivity of non-COVID-19, F-measure, and area under the ROC curve (AUC) of 0.9833, 1.0000, 0.9720, 0.9799, and 0.9860, respectively, is more successful than other classifiers. Finally, the best and most effective features were determined for each classifier using the sequential forward selection (SFS) method. According to the results, the proposed system is excellent compared with similar studies in the literature and can be easily used in smartphones and facilitate the diagnosis of COVID-19 patients. In addition, since the used data set includes reflex and unconscious coughs, the results showed that conscious or unconscious coughing has no effect on the diagnosis of COVID-19 patients based on the cough sound.
△ Less
Submitted 8 February, 2021;
originally announced February 2021.
-
Impact of Autonomous Vehicle Technology on Long Distance Travel Behavior
Authors:
Maryam Maleki,
Yupo Chan,
Mohammad Arani
Abstract:
Although rapid progress in-vehicle automated technology has sped up the possibility of using fully automated technology for public use, little research has been done on the possible influences of autonomous vehicles (AVs) technology on long-distance travel. This technology has the potential to have a significant effect on intercity trips. This study analyzed a travel survey to anticipate the impac…
▽ More
Although rapid progress in-vehicle automated technology has sped up the possibility of using fully automated technology for public use, little research has been done on the possible influences of autonomous vehicles (AVs) technology on long-distance travel. This technology has the potential to have a significant effect on intercity trips. This study analyzed a travel survey to anticipate the impact of this technology on long-distance trips. We have divided trips into two different categories including trips for pleasure and trips for business. Different hypotheses based on the authors' knowledge and assisted by existing literature have been defined for each type of trip. By using the Pearson method these hypotheses have been tested and the positive or negative responses from respondents have been evaluated. The findings show that using AVs for pleasure trips can increase the number of travelers and stimulate people to choose longer distances for their trips. In addition, people enjoy more and will be interested to travel more frequently. For business trips, AV technology can reduce travel costs and job-related stress. Unlike pleasure trips for which people are not interested in traveling at night, business travelers prefer to travel at night.
△ Less
Submitted 12 January, 2021;
originally announced January 2021.
-
REBD:A Conceptual Framework for Big Data Requirements Engineering
Authors:
Sandhya Rani Kourla,
Eesha Putti,
Mina Maleki
Abstract:
Requirements engineering (RE), as a part of the project development life cycle, has increasingly been recognized as the key to ensuring on-time, on-budget, and goal-based delivery of software projects;compromising this vital phase is nothing but project failures. RE of big data projects is even more crucial because of the main characteristics of big data, including high volume, velocity, and varie…
▽ More
Requirements engineering (RE), as a part of the project development life cycle, has increasingly been recognized as the key to ensuring on-time, on-budget, and goal-based delivery of software projects;compromising this vital phase is nothing but project failures. RE of big data projects is even more crucial because of the main characteristics of big data, including high volume, velocity, and variety. As the traditional RE methods and tools are user-centric rather than data-centric, employing these methodologies is insufficient to fulfill the RE processes for big data projects. Because of the importance of RE and limitations of traditional RE methodologies in the context of big data software projects, in this paper, a big data requirements engineering framework, named REBD, has been proposed. This conceptual framework describes the systematic plan to carry out big data projects starting from requirements engineering to the development, assuring successful execution, and increased productivity of the big data projects.
△ Less
Submitted 19 June, 2020;
originally announced June 2020.
-
A Novel Deep Learning Based Approach for Left Ventricle Segmentation in Echocardiography: MFP-Unet
Authors:
Shakiba Moradi,
Mostafa Ghelich-Oghli,
Azin Alizadehasl,
Isaac Shiri,
Niki Oveisi,
Mehrdad Oveisi,
Majid Maleki,
Jan Dhooge
Abstract:
Segmentation of the Left ventricle (LV) is a crucial step for quantitative measurements such as area, volume, and ejection fraction. However, the automatic LV segmentation in 2D echocardiographic images is a challenging task due to ill-defined borders, and operator dependence issues (insufficient reproducibility). U-net, which is a well-known architecture in medical image segmentation, addressed t…
▽ More
Segmentation of the Left ventricle (LV) is a crucial step for quantitative measurements such as area, volume, and ejection fraction. However, the automatic LV segmentation in 2D echocardiographic images is a challenging task due to ill-defined borders, and operator dependence issues (insufficient reproducibility). U-net, which is a well-known architecture in medical image segmentation, addressed this problem through an encoder-decoder path. Despite outstanding overall performance, U-net ignores the contribution of all semantic strengths in the segmentation procedure. In the present study, we have proposed a novel architecture to tackle this drawback. Feature maps in all levels of the decoder path of U-net are concatenated, their depths are equalized, and up-sampled to a fixed dimension. This stack of feature maps would be the input of the semantic segmentation layer. The proposed network yielded state-of-the-art results when comparing with results from U-net, dilated U-net, and deeplabv3, using the same dataset. An average Dice Metric (DM) of 0.945, Hausdorff Distance (HD) of 1.62, Jaccard Coefficient (JC) of 0.97, and Mean Absolute Distance (MAD) of 1.32 are achieved. The correlation graph, bland-altman analysis, and box plot showed a great agreement between automatic and manually calculated volume, area, and length.
△ Less
Submitted 22 December, 2019; v1 submitted 25 June, 2019;
originally announced June 2019.
-
Deep Learning with Attention to Predict Gestational Age of the Fetal Brain
Authors:
Liyue Shen,
Katie Shpanskaya,
Edward Lee,
Emily McKenna,
Maryam Maleki,
Quin Lu,
Safwan Halabi,
John Pauly,
Kristen Yeom
Abstract:
Fetal brain imaging is a cornerstone of prenatal screening and early diagnosis of congenital anomalies. Knowledge of fetal gestational age is the key to the accurate assessment of brain development. This study develops an attention-based deep learning model to predict gestational age of the fetal brain. The proposed model is an end-to-end framework that combines key insights from multi-view MRI in…
▽ More
Fetal brain imaging is a cornerstone of prenatal screening and early diagnosis of congenital anomalies. Knowledge of fetal gestational age is the key to the accurate assessment of brain development. This study develops an attention-based deep learning model to predict gestational age of the fetal brain. The proposed model is an end-to-end framework that combines key insights from multi-view MRI including axial, coronal, and sagittal views. The model also uses age-activated weakly-supervised attention maps to enable rotation-invariant localization of the fetal brain among background noise. We evaluate our methods on the collected fetal brain MRI cohort with a large age distribution from 125 to 273 days. Our extensive experiments show age prediction performance with R2 = 0.94 using multi-view MRI and attention.
△ Less
Submitted 9 December, 2018;
originally announced December 2018.
-
Spatial Sensing and Cognitive Radio Communication in the Presence of A $K$-User Interference Primary Network
Authors:
Ardalan Alizadeh,
Hamid Reza Bahrami,
Mehdi Maleki,
Shivakumar Sastry
Abstract:
We study the feasibility of cognitive radio (CR) communication in the presence of a $K$-user multi-input multi-output (MIMO) interference channel as the primary network. Assuming that the primary interference network has unused spatial degrees of freedom (DoFs), we first investigate the sufficient condition on the number of antennas at the secondary transmitter under which the secondary system can…
▽ More
We study the feasibility of cognitive radio (CR) communication in the presence of a $K$-user multi-input multi-output (MIMO) interference channel as the primary network. Assuming that the primary interference network has unused spatial degrees of freedom (DoFs), we first investigate the sufficient condition on the number of antennas at the secondary transmitter under which the secondary system can communicate while causing no interference to the primary receivers. We show that, to maximize the benefit, the secondary transmitter should have at least the same number of antennas as the spatial DoFs of the primary system. We then derive the secondary precoding and decoding matrices to have zero interference leakage into the primary network while the signal-to-interference plus noise ratio (SINR) at the secondary receiver is maximized. As the success of the secondary communication depends on the availability of unused DoFs, we then propose a fast sensing method based on the eigenvalue analysis of the received signal covariance matrix to determine the availability of unused DoFs or equivalently spatial holes. Since the proposed fast sensing method cannot identify the indices of inactive primary streams, we also provide a fine sensing method based on the generalized likelihood ratio test (GLRT) to decide the absence of individual primary streams. Simulation results show that the proposed CR sensing and transmission scheme can, in practice, provide a significant throughput while causing no interference to the primary receivers, and that the sensing detects the spatial holes of the primary network with high detection probability.
△ Less
Submitted 12 September, 2014;
originally announced September 2014.
