-
Evolutionary Deep Reinforcement Learning Using Elite Buffer: A Novel Approach Towards DRL Combined with EA in Continuous Control Tasks
Authors:
Marzieh Sadat Esmaeeli,
Hamed Malek
Abstract:
Despite the numerous applications and success of deep reinforcement learning in many control tasks, it still suffers from many crucial problems and limitations, including temporal credit assignment with sparse reward, absence of effective exploration, and a brittle convergence that is extremely sensitive to the hyperparameters of the problem. The problems of deep reinforcement learning in continuo…
▽ More
Despite the numerous applications and success of deep reinforcement learning in many control tasks, it still suffers from many crucial problems and limitations, including temporal credit assignment with sparse reward, absence of effective exploration, and a brittle convergence that is extremely sensitive to the hyperparameters of the problem. The problems of deep reinforcement learning in continuous control, along with the success of evolutionary algorithms in facing some of these problems, have emerged the idea of evolutionary reinforcement learning, which attracted many controversies. Despite successful results in a few studies in this field, a proper and fitting solution to these problems and their limitations is yet to be presented. The present study aims to study the efficiency of combining the two fields of deep reinforcement learning and evolutionary computations further and take a step towards improving methods and the existing challenges. The "Evolutionary Deep Reinforcement Learning Using Elite Buffer" algorithm introduced a novel mechanism through inspiration from interactive learning capability and hypothetical outcomes in the human brain. In this method, the utilization of the elite buffer (which is inspired by learning based on experience generalization in the human mind), along with the existence of crossover and mutation operators, and interactive learning in successive generations, have improved efficiency, convergence, and proper advancement in the field of continuous control. According to the results of experiments, the proposed method surpasses other well-known methods in environments with high complexity and dimension and is superior in resolving the mentioned problems and limitations.
△ Less
Submitted 18 September, 2022;
originally announced September 2022.
-
RecoMed: A Knowledge-Aware Recommender System for Hypertension Medications
Authors:
Maryam Sajde,
Hamed Malek,
Mehran Mohsenzadeh
Abstract:
Background and Objective High medicine diversity has always been a significant challenge for prescription, causing confusion or doubt in physicians' decision-making process. This paper aims to develop a medicine recommender system called RecoMed to aid the physician in the prescription process of hypertension by providing information about what medications have been prescribed by other doctors and…
▽ More
Background and Objective High medicine diversity has always been a significant challenge for prescription, causing confusion or doubt in physicians' decision-making process. This paper aims to develop a medicine recommender system called RecoMed to aid the physician in the prescription process of hypertension by providing information about what medications have been prescribed by other doctors and figuring out what other medicines can be recommended in addition to the one in question. Methods There are two steps to the developed method: First, association rule mining algorithms are employed to find medicine association rules. The second step entails graph mining and clustering to present an enriched recommendation via ATC code, which itself comprises several steps. First, the initial graph is constructed from historical prescription data. Then, data pruning is performed in the second step, after which the medicines with a high repetition rate are removed at the discretion of a general medical practitioner. Next, the medicines are matched to a well-known medicine classification system called the ATC code to provide an enriched recommendation. And finally, the DBSCAN and Louvain algorithms cluster medicines in the final step. Results A list of recommended medicines is provided as the system's output, and physicians can choose one or more of the medicines based on the patient's clinical symptoms. Only the medicines of class 2, related to high blood pressure medications, are used to assess the system's performance. The results obtained from this system have been reviewed and confirmed by an expert in this field.
△ Less
Submitted 9 June, 2022; v1 submitted 9 January, 2022;
originally announced January 2022.
-
Hybrid Self-Attention NEAT: A novel evolutionary approach to improve the NEAT algorithm
Authors:
Saman Khamesian,
Hamed Malek
Abstract:
This article presents a "Hybrid Self-Attention NEAT" method to improve the original NeuroEvolution of Augmenting Topologies (NEAT) algorithm in high-dimensional inputs. Although the NEAT algorithm has shown a significant result in different challenging tasks, as input representations are high dimensional, it cannot create a well-tuned network. Our study addresses this limitation by using self-atte…
▽ More
This article presents a "Hybrid Self-Attention NEAT" method to improve the original NeuroEvolution of Augmenting Topologies (NEAT) algorithm in high-dimensional inputs. Although the NEAT algorithm has shown a significant result in different challenging tasks, as input representations are high dimensional, it cannot create a well-tuned network. Our study addresses this limitation by using self-attention as an indirect encoding method to select the most important parts of the input. In addition, we improve its overall performance with the help of a hybrid method to evolve the final network weights. The main conclusion is that Hybrid Self- Attention NEAT can eliminate the restriction of the original NEAT. The results indicate that in comparison with evolutionary algorithms, our model can get comparable scores in Atari games with raw pixels input with a much lower number of parameters.
△ Less
Submitted 14 August, 2022; v1 submitted 7 December, 2021;
originally announced December 2021.
-
Cardiac SPECT Radiomics Features Repeatability and Reproducibility: A Multi Scanner Phantom Study
Authors:
Mohammad Edalat-Javid,
Isaac Shiri,
Ghasem Hajianfar,
Hamid Abdollahi,
Niki Oveisi,
Mohammad Javadian,
Mojtaba Shamsaei Zafarghandi,
Hadi Malek,
Ahmad Bitarafan-Rajabi,
Mehrdad Oveisi
Abstract:
Background: The aim of this study was to assess the robustness of cardiac SPECT radiomics features against changes in imaging settings including acquisition and reconstruction settings.
Methods: Four scanners were used to acquire SPECT scans of a cardiac phantom with 5mCi of 99mTc. The effects of different image acquisition and reconstruction settings including the Number of View, View Matrix Si…
▽ More
Background: The aim of this study was to assess the robustness of cardiac SPECT radiomics features against changes in imaging settings including acquisition and reconstruction settings.
Methods: Four scanners were used to acquire SPECT scans of a cardiac phantom with 5mCi of 99mTc. The effects of different image acquisition and reconstruction settings including the Number of View, View Matrix Size, attenuation correction, image reconstruction algorithm, number of iterations, number of subsets, type of filter, full width at half maximum (FWHM) of Gaussian filter, Butterworth filter order, and Butterworth filter cut-off were studied. In total 5263 different images were reconstructed. Eighty-seven radiomic features including first, second, and high order textures were extracted from images. To assess reproducibility and repeatability the coefficient of variation (COV) was used for each image feature over the different imaging settings.
Result: IDMN and IDN features from GLCM, RP from GLRLM, ZE from GLSZM, and DE from GLDM feature sets were the only features that were the most reproducible (COV < 5) against changes in all imaging settings. In addition, the IDMN feature from GLCM, LALGLE, SALGLE and LGLZE from GLSZM, and SDLGLE from GLDM feature sets were the features that were less reproducible (COV>20 ) against changes in all imaging settings. Matrix size has the greatest impact on feature variability as most of features are not repeatable and 82.76 of them had (COV>20 ).
Conclusion: Repeatability and reproducibility of SPECT radiomics texture features in different imaging settings is feature-dependent, and different image acquisitions and reconstructions have different effects on radiomics texture features. Low COV radiomics features could be consider for further clinical studies.
Keywords: SPECT, Radiomics, Cardiac, Repeatability, Reproducibility
△ Less
Submitted 11 September, 2019;
originally announced September 2019.
