Showing 1–5 of 5 results for author: Noman, N

Boosting Exploration in Actor-Critic Algorithms by Incentivizing Plausible Novel States

Authors: Chayan Banerjee, Zhiyong Chen, Nasimul Noman

Abstract: Actor-critic (AC) algorithms are a class of model-free deep reinforcement learning algorithms, which have proven their efficacy in diverse domains, especially in solving continuous control problems. Improvement of exploration (action entropy) and exploitation (expected return) using more efficient samples is a critical issue in AC algorithms. A basic strategy of a learning algorithm is to facilita… ▽ More Actor-critic (AC) algorithms are a class of model-free deep reinforcement learning algorithms, which have proven their efficacy in diverse domains, especially in solving continuous control problems. Improvement of exploration (action entropy) and exploitation (expected return) using more efficient samples is a critical issue in AC algorithms. A basic strategy of a learning algorithm is to facilitate indiscriminately exploring all of the environment state space, as well as to encourage exploring rarely visited states rather than frequently visited one. Under this strategy, we propose a new method to boost exploration through an intrinsic reward, based on measurement of a state's novelty and the associated benefit of exploring the state (with regards to policy optimization), altogether called plausible novelty. With incentivized exploration of plausible novel states, an AC algorithm is able to improve its sample efficiency and hence training performance. The new method is verified by extensive simulations of continuous control tasks of MuJoCo environments on a variety of prominent off-policy AC algorithms. △ Less

Submitted 1 October, 2022; originally announced October 2022.

Application of E-Commerce Technologies in accelerating the Success of SME Operation

Authors: Ziad Hmwd A Almtiri, Shah J Miah, Nasimul Noman

Abstract: Electronic commerce (e-Commerce) technologies have been increased over the past two decades in different business sectors. In particular, the technologies of B2C operations have significantly improved the productivity of online small businesses such as SMEs. Systematic literature review in this domain categorized different benefits but a limited number of studies on SME success from a view of an i… ▽ More Electronic commerce (e-Commerce) technologies have been increased over the past two decades in different business sectors. In particular, the technologies of B2C operations have significantly improved the productivity of online small businesses such as SMEs. Systematic literature review in this domain categorized different benefits but a limited number of studies on SME success from a view of an information systems (IS) research exists, which needs to be taken for further attention. Informing through a comprehensive analysis this study introduces a conceptual framework of the application of e-Commerce technologies in accelerating the SME operation. Content analysis methodology was adopted for generating the outcome associated with the success of the technologies in SMEs. △ Less

Submitted 25 February, 2022; v1 submitted 20 October, 2021; originally announced October 2021.

Comments: Submitted to ICICT Conf 2022

Different Applications and Technologies of Internet of Things (IoT)

Authors: Feisal Masmali, Shah J. Miah, Nasimul Noman

Abstract: Internet of things (IoT) has significantly altered the traditional lifestyle to a highly technologically advanced society. Some of the significant transformations that have been achieved through IoT are smart homes, smart transportation, smart city, and control of pollution. A considerable number of studies have been conducted and continue to be done to increase the use of technology through IoT.… ▽ More Internet of things (IoT) has significantly altered the traditional lifestyle to a highly technologically advanced society. Some of the significant transformations that have been achieved through IoT are smart homes, smart transportation, smart city, and control of pollution. A considerable number of studies have been conducted and continue to be done to increase the use of technology through IoT. Furthermore, the research about IoT has not been done fully in improving the application of technology through IoT. Besides, IoT experiences several problems that need to be considered in order to get the full capability of IoT in changing society. This research paper addresses the key applications of IoT, the architecture of IoT, and the key issues affecting IoT. In addition, the paper highlights how big data analytics is essential in improving the effectiveness of IoT in various applications within society. △ Less

Submitted 25 February, 2022; v1 submitted 20 October, 2021; originally announced October 2021.

Comments: Paper is presented to ICICT 2022 Conference

Improved Soft Actor-Critic: Mixing Prioritized Off-Policy Samples with On-Policy Experience

Authors: Chayan Banerjee, Zhiyong Chen, Nasimul Noman

Abstract: Soft Actor-Critic (SAC) is an off-policy actor-critic reinforcement learning algorithm, essentially based on entropy regularization. SAC trains a policy by maximizing the trade-off between expected return and entropy (randomness in the policy). It has achieved state-of-the-art performance on a range of continuous-control benchmark tasks, outperforming prior on-policy and off-policy methods. SAC wo… ▽ More Soft Actor-Critic (SAC) is an off-policy actor-critic reinforcement learning algorithm, essentially based on entropy regularization. SAC trains a policy by maximizing the trade-off between expected return and entropy (randomness in the policy). It has achieved state-of-the-art performance on a range of continuous-control benchmark tasks, outperforming prior on-policy and off-policy methods. SAC works in an off-policy fashion where data are sampled uniformly from past experiences (stored in a buffer) using which parameters of the policy and value function networks are updated. We propose certain crucial modifications for boosting the performance of SAC and make it more sample efficient. In our proposed improved SAC, we firstly introduce a new prioritization scheme for selecting better samples from the experience replay buffer. Secondly we use a mixture of the prioritized off-policy data with the latest on-policy data for training the policy and the value function networks. We compare our approach with the vanilla SAC and some recent variants of SAC and show that our approach outperforms the said algorithmic benchmarks. It is comparatively more stable and sample efficient when tested on a number of continuous control tasks in MuJoCo environments. △ Less

Submitted 24 September, 2021; originally announced September 2021.

Optimal Actor-Critic Policy with Optimized Training Datasets

Authors: Chayan Banerjee, Zhiyong Chen, Nasimul Noman, Mohsen Zamani

Abstract: Actor-critic (AC) algorithms are known for their efficacy and high performance in solving reinforcement learning problems, but they also suffer from low sampling efficiency. An AC based policy optimization process is iterative and needs to frequently access the agent-environment system to evaluate and update the policy by rolling out the policy, collecting rewards and states (i.e. samples), and le… ▽ More Actor-critic (AC) algorithms are known for their efficacy and high performance in solving reinforcement learning problems, but they also suffer from low sampling efficiency. An AC based policy optimization process is iterative and needs to frequently access the agent-environment system to evaluate and update the policy by rolling out the policy, collecting rewards and states (i.e. samples), and learning from them. It ultimately requires a huge number of samples to learn an optimal policy. To improve sampling efficiency, we propose a strategy to optimize the training dataset that contains significantly less samples collected from the AC process. The dataset optimization is made of a best episode only operation, a policy parameter-fitness model, and a genetic algorithm module. The optimal policy network trained by the optimized training dataset exhibits superior performance compared to many contemporary AC algorithms in controlling autonomous dynamical systems. Evaluation on standard benchmarks show that the method improves sampling efficiency, ensures faster convergence to optima, and is more data-efficient than its counterparts. △ Less

Submitted 1 December, 2021; v1 submitted 16 August, 2021; originally announced August 2021.