-
Sampling-based Safe Reinforcement Learning for Nonlinear Dynamical Systems
Authors:
Wesley A. Suttle,
Vipul K. Sharma,
Krishna C. Kosaraju,
S. Sivaranjani,
Ji Liu,
Vijay Gupta,
Brian M. Sadler
Abstract:
We develop provably safe and convergent reinforcement learning (RL) algorithms for control of nonlinear dynamical systems, bridging the gap between the hard safety guarantees of control theory and the convergence guarantees of RL theory. Recent advances at the intersection of control and RL follow a two-stage, safety filter approach to enforcing hard safety constraints: model-free RL is used to le…
▽ More
We develop provably safe and convergent reinforcement learning (RL) algorithms for control of nonlinear dynamical systems, bridging the gap between the hard safety guarantees of control theory and the convergence guarantees of RL theory. Recent advances at the intersection of control and RL follow a two-stage, safety filter approach to enforcing hard safety constraints: model-free RL is used to learn a potentially unsafe controller, whose actions are projected onto safe sets prescribed, for example, by a control barrier function. Though safe, such approaches lose any convergence guarantees enjoyed by the underlying RL methods. In this paper, we develop a single-stage, sampling-based approach to hard constraint satisfaction that learns RL controllers enjoying classical convergence guarantees while satisfying hard safety constraints throughout training and deployment. We validate the efficacy of our approach in simulation, including safe control of a quadcopter in a challenging obstacle avoidance problem, and demonstrate that it outperforms existing benchmarks.
△ Less
Submitted 6 March, 2024;
originally announced March 2024.
-
Eye Gaze Controlled Robotic Arm for Persons with SSMI
Authors:
Vinay Krishna Sharma,
L. R. D. Murthy,
KamalPreet Singh Saluja,
Vimal Mollyn,
Gourav Sharma,
Pradipta Biswas
Abstract:
Background: People with severe speech and motor impairment (SSMI) often uses a technique called eye pointing to communicate with outside world. One of their parents, caretakers or teachers hold a printed board in front of them and by analyzing their eye gaze manually, their intentions are interpreted. This technique is often error prone and time consuming and depends on a single caretaker.
Objec…
▽ More
Background: People with severe speech and motor impairment (SSMI) often uses a technique called eye pointing to communicate with outside world. One of their parents, caretakers or teachers hold a printed board in front of them and by analyzing their eye gaze manually, their intentions are interpreted. This technique is often error prone and time consuming and depends on a single caretaker.
Objective: We aimed to automate the eye tracking process electronically by using commercially available tablet, computer or laptop and without requiring any dedicated hardware for eye gaze tracking. The eye gaze tracker is used to develop a video see through based AR (augmented reality) display that controls a robotic device with eye gaze and deployed for a fabric printing task.
Methodology: We undertook a user centred design process and separately evaluated the web cam based gaze tracker and the video see through based human robot interaction involving users with SSMI. We also reported a user study on manipulating a robotic arm with webcam based eye gaze tracker.
Results: Using our bespoke eye gaze controlled interface, able bodied users can select one of nine regions of screen at a median of less than 2 secs and users with SSMI can do so at a median of 4 secs. Using the eye gaze controlled human-robot AR display, users with SSMI could undertake representative pick and drop task at an average duration less than 15 secs and reach a randomly designated target within 60 secs using a COTS eye tracker and at an average time of 2 mins using the webcam based eye gaze tracker.
△ Less
Submitted 25 May, 2020;
originally announced May 2020.
-
Design and Implementation of Modified Fuzzy based CPU Scheduling Algorithm
Authors:
Rajani Kumari,
Vivek Kumar Sharma,
Sandeep Kumar
Abstract:
CPU Scheduling is the base of multiprogramming. Scheduling is a process which decides order of task from a set of multiple tasks that are ready to execute. There are number of CPU scheduling algorithms available, but it is very difficult task to decide which one is better. This paper discusses the design and implementation of modified fuzzy based CPU scheduling algorithm. This paper present a new…
▽ More
CPU Scheduling is the base of multiprogramming. Scheduling is a process which decides order of task from a set of multiple tasks that are ready to execute. There are number of CPU scheduling algorithms available, but it is very difficult task to decide which one is better. This paper discusses the design and implementation of modified fuzzy based CPU scheduling algorithm. This paper present a new set of fuzzy rules. It demonstrates that scheduling done with new priority improves average waiting time and average turnaround time.
△ Less
Submitted 26 May, 2017;
originally announced June 2017.
-
Randomized Memetic Artificial Bee Colony Algorithm
Authors:
Sandeep Kumar,
Vivek Kumar Sharma,
Rajani Kumari
Abstract:
Artificial Bee Colony (ABC) optimization algorithm is one of the recent population based probabilistic approach developed for global optimization. ABC is simple and has been showed significant improvement over other Nature Inspired Algorithms (NIAs) when tested over some standard benchmark functions and for some complex real world optimization problems. Memetic Algorithms also become one of the ke…
▽ More
Artificial Bee Colony (ABC) optimization algorithm is one of the recent population based probabilistic approach developed for global optimization. ABC is simple and has been showed significant improvement over other Nature Inspired Algorithms (NIAs) when tested over some standard benchmark functions and for some complex real world optimization problems. Memetic Algorithms also become one of the key methodologies to solve the very large and complex real-world optimization problems. The solution search equation of Memetic ABC is based on Golden Section Search and an arbitrary value which tries to balance exploration and exploitation of search space. But still there are some chances to skip the exact solution due to its step size. In order to balance between diversification and intensification capability of the Memetic ABC, it is randomized the step size in Memetic ABC. The proposed algorithm is named as Randomized Memetic ABC (RMABC). In RMABC, new solutions are generated nearby the best so far solution and it helps to increase the exploitation capability of Memetic ABC. The experiments on some test problems of different complexities and one well known engineering optimization application show that the proposed algorithm outperforms over Memetic ABC (MeABC) and some other variant of ABC algorithm(like Gbest guided ABC (GABC),Hooke Jeeves ABC (HJABC), Best-So-Far ABC (BSFABC) and Modified ABC (MABC) in case of almost all the problems.
△ Less
Submitted 1 August, 2014;
originally announced August 2014.
-
Memetic Search in Differential Evolution Algorithm
Authors:
Sandeep Kumar,
Vivek Kumar Sharma,
Rajani Kumari
Abstract:
Differential Evolution (DE) is a renowned optimization stratagem that can easily solve nonlinear and comprehensive problems. DE is a well known and uncomplicated population based probabilistic approach for comprehensive optimization. It has apparently outperformed a number of Evolutionary Algorithms and further search heuristics in the vein of Particle Swarm Optimization at what time of testing ov…
▽ More
Differential Evolution (DE) is a renowned optimization stratagem that can easily solve nonlinear and comprehensive problems. DE is a well known and uncomplicated population based probabilistic approach for comprehensive optimization. It has apparently outperformed a number of Evolutionary Algorithms and further search heuristics in the vein of Particle Swarm Optimization at what time of testing over both yardstick and actual world problems. Nevertheless, DE, like other probabilistic optimization algorithms, from time to time exhibits precipitate convergence and stagnates at suboptimal position. In order to stay away from stagnation behavior while maintaining an excellent convergence speed, an innovative search strategy is introduced, named memetic search in DE. In the planned strategy, positions update equation customized as per a memetic search stratagem. In this strategy a better solution participates more times in the position modernize procedure. The position update equation is inspired from the memetic search in artificial bee colony algorithm. The proposed strategy is named as Memetic Search in Differential Evolution (MSDE). To prove efficiency and efficacy of MSDE, it is tested over 8 benchmark optimization problems and three real world optimization problems. A comparative analysis has also been carried out among proposed MSDE and original DE. Results show that the anticipated algorithm go one better than the basic DE and its recent deviations in a good number of the experiments.
△ Less
Submitted 1 August, 2014;
originally announced August 2014.
-
Improved Onlooker Bee Phase in Artificial Bee Colony Algorithm
Authors:
Sandeep Kumar,
Vivek Kumar Sharma,
Rajani Kumari
Abstract:
Artificial Bee Colony (ABC) is a distinguished optimization strategy that can resolve nonlinear and multifaceted problems. It is comparatively a straightforward and modern population based probabilistic approach for comprehensive optimization. In the vein of the other population based algorithms, ABC is moreover computationally classy due to its slow nature of search procedure. The solution explor…
▽ More
Artificial Bee Colony (ABC) is a distinguished optimization strategy that can resolve nonlinear and multifaceted problems. It is comparatively a straightforward and modern population based probabilistic approach for comprehensive optimization. In the vein of the other population based algorithms, ABC is moreover computationally classy due to its slow nature of search procedure. The solution exploration equation of ABC is extensively influenced by a arbitrary quantity which helps in exploration at the cost of exploitation of the better search space. In the solution exploration equation of ABC due to the outsized step size the chance of skip** the factual solution is high. Therefore, here this paper improve onlooker bee phase with help of a local search strategy inspired by memetic algorithm to balance the diversity and convergence capability of the ABC. The proposed algorithm is named as Improved Onlooker Bee Phase in ABC (IoABC). It is tested over 12 well known un-biased test problems of diverse complexities and two engineering optimization problems; results show that the anticipated algorithm go one better than the basic ABC and its recent deviations in a good number of the experiments.
△ Less
Submitted 22 July, 2014;
originally announced July 2014.
-
A Novel Hybrid Crossover based Artificial Bee Colony Algorithm for Optimization Problem
Authors:
Sandeep Kumar,
Vivek Kumar Sharma,
Rajani Kumari
Abstract:
Artificial bee colony (ABC) algorithm has proved its importance in solving a number of problems including engineering optimization problems. ABC algorithm is one of the most popular and youngest member of the family of population based nature inspired meta-heuristic swarm intelligence method. ABC has been proved its superiority over some other Nature Inspired Algorithms (NIA) when applied for both…
▽ More
Artificial bee colony (ABC) algorithm has proved its importance in solving a number of problems including engineering optimization problems. ABC algorithm is one of the most popular and youngest member of the family of population based nature inspired meta-heuristic swarm intelligence method. ABC has been proved its superiority over some other Nature Inspired Algorithms (NIA) when applied for both benchmark functions and real world problems. The performance of search process of ABC depends on a random value which tries to balance exploration and exploitation phase. In order to increase the performance it is required to balance the exploration of search space and exploitation of optimal solution of the ABC. This paper outlines a new hybrid of ABC algorithm with Genetic Algorithm. The proposed method integrates crossover operation from Genetic Algorithm (GA) with original ABC algorithm. The proposed method is named as Crossover based ABC (CbABC). The CbABC strengthens the exploitation phase of ABC as crossover enhances exploration of search space. The CbABC tested over four standard benchmark functions and a popular continuous optimization problem.
△ Less
Submitted 21 July, 2014;
originally announced July 2014.
