-
Correlation functions between singular values and eigenvalues
Authors:
Matthias Allard,
Mario Kieburg
Abstract:
Exploiting the explicit bijection between the density of singular values and the density of eigenvalues for bi-unitarily invariant complex random matrix ensembles of finite matrix size we aim at finding the induced probability measure on $j$ eigenvalues and $k$ singular values that we coin $j,k$-point correlation measure. We fully derive all $j,k$-point correlation measures in the simplest cases f…
▽ More
Exploiting the explicit bijection between the density of singular values and the density of eigenvalues for bi-unitarily invariant complex random matrix ensembles of finite matrix size we aim at finding the induced probability measure on $j$ eigenvalues and $k$ singular values that we coin $j,k$-point correlation measure. We fully derive all $j,k$-point correlation measures in the simplest cases for matrices of size $n=1$ and $n=2$. For $n>2$, we find a general formula for the $1,1$-point correlation measure. This formula reduces drastically when assuming the singular values are drawn from a polynomial ensemble, yielding an explicit formula in terms of the kernel corresponding to the singular value statistics. These expressions simplify even further when the singular values are drawn from a Pólya ensemble and extend known results between the eigenvalue and singular value statistics of the corresponding bi-unitarily invariant ensemble.
△ Less
Submitted 10 April, 2024; v1 submitted 28 March, 2024;
originally announced March 2024.
-
QDax: A Library for Quality-Diversity and Population-based Algorithms with Hardware Acceleration
Authors:
Felix Chalumeau,
Bryan Lim,
Raphael Boige,
Maxime Allard,
Luca Grillotti,
Manon Flageat,
Valentin Macé,
Arthur Flajolet,
Thomas Pierrot,
Antoine Cully
Abstract:
QDax is an open-source library with a streamlined and modular API for Quality-Diversity (QD) optimization algorithms in Jax. The library serves as a versatile tool for optimization purposes, ranging from black-box optimization to continuous control. QDax offers implementations of popular QD, Neuroevolution, and Reinforcement Learning (RL) algorithms, supported by various examples. All the implemen…
▽ More
QDax is an open-source library with a streamlined and modular API for Quality-Diversity (QD) optimization algorithms in Jax. The library serves as a versatile tool for optimization purposes, ranging from black-box optimization to continuous control. QDax offers implementations of popular QD, Neuroevolution, and Reinforcement Learning (RL) algorithms, supported by various examples. All the implementations can be just-in-time compiled with Jax, facilitating efficient execution across multiple accelerators, including GPUs and TPUs. These implementations effectively demonstrate the framework's flexibility and user-friendliness, easing experimentation for research purposes. Furthermore, the library is thoroughly documented and tested with 95\% coverage.
△ Less
Submitted 7 August, 2023;
originally announced August 2023.
-
Benchmarking Quality-Diversity Algorithms on Neuroevolution for Reinforcement Learning
Authors:
Manon Flageat,
Bryan Lim,
Luca Grillotti,
Maxime Allard,
Simón C. Smith,
Antoine Cully
Abstract:
We present a Quality-Diversity benchmark suite for Deep Neuroevolution in Reinforcement Learning domains for robot control. The suite includes the definition of tasks, environments, behavioral descriptors, and fitness. We specify different benchmarks based on the complexity of both the task and the agent controlled by a deep neural network. The benchmark uses standard Quality-Diversity metrics, in…
▽ More
We present a Quality-Diversity benchmark suite for Deep Neuroevolution in Reinforcement Learning domains for robot control. The suite includes the definition of tasks, environments, behavioral descriptors, and fitness. We specify different benchmarks based on the complexity of both the task and the agent controlled by a deep neural network. The benchmark uses standard Quality-Diversity metrics, including coverage, QD-score, maximum fitness, and an archive profile metric to quantify the relation between coverage and fitness. We also present how to quantify the robustness of the solutions with respect to environmental stochasticity by introducing corrected versions of the same metrics. We believe that our benchmark is a valuable tool for the community to compare and improve their findings. The source code is available online: https://github.com/adaptive-intelligent-robotics/QDax
△ Less
Submitted 3 November, 2022;
originally announced November 2022.
-
Online Damage Recovery for Physical Robots with Hierarchical Quality-Diversity
Authors:
Maxime Allard,
Simón C. Smith,
Konstantinos Chatzilygeroudis,
Bryan Lim,
Antoine Cully
Abstract:
In real-world environments, robots need to be resilient to damages and robust to unforeseen scenarios. Quality-Diversity (QD) algorithms have been successfully used to make robots adapt to damages in seconds by leveraging a diverse set of learned skills. A high diversity of skills increases the chances of a robot to succeed at overcoming new situations since there are more potential alternatives t…
▽ More
In real-world environments, robots need to be resilient to damages and robust to unforeseen scenarios. Quality-Diversity (QD) algorithms have been successfully used to make robots adapt to damages in seconds by leveraging a diverse set of learned skills. A high diversity of skills increases the chances of a robot to succeed at overcoming new situations since there are more potential alternatives to solve a new task.However, finding and storing a large behavioural diversity of multiple skills often leads to an increase in computational complexity. Furthermore, robot planning in a large skill space is an additional challenge that arises with an increased number of skills. Hierarchical structures can help reducing this search and storage complexity by breaking down skills into primitive skills. In this paper, we introduce the Hierarchical Trial and Error algorithm, which uses a hierarchical behavioural repertoire to learn diverse skills and leverages them to make the robot adapt quickly in the physical world. We show that the hierarchical decomposition of skills enables the robot to learn more complex behaviours while kee** the learning of the repertoire tractable. Experiments with a hexapod robot show that our method solves a maze navigation tasks with 20% less actions in simulation, and 43% less actions in the physical world, for the most challenging scenarios than the best baselines while having 78% less complete failures.
△ Less
Submitted 18 October, 2022;
originally announced October 2022.
-
Neuroevolution is a Competitive Alternative to Reinforcement Learning for Skill Discovery
Authors:
Felix Chalumeau,
Raphael Boige,
Bryan Lim,
Valentin Macé,
Maxime Allard,
Arthur Flajolet,
Antoine Cully,
Thomas Pierrot
Abstract:
Deep Reinforcement Learning (RL) has emerged as a powerful paradigm for training neural policies to solve complex control tasks. However, these policies tend to be overfit to the exact specifications of the task and environment they were trained on, and thus do not perform well when conditions deviate slightly or when composed hierarchically to solve even more complex tasks. Recent work has shown…
▽ More
Deep Reinforcement Learning (RL) has emerged as a powerful paradigm for training neural policies to solve complex control tasks. However, these policies tend to be overfit to the exact specifications of the task and environment they were trained on, and thus do not perform well when conditions deviate slightly or when composed hierarchically to solve even more complex tasks. Recent work has shown that training a mixture of policies, as opposed to a single one, that are driven to explore different regions of the state-action space can address this shortcoming by generating a diverse set of behaviors, referred to as skills, that can be collectively used to great effect in adaptation tasks or for hierarchical planning. This is typically realized by including a diversity term - often derived from information theory - in the objective function optimized by RL. However these approaches often require careful hyperparameter tuning to be effective. In this work, we demonstrate that less widely-used neuroevolution methods, specifically Quality Diversity (QD), are a competitive alternative to information-theory-augmented RL for skill discovery. Through an extensive empirical evaluation comparing eight state-of-the-art algorithms (four flagship algorithms from each line of work) on the basis of (i) metrics directly evaluating the skills' diversity, (ii) the skills' performance on adaptation tasks, and (iii) the skills' performance when used as primitives for hierarchical planning; QD methods are found to provide equal, and sometimes improved, performance whilst being less sensitive to hyperparameters and more scalable. As no single method is found to provide near-optimal performance across all environments, there is a rich scope for further research which we support by proposing future directions and providing optimized open-source implementations.
△ Less
Submitted 8 September, 2023; v1 submitted 6 October, 2022;
originally announced October 2022.
-
Hierarchical Quality-Diversity for Online Damage Recovery
Authors:
Maxime Allard,
Simón C. Smith,
Konstantinos Chatzilygeroudis,
Antoine Cully
Abstract:
Adaptation capabilities, like damage recovery, are crucial for the deployment of robots in complex environments. Several works have demonstrated that using repertoires of pre-trained skills can enable robots to adapt to unforeseen mechanical damages in a few minutes. These adaptation capabilities are directly linked to the behavioural diversity in the repertoire. The more alternatives the robot ha…
▽ More
Adaptation capabilities, like damage recovery, are crucial for the deployment of robots in complex environments. Several works have demonstrated that using repertoires of pre-trained skills can enable robots to adapt to unforeseen mechanical damages in a few minutes. These adaptation capabilities are directly linked to the behavioural diversity in the repertoire. The more alternatives the robot has to execute a skill, the better are the chances that it can adapt to a new situation. However, solving complex tasks, like maze navigation, usually requires multiple different skills. Finding a large behavioural diversity for these multiple skills often leads to an intractable exponential growth of the number of required solutions. In this paper, we introduce the Hierarchical Trial and Error algorithm, which uses a hierarchical behavioural repertoire to learn diverse skills and leverages them to make the robot more adaptive to different situations. We show that the hierarchical decomposition of skills enables the robot to learn more complex behaviours while kee** the learning of the repertoire tractable. The experiments with a hexapod robot show that our method solves maze navigation tasks with 20% less actions in the most challenging scenarios than the best baseline while having 57% less complete failures.
△ Less
Submitted 12 April, 2022;
originally announced April 2022.
-
Accelerated Quality-Diversity through Massive Parallelism
Authors:
Bryan Lim,
Maxime Allard,
Luca Grillotti,
Antoine Cully
Abstract:
Quality-Diversity (QD) optimization algorithms are a well-known approach to generate large collections of diverse and high-quality solutions. However, derived from evolutionary computation, QD algorithms are population-based methods which are known to be data-inefficient and requires large amounts of computational resources. This makes QD algorithms slow when used in applications where solution ev…
▽ More
Quality-Diversity (QD) optimization algorithms are a well-known approach to generate large collections of diverse and high-quality solutions. However, derived from evolutionary computation, QD algorithms are population-based methods which are known to be data-inefficient and requires large amounts of computational resources. This makes QD algorithms slow when used in applications where solution evaluations are computationally costly. A common approach to speed up QD algorithms is to evaluate solutions in parallel, for instance by using physical simulators in robotics. Yet, this approach is limited to several dozen of parallel evaluations as most physics simulators can only be parallelized more with a greater number of CPUs. With recent advances in simulators that run on accelerators, thousands of evaluations can now be performed in parallel on single GPU/TPU. In this paper, we present QDax, an accelerated implementation of MAP-Elites which leverages massive parallelism on accelerators to make QD algorithms more accessible. We show that QD algorithms are ideal candidates to take advantage of progress in hardware acceleration. We demonstrate that QD algorithms can scale with massive parallelism to be run at interactive timescales without any significant effect on the performance. Results across standard optimization functions and four neuroevolution benchmark environments shows that experiment runtimes are reduced by two factors of magnitudes, turning days of computation into minutes. More surprising, we observe that reducing the number of generations by two orders of magnitude, and thus having significantly shorter lineage does not impact the performance of QD algorithms. These results show that QD can now benefit from hardware acceleration, which contributed significantly to the bloom of deep learning.
△ Less
Submitted 10 October, 2022; v1 submitted 2 February, 2022;
originally announced February 2022.
-
Automated Process Planning for Turning: A Feature-Free Approach
Authors:
Morad Behandish,
Saigopal Nelaturi,
Chaman Singh Verma,
Mats Allard
Abstract:
Turning is the most commonly available and least expensive machining operation, in terms of both machine-hour rates and tool insert prices. A practical CNC process planner has to maximize the utilization of turning, not only to attain precision requirements for turnable surfaces, but also to minimize the machining cost, while non-turnable features can be left for other processes such as milling. M…
▽ More
Turning is the most commonly available and least expensive machining operation, in terms of both machine-hour rates and tool insert prices. A practical CNC process planner has to maximize the utilization of turning, not only to attain precision requirements for turnable surfaces, but also to minimize the machining cost, while non-turnable features can be left for other processes such as milling. Most existing methods rely on separation of surface features and lack guarantees when analyzing complex parts with interacting features. In a previous study, we demonstrated successful implementation of a feature-free milling process planner based on configuration space methods used for spatial reasoning and AI search for planning. This paper extends the feature-free method to include turning process planning. It opens up the opportunity for seamless integration of turning actions into a mill-turn process planner that can handle arbitrarily complex shapes with or without a priori knowledge of feature semantics.
△ Less
Submitted 2 July, 2019; v1 submitted 22 May, 2019;
originally announced May 2019.
