-
Model Predictive Optimized Path Integral Strategies
Authors:
Dylan M. Asmar,
Ransalu Senanayake,
Shawn Manuel,
Mykel J. Kochenderfer
Abstract:
We generalize the derivation of model predictive path integral control (MPPI) to allow for a single joint distribution across controls in the control sequence. This reformation allows for the implementation of adaptive importance sampling (AIS) algorithms into the original importance sampling step while still maintaining the benefits of MPPI such as working with arbitrary system dynamics and cost…
▽ More
We generalize the derivation of model predictive path integral control (MPPI) to allow for a single joint distribution across controls in the control sequence. This reformation allows for the implementation of adaptive importance sampling (AIS) algorithms into the original importance sampling step while still maintaining the benefits of MPPI such as working with arbitrary system dynamics and cost functions. The benefit of optimizing the proposal distribution by integrating AIS at each control step is demonstrated in simulated environments including controlling multiple cars around a track. The new algorithm is more sample efficient than MPPI, achieving better performance with fewer samples. This performance disparity grows as the dimension of the action space increases. Results from simulations suggest the new algorithm can be used as an anytime algorithm, increasing the value of control at each iteration versus relying on a large set of samples.
△ Less
Submitted 1 March, 2023; v1 submitted 30 March, 2022;
originally announced March 2022.
-
LASER: Learning a Latent Action Space for Efficient Reinforcement Learning
Authors:
Arthur Allshire,
Roberto Martín-Martín,
Charles Lin,
Shawn Manuel,
Silvio Savarese,
Animesh Garg
Abstract:
The process of learning a manipulation task depends strongly on the action space used for exploration: posed in the incorrect action space, solving a task with reinforcement learning can be drastically inefficient. Additionally, similar tasks or instances of the same task family impose latent manifold constraints on the most effective action space: the task family can be best solved with actions i…
▽ More
The process of learning a manipulation task depends strongly on the action space used for exploration: posed in the incorrect action space, solving a task with reinforcement learning can be drastically inefficient. Additionally, similar tasks or instances of the same task family impose latent manifold constraints on the most effective action space: the task family can be best solved with actions in a manifold of the entire action space of the robot. Combining these insights we present LASER, a method to learn latent action spaces for efficient reinforcement learning. LASER factorizes the learning problem into two sub-problems, namely action space learning and policy learning in the new action space. It leverages data from similar manipulation task instances, either from an offline expert or online during policy learning, and learns from these trajectories a map** from the original to a latent action space. LASER is trained as a variational encoder-decoder model to map raw actions into a disentangled latent action space while maintaining action reconstruction and latent space dynamic consistency. We evaluate LASER on two contact-rich robotic tasks in simulation, and analyze the benefit of policy learning in the generated latent action space. We show improved sample efficiency compared to the original action space from better alignment of the action space to the task space, as we observe with visualizations of the learned action space manifold. Additional details: https://www.pair.toronto.edu/laser
△ Less
Submitted 30 March, 2021; v1 submitted 29 March, 2021;
originally announced March 2021.
-
Oscillation Criteria for Higher Order Nonlinear Generalized Neutral Difference Equations
Authors:
Adem Kilicman,
P. Venkata Mohan Reddy,
M. Maria Susai Manuel
Abstract:
In the present study we highlight some results related to the oscillation for high order nonlinear generalized neutral difference equation in the following form \begin{equation*} Δ_{\ell}\left(a(k)Δ_{\ell}^{m-1}z(k)\right)+q(k)f(x({k-ρ\ell}))=0, \label{1e} \end{equation*} where $z(k)=x(k)+p(k)x({τ(k)})$.
In the present study we highlight some results related to the oscillation for high order nonlinear generalized neutral difference equation in the following form \begin{equation*} Δ_{\ell}\left(a(k)Δ_{\ell}^{m-1}z(k)\right)+q(k)f(x({k-ρ\ell}))=0, \label{1e} \end{equation*} where $z(k)=x(k)+p(k)x({τ(k)})$.
△ Less
Submitted 8 August, 2018;
originally announced August 2018.
