AC4MPC: Actor-Critic Reinforcement Learning for Nonlinear Model Predictive Control
Authors:
Rudolf Reiter,
Andrea Ghezzi,
Katrin Baumgärtner,
Jasper Hoffmann,
Robert D. McAllister,
Moritz Diehl
Abstract:
\Ac{MPC} and \ac{RL} are two powerful control strategies with, arguably, complementary advantages. In this work, we show how actor-critic \ac{RL} techniques can be leveraged to improve the performance of \ac{MPC}. The \ac{RL} critic is used as an approximation of the optimal value function, and an actor roll-out provides an initial guess for primal variables of the \ac{MPC}. A parallel control arc…
▽ More
\Ac{MPC} and \ac{RL} are two powerful control strategies with, arguably, complementary advantages. In this work, we show how actor-critic \ac{RL} techniques can be leveraged to improve the performance of \ac{MPC}. The \ac{RL} critic is used as an approximation of the optimal value function, and an actor roll-out provides an initial guess for primal variables of the \ac{MPC}. A parallel control architecture is proposed where each \ac{MPC} instance is solved twice for different initial guesses. Besides the actor roll-out initialization, a shifted initialization from the previous solution is used. Thereafter, the actor and the critic are again used to approximately evaluate the infinite horizon cost of these trajectories. The control actions from the lowest-cost trajectory are applied to the system at each time step. We establish that the proposed algorithm is guaranteed to outperform the original \ac{RL} policy plus an error term that depends on the accuracy of the critic and decays with the horizon length of the \ac{MPC} formulation. Moreover, we do not require globally optimal solutions for these guarantees to hold. The approach is demonstrated on an illustrative toy example and an \ac{AD} overtaking scenario.
△ Less
Submitted 6 June, 2024;
originally announced June 2024.
Giga-voxel multidimensional fluorescence imaging combining single-pixel detection and data fusion
Authors:
Fernando Soldevila,
Armin Lenz,
Alberto Ghezzi,
Andrea Farina,
Cosimo D'Andrea,
Enrique Tajahuerce
Abstract:
Time-resolved fluorescence imaging is a key tool in biomedical applications, as it allows to non-invasively obtain functional and structural information. However, the big amount of collected data introduces challenges in both acquisition speed and processing needs. Here, we introduce a novel technique that allows to reconstruct a Giga-voxel 4D hypercube in a fast manner while only measuring 0.03 %…
▽ More
Time-resolved fluorescence imaging is a key tool in biomedical applications, as it allows to non-invasively obtain functional and structural information. However, the big amount of collected data introduces challenges in both acquisition speed and processing needs. Here, we introduce a novel technique that allows to reconstruct a Giga-voxel 4D hypercube in a fast manner while only measuring 0.03 % of the information. The system combines two single-pixel cameras and a conventional 2D array detector working in parallel. Data fusion techniques are introduced to combine the individual 2D and 3D projections acquired by each sensor in the final high-resolution 4D hypercube, which can be used to identify different fluorophore species by their spectral and temporal signatures.
△ Less
Submitted 26 July, 2021;
originally announced July 2021.
