-
Long-term memory induced correction to Arrhenius law
Authors:
A. Barbier-Chebbah,
O. Bénichou,
R. Voituriez,
T. Guérin
Abstract:
The Kramers escape problem is a paradigmatic model for the kinetics of rare events, which are usually characterized by Arrhenius law. So far, analytical approaches have failed to capture the kinetics of rare events in the important case of non-Markovian processes with long-term memory, as occurs in the context of reactions involving proteins, long polymers, or strongly viscoelastic fluids. Here, b…
▽ More
The Kramers escape problem is a paradigmatic model for the kinetics of rare events, which are usually characterized by Arrhenius law. So far, analytical approaches have failed to capture the kinetics of rare events in the important case of non-Markovian processes with long-term memory, as occurs in the context of reactions involving proteins, long polymers, or strongly viscoelastic fluids. Here, based on a minimal model of non-Markovian Gaussian process with long-term memory, we determine quantitatively the mean FPT to a rare configuration and provide its asymptotics in the limit of a large energy barrier $E$. Our analysis unveils a correction to Arrhenius law, induced by long-term memory, which we determine analytically. This correction, which we show can be quantitatively significant, takes the form of a second effective energy barrier $E'<E$ and captures the dependence of rare event kinetics on initial conditions, which is a hallmark of long-term memory. Altogether, our results quantify the impact of long-term memory on rare event kinetics, beyond Arrhenius law.
△ Less
Submitted 7 June, 2024;
originally announced June 2024.
-
Approximate information maximization for bandit games
Authors:
Alex Barbier-Chebbah,
Christian L. Vestergaard,
Jean-Baptiste Masson,
Etienne Boursier
Abstract:
Entropy maximization and free energy minimization are general physical principles for modeling the dynamics of various physical systems. Notable examples include modeling decision-making within the brain using the free-energy principle, optimizing the accuracy-complexity trade-off when accessing hidden variables with the information bottleneck principle (Tishby et al., 2000), and navigation in ran…
▽ More
Entropy maximization and free energy minimization are general physical principles for modeling the dynamics of various physical systems. Notable examples include modeling decision-making within the brain using the free-energy principle, optimizing the accuracy-complexity trade-off when accessing hidden variables with the information bottleneck principle (Tishby et al., 2000), and navigation in random environments using information maximization (Vergassola et al., 2007). Built on this principle, we propose a new class of bandit algorithms that maximize an approximation to the information of a key variable within the system. To this end, we develop an approximated analytical physics-based representation of an entropy to forecast the information gain of each action and greedily choose the one with the largest information gain. This method yields strong performances in classical bandit settings. Motivated by its empirical success, we prove its asymptotic optimality for the two-armed bandit problem with Gaussian rewards. Owing to its ability to encompass the system's properties in a global physical functional, this approach can be efficiently adapted to more complex bandit settings, calling for further investigation of information maximization approaches for multi-armed bandit problems.
△ Less
Submitted 30 October, 2023; v1 submitted 19 October, 2023;
originally announced October 2023.
-
Approximate information for efficient exploration-exploitation strategies
Authors:
Alex Barbier-Chebbah,
Christian L. Vestergaard,
Jean-Baptiste Masson
Abstract:
This paper addresses the exploration-exploitation dilemma inherent in decision-making, focusing on multi-armed bandit problems. The problems involve an agent deciding whether to exploit current knowledge for immediate gains or explore new avenues for potential long-term rewards. We here introduce a novel algorithm, approximate information maximization (AIM), which employs an analytical approximati…
▽ More
This paper addresses the exploration-exploitation dilemma inherent in decision-making, focusing on multi-armed bandit problems. The problems involve an agent deciding whether to exploit current knowledge for immediate gains or explore new avenues for potential long-term rewards. We here introduce a novel algorithm, approximate information maximization (AIM), which employs an analytical approximation of the entropy gradient to choose which arm to pull at each point in time. AIM matches the performance of Infomax and Thompson sampling while also offering enhanced computational speed, determinism, and tractability. Empirical evaluation of AIM indicates its compliance with the Lai-Robbins asymptotic bound and demonstrates its robustness for a range of priors. Its expression is tunable, which allows for specific optimization in various settings.
△ Less
Submitted 4 July, 2023;
originally announced July 2023.
-
Joint statistics of space and time exploration of $1d$ random walks
Authors:
J. Klinger,
A. Barbier-Chebbah,
R. Voituriez,
O. Bénichou
Abstract:
The statistics of first-passage times of random walks to target sites has proved to play a key role in determining the kinetics of space exploration in various contexts. In parallel, the number of distinct sites visited by a random walker and related observables have been introduced to characterize the geometry of space exploration. Here, we address the question of the joint distribution of the fi…
▽ More
The statistics of first-passage times of random walks to target sites has proved to play a key role in determining the kinetics of space exploration in various contexts. In parallel, the number of distinct sites visited by a random walker and related observables have been introduced to characterize the geometry of space exploration. Here, we address the question of the joint distribution of the first-passage time to a target and the number of distinct sites visited when the target is reached, which fully quantifies the coupling between kinetics and geometry of search trajectories. Focusing on 1-dimensional systems, we present a general method and derive explicit expressions of this joint distribution for several representative examples of Markovian search processes. In addition, we obtain a general scaling form, which holds also for non Markovian processes and captures the general dependence of the joint distribution on its space and time variables. We argue that the joint distribution has important applications to various problems, such as a conditional form of the Rosenstock trap** model, and the persistence properties of self-interacting random walks.
△ Less
Submitted 28 September, 2021;
originally announced September 2021.
