Showing 1–8 of 8 results for author: Kobeissi, Z

Mean-field games for harvesting problems: Uniqueness, long-time behaviour and weak KAM theory

Authors: Ziad Kobeissi, Idriss Mazari-Fouquer, Domènec Ruiz-Balet

Abstract: The goal of this paper is to study a Mean Field Game (MFG) system stemming from the harvesting of resources. Modelling the latter through a reaction-diffusion equation and the harvesters as competing rational agents, we are led to a non-local (in time and space) MFG system that consists of three equations, the study of which is quite delicate. The main focus of this paper is on the derivation of a… ▽ More The goal of this paper is to study a Mean Field Game (MFG) system stemming from the harvesting of resources. Modelling the latter through a reaction-diffusion equation and the harvesters as competing rational agents, we are led to a non-local (in time and space) MFG system that consists of three equations, the study of which is quite delicate. The main focus of this paper is on the derivation of analytical results (e.g existence, uniqueness) and of long time behaviour (here, convergence to the ergodic system). We provide some explicit solutions to this ergodic system. △ Less

Submitted 10 June, 2024; originally announced June 2024.

The tragedy of the commons: A Mean-Field Game approach to the reversal of travelling waves

Authors: Ziad Kobeissi, Idriss Mazari-Fouquer, Domenec Ruiz-Balet

Abstract: The goal of this paper is to investigate an instance of the tragedy of the commons in spatially distributed harvesting games. The model we choose is that of a fishes' population that is governed by a parabolic bistable equation and that fishermen harvest. We assume that, when no fisherman is present, the fishes' population is invading (mathematically, there is an invading travelling front). Is it… ▽ More The goal of this paper is to investigate an instance of the tragedy of the commons in spatially distributed harvesting games. The model we choose is that of a fishes' population that is governed by a parabolic bistable equation and that fishermen harvest. We assume that, when no fisherman is present, the fishes' population is invading (mathematically, there is an invading travelling front). Is it possible that fishermen, when acting selfishly, each in his or her own best interest, might lead to a reversal of the travelling wave and, consequently, to an extinction of the global population? To answer this question, we model the behaviour of individual fishermen using a Mean Field Game approach, and we show that the answer is yes. We then show that, at least in some cases, if the fishermen coordinated instead of acting selfishly, each of them could make more benefit, while still guaranteeing the survival of the population. Our study is illustrated by several numerical simulations. △ Less

Submitted 3 March, 2023; v1 submitted 2 March, 2023; originally announced March 2023.

MSC Class: 35C07; 35K10; 35K57; 35Q89; 49N80

A Non-asymptotic Analysis of Non-parametric Temporal-Difference Learning

Authors: Eloïse Berthier, Ziad Kobeissi, Francis Bach

Abstract: Temporal-difference learning is a popular algorithm for policy evaluation. In this paper, we study the convergence of the regularized non-parametric TD(0) algorithm, in both the independent and Markovian observation settings. In particular, when TD is performed in a universal reproducing kernel Hilbert space (RKHS), we prove convergence of the averaged iterates to the optimal value function, even… ▽ More Temporal-difference learning is a popular algorithm for policy evaluation. In this paper, we study the convergence of the regularized non-parametric TD(0) algorithm, in both the independent and Markovian observation settings. In particular, when TD is performed in a universal reproducing kernel Hilbert space (RKHS), we prove convergence of the averaged iterates to the optimal value function, even when it does not belong to the RKHS. We provide explicit convergence rates that depend on a source condition relating the regularity of the optimal value function to the RKHS. We illustrate this convergence numerically on a simple continuous-state Markov reward process. △ Less

Submitted 24 May, 2022; originally announced May 2022.

Temporal Difference Learning with Continuous Time and State in the Stochastic Setting

Authors: Ziad Kobeissi, Francis Bach

Abstract: We consider the problem of continuous-time policy evaluation. This consists in learning through observations the value function associated with an uncontrolled continuous-time stochastic dynamic and a reward function. We propose two original variants of the well-known TD(0) method using vanishing time steps. One is model-free and the other is model-based. For both methods, we prove theoretical con… ▽ More We consider the problem of continuous-time policy evaluation. This consists in learning through observations the value function associated with an uncontrolled continuous-time stochastic dynamic and a reward function. We propose two original variants of the well-known TD(0) method using vanishing time steps. One is model-free and the other is model-based. For both methods, we prove theoretical convergence rates that we subsequently verify through numerical simulations. Alternatively, those methods can be interpreted as novel reinforcement learning approaches for approximating solutions of linear PDEs (partial differential equations) or linear BSDEs (backward stochastic differential equations). △ Less

Submitted 7 June, 2023; v1 submitted 16 February, 2022; originally announced February 2022.

Mean Field Games with monotonous interactions through the law of states and controls of the agents

Authors: Z Kobeissi

Abstract: We consider a class of Mean Field Games in which the agents may interact through the statistical distribution of their states and controls. It is supposed that the Hamiltonian behaves like a power of its arguments as they tend to infinity, with an exponent larger than one. A monotonicity assumption is also made. Existence and uniqueness are proved using a priori estimates which stem from the monot… ▽ More We consider a class of Mean Field Games in which the agents may interact through the statistical distribution of their states and controls. It is supposed that the Hamiltonian behaves like a power of its arguments as they tend to infinity, with an exponent larger than one. A monotonicity assumption is also made. Existence and uniqueness are proved using a priori estimates which stem from the monotonicity assumptions and Leray-Schauder theorem. Applications of the results are given. △ Less

Submitted 23 June, 2020; originally announced June 2020.

Mean Field Games of Controls: Finite Difference Approximations

Authors: Y Achdou, Z Kobeissi

Abstract: We consider a class of mean field games in which the agents interact through both their states and controls, and we focus on situations in which a generic agent tries to adjust her speed (control) to an average speed (the average is made in a neighborhood in the state space). In such cases, the monotonicity assumptions that are frequently made in the theory of mean field games do not hold, and uni… ▽ More We consider a class of mean field games in which the agents interact through both their states and controls, and we focus on situations in which a generic agent tries to adjust her speed (control) to an average speed (the average is made in a neighborhood in the state space). In such cases, the monotonicity assumptions that are frequently made in the theory of mean field games do not hold, and uniqueness cannot be expected in general. Such model lead to systems of forward-backward nonlinear nonlocal parabolic equations; the latter are supplemented with various kinds of boundary conditions, in particular Neumann-like boundary conditions stemming from reflection conditions on the underlying controled stochastic processes. The present work deals with numerical approximations of the above mentioned systems. After describing the finite difference scheme, we propose an iterative method for solving the systems of nonlinear equations that arise in the discrete setting; it combines a continuation method, Newton iterations and inner loops of a bigradient like solver. The numerical method is used for simulating two examples. We also make experiments on the behaviour of the iterative algorithm when the parameters of the model vary. The theory of mean field games, (MFGs for short), aims at studying deterministic or stochastic differential games (Nash equilibria) as the number of agents tends to infinity. It supposes that the rational agents are indistinguishable and individually have a negligible influence on the game, and that each individual strategy is influenced by some averages of quantities depending on the states (or the controls as in the present work) of the other agents. MFGs have been introduced in the pioneering works of J-M. Lasry and P-L. Lions [17, 18, 19]. Independently and at approximately the same time, the notion of mean field games arose in the engineering literature, see the works of M.Y. Huang, P.E. Caines and R.Malham{é} [14, 15]. The present work deals with numerical approximations of mean field games in which the agents interact through both their states and controls; it follows a more theoretical work by the second author, [16], which is devoted to the mathematical analysis of the related systems of nonlocal partial differential equations. There is not much literature on MFGs in which the agents also interact through their controls, see [13, 12, 8, 10, 7, 16]. To stress the fact that the latter situation is considered, we will sometimes use the terminology mean field games of control and the acronym MFGC. △ Less

Submitted 9 March, 2020; originally announced March 2020.

On Classical Solutions to the Mean Field Game System of Controls

Authors: Z Kobeissi

Abstract: In this paper, we consider a class of mean field games in which the optimal strategy of a representative agent depends on the statistical distribution of the states and controls. We prove some existence results for the forward-backward system of PDEs under rather natural assumptions. The main step of the proof consists of obtaining a priori estimates on the gradient of the cost function by Bernste… ▽ More In this paper, we consider a class of mean field games in which the optimal strategy of a representative agent depends on the statistical distribution of the states and controls. We prove some existence results for the forward-backward system of PDEs under rather natural assumptions. The main step of the proof consists of obtaining a priori estimates on the gradient of the cost function by Bernstein's method. Uniqueness is also proved under more restrictive assumptions. The last section contains some examples to which the previously mentioned existence (and possibly uniqueness) results apply. △ Less

Submitted 10 July, 2020; v1 submitted 25 April, 2019; originally announced April 2019.

On the implementation of a primal-dual algorithm for second order time-dependent mean field games with local couplings

Authors: Luis Briceño-Arias, Dante Kalise, Ziad Kobeissi, Mathieu Laurière, Álvaro Mateos González, Francisco José Silva

Abstract: We study a numerical approximation of a time-dependent Mean Field Game (MFG) system with local couplings. The discretization we consider stems from a variational approach described in [Briceno-Arias, Kalise, and Silva, SIAM J. Control Optim., 2017] for the stationary problem and leads to the finite difference scheme introduced by Achdou and Capuzzo-Dolcetta in [SIAM J. Numer. Anal., 48(3):1136-116… ▽ More We study a numerical approximation of a time-dependent Mean Field Game (MFG) system with local couplings. The discretization we consider stems from a variational approach described in [Briceno-Arias, Kalise, and Silva, SIAM J. Control Optim., 2017] for the stationary problem and leads to the finite difference scheme introduced by Achdou and Capuzzo-Dolcetta in [SIAM J. Numer. Anal., 48(3):1136-1162, 2010]. In order to solve the finite dimensional variational problems, in [Briceno-Arias, Kalise, and Silva, SIAM J. Control Optim., 2017] the authors implement the primal-dual algorithm introduced by Chambolle and Pock in [J. Math. Imaging Vision, 40(1):120-145, 2011], whose core consists in iteratively solving linear systems and applying a proximity operator. We apply that method to time-dependent MFG and, for large viscosity parameters, we improve the linear system solution by replacing the direct approach used in [Briceno-Arias, Kalise, and Silva, SIAM J. Control Optim., 2017] by suitable preconditioned iterative algorithms. △ Less

Submitted 4 November, 2018; v1 submitted 21 February, 2018; originally announced February 2018.