-
Asymptotic Nash Equilibria of Finite-State Ergodic Markovian Mean Field Games
Authors:
Asaf Cohen,
Ethan Zell
Abstract:
Mean field games (MFGs) model equilibria in games with a continuum of weakly interacting players as limiting systems of symmetric $n$-player games. We consider the finite-state, infinite-horizon problem with ergodic cost. Assuming Markovian strategies, we first prove that any solution to the MFG system gives rise to a $(C/\sqrt{n})$-Nash equilibrium in the $n$-player game. We follow this result by…
▽ More
Mean field games (MFGs) model equilibria in games with a continuum of weakly interacting players as limiting systems of symmetric $n$-player games. We consider the finite-state, infinite-horizon problem with ergodic cost. Assuming Markovian strategies, we first prove that any solution to the MFG system gives rise to a $(C/\sqrt{n})$-Nash equilibrium in the $n$-player game. We follow this result by proving the same is true for the strategy profile derived from the master equation. We conclude the main theoretical portion of the paper by establishing a large deviation principle for empirical measures associated with the asymptotic Nash equilibria. Then, we contrast the asymptotic Nash equilibria using an example. We solve the MFG system directly and numerically solve the ergodic master equation by adapting the deep Galerkin method of Sirignano and Spiliopoulos. We use these results to derive the strategies of the asymptotic Nash equilibria and compare them. Finally, we derive an explicit form for the rate functions in dimension two.
△ Less
Submitted 17 April, 2024;
originally announced April 2024.
-
Deep Backward and Galerkin Methods for the Finite State Master Equation
Authors:
Asaf Cohen,
Mathieu Laurière,
Ethan Zell
Abstract:
This paper proposes and analyzes two neural network methods to solve the master equation for finite-state mean field games (MFGs). Solving MFGs provides approximate Nash equilibria for stochastic, differential games with finite but large populations of agents. The master equation is a partial differential equation (PDE) whose solution characterizes MFG equilibria for any possible initial distribut…
▽ More
This paper proposes and analyzes two neural network methods to solve the master equation for finite-state mean field games (MFGs). Solving MFGs provides approximate Nash equilibria for stochastic, differential games with finite but large populations of agents. The master equation is a partial differential equation (PDE) whose solution characterizes MFG equilibria for any possible initial distribution. The first method we propose relies on backward induction in a time component while the second method directly tackles the PDE without discretizing time. For both approaches, we prove two types of results: there exist neural networks that make the algorithms' loss functions arbitrarily small, and conversely, if the losses are small, then the neural networks are good approximations of the master equation's solution. We conclude the paper with numerical experiments on benchmark problems from the literature up to dimension 15, and a comparison with solutions computed by a classical method for fixed initial distributions.
△ Less
Submitted 7 March, 2024;
originally announced March 2024.
-
Image-Coupled Volume Propagation for Stereo Matching
Authors:
Oh-Hun Kwon,
Eduard Zell
Abstract:
Several leading methods on public benchmarks for depth-from-stereo rely on memory-demanding 4D cost volumes and computationally intensive 3D convolutions for feature matching. We suggest a new way to process the 4D cost volume where we merge two different concepts in one deeply integrated framework to achieve a symbiotic relationship. A feature matching part is responsible for identifying matching…
▽ More
Several leading methods on public benchmarks for depth-from-stereo rely on memory-demanding 4D cost volumes and computationally intensive 3D convolutions for feature matching. We suggest a new way to process the 4D cost volume where we merge two different concepts in one deeply integrated framework to achieve a symbiotic relationship. A feature matching part is responsible for identifying matching pixels pairs along the baseline while a concurrent image volume part is inspired by depth-from-mono CNNs. However, instead of predicting depth directly from image features, it provides additional context to resolve ambiguities during pixel matching. More technically, the processing of the 4D cost volume is separated into a 2D propagation and a 3D propagation part. Starting from feature maps of the left image, the 2D propagation assists the 3D propagation part of the cost volume at different layers by adding visual features to the geometric context. By combining both parts, we can safely reduce the scale of 3D convolution layers in the matching part without sacrificing accuracy. Experiments demonstrate that our end-to-end trained CNN is ranked 2nd on KITTI2012 and ETH3D benchmarks while being significantly faster than the 1st-ranked method. Furthermore, we notice that the coupling of image and matching-volume improves fine-scale details as demonstrated by our qualitative analysis.
△ Less
Submitted 30 December, 2022;
originally announced January 2023.
-
Analysis of the Finite-State Ergodic Master Equation
Authors:
Asaf Cohen,
Ethan Zell
Abstract:
Mean field games model equilibria in games with a continuum of players as limiting systems of symmetric $n$-player games with weak interaction between the players. We consider a finite-state, infinite-horizon problem with two cost criteria: discounted and ergodic. Under the Lasry--Lions monotonicity condition we characterize the stationary ergodic mean field game equilibrium by a mean field game s…
▽ More
Mean field games model equilibria in games with a continuum of players as limiting systems of symmetric $n$-player games with weak interaction between the players. We consider a finite-state, infinite-horizon problem with two cost criteria: discounted and ergodic. Under the Lasry--Lions monotonicity condition we characterize the stationary ergodic mean field game equilibrium by a mean field game system of two coupled equations: one for the value and the other for the stationary measure. This system is linked with the ergodic master equation. Several discounted mean field game systems are utilized in order to set up the relevant discounted master equations. We show that the discounted master equations are smooth, uniformly in the discount factor. Taking the discount factor to zero, we achieve the smoothness of the ergodic master equation.
△ Less
Submitted 16 November, 2022; v1 submitted 11 April, 2022;
originally announced April 2022.
-
Survey of Self-Play in Reinforcement Learning
Authors:
Anthony DiGiovanni,
Ethan C. Zell
Abstract:
In reinforcement learning (RL), the term self-play describes a kind of multi-agent learning (MAL) that deploys an algorithm against copies of itself to test compatibility in various stochastic environments. As is typical in MAL, the literature draws heavily from well-established concepts in classical game theory and so this survey quickly reviews some fundamental concepts. In what follows, we pres…
▽ More
In reinforcement learning (RL), the term self-play describes a kind of multi-agent learning (MAL) that deploys an algorithm against copies of itself to test compatibility in various stochastic environments. As is typical in MAL, the literature draws heavily from well-established concepts in classical game theory and so this survey quickly reviews some fundamental concepts. In what follows, we present a brief survey of self-play literature, its major themes, criteria, and techniques, and then conclude with an assessment of current shortfalls of the literature as well as suggestions for future directions.
△ Less
Submitted 6 July, 2021;
originally announced July 2021.
-
Limiting speed of a second class particle in ASEP
Authors:
Promit Ghosal,
Axel Saenz,
Ethan C. Zell
Abstract:
We study the asymptotic speed of a second class particle in the two-species asymmetric simple exclusion process (ASEP) on $\mathbb{Z}$ with each particle belonging either to the first class or the second class. For any fixed non-negative integer $L$, we consider the two-species ASEP started from the initial data with all the sites of $\mathbb{Z}_{<-L}$ occupied by first class particles, all the si…
▽ More
We study the asymptotic speed of a second class particle in the two-species asymmetric simple exclusion process (ASEP) on $\mathbb{Z}$ with each particle belonging either to the first class or the second class. For any fixed non-negative integer $L$, we consider the two-species ASEP started from the initial data with all the sites of $\mathbb{Z}_{<-L}$ occupied by first class particles, all the sites of $\mathbb{Z}_{[-L,0]}$ occupied by second class particles, and the rest of the sites of $\mathbb{Z}$ unoccupied. With these initial conditions, we show that the speed of the leftmost second class particle converges weakly to a distribution supported on a symmetric compact interval $Γ\subset \mathbb{R}$. Furthermore, the limiting distribution is shown to have the same law as the minimum of $L+1$ independent random samples drawn uniformly from the interval $Γ$.
△ Less
Submitted 25 March, 2019; v1 submitted 22 March, 2019;
originally announced March 2019.
-
Some tables of right set properties in affine Weyl groups of type A
Authors:
Leonard L. Scott,
Ethan C. Zell
Abstract:
The tables of this title are a first attempt to understand empirically the sizes of certain distinguished sets, introduced by Hankyung Ko, of elements in affine Weyl groups. The sizes are relevant to the computational efficiency of direct approaches to computing characters of modular representations of algebraic groups from characters of corresponding irreducible representations of quantum groups.
The tables of this title are a first attempt to understand empirically the sizes of certain distinguished sets, introduced by Hankyung Ko, of elements in affine Weyl groups. The sizes are relevant to the computational efficiency of direct approaches to computing characters of modular representations of algebraic groups from characters of corresponding irreducible representations of quantum groups.
△ Less
Submitted 7 June, 2018;
originally announced June 2018.
