-
How does a Rational Agent Act in an Epidemic?
Authors:
S. Yagiz Olmez,
Shubham Aggarwal,
** Won Kim,
Erik Miehling,
Tamer Başar,
Matthew West,
Prashant G. Mehta
Abstract:
Evolution of disease in a large population is a function of the top-down policy measures from a centralized planner, as well as the self-interested decisions (to be socially active) of individual agents in a large heterogeneous population. This paper is concerned with understanding the latter based on a mean-field type optimal control model. Specifically, the model is used to investigate the role…
▽ More
Evolution of disease in a large population is a function of the top-down policy measures from a centralized planner, as well as the self-interested decisions (to be socially active) of individual agents in a large heterogeneous population. This paper is concerned with understanding the latter based on a mean-field type optimal control model. Specifically, the model is used to investigate the role of partial information on an agent's decision-making, and study the impact of such decisions by a large number of agents on the spread of the virus in the population. The motivation comes from the presymptomatic and asymptomatic spread of the COVID-19 virus where an agent unwittingly spreads the virus. We show that even in a setting with fully rational agents, limited information on the viral state can result in an epidemic growth.
△ Less
Submitted 5 June, 2022;
originally announced June 2022.
-
Modeling Presymptomatic Spread in Epidemics via Mean-Field Games
Authors:
S. Yagiz Olmez,
Shubham Aggarwal,
** Won Kim,
Erik Miehling,
Tamer Başar,
Matthew West,
Prashant G. Mehta
Abstract:
This paper is concerned with develo** mean-field game models for the evolution of epidemics. Specifically, an agent's decision -- to be socially active in the midst of an epidemic -- is modeled as a mean-field game with health-related costs and activity-related rewards. By considering the fully and partially observed versions of this problem, the role of information in guiding an agent's rationa…
▽ More
This paper is concerned with develo** mean-field game models for the evolution of epidemics. Specifically, an agent's decision -- to be socially active in the midst of an epidemic -- is modeled as a mean-field game with health-related costs and activity-related rewards. By considering the fully and partially observed versions of this problem, the role of information in guiding an agent's rational decision is highlighted. The main contributions of the paper are to derive the equations for the mean-field game in both fully and partially observed settings of the problem, to present a complete analysis of the fully observed case, and to present some analytical results for the partially observed case.
△ Less
Submitted 19 November, 2021;
originally announced November 2021.
-
Discrete-Time Linear-Quadratic Regulation via Optimal Transport
Authors:
Mathias Hudoba de Badyn,
Erik Miehling,
Dylan Janak,
Behçet Açıkmeşe,
Mehran Mesbahi,
Tamer Başar,
John Lygeros,
Roy S. Smith
Abstract:
In this paper, we consider a discrete-time stochastic control problem with uncertain initial and target states. We first discuss the connection between optimal transport and stochastic control problems of this form. Next, we formulate a linear-quadratic regulator problem where the initial and terminal states are distributed according to specified probability densities. A closed-form solution for t…
▽ More
In this paper, we consider a discrete-time stochastic control problem with uncertain initial and target states. We first discuss the connection between optimal transport and stochastic control problems of this form. Next, we formulate a linear-quadratic regulator problem where the initial and terminal states are distributed according to specified probability densities. A closed-form solution for the optimal transport map in the case of linear-time varying systems is derived, along with an algorithm for computing the optimal map. Two numerical examples pertaining to swarm deployment demonstrate the practical applicability of the model, and performance of the numerical method.
△ Less
Submitted 6 September, 2021;
originally announced September 2021.
-
Strategic Inference with a Single Private Sample
Authors:
Erik Miehling,
Roy Dong,
Cédric Langbort,
Tamer Başar
Abstract:
Motivated by applications in cyber security, we develop a simple game model for describing how a learning agent's private information influences an observing agent's inference process. The model describes a situation in which one of the agents (attacker) is deciding which of two targets to attack, one with a known reward and another with uncertain reward. The attacker receives a single private sam…
▽ More
Motivated by applications in cyber security, we develop a simple game model for describing how a learning agent's private information influences an observing agent's inference process. The model describes a situation in which one of the agents (attacker) is deciding which of two targets to attack, one with a known reward and another with uncertain reward. The attacker receives a single private sample from the uncertain target's distribution and updates its belief of the target quality. The other agent (defender) knows the true rewards, but does not see the sample that the attacker has received. This leads to agents possessing asymmetric information: the attacker is uncertain over the parameter of the distribution, whereas the defender is uncertain about the observed sample. After the attacker updates its belief, both the attacker and the defender play a simultaneous move game based on their respective beliefs. We offer a characterization of the pure strategy equilibria of the game and explain how the players' decisions are influenced by their prior knowledge and the payoffs/costs.
△ Less
Submitted 13 September, 2019;
originally announced September 2019.
-
Online Planning for Decentralized Stochastic Control with Partial History Sharing
Authors:
Kaiqing Zhang,
Erik Miehling,
Tamer Başar
Abstract:
In decentralized stochastic control, standard approaches for sequential decision-making, e.g. dynamic programming, quickly become intractable due to the need to maintain a complex information state. Computational challenges are further compounded if agents do not possess complete model knowledge. In this paper, we take advantage of the fact that in many problems agents share some common informatio…
▽ More
In decentralized stochastic control, standard approaches for sequential decision-making, e.g. dynamic programming, quickly become intractable due to the need to maintain a complex information state. Computational challenges are further compounded if agents do not possess complete model knowledge. In this paper, we take advantage of the fact that in many problems agents share some common information, or history, termed partial history sharing. Under this information structure the policy search space is greatly reduced. We propose a provably convergent, online tree-search based algorithm that does not require a closed-form model or explicit communication among agents. Interestingly, our algorithm can be viewed as a generalization of several existing heuristic solvers for decentralized partially observable Markov decision processes. To demonstrate the applicability of the model, we propose a novel collaborative intrusion response model, where multiple agents (defenders) possessing asymmetric information aim to collaboratively defend a computer network. Numerical results demonstrate the performance of our algorithm.
△ Less
Submitted 6 August, 2019;
originally announced August 2019.
-
A Decentralized Mechanism for Computing Competitive Equilibria in Deregulated Electricity Markets
Authors:
Erik Miehling,
Demosthenis Teneketzis
Abstract:
With the increased level of distributed generation and demand response comes the need for associated mechanisms that can perform well in the face of increasingly complex deregulated energy market structures. Using Lagrangian duality theory, we develop a decentralized market mechanism that ensures that, under the guidance of a market operator, self-interested market participants: generation compani…
▽ More
With the increased level of distributed generation and demand response comes the need for associated mechanisms that can perform well in the face of increasingly complex deregulated energy market structures. Using Lagrangian duality theory, we develop a decentralized market mechanism that ensures that, under the guidance of a market operator, self-interested market participants: generation companies (GenCos), distribution companies (DistCos), and transmission companies (TransCos), reach a competitive equilibrium. We show that even in the presence of informational asymmetries and nonlinearities (such as power losses and transmission constraints), the resulting competitive equilibrium is Pareto efficient.
△ Less
Submitted 23 March, 2016; v1 submitted 9 March, 2016;
originally announced March 2016.
-
Sequential Detection with Mutual Information Stop** Cost
Authors:
Vikram Krishnamurthy,
Robert Bitmead,
Michel Gevers,
Erik Miehling
Abstract:
This paper formulates and solves a sequential detection problem that involves the mutual information (stochastic observability) of a Gaussian process observed in noise with missing measurements. The main result is that the optimal decision is characterized by a monotone policy on the partially ordered set of positive definite covariance matrices. This monotone structure implies that numerically ef…
▽ More
This paper formulates and solves a sequential detection problem that involves the mutual information (stochastic observability) of a Gaussian process observed in noise with missing measurements. The main result is that the optimal decision is characterized by a monotone policy on the partially ordered set of positive definite covariance matrices. This monotone structure implies that numerically efficient algorithms can be designed to estimate and implement monotone parametrized decision policies.The sequential detection problem is motivated by applications in radar scheduling where the aim is to maintain the mutual information of all targets within a specified bound. We illustrate the problem formulation and performance of monotone parametrized policies via numerical examples in fly-by and persistent-surveillance applications involving a GMTI (Ground Moving Target Indicator) radar.
△ Less
Submitted 19 October, 2011;
originally announced October 2011.