-
Collaborative Decision-Making and the k-Strong Price of Anarchy in Common Interest Games
Authors:
Bryce L. Ferguson,
Dario Paccagnan,
Bary S. R. Pradelski,
Jason R. Marden
Abstract:
The control of large-scale, multi-agent systems often entails distributing decision-making across the system components. However, with advances in communication and computation technologies, we can consider new collaborative decision-making paradigms that exist somewhere between centralized and distributed control. In this work, we seek to understand the benefits and costs of increased collaborati…
▽ More
The control of large-scale, multi-agent systems often entails distributing decision-making across the system components. However, with advances in communication and computation technologies, we can consider new collaborative decision-making paradigms that exist somewhere between centralized and distributed control. In this work, we seek to understand the benefits and costs of increased collaborative communication in multi-agent systems. We specifically study this in the context of common interest games in which groups of up to k agents can coordinate their actions in maximizing the common objective function. The equilibria that emerge in these systems are the k-strong Nash equilibria of the common interest game; studying the properties of these states can provide relevant insights into the efficacy of inter-agent collaboration. Our contributions come threefold: 1) provide bounds on how well k-strong Nash equilibria approximate the optimal system welfare, formalized by the k-strong price of anarchy, 2) study the run-time and transient performance of collaborative agent-based dynamics, and 3) consider the task of redesigning objectives for groups of agents which improve system performance. We study these three facets generally as well as in the context of resource allocation problems, in which we provide tractable linear programs that give tight bounds on the k-strong price of anarchy.
△ Less
Submitted 2 July, 2024; v1 submitted 2 November, 2023;
originally announced November 2023.
-
Collaborative Coalitions in Multi-Agent Systems: Quantifying the Strong Price of Anarchy for Resource Allocation Games
Authors:
Bryce L. Ferguson,
Dario Paccagnan,
Bary S. R. Pradelski,
Jason R. Marden
Abstract:
The emergence of new communication technologies allows us to expand our understanding of distributed control and consider collaborative decision-making paradigms. With collaborative algorithms, certain local decision-making entities (or agents) are enabled to communicate and collaborate on their actions with one another to attain better system behavior. By limiting the amount of communication, the…
▽ More
The emergence of new communication technologies allows us to expand our understanding of distributed control and consider collaborative decision-making paradigms. With collaborative algorithms, certain local decision-making entities (or agents) are enabled to communicate and collaborate on their actions with one another to attain better system behavior. By limiting the amount of communication, these algorithms exist somewhere between centralized and fully distributed approaches. To understand the possible benefits of this inter-agent collaboration, we model a multi-agent system as a common-interest game in which groups of agents can collaborate on their actions to jointly increase the system welfare. We specifically consider $k$-strong Nash equilibria as the emergent behavior of these systems and address how well these states approximate the system optimal, formalized by the $k$-strong price of anarchy ratio. Our main contributions are in generating tight bounds on the $k$-strong price of anarchy in finite resource allocation games as the solution to a tractable linear program. By varying $k$ --the maximum size of a collaborative coalition--we observe exactly how much performance is gained from inter-agent collaboration. To investigate further opportunities for improvement, we generate upper bounds on the maximum attainable $k$-strong price of anarchy when the agents' utility function can be designed.
△ Less
Submitted 15 August, 2023;
originally announced August 2023.
-
The Cost of Informing Decision-Makers in Multi-Agent Maximum Coverage Problems with Random Resource Values
Authors:
Bryce L. Ferguson,
Dario Paccagnan,
Jason R. Marden
Abstract:
The emergent behavior of a distributed system is conditioned by the information available to the local decision-makers. Therefore, one may expect that providing decision-makers with more information will improve system performance; in this work, we find that this is not necessarily the case. In multi-agent maximum coverage problems, we find that even when agents' objectives are aligned with the gl…
▽ More
The emergent behavior of a distributed system is conditioned by the information available to the local decision-makers. Therefore, one may expect that providing decision-makers with more information will improve system performance; in this work, we find that this is not necessarily the case. In multi-agent maximum coverage problems, we find that even when agents' objectives are aligned with the global welfare, informing agents about the realization of the resource's random values can reduce equilibrium performance by a factor of 1/2. This affirms an important aspect of designing distributed systems: information need be shared carefully. We further this understanding by providing lower and upper bounds on the ratio of system welfare when information is (fully or partially) revealed and when it is not, termed the value-of-informing. We then identify a trade-off that emerges when optimizing the performance of the best-case and worst-case equilibrium.
△ Less
Submitted 21 June, 2023;
originally announced June 2023.
-
Markov Games with Decoupled Dynamics: Price of Anarchy and Sample Complexity
Authors:
Runyu Zhang,
Yuyang Zhang,
Rohit Konda,
Bryce Ferguson,
Jason Marden,
Na Li
Abstract:
This paper studies the finite-time horizon Markov games where the agents' dynamics are decoupled but the rewards can possibly be coupled across agents. The policy class is restricted to local policies where agents make decisions using their local state. We first introduce the notion of smooth Markov games which extends the smoothness argument for normal form games to our setting, and leverage the…
▽ More
This paper studies the finite-time horizon Markov games where the agents' dynamics are decoupled but the rewards can possibly be coupled across agents. The policy class is restricted to local policies where agents make decisions using their local state. We first introduce the notion of smooth Markov games which extends the smoothness argument for normal form games to our setting, and leverage the smoothness property to bound the price of anarchy of the Markov game. For a specific type of Markov game called the Markov potential game, we also develop a distributed learning algorithm, multi-agent soft policy iteration (MA-SPI), which provably converges to a Nash equilibrium. Sample complexity of the algorithm is also provided. Lastly, our results are validated using a dynamic covering game.
△ Less
Submitted 7 April, 2023;
originally announced April 2023.
-
Avoiding Unintended Consequences: How Incentives Aid Information Provisioning in Bayesian Congestion Games
Authors:
Bryce L. Ferguson,
Philip N. Brown,
Jason R. Marden
Abstract:
When users lack specific knowledge of various system parameters, their uncertainty may lead them to make undesirable deviations in their decision making. To alleviate this, an informed system operator may elect to signal information to uninformed users with the hope of persuading them to take more preferable actions. In this work, we study public and truthful signalling mechanisms in the context o…
▽ More
When users lack specific knowledge of various system parameters, their uncertainty may lead them to make undesirable deviations in their decision making. To alleviate this, an informed system operator may elect to signal information to uninformed users with the hope of persuading them to take more preferable actions. In this work, we study public and truthful signalling mechanisms in the context of Bayesian congestion games on parallel networks. We provide bounds on the possible benefit a signalling policy can provide with and without the concurrent use of monetary incentives. We find that though revealing information can reduce system cost in some settings, it can also be detrimental and cause worse performance than not signalling at all. However, by utilizing both signalling and incentive mechanisms, the system operator can guarantee that revealing information does not worsen performance while offering similar opportunities for improvement. These findings emerge from the closed form bounds we derive on the benefit a signalling policy can provide. We provide a numerical example which illustrates the phenomenon that revealing more information can degrade performance when incentives are not used and improves performance when incentives are used.
△ Less
Submitted 30 March, 2023; v1 submitted 12 April, 2022;
originally announced April 2022.
-
Path Defense in Dynamic Defender-Attacker Blotto Games (dDAB) with Limited Information
Authors:
Austin K. Chen,
Bryce L. Ferguson,
Daigo Shishika,
Michael Dorothy,
Jason R. Marden,
George J. Pappas,
Vijay Kumar
Abstract:
We consider a path guarding problem in dynamic Defender-Attacker Blotto games (dDAB), where a team of robots must defend a path in a graph against adversarial agents. Multi-robot systems are particularly well suited to this application, as recent work has shown the effectiveness of these systems in related areas such as perimeter defense and surveillance. When designing a defender policy that guar…
▽ More
We consider a path guarding problem in dynamic Defender-Attacker Blotto games (dDAB), where a team of robots must defend a path in a graph against adversarial agents. Multi-robot systems are particularly well suited to this application, as recent work has shown the effectiveness of these systems in related areas such as perimeter defense and surveillance. When designing a defender policy that guarantees the defense of a path, information about the adversary and the environment can be helpful and may reduce the number of resources required by the defender to achieve a sufficient level of security. In this work, we characterize the necessary and sufficient number of assets needed to guarantee the defense of a shortest path between two nodes in dDAB games when the defender can only detect assets within $k$-hops of a shortest path. By characterizing the relationship between sensing horizon and required resources, we show that increasing the sensing capability of the defender greatly reduces the number of defender assets needed to defend the path.
△ Less
Submitted 25 May, 2023; v1 submitted 8 April, 2022;
originally announced April 2022.
-
The Effectiveness of Subsidies and Tolls in Congestion Games
Authors:
Bryce L. Ferguson,
Philip N. Brown,
Jason R. Marden
Abstract:
Are rewards or penalties more effective in influencing user behavior? This work compares the effectiveness of subsidies and tolls in incentivizing user behavior in congestion games. The predominantly studied method of influencing user behavior in network routing problems is to institute taxes which alter users' observed costs in a manner that causes their self-interested choices to more closely al…
▽ More
Are rewards or penalties more effective in influencing user behavior? This work compares the effectiveness of subsidies and tolls in incentivizing user behavior in congestion games. The predominantly studied method of influencing user behavior in network routing problems is to institute taxes which alter users' observed costs in a manner that causes their self-interested choices to more closely align with a system-level objective. Another conceivable method to accomplish the same goal is to subsidize the users' actions that are preferable from a system-level perspective. We show that, when users behave similarly and predictably, subsidies offer superior performance guarantees to tolls under similar budgetary constraints; however, in the presence of unknown player heterogeneity, subsidies fail to offer the same robustness as tolls.
△ Less
Submitted 18 February, 2021;
originally announced February 2021.
-
Optimal Taxes in Atomic Congestion Games
Authors:
Dario Paccagnan,
Rahul Chandan,
Bryce L Ferguson,
Jason R Marden
Abstract:
How can we design mechanisms to promote efficient use of shared resources? Here, we answer this question in relation to the well-studied class of atomic congestion games, used to model a variety of problems, including traffic routing. Within this context, a methodology for designing tolling mechanisms that minimize the system inefficiency (price of anarchy) exploiting solely local information is s…
▽ More
How can we design mechanisms to promote efficient use of shared resources? Here, we answer this question in relation to the well-studied class of atomic congestion games, used to model a variety of problems, including traffic routing. Within this context, a methodology for designing tolling mechanisms that minimize the system inefficiency (price of anarchy) exploiting solely local information is so far missing in spite of the scientific interest. In this manuscript we resolve this problem through a tractable linear programming formulation that applies to and beyond polynomial congestion games. When specializing our approach to the polynomial case, we obtain tight values for the optimal price of anarchy and corresponding tolls, uncovering an unexpected link with load balancing games. We also derive optimal tolling mechanisms that are constant with the congestion level, generalizing the results of Caragiannis et al. [ACM Transactions on Algorithms, 2010] to polynomial congestion games and beyond. Finally, we apply our techniques to compute the efficiency of the marginal cost mechanism. Surprisingly, optimal tolling mechanism using only local information perform closely to existing mechanism that utilize global information [Bilò and Vinci, ACM Transactions on Economics and Computation, 2019], while the marginal cost mechanism, known to be optimal in the continuous-flow model, has lower efficiency than that encountered levying no toll. All results are tight for pure Nash equilibria, and extend to coarse correlated equilibria.
△ Less
Submitted 11 March, 2021; v1 submitted 21 November, 2019;
originally announced November 2019.
-
Carrots or Sticks? The Effectiveness of Subsidies and Tolls in Congestion Games
Authors:
Bryce L. Ferguson,
Philip N. Brown,
Jason R. Marden
Abstract:
Are rewards or penalties more effective in influencing user behavior? This work compares the effectiveness of subsidies and tolls in incentivizing users in congestion games. The predominantly studied method of influencing user behavior in network routing problems is to institute taxes which alter users' observed costs in a manner that causes their self-interested choices to more closely align with…
▽ More
Are rewards or penalties more effective in influencing user behavior? This work compares the effectiveness of subsidies and tolls in incentivizing users in congestion games. The predominantly studied method of influencing user behavior in network routing problems is to institute taxes which alter users' observed costs in a manner that causes their self-interested choices to more closely align with a system-level objective. Another feasible method to accomplish the same goal is to subsidize the users' actions that are preferable from a system-level perspective. We show that, when users behave similarly and predictably, subsidies offer comparable performance guarantees to tolls while requiring smaller monetary transactions with users; however, in the presence of unknown player heterogeneity, subsidies fail to offer the same performance as tolls. We further investigate these relationships in affine congestion games, deriving explicit performance bounds under optimal tolls and subsidies with and without user heterogeneity; we show that the differences in performance can be significant.
△ Less
Submitted 5 October, 2019;
originally announced October 2019.
-
Utilizing Information Optimally to Influence Distributed Network Routing
Authors:
Bryce L. Ferguson,
Philip N. Brown,
Jason R. Marden
Abstract:
How can a system designer exploit system-level knowledge to derive incentives to optimally influence social behavior? The literature on network routing contains many results studying the application of monetary tolls to influence behavior and improve the efficiency of self-interested network traffic routing. These results typically fall into two categories: (1) optimal tolls which incentivize soci…
▽ More
How can a system designer exploit system-level knowledge to derive incentives to optimally influence social behavior? The literature on network routing contains many results studying the application of monetary tolls to influence behavior and improve the efficiency of self-interested network traffic routing. These results typically fall into two categories: (1) optimal tolls which incentivize socially-optimal behavior for a known realization of the network and population, or (2) robust tolls which provably reduce congestion given uncertainty regarding networks and user types, but may fail to optimize routing in general. This paper advances the study of robust influencing, mechanisms asking how a system designer can optimally exploit additional information regarding the network structure and user price sensitivities to design pricing mechanisms which influence behavior. We design optimal scaled marginal-cost pricing mechanisms for a class of parallel-network routing games and derive the tight performance guarantees when the network structure and/or the average user price-sensitivity is known. Our results demonstrate that from the standpoint of the system operator, in general it is more important to know the structure of the network than it is to know distributional information regarding the user population.
△ Less
Submitted 23 July, 2019;
originally announced July 2019.
-
An Online Pricing Mechanism for Electric Vehicle Parking Assignment and Charge Scheduling
Authors:
Nathaniel Tucker,
Bryce Ferguson,
Mahnoosh Alizadeh
Abstract:
In this paper, we design a pricing framework for online electric vehicle (EV) parking assignment and charge scheduling. Here, users with electric vehicles want to park and charge at electric-vehicle-supply-equipment (EVSEs) at different locations and arrive/depart throughout the day. The goal is to assign and schedule users to the available EVSEs while maximizing user utility and minimizing operat…
▽ More
In this paper, we design a pricing framework for online electric vehicle (EV) parking assignment and charge scheduling. Here, users with electric vehicles want to park and charge at electric-vehicle-supply-equipment (EVSEs) at different locations and arrive/depart throughout the day. The goal is to assign and schedule users to the available EVSEs while maximizing user utility and minimizing operational costs. Our formulation can accommodate multiple locations, limited resources, operational costs, as well as variable arrival patterns. With this formulation, the parking facility management can optimize for behind-the-meter solar integration and reduce costs due to procuring electricity from the grid. We use an online pricing mechanism to approximate the EVSE reservation problem's solution and we analyze the performance compared to the offline solution. Our numerical simulation validates the performance of the EVSE reservation system in a downtown area with multiple parking locations equipped with EVSEs.
△ Less
Submitted 15 May, 2019;
originally announced May 2019.
-
Set-valued dynamic treatment regimes for competing outcomes
Authors:
Eric B. Laber,
Daniel J. Lizotte,
Bradley Ferguson
Abstract:
Dynamic treatment regimes operationalize the clinical decision process as a sequence of functions, one for each clinical decision, where each function takes as input up-to-date patient information and gives as output a single recommended treatment. Current methods for estimating optimal dynamic treatment regimes, for example Q-learning, require the specification of a single outcome by which the `g…
▽ More
Dynamic treatment regimes operationalize the clinical decision process as a sequence of functions, one for each clinical decision, where each function takes as input up-to-date patient information and gives as output a single recommended treatment. Current methods for estimating optimal dynamic treatment regimes, for example Q-learning, require the specification of a single outcome by which the `goodness' of competing dynamic treatment regimes are measured. However, this is an over-simplification of the goal of clinical decision making, which aims to balance several potentially competing outcomes. For example, often a balance must be struck between treatment effectiveness and side-effect burden. We propose a method for constructing dynamic treatment regimes that accommodates competing outcomes by recommending sets of treatments at each decision point. Formally, we construct a sequence of set-valued functions that take as input up-to-date patient information and give as output a recommended subset of the possible treatments. For a given patient history, the recommended set of treatments contains all treatments that are not inferior according to any of the competing outcomes. When there is more than one decision point, constructing these set-valued functions requires solving a non-trivial enumeration problem. We offer an exact enumeration algorithm by recasting the problem as a linear mixed integer program. The proposed methods are illustrated using data from a depression study and the CATIE schizophrenia study.
△ Less
Submitted 7 August, 2012; v1 submitted 12 July, 2012;
originally announced July 2012.
