-
Adaptive Incentive Design with Learning Agents
Authors:
Chinmay Maheshwari,
Kshitij Kulkarni,
Manxi Wu,
Shankar Sastry
Abstract:
How can the system operator learn an incentive mechanism that achieves social optimality based on limited information about the agents' behavior, who are dynamically updating their strategies? To answer this question, we propose an \emph{adaptive} incentive mechanism. This mechanism updates the incentives of agents based on the feedback of each agent's externality, evaluated as the difference betw…
▽ More
How can the system operator learn an incentive mechanism that achieves social optimality based on limited information about the agents' behavior, who are dynamically updating their strategies? To answer this question, we propose an \emph{adaptive} incentive mechanism. This mechanism updates the incentives of agents based on the feedback of each agent's externality, evaluated as the difference between the player's marginal cost and society's marginal cost at each time step. The proposed mechanism updates the incentives on a slower timescale compared to the agents' learning dynamics, resulting in a two-timescale coupled dynamical system. Notably, this mechanism is agnostic to the specific learning dynamics used by agents to update their strategies. We show that any fixed point of this adaptive incentive mechanism corresponds to the optimal incentive mechanism, ensuring that the Nash equilibrium coincides with the socially optimal strategy. Additionally, we provide sufficient conditions that guarantee the convergence of the adaptive incentive mechanism to a fixed point. Our results apply to both atomic and non-atomic games. To demonstrate the effectiveness of our proposed mechanism, we verify the convergence conditions in two practically relevant games: atomic networked quadratic aggregative games and non-atomic network routing games.
△ Less
Submitted 26 May, 2024;
originally announced May 2024.
-
Understanding the Impact of Coalitions between EV Charging Stations
Authors:
Sukanya Kudva,
Kshitij Kulkarni,
Chinmay Maheshwari,
Anil Aswani,
Shankar Sastry
Abstract:
The rapid growth of electric vehicles (EVs) is driving the expansion of charging infrastructure globally. This expansion, however, places significant charging demand on the electricity grid, impacting grid operations and electricity pricing. While coordination among all charging stations is beneficial, it may not be always feasible. However, a subset of charging stations, which could be jointly op…
▽ More
The rapid growth of electric vehicles (EVs) is driving the expansion of charging infrastructure globally. This expansion, however, places significant charging demand on the electricity grid, impacting grid operations and electricity pricing. While coordination among all charging stations is beneficial, it may not be always feasible. However, a subset of charging stations, which could be jointly operated by a company, could coordinate to decide their charging profile. In this paper we investigate whether such coalitions between charging stations is better than no coordination.
We model EV charging as a non-cooperative aggregative game, where each station's cost is determined by both monetary payments tied to reactive electricity prices on the grid and its sensitivity to deviations from a nominal charging profile. We consider a solution concept that we call $\mathcal{C}$-Nash equilibrium, which is tied to a coalition $\mathcal{C}$ of charging stations coordinating to reduce their cumulative costs. We provide sufficient conditions, in terms of the demand and sensitivity of charging stations, to determine when independent (uncoordinated) operation of charging stations could result in lower overall costs to charging stations, the coalition, and charging stations outside the coalition. Somewhat counter to intuition, we demonstrate scenarios where allowing charging stations to operate independently is better than coordinating as a coalition. Jointly, these results provide operators of charging stations insights into how to coordinate their charging behavior, and open several research directions.
△ Less
Submitted 5 April, 2024;
originally announced April 2024.
-
Incentive-Compatible Vertiport Reservation in Advanced Air Mobility: An Auction-Based Approach
Authors:
Pan-Yang Su,
Chinmay Maheshwari,
Victoria Tuck,
Shankar Sastry
Abstract:
The rise of advanced air mobility (AAM) is expected to become a multibillion-dollar industry in the near future. Market-based mechanisms are touted to be an integral part of AAM operations, which comprise heterogeneous operators with private valuations. In this work, we study the problem of designing a mechanism to coordinate the movement of electric vertical take-off and landing (eVTOL) aircraft,…
▽ More
The rise of advanced air mobility (AAM) is expected to become a multibillion-dollar industry in the near future. Market-based mechanisms are touted to be an integral part of AAM operations, which comprise heterogeneous operators with private valuations. In this work, we study the problem of designing a mechanism to coordinate the movement of electric vertical take-off and landing (eVTOL) aircraft, operated by multiple operators each having heterogeneous valuations associated with their fleet, between vertiports, while enforcing the arrival, departure, and parking constraints at vertiports. Particularly, we propose an incentive-compatible and individually rational vertiport reservation mechanism that maximizes a social welfare metric, which encapsulates the objective of maximizing the overall valuations of all operators while minimizing the congestion at vertiports. Additionally, we improve the computational tractability of designing the reservation mechanism by proposing a mixed binary linear programming approach that is based on constructing network flow graph corresponding to the underlying problem.
△ Less
Submitted 26 March, 2024;
originally announced March 2024.
-
Congestion Pricing for Efficiency and Equity: Theory and Applications to the San Francisco Bay Area
Authors:
Chinmay Maheshwari,
Kshitij Kulkarni,
Druv Pai,
Jiarui Yang,
Manxi Wu,
Shankar Sastry
Abstract:
Congestion pricing, while adopted by many cities to alleviate traffic congestion, raises concerns about widening socioeconomic disparities due to its disproportionate impact on low-income travelers. In this study, we address this concern by proposing a new class of congestion pricing schemes that not only minimize congestion levels but also incorporate an equity objective to reduce cost disparitie…
▽ More
Congestion pricing, while adopted by many cities to alleviate traffic congestion, raises concerns about widening socioeconomic disparities due to its disproportionate impact on low-income travelers. In this study, we address this concern by proposing a new class of congestion pricing schemes that not only minimize congestion levels but also incorporate an equity objective to reduce cost disparities among travelers with different willingness-to-pay. Our analysis builds on a congestion game model with heterogeneous traveler populations. We present four pricing schemes that account for practical considerations, such as the ability to charge differentiated tolls to various traveler populations and the option to toll all or only a subset of edges in the network. We evaluate our pricing schemes in the calibrated freeway network of the San Francisco Bay Area. We demonstrate that the proposed congestion pricing schemes improve both efficiency (in terms of reduced average travel time) and equity (the disparities of travel costs experienced by different populations) compared to the current pricing scheme. Moreover, our pricing schemes also generate a total revenue comparable to the current pricing scheme. Our results further show that pricing schemes charging differentiated prices to traveler populations with varying willingness-to-pay lead to a more equitable distribution of travel costs compared to those that charge a homogeneous price to all.
△ Less
Submitted 30 January, 2024;
originally announced January 2024.
-
Music Recommendation on Spotify using Deep Learning
Authors:
Chhavi Maheshwari
Abstract:
Hosting about 50 million songs and 4 billion playlists, there is an enormous amount of data generated at Spotify every single day - upwards of 600 gigabytes of data (harvard.edu). Since the algorithms that Spotify uses in recommendation systems is proprietary and confidential, code for big data analytics and recommendation can only be speculated. However, it is widely theorized that Spotify uses t…
▽ More
Hosting about 50 million songs and 4 billion playlists, there is an enormous amount of data generated at Spotify every single day - upwards of 600 gigabytes of data (harvard.edu). Since the algorithms that Spotify uses in recommendation systems is proprietary and confidential, code for big data analytics and recommendation can only be speculated. However, it is widely theorized that Spotify uses two main strategies to target users' playlists and personalized mixes that are infamous for their retention - exploration and exploitation (kaggle.com). This paper aims to appropriate filtering using the approach of deep learning for maximum user likeability. The architecture achieves 98.57% and 80% training and validation accuracy respectively.
△ Less
Submitted 10 December, 2023;
originally announced December 2023.
-
Markov $α$-Potential Games
Authors:
Xin Guo,
Xinyu Li,
Chinmay Maheshwari,
Shankar Sastry,
Manxi Wu
Abstract:
This paper proposes a new framework of Markov $α$-potential games to study Markov games. In this new framework, Markov games are shown to be Markov $α$-potential games, and the existence of an associated $α$-potential function is established. Any optimizer of an $α$-potential function is shown to be an $α$-stationary NE. Two important classes of practically significant Markov games, Markov congest…
▽ More
This paper proposes a new framework of Markov $α$-potential games to study Markov games. In this new framework, Markov games are shown to be Markov $α$-potential games, and the existence of an associated $α$-potential function is established. Any optimizer of an $α$-potential function is shown to be an $α$-stationary NE. Two important classes of practically significant Markov games, Markov congestion games and the perturbed Markov team games, are studied via this framework of Markov $α$-potential games, with explicit characterization of an upper bound for $α$ and its relation to game parameters. Additionally, a semi-infinite linear programming based formulation is presented to obtain an upper bound for $α$ for any Markov game. Furthermore, two equilibrium approximation algorithms, namely the projected gradient-ascent algorithm and the sequential maximum improvement algorithm, are presented along with their Nash regret analysis, and corroborated by numerical experiments.
△ Less
Submitted 9 March, 2024; v1 submitted 21 May, 2023;
originally announced May 2023.
-
Synthetic DOmain-Targeted Augmentation (S-DOTA) Improves Model Generalization in Digital Pathology
Authors:
Sai Chowdary Gullapally,
Yibo Zhang,
Nitin Kumar Mittal,
Deeksha Kartik,
Sandhya Srinivasan,
Kevin Rose,
Daniel Shenker,
Dinkar Juyal,
Harshith Padigela,
Raymond Biju,
Victor Minden,
Chirag Maheshwari,
Marc Thibault,
Zvi Goldstein,
Luke Novak,
Nidhi Chandra,
Justin Lee,
Aaditya Prakash,
Chintan Shah,
John Abel,
Darren Fahy,
Amaro Taylor-Weiner,
Anand Sampat
Abstract:
Machine learning algorithms have the potential to improve patient outcomes in digital pathology. However, generalization of these tools is currently limited by sensitivity to variations in tissue preparation, staining procedures and scanning equipment that lead to domain shift in digitized slides. To overcome this limitation and improve model generalization, we studied the effectiveness of two Syn…
▽ More
Machine learning algorithms have the potential to improve patient outcomes in digital pathology. However, generalization of these tools is currently limited by sensitivity to variations in tissue preparation, staining procedures and scanning equipment that lead to domain shift in digitized slides. To overcome this limitation and improve model generalization, we studied the effectiveness of two Synthetic DOmain-Targeted Augmentation (S-DOTA) methods, namely CycleGAN-enabled Scanner Transform (ST) and targeted Stain Vector Augmentation (SVA), and compared them against the International Color Consortium (ICC) profile-based color calibration (ICC Cal) method and a baseline method using traditional brightness, color and noise augmentations. We evaluated the ability of these techniques to improve model generalization to various tasks and settings: four models, two model types (tissue segmentation and cell classification), two loss functions, six labs, six scanners, and three indications (hepatocellular carcinoma (HCC), nonalcoholic steatohepatitis (NASH), prostate adenocarcinoma). We compared these methods based on the macro-averaged F1 scores on in-distribution (ID) and out-of-distribution (OOD) test sets across multiple domains, and found that S-DOTA methods (i.e., ST and SVA) led to significant improvements over ICC Cal and baseline on OOD data while maintaining comparable performance on ID data. Thus, we demonstrate that S-DOTA may help address generalization due to domain shift in real world applications.
△ Less
Submitted 3 May, 2023;
originally announced May 2023.
-
Follower Agnostic Methods for Stackelberg Games
Authors:
Chinmay Maheshwari,
James Cheng,
S. Shankar Sasty,
Lillian Ratliff,
Eric Mazumdar
Abstract:
In this paper, we present an efficient algorithm to solve online Stackelberg games, featuring multiple followers, in a follower-agnostic manner. Unlike previous works, our approach works even when leader has no knowledge about the followers' utility functions or strategy space. Our algorithm introduces a unique gradient estimator, leveraging specially designed strategies to probe followers. In a d…
▽ More
In this paper, we present an efficient algorithm to solve online Stackelberg games, featuring multiple followers, in a follower-agnostic manner. Unlike previous works, our approach works even when leader has no knowledge about the followers' utility functions or strategy space. Our algorithm introduces a unique gradient estimator, leveraging specially designed strategies to probe followers. In a departure from traditional assumptions of optimal play, we model followers' responses using a convergent adaptation rule, allowing for realistic and dynamic interactions. The leader constructs the gradient estimator solely based on observations of followers' actions. We provide both non-asymptotic convergence rates to stationary points of the leader's objective and demonstrate asymptotic convergence to a \emph{local Stackelberg equilibrium}. To validate the effectiveness of our algorithm, we use this algorithm to solve the problem of incentive design on a large-scale transportation network, showcasing its robustness even when the leader lacks access to followers' demand.
△ Less
Submitted 26 March, 2024; v1 submitted 2 February, 2023;
originally announced February 2023.
-
Competing Bandits in Time Varying Matching Markets
Authors:
Deepan Muthirayan,
Chinmay Maheshwari,
Pramod P. Khargonekar,
Shankar Sastry
Abstract:
We study the problem of online learning in two-sided non-stationary matching markets, where the objective is to converge to a stable match. In particular, we consider the setting where one side of the market, the arms, has fixed known set of preferences over the other side, the players. While this problem has been studied when the players have fixed but unknown preferences, in this work we study t…
▽ More
We study the problem of online learning in two-sided non-stationary matching markets, where the objective is to converge to a stable match. In particular, we consider the setting where one side of the market, the arms, has fixed known set of preferences over the other side, the players. While this problem has been studied when the players have fixed but unknown preferences, in this work we study the problem of how to learn when the preferences of the players are time varying and unknown. Our contribution is a methodology that can handle any type of preference structure and variation scenario. We show that, with the proposed algorithm, each player receives a uniform sub-linear regret of {$\widetilde{\mathcal{O}}(L^{1/2}_TT^{1/2})$} up to the number of changes in the underlying preferences of the agents, $L_T$. Therefore, we show that the optimal rates for single-agent learning can be achieved in spite of the competition up to a difference of a constant factor. We also discuss extensions of this algorithm to the case where the number of changes need not be known a priori.
△ Less
Submitted 12 January, 2023; v1 submitted 20 October, 2022;
originally announced October 2022.
-
Decentralized, Communication- and Coordination-free Learning in Structured Matching Markets
Authors:
Chinmay Maheshwari,
Eric Mazumdar,
Shankar Sastry
Abstract:
We study the problem of online learning in competitive settings in the context of two-sided matching markets. In particular, one side of the market, the agents, must learn about their preferences over the other side, the firms, through repeated interaction while competing with other agents for successful matches. We propose a class of decentralized, communication- and coordination-free algorithms…
▽ More
We study the problem of online learning in competitive settings in the context of two-sided matching markets. In particular, one side of the market, the agents, must learn about their preferences over the other side, the firms, through repeated interaction while competing with other agents for successful matches. We propose a class of decentralized, communication- and coordination-free algorithms that agents can use to reach to their stable match in structured matching markets. In contrast to prior works, the proposed algorithms make decisions based solely on an agent's own history of play and requires no foreknowledge of the firms' preferences. Our algorithms are constructed by splitting up the statistical problem of learning one's preferences, from noisy observations, from the problem of competing for firms. We show that under realistic structural assumptions on the underlying preferences of the agents and firms, the proposed algorithms incur a regret which grows at most logarithmically in the time horizon. Our results show that, in the case of matching markets, competition need not drastically affect the performance of decentralized, communication and coordination free online learning algorithms.
△ Less
Submitted 6 June, 2022;
originally announced June 2022.
-
Independent and Decentralized Learning in Markov Potential Games
Authors:
Chinmay Maheshwari,
Manxi Wu,
Druv Pai,
Shankar Sastry
Abstract:
We propose a multi-agent reinforcement learning dynamics, and analyze its convergence in infinite-horizon discounted Markov potential games. We focus on the independent and decentralized setting, where players do not have knowledge of the game model and cannot coordinate. In each stage, players update their estimate of Q-function that evaluates their total contingent payoff based on the realized o…
▽ More
We propose a multi-agent reinforcement learning dynamics, and analyze its convergence in infinite-horizon discounted Markov potential games. We focus on the independent and decentralized setting, where players do not have knowledge of the game model and cannot coordinate. In each stage, players update their estimate of Q-function that evaluates their total contingent payoff based on the realized one-stage reward in an asynchronous manner. Then, players independently update their policies by incorporating an optimal one-stage deviation strategy based on the estimated Q-function. A key feature of the learning dynamics is that the Q-function estimates are updated at a faster timescale than the policies. We prove that the policies induced by our learning dynamics converge to the set of stationary Nash equilibria in Markov potential games with probability 1. Our results highlight the efficacy of simple learning dynamics in reaching to the set of stationary Nash equilibrium even in environments with minimal information available.
△ Less
Submitted 10 November, 2023; v1 submitted 29 May, 2022;
originally announced May 2022.
-
Inducing Social Optimality in Games via Adaptive Incentive Design
Authors:
Chinmay Maheshwari,
Kshitij Kulkarni,
Manxi Wu,
Shankar Sastry
Abstract:
How can a social planner adaptively incentivize selfish agents who are learning in a strategic environment to induce a socially optimal outcome in the long run? We propose a two-timescale learning dynamics to answer this question in both atomic and non-atomic games. In our learning dynamics, players adopt a class of learning rules to update their strategies at a faster timescale, while a social pl…
▽ More
How can a social planner adaptively incentivize selfish agents who are learning in a strategic environment to induce a socially optimal outcome in the long run? We propose a two-timescale learning dynamics to answer this question in both atomic and non-atomic games. In our learning dynamics, players adopt a class of learning rules to update their strategies at a faster timescale, while a social planner updates the incentive mechanism at a slower timescale. In particular, the update of the incentive mechanism is based on each player's externality, which is evaluated as the difference between the player's marginal cost and the society's marginal cost in each time step. We show that any fixed point of our learning dynamics corresponds to the optimal incentive mechanism such that the corresponding Nash equilibrium also achieves social optimality. We also provide sufficient conditions for the learning dynamics to converge to a fixed point so that the adaptive incentive mechanism eventually induces a socially optimal outcome. Finally, we demonstrate that the sufficient conditions for convergence are satisfied in a variety of games, including (i) atomic networked quadratic aggregative games, (ii) atomic Cournot competition, and (iii) non-atomic network routing games.
△ Less
Submitted 11 April, 2022;
originally announced April 2022.
-
Dynamic Tolling for Inducing Socially Optimal Traffic Loads
Authors:
Chinmay Maheshwari,
Kshitij Kulkarni,
Manxi Wu,
Shankar Sastry
Abstract:
How to design tolls that induce socially optimal traffic loads with dynamically arriving travelers who make selfish routing decisions? We propose a two-timescale discrete-time stochastic dynamics that adaptively adjusts the toll prices on a parallel link network while accounting for the updates of traffic loads induced by the incoming and outgoing travelers and their route choices. The updates of…
▽ More
How to design tolls that induce socially optimal traffic loads with dynamically arriving travelers who make selfish routing decisions? We propose a two-timescale discrete-time stochastic dynamics that adaptively adjusts the toll prices on a parallel link network while accounting for the updates of traffic loads induced by the incoming and outgoing travelers and their route choices. The updates of loads and tolls in our dynamics have three key features: (i) The total demand of incoming and outgoing travelers is stochastically realized; (ii) Travelers are myopic and selfish in that they choose routes according to a perturbed best response given the current latency and tolls on parallel links; (iii) The update of tolls is at a slower timescale as compared to the the update of loads. We show that the loads and the tolls eventually concentrate in a neighborhood of the fixed point, which corresponds to the socially optimal load and toll price. Moreover, the fixed point load is also a stochastic user equilibrium with respect to the toll price. Our results are useful for traffic authorities to efficiently manage traffic loads in response to the arrival and departure of travelers.
△ Less
Submitted 17 October, 2021;
originally announced October 2021.
-
Toward Interlanguage Parallel Scripting for Distributed-Memory Scientific Computing
Authors:
Justin M. Wozniak,
Timothy G. Armstrong,
Ketan C. Maheshwari,
Daniel S. Katz,
Michael Wilde,
Ian T. Foster
Abstract:
Scripting languages such as Python and R have been widely adopted as tools for the productive development of scientific software because of the power and expressiveness of the languages and available libraries. However, deploying scripted applications on large-scale parallel computer systems such as the IBM Blue Gene/Q or Cray XE6 is a challenge because of issues including operating system limitat…
▽ More
Scripting languages such as Python and R have been widely adopted as tools for the productive development of scientific software because of the power and expressiveness of the languages and available libraries. However, deploying scripted applications on large-scale parallel computer systems such as the IBM Blue Gene/Q or Cray XE6 is a challenge because of issues including operating system limitations, interoperability challenges, parallel filesystem overheads due to the small file system accesses common in scripted approaches, and other issues. We present here a new approach to these problems in which the Swift scripting system is used to integrate high-level scripts written in Python, R, and Tcl, with native code developed in C, C++, and Fortran, by linking Swift to the library interfaces to the script interpreters. In this approach, Swift handles data management, movement, and marshaling among distributed-memory processes without direct user manipulation of low-level communication libraries such as MPI. We present a technique to efficiently launch scripted applications on large-scale supercomputers using a hierarchical programming model.
△ Less
Submitted 6 July, 2021;
originally announced July 2021.
-
Zeroth-Order Methods for Convex-Concave Minmax Problems: Applications to Decision-Dependent Risk Minimization
Authors:
Chinmay Maheshwari,
Chih-Yuan Chiu,
Eric Mazumdar,
S. Shankar Sastry,
Lillian J. Ratliff
Abstract:
Min-max optimization is emerging as a key framework for analyzing problems of robustness to strategically and adversarially generated data. We propose a random reshuffling-based gradient free Optimistic Gradient Descent-Ascent algorithm for solving convex-concave min-max problems with finite sum structure.
We prove that the algorithm enjoys the same convergence rate as that of zeroth-order algor…
▽ More
Min-max optimization is emerging as a key framework for analyzing problems of robustness to strategically and adversarially generated data. We propose a random reshuffling-based gradient free Optimistic Gradient Descent-Ascent algorithm for solving convex-concave min-max problems with finite sum structure.
We prove that the algorithm enjoys the same convergence rate as that of zeroth-order algorithms for convex minimization problems. We further specialize the algorithm to solve distributionally robust, decision-dependent learning problems, where gradient information is not readily available. Through illustrative simulations, we observe that our proposed approach learns models that are simultaneously robust against adversarial distribution shifts and strategic decisions from the data sources, and outperforms existing methods from the strategic classification literature.
△ Less
Submitted 19 February, 2022; v1 submitted 16 June, 2021;
originally announced June 2021.
-
Machine Learning Enabled Computational Screening of Inorganic Solid Electrolytes for Dendrite Suppression with Li Metal Anode
Authors:
Zeeshan Ahmad,
Tian Xie,
Chinmay Maheshwari,
Jeffrey C. Grossman,
Venkatasubramanian Viswanathan
Abstract:
Next generation batteries based on lithium (Li) metal anodes have been plagued by the dendritic electrodeposition of Li metal on the anode during cycling, resulting in short circuit and capacity loss. Suppression of dendritic growth through the use of solid electrolytes has emerged as one of the most promising strategies for enabling the use of Li metal anodes. We perform a computational screening…
▽ More
Next generation batteries based on lithium (Li) metal anodes have been plagued by the dendritic electrodeposition of Li metal on the anode during cycling, resulting in short circuit and capacity loss. Suppression of dendritic growth through the use of solid electrolytes has emerged as one of the most promising strategies for enabling the use of Li metal anodes. We perform a computational screening of over 12,000 inorganic solids based on their ability to suppress dendrite initiation in contact with Li metal anode. Properties for mechanically isotropic and anisotropic interfaces that can be used in stability criteria for determining the propensity of dendrite initiation are usually obtained from computationally expensive first-principles methods. In order to obtain a large dataset for screening, we use machine learning models to predict the mechanical properties of several new solid electrolytes. We train a convolutional neural network on the shear and bulk moduli purely on structural features of the material. We use AdaBoost, Lasso and Bayesian ridge regression to train the elastic constants, where the choice of the model depended on the size of the training data and the noise that it can handle. Our models give us direct interpretability by revealing the dominant structural features affecting the elastic constants. The stiffness is found to increase with a decrease in volume per atom, increase in minimum anion-anion separation, and increase in sublattice (all but Li) packing fraction. Cross-validation/test performance suggests our models generalize well. We predict over 20 mechanically anisotropic interfaces between Li metal and 6 solid electrolytes which can be used to suppress dendrite growth. Our screened candidates are generally soft and highly anisotropic, and present opportunities for simultaneously obtaining dendrite suppression and high ionic conductivity in solid electrolytes.
△ Less
Submitted 12 April, 2018;
originally announced April 2018.
