-
Infinite Horizon Average Cost Optimality Criteria for Mean-Field Control
Authors:
Erhan Bayraktar,
Ali D. Kara
Abstract:
We study mean-field control problems in discrete-time under the infinite horizon average cost optimality criteria. We focus on both the finite population and the infinite population setups. We show the existence of a solution to the average cost optimality equation (ACOE) and the existence of optimal stationary Markov policies for finite population problems under (i) a minorization condition that…
▽ More
We study mean-field control problems in discrete-time under the infinite horizon average cost optimality criteria. We focus on both the finite population and the infinite population setups. We show the existence of a solution to the average cost optimality equation (ACOE) and the existence of optimal stationary Markov policies for finite population problems under (i) a minorization condition that provides geometric ergodicity on the collective state process of the agents, and (ii) under standard Lipschitz continuity assumptions on the stage-wise cost and transition function of the agents when the Lipschitz constant of the transition function satisfies a certain bound. For the infinite population problem, we establish the existence of a solution to the ACOE, and the existence of optimal policies under the continuity assumptions on the cost and the transition functions. Finally, we relate the finite population and infinite population control problems: (i) we prove that the optimal value of the finite population problem converges to the optimal value of the infinite population problem as the number of agents grows to infinity; (ii) we show that the accumulation points of the finite population optimal solution corresponds to an optimal solution for the infinite population problem, and finally (iii), we show that one can use the solution of the infinite population problem for the finite population problem symmetrically across the agents to achieve near optimal performance when the population is sufficiently large.
△ Less
Submitted 17 April, 2024; v1 submitted 20 September, 2023;
originally announced September 2023.
-
Analysis of feature detector and descriptor combinations with a localization experiment for various performance metrics
Authors:
Ertugrul Bayraktar,
Pinar Boyraz
Abstract:
The purpose of this study is to provide a detailed performance comparison of feature detector/descriptor methods, particularly when their various combinations are used for image-matching. The localization experiments of a mobile robot in an indoor environment are presented as a case study. In these experiments, 3090 query images and 127 dataset images were used. This study includes five methods fo…
▽ More
The purpose of this study is to provide a detailed performance comparison of feature detector/descriptor methods, particularly when their various combinations are used for image-matching. The localization experiments of a mobile robot in an indoor environment are presented as a case study. In these experiments, 3090 query images and 127 dataset images were used. This study includes five methods for feature detectors (features from accelerated segment test (FAST), oriented FAST and rotated binary robust independent elementary features (BRIEF) (ORB), speeded-up robust features (SURF), scale invariant feature transform (SIFT), and binary robust invariant scalable keypoints (BRISK)) and five other methods for feature descriptors (BRIEF, BRISK, SIFT, SURF, and ORB). These methods were used in 23 different combinations and it was possible to obtain meaningful and consistent comparison results using the performance criteria defined in this study. All of these methods were used independently and separately from each other as either feature detector or descriptor. The performance analysis shows the discriminative power of various combinations of detector and descriptor methods. The analysis is completed using five parameters: (i) accuracy, (ii) time, (iii) angle difference between keypoints, (iv) number of correct matches, and (v) distance between correctly matched keypoints. In a range of 60°, covering five rotational pose points for our system, the FAST-SURF combination had the lowest distance and angle difference values and the highest number of matched keypoints. SIFT-SURF was the most accurate combination with a 98.41% correct classification rate. The fastest algorithm was ORB-BRIEF, with a total running time of 21,303.30 s to match 560 images captured during motion with 127 dataset images.
△ Less
Submitted 17 October, 2017;
originally announced October 2017.
-
Byzantine Fault Tolerant Distributed Quickest Change Detection
Authors:
Erhan Bayraktar,
Lifeng Lai
Abstract:
We introduce and solve the problem of Byzantine fault tolerant distributed quickest change detection in both continuous and discrete time setups. In this problem, multiple sensors sequentially observe random signals from the environment and send their observations to a control center that will determine whether there is a change in the statistical behavior of the observations. We assume that the s…
▽ More
We introduce and solve the problem of Byzantine fault tolerant distributed quickest change detection in both continuous and discrete time setups. In this problem, multiple sensors sequentially observe random signals from the environment and send their observations to a control center that will determine whether there is a change in the statistical behavior of the observations. We assume that the signals are independent and identically distributed across sensors. An unknown subset of sensors are compromised and will send arbitrarily modified and even artificially generated signals to the control center. It is shown that the performance of the the so-called CUSUM statistic, which is optimal when all sensors are honest, will be significantly degraded in the presence of even a single dishonest sensor. In particular, instead of in a logarithmically the detection delay grows linearly with the average run length (ARL) to false alarm. To mitigate such a performance degradation, we propose a fully distributed low complexity detection scheme. We show that the proposed scheme can recover the log scaling. We also propose a centralized group-wise scheme that can further reduce the detection delay.
△ Less
Submitted 29 December, 2014; v1 submitted 9 June, 2013;
originally announced June 2013.
-
On the Robust Optimal Stop** Problem
Authors:
Erhan Bayraktar,
Song Yao
Abstract:
We study a robust optimal stop** problem with respect to a set $\cP$ of mutually singular probabilities. This can be interpreted as a zero-sum controller-stopper game in which the stopper is trying to maximize its pay-off while an adverse player wants to minimize this payoff by choosing an evaluation criteria from $\cP$. We show that the \emph{upper Snell envelope $\ol{Z}$} of the reward process…
▽ More
We study a robust optimal stop** problem with respect to a set $\cP$ of mutually singular probabilities. This can be interpreted as a zero-sum controller-stopper game in which the stopper is trying to maximize its pay-off while an adverse player wants to minimize this payoff by choosing an evaluation criteria from $\cP$. We show that the \emph{upper Snell envelope $\ol{Z}$} of the reward process $Y$ is a supermartingale with respect to an appropriately defined nonlinear expectation $\ul{\sE}$, and $\ol{Z}$ is further an $\ul{\sE}-$martingale up to the first time $\t^*$ when $\ol{Z}$ meets $Y$. Consequently, $\t^*$ is the optimal stop** time for the robust optimal stop** problem and the corresponding zero-sum game has a value. Although the result seems similar to the one obtained in the classical optimal stop** theory, the mutual singularity of probabilities and the game aspect of the problem give rise to major technical hurdles, which we circumvent using some new methods.
△ Less
Submitted 11 April, 2016; v1 submitted 1 January, 2013;
originally announced January 2013.
-
Quickest Detection with Discretely Controlled Observations
Authors:
Erhan Bayraktar,
Ross Kravitz
Abstract:
We study a continuous time Bayesian quickest detection problem in which observation times are a scarce resource. The agent, limited to making a finite number of discrete observations, must adaptively decide his observation strategy to minimize detection delay and the probability of false alarm. Under two different models of observation rights, we establish the existence of optimal strategies, and…
▽ More
We study a continuous time Bayesian quickest detection problem in which observation times are a scarce resource. The agent, limited to making a finite number of discrete observations, must adaptively decide his observation strategy to minimize detection delay and the probability of false alarm. Under two different models of observation rights, we establish the existence of optimal strategies, and formulate an algorithmic approach to the problem via jump operators. We describe algorithms for these problems, and illustrate them with some numerical results. As the number of observation rights tends to infinity, we also show convergence to the classical continuous observation problem of Shiryaev.
△ Less
Submitted 2 December, 2014; v1 submitted 19 December, 2012;
originally announced December 2012.
-
Stochastic Perron's method for Hamilton-Jacobi-Bellman equations
Authors:
Erhan Bayraktar,
Mihai Sirbu
Abstract:
We show that the value function of a stochastic control problem is the unique solution of the associated Hamilton-Jacobi-Bellman (HJB) equation, completely avoiding the proof of the so-called dynamic programming principle (DPP). Using Stochastic Perron's method we construct a super-solution lying below the value function and a sub-solution dominating it. A comparison argument easily closes the pro…
▽ More
We show that the value function of a stochastic control problem is the unique solution of the associated Hamilton-Jacobi-Bellman (HJB) equation, completely avoiding the proof of the so-called dynamic programming principle (DPP). Using Stochastic Perron's method we construct a super-solution lying below the value function and a sub-solution dominating it. A comparison argument easily closes the proof. The program has the precise meaning of verification for viscosity-solutions, obtaining the DPP as a conclusion. It also immediately follows that the weak and strong formulations of the stochastic control problem have the same value. Using this method we also capture the possible face-lifting phenomenon in a straightforward manner.
△ Less
Submitted 24 September, 2013; v1 submitted 10 December, 2012;
originally announced December 2012.
-
Liquidation in Limit Order Books with Controlled Intensity
Authors:
Erhan Bayraktar,
Michael Ludkovski
Abstract:
We consider a framework for solving optimal liquidation problems in limit order books. In particular, order arrivals are modeled as a point process whose intensity depends on the liquidation price. We set up a stochastic control problem in which the goal is to maximize the expected revenue from liquidating the entire position held. We solve this optimal liquidation problem for power-law and expone…
▽ More
We consider a framework for solving optimal liquidation problems in limit order books. In particular, order arrivals are modeled as a point process whose intensity depends on the liquidation price. We set up a stochastic control problem in which the goal is to maximize the expected revenue from liquidating the entire position held. We solve this optimal liquidation problem for power-law and exponential-decay order book models and discuss several extensions. We also consider the continuous selling (or fluid) limit when the trading units are ever smaller and the intensity is ever larger. This limit provides an analytical approximation to the value function and the optimal solution. Using techniques from viscosity solutions we show that the discrete state problem and its optimal solution converge to the corresponding quantities in the continuous selling limit uniformly on compacts.
△ Less
Submitted 26 January, 2012; v1 submitted 2 May, 2011;
originally announced May 2011.
-
Stochatic Perron's method and verification without smoothness using viscosity comparison: the linear case
Authors:
Erhan Bayraktar,
Mihai Sirbu
Abstract:
We introduce a probabilistic version of the classical Perron's method to construct viscosity solutions to linear parabolic equations associated to stochastic differential equations. Using this method, we construct easily two viscosity (sub and super) solutions that squeeze in between the expected payoff. If a comparison result holds true, then there exists a unique viscosity solution which is a ma…
▽ More
We introduce a probabilistic version of the classical Perron's method to construct viscosity solutions to linear parabolic equations associated to stochastic differential equations. Using this method, we construct easily two viscosity (sub and super) solutions that squeeze in between the expected payoff. If a comparison result holds true, then there exists a unique viscosity solution which is a martingale along the solutions of the stochastic differential equation. The unique viscosity solution is actually equal to the expected payoff. This amounts to a verification result (Ito's Lemma) for non-smooth viscosity solutions of the linear parabolic equation. This is the first step in a larger program to prove verification for viscosity solutions and the Dynamic Programming Principle for stochastic control problems and games
△ Less
Submitted 13 July, 2011; v1 submitted 2 March, 2011;
originally announced March 2011.
-
On the Multi-Dimensional Controller and Stopper Games
Authors:
Erhan Bayraktar,
Yu-Jui Huang
Abstract:
We consider a zero-sum stochastic differential controller-and-stopper game in which the state process is a controlled diffusion evolving in a multi-dimensional Euclidean space. In this game, the controller affects both the drift and the volatility terms of the state process. Under appropriate conditions, we show that the game has a value and the value function is the unique viscosity solution to a…
▽ More
We consider a zero-sum stochastic differential controller-and-stopper game in which the state process is a controlled diffusion evolving in a multi-dimensional Euclidean space. In this game, the controller affects both the drift and the volatility terms of the state process. Under appropriate conditions, we show that the game has a value and the value function is the unique viscosity solution to an obstacle problem for a Hamilton-Jacobi-Bellman equation.
△ Less
Submitted 13 January, 2013; v1 submitted 5 September, 2010;
originally announced September 2010.
-
Minimizing the Probability of Lifetime Ruin under Stochastic Volatility
Authors:
Erhan Bayraktar,
Xueying Hu,
Virginia R. Young
Abstract:
We assume that an individual invests in a financial market with one riskless and one risky asset, with the latter's price following a diffusion with stochastic volatility. In the current financial market especially, it is important to include stochastic volatility in the risky asset's price process. Given the rate of consumption, we find the optimal investment strategy for the individual who wishe…
▽ More
We assume that an individual invests in a financial market with one riskless and one risky asset, with the latter's price following a diffusion with stochastic volatility. In the current financial market especially, it is important to include stochastic volatility in the risky asset's price process. Given the rate of consumption, we find the optimal investment strategy for the individual who wishes to minimize the probability of going bankrupt. To solve this minimization problem, we use techniques from stochastic optimal control.
△ Less
Submitted 5 May, 2011; v1 submitted 18 March, 2010;
originally announced March 2010.
