-
Transfer in Sequential Multi-armed Bandits via Reward Samples
Authors:
Rahul N R,
Vaibhav Katewa
Abstract:
We consider a sequential stochastic multi-armed bandit problem where the agent interacts with bandit over multiple episodes. The reward distribution of the arms remain constant throughout an episode but can change over different episodes. We propose an algorithm based on UCB to transfer the reward samples from the previous episodes and improve the cumulative regret performance over all the episode…
▽ More
We consider a sequential stochastic multi-armed bandit problem where the agent interacts with bandit over multiple episodes. The reward distribution of the arms remain constant throughout an episode but can change over different episodes. We propose an algorithm based on UCB to transfer the reward samples from the previous episodes and improve the cumulative regret performance over all the episodes. We provide regret analysis and empirical results for our algorithm, which show significant improvement over the standard UCB algorithm without transfer.
△ Less
Submitted 19 March, 2024;
originally announced March 2024.
-
Control Barrier Functions in UGVs for Kinematic Obstacle Avoidance: A Collision Cone Approach
Authors:
Phani Thontepu,
Bhavya Giri Goswami,
Manan Tayal,
Neelaksh Singh,
Shyamsundar P I,
Shyam Sundar M G,
Suresh Sundaram,
Vaibhav Katewa,
Shishir Kolathaya
Abstract:
In this paper, we propose a new class of Control Barrier Functions (CBFs) for Unmanned Ground Vehicles (UGVs) that help avoid collisions with kinematic (non-zero velocity) obstacles. While the current forms of CBFs have been successful in guaranteeing safety/collision avoidance with static obstacles, extensions for the dynamic case have seen limited success. Moreover, with the UGV models like the…
▽ More
In this paper, we propose a new class of Control Barrier Functions (CBFs) for Unmanned Ground Vehicles (UGVs) that help avoid collisions with kinematic (non-zero velocity) obstacles. While the current forms of CBFs have been successful in guaranteeing safety/collision avoidance with static obstacles, extensions for the dynamic case have seen limited success. Moreover, with the UGV models like the unicycle or the bicycle, applications of existing CBFs have been conservative in terms of control, i.e., steering/thrust control has not been possible under certain scenarios. Drawing inspiration from the classical use of collision cones for obstacle avoidance in trajectory planning, we introduce its novel CBF formulation with theoretical guarantees on safety for both the unicycle and bicycle models. The main idea is to ensure that the velocity of the obstacle w.r.t. the vehicle is always pointing away from the vehicle. Accordingly, we construct a constraint that ensures that the velocity vector always avoids a cone of vectors pointing at the vehicle. The efficacy of this new control methodology is later verified by Pybullet simulations on TurtleBot3 and F1Tenth.
△ Less
Submitted 16 October, 2023; v1 submitted 23 September, 2022;
originally announced September 2022.
-
On Connections between Opacity and Security in Linear Systems
Authors:
Varkey M. John,
Vaibhav Katewa
Abstract:
Opacity and attack detectability are important properties for any system as they allow the states to remain private and malicious attacks to be detected, respectively. In this paper, we show that a fundamental trade-off exists between these properties for a linear dynamical system, in the sense that if an opaque system is subjected to attacks, all attacks cannot be detected. We first characterize…
▽ More
Opacity and attack detectability are important properties for any system as they allow the states to remain private and malicious attacks to be detected, respectively. In this paper, we show that a fundamental trade-off exists between these properties for a linear dynamical system, in the sense that if an opaque system is subjected to attacks, all attacks cannot be detected. We first characterize the opacity conditions for the system in terms of its weakly unobservable subspace (WUS) and show that the number of opaque states is proportional to the size of the WUS. Further, we establish conditions under which increasing the opaque sets also increases the set of undetectable attacks. This highlights a fundamental trade-off between security and privacy. We demonstrate application of our results on a remotely controlled automotive system.
△ Less
Submitted 13 June, 2022;
originally announced June 2022.
-
Robust Adversarial Classification via Abstaining
Authors:
Abed AlRahman Al Makdah,
Vaibhav Katewa,
Fabio Pasqualetti
Abstract:
In this work, we consider a binary classification problem and cast it into a binary hypothesis testing framework, where the observations can be perturbed by an adversary. To improve the adversarial robustness of a classifier, we include an abstain option, where the classifier abstains from making a decision when it has low confidence about the prediction. We propose metrics to quantify the nominal…
▽ More
In this work, we consider a binary classification problem and cast it into a binary hypothesis testing framework, where the observations can be perturbed by an adversary. To improve the adversarial robustness of a classifier, we include an abstain option, where the classifier abstains from making a decision when it has low confidence about the prediction. We propose metrics to quantify the nominal performance of a classifier with an abstain option and its robustness against adversarial perturbations. We show that there exist a tradeoff between the two metrics regardless of what method is used to choose the abstain region. Our results imply that the robustness of a classifier with an abstain option can only be improved at the expense of its nominal performance. Further, we provide necessary conditions to design the abstain region for a 1- dimensional binary classification problem. We validate our theoretical results on the MNIST dataset, where we numerically show that the tradeoff between performance and robustness also exist for the general multi-class classification problems.
△ Less
Submitted 30 September, 2021; v1 submitted 6 April, 2021;
originally announced April 2021.
-
On the Robustness of Data-Driven Controllers for Linear Systems
Authors:
Rajasekhar Anguluri,
Abed AlRahman Al Makdah,
Vaibhav Katewa,
Fabio Pasqualetti
Abstract:
This paper proposes a new framework and several results to quantify the performance of data-driven state-feedback controllers for linear systems against targeted perturbations of the training data. We focus on the case where subsets of the training data are randomly corrupted by an adversary, and derive lower and upper bounds for the stability of the closed-loop system with compromised controller…
▽ More
This paper proposes a new framework and several results to quantify the performance of data-driven state-feedback controllers for linear systems against targeted perturbations of the training data. We focus on the case where subsets of the training data are randomly corrupted by an adversary, and derive lower and upper bounds for the stability of the closed-loop system with compromised controller as a function of the perturbation statistics, size of the training data, sensitivity of the data-driven algorithm to perturbation of the training data, and properties of the nominal closed-loop system. Our stability and convergence bounds are probabilistic in nature, and rely on a first-order approximation of the data-driven procedure that designs the state-feedback controller, which can be computed directly using the training data. We illustrate our findings via multiple numerical studies.
△ Less
Submitted 21 December, 2019;
originally announced December 2019.
-
A Fundamental Performance Limitation for Adversarial Classification
Authors:
Abed AlRahman Al Makdah,
Vaibhav Katewa,
Fabio Pasqualetti
Abstract:
Despite the widespread use of machine learning algorithms to solve problems of technological, economic, and social relevance, provable guarantees on the performance of these data-driven algorithms are critically lacking, especially when the data originates from unreliable sources and is transmitted over unprotected and easily accessible channels. In this paper we take an important step to bridge t…
▽ More
Despite the widespread use of machine learning algorithms to solve problems of technological, economic, and social relevance, provable guarantees on the performance of these data-driven algorithms are critically lacking, especially when the data originates from unreliable sources and is transmitted over unprotected and easily accessible channels. In this paper we take an important step to bridge this gap and formally show that, in a quest to optimize their accuracy, binary classification algorithms -- including those based on machine-learning techniques -- inevitably become more sensitive to adversarial manipulation of the data. Further, for a given class of algorithms with the same complexity (i.e., number of classification boundaries), the fundamental tradeoff curve between accuracy and sensitivity depends solely on the statistics of the data, and cannot be improved by tuning the algorithm.
△ Less
Submitted 14 March, 2019; v1 submitted 3 March, 2019;
originally announced March 2019.
