Showing 1–14 of 14 results for author: Roy, B

OffRAMPS: An FPGA-based Intermediary for Analysis and Modification of Additive Manufacturing Control Systems

Authors: Jason Blocklove, Md Raz, Prithwish Basu Roy, Hammond Pearce, Prashanth Krishnamurthy, Farshad Khorrami, Ramesh Karri

Abstract: Cybersecurity threats in Additive Manufacturing (AM) are an increasing concern as AM adoption continues to grow. AM is now being used for parts in the aerospace, transportation, and medical domains. Threat vectors which allow for part compromise are particularly concerning, as any failure in these domains would have life-threatening consequences. A major challenge to investigation of AM part-compr… ▽ More Cybersecurity threats in Additive Manufacturing (AM) are an increasing concern as AM adoption continues to grow. AM is now being used for parts in the aerospace, transportation, and medical domains. Threat vectors which allow for part compromise are particularly concerning, as any failure in these domains would have life-threatening consequences. A major challenge to investigation of AM part-compromises comes from the difficulty in evaluating and benchmarking both identified threat vectors as well as methods for detecting adversarial actions. In this work, we introduce a generalized platform for systematic analysis of attacks against and defenses for 3D printers. Our "OFFRAMPS" platform is based on the open-source 3D printer control board "RAMPS." OFFRAMPS allows analysis, recording, and modification of all control signals and I/O for a 3D printer. We show the efficacy of OFFRAMPS by presenting a series of case studies based on several Trojans, including ones identified in the literature, and show that OFFRAMPS can both emulate and detect these attacks, i.e., it can both change and detect arbitrary changes to the g-code print commands. △ Less

Submitted 23 April, 2024; originally announced April 2024.

Geometric Tracking Control of a Multi-rotor UAV for Partially Known Trajectories

Authors: Yogesh Kumar, S. B. Roy, P. B. Sujit

Abstract: This paper presents a trajectory-tracking controller for multi-rotor unmanned aerial vehicles (UAVs) in scenarios where only the desired position and heading are known without the higher-order derivatives. The proposed solution modifies the state-of-the-art geometric controller, effectively addressing challenges related to the non-existence of the desired attitude and ensuring positive total thrus… ▽ More This paper presents a trajectory-tracking controller for multi-rotor unmanned aerial vehicles (UAVs) in scenarios where only the desired position and heading are known without the higher-order derivatives. The proposed solution modifies the state-of-the-art geometric controller, effectively addressing challenges related to the non-existence of the desired attitude and ensuring positive total thrust input for all time. We tackle the additional challenge of the non-availability of the higher derivatives of the trajectory by introducing novel nonlinear filter structures. We formalize theoretically the effect of these filter structures on the system error dynamics. Subsequently, through a rigorous theoretical analysis, we demonstrate that the proposed controller leads to uniformly ultimately bounded system error dynamics. △ Less

Submitted 13 November, 2023; originally announced November 2023.

On Continuous Full-Order Integral-Terminal Sliding Mode Control with Unknown Apriori Bound on Uncertainty

Authors: Jit Koley, Dinesh Patra, Binoy Krishna Roy

Abstract: This study aims at providing a solution to the problem of designing a continuous and finite-time control for a class of nonlinear systems in the presence of matched uncertainty with an unknown apriori bound. First, we propose a Full-Order Integral-Terminal Sliding Manifold (FOITSM) with a conventional (discontinuous) sliding mode to show that it provides the combined attributes of the nonsingular… ▽ More This study aims at providing a solution to the problem of designing a continuous and finite-time control for a class of nonlinear systems in the presence of matched uncertainty with an unknown apriori bound. First, we propose a Full-Order Integral-Terminal Sliding Manifold (FOITSM) with a conventional (discontinuous) sliding mode to show that it provides the combined attributes of the nonsingular terminal and integral sliding mode algorithms. Secondly, an Adaptive Disturbance Observer (ADO) has been designed to alleviate the effect of the uncertainty acting on the system. On application of the ADO-based Full-Order Integral-Terminal Sliding Mode Control (FOITSMC), the chattering phenomenon in control input has been reduced substantially in the presence of conditionally known matched disturbances. Moreover, the adaptive gains of ADO are updated non-monotonically without over-bounding the acting disturbance, yet sustain the global boundedness of state trajectories within a specific bound. %Finally, an application of the proposed algorithm for attitude stabilization of a rigid spacecraft has been successively shown. △ Less

Submitted 20 March, 2024; v1 submitted 5 April, 2023; originally announced April 2023.

Comments: 14 pages, 9 figures

MRAC with Memory for Switched Linear Systems

Authors: Pritesh Patel, Sayan Basu Roy, Shubhendu Bhasin

Abstract: This work proposes a switched model reference adaptive control (S-MRAC) architecture for a multi-input multi-output (MIMO) switched linear system with memory for enhanced learning. A salient feature of the proposed method that separates it from most previous results is the use of memory that store the estimator states at switching and facilitate parameter learning during both active and inactive p… ▽ More This work proposes a switched model reference adaptive control (S-MRAC) architecture for a multi-input multi-output (MIMO) switched linear system with memory for enhanced learning. A salient feature of the proposed method that separates it from most previous results is the use of memory that store the estimator states at switching and facilitate parameter learning during both active and inactive phases of a subsystem, thereby improving the tracking performance of the overall switched system. Specifically, the learning experience from the previous active duration of a subsystem is retained in the memory and reused when the subsystem is inactive and when the subsystem becomes active again. Parameter convergence is shown based on an intermittent initial excitation (IIE), which is significantly relaxed than the classical persistence of excitation (PE) condition. A common Lyapunov function is considered to ensure closed-loop stability with S-MRAC. Further under IIE, the exponential stability of tracking and parameter estimation error dynamics are guaranteed. △ Less

Submitted 28 January, 2023; originally announced January 2023.

Comments: arXiv admin note: text overlap with arXiv:2204.03338

Composite Adaptive Control for Time-varying Systems with Dual Adaptation

Authors: Raghavv Goel, Sayan Basu Roy

Abstract: This paper proposes a composite adaptive control architecture using dual adaptation scheme for dynamical systems comprising time-varying uncertain parameters. While majority of the adaptive control schemes in literature address the case of constant parameters, recent research has conceptualized improved adaptive control techniques for time-varying systems with rigorous stability proofs. The propos… ▽ More This paper proposes a composite adaptive control architecture using dual adaptation scheme for dynamical systems comprising time-varying uncertain parameters. While majority of the adaptive control schemes in literature address the case of constant parameters, recent research has conceptualized improved adaptive control techniques for time-varying systems with rigorous stability proofs. The proposed work is an effort towards a similar direction, where a novel dual adaptation mechanism is introduced to efficiently tackle the time-varying nature of the parameters. Projection and $σ$-modification algorithms are strategically combined using congelation of variables to claim a global result for the tracking error space. While the classical adaptive systems demand a restrictive condition of persistence of excitation (PE) for accurate parameter estimation, the proposed work relies on a milder condition, called initial excitation (IE) for the same. A rigorous Lyapunov stability analysis is carried out to establish uniformly ultimately bounded (UUB) stability of the closed-loop system. Further it is analytically shown that the proposed work can recover the performance of previously designed IE-based adaptive controller in case of time invariant systems. △ Less

Submitted 3 June, 2022; originally announced June 2022.

Comments: 6 pages

Online Adaptive Identification of Switched Affine Systems Using a Two-Tier Filter Architecture with Memory

Authors: Pritesh Patel, Sayan Basu Roy, Shubhendu Bhasin

Abstract: This work proposes an online adaptive identification method for multi-input multi-output (MIMO) switched affine systems with guaranteed parameter convergence. A family of online parameter estimators is used that is equipped with a dual-layer low pass filter architecture to facilitate parameter learning and identification of each subsystem. The filters capture information about the unknown paramete… ▽ More This work proposes an online adaptive identification method for multi-input multi-output (MIMO) switched affine systems with guaranteed parameter convergence. A family of online parameter estimators is used that is equipped with a dual-layer low pass filter architecture to facilitate parameter learning and identification of each subsystem. The filters capture information about the unknown parameters in the form of a prediction error which is used in the parameter estimation algorithm. A salient feature of the proposed method that distinguishes it from most previous results is the use of a memory bank that stores filter values and promotes parameter learning during both active and inactive phases of a subsystem. Specifically, the learnt experience from the previous active phase of a subsystem is retained in the memory and leveraged for parameter learning in its subsequent active and inactive phases. Further, a new notion of intermittent initial excitation (IIE) is introduced that extends the previously established initial excitation (IE) condition to the switched system framework. IIE is shown to be sufficient to ensure exponential convergence of the switched system parameters. △ Less

Submitted 7 April, 2022; originally announced April 2022.

Carbon-Aware Computing for Datacenters

Authors: Ana Radovanovic, Ross Koningstein, Ian Schneider, Bokan Chen, Alexandre Duarte, Binz Roy, Diyue Xiao, Maya Haridasan, Patrick Hung, Nick Care, Saurav Talukdar, Eric Mullen, Kendal Smith, MariEllen Cottman, Walfredo Cirne

Abstract: The amount of CO$_2$ emitted per kilowatt-hour on an electricity grid varies by time of day and substantially varies by location due to the types of generation. Networked collections of warehouse scale computers, sometimes called Hyperscale Computing, emit more carbon than needed if operated without regard to these variations in carbon intensity. This paper introduces Google's system for Carbon-In… ▽ More The amount of CO$_2$ emitted per kilowatt-hour on an electricity grid varies by time of day and substantially varies by location due to the types of generation. Networked collections of warehouse scale computers, sometimes called Hyperscale Computing, emit more carbon than needed if operated without regard to these variations in carbon intensity. This paper introduces Google's system for Carbon-Intelligent Compute Management, which actively minimizes electricity-based carbon footprint and power infrastructure costs by delaying temporally flexible workloads. The core component of the system is a suite of analytical pipelines used to gather the next day's carbon intensity forecasts, train day-ahead demand prediction models, and use risk-aware optimization to generate the next day's carbon-aware Virtual Capacity Curves (VCCs) for all datacenter clusters across Google's fleet. VCCs impose hourly limits on resources available to temporally flexible workloads while preserving overall daily capacity, enabling all such workloads to complete within a day. Data from operation shows that VCCs effectively limit hourly capacity when the grid's energy supply mix is carbon intensive and delay the execution of temporally flexible workloads to "greener" times. △ Less

Submitted 11 June, 2021; originally announced June 2021.

Safe Exploration for Identifying Linear Systems via Robust Optimization

Authors: Tyler Lu, Martin Zinkevich, Craig Boutilier, Binz Roy, Dale Schuurmans

Abstract: Safely exploring an unknown dynamical system is critical to the deployment of reinforcement learning (RL) in physical systems where failures may have catastrophic consequences. In scenarios where one knows little about the dynamics, diverse transition data covering relevant regions of state-action space is needed to apply either model-based or model-free RL. Motivated by the cooling of Google's da… ▽ More Safely exploring an unknown dynamical system is critical to the deployment of reinforcement learning (RL) in physical systems where failures may have catastrophic consequences. In scenarios where one knows little about the dynamics, diverse transition data covering relevant regions of state-action space is needed to apply either model-based or model-free RL. Motivated by the cooling of Google's data centers, we study how one can safely identify the parameters of a system model with a desired accuracy and confidence level. In particular, we focus on learning an unknown linear system with Gaussian noise assuming only that, initially, a nominal safe action is known. Define safety as satisfying specific linear constraints on the state space (e.g., requirements on process variable) that must hold over the span of an entire trajectory, and given a Probably Approximately Correct (PAC) style bound on the estimation error of model parameters, we show how to compute safe regions of action space by gradually growing a ball around the nominal safe action. One can apply any exploration strategy where actions are chosen from such safe regions. Experiments on a stylized model of data center cooling dynamics show how computing proper safe regions can increase the sample efficiency of safe exploration. △ Less

Submitted 29 November, 2017; originally announced November 2017.

Theory and Application on Adaptive-Robust Control of Euler-Lagrange Systems with Linearly Parametrizable Uncertainty Bound

Authors: Spandan Roy, Sayan Basu Roy, Indra Narayan Kar

Abstract: This work proposes a new adaptive-robust control (ARC) architecture for a class of uncertain Euler-Lagrange (EL) systems where the upper bound of the uncertainty satisfies linear in parameters (LIP) structure. Conventional ARC strategies either require structural knowledge of the system or presume that the overall uncertainties or its time derivative are norm bounded by a constant. Due to unmodell… ▽ More This work proposes a new adaptive-robust control (ARC) architecture for a class of uncertain Euler-Lagrange (EL) systems where the upper bound of the uncertainty satisfies linear in parameters (LIP) structure. Conventional ARC strategies either require structural knowledge of the system or presume that the overall uncertainties or its time derivative are norm bounded by a constant. Due to unmodelled dynamics and modelling imperfection, true structural knowledge of the system is not always available. Further, for the class of systems under consideration, prior assumption regarding the uncertainties (or its time derivative) being upper bounded by a constant, puts a restriction on states beforehand. Conventional ARC laws invite overestimation-underestimation problem of switching gain. Towards this front, Adaptive Switching-gain based Robust Control (ASRC) is proposed which alleviates the overestimation-underestimation problem of switching gain. Moreover, ASRC avoids any presumption of constant upper bound on the overall uncertainties and can negotiate uncertainties regardless of being linear or nonlinear in parameters. Experimental results of ASRC using a wheeled mobile robot notes improved control performance in comparison to adaptive sliding mode control. △ Less

Submitted 4 August, 2017; originally announced August 2017.

Comments: 10 pages, 9 figures

Journal ref: IEEE Transactions on Control Systems Technology, 2018

Deep Exploration via Bootstrapped DQN

Authors: Ian Osband, Charles Blundell, Alexander Pritzel, Benjamin Van Roy

Abstract: Efficient exploration in complex environments remains a major challenge for reinforcement learning. We propose bootstrapped DQN, a simple algorithm that explores in a computationally and statistically efficient manner through use of randomized value functions. Unlike dithering strategies such as epsilon-greedy exploration, bootstrapped DQN carries out temporally-extended (or deep) exploration; thi… ▽ More Efficient exploration in complex environments remains a major challenge for reinforcement learning. We propose bootstrapped DQN, a simple algorithm that explores in a computationally and statistically efficient manner through use of randomized value functions. Unlike dithering strategies such as epsilon-greedy exploration, bootstrapped DQN carries out temporally-extended (or deep) exploration; this can lead to exponentially faster learning. We demonstrate these benefits in complex stochastic MDPs and in the large-scale Arcade Learning Environment. Bootstrapped DQN substantially improves learning times and performance across most Atari games. △ Less

Submitted 4 July, 2016; v1 submitted 15 February, 2016; originally announced February 2016.

Memory-Based Data-Driven MRAC Architecture Ensuring Parameter Convergence

Authors: Sayan Basu Roy, Shubhendu Bhasin, Indra Narayan Kar

Abstract: Convergence of controller parameters in standard model reference adaptive control (MRAC) requires the system states to be persistently exciting (PE), a restrictive condition to be verified online. A recent data-driven approach, concurrent learning, uses information-rich past data concurrently with the standard parameter update laws to guarantee parameter convergence without the need of the PE cond… ▽ More Convergence of controller parameters in standard model reference adaptive control (MRAC) requires the system states to be persistently exciting (PE), a restrictive condition to be verified online. A recent data-driven approach, concurrent learning, uses information-rich past data concurrently with the standard parameter update laws to guarantee parameter convergence without the need of the PE condition. This method guarantees exponential convergence of both the tracking and the controller parameter estimation errors to zero, whereas, the classical MRAC merely ensures asymptotic convergence of tracking error to zero. However, the method requires knowledge of the state derivative, at least at the time instances when the state values are stored in memory. The method further assumes knowledge of the control allocation matrix. This paper addresses these limitations by using a memory-based finite-time system identifier in conjunction with a data-driven approach, leading to convergence of both the tracking and the controller parameter estimation errors without the PE condition and knowledge of the system matrices and the state derivative. A Lyapunov based stability proof is included to justify the validity of the proposed data-driven approach. Simulation results demonstrate the efficacy of the suggested method. △ Less

Submitted 1 February, 2016; originally announced February 2016.

Generalization and Exploration via Randomized Value Functions

Authors: Ian Osband, Benjamin Van Roy, Zheng Wen

Abstract: We propose randomized least-squares value iteration (RLSVI) -- a new reinforcement learning algorithm designed to explore and generalize efficiently via linearly parameterized value functions. We explain why versions of least-squares value iteration that use Boltzmann or epsilon-greedy exploration can be highly inefficient, and we present computational results that demonstrate dramatic efficiency… ▽ More We propose randomized least-squares value iteration (RLSVI) -- a new reinforcement learning algorithm designed to explore and generalize efficiently via linearly parameterized value functions. We explain why versions of least-squares value iteration that use Boltzmann or epsilon-greedy exploration can be highly inefficient, and we present computational results that demonstrate dramatic efficiency gains enjoyed by RLSVI. Further, we establish an upper bound on the expected regret of RLSVI that demonstrates near-optimality in a tabula rasa learning context. More broadly, our results suggest that randomized value functions offer a promising approach to tackling a critical challenge in reinforcement learning: synthesizing efficient exploration and effective generalization. △ Less

Submitted 15 February, 2016; v1 submitted 4 February, 2014; originally announced February 2014.

Comments: arXiv admin note: text overlap with arXiv:1307.4847

Efficient Reinforcement Learning in Deterministic Systems with Value Function Generalization

Authors: Zheng Wen, Benjamin Van Roy

Abstract: We consider the problem of reinforcement learning over episodes of a finite-horizon deterministic system and as a solution propose optimistic constraint propagation (OCP), an algorithm designed to synthesize efficient exploration and value function generalization. We establish that when the true value function lies within a given hypothesis class, OCP selects optimal actions over all but at most K… ▽ More We consider the problem of reinforcement learning over episodes of a finite-horizon deterministic system and as a solution propose optimistic constraint propagation (OCP), an algorithm designed to synthesize efficient exploration and value function generalization. We establish that when the true value function lies within a given hypothesis class, OCP selects optimal actions over all but at most K episodes, where K is the eluder dimension of the given hypothesis class. We establish further efficiency and asymptotic performance guarantees that apply even if the true value function does not lie in the given hypothesis class, for the special case where the hypothesis class is the span of pre-specified indicator functions over disjoint sets. We also discuss the computational complexity of OCP and present computational results involving two illustrative examples. △ Less

Submitted 6 July, 2016; v1 submitted 18 July, 2013; originally announced July 2013.

Directed Time Series Regression for Control

Authors: Yi-Hao Kao, Benjamin Van Roy

Abstract: We propose directed time series regression, a new approach to estimating parameters of time-series models for use in certainty equivalent model predictive control. The approach combines merits of least squares regression and empirical optimization. Through a computational study involving a stochastic version of a well known inverted pendulum balancing problem, we demonstrate that directed time ser… ▽ More We propose directed time series regression, a new approach to estimating parameters of time-series models for use in certainty equivalent model predictive control. The approach combines merits of least squares regression and empirical optimization. Through a computational study involving a stochastic version of a well known inverted pendulum balancing problem, we demonstrate that directed time series regression can generate significant improvements in controller performance over either of the aforementioned alternatives. △ Less

Submitted 26 June, 2012; originally announced June 2012.