-
Tolerance of Reinforcement Learning Controllers against Deviations in Cyber Physical Systems
Authors:
Changjian Zhang,
Parv Kapoor,
Eunsuk Kang,
Romulo Meira-Goes,
David Garlan,
Akila Ganlath,
Shatadal Mishra,
Nejib Ammar
Abstract:
Cyber-physical systems (CPS) with reinforcement learning (RL)-based controllers are increasingly being deployed in complex physical environments such as autonomous vehicles, the Internet-of-Things(IoT), and smart cities. An important property of a CPS is tolerance; i.e., its ability to function safely under possible disturbances and uncertainties in the actual operation. In this paper, we introduc…
▽ More
Cyber-physical systems (CPS) with reinforcement learning (RL)-based controllers are increasingly being deployed in complex physical environments such as autonomous vehicles, the Internet-of-Things(IoT), and smart cities. An important property of a CPS is tolerance; i.e., its ability to function safely under possible disturbances and uncertainties in the actual operation. In this paper, we introduce a new, expressive notion of tolerance that describes how well a controller is capable of satisfying a desired system requirement, specified using Signal Temporal Logic (STL), under possible deviations in the system. Based on this definition, we propose a novel analysis problem, called the tolerance falsification problem, which involves finding small deviations that result in a violation of the given requirement. We present a novel, two-layer simulation-based analysis framework and a novel search heuristic for finding small tolerance violations. To evaluate our approach, we construct a set of benchmark problems where system parameters can be configured to represent different types of uncertainties and disturbancesin the system. Our evaluation shows that our falsification approach and heuristic can effectively find small tolerance violations.
△ Less
Submitted 24 June, 2024;
originally announced June 2024.
-
Transformer-based Model for ASR N-Best Rescoring and Rewriting
Authors:
Iwen E. Kang,
Christophe Van Gysel,
Man-Hung Siu
Abstract:
Voice assistants increasingly use on-device Automatic Speech Recognition (ASR) to ensure speed and privacy. However, due to resource constraints on the device, queries pertaining to complex information domains often require further processing by a search engine. For such applications, we propose a novel Transformer based model capable of rescoring and rewriting, by exploring full context of the N-…
▽ More
Voice assistants increasingly use on-device Automatic Speech Recognition (ASR) to ensure speed and privacy. However, due to resource constraints on the device, queries pertaining to complex information domains often require further processing by a search engine. For such applications, we propose a novel Transformer based model capable of rescoring and rewriting, by exploring full context of the N-best hypotheses in parallel. We also propose a new discriminative sequence training objective that can work well for both rescore and rewrite tasks. We show that our Rescore+Rewrite model outperforms the Rescore-only baseline, and achieves up to an average 8.6% relative Word Error Rate (WER) reduction over the ASR system by itself.
△ Less
Submitted 12 June, 2024;
originally announced June 2024.
-
Safe Planning through Incremental Decomposition of Signal Temporal Logic Specifications
Authors:
Parv Kapoor,
Eunsuk Kang,
Romulo Meira-Goes
Abstract:
Trajectory planning is a critical process that enables autonomous systems to safely navigate complex environments. Signal temporal logic (STL) specifications are an effective way to encode complex temporally extended objectives for trajectory planning in cyber-physical systems (CPS). However, planning from these specifications using existing techniques scale exponentially with the number of nested…
▽ More
Trajectory planning is a critical process that enables autonomous systems to safely navigate complex environments. Signal temporal logic (STL) specifications are an effective way to encode complex temporally extended objectives for trajectory planning in cyber-physical systems (CPS). However, planning from these specifications using existing techniques scale exponentially with the number of nested operators and the horizon of specification. Additionally, performance is exacerbated at runtime due to limited computational budgets and compounding modeling errors. Decomposing a complex specification into smaller subtasks and incrementally planning for them can remedy these issues. In this work, we present a way to decompose STL requirements temporally to improve planning efficiency and performance. The key insight in our work is to encode all specifications as a set of reachability and invariance constraints and scheduling these constraints sequentially at runtime. Our proposed technique outperforms the state-of-the-art trajectory synthesis techniques for both linear and non linear dynamical systems.
△ Less
Submitted 18 March, 2024; v1 submitted 13 March, 2024;
originally announced March 2024.
-
Integrating Graceful Degradation and Recovery through Requirement-driven Adaptation
Authors:
Simon Chu,
Justin Koe,
David Garlan,
Eunsuk Kang
Abstract:
Cyber-physical systems (CPS) are subject to environmental uncertainties such as adverse operating conditions, malicious attacks, and hardware degradation. These uncertainties may lead to failures that put the system in a sub-optimal or unsafe state. Systems that are resilient to such uncertainties rely on two types of operations: (1) graceful degradation, to ensure that the system maintains an acc…
▽ More
Cyber-physical systems (CPS) are subject to environmental uncertainties such as adverse operating conditions, malicious attacks, and hardware degradation. These uncertainties may lead to failures that put the system in a sub-optimal or unsafe state. Systems that are resilient to such uncertainties rely on two types of operations: (1) graceful degradation, to ensure that the system maintains an acceptable level of safety during unexpected environmental conditions and (2) recovery, to facilitate the resumption of normal system functions. Typically, mechanisms for degradation and recovery are developed independently from each other, and later integrated into a system, requiring the designer to develop an additional, ad-hoc logic for activating and coordinating between the two operations. In this paper, we propose a self-adaptation approach for improving system resiliency through automated triggering and coordination of graceful degradation and recovery. The key idea behind our approach is to treat degradation and recovery as requirement-driven adaptation tasks: Degradation can be thought of as temporarily weakening original (i.e., ideal) system requirements to be achieved by the system, and recovery as strengthening the weakened requirements when the environment returns within an expected operating boundary. Furthermore, by treating weakening and strengthening as dual operations, we argue that a single requirement-based adaptation method is sufficient to enable coordination between degradation and recovery. Given system requirements specified in signal temporal logic (STL), we propose a run-time adaptation framework that performs degradation and recovery in response to environmental changes. We describe a prototype implementation of our framework and demonstrate the feasibility of the proposed approach using a case study in unmanned underwater vehicles.
△ Less
Submitted 8 April, 2024; v1 submitted 17 January, 2024;
originally announced January 2024.
-
Investigating Robustness in Cyber-Physical Systems: Specification-Centric Analysis in the face of System Deviations
Authors:
Changjian Zhang,
Parv Kapoor,
Romulo Meira-Goes,
David Garlan,
Eunsuk Kang,
Akila Ganlath,
Shatadal Mishra,
Nejib Ammar
Abstract:
The adoption of cyber-physical systems (CPS) is on the rise in complex physical environments, encompassing domains such as autonomous vehicles, the Internet of Things (IoT), and smart cities. A critical attribute of CPS is robustness, denoting its capacity to operate safely despite potential disruptions and uncertainties in the operating environment. This paper proposes a novel specification-based…
▽ More
The adoption of cyber-physical systems (CPS) is on the rise in complex physical environments, encompassing domains such as autonomous vehicles, the Internet of Things (IoT), and smart cities. A critical attribute of CPS is robustness, denoting its capacity to operate safely despite potential disruptions and uncertainties in the operating environment. This paper proposes a novel specification-based robustness, which characterizes the effectiveness of a controller in meeting a specified system requirement, articulated through Signal Temporal Logic (STL) while accounting for possible deviations in the system. This paper also proposes the robustness falsification problem based on the definition, which involves identifying minor deviations capable of violating the specified requirement. We present an innovative two-layer simulation-based analysis framework designed to identify subtle robustness violations. To assess our methodology, we devise a series of benchmark problems wherein system parameters can be adjusted to emulate various forms of uncertainties and disturbances. Initial evaluations indicate that our falsification approach proficiently identifies robustness violations, providing valuable insights for comparing robustness between conventional and reinforcement learning (RL)-based controllers
△ Less
Submitted 25 March, 2024; v1 submitted 13 November, 2023;
originally announced November 2023.
-
Runtime Resolution of Feature Interactions through Adaptive Requirement Weakening
Authors:
Simon Chu,
Emma Shedden,
Changjian Zhang,
Rômulo Meira-Góes,
Gabriel A. Moreno,
David Garlan,
Eunsuk Kang
Abstract:
The feature interaction problem occurs when two or more independently developed components interact with each other in unanticipated ways, resulting in undesirable system behaviors. Feature interaction problems remain a challenge for emerging domains in cyber-physical systems (CPS), such as the Internet of Things and autonomous drones. Existing techniques for resolving feature interactions take a…
▽ More
The feature interaction problem occurs when two or more independently developed components interact with each other in unanticipated ways, resulting in undesirable system behaviors. Feature interaction problems remain a challenge for emerging domains in cyber-physical systems (CPS), such as the Internet of Things and autonomous drones. Existing techniques for resolving feature interactions take a "winner-takes-all" approach, where one out of the conflicting features is selected as the most desirable one, and the rest are disabled. However, when multiple of the conflicting features fulfill important system requirements, being forced to select one of them can result in an undesirable system outcome. In this paper, we propose a new resolution approach that allows all of the conflicting features to continue to partially fulfill their requirements during the resolution process. In particular, our approach leverages the idea of adaptive requirement weakening, which involves one or more features temporarily weakening their level of performance in order to co-exist with the other features in a consistent manner. Given feature requirements specified in Signal Temporal Logic (STL), we propose an automated method and a runtime architecture for automatically weakening the requirements to resolve a conflict. We demonstrate our approach through case studies on feature interactions in autonomous drones.
△ Less
Submitted 27 October, 2023;
originally announced October 2023.
-
A Quantitatively Interpretable Model for Alzheimer's Disease Prediction Using Deep Counterfactuals
Authors:
Kwanseok Oh,
Da-Woon Heo,
Ahmad Wisnu Mulyadi,
Wonsik Jung,
Eunsong Kang,
Kun Ho Lee,
Heung-Il Suk
Abstract:
Deep learning (DL) for predicting Alzheimer's disease (AD) has provided timely intervention in disease progression yet still demands attentive interpretability to explain how their DL models make definitive decisions. Recently, counterfactual reasoning has gained increasing attention in medical research because of its ability to provide a refined visual explanatory map. However, such visual explan…
▽ More
Deep learning (DL) for predicting Alzheimer's disease (AD) has provided timely intervention in disease progression yet still demands attentive interpretability to explain how their DL models make definitive decisions. Recently, counterfactual reasoning has gained increasing attention in medical research because of its ability to provide a refined visual explanatory map. However, such visual explanatory maps based on visual inspection alone are insufficient unless we intuitively demonstrate their medical or neuroscientific validity via quantitative features. In this study, we synthesize the counterfactual-labeled structural MRIs using our proposed framework and transform it into a gray matter density map to measure its volumetric changes over the parcellated region of interest (ROI). We also devised a lightweight linear classifier to boost the effectiveness of constructed ROIs, promoted quantitative interpretation, and achieved comparable predictive performance to DL methods. Throughout this, our framework produces an ``AD-relatedness index'' for each ROI and offers an intuitive understanding of brain status for an individual patient and across patient groups with respect to AD progression.
△ Less
Submitted 5 October, 2023;
originally announced October 2023.
-
Safe Environmental Envelopes of Discrete Systems
Authors:
Rômulo Meira-Góes,
Ian Dardik,
Eunsuk Kang,
Stéphane Lafortune,
Stavros Tripakis
Abstract:
A safety verification task involves verifying a system against a desired safety property under certain assumptions about the environment. However, these environmental assumptions may occasionally be violated due to modeling errors or faults. Ideally, the system guarantees its critical properties even under some of these violations, i.e., the system is \emph{robust} against environmental deviations…
▽ More
A safety verification task involves verifying a system against a desired safety property under certain assumptions about the environment. However, these environmental assumptions may occasionally be violated due to modeling errors or faults. Ideally, the system guarantees its critical properties even under some of these violations, i.e., the system is \emph{robust} against environmental deviations. This paper proposes a notion of \emph{robustness} as an explicit, first-class property of a transition system that captures how robust it is against possible \emph{deviations} in the environment. We modeled deviations as a set of \emph{transitions} that may be added to the original environment. Our robustness notion then describes the safety envelope of this system, i.e., it captures all sets of extra environment transitions for which the system still guarantees a desired property. We show that being able to explicitly reason about robustness enables new types of system analysis and design tasks beyond the common verification problem stated above. We demonstrate the application of our framework on case studies involving a radiation therapy interface, an electronic voting machine, a fare collection protocol, and a medical pump device.
△ Less
Submitted 1 June, 2023;
originally announced June 2023.
-
Discovering novel systemic biomarkers in photos of the external eye
Authors:
Boris Babenko,
Ilana Traynis,
Christina Chen,
Preeti Singh,
Akib Uddin,
Jorge Cuadros,
Lauren P. Daskivich,
April Y. Maa,
Ramasamy Kim,
Eugene Yu-Chuan Kang,
Yossi Matias,
Greg S. Corrado,
Lily Peng,
Dale R. Webster,
Christopher Semturs,
Jonathan Krause,
Avinash V. Varadarajan,
Naama Hammel,
Yun Liu
Abstract:
External eye photos were recently shown to reveal signs of diabetic retinal disease and elevated HbA1c. In this paper, we evaluate if external eye photos contain information about additional systemic medical conditions. We developed a deep learning system (DLS) that takes external eye photos as input and predicts multiple systemic parameters, such as those related to the liver (albumin, AST); kidn…
▽ More
External eye photos were recently shown to reveal signs of diabetic retinal disease and elevated HbA1c. In this paper, we evaluate if external eye photos contain information about additional systemic medical conditions. We developed a deep learning system (DLS) that takes external eye photos as input and predicts multiple systemic parameters, such as those related to the liver (albumin, AST); kidney (eGFR estimated using the race-free 2021 CKD-EPI creatinine equation, the urine ACR); bone & mineral (calcium); thyroid (TSH); and blood count (Hgb, WBC, platelets). Development leveraged 151,237 images from 49,015 patients with diabetes undergoing diabetic eye screening in 11 sites across Los Angeles county, CA. Evaluation focused on 9 pre-specified systemic parameters and leveraged 3 validation sets (A, B, C) spanning 28,869 patients with and without diabetes undergoing eye screening in 3 independent sites in Los Angeles County, CA, and the greater Atlanta area, GA. We compared against baseline models incorporating available clinicodemographic variables (e.g. age, sex, race/ethnicity, years with diabetes). Relative to the baseline, the DLS achieved statistically significant superior performance at detecting AST>36, calcium<8.6, eGFR<60, Hgb<11, platelets<150, ACR>=300, and WBC<4 on validation set A (a patient population similar to the development sets), where the AUC of DLS exceeded that of the baseline by 5.2-19.4%. On validation sets B and C, with substantial patient population differences compared to the development sets, the DLS outperformed the baseline for ACR>=300 and Hgb<11 by 7.3-13.2%. Our findings provide further evidence that external eye photos contain important biomarkers of systemic health spanning multiple organ systems. Further work is needed to investigate whether and how these biomarkers can be translated into clinical impact.
△ Less
Submitted 18 July, 2022;
originally announced July 2022.
-
On tolerance of discrete systems with respect to transition perturbations
Authors:
Rômulo Meira-Góes,
Eunsuk Kang,
Stéphane Lafortune,
Stavros Tripakis
Abstract:
Control systems should enforce a desired property for both expected modeled situations as well as unexpected unmodeled environmental situations. Existing methods focus on designing controllers to enforce the desired property only when the environment behaves as expected. However, these methods lack discussion on how the system behaves when the environment is perturbed. In this paper, we propose an…
▽ More
Control systems should enforce a desired property for both expected modeled situations as well as unexpected unmodeled environmental situations. Existing methods focus on designing controllers to enforce the desired property only when the environment behaves as expected. However, these methods lack discussion on how the system behaves when the environment is perturbed. In this paper, we propose an approach for analyzing control systems with respect to their tolerance against environmental perturbations. A control system tolerates certain environmental perturbations when it remains capable of guaranteeing the desired property despite the perturbations. Each controller inherently has a level of tolerance against environmental perturbations. We formally define this notion of tolerance and describe a general technique to compute it, for any given regular property. We also present a more efficient method to compute tolerance with respect to invariance properties. Moreover, we introduce a new controller synthesis problem based on our notion of tolerance. We demonstrate the application of our framework on an autonomous surveillance example.
△ Less
Submitted 18 October, 2021; v1 submitted 8 October, 2021;
originally announced October 2021.
-
Synthesis of Sensor Deception Attacks at the Supervisory Layer of Cyber-Physical Systems
Authors:
Romulo Meira-Goes,
Eunsuk Kang,
Raymond H. Kwong,
Stephane Lafortune
Abstract:
We study the security of Cyber-Physical Systems (CPS) in the context of the supervisory control layer. Specifically, we propose a general model of a CPS attacker in the framework of discrete event systems and investigate the problem of synthesizing an attack strategy for a given feedback control system. Our model captures a class of deception attacks, where the attacker has the ability to hijack a…
▽ More
We study the security of Cyber-Physical Systems (CPS) in the context of the supervisory control layer. Specifically, we propose a general model of a CPS attacker in the framework of discrete event systems and investigate the problem of synthesizing an attack strategy for a given feedback control system. Our model captures a class of deception attacks, where the attacker has the ability to hijack a subset of sensor readings and mislead the supervisor, with the goal of inducing the system into an undesirable state. We utilize a game-like discrete transition structure, called Insertion-Deletion Attack structure (IDA), to capture the interaction between the supervisor and the environment (which includes the system and the attacker). We show how to use IDAs to synthesize three different types of successful stealthy attacks, i.e., attacks that avoid detection from the supervisor and cause damage to the system.
△ Less
Submitted 4 August, 2020;
originally announced August 2020.