-
Falsification of Cyber-Physical Systems using Bayesian Optimization
Authors:
Zahra Ramezani,
Kenan Šehić,
Luigi Nardi,
Knut Åkesson
Abstract:
Cyber-physical systems (CPSs) are usually complex and safety-critical; hence, it is difficult and important to guarantee that the system's requirements, i.e., specifications, are fulfilled. Simulation-based falsification of CPSs is a practical testing method that can be used to raise confidence in the correctness of the system by only requiring that the system under test can be simulated. As each…
▽ More
Cyber-physical systems (CPSs) are usually complex and safety-critical; hence, it is difficult and important to guarantee that the system's requirements, i.e., specifications, are fulfilled. Simulation-based falsification of CPSs is a practical testing method that can be used to raise confidence in the correctness of the system by only requiring that the system under test can be simulated. As each simulation is typically computationally intensive, an important step is to reduce the number of simulations needed to falsify a specification. We study Bayesian optimization (BO), a sample-efficient method that learns a surrogate model that describes the relationship between the parametrization of possible input signals and the evaluation of the specification.
In this paper, we improve the falsification using BO by; first adopting two prominent BO methods, one fits local surrogate models, and the other exploits the user's prior knowledge. Secondly, the formulation of acquisition functions for falsification is addressed in this paper. Benchmark evaluation shows significant improvements in using local surrogate models of BO for falsifying benchmark examples that were previously hard to falsify. Using prior knowledge in the falsification process is shown to be particularly important when the simulation budget is limited. For some of the benchmark problems, the choice of acquisition function clearly affects the number of simulations needed for successful falsification.
△ Less
Submitted 12 February, 2023; v1 submitted 14 September, 2022;
originally announced September 2022.
-
Vision for Bosnia and Herzegovina in Artificial Intelligence Age: Global Trends, Potential Opportunities, Selected Use-cases and Realistic Goals
Authors:
Zlatan Ajanović,
Emina Aličković,
Aida Branković,
Sead Delalić,
Eldar Kurtić,
Salem Malikić,
Adnan Mehonić,
Hamza Merzić,
Kenan Šehić,
Bahrudin Trbalić
Abstract:
Artificial Intelligence (AI) is one of the most promising technologies of the 21. century, with an already noticeable impact on society and the economy. With this work, we provide a short overview of global trends, applications in industry and selected use-cases from our international experience and work in industry and academia. The goal is to present global and regional positive practices and pr…
▽ More
Artificial Intelligence (AI) is one of the most promising technologies of the 21. century, with an already noticeable impact on society and the economy. With this work, we provide a short overview of global trends, applications in industry and selected use-cases from our international experience and work in industry and academia. The goal is to present global and regional positive practices and provide an informed opinion on the realistic goals and opportunities for positioning B&H on the global AI scene.
△ Less
Submitted 8 September, 2022;
originally announced September 2022.
-
LassoBench: A High-Dimensional Hyperparameter Optimization Benchmark Suite for Lasso
Authors:
Kenan Šehić,
Alexandre Gramfort,
Joseph Salmon,
Luigi Nardi
Abstract:
While Weighted Lasso sparse regression has appealing statistical guarantees that would entail a major real-world impact in finance, genomics, and brain imaging applications, it is typically scarcely adopted due to its complex high-dimensional space composed by thousands of hyperparameters. On the other hand, the latest progress with high-dimensional hyperparameter optimization (HD-HPO) methods for…
▽ More
While Weighted Lasso sparse regression has appealing statistical guarantees that would entail a major real-world impact in finance, genomics, and brain imaging applications, it is typically scarcely adopted due to its complex high-dimensional space composed by thousands of hyperparameters. On the other hand, the latest progress with high-dimensional hyperparameter optimization (HD-HPO) methods for black-box functions demonstrates that high-dimensional applications can indeed be efficiently optimized. Despite this initial success, HD-HPO approaches are mostly applied to synthetic problems with a moderate number of dimensions, which limits its impact in scientific and engineering applications. We propose LassoBench, the first benchmark suite tailored for Weighted Lasso regression. LassoBench consists of benchmarks for both well-controlled synthetic setups (number of samples, noise level, ambient and effective dimensionalities, and multiple fidelities) and real-world datasets, which enables the use of many flavors of HPO algorithms to be studied and extended to the high-dimensional Lasso setting. We evaluate 6 state-of-the-art HPO methods and 3 Lasso baselines, and demonstrate that Bayesian optimization and evolutionary strategies can improve over the methods commonly used for sparse regression while highlighting limitations of these frameworks in very high-dimensional and noisy settings.
△ Less
Submitted 10 June, 2022; v1 submitted 4 November, 2021;
originally announced November 2021.
-
Low-dimensional offshore wave input for extreme event quantification
Authors:
Kenan Šehić,
Henrik Bredmose,
John D. Sørensen,
Mirza Karamehmedović
Abstract:
In offshore engineering design, nonlinear wave models are often used to propagate stochastic waves from an input boundary to the location of an offshore structure. Each wave realization is typically characterized by a high-dimensional input time series, and a reliable determination of the extreme events is associated with substantial computational effort. As the sea depth decreases, extreme events…
▽ More
In offshore engineering design, nonlinear wave models are often used to propagate stochastic waves from an input boundary to the location of an offshore structure. Each wave realization is typically characterized by a high-dimensional input time series, and a reliable determination of the extreme events is associated with substantial computational effort. As the sea depth decreases, extreme events become more difficult to evaluate. We here construct a low-dimensional characterization of the candidate input time series to circumvent the search for extreme wave events in a high-dimensional input probability space. Each wave input is represented by a unique low-dimensional set of parameters for which standard surrogate approximations, such as Gaussian processes, can estimate the short-term exceedance probability efficiently and accurately. We demonstrate the advantages of the new approach with a simple shallow-water wave model based on the Korteweg-de Vries equation for which we can provide an accurate reference solution based on the simple Monte Carlo method. We furthermore apply the method to a fully nonlinear wave model for wave propagation over a slo** seabed. The results demonstrate that the Gaussian process can learn accurately the tail of the heavy-tailed distribution of the maximum wave crest elevation based on only $1.7\%$ of the required Monte Carlo evaluations.
△ Less
Submitted 22 April, 2020;
originally announced April 2020.
-
Estimation of Failure Probabilities via Local Subset Approximations
Authors:
Kenan Šehić,
Mirza Karamehmedović
Abstract:
We here consider the subset simulation method which approaches a failure event using a decreasing sequence of nested intermediate failure events. The method resembles importance sampling, which actively explores a probability space by conditioning the next evaluation on the previous evaluations using a Markov chain Monte Carlo (MCMC) algorithm. A Markov chain typically requires many steps to estim…
▽ More
We here consider the subset simulation method which approaches a failure event using a decreasing sequence of nested intermediate failure events. The method resembles importance sampling, which actively explores a probability space by conditioning the next evaluation on the previous evaluations using a Markov chain Monte Carlo (MCMC) algorithm. A Markov chain typically requires many steps to estimate the target distribution, which is impractical with expensive numerical models. Therefore, we propose to approximate each step of a Markov chain locally with Gaussian process (GP) regression. Benchmark examples of reliability analysis show that local approximations significantly improve overall efficiency of subset simulation. They reduce the number of expensive limit-state evaluations by over $80\%$. However, GP regression becomes computationally impractical with increasing dimension. Therefore, to make our use of a GP feasible, we employ the partial least squares (PLS) regression, a gradient-free reduction method, locally to explore and utilize a low-dimensional subspace within a Markov chain. Numerical experiments illustrate a significant computational gain with maintained sufficient accuracy.
△ Less
Submitted 12 March, 2020;
originally announced March 2020.
-
Active-Subspace Analysis of Exceedance Probability for Shallow-Water Waves
Authors:
Kenan Šehić,
Henrik Bredmose,
John D. Sørensen,
Mirza Karamehmedović
Abstract:
We model shallow-water waves using a one-dimensional Korteweg-de Vries equation with the wave generation parameterized by random wave amplitudes for a predefined sea state. These wave amplitudes define the high-dimensional stochastic input vector for which we estimate the short-term wave crest exceedance probability at a reference point. For this high-dimensional and complex problem, most reliabil…
▽ More
We model shallow-water waves using a one-dimensional Korteweg-de Vries equation with the wave generation parameterized by random wave amplitudes for a predefined sea state. These wave amplitudes define the high-dimensional stochastic input vector for which we estimate the short-term wave crest exceedance probability at a reference point. For this high-dimensional and complex problem, most reliability methods fail, while Monte Carlo methods become impractical due to the slow convergence rate. Therefore, first within offshore applications, we employ the dimensionality reduction method called \textit{Active-Subspace Analysis}. This method identifies a low-dimensional subspace of the input space that is most significant to the input-output variability. We exploit this to efficiently train a Gaussian process that models the maximum 10-minute crest elevation at the reference point, and to thereby efficiently estimate the short-term wave crest exceedance probability. The active low-dimensional subspace for the Korteweg-de Vries model also exposes the expected incident wave groups associated with extreme waves and loads. Our results show the advantages and the effectiveness of the active-subspace analysis against the Monte Carlo implementation for offshore applications.
△ Less
Submitted 30 December, 2019;
originally announced January 2020.
