-
A survey on robustness in trajectory prediction for autonomous vehicles
Authors:
Jeroen Hagenus,
Frederik Baymler Mathiesen,
Julian F. Schumann,
Arkady Zgonnikov
Abstract:
Autonomous vehicles rely on accurate trajectory prediction to inform decision-making processes related to navigation and collision avoidance. However, current trajectory prediction models show signs of overfitting, which may lead to unsafe or suboptimal behavior. To address these challenges, this paper presents a comprehensive framework that categorizes and assesses the definitions and strategies…
▽ More
Autonomous vehicles rely on accurate trajectory prediction to inform decision-making processes related to navigation and collision avoidance. However, current trajectory prediction models show signs of overfitting, which may lead to unsafe or suboptimal behavior. To address these challenges, this paper presents a comprehensive framework that categorizes and assesses the definitions and strategies used in the literature on evaluating and improving the robustness of trajectory prediction models. This involves a detailed exploration of various approaches, including data slicing methods, perturbation techniques, model architecture changes, and post-training adjustments. In the literature, we see many promising methods for increasing robustness, which are necessary for safe and reliable autonomous driving.
△ Less
Submitted 20 April, 2024; v1 submitted 2 February, 2024;
originally announced February 2024.
-
Robust Multi-Modal Density Estimation
Authors:
Anna Mészáros,
Julian F. Schumann,
Javier Alonso-Mora,
Arkady Zgonnikov,
Jens Kober
Abstract:
The estimation of probability density functions is a fundamental problem in science and engineering. However, common methods such as kernel density estimation (KDE) have been demonstrated to lack robustness, while more complex methods have not been evaluated in multi-modal estimation problems. In this paper, we present ROME (RObust Multi-modal Estimator), a non-parametric approach for density esti…
▽ More
The estimation of probability density functions is a fundamental problem in science and engineering. However, common methods such as kernel density estimation (KDE) have been demonstrated to lack robustness, while more complex methods have not been evaluated in multi-modal estimation problems. In this paper, we present ROME (RObust Multi-modal Estimator), a non-parametric approach for density estimation which addresses the challenge of estimating multi-modal, non-normal, and highly correlated distributions. ROME utilizes clustering to segment a multi-modal set of samples into multiple uni-modal ones and then combines simple KDE estimates obtained for individual clusters in a single multi-modal estimate. We compared our approach to state-of-the-art methods for density estimation as well as ablations of ROME, showing that it not only outperforms established methods but is also more robust to a variety of distributions. Our results demonstrate that ROME can overcome the issues of over-fitting and over-smoothing exhibited by other estimators.
△ Less
Submitted 6 May, 2024; v1 submitted 19 January, 2024;
originally announced January 2024.
-
Runway Sign Classifier: A DAL C Certifiable Machine Learning System
Authors:
Konstantin Dmitriev,
Johann Schumann,
Islam Bostanov,
Mostafa Abdelhamid,
Florian Holzapfel
Abstract:
In recent years, the remarkable progress of Machine Learning (ML) technologies within the domain of Artificial Intelligence (AI) systems has presented unprecedented opportunities for the aviation industry, paving the way for further advancements in automation, including the potential for single pilot or fully autonomous operation of large commercial airplanes. However, ML technology faces major in…
▽ More
In recent years, the remarkable progress of Machine Learning (ML) technologies within the domain of Artificial Intelligence (AI) systems has presented unprecedented opportunities for the aviation industry, paving the way for further advancements in automation, including the potential for single pilot or fully autonomous operation of large commercial airplanes. However, ML technology faces major incompatibilities with existing airborne certification standards, such as ML model traceability and explainability issues or the inadequacy of traditional coverage metrics. Certification of ML-based airborne systems using current standards is problematic due to these challenges. This paper presents a case study of an airborne system utilizing a Deep Neural Network (DNN) for airport sign detection and classification. Building upon our previous work, which demonstrates compliance with Design Assurance Level (DAL) D, we upgrade the system to meet the more stringent requirements of Design Assurance Level C. To achieve DAL C, we employ an established architectural mitigation technique involving two redundant and dissimilar Deep Neural Networks. The application of novel ML-specific data management techniques further enhances this approach. This work is intended to illustrate how the certification challenges of ML-based systems can be addressed for medium criticality airborne applications.
△ Less
Submitted 10 October, 2023;
originally announced October 2023.
-
Smooth-Trajectron++: Augmenting the Trajectron++ behaviour prediction model with smooth attention
Authors:
Frederik S. B. Westerhout,
Julian F. Schumann,
Arkady Zgonnikov
Abstract:
Understanding traffic participants' behaviour is crucial for predicting their future trajectories, aiding in develo** safe and reliable planning systems for autonomous vehicles. Integrating cognitive processes and machine learning models has shown promise in other domains but is lacking in the trajectory forecasting of multiple traffic agents in large-scale autonomous driving datasets. This work…
▽ More
Understanding traffic participants' behaviour is crucial for predicting their future trajectories, aiding in develo** safe and reliable planning systems for autonomous vehicles. Integrating cognitive processes and machine learning models has shown promise in other domains but is lacking in the trajectory forecasting of multiple traffic agents in large-scale autonomous driving datasets. This work investigates the state-of-the-art trajectory forecasting model Trajectron++ which we enhance by incorporating a smoothing term in its attention module. This attention mechanism mimics human attention inspired by cognitive science research indicating limits to attention switching. We evaluate the performance of the resulting Smooth-Trajectron++ model and compare it to the original model on various benchmarks, revealing the potential of incorporating insights from human cognition into trajectory prediction models.
△ Less
Submitted 2 June, 2023; v1 submitted 31 May, 2023;
originally announced May 2023.
-
Using Models Based on Cognitive Theory to Predict Human Behavior in Traffic: A Case Study
Authors:
Julian F. Schumann,
Aravinda Ramakrishnan Srinivasan,
Jens Kober,
Gustav Markkula,
Arkady Zgonnikov
Abstract:
The development of automated vehicles has the potential to revolutionize transportation, but they are currently unable to ensure a safe and time-efficient driving style. Reliable models predicting human behavior are essential for overcoming this issue. While data-driven models are commonly used to this end, they can be vulnerable in safety-critical edge cases. This has led to an interest in models…
▽ More
The development of automated vehicles has the potential to revolutionize transportation, but they are currently unable to ensure a safe and time-efficient driving style. Reliable models predicting human behavior are essential for overcoming this issue. While data-driven models are commonly used to this end, they can be vulnerable in safety-critical edge cases. This has led to an interest in models incorporating cognitive theory, but as such models are commonly developed for explanatory purposes, this approach's effectiveness in behavior prediction has remained largely untested so far. In this article, we investigate the usefulness of the \emph{Commotions} model -- a novel cognitively plausible model incorporating the latest theories of human perception, decision-making, and motor control -- for predicting human behavior in gap acceptance scenarios, which entail many important traffic interactions such as lane changes and intersections. We show that this model can compete with or even outperform well-established data-driven prediction models across several naturalistic datasets. These results demonstrate the promise of incorporating cognitive theory in behavior prediction models for automated vehicles.
△ Less
Submitted 9 October, 2023; v1 submitted 24 May, 2023;
originally announced May 2023.
-
The COMMOTIONS Urban Interactions Driving Simulator Study Dataset
Authors:
Aravinda Ramakrishnan Srinivasan,
Julian Schumann,
Yueyang Wang,
Yi-Shin Lin,
Michael Daly,
Albert Solernou,
Arkady Zgonnikov,
Matteo Leonetti,
Jac Billington,
Gustav Markkula
Abstract:
Accurate modelling of road user interaction has received lot of attention in recent years due to the advent of increasingly automated vehicles. To support such modelling, there is a need to complement naturalistic datasets of road user interaction with targeted, controlled study data. This paper describes a dataset collected in a simulator study conducted in the project COMMOTIONS, addressing urba…
▽ More
Accurate modelling of road user interaction has received lot of attention in recent years due to the advent of increasingly automated vehicles. To support such modelling, there is a need to complement naturalistic datasets of road user interaction with targeted, controlled study data. This paper describes a dataset collected in a simulator study conducted in the project COMMOTIONS, addressing urban driving interactions, in a state of the art moving base driving simulator. The study focused on two types of near-crash situations that can arise in urban driving interactions, and also collected data on human driver gap acceptance across a range of controlled gap sequences.
△ Less
Submitted 2 July, 2024; v1 submitted 17 May, 2023;
originally announced May 2023.
-
Learning Distributions over Trajectories for Human Behavior Prediction
Authors:
Anna Mészáros,
Julian F. Schumann,
Javier Alonso-Mora,
Arkady Zgonnikov,
Jens Kober
Abstract:
Predicting the future behavior of human road users is an important aspect for the development of risk-aware autonomous vehicles. While many models have been developed towards this end, effectively capturing and predicting the variability inherent to human behavior still remains an open challenge. This paper proposes TrajFlow - a new approach for probabilistic trajectory prediction based on Normali…
▽ More
Predicting the future behavior of human road users is an important aspect for the development of risk-aware autonomous vehicles. While many models have been developed towards this end, effectively capturing and predicting the variability inherent to human behavior still remains an open challenge. This paper proposes TrajFlow - a new approach for probabilistic trajectory prediction based on Normalizing Flows. We reformulate the problem of capturing distributions over trajectories into capturing distributions over abstracted trajectory features using an autoencoder, simplifying the learning task of the Normalizing Flows. TrajFlow outperforms state-of-the-art behavior prediction models in capturing full trajectory distributions in two synthetic benchmarks with known true distributions, and is competitive on the naturalistic datasets ETH/UCY, rounD, and nuScenes. Our results demonstrate the effectiveness of TrajFlow in probabilistic prediction of human behavior.
△ Less
Submitted 19 April, 2024; v1 submitted 11 April, 2023;
originally announced April 2023.
-
Benchmark for Models Predicting Human Behavior in Gap Acceptance Scenarios
Authors:
Julian Frederik Schumann,
Jens Kober,
Arkady Zgonnikov
Abstract:
Autonomous vehicles currently suffer from a time-inefficient driving style caused by uncertainty about human behavior in traffic interactions. Accurate and reliable prediction models enabling more efficient trajectory planning could make autonomous vehicles more assertive in such interactions. However, the evaluation of such models is commonly oversimplistic, ignoring the asymmetric importance of…
▽ More
Autonomous vehicles currently suffer from a time-inefficient driving style caused by uncertainty about human behavior in traffic interactions. Accurate and reliable prediction models enabling more efficient trajectory planning could make autonomous vehicles more assertive in such interactions. However, the evaluation of such models is commonly oversimplistic, ignoring the asymmetric importance of prediction errors and the heterogeneity of the datasets used for testing. We examine the potential of recasting interactions between vehicles as gap acceptance scenarios and evaluating models in this structured environment. To that end, we develop a framework aiming to facilitate the evaluation of any model, by any metric, and in any scenario. We then apply this framework to state-of-the-art prediction models, which all show themselves to be unreliable in the most safety-critical situations.
△ Less
Submitted 20 February, 2023; v1 submitted 10 November, 2022;
originally announced November 2022.
-
Toward Certification of Machine-Learning Systems for Low Criticality Airborne Applications
Authors:
K. Dmitriev,
J. Schumann,
F. Holzapfel
Abstract:
The exceptional progress in the field of machine learning (ML) in recent years has attracted a lot of interest in using this technology in aviation. Possible airborne applications of ML include safety-critical functions, which must be developed in compliance with rigorous certification standards of the aviation industry. Current certification standards for the aviation industry were developed prio…
▽ More
The exceptional progress in the field of machine learning (ML) in recent years has attracted a lot of interest in using this technology in aviation. Possible airborne applications of ML include safety-critical functions, which must be developed in compliance with rigorous certification standards of the aviation industry. Current certification standards for the aviation industry were developed prior to the ML renaissance without taking specifics of ML technology into account. There are some fundamental incompatibilities between traditional design assurance approaches and certain aspects of ML-based systems. In this paper, we analyze the current airborne certification standards and show that all objectives of the standards can be achieved for a low-criticality ML-based system if certain assumptions about ML development workflow are applied.
△ Less
Submitted 28 September, 2022;
originally announced September 2022.
-
A machine learning approach for fighting the curse of dimensionality in global optimization
Authors:
Julian F. Schumann,
Alejandro M. Aragón
Abstract:
Finding global optima in high-dimensional optimization problems is extremely challenging since the number of function evaluations required to sufficiently explore the search space increases exponentially with its dimensionality. Furthermore, multimodal cost functions render local gradient-based search techniques ineffective. To overcome these difficulties, we propose to trim uninteresting regions…
▽ More
Finding global optima in high-dimensional optimization problems is extremely challenging since the number of function evaluations required to sufficiently explore the search space increases exponentially with its dimensionality. Furthermore, multimodal cost functions render local gradient-based search techniques ineffective. To overcome these difficulties, we propose to trim uninteresting regions of the search space where global optima are unlikely to be found by means of autoencoders, exploiting the lower intrinsic dimensionality of certain cost functions; optima are then searched over lower-dimensional latent spaces. The methodology is tested on benchmark functions and on multiple variations of a structural topology optimization problem, where we show that we can estimate this intrinsic lower dimensionality and based thereon obtain the global optimum at best or superior results compared to established optimization procedures at worst.
△ Less
Submitted 3 November, 2022; v1 submitted 28 October, 2021;
originally announced October 2021.
-
Presynaptic modulation as fast synaptic switching: state-dependent modulation of task performance
Authors:
Gabriele Scheler,
Johann Schumann
Abstract:
Neuromodulatory receptors in presynaptic position have the ability to suppress synaptic transmission for seconds to minutes when fully engaged. This effectively alters the synaptic strength of a connection. Much work on neuromodulation has rested on the assumption that these effects are uniform at every neuron. However, there is considerable evidence to suggest that presynaptic regulation may be…
▽ More
Neuromodulatory receptors in presynaptic position have the ability to suppress synaptic transmission for seconds to minutes when fully engaged. This effectively alters the synaptic strength of a connection. Much work on neuromodulation has rested on the assumption that these effects are uniform at every neuron. However, there is considerable evidence to suggest that presynaptic regulation may be in effect synapse-specific. This would define a second "weight modulation" matrix, which reflects presynaptic receptor efficacy at a given site. Here we explore functional consequences of this hypothesis. By analyzing and comparing the weight matrices of networks trained on different aspects of a task, we identify the potential for a low complexity "modulation matrix", which allows to switch between differently trained subtasks while retaining general performance characteristics for the task. This means that a given network can adapt itself to different task demands by regulating its release of neuromodulators. Specifically, we suggest that (a) a network can provide optimized responses for related classification tasks without the need to train entirely separate networks and (b) a network can blend a "memory mode" which aims at reproducing memorized patterns and a "novelty mode" which aims to facilitate classification of new patterns. We relate this work to the known effects of neuromodulators on brain-state dependent processing.
△ Less
Submitted 24 January, 2004;
originally announced January 2004.
-
Automatic Debugging Support for UML Designs
Authors:
Johann Schumann
Abstract:
Design of large software systems requires rigorous application of software engineering methods covering all phases of the software process. Debugging during the early design phases is extremely important, because late bug-fixes are expensive.
In this paper, we describe an approach which facilitates debugging of UML requirements and designs. The Unified Modeling Language (UML) is a set of notat…
▽ More
Design of large software systems requires rigorous application of software engineering methods covering all phases of the software process. Debugging during the early design phases is extremely important, because late bug-fixes are expensive.
In this paper, we describe an approach which facilitates debugging of UML requirements and designs. The Unified Modeling Language (UML) is a set of notations for object-orient design of a software system. We have developed an algorithm which translates requirement specifications in the form of annotated sequence diagrams into structured statecharts. This algorithm detects conflicts between sequence diagrams and inconsistencies in the domain knowledge. After synthesizing statecharts from sequence diagrams, these statecharts usually are subject to manual modification and refinement. By using the ``backward'' direction of our synthesis algorithm, we are able to map modifications made to the statechart back into the requirements (sequence diagrams) and check for conflicts there. Fed back to the user conflicts detected by our algorithm are the basis for deductive-based debugging of requirements and domain theory in very early development stages. Our approach allows to generate explanations on why there is a conflict and which parts of the specifications are affected.
△ Less
Submitted 13 November, 2000;
originally announced November 2000.
