-
scenario.center: Methods from Real-world Data to a Scenario Database
Authors:
Michael Schuldes,
Christoph Glasmacher,
Lutz Eckstein
Abstract:
Scenario-based testing is a promising method to develop, verify and validate automated driving systems (ADS) since pure on-road testing seems inefficient for complex traffic environments. A major challenge for this approach is the provision and management of a sufficient number of scenarios to test a system. The provision, generation, and management of scenario at scale is investigated in current…
▽ More
Scenario-based testing is a promising method to develop, verify and validate automated driving systems (ADS) since pure on-road testing seems inefficient for complex traffic environments. A major challenge for this approach is the provision and management of a sufficient number of scenarios to test a system. The provision, generation, and management of scenario at scale is investigated in current research. This paper presents the scenario database scenario.center ( https://scenario.center ) to process and manage scenario data covering the needs of scenario-based testing approaches comprehensively and automatically. Thereby, requirements for such databases are described. Based on those, a four-step approach is proposed. Firstly, a common input format with defined quality requirements is defined. This is utilized for detecting events and base scenarios automatically. Furthermore, methods for searchability, evaluation of data quality and different scenario generation methods are proposed to allow a broad applicability serving different needs. For evaluation, the methodology is compared to state-of-the-art scenario databases. Finally, the application and capabilities of the database are shown by applying the methodology to the inD dataset. A public demonstration of the database interface is provided at https://scenario.center .
△ Less
Submitted 19 April, 2024; v1 submitted 3 April, 2024;
originally announced April 2024.
-
Causality-based Transfer of Driving Scenarios to Unseen Intersections
Authors:
Christoph Glasmacher,
Michael Schuldes,
Sleiman El Masri,
Lutz Eckstein
Abstract:
Scenario-based testing of automated driving functions has become a promising method to reduce time and cost compared to real-world testing. In scenario-based testing automated functions are evaluated in a set of pre-defined scenarios. These scenarios provide information about vehicle behaviors, environmental conditions, or road characteristics using parameters. To create realistic scenarios, param…
▽ More
Scenario-based testing of automated driving functions has become a promising method to reduce time and cost compared to real-world testing. In scenario-based testing automated functions are evaluated in a set of pre-defined scenarios. These scenarios provide information about vehicle behaviors, environmental conditions, or road characteristics using parameters. To create realistic scenarios, parameters and parameter dependencies have to be fitted utilizing real-world data. However, due to the large variety of intersections and movement constellations found in reality, data may not be available for certain scenarios. This paper proposes a methodology to systematically analyze relations between parameters of scenarios. Bayesian networks are utilized to analyze causal dependencies in order to decrease the amount of required data and to transfer causal patterns creating unseen scenarios. Thereby, infrastructural influences on movement patterns are investigated to generate realistic scenarios on unobserved intersections. For evaluation, scenarios and underlying parameters are extracted from the inD dataset. Movement patterns are estimated, transferred and checked against recorded data from those initially unseen intersections.
△ Less
Submitted 2 April, 2024;
originally announced April 2024.
-
Towards a Completeness Argumentation for Scenario Concepts
Authors:
Christoph Glasmacher,
Hendrik Weber,
Lutz Eckstein
Abstract:
Scenario-based testing has become a promising approach to overcome the complexity of real-world traffic for safety assurance of automated vehicles. Within scenario-based testing, a system under test is confronted with a set of predefined scenarios. This set shall ensure more efficient testing of an automated vehicle operating in an open context compared to real-world testing. However, the question…
▽ More
Scenario-based testing has become a promising approach to overcome the complexity of real-world traffic for safety assurance of automated vehicles. Within scenario-based testing, a system under test is confronted with a set of predefined scenarios. This set shall ensure more efficient testing of an automated vehicle operating in an open context compared to real-world testing. However, the question arises if a scenario catalog can cover the open context sufficiently to allow an argumentation for sufficiently safe driving functions and how this can be proven. Within this paper, a methodology is proposed to argue a sufficient completeness of a scenario concept using a goal structured notation. Thereby, the distinction between completeness and coverage is discussed. For both, methods are proposed for a streamlined argumentation and regarding evidence. These methods are applied to a scenario concept and the inD dataset to prove the usability.
△ Less
Submitted 2 April, 2024;
originally announced April 2024.
-
Acquire Driving Scenarios Efficiently: A Framework for Prospective Assessment of Cost-Optimal Scenario Acquisition
Authors:
Christoph Glasmacher,
Michael Schuldes,
Hendrik Weber,
Nicolas Wagener,
Lutz Eckstein
Abstract:
Scenario-based testing is becoming increasingly important in safety assurance for automated driving. However, comprehensive and sufficiently complete coverage of the scenario space requires significant effort and resources if using only real-world data. To address this issue, driving scenario generation methods are developed and used more frequently, but the benefit of substituting generated data…
▽ More
Scenario-based testing is becoming increasingly important in safety assurance for automated driving. However, comprehensive and sufficiently complete coverage of the scenario space requires significant effort and resources if using only real-world data. To address this issue, driving scenario generation methods are developed and used more frequently, but the benefit of substituting generated data for real-world data has not yet been quantified. Additionally, the coverage of a set of concrete scenarios within a given logical scenario space has not been predicted yet. This paper proposes a methodology to quantify the cost-optimal usage of scenario generation approaches to reach a certainly complete scenario space coverage under given quality constraints and parametrization. Therefore, individual process steps for scenario generation and usage are investigated and evaluated using a meta model for the abstraction of knowledge-based and data-driven methods. Furthermore, a methodology is proposed to fit the meta model including the prediction of reachable complete coverage, quality criteria, and costs. Finally, the paper exemplary examines the suitability of a hybrid generation model under technical, economical, and quality constraints in comparison to different real-world scenario mining methods.
△ Less
Submitted 21 July, 2023;
originally announced July 2023.
-
An Automated Analysis Framework for Trajectory Datasets
Authors:
Christoph Glasmacher,
Robert Krajewski,
Lutz Eckstein
Abstract:
Trajectory datasets of road users have become more important in the last years for safety validation of highly automated vehicles. Several naturalistic trajectory datasets with each more than 10.000 tracks were released and others will follow. Considering this amount of data, it is necessary to be able to compare these datasets in-depth with ease to get an overview. By now, the datasets' own provi…
▽ More
Trajectory datasets of road users have become more important in the last years for safety validation of highly automated vehicles. Several naturalistic trajectory datasets with each more than 10.000 tracks were released and others will follow. Considering this amount of data, it is necessary to be able to compare these datasets in-depth with ease to get an overview. By now, the datasets' own provided information is mainly limited to meta-data and qualitative descriptions which are mostly not consistent with other datasets. This is insufficient for users to differentiate the emerging datasets for application-specific selection. Therefore, an automated analysis framework is proposed in this work. Starting with analyzing individual tracks, fourteen elementary characteristics, so-called detection types, are derived and used as the base of this framework. To describe each traffic scenario precisely, the detections are subdivided into common metrics, clustering methods and anomaly detection. Those are combined using a modular approach. The detections are composed into new scores to describe three defined attributes of each track data quantitatively: interaction, anomaly and relevance. These three scores are calculated hierarchically for different abstract layers to provide an overview not just between datasets but also for tracks, spatial regions and individual situations. So, an objective comparison between datasets can be realized. Furthermore, it can help to get a deeper understanding of the recorded infrastructure and its effect on road user behavior. To test the validity of the framework, a study is conducted to compare the scores with human perception. Additionally, several datasets are compared.
△ Less
Submitted 12 February, 2022;
originally announced February 2022.
