Showing 1–10 of 10 results for author: de Gelder, E

A Quantitative Method to Determine What Collisions Are Reasonably Foreseeable and Preventable

Authors: Erwin de Gelder, Olaf Op den Camp

Abstract: The development of Automated Driving Systems (ADSs) has made significant progress in the last years. To enable the deployment of Automated Vehicles (AVs) equipped with such ADSs, regulations concerning the approval of these systems need to be established. In 2021, the World Forum for Harmonization of Vehicle Regulations has approved a new United Nations regulation concerning the approval of Automa… ▽ More The development of Automated Driving Systems (ADSs) has made significant progress in the last years. To enable the deployment of Automated Vehicles (AVs) equipped with such ADSs, regulations concerning the approval of these systems need to be established. In 2021, the World Forum for Harmonization of Vehicle Regulations has approved a new United Nations regulation concerning the approval of Automated Lane Kee** Systems (ALKSs). An important aspect of this regulation is that "the activated system shall not cause any collisions that are reasonably foreseeable and preventable." The phrasing of "reasonably foreseeable and preventable" might be subjected to different interpretations and, therefore, this might result in disagreements among AV developers and the authorities that are requested to approve AVs. The objective of this work is to propose a method for quantifying what is "reasonably foreseeable and preventable". The proposed method considers the Operational Design Domain (ODD) of the system and can be applied to any ODD. Having a quantitative method for determining what is reasonably foreseeable and preventable provides developers, authorities, and the users of ADSs a better understanding of the residual risks to be expected when deploying these systems in real traffic. Using our proposed method, we can estimate what collisions are reasonably foreseeable and preventable. This will help in setting requirements regarding the safety of ADSs and can lead to stronger justification for design decisions and test coverage for develo** ADSs. △ Less

Submitted 5 September, 2023; originally announced September 2023.

Comments: 25 pages, 9 figures, 2 tables

Journal ref: Safety Science, Volume 167, 2023, 106233

Scenario Extraction from a Large Real-World Dataset for the Assessment of Automated Vehicles

Authors: Detian Guo, Manuel Muñoz Sánchez, Erwin de Gelder, Tom P. J. van der Sande

Abstract: Many players in the automotive field support scenario-based assessment of automated vehicles (AVs), where individual traffic situations can be tested and, thus, facilitate concluding on the performance of AVs in different situations. Since a large number of different scenarios can occur in real-world traffic, the question is how to find a finite set of relevant scenarios. Scenarios extracted from… ▽ More Many players in the automotive field support scenario-based assessment of automated vehicles (AVs), where individual traffic situations can be tested and, thus, facilitate concluding on the performance of AVs in different situations. Since a large number of different scenarios can occur in real-world traffic, the question is how to find a finite set of relevant scenarios. Scenarios extracted from large real-world datasets represent real-world traffic since real driving data is used. Extracting scenarios, however, is challenging because (1) the scenarios to be tested should assess the AVs behave safely, which conflicts with the fact that the majority of the data contains scenarios that are not interesting from a safety perspective, and (2) extensive data processing is required, which hinders the utilization of large real-world datasets. In this work, we propose an approach for extracting scenarios from real-world driving data. The first step is data preprocessing to tackle the errors and noise in real-world data by reconstructing the data. The second step performs data tagging to label actors' activities, their interactions with each other and the environment. Finally, the scenarios are extracted by searching for combinations of tags. The proposed approach is evaluated using data simulated with CARLA and applied to a part of a large real-world driving dataset, i.e., the Waymo Open Motion Dataset (WOMD). The code and scenarios extracted from WOMD are open to the research community to facilitate the assessment of the automated driving functions in different scenarios. △ Less

Submitted 15 August, 2023; v1 submitted 13 June, 2023; originally announced June 2023.

Comments: 6 pages, accepted by ITSC 2023

PRISMA: A Novel Approach for Deriving Probabilistic Surrogate Safety Measures for Risk Evaluation

Authors: Erwin de Gelder, Kingsley Adjenughwure, Jeroen Manders, Ron Snijders, Jan-Pieter Paardekooper, Olaf Op den Camp, Arturo Tejada, Bart De Schutter

Abstract: Surrogate Safety Measures (SSMs) are used to express road safety in terms of the safety risk in traffic conflicts. Typically, SSMs rely on assumptions regarding the future evolution of traffic participant trajectories to generate a measure of risk, restricting their applicability to scenarios where these assumptions are valid. In response to this limitation, we present the novel Probabilistic RISk… ▽ More Surrogate Safety Measures (SSMs) are used to express road safety in terms of the safety risk in traffic conflicts. Typically, SSMs rely on assumptions regarding the future evolution of traffic participant trajectories to generate a measure of risk, restricting their applicability to scenarios where these assumptions are valid. In response to this limitation, we present the novel Probabilistic RISk Measure derivAtion (PRISMA) method. The objective of the PRISMA method is to derive SSMs that can be used to calculate in real time the probability of a specific event (e.g., a crash). The PRISMA method adopts a data-driven approach to predict the possible future traffic participant trajectories, thereby reducing the reliance on specific assumptions regarding these trajectories. Since the PRISMA is not bound to specific assumptions, the PRISMA method offers the ability to derive multiple SSMs for various scenarios. The occurrence probability of the specified event is based on simulations and combined with a regression model, this enables our derived SSMs to make real-time risk estimations. To illustrate the PRISMA method, an SSM is derived for risk evaluation during longitudinal traffic interactions. Since there is no known method to objectively estimate risk from first principles, i.e., there is no known risk ground truth, it is very difficult, if not impossible, to objectively compare the relative merits of two SSMs. Instead, we provide a method for benchmarking our derived SSM with respect to expected risk trends. The application of the benchmarking illustrates that the SSM matches the expected risk trends. Whereas the derived SSM shows the potential of the PRISMA method, future work involves applying the approach for other types of traffic conflicts, such as lateral traffic conflicts or interactions with vulnerable road users. △ Less

Submitted 4 September, 2023; v1 submitted 14 March, 2023; originally announced March 2023.

Comments: Accepted for publication in Accident Analysis & Prevention. 28 pages, 5 figures, 1 table

Scenario Parameter Generation Method and Scenario Representativeness Metric for Scenario-Based Assessment of Automated Vehicles

Authors: Erwin de Gelder, Jasper Hof, Eric Cator, Jan-Pieter Paardekooper, Olaf Op den Camp, Jeroen Ploeg, Bart De Schutter

Abstract: The development of assessment methods for the performance of Automated Vehicles (AVs) is essential to enable the deployment of automated driving technologies, due to the complex operational domain of AVs. One candidate is scenario-based assessment, in which test cases are derived from real-world road traffic scenarios obtained from driving data. Because of the high variety of the possible scenario… ▽ More The development of assessment methods for the performance of Automated Vehicles (AVs) is essential to enable the deployment of automated driving technologies, due to the complex operational domain of AVs. One candidate is scenario-based assessment, in which test cases are derived from real-world road traffic scenarios obtained from driving data. Because of the high variety of the possible scenarios, using only observed scenarios for the assessment is not sufficient. Therefore, methods for generating additional scenarios are necessary. Our contribution is twofold. First, we propose a method to determine the parameters that describe the scenarios to a sufficient degree without relying on strong assumptions on the parameters that characterize the scenarios. By estimating the probability density function (pdf) of these parameters, realistic parameter values can be generated. Second, we present the Scenario Representativeness (SR) metric based on the Wasserstein distance, which quantifies to what extent the scenarios with the generated parameter values are representative of real-world scenarios while covering the actual variety found in the real-world scenarios. A comparison of our proposed method with methods relying on assumptions of the scenario parametrization and pdf estimation shows that the proposed method can automatically determine the optimal scenario parametrization and pdf estimation. Furthermore, we demonstrate that our SR metric can be used to choose the (number of) parameters that best describe a scenario. The presented method is promising, because the parameterization and pdf estimation can directly be applied to already available importance sampling strategies for accelerating the evaluation of AVs. △ Less

Submitted 24 February, 2022; originally announced February 2022.

Comments: 14 pages, 11 figures, 4 tables, accepted for publication in IEEE Transactions on Intelligent Transportation Systems

Scenario-Based Safety Assessment Framework for Automated Vehicles

Authors: J. Ploeg, E. de Gelder, M. Slavík, E. Querner, T. Webster, N. de Boer

Abstract: Automated vehicles (AVs) are expected to increase traffic safety and traffic efficiency, among others by enabling flexible mobility-on-demand systems. This is particularly important in Singapore, being one of the world's most densely populated countries, which is why the Singaporean authorities are currently actively facilitating the deployment of AVs. As a consequence, however, the need arises fo… ▽ More Automated vehicles (AVs) are expected to increase traffic safety and traffic efficiency, among others by enabling flexible mobility-on-demand systems. This is particularly important in Singapore, being one of the world's most densely populated countries, which is why the Singaporean authorities are currently actively facilitating the deployment of AVs. As a consequence, however, the need arises for a formal AV road approval procedure. To this end, a safety assessment framework is proposed, which combines aspects of the standardized functional safety design methodology with a traffic scenario-based approach. The latter involves using driving data to extract AV-relevant traffic scenarios. The underlying approach is based on decomposition of scenarios into elementary events, subsequent scenario parametrization, and sampling of the estimated probability density functions of the scenario parameters to create test scenarios. The resulting test scenarios are subsequently employed for virtual testing in a simulation environment and physical testing on a proving ground and in real life. As a result, the proposed assessment pipeline thus provides statistically relevant and quantitative measures for the AV performance in a relatively short time frame due to the simulation-based approach. Ultimately, the proposed methodology provides authorities with a formal road approval procedure for AVs. In particular, the proposed methodology will support the Singaporean Land Transport Authority for road approval of AVs. △ Less

Submitted 17 December, 2021; originally announced December 2021.

Comments: 10 pages, 7 figures

Journal ref: 16th ITS Asia-Pacific Forum, 2018

Constrained Sampling from a Kernel Density Estimator to Generate Scenarios for the Assessment of Automated Vehicles

Authors: Erwin de Gelder, Eric Cator, Jan-Pieter Paardekooper, Olaf Op den Camp, Bart De Schutter

Abstract: The safety assessment of automated vehicles (AVs) is an important aspect of the development cycle of AVs. A scenario-based assessment approach is accepted by many players in the field as part of the complete safety assessment. A scenario is a representation of a situation on the road to which the AV needs to respond appropriately. One way to generate the required scenario-based test descriptions i… ▽ More The safety assessment of automated vehicles (AVs) is an important aspect of the development cycle of AVs. A scenario-based assessment approach is accepted by many players in the field as part of the complete safety assessment. A scenario is a representation of a situation on the road to which the AV needs to respond appropriately. One way to generate the required scenario-based test descriptions is to parameterize the scenarios and to draw these parameters from a probability density function (pdf). Because the shape of the pdf is unknown beforehand, assuming a functional form of the pdf and fitting the parameters to the data may lead to inaccurate fits. As an alternative, Kernel Density Estimation (KDE) is a promising candidate for estimating the underlying pdf, because it is flexible with the underlying distribution of the parameters. Drawing random samples from a pdf estimated with KDE is possible without the need of evaluating the actual pdf, which makes it suitable for drawing random samples for, e.g., Monte Carlo methods. Sampling from a KDE while the samples satisfy a linear equality constraint, however, has not been described in the literature, as far as the authors know. In this paper, we propose a method to sample from a pdf estimated using KDE, such that the samples satisfy a linear equality constraint. We also present an algorithm of our method in pseudo-code. The method can be used to generating scenarios that have, e.g., a predetermined starting speed or to generate different types of scenarios. This paper also shows that the method for sampling scenarios can be used in case a Singular Value Decomposition (SVD) is used to reduce the dimension of the parameter vectors. △ Less

Submitted 12 July, 2021; originally announced July 2021.

Comments: 6 pages, 3 figures, to be published in the proceedings of the IEEE Intelligent Vehicle Symposium Workshops (IV workshop)

Tagging Real-World Scenarios for the Assessment of Autonomous Vehicles

Authors: Erwin de Gelder, Olaf Op den Camp

Abstract: The development of Autonomous Vehicles (AVs) has made significant progress in the last years. An essential aspect in the development of AVs is the assessment of quality and performance aspects of the AVs, such as safety, comfort, and efficiency. Among other methods, a scenario-based approach has been proposed. With scenario-based testing, the AV is subjected to a collection of scenarios that repre… ▽ More The development of Autonomous Vehicles (AVs) has made significant progress in the last years. An essential aspect in the development of AVs is the assessment of quality and performance aspects of the AVs, such as safety, comfort, and efficiency. Among other methods, a scenario-based approach has been proposed. With scenario-based testing, the AV is subjected to a collection of scenarios that represent real-world situations. The collection of scenarios needs to cover the variety of what an AV can encounter in real traffic. As a result, many different scenarios are considered, that are grouped into so-called scenario categories. We propose a method for defining the scenario categories using a system of tags, where each tag describes a particular characteristic of a scenario category. There is a balance between having generic scenario categories - very specific set of scenarios, while for another system one might be interested in a set of scenarios with a high variety. To accommodate this, tags are structured in trees. The different layers of the trees can be regarded as different abstraction levels. Next to presenting the method for describing scenario categories using tags, we will illustrate the method by showing applicable trees of tags using concrete examples in the Singapore traffic system. Trees of tags are shown for the vehicle under test, the dynamic environment (e.g., the other road users), the static environment (e.g., the road layout), and the environmental conditions (weather and lighting conditions). Few examples are presented to illustrate the proposed method for defining the scenario categories using tags. △ Less

Submitted 2 December, 2020; originally announced December 2020.

Comments: Accepted for publication in the proceedings of the FISITA 2020 conference

Report number: F2020-PIF-048

Procedure for the Safety Assessment of an Autonomous Vehicle Using Real-World Scenarios

Authors: Erwin de Gelder, Olaf Op den Camp

Abstract: The development of Autonomous Vehicles (AVs) has made significant progress in the last years. An important aspect in the development of AVs is the assessment of their safety. New approaches need to be worked out. Among these, real-world scenario-based assessment is widely supported by many players in the automotive field. Scenario-based assessment allows for using virtual simulation tools in addit… ▽ More The development of Autonomous Vehicles (AVs) has made significant progress in the last years. An important aspect in the development of AVs is the assessment of their safety. New approaches need to be worked out. Among these, real-world scenario-based assessment is widely supported by many players in the automotive field. Scenario-based assessment allows for using virtual simulation tools in addition to physical tests, such as on a test track, proving ground, or public road. We propose a procedure for real-world scenario-based road-approval assessment considering three stakeholders: the applicant, the assessor, and the (road or vehicle) authority. The challenges are as follows. Firstly, the tests need to be tailored to the operational design domain (ODD) and dynamic driving task (DDT) description of the AV. Secondly, it is assumed that the applicant does not want to disclose all of the detailed test results because of proprietary or confidential information contained in these results. Thirdly, due to the complex ODD and DDT, many test scenarios are required to obtain sufficient confidence in the assessment of the AV. Consequently, it is assumed that due to limited resources, it is infeasible for the assessor to conduct all (physical) tests. We propose a systematic approach for determining the tests that are based on the requirements set by the authority and the AV's ODD and DDT description, such that the tests are tailored to the applicable ODD and DDT. Each test comes with metrics that enables the applicant to provide a performance rating of the AV for each of the tests. By only providing a performance rating for each test, the applicant does not need to disclose the details of the test results. In our proposed procedure, the assessor only conducts a limited number of tests. The main purpose of these tests is to verify the fidelity of the results provided by the applicant. △ Less

Submitted 2 December, 2020; v1 submitted 1 December, 2020; originally announced December 2020.

Comments: Accepted for publication in the proceedings of the FISITA 2020 conference

Report number: F2020-VES-014

Real-World Scenario Mining for the Assessment of Automated Vehicles

Authors: Erwin de Gelder, Jeroen Manders, Corrado Grappiolo, Jan-Pieter Paardekooper, Olaf Op den Camp, Bart De Schutter

Abstract: Scenario-based methods for the assessment of Automated Vehicles (AVs) are widely supported by many players in the automotive field. Scenarios captured from real-world data can be used to define the scenarios for the assessment and to estimate their relevance. Therefore, different techniques are proposed for capturing scenarios from real-world data. In this paper, we propose a new method to capture… ▽ More Scenario-based methods for the assessment of Automated Vehicles (AVs) are widely supported by many players in the automotive field. Scenarios captured from real-world data can be used to define the scenarios for the assessment and to estimate their relevance. Therefore, different techniques are proposed for capturing scenarios from real-world data. In this paper, we propose a new method to capture scenarios from real-world data using a two-step approach. The first step consists in automatically labeling the data with tags. Second, we mine the scenarios, represented by a combination of tags, based on the labeled tags. One of the benefits of our approach is that the tags can be used to identify characteristics of a scenario that are shared among different type of scenarios. In this way, these characteristics need to be identified only once. Furthermore, the method is not specific for one type of scenario and, therefore, it can be applied to a large variety of scenarios. We provide two examples to illustrate the method. This paper is concluded with some promising future possibilities for our approach, such as automatic generation of scenarios for the assessment of automated vehicles. △ Less

Submitted 12 July, 2021; v1 submitted 31 May, 2020; originally announced June 2020.

Comments: 8 pages, 8 figures, 4 tables

Journal ref: Proceedings of the IEEE 2020 Intelligent Transportation Systems Conference (ITSC)

Towards an Ontology for Scenario Definition for the Assessment of Automated Vehicles: An Object-Oriented Framework

Authors: E. de Gelder, J. -P. Paardekooper, A. Khabbaz Saberi, H. Elrofai, O. Op den Camp., S. Kraines, J. Ploeg, B. De Schutter

Abstract: The development of new assessment methods for the performance of automated vehicles is essential to enable the deployment of automated driving technologies, due to the complex operational domain of automated vehicles. One contributing method is scenario-based assessment in which test cases are derived from real-world road traffic scenarios obtained from driving data. Given the complexity of the re… ▽ More The development of new assessment methods for the performance of automated vehicles is essential to enable the deployment of automated driving technologies, due to the complex operational domain of automated vehicles. One contributing method is scenario-based assessment in which test cases are derived from real-world road traffic scenarios obtained from driving data. Given the complexity of the reality that is being modeled in these scenarios, it is a challenge to define a structure for capturing these scenarios. An intensional definition that provides a set of characteristics that are deemed to be both necessary and sufficient to qualify as a scenario assures that the scenarios constructed are both complete and intercomparable. In this article, we develop a comprehensive and operable definition of the notion of scenario while considering existing definitions in the literature. This is achieved by proposing an object-oriented framework in which scenarios and their building blocks are defined as classes of objects having attributes, methods, and relationships with other objects. The object-oriented approach promotes clarity, modularity, reusability, and encapsulation of the objects. We provide definitions and justifications of each of the terms. Furthermore, the framework is used to translate the terms in a coding language that is publicly available. △ Less

Submitted 13 December, 2021; v1 submitted 30 January, 2020; originally announced January 2020.

Comments: 16 pages, 7 figures, 1 table