Showing 1–23 of 23 results for author: Özgüner, Ü

Using Collision Momentum in Deep Reinforcement Learning Based Adversarial Pedestrian Modeling

Authors: Dianwei Chen, Ekim Yurtsever, Keith Redmill, Umit Ozguner

Abstract: Recent research in pedestrian simulation often aims to develop realistic behaviors in various situations, but it is challenging for existing algorithms to generate behaviors that identify weaknesses in automated vehicles' performance in extreme and unlikely scenarios and edge cases. To address this, specialized pedestrian behavior algorithms are needed. Current research focuses on realistic trajec… ▽ More Recent research in pedestrian simulation often aims to develop realistic behaviors in various situations, but it is challenging for existing algorithms to generate behaviors that identify weaknesses in automated vehicles' performance in extreme and unlikely scenarios and edge cases. To address this, specialized pedestrian behavior algorithms are needed. Current research focuses on realistic trajectories using social force models and reinforcement learning based models. However, we propose a reinforcement learning algorithm that specifically targets collisions and better uncovers unique failure modes of automated vehicle controllers. Our algorithm is efficient and generates more severe collisions, allowing for the identification and correction of weaknesses in autonomous driving algorithms in complex and varied scenarios. △ Less

Submitted 12 June, 2023; originally announced June 2023.

A Finite-Sampling, Operational Domain Specific, and Provably Unbiased Connected and Automated Vehicle Safety Metric

Authors: Bowen Weng, Linda Capito, Umit Ozguner, Keith Redmill

Abstract: A connected and automated vehicle safety metric determines the performance of a subject vehicle (SV) by analyzing the data involving the interactions among the SV and other dynamic road users and environmental features. When the data set contains only a finite set of samples collected from the naturalistic mixed-traffic driving environment, a metric is expected to generalize the safety assessment… ▽ More A connected and automated vehicle safety metric determines the performance of a subject vehicle (SV) by analyzing the data involving the interactions among the SV and other dynamic road users and environmental features. When the data set contains only a finite set of samples collected from the naturalistic mixed-traffic driving environment, a metric is expected to generalize the safety assessment outcome from the observed finite samples to the unobserved cases by specifying in what domain the SV is expected to be safe and how safe the SV is, statistically, in that domain. However, to the best of our knowledge, none of the existing safety metrics are able to justify the above properties with an operational domain specific, guaranteed complete, and provably unbiased safety evaluation outcome. In this paper, we propose a novel safety metric that involves the $α$-shape and the $ε$-almost robustly forward invariant set to characterize the SV's almost safe operable domain and the probability for the SV to remain inside the safe domain indefinitely, respectively. The empirical performance of the proposed method is demonstrated in several different operational design domains through a series of cases covering a variety of fidelity levels (real-world and simulators), driving environments (highway, urban, and intersections), road users (car, truck, and pedestrian), and SV driving behaviors (human driver and self driving algorithms). △ Less

Submitted 2 February, 2022; v1 submitted 15 November, 2021; originally announced November 2021.

A Formal Characterization of Black-Box System Safety Performance with Scenario Sampling

Authors: Bowen Weng, Linda Capito, Umit Ozguner, Keith Redmill

Abstract: A typical scenario-based evaluation framework seeks to characterize a black-box system's safety performance (e.g., failure rate) through repeatedly sampling initialization configurations (scenario sampling) and executing a certain test policy for scenario propagation (scenario testing) with the black-box system involved as the test subject. In this letter, we first present a novel safety evaluatio… ▽ More A typical scenario-based evaluation framework seeks to characterize a black-box system's safety performance (e.g., failure rate) through repeatedly sampling initialization configurations (scenario sampling) and executing a certain test policy for scenario propagation (scenario testing) with the black-box system involved as the test subject. In this letter, we first present a novel safety evaluation criterion that seeks to characterize the actual operational domain within which the test subject would remain safe indefinitely with high probability. By formulating the black-box testing scenario as a dynamic system, we show that the presented problem is equivalent to finding a certain "almost" robustly forward invariant set for the given system. Second, for an arbitrary scenario testing strategy, we propose a scenario sampling algorithm that is provably asymptotically optimal in obtaining the safe invariant set with arbitrarily high accuracy. Moreover, as one considers different testing strategies (e.g., biased sampling of safety-critical cases), we show that the proposed algorithm still converges to the unbiased approximation of the safety characterization outcome if the scenario testing satisfies a certain condition. Finally, the effectiveness of the presented scenario sampling algorithms and various theoretical properties are demonstrated in a case study of the safety evaluation of a control barrier function-based mobile robot collision avoidance system. △ Less

Submitted 5 October, 2021; originally announced October 2021.

Comments: A shorter version of this manuscript has been accepted to be published at IEEE Robotics and Automation Letters (RA-L)

Journal ref: IEEE Robotics and Automation Letters, vol. 7, no. 1, pp. 199-206, Jan. 2022

Pedestrian Emergence Estimation and Occlusion-Aware Risk Assessment for Urban Autonomous Driving

Authors: Mert Koc, Ekim Yurtsever, Keith Redmill, Umit Ozguner

Abstract: Avoiding unseen or partially occluded vulnerable road users (VRUs) is a major challenge for fully autonomous driving in urban scenes. However, occlusion-aware risk assessment systems have not been widely studied. Here, we propose a pedestrian emergence estimation and occlusion-aware risk assessment system for urban autonomous driving. First, the proposed system utilizes available contextual inform… ▽ More Avoiding unseen or partially occluded vulnerable road users (VRUs) is a major challenge for fully autonomous driving in urban scenes. However, occlusion-aware risk assessment systems have not been widely studied. Here, we propose a pedestrian emergence estimation and occlusion-aware risk assessment system for urban autonomous driving. First, the proposed system utilizes available contextual information, such as visible cars and pedestrians, to estimate pedestrian emergence probabilities in occluded regions. These probabilities are then used in a risk assessment framework, and incorporated into a longitudinal motion controller. The proposed controller is tested against several baseline controllers that recapitulate some commonly observed driving styles. The simulated test scenarios include randomly placed parked cars and pedestrians, most of whom are occluded from the ego vehicle's view and emerges randomly. The proposed controller outperformed the baselines in terms of safety and comfort measures. △ Less

Submitted 5 July, 2021; originally announced July 2021.

Comments: Accepted to ITSC2021

Towards Guaranteed Safety Assurance of Automated Driving Systems with Scenario Sampling: An Invariant Set Perspective (Extended Version)

Authors: Bowen Weng, Linda Capito, Umit Ozguner, Keith Redmill

Abstract: How many scenarios are sufficient to validate the safe Operational Design Domain (ODD) of an Automated Driving System (ADS) equipped vehicle? Is a more significant number of sampled scenarios guaranteeing a more accurate safety assessment of the ADS? Despite the various empirical success of ADS safety evaluation with scenario sampling in practice, some of the fundamental properties are largely unk… ▽ More How many scenarios are sufficient to validate the safe Operational Design Domain (ODD) of an Automated Driving System (ADS) equipped vehicle? Is a more significant number of sampled scenarios guaranteeing a more accurate safety assessment of the ADS? Despite the various empirical success of ADS safety evaluation with scenario sampling in practice, some of the fundamental properties are largely unknown. This paper seeks to remedy this gap by formulating and tackling the scenario sampling safety assurance problem from a set invariance perspective. First, a novel conceptual equivalence is drawn between the scenario sampling safety assurance problem and the data-driven robustly controlled forward invariant set validation and quantification problem. This paper then provides a series of resolution complete and probabilistic complete solutions with finite-sampling analyses for the safety validation problem that authenticates a given ODD. On the other hand, the quantification problem escalates the validation challenge and starts looking for a safe sub-domain of a particular property. This inspires various algorithms that are provably probabilistic incomplete, probabilistic complete but sub-optimal, and asymptotically optimal. Finally, the proposed asymptotically optimal scenario sampling safety quantification algorithm is also empirically demonstrated through simulation experiments. △ Less

Submitted 29 September, 2021; v1 submitted 19 April, 2021; originally announced April 2021.

Comments: A shorter version of this manuscript has been accepted by the IEEE Transactions on Intelligent Vehicles

Predicting Pedestrian Crossing Intention with Feature Fusion and Spatio-Temporal Attention

Authors: Dongfang Yang, Haolin Zhang, Ekim Yurtsever, Keith Redmill, Ümit Özgüner

Abstract: Predicting vulnerable road user behavior is an essential prerequisite for deploying Automated Driving Systems (ADS) in the real-world. Pedestrian crossing intention should be recognized in real-time, especially for urban driving. Recent works have shown the potential of using vision-based deep neural network models for this task. However, these models are not robust and certain issues still need t… ▽ More Predicting vulnerable road user behavior is an essential prerequisite for deploying Automated Driving Systems (ADS) in the real-world. Pedestrian crossing intention should be recognized in real-time, especially for urban driving. Recent works have shown the potential of using vision-based deep neural network models for this task. However, these models are not robust and certain issues still need to be resolved. First, the global spatio-temproal context that accounts for the interaction between the target pedestrian and the scene has not been properly utilized. Second, the optimum strategy for fusing different sensor data has not been thoroughly investigated. This work addresses the above limitations by introducing a novel neural network architecture to fuse inherently different spatio-temporal features for pedestrian crossing intention prediction. We fuse different phenomena such as sequences of RGB imagery, semantic segmentation masks, and ego-vehicle speed in an optimum way using attention mechanisms and a stack of recurrent neural networks. The optimum architecture was obtained through exhaustive ablation and comparison studies. Extensive comparative experiments on the JAAD pedestrian action prediction benchmark demonstrate the effectiveness of the proposed method, where state-of-the-art performance was achieved. Our code is open-source and publicly available. △ Less

Submitted 13 October, 2021; v1 submitted 12 April, 2021; originally announced April 2021.

Comments: Submitted to IEEE Transactions on Intelligent Vehicles

On the Generalizability of Motion Models for Road Users in Heterogeneous Shared Traffic Spaces

Authors: Fatema T. Johora, Dongfang Yang, Jörg P. Müller, Ümit Özgüner

Abstract: Modeling mixed-traffic motion and interactions is crucial to assess safety, efficiency, and feasibility of future urban areas. The lack of traffic regulations, diverse transport modes, and the dynamic nature of mixed-traffic zones like shared spaces make realistic modeling of such environments challenging. This paper focuses on the generalizability of the motion model, i.e., its ability to generat… ▽ More Modeling mixed-traffic motion and interactions is crucial to assess safety, efficiency, and feasibility of future urban areas. The lack of traffic regulations, diverse transport modes, and the dynamic nature of mixed-traffic zones like shared spaces make realistic modeling of such environments challenging. This paper focuses on the generalizability of the motion model, i.e., its ability to generate realistic behavior in different environmental settings, an aspect which is lacking in existing works. Specifically, our first contribution is a novel and systematic process of formulating general motion models and application of this process is to extend our Game-Theoretic Social Force Model (GSFM) towards a general model for generating a large variety of motion behaviors of pedestrians and cars from different shared spaces. Our second contribution is to consider different motion patterns of pedestrians by calibrating motion-related features of individual pedestrian and clustering them into groups. We analyze two clustering approaches. The calibration and evaluation of our model are performed on three different shared space data sets. The results indicate that our model can realistically simulate a wide range of motion behaviors and interaction scenarios, and that adding different motion patterns of pedestrians into our model improves its performance. △ Less

Submitted 18 January, 2021; originally announced January 2021.

Sub-Goal Social Force Model for Collective Pedestrian Motion Under Vehicle Influence

Authors: Dongfang Yang, Fatema T. Johora, Keith A. Redmill, Ümit Özgüner, Jörg P. Müller

Abstract: In mixed traffic scenarios, a certain number of pedestrians might coexist in a small area while interacting with vehicles. In this situation, every pedestrian must simultaneously react to the surrounding pedestrians and vehicles. Analytical modeling of such collective pedestrian motion can benefit intelligent transportation practices like shared space design and urban autonomous driving. This work… ▽ More In mixed traffic scenarios, a certain number of pedestrians might coexist in a small area while interacting with vehicles. In this situation, every pedestrian must simultaneously react to the surrounding pedestrians and vehicles. Analytical modeling of such collective pedestrian motion can benefit intelligent transportation practices like shared space design and urban autonomous driving. This work proposed the sub-goal social force model (SG-SFM) to describe the collective pedestrian motion under vehicle influence. The proposed model introduced a new design of vehicle influence on pedestrian motion, which was smoothly combined with the influence of surrounding pedestrians using the sub-goal concept. This model aims to describe generalized pedestrian motion, i.e., it is applicable to various vehicle-pedestrian interaction patterns. The generalization was verified by both quantitative and qualitative evaluation. The quantitative evaluation was conducted to reproduce pedestrian motion in three different datasets, HBS, CITR, and DUT. It also compared two different ways of calibrating the model parameters. The qualitative evaluation examined the simulation of collective pedestrian motion in a series of fundamental vehicle-pedestrian interaction scenarios. The above evaluation results demonstrated the effectiveness of the proposed model. △ Less

Submitted 10 January, 2021; originally announced January 2021.

Comments: submitted to IEEE Transactions on Intelligent Transportation Systems

Faraway-Frustum: Dealing with Lidar Sparsity for 3D Object Detection using Fusion

Authors: Haolin Zhang, Dongfang Yang, Ekim Yurtsever, Keith A. Redmill, Ümit Özgüner

Abstract: Learned pointcloud representations do not generalize well with an increase in distance to the sensor. For example, at a range greater than 60 meters, the sparsity of lidar pointclouds reaches to a point where even humans cannot discern object shapes from each other. However, this distance should not be considered very far for fast-moving vehicles: A vehicle can traverse 60 meters under two seconds… ▽ More Learned pointcloud representations do not generalize well with an increase in distance to the sensor. For example, at a range greater than 60 meters, the sparsity of lidar pointclouds reaches to a point where even humans cannot discern object shapes from each other. However, this distance should not be considered very far for fast-moving vehicles: A vehicle can traverse 60 meters under two seconds while moving at 70 mph. For safe and robust driving automation, acute 3D object detection at these ranges is indispensable. Against this backdrop, we introduce faraway-frustum: a novel fusion strategy for detecting faraway objects. The main strategy is to depend solely on the 2D vision for recognizing object class, as object shape does not change drastically with an increase in depth, and use pointcloud data for object localization in the 3D space for faraway objects. For closer objects, we use learned pointcloud representations instead, following state-of-the-art. This strategy alleviates the main shortcoming of object detection with learned pointcloud representations. Experiments on the KITTI dataset demonstrate that our method outperforms state-of-the-art by a considerable margin for faraway object detection in bird's-eye-view and 3D. Our code is open-source and publicly available: https://github.com/dongfang-steven-yang/faraway-frustum. △ Less

Submitted 27 March, 2021; v1 submitted 2 November, 2020; originally announced November 2020.

A Modeled Approach for Online Adversarial Test of Operational Vehicle Safety (extended version)

Authors: Linda Capito, Bowen Weng, Umit Ozguner, Keith Redmill

Abstract: The scenario-based testing of operational vehicle safety presents a set of principal other vehicle (POV) trajectories that seek to force the subject vehicle (SV) into a certain safety-critical situation. Current scenarios are mostly (i) statistics-driven: inspired by human driver crash data, (ii) deterministic: POV trajectories are pre-determined and are independent of SV responses, and (iii) over… ▽ More The scenario-based testing of operational vehicle safety presents a set of principal other vehicle (POV) trajectories that seek to force the subject vehicle (SV) into a certain safety-critical situation. Current scenarios are mostly (i) statistics-driven: inspired by human driver crash data, (ii) deterministic: POV trajectories are pre-determined and are independent of SV responses, and (iii) overly simplified: defined over a finite set of actions performed at the abstracted motion planning level. Such scenario-based testing (i) lacks severity guarantees, (ii) has predefined maneuvers making it easy for an SV with intelligent driving policies to game the test, and (iii) is inefficient in producing safety-critical instances with limited and expensive testing effort. We propose a model-driven online feedback control policy for multiple POVs which propagates efficient adversarial trajectories while respecting traffic rules and other concerns formulated as an admissible state-action space. The approach is formulated in an anchor-template hierarchy structure, with the template model planning inducing a theoretical SV capturing guarantee under standard assumptions. The planned adversarial trajectory is then tracked by a lower-level controller applied to the full-system or the anchor model. The effectiveness of the methodology is illustrated through various simulated examples with the SV controlled by either parameterized self-driving policies or human drivers. △ Less

Submitted 20 May, 2021; v1 submitted 25 September, 2020; originally announced September 2020.

Comments: This document is the extended version of our paper accepted to the 2021 IEEE American Control Conference

A Vision-based Social Distancing and Critical Density Detection System for COVID-19

Authors: Dongfang Yang, Ekim Yurtsever, Vishnu Renganathan, Keith A. Redmill, Ümit Özgüner

Abstract: Social distancing has been proven as an effective measure against the spread of the infectious COronaVIrus Disease 2019 (COVID-19). However, individuals are not used to tracking the required 6-feet (2-meters) distance between themselves and their surroundings. An active surveillance system capable of detecting distances between individuals and warning them can slow down the spread of the deadly di… ▽ More Social distancing has been proven as an effective measure against the spread of the infectious COronaVIrus Disease 2019 (COVID-19). However, individuals are not used to tracking the required 6-feet (2-meters) distance between themselves and their surroundings. An active surveillance system capable of detecting distances between individuals and warning them can slow down the spread of the deadly disease. Furthermore, measuring social density in a region of interest (ROI) and modulating inflow can decrease social distancing violation occurrence chance. On the other hand, recording data and labeling individuals who do not follow the measures will breach individuals' rights in free-societies. Here we propose an Artificial Intelligence (AI) based real-time social distancing detection and warning system considering four important ethical factors: (1) the system should never record/cache data, (2) the warnings should not target the individuals, (3) no human supervisor should be in the detection/warning loop, and (4) the code should be open-source and accessible to the public. Against this backdrop, we propose using a monocular camera and deep learning-based real-time object detectors to measure social distancing. If a violation is detected, a non-intrusive audio-visual warning signal is emitted without targeting the individual who breached the social distancing measure. Also, if the social density is over a critical value, the system sends a control signal to modulate inflow into the ROI. We tested the proposed method across real-world datasets to measure its generality and performance. The proposed method is ready for deployment, and our code is open-sourced. △ Less

Submitted 8 July, 2020; v1 submitted 7 July, 2020; originally announced July 2020.

An online evolving framework for advancing reinforcement-learning based automated vehicle control

Authors: Teawon Han, Subramanya Nageshrao, Dimitar P. Filev, Umit Ozguner

Abstract: In this paper, an online evolving framework is proposed to detect and revise a controller's imperfect decision-making in advance. The framework consists of three modules: the evolving Finite State Machine (e-FSM), action-reviser, and controller modules. The e-FSM module evolves a stochastic model (e.g., Discrete-Time Markov Chain) from scratch by determining new states and identifying transition p… ▽ More In this paper, an online evolving framework is proposed to detect and revise a controller's imperfect decision-making in advance. The framework consists of three modules: the evolving Finite State Machine (e-FSM), action-reviser, and controller modules. The e-FSM module evolves a stochastic model (e.g., Discrete-Time Markov Chain) from scratch by determining new states and identifying transition probabilities repeatedly. With the latest stochastic model and given criteria, the action-reviser module checks validity of the controller's chosen action by predicting future states. Then, if the chosen action is not appropriate, another action is inspected and selected. In order to show the advantage of the proposed framework, the Deep Deterministic Policy Gradient (DDPG) w/ and w/o the online evolving framework are applied to control an ego-vehicle in the car-following scenario where control criteria are set by speed and safety. Experimental results show that inappropriate actions chosen by the DDPG controller are detected and revised appropriately through our proposed framework, resulting in no control failures after a few iterations. △ Less

Submitted 16 June, 2020; v1 submitted 14 June, 2020; originally announced June 2020.

Comments: Accepted in IFAC 2020 WC

Optical Flow based Visual Potential Field for Autonomous Driving

Authors: Linda Capito, Keith Redmill, Umit Ozguner

Abstract: Monocular vision-based navigation for automated driving is a challenging task due to the lack of enough information to compute temporal relationships among objects on the road. Optical flow is an option to obtain temporal information from monocular camera images and has been used widely with the purpose of identifying objects and their relative motion. This work proposes to generate an artificial… ▽ More Monocular vision-based navigation for automated driving is a challenging task due to the lack of enough information to compute temporal relationships among objects on the road. Optical flow is an option to obtain temporal information from monocular camera images and has been used widely with the purpose of identifying objects and their relative motion. This work proposes to generate an artificial potential field, i.e. visual potential field, from a sequence of images using sparse optical flow, which is used together with a gradient tracking sliding mode controller to navigate the vehicle to destination without collision with obstacles. The angular reference for the vehicle is computed online. This work considers that the vehicle does not require to have a priori information from the map or obstacles to navigate successfully. The proposed technique is tested both in synthetic and real images. △ Less

Submitted 30 May, 2020; originally announced June 2020.

Comments: This paper was accepted in the 31st IEEE Intelligent Vehicles Symposium (IV2020)

A Multi-State Social Force Based Framework for Vehicle-Pedestrian Interaction in Uncontrolled Pedestrian Crossing Scenarios

Authors: Dongfang Yang, Keith Redmill, Umit Ozguner

Abstract: Vehicle-pedestrian interaction (VPI) is one of the most challenging tasks for automated driving systems. The design of driving strategies for such systems usually starts with verifying VPI in simulation. This work proposed an improved framework for the study of VPI in uncontrolled pedestrian crossing scenarios. The framework admits the mutual effect between the pedestrian and the vehicle. A multi-… ▽ More Vehicle-pedestrian interaction (VPI) is one of the most challenging tasks for automated driving systems. The design of driving strategies for such systems usually starts with verifying VPI in simulation. This work proposed an improved framework for the study of VPI in uncontrolled pedestrian crossing scenarios. The framework admits the mutual effect between the pedestrian and the vehicle. A multi-state social force based pedestrian motion model was designed to describe the microscopic motion of the pedestrian crossing behavior. The pedestrian model considers major interaction factors such as the accepted gap of the pedestrian's decision on when to start crossing, the desired speed of the pedestrian, and the effect of the vehicle on the pedestrian while the pedestrian is crossing the road. Vehicle driving strategies focus on the longitudinal motion control, for which the feedback obstacle avoidance control and the model predictive control were tested and compared in the framework. The simulation results verified that the proposed framework can generate a variety of VPI scenarios, consisting of either the pedestrian yielding to the vehicle or the vehicle yielding to the pedestrian. The framework can be easily extended to apply different approaches to the VPI problems. △ Less

Submitted 15 May, 2020; originally announced May 2020.

Comments: This paper was accepted in the 31st IEEE Intelligent Vehicles Symposium (IV2020)

Integrating Deep Reinforcement Learning with Model-based Path Planners for Automated Driving

Authors: Ekim Yurtsever, Linda Capito, Keith Redmill, Umit Ozguner

Abstract: Automated driving in urban settings is challenging. Human participant behavior is difficult to model, and conventional, rule-based Automated Driving Systems (ADSs) tend to fail when they face unmodeled dynamics. On the other hand, the more recent, end-to-end Deep Reinforcement Learning (DRL) based model-free ADSs have shown promising results. However, pure learning-based approaches lack the hard-c… ▽ More Automated driving in urban settings is challenging. Human participant behavior is difficult to model, and conventional, rule-based Automated Driving Systems (ADSs) tend to fail when they face unmodeled dynamics. On the other hand, the more recent, end-to-end Deep Reinforcement Learning (DRL) based model-free ADSs have shown promising results. However, pure learning-based approaches lack the hard-coded safety measures of model-based controllers. Here we propose a hybrid approach for integrating a path planning pipe into a vision based DRL framework to alleviate the shortcomings of both worlds. In summary, the DRL agent is trained to follow the path planner's waypoints as close as possible. The agent learns this policy by interacting with the environment. The reward function contains two major terms: the penalty of straying away from the path planner and the penalty of having a collision. The latter has precedence in the form of having a significantly greater numerical value. Experimental results show that the proposed method can plan its path and navigate between randomly chosen origin-destination points in CARLA, a dynamic urban simulation environment. Our code is open-source and available online. △ Less

Submitted 19 May, 2020; v1 submitted 2 February, 2020; originally announced February 2020.

Comments: 6 pages, 5 figures. Accepted for IEEE Intelligent Vehicles Symposium 2020

An Online Evolving Framework for Modeling the Safe Autonomous Vehicle Control System via Online Recognition of Latent Risks

Authors: Teawon Han, Dimitar Filev, Umit Ozguner

Abstract: An online evolving framework is proposed to support modeling the safe Automated Vehicle (AV) control system by making the controller able to recognize unexpected situations and react appropriately by choosing a better action. Within the framework, the evolving Finite State Machine (e-FSM), which is an online model able to (1) determine states uniquely as needed, (2) recognize states, and (3) ident… ▽ More An online evolving framework is proposed to support modeling the safe Automated Vehicle (AV) control system by making the controller able to recognize unexpected situations and react appropriately by choosing a better action. Within the framework, the evolving Finite State Machine (e-FSM), which is an online model able to (1) determine states uniquely as needed, (2) recognize states, and (3) identify state-transitions, is introduced. In this study, the e-FSM's capabilities are explained and illustrated by simulating a simple car-following scenario. As a vehicle controller, the Intelligent Driver Model (IDM) is implemented, and different sets of IDM parameters are assigned to the following vehicle for simulating various situations (including the collision). While simulating the car-following scenario, e-FSM recognizes and determines the states and identifies the transition matrices by suggested methods. To verify if e-FSM can recognize and determine states uniquely, we analyze whether the same state is recognized under the identical situation. The difference between probability distributions of predicted and recognized states is measured by the Jensen-Shannon divergence (JSD) method to validate the accuracy of identified transition-matrices. As shown in the results, the Dead-End state which has latent-risk of the collision is uniquely determined and consistently recognized. Also, the probability distributions of the predicted state are significantly similar to the recognized state, declaring that the state-transitions are precisely identified. △ Less

Submitted 28 August, 2019; originally announced August 2019.

Comments: Under review in the Transportation Research Record: Journal of the Transportation Research Board

Driving Intention Recognition and Lane Change Prediction on the Highway

Authors: Teawon Han, Junbo **g, Umit Ozguner

Abstract: This paper proposes a framework to recognize driving intentions and to predict driving behaviors of lane changing on the highway by using externally sensable traffic data from the host-vehicle. The framework consists of a driving characteristic estimator and a driving behavior predictor. A driver's implicit driving characteristic information is uniquely determined and detected by proposed the onli… ▽ More This paper proposes a framework to recognize driving intentions and to predict driving behaviors of lane changing on the highway by using externally sensable traffic data from the host-vehicle. The framework consists of a driving characteristic estimator and a driving behavior predictor. A driver's implicit driving characteristic information is uniquely determined and detected by proposed the online-estimator. Neural-network based behavior predictor is developed and validated by testing with the real naturalistic traffic data from Next Generation Simulation (NGSIM), which demonstrates the effectiveness in identifying the driving characteristics and transforming into accurate behavior prediction in real-world traffic situations. △ Less

Submitted 28 August, 2019; originally announced August 2019.

Comments: This paper was accepted into the 30th IEEE Intelligent Vehicles Symposium. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses

Combining Social Force Model with Model Predictive Control for Vehicle's Longitudinal Speed Regulation in Pedestrian-Dense Scenarios

Authors: Dongfang Yang, Ümit Özgüner

Abstract: In pedestrian-dense traffic scenarios, an autonomous vehicle may have to safely drive through a crowd of pedestrians while the vehicle tries to keep the desired speed as much as possible. This requires a model that can predict the motion of crowd pedestrians and a method for the vehicle to predictively adjust its speed. In this study, the model-based predictive control (MPC) was combined with a so… ▽ More In pedestrian-dense traffic scenarios, an autonomous vehicle may have to safely drive through a crowd of pedestrians while the vehicle tries to keep the desired speed as much as possible. This requires a model that can predict the motion of crowd pedestrians and a method for the vehicle to predictively adjust its speed. In this study, the model-based predictive control (MPC) was combined with a social-force based vehicle-crowd interaction (VCI) model to regulate the longitudinal speed of the autonomous vehicle. The predictive feature of the VCI model can be precisely utilized by the MPC. A criterion for simultaneously guaranteeing pedestrian safety and kee** the desired speed was designed, and consequently, the MPC was formulated as a standard quadratic programming (QP) problem, which can be easily solved by standard QP toolbox. The proposed approach was compared with the traditional proportional-integral-derivative (PID) control approach for regulating longitudinal speed. Scenarios of different pedestrian density were evaluated in simulation. The results demonstrated the merits of the proposed method to address this type of problem. It also shows the potential of extending the method to address more complex vehicle-pedestrian interaction situations. △ Less

Submitted 11 July, 2019; originally announced July 2019.

Comments: This paper has been accepted and presented on The 8th Biennial Workshop on Digital Signal Processing for In-Vehicle Systems, Oct 7-9, 2018, at Nagoya University, Japan

Top-view Trajectories: A Pedestrian Dataset of Vehicle-Crowd Interaction from Controlled Experiments and Crowded Campus

Authors: Dongfang Yang, Linhui Li, Keith Redmill, Ümit Özgüner

Abstract: Predicting the collective motion of a group of pedestrians (a crowd) under the vehicle influence is essential for the development of autonomous vehicles to deal with mixed urban scenarios where interpersonal interaction and vehicle-crowd interaction (VCI) are significant. This usually requires a model that can describe individual pedestrian motion under the influence of nearby pedestrians and the… ▽ More Predicting the collective motion of a group of pedestrians (a crowd) under the vehicle influence is essential for the development of autonomous vehicles to deal with mixed urban scenarios where interpersonal interaction and vehicle-crowd interaction (VCI) are significant. This usually requires a model that can describe individual pedestrian motion under the influence of nearby pedestrians and the vehicle. This study proposed two pedestrian trajectory datasets, CITR dataset and DUT dataset, so that the pedestrian motion models can be further calibrated and verified, especially when vehicle influence on pedestrians plays an important role. CITR dataset consists of experimentally designed fundamental VCI scenarios (front, back, and lateral VCIs) and provides unique ID for each pedestrian, which is suitable for exploring a specific aspect of VCI. DUT dataset gives two ordinary and natural VCI scenarios in crowded university campus, which can be used for more general purpose VCI exploration. The trajectories of pedestrians, as well as vehicles, were extracted by processing video frames that come from a down-facing camera mounted on a hovering drone as the recording equipment. The final trajectories of pedestrians and vehicles were refined by Kalman filters with linear point-mass model and nonlinear bicycle model, respectively, in which xy-velocity of pedestrians and longitudinal speed and orientation of vehicles were estimated. The statistics of the velocity magnitude distribution demonstrated the validity of the proposed dataset. In total, there are approximate 340 pedestrian trajectories in CITR dataset and 1793 pedestrian trajectories in DUT dataset. The dataset is available at GitHub. △ Less

Submitted 19 April, 2019; v1 submitted 1 February, 2019; originally announced February 2019.

Comments: This paper was accepted into the 30th IEEE Intelligent Vehicles Symposium. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses

Model predictive trajectory optimization and tracking for on-road autonomous vehicles

Authors: Peng Liu, Brian Paden, Umit Ozguner

Abstract: Motion planning for autonomous vehicles requires spatio-temporal motion plans (i.e. state trajectories) to account for dynamic obstacles. This requires a trajectory tracking control process which faithfully tracks planned trajectories. In this paper, a control scheme is presented which first optimizes a planned trajectory and then tracks the optimized trajectory using a feedback-feedforward contro… ▽ More Motion planning for autonomous vehicles requires spatio-temporal motion plans (i.e. state trajectories) to account for dynamic obstacles. This requires a trajectory tracking control process which faithfully tracks planned trajectories. In this paper, a control scheme is presented which first optimizes a planned trajectory and then tracks the optimized trajectory using a feedback-feedforward controller. The feedforward element is calculated in a model predictive manner with a cost function focusing on driving performance. Stability of the error dynamic is then guaranteed by the design of the feedback-feedforward controller. The tracking performance of the control system is tested in a realistic simulated scenario where the control system must track an evasive lateral maneuver. The proposed controller performs well in simulation and can be easily adapted to different dynamic vehicle models. The uniqueness of the solution to the control synthesis eliminates any nondeterminism that could arise with switching between numerical solvers for the underlying mathematical program. △ Less

Submitted 10 November, 2018; originally announced November 2018.

Comments: 6 pages, 7 figures

Identification of Risk Significant Automotive Scenarios Under Hardware Failures

Authors: Mohammad Hejase, Arda Kurt, Tunc Aldemir, Umit Ozguner

Abstract: The level of autonomous functions in vehicular control systems has been on a steady rise. This rise makes it more challenging for control system engineers to ensure a high level of safety, especially against unexpected failures such as stochastic hardware failures. A generic Backtracking Process Algorithm (BPA) based on a deductive implementation of the Markov/Cell-to-Cell Map** technique is pro… ▽ More The level of autonomous functions in vehicular control systems has been on a steady rise. This rise makes it more challenging for control system engineers to ensure a high level of safety, especially against unexpected failures such as stochastic hardware failures. A generic Backtracking Process Algorithm (BPA) based on a deductive implementation of the Markov/Cell-to-Cell Map** technique is proposed for the identification of critical scenarios leading to the violation of safety goals. A discretized state-space representation of the system allows tracing of fault propagation throughout the system, and the quantification of probabilistic system evolution in time. A case study of a Hybrid State Control System for an autonomous vehicle prone to a brake-by-wire failure is constructed. The hazard of interest is collision with a stationary vehicle. The BPA is implemented to identify the risk significant scenarios leading to the hazard of interest. △ Less

Submitted 12 April, 2018; originally announced April 2018.

Comments: In Proceedings SCAV 2018, arXiv:1804.03406

Journal ref: EPTCS 269, 2018, pp. 59-73

Distributed Model Predictive Control of Spatially Interconnected Systems Using Switched Cost Functions

Authors: Peng Liu, Umit Ozguner

Abstract: This note proposes a distributed model predictive control (DMPC) scheme with switched cost functions for a class of spatially interconnected systems with communication constraints. Non-iterative and parallel communication strategy is considered to ensure that all distributed controllers complete input updates at each single information exchange step. The proposed DMPC scheme switches the optimizat… ▽ More This note proposes a distributed model predictive control (DMPC) scheme with switched cost functions for a class of spatially interconnected systems with communication constraints. Non-iterative and parallel communication strategy is considered to ensure that all distributed controllers complete input updates at each single information exchange step. The proposed DMPC scheme switches the optimization index on a switching surface generated by control invariant sets. With the index-switching strategy, stability of the origin is ensured by a terminal control law. Convergence conditions of the optimal cost to zero are established taking into account the causal link between the presumed trajectory and the optimized trajectory of the previous step. The compatibility constraints preserve the quadratic program property that is desired in practical applications. It is also observed that the proposed DMPC scheme has benefits dealing with systems that need to take into account safety-related spatial constraints. △ Less

Submitted 3 June, 2017; v1 submitted 7 June, 2016; originally announced June 2016.

Comments: 18 pages, 3 figures

A Hands-on Education Program on Cyber Physical Systems for High School Students

Authors: Vijay Gadepally, Ashok Krishnamurthy, Umit Ozguner

Abstract: Cyber Physical Systems (CPS) are the conjoining of an entities' physical and computational elements. The development of a typical CPS system follows a sequence from conceptual modeling, testing in simulated (virtual) worlds, testing in controlled (possibly laboratory) environments and finally deployment. Throughout each (repeatable) stage, the behavior of the physical entities, the sensing and sit… ▽ More Cyber Physical Systems (CPS) are the conjoining of an entities' physical and computational elements. The development of a typical CPS system follows a sequence from conceptual modeling, testing in simulated (virtual) worlds, testing in controlled (possibly laboratory) environments and finally deployment. Throughout each (repeatable) stage, the behavior of the physical entities, the sensing and situation assessment, and the computation and control options have to be understood and carefully represented through abstraction. The CPS Group at the Ohio State University, as part of an NSF funded CPS project on "Autonomous Driving in Mixed Environments", has been develo** CPS related educational activities at the K-12, undergraduate and graduate levels. The aim of these educational activities is to train students in the principles and design issues in CPS and to broaden the participation in science and engineering. The project team has a strong commitment to impact STEM education across the entire K-20 community. In this paper, we focus on the K-12 community and present a two-week Summer Program for high school juniors and seniors that introduces them to the principles of CPS design and walks them through several of the design steps. We also provide an online repository that aids CPS researchers in providing a similar educational experience. △ Less

Submitted 3 August, 2014; originally announced August 2014.