Showing 1–18 of 18 results for author: Markkula, G

Validation of human benchmark models for Automated Driving System approval: How competent and careful are they really?

Authors: Pierluigi Olleja, Gustav Markkula, Jonas Bärgman

Abstract: Advanced Driver Assistance Systems (ADAS) and Automated Driving Systems (ADS) are expected to improve comfort, productivity and, most importantly, safety for all road users. To ensure that the systems are safe, rules and regulations describing the systems' approval and validation procedures are in effect in Europe. The UNECE Regulation 157 (R157) is one of those. Annex 3 of R157 describes two driv… ▽ More Advanced Driver Assistance Systems (ADAS) and Automated Driving Systems (ADS) are expected to improve comfort, productivity and, most importantly, safety for all road users. To ensure that the systems are safe, rules and regulations describing the systems' approval and validation procedures are in effect in Europe. The UNECE Regulation 157 (R157) is one of those. Annex 3 of R157 describes two driver models, representing the performance of a "competent and careful" driver, which can be used as benchmarks to determine whether, in certain situations, a crash would be preventable by a human driver. However, these models have not been validated against human behavior in real safety-critical events. Therefore, this study uses counterfactual simulation to assess the performance of the two models when applied to 38 safety-critical cut-in near-crashes from the SHRP2 naturalistic driving study. The results show that the two computational models performed rather differently from the human drivers: one model showed a generally delayed braking reaction compared to the human drivers, causing crashes in three of the original near-crashes. The other model demonstrated, in general, brake onsets substantially earlier than the human drivers, possibly being overly sensitive to lateral perturbations. That is, the first model does not seem to behave as the competent and careful driver it is supposed to represent, while the second seems to be overly careful. Overall, our results show that, if models are to be included in regulations, they need to be substantially improved. We argue that achieving this will require better validation across the scenario types that the models are intended to cover (e.g., cut-in conflicts), a process which should include applying the models counterfactually to near-crashes and validating them against several different safety related metrics. △ Less

Submitted 13 June, 2024; originally announced June 2024.

Pedestrian crossing decisions can be explained by bounded optimal decision-making under noisy visual perception

Authors: Yueyang Wang, Aravinda Ramakrishnan Srinivasan, Jussi P. P. Jokinen, Antti Oulasvirta, Gustav Markkula

Abstract: This paper presents a model of pedestrian crossing decisions, based on the theory of computational rationality. It is assumed that crossing decisions are boundedly optimal, with bounds on optimality arising from human cognitive limitations. While previous models of pedestrian behaviour have been either 'black-box' machine learning models or mechanistic models with explicit assumptions about cognit… ▽ More This paper presents a model of pedestrian crossing decisions, based on the theory of computational rationality. It is assumed that crossing decisions are boundedly optimal, with bounds on optimality arising from human cognitive limitations. While previous models of pedestrian behaviour have been either 'black-box' machine learning models or mechanistic models with explicit assumptions about cognitive factors, we combine both approaches. Specifically, we model mechanistically noisy human visual perception and assumed rewards in crossing, but we use reinforcement learning to learn bounded optimal behaviour policy. The model reproduces a larger number of known empirical phenomena than previous models, in particular: (1) the effect of the time to arrival of an approaching vehicle on whether the pedestrian accepts the gap, the effect of the vehicle's speed on both (2) gap acceptance and (3) pedestrian timing of crossing in front of yielding vehicles, and (4) the effect on this crossing timing of the stop** distance of the yielding vehicle. Notably, our findings suggest that behaviours previously framed as 'biases' in decision-making, such as speed-dependent gap acceptance, might instead be a product of rational adaptation to the constraints of visual perception. Our approach also permits fitting the parameters of cognitive constraints and rewards per individual, to better account for individual differences. To conclude, by leveraging both RL and mechanistic modelling, our model offers novel insights about pedestrian behaviour, and may provide a useful foundation for more accurate and scalable pedestrian models. △ Less

Submitted 6 February, 2024; originally announced February 2024.

Using Models Based on Cognitive Theory to Predict Human Behavior in Traffic: A Case Study

Authors: Julian F. Schumann, Aravinda Ramakrishnan Srinivasan, Jens Kober, Gustav Markkula, Arkady Zgonnikov

Abstract: The development of automated vehicles has the potential to revolutionize transportation, but they are currently unable to ensure a safe and time-efficient driving style. Reliable models predicting human behavior are essential for overcoming this issue. While data-driven models are commonly used to this end, they can be vulnerable in safety-critical edge cases. This has led to an interest in models… ▽ More The development of automated vehicles has the potential to revolutionize transportation, but they are currently unable to ensure a safe and time-efficient driving style. Reliable models predicting human behavior are essential for overcoming this issue. While data-driven models are commonly used to this end, they can be vulnerable in safety-critical edge cases. This has led to an interest in models incorporating cognitive theory, but as such models are commonly developed for explanatory purposes, this approach's effectiveness in behavior prediction has remained largely untested so far. In this article, we investigate the usefulness of the \emph{Commotions} model -- a novel cognitively plausible model incorporating the latest theories of human perception, decision-making, and motor control -- for predicting human behavior in gap acceptance scenarios, which entail many important traffic interactions such as lane changes and intersections. We show that this model can compete with or even outperform well-established data-driven prediction models across several naturalistic datasets. These results demonstrate the promise of incorporating cognitive theory in behavior prediction models for automated vehicles. △ Less

Submitted 9 October, 2023; v1 submitted 24 May, 2023; originally announced May 2023.

Comments: 6 pages, 2 figures

The COMMOTIONS Urban Interactions Driving Simulator Study Dataset

Authors: Aravinda Ramakrishnan Srinivasan, Julian Schumann, Yueyang Wang, Yi-Shin Lin, Michael Daly, Albert Solernou, Arkady Zgonnikov, Matteo Leonetti, Jac Billington, Gustav Markkula

Abstract: Accurate modelling of road user interaction has received lot of attention in recent years due to the advent of increasingly automated vehicles. To support such modelling, there is a need to complement naturalistic datasets of road user interaction with targeted, controlled study data. This paper describes a dataset collected in a simulator study conducted in the project COMMOTIONS, addressing urba… ▽ More Accurate modelling of road user interaction has received lot of attention in recent years due to the advent of increasingly automated vehicles. To support such modelling, there is a need to complement naturalistic datasets of road user interaction with targeted, controlled study data. This paper describes a dataset collected in a simulator study conducted in the project COMMOTIONS, addressing urban driving interactions, in a state of the art moving base driving simulator. The study focused on two types of near-crash situations that can arise in urban driving interactions, and also collected data on human driver gap acceptance across a range of controlled gap sequences. △ Less

Submitted 2 July, 2024; v1 submitted 17 May, 2023; originally announced May 2023.

Comments: 5 pages, 8 figures, 6 tables, data techincal description paper, Open Science Foundation - https://osf.io/eazg5/

Cross or Wait? Predicting Pedestrian Interaction Outcomes at Unsignalized Crossings

Authors: Chi Zhang, Amir Hossein Kalantari, Yue Yang, Zhongjun Ni, Gustav Markkula, Natasha Merat, Christian Berger

Abstract: Predicting pedestrian behavior when interacting with vehicles is one of the most critical challenges in the field of automated driving. Pedestrian crossing behavior is influenced by various interaction factors, including time to arrival, pedestrian waiting time, the presence of zebra crossing, and the properties and personality traits of both pedestrians and drivers. However, these factors have no… ▽ More Predicting pedestrian behavior when interacting with vehicles is one of the most critical challenges in the field of automated driving. Pedestrian crossing behavior is influenced by various interaction factors, including time to arrival, pedestrian waiting time, the presence of zebra crossing, and the properties and personality traits of both pedestrians and drivers. However, these factors have not been fully explored for use in predicting interaction outcomes. In this paper, we use machine learning to predict pedestrian crossing behavior including pedestrian crossing decision, crossing initiation time (CIT), and crossing duration (CD) when interacting with vehicles at unsignalized crossings. Distributed simulator data are utilized for predicting and analyzing the interaction factors. Compared with the logistic regression baseline model, our proposed neural network model improves the prediction accuracy and F1 score by 4.46% and 3.23%, respectively. Our model also reduces the root mean squared error (RMSE) for CIT and CD by 21.56% and 30.14% compared with the linear regression model. Additionally, we have analyzed the importance of interaction factors, and present the results of models using fewer factors. This provides information for model selection in different scenarios with limited input features. △ Less

Submitted 19 March, 2024; v1 submitted 17 April, 2023; originally announced April 2023.

Comments: 8 pages, 7 figures, 9 tables. Accepted in 2023 IEEE Intelligent Vehicles Symposium (IV). DOI: 10.1109/IV55152.2023.10186616

MSC Class: 68T40 ACM Class: I.2.10

Journal ref: C. Zhang et al, "Cross or Wait? Predicting Pedestrian Interaction Outcomes at Unsignalized Crossings," 2023 IEEE Intelligent Vehicles Symposium (IV), Anchorage, AK, USA, 2023, pp. 1-8

An active inference model of car following: Advantages and applications

Authors: Ran Wei, Anthony D. McDonald, Alfredo Garcia, Gustav Markkula, Johan Engstrom, Matthew O'Kelly

Abstract: Driver process models play a central role in the testing, verification, and development of automated and autonomous vehicle technologies. Prior models developed from control theory and physics-based rules are limited in automated vehicle applications due to their restricted behavioral repertoire. Data-driven machine learning models are more capable than rule-based models but are limited by the nee… ▽ More Driver process models play a central role in the testing, verification, and development of automated and autonomous vehicle technologies. Prior models developed from control theory and physics-based rules are limited in automated vehicle applications due to their restricted behavioral repertoire. Data-driven machine learning models are more capable than rule-based models but are limited by the need for large training datasets and their lack of interpretability, i.e., an understandable link between input data and output behaviors. We propose a novel car following modeling approach using active inference, which has comparable behavioral flexibility to data-driven models while maintaining interpretability. We assessed the proposed model, the Active Inference Driving Agent (AIDA), through a benchmark analysis against the rule-based Intelligent Driver Model, and two neural network Behavior Cloning models. The models were trained and tested on a real-world driving dataset using a consistent process. The testing results showed that the AIDA predicted driving controls significantly better than the rule-based Intelligent Driver Model and had similar accuracy to the data-driven neural network models in three out of four evaluations. Subsequent interpretability analyses illustrated that the AIDA's learned distributions were consistent with driver behavior theory and that visualizations of the distributions could be used to directly comprehend the model's decision making process and correct model errors attributable to limited training data. The results indicate that the AIDA is a promising alternative to black-box data-driven models and suggest a need for further research focused on modeling driving style and model training with more diverse datasets. △ Less

Submitted 27 March, 2023; originally announced March 2023.

Modeling human road crossing decisions as reward maximization with visual perception limitations

Authors: Yueyang Wang, Aravinda Ramakrishnan Srinivasan, Jussi P. P. Jokinen, Antti Oulasvirta, Gustav Markkula

Abstract: Understanding the interaction between different road users is critical for road safety and automated vehicles (AVs). Existing mathematical models on this topic have been proposed based mostly on either cognitive or machine learning (ML) approaches. However, current cognitive models are incapable of simulating road user trajectories in general scenarios, and ML models lack a focus on the mechanisms… ▽ More Understanding the interaction between different road users is critical for road safety and automated vehicles (AVs). Existing mathematical models on this topic have been proposed based mostly on either cognitive or machine learning (ML) approaches. However, current cognitive models are incapable of simulating road user trajectories in general scenarios, and ML models lack a focus on the mechanisms generating the behavior and take a high-level perspective which can cause failures to capture important human-like behaviors. Here, we develop a model of human pedestrian crossing decisions based on computational rationality, an approach using deep reinforcement learning (RL) to learn boundedly optimal behavior policies given human constraints, in our case a model of the limited human visual system. We show that the proposed combined cognitive-RL model captures human-like patterns of gap acceptance and crossing initiation time. Interestingly, our model's decisions are sensitive to not only the time gap, but also the speed of the approaching vehicle, something which has been described as a "bias" in human gap acceptance behavior. However, our results suggest that this is instead a rational adaption to human perceptual limitations. Moreover, we demonstrate an approach to accounting for individual differences in computational rationality models, by conditioning the RL policy on the parameters of the human constraints. Our results demonstrate the feasibility of generating more human-like road user behavior by combining RL with cognitive models. △ Less

Submitted 27 January, 2023; originally announced January 2023.

Comments: 6 pages, 5 figures,1 table, manuscript created for consideration at IEEE IV 2023 conference

Deconstructing Pedestrian Crossing Decision-making in Interactions with Continuous Traffic: an Anthropomorphic Model

Authors: Kai Tian, Gustav Markkula, Chongfeng Wei, Yee Mun Lee, Ruth Madigan, Toshiya Hirose, Natasha Merat, Richard Romano

Abstract: As safe and comfortable interactions with pedestrians could contribute to automated vehicles' (AVs) social acceptance and scale, increasing attention has been drawn to computational pedestrian behavior models. However, very limited studies characterize pedestrian crossing behavior based on specific behavioral mechanisms, as those mechanisms underpinning pedestrian road behavior are not yet clear.… ▽ More As safe and comfortable interactions with pedestrians could contribute to automated vehicles' (AVs) social acceptance and scale, increasing attention has been drawn to computational pedestrian behavior models. However, very limited studies characterize pedestrian crossing behavior based on specific behavioral mechanisms, as those mechanisms underpinning pedestrian road behavior are not yet clear. Here, we reinterpret pedestrian crossing behavior based on a deconstructed crossing decision process at uncontrolled intersections with continuous traffic. Notably, we explain and model pedestrian crossing behavior as they wait for crossing opportunities, optimizing crossing decisions by comparing the visual collision risk of approaching vehicles around them. A collision risk-based crossing initiation model is proposed to characterize the time-dynamic nature of pedestrian crossing decisions. A simulation tool is established to reproduce pedestrian behavior by employing the proposed model and a social force model. Two datasets collected in a CAVE-based immersive pedestrian simulator are applied to calibrate and validate the model. The model predicts pedestrian crossing decisions across all traffic scenarios well. In particular, by considering the decision strategy that pedestrians compare the collision risk of surrounding traffic gaps, model performance is significantly improved. Moreover, the collision risk-based crossing initiation model accurately captures the timing of pedestrian crossing initiations within each gap. This work concisely demonstrates how pedestrians dynamically adapt their crossings in continuous traffic based on perceived collision risk, potentially providing insights into modeling coupled human-AV interactions or serving as a tool to realize human-like pedestrian road behavior in virtual AVs test platforms. △ Less

Submitted 25 January, 2023; originally announced January 2023.

Beyond RMSE: Do machine-learned models of road user interaction produce human-like behavior?

Authors: Aravinda Ramakrishnan Srinivasan, Yi-Shin Lin, Morris Antonello, Anthony Knittel, Mohamed Hasan, Majd Hawasly, John Redford, Subramanian Ramamoorthy, Matteo Leonetti, Jac Billington, Richard Romano, Gustav Markkula

Abstract: Autonomous vehicles use a variety of sensors and machine-learned models to predict the behavior of surrounding road users. Most of the machine-learned models in the literature focus on quantitative error metrics like the root mean square error (RMSE) to learn and report their models' capabilities. This focus on quantitative error metrics tends to ignore the more important behavioral aspect of the… ▽ More Autonomous vehicles use a variety of sensors and machine-learned models to predict the behavior of surrounding road users. Most of the machine-learned models in the literature focus on quantitative error metrics like the root mean square error (RMSE) to learn and report their models' capabilities. This focus on quantitative error metrics tends to ignore the more important behavioral aspect of the models, raising the question of whether these models really predict human-like behavior. Thus, we propose to analyze the output of machine-learned models much like we would analyze human data in conventional behavioral research. We introduce quantitative metrics to demonstrate presence of three different behavioral phenomena in a naturalistic highway driving dataset: 1) The kinematics-dependence of who passes a merging point first 2) Lane change by an on-highway vehicle to accommodate an on-ramp vehicle 3) Lane changes by vehicles on the highway to avoid lead vehicle conflicts. Then, we analyze the behavior of three machine-learned models using the same metrics. Even though the models' RMSE value differed, all the models captured the kinematic-dependent merging behavior but struggled at varying degrees to capture the more nuanced courtesy lane change and highway lane change behavior. Additionally, the collision aversion analysis during lane changes showed that the models struggled to capture the physical aspect of human driving: leaving adequate gap between the vehicles. Thus, our analysis highlighted the inadequacy of simple quantitative metrics and the need to take a broader behavioral perspective when analyzing machine-learned models of human driving predictions. △ Less

Submitted 28 March, 2023; v1 submitted 22 June, 2022; originally announced June 2022.

Comments: This work has been accepted for publication in the IEEE Transactions on Intelligent Transportation Systems journal on 13th March 2023

Models of human behavior for human-robot interaction and automated driving: How accurate do the models of human behavior need to be?

Authors: Gustav Markkula, Mehmet Dogar

Abstract: There are many examples of cases where access to improved models of human behavior and cognition has allowed creation of robots which can better interact with humans, and not least in road vehicle automation this is a rapidly growing area of research. Human-robot interaction (HRI) therefore provides an important applied setting for human behavior modeling - but given the vast complexity of human b… ▽ More There are many examples of cases where access to improved models of human behavior and cognition has allowed creation of robots which can better interact with humans, and not least in road vehicle automation this is a rapidly growing area of research. Human-robot interaction (HRI) therefore provides an important applied setting for human behavior modeling - but given the vast complexity of human behavior, how complete and accurate do these models need to be? Here, we outline some possible ways of thinking about this problem, starting from the suggestion that modelers need to keep the right end goal in sight: A successful human-robot interaction, in terms of safety, performance, and human satisfaction. Efforts toward model completeness and accuracy should be focused on those aspects of human behavior to which interaction success is most sensitive. We emphasise that identifying which those aspects are is a difficult scientific objective in its own right, distinct for each given HRI context. We propose and exemplify an approach to formulating a priori hypotheses on this matter, in cases where robots are to be involved in interactions which currently take place between humans, such as in automated driving. Our perspective also highlights some possible risks of overreliance on machine-learned models of human behavior in HRI, and how to mitigate against those risks. △ Less

Submitted 24 August, 2022; v1 submitted 12 February, 2022; originally announced February 2022.

Comments: Accepted for publication in IEEE Robotics & Automation Magazine. In press

A Utility Maximization Model of Pedestrian and Driver Interactions

Authors: Yi-Shin Lin, Aravinda Ramakrishnan Srinivasan, Matteo Leonetti, Jac Billington, Gustav Markkula

Abstract: Many models account for the traffic flow of road users but few take the details of local interactions into consideration and how they could deteriorate into safety-critical situations. Building on the concept of sensorimotor control, we develop a modeling framework applying the principles of utility maximization, motor primitives, and intermittent action decisions to account for the details of int… ▽ More Many models account for the traffic flow of road users but few take the details of local interactions into consideration and how they could deteriorate into safety-critical situations. Building on the concept of sensorimotor control, we develop a modeling framework applying the principles of utility maximization, motor primitives, and intermittent action decisions to account for the details of interactive behaviors among road users. The framework connects these principles to the decision theory and is applied to determine whether such an approach can reproduce the following phenomena: When two pedestrians travel on crossing paths, (a) their interaction is sensitive to initial asymmetries, and (b) based on which, they rapidly resolve collision conflict by adapting their behaviors. When a pedestrian crosses the road while facing an approaching car, (c) either road user yields to the other to resolve their conflict, akin to the pedestrian interaction, and (d) the outcome reveals a specific situational kinematics, associated with the nature of vehicle acceleration. We show that these phenomena emerge naturally from our modeling framework when the model can evolve its parameters as a consequence of the situations. We believe that the modeling framework and phenomenon-centered analysis offer promising tools to understand road user interactions. We conclude with a discussion on how the model can be instrumental in studying the safety-critical situations when including other variables in road-user interactions. △ Less

Submitted 21 October, 2021; originally announced October 2021.

Comments: 10 pages, 7 figures

Maneuver-Aware Pooling for Vehicle Trajectory Prediction

Authors: Mohamed Hasan, Albert Solernou, Evangelos Paschalidis, He Wang, Gustav Markkula, Richard Romano

Abstract: Autonomous vehicles should be able to predict the future states of its environment and respond appropriately. Specifically, predicting the behavior of surrounding human drivers is vital for such platforms to share the same road with humans. Behavior of each of the surrounding vehicles is governed by the motion of its neighbor vehicles. This paper focuses on predicting the behavior of the surroundi… ▽ More Autonomous vehicles should be able to predict the future states of its environment and respond appropriately. Specifically, predicting the behavior of surrounding human drivers is vital for such platforms to share the same road with humans. Behavior of each of the surrounding vehicles is governed by the motion of its neighbor vehicles. This paper focuses on predicting the behavior of the surrounding vehicles of an autonomous vehicle on highways. We are motivated by improving the prediction accuracy when a surrounding vehicle performs lane change and highway merging maneuvers. We propose a novel pooling strategy to capture the inter-dependencies between the neighbor vehicles. Depending solely on Euclidean trajectory representation, the existing pooling strategies do not model the context information of the maneuvers intended by a surrounding vehicle. In contrast, our pooling mechanism employs polar trajectory representation, vehicles orientation and radial velocity. This results in an implicitly maneuver-aware pooling operation. We incorporated the proposed pooling mechanism into a generative encoder-decoder model, and evaluated our method on the public NGSIM dataset. The results of maneuver-based trajectory predictions demonstrate the effectiveness of the proposed method compared with the state-of-the-art approaches. Our "Pooling Toolbox" code is available at https://github.com/m-hasan-n/pooling. △ Less

Submitted 28 April, 2021; originally announced April 2021.

Comments: Preprint (under review IROS'21). arXiv admin note: text overlap with arXiv:2104.11180

Maneuver-based Anchor Trajectory Hypotheses at Roundabouts

Authors: Mohamed Hasan, Evangelos Paschalidis, Albert Solernou, He Wang, Gustav Markkula, Richard Romano

Abstract: Predicting future behavior of the surrounding vehicles is crucial for self-driving platforms to safely navigate through other traffic. This is critical when making decisions like crossing an unsignalized intersection. We address the problem of vehicle motion prediction in a challenging roundabout environment by learning from human driver data. We extend existing recurrent encoder-decoder models to… ▽ More Predicting future behavior of the surrounding vehicles is crucial for self-driving platforms to safely navigate through other traffic. This is critical when making decisions like crossing an unsignalized intersection. We address the problem of vehicle motion prediction in a challenging roundabout environment by learning from human driver data. We extend existing recurrent encoder-decoder models to be advantageously combined with anchor trajectories to predict vehicle behaviors on a roundabout. Drivers' intentions are encoded by a set of maneuvers that correspond to semantic driving concepts. Accordingly, our model employs a set of maneuver-specific anchor trajectories that cover the space of possible outcomes at the roundabout. The proposed model can output a multi-modal distribution over the predicted future trajectories based on the maneuver-specific anchors. We evaluate our model using the public RounD dataset and the experiment results show the effectiveness of the proposed maneuver-based anchor regression in improving prediction accuracy, reducing the average RMSE to 28% less than the best baseline. Our code is available at https://github.com/m-hasan-n/roundabout. △ Less

Submitted 22 April, 2021; originally announced April 2021.

Comments: Under Review IROS 2021

Comparing merging behaviors observed in naturalistic data with behaviors generated by a machine learned model

Authors: Aravinda Ramakrishnan Srinivasan, Mohamed Hasan, Yi-Shin Lin, Matteo Leonetti, Jac Billington, Richard Romano, Gustav Markkula

Abstract: There is quickly growing literature on machine-learned models that predict human driving trajectories in road traffic. These models focus their learning on low-dimensional error metrics, for example average distance between model-generated and observed trajectories. Such metrics permit relative comparison of models, but do not provide clearly interpretable information on how close to human behavio… ▽ More There is quickly growing literature on machine-learned models that predict human driving trajectories in road traffic. These models focus their learning on low-dimensional error metrics, for example average distance between model-generated and observed trajectories. Such metrics permit relative comparison of models, but do not provide clearly interpretable information on how close to human behavior the models actually come, for example in terms of higher-level behavior phenomena that are known to be present in human driving. We study highway driving as an example scenario, and introduce metrics to quantitatively demonstrate the presence, in a naturalistic dataset, of two familiar behavioral phenomena: (1) The kinematics-dependent contest, between on-highway and on-ramp vehicles, of who passes the merging point first. (2) Courtesy lane changes away from the outermost lane, to leave space for a merging vehicle. Applying the exact same metrics to the output of a state-of-the-art machine-learned model, we show that the model is capable of reproducing the former phenomenon, but not the latter. We argue that this type of behavioral analysis provides information that is not available from conventional model-fitting metrics, and that it may be useful to analyze (and possibly fit) models also based on these types of behavioral criteria. △ Less

Submitted 21 April, 2021; originally announced April 2021.

Comments: This paper has been submitted to 24th IEEE International Conference on Intelligent Transportation - ITSC2021, September 19-22, 2021 Indianapolis, IN, United States

Pedestrian Models for Autonomous Driving Part II: High-Level Models of Human Behavior

Authors: Fanta Camara, Nicola Bellotto, Serhan Cosar, Florian Weber, Dimitris Nathanael, Matthias Althoff, **gyuan Wu, Johannes Ruenz, André Dietrich, Gustav Markkula, Anna Schieben, Fabio Tango, Natasha Merat, Charles W. Fox

Abstract: Autonomous vehicles (AVs) must share space with pedestrians, both in carriageway cases such as cars at pedestrian crossings and off-carriageway cases such as delivery vehicles navigating through crowds on pedestrianized high-streets. Unlike static obstacles, pedestrians are active agents with complex, interactive motions. Planning AV actions in the presence of pedestrians thus requires modelling o… ▽ More Autonomous vehicles (AVs) must share space with pedestrians, both in carriageway cases such as cars at pedestrian crossings and off-carriageway cases such as delivery vehicles navigating through crowds on pedestrianized high-streets. Unlike static obstacles, pedestrians are active agents with complex, interactive motions. Planning AV actions in the presence of pedestrians thus requires modelling of their probable future behaviour as well as detecting and tracking them. This narrative review article is Part II of a pair, together surveying the current technology stack involved in this process, organising recent research into a hierarchical taxonomy ranging from low-level image detection to high-level psychological models, from the perspective of an AV designer. This self-contained Part II covers the higher levels of this stack, consisting of models of pedestrian behaviour, from prediction of individual pedestrians' likely destinations and paths, to game-theoretic models of interactions between pedestrians and autonomous vehicles. This survey clearly shows that, although there are good models for optimal walking behaviour, high-level psychological and social modelling of pedestrian behaviour still remains an open research question that requires many conceptual issues to be clarified. Early work has been done on descriptive and qualitative models of behaviour, but much work is still needed to translate them into quantitative algorithms for practical AV control. △ Less

Submitted 20 July, 2020; v1 submitted 26 March, 2020; originally announced March 2020.

Comments: Accepted for publication in the IEEE Transactions on Intelligent Transportation Systems

An Objective Assessment of the Utility of a Driving Simulator for Low Mu Testing

Authors: Richard Romano, Gustav Markkula, Erwin Boer, Hamish Jamson, Alex Bean, Andrew Tomlinson, Anthony Horrobin, Ehsan Sadraei

Abstract: Driving simulators can be used to test vehicle designs earlier, prior to building physical prototypes. One area of particular interest is winter testing since testing is limited to specific times of year and specific regions in the world. To ensure that the simulator is fit for purpose, an objective assessment is required. In this study a simulator and real world comparison was performed with thre… ▽ More Driving simulators can be used to test vehicle designs earlier, prior to building physical prototypes. One area of particular interest is winter testing since testing is limited to specific times of year and specific regions in the world. To ensure that the simulator is fit for purpose, an objective assessment is required. In this study a simulator and real world comparison was performed with three simulator configurations (standard, no steering torque, no motion) to assess the ability of a utility triplet of analyses to be able to quantify the differences between the real world and the different simulator configurations. The results suggest that the utility triplet is effective in measuring the differences in simulator configurations and that the developed Virtual Sweden environment achieved rather good behavioural fidelity in the sense of preserving absolute levels of many measures of behaviour. The main limitation in the simulated environment seemed to be the poor match of the dynamic lateral friction limit on snow and ice when compared to the real world. △ Less

Submitted 7 December, 2018; originally announced December 2018.

Modelling visual-vestibular integration and behavioural adaptation in the driving simulator

Authors: Gustav Markkula, Richard Romano, Rachel Waldram, Oscar Giles, Callum Mole, Richard Wilkie

Abstract: It is well established that not only vision but also other sensory modalities affect drivers' control of their vehicles, and that drivers adapt over time to persistent changes in sensory cues (for example in driving simulators), but the mechanisms underlying these behavioural phenomena are poorly understood. Here, we consider the existing literature on how driver steering in slalom tasks is affect… ▽ More It is well established that not only vision but also other sensory modalities affect drivers' control of their vehicles, and that drivers adapt over time to persistent changes in sensory cues (for example in driving simulators), but the mechanisms underlying these behavioural phenomena are poorly understood. Here, we consider the existing literature on how driver steering in slalom tasks is affected by the down-scaling of vestibular cues, and propose a driver model that can explain the empirically observed effects, namely: decreased task performance and increased steering effort during initial exposure, followed by a partial reversal of these effects as task exposure is prolonged. Unexpectedly, the model also reproduced another empirical finding: a local optimum for motion down-scaling, where path-tracking is better than when one-to-one motion cues are available. Overall, the results imply that: (1) drivers make direct use of vestibular information as part of determining appropriate steering, and (2) motion down-scaling causes a yaw rate underestimation phenomenon, where drivers behave as if the simulated vehicle is rotating more slowly than it is. However, (3) in the slalom task, a certain degree of such yaw rate underestimation is beneficial to path tracking performance. Furthermore, (4) behavioural adaptation, as empirically observed in slalom tasks, may occur due to (a) down-weighting of vestibular cues, and/or (b) increased sensitivity to control errors, in determining when to adjust steering and by how much, but (c) seemingly not in the form of a full compensatory rescaling of the received vestibular input. The analyses presented here provide new insights and hypotheses about simulator driving, and the developed models can be used to support research on multisensory integration and behavioural adaptation in both driving and other task domains. △ Less

Submitted 7 November, 2018; v1 submitted 29 October, 2018; originally announced October 2018.

Comments: Changes in v2: Minor language improvements to Abstract and Conclusion; Changes in v3: Added acknowledgments and data statement

Sustained sensorimotor control as intermittent decisions about prediction errors: Computational framework and application to ground vehicle steering

Authors: Gustav Markkula, Erwin Boer, Richard Romano, Natasha Merat

Abstract: A conceptual and computational framework is proposed for modelling of human sensorimotor control, and is exemplified for the sensorimotor task of steering a car. The framework emphasises control intermittency, and extends on existing models by suggesting that the nervous system implements intermittent control using a combination of (1) motor primitives, (2) prediction of sensory outcomes of motor… ▽ More A conceptual and computational framework is proposed for modelling of human sensorimotor control, and is exemplified for the sensorimotor task of steering a car. The framework emphasises control intermittency, and extends on existing models by suggesting that the nervous system implements intermittent control using a combination of (1) motor primitives, (2) prediction of sensory outcomes of motor actions, and (3) evidence accumulation of prediction errors. It is shown that approximate but useful sensory predictions in the intermittent control context can be constructed without detailed forward models, as a superposition of simple prediction primitives, resembling neurobiologically observed corollary discharges. The proposed mathematical framework allows straightforward extension to intermittent behaviour from existing one-dimensional continuous models in the linear control and ecological psychology traditions. Empirical observations from a driving simulator provide support for some of the framework assumptions: It is shown that human steering control, in routine lane-kee** and in a demanding near-limit task, is better described as a sequence of discrete stepwise steering adjustments, than as continuous control. Furthermore, the amplitudes of individual steering adjustments are well predicted by a compound visual cue signalling steering error, and even better so if also adjusting for predictions of how the same cue is affected by previous control. Finally, evidence accumulation is shown to explain observed covariability between inter-adjustment durations and adjustment amplitudes, seemingly better so than the type of threshold mechanisms that are typically assumed in existing models of intermittent control. △ Less

Submitted 8 March, 2017; originally announced March 2017.