-
Amodal Optical Flow
Authors:
Maximilian Luz,
Rohit Mohan,
Ahmed Rida Sekkat,
Oliver Sawade,
Elmar Matthes,
Thomas Brox,
Abhinav Valada
Abstract:
Optical flow estimation is very challenging in situations with transparent or occluded objects. In this work, we address these challenges at the task level by introducing Amodal Optical Flow, which integrates optical flow with amodal perception. Instead of only representing the visible regions, we define amodal optical flow as a multi-layered pixel-level motion field that encompasses both visible…
▽ More
Optical flow estimation is very challenging in situations with transparent or occluded objects. In this work, we address these challenges at the task level by introducing Amodal Optical Flow, which integrates optical flow with amodal perception. Instead of only representing the visible regions, we define amodal optical flow as a multi-layered pixel-level motion field that encompasses both visible and occluded regions of the scene. To facilitate research on this new task, we extend the AmodalSynthDrive dataset to include pixel-level labels for amodal optical flow estimation. We present several strong baselines, along with the Amodal Flow Quality metric to quantify the performance in an interpretable manner. Furthermore, we propose the novel AmodalFlowNet as an initial step toward addressing this task. AmodalFlowNet consists of a transformer-based cost-volume encoder paired with a recurrent transformer decoder which facilitates recurrent hierarchical feature propagation and amodal semantic grounding. We demonstrate the tractability of amodal optical flow in extensive experiments and show its utility for downstream tasks such as panoptic tracking. We make the dataset, code, and trained models publicly available at http://amodal-flow.cs.uni-freiburg.de.
△ Less
Submitted 7 May, 2024; v1 submitted 13 November, 2023;
originally announced November 2023.
-
AmodalSynthDrive: A Synthetic Amodal Perception Dataset for Autonomous Driving
Authors:
Ahmed Rida Sekkat,
Rohit Mohan,
Oliver Sawade,
Elmar Matthes,
Abhinav Valada
Abstract:
Unlike humans, who can effortlessly estimate the entirety of objects even when partially occluded, modern computer vision algorithms still find this aspect extremely challenging. Leveraging this amodal perception for autonomous driving remains largely untapped due to the lack of suitable datasets. The curation of these datasets is primarily hindered by significant annotation costs and mitigating a…
▽ More
Unlike humans, who can effortlessly estimate the entirety of objects even when partially occluded, modern computer vision algorithms still find this aspect extremely challenging. Leveraging this amodal perception for autonomous driving remains largely untapped due to the lack of suitable datasets. The curation of these datasets is primarily hindered by significant annotation costs and mitigating annotator subjectivity in accurately labeling occluded regions. To address these limitations, we introduce AmodalSynthDrive, a synthetic multi-task multi-modal amodal perception dataset. The dataset provides multi-view camera images, 3D bounding boxes, LiDAR data, and odometry for 150 driving sequences with over 1M object annotations in diverse traffic, weather, and lighting conditions. AmodalSynthDrive supports multiple amodal scene understanding tasks including the introduced amodal depth estimation for enhanced spatial understanding. We evaluate several baselines for each of these tasks to illustrate the challenges and set up public benchmarking servers. The dataset is available at http://amodalsynthdrive.cs.uni-freiburg.de.
△ Less
Submitted 11 March, 2024; v1 submitted 12 September, 2023;
originally announced September 2023.
-
Vehicle Automation Field Test: Impact on Driver Behavior and Trust
Authors:
Walter Morales Alvarez,
Nikita Smirnov,
Elmar Matthes,
Cristina Olaverri-Monreal
Abstract:
With the growing technological advances in autonomous driving, the transport industry and research community seek to determine the impact that autonomous vehicles (AV) will have on consumers, as well as identify the different factors that will influence their use. Most of the research performed so far relies on laboratory-controlled conditions using driving simulators, as they offer a safe environ…
▽ More
With the growing technological advances in autonomous driving, the transport industry and research community seek to determine the impact that autonomous vehicles (AV) will have on consumers, as well as identify the different factors that will influence their use. Most of the research performed so far relies on laboratory-controlled conditions using driving simulators, as they offer a safe environment for testing advanced driving assistance systems (ADAS). In this study we analyze the behavior of drivers that are placed in control of an automated vehicle in a real life driving environment. The vehicle is equipped with advanced autonomy, making driver control of the vehicle unnecessary in many scenarios, although a driver take over is possible and sometimes required. In doing so, we aim to determine the impact of such a system on the driver and their driving performance. To this end road users' behavior from naturalistic driving data is analyzed focusing on awareness and diagnosis of the road situation. Results showed that the road features determined the level of visual attention and trust in the automation. They also showed that the activities performed during the automation affected the reaction time to take over the control of the vehicle.
△ Less
Submitted 10 January, 2022; v1 submitted 4 June, 2020;
originally announced June 2020.
