-
Convolutions for Spatial Interaction Modeling
Authors:
Zhaoen Su,
Chao Wang,
David Bradley,
Carlos Vallespi-Gonzalez,
Carl Wellington,
Nemanja Djuric
Abstract:
In many different fields interactions between objects play a critical role in determining their behavior. Graph neural networks (GNNs) have emerged as a powerful tool for modeling interactions, although often at the cost of adding considerable complexity and latency. In this paper, we consider the problem of spatial interaction modeling in the context of predicting the motion of actors around auto…
▽ More
In many different fields interactions between objects play a critical role in determining their behavior. Graph neural networks (GNNs) have emerged as a powerful tool for modeling interactions, although often at the cost of adding considerable complexity and latency. In this paper, we consider the problem of spatial interaction modeling in the context of predicting the motion of actors around autonomous vehicles, and investigate alternatives to GNNs. We revisit 2D convolutions and show that they can demonstrate comparable performance to graph networks in modeling spatial interactions with lower latency, thus providing an effective and efficient alternative in time-critical systems. Moreover, we propose a novel interaction loss to further improve the interaction modeling of the considered methods.
△ Less
Submitted 8 June, 2022; v1 submitted 14 April, 2021;
originally announced April 2021.
-
MultiXNet: Multiclass Multistage Multimodal Motion Prediction
Authors:
Nemanja Djuric,
Henggang Cui,
Zhaoen Su,
Shangxuan Wu,
Huahua Wang,
Fang-Chieh Chou,
Luisa San Martin,
Song Feng,
Rui Hu,
Yang Xu,
Alyssa Dayan,
Sidney Zhang,
Brian C. Becker,
Gregory P. Meyer,
Carlos Vallespi-Gonzalez,
Carl K. Wellington
Abstract:
One of the critical pieces of the self-driving puzzle is understanding the surroundings of a self-driving vehicle (SDV) and predicting how these surroundings will change in the near future. To address this task we propose MultiXNet, an end-to-end approach for detection and motion prediction based directly on lidar sensor data. This approach builds on prior work by handling multiple classes of traf…
▽ More
One of the critical pieces of the self-driving puzzle is understanding the surroundings of a self-driving vehicle (SDV) and predicting how these surroundings will change in the near future. To address this task we propose MultiXNet, an end-to-end approach for detection and motion prediction based directly on lidar sensor data. This approach builds on prior work by handling multiple classes of traffic actors, adding a jointly trained second-stage trajectory refinement step, and producing a multimodal probability distribution over future actor motion that includes both multiple discrete traffic behaviors and calibrated continuous position uncertainties. The method was evaluated on large-scale, real-world data collected by a fleet of SDVs in several cities, with the results indicating that it outperforms existing state-of-the-art approaches.
△ Less
Submitted 24 May, 2021; v1 submitted 2 June, 2020;
originally announced June 2020.
-
RV-FuseNet: Range View Based Fusion of Time-Series LiDAR Data for Joint 3D Object Detection and Motion Forecasting
Authors:
Ankit Laddha,
Shivam Gautam,
Gregory P. Meyer,
Carlos Vallespi-Gonzalez,
Carl K. Wellington
Abstract:
Robust real-time detection and motion forecasting of traffic participants is necessary for autonomous vehicles to safely navigate urban environments. In this paper, we present RV-FuseNet, a novel end-to-end approach for joint detection and trajectory estimation directly from time-series LiDAR data. Instead of the widely used bird's eye view (BEV) representation, we utilize the native range view (R…
▽ More
Robust real-time detection and motion forecasting of traffic participants is necessary for autonomous vehicles to safely navigate urban environments. In this paper, we present RV-FuseNet, a novel end-to-end approach for joint detection and trajectory estimation directly from time-series LiDAR data. Instead of the widely used bird's eye view (BEV) representation, we utilize the native range view (RV) representation of LiDAR data. The RV preserves the full resolution of the sensor by avoiding the voxelization used in the BEV. Furthermore, RV can be processed efficiently due to its compactness. Previous approaches project time-series data to a common viewpoint for temporal fusion, and often this viewpoint is different from where it was captured. This is sufficient for BEV methods, but for RV methods, this can lead to loss of information and data distortion which has an adverse impact on performance. To address this challenge we propose a simple yet effective novel architecture, \textit{Incremental Fusion}, that minimizes the information loss by sequentially projecting each RV sweep into the viewpoint of the next sweep in time. We show that our approach significantly improves motion forecasting performance over the existing state-of-the-art. Furthermore, we demonstrate that our sequential fusion approach is superior to alternative RV based fusion methods on multiple datasets.
△ Less
Submitted 22 March, 2021; v1 submitted 21 May, 2020;
originally announced May 2020.
-
LaserFlow: Efficient and Probabilistic Object Detection and Motion Forecasting
Authors:
Gregory P. Meyer,
Jake Charland,
Shreyash Pandey,
Ankit Laddha,
Shivam Gautam,
Carlos Vallespi-Gonzalez,
Carl K. Wellington
Abstract:
In this work, we present LaserFlow, an efficient method for 3D object detection and motion forecasting from LiDAR. Unlike the previous work, our approach utilizes the native range view representation of the LiDAR, which enables our method to operate at the full range of the sensor in real-time without voxelization or compression of the data. We propose a new multi-sweep fusion architecture, which…
▽ More
In this work, we present LaserFlow, an efficient method for 3D object detection and motion forecasting from LiDAR. Unlike the previous work, our approach utilizes the native range view representation of the LiDAR, which enables our method to operate at the full range of the sensor in real-time without voxelization or compression of the data. We propose a new multi-sweep fusion architecture, which extracts and merges temporal features directly from the range images. Furthermore, we propose a novel technique for learning a probability distribution over future trajectories inspired by curriculum learning. We evaluate LaserFlow on two autonomous driving datasets and demonstrate competitive results when compared to the existing state-of-the-art methods.
△ Less
Submitted 15 October, 2020; v1 submitted 12 March, 2020;
originally announced March 2020.
-
3D Point Cloud Processing and Learning for Autonomous Driving
Authors:
Siheng Chen,
Baoan Liu,
Chen Feng,
Carlos Vallespi-Gonzalez,
Carl Wellington
Abstract:
We present a review of 3D point cloud processing and learning for autonomous driving. As one of the most important sensors in autonomous vehicles, light detection and ranging (LiDAR) sensors collect 3D point clouds that precisely record the external surfaces of objects and scenes. The tools for 3D point cloud processing and learning are critical to the map creation, localization, and perception mo…
▽ More
We present a review of 3D point cloud processing and learning for autonomous driving. As one of the most important sensors in autonomous vehicles, light detection and ranging (LiDAR) sensors collect 3D point clouds that precisely record the external surfaces of objects and scenes. The tools for 3D point cloud processing and learning are critical to the map creation, localization, and perception modules in an autonomous vehicle. While much attention has been paid to data collected from cameras, such as images and videos, an increasing number of researchers have recognized the importance and significance of LiDAR in autonomous driving and have proposed processing and learning algorithms to exploit 3D point clouds. We review the recent progress in this research area and summarize what has been tried and what is needed for practical and safe autonomous vehicles. We also offer perspectives on open issues that are needed to be solved in the future.
△ Less
Submitted 1 March, 2020;
originally announced March 2020.
-
LaserNet: An Efficient Probabilistic 3D Object Detector for Autonomous Driving
Authors:
Gregory P. Meyer,
Ankit Laddha,
Eric Kee,
Carlos Vallespi-Gonzalez,
Carl K. Wellington
Abstract:
In this paper, we present LaserNet, a computationally efficient method for 3D object detection from LiDAR data for autonomous driving. The efficiency results from processing LiDAR data in the native range view of the sensor, where the input data is naturally compact. Operating in the range view involves well known challenges for learning, including occlusion and scale variation, but it also provid…
▽ More
In this paper, we present LaserNet, a computationally efficient method for 3D object detection from LiDAR data for autonomous driving. The efficiency results from processing LiDAR data in the native range view of the sensor, where the input data is naturally compact. Operating in the range view involves well known challenges for learning, including occlusion and scale variation, but it also provides contextual information based on how the sensor data was captured. Our approach uses a fully convolutional network to predict a multimodal distribution over 3D boxes for each point and then it efficiently fuses these distributions to generate a prediction for each object. Experiments show that modeling each detection as a distribution rather than a single deterministic box leads to better overall detection performance. Benchmark results show that this approach has significantly lower runtime than other recent detectors and that it achieves state-of-the-art performance when compared on a large dataset that has enough data to overcome the challenges of training on the range view.
△ Less
Submitted 20 March, 2019;
originally announced March 2019.
-
Comparing Apples and Oranges: Off-Road Pedestrian Detection on the NREC Agricultural Person-Detection Dataset
Authors:
Zachary Pezzementi,
Trenton Tabor,
Peiyun Hu,
Jonathan K. Chang,
Deva Ramanan,
Carl Wellington,
Benzun P. Wisely Babu,
Herman Herman
Abstract:
Person detection from vehicles has made rapid progress recently with the advent of multiple highquality datasets of urban and highway driving, yet no large-scale benchmark is available for the same problem in off-road or agricultural environments. Here we present the NREC Agricultural Person-Detection Dataset to spur research in these environments. It consists of labeled stereo video of people in…
▽ More
Person detection from vehicles has made rapid progress recently with the advent of multiple highquality datasets of urban and highway driving, yet no large-scale benchmark is available for the same problem in off-road or agricultural environments. Here we present the NREC Agricultural Person-Detection Dataset to spur research in these environments. It consists of labeled stereo video of people in orange and apple orchards taken from two perception platforms (a tractor and a pickup truck), along with vehicle position data from RTK GPS. We define a benchmark on part of the dataset that combines a total of 76k labeled person images and 19k sampled person-free images. The dataset highlights several key challenges of the domain, including varying environment, substantial occlusion by vegetation, people in motion and in non-standard poses, and people seen from a variety of distances; meta-data are included to allow targeted evaluation of each of these effects. Finally, we present baseline detection performance results for three leading approaches from urban pedestrian detection and our own convolutional neural network approach that benefits from the incorporation of additional image context. We show that the success of existing approaches on urban data does not transfer directly to this domain.
△ Less
Submitted 26 October, 2017; v1 submitted 22 July, 2017;
originally announced July 2017.
