-
Digital twins to alleviate the need for real field data in vision-based vehicle speed detection systems
Authors:
Antonio Hernández Martínez,
Iván García Daza,
Carlos Fernández López,
David Fernández Llorca
Abstract:
Accurate vision-based speed estimation is much more cost-effective than traditional methods based on radar or LiDAR. However, it is also challenging due to the limitations of perspective projection on a discrete sensor, as well as the high sensitivity to calibration, lighting and weather conditions. Interestingly, deep learning approaches (which dominate the field of computer vision) are very limi…
▽ More
Accurate vision-based speed estimation is much more cost-effective than traditional methods based on radar or LiDAR. However, it is also challenging due to the limitations of perspective projection on a discrete sensor, as well as the high sensitivity to calibration, lighting and weather conditions. Interestingly, deep learning approaches (which dominate the field of computer vision) are very limited in this context due to the lack of available data. Indeed, obtaining video sequences of real road traffic with accurate speed values associated with each vehicle is very complex and costly, and the number of available datasets is very limited. Recently, some approaches are focusing on the use of synthetic data. However, it is still unclear how models trained on synthetic data can be effectively applied to real world conditions. In this work, we propose the use of digital-twins using CARLA simulator to generate a large dataset representative of a specific real-world camera. The synthetic dataset contains a large variability of vehicle types, colours, speeds, lighting and weather conditions. A 3D CNN model is trained on the digital twin and tested on the real sequences. Unlike previous approaches that generate multi-camera sequences, we found that the gap between the the real and the virtual conditions is a key factor in obtaining low speed estimation errors. Even with a preliminary approach, the mean absolute error obtained remains below 3km/h.
△ Less
Submitted 11 July, 2024;
originally announced July 2024.
-
MaskedFusion360: Reconstruct LiDAR Data by Querying Camera Features
Authors:
Royden Wagner,
Marvin Klemp,
Carlos Fernandez Lopez
Abstract:
In self-driving applications, LiDAR data provides accurate information about distances in 3D but lacks the semantic richness of camera data. Therefore, state-of-the-art methods for perception in urban scenes fuse data from both sensor types. In this work, we introduce a novel self-supervised method to fuse LiDAR and camera data for self-driving applications. We build upon masked autoencoders (MAEs…
▽ More
In self-driving applications, LiDAR data provides accurate information about distances in 3D but lacks the semantic richness of camera data. Therefore, state-of-the-art methods for perception in urban scenes fuse data from both sensor types. In this work, we introduce a novel self-supervised method to fuse LiDAR and camera data for self-driving applications. We build upon masked autoencoders (MAEs) and train deep learning models to reconstruct masked LiDAR data from fused LiDAR and camera features. In contrast to related methods that use birds-eye-view representations, we fuse features from dense spherical LiDAR projections and features from fish-eye camera crops with a similar field of view. Therefore, we reduce the learned spatial transformations to moderate perspective transformations and do not require additional modules to generate dense LiDAR representations. Code is available at: https://github.com/KIT-MRT/masked-fusion-360
△ Less
Submitted 12 June, 2023;
originally announced June 2023.
-
Self-supervised pseudo-colorizing of masked cells
Authors:
Royden Wagner,
Carlos Fernandez Lopez,
Christoph Stiller
Abstract:
Self-supervised learning, which is strikingly referred to as the dark matter of intelligence, is gaining more attention in biomedical applications of deep learning. In this work, we introduce a novel self-supervision objective for the analysis of cells in biomedical microscopy images. We propose training deep learning models to pseudo-colorize masked cells. We use a physics-informed pseudo-spectra…
▽ More
Self-supervised learning, which is strikingly referred to as the dark matter of intelligence, is gaining more attention in biomedical applications of deep learning. In this work, we introduce a novel self-supervision objective for the analysis of cells in biomedical microscopy images. We propose training deep learning models to pseudo-colorize masked cells. We use a physics-informed pseudo-spectral colormap that is well suited for colorizing cell topology. Our experiments reveal that approximating semantic segmentation by pseudo-colorization is beneficial for subsequent fine-tuning on cell detection. Inspired by the recent success of masked image modeling, we additionally mask out cell parts and train to reconstruct these parts to further enrich the learned representations. We compare our pre-training method with self-supervised frameworks including contrastive learning (SimCLR), masked autoencoders (MAEs), and edge-based self-supervision. We build upon our previous work and train hybrid models for cell detection, which contain both convolutional and vision transformer modules. Our pre-training method can outperform SimCLR, MAE-like masked image modeling, and edge-based self-supervision when pre-training on a diverse set of six fluorescence microscopy datasets. Code is available at: https://github.com/roydenwa/pseudo-colorize-masked-cells
△ Less
Submitted 28 August, 2023; v1 submitted 12 February, 2023;
originally announced February 2023.
-
Risk-Aware High-level Decisions for Automated Driving at Occluded Intersections with Reinforcement Learning
Authors:
Danial Kamran,
Carlos Fernandez Lopez,
Martin Lauer,
Christoph Stiller
Abstract:
Reinforcement learning is nowadays a popular framework for solving different decision making problems in automated driving. However, there are still some remaining crucial challenges that need to be addressed for providing more reliable policies. In this paper, we propose a generic risk-aware DQN approach in order to learn high level actions for driving through unsignalized occluded intersections.…
▽ More
Reinforcement learning is nowadays a popular framework for solving different decision making problems in automated driving. However, there are still some remaining crucial challenges that need to be addressed for providing more reliable policies. In this paper, we propose a generic risk-aware DQN approach in order to learn high level actions for driving through unsignalized occluded intersections. The proposed state representation provides lane based information which allows to be used for multi-lane scenarios. Moreover, we propose a risk based reward function which punishes risky situations instead of only collision failures. Such rewarding approach helps to incorporate risk prediction into our deep Q network and learn more reliable policies which are safer in challenging situations. The efficiency of the proposed approach is compared with a DQN learned with conventional collision based rewarding scheme and also with a rule-based intersection navigation policy. Evaluation results show that the proposed approach outperforms both of these methods. It provides safer actions than collision-aware DQN approach and is less overcautious than the rule-based policy.
△ Less
Submitted 9 April, 2020;
originally announced April 2020.
-
A Study of Delay Drifts on Massive MIMO Wideband Channel Models
Authors:
Carlos F. Lopez,
Cheng-Xiang Wang
Abstract:
In this paper, we study the effects of the variations of the propagation delay over large-scale antenna-arrays used in massive multiple-input multiple-output (MIMO) wideband communication systems on the statistical properties of the channel. Due to its simplicity and popularity, the Elliptical geometry-based stochastic channel model (GBSM) is employed to demonstrate new non-stationary properties o…
▽ More
In this paper, we study the effects of the variations of the propagation delay over large-scale antenna-arrays used in massive multiple-input multiple-output (MIMO) wideband communication systems on the statistical properties of the channel. Due to its simplicity and popularity, the Elliptical geometry-based stochastic channel model (GBSM) is employed to demonstrate new non-stationary properties of the channel in the frequency and spatial domains caused by the drift of delays. In addition, we show that the time of travel of multi-path components (MPCs) over large-scale arrays may result in overlooked frequency and spatial decorrelation effects. These are theoretically demonstrated by deriving the space-time-frequency correlation functions (STFCFs) of both narrowband and wideband Elliptical models. Closed-form expressions of the array-variant frequency correlation function (FCF), power delay profile (PDP), mean delay, and delay spread of single- and multi-confocal Elliptical models are derived when the angles of arrival (AOAs) are von Mises distributed. In such conditions, we find that the large dimensions of the antenna array may limit the narrowband characteristic of the single-ellipse model and alter the wideband characteristics (PDP and FCF) of the multi-confocal Elliptical channel model. Although we present and analyze numerical and simulation results for a particular GBSM, similar conclusions can be extended to other GBSMs.
△ Less
Submitted 22 March, 2018;
originally announced March 2018.
-
A novel 2D non-stationary wideband massive MIMO channel model
Authors:
C. F. Lopez,
C. -X. Wang,
R. Feng
Abstract:
In this paper, a novel two-dimensional (2D) non-stationary wideband geometry-based stochastic model (GBSM) for massive multiple-input multiple-output (MIMO) communication systems is proposed. Key characteristics of massive MIMO channels such as near field effects and cluster evolution along the array are addressed in this model. Near field effects are modelled by a second-order approximation to sp…
▽ More
In this paper, a novel two-dimensional (2D) non-stationary wideband geometry-based stochastic model (GBSM) for massive multiple-input multiple-output (MIMO) communication systems is proposed. Key characteristics of massive MIMO channels such as near field effects and cluster evolution along the array are addressed in this model. Near field effects are modelled by a second-order approximation to spherical wavefronts, i.e., parabolic wavefronts, leading to linear drifts of the angles of multipath components (MPCs) and non-stationarity along the array. Cluster evolution along the array involving cluster (dis)appearance and smooth average power variations is considered. Cluster (dis)appearance is modeled by a two-state Markov process and smooth average power variations are modelled by a spatial lognormal process. Statistical properties of the channel model such as time autocorrelation function (ACF), spatial cross-correlation function (CCF), and cluster average power and Rician factor variations over the array are derived. Finally, simulation results are presented and analyzed, demonstrating that parabolic wavefronts and cluster soft evolution are good candidates to model important massive MIMO channel characteristics.
△ Less
Submitted 2 November, 2016;
originally announced November 2016.
-
An unbiased metric of antiproliferative drug effect in vitro
Authors:
Leonard A. Harris,
Peter L. Frick,
Shawn P. Garbett,
Keisha N. Hardeman,
B. Bishal Paudel,
Carlos F. Lopez,
Vito Quaranta,
Darren R. Tyson
Abstract:
In vitro cell proliferation assays are widely used in pharmacology, molecular biology, and drug discovery. Using theoretical modeling and experimentation, we show that current antiproliferative drug effect metrics suffer from time-dependent bias, leading to inaccurate assessments of parameters such as drug potency and efficacy. We propose the drug-induced proliferation (DIP) rate, the slope of the…
▽ More
In vitro cell proliferation assays are widely used in pharmacology, molecular biology, and drug discovery. Using theoretical modeling and experimentation, we show that current antiproliferative drug effect metrics suffer from time-dependent bias, leading to inaccurate assessments of parameters such as drug potency and efficacy. We propose the drug-induced proliferation (DIP) rate, the slope of the line on a plot of cell population doublings versus time, as an alternative, time-independent metric.
△ Less
Submitted 22 May, 2016;
originally announced May 2016.
