-
Reimagining an autonomous vehicle
Authors:
Jeffrey Hawke,
Haibo E,
Vijay Badrinarayanan,
Alex Kendall
Abstract:
The self driving challenge in 2021 is this century's technological equivalent of the space race, and is now entering the second major decade of development. Solving the technology will create social change which parallels the invention of the automobile itself. Today's autonomous driving technology is laudable, though rooted in decisions made a decade ago. We argue that a rethink is required, reco…
▽ More
The self driving challenge in 2021 is this century's technological equivalent of the space race, and is now entering the second major decade of development. Solving the technology will create social change which parallels the invention of the automobile itself. Today's autonomous driving technology is laudable, though rooted in decisions made a decade ago. We argue that a rethink is required, reconsidering the autonomous vehicle (AV) problem in the light of the body of knowledge that has been gained since the DARPA challenges which seeded the industry. What does AV2.0 look like? We present an alternative vision: a recipe for driving with machine learning, and grand challenges for research in driving.
△ Less
Submitted 12 August, 2021;
originally announced August 2021.
-
FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Cameras
Authors:
Anthony Hu,
Zak Murez,
Nikhil Mohan,
Sofía Dudas,
Jeffrey Hawke,
Vijay Badrinarayanan,
Roberto Cipolla,
Alex Kendall
Abstract:
Driving requires interacting with road agents and predicting their future behaviour in order to navigate safely. We present FIERY: a probabilistic future prediction model in bird's-eye view from monocular cameras. Our model predicts future instance segmentation and motion of dynamic agents that can be transformed into non-parametric future trajectories. Our approach combines the perception, sensor…
▽ More
Driving requires interacting with road agents and predicting their future behaviour in order to navigate safely. We present FIERY: a probabilistic future prediction model in bird's-eye view from monocular cameras. Our model predicts future instance segmentation and motion of dynamic agents that can be transformed into non-parametric future trajectories. Our approach combines the perception, sensor fusion and prediction components of a traditional autonomous driving stack by estimating bird's-eye-view prediction directly from surround RGB monocular camera inputs. FIERY learns to model the inherent stochastic nature of the future solely from camera driving data in an end-to-end manner, without relying on HD maps, and predicts multimodal future trajectories. We show that our model outperforms previous prediction baselines on the NuScenes and Lyft datasets. The code and trained models are available at https://github.com/wayveai/fiery.
△ Less
Submitted 18 October, 2021; v1 submitted 21 April, 2021;
originally announced April 2021.
-
Urban Driving with Conditional Imitation Learning
Authors:
Jeffrey Hawke,
Richard Shen,
Corina Gurau,
Siddharth Sharma,
Daniele Reda,
Nikolay Nikolov,
Przemyslaw Mazur,
Sean Micklethwaite,
Nicolas Griffiths,
Amar Shah,
Alex Kendall
Abstract:
Hand-crafting generalised decision-making rules for real-world urban autonomous driving is hard. Alternatively, learning behaviour from easy-to-collect human driving demonstrations is appealing. Prior work has studied imitation learning (IL) for autonomous driving with a number of limitations. Examples include only performing lane-following rather than following a user-defined route, only using a…
▽ More
Hand-crafting generalised decision-making rules for real-world urban autonomous driving is hard. Alternatively, learning behaviour from easy-to-collect human driving demonstrations is appealing. Prior work has studied imitation learning (IL) for autonomous driving with a number of limitations. Examples include only performing lane-following rather than following a user-defined route, only using a single camera view or heavily cropped frames lacking state observability, only lateral (steering) control, but not longitudinal (speed) control and a lack of interaction with traffic. Importantly, the majority of such systems have been primarily evaluated in simulation - a simple domain, which lacks real-world complexities. Motivated by these challenges, we focus on learning representations of semantics, geometry and motion with computer vision for IL from human driving demonstrations. As our main contribution, we present an end-to-end conditional imitation learning approach, combining both lateral and longitudinal control on a real vehicle for following urban routes with simple traffic. We address inherent dataset bias by data balancing, training our final policy on approximately 30 hours of demonstrations gathered over six months. We evaluate our method on an autonomous vehicle by driving 35km of novel routes in European urban streets.
△ Less
Submitted 5 December, 2019; v1 submitted 30 November, 2019;
originally announced December 2019.
-
Learning to Drive from Simulation without Real World Labels
Authors:
Alex Bewley,
Jessica Rigley,
Yuxuan Liu,
Jeffrey Hawke,
Richard Shen,
Vinh-Dieu Lam,
Alex Kendall
Abstract:
Simulation can be a powerful tool for understanding machine learning systems and designing methods to solve real-world problems. Training and evaluating methods purely in simulation is often "doomed to succeed" at the desired task in a simulated environment, but the resulting models are incapable of operation in the real world. Here we present and evaluate a method for transferring a vision-based…
▽ More
Simulation can be a powerful tool for understanding machine learning systems and designing methods to solve real-world problems. Training and evaluating methods purely in simulation is often "doomed to succeed" at the desired task in a simulated environment, but the resulting models are incapable of operation in the real world. Here we present and evaluate a method for transferring a vision-based lane following driving policy from simulation to operation on a rural road without any real-world labels. Our approach leverages recent advances in image-to-image translation to achieve domain transfer while jointly learning a single-camera control policy from simulation control labels. We assess the driving performance of this method using both open-loop regression metrics, and closed-loop performance operating an autonomous vehicle on rural and urban roads.
△ Less
Submitted 13 December, 2018; v1 submitted 10 December, 2018;
originally announced December 2018.
-
Learning to Drive in a Day
Authors:
Alex Kendall,
Jeffrey Hawke,
David Janz,
Przemyslaw Mazur,
Daniele Reda,
John-Mark Allen,
Vinh-Dieu Lam,
Alex Bewley,
Amar Shah
Abstract:
We demonstrate the first application of deep reinforcement learning to autonomous driving. From randomly initialised parameters, our model is able to learn a policy for lane following in a handful of training episodes using a single monocular image as input. We provide a general and easy to obtain reward: the distance travelled by the vehicle without the safety driver taking control. We use a cont…
▽ More
We demonstrate the first application of deep reinforcement learning to autonomous driving. From randomly initialised parameters, our model is able to learn a policy for lane following in a handful of training episodes using a single monocular image as input. We provide a general and easy to obtain reward: the distance travelled by the vehicle without the safety driver taking control. We use a continuous, model-free deep reinforcement learning algorithm, with all exploration and optimisation performed on-vehicle. This demonstrates a new framework for autonomous driving which moves away from reliance on defined logical rules, map**, and direct supervision. We discuss the challenges and opportunities to scale this approach to a broader range of autonomous driving tasks.
△ Less
Submitted 11 September, 2018; v1 submitted 1 July, 2018;
originally announced July 2018.
-
What Makes a Place? Building Bespoke Place Dependent Object Detectors for Robotics
Authors:
Jeffrey Hawke,
Alex Bewley,
Ingmar Posner
Abstract:
This paper is about enabling robots to improve their perceptual performance through repeated use in their operating environment, creating local expert detectors fitted to the places through which a robot moves. We leverage the concept of 'experiences' in visual perception for robotics, accounting for bias in the data a robot sees by fitting object detector models to a particular place. The key que…
▽ More
This paper is about enabling robots to improve their perceptual performance through repeated use in their operating environment, creating local expert detectors fitted to the places through which a robot moves. We leverage the concept of 'experiences' in visual perception for robotics, accounting for bias in the data a robot sees by fitting object detector models to a particular place. The key question we seek to answer in this paper is simply: how do we define a place? We build bespoke pedestrian detector models for autonomous driving, highlighting the necessary trade off between generalisation and model capacity as we vary the extent of the place we fit to. We demonstrate a sizeable performance gain over a current state-of-the-art detector when using computationally lightweight bespoke place-fitted detector models.
△ Less
Submitted 7 August, 2017;
originally announced August 2017.
-
Impact of Resonant Magnetic Perturbations on the L-H Transition on MAST
Authors:
R. Scannell,
A. Kirk,
M. Carr,
J. Hawke,
S. S. Henderson,
T. O'Gorman,
A. Patel,
A. Shaw,
A. Thornton,
the MAST Team
Abstract:
The impact of resonant magnetic perturbations (RMPs) on the power required to access H-mode is examined experimentally on MAST. Applying RMP in n=2,3,4 and 6 configurations causes significant delays to the timing of the L-H transition at low applied fields and prevents the transition at high fields. The experiment was primarily performed at RMP fields sufficient to cause moderate increases in ELM…
▽ More
The impact of resonant magnetic perturbations (RMPs) on the power required to access H-mode is examined experimentally on MAST. Applying RMP in n=2,3,4 and 6 configurations causes significant delays to the timing of the L-H transition at low applied fields and prevents the transition at high fields. The experiment was primarily performed at RMP fields sufficient to cause moderate increases in ELM frequency, f mitigated/f natural~3. To obtain H-mode with RMPs at this field, an increase of injected beam power is required of at least 50% for n=3 and n=4 RMP and 100% for n=6 RMP. In terms of power threshold, this corresponds to increases of at least 20% for n=3 and n=4 RMPs and 60% for n=6 RMPs. This 'RMP affected' power threshold is found to increase with RMP magnitude above a certain minimum perturbed field, below which there is no impact on the power threshold. Extrapolations from these results indicate large increases in the L-H power threshold will be required for discharges requiring large mitigated ELM frequency.
△ Less
Submitted 12 December, 2014;
originally announced December 2014.
