AdCraft: An Advanced Reinforcement Learning Benchmark Environment for Search Engine Marketing Optimization
Authors:
Maziar Gomrokchi,
Owen Levin,
Jeffrey Roach,
Jonah White
Abstract:
We introduce AdCraft, a novel benchmark environment for the Reinforcement Learning (RL) community distinguished by its stochastic and non-stationary properties. The environment simulates bidding and budgeting dynamics within Search Engine Marketing (SEM), a digital marketing technique utilizing paid advertising to enhance the visibility of websites on search engine results pages (SERPs). The perfo…
▽ More
We introduce AdCraft, a novel benchmark environment for the Reinforcement Learning (RL) community distinguished by its stochastic and non-stationary properties. The environment simulates bidding and budgeting dynamics within Search Engine Marketing (SEM), a digital marketing technique utilizing paid advertising to enhance the visibility of websites on search engine results pages (SERPs). The performance of SEM advertisement campaigns depends on several factors, including keyword selection, ad design, bid management, budget adjustments, and performance monitoring. Deep RL recently emerged as a potential strategy to optimize campaign profitability within the complex and dynamic landscape of SEM, but it requires substantial data, which may be costly or infeasible to acquire in practice. Our customizable environment enables practitioners to assess and enhance the robustness of RL algorithms pertinent to SEM bid and budget management without such costs. Through a series of experiments within the environment, we demonstrate the challenges imposed on agent convergence and performance by sparsity and non-stationarity. We hope these challenges further encourage discourse and development around effective strategies for managing real-world uncertainties.
△ Less
Submitted 14 November, 2023; v1 submitted 20 June, 2023;
originally announced June 2023.
The NVIDIA PilotNet Experiments
Authors:
Mariusz Bojarski,
Chenyi Chen,
Joyjit Daw,
Alperen Değirmenci,
Joya Deri,
Bernhard Firner,
Beat Flepp,
Sachin Gogri,
Jesse Hong,
Lawrence Jackel,
Zhenhua Jia,
BJ Lee,
Bo Liu,
Fei Liu,
Urs Muller,
Samuel Payne,
Nischal Kota Nagendra Prasad,
Artem Provodin,
John Roach,
Timur Rvachov,
Neha Tadimeti,
Jesper van Engelen,
Haiguang Wen,
Eric Yang,
Zongyi Yang
Abstract:
Four years ago, an experimental system known as PilotNet became the first NVIDIA system to steer an autonomous car along a roadway. This system represents a departure from the classical approach for self-driving in which the process is manually decomposed into a series of modules, each performing a different task. In PilotNet, on the other hand, a single deep neural network (DNN) takes pixels as i…
▽ More
Four years ago, an experimental system known as PilotNet became the first NVIDIA system to steer an autonomous car along a roadway. This system represents a departure from the classical approach for self-driving in which the process is manually decomposed into a series of modules, each performing a different task. In PilotNet, on the other hand, a single deep neural network (DNN) takes pixels as input and produces a desired vehicle trajectory as output; there are no distinct internal modules connected by human-designed interfaces. We believe that handcrafted interfaces ultimately limit performance by restricting information flow through the system and that a learned approach, in combination with other artificial intelligence systems that add redundancy, will lead to better overall performing systems. We continue to conduct research toward that goal.
This document describes the PilotNet lane-kee** effort, carried out over the past five years by our NVIDIA PilotNet group in Holmdel, New Jersey. Here we present a snapshot of system status in mid-2020 and highlight some of the work done by the PilotNet group.
△ Less
Submitted 17 October, 2020;
originally announced October 2020.
