-
Towards Real-World Deployment of Reinforcement Learning for Traffic Signal Control
Authors:
Arthur Müller,
Vishal Rangras,
Georg Schnittker,
Michael Waldmann,
Maxim Friesen,
Tobias Ferfers,
Lukas Schreckenberg,
Florian Hufen,
Jürgen Jasperneite,
Marco Wiering
Abstract:
Sub-optimal control policies in intersection traffic signal controllers (TSC) contribute to congestion and lead to negative effects on human health and the environment. Reinforcement learning (RL) for traffic signal control is a promising approach to design better control policies and has attracted considerable research interest in recent years. However, most work done in this area used simplified…
▽ More
Sub-optimal control policies in intersection traffic signal controllers (TSC) contribute to congestion and lead to negative effects on human health and the environment. Reinforcement learning (RL) for traffic signal control is a promising approach to design better control policies and has attracted considerable research interest in recent years. However, most work done in this area used simplified simulation environments of traffic scenarios to train RL-based TSC. To deploy RL in real-world traffic systems, the gap between simplified simulation environments and real-world applications has to be closed. Therefore, we propose LemgoRL, a benchmark tool to train RL agents as TSC in a realistic simulation environment of Lemgo, a medium-sized town in Germany. In addition to the realistic simulation model, LemgoRL encompasses a traffic signal logic unit that ensures compliance with all regulatory and safety requirements. LemgoRL offers the same interface as the wellknown OpenAI gym toolkit to enable easy deployment in existing research work. To demonstrate the functionality and applicability of LemgoRL, we train a state-of-the-art Deep RL algorithm on a CPU cluster utilizing a framework for distributed and parallel RL and compare its performance with other methods. Our benchmark tool drives the development of RL algorithms towards real-world applications.
△ Less
Submitted 11 January, 2022; v1 submitted 30 March, 2021;
originally announced March 2021.
-
An Agent-Based Model of Message Propagation in the Facebook Electronic Social Network
Authors:
Hamid Reza Nasrinpour,
Marcia R. Friesen,
Robert D.,
McLeod
Abstract:
A large scale agent-based model of common Facebook users was designed to develop an understanding of the underlying mechanism of information diffusion within online social networks at a micro-level analysis. The agent-based model network structure is based on a sample from Facebook. Using an erased configuration model and the idea of common neighbours, a new correction procedure was investigated t…
▽ More
A large scale agent-based model of common Facebook users was designed to develop an understanding of the underlying mechanism of information diffusion within online social networks at a micro-level analysis. The agent-based model network structure is based on a sample from Facebook. Using an erased configuration model and the idea of common neighbours, a new correction procedure was investigated to overcome the problem of missing graph edges to construct a representative sample of the Facebook network graph. The model parameters are based on assumptions and general activity patterns (such as posting rate, time spent on Facebook etc.) taken from general data on Facebook. Using the agent-based model, the impact of post length, post score and publisher's friend count on the spread of wall posts in several scenarios was analyzed. Findings indicated that post content has the highest impact on the success of post propagation. However, amusing and absorbing but lengthy posts (e.g. a funny video) do not spread as well as short but unremarkable ones (e.g. an interesting photo). In contrast to product adoption and disease spread propagation models, the absence of a similar "epidemic" threshold in Facebook post diffusion is observed.
△ Less
Submitted 22 November, 2016;
originally announced November 2016.
-
State Estimation for Tensegrity Robots
Authors:
Ken Caluwaerts,
Jonathan Bruce,
Jeffrey M. Friesen,
Vytas SunSpiral
Abstract:
Tensegrity robots are a class of compliant robots that have many desirable traits when designing mass efficient systems that must interact with uncertain environments. Various promising control approaches have been proposed for tensegrity systems in simulation. Unfortunately, state estimation methods for tensegrity robots have not yet been thoroughly studied.
In this paper, we present the design…
▽ More
Tensegrity robots are a class of compliant robots that have many desirable traits when designing mass efficient systems that must interact with uncertain environments. Various promising control approaches have been proposed for tensegrity systems in simulation. Unfortunately, state estimation methods for tensegrity robots have not yet been thoroughly studied.
In this paper, we present the design and evaluation of a state estimator for tensegrity robots. This state estimator will enable existing and future control algorithms to transfer from simulation to hardware. Our approach is based on the unscented Kalman filter (UKF) and combines inertial measurements, ultra wideband time-of-flight ranging measurements, and actuator state information.
We evaluate the effectiveness of our method on the SUPERball, a tensegrity based planetary exploration robotic prototype. In particular, we conduct tests for evaluating both the robot's success in estimating global position in relation to fixed ranging base stations during rolling maneuvers as well as local behavior due to small-amplitude deformations induced by cable actuation.
△ Less
Submitted 19 February, 2016; v1 submitted 5 October, 2015;
originally announced October 2015.
-
GPU accelerated Nature Inspired Methods for Modelling Large Scale Bi-Directional Pedestrian Movement
Authors:
Sankha Baran Dutta,
Robert McLeod,
Marcia Friesen
Abstract:
Pedestrian movement, although ubiquitous and well-studied, is still not that well understood due to the complicating nature of the embedded social dynamics. Interest among researchers in simulating pedestrian movement and interactions has grown significantly in part due to increased computational and visualization capabilities afforded by high power computing. Different approaches have been adopte…
▽ More
Pedestrian movement, although ubiquitous and well-studied, is still not that well understood due to the complicating nature of the embedded social dynamics. Interest among researchers in simulating pedestrian movement and interactions has grown significantly in part due to increased computational and visualization capabilities afforded by high power computing. Different approaches have been adopted to simulate pedestrian movement under various circumstances and interactions. In the present work, bi-directional crowd movement is simulated where an equal numbers of individuals try to reach the opposite sides of an environment. Two movement methods are considered. First a Least Effort Model (LEM) is investigated where agents try to take an optimal path with as minimal changes from their intended path as possible. Following this, a modified form of Ant Colony Optimization (ACO) is proposed, where individuals are guided by a goal of reaching the other side in a least effort mode as well as a pheromone trail left by predecessors. The basic idea is to increase agent interaction, thereby more closely reflecting a real world scenario. The methodology utilizes Graphics Processing Units (GPUs) for general purpose computing using the CUDA platform. Because of the inherent parallel properties associated with pedestrian movement such as proximate interactions of individuals on a 2D grid, GPUs are well suited. The main feature of the implementation undertaken here is that the parallelism is data driven. The data driven implementation leads to a speedup up to 18x compared to its sequential counterpart running on a single threaded CPU. The numbers of pedestrians considered in the model ranged from 2K to 100K representing numbers typical of mass gathering events. A detailed discussion addresses implementation challenges faced and averted.
△ Less
Submitted 16 December, 2014;
originally announced December 2014.
-
Fooling sets and rank
Authors:
Mirjam Friesen,
Aya Hamed,
Troy Lee,
Dirk Oliver Theis
Abstract:
An $n\times n$ matrix $M$ is called a \textit{fooling-set matrix of size $n$} if its diagonal entries are nonzero and $M_{k,\ell} M_{\ell,k} = 0$ for every $k\ne \ell$. Dietzfelbinger, Hromkovi{č}, and Schnitger (1996) showed that $n \le (\mbox{rk} M)^2$, regardless of over which field the rank is computed, and asked whether the exponent on $\mbox{rk} M$ can be improved.
We settle this question.…
▽ More
An $n\times n$ matrix $M$ is called a \textit{fooling-set matrix of size $n$} if its diagonal entries are nonzero and $M_{k,\ell} M_{\ell,k} = 0$ for every $k\ne \ell$. Dietzfelbinger, Hromkovi{č}, and Schnitger (1996) showed that $n \le (\mbox{rk} M)^2$, regardless of over which field the rank is computed, and asked whether the exponent on $\mbox{rk} M$ can be improved.
We settle this question. In characteristic zero, we construct an infinite family of rational fooling-set matrices with size $n = \binom{\mbox{rk} M+1}{2}$. In nonzero characteristic, we construct an infinite family of matrices with $n= (1+o(1))(\mbox{rk} M)^2$.
△ Less
Submitted 16 January, 2014; v1 submitted 14 August, 2012;
originally announced August 2012.
