-
How Many Parameters Does it Take to Change a Light Bulb? Evaluating Performance in Self-Play of Conversational Games as a Function of Model Characteristics
Authors:
Nidhir Bhavsar,
Jonathan Jordan,
Sherzod Hakimov,
David Schlangen
Abstract:
What makes a good Large Language Model (LLM)? That it performs well on the relevant benchmarks -- which hopefully measure, with some validity, the presence of capabilities that are also challenged in real application. But what makes the model perform well? What gives a model its abilities? We take a recently introduced type of benchmark that is meant to challenge capabilities in a goal-directed, a…
▽ More
What makes a good Large Language Model (LLM)? That it performs well on the relevant benchmarks -- which hopefully measure, with some validity, the presence of capabilities that are also challenged in real application. But what makes the model perform well? What gives a model its abilities? We take a recently introduced type of benchmark that is meant to challenge capabilities in a goal-directed, agentive context through self-play of conversational games, and analyse how performance develops as a function of model characteristics like number of parameters, or type of training. We find that while there is a clear relationship between number of parameters and performance, there is still a wide spread of performance points within a given size bracket, which is to be accounted for by training parameters such as fine-tuning data quality and method. From a more practical angle, we also find a certain degree of unpredictability about performance across access methods, possible due to unexposed sampling parameters, and a, very welcome, performance stability against at least moderate weight quantisation during inference.
△ Less
Submitted 20 June, 2024;
originally announced June 2024.
-
Backpropagation through space, time, and the brain
Authors:
Benjamin Ellenberger,
Paul Haider,
Jakob Jordan,
Kevin Max,
Ismael Jaras,
Laura Kriener,
Federico Benitez,
Mihai A. Petrovici
Abstract:
Effective learning in neuronal networks requires the adaptation of individual synapses given their relative contribution to solving a task. However, physical neuronal systems -- whether biological or artificial -- are constrained by spatio-temporal locality. How such networks can perform efficient credit assignment, remains, to a large extent, an open question. In Machine Learning, the answer is a…
▽ More
Effective learning in neuronal networks requires the adaptation of individual synapses given their relative contribution to solving a task. However, physical neuronal systems -- whether biological or artificial -- are constrained by spatio-temporal locality. How such networks can perform efficient credit assignment, remains, to a large extent, an open question. In Machine Learning, the answer is almost universally given by the error backpropagation algorithm, through both space (BP) and time (BPTT). However, BP(TT) is well-known to rely on biologically implausible assumptions, in particular with respect to spatiotemporal (non-)locality, while forward-propagation models such as real-time recurrent learning (RTRL) suffer from prohibitive memory constraints. We introduce Generalized Latent Equilibrium (GLE), a computational framework for fully local spatio-temporal credit assignment in physical, dynamical networks of neurons. We start by defining an energy based on neuron-local mismatches, from which we derive both neuronal dynamics via stationarity and parameter dynamics via gradient descent. The resulting dynamics can be interpreted as a real-time, biologically plausible approximation of BPTT in deep cortical networks with continuous-time neuronal dynamics and continuously active, local synaptic plasticity. In particular, GLE exploits the ability of biological neurons to phase-shift their output rate with respect to their membrane potential, which is essential in both directions of information propagation. For the forward computation, it enables the map** of time-continuous inputs to neuronal space, performing an effective spatiotemporal convolution. For the backward computation, it permits the temporal inversion of feedback signals, which consequently approximate the adjoint states necessary for useful parameter updates.
△ Less
Submitted 25 March, 2024;
originally announced March 2024.
-
Learning beyond sensations: how dreams organize neuronal representations
Authors:
Nicolas Deperrois,
Mihai A. Petrovici,
Walter Senn,
Jakob Jordan
Abstract:
Semantic representations in higher sensory cortices form the basis for robust, yet flexible behavior. These representations are acquired over the course of development in an unsupervised fashion and continuously maintained over an organism's lifespan. Predictive learning theories propose that these representations emerge from predicting or reconstructing sensory inputs. However, brains are known t…
▽ More
Semantic representations in higher sensory cortices form the basis for robust, yet flexible behavior. These representations are acquired over the course of development in an unsupervised fashion and continuously maintained over an organism's lifespan. Predictive learning theories propose that these representations emerge from predicting or reconstructing sensory inputs. However, brains are known to generate virtual experiences, such as during imagination and dreaming, that go beyond previously experienced inputs. Here, we suggest that virtual experiences may be just as relevant as actual sensory inputs in sha** cortical representations. In particular, we discuss two complementary learning principles that organize representations through the generation of virtual experiences. First, "adversarial dreaming" proposes that creative dreams support a cortical implementation of adversarial learning in which feedback and feedforward pathways engage in a productive game of trying to fool each other. Second, "contrastive dreaming" proposes that the invariance of neuronal representations to irrelevant factors of variation is acquired by trying to map similar virtual experiences together via a contrastive learning process. These principles are compatible with known cortical structure and dynamics and the phenomenology of sleep thus providing promising directions to explain cortical learning beyond the classical predictive learning paradigm.
△ Less
Submitted 5 December, 2023; v1 submitted 3 August, 2023;
originally announced August 2023.
-
Quasi Real-Time Autonomous Satellite Detection and Orbit Estimation
Authors:
Jarred Jordan,
Daniel Posada,
Matthew Gillette,
David Zuehlke,
Troy Henderson
Abstract:
A method of near real-time detection and tracking of resident space objects (RSOs) using a convolutional neural network (CNN) and linear quadratic estimator (LQE) is proposed. Advances in machine learning architecture allow the use of low-power/cost embedded devices to perform complex classification tasks. In order to reduce the costs of tracking systems, a low-cost embedded device will be used to…
▽ More
A method of near real-time detection and tracking of resident space objects (RSOs) using a convolutional neural network (CNN) and linear quadratic estimator (LQE) is proposed. Advances in machine learning architecture allow the use of low-power/cost embedded devices to perform complex classification tasks. In order to reduce the costs of tracking systems, a low-cost embedded device will be used to run a CNN detection model for RSOs in unresolved images captured by a gray-scale camera and small telescope. Detection results computed in near real-time are then passed to an LQE to compute tracking updates for the telescope mount, resulting in a fully autonomous method of optical RSO detection and tracking. Keywords: Space Domain Awareness, Neural Networks, Real-Time, Object Detection, Embedded Systems.
△ Less
Submitted 12 April, 2023;
originally announced April 2023.
-
LMR: Lane Distance-Based Metric for Trajectory Prediction
Authors:
Julian Schmidt,
Thomas Monninger,
Julian Jordan,
Klaus Dietmayer
Abstract:
The development of approaches for trajectory prediction requires metrics to validate and compare their performance. Currently established metrics are based on Euclidean distance, which means that errors are weighted equally in all directions. Euclidean metrics are insufficient for structured environments like roads, since they do not properly capture the agent's intent relative to the underlying l…
▽ More
The development of approaches for trajectory prediction requires metrics to validate and compare their performance. Currently established metrics are based on Euclidean distance, which means that errors are weighted equally in all directions. Euclidean metrics are insufficient for structured environments like roads, since they do not properly capture the agent's intent relative to the underlying lane. In order to provide a reasonable assessment of trajectory prediction approaches with regard to the downstream planning task, we propose a new metric that is lane distance-based: Lane Miss Rate (LMR). For the calculation of LMR, the ground-truth and predicted endpoints are assigned to lane segments, more precisely their centerlines. Measured by the distance along the lane segments, predictions that are within a certain threshold distance to the ground-truth count as hits, otherwise they count as misses. LMR is then defined as the ratio of sequences that yield a miss. Our results on three state-of-the-art trajectory prediction models show that LMR preserves the order of Euclidean distance-based metrics. In contrast to the Euclidean Miss Rate, qualitative results show that LMR yields misses for sequences where predictions are located on wrong lanes. Hits on the other hand result for sequences where predictions are located on the correct lane. This means that LMR implicitly weights Euclidean error relative to the lane and goes into the direction of capturing intents of traffic agents. The source code of LMR for Argoverse 2 is publicly available.
△ Less
Submitted 13 April, 2023; v1 submitted 12 April, 2023;
originally announced April 2023.
-
RESET: Revisiting Trajectory Sets for Conditional Behavior Prediction
Authors:
Julian Schmidt,
Pascal Huissel,
Julian Wiederer,
Julian Jordan,
Vasileios Belagiannis,
Klaus Dietmayer
Abstract:
It is desirable to predict the behavior of traffic participants conditioned on different planned trajectories of the autonomous vehicle. This allows the downstream planner to estimate the impact of its decisions. Recent approaches for conditional behavior prediction rely on a regression decoder, meaning that coordinates or polynomial coefficients are regressed. In this work we revisit set-based tr…
▽ More
It is desirable to predict the behavior of traffic participants conditioned on different planned trajectories of the autonomous vehicle. This allows the downstream planner to estimate the impact of its decisions. Recent approaches for conditional behavior prediction rely on a regression decoder, meaning that coordinates or polynomial coefficients are regressed. In this work we revisit set-based trajectory prediction, where the probability of each trajectory in a predefined trajectory set is determined by a classification model, and first-time employ it to the task of conditional behavior prediction. We propose RESET, which combines a new metric-driven algorithm for trajectory set generation with a graph-based encoder. For unconditional prediction, RESET achieves comparable performance to a regression-based approach. Due to the nature of set-based approaches, it has the advantageous property of being able to predict a flexible number of trajectories without influencing runtime or complexity. For conditional prediction, RESET achieves reasonable results with late fusion of the planned trajectory, which was not observed for regression-based approaches before. This means that RESET is computationally lightweight to combine with a planner that proposes multiple future plans of the autonomous vehicle, as large parts of the forward pass can be reused.
△ Less
Submitted 12 April, 2023;
originally announced April 2023.
-
Exploring Navigation Maps for Learning-Based Motion Prediction
Authors:
Julian Schmidt,
Julian Jordan,
Franz Gritschneder,
Thomas Monninger,
Klaus Dietmayer
Abstract:
The prediction of surrounding agents' motion is a key for safe autonomous driving. In this paper, we explore navigation maps as an alternative to the predominant High Definition (HD) maps for learning-based motion prediction. Navigation maps provide topological and geometrical information on road-level, HD maps additionally have centimeter-accurate lane-level information. As a result, HD maps are…
▽ More
The prediction of surrounding agents' motion is a key for safe autonomous driving. In this paper, we explore navigation maps as an alternative to the predominant High Definition (HD) maps for learning-based motion prediction. Navigation maps provide topological and geometrical information on road-level, HD maps additionally have centimeter-accurate lane-level information. As a result, HD maps are costly and time-consuming to obtain, while navigation maps with near-global coverage are freely available. We describe an approach to integrate navigation maps into learning-based motion prediction models. To exploit locally available HD maps during training, we additionally propose a model-agnostic method for knowledge distillation. In experiments on the publicly available Argoverse dataset with navigation maps obtained from OpenStreetMap, our approach shows a significant improvement over not using a map at all. Combined with our method for knowledge distillation, we achieve results that are close to the original HD map-reliant models. Our publicly available navigation map API for Argoverse enables researchers to develop and evaluate their own approaches using navigation maps.
△ Less
Submitted 13 February, 2023;
originally announced February 2023.
-
SCENE: Reasoning about Traffic Scenes using Heterogeneous Graph Neural Networks
Authors:
Thomas Monninger,
Julian Schmidt,
Jan Rupprecht,
David Raba,
Julian Jordan,
Daniel Frank,
Steffen Staab,
Klaus Dietmayer
Abstract:
Understanding traffic scenes requires considering heterogeneous information about dynamic agents and the static infrastructure. In this work we propose SCENE, a methodology to encode diverse traffic scenes in heterogeneous graphs and to reason about these graphs using a heterogeneous Graph Neural Network encoder and task-specific decoders. The heterogeneous graphs, whose structures are defined by…
▽ More
Understanding traffic scenes requires considering heterogeneous information about dynamic agents and the static infrastructure. In this work we propose SCENE, a methodology to encode diverse traffic scenes in heterogeneous graphs and to reason about these graphs using a heterogeneous Graph Neural Network encoder and task-specific decoders. The heterogeneous graphs, whose structures are defined by an ontology, consist of different nodes with type-specific node features and different relations with type-specific edge features. In order to exploit all the information given by these graphs, we propose to use cascaded layers of graph convolution. The result is an encoding of the scene. Task-specific decoders can be applied to predict desired attributes of the scene. Extensive evaluation on two diverse binary node classification tasks show the main strength of this methodology: despite being generic, it even manages to outperform task-specific baselines. The further application of our methodology to the task of node classification in various knowledge graphs shows its transferability to other domains.
△ Less
Submitted 9 January, 2023;
originally announced January 2023.
-
Three-Dimensional Coherent Diffractive Imaging of Isolated Faceted Nanostructures
Authors:
Alessandro Colombo,
Simon Dold,
Patrice Kolb,
Nils Bernhardt,
Patrick Behrens,
Jonathan Correa,
Stefan Düsterer,
Benjamin Erk,
Linos Hecht,
Andrea Heilrath,
Robert Irsig,
Norman Iwe,
Jakob Jordan,
Björn Kruse,
Bruno Langbehn,
Bastian Manschwetus,
Franklin Martinez,
Karl-Heinz Meiwes-Broer,
Kevin Oldenburg,
Christopher Passow,
Christian Peltz,
Mario Sauppe,
Fabian Seel,
Rico Mayro P. Tanyag,
Rolf Treusch
, et al. (7 additional authors not shown)
Abstract:
The structure and dynamics of isolated nanosamples in free flight can be directly visualized via single-shot coherent diffractive imaging using the intense and short pulses of X-ray free-electron lasers. Wide-angle scattering images even encode three-dimensional morphological information of the samples, but the retrieval of this information remains a challenge. Up to now, effective three-dimension…
▽ More
The structure and dynamics of isolated nanosamples in free flight can be directly visualized via single-shot coherent diffractive imaging using the intense and short pulses of X-ray free-electron lasers. Wide-angle scattering images even encode three-dimensional morphological information of the samples, but the retrieval of this information remains a challenge. Up to now, effective three-dimensional morphology reconstructions from single shots were only achieved via fitting with highly constrained models, requiring a priori knowledge about possible geometrical shapes. Here we present a much more generic imaging approach. Relying on a model that allows for any sample morphology described by a convex polyhedron, we reconstruct wide-angle diffraction patterns from individual silver nanoparticles. In addition to known structural motives with high symmetries, we retrieve imperfect shapes and agglomerates which were not accessible previously. Our results open new routes towards true 3D structure determination of single nanoparticles and, ultimately, 3D movies of ultrafast nanoscale dynamics.
△ Less
Submitted 8 August, 2022;
originally announced August 2022.
-
RGB-D Robotic Pose Estimation For a Servicing Robotic Arm
Authors:
Jared Herron,
Daniel Lopez,
Jarred Jordan,
Jillian Rudy,
Aryslan Malik,
Daniel Posada,
Mehran Andalibi,
Troy Henderson
Abstract:
A large number of robotic and human-assisted missions to the Moon and Mars are forecast. NASA's efforts to learn about the geology and makeup of these celestial bodies rely heavily on the use of robotic arms. The safety and redundancy aspects will be crucial when humans will be working alongside the robotic explorers. Additionally, robotic arms are crucial to satellite servicing and planned orbit…
▽ More
A large number of robotic and human-assisted missions to the Moon and Mars are forecast. NASA's efforts to learn about the geology and makeup of these celestial bodies rely heavily on the use of robotic arms. The safety and redundancy aspects will be crucial when humans will be working alongside the robotic explorers. Additionally, robotic arms are crucial to satellite servicing and planned orbit debris mitigation missions. The goal of this work is to create a custom Computer Vision (CV) based Artificial Neural Network (ANN) that would be able to rapidly identify the posture of a 7 Degree of Freedom (DoF) robotic arm from a single (RGB-D) image - just like humans can easily identify if an arm is pointing in some general direction. The Sawyer robotic arm is used for develo** and training this intelligent algorithm. Since Sawyer's joint space spans 7 dimensions, it is an insurmountable task to cover the entire joint configuration space. In this work, orthogonal arrays are used, similar to the Taguchi method, to efficiently span the joint space with the minimal number of training images. This ``optimally'' generated database is used to train the custom ANN and its degree of accuracy is on average equal to twice the smallest joint displacement step used for database generation. A pre-trained ANN will be useful for estimating the postures of robotic manipulators used on space stations, spacecraft, and rovers as an auxiliary tool or for contingency plans.
△ Less
Submitted 23 July, 2022;
originally announced July 2022.
-
Detection and Initial Assessment of Lunar Landing Sites Using Neural Networks
Authors:
Daniel Posada,
Jarred Jordan,
Angelica Radulovic,
Lillian Hong,
Aryslan Malik,
Troy Henderson
Abstract:
Robotic and human lunar landings are a focus of future NASA missions. Precision landing capabilities are vital to guarantee the success of the mission, and the safety of the lander and crew. During the approach to the surface there are multiple challenges associated with Hazard Relative Navigation to ensure safe landings. This paper will focus on a passive autonomous hazard detection and avoidance…
▽ More
Robotic and human lunar landings are a focus of future NASA missions. Precision landing capabilities are vital to guarantee the success of the mission, and the safety of the lander and crew. During the approach to the surface there are multiple challenges associated with Hazard Relative Navigation to ensure safe landings. This paper will focus on a passive autonomous hazard detection and avoidance sub-system to generate an initial assessment of possible landing regions for the guidance system. The system uses a single camera and the MobileNetV2 neural network architecture to detect and discern between safe landing sites and hazards such as rocks, shadows, and craters. Then a monocular structure from motion will recreate the surface to provide slope and roughness analysis.
△ Less
Submitted 23 July, 2022;
originally announced July 2022.
-
Satellite Detection in Unresolved Space Imagery for Space Domain Awareness Using Neural Networks
Authors:
Jarred Jordan,
Daniel Posada,
David Zuehlke,
Angelica Radulovic,
Aryslan Malik,
Troy Henderson
Abstract:
This work utilizes a MobileNetV2 Convolutional Neural Network (CNN) for fast, mobile detection of satellites, and rejection of stars, in cluttered unresolved space imagery. First, a custom database is created using imagery from a synthetic satellite image program and labeled with bounding boxes over satellites for "satellite-positive" images. The CNN is then trained on this database and the infere…
▽ More
This work utilizes a MobileNetV2 Convolutional Neural Network (CNN) for fast, mobile detection of satellites, and rejection of stars, in cluttered unresolved space imagery. First, a custom database is created using imagery from a synthetic satellite image program and labeled with bounding boxes over satellites for "satellite-positive" images. The CNN is then trained on this database and the inference is validated by checking the accuracy of the model on an external dataset constructed of real telescope imagery. In doing so, the trained CNN provides a method of rapid satellite identification for subsequent utilization in ground-based orbit estimation.
△ Less
Submitted 23 July, 2022;
originally announced July 2022.
-
MEAT: Maneuver Extraction from Agent Trajectories
Authors:
Julian Schmidt,
Julian Jordan,
David Raba,
Tobias Welz,
Klaus Dietmayer
Abstract:
Advances in learning-based trajectory prediction are enabled by large-scale datasets. However, in-depth analysis of such datasets is limited. Moreover, the evaluation of prediction models is limited to metrics averaged over all samples in the dataset. We propose an automated methodology that allows to extract maneuvers (e.g., left turn, lane change) from agent trajectories in such datasets. The me…
▽ More
Advances in learning-based trajectory prediction are enabled by large-scale datasets. However, in-depth analysis of such datasets is limited. Moreover, the evaluation of prediction models is limited to metrics averaged over all samples in the dataset. We propose an automated methodology that allows to extract maneuvers (e.g., left turn, lane change) from agent trajectories in such datasets. The methodology considers information about the agent dynamics and information about the lane segments the agent traveled along. Although it is possible to use the resulting maneuvers for training classification networks, we exemplary use them for extensive trajectory dataset analysis and maneuver-specific evaluation of multiple state-of-the-art trajectory prediction models. Additionally, an analysis of the datasets and an evaluation of the prediction models based on the agent dynamics is provided.
△ Less
Submitted 10 June, 2022;
originally announced June 2022.
-
CRAT-Pred: Vehicle Trajectory Prediction with Crystal Graph Convolutional Neural Networks and Multi-Head Self-Attention
Authors:
Julian Schmidt,
Julian Jordan,
Franz Gritschneder,
Klaus Dietmayer
Abstract:
Predicting the motion of surrounding vehicles is essential for autonomous vehicles, as it governs their own motion plan. Current state-of-the-art vehicle prediction models heavily rely on map information. In reality, however, this information is not always available. We therefore propose CRAT-Pred, a multi-modal and non-rasterization-based trajectory prediction model, specifically designed to effe…
▽ More
Predicting the motion of surrounding vehicles is essential for autonomous vehicles, as it governs their own motion plan. Current state-of-the-art vehicle prediction models heavily rely on map information. In reality, however, this information is not always available. We therefore propose CRAT-Pred, a multi-modal and non-rasterization-based trajectory prediction model, specifically designed to effectively model social interactions between vehicles, without relying on map information. CRAT-Pred applies a graph convolution method originating from the field of material science to vehicle prediction, allowing to efficiently leverage edge features, and combines it with multi-head self-attention. Compared to other map-free approaches, the model achieves state-of-the-art performance with a significantly lower number of model parameters. In addition to that, we quantitatively show that the self-attention mechanism is able to learn social interactions between vehicles, with the weights representing a measurable interaction score. The source code is publicly available.
△ Less
Submitted 10 February, 2022; v1 submitted 9 February, 2022;
originally announced February 2022.
-
DELAUNAY: a dataset of abstract art for psychophysical and machine learning research
Authors:
Camille Gontier,
Jakob Jordan,
Mihai A. Petrovici
Abstract:
Image datasets are commonly used in psychophysical experiments and in machine learning research. Most publicly available datasets are comprised of images of realistic and natural objects. However, while typical machine learning models lack any domain specific knowledge about natural objects, humans can leverage prior experience for such data, making comparisons between artificial and natural learn…
▽ More
Image datasets are commonly used in psychophysical experiments and in machine learning research. Most publicly available datasets are comprised of images of realistic and natural objects. However, while typical machine learning models lack any domain specific knowledge about natural objects, humans can leverage prior experience for such data, making comparisons between artificial and natural learning challenging. Here, we introduce DELAUNAY, a dataset of abstract paintings and non-figurative art objects labelled by the artists' names. This dataset provides a middle ground between natural images and artificial patterns and can thus be used in a variety of contexts, for example to investigate the sample efficiency of humans and artificial neural networks. Finally, we train an off-the-shelf convolutional neural network on DELAUNAY, highlighting several of its intriguing features.
△ Less
Submitted 28 January, 2022;
originally announced January 2022.
-
Latent Equilibrium: A unified learning theory for arbitrarily fast computation with arbitrarily slow neurons
Authors:
Paul Haider,
Benjamin Ellenberger,
Laura Kriener,
Jakob Jordan,
Walter Senn,
Mihai A. Petrovici
Abstract:
The response time of physical computational elements is finite, and neurons are no exception. In hierarchical models of cortical networks each layer thus introduces a response lag. This inherent property of physical dynamical systems results in delayed processing of stimuli and causes a timing mismatch between network output and instructive signals, thus afflicting not only inference, but also lea…
▽ More
The response time of physical computational elements is finite, and neurons are no exception. In hierarchical models of cortical networks each layer thus introduces a response lag. This inherent property of physical dynamical systems results in delayed processing of stimuli and causes a timing mismatch between network output and instructive signals, thus afflicting not only inference, but also learning. We introduce Latent Equilibrium, a new framework for inference and learning in networks of slow components which avoids these issues by harnessing the ability of biological neurons to phase-advance their output with respect to their membrane potential. This principle enables quasi-instantaneous inference independent of network depth and avoids the need for phased plasticity or computationally expensive network relaxation phases. We jointly derive disentangled neuron and synapse dynamics from a prospective energy function that depends on a network's generalized position and momentum. The resulting model can be interpreted as a biologically plausible approximation of error backpropagation in deep cortical networks with continuous-time, leaky neuronal dynamics and continuously active, local plasticity. We demonstrate successful learning of standard benchmark datasets, achieving competitive performance using both fully-connected and convolutional architectures, and show how our principle can be applied to detailed models of cortical microcircuitry. Furthermore, we study the robustness of our model to spatio-temporal substrate imperfections to demonstrate its feasibility for physical realization, be it in vivo or in silico.
△ Less
Submitted 27 October, 2021;
originally announced October 2021.
-
Routing brain traffic through the von Neumann bottleneck: Efficient cache usage in spiking neural network simulation code on general purpose computers
Authors:
Jari Pronold,
Jakob Jordan,
Brian J. N. Wylie,
Itaru Kitayama,
Markus Diesmann,
Susanne Kunkel
Abstract:
Simulation is a third pillar next to experiment and theory in the study of complex dynamic systems such as biological neural networks. Contemporary brain-scale networks correspond to directed graphs of a few million nodes, each with an in-degree and out-degree of several thousands of edges, where nodes and edges correspond to the fundamental biological units, neurons and synapses, respectively. Wh…
▽ More
Simulation is a third pillar next to experiment and theory in the study of complex dynamic systems such as biological neural networks. Contemporary brain-scale networks correspond to directed graphs of a few million nodes, each with an in-degree and out-degree of several thousands of edges, where nodes and edges correspond to the fundamental biological units, neurons and synapses, respectively. When considering a random graph, each node's edges are distributed across thousands of parallel processes. The activity in neuronal networks is also sparse. Each neuron occasionally transmits a brief signal, called spike, via its outgoing synapses to the corresponding target neurons. This spatial and temporal sparsity represents an inherent bottleneck for simulations on conventional computers: Fundamentally irregular memory-access patterns cause poor cache utilization. Using an established neuronal network simulation code as a reference implementation, we investigate how common techniques to recover cache performance such as software-induced prefetching and software pipelining can benefit a real-world application. The algorithmic changes reduce simulation time by up to 50%. The study exemplifies that many-core systems assigned with an intrinsically parallel computational problem can overcome the von Neumann bottleneck of conventional computer architectures.
△ Less
Submitted 11 March, 2022; v1 submitted 27 September, 2021;
originally announced September 2021.
-
Learning cortical representations through perturbed and adversarial dreaming
Authors:
Nicolas Deperrois,
Mihai A. Petrovici,
Walter Senn,
Jakob Jordan
Abstract:
Humans and other animals learn to extract general concepts from sensory experience without extensive teaching. This ability is thought to be facilitated by offline states like sleep where previous experiences are systemically replayed. However, the characteristic creative nature of dreams suggests that learning semantic representations may go beyond merely replaying previous experiences. We suppor…
▽ More
Humans and other animals learn to extract general concepts from sensory experience without extensive teaching. This ability is thought to be facilitated by offline states like sleep where previous experiences are systemically replayed. However, the characteristic creative nature of dreams suggests that learning semantic representations may go beyond merely replaying previous experiences. We support this hypothesis by implementing a cortical architecture inspired by generative adversarial networks (GANs). Learning in our model is organized across three different global brain states mimicking wakefulness, NREM and REM sleep, optimizing different, but complementary objective functions. We train the model on standard datasets of natural images and evaluate the quality of the learned representations. Our results suggest that generating new, virtual sensory inputs via adversarial dreaming during REM sleep is essential for extracting semantic concepts, while replaying episodic memories via perturbed dreaming during NREM sleep improves the robustness of latent representations. The model provides a new computational perspective on sleep states, memory replay and dreams and suggests a cortical implementation of GANs.
△ Less
Submitted 18 February, 2022; v1 submitted 9 September, 2021;
originally announced September 2021.
-
DINs: Deep Interactive Networks for Neurofibroma Segmentation in Neurofibromatosis Type 1 on Whole-Body MRI
Authors:
Jian-Wei Zhang,
Wei Chen,
K. Ina Ly,
Xubin Zhang,
Fan Yan,
Justin Jordan,
Gordon Harris,
Scott Plotkin,
Pengyi Hao,
Wenli Cai
Abstract:
Neurofibromatosis type 1 (NF1) is an autosomal dominant tumor predisposition syndrome that involves the central and peripheral nervous systems. Accurate detection and segmentation of neurofibromas are essential for assessing tumor burden and longitudinal tumor size changes. Automatic convolutional neural networks (CNNs) are sensitive and vulnerable as tumors' variable anatomical location and heter…
▽ More
Neurofibromatosis type 1 (NF1) is an autosomal dominant tumor predisposition syndrome that involves the central and peripheral nervous systems. Accurate detection and segmentation of neurofibromas are essential for assessing tumor burden and longitudinal tumor size changes. Automatic convolutional neural networks (CNNs) are sensitive and vulnerable as tumors' variable anatomical location and heterogeneous appearance on MRI. In this study, we propose deep interactive networks (DINs) to address the above limitations. User interactions guide the model to recognize complicated tumors and quickly adapt to heterogeneous tumors. We introduce a simple but effective Exponential Distance Transform (ExpDT) that converts user interactions into guide maps regarded as the spatial and appearance prior. Comparing with popular Euclidean and geodesic distances, ExpDT is more robust to various image sizes, which reserves the distribution of interactive inputs. Furthermore, to enhance the tumor-related features, we design a deep interactive module to propagate the guides into deeper layers. We train and evaluate DINs on three MRI data sets from NF1 patients. The experiment results yield significant improvements of 44% and 14% in DSC comparing with automated and other interactive methods, respectively. We also experimentally demonstrate the efficiency of DINs in reducing user burden when comparing with conventional interactive methods. The source code of our method is available at \url{https://github.com/Jarvis73/DINs}.
△ Less
Submitted 7 June, 2021;
originally announced June 2021.
-
Evolving Neuronal Plasticity Rules using Cartesian Genetic Programming
Authors:
Henrik D. Mettler,
Maximilian Schmidt,
Walter Senn,
Mihai A. Petrovici,
Jakob Jordan
Abstract:
We formulate the search for phenomenological models of synaptic plasticity as an optimization problem. We employ Cartesian genetic programming to evolve biologically plausible human-interpretable plasticity rules that allow a given network to successfully solve tasks from specific task families. While our evolving-to-learn approach can be applied to various learning paradigms, here we illustrate i…
▽ More
We formulate the search for phenomenological models of synaptic plasticity as an optimization problem. We employ Cartesian genetic programming to evolve biologically plausible human-interpretable plasticity rules that allow a given network to successfully solve tasks from specific task families. While our evolving-to-learn approach can be applied to various learning paradigms, here we illustrate its power by evolving plasticity rules that allow a network to efficiently determine the first principal component of its input distribution. We demonstrate that the evolved rules perform competitively with known hand-designed solutions. We explore how the statistical properties of the datasets used during the evolutionary search influences the form of the plasticity rules and discover new rules which are adapted to the structure of the corresponding datasets.
△ Less
Submitted 8 February, 2021;
originally announced February 2021.
-
Economic and Business Dimensions Cloud Computing and Electricity: Beyond the Utility Model
Authors:
Erik Brynjolfsson,
Paul Hofmann,
John Jordan
Abstract:
An overly simplistic reliance on the utility model risks blinding us to the real opportunities and challenges of cloud computing.
An overly simplistic reliance on the utility model risks blinding us to the real opportunities and challenges of cloud computing.
△ Less
Submitted 7 June, 2020;
originally announced June 2020.
-
Deploying South African Social Honeypots on Twitter
Authors:
Laurenz A Cornelissen,
Richard J Barnett,
Morakane AM Kepa,
Daniel Loebenberg-Novitzkas,
Jacques Jordaan
Abstract:
Inspired by the simple, yet effective, method of tweeting gibberish to attract automated social agents (bots), we attempt to create localised honeypots in the South African political context. We produce a series of defined techniques and combine them to generate interactions from users on Twitter. The paper offers two key contributions. Conceptually, an argument is made that honeypots should not b…
▽ More
Inspired by the simple, yet effective, method of tweeting gibberish to attract automated social agents (bots), we attempt to create localised honeypots in the South African political context. We produce a series of defined techniques and combine them to generate interactions from users on Twitter. The paper offers two key contributions. Conceptually, an argument is made that honeypots should not be confused for bot detection methods, but are rather methods to capture low-quality users. Secondly, we successfully generate a list of 288 local low quality users active in the political context.
△ Less
Submitted 17 September, 2018;
originally announced September 2018.
-
Statistical physics of human cooperation
Authors:
Matjaz Perc,
Jillian J. Jordan,
David G. Rand,
Zhen Wang,
Stefano Boccaletti,
Attila Szolnoki
Abstract:
Extensive cooperation among unrelated individuals is unique to humans, who often sacrifice personal benefits for the common good and work together to achieve what they are unable to execute alone. The evolutionary success of our species is indeed due, to a large degree, to our unparalleled other-regarding abilities. Yet, a comprehensive understanding of human cooperation remains a formidable chall…
▽ More
Extensive cooperation among unrelated individuals is unique to humans, who often sacrifice personal benefits for the common good and work together to achieve what they are unable to execute alone. The evolutionary success of our species is indeed due, to a large degree, to our unparalleled other-regarding abilities. Yet, a comprehensive understanding of human cooperation remains a formidable challenge. Recent research in social science indicates that it is important to focus on the collective behavior that emerges as the result of the interactions among individuals, groups, and even societies. Non-equilibrium statistical physics, in particular Monte Carlo methods and the theory of collective behavior of interacting particles near phase transition points, has proven to be very valuable for understanding counterintuitive evolutionary outcomes. By studying models of human cooperation as classical spin models, a physicist can draw on familiar settings from statistical physics. However, unlike pairwise interactions among particles that typically govern solid-state physics systems, interactions among humans often involve group interactions, and they also involve a larger number of possible states even for the most simplified description of reality. The complexity of solutions therefore often surpasses that observed in physical systems. Here we review experimental and theoretical research that advances our understanding of human cooperation, focusing on spatial pattern formation, on the spatiotemporal dynamics of observed solutions, and on self-organization that may either promote or hinder socially favorable states.
△ Less
Submitted 19 May, 2017;
originally announced May 2017.
-
Feeder Load Balancing using Neural Network
Authors:
A. Ukil,
W. Siti,
J. Jordaan
Abstract:
The distribution system problems, such as planning, loss minimization, and energy restoration, usually involve the phase balancing or network reconfiguration procedures. The determination of an optimal phase balance is, in general, a combinatorial optimization problem. This paper proposes optimal reconfiguration of the phase balancing using the neural network, to switch on and off the different sw…
▽ More
The distribution system problems, such as planning, loss minimization, and energy restoration, usually involve the phase balancing or network reconfiguration procedures. The determination of an optimal phase balance is, in general, a combinatorial optimization problem. This paper proposes optimal reconfiguration of the phase balancing using the neural network, to switch on and off the different switches, allowing the three phases supply by the transformer to the end-users to be balanced. This paper presents the application examples of the proposed method using the real and simulated test data.
△ Less
Submitted 20 March, 2015;
originally announced March 2015.
-
Game-theoretic Approach for Non-Cooperative Planning
Authors:
Jaume Jordán,
Eva Onaindia
Abstract:
When two or more self-interested agents put their plans to execution in the same environment, conflicts may arise as a consequence, for instance, of a common utilization of resources. In this case, an agent can postpone the execution of a particular action, if this punctually solves the conflict, or it can resort to execute a different plan if the agent's payoff significantly diminishes due to the…
▽ More
When two or more self-interested agents put their plans to execution in the same environment, conflicts may arise as a consequence, for instance, of a common utilization of resources. In this case, an agent can postpone the execution of a particular action, if this punctually solves the conflict, or it can resort to execute a different plan if the agent's payoff significantly diminishes due to the action deferral. In this paper, we present a game-theoretic approach to non-cooperative planning that helps predict before execution what plan schedules agents will adopt so that the set of strategies of all agents constitute a Nash equilibrium. We perform some experiments and discuss the solutions obtained with our game-theoretical approach, analyzing how the conflicts between the plans determine the strategic behavior of the agents.
△ Less
Submitted 4 March, 2015;
originally announced March 2015.
-
Heuristics guide the implementation of social preferences in one-shot Prisoner's Dilemma experiments
Authors:
Valerio Capraro,
Jillian J. Jordan,
David G. Rand
Abstract:
Cooperation in one-shot anonymous interactions is a widely documented aspect of human behaviour. Here we shed light on the motivations behind this behaviour by experimentally exploring cooperation in a one-shot continuous-strategy Prisoner's Dilemma (i.e. one-shot two-player Public Goods Game). We examine the distribution of cooperation amounts, and how that distribution varies based on the benefi…
▽ More
Cooperation in one-shot anonymous interactions is a widely documented aspect of human behaviour. Here we shed light on the motivations behind this behaviour by experimentally exploring cooperation in a one-shot continuous-strategy Prisoner's Dilemma (i.e. one-shot two-player Public Goods Game). We examine the distribution of cooperation amounts, and how that distribution varies based on the benefit-to-cost ratio of cooperation (b/c). Interestingly, we find a trimodal distribution at all b/c values investigated. Increasing b/c decreases the fraction of participants engaging in zero cooperation and increases the fraction engaging in maximal cooperation, suggesting a role for efficiency concerns. However, a substantial fraction of participants consistently engage in 50% cooperation regardless of b/c. The presence of these persistent 50% cooperators is surprising, and not easily explained by standard models of social preferences. We present evidence that this behaviour is a result of social preferences guided by simple decision heuristics, rather than the rational examination of payoffs assumed by most social preference models. We also find a strong correlation between play in the Prisoner's Dilemma and in a subsequent Dictator Game, confirming previous findings suggesting a common prosocial motivation underlying altruism and cooperation.
△ Less
Submitted 13 August, 2014; v1 submitted 28 April, 2014;
originally announced April 2014.
-
An Evaluation of Popular Copy-Move Forgery Detection Approaches
Authors:
Vincent Christlein,
Christian Riess,
Johannes Jordan,
Corinna Riess,
Elli Angelopoulou
Abstract:
A copy-move forgery is created by copying and pasting content within the same image, and potentially post-processing it. In recent years, the detection of copy-move forgeries has become one of the most actively researched topics in blind image forensics. A considerable number of different algorithms have been proposed focusing on different types of postprocessed copies. In this paper, we aim to an…
▽ More
A copy-move forgery is created by copying and pasting content within the same image, and potentially post-processing it. In recent years, the detection of copy-move forgeries has become one of the most actively researched topics in blind image forensics. A considerable number of different algorithms have been proposed focusing on different types of postprocessed copies. In this paper, we aim to answer which copy-move forgery detection algorithms and processing steps (e.g., matching, filtering, outlier detection, affine transformation estimation) perform best in various postprocessing scenarios. The focus of our analysis is to evaluate the performance of previously proposed feature sets. We achieve this by casting existing algorithms in a common pipeline. In this paper, we examined the 15 most prominent feature sets. We analyzed the detection performance on a per-image basis and on a per-pixel basis. We created a challenging real-world copy-move dataset, and a software framework for systematic image manipulation. Experiments show, that the keypoint-based features SIFT and SURF, as well as the block-based DCT, DWT, KPCA, PCA and Zernike features perform very well. These feature sets exhibit the best robustness against various noise sources and downsampling, while reliably identifying the copied regions.
△ Less
Submitted 26 November, 2012; v1 submitted 17 August, 2012;
originally announced August 2012.
