-
Higher-Order Spatial Information for Self-Supervised Place Cell Learning
Authors:
Jared Deighton,
Wyatt Mackey,
Ioannis Schizas,
David L. Boothe Jr.,
Vasileios Maroulas
Abstract:
Mammals navigate novel environments and exhibit resilience to sparse environmental sensory cues via place and grid cells, which encode position in space. While the efficiency of grid cell coding has been extensively studied, the computational role of place cells is less well understood. This gap arises partially because spatial information measures have, until now, been limited to single place cel…
▽ More
Mammals navigate novel environments and exhibit resilience to sparse environmental sensory cues via place and grid cells, which encode position in space. While the efficiency of grid cell coding has been extensively studied, the computational role of place cells is less well understood. This gap arises partially because spatial information measures have, until now, been limited to single place cells. We derive and implement a higher-order spatial information measure, allowing for the study of the emergence of multiple place cells in a self-supervised manner. We show that emergent place cells have many desirable features, including high-accuracy spatial decoding. This is the first work in which higher-order spatial information measures that depend solely on place cells' firing rates have been derived and which focuses on the emergence of multiple place cells via self-supervised learning. By quantifying the spatial information of multiple place cells, we enhance our understanding of place cell formation and capabilities in recurrent neural networks, thereby improving the potential navigation capabilities of artificial systems in novel environments without objective location information.
△ Less
Submitted 10 June, 2024;
originally announced July 2024.
-
Geometric sparsification in recurrent neural networks
Authors:
Wyatt Mackey,
Ioannis Schizas,
Jared Deighton,
David L. Boothe, Jr.,
Vasileios Maroulas
Abstract:
A common technique for ameliorating the computational costs of running large neural models is sparsification, or the removal of neural connections during training. Sparse models are capable of maintaining the high accuracy of state of the art models, while functioning at the cost of more parsimonious models. The structures which underlie sparse architectures are, however, poorly understood and not…
▽ More
A common technique for ameliorating the computational costs of running large neural models is sparsification, or the removal of neural connections during training. Sparse models are capable of maintaining the high accuracy of state of the art models, while functioning at the cost of more parsimonious models. The structures which underlie sparse architectures are, however, poorly understood and not consistent between differently trained models and sparsification schemes. In this paper, we propose a new technique for sparsification of recurrent neural nets (RNNs), called moduli regularization, in combination with magnitude pruning. Moduli regularization leverages the dynamical system induced by the recurrent structure to induce a geometric relationship between neurons in the hidden state of the RNN. By making our regularizing term explicitly geometric, we provide the first, to our knowledge, a priori description of the desired sparse architecture of our neural net. We verify the effectiveness of our scheme for navigation and natural language processing RNNs. Navigation is a structurally geometric task, for which there are known moduli spaces, and we show that regularization can be used to reach 90% sparsity while maintaining model performance only when coefficients are chosen in accordance with a suitable moduli space. Natural language processing, however, has no known moduli space in which computations are performed. Nevertheless, we show that moduli regularization induces more stable recurrent neural nets with a variety of moduli regularizers, and achieves high fidelity models at 98% sparsity.
△ Less
Submitted 10 June, 2024;
originally announced June 2024.
-
A pragmatic workflow for research software engineering in computational science
Authors:
Tomislav Marić,
Dennis Gläser,
Jan-Patrick Lehr,
Ioannis Papagiannidis,
Benjamin Lambie,
Christian Bischof,
Dieter Bothe
Abstract:
University research groups in Computational Science and Engineering (CSE) generally lack dedicated funding and personnel for Research Software Engineering (RSE), which, combined with the pressure to maximize the number of scientific publications, shifts the focus away from sustainable research software development and reproducible results. The neglect of RSE in CSE at University research groups ne…
▽ More
University research groups in Computational Science and Engineering (CSE) generally lack dedicated funding and personnel for Research Software Engineering (RSE), which, combined with the pressure to maximize the number of scientific publications, shifts the focus away from sustainable research software development and reproducible results. The neglect of RSE in CSE at University research groups negatively impacts the scientific output: research data - including research software - related to a CSE publication cannot be found, reproduced, or re-used, different ideas are not combined easily into new ideas, and published methods must very often be re-implemented to be investigated further. This slows down CSE research significantly, resulting in considerable losses in time and, consequentially, public funding.
We propose a RSE workflow for Computational Science and Engineering (CSE) that addresses these challenges, that improves the quality of research output in CSE. Our workflow applies established software engineering practices adapted for CSE: software testing, result visualization, and periodical cross-linking of software with reports/publications and data, timed by milestones in the scientific publication process. The workflow introduces minimal work overhead, crucial for university research groups, and delivers modular and tested software linked to publications whose results can easily be reproduced. We define research software quality from a perspective of a pragmatic researcher: the ability to quickly find the publication, data, and software related to a published research idea, quickly reproduce results, understand or re-use a CSE method, and finally extend the method with new research ideas.
△ Less
Submitted 2 October, 2023;
originally announced October 2023.
-
A Topological Deep Learning Framework for Neural Spike Decoding
Authors:
Edward C. Mitchell,
Brittany Story,
David Boothe,
Piotr J. Franaszczuk,
Vasileios Maroulas
Abstract:
The brain's spatial orientation system uses different neuron ensembles to aid in environment-based navigation. Two of the ways brains encode spatial information is through head direction cells and grid cells. Brains use head direction cells to determine orientation whereas grid cells consist of layers of decked neurons that overlay to provide environment-based navigation. These neurons fire in ens…
▽ More
The brain's spatial orientation system uses different neuron ensembles to aid in environment-based navigation. Two of the ways brains encode spatial information is through head direction cells and grid cells. Brains use head direction cells to determine orientation whereas grid cells consist of layers of decked neurons that overlay to provide environment-based navigation. These neurons fire in ensembles where several neurons fire at once to activate a single head direction or grid. We want to capture this firing structure and use it to decode head direction grid cell data. Understanding, representing, and decoding these neural structures requires models that encompass higher order connectivity, more than the 1-dimensional connectivity that traditional graph-based models provide. To that end, in this work, we develop a topological deep learning framework for neural spike train decoding. Our framework combines unsupervised simplicial complex discovery with the power of deep learning via a new architecture we develop herein called a simplicial convolutional recurrent neural network. Simplicial complexes, topological spaces that use not only vertices and edges but also higher-dimensional objects, naturally generalize graphs and capture more than just pairwise relationships. Additionally, this approach does not require prior knowledge of the neural activity beyond spike counts, which removes the need for similarity measurements. The effectiveness and versatility of the simplicial convolutional neural network is demonstrated on head direction and trajectory prediction via head direction and grid cell datasets.
△ Less
Submitted 6 September, 2023; v1 submitted 1 December, 2022;
originally announced December 2022.
-
A Research Software Engineering Workflow for Computational Science and Engineering
Authors:
Tomislav Maric,
Dennis Gläser,
Jan-Patrick Lehr,
Ioannis Papagiannidis,
Benjamin Lambie,
Christian Bischof,
Dieter Bothe
Abstract:
University research groups in Computational Science and Engineering (CSE) generally lack dedicated funding and personnel for Research Software Engineering (RSE), which, combined with the pressure to maximize the number of scientific publications, shifts the focus away from sustainable research software development and reproducible results. The neglect of RSE in CSE at University research groups ne…
▽ More
University research groups in Computational Science and Engineering (CSE) generally lack dedicated funding and personnel for Research Software Engineering (RSE), which, combined with the pressure to maximize the number of scientific publications, shifts the focus away from sustainable research software development and reproducible results. The neglect of RSE in CSE at University research groups negatively impacts the scientific output: research data - including research software - related to a CSE publication cannot be found, reproduced, or re-used, different ideas are not combined easily into new ideas, and published methods must very often be re-implemented to be investigated further. This slows down CSE research significantly, resulting in considerable losses in time and, consequentially, public funding.
We propose a RSE workflow for Computational Science and Engineering (CSE) that addresses these challenges, that improves the quality of research output in CSE. Our workflow applies established software engineering practices adapted for CSE: software testing, result visualization, and periodical cross-linking of software with reports/publications and data, timed by milestones in the scientific publication process. The workflow introduces minimal work overhead, crucial for university research groups, and delivers modular and tested software linked to publications whose results can easily be reproduced. We define research software quality from a perspective of a pragmatic researcher: the ability to quickly find the publication, data, and software related to a published research idea, quickly reproduce results, understand or re-use a CSE method, and finally extend the method with new research ideas.
△ Less
Submitted 15 August, 2022;
originally announced August 2022.
-
Third-order accurate initialization of VOF volume fractions on unstructured meshes with arbitrary polyhedral cells
Authors:
Johannes Kromer,
Dieter Bothe
Abstract:
This paper introduces a novel method for the efficient and accurate computation of volume fractions on unstructured polyhedral meshes, where the phase boundary is an orientable hypersurface, implicitly given as the iso-contour of a sufficiently smooth level-set function. Locally, i.e. in each mesh cell, we compute a principal coordinate system in which the hypersurface can be approximated as the g…
▽ More
This paper introduces a novel method for the efficient and accurate computation of volume fractions on unstructured polyhedral meshes, where the phase boundary is an orientable hypersurface, implicitly given as the iso-contour of a sufficiently smooth level-set function. Locally, i.e. in each mesh cell, we compute a principal coordinate system in which the hypersurface can be approximated as the graph of an osculating paraboloid. A recursive application of the \textsc{Gaussian} divergence theorem then allows to analytically transform the volume integrals to curve integrals associated to the polyhedron faces, which can be easily approximated numerically by means of standard \textsc{Gauss-Legendre} quadrature. This face-based formulation enables the applicability to unstructured meshes and considerably simplifies the numerical procedure for applications in three spatial dimensions. We discuss the theoretical foundations and provide details of the numerical algorithm. Finally, we present numerical results for convex and non-convex hypersurfaces embedded in cuboidal and tetrahedral meshes, showing both high accuracy and third- to fourth-order convergence with spatial resolution.
△ Less
Submitted 17 November, 2021; v1 submitted 1 November, 2021;
originally announced November 2021.
-
Efficient sequential PLIC interface positioning for enhanced performance of the three-phase VoF Method
Authors:
Johannes Kromer,
Johanna Potyka,
Kathrin Schulte,
Dieter Bothe
Abstract:
This paper presents an efficient algorithm for the sequential positioning, also called nested dissection, of two planes in an arbitrary polyhedron. Two planar interfaces are positioned such that the first plane truncates a given volume from this arbitrary polyhedron and the next plane truncates a second given volume from the residual polyhedron. This is a relevant task in the numerical simulation…
▽ More
This paper presents an efficient algorithm for the sequential positioning, also called nested dissection, of two planes in an arbitrary polyhedron. Two planar interfaces are positioned such that the first plane truncates a given volume from this arbitrary polyhedron and the next plane truncates a second given volume from the residual polyhedron. This is a relevant task in the numerical simulation of three-phase flows when resorting to the geometric Volume-of-Fluid (VoF) method with a Piecewise Linear Interface Calculation (PLIC). An efficient algorithm for this task significantly speeds up the three-phase PLIC algorithm. The present study describes a method based on a recursive application of the Gaussian divergence theorem, where the fact that the truncated polyhedron shares multiple faces with the original polyhedron can be exploited to reduce the computational effort. A careful choice of the coordinate system origin for the volume computation allows for successive positioning of two planes without reestablishing polyhedron connectivity. Combined with a highly efficient root finding, this results in a significant performance gain in the reconstruction of the three-phase interface configurations.
The performance of the new method is assessed in a series of carefully designed numerical experiments. Compared to a conventional decomposition-based approach, the number of iterations and, thus, of the required truncations was reduced by up to an order of magnitude. The PLIC positioning run-time was reduced by about 90% in our reference implementation. Integrated into the multi-phase flow solver Free Surface 3D (FS3D), an overall performance gain of about 20% was achieved. Allowing for simple integration into existing numerical schemes, the proposed algorithm is self-contained (example Fortran Module see https://doi.org/10.18419/darus-2488), requiring no external decomposition libraries.
△ Less
Submitted 27 June, 2023; v1 submitted 19 May, 2021;
originally announced May 2021.
-
Face-based Volume-of-Fluid interface positioning in arbitrary polyhedra
Authors:
Johannes Kromer,
Dieter Bothe
Abstract:
We introduce a fast and robust algorithm for finding a plane $Γ$ with given normal $\vec{n}_Γ$, which truncates an arbitrary polyhedron $\mathcal{P}$ such that the remaining sub-polyhedron admits a given volume $α|\mathcal{P}|$. In the literature, this is commonly referred to as Volume-of-Fluid (VoF) interface positioning problem. The novelty of our work is twofold: firstly, by recursive applicati…
▽ More
We introduce a fast and robust algorithm for finding a plane $Γ$ with given normal $\vec{n}_Γ$, which truncates an arbitrary polyhedron $\mathcal{P}$ such that the remaining sub-polyhedron admits a given volume $α|\mathcal{P}|$. In the literature, this is commonly referred to as Volume-of-Fluid (VoF) interface positioning problem. The novelty of our work is twofold: firstly, by recursive application of the Gaussian divergence theorem, the volume of a truncated polyhedron can be computed at high efficiency, based on summation over quantities associated to the faces of the polyhedron. One obtains a very convenient piecewise parametrization (within so-called brackets) in terms of the signed distance s to the plane $Γ$. As an implication, one can restrain from the costly necessity to establish topological connectivity, rendering the present approach most suitable for the application to unstructured computational meshes. Secondly, in the vicinity of the truncation position s, the volume can be expressed exactly, i.e. in terms of a cubic polynomial of the normal distance to the PLIC plane. The local knowledge of derivatives enables to construct a root-finding algorithm that pairs bracketing and higher-order approximation. The performance is assessed by conducting an extensive set of numerical experiments, considering convex and non-convex polyhedra of genus (i.e., number of holes) zero and one in combination with carefully selected volume fractions $α$ (including $α\approx0$ and $α\approx1$) and normal orientations $\vec{n}_Γ$. For all configurations we obtain a significant reduction of the number of (computationally costly) truncations required for the positioning: on average, our algorithm requires between one and two polyhedron truncations to find the position of the plane $Γ$, outperforming existing methods.
△ Less
Submitted 11 January, 2021;
originally announced January 2021.
-
Bayesian Topological Learning for Brain State Classification
Authors:
Farzana Nasrin,
Christopher Oballe,
David L. Boothe,
Vasileios Maroulas
Abstract:
Investigation of human brain states through electroencephalograph (EEG) signals is a crucial step in human-machine communications. However, classifying and analyzing EEG signals are challenging due to their noisy, nonlinear and nonstationary nature. Current methodologies for analyzing these signals often fall short because they have several regularity assumptions baked in. This work provides an ef…
▽ More
Investigation of human brain states through electroencephalograph (EEG) signals is a crucial step in human-machine communications. However, classifying and analyzing EEG signals are challenging due to their noisy, nonlinear and nonstationary nature. Current methodologies for analyzing these signals often fall short because they have several regularity assumptions baked in. This work provides an effective, flexible and noise-resilient scheme to analyze EEG by extracting pertinent information while abiding by the 3N (noisy, nonlinear and nonstationary) nature of data. We implement a topological tool, namely persistent homology, that tracks the evolution of topological features over time intervals and incorporates individual's expectations as prior knowledge by means of a Bayesian framework to compute posterior distributions. Relying on these posterior distributions, we apply Bayes factor classification to noisy EEG measurements. The performance of this Bayesian classification scheme is then compared with other existing methods for EEG signals.
△ Less
Submitted 17 December, 2019;
originally announced December 2019.
-
Boundedness-Preserving Implicit Correction of Mesh-Induced Errors for VoF Based Heat and Mass Transfer
Authors:
Simon Hill,
Daniel Deising,
Thomas Acher,
Harald Klein,
Dieter Bothe,
Holger Marschall
Abstract:
Spatial discretisation of geometrically complex computational domains often entails unstructured meshes of general topology for Computational Fluid Dynamics (CFD). Mesh skewness is then typically encountered causing severe deterioration of the formal order of accuracy of the discretisation, or boundedness of the solution, or both. Particularly methods inherently relying on the accurate and bounded…
▽ More
Spatial discretisation of geometrically complex computational domains often entails unstructured meshes of general topology for Computational Fluid Dynamics (CFD). Mesh skewness is then typically encountered causing severe deterioration of the formal order of accuracy of the discretisation, or boundedness of the solution, or both. Particularly methods inherently relying on the accurate and bounded transport of sharp fields suffer from all types of mesh-induced skewness errors, namely both non-orthogonality and non-conjunctionality errors. This work is devoted to a boundedness-preserving strategy to correct for skewness errors arising from discretisation of advection and diffusion terms within the context of interfacial heat and mass transfer based on the Volume-of-Fluid methodology. The implementation has been accomplished using a second-order finite volume method with support for unstructured meshes of general topology. We examine and advance suitable corrections for the finite volume discretisation of a consistent single-field model, where both accurate and bounded transport due to diffusion and advection is crucial. In order to ensure consistency of both the volume fraction and the species concentration transport, i.e. to avoid artificial heat or species transfer, corrections are studied for both cases. The cross interfacial jump and adjacent sharp gradients of species concentration render the correction for skewness-induced diffusion and advection errors additionally demanding and has not so far been addressed in the literature.
△ Less
Submitted 21 June, 2017; v1 submitted 31 May, 2017;
originally announced May 2017.
-
A graph-theoretical approach for the computation of connected iso-surfaces based on volumetric data
Authors:
Abdulaziz Ali,
Dieter Bothe
Abstract:
The existing combinatorial methods for iso-surface computation are efficient for pure visualization purposes, but it is known that the resulting iso-surfaces can have holes, and topological problems like missing or wrong connectivity can appear. To avoid such problems, we introduce a graph-theoretical method for the computation of iso-surfaces on cuboid meshes in $\mathbb{R}^3$. The method for the…
▽ More
The existing combinatorial methods for iso-surface computation are efficient for pure visualization purposes, but it is known that the resulting iso-surfaces can have holes, and topological problems like missing or wrong connectivity can appear. To avoid such problems, we introduce a graph-theoretical method for the computation of iso-surfaces on cuboid meshes in $\mathbb{R}^3$. The method for the generation of iso-surfaces employs labeled cuboid graphs $G(V,E,\mathcal{F})$ such that $V$ is the set of vertices of a cuboid $C\subset\mathbb{R}^3$, $E$ is the set of edges of $C$ and $\mathcal{F}\,:\,V\rightarrow [0,1]$. The nodes of $G$ are weighted by the values of $\mathcal{F}$ which represents the volumetric information, e.g.\ from a Volume of Fluid method. Using a given iso-level $c\in (0,1)$, we first obtain all iso-points, i.e.\ points where the value $c$ is attained by the edge-interpolated $\mathcal{F}$-field. The iso-surface is then built from iso-elements which are composed of triangles and are such that their polygonal boundary has only iso-points as vertices. All vertices lie on the faces of a single mesh cell.
We give a proof that the generated iso-surface is connected up to the boundary of the domain and it can be decomposed into different oriented components. Two different components may have discrete points or line segments in common. The graph-theoretical method for the computation of iso-surfaces developed in this paper enables to recover local information of the iso-surface that can be used e.g.\ to compute discrete mean curvature and to solve surface PDEs. Concerning the computational effort, the resulting algorithm is as efficient as existing combinatorial methods.
△ Less
Submitted 13 February, 2014;
originally announced February 2014.
-
Experiments and Direct Numerical Simulations of binary collisions of miscible liquid droplets with different viscosities
Authors:
C. Focke,
D. Bothe,
M. Kuschel,
M. Sommerfeld
Abstract:
Binary droplet collisions are of importance in a variety of practical applications comprising dispersed two-phase flows. The background of our research is the prediction of properties of particulate products formed in spray processes. To gain a more thorough understanding of the elementary sub-processes inside a spray, experiments and direct numerical simulations of binary droplet collisions are u…
▽ More
Binary droplet collisions are of importance in a variety of practical applications comprising dispersed two-phase flows. The background of our research is the prediction of properties of particulate products formed in spray processes. To gain a more thorough understanding of the elementary sub-processes inside a spray, experiments and direct numerical simulations of binary droplet collisions are used. The aim of these investigations is to develop semi-analytical descriptions for the outcome of droplet collisions. Such collision models can then be employed as closure terms for scale-reduced simulations. In the present work we focus on the collision of droplets of different liquids. These kinds of collisions take place in every spray drying process when droplets with different solids contents collide in recirculation zones. A new experimental method has been developed allowing for high spatial and time resolved recordings via Laser-induced fluorescence. The results obtained with the proposed method will be compared with DNS simulations. The viscosities of the droplets are different whereas the interfacial tension and density are equal. The liquids are miscible and no surface tension is acting between the two liquids. Our intention is to discover elementary phenomena caused by the viscosity ratio of the droplets.
△ Less
Submitted 23 October, 2012;
originally announced October 2012.
