Attending to Topological Spaces: The Cellular Transformer
Authors:
Rubén Ballester,
Pablo Hernández-García,
Mathilde Papillon,
Claudio Battiloro,
Nina Miolane,
Tolga Birdal,
Carles Casacuberta,
Sergio Escalera,
Mustafa Hajij
Abstract:
Topological Deep Learning seeks to enhance the predictive performance of neural network models by harnessing topological structures in input data. Topological neural networks operate on spaces such as cell complexes and hypergraphs, that can be seen as generalizations of graphs. In this work, we introduce the Cellular Transformer (CT), a novel architecture that generalizes graph-based transformers…
▽ More
Topological Deep Learning seeks to enhance the predictive performance of neural network models by harnessing topological structures in input data. Topological neural networks operate on spaces such as cell complexes and hypergraphs, that can be seen as generalizations of graphs. In this work, we introduce the Cellular Transformer (CT), a novel architecture that generalizes graph-based transformers to cell complexes. First, we propose a new formulation of the usual self- and cross-attention mechanisms, tailored to leverage incidence relations in cell complexes, e.g., edge-face and node-edge relations. Additionally, we propose a set of topological positional encodings specifically designed for cell complexes. By transforming three graph datasets into cell complex datasets, our experiments reveal that CT not only achieves state-of-the-art performance, but it does so without the need for more complex enhancements such as virtual nodes, in-domain structural encodings, or graph rewiring.
△ Less
Submitted 26 May, 2024; v1 submitted 22 May, 2024;
originally announced May 2024.
TopoX: A Suite of Python Packages for Machine Learning on Topological Domains
Authors:
Mustafa Hajij,
Mathilde Papillon,
Florian Frantzen,
Jens Agerberg,
Ibrahem AlJabea,
Ruben Ballester,
Claudio Battiloro,
Guillermo Bernárdez,
Tolga Birdal,
Aiden Brent,
Peter Chin,
Sergio Escalera,
Simone Fiorellino,
Odin Hoff Gardaa,
Gurusankar Gopalakrishnan,
Devendra Govil,
Josef Hoppe,
Maneel Reddy Karri,
Jude Khouja,
Manuel Lecha,
Neal Livesay,
Jan Meißner,
Soham Mukherjee,
Alexander Nikitin,
Theodore Papamarkou
, et al. (18 additional authors not shown)
Abstract:
We introduce TopoX, a Python software suite that provides reliable and user-friendly building blocks for computing and machine learning on topological domains that extend graphs: hypergraphs, simplicial, cellular, path and combinatorial complexes. TopoX consists of three packages: TopoNetX facilitates constructing and computing on these domains, including working with nodes, edges and higher-order…
▽ More
We introduce TopoX, a Python software suite that provides reliable and user-friendly building blocks for computing and machine learning on topological domains that extend graphs: hypergraphs, simplicial, cellular, path and combinatorial complexes. TopoX consists of three packages: TopoNetX facilitates constructing and computing on these domains, including working with nodes, edges and higher-order cells; TopoEmbedX provides methods to embed topological domains into vector spaces, akin to popular graph-based embedding algorithms such as node2vec; TopoModelx is built on top of PyTorch and offers a comprehensive toolbox of higher-order message passing functions for neural networks on topological domains. The extensively documented and unit-tested source code of TopoX is available under MIT license at https://pyt-team.github.io/.
△ Less
Submitted 17 February, 2024; v1 submitted 4 February, 2024;
originally announced February 2024.
