-
Autobahn: Automorphism-based Graph Neural Nets
Authors:
Erik Henning Thiede,
Wenda Zhou,
Risi Kondor
Abstract:
We introduce Automorphism-based graph neural networks (Autobahn), a new family of graph neural networks. In an Autobahn, we decompose the graph into a collection of subgraphs and apply local convolutions that are equivariant to each subgraph's automorphism group. Specific choices of local neighborhoods and subgraphs recover existing architectures such as message passing neural networks. Our formal…
▽ More
We introduce Automorphism-based graph neural networks (Autobahn), a new family of graph neural networks. In an Autobahn, we decompose the graph into a collection of subgraphs and apply local convolutions that are equivariant to each subgraph's automorphism group. Specific choices of local neighborhoods and subgraphs recover existing architectures such as message passing neural networks. Our formalism also encompasses novel architectures: as an example, we introduce a graph neural network that decomposes the graph into paths and cycles. The resulting convolutions reflect the natural way that parts of the graph can transform, preserving the intuitive meaning of convolution without sacrificing global permutation equivariance. We validate our approach by applying Autobahn to molecular graphs, where it achieves results competitive with state-of-the-art message passing algorithms.
△ Less
Submitted 2 February, 2022; v1 submitted 2 March, 2021;
originally announced March 2021.
-
A community-powered search of machine learning strategy space to find NMR property prediction models
Authors:
Lars A. Bratholm,
Will Gerrard,
Brandon Anderson,
Shaojie Bai,
Sunghwan Choi,
Lam Dang,
Pavel Hanchar,
Addison Howard,
Guillaume Huard,
Sanghoon Kim,
Zico Kolter,
Risi Kondor,
Mordechai Kornbluth,
Youhan Lee,
Youngsoo Lee,
Jonathan P. Mailoa,
Thanh Tu Nguyen,
Milos Popovic,
Goran Rakocevic,
Walter Reade,
Wonho Song,
Luka Stojanovic,
Erik H. Thiede,
Nebojsa Tijanic,
Andres Torrubia
, et al. (4 additional authors not shown)
Abstract:
The rise of machine learning (ML) has created an explosion in the potential strategies for using data to make scientific predictions. For physical scientists wishing to apply ML strategies to a particular domain, it can be difficult to assess in advance what strategy to adopt within a vast space of possibilities. Here we outline the results of an online community-powered effort to swarm search the…
▽ More
The rise of machine learning (ML) has created an explosion in the potential strategies for using data to make scientific predictions. For physical scientists wishing to apply ML strategies to a particular domain, it can be difficult to assess in advance what strategy to adopt within a vast space of possibilities. Here we outline the results of an online community-powered effort to swarm search the space of ML strategies and develop algorithms for predicting atomic-pairwise nuclear magnetic resonance (NMR) properties in molecules. Using an open-source dataset, we worked with Kaggle to design and host a 3-month competition which received 47,800 ML model predictions from 2,700 teams in 84 countries. Within 3 weeks, the Kaggle community produced models with comparable accuracy to our best previously published "in-house" efforts. A meta-ensemble model constructed as a linear combination of the top predictions has a prediction accuracy which exceeds that of any individual model, 7-19x better than our previous state-of-the-art. The results highlight the potential of transformer architectures for predicting quantum mechanical (QM) molecular properties.
△ Less
Submitted 13 August, 2020;
originally announced August 2020.
-
The general theory of permutation equivarant neural networks and higher order graph variational encoders
Authors:
Erik Henning Thiede,
Truong Son Hy,
Risi Kondor
Abstract:
Previous work on symmetric group equivariant neural networks generally only considered the case where the group acts by permuting the elements of a single vector. In this paper we derive formulae for general permutation equivariant layers, including the case where the layer acts on matrices by permuting their rows and columns simultaneously. This case arises naturally in graph learning and relatio…
▽ More
Previous work on symmetric group equivariant neural networks generally only considered the case where the group acts by permuting the elements of a single vector. In this paper we derive formulae for general permutation equivariant layers, including the case where the layer acts on matrices by permuting their rows and columns simultaneously. This case arises naturally in graph learning and relation learning applications. As a specific case of higher order permutation equivariant networks, we present a second order graph variational encoder, and show that the latent distribution of equivariant generative models must be exchangeable. We demonstrate the efficacy of this architecture on the tasks of link prediction in citation graphs and molecular graph generation.
△ Less
Submitted 8 April, 2020;
originally announced April 2020.
