-
ICML 2023 Topological Deep Learning Challenge : Design and Results
Authors:
Mathilde Papillon,
Mustafa Hajij,
Helen Jenne,
Johan Mathe,
Audun Myers,
Theodore Papamarkou,
Tolga Birdal,
Tamal Dey,
Tim Doster,
Tegan Emerson,
Gurusankar Gopalakrishnan,
Devendra Govil,
Aldo Guzmán-Sáenz,
Henry Kvinge,
Neal Livesay,
Soham Mukherjee,
Shreyas N. Samaga,
Karthikeyan Natesan Ramamurthy,
Maneel Reddy Karri,
Paul Rosen,
Sophia Sanborn,
Robin Walters,
Jens Agerberg,
Sadrodin Barikbin,
Claudio Battiloro
, et al. (31 additional authors not shown)
Abstract:
This paper presents the computational challenge on topological deep learning that was hosted within the ICML 2023 Workshop on Topology and Geometry in Machine Learning. The competition asked participants to provide open-source implementations of topological neural networks from the literature by contributing to the python packages TopoNetX (data processing) and TopoModelX (deep learning). The chal…
▽ More
This paper presents the computational challenge on topological deep learning that was hosted within the ICML 2023 Workshop on Topology and Geometry in Machine Learning. The competition asked participants to provide open-source implementations of topological neural networks from the literature by contributing to the python packages TopoNetX (data processing) and TopoModelX (deep learning). The challenge attracted twenty-eight qualifying submissions in its two-month duration. This paper describes the design of the challenge and summarizes its main findings.
△ Less
Submitted 18 January, 2024; v1 submitted 26 September, 2023;
originally announced September 2023.
-
Evaluating Robustness and Uncertainty of Graph Models Under Structural Distributional Shifts
Authors:
Gleb Bazhenov,
Denis Kuznedelev,
Andrey Malinin,
Artem Babenko,
Liudmila Prokhorenkova
Abstract:
In reliable decision-making systems based on machine learning, models have to be robust to distributional shifts or provide the uncertainty of their predictions. In node-level problems of graph learning, distributional shifts can be especially complex since the samples are interdependent. To evaluate the performance of graph models, it is important to test them on diverse and meaningful distributi…
▽ More
In reliable decision-making systems based on machine learning, models have to be robust to distributional shifts or provide the uncertainty of their predictions. In node-level problems of graph learning, distributional shifts can be especially complex since the samples are interdependent. To evaluate the performance of graph models, it is important to test them on diverse and meaningful distributional shifts. However, most graph benchmarks considering distributional shifts for node-level problems focus mainly on node features, while structural properties are also essential for graph problems. In this work, we propose a general approach for inducing diverse distributional shifts based on graph structure. We use this approach to create data splits according to several structural node properties: popularity, locality, and density. In our experiments, we thoroughly evaluate the proposed distributional shifts and show that they can be quite challenging for existing graph models. We also reveal that simple models often outperform more sophisticated methods on the considered structural shifts. Finally, our experiments provide evidence that there is a trade-off between the quality of learned representations for the base classification task under structural distributional shift and the ability to separate the nodes from different distributions using these representations.
△ Less
Submitted 1 November, 2023; v1 submitted 27 February, 2023;
originally announced February 2023.
-
Towards OOD Detection in Graph Classification from Uncertainty Estimation Perspective
Authors:
Gleb Bazhenov,
Sergei Ivanov,
Maxim Panov,
Alexey Zaytsev,
Evgeny Burnaev
Abstract:
The problem of out-of-distribution detection for graph classification is far from being solved. The existing models tend to be overconfident about OOD examples or completely ignore the detection task. In this work, we consider this problem from the uncertainty estimation perspective and perform the comparison of several recently proposed methods. In our experiment, we find that there is no univers…
▽ More
The problem of out-of-distribution detection for graph classification is far from being solved. The existing models tend to be overconfident about OOD examples or completely ignore the detection task. In this work, we consider this problem from the uncertainty estimation perspective and perform the comparison of several recently proposed methods. In our experiment, we find that there is no universal approach for OOD detection, and it is important to consider both graph representations and predictive categorical distribution.
△ Less
Submitted 21 June, 2022;
originally announced June 2022.
