Skip to main content

Showing 1–18 of 18 results for author: Keriven, N

Searching in archive stat. Search in all archives.
.
  1. arXiv:2405.18127  [pdf, other

    cs.LG stat.ML

    Graph Coarsening with Message-Passing Guarantees

    Authors: Antonin Joly, Nicolas Keriven

    Abstract: Graph coarsening aims to reduce the size of a large graph while preserving some of its key properties, which has been used in many applications to reduce computational load and memory footprint. For instance, in graph machine learning, training Graph Neural Networks (GNNs) on coarsened graphs leads to drastic savings in time and memory. However, GNNs rely on the Message-Passing (MP) paradigm, and… ▽ More

    Submitted 28 May, 2024; originally announced May 2024.

  2. arXiv:2305.14814  [pdf, other

    cs.LG stat.ML

    What functions can Graph Neural Networks compute on random graphs? The role of Positional Encoding

    Authors: Nicolas Keriven, Samuel Vaiter

    Abstract: We aim to deepen the theoretical understanding of Graph Neural Networks (GNNs) on large graphs, with a focus on their expressive power. Existing analyses relate this notion to the graph isomorphism problem, which is mostly relevant for graphs of small sizes, or studied graph classification or regression tasks, while prediction tasks on nodes are far more relevant on large graphs. Recently, several… ▽ More

    Submitted 24 May, 2023; originally announced May 2023.

  3. arXiv:2304.11140  [pdf, other

    stat.ML cs.LG

    Convergence of Message Passing Graph Neural Networks with Generic Aggregation On Large Random Graphs

    Authors: Matthieu Cordonnier, Nicolas Keriven, Nicolas Tremblay, Samuel Vaiter

    Abstract: We study the convergence of message passing graph neural networks on random graph models to their continuous counterpart as the number of nodes tends to infinity. Until now, this convergence was only known for architectures with aggregation functions in the form of normalized means, or, equivalently, of an application of classical operators like the adjacency matrix or the graph Laplacian. We exte… ▽ More

    Submitted 13 July, 2023; v1 submitted 21 April, 2023; originally announced April 2023.

  4. arXiv:2210.10535  [pdf, other

    cs.CG cs.LG stat.OT

    Stability of Entropic Wasserstein Barycenters and application to random geometric graphs

    Authors: Marc Theveneau, Nicolas Keriven

    Abstract: As interest in graph data has grown in recent years, the computation of various geometric tools has become essential. In some area such as mesh processing, they often rely on the computation of geodesics and shortest paths in discretized manifolds. A recent example of such a tool is the computation of Wasserstein barycenters (WB), a very general notion of barycenters derived from the theory of Opt… ▽ More

    Submitted 27 March, 2023; v1 submitted 19 October, 2022; originally announced October 2022.

  5. arXiv:2205.12156  [pdf, other

    stat.ML cs.LG

    Not too little, not too much: a theoretical analysis of graph (over)smoothing

    Authors: Nicolas Keriven

    Abstract: We analyze graph smoothing with \emph{mean aggregation}, where each node successively receives the average of the features of its neighbors. Indeed, it has quickly been observed that Graph Neural Networks (GNNs), which generally follow some variant of Message-Passing (MP) with repeated aggregation, may be subject to the oversmoothing phenomenon: by performing too many rounds of MP, the node featur… ▽ More

    Submitted 10 October, 2022; v1 submitted 24 May, 2022; originally announced May 2022.

    Journal ref: Advances in Neural Information Processing 2022

  6. arXiv:2201.03949  [pdf, other

    stat.ML cs.LG math.PR

    Entropic Optimal Transport in Random Graphs

    Authors: Nicolas Keriven

    Abstract: In graph analysis, a classic task consists in computing similarity measures between (groups of) nodes. In latent space random graphs, nodes are associated to unknown latent variables. One may then seek to compute distances directly in the latent space, using only the graph structure. In this paper, we show that it is possible to consistently estimate entropic-regularized Optimal Transport (OT) dis… ▽ More

    Submitted 11 January, 2022; originally announced January 2022.

  7. arXiv:2112.07990  [pdf, other

    math.OC stat.ML

    Supervised learning of analysis-sparsity priors with automatic differentiation

    Authors: Hashem Ghanem, Joseph Salmon, Nicolas Keriven, Samuel Vaiter

    Abstract: Sparsity priors are commonly used in denoising and image reconstruction. For analysis-type priors, a dictionary defines a representation of signals that is likely to be sparse. In most situations, this dictionary is not known, and is to be recovered from pairs of ground-truth signals and measurements, by minimizing the reconstruction error. This defines a hierarchical optimization problem, which c… ▽ More

    Submitted 15 December, 2021; originally announced December 2021.

    Comments: 5 pages, 4 figures

  8. arXiv:2105.13099  [pdf, other

    stat.ML cs.LG

    On the Universality of Graph Neural Networks on Large Random Graphs

    Authors: Nicolas Keriven, Alberto Bietti, Samuel Vaiter

    Abstract: We study the approximation power of Graph Neural Networks (GNNs) on latent position random graphs. In the large graph limit, GNNs are known to converge to certain "continuous" models known as c-GNNs, which directly enables a study of their approximation power on random graph models. In the absence of input node features however, just as GNNs are limited by the Weisfeiler-Lehman isomorphism test, c… ▽ More

    Submitted 28 May, 2021; v1 submitted 27 May, 2021; originally announced May 2021.

  9. arXiv:2010.08270  [pdf, other

    cs.LG stat.ML

    Fast Graph Kernel with Optical Random Features

    Authors: Hashem Ghanem, Nicolas Keriven, Nicolas Tremblay

    Abstract: The graphlet kernel is a classical method in graph classification. It however suffers from a high computation cost due to the isomorphism test it includes. As a generic proxy, and in general at the cost of losing some information, this test can be efficiently replaced by a user-defined map** that computes various graph characteristics. In this paper, we propose to leverage kernel random features… ▽ More

    Submitted 16 October, 2020; originally announced October 2020.

  10. arXiv:2008.01839  [pdf, other

    stat.ML cs.IT cs.LG

    Sketching Datasets for Large-Scale Learning (long version)

    Authors: Rémi Gribonval, Antoine Chatalic, Nicolas Keriven, Vincent Schellekens, Laurent Jacques, Philip Schniter

    Abstract: This article considers "compressive learning," an approach to large-scale machine learning where datasets are massively compressed before learning (e.g., clustering, classification, or regression) is performed. In particular, a "sketch" is first constructed by computing carefully chosen nonlinear random features (e.g., random Fourier features) and averaging them over the whole dataset. Parameters… ▽ More

    Submitted 24 June, 2021; v1 submitted 4 August, 2020; originally announced August 2020.

  11. arXiv:2006.01868  [pdf, other

    stat.ML cs.LG

    Convergence and Stability of Graph Convolutional Networks on Large Random Graphs

    Authors: Nicolas Keriven, Alberto Bietti, Samuel Vaiter

    Abstract: We study properties of Graph Convolutional Networks (GCNs) by analyzing their behavior on standard models of random graphs, where nodes are represented by random latent variables and edges are drawn according to a similarity kernel. This allows us to overcome the difficulties of dealing with discrete notions such as isomorphisms on very large graphs, by considering instead more natural geometric a… ▽ More

    Submitted 23 October, 2020; v1 submitted 2 June, 2020; originally announced June 2020.

  12. arXiv:2002.02892  [pdf, other

    stat.ML cs.LG math.ST

    Sparse and Smooth: improved guarantees for Spectral Clustering in the Dynamic Stochastic Block Model

    Authors: Nicolas Keriven, Samuel Vaiter

    Abstract: In this paper, we analyse classical variants of the Spectral Clustering (SC) algorithm in the Dynamic Stochastic Block Model (DSBM). Existing results show that, in the relatively sparse case where the expected degree grows logarithmically with the number of nodes, guarantees in the static case can be extended to the dynamic case and yield improved error bounds when the DSBM is sufficiently smooth… ▽ More

    Submitted 10 February, 2020; v1 submitted 7 February, 2020; originally announced February 2020.

  13. arXiv:1905.04943  [pdf, other

    cs.LG stat.ML

    Universal Invariant and Equivariant Graph Neural Networks

    Authors: Nicolas Keriven, Gabriel Peyré

    Abstract: Graph Neural Networks (GNN) come in many flavors, but should always be either invariant (permutation of the nodes of the input graph does not affect the output) or equivariant (permutation of the input permutes the output). In this paper, we consider a specific class of invariant and equivariant networks, for which we prove new universality theorems. More precisely, we consider networks with a sin… ▽ More

    Submitted 24 October, 2019; v1 submitted 13 May, 2019; originally announced May 2019.

  14. arXiv:1805.08061  [pdf, other

    cs.LG stat.ML

    NEWMA: a new method for scalable model-free online change-point detection

    Authors: Nicolas Keriven, Damien Garreau, Iacopo Poli

    Abstract: We consider the problem of detecting abrupt changes in the distribution of a multi-dimensional time series, with limited computing power and memory. In this paper, we propose a new, simple method for model-free online change-point detection that relies only on fast and light recursive statistics, inspired by the classical Exponential Weighted Moving Average algorithm (EWMA). The proposed idea is t… ▽ More

    Submitted 1 April, 2020; v1 submitted 21 May, 2018; originally announced May 2018.

  15. arXiv:1711.04460  [pdf, other

    stat.ML cs.SD eess.AS

    Blind Source Separation Using Mixtures of Alpha-Stable Distributions

    Authors: Nicolas Keriven, Antoine Deleforge, Antoine Liutkus

    Abstract: We propose a new blind source separation algorithm based on mixtures of alpha-stable distributions. Complex symmetric alpha-stable distributions have been recently showed to better model audio signals in the time-frequency domain than classical Gaussian distributions thanks to their larger dynamic range. However, inference of these models is notoriously hard to perform because their probability de… ▽ More

    Submitted 12 February, 2018; v1 submitted 13 November, 2017; originally announced November 2017.

    Comments: International Conference on Acoustics, Speech and Signal Processing (ICASSP), Apr 2018, Calgary, Canada

  16. arXiv:1706.07180  [pdf, ps, other

    stat.ML cs.IT cs.LG math.ST

    Compressive Statistical Learning with Random Feature Moments

    Authors: Rémi Gribonval, Gilles Blanchard, Nicolas Keriven, Yann Traonmilin

    Abstract: We describe a general framework -- compressive statistical learning -- for resource-efficient large-scale learning: the training collection is compressed in one pass into a low-dimensional sketch (a vector of random empirical generalized moments) that captures the information relevant to the considered learning task. A near-minimizer of the risk is computed from the sketch through the solution of… ▽ More

    Submitted 22 June, 2021; v1 submitted 22 June, 2017; originally announced June 2017.

    Comments: Main novelties between version 1 and version 2: improved concentration bounds, improved sketch sizes for compressive k-means and compressive GMM that now scale linearly with the ambient dimensionMain novelties of version 3: all content on compressive clustering and compressive GMM is now developed in the companion paper hal-02536818; improved statistical guarantees in a generic framework with illustration of the improvements on compressive PCA. Mathematical Statistics and Learning, EMS Publishing House, In press

  17. arXiv:1610.08738  [pdf, other

    cs.LG stat.ML

    Compressive K-means

    Authors: Nicolas Keriven, Nicolas Tremblay, Yann Traonmilin, Rémi Gribonval

    Abstract: The Lloyd-Max algorithm is a classical approach to perform K-means clustering. Unfortunately, its cost becomes prohibitive as the training dataset grows large. We propose a compressive version of K-means (CKM), that estimates cluster centers from a sketch, i.e. from a drastically compressed representation of the training dataset. We demonstrate empirically that CKM performs similarly to Lloyd-Max,… ▽ More

    Submitted 10 February, 2017; v1 submitted 27 October, 2016; originally announced October 2016.

  18. arXiv:1606.02838  [pdf, other

    cs.LG stat.ML

    Sketching for Large-Scale Learning of Mixture Models

    Authors: Nicolas Keriven, Anthony Bourrier, Rémi Gribonval, Patrick Pérez

    Abstract: Learning parameters from voluminous data can be prohibitive in terms of memory and computational requirements. We propose a "compressive learning" framework where we estimate model parameters from a sketch of the training data. This sketch is a collection of generalized moments of the underlying probability distribution of the data. It can be computed in a single pass on the training set, and is e… ▽ More

    Submitted 5 May, 2017; v1 submitted 9 June, 2016; originally announced June 2016.