-
Croissant: A Metadata Format for ML-Ready Datasets
Authors:
Mubashara Akhtar,
Omar Benjelloun,
Costanza Conforti,
Pieter Gijsbers,
Joan Giner-Miguelez,
Nitisha Jain,
Michael Kuchnik,
Quentin Lhoest,
Pierre Marcenac,
Manil Maskey,
Peter Mattson,
Luis Oala,
Pierre Ruyssen,
Rajat Shinde,
Elena Simperl,
Goeffry Thomas,
Slava Tykhonov,
Joaquin Vanschoren,
Jos van der Velde,
Steffen Vogler,
Carole-Jean Wu
Abstract:
Data is a critical resource for Machine Learning (ML), yet working with data remains a key friction point. This paper introduces Croissant, a metadata format for datasets that simplifies how data is used by ML tools and frameworks. Croissant makes datasets more discoverable, portable and interoperable, thereby addressing significant challenges in ML data management and responsible AI. Croissant is…
▽ More
Data is a critical resource for Machine Learning (ML), yet working with data remains a key friction point. This paper introduces Croissant, a metadata format for datasets that simplifies how data is used by ML tools and frameworks. Croissant makes datasets more discoverable, portable and interoperable, thereby addressing significant challenges in ML data management and responsible AI. Croissant is already supported by several popular dataset repositories, spanning hundreds of thousands of datasets, ready to be loaded into the most popular ML frameworks.
△ Less
Submitted 30 May, 2024; v1 submitted 28 March, 2024;
originally announced March 2024.
-
Optimal Stop** via Randomized Neural Networks
Authors:
Calypso Herrera,
Florian Krach,
Pierre Ruyssen,
Josef Teichmann
Abstract:
This paper presents the benefits of using randomized neural networks instead of standard basis functions or deep neural networks to approximate the solutions of optimal stop** problems. The key idea is to use neural networks, where the parameters of the hidden layers are generated randomly and only the last layer is trained, in order to approximate the continuation value. Our approaches are appl…
▽ More
This paper presents the benefits of using randomized neural networks instead of standard basis functions or deep neural networks to approximate the solutions of optimal stop** problems. The key idea is to use neural networks, where the parameters of the hidden layers are generated randomly and only the last layer is trained, in order to approximate the continuation value. Our approaches are applicable to high dimensional problems where the existing approaches become increasingly impractical. In addition, since our approaches can be optimized using simple linear regression, they are easy to implement and theoretical guarantees can be provided. We test our approaches for American option pricing on Black--Scholes, Heston and rough Heston models and for optimally stop** a fractional Brownian motion. In all cases, our algorithms outperform the state-of-the-art and other relevant machine learning approaches in terms of computation time while achieving comparable results. Moreover, we show that they can also be used to efficiently compute Greeks of American options.
△ Less
Submitted 1 December, 2023; v1 submitted 28 April, 2021;
originally announced April 2021.
-
Denise: Deep Robust Principal Component Analysis for Positive Semidefinite Matrices
Authors:
Calypso Herrera,
Florian Krach,
Anastasis Kratsios,
Pierre Ruyssen,
Josef Teichmann
Abstract:
The robust PCA of covariance matrices plays an essential role when isolating key explanatory features. The currently available methods for performing such a low-rank plus sparse decomposition are matrix specific, meaning, those algorithms must re-run for every new matrix. Since these algorithms are computationally expensive, it is preferable to learn and store a function that nearly instantaneousl…
▽ More
The robust PCA of covariance matrices plays an essential role when isolating key explanatory features. The currently available methods for performing such a low-rank plus sparse decomposition are matrix specific, meaning, those algorithms must re-run for every new matrix. Since these algorithms are computationally expensive, it is preferable to learn and store a function that nearly instantaneously performs this decomposition when evaluated. Therefore, we introduce Denise, a deep learning-based algorithm for robust PCA of covariance matrices, or more generally, of symmetric positive semidefinite matrices, which learns precisely such a function. Theoretical guarantees for Denise are provided. These include a novel universal approximation theorem adapted to our geometric deep learning problem and convergence to an optimal solution to the learning problem. Our experiments show that Denise matches state-of-the-art performance in terms of decomposition quality, while being approximately $2000\times$ faster than the state-of-the-art, principal component pursuit (PCP), and $200 \times$ faster than the current speed-optimized method, fast PCP.
△ Less
Submitted 6 June, 2023; v1 submitted 28 April, 2020;
originally announced April 2020.
-
A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmark
Authors:
Xiaohua Zhai,
Joan Puigcerver,
Alexander Kolesnikov,
Pierre Ruyssen,
Carlos Riquelme,
Mario Lucic,
Josip Djolonga,
Andre Susano Pinto,
Maxim Neumann,
Alexey Dosovitskiy,
Lucas Beyer,
Olivier Bachem,
Michael Tschannen,
Marcin Michalski,
Olivier Bousquet,
Sylvain Gelly,
Neil Houlsby
Abstract:
Representation learning promises to unlock deep learning for the long tail of vision tasks without expensive labelled datasets. Yet, the absence of a unified evaluation for general visual representations hinders progress. Popular protocols are often too constrained (linear classification), limited in diversity (ImageNet, CIFAR, Pascal-VOC), or only weakly related to representation quality (ELBO, r…
▽ More
Representation learning promises to unlock deep learning for the long tail of vision tasks without expensive labelled datasets. Yet, the absence of a unified evaluation for general visual representations hinders progress. Popular protocols are often too constrained (linear classification), limited in diversity (ImageNet, CIFAR, Pascal-VOC), or only weakly related to representation quality (ELBO, reconstruction error). We present the Visual Task Adaptation Benchmark (VTAB), which defines good representations as those that adapt to diverse, unseen tasks with few examples. With VTAB, we conduct a large-scale study of many popular publicly-available representation learning algorithms. We carefully control confounders such as architecture and tuning budget. We address questions like: How effective are ImageNet representations beyond standard natural datasets? How do representations trained via generative and discriminative models compare? To what extent can self-supervision replace labels? And, how close are we to general visual representations?
△ Less
Submitted 21 February, 2020; v1 submitted 1 October, 2019;
originally announced October 2019.
-
Low-Rank plus Sparse Decomposition of Covariance Matrices using Neural Network Parametrization
Authors:
Michel Baes,
Calypso Herrera,
Ariel Neufeld,
Pierre Ruyssen
Abstract:
This paper revisits the problem of decomposing a positive semidefinite matrix as a sum of a matrix with a given rank plus a sparse matrix. An immediate application can be found in portfolio optimization, when the matrix to be decomposed is the covariance between the different assets in the portfolio. Our approach consists in representing the low-rank part of the solution as the product $MM^{T}$, w…
▽ More
This paper revisits the problem of decomposing a positive semidefinite matrix as a sum of a matrix with a given rank plus a sparse matrix. An immediate application can be found in portfolio optimization, when the matrix to be decomposed is the covariance between the different assets in the portfolio. Our approach consists in representing the low-rank part of the solution as the product $MM^{T}$, where $M$ is a rectangular matrix of appropriate size, parametrized by the coefficients of a deep neural network. We then use a gradient descent algorithm to minimize an appropriate loss function over the parameters of the network. We deduce its convergence rate to a local optimum from the Lipschitz smoothness of our loss function. We show that the rate of convergence grows polynomially in the dimensions of the input, output, and the size of each of the hidden layers.
△ Less
Submitted 15 June, 2021; v1 submitted 1 August, 2019;
originally announced August 2019.
