-
On Closed-Form Expressions for the Fisher-Rao Distance
Authors:
Henrique K. Miyamoto,
Fábio C. C. Meneghetti,
Julianna Pinele,
Sueli I. R. Costa
Abstract:
The Fisher-Rao distance is the geodesic distance between probability distributions in a statistical manifold equipped with the Fisher metric, which is a natural choice of Riemannian metric on such manifolds. It has recently been applied to supervised and unsupervised problems in machine learning, in various contexts. Finding closed-form expressions for the Fisher-Rao distance is generally a non-tr…
▽ More
The Fisher-Rao distance is the geodesic distance between probability distributions in a statistical manifold equipped with the Fisher metric, which is a natural choice of Riemannian metric on such manifolds. It has recently been applied to supervised and unsupervised problems in machine learning, in various contexts. Finding closed-form expressions for the Fisher-Rao distance is generally a non-trivial task, and those are only available for a few families of probability distributions. In this survey, we collect examples of closed-form expressions for the Fisher-Rao distance of both discrete and continuous distributions, aiming to present them in a unified and accessible language. In doing so, we also: illustrate the relation between negative multinomial distributions and the hyperbolic model, include a few new examples, and write a few more in the standard form of elliptical distributions.
△ Less
Submitted 27 February, 2024; v1 submitted 28 April, 2023;
originally announced April 2023.
-
Information Properties of a Random Variable Decomposition through Lattices
Authors:
Fábio C. C. Meneghetti,
Henrique K. Miyamoto,
Sueli I. R. Costa
Abstract:
A full-rank lattice in the Euclidean space is a discrete set formed by all integer linear combinations of a basis. Given a probability distribution on $\mathbb{R}^n$, two operations can be induced by considering the quotient of the space by such a lattice: wrap** and quantization. For a lattice $Λ$, and a fundamental domain $D$ which tiles $\mathbb{R}^n$ through $Λ$, the wrapped distribution ove…
▽ More
A full-rank lattice in the Euclidean space is a discrete set formed by all integer linear combinations of a basis. Given a probability distribution on $\mathbb{R}^n$, two operations can be induced by considering the quotient of the space by such a lattice: wrap** and quantization. For a lattice $Λ$, and a fundamental domain $D$ which tiles $\mathbb{R}^n$ through $Λ$, the wrapped distribution over the quotient is obtained by summing the density over each coset, while the quantized distribution over the lattice is defined by integrating over each fundamental domain translation. These operations define wrapped and quantized random variables over $D$ and $Λ$, respectively, which sum up to the original random variable. We investigate information-theoretic properties of this decomposition, such as entropy, mutual information and the Fisher information matrix, and show that it naturally generalizes to the more abstract context of locally compact topological groups.
△ Less
Submitted 16 November, 2022; v1 submitted 7 November, 2022;
originally announced November 2022.
-
The Fisher-Rao Loss for Learning under Label Noise
Authors:
Henrique K. Miyamoto,
Fábio C. C. Meneghetti,
Sueli I. R. Costa
Abstract:
Choosing a suitable loss function is essential when learning by empirical risk minimisation. In many practical cases, the datasets used for training a classifier may contain incorrect labels, which prompts the interest for using loss functions that are inherently robust to label noise. In this paper, we study the Fisher-Rao loss function, which emerges from the Fisher-Rao distance in the statistic…
▽ More
Choosing a suitable loss function is essential when learning by empirical risk minimisation. In many practical cases, the datasets used for training a classifier may contain incorrect labels, which prompts the interest for using loss functions that are inherently robust to label noise. In this paper, we study the Fisher-Rao loss function, which emerges from the Fisher-Rao distance in the statistical manifold of discrete distributions. We derive an upper bound for the performance degradation in the presence of label noise, and analyse the learning speed of this loss. Comparing with other commonly used losses, we argue that the Fisher-Rao loss provides a natural trade-off between robustness and training dynamics. Numerical experiments with synthetic and MNIST datasets illustrate this performance.
△ Less
Submitted 26 November, 2022; v1 submitted 28 October, 2022;
originally announced October 2022.
