-
A generative flow for conditional sampling via optimal transport
Authors:
Jason Alfonso,
Ricardo Baptista,
Anupam Bhakta,
Noam Gal,
Alfin Hou,
Isa Lyubimova,
Daniel Pocklington,
Josef Sajonz,
Giulio Trigila,
Ryan Tsai
Abstract:
Sampling conditional distributions is a fundamental task for Bayesian inference and density estimation. Generative models, such as normalizing flows and generative adversarial networks, characterize conditional distributions by learning a transport map that pushes forward a simple reference (e.g., a standard Gaussian) to a target distribution. While these approaches successfully describe many non-…
▽ More
Sampling conditional distributions is a fundamental task for Bayesian inference and density estimation. Generative models, such as normalizing flows and generative adversarial networks, characterize conditional distributions by learning a transport map that pushes forward a simple reference (e.g., a standard Gaussian) to a target distribution. While these approaches successfully describe many non-Gaussian problems, their performance is often limited by parametric bias and the reliability of gradient-based (adversarial) optimizers to learn these transformations. This work proposes a non-parametric generative model that iteratively maps reference samples to the target. The model uses block-triangular transport maps, whose components are shown to characterize conditionals of the target distribution. These maps arise from solving an optimal transport problem with a weighted $L^2$ cost function, thereby extending the data-driven approach in [Trigila and Tabak, 2016] for conditional sampling. The proposed approach is demonstrated on a two dimensional example and on a parameter inference problem involving nonlinear ODEs.
△ Less
Submitted 9 July, 2023;
originally announced July 2023.
-
Distributional barycenter problem through data-driven flows
Authors:
Esteban G. Tabak,
Giulio Trigila,
Wenjun Zhao
Abstract:
A new method is proposed for the solution of the data-driven optimal transport barycenter problem and of the more general distributional barycenter problem that the article introduces. The method improves on previous approaches based on adversarial games, by slaving the discriminator to the generator, minimizing the need for parameterizations and by allowing the adoption of general cost functions.…
▽ More
A new method is proposed for the solution of the data-driven optimal transport barycenter problem and of the more general distributional barycenter problem that the article introduces. The method improves on previous approaches based on adversarial games, by slaving the discriminator to the generator, minimizing the need for parameterizations and by allowing the adoption of general cost functions. It is applied to numerical examples, which include analyzing the MNIST data set with a new cost function that penalizes non-isometric maps.
△ Less
Submitted 29 April, 2021;
originally announced April 2021.
-
Data Driven Conditional Optimal Transport
Authors:
Esteban G. Tabak,
Giulio Trigila,
Wenjun Zhao
Abstract:
A data driven procedure is developed to compute the optimal map between two conditional probabilities $ρ(x|z_{1},...,z_{L})$ and $μ(y|z_{1},...,z_{L})$ depending on a set of covariates $z_{i}$. The procedure is tested on synthetic data from the ACIC Data Analysis Challenge 2017 and it is applied to non uniform lightness transfer between images. Exactly solvable examples and simulations are perform…
▽ More
A data driven procedure is developed to compute the optimal map between two conditional probabilities $ρ(x|z_{1},...,z_{L})$ and $μ(y|z_{1},...,z_{L})$ depending on a set of covariates $z_{i}$. The procedure is tested on synthetic data from the ACIC Data Analysis Challenge 2017 and it is applied to non uniform lightness transfer between images. Exactly solvable examples and simulations are performed to highlight the differences with ordinary optimal transport.
△ Less
Submitted 24 October, 2019;
originally announced October 2019.
-
An implicit gradient-descent procedure for minimax problems
Authors:
Montacer Essid,
Esteban Tabak,
Giulio Trigila
Abstract:
A game theory inspired methodology is proposed for finding a function's saddle points. While explicit descent methods are known to have severe convergence issues, implicit methods are natural in an adversarial setting, as they take the other player's optimal strategy into account. The implicit scheme proposed has an adaptive learning rate that makes it transition to Newton's method in the neighbor…
▽ More
A game theory inspired methodology is proposed for finding a function's saddle points. While explicit descent methods are known to have severe convergence issues, implicit methods are natural in an adversarial setting, as they take the other player's optimal strategy into account. The implicit scheme proposed has an adaptive learning rate that makes it transition to Newton's method in the neighborhood of saddle points. Convergence is shown through local analysis and, in non convex-concave settings, thorough numerical examples in optimal transport and linear programming. An ad-hoc quasi Newton method is developed for high dimensional problems, for which the inversion of the Hessian of the objective function may entail a high computational cost.
△ Less
Submitted 1 June, 2019;
originally announced June 2019.
-
The data-driven Schroedinger bridge
Authors:
Michele Pavon,
Esteban G Tabak,
Giulio Trigila
Abstract:
Erwin Schroedinger posed, and to a large extent solved in 1931/32 the problem of finding the most likely random evolution between two continuous probability distributions. This article considers this problem in the case when only samples of the two distributions are available. A novel iterative procedure is proposed, inspired by Fortet-Sinkhorn type algorithms. Since only samples of the marginals…
▽ More
Erwin Schroedinger posed, and to a large extent solved in 1931/32 the problem of finding the most likely random evolution between two continuous probability distributions. This article considers this problem in the case when only samples of the two distributions are available. A novel iterative procedure is proposed, inspired by Fortet-Sinkhorn type algorithms. Since only samples of the marginals are available, the new approach features constrained maximum likelihood estimation in place of the nonlinear boundary couplings, and importance sampling to propagate the functions $\varphi$ and $\hat{\varphi}$ solving the Schroedinger system. This method is well-suited to high-dimensional settings, where introducing grids leads to numerically unfeasible or unreliable methods. The methodology is illustrated in two applications: entropic interpolation of two-dimensional Gaussian mixtures, and the estimation of integrals through a variation of importance sampling.
△ Less
Submitted 5 June, 2018; v1 submitted 4 June, 2018;
originally announced June 2018.
