-
On Semi-supervised Estimation of Discrete Distributions under f-divergences
Authors:
Hasan Sabri Melihcan Erol,
Lizhong Zheng
Abstract:
We study the problem of estimating the joint probability mass function (pmf) over two random variables. In particular, the estimation is based on the observation of $m$ samples containing both variables and $n$ samples missing one fixed variable. We adopt the minimax framework with $l^p_p$ loss functions. Recent work established that univariate minimax estimator combinations achieve minimax risk w…
▽ More
We study the problem of estimating the joint probability mass function (pmf) over two random variables. In particular, the estimation is based on the observation of $m$ samples containing both variables and $n$ samples missing one fixed variable. We adopt the minimax framework with $l^p_p$ loss functions. Recent work established that univariate minimax estimator combinations achieve minimax risk with the optimal first-order constant for $p \ge 2$ in the regime $m = o(n)$, questions remained for $p \le 2$ and various $f$-divergences. In our study, we affirm that these composite estimators are indeed minimax optimal for $l^p_p$ loss functions, specifically for the range $1 \le p \le 2$, including the critical $l_1$ loss. Additionally, we ascertain their optimality for a suite of $f$-divergences, such as KL, $χ^2$, Squared Hellinger, and Le Cam divergences.
△ Less
Submitted 15 May, 2024;
originally announced May 2024.
-
Operator SVD with Neural Networks via Nested Low-Rank Approximation
Authors:
J. Jon Ryu,
Xiangxiang Xu,
H. S. Melihcan Erol,
Yuheng Bu,
Lizhong Zheng,
Gregory W. Wornell
Abstract:
Computing eigenvalue decomposition (EVD) of a given linear operator, or finding its leading eigenvalues and eigenfunctions, is a fundamental task in many machine learning and scientific computing problems. For high-dimensional eigenvalue problems, training neural networks to parameterize the eigenfunctions is considered as a promising alternative to the classical numerical linear algebra technique…
▽ More
Computing eigenvalue decomposition (EVD) of a given linear operator, or finding its leading eigenvalues and eigenfunctions, is a fundamental task in many machine learning and scientific computing problems. For high-dimensional eigenvalue problems, training neural networks to parameterize the eigenfunctions is considered as a promising alternative to the classical numerical linear algebra techniques. This paper proposes a new optimization framework based on the low-rank approximation characterization of a truncated singular value decomposition, accompanied by new techniques called nesting for learning the top-$L$ singular values and singular functions in the correct order. The proposed method promotes the desired orthogonality in the learned functions implicitly and efficiently via an unconstrained optimization formulation, which is easy to solve with off-the-shelf gradient-based optimization algorithms. We demonstrate the effectiveness of the proposed optimization framework for use cases in computational physics and machine learning.
△ Less
Submitted 5 February, 2024;
originally announced February 2024.
-
On Semi-Supervised Estimation of Distributions
Authors:
H. S. Melihcan Erol,
Erixhen Sula,
Lizhong Zheng
Abstract:
We study the problem of estimating the joint probability mass function (pmf) over two random variables. In particular, the estimation is based on the observation of $m$ samples containing both variables and $n$ samples missing one fixed variable. We adopt the minimax framework with $l^p_p$ loss functions, and we show that the composition of uni-variate minimax estimators achieves minimax risk with…
▽ More
We study the problem of estimating the joint probability mass function (pmf) over two random variables. In particular, the estimation is based on the observation of $m$ samples containing both variables and $n$ samples missing one fixed variable. We adopt the minimax framework with $l^p_p$ loss functions, and we show that the composition of uni-variate minimax estimators achieves minimax risk with the optimal first-order constant for $p \ge 2$, in the regime $m = o(n)$.
△ Less
Submitted 15 May, 2023; v1 submitted 13 May, 2023;
originally announced May 2023.
-
Variations on Hammersley's interacting particle process
Authors:
Arda Atalik,
H. S. Melihcan Erol,
Gökhan Yıldırım,
Mustafa Yilmaz
Abstract:
The longest increasing subsequence problem for permutations has been studied extensively in the last fifty years. The interpretation of the longest increasing subsequence as the longest 21-avoiding subsequence in the context of permutation patterns leads to many interesting research directions. We introduce and study the statistical properties of Hammersleytype interacting particle processes relat…
▽ More
The longest increasing subsequence problem for permutations has been studied extensively in the last fifty years. The interpretation of the longest increasing subsequence as the longest 21-avoiding subsequence in the context of permutation patterns leads to many interesting research directions. We introduce and study the statistical properties of Hammersleytype interacting particle processes related to these generalizations and explore the finer structures of their distributions. We also propose three different interacting particle systems in the plane analogous to the Hammersley process in one dimension and obtain estimates for the asymptotic orders of the mean and variance of the number of particles in the systems.
△ Less
Submitted 21 June, 2021;
originally announced June 2021.
