-
On the conservation laws and the structure of the nonlinearity for SQG and its generalizations
Authors:
Philip Isett,
Andrew Ma
Abstract:
Using a new definition for the nonlinear term, we prove that all weak solutions to the SQG equation (and mSQG) conserve the angular momentum. This result is new for the weak solutions of [Resnick, '95] and rules out the possibility of anomalous dissipation of angular momentum. We also prove conservation of the Hamiltonian under conjecturally optimal assumptions, sharpening a well-known criterion o…
▽ More
Using a new definition for the nonlinear term, we prove that all weak solutions to the SQG equation (and mSQG) conserve the angular momentum. This result is new for the weak solutions of [Resnick, '95] and rules out the possibility of anomalous dissipation of angular momentum. We also prove conservation of the Hamiltonian under conjecturally optimal assumptions, sharpening a well-known criterion of [Cheskidov-Constantin-Friedlander-Shvydkoy, '08]. Moreover, we show that our new estimate for the nonlinearity is optimal and that it characterizes the mSQG nonlinearity uniquely among active scalar nonlinearities with a scaling symmetry.
△ Less
Submitted 13 March, 2024;
originally announced March 2024.
-
Degree conditions for disjoint path covers in digraphs
Authors:
Ansong Ma,
Yuefang Sun
Abstract:
In this paper, we study degree conditions for three types of disjoint directed path cover problems: many-to-many $k$-DDPC, one-to-many $k$-DDPC and one-to-one $k$-DDPC, which are intimately connected to other famous topics in graph theory, such as Hamiltonicity and $k$-linkage, and have a strong background of applications.
Firstly, we get two sharp minimum semi-degree sufficient conditions for t…
▽ More
In this paper, we study degree conditions for three types of disjoint directed path cover problems: many-to-many $k$-DDPC, one-to-many $k$-DDPC and one-to-one $k$-DDPC, which are intimately connected to other famous topics in graph theory, such as Hamiltonicity and $k$-linkage, and have a strong background of applications.
Firstly, we get two sharp minimum semi-degree sufficient conditions for the unpaired many-to-many $k$-DDPC problem and a sharp Ore-type degree condition for the paired many-to-many $2$-DDPC problem. Secondly, we obtain a minimum semi-degree sufficient condition for the one-to-many $k$-DDPC problem on a digraph with order $n$, and show that the bound for the minimum semi-degree is sharp when $n+k$ is even and is sharp up to an additive constant 1 otherwise. Finally, we give a minimum semi-degree sufficient condition for the one-to-one $k$-DDPC problem on a digraph with order $n$, and show that the bound for the minimum semi-degree is sharp when $n+k$ is odd and is sharp up to an additive constant 1 otherwise.
△ Less
Submitted 28 February, 2024; v1 submitted 21 February, 2024;
originally announced February 2024.
-
Block-missing data in linear systems: An unbiased stochastic gradient descent approach
Authors:
Chelsea Huynh,
Anna Ma,
Michael Strand
Abstract:
Achieving accurate approximations to solutions of large linear systems is crucial, especially when those systems utilize real-world data. A consequence of using real-world data is that there will inevitably be missingness. Current approaches for dealing with missing data, such as deletion and imputation, can introduce bias. Recent studies proposed an adaptation of stochastic gradient descent (SGD)…
▽ More
Achieving accurate approximations to solutions of large linear systems is crucial, especially when those systems utilize real-world data. A consequence of using real-world data is that there will inevitably be missingness. Current approaches for dealing with missing data, such as deletion and imputation, can introduce bias. Recent studies proposed an adaptation of stochastic gradient descent (SGD) in specific missing-data models. In this work, we propose a new algorithm, $\ell$-tuple mSGD, for the setting in which data is missing in a block-wise, tuple pattern. We prove that our proposed method uses unbiased estimates of the gradient of the least squares objective in the presence of tuple missing data. We also draw connections between $\ell$-tuple mSGD and previously established SGD-type methods for missing data. Furthermore, we prove our algorithm converges when using updating step sizes and empirically demonstrate the convergence of $\ell$-tuple mSGD on synthetic data. Lastly, we evaluate $\ell$-tuple mSGD applied to real-world continuous glucose monitoring (CGM) device data.
△ Less
Submitted 16 October, 2023;
originally announced October 2023.
-
A Note on Randomized Kaczmarz Algorithm for Solving Doubly-Noisy Linear Systems
Authors:
El Houcine Bergou,
Soumia Boucherouite,
Aritra Dutta,
Xin Li,
Anna Ma
Abstract:
Large-scale linear systems, $Ax=b$, frequently arise in practice and demand effective iterative solvers. Often, these systems are noisy due to operational errors or faulty data-collection processes. In the past decade, the randomized Kaczmarz (RK) algorithm has been studied extensively as an efficient iterative solver for such systems. However, the convergence study of RK in the noisy regime is li…
▽ More
Large-scale linear systems, $Ax=b$, frequently arise in practice and demand effective iterative solvers. Often, these systems are noisy due to operational errors or faulty data-collection processes. In the past decade, the randomized Kaczmarz (RK) algorithm has been studied extensively as an efficient iterative solver for such systems. However, the convergence study of RK in the noisy regime is limited and considers measurement noise in the right-hand side vector, $b$. Unfortunately, in practice, that is not always the case; the coefficient matrix $A$ can also be noisy. In this paper, we analyze the convergence of RK for noisy linear systems when the coefficient matrix, $A$, is corrupted with both additive and multiplicative noise, along with the noisy vector, $b$. In our analyses, the quantity $\tilde R=\| \tilde A^{\dagger} \|_2^2 \|\tilde A \|_F^2$ influences the convergence of RK, where $\tilde A$ represents a noisy version of $A$. We claim that our analysis is robust and realistically applicable, as we do not require information about the noiseless coefficient matrix, $A$, and considering different conditions on noise, we can control the convergence of RK. We substantiate our theoretical findings by performing comprehensive numerical experiments.
△ Less
Submitted 31 August, 2023;
originally announced August 2023.
-
On Subsampled Quantile Randomized Kaczmarz
Authors:
Jamie Haddock,
Anna Ma,
Elizaveta Rebrova
Abstract:
When solving noisy linear systems Ax = b + c, the theoretical and empirical performance of stochastic iterative methods, such as the Randomized Kaczmarz algorithm, depends on the noise level. However, if there are a small number of highly corrupt measurements, one can instead use quantile-based methods to guarantee convergence to the solution x of the system, despite the presence of noise. Such me…
▽ More
When solving noisy linear systems Ax = b + c, the theoretical and empirical performance of stochastic iterative methods, such as the Randomized Kaczmarz algorithm, depends on the noise level. However, if there are a small number of highly corrupt measurements, one can instead use quantile-based methods to guarantee convergence to the solution x of the system, despite the presence of noise. Such methods require the computation of the entire residual vector, which may not be desirable or even feasible in some cases. In this work, we analyze the sub-sampled quantile Randomized Kaczmarz (sQRK) algorithm for solving large-scale linear systems which utilize a sub-sampled residual to approximate the quantile threshold. We prove that this method converges to the unique solution to the linear system and provide numerical experiments that support our theoretical findings. We additionally remark on the extremely small sample size case and demonstrate the importance of interplay between the choice of quantile and subset size.
△ Less
Submitted 15 August, 2023;
originally announced August 2023.
-
Stochastic Natural Thresholding Algorithms
Authors:
Rachel Grotheer,
Shuang Li,
Anna Ma,
Deanna Needell,
**g Qin
Abstract:
Sparse signal recovery is one of the most fundamental problems in various applications, including medical imaging and remote sensing. Many greedy algorithms based on the family of hard thresholding operators have been developed to solve the sparse signal recovery problem. More recently, Natural Thresholding (NT) has been proposed with improved computational efficiency. This paper proposes and disc…
▽ More
Sparse signal recovery is one of the most fundamental problems in various applications, including medical imaging and remote sensing. Many greedy algorithms based on the family of hard thresholding operators have been developed to solve the sparse signal recovery problem. More recently, Natural Thresholding (NT) has been proposed with improved computational efficiency. This paper proposes and discusses convergence guarantees for stochastic natural thresholding algorithms by extending the NT from the deterministic version with linear measurements to the stochastic version with a general objective function. We also conduct various numerical experiments on linear and nonlinear measurements to demonstrate the performance of StoNT.
△ Less
Submitted 7 June, 2023;
originally announced June 2023.
-
Efficient and Robust Bayesian Selection of Hyperparameters in Dimension Reduction for Visualization
Authors:
Yin-Ting Liao,
Hengrui Luo,
Anna Ma
Abstract:
We introduce an efficient and robust auto-tuning framework for hyperparameter selection in dimension reduction (DR) algorithms, focusing on large-scale datasets and arbitrary performance metrics. By leveraging Bayesian optimization (BO) with a surrogate model, our approach enables efficient hyperparameter selection with multi-objective trade-offs and allows us to perform data-driven sensitivity an…
▽ More
We introduce an efficient and robust auto-tuning framework for hyperparameter selection in dimension reduction (DR) algorithms, focusing on large-scale datasets and arbitrary performance metrics. By leveraging Bayesian optimization (BO) with a surrogate model, our approach enables efficient hyperparameter selection with multi-objective trade-offs and allows us to perform data-driven sensitivity analysis. By incorporating normalization and subsampling, the proposed framework demonstrates versatility and efficiency, as shown in applications to visualization techniques such as t-SNE and UMAP. We evaluate our results on various synthetic and real-world datasets using multiple quality metrics, providing a robust and efficient solution for hyperparameter selection in DR algorithms.
△ Less
Submitted 1 June, 2023;
originally announced June 2023.
-
Existence and concentration of ground state solution to a nonlocal Schrödinger equation
Authors:
Anmin Mao,
Qian Zhang
Abstract:
We study a class of Schrödinger-Kirchhoff system involving critical exponent. We aim to find suitable conditions to assure the existence of a positive ground state solution of Nehari-Pohouzaev type $u_{\varepsilon}$ with exponential decay at infinity for $\varepsilon$ and $ u_{\varepsilon}$ concentrates around a global minimum point of $ V$ as $ \varepsilon\rightarrow0^{+}.$ The nonlinear term inc…
▽ More
We study a class of Schrödinger-Kirchhoff system involving critical exponent. We aim to find suitable conditions to assure the existence of a positive ground state solution of Nehari-Pohouzaev type $u_{\varepsilon}$ with exponential decay at infinity for $\varepsilon$ and $ u_{\varepsilon}$ concentrates around a global minimum point of $ V$ as $ \varepsilon\rightarrow0^{+}.$ The nonlinear term includes the nonlinearity $f(u)\sim|u|^{p-1}u$ for the well-studied case $ p\in[3,5)$, and the less-studied case $p\in(2,3)$.
△ Less
Submitted 25 May, 2023;
originally announced May 2023.
-
Iterative Singular Tube Hard Thresholding Algorithms for Tensor Recovery
Authors:
Rachel Grotheer,
Shuang Li,
Anna Ma,
Deanna Needell,
**g Qin
Abstract:
Due to the explosive growth of large-scale data sets, tensors have been a vital tool to analyze and process high-dimensional data. Different from the matrix case, tensor decomposition has been defined in various formats, which can be further used to define the best low-rank approximation of a tensor to significantly reduce the dimensionality for signal compression and recovery. In this paper, we c…
▽ More
Due to the explosive growth of large-scale data sets, tensors have been a vital tool to analyze and process high-dimensional data. Different from the matrix case, tensor decomposition has been defined in various formats, which can be further used to define the best low-rank approximation of a tensor to significantly reduce the dimensionality for signal compression and recovery. In this paper, we consider the low-rank tensor recovery problem when the tubal rank of the underlying tensor is given or estimated a priori. We propose a novel class of iterative singular tube hard thresholding algorithms for tensor recovery based on the low-tubal-rank tensor approximation, including basic, accelerated deterministic and stochastic versions. Convergence guarantees are provided along with the special case when the measurements are linear. Numerical experiments on tensor compressive sensing and color image inpainting are conducted to demonstrate convergence and computational efficiency in practice.
△ Less
Submitted 26 December, 2023; v1 submitted 10 April, 2023;
originally announced April 2023.
-
On the factorization invariants of arithmetical congruence monoids
Authors:
Scott T. Chapman,
Caroline Liu,
Annabel Ma,
Andrew Zhang
Abstract:
In this paper, we study various factorization invariants of arithmetical congruence monoids. The invariants we investigate are the catenary degree, a measure of the maximum distance between any two factorizations of the same element, the length density, which describes the distribution of the factorization lengths of an element, and the omega primality, which measures how far an element is from be…
▽ More
In this paper, we study various factorization invariants of arithmetical congruence monoids. The invariants we investigate are the catenary degree, a measure of the maximum distance between any two factorizations of the same element, the length density, which describes the distribution of the factorization lengths of an element, and the omega primality, which measures how far an element is from being prime.
△ Less
Submitted 12 January, 2023; v1 submitted 3 October, 2022;
originally announced October 2022.
-
A minimum semi-degree condition for unpaired many-to-many disjoint path covers in digraphs
Authors:
Ansong Ma,
Yuefang Sun
Abstract:
For a digraph $D$, let $δ^{0}(D) = \min \{δ^{+}(D), δ^{-}(D)\}$ be the minimum semi-degree of $D$. A set of $k$ vertex-disjoint paths, $\{P_{1}, \dots, P_{k}\}$, joining a disjoint source set $S = \{s_{1}, \dots, s_{k}\}$ and sink set $T = \{t_{1}, \dots, t_{k}\}$ is called an unpaired many-to-many $k$-disjoint directed path cover ($k$-DDPC for short) of $D$, if each $P_{j}$ joins $s_{j}$ and…
▽ More
For a digraph $D$, let $δ^{0}(D) = \min \{δ^{+}(D), δ^{-}(D)\}$ be the minimum semi-degree of $D$. A set of $k$ vertex-disjoint paths, $\{P_{1}, \dots, P_{k}\}$, joining a disjoint source set $S = \{s_{1}, \dots, s_{k}\}$ and sink set $T = \{t_{1}, \dots, t_{k}\}$ is called an unpaired many-to-many $k$-disjoint directed path cover ($k$-DDPC for short) of $D$, if each $P_{j}$ joins $s_{j}$ and $t_{σ(j)}$ for some permutation $σ$ on $\{1, \dots , k\}$ and $\bigcup^{k}_{j=1} V(P_{j}) = V(D)$.
In this paper, we give a new proof for the following result that every digraph $D$ with $δ^{0}(D) \geq \lceil (n+k) / 2 \rceil$ has an unpaired many-to-many $k$-DDPC joining any disjoint source set $S$ and sink set $T$, where $S = \{s_{1}, \dots, s_{k}\}$ and $T = \{t_{1}, \dots, t_{k}\}$. Moreover, we show that the bound on the minimum semi-degree is best possible when $n \geq 3k$.
△ Less
Submitted 25 October, 2022; v1 submitted 1 October, 2022;
originally announced October 2022.
-
A minimum semi-degree sufficient condition for one-to-many disjoint path covers in semicomplete digraphs
Authors:
Ansong Ma,
Yuefang Sun,
Xiaoyan Zhang
Abstract:
Let $D$ be a digraph. We define the minimum semi-degree of $D$ as $δ^{0}(D) := \min \{δ^{+}(D), δ^{-}(D)\}$. Let $k$ be a positive integer, and let $S = \{s\}$ and $T = \{t_{1}, \dots ,t_{k}\}$ be any two disjoint subsets of $V(D)$. A set of $k$ internally disjoint paths joining source set $S$ and sink set $T$ that cover all vertices $D$ are called a one-to-many $k$-disjoint directed path cover (…
▽ More
Let $D$ be a digraph. We define the minimum semi-degree of $D$ as $δ^{0}(D) := \min \{δ^{+}(D), δ^{-}(D)\}$. Let $k$ be a positive integer, and let $S = \{s\}$ and $T = \{t_{1}, \dots ,t_{k}\}$ be any two disjoint subsets of $V(D)$. A set of $k$ internally disjoint paths joining source set $S$ and sink set $T$ that cover all vertices $D$ are called a one-to-many $k$-disjoint directed path cover ($k$-DDPC for short) of $D$. A digraph $D$ is semicomplete if for every pair $x,y$ of vertices of it, there is at least one arc between $x$ and $y$.
In this paper, we prove that every semicomplete digraph $D$ of sufficiently large order $n$ with $δ^{0}(D) \geq \lceil (n+k-1)/2\rceil$ has a one-to-many $k$-DDPC joining any disjoint source set $S$ and sink set $T$, where $S = \{s\}, T = \{t_{1}, \dots, t_{k}\}$.
△ Less
Submitted 19 August, 2022;
originally announced August 2022.
-
Robust recovery of low-rank matrices and low-tubal-rank tensors from noisy sketches
Authors:
Anna Ma,
Dominik Stöger,
Yizhe Zhu
Abstract:
A common approach for compressing large-scale data is through matrix sketching. In this work, we consider the problem of recovering low-rank matrices from two noisy linear sketches using the double sketching scheme discussed in Fazel et al. (2008), which is based on an approach by Woolfe et al. (2008). Using tools from non-asymptotic random matrix theory, we provide the first theoretical guarantee…
▽ More
A common approach for compressing large-scale data is through matrix sketching. In this work, we consider the problem of recovering low-rank matrices from two noisy linear sketches using the double sketching scheme discussed in Fazel et al. (2008), which is based on an approach by Woolfe et al. (2008). Using tools from non-asymptotic random matrix theory, we provide the first theoretical guarantees characterizing the error between the output of the double sketch algorithm and the ground truth low-rank matrix. We apply our result to the problems of low-rank matrix approximation and low-tubal-rank tensor recovery.
△ Less
Submitted 14 July, 2023; v1 submitted 1 June, 2022;
originally announced June 2022.
-
On the Number of Faces and Radii of Cells Induced by Gaussian Spherical Tessellations
Authors:
Eric Lybrand,
Anna Ma,
Rayan Saab
Abstract:
We study a geometric property related to spherical hyperplane tessellations in $\mathbb{R}^{d}$. We first consider a fixed $x$ on the Euclidean sphere and tessellations with $M \gg d$ hyperplanes passing through the origin having normal vectors distributed according to a Gaussian distribution. We show that with high probability there exists a subset of the hyperplanes whose cardinality is on the o…
▽ More
We study a geometric property related to spherical hyperplane tessellations in $\mathbb{R}^{d}$. We first consider a fixed $x$ on the Euclidean sphere and tessellations with $M \gg d$ hyperplanes passing through the origin having normal vectors distributed according to a Gaussian distribution. We show that with high probability there exists a subset of the hyperplanes whose cardinality is on the order of $d\log(d)\log(M)$ such that the radius of the cell containing $x$ induced by these hyperplanes is bounded above by, up to constants, $d\log(d)\log(M)/M$. We extend this result to hold for all cells in the tessellation with high probability. Up to logarithmic terms, this upper bound matches the previously established lower bound of Goyal et al. (IEEE T. Inform. Theory 44(1):16-31, 1998).
△ Less
Submitted 30 August, 2021;
originally announced August 2021.
-
A direct approach to nonuniqueness and failure of compactness for the SQG equation
Authors:
Philip Isett,
Andrew Ma
Abstract:
We give an alternative proof of the nonuniqueness of weak solutions to the surface quasigeostrophic equation (SQG) first shown in [Buckmaster-Shkoller-Vicol, '16]. Our approach proceeds directly at the level of the scalar field. Furthermore, we prove that every smooth scalar field with compact support that conserves the integral can be realized as a weak limit of solutions to SQG.
We give an alternative proof of the nonuniqueness of weak solutions to the surface quasigeostrophic equation (SQG) first shown in [Buckmaster-Shkoller-Vicol, '16]. Our approach proceeds directly at the level of the scalar field. Furthermore, we prove that every smooth scalar field with compact support that conserves the integral can be realized as a weak limit of solutions to SQG.
△ Less
Submitted 26 July, 2020; v1 submitted 6 July, 2020;
originally announced July 2020.
-
Least Squares Estimator for Vasicek Model Driven by Sub-fractional Brownian Processes from Discrete Observations
Authors:
Cuiyun Zhang,
**gjun Guo,
Aiqin Ma,
Bo Peng
Abstract:
We study the parameter estimation problem of Vasicek Model driven by sub-fractional Brownian processes from discrete observations, and let {S_t^H,t>=0} denote a sub-fractional Brownian motion whose Hurst parameter 1/2<H<1 . The studies are as follows: firstly, two unknown parameters in the model are estimated by the least squares method. Secondly, the strong consistency and the asymptotic distribu…
▽ More
We study the parameter estimation problem of Vasicek Model driven by sub-fractional Brownian processes from discrete observations, and let {S_t^H,t>=0} denote a sub-fractional Brownian motion whose Hurst parameter 1/2<H<1 . The studies are as follows: firstly, two unknown parameters in the model are estimated by the least squares method. Secondly, the strong consistency and the asymptotic distribution of the estimators are studied respectively. Finally, our estimators are validated by numerical simulation.
△ Less
Submitted 2 July, 2020;
originally announced July 2020.
-
Randomized Kaczmarz for Tensor Linear Systems
Authors:
Anna Ma,
Denali Molitor
Abstract:
Solving linear systems of equations is a fundamental problem in mathematics. When the linear system is so large that it cannot be loaded into memory at once, iterative methods such as the randomized Kaczmarz method excel. Here, we extend the randomized Kaczmarz method to solve multi-linear (tensor) systems under the tensor-tensor t-product. We provide convergence guarantees for the proposed tensor…
▽ More
Solving linear systems of equations is a fundamental problem in mathematics. When the linear system is so large that it cannot be loaded into memory at once, iterative methods such as the randomized Kaczmarz method excel. Here, we extend the randomized Kaczmarz method to solve multi-linear (tensor) systems under the tensor-tensor t-product. We provide convergence guarantees for the proposed tensor randomized Kaczmarz that are analogous to those of the randomized Kaczmarz method for matrix linear systems. We demonstrate experimentally that the tensor randomized Kaczmarz method converges faster than traditional randomized Kaczmarz applied to a naively matricized version of the linear system. In addition, we draw connections between the proposed algorithm and a previously known extension of the randomized Kaczmarz algorithm for matrix linear systems.
△ Less
Submitted 1 June, 2020;
originally announced June 2020.
-
Computer-based and paper-and-pencil tests: A study in calculus for STEM majors
Authors:
Lawrence Smolinsky,
Brian D. Marx,
Gestur Olafsson,
Yanxia A. Ma
Abstract:
Computer-based testing is an expanding use of technology offering advantages to teachers and students. We studied Calculus II classes for STEM majors using different testing modes. Three sections with 324 students employed: Paper-and-pencil testing, computer-based testing, and both. Computer tests gave immediate feedback, allowed multiple submissions, and pooling. Paper-and-pencil tests required w…
▽ More
Computer-based testing is an expanding use of technology offering advantages to teachers and students. We studied Calculus II classes for STEM majors using different testing modes. Three sections with 324 students employed: Paper-and-pencil testing, computer-based testing, and both. Computer tests gave immediate feedback, allowed multiple submissions, and pooling. Paper-and-pencil tests required work and explanation allowing inspection of high cognitive demand tasks. Each test mode used the strength of its method. Students were given the same lecture by the same instructor on the same day and the same homework assignments and due dates. The design is quasi-experimental, but students were not aware of the testing mode at registration. Two basic questions examined were: (1) Do paper-and-pencil and computer-based tests measure knowledge and skill in STEM Calculus II in a consistent manner? (2) How does the knowledge and skill gained by students in a fully computer-based Calculus II class compare to students in a class requiring pencil-and-paper tests and hence some paper-and-pencil work. These results indicate that computer-based tests are as consistent with paper-and-pencil tests as computer-based tests are with themselves. Results are also consistent with classes using paper-and-pencil tests having slightly better outcomes than fully computer-based classes using only computer assessments.
△ Less
Submitted 11 May, 2020;
originally announced May 2020.
-
Greed Works: An Improved Analysis of Sampling Kaczmarz-Motzkin
Authors:
Jamie Haddock,
Anna Ma
Abstract:
Stochastic iterative algorithms have gained recent interest in machine learning and signal processing for solving large-scale systems of equations, $Ax=b$. One such example is the Randomized Kaczmarz (RK) algorithm, which acts only on single rows of the matrix $A$ at a time. While RK randomly selects a row of $A$ to work with, Motzkin's Method (MM) employs a greedy row selection. Connections betwe…
▽ More
Stochastic iterative algorithms have gained recent interest in machine learning and signal processing for solving large-scale systems of equations, $Ax=b$. One such example is the Randomized Kaczmarz (RK) algorithm, which acts only on single rows of the matrix $A$ at a time. While RK randomly selects a row of $A$ to work with, Motzkin's Method (MM) employs a greedy row selection. Connections between the two algorithms resulted in the Sampling Kaczmarz-Motzkin (SKM) algorithm which samples a random subset of $β$ rows of $A$ and then greedily selects the best row of the subset. Despite their variable computational costs, all three algorithms have been proven to have the same theoretical upper bound on the convergence rate. In this work, an improved analysis of the range of random (RK) to greedy (MM) methods is presented. This analysis improves upon previous known convergence bounds for SKM, capturing the benefit of partially greedy selection schemes. This work also further generalizes previous known results, removing the theoretical assumptions that $β$ must be fixed at every iteration and that $A$ must have normalized rows.
△ Less
Submitted 24 July, 2020; v1 submitted 7 December, 2019;
originally announced December 2019.
-
Stochastic Iterative Hard Thresholding for Low-Tucker-Rank Tensor Recovery
Authors:
Rachel Grotheer,
Shuang Li,
Anna Ma,
Deanna Needell,
**g Qin
Abstract:
Low-rank tensor recovery problems have been widely studied in many applications of signal processing and machine learning. Tucker decomposition is known as one of the most popular decompositions in the tensor framework. In recent years, researchers have developed many state-of-the-art algorithms to address the problem of low-Tucker-rank tensor recovery. Motivated by the favorable properties of the…
▽ More
Low-rank tensor recovery problems have been widely studied in many applications of signal processing and machine learning. Tucker decomposition is known as one of the most popular decompositions in the tensor framework. In recent years, researchers have developed many state-of-the-art algorithms to address the problem of low-Tucker-rank tensor recovery. Motivated by the favorable properties of the stochastic algorithms, such as stochastic gradient descent and stochastic iterative hard thresholding, we aim to extend the well-known stochastic iterative hard thresholding algorithm to the tensor framework in order to address the problem of recovering a low-Tucker-rank tensor from its linear measurements. We have also developed linear convergence analysis for the proposed method and conducted a series of experiments with both synthetic and real data to illustrate the performance of the proposed method.
△ Less
Submitted 16 July, 2020; v1 submitted 22 September, 2019;
originally announced September 2019.
-
Iterative Hard Thresholding for Low CP-rank Tensor Models
Authors:
Rachel Grotheer,
Shuang Li,
Anna Ma,
Deanna Needell,
**g Qin
Abstract:
Recovery of low-rank matrices from a small number of linear measurements is now well-known to be possible under various model assumptions on the measurements. Such results demonstrate robustness and are backed with provable theoretical guarantees. However, extensions to tensor recovery have only recently began to be studied and developed, despite an abundance of practical tensor applications. Rece…
▽ More
Recovery of low-rank matrices from a small number of linear measurements is now well-known to be possible under various model assumptions on the measurements. Such results demonstrate robustness and are backed with provable theoretical guarantees. However, extensions to tensor recovery have only recently began to be studied and developed, despite an abundance of practical tensor applications. Recently, a tensor variant of the Iterative Hard Thresholding method was proposed and theoretical results were obtained that guarantee exact recovery of tensors with low Tucker rank. In this paper, we utilize the same tensor version of the Restricted Isometry Property (RIP) to extend these results for tensors with low CANDECOMP/PARAFAC (CP) rank. In doing so, we leverage recent results on efficient approximations of CP decompositions that remove the need for challenging assumptions in prior works. We complement our theoretical findings with empirical results that showcase the potential of the approach.
△ Less
Submitted 22 August, 2019;
originally announced August 2019.
-
Data-driven Algorithm Selection and Parameter Tuning: Two Case studies in Optimization and Signal Processing
Authors:
Jesus A. De Loera,
Jamie Haddock,
Anna Ma,
Deanna Needell
Abstract:
Machine learning algorithms typically rely on optimization subroutines and are well-known to provide very effective outcomes for many types of problems. Here, we flip the reliance and ask the reverse question: can machine learning algorithms lead to more effective outcomes for optimization problems? Our goal is to train machine learning methods to automatically improve the performance of optimizat…
▽ More
Machine learning algorithms typically rely on optimization subroutines and are well-known to provide very effective outcomes for many types of problems. Here, we flip the reliance and ask the reverse question: can machine learning algorithms lead to more effective outcomes for optimization problems? Our goal is to train machine learning methods to automatically improve the performance of optimization and signal processing algorithms. As a proof of concept, we use our approach to improve two popular data processing subroutines in data science: stochastic gradient descent and greedy methods in compressed sensing. We provide experimental results that demonstrate the answer is ``yes'', machine learning algorithms do lead to more effective outcomes for optimization problems, and show the future potential for this research direction.
△ Less
Submitted 26 July, 2019; v1 submitted 30 May, 2019;
originally announced May 2019.
-
Variational training of neural network approximations of solution maps for physical models
Authors:
Yingzhou Li,
Jianfeng Lu,
Anqi Mao
Abstract:
A novel solve-training framework is proposed to train neural network in representing low dimensional solution maps of physical models. Solve-training framework uses the neural network as the ansatz of the solution map and train the network variationally via loss functions from the underlying physical models. Solve-training framework avoids expensive data preparation in the traditional supervised t…
▽ More
A novel solve-training framework is proposed to train neural network in representing low dimensional solution maps of physical models. Solve-training framework uses the neural network as the ansatz of the solution map and train the network variationally via loss functions from the underlying physical models. Solve-training framework avoids expensive data preparation in the traditional supervised training procedure, which prepares labels for input data, and still achieves effective representation of the solution map adapted to the input data distribution. The efficiency of solve-training framework is demonstrated through obtaining solutions maps for linear and nonlinear elliptic equations, and maps from potentials to ground states of linear and nonlinear Schrödinger equations.
△ Less
Submitted 14 October, 2020; v1 submitted 7 May, 2019;
originally announced May 2019.
-
Compressed Anomaly Detection with Multiple Mixed Observations
Authors:
Natalie Durgin,
Rachel Grotheer,
Chenxi Huang,
Shuang Li,
Anna Ma,
Deanna Needell,
**g Qin
Abstract:
We consider a collection of independent random variables that are identically distributed, except for a small subset which follows a different, anomalous distribution. We study the problem of detecting which random variables in the collection are governed by the anomalous distribution. Recent work proposes to solve this problem by conducting hypothesis tests based on mixed observations (e.g. linea…
▽ More
We consider a collection of independent random variables that are identically distributed, except for a small subset which follows a different, anomalous distribution. We study the problem of detecting which random variables in the collection are governed by the anomalous distribution. Recent work proposes to solve this problem by conducting hypothesis tests based on mixed observations (e.g. linear combinations) of the random variables. Recognizing the connection between taking mixed observations and compressed sensing, we view the problem as recovering the "support" (index set) of the anomalous random variables from multiple measurement vectors (MMVs). Many algorithms have been developed for recovering jointly sparse signals and their support from MMVs. We establish the theoretical and empirical effectiveness of these algorithms at detecting anomalies. We also extend the LASSO algorithm to an MMV version for our purpose. Further, we perform experiments on synthetic data, consisting of samples from the random variables, to explore the trade-off between the number of mixed observations per sample and the number of samples required to detect anomalies.
△ Less
Submitted 19 June, 2018; v1 submitted 30 January, 2018;
originally announced January 2018.
-
Sparse Randomized Kaczmarz for Support Recovery of Jointly Sparse Corrupted Multiple Measurement Vectors
Authors:
Natalie Durgin,
Rachel Grotheer,
Chenxi Huang,
Shuang Li,
Anna Ma,
Deanna Needell,
**g Qin
Abstract:
While single measurement vector (SMV) models have been widely studied in signal processing, there is a surging interest in addressing the multiple measurement vectors (MMV) problem. In the MMV setting, more than one measurement vector is available and the multiple signals to be recovered share some commonalities such as a common support. Applications in which MMV is a naturally occurring phenomeno…
▽ More
While single measurement vector (SMV) models have been widely studied in signal processing, there is a surging interest in addressing the multiple measurement vectors (MMV) problem. In the MMV setting, more than one measurement vector is available and the multiple signals to be recovered share some commonalities such as a common support. Applications in which MMV is a naturally occurring phenomenon include online streaming, medical imaging, and video recovery. This work presents a stochastic iterative algorithm for the support recovery of jointly sparse corrupted MMV. We present a variant of the Sparse Randomized Kaczmarz algorithm for corrupted MMV and compare our proposed method with an existing Kaczmarz type algorithm for MMV problems. We also showcase the usefulness of our approach in the online (streaming) setting and provide empirical evidence that suggests the robustness of the proposed method to the distribution of the corruption and the number of corruptions occurring.
△ Less
Submitted 14 June, 2018; v1 submitted 7 November, 2017;
originally announced November 2017.
-
Stochastic Greedy Algorithms For Multiple Measurement Vectors
Authors:
**g Qin,
Shuang Li,
Deanna Needell,
Anna Ma,
Rachel Grotheer,
Chenxi Huang,
Natalie Durgin
Abstract:
Sparse representation of a single measurement vector (SMV) has been explored in a variety of compressive sensing applications. Recently, SMV models have been extended to solve multiple measurement vectors (MMV) problems, where the underlying signal is assumed to have joint sparse structures. To circumvent the NP-hardness of the $\ell_0$ minimization problem, many deterministic MMV algorithms solve…
▽ More
Sparse representation of a single measurement vector (SMV) has been explored in a variety of compressive sensing applications. Recently, SMV models have been extended to solve multiple measurement vectors (MMV) problems, where the underlying signal is assumed to have joint sparse structures. To circumvent the NP-hardness of the $\ell_0$ minimization problem, many deterministic MMV algorithms solve the convex relaxed models with limited efficiency. In this paper, we develop stochastic greedy algorithms for solving the joint sparse MMV reconstruction problem. In particular, we propose the MMV Stochastic Iterative Hard Thresholding (MStoIHT) and MMV Stochastic Gradient Matching Pursuit (MStoGradMP) algorithms, and we also utilize the mini-batching technique to further improve their performance. Convergence analysis indicates that the proposed algorithms are able to converge faster than their SMV counterparts, i.e., concatenated StoIHT and StoGradMP, under certain conditions. Numerical experiments have illustrated the superior effectiveness of the proposed algorithms over their SMV counterparts.
△ Less
Submitted 22 August, 2020; v1 submitted 4 November, 2017;
originally announced November 2017.
-
Existence and non-existence of transition fronts in mixed ignition-monostable media
Authors:
Cole Graham,
Tau Shean Lim,
Andrew Ma,
David Weber
Abstract:
We study transition fronts for one-dimensional reaction-diffusion equations with compactly perturbed ignition-monostable reactions. We establish an almost sharp condition on reactions which characterizes the existence and non-existence of fronts. In particular, we prove that a strong inhomogeneity in the reaction prevents formation of transition fronts, while a weak inhomogeneity gives rise to a f…
▽ More
We study transition fronts for one-dimensional reaction-diffusion equations with compactly perturbed ignition-monostable reactions. We establish an almost sharp condition on reactions which characterizes the existence and non-existence of fronts. In particular, we prove that a strong inhomogeneity in the reaction prevents formation of transition fronts, while a weak inhomogeneity gives rise to a front. Our work extends results and methods introduced by J. Nolen, J.M. Roquejoffre, L. Ryzhik, and A. Zlatoš.
△ Less
Submitted 9 May, 2017;
originally announced May 2017.
-
Stochastic Gradient Descent for Linear Systems with Missing Data
Authors:
Anna Ma,
Deanna Needell
Abstract:
Traditional methods for solving linear systems have quickly become impractical due to an increase in the size of available data. Utilizing massive amounts of data is further complicated when the data is incomplete or has missing entries. In this work, we address the obstacles presented when working with large data and incomplete data simultaneously. In particular, we propose to adapt the Stochasti…
▽ More
Traditional methods for solving linear systems have quickly become impractical due to an increase in the size of available data. Utilizing massive amounts of data is further complicated when the data is incomplete or has missing entries. In this work, we address the obstacles presented when working with large data and incomplete data simultaneously. In particular, we propose to adapt the Stochastic Gradient Descent method to address missing data in linear systems. Our proposed algorithm, the Stochastic Gradient Descent for Missing Data method (mSGD), is introduced and theoretical convergence guarantees are provided. In addition, we include numerical experiments on simulated and real world data that demonstrate the usefulness of our method.
△ Less
Submitted 7 January, 2019; v1 submitted 23 February, 2017;
originally announced February 2017.
-
Iterative methods for solving factorized linear systems
Authors:
Anna Ma,
Deanna Needell,
Aaditya Ramdas
Abstract:
Stochastic iterative algorithms such as the Kaczmarz and Gauss-Seidel methods have gained recent attention because of their speed, simplicity, and the ability to approximately solve large-scale linear systems of equations without needing to access the entire matrix. In this work, we consider the setting where we wish to solve a linear system in a large matrix X that is stored in a factorized form,…
▽ More
Stochastic iterative algorithms such as the Kaczmarz and Gauss-Seidel methods have gained recent attention because of their speed, simplicity, and the ability to approximately solve large-scale linear systems of equations without needing to access the entire matrix. In this work, we consider the setting where we wish to solve a linear system in a large matrix X that is stored in a factorized form, X = UV; this setting either arises naturally in many applications or may be imposed when working with large low-rank datasets for reasons of space required for storage. We propose a variant of the randomized Kaczmarz method for such systems that takes advantage of the factored form, and avoids computing X. We prove an exponential convergence rate and supplement our theoretical guarantees with experimental evidence demonstrating that the factored variant yields significant acceleration in convergence.
△ Less
Submitted 9 January, 2019; v1 submitted 25 January, 2017;
originally announced January 2017.
-
A Kochen-Specker theorem for integer matrices and noncommutative spectrum functors
Authors:
Michael Ben-Zvi,
Alexander Ma,
Manuel Reyes
Abstract:
We investigate the possibility of constructing Kochen-Specker uncolorable sets of idempotent matrices whose entries lie in various rings, including the rational numbers, the integers, and finite fields. Most notably, we show that there is no Kochen-Specker coloring of the $n \times n$ idempotent integer matrices for $n \geq 3$, thereby illustrating that Kochen-Specker contextuality is an inherent…
▽ More
We investigate the possibility of constructing Kochen-Specker uncolorable sets of idempotent matrices whose entries lie in various rings, including the rational numbers, the integers, and finite fields. Most notably, we show that there is no Kochen-Specker coloring of the $n \times n$ idempotent integer matrices for $n \geq 3$, thereby illustrating that Kochen-Specker contextuality is an inherent feature of pure matrix algebra. We apply this to generalize recent no-go results on noncommutative spectrum functors, showing that any contravariant functor from rings to sets (respectively, topological spaces or locales) that restricts to the Zariski prime spectrum functor for commutative rings must assign the empty set (respectively, empty space or locale) to the matrix ring $M_n(R)$ for any integer $n \geq 3$ and any ring $R$. An appendix by Alexandru Chirvasitu shows that Kochen-Specker colorings of idempotents in partial subalgebras of $M_3(F)$ for a perfect field $F$ can be extended to partial algebra morphisms into the algebraic closure of $F$.
△ Less
Submitted 11 August, 2017; v1 submitted 11 September, 2015;
originally announced September 2015.
-
Convergence properties of the randomized extended Gauss-Seidel and Kaczmarz methods
Authors:
Anna Ma,
Deanna Needell,
Aaditya Ramdas
Abstract:
The Kaczmarz and Gauss-Seidel methods both solve a linear system $\bf{X}\bfβ = \bf{y}$ by iteratively refining the solution estimate. Recent interest in these methods has been sparked by a proof of Strohmer and Vershynin which shows the randomized Kaczmarz method converges linearly in expectation to the solution. Lewis and Leventhal then proved a similar result for the randomized Gauss-Seidel algo…
▽ More
The Kaczmarz and Gauss-Seidel methods both solve a linear system $\bf{X}\bfβ = \bf{y}$ by iteratively refining the solution estimate. Recent interest in these methods has been sparked by a proof of Strohmer and Vershynin which shows the randomized Kaczmarz method converges linearly in expectation to the solution. Lewis and Leventhal then proved a similar result for the randomized Gauss-Seidel algorithm. However, the behavior of both methods depends heavily on whether the system is under or overdetermined, and whether it is consistent or not. Here we provide a unified theory of both methods, their variants for these different settings, and draw connections between both approaches. In doing so, we also provide a proof that an extended version of randomized Gauss-Seidel converges linearly to the least norm solution in the underdetermined case (where the usual randomized Gauss Seidel fails to converge). We detail analytically and empirically the convergence properties of both methods and their extended variants in all possible system settings. With this result, a complete and rigorous theory of both methods is furnished.
△ Less
Submitted 1 February, 2018; v1 submitted 27 March, 2015;
originally announced March 2015.
