-
Adaptive Ridge Approach to Heteroscedastic Regression
Authors:
Ka Long Keith Ho,
Hiroki Masuda
Abstract:
We propose an adaptive ridge (AR) based estimation scheme for a heteroscedastic linear model equipped with log-linear errors. We simultaneously estimate the mean and variance parameters and show new asymptotic distributional and tightness properties in a sparse setting. We also show that estimates for zero parameters shrink with more iterations under suitable assumptions for tuning parameters. We…
▽ More
We propose an adaptive ridge (AR) based estimation scheme for a heteroscedastic linear model equipped with log-linear errors. We simultaneously estimate the mean and variance parameters and show new asymptotic distributional and tightness properties in a sparse setting. We also show that estimates for zero parameters shrink with more iterations under suitable assumptions for tuning parameters. We observe possible generalizations of this paper's results through simulations and will apply the estimation method in forecasting electricity consumption.
△ Less
Submitted 4 May, 2024; v1 submitted 21 February, 2024;
originally announced February 2024.
-
Multidimensional Phase Recovery and Interpolative Decomposition Butterfly Factorization
Authors:
Ze Chen,
Juan Zhang,
Kenneth L. Ho,
Haizhao Yang
Abstract:
This paper focuses on the fast evaluation of the matvec $g=Kf$ for $K\in \mathbb{C}^{N\times N}$, which is the discretization of a multidimensional oscillatory integral transform $g(x) = \int K(x,ξ) f(ξ)dξ$ with a kernel function $K(x,ξ)=e^{2πıΦ(x,ξ)}$, where $Φ(x,ξ)$ is a piecewise smooth phase function with $x$ and $ξ$ in $\mathbb{R}^d$ for $d=2$ or $3$. A new framework is introduced to compute…
▽ More
This paper focuses on the fast evaluation of the matvec $g=Kf$ for $K\in \mathbb{C}^{N\times N}$, which is the discretization of a multidimensional oscillatory integral transform $g(x) = \int K(x,ξ) f(ξ)dξ$ with a kernel function $K(x,ξ)=e^{2πıΦ(x,ξ)}$, where $Φ(x,ξ)$ is a piecewise smooth phase function with $x$ and $ξ$ in $\mathbb{R}^d$ for $d=2$ or $3$. A new framework is introduced to compute $Kf$ with $O(N\log N)$ time and memory complexity in the case that only indirect access to the phase function $Φ$ is available. This framework consists of two main steps: 1) an $O(N\log N)$ algorithm for recovering the multidimensional phase function $Φ$ from indirect access is proposed; 2) a multidimensional interpolative decomposition butterfly factorization (MIDBF) is designed to evaluate the matvec $Kf$ with an $O(N\log N)$ complexity once $Φ$ is available. Numerical results are provided to demonstrate the effectiveness of the proposed framework.
△ Less
Submitted 23 March, 2020; v1 submitted 25 August, 2019;
originally announced August 2019.
-
Interpolative Decomposition Butterfly Factorization
Authors:
Qiyuan Pang,
Kenneth L. Ho,
Haizhao Yang
Abstract:
This paper introduces a "kernel-independent" interpolative decomposition butterfly factorization (IDBF) as a data-sparse approximation for matrices that satisfy a complementary low-rank property. The IDBF can be constructed in $O(N\log N)$ operations for an $N\times N$ matrix via hierarchical interpolative decompositions (IDs), if matrix entries can be sampled individually and each sample takes…
▽ More
This paper introduces a "kernel-independent" interpolative decomposition butterfly factorization (IDBF) as a data-sparse approximation for matrices that satisfy a complementary low-rank property. The IDBF can be constructed in $O(N\log N)$ operations for an $N\times N$ matrix via hierarchical interpolative decompositions (IDs), if matrix entries can be sampled individually and each sample takes $O(1)$ operations. The resulting factorization is a product of $O(\log N)$ sparse matrices, each with $O(N)$ non-zero entries. Hence, it can be applied to a vector rapidly in $O(N\log N)$ operations. IDBF is a general framework for nearly optimal fast matvec useful in a wide range of applications, e.g., special function transformation, Fourier integral operators, high-frequency wave computation. Numerical results are provided to demonstrate the effectiveness of the butterfly factorization and its construction algorithms.
△ Less
Submitted 7 October, 2018; v1 submitted 27 September, 2018;
originally announced September 2018.
-
Recursively Preconditioned Hierarchical Interpolative Factorization for Elliptic Partial Differential Equations
Authors:
Jordi Feliu-Fabà,
Kenneth L. Ho,
Lexing Ying
Abstract:
The hierarchical interpolative factorization for elliptic partial differential equations is a fast algorithm for approximate sparse matrix inversion in linear or quasilinear time. Its accuracy can degrade, however, when applied to strongly ill-conditioned problems. Here, we propose a simple modification that can significantly improve the accuracy at no additional asymptotic cost: applying a block…
▽ More
The hierarchical interpolative factorization for elliptic partial differential equations is a fast algorithm for approximate sparse matrix inversion in linear or quasilinear time. Its accuracy can degrade, however, when applied to strongly ill-conditioned problems. Here, we propose a simple modification that can significantly improve the accuracy at no additional asymptotic cost: applying a block Jacobi preconditioner before each level of skeletonization. This dramatically limits the impact of the underlying system conditioning and enables the construction of robust and highly efficient preconditioners even at quite modest compression tolerances. Numerical examples demonstrate the performance of the new approach.
△ Less
Submitted 8 April, 2019; v1 submitted 3 August, 2018;
originally announced August 2018.
-
A recursive skeletonization factorization based on strong admissibility
Authors:
Victor Minden,
Kenneth L. Ho,
Anil Damle,
Lexing Ying
Abstract:
We introduce the strong recursive skeletonization factorization (RS-S), a new approximate matrix factorization based on recursive skeletonization for solving discretizations of linear integral equations associated with elliptic partial differential equations in two and three dimensions (and other matrices with similar hierarchical rank structure). Unlike previous skeletonization-based factorizatio…
▽ More
We introduce the strong recursive skeletonization factorization (RS-S), a new approximate matrix factorization based on recursive skeletonization for solving discretizations of linear integral equations associated with elliptic partial differential equations in two and three dimensions (and other matrices with similar hierarchical rank structure). Unlike previous skeletonization-based factorizations, RS-S uses a simple modification of skeletonization, strong skeletonization, which compresses only far-field interactions. This leads to an approximate factorization in the form of a product of many block unit-triangular matrices that may be used as a preconditioner or moderate-accuracy direct solver, with dramatically reduced rank growth. We further combine the strong skeletonization procedure with alternating near-field compression to obtain the hybrid recursive skeletonization factorization (RS-WS), a modification of RS-S that exhibits reduced storage cost in many settings. Under suitable rank assumptions both RS-S and RS-WS exhibit linear computational complexity, which we demonstrate with a number of numerical examples.
△ Less
Submitted 6 January, 2017; v1 submitted 26 September, 2016;
originally announced September 2016.
-
Differential Algebra for Model Comparison
Authors:
Heather A. Harrington,
Kenneth L. Ho,
Nicolette Meshkat
Abstract:
We present a method for rejecting competing models from noisy time-course data that does not rely on parameter inference. First we characterize ordinary differential equation models in only measurable variables using differential algebra elimination. Next we extract additional information from the given data using Gaussian Process Regression (GPR) and then transform the differential invariants. We…
▽ More
We present a method for rejecting competing models from noisy time-course data that does not rely on parameter inference. First we characterize ordinary differential equation models in only measurable variables using differential algebra elimination. Next we extract additional information from the given data using Gaussian Process Regression (GPR) and then transform the differential invariants. We develop a test using linear algebra and statistics to reject transformed models with the given data in a parameter-free manner. This algorithm exploits the information about transients that is encoded in the model's structure. We demonstrate the power of this approach by discriminating between different models from mathematical biology.
△ Less
Submitted 31 March, 2016;
originally announced March 2016.
-
Fast spatial Gaussian process maximum likelihood estimation via skeletonization factorizations
Authors:
Victor Minden,
Anil Damle,
Kenneth L. Ho,
Lexing Ying
Abstract:
Maximum likelihood estimation for parameter-fitting given observations from a Gaussian process in space is a computationally-demanding task that restricts the use of such methods to moderately-sized datasets. We present a framework for unstructured observations in two spatial dimensions that allows for evaluation of the log-likelihood and its gradient (i.e., the score equations) in…
▽ More
Maximum likelihood estimation for parameter-fitting given observations from a Gaussian process in space is a computationally-demanding task that restricts the use of such methods to moderately-sized datasets. We present a framework for unstructured observations in two spatial dimensions that allows for evaluation of the log-likelihood and its gradient (i.e., the score equations) in $\tilde O(n^{3/2})$ time under certain assumptions, where $n$ is the number of observations. Our method relies on the skeletonization procedure described by Martinsson & Rokhlin in the form of the recursive skeletonization factorization of Ho & Ying. Combining this with an adaptation of the matrix peeling algorithm of Lin et al. for constructing $\mathcal{H}$-matrix representations of black-box operators, we obtain a framework that can be used in the context of any first-order optimization routine to quickly and accurately compute maximum-likelihood estimates.
△ Less
Submitted 7 September, 2017; v1 submitted 25 March, 2016;
originally announced March 2016.
-
Numerical algebraic geometry for model selection and its application to the life sciences
Authors:
Elizabeth Gross,
Brent Davis,
Kenneth L. Ho,
Daniel J. Bates,
Heather A. Harrington
Abstract:
Researchers working with mathematical models are often confronted by the related problems of parameter estimation, model validation, and model selection. These are all optimization problems, well-known to be challenging due to non-linearity, non-convexity and multiple local optima. Furthermore, the challenges are compounded when only partial data is available. Here, we consider polynomial models (…
▽ More
Researchers working with mathematical models are often confronted by the related problems of parameter estimation, model validation, and model selection. These are all optimization problems, well-known to be challenging due to non-linearity, non-convexity and multiple local optima. Furthermore, the challenges are compounded when only partial data is available. Here, we consider polynomial models (e.g., mass-action chemical reaction networks at steady state) and describe a framework for their analysis based on optimization using numerical algebraic geometry. Specifically, we use probability-one polynomial homotopy continuation methods to compute all critical points of the objective function, then filter to recover the global optima. Our approach exploits the geometric structures relating models and data, and we demonstrate its utility on examples from cell signaling, synthetic biology, and epidemiology.
△ Less
Submitted 1 April, 2016; v1 submitted 15 July, 2015;
originally announced July 2015.
-
A technique for updating hierarchical skeletonization-based factorizations of integral operators
Authors:
Victor Minden,
Anil Damle,
Kenneth L. Ho,
Lexing Ying
Abstract:
We present a method for updating certain hierarchical factorizations for solving linear integral equations with elliptic kernels. In particular, given a factorization corresponding to some initial geometry or material parameters, we can locally perturb the geometry or coefficients and update the initial factorization to reflect this change with asymptotic complexity that is polylogarithmic in the…
▽ More
We present a method for updating certain hierarchical factorizations for solving linear integral equations with elliptic kernels. In particular, given a factorization corresponding to some initial geometry or material parameters, we can locally perturb the geometry or coefficients and update the initial factorization to reflect this change with asymptotic complexity that is polylogarithmic in the total number of unknowns and linear in the number of perturbed unknowns. We apply our method to the recursive skeletonization factorization and hierarchical interpolative factorization and demonstrate scaling results for a number of different 2D problem setups.
△ Less
Submitted 2 November, 2015; v1 submitted 20 November, 2014;
originally announced November 2014.
-
A fast direct solver for scattering from periodic structures with multiple material interfaces in two dimensions
Authors:
Leslie Greengard,
Kenneth L. Ho,
June-Yub Lee
Abstract:
We present a new integral equation method for the calculation of two-dimensional scattering from periodic structures involving triple-points (multiple materials meeting at a single point). The combination of a robust and high-order accurate integral representation and a fast direct solver permits the efficient simulation of scattering from fixed structures at multiple angles of incidence. We demon…
▽ More
We present a new integral equation method for the calculation of two-dimensional scattering from periodic structures involving triple-points (multiple materials meeting at a single point). The combination of a robust and high-order accurate integral representation and a fast direct solver permits the efficient simulation of scattering from fixed structures at multiple angles of incidence. We demonstrate the performance of the scheme with several numerical examples.
△ Less
Submitted 29 July, 2013;
originally announced July 2013.
-
Hierarchical interpolative factorization for elliptic operators: differential equations
Authors:
Kenneth L. Ho,
Lexing Ying
Abstract:
This paper introduces the hierarchical interpolative factorization for elliptic partial differential equations (HIF-DE) in two (2D) and three dimensions (3D). This factorization takes the form of an approximate generalized LU/LDL decomposition that facilitates the efficient inversion of the discretized operator. HIF-DE is based on the multifrontal method but uses skeletonization on the separator f…
▽ More
This paper introduces the hierarchical interpolative factorization for elliptic partial differential equations (HIF-DE) in two (2D) and three dimensions (3D). This factorization takes the form of an approximate generalized LU/LDL decomposition that facilitates the efficient inversion of the discretized operator. HIF-DE is based on the multifrontal method but uses skeletonization on the separator fronts to sparsify the dense frontal matrices and thus reduce the cost. We conjecture that this strategy yields linear complexity in 2D and quasilinear complexity in 3D. Estimated linear complexity in 3D can be achieved by skeletonizing the compressed fronts themselves, which amounts geometrically to a recursive dimensional reduction scheme. Numerical experiments support our claims and further demonstrate the performance of our algorithm as a fast direct solver and preconditioner. MATLAB codes are freely available.
△ Less
Submitted 20 April, 2015; v1 submitted 10 July, 2013;
originally announced July 2013.
-
Hierarchical interpolative factorization for elliptic operators: integral equations
Authors:
Kenneth L. Ho,
Lexing Ying
Abstract:
This paper introduces the hierarchical interpolative factorization for integral equations (HIF-IE) associated with elliptic problems in two and three dimensions. This factorization takes the form of an approximate generalized LU decomposition that permits the efficient application of the discretized operator and its inverse. HIF-IE is based on the recursive skeletonization algorithm but incorporat…
▽ More
This paper introduces the hierarchical interpolative factorization for integral equations (HIF-IE) associated with elliptic problems in two and three dimensions. This factorization takes the form of an approximate generalized LU decomposition that permits the efficient application of the discretized operator and its inverse. HIF-IE is based on the recursive skeletonization algorithm but incorporates a novel combination of two key features: (1) a matrix factorization framework for sparsifying structured dense matrices and (2) a recursive dimensional reduction strategy to decrease the cost. Thus, higher-dimensional problems are effectively mapped to one dimension, and we conjecture that constructing, applying, and inverting the factorization all have linear or quasilinear complexity. Numerical experiments support this claim and further demonstrate the performance of our algorithm as a generalized fast multipole method, direct solver, and preconditioner. HIF-IE is compatible with geometric adaptivity and can handle both boundary and volume problems. MATLAB codes are freely available.
△ Less
Submitted 20 April, 2015; v1 submitted 10 July, 2013;
originally announced July 2013.
-
A fast semi-direct least squares algorithm for hierarchically block separable matrices
Authors:
Kenneth L. Ho,
Leslie Greengard
Abstract:
We present a fast algorithm for linear least squares problems governed by hierarchically block separable (HBS) matrices. Such matrices are generally dense but data-sparse and can describe many important operators including those derived from asymptotically smooth radial kernels that are not too oscillatory. The algorithm is based on a recursive skeletonization procedure that exposes this sparsity…
▽ More
We present a fast algorithm for linear least squares problems governed by hierarchically block separable (HBS) matrices. Such matrices are generally dense but data-sparse and can describe many important operators including those derived from asymptotically smooth radial kernels that are not too oscillatory. The algorithm is based on a recursive skeletonization procedure that exposes this sparsity and solves the dense least squares problem as a larger, equality-constrained, sparse one. It relies on a sparse QR factorization coupled with iterative weighted least squares methods. In essence, our scheme consists of a direct component, comprised of matrix compression and factorization, followed by an iterative component to enforce certain equality constraints. At most two iterations are typically required for problems that are not too ill-conditioned. For an $M \times N$ HBS matrix with $M \geq N$ having bounded off-diagonal block rank, the algorithm has optimal $\mathcal{O} (M + N)$ complexity. If the rank increases with the spatial dimension as is common for operators that are singular at the origin, then this becomes $\mathcal{O} (M + N)$ in 1D, $\mathcal{O} (M + N^{3/2})$ in 2D, and $\mathcal{O} (M + N^{2})$ in 3D. We illustrate the performance of the method on both over- and underdetermined systems in a variety of settings, with an emphasis on radial basis function approximation and efficient updating and downdating.
△ Less
Submitted 9 April, 2014; v1 submitted 14 December, 2012;
originally announced December 2012.
-
A fast direct solver for structured linear systems by recursive skeletonization
Authors:
Kenneth L. Ho,
Leslie Greengard
Abstract:
We present a fast direct solver for structured linear systems based on multilevel matrix compression. Using the recently developed interpolative decomposition of a low-rank matrix in a recursive manner, we embed an approximation of the original matrix into a larger, but highly structured sparse one that allows fast factorization and application of the inverse. The algorithm extends the Martinsson/…
▽ More
We present a fast direct solver for structured linear systems based on multilevel matrix compression. Using the recently developed interpolative decomposition of a low-rank matrix in a recursive manner, we embed an approximation of the original matrix into a larger, but highly structured sparse one that allows fast factorization and application of the inverse. The algorithm extends the Martinsson/Rokhlin method developed for 2D boundary integral equations and proceeds in two phases: a precomputation phase, consisting of matrix compression and factorization, followed by a solution phase to apply the matrix inverse. For boundary integral equations which are not too oscillatory, e.g., based on the Green's functions for the Laplace or low-frequency Helmholtz equations, both phases typically have complexity O(N) in two dimensions, where $N$ is the number of discretization points. In our current implementation, the corresponding costs in three dimensions are $O(N^{3/2})$ and $O(N \log N)$ for precomputation and solution, respectively. Extensive numerical experiments show a speedup of $\sim 100$ for the solution phase over modern fast multipole methods; however, the cost of precomputation remains high. Thus, the solver is particularly suited to problems where large numbers of iterations would be required. Such is the case with ill-conditioned linear systems or when the same system is to be solved with multiple right-hand sides. Our algorithm is implemented in Fortran and freely available.
△ Less
Submitted 19 July, 2012; v1 submitted 13 October, 2011;
originally announced October 2011.
-
A parameter-free model discrimination criterion based on steady-state coplanarity
Authors:
Heather A. Harrington,
Kenneth L. Ho,
Thomas Thorne,
Michael P. H. Stumpf
Abstract:
We describe a novel procedure for deciding when a mass-action model is incompatible with observed steady-state data that does not require any parameter estimation. Thus, we avoid the difficulties of nonlinear optimization typically associated with methods based on parameter fitting. The key idea is to use the model equations to construct a transformation of the original variables such that any set…
▽ More
We describe a novel procedure for deciding when a mass-action model is incompatible with observed steady-state data that does not require any parameter estimation. Thus, we avoid the difficulties of nonlinear optimization typically associated with methods based on parameter fitting. The key idea is to use the model equations to construct a transformation of the original variables such that any set of steady states of the model under that transformation lies on a common plane, irrespective of the values of the model parameters. Model rejection can then be performed by assessing the degree to which the transformed data deviate from coplanarity. We demonstrate our method by applying it to models of multisite phosphorylation and cell death signaling. Although somewhat limited at present, our work provides an important first step towards a parameter-free framework for data-driven model selection.
△ Less
Submitted 16 August, 2012; v1 submitted 16 September, 2011;
originally announced September 2011.
