-
Generalized Choi-Davis-Jensen's Operator Inequalities and Their Applications
Authors:
Shih Yu Chang,
Yimin Wei
Abstract:
The original Choi-Davis-Jensen's inequality, with its wide-ranging applications in diverse scientific and engineering fields, has motivated researchers to explore generalizations. In this study, we extend Davis-Choi-Jensen's inequality by considering a nonlinear map instead of a normalized linear map and generalize operator convex function to any continuous function defined in a compact region. Th…
▽ More
The original Choi-Davis-Jensen's inequality, with its wide-ranging applications in diverse scientific and engineering fields, has motivated researchers to explore generalizations. In this study, we extend Davis-Choi-Jensen's inequality by considering a nonlinear map instead of a normalized linear map and generalize operator convex function to any continuous function defined in a compact region. The Stone-Weierstrass theorem and Kantorovich function are instrumental in formulating and proving generalized Choi-Davis-Jensen's inequalities. Additionally, we present an application of this generalized inequality in the context of statistical physics.
△ Less
Submitted 7 March, 2024;
originally announced March 2024.
-
Algebraic Riccati Tensor Equations with Applications in Multilinear Control Systems
Authors:
Yuchao Wang,
Yimin Wei,
Guofeng Zhang,
Shih Yu Chang
Abstract:
In a recent interesting paper [8], Chen et al. initialized the control-theoretic study of a class of discrete-time multilinear time-invariant (MLTI) control systems, where system states, inputs and outputs are all tensors endowed with the Einstein product. Criteria for fundamental system-theoretic notions such as stability, reachability and observability are established by means of tensor decompos…
▽ More
In a recent interesting paper [8], Chen et al. initialized the control-theoretic study of a class of discrete-time multilinear time-invariant (MLTI) control systems, where system states, inputs and outputs are all tensors endowed with the Einstein product. Criteria for fundamental system-theoretic notions such as stability, reachability and observability are established by means of tensor decomposition. The purpose of this paper is to continue this novel research direction. Specifically, we focus on continuous-time MLTI control systems. We define Hamiltonian tensors and symplectic tensors and establish the Schur-Hamiltonian tensor decomposition and symplectic tensor singular value decomposition (SVD). Based on these we propose the algebraic Riccati tensor equation (ARTE) and show that it has a unique positive semidefinite solution if the system is stablizable and detectable. A tensor-based Newton method is proposed to find numerical solutions of the ARTE. The tensor version of the bounded real lemma is also established. A first-order robustness analysis of the ARTE is conducted. Finally, a numerical example is used to demonstrate the proposed theory and algorithms.
△ Less
Submitted 20 February, 2024;
originally announced February 2024.
-
Random Parametrization Double Tensors Integrals and Their Applications
Authors:
Shih Yu Chang
Abstract:
In this work, we extend double tensor integrals (DTI) from our previous work to parametrization double tensors integrals (PDTI) by applying integral kernel transform bounds to upper bound PDTI norm and establishing a new perturbation formula. Besides, the convergence property of random PDTI is investigated and this property is utilized to characterize the relation between the original derivative t…
▽ More
In this work, we extend double tensor integrals (DTI) from our previous work to parametrization double tensors integrals (PDTI) by applying integral kernel transform bounds to upper bound PDTI norm and establishing a new perturbation formula. Besides, the convergence property of random PDTI is investigated and this property is utilized to characterize the relation between the original derivative tensor and the action result of PDTI to the original derivative tensor. These tools help us to derive new tail bounds for random tensors according to more general operator inequalities, e.g., Heinz inequality and Birman-Koplienko-Solomyak inequality. Moreover, new tail bounds about random tensors are also obtained according to our new derived perturbation formula and integral kernel transform bounds.
△ Less
Submitted 6 May, 2022;
originally announced May 2022.
-
Random Double Tensors Integrals
Authors:
Shih Yu Chang
Abstract:
In this work, we try to build a theory for random double tensor integrals (DTI). We begin with the definition of DTI and discuss how randomness structure is built upon DTI. Then, the tail bound of the unitarily invariant norm for the random DTI is established and this bound can help us to derive tail bounds of the unitarily invariant norm for various types of two tensors means, e.g., arithmetic me…
▽ More
In this work, we try to build a theory for random double tensor integrals (DTI). We begin with the definition of DTI and discuss how randomness structure is built upon DTI. Then, the tail bound of the unitarily invariant norm for the random DTI is established and this bound can help us to derive tail bounds of the unitarily invariant norm for various types of two tensors means, e.g., arithmetic mean, geometric mean, harmonic mean, and general mean. By associating DTI with perturbation formula, i.e., a formula to relate the tensor-valued function difference with respect the difference of the function input tensors, the tail bounds of the unitarily invariant norm for the Lipschitz estimate of tensor-valued function with random tensors as arguments are derived for vanilla case and quasi-commutator case, respectively. We also establish the continuity property for random DTI in the sense of convergence in the random tensor mean, and we apply this continuity property to obtain the tail bound of the unitarily invariant norm for the derivative of the tensor-valued function.
△ Less
Submitted 4 April, 2022;
originally announced April 2022.
-
Generalized Hanson-Wright Inequality for Random Tensors
Authors:
Shih Yu Chang
Abstract:
The Hanson-Wright inequality is an upper bound for tails of real quadratic forms in independent random variables. In this work, we extend the Hanson-Wright inequality for the Ky Fan k-norm for the polynomial function of the quadratic sum of random tensors under Einstein product. We decompose the quadratic tensors sum into the diagonal part and the coupling part. For the diagonal part, we can apply…
▽ More
The Hanson-Wright inequality is an upper bound for tails of real quadratic forms in independent random variables. In this work, we extend the Hanson-Wright inequality for the Ky Fan k-norm for the polynomial function of the quadratic sum of random tensors under Einstein product. We decompose the quadratic tensors sum into the diagonal part and the coupling part. For the diagonal part, we can apply the generalized tensor Chernoff bound directly. But, for the coupling part, we have to apply decoupling method first, i.e., decoupling inequality to bound expressions with dependent random tensors with independent random tensors, before applying generalized tensor Chernoff bound again to get the the tail probability of the Ky Fan $k$-norm of the coupling part sum of independent random tensors. At the end, the generalized Hanson-Wright inequality for the Ky Fan k-norm for the polynomial function of the quadratic sum of random tensors can be obtained by the combination of the bound from the diagonal sum part and the bound from the coupling sum part.
△ Less
Submitted 1 March, 2022;
originally announced March 2022.
-
Hanson-Wright Inequality for Random Tensors under Einstein Product
Authors:
Shih Yu Chang
Abstract:
The Hanson-Wright inequality is an upper bound for tails of real quadratic forms in independent subgaussian random variables. In this work, we extend the Hanson-Wright inequality for the maximum eigenvalue of the quadratic sum of random Hermitian tensors under Einstein product. We first prove Weyl inequality for tensors under Einstein product and apply this fact to separate the quadratic form of r…
▽ More
The Hanson-Wright inequality is an upper bound for tails of real quadratic forms in independent subgaussian random variables. In this work, we extend the Hanson-Wright inequality for the maximum eigenvalue of the quadratic sum of random Hermitian tensors under Einstein product. We first prove Weyl inequality for tensors under Einstein product and apply this fact to separate the quadratic form of random Hermitian tensors into diagonal sum and coupling (non-diagonal) sum parts. For the diagonal part, we can apply Bernstein inequality to bound the tail probability of the maximum eigenvalue of the sum of independent random Hermitian tensors directly. For coupling sum part, we have to apply decoupling method first, i.e., decoupling inequality to bound expressions with dependent random Hermitian tensors with independent random Hermitian tensors, before applying Bernstein inequality again to bound the tail probability of the maximum eigenvalue of the coupling sum of independent random Hermitian tensors. Finally, the Hanson-Wright inequality for the maximum eigenvalue of the quadratic sum of random Hermitian tensors under Einstein product can be obtained by the combination of the bound from the diagonal sum part and the bound from the coupling (non-diagonal) sum part. In Appendix of this work, we also include the Hanson-Wright inequality under T-product tensor, which can be derived by the same method of establishing the Hanson-Wright inequality under Einstein product except changing the rule of tensors product operation.
△ Less
Submitted 1 March, 2022; v1 submitted 23 November, 2021;
originally announced November 2021.
-
T-product Tensor Expander Chernoff Bound
Authors:
Shih Yu Chang
Abstract:
In probability theory, the Chernoff bound gives exponentially decreasing bounds on tail distributions for sums of independent random variables and such bound is applied at different fields in science and engineering. In this work, we generalize the conventional Chernoff bound from the summation of independent random variables to the summation of dependent random T-product tensors. Our main tool us…
▽ More
In probability theory, the Chernoff bound gives exponentially decreasing bounds on tail distributions for sums of independent random variables and such bound is applied at different fields in science and engineering. In this work, we generalize the conventional Chernoff bound from the summation of independent random variables to the summation of dependent random T-product tensors. Our main tool used at this work is majorization technique. We first apply majorizaton method to establish norm inequalitites for T-product tensors and these norm inequalities are used to derive T-product tensor expander Chernoff bound. Compared with the matrix expander Chernoff bound obtained by Garg et al., the T-product tensor expander Chernoff bound proved at this work contributes following aspects: (1) the random objects dimensions are increased from matrices (two-dimensional data array) to T-product tensors (three-dimensional data array); (2) this bound generalizes the identity map of the random objects summation to any polynomial function of the random objects summation; (3) Ky Fan norm, instead only the maximum or the minimum eigenvalues, for the function of the random T-product tensors summation is considered; (4) we remove the restriction about the summation of all mapped random objects is zero, which is required in the matrix expander Chernoff bound derivation.
△ Less
Submitted 28 September, 2021;
originally announced September 2021.
-
Generalized T-product Tensor Bernstein Bounds
Authors:
Shih Yu Chang,
Yimin Wei
Abstract:
Since Kilmer et al. introduced the new multiplication method between two third-order tensors around 2008 and third-order tensors with such multiplication structure are also called as T-product tensors, T-product tensors have been applied to many fields in science and engineering, such as low-rank tensor approximation, signal processing, image feature extraction, machine learning, computer vision,…
▽ More
Since Kilmer et al. introduced the new multiplication method between two third-order tensors around 2008 and third-order tensors with such multiplication structure are also called as T-product tensors, T-product tensors have been applied to many fields in science and engineering, such as low-rank tensor approximation, signal processing, image feature extraction, machine learning, computer vision, and the multi-view clustering problem, etc. However, there are very few works dedicated to exploring the behavior of random T-product tensors. This work considers the problem about the tail behavior of the unitarily invariant norm for the summation of random symmetric T-product tensors. Majorization and antisymmetric Kronecker product tools are main techniques utilized to establish inequalities for unitarily norms of multivariate T-product tensors. The Laplace transform method is integrated with these inequalities for unitarily norms of multivariate T-product tensors to provide us with Bernstein Bounds estimation of Ky Fan $k$-norm for functions of the symmetric random T-product tensors summation.
△ Less
Submitted 5 October, 2021; v1 submitted 22 September, 2021;
originally announced September 2021.
-
T product Tensors Part I: Inequalities
Authors:
Shih Yu Chang,
Yimin Wei
Abstract:
The T product operation between two three order tensors was invented around 2011 and it arises from many applications, such as signal processing, image feature extraction, machine learning, computer vision, and the multiview clustering problem. Although there are many pioneer works about T product tensors, there are no works dedicated to inequalities associated with T product tensors. In this work…
▽ More
The T product operation between two three order tensors was invented around 2011 and it arises from many applications, such as signal processing, image feature extraction, machine learning, computer vision, and the multiview clustering problem. Although there are many pioneer works about T product tensors, there are no works dedicated to inequalities associated with T product tensors. In this work, we first attempt to build inequalities at the following aspects: (1) trace function nondecreasing and convexity; (2) Golden Thompson inequality for T product tensors; (3) Jensen T product inequality; (4) Klein T product inequality. All these inequalities are related to generalize celebrated Lieb concavity theorem from matrices to T product tensors. This new version of Lieb concavity theorem under T product tensor will be used to determine the tail bound for the maximum eigenvalue induced by independent sums of random Hermitian T product, which is the key tool to derive various new tail bounds for random T product tensors. Besides, Qi et. al introduces a new concept, named eigentuple, about T product tensors and they apply this concept to study nonnegative (positive) definite properties of T product tensors. The final main contribution of this work is to develop the Courant Fischer Theorem with respect to eigentuples, and this theorem helps us to understand the relationship between the minimum eigentuple and the maximum eigentuple. The main content of this paper is Part I of a serious task about T product tensors. The Part II of this work will utilize these new inequalities and Courant Fischer Theorem under T product tensors to derive tail bounds of the extreme eigenvalue and the maximum eigentuple for sums of random T product tensors, e.g., T product tensor Chernoff and T product tensor Bernstein bounds.
△ Less
Submitted 10 August, 2021; v1 submitted 13 July, 2021;
originally announced July 2021.
-
T product Tensors Part II: Tail Bounds for Sums of Random T product Tensors
Authors:
Shih Yu Chang,
Yimin Wei
Abstract:
This paper is the Part II of a serious work about T product tensors focusing at establishing new probability bounds for sums of random, independent, T product tensors. These probability bounds characterize large deviation behavior of the extreme eigenvalue of the sums of random T product tensors. We apply Lapalace transform method and Lieb concavity theorem for T product tensors obtained from our…
▽ More
This paper is the Part II of a serious work about T product tensors focusing at establishing new probability bounds for sums of random, independent, T product tensors. These probability bounds characterize large deviation behavior of the extreme eigenvalue of the sums of random T product tensors. We apply Lapalace transform method and Lieb concavity theorem for T product tensors obtained from our Part I paper, and apply these tools to generalize the classical bounds associated with the names Chernoff, and Bernstein from the scalar to the T product tensor setting. Tail bounds for the norm of a sum of random rectangular T product tensors are also derived from corollaries of random Hermitian T product tensors cases. The proof mechanism is also applied to T product tensor valued martingales and T product tensor based Azuma, Hoeffding and McDiarmid inequalities are derived.
△ Less
Submitted 8 December, 2021; v1 submitted 13 July, 2021;
originally announced July 2021.
-
Tensor Expander Chernoff Bounds
Authors:
Shih Yu Chang
Abstract:
The Chernoff bound is an important inequality relation in probability theory. The original version of the Chernoff bound is to give an exponential decreasing bound on the tail distribution of sums of independent random variables. Recent years, several works have been done by extending the original version of the Chernoff bound to high-dimensional random objects, e.g., random matrices, or/and to co…
▽ More
The Chernoff bound is an important inequality relation in probability theory. The original version of the Chernoff bound is to give an exponential decreasing bound on the tail distribution of sums of independent random variables. Recent years, several works have been done by extending the original version of the Chernoff bound to high-dimensional random objects, e.g., random matrices, or/and to consider the relaxation that there is no requirement of independent assumptions among random objects. In this work, we generalize the matrix expander Chernoff bound studied by Garg et al. at work: A Matrix Expander Chernoff Bound, to tensor expander Chernoff bounds. Our main tool is to develop new tensor norm inequalities based on log-majorization techniques. These new tensor norm inequalities are used to bound the expectation of Ky Fan norm of the random tensor exponential function, then tensor expander Chernoff bounds can be established. Compared with the matrix expander Chernoff bound, the tensor expander Chernoff bounds proved at this work contributes following aspects: (1) the random objects dimensions are increased from matrices (two-dimensional data array) to tensors (multidimensional data array); (2) this bound generalizes the identity map of the random objects summation to any polynomial function of the random objects summation; (3) Ky Fan norm, instead only the maximum or the minimum eigenvalues, for the function of the random objects summation is considered; (4) we remove the restriction about the summation of all mapped random objects is zero, which is required in the matrix expander Chernoff bound derivation.
△ Less
Submitted 16 May, 2021; v1 submitted 13 May, 2021;
originally announced May 2021.
-
General Tail Bounds for Random Tensors Summation: Majorization Approach
Authors:
Shih Yu Chang
Abstract:
In recent years, tensors have been applied to different applications in science and engineering fields. In order to establish theory about tail bounds of the tensors summation behavior, this work extends previous work by considering the tensors summation tail behavior of the top $k$-largest singular values of a function of the tensors summation, instead of the largest/smallest singular value of th…
▽ More
In recent years, tensors have been applied to different applications in science and engineering fields. In order to establish theory about tail bounds of the tensors summation behavior, this work extends previous work by considering the tensors summation tail behavior of the top $k$-largest singular values of a function of the tensors summation, instead of the largest/smallest singular value of the tensors summation directly (identity function) explored in Shih Yu's work: Convenient tail bounds for sums of random tensors. Majorization and antisymmetric tensor product tools are main techniques utilized to establish inequalities for unitarily norms of multivariate tensors. The Laplace transform method is integrated with these inequalities for unitarily norms of multivariate tensors to give us tail bounds estimation for Ky Fan $k$-norm for a function of the tensors summation. By restricting different random tensor conditions, we obtain generalized tensor Chernoff and Bernstein inequalities.
△ Less
Submitted 3 October, 2021; v1 submitted 13 May, 2021;
originally announced May 2021.
-
Convenient tail bounds for sums of random tensors
Authors:
Shih Yu Chang
Abstract:
This work prepares new probability bounds for sums of random, independent, Hermitian tensors. These probability bounds characterize large-deviation behavior of the extreme eigenvalue of the sums of random tensors. We extend Lapalace transform method and Lieb's concavity theorem from matrices to tensors, and apply these tools to generalize the classical bounds associated with the names Chernoff, Be…
▽ More
This work prepares new probability bounds for sums of random, independent, Hermitian tensors. These probability bounds characterize large-deviation behavior of the extreme eigenvalue of the sums of random tensors. We extend Lapalace transform method and Lieb's concavity theorem from matrices to tensors, and apply these tools to generalize the classical bounds associated with the names Chernoff, Bennett, and Bernstein from the scalar to the tensor setting. Tail bounds for the norm of a sum of random rectangular tensors are also derived from corollaries of random Hermitian tensors cases. The proof mechanism can also be applied to tensor-valued martingales and tensor-based Azuma, Hoeffding and McDiarmid inequalities are established.
△ Less
Submitted 30 December, 2020;
originally announced December 2020.
-
Sherman-Morrison-Woodbury Identity for Tensors
Authors:
Shih Yu Chang
Abstract:
In linear algebra, the sherman-morrison-woodbury identity says that the inverse of a rank-$k$ correction of some matrix can be computed by doing a rank-k correction to the inverse of the original matrix. This identity is crucial to accelerate the matrix inverse computation when the matrix involves correction. Many scientific and engineering applications have to deal with this matrix inverse proble…
▽ More
In linear algebra, the sherman-morrison-woodbury identity says that the inverse of a rank-$k$ correction of some matrix can be computed by doing a rank-k correction to the inverse of the original matrix. This identity is crucial to accelerate the matrix inverse computation when the matrix involves correction. Many scientific and engineering applications have to deal with this matrix inverse problem after updating the matrix, e.g., sensitivity analysis of linear systems, covariance matrix update in kalman filter, etc. However, there is no similar identity in tensors. In this work, we will derive the sherman-morrison-woodbury identity for invertible tensors first. Since not all tensors are invertible, we further generalize the sherman-morrison-woodbury identity for tensors with moore-penrose generalized inverse by utilizing orthogonal projection of the correction tensor part into the original tensor and its Hermitian tensor. According to this new established the sherman-morrison-woodbury identity for tensors, we can perform sensitivity analysis for multi-linear systems by deriving the normalized upper bound for the solution of a multilinear system. Several numerical examples are also presented to demonstrate how the normalized error upper bounds are affected by perturbation degree of tensor coefficients.
△ Less
Submitted 3 July, 2020;
originally announced July 2020.
-
A conformally invariant sphere theorem in four dimensions
Authors:
S. Y. A Chang,
Matthew J. Gursky,
Paul Yang
Abstract:
In this paper we provide a sharp characterization of the smooth four-dimensional sphere. The assumptions of the theorem are conformally invariant, and can be reduced to an L^2 inequality of the Weyl tensor and positivity of the Yamabe invariant.
In this paper we provide a sharp characterization of the smooth four-dimensional sphere. The assumptions of the theorem are conformally invariant, and can be reduced to an L^2 inequality of the Weyl tensor and positivity of the Yamabe invariant.
△ Less
Submitted 17 September, 2003;
originally announced September 2003.
