-
An Elementary proof for Bertrand's Postulate
Authors:
Pranav Narayan Sharma
Abstract:
In this paper we give an elementary proof for Bertrand's postulate also known as Bertrand-Chebyshev theorem.
In this paper we give an elementary proof for Bertrand's postulate also known as Bertrand-Chebyshev theorem.
△ Less
Submitted 11 July, 2024; v1 submitted 10 July, 2024;
originally announced July 2024.
-
Eigenvalue backward errors of Rosenbrock systems and optimization of sums of Rayleigh quotient
Authors:
Ding Lu,
Anshul Prajapati,
Punit Sharma,
Shreemayee Bora
Abstract:
We address the problem of computing the eigenvalue backward error of the Rosenbrock system matrix under various types of block perturbations. We establish computable formulas for these backward errors using a class of minimization problems involving the Sum of Two generalized Rayleigh Quotients (SRQ2). For computational purposes and analysis, we reformulate such optimization problems as minimizati…
▽ More
We address the problem of computing the eigenvalue backward error of the Rosenbrock system matrix under various types of block perturbations. We establish computable formulas for these backward errors using a class of minimization problems involving the Sum of Two generalized Rayleigh Quotients (SRQ2). For computational purposes and analysis, we reformulate such optimization problems as minimization of a rational function over the joint numerical range of three Hermitian matrices. This reformulation eliminates certain local minimizers of the original SRQ2 minimization and allows for convenient visualization of the solution. Furthermore, by exploiting the convexity within the joint numerical range, we derive a characterization of the optimal solution using a Nonlinear Eigenvalue Problem with Eigenvector dependency (NEPv). The NEPv characterization enables a more efficient solution of the SRQ2 minimization compared to traditional optimization techniques. Our numerical experiments demonstrate the benefits and effectiveness of the NEPv approach for SRQ2 minimization in computing eigenvalue backward errors of Rosenbrock systems.
△ Less
Submitted 4 July, 2024;
originally announced July 2024.
-
Structured singular values and their application in computing eigenvalue backward errors of the Rosenbrock system matrix
Authors:
Anshul Prajapati,
Punit Sharma
Abstract:
The structured singular values (aka the μ-values) are essential in analyzing the stability of control systems and in the structured eigenvalue perturbation theory of matrices and matrix polynomials. In this paper, we study the μ-value of a matrix under block-diagonal structured perturbations (full blocks but possibly rectangular). We provide an explicit expression for the μ-value and also obtain a…
▽ More
The structured singular values (aka the μ-values) are essential in analyzing the stability of control systems and in the structured eigenvalue perturbation theory of matrices and matrix polynomials. In this paper, we study the μ-value of a matrix under block-diagonal structured perturbations (full blocks but possibly rectangular). We provide an explicit expression for the μ-value and also obtain a computable upper bound in terms of minimizing the largest singular value of a parameter-dependent matrix. This upper bound equals the μ-value when the perturbation matrix has no more than three blocks on the diagonal. We then apply the μ-value results in computing eigenvalue backward errors of a Rosenbrock system matrix corresponding to a rational matrix function when some or all blocks of the Rosenbrock system matrix are subject to perturbation. The results are illustrated through numerical experiments.
△ Less
Submitted 20 May, 2024;
originally announced May 2024.
-
Transitivity And Related Notions For Graph Induced Symbolic Systems
Authors:
Prashant Kumar,
Puneet Sharma
Abstract:
In this paper, we investigate the dynamical behavior of a two dimensional shift $X_G$ (generated by a two dimensional graph $G=(\mathcal{H},\mathcal{V})$) using the adjacency matrices of the generating graph $G$. In particular, we investigate properties such as transitivity, directional transitivity, weak mixing, directional weak mixing and mixing for the shift space $X_G$. We prove that if…
▽ More
In this paper, we investigate the dynamical behavior of a two dimensional shift $X_G$ (generated by a two dimensional graph $G=(\mathcal{H},\mathcal{V})$) using the adjacency matrices of the generating graph $G$. In particular, we investigate properties such as transitivity, directional transitivity, weak mixing, directional weak mixing and mixing for the shift space $X_G$. We prove that if $(HV)_{ij}\neq 0 \Leftrightarrow (VH)_{ij}\neq 0$ (for all $i,j$), while doubly transitivity (weak mixing) of $X_H$ (or $X_V$) ensures the same for two dimensional shift generated by the graph $G$, directional transitivity (in the direction $(r,s)$) can be characterized through the block representation of $H^rV^s$. We provide necessary and sufficient criteria to establish horizontal (vertical) transitivity for the shift space $X_G$. We also provide examples to establish the necessity of the conditions imposed. Finally, we investigate the decomposability of a given graph into product of graphs with reduced complexity.
△ Less
Submitted 2 November, 2023;
originally announced November 2023.
-
High-probability Convergence Bounds for Nonlinear Stochastic Gradient Descent Under Heavy-tailed Noise
Authors:
Aleksandar Armacki,
Pranay Sharma,
Gauri Joshi,
Dragana Bajovic,
Dusan Jakovetic,
Soummya Kar
Abstract:
We study high-probability convergence guarantees of learning on streaming data in the presence of heavy-tailed noise. In the proposed scenario, the model is updated in an online fashion, as new information is observed, without storing any additional data. To combat the heavy-tailed noise, we consider a general framework of nonlinear stochastic gradient descent (SGD), providing several strong resul…
▽ More
We study high-probability convergence guarantees of learning on streaming data in the presence of heavy-tailed noise. In the proposed scenario, the model is updated in an online fashion, as new information is observed, without storing any additional data. To combat the heavy-tailed noise, we consider a general framework of nonlinear stochastic gradient descent (SGD), providing several strong results. First, for non-convex costs and component-wise nonlinearities, we establish a convergence rate arbitrarily close to $\mathcal{O}\left(t^{-\frac{1}{4}}\right)$, whose exponent is independent of noise and problem parameters. Second, for strongly convex costs and component-wise nonlinearities, we establish a rate arbitrarily close to $\mathcal{O}\left(t^{-\frac{1}{2}}\right)$ for the weighted average of iterates, with exponent again independent of noise and problem parameters. Finally, for strongly convex costs and a broader class of nonlinearities, we establish convergence of the last iterate, with a rate $\mathcal{O}\left(t^{-ζ} \right)$, where $ζ\in (0,1)$ depends on problem parameters, noise and nonlinearity. As we show analytically and numerically, $ζ$ can be used to inform the preferred choice of nonlinearity for given problem settings. Compared to state-of-the-art, who only consider clip**, require bounded noise moments of order $η\in (1,2]$, and establish convergence rates whose exponents go to zero as $η\rightarrow 1$, we provide high-probability guarantees for a much broader class of nonlinearities and symmetric density noise, with convergence rates whose exponents are bounded away from zero, even when the noise has finite first moment only. Moreover, in the case of strongly convex functions, we demonstrate analytically and numerically that clip** is not always the optimal nonlinearity, further underlining the value of our general framework.
△ Less
Submitted 30 April, 2024; v1 submitted 28 October, 2023;
originally announced October 2023.
-
On Equicontinuity and Related Notions in Nonautonomous Dynamical Systems
Authors:
Sushmita Yadav,
Puneet Sharma
Abstract:
In this work, we investigate the dynamics of a general non-autonomous system generated by a commutative family of homeomorphisms. In particular, we investigate properties such as periodicity, equicontinuity, minimality and transitivity for a general non-autonomous dynamical system. In \cite{sk2}, the authors derive necessary and sufficient conditions for a system to be minimal. We claim the result…
▽ More
In this work, we investigate the dynamics of a general non-autonomous system generated by a commutative family of homeomorphisms. In particular, we investigate properties such as periodicity, equicontinuity, minimality and transitivity for a general non-autonomous dynamical system. In \cite{sk2}, the authors derive necessary and sufficient conditions for a system to be minimal. We claim the result to be false and provide an example in support of our claim. Further, we correct the result to derive necessary and sufficient conditions for a non-autonomous system to be minimal. We prove that for an equicontinuous flow generated by a commutative family, while the system need not exhibit almost periodic points, if $x$ is almost periodic then every point in $\overline{\mathcal{O}_H(x)}$ is almost periodic. We further prove that in such a case, the set $\overline{\mathcal{O}_H(x)}$ is uniformly almost periodic and hence provide an analogous extension to a result known for the autonomous systems. We prove that a system generated by a commutative family is transitive if and only if it exhibits a point with dense orbit. We also prove that any minimal system generated by commutative family is either equicontinuous or has a dense set of sensitive points.
△ Less
Submitted 5 October, 2023;
originally announced October 2023.
-
Partial Group Representations on Semialgebras
Authors:
Thakur Meenakshi,
R. P. Sharma
Abstract:
Let $A$ be an additively cancellative semialgebra over an additively cancellative semifield $K$ as defined in [9]. For a given partial action $α$ of a group $G$ on an algebra, the associativity of partial skew group ring together with the existence and uniqueness of envelo** (global) action were studied by M. Dokuchaev and R. Exel [2] which were extended for semialgebras with some restriction by…
▽ More
Let $A$ be an additively cancellative semialgebra over an additively cancellative semifield $K$ as defined in [9]. For a given partial action $α$ of a group $G$ on an algebra, the associativity of partial skew group ring together with the existence and uniqueness of envelo** (global) action were studied by M. Dokuchaev and R. Exel [2] which were extended for semialgebras with some restriction by Sharma et. al. using the ring of differences. In a similar way, we extend the results of [2,3] for semialgebras regarding partial representations.
△ Less
Submitted 23 June, 2023; v1 submitted 20 March, 2023;
originally announced June 2023.
-
Characterisation of equivalent norms on a linear space using exponential vector space
Authors:
Dhruba Prakash Biswas,
Priti Sharma,
Sandip Jana
Abstract:
In this paper we have found a necessary and sufficient condition for equivalence of two norms on a linear space using the theory of exponential vector space. Exponential vector space is an ordered algebraic structure which can be considered as an algebraic ordered extension of vector space. This structure is axiomatised on the basis of the intrinsic properties of the hyperspace…
▽ More
In this paper we have found a necessary and sufficient condition for equivalence of two norms on a linear space using the theory of exponential vector space. Exponential vector space is an ordered algebraic structure which can be considered as an algebraic ordered extension of vector space. This structure is axiomatised on the basis of the intrinsic properties of the hyperspace $\mathscr{C}(\mathcal X)$ comprising all nonempty compact subsets of a Hausdorff topological vector space $\mathcal X$. Exponential vector space is a conglomeration of a semigroup structure, a scalar multiplication and a compatible partial order. We have shown that the collection of all norms defined on a linear space, together with the constant function zero, forms a topological exponential vector space. Then using the concept of comparing function (a concept defined on a topological exponential vector space) we have proved the aforesaid necessary and sufficient condition; also we have proved using comparing function that in an infinite dimensional linear space there are uncountably many non-equivalent norms.
△ Less
Submitted 20 May, 2023;
originally announced May 2023.
-
Additional results on convergence and semiconvergence of three-step alternating iteration scheme for singular linear systems
Authors:
Vaibhav Shekhar,
Punit Sharma
Abstract:
The three-step alternating iteration scheme for finding an iterative solution of a singular (non-singular) linear systems in a faster way was introduced by Nandi {\it et al.} [Numer. Algorithms; 84 (2) (2020) 457-483], recently. The authors then provided its convergence criteria for a class of matrix splitting called proper G-weak regular splittings of type I. In this note, we analyze further the…
▽ More
The three-step alternating iteration scheme for finding an iterative solution of a singular (non-singular) linear systems in a faster way was introduced by Nandi {\it et al.} [Numer. Algorithms; 84 (2) (2020) 457-483], recently. The authors then provided its convergence criteria for a class of matrix splitting called proper G-weak regular splittings of type I. In this note, we analyze further the convergence criteria of the same scheme. In this aspect, we obtain sufficient conditions for the convergence of the same scheme for another class of matrix splittings called proper G-weak regular splittings of type II. We then show that this scheme converges faster than the two-step alternating and usual iteration schemes, even for this class of splittings. As a particular case, we also establish faster convergence criteria of three-step in a nonsingular matrix setting. This is shown that a large amount of computational time and memory are required in single-step and two-step alternating iterative methods to solve the nonsingular linear systems more efficiently than the three-step alternating iteration method. Finally, the semiconvergence of a three-step alternating iterative scheme is established. Its faster semiconvergence is demonstrated by considering a singular linear system arising from the Markov process.
△ Less
Submitted 8 May, 2023;
originally announced May 2023.
-
Moments of derivatives of modular $L$-functions
Authors:
Sumit Kumar,
Kummari Mallesham,
Prahlad Sharma,
Saurabh Kumar Singh
Abstract:
Let $f$ be an Hecke eigenform for the group $Γ_{0}(q)$ and $χ_{d}$ be a primitive quadratic character of conductor $|d|$. In this article, we prove an asymptotic for the second moment of the derivative of $L(s, f \otimes χ_{8d})$ at the central point $1/2$, which was previously known under GRH by Petrow \cite{petrow}.
Let $f$ be an Hecke eigenform for the group $Γ_{0}(q)$ and $χ_{d}$ be a primitive quadratic character of conductor $|d|$. In this article, we prove an asymptotic for the second moment of the derivative of $L(s, f \otimes χ_{8d})$ at the central point $1/2$, which was previously known under GRH by Petrow \cite{petrow}.
△ Less
Submitted 24 October, 2023; v1 submitted 29 March, 2023;
originally announced March 2023.
-
Bilinear sums with $GL(2)$ coefficients and the exponent of distribution of $d_3$
Authors:
Prahlad Sharma
Abstract:
We obtain the exponent of distribution $1/2+1/30$ for the ternary divisor function $d_3$ to square-free and prime power moduli, improving the previous results of Fouvry--Kowalski--Michel, Heath-Brown, and Friedlander--Iwaniec. The key input is certain estimates on bilinear sums with $GL(2)$ coefficients obtained using the delta symbol approach.
We obtain the exponent of distribution $1/2+1/30$ for the ternary divisor function $d_3$ to square-free and prime power moduli, improving the previous results of Fouvry--Kowalski--Michel, Heath-Brown, and Friedlander--Iwaniec. The key input is certain estimates on bilinear sums with $GL(2)$ coefficients obtained using the delta symbol approach.
△ Less
Submitted 15 February, 2024; v1 submitted 10 March, 2023;
originally announced March 2023.
-
Bioconvection in a phototactic algae suspension with oblique irradiation and forward anisotropic scattering
Authors:
Sandeep Kumar,
Preeti Sharma
Abstract:
In this study, we analyze the bioconvection in a suspension of phototactic algae that exhibits anisotropic scattering. The top layer of the suspension is illuminated by oblique collimated irradiation. During the study, the bottom boundary is considered as rigid whereas the top boundary is considered stress-free. In order to solve the eigenvalue problem, the Newton-Raphson-Kantorovich finite differ…
▽ More
In this study, we analyze the bioconvection in a suspension of phototactic algae that exhibits anisotropic scattering. The top layer of the suspension is illuminated by oblique collimated irradiation. During the study, the bottom boundary is considered as rigid whereas the top boundary is considered stress-free. In order to solve the eigenvalue problem, the Newton-Raphson-Kantorovich finite difference method of order four is used. Linear analysis of the basic state is performed using neutral curves. The results demonstrate a change in the most unstable mode from an overstable to a stationary state or vice versa for particular parameters in response to a variation in the incidence angle. The position of the maximum basic concentration shifts toward the top of the suspension as the incidence angle is increased. In most cases, the system becomes more unstable with an increment in the incidence angle.
△ Less
Submitted 4 March, 2023;
originally announced March 2023.
-
Federated Minimax Optimization with Client Heterogeneity
Authors:
Pranay Sharma,
Rohan Panda,
Gauri Joshi
Abstract:
Minimax optimization has seen a surge in interest with the advent of modern applications such as GANs, and it is inherently more challenging than simple minimization. The difficulty is exacerbated by the training data residing at multiple edge devices or \textit{clients}, especially when these clients can have heterogeneous datasets and local computation capabilities. We propose a general federate…
▽ More
Minimax optimization has seen a surge in interest with the advent of modern applications such as GANs, and it is inherently more challenging than simple minimization. The difficulty is exacerbated by the training data residing at multiple edge devices or \textit{clients}, especially when these clients can have heterogeneous datasets and local computation capabilities. We propose a general federated minimax optimization framework that subsumes such settings and several existing methods like Local SGDA. We show that naive aggregation of heterogeneous local progress results in optimizing a mismatched objective function -- a phenomenon previously observed in standard federated minimization. To fix this problem, we propose normalizing the client updates by the number of local steps undertaken between successive communication rounds. We analyze the convergence of the proposed algorithm for classes of nonconvex-concave and nonconvex-nonconcave functions and characterize the impact of heterogeneous client data, partial client participation, and heterogeneous local computations. Our analysis works under more general assumptions on the intra-client noise and inter-client heterogeneity than so far considered in the literature. For all the function classes considered, we significantly improve the existing computation and communication complexity results. Experimental results support our theoretical claims.
△ Less
Submitted 9 February, 2023; v1 submitted 8 February, 2023;
originally announced February 2023.
-
Inductive algebras for the motion group of the plane
Authors:
Promod Sharma,
M. K. Vemuri
Abstract:
Each irreducible representation of the motion group of the plane has a unique maximal inductive algebra, and it is self adjoint.
Each irreducible representation of the motion group of the plane has a unique maximal inductive algebra, and it is self adjoint.
△ Less
Submitted 7 December, 2022;
originally announced December 2022.
-
Inductive algebras for compact groups
Authors:
Promod Sharma,
M. K. Vemuri
Abstract:
Inductive algebras for a compact group are self-adjoint
Inductive algebras for a compact group are self-adjoint
△ Less
Submitted 23 November, 2022;
originally announced November 2022.
-
Characterizing matrices with eigenvalues in an LMI region: A dissipative-Hamiltonian approach
Authors:
Neelam Choudhary,
Nicolas Gillis,
Punit Sharma
Abstract:
In this paper, we provide a dissipative Hamiltonian (DH) characterization for the set of matrices whose eigenvalues belong to a given LMI region. This characterization is a generalization of that of Choudhary et al. (Numer. Linear Algebra Appl., 2020) to any LMI region. It can be used in various contexts, which we illustrate on the nearest $Ω$-stable matrix problem: given an LMI region…
▽ More
In this paper, we provide a dissipative Hamiltonian (DH) characterization for the set of matrices whose eigenvalues belong to a given LMI region. This characterization is a generalization of that of Choudhary et al. (Numer. Linear Algebra Appl., 2020) to any LMI region. It can be used in various contexts, which we illustrate on the nearest $Ω$-stable matrix problem: given an LMI region $Ω\subseteq \mathbb{C}$ and a matrix $A \in \mathbb{C}^{n,n}$, find the nearest matrix to $A$ whose eigenvalues belong to $Ω$. Finally, we generalize our characterization to more general regions that can be expressed using LMIs involving complex matrices.
△ Less
Submitted 13 October, 2022;
originally announced October 2022.
-
Structured eigenvalue backward errors for rational matrix polynomials with symmetry structures
Authors:
Anshul Prajapati,
Punit Sharma
Abstract:
We derive computable formulas for the structured backward errors of a complex number $λ$ when considered as an approximate eigenvalue of rational matrix polynomials that carry a symmetry structure. We consider symmetric, skew-symmetric, T-even, T-odd, Hermitian, skew-Hermitian, $*$-even, $*$-odd, and $*$-palindromic structures. Numerical experiments show that the backward errors with respect to st…
▽ More
We derive computable formulas for the structured backward errors of a complex number $λ$ when considered as an approximate eigenvalue of rational matrix polynomials that carry a symmetry structure. We consider symmetric, skew-symmetric, T-even, T-odd, Hermitian, skew-Hermitian, $*$-even, $*$-odd, and $*$-palindromic structures. Numerical experiments show that the backward errors with respect to structure-preserving and arbitrary perturbations are significantly different.
△ Less
Submitted 29 August, 2022;
originally announced August 2022.
-
Doubly structured map** problems of the form $Δx=y$ and $Δ^*z=w$
Authors:
Mohit Kumar Baghel,
Punit Sharma
Abstract:
For a given class of structured matrices $\mathbb S$, we find necessary and sufficient conditions on vectors $x,w\in \C^{n+m}$ and $y,z \in \C^{n}$ for which there exists $Δ=[Δ_1~Δ_2]$ with $Δ_1 \in \mathbb S$ and $Δ_2 \in \C^{n,m}$ such that $Δx=y$ and $Δ^*z=w$. We also characterize the set of all such map**s $Δ$ and provide sufficient conditions on vectors $x,y,z$, and $w$ to investigate a…
▽ More
For a given class of structured matrices $\mathbb S$, we find necessary and sufficient conditions on vectors $x,w\in \C^{n+m}$ and $y,z \in \C^{n}$ for which there exists $Δ=[Δ_1~Δ_2]$ with $Δ_1 \in \mathbb S$ and $Δ_2 \in \C^{n,m}$ such that $Δx=y$ and $Δ^*z=w$. We also characterize the set of all such map**s $Δ$ and provide sufficient conditions on vectors $x,y,z$, and $w$ to investigate a $Δ$ with minimal Frobenius norm. The structured classes $\mathbb S$ we consider include (skew)-Hermitian, (skew)-symmetric, pseudo(skew)-symmetric, $J$-(skew)-symmetric, pseudo(skew)-Hermitian, positive (semi)definite, and dissipative matrices. These map**s are then used in computing the structured eigenvalue/eigenpair backward errors of matrix pencils arising in optimal control.
△ Less
Submitted 26 August, 2022;
originally announced August 2022.
-
Real interpolation of functions with applications to accretive operators on Banach spaces
Authors:
Ralph Chill,
Praveen Sharma,
Sachi Srivastava
Abstract:
We study real interpolation, but instead of interpolating between Banach spaces, we interpolate between general functions taking values in $[0,\infty].$ We show the equivalence of the mean method and the $K$-method and apply the general theory to interpolation between the norm on a Banach space and the set norm associated with an m-accretive operator on such a space.
We study real interpolation, but instead of interpolating between Banach spaces, we interpolate between general functions taking values in $[0,\infty].$ We show the equivalence of the mean method and the $K$-method and apply the general theory to interpolation between the norm on a Banach space and the set norm associated with an m-accretive operator on such a space.
△ Less
Submitted 27 June, 2022;
originally announced June 2022.
-
Federated Minimax Optimization: Improved Convergence Analyses and Algorithms
Authors:
Pranay Sharma,
Rohan Panda,
Gauri Joshi,
Pramod K. Varshney
Abstract:
In this paper, we consider nonconvex minimax optimization, which is gaining prominence in many modern machine learning applications such as GANs. Large-scale edge-based collection of training data in these applications calls for communication-efficient distributed optimization algorithms, such as those used in federated learning, to process the data. In this paper, we analyze Local stochastic grad…
▽ More
In this paper, we consider nonconvex minimax optimization, which is gaining prominence in many modern machine learning applications such as GANs. Large-scale edge-based collection of training data in these applications calls for communication-efficient distributed optimization algorithms, such as those used in federated learning, to process the data. In this paper, we analyze Local stochastic gradient descent ascent (SGDA), the local-update version of the SGDA algorithm. SGDA is the core algorithm used in minimax optimization, but it is not well-understood in a distributed setting. We prove that Local SGDA has \textit{order-optimal} sample complexity for several classes of nonconvex-concave and nonconvex-nonconcave minimax problems, and also enjoys \textit{linear speedup} with respect to the number of clients. We provide a novel and tighter analysis, which improves the convergence and communication guarantees in the existing literature. For nonconvex-PL and nonconvex-one-point-concave functions, we improve the existing complexity results for centralized minimax problems. Furthermore, we propose a momentum-based local-update algorithm, which has the same convergence guarantees, but outperforms Local SGDA as demonstrated in our experiments.
△ Less
Submitted 9 March, 2022;
originally announced March 2022.
-
Solving matrix nearness problems via Hamiltonian systems, matrix factorization, and optimization
Authors:
Nicolas Gillis,
Punit Sharma
Abstract:
In these lectures notes, we review our recent works addressing various problems of finding the nearest stable system to an unstable one. After the introduction, we provide some preliminary background, namely, defining Port-Hamiltonian systems and dissipative Hamiltonian systems and their properties, briefly discussing matrix factorizations, and describing the optimization methods that we will use…
▽ More
In these lectures notes, we review our recent works addressing various problems of finding the nearest stable system to an unstable one. After the introduction, we provide some preliminary background, namely, defining Port-Hamiltonian systems and dissipative Hamiltonian systems and their properties, briefly discussing matrix factorizations, and describing the optimization methods that we will use in these notes. In the third chapter, we present our approach to tackle the distance to stability for standard continuous linear time invariant (LTI) systems. The main idea is to rely on the characterization of stable systems as dissipative Hamiltonian systems. We show how this idea can be generalized to compute the nearest $Ω$-stable matrix, where the eigenvalues of the sought system matrix $A$ are required to belong a rather general set $Ω$. We also show how these ideas can be used to compute minimal-norm static feedbacks, that is, stabilize a system by choosing a proper input $u(t)$ that linearly depends on $x(t)$ (static-state feedback), or on $y(t)$ (static-output feedback). In the fourth chapter, we present our approach to tackle the distance to passivity. The main idea is to rely on the characterization of stable systems as port-Hamiltonian systems. We also discuss in more details the special case of computing the nearest stable matrix pairs. In the last chapter, we focus on discrete-time LTI systems. Similarly as for the continuous case, we propose a parametrization that allows efficiently compute the nearest stable system (for matrices and matrix pairs), allowing to compute the distance to stability. We show how this idea can be used in data-driven system identification, that is, given a set of input-output pairs, identify the system $A$.
△ Less
Submitted 5 February, 2022;
originally announced February 2022.
-
Distributed Cooperative Multi-Agent Reinforcement Learning with Directed Coordination Graph
Authors:
Gangshan **g,
He Bai,
Jemin George,
Aranya Chakrabortty,
Piyush. K. Sharma
Abstract:
Existing distributed cooperative multi-agent reinforcement learning (MARL) frameworks usually assume undirected coordination graphs and communication graphs while estimating a global reward via consensus algorithms for policy evaluation. Such a framework may induce expensive communication costs and exhibit poor scalability due to requirement of global consensus. In this work, we study MARLs with d…
▽ More
Existing distributed cooperative multi-agent reinforcement learning (MARL) frameworks usually assume undirected coordination graphs and communication graphs while estimating a global reward via consensus algorithms for policy evaluation. Such a framework may induce expensive communication costs and exhibit poor scalability due to requirement of global consensus. In this work, we study MARLs with directed coordination graphs, and propose a distributed RL algorithm where the local policy evaluations are based on local value functions. The local value function of each agent is obtained by local communication with its neighbors through a directed learning-induced communication graph, without using any consensus algorithm. A zeroth-order optimization (ZOO) approach based on parameter perturbation is employed to achieve gradient estimation. By comparing with existing ZOO-based RL algorithms, we show that our proposed distributed RL algorithm guarantees high scalability. A distributed resource allocation example is shown to illustrate the effectiveness of our algorithm.
△ Less
Submitted 9 January, 2022;
originally announced January 2022.
-
On Graph Induced Symbolic Systems
Authors:
Prashant Kumar,
Puneet Sharma
Abstract:
\begin{abstract} In this paper, we investigate a shift arising from graph $G$. We prove that any $k$-dimensional shift of finite type can be generated through a $k$-dimensional graph. We investigate the structure of the shift space using the generating matrices for the shift space. We prove that a two dimensional shift space has a horizontally (vertically) periodic point if and only if it possesse…
▽ More
\begin{abstract} In this paper, we investigate a shift arising from graph $G$. We prove that any $k$-dimensional shift of finite type can be generated through a $k$-dimensional graph. We investigate the structure of the shift space using the generating matrices for the shift space. We prove that a two dimensional shift space has a horizontally (vertically) periodic point if and only if it possesses a $(m,n)$-periodic point (for some $m,n\in \mathbb{Z}\setminus \{0\}$). We prove that a shift space is finite if and only if it can be generated by permutation matrices. We study the non-emptiness problem and existence of periodic points in terms of the generating matrices.
△ Less
Submitted 28 December, 2021;
originally announced December 2021.
-
Asynchronous Distributed Reinforcement Learning for LQR Control via Zeroth-Order Block Coordinate Descent
Authors:
Gangshan **g,
He Bai,
Jemin George,
Aranya Chakrabortty,
Piyush K. Sharma
Abstract:
Recently introduced distributed zeroth-order optimization (ZOO) algorithms have shown their utility in distributed reinforcement learning (RL). Unfortunately, in the gradient estimation process, almost all of them require random samples with the same dimension as the global variable and/or require evaluation of the global cost function, which may induce high estimation variance for large-scale net…
▽ More
Recently introduced distributed zeroth-order optimization (ZOO) algorithms have shown their utility in distributed reinforcement learning (RL). Unfortunately, in the gradient estimation process, almost all of them require random samples with the same dimension as the global variable and/or require evaluation of the global cost function, which may induce high estimation variance for large-scale networks. In this paper, we propose a novel distributed zeroth-order algorithm by leveraging the network structure inherent in the optimization objective, which allows each agent to estimate its local gradient by local cost evaluation independently, without use of any consensus protocol. The proposed algorithm exhibits an asynchronous update scheme, and is designed for stochastic non-convex optimization with a possibly non-convex feasible domain based on the block coordinate descent method. The algorithm is later employed as a distributed model-free RL algorithm for distributed linear quadratic regulator design, where a learning graph is designed to describe the required interaction relationship among agents in distributed learning. We provide an empirical validation of the proposed algorithm to benchmark its performance on convergence rate and variance against a centralized ZOO algorithm.
△ Less
Submitted 2 May, 2024; v1 submitted 26 July, 2021;
originally announced July 2021.
-
Optimizing Rayleigh quotient with symmetric constraints and their applications to perturbations of the structured polynomial eigenvalue problem
Authors:
Anshul Prajapati,
Punit Sharma
Abstract:
For a Hermitian matrix $H \in \mathbb C^{n,n}$ and symmetric matrices $S_0, S_1,\ldots,S_k \in \mathbb C^{n,n}$, we consider the problem of computing the supremum of $\left\{ \frac{v^*Hv}{v^*v}:~v\in \mathbb C^{n}\setminus \{0\},\,v^TS_iv=0~\text{for}~i=0,\ldots,k\right\}$. For this, we derive an estimation in the form of minimizing the second largest eigenvalue of a parameter depending Hermitian…
▽ More
For a Hermitian matrix $H \in \mathbb C^{n,n}$ and symmetric matrices $S_0, S_1,\ldots,S_k \in \mathbb C^{n,n}$, we consider the problem of computing the supremum of $\left\{ \frac{v^*Hv}{v^*v}:~v\in \mathbb C^{n}\setminus \{0\},\,v^TS_iv=0~\text{for}~i=0,\ldots,k\right\}$. For this, we derive an estimation in the form of minimizing the second largest eigenvalue of a parameter depending Hermitian matrix, which is exact when the eigenvalue at the optimal is simple. The results are then applied to compute the eigenvalue backward errors of higher degree matrix polynomials with T-palindromic, T-antipalindromic, T-even, T-odd, and skew-symmetric structures. The results are illustrated by numerical experiments.
△ Less
Submitted 14 June, 2021;
originally announced June 2021.
-
STEM: A Stochastic Two-Sided Momentum Algorithm Achieving Near-Optimal Sample and Communication Complexities for Federated Learning
Authors:
Prashant Khanduri,
Pranay Sharma,
Haibo Yang,
Mingyi Hong,
Jia Liu,
Ketan Rajawat,
Pramod K. Varshney
Abstract:
Federated Learning (FL) refers to the paradigm where multiple worker nodes (WNs) build a joint model by using local data. Despite extensive research, for a generic non-convex FL problem, it is not clear, how to choose the WNs' and the server's update directions, the minibatch sizes, and the local update frequency, so that the WNs use the minimum number of samples and communication rounds to achiev…
▽ More
Federated Learning (FL) refers to the paradigm where multiple worker nodes (WNs) build a joint model by using local data. Despite extensive research, for a generic non-convex FL problem, it is not clear, how to choose the WNs' and the server's update directions, the minibatch sizes, and the local update frequency, so that the WNs use the minimum number of samples and communication rounds to achieve the desired solution. This work addresses the above question and considers a class of stochastic algorithms where the WNs perform a few local updates before communication. We show that when both the WN's and the server's directions are chosen based on a stochastic momentum estimator, the algorithm requires $\tilde{\mathcal{O}}(ε^{-3/2})$ samples and $\tilde{\mathcal{O}}(ε^{-1})$ communication rounds to compute an $ε$-stationary solution. To the best of our knowledge, this is the first FL algorithm that achieves such {\it near-optimal} sample and communication complexities simultaneously. Further, we show that there is a trade-off curve between local update frequencies and local minibatch sizes, on which the above sample and communication complexities can be maintained. Finally, we show that for the classical FedAvg (a.k.a. Local SGD, which is a momentum-less special case of the STEM), a similar trade-off curve exists, albeit with worse sample and communication complexities. Our insights on this trade-off provides guidelines for choosing the four important design elements for FL algorithms, the update frequency, directions, and minibatch sizes to achieve the best performance.
△ Less
Submitted 19 June, 2021;
originally announced June 2021.
-
Estimation to structured distances to singularity for matrix pencils with symmetry structures: A linear algebra-based approach
Authors:
Anshul Prajapati,
Punit Sharma
Abstract:
We study the structured distance to singularity for a given regular matrix pencil $A+sE$, where $(A,E)\in \mathbb S \subseteq (\mathbb C^{n,n})^2$. This includes Hermitian, skew-Hermitian, $*$-even, $*$-odd, $*$-palindromic, T-palindromic, and dissipative Hamiltonian pencils. We present a purely linear algebra-based approach to derive explicit computable formulas for the distance to the nearest st…
▽ More
We study the structured distance to singularity for a given regular matrix pencil $A+sE$, where $(A,E)\in \mathbb S \subseteq (\mathbb C^{n,n})^2$. This includes Hermitian, skew-Hermitian, $*$-even, $*$-odd, $*$-palindromic, T-palindromic, and dissipative Hamiltonian pencils. We present a purely linear algebra-based approach to derive explicit computable formulas for the distance to the nearest structured pencil $(A-Δ_A)+s(E-Δ_E)$ such that $A-Δ_A$ and $E-Δ_E$ have a common null vector. We then obtain a family of computable lower bounds for the unstructured and structured distances to singularity. Numerical experiments suggest that in many cases, there is a significant difference between structured and unstructured distances.
This approach extends to structured matrix polynomials with higher degrees.
△ Less
Submitted 28 May, 2021;
originally announced May 2021.
-
On the non-symmetric semidefinite Procrustes problem
Authors:
Mohit Kumar Baghel,
Nicolas Gillis,
Punit Sharma
Abstract:
In this paper, we consider the non-symmetric positive semidefinite Procrustes (NSPSDP) problem: Given two matrices $X,Y \in \mathbb{R}^{n,m}$, find the matrix $A \in \mathbb{R}^{n,n}$ that minimizes the Frobenius norm of $AX-Y$ and which is such that $A+A^T$ is positive semidefinite. We generalize the semi-analytical approach for the symmetric positive semidefinite Procrustes problem, where $A$ is…
▽ More
In this paper, we consider the non-symmetric positive semidefinite Procrustes (NSPSDP) problem: Given two matrices $X,Y \in \mathbb{R}^{n,m}$, find the matrix $A \in \mathbb{R}^{n,n}$ that minimizes the Frobenius norm of $AX-Y$ and which is such that $A+A^T$ is positive semidefinite. We generalize the semi-analytical approach for the symmetric positive semidefinite Procrustes problem, where $A$ is required to be positive semidefinite, that was proposed by Gillis and Sharma (A semi-analytical approach for the positive semidefinite Procrustes problem, Linear Algebra Appl. 540, 112-137, 2018). As for the symmetric case, we first show that the NSPSDP problem can be reduced to a smaller NSPSDP problem that always has a unique solution and where the matrix $X$ is diagonal and has full rank. Then, an efficient semi-analytical algorithm to solve the NSPSDP problem is proposed, solving the smaller and well-posed problem with a fast gradient method which guarantees a linear rate of convergence. This algorithm is also applicable to solve the complex NSPSDP problem, where $X,Y \in \mathbb{C}^{n,m}$, as we show the complex NSPSDP problem can be written as an overparametrized real NSPSDP problem. The efficiency of the proposed algorithm is illustrated on several numerical examples.
△ Less
Submitted 15 April, 2021; v1 submitted 13 April, 2021;
originally announced April 2021.
-
Characterization of the dissipative map**s and their application to perturbations of dissipative-Hamiltonian systems
Authors:
Mohit Kumar Baghel,
Nicolas Gillis,
Punit Sharma
Abstract:
In this paper, we find necessary and sufficient conditions to identify pairs of matrices $X$ and $Y$ for which there exists $Δ\in \mathbb C^{n,n}$ such that $Δ+Δ^*$ is positive semidefinite and $ΔX=Y$. Such a $Δ$ is called a dissipative map** taking $X$ to $Y$. We also provide two different characterizations for the set of all dissipative map**s, and use them to characterize the unique dissipa…
▽ More
In this paper, we find necessary and sufficient conditions to identify pairs of matrices $X$ and $Y$ for which there exists $Δ\in \mathbb C^{n,n}$ such that $Δ+Δ^*$ is positive semidefinite and $ΔX=Y$. Such a $Δ$ is called a dissipative map** taking $X$ to $Y$. We also provide two different characterizations for the set of all dissipative map**s, and use them to characterize the unique dissipative map** with minimal Frobenius norm. The minimal-norm dissipative map** is then used to determine the distance to asymptotic instability for dissipative-Hamiltonian systems under general structure-preserving perturbations. We illustrate our results over some numerical examples and compare them with those of Mehl, Mehrmann and Sharma (Stability Radii for Linear Hamiltonian Systems with Dissipation Under Structure-Preserving Perturbations, SIAM J. Mat. Anal. Appl.\ 37 (4): 1625-1654, 2016).
△ Less
Submitted 2 April, 2021;
originally announced April 2021.
-
Bicomplex Mittag-Leffler Function and Properties
Authors:
Ritu Agarwal,
Urvashi Purohit Sharma,
Ravi P. Agarwal
Abstract:
With the increasing importance of the Mittag-Leffler function in the physical applications, these days many researchers are studying various generalizations and extensions of the Mittag-Leffler function. In this paper efforts are made to define bicomplex extension of the Mittag-Leffler function and also its analyticity and region of convergence are discussed. Various properties of the bicomplex Mi…
▽ More
With the increasing importance of the Mittag-Leffler function in the physical applications, these days many researchers are studying various generalizations and extensions of the Mittag-Leffler function. In this paper efforts are made to define bicomplex extension of the Mittag-Leffler function and also its analyticity and region of convergence are discussed. Various properties of the bicomplex Mittag-Leffler function including integral representation, recurrence relations, duplication formula and differential relations are established.
△ Less
Submitted 18 March, 2021;
originally announced March 2021.
-
Learning Distributed Stabilizing Controllers for Multi-Agent Systems
Authors:
Gangshan **g,
He Bai,
Jemin George,
Aranya Chakrabortty,
Piyush K. Sharma
Abstract:
We address the problem of model-free distributed stabilization of heterogeneous multi-agent systems using reinforcement learning (RL). Two algorithms are developed. The first algorithm solves a centralized linear quadratic regulator (LQR) problem without knowing any initial stabilizing gain in advance. The second algorithm builds upon the results of the first algorithm, and extends it to distribut…
▽ More
We address the problem of model-free distributed stabilization of heterogeneous multi-agent systems using reinforcement learning (RL). Two algorithms are developed. The first algorithm solves a centralized linear quadratic regulator (LQR) problem without knowing any initial stabilizing gain in advance. The second algorithm builds upon the results of the first algorithm, and extends it to distributed stabilization of multi-agent systems with predefined interaction graphs. Rigorous proofs are provided to show that the proposed algorithms achieve guaranteed convergence if specific conditions hold. A simulation example is presented to demonstrate the theoretical results.
△ Less
Submitted 7 March, 2021;
originally announced March 2021.
-
A method for determining the parameters in a rheological model for viscoelastic materials by minimizing Tikhonov functionals
Authors:
Rebecca Rothermel,
Wladimir Panfilenko,
Prateek Sharma,
Anne Wald,
Thomas Schuster,
Anne Jung,
Stefan Diebels
Abstract:
Mathematical models describing the behavior of viscoelastic materials are often based on evolution equations that measure the change in stress depending on its material parameters such as stiffness, viscosity or relaxation time. In this article, we introduce a Maxwell-based rheological model, define the associated forward operator and the inverse problem in order to determine the number of Maxwell…
▽ More
Mathematical models describing the behavior of viscoelastic materials are often based on evolution equations that measure the change in stress depending on its material parameters such as stiffness, viscosity or relaxation time. In this article, we introduce a Maxwell-based rheological model, define the associated forward operator and the inverse problem in order to determine the number of Maxwell elements and the material parameters of the underlying viscoelastic material. We perform a relaxation experiment by applying a strain to the material and measure the generated stress. Since the measured data varies with the number of Maxwell elements, the forward operator of the underlying inverse problem depends on parts of the solution. By introducing assumptions on the relaxation times, we propose a clustering algorithm to resolve this problem. We provide the calculations that are necessary for the minimization process and conclude with numerical results by investigating unperturbed as well as noisy data. We present different reconstruction approaches based on minimizing a least squares functional. Furthermore, we look at individual stress components to analyze different displacement rates. Finally, we study reconstructions with shortened data sets to obtain assertions on how long experiments have to be performed to identify conclusive material parameters.
△ Less
Submitted 26 February, 2021;
originally announced February 2021.
-
Zeroth-Order Hybrid Gradient Descent: Towards A Principled Black-Box Optimization Framework
Authors:
Pranay Sharma,
Kaidi Xu,
Sijia Liu,
Pin-Yu Chen,
Xue Lin,
Pramod K. Varshney
Abstract:
In this work, we focus on the study of stochastic zeroth-order (ZO) optimization which does not require first-order gradient information and uses only function evaluations. The problem of ZO optimization has emerged in many recent machine learning applications, where the gradient of the objective function is either unavailable or difficult to compute. In such cases, we can approximate the full gra…
▽ More
In this work, we focus on the study of stochastic zeroth-order (ZO) optimization which does not require first-order gradient information and uses only function evaluations. The problem of ZO optimization has emerged in many recent machine learning applications, where the gradient of the objective function is either unavailable or difficult to compute. In such cases, we can approximate the full gradients or stochastic gradients through function value based gradient estimates. Here, we propose a novel hybrid gradient estimator (HGE), which takes advantage of the query-efficiency of random gradient estimates as well as the variance-reduction of coordinate-wise gradient estimates. We show that with a graceful design in coordinate importance sampling, the proposed HGE-based ZO optimization method is efficient both in terms of iteration complexity as well as function query cost. We provide a thorough theoretical analysis of the convergence of our proposed method for non-convex, convex, and strongly-convex optimization. We show that the convergence rate that we derive generalizes the results for some prominent existing methods in the nonconvex case, and matches the optimal result in the convex case. We also corroborate the theory with a real-world black-box attack generation application to demonstrate the empirical advantage of our method over state-of-the-art ZO optimization approaches.
△ Less
Submitted 21 December, 2020;
originally announced December 2020.
-
$t$-aspect subconvexity for $GL(2) \times GL(2)$ $L$-function
Authors:
Ratnadeep Acharya,
Prahlad Sharma,
Saurabh Kumar Singh
Abstract:
In this paper we shall prove a subconvexity bound for $GL(2) \times GL(2)$ $L$-function in $t$-aspect by using a $GL(1)$ circle method.
In this paper we shall prove a subconvexity bound for $GL(2) \times GL(2)$ $L$-function in $t$-aspect by using a $GL(1)$ circle method.
△ Less
Submitted 2 November, 2020;
originally announced November 2020.
-
Subconvexity for $GL(3)\times GL(2)$ $L$-functions in $GL(3)$ spectral aspect
Authors:
Prahlad Sharma
Abstract:
Let $f$ be a $SL(2,\mathbb{Z})$ holomorphic cusp form or the Eisenstien series $E(z,1/2)$ and $π$ be a $SL(3,\mathbb{Z})$ Hecke-Maass cusp form with its Langlands parameter $μ$ in generic position i.e. away from Weyl chamber walls and away from self dual forms. We study an amplified second moment $\sum_{j} A(π_j)|L(1/2,π_j\times f)|^2$ and deduce the subconvexity bound \begin{equation*} L(1/2,π\ti…
▽ More
Let $f$ be a $SL(2,\mathbb{Z})$ holomorphic cusp form or the Eisenstien series $E(z,1/2)$ and $π$ be a $SL(3,\mathbb{Z})$ Hecke-Maass cusp form with its Langlands parameter $μ$ in generic position i.e. away from Weyl chamber walls and away from self dual forms. We study an amplified second moment $\sum_{j} A(π_j)|L(1/2,π_j\times f)|^2$ and deduce the subconvexity bound \begin{equation*} L(1/2,π\times f)\ll_{f,ε} \|μ\|^{3/2-1/2022+ε}. \end{equation*} As a corollary, when $f=E(z,1/2)$, we also obtain the subconvexity bound \begin{equation*} L(1/2,π)\ll_ε \|μ\|^{3/4-1/4044+ε}. \end{equation*}
△ Less
Submitted 22 June, 2022; v1 submitted 20 October, 2020;
originally announced October 2020.
-
Some Special Sets in an Exponential Vector Space
Authors:
Priti Sharma,
Sandip Jana
Abstract:
In this paper, we have studied 'absorbing' and 'balanced' sets in an Exponential Vector Space (\emph{evs} in short) over the field $\mathbb K$ of real or complex. These sets play pivotal role to describe several aspects of a topological evs. We have characterised a local base at the additive identity in terms of balanced and absorbing sets in a topological evs over the field $\mathbb K$. Also, we…
▽ More
In this paper, we have studied 'absorbing' and 'balanced' sets in an Exponential Vector Space (\emph{evs} in short) over the field $\mathbb K$ of real or complex. These sets play pivotal role to describe several aspects of a topological evs. We have characterised a local base at the additive identity in terms of balanced and absorbing sets in a topological evs over the field $\mathbb K$. Also, we have found a sufficient condition under which an evs can be topologised to form a topological evs. Next, we have introduced the concept of 'bounded sets' in a topological evs over the field $\mathbb K$ and characterised them with the help of balanced sets. Also we have shown that compactness implies boundedness of a set in a topological evs. In the last section we have introduced the concept of `radial' evs which characterises an evs over the field $\mathbb K$ up to order-isomorphism. Also, we have shown that every topological evs is radial. Further, it has been shown that "the usual subspace topology is the finest topology with respect to which $[0,\infty)$ forms a topological evs over the field $\mathbb K$".
△ Less
Submitted 5 June, 2020;
originally announced June 2020.
-
Fractal basins of attraction in a binary quasar model
Authors:
Vinay Kumar,
Pankaj Sharma,
Rajiv Aggarwal,
Bhavneet Kaur
Abstract:
The present paper investigates the binary system of quasars in the framework of the Circular Restricted Three-Body Problem. The parametric evolution of libration points, the geometry of zero-velocity curves are one of the crucial aspects of our study. The multivariate form of NR method is applied to study the basin of attraction connected with libration points. The algorithm for using the Newton-R…
▽ More
The present paper investigates the binary system of quasars in the framework of the Circular Restricted Three-Body Problem. The parametric evolution of libration points, the geometry of zero-velocity curves are one of the crucial aspects of our study. The multivariate form of NR method is applied to study the basin of attraction connected with libration points. The algorithm for using the Newton-Raphson method is slightly modified in order to avoid the unnecessary delay in the convergence of initial conditions. The impact of parameters on the shape of the basin of attraction and the number of iterations needed for the convergence of initial conditions are explored. We carry out an exhaustive (numerical) study to show the influence of these parameters on converging regions in basins of convergence. We unveil the existence of fractal structure in the basin of attraction using the method of basin entropy. In almost all cases, the existence of fractal structure is found throughout the basins of attraction.
△ Less
Submitted 11 May, 2020;
originally announced May 2020.
-
Distributed Stochastic Non-Convex Optimization: Momentum-Based Variance Reduction
Authors:
Prashant Khanduri,
Pranay Sharma,
Swatantra Kafle,
Saikiran Bulusu,
Ketan Rajawat,
Pramod K. Varshney
Abstract:
In this work, we propose a distributed algorithm for stochastic non-convex optimization. We consider a worker-server architecture where a set of $K$ worker nodes (WNs) in collaboration with a server node (SN) jointly aim to minimize a global, potentially non-convex objective function. The objective function is assumed to be the sum of local objective functions available at each WN, with each node…
▽ More
In this work, we propose a distributed algorithm for stochastic non-convex optimization. We consider a worker-server architecture where a set of $K$ worker nodes (WNs) in collaboration with a server node (SN) jointly aim to minimize a global, potentially non-convex objective function. The objective function is assumed to be the sum of local objective functions available at each WN, with each node having access to only the stochastic samples of its local objective function. In contrast to the existing approaches, we employ a momentum based "single loop" distributed algorithm which eliminates the need of computing large batch size gradients to achieve variance reduction. We propose two algorithms one with "adaptive" and the other with "non-adaptive" learning rates. We show that the proposed algorithms achieve the optimal computational complexity while attaining linear speedup with the number of WNs. Specifically, the algorithms reach an $ε$-stationary point $x_a$ with $\mathbb{E}\| \nabla f(x_a) \| \leq \tilde{O}(K^{-1/3}T^{-1/2} + K^{-1/3}T^{-1/3})$ in $T$ iterations, thereby requiring $\tilde{O}(K^{-1} ε^{-3})$ gradient computations at each WN. Moreover, our approach does not assume identical data distributions across WNs making the approach general enough for federated learning applications.
△ Less
Submitted 1 May, 2020;
originally announced May 2020.
-
On Distributed Online Convex Optimization with Sublinear Dynamic Regret and Fit
Authors:
Pranay Sharma,
Prashant Khanduri,
Lixin Shen,
Donald J. Bucci Jr.,
Pramod K. Varshney
Abstract:
In this work, we consider a distributed online convex optimization problem, with time-varying (potentially adversarial) constraints. A set of nodes, jointly aim to minimize a global objective function, which is the sum of local convex functions. The objective and constraint functions are revealed locally to the nodes, at each time, after taking an action. Naturally, the constraints cannot be insta…
▽ More
In this work, we consider a distributed online convex optimization problem, with time-varying (potentially adversarial) constraints. A set of nodes, jointly aim to minimize a global objective function, which is the sum of local convex functions. The objective and constraint functions are revealed locally to the nodes, at each time, after taking an action. Naturally, the constraints cannot be instantaneously satisfied. Therefore, we reformulate the problem to satisfy these constraints in the long term. To this end, we propose a distributed primal-dual mirror descent based approach, in which the primal and dual updates are carried out locally at all the nodes. This is followed by sharing and mixing of the primal variables by the local nodes via communication with the immediate neighbors. To quantify the performance of the proposed algorithm, we utilize the challenging, but more realistic metrics of dynamic regret and fit. Dynamic regret measures the cumulative loss incurred by the algorithm, compared to the best dynamic strategy. On the other hand, fit measures the long term cumulative constraint violations. Without assuming the restrictive Slater's conditions, we show that the proposed algorithm achieves sublinear regret and fit under mild, commonly used assumptions.
△ Less
Submitted 5 May, 2021; v1 submitted 9 January, 2020;
originally announced January 2020.
-
Parallel Restarted SPIDER -- Communication Efficient Distributed Nonconvex Optimization with Optimal Computation Complexity
Authors:
Pranay Sharma,
Swatantra Kafle,
Prashant Khanduri,
Saikiran Bulusu,
Ketan Rajawat,
Pramod K. Varshney
Abstract:
In this paper, we propose a distributed algorithm for stochastic smooth, non-convex optimization. We assume a worker-server architecture where $N$ nodes, each having $n$ (potentially infinite) number of samples, collaborate with the help of a central server to perform the optimization task. The global objective is to minimize the average of local cost functions available at individual nodes. The p…
▽ More
In this paper, we propose a distributed algorithm for stochastic smooth, non-convex optimization. We assume a worker-server architecture where $N$ nodes, each having $n$ (potentially infinite) number of samples, collaborate with the help of a central server to perform the optimization task. The global objective is to minimize the average of local cost functions available at individual nodes. The proposed approach is a non-trivial extension of the popular parallel-restarted SGD algorithm, incorporating the optimal variance-reduction based SPIDER gradient estimator into it. We prove convergence of our algorithm to a first-order stationary solution. The proposed approach achieves the best known communication complexity $O(ε^{-1})$ along with the optimal computation complexity. For finite-sum problems (finite $n$), we achieve the optimal computation (IFO) complexity $O(\sqrt{Nn}ε^{-1})$. For online problems ($n$ unknown or infinite), we achieve the optimal IFO complexity $O(ε^{-3/2})$. In both the cases, we maintain the linear speedup achieved by existing methods. This is a massive improvement over the $O(ε^{-2})$ IFO complexity of the existing approaches. Additionally, our algorithm is general enough to allow non-identical distributions of data across workers, as in the recently proposed federated learning paradigm.
△ Less
Submitted 6 November, 2020; v1 submitted 12 December, 2019;
originally announced December 2019.
-
Byzantine Resilient Non-Convex SVRG with Distributed Batch Gradient Computations
Authors:
Prashant Khanduri,
Saikiran Bulusu,
Pranay Sharma,
Pramod K. Varshney
Abstract:
In this work, we consider the distributed stochastic optimization problem of minimizing a non-convex function $f(x) = \mathbb{E}_{ξ\sim \mathcal{D}} f(x; ξ)$ in an adversarial setting, where the individual functions $f(x; ξ)$ can also be potentially non-convex. We assume that at most $α$-fraction of a total of $K$ nodes can be Byzantines. We propose a robust stochastic variance-reduced gradient (S…
▽ More
In this work, we consider the distributed stochastic optimization problem of minimizing a non-convex function $f(x) = \mathbb{E}_{ξ\sim \mathcal{D}} f(x; ξ)$ in an adversarial setting, where the individual functions $f(x; ξ)$ can also be potentially non-convex. We assume that at most $α$-fraction of a total of $K$ nodes can be Byzantines. We propose a robust stochastic variance-reduced gradient (SVRG) like algorithm for the problem, where the batch gradients are computed at the worker nodes (WNs) and the stochastic gradients are computed at the server node (SN). For the non-convex optimization problem, we show that we need $\tilde{O}\left( \frac{1}{ε^{5/3} K^{2/3}} + \frac{α^{4/3}}{ε^{5/3}} \right)$ gradient computations on average at each node (SN and WNs) to reach an $ε$-stationary point. The proposed algorithm guarantees convergence via the design of a novel Byzantine filtering rule which is independent of the problem dimension. Importantly, we capture the effect of the fraction of Byzantine nodes $α$ present in the network on the convergence performance of the algorithm.
△ Less
Submitted 10 December, 2019;
originally announced December 2019.
-
Inductive algebras for the affine group of a finite field
Authors:
Promod Sharma,
M. K. Vemuri
Abstract:
Each irreducible representation of the affine group of a finite field has a unique maximal inductive algebra, and it is self adjoint.
Each irreducible representation of the affine group of a finite field has a unique maximal inductive algebra, and it is self adjoint.
△ Less
Submitted 26 July, 2019;
originally announced July 2019.
-
Minimal-norm static feedbacks using dissipative Hamiltonian matrices
Authors:
Nicolas Gillis,
Punit Sharma
Abstract:
In this paper, we characterize the set of static-state feedbacks that stabilize a given continuous linear-time invariant system pair using dissipative Hamiltonian matrices. This characterization results in a parametrization of feedbacks in terms of skew-symmetric and symmetric positive semidefinite matrices, and leads to a semidefinite program that computes a static-state stabilizing feedback. Thi…
▽ More
In this paper, we characterize the set of static-state feedbacks that stabilize a given continuous linear-time invariant system pair using dissipative Hamiltonian matrices. This characterization results in a parametrization of feedbacks in terms of skew-symmetric and symmetric positive semidefinite matrices, and leads to a semidefinite program that computes a static-state stabilizing feedback. This characterization also allows us to propose an algorithm that computes minimal-norm static feedbacks. The theoretical results extend to the static-output feedback (SOF) problem, and we also propose an algorithm to tackle this problem. We illustrate the effectiveness of our algorithm compared to state-of-the-art methods for the SOF problem on numerous numerical examples from the COMPLeIB library.
△ Less
Submitted 16 July, 2019;
originally announced July 2019.
-
Subconvexity for $GL(3)\times GL(2)$ twists (with an appendix by Will Sawin)
Authors:
Prahlad Sharma
Abstract:
Let $π$ be a $SL(3,\mathbb{Z})$ Hecke-Maass cusp form, $f$ be a $SL(2,\mathbb{Z})$ holomorphic cusp form or Maass cusp form and $χ$ be any non-trivial character $\bmod \, p$, where $p$ is prime. We show that the $L$-function associated with this triplet satisfy \begin{equation*} L\left(\frac{1}{2},π\times f\timesχ\right)\ll_{π,f,ε} p^{\frac{3}{2}-\frac{1}{16}+ε}. \end{equation*} The method also yi…
▽ More
Let $π$ be a $SL(3,\mathbb{Z})$ Hecke-Maass cusp form, $f$ be a $SL(2,\mathbb{Z})$ holomorphic cusp form or Maass cusp form and $χ$ be any non-trivial character $\bmod \, p$, where $p$ is prime. We show that the $L$-function associated with this triplet satisfy \begin{equation*} L\left(\frac{1}{2},π\times f\timesχ\right)\ll_{π,f,ε} p^{\frac{3}{2}-\frac{1}{16}+ε}. \end{equation*} The method also yields the subconvex bound \begin{equation*} L\left(\frac{1}{2},π\otimes χ\right)\ll_{π,ε}p^{\frac{3}{4}-\frac{1}{32}+ε}. \end{equation*}
△ Less
Submitted 10 May, 2022; v1 submitted 22 June, 2019;
originally announced June 2019.
-
Powers Vs. Powers
Authors:
Pramod K Sharma
Abstract:
Let $ A \subset B$ be rings. An ideal $ J \subset B$ is called power stable in $A$ if $ J^n \cap A = (J\cap A)^n$ for all $ n\geq 1$. Further, $J$ is called ultimately power stable in $A$ if $ J^n \cap A = (J\cap A)^n$ for all $n$ large i.e., $ n \gg 0$. In this note, our focus is to study these concepts for pair of rings $ R \subset R[X]$ where $R$ is an integral domain. Some of the results we pr…
▽ More
Let $ A \subset B$ be rings. An ideal $ J \subset B$ is called power stable in $A$ if $ J^n \cap A = (J\cap A)^n$ for all $ n\geq 1$. Further, $J$ is called ultimately power stable in $A$ if $ J^n \cap A = (J\cap A)^n$ for all $n$ large i.e., $ n \gg 0$. In this note, our focus is to study these concepts for pair of rings $ R \subset R[X]$ where $R$ is an integral domain. Some of the results we prove are: A maximal ideal $\textbf{m}$ in $R[X]$ is power stable in $R$ if and only if $ \wp^t $ is $ \wp-$primary for all $ t \geq 1$ for the prime ideal $\wp = \textbf{m}\cap R$. We use this to prove that for a Hilbert domain $R$, any radical ideal in $R[X]$ which is a finite intersection of G-ideals is power stable in $R$. Further, we prove that if $R$ is a Noetherian integral domain of dimension 1 then any radical ideal in $R[X] $ is power stable in $R$, and if every ideal in $R[X]$ is power stable in $R$ then $R$ is a field. We also show that if $ A \subset B$ are Noetherian rings, and $ I $ is an ideal in $B$ which is ultimately power stable in $A$, then if $ I \cap A = J$ is a radical ideal generated by a regular $A$-sequence, it is power stable. Finally, we give a relationship in power stability and ultimate power stability using the concept of reduction of an ideal (Theorem 3.22).
△ Less
Submitted 26 March, 2019;
originally announced March 2019.
-
Ideal containment vs. powers
Authors:
Pramod K. Sharma
Abstract:
Let $R$ be a commutative ring with identity. In this note, we study the property: If $ I \subsetneqq J$ are ideals in $R$, then $ I^n \subsetneqq J^n$ for all $ n\geq 1$. We define the notion of a big ideal (Definition 1.2). It is noted that the property has close relationship with the notions of reduction of an ideal and Ratliff-Rush ideal [7]. Apart from other results, it is proved that a Noethe…
▽ More
Let $R$ be a commutative ring with identity. In this note, we study the property: If $ I \subsetneqq J$ are ideals in $R$, then $ I^n \subsetneqq J^n$ for all $ n\geq 1$. We define the notion of a big ideal (Definition 1.2). It is noted that the property has close relationship with the notions of reduction of an ideal and Ratliff-Rush ideal [7]. Apart from other results, it is proved that a Noetherian domain satifies the property if and only if every ideal in $R$ is a Ratliff-Rush ideal. We also prove that ideals having no proper reduction are big ideals, and maximal ideals in regular rings are big.
△ Less
Submitted 26 March, 2019;
originally announced March 2019.
-
Multidimensional Shifts And Finite Matrices
Authors:
Puneet Sharma,
Dileep Kumar
Abstract:
Let $X$ be a $2$-dimensional subshift of finite type generated by a finite set of forbidden blocks (of finite size). We give an algorithm for generating the elements of the shift space using sequence of finite matrices (of increasing size). We prove that the sequence generated yields precisely the elements of the shift space $X$ and hence characterizes the elements of the shift space $X$. We exten…
▽ More
Let $X$ be a $2$-dimensional subshift of finite type generated by a finite set of forbidden blocks (of finite size). We give an algorithm for generating the elements of the shift space using sequence of finite matrices (of increasing size). We prove that the sequence generated yields precisely the elements of the shift space $X$ and hence characterizes the elements of the shift space $X$. We extend our investigations to a general $d$-dimensional shift of finite type. In the process, we prove that that elements of $d$-dimensional shift of finite type can be characterized by a sequence of finite matrices (of increasing size).
△ Less
Submitted 5 February, 2019; v1 submitted 21 January, 2019;
originally announced January 2019.
-
On approximating the nearest Ω-stable matrix
Authors:
Neelam Choudhary,
Nicolas Gillis,
Punit Sharma
Abstract:
In this paper, we consider the problem of approximating a given matrix with a matrix whose eigenvalues lie in some specific region Ω, within the complex plane. More precisely, we consider three types of regions and their intersections: conic sectors, vertical strips and disks. We refer to this problem as the nearest Ω-stable matrix problem. This includes as special cases the stable matrices for co…
▽ More
In this paper, we consider the problem of approximating a given matrix with a matrix whose eigenvalues lie in some specific region Ω, within the complex plane. More precisely, we consider three types of regions and their intersections: conic sectors, vertical strips and disks. We refer to this problem as the nearest Ω-stable matrix problem. This includes as special cases the stable matrices for continuous and discrete time linear time-invariant systems. In order to achieve this goal, we parametrize this problem using dissipative Hamiltonian matrices and linear matrix inequalities. This leads to a reformulation of the problem with a convex feasible set. By applying a block coordinate descent method on this reformulation, we are able to compute solutions to the approximation problem, which is illustrated on some examples.
△ Less
Submitted 10 January, 2019;
originally announced January 2019.
-
Dynamics Of Finitely Generated Non-Autonomous Systems
Authors:
Manish Raghav,
Puneet Sharma
Abstract:
In this paper, we discuss dynamical behavior of a non-autonomous system generated by a finite family $\mathbb{F}$. In the process, we relate the dynamical behavior of the non-autonomous system generated by the family $\mathbb{F}=\{f_1,f_2,\ldots,f_k\}$ with the dynamical behavior of the system $(X,f_k\circ f_{k-1}\circ\ldots\circ f_1)$. We discuss properties like minimality, equicontinuity, proxim…
▽ More
In this paper, we discuss dynamical behavior of a non-autonomous system generated by a finite family $\mathbb{F}$. In the process, we relate the dynamical behavior of the non-autonomous system generated by the family $\mathbb{F}=\{f_1,f_2,\ldots,f_k\}$ with the dynamical behavior of the system $(X,f_k\circ f_{k-1}\circ\ldots\circ f_1)$. We discuss properties like minimality, equicontinuity, proximality and various forms of sensitivities for the two systems. We derive conditions under which the dynamical behavior of $(X,f_k\circ f_{k-1}\circ\ldots\circ f_1)$ is carried forward to $(X,\mathbb{F})$ (and vice-versa). We also give examples to illustrate the necessity of the conditions imposed.
△ Less
Submitted 2 October, 2018;
originally announced October 2018.
-
A note on approximating the nearest stable discrete-time descriptor system with fixed rank
Authors:
Nicolas Gillis,
Michael Karow,
Punit Sharma
Abstract:
Consider a discrete-time linear time-invariant descriptor system $Ex(k+1)=Ax(k)$ for $k \in \mathbb Z_{+}$. In this paper, we tackle for the first time the problem of stabilizing such systems by computing a nearby regular index one stable system $\hat E x(k+1)= \hat A x(k)$ with $\text{rank}(\hat E)=r$. We reformulate this highly nonconvex problem into an equivalent optimization problem with a rel…
▽ More
Consider a discrete-time linear time-invariant descriptor system $Ex(k+1)=Ax(k)$ for $k \in \mathbb Z_{+}$. In this paper, we tackle for the first time the problem of stabilizing such systems by computing a nearby regular index one stable system $\hat E x(k+1)= \hat A x(k)$ with $\text{rank}(\hat E)=r$. We reformulate this highly nonconvex problem into an equivalent optimization problem with a relatively simple feasible set onto which it is easy to project. This allows us to employ a block coordinate descent method to obtain a nearby regular index one stable system. We illustrate the effectiveness of the algorithm on several examples.
△ Less
Submitted 12 July, 2018;
originally announced July 2018.
