-
Cryptography-Based Privacy-Preserving Method for Distributed Optimization over Time-Varying Directed Graphs with Enhanced Efficiency
Authors:
Bing Liu,
Furan Xie,
Li Chai
Abstract:
In this paper, we study the privacy-preserving distributed optimization problem, aiming to prevent attackers from stealing the private information of agents. For this purpose, we propose a novel privacy-preserving algorithm based on the Advanced Encryption Standard (AES), which is both secure and computationally efficient. By appropriately constructing the underlying weight matrices, our algorithm…
▽ More
In this paper, we study the privacy-preserving distributed optimization problem, aiming to prevent attackers from stealing the private information of agents. For this purpose, we propose a novel privacy-preserving algorithm based on the Advanced Encryption Standard (AES), which is both secure and computationally efficient. By appropriately constructing the underlying weight matrices, our algorithm can be applied to time-varying directed networks. We show that the proposed algorithm can protect an agent's privacy if the agent has at least one legitimate neighbor at the initial iteration. Under the assumption that the objective function is strongly convex and Lipschitz smooth, we rigorously prove that the proposed algorithm has a linear convergence rate. Finally, the effectiveness of the proposed algorithm is demonstrated by numerical simulations of the canonical sensor fusion problem.
△ Less
Submitted 14 May, 2024;
originally announced May 2024.
-
Frozen Gaussian approximation for the fractional Schrödinger equation
Authors:
Lihui Chai,
Hengzhun Chen,
Xu Yang
Abstract:
We develop the frozen Gaussian approximation (FGA) for the fractional Schrödinger equation in the semi-classical regime, where the solution is highly oscillatory when the scaled Planck constant $\varepsilon$ is small. This method approximates the solution to the Schrödinger equation by an integral representation based on asymptotic analysis and provides a highly efficient computational method for…
▽ More
We develop the frozen Gaussian approximation (FGA) for the fractional Schrödinger equation in the semi-classical regime, where the solution is highly oscillatory when the scaled Planck constant $\varepsilon$ is small. This method approximates the solution to the Schrödinger equation by an integral representation based on asymptotic analysis and provides a highly efficient computational method for high-frequency wave function evolution. In particular, we revise the standard FGA formula to address the singularities arising in the higher-order derivatives of coefficients of the associated Hamiltonian flow that are second-order continuously differentiable or smooth in conventional FGA analysis. We then establish its convergence to the true solution. Additionally, we provide some numerical examples to verify the accuracy and convergence behavior of the frozen Gaussian approximation method.
△ Less
Submitted 27 March, 2024;
originally announced March 2024.
-
Distributed Stochastic Bilevel Optimization: Improved Complexity and Heterogeneity Analysis
Authors:
Youcheng Niu,
**ming Xu,
Ying Sun,
Yan Huang,
Li Chai
Abstract:
This paper consider solving a class of nonconvex-strongly-convex distributed stochastic bilevel optimization (DSBO) problems with personalized inner-level objectives. Most existing algorithms require computational loops for hypergradient estimation, leading to computational inefficiency. Moreover, the impact of data heterogeneity on convergence in bilevel problems is not explicitly characterized y…
▽ More
This paper consider solving a class of nonconvex-strongly-convex distributed stochastic bilevel optimization (DSBO) problems with personalized inner-level objectives. Most existing algorithms require computational loops for hypergradient estimation, leading to computational inefficiency. Moreover, the impact of data heterogeneity on convergence in bilevel problems is not explicitly characterized yet. To address these issues, we propose LoPA, a loopless personalized distributed algorithm that leverages a tracking mechanism for iterative approximation of inner-level solutions and Hessian-inverse matrices without relying on extra computation loops. Our theoretical analysis explicitly characterizes the heterogeneity across nodes denoted by $b$, and establishes a sublinear rate of ${\mathcal{O}}( {\frac{1}{(1-ρ)^2K} + \frac{{b^{\frac{2}{3}} }}{{\left( {1 - ρ} \right)^{\frac{2}{3}}K^{\frac{2}{3}} }} + \frac{1}{\sqrt{ K }} ( σ_{\rm{p}} + \frac{1}{\sqrt{m}}σ_{\rm{c}}) } )$ without the boundedness of local hypergradients, where $σ_{\rm p}$ and $σ_{\rm c}$ represent the gradient sampling variances associated with the inner- and outer-level variables, respectively. We also develop a variant of LoPA based on gradient tracking to eliminate the impact of data heterogeneity, yielding an improved rate of ${\mathcal{O}}(\frac{1}{ (1-ρ)^4K } + \frac{1}{\sqrt{K}}( σ_{\rm{p}} + \frac{1}{\sqrt{m}}σ_{\rm{c}} ) )$. The computational complexity of LoPA is of ${\mathcal{O}}({ε^{-2}})$ to an $ε$-stationary point, matching the communication complexity due to the loopless structure, which outperforms existing counterparts for DSBO. Numerical experiments validate the effectiveness of the proposed algorithm.
△ Less
Submitted 8 February, 2024; v1 submitted 22 December, 2023;
originally announced December 2023.
-
Accelerating the Convergence Rate of Consensus for Second-Order Multi-Agent Systems by Memory Information
Authors:
Jiahao Dai,
**g-Wen Yi,
Li Chai
Abstract:
This paper utilizes the agent's memory in accelerated consensus for second-order multi-agent systems (MASs). In the case of one-tap memory, explicit formulas for the optimal consensus convergence rate and control parameters are derived by applying the Jury stability criterion. It is proved that the optimal consensus convergence rate with one-tap memory is faster than that without memory. In the ca…
▽ More
This paper utilizes the agent's memory in accelerated consensus for second-order multi-agent systems (MASs). In the case of one-tap memory, explicit formulas for the optimal consensus convergence rate and control parameters are derived by applying the Jury stability criterion. It is proved that the optimal consensus convergence rate with one-tap memory is faster than that without memory. In the case of M-tap memory, an iterative algorithm is given to derive the control parameters to accelerate the convergence rate. Moreover, the accelerated consensus with one-tap memory is extended to the formation control, and the control parameters to achieve the fastest formation are obtained. Numerical examples further illustrate the theoretical results.
△ Less
Submitted 24 March, 2023;
originally announced March 2023.
-
Frozen Gaussian Sampling for Scalar Wave Equations
Authors:
Lihui Chai,
Ye Feng,
Zhennan Zhou
Abstract:
In this article, we introduce the frozen Gaussian sampling (FGS) algorithm to solve the scalar wave equation in the high-frequency regime. The FGS algorithm is a Monte Carlo sampling strategy based on the frozen Gaussian approximation, which greatly reduces the computation workload in the wave propagation and reconstruction. In this work, we propose feasible and detailed procedures to implement th…
▽ More
In this article, we introduce the frozen Gaussian sampling (FGS) algorithm to solve the scalar wave equation in the high-frequency regime. The FGS algorithm is a Monte Carlo sampling strategy based on the frozen Gaussian approximation, which greatly reduces the computation workload in the wave propagation and reconstruction. In this work, we propose feasible and detailed procedures to implement the FGS algorithm to approximate scalar wave equations with Gaussian initial conditions and WKB initial conditions respectively. For both initial data cases, we rigorously analyze the error of applying this algorithm to wave equations of dimensionality $d \geq 3$. In Gaussian initial data cases, we prove that the sampling error due to the Monte Carlo method is independent of the typical wave number. We also derive a quantitative bound of the sampling error in WKB initial data cases. Finally, we validate the performance of the FGS and the theoretical estimates about the sampling error through various numerical examples, which include using the FGS to solve wave equations with both Gaussian and WKB initial data of dimensionality $d = 1, 2$, and $3$.
△ Less
Submitted 2 September, 2023; v1 submitted 9 November, 2022;
originally announced November 2022.
-
Fast consensus of high-order multi-agent systems
Authors:
Jiahao Dai,
**g-Wen Yi,
Li Chai
Abstract:
In this paper, the fast consensus problem of high-order multi-agent systems under undirected topologies is considered. The direct link between the consensus convergence rate and the control gains is established. An accelerated consensus algorithm based on gradient descent is proposed to optimize the convergence rate. By applying the Routh-Hurwitz stability criterion, the lower bound on the converg…
▽ More
In this paper, the fast consensus problem of high-order multi-agent systems under undirected topologies is considered. The direct link between the consensus convergence rate and the control gains is established. An accelerated consensus algorithm based on gradient descent is proposed to optimize the convergence rate. By applying the Routh-Hurwitz stability criterion, the lower bound on the convergence rate is derived, and explicit control gains are derived as the necessary condition to achieve the optimal convergence rate. Moreover, a protocol with time-varying control gains is designed to achieve the finite-time consensus. Explicit formulas for the time-varying control gains and the final consensus state are given. Numerical examples and simulation results are presented to illustrate the obtained theoretical results.
△ Less
Submitted 16 May, 2022;
originally announced May 2022.
-
Optimal Memory Scheme for Accelerated Consensus Over Multi-Agent Networks
Authors:
Jiahao Dai,
**g-Wen Yi,
Li Chai
Abstract:
The consensus over multi-agent networks can be accelerated by introducing agent's memory to the control protocol. In this paper, a more general protocol with the node memory and the state deviation memory is designed. We aim to provide the optimal memory scheme to accelerate consensus. The contributions of this paper are three: (i) For the one-tap memory scheme, we demonstrate that the state devia…
▽ More
The consensus over multi-agent networks can be accelerated by introducing agent's memory to the control protocol. In this paper, a more general protocol with the node memory and the state deviation memory is designed. We aim to provide the optimal memory scheme to accelerate consensus. The contributions of this paper are three: (i) For the one-tap memory scheme, we demonstrate that the state deviation memory is useless for the optimal convergence. (ii) In the worst case, we prove that it is a vain to add any tap of the state deviation memory, and the one-tap node memory is sufficient to achieve the optimal convergence. (iii) We show that the two-tap state deviation memory is effective on some special networks, such as star networks. Numerical examples are listed to illustrate the validity and correctness of the obtained results.
△ Less
Submitted 13 December, 2021;
originally announced December 2021.
-
Convergence Rate of Accelerated Average Consensus with Local Node Memory: Optimization and Analytic Solutions
Authors:
**g-Wen Yi,
Li Chai,
**gxin Zhang
Abstract:
Previous researches have shown that adding local memory can accelerate the consensus. It is natural to ask questions like what is the fastest rate achievable by the $M$-tap memory acceleration, and what are the corresponding control parameters. This paper introduces a set of effective and previously unused techniques to analyze the convergence rate of accelerated consensus with $M$-tap memory of l…
▽ More
Previous researches have shown that adding local memory can accelerate the consensus. It is natural to ask questions like what is the fastest rate achievable by the $M$-tap memory acceleration, and what are the corresponding control parameters. This paper introduces a set of effective and previously unused techniques to analyze the convergence rate of accelerated consensus with $M$-tap memory of local nodes and to design the control protocols. These effective techniques, including the Kharitonov stability theorem, the Routh stability criterion and the robust stability margin, have led to the following new results: 1) the direct link between the convergence rate and the control parameters; 2) explicit formulas of the optimal convergence rate and the corresponding optimal control parameters for $M \leq 2$ on a given graph; 3) the optimal worst-case convergence rate and the corresponding optimal control parameters for the memory $M \geq 1$ on a set of uncertain graphs. We show that the acceleration with the memory $M = 1$ provides the optimal convergence rate in the sense of worst-case performance. Several numerical examples are given to demonstrate the validity and performance of the theoretical results.
△ Less
Submitted 10 December, 2021; v1 submitted 18 October, 2021;
originally announced October 2021.
-
Seismic Tomography with Random Batch Gradient Reconstruction
Authors:
Yixiao Hu,
Lihui Chai,
Zhongyi Huang,
Xu Yang
Abstract:
Seismic tomography solves high-dimensional optimization problems to image subsurface structures of Earth. In this paper, we propose to use random batch methods to construct the gradient used for iterations in seismic tomography. Specifically, we use the frozen Gaussian approximation to compute seismic wave propagation, and then construct stochastic gradients by random batch methods. The method inh…
▽ More
Seismic tomography solves high-dimensional optimization problems to image subsurface structures of Earth. In this paper, we propose to use random batch methods to construct the gradient used for iterations in seismic tomography. Specifically, we use the frozen Gaussian approximation to compute seismic wave propagation, and then construct stochastic gradients by random batch methods. The method inherits the spirit of stochastic gradient descent methods for solving high-dimensional optimization problems. The proposed idea is general in the sense that it does not rely on the usage of the frozen Gaussian approximation, and one can replace it with any other efficient wave propagation solvers, e.g., Gaussian beam methods and spectral element methods. We prove the convergence of the random batch method in the mean-square sense, and show the numerical performance of the proposed method by two-dimensional and three-dimensional examples of wave-equation-based travel-time inversion and full-waveform inversion, respectively. As a byproduct, we also prove the convergence of the accelerated full-waveform inversion using dynamic mini-batches and spectral element methods.
△ Less
Submitted 11 February, 2023; v1 submitted 12 October, 2021;
originally announced October 2021.
-
Opinion Dynamics Models with Memory in Coopetitive Social Networks: Analysis, Application and Simulation
Authors:
Qingsong Liu,
Li Chai
Abstract:
In some social networks, the opinion forming is based on its own and neighbors' (initial) opinions, whereas the evolution of the individual opinions is also influenced by the individual's past opinions in the real world. Unlike existing social network models, in this paper, a novel model of opinion dynamics is proposed, which describes the evolution of the individuals' opinions not only depends on…
▽ More
In some social networks, the opinion forming is based on its own and neighbors' (initial) opinions, whereas the evolution of the individual opinions is also influenced by the individual's past opinions in the real world. Unlike existing social network models, in this paper, a novel model of opinion dynamics is proposed, which describes the evolution of the individuals' opinions not only depends on its own and neighbors' current opinions, but also depends on past opinions. Memory and memoryless communication rules are simultaneously established for the proposed opinion dynamics model. Sufficient and/or necessary conditions for the equal polarization, consensus and neutralizability of the opinions are respectively presented in terms of the network topological structure and the spectral analysis. We apply our model to simulate Kahneman's seminal experiments on choices in risky and riskless contexts, which fits in with the experiment results. Simulation analysis shows that the memory capacity of the individuals is inversely proportional to the speeds of the ultimate opinions formational.
△ Less
Submitted 5 August, 2021;
originally announced August 2021.
-
A multi-band semiclassical model for surface hop** quantum dynamics
Authors:
Lihui Chai,
Shi **,
Qin Li,
Omar Morandi
Abstract:
In the paper we derive a semiclassical model for surface hop** allowing quantum dynamical non-adiabatic transition between different potential energy surfaces in which cases the classical Born-Oppenheimer approximation breaks down. The model is derived using the Wigner transform and Weyl quantization, and the central idea is to evolve the entire Wigner matrix rather than just the diagonal entrie…
▽ More
In the paper we derive a semiclassical model for surface hop** allowing quantum dynamical non-adiabatic transition between different potential energy surfaces in which cases the classical Born-Oppenheimer approximation breaks down. The model is derived using the Wigner transform and Weyl quantization, and the central idea is to evolve the entire Wigner matrix rather than just the diagonal entries as was done previously in the adiabatic case. The off-diagonal entries of the Wigner matrix suitably describe the non-adiabatic transition, such as the Berry connection, for avoided crossings. We study the numerical approximation issues of the model, and then conduct numerical experiments to validate the model.
△ Less
Submitted 4 May, 2014;
originally announced May 2014.
