Showing 1–25 of 25 results for author: Koskela, A

Differentially Private Convex Approximation of Two-Layer ReLU Networks

Authors: Antti Koskela

Abstract: We show that it is possible to privately train convex problems that give models with similar privacy-utility trade-off as one hidden-layer ReLU networks trained with differentially private stochastic gradient descent (DP-SGD). As we show, this is possible via a certain dual formulation of the ReLU minimization problem. We derive a stochastic approximation of the dual problem that leads to a strong… ▽ More We show that it is possible to privately train convex problems that give models with similar privacy-utility trade-off as one hidden-layer ReLU networks trained with differentially private stochastic gradient descent (DP-SGD). As we show, this is possible via a certain dual formulation of the ReLU minimization problem. We derive a stochastic approximation of the dual problem that leads to a strongly convex problem which allows applying, for example, the privacy amplification by iteration type of analysis for gradient-based private optimizers, and in particular allows giving accurate privacy bounds for the noisy cyclic mini-batch gradient descent with fixed disjoint mini-batches. We obtain on the MNIST and FashionMNIST problems for the noisy cyclic mini-batch gradient descent first empirical results that show similar privacy-utility-trade-offs as DP-SGD applied to a ReLU network. We outline theoretical utility bounds that illustrate the speed-ups of the private convex approximation of ReLU networks. △ Less

Submitted 5 July, 2024; originally announced July 2024.

Black Box Differential Privacy Auditing Using Total Variation Distance

Authors: Antti Koskela, Jafar Mohammadi

Abstract: We present a practical method to audit the differential privacy (DP) guarantees of a machine learning model using a small hold-out dataset that is not exposed to the model during the training. Having a score function such as the loss function employed during the training, our method estimates the total variation (TV) distance between scores obtained with a subset of the training data and the hold-… ▽ More We present a practical method to audit the differential privacy (DP) guarantees of a machine learning model using a small hold-out dataset that is not exposed to the model during the training. Having a score function such as the loss function employed during the training, our method estimates the total variation (TV) distance between scores obtained with a subset of the training data and the hold-out dataset. With some meta information about the underlying DP training algorithm, these TV distance values can be converted to $(\varepsilon,δ)$-guarantees for any $δ$. We show that these score distributions asymptotically give lower bounds for the DP guarantees of the underlying training algorithm, however, we perform a one-shot estimation for practicality reasons. We specify conditions that lead to lower bounds for the DP guarantees with high probability. To estimate the TV distance between the score distributions, we use a simple density estimation method based on histograms. We show that the TV distance gives a very close to optimally robust estimator and has an error rate $\mathcal{O}(k^{-1/3})$, where $k$ is the total number of samples. Numerical experiments on benchmark datasets illustrate the effectiveness of our approach and show improvements over baseline methods for black-box auditing. △ Less

Submitted 5 July, 2024; v1 submitted 7 June, 2024; originally announced June 2024.

Privacy Profiles for Private Selection

Authors: Antti Koskela, Rachel Redberg, Yu-Xiang Wang

Abstract: Private selection mechanisms (e.g., Report Noisy Max, Sparse Vector) are fundamental primitives of differentially private (DP) data analysis with wide applications to private query release, voting, and hyperparameter tuning. Recent work (Liu and Talwar, 2019; Papernot and Steinke, 2022) has made significant progress in both generalizing private selection mechanisms and tightening their privacy ana… ▽ More Private selection mechanisms (e.g., Report Noisy Max, Sparse Vector) are fundamental primitives of differentially private (DP) data analysis with wide applications to private query release, voting, and hyperparameter tuning. Recent work (Liu and Talwar, 2019; Papernot and Steinke, 2022) has made significant progress in both generalizing private selection mechanisms and tightening their privacy analysis using modern numerical privacy accounting tools, e.g., Rényi DP. But Rényi DP is known to be lossy when $(ε,δ)$-DP is ultimately needed, and there is a trend to close the gap by directly handling privacy profiles, i.e., $δ$ as a function of $ε$ or its equivalent dual form known as $f$-DPs. In this paper, we work out an easy-to-use recipe that bounds the privacy profiles of ReportNoisyMax and PrivateTuning using the privacy profiles of the base algorithms they corral. Numerically, our approach improves over the RDP-based accounting in all regimes of interest and leads to substantial benefits in end-to-end private learning experiments. Our analysis also suggests new distributions, e.g., binomial distribution for randomizing the number of rounds that leads to more substantial improvements in certain regimes. △ Less

Submitted 9 February, 2024; originally announced February 2024.

Improving the Privacy and Practicality of Objective Perturbation for Differentially Private Linear Learners

Authors: Rachel Redberg, Antti Koskela, Yu-Xiang Wang

Abstract: In the arena of privacy-preserving machine learning, differentially private stochastic gradient descent (DP-SGD) has outstripped the objective perturbation mechanism in popularity and interest. Though unrivaled in versatility, DP-SGD requires a non-trivial privacy overhead (for privately tuning the model's hyperparameters) and a computational complexity which might be extravagant for simple models… ▽ More In the arena of privacy-preserving machine learning, differentially private stochastic gradient descent (DP-SGD) has outstripped the objective perturbation mechanism in popularity and interest. Though unrivaled in versatility, DP-SGD requires a non-trivial privacy overhead (for privately tuning the model's hyperparameters) and a computational complexity which might be extravagant for simple models such as linear and logistic regression. This paper revamps the objective perturbation mechanism with tighter privacy analyses and new computational tools that boost it to perform competitively with DP-SGD on unconstrained convex generalized linear problems. △ Less

Submitted 31 December, 2023; originally announced January 2024.

Practical Differentially Private Hyperparameter Tuning with Subsampling

Authors: Antti Koskela, Tejas Kulkarni

Abstract: Tuning the hyperparameters of differentially private (DP) machine learning (ML) algorithms often requires use of sensitive data and this may leak private information via hyperparameter values. Recently, Papernot and Steinke (2022) proposed a certain class of DP hyperparameter tuning algorithms, where the number of random search samples is randomized itself. Commonly, these algorithms still conside… ▽ More Tuning the hyperparameters of differentially private (DP) machine learning (ML) algorithms often requires use of sensitive data and this may leak private information via hyperparameter values. Recently, Papernot and Steinke (2022) proposed a certain class of DP hyperparameter tuning algorithms, where the number of random search samples is randomized itself. Commonly, these algorithms still considerably increase the DP privacy parameter $\varepsilon$ over non-tuned DP ML model training and can be computationally heavy as evaluating each hyperparameter candidate requires a new training run. We focus on lowering both the DP bounds and the computational cost of these methods by using only a random subset of the sensitive data for the hyperparameter tuning and by extrapolating the optimal values to a larger dataset. We provide a Rényi differential privacy analysis for the proposed method and experimentally show that it consistently leads to better privacy-utility trade-off than the baseline method by Papernot and Steinke. △ Less

Submitted 13 February, 2024; v1 submitted 27 January, 2023; originally announced January 2023.

Journal ref: NeurIPS 2023

Individual Privacy Accounting with Gaussian Differential Privacy

Authors: Antti Koskela, Marlon Tobaben, Antti Honkela

Abstract: Individual privacy accounting enables bounding differential privacy (DP) loss individually for each participant involved in the analysis. This can be informative as often the individual privacy losses are considerably smaller than those indicated by the DP bounds that are based on considering worst-case bounds at each data access. In order to account for the individual privacy losses in a principl… ▽ More Individual privacy accounting enables bounding differential privacy (DP) loss individually for each participant involved in the analysis. This can be informative as often the individual privacy losses are considerably smaller than those indicated by the DP bounds that are based on considering worst-case bounds at each data access. In order to account for the individual privacy losses in a principled manner, we need a privacy accountant for adaptive compositions of randomised mechanisms, where the loss incurred at a given data access is allowed to be smaller than the worst-case loss. This kind of analysis has been carried out for the Rényi differential privacy (RDP) by Feldman and Zrnic (2021), however not yet for the so-called optimal privacy accountants. We make first steps in this direction by providing a careful analysis using the Gaussian differential privacy which gives optimal bounds for the Gaussian mechanism, one of the most versatile DP mechanisms. This approach is based on determining a certain supermartingale for the hockey-stick divergence and on extending the Rényi divergence-based fully adaptive composition results by Feldman and Zrnic. We also consider measuring the individual $(\varepsilon,δ)$-privacy losses using the so-called privacy loss distributions. With the help of the Blackwell theorem, we can then make use of the RDP analysis to construct an approximative individual $(\varepsilon,δ)$-accountant. △ Less

Submitted 24 August, 2023; v1 submitted 30 September, 2022; originally announced September 2022.

Comments: 31 pages, 10 figures

Journal ref: International Conference on Learning Representations 2023

Differentially Private Hamiltonian Monte Carlo

Authors: Ossi Räisä, Antti Koskela, Antti Honkela

Abstract: Markov chain Monte Carlo (MCMC) algorithms have long been the main workhorses of Bayesian inference. Among them, Hamiltonian Monte Carlo (HMC) has recently become very popular due to its efficiency resulting from effective use of the gradients of the target distribution. In privacy-preserving machine learning, differential privacy (DP) has become the gold standard in ensuring that the privacy of d… ▽ More Markov chain Monte Carlo (MCMC) algorithms have long been the main workhorses of Bayesian inference. Among them, Hamiltonian Monte Carlo (HMC) has recently become very popular due to its efficiency resulting from effective use of the gradients of the target distribution. In privacy-preserving machine learning, differential privacy (DP) has become the gold standard in ensuring that the privacy of data subjects is not violated. Existing DP MCMC algorithms either use random-walk proposals, or do not use the Metropolis--Hastings (MH) acceptance test to ensure convergence without decreasing their step size to zero. We present a DP variant of HMC using the MH acceptance test that builds on a recently proposed DP MCMC algorithm called the penalty algorithm, and adds noise to the gradient evaluations of HMC. We prove that the resulting algorithm converges to the correct distribution, and is ergodic. We compare DP-HMC with the existing penalty, DP-SGLD and DP-SGNHT algorithms, and find that DP-HMC has better or equal performance than the penalty algorithm, and performs more consistently than DP-SGLD or DP-SGNHT. △ Less

Submitted 17 June, 2021; originally announced June 2021.

Comments: 18 pages, 3 figures

Tight Accounting in the Shuffle Model of Differential Privacy

Authors: Antti Koskela, Mikko A. Heikkilä, Antti Honkela

Abstract: Shuffle model of differential privacy is a novel distributed privacy model based on a combination of local privacy mechanisms and a secure shuffler. It has been shown that the additional randomisation provided by the shuffler improves privacy bounds compared to the purely local mechanisms. Accounting tight bounds, however, is complicated by the complexity brought by the shuffler. The recently prop… ▽ More Shuffle model of differential privacy is a novel distributed privacy model based on a combination of local privacy mechanisms and a secure shuffler. It has been shown that the additional randomisation provided by the shuffler improves privacy bounds compared to the purely local mechanisms. Accounting tight bounds, however, is complicated by the complexity brought by the shuffler. The recently proposed numerical techniques for evaluating $(\varepsilon,δ)$-differential privacy guarantees have been shown to give tighter bounds than commonly used methods for compositions of various complex mechanisms. In this paper, we show how to obtain accurate bounds for adaptive compositions of general $\varepsilon$-LDP shufflers using the analysis by Feldman et al. (2021) and tight bounds for adaptive compositions of shufflers of $k$-randomised response mechanisms, using the analysis by Balle et al. (2019). We show how to speed up the evaluation of the resulting privacy loss distribution from $\mathcal{O}(n^2)$ to $\mathcal{O}(n)$, where $n$ is the number of users, without noticeable change in the resulting $δ(\varepsilon)$-upper bounds. We also demonstrate looseness of the existing bounds and methods found in the literature, improving previous composition results significantly. △ Less

Submitted 31 January, 2022; v1 submitted 1 June, 2021; originally announced June 2021.

Comments: 21 pages, 5 figures

Computing Differential Privacy Guarantees for Heterogeneous Compositions Using FFT

Authors: Antti Koskela, Antti Honkela

Abstract: The recently proposed Fast Fourier Transform (FFT)-based accountant for evaluating $(\varepsilon,δ)$-differential privacy guarantees using the privacy loss distribution formalism has been shown to give tighter bounds than commonly used methods such as Rényi accountants when applied to homogeneous compositions, i.e., to compositions of identical mechanisms. In this paper, we extend this approach to… ▽ More The recently proposed Fast Fourier Transform (FFT)-based accountant for evaluating $(\varepsilon,δ)$-differential privacy guarantees using the privacy loss distribution formalism has been shown to give tighter bounds than commonly used methods such as Rényi accountants when applied to homogeneous compositions, i.e., to compositions of identical mechanisms. In this paper, we extend this approach to heterogeneous compositions. We carry out a full error analysis that allows choosing the parameters of the algorithm such that a desired accuracy is obtained. The analysis also extends previous results by taking into account all the parameters of the algorithm. Using the error analysis, we also give a bound for the computational complexity in terms of the error which is analogous to and slightly tightens the one given by Murtagh and Vadhan (2018). We also show how to speed up the evaluation of tight privacy guarantees using the Plancherel theorem at the cost of increased pre-computation and memory usage. △ Less

Submitted 21 June, 2021; v1 submitted 24 February, 2021; originally announced February 2021.

Comments: 44 pages, 10 figures

Differentially Private Bayesian Inference for Generalized Linear Models

Authors: Tejas Kulkarni, Joonas Jälkö, Antti Koskela, Samuel Kaski, Antti Honkela

Abstract: Generalized linear models (GLMs) such as logistic regression are among the most widely used arms in data analyst's repertoire and often used on sensitive datasets. A large body of prior works that investigate GLMs under differential privacy (DP) constraints provide only private point estimates of the regression coefficients, and are not able to quantify parameter uncertainty. In this work, with lo… ▽ More Generalized linear models (GLMs) such as logistic regression are among the most widely used arms in data analyst's repertoire and often used on sensitive datasets. A large body of prior works that investigate GLMs under differential privacy (DP) constraints provide only private point estimates of the regression coefficients, and are not able to quantify parameter uncertainty. In this work, with logistic and Poisson regression as running examples, we introduce a generic noise-aware DP Bayesian inference method for a GLM at hand, given a noisy sum of summary statistics. Quantifying uncertainty allows us to determine which of the regression coefficients are statistically significantly different from zero. We provide a previously unknown tight privacy analysis and experimentally demonstrate that the posteriors obtained from our model, while adhering to strong privacy guarantees, are close to the non-private posteriors. △ Less

Submitted 12 May, 2021; v1 submitted 1 November, 2020; originally announced November 2020.

Computing low-rank approximations of the Fréchet derivative of a matrix function using Krylov subspace methods

Authors: Peter Kandolf, Antti Koskela, Samuel D. Relton, Marcel Schweitzer

Abstract: The Fréchet derivative $L_f(A,E)$ of the matrix function $f(A)$ plays an important role in many different applications, including condition number estimation and network analysis. We present several different Krylov subspace methods for computing low-rank approximations of $L_f(A,E)$ when the direction term $E$ is of rank one (which can easily be extended to general low-rank). We analyze the conve… ▽ More The Fréchet derivative $L_f(A,E)$ of the matrix function $f(A)$ plays an important role in many different applications, including condition number estimation and network analysis. We present several different Krylov subspace methods for computing low-rank approximations of $L_f(A,E)$ when the direction term $E$ is of rank one (which can easily be extended to general low-rank). We analyze the convergence of the resulting method for the important special case that $A$ is Hermitian and $f$ is either the exponential, the logarithm or a Stieltjes function. In a number of numerical tests, both including matrices from benchmark collections and from real-world applications, we demonstrate and compare the accuracy and efficiency of the proposed methods. △ Less

Submitted 29 August, 2020; originally announced August 2020.

MSC Class: 65F50; 65F55; 65F60

Differentially private cross-silo federated learning

Authors: Mikko A. Heikkilä, Antti Koskela, Kana Shimizu, Samuel Kaski, Antti Honkela

Abstract: Strict privacy is of paramount importance in distributed machine learning. Federated learning, with the main idea of communicating only what is needed for learning, has been recently introduced as a general approach for distributed learning to enhance learning and improve security. However, federated learning by itself does not guarantee any privacy for data subjects. To quantify and control how m… ▽ More Strict privacy is of paramount importance in distributed machine learning. Federated learning, with the main idea of communicating only what is needed for learning, has been recently introduced as a general approach for distributed learning to enhance learning and improve security. However, federated learning by itself does not guarantee any privacy for data subjects. To quantify and control how much privacy is compromised in the worst-case, we can use differential privacy. In this paper we combine additively homomorphic secure summation protocols with differential privacy in the so-called cross-silo federated learning setting. The goal is to learn complex models like neural networks while guaranteeing strict privacy for the individual data subjects. We demonstrate that our proposed solutions give prediction accuracy that is comparable to the non-distributed setting, and are fast enough to enable learning models with millions of parameters in a reasonable time. To enable learning under strict privacy guarantees that need privacy amplification by subsampling, we present a general algorithm for oblivious distributed subsampling. However, we also argue that when malicious parties are present, a simple approach using distributed Poisson subsampling gives better privacy. Finally, we show that by leveraging random projections we can further scale-up our approach to larger models while suffering only a modest performance loss. △ Less

Submitted 10 July, 2020; originally announced July 2020.

Comments: 14 pages, 5 figures

Tight Differential Privacy for Discrete-Valued Mechanisms and for the Subsampled Gaussian Mechanism Using FFT

Authors: Antti Koskela, Joonas Jälkö, Lukas Prediger, Antti Honkela

Abstract: We propose a numerical accountant for evaluating the tight $(\varepsilon,δ)$-privacy loss for algorithms with discrete one dimensional output. The method is based on the privacy loss distribution formalism and it uses the recently introduced fast Fourier transform based accounting technique. We carry out an error analysis of the method in terms of moment bounds of the privacy loss distribution whi… ▽ More We propose a numerical accountant for evaluating the tight $(\varepsilon,δ)$-privacy loss for algorithms with discrete one dimensional output. The method is based on the privacy loss distribution formalism and it uses the recently introduced fast Fourier transform based accounting technique. We carry out an error analysis of the method in terms of moment bounds of the privacy loss distribution which leads to rigorous lower and upper bounds for the true $(\varepsilon,δ)$-values. As an application, we present a novel approach to accurate privacy accounting of the subsampled Gaussian mechanism. This completes the previously proposed analysis by giving strict lower and upper bounds for the privacy parameters. We demonstrate the performance of the accountant on the binomial mechanism and show that our approach allows decreasing noise variance up to 75 percent at equal privacy compared to existing bounds in the literature. We also illustrate how to compute tight bounds for the exponential mechanism applied to counting queries. △ Less

Submitted 21 June, 2021; v1 submitted 12 June, 2020; originally announced June 2020.

Comments: 41 pages, 5 figures

Journal ref: AISTATS (2021) 3358-3366

Sampling of Stochastic Differential Equations using the Karhunen-Loève Expansion and Matrix Functions

Authors: Antti Koskela, Samuel D. Relton

Abstract: We consider linearizations of stochastic differential equations with additive noise using the Karhunen-Loève expansion. We obtain our linearizations by truncating the expansion and writing the solution as a series of matrix-vector products using the theory of matrix functions. Moreover, we restate the solution as the solution of a system of linear differential equations. We obtain strong and weak… ▽ More We consider linearizations of stochastic differential equations with additive noise using the Karhunen-Loève expansion. We obtain our linearizations by truncating the expansion and writing the solution as a series of matrix-vector products using the theory of matrix functions. Moreover, we restate the solution as the solution of a system of linear differential equations. We obtain strong and weak error bounds for the truncation procedure and show that, under suitable conditions, the mean square error has order of convergence $\mathcal{O}(\frac{1}{m})$ and the second moment has a weak order of convergence $\mathcal{O}(\frac{1}{m})$, where $m$ denotes the size of the expansion. We also discuss efficient numerical linear algebraic techniques to approximate the series of matrix functions and the linearized system of differential equations. These theoretical results are supported by experiments showing the effectiveness of our algorithms when compared to standard methods such as the Euler-Maruyama scheme. △ Less

Submitted 12 April, 2020; originally announced April 2020.

Comments: 19 pages, 4 figures

Computing Tight Differential Privacy Guarantees Using FFT

Authors: Antti Koskela, Joonas Jälkö, Antti Honkela

Abstract: Differentially private (DP) machine learning has recently become popular. The privacy loss of DP algorithms is commonly reported using $(\varepsilon,δ)$-DP. In this paper, we propose a numerical accountant for evaluating the privacy loss for algorithms with continuous one dimensional output. This accountant can be applied to the subsampled multidimensional Gaussian mechanism which underlies the po… ▽ More Differentially private (DP) machine learning has recently become popular. The privacy loss of DP algorithms is commonly reported using $(\varepsilon,δ)$-DP. In this paper, we propose a numerical accountant for evaluating the privacy loss for algorithms with continuous one dimensional output. This accountant can be applied to the subsampled multidimensional Gaussian mechanism which underlies the popular DP stochastic gradient descent. The proposed method is based on a numerical approximation of an integral formula which gives the exact $(\varepsilon,δ)$-values. The approximation is carried out by discretising the integral and by evaluating discrete convolutions using the fast Fourier transform algorithm. We give both theoretical error bounds and numerical error estimates for the approximation. Experimental comparisons with state-of-the-art techniques demonstrate significant improvements in bound tightness and/or computation time. Python code for the method can be found in Github (https://github.com/DPBayes/PLD-Accountant/). △ Less

Submitted 4 November, 2019; v1 submitted 7 June, 2019; originally announced June 2019.

Comments: 43 pages, 7 figures

Journal ref: AISTATS (2020) 2560-2569

Learning Rate Adaptation for Federated and Differentially Private Learning

Authors: Antti Koskela, Antti Honkela

Abstract: We propose an algorithm for the adaptation of the learning rate for stochastic gradient descent (SGD) that avoids the need for validation set use. The idea for the adaptiveness comes from the technique of extrapolation: to get an estimate for the error against the gradient flow which underlies SGD, we compare the result obtained by one full step and two half-steps. The algorithm is applied in two… ▽ More We propose an algorithm for the adaptation of the learning rate for stochastic gradient descent (SGD) that avoids the need for validation set use. The idea for the adaptiveness comes from the technique of extrapolation: to get an estimate for the error against the gradient flow which underlies SGD, we compare the result obtained by one full step and two half-steps. The algorithm is applied in two separate frameworks: federated and differentially private learning. Using examples of deep neural networks we empirically show that the adaptive algorithm is competitive with manually tuned commonly used optimisation methods for differentially privately training. We also show that it works robustly in the case of federated learning unlike commonly used optimisation methods. △ Less

Submitted 31 May, 2019; v1 submitted 11 September, 2018; originally announced September 2018.

Comments: 17 pages, 9 figures

Journal ref: AISTATS (2020) 2465-2475

On a generalization of the Bessel function Neumann expansion

Authors: Antti Koskela, Elias Jarlebring

Abstract: The Bessel-Neumann expansion (of integer order) of a function $g:\mathbb{C}\rightarrow\mathbb{C}$ corresponds to representing $g$ as a linear combination of basis functions $φ_0,φ_1,\ldots$, i.e., $g(z)=\sum_{\ell = 0}^\infty w_\ell φ_\ell(s)$, where $φ_i(z)=J_i(z)$, $i=0,\ldots$, are the Bessel functions. In this work, we study an expansion for a more general class of basis functions. More precis… ▽ More The Bessel-Neumann expansion (of integer order) of a function $g:\mathbb{C}\rightarrow\mathbb{C}$ corresponds to representing $g$ as a linear combination of basis functions $φ_0,φ_1,\ldots$, i.e., $g(z)=\sum_{\ell = 0}^\infty w_\ell φ_\ell(s)$, where $φ_i(z)=J_i(z)$, $i=0,\ldots$, are the Bessel functions. In this work, we study an expansion for a more general class of basis functions. More precisely, we assume that the basis functions satisfy an infinite dimensional linear ordinary differential equation associated with a Hessenberg matrix, motivated by the fact that these basis functions occur in certain iterative methods. A procedure to compute the basis functions as well as the coefficients is proposed. Theoretical properties of the expansion are studied. We illustrate that non-standard basis functions can give faster convergence than the Bessel functions. △ Less

Submitted 12 December, 2017; originally announced December 2017.

Comments: 8 pages, 2 figures

Krylov integrators for Hamiltonian systems

Authors: Antti Koskela

Abstract: We consider Arnoldi like processes to obtain symplectic subspaces for Hamiltonian systems. Large systems are locally approximated by ones living in low dimensional subspaces; we especially consider Krylov subspaces and some extensions. This will be utilized in two ways: solve numerically local small dimensional systems or in a given numerical, e.g. exponential, integrator, use the subspace for app… ▽ More We consider Arnoldi like processes to obtain symplectic subspaces for Hamiltonian systems. Large systems are locally approximated by ones living in low dimensional subspaces; we especially consider Krylov subspaces and some extensions. This will be utilized in two ways: solve numerically local small dimensional systems or in a given numerical, e.g. exponential, integrator, use the subspace for approximations of necessary functions. In the former case one can expect an excellent energy preservation. For the latter this is so for linear systems. For some second order exponential integrators we consider these two approaches are shown to be equivalent. In numerical experiments with nonlinear Hamiltonian problems their behaviour seems promising. △ Less

Submitted 14 December, 2017; v1 submitted 11 December, 2017; originally announced December 2017.

Comments: 24 pages, 10 figures

MSC Class: 65P10; 37M15; 65F60; 65F10

Journal ref: BIT Numerical Mathematics 59 (1), 57-76 (2019)

Analysis of Krylov Subspace Approximation to Large Scale Differential Riccati Equations

Authors: Antti Koskela, Hermann Mena

Abstract: We consider a Krylov subspace approximation method for the symmetric differential Riccati equation $\dot{X} = AX + XA^T + Q - XSX$, $X(0)=X_0$. The method we consider is based on projecting the large scale equation onto a Krylov subspace spanned by the matrix $A$ and the low rank factors of $X_0$ and $Q$. We prove that the method is structure preserving in the sense that it preserves two important… ▽ More We consider a Krylov subspace approximation method for the symmetric differential Riccati equation $\dot{X} = AX + XA^T + Q - XSX$, $X(0)=X_0$. The method we consider is based on projecting the large scale equation onto a Krylov subspace spanned by the matrix $A$ and the low rank factors of $X_0$ and $Q$. We prove that the method is structure preserving in the sense that it preserves two important properties of the exact flow, namely the positivity of the exact flow, and also the property of monotonicity. We also provide a theoretical a priori error analysis which shows a superlinear convergence of the method. This behavior is illustrated in the numerical experiments. Moreover, we derive an efficient a posteriori error estimate as well as discuss multiple time step** combined with a cut of the rank of the numerical solution. △ Less

Submitted 29 December, 2018; v1 submitted 21 May, 2017; originally announced May 2017.

Comments: 26 pages, 10 figures

MSC Class: 65F10; 65F60; 65L20; 65M22; 93A15; 93C05

Journal ref: Electronic Transactions on Numerical Analysis 52, 431--454 (2020)

Disguised and new Quasi-Newton methods for nonlinear eigenvalue problems

Authors: Elias Jarlebring, Antti Koskela, Giampaolo Mele

Abstract: In this paper we take a quasi-Newton approach to nonlinear eigenvalue problems (NEPs) of the type $M(λ)v=0$, where $M:\mathbb{C}\rightarrow\mathbb{C}^{n\times n}$ is a holomorphic function. We investigate which types of approximations of the Jacobian matrix lead to competitive algorithms, and provide convergence theory. The convergence analysis is based on theory for quasi-Newton methods and Keldy… ▽ More In this paper we take a quasi-Newton approach to nonlinear eigenvalue problems (NEPs) of the type $M(λ)v=0$, where $M:\mathbb{C}\rightarrow\mathbb{C}^{n\times n}$ is a holomorphic function. We investigate which types of approximations of the Jacobian matrix lead to competitive algorithms, and provide convergence theory. The convergence analysis is based on theory for quasi-Newton methods and Keldysh's theorem for NEPs. We derive new algorithms and also show that several well-established methods for NEPs can be interpreted as quasi-Newton methods, and thereby provide insight to their convergence behavior. In particular, we establish quasi-Newton interpretations of Neumaier's residual inverse iteration and Ruhe's method of successive linear problems. △ Less

Submitted 27 February, 2017; originally announced February 2017.

Splitting methods for time integration of trajectories in combined electric and magnetic fields

Authors: Christian Knapp, Alexander Kendl, Antti Koskela, Alexander Ostermann

Abstract: The equations of motion of a single particle subject to an arbitrary electric and a static magnetic field form a Poisson system. We present a second-order time integration method which preserves well the Poisson structure and compare it to commonly used algorithms, such as the Boris scheme. All the methods are represented in a general framework of splitting methods. We use the so-called $φ$ functi… ▽ More The equations of motion of a single particle subject to an arbitrary electric and a static magnetic field form a Poisson system. We present a second-order time integration method which preserves well the Poisson structure and compare it to commonly used algorithms, such as the Boris scheme. All the methods are represented in a general framework of splitting methods. We use the so-called $φ$ functions, which give efficient ways for both analyzing and implementing the algorithms. Numerical experiments show an excellent long term stability for the new method considered. △ Less

Submitted 6 October, 2015; originally announced October 2015.

Comments: 14 pages, 8 figures

Journal ref: Phys. Rev. E 92, 063310 (2015)

Krylov approximation of ODEs with polynomial parameterization

Authors: Antti Koskela, Elias Jarlebring, Michiel E. Hochstenbach

Abstract: We propose a new numerical method to solve linear ordinary differential equations of the type $\frac{\partial u}{\partial t}(t,\varepsilon) = A(\varepsilon) \, u(t,\varepsilon)$, where $A:\mathbb{C}\rightarrow\mathbb{C}^{n\times n}$ is a matrix polynomial with large and sparse matrix coefficients. The algorithm computes an explicit parameterization of approximations of $u(t,\varepsilon)$ such that… ▽ More We propose a new numerical method to solve linear ordinary differential equations of the type $\frac{\partial u}{\partial t}(t,\varepsilon) = A(\varepsilon) \, u(t,\varepsilon)$, where $A:\mathbb{C}\rightarrow\mathbb{C}^{n\times n}$ is a matrix polynomial with large and sparse matrix coefficients. The algorithm computes an explicit parameterization of approximations of $u(t,\varepsilon)$ such that approximations for many different values of $\varepsilon$ and $t$ can be obtained with a very small additional computational effort. The derivation of the algorithm is based on a reformulation of the parameterization as a linear parameter-free ordinary differential equation and on approximating the product of the matrix exponential and a vector with a Krylov method. The Krylov approximation is generated with Arnoldi's method and the structure of the coefficient matrix turns out to have an independence on the truncation parameter so that it can also be interpreted as Arnoldi's method applied to an infinite dimensional matrix. We prove the superlinear convergence of the algorithm and provide a posteriori error estimates to be used as termination criteria. The behavior of the algorithm is illustrated with examples stemming from spatial discretizations of partial differential equations. △ Less

Submitted 27 July, 2015; originally announced July 2015.

Comments: 22 pages, 10 figures

Journal ref: SIMAX 37.2 (2016) 519-538

The infinite Arnoldi exponential integrator for linear inhomogeneous ODEs

Authors: Antti Koskela, Elias Jarlebring

Abstract: Exponential integrators that use Krylov approximations of matrix functions have turned out to be efficient for the time-integration of certain ordinary differential equations (ODEs). This holds in particular for linear homogeneous ODEs, where the exponential integrator is equivalent to approximating the product of the matrix exponential and a vector. In this paper, we consider linear inhomogeneous… ▽ More Exponential integrators that use Krylov approximations of matrix functions have turned out to be efficient for the time-integration of certain ordinary differential equations (ODEs). This holds in particular for linear homogeneous ODEs, where the exponential integrator is equivalent to approximating the product of the matrix exponential and a vector. In this paper, we consider linear inhomogeneous ODEs, $y'(t)=Ay(t)+g(t)$, where the function $g(t)$ is assumed to satisfy certain regularity conditions. We derive an algorithm for this problem which is equivalent to approximating the product of the matrix exponential and a vector using Arnoldi's method. The construction is based on expressing the function $g(t)$ as a linear combination of given basis functions $[φ_i]_{i=0}^\infty$ with particular properties. The properties are such that the inhomogeneous ODE can be restated as an infinite-dimensional linear homogeneous ODE. Moreover, the linear homogeneous infinite-dimensional ODE has properties that directly allow us to extend a Krylov method for finite-dimensional linear ODEs. Although the construction is based on an infinite-dimensional operator, the algorithm can be carried out with operations involving matrices and vectors of finite size. This type of construction resembles in many ways the infinite Arnoldi method for nonlinear eigenvalue problems. We prove convergence of the algorithm under certain natural conditions, and illustrate properties of the algorithm with examples stemming from the discretization of partial differential equations. △ Less

Submitted 24 February, 2015; v1 submitted 5 February, 2015; originally announced February 2015.

Comments: 25 pages, 10 figures

MSC Class: 65F10; 65F60; 65L05; 65L20

Differential Geometrically Consistent Artificial Viscosity in Comoving Curvilinear Coordinates

Authors: Harald Höller, Antti Koskela, Ernst Dorfi, Werner Benger

Abstract: Context. High-resolution numerical methods have been developed for nonlinear, discontinuous problems as they appear in simulations of astrophysical objects. One of the strategies applied is the concept of artificial viscosity. Aims. Grid-based numerical simulations ideally utilize problem-oriented grids in order to minimize the necessary number of cells at a given (desired) spatial resolution. We… ▽ More Context. High-resolution numerical methods have been developed for nonlinear, discontinuous problems as they appear in simulations of astrophysical objects. One of the strategies applied is the concept of artificial viscosity. Aims. Grid-based numerical simulations ideally utilize problem-oriented grids in order to minimize the necessary number of cells at a given (desired) spatial resolution. We want to propose a modified tensor of artificial viscosity which is employable for generally comoving, curvilinear grids. Methods. We study a differential geometrically consistent artificial viscosity analytically and visualize a comparison of our result to previous implementations by applying it to a simple self-similar velocity field. We give a general introduction to artificial viscosity first and motivate its application in numerical analysis. Then we present how a tensor of artificial viscosity has to be designed when going beyond common static Eulerian or Lagrangian comoving rectangular grids. Results. We find that in comoving, curvilinear coordinates the isotropic (pressure) part of the tensor of artificial viscosity has to be modified metrically in order for it to fulfill all its desired properties. △ Less

Submitted 17 June, 2013; v1 submitted 22 April, 2013; originally announced April 2013.

Comments: Submitted to Computer Physics Communications

Investigation of the recombination of the retarded shell of "born-again" CSPNe by time-dependent radiative transfer models

Authors: Antti Koskela, Silvia Dalnodar, Ralf Kissmann, Anita Reimer, Alexander Ostermann, Stefan Kimeswenger

Abstract: A standard planetary nebula stays more than 10 000 years in the state of a photoionized nebula. As long as the timescales of the most important ionizing processes are much smaller, the ionization state can be characterized by a static photoionization model and simulated with codes like CLOUDY (Ferland et al. 1998). When the star exhibits a late Helium flash, however, its ionizing flux stops within… ▽ More A standard planetary nebula stays more than 10 000 years in the state of a photoionized nebula. As long as the timescales of the most important ionizing processes are much smaller, the ionization state can be characterized by a static photoionization model and simulated with codes like CLOUDY (Ferland et al. 1998). When the star exhibits a late Helium flash, however, its ionizing flux stops within a very short period. The star then re-appears from itsopaque shell after a few years (or centuries) as a cold giant star without any hard ionizing photons. Describing the physics of such behavior requires a fully time-dependent radiative transfer model. Pollacco (1999), Kerber et al. (1999) and Lechner & Kimeswenger (2004) used data of the old nebulae around V605 Aql and V4334 Sgr to derive a model of the pre-outburst state of the CSPN in a static model. Their argument was the long recombination time scale for such thin media. With regard to these models Schoenberner (2008) critically raised the question whether a significant change in the ionization state (and thus the spectrum) has to be expected after a time of up to 80 years, and whether static models are applicable at all. △ Less

Submitted 9 September, 2011; originally announced September 2011.

Comments: (3 pages, 1 figure, to appear in proceedings of the IAU Symposium 283: "Planetary Nebulae: An Eye to the Future", Eds.: A. Manchado, L. Stanghellini and D. Schoenberner; presenting author: Stefan Kimeswenger)