-
Anatomy of an extreme event: What can we infer about the history of a heavy-tailed random walk?
Authors:
Wesley W. Erickson,
Daniel A. Steck
Abstract:
Extreme events are by nature rare and difficult to predict, yet are often much more important than frequent, typical events. An interesting counterpoint to the prediction of such events is their retrodiction -- given a process in an outlier state, how did the events leading up to this endpoint unfold? In particular, was there only a single, massive event, or was the history a composite of multiple…
▽ More
Extreme events are by nature rare and difficult to predict, yet are often much more important than frequent, typical events. An interesting counterpoint to the prediction of such events is their retrodiction -- given a process in an outlier state, how did the events leading up to this endpoint unfold? In particular, was there only a single, massive event, or was the history a composite of multiple, smaller but still significant events? To investigate this problem we take heavy-tailed stochastic processes (specifically,the symmetric, $α$-stable Lévy processes) as prototypical random walks. A natural and useful characteristic scale arises from the analysis of processes conditioned to arrive in a particular final state (Lévy bridges). For final displacements longer than this scale, the scenario of a single, long jump is most likely, even though it corresponds to a rare, extreme event. On the other hand, for small final displacements, histories involving extreme events tend to be suppressed. To further illustrate the utility of this analysis, we show how it provides an intuitive framework for understanding three problems related to boundary crossings of heavy-tailed processes. These examples illustrate how intuition fails to carry over from diffusive processes, even very close to the Gaussian limit. One example yields a computationally and conceptually useful representation of Lévy bridges that illustrates how conditioning impacts the extreme-event content of a random walk. The other examples involve the conditioned boundary-crossing problem and the ordinary first-escape problem; we discuss the observability of the latter example in experiments with laser-cooled atoms.
△ Less
Submitted 18 November, 2022; v1 submitted 6 February, 2020;
originally announced February 2020.
-
Regularization of Limited Memory Quasi-Newton Methods for Large-Scale Nonconvex Minimization
Authors:
Daniel Steck,
Christian Kanzow
Abstract:
This paper deals with regularized Newton methods, a flexible class of unconstrained optimization algorithms that is competitive with line search and trust region methods and potentially combines attractive elements of both. The particular focus is on combining regularization with limited memory quasi-Newton methods by exploiting the special structure of limited memory algorithms. Global convergenc…
▽ More
This paper deals with regularized Newton methods, a flexible class of unconstrained optimization algorithms that is competitive with line search and trust region methods and potentially combines attractive elements of both. The particular focus is on combining regularization with limited memory quasi-Newton methods by exploiting the special structure of limited memory algorithms. Global convergence of regularization methods is shown under mild assumptions and the details of regularized limited memory quasi-Newton updates are discussed including their compact representations.
Numerical results using all large-scale test problems from the CUTEst collection indicate that our regularized version of L-BFGS is competitive with state-of-the-art line search and trust-region L-BFGS algorithms and previous attempts at combining L-BFGS with regularization, while potentially outperforming some of them, especially when nonmonotonicity is involved.
△ Less
Submitted 11 July, 2022; v1 submitted 11 November, 2019;
originally announced November 2019.
-
Brezis pseudomonotonicity is strictly weaker than Ky-Fan hemicontinuity
Authors:
Daniel Steck
Abstract:
In 1968, H. Brezis introduced a notion of operator pseudomonotonicity which provides a unified approach to monotone and nonmonotone variational inequalities (VIs). A closely related notion is that of Ky-Fan hemicontinuity, a continuity property which arises if the famous Ky-Fan minimax inequality is applied to the VI framework. It is clear from the corresponding definitions that Ky-Fan hemicontinu…
▽ More
In 1968, H. Brezis introduced a notion of operator pseudomonotonicity which provides a unified approach to monotone and nonmonotone variational inequalities (VIs). A closely related notion is that of Ky-Fan hemicontinuity, a continuity property which arises if the famous Ky-Fan minimax inequality is applied to the VI framework. It is clear from the corresponding definitions that Ky-Fan hemicontinuity implies Brezis pseudomonotonicity, but quite surprisingly, a recent publication by Sadeqi and Paydar (J. Optim. Theory Appl., 165(2):344-358, 2015) claims the equivalence of the two properties. The purpose of the present note is to show that this equivalence is false; this is achieved by providing a concrete example of a nonlinear operator which is Brezis pseudomonotone but not Ky--Fan hemicontinuous.
△ Less
Submitted 9 October, 2018;
originally announced October 2018.
-
Quasi-Variational Inequalities in Banach Spaces: Theory and Augmented Lagrangian Methods
Authors:
Christian Kanzow,
Daniel Steck
Abstract:
This paper deals with quasi-variational inequality problems (QVIs) in a generic Banach space setting. We provide a theoretical framework for the analysis of such problems which is based on two key properties: the pseudomonotonicity (in the sense of Brezis) of the variational operator and a Mosco-type continuity of the feasible set map**. We show that these assumptions can be used to establish th…
▽ More
This paper deals with quasi-variational inequality problems (QVIs) in a generic Banach space setting. We provide a theoretical framework for the analysis of such problems which is based on two key properties: the pseudomonotonicity (in the sense of Brezis) of the variational operator and a Mosco-type continuity of the feasible set map**. We show that these assumptions can be used to establish the existence of solutions and their computability via suitable approximation techniques. In addition, we provide a practical and easily verifiable sufficient condition for the Mosco-type continuity property in terms of suitable constraint qualifications.
Based on the theoretical framework, we construct an algorithm of augmented Lagrangian type which reduces the QVI to a sequence of standard variational inequalities. A full convergence analysis is provided which includes the existence of solutions of the subproblems as well as the attainment of feasibility and optimality. Applications and numerical results are included to demonstrate the practical viability of the method.
△ Less
Submitted 3 December, 2018; v1 submitted 30 September, 2018;
originally announced October 2018.
-
Relaxation schemes for mathematical programs with switching constraints
Authors:
Christian Kanzow,
Patrick Mehlitz,
Daniel Steck
Abstract:
Switching-constrained optimization problems form a difficult class of mathematical programs since their feasible set is almost disconnected while standard constraint qualifications are likely to fail at several feasible points. That is why the application of standard methods from nonlinear programming does not seem to be promising in order to solve such problems. In this paper, we adapt the relaxa…
▽ More
Switching-constrained optimization problems form a difficult class of mathematical programs since their feasible set is almost disconnected while standard constraint qualifications are likely to fail at several feasible points. That is why the application of standard methods from nonlinear programming does not seem to be promising in order to solve such problems. In this paper, we adapt the relaxation method from Kanzow and Schwartz (SIAM J. Optim., 23(2):770-798, 2013) for the numerical treatment of mathematical programs with complementarity constraints to the setting of switching-constrained optimization. It is shown that the proposed method computes M-stationary points under mild assumptions. Furthermore, we comment on other possible relaxation approaches which can be used to tackle mathematical programs with switching constraints. As it turns out, adapted versions of Scholtes' global relaxation scheme as well as the relaxation scheme of Steffensen and Ulbrich only find W-stationary points of switching-constrained optimization problems in general. Some computational experiments visualize the performance of the proposed relaxation method.
△ Less
Submitted 7 September, 2018;
originally announced September 2018.
-
Augmented Lagrangian Methods for the Solution of Generalized Nash Equilibrium Problems
Authors:
Christian Kanzow,
Daniel Steck
Abstract:
We propose an augmented Lagrangian-type algorithm for the solution of generalized Nash equilibrium problems (GNEPs). Specifically, we discuss the convergence properties with regard to both feasibility and optimality of limit points. This is done by introducing a secondary GNEP as a new optimality concept. In this context, special consideration is given to the role of suitable constraint qualificat…
▽ More
We propose an augmented Lagrangian-type algorithm for the solution of generalized Nash equilibrium problems (GNEPs). Specifically, we discuss the convergence properties with regard to both feasibility and optimality of limit points. This is done by introducing a secondary GNEP as a new optimality concept. In this context, special consideration is given to the role of suitable constraint qualifications that take into account the particular structure of GNEPs. Furthermore, we consider the behaviour of the method for jointly-convex GNEPs and describe a modification which is tailored towards the computation of variational equilibria. Numerical results are included to illustrate the practical performance of the overall method.
△ Less
Submitted 12 July, 2018;
originally announced July 2018.
-
An Augmented Lagrangian Method for Optimization Problems in Banach Spaces
Authors:
Christian Kanzow,
Daniel Steck,
Daniel Wachsmuth
Abstract:
We propose a variant of the classical augmented Lagrangian method for constrained optimization problems in Banach spaces. Our theoretical framework does not require any convexity or second-order assumptions and allows the treatment of inequality constraints with infinite-dimensional image space. Moreover, we discuss the convergence properties of our algorithm with regard to feasibility, global opt…
▽ More
We propose a variant of the classical augmented Lagrangian method for constrained optimization problems in Banach spaces. Our theoretical framework does not require any convexity or second-order assumptions and allows the treatment of inequality constraints with infinite-dimensional image space. Moreover, we discuss the convergence properties of our algorithm with regard to feasibility, global optimality, and KKT conditions. Some numerical results are given to illustrate the practical viability of the method.
△ Less
Submitted 12 July, 2018;
originally announced July 2018.
-
On Error Bounds and Multiplier Methods for Variational Problems in Banach Spaces
Authors:
Christian Kanzow,
Daniel Steck
Abstract:
This paper deals with a general form of variational problems in Banach spaces which encompasses variational inequalities as well as minimization problems. We prove a characterization of local error bounds for the distance to the (primal-dual) solution set and give a sufficient condition for such an error bound to hold. In the second part of the paper, we consider an algorithm of augmented Lagrangi…
▽ More
This paper deals with a general form of variational problems in Banach spaces which encompasses variational inequalities as well as minimization problems. We prove a characterization of local error bounds for the distance to the (primal-dual) solution set and give a sufficient condition for such an error bound to hold. In the second part of the paper, we consider an algorithm of augmented Lagrangian type for the solution of such variational problems. We give some global convergence properties of the method and then use the error bound theory to provide estimates for the rate of convergence and to deduce boundedness of the sequence of penalty parameters. Finally, numerical results for optimal control, Nash equilibrium problems, and elliptic parameter estimation problems are presented.
△ Less
Submitted 11 July, 2018;
originally announced July 2018.
