-
Proximal-Proximal-Gradient Method
Authors:
Ernest K. Ryu,
Wotao Yin
Abstract:
In this paper, we present the proximal-proximal-gradient method (PPG), a novel optimization method that is simple to implement and simple to parallelize. PPG generalizes the proximal-gradient method and ADMM and is applicable to minimization problems written as a sum of many differentiable and many non-differentiable convex functions. The non-differentiable functions can be coupled. We furthermore…
▽ More
In this paper, we present the proximal-proximal-gradient method (PPG), a novel optimization method that is simple to implement and simple to parallelize. PPG generalizes the proximal-gradient method and ADMM and is applicable to minimization problems written as a sum of many differentiable and many non-differentiable convex functions. The non-differentiable functions can be coupled. We furthermore present a related stochastic variation, which we call stochastic PPG (S-PPG). S-PPG can be interpreted as a generalization of Finito and MISO over to the sum of many coupled non-differentiable convex functions. We present many applications that can benefit from PPG and S-PPG and prove convergence for both methods. A key strength of PPG and S-PPG is, compared to existing methods, its ability to directly handle a large sum of non-differentiable non-separable functions with a constant stepsize independent of the number of functions. Such non-diminishing stepsizes allows them to be fast.
△ Less
Submitted 18 October, 2017; v1 submitted 23 August, 2017;
originally announced August 2017.
-
A New Use of Douglas-Rachford Splitting and ADMM for Identifying Infeasible, Unbounded, and Pathological Conic Programs
Authors:
Yanli Liu,
Ernest K. Ryu,
Wotao Yin
Abstract:
In this paper, we present a method for identifying infeasible, unbounded, and pathological conic programs based on Douglas-Rachford splitting, or equivalently ADMM. When an optimization program is infeasible, unbounded, or pathological, the iterates of Douglas-Rachford splitting diverge. Somewhat surprisingly, such divergent iterates still provide useful information, which our method uses for iden…
▽ More
In this paper, we present a method for identifying infeasible, unbounded, and pathological conic programs based on Douglas-Rachford splitting, or equivalently ADMM. When an optimization program is infeasible, unbounded, or pathological, the iterates of Douglas-Rachford splitting diverge. Somewhat surprisingly, such divergent iterates still provide useful information, which our method uses for identification. In addition, for strongly infeasible problems the method produces a separating hyperplane and informs the user on how to minimally modify the given problem to achieve strong feasibility. As a first-order method, the proposed algorithm relies on simple subroutines, and therefore is simple to implement and has low per-iteration cost.
△ Less
Submitted 15 October, 2017; v1 submitted 7 June, 2017;
originally announced June 2017.
-
Risk-Constrained Kelly Gambling
Authors:
Enzo Busseti,
Ernest K. Ryu,
Stephen Boyd
Abstract:
We consider the classic Kelly gambling problem with general distribution of outcomes, and an additional risk constraint that limits the probability of a drawdown of wealth to a given undesirable level. We develop a bound on the drawdown probability; using this bound instead of the original risk constraint yields a convex optimization problem that guarantees the drawdown risk constraint holds. Nume…
▽ More
We consider the classic Kelly gambling problem with general distribution of outcomes, and an additional risk constraint that limits the probability of a drawdown of wealth to a given undesirable level. We develop a bound on the drawdown probability; using this bound instead of the original risk constraint yields a convex optimization problem that guarantees the drawdown risk constraint holds. Numerical experiments show that our bound on drawdown probability is reasonably close to the actual drawdown risk, as computed by Monte Carlo simulation. Our method is parametrized by a single parameter that has a natural interpretation as a risk-aversion parameter, allowing us to systematically trade off asymptotic growth rate and drawdown risk. Simulations show that this method yields bets that out perform fractional-Kelly bets for the same drawdown risk level or growth rate. Finally, we show that a natural quadratic approximation of our convex problem is closely connected to the classical mean-variance Markowitz portfolio selection problem.
△ Less
Submitted 20 March, 2016;
originally announced March 2016.
-
Feasibility Study of Stochastic Streaming with 4K UHD Video Traces
Authors:
Joongheon Kim,
Eun-Seok Ryu
Abstract:
This paper performs the feasibility study of stochastic video streaming algorithms with up-to-date 4K ultra-high-definition (UHD) video traces. In previous work, various stochastic video streaming algorithms were proposed which maximize time-average video streaming quality subject to queue stability based on the information of queue-backlog length. The performance improvements with the stochastic…
▽ More
This paper performs the feasibility study of stochastic video streaming algorithms with up-to-date 4K ultra-high-definition (UHD) video traces. In previous work, various stochastic video streaming algorithms were proposed which maximize time-average video streaming quality subject to queue stability based on the information of queue-backlog length. The performance improvements with the stochastic video streaming algorithms were verified with traditional MPEG test sequences; but there is no study how much the proposed stochastic algorithm is better when we consider up-to-date 4K UHD video traces. Therefore, this paper evaluates the stochastic streaming algorithms with 4K UHD video traces; and verifies that the stochastic algorithms perform better than queue-independent algorithms, as desired.
△ Less
Submitted 17 November, 2015;
originally announced November 2015.
-
Adaptive Importance Sampling via Stochastic Convex Programming
Authors:
Ernest K. Ryu,
Stephen P. Boyd
Abstract:
We show that the variance of the Monte Carlo estimator that is importance sampled from an exponential family is a convex function of the natural parameter of the distribution. With this insight, we propose an adaptive importance sampling algorithm that simultaneously improves the choice of sampling distribution while accumulating a Monte Carlo estimate. Exploiting convexity, we prove that the meth…
▽ More
We show that the variance of the Monte Carlo estimator that is importance sampled from an exponential family is a convex function of the natural parameter of the distribution. With this insight, we propose an adaptive importance sampling algorithm that simultaneously improves the choice of sampling distribution while accumulating a Monte Carlo estimate. Exploiting convexity, we prove that the method's unbiased estimator has variance that is asymptotically optimal over the exponential family.
△ Less
Submitted 8 January, 2015; v1 submitted 15 December, 2014;
originally announced December 2014.
-
Computing reaction rates in bio-molecular systems using discrete macro-states
Authors:
Eric Darve,
Ernest Ryu
Abstract:
Computing reaction rates in biomolecular systems is a common goal of molecular dynamics simulations. The reactions considered often involve conformational changes in the molecule, either changes in the structure of a protein or the relative position of two molecules, for example when modeling the binding of a protein and ligand. Here we will consider the general problem of computing the rate of tr…
▽ More
Computing reaction rates in biomolecular systems is a common goal of molecular dynamics simulations. The reactions considered often involve conformational changes in the molecule, either changes in the structure of a protein or the relative position of two molecules, for example when modeling the binding of a protein and ligand. Here we will consider the general problem of computing the rate of transfer from a subset A of the conformational space Omega to a subset B of Omega. It is assumed that A and B are associated with minimum energy basins and are long-lived states.
Rates can be obtained using many different methods. We review some of the most popular approaches. We organize the different approaches roughly in chronological order and under four main categories: reactive flux, transition path sampling, conformation dynamics. The fourth class of methods, to which we do not give any specific name, in some sense attempts to combine features from transition path sampling and conformation dynamics. They include non-equilibrium umbrella sampling (Warmflash et al. [2007], Dickson et al. [2009b]), and weighted ensemble dynamics (Huber and Kim [1996]).
△ Less
Submitted 2 July, 2013;
originally announced July 2013.
-
Stochastic Kronecker Graph on Vertex-Centric BSP
Authors:
Ernest Ryu,
Sean Choi
Abstract:
Recently Stochastic Kronecker Graph (SKG), a network generation model, and vertex-centric BSP, a graph processing framework like Pregel, have attracted much attention in the network analysis community. Unfortunately the two are not very well-suited for each other and thus an implementation of SKG on vertex-centric BSP must either be done serially or in an unnatural manner.
In this paper, we pres…
▽ More
Recently Stochastic Kronecker Graph (SKG), a network generation model, and vertex-centric BSP, a graph processing framework like Pregel, have attracted much attention in the network analysis community. Unfortunately the two are not very well-suited for each other and thus an implementation of SKG on vertex-centric BSP must either be done serially or in an unnatural manner.
In this paper, we present a new network generation model, which we call Poisson Stochastic Kronecker Graph (PSKG), that generate edges according to the Poisson distribution. The advantage of PSKG is that it is easily parallelizable on vertex-centric BSP, requires no communication between computational nodes, and yet retains all the desired properties of SKG.
△ Less
Submitted 10 August, 2012;
originally announced August 2012.
