Showing 1–50 of 59 results for author: Friedlander, M

Infinite dimensional modules for linear algebraic groups

Authors: Eric M. Friedlander

Abstract: We investigate infinite dimensional modules for a linear algebraic group $\mathbb G$ over a field of positive characteristic $p$. For any subcoalgebra $C \subset \mathcal O(\mathbb G)$ of the coordinate algebra of $\mathbb G$, we consider the abelian subcategory $CoMod(C) \subset Mod(\mathbb G)$ and the left exact functor $(-)_C: Mod(\mathbb G) \to CoMod(C)$ that is right adjoint to the inclusion… ▽ More We investigate infinite dimensional modules for a linear algebraic group $\mathbb G$ over a field of positive characteristic $p$. For any subcoalgebra $C \subset \mathcal O(\mathbb G)$ of the coordinate algebra of $\mathbb G$, we consider the abelian subcategory $CoMod(C) \subset Mod(\mathbb G)$ and the left exact functor $(-)_C: Mod(\mathbb G) \to CoMod(C)$ that is right adjoint to the inclusion functor. The class of cofinite $\mathbb G$-modules is formulated using finite dimensional subcoalgebras of $\mathcal O(\mathbb G)$ and the new invariant of "cofinite type" is introduced. We are particularly interested in mock injective $\mathbb G$-modules, $\mathbb G$-modules which are not seen by earlier support theories. Various properties of these ghostly $\mathbb G$-modules are established. The stable category $StMock(\mathbb G)$ is introduced, enabling mock injective $\mathbb G$-modules to fit into the framework of tensor triangulated categories. △ Less

Submitted 18 June, 2024; originally announced June 2024.

Comments: This submission replaces arXiv:2305.10921 entitled "Filtrations and growth of G-modules"

Reformulation of the stable Adams conjecture

Authors: Eric M. Friedlander

Abstract: We revisit methods of proof of the Adams Conjecture in order to correct and supplement earlier efforts to prove analogous conjectures in the stable homotopy category. We utilize simplicial schemes over an algebraically closed field of positive characteristic and a rigid version of Artin-Mazur étale homotopy theory. Consideration of special $\mathcal F$-spaces and together with Bousfield-Kan… ▽ More We revisit methods of proof of the Adams Conjecture in order to correct and supplement earlier efforts to prove analogous conjectures in the stable homotopy category. We utilize simplicial schemes over an algebraically closed field of positive characteristic and a rigid version of Artin-Mazur étale homotopy theory. Consideration of special $\mathcal F$-spaces and together with Bousfield-Kan $\mathbb Z/\ell$-completion enables us to employ an "étale functor" which commutes up to homotopy with products of simplicial schemes. In order to prove the Stable Adams Conjecture, we construct the universal $\mathbb Z/\ell$-completed $X$-fibrations for various pointed simplicial sets $X$. Thus, two maps from a given $\mathcal F$-space $\underline{\mathcal B}$ to the base $\mathcal F$-space of the universal $\mathbb Z/\ell$-completed $X$-fibration $π_{X,\ell}: \underline {\mathcal B} (G_\ell(X),X_\ell) \to \underline {\mathcal B} G_\ell(X)$ determine homotopy equivalent maps of spectra if and only they correspond via pull-back of $π_{X,\ell}$ to fiber homotopy equivalent $\mathbb Z/\ell$-completed $X$-fibrations over $\underline {\mathcal B}$. For the proof of the Stable Adams Conjecture, we consider maps of $\mathcal F$-spaces $\underline {\mathcal B }\to \underline {\mathcal B} G_\ell(S^2)$ where $\underline {\mathcal B}$ is an $\mathcal F$-space model of connective $\ell$-completed connective $K$-theory. △ Less

Submitted 29 February, 2024; v1 submitted 22 October, 2023; originally announced October 2023.

Comments: Improved exposition with some corrected formulations

MSC Class: 55N20; 55R15

Filtrations and Growth of $\mathbb G$-modules

Authors: Eric M. Friedlander

Abstract: We investigate infinite dimensional modules for an affine group scheme $\mathbb G$ of finite type over a field of positive characteristic $p$. For any subspace $X \subset \mathcal O(\mathbb G)$ of the coordinate algebra of $\mathbb G$, we consider the abelian subcategory $Mod(\mathbb G,X) \subset Mod(\mathbb G)$ of ``$X$-comodules" and the left exact functor… ▽ More We investigate infinite dimensional modules for an affine group scheme $\mathbb G$ of finite type over a field of positive characteristic $p$. For any subspace $X \subset \mathcal O(\mathbb G)$ of the coordinate algebra of $\mathbb G$, we consider the abelian subcategory $Mod(\mathbb G,X) \subset Mod(\mathbb G)$ of ``$X$-comodules" and the left exact functor $(-)_X: Mod(\mathbb G) \to Mod(\mathbb G,X)$ which is right adjoint to the inclusion functor. We employ ``ascending converging sequences" $\{ X_i \}$ of subspaces of $\mathcal O(\mathbb G)$ to provide functorial filtrations $\{ M_{X_i }\}$ of each $\mathbb G$-module $M$. A $\mathbb G$-module $M$ is injective if and only if each $M_{X_i}$ is an injective $X_i$-comodule for some (or, equivalently, for all) such $\{ X_i \}$. We consider the explicit ascending converging sequence $ \{ \mathcal O(\mathbb G)_{\leq d,φ} \}$ of finite dimensional subcoalgebras of $\mathcal O(\mathbb G)$ depending upon a closed embedding $φ: \mathbb G \ \hookrightarrow \ GL_N$. Of particular interest to us are mock injective $\mathbb G$-modules, modules whose support varieties are empty. Restrictions of a $\mathbb G$-module to each $\mathcal O(\mathbb G)_{\leq d,φ}$ provide new invariants for $\mathbb G$-modules. For cofinite $\mathbb G$-modules $M$, we explore the the growth of $d \mapsto M_{\cal O(\mathbb G)_{\leq d,φ}}$. △ Less

Submitted 8 February, 2024; v1 submitted 18 May, 2023; originally announced May 2023.

Comments: Major changes implemented in this version

MSC Class: 20G05; 20C20; 20G10

Knowledge-Injected Federated Learning

Authors: Zhenan Fan, Zirui Zhou, Jian Pei, Michael P. Friedlander, Jiajie Hu, Chengliang Li, Yong Zhang

Abstract: Federated learning is an emerging technique for training models from decentralized data sets. In many applications, data owners participating in the federated learning system hold not only the data but also a set of domain knowledge. Such knowledge includes human know-how and craftsmanship that can be extremely helpful to the federated learning task. In this work, we propose a federated learning f… ▽ More Federated learning is an emerging technique for training models from decentralized data sets. In many applications, data owners participating in the federated learning system hold not only the data but also a set of domain knowledge. Such knowledge includes human know-how and craftsmanship that can be extremely helpful to the federated learning task. In this work, we propose a federated learning framework that allows the injection of participants' domain knowledge, where the key idea is to refine the global model with knowledge locally. The scenario we consider is motivated by a real industry-level application, and we demonstrate the effectiveness of our approach to this application. △ Less

Submitted 16 August, 2022; originally announced August 2022.

A dual approach for federated learning

Authors: Zhenan Fan, Huang Fang, Michael P. Friedlander

Abstract: We study the federated optimization problem from a dual perspective and propose a new algorithm termed federated dual coordinate descent (FedDCD), which is based on a type of coordinate descent method developed by Necora et al.[Journal of Optimization Theory and Applications, 2017]. Additionally, we enhance the FedDCD method with inexact gradient oracles and Nesterov's acceleration. We demonstrate… ▽ More We study the federated optimization problem from a dual perspective and propose a new algorithm termed federated dual coordinate descent (FedDCD), which is based on a type of coordinate descent method developed by Necora et al.[Journal of Optimization Theory and Applications, 2017]. Additionally, we enhance the FedDCD method with inexact gradient oracles and Nesterov's acceleration. We demonstrate theoretically that our proposed approach achieves better convergence rates than the state-of-the-art primal federated optimization algorithms under certain situations. Numerical experiments on real-world datasets support our analysis. △ Less

Submitted 3 February, 2022; v1 submitted 26 January, 2022; originally announced January 2022.

Fair and efficient contribution valuation for vertical federated learning

Authors: Zhenan Fan, Huang Fang, Zirui Zhou, Jian Pei, Michael P. Friedlander, Yong Zhang

Abstract: Federated learning is a popular technology for training machine learning models on distributed data sources without sharing data. Vertical federated learning or feature-based federated learning applies to the cases that different data sources share the same sample ID space but differ in feature space. To ensure the data owners' long-term engagement, it is critical to objectively assess the contrib… ▽ More Federated learning is a popular technology for training machine learning models on distributed data sources without sharing data. Vertical federated learning or feature-based federated learning applies to the cases that different data sources share the same sample ID space but differ in feature space. To ensure the data owners' long-term engagement, it is critical to objectively assess the contribution from each data source and recompense them accordingly. The Shapley value (SV) is a provably fair contribution valuation metric originated from cooperative game theory. However, computing the SV requires extensively retraining the model on each subset of data sources, which causes prohibitively high communication costs in federated learning. We propose a contribution valuation metric called vertical federated Shapley value (VerFedSV) based on SV. We show that VerFedSV not only satisfies many desirable properties for fairness but is also efficient to compute, and can be adapted to both synchronous and asynchronous vertical federated learning algorithms. Both theoretical analysis and extensive experimental results verify the fairness, efficiency, and adaptability of VerFedSV. △ Less

Submitted 7 January, 2022; originally announced January 2022.

Support Varieties and stable categories for algebraic groups

Authors: Eric M. Friedlander

Abstract: We consider rational representations of a connected linear algebraic group $\mathbb G$ over a field $k$ of positive characteristic $p > 0$. We introduce a natural extension $M \mapsto Π(\mathbb G)_M$ to $\mathbb G$-modules of the $π$-point support theory for modules $M$ for a finite group scheme $G$ and show that this theory is essentially equivalent to the more "intrinsic" and "explicit" theory… ▽ More We consider rational representations of a connected linear algebraic group $\mathbb G$ over a field $k$ of positive characteristic $p > 0$. We introduce a natural extension $M \mapsto Π(\mathbb G)_M$ to $\mathbb G$-modules of the $π$-point support theory for modules $M$ for a finite group scheme $G$ and show that this theory is essentially equivalent to the more "intrinsic" and "explicit" theory $M \mapsto \mathbb P\mathfrak C(\mathbb G)_M$ of supports for an algebraic group of exponential type, a theory which uses 1-parameter subgroups $\mathbb G_a \to \mathbb G$. We extend our support theory to bounded complexes of $\mathbb G$-modules, $C^\bullet \mapsto Π(\mathbb G)_{C^\bullet}$. We introduce the tensor triangulated category $StMod(\mathbb G)$, the Verdier quotient of the bounded derived category $D^b(Mod(\mathbb G))$ by the thick subcategory of mock injective modules. Our support theory satisfies all the standard properties" for a theory of supports for $StMod(\mathbb G)$. As an application, we employ $C^\bullet \mapsto Π(\mathbb G)_{C^\bullet}$ to establish the classification of $(r)$-complete, thick tensor ideals of $stmod(\mathbb G)$ in terms of $stmod(\mathbb G)$-realizable subsets of $Π(\mathbb G)$ and the classification of $(r)$-complete, localizing subcategories of $StMod(\mathbb G)$ in terms of $StMod(\mathbb G)$-realizable subsets of $Π(\mathbb G)$. △ Less

Submitted 23 May, 2022; v1 submitted 20 December, 2021; originally announced December 2021.

Comments: This version differs from the original in its organization, formulation of results, and proofs

Improving Fairness for Data Valuation in Horizontal Federated Learning

Authors: Zhenan Fan, Huang Fang, Zirui Zhou, Jian Pei, Michael P. Friedlander, Changxin Liu, Yong Zhang

Abstract: Federated learning is an emerging decentralized machine learning scheme that allows multiple data owners to work collaboratively while ensuring data privacy. The success of federated learning depends largely on the participation of data owners. To sustain and encourage data owners' participation, it is crucial to fairly evaluate the quality of the data provided by the data owners and reward them c… ▽ More Federated learning is an emerging decentralized machine learning scheme that allows multiple data owners to work collaboratively while ensuring data privacy. The success of federated learning depends largely on the participation of data owners. To sustain and encourage data owners' participation, it is crucial to fairly evaluate the quality of the data provided by the data owners and reward them correspondingly. Federated Shapley value, recently proposed by Wang et al. [Federated Learning, 2020], is a measure for data value under the framework of federated learning that satisfies many desired properties for data valuation. However, there are still factors of potential unfairness in the design of federated Shapley value because two data owners with the same local data may not receive the same evaluation. We propose a new measure called completed federated Shapley value to improve the fairness of federated Shapley value. The design depends on completing a matrix consisting of all the possible contributions by different subsets of the data owners. It is shown under mild conditions that this matrix is approximately low-rank by leveraging concepts and tools from optimization. Both theoretical analysis and empirical evaluation verify that the proposed measure does improve fairness in many circumstances. △ Less

Submitted 23 May, 2022; v1 submitted 18 September, 2021; originally announced September 2021.

Journal ref: 2022 IEEE 38th International Conference on Data Engineering (ICDE)

Support theory for Drinfeld doubles of some infinitesimal group schemes

Authors: Eric M. Friedlander, Cris Negron

Abstract: Consider a Frobenius kernel G in a split semisimple algebraic group, in very good characteristic. We provide an analysis of support for the Drinfeld center Z(rep(G)) of the representation category for G, or equivalently for the representation category of the Drinfeld double of kG. We show that thick ideals in the corresponding stable category are classified by cohomological support, and calculate… ▽ More Consider a Frobenius kernel G in a split semisimple algebraic group, in very good characteristic. We provide an analysis of support for the Drinfeld center Z(rep(G)) of the representation category for G, or equivalently for the representation category of the Drinfeld double of kG. We show that thick ideals in the corresponding stable category are classified by cohomological support, and calculate the Balmer spectrum of the stable category of Z(rep(G)). We also construct a $π$-point style rank variety for the Drinfeld double, identify $π$-point support with cohomological support, and show that both support theories satisfy the tensor product property. Our results hold, more generally, for Drinfeld doubles of Frobenius kernels in any smooth algebraic group which admits a quasi-logarithm, such as a Borel subgroup in a split semisimple group in very good characteristic. △ Less

Submitted 27 July, 2021; originally announced July 2021.

Comments: 44 pages

Journal ref: Alg. Number Th. 17 (2023) 217-260

Cardinality-constrained structured data-fitting problems

Authors: Zhenan Fan, Huang Fang, Michael P. Friedlander

Abstract: A memory-efficient framework is described for the cardinality-constrained structured data-fitting problem. Dual-based atom-identification rules are proposed that reveal the structure of the optimal primal solution from near-optimal dual solutions. These rules allow for a simple and computationally cheap algorithm for translating any feasible dual solution to a primal solution that satisfies the ca… ▽ More A memory-efficient framework is described for the cardinality-constrained structured data-fitting problem. Dual-based atom-identification rules are proposed that reveal the structure of the optimal primal solution from near-optimal dual solutions. These rules allow for a simple and computationally cheap algorithm for translating any feasible dual solution to a primal solution that satisfies the cardinality constraint. Rigorous guarantees are provided for obtaining a near-optimal primal solution given any dual-based method that generates dual iterates converging to an optimal dual solution. Numerical experiments on real-world datasets support confirm the analysis and demonstrate the efficiency of the proposed approach. △ Less

Submitted 19 July, 2022; v1 submitted 23 July, 2021; originally announced July 2021.

From perspective maps to epigraphical projections

Authors: Michael P. Friedlander, Ariel Goodwin, Tim Hoheisel

Abstract: The projection onto the epigraph or a level set of a closed proper convex function can be achieved by finding a root of a scalar equation that involves the proximal operator as a function of the proximal parameter. This paper develops the variational analysis of this scalar equation. The approach is based on a study of the variational-analytic properties of general convex optimization problems tha… ▽ More The projection onto the epigraph or a level set of a closed proper convex function can be achieved by finding a root of a scalar equation that involves the proximal operator as a function of the proximal parameter. This paper develops the variational analysis of this scalar equation. The approach is based on a study of the variational-analytic properties of general convex optimization problems that are (partial) infimal projections of the the sum of the function in question and the perspective map of a convex kernel. When the kernel is the Euclidean norm squared, the solution map corresponds to the proximal map, and thus the variational properties derived for the general case apply to the proximal case. Properties of the value function and the corresponding solution map -- including local Lipschitz continuity, directional differentiability, and semismoothness -- are derived. An SC$^1$ optimization framework for computing epigraphical and level-set projections is thus established. Numerical experiments on 1-norm projection illustrate the effectiveness of the approach as compared with specialized algorithms △ Less

Submitted 12 February, 2021; originally announced February 2021.

MSC Class: 52A4; 65K10; 90C25; 90C46

Support Theory for Extended Drinfeld Doubles

Authors: Eric M. Friedlander

Abstract: Following earlier work with Cris Negron on the cohomology of Drinfeld doubles $D(\mathbb G_{(r)})$, we develop a "geometric theory" of support varieties for "extended Drinfeld doubles" $\tilde D(\mathbb G_{(r)})$ of Frobenius kernels $\mathbb G_{(r)}$ of smooth linear algebraic groups $\mathbb G$ over a field $k$ of characteristic $p > 0$. To a $\tilde D(\mathbb G_{(r)})$-module $M$ we associate t… ▽ More Following earlier work with Cris Negron on the cohomology of Drinfeld doubles $D(\mathbb G_{(r)})$, we develop a "geometric theory" of support varieties for "extended Drinfeld doubles" $\tilde D(\mathbb G_{(r)})$ of Frobenius kernels $\mathbb G_{(r)}$ of smooth linear algebraic groups $\mathbb G$ over a field $k$ of characteristic $p > 0$. To a $\tilde D(\mathbb G_{(r)})$-module $M$ we associate the space $Π(\tilde D(\mathbb G_{(r)}))_M$ of equivalence classes of "pairs of $π$-points" and prove most of the desired properties of $M \mapsto Π(\tilde D(\mathbb G_{(r)}))_M$. Namely, this association satisfies the "tensor product property" and admits a natural continuous map $Ψ_{\tilde D}$ to cohomological support theory. Moreover, for $M$ finite dimensional and with suitable conditions on $\mathbb G_{(r)}$, this association provides a "projectivity test", $Ψ_{\tilde D}$ is a homeomorphism, and identifies $Π(\tilde D(\mathbb G_{(r)}))_M$ with the cohomological support variety of $M$ for various classes of $\tilde D(\mathbb G_{(r)})$-modules $M$. △ Less

Submitted 4 February, 2021; originally announced February 2021.

MSC Class: 16G99; 16S40; 16T05

NBIHT: An Efficient Algorithm for 1-bit Compressed Sensing with Optimal Error Decay Rate

Authors: Michael P. Friedlander, Halyun Jeong, Yaniv Plan, Ozgur Yilmaz

Abstract: The Binary Iterative Hard Thresholding (BIHT) algorithm is a popular reconstruction method for one-bit compressed sensing due to its simplicity and fast empirical convergence. There have been several works about BIHT but a theoretical understanding of the corresponding approximation error and convergence rate still remains open. This paper shows that the normalized version of BIHT (NBHIT) achiev… ▽ More The Binary Iterative Hard Thresholding (BIHT) algorithm is a popular reconstruction method for one-bit compressed sensing due to its simplicity and fast empirical convergence. There have been several works about BIHT but a theoretical understanding of the corresponding approximation error and convergence rate still remains open. This paper shows that the normalized version of BIHT (NBHIT) achieves an approximation error rate optimal up to logarithmic factors. More precisely, using $m$ one-bit measurements of an $s$-sparse vector $x$, we prove that the approximation error of NBIHT is of order $O \left(1 \over m \right)$ up to logarithmic factors, which matches the information-theoretic lower bound $Ω\left(1 \over m \right)$ proved by Jacques, Laska, Boufounos, and Baraniuk in 2013. To our knowledge, this is the first theoretical analysis of a BIHT-type algorithm that explains the optimal rate of error decay empirically observed in the literature. This also makes NBIHT the first provable computationally-efficient one-bit compressed sensing algorithm that breaks the inverse square root error decay rate $O \left(1 \over m^{1/2} \right)$. △ Less

Submitted 23 December, 2020; originally announced December 2020.

Comments: Submitted to a journal

MSC Class: 94-XX

Polar Deconvolution of Mixed Signals

Authors: Zhenan Fan, Halyun Jeong, Babhru Joshi, Michael P. Friedlander

Abstract: The signal demixing problem seeks to separate a superposition of multiple signals into its constituent components. This paper studies a two-stage approach that first decompresses and subsequently deconvolves the noisy and undersampled observations of the superposition using two convex programs. Probabilistic error bounds are given on the accuracy with which this process approximates the individual… ▽ More The signal demixing problem seeks to separate a superposition of multiple signals into its constituent components. This paper studies a two-stage approach that first decompresses and subsequently deconvolves the noisy and undersampled observations of the superposition using two convex programs. Probabilistic error bounds are given on the accuracy with which this process approximates the individual signals. The theory of polar convolution of convex sets and gauge functions plays a central role in the analysis and solution process. If the measurements are random and the noise is bounded, this approach stably recovers low-complexity and mutually incoherent signals, with high probability and with near-optimal sample complexity. We develop an efficient algorithm, based on level-set and conditional-gradient methods, that solves the convex optimization problems with sublinear iteration complexity and linear space requirements. Numerical experiments on both real and synthetic data confirm the theory and the efficiency of the approach. △ Less

Submitted 23 May, 2022; v1 submitted 14 October, 2020; originally announced October 2020.

Online mirror descent and dual averaging: kee** pace in the dynamic case

Authors: Huang Fang, Nicholas J. A. Harvey, Victor S. Portella, Michael P. Friedlander

Abstract: Online mirror descent (OMD) and dual averaging (DA) -- two fundamental algorithms for online convex optimization -- are known to have very similar (and sometimes identical) performance guarantees when used with a fixed learning rate. Under dynamic learning rates, however, OMD is provably inferior to DA and suffers a linear regret, even in common settings such as prediction with expert advice. We m… ▽ More Online mirror descent (OMD) and dual averaging (DA) -- two fundamental algorithms for online convex optimization -- are known to have very similar (and sometimes identical) performance guarantees when used with a fixed learning rate. Under dynamic learning rates, however, OMD is provably inferior to DA and suffers a linear regret, even in common settings such as prediction with expert advice. We modify the OMD algorithm through a simple technique that we call stabilization. We give essentially the same abstract regret bound for OMD with stabilization and for DA by modifying the classical OMD convergence analysis in a careful and modular way that allows for straightforward and flexible proofs. Simple corollaries of these bounds show that OMD with stabilization and DA enjoy the same performance guarantees in many applications -- even under dynamic learning rates. We also shed light on the similarities between OMD and DA and show simple conditions under which stabilized-OMD and DA generate the same iterates. △ Less

Submitted 3 September, 2021; v1 submitted 3 June, 2020; originally announced June 2020.

Comments: 27 pages main text, 37 pages in total, 1 figure. Version 2: Revision for camera-ready version of ICML 2020, with a new abstract, new discussion and acknowledgements sections, and some other minor modifications. Version 3: Technical report version of JMLR submission, with minor revisions, full proofs, and more details on the setting with composite functions

Approximate methods for phase retrieval via gauge duality

Authors: Ron Estrin, Yifan Sun, Halyun Jeong, Michael Friedlander

Abstract: We consider the problem of finding a low rank symmetric matrix satisfying a system of linear equations, as appears in phase retrieval. In particular, we solve the gauge dual formulation, but use a fast approximation of the spectral computations to achieve a noisy solution estimate. This estimate is then used as the initialization of an alternating gradient descent scheme over a nonconvex rank-1 ma… ▽ More We consider the problem of finding a low rank symmetric matrix satisfying a system of linear equations, as appears in phase retrieval. In particular, we solve the gauge dual formulation, but use a fast approximation of the spectral computations to achieve a noisy solution estimate. This estimate is then used as the initialization of an alternating gradient descent scheme over a nonconvex rank-1 matrix factorization formulation. Numerical results on small problems show consistent recovery, with very low computational cost. △ Less

Submitted 1 June, 2020; originally announced June 2020.

MSC Class: 49M29 ACM Class: G.1.6

A perturbation view of level-set methods for convex optimization

Authors: Ron Estrin, Michael P. Friedlander

Abstract: Level-set methods for convex optimization are predicated on the idea that certain problems can be parameterized so that their solutions can be recovered as the limiting process of a root-finding procedure. This idea emerges time and again across a range of algorithms for convex problems. Here we demonstrate that strong duality is a necessary condition for the level-set approach to succeed. In the… ▽ More Level-set methods for convex optimization are predicated on the idea that certain problems can be parameterized so that their solutions can be recovered as the limiting process of a root-finding procedure. This idea emerges time and again across a range of algorithms for convex problems. Here we demonstrate that strong duality is a necessary condition for the level-set approach to succeed. In the absence of strong duality, the level-set method identifies $ε$-infeasible points that do not converge to a feasible point as $ε$ tends to zero. The level-set approach is also used as a proof technique for establishing sufficient conditions for strong duality that are different from Slater's constraint qualification. △ Less

Submitted 15 May, 2020; v1 submitted 17 January, 2020; originally announced January 2020.

MSC Class: 90C25; 65K10; 49M29

Polar Alignment and Atomic Decomposition

Authors: Zhenan Fan, Halyun Jeong, Yifan Sun, Michael P. Friedlander

Abstract: Structured optimization uses a prescribed set of atoms to assemble a solution that fits a model to data. Polarity, which extends the familiar notion of orthogonality from linear sets to general convex sets, plays a special role in a simple and geometric form of convex duality. This duality correspondence yields a general notion of alignment that leads to an intuitive and complete description of ho… ▽ More Structured optimization uses a prescribed set of atoms to assemble a solution that fits a model to data. Polarity, which extends the familiar notion of orthogonality from linear sets to general convex sets, plays a special role in a simple and geometric form of convex duality. This duality correspondence yields a general notion of alignment that leads to an intuitive and complete description of how atoms participate in the final decomposition of the solution. The resulting geometric perspective leads to variations of existing algorithms effective for large-scale problems. We illustrate these ideas with many examples, including applications in matrix completion and morphological component analysis for the separation of mixtures of signals. △ Less

Submitted 10 December, 2019; originally announced December 2019.

Comments: 39 pages

MSC Class: 90C25; 90C22; 65K05; 65F99

Implementing a smooth exact penalty function for general constrained nonlinear optimization

Authors: Ron Estrin, Michael Friedlander, Dominique Orban, Michael Saunders

Abstract: We build upon Estrin et al. (2019) to develop a general constrained nonlinear optimization algorithm based on a smooth penalty function proposed by Fletcher (1970, 1973b). Although Fletcher's approach has historically been considered impractical, we show that the computational kernels required are no more expensive than those in other widely accepted methods for nonlinear optimization. The main ke… ▽ More We build upon Estrin et al. (2019) to develop a general constrained nonlinear optimization algorithm based on a smooth penalty function proposed by Fletcher (1970, 1973b). Although Fletcher's approach has historically been considered impractical, we show that the computational kernels required are no more expensive than those in other widely accepted methods for nonlinear optimization. The main kernel for evaluating the penalty function and its derivatives solves structured linear systems. When the matrices are available explicitly, we store a single factorization each iteration. Otherwise, we obtain a factorization-free optimization algorithm by solving each linear system iteratively. The penalty function shows promise in cases where the linear systems can be solved efficiently, e.g., PDE-constrained optimization problems when efficient preconditioners exist. We demonstrate the merits of the approach, and give numerical results on several PDE-constrained and standard test problems. △ Less

Submitted 3 December, 2019; originally announced December 2019.

Comments: 25 pages. arXiv admin note: text overlap with arXiv:1910.04300

Report number: Cahier du Gerad G-2019-27

Journal ref: SIAM J. Sci. Comput., 42(3), A1836-A1859, 2020

Bundle methods for dual atomic pursuit

Authors: Zhenan Fan, Yifan Sun, Michael P. Friedlander

Abstract: The aim of structured optimization is to assemble a solution, using a given set of (possibly uncountably infinite) atoms, to fit a model to data. A two-stage algorithm based on gauge duality and bundle method is proposed. The first stage discovers the optimal atomic support for the primal problem by solving a sequence of approximations of the dual problem using a bundle-type method. The second sta… ▽ More The aim of structured optimization is to assemble a solution, using a given set of (possibly uncountably infinite) atoms, to fit a model to data. A two-stage algorithm based on gauge duality and bundle method is proposed. The first stage discovers the optimal atomic support for the primal problem by solving a sequence of approximations of the dual problem using a bundle-type method. The second stage recovers the approximate primal solution using the atoms discovered in the first stage. The overall approach leads to implementable and efficient algorithms for large problems. △ Less

Submitted 2 November, 2019; v1 submitted 29 October, 2019; originally announced October 2019.

Comments: 53rd Annual Asilomar Conference on Signals, Systems, and Computers

Implementing a smooth exact penalty function for equality-constrained nonlinear optimization

Authors: Ron Estrin, Michael P. Friedlander, Dominique Orban, Michael A. Saunders

Abstract: We develop a general equality-constrained nonlinear optimization algorithm based on a smooth penalty function proposed by Fletcher (1970). Although it was historically considered to be computationally prohibitive in practice, we demonstrate that the computational kernels required are no more expensive than other widely accepted methods for nonlinear optimization. The main kernel required to evalua… ▽ More We develop a general equality-constrained nonlinear optimization algorithm based on a smooth penalty function proposed by Fletcher (1970). Although it was historically considered to be computationally prohibitive in practice, we demonstrate that the computational kernels required are no more expensive than other widely accepted methods for nonlinear optimization. The main kernel required to evaluate the penalty function and its derivatives is solving a structured linear system. We show how to solve this system efficiently by storing a single factorization each iteration when the matrices are available explicitly. We further show how to adapt the penalty function to the class of factorization-free algorithms by solving the linear system iteratively. The penalty function therefore has promise when the linear system can be solved efficiently, e.g., for PDE-constrained optimization problems where efficient preconditioners exist. We discuss extensions including handling simple constraints explicitly, regularizing the penalty function, and inexact evaluation of the penalty function and its gradients. We demonstrate the merits of the approach and its various features on some nonlinear programs from a standard test set, and some PDE-constrained optimization problems. △ Less

Submitted 9 October, 2019; originally announced October 2019.

Report number: Cahier du GERAD G-2019-04

Journal ref: SIAM J. Sci. Comput., 42(3), A1809-A1835, 2020

Geometric Invariants of Representations of Finite Groups

Authors: Eric M. Friedlander

Abstract: J. Pevtsova and the author constructed a ``universal $p$-nilpotent operator" for an infinitesimal group scheme $G$ over a field $k$ of characteristic $p > 0$ which led to coherent sheaves on the scheme of 1-parameter subgroups of $G$ associated to a $G$-module $M$. Of special interest is the fact that these coherent sheaves are vector bundles if $M$ is of constant Jordan type. In this paper, we pr… ▽ More J. Pevtsova and the author constructed a ``universal $p$-nilpotent operator" for an infinitesimal group scheme $G$ over a field $k$ of characteristic $p > 0$ which led to coherent sheaves on the scheme of 1-parameter subgroups of $G$ associated to a $G$-module $M$. Of special interest is the fact that these coherent sheaves are vector bundles if $M$ is of constant Jordan type. In this paper, we provide similar invariants for a finite group $τ$ which recover the invariants earlier obtained for elementary abelian $p$-groups. To do this, we replace the analogue of 1-parameter subgroups by a refined version of equivalence classes of $π$-points for $kτ$. More generally, we provide a construction of vector bundles for the semi-direct product $G\rtimes τ$ of an infinitesimal group scheme $G$ and a finite group $τ$. A major motivation for this study is to further our understanding of the relationship between representations of $\mathbb G(\mathbb F_p)$ and $\mathbb G_{(r)}$ associated to a finite dimensional rational $\mathbb G$-module $M$, where $\mathbb G$ is a reductive group with $r$-th Fobenius kernel $\mathbb G_{(r)}$. Using vector bundles, we extend and sharpen earlier results comparing support varieties. △ Less

Submitted 16 June, 2019; originally announced June 2019.

MSC Class: 20G05; 20C20; 20G10

Quantum Algorithms for Structured Prediction

Authors: Behrooz Sepehry, Ehsan Iranmanesh, Michael P. Friedlander, Pooya Ronagh

Abstract: We introduce two quantum algorithms for solving structured prediction problems. We first show that a stochastic gradient descent that uses the quantum minimum finding algorithm and takes its probabilistic failure into account solves the structured prediction problem with a runtime that scales with the square root of the size of the label space, and in $\widetilde O\left(1/ε\right)$ with respect to… ▽ More We introduce two quantum algorithms for solving structured prediction problems. We first show that a stochastic gradient descent that uses the quantum minimum finding algorithm and takes its probabilistic failure into account solves the structured prediction problem with a runtime that scales with the square root of the size of the label space, and in $\widetilde O\left(1/ε\right)$ with respect to the precision, $ε$, of the solution. Motivated by robust inference techniques in machine learning, we then introduce another quantum algorithm that solves a smooth approximation of the structured prediction problem with a similar quantum speedup in the size of the label space and a similar scaling in the precision parameter. In doing so, we analyze a variant of stochastic gradient descent for convex optimization in the presence of an additive error in the calculation of the gradients, and show that its convergence rate does not deteriorate if the additive errors are of the order $O(\sqrtε)$. This algorithm uses quantum Gibbs sampling at temperature $Ω(ε)$ as a subroutine. Based on these theoretical observations, we propose a method for using quantum Gibbs samplers to combine feedforward neural networks with probabilistic graphical models for quantum machine learning. Our numerical results using Monte Carlo simulations on an image tagging task demonstrate the benefit of the approach. △ Less

Submitted 1 July, 2021; v1 submitted 11 September, 2018; originally announced September 2018.

Polar Convolution

Authors: Michael P. Friedlander, Ives Macêdo, Ting Kei Pong

Abstract: The Moreau envelope is one of the key convexity-preserving functional operations in convex analysis, and it is central to the development and analysis of many approaches for convex optimization. This paper develops the theory for an analogous convolution operation, called the polar envelope, specialized to gauge functions. Many important properties of the Moreau envelope and the proximal map are m… ▽ More The Moreau envelope is one of the key convexity-preserving functional operations in convex analysis, and it is central to the development and analysis of many approaches for convex optimization. This paper develops the theory for an analogous convolution operation, called the polar envelope, specialized to gauge functions. Many important properties of the Moreau envelope and the proximal map are mirrored by the polar envelope and its corresponding proximal map. These properties include smoothness of the envelope function, uniqueness and continuity of the proximal map, which play important roles in duality and in the construction of algorithms for gauge optimization. A suite of tools with which to build algorithms for this family of optimization problems is thus established. △ Less

Submitted 3 February, 2019; v1 submitted 21 August, 2018; originally announced August 2018.

MSC Class: 90C15; 90C25

Rational Cohomology and Supports for Linear Algebraic Groups

Authors: Eric M. Friedlander

Abstract: This paper is an extended version of four lectures at PIMS in Vancouver given June 27 - 30, 2016. The primary goal of these lectures was to publicize the author's recent efforts to extend to representations of linear algebraic groups the "theory of support varieties" which has proved successful in the study of representations of finite group schemes. The lectures offer readers an introduction to t… ▽ More This paper is an extended version of four lectures at PIMS in Vancouver given June 27 - 30, 2016. The primary goal of these lectures was to publicize the author's recent efforts to extend to representations of linear algebraic groups the "theory of support varieties" which has proved successful in the study of representations of finite group schemes. The lectures offer readers an introduction to the subject together with "homework problems, simplify and clarify some points in the literature, and mention some directions for future research. △ Less

Submitted 18 December, 2017; originally announced December 2017.

Comments: This will appear in "Geometric and Toplogical Aspects of Representation Theory of Finite Groups", published by Springer-Verlag

MSC Class: 20G05; 20C20; 20G10

Foundations of gauge and perspective duality

Authors: Alexandre Y. Aravkin, James V. Burke, Dmitriy Drusvyatskiy, Michael P. Friedlander, Kellie MacPhee

Abstract: We revisit the foundations of gauge duality and demonstrate that it can be explained using a modern approach to duality based on a perturbation framework. We therefore put gauge duality and Fenchel-Rockafellar duality on equal footing, including explaining gauge dual variables as sensitivity measures, and showing how to recover primal solutions from those of the gauge dual. This vantage point allo… ▽ More We revisit the foundations of gauge duality and demonstrate that it can be explained using a modern approach to duality based on a perturbation framework. We therefore put gauge duality and Fenchel-Rockafellar duality on equal footing, including explaining gauge dual variables as sensitivity measures, and showing how to recover primal solutions from those of the gauge dual. This vantage point allows a direct proof that optimal solutions of the Fenchel-Rockafellar dual of the gauge dual are precisely the primal solutions rescaled by the optimal value. We extend the gauge duality framework to the setting in which the functional components are general nonnegative convex functions, including problems with piecewise linear quadratic functions and constraints that arise from generalized linear models used in regression. △ Less

Submitted 18 June, 2018; v1 submitted 28 February, 2017; originally announced February 2017.

Comments: 29 pages

Cohomology of unipotent group schemes

Authors: Eric M. Friedlander

Abstract: We verify that universal classes in the cohomology of $GL_N$ determine explicit cohomology classes of Frobenius kernels $G_{(r)}$ of various linear algebraic groups $G$ . We consider the relationship of $\varprojlim_r H^*(U_{(r)},k)$ to the rational cohomology $H^*(U,k)$ of many unipotent algebraic groups $U$. The second half of this paper investigates in detail the cohomology of Frobenius kernels… ▽ More We verify that universal classes in the cohomology of $GL_N$ determine explicit cohomology classes of Frobenius kernels $G_{(r)}$ of various linear algebraic groups $G$ . We consider the relationship of $\varprojlim_r H^*(U_{(r)},k)$ to the rational cohomology $H^*(U,k)$ of many unipotent algebraic groups $U$. The second half of this paper investigates in detail the cohomology of Frobenius kernels $(U_3)_{(r)}$ of the Heisenberg group $U_3 \subset GL_3$. △ Less

Submitted 11 July, 2019; v1 submitted 15 February, 2017; originally announced February 2017.

Comments: Total revision of previous version: new title and abstract; new ordering of material; some results limited to the special case of the Heisenberg group

MSC Class: 20G05; 20C20; 20G10

The Bayesian analysis of contingency table data using the bayesloglin R package

Authors: Matthew Friedlander

Abstract: For log-linear analysis, the hyper Dirichlet conjugate prior is available to work in the Bayesian paradigm. With this prior, the MC3 algorithm allows for exploration of the space of models to try to find those with the highest posterior probability. Once top models have been identified, a block Gibbs sampler can be constructed to sample from the posterior distribution and to estimate parameters of… ▽ More For log-linear analysis, the hyper Dirichlet conjugate prior is available to work in the Bayesian paradigm. With this prior, the MC3 algorithm allows for exploration of the space of models to try to find those with the highest posterior probability. Once top models have been identified, a block Gibbs sampler can be constructed to sample from the posterior distribution and to estimate parameters of interest. Our aim in this paper, is to introduce the bayesloglin R package \citep{R} which contains functions to carry out these tasks. △ Less

Submitted 16 December, 2016; originally announced December 2016.

Analyzing Genome-wide Association Study Data with the R Package genMOSS

Authors: Matthew Friedlander, Adrian Dobra, Helene Massam, Laurent Briollais

Abstract: The R package (R Core Team (2016)) genMOSS is specifically designed for the Bayesian analysis of genome-wide association study data. The package implements the mode oriented stochastic search (MOSS) procedure as well as a simple moving window approach to identify combinations of single nucleotide polymorphisms associated with a response. The prior used in Bayesian computations is the generalized h… ▽ More The R package (R Core Team (2016)) genMOSS is specifically designed for the Bayesian analysis of genome-wide association study data. The package implements the mode oriented stochastic search (MOSS) procedure as well as a simple moving window approach to identify combinations of single nucleotide polymorphisms associated with a response. The prior used in Bayesian computations is the generalized hyper Dirichlet. △ Less

Submitted 22 November, 2016; originally announced November 2016.

Comments: 10 pages

MSC Class: 62-07 (Primary)

Fitting log-linear models in sparse contingency tables using the eMLEloglin R package

Authors: Matthew Friedlander

Abstract: Log-linear modeling is a popular method for the analysis of contingency table data. When the table is sparse, and the data falls on a proper face $F$ of the convex support, there are consequences on model inference and model selection. Knowledge of the cells determining $F$ is crucial to mitigating these effects. We introduce the R package (R Core Team (2016)) eMLEloglin for determining $F$ and pa… ▽ More Log-linear modeling is a popular method for the analysis of contingency table data. When the table is sparse, and the data falls on a proper face $F$ of the convex support, there are consequences on model inference and model selection. Knowledge of the cells determining $F$ is crucial to mitigating these effects. We introduce the R package (R Core Team (2016)) eMLEloglin for determining $F$ and passing that information on to the glm package to fit the model properly. △ Less

Submitted 16 December, 2016; v1 submitted 22 November, 2016; originally announced November 2016.

Comments: 13 pages

MSC Class: 62H17 (Primary)

Satisfying Real-world Goals with Dataset Constraints

Authors: Gabriel Goh, Andrew Cotter, Maya Gupta, Michael Friedlander

Abstract: The goal of minimizing misclassification error on a training set is often just one of several real-world goals that might be defined on different datasets. For example, one may require a classifier to also make positive predictions at some specified rate for some subpopulation (fairness), or to achieve a specified empirical recall. Other real-world goals include reducing churn with respect to a pr… ▽ More The goal of minimizing misclassification error on a training set is often just one of several real-world goals that might be defined on different datasets. For example, one may require a classifier to also make positive predictions at some specified rate for some subpopulation (fairness), or to achieve a specified empirical recall. Other real-world goals include reducing churn with respect to a previously deployed model, or stabilizing online training. In this paper we propose handling multiple goals on multiple datasets by training with dataset constraints, using the ramp penalty to accurately quantify costs, and present an efficient algorithm to approximately optimize the resulting non-convex constrained optimization problem. Experiments on both benchmark and real-world industry datasets demonstrate the effectiveness of our approach. △ Less

Submitted 3 May, 2017; v1 submitted 23 June, 2016; originally announced June 2016.

Efficient evaluation of scaled proximal operators

Authors: Michael P. Friedlander, Gabriel Goh

Abstract: Quadratic-support functions [Aravkin, Burke, and Pillonetto; J. Mach. Learn. Res. 14(1), 2013] constitute a parametric family of convex functions that includes a range of useful regularization terms found in applications of convex optimization. We show how an interior method can be used to efficiently compute the proximal operator of a quadratic-support function under different metrics. When the m… ▽ More Quadratic-support functions [Aravkin, Burke, and Pillonetto; J. Mach. Learn. Res. 14(1), 2013] constitute a parametric family of convex functions that includes a range of useful regularization terms found in applications of convex optimization. We show how an interior method can be used to efficiently compute the proximal operator of a quadratic-support function under different metrics. When the metric and the function have the right structure, the proximal map can be computed with cost nearly linear in the input size. We describe how to use this approach to implement quasi-Newton methods for a rich class of nonsmooth problems that arise, for example, in sparse optimization, image denoising, and sparse logistic regression. △ Less

Submitted 19 December, 2016; v1 submitted 17 March, 2016; originally announced March 2016.

Comments: 23 pages

Journal ref: Electronic Transactions on Numerical Analysis, 46:1-22, 2017

Level-set methods for convex optimization

Authors: Aleksandr Y. Aravkin, James V. Burke, Dmitriy Drusvyatskiy, Michael P. Friedlander, Scott Roy

Abstract: Convex optimization problems arising in applications often have favorable objective functions and complicated constraints, thereby precluding first-order methods from being immediately applicable. We describe an approach that exchanges the roles of the objective and constraint functions, and instead approximately solves a sequence of parametric level-set problems. A zero-finding procedure, based o… ▽ More Convex optimization problems arising in applications often have favorable objective functions and complicated constraints, thereby precluding first-order methods from being immediately applicable. We describe an approach that exchanges the roles of the objective and constraint functions, and instead approximately solves a sequence of parametric level-set problems. A zero-finding procedure, based on inexact function evaluations and possibly inexact derivative information, leads to an efficient solution scheme for the original problem. We describe the theoretical and practical properties of this approach for a broad range of problems, including low-rank semidefinite optimization, sparse optimization, and generalized linear models for inference. △ Less

Submitted 3 February, 2016; originally announced February 2016.

Comments: 38 pages

Low-rank spectral optimization via gauge duality

Authors: Michael P. Friedlander, Ives Macedo

Abstract: Various applications in signal processing and machine learning give rise to highly structured spectral optimization problems characterized by low-rank solutions. Two important examples that motivate this work are optimization problems from phase retrieval and from blind deconvolution, which are designed to yield rank-1 solutions. An algorithm is described that is based on solving a certain constra… ▽ More Various applications in signal processing and machine learning give rise to highly structured spectral optimization problems characterized by low-rank solutions. Two important examples that motivate this work are optimization problems from phase retrieval and from blind deconvolution, which are designed to yield rank-1 solutions. An algorithm is described that is based on solving a certain constrained eigenvalue optimization problem that corresponds to the gauge dual which, unlike the more typical Lagrange dual, has an especially simple constraint. The dominant cost at each iteration is the computation of rightmost eigenpairs of a Hermitian operator. A range of numerical examples illustrate the scalability of the approach. △ Less

Submitted 23 March, 2016; v1 submitted 3 August, 2015; originally announced August 2015.

Comments: Final version. To appear in SIAM J. Scientific Computing

MSC Class: 90C15; 90C25

Journal ref: SIAM Journal on Scientific Computing, 38(3):A1616-A1638, 2016

Coordinate Descent Converges Faster with the Gauss-Southwell Rule Than Random Selection

Authors: Julie Nutini, Mark Schmidt, Issam H. Laradji, Michael Friedlander, Hoyt Koepke

Abstract: There has been significant recent work on the theory and application of randomized coordinate descent algorithms, beginning with the work of Nesterov [SIAM J. Optim., 22(2), 2012], who showed that a random-coordinate selection rule achieves the same convergence rate as the Gauss-Southwell selection rule. This result suggests that we should never use the Gauss-Southwell rule, as it is typically muc… ▽ More There has been significant recent work on the theory and application of randomized coordinate descent algorithms, beginning with the work of Nesterov [SIAM J. Optim., 22(2), 2012], who showed that a random-coordinate selection rule achieves the same convergence rate as the Gauss-Southwell selection rule. This result suggests that we should never use the Gauss-Southwell rule, as it is typically much more expensive than random selection. However, the empirical behaviours of these algorithms contradict this theoretical result: in applications where the computational costs of the selection rules are comparable, the Gauss-Southwell selection rule tends to perform substantially better than random coordinate selection. We give a simple analysis of the Gauss-Southwell rule showing that---except in extreme cases---its convergence rate is faster than choosing random coordinates. Further, in this work we (i) show that exact coordinate optimization improves the convergence rate for certain sparse problems, (ii) propose a Gauss-Southwell-Lipschitz rule that gives an even faster convergence rate given knowledge of the Lipschitz constants of the partial derivatives, (iii) analyze the effect of approximate Gauss-Southwell rules, and (iv) analyze proximal-gradient variants of the Gauss-Southwell rule. △ Less

Submitted 28 October, 2018; v1 submitted 1 June, 2015; originally announced June 2015.

Comments: ICML 2015. v2: Updated the Gauss-Southwell-q result in Section 8 and Appendix H, to remove the part depending on mu_1 (the proof had an error). Added Section 8.1, which discusses conditions under which a rate depending on mu_1 does hold

Vector Bundles Associated to Lie Algebras

Authors: Jon F. Carlson, Eric M. Friedlander, Julia Pevtsova

Abstract: We introduce and investigate a functorial construction which associates coherent sheaves to finite dimensional (restricted) representations of a restricted Lie algebra $\mathfrak g$. These are sheaves on locally closed subvarieties of the projective variety $\mathbb E(r,\mathfrak g)$ of elementary subalgebras of $\mathfrak g$ of dimension $r$. We show that representations of constant radical or so… ▽ More We introduce and investigate a functorial construction which associates coherent sheaves to finite dimensional (restricted) representations of a restricted Lie algebra $\mathfrak g$. These are sheaves on locally closed subvarieties of the projective variety $\mathbb E(r,\mathfrak g)$ of elementary subalgebras of $\mathfrak g$ of dimension $r$. We show that representations of constant radical or socle rank studied in \cite{CFP3} which generalize modules of constant Jordan type lead to algebraic vector bundles on $\mathbb E(r,\mathfrak g)$. For $\mathfrak g = Lie(G)$, the Lie algebra of an algebraic group $G$, rational representations of $G$ enable us to realize familiar algebraic vector bundles on $G$-orbits of $\mathbb E(r, \mathfrak g)$. △ Less

Submitted 18 August, 2014; originally announced August 2014.

Comments: Replaces second half of arXiv:1207.5898 . To appear in Crelle

MSC Class: 17B50; 16G10

Elementary Subalgebrs of Lie Algebras

Authors: Jon F. Carlson, Eric M. Friedlander, Julia Pevtsova

Abstract: We initiate the investigation of the projective varieties $\mathbb E(r,\mathfrak g)$ of elementary subalgebras of dimension $r$ of a ($p$-restricted) Lie algebra $\mathfrak g$ for various $r \geq 1$. These varieties $\mathbb E(r,\mathfrak g)$ are the natural ambient varieties for generalized support varieties for restricted representations of $\mathfrak g$. We identify these varieties in special c… ▽ More We initiate the investigation of the projective varieties $\mathbb E(r,\mathfrak g)$ of elementary subalgebras of dimension $r$ of a ($p$-restricted) Lie algebra $\mathfrak g$ for various $r \geq 1$. These varieties $\mathbb E(r,\mathfrak g)$ are the natural ambient varieties for generalized support varieties for restricted representations of $\mathfrak g$. We identify these varieties in special cases, revealing their interesting and varied geometric structures. We also introduce invariants for a finite dimensional $\mathfrak u(\mathfrak g)$-module $M$, the local $(r,j)$-radical rank and local $(r,j)$-socle rank, functions which are lower/upper semicontinuous on $\mathbb E(r,\mathfrak g)$. Examples are given of $\mathfrak u(\mathfrak g)$-modules for which some of these rank functions are constant. △ Less

Submitted 18 August, 2014; originally announced August 2014.

Comments: Replaces 1st half of arXiv:1207.5898 (with same title). To appear in the Journal of Algebra

MSC Class: 17B50; 16G10

Filtrations, 1-parameter Subgroups, and Rational Injectivity

Authors: Eric M. Friedlander

Abstract: We investigate rational $G$-modules $M$ for a linear algebraic group $G$ over an algebraically closed field $k$ of characteristic $p > 0$ using filtrations by sub-coalgebras of the coordinate algebra $k[G]$ of $G$. Even in the special case of the additive group $\mathbb G_a$, interesting structures and examples are revealed. The "degree" filtration we consider for unipotent algebraic groups leads… ▽ More We investigate rational $G$-modules $M$ for a linear algebraic group $G$ over an algebraically closed field $k$ of characteristic $p > 0$ using filtrations by sub-coalgebras of the coordinate algebra $k[G]$ of $G$. Even in the special case of the additive group $\mathbb G_a$, interesting structures and examples are revealed. The "degree" filtration we consider for unipotent algebraic groups leads to a "filtration by exponential degree" applicable to rational $G$ modules for any linear algebraic group $G$ of exponential type; this filtration is defined in terms of 1-parameter subgroups and is related to support varieties introduced recently by the author for such rational $G$-modules. We formulate in terms of this filtration a necessary and sufficient condition for rational injectivity for rational $G$-modules. Our investigation leads to the consideration of two new classes of rational $G$-modules: those that are "mock injective" and those that are "mock trivial". △ Less

Submitted 25 October, 2015; v1 submitted 13 August, 2014; originally announced August 2014.

Comments: Slight title change, exposition drastically revised, added discussion of mock injectives and mock trivial modules

MSC Class: 20G05; 20C20; 20G10

Support varieties for rational representations

Authors: Eric M. Friedlander

Abstract: We introduce support varieties for rational representations of a linear algebraic group $G$ of exponential type over an algebraically closed field $k$ of characteristic $p > 0$. These varieties are closed subspaces of the space $V(G)$ of all 1-parameter subgroups of $G$. The functor $M \mapsto V(G)_M$ satisfies many of the standard properties of support varieties satisfied by finite groups and oth… ▽ More We introduce support varieties for rational representations of a linear algebraic group $G$ of exponential type over an algebraically closed field $k$ of characteristic $p > 0$. These varieties are closed subspaces of the space $V(G)$ of all 1-parameter subgroups of $G$. The functor $M \mapsto V(G)_M$ satisfies many of the standard properties of support varieties satisfied by finite groups and other finite group schemes. Furthermore, there is a close relationship between $V(G)_M$ and the family of support varieties $V_r(G)_M$ obtained by restricting the $G$ action to Frobenius kernels $G_{(r)} \subset G$. These support varieties seem particularly appropriate for the investigation of infinite dimensional rational $G$-modules. △ Less

Submitted 29 June, 2014; originally announced June 2014.

MSC Class: Primary: 20G05; secondary: 20C20; 20G10

Journal ref: Compositio Math. 151 (2015) 765-792

An approach to intersection theory on singular varieties using motivic complexes

Authors: Eric M. Friedlander, Joseph Ross

Abstract: We introduce techniques of Suslin, Voevodsky, and others into the study of singular varieties. Our approach is modeled after Goresky-MacPherson intersection homology. We provide a formulation of perversity cycle spaces leading to perversity homology theory and a companion perversity cohomology theory based upon generalized cocycle spaces. These theories lead to conditions on pairs of cycles which… ▽ More We introduce techniques of Suslin, Voevodsky, and others into the study of singular varieties. Our approach is modeled after Goresky-MacPherson intersection homology. We provide a formulation of perversity cycle spaces leading to perversity homology theory and a companion perversity cohomology theory based upon generalized cocycle spaces. These theories lead to conditions on pairs of cycles which can be intersected and a suitable equivalence relation on cocycles/cycles enabling pairings on equivalence classes. We establish suspension and splitting theorems, as well as a localization property. Some examples of intersections on singular varieties are computed. △ Less

Submitted 20 May, 2016; v1 submitted 21 November, 2013; originally announced November 2013.

Comments: revised version, to appear in Compositio Mathematica

Journal ref: Compositio Math. 152 (2016) 2371-2404

Gauge optimization and duality

Authors: Michael P. Friedlander, Ives Macedo, Ting Kei Pong

Abstract: Gauge functions significantly generalize the notion of a norm, and gauge optimization, as defined by Freund (1987}, seeks the element of a convex set that is minimal with respect to a gauge function. This conceptually simple problem can be used to model a remarkable array of useful problems, including a special case of conic optimization, and related problems that arise in machine learning and sig… ▽ More Gauge functions significantly generalize the notion of a norm, and gauge optimization, as defined by Freund (1987}, seeks the element of a convex set that is minimal with respect to a gauge function. This conceptually simple problem can be used to model a remarkable array of useful problems, including a special case of conic optimization, and related problems that arise in machine learning and signal processing. The gauge structure of these problems allows for a special kind of duality framework. This paper explores the duality framework proposed by Freund, and proposes a particular form of the problem that exposes some useful properties of the gauge optimization framework (such as the variational properties of its value function), and yet maintains most of the generality of the abstract form of gauge optimization. △ Less

Submitted 20 March, 2014; v1 submitted 9 October, 2013; originally announced October 2013.

Comments: 24 pp

Journal ref: SIAM Journal on Optimization, 24(4):1999-2022, 2014

Fast Dual Variational Inference for Non-Conjugate LGMs

Authors: Mohammad Emtiyaz Khan, Aleksandr Y. Aravkin, Michael P. Friedlander, Matthias Seeger

Abstract: Latent Gaussian models (LGMs) are widely used in statistics and machine learning. Bayesian inference in non-conjugate LGMs is difficult due to intractable integrals involving the Gaussian prior and non-conjugate likelihoods. Algorithms based on variational Gaussian (VG) approximations are widely employed since they strike a favorable balance between accuracy, generality, speed, and ease of use. Ho… ▽ More Latent Gaussian models (LGMs) are widely used in statistics and machine learning. Bayesian inference in non-conjugate LGMs is difficult due to intractable integrals involving the Gaussian prior and non-conjugate likelihoods. Algorithms based on variational Gaussian (VG) approximations are widely employed since they strike a favorable balance between accuracy, generality, speed, and ease of use. However, the structure of the optimization problems associated with these approximations remains poorly understood, and standard solvers take too long to converge. We derive a novel dual variational inference approach that exploits the convexity property of the VG approximations. We obtain an algorithm that solves a convex optimization problem, reduces the number of variational parameters, and converges much faster than previous methods. Using real-world data, we demonstrate these advantages on a variety of LGMs, including Gaussian process classification, and latent Gaussian Markov random fields. △ Less

Submitted 5 June, 2013; originally announced June 2013.

Comments: 9 pages, 3 figures

MSC Class: 62F15; 65K10; 49M29; 90C06

Tail bounds for stochastic approximation

Authors: Michael P. Friedlander, Gabriel Goh

Abstract: Stochastic-approximation gradient methods are attractive for large-scale convex optimization because they offer inexpensive iterations. They are especially popular in data-fitting and machine-learning applications where the data arrives in a continuous stream, or it is necessary to minimize large sums of functions. It is known that by appropriately decreasing the variance of the error at each iter… ▽ More Stochastic-approximation gradient methods are attractive for large-scale convex optimization because they offer inexpensive iterations. They are especially popular in data-fitting and machine-learning applications where the data arrives in a continuous stream, or it is necessary to minimize large sums of functions. It is known that by appropriately decreasing the variance of the error at each iteration, the expected rate of convergence matches that of the underlying deterministic gradient method. Conditions are given under which this happens with overwhelming probability. △ Less

Submitted 8 January, 2014; v1 submitted 20 April, 2013; originally announced April 2013.

Variational properties of value functions

Authors: Aleksandr Y. Aravkin, James V. Burke, Michael P. Friedlander

Abstract: Regularization plays a key role in a variety of optimization formulations of inverse problems. A recurring theme in regularization approaches is the selection of regularization parameters, and their effect on the solution and on the optimal value of the optimization problem. The sensitivity of the value function to the regularization parameter can be linked directly to the Lagrange multipliers. Th… ▽ More Regularization plays a key role in a variety of optimization formulations of inverse problems. A recurring theme in regularization approaches is the selection of regularization parameters, and their effect on the solution and on the optimal value of the optimization problem. The sensitivity of the value function to the regularization parameter can be linked directly to the Lagrange multipliers. This paper characterizes the variational properties of the value functions for a broad class of convex formulations, which are not all covered by standard Lagrange multiplier theory. An inverse function theorem is given that links the value functions of different regularization formulations (not necessarily convex). These results have implications for the selection of regularization parameters, and the development of specialized algorithms. Numerical examples illustrate the theoretical results. △ Less

Submitted 23 May, 2013; v1 submitted 15 November, 2012; originally announced November 2012.

Comments: 30 pages

Journal ref: SIAM Journal on Optimization, 23(3):1689-1717, 2013

Elementary subalgebras of Lie algebras

Authors: Jon F. Carlson, Eric M. Friedlander, Julia Pevtsova

Abstract: We initiate the investigation of the projective variety $E(r,g)$ of elementary subalgebras of dimension $r$ of a ($p$-restricted) Lie algebra $g$ for some $r > 0$ and demonstrate that this variety encodes considerable information about the representations of $g$. For various choices of $g$ and $r$, we identify the geometric structure of $E(r,g)$. We show that special classes of (restricted) repres… ▽ More We initiate the investigation of the projective variety $E(r,g)$ of elementary subalgebras of dimension $r$ of a ($p$-restricted) Lie algebra $g$ for some $r > 0$ and demonstrate that this variety encodes considerable information about the representations of $g$. For various choices of $g$ and $r$, we identify the geometric structure of $E(r,g)$. We show that special classes of (restricted) representations of $g$ lead to algebraic vector bundles on $E(r,g)$. For $g = Lie(G)$ the Lie algebra of an algebraic group $G$, rational representations of $G$ enable us to realize familiar algebraic vector bundles on $G$-orbits of $E(r, g)$. △ Less

Submitted 23 September, 2014; v1 submitted 25 July, 2012; originally announced July 2012.

Comments: The paper has been split into two at the request of the referee, the first part has the same title as the older combined version

Robust inversion via semistochastic dimensionality reduction

Authors: Aleksandr Aravkin, Michael P. Friedlander, Tristan van Leeuwen

Abstract: We consider a class of inverse problems where it is possible to aggregate the results of multiple experiments. This class includes problems where the forward model is the solution operator to linear ODEs or PDEs. The tremendous size of such problems motivates dimensionality reduction techniques based on randomly mixing experiments. These techniques break down, however, when robust data-fitting for… ▽ More We consider a class of inverse problems where it is possible to aggregate the results of multiple experiments. This class includes problems where the forward model is the solution operator to linear ODEs or PDEs. The tremendous size of such problems motivates dimensionality reduction techniques based on randomly mixing experiments. These techniques break down, however, when robust data-fitting formulations are used, which are essential in cases of missing data, unusually large errors, and systematic features in the data unexplained by the forward model. We survey robust methods within a statistical framework, and propose a semistochastic optimization approach that allows dimensionality reduction. The efficacy of the methods are demonstrated for a large-scale seismic inverse problem using the robust Student's t-distribution, where a useful synthetic velocity model is recovered in the extreme scenario of 60% data missing at random. The semistochastic approach achieves this recovery using 20% of the effort required by a direct robust approach. △ Less

Submitted 2 July, 2012; v1 submitted 5 October, 2011; originally announced October 2011.

Comments: Mathematical Programming, 2012

Journal ref: Mathematical Programming 134 (1), 101-125, 2012

Representations of elementary abelian p-groups and bundles on Grassmannians

Authors: Jon F. Carlson, Eric M. Friedlander, Julia Pevtsova

Abstract: We initiate the study of representations of elementary abelian $p$-groups via restrictions to truncated polynomial subalgebras of the group algebra generated by $r$ nilpotent elements, $k[t_1,..., t_r]/(t^p_1,..., t_r^p)$. We introduce new geometric invariants based on the behavior of modules upon restrictions to such subalgebras. We also introduce modules of constant radical and socle type genera… ▽ More We initiate the study of representations of elementary abelian $p$-groups via restrictions to truncated polynomial subalgebras of the group algebra generated by $r$ nilpotent elements, $k[t_1,..., t_r]/(t^p_1,..., t_r^p)$. We introduce new geometric invariants based on the behavior of modules upon restrictions to such subalgebras. We also introduce modules of constant radical and socle type generalizing modules of constant Jordan type and provide several general constructions of modules with these properties. We show that modules of constant radical and socle type lead to families of algebraic vector bundles on Grassmannians and illustrate our theory with numerous examples. △ Less

Submitted 22 June, 2011; originally announced June 2011.

Generalized support varieties for finite group schemes

Authors: Eric M. Friedlander, Julia Pevtsova

Abstract: We construct two families of refinements of the (projectivized) support variety of a finite dimensional module $M$ for a finite group scheme $G$. For an arbitrary finite group scheme, we associate a family of {\it non maximal rank varieties} $Γ^j(G)_M$, $1\leq j \leq p-1$, to a $kG$-module $M$. For $G$ infinitesimal, we construct a finer family of locally closed subvarieties $V^{\ul a}(G)_M$ of th… ▽ More We construct two families of refinements of the (projectivized) support variety of a finite dimensional module $M$ for a finite group scheme $G$. For an arbitrary finite group scheme, we associate a family of {\it non maximal rank varieties} $Γ^j(G)_M$, $1\leq j \leq p-1$, to a $kG$-module $M$. For $G$ infinitesimal, we construct a finer family of locally closed subvarieties $V^{\ul a}(G)_M$ of the variety of one parameter subgroups of $G$ for any partition $\ul a$ of $\dim M$. For an arbitrary finite group scheme $G$, a $kG$-module $M$ of constant rank, and a cohomology class $ζ$ in $\HHH^1(G,M)$ we introduce the {\it zero locus} $Z(ζ) \subset Π(G)$. We show that $Z(ζ)$ is a closed subvariety, and relate it to the non-maximal rank varieties. We also extend the construction of $Z(ζ)$ to an arbitrary extension class $ζ\in \Ext^n_G(M,N)$ whenever $M$ and $N$ are $kG$-modules of constant Jordan type. △ Less

Submitted 21 June, 2011; originally announced June 2011.

MSC Class: 16G10; 20C20; 20G10

Journal ref: Documenta Mathematica, Extra Volume Suslin (2011) 191-217

Hybrid Deterministic-Stochastic Methods for Data Fitting

Authors: Michael P. Friedlander, Mark Schmidt

Abstract: Many structured data-fitting applications require the solution of an optimization problem involving a sum over a potentially large number of measurements. Incremental gradient algorithms offer inexpensive iterations by sampling a subset of the terms in the sum. These methods can make great progress initially, but often slow as they approach a solution. In contrast, full-gradient methods achieve st… ▽ More Many structured data-fitting applications require the solution of an optimization problem involving a sum over a potentially large number of measurements. Incremental gradient algorithms offer inexpensive iterations by sampling a subset of the terms in the sum. These methods can make great progress initially, but often slow as they approach a solution. In contrast, full-gradient methods achieve steady convergence at the expense of evaluating the full objective and gradient on each iteration. We explore hybrid methods that exhibit the benefits of both approaches. Rate-of-convergence analysis shows that by controlling the sample size in an incremental gradient algorithm, it is possible to maintain the steady convergence rates of full-gradient methods. We detail a practical quasi-Newton implementation based on this approach. Numerical experiments illustrate its potential benefits. △ Less

Submitted 9 February, 2013; v1 submitted 13 April, 2011; originally announced April 2011.

Comments: 26 pages. Revised proofs of Theorems 2.6 and 3.1, results unchanged

Journal ref: SIAM Journal on Scientific Computing, 34(3):A1380-A1405, 2012

Recovering Compressively Sampled Signals Using Partial Support Information

Authors: Michael P. Friedlander, Hassan Mansour, Rayan Saab, Ozgur Yilmaz

Abstract: In this paper we study recovery conditions of weighted $\ell_1$ minimization for signal reconstruction from compressed sensing measurements when partial support information is available. We show that if at least 50% of the (partial) support information is accurate, then weighted $\ell_1$ minimization is stable and robust under weaker conditions than the analogous conditions for standard $\ell_1$ m… ▽ More In this paper we study recovery conditions of weighted $\ell_1$ minimization for signal reconstruction from compressed sensing measurements when partial support information is available. We show that if at least 50% of the (partial) support information is accurate, then weighted $\ell_1$ minimization is stable and robust under weaker conditions than the analogous conditions for standard $\ell_1$ minimization. Moreover, weighted $\ell_1$ minimization provides better bounds on the reconstruction error in terms of the measurement noise and the compressibility of the signal to be recovered. We illustrate our results with extensive numerical experiments on synthetic data and real audio and video signals. △ Less

Submitted 22 July, 2011; v1 submitted 22 October, 2010; originally announced October 2010.

Comments: 22 pages, 10 figures