-
Applied Measure Theory for Probabilistic Modeling
Authors:
Chad Scherrer,
Moritz Schauer
Abstract:
Probabilistic programming and statistical computing are vibrant areas in the development of the Julia programming language, but the underlying infrastructure dramatically predates recent developments. The goal of MeasureTheory.jl is to provide Julia with the right vocabulary and tools for these tasks.
In the package we introduce a well-chosen set of notions from the foundations of probability to…
▽ More
Probabilistic programming and statistical computing are vibrant areas in the development of the Julia programming language, but the underlying infrastructure dramatically predates recent developments. The goal of MeasureTheory.jl is to provide Julia with the right vocabulary and tools for these tasks.
In the package we introduce a well-chosen set of notions from the foundations of probability together with powerful combinators and transforms, giving a gentle introduction to the concepts in this article.
The task is foremost achieved by recognizing measure as the central object. This enables us to develop a proper concept of densities as objects relating measures with each others. As densities provide local perspective on measures, they are the key to efficient implementations.
The need to preserve this computationally so important locality leads to the new notion of locally-dominated measure solving the so-called base measure problem and making work with densities and distributions in Julia easier and more flexible.
△ Less
Submitted 28 June, 2022; v1 submitted 1 October, 2021;
originally announced October 2021.
-
What's the Over/Under? Probabilistic Bounds on Information Leakage
Authors:
Ian Sweet,
Jose Manuel Calderon Trilla,
Chad Scherrer,
Michael Hicks,
Stephen Magill
Abstract:
Quantitative information flow (QIF) is concerned with measuring how much of a secret is leaked to an adversary who observes the result of a computation that uses it. Prior work has shown that QIF techniques based on abstract interpretation with probabilistic polyhedra can be used to analyze the worst-case leakage of a query, on-line, to determine whether that query can be safely answered. While th…
▽ More
Quantitative information flow (QIF) is concerned with measuring how much of a secret is leaked to an adversary who observes the result of a computation that uses it. Prior work has shown that QIF techniques based on abstract interpretation with probabilistic polyhedra can be used to analyze the worst-case leakage of a query, on-line, to determine whether that query can be safely answered. While this approach can provide precise estimates, it does not scale well. This paper shows how to solve the scalability problem by augmenting the baseline technique with sampling and symbolic execution. We prove that our approach never underestimates a query's leakage (it is sound), and detailed experimental results show that we can match the precision of the baseline technique but with orders of magnitude better performance.
△ Less
Submitted 22 February, 2018;
originally announced February 2018.
-
Feature Clustering for Accelerating Parallel Coordinate Descent
Authors:
Chad Scherrer,
Ambuj Tewari,
Mahantesh Halappanavar,
David Haglin
Abstract:
Large-scale L1-regularized loss minimization problems arise in high-dimensional applications such as compressed sensing and high-dimensional supervised learning, including classification and regression problems. High-performance algorithms and implementations are critical to efficiently solving these problems. Building upon previous work on coordinate descent algorithms for L1-regularized problems…
▽ More
Large-scale L1-regularized loss minimization problems arise in high-dimensional applications such as compressed sensing and high-dimensional supervised learning, including classification and regression problems. High-performance algorithms and implementations are critical to efficiently solving these problems. Building upon previous work on coordinate descent algorithms for L1-regularized problems, we introduce a novel family of algorithms called block-greedy coordinate descent that includes, as special cases, several existing algorithms such as SCD, Greedy CD, Shotgun, and Thread-Greedy. We give a unified convergence analysis for the family of block-greedy algorithms. The analysis suggests that block-greedy coordinate descent can better exploit parallelism if features are clustered so that the maximum inner product between features in different blocks is small. Our theoretical convergence analysis is supported with experimental re- sults using data from diverse real-world applications. We hope that algorithmic approaches and convergence analysis we provide will not only advance the field, but will also encourage researchers to systematically explore the design space of algorithms for solving large-scale L1-regularization problems.
△ Less
Submitted 17 December, 2012;
originally announced December 2012.
-
Scaling Up Coordinate Descent Algorithms for Large $\ell_1$ Regularization Problems
Authors:
Chad Scherrer,
Mahantesh Halappanavar,
Ambuj Tewari,
David Haglin
Abstract:
We present a generic framework for parallel coordinate descent (CD) algorithms that includes, as special cases, the original sequential algorithms Cyclic CD and Stochastic CD, as well as the recent parallel Shotgun algorithm. We introduce two novel parallel algorithms that are also special cases---Thread-Greedy CD and Coloring-Based CD---and give performance measurements for an OpenMP implementati…
▽ More
We present a generic framework for parallel coordinate descent (CD) algorithms that includes, as special cases, the original sequential algorithms Cyclic CD and Stochastic CD, as well as the recent parallel Shotgun algorithm. We introduce two novel parallel algorithms that are also special cases---Thread-Greedy CD and Coloring-Based CD---and give performance measurements for an OpenMP implementation of these.
△ Less
Submitted 27 June, 2012;
originally announced June 2012.
-
Flavor Physics in an SO(10) Grand Unified Model
Authors:
Jennifer Girrbach,
Sebastian Jager,
Markus Knopf,
Waldemar Martens,
Ulrich Nierste,
Christian Scherrer,
Soren Wiesenfeldt
Abstract:
In supersymmetric grand-unified models, the lepton mixing matrix can possibly affect flavor-changing transitions in the quark sector. We present a detailed analysis of a model proposed by Chang, Masiero and Murayama, in which the near-maximal atmospheric neutrino mixing angle governs large new b -> s transitions. Relating the supersymmetric low-energy parameters to seven new parameters of this SO(…
▽ More
In supersymmetric grand-unified models, the lepton mixing matrix can possibly affect flavor-changing transitions in the quark sector. We present a detailed analysis of a model proposed by Chang, Masiero and Murayama, in which the near-maximal atmospheric neutrino mixing angle governs large new b -> s transitions. Relating the supersymmetric low-energy parameters to seven new parameters of this SO(10) GUT model, we perform a correlated study of several flavor-changing neutral current (FCNC) processes. We find the current bound on B(tau -> mu gamma) more constraining than B(B -> X_s gamma). The LEP limit on the lightest Higgs boson mass implies an important lower bound on tan beta, which in turn limits the size of the new FCNC transitions. Remarkably, the combined analysis does not rule out large effects in B_s-B_s-bar mixing and we can easily accomodate the large CP phase in the B_s-B_s-bar system which has recently been inferred from a global analysis of CDF and DO data. The model predicts a particle spectrum which is different from the popular Constrained Minimal Supersymmetric Standard Model (CMSSM). B(tau -> mu gamma) enforces heavy masses, typically above 1 TeV, for the sfermions of the degenerate first two generations. However, the ratio of the third-generation and first-generation sfermion masses is smaller than in the CMSSM and a (dominantly right-handed) stop with mass below 500 GeV is possible.
△ Less
Submitted 23 June, 2011; v1 submitted 31 January, 2011;
originally announced January 2011.
