Skip to main content

Showing 1–20 of 20 results for author: Orbanz, P

.
  1. arXiv:2406.12437  [pdf, other

    math.ST math.PR

    Slow rates of approximation of U-statistics and V-statistics by quadratic forms of Gaussians

    Authors: Kevin Han Huang, Peter Orbanz

    Abstract: We construct examples of degree-two U- and V-statistics of $n$ i.i.d.~heavy-tailed random vectors in $\mathbb{R}^{d(n)}$, whose $ν$-th moments exist for ${ν> 2}$, and provide tight bounds on the error of approximating both statistics by a quadratic form of Gaussians. In the case ${ν=3}$, the error of approximation is $Θ(n^{-1/12})$. The proof adapts a result of Huang, Austern and Orbanz [12] to U-… ▽ More

    Submitted 18 June, 2024; originally announced June 2024.

  2. arXiv:2403.10711  [pdf, ps, other

    math.PR math.ST

    Gaussian universality for approximately polynomial functions of high-dimensional data

    Authors: Kevin Han Huang, Morgane Austern, Peter Orbanz

    Abstract: We establish an invariance principle for polynomial functions of $n$ independent high-dimensional random vectors, and also show that the obtained rates are nearly optimal. Both the dimension of the vectors and the degree of the polynomial are permitted to grow with $n$. Specifically, we obtain a finite sample upper bound for the error of approximation by a polynomial of Gaussians, measured in Kolm… ▽ More

    Submitted 15 March, 2024; originally announced March 2024.

  3. arXiv:2402.07613  [pdf, other

    math.ST cs.LG stat.ML

    Global optimality under amenable symmetry constraints

    Authors: Peter Orbanz

    Abstract: We ask whether there exists a function or measure that (1) minimizes a given convex functional or risk and (2) satisfies a symmetry property specified by an amenable group of transformations. Examples of such symmetry properties are invariance, equivariance, or quasi-invariance. Our results draw on old ideas of Stein and Le Cam and on approximate group averages that appear in ergodic theorems for… ▽ More

    Submitted 12 February, 2024; originally announced February 2024.

  4. arXiv:2402.00188  [pdf, other

    cs.DM math.CO math.ST

    The Graph Pencil Method: Map** Subgraph Densities to Stochastic Block Models

    Authors: Lee M Gunderson, Gecia Bravo-Hermsdorff, Peter Orbanz

    Abstract: In this work, we describe a method that determines an exact map from a finite set of subgraph densities to the parameters of a stochastic block model (SBM) matching these densities. Given a number $K$ of blocks, the subgraph densities of a finite number of stars and bistars uniquely determines a single element of the class of all degree-separated stochastic block models with $K$ blocks. Our method… ▽ More

    Submitted 31 January, 2024; originally announced February 2024.

    Comments: NeurIPS 2023

    Journal ref: NeurIPS 2023

  5. arXiv:2401.10060  [pdf, ps, other

    math.PR math.ST

    Poisson approximation for stochastic processes summed over amenable groups

    Authors: Haoyu Ye, Peter Orbanz, Morgane Austern

    Abstract: We generalize the Poisson limit theorem to binary functions of random objects whose law is invariant under the action of an amenable group. Examples include stationary random fields, exchangeable sequences, and exchangeable graphs. A celebrated result of E. Lindenstrauss shows that normalized sums over certain increasing subsets of such groups approximate expectations. Our results clarify that the… ▽ More

    Submitted 18 January, 2024; originally announced January 2024.

  6. arXiv:2306.05261  [pdf, other

    stat.ML cond-mat.mtrl-sci cs.LG

    Representing and Learning Functions Invariant Under Crystallographic Groups

    Authors: Ryan P. Adams, Peter Orbanz

    Abstract: Crystallographic groups describe the symmetries of crystals and other repetitive structures encountered in nature and the sciences. These groups include the wallpaper and space groups. We derive linear and nonlinear representations of functions that are (1) smooth and (2) invariant under such a group. The linear representation generalizes the Fourier basis to crystallographically invariant basis f… ▽ More

    Submitted 8 June, 2023; originally announced June 2023.

  7. arXiv:2202.09134  [pdf, other

    cs.LG math.ST stat.ML

    Data Augmentation in the Underparameterized and Overparameterized Regimes

    Authors: Kevin Han Huang, Peter Orbanz, Morgane Austern

    Abstract: We provide results that exactly quantify how data augmentation affects the variance and limiting distribution of estimates, and analyze several specific models in detail. The results confirm some observations made in machine learning practice, but also lead to unexpected findings: Data augmentation may increase rather than decrease the uncertainty of estimates, such as the empirical prediction ris… ▽ More

    Submitted 28 September, 2023; v1 submitted 18 February, 2022; originally announced February 2022.

    Comments: Changed title and added an analysis on the effect of augmentations on the double-descent risk curve of a high-dimensional ridgeless estimator

  8. arXiv:1806.10701  [pdf, other

    stat.ML cs.LG cs.SI

    Empirical Risk Minimization and Stochastic Gradient Descent for Relational Data

    Authors: Victor Veitch, Morgane Austern, Wenda Zhou, David M. Blei, Peter Orbanz

    Abstract: Empirical risk minimization is the main tool for prediction problems, but its extension to relational data remains unsolved. We solve this problem using recent ideas from graph sampling theory to (i) define an empirical risk for relational data and (ii) obtain stochastic gradients for this empirical risk that are automatically unbiased. This is achieved by considering the method by which data is s… ▽ More

    Submitted 22 February, 2019; v1 submitted 27 June, 2018; originally announced June 2018.

    Comments: Accepted as AISTATS 2019 Oral

  9. arXiv:1806.10661  [pdf, other

    math.ST math.PR

    Limit theorems for invariant distributions

    Authors: Morgane Austern, Peter Orbanz

    Abstract: A distributional symmetry is invariance of a distribution under a group of transformations. Exchangeability and stationarity are examples. We explain that a result of ergodic theory provides a law of large numbers: If the group satisfies suitable conditions, expectations can be estimated by averaging over subsets of transformations, and these estimators are strongly consistent. We show that, if a… ▽ More

    Submitted 28 November, 2021; v1 submitted 27 June, 2018; originally announced June 2018.

  10. arXiv:1804.05862  [pdf, other

    stat.ML cs.LG

    Non-Vacuous Generalization Bounds at the ImageNet Scale: A PAC-Bayesian Compression Approach

    Authors: Wenda Zhou, Victor Veitch, Morgane Austern, Ryan P. Adams, Peter Orbanz

    Abstract: Modern neural networks are highly overparameterized, with capacity to substantially overfit to training data. Nevertheless, these networks often generalize well in practice. It has also been observed that trained networks can often be "compressed" to much smaller representations. The purpose of this paper is to connect these two empirical observations. Our main technical result is a generalization… ▽ More

    Submitted 24 February, 2019; v1 submitted 16 April, 2018; originally announced April 2018.

    Comments: 16 pages, 1 figure. Accepted at ICLR 2019

  11. arXiv:1710.04217  [pdf, other

    math.ST math.PR

    Subsampling large graphs and invariance in networks

    Authors: Peter Orbanz

    Abstract: Specify a randomized algorithm that, given a very large graph or network, extracts a random subgraph. What can we learn about the input graph from a single subsample? We derive laws of large numbers for the sampler output, by relating randomized subsampling to distributional invariance: Assuming an invariance holds is tantamount to assuming the sample has been generated by a specific algorithm. Th… ▽ More

    Submitted 11 October, 2017; originally announced October 2017.

  12. arXiv:1710.02159  [pdf, other

    math.PR cs.SI math.ST physics.soc-ph

    Preferential Attachment and Vertex Arrival Times

    Authors: Benjamin Bloem-Reddy, Peter Orbanz

    Abstract: We study preferential attachment mechanisms in random graphs that are parameterized by (i) a constant bias affecting the degree-biased distribution on the vertex set and (ii) the distribution of times at which new vertices are created by the model. The class of random graphs so defined admits a representation theorem reminiscent of residual allocation, or "stick-breaking" schemes. We characterize… ▽ More

    Submitted 5 October, 2017; originally announced October 2017.

    Comments: 34 pages, 1 figure

  13. arXiv:1703.03412  [pdf, other

    math.ST

    Uniform estimation in stochastic block models is slow

    Authors: Ismaël Castillo, Peter Orbanz

    Abstract: We explicitly quantify the empirically observed phenomenon that estimation under a stochastic block model (SBM) is hard if the model contains classes that are similar. More precisely, we consider estimation of certain functionals of random graphs generated by a SBM. The SBM may or may not be sparse, and the number of classes may be fixed or grow with the number of vertices. Minimax lower and upper… ▽ More

    Submitted 26 April, 2022; v1 submitted 9 March, 2017; originally announced March 2017.

  14. arXiv:1703.02054  [pdf, other

    math.PR

    Independence by Random Scaling

    Authors: Lancelot F. James, Peter Orbanz

    Abstract: We give conditions under which a scalar random variable T can be coupled to a random scaling factor $ξ$ such that T and $ξ$T are rendered stochastically independent. A similar result is obtained for random measures. One consequence is a generalization of a result by Pitman and Yor on the Poisson-Dirichlet distribution to its negative parameter range. Another application are diffusion excursions st… ▽ More

    Submitted 6 March, 2017; originally announced March 2017.

  15. arXiv:1612.06404  [pdf, other

    stat.ME stat.ML

    Random Walk Models of Network Formation and Sequential Monte Carlo Methods for Graphs

    Authors: Benjamin Bloem-Reddy, Peter Orbanz

    Abstract: We introduce a class of generative network models that insert edges by connecting the starting and terminal vertices of a random walk on the network graph. Within the taxonomy of statistical network models, this class is distinguished by permitting the location of a new edge to explicitly depend on the structure of the graph, but being nonetheless statistically and computationally tractable. In th… ▽ More

    Submitted 7 July, 2018; v1 submitted 19 December, 2016; originally announced December 2016.

  16. arXiv:1510.07309  [pdf, other

    math.PR

    Scaled subordinators and generalizations of the Indian buffet process

    Authors: Lancelot F. James, Peter Orbanz, Yee Whye Teh

    Abstract: We study random families of subsets of $\mathbb{N}$ that are similar to exchangeable random partitions, but do not require constituent sets to be disjoint: Each element of ${\mathbb{N}}$ may be contained in multiple subsets. One class of such objects, known as Indian buffet processes, has become a popular tool in machine learning. Based on an equivalence between Indian buffet and scale-invariant P… ▽ More

    Submitted 25 October, 2015; originally announced October 2015.

  17. arXiv:1312.7857  [pdf, other

    math.ST stat.ML

    Bayesian Models of Graphs, Arrays and Other Exchangeable Random Structures

    Authors: Peter Orbanz, Daniel M. Roy

    Abstract: The natural habitat of most Bayesian methods is data represented by exchangeable sequences of observations, for which de Finetti's theorem provides the theoretical foundation. Dirichlet process clustering, Gaussian process regression, and many other parametric and nonparametric Bayesian models fall within the remit of this framework; many problems arising in modern data analysis do not. This artic… ▽ More

    Submitted 13 February, 2015; v1 submitted 30 December, 2013; originally announced December 2013.

    Journal ref: IEEE Transactions Pattern Analysis and Machine Intelligence 2015, Vol. 37, No. 2, pp. 437-461

  18. arXiv:1312.7351  [pdf, other

    math.PR math.CO

    Borel Liftings of Graph Limits

    Authors: Peter Orbanz, Balazs Szegedy

    Abstract: The cut pseudo-metric on the space of graph limits induces an equivalence relation. The quotient space obtained by collapsing each equivalence class to a point is a metric space with appealing analytic properties. We show that the equivalence relation admits a Borel lifting: There exists a Borel-measurable map** which maps each equivalence class to one of its elements.

    Submitted 27 December, 2013; originally announced December 2013.

  19. arXiv:1101.4657  [pdf, ps, other

    math.ST stat.ML

    Projective Limit Random Probabilities on Polish Spaces

    Authors: Peter Orbanz

    Abstract: A pivotal problem in Bayesian nonparametrics is the construction of prior distributions on the space M(V) of probability measures on a given domain V. In principle, such distributions on the infinite-dimensional space M(V) can be constructed from their finite-dimensional marginals---the most prominent example being the construction of the Dirichlet process from finite-dimensional Dirichlet distrib… ▽ More

    Submitted 19 October, 2011; v1 submitted 24 January, 2011; originally announced January 2011.

    Comments: 20 pages, 3 figures. Published in the Electronic Journal of Statistics by the Institute of Mathematical Statistics

    Journal ref: Electronic Journal of Statistics 2011, Vol. 5, 1354-1373

  20. arXiv:1012.0363  [pdf, other

    math.ST stat.ML

    Conjugate Projective Limits

    Authors: Peter Orbanz

    Abstract: We characterize conjugate nonparametric Bayesian models as projective limits of conjugate, finite-dimensional Bayesian models. In particular, we identify a large class of nonparametric models representable as infinite-dimensional analogues of exponential family distributions and their canonical conjugate priors. This class contains most models studied in the literature, including Dirichlet process… ▽ More

    Submitted 7 January, 2011; v1 submitted 1 December, 2010; originally announced December 2010.

    Comments: 49 pages; improved version: revised proof of theorem 3 (results unchanged), discussion added, exposition revised