-
Gemini: A Family of Highly Capable Multimodal Models
Authors:
Gemini Team,
Rohan Anil,
Sebastian Borgeaud,
Jean-Baptiste Alayrac,
Jiahui Yu,
Radu Soricut,
Johan Schalkwyk,
Andrew M. Dai,
Anja Hauth,
Katie Millican,
David Silver,
Melvin Johnson,
Ioannis Antonoglou,
Julian Schrittwieser,
Amelia Glaese,
Jilin Chen,
Emily Pitler,
Timothy Lillicrap,
Angeliki Lazaridou,
Orhan Firat,
James Molloy,
Michael Isard,
Paul R. Barham,
Tom Hennigan,
Benjamin Lee
, et al. (1325 additional authors not shown)
Abstract:
This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding. The Gemini family consists of Ultra, Pro, and Nano sizes, suitable for applications ranging from complex reasoning tasks to on-device memory-constrained use-cases. Evaluation on a broad range of benchmarks shows that our most-capable Gemini Ultr…
▽ More
This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding. The Gemini family consists of Ultra, Pro, and Nano sizes, suitable for applications ranging from complex reasoning tasks to on-device memory-constrained use-cases. Evaluation on a broad range of benchmarks shows that our most-capable Gemini Ultra model advances the state of the art in 30 of 32 of these benchmarks - notably being the first model to achieve human-expert performance on the well-studied exam benchmark MMLU, and improving the state of the art in every one of the 20 multimodal benchmarks we examined. We believe that the new capabilities of the Gemini family in cross-modal reasoning and language understanding will enable a wide variety of use cases. We discuss our approach toward post-training and deploying Gemini models responsibly to users through services including Gemini, Gemini Advanced, Google AI Studio, and Cloud Vertex AI.
△ Less
Submitted 17 June, 2024; v1 submitted 18 December, 2023;
originally announced December 2023.
-
The James Webb Space Telescope Mission
Authors:
Jonathan P. Gardner,
John C. Mather,
Randy Abbott,
James S. Abell,
Mark Abernathy,
Faith E. Abney,
John G. Abraham,
Roberto Abraham,
Yasin M. Abul-Huda,
Scott Acton,
Cynthia K. Adams,
Evan Adams,
David S. Adler,
Maarten Adriaensen,
Jonathan Albert Aguilar,
Mansoor Ahmed,
Nasif S. Ahmed,
Tanjira Ahmed,
Rüdeger Albat,
Loïc Albert,
Stacey Alberts,
David Aldridge,
Mary Marsha Allen,
Shaune S. Allen,
Martin Altenburg
, et al. (983 additional authors not shown)
Abstract:
Twenty-six years ago a small committee report, building on earlier studies, expounded a compelling and poetic vision for the future of astronomy, calling for an infrared-optimized space telescope with an aperture of at least $4m$. With the support of their governments in the US, Europe, and Canada, 20,000 people realized that vision as the $6.5m$ James Webb Space Telescope. A generation of astrono…
▽ More
Twenty-six years ago a small committee report, building on earlier studies, expounded a compelling and poetic vision for the future of astronomy, calling for an infrared-optimized space telescope with an aperture of at least $4m$. With the support of their governments in the US, Europe, and Canada, 20,000 people realized that vision as the $6.5m$ James Webb Space Telescope. A generation of astronomers will celebrate their accomplishments for the life of the mission, potentially as long as 20 years, and beyond. This report and the scientific discoveries that follow are extended thank-you notes to the 20,000 team members. The telescope is working perfectly, with much better image quality than expected. In this and accompanying papers, we give a brief history, describe the observatory, outline its objectives and current observing program, and discuss the inventions and people who made it possible. We cite detailed reports on the design and the measured performance on orbit.
△ Less
Submitted 10 April, 2023;
originally announced April 2023.
-
The Bernoulli clock: probabilistic and combinatorial interpretations of the Bernoulli polynomials by circular convolution
Authors:
Yassine El Maazouz,
Jim Pitman
Abstract:
The factorially normalized Bernoulli polynomials $b_n(x) = B_n(x)/n!$ are known to be characterized by $b_0(x) = 1$ and $b_n(x)$ for $n >0$ is the antiderivative of $b_{n-1}(x)$ subject to $\int_0^1 b_n(x) dx = 0$. We offer a related characterization: $b_1(x) = x - 1/2$ and $(-1)^{n-1} b_n(x)$ for $n >0$ is the $n$-fold circular convolution of $b_1(x)$ with itself. Equivalently, $1 - 2^n b_n(x)$ i…
▽ More
The factorially normalized Bernoulli polynomials $b_n(x) = B_n(x)/n!$ are known to be characterized by $b_0(x) = 1$ and $b_n(x)$ for $n >0$ is the antiderivative of $b_{n-1}(x)$ subject to $\int_0^1 b_n(x) dx = 0$. We offer a related characterization: $b_1(x) = x - 1/2$ and $(-1)^{n-1} b_n(x)$ for $n >0$ is the $n$-fold circular convolution of $b_1(x)$ with itself. Equivalently, $1 - 2^n b_n(x)$ is the probability density at $x \in (0,1)$ of the fractional part of a sum of $n$ independent random variables, each with the beta$(1,2)$ probability density $2(1-x)$ at $x \in (0,1)$. This result has a novel combinatorial analog, the {\em Bernoulli clock}: mark the hours of a $2 n$ hour clock by a uniform random permutation of the multiset $\{1,1, 2,2, \ldots, n,n\}$, meaning pick two different hours uniformly at random from the $2 n$ hours and mark them $1$, then pick two different hours uniformly at random from the remaining $2 n - 2$ hours and mark them $2$, and so on. Starting from hour $0 = 2n$, move clockwise to the first hour marked $1$, continue clockwise to the first hour marked $2$, and so on, continuing clockwise around the Bernoulli clock until the first of the two hours marked $n$ is encountered, at a random hour $I_n$ between $1$ and $2n$. We show that for each positive integer $n$, the event $( I_n = 1)$ has probability $(1 - 2^n b_n(0))/(2n)$, where $n! b_n(0) = B_n(0)$ is the $n$th Bernoulli number. For $ 1 \le k \le 2 n$, the difference $δ_n(k):= 1/(2n) - ¶( I_n = k)$ is a polynomial function of $k$ with the surprising symmetry $δ_n( 2 n + 1 - k) = (-1)^n δ_n(k)$, which is a combinatorial analog of the well known symmetry of Bernoulli polynomials $b_n(1-x) = (-1)^n b_n(x)$.
△ Less
Submitted 11 January, 2024; v1 submitted 5 October, 2022;
originally announced October 2022.
-
The Science Performance of JWST as Characterized in Commissioning
Authors:
Jane Rigby,
Marshall Perrin,
Michael McElwain,
Randy Kimble,
Scott Friedman,
Matt Lallo,
René Doyon,
Lee Feinberg,
Pierre Ferruit,
Alistair Glasse,
Marcia Rieke,
George Rieke,
Gillian Wright,
Chris Willott,
Knicole Colon,
Stefanie Milam,
Susan Neff,
Christopher Stark,
Jeff Valenti,
Jim Abell,
Faith Abney,
Yasin Abul-Huda,
D. Scott Acton,
Evan Adams,
David Adler
, et al. (601 additional authors not shown)
Abstract:
This paper characterizes the actual science performance of the James Webb Space Telescope (JWST), as determined from the six month commissioning period. We summarize the performance of the spacecraft, telescope, science instruments, and ground system, with an emphasis on differences from pre-launch expectations. Commissioning has made clear that JWST is fully capable of achieving the discoveries f…
▽ More
This paper characterizes the actual science performance of the James Webb Space Telescope (JWST), as determined from the six month commissioning period. We summarize the performance of the spacecraft, telescope, science instruments, and ground system, with an emphasis on differences from pre-launch expectations. Commissioning has made clear that JWST is fully capable of achieving the discoveries for which it was built. Moreover, almost across the board, the science performance of JWST is better than expected; in most cases, JWST will go deeper faster than expected. The telescope and instrument suite have demonstrated the sensitivity, stability, image quality, and spectral range that are necessary to transform our understanding of the cosmos through observations spanning from near-earth asteroids to the most distant galaxies.
△ Less
Submitted 10 April, 2023; v1 submitted 12 July, 2022;
originally announced July 2022.
-
The range of a self-similar additive gamma process is a scale invariant Poisson point process
Authors:
Jim Pitman,
Zhiyi You
Abstract:
It is shown that for a non-decreasing self-similar stochastic process $T$ with independent increments, the range of $T$ forms a Poisson point process with $σ$-finite intensity if and only if the one-dimensional distribution of $T(1)$ is of the gamma type. This follows from a general hold-jump description of such processes $T$, and implies the known result that the spacings between consecutive poin…
▽ More
It is shown that for a non-decreasing self-similar stochastic process $T$ with independent increments, the range of $T$ forms a Poisson point process with $σ$-finite intensity if and only if the one-dimensional distribution of $T(1)$ is of the gamma type. This follows from a general hold-jump description of such processes $T$, and implies the known result that the spacings between consecutive points of a scale invariant Poisson point process, with intensity $θx^{-1} dx$, are the points of another scale invariant Poisson point process with the same intensity.
△ Less
Submitted 12 April, 2022; v1 submitted 17 November, 2021;
originally announced November 2021.
-
Hidden symmetries and limit laws in the extreme order statistics of the Laplace random walk
Authors:
Jim Pitman,
Wenpin Tang
Abstract:
This paper is concerned with the limit laws of the extreme order statistics derived from a symmetric Laplace walk. We provide two different descriptions of the point process of the limiting extreme order statistics: a branching representation and a squared Bessel representation. These complementary descriptions expose various hidden symmetries in branching processes and Brownian motion which lie b…
▽ More
This paper is concerned with the limit laws of the extreme order statistics derived from a symmetric Laplace walk. We provide two different descriptions of the point process of the limiting extreme order statistics: a branching representation and a squared Bessel representation. These complementary descriptions expose various hidden symmetries in branching processes and Brownian motion which lie behind some striking formulas found by Schehr and Majumdar (Phys. Rev. Lett., 108:040601). In particular, the Bessel process of dimension $4 = 2+2$ appears in the descriptions as a path decomposition of Brownian motion at a local minimum and the Ray-Knight description of Brownian local times near the minimum.
△ Less
Submitted 11 July, 2021;
originally announced July 2021.
-
Markovian structure in the concave majorant of Brownian motion
Authors:
Mehdi Ouaki,
Jim Pitman
Abstract:
The purpose of this paper is to highlight some hidden Markovian structure of the concave majorant of the Brownian motion. Several distributional identities are implied by the joint law of a standard one-dimensional Brownian motion $B$ and its almost surely unique concave majorant $K$ on $[0,\infty)$. In particular, the one-dimensional distribution of $2 K_t - B_t$ is that of $R_5(t)$, where $R_5$…
▽ More
The purpose of this paper is to highlight some hidden Markovian structure of the concave majorant of the Brownian motion. Several distributional identities are implied by the joint law of a standard one-dimensional Brownian motion $B$ and its almost surely unique concave majorant $K$ on $[0,\infty)$. In particular, the one-dimensional distribution of $2 K_t - B_t$ is that of $R_5(t)$, where $R_5$ is a $5-$dimensional Bessel process with $R_5(0) = 0$. The process $2K-B$ shares a number of other properties with $R_5$, and we conjecture that it may have the distribution of $R_5$. We also describe the distribution of the convex minorant of a three-dimensional Bessel process with drift.
△ Less
Submitted 20 April, 2022; v1 submitted 23 May, 2021;
originally announced May 2021.
-
Stationary 1-dependent Counting Processes: from Runs to Bivariate Generating Functions
Authors:
Jim Pitman,
Zhiyi You
Abstract:
We give a formula for the bivariate generating function of a stationary 1-dependent counting process in terms of its run probability generating function, with a probabilistic proof. The formula reduces to the well known bivariate generating function of the Eulerian distribution in the case of descents of a sequence of indepependent and identically distributed random variables. The formula is compa…
▽ More
We give a formula for the bivariate generating function of a stationary 1-dependent counting process in terms of its run probability generating function, with a probabilistic proof. The formula reduces to the well known bivariate generating function of the Eulerian distribution in the case of descents of a sequence of indepependent and identically distributed random variables. The formula is compared with alternative expressions from the theory of determinantal point processes and the combinatorics of sequences.
△ Less
Submitted 17 May, 2021;
originally announced May 2021.
-
The science enabled by a dedicated solar system space telescope
Authors:
Cindy L. Young,
Michael H. Wong,
Kunio M. Sayanagi,
Shannon Curry,
Kandis L. Jessup,
Tracy Becker,
Amanda Hendrix,
Nancy Chanover,
Stephanie Milam,
Bryan J. Holler,
Gregory Holsclaw,
Javier Peralta,
John Clarke,
John Spencer,
Michael S. P. Kelley,
Janet Luhmann,
David MacDonnell,
Ronald J. Vervack Jr.,
Kurt Retherford,
Leigh N. Fletcher,
Imke de Pater,
Faith Vilas,
Lori Feaga,
Oswald Siegmund,
Jim Bell
, et al. (13 additional authors not shown)
Abstract:
The National Academy Committee on Astrobiology and Planetary Science (CAPS) made a recommendation to study a large/medium-class dedicated space telescope for planetary science, going beyond the Discovery-class dedicated planetary space telescope endorsed in Visions and Voyages. Such a telescope would observe targets across the entire solar system, engaging a broad spectrum of the science community…
▽ More
The National Academy Committee on Astrobiology and Planetary Science (CAPS) made a recommendation to study a large/medium-class dedicated space telescope for planetary science, going beyond the Discovery-class dedicated planetary space telescope endorsed in Visions and Voyages. Such a telescope would observe targets across the entire solar system, engaging a broad spectrum of the science community. It would ensure that the high-resolution, high-sensitivity observations of the solar system in visible and UV wavelengths revolutionized by the Hubble Space Telescope (HST) could be extended. A dedicated telescope for solar system science would: (a) transform our understanding of time-dependent phenomena in our solar system that cannot be studied currently under programs to observe and visit new targets and (b) enable a comprehensive survey and spectral characterization of minor bodies across the solar system, which requires a large time allocation not supported by existing facilities. The time-domain phenomena to be explored are critically reliant on high spatial resolution UV-visible observations. This paper presents science themes and key questions that require a long-lasting space telescope dedicated to planetary science that can capture high-quality, consistent data at the required cadences that are free from effects of the terrestrial atmosphere and differences across observing facilities. Such a telescope would have excellent synergy with astrophysical facilities by placing planetary discoveries made by astrophysics assets in temporal context, as well as triggering detailed follow-up observations using larger telescopes. The telescope would support future missions to the Ice Giants, Ocean Worlds, and minor bodies across the solar system by placing the results of such targeted missions in the context of longer records of temporal activities and larger sample populations.
△ Less
Submitted 18 August, 2020;
originally announced August 2020.
-
Architectures and Technologies for a Space Telescope for Solar System Science
Authors:
Kunio M. Sayanagi,
Cindy L. Young,
Lynn Bowman,
Joseph Pitman,
Bo Naasz,
Bonnie Meinke,
Tracy Becker,
Jim Bell,
Richard Cartwright,
Nancy Chanover,
John Clarke,
Joshua Colwell,
Shannon Curry,
Imke de Pater,
Gregory Delory,
Lori Feaga,
Leigh N. Fletcher,
Thomas Greathouse,
Amanda Hendrix,
Bryan J. Holler,
Gregory Holsclaw,
Kandis L. Jessup,
Michael S. P. Kelley,
Robert Lillis,
Rosaly M. C. Lopes
, et al. (15 additional authors not shown)
Abstract:
We advocate for a mission concept study for a space telescope dedicated to solar system science in Earth orbit. Such a study was recommended by the Committee on Astrobiology and Planetary Science (CAPS) report "Getting Ready for the Next Planetary Science Decadal Survey." The Mid-Decadal Review also recommended NASA to assess the role and value of space telescopes for planetary science. The need f…
▽ More
We advocate for a mission concept study for a space telescope dedicated to solar system science in Earth orbit. Such a study was recommended by the Committee on Astrobiology and Planetary Science (CAPS) report "Getting Ready for the Next Planetary Science Decadal Survey." The Mid-Decadal Review also recommended NASA to assess the role and value of space telescopes for planetary science. The need for high-resolution, UV-Visible capabilities is especially acute for planetary science with the impending end of the Hubble Space Telescope (HST); however, NASA has not funded a planetary telescope concept study, and the need to assess its value remains. Here, we present potential design options that should be explored to inform the decadal survey.
△ Less
Submitted 15 August, 2020;
originally announced August 2020.
-
Extreme order statistics of random walks
Authors:
Jim Pitman,
Wenpin Tang
Abstract:
This paper is concerned with the limit theory of the extreme order statistics derived from random walks. We establish the joint convergence of the order statistics near the minimum of a random walk in terms of the Feller chains. Detailed descriptions of the limit process are given in the case of simple symmetric walks and Gaussian walks. Some open problems are also presented.
This paper is concerned with the limit theory of the extreme order statistics derived from random walks. We establish the joint convergence of the order statistics near the minimum of a random walk in terms of the Feller chains. Detailed descriptions of the limit process are given in the case of simple symmetric walks and Gaussian walks. Some open problems are also presented.
△ Less
Submitted 27 September, 2021; v1 submitted 28 July, 2020;
originally announced July 2020.
-
Feller coupling of cycles and Poisson spacings
Authors:
Joseph Najnudel,
Jim Pitman
Abstract:
Feller (1945) provided a coupling between the counts of cycles of various sizes in a uniform random permutation of $[n]$ and the spacings between successes in a sequence of $n$ independent Bernoulli trials with success probability $1/n$ at the $n$th trial. Arratia, Barbour and Tavaré (1992) extended Feller's coupling, to associate cycles of random permutations governed by the Ewens $(θ)$ distribut…
▽ More
Feller (1945) provided a coupling between the counts of cycles of various sizes in a uniform random permutation of $[n]$ and the spacings between successes in a sequence of $n$ independent Bernoulli trials with success probability $1/n$ at the $n$th trial. Arratia, Barbour and Tavaré (1992) extended Feller's coupling, to associate cycles of random permutations governed by the Ewens $(θ)$ distribution with spacings derived from independent Bernoulli trials with success probability $θ/(n-1+θ)$ at the $n$th trial, and to conclude that in an infinite sequence of such trials, the numbers of spacings of length $\ell$ are independent Poisson variables with means $θ/\ell$. Ignatov (1978) first discovered this remarkable result in the uniform case $θ= 1$, by constructing Bernoulli $(1/n)$ trials as the indicators of record values in a sequence of i.i.d. uniform $[0,1]$ variables. In the present article, the Poisson property of inhomogeneous Bernoulli spacings is explained by a variation of Ignatov's approach for a general $θ>0$. Moreover, our approach naturally provides random permutations of infinite sets whose cycle counts are exactly given by independent Poisson random variables.
△ Less
Submitted 13 November, 2020; v1 submitted 22 July, 2019;
originally announced July 2019.
-
Bounds on the probability of radically different opinions
Authors:
Krzysztof Burdzy,
Jim Pitman
Abstract:
We establish bounds on the probability that two different agents, who share an initial opinion expressed as a probability distribution on an abstract probability space, given two different sources of information, may come to radically different opinions regarding the conditional probability of the same event.
We establish bounds on the probability that two different agents, who share an initial opinion expressed as a probability distribution on an abstract probability space, given two different sources of information, may come to radically different opinions regarding the conditional probability of the same event.
△ Less
Submitted 18 March, 2019;
originally announced March 2019.
-
Distribution-free properties of isotonic regression
Authors:
Jake A. Soloff,
Adityanand Guntuboyina,
Jim Pitman
Abstract:
It is well known that the isotonic least squares estimator is characterized as the derivative of the greatest convex minorant of a random walk. Provided the walk has exchangeable increments, we prove that the slopes of the greatest convex minorant are distributed as order statistics of the running averages. This result implies an exact non-asymptotic formula for the squared error risk of least squ…
▽ More
It is well known that the isotonic least squares estimator is characterized as the derivative of the greatest convex minorant of a random walk. Provided the walk has exchangeable increments, we prove that the slopes of the greatest convex minorant are distributed as order statistics of the running averages. This result implies an exact non-asymptotic formula for the squared error risk of least squares in isotonic regression when the true sequence is constant that holds for every exchangeable error distribution.
△ Less
Submitted 11 December, 2018;
originally announced December 2018.
-
Gaps and interleaving of point processes in sampling from a residual allocation model
Authors:
Jim Pitman,
Yuri Yakubovich
Abstract:
This article presents a limit theorem for the gaps $\widehat{G}_{i:n}:= X_{n-i+1:n} - X_{n-i:n}$ between order statistics $X_{1:n} \le \cdots \le X_{n:n}$ of a sample of size $n$ from a random discrete distribution on the positive integers $(P_1, P_2, \ldots)$ governed by a residual allocation model (also called a Bernoulli sieve) $P_j:= H_j \prod_{i=1}^{j-1}(1-H_i)$ for a sequence of independent…
▽ More
This article presents a limit theorem for the gaps $\widehat{G}_{i:n}:= X_{n-i+1:n} - X_{n-i:n}$ between order statistics $X_{1:n} \le \cdots \le X_{n:n}$ of a sample of size $n$ from a random discrete distribution on the positive integers $(P_1, P_2, \ldots)$ governed by a residual allocation model (also called a Bernoulli sieve) $P_j:= H_j \prod_{i=1}^{j-1}(1-H_i)$ for a sequence of independent random hazard variables $H_i$ which are identically distributed according to some distribution of $H \in (0,1)$ such that $- \log(1 - H)$ has a non-lattice distribution with finite mean $μ_{\mbox{log}}$. As $n\to \infty$ the finite dimensional distributions of the gaps $\widehat{G}_{i:n}$ converge to those of limiting gaps $G_i$ which are the numbers of points in a stationary renewal process with i.i.d. spacings $- \log(1 - H_j)$ between times $T_{i-1}$ and $T_i$ of births in a Yule process, that is $T_i := \sum_{k=1}^i \varepsilon_{k}/k$ for a sequence of i.i.d. exponential variables $\varepsilon_k$ with mean 1. A consequence is that the mean of $\widehat{G}_{i:n}$ converges to the mean of $G_i$, which is $1/(i μ_{\mbox{log}} )$. This limit theorem simplifies and extends a result of Gnedin, Iksanov and Roesler for the Bernoulli sieve.
△ Less
Submitted 26 April, 2018;
originally announced April 2018.
-
Random weighted averages, partition structures and generalized arcsine laws
Authors:
Jim Pitman
Abstract:
This article offers a simplified approach to the distribution theory of randomly weighted averages or $P$-means $M_P(X):= \sum_{j} X_j P_j$, for a sequence of i.i.d.random variables $X, X_1, X_2, \ldots$, and independent random weights $P:= (P_j)$ with $P_j \ge 0$ and $\sum_{j} P_j = 1$. The collection of distributions of $M_P(X)$, indexed by distributions of $X$, is shown to encode Kingman's part…
▽ More
This article offers a simplified approach to the distribution theory of randomly weighted averages or $P$-means $M_P(X):= \sum_{j} X_j P_j$, for a sequence of i.i.d.random variables $X, X_1, X_2, \ldots$, and independent random weights $P:= (P_j)$ with $P_j \ge 0$ and $\sum_{j} P_j = 1$. The collection of distributions of $M_P(X)$, indexed by distributions of $X$, is shown to encode Kingman's partition structure derived from $P$. For instance, if $X_p$ has Bernoulli$(p)$ distribution on $\{0,1\}$, the $n$th moment of $M_P(X_p)$ is a polynomial function of $p$ which equals the probability generating function of the number $K_n$ of distinct values in a sample of size $n$ from $P$: $E (M_P(X_p))^n = E p^{K_n}$. This elementary identity illustrates a general moment formula for $P$-means in terms of the partition structure associated with random samples from $P$, first developed by Diaconis and Kemperman (1996) and Kerov (1998) in terms of random permutations. As shown by Tsilevich (1997) if the partition probabilities factorize in a way characteristic of the generalized Ewens sampling formula with two parameters $(α,θ)$, found by Pitman (1992), then the moment formula yields the Cauchy-Stieltjes transform of an $(α,θ)$ mean. The analysis of these random means includes the characterization of $(0,θ)$-means, known as Dirichlet means, due to Von Neumann (1941), Watson (1956) and Cifarelli and Regazzini (1990) and generalizations of Lévy's arcsine law for the time spent positive by a Brownian motion, due to Darling (1949) Lamperti (1958) and Barlow, Pitman and Yor (1989).
△ Less
Submitted 21 April, 2018;
originally announced April 2018.
-
Squared Bessel processes of positive and negative dimension embedded in Brownian local times
Authors:
Jim Pitman,
Matthias Winkel
Abstract:
The Ray--Knight theorems show that the local time processes of various path fragments derived from a one-dimensional Brownian motion $B$ are squared Bessel processes of dimensions $0$, $2$, and $4$. It is also known that for various singular perturbations $X= |B| + μ\ell$ of a reflecting Brownian motion $|B|$ by a multiple $μ$ of its local time process $\ell$ at $0$, corresponding local time proce…
▽ More
The Ray--Knight theorems show that the local time processes of various path fragments derived from a one-dimensional Brownian motion $B$ are squared Bessel processes of dimensions $0$, $2$, and $4$. It is also known that for various singular perturbations $X= |B| + μ\ell$ of a reflecting Brownian motion $|B|$ by a multiple $μ$ of its local time process $\ell$ at $0$, corresponding local time processes of $X$ are squared Bessel with other real dimension parameters, both positive and negative. Here, we embed squared Bessel processes of all real dimensions directly in the local time process of $B$. This is done by decomposing the path of $B$ into its excursions above and below a family of continuous random levels determined by the Harrison--Shepp construction of skew Brownian motion as the strong solution of an SDE driven by $B$. This embedding connects to Brownian local times a framework of point processes of squared Bessel excursions of negative dimension and associated stable processes, recently introduced by Forman, Pal, Rizzolo and Winkel to set up interval partition evolutions that arise in their approach to the Aldous diffusion on a space of continuum trees.
△ Less
Submitted 19 April, 2018;
originally announced April 2018.
-
A guide to Brownian motion and related stochastic processes
Authors:
Jim Pitman,
Marc Yor
Abstract:
This is a guide to the mathematical theory of Brownian motion and related stochastic processes, with indications of how this theory is related to other branches of mathematics, most notably the classical theory of partial differential equations associated with the Laplace and heat operators, and various generalizations thereof. As a typical reader, we have in mind a student, familiar with the basi…
▽ More
This is a guide to the mathematical theory of Brownian motion and related stochastic processes, with indications of how this theory is related to other branches of mathematics, most notably the classical theory of partial differential equations associated with the Laplace and heat operators, and various generalizations thereof. As a typical reader, we have in mind a student, familiar with the basic concepts of probability based on measure theory, at the level of the graduate texts of Billingsley and Durrett , and who wants a broader perspective on the theory of Brownian motion and related stochastic processes than can be found in these texts.
△ Less
Submitted 26 February, 2018;
originally announced February 2018.
-
Renewal sequences and record chains related to multiple zeta sums
Authors:
Jean-Jil Duchamps,
Jim Pitman,
Wenpin Tang
Abstract:
For the random interval partition of $[0,1]$ generated by the uniform stick-breaking scheme known as GEM$(1)$, let $u_k$ be the probability that the first $k$ intervals created by the stick-breaking scheme are also the first $k$ intervals to be discovered in a process of uniform random sampling of points from $[0,1]$. Then $u_k$ is a renewal sequence. We prove that $u_k$ is a rational linear combi…
▽ More
For the random interval partition of $[0,1]$ generated by the uniform stick-breaking scheme known as GEM$(1)$, let $u_k$ be the probability that the first $k$ intervals created by the stick-breaking scheme are also the first $k$ intervals to be discovered in a process of uniform random sampling of points from $[0,1]$. Then $u_k$ is a renewal sequence. We prove that $u_k$ is a rational linear combination of the real numbers $1, ζ(2), \ldots, ζ(k)$ where $ζ$ is the Riemann zeta function, and show that $u_k$ has limit $1/3$ as $k \to \infty$. Related results provide probabilistic interpretations of some multiple zeta values in terms of a Markov chain derived from the interval partition. This Markov chain has the structure of a weak record chain. Similar results are given for the GEM$(θ)$ model, with beta$(1,θ)$ instead of uniform stick-breaking factors, and for another more algebraic derivation of renewal sequences from the Riemann zeta function.
△ Less
Submitted 15 June, 2019; v1 submitted 24 July, 2017;
originally announced July 2017.
-
An ergodic theorem for partially exchangeable random partitions
Authors:
Jim Pitman,
Yuri Yakubovich
Abstract:
We consider shifts $Π_{n,m}$ of a partially exchangeable random partition $Π_\infty$ of $\mathbb{N}$ obtained by restricting $Π_\infty$ to $\{n+1,n+2,\dots, n+m\}$ and then subtracting $n$ from each element to get a partition of $[m]:= \{1, \ldots, m \}$. We show that for each fixed $m$ the distribution of $Π_{n,m}$ converges to the distribution of the restriction to $[m]$ of the exchangeable rand…
▽ More
We consider shifts $Π_{n,m}$ of a partially exchangeable random partition $Π_\infty$ of $\mathbb{N}$ obtained by restricting $Π_\infty$ to $\{n+1,n+2,\dots, n+m\}$ and then subtracting $n$ from each element to get a partition of $[m]:= \{1, \ldots, m \}$. We show that for each fixed $m$ the distribution of $Π_{n,m}$ converges to the distribution of the restriction to $[m]$ of the exchangeable random partition of $\mathbb{N}$ with the same ranked frequencies as $Π_\infty$. As a consequence, the partially exchangeable random partition $Π_\infty$ is exchangeable if and only if $Π_\infty$ is stationary in the sense that for each fixed $m$ the distribution of $Π_{n,m}$ on partitions of $[m]$ is the same for all $n$. We also describe the evolution of the frequencies of a partially exchangeable random partition under the shift transformation. For an exchangeable random partition with proper frequencies, the time reversal of this evolution is the heaps process studied by Donnelly and others.
△ Less
Submitted 2 July, 2017;
originally announced July 2017.
-
Ordered and size-biased frequencies in GEM and Gibbs models for species sampling
Authors:
Jim Pitman,
Yuri Yakubovich
Abstract:
We describe the distribution of frequencies ordered by sample values in a random sample of size $n$ from the two parameter GEM$(α,θ)$ random discrete distribution on the positive integers. These frequencies are a $($size$-α)$-biased random permutation of the sample frequencies in either ranked order, or in the order of appearance of values in the sampling process. This generalizes a well known ide…
▽ More
We describe the distribution of frequencies ordered by sample values in a random sample of size $n$ from the two parameter GEM$(α,θ)$ random discrete distribution on the positive integers. These frequencies are a $($size$-α)$-biased random permutation of the sample frequencies in either ranked order, or in the order of appearance of values in the sampling process. This generalizes a well known identity in distribution due to Donnelly and Tavaré (1986) for $α= 0$ to the case $0 \le α< 1$. This description extends to sampling from Gibbs$(α)$ frequencies obtained by suitable conditioning of the GEM$(α,θ)$ model, and yields a value-ordered version of the Chinese Restaurant construction of GEM$(α,θ)$ and Gibbs$(α)$ frequencies in the more usual size-biased order of their appearance. The proofs are based on a general construction of a finite sample $(X_1,\dots,X_n)$ from any random frequencies in size-biased order from the associated exchangeable random partition $Π_\infty$ of $\mathbb{N}$ which they generate.
△ Less
Submitted 26 August, 2017; v1 submitted 16 April, 2017;
originally announced April 2017.
-
Regenerative random permutations of integers
Authors:
Jim Pitman,
Wenpin Tang
Abstract:
Motivated by recent studies of large Mallows$(q)$ permutations, we propose a class of random permutations of $\mathbb{N}_{+}$ and of $\mathbb{Z}$, called regenerative permutations. Many previous results of the limiting Mallows$(q)$ permutations are recovered and extended. Three special examples: blocked permutations, p-shifted permutations and p-biased permutations are studied.
Motivated by recent studies of large Mallows$(q)$ permutations, we propose a class of random permutations of $\mathbb{N}_{+}$ and of $\mathbb{Z}$, called regenerative permutations. Many previous results of the limiting Mallows$(q)$ permutations are recovered and extended. Three special examples: blocked permutations, p-shifted permutations and p-biased permutations are studied.
△ Less
Submitted 15 June, 2019; v1 submitted 4 April, 2017;
originally announced April 2017.
-
Extremes and gaps in sampling from a GEM random discrete distribution
Authors:
Jim Pitman,
Yuri Yakubovich
Abstract:
We show that in a sample of size $n$ from a GEM$(0,θ)$ random discrete distribution, the gaps $G_{i:n}:= X_{n-i+1:n} - X_{n-i:n}$ between order statistics $X_{1:n} \le \cdots \le X_{n:n}$ of the sample, with the convention $G_{n:n} := X_{1:n} - 1$, are distributed like the first $n$ terms of an infinite sequence of independent geometric$(i/(i+θ))$ variables $G_i$. This extends a known result for t…
▽ More
We show that in a sample of size $n$ from a GEM$(0,θ)$ random discrete distribution, the gaps $G_{i:n}:= X_{n-i+1:n} - X_{n-i:n}$ between order statistics $X_{1:n} \le \cdots \le X_{n:n}$ of the sample, with the convention $G_{n:n} := X_{1:n} - 1$, are distributed like the first $n$ terms of an infinite sequence of independent geometric$(i/(i+θ))$ variables $G_i$. This extends a known result for the minimum $X_{1:n}$ to other gaps in the range of the sample, and implies that the maximum $X_{n:n}$ has the distribution of $1 + \sum_{i=1}^n G_i$, hence the known result that $X_{n:n}$ grows like $θ\log(n)$ as $n\to\infty$, with an asymptotically normal distribution. Other consequences include most known formulas for the exact distributions of GEM$(0,θ)$ sampling statistics, including the Ewens and Donnelly--Tavaré sampling formulas. For the two-parameter GEM$(α,θ)$ distribution we show that the maximal value grows like a random multiple of $n^{α/(1-α)}$ and find the limit distribution of the multiplier.
△ Less
Submitted 23 January, 2017;
originally announced January 2017.
-
The argmin process of random walks and Lévy processes
Authors:
Jim Pitman,
Wenpin Tang
Abstract:
In this paper we consider the argmin process of random walks and Lévy processes. We prove that they enjoy the Markov property, and provide their transition kernels in some special cases.
In this paper we consider the argmin process of random walks and Lévy processes. We prove that they enjoy the Markov property, and provide their transition kernels in some special cases.
△ Less
Submitted 20 June, 2018; v1 submitted 19 October, 2016;
originally announced October 2016.
-
The argmin process of random walks, Brownian motion and Lévy processes
Authors:
Jim Pitman,
Wenpin Tang
Abstract:
In this paper we investigate the argmin process of Brownian motion $B$ defined by $α_t:=\sup\left\{s \in [0,1]: B_{t+s}=\min_{u \in [0,1]}B_{t+u} \right\}$ for $t \geq 0$. The argmin process $α$ is stationary,with invariant measure which is arcsine distributed. We prove that $(α_t; t \geq 0)$ is a Markov process with the Feller property, and provide its transition kernel $Q_t(x,\cdot)$ for $t>0$ a…
▽ More
In this paper we investigate the argmin process of Brownian motion $B$ defined by $α_t:=\sup\left\{s \in [0,1]: B_{t+s}=\min_{u \in [0,1]}B_{t+u} \right\}$ for $t \geq 0$. The argmin process $α$ is stationary,with invariant measure which is arcsine distributed. We prove that $(α_t; t \geq 0)$ is a Markov process with the Feller property, and provide its transition kernel $Q_t(x,\cdot)$ for $t>0$ and $x \in [0,1]$. Similar results for the argmin process of random walks and Lévy processes are derived. We also consider Brownian extrema of a given length. We prove that these extrema form a delayed renewal process with an explicit path construction. We also give a path decomposition for Brownian motion at these extrema
△ Less
Submitted 20 June, 2018; v1 submitted 5 October, 2016;
originally announced October 2016.
-
Successive maxima of samples from a GEM distribution
Authors:
Jim Pitman,
Yuri Yakubovich
Abstract:
We show that the maximal value in a size $n$ sample from GEM$(θ)$ distribution is distributed as a sum of independent geometric random variables. This implies that the maximal value grows as $θ\log(n)$ as $n\to\infty$. For the two-parametric GEM$(α,θ)$ distribution we show that the maximal value grows as a random factor of $n^{α/(1-α)}$ and find the limiting distribution.
We show that the maximal value in a size $n$ sample from GEM$(θ)$ distribution is distributed as a sum of independent geometric random variables. This implies that the maximal value grows as $θ\log(n)$ as $n\to\infty$. For the two-parametric GEM$(α,θ)$ distribution we show that the maximal value grows as a random factor of $n^{α/(1-α)}$ and find the limiting distribution.
△ Less
Submitted 6 September, 2016;
originally announced September 2016.
-
A direct approach to the stable distributions
Authors:
E. J. G. Pitman,
Jim Pitman
Abstract:
The explicit form for the characteristic function of a stable distribution on the line is derived analytically by solving the associated functional equation and applying theory of regular variation, without appeal to the general Lévy-Khintchine integral representation of infinitely divisible distributions.
The explicit form for the characteristic function of a stable distribution on the line is derived analytically by solving the associated functional equation and applying theory of regular variation, without appeal to the general Lévy-Khintchine integral representation of infinitely divisible distributions.
△ Less
Submitted 25 April, 2016;
originally announced April 2016.
-
Tree formulas, mean first passage times and Kemeny's constant of a Markov chain
Authors:
Jim Pitman,
Wenpin Tang
Abstract:
In this paper, we aim to provide probabilistic and combinatorial insights into tree formulas for the Green function and hitting probabilities of Markov chains on a finite state space. These tree formulas are closely related to loop-erased random walks by Wilson's algorithm for random spanning trees, and to mixing times by the Markov chain tree theorem. Let $m_{ij}$ be the mean first passage time f…
▽ More
In this paper, we aim to provide probabilistic and combinatorial insights into tree formulas for the Green function and hitting probabilities of Markov chains on a finite state space. These tree formulas are closely related to loop-erased random walks by Wilson's algorithm for random spanning trees, and to mixing times by the Markov chain tree theorem. Let $m_{ij}$ be the mean first passage time from $i$ to $j$ for an irreducible chain with finite state space $S$ and transition matrix $(p_{ij}; i, j \in S)$. It is well-known that $m_{jj} = 1/π_j = Σ^{(1)}/Σ_j$, where $π$ is the stationary distribution for the chain, $Σ_j$ is the tree sum, over $n^{n-2}$ trees $\textbf{t}$ spanning $S$ with root $j$ and edges $i \rightarrow k$ directed to $j$, of the tree product $\prod_{i \rightarrow k \in \textbf{t} }p_{ik}$, and $Σ^{(1)}:= \sum_{j \in S} Σ_j$. Chebotarev and Agaev derived further results from {\em Kirchhoff's matrix tree theorem}. We deduce that for $i \ne j$, $m_{ij} = Σ_{ij}/Σ_j$, where $Σ_{ij}$ is the sum over the same set of $n^{n-2}$ spanning trees of the same tree product as for $Σ_j$, except that in each product the factor $p_{kj}$ is omitted where $k = k(i,j,\textbf{t})$ is the last state before $j$ in the path from $i$ to $j$ in $\textbf{t}$. It follows that Kemeny's constant $\sum_{j \in S} m_{ij}/m_{jj}$ equals to $ Σ^{(2)}/Σ^{(1)}$, where $Σ^{(r)}$ is the sum, over all forests $\textbf{f}$ labeled by $S$ with $r$ trees, of the product of $p_{ij}$ over edges $i \rightarrow j$ of $\textbf{t}$. We show that these results can be derived without appeal to the matrix tree theorem. A list of relevant literature is also reviewed.
△ Less
Submitted 7 February, 2018; v1 submitted 29 March, 2016;
originally announced March 2016.
-
Beta-gamma tail asymptotics
Authors:
Jim Pitman,
Miklos Z. Racz
Abstract:
We compute the tail asymptotics of the product of a beta random variable and a generalized gamma random variable which are independent and have general parameters. A special case of these asymptotics were proved and used in a recent work of Bubeck, Mossel, and Rácz in order to determine the tail asymptotics of the maximum degree of the preferential attachment tree. The proof presented here is simp…
▽ More
We compute the tail asymptotics of the product of a beta random variable and a generalized gamma random variable which are independent and have general parameters. A special case of these asymptotics were proved and used in a recent work of Bubeck, Mossel, and Rácz in order to determine the tail asymptotics of the maximum degree of the preferential attachment tree. The proof presented here is simpler and highlights why these asymptotics hold.
△ Less
Submitted 8 September, 2015;
originally announced September 2015.
-
The spans in Brownian motion
Authors:
Steven N. Evans,
Jim Pitman,
Wenpin Tang
Abstract:
For $d \in \{1,2,3\}$, let $(B^d_t;~ t \geq 0)$ be a $d$-dimensional standard Brownian motion. We study the $d$-Brownian span set $Span(d):=\{t-s;~ B^d_s=B^d_t~\mbox{for some}~0 \leq s \leq t\}$. We prove that almost surely the random set $Span(d)$ is $σ$-compact and dense in $\mathbb{R}_{+}$. In addition, we show that $Span(1)=\mathbb{R}_{+}$ almost surely; the Lebesgue measure of $Span(2)$ is…
▽ More
For $d \in \{1,2,3\}$, let $(B^d_t;~ t \geq 0)$ be a $d$-dimensional standard Brownian motion. We study the $d$-Brownian span set $Span(d):=\{t-s;~ B^d_s=B^d_t~\mbox{for some}~0 \leq s \leq t\}$. We prove that almost surely the random set $Span(d)$ is $σ$-compact and dense in $\mathbb{R}_{+}$. In addition, we show that $Span(1)=\mathbb{R}_{+}$ almost surely; the Lebesgue measure of $Span(2)$ is $0$ almost surely and its Hausdorff dimension is $1$ almost surely; and the Hausdorff dimension of $Span(3)$ is $\frac{1}{2}$ almost surely. We also list a number of conjectures and open problems.
△ Less
Submitted 23 July, 2017; v1 submitted 5 June, 2015;
originally announced June 2015.
-
Random Dirichlet series arising from records
Authors:
Ron Peled,
Yuval Peres,
Jim Pitman,
Ryokichi Tanaka
Abstract:
We study the distributions of the random Dirichlet series with parameters $(s, β)$ defined by $$ S=\sum_{n=1}^{\infty}\frac{I_n}{n^s}, $$ where $(I_n)$ is a sequence of independent Bernoulli random variables, $I_n$ taking value $1$ with probability $1/n^β$ and value $0$ otherwise. Random series of this type are motivated by the record indicator sequences which have been studied in extreme value th…
▽ More
We study the distributions of the random Dirichlet series with parameters $(s, β)$ defined by $$ S=\sum_{n=1}^{\infty}\frac{I_n}{n^s}, $$ where $(I_n)$ is a sequence of independent Bernoulli random variables, $I_n$ taking value $1$ with probability $1/n^β$ and value $0$ otherwise. Random series of this type are motivated by the record indicator sequences which have been studied in extreme value theory in statistics. We show that when $s>0$ and $0< β\le 1$ with $s+β>1$ the distribution of $S$ has a density; otherwise it is purely atomic or not defined because of divergence. In particular, in the case when $s>0$ and $β=1$, we prove that for every $0<s<1$ the density is bounded and continuous, whereas for every $s>1$ it is unbounded. In the case when $s>0$ and $0<β<1$ with $s+β>1$, the density is smooth. To show the absolute continuity, we obtain estimates of the Fourier transforms, employing van der Corput's method to deal with number-theoretic problems. We also give further regularity results of the densities, and present an example of non atomic singular distribution which is induced by the series restricted to the primes.
△ Less
Submitted 12 August, 2015; v1 submitted 24 May, 2015;
originally announced May 2015.
-
Martingale marginals do not always determine convergence
Authors:
Jim Pitman
Abstract:
Baez-Duarte (1971) and Gilat (1972) gave examples of martingales that converge in probability (and hence in distribution) but not almost surely. Here such a martingale is constructed with uniformly bounded increments, and a construction is provided of two martingales with the same marginals, one of which converges almost surely, while the other does not converge in probability.
Baez-Duarte (1971) and Gilat (1972) gave examples of martingales that converge in probability (and hence in distribution) but not almost surely. Here such a martingale is constructed with uniformly bounded increments, and a construction is provided of two martingales with the same marginals, one of which converges almost surely, while the other does not converge in probability.
△ Less
Submitted 26 March, 2015;
originally announced March 2015.
-
Patterns in random walks and Brownian motion
Authors:
Jim Pitman,
Wenpin Tang
Abstract:
We ask if it is possible to find some particular continuous paths of unit length in linear Brownian motion. Beginning with a discrete version of the problem, we derive the asymptotics of the expected waiting time for several interesting patterns. These suggest corresponding results on the existence/non-existence of continuous paths embedded in Brownian motion. With further effort we are able to pr…
▽ More
We ask if it is possible to find some particular continuous paths of unit length in linear Brownian motion. Beginning with a discrete version of the problem, we derive the asymptotics of the expected waiting time for several interesting patterns. These suggest corresponding results on the existence/non-existence of continuous paths embedded in Brownian motion. With further effort we are able to prove some of these existence and non-existence results by various stochastic analysis arguments. A list of open problems is presented.
△ Less
Submitted 10 September, 2015; v1 submitted 31 October, 2014;
originally announced November 2014.
-
The Slepian zero set, and Brownian bridge embedded in Brownian motion by a spacetime shift
Authors:
Jim Pitman,
Wenpin Tang
Abstract:
This paper is concerned with various aspects of the Slepian process $(B_{t+1} - B_t, t \ge 0)$ derived from a one-dimensional Brownian motion $(B_t, t \ge 0 )$. In particular, we offer an analysis of the local structure of the Slepian zero set $\{t : B_{t+1} = B_t \}$, including a path decomposition of the Slepian process for $0 \le t \le 1$. We also establish the existence of a random time $T$ su…
▽ More
This paper is concerned with various aspects of the Slepian process $(B_{t+1} - B_t, t \ge 0)$ derived from a one-dimensional Brownian motion $(B_t, t \ge 0 )$. In particular, we offer an analysis of the local structure of the Slepian zero set $\{t : B_{t+1} = B_t \}$, including a path decomposition of the Slepian process for $0 \le t \le 1$. We also establish the existence of a random time $T$ such that $T$ falls in the the Slepian zero set almost surely and the process $(B_{T+u} - B_T, 0 \le u \le 1)$ is standard Brownian bridge.
△ Less
Submitted 11 June, 2015; v1 submitted 31 October, 2014;
originally announced November 2014.
-
The Vervaat transform of Brownian bridges and Brownian motion
Authors:
Titus Lupu,
Jim Pitman,
Wenpin Tang
Abstract:
For a continuous function $f \in \mathcal{C}([0,1])$, define the Vervaat transform $V(f)(t):=f(τ(f)+t \mod1)+f(1)1_{\{t+τ(f) \geq 1\}}-f(τ(f))$, where $τ(f)$ corresponds to the first time at which the minimum of $f$ is attained. Motivated by recent study of quantile transforms of random walks and Brownian motion, we investigate the Vervaat transform of Brownian motion and Brownian bridges with arb…
▽ More
For a continuous function $f \in \mathcal{C}([0,1])$, define the Vervaat transform $V(f)(t):=f(τ(f)+t \mod1)+f(1)1_{\{t+τ(f) \geq 1\}}-f(τ(f))$, where $τ(f)$ corresponds to the first time at which the minimum of $f$ is attained. Motivated by recent study of quantile transforms of random walks and Brownian motion, we investigate the Vervaat transform of Brownian motion and Brownian bridges with arbitrary endpoints. When the two endpoints of the bridge are not the same, the Vervaat transform is not Markovian. We describe its distribution by path decomposition and study its semi-martingale property. The same study is done for the Vervaat transform of unconditioned Brownian motion, the expectation and variance of which are also derived.
△ Less
Submitted 7 May, 2015; v1 submitted 14 October, 2013;
originally announced October 2013.
-
On Vervaat transform of Brownian bridges and Brownian motion
Authors:
Jim Pitman,
Wenpin Tang
Abstract:
For a continuous function $f \in \mathcal{C}([0,1])$, define the Vervaat transform $V(f)(t):=f(τ(f)+t \mod1)+f(1)1_{\{t+τ(f) \geq 1\}}-f(τ(f))$, where $τ(f)$ corresponds to the first time at which the minimum of $f$ is attained. Motivated by recent study of quantile transforms for random walks and Brownian motion, we study the Vervaat transform of Brownian motion and Brownian bridges with arbitary…
▽ More
For a continuous function $f \in \mathcal{C}([0,1])$, define the Vervaat transform $V(f)(t):=f(τ(f)+t \mod1)+f(1)1_{\{t+τ(f) \geq 1\}}-f(τ(f))$, where $τ(f)$ corresponds to the first time at which the minimum of $f$ is attained. Motivated by recent study of quantile transforms for random walks and Brownian motion, we study the Vervaat transform of Brownian motion and Brownian bridges with arbitary endpoints. When the two endpoints of the bridge are not the same, the Vervaat transform is not Markovian. We describe its distribution by path decompositions and study its semimartingale properties. The expectation and variance of the Vervaat transform of Brownian motion are also derived.
△ Less
Submitted 14 October, 2013; v1 submitted 30 July, 2013;
originally announced July 2013.
-
The quantile transform of a simple walk
Authors:
Sami Assaf,
Noah Forman,
Jim Pitman
Abstract:
We examine a new path transform on 1-dimensional simple random walks and Brownian motion, the quantile transform. This transformation relates to identities in fluctuation theory due to Wendel, Port, Dassios and others, and to discrete and Brownian versions of Tanaka's formula. For an n-step random walk, the quantile transform reorders increments according to the value of the walk at the start of e…
▽ More
We examine a new path transform on 1-dimensional simple random walks and Brownian motion, the quantile transform. This transformation relates to identities in fluctuation theory due to Wendel, Port, Dassios and others, and to discrete and Brownian versions of Tanaka's formula. For an n-step random walk, the quantile transform reorders increments according to the value of the walk at the start of each increment. We describe the distribution of the quantile transform of a simple random walk of n steps, using a bijection to characterize the number of pre-images of each possible transformed path. We deduce, both for simple random walks and for Brownian motion, that the quantile transform has the same distribution as Vervaat's transform. For Brownian motion, the quantile transforms of the embedded simple random walks converge to a time change of the local time profile. We characterize the distribution of the local time profile, giving rise to an identity that generalizes a variant of Jeulin's description of the local time profile of a Brownian bridge or excursion.
△ Less
Submitted 18 July, 2013;
originally announced July 2013.
-
Regenerative tree growth: Markovian embedding of fragmenters, bifurcators, and bead splitting processes
Authors:
Jim Pitman,
Matthias Winkel
Abstract:
Some, but not all processes of the form $M_t=\exp(-ξ_t)$ for a pure-jump subordinator $ξ$ with Laplace exponent $Φ$ arise as residual mass processes of particle 1 (tagged particle) in Bertoin's partition-valued exchangeable fragmentation processes. We introduce the notion of a Markovian embedding of $M=(M_t,t\ge 0)$ in a fragmentation process, and we show that for each $Φ$, there is a unique (in d…
▽ More
Some, but not all processes of the form $M_t=\exp(-ξ_t)$ for a pure-jump subordinator $ξ$ with Laplace exponent $Φ$ arise as residual mass processes of particle 1 (tagged particle) in Bertoin's partition-valued exchangeable fragmentation processes. We introduce the notion of a Markovian embedding of $M=(M_t,t\ge 0)$ in a fragmentation process, and we show that for each $Φ$, there is a unique (in distribution) binary fragmentation process in which $M$ has a Markovian embedding. The identification of the Laplace exponent $Φ^*$ of its tagged particle process $M^*$ gives rise to a symmetrisation operation $Φ\mapstoΦ^*$, which we investigate in a general study of pairs $(M,M^*)$ that coincide up to a random time and then evolve independently. We call $M$ a fragmenter and $(M,M^*)$ a bifurcator. For $α>0$, we equip the interval $R_1=[0,\int_0^{\infty}M_t^α\,dt]$ with a purely atomic probability measure $μ_1$, which captures the jump sizes of $M$ suitably placed on $R_1$. We study binary tree growth processes that in the $n$th step sample an atom (``bead'') from $μ_n$ and build $(R_{n+1},μ_{n+1})$ by replacing the atom by a rescaled independent copy of $(R_1,μ_1)$ that we tie to the position of the atom. We show that any such bead splitting process $((R_n,μ_n),n\ge1)$ converges almost surely to an $α$-self-similar continuum random tree of Haas and Miermont, in the Gromov-Hausdorff-Prohorov sense. This generalises Aldous's line-breaking construction of the Brownian continuum random tree.
△ Less
Submitted 17 November, 2015; v1 submitted 2 April, 2013;
originally announced April 2013.
-
Simultaneous Exoplanet Characterization and deep wide-field imaging with a diffractive pupil telescope
Authors:
Olivier Guyon,
Josh A. Eisner,
Roger Angel,
Neville J. Woolf,
Eduardo A. Bendek,
Thomas D. Milster,
Stephen M. Ammons,
Michael Shao,
Stuart Shaklan,
Marie Levine,
Bijan Nemati,
Frantz Martinache,
Joe Pitman,
Robert A. Woodruff,
Ruslan Belikov
Abstract:
High-precision astrometry can identify exoplanets and measure their orbits and masses, while coronagraphic imaging enables detailed characterization of their physical properties and atmospheric compositions through spectroscopy. In a previous paper, we showed that a diffractive pupil telescope (DPT) in space can enable sub-microarcsecond accuracy astrometric measurements from wide-field images by…
▽ More
High-precision astrometry can identify exoplanets and measure their orbits and masses, while coronagraphic imaging enables detailed characterization of their physical properties and atmospheric compositions through spectroscopy. In a previous paper, we showed that a diffractive pupil telescope (DPT) in space can enable sub-microarcsecond accuracy astrometric measurements from wide-field images by creating faint but sharp diffraction spikes around the bright target star. The DPT allows simultaneous astrometric measurement and coronagraphic imaging, and we discuss and quantify in this paper the scientific benefits of this combination for exoplanet science investigations: identification of exoplanets with increased sensitivity and robustness, and ability to measure planetary masses to high accuracy. We show how using both measurements to identify planets and measure their masses offers greater sensitivity and provides more reliable measurements than possible with separate missions, and therefore results in a large gain in mission efficiency. The combined measurements reliably identify potentially habitable planets in multiple systems with a few observations, while astrometry or imaging alone would require many measurements over a long time baseline. In addition, the combined measurement allows direct determination of stellar masses to percent-level accuracy, using planets as test particles. We also show that the DPT maintains the full sensitivity of the telescope for deep wide-field imaging, and is therefore compatible with simultaneous scientific observations unrelated to exoplanets. We conclude that astrometry, coronagraphy, and deep wide-field imaging can be performed simultaneously on a single telescope without significant negative impact on the performance of any of the three techniques.
△ Less
Submitted 1 April, 2013;
originally announced April 2013.
-
High precision astrometry with a diffractive pupil telescope
Authors:
Olivier Guyon,
Eduardo A. Bendek,
Thomas D. Milster,
Josh A. Eisner,
Roger Angel,
Neville J. Woolf,
Stephen M. Ammons,
Michael Shao,
Stuart Shaklan,
Marie Levine,
Bijan Nemati,
Joe Pitman,
Robert A. Woodruff,
Ruslan Belikov
Abstract:
Astrometric detection and mass determination of Earth-mass exoplanets requires sub-microarcsec accuracy, which is theoretically possible with an imaging space telescope using field stars as an astrometric reference. The measurement must however overcome astrometric distortions which are much larger than the photon noise limit. To address this issue, we propose to generate faint stellar diffraction…
▽ More
Astrometric detection and mass determination of Earth-mass exoplanets requires sub-microarcsec accuracy, which is theoretically possible with an imaging space telescope using field stars as an astrometric reference. The measurement must however overcome astrometric distortions which are much larger than the photon noise limit. To address this issue, we propose to generate faint stellar diffraction spikes using a two-dimensional grid of regularly spaced small dark spots added to the surface of the primary mirror (PM). Accurate astrometric motion of the host star is obtained by comparing the position of the spikes to the background field stars. The spikes do not contribute to scattered light in the central part of the field and therefore allow unperturbed coronagraphic observation of the star's immediate surrounding. Because the diffraction spikes are created on the PM and imaged on the same focal plane detector as the background stars, astrometric distortions affect equally the diffraction spikes and the background stars, and are therefore calibrated. We describe the technique, detail how the data collected by the wide-field camera are used to derive astrometric motion, and identify the main sources of astrometric error using numerical simulations and analytical derivations. We find that the 1.4 m diameter telescope, 0.3 sq.deg field we adopt as a baseline design achieves 0.2 microarcsec single measurement astrometric accuracy. The diffractive pupil concept thus enables sub-microarcsec astrometry without relying on the accurate pointing, external metrology or high stability hardware required with previously proposed high precision astrometry concepts.
△ Less
Submitted 1 April, 2013;
originally announced April 2013.
-
Feature allocations, probability functions, and paintboxes
Authors:
Tamara Broderick,
Jim Pitman,
Michael I. Jordan
Abstract:
The problem of inferring a clustering of a data set has been the subject of much research in Bayesian analysis, and there currently exists a solid mathematical foundation for Bayesian approaches to clustering. In particular, the class of probability distributions over partitions of a data set has been characterized in a number of ways, including via exchangeable partition probability functions (EP…
▽ More
The problem of inferring a clustering of a data set has been the subject of much research in Bayesian analysis, and there currently exists a solid mathematical foundation for Bayesian approaches to clustering. In particular, the class of probability distributions over partitions of a data set has been characterized in a number of ways, including via exchangeable partition probability functions (EPPFs) and the Kingman paintbox. Here, we develop a generalization of the clustering problem, called feature allocation, where we allow each data point to belong to an arbitrary, non-negative integer number of groups, now called features or topics. We define and study an "exchangeable feature probability function" (EFPF)---analogous to the EPPF in the clustering setting---for certain types of feature models. Moreover, we introduce a "feature paintbox" characterization---analogous to the Kingman paintbox for clustering---of the class of exchangeable feature models. We provide a further characterization of the subclass of feature allocations that have EFPF representations.
△ Less
Submitted 29 January, 2013; v1 submitted 28 January, 2013;
originally announced January 2013.
-
Size-biased permutation of a finite sequence with independent and identically distributed terms
Authors:
Jim Pitman,
Ngoc M. Tran
Abstract:
This paper focuses on the size-biased permutation of $n$ independent and identically distributed (i.i.d.) positive random variables. This is a finite dimensional analogue of the size-biased permutation of ranked jumps of a subordinator studied in Perman-Pitman-Yor (PPY) [Probab. Theory Related Fields 92 (1992) 21-39], as well as a special form of induced order statistics [Bull. Inst. Internat. Sta…
▽ More
This paper focuses on the size-biased permutation of $n$ independent and identically distributed (i.i.d.) positive random variables. This is a finite dimensional analogue of the size-biased permutation of ranked jumps of a subordinator studied in Perman-Pitman-Yor (PPY) [Probab. Theory Related Fields 92 (1992) 21-39], as well as a special form of induced order statistics [Bull. Inst. Internat. Statist. 45 (1973) 295-300; Ann. Statist. 2 (1974) 1034-1039]. This intersection grants us different tools for deriving distributional properties. Their comparisons lead to new results, as well as simpler proofs of existing ones. Our main contribution, Theorem 25 in Section 6, describes the asymptotic distribution of the last few terms in a finite i.i.d. size-biased permutation via a Poisson coupling with its few smallest order statistics.
△ Less
Submitted 29 September, 2015; v1 submitted 29 October, 2012;
originally announced October 2012.
-
Regenerative tree growth: structural results and convergence
Authors:
Jim Pitman,
Douglas Rizzolo,
Matthias Winkel
Abstract:
We introduce regenerative tree growth processes as consistent families of random trees with n labelled leaves, n>=1, with a regenerative property at branch points. This framework includes growth processes for exchangeably labelled Markov branching trees, as well as non-exchangeable models such as the alpha-theta model, the alpha-gamma model and all restricted exchangeable models previously studied…
▽ More
We introduce regenerative tree growth processes as consistent families of random trees with n labelled leaves, n>=1, with a regenerative property at branch points. This framework includes growth processes for exchangeably labelled Markov branching trees, as well as non-exchangeable models such as the alpha-theta model, the alpha-gamma model and all restricted exchangeable models previously studied. Our main structural result is a representation of the growth rule by a sigma-finite dislocation measure kappa on the set of partitions of the natural numbers extending Bertoin's notion of exchangeable dislocation measures from the setting of homogeneous fragmentations. We use this representation to establish necessary and sufficient conditions on the growth rule under which we can apply results by Haas and Miermont for unlabelled and not necessarily consistent trees to establish self-similar random trees and residual mass processes as scaling limits. While previous studies exploited some form of exchangeability, our scaling limit results here only require a regularity condition on the convergence of asymptotic frequencies under kappa, in addition to a regular variation condition.
△ Less
Submitted 27 September, 2013; v1 submitted 15 July, 2012;
originally announced July 2012.
-
Cluster and Feature Modeling from Combinatorial Stochastic Processes
Authors:
Tamara Broderick,
Michael I. Jordan,
Jim Pitman
Abstract:
One of the focal points of the modern literature on Bayesian nonparametrics has been the problem of clustering, or partitioning, where each data point is modeled as being associated with one and only one of some collection of groups called clusters or partition blocks. Underlying these Bayesian nonparametric models are a set of interrelated stochastic processes, most notably the Dirichlet process…
▽ More
One of the focal points of the modern literature on Bayesian nonparametrics has been the problem of clustering, or partitioning, where each data point is modeled as being associated with one and only one of some collection of groups called clusters or partition blocks. Underlying these Bayesian nonparametric models are a set of interrelated stochastic processes, most notably the Dirichlet process and the Chinese restaurant process. In this paper we provide a formal development of an analogous problem, called feature modeling, for associating data points with arbitrary nonnegative integer numbers of groups, now called features or topics. We review the existing combinatorial stochastic process representations for the clustering problem and develop analogous representations for the feature modeling problem. These representations include the beta process and the Indian buffet process as well as new representations that provide insight into the connections between these processes. We thereby bring the same level of completeness to the treatment of Bayesian nonparametric feature modeling that has previously been achieved for Bayesian nonparametric clustering.
△ Less
Submitted 1 October, 2013; v1 submitted 25 June, 2012;
originally announced June 2012.
-
A Brief History of the Statistics Department of the University of California at Berkeley
Authors:
Terry Speed,
Jim Pitman,
John Rice
Abstract:
The early history of our department was dominated by Jerzy Neyman (1894-1981), while the next phase was largely in the hands of Neyman's students, with Erich Lehmann (1917-2009) being a central, long-lived and much-loved member of this group. We are very fortunate in having Constance Reid's biography "Neyman -- From Life" and Erich's "Reminiscences of a Statistician: The Company I Kept" and other…
▽ More
The early history of our department was dominated by Jerzy Neyman (1894-1981), while the next phase was largely in the hands of Neyman's students, with Erich Lehmann (1917-2009) being a central, long-lived and much-loved member of this group. We are very fortunate in having Constance Reid's biography "Neyman -- From Life" and Erich's "Reminiscences of a Statistician: The Company I Kept" and other historical material documenting the founding and growth of the department, and the people in it. In what follows, we will draw heavily from these sources, describing what seems to us to be a remarkable success story: one person starting "a cell of statistical research and teaching ... not being hampered by any existing traditions and routines" and seeing that cell grow rapidly into a major force in academic statistics worldwide. That it has remained so for (at least) the half-century after its founding is a testament to the strength of Neyman's model for a department of statistics.
△ Less
Submitted 31 January, 2012;
originally announced January 2012.
-
Archimedes, Gauss, and Stein
Authors:
Jim Pitman,
Nathan Ross
Abstract:
We discuss a characterization of the centered Gaussian distribution which can be read from results of Archimedes and Maxwell, and relate it to Charles Stein's well-known characterization of the same distribution. These characterizations fit into a more general framework involving the beta-gamma algebra, which explains some other characterizations appearing in the Stein's method literature.
We discuss a characterization of the centered Gaussian distribution which can be read from results of Archimedes and Maxwell, and relate it to Charles Stein's well-known characterization of the same distribution. These characterizations fit into a more general framework involving the beta-gamma algebra, which explains some other characterizations appearing in the Stein's method literature.
△ Less
Submitted 20 January, 2012;
originally announced January 2012.
-
Schröder's problems and scaling limits of random trees
Authors:
Jim Pitman,
Douglas Rizzolo
Abstract:
In a classic paper Schröder posed four combinatorial problems about the number of certain types of bracketings of words and sets. Here we address what these bracketings look like on average. For each of the four problems we prove that a uniform pick from the appropriate set of bracketings, when considered as a tree, has the Brownian continuum random tree as its scaling limit as the size of the wor…
▽ More
In a classic paper Schröder posed four combinatorial problems about the number of certain types of bracketings of words and sets. Here we address what these bracketings look like on average. For each of the four problems we prove that a uniform pick from the appropriate set of bracketings, when considered as a tree, has the Brownian continuum random tree as its scaling limit as the size of the word or set goes to infinity.
△ Less
Submitted 22 September, 2013; v1 submitted 9 July, 2011;
originally announced July 2011.
-
Beta processes, stick-breaking, and power laws
Authors:
Tamara Broderick,
Michael I. Jordan,
Jim Pitman
Abstract:
The beta-Bernoulli process provides a Bayesian nonparametric prior for models involving collections of binary-valued features. A draw from the beta process yields an infinite collection of probabilities in the unit interval, and a draw from the Bernoulli process turns these into binary-valued features. Recent work has provided stick-breaking representations for the beta process analogous to the we…
▽ More
The beta-Bernoulli process provides a Bayesian nonparametric prior for models involving collections of binary-valued features. A draw from the beta process yields an infinite collection of probabilities in the unit interval, and a draw from the Bernoulli process turns these into binary-valued features. Recent work has provided stick-breaking representations for the beta process analogous to the well-known stick-breaking representation for the Dirichlet process. We derive one such stick-breaking representation directly from the characterization of the beta process as a completely random measure. This approach motivates a three-parameter generalization of the beta process, and we study the power laws that can be obtained from this generalized beta process. We present a posterior inference algorithm for the beta-Bernoulli process that exploits the stick-breaking representation, and we present experimental results for a discrete factor-analysis model.
△ Less
Submitted 15 September, 2011; v1 submitted 2 June, 2011;
originally announced June 2011.
-
Convex minorants of random walks and Lévy processes
Authors:
Josh Abramson,
Jim Pitman,
Nathan Ross,
Gerónimo Uribe Bravo
Abstract:
This article provides an overview of recent work on descriptions and properties of the convex minorant of random walks and Lévy processes which summarize and extend the literature on these subjects.
The results surveyed include point process descriptions of the convex minorant of random walks and Lévy processes on a fixed finite interval, up to an independent exponential time, and in the infinit…
▽ More
This article provides an overview of recent work on descriptions and properties of the convex minorant of random walks and Lévy processes which summarize and extend the literature on these subjects.
The results surveyed include point process descriptions of the convex minorant of random walks and Lévy processes on a fixed finite interval, up to an independent exponential time, and in the infinite horizon case. These descriptions follow from the invariance of these processes under an adequate path transformation. In the case of Brownian motion, we note how further special properties of this process, including time-inversion, imply a sequential description for the convex minorant of the Brownian meander.
△ Less
Submitted 3 February, 2011;
originally announced February 2011.
-
A representation of exchangeable hierarchies by sampling from real trees
Authors:
Noah Forman,
Chris Haulk,
Jim Pitman
Abstract:
A hierarchy on a set $S$, also called a total partition of $S$, is a collection $\mathcal{H}$ of subsets of $S$ such that $S \in \mathcal{H}$, each singleton subset of $S$ belongs to $\mathcal{H}$, and if $A, B \in \mathcal{H}$ then $A \cap B$ equals either $A$ or $B$ or $\varnothing$. Every exchangeable random hierarchy of positive integers has the same distribution as a random hierarchy…
▽ More
A hierarchy on a set $S$, also called a total partition of $S$, is a collection $\mathcal{H}$ of subsets of $S$ such that $S \in \mathcal{H}$, each singleton subset of $S$ belongs to $\mathcal{H}$, and if $A, B \in \mathcal{H}$ then $A \cap B$ equals either $A$ or $B$ or $\varnothing$. Every exchangeable random hierarchy of positive integers has the same distribution as a random hierarchy $\mathcal{H}$ associated as follows with a random real tree $\mathcal{T}$ equipped with root element $0$ and a random probability distribution $p$ on the Borel subsets of $\mathcal{T}$: given $(\mathcal{T},p)$, let $t_1,t_2, ...$ be independent and identically distributed according to $p$, and let $\mathcal{H}$ comprise all singleton subsets of $\mathbb{N}$, and every subset of the form $\{j: t_j \in F_x\}$ as $x$ ranges over $\mathcal{T}$, where $F_x$ is the fringe subtree of $\mathcal{T}$ rooted at $x$. There is also the alternative characterization: every exchangeable random hierarchy of positive integers has the same distribution as a random hierarchy $\mathcal{H}$ derived as follows from a random hierarchy $\mathscr{H}$ on $[0,1]$ and a family $(U_j)$ of IID uniform [0,1] random variables independent of $\mathscr{H}$: let $\mathcal{H}$ comprise all sets of the form $\{j: U_j \in B\}$ as $B$ ranges over the members of $\mathscr{H}$.
△ Less
Submitted 12 September, 2017; v1 submitted 28 January, 2011;
originally announced January 2011.
