-
Anytime-Valid Tests of Group Invariance through Conformal Prediction
Authors:
Tyron Lardy,
Muriel Felipe Pérez-Ortiz
Abstract:
We develop anytime-valid tests of invariance under the action of compact groups. The resulting test statistics are optimal in a logarithmic-growth sense. We apply our method to extend recent anytime-valid tests of independence and to construct tests of normality.
We develop anytime-valid tests of invariance under the action of compact groups. The resulting test statistics are optimal in a logarithmic-growth sense. We apply our method to extend recent anytime-valid tests of independence and to construct tests of normality.
△ Less
Submitted 23 May, 2024; v1 submitted 27 January, 2024;
originally announced January 2024.
-
Exponential Stochastic Inequality
Authors:
Peter D. Grünwald,
Muriel F. Pérez-Ortiz,
Zakaria Mhammedi
Abstract:
We develop the concept of exponential stochastic inequality (ESI), a novel notation that simultaneously captures high-probability and in-expectation statements. It is especially well suited to succinctly state, prove, and reason about excess-risk and generalization bounds in statistical learning, specifically, but not restricted to, the PAC-Bayesian type. We show that the ESI satisfies transitivit…
▽ More
We develop the concept of exponential stochastic inequality (ESI), a novel notation that simultaneously captures high-probability and in-expectation statements. It is especially well suited to succinctly state, prove, and reason about excess-risk and generalization bounds in statistical learning, specifically, but not restricted to, the PAC-Bayesian type. We show that the ESI satisfies transitivity and other properties which allow us to use it like standard, nonstochastic inequalities. We substantially extend the original definition from Koolen et al. (2016) and show that general ESIs satisfy a host of useful additional properties, including a novel Markov-like inequality. We show how ESIs relate to, and clarify, PAC-Bayesian bounds, subcentered subgamma random variables and *fast-rate conditions* such as the central and Bernstein conditions. We also show how the ideas can be extended to random scaling factors (learning rates).
△ Less
Submitted 27 April, 2023;
originally announced April 2023.
-
E-Statistics, Group Invariance and Anytime Valid Testing
Authors:
Muriel Felipe Pérez-Ortiz,
Tyron Lardy,
Rianne de Heide,
Peter Grünwald
Abstract:
We study worst-case-growth-rate-optimal (GROW) e-statistics for hypothesis testing between two group models. It is known that under a mild condition on the action of the underlying group G on the data, there exists a maximally invariant statistic. We show that among all e-statistics, invariant or not, the likelihood ratio of the maximally invariant statistic is GROW, both in the absolute and in th…
▽ More
We study worst-case-growth-rate-optimal (GROW) e-statistics for hypothesis testing between two group models. It is known that under a mild condition on the action of the underlying group G on the data, there exists a maximally invariant statistic. We show that among all e-statistics, invariant or not, the likelihood ratio of the maximally invariant statistic is GROW, both in the absolute and in the relative sense, and that an anytime-valid test can be based on it. The GROW e-statistic is equal to a Bayes factor with a right Haar prior on G. Our treatment avoids nonuniqueness issues that sometimes arise for such priors in Bayesian contexts. A crucial assumption on the group G is its amenability, a well-known group-theoretical condition, which holds, for instance, in scale-location families. Our results also apply to finite-dimensional linear regression.
△ Less
Submitted 17 October, 2023; v1 submitted 16 August, 2022;
originally announced August 2022.
-
The Anytime-Valid Logrank Test: Error Control Under Continuous Monitoring with Unlimited Horizon
Authors:
J. ter Schure,
M. F. Perez-Ortiz,
A. Ly,
P. Grunwald
Abstract:
We introduce the anytime-valid (AV) logrank test, a version of the logrank test that provides type-I error guarantees under optional stop** and optional continuation. The test is sequential without the need to specify a maximum sample size or stop** rule, and allows for cumulative meta-analysis with type-I error control. The method can be extended to define anytime-valid confidence intervals.…
▽ More
We introduce the anytime-valid (AV) logrank test, a version of the logrank test that provides type-I error guarantees under optional stop** and optional continuation. The test is sequential without the need to specify a maximum sample size or stop** rule, and allows for cumulative meta-analysis with type-I error control. The method can be extended to define anytime-valid confidence intervals. The logrank test is an instance of the martingale tests based on E-variables that have been recently developed. We demonstrate type-I error guarantees for the test in a semiparametric setting of proportional hazards and show how to extend it to ties, Cox' regression and confidence sequences. Using a Gaussian approximation on the logrank statistic, we show that the AV logrank test (which itself is always exact) has a similar rejection region to O'Brien-Fleming alpha-spending but with the potential to achieve 100% power by optional continuation. Although our approach to study design requires a larger sample size, the *expected* sample size is competitive by optional stop**.
△ Less
Submitted 1 May, 2023; v1 submitted 13 November, 2020;
originally announced November 2020.
-
Machine learning techniques to select Be star candidates. An application in the OGLE-IV Gaia south ecliptic pole field
Authors:
M. F. Pérez-Ortiz,
A. García-Varela,
A. J. Quiroz,
B. E. Sabogal,
J. Hernández
Abstract:
Statistical pattern recognition methods have provided competitive solutions for variable star classification at a relatively low computational cost. In order to perform supervised classification, a set of features is proposed and used to train an automatic classification system. Quantities related to the magnitude density of the light curves and their Fourier coefficients have been chosen as featu…
▽ More
Statistical pattern recognition methods have provided competitive solutions for variable star classification at a relatively low computational cost. In order to perform supervised classification, a set of features is proposed and used to train an automatic classification system. Quantities related to the magnitude density of the light curves and their Fourier coefficients have been chosen as features in previous studies. However, some of these features are not robust to the presence of outliers and the calculation of Fourier coefficients is computationally expensive for large data sets. We propose and evaluate the performance of a new robust set of features using supervised classifiers in order to look for new Be star candidates in the OGLE-IV Gaia south ecliptic pole field. We calculated the proposed set of features on six types of variable stars and on a set of Be star candidates reported in the literature. We evaluated the performance of these features using classification trees and random forests along with K-nearest neighbours, support vector machines, and gradient boosted trees methods. We tuned the classifiers with a 10-fold cross-validation and grid search. We validated the performance of the best classifier on a set of OGLE-IV light curves and applied this to find new Be star candidates. The random forest classifier outperformed the others. By using the random forest classifier and colour criteria we found 50 Be star candidates in the direction of the Gaia south ecliptic pole field, four of which have infrared colours consistent with Herbig Ae/Be stars. Supervised methods are very useful in order to obtain preliminary samples of variable stars extracted from large databases. As usual, the stars classified as Be stars candidates must be checked for the colours and spectroscopic characteristics expected for them.
△ Less
Submitted 14 July, 2017;
originally announced July 2017.
