Showing 1–14 of 14 results for author: Muller, A

Truly No-Regret Learning in Constrained MDPs

Authors: Adrian Müller, Pragnya Alatur, Volkan Cevher, Giorgia Ramponi, Niao He

Abstract: Constrained Markov decision processes (CMDPs) are a common way to model safety constraints in reinforcement learning. State-of-the-art methods for efficiently solving CMDPs are based on primal-dual algorithms. For these algorithms, all currently known regret bounds allow for error cancellations -- one can compensate for a constraint violation in one round with a strict constraint satisfaction in a… ▽ More Constrained Markov decision processes (CMDPs) are a common way to model safety constraints in reinforcement learning. State-of-the-art methods for efficiently solving CMDPs are based on primal-dual algorithms. For these algorithms, all currently known regret bounds allow for error cancellations -- one can compensate for a constraint violation in one round with a strict constraint satisfaction in another. This makes the online learning process unsafe since it only guarantees safety for the final (mixture) policy but not during learning. As Efroni et al. (2020) pointed out, it is an open question whether primal-dual algorithms can provably achieve sublinear regret if we do not allow error cancellations. In this paper, we give the first affirmative answer. We first generalize a result on last-iterate convergence of regularized primal-dual schemes to CMDPs with multiple constraints. Building upon this insight, we propose a model-based primal-dual algorithm to learn in an unknown CMDP. We prove that our algorithm achieves sublinear regret without error cancellations. △ Less

Submitted 18 March, 2024; v1 submitted 24 February, 2024; originally announced February 2024.

Cancellation-Free Regret Bounds for Lagrangian Approaches in Constrained Markov Decision Processes

Authors: Adrian Müller, Pragnya Alatur, Giorgia Ramponi, Niao He

Abstract: Constrained Markov Decision Processes (CMDPs) are one of the common ways to model safe reinforcement learning problems, where constraint functions model the safety objectives. Lagrangian-based dual or primal-dual algorithms provide efficient methods for learning in CMDPs. For these algorithms, the currently known regret bounds in the finite-horizon setting allow for a "cancellation of errors"; one… ▽ More Constrained Markov Decision Processes (CMDPs) are one of the common ways to model safe reinforcement learning problems, where constraint functions model the safety objectives. Lagrangian-based dual or primal-dual algorithms provide efficient methods for learning in CMDPs. For these algorithms, the currently known regret bounds in the finite-horizon setting allow for a "cancellation of errors"; one can compensate for a constraint violation in one episode with a strict constraint satisfaction in another. However, we do not consider such a behavior safe in practical applications. In this paper, we overcome this weakness by proposing a novel model-based dual algorithm OptAug-CMDP for tabular finite-horizon CMDPs. Our algorithm is motivated by the augmented Lagrangian method and can be performed efficiently. We show that during $K$ episodes of exploring the CMDP, our algorithm obtains a regret of $\tilde{O}(\sqrt{K})$ for both the objective and the constraint violation. Unlike existing Lagrangian approaches, our algorithm achieves this regret without the need for the cancellation of errors. △ Less

Submitted 30 August, 2023; v1 submitted 12 June, 2023; originally announced June 2023.

The Topology-Overlap Trade-Off in Retinal Arteriole-Venule Segmentation

Authors: Angel Victor Juanco Muller, Joao F. C. Mota, Keith A. Goatman, Corne Hoogendoorn

Abstract: Retinal fundus images can be an invaluable diagnosis tool for screening epidemic diseases like hypertension or diabetes. And they become especially useful when the arterioles and venules they depict are clearly identified and annotated. However, manual annotation of these vessels is extremely time demanding and taxing, which calls for automatic segmentation. Although convolutional neural networks… ▽ More Retinal fundus images can be an invaluable diagnosis tool for screening epidemic diseases like hypertension or diabetes. And they become especially useful when the arterioles and venules they depict are clearly identified and annotated. However, manual annotation of these vessels is extremely time demanding and taxing, which calls for automatic segmentation. Although convolutional neural networks can achieve high overlap between predictions and expert annotations, they often fail to produce topologically correct predictions of tubular structures. This situation is exacerbated by the bifurcation versus crossing ambiguity which causes classification mistakes. This paper shows that including a topology preserving term in the loss function improves the continuity of the segmented vessels, although at the expense of artery-vein misclassification and overall lower overlap metrics. However, we show that by including an orientation score guided convolutional module, based on the anisotropic single sided cake wavelet, we reduce such misclassification and further increase the topology correctness of the results. We evaluate our model on public datasets with conveniently chosen metrics to assess both overlap and topology correctness, showing that our model is able to produce results on par with state-of-the-art from the point of view of overlap, while increasing topological accuracy. △ Less

Submitted 31 March, 2023; originally announced March 2023.

Comments: To be published in proceedings of SPIE Medical Imaging 2023 Image Processing

Joint lifetime modelling with matrix distributions

Authors: Albrecher Hansjörg, Bladt Martin, Alaric J. A Müller

Abstract: Acyclic phase-type (PH) distributions have been a popular tool in survival analysis, thanks to their natural interpretation in terms of ageing towards its inevitable absorption. In this paper, we consider an extension to the bivariate setting for the modelling of joint lifetimes. In contrast to previous models in the literature that were based on a separate estimation of the marginal behavior and… ▽ More Acyclic phase-type (PH) distributions have been a popular tool in survival analysis, thanks to their natural interpretation in terms of ageing towards its inevitable absorption. In this paper, we consider an extension to the bivariate setting for the modelling of joint lifetimes. In contrast to previous models in the literature that were based on a separate estimation of the marginal behavior and the dependence structure through a copula, we propose a new time-inhomogeneous version of a multivariate PH class (mIPH) that leads to a model for joint lifetimes without that separation. We study properties of mIPH class members and provide an adapted estimation procedure that allows for right-censoring and covariate information. We show that initial distribution vectors in our construction can be tailored to reflect the dependence of the random variables, and use multinomial regression to determine the influence of covariates on starting probabilities. Moreover, we highlight the flexibility and parsimony, in terms of needed phases, introduced by the time-inhomogeneity. Numerical illustrations are given for the famous data set of joint lifetimes of Frees et al. [15], where 10 phases turn out to be sufficient for a reasonable fitting performance. As a by-product, the proposed approach enables a natural causal interpretation of the association in the ageing mechanism of joint lifetimes that goes beyond a statistical fit. △ Less

Submitted 3 October, 2022; v1 submitted 4 July, 2022; originally announced July 2022.

Hypothesis-based acceptance sampling for modules F and F1 of the European Measuring Instruments Directive

Authors: Katy Klauenberg, Cord A. Müller, Clemens Elster

Abstract: Millions of measuring instruments are verified each year before being placed on the markets worldwide. In the EU, such initial conformity assessments are regulated by the Measuring Instruments Directive (MID). The MID modules F and F1 on product verification allow for statistical acceptance sampling, whereby only random subsets of instruments need to be inspected. This paper re-interprets the acce… ▽ More Millions of measuring instruments are verified each year before being placed on the markets worldwide. In the EU, such initial conformity assessments are regulated by the Measuring Instruments Directive (MID). The MID modules F and F1 on product verification allow for statistical acceptance sampling, whereby only random subsets of instruments need to be inspected. This paper re-interprets the acceptance sampling conditions formulated by the MID. The new interpretation is contrasted with the one advanced in WELMEC guide 8.10, and three advantages have become apparent. Firstly, an economic advantage of the new interpretation is a producers' risk bounded from above, such that measuring instruments with sufficient quality are accepted with a guaranteed probability of no less than 95 %. Secondly, a conceptual advantage is that the new MID interpretation fits into the well-known, formal framework of statistical hypothesis testing. Thirdly, the new interpretation applies unambiguously to finite-sized lots, even very small ones. We conclude that the new interpretation is to be preferred and suggest re-formulating the statistical sampling conditions in the MID. Re-interpreting the MID conditions implies that currently available sampling plans are either not admissible or not optimal. We derive a new acceptance sampling scheme and recommend its application. △ Less

Submitted 12 March, 2021; originally announced March 2021.

Comments: accepted, Statistics and Public Policy

Journal ref: Statistics and Public Policy, 2021, Vol. 8, No. 1, 9-17

matrixdist: An R Package for Statistical Analysis of Matrix Distributions

Authors: Martin Bladt, Alaric Mueller, Jorge Yslas

Abstract: The matrixdist R package provides a comprehensive suite of tools for the statistical analysis of matrix distributions, including phase-type, inhomogeneous phase-type, discrete phase-type, and related multivariate distributions. This paper introduces the package and its key features, including the estimation of these distributions and their extensions through expectation-maximisation algorithms, as… ▽ More The matrixdist R package provides a comprehensive suite of tools for the statistical analysis of matrix distributions, including phase-type, inhomogeneous phase-type, discrete phase-type, and related multivariate distributions. This paper introduces the package and its key features, including the estimation of these distributions and their extensions through expectation-maximisation algorithms, as well as the implementation of regression through the proportional intensities and mixture-of-experts models. Additionally, the paper provides an overview of the theoretical background, discusses the algorithms and methods implemented in the package, and offers practical examples to illustrate the application of matrixdist in real-world scenarios. The matrixdist R package aims to provide researchers and practitioners a wide set of tools for analysing and modelling complex data using matrix distributions. △ Less

Submitted 15 August, 2023; v1 submitted 20 January, 2021; originally announced January 2021.

Importance of Tuning Hyperparameters of Machine Learning Algorithms

Authors: Hilde J. P. Weerts, Andreas C. Mueller, Joaquin Vanschoren

Abstract: The performance of many machine learning algorithms depends on their hyperparameter settings. The goal of this study is to determine whether it is important to tune a hyperparameter or whether it can be safely set to a default value. We present a methodology to determine the importance of tuning a hyperparameter based on a non-inferiority test and tuning risk: the performance loss that is incurred… ▽ More The performance of many machine learning algorithms depends on their hyperparameter settings. The goal of this study is to determine whether it is important to tune a hyperparameter or whether it can be safely set to a default value. We present a methodology to determine the importance of tuning a hyperparameter based on a non-inferiority test and tuning risk: the performance loss that is incurred when a hyperparameter is not tuned, but set to a default value. Because our methods require the notion of a default parameter, we present a simple procedure that can be used to determine reasonable default parameters. We apply our methods in a benchmark study using 59 datasets from OpenML. Our results show that leaving particular hyperparameters at their default value is non-inferior to tuning these hyperparameters. In some cases, leaving the hyperparameter at its default value even outperforms tuning it using a search procedure with a limited number of iterations. △ Less

Submitted 15 July, 2020; originally announced July 2020.

Dependence uncertainty bounds for the energy score and the multivariate Gini mean difference

Authors: Carole Bernard, Alfred Müller

Abstract: The energy distance and energy scores became important tools in multivariate statistics and multivariate probabilistic forecasting in recent years. They are both based on the expected distance of two independent samples. In this paper we study dependence uncertainty bounds for these quantities under the assumption that we know the marginals but do not know the dependence structure. We find some in… ▽ More The energy distance and energy scores became important tools in multivariate statistics and multivariate probabilistic forecasting in recent years. They are both based on the expected distance of two independent samples. In this paper we study dependence uncertainty bounds for these quantities under the assumption that we know the marginals but do not know the dependence structure. We find some interesting sharp analytic bounds, where one of them is obtained for an unusual spherically symmetric copula. These results should help to better understand the sensitivity of these measures to misspecifications in the copula. △ Less

Submitted 24 April, 2020; originally announced April 2020.

MSC Class: 62H05

A copula-based time series model for global horizontal irradiation

Authors: Alfred Müller, Matthias Reuber

Abstract: The increasing importance of solar power for electricity generation leads to an increasing demand for probabilistic forecasting of local and aggregated PV yields. In this paper we use an indirect modeling approach for hourly medium to long term local PV yields based on publicly available irradiation data. We suggest a time series model for global horizontal irradiation for which it is easy to gene… ▽ More The increasing importance of solar power for electricity generation leads to an increasing demand for probabilistic forecasting of local and aggregated PV yields. In this paper we use an indirect modeling approach for hourly medium to long term local PV yields based on publicly available irradiation data. We suggest a time series model for global horizontal irradiation for which it is easy to generate an arbitrary number of scenarios and thus allows for multivariate probabilistic forecasts for arbitrary time horizons. In contrast to many simplified models that have been considered in the literature so far it features several important stylized facts. Sharp time dependent lower and upper bounds of global horizontal irradiations are estimated that improve the often used physical bounds. The parameters of the beta distributed marginals of the transformed data are allowed to be time dependent. A copula-based time series model is introduced for the hourly and daily dependence structure based on a simple graphical structure known from the theory of vine copulas. Non-Gaussian copulas like Gumbel and BB1 copulas are used that allow for the important feature of so-called tail dependence. Evaluation methods like the continuous ranked probability score (CRPS), the energy score (ES) and the variogram score (VS) are used to compare the power of the model for multivariate probabilistic forecasting with other models used in the literature showing that our model outperforms other models in many respects. △ Less

Submitted 21 February, 2020; originally announced February 2020.

Meta-modal Information Flow: A Method for Capturing Multimodal Modular Disconnectivity in Schizophrenia

Authors: Haleh Falakshahi, Victor M. Vergara, **gyu Liu, Daniel H. Mathalon, Judith M. Ford, James Voyvodic, Bryon A. Mueller, Aysenil Belger, Sarah McEwen, Steven G. Potkin, Adrian Preda, Hooman Rokham, **g Sui, Jessica A. Turner, Sergey Plis, Vince D. Calhoun

Abstract: Objective: Multimodal measurements of the same phenomena provide complementary information and highlight different perspectives, albeit each with their own limitations. A focus on a single modality may lead to incorrect inferences, which is especially important when a studied phenomenon is a disease. In this paper, we introduce a method that takes advantage of multimodal data in addressing the hyp… ▽ More Objective: Multimodal measurements of the same phenomena provide complementary information and highlight different perspectives, albeit each with their own limitations. A focus on a single modality may lead to incorrect inferences, which is especially important when a studied phenomenon is a disease. In this paper, we introduce a method that takes advantage of multimodal data in addressing the hypotheses of disconnectivity and dysfunction within schizophrenia (SZ). Methods: We start with estimating and visualizing links within and among extracted multimodal data features using a Gaussian graphical model (GGM). We then propose a modularity-based method that can be applied to the GGM to identify links that are associated with mental illness across a multimodal data set. Through simulation and real data, we show our approach reveals important information about disease-related network disruptions that are missed with a focus on a single modality. We use functional MRI (fMRI), diffusion MRI (dMRI), and structural MRI (sMRI) to compute the fractional amplitude of low frequency fluctuations (fALFF), fractional anisotropy (FA), and gray matter (GM) concentration maps. These three modalities are analyzed using our modularity method. Results: Our results show missing links that are only captured by the cross-modal information that may play an important role in disconnectivity between the components. Conclusion: We identified multimodal (fALFF, FA and GM) disconnectivity in the default mode network area in patients with SZ, which would not have been detectable in a single modality. Significance: The proposed approach provides an important new tool for capturing information that is distributed among multiple imaging modalities. △ Less

Submitted 6 January, 2020; originally announced January 2020.

Journal ref: IEEE Transactions on Biomedical Engineering, 2019

OpenML-Python: an extensible Python API for OpenML

Authors: Matthias Feurer, Jan N. van Rijn, Arlind Kadra, Pieter Gijsbers, Neeratyoy Mallik, Sahithya Ravi, Andreas Müller, Joaquin Vanschoren, Frank Hutter

Abstract: OpenML is an online platform for open science collaboration in machine learning, used to share datasets and results of machine learning experiments. In this paper we introduce OpenML-Python, a client API for Python, opening up the OpenML platform for a wide range of Python-based tools. It provides easy access to all datasets, tasks and experiments on OpenML from within Python. It also provides fun… ▽ More OpenML is an online platform for open science collaboration in machine learning, used to share datasets and results of machine learning experiments. In this paper we introduce OpenML-Python, a client API for Python, opening up the OpenML platform for a wide range of Python-based tools. It provides easy access to all datasets, tasks and experiments on OpenML from within Python. It also provides functionality to conduct machine learning experiments, upload the results to OpenML, and reproduce results which are stored on OpenML. Furthermore, it comes with a scikit-learn plugin and a plugin mechanism to easily integrate other machine learning libraries written in Python into the OpenML ecosystem. Source code and documentation is available at https://github.com/openml/openml-python/. △ Less

Submitted 23 June, 2021; v1 submitted 6 November, 2019; originally announced November 2019.

Journal ref: Journal of Machine Learning Research 22(100), 2021

Optimal acceptance sampling for modules F and F1 of the European Measuring Instruments Directive

Authors: Cord A. Müller

Abstract: Acceptance sampling plans offered by ISO 2859-1 are far from optimal under the conditions for statistical verification in modules F and F1 as prescribed by Annex II of the Measuring Instruments Directive (MID) 2014/32/EU, resulting in sample sizes that are larger than necessary. An optimised single-sampling scheme is derived, both for large lots using the binomial distribution and for finite-sized… ▽ More Acceptance sampling plans offered by ISO 2859-1 are far from optimal under the conditions for statistical verification in modules F and F1 as prescribed by Annex II of the Measuring Instruments Directive (MID) 2014/32/EU, resulting in sample sizes that are larger than necessary. An optimised single-sampling scheme is derived, both for large lots using the binomial distribution and for finite-sized lots using the exact hypergeometric distribution, resulting in smaller sample sizes that are economically more efficient while offering the full statistical protection required by the MID. △ Less

Submitted 28 March, 2019; v1 submitted 15 January, 2019; originally announced January 2019.

Comments: accepted by J. Appl. Stat. (2019)

Learning Multiple Defaults for Machine Learning Algorithms

Authors: Florian Pfisterer, Jan N. van Rijn, Philipp Probst, Andreas Müller, Bernd Bischl

Abstract: The performance of modern machine learning methods highly depends on their hyperparameter configurations. One simple way of selecting a configuration is to use default settings, often proposed along with the publication and implementation of a new algorithm. Those default values are usually chosen in an ad-hoc manner to work good enough on a wide variety of datasets. To address this problem, diffe… ▽ More The performance of modern machine learning methods highly depends on their hyperparameter configurations. One simple way of selecting a configuration is to use default settings, often proposed along with the publication and implementation of a new algorithm. Those default values are usually chosen in an ad-hoc manner to work good enough on a wide variety of datasets. To address this problem, different automatic hyperparameter configuration algorithms have been proposed, which select an optimal configuration per dataset. This principled approach usually improves performance but adds additional algorithmic complexity and computational costs to the training procedure. As an alternative to this, we propose learning a set of complementary default values from a large database of prior empirical results. Selecting an appropriate configuration on a new dataset then requires only a simple, efficient and embarrassingly parallel search over this set. We demonstrate the effectiveness and efficiency of the approach we propose in comparison to random search and Bayesian Optimization. △ Less

Submitted 30 April, 2021; v1 submitted 23 November, 2018; originally announced November 2018.

Machine Learning for Neuroimaging with Scikit-Learn

Authors: Alexandre Abraham, Fabian Pedregosa, Michael Eickenberg, Philippe Gervais, Andreas Muller, Jean Kossaifi, Alexandre Gramfort, Bertrand Thirion, Gäel Varoquaux

Abstract: Statistical machine learning methods are increasingly used for neuroimaging data analysis. Their main virtue is their ability to model high-dimensional datasets, e.g. multivariate analysis of activation images or resting-state time series. Supervised learning is typically used in decoding or encoding settings to relate brain images to behavioral or clinical observations, while unsupervised learnin… ▽ More Statistical machine learning methods are increasingly used for neuroimaging data analysis. Their main virtue is their ability to model high-dimensional datasets, e.g. multivariate analysis of activation images or resting-state time series. Supervised learning is typically used in decoding or encoding settings to relate brain images to behavioral or clinical observations, while unsupervised learning can uncover hidden structures in sets of images (e.g. resting state functional MRI) or find sub-populations in large cohorts. By considering different functional neuroimaging applications, we illustrate how scikit-learn, a Python machine learning library, can be used to perform some key analysis steps. Scikit-learn contains a very large set of statistical learning algorithms, both supervised and unsupervised, and its application to neuroimaging data provides a versatile tool to study the brain. △ Less

Submitted 12 December, 2014; originally announced December 2014.

Comments: Frontiers in neuroscience, Frontiers Research Foundation, 2013, pp.15