Showing 1–50 of 51 results for author: Wallin, J

Unveiling low-dimensional patterns induced by convex non-differentiable regularizers

Authors: Ivan Hejný, Jonas Wallin, Małgorzata Bogdan, Michał Kos

Abstract: Popular regularizers with non-differentiable penalties, such as Lasso, Elastic Net, Generalized Lasso, or SLOPE, reduce the dimension of the parameter space by inducing sparsity or clustering in the estimators' coordinates. In this paper, we focus on linear regression and explore the asymptotic distributions of the resulting low-dimensional patterns when the number of regressors $p$ is fixed, the… ▽ More Popular regularizers with non-differentiable penalties, such as Lasso, Elastic Net, Generalized Lasso, or SLOPE, reduce the dimension of the parameter space by inducing sparsity or clustering in the estimators' coordinates. In this paper, we focus on linear regression and explore the asymptotic distributions of the resulting low-dimensional patterns when the number of regressors $p$ is fixed, the number of observations $n$ goes to infinity, and the penalty function increases at the rate of $\sqrt{n}$. While the asymptotic distribution of the rescaled estimation error can be derived by relatively standard arguments, the convergence of the pattern does not simply follow from the convergence in distribution, and requires a careful and separate treatment. For this purpose, we use the Hausdorff distance as a suitable mode of convergence for subdifferentials, resulting in the desired pattern convergence. Furthermore, we derive the exact limiting probability of recovering the true model pattern. This probability goes to 1 if and only if the penalty scaling constant diverges to infinity and the regularizer-specific asymptotic irrepresentability condition is satisfied. We then propose simple two-step procedures that asymptotically recover the model patterns, irrespective whether the irrepresentability condition holds. Interestingly, our theory shows that Fused Lasso cannot reliably recover its own clustering pattern, even for independent regressors. It also demonstrates how this problem can be resolved by ``concavifying'' the Fused Lasso penalty coefficients. Additionally, sampling from the asymptotic error distribution facilitates comparisons between different regularizers. We provide short simulation studies showcasing an illustrative comparison between the asymptotic properties of Lasso, Fused Lasso, and SLOPE. △ Less

Submitted 13 May, 2024; originally announced May 2024.

Comments: 30 pages, 5 figures, for associated code, see https://github.com/IvanHejny/pyslope.git

Spatial confounding under infill asymptotics

Authors: David Bolin, Jonas Wallin

Abstract: The estimation of regression parameters in spatially referenced data plays a crucial role across various scientific domains. A common approach involves employing an additive regression model to capture the relationship between observations and covariates, accounting for spatial variability not explained by the covariates through a Gaussian random field. While theoretical analyses of such models ha… ▽ More The estimation of regression parameters in spatially referenced data plays a crucial role across various scientific domains. A common approach involves employing an additive regression model to capture the relationship between observations and covariates, accounting for spatial variability not explained by the covariates through a Gaussian random field. While theoretical analyses of such models have predominantly focused on prediction and covariance parameter inference, recent attention has shifted towards understanding the theoretical properties of regression coefficient estimates, particularly in the context of spatial confounding. This article studies the effect of misspecified covariates, in particular when the misspecification changes the smoothness. We analyze the theoretical properties of the generalize least-square estimator under infill asymptotics, and show that the estimator can have counter-intuitive properties. In particular, the estimated regression coefficients can converge to zero as the number of observations increases, despite high correlations between observations and covariates. Perhaps even more surprising, the estimates can diverge to infinity under certain conditions. Through an application to temperature and precipitation data, we show that both behaviors can be observed for real data. Finally, we propose a simple fix to the problem by adding a smoothing step in the regression. △ Less

Submitted 27 March, 2024; originally announced March 2024.

Tomographic projection optimization for volumetric additive manufacturing with general band constraint Lp-norm minimization

Authors: Chi Chung Li, Joseph Toombs, Hayden K. Taylor, Thomas J. Wallin

Abstract: Tomographic volumetric additive manufacturing is a rapidly growing fabrication technology that enables rapid production of 3D objects through a single build step. In this process, the design of projections directly impacts geometric resolution, material properties, and manufacturing yield of the final printed part. Herein, we identify the hidden equivalent operations of three major existing projec… ▽ More Tomographic volumetric additive manufacturing is a rapidly growing fabrication technology that enables rapid production of 3D objects through a single build step. In this process, the design of projections directly impacts geometric resolution, material properties, and manufacturing yield of the final printed part. Herein, we identify the hidden equivalent operations of three major existing projection optimization schemes and reformulate them into a general loss function where the optimization behavior can be systematically studied, and unique capabilities of the individual schemes can coalesce. The loss function formulation proposed in this study unified the optimization for binary and greyscale targets and generalized problem relaxation strategies with local tolerancing and weighting. Additionally, this formulation offers control on error sparsity and consistent dose response map** throughout initialization, optimization, and evaluation. A parameter-sweep analysis in this study guides users in tuning optimization parameters for application-specific goals. △ Less

Submitted 13 February, 2024; v1 submitted 3 December, 2023; originally announced December 2023.

Comments: 49 pages, 12 figures

MSC Class: 90C26 ACM Class: J.2; J.6

Statistical inference for Gaussian Whittle-Matérn fields on metric graphs

Authors: David Bolin, Alexandre Simas, Jonas Wallin

Abstract: Whittle-Matérn fields are a recently introduced class of Gaussian processes on metric graphs, which are specified as solutions to a fractional-order stochastic differential equation. Unlike earlier covariance-based approaches for specifying Gaussian fields on metric graphs, the Whittle-Matérn fields are well-defined for any compact metric graph and can provide Gaussian processes with differentiabl… ▽ More Whittle-Matérn fields are a recently introduced class of Gaussian processes on metric graphs, which are specified as solutions to a fractional-order stochastic differential equation. Unlike earlier covariance-based approaches for specifying Gaussian fields on metric graphs, the Whittle-Matérn fields are well-defined for any compact metric graph and can provide Gaussian processes with differentiable sample paths. We derive the main statistical properties of the model class, particularly the consistency and asymptotic normality of maximum likelihood estimators of model parameters and the necessary and sufficient conditions for asymptotic optimality properties of linear prediction based on the model with misspecified parameters. The covariance function of the Whittle-Matérn fields is generally unavailable in closed form, and they have therefore been challenging to use for statistical inference. However, we show that for specific values of the fractional exponent, when the fields have Markov properties, likelihood-based inference and spatial prediction can be performed exactly and computationally efficiently. This facilitates using the Whittle-Matérn fields in statistical applications involving big datasets without the need for any approximations. The methods are illustrated via an application to modeling of traffic data, where allowing for differentiable processes dramatically improves the results. △ Less

Submitted 25 October, 2023; v1 submitted 20 April, 2023; originally announced April 2023.

Comments: 45 pages, 5 figures. arXiv admin note: text overlap with arXiv:2205.06163

MSC Class: Primary 62M30; secondary 62M05; 60G60; 35R02

Markov properties of Gaussian random fields on compact metric graphs

Authors: David Bolin, Alexandre B. Simas, Jonas Wallin

Abstract: There has recently been much interest in Gaussian fields on linear networks and, more generally, on compact metric graphs. One proposed strategy for defining such fields on a metric graph $Γ$ is through a covariance function that is isotropic in a metric on the graph. Another is through a fractional-order differential equation $L^{α/2} (τu) = \mathcal{W}$ on $Γ$, where $L = κ^2 - \nabla(a\nabla)$… ▽ More There has recently been much interest in Gaussian fields on linear networks and, more generally, on compact metric graphs. One proposed strategy for defining such fields on a metric graph $Γ$ is through a covariance function that is isotropic in a metric on the graph. Another is through a fractional-order differential equation $L^{α/2} (τu) = \mathcal{W}$ on $Γ$, where $L = κ^2 - \nabla(a\nabla)$ for (sufficiently nice) functions $κ, a$, and $\mathcal{W}$ is Gaussian white noise. We study Markov properties of these two types of fields. First, we show that no Gaussian random fields exist on general metric graphs that are both isotropic and Markov. Then, we show that the second type of fields, the generalized Whittle--Matérn fields, are Markov if and only if $α\in\mathbb{N}$. Further, if $α\in\mathbb{N}$, a generalized Whittle--Matérn field $u$ is Markov of order $α$, which means that the field $u$ in one region $S\subsetΓ$ is conditionally independent of $u$ in $Γ\setminus S$ given the values of $u$ and its $α-1$ derivatives on $\partial S$. Finally, we provide two results as consequences of the theory developed: first we prove that the Markov property implies an explicit characterization of $u$ on a fixed edge $e$, revealing that the conditional distribution of $u$ on $e$ given the values at the two vertices connected to $e$ is independent of the geometry of $Γ$; second, we show that the solution to $L^{1/2}(τu) = \mathcal{W}$ on $Γ$ can obtained by conditioning independent generalized Whittle--Matérn processes on the edges, with $α=1$ and Neumann boundary conditions, on being continuous at the vertices. △ Less

Submitted 30 August, 2023; v1 submitted 6 April, 2023; originally announced April 2023.

Comments: 29 pages

MSC Class: 60G60 (Primary) 60G15; 60H15 (Secondary)

Weak pattern convergence for SLOPE and its robust versions

Authors: Ivan Hejný, Jonas Wallin, Małgorzata Bogdan

Abstract: The Sorted L-One Estimator (SLOPE) is a popular regularization method in regression, which induces clustering of the estimated coefficients. That is, the estimator can have coefficients of identical magnitude. In this paper, we derive an asymptotic distribution of SLOPE for the ordinary least squares, Huber, and Quantile loss functions, and use it to study the clustering behavior in the limit. Thi… ▽ More The Sorted L-One Estimator (SLOPE) is a popular regularization method in regression, which induces clustering of the estimated coefficients. That is, the estimator can have coefficients of identical magnitude. In this paper, we derive an asymptotic distribution of SLOPE for the ordinary least squares, Huber, and Quantile loss functions, and use it to study the clustering behavior in the limit. This requires a stronger type of convergence since clustering properties do not follow merely from the classical weak convergence. For this aim, we utilize the Hausdorff distance, which provides a suitable notion of convergence for the penalty subdifferentials and a bridge toward weak convergence of the clustering pattern. We establish asymptotic control of the false discovery rate for the asymptotic orthogonal design of the regressor. We also show how to extend the framework to a broader class of regularizers other than SLOPE. △ Less

Submitted 14 April, 2023; v1 submitted 20 March, 2023; originally announced March 2023.

Comments: 30 pages

Incorporating Crowdsourced Annotator Distributions into Ensemble Modeling to Improve Classification Trustworthiness for Ancient Greek Papyri

Authors: Graham West, Matthew I. Swindall, Ben Keener, Timothy Player, Alex C. Williams, James H. Brusuelas, John F. Wallin

Abstract: Performing classification on noisy, crowdsourced image datasets can prove challenging even for the best neural networks. Two issues which complicate the problem on such datasets are class imbalance and ground-truth uncertainty in labeling. The AL-ALL and AL-PUB datasets - consisting of tightly cropped, individual characters from images of ancient Greek papyri - are strongly affected by both issues… ▽ More Performing classification on noisy, crowdsourced image datasets can prove challenging even for the best neural networks. Two issues which complicate the problem on such datasets are class imbalance and ground-truth uncertainty in labeling. The AL-ALL and AL-PUB datasets - consisting of tightly cropped, individual characters from images of ancient Greek papyri - are strongly affected by both issues. The application of ensemble modeling to such datasets can help identify images where the ground-truth is questionable and quantify the trustworthiness of those samples. As such, we apply stacked generalization consisting of nearly identical ResNets with different loss functions: one utilizing sparse cross-entropy (CXE) and the other Kullback-Liebler Divergence (KLD). Both networks use labels drawn from a crowd-sourced consensus. This consensus is derived from a Normalized Distribution of Annotations (NDA) based on all annotations for a given character in the dataset. For the second network, the KLD is calculated with respect to the NDA. For our ensemble model, we apply a k-nearest neighbors model to the outputs of the CXE and KLD networks. Individually, the ResNet models have approximately 93% accuracy, while the ensemble model achieves an accuracy of > 95%, increasing the classification trustworthiness. We also perform an analysis of the Shannon entropy of the various models' output distributions to measure classification uncertainty. Our results suggest that entropy is useful for predicting model misclassifications. △ Less

Submitted 26 January, 2024; v1 submitted 28 October, 2022; originally announced October 2022.

Journal ref: Journal of Data Mining & Digital Humanities, Historical Documents and automatic text recognition, Digital humanities in languages (February 7, 2024) jdmdh:10297

Coordinate Descent for SLOPE

Authors: Johan Larsson, Quentin Klopfenstein, Mathurin Massias, Jonas Wallin

Abstract: The lasso is the most famous sparse regression and feature selection method. One reason for its popularity is the speed at which the underlying optimization problem can be solved. Sorted L-One Penalized Estimation (SLOPE) is a generalization of the lasso with appealing statistical properties. In spite of this, the method has not yet reached widespread interest. A major reason for this is that curr… ▽ More The lasso is the most famous sparse regression and feature selection method. One reason for its popularity is the speed at which the underlying optimization problem can be solved. Sorted L-One Penalized Estimation (SLOPE) is a generalization of the lasso with appealing statistical properties. In spite of this, the method has not yet reached widespread interest. A major reason for this is that current software packages that fit SLOPE rely on algorithms that perform poorly in high dimensions. To tackle this issue, we propose a new fast algorithm to solve the SLOPE optimization problem, which combines proximal gradient descent and proximal coordinate descent steps. We provide new results on the directional derivative of the SLOPE penalty and its related SLOPE thresholding operator, as well as provide convergence guarantees for our proposed solver. In extensive benchmarks on simulated and real data, we show that our method outperforms a long list of competing algorithms. △ Less

Submitted 26 October, 2022; originally announced October 2022.

Comments: 18 pages

MSC Class: 62-08 ACM Class: G.1.6; G.4; G.3

Journal ref: Proceedings of the 26th international conference on artificial intelligence and statistics. Valencia, Spain: PMLR; 2023. p. 4802-21. (PMLR; vol. 206)

Gaussian Whittle-Matérn fields on metric graphs

Authors: David Bolin, Alexandre B. Simas, Jonas Wallin

Abstract: We define a new class of Gaussian processes on compact metric graphs such as street or river networks. The proposed models, the Whittle--Matérn fields, are defined via a fractional stochastic differential equation on the compact metric graph and are a natural extension of Gaussian fields with Matérn covariance functions on Euclidean domains to the non-Euclidean metric graph setting. Existence of t… ▽ More We define a new class of Gaussian processes on compact metric graphs such as street or river networks. The proposed models, the Whittle--Matérn fields, are defined via a fractional stochastic differential equation on the compact metric graph and are a natural extension of Gaussian fields with Matérn covariance functions on Euclidean domains to the non-Euclidean metric graph setting. Existence of the processes, as well as some of their main properties, such as sample path regularity are derived. The model class in particular contains differentiable processes. To the best of our knowledge, this is the first construction of a differentiable Gaussian process on general compact metric graphs. Further, we prove an intrinsic property of these processes: that they do not change upon addition or removal of vertices with degree two. Finally, we obtain Karhunen--Loève expansions of the processes, provide numerical experiments, and compare them to Gaussian processes with isotropic covariance functions. △ Less

Submitted 6 April, 2023; v1 submitted 12 May, 2022; originally announced May 2022.

Comments: 28 pages, 4 figures

MSC Class: 60G60 (Primary) 60G17; 60H15 (Secondary)

One-Pot Printing of Robust Multimaterial Devices

Authors: Sijia Huang, Steven Adelmund, Pradip S. Pichumani, Johanna J. Schwartz, Yigit Menguc, Maxim Shusteff, Thomas J. Wallin

Abstract: Polymer 3D printing is a broad set of manufacturing methods that permit the fabrication of complex architectures, and, as a result, numerous efforts focus on formulating processible chemistries that produce desirable material behavior in printed parts. However, current resin chemistries typically result in a single fixed set of properties once fully polymerized, a fact that poses significant engin… ▽ More Polymer 3D printing is a broad set of manufacturing methods that permit the fabrication of complex architectures, and, as a result, numerous efforts focus on formulating processible chemistries that produce desirable material behavior in printed parts. However, current resin chemistries typically result in a single fixed set of properties once fully polymerized, a fact that poses significant engineering challenges to obtaining multimaterial devices. As an alternative to single-property materials, we introduce a ternary sequential reaction scheme that exhibits diverse multimaterial properties by profoundly altering the polymer microstructure from within a single resin composition. In this system, the photodosage during 3D printing sets both the shape and extent of conversion for each subsequent reaction. This different polymerization mechanisms of the subsequent stages yield disparate crosslink densities and viscoelastic properties. As a result, our materials possess Young's Moduli spanning over three orders of magnitude (400 kPa < E < 1.6 GPa) with smooth transitions between soft and stiff regions. We successfully pattern a 500x change in modulus in under a millimeter while the sequential assembly of our polymer networks ensures robust interfaces and enhances toughness by 10x compared to the single property materials. Most importantly, the final objects remain stable to UV and thermal aging, a key limitation to applications of previous multimaterial chemistries. We demonstrate the ability to 3D print intricate multimaterial architectures by fabricating a soft, wearable braille display. △ Less

Submitted 30 September, 2022; v1 submitted 20 November, 2021; originally announced November 2021.

Comments: 54 pages including supplemental information, 5 main text figures

Efficient methods for Gaussian Markov random fields under sparse linear constraints

Authors: David Bolin, Jonas Wallin

Abstract: Methods for inference and simulation of linearly constrained Gaussian Markov Random Fields (GMRF) are computationally prohibitive when the number of constraints is large. In some cases, such as for intrinsic GMRFs, they may even be unfeasible. We propose a new class of methods to overcome these challenges in the common case of sparse constraints, where one has a large number of constraints and eac… ▽ More Methods for inference and simulation of linearly constrained Gaussian Markov Random Fields (GMRF) are computationally prohibitive when the number of constraints is large. In some cases, such as for intrinsic GMRFs, they may even be unfeasible. We propose a new class of methods to overcome these challenges in the common case of sparse constraints, where one has a large number of constraints and each only involves a few elements. Our methods rely on a basis transformation into blocks of constrained versus non-constrained subspaces, and we show that the methods greatly outperform existing alternatives in terms of computational cost. By combining the proposed methods with the stochastic partial differential equation approach for Gaussian random fields, we also show how to formulate Gaussian process regression with linear constraints in a GMRF setting to reduce computational cost. This is illustrated in two applications with simulated data. △ Less

Submitted 3 June, 2021; originally announced June 2021.

MSC Class: 62M40; 60G15; 65C30 ACM Class: G.3; G.1.3

The Hessian Screening Rule

Authors: Johan Larsson, Jonas Wallin

Abstract: Predictor screening rules, which discard predictors before fitting a model, have had considerable impact on the speed with which sparse regression problems, such as the lasso, can be solved. In this paper we present a new screening rule for solving the lasso path: the Hessian Screening Rule. The rule uses second-order information from the model to provide both effective screening, particularly in… ▽ More Predictor screening rules, which discard predictors before fitting a model, have had considerable impact on the speed with which sparse regression problems, such as the lasso, can be solved. In this paper we present a new screening rule for solving the lasso path: the Hessian Screening Rule. The rule uses second-order information from the model to provide both effective screening, particularly in the case of high correlation, as well as accurate warm starts. The proposed rule outperforms all alternatives we study on simulated data sets with both low and high correlation for $\ell_1$-regularized least-squares (the lasso) and logistic regression. It also performs best in general on the real data sets that we examine. △ Less

Submitted 4 October, 2022; v1 submitted 27 April, 2021; originally announced April 2021.

Comments: 25 pages, 14 figures

MSC Class: 62J07 ACM Class: G.3; G.4

Journal ref: Advances in neural information processing systems 35 (eds. Koyejo, S. et al.) vol. 35 15823-15835 (Curran Associates, Inc., New Orleans, USA, 2022)

Nowcasting Covid-19 statistics reported withdelay: a case-study of Sweden

Authors: Adam Altmejd, Joacim Rocklöv, Jonas Wallin

Abstract: The new corona virus disease -- COVID-2019 -- is rapidly spreading through the world. The availability of unbiased timely statistics of trends in disease events are a key to effective responses. But due to reporting delays, the most recently reported numbers are frequently underestimating of the total number of infections, hospitalizations and deaths creating an illusion of a downward trend. Here… ▽ More The new corona virus disease -- COVID-2019 -- is rapidly spreading through the world. The availability of unbiased timely statistics of trends in disease events are a key to effective responses. But due to reporting delays, the most recently reported numbers are frequently underestimating of the total number of infections, hospitalizations and deaths creating an illusion of a downward trend. Here we describe a statistical methodology for predicting true daily quantities and their uncertainty, estimated using historical reporting delays. The methodology takes into account the observed distribution pattern of the lag. It is derived from the removal method, a well-established estimation framework in the field of ecology. △ Less

Submitted 11 June, 2020; originally announced June 2020.

The Strong Screening Rule for SLOPE

Authors: Johan Larsson, Małgorzata Bogdan, Jonas Wallin

Abstract: Extracting relevant features from data sets where the number of observations ($n$) is much smaller then the number of predictors ($p$) is a major challenge in modern statistics. Sorted L-One Penalized Estimation (SLOPE), a generalization of the lasso, is a promising method within this setting. Current numerical procedures for SLOPE, however, lack the efficiency that respective tools for the lasso… ▽ More Extracting relevant features from data sets where the number of observations ($n$) is much smaller then the number of predictors ($p$) is a major challenge in modern statistics. Sorted L-One Penalized Estimation (SLOPE), a generalization of the lasso, is a promising method within this setting. Current numerical procedures for SLOPE, however, lack the efficiency that respective tools for the lasso enjoy, particularly in the context of estimating a complete regularization path. A key component in the efficiency of the lasso is predictor screening rules: rules that allow predictors to be discarded before estimating the model. This is the first paper to establish such a rule for SLOPE. We develop a screening rule for SLOPE by examining its subdifferential and show that this rule is a generalization of the strong rule for the lasso. Our rule is heuristic, which means that it may discard predictors erroneously. We present conditions under which this may happen and show that such situations are rare and easily safeguarded against by a simple check of the optimality conditions. Our numerical experiments show that the rule performs well in practice, leading to improvements by orders of magnitude for data in the $p \gg n$ domain, as well as incurring no additional computational overhead when $n \gg p$. We also examine the effect of correlation structures in the design matrix on the rule and discuss algorithmic strategies for employing the rule. Finally, we provide an efficient implementation of the rule in our R package SLOPE. △ Less

Submitted 22 April, 2022; v1 submitted 7 May, 2020; originally announced May 2020.

Comments: 15 pages, 5 figures

MSC Class: 62J07 ACM Class: G.3; G.4

Journal ref: Advances in neural information processing systems 33 (eds. Larochelle, H. et al.) vol. 33 14592-14603 (Curran Associates, Inc., Virtual, 2020)

Local scale invariance and robustness of proper scoring rules

Authors: David Bolin, Jonas Wallin

Abstract: Averages of proper scoring rules are often used to rank probabilistic forecasts. In many cases, the individual terms in these averages are based on observations and forecasts from different distributions. We show that some of the most popular proper scoring rules, such as the continuous ranked probability score (CRPS), give more importance to observations with large uncertainty which can lead to u… ▽ More Averages of proper scoring rules are often used to rank probabilistic forecasts. In many cases, the individual terms in these averages are based on observations and forecasts from different distributions. We show that some of the most popular proper scoring rules, such as the continuous ranked probability score (CRPS), give more importance to observations with large uncertainty which can lead to unintuitive rankings. To describe this issue, we define the concept of local scale invariance for scoring rules. A new class of generalized proper kernel scoring rules is derived and as a member of this class we propose the scaled CRPS (SCRPS). This new proper scoring rule is locally scale invariant and therefore works in the case of varying uncertainty. Like CRPS it is computationally available for output from ensemble forecasts, and does not require the ability to evaluate densities of forecasts. We further define robustness of scoring rules, show why this also is an important concept for average scores, and derive new proper scoring rules that are robust against outliers. The theoretical findings are illustrated in three different applications from spatial statistics, stochastic volatility models, and regression for count data. △ Less

Submitted 26 March, 2022; v1 submitted 11 December, 2019; originally announced December 2019.

MSC Class: 62C99; 62A99

Robust joint modelling of longitudinal and survival data with a time-varying degrees-of-freedom parameter

Authors: Lisa McFetridge, Ozgur Asar, Jonas Wallin

Abstract: Repeated measures of biomarkers have the potential of explaining hazards of survival outcomes. In practice, these measurements are intermittently measured and are known to be subject to substantial measurement error. Joint modelling of longitudinal and survival data enables us to associate intermittently measured error-prone biomarkers with risks of survival outcomes. Most of the joint models avai… ▽ More Repeated measures of biomarkers have the potential of explaining hazards of survival outcomes. In practice, these measurements are intermittently measured and are known to be subject to substantial measurement error. Joint modelling of longitudinal and survival data enables us to associate intermittently measured error-prone biomarkers with risks of survival outcomes. Most of the joint models available in the literature have been built on the Gaussian assumption. This makes them sensitive to outliers. In this work, we study a range of robust models to address this issue. For medical data, it has been observed that outliers might occur with different frequencies over time. To address this, a new model with a time varying robustness is introduced. Through both a simulation study and analysis of two real-life data examples, this research not only stresses the need to account for longitudinal outliers in joint modelling research but also highlights the bias and inefficiency from not properly estimating the degrees-of-freedom parameter. Each technique presented in this work can be fitted using the R package robjm. △ Less

Submitted 11 December, 2019; originally announced December 2019.

Generalized bounds for active subspaces

Authors: Mario Teixeira Parente, Jonas Wallin, Barbara Wohlmuth

Abstract: In this article, we consider scenarios in which traditional estimates for the active subspace method based on probabilistic Poincaré inequalities are not valid due to unbounded Poincaré constants. Consequently, we propose a framework that allows to derive generalized estimates in the sense that it enables to control the trade-off between the size of the Poincaré constant and a weaker order of the… ▽ More In this article, we consider scenarios in which traditional estimates for the active subspace method based on probabilistic Poincaré inequalities are not valid due to unbounded Poincaré constants. Consequently, we propose a framework that allows to derive generalized estimates in the sense that it enables to control the trade-off between the size of the Poincaré constant and a weaker order of the final error bound. In particular, we investigate independently exponentially distributed random variables in dimension two or larger and give explicit expressions for corresponding Poincaré constants showing their dependence on the dimension of the problem. Finally, we suggest possibilities for future work that aim for extending the class of distributions applicable to the active subspace method as we regard this as an opportunity to enlarge its usability. △ Less

Submitted 10 February, 2020; v1 submitted 3 October, 2019; originally announced October 2019.

Comments: 27 pages, 6 figures

Journal ref: Electronic Journal of Statistics 14 (1), 917-943, 2020

Nonparametric shrinkage estimation in high dimensional generalized linear models via Polya trees

Authors: Asaf Weinstein, Jonas Wallin, Daniel Yekutieli, Małgorzata Bogdan

Abstract: In a given generalized linear model with fixed effects, and under a specified loss function, what is the optimal estimator of the coefficients? We propose as a contender an ideal (oracle) shrinkage estimator, specifically, the Bayes estimator under the particular prior that assigns equal mass to every permutation of the true coefficient vector. We first study this ideal shrinker, showing some opti… ▽ More In a given generalized linear model with fixed effects, and under a specified loss function, what is the optimal estimator of the coefficients? We propose as a contender an ideal (oracle) shrinkage estimator, specifically, the Bayes estimator under the particular prior that assigns equal mass to every permutation of the true coefficient vector. We first study this ideal shrinker, showing some optimality properties in both frequentist and Bayesian frameworks by extending notions from Robbins's compound decision theory. To compete with the ideal estimator, taking advantage of the fact that it depends on the true coefficients only through their {\it empirical distribution}, we postulate a hierarchical Bayes model, that can be viewed as a nonparametric counterpart of the usual Gaussian hierarchical model. More concretely, the individual coefficients are modeled as i.i.d.~draws from a common distribution $π$, which is itself modeled as random and assigned a Polya tree prior to reflect indefiniteness. We show in simulations that the posterior mean of $π$ approximates well the empirical distribution of the true, {\it fixed} coefficients, effectively solving a nonparametric deconvolution problem. This allows the posterior estimates of the coefficient vector to learn the correct shrinkage pattern without parametric restrictions. We compare our method with popular parametric alternatives on the challenging task of gene map** in the presence of polygenic effects. In this scenario, the regressors exhibit strong spatial correlation, and the signal consists of a dense polygenic component along with several prominent spikes. Our analysis demonstrates that, unlike standard high-dimensional methods such as ridge regression or Lasso, the proposed approach recovers the intricate signal structure, and results in better estimation and prediction accuracy in supporting simulations. △ Less

Submitted 22 August, 2023; v1 submitted 22 August, 2019; originally announced August 2019.

Bestow and Atomic: Concurrent Programming using Isolation, Delegation and Grou**

Authors: Elias Castegren, Joel Wallin, Tobias Wrigstad

Abstract: Any non-trivial concurrent system warrants synchronisation, regardless of the concurrency model. Actor-based concurrency serialises all computations in an actor through asynchronous message passing. In contrast, lock-based concurrency serialises some computations by following a lock--unlock protocol for accessing certain data. Both systems require sound reasoning about pointers and aliasing to exc… ▽ More Any non-trivial concurrent system warrants synchronisation, regardless of the concurrency model. Actor-based concurrency serialises all computations in an actor through asynchronous message passing. In contrast, lock-based concurrency serialises some computations by following a lock--unlock protocol for accessing certain data. Both systems require sound reasoning about pointers and aliasing to exclude data-races. If actor isolation is broken, so is the single-thread-of-control abstraction. Similarly for locks, if a datum is accessible outside of the scope of the lock, the datum is not governed by the lock. In this paper we discuss how to balance aliasing and synchronisation. In previous work, we defined a type system that guarantees data-race freedom of actor-based concurrency and lock-based concurrency. This paper extends this work by the introduction of two programming constructs; one for decoupling isolation and synchronisation and one for constructing higher-level atomicity guarantees from lower-level synchronisation. We focus predominantly on actors, and in particular the Encore programming language, but our ultimate goal is to define our constructs in such a way that they can be used both with locks and actors, given that combinations of both models occur frequently in actual systems. We discuss the design space, provide several formalisations of different semantics and discuss their properties, and connect them to case studies showing how our proposed constructs can be useful. We also report on an on-going implementation of our proposed constructs in Encore. △ Less

Submitted 26 July, 2018; originally announced July 2018.

Journal ref: Journal of Logical and Algebraic Methods in Programming, Vol. 100 (2018), 130-151

Infinite dimensional adaptive MCMC for Gaussian processes

Authors: Jonas Wallin, Sreekar Vadlamani

Abstract: Latent Gaussian processes are widely applied in many fields like, statistics, inverse problems and machine learning. A popular method for inference is through the posterior distribution, which is typically carried out by Markov Chain Monte Carlo (MCMC) algorithms. Most Gaussian processes can be represented as a Gaussian measure in a infinite dimensional space. This is an issue for standard algorit… ▽ More Latent Gaussian processes are widely applied in many fields like, statistics, inverse problems and machine learning. A popular method for inference is through the posterior distribution, which is typically carried out by Markov Chain Monte Carlo (MCMC) algorithms. Most Gaussian processes can be represented as a Gaussian measure in a infinite dimensional space. This is an issue for standard algorithms as they break down in an infinite dimensional setting, thus the need for appropriate infinite dimensional samplers for implementing probabilistic inference in such framework. In this paper, we introduce several adaptive versions of the preconditioned Crank-Nicolson Langevin (pCNL) algorithm, which can be viewed as an infinite dimensional version of the well known Metropolis adjusted Langevin algorithm (MALA) algorithm for Gaussian processes. The basic premise for all our proposals lies in the idea of implementing change of measure formulation to adapt the algorithms to greatly improve their efficiency. A gradient-free version of pCNL is introduced, which is a hybrid of an adaptive independence sampler and an adaptive random walk sampler, and is shown to outperform the standard preconditioned Crank-Nicolson (pCN) scheme. Finally, we demonstrate the efficiency of our proposed algorithm for three different statistical models. △ Less

Submitted 13 April, 2018; originally announced April 2018.

Linear Mixed-Effects Models for Non-Gaussian Repeated Measurement Data

Authors: Özgür Asar, David Bolin, Peter J. Diggle, Jonas Wallin

Abstract: We consider the analysis of continuous repeated measurement outcomes that are collected through time, also known as longitudinal data. A standard framework for analysing data of this kind is a linear Gaussian mixed-effects model within which the outcome variable can be decomposed into fixed-effects, time-invariant and time-varying random-effects, and measurement noise. We develop methodology that,… ▽ More We consider the analysis of continuous repeated measurement outcomes that are collected through time, also known as longitudinal data. A standard framework for analysing data of this kind is a linear Gaussian mixed-effects model within which the outcome variable can be decomposed into fixed-effects, time-invariant and time-varying random-effects, and measurement noise. We develop methodology that, for the first time, allows any combination of these stochastic components to be non-Gaussian, using multivariate Normal variance-mean mixtures. We estimate parameters by max- imum likelihood, implemented with a novel, computationally efficient stochastic gradient algorithm. We obtain standard error estimates by inverting the observed Fisher-information matrix, and obtain the predictive distributions for the random-effects in both filtering (conditioning on past and current data) and smoothing (conditioning on all data) contexts. To implement these procedures, we intro- duce an R package, ngme. We re-analyse two data-sets, from cystic fibrosis and nephrology research, that were previously analysed using Gaussian linear mixed effects models. △ Less

Submitted 7 April, 2018; originally announced April 2018.

EG Weighting Districts

Authors: Ray J Wallin

Abstract: This past decade has seen a noticeable uptick in asymmetric election results along with the inevitable claims of gerrymandering and litigation. Research, too, has followed, giving rise to intense scrutiny of elections, where the goal is to understand not only what goes into gerrymandering, but how to measure what comes out. Perhaps the most cited symmetry measure of this decade is the EG. The EG h… ▽ More This past decade has seen a noticeable uptick in asymmetric election results along with the inevitable claims of gerrymandering and litigation. Research, too, has followed, giving rise to intense scrutiny of elections, where the goal is to understand not only what goes into gerrymandering, but how to measure what comes out. Perhaps the most cited symmetry measure of this decade is the EG. The EG has been commonplace in gerrymandering litigation nationwide and the focus of numerous articles, both popular and scholarly. This article shows how the EG can be represented as a weighting function. This is not the full WDM article though it shows up in Google searches. For the full WDM article, please see https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3308888 △ Less

Submitted 24 January, 2019; v1 submitted 29 March, 2018; originally announced March 2018.

The Astrophysics Source Code Library: What's new, what's coming

Authors: Alice Allen, G. Bruce Berriman, Kimberly DuPrie, Jessica Mink, Robert Nemiroff, P. Wesley Ryan, Judy Schmidt, Lior Shamir, Keith Shortridge, Mark Taylor, Peter Teuben, John Wallin, Rein H. Warmels

Abstract: The Astrophysics Source Code Library (ASCL, ascl.net), established in 1999, is a citable online registry of source codes used in research that are available for download; the ASCL's main purpose is to improve the transparency, reproducibility, and falsifiability of research. In 2017, improvements to the resource included real-time data backup for submissions and newly-published entries, improved c… ▽ More The Astrophysics Source Code Library (ASCL, ascl.net), established in 1999, is a citable online registry of source codes used in research that are available for download; the ASCL's main purpose is to improve the transparency, reproducibility, and falsifiability of research. In 2017, improvements to the resource included real-time data backup for submissions and newly-published entries, improved cross-matching of research papers with software entries in ADS, and expansion of preferred citation information for the software in the ASCL. △ Less

Submitted 8 December, 2017; originally announced December 2017.

Comments: 4 pages; to be published in proceedings of the October 22-26 2017 ADASS XXVII meeting in Santiago, Chile

Level set Cox processes

Authors: Anders Hildeman, David Bolin, Jonas Wallin, Janine B. Illian

Abstract: The log-Gaussian Cox process (LGCP) is a popular point process for modeling non-interacting spatial point patterns. This paper extends the LGCP model to handle data exhibiting fundamentally different behaviors in different subregions of the spatial domain. The aim of the analyst might be either to identify and classify these regions, to perform kriging, or to derive some properties of the paramete… ▽ More The log-Gaussian Cox process (LGCP) is a popular point process for modeling non-interacting spatial point patterns. This paper extends the LGCP model to handle data exhibiting fundamentally different behaviors in different subregions of the spatial domain. The aim of the analyst might be either to identify and classify these regions, to perform kriging, or to derive some properties of the parameters driving the random field in one or several of the subregions. The extension is based on replacing the latent Gaussian random field in the LGCP by a latent spatial mixture model. The mixture model is specified using a latent, categorically valued, random field induced by level set operations on a Gaussian random field. Conditional on the classification, the intensity surface for each class is modeled by a set of independent Gaussian random fields. This allows for standard stationary covariance structures, such as the Matérn family, to be used to model Gaussian random fields with some degree of general smoothness but also occasional and structured sharp discontinuities. A computationally efficient MCMC method is proposed for Bayesian inference and we show consistency of finite dimensional approximations of the model. Finally, the model is fitted to point pattern data derived from a tropical rainforest on Barro Colorado island, Panama. We show that the proposed model is able to capture behavior for which inference based on the standard LGCP is biased. △ Less

Submitted 2 November, 2017; v1 submitted 23 August, 2017; originally announced August 2017.

MSC Class: 62M30

Astrophysics Source Code Library: Here we grow again!

Authors: Alice Allen, G. Bruce Berriman, Kimberly DuPrie, Jessica Mink, Robert Nemiroff, Thomas Robitaille, Judy Schmidt, Lior Shamir, Keith Shortridge, Mark Taylor, Peter Teuben, John Wallin

Abstract: The Astrophysics Source Code Library (ASCL) is a free online registry of research codes; it is indexed by ADS and Web of Science and has over 1300 code entries. Its entries are increasingly used to cite software; citations have been doubling each year since 2012 and every major astronomy journal accepts citations to the ASCL. Codes in the resource cover all aspects of astrophysics research and man… ▽ More The Astrophysics Source Code Library (ASCL) is a free online registry of research codes; it is indexed by ADS and Web of Science and has over 1300 code entries. Its entries are increasingly used to cite software; citations have been doubling each year since 2012 and every major astronomy journal accepts citations to the ASCL. Codes in the resource cover all aspects of astrophysics research and many programming languages are represented. In the past year, the ASCL added dashboards for users and administrators, started minting Digital Objective Identifiers (DOIs) for software it houses, and added metadata fields requested by users. This presentation covers the ASCL's growth in the past year and the opportunities afforded it as one of the few domain libraries for science research codes. △ Less

Submitted 18 November, 2016; originally announced November 2016.

Comments: 4 pages, 2 figures; to be published in ADASS XXVI (held October 16-20, 2016) proceedings

Estimating the unobservable moose - converting index to population size using a Bayesian Hierarchical state space model

Authors: Jonas Wallin, Kjell Wallin

Abstract: Indirect information on population size, like pellet counts or volunteer counts, is the main source of information in most ecological studies and applied population management situations. Often, such observations are treaded as if they were actual measurements of population size. This assumption results in incorrect conclusions about a population's size and its dynamics. We propose a model with a… ▽ More Indirect information on population size, like pellet counts or volunteer counts, is the main source of information in most ecological studies and applied population management situations. Often, such observations are treaded as if they were actual measurements of population size. This assumption results in incorrect conclusions about a population's size and its dynamics. We propose a model with a temporal varying link, denoted countability, between indirect observations and actual population size. We show that, when indirect measurement has high precision (for instance many observation hours) the assumption of temporal varying countability can have a crucial effect on the estimated population dynamic. We apply the model on two local moose populations in Sweden. The estimated population dynamics is found to explain 30-50 percent of the total variability in the observation data; thus, countability accounts for most of the variation. This unreliability of the estimated dynamics has a substantial negative impact on the ability to manage populations; for example, reducing (increasing) the number of animals that needs to be harvested in order to sustain the population above (below) a fixed level. Finally, large difference in countability between two study areas implies a substantial spatial variation in the countability; this variation in itself is highly worthy of study. △ Less

Submitted 25 September, 2016; v1 submitted 21 July, 2016; originally announced July 2016.

Whole-brain substitute CT generation using Markov random field mixture models

Authors: Anders Hildeman, David Bolin, Jonas Wallin, Adam Johansson, Tufve Nyholm, Thomas Asklund, Jun Yu

Abstract: Computed tomography (CT) equivalent information is needed for attenuation correction in PET imaging and for dose planning in radiotherapy. Prior work has shown that Gaussian mixture models can be used to generate a substitute CT (s-CT) image from a specific set of MRI modalities. This work introduces a more flexible class of mixture models for s-CT generation, that incorporates spatial dependency… ▽ More Computed tomography (CT) equivalent information is needed for attenuation correction in PET imaging and for dose planning in radiotherapy. Prior work has shown that Gaussian mixture models can be used to generate a substitute CT (s-CT) image from a specific set of MRI modalities. This work introduces a more flexible class of mixture models for s-CT generation, that incorporates spatial dependency in the data through a Markov random field prior on the latent field of class memberships associated with a mixture model. Furthermore, the mixture distributions are extended from Gaussian to normal inverse Gaussian (NIG), allowing heavier tails and skewness. The amount of data needed to train a model for s-CT generation is of the order of 100 million voxels. The computational efficiency of the parameter estimation and prediction methods are hence paramount, especially when spatial dependency is included in the models. A stochastic Expectation Maximization (EM) gradient algorithm is proposed in order to tackle this challenge. The advantages of the spatial model and NIG distributions are evaluated with a cross-validation study based on data from 14 patients. The study show that the proposed model enhances the predictive quality of the s-CT images by reducing the mean absolute error with 17.9%. Also, the distribution of CT values conditioned on the MR images are better explained by the proposed model as evaluated using continuous ranked probability scores. △ Less

Submitted 28 September, 2016; v1 submitted 7 July, 2016; originally announced July 2016.

Multivariate type G Matérn stochastic partial differential equation random fields

Authors: David Bolin, Jonas Wallin

Abstract: For many applications with multivariate data, random field models capturing departures from Gaussianity within realisations are appropriate. For this reason, we formulate a new class of multivariate non-Gaussian models based on systems of stochastic partial differential equations with additive type G noise whose marginal covariance functions are of Matérn type. We consider four increasingly flexib… ▽ More For many applications with multivariate data, random field models capturing departures from Gaussianity within realisations are appropriate. For this reason, we formulate a new class of multivariate non-Gaussian models based on systems of stochastic partial differential equations with additive type G noise whose marginal covariance functions are of Matérn type. We consider four increasingly flexible constructions of the noise, where the first two are similar to existing copula-based models. In contrast to these, the latter two constructions can model non-Gaussian spatial data without replicates. Computationally efficient methods for likelihood-based parameter estimation and probabilistic prediction are proposed, and the flexibility of the suggested models is illustrated by numerical examples and two statistical applications. △ Less

Submitted 31 December, 2019; v1 submitted 27 June, 2016; originally announced June 2016.

Journal ref: J. Royal Stat. Soc. Series B (2019)

Galaxy Zoo: Mergers - Dynamical Models of Interacting Galaxies

Authors: Anthony J. Holincheck, John F. Wallin, Kirk Borne, Lucy Fortson, Chris Lintott, Arfon M. Smith, Steven Bamford, William C. Keel, Michael Parrish

Abstract: The dynamical history of most merging galaxies is not well understood. Correlations between galaxy interaction and star formation have been found in previous studies, but require the context of the physical history of merging systems for full insight into the processes that lead to enhanced star formation. We present the results of simulations that reconstruct the orbit trajectories and disturbed… ▽ More The dynamical history of most merging galaxies is not well understood. Correlations between galaxy interaction and star formation have been found in previous studies, but require the context of the physical history of merging systems for full insight into the processes that lead to enhanced star formation. We present the results of simulations that reconstruct the orbit trajectories and disturbed morphologies of pairs of interacting galaxies. With the use of a restricted three-body simulation code and the help of Citizen Scientists, we sample 10^5 points in parameter space for each system. We demonstrate a successful recreation of the morphologies of 62 pairs of interacting galaxies through the review of more than 3 million simulations. We examine the level of convergence and uniqueness of the dynamical properties of each system. These simulations represent the largest collection of models of interacting galaxies to date, providing a valuable resource for the investigation of mergers. This paper presents the simulation parameters generated by the project. They are now publicly available in electronic format at http://data.galaxyzoo.org/mergers.html. Though our best-fit model parameters are not an exact match to previously published models, our method for determining uncertainty measurements will aid future comparisons between models. The dynamical clocks from our models agree with previous results of the time since the onset of star formation from star burst models in interacting systems and suggests that tidally induced star formation is triggered very soon after closest approach. △ Less

Submitted 1 April, 2016; originally announced April 2016.

Comments: 29 pages, 17 figures, 7 tables, Accepted for publication in MNRAS

Online estimation of driving events and fatigue damage on vehicles

Authors: Roza Maghsood, Jonas Wallin

Abstract: Driving events, such as maneuvers at slow speed and turns, are important for durability assessments of vehicle components. By counting the number of driving events, one can estimate the fatigue damage caused by the same kind of events. Through knowledge of the distribution of driving events for a group of customers, the vehicles producers can tailor the design, of vehicles, for the group. In this… ▽ More Driving events, such as maneuvers at slow speed and turns, are important for durability assessments of vehicle components. By counting the number of driving events, one can estimate the fatigue damage caused by the same kind of events. Through knowledge of the distribution of driving events for a group of customers, the vehicles producers can tailor the design, of vehicles, for the group. In this article, we propose an algorithm that can be applied on-board a vehicle to online estimate the expected number of driving events occurring, and thus be used to estimate the distribution of driving events for a certain group of customers. Since the driving events are not observed directly, the algorithm uses a hidden Markov model to extract the events. The parameters of the HMM are estimated using an online EM algorithm. The introduction of the online EM is crucial for practical usage, on-board vehicles, due to that its complexity of an iteration is fixed. Typically, the EM algorithm is used to find the, fixed, parameters that maximizes the likelihood. By introducing a fixed forgetting factor in the online EM, an adaptive algorithm is acquired. This is important in practice since the driving conditions changes over time and a single trip can contain different road types such as city and highway, making the assumption of fixed parameters unrealistic. Finally, we also derive a method to online compute the expected damage. △ Less

Submitted 21 March, 2016; originally announced March 2016.

Comments: 23 pages, 8 figures

Improving Software Citation and Credit

Authors: Alice Allen, G. Bruce Berriman, Kimberly DuPrie, Jessica Mink, Robert Nemiroff, Thomas Robitaille, Lior Shamir, Keith Shortridge, Mark Taylor, Peter Teuben, John Wallin

Abstract: The past year has seen movement on several fronts for improving software citation, including the Center for Open Science's Transparency and Openness Promotion (TOP) Guidelines, the Software Publishing Special Interest Group that was started at January's AAS meeting in Seattle at the request of that organization's Working Group on Astronomical Software, a Sloan-sponsored meeting at GitHub in San Fr… ▽ More The past year has seen movement on several fronts for improving software citation, including the Center for Open Science's Transparency and Openness Promotion (TOP) Guidelines, the Software Publishing Special Interest Group that was started at January's AAS meeting in Seattle at the request of that organization's Working Group on Astronomical Software, a Sloan-sponsored meeting at GitHub in San Francisco to begin work on a cohesive research software citation-enabling platform, the work of Force11 to "transform and improve" research communication, and WSSSPE's ongoing efforts that include software publication, citation, credit, and sustainability. Brief reports on these efforts were shared at the BoF, after which participants discussed ideas for improving software citation, generating a list of recommendations to the community of software authors, journal publishers, ADS, and research authors. The discussion, recommendations, and feedback will help form recommendations for software citation to those publishers represented in the Software Publishing Special Interest Group and the broader community. △ Less

Submitted 24 December, 2015; originally announced December 2015.

Comments: Birds of a Feather session organized by the Astrophysics Source Code Library (ASCL, http://ascl.net/ ); to be published in Proceedings of ADASS XXV (Sydney, Australia; October, 2015). 4 pages

JSPAM: A restricted three-body code for simulating interacting galaxies

Authors: John Wallin, Anthony Holincheck, Allen Harvey

Abstract: Restricted three-body codes have a proven ability to recreate much of the disturbed morphology of actual interacting galaxies. As more sophisticated n-body models were developed and computer speed increased, restricted three-body codes fell out of favor. However, their supporting role for performing wide searches of parameter space when fitting orbits to real systems demonstrates a continuing need… ▽ More Restricted three-body codes have a proven ability to recreate much of the disturbed morphology of actual interacting galaxies. As more sophisticated n-body models were developed and computer speed increased, restricted three-body codes fell out of favor. However, their supporting role for performing wide searches of parameter space when fitting orbits to real systems demonstrates a continuing need for their use. Here we present the model and algorithm used in the JSPAM code. A precursor of this code was originally described in 1990, and was called SPAM. We have recently updated the software with an alternate potential and a treatment of dynamical friction to more closely mimic the results from n-body tree codes. The code is released publicly for use under the terms of the Academic Free License (AFL) v.3.0 and has been added to the Astrophysics Source Code Library. △ Less

Submitted 13 April, 2016; v1 submitted 16 November, 2015; originally announced November 2015.

Comments: 23 pages, 7 figures, fixed typos and updated for accepted version

Spatially adaptive covariance tapering

Authors: David Bolin, Jonas Wallin

Abstract: Covariance tapering is a popular approach for reducing the computational cost of spatial prediction and parameter estimation for Gaussian process models. However, tapering can have poor performance when the process is sampled at spatially irregular locations or when non-stationary covariance models are used. This work introduces an adaptive tapering method in order to improve the performance of ta… ▽ More Covariance tapering is a popular approach for reducing the computational cost of spatial prediction and parameter estimation for Gaussian process models. However, tapering can have poor performance when the process is sampled at spatially irregular locations or when non-stationary covariance models are used. This work introduces an adaptive tapering method in order to improve the performance of tapering in these problematic cases. This is achieved by introducing a computationally convenient class of compactly supported non-stationary covariance functions, combined with a new method for choosing spatially varying taper ranges. Numerical experiments are used to show that the performance of both kriging prediction and parameter estimation can be improved by allowing for spatially varying taper ranges. However, although adaptive tapering outperforms regular tapering, simply dividing the data into blocks and ignoring the dependence between the blocks is often a better method for parameter estimation. △ Less

Submitted 19 February, 2016; v1 submitted 11 June, 2015; originally announced June 2015.

Efficient adaptive MCMC through precision estimation

Authors: Jonas Wallin, David Bolin

Abstract: A novel adaptive Markov chain Monte Carlo algorithm is presented. The algorithm utilizes sparsity in the partial correlation structure of a density to efficiently estimate the covariance matrix through the Cholesky factor of the precision matrix. The algorithm also utilizes the sparsity to sample efficiently from both MALA and Metropolis Hasting random walk proposals. Further, an algorithm that es… ▽ More A novel adaptive Markov chain Monte Carlo algorithm is presented. The algorithm utilizes sparsity in the partial correlation structure of a density to efficiently estimate the covariance matrix through the Cholesky factor of the precision matrix. The algorithm also utilizes the sparsity to sample efficiently from both MALA and Metropolis Hasting random walk proposals. Further, an algorithm that estimates the partial correlation structure of a density is proposed. Combining this with the Cholesky factor estimation algorithm results in an efficient black-box AMCMC method that can be used for general densities with unknown dependency structure. The method is compared with regular empirical covariance adaption for two examples. In both examples, the proposed method's covariance estimates converge faster to the true covariance matrix and the computational cost for each iteration is lower. △ Less

Submitted 6 February, 2016; v1 submitted 14 May, 2015; originally announced May 2015.

Latent modeling of flow cytometry cell populations

Authors: Jonas Wallin, Kerstin Johnsson, Magnus Fontes

Abstract: Flow cytometry is a widespread single-cell measurement technology with a multitude of clinical and research applications. Interpretation of flow cytometry data is hard; the instrumentation is delicate and can not render absolute measurements, hence samples can only be interpreted in relation to each other while at the same time comparisons are confounded by inter-sample variation. Despite this, cu… ▽ More Flow cytometry is a widespread single-cell measurement technology with a multitude of clinical and research applications. Interpretation of flow cytometry data is hard; the instrumentation is delicate and can not render absolute measurements, hence samples can only be interpreted in relation to each other while at the same time comparisons are confounded by inter-sample variation. Despite this, current automated flow cytometry data analysis methods either treat samples individually or ignore the variation by for example pooling the data. In this article we introduce a Bayesian hierarchical model for studying latent relations between cell populations in flow cytometry samples, thereby systematizing inter-sample variation. The model is applied to a data set containing replicated flow cytometry measurements of samples from healthy individuals, with informative priors capturing expert knowledge. It is shown that the technical variation in the inferred cell population sizes is small in comparison to the intrinsic biological variation. The large size of flow cytometry data, where a single sample can contain measurements on hundreds of thousands of cells, necessitates computationally efficient methods. To address this, we have implemented a parallel Markov Chain Monte Carlo scheme for sampling the posterior distribution. △ Less

Submitted 13 February, 2015; originally announced February 2015.

Comments: Supplemental material provided

MSC Class: 62P10

Astrophysics Source Code Library Enhancements

Authors: Robert J. Hanisch, Alice Allen, G. Bruce Berriman, Kimberly DuPrie, Jessica Mink, Robert J. Nemiroff, Judy Schmidt, Lior Shamir, Keith Shortridge, Mark Taylor, Peter J. Teuben, John Wallin

Abstract: The Astrophysics Source Code Library (ASCL; ascl.net) is a free online registry of codes used in astronomy research; it currently contains over 900 codes and is indexed by ADS. The ASCL has recently moved a new infrastructure into production. The new site provides a true database for the code entries and integrates the WordPress news and information pages and the discussion forum into one site. Pr… ▽ More The Astrophysics Source Code Library (ASCL; ascl.net) is a free online registry of codes used in astronomy research; it currently contains over 900 codes and is indexed by ADS. The ASCL has recently moved a new infrastructure into production. The new site provides a true database for the code entries and integrates the WordPress news and information pages and the discussion forum into one site. Previous capabilities are retained and permalinks to ascl.net continue to work. This improvement offers more functionality and flexibility than the previous site, is easier to maintain, and offers new possibilities for collaboration. This presentation covers these recent changes to the ASCL. △ Less

Submitted 7 November, 2014; originally announced November 2014.

Comments: 4 pages; to be published in ADASS XXIV Proceedings. ASCL can be accessed at http://ascl.net/

Combining human and machine learning for morphological analysis of galaxy images

Authors: Evan Kuminski, Joe George, John Wallin, Lior Shamir

Abstract: The increasing importance of digital sky surveys collecting many millions of galaxy images has reinforced the need for robust methods that can perform morphological analysis of large galaxy image databases. Citizen science initiatives such as Galaxy Zoo showed that large datasets of galaxy images can be analyzed effectively by non-scientist volunteers, but since databases generated by robotic tele… ▽ More The increasing importance of digital sky surveys collecting many millions of galaxy images has reinforced the need for robust methods that can perform morphological analysis of large galaxy image databases. Citizen science initiatives such as Galaxy Zoo showed that large datasets of galaxy images can be analyzed effectively by non-scientist volunteers, but since databases generated by robotic telescopes grow much faster than the processing power of any group of citizen scientists, it is clear that computer analysis is required. Here we propose to use citizen science data for training machine learning systems, and show experimental results demonstrating that machine learning systems can be trained with citizen science data. Our findings show that the performance of machine learning depends on the quality of the data, which can be improved by using samples that have a high degree of agreement between the citizen scientists. The source code of the method is publicly available. △ Less

Submitted 28 September, 2014; originally announced September 2014.

Comments: PASP, accepted

Automatic detection of peculiar galaxy pairs in Sloan Digital Sky Survey

Authors: Lior Shamir, John Wallin

Abstract: We applied computational tools for automatic detection of peculiar galaxy pairs. We first detected in SDSS DR7 ~400,000 galaxy images with i magnitude <18 that had more than one point spread function, and then applied a machine learning algorithm that detected ~26,000 galaxy images that had morphology similar to the morphology of galaxy mergers. That dataset was mined using a novelty detection alg… ▽ More We applied computational tools for automatic detection of peculiar galaxy pairs. We first detected in SDSS DR7 ~400,000 galaxy images with i magnitude <18 that had more than one point spread function, and then applied a machine learning algorithm that detected ~26,000 galaxy images that had morphology similar to the morphology of galaxy mergers. That dataset was mined using a novelty detection algorithm, producing a short list of 500 most peculiar galaxies as quantitatively determined by the algorithm. Manual examination of these galaxies showed that while most of the galaxy pairs in the list were not necessarily peculiar, numerous unusual galaxy pairs were detected. In this paper we describe the protocol and computational tools used for the detection of peculiar mergers, and provide examples of peculiar galaxy pairs that were detected. △ Less

Submitted 18 July, 2014; originally announced July 2014.

Comments: MNRAS, accepted

Ideas for Advancing Code Sharing (A Different Kind of Hack Day)

Authors: Peter Teuben, Alice Allen, Bruce Berriman, Kimberly DuPrie, Robert J. Hanisch, Jessica Mink, Robert Nemiroff, Lior Shamir, Keith Shortridge, Mark Taylor, John Wallin

Abstract: How do we as a community encourage the reuse of software for telescope operations, data processing, and calibration? How can we support making codes used in research available for others to examine? Continuing the discussion from last year Bring out your codes! BoF session, participants separated into groups to brainstorm ideas to mitigate factors which inhibit code sharing and nurture those which… ▽ More How do we as a community encourage the reuse of software for telescope operations, data processing, and calibration? How can we support making codes used in research available for others to examine? Continuing the discussion from last year Bring out your codes! BoF session, participants separated into groups to brainstorm ideas to mitigate factors which inhibit code sharing and nurture those which encourage code sharing. The BoF concluded with the sharing of ideas that arose from the brainstorming sessions and a brief summary by the moderator. △ Less

Submitted 27 December, 2013; originally announced December 2013.

Comments: To be published in Proceedings of ADASS XXIII. Links to notes from brainstorming sessions are available here: http://asterisk.apod.com/wp/?p=543

Astrophysics Source Code Library: Incite to Cite!

Authors: Kimberly DuPrie, Alice Allen, Bruce Berriman, Robert J. Hanisch, Jessica Mink, Robert J. Nemiroff, Lior Shamir, Keith Shortridge, Mark B. Taylor, Peter Teuben, John F. Wallin

Abstract: The Astrophysics Source Code Library (ASCL, http://ascl.net/) is an online registry of over 700 source codes that are of interest to astrophysicists, with more being added regularly. The ASCL actively seeks out codes as well as accepting submissions from the code authors, and all entries are citable and indexed by ADS. All codes have been used to generate results published in or submitted to a ref… ▽ More The Astrophysics Source Code Library (ASCL, http://ascl.net/) is an online registry of over 700 source codes that are of interest to astrophysicists, with more being added regularly. The ASCL actively seeks out codes as well as accepting submissions from the code authors, and all entries are citable and indexed by ADS. All codes have been used to generate results published in or submitted to a refereed journal and are available either via a download site or froman identified source. In addition to being the largest directory of scientist-written astrophysics programs available, the ASCL is also an active participant in the reproducible research movement with presentations at various conferences, numerous blog posts and a journal article. This poster provides a description of the ASCL and the changes that we are starting to see in the astrophysics community as a result of the work we are doing. △ Less

Submitted 23 December, 2013; originally announced December 2013.

Comments: Four pages, two figures. To be published in the Proceedings of ADASS XXIII, which took place September 29-October 3, 2013. The Astrophysics Source Code Library can be accessed at http://www.ascl.net; a concise directory of codes can be accessed at http://asterisk.apod.com/wp/?page_id=12

The Astrophysics Source Code Library: Where do we go from here?

Authors: Alice Allen, Bruce Berriman, Kimberly DuPrie, Robert J. Hanisch, Jessica Mink, Robert Nemiroff, Lior Shamir, Keith Shortridge, Mark Taylor, Peter Teuben, John Wallin

Abstract: The Astrophysics Source Code Library, started in 1999, has in the past three years grown from a repository for 40 codes to a registry of over 700 codes that are now indexed by ADS. What comes next? We examine the future of the ASCL, the challenges facing it, the rationale behind its practices, and the need to balance what we might do with what we have the resources to accomplish. The Astrophysics Source Code Library, started in 1999, has in the past three years grown from a repository for 40 codes to a registry of over 700 codes that are now indexed by ADS. What comes next? We examine the future of the ASCL, the challenges facing it, the rationale behind its practices, and the need to balance what we might do with what we have the resources to accomplish. △ Less

Submitted 18 December, 2013; originally announced December 2013.

Comments: To be published in ADASS XXIII Proceedings. ASCL can be accessed at http://www.ascl.net

Automatic quantitative morphological analysis of interacting galaxies

Authors: Lior Shamir, Anthony Holincheck, John Wallin

Abstract: The large number of galaxies imaged by digital sky surveys reinforces the need for computational methods for analyzing galaxy morphology. While the morphology of most galaxies can be associated with a stage on the Hubble sequence, morphology of galaxy mergers is far more complex due to the combination of two or more galaxies with different morphologies and the interaction between them. Here we pro… ▽ More The large number of galaxies imaged by digital sky surveys reinforces the need for computational methods for analyzing galaxy morphology. While the morphology of most galaxies can be associated with a stage on the Hubble sequence, morphology of galaxy mergers is far more complex due to the combination of two or more galaxies with different morphologies and the interaction between them. Here we propose a computational method based on unsupervised machine learning that can quantitatively analyze morphologies of galaxy mergers and associate galaxies by their morphology. The method works by first generating multiple synthetic galaxy models for each galaxy merger, and then extracting a large set of numerical image content descriptors for each galaxy model. These numbers are weighted using Fisher discriminant scores, and then the similarities between the galaxy mergers are deduced using a variation of Weighted Nearest Neighbor analysis such that the Fisher scores are used as weights. The similarities between the galaxy mergers are visualized using phylogenies to provide a graph that reflects the morphological similarities between the different galaxy mergers, and thus quantitatively profile the morphology of galaxy mergers. △ Less

Submitted 16 September, 2013; originally announced September 2013.

Comments: Astronomy & Computing, accepted

Non-Gaussian Matérn fields with an application to precipitation modeling

Authors: David Bolin, Jonas Wallin

Abstract: The recently proposed non-Gaussian Matérn random field models, generated through Stochastic Partial differential equations (SPDEs), are extended by considering the class of Generalized Hyperbolic processes as noise forcings. The models are also extended to the standard geostatistical setting where irregularly spaced observations are modeled using measurement errors and covariates. A maximum likeli… ▽ More The recently proposed non-Gaussian Matérn random field models, generated through Stochastic Partial differential equations (SPDEs), are extended by considering the class of Generalized Hyperbolic processes as noise forcings. The models are also extended to the standard geostatistical setting where irregularly spaced observations are modeled using measurement errors and covariates. A maximum likelihood estimation technique based on the Monte Carlo Expectation Maximization (MCEM) algorithm is presented, and it is shown how the model can be used to do predictions at unobserved locations. Finally, an application to precipitation data over the United States for two month in 1997 is presented, and the performance of the non-Gaussian models is compared with standard Gaussian and transformed Gaussian models through cross-validation. △ Less

Submitted 24 July, 2013; originally announced July 2013.

Practices in source code sharing in astrophysics

Authors: Lior Shamir, John F. Wallin, Alice Allen, Bruce Berriman, Peter Teuben, Robert J. Nemiroff, Jessica Mink, Robert J. Hanisch, Kimberly DuPrie

Abstract: While software and algorithms have become increasingly important in astronomy, the majority of authors who publish computational astronomy research do not share the source code they develop, making it difficult to replicate and reuse the work. In this paper we discuss the importance of sharing scientific source code with the entire astrophysics community, and propose that journals require authors… ▽ More While software and algorithms have become increasingly important in astronomy, the majority of authors who publish computational astronomy research do not share the source code they develop, making it difficult to replicate and reuse the work. In this paper we discuss the importance of sharing scientific source code with the entire astrophysics community, and propose that journals require authors to make their code publicly available when a paper is published. That is, we suggest that a paper that involves a computer program not be accepted for publication unless the source code becomes publicly available. The adoption of such a policy by editors, editorial boards, and reviewers will improve the ability to replicate scientific results, and will also make the computational astronomy methods more available to other researchers who wish to apply them to their data. △ Less

Submitted 24 April, 2013; originally announced April 2013.

Comments: Accepted by Astronomy and Computing. 10 pages

Galaxy Zoo: Morphological Classification and Citizen Science

Authors: Lucy Fortson, Karen Masters, Robert Nichol, Kirk Borne, Edd Edmondson, Chris Lintott, Jordan Raddick, Kevin Schawinski, John Wallin

Abstract: We provide a brief overview of the Galaxy Zoo and Zooniverse projects, including a short discussion of the history of, and motivation for, these projects as well as reviewing the science these innovative internet-based citizen science projects have produced so far. We briefly describe the method of applying en-masse human pattern recognition capabilities to complex data in data-intensive research.… ▽ More We provide a brief overview of the Galaxy Zoo and Zooniverse projects, including a short discussion of the history of, and motivation for, these projects as well as reviewing the science these innovative internet-based citizen science projects have produced so far. We briefly describe the method of applying en-masse human pattern recognition capabilities to complex data in data-intensive research. We also provide a discussion of the lessons learned from develo** and running these community--based projects including thoughts on future applications of this methodology. This review is intended to give the reader a quick and simple introduction to the Zooniverse. △ Less

Submitted 28 April, 2011; originally announced April 2011.

Comments: 11 pages, 1 figure; to be published in Advances in Machine Learning and Data Mining for Astronomy

The Revolution in Astronomy Education: Data Science for the Masses

Authors: Kirk D. Borne, Suzanne Jacoby, K. Carney, A. Connolly, T. Eastman, M. J. Raddick, J. A. Tyson, J. Wallin

Abstract: As our capacity to study ever-expanding domains of our science has increased (including the time domain, non-electromagnetic phenomena, magnetized plasmas, and numerous sky surveys in multiple wavebands with broad spatial coverage and unprecedented depths), so have the horizons of our understanding of the Universe been similarly expanding. This expansion is coupled to the exponential data deluge… ▽ More As our capacity to study ever-expanding domains of our science has increased (including the time domain, non-electromagnetic phenomena, magnetized plasmas, and numerous sky surveys in multiple wavebands with broad spatial coverage and unprecedented depths), so have the horizons of our understanding of the Universe been similarly expanding. This expansion is coupled to the exponential data deluge from multiple sky surveys, which have grown from gigabytes into terabytes during the past decade, and will grow from terabytes into Petabytes (even hundreds of Petabytes) in the next decade. With this increased vastness of information, there is a growing gap between our awareness of that information and our understanding of it. Training the next generation in the fine art of deriving intelligent understanding from data is needed for the success of sciences, communities, projects, agencies, businesses, and economies. This is true for both specialists (scientists) and non-specialists (everyone else: the public, educators and students, workforce). Specialists must learn and apply new data science research techniques in order to advance our understanding of the Universe. Non-specialists require information literacy skills as productive members of the 21st century workforce, integrating foundational skills for lifelong learning in a world increasingly dominated by data. We address the impact of the emerging discipline of data science on astronomy education within two contexts: formal education and lifelong learners. △ Less

Submitted 21 September, 2009; originally announced September 2009.

Comments: 12 pages total: 1 cover page, 1 page of co-signers, plus 10 pages, State of the Profession Position Paper submitted to the Astro2010 Decadal Survey (March 2009)

How Well Do We Know the Orbits of the Outer Planets?

Authors: Gary L. Page, John F. Wallin, David S. Dixon

Abstract: This paper deals with the problem of astrometric determination of the orbital elements of the outer planets, in particular by assessing the ability of astrometric observations to detect perturbations of the sort expected from the Pioneer effect or other small perturbations to gravity. We also show that while using simplified models of the dynamics can lead to some insights, one must be careful t… ▽ More This paper deals with the problem of astrometric determination of the orbital elements of the outer planets, in particular by assessing the ability of astrometric observations to detect perturbations of the sort expected from the Pioneer effect or other small perturbations to gravity. We also show that while using simplified models of the dynamics can lead to some insights, one must be careful to not over-simplify the issues involved lest one be misled by the analysis onto false paths. Specifically, we show that the current ephemeris of Pluto does not preclude the existence of the Pioneer effect. We show that the orbit of Pluto is simply not well enough characterized at present to make such an assertion. A number of misunderstandings related to these topics have now propagated through the literature and have been used as a basis for drawing conclusions about the dynamics of the solar system. Thus, the objective of this paper is to address these issues. Finally, we offer some comments dealing with the complex topic of model selection and comparison. △ Less

Submitted 30 April, 2009; originally announced May 2009.

Comments: Accepted for publication in the ApJ

Journal ref: Astrophys.J.697:1226-1241,2009

A sampling inequality for fractional order Sobolev semi-norms using arbitrary order data

Authors: Andrew Corrigan, John Wallin, Thomas Wanner

Abstract: To improve convergence results obtained using a framework for unsymmetric meshless methods due to Schaback (Preprint Göttingen 2006), we extend, in two directions, the Sobolev bound due to Arcangéli et al. (Numer Math 107, 181-211, 2007), which itself extends two others due to Wendland and Rieger (Numer Math 101, 643-662, 2005) and Madych (J. Approx Theory 142, 116-128, 2006). The first is to in… ▽ More To improve convergence results obtained using a framework for unsymmetric meshless methods due to Schaback (Preprint Göttingen 2006), we extend, in two directions, the Sobolev bound due to Arcangéli et al. (Numer Math 107, 181-211, 2007), which itself extends two others due to Wendland and Rieger (Numer Math 101, 643-662, 2005) and Madych (J. Approx Theory 142, 116-128, 2006). The first is to incorporate discrete samples of arbitrary order derivatives into the bound, which are used to obtain higher order convergence in higher order Sobolev norms. The second is to optimally bound fractional order Sobolev semi-norms, which are used to obtain more optimal convergence rates when solving problems requiring fractional order Sobolev spaces, notably inhomogeneous boundary value problems. △ Less

Submitted 13 May, 2009; v1 submitted 26 January, 2008; originally announced January 2008.

Comments: v3. fixes typos, notation, wording. v2. 14 pages, major conceptual changes

MSC Class: 65N12; 65N15; 65N22; 65N35;

Testing Gravity in the Outer Solar System: Results from Trans-Neptunian Objects

Authors: John F. Wallin, David S. Dixon, Gary L. Page

Abstract: The inverse square law of gravity is poorly probed by experimental tests at distances of ~ 10 AUs. Recent analysis of the trajectory of the Pioneer 10 and 11 spacecraft have shown an unmodeled acceleration directed toward the Sun which was not explained by any obvious spacecraft systematics, and occurred when at distances greater than 20 AUs from the Sun. If this acceleration represents a depart… ▽ More The inverse square law of gravity is poorly probed by experimental tests at distances of ~ 10 AUs. Recent analysis of the trajectory of the Pioneer 10 and 11 spacecraft have shown an unmodeled acceleration directed toward the Sun which was not explained by any obvious spacecraft systematics, and occurred when at distances greater than 20 AUs from the Sun. If this acceleration represents a departure from Newtonian gravity or is indicative of an additional mass distribution in the outer solar system, it should be detectable in the orbits of Trans-Neptunian Objects (TNOs). To place limits on deviations from Newtonian gravity, we have selected a well observed sample of TNOs found orbiting between 20 and 100 AU from the Sun. By examining their orbits with modified orbital fitting software, we place tight limits on the perturbations of gravity that could exist in this region of the solar system. △ Less

Submitted 23 May, 2007; originally announced May 2007.

Comments: 20 pages, 4 figures, 2 tables, uses AASTex v5.x macros

Journal ref: Astrophys.J.666:1296-1302,2007

Can Minor Planets be Used to Assess Gravity in the Outer Solar System?

Authors: Gary L. Page, David S. Dixon, John F. Wallin

Abstract: The twin Pioneer spacecraft have been tracked for over thirty years as they headed out of the solar system. After passing 20 AU from the Sun, both exhibited a systematic error in their trajectories that can be interpreted as a constant acceleration towards the Sun. This Pioneer Effect is most likely explained by spacecraft systematics, but there have been no convincing arguments that that is the… ▽ More The twin Pioneer spacecraft have been tracked for over thirty years as they headed out of the solar system. After passing 20 AU from the Sun, both exhibited a systematic error in their trajectories that can be interpreted as a constant acceleration towards the Sun. This Pioneer Effect is most likely explained by spacecraft systematics, but there have been no convincing arguments that that is the case. The alternative is that the Pioneer Effect represents a real phenomenon and perhaps new physics. What is lacking is a means of measuring the effect, its variation, its potential anisotropies, and its region of influence. We show that minor planets provide an observational vehicle for investigating the gravitational field in the outer solar system, and that a sustained observation campaign against properly chosen minor planets could confirm or refute the existence of the Pioneer Effect. Additionally, even if the Pioneer Effect does not represent a new physical phenomenon, minor planets can be used to probe the gravitational field in the outer Solar System and since there are very few intermediate range tests of gravity at the multiple AU distance scale, this is a worthwhile endeavor in its own right. △ Less

Submitted 2 January, 2006; v1 submitted 17 April, 2005; originally announced April 2005.

Comments: Accepted for publication in The Astrophysical Journal