Showing 1–50 of 297 results for author: Hogg, D W

NotPlaNET: Removing False Positives from Planet Hunters TESS with Machine Learning

Authors: Valentina Tardugno Poleo, Nora Eisner, David W. Hogg

Abstract: Differentiating between real transit events and false positive signals in photometric time series data is a bottleneck in the identification of transiting exoplanets, particularly long-period planets. This differentiation typically requires visual inspection of a large number of transit-like signals to rule out instrumental and astrophysical false positives that mimic planetary transit signals. We… ▽ More Differentiating between real transit events and false positive signals in photometric time series data is a bottleneck in the identification of transiting exoplanets, particularly long-period planets. This differentiation typically requires visual inspection of a large number of transit-like signals to rule out instrumental and astrophysical false positives that mimic planetary transit signals. We build a one-dimensional convolutional neural network (CNN) to separate eclipsing binaries and other false positives from potential planet candidates, reducing the number of light curves that require human vetting. Our CNN is trained using the TESS light curves that were identified by Planet Hunters citizen scientists as likely containing a transit. We also include the background flux and centroid information. The light curves are visually inspected and labeled by project scientists and are minimally pre-processed, with only normalization and data augmentation taking place before training. The median percentage of contaminants flagged across the test sectors is 18% with a maximum of 37% and a minimum of 10%. Our model keeps 100% of the planets for 16 of the 18 test sectors, while incorrectly flagging one planet candidate (0.3%) for one sector and two (0.6%) for the remaining sector. Our method shows potential to reduce the number of light curves requiring manual vetting by up to a third with minimal misclassification of planet candidates. △ Less

Submitted 28 May, 2024; originally announced May 2024.

Comments: Under review at The Astronomical Journal

Is machine learning good or bad for the natural sciences?

Authors: David W. Hogg, Soledad Villar

Abstract: Machine learning (ML) methods are having a huge impact across all of the sciences. However, ML has a strong ontology - in which only the data exist - and a strong epistemology - in which a model is considered good if it performs well on held-out training data. These philosophies are in strong conflict with both standard practices and key philosophies in the natural sciences. Here we identify some… ▽ More Machine learning (ML) methods are having a huge impact across all of the sciences. However, ML has a strong ontology - in which only the data exist - and a strong epistemology - in which a model is considered good if it performs well on held-out training data. These philosophies are in strong conflict with both standard practices and key philosophies in the natural sciences. Here we identify some locations for ML in the natural sciences at which the ontology and epistemology are valuable. For example, when an expressive machine learning model is used in a causal inference to represent the effects of confounders, such as foregrounds, backgrounds, or instrument calibration parameters, the model capacity and loose philosophy of ML can make the results more trustworthy. We also show that there are contexts in which the introduction of ML introduces strong, unwanted statistical biases. For one, when ML models are used to emulate physical (or first-principles) simulations, they amplify confirmation biases. For another, when expressive regressions are used to label datasets, those labels cannot be used in downstream joint or ensemble analyses without taking on uncontrolled biases. The question in the title is being asked of all of the natural sciences; that is, we are calling on the scientific communities to take a step back and consider the role and value of ML in their fields; the (partial) answers we give here come from the particular perspective of physics. △ Less

Submitted 31 May, 2024; v1 submitted 28 May, 2024; originally announced May 2024.

Comments: A Position Paper accepted for publication in the 2024 International Conference on Machine Learning (ICML)

Planet Hunters TESS V: a planetary system around a binary star, including a mini-Neptune in the habitable zone

Authors: Nora L. Eisner, Samuel K. Grunblatt, Oscar Barragán, Thea H. Faridani, Chris Lintott, Suzanne Aigrain, Cole Johnston, Ian R. Mason, Keivan G. Stassun, Megan Bedell, Andrew W. Boyle, David R. Ciardi, Catherine A. Clark, Guillaume Hebrard, David W. Hogg, Steve B. Howell, Baptiste Klein, Joe Llama, Joshua N. Winn, Lily L. Zhao, Joseph M. Akana Murphy, Corey Beard, Casey L. Brinkman, Ashley Chontos, Pia Cortes-Zuleta , et al. (39 additional authors not shown)

Abstract: We report on the discovery and validation of a transiting long-period mini-Neptune orbiting a bright (V = 9.0 mag) G dwarf (TOI 4633; R = 1.05 RSun, M = 1.10 MSun). The planet was identified in data from the Transiting Exoplanet Survey Satellite by citizen scientists taking part in the Planet Hunters TESS project. Modeling of the transit events yields an orbital period of 271.9445 +/- 0.0040 days… ▽ More We report on the discovery and validation of a transiting long-period mini-Neptune orbiting a bright (V = 9.0 mag) G dwarf (TOI 4633; R = 1.05 RSun, M = 1.10 MSun). The planet was identified in data from the Transiting Exoplanet Survey Satellite by citizen scientists taking part in the Planet Hunters TESS project. Modeling of the transit events yields an orbital period of 271.9445 +/- 0.0040 days and radius of 3.2 +/- 0.20 REarth. The Earth-like orbital period and an incident flux of 1.56 +/- 0.2 places it in the optimistic habitable zone around the star. Doppler spectroscopy of the system allowed us to place an upper mass limit on the transiting planet and revealed a non-transiting planet candidate in the system with a period of 34.15 +/- 0.15 days. Furthermore, the combination of archival data dating back to 1905 with new high angular resolution imaging revealed a stellar companion orbiting the primary star with an orbital period of around 230 years and an eccentricity of about 0.9. The long period of the transiting planet, combined with the high eccentricity and close approach of the companion star makes this a valuable system for testing the formation and stability of planets in binary systems. △ Less

Submitted 29 April, 2024; originally announced April 2024.

Comments: 24 pages, 16 figures, 4 tables

Journal ref: Published in AJ, 2024

Signal-preserving CMB component separation with machine learning

Authors: Fiona McCarthy, J. Colin Hill, William R. Coulton, David W. Hogg

Abstract: Analysis of microwave sky signals, such as the cosmic microwave background, often requires component separation with multi-frequency methods, where different signals are isolated by their frequency behaviors. Many so-called "blind" methods, such as the internal linear combination (ILC), make minimal assumptions about the spatial distribution of the signal or contaminants, and only assume knowledge… ▽ More Analysis of microwave sky signals, such as the cosmic microwave background, often requires component separation with multi-frequency methods, where different signals are isolated by their frequency behaviors. Many so-called "blind" methods, such as the internal linear combination (ILC), make minimal assumptions about the spatial distribution of the signal or contaminants, and only assume knowledge of the frequency dependence of the signal. The ILC is a minimum-variance linear combination of the measured frequency maps. In the case of Gaussian, statistically isotropic fields, this is the optimal linear combination, as the variance is the only statistic of interest. However, in many cases the signal we wish to isolate, or the foregrounds we wish to remove, are non-Gaussian and/or statistically anisotropic (in particular for Galactic foregrounds). In such cases, it is possible that machine learning (ML) techniques can be used to exploit the non-Gaussian features of the foregrounds and thereby improve component separation. However, many ML techniques require the use of complex, difficult-to-interpret operations on the data. We propose a hybrid method whereby we train an ML model using only combinations of the data that $\textit{do not contain the signal}$, and combine the resulting ML-predicted foreground estimate with the ILC solution to reduce the error from the ILC. We demonstrate our methods on simulations of extragalactic temperature and Galactic polarization foregrounds, and show that our ML model can exploit non-Gaussian features, such as point sources and spatially-varying spectral indices, to produce lower-variance maps than ILC - eg, reducing the variance of the B-mode residual by factors of up to 5 - while preserving the signal of interest in an unbiased manner. Moreover, we often find improved performance when applying our model to foreground models on which it was not trained. △ Less

Submitted 4 April, 2024; originally announced April 2024.

Comments: 22 pages, 13 figures

A Data-Driven Search For Mid-Infrared Excesses Among Five Million Main-Sequence FGK Stars

Authors: Gabriella Contardo, David W. Hogg

Abstract: Stellar infrared excesses can indicate various phenomena of interest, from protoplanetary disks to debris disks, or (more speculatively) techno-signatures along the lines of Dyson spheres. In this paper, we conduct a large search for such excesses, designed as a data-driven contextual anomaly detection pipeline. We focus our search on FGK stars close to the main sequence to favour non-young host s… ▽ More Stellar infrared excesses can indicate various phenomena of interest, from protoplanetary disks to debris disks, or (more speculatively) techno-signatures along the lines of Dyson spheres. In this paper, we conduct a large search for such excesses, designed as a data-driven contextual anomaly detection pipeline. We focus our search on FGK stars close to the main sequence to favour non-young host stars. We look for excess in the mid-infrared, unlocking a large sample to search in while favouring extreme IR excess akin to the ones produced by Extreme Debris Disks (EDD). We combine observations from ESA Gaia DR3, 2MASS, and the unWISE of NASA WISE, and create a catalogue of 4,898,812 stars with $G < 16$ mag. We consider a star to have an excess if it is substantially brighter in $W1$ and $W2$ bands than what is predicted from an ensemble of machine-learning models trained on the data, taking optical and near-infrared information as input features. We apply a set of additional cuts (derived from the ML models and the objects' astronomical features) to avoid false-positive and identify a set of 53 objects (a rate of $1.1\times 10^{-5}$), including one previously identified EDD candidate. Typical infrared-excess fractional luminosities we find are in the range 0.005 to 0.1, consistent with known EDDs. △ Less

Submitted 27 March, 2024; originally announced March 2024.

Comments: 23 pages, 17 figures, 2 tables. Candidates' IDs and (some) code available at https://github.com/contardog/NotATechnosignatureSearch . Comments welcomed!

Frizzle: Combining spectra or images by forward modeling

Authors: David W. Hogg, Andrew R. Casey

Abstract: When there are many observations of an astronomical source - many images with different dithers, or many spectra taken at different barycentric velocities - it is standard practice to shift and stack the data, to (for example) make a high signal-to-noise average image or mean spectrum. Bound-saturating measurements are made by manipulating a likelihood function, where the data are treated as fixed… ▽ More When there are many observations of an astronomical source - many images with different dithers, or many spectra taken at different barycentric velocities - it is standard practice to shift and stack the data, to (for example) make a high signal-to-noise average image or mean spectrum. Bound-saturating measurements are made by manipulating a likelihood function, where the data are treated as fixed, and model parameters are modified to fit the data. Traditional shifting and stacking of data can be converted into a model-fitting procedure, such that the data are not modified, and yet the output is the shift-adjusted mean. The key component of this conversion is a spectral model that is completely flexible but also a continuous function of wavelength (or position in the case of imaging) that can represent any signal being measured by the device after any reasonable translation (or rotation or field distortion). The benefits of a modeling approach are myriad: The sacred data never are modified. Noise maps, data gaps, and bad-data masks don't require interpolation. The output can take the form of an image or spectrum evaluated on a pixel grid, as is traditional. In addition to shifts, the model can account for line-spread or point-spread function variations, world-coordinate-system variations, and calibration or normalization variations. The noise in the output becomes uncorrelated across neighboring pixels as the shifts deliver good coverage in some sense. The only cost is a small increase in computational complexity over that of traditional methods. We demonstrate the method with a small data example and we provide open source sample code for re-use. △ Less

Submitted 16 March, 2024; originally announced March 2024.

Comments: submitted to AAS Journals

zoomies: A tool to infer stellar age from vertical action in Gaia data

Authors: Sheila Sagear, Adrian M. Price-Whelan, Sarah Ballard, Yuxi, Lu, Ruth Angus, David W. Hogg

Abstract: Stellar age measurements are fundamental to understanding a wide range of astronomical processes, including galactic dynamics, stellar evolution, and planetary system formation. However, extracting age information from Main Sequence stars is complicated, with techniques often relying on age proxies in the absence of direct measurements. The Gaia data releases have enabled detailed studies of the d… ▽ More Stellar age measurements are fundamental to understanding a wide range of astronomical processes, including galactic dynamics, stellar evolution, and planetary system formation. However, extracting age information from Main Sequence stars is complicated, with techniques often relying on age proxies in the absence of direct measurements. The Gaia data releases have enabled detailed studies of the dynamical properties of stars within the Milky Way, offering new opportunities to understand the relationship between stellar age and dynamics. In this study, we leverage high-precision astrometric data from Gaia DR3 to construct a stellar age prediction model based only on stellar dynamical properties; namely, the vertical action. We calibrate two distinct, hierarchical stellar age--vertical action relations, first employing asteroseismic ages for red giant branch stars, then isochrone ages for main-sequence turn-off stars. We describe a framework called "zoomies" based on this calibration, by which we can infer ages for any star given its vertical action. This tool is open-source and intended for community use. We compare dynamical age estimates from "zoomies" with ages derived from other techniques for a sample of open clusters and main-sequence stars with asteroseismic age measurements. We also compare dynamical age estimates for stellar samples from the Kepler, K2, and TESS exoplanet transit surveys. While dynamical age relations are associated with large uncertainty, they are generally mass-independent and depend on homogeneously measured astrometric data. These age predictions are uniquely useful for large-scale demographic investigations, especially in disentangling the relationship between planet occurrence, metallicity, and age for low-mass stars. △ Less

Submitted 14 March, 2024; originally announced March 2024.

Comments: 23 pages, 12 figures, 2 tables. Submitted to AAS Journals

Data-driven Dynamics with Orbital Torus Imaging: A Flexible Model of the Vertical Phase Space of the Galaxy

Authors: Adrian M. Price-Whelan, Jason A. S. Hunt, Danny Horta, Micah Oeur, David W. Hogg, Kathryn V. Johnston, Lawrence Widrow

Abstract: The vertical kinematics of stars near the Sun can be used to measure the total mass distribution near the Galactic disk and to study out-of-equilibrium dynamics. With contemporary stellar surveys, the tracers of vertical dynamics are so numerous and so well measured that the shapes of underlying orbits are almost directly visible in the data through element abundances or even stellar density. Thes… ▽ More The vertical kinematics of stars near the Sun can be used to measure the total mass distribution near the Galactic disk and to study out-of-equilibrium dynamics. With contemporary stellar surveys, the tracers of vertical dynamics are so numerous and so well measured that the shapes of underlying orbits are almost directly visible in the data through element abundances or even stellar density. These orbits can be used to infer a mass model for the Milky Way, enabling constraints on the dark matter distribution in the inner galaxy. Here we present a flexible model for foliating the vertical position-velocity phase space with orbits, for use in data-driven studies of dynamics. The vertical acceleration profile in the vicinity of the disk, along with the orbital actions, angles, and frequencies for individual stars, can all be derived from that orbit foliation. We show that this framework - "Orbital Torus Imaging" (OTI) - is rigorously justified in the context of dynamical theory, and does a good job of fitting orbits to simulated stellar abundance data with varying degrees of realism. OTI (1) does not require a global model for the Milky Way mass distribution, and (2) does not require detailed modeling of the selection function of the input survey data. We discuss the approximations and limitations of the OTI framework, which currently trades dynamical interpretability for flexibility in representing the data in some regimes, and which also presently separates the vertical and radial dynamics. We release an open-source tool, torusimaging, to accompany this article. △ Less

Submitted 15 January, 2024; originally announced January 2024.

Comments: 32 pages, 10 figures. See arXiv:2312.07664 for an application of this method with data from APOGEE and Gaia

Orbital Torus Imaging: Acceleration, density, and dark matter in the Galactic disk measured with element abundance gradients

Authors: Danny Horta, Adrian M. Price-Whelan, David W. Hogg, Kathryn V. Johnston, Lawrence Widrow, Julianne J. Dalcanton, Melissa K. Ness, Jason A. S. Hunt

Abstract: Under the assumption of a simple and time-invariant gravitational potential, many Galactic dynamics techniques infer the Milky Way's mass and dark matter distribution from stellar kinematic observations. These methods typically rely on parameterized potential models of the Galaxy and must take into account non-trivial survey selection effects, because they make use of the density of stars in phase… ▽ More Under the assumption of a simple and time-invariant gravitational potential, many Galactic dynamics techniques infer the Milky Way's mass and dark matter distribution from stellar kinematic observations. These methods typically rely on parameterized potential models of the Galaxy and must take into account non-trivial survey selection effects, because they make use of the density of stars in phase space. Large-scale spectroscopic surveys now supply information beyond kinematics in the form of precise stellar label measurements (especially element abundances). These element abundances are known to correlate with orbital actions or other dynamical invariants. Here, we use the Orbital Torus Imaging (OTI) framework that uses abundance gradients in phase space to map orbits. In many cases these gradients can be measured without detailed knowledge of the selection function. We use stellar surface abundances from the APOGEE survey combined with kinematic data from the Gaia mission. Our method reveals the vertical ($z$-direction) orbit structure in the Galaxy and enables empirical measurements of the vertical acceleration field and orbital frequencies in the disk. From these measurements, we infer the total surface mass density, $Σ$, and midplane volume density, $ρ_0$, as a function of Galactocentric radius and height. Around the Sun, we find $Σ_{\odot}(z=1.1$ kpc)$=72^{+6}_{-9}$M$_{\odot}$pc$^{-2}$ and $ρ_{\odot}(z=0)=0.081^{+0.015}_{-0.009}$ M$_{\odot}$pc$^{-3}$ using the most constraining abundance ratio, [Mg/Fe]. This corresponds to a dark matter contribution in surface density of $Σ_{\odot,\mathrm{DM}}(z=1.1$ kpc)$=24\pm4$ M$_{\odot}$pc$^{-2}$, and in total volume mass density of $ρ_{\odot,\mathrm{DM}}(z=0)=0.011\pm0.002$ M$_{\odot}$pc$^{-3}$. Moreover, using these mass density values we estimate the scale length of the low-$α$ disc to be $h_R=2.24\pm0.06$kpc. △ Less

Submitted 18 December, 2023; v1 submitted 12 December, 2023; originally announced December 2023.

Comments: Accepted for publication in ApJ. 19 pages, 11 figures, 3 Tables

AspGap: Augmented Stellar Parameters and Abundances for 23 million RGB stars from Gaia XP low-resolution spectra

Authors: Jiadong Li, Kaze W. K. Wong, David W. Hogg, Hans-Walter Rix, Vedant Chandra

Abstract: We present AspGap, a new approach to infer stellar labels from low-resolution Gaia XP spectra, including precise [$α$/M] estimates for the first time. AspGap is a neural-network based regression model trained on APOGEE spectra. In the training step, AspGap learns to use XP spectra not only to predict stellar labels but also the high-resolution APOGEE spectra that lead to the same stellar labels. T… ▽ More We present AspGap, a new approach to infer stellar labels from low-resolution Gaia XP spectra, including precise [$α$/M] estimates for the first time. AspGap is a neural-network based regression model trained on APOGEE spectra. In the training step, AspGap learns to use XP spectra not only to predict stellar labels but also the high-resolution APOGEE spectra that lead to the same stellar labels. The inclusion of this last model component -- dubbed the hallucinator -- creates a more physically motivated map** and significantly improves the prediction of stellar labels in the validation, particularly of [$α$/M]. For giant stars, we find cross-validated rms accuracies for Teff, log g, [M/H], [$α$/M] of ~1%, 0.12 dex, 0.07 dex, 0.03 dex, respectively. We also validate our labels through comparison with external datasets and through a range of astrophysical tests that demonstrate that we are indeed determining [$α$/M] from the XP spectra, rather than just inferring it indirectly from correlations with other labels. We publicly release the AspGap codebase, along with our stellar parameter catalog for all giants observed by Gaia XP. AspGap enables new insights into the formation and chemo-dynamics of our Galaxy by providing precise [$α$/M] estimates for 23 million giant stars, including 12 million with radial velocities from Gaia. △ Less

Submitted 25 September, 2023; originally announced September 2023.

Comments: 29 pages, 19 figures, submitted to ApJS

KPM: A Flexible and Data-Driven K-Process Model for Nucleosynthesis

Authors: Emily J. Griffith, David W. Hogg, Julianne J. Dalcanton, Sten Hasselquist, Bridget Ratcliffe, Melissa Ness, David H. Weinberg

Abstract: The element abundance pattern found in Milky Way disk stars is close to two-dimensional, dominated by production from one prompt process and one delayed process. This simplicity is remarkable, since the elements are produced by a multitude of nucleosynthesis mechanisms operating in stars with a wide range of progenitor masses. We fit the abundances of 14 elements for 48,659 red-giant stars from AP… ▽ More The element abundance pattern found in Milky Way disk stars is close to two-dimensional, dominated by production from one prompt process and one delayed process. This simplicity is remarkable, since the elements are produced by a multitude of nucleosynthesis mechanisms operating in stars with a wide range of progenitor masses. We fit the abundances of 14 elements for 48,659 red-giant stars from APOGEE DR17 using a flexible, data-driven K-process model -- dubbed KPM. In our fiducial model, with $K=2$, each abundance in each star is described as the sum of a prompt and a delayed process contribution. We find that KPM with $K=2$ is able to explain the abundances well, recover the observed abundance bimodality, and detect the bimodality over a greater range in metallicity than previously has been possible. We compare to prior work by Weinberg et al. (2022), finding that KPM produces similar results, but that KPM better predicts stellar abundances, especially for elements C+N and Mn and for stars at super-solar metallicities. The model fixes the relative contribution of the prompt and delayed process to two elements to break degeneracies and improve interpretability; we find that some of the nucleosynthetic implications are dependent upon these detailed choices. We find that moving to four processes adds flexibility and improves the model's ability to predict the stellar abundances, but doesn't qualitatively change the story. The results of KPM will help us to interpret and constrain the formation of the Galaxy disk, the relationship between abundances and ages, and the physics of nucleosynthesis. △ Less

Submitted 8 December, 2023; v1 submitted 11 July, 2023; originally announced July 2023.

Comments: 33 pages, 13 figures, 3 tables

Quaia, the Gaia-unWISE Quasar Catalog: An All-Sky Spectroscopic Quasar Sample

Authors: Kate Storey-Fisher, David W. Hogg, Hans-Walter Rix, Anna-Christina Eilers, Giulio Fabbian, Michael Blanton, David Alonso

Abstract: We present a new, all-sky quasar catalog, Quaia, that samples the largest comoving volume of any existing spectroscopic quasar sample. The catalog draws on the 6,649,162 quasar candidates identified by the Gaia mission that have redshift estimates from the space observatory's low-resolution BP/RP spectra. This initial sample is highly homogeneous and complete, but has low purity, and 18% of even t… ▽ More We present a new, all-sky quasar catalog, Quaia, that samples the largest comoving volume of any existing spectroscopic quasar sample. The catalog draws on the 6,649,162 quasar candidates identified by the Gaia mission that have redshift estimates from the space observatory's low-resolution BP/RP spectra. This initial sample is highly homogeneous and complete, but has low purity, and 18% of even the bright ($G<20.0$) confirmed quasars have discrepant redshift estimates ($|Δz/(1+z)|>0.2$) compared to those from the Sloan Digital Sky Survey (SDSS). In this work, we combine the Gaia candidates with unWISE infrared data (based on the Wide-field Infrared Survey Explorer survey) to construct a catalog useful for cosmological and astrophysical quasar studies. We apply cuts based on proper motions and Gaia and unWISE colors, reducing the number of contaminants by $\sim$4$\times$. We improve the redshifts by training a $k$-nearest neighbors model on SDSS redshifts, and achieve estimates on the $G<20.0$ sample with only 6% (10%) catastrophic errors with $|Δz/(1+z)|>0.2$ ($0.1$), a reduction of $\sim$3$\times$ ($\sim$2$\times$) compared to the Gaia redshifts. The final catalog has 1,295,502 quasars with $G<20.5$, and 755,850 candidates in an even cleaner $G<20.0$ sample, with accompanying rigorous selection function models. We compare Quaia to existing quasar catalogs, showing that its large effective volume makes it a highly competitive sample for cosmological large-scale structure analyses. The catalog is publicly available at https://zenodo.org/records/10403370. △ Less

Submitted 18 March, 2024; v1 submitted 30 June, 2023; originally announced June 2023.

Comments: Published in the Astrophysical Journal; updated to match published version. Catalog available at https://doi.org/10.5281/zenodo.10403370. Animation available at https://cosmo.nyu.edu/ksf/quaia.mp4

Journal ref: Storey-Fisher (2024) 964, 69

Constraining cosmology with the Gaia-unWISE Quasar Catalog and CMB lensing: structure growth

Authors: David Alonso, Giulio Fabbian, Kate Storey-Fisher, Anna-Christina Eilers, Carlos García-García, David W. Hogg, Hans-Walter Rix

Abstract: We study the angular clustering of Quaia, a Gaia- and unWISE-based catalog of over a million quasars with an exceptionally well-defined selection function. With it, we derive cosmology constraints from the amplitude and growth of structure across cosmic time. We divide the sample into two redshift bins, centered at $z=1.0$ and $z=2.1$, and measure both overdensity auto-correlations and cross-corre… ▽ More We study the angular clustering of Quaia, a Gaia- and unWISE-based catalog of over a million quasars with an exceptionally well-defined selection function. With it, we derive cosmology constraints from the amplitude and growth of structure across cosmic time. We divide the sample into two redshift bins, centered at $z=1.0$ and $z=2.1$, and measure both overdensity auto-correlations and cross-correlations with maps of the Cosmic Microwave Background convergence measured by Planck. From these data, and including a prior from measurements of the baryon acoustic oscillations scale, we place constraints on the amplitude of the matter power spectrum $σ_8=0.766\pm 0.034$, and on the matter density parameter $Ω_m=0.343^{+0.017}_{-0.019}$. These measurements are in reasonable agreement with \planck at the $\sim$ 1.4$σ$ level, and are found to be robust with respect to observational and theoretical uncertainties. We find that our slightly lower value of $σ_8$ is driven by the higher-redshift sample, which favours a low amplitude of matter fluctuations. We present plausible arguments showing that this could be driven by contamination of the CMB lensing map by high-redshift extragalactic foregrounds, which should also affect other cross-correlations with tracers of large-scale structure beyond $z\sim1.5$. Our constraints are competitive with those from state-of-the-art 3$\times$2-point analyses, but arise from a range of scales and redshifts that is highly complementary to those covered by cosmic shear data and most galaxy clustering samples. This, coupled with the unprecedented combination of volume and redshift precision achieved by Quaia allows us to break the usual degeneracy between $Ω_m$ and $σ_8$. △ Less

Submitted 3 July, 2023; v1 submitted 30 June, 2023; originally announced June 2023.

Comments: Submitted to JCAP, corrects links to public data release. The per-bin selection functions used in this analysis, and the complete set of power spectra, covariance, and associated metadata used in our fiducial analysis are publicly available at https://doi.org/10.5281/zenodo.8098635 . The public Quaia catalog can be found at https://doi.org/10.5281/zenodo.8060755

GeometricImageNet: Extending convolutional neural networks to vector and tensor images

Authors: Wilson Gregory, David W. Hogg, Ben Blum-Smith, Maria Teresa Arias, Kaze W. K. Wong, Soledad Villar

Abstract: Convolutional neural networks and their ilk have been very successful for many learning tasks involving images. These methods assume that the input is a scalar image representing the intensity in each pixel, possibly in multiple channels for color images. In natural-science domains however, image-like data sets might have vectors (velocity, say), tensors (polarization, say), pseudovectors (magneti… ▽ More Convolutional neural networks and their ilk have been very successful for many learning tasks involving images. These methods assume that the input is a scalar image representing the intensity in each pixel, possibly in multiple channels for color images. In natural-science domains however, image-like data sets might have vectors (velocity, say), tensors (polarization, say), pseudovectors (magnetic field, say), or other geometric objects in each pixel. Treating the components of these objects as independent channels in a CNN neglects their structure entirely. Our formulation -- the GeometricImageNet -- combines a geometric generalization of convolution with outer products, tensor index contractions, and tensor index permutations to construct geometric-image functions of geometric images that use and benefit from the tensor structure. The framework permits, with a very simple adjustment, restriction to function spaces that are exactly equivariant to translations, discrete rotations, and reflections. We use representation theory to quantify the dimension of the space of equivariant polynomial functions on 2-dimensional vector images. We give partial results on the expressivity of GeometricImageNet on small images. In numerical experiments, we find that GeometricImageNet has good generalization for a small simulated physics system, even when trained with a small training set. We expect this tool will be valuable for scientific and engineering machine learning, for example in cosmology or ocean dynamics. △ Less

Submitted 21 May, 2023; originally announced May 2023.

The Panchromatic Hubble Andromeda Treasury XX: The Disk of M31 is Thick

Authors: Julianne J. Dalcanton, Eric F. Bell, Yumi Choi, Andrew E. Dolphin, Morgan Fouesneau, Léo Girardi, David W. Hogg, Anil C. Seth, Benjamin F. Williams

Abstract: We present a new approach to measuring the thickness of a partially face-on stellar disk, using dust geometry. In a moderately-inclined disk galaxy, the fraction of reddened stars is expected to be 50% everywhere, assuming that dust lies in a thin midplane. In a thickened disk, however, a wide range of radii project onto the line of sight. Assuming stellar density declines with radius, this geomet… ▽ More We present a new approach to measuring the thickness of a partially face-on stellar disk, using dust geometry. In a moderately-inclined disk galaxy, the fraction of reddened stars is expected to be 50% everywhere, assuming that dust lies in a thin midplane. In a thickened disk, however, a wide range of radii project onto the line of sight. Assuming stellar density declines with radius, this geometrical projection leads to differences in the numbers of stars on the near and far sides of the thin dust layer. The fraction of reddened stars will thus differ from the 50% prediction, with a deviation that becomes larger for puffier disks. We map the fraction of reddened red giant branch (RGB) stars across M31, which shows prominent dust lanes on only one side of the major axis. The fraction of reddened stars varies systematically from 20% to 80%, which requires that these stars have an exponential scale height h_z that is 0.14+/-0.015 times the exponential scale length (h_r~5.5kpc). M31's RGB stars must therefore have h_z=770+/-80pc, which is far thicker than the Milky Way's thin disk, but comparable to its thick disk. The lack of a significant thin disk in M31 is unexpected, but consistent with its interaction history and high disk velocity dispersion. We suggest that asymmetric reddening be used as a generic criteria for identifying ``thick disk'' dominated systems, and discuss prospects for future 3-dimensional tomographic map** of the gas and stars in M31. △ Less

Submitted 17 April, 2023; originally announced April 2023.

Comments: 22 pages. Accepted to the Astrophysical Journal

Towards fully covariant machine learning

Authors: Soledad Villar, David W. Hogg, Weichi Yao, George A. Kevrekidis, Bernhard Schölkopf

Abstract: Any representation of data involves arbitrary investigator choices. Because those choices are external to the data-generating process, each choice leads to an exact symmetry, corresponding to the group of transformations that takes one possible representation to another. These are the passive symmetries; they include coordinate freedom, gauge symmetry, and units covariance, all of which have led t… ▽ More Any representation of data involves arbitrary investigator choices. Because those choices are external to the data-generating process, each choice leads to an exact symmetry, corresponding to the group of transformations that takes one possible representation to another. These are the passive symmetries; they include coordinate freedom, gauge symmetry, and units covariance, all of which have led to important results in physics. In machine learning, the most visible passive symmetry is the relabeling or permutation symmetry of graphs. Our goal is to understand the implications for machine learning of the many passive symmetries in play. We discuss dos and don'ts for machine learning practice if passive symmetries are to be respected. We discuss links to causal modeling, and argue that the implementation of passive symmetries is particularly valuable when the goal of the learning problem is to generalize out of sample. This paper is conceptual: It translates among the languages of physics, mathematics, and machine-learning. We believe that consideration and implementation of passive symmetries might help machine learning in the same ways that it transformed physics in the twentieth century. △ Less

Submitted 28 June, 2023; v1 submitted 31 January, 2023; originally announced January 2023.

Comments: substantial revision from v1; submitted to TMLR

The Eighteenth Data Release of the Sloan Digital Sky Surveys: Targeting and First Spectra from SDSS-V

Authors: Andrés Almeida, Scott F. Anderson, Maria Argudo-Fernández, Carles Badenes, Kat Barger, Jorge K. Barrera-Ballesteros, Chad F. Bender, Erika Benitez, Felipe Besser, Dmitry Bizyaev, Michael R. Blanton, John Bochanski, Jo Bovy, William Nielsen Brandt, Joel R. Brownstein, Johannes Buchner, Esra Bulbul, Joseph N. Burchett, Mariana Cano Díaz, Joleen K. Carlberg, Andrew R. Casey, Vedant Chandra, Brian Cherinka, Cristina Chiappini, Abigail A. Coker , et al. (129 additional authors not shown)

Abstract: The eighteenth data release of the Sloan Digital Sky Surveys (SDSS) is the first one for SDSS-V, the fifth generation of the survey. SDSS-V comprises three primary scientific programs, or "Mappers": Milky Way Mapper (MWM), Black Hole Mapper (BHM), and Local Volume Mapper (LVM). This data release contains extensive targeting information for the two multi-object spectroscopy programs (MWM and BHM),… ▽ More The eighteenth data release of the Sloan Digital Sky Surveys (SDSS) is the first one for SDSS-V, the fifth generation of the survey. SDSS-V comprises three primary scientific programs, or "Mappers": Milky Way Mapper (MWM), Black Hole Mapper (BHM), and Local Volume Mapper (LVM). This data release contains extensive targeting information for the two multi-object spectroscopy programs (MWM and BHM), including input catalogs and selection functions for their numerous scientific objectives. We describe the production of the targeting databases and their calibration- and scientifically-focused components. DR18 also includes ~25,000 new SDSS spectra and supplemental information for X-ray sources identified by eROSITA in its eFEDS field. We present updates to some of the SDSS software pipelines and preview changes anticipated for DR19. We also describe three value-added catalogs (VACs) based on SDSS-IV data that have been published since DR17, and one VAC based on the SDSS-V data in the eFEDS field. △ Less

Submitted 6 July, 2023; v1 submitted 18 January, 2023; originally announced January 2023.

Comments: Accepted to ApJS

Vertical motion in the Galactic disc: unwinding the Snail

Authors: Neige Frankel, Jo Bovy, Scott Tremaine, David W. Hogg

Abstract: The distribution of stars in the Milky Way disc shows a spiral structure--the Snail--in the space of velocity and position normal to the Galactic mid-plane. The Snail appears as straight lines in the vertical frequency--vertical phase plane when effects from sample selection are removed. Their slope has the dimension of inverse time, with the simplest interpretation being the inverse age of the Sn… ▽ More The distribution of stars in the Milky Way disc shows a spiral structure--the Snail--in the space of velocity and position normal to the Galactic mid-plane. The Snail appears as straight lines in the vertical frequency--vertical phase plane when effects from sample selection are removed. Their slope has the dimension of inverse time, with the simplest interpretation being the inverse age of the Snail. Here, we devise and fit a simple model in which the spiral starts as a lopsided perturbation from steady state, that winds up into the present-day morphology. The winding occurs because the vertical frequency decreases with vertical action. We use data from stars in Gaia EDR3 that have measured radial velocities, pruned by simple distance and photometric selection functions. We divide the data into boxels of dynamical invariants (radial action, angular momentum); our model fits the data well in many of the boxels. The model parameters have physical interpretations: one, $A$, is a perturbation amplitude, and one, $t$, is interpretable in the simplest models as the time since the event that caused the Snail. We find trends relating the strength and age to angular momentum: (i) the amplitude $A$ is small at low angular momentum ($<1\,600\mathrm{\,kpc\,km\,s}^{-1}$ or guiding-centre radius $< 7.3\,$kpc), and over a factor of three larger, with strong variations, in the outer disc; (ii) there is no single well-defined perturbation time, with $t$ varying between 0.2 and 0.6 Gyr. Residuals between the data and the model display systematic trends, implying that the data call for more complex models. △ Less

Submitted 22 December, 2022; originally announced December 2022.

Comments: 9 pages, 8 figures. Submitted to MNRAS (constructive comments are welcome)

A Generative Model for Quasar Spectra

Authors: Anna-Christina Eilers, David W. Hogg, Bernhard Schölkopf, Daniel Foreman-Mackey, Frederick B. Davies, Jan-Torge Schindler

Abstract: We build a multi-output generative model for quasar spectra and the properties of their black hole engines, based on a Gaussian process latent-variable model. This model treats every quasar as a vector of latent properties such that the spectrum and all physical properties of the quasar are associated with non-linear functions of those latent parameters; the Gaussian process kernel functions defin… ▽ More We build a multi-output generative model for quasar spectra and the properties of their black hole engines, based on a Gaussian process latent-variable model. This model treats every quasar as a vector of latent properties such that the spectrum and all physical properties of the quasar are associated with non-linear functions of those latent parameters; the Gaussian process kernel functions define priors on the function space. Our generative model is trained with a justifiable likelihood function that allows us to treat heteroscedastic noise and missing data correctly, which is crucial for all astrophysical applications. It can predict simultaneously unobserved spectral regions, as well as the physical properties of quasars in held-out test data. We apply the model to rest-frame ultraviolet and optical quasar spectra for which precise black hole masses (based on reverberation map** measurements) are available. Unlike reverberation-map** studies, which require multi-epoch data, our model predicts black hole masses from single-epoch spectra, even with limited spectral coverage. We demonstrate the capabilities of the model by predicting black hole masses and unobserved spectral regions. We find that we predict black hole masses at close to the best possible accuracy. △ Less

Submitted 6 September, 2022; originally announced September 2022.

Comments: accepted for publication in ApJ

The Poor Old Heart of the Milky Way

Authors: Hans-Walter Rix, Vedant Chandra, René Andrae, Adrian M. Price-Whelan, David H. Weinberg, Charlie Conroy, Morgan Fouesneau, David W. Hogg, Francesca De Angeli, Rohan P. Naidu, Maosheng Xiang, Daniela Ruz-Mieres

Abstract: Massive disk galaxies like our Milky Way should host an ancient, metal-poor, and centrally concentrated stellar population. This population reflects the star formation and enrichment in the few most massive progenitor components that coalesced at high redshift to form the proto-Galaxy. While metal-poor stars are known to reside in the inner few kiloparsecs of our Galaxy, current data do not yet pr… ▽ More Massive disk galaxies like our Milky Way should host an ancient, metal-poor, and centrally concentrated stellar population. This population reflects the star formation and enrichment in the few most massive progenitor components that coalesced at high redshift to form the proto-Galaxy. While metal-poor stars are known to reside in the inner few kiloparsecs of our Galaxy, current data do not yet provide a comprehensive picture of such a metal-poor "heart" of the Milky Way. We use information from Gaia DR3, especially the XP spectra, to construct a sample of 2 million bright (BP $<15.5$ mag) giant stars within $30^\circ$ of the Galactic Center with robust [M/H] estimates, $δ$ [M/H] $\lesssim 0.1$. For most sample members we can calculate orbits based on Gaia RVS velocities and astrometry. This sample reveals an extensive, ancient, and metal-poor population that includes $\sim 18,000$ stars with $-2.7<$ [M/H] $<-1.5$, representing a stellar mass of $\gtrsim 5\times 10^7$ M$_\odot$. The spatial distribution of these [M/H] $<-1.5$ stars has a Gaussian extent of only $σ_{\mathrm{R_{GC}}} \sim 2.7$ kpc around the Galactic center, with most of these orbits being confined to the inner Galaxy. At high orbital eccentricities, there is clear evidence for accreted halo stars in their pericentral orbit phase. Stars with [M/H] $< -2$ show no net rotation, whereas those with [M/H] $\sim -1$ are rotation dominated. Most of the tightly bound stars show $[α/\text{Fe}]$-enhancement and [Al/Fe]-[Mn/Fe] abundance patterns expected for an origin in the more massive portions of the proto-Galaxy. These central, metal-poor stars most likely predate the oldest part of the disk ($τ_{\text{age}}\approx 12.5$ Gyrs), which implies that they formed at $z\gtrsim 5$, forging the proto-Milky Way. △ Less

Submitted 6 September, 2022; originally announced September 2022.

Comments: 21 pages, 9 figures, submitted to ApJ

An empirical model of the Gaia DR3 selection function

Authors: Tristan Cantat-Gaudin, Morgan Fouesneau, Hans-Walter Rix, Anthony G. A. Brown, Alfred Castro-Ginard, Ronald Drimmel, David W. Hogg, Andrew R. Casey, Shourya Khanna, Semyeong Oh, Adrian M. Price Whelan, Vasily Belokurov, Andrew K. Saydjari, Gregory M. Green

Abstract: Interpreting and modelling astronomical catalogues requires an understanding of the catalogues' completeness or selection function: objects of what properties had a chance to end up in the catalogue. Here we set out to empirically quantify the completeness of the overall Gaia DR3 catalogue. This task is not straightforward because Gaia is the all-sky optical survey with the highest angular resolut… ▽ More Interpreting and modelling astronomical catalogues requires an understanding of the catalogues' completeness or selection function: objects of what properties had a chance to end up in the catalogue. Here we set out to empirically quantify the completeness of the overall Gaia DR3 catalogue. This task is not straightforward because Gaia is the all-sky optical survey with the highest angular resolution to date and no consistent ``ground truth'' exists to allow direct comparisons. However, well-characterised deeper imaging enables an empirical assessment of Gaia's $G$-band completeness across parts of the sky. On this basis, we devised a simple analytical completeness model of Gaia as a function of the observed $G$ magnitude and position over the sky, which accounts for both the effects of crowding and the complex Gaia scanning law. Our model only depends on a single quantity: the median magnitude $M_{10}$ in a patch of the sky of catalogued sources with $\texttt{astrometric_matched_transits}$ $\leq 10$. $M_{10}$ reflects elementary completeness decisions in the Gaia pipeline and is computable from the Gaia DR3 catalogue itself and therefore applicable across the whole sky. We calibrate our model using the Dark Energy Camera Plane Survey (DECaPS) and test its predictions against Hubble Space Telescope observations of globular clusters. We find that our model predicts Gaia's completeness values to a few per cent across the sky. We make the model available as a part of the $\texttt{gaiasf}$ Python package built and maintained by the GaiaUnlimited project: $\texttt{https://github.com/gaia-unlimited/gaiaunlimited}$ △ Less

Submitted 6 September, 2022; v1 submitted 19 August, 2022; originally announced August 2022.

Comments: submitted to A&A

Journal ref: A&A 669, A55 (2023)

Magnitudes, distance moduli, bolometric corrections, and so much more

Authors: David W. Hogg

Abstract: This pedagogical document about stellar photometry - aimed at those for whom astronomical arcana seem arcane - endeavours to explain the concepts of magnitudes, color indices, absolute magnitudes, distance moduli, extinctions, attenuations, color excesses, K corrections, and bolometric corrections. I include some discussion of observational technique, and some discussion of epistemology, but the p… ▽ More This pedagogical document about stellar photometry - aimed at those for whom astronomical arcana seem arcane - endeavours to explain the concepts of magnitudes, color indices, absolute magnitudes, distance moduli, extinctions, attenuations, color excesses, K corrections, and bolometric corrections. I include some discussion of observational technique, and some discussion of epistemology, but the primary focus here is on the theoretical or interpretive connections between the observational astronomical quantities and the physical properties of the observational targets. △ Less

Submitted 10 September, 2022; v1 submitted 2 June, 2022; originally announced June 2022.

Comments: Not submitted anywhere (yet)

Searching for quasi-periodic oscillations in astrophysical transients using Gaussian processes

Authors: M. Hübner, D. Huppenkothen, P. D. Lasky, A. R. Inglis, C. Ick, D. W. Hogg

Abstract: Analyses of quasi-periodic oscillations (QPOs) are important to understanding the dynamic behaviour in many astrophysical objects during transient events like gamma-ray bursts, solar flares, magnetar flares and fast radio bursts. Astrophysicists often search for QPOs with frequency-domain methods such as (Lomb-Scargle) periodograms, which generally assume power-law models plus some excess around t… ▽ More Analyses of quasi-periodic oscillations (QPOs) are important to understanding the dynamic behaviour in many astrophysical objects during transient events like gamma-ray bursts, solar flares, magnetar flares and fast radio bursts. Astrophysicists often search for QPOs with frequency-domain methods such as (Lomb-Scargle) periodograms, which generally assume power-law models plus some excess around the QPO frequency. Time-series data can alternatively be investigated directly in the time domain using Gaussian Process (GP) regression. While GP regression is computationally expensive in the general case, the properties of astrophysical data and models allow fast likelihood strategies. Heteroscedasticity and non-stationarity in data have been shown to cause bias in periodogram-based analyses. Gaussian processes can take account of these properties. Using GPs, we model QPOs as a stochastic process on top of a deterministic flare shape. Using Bayesian inference, we demonstrate how to infer GP hyperparameters and assign them physical meaning, such as the QPO frequency. We also perform model selection between QPOs and alternative models such as red noise and show that this can be used to reliably find QPOs. This method is easily applicable to a variety of different astrophysical data sets. We demonstrate the use of this method on a range of short transients: a gamma-ray burst, a magnetar flare, a magnetar giant flare, and simulated solar flare data. △ Less

Submitted 25 May, 2022; originally announced May 2022.

Comments: 24 pages, 19 figures, submitted to The Astrophysical Journal

Map** Dark Matter with Extragalactic Stellar Streams: the Case of Centaurus A

Authors: Sarah Pearson, Adrian M. Price-Whelan, David W. Hogg, Anil C. Seth, David J. Sand, Jason A. S. Hunt, Denija Crnojevic

Abstract: In the coming decade, thousands of stellar streams will be observed in the halos of external galaxies. What fundamental discoveries will we make about dark matter from these streams? As a first attempt to look at these questions, we model Magellan/Megacam imaging of the Centaurus A's (Cen A) disrupting dwarf companion Dwarf 3 (Dw3) and its associated stellar stream, to find out what can be learned… ▽ More In the coming decade, thousands of stellar streams will be observed in the halos of external galaxies. What fundamental discoveries will we make about dark matter from these streams? As a first attempt to look at these questions, we model Magellan/Megacam imaging of the Centaurus A's (Cen A) disrupting dwarf companion Dwarf 3 (Dw3) and its associated stellar stream, to find out what can be learned about the Cen A dark-matter halo. We develop a novel external galaxy stream-fitting technique and generate model stellar streams that reproduce the stream morphology visible in the imaging. We find that there are many viable stream models that fit the data well, with reasonable parameters, provided that Cen A has a halo mass larger than M$_{200}$ $>4.70\times 10^{12}$ M$_{\odot}$. There is a second stream in Cen A's halo that is also reproduced within the context of this same dynamical model. However, stream morphology in the imaging alone does not uniquely determine the mass or mass distribution for the Cen A halo. In particular, the stream models with high likelihood show covariances between the inferred Cen A mass distribution, the inferred Dw3 progenitor mass, the Dw3 velocity, and the Dw3 line-of-sight position. We show that these degeneracies can be broken with radial-velocity measurements along the stream, and that a single radial velocity measurement puts a substantial lower limit on the halo mass. These results suggest that targeted radial-velocity measurements will be critical if we want to learn about dark matter from extragalactic stellar streams. △ Less

Submitted 20 October, 2022; v1 submitted 24 May, 2022; originally announced May 2022.

Comments: 11 figures, 23 pages, Accepted to ApJ

Dimensionless machine learning: Imposing exact units equivariance

Authors: Soledad Villar, Weichi Yao, David W. Hogg, Ben Blum-Smith, Bianca Dumitrascu

Abstract: Units equivariance (or units covariance) is the exact symmetry that follows from the requirement that relationships among measured quantities of physics relevance must obey self-consistent dimensional scalings. Here, we express this symmetry in terms of a (non-compact) group action, and we employ dimensional analysis and ideas from equivariant machine learning to provide a methodology for exactly… ▽ More Units equivariance (or units covariance) is the exact symmetry that follows from the requirement that relationships among measured quantities of physics relevance must obey self-consistent dimensional scalings. Here, we express this symmetry in terms of a (non-compact) group action, and we employ dimensional analysis and ideas from equivariant machine learning to provide a methodology for exactly units-equivariant machine learning: For any given learning task, we first construct a dimensionless version of its inputs using classic results from dimensional analysis, and then perform inference in the dimensionless space. Our approach can be used to impose units equivariance across a broad range of machine learning methods which are equivariant to rotations and other groups. We discuss the in-sample and out-of-sample prediction accuracy gains one can obtain in contexts like symbolic regression and emulation, where symmetry is important. We illustrate our approach with simple numerical examples involving dynamical systems in physics and ecology. △ Less

Submitted 31 December, 2022; v1 submitted 2 April, 2022; originally announced April 2022.

Journal ref: Journal of Machine Learning Research 24 (2023) 1--32

Kepler K2 Campaign 9: II. First space-based discovery of an exoplanet using microlensing

Authors: D. Specht, R. Poleski, M. T. Penny, E. Kerins, I. McDonald, Chung-Uk Lee, A. Udalski, I. A. Bond, Y. Shvartzvald, Weicheng Zang, R. A. Street, D. W. Hogg, B. S. Gaudi, T. Barclay, G. Barentsen, S. B. Howell, F. Mullally, C. B. Henderson, S. T. Bryson, D. A. Caldwell, M. R. Haas, J. E. Van Cleve, K. Larson, K. McCalmont, C. Peterson , et al. (61 additional authors not shown)

Abstract: We present K2-2016-BLG-0005Lb, a densely sampled, planetary binary caustic-crossing microlensing event found from a blind search of data gathered from Campaign 9 of the Kepler K2 mission (K2C9). K2-2016-BLG-0005Lb is the first bound microlensing exoplanet discovered from space-based data. The event has caustic entry and exit points that are resolved in the K2C9 data, enabling the lens--source rela… ▽ More We present K2-2016-BLG-0005Lb, a densely sampled, planetary binary caustic-crossing microlensing event found from a blind search of data gathered from Campaign 9 of the Kepler K2 mission (K2C9). K2-2016-BLG-0005Lb is the first bound microlensing exoplanet discovered from space-based data. The event has caustic entry and exit points that are resolved in the K2C9 data, enabling the lens--source relative proper motion to be measured. We have fitted a binary microlens model to the Kepler data, and to simultaneous observations from multiple ground-based surveys. Whilst the ground-based data only sparsely sample the binary caustic, they provide a clear detection of parallax that allows us to break completely the microlensing mass--position--velocity degeneracy and measure the planet's mass directly. We find a host mass of $0.58\pm0.04 ~{\rm M}_\odot$ and a planetary mass of $1.1\pm0.1 ~{\rm M_J}$. The system lies at a distance of $5.2\pm0.2~$kpc from Earth towards the Galactic bulge, more than twice the distance of the previous most distant planet found by Kepler. The sky-projected separation of the planet from its host is found to be $4.2\pm0.3~$au which, for circular orbits, deprojects to a host separation $a = 4.4^{+1.9}_{-0.4}~$au and orbital period $P = 13^{+9}_{-2}~$yr. This makes K2-2016-BLG-0005Lb a close Jupiter analogue orbiting a low-mass host star. According to current planet formation models, this system is very close to the host mass threshold below which Jupiters are not expected to form. Upcoming space-based exoplanet microlensing surveys by NASA's Nancy Grace Roman Space Telescope and, possibly, ESA's Euclid mission, will provide demanding tests of current planet formation models. △ Less

Submitted 2 February, 2023; v1 submitted 31 March, 2022; originally announced March 2022.

Comments: 18 pages. Accepted for publication in MNRAS Published by Oxford University Press on behalf of the Royal Astronomical Society

Map** Interstellar Dust with Gaussian Processes

Authors: Andrew C. Miller, Lauren Anderson, Boris Leistedt, John P. Cunningham, David W. Hogg, David M. Blei

Abstract: Interstellar dust corrupts nearly every stellar observation, and accounting for it is crucial to measuring physical properties of stars. We model the dust distribution as a spatially varying latent field with a Gaussian process (GP) and develop a likelihood model and inference method that scales to millions of astronomical observations. Modeling interstellar dust is complicated by two factors. The… ▽ More Interstellar dust corrupts nearly every stellar observation, and accounting for it is crucial to measuring physical properties of stars. We model the dust distribution as a spatially varying latent field with a Gaussian process (GP) and develop a likelihood model and inference method that scales to millions of astronomical observations. Modeling interstellar dust is complicated by two factors. The first is integrated observations. The data come from a vantage point on Earth and each observation is an integral of the unobserved function along our line of sight, resulting in a complex likelihood and a more difficult inference problem than in classical GP inference. The second complication is scale; stellar catalogs have millions of observations. To address these challenges we develop ziggy, a scalable approach to GP inference with integrated observations based on stochastic variational inference. We study ziggy on synthetic data and the Ananke dataset, a high-fidelity mechanistic model of the Milky Way with millions of stars. ziggy reliably infers the spatial dust map with well-calibrated posterior uncertainties. △ Less

Submitted 14 February, 2022; originally announced February 2022.

The Thresher: Lucky Imaging without the Waste

Authors: James A. Hitchcock, D. M. Bramich, Daniel Foreman-Mackey, David W. Hogg, Markus Hundertmark

Abstract: In traditional lucky imaging (TLI), many consecutive images of the same scene are taken with a high frame-rate camera, and all but the sharpest images are discarded before constructing the final shift-and-add image. Here we present an alternative image analysis pipeline -- The Thresher -- for these kinds of data, based on online multi-frame blind deconvolution. It makes use of all available data t… ▽ More In traditional lucky imaging (TLI), many consecutive images of the same scene are taken with a high frame-rate camera, and all but the sharpest images are discarded before constructing the final shift-and-add image. Here we present an alternative image analysis pipeline -- The Thresher -- for these kinds of data, based on online multi-frame blind deconvolution. It makes use of all available data to obtain a best estimate of the astronomical scene in the context of reasonable computational limits; it does not require prior estimates of the point-spread functions in the images, or knowledge of point sources in the scene that could provide such estimates. Most importantly, the scene it aims to return is the optimum of a justified scalar objective based on the likelihood function. Because it uses the full set of images in the stack, The Thresher outperforms TLI in signal-to-noise; as it accounts for the individual-frame PSFs, it does this without loss of angular resolution. We demonstrate the effectiveness of our algorithm on both simulated data and real Electron-Multiplying CCD images obtained at the Danish 1.54m telescope (hosted by ESO, La Silla). We also explore the current limitations of the algorithm, and find that for the choice of image model presented here, non-linearities in flux are introduced into the returned scene. Ongoing development of the software can be viewed at https://github.com/jah1994/TheThresher. △ Less

Submitted 9 February, 2022; originally announced February 2022.

Comments: 14 pages, 9 figures, Accepted for publication in Monthly Notices of the Royal Astronomical Society

The emptiness inside: Finding gaps, valleys, and lacunae with geometric data analysis

Authors: Gabriella Contardo, David W. Hogg, Jason A. S. Hunt, Joshua E. G. Peek, Yen-Chi Chen

Abstract: Discoveries of gaps in data have been important in astrophysics. For example, there are kinematic gaps opened by resonances in dynamical systems, or exoplanets of a certain radius that are empirically rare. A gap in a data set is a kind of anomaly, but in an unusual sense: Instead of being a single outlier data point, situated far from other data points, it is a region of the space, or a set of po… ▽ More Discoveries of gaps in data have been important in astrophysics. For example, there are kinematic gaps opened by resonances in dynamical systems, or exoplanets of a certain radius that are empirically rare. A gap in a data set is a kind of anomaly, but in an unusual sense: Instead of being a single outlier data point, situated far from other data points, it is a region of the space, or a set of points, that is anomalous compared to its surroundings. Gaps are both interesting and hard to find and characterize, especially when they have non-trivial shapes. We present in this paper a statistic that can be used to estimate the (local) "gappiness" of a point in the data space. It uses the gradient and Hessian of the density estimate (and thus requires a twice-differentiable density estimator). This statistic can be computed at (almost) any point in the space and does not rely on optimization; it allows to highlight under-dense regions of any dimensionality and shape in a general and efficient way. We illustrate our method on the velocity distribution of nearby stars in the Milky Way disk plane, which exhibits gaps that could originate from different processes. Identifying and characterizing those gaps could help determine their origins. We provide in an Appendix implementation notes and additional considerations for finding under-densities in data, using critical points and the properties of the Hessian of the density. △ Less

Submitted 5 September, 2022; v1 submitted 25 January, 2022; originally announced January 2022.

Comments: 17 pages, 10 figures. Submitted to AJ. Comments welcomed. Revision: added 3D gridding + restructured outline: implementation notes (Quadratic Kernel) and methods for approx critical points and 1d-valley now in Annex

The EXPRES Stellar Signals Project II. State of the Field in Disentangling Photospheric Velocities

Authors: Lily L. Zhao, Debra A. Fischer, Eric B. Ford, Alex Wise, Michaël Cretignier, Suzanne Aigrain, Oscar Barragan, Megan Bedell, Lars A. Buchhave, João D. Camacho, Heather M. Cegla, Jessi Cisewski-Kehe, Andrew Collier Cameron, Zoe L. de Beurs, Sally Dodson-Robinson, Xavier Dumusque, João P. Faria, Christian Gilbertson, Charlotte Haley, Justin Harrell, David W. Hogg, Parker Holzer, Ancy Anna John, Baptiste Klein, Marina Lafarga , et al. (18 additional authors not shown)

Abstract: Measured spectral shifts due to intrinsic stellar variability (e.g., pulsations, granulation) and activity (e.g., spots, plages) are the largest source of error for extreme precision radial velocity (EPRV) exoplanet detection. Several methods are designed to disentangle stellar signals from true center-of-mass shifts due to planets. The EXPRES Stellar Signals Project (ESSP) presents a self-consist… ▽ More Measured spectral shifts due to intrinsic stellar variability (e.g., pulsations, granulation) and activity (e.g., spots, plages) are the largest source of error for extreme precision radial velocity (EPRV) exoplanet detection. Several methods are designed to disentangle stellar signals from true center-of-mass shifts due to planets. The EXPRES Stellar Signals Project (ESSP) presents a self-consistent comparison of 22 different methods tested on the same extreme-precision spectroscopic data from EXPRES. Methods derived new activity indicators, constructed models for map** an indicator to the needed RV correction, or separated out shape- and shift-driven RV components. Since no ground truth is known when using real data, relative method performance is assessed using the total and nightly scatter of returned RVs and agreement between the results of different methods. Nearly all submitted methods return a lower RV RMS than classic linear decorrelation, but no method is yet consistently reducing the RV RMS to sub-meter-per-second levels. There is a concerning lack of agreement between the RVs returned by different methods. These results suggest that continued progress in this field necessitates increased interpretability of methods, high-cadence data to capture stellar signals at all timescales, and continued tests like the ESSP using consistent data sets with more advanced metrics for method performance. Future comparisons should make use of various well-characterized data sets -- such as solar data or data with known injected planetary and/or stellar signals -- to better understand method performance and whether planetary signals are preserved. △ Less

Submitted 25 January, 2022; originally announced January 2022.

Comments: 33 pages (+12 pages of Appendix), 10 figures, 8 tables, accepted for publication in AJ

Stellar Abundance Maps of the Milky Way Disk

Authors: Anna-Christina Eilers, David W. Hogg, Hans-Walter Rix, Melissa K. Ness, Adrian M. Price-Whelan, Szabolcs Meszaros, Christian Nitschelm

Abstract: To understand the formation of the Milky Way's prominent bar it is important to know whether stars in the bar differ in the chemical element composition of their birth material as compared to disk stars. This requires stellar abundance measurements for large samples across the Milky Way's body. Such samples, e.g. luminous red giant stars observed by SDSS's Apogee survey, will inevitably span a ran… ▽ More To understand the formation of the Milky Way's prominent bar it is important to know whether stars in the bar differ in the chemical element composition of their birth material as compared to disk stars. This requires stellar abundance measurements for large samples across the Milky Way's body. Such samples, e.g. luminous red giant stars observed by SDSS's Apogee survey, will inevitably span a range of stellar parameters; as a consequence, both modelling imperfections and stellar evolution may preclude consistent and precise estimates of their chemical composition at a level of purported bar signatures, which has left current analyses of a chemically distinct bar inconclusive. Here, we develop a new self-calibration approach to eliminate both modelling and astrophysical abundance systematics among red giant branch (RGB) stars of different luminosities (and hence surface gravity $\log g$). We apply our method to $48,853$ luminous Apogee DR16 RGB stars to construct spatial abundance maps of $20$ chemical elements near the Milky Way's mid-plane, covering Galactocentric radii of $0\,{\rm kpc}<R_{\rm GC}<20\,\rm kpc$. Our results indicate that there are no abundance variations whose geometry matches that of the bar, and that the mean abundance gradients vary smoothly and monotonically with Galactocentric radius. We confirm that the high-$α$ disk is chemically homogeneous, without spatial gradients. Furthermore, we present the most precise [Fe/H] vs. $R_{\rm GC}$ gradient to date with a slope of $-0.057\pm0.001\rm~dex\,kpc^{-1}$ out to approximately $15$ kpc. △ Less

Submitted 18 February, 2022; v1 submitted 6 December, 2021; originally announced December 2021.

Comments: ApJ accepted

The Seventeenth Data Release of the Sloan Digital Sky Surveys: Complete Release of MaNGA, MaStar and APOGEE-2 Data

Authors: Abdurro'uf, Katherine Accetta, Conny Aerts, Victor Silva Aguirre, Romina Ahumada, Nikhil Ajgaonkar, N. Filiz Ak, Shadab Alam, Carlos Allende Prieto, Andres Almeida, Friedrich Anders, Scott F. Anderson, Brett H. Andrews, Borja Anguiano, Erik Aquino-Ortiz, Alfonso Aragon-Salamanca, Maria Argudo-Fernandez, Metin Ata, Marie Aubert, Vladimir Avila-Reese, Carles Badenes, Rodolfo H. Barba, Kat Barger, Jorge K. Barrera-Ballesteros, Rachael L. Beaton , et al. (316 additional authors not shown)

Abstract: This paper documents the seventeenth data release (DR17) from the Sloan Digital Sky Surveys; the fifth and final release from the fourth phase (SDSS-IV). DR17 contains the complete release of the Map** Nearby Galaxies at Apache Point Observatory (MaNGA) survey, which reached its goal of surveying over 10,000 nearby galaxies. The complete release of the MaNGA Stellar Library (MaStar) accompanies… ▽ More This paper documents the seventeenth data release (DR17) from the Sloan Digital Sky Surveys; the fifth and final release from the fourth phase (SDSS-IV). DR17 contains the complete release of the Map** Nearby Galaxies at Apache Point Observatory (MaNGA) survey, which reached its goal of surveying over 10,000 nearby galaxies. The complete release of the MaNGA Stellar Library (MaStar) accompanies this data, providing observations of almost 30,000 stars through the MaNGA instrument during bright time. DR17 also contains the complete release of the Apache Point Observatory Galactic Evolution Experiment 2 (APOGEE-2) survey which publicly releases infra-red spectra of over 650,000 stars. The main sample from the Extended Baryon Oscillation Spectroscopic Survey (eBOSS), as well as the sub-survey Time Domain Spectroscopic Survey (TDSS) data were fully released in DR16. New single-fiber optical spectroscopy released in DR17 is from the SPectroscipic IDentification of ERosita Survey (SPIDERS) sub-survey and the eBOSS-RM program. Along with the primary data sets, DR17 includes 25 new or updated Value Added Catalogs (VACs). This paper concludes the release of SDSS-IV survey data. SDSS continues into its fifth phase with observations already underway for the Milky Way Mapper (MWM), Local Volume Mapper (LVM) and Black Hole Mapper (BHM) surveys. △ Less

Submitted 13 January, 2022; v1 submitted 3 December, 2021; originally announced December 2021.

Comments: 40 pages, 8 figures, 6 tables. In press at ApJSS (arxiv v2 corrects some minor typos and updates references)

Map** stellar surfaces III: An Efficient, Scalable, and Open-Source Doppler Imaging Model

Authors: Rodrigo Luger, Megan Bedell, Daniel Foreman-Mackey, Ian J. M. Crossfield, Lily L. Zhao, David W. Hogg

Abstract: The study of stellar surfaces can reveal information about the chemical composition, interior structure, and magnetic properties of stars. It is also critical to the detection and characterization of extrasolar planets, in particular those targeted in extreme precision radial velocity (EPRV) searches, which must contend with stellar variability that is often orders of magnitude stronger than the p… ▽ More The study of stellar surfaces can reveal information about the chemical composition, interior structure, and magnetic properties of stars. It is also critical to the detection and characterization of extrasolar planets, in particular those targeted in extreme precision radial velocity (EPRV) searches, which must contend with stellar variability that is often orders of magnitude stronger than the planetary signal. One of the most successful methods to map the surfaces of stars is Doppler imaging, in which the presence of inhomogeneities is inferred from subtle line shape changes in high resolution stellar spectra. In this paper, we present a novel, efficient, and closed-form solution to the problem of Doppler imaging of stellar surfaces. Our model explicitly allows for incomplete knowledge of the local (rest frame) stellar spectrum, allowing one to learn differences from spectral templates while simultaneously map** the stellar surface. It therefore works on blended lines, regions of the spectrum where line formation mechanisms are not well understood, or stars whose spots have intrinsically different spectra from the rest of the photosphere. We implement the model within the open source starry framework, making it fast, differentiable, and easy to use in both optimization and posterior inference settings. As a proof-of-concept, we use our model to infer the surface map of the brown dwarf WISE 1049-5319B, finding close agreement with the solution of Crossfield et al. (2014). We also discuss Doppler imaging in the context of EPRV studies and describe an interpretable spectral-temporal Gaussian process for stellar spectral variability that we expect will be important for EPRV exoplanet searches. △ Less

Submitted 12 October, 2021; originally announced October 2021.

Comments: 59 pages, 18 figures. Prepared using the showyourwork (github.com/rodluger/showyourwork) open source scientific article workflow

A simple equivariant machine learning method for dynamics based on scalars

Authors: Weichi Yao, Kate Storey-Fisher, David W. Hogg, Soledad Villar

Abstract: Physical systems obey strict symmetry principles. We expect that machine learning methods that intrinsically respect these symmetries should have higher prediction accuracy and better generalization in prediction of physical dynamics. In this work we implement a principled model based on invariant scalars, and release open-source code. We apply this Scalars method to a simple chaotic dynamical sys… ▽ More Physical systems obey strict symmetry principles. We expect that machine learning methods that intrinsically respect these symmetries should have higher prediction accuracy and better generalization in prediction of physical dynamics. In this work we implement a principled model based on invariant scalars, and release open-source code. We apply this Scalars method to a simple chaotic dynamical system, the springy double pendulum. We show that the Scalars method outperforms state-of-the-art approaches for learning the properties of physical systems with symmetries, both in terms of accuracy and speed. Because the method incorporates the fundamental symmetries, we expect it to generalize to different settings, such as changes in the force laws in the system. △ Less

Submitted 30 October, 2021; v1 submitted 7 October, 2021; originally announced October 2021.

Snails Across Scales: Local and Global Phase-Mixing Structures as Probes of the Past and Future Milky Way

Authors: Suroor S. Gandhi, Kathryn V. Johnston, Jason A. S. Hunt, Adrian M. Price-Whelan, Chervin F. P. Laporte, David W. Hogg

Abstract: Signatures of vertical disequilibrium have been observed across the Milky Way's disk. These signatures manifest locally as unmixed phase-spirals in $z$--$v_z$ space ("snails-in-phase") and globally as nonzero mean $z$ and $v_z$ which wraps around as a physical spiral across the $x$--$y$ plane ("snails-in-space"). We explore the connection between these local and global spirals through the example… ▽ More Signatures of vertical disequilibrium have been observed across the Milky Way's disk. These signatures manifest locally as unmixed phase-spirals in $z$--$v_z$ space ("snails-in-phase") and globally as nonzero mean $z$ and $v_z$ which wraps around as a physical spiral across the $x$--$y$ plane ("snails-in-space"). We explore the connection between these local and global spirals through the example of a satellite perturbing a test-particle Milky Way (MW)-like disk. We anticipate our results to broadly apply to any vertical perturbation. Using a $z$--$v_z$ asymmetry metric we demonstrate that in test-particle simulations: (a) multiple local phase-spiral morphologies appear when stars are binned by azimuthal action $J_φ$, excited by a single event (in our case, a satellite disk-crossing); (b) these distinct phase-spirals are traced back to distinct disk locations; and (c) they are excited at distinct times. Thus, local phase-spirals offer a global view of the MW's perturbation history from multiple perspectives. Using a toy model for a Sagittarius (Sgr)-like satellite crossing the disk, we show that the full interaction takes place on timescales comparable to orbital periods of disk stars within $R \lesssim 10$ kpc. Hence such perturbations have widespread influence which peaks in distinct regions of the disk at different times. This leads us to examine the ongoing MW-Sgr interaction. While Sgr has not yet crossed the disk (currently, $z_{Sgr} \approx -6$ kpc, $v_{z,Sgr} \approx 210$ km/s), we demonstrate that the peak of the impact has already passed. Sgr's pull over the past 150 Myr creates a global $v_z$ signature with amplitude $\propto M_{Sgr}$, which might be detectable in future spectroscopic surveys. △ Less

Submitted 7 July, 2021; originally announced July 2021.

Journal ref: ApJ 928 80 (2022)

The unpopular Package: a Data-driven Approach to De-trend TESS Full Frame Image Light Curves

Authors: Soichiro Hattori, Daniel Foreman-Mackey, David W. Hogg, Benjamin T. Montet, Ruth Angus, T. A. Pritchard, Jason L. Curtis, Bernhard Schölkopf

Abstract: The majority of observed pixels on the Transiting Exoplanet Survey Satellite (TESS) are delivered in the form of full frame images (FFI). However, the FFIs contain systematic effects such as pointing jitter and scattered light from the Earth and Moon that must be removed before downstream analysis. We present unpopular, an open-source Python package to de-trend TESS FFI light curves based on the c… ▽ More The majority of observed pixels on the Transiting Exoplanet Survey Satellite (TESS) are delivered in the form of full frame images (FFI). However, the FFIs contain systematic effects such as pointing jitter and scattered light from the Earth and Moon that must be removed before downstream analysis. We present unpopular, an open-source Python package to de-trend TESS FFI light curves based on the causal pixel model method. Under the assumption that shared flux variations across multiple distant pixels are likely to be systematics, unpopular removes these common (i.e., popular) trends by modeling the systematics in a given pixel's light curve as a linear combination of light curves from many other distant pixels. To prevent overfitting we employ ridge regression and a train-and-test framework where the data points being de-trended are separated from those used to obtain the model coefficients. We also allow for simultaneous fitting with a polynomial model to capture any long-term astrophysical trends. We validate our method by de-trending different sources (e.g., supernova, tidal disruption event, exoplanet-hosting star, fast rotating star) and comparing our light curves to those obtained by other pipelines when appropriate. We also show that unpopular is able to preserve sector-length astrophysical signals, allowing for the extraction of multi-sector light curves from the FFI data. The unpopular source code and tutorials are freely available online. △ Less

Submitted 4 April, 2022; v1 submitted 28 June, 2021; originally announced June 2021.

Comments: 35 pages, 13 figures, accepted for publication in The Astronomical Journal

Selection Functions in Astronomical Data Modeling, with the Space Density of White Dwarfs as Worked Example

Authors: Hans-Walter Rix, David W. Hogg, Douglas Boubert, Anthony G. A. Brown, Andrew Casey, Ronald Drimmel, Andrew Everall, Morgan Fouesneau, Adrian M. Price-Whelan

Abstract: Statistical studies of astronomical data sets, in particular of cataloged properties for discrete objects, are central to astrophysics. One cannot model those objects' population properties or incidences without a quantitative understanding of the conditions under which these objects ended up in a catalog or sample, the sample's selection function. As systematic and didactic introductions to this… ▽ More Statistical studies of astronomical data sets, in particular of cataloged properties for discrete objects, are central to astrophysics. One cannot model those objects' population properties or incidences without a quantitative understanding of the conditions under which these objects ended up in a catalog or sample, the sample's selection function. As systematic and didactic introductions to this topic are scarce in the astrophysical literature, we aim to provide one, addressing generically the following questions: What is a selection function? What arguments $\vec{q}$ should a selection function depend on? Over what domain must a selection function be defined? What approximations and simplifications can be made? And, how is a selection function used in `modelling'? We argue that volume-complete samples, with the volume drastically curtailed by the faintest objects, reflect a highly sub-optimal selection function that needlessly reduces the number of bright and usually rare objects in the sample. We illustrate these points by a worked example, deriving the space density of white dwarfs (WD) in the Galactic neighbourhood as a function of their luminosity and Gaia color, $Φ_0(M_G,B-R)$ in [mag$^{-2}$pc$^{-3}$]. We construct a sample of $10^5$ presumed WDs through straightforward selection cuts on the Gaia EDR3 catalog, in magnitude, color, parallax, and astrometric fidelity $\vec{q}=(m_G,B-R,\varpi,p_{af})$. We then combine a simple model for $Φ_0$ with the effective survey volume derived from this selection function $S_C(\vec{q})$ to derive a detailed and robust estimate of $Φ_0(M_G,B-R)$. This resulting white dwarf luminosity-color function $Φ_0(M_G,B-R)$ differs dramatically from the initial number density distribution in the luminosity-color plane: by orders of magnitude in density and by four magnitudes in density peak location. △ Less

Submitted 14 June, 2021; originally announced June 2021.

Journal ref: ApJ 2021

Scalars are universal: Equivariant machine learning, structured like classical physics

Authors: Soledad Villar, David W. Hogg, Kate Storey-Fisher, Weichi Yao, Ben Blum-Smith

Abstract: There has been enormous progress in the last few years in designing neural networks that respect the fundamental symmetries and coordinate freedoms of physical law. Some of these frameworks make use of irreducible representations, some make use of high-order tensor objects, and some apply symmetry-enforcing constraints. Different physical laws obey different combinations of fundamental symmetries,… ▽ More There has been enormous progress in the last few years in designing neural networks that respect the fundamental symmetries and coordinate freedoms of physical law. Some of these frameworks make use of irreducible representations, some make use of high-order tensor objects, and some apply symmetry-enforcing constraints. Different physical laws obey different combinations of fundamental symmetries, but a large fraction (possibly all) of classical physics is equivariant to translation, rotation, reflection (parity), boost (relativity), and permutations. Here we show that it is simple to parameterize universally approximating polynomial functions that are equivariant under these symmetries, or under the Euclidean, Lorentz, and Poincaré groups, at any dimensionality $d$. The key observation is that nonlinear O($d$)-equivariant (and related-group-equivariant) functions can be universally expressed in terms of a lightweight collection of scalars -- scalar products and scalar contractions of the scalar, vector, and tensor inputs. We complement our theory with numerical examples that show that the scalar-based method is simple, efficient, and scalable. △ Less

Submitted 7 February, 2023; v1 submitted 11 June, 2021; originally announced June 2021.

Comments: NeurIPS 2021

Journal ref: Advances in Neural Information Processing Systems, 34, 28848-28863. 2021

Map** stellar surfaces I: Degeneracies in the rotational light curve problem

Authors: Rodrigo Luger, Daniel Foreman-Mackey, Christina Hedges, David W. Hogg

Abstract: Thanks to missions like Kepler and TESS, we now have access to tens of thousands of high precision, fast cadence, and long baseline stellar photometric observations. In principle, these light curves encode a vast amount of information about stellar variability and, in particular, about the distribution of starspots and other features on their surfaces. Unfortunately, the problem of inferring stell… ▽ More Thanks to missions like Kepler and TESS, we now have access to tens of thousands of high precision, fast cadence, and long baseline stellar photometric observations. In principle, these light curves encode a vast amount of information about stellar variability and, in particular, about the distribution of starspots and other features on their surfaces. Unfortunately, the problem of inferring stellar surface properties from a rotational light curve is famously ill-posed, as it often does not admit a unique solution. Inference about the number, size, contrast, and location of spots can therefore depend very strongly on the assumptions of the model, the regularization scheme, or the prior. The goal of this paper is twofold: (1) to explore the various degeneracies affecting the stellar light curve "inversion" problem and their effect on what can and cannot be learned from a stellar surface given unresolved photometric measurements; and (2) to motivate ensemble analyses of the light curves of many stars at once as a powerful data-driven alternative to common priors adopted in the literature. We further derive novel results on the dependence of the null space on stellar inclination and limb darkening and show that single-band photometric measurements cannot uniquely constrain quantities like the total spot coverage without the use of strong priors. This is the first in a series of papers devoted to the development of novel algorithms and tools for the analysis of stellar light curves and spectral time series, with the explicit goal of enabling statistically robust inference about their surface properties. △ Less

Submitted 9 February, 2021; v1 submitted 29 January, 2021; originally announced February 2021.

Comments: 33 pages, 12 figures. To be submitted to AAS Journals. Updated references

Fitting very flexible models: Linear regression with large numbers of parameters

Authors: David W. Hogg, Soledad Villar

Abstract: There are many uses for linear fitting; the context here is interpolation and denoising of data, as when you have calibration data and you want to fit a smooth, flexible function to those data. Or you want to fit a flexible function to de-trend a time series or normalize a spectrum. In these contexts, investigators often choose a polynomial basis, or a Fourier basis, or wavelets, or something equa… ▽ More There are many uses for linear fitting; the context here is interpolation and denoising of data, as when you have calibration data and you want to fit a smooth, flexible function to those data. Or you want to fit a flexible function to de-trend a time series or normalize a spectrum. In these contexts, investigators often choose a polynomial basis, or a Fourier basis, or wavelets, or something equally general. They also choose an order, or number of basis functions to fit, and (often) some kind of regularization. We discuss how this basis-function fitting is done, with ordinary least squares and extensions thereof. We emphasize that it is often valuable to choose far more parameters than data points, despite folk rules to the contrary: Suitably regularized models with enormous numbers of parameters generalize well and make good predictions for held-out data; over-fitting is not (mainly) a problem of having too many parameters. It is even possible to take the limit of infinite parameters, at which, if the basis and regularization are chosen correctly, the least-squares fit becomes the mean of a Gaussian process. We recommend cross-validation as a good empirical method for model selection (for example, setting the number of parameters and the form of the regularization), and jackknife resampling as a good empirical method for estimating the uncertainties of the predictions made by the model. We also give advice for building stable computational implementations. △ Less

Submitted 15 January, 2021; originally announced January 2021.

Comments: all code used to make the figures is available at https://github.com/davidwhogg/FlexibleLinearModels

Principled point-source detection in collections of astronomical images

Authors: Dustin Lang, David W. Hogg

Abstract: We review the well-known matched filter method for the detection of point sources in astronomical images. This is shown to be optimal (that is, to saturate the Cramer--Rao bound) under stated conditions that are very strong: an isolated source in background-dominated imaging with perfectly known background level, point-spread function, and noise models. We show that the matched filter produces a m… ▽ More We review the well-known matched filter method for the detection of point sources in astronomical images. This is shown to be optimal (that is, to saturate the Cramer--Rao bound) under stated conditions that are very strong: an isolated source in background-dominated imaging with perfectly known background level, point-spread function, and noise models. We show that the matched filter produces a maximum-likelihood estimate of the brightness of a purported point source, and this leads to a simple way to combine multiple images---taken through the same bandpass filter but with different noise levels and point-spread functions---to produce an optimal point source detection map. We then extend the approach to images taken through different bandpass filters, introducing the SED-matched filter, which allows us to combine images taken through different filters, but requires us to specify the colors of the objects we wish to detect. We show that this approach is superior to some methods traditionally employed, and that other traditional methods can be seen as instances of SED-matched filtering with implied (and often unreasonable) priors. We present a Bayesian formulation, including a flux prior that leads to a closed-form expression with low computational cost. △ Less

Submitted 31 December, 2020; originally announced December 2020.

Orbital Torus Imaging: Using Element Abundances to Map Orbits and Mass in the Milky Way

Authors: Adrian M. Price-Whelan, David W. Hogg, Kathryn V. Johnston, Melissa K. Ness, Hans-Walter Rix, Rachael L. Beaton, Joel R. Brownstein, Domingo Aníbal García-Hernández, Sten Hasselquist, Christian R. Hayes, Richard R. Lane, Gail Zasowski

Abstract: Many approaches to galaxy dynamics assume that the gravitational potential is simple and the distribution function is time-invariant. Under these assumptions there are traditional tools for inferring potential parameters given observations of stellar kinematics (e.g., Jeans models). However, spectroscopic surveys measure many stellar properties beyond kinematics. Here we present a new approach for… ▽ More Many approaches to galaxy dynamics assume that the gravitational potential is simple and the distribution function is time-invariant. Under these assumptions there are traditional tools for inferring potential parameters given observations of stellar kinematics (e.g., Jeans models). However, spectroscopic surveys measure many stellar properties beyond kinematics. Here we present a new approach for dynamical inference, Orbital Torus Imaging, which makes use of kinematic measurements and element abundances (or other invariant labels). We exploit the fact that, in steady state, stellar labels vary systematically with orbit characteristics (actions), yet must be invariant with respect to orbital phases (conjugate angles). The orbital foliation of phase space must therefore coincide with surfaces along which all moments of all stellar label distributions are constant. Both classical-statistics and Bayesian methods can be built on this; these methods will be more robust and require fewer assumptions than traditional tools because they require no knowledge of the (spatial) survey selection function and they do not involve second moments of velocity distributions. We perform a classical-statistics demonstration with red giant branch stars from the APOGEE surveys: We model the vertical orbit structure in the Milky Way disk to constrain the local disk mass, scale height, and the disk--halo mass ratio (at fixed local circular velocity). We find that the disk mass can be constrained (naïvely) at the few-percent level with Orbital Torus Imaging using only eight element-abundance ratios, demonstrating the promise of combining stellar labels with dynamical invariants. △ Less

Submitted 29 March, 2021; v1 submitted 30 November, 2020; originally announced December 2020.

Comments: 34 pages, 8 figures. Published in ApJ

Journal ref: The Astrophysical Journal, Volume 910, Issue 1, id.17, 16 pp., 2021

TRAP: A temporal systematics model for improved direct detection of exoplanets at small angular separations

Authors: M. Samland, J. Bouwman, D. W. Hogg, W. Brandner, T. Henning, M. Janson

Abstract: High-contrast imaging surveys for exoplanet detection have shown giant planets at large separations to be rare. It is important to push towards detections at smaller separations, the part of the parameter space containing most planets. The performance of traditional methods for post-processing of pupil-stabilized observations decreases at smaller separations, due to the larger field-rotation requi… ▽ More High-contrast imaging surveys for exoplanet detection have shown giant planets at large separations to be rare. It is important to push towards detections at smaller separations, the part of the parameter space containing most planets. The performance of traditional methods for post-processing of pupil-stabilized observations decreases at smaller separations, due to the larger field-rotation required to displace a source on the detector in addition to the intrinsic difficulty of higher stellar contamination. We developed a method of extracting exoplanet signals that improves performance at small angular separations. A data-driven model of the temporal behavior of the systematics for each pixel can be created using reference pixels at a different position, assuming the underlying causes of the systematics are shared across multiple pixels. This is mostly true for the speckle pattern in high-contrast imaging. In our causal regression model, we simultaneously fit the model of a planet signal "transiting" over detector pixels and non-local reference lightcurves describing a basis of shared temporal trends of the speckle pattern to find the best fitting temporal model describing the signal. With our implementation of a spatially non-local, temporal systematics model, called TRAP, we show that it is possible to gain up to a factor of 6 in contrast at close separations ($<3λ/D$) compared to a model based on spatial correlations between images displaced in time. We show that better temporal sampling resulting in significantly better contrasts. At short integration times for $β$ Pic data, we increase the SNR of the planet by a factor of 4 compared to the spatial systematics model. Finally, we show that the temporal model can be used on unaligned data which has only been dark and flat corrected, without the need for further pre-processing. △ Less

Submitted 14 March, 2021; v1 submitted 24 November, 2020; originally announced November 2020.

Comments: This paper has 21 pages of which 17 are the main body and 4 pages are appendix. 16 main figures and 4 figures in appendix

Journal ref: A&A 646, A24 (2021)

Dimensionality reduction, regularization, and generalization in overparameterized regressions

Authors: Ningyuan Huang, David W. Hogg, Soledad Villar

Abstract: Overparameterization in deep learning is powerful: Very large models fit the training data perfectly and yet often generalize well. This realization brought back the study of linear models for regression, including ordinary least squares (OLS), which, like deep learning, shows a "double-descent" behavior: (1) The risk (expected out-of-sample prediction error) can grow arbitrarily when the number o… ▽ More Overparameterization in deep learning is powerful: Very large models fit the training data perfectly and yet often generalize well. This realization brought back the study of linear models for regression, including ordinary least squares (OLS), which, like deep learning, shows a "double-descent" behavior: (1) The risk (expected out-of-sample prediction error) can grow arbitrarily when the number of parameters $p$ approaches the number of samples $n$, and (2) the risk decreases with $p$ for $p>n$, sometimes achieving a lower value than the lowest risk for $p<n$. The divergence of the risk for OLS can be avoided with regularization. In this work, we show that for some data models it can also be avoided with a PCA-based dimensionality reduction (PCA-OLS, also known as principal component regression). We provide non-asymptotic bounds for the risk of PCA-OLS by considering the alignments of the population and empirical principal components. We show that dimensionality reduction improves robustness while OLS is arbitrarily susceptible to adversarial attacks, particularly in the overparameterized regime. We compare PCA-OLS theoretically and empirically with a wide range of projection-based methods, including random projections, partial least squares (PLS), and certain classes of linear two-layer neural networks. These comparisons are made for different data generation models to assess the sensitivity to signal-to-noise and the alignment of regression coefficients with the features. We find that methods in which the projection depends on the training data can outperform methods where the projections are chosen independently of the training data, even those with oracle knowledge of population quantities, another seemingly paradoxical phenomenon that has been identified previously. This suggests that overparameterization may not be necessary for good generalization. △ Less

Submitted 19 October, 2021; v1 submitted 23 November, 2020; originally announced November 2020.

Journal ref: SIAM Journal on Mathematics of Data Science Vol.4 Iss.1, 2022

Two-point statistics without bins: A continuous-function generalization of the correlation function estimator for large-scale structure

Authors: Kate Storey-Fisher, David W. Hogg

Abstract: The two-point correlation function (2pcf) is the key statistic in structure formation; it measures the clustering of galaxies or other density field tracers. Estimators of the 2pcf, including the standard Landy-Szalay (LS) estimator, evaluate the 2pcf in hard-edged separation bins, which is scientifically inappropriate and results in a poor trade-off between bias and variance. We present a new 2pc… ▽ More The two-point correlation function (2pcf) is the key statistic in structure formation; it measures the clustering of galaxies or other density field tracers. Estimators of the 2pcf, including the standard Landy-Szalay (LS) estimator, evaluate the 2pcf in hard-edged separation bins, which is scientifically inappropriate and results in a poor trade-off between bias and variance. We present a new 2pcf estimator, the Continuous-Function Estimator, which generalizes LS to a continuous representation and obviates binning in separation or any other pair property. Our estimator, inspired by the mathematics of least-squares fitting, replaces binned pair counts with projections onto basis functions; it outputs the best linear combination of basis functions to describe the 2pcf. The choice of basis can take into account the expected form of the 2pcf, as well as its dependence on pair properties other than separation. We show that the Continuous-Function Estimator with a cubic-spline basis better represents the shape of the 2pcf compared to LS. We also estimate directly the baryon acoustic scale, using a small number of physically-motivated basis functions. Critically, this leads to a reduction in the number of mock catalogs required for covariance estimation, which is currently the limiting step in many 2pcf analyses. We discuss further applications of the Continuous-Function Estimator, including determination of the dependence of clustering on galaxy properties and searches for potential inhomogeneities or anisotropies in large-scale structure. △ Less

Submitted 3 February, 2021; v1 submitted 3 November, 2020; originally announced November 2020.

Comments: Accepted to ApJ

Excalibur: A Non-Parametric, Hierarchical Wavelength-Calibration Method for a Precision Spectrograph

Authors: L. L. Zhao, D. W. Hogg, M. Bedell, D. A. Fischer

Abstract: Excalibur is a non-parametric, hierarchical framework for precision wavelength-calibration of spectrographs. It is designed with the needs of extreme-precision radial velocity (EPRV) in mind, which require that instruments be calibrated or stabilized to better than $10^{-4}$ pixels. Instruments vary along only a few dominant degrees of freedom, especially EPRV instruments that feature highly stabi… ▽ More Excalibur is a non-parametric, hierarchical framework for precision wavelength-calibration of spectrographs. It is designed with the needs of extreme-precision radial velocity (EPRV) in mind, which require that instruments be calibrated or stabilized to better than $10^{-4}$ pixels. Instruments vary along only a few dominant degrees of freedom, especially EPRV instruments that feature highly stabilized optical systems and detectors. Excalibur takes advantage of this property by using all calibration data to construct a low-dimensional representation of all accessible calibration states for an instrument. Excalibur also takes advantage of laser frequency combs or etalons, which generate a dense set of stable calibration points. This density permits the use of a non-parametric wavelength solution that can adapt to any instrument or detector oddities better than parametric models, such as a polynomial. We demonstrate the success of this method with data from the EXtreme PREcision Spectrograph (EXPRES), which uses a laser frequency comb. When wavelengths are assigned to laser comb lines using excalibur, the RMS of the residuals is about five times lower than wavelengths assigned using polynomial fits to individual exposures. Radial-velocity measurements of HD 34411 showed a reduction in RMS scatter over a 10-month time baseline from $1.17$ to $1.05\, m\,s^{-1}$. △ Less

Submitted 26 October, 2020; originally announced October 2020.

Comments: 16 pages, 10 figures, presented here following first referee report

An unsupervised method for identifying $X$-enriched stars directly from spectra: Li in LAMOST

Authors: Adam Wheeler, Melissa Ness, David W. Hogg

Abstract: Stars with peculiar element abundances are important markers of chemical enrichment mechanisms. We present a simple method, tangent space projection (TSP), for the detection of $X$-enriched stars, for arbitrary elements $X$, even from blended lines. Our method does not require stellar labels, but instead directly estimates the counterfactual unrenriched spectrum from other unlabelled spectra. As a… ▽ More Stars with peculiar element abundances are important markers of chemical enrichment mechanisms. We present a simple method, tangent space projection (TSP), for the detection of $X$-enriched stars, for arbitrary elements $X$, even from blended lines. Our method does not require stellar labels, but instead directly estimates the counterfactual unrenriched spectrum from other unlabelled spectra. As a case study, we apply this method to the $6708~$Å Li doublet in LAMOST DR5, identifying 8,428 Li-enriched stars seamlessly across evolutionary state. We comment on the explanation for Li-enrichement for different subpopulations, including planet accretion, nonstandard mixing, and youth. △ Less

Submitted 8 March, 2021; v1 submitted 8 September, 2020; originally announced September 2020.

Comments: Published in ApJ; Data available at https://doi.org/10.7916/d8-3ap9-qe35

Journal ref: ApJ 908 247 (2021)

How to obtain the redshift distribution from probabilistic redshift estimates

Authors: Alex I. Malz, David W. Hogg

Abstract: A trustworthy estimate of the redshift distribution $n(z)$ is crucial for using weak gravitational lensing and large-scale structure of galaxy catalogs to study cosmology. Spectroscopic redshifts for the dim and numerous galaxies of next-generation weak-lensing surveys are expected to be unavailable, making photometric redshift (photo-$z$) probability density functions (PDFs) the next-best alterna… ▽ More A trustworthy estimate of the redshift distribution $n(z)$ is crucial for using weak gravitational lensing and large-scale structure of galaxy catalogs to study cosmology. Spectroscopic redshifts for the dim and numerous galaxies of next-generation weak-lensing surveys are expected to be unavailable, making photometric redshift (photo-$z$) probability density functions (PDFs) the next-best alternative for comprehensively encapsulating the nontrivial systematics affecting photo-$z$ point estimation. The established stacked estimator of $n(z)$ avoids reducing photo-$z$ PDFs to point estimates but yields a systematically biased estimate of $n(z)$ that worsens with decreasing signal-to-noise, the very regime where photo-$z$ PDFs are most necessary. We introduce Cosmological Hierarchical Inference with Probabilistic Photometric Redshifts (CHIPPR), a statistically rigorous probabilistic graphical model of redshift-dependent photometry, which correctly propagates the redshift uncertainty information beyond the best-fit estimator of $n(z)$ produced by traditional procedures and is provably the only self-consistent way to recover $n(z)$ from photo-$z$ PDFs. We present the $\texttt{chippr}$ prototype code, noting that the mathematically justifiable approach incurs computational expense. The CHIPPR approach is applicable to any one-point statistic of any random variable, provided the prior probability density used to produce the posteriors is explicitly known; if the prior is implicit, as may be the case for popular photo-$z$ techniques, then the resulting posterior PDFs cannot be used for scientific inference. We therefore recommend that the photo-$z$ community focus on develo** methodologies that enable the recovery of photo-$z$ likelihoods with support over all redshifts, either directly or via a known prior probability density. △ Less

Submitted 23 July, 2020; originally announced July 2020.

Comments: submitted to ApJ

The power of co-ordinate transformations in dynamical interpretations of Galactic structure

Authors: Jason A. S. Hunt, Kathryn V. Johnston, Alex R. Pettitt, Emily C. Cunningham, Daisuke Kawata, David W. Hogg

Abstract: $Gaia… ▽ More $Gaia$ DR2 has provided an unprecedented wealth of information about the positions and motions of stars in our Galaxy, and has highlighted the degree of disequilibria in the disc. As we collect data over a wider area of the disc it becomes increasingly appealing to start analysing stellar actions and angles, which specifically label orbit space, instead of their current phase space location. Conceptually, while $\bar{x}$ and $\bar{v}$ tell us about the potential and local interactions, grou** in action puts together stars that have similar frequencies and hence similar responses to dynamical effects occurring over several orbits. Grou** in actions and angles refines this further to isolate stars which are travelling together through space and hence have shared histories. Mixing these coordinate systems can confuse the interpretation. For example, it has been suggested that by moving stars to their guiding radius, the Milky Way spiral structure is visible as ridge-like overdensities in the $Gaia$ data \citep{Khoperskov+19b}. However, in this work, we show that these features are in fact the known kinematic moving groups, both in the $L_z-φ$ and the $v_{\mathrm{R}}-v_φ$ planes. Using simulations we show how this distinction will become even more important as we move to a global view of the Milky Way. As an example, we show that the radial velocity wave seen in the Galactic disc in $Gaia$ and APOGEE should become stronger in the action-angle frame, and that it can be reproduced by transient spiral structure. △ Less

Submitted 5 June, 2020; originally announced June 2020.

Comments: 12 pages, 10 Figures

Data Analysis Recipes: Products of multivariate Gaussians in Bayesian inferences

Authors: David W. Hogg, Adrian M. Price-Whelan, Boris Leistedt

Abstract: A product of two Gaussians (or normal distributions) is another Gaussian. That's a valuable and useful fact! Here we use it to derive a refactoring of a common product of multivariate Gaussians: The product of a Gaussian likelihood times a Gaussian prior, where some or all of those parameters enter the likelihood only in the mean and only linearly. That is, a linear, Gaussian, Bayesian model. This… ▽ More A product of two Gaussians (or normal distributions) is another Gaussian. That's a valuable and useful fact! Here we use it to derive a refactoring of a common product of multivariate Gaussians: The product of a Gaussian likelihood times a Gaussian prior, where some or all of those parameters enter the likelihood only in the mean and only linearly. That is, a linear, Gaussian, Bayesian model. This product of a likelihood times a prior pdf can be refactored into a product of a marginalized likelihood (or a Bayesian evidence) times a posterior pdf, where (in this case) both of these are also Gaussian. The means and variance tensors of the refactored Gaussians are straightforward to obtain as closed-form expressions; here we deliver these expressions, with discussion. The closed-form expressions can be used to speed up and improve the precision of inferences that contain linear parameters with Gaussian priors. We connect these methods to inferences that arise frequently in physics and astronomy. If all you want is the answer, the question is posed and answered at the beginning of Section 3. We show two toy examples, in the form of worked exercises, in Section 4. The solutions, discussion, and exercises in this Note are aimed at someone who is already familiar with the basic ideas of Bayesian inference and probability. △ Less

Submitted 28 May, 2020; originally announced May 2020.

Comments: a chapter of a book we will never write