-
Anomaly Detection and Approximate Similarity Searches of Transients in Real-time Data Streams
Authors:
P. D. Aleo,
A. W. Engel,
G. Narayan,
C. R. Angus,
K. Malanchev,
K. Auchettl,
V. F. Baldassare,
A. Berres,
T. J. L. de Boer,
B. M. Boyd,
K. C. Chambers,
K. W. Davis,
N. Esquivel,
D. Farias,
R. J. Foley,
A. Gagliano,
C. Gall,
H. Gao,
S. Gomez,
M. Grayling,
C. -C. Lin,
E. A. Magnier,
K. S. Mandel,
T. Matheson,
S. I. Raimundo
, et al. (5 additional authors not shown)
Abstract:
We present LAISS (Lightcurve Anomaly Identification and Similarity Search), an automated pipeline to detect anomalous astrophysical transients in real-time data streams. We deploy our anomaly detection model on the nightly ZTF Alert Stream via the ANTARES broker, identifying a manageable $\sim$1-5 candidates per night for expert vetting and coordinating follow-up observations. Our method leverages…
▽ More
We present LAISS (Lightcurve Anomaly Identification and Similarity Search), an automated pipeline to detect anomalous astrophysical transients in real-time data streams. We deploy our anomaly detection model on the nightly ZTF Alert Stream via the ANTARES broker, identifying a manageable $\sim$1-5 candidates per night for expert vetting and coordinating follow-up observations. Our method leverages statistical light-curve and contextual host-galaxy features within a random forest classifier, tagging transients of rare classes (spectroscopic anomalies), of uncommon host-galaxy environments (contextual anomalies), and of peculiar or interaction-powered phenomena (behavioral anomalies). Moreover, we demonstrate the power of a low-latency ($\sim$ms) approximate similarity search method to find transient analogs with similar light-curve evolution and host-galaxy environments. We use analogs for data-driven discovery, characterization, (re-)classification, and imputation in retrospective and real-time searches. To date we have identified $\sim$50 previously known and previously missed rare transients from real-time and retrospective searches, including but not limited to: SLSNe, TDEs, SNe IIn, SNe IIb, SNe Ia-CSM, SNe Ia-91bg-like, SNe Ib, SNe Ic, SNe Ic-BL, and M31 novae. Lastly, we report the discovery of 325 total transients, all observed between 2018-2021 and absent from public catalogs ($\sim$1% of all ZTF Astronomical Transient reports to the Transient Name Server through 2021). These methods enable a systematic approach to finding the "needle in the haystack" in large-volume data streams. Because of its integration with the ANTARES broker, LAISS is built to detect exciting transients in Rubin data.
△ Less
Submitted 1 April, 2024;
originally announced April 2024.
-
Superphot+: Realtime Fitting and Classification of Supernova Light Curves
Authors:
Kaylee M. de Soto,
Ashley Villar,
Edo Berger,
Sebastian Gomez,
Griffin Hosseinzadeh,
Doug Branton,
Sandro Campos,
Melissa DeLucchi,
Jeremy Kubica,
Olivia Lynn,
Konstantin Malanchev,
Alex I. Malz
Abstract:
Photometric classifications of supernova (SN) light curves have become necessary to utilize the full potential of large samples of observations obtained from wide-field photometric surveys, such as the Zwicky Transient Facility (ZTF) and the Vera C. Rubin Observatory. Here, we present a photometric classifier for SN light curves that does not rely on redshift information and still maintains compar…
▽ More
Photometric classifications of supernova (SN) light curves have become necessary to utilize the full potential of large samples of observations obtained from wide-field photometric surveys, such as the Zwicky Transient Facility (ZTF) and the Vera C. Rubin Observatory. Here, we present a photometric classifier for SN light curves that does not rely on redshift information and still maintains comparable accuracy to redshift-dependent classifiers. Our new package, Superphot+, uses a parametric model to extract meaningful features from multiband SN light curves. We train a gradient-boosted machine with fit parameters from 6,061 ZTF SNe that pass data quality cuts and are spectroscopically classified as one of five classes: SN Ia, SN II, SN Ib/c, SN IIn, and SLSN-I. Without redshift information, our classifier yields a class-averaged F1-score of 0.61 +/- 0.02 and a total accuracy of 0.83 +/- 0.01. Including redshift information improves these metrics to 0.71 +/- 0.02 and 0.88 +/- 0.01, respectively. We assign new class probabilities to 3,558 ZTF transients that show SN-like characteristics (based on the ALeRCE Broker light curve and stamp classifiers), but lack spectroscopic classifications. Finally, we compare our predicted SN labels with those generated by the ALeRCE light curve classifier, finding that the two classifiers agree on photometric labels for 82 +/- 2% of light curves with spectroscopic labels and 72% of light curves without spectroscopic labels. Superphot+ is currently classifying ZTF SNe in real time via the ANTARES Broker, and is designed for simple adaptation to six-band Rubin light curves in the future.
△ Less
Submitted 12 March, 2024;
originally announced March 2024.
-
Hierarchical Cross-entropy Loss for Classification of Astrophysical Transients
Authors:
V. Ashley Villar,
Kaylee de Soto,
Alex Gagliano
Abstract:
Astrophysical transient phenomena are traditionally classified spectroscopically in a hierarchical taxonomy; however, this graph structure is currently not utilized in neural net-based photometric classifiers for time-domain astrophysics. Instead, independent classifiers are trained for different tiers of classified data, and events are excluded if they fall outside of these well-defined but flat…
▽ More
Astrophysical transient phenomena are traditionally classified spectroscopically in a hierarchical taxonomy; however, this graph structure is currently not utilized in neural net-based photometric classifiers for time-domain astrophysics. Instead, independent classifiers are trained for different tiers of classified data, and events are excluded if they fall outside of these well-defined but flat classification schemes. Here, we introduce a weighted hierarchical cross-entropy objective function for classification of astrophysical transients. Our method allows users to directly build and use physics- or observationally-motivated tree-based taxonomies. Our weighted hierarchical cross-entropy loss directly uses this graph to accurately classify all targets into any node of the tree, re-weighting imbalanced classes. We test our novel loss on a set of variable stars and extragalactic transients from the Zwicky Transient Facility, showing that we can achieve similar performance to fine-tuned classifiers with the advantage of notably more flexibility in downstream classification tasks.
△ Less
Submitted 4 December, 2023;
originally announced December 2023.
-
The LIGO HET Response (LIGHETR) Project to Discover and Spectroscopically Follow Optical Transients Associated with Neutron Star Mergers
Authors:
M. J. Bustamante-Rosell,
Greg Zeimann,
J. Craig Wheeler,
Karl Gebhardt,
Aaron Zimmerman,
Chris Fryer,
Oleg Korobkin,
Richard Matzner,
V. Ashley Villar,
S. Karthik Yadavalli,
Kaylee M. de Soto,
Matthew Shetrone,
Steven Janowiecki,
Pawan Kumar,
David Pooley,
Benjamin P. Thomas,
Hsin-Yu Chen,
Lifan Wang,
Jozsef Vinko,
David J. Sand,
Ryan Wollaeger,
Frederic V. Hessman,
Kristen B. McQuinn
Abstract:
The LIGO HET Response (LIGHETR) project is an enterprise to follow up optical transients (OT) discovered as gravitational wave merger sources by the LIGO/Virgo collaboration (LVC). Early spectroscopy has the potential to constrain crucial parameters such as the aspect angle. The LIGHETR collaboration also includes the capacity to model the spectroscopic evolution of mergers to facilitate a real-ti…
▽ More
The LIGO HET Response (LIGHETR) project is an enterprise to follow up optical transients (OT) discovered as gravitational wave merger sources by the LIGO/Virgo collaboration (LVC). Early spectroscopy has the potential to constrain crucial parameters such as the aspect angle. The LIGHETR collaboration also includes the capacity to model the spectroscopic evolution of mergers to facilitate a real-time direct comparison of models with our data. The principal facility is the Hobby-Eberly Telescope. LIGHETR uses the massively-replicated VIRUS array of spectrographs to search for associated OTs and obtain early blue spectra and in a complementary role, the low-resolution LRS-2 spectrograph is used to obtain spectra of viable candidates as well as a densely-sampled series of spectra of true counterparts. Once an OT is identified, the anticipated cadence of spectra would match or considerably exceed anything achieved for GW170817 = AT2017gfo for which there were no spectra in the first 12 hours and thereafter only roughly once daily. We describe special HET-specific software written to facilitate the program and attempts to determine the flux limits to undetected sources. We also describe our campaign to follow up OT candidates during the third observational campaign of the LIGO and Virgo Scientific Collaborations. We obtained VIRUS spectroscopy of candidate galaxy hosts for 5 LVC gravitational wave events and LRS-2 spectra of one candidate for the OT associated with S190901ap. We identified that candidate, ZTF19abvionh = AT2019pip, as a possible Wolf-Rayet star in an otherwise unrecognized nearby dwarf galaxy.
△ Less
Submitted 27 June, 2023;
originally announced June 2023.
-
The Young Supernova Experiment Data Release 1 (YSE DR1): Light Curves and Photometric Classification of 1975 Supernovae
Authors:
P. D. Aleo,
K. Malanchev,
S. Sharief,
D. O. Jones,
G. Narayan,
R. J. Foley,
V. A. Villar,
C. R. Angus,
V. F. Baldassare,
M. J. Bustamante-Rosell,
D. Chatterjee,
C. Cold,
D. A. Coulter,
K. W. Davis,
S. Dhawan,
M. R. Drout,
A. Engel,
K. D. French,
A. Gagliano,
C. Gall,
J. Hjorth,
M. E. Huber,
W. V. Jacobson-Galán,
C. D. Kilpatrick,
D. Langeroodi
, et al. (58 additional authors not shown)
Abstract:
We present the Young Supernova Experiment Data Release 1 (YSE DR1), comprised of processed multi-color Pan-STARRS1 (PS1) griz and Zwicky Transient Facility (ZTF) gr photometry of 1975 transients with host-galaxy associations, redshifts, spectroscopic/photometric classifications, and additional data products from 2019 November 24 to 2021 December 20. YSE DR1 spans discoveries and observations from…
▽ More
We present the Young Supernova Experiment Data Release 1 (YSE DR1), comprised of processed multi-color Pan-STARRS1 (PS1) griz and Zwicky Transient Facility (ZTF) gr photometry of 1975 transients with host-galaxy associations, redshifts, spectroscopic/photometric classifications, and additional data products from 2019 November 24 to 2021 December 20. YSE DR1 spans discoveries and observations from young and fast-rising supernovae (SNe) to transients that persist for over a year, with a redshift distribution reaching z~0.5. We present relative SN rates from YSE's magnitude- and volume-limited surveys, which are consistent with previously published values within estimated uncertainties for untargeted surveys. We combine YSE and ZTF data, and create multi-survey SN simulations to train the ParSNIP and SuperRAENN photometric classification algorithms; when validating our ParSNIP classifier on 472 spectroscopically classified YSE DR1 SNe, we achieve 82% accuracy across three SN classes (SNe Ia, II, Ib/Ic) and 90% accuracy across two SN classes (SNe Ia, core-collapse SNe). Our classifier performs particularly well on SNe Ia, with high (>90%) individual completeness and purity, which will help build an anchor photometric SNe Ia sample for cosmology. We then use our photometric classifier to characterize our photometric sample of 1483 SNe, labeling 1048 (~71%) SNe Ia, 339 (~23%) SNe II, and 96 (~6%) SNe Ib/Ic. YSE DR1 provides a training ground for building discovery, anomaly detection, and classification algorithms, performing cosmological analyses, understanding the nature of red and rare transients, exploring tidal disruption events and nuclear variability, and preparing for the forthcoming Vera C. Rubin Observatory Legacy Survey of Space and Time.
△ Less
Submitted 21 February, 2023; v1 submitted 14 November, 2022;
originally announced November 2022.
-
Relative intrinsic scatter in hierarchical Type Ia supernova siblings analyses: Application to SNe 2021hpr, 1997bq & 2008fv in NGC 3147
Authors:
Sam M. Ward,
Stephen Thorp,
Kaisey S. Mandel,
Suhail Dhawan,
David O. Jones,
Kirsty Taggart,
Ryan J. Foley,
Gautham Narayan,
Kenneth C. Chambers,
David A. Coulter,
Kyle W. Davis,
Thomas de Boer,
Kaylee de Soto,
Nicholas Earl,
Alex Gagliano,
Hua Gao,
Jens Hjorth,
Mark E. Huber,
Luca Izzo,
Danial Langeroodi,
Eugene A. Magnier,
Peter McGill,
Armin Rest,
César Rojas-Bravo,
Radosław Wojtak
Abstract:
We present Young Supernova Experiment $grizy$ photometry of SN 2021hpr, the third Type Ia supernova sibling to explode in the Cepheid calibrator galaxy, NGC 3147. Siblings are useful for improving SN-host distance estimates, and investigating the contributions towards the SN Ia intrinsic scatter (post-standardisation residual scatter in distance estimates). We thus develop a principled Bayesian fr…
▽ More
We present Young Supernova Experiment $grizy$ photometry of SN 2021hpr, the third Type Ia supernova sibling to explode in the Cepheid calibrator galaxy, NGC 3147. Siblings are useful for improving SN-host distance estimates, and investigating the contributions towards the SN Ia intrinsic scatter (post-standardisation residual scatter in distance estimates). We thus develop a principled Bayesian framework for analyzing SN Ia siblings. At its core is the cosmology-independent relative intrinsic scatter parameter, $σ_{Rel}$: the dispersion of siblings distance estimates relative to one another within a galaxy. It quantifies the contribution towards the total intrinsic scatter, $σ_0$, from within-galaxy variations about the siblings' common properties. It also affects the combined-distance uncertainty. We present analytic formulae for computing a $σ_{Rel}$-posterior from individual siblings distances (estimated using any SN-model). Applying a newly trained BayeSN model, we fit the light curves of each sibling in NGC 3147 individually, to yield consistent distance estimates. However, the wide $σ_{Rel}$-posterior means $σ_{Rel}\approxσ_0$ is not ruled out. We thus combine the distances by marginalizing over $σ_{Rel}$ with an informative prior: $σ_{Rel}\sim U(0,σ_0)$. Simultaneously fitting the trio's light curves improves constraints on distance, and each sibling's individual dust parameters, compared to individual fits. Higher correlation also tightens dust parameter constraints. Therefore, $σ_{Rel}$-marginalization yields robust estimates of siblings distances for cosmology, and dust parameters for siblings-host correlation studies. Incorporating NGC 3147's Cepheid-distance yields $H_0=78.4\pm 6.5\,$km/s/Mpc. Our work motivates analyses of homogeneous siblings samples, to constrain $σ_{Rel}$, and its SN-model dependence.
△ Less
Submitted 1 September, 2023; v1 submitted 21 September, 2022;
originally announced September 2022.
-
Tracing Milky Way substructure with an RR Lyrae hierarchical clustering forest
Authors:
Brian T. Cook,
Deborah F. Woods,
Jessica D. Ruprecht,
Jacob Varey,
Radha Mastandrea,
Kaylee de Soto,
Jacob F. Harburg,
Umaa Rebbapragada,
Ashish A. Mahabal
Abstract:
RR Lyrae variable stars have long been reliable standard candles used to discern structure in the Local Group. With this in mind, we present a routine to identify grou**s containing a statistically significant number of RR Lyrae variables in the Milky Way environment. RR Lyrae variable grou**s, or substructures, with potential Galactic archaeology applications are found using a forest of agglo…
▽ More
RR Lyrae variable stars have long been reliable standard candles used to discern structure in the Local Group. With this in mind, we present a routine to identify grou**s containing a statistically significant number of RR Lyrae variables in the Milky Way environment. RR Lyrae variable grou**s, or substructures, with potential Galactic archaeology applications are found using a forest of agglomerative, hierarchical clustering trees, whose leaves are Milky Way RR Lyrae variables. Each grou** is validated by ensuring that the internal RR Lyrae variable proper motions are sufficiently correlated. Photometric information was collected from the Gaia second data release and proper motions from the (early) third data release. After applying this routine to the catalogue of 91234 variables, we are able to report sixteen unique RR Lyrae substructures with physical sizes of less than 1 kpc. Five of these substructures are in close proximity to Milky Way globular clusters with previously known tidal tails and/or a potential connection to Galactic merger events. One candidate substructure is in the neighbourhood of the Large Magellanic Cloud but is more distant (and older) than known satellites of the dwarf galaxy. Our study ends with a discussion of ways in which future surveys could be applied to the discovery of Milky Way stellar streams.
△ Less
Submitted 12 April, 2022;
originally announced April 2022.
-
What Makes Quadruply Lensed Quasars Quadruple?
Authors:
Richard Luhtaru,
Paul L. Schechter,
Kaylee M. de Soto
Abstract:
Among known strongly lensed quasar systems, ~25% have gravitational potentials sufficiently flat (and sources sufficiently well aligned) to produce four images rather than two. The projected flattening of the lensing galaxy and tides from neighboring galaxies both contribute to the potential's quadrupole. Witt's hyperbola and Wynne's ellipse permit determination of the overall quadrupole from the…
▽ More
Among known strongly lensed quasar systems, ~25% have gravitational potentials sufficiently flat (and sources sufficiently well aligned) to produce four images rather than two. The projected flattening of the lensing galaxy and tides from neighboring galaxies both contribute to the potential's quadrupole. Witt's hyperbola and Wynne's ellipse permit determination of the overall quadrupole from the positions of the quasar images. The position of the lensing galaxy resolves the distinct contributions of intrinsic ellipticity and tidal shear to that quadrupole. Among 31 quadruply lensed quasars systems with statistically significant decompositions, 15 are either reliably ($2σ$) or provisionally ($1σ$) shear-dominated and 11 are either reliably or provisionally ellipticity-dominated. For the remaining 8, the two effects make roughly equal contributions to the combined cross section (newly derived here) for quadruple lensing. This observational result is strongly at variance with the ellipticity-dominated forecast of Oguri & Marshall (2010).
△ Less
Submitted 28 April, 2021; v1 submitted 16 February, 2021;
originally announced February 2021.
