-
Anticipatory Music Transformer
Authors:
John Thickstun,
David Hall,
Chris Donahue,
Percy Liang
Abstract:
We introduce anticipation: a method for constructing a controllable generative model of a temporal point process (the event process) conditioned asynchronously on realizations of a second, correlated process (the control process). We achieve this by interleaving sequences of events and controls, such that controls appear following stop** times in the event sequence. This work is motivated by pro…
▽ More
We introduce anticipation: a method for constructing a controllable generative model of a temporal point process (the event process) conditioned asynchronously on realizations of a second, correlated process (the control process). We achieve this by interleaving sequences of events and controls, such that controls appear following stop** times in the event sequence. This work is motivated by problems arising in the control of symbolic music generation. We focus on infilling control tasks, whereby the controls are a subset of the events themselves, and conditional generation completes a sequence of events given the fixed control events. We train anticipatory infilling models using the large and diverse Lakh MIDI music dataset. These models match the performance of autoregressive models for prompted music generation, with the additional capability to perform infilling control tasks, including accompaniment. Human evaluators report that an anticipatory model produces accompaniments with similar musicality to even music composed by humans over a 20-second clip.
△ Less
Submitted 14 June, 2023;
originally announced June 2023.
-
Learning the Night Sky with Deep Generative Priors
Authors:
Fausto Navarro,
Daniel Hall,
Tamas Budavari,
Yashil Sukurdeep
Abstract:
Recovering sharper images from blurred observations, referred to as deconvolution, is an ill-posed problem where classical approaches often produce unsatisfactory results. In ground-based astronomy, combining multiple exposures to achieve images with higher signal-to-noise ratios is complicated by the variation of point-spread functions across exposures due to atmospheric effects. We develop an un…
▽ More
Recovering sharper images from blurred observations, referred to as deconvolution, is an ill-posed problem where classical approaches often produce unsatisfactory results. In ground-based astronomy, combining multiple exposures to achieve images with higher signal-to-noise ratios is complicated by the variation of point-spread functions across exposures due to atmospheric effects. We develop an unsupervised multi-frame method for denoising, deblurring, and coadding images inspired by deep generative priors. We use a carefully chosen convolutional neural network architecture that combines information from multiple observations, regularizes the joint likelihood over these observations, and allows us to impose desired constraints, such as non-negativity of pixel values in the sharp, restored image. With an eye towards the Rubin Observatory, we analyze 4K by 4K Hyper Suprime-Cam exposures and obtain preliminary results which yield promising restored images and extracted source lists.
△ Less
Submitted 3 February, 2023;
originally announced February 2023.
-
Tropical and Extratropical Cyclone Detection Using Deep Learning
Authors:
Christina Kumler-Bonfanti,
Jebb Stewart,
David Hall,
Mark Govett
Abstract:
Extracting valuable information from large sets of diverse meteorological data is a time-intensive process. Machine learning methods can help improve both speed and accuracy of this process. Specifically, deep learning image segmentation models using the U-Net structure perform faster and can identify areas missed by more restrictive approaches, such as expert hand-labeling and a priori heuristic…
▽ More
Extracting valuable information from large sets of diverse meteorological data is a time-intensive process. Machine learning methods can help improve both speed and accuracy of this process. Specifically, deep learning image segmentation models using the U-Net structure perform faster and can identify areas missed by more restrictive approaches, such as expert hand-labeling and a priori heuristic methods. This paper discusses four different state-of-the-art U-Net models designed for detection of tropical and extratropical cyclone Regions Of Interest (ROI) from two separate input sources: total precipitable water output from the Global Forecasting System (GFS) model and water vapor radiance images from the Geostationary Operational Environmental Satellite (GOES). These models are referred to as IBTrACS-GFS, Heuristic-GFS, IBTrACS-GOES, and Heuristic-GOES. All four U-Nets are fast information extraction tools and perform with a ROI detection accuracy ranging from 80% to 99%. These are additionally evaluated with the Dice and Tversky Intersection over Union (IoU) metrics, having Dice coefficient scores ranging from 0.51 to 0.76 and Tversky coefficients ranging from 0.56 to 0.74. The extratropical cyclone U-Net model performed 3 times faster than the comparable heuristic model used to detect the same ROI. The U-Nets were specifically selected for their capabilities in detecting cyclone ROI beyond the scope of the training labels. These machine learning models identified more ambiguous and active ROI missed by the heuristic model and hand-labeling methods commonly used in generating real-time weather alerts, having a potentially direct impact on public safety.
△ Less
Submitted 18 May, 2020;
originally announced May 2020.
