Skip to main content

Showing 1–17 of 17 results for author: Morningstar, W R

.
  1. arXiv:2312.01187  [pdf, other

    cs.CV cs.LG stat.ML

    SASSL: Enhancing Self-Supervised Learning via Neural Style Transfer

    Authors: Renan A. Rojas-Gomez, Karan Singhal, Ali Etemad, Alex Bijamov, Warren R. Morningstar, Philip Andrew Mansfield

    Abstract: Existing data augmentation in self-supervised learning, while diverse, fails to preserve the inherent structure of natural images. This results in distorted augmented samples with compromised semantic information, ultimately impacting downstream performance. To overcome this, we propose SASSL: Style Augmentations for Self Supervised Learning, a novel augmentation technique based on Neural Style Tr… ▽ More

    Submitted 3 February, 2024; v1 submitted 2 December, 2023; originally announced December 2023.

  2. arXiv:2311.03629  [pdf, other

    cs.CV cs.LG

    Random Field Augmentations for Self-Supervised Representation Learning

    Authors: Philip Andrew Mansfield, Arash Afkanpour, Warren Richard Morningstar, Karan Singhal

    Abstract: Self-supervised representation learning is heavily dependent on data augmentations to specify the invariances encoded in representations. Previous work has shown that applying diverse data augmentations is crucial to downstream performance, but augmentation techniques remain under-explored. In this work, we propose a new family of local transformations based on Gaussian random fields to generate i… ▽ More

    Submitted 6 November, 2023; originally announced November 2023.

    ACM Class: I.2.6; I.2.10; I.5.1

  3. arXiv:2309.05213  [pdf, other

    cs.LG cs.AI cs.DC

    Towards Federated Learning Under Resource Constraints via Layer-wise Training and Depth Dropout

    Authors: Pengfei Guo, Warren Richard Morningstar, Raviteja Vemulapalli, Karan Singhal, Vishal M. Patel, Philip Andrew Mansfield

    Abstract: Large machine learning models trained on diverse data have recently seen unprecedented success. Federated learning enables training on private data that may otherwise be inaccessible, such as domain-specific datasets decentralized across many clients. However, federated learning can be difficult to scale to large models when clients have limited resources. This challenge often results in a trade-o… ▽ More

    Submitted 10 September, 2023; originally announced September 2023.

  4. arXiv:2305.13672  [pdf, other

    cs.LG cs.DC

    Federated Variational Inference: Towards Improved Personalization and Generalization

    Authors: Elahe Vedadi, Joshua V. Dillon, Philip Andrew Mansfield, Karan Singhal, Arash Afkanpour, Warren Richard Morningstar

    Abstract: Conventional federated learning algorithms train a single global model by leveraging all participating clients' data. However, due to heterogeneity in client generative distributions and predictive models, these approaches may not appropriately approximate the predictive process, converge to an optimal state, or generalize to new clients. We study personalization and generalization in stateless cr… ▽ More

    Submitted 25 May, 2023; v1 submitted 23 May, 2023; originally announced May 2023.

    Comments: 16 pages, 6 figures

  5. arXiv:2210.00092  [pdf, other

    cs.LG cs.CV

    Federated Training of Dual Encoding Models on Small Non-IID Client Datasets

    Authors: Raviteja Vemulapalli, Warren Richard Morningstar, Philip Andrew Mansfield, Hubert Eichner, Karan Singhal, Arash Afkanpour, Bradley Green

    Abstract: Dual encoding models that encode a pair of inputs are widely used for representation learning. Many approaches train dual encoding models by maximizing agreement between pairs of encodings on centralized training data. However, in many scenarios, datasets are inherently decentralized across many clients (user devices or organizations) due to privacy concerns, motivating federated learning. In this… ▽ More

    Submitted 10 April, 2023; v1 submitted 30 September, 2022; originally announced October 2022.

    Comments: ICLR 2023 Workshop on Pitfalls of Limited Data and Computation for Trustworthy ML

  6. arXiv:2011.08711  [pdf, other

    stat.ML cs.LG

    VIB is Half Bayes

    Authors: Alexander A Alemi, Warren R Morningstar, Ben Poole, Ian Fischer, Joshua V Dillon

    Abstract: In discriminative settings such as regression and classification there are two random variables at play, the inputs X and the targets Y. Here, we demonstrate that the Variational Information Bottleneck can be viewed as a compromise between fully empirical and fully Bayesian objectives, attempting to minimize the risks due to finite sampling of Y only. We argue that this approach provides some of t… ▽ More

    Submitted 17 November, 2020; originally announced November 2020.

  7. arXiv:2010.09629  [pdf, other

    cs.LG stat.ML

    PAC$^m$-Bayes: Narrowing the Empirical Risk Gap in the Misspecified Bayesian Regime

    Authors: Warren R. Morningstar, Alexander A. Alemi, Joshua V. Dillon

    Abstract: The Bayesian posterior minimizes the "inferential risk" which itself bounds the "predictive risk". This bound is tight when the likelihood and prior are well-specified. However since misspecification induces a gap, the Bayesian posterior predictive distribution may have poor generalization performance. This work develops a multi-sample loss (PAC$^m$) which can close the gap by spanning a trade-off… ▽ More

    Submitted 23 May, 2022; v1 submitted 19 October, 2020; originally announced October 2020.

    Comments: Accepted at AISTATS2022

    Journal ref: International Conference on Artificial Intelligence and Statistics, 8270-8298, (2022)

  8. arXiv:2006.09273  [pdf, other

    cs.LG stat.ML

    Density of States Estimation for Out-of-Distribution Detection

    Authors: Warren R. Morningstar, Cusuh Ham, Andrew G. Gallagher, Balaji Lakshminarayanan, Alexander A. Alemi, Joshua V. Dillon

    Abstract: Perhaps surprisingly, recent studies have shown probabilistic model likelihoods have poor specificity for out-of-distribution (OOD) detection and often assign higher likelihoods to OOD data than in-distribution data. To ameliorate this issue we propose DoSE, the density of states estimator. Drawing on the statistical physics notion of ``density of states,'' the DoSE decision rule avoids direct com… ▽ More

    Submitted 22 June, 2020; v1 submitted 16 June, 2020; originally announced June 2020.

    Comments: Submitted to NeurIPS. Corrected footnote from: "34th Conference on Neural Information Processing Systems (NeurIPS 2020), Vancouver, Canada" to "Preprint. Under review."

  9. arXiv:2003.01687  [pdf, other

    cs.LG stat.ML

    Automatic Differentiation Variational Inference with Mixtures

    Authors: Warren R. Morningstar, Sharad M. Vikram, Cusuh Ham, Andrew Gallagher, Joshua V. Dillon

    Abstract: Automatic Differentiation Variational Inference (ADVI) is a useful tool for efficiently learning probabilistic models in machine learning. Generally approximate posteriors learned by ADVI are forced to be unimodal in order to facilitate use of the reparameterization trick. In this paper, we show how stratified sampling may be used to enable mixture distributions as the approximate posterior, and d… ▽ More

    Submitted 24 June, 2020; v1 submitted 3 March, 2020; originally announced March 2020.

    Comments: Submitted to NeurIPS 2020, Corrected footnote from: "34th Conference on Neural Information Processing Systems (NeurIPS 2020), Vancouver, Canada" to "Preprint. Under review."

  10. Source structure and molecular gas properties from high-resolution CO imaging of SPT-selected dusty star-forming galaxies

    Authors: Chenxing Dong, Justin S. Spilker, Anthony H. Gonzalez, Yordanka Apostolovski, Manuel Aravena, Matthieu Béthermin, Scott C. Chapman, Chian-Chou Chen, Christopher C. Hayward, Yashar D. Hezaveh, Katrina C. Litke, **gzhe Ma, Daniel P. Marrone, Warren R. Morningstar, Kedar A. Phadke, Cassie A. Reuter, Jarugula Sreevani, Antony A. Stark, Joaquin D. Vieira, Axel Weiß

    Abstract: We present Atacama Large Millimeter/submillimeter Array (ALMA) observations of high-J CO lines ($J_\mathrm{up}=6$, 7, 8) and associated dust continuum towards five strongly lensed, dusty, star-forming galaxies (DSFGs) at redshift $z = 2.7$-5.7. These galaxies, discovered in the South Pole Telescope survey, are observed at $0.2''$-$0.4''$ resolution with ALMA. Our high-resolution imaging coupled wi… ▽ More

    Submitted 29 January, 2019; originally announced January 2019.

    Comments: 19 pages, 12 figures, accepted by ApJ

  11. arXiv:1901.01359  [pdf, other

    astro-ph.IM astro-ph.CO astro-ph.GA

    Data-Driven Reconstruction of Gravitationally Lensed Galaxies using Recurrent Inference Machines

    Authors: Warren R. Morningstar, Laurence Perreault Levasseur, Yashar D. Hezaveh, Roger Blandford, Phil Marshall, Patrick Putzky, Thomas D. Rueter, Risa Wechsler, Max Welling

    Abstract: We present a machine learning method for the reconstruction of the undistorted images of background sources in strongly lensed systems. This method treats the source as a pixelated image and utilizes the Recurrent Inference Machine (RIM) to iteratively reconstruct the background source given a lens model. Our architecture learns to minimize the likelihood of the model parameters (source pixels) gi… ▽ More

    Submitted 4 January, 2019; originally announced January 2019.

    Comments: Submitted to ApJ

  12. Fast Molecular Outflow from a Dusty Star-Forming Galaxy in the Early Universe

    Authors: J. S. Spilker, M. Aravena, M. Bethermin, S. C. Chapman, C. -C. Chen, D. J. M. Cunningham, C. De Breuck, C. Dong, A. H. Gonzalez, C. C. Hayward, Y. D. Hezaveh, K. C. Litke, J. Ma, M. Malkan, D. P. Marrone, T. B. Miller, W. R. Morningstar, D. Narayanan, K. A. Phadke, J. Sreevani, A. A. Stark, J. D. Vieira, A. Weiss

    Abstract: Galaxies grow inefficiently, with only a few percent of the available gas converted into stars each free-fall time. Feedback processes, such as outflowing winds driven by radiation pressure, supernovae or supermassive black hole accretion, can act to halt star formation if they heat or expel the gas supply. We report a molecular outflow launched from a dust-rich star-forming galaxy at redshift 5.3… ▽ More

    Submitted 5 September, 2018; originally announced September 2018.

    Comments: Author's version of paper to appear in Science on 7 September 2018

  13. arXiv:1808.00011  [pdf, other

    astro-ph.IM astro-ph.CO

    Analyzing interferometric observations of strong gravitational lenses with recurrent and convolutional neural networks

    Authors: Warren R. Morningstar, Yashar D. Hezaveh, Laurence Perreault Levasseur, Roger D. Blandford, Philip J. Marshall, Patrick Putzky, Risa H. Wechsler

    Abstract: We use convolutional neural networks (CNNs) and recurrent neural networks (RNNs) to estimate the parameters of strong gravitational lenses from interferometric observations. We explore multiple strategies and find that the best results are obtained when the effects of the dirty beam are first removed from the images with a deconvolution performed with an RNN-based structure before estimating the p… ▽ More

    Submitted 31 July, 2018; originally announced August 2018.

    Comments: To be submitted to ApJ

  14. Galaxy growth in a massive halo in the first billion years of cosmic history

    Authors: D. P. Marrone, J. S. Spilker, C. C. Hayward, J. D. Vieira, M. Aravena, M. L. N. Ashby, M. B. Bayliss, M. Bethermin, M. Brodwin, M. S. Bothwell, J. E. Carlstrom, S. C. Chapman, Chian-Chou Chen, T. M. Crawford, D. J. M. Cunningham, C. De Breuck, C. D. Fassnacht, A. H. Gonzalez, T. R. Greve, Y. D. Hezaveh, K. Lacaille, K. C. Litke, S. Lower, J. Ma, M. Malkan , et al. (12 additional authors not shown)

    Abstract: According to the current understanding of cosmic structure formation, the precursors of the most massive structures in the Universe began to form shortly after the Big Bang, in regions corresponding to the largest fluctuations in the cosmic density field. Observing these structures during their period of active growth and assembly - the first few hundred million years of the Universe - is challeng… ▽ More

    Submitted 8 December, 2017; originally announced December 2017.

    Comments: Nature, published online on 06 December 2017

  15. The spin of the black hole 4U 1543-47

    Authors: W. R. Morningstar, J. M. Miller

    Abstract: We present a new analysis of Rossi X-ray Timing Explorer observations of the 2002 outburst of the transient X-ray nova 4U 1543-47. We focus on observations in the High/Soft state, and attempt to measure the spin of the black hole by simultaneously fitting the thermal disk continuum and by modeling the broadened iron k-shell emission lines and additional blurred reflection features. Previous works… ▽ More

    Submitted 29 August, 2014; originally announced August 2014.

    Comments: Accepted for publication in ApJ Letters

  16. A Seyfert-2-like Spectrum in the High-Mass X-ray Binary Microquasar V4641 Sgr

    Authors: W. R. Morningstar, J. M. Miller, M. T. Reynolds, D. Maitra

    Abstract: We present an analysis of three archival Chandra observations of the black hole V4641 Sgr, performed during a decline into quiescence. The last two observations in the sequence can be modeled with a simple power-law. The first spectrum, however, is remarkably similar to spectra observed in Seyfert-2 active galactic nuclei, which arise through a combination of obscuration and reflection from distan… ▽ More

    Submitted 26 February, 2014; originally announced February 2014.

    Comments: Accepted for publication in ApJ Letters

  17. The Spin of the Black Hole GS 1124-683: Observation of a Retrograde Accretion Disk?

    Authors: Warren R. Morningstar, Jon M. Miller, Rubens C. Reis, Ken Ebisawa

    Abstract: We re-examine archival Ginga data for the black hole binary system GS 1124-683, obtained when the system was undergoing its 1991 outburst. Our analysis estimates the dimensionless spin parameter a=cJ/GM^2 by fitting the X-ray continuum spectra obtained while the system was in the "Thermal Dominant" state. For likely values of mass and distance, we find the spin to be a=-0.25 (-0.64, +0.05) (90% co… ▽ More

    Submitted 8 January, 2014; originally announced January 2014.

    Comments: Accepted for publication in ApJ Letters