Showing 1–39 of 39 results for author: Metzler, A

Event3DGS: Event-Based 3D Gaussian Splatting for High-Speed Robot Egomotion

Authors: Tianyi Xiong, Jiayi Wu, Botao He, Cornelia Fermuller, Yiannis Aloimonos, Heng Huang, Christopher A. Metzler

Abstract: By combining differentiable rendering with explicit point-based scene representations, 3D Gaussian Splatting (3DGS) has demonstrated breakthrough 3D reconstruction capabilities. However, to date 3DGS has had limited impact on robotics, where high-speed egomotion is pervasive: Egomotion introduces motion blur and leads to artifacts in existing frame-based 3DGS reconstruction methods. To address thi… ▽ More By combining differentiable rendering with explicit point-based scene representations, 3D Gaussian Splatting (3DGS) has demonstrated breakthrough 3D reconstruction capabilities. However, to date 3DGS has had limited impact on robotics, where high-speed egomotion is pervasive: Egomotion introduces motion blur and leads to artifacts in existing frame-based 3DGS reconstruction methods. To address this challenge, we introduce Event3DGS, an {\em event-based} 3DGS framework. By exploiting the exceptional temporal resolution of event cameras, Event3GDS can reconstruct high-fidelity 3D structure and appearance under high-speed egomotion. Extensive experiments on multiple synthetic and real-world datasets demonstrate the superiority of Event3DGS compared with existing event-based dense 3D scene reconstruction frameworks; Event3DGS substantially improves reconstruction quality (+3dB) while reducing computational costs by 95\%. Our framework also allows one to incorporate a few motion-blurred frame-based measurements into the reconstruction process to further improve appearance fidelity without loss of structural accuracy. △ Less

Submitted 18 June, 2024; v1 submitted 5 June, 2024; originally announced June 2024.

WaveMo: Learning Wavefront Modulations to See Through Scattering

Authors: Mingyang Xie, Haiyun Guo, Brandon Y. Feng, Lingbo **, Ashok Veeraraghavan, Christopher A. Metzler

Abstract: Imaging through scattering media is a fundamental and pervasive challenge in fields ranging from medical diagnostics to astronomy. A promising strategy to overcome this challenge is wavefront modulation, which induces measurement diversity during image acquisition. Despite its importance, designing optimal wavefront modulations to image through scattering remains under-explored. This paper introdu… ▽ More Imaging through scattering media is a fundamental and pervasive challenge in fields ranging from medical diagnostics to astronomy. A promising strategy to overcome this challenge is wavefront modulation, which induces measurement diversity during image acquisition. Despite its importance, designing optimal wavefront modulations to image through scattering remains under-explored. This paper introduces a novel learning-based framework to address the gap. Our approach jointly optimizes wavefront modulations and a computationally lightweight feedforward "proxy" reconstruction network. This network is trained to recover scenes obscured by scattering, using measurements that are modified by these modulations. The learned modulations produced by our framework generalize effectively to unseen scattering scenarios and exhibit remarkable versatility. During deployment, the learned modulations can be decoupled from the proxy network to augment other more computationally expensive restoration algorithms. Through extensive experiments, we demonstrate our approach significantly advances the state of the art in imaging through scattering media. Our project webpage is at https://wavemo-2024.github.io/. △ Less

Submitted 11 April, 2024; originally announced April 2024.

Adaptive LPD Radar Waveform Design with Generative Deep Learning

Authors: Matthew R. Ziemann, Christopher A. Metzler

Abstract: We propose a novel, learning-based method for adaptively generating low probability of detection (LPD) radar waveforms that blend into their operating environment. Our waveforms are designed to follow a distribution that is indistinguishable from the ambient radio frequency (RF) background -- while still being effective at ranging and sensing. To do so, we use an unsupervised, adversarial learning… ▽ More We propose a novel, learning-based method for adaptively generating low probability of detection (LPD) radar waveforms that blend into their operating environment. Our waveforms are designed to follow a distribution that is indistinguishable from the ambient radio frequency (RF) background -- while still being effective at ranging and sensing. To do so, we use an unsupervised, adversarial learning framework; our generator network produces waveforms designed to confuse a critic network, which is optimized to differentiate generated waveforms from the background. To ensure our generated waveforms are still effective for sensing, we introduce and minimize an ambiguity function-based loss on the generated waveforms. We evaluate the performance of our method by comparing the single-pulse detectability of our generated waveforms with traditional LPD waveforms using a separately trained detection neural network. We find that our method can generate LPD waveforms that reduce detectability by up to 90% while simultaneously offering improved ambiguity function (sensing) characteristics. Our framework also provides a mechanism to trade-off detectability and sensing performance. △ Less

Submitted 18 March, 2024; originally announced March 2024.

Comments: 11 pages, 11 figures

Bagged Deep Image Prior for Recovering Images in the Presence of Speckle Noise

Authors: Xi Chen, Zhewen Hou, Christopher A. Metzler, Arian Maleki, Shirin Jalali

Abstract: We investigate both the theoretical and algorithmic aspects of likelihood-based methods for recovering a complex-valued signal from multiple sets of measurements, referred to as looks, affected by speckle (multiplicative) noise. Our theoretical contributions include establishing the first existing theoretical upper bound on the Mean Squared Error (MSE) of the maximum likelihood estimator under the… ▽ More We investigate both the theoretical and algorithmic aspects of likelihood-based methods for recovering a complex-valued signal from multiple sets of measurements, referred to as looks, affected by speckle (multiplicative) noise. Our theoretical contributions include establishing the first existing theoretical upper bound on the Mean Squared Error (MSE) of the maximum likelihood estimator under the deep image prior hypothesis. Our theoretical results capture the dependence of MSE upon the number of parameters in the deep image prior, the number of looks, the signal dimension, and the number of measurements per look. On the algorithmic side, we introduce the concept of bagged Deep Image Priors (Bagged-DIP) and integrate them with projected gradient descent. Furthermore, we show how employing Newton-Schulz algorithm for calculating matrix inverses within the iterations of PGD reduces the computational complexity of the algorithm. We will show that this method achieves the state-of-the-art performance. △ Less

Submitted 23 February, 2024; originally announced February 2024.

Hyper-Diffusion: Estimating Epistemic and Aleatoric Uncertainty with a Single Model

Authors: Matthew A. Chan, Maria J. Molina, Christopher A. Metzler

Abstract: Estimating and disentangling epistemic uncertainty (uncertainty that can be reduced with more training data) and aleatoric uncertainty (uncertainty that is inherent to the task at hand) is critically important when applying machine learning (ML) to high-stakes applications such as medical imaging and weather forecasting. Conditional diffusion models' breakthrough ability to accurately and efficien… ▽ More Estimating and disentangling epistemic uncertainty (uncertainty that can be reduced with more training data) and aleatoric uncertainty (uncertainty that is inherent to the task at hand) is critically important when applying machine learning (ML) to high-stakes applications such as medical imaging and weather forecasting. Conditional diffusion models' breakthrough ability to accurately and efficiently sample from the posterior distribution of a dataset now makes uncertainty estimation conceptually straightforward: One need only train and sample from a large ensemble of diffusion models. Unfortunately, training such an ensemble becomes computationally intractable as the complexity of the model architecture grows. In this work we introduce a new approach to ensembling, hyper-diffusion, which allows one to accurately estimate epistemic and aleatoric uncertainty with a single model. Unlike existing Monte Carlo dropout based single-model ensembling methods, hyper-diffusion offers the same prediction accuracy as multi-model ensembles. We validate our approach on two distinct tasks: x-ray computed tomography (CT) reconstruction and weather temperature forecasting. △ Less

Submitted 5 February, 2024; originally announced February 2024.

Comments: 10 pages, 7 figures

AONeuS: A Neural Rendering Framework for Acoustic-Optical Sensor Fusion

Authors: Mohamad Qadri, Kevin Zhang, Akshay Hinduja, Michael Kaess, Adithya Pediredla, Christopher A. Metzler

Abstract: Underwater perception and 3D surface reconstruction are challenging problems with broad applications in construction, security, marine archaeology, and environmental monitoring. Treacherous operating conditions, fragile surroundings, and limited navigation control often dictate that submersibles restrict their range of motion and, thus, the baseline over which they can capture measurements. In the… ▽ More Underwater perception and 3D surface reconstruction are challenging problems with broad applications in construction, security, marine archaeology, and environmental monitoring. Treacherous operating conditions, fragile surroundings, and limited navigation control often dictate that submersibles restrict their range of motion and, thus, the baseline over which they can capture measurements. In the context of 3D scene reconstruction, it is well-known that smaller baselines make reconstruction more challenging. Our work develops a physics-based multimodal acoustic-optical neural surface reconstruction framework (AONeuS) capable of effectively integrating high-resolution RGB measurements with low-resolution depth-resolved imaging sonar measurements. By fusing these complementary modalities, our framework can reconstruct accurate high-resolution 3D surfaces from measurements captured over heavily-restricted baselines. Through extensive simulations and in-lab experiments, we demonstrate that AONeuS dramatically outperforms recent RGB-only and sonar-only inverse-differentiable-rendering--based surface reconstruction methods. A website visualizing the results of our paper is located at this address: https://aoneus.github.io/ △ Less

Submitted 21 May, 2024; v1 submitted 5 February, 2024; originally announced February 2024.

Comments: SIGGRAPH 2024 (conference track full paper). First two authors contributed equally. Paper website: https://aoneus.github.io/

FPM-INR: Fourier ptychographic microscopy image stack reconstruction using implicit neural representations

Authors: Haowen Zhou, Brandon Y. Feng, Haiyun Guo, Siyu Lin, Mingshu Liang, Christopher A. Metzler, Changhuei Yang

Abstract: Image stacks provide invaluable 3D information in various biological and pathological imaging applications. Fourier ptychographic microscopy (FPM) enables reconstructing high-resolution, wide field-of-view image stacks without z-stack scanning, thus significantly accelerating image acquisition. However, existing FPM methods take tens of minutes to reconstruct and gigabytes of memory to store a hig… ▽ More Image stacks provide invaluable 3D information in various biological and pathological imaging applications. Fourier ptychographic microscopy (FPM) enables reconstructing high-resolution, wide field-of-view image stacks without z-stack scanning, thus significantly accelerating image acquisition. However, existing FPM methods take tens of minutes to reconstruct and gigabytes of memory to store a high-resolution volumetric scene, impeding fast gigapixel-scale remote digital pathology. While deep learning approaches have been explored to address this challenge, existing methods poorly generalize to novel datasets and can produce unreliable hallucinations. This work presents FPM-INR, a compact and efficient framework that integrates physics-based optical models with implicit neural representations (INR) to represent and reconstruct FPM image stacks. FPM-INR is agnostic to system design or sample types and does not require external training data. In our demonstrated experiments, FPM-INR substantially outperforms traditional FPM algorithms with up to a 25-fold increase in speed and an 80-fold reduction in memory usage for continuous image stack representations. △ Less

Submitted 31 October, 2023; v1 submitted 27 October, 2023; originally announced October 2023.

Comments: Project Page: https://hwzhou2020.github.io/FPM-INR-Web/

TiDy-PSFs: Computational Imaging with Time-Averaged Dynamic Point-Spread-Functions

Authors: Sachin Shah, Sakshum Kulshrestha, Christopher A. Metzler

Abstract: Point-spread-function (PSF) engineering is a powerful computational imaging techniques wherein a custom phase mask is integrated into an optical system to encode additional information into captured images. Used in combination with deep learning, such systems now offer state-of-the-art performance at monocular depth estimation, extended depth-of-field imaging, lensless imaging, and other tasks. In… ▽ More Point-spread-function (PSF) engineering is a powerful computational imaging techniques wherein a custom phase mask is integrated into an optical system to encode additional information into captured images. Used in combination with deep learning, such systems now offer state-of-the-art performance at monocular depth estimation, extended depth-of-field imaging, lensless imaging, and other tasks. Inspired by recent advances in spatial light modulator (SLM) technology, this paper answers a natural question: Can one encode additional information and achieve superior performance by changing a phase mask dynamically over time? We first prove that the set of PSFs described by static phase masks is non-convex and that, as a result, time-averaged PSFs generated by dynamic phase masks are fundamentally more expressive. We then demonstrate, in simulation, that time-averaged dynamic (TiDy) phase masks can offer substantially improved monocular depth estimation and extended depth-of-field imaging performance. △ Less

Submitted 30 March, 2023; originally announced March 2023.

Comments: 13 pages, 16 figures

SUD$^2$: Supervision by Denoising Diffusion Models for Image Reconstruction

Authors: Matthew A. Chan, Sean I. Young, Christopher A. Metzler

Abstract: Many imaging inverse problems$\unicode{x2014}$such as image-dependent in-painting and dehazing$\unicode{x2014}$are challenging because their forward models are unknown or depend on unknown latent parameters. While one can solve such problems by training a neural network with vast quantities of paired training data, such paired training data is often unavailable. In this paper, we propose a general… ▽ More Many imaging inverse problems$\unicode{x2014}$such as image-dependent in-painting and dehazing$\unicode{x2014}$are challenging because their forward models are unknown or depend on unknown latent parameters. While one can solve such problems by training a neural network with vast quantities of paired training data, such paired training data is often unavailable. In this paper, we propose a generalized framework for training image reconstruction networks when paired training data is scarce. In particular, we demonstrate the ability of image denoising algorithms and, by extension, denoising diffusion models to supervise network training in the absence of paired training data. △ Less

Submitted 3 April, 2023; v1 submitted 16 March, 2023; originally announced March 2023.

Comments: 18 pages, 15 figures

Roadmap on Deep Learning for Microscopy

Authors: Giovanni Volpe, Carolina Wählby, Lei Tian, Michael Hecht, Artur Yakimovich, Kristina Monakhova, Laura Waller, Ivo F. Sbalzarini, Christopher A. Metzler, Mingyang Xie, Kevin Zhang, Isaac C. D. Lenton, Halina Rubinsztein-Dunlop, Daniel Brunner, Bijie Bai, Aydogan Ozcan, Daniel Midtvedt, Hao Wang, Nataša Sladoje, Joakim Lindblad, Jason T. Smith, Marien Ochoa, Margarida Barroso, Xavier Intes, Tong Qiu , et al. (50 additional authors not shown)

Abstract: Through digital imaging, microscopy has evolved from primarily being a means for visual observation of life at the micro- and nano-scale, to a quantitative tool with ever-increasing resolution and throughput. Artificial intelligence, deep neural networks, and machine learning are all niche terms describing computational methods that have gained a pivotal role in microscopy-based research over the… ▽ More Through digital imaging, microscopy has evolved from primarily being a means for visual observation of life at the micro- and nano-scale, to a quantitative tool with ever-increasing resolution and throughput. Artificial intelligence, deep neural networks, and machine learning are all niche terms describing computational methods that have gained a pivotal role in microscopy-based research over the past decade. This Roadmap is written collectively by prominent researchers and encompasses selected aspects of how machine learning is applied to microscopy image data, with the aim of gaining scientific knowledge by improved image quality, automated detection, segmentation, classification and tracking of objects, and efficient merging of information from multiple imaging modalities. We aim to give the reader an overview of the key developments and an understanding of possibilities and limitations of machine learning for microscopy. It will be of interest to a wide cross-disciplinary audience in the physical sciences and life sciences. △ Less

Submitted 7 March, 2023; originally announced March 2023.

Weakly-Supervised Semantic Segmentation of Ships Using Thermal Imagery

Authors: Rushil Joshi, Ethan Adams, Matthew Ziemann, Christopher A. Metzler

Abstract: The United States coastline spans 95,471 miles; a distance that cannot be effectively patrolled or secured by manual human effort alone. Unmanned Aerial Vehicles (UAVs) equipped with infrared cameras and deep-learning based algorithms represent a more efficient alternative for identifying and segmenting objects of interest - namely, ships. However, standard approaches to training these algorithms… ▽ More The United States coastline spans 95,471 miles; a distance that cannot be effectively patrolled or secured by manual human effort alone. Unmanned Aerial Vehicles (UAVs) equipped with infrared cameras and deep-learning based algorithms represent a more efficient alternative for identifying and segmenting objects of interest - namely, ships. However, standard approaches to training these algorithms require large-scale datasets of densely labeled infrared maritime images. Such datasets are not publicly available and manually annotating every pixel in a large-scale dataset would have an extreme labor cost. In this work we demonstrate that, in the context of segmenting ships in infrared imagery, weakly-supervising an algorithm with sparsely labeled data can drastically reduce data labeling costs with minimal impact on system performance. We apply weakly-supervised learning to an unlabeled dataset of 7055 infrared images sourced from the Naval Air Warfare Center Aircraft Division (NAWCAD). We find that by sparsely labeling only 32 points per image, weakly-supervised segmentation models can still effectively detect and segment ships, with a Jaccard score of up to 0.756. △ Less

Submitted 26 December, 2022; originally announced December 2022.

MSC Class: I.4

City-Wide Perceptions of Neighbourhood Quality using Street View Images

Authors: Emily Muller, Emily Gemmell, Ishmam Choudhury, Ricky Nathvani, Antje Barbara Metzler, James Bennett, Emily Denton, Seth Flaxman, Majid Ezzati

Abstract: The interactions of individuals with city neighbourhoods is determined, in part, by the perceived quality of urban environments. Perceived neighbourhood quality is a core component of urban vitality, influencing social cohesion, sense of community, safety, activity and mental health of residents. Large-scale assessment of perceptions of neighbourhood quality was pioneered by the Place Pulse projec… ▽ More The interactions of individuals with city neighbourhoods is determined, in part, by the perceived quality of urban environments. Perceived neighbourhood quality is a core component of urban vitality, influencing social cohesion, sense of community, safety, activity and mental health of residents. Large-scale assessment of perceptions of neighbourhood quality was pioneered by the Place Pulse projects. Researchers demonstrated the efficacy of crowd-sourcing perception ratings of image pairs across 56 cities and training a model to predict perceptions from street-view images. Variation across cities may limit Place Pulse's usefulness for assessing within-city perceptions. In this paper, we set forth a protocol for city-specific dataset collection for the perception: 'On which street would you prefer to walk?'. This paper describes our methodology, based in London, including collection of images and ratings, web development, model training and map**. Assessment of within-city perceptions of neighbourhoods can identify inequities, inform planning priorities, and identify temporal dynamics. Code available: https://emilymuller1991.github.io/urban-perceptions/. △ Less

Submitted 24 November, 2022; v1 submitted 22 November, 2022; originally announced November 2022.

MetaDIP: Accelerating Deep Image Prior with Meta Learning

Authors: Kevin Zhang, Mingyang Xie, Maharshi Gor, Yi-Ting Chen, Yvonne Zhou, Christopher A. Metzler

Abstract: Deep image prior (DIP) is a recently proposed technique for solving imaging inverse problems by fitting the reconstructed images to the output of an untrained convolutional neural network. Unlike pretrained feedforward neural networks, the same DIP can generalize to arbitrary inverse problems, from denoising to phase retrieval, while offering competitive performance at each task. The central disad… ▽ More Deep image prior (DIP) is a recently proposed technique for solving imaging inverse problems by fitting the reconstructed images to the output of an untrained convolutional neural network. Unlike pretrained feedforward neural networks, the same DIP can generalize to arbitrary inverse problems, from denoising to phase retrieval, while offering competitive performance at each task. The central disadvantage of DIP is that, while feedforward neural networks can reconstruct an image in a single pass, DIP must gradually update its weights over hundreds to thousands of iterations, at a significant computational cost. In this work we use meta-learning to massively accelerate DIP-based reconstructions. By learning a proper initialization for the DIP weights, we demonstrate a 10x improvement in runtimes across a range of inverse imaging tasks. Moreover, we demonstrate that a network trained to quickly reconstruct faces also generalizes to reconstructing natural image patches. △ Less

Submitted 17 September, 2022; originally announced September 2022.

Denoising Generalized Expectation-Consistent Approximation for MR Image Recovery

Authors: Saurav K. Shastri, Rizwan Ahmad, Christopher A. Metzler, Philip Schniter

Abstract: To solve inverse problems, plug-and-play (PnP) methods replace the proximal step in a convex optimization algorithm with a call to an application-specific denoiser, often implemented using a deep neural network (DNN). Although such methods yield accurate solutions, they can be improved. For example, denoisers are usually designed/trained to remove white Gaussian noise, but the denoiser input error… ▽ More To solve inverse problems, plug-and-play (PnP) methods replace the proximal step in a convex optimization algorithm with a call to an application-specific denoiser, often implemented using a deep neural network (DNN). Although such methods yield accurate solutions, they can be improved. For example, denoisers are usually designed/trained to remove white Gaussian noise, but the denoiser input error in PnP algorithms is usually far from white or Gaussian. Approximate message passing (AMP) methods provide white and Gaussian denoiser input error, but only when the forward operator is sufficiently random. In this work, for Fourier-based forward operators, we propose a PnP algorithm based on generalized expectation-consistent (GEC) approximation -- a close cousin of AMP -- that offers predictable error statistics at each iteration, as well as a new DNN denoiser that leverages those statistics. We apply our approach to magnetic resonance (MR) image recovery and demonstrate its advantages over existing PnP and AMP methods. △ Less

Submitted 7 September, 2022; v1 submitted 8 June, 2022; originally announced June 2022.

TurbuGAN: An Adversarial Learning Approach to Spatially-Varying Multiframe Blind Deconvolution with Applications to Imaging Through Turbulence

Authors: Brandon Yushan Feng, Mingyang Xie, Christopher A. Metzler

Abstract: We present a self-supervised and self-calibrating multi-shot approach to imaging through atmospheric turbulence, called TurbuGAN. Our approach requires no paired training data, adapts itself to the distribution of the turbulence, leverages domain-specific data priors, and can generalize from tens to thousands of measurements. We achieve such functionality through an adversarial sensing framework a… ▽ More We present a self-supervised and self-calibrating multi-shot approach to imaging through atmospheric turbulence, called TurbuGAN. Our approach requires no paired training data, adapts itself to the distribution of the turbulence, leverages domain-specific data priors, and can generalize from tens to thousands of measurements. We achieve such functionality through an adversarial sensing framework adapted from CryoGAN, which uses a discriminator network to match the distributions of captured and simulated measurements. Our framework builds on CryoGAN by (1) generalizing the forward measurement model to incorporate physically accurate and computationally efficient models for light propagation through anisoplanatic turbulence, (2) enabling adaptation to slightly misspecified forward models, and (3) leveraging domain-specific prior knowledge using pretrained generative networks, when available. We validate TurbuGAN on both computationally simulated and experimentally captured images distorted with anisoplanatic turbulence. △ Less

Submitted 2 January, 2023; v1 submitted 13 March, 2022; originally announced March 2022.

Expectation Consistent Plug-and-Play for MRI

Authors: Saurav K Shastri, Rizwan Ahmad, Christopher A Metzler, Philip Schniter

Abstract: For image recovery problems, plug-and-play (PnP) methods have been developed that replace the proximal step in an optimization algorithm with a call to an application-specific denoiser, often implemented using a deep neural network. Although such methods have been successful, they can be improved. For example, the denoiser is often trained using white Gaussian noise, while PnP's denoiser input err… ▽ More For image recovery problems, plug-and-play (PnP) methods have been developed that replace the proximal step in an optimization algorithm with a call to an application-specific denoiser, often implemented using a deep neural network. Although such methods have been successful, they can be improved. For example, the denoiser is often trained using white Gaussian noise, while PnP's denoiser input error is often far from white and Gaussian, with statistics that are difficult to predict from iteration to iteration. PnP methods based on approximate message passing (AMP) are an exception, but only when the forward operator behaves like a large random matrix. In this work, we design a PnP method using the expectation consistent (EC) approximation algorithm, a generalization of AMP, that offers predictable error statistics at each iteration, from which a deep-net denoiser can be effectively trained. △ Less

Submitted 11 February, 2022; originally announced February 2022.

Supervision by Denoising for Medical Image Segmentation

Authors: Sean I. Young, Adrian V. Dalca, Enzo Ferrante, Polina Golland, Christopher A. Metzler, Bruce Fischl, Juan Eugenio Iglesias

Abstract: Learning-based image reconstruction models, such as those based on the U-Net, require a large set of labeled images if good generalization is to be guaranteed. In some imaging domains, however, labeled data with pixel- or voxel-level label accuracy are scarce due to the cost of acquiring them. This problem is exacerbated further in domains like medical imaging, where there is no single ground trut… ▽ More Learning-based image reconstruction models, such as those based on the U-Net, require a large set of labeled images if good generalization is to be guaranteed. In some imaging domains, however, labeled data with pixel- or voxel-level label accuracy are scarce due to the cost of acquiring them. This problem is exacerbated further in domains like medical imaging, where there is no single ground truth label, resulting in large amounts of repeat variability in the labels. Therefore, training reconstruction networks to generalize better by learning from both labeled and unlabeled examples (called semi-supervised learning) is problem of practical and theoretical interest. However, traditional semi-supervised learning methods for image reconstruction often necessitate handcrafting a differentiable regularizer specific to some given imaging problem, which can be extremely time-consuming. In this work, we propose "supervision by denoising" (SUD), a framework that enables us to supervise reconstruction models using their own denoised output as soft labels. SUD unifies stochastic averaging and spatial denoising techniques under a spatio-temporal denoising framework and alternates denoising and model weight update steps in an optimization framework for semi-supervision. As example applications, we apply SUD to two problems arising from biomedical imaging -- anatomical brain reconstruction (3D) and cortical parcellation (2D) -- to demonstrate a significant improvement in the image reconstructions over supervised-only and stochastic averaging baselines. △ Less

Submitted 4 January, 2024; v1 submitted 7 February, 2022; originally announced February 2022.

Comments: To appear in the IEEE Transactions on Pattern Analysis and Machine Intelligence

Measuring Financial Advice: aligning client elicited and revealed risk

Authors: John R. J. Thompson, Longlong Feng, R. Mark Reesor, Chuck Grace, Adam Metzler

Abstract: Financial advisors use questionnaires and discussions with clients to determine a suitable portfolio of assets that will allow clients to reach their investment objectives. Financial institutions assign risk ratings to each security they offer, and those ratings are used to guide clients and advisors to choose an investment portfolio risk that suits their stated risk tolerance. This paper compares… ▽ More Financial advisors use questionnaires and discussions with clients to determine a suitable portfolio of assets that will allow clients to reach their investment objectives. Financial institutions assign risk ratings to each security they offer, and those ratings are used to guide clients and advisors to choose an investment portfolio risk that suits their stated risk tolerance. This paper compares client Know Your Client (KYC) profile risk allocations to their investment portfolio risk selections using a value-at-risk discrepancy methodology. Value-at-risk is used to measure elicited and revealed risk to show whether clients are over-risked or under-risked, changes in KYC risk lead to changes in portfolio configuration, and cash flow affects a client's portfolio risk. We demonstrate the effectiveness of value-at-risk at measuring clients' elicited and revealed risk on a dataset provided by a private Canadian financial dealership of over $50,000$ accounts for over $27,000$ clients and $300$ advisors. By measuring both elicited and revealed risk using the same measure, we can determine how well a client's portfolio aligns with their stated goals. We believe that using value-at-risk to measure client risk provides valuable insight to advisors to ensure that their practice is KYC compliant, to better tailor their client portfolios to stated goals, communicate advice to clients to either align their portfolios to stated goals or refresh their goals, and to monitor changes to the clients' risk positions across their practice. △ Less

Submitted 25 May, 2021; originally announced May 2021.

Comments: 36 pages, 17 figures, 5 tables

MSC Class: 91G70 (Primary) 91C05; 91-10 (Secondary)

SUREMap: Predicting Uncertainty in CNN-based Image Reconstruction Using Stein's Unbiased Risk Estimate

Authors: Ruangrawee Kitichotkul, Christopher A. Metzler, Frank Ong, Gordon Wetzstein

Abstract: Convolutional neural networks (CNN) have emerged as a powerful tool for solving computational imaging reconstruction problems. However, CNNs are generally difficult-to-understand black-boxes. Accordingly, it is challenging to know when they will work and, more importantly, when they will fail. This limitation is a major barrier to their use in safety-critical applications like medical imaging: Is… ▽ More Convolutional neural networks (CNN) have emerged as a powerful tool for solving computational imaging reconstruction problems. However, CNNs are generally difficult-to-understand black-boxes. Accordingly, it is challenging to know when they will work and, more importantly, when they will fail. This limitation is a major barrier to their use in safety-critical applications like medical imaging: Is that blob in the reconstruction an artifact or a tumor? In this work we use Stein's unbiased risk estimate (SURE) to develop per-pixel confidence intervals, in the form of heatmaps, for compressive sensing reconstruction using the approximate message passing (AMP) framework with CNN-based denoisers. These heatmaps tell end-users how much to trust an image formed by a CNN, which could greatly improve the utility of CNNs in various computational imaging applications. △ Less

Submitted 19 April, 2021; v1 submitted 25 October, 2020; originally announced October 2020.

D-VDAMP: Denoising-based Approximate Message Passing for Compressive MRI

Authors: Christopher A. Metzler, Gordon Wetzstein

Abstract: Plug and play (P&P) algorithms iteratively apply highly optimized image denoisers to impose priors and solve computational image reconstruction problems, to great effect. However, in general the "effective noise", that is the difference between the true signal and the intermediate solution, within the iterations of P&P algorithms is neither Gaussian nor white. This fact makes existing denoising al… ▽ More Plug and play (P&P) algorithms iteratively apply highly optimized image denoisers to impose priors and solve computational image reconstruction problems, to great effect. However, in general the "effective noise", that is the difference between the true signal and the intermediate solution, within the iterations of P&P algorithms is neither Gaussian nor white. This fact makes existing denoising algorithms suboptimal. In this work, we propose a CNN architecture for removing colored Gaussian noise and combine it with the recently proposed VDAMP algorithm, whose effective noise follows a predictable colored Gaussian distribution. We apply the resulting denoising-based VDAMP (D-VDAMP) algorithm to variable density sampled compressive MRI where it substantially outperforms existing techniques. △ Less

Submitted 25 October, 2020; originally announced October 2020.

Deep Learning Techniques for Inverse Problems in Imaging

Authors: Gregory Ongie, Ajil Jalal, Christopher A. Metzler, Richard G. Baraniuk, Alexandros G. Dimakis, Rebecca Willett

Abstract: Recent work in machine learning shows that deep neural networks can be used to solve a wide variety of inverse problems arising in computational imaging. We explore the central prevailing themes of this emerging area and present a taxonomy that can be used to categorize different problems and reconstruction methods. Our taxonomy is organized along two central axes: (1) whether or not a forward mod… ▽ More Recent work in machine learning shows that deep neural networks can be used to solve a wide variety of inverse problems arising in computational imaging. We explore the central prevailing themes of this emerging area and present a taxonomy that can be used to categorize different problems and reconstruction methods. Our taxonomy is organized along two central axes: (1) whether or not a forward model is known and to what extent it is used in training and testing, and (2) whether or not the learning is supervised or unsupervised, i.e., whether or not the training relies on access to matched ground truth image and measurement pairs. We also discuss the trade-offs associated with these different reconstruction approaches, caveats and common failure modes, plus open problems and avenues for future work. △ Less

Submitted 12 May, 2020; originally announced May 2020.

Deep S$^3$PR: Simultaneous Source Separation and Phase Retrieval Using Deep Generative Models

Authors: Christopher A. Metzler, Gordon Wetzstein

Abstract: This paper introduces and solves the simultaneous source separation and phase retrieval (S$^3$PR) problem. S$^3$PR is an important but largely unsolved problem in a number application domains, including microscopy, wireless communication, and imaging through scattering media, where one has multiple independent coherent sources whose phase is difficult to measure. In general, S$^3$PR is highly unde… ▽ More This paper introduces and solves the simultaneous source separation and phase retrieval (S$^3$PR) problem. S$^3$PR is an important but largely unsolved problem in a number application domains, including microscopy, wireless communication, and imaging through scattering media, where one has multiple independent coherent sources whose phase is difficult to measure. In general, S$^3$PR is highly under-determined, non-convex, and difficult to solve. In this work, we demonstrate that by restricting the solutions to lie in the range of a deep generative model, we can constrain the search space sufficiently to solve S$^3$PR. △ Less

Submitted 13 October, 2020; v1 submitted 13 February, 2020; originally announced February 2020.

Keyhole Imaging: Non-Line-of-Sight Imaging and Tracking of Moving Objects Along a Single Optical Path

Authors: Christopher A. Metzler, David B. Lindell, Gordon Wetzstein

Abstract: Non-line-of-sight (NLOS) imaging and tracking is an emerging technology that allows the shape or position of objects around corners or behind diffusers to be recovered from transient, time-of-flight measurements. However, existing NLOS approaches require the imaging system to scan a large area on a visible surface, where the indirect light paths of hidden objects are sampled. In many applications,… ▽ More Non-line-of-sight (NLOS) imaging and tracking is an emerging technology that allows the shape or position of objects around corners or behind diffusers to be recovered from transient, time-of-flight measurements. However, existing NLOS approaches require the imaging system to scan a large area on a visible surface, where the indirect light paths of hidden objects are sampled. In many applications, such as robotic vision or autonomous driving, optical access to a large scanning area may not be available, which severely limits the practicality of existing NLOS techniques. Here, we propose a new approach, dubbed keyhole imaging, that captures a sequence of transient measurements along a single optical path, for example, through a keyhole. Assuming that the hidden object of interest moves during the acquisition time, we effectively capture a series of time-resolved projections of the object's shape from unknown viewpoints. We derive inverse methods based on expectation-maximization to recover the object's shape and location using these measurements. Then, with the help of long exposure times and retroreflective tape, we demonstrate successful experimental results with a prototype keyhole imaging system. △ Less

Submitted 5 January, 2021; v1 submitted 13 December, 2019; originally announced December 2019.

Deep Optics for Single-shot High-dynamic-range Imaging

Authors: Christopher A. Metzler, Hayato Ikoma, Yifan Peng, Gordon Wetzstein

Abstract: High-dynamic-range (HDR) imaging is crucial for many computer graphics and vision applications. Yet, acquiring HDR images with a single shot remains a challenging problem. Whereas modern deep learning approaches are successful at hallucinating plausible HDR content from a single low-dynamic-range (LDR) image, saturated scene details often cannot be faithfully recovered. Inspired by recent deep opt… ▽ More High-dynamic-range (HDR) imaging is crucial for many computer graphics and vision applications. Yet, acquiring HDR images with a single shot remains a challenging problem. Whereas modern deep learning approaches are successful at hallucinating plausible HDR content from a single low-dynamic-range (LDR) image, saturated scene details often cannot be faithfully recovered. Inspired by recent deep optical imaging approaches, we interpret this problem as jointly training an optical encoder and electronic decoder where the encoder is parameterized by the point spread function (PSF) of the lens, the bottleneck is the sensor with a limited dynamic range, and the decoder is a convolutional neural network (CNN). The lens surface is then jointly optimized with the CNN in a training phase; we fabricate this optimized optical element and attach it as a hardware add-on to a conventional camera during inference. In extensive simulations and with a physical prototype, we demonstrate that this end-to-end deep optical imaging approach to single-shot HDR imaging outperforms both purely CNN-based approaches and other PSF engineering approaches. △ Less

Submitted 1 August, 2019; originally announced August 2019.

An Expectation-Maximization Approach to Tuning Generalized Vector Approximate Message Passing

Authors: Christopher A. Metzler, Philip Schniter, Richard G. Baraniuk

Abstract: Generalized Vector Approximate Message Passing (GVAMP) is an efficient iterative algorithm for approximately minimum-mean-squared-error estimation of a random vector $\mathbf{x}\sim p_{\mathbf{x}}(\mathbf{x})$ from generalized linear measurements, i.e., measurements of the form $\mathbf{y}=Q(\mathbf{z})$ where $\mathbf{z}=\mathbf{Ax}$ with known $\mathbf{A}$, and $Q(\cdot)$ is a noisy, potentially… ▽ More Generalized Vector Approximate Message Passing (GVAMP) is an efficient iterative algorithm for approximately minimum-mean-squared-error estimation of a random vector $\mathbf{x}\sim p_{\mathbf{x}}(\mathbf{x})$ from generalized linear measurements, i.e., measurements of the form $\mathbf{y}=Q(\mathbf{z})$ where $\mathbf{z}=\mathbf{Ax}$ with known $\mathbf{A}$, and $Q(\cdot)$ is a noisy, potentially nonlinear, componentwise function. Problems of this form show up in numerous applications, including robust regression, binary classification, quantized compressive sensing, and phase retrieval. In some cases, the prior $p_{\mathbf{x}}$ and/or channel $Q(\cdot)$ depend on unknown deterministic parameters $\boldsymbolθ$, which prevents a direct application of GVAMP. In this paper we propose a way to combine expectation maximization (EM) with GVAMP to jointly estimate $\mathbf{x}$ and $\boldsymbolθ$. We then demonstrate how EM-GVAMP can solve the phase retrieval problem with unknown measurement-noise variance. △ Less

Submitted 26 June, 2018; originally announced June 2018.

Unsupervised Learning with Stein's Unbiased Risk Estimator

Authors: Christopher A. Metzler, Ali Mousavi, Reinhard Heckel, Richard G. Baraniuk

Abstract: Learning from unlabeled and noisy data is one of the grand challenges of machine learning. As such, it has seen a flurry of research with new ideas proposed continuously. In this work, we revisit a classical idea: Stein's Unbiased Risk Estimator (SURE). We show that, in the context of image recovery, SURE and its generalizations can be used to train convolutional neural networks (CNNs) for a range… ▽ More Learning from unlabeled and noisy data is one of the grand challenges of machine learning. As such, it has seen a flurry of research with new ideas proposed continuously. In this work, we revisit a classical idea: Stein's Unbiased Risk Estimator (SURE). We show that, in the context of image recovery, SURE and its generalizations can be used to train convolutional neural networks (CNNs) for a range of image denoising and recovery problems without any ground truth data. Specifically, our goal is to reconstruct an image $x$ from a noisy linear transformation (measurement) of the image. We consider two scenarios: one where no additional data is available and one where we have measurements of other images that are drawn from the same noisy distribution as $x$, but have no access to the clean images. Such is the case, for instance, in the context of medical imaging, microscopy, and astronomy, where noise-less ground truth data is rarely available. We show that in this situation, SURE can be used to estimate the mean-squared-error loss associated with an estimate of $x$. Using this estimate of the loss, we train networks to perform denoising and compressed sensing recovery. In addition, we also use the SURE framework to partially explain and improve upon an intriguing results presented by Ulyanov et al. in "Deep Image Prior": that a network initialized with random weights and fit to a single noisy image can effectively denoise that image. Public implementations of the networks and methods described in this paper can be found at https://github.com/ricedsp/D-AMP_Toolbox. △ Less

Submitted 22 July, 2020; v1 submitted 26 May, 2018; originally announced May 2018.

prDeep: Robust Phase Retrieval with a Flexible Deep Network

Authors: Christopher A. Metzler, Philip Schniter, Ashok Veeraraghavan, Richard G. Baraniuk

Abstract: Phase retrieval algorithms have become an important component in many modern computational imaging systems. For instance, in the context of ptychography and speckle correlation imaging, they enable imaging past the diffraction limit and through scattering media, respectively. Unfortunately, traditional phase retrieval algorithms struggle in the presence of noise. Progress has been made recently on… ▽ More Phase retrieval algorithms have become an important component in many modern computational imaging systems. For instance, in the context of ptychography and speckle correlation imaging, they enable imaging past the diffraction limit and through scattering media, respectively. Unfortunately, traditional phase retrieval algorithms struggle in the presence of noise. Progress has been made recently on more robust algorithms using signal priors, but at the expense of limiting the range of supported measurement models (e.g., to Gaussian or coded diffraction patterns). In this work we leverage the regularization-by-denoising framework and a convolutional neural network denoiser to create prDeep, a new phase retrieval algorithm that is both robust and broadly applicable. We test and validate prDeep in simulation to demonstrate that it is robust to noise and can handle a variety of system models. A MatConvNet implementation of prDeep is available at https://github.com/ricedsp/prDeep. △ Less

Submitted 29 June, 2018; v1 submitted 28 February, 2018; originally announced March 2018.

Learned D-AMP: Principled Neural Network based Compressive Image Recovery

Authors: Christopher A. Metzler, Ali Mousavi, Richard G. Baraniuk

Abstract: Compressive image recovery is a challenging problem that requires fast and accurate algorithms. Recently, neural networks have been applied to this problem with promising results. By exploiting massively parallel GPU processing architectures and oodles of training data, they can run orders of magnitude faster than existing techniques. However, these methods are largely unprincipled black boxes tha… ▽ More Compressive image recovery is a challenging problem that requires fast and accurate algorithms. Recently, neural networks have been applied to this problem with promising results. By exploiting massively parallel GPU processing architectures and oodles of training data, they can run orders of magnitude faster than existing techniques. However, these methods are largely unprincipled black boxes that are difficult to train and often-times specific to a single measurement matrix. It was recently demonstrated that iterative sparse-signal-recovery algorithms can be "unrolled" to form interpretable deep networks. Taking inspiration from this work, we develop a novel neural network architecture that mimics the behavior of the denoising-based approximate message passing (D-AMP) algorithm. We call this new network Learned D-AMP (LDAMP). The LDAMP network is easy to train, can be applied to a variety of different measurement matrices, and comes with a state-evolution heuristic that accurately predicts its performance. Most importantly, it outperforms the state-of-the-art BM3D-AMP and NLR-CS algorithms in terms of both accuracy and run time. At high resolutions, and when used with sensing matrices that have fast implementations, LDAMP runs over $50\times$ faster than BM3D-AMP and hundreds of times faster than NLR-CS. △ Less

Submitted 6 November, 2017; v1 submitted 21 April, 2017; originally announced April 2017.

From Denoising to Compressed Sensing

Authors: Christopher A. Metzler, Arian Maleki, Richard G. Baraniuk

Abstract: A denoising algorithm seeks to remove noise, errors, or perturbations from a signal. Extensive research has been devoted to this arena over the last several decades, and as a result, today's denoisers can effectively remove large amounts of additive white Gaussian noise. A compressed sensing (CS) reconstruction algorithm seeks to recover a structured signal acquired using a small number of randomi… ▽ More A denoising algorithm seeks to remove noise, errors, or perturbations from a signal. Extensive research has been devoted to this arena over the last several decades, and as a result, today's denoisers can effectively remove large amounts of additive white Gaussian noise. A compressed sensing (CS) reconstruction algorithm seeks to recover a structured signal acquired using a small number of randomized measurements. Typical CS reconstruction algorithms can be cast as iteratively estimating a signal from a perturbed observation. This paper answers a natural question: How can one effectively employ a generic denoiser in a CS reconstruction algorithm? In response, we develop an extension of the approximate message passing (AMP) framework, called Denoising-based AMP (D-AMP), that can integrate a wide class of denoisers within its iterations. We demonstrate that, when used with a high performance denoiser for natural images, D-AMP offers state-of-the-art CS recovery performance while operating tens of times faster than competing methods. We explain the exceptional performance of D-AMP by analyzing some of its theoretical features. A key element in D-AMP is the use of an appropriate Onsager correction term in its iterations, which coerces the signal perturbation at each iteration to be very close to the white Gaussian noise that denoisers are typically designed to remove. △ Less

Submitted 17 April, 2016; v1 submitted 16 June, 2014; originally announced June 2014.

Theoretical characterization of a model of aragonite crystal orientation in red abalone nacre

Authors: S. N. Coppersmith, P. U. P. A. Gilbert, R. A. Metzler

Abstract: Nacre, commonly known as mother-of-pearl, is a remarkable biomineral that in red abalone consists of layers of 400-nm thick aragonite crystalline tablets confined by organic matrix sheets, with the $(001)$ crystal axes of the aragonite tablets oriented to within $\pm$ 12 degrees from the normal to the layer planes. Recent experiments demonstrate that this orientational order develops over a dist… ▽ More Nacre, commonly known as mother-of-pearl, is a remarkable biomineral that in red abalone consists of layers of 400-nm thick aragonite crystalline tablets confined by organic matrix sheets, with the $(001)$ crystal axes of the aragonite tablets oriented to within $\pm$ 12 degrees from the normal to the layer planes. Recent experiments demonstrate that this orientational order develops over a distance of tens of layers from the prismatic boundary at which nacre formation begins. Our previous simulations of a model in which the order develops because of differential tablet growth rates (oriented tablets growing faster than misoriented ones) yield patterns of tablets that agree qualitatively and quantitatively with the experimental measurements. This paper presents an analytical treatment of this model, focusing on how the dynamical development and eventual degree of order depend on model parameters. Dynamical equations for the probability distributions governing tablet orientations are introduced whose form can be determined from symmetry considerations and for which substantial analytic progress can be made. Numerical simulations are performed to relate the parameters used in the analytic theory to those in the microscopic growth model. The analytic theory demonstrates that the dynamical mechanism is able to achieve a much higher degree of order than naive estimates would indicate. △ Less

Submitted 28 January, 2009; originally announced January 2009.

Comments: 20 pages, 3 figures

Ordering in red abalone nacre

Authors: Rebecca A. Metzler, Dong Zhou, Mike Abrecht, Susan N. Coppersmith, P. U. P. A. Gilbert

Abstract: Red abalone nacre is an intensely studied biomineral, and yet its formation mechanism remains poorly understood. Here we report quantitative measurements probing the degree of order of the aragonite tablets in nacre, and show that order develops over a distance of about 50 microns. These observations indicate that the orientational order of aragonite tablets in nacre is established gradually and… ▽ More Red abalone nacre is an intensely studied biomineral, and yet its formation mechanism remains poorly understood. Here we report quantitative measurements probing the degree of order of the aragonite tablets in nacre, and show that order develops over a distance of about 50 microns. These observations indicate that the orientational order of aragonite tablets in nacre is established gradually and dynamically, and we show that a model of controlled assembly based on suppression of the crystal growth rate along a specific direction, when growth is confined in a layered structure, yields a tablet pattern consistent with those revealed by detailed experimental measurements. This work provides strong evidence that the organism s control of crystal orientation in nacre occurs via regulation of crystal nucleation and growth as opposed to direct templation or heteroepitaxial growth on organic molecules on the organic matrix sheets. △ Less

Submitted 25 October, 2007; originally announced October 2007.

Effects of clum** on temperature I: externally heated clouds

Authors: S. D. Doty, R. A. Metzler, M. L. Palotti

Abstract: We present a study of radiative transfer in dusty, clumpy star-forming regions. A series of self-consistent, 3-D, continuum radiative transfer models are constructed for a grid of models parameterized by central luminosity, filling factor, clump radius, and face-averaged optical depth. The temperature distribution within the clouds is studied as a function of this parameterization. Among our res… ▽ More We present a study of radiative transfer in dusty, clumpy star-forming regions. A series of self-consistent, 3-D, continuum radiative transfer models are constructed for a grid of models parameterized by central luminosity, filling factor, clump radius, and face-averaged optical depth. The temperature distribution within the clouds is studied as a function of this parameterization. Among our results, we find that: (a) the effective optical depth is smaller in clumpy regions than in equivalent homogeneous regions; (b) penetration of radiation is drive by the fraction of open sky (FOS) -- which measures the fraction of solid angle which is devoid of clumps; (c) FOS increases as clump radius increases and filling factor decreases; (d) for FOS > 0.6-0.8 the sky is sufficiently open that the temperature is relatively insensitive to FOS; (e) the physical process by which radiation penetrates is streaming between clumps; (f) filling factor dominates the temperature distribution for large optical depths, and at small clump radii for small optical depths; (g) at lower optical depths, the temperature distribution is most sensitive to filling factors of 1-10 per cent, in accordance with many observations; (h) direct shadowing can be important approximately one clump radius behind a clump. △ Less

Submitted 29 June, 2005; originally announced June 2005.

Comments: 12 pages, 17 figures, accepted by MNRAS

Journal ref: Mon.Not.Roy.Astron.Soc.362:737-747,2005

The Effect of the Cosmic Web on Cluster Weak Lensing Mass Estimates

Authors: Christopher A. Metzler, Martin White, Chris Loken

Abstract: In modern hierarchical theories of structure formation, rich clusters of galaxies form at the vertices of a weblike distribution of matter, with filaments emanating from them to large distances and with smaller objects forming and draining in along these filaments. The amount of mass contained in structure near the cluster can be comparable to the collapsed mass of the cluster itself. As the len… ▽ More In modern hierarchical theories of structure formation, rich clusters of galaxies form at the vertices of a weblike distribution of matter, with filaments emanating from them to large distances and with smaller objects forming and draining in along these filaments. The amount of mass contained in structure near the cluster can be comparable to the collapsed mass of the cluster itself. As the lensing kernel is quite broad along the line of sight around cluster lenses with typical redshifts near z=0.5, structures many Mpc away from the cluster are essentially at the same location as the cluster itself, when considering their effect on the cluster's weak lensing signal. We use large-scale numerical simulations of structure formation in a Lambda-dominated cold dark matter model to quantify the effect that large-scale structure near clusters has upon the cluster masses deduced from weak lensing analysis. A correction for the scatter in possible observed lensing masses should be included when interpreting mass functions from weak lensing surveys. △ Less

Submitted 8 March, 2001; v1 submitted 23 May, 2000; originally announced May 2000.

Comments: 14 pages, 11 figures. LaTeX2e, uses emulateapj.sty and onecolfloat.sty

Journal ref: 2001, ApJ, 547, 560

Weak Gravitational Lensing and Cluster Mass Estimates

Authors: Christopher A. Metzler, Martin White, Michael Norman, Chris Loken

Abstract: Hierarchical theories of structure formation predict that clusters of galaxies should be embedded in a web like structure, with filaments emanating from them to large distances. The amount of mass contained within such filaments near a cluster can be comparable to the collapsed mass of the cluster itself. Diffuse infalling material also contains a large amount of mass. Both these components can… ▽ More Hierarchical theories of structure formation predict that clusters of galaxies should be embedded in a web like structure, with filaments emanating from them to large distances. The amount of mass contained within such filaments near a cluster can be comparable to the collapsed mass of the cluster itself. Diffuse infalling material also contains a large amount of mass. Both these components can contribute to the cluster weak lensing signal. This ``projection bias'' is maximized if a filament lies close to the line-of-sight to a cluster. Using large--scale numerical simulations of structure formation in a cosmological constant dominated cold dark matter model, we show that the projected mass typically exceeds the actual mass by several tens of percent. This effect is significant for attempts to estimate cluster masses through weak lensing observations, and will affect weak lensing surveys aimed at constructing the cluster mass function. △ Less

Submitted 13 April, 1999; originally announced April 1999.

Comments: 4 pages, 3 figures. LaTeX2e, uses emulateapj.sty and onecolfloat.sty. To be submitted to the Astrophysical Journal Letters

Journal ref: Astrophys.J.520:L9,1999

The Cluster SZ -- Mass Correlation

Authors: Christopher A. Metzler

Abstract: N-body + hydrodynamic simulations of galaxy clusters are used to demonstrate a correlation between galaxy cluster mass and the strength of the Sunyaev-Zel'dovich (SZ) effect induced by the cluster. The intrinsic scatter in the correlaton is larger than seen in the cluster mass -- X-ray temperature correlation, but smaller than seen in the correlation between mass (or temperature) and X-ray lumin… ▽ More N-body + hydrodynamic simulations of galaxy clusters are used to demonstrate a correlation between galaxy cluster mass and the strength of the Sunyaev-Zel'dovich (SZ) effect induced by the cluster. The intrinsic scatter in the correlaton is larger than seen in the cluster mass -- X-ray temperature correlation, but smaller than seen in the correlation between mass (or temperature) and X-ray luminosity, as expected. Using the convergence to self-similarity of cluster structure at larger radii, a simple area-averaged SZ value derived from mock SZ maps also correlates well with mass; the intrinsic scatter in this correlation is comparable to that seen in simulations for the mass -- temperature correlation. Such a relation may prove a powerful tool for estimating cluster masses at higher redshifts. △ Less

Submitted 16 December, 1998; originally announced December 1998.

Comments: 4 pages, 2 figures. LaTeX2e, uses emulateapj.sty and onecolfloat.sty. To be submitted to the Astrophysical Journal Letters

Report number: FERMILAB-Pub-98/400-A

Simulations Of Galaxy Clusters With And Without Winds: I. The Structure Of Clusters

Authors: Christopher A. Metzler, August E. Evrard

Abstract: We use gas dynamic simulations to explore the effects of galactic winds on galaxy clusters. Two ensembles of 18 realizations, spanning a decade in temperature, are evolved with and without winds in an underlying biased CDM cosmology. Galaxies are identified as peaks in the initial density field, and are assumed to lose half their initial mass over a Hubble time in winds with an effective tempera… ▽ More We use gas dynamic simulations to explore the effects of galactic winds on galaxy clusters. Two ensembles of 18 realizations, spanning a decade in temperature, are evolved with and without winds in an underlying biased CDM cosmology. Galaxies are identified as peaks in the initial density field, and are assumed to lose half their initial mass over a Hubble time in winds with an effective temperature of 8 keV. The energy input through winds boosts the entropy of the ICM above that generated by infall, resulting in a more extended density profile. The magnitude of this effect increases with decreasing mass or temperature, consistent with observations. The extended nature of the ICM with winds can lead to underestimates of the global baryon fraction, with low-temperature clusters particularly deficient. We calibrate the amplitude of this effect at density contrasts of 170 and 500. On the other hand, the final temperature of the gas is relatively unaffected; a similar mass-temperature relation exists with or without winds. The input wind energy is effectively consumed as work to lift the gas in the dark matter dominated potential. While the dark matter profile exhibits a self-similar form seen in other studies, the galaxy distribution is more centrally concentrated than either the dark matter or gas, and evidences a mild velocity bias. The gradient of galaxies with respect to primordial gas produces strong, negative radial gradients in metallicity within the ICM. △ Less

Submitted 28 October, 1997; originally announced October 1997.

Comments: 46 pages, 18 figures. Uses AAS style aaspp4.sty

Report number: FERMILAB-Pub-97/349-A

Mass Estimates of X-Ray Clusters

Authors: August E. Evrard, Christopher A. Metzler, Julio F. Navarro

Abstract: We use cosmological gas dynamic simulations to investigate the accuracy of galaxy cluster mass estimates based on X-ray observations. The experiments follow the formation of clusters in different cosmological models and include the effects of gravity, pressure gradients, and hydrodynamical shocks. A subset of our ensemble also allows for feedback of mass and energy from galactic winds into the i… ▽ More We use cosmological gas dynamic simulations to investigate the accuracy of galaxy cluster mass estimates based on X-ray observations. The experiments follow the formation of clusters in different cosmological models and include the effects of gravity, pressure gradients, and hydrodynamical shocks. A subset of our ensemble also allows for feedback of mass and energy from galactic winds into the intracluster medium. We find that mass estimates based on the hydrostatic, isothermal beta-model are remarkably accurate when evaluated at radii where the cluster mean density is between 500-2500 times the critical density. Applied to 174 artificial ROSAT images constructed from the simulations, the distribution of the estimated-to-true mass ratio is nearly unbiased and has a standard deviation of 14-29%. The scatter can be considerably reduced (to 8-15%) by using an alternative mass estimator that exploits the tightness of the mass-temperature relation found in the simulations. The improvement over beta-model estimates is due to the elimination of the variance contributed by the gas outer slope parameter. We discuss these findings and their implications for recent measurements of cluster baryon fractions. △ Less

Submitted 10 October, 1995; originally announced October 1995.

Comments: TeX, 24p; 11 Postscript figs. Submitted to the Astrophysical Journal

Report number: FERMILAB-Pub-95/337-A, UofA TAP 95-50

Journal ref: Astrophys.J. 469 (1996) 494

The Velocity Dispersion -- Temperature Correlation from a Limited Cluster Sample

Authors: Christina M. Bird, Richard F. Mushotzky, Christopher A. Metzler

Abstract: Most studies of correlations between X-ray and optical properties of galaxy clusters have used the largest samples of data available, regardless of the morphological types of clusters included. Given the increasing evidence that morphology is related to a cluster's degree of dynamical evolution, we approach the study of X-ray and optical correlations differently. We evaluate the relationship bet… ▽ More Most studies of correlations between X-ray and optical properties of galaxy clusters have used the largest samples of data available, regardless of the morphological types of clusters included. Given the increasing evidence that morphology is related to a cluster's degree of dynamical evolution, we approach the study of X-ray and optical correlations differently. We evaluate the relationship between velocity dispersion and temperature for a limited set of galaxy clusters taken from Bird (1994), which all possess dominant central galaxies and which have been explicitly corrected for the presence of substructure. We find that $σ_r \propto T^{0.61 \pm 0.13}$. We use a Monte Carlo computer routine to estimate the significance of this deviation from the $σ_r \propto T^{0.5}$ relationship predicted by the virial theorem. We find that the simulated correlation is steeper than the observed value only 4\% of the time, suggesting that the deviation is significant. The combination of protogalactic winds and dynamical friction reproduces nearly exactly the observed relationship between $σ_r$ and $T$. △ Less

Submitted 4 May, 1995; originally announced May 1995.

Comments: 27 pages, LaTeX, requires aasms.sty and epsf.tex; accepted for publication in The Astrophysical Journal. 2 PostScript figures, as well as PostScript version of complete paper, available by anonymous ftp from ftp://kula.phsx.ukans.edu/

Journal ref: Astrophys.J. 453 (1995) 40

A Simulation of the Intracluster Medium with Feedback from Cluster Galaxies

Authors: Christopher A. Metzler, August E. Evrard

Abstract: The formation and evolution of an X--ray cluster is studied using a 3--D N-body + hydrodynamical simulation which includes feedback of energy and iron from cluster galaxies. The differences in evolution and final state between the simulation and a similar run without feedback are highlighted. We address the energetics of cluster gas; the distribution of dark matter, gas, and galaxies; the thermo… ▽ More The formation and evolution of an X--ray cluster is studied using a 3--D N-body + hydrodynamical simulation which includes feedback of energy and iron from cluster galaxies. The differences in evolution and final state between the simulation and a similar run without feedback are highlighted. We address the energetics of cluster gas; the distribution of dark matter, gas, and galaxies; the thermodynamic state of cluster gas, the beta model, and the beta discrepancy; x-ray evolution; the validity of estimates of the binding mass and of the baryon fraction of the cluster; dynamics of the galaxy population and velocity bias; and of the evolution of the iron abundance distribution in the cluster. We find that preheating by galactic winds results in a lower core density for the ICM, which in turn makes the simulated cluster fainter at high redshift and undergo rapid luminosity evolution to the present. A strong iron abundance gradient is present, even at high redshift. Velocity bias is present in the galaxy population; this bias combined with the preheating from winds drives the value of $β_{spec}\,=\,σ^{2}/\left(kT/μm_{p}\right)$ to values less than one. The two fluid simulation, modelling infall only, predicts $β_{spec}\,\geq\,1$, less in line with observation. △ Less

Submitted 29 September, 1993; originally announced September 1993.

Comments: 36 pages (15 figures available upon request), LaTeX (modified AAS style included), submitted to the Astrophysical Journal, UM AC 93-12

Journal ref: Astrophys.J.437:564-583,1994