Improving the Reconstruction of Disentangled Representation Learners via Multi-Stage Modeling
Authors:
Akash Srivastava,
Yamini Bansal,
Yukun Ding,
Cole Lincoln Hurwitz,
Kai Xu,
Bernhard Egger,
Prasanna Sattigeri,
Joshua B. Tenenbaum,
Phuong Le,
Arun Prakash R,
Nengfeng Zhou,
Joel Vaughan,
Yaquan Wang,
Anwesha Bhattacharyya,
Kristjan Greenewald,
David D. Cox,
Dan Gutfreund
Abstract:
Current autoencoder-based disentangled representation learning methods achieve disentanglement by penalizing the (aggregate) posterior to encourage statistical independence of the latent factors. This approach introduces a trade-off between disentangled representation learning and reconstruction quality since the model does not have enough capacity to learn correlated latent variables that capture…
▽ More
Current autoencoder-based disentangled representation learning methods achieve disentanglement by penalizing the (aggregate) posterior to encourage statistical independence of the latent factors. This approach introduces a trade-off between disentangled representation learning and reconstruction quality since the model does not have enough capacity to learn correlated latent variables that capture detail information present in most image data. To overcome this trade-off, we present a novel multi-stage modeling approach where the disentangled factors are first learned using a penalty-based disentangled representation learning method; then, the low-quality reconstruction is improved with another deep generative model that is trained to model the missing correlated latent variables, adding detail information while maintaining conditioning on the previously learned disentangled factors. Taken together, our multi-stage modelling approach results in a single, coherent probabilistic model that is theoretically justified by the principal of D-separation and can be realized with a variety of model classes including likelihood-based models such as variational autoencoders, implicit models such as generative adversarial networks, and tractable models like normalizing flows or mixtures of Gaussians. We demonstrate that our multi-stage model has higher reconstruction quality than current state-of-the-art methods with equivalent disentanglement performance across multiple standard benchmarks. In addition, we apply the multi-stage model to generate synthetic tabular datasets, showcasing an enhanced performance over benchmark models across a variety of metrics. The interpretability analysis further indicates that the multi-stage model can effectively uncover distinct and meaningful features of variations from which the original distribution can be recovered.
△ Less
Submitted 3 April, 2024; v1 submitted 25 October, 2020;
originally announced October 2020.
Scalable Spike Source Localization in Extracellular Recordings using Amortized Variational Inference
Authors:
Cole L. Hurwitz,
Kai Xu,
Akash Srivastava,
Alessio P. Buccino,
Matthias H. Hennig
Abstract:
Determining the positions of neurons in an extracellular recording is useful for investigating functional properties of the underlying neural circuitry. In this work, we present a Bayesian modelling approach for localizing the source of individual spikes on high-density, microelectrode arrays. To allow for scalable inference, we implement our model as a variational autoencoder and perform amortize…
▽ More
Determining the positions of neurons in an extracellular recording is useful for investigating functional properties of the underlying neural circuitry. In this work, we present a Bayesian modelling approach for localizing the source of individual spikes on high-density, microelectrode arrays. To allow for scalable inference, we implement our model as a variational autoencoder and perform amortized variational inference. We evaluate our method on both biophysically realistic simulated and real extracellular datasets, demonstrating that it is more accurate than and can improve spike sorting performance over heuristic localization methods such as center of mass.
△ Less
Submitted 26 January, 2022; v1 submitted 29 May, 2019;
originally announced May 2019.
