-
Metropolis Sampling for Constrained Diffusion Models
Authors:
Nic Fishman,
Leo Klarner,
Emile Mathieu,
Michael Hutchinson,
Valentin de Bortoli
Abstract:
Denoising diffusion models have recently emerged as the predominant paradigm for generative modelling on image domains. In addition, their extension to Riemannian manifolds has facilitated a range of applications across the natural sciences. While many of these problems stand to benefit from the ability to specify arbitrary, domain-informed constraints, this setting is not covered by the existing…
▽ More
Denoising diffusion models have recently emerged as the predominant paradigm for generative modelling on image domains. In addition, their extension to Riemannian manifolds has facilitated a range of applications across the natural sciences. While many of these problems stand to benefit from the ability to specify arbitrary, domain-informed constraints, this setting is not covered by the existing (Riemannian) diffusion model methodology. Recent work has attempted to address this issue by constructing novel noising processes based on the reflected Brownian motion and logarithmic barrier methods. However, the associated samplers are either computationally burdensome or only apply to convex subsets of Euclidean space. In this paper, we introduce an alternative, simple noising scheme based on Metropolis sampling that affords substantial gains in computational efficiency and empirical performance compared to the earlier samplers. Of independent interest, we prove that this new process corresponds to a valid discretisation of the reflected Brownian motion. We demonstrate the scalability and flexibility of our approach on a range of problem settings with convex and non-convex constraints, including applications from geospatial modelling, robotics and protein design.
△ Less
Submitted 9 November, 2023; v1 submitted 11 July, 2023;
originally announced July 2023.
-
Diffusion Models for Constrained Domains
Authors:
Nic Fishman,
Leo Klarner,
Valentin De Bortoli,
Emile Mathieu,
Michael Hutchinson
Abstract:
Denoising diffusion models are a novel class of generative algorithms that achieve state-of-the-art performance across a range of domains, including image generation and text-to-image tasks. Building on this success, diffusion models have recently been extended to the Riemannian manifold setting, broadening their applicability to a range of problems from the natural and engineering sciences. Howev…
▽ More
Denoising diffusion models are a novel class of generative algorithms that achieve state-of-the-art performance across a range of domains, including image generation and text-to-image tasks. Building on this success, diffusion models have recently been extended to the Riemannian manifold setting, broadening their applicability to a range of problems from the natural and engineering sciences. However, these Riemannian diffusion models are built on the assumption that their forward and backward processes are well-defined for all times, preventing them from being applied to an important set of tasks that consider manifolds defined via a set of inequality constraints. In this work, we introduce a principled framework to bridge this gap. We present two distinct noising processes based on (i) the logarithmic barrier metric and (ii) the reflected Brownian motion induced by the constraints. As existing diffusion model techniques cannot be applied in this setting, we derive new tools to define such models in our framework. We then demonstrate the practical utility of our methods on a number of synthetic and real-world tasks, including applications from robotics and protein design.
△ Less
Submitted 7 March, 2024; v1 submitted 11 April, 2023;
originally announced April 2023.
-
Human mobility networks reveal increased segregation in large cities
Authors:
Hamed Nilforoshan,
Wenli Looi,
Emma Pierson,
Blanca Villanueva,
Nic Fishman,
Yiling Chen,
John Sholar,
Beth Redbird,
David Grusky,
Jure Leskovec
Abstract:
A long-standing expectation is that large, dense, and cosmopolitan areas support socioeconomic mixing and exposure between diverse individuals. It has been difficult to assess this hypothesis because past approaches to measuring socioeconomic mixing have relied on static residential housing data rather than real-life exposures between people at work, in places of leisure, and in home neighborhoods…
▽ More
A long-standing expectation is that large, dense, and cosmopolitan areas support socioeconomic mixing and exposure between diverse individuals. It has been difficult to assess this hypothesis because past approaches to measuring socioeconomic mixing have relied on static residential housing data rather than real-life exposures between people at work, in places of leisure, and in home neighborhoods. Here we develop a new measure of exposure segregation (ES) that captures the socioeconomic diversity of everyday encounters. Leveraging cell phone mobility data to represent 1.6 billion exposures among 9.6 million people in the United States, we measure exposure segregation across 382 Metropolitan Statistical Areas (MSAs) and 2829 counties. We discover that exposure segregation is 67% higher in the 10 largest Metropolitan Statistical Areas (MSAs) than in small MSAs with fewer than 100,000 residents. This means that, contrary to expectation, residents of large cosmopolitan areas have significantly less exposure to diverse individuals. Second, we find evidence that large cities offer a greater choice of differentiated spaces targeted to specific socioeconomic groups, a dynamic that accounts for this increase in everyday socioeconomic segregation. Third, we discover that this segregation-increasing effect is countered when a city's hubs (e.g. shop** malls) are positioned to bridge diverse neighborhoods and thus attract people of all socioeconomic statuses. Overall, our findings challenge a long-standing conjecture in human geography and urban design, and highlight how built environment can both prevent and facilitate exposure between diverse individuals.
△ Less
Submitted 24 July, 2023; v1 submitted 13 October, 2022;
originally announced October 2022.
-
Should attention be all we need? The epistemic and ethical implications of unification in machine learning
Authors:
Nic Fishman,
Leif Hancox-Li
Abstract:
"Attention is all you need" has become a fundamental precept in machine learning research. Originally designed for machine translation, transformers and the attention mechanisms that underpin them now find success across many problem domains. With the apparent domain-agnostic success of transformers, many researchers are excited that similar model architectures can be successfully deployed across…
▽ More
"Attention is all you need" has become a fundamental precept in machine learning research. Originally designed for machine translation, transformers and the attention mechanisms that underpin them now find success across many problem domains. With the apparent domain-agnostic success of transformers, many researchers are excited that similar model architectures can be successfully deployed across diverse applications in vision, language and beyond. We consider the benefits and risks of these waves of unification on both epistemic and ethical fronts. On the epistemic side, we argue that many of the arguments in favor of unification in the natural sciences fail to transfer over to the machine learning case, or transfer over only under assumptions that might not hold. Unification also introduces epistemic risks related to portability, path dependency, methodological diversity, and increased black-boxing. On the ethical side, we discuss risks emerging from epistemic concerns, further marginalizing underrepresented perspectives, the centralization of power, and having fewer models across more domains of application
△ Less
Submitted 9 May, 2022;
originally announced May 2022.
