-
Latent Directions: A Simple Pathway to Bias Mitigation in Generative AI
Authors:
Carolina Lopez Olmos,
Alexandros Neophytou,
Sunando Sengupta,
Dim P. Papadopoulos
Abstract:
Mitigating biases in generative AI and, particularly in text-to-image models, is of high importance given their growing implications in society. The biased datasets used for training pose challenges in ensuring the responsible development of these models, and mitigation through hard prompting or embedding alteration, are the most common present solutions. Our work introduces a novel approach to ac…
▽ More
Mitigating biases in generative AI and, particularly in text-to-image models, is of high importance given their growing implications in society. The biased datasets used for training pose challenges in ensuring the responsible development of these models, and mitigation through hard prompting or embedding alteration, are the most common present solutions. Our work introduces a novel approach to achieve diverse and inclusive synthetic images by learning a direction in the latent space and solely modifying the initial Gaussian noise provided for the diffusion process. Maintaining a neutral prompt and untouched embeddings, this approach successfully adapts to diverse debiasing scenarios, such as geographical biases. Moreover, our work proves it is possible to linearly combine these learned latent directions to introduce new mitigations, and if desired, integrate it with text embedding adjustments. Furthermore, text-to-image models lack transparency for assessing bias in outputs, unless visually inspected. Thus, we provide a tool to empower developers to select their desired concepts to mitigate. The project page with code is available online.
△ Less
Submitted 10 June, 2024;
originally announced June 2024.
-
Effect of coat-protein concentration on the self-assembly of bacteriophage MS2 capsids around RNA
Authors:
LaNell A. Williams,
Andreas Neophytou,
Rees F. Garmann,
Dwaipayan Chakrabarti,
Vinothan N. Manoharan
Abstract:
Self-assembly is a vital part of the life cycle of certain icosahedral RNA viruses. Furthermore, the assembly process can be harnessed to make icosahedral virus-like particles (VLPs) from coat protein and RNA in vitro. Although much previous work has explored the effects of RNA-protein interactions on the assembly products, relatively little research has explored the effects of coat-protein concen…
▽ More
Self-assembly is a vital part of the life cycle of certain icosahedral RNA viruses. Furthermore, the assembly process can be harnessed to make icosahedral virus-like particles (VLPs) from coat protein and RNA in vitro. Although much previous work has explored the effects of RNA-protein interactions on the assembly products, relatively little research has explored the effects of coat-protein concentration. We mix coat protein and RNA from bacteriophage MS2, and we use a combination of gel electrophoresis, dynamic light scattering, and transmission electron microscopy to investigate the assembly products. We show that with increasing coat-protein concentration, the products transition from well-formed MS2 VLPs to ``monster'' particles consisting of multiple partial capsids to RNA-protein condensates consisting of large networks of RNA and partially assembled capsids. We argue that the transition from well-formed to monster particles arises because the assembly follows a nucleation-and-growth pathway in which the nucleation rate depends sensitively on the coat-protein concentration, such that at high protein concentrations, multiple nuclei can form on each RNA strand. To understand the formation of the condensates, which occurs at even higher coat-protein concentrations, we use Monte Carlo simulations with coarse-grained models of capsomers and RNA. These simulations suggest that the the formation of condensates occurs by the adsorption of protein to the RNA followed by the assembly of capsids. Multiple RNA molecules can become trapped when a capsid grows from capsomers attached to two different RNA molecules or when excess protein bridges together growing capsids on different RNA molecules. Our results provide insight into an important biophysical process and could inform design rules for making VLPs for various applications.
△ Less
Submitted 16 January, 2024; v1 submitted 9 July, 2023;
originally announced July 2023.
-
A Bayesian Joint Model for Compositional Mediation Effect Selection in Microbiome Data
Authors:
**gyan Fu,
Matthew D. Koslovsky,
Andreas M. Neophytou,
Marina Vannucci
Abstract:
Analyzing multivariate count data generated by high-throughput sequencing technology in microbiome research studies is challenging due to the high-dimensional and compositional structure of the data and overdispersion. In practice, researchers are often interested in investigating how the microbiome may mediate the relation between an assigned treatment and an observed phenotypic response. Existin…
▽ More
Analyzing multivariate count data generated by high-throughput sequencing technology in microbiome research studies is challenging due to the high-dimensional and compositional structure of the data and overdispersion. In practice, researchers are often interested in investigating how the microbiome may mediate the relation between an assigned treatment and an observed phenotypic response. Existing approaches designed for compositional mediation analysis are unable to simultaneously determine the presence of direct effects, relative indirect effects, and overall indirect effects, while quantifying their uncertainty. We propose a formulation of a Bayesian joint model for compositional data that allows for the identification, estimation, and uncertainty quantification of various causal estimands in high-dimensional mediation analysis. We conduct simulation studies and compare our method's mediation effects selection performance with existing methods. Finally, we apply our method to a benchmark data set investigating the sub-therapeutic antibiotic treatment effect on body weight in early-life mice.
△ Less
Submitted 26 April, 2023; v1 submitted 22 September, 2022;
originally announced September 2022.
-
NP-Match: When Neural Processes meet Semi-Supervised Learning
Authors:
Jianfeng Wang,
Thomas Lukasiewicz,
Daniela Massiceti,
Xiaolin Hu,
Vladimir Pavlovic,
Alexandros Neophytou
Abstract:
Semi-supervised learning (SSL) has been widely explored in recent years, and it is an effective way of leveraging unlabeled data to reduce the reliance on labeled data. In this work, we adjust neural processes (NPs) to the semi-supervised image classification task, resulting in a new method named NP-Match. NP-Match is suited to this task for two reasons. Firstly, NP-Match implicitly compares data…
▽ More
Semi-supervised learning (SSL) has been widely explored in recent years, and it is an effective way of leveraging unlabeled data to reduce the reliance on labeled data. In this work, we adjust neural processes (NPs) to the semi-supervised image classification task, resulting in a new method named NP-Match. NP-Match is suited to this task for two reasons. Firstly, NP-Match implicitly compares data points when making predictions, and as a result, the prediction of each unlabeled data point is affected by the labeled data points that are similar to it, which improves the quality of pseudo-labels. Secondly, NP-Match is able to estimate uncertainty that can be used as a tool for selecting unlabeled samples with reliable pseudo-labels. Compared with uncertainty-based SSL methods implemented with Monte Carlo (MC) dropout, NP-Match estimates uncertainty with much less computational overhead, which can save time at both the training and the testing phases. We conducted extensive experiments on four public datasets, and NP-Match outperforms state-of-the-art (SOTA) results or achieves competitive results on them, which shows the effectiveness of NP-Match and its potential for SSL.
△ Less
Submitted 3 July, 2022;
originally announced July 2022.
-
Cross-modal Spectrum Transformation Network For Acoustic Scene classification
Authors:
Yang Liu,
Alexandros Neophytou,
Sunando Sengupta,
Eric Sommerlade
Abstract:
Convolutional neural networks (CNNs) with log-mel spectrum features have shown promising results for acoustic scene classification tasks. However, the performance of these CNN based classifiers is still lacking as they do not generalise well for unknown environments. To address this issue, we introduce an acoustic spectrum transformation network where traditional log-mel spectrums are transformed…
▽ More
Convolutional neural networks (CNNs) with log-mel spectrum features have shown promising results for acoustic scene classification tasks. However, the performance of these CNN based classifiers is still lacking as they do not generalise well for unknown environments. To address this issue, we introduce an acoustic spectrum transformation network where traditional log-mel spectrums are transformed into imagined visual features (IVF). The imagined visual features are learned by exploiting the relationship between audio and visual features present in video recordings. An auto-encoder is used to encode images as visual features and a transformation network learns how to generate imagined visual features from log-mel. Our model is trained on a large dataset of Youtube videos. We test our proposed method on the scene classification task of DCASE and ESC-50, where our method outperforms other spectrum features, especially for unseen environments.
△ Less
Submitted 13 August, 2021;
originally announced August 2021.
-
Relighting Images in the Wild with a Self-Supervised Siamese Auto-Encoder
Authors:
Yang Liu,
Alexandros Neophytou,
Sunando Sengupta,
Eric Sommerlade
Abstract:
We propose a self-supervised method for image relighting of single view images in the wild. The method is based on an auto-encoder which deconstructs an image into two separate encodings, relating to the scene illumination and content, respectively. In order to disentangle this embedding information without supervision, we exploit the assumption that some augmentation operations do not affect the…
▽ More
We propose a self-supervised method for image relighting of single view images in the wild. The method is based on an auto-encoder which deconstructs an image into two separate encodings, relating to the scene illumination and content, respectively. In order to disentangle this embedding information without supervision, we exploit the assumption that some augmentation operations do not affect the image content and only affect the direction of the light. A novel loss function, called spherical harmonic loss, is introduced that forces the illumination embedding to convert to a spherical harmonic vector. We train our model on large-scale datasets such as Youtube 8M and CelebA. Our experiments show that our method can correctly estimate scene illumination and realistically re-light input images, without any supervision or a prior shape model. Compared to supervised methods, our approach has similar performance and avoids common lighting artifacts.
△ Less
Submitted 23 August, 2021; v1 submitted 11 December, 2020;
originally announced December 2020.
