-
FGBERT: Function-Driven Pre-trained Gene Language Model for Metagenomics
Authors:
ChenRui Duan,
Zelin Zang,
Yongjie Xu,
Hang He,
Zihan Liu,
Zijia Song,
Ju-Sheng Zheng,
Stan Z. Li
Abstract:
Metagenomic data, comprising mixed multi-species genomes, are prevalent in diverse environments like oceans and soils, significantly impacting human health and ecological functions. However, current research relies on K-mer representations, limiting the capture of structurally relevant gene contexts. To address these limitations and further our understanding of complex relationships between metage…
▽ More
Metagenomic data, comprising mixed multi-species genomes, are prevalent in diverse environments like oceans and soils, significantly impacting human health and ecological functions. However, current research relies on K-mer representations, limiting the capture of structurally relevant gene contexts. To address these limitations and further our understanding of complex relationships between metagenomic sequences and their functions, we introduce a protein-based gene representation as a context-aware and structure-relevant tokenizer. Our approach includes Masked Gene Modeling (MGM) for gene group-level pre-training, providing insights into inter-gene contextual information, and Triple Enhanced Metagenomic Contrastive Learning (TEM-CL) for gene-level pre-training to model gene sequence-function relationships. MGM and TEM-CL constitute our novel metagenomic language model {\NAME}, pre-trained on 100 million metagenomic sequences. We demonstrate the superiority of our proposed {\NAME} on eight datasets.
△ Less
Submitted 24 February, 2024;
originally announced February 2024.
-
Symmetry-Informed Geometric Representation for Molecules, Proteins, and Crystalline Materials
Authors:
Shengchao Liu,
Weitao Du,
Yan**g Li,
Zhuoxinran Li,
Zhiling Zheng,
Chenru Duan,
Zhiming Ma,
Omar Yaghi,
Anima Anandkumar,
Christian Borgs,
Jennifer Chayes,
Hongyu Guo,
Jian Tang
Abstract:
Artificial intelligence for scientific discovery has recently generated significant interest within the machine learning and scientific communities, particularly in the domains of chemistry, biology, and material discovery. For these scientific problems, molecules serve as the fundamental building blocks, and machine learning has emerged as a highly effective and powerful tool for modeling their g…
▽ More
Artificial intelligence for scientific discovery has recently generated significant interest within the machine learning and scientific communities, particularly in the domains of chemistry, biology, and material discovery. For these scientific problems, molecules serve as the fundamental building blocks, and machine learning has emerged as a highly effective and powerful tool for modeling their geometric structures. Nevertheless, due to the rapidly evolving process of the field and the knowledge gap between science (e.g., physics, chemistry, & biology) and machine learning communities, a benchmarking study on geometrical representation for such data has not been conducted. To address such an issue, in this paper, we first provide a unified view of the current symmetry-informed geometric methods, classifying them into three main categories: invariance, equivariance with spherical frame basis, and equivariance with vector frame basis. Then we propose a platform, coined Geom3D, which enables benchmarking the effectiveness of geometric strategies. Geom3D contains 16 advanced symmetry-informed geometric representation models and 14 geometric pretraining methods over 46 diverse datasets, including small molecules, proteins, and crystalline materials. We hope that Geom3D can, on the one hand, eliminate barriers for machine learning researchers interested in exploring scientific problems; and, on the other hand, provide valuable guidance for researchers in computational chemistry, structural biology, and materials science, aiding in the informed selection of representation techniques for specific applications.
△ Less
Submitted 15 June, 2023;
originally announced June 2023.
-
Active Learning Exploration of Transition Metal Complexes to Discover Method-Insensitive and Synthetically Accessible Chromophores
Authors:
Chenru Duan,
Aditya Nandy,
Gianmarco Terrones,
David W. Kastner,
Heather J. Kulik
Abstract:
Transition metal chromophores with earth-abundant transition metals are an important design target for their applications in lighting and non-toxic bioimaging, but their design is challenged by the scarcity of complexes that simultaneously have optimal target absorption energies in the visible region as well as well-defined ground states. Machine learning (ML) accelerated discovery could overcome…
▽ More
Transition metal chromophores with earth-abundant transition metals are an important design target for their applications in lighting and non-toxic bioimaging, but their design is challenged by the scarcity of complexes that simultaneously have optimal target absorption energies in the visible region as well as well-defined ground states. Machine learning (ML) accelerated discovery could overcome such challenges by enabling screening of a larger space, but is limited by the fidelity of the data used in ML model training, which is typically from a single approximate density functional. To address this limitation, we search for consensus in predictions among 23 density functional approximations across multiple rungs of Jacobs ladder. To accelerate the discovery of complexes with absorption energies in the visible region while minimizing MR character, we use 2D efficient global optimization to sample candidate low-spin chromophores from multi-million complex spaces. Despite the scarcity (i.e., approx. 0.01\%) of potential chromophores in this large chemical space, we identify candidates with high likelihood (i.e., > 10\%) of computational validation as the ML models improve during active learning, representing a 1,000-fold acceleration in discovery. Absorption spectra of promising chromophores from time-dependent density functional theory verify that 2/3 of candidates have the desired excited state properties. The observation that constituent ligands from our leads have demonstrated interesting optical properties in the literature exemplifies the effectiveness of our construction of a realistic design space and active learning approach.
△ Less
Submitted 15 September, 2022; v1 submitted 10 August, 2022;
originally announced August 2022.
-
Network structural origin of instabilities in large complex systems
Authors:
Chao Duan,
Takashi Nishikawa,
Deniz Eroglu,
Adilson E. Motter
Abstract:
A central issue in the study of large complex network systems, such as power grids, financial networks, and ecological systems, is to understand their response to dynamical perturbations. Recent studies recognize that many real networks show nonnormality and that nonnormality can give rise to reactivity--the capacity of a linearly stable system to amplify its response to perturbations, oftentimes…
▽ More
A central issue in the study of large complex network systems, such as power grids, financial networks, and ecological systems, is to understand their response to dynamical perturbations. Recent studies recognize that many real networks show nonnormality and that nonnormality can give rise to reactivity--the capacity of a linearly stable system to amplify its response to perturbations, oftentimes exciting nonlinear instabilities. Here, we identify network structural properties underlying the pervasiveness of nonnormality and reactivity in real directed networks, which we establish using the most extensive data set of such networks studied in this context to date. The identified properties are imbalances between incoming and outgoing network links and paths at each node. Based on this characterization, we develop a theory that quantitatively predicts nonnormality and reactivity and explains the observed pervasiveness. We suggest that these results can be used to design, upgrade, control, and manage networks to avoid or promote network instabilities.
△ Less
Submitted 19 July, 2022; v1 submitted 15 July, 2022;
originally announced July 2022.
