In-Context Learning of Physical Properties: Few-Shot Adaptation to Out-of-Distribution Molecular Graphs
Authors:
Grzegorz Kaszuba,
Amirhossein D. Naghdi,
Dario Massa,
Stefanos Papanikolaou,
Andrzej Jaszkiewicz,
Piotr Sankowski
Abstract:
Large language models manifest the ability of few-shot adaptation to a sequence of provided examples. This behavior, known as in-context learning, allows for performing nontrivial machine learning tasks during inference only. In this work, we address the question: can we leverage in-context learning to predict out-of-distribution materials properties? However, this would not be possible for struct…
▽ More
Large language models manifest the ability of few-shot adaptation to a sequence of provided examples. This behavior, known as in-context learning, allows for performing nontrivial machine learning tasks during inference only. In this work, we address the question: can we leverage in-context learning to predict out-of-distribution materials properties? However, this would not be possible for structure property prediction tasks unless an effective method is found to pass atomic-level geometric features to the transformer model. To address this problem, we employ a compound model in which GPT-2 acts on the output of geometry-aware graph neural networks to adapt in-context information. To demonstrate our model's capabilities, we partition the QM9 dataset into sequences of molecules that share a common substructure and use them for in-context learning. This approach significantly improves the performance of the model on out-of-distribution examples, surpassing the one of general graph neural network models.
△ Less
Submitted 3 June, 2024;
originally announced June 2024.
Compositional Search of Stable Crystalline Structures in Multi-Component Alloys Using Generative Diffusion Models
Authors:
Grzegorz Kaszuba,
Amirhossein Naghdi Dorabati,
Stefanos Papanikolaou,
Andrzej Jaszkiewicz,
Piotr Sankowski
Abstract:
Exploring the vast composition space of multi-component alloys presents a challenging task for both \textit{ab initio} (first principles) and experimental methods due to the time-consuming procedures involved. This ultimately impedes the discovery of novel, stable materials that may display exceptional properties. Here, the Crystal Diffusion Variational Autoencoder (CDVAE) model is adapted to char…
▽ More
Exploring the vast composition space of multi-component alloys presents a challenging task for both \textit{ab initio} (first principles) and experimental methods due to the time-consuming procedures involved. This ultimately impedes the discovery of novel, stable materials that may display exceptional properties. Here, the Crystal Diffusion Variational Autoencoder (CDVAE) model is adapted to characterize the stable compositions of a well studied multi-component alloy, NiFeCr, with two distinct crystalline phases known to be stable across its compositional space. To this end, novel extensions to CDVAE were proposed, enhancing the model's ability to reconstruct configurations from their latent space within the test set by approximately 30\% . A fact that increases a model's probability of discovering new materials when dealing with various crystalline structures. Afterwards, the new model is applied for materials generation, demonstrating excellent agreement in identifying stable configurations within the ternary phase space when compared to first principles data. Finally, a computationally efficient framework for inverse design is proposed, employing Molecular Dynamics (MD) simulations of multi-component alloys with reliable interatomic potentials, enabling the optimization of materials property across the phase space.
△ Less
Submitted 26 December, 2023;
originally announced December 2023.
