-
Multitask Extension of Geometrically Aligned Transfer Encoder
Authors:
Sung Moon Ko,
Sumin Lee,
Dae-Woong Jeong,
Hyunseung Kim,
Chanhui Lee,
Soorin Yim,
Sehui Han
Abstract:
Molecular datasets often suffer from a lack of data. It is well-known that gathering data is difficult due to the complexity of experimentation or simulation involved. Here, we leverage mutual information across different tasks in molecular data to address this issue. We extend an algorithm that utilizes the geometric characteristics of the encoding space, known as the Geometrically Aligned Transf…
▽ More
Molecular datasets often suffer from a lack of data. It is well-known that gathering data is difficult due to the complexity of experimentation or simulation involved. Here, we leverage mutual information across different tasks in molecular data to address this issue. We extend an algorithm that utilizes the geometric characteristics of the encoding space, known as the Geometrically Aligned Transfer Encoder (GATE), to a multi-task setup. Thus, we connect multiple molecular tasks by aligning the curved coordinates onto locally flat coordinates, ensuring the flow of information from source tasks to support performance on target data.
△ Less
Submitted 3 May, 2024;
originally announced May 2024.
-
Geometrically Aligned Transfer Encoder for Inductive Transfer in Regression Tasks
Authors:
Sung Moon Ko,
Sumin Lee,
Dae-Woong Jeong,
Woohyung Lim,
Sehui Han
Abstract:
Transfer learning is a crucial technique for handling a small amount of data that is potentially related to other abundant data. However, most of the existing methods are focused on classification tasks using images and language datasets. Therefore, in order to expand the transfer learning scheme to regression tasks, we propose a novel transfer technique based on differential geometry, namely the…
▽ More
Transfer learning is a crucial technique for handling a small amount of data that is potentially related to other abundant data. However, most of the existing methods are focused on classification tasks using images and language datasets. Therefore, in order to expand the transfer learning scheme to regression tasks, we propose a novel transfer technique based on differential geometry, namely the Geometrically Aligned Transfer Encoder (GATE). In this method, we interpret the latent vectors from the model to exist on a Riemannian curved manifold. We find a proper diffeomorphism between pairs of tasks to ensure that every arbitrary point maps to a locally flat coordinate in the overlap** region, allowing the transfer of knowledge from the source to the target data. This also serves as an effective regularizer for the model to behave in extrapolation regions. In this article, we demonstrate that GATE outperforms conventional methods and exhibits stable behavior in both the latent space and extrapolation regions for various molecular graph datasets.
△ Less
Submitted 10 October, 2023;
originally announced October 2023.
-
3D Denoisers are Good 2D Teachers: Molecular Pretraining via Denoising and Cross-Modal Distillation
Authors:
Sungjun Cho,
Dae-Woong Jeong,
Sung Moon Ko,
**woo Kim,
Sehui Han,
Seunghoon Hong,
Honglak Lee,
Moontae Lee
Abstract:
Pretraining molecular representations from large unlabeled data is essential for molecular property prediction due to the high cost of obtaining ground-truth labels. While there exist various 2D graph-based molecular pretraining approaches, these methods struggle to show statistically significant gains in predictive performance. Recent work have thus instead proposed 3D conformer-based pretraining…
▽ More
Pretraining molecular representations from large unlabeled data is essential for molecular property prediction due to the high cost of obtaining ground-truth labels. While there exist various 2D graph-based molecular pretraining approaches, these methods struggle to show statistically significant gains in predictive performance. Recent work have thus instead proposed 3D conformer-based pretraining under the task of denoising, which led to promising results. During downstream finetuning, however, models trained with 3D conformers require accurate atom-coordinates of previously unseen molecules, which are computationally expensive to acquire at scale. In light of this limitation, we propose D&D, a self-supervised molecular representation learning framework that pretrains a 2D graph encoder by distilling representations from a 3D denoiser. With denoising followed by cross-modal knowledge distillation, our approach enjoys use of knowledge obtained from denoising as well as painless application to downstream tasks with no access to accurate conformers. Experiments on real-world molecular property prediction datasets show that the graph encoder trained via D&D can infer 3D information based on the 2D graph and shows superior performance and label-efficiency against other baselines.
△ Less
Submitted 7 September, 2023;
originally announced September 2023.
-
Grou**-matrix based Graph Pooling with Adaptive Number of Clusters
Authors:
Sung Moon Ko,
Sungjun Cho,
Dae-Woong Jeong,
Sehui Han,
Moontae Lee,
Honglak Lee
Abstract:
Graph pooling is a crucial operation for encoding hierarchical structures within graphs. Most existing graph pooling approaches formulate the problem as a node clustering task which effectively captures the graph topology. Conventional methods ask users to specify an appropriate number of clusters as a hyperparameter, then assume that all input graphs share the same number of clusters. In inductiv…
▽ More
Graph pooling is a crucial operation for encoding hierarchical structures within graphs. Most existing graph pooling approaches formulate the problem as a node clustering task which effectively captures the graph topology. Conventional methods ask users to specify an appropriate number of clusters as a hyperparameter, then assume that all input graphs share the same number of clusters. In inductive settings where the number of clusters can vary, however, the model should be able to represent this variation in its pooling layers in order to learn suitable clusters. Thus we propose GMPool, a novel differentiable graph pooling architecture that automatically determines the appropriate number of clusters based on the input data. The main intuition involves a grou** matrix defined as a quadratic form of the pooling operator, which induces use of binary classification probabilities of pairwise combinations of nodes. GMPool obtains the pooling operator by first computing the grou** matrix, then decomposing it. Extensive evaluations on molecular property prediction tasks demonstrate that our method outperforms conventional methods.
△ Less
Submitted 7 September, 2022;
originally announced September 2022.
-
The rotation curve of a point particle in stringy gravity
Authors:
Sung Moon Ko,
Jeong-Hyuck Park,
Minwoo Suh
Abstract:
Double Field Theory suggests to view the whole massless sector of closed strings as the gravitational unity. The fundamental symmetries therein, including the $\mathbf{O}(D,D)$ covariance, can determine unambiguously how the Standard Model as well as a relativistic point particle should couple to the closed string massless sector. The theory also refines the notion of singularity. We consider the…
▽ More
Double Field Theory suggests to view the whole massless sector of closed strings as the gravitational unity. The fundamental symmetries therein, including the $\mathbf{O}(D,D)$ covariance, can determine unambiguously how the Standard Model as well as a relativistic point particle should couple to the closed string massless sector. The theory also refines the notion of singularity. We consider the most general, spherically symmetric, asymptotically flat, static vacuum solution to ${D=4}$ Double Field Theory, which contains three free parameters and consequently generalizes the Schwarzschild geometry. Analyzing the circular geodesic of a point particle in string frame, we obtain the orbital velocity as a function of $R/(M_{\scriptscriptstyle{\infty}}G)$ which is the dimensionless radial variable normalized by mass. The rotation curve generically features a maximum and thus non-Keplerian over a finite range, while becoming asymptotically Keplerian at infinity, $R/(M_{\scriptscriptstyle{\infty}}G)\rightarrow \infty$. The adoption of the string frame rather than Einstein frame is the consequence of the fundamental symmetry principle. Our result opens up a new scheme to solve the dark matter/energy problems by modifying General Relativity at `short' range of $R/(M_{\scriptscriptstyle{\infty}}G)$.
△ Less
Submitted 20 May, 2017; v1 submitted 29 June, 2016;
originally announced June 2016.
-
Dynamics of Perturbations in Double Field Theory & Non-Relativistic String Theory
Authors:
Sung Moon Ko,
Charles Melby-Thompson,
Rene Meyer,
Jeong-Hyuck Park
Abstract:
Double Field Theory provides a geometric framework capable of describing string theory backgrounds that cannot be understood purely in terms of Riemannian geometry -- not only globally (`non-geometry'), but even locally (`non-Riemannian'). In this work, we show that the non-relativistic closed string theory of Gomis and Ooguri [1] arises precisely as such a non-Riemannian string background, and th…
▽ More
Double Field Theory provides a geometric framework capable of describing string theory backgrounds that cannot be understood purely in terms of Riemannian geometry -- not only globally (`non-geometry'), but even locally (`non-Riemannian'). In this work, we show that the non-relativistic closed string theory of Gomis and Ooguri [1] arises precisely as such a non-Riemannian string background, and that the Gomis-Ooguri sigma model is equivalent to the Double Field Theory sigma model of [2] on this background. We further show that the target-space formulation of Double Field Theory on this non-Riemannian background correctly reproduces the appropriate sector of the Gomis-Ooguri string spectrum. To do this, we develop a general semi-covariant formalism describing perturbations in Double Field Theory. We derive compact expressions for the linearized equations of motion around a generic on-shell background, and construct the corresponding fluctuation Lagrangian in terms of novel completely covariant second order differential operators. We also present a new non-Riemannian solution featuring Schrödinger conformal symmetry.
△ Less
Submitted 1 December, 2015; v1 submitted 5 August, 2015;
originally announced August 2015.
-
Superconformal Yang-Mills quantum mechanics and Calogero model with OSp(N|2,R) symmetry
Authors:
Neil B. Copland,
Sung Moon Ko,
Jeong-Hyuck Park
Abstract:
In spacetime dimension two, pure Yang-Mills possesses no physical degrees of freedom, and consequently it admits a supersymmetric extension to couple to an arbitrary number, N say, of Majorana-Weyl gauginos. This results in (N,0) super Yang-Mills. Further, its dimensional reduction to mechanics doubles the number of supersymmetries, from N to N+N, to include conformal supercharges, and leads to a…
▽ More
In spacetime dimension two, pure Yang-Mills possesses no physical degrees of freedom, and consequently it admits a supersymmetric extension to couple to an arbitrary number, N say, of Majorana-Weyl gauginos. This results in (N,0) super Yang-Mills. Further, its dimensional reduction to mechanics doubles the number of supersymmetries, from N to N+N, to include conformal supercharges, and leads to a superconformal Yang-Mills quantum mechanics with symmetry group OSp(N|2,R). We comment on its connection to AdS_2 \times S^{N-1} and reduction to a supersymmetric Calogero model.
△ Less
Submitted 6 July, 2012; v1 submitted 17 May, 2012;
originally announced May 2012.
