-
OrthoNets: Orthogonal Channel Attention Networks
Authors:
Hadi Salman,
Caleb Parks,
Matthew Swan,
John Gauch
Abstract:
Designing an effective channel attention mechanism implores one to find a lossy-compression method allowing for optimal feature representation. Despite recent progress in the area, it remains an open problem. FcaNet, the current state-of-the-art channel attention mechanism, attempted to find such an information-rich compression using Discrete Cosine Transforms (DCTs). One drawback of FcaNet is tha…
▽ More
Designing an effective channel attention mechanism implores one to find a lossy-compression method allowing for optimal feature representation. Despite recent progress in the area, it remains an open problem. FcaNet, the current state-of-the-art channel attention mechanism, attempted to find such an information-rich compression using Discrete Cosine Transforms (DCTs). One drawback of FcaNet is that there is no natural choice of the DCT frequencies. To circumvent this issue, FcaNet experimented on ImageNet to find optimal frequencies. We hypothesize that the choice of frequency plays only a supporting role and the primary driving force for the effectiveness of their attention filters is the orthogonality of the DCT kernels. To test this hypothesis, we construct an attention mechanism using randomly initialized orthogonal filters. Integrating this mechanism into ResNet, we create OrthoNet. We compare OrthoNet to FcaNet (and other attention mechanisms) on Birds, MS-COCO, and Places356 and show superior performance. On the ImageNet dataset, our method competes with or surpasses the current state-of-the-art. Our results imply that an optimal choice of filter is elusive and generalization can be achieved with a sufficiently large number of orthogonal filters. We further investigate other general principles for implementing channel attention, such as its position in the network and channel grou**s. Our code is publicly available at https://github.com/hady1011/OrthoNets/
△ Less
Submitted 6 November, 2023; v1 submitted 6 November, 2023;
originally announced November 2023.
-
WaveNets: Wavelet Channel Attention Networks
Authors:
Hadi Salman,
Caleb Parks,
Shi Yin Hong,
Justin Zhan
Abstract:
Channel Attention reigns supreme as an effective technique in the field of computer vision. However, the proposed channel attention by SENet suffers from information loss in feature learning caused by the use of Global Average Pooling (GAP) to represent channels as scalars. Thus, designing effective channel attention mechanisms requires finding a solution to enhance features preservation in modeli…
▽ More
Channel Attention reigns supreme as an effective technique in the field of computer vision. However, the proposed channel attention by SENet suffers from information loss in feature learning caused by the use of Global Average Pooling (GAP) to represent channels as scalars. Thus, designing effective channel attention mechanisms requires finding a solution to enhance features preservation in modeling channel inter-dependencies. In this work, we utilize Wavelet transform compression as a solution to the channel representation problem. We first test wavelet transform as an Auto-Encoder model equipped with conventional channel attention module. Next, we test wavelet transform as a standalone channel compression method. We prove that global average pooling is equivalent to the recursive approximate Haar wavelet transform. With this proof, we generalize channel attention using Wavelet compression and name it WaveNet. Implementation of our method can be embedded within existing channel attention methods with a couple of lines of code. We test our proposed method using ImageNet dataset for image classification task. Our method outperforms the baseline SENet, and achieves the state-of-the-art results. Our code implementation is publicly available at https://github.com/hady1011/WaveNet-C.
△ Less
Submitted 12 March, 2024; v1 submitted 4 November, 2022;
originally announced November 2022.
-
Mining for Potent Inhibitors through Artificial Intelligence and Physics: A Unified Methodology for Ligand Based and Structure Based Drug Design
Authors:
Jie Li,
Oufan Zhang,
Yingze Wang,
Kunyang Sun,
Xingyi Guan,
Dorian Bagni,
Mojtaba Haghighatlari,
Fiona L. Kearns,
Conor Parks,
Rommie E. Amaro,
Teresa Head-Gordon
Abstract:
The viability of a new drug molecule is a time and resource intensive task that makes computer-aided assessments a vital approach to rapid drug discovery. Here we develop a machine learning algorithm, iMiner, that generates novel inhibitor molecules for target proteins by combining deep reinforcement learning with real-time 3D molecular docking using AutoDock Vina, thereby simultaneously creating…
▽ More
The viability of a new drug molecule is a time and resource intensive task that makes computer-aided assessments a vital approach to rapid drug discovery. Here we develop a machine learning algorithm, iMiner, that generates novel inhibitor molecules for target proteins by combining deep reinforcement learning with real-time 3D molecular docking using AutoDock Vina, thereby simultaneously creating chemical novelty while constraining molecules for shape and molecular compatibility with target active sites. Moreover, through the use of various types of reward functions, we can generate new molecules that are chemically similar to a target ligand, which can be grown from known protein bound fragments, as well as to create molecules that enforce interactions with target residues in the protein active site. The iMiner algorithm is embedded in a composite workflow that filters out Pan-assay interference compounds, Lipinski rule violations, and poor synthetic accessibility, with options for cross-validation against other docking scoring functions and automation of a molecular dynamics simulation to measure pose stability. Because our approach only relies on the structure of the target protein, iMiner can be easily adapted for future development of other inhibitors or small molecule therapeutics of any target protein.
△ Less
Submitted 10 January, 2024; v1 submitted 4 October, 2021;
originally announced October 2021.
-
Interpolation and Sampling in Analytic Tent Spaces
Authors:
Caleb Parks
Abstract:
Introduced by Coifman, Meyer, and Stein, the tent spaces have seen wide applications in Harmonic Analysis. Their analytic cousins have seen some applications involving the derivatives of Hardy space functions. Moreover, the tent spaces have been a recent focus of research. We introduce the concept of interpolating and sampling sequences for analytic tent spaces analogously to the same concepts for…
▽ More
Introduced by Coifman, Meyer, and Stein, the tent spaces have seen wide applications in Harmonic Analysis. Their analytic cousins have seen some applications involving the derivatives of Hardy space functions. Moreover, the tent spaces have been a recent focus of research. We introduce the concept of interpolating and sampling sequences for analytic tent spaces analogously to the same concepts for Bergman spaces. We then characterize such sequences in terms of Seip's upper and lower uniform density. We accomplish this by exploiting a kind of Möbius invariance for the tent spaces.
△ Less
Submitted 20 April, 2020; v1 submitted 16 January, 2020;
originally announced January 2020.
-
Improvements of the REDCRAFT Software Package
Authors:
Casey A Cole,
Caleb Parks,
Julian Rachele,
Homayoun Valafar
Abstract:
Traditional approaches to elucidation of protein structures by NMR spectroscopy rely on distance restraints also known as nuclear Overhauser effects (NOEs). The use of NOEs as the primary source of structure determination by NMR spectroscopy is time consuming and expensive. Residual Dipolar Couplings (RDCs) have become an alternate approach for structure calculation by NMR spectroscopy. In previou…
▽ More
Traditional approaches to elucidation of protein structures by NMR spectroscopy rely on distance restraints also known as nuclear Overhauser effects (NOEs). The use of NOEs as the primary source of structure determination by NMR spectroscopy is time consuming and expensive. Residual Dipolar Couplings (RDCs) have become an alternate approach for structure calculation by NMR spectroscopy. In previous works, the software package REDCRAFT has been presented as a means of harnessing the information containing in RDCs for structure calculation of proteins. In this work, we present significant improvements to the REDCRAFT package including: refinement of the decimation procedure, the inclusion of graphical user interface, adoption of NEF standards, and addition of scripts for enhanced protein modeling options. The improvements to REDCRAFT have resulted in the ability to fold proteins that the previous versions were unable to fold. For instance, we report the results of folding of the protein 1A1Z in the presence of highly erroneous data.
△ Less
Submitted 19 November, 2019;
originally announced November 2019.
-
Wronskians and Linear Dependence of Formal Power Series
Authors:
Keith Ball,
Cynthia Parks,
Wai Yan Pong
Abstract:
We give a new proof of the fact that the vanishing of generalized Wronskians implies linear dependence of formal power series in serveral variables. Our results are also valid for quotients of germs of analytic functions.
We give a new proof of the fact that the vanishing of generalized Wronskians implies linear dependence of formal power series in serveral variables. Our results are also valid for quotients of germs of analytic functions.
△ Less
Submitted 29 October, 2016;
originally announced November 2016.
-
Hydrostatic pressure response of an oxide two-dimensional electron system
Authors:
J. Zabaleta,
V. S. Borisov,
R. Wanke,
H. O. Jeschke,
S. C. Parks,
B. Baum,
A. Teker,
T. Harada,
K. Syassen,
T. Kopp,
N. Pavlenko,
R. ValentÃ,
J. Mannhart
Abstract:
Two-dimensional electron systems with fascinating properties exist in multilayers of standard semiconductors, on helium surfaces, and in oxides. Compared to the two-dimensional (2D) electron gases of semiconductors, the 2D electron systems in oxides are typically more strongly correlated and more sensitive to the microscopic structure of the hosting lattice. This sensitivity suggests that the oxid…
▽ More
Two-dimensional electron systems with fascinating properties exist in multilayers of standard semiconductors, on helium surfaces, and in oxides. Compared to the two-dimensional (2D) electron gases of semiconductors, the 2D electron systems in oxides are typically more strongly correlated and more sensitive to the microscopic structure of the hosting lattice. This sensitivity suggests that the oxide 2D systems are highly tunable by hydrostatic pressure. Here we explore the effects of hydrostatic pressure on the well-characterized 2D electron system formed at LaAlO$_{3}$ -SrTiO$_{3}$ interfaces[1] and measure a pronounced, unexpected response. Pressure of $\sim$2 GPa reversibly doubles the 2D carrier density $n_{s}$ at 4 K. Along with the increase of $n_{s}$, the conductivity and mobility are reduced under pressure. First-principles pressure simulations reveal the same behavior of the carrier density and suggest a possible mechanism of the mobility reduction, based on the dielectric properties of both materials and their variation under external pressure.
△ Less
Submitted 27 May, 2016;
originally announced May 2016.
