-
Empowering Dual-Level Graph Self-Supervised Pretraining with Motif Discovery
Authors:
Pengwei Yan,
Kaisong Song,
Zhuoren Jiang,
Yangyang Kang,
Tianqian** Lin,
Changlong Sun,
Xiaozhong Liu
Abstract:
While self-supervised graph pretraining techniques have shown promising results in various domains, their application still experiences challenges of limited topology learning, human knowledge dependency, and incompetent multi-level interactions. To address these issues, we propose a novel solution, Dual-level Graph self-supervised Pretraining with Motif discovery (DGPM), which introduces a unique…
▽ More
While self-supervised graph pretraining techniques have shown promising results in various domains, their application still experiences challenges of limited topology learning, human knowledge dependency, and incompetent multi-level interactions. To address these issues, we propose a novel solution, Dual-level Graph self-supervised Pretraining with Motif discovery (DGPM), which introduces a unique dual-level pretraining structure that orchestrates node-level and subgraph-level pretext tasks. Unlike prior approaches, DGPM autonomously uncovers significant graph motifs through an edge pooling module, aligning learned motif similarities with graph kernel-based similarities. A cross-matching task enables sophisticated node-motif interactions and novel representation learning. Extensive experiments on 15 datasets validate DGPM's effectiveness and generalizability, outperforming state-of-the-art methods in unsupervised representation learning and transfer learning settings. The autonomously discovered motifs demonstrate the potential of DGPM to enhance robustness and interpretability.
△ Less
Submitted 19 December, 2023;
originally announced December 2023.
-
STRAW: Structure-Adaptive Weighting Procedure for Large-Scale Spatial Multiple Testing
Authors:
Pengfei Wang,
Pengyu Yan,
Canhui Li
Abstract:
The problem of large-scale spatial multiple testing is often encountered in various scientific research fields, where the signals are usually enriched on some regions while sparse on others. To integrate spatial structure information from nearby locations, we propose a novel approach, called {\bf STR}ucture-{\bf A}daptive {\bf W}eighting (STRAW) procedure, for large-scale spatial multiple testing.…
▽ More
The problem of large-scale spatial multiple testing is often encountered in various scientific research fields, where the signals are usually enriched on some regions while sparse on others. To integrate spatial structure information from nearby locations, we propose a novel approach, called {\bf STR}ucture-{\bf A}daptive {\bf W}eighting (STRAW) procedure, for large-scale spatial multiple testing. The STRAW procedure is capable of handling a broad range of spatial settings by leveraging a class of weighted p-values and is fully data-driven. Theoretical results show that the proposed method controls the false discovery rate (FDR) at the pre-specified level under some mild conditions. In practice, the local sparsity level, defined as the probability of the null hypothesis being not true, is commonly unknown. To address this issue, we develop a new method for estimating the local sparsity level by employing the kernel-smooth local false discovery rate (Lfdr) statistic. The superior numerical performance of the STRAW procedure is demonstrated by performing extensive simulation studies and a real data analysis.
△ Less
Submitted 27 September, 2023;
originally announced September 2023.
-
On a Sparse Shortcut Topology of Artificial Neural Networks
Authors:
Fenglei Fan,
Dayang Wang,
Hengtao Guo,
Qikui Zhu,
**kun Yan,
Ge Wang,
Hengyong Yu
Abstract:
In established network architectures, shortcut connections are often used to take the outputs of earlier layers as additional inputs to later layers. Despite the extraordinary effectiveness of shortcuts, there remain open questions on the mechanism and characteristics. For example, why are shortcuts powerful? Why do shortcuts generalize well? In this paper, we investigate the expressivity and gene…
▽ More
In established network architectures, shortcut connections are often used to take the outputs of earlier layers as additional inputs to later layers. Despite the extraordinary effectiveness of shortcuts, there remain open questions on the mechanism and characteristics. For example, why are shortcuts powerful? Why do shortcuts generalize well? In this paper, we investigate the expressivity and generalizability of a novel sparse shortcut topology. First, we demonstrate that this topology can empower a one-neuron-wide deep network to approximate any univariate continuous function. Then, we present a novel width-bounded universal approximator in contrast to depth-bounded universal approximators and extend the approximation result to a family of equally competent networks. Furthermore, with generalization bound theory, we show that the proposed shortcut topology enjoys excellent generalizability. Finally, we corroborate our theoretical analyses by comparing the proposed topology with popular architectures, including ResNet and DenseNet, on well-known benchmarks and perform a saliency map analysis to interpret the proposed topology. Our work helps enhance the understanding of the role of shortcuts and suggests further opportunities to innovate neural architectures.
△ Less
Submitted 11 November, 2021; v1 submitted 21 November, 2018;
originally announced November 2018.
