Showing 1–2 of 2 results for author: Xun, X

On Representation Knowledge Distillation for Graph Neural Networks

Authors: Chaitanya K. Joshi, Fayao Liu, Xu Xun, Jie Lin, Chuan-Sheng Foo

Abstract: Knowledge distillation is a learning paradigm for boosting resource-efficient graph neural networks (GNNs) using more expressive yet cumbersome teacher models. Past work on distillation for GNNs proposed the Local Structure Preserving loss (LSP), which matches local structural relationships defined over edges across the student and teacher's node embeddings. This paper studies whether preserving t… ▽ More Knowledge distillation is a learning paradigm for boosting resource-efficient graph neural networks (GNNs) using more expressive yet cumbersome teacher models. Past work on distillation for GNNs proposed the Local Structure Preserving loss (LSP), which matches local structural relationships defined over edges across the student and teacher's node embeddings. This paper studies whether preserving the global topology of how the teacher embeds graph data can be a more effective distillation objective for GNNs, as real-world graphs often contain latent interactions and noisy edges. We propose Graph Contrastive Representation Distillation (G-CRD), which uses contrastive learning to implicitly preserve global topology by aligning the student node embeddings to those of the teacher in a shared representation space. Additionally, we introduce an expanded set of benchmarks on large-scale real-world datasets where the performance gap between teacher and student GNNs is non-negligible. Experiments across 4 datasets and 14 heterogeneous GNN architectures show that G-CRD consistently boosts the performance and robustness of lightweight GNNs, outperforming LSP (and a global structure preserving variant of LSP) as well as baselines from 2D computer vision. An analysis of the representational similarity among teacher and student embedding spaces reveals that G-CRD balances preserving local and global relationships, while structure preserving approaches are best at preserving one or the other. Our code is available at https://github.com/chaitjo/efficient-gnns △ Less

Submitted 4 February, 2023; v1 submitted 9 November, 2021; originally announced November 2021.

Comments: IEEE Transactions on Neural Networks and Learning Representation (TNNLS), Special Issue on Deep Neural Networks for Graphs: Theory, Models, Algorithms and Applications

Research on innovation in business and management about China and Latin America: bibliometrics insights using Google Scholar, Dimensions and Microsoft Academic

Authors: Julian D. Cortes, Xiaolei Lin, Xiaolei Xun

Abstract: Trade and investment between develo** regions such as China and Latin America (LATAM) are growing prominently. However, insights on crucial factors such as innovation in business and management (iBM) about both regions have not been scrutinized. This study presents the research output, impact, and structure of iBM research published about China and LATAM in a comparative framework using Google S… ▽ More Trade and investment between develo** regions such as China and Latin America (LATAM) are growing prominently. However, insights on crucial factors such as innovation in business and management (iBM) about both regions have not been scrutinized. This study presents the research output, impact, and structure of iBM research published about China and LATAM in a comparative framework using Google Scholar, Dimensions, and Microsoft Academic. Findings showed i) that iBM topics of both regions were framed within research and development management, and technological development topics, ii) significant differences in output and impact between regions, and iii) the same case for platforms. △ Less

Submitted 13 April, 2021; originally announced April 2021.