-
High-Discriminative Attribute Feature Learning for Generalized Zero-Shot Learning
Authors:
Yu Lei,
Guoshuai Sheng,
Fangfang Li,
Quanxue Gao,
Cheng Deng,
Qin Li
Abstract:
Zero-shot learning(ZSL) aims to recognize new classes without prior exposure to their samples, relying on semantic knowledge from observed classes. However, current attention-based models may overlook the transferability of visual features and the distinctiveness of attribute localization when learning regional features in images. Additionally, they often overlook shared attributes among different…
▽ More
Zero-shot learning(ZSL) aims to recognize new classes without prior exposure to their samples, relying on semantic knowledge from observed classes. However, current attention-based models may overlook the transferability of visual features and the distinctiveness of attribute localization when learning regional features in images. Additionally, they often overlook shared attributes among different objects. Highly discriminative attribute features are crucial for identifying and distinguishing unseen classes. To address these issues, we propose an innovative approach called High-Discriminative Attribute Feature Learning for Generalized Zero-Shot Learning (HDAFL). HDAFL optimizes visual features by learning attribute features to obtain discriminative visual embeddings. Specifically, HDAFL utilizes multiple convolutional kernels to automatically learn discriminative regions highly correlated with attributes in images, eliminating irrelevant interference in image features. Furthermore, we introduce a Transformer-based attribute discrimination encoder to enhance the discriminative capability among attributes. Simultaneously, the method employs contrastive loss to alleviate dataset biases and enhance the transferability of visual features, facilitating better semantic transfer between seen and unseen classes. Experimental results demonstrate the effectiveness of HDAFL across three widely used datasets.
△ Less
Submitted 7 April, 2024;
originally announced April 2024.
-
MSPipe: Efficient Temporal GNN Training via Staleness-aware Pipeline
Authors:
Guangming Sheng,
Junwei Su,
Chao Huang,
Chuan Wu
Abstract:
Memory-based Temporal Graph Neural Networks (MTGNNs) are a class of temporal graph neural networks that utilize a node memory module to capture and retain long-term temporal dependencies, leading to superior performance compared to memory-less counterparts. However, the iterative reading and updating process of the memory module in MTGNNs to obtain up-to-date information needs to follow the tempor…
▽ More
Memory-based Temporal Graph Neural Networks (MTGNNs) are a class of temporal graph neural networks that utilize a node memory module to capture and retain long-term temporal dependencies, leading to superior performance compared to memory-less counterparts. However, the iterative reading and updating process of the memory module in MTGNNs to obtain up-to-date information needs to follow the temporal dependencies. This introduces significant overhead and limits training throughput. Existing optimizations for static GNNs are not directly applicable to MTGNNs due to differences in training paradigm, model architecture, and the absence of a memory module. Moreover, they do not effectively address the challenges posed by temporal dependencies, making them ineffective for MTGNN training. In this paper, we propose MSPipe, a general and efficient framework for MTGNNs that maximizes training throughput while maintaining model accuracy. Our design addresses the unique challenges associated with fetching and updating node memory states in MTGNNs by integrating staleness into the memory module. However, simply introducing a predefined staleness bound in the memory module to break temporal dependencies may lead to suboptimal performance and lack of generalizability across different models and datasets. To solve this, we introduce an online pipeline scheduling algorithm in MSPipe that strategically breaks temporal dependencies with minimal staleness and delays memory fetching to obtain fresher memory states. Moreover, we design a staleness mitigation mechanism to enhance training convergence and model accuracy. We provide convergence analysis and prove that MSPipe maintains the same convergence rate as vanilla sample-based GNN training. Experimental results show that MSPipe achieves up to 2.45x speed-up without sacrificing accuracy, making it a promising solution for efficient MTGNN training.
△ Less
Submitted 23 February, 2024;
originally announced February 2024.
-
RotaTR: Detection Transformer for Dense and Rotated Object
Authors:
Zhu Yuke,
Ruan Yumeng,
Yang Lei,
Guo Sheng
Abstract:
Detecting the objects in dense and rotated scenes is a challenging task. Recent works on this topic are mostly based on Faster RCNN or Retinanet. As they are highly dependent on the pre-set dense anchors and the NMS operation, the approach is indirect and suboptimal.The end-to-end DETR-based detectors have achieved great success in horizontal object detection and many other areas like segmentation…
▽ More
Detecting the objects in dense and rotated scenes is a challenging task. Recent works on this topic are mostly based on Faster RCNN or Retinanet. As they are highly dependent on the pre-set dense anchors and the NMS operation, the approach is indirect and suboptimal.The end-to-end DETR-based detectors have achieved great success in horizontal object detection and many other areas like segmentation, tracking, action recognition and etc.However, the DETR-based detectors perform poorly on dense rotated target tasks and perform worse than most modern CNN-based detectors. In this paper, we find the most significant reason for the poor performance is that the original attention can not accurately focus on the oriented targets. Accordingly, we propose Rotated object detection TRansformer (RotaTR) as an extension of DETR to oriented detection. Specifically, we design Rotation Sensitive deformable (RSDeform) attention to enhance the DETR's ability to detect oriented targets. It is used to build the feature alignment module and rotation-sensitive decoder for our model. We test RotaTR on four challenging-oriented benchmarks. It shows a great advantage in detecting dense and oriented objects compared to the original DETR. It also achieves competitive results when compared to the state-of-the-art.
△ Less
Submitted 5 December, 2023;
originally announced December 2023.
-
GNNFlow: A Distributed Framework for Continuous Temporal GNN Learning on Dynamic Graphs
Authors:
Yuchen Zhong,
Guangming Sheng,
Tianzuo Qin,
Minjie Wang,
Quan Gan,
Chuan Wu
Abstract:
Graph Neural Networks (GNNs) play a crucial role in various fields. However, most existing deep graph learning frameworks assume pre-stored static graphs and do not support training on graph streams. In contrast, many real-world graphs are dynamic and contain time domain information. We introduce GNNFlow, a distributed framework that enables efficient continuous temporal graph representation learn…
▽ More
Graph Neural Networks (GNNs) play a crucial role in various fields. However, most existing deep graph learning frameworks assume pre-stored static graphs and do not support training on graph streams. In contrast, many real-world graphs are dynamic and contain time domain information. We introduce GNNFlow, a distributed framework that enables efficient continuous temporal graph representation learning on dynamic graphs on multi-GPU machines. GNNFlow introduces an adaptive time-indexed block-based data structure that effectively balances memory usage with graph update and sampling operation efficiency. It features a hybrid GPU-CPU graph data placement for rapid GPU-based temporal neighborhood sampling and kernel optimizations for enhanced sampling processes. A dynamic GPU cache for node and edge features is developed to maximize cache hit rates through reuse and restoration strategies. GNNFlow supports distributed training across multiple machines with static scheduling to ensure load balance. We implement GNNFlow based on DGL and PyTorch. Our experimental results show that GNNFlow provides up to 21.1x faster continuous learning than existing systems.
△ Less
Submitted 29 November, 2023; v1 submitted 29 November, 2023;
originally announced November 2023.
-
SWIFT: Expedited Failure Recovery for Large-scale DNN Training
Authors:
Yuchen Zhong,
Guangming Sheng,
Juncheng Liu,
**hui Yuan,
Chuan Wu
Abstract:
As the size of deep learning models gets larger and larger, training takes longer time and more resources, making fault tolerance more and more critical. Existing state-of-the-art methods like CheckFreq and Elastic Horovod need to back up a copy of the model state (i.e., parameters and optimizer states) in memory, which is costly for large models and leads to non-trivial overhead. This paper prese…
▽ More
As the size of deep learning models gets larger and larger, training takes longer time and more resources, making fault tolerance more and more critical. Existing state-of-the-art methods like CheckFreq and Elastic Horovod need to back up a copy of the model state (i.e., parameters and optimizer states) in memory, which is costly for large models and leads to non-trivial overhead. This paper presents SWIFT, a novel recovery design for distributed deep neural network training that significantly reduces the failure recovery overhead without affecting training throughput and model accuracy. Instead of making an additional copy of the model state, SWIFT resolves the inconsistencies of the model state caused by the failure and exploits the replicas of the model state in data parallelism for failure recovery. We propose a logging-based approach when replicas are unavailable, which records intermediate data and replays the computation to recover the lost state upon a failure. The re-computation is distributed across multiple machines to accelerate failure recovery further. We also log intermediate data selectively, exploring the trade-off between recovery time and intermediate data storage overhead. Evaluations show that SWIFT significantly reduces the failure recovery time and achieves similar or better training throughput during failure-free execution compared to state-of-the-art methods without degrading final model accuracy. SWIFT can also achieve up to 1.16x speedup in total training time compared to state-of-the-art methods.
△ Less
Submitted 13 February, 2023;
originally announced February 2023.
-
Classification of second harmonic generation effect in magnetically ordered materials
Authors:
Rui-Chun Xiao,
Ding-Fu Shao,
Wei Gan,
Huan-Wen Wang,
Hui Han,
Z. G. Sheng,
Chang** Zhang,
Hua Jiang,
Hui Li
Abstract:
The relationship between magnetic order and the second harmonic generation (SHG) effect is a fundamental area of study in condensed matter physics with significant practical implications. In order to gain a clearer understanding of this intricate relation, this study presents a comprehensive classification scheme for the SHG effect in magnetically ordered materials. This framework offers a straigh…
▽ More
The relationship between magnetic order and the second harmonic generation (SHG) effect is a fundamental area of study in condensed matter physics with significant practical implications. In order to gain a clearer understanding of this intricate relation, this study presents a comprehensive classification scheme for the SHG effect in magnetically ordered materials. This framework offers a straightforward approach to connect magnetic order and SHG effect. The characteristics of the SHG tensors in all magnetic point groups are studied using the isomorphic group method, followed by a comprehensive SHG effect classification scheme that includes seven types based on the symmetries of the magnetic phases and their corresponding parent phases. In addition, a tensor dictionary containing the SHG and linear magneto-optic (LMO) effect is established. Furthermore, an extensive SHG database of magnetically ordered materials is also built up. This classification strategy exposes an anomalous SHG effect with even characteristic under time-reversal symmetry, which is solely contributed by magnetic structure. Moreover, the proposed classification scheme facilitates the determination of magnetic structures through SHG effect.
△ Less
Submitted 15 August, 2023; v1 submitted 8 November, 2022;
originally announced November 2022.
-
Event Occurrence Model for Power Distribution System Based on Data Eigenvalue Behavior Analytic
Authors:
Nan Zhou,
Lingen Luo,
Gehao Sheng,
Xiuchen Jiang
Abstract:
This paper describes a model for power distribution network, which is hten utilized for eigenvalue behavior distribution analysis. We utilized the compensation theory to model the event that occurs in the network and derived the relation between the system nodal voltage eigenvalues and the occurrence of events. The paper combines the power system model with the eigenvalue distribution analysis.
This paper describes a model for power distribution network, which is hten utilized for eigenvalue behavior distribution analysis. We utilized the compensation theory to model the event that occurs in the network and derived the relation between the system nodal voltage eigenvalues and the occurrence of events. The paper combines the power system model with the eigenvalue distribution analysis.
△ Less
Submitted 3 February, 2021;
originally announced February 2021.
-
Partial Discharge Direction of Arrival Estimation in Air-insulated Substation by UHF Wireless Array and RSSI Maximum Likelihood Estimator
Authors:
Bei Han,
Lingen Luo,
Gehao Sheng,
Xiuchen Jiang
Abstract:
The quick detection and localization of partial discharge (PD) in an air-insulated substation (AIS) based on ultrahigh-frequency (UHF) sensor arrays are efficient for power equipment monitoring. The adopted UHF PD time difference of arrival (TDOA) methods mainly use the time difference of electromagnetic wave signals. Thus, the system requires both a high sampling rate and time synchronization acc…
▽ More
The quick detection and localization of partial discharge (PD) in an air-insulated substation (AIS) based on ultrahigh-frequency (UHF) sensor arrays are efficient for power equipment monitoring. The adopted UHF PD time difference of arrival (TDOA) methods mainly use the time difference of electromagnetic wave signals. Thus, the system requires both a high sampling rate and time synchronization accuracy, leading to a high cost and large size. In this study, the feasibility and accuracy of PD DOA in an AIS were investigated using a UHF wireless sensor array and the received signal strength indicator. First, the power pattern of the designed UHF wireless sensor array was obtained via an offline experiment. Then, a statistical approach to the PD DOA method based on the maximum likelihood estimator was employed to obtain the preliminary DOA result. Finally, interpolation and clustering algorithms were used to improve the accuracy of the DOA. A laboratory test was conducted. The average error of the PD DOA was less than 6°, and the cost-effectiveness and portability were clearly improved.
△ Less
Submitted 16 October, 2020;
originally announced October 2020.
-
Critical behavior of two-dimensional intrinsically ferromagnetic semiconductor CrI3
Authors:
G. T. Lin,
X. Luo,
F. C. Chen,
J. Yan,
J. J. Gao,
Y. Sun,
W. Tong,
P. Tong,
W. J. Lu,
Z. G. Sheng,
W. H. Song,
X. B. Zhu,
Y. P. Sun
Abstract:
CrI3, which belongs to a rare category of two-dimensional (2D) ferromagnetic semiconductors, is of great interest for spintronic device applications. Unlike CrCl3 whose magnetism presents a 2D-Heisenberg behavior, CrI3 exhibits a larger van der Waals gap, smaller cleavage energy, and stronger magnetic anisotropy which could lead to a 3D magnetic characteristic. Hence, we investigate the critical b…
▽ More
CrI3, which belongs to a rare category of two-dimensional (2D) ferromagnetic semiconductors, is of great interest for spintronic device applications. Unlike CrCl3 whose magnetism presents a 2D-Heisenberg behavior, CrI3 exhibits a larger van der Waals gap, smaller cleavage energy, and stronger magnetic anisotropy which could lead to a 3D magnetic characteristic. Hence, we investigate the critical behavior of CrI3 in the vicinity of magnetic transition. We use the modified Arrott plot and Kouvel-Fisher method, and conduct critical isotherm analysis to estimate the critical exponents near the ferromagnetic phase transition. This shows that the magnetism of CrI3 follows the crossover behavior of a 3D-Ising model with mean field type interactions where the critical exponents \b{eta}, γ, and δ are 0.323, 0.835, and 3.585, respectively, at the Curie temperature of 64 K. We propose the crossover behavior can be attributed to the strong uniaxial anisotropy and inevitable interlayer coupling. Our experiment demonstrates the applicability of crossover behavior to a 2D ferromagnetic semiconductor.
△ Less
Submitted 30 January, 2018;
originally announced January 2018.
-
Tricritical behavior of two-dimensional intrinsic ferromagnetic semiconducting CrGeTe3
Authors:
G. T. Lin,
H. L. Zhuang,
X. Luo,
B. J. Liu,
F. C. Chen,
J. Yan,
Y. Sun,
J. Zhou,
W. J. Lu,
P. Tong,
Z. G. Sheng,
Z. Qu,
W. H. Song,
X. B. Zhu,
Y. P. Sun
Abstract:
CrGeTe3 recently emerges as a new two-dimensional (2D) ferromagnetic semiconductor that is promising for spintronic device applications. Unlike CrSiTe3 whose magnetism can be understood using the 2D-Ising model, CrGeTe3 exhibits a smaller van der Waals gap and larger cleavage energy, which could lead to a transition of magnetic mechanism from 2D to 3D. To confirm this speculation, we investigate t…
▽ More
CrGeTe3 recently emerges as a new two-dimensional (2D) ferromagnetic semiconductor that is promising for spintronic device applications. Unlike CrSiTe3 whose magnetism can be understood using the 2D-Ising model, CrGeTe3 exhibits a smaller van der Waals gap and larger cleavage energy, which could lead to a transition of magnetic mechanism from 2D to 3D. To confirm this speculation, we investigate the critical behavior CrGeTe3 around the second-order paramagnetic-ferromagnetic phase transition. We obtain the critical exponents estimated by several common experimental techniques including the modified Arrott plot, Kouvel-Fisher method and critical isotherm analysis, which show that the magnetism of CrGeTe3 follows the tricritical mean-field model with the critical exponents \b{eta}, γ, and δ of 0.240, 1.000, and 5.070, respectively, at the Curie temperature of 67.9 K. We therefore suggest that the magnetic phase transition from 2D to 3D for CrGeTe3 should locate near a tricritical point. Our experiment provides a direct demonstration of the applicability of the tricritical mean-field model to a 2D ferromagnetic semiconductor.
△ Less
Submitted 10 June, 2017;
originally announced June 2017.
-
Structural, magnetic and transport properties in the Pr-doped manganites La0.9-xPrxTe0.1MnO3
Authors:
J. Yang,
W. H. Song,
Y. Q. Ma,
R. L. Zhang,
B. C. Zhao,
Z. G. Sheng,
G. H. Zheng,
J. M. Dai,
Y. P. Sun
Abstract:
The effect of Pr-do** on structural, magnetic and transport properties in electron-doped manganites La0.9-xPrxTe0.1MnO3 with fixed carrier concentration are investigated. The room temperature structural transition from rhombohedral (R3C) to orthorhombic (Pbnm) symmetry is found in the samples with by the Rietveld refinement of x-ray powder diffraction patterns. The Curie temperature of samples…
▽ More
The effect of Pr-do** on structural, magnetic and transport properties in electron-doped manganites La0.9-xPrxTe0.1MnO3 with fixed carrier concentration are investigated. The room temperature structural transition from rhombohedral (R3C) to orthorhombic (Pbnm) symmetry is found in the samples with by the Rietveld refinement of x-ray powder diffraction patterns. The Curie temperature of samples decreases and the transition becomes broader with increasing Pr-do** level. For the samples with, there exist insulator-metal (I-M) transition. And the low-temperature I-M transition is observed at about 66K for the sample with x = 0.36, which may be related to the opening of a new percolation channel. For the samples with, r(T) curves display the semiconducting behavior (dρ/dT < 0) in both high-temperature PM phase and low-temperature FM phase. The results are discussed in terms of the increased bending of the Mn-O-Mn bond with decreasing the average ionic radius of the A-site element <r_A> and the tolerance factor, resulting in the narrowing of the bandwidth, the decrease of the mobility of electrons and the weakening of DE interaction caused by the substitution of smaller Pr3+ ions for larger La3+ ion.
△ Less
Submitted 13 August, 2004;
originally announced August 2004.
-
Insulator-metal transition and the magnetic phase diagram of La1-xTexMnO3
Authors:
J. Yang,
W. H. Song,
Y. Q. Ma,
R. L. Zhang,
B. C. Zhao,
Z. G. Sheng,
G. H. Zheng,
J. M. Dai,
Y. P. Sun
Abstract:
The structural, electrical transport and magnetic properties of perovskite oxides La1-xTexMnO3 have been investigated and thus the magnetic phase diagram of La1-xTexMnO3 compounds as a function of temperature and the do** level x has been obtained. All samples have rhombohedral structure and undergo paramagnetic-ferromagnetic (PM-FM) transition accompanied with metal-insulator transition (MIT)…
▽ More
The structural, electrical transport and magnetic properties of perovskite oxides La1-xTexMnO3 have been investigated and thus the magnetic phase diagram of La1-xTexMnO3 compounds as a function of temperature and the do** level x has been obtained. All samples have rhombohedral structure and undergo paramagnetic-ferromagnetic (PM-FM) transition accompanied with metal-insulator transition (MIT). Whereas a charge ordering (CO) transition begins to appear at for the sample with x=0.60. Moreover, the variation of the Curie temperature and the MIT temperature is quite complex and the results are discussed in terms of three factors including the average A-site cation radius <rA>, the size mismatch and the Te content. In addition, there has an evident magnetoresistance (MR) at low temperatures for all samples.
△ Less
Submitted 13 August, 2004;
originally announced August 2004.
-
The effect of grain size on electrical transport and magnetic properties of La0.9Te0.1MnO3
Authors:
J. Yang,
B. C. Zhao,
R. L. Zhang,
Y. Q. Ma,
Z. G. Sheng,
W. H. Song,
Y. P. Sun
Abstract:
The effect of grain size on structural, magnetic and transport properties in electron-doped manganites La0.9Te0.1MnO3 has been investigated. All samples show a rhombohedral structure with the space group at room temperature. It shows that the Mn-O-Mn bond angle decreases and the Mn-O bond length increases with the increase of grain size. All samples undergo paramagnetic (PM)-ferromagnetic (FM) p…
▽ More
The effect of grain size on structural, magnetic and transport properties in electron-doped manganites La0.9Te0.1MnO3 has been investigated. All samples show a rhombohedral structure with the space group at room temperature. It shows that the Mn-O-Mn bond angle decreases and the Mn-O bond length increases with the increase of grain size. All samples undergo paramagnetic (PM)-ferromagnetic (FM) phase transition and an interesting phenomenon that both magnetization and the Curie temperature decrease with increasing grain size is observed, which is suggested to mainly originate from the increase of the Mn-O bond length . Additionally, obviously increases with decreasing grain size due to the increase of both the height and width of tunneling barriers with decreasing the grain size. The results indicate that both the intrinsic colossal magnetoresistance (CMR) and the extrinsic the extrinsic interfacial magnetoresistance (IMR) can be effectively tuned in La0.9Te0.1MnO3 by changing grain size.
△ Less
Submitted 26 August, 2004; v1 submitted 13 August, 2004;
originally announced August 2004.
-
Structural, magnetic and transport properties in the Cu-doped manganites La0.85Te0.15Mn1-xCuxO3 (0 \leq x \leq 0.20)
Authors:
J. Yang,
W. H. Song,
Y. Q. Ma,
R. L. Zhang,
B. C. Zhao,
Z. G. Sheng,
G. H. Zheng,
J. M. Dai,
Y. P. Sun
Abstract:
The effect of Cu-do** at Mn-site on structural, magnetic and transport properties in electron-doped manganites La0.85Te0.15Mn1-xCuxO3 has been investigated. Based on the analysis of structural parameter variations, the valence state of the Cu ion in Cu-doped manganites is suggested to be +2. All samples undergo the paramagnetic-ferromagnetic (PM-FM) phase transition. The Curie temperature decr…
▽ More
The effect of Cu-do** at Mn-site on structural, magnetic and transport properties in electron-doped manganites La0.85Te0.15Mn1-xCuxO3 has been investigated. Based on the analysis of structural parameter variations, the valence state of the Cu ion in Cu-doped manganites is suggested to be +2. All samples undergo the paramagnetic-ferromagnetic (PM-FM) phase transition. The Curie temperature decreases and the transition becomes broader with increasing Cu-do** level, in contrast, the magnetization magnitude of Cu-do** samples at low temperatures increase as x \leq 0.15. The insulator-metal (I-M) transition moves to lower temperatures with increasing Cu-do** content and disappears as x > 0.1. In addition, the higher temperature resistivity r peak in double-peak-like r(T) curves observed in no Cu-do** sample is completely suppressed as Cu-do** level x = 0.1 and r(T) curve only shows single I-M transition at the low temperature well below . The results are discussed according to the change of magnetic exchange interaction caused by Cu-do**.
△ Less
Submitted 20 September, 2004; v1 submitted 12 August, 2004;
originally announced August 2004.
-
The effect of oxygen stoichiometry on electrical transport and magnetic properties of La0.9Te0.1MnOy
Authors:
J. Yang,
W. H. Song,
R. L. Zhang,
Y. Q. Ma,
B. C. Zhao,
Z. G. Sheng,
G. H. Zheng,
J. M. Dai,
Y. P. Sun
Abstract:
The effect of the variation of oxygen content on structural, magnetic and transport properties in the electron-doped manganites La0.9Te0.1MnOy has been investigated. All samples show a rhombohedral structure with the space group . The Curie temperature decreases and the paramagnetic-ferromagnetic (PM-FM) transition becomes broader with the reduction of oxygen content. The resistivity of the anne…
▽ More
The effect of the variation of oxygen content on structural, magnetic and transport properties in the electron-doped manganites La0.9Te0.1MnOy has been investigated. All samples show a rhombohedral structure with the space group . The Curie temperature decreases and the paramagnetic-ferromagnetic (PM-FM) transition becomes broader with the reduction of oxygen content. The resistivity of the annealed samples increases slightly with a small reduction of oxygen content. Further reduction in the oxygen content, the resistivity maximum increases by six orders of magnitude compared with that of the as-prepared sample, and the r(T) curves of samples with y = 2.86 and y = 2.83 display the semiconducting behavior () in both high-temperature PM phase and low-temperature FM phase, which is considered to be related to the appearance of superexchange ferromagnetism (SFM) and the localization of carriers. The results are discussed in terms of the combined effects of the increase in the Mn2+/(Mn2++Mn3+) ratio, the partial destruction of double exchange (DE) interaction, and the localization of carriers due to the introduction of oxygen vacancies in the Mn-O-Mn network.
△ Less
Submitted 12 August, 2004;
originally announced August 2004.