A research group has unveiled Capsule, an innovative out-of-core technique designed for training large-scale Graph Neural Networks (GNNs). This method boasts up to a 12.02-fold enhancement in runtime efficiency and utilizes merely 22.24% of the primary memory when contrasted with leading out-of-core GNN frameworks currently available. The study findings have been made public through publication.
ACM Proceedings on Data Management
The team comprised the Data Darkness Lab (DDL) from the Medical Imaging Intelligence and Robotics Research Center at the USTC Suzhou Institute of the University of Science and Technology of China (USTC).
Graph Neural Networks (GNNs) have shown their prowess in fields like recommendation systems, natural language processing, computational chemistry, and bioinformatics. Widely used training platforms for GNNs, including DGL and PyG, utilize the parallel processing capabilities of GPUs to analyze structural details within graph data.
Even though GPUs offer computational benefits for training Graph Neural Networks (GNNs), their restricted memory capacity makes it difficult to handle extensive graph datasets, posing a major hurdle for current GNN systems. To tackle this problem, the DDL group introduced an innovative out-of-core (OOC) GNN training framework called Capsule. This approach offers a viable solution for managing large-scale GNN training efficiently.
In contrast to current out-of-core Graph Neural Network (GNN) systems, Capsule minimizes the input/output bottleneck between the CPU and GPU during backpropagation through techniques like graph partitioning and pruning. By doing so, it ensures that all necessary training subgraphs and their attributes can be accommodated within the GPU’s memory limits, thereby enhancing overall efficiency.
Furthermore, Capsule enhances efficiency through a subgraph-loading method designed around the shortest Hamiltonian cycle along with a pipeline-style parallel approach. Additionally, Capsule is user-friendly and can be easily integrated into leading open-source GNN training platforms.
When tested with extensive real-world graph data sets, Capsule surpassed current leading systems, showing as much as a 12.02-fold increase in performance while utilizing just 22.24% of the memory capacity. Additionally, it offers a theoretical limit for the variability of the embeddings generated throughout the training process.
This work provides a new approach to the colossal graphical structures processed and the limited memory capacities of GPUs.
More information:
Yongan Xiang et al., Capsule: An Out-of-Core Training Method for Massive Graph Neural Networks,
ACM Transactions on Management of Data Proceedings
(2025).
DOI: 10.1145/3709669
Furnished by the University of Science and Technology of China
This tale was initially released on
Tech Xplore
. Subscribe to our
newsletter
For the most recent science and technology news updates.
