Morzsák

Oldal címe

Distributed GraphSAGE Training with DistDGL: A Multi-GPU Scalability Study

Címlapos tartalom

Graph Neural Network (GNN) training on largescale graphs is often limited by memory capacity, data movement, and synchronization overheads. This paper continues our previous evaluation of distributed deep graph learning with Distributed Deep Graph Library (DistDGL) by moving from a cloud-based environment to the Graphics Processing Unit (GPU) partition of the Komondor High-Performance Computing (HPC) system. We evaluate distributed GraphSAGE training on the Reddit and OGBN-Products datasets using multiple batch sizes and configurations ranging from one to four compute nodes and one or two GPUs per compute node. The experiments show that the GPU partition of the Komondor HPC system provides substantially better scalability than the previously studied cloud setup. In particular, single-node two-GPU configurations achieve strong scaling efficiency, and in some cases exceed the ideal speedup relative to the single-GPU baseline. At the same time, the results reveal that scalability is not determined by GPU count alone: dataset characteristics, batch size, sampling and copy time, backward-pass synchronization, and runtime overhead all influence the observed performance. The componentlevel analysis shows that dense and sparse graph workloads exhibit different scaling behavior, highlighting the importance of evaluating distributed GNN systems on multiple datasets. Overall, the results demonstrate that DistDGL can effectively exploit GPU-accelerated HPC infrastructure, while also identifying communication and runtime overheads as important targets for further optimization.