Morzsák

Oldal címe

Bridging the Asymmetry Gap: A Threshold Model for Hybrid Cloud GPU Offloading

Címlapos tartalom

Hybrid cloud models are increasingly used for large GPU workloads. When on-premises capacity is limited, jobs must either wait for local GPUs or be offloaded to the cloud. Because on-premises GPU refresh cycles lag behind those of cloud platforms, cloud GPUs are often faster but incur data transfer and startup overhead. This paper introduces a threshold-based model that identifies the minimum local waiting time beyond which offloading reduces total job completion time. We evaluate ResNet-50 and ViT-B/16 training on an on-premises NVIDIA Tesla V100 versus NVIDIA A10G and L4 cloud GPUs, under eager and compiled execution modes. Our results show that the offloading threshold depends on the interaction between model architecture and software optimizations. ResNet-50 requires a positive local waiting time for offloading to be beneficial in most evaluated configurations, whereas for ViT-B/16, enabling graph compilation with TensorFloat-32 (TF32) precision on the L4 makes offloading preferable regardless of local waiting time under warm-start conditions. Even with cold-start overhead, the advantage of the compiled execution path outweighs provisioning delays as the dataset scale increases. These findings indicate that hybrid cloud scheduling decisions should account for the software execution path and instance state alongside hardware characteristics and transfer delays.