Dynamic Sharding and Load Balancing for Federated Learning in Distributed NoSQL Environments

Abstract

The spread of decentralized information in distributed NoSQL systems has posed considerable problems to Federated Learning (FL) mainly because there is an obvious discrepancy between the distribution of data and their computation. The study presents the concept of the Dynamic Sharding and Load Balancing (DSLB-FL) framework that integrates the concept of the so-called straggler node by converting the fixed database shards into liquid and compute-sensitive entities. Through the application of a real-time telemetry-based orchestration layer, the system will detect the existence of computational bottlenecks, hence take the initiative of moving data shards off the overloaded or undercapacity nodes to those low-utilized and high-performance nodes. This artificial synchronization makes the process of global model aggregation no longer controlled by the slowest cluster member, and maximizes the duty cycle of heterogeneous hardware, preserves the integrity of the federated process in a privacy-preserving manner.