분산학습1 [인공지능] Federated Learning , Distributed Learning Summary ✨ Federated learning ◾ Exploits huge amount of data across clients ◾ data is non-IID ◾ Brief indroduction of FedAvg ✨ Distributed learning ◾ Make training Faster ◾ Data evenly distributed ◾ data parallel : Forward pass, Backward pass , Weight update 👉 Communication : Allreduce - ring reduction, One-host reduction ◾ Model parallel 👉 inter-layer parallel (pipeline) : sub-minibatches, inter.. 2022. 6. 15. 이전 1 다음 728x90
