On the Necessity of Aligning Gradients for Wireless Federated Learning

Wei Ting Chang, Mohamed Seif Eldin Mohamed, Ravi Tandon

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In this paper, we consider the problem of wireless federated learning, where the users wish to jointly train a machine learning model with the help of a parameter server. During the training, the local gradients from the users are aggregated over a wireless channel. Typically, coefficients of the local gradients are aligned by power control techniques to ensure that the estimated aggregated gradient is an unbiased estimator of the true gradient. However, schemes that align gradients require coordination, can be challenging to implement in practice, and often lead to degraded performance due to heterogeneity of users' channels. In this paper, we show that alignment of gradients for wireless FL is not always necessary for convergence. Specifically, we consider non-convex loss functions, and derive conditions under which misaligned wireless gradient aggregation still converges to a stationary point. We also present experimental results to show that transmitting at full power can outperform aligned gradient aggregation depending on the heterogeneity of users' channels.

Original languageEnglish (US)
Title of host publication2021 IEEE 22nd International Workshop on Signal Processing Advances in Wireless Communications, SPAWC 2021
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages226-230
Number of pages5
ISBN (Electronic)9781665428514
DOIs
StatePublished - 2021
Event22nd IEEE International Workshop on Signal Processing Advances in Wireless Communications, SPAWC 2021 - Lucca, Italy
Duration: Sep 27 2021Sep 30 2021

Publication series

NameIEEE Workshop on Signal Processing Advances in Wireless Communications, SPAWC
Volume2021-September

Conference

Conference22nd IEEE International Workshop on Signal Processing Advances in Wireless Communications, SPAWC 2021
Country/TerritoryItaly
CityLucca
Period9/27/219/30/21

Keywords

  • Federated learning
  • Stochastic Gradient Descent
  • Wireless Aggregation

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Computer Science Applications
  • Information Systems

Fingerprint

Dive into the research topics of 'On the Necessity of Aligning Gradients for Wireless Federated Learning'. Together they form a unique fingerprint.

Cite this