Differentially Private Federated Learning with Drift Control

Wei Ting Chang, Mohamed Seif, Ravi Tandon

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In this paper, we consider the problem of differentially private federated learning with statistical data heterogeneity. More specifically, users collaborate with the parameter server (PS) to jointly train a machine learning model using their local datasets that are non-i.i.d. across users. The PS is assumed to be honest-but-curious so that the data at users need to be kept private from the PS. More specifically, interactions between the PS and users must satisfy differential privacy (DP) for each user. In this work, we propose a differentially private mechanism that simultaneously deals with user-drift caused by non-i.i.d. data and the randomized user participation in the training process. Specifically, we study SCAFFOLD, a popular federated learning algorithm, that has shown better performance on dealing with non-i.i.d. data than previous federated averaging algorithms. We study the convergence rate of SCAFFOLD under differential privacy constraint. Our convergence results take into account time-varying perturbation noises used by the users, and data and user sampling. We propose two time-varying noise allocation schemes in order to achieve better convergence rate and satisfy a total DP privacy budget. We also conduct experiments to confirm our theoretical findings on real world dataset.

Original languageEnglish (US)
Title of host publication2022 56th Annual Conference on Information Sciences and Systems, CISS 2022
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages240-245
Number of pages6
ISBN (Electronic)9781665417969
DOIs
StatePublished - 2022
Event56th Annual Conference on Information Sciences and Systems, CISS 2022 - Princeton, United States
Duration: Mar 9 2022Mar 11 2022

Publication series

Name2022 56th Annual Conference on Information Sciences and Systems, CISS 2022

Conference

Conference56th Annual Conference on Information Sciences and Systems, CISS 2022
Country/TerritoryUnited States
CityPrinceton
Period3/9/223/11/22

Keywords

  • Federated learning
  • Rényi Differential Privacy
  • Sampling
  • Stochastic Gradient Descent

ASJC Scopus subject areas

  • Information Systems and Management
  • Artificial Intelligence
  • Computer Science Applications
  • Information Systems

Fingerprint

Dive into the research topics of 'Differentially Private Federated Learning with Drift Control'. Together they form a unique fingerprint.

Cite this