西安电子科技大学学报 ›› 2024, Vol. 51 ›› Issue (3): 158-169.doi: 10.19665/j.issn1001-2400.20230706

• 网络空间安全 • 上一篇    下一篇

可实现双向自适应差分隐私的联邦学习方案

李洋1,2(), 徐进(), 朱建明1(), 王友卫1,2()   

  1. 1.中央财经大学 信息学院,北京 100081
    2.中央财经大学 国家金融安全教育部工程研究中心,北京 100081
  • 收稿日期:2023-05-04 出版日期:2024-06-20 发布日期:2023-08-22
  • 作者简介:李 洋(1981—),男,副教授,E-mail:liyang@cufe.edu.cn
    徐 进(1999—),男,中央财经大学硕士研究生,E-mail:2022212369@email.cufe.edu.cn
    朱建明(1965—),男,教授,E-mail:zjm@cufe.edu.cn
    王友卫(1987—),男,副教授,E-mail:wangcufe@163.com
  • 基金资助:
    国家重点研发计划(2017YFB1400700);教育部人文社科项目(19YJCZH178);中央财经大学教育教学改革基金2022年度课题(2022ZXJG35);中央财经大学新兴交叉学科建设项目资助

Bidirectional adaptive differential privacy federated learning scheme

LI Yang1,2(), XU Jin(), ZHU Jianming1(), WANG Youwei1,2()   

  1. 1. School of Information,Central University of Finance and Economics,Beijing 100081,China
    2. Ministry of Education Engineering Research Center of State Financial Security,Central University of Finance and Economics,Beijing 100081,China
  • Received:2023-05-04 Online:2024-06-20 Published:2023-08-22

摘要:

随着个人数据的爆发式增长,基于差分隐私的联邦学习模型可用于解决数据孤岛问题和保护用户数据隐私,参与者通过训练本地数据,将添加噪声后的参数共享到中心服务器进行聚合,实现分布式机器学习训练。此过程中存在两方面问题:① 中心服务器广播参数的过程中数据信息仍未受到保护,有泄露用户隐私的风险;② 对参数过度添加噪声会导致参数聚合质量降低,影响最终联邦学习的模型精度。为解决以上问题,提出了一种可实现双向自适应差分隐私的联邦学习方案(FedBADP),对客户端和中心服务器之间传输的梯度进行自适应加噪,在保护数据安全的同时不影响模型准确率。考虑到参与者硬件设备的性能限制,文中对其梯度进行采样以减少通信开销,并在客户端和中心服务器使用均方根传递加速模型的收敛提高模型精度。实验结果证明,文中提出的模型框架在保持较好准确率的同时,也增强了用户的隐私保护能力。

关键词: 双向自适应噪声, 均方根传递, 采样, 差分隐私, 联邦学习

Abstract:

With the explosive growth of personal data,the federated learning based on differential privacy can be used to solve the problem of data islands and preserve user data privacy.Participants share the parameters with noise to the central server for aggregation by training local data,and realize distributed machine learning training.However,there are two defects in this model:on the one hand,the data information in the process of parameters broadcasting by the central server is still compromised,with the risk of user privacy leakage;on the other hand,adding too much noise to parameters will reduce the quality of parameter aggregation and affect the model accuracy of federated learning.In order to solve the above problems,a bidirectional adaptive differential privacy federated learning scheme(Federated Learning Approach with Bidirectional Adaptive Differential Privacy,FedBADP) is proposed,which can adaptively add noise to the gradients transmitted by participants and central servers,and keep data security without affecting the model accuracy.Meanwhile,considering the performance limitations of the participants hardware devices,this model samples their gradients to reduce the communication overhead,and uses the RMSprop to accelerate the convergence of the model on the participants and central server to improve the accuracy of the model.Experiments show that our novel model can enhance the user privacy preserving while maintaining a good accuracy.

Key words: bidirectional adaptive noise, RMSprop, sampling, differential privacy, federated learning

中图分类号: 

  • TP309