可实现双向自适应差分隐私的联邦学习方案

李洋; 徐进; 朱建明; 王友卫

doi:10.19665/j.issn1001-2400.20230706

您当前的位置：

首页 >

文章列表页 >

可实现双向自适应差分隐私的联邦学习方案

网络空间安全 | 更新时间：2024-07-23

- 可实现双向自适应差分隐私的联邦学习方案
- Bidirectional adaptive differential privacy federated learning scheme
- 西安电子科技大学学报 2024年51卷第3期页码：158-169
- 作者机构：
  
  1. 中央财经大学信息学院,北京 100081
  2. 中央财经大学国家金融安全教育部工程研究中心,北京 100081
- 作者简介：
  
  [ "李洋(1981—),男,副教授,E-mail:[email protected]" ]
  [ "徐进(1999—),男,中央财经大学硕士研究生,E-mail:[email protected]" ]
  [ "朱建明(1965—),男,教授,E-mail:[email protected]" ]
  [ "王友卫(1987—),男,副教授,E-mail:[email protected]" ]
- 基金信息：
  
  国家重点研发计划(2017YFB1400700);教育部人文社科项目(19YJCZH178);中央财经大学教育教学改革基金2022年度课题(2022ZXJG35);中央财经大学新兴交叉学科建设项目资助
- DOI：10.19665/j.issn1001-2400.20230706
  中图分类号： TP309;TP18
- 纸质出版日期：2024-06-20，
  
  网络出版日期：2023-08-22，
  
  收稿日期：2023-05-04，
- 稿件说明：
扫描看全文
李洋, 徐进, 朱建明, 等. 可实现双向自适应差分隐私的联邦学习方案[J]. 西安电子科技大学学报, 2024,51(3):158-169.

Yang LI, Jin XU, Jianming ZHU, et al. Bidirectional adaptive differential privacy federated learning scheme[J]. Journal of Xidian University, 2024,51(3):158-169.
李洋, 徐进, 朱建明, 等. 可实现双向自适应差分隐私的联邦学习方案[J]. 西安电子科技大学学报, 2024,51(3):158-169. DOI： 10.19665/j.issn1001-2400.20230706.

Yang LI, Jin XU, Jianming ZHU, et al. Bidirectional adaptive differential privacy federated learning scheme[J]. Journal of Xidian University, 2024,51(3):158-169. DOI： 10.19665/j.issn1001-2400.20230706.

摘要

随着个人数据的爆发式增长

基于差分隐私的联邦学习模型可用于解决数据孤岛问题和保护用户数据隐私

参与者通过训练本地数据

将添加噪声后的参数共享到中心服务器进行聚合

实现分布式机器学习训练。此过程中存在两方面问题:① 中心服务器广播参数的过程中数据信息仍未受到保护

有泄露用户隐私的风险;② 对参数过度添加噪声会导致参数聚合质量降低

影响最终联邦学习的模型精度。为解决以上问题

提出了一种可实现双向自适应差分隐私的联邦学习方案(FedBADP)

对客户端和中心服务器之间传输的梯度进行自适应加噪

在保护数据安全的同时不影响模型准确率。考虑到参与者硬件设备的性能限制

文中对其梯度进行采样以减少通信开销

并在客户端和中心服务器使用均方根传递加速模型的收敛提高模型精度。实验结果证明

文中提出的模型框架在保持较好准确率的同时

也增强了用户的隐私保护能力。

Abstract

With the explosive growth of personal data

the federated learning based on differential privacy can be used to solve the problem of data islands and preserve user data privacy.Participants share the parameters with noise to the central server for aggregation by training local data

and realize distributed machine learning training.However

there are two defects in this model:on the one hand

the data information in the process of parameters broadcasting by the central server is still compromised

with the risk of user privacy leakage;on the other hand

adding too much noise to parameters will reduce the quality of parameter aggregation and affect the model accuracy of federated learning.In order to solve the above problems

a bidirectional adaptive differential privacy federated learning scheme(Federated Learning Approach with Bidirectional Adaptive Differential Privacy

FedBADP) is proposed

which can adaptively add noise to the gradients transmitted by participants and central servers

and keep data security without affecting the model accuracy.Meanwhile

considering the performance limitations of the participants hardware devices

this model samples their gradients to reduce the communication overhead

and uses the RMSprop to accelerate the convergence of the model on the participants and central server to improve the accuracy of the model.Experiments show that our novel model can enhance the user privacy preserving while maintaining a good accuracy.

关键词

双向自适应噪声均方根传递采样差分隐私联邦学习

Keywords

bidirectional adaptive noiseRMSpropsamplingdifferential privacyfederated learning

references

MCMAHAN H B, MOORE E, RAMAGE D, et al. Communication-Efficient Learning of Deep Networks from Decentralized Data[C]//Proceedings of the 20th International Conference on Artificial Intelligence and Statistics. New York: PMLR, 2017:1273-1282.

JIANG J, KANTARCI B, OKTUG S F, et al. Federated Learning in Smart City Sensing:Challenges and Opportunities[J]. Sensors, 2020, 20(21):6230.

XU J, GLICKSBERG B S, SU C, et al. Federated Learning for Healthcare Informatics[J]. Journal of Healthcare Informatics Research 5, 2021, 5(1):1-19.

TU X, ZHU K, LUONG N C, et al. Incentive Mechanisms for Federated Learning:From Economic and Game Theoretic Perspective(2021)[J/OL].[2021-11-20]. https://arxiv.org/abs/2111.11850v1.https://arxiv.org/abs/2111.11850v1https://arxiv.org/abs/2111.11850v1

陈嘉翊, 孙晨雨, 周欣桐, 等. 基于联邦学习和同态加密的电力数据预测模型本地保护[J]. 信息安全研究, 2023, 9(3):228-234.

CHEN Jiayi, SUN Chenyu, ZHOU Xintong, et al. Local Privacy Protection for Power Data Prediction Model Based on Federated Learning and Homomorphic Encryption[J]. Information Security Research, 2023, 9(3):228-234.

徐花, 田有亮. 差分隐私下的权重社交网络隐私保护[J]. 西安电子科技大学学报, 2022, 49(1):17-25.

XU Hua, TIAN Youliang. Protection of Privacy of the Weighted Social Network under Differential Privacy[J]. Journal of Xidian University, 2022, 49(1):17-25.

WANG F, XIE M, TAN Z, et al. Preserving Differential Privacy in Deep Learning Based on Feature Relevance Region Segmentation[J]. IEEE Transactions on Emerging Topics in Computing, 2023, 12(1):307-315.

FU J, CHEN Z, HAN X. Adap DP-FL:Differentially Private Federated Learning with Adaptive Noise[C]//2022 IEEE International Conference on Trust,Security and Privacy in Computing and Communications(TrustCom).Piscataway:IEEE, 2022:656-663.

粟勇, 刘文龙, 刘圣龙, 等. 基于安全洗牌和差分隐私的联邦学习模型安全防护方法[J]. 信息安全研究, 2022, 8(3):270-276.

SU Yong, LIU Wenlong, LIU Shenglong, et al. Secure Protection Method for Federated Learning Model Based on Secure Shuffling and Differential Privacy[J]. Information Security Research, 2022, 8(3):270-276.

晏燕, 董卓越, 徐飞, 等. 一种Hilbert编码的本地化位置隐私保护方法[J]. 西安电子科技大学学报, 2022, 50(2):147-160.

YAN Yan, DONG Zhuofei, XU Fei, et al. Localized Location Privacy Protection Method Using the Hilbert Encoding[J]. Journal of Xidian University, 2022, 50(2):147-160.

KONEN J, MCMAHAN H B, YU F X, et al. Federated Learning:Strategies for Improving Communication Efficiency(2016)[J/OL].[2016-10-18].https://arxiv.org/abs/1610.05492.https://arxiv.org/abs/1610.05492https://arxiv.org/abs/1610.05492

LI T, SAHU A K, ZAHEER M, et al. Federated Optimization in Heterogeneous Networks(2018)[J/OL].[2018-12-14]. https://arxiv.org/abs/1812.06127v5.https://arxiv.org/abs/1812.06127v5https://arxiv.org/abs/1812.06127v5

LIU W, CHEN L, CHEN Y, et al. Accelerating Federated Learning via Momentum Gradient Descent[J]. IEEE Transactions on Parallel and Distributed Systems, 2020, 31(8):1754-66.

LIU X, LI Y, WANG Q, et al. Sparse Personalized Federated Learning via Maximizing Correlation(2021)[J/OL].[2021-07-12]. https://arxiv.org/abs/2107.05330v3.https://arxiv.org/abs/2107.05330v3https://arxiv.org/abs/2107.05330v3

MELIS L, SONG C, CRISTOFARO E D, et al. Inference Attacks Against Collaborative Learning(2018)[J/OL].[2018-05-10]. https://arxiv.org/abs/1805.04049v1.https://arxiv.org/abs/1805.04049v1https://arxiv.org/abs/1805.04049v1

ABADI M, CHU A, GOODFELLOW I, et al. Deep Learning with Differential Privacy[C]//Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. New York: ACM, 2016:308-318.

WEI K, LI J, DING M, et al. Federated Learning with Differential Privacy:Algorithms and Performance Analysis[J]. IEEE Transactions on Information Forensics and Security, 2020, 15:3454-3469.

WEI K, LI J, DING M, et al. User-Level Privacy-Preserving Federated Learning:Analysis and Performance Optimization[J]. IEEE Transactions on Mobile Computing, 2022, 21(9):3388-3401.

WU X, ZHANG Y, SHI M, et al. AnAdaptive Federated Learning Scheme with Differential Privacy Preserving[J]. Future Generation Computer Systems, 2022, 127:362-72.

XU Z, SHI S, LIU A X, et al. An Adaptive and Fast Convergent Approach to Differentially Private Deep Learning[C]//Proceedings of the IEEE INFOCOM 2020:International Conference on Computer Communications(INFOCOM). Piscataway:IEEE, 2020:1867-1876.

KUMAR G, PRIYA G, DILEEP M, et al. Image Deconvolution using Deep Learning-based Adam Optimizer[C]//2022 6th International Conference on Electronics,Communication and Aerospace Technology.Piscataway:IEEE, 2022:901-904.

XIANG L, YANG J, LI B. Differentially-Private Deep Learning from an Optimization Perspective[C]//Proceedings of the IEEE Conference on Computer Communications. Piscataway:IEEE, 2019:559-567.

DWORK C. Differential Privacy[C].Proceedings of the 33rd International Conference on Automata,Languages and Programming. Heidelberg:Springer, 2006:1-10.

LI N, LYU M, SU D, et al. Differential Privacy:From Theory to Practice[J]. Synthesis Lectures on Information Security,Privacy,and Trust, 2016, 8(4):1-138.

GEYER R C, KLEIN T, NABI M. Differentially Private Federated Learning:A Client Level Perspective(2017)[J/OL].[2017-12-20]. https://arxiv.org/abs/1712.07557v2.https://arxiv.org/abs/1712.07557v2https://arxiv.org/abs/1712.07557v2

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

自适应裁剪的差分隐私联邦学习框架

隐私保护的拜占庭鲁棒联邦学习算法

联邦加密流量分类中的细粒度防御方法

一种高效的联邦学习隐私保护方案

面向医疗数据的隐私保护联邦学习架构