自适应裁剪的差分隐私联邦学习框架

王方伟; 谢美云; 李青茹; 王长广

doi:10.19665/j.issn1001-2400.2023.04.011

您当前的位置：

首页 >

文章列表页 >

自适应裁剪的差分隐私联邦学习框架

网络空间安全专栏 | 更新时间：2023-10-18

- 自适应裁剪的差分隐私联邦学习框架
- 暂无标题
- 西安电子科技大学学报 2023年50卷第4期页码：111-120
- 作者机构：
  
  河北师范大学计算机与网络空间安全学院河北省网络与信息安全重点实验室,河北石家庄 050024
- 作者简介：
  
  [ "王方伟(1976—),男,教授,E-mail:[email protected];" ]
  [ "谢美云(1998—),女,河北师范大学硕士研究生,E-mail:[email protected];" ]
  [ "李青茹(1971—),女,副教授,E-mail:[email protected]" ]
  王长广(1971—),男,教授,E-mail:[email protected]
- 基金信息：
  
  国家自然科学基金(61572170);河北省自然科学基金(F2021205004);河北省教育厅重点基金(ZD2021062)
- DOI：10.19665/j.issn1001-2400.2023.04.011
  中图分类号：
扫描看全文
王方伟, 谢美云, 李青茹, 等. 自适应裁剪的差分隐私联邦学习框架[J]. 西安电子科技大学学报, 2023,50(4):111-120.
王方伟, 谢美云, 李青茹, 等. 自适应裁剪的差分隐私联邦学习框架[J]. 西安电子科技大学学报, 2023,50(4):111-120. DOI： 10.19665/j.issn1001-2400.2023.04.011.

DOI：

摘要

联邦学习允许参与训练的各方在不共享自己数据的前提下,实现协同建模,其数据隔离策略在一定程度上保障了用户数据的隐私安全,有效缓解了数据孤岛问题。然而,联邦学习的训练过程涉及参与者和服务器之间大量的参数交互,仍存在隐私泄露风险。为解决联邦学习数据传输过程中的隐私保护问题,提出了一种基于自适应裁剪的差分隐私联邦学习ADP_FL框架。在该框架中,各参与方使用自己的数据在本地执行多次迭代来训练模型,在每个迭代中自适应地选取裁剪阈值对梯度进行裁剪,将梯度限制在一个合理范围内;仅向上传的模型参数中添加动态的高斯噪声,以掩藏各参与者的贡献,服务器聚合接收到的噪声参数来更新全局模型。自适应梯度裁剪策略不仅可以实现对梯度的合理校准,同时裁剪阈值作为敏感度当中的一项参数,通过动态改变敏感度来控制着添加的噪声规模。理论分析和实验表明,所提出的框架在强隐私约束下,仍能够实现良好的模型精度。

Abstract

Federation learning allows the parties involved in training to achieve collaborative modeling without sharing their own data.Its data isolation strategy safeguards the privacy and security of user data to a certain extent and effectively alleviates the problem of data silos.However,the training process of federation learning involves a large number of parameter interactions among the participants and the server,and there is still a risk of privacy disclosure.So a differentially private federated learning framework ADP_FL based on adaptive cropping is proposed to address the privacy protection problem during data transmission.In this framework,each participant uses its own data to train the model by performing multiple iterations locally.The gradient is trimmed by adaptively selecting the trimming threshold in each iteration in order to limit the gradient to a reasonable range.Only dynamic Gaussian noise is added to the uploaded model parameters to mask the contribution of each participant.The server aggregates the received noise parameters to update the global model.The adaptive gradient clipping strategy can not only achieve a reasonable calibration of the gradient,but also control the noise scale by dynamically changing the sensitivity while considering the clipping threshold as a parameter in the sensitivity.The results of theoretical analysis and experiments show that the proposed framework can still achieve a great model accuracy under strong privacy constraints.

关键词

联邦学习差分隐私隐私泄漏自适应裁剪

Keywords

federated learningdifferential privacyprivacy disclosureadaptive clipping

references

MCMAHAN H B, MOORE E, RAMAGE D, et al. Communication-Efficient Learning of Deep Networks from Decentralized Data[C]// Proceedings of the 20th International Conference on Artificial Intelligence and Statistics.Piscataway:IEEE, 2017:1273-1282.

顾育豪, 白跃彬. 联邦学习模型安全与隐私研究进展(2022)[J/OL].[2022-09-20]. http://www.jos.org.cn/1000-9825/6658.htm. http://www.jos.org.cn/1000-9825/6658.htmhttp://www.jos.org.cn/1000-9825/6658.htm

GU Yuhao, BAI Yuebin. Survey on Security and Privacy of Federated Learning Models (2022)[J/OL].[2022-09-20]. http://www.jos.org.cn/1000-9825/6658.htm. http://www.jos.org.cn/1000-9825/6658.htmhttp://www.jos.org.cn/1000-9825/6658.htm

HOSSEINI S M, SIKAROUDI M, BABAEI M, et al. Cluster Based Secure Multi-Party Computation in Federated Learning for Histopathology Images[C]// International Workshop on Distributed,Collaborative,and Federated Learning,Workshop on Affordable Healthcare and AI for Resource Diverse Global Health.Heidelberg:Springer, 2022:110-118.

KANAGAVELU R, WEI Q, LI Z, et al. CE-Fed:Communication Efficient Multi-Party Computation Enabled Federated Learning[J]. Array, 2022, 15(9):100207. DOI:10.1016/j.array.2022.100207http://doi.org/10.1016/j.array.2022.100207https://linkinghub.elsevier.com/retrieve/pii/S2590005622000546https://linkinghub.elsevier.com/retrieve/pii/S2590005622000546

MA J, NAAS S A, SIGG S, et al. Privacy-Preserving Federated Learning Based on Multi-Key Homomorphic Encryption[J]. International Journal of Intelligent Systems, 2022, 37(9):5880-5901. DOI:10.1002/int.v37.9http://doi.org/10.1002/int.v37.9https://onlinelibrary.wiley.com/toc/1098111x/37/9https://onlinelibrary.wiley.com/toc/1098111x/37/9

PARK J, LIM H. Privacy-Preserving Federated Learning Using Homomorphic Encryption[J]. Applied Sciences, 2022, 12(2):734. DOI:10.3390/app12020734http://doi.org/10.3390/app12020734https://www.mdpi.com/2076-3417/12/2/734https://www.mdpi.com/2076-3417/12/2/734

张泽辉, 富瑶, 高铁杠. 支持数据隐私保护的联邦深度神经网络模型研究[J]. 自动化学报, 2022, 48(5):1273-1284.

ZHANG Zehui, FU Yao, GAO Tiegang. Research on Federated Deep Neural Network Model for Data Privacy Preserving[J]. Acta Automatica Sinica, 2022, 48(5):1273-1284.

徐花, 田有亮. 差分隐私下的权重社交网络隐私保护[J]. 西安电子科技大学学报, 2022, 49(1):17-25.

XU Hua, TIAN Youliang. Protection of Privacy of the Weighted Social Network under Differential Privacy[J]. Journal of Xidian University, 2022, 48(5):17-25.

刘艺璇, 陈红, 刘宇涵, 等. 联邦学习中的隐私保护技术[J]. 软件学报, 2022, 33(3):1057-1092.

LIU Yixuan, CHEN Hong, LIU Yuhan, et al. Privacy-preserving Techniques in Federated Learning[J]. Journal of Software, 2022, 33(3):1057-1092.

TRUEX S, LIU L, CHOW K H, et al. LDP-Fed:Federated Learning with Local Differential Privacy[C]// Proceedings of the Third ACM International Workshop on Edge Systems,Analytics and Networking. New York: ACM, 2020:61-66.

SUN L, QIAN J, CHEN X. LDP-FL:Practical Private Aggregation in Federated Learning with Local Differential Privacy (2021)[J/OL].[2021-05-21]. https://arxiv.org/pdf/2007.15789v2.pdf. https://arxiv.org/pdf/2007.15789v2.pdfhttps://arxiv.org/pdf/2007.15789v2.pdf

ZHAO Y, ZHAO J, YANG M, et al. Local Differential Privacy-Based Federated Learning for Internet of Things[J]. IEEE Internet of Things Journal, 2020, 8(11):8836-8853. DOI:10.1109/JIOT.2020.3037194http://doi.org/10.1109/JIOT.2020.3037194https://ieeexplore.ieee.org/document/9253545/https://ieeexplore.ieee.org/document/9253545/

CHAMIKAPA M P A, LIU D, CAMTEPE S, et al. Local Differential Privacy for Federated Learning[C]// European Symposium on Research in Computer Security.Heidelberg:Springer, 2022:195-216.

ZHAO J, YANG M, ZHANG R, et al. Privacy-Enhanced Federated Learning:A Restrictively Self-Sampled and Data-Perturbed Local Differential Privacy Method[J]. Electronics, 2022, 11(23):4007. DOI:10.3390/electronics11234007http://doi.org/10.3390/electronics11234007https://www.mdpi.com/2079-9292/11/23/4007https://www.mdpi.com/2079-9292/11/23/4007

LIU X, LI H, XU G, et al. Adaptive Privacy-Preserving Federated Learning[J]. Peer-to-Peer Networking and Applications, 2020, 13(6):2356-2366. DOI:10.1007/s12083-019-00869-2http://doi.org/10.1007/s12083-019-00869-2

WU X, ZHANG Y, SHI M, et al. An Adaptive Federated Learning Scheme with Differential Privacy Preserving[J]. Future Generation Computer Systems, 2022, 127(2):362-372. DOI:10.1016/j.future.2021.09.015http://doi.org/10.1016/j.future.2021.09.015https://linkinghub.elsevier.com/retrieve/pii/S0167739X21003617https://linkinghub.elsevier.com/retrieve/pii/S0167739X21003617

朱建明, 张沁楠, 高胜, 等. 基于区块链的隐私保护可信联邦学习模型[J]. 计算机学报, 2021, 44(12):2464-2484.

ZHU Jianming, ZHANG Qinnan, GAO Sheng, et al. Privacy Preserving and Trustworthy Federated Learning Model Based on Blockchain[J]. Chinses Journal of Computers, 2021, 44(12):2464-2484.

HU R, GONG Y, GUO Y. Federated Learning with Sparsified Model Perturbation:Improving Accuracy under Client-Level Differential Privacy (2022)[J/OL].[2022-11-15]. https://arxiv.org/pdf/2202.07178v2.pdf. https://arxiv.org/pdf/2202.07178v2.pdfhttps://arxiv.org/pdf/2202.07178v2.pdf

LIU W, CHENG J, WANG X, et al. Hybrid Differential Privacy Based Federated Learning for Internet of Things[J]. Journal of Systems Architecture, 2022, 124(3):102418. DOI:10.1016/j.sysarc.2022.102418http://doi.org/10.1016/j.sysarc.2022.102418https://linkinghub.elsevier.com/retrieve/pii/S1383762122000200https://linkinghub.elsevier.com/retrieve/pii/S1383762122000200

SHEN X, LIU Y, ZHANG Z. Performance-Enhanced Federated Learning with Differential Privacy for Internet of Things[J]. IEEE Internet of Things Journal, 2022, 9(23):24079-24094. DOI:10.1109/JIOT.2022.3189361http://doi.org/10.1109/JIOT.2022.3189361https://ieeexplore.ieee.org/document/9820771/https://ieeexplore.ieee.org/document/9820771/

LIAN Z, WANG W, HUANG H, et al. Layer-Based Communication-Efficient Federated Learning with Privacy Preservation[J]. IEICE Transactions on Information and Systems, 2022, 105(2):256-263.

BAEK C, KIM S, NAM D, et al. EnhancingDifferential Privacy for Federated Learning at Scale[J]. IEEE Access, 2021, 9(10):148090-148103. DOI:10.1109/ACCESS.2021.3124020http://doi.org/10.1109/ACCESS.2021.3124020https://ieeexplore.ieee.org/document/9592806/https://ieeexplore.ieee.org/document/9592806/

YANG Q, LIU Y, CHEN T, et al. Federated Machine Learning:Concept and Applications[J]. ACM Transactions on Intelligent Systems and Technology (TIST), 2019, 10(2):1-19.

DWORK C, ROTH A. The Algorithmic Foundations of Differential Privacy[J]. Foundations and Trends in Theoretical Computer Science, 2014, 9(3):211-407. DOI:10.1561/0400000042http://doi.org/10.1561/0400000042http://www.nowpublishers.com/articles/foundations-and-trends-in-theoretical-computer-science/TCS-042http://www.nowpublishers.com/articles/foundations-and-trends-in-theoretical-computer-science/TCS-042

DWORK C, ROTHBLUM G N, VADHAN S. Boosting and differential privacy[C]// 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.Piscataway:IEEE, 2010:51-60.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

隐私保护的拜占庭鲁棒联邦学习算法

一种高效的联邦学习隐私保护方案

面向医疗数据的隐私保护联邦学习架构

反迁移学习的隐私保护联邦学习

自适应差分隐私的高效深度学习方案