自适应差分隐私的高效深度学习方案

doi:10.19665/j.issn1001-2400.2023.04.006

摘要/Abstract

摘要：

深度学习在诸多领域取得成功的同时,也逐渐暴露出严重的隐私安全问题。作为一种轻量级隐私保护技术,差分隐私通过对模型添加噪声使得输出结果对数据集中的任意一条数据都不敏感,更适合现实中个人用户隐私保护的场景。针对现有大多差分隐私深度学习方案中迭代次数对隐私预算的依赖、数据可用性较低和模型收敛速度较慢等问题,提出了一种自适应差分隐私的高效深度学习方案。首先,基于沙普利加性解释模型设计了一种自适应差分隐私机制,通过对样本特征加噪使得迭代次数独立于隐私预算,再利用函数机制扰动损失函数,从而实现对原始样本和标签的双重保护,同时增强数据可用性。其次,利用自适应矩估计算法调整学习率来加快模型收敛速度。并且,引入零集中差分隐私作为隐私损失统计机制,降低因隐私损失超过隐私预算带来的隐私泄露风险。最后,对方案的隐私性进行理论分析,并在MNIST和Fashion-MNIST数据集上通过对比实验,验证了所提方案的有效性。

关键词: 深度学习, 差分隐私, 自适应, 隐私损失, 模型收敛

Abstract:

While deep learning has achieved a great success in many fields,it has also gradually exposed a series of serious privacy security issues.As a lightweight privacy protection technology,differential privacy makes the output insensitive to any data in the dataset by adding noise to the model,which is more suitable for the privacy protection of individual users in reality.Aiming at the problems of the dependence of iterations on the privacy budget,low data availability and slow model convergence in most existing differential private deep learning schemes,an efficient deep learning scheme based on adaptive differential privacy is proposed.First,an adaptive differential privacy mechanism is designed based on the Shapley additive explanation model.By adding noise to the sample features,the number of iterations is independent of the privacy budget,and then the loss function is perturbed by the function mechanism,thus achieving the dual protection of original samples and labels while enhancing the utility of data.Second,the adaptive moment estimation algorithm is used to adjust the learning rate to accelerate the model convergence.Additionally,zero-centralized difference privacy is introduced as a statistical mechanism of privacy loss,which reduces the risk of privacy leakage caused by the privacy loss exceeding the privacy budget.Finally,a theoretical analysis of privacy is made,with the effectiveness of the proposed scheme verified by comparative experiments on the MNIST and Fashion-MNIST datasets.

Key words: deep learning, differential privacy, self-adaptation, privacy loss, model convergence

中图分类号:

TP309

王玉画,高胜,朱建明,黄晨. 自适应差分隐私的高效深度学习方案[J]. 西安电子科技大学学报, 2023, 50(4): 54-64.

WANG Yuhua,GAO Sheng,ZHU Jianming,HUANG Chen. Efficient deep learning scheme with adaptive differential privacy[J]. Journal of Xidian University, 2023, 50(4): 54-64.

图/表 5

参考文献 25

[1]	SONG M K, WANG Z B, ZHANG Z F, et al. Analyzing User-Level Privacy Attack Against Federated Learning[J]. IEEE Journal on Selected Areas in Communications, 2020, 38(10):2430-2444. doi: 10.1109/JSAC.49
[2]	SHOKRI R, STRONATI M, SONG C, et al. Membership Inference Attacks Against Machine Learning Models[C]// 2017 IEEE Symposium on Security and Privacy.Piscataway:IEEE, 2017:3-18.
[3]	SALEM A, ZHANG Y, HUMBERT M, et al. ML-Leaks:Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models[C]// The 26th Annual Network and Distributed System Security Symposium.California:NDSS, 2019:24-27.
[4]	YU L, LIU L, PU C, et al. Differentially Private Model Publishing for Deep Learning[C]// 2019 IEEE Symposium on Security and Privacy (SP).Piscataway:IEEE, 2019:332-349.
[5]	康海燕, 冀源蕊. 基于本地化差分隐私的联邦学习方法研究[J]. 通信学报, 2022, 43(10):94-105. doi: 10.11959/j.issn.1000-436x.2022189
	KANG Haiyan, JI Yuanrui. Research on Federated Learning Approach Based on Local Differential Privacy[J]. Journal on Communications, 2022, 43(10):94-105. doi: 10.11959/j.issn.1000-436x.2022189
[6]	PODSCHWADT R, TAKABI D, HU P, et al. A Survey of Deep Learning Architectures for Privacy-Preserving Machine Learning with Fully Homomorphic Encryption[J]. IEEE Access, 2022, 10:117477-117500. doi: 10.1109/ACCESS.2022.3219049
[7]	CHEN J, LI K, YU P. Privacy-Preserving Deep Learning Model for Decentralized VANETs Using Fully Homomorphic Encryption and Blockchain[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(8):11633-11642. doi: 10.1109/TITS.2021.3105682
[8]	RESENDE A, RAILSBACK D, DOWSLEY R, et al. Fast Privacy-Preserving Text Classification Based on Secure Multiparty Computation[J]. IEEE Transactions on Information Forensics and Security, 2022, 17:428-442. doi: 10.1109/TIFS.2022.3144007
[9]	FENG Q, HE D, SHEN J, et al. PPNNT:Multi-Party Privacy-Preserving Neural Network Training System[J]. IEEE Transactions on Artificial Intelligence, 2023 (Early Access):1-14.
[10]	DWORK C. Differential Privacy[C]// The 33rd International Conference on Automata,Languages and Programming - Volume Part II,ICALP’06.Berlin:Springer, 2006:1-12.
[11]	徐花, 田有亮. 差分隐私下的权重社交网络隐私保护[J]. 西安电子科技大学学报, 2022, 49(1):17-25.
	XU Hua, TIAN Youliang. Protection of Privacy of the Weighted Social Network under Differential Privacy[J]. Journal of Xidian University, 2022, 49(1):17-25.
[12]	晏燕, 董卓越, 徐飞, 等. 一种Hilbert编码的本地化位置隐私保护方法[J]. 西安电子科技大学学报, 2023, 50(2):147-160.
	YAN Yan, DONG Zhuoyue, XU Fei, et al. Localized Location Privacy Protection Method Using the Hilbert Encoding[J]. Journal of Xidian University, 2023, 50(2):147-160.
[13]	HU Y, TAN Z, LI X, et al. Adaptive Clipping Bound of Deep Learning with Differential Privacy[C]// 2021 IEEE International Conference on Trust,Security and Privacy in Computing and Communications (TrustCom).Piscataway:IEEE, 2021:428-435.
[14]	FU J, CHEN Z, HAN X. Adap DP-FL:Differentially Private Federated Learning with Adaptive Noise[C]// 2022 IEEE International Conference on Trust,Security and Privacy in Computing and Communications (TrustCom).Piscataway:IEEE, 2022:656-663.
[15]	WANG F, XIE M, TAN Z, et al. Preserving Differential Privacy in Deep Learning Based on Feature Relevance Region Segmentation[J]. IEEE Transactions on Emerging Topics in Computing, 2023 (Early Access):1-11.
[16]	ABADI M, CHU A, GOODFELLOW I, et al. Deep Learning with Differential Privacy[C]// The 2016 ACM SIGSAC Conference on Computer and Communications Security (CCS '16). New York: ACM, 2016:308-318.
[17]	ZHANG X, DIING J, WU M, et al. Adaptive Privacy Preserving Deep Learning Algorithms for Medical Data[C]// 2021 IEEE Winter Conference on Applications of Computer Vision (WACV).Piscataway:IEEE, 2021:1168-1177
[18]	PHAN N, WU X, HU H. Adaptive Laplace Mechanism:Differential Privacy Preservation in Deep Learning[C]// 2017 IEEE International Conference on Data Mining (ICDM).Piscataway:IEEE, 2017:385-394.
[19]	ZHANG Y, BAI S. An Improved LRP-Based Differential Privacy Preserving Deep Learning Framework[C]// 2021 17th International Conference on Computational Intelligence and Security (CIS).Piscataway:IEEE, 2021:484-488.
[20]	LIU X, LI H, XU G, et al. Adaptive Privacy-Preserving Federated Learning[J]. Peer-to-Peer Networking and Applications, 2020, 13:2356-2366. doi: 10.1007/s12083-019-00869-2
[21]	BUN M, STEINKE T. Concentrated Differential Privacy:Simplifications,Extensions,and Lower Bounds[C]// Theory of Cryptography Conference.Berlin:Springer, 2016:635-658.
[22]	LI C, LOU J, LIU S, et al. Shapley Explainer-An Interpretation Method for GNNs Used in SDN[C]// GLOBECOM 2022-2022 IEEE Global Communications Conference.Piscataway:IEEE, 2022:5534-5540.
[23]	纪守领, 杜天宇, 李进锋, 等. 机器学习模型安全与隐私研究综述[J]. 软件学报, 2021, 32(1):41-67.
	JI Shouling, DU Tianyu, LI Jinfeng, et al. Research on Security and Privacy of Machine Learning Models[J]. Journal of Software, 2021, 32(1):41-67.
[24]	ZHANG J, ZHANG Z, XIAO X, et al. Functional Mechanism:Regression Analysis under Differential Privacy[J]. Proceedings of the VLDB Endowment, 2012, 5(11):1364-1375. doi: 10.14778/2350229.2350253
[25]	KUMAR G, PRIYA G, DILEEP M, et al. Image Deconvolution Using Deep Learning-Based Adam Optimizer[C]// 2022 6th International Conference on Electronics,Communication and Aerospace Technology.Piscataway:IEEE, 2022:901-904.