面向对抗样本干扰的人脸识别隐私保护方案

doi:10.19665/j.issn1001-2400.20241105

西安电子科技大学学报 ›› 2025, Vol. 52 ›› Issue (2): 190-200.doi: 10.19665/j.issn1001-2400.20241105

面向对抗样本干扰的人脸识别隐私保护方案

马彩霞¹^,²(), 贾春福¹^,²(), 蔡智鹏¹^,²(), 杜瑞忠³^,⁴(), 李明月³^,⁴(), 哈冠雄¹^,²()

1.南开大学网络空间安全学院,天津 300350
2.天津市网络与数据安全技术重点实验室,天津 300350
3.河北大学网络空间安全与计算机学院,河北保定 071002
4.河北省高可信信息系统重点实验室,河北保定 071002

收稿日期:2024-09-12 出版日期:2025-04-20 发布日期:2024-11-27
通讯作者: 贾春福(1966—),男,教授,E-mail:cfjia@nankai.edu.cn
作者简介:马彩霞(1997—),女,南开大学博士研究生,E-mail:macaixia20@163.com;
蔡智鹏(2001—),男,南开大学硕士研究生,E-mail:nkucaizp@mail.nankai.edu.cn;
杜瑞忠(1975—),男,教授,E-mail:durz@hbu.edu.cn;
李明月(1993—),女,副教授,E-mail:limingyue@hbu.edu.cn;
哈冠雄(1995—),男,南开大学博士研究生,E-mail:hgx1995@mail.nankai.edu.cn
基金资助:
国家重点研发计划(2018YFA0704703);国家自然科学基金(61972215);国家自然科学基金(62172238);国家自然科学基金(61972073);天津市自然科学基金(20JCZDJC00640);中央高校基本科研业务费专项资金资助

Privacy-preserving face recognition against adversarial sample perturbations

MA Caixia¹^,²(), JIA Chunfu¹^,²(), CAI Zhipeng¹^,²(), DU Ruizhong³^,⁴(), LI Mingyue³^,⁴(), HA Guanxiong¹^,²()

1. College of Cyber Science,Nankai University,Tianjin 300350,China
2. Tianjin Key Laboratory of Network and Data Security Technology,Tianjin 300350,China
3. School of Cyber Security and Computer,Hebei University,Baoding 071002,China
4. Hebei Province Key Laboratory of High Confidence Information System,Baoding 071002,China

Received:2024-09-12 Online:2025-04-20 Published:2024-11-27

摘要/Abstract

摘要：

人脸识别技术的广泛应用给人们带来了极大的便利,但也面临着身份等隐私信息被泄露的风险。针对人脸识别的攻击可通过重构人脸图像来获取原始图像的隐私信息,为防范此类攻击,提出了一种面向对抗样本干扰的人脸识别隐私保护方案(PPFR-ASP)。该方案采用频域技术中的协同推理机制,首先将人脸图像转换为频域特征,并分成高频成分特征和低频成分特征,然后在频域特征上添加对抗样本干扰。其次,为每个原始图像准备一个目标图像,对目标图像做和原始图像相同的频域特征转换处理,然后使用目标图像的频率成分通道遮盖原始图像中的相应成分,使攻击者将重构图像识别为目标图像,从而隐藏原始图像的真实身份。最后,文中所提方案在多个数据集上进行了大量的实验,结果表明其人脸图像隐私保护性能高于对比方案,查询精确度和计算开销与无隐私保护的ArcFace方案相接近。

关键词: 人脸识别, 隐私保护, 频域技术, 对抗样本干扰

Abstract:

While the wide application of face recognition technology brings great convenience to people,this technology also faces the risk of identity and other private information being leaked.Some current attacks against face recognition can obtain the privacy information on the original image by reconstructing the face image.To prevent such attacks,this paper proposes a privacy-preserving face recognition model against adversarial sample perturbation(PPFR-ASP).Specifically,our scheme employs collaborative inference in the frequency domain.First,the face image is transformed into frequency domain features,which are divided into high-frequency and low-frequency components.Then we add adversarial sample perturbation on the frequency domain features.Furthermore,a target image is prepared for each original image,and the same frequency domain feature transformation is applied to the target image.The frequency component channels of the target image are used to mask the corresponding components in the original image so as to cause the attacker to reconstruct the image as the target image,thereby concealing the true identity of the original image.Finally,extensive experiments are conducted on multiple datasets,demonstrating that the privacy protection performance of this solution for facial image data exceeds that of comparative approaches.The query accuracy and computational overhead of this solution are comparable to those of the unprotected ArcFace scheme.

Key words: face recognition, privacy-preserving, frequency domain technique, adversarial sample perturbations

中图分类号:

TP391

马彩霞, 贾春福, 蔡智鹏, 杜瑞忠, 李明月, 哈冠雄. 面向对抗样本干扰的人脸识别隐私保护方案[J]. 西安电子科技大学学报, 2025, 52(2): 190-200.

MA Caixia, JIA Chunfu, CAI Zhipeng, DU Ruizhong, LI Mingyue, HA Guanxiong. Privacy-preserving face recognition against adversarial sample perturbations[J]. Journal of Xidian University, 2025, 52(2): 190-200.

图/表 14

图1

表1

图2

图3

图4

图5

表2

图6

图7

图8

图9

表3

图10

表4

参考文献 36

[1]	蔡英, 张猛, 李新, 等. 面向ASPE的抗合谋攻击图像检索隐私保护方案[J]. 西安电子科技大学学报, 2023, 50(5):156-165.
	CAI Ying, ZHANG Meng, LI Xin, et al. Privacy-Preserving Scheme for Image Retrieval Against Collusion Attacks Targeting ASPE[J]. Journal of Xidian University, 2023, 50(5):156-165.
[2]	SUN X, TIAN C L, HU C H, et al. Privacy-Preserving and Verifiable SRC-Based Face Recognition with Cloud/Edge Server Assistance[J]. Computers & Security, 2022, 118:102740.
[3]	LIU M, HU H S, XIANG H L, et al. Clustering-Based Efficient Privacy-Preserving Face Recognition Scheme without Compromising Accuracy[J]. ACM Transactions on Sensor Networks, 2021, 17(3):1-27.
[4]	HUANG Y S B, SONG Z, LI K, et al. Instahide:Instance-Hiding Schemes for Private Distributed Learning[C]// Proceedings of the 37th International Conference on Machine Learning. New York: ICML, 2020:4507-4518.
[5]	CHAMIKARA M A P, BERT'OK P, KHALIL I, et al. Privacy Preserving Face Recognition Utilizing Differential Privacy[J]. Computers & Security, 2020, 97:101951.
[6]	JI J Z, WANG H, HUANG Y G, et al. Privacy-Preserving Face Recognition with Learnable Privacy Budgets in Frequency Domain[C]//European Conference on Computer Vision-ECCV. Berlin:Springer, 2022:475-491.
[7]	MIRJALILI V, ROSS A. Soft Biometric Privacy:Retaining Biometric Utility of Face Images while Perturbing Gender[C]// 2017 IEEE International Joint Conference on Biometrics.Piscataway:IEEE, 2017: 564-573.
[8]	WANG Y G, LIU J, LUO M, et al. Privacy-Preserving Face Recognition in the Frequency Domain[C]// Thirty-Sixth AAAI Conference on Artificial Intelligence. Palo Alto: AAAI, 2022:2558-2566.
[9]	MIRESHGHALLAH F, TARAM M, JALALI A, et al. Not All Features are Equal:Discovering Essential Features for Preserving Prediction Privacy[C]// WWW’21:The Web Conference 2021. New York: ACM, 2021:669-680.
[10]	GOSWAMI G, AGARWAL A, NALINI K, et al. Detecting and Mitigating Adversarial Perturbations for Robust Face Recognition[J]. International Journal of Computer Vision, 2019, 127(6-7):719-742.
[11]	KOMKOV S, PETIUSHKO A. Advhat:Real-World Adversarial Attack on Arcface Face ID System[C]//25th International Conference on Pattern Recognition. Piscataway:IEEE, 2021:819-826.
[12]	EVTIMOV I, EYKHOLT K, FERNANDES E, et al. Robust Physical-World Attacks on Machine Learning Models(2017)[J/OL].[2018-04-10]. https://arxiv.org/abs/1707.08945v2.
[13]	SONG D, EYKHOLT K, EVTIMOV I, et al. Physical Adversarial Examples for Object Detectors[C]//12th USENIX Workshop on Offensive Technologies. Berkeley:USENIX, 2018:1-10.
[14]	WU Z X, LIM S N, LARRY S, et al. Making an Invisibility Cloak:Real World Adversarial Attacks on Object Detectors[C]//European Conference on Computer Vision-ECCV 2020. Berlin:Springer, 2020:1-17.
[15]	DONG Y P, SU H, WU B Y, et al. Efficient Decision-Based Black-Box Adversarial Attacks on Face Recognition[C]// IEEE Conference on Computer Vision and Pattern Recognition,CVPR 2019.Piscataway:IEEE, 2019:7714-7722.
[16]	ZHONG Y Y, DENG W H. Towards Transferable Adversarial Attack Against Deep Face Recognition[J]. IEEE Transactions on Information Forensics and Security, 2021, 16:1452-1466.
[17]	SHARIF M, BHAGAVATULA S, BAUER L, et al. A General Framework for Adversarial Examples with Objectives[J]. ACM Transactions on Privacy and Security, 2019, 22(3):1-30.
[18]	YANG L, SONG Q, WU Y Q. Attacks on State-of-the-Art Face Recognition Using Attentional Adversarial Attack Generative Network[J]. Multimedia Tools and Applications, 2021, 80(1):855-875.
[19]	DEB D, ZHANG J B, ANIL K. Advfaces:Adversarial Face Synthesis[C]// 2020 IEEE International Joint Conference on Biometrics,IJCB 2020.Piscataway:IEEE, 2020: 1-10.
[20]	XIAO Z H, GAO X F, FU C L, et al. Improving Transferability of Adversarial Patches on Face Recognition with Generative Models[C]// IEEE Conference on Computer Vision and Pattern Recognition,CVPR 2021.Piscataway:IEEE, 2021:11845-11854.
[21]	魏梓钰, 杨曦, 王楠楠, 等. 互惠双向生成对抗网络用于跨模态行人重识别[J]. 西安电子科技大学学报, 2021, 48(2):205-212.
	WEI Ziyu, YANG Xi, WANG Nannan, et al. Reciprocal Bi-Directional Generative Adversarial Network for Cross-Modal Pedestrian Re-Identification[J]. Journal of Xidian University, 2021, 48(2):205-212.
[22]	YANG X, DONG Y P, PANG T Y, et al. Towards Face Encryption by Generating Adversarial Identity Masks[C]// 2021 IEEE/CVF International Conference on Computer Vision,ICCV 2021.Piscataway:IEEE, 2021: 3877-3887.
[23]	XIE C H, ZHANG Z H, ZHOU Y Y, et al. Improving Transferability of Adversarial Examples with Input Diversity[C]// IEEE Conference on Computer Vision and Pattern Recognition,CVPR 2019.Piscataway:IEEE, 2019:2730-2739.
[24]	DONG Y P, LIAO F Z, PANG T Y, et al. Boosting Adversarial Attacks with Momentum[C]// 2018 IEEE Conference on Computer Vision and Pattern Recognition,CVPR 2018.Piscataway:IEEE, 2018: 9185-9193.
[25]	KURAKIN A, GOODFELLOW I J, BENGIO S. Adversarial Machine Learning at Scale[C]// 5th International Conference on Learning Representations,ICLR 2017. La Jolla: ICLR, 2017:1-17.
[26]	FRANK J, EISENHOFER T, SCHÖNHERR L, et al. Leveraging Frequency Analysis for Deep Fake Image Recognition[C]// Proceedings of the 37th International Conference on Machine Learning,ICML 2020. New York: ICML, 2020:3247-3258.
[27]	HUANG J X, GUAN D, XIAO A, et al. FSDR:Frequency Space Domain Randomization for Domain Generalization[C]// IEEE Conference on Computer Vision and Pattern Recognition,CVPR 2021.Piscataway:IEEE, 2021:6891-6902.
[28]	KAGAWADEV C, ANGADI S A. Fusion of Frequency Domain Features of Face and Iris Traits for Person Identification[J]. Journal of the Institution of Engineers(India) Series B, 2021, 102(5):987-996.
[29]	EHRLICH M, DAVIS L. Deep Residual Learning in the JPEG Transform Domain[C]// 2019 IEEE/CVF International Conference on Computer Vision,ICCV 2019.Piscataway:IEEE, 2019: 3483-3492.
[30]	XU K, QIN M G, SUN F, et al. Learning in the Frequency Domain[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway:IEEE, 2020:1740-1749.
[31]	SANTOS S F D, ALMEIDA J. Less is More:Accelerating Faster Neural Networks Straight from JPEG[C]// Progress in Pattern Recognition,Image Analysis,Computer Vision,and Applications,CIARP 2021.Berlin:Springer, 2021:237-247.
[32]	DENG J K, GUO J, YANG J, et al. Arcface:Additive Angular Margin Loss for Deep Face Recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(10):5962-5979.
[33]	WAHIDA F, CHAMIKARA M, KHALIL I. An Adversarial Machine Learning Based Approach for Privacy Preserving Face Recognition in Distributed Smart City Surveillance[J]. Computer Networks, 2024,254:110798.
[34]	王滨, 陈思, 陈加栋, 等. DWB-AES:基于AES的动态白盒实现方法[J]. 通信学报, 2021, 42(2):177-186. doi: 10.11959/j.issn.1000-436x.2021020
	WANG Bin, CHEN Si, CHEN Jiadong, et al. DWB-AES:An Implementation of Dynamic White-Box Based on AES[J]. Journal on Communications, 2021, 42(2):177-186. doi: 10.11959/j.issn.1000-436x.2021020
[35]	HUANG Q, KATSMAN I, GU Z Q, et al. Enhancing Adversarial Example Transferability with An Intermediate Level Attack[C]// 2019 IEEE/CVF International Conference on Computer Vision,ICCV 2019.Piscataway:IEEE, 2019: 4732-4741.
[36]	YANG Y M, HU W P, HU H F. Syncretic Space Learning Network for NIR-VIS Face Recognition[J]. ACM Transactions on Multimedia Computing,Communications,and Applications, 2023, 20(1):1-25.

参数	含义
x^o	原始图像
x^t	目标图像
x^p	低频干扰后的图像
x^p'	高频最低值干扰后的图像
D	原始图像数据集
D_id	原始图像数据集的子集
X-O	重构的原始图像
X-T	重构的目标图像
T_i	目标图像的身份
id	原始图像的真实身份

方案	LFW/%	CFP/%	CALFW/%	CPLFW/%	AgeDB/%	Vggface/%	隐私保护
ArcFace^[32]	99.82	97.63	96.05	92.50	97.92	95.12	×
PPFR-ASP	99.81	97.36	95.89	91.67	97.61	94.44	√
PPFR-FD^[8]	99.68	95.04	95.72	90.78	97.37	94.08	√
PEEP^[5]	98.41	74.47	97.47	79.58	90.06	88.85	√
InstalHide^[4]	98.45	94.43	92.98	89.15	96.01	92.87	√
FRLBP^[6]	99.48	97.20	93.47	90.60	94.37	93.11	√
WCKA^[33]	98.55	89.36	92.95	88.85	90.15	94.91	√

方案	LFW	CFP	CALFW	CPLFW	AgeDB	Vggface
ArcFace^[32]	99.82	97.63	96.05	92.50	97.92	95.12
PPFR-ASP	99.81	97.36	95.89	91.67	97.61	94.44
ASP-LF	99.82	97.56	96.02	92.45	97.89	95.05
ASP-HFMin	99.81	96.45	95.98	92.27	97.72	94.89

处理过程	复杂度
识别过程时间复杂度	O(f(n))
识别过程空间复杂度	O(m)
干扰过程时间复杂度	O(3×g(n))
干扰过程空间复杂度	O(h)
整体时间复杂度	O(f(n)+3×g(n))
整体空间复杂度	O(m+h)

面向对抗样本干扰的人脸识别隐私保护方案

Privacy-preserving face recognition against adversarial sample perturbations

RichHTML

PDF (PC)

赞

可视化

摘要/Abstract

引用本文

使用本文

图/表 14

参考文献 36

相关文章 15

Metrics

本文评价

推荐阅读 0

[1]	谢玮璇, 郭子裕, 左金鑫, 郭辰青, 陆月明. 一种面向数据扰动的动态均衡隐私模型[J]. 西安电子科技大学学报, 2025, 52(2): 201-213.
[2]	曹来成, 后杨宁, 冯涛, 郭显. 面向动态博弈的k-匿名隐私保护数据共享方案[J]. 西安电子科技大学学报, 2024, 51(4): 170-179.
[3]	郭庆, 田有亮. 支持受控共享的医疗数据隐私保护方案[J]. 西安电子科技大学学报, 2024, 51(1): 165-176.
[4]	丁红发,傅培旺,彭长根,龙士工,吴宁博. 混洗差分隐私保护的度分布直方图发布算法[J]. 西安电子科技大学学报, 2023, 50(6): 219-236.
[5]	蔡英,张猛,李新,张宇,范艳芳. 面向ASPE的抗合谋攻击图像检索隐私保护方案[J]. 西安电子科技大学学报, 2023, 50(5): 156-165.
[6]	王波,李洪涛,王洁,郭一娜. 面向医疗数据的隐私保护联邦学习架构[J]. 西安电子科技大学学报, 2023, 50(5): 166-177.
[7]	宋成,程道晨,彭维平. 一种高效的联邦学习隐私保护方案[J]. 西安电子科技大学学报, 2023, 50(5): 178-187.
[8]	曹来成,吴文涛,冯涛,郭显. 面向数据质量的隐私保护多分类LR方案[J]. 西安电子科技大学学报, 2023, 50(5): 188-198.
[9]	杨晓晖,庄海晶. 一种标识符动态协商的匿名通信模型[J]. 西安电子科技大学学报, 2023, 50(4): 100-110.
[10]	李海洋,郭晶晶,刘玖樽,刘志全. 隐私保护的拜占庭鲁棒联邦学习算法[J]. 西安电子科技大学学报, 2023, 50(4): 121-131.
[11]	丁红发,唐明丽,刘海,蒋合领,傅培旺,于莹莹. 邻居子图扰动下的k-度匿名隐私保护模型[J]. 西安电子科技大学学报, 2023, 50(4): 180-193.
[12]	许勐璠,李兴华. 反迁移学习的隐私保护联邦学习[J]. 西安电子科技大学学报, 2023, 50(4): 89-99.
[13]	李文华,董丽华,曾勇. key-nets同态加密方案的安全性分析及改进[J]. 西安电子科技大学学报, 2023, 50(1): 192-202.
[14]	左开中,刘蕊,赵俊,谌章义,陈付龙. 融合语义信息的时空关联位置隐私保护方法[J]. 西安电子科技大学学报, 2022, 49(1): 67-77.
[15]	郑献春,李晖,王瑞,闫皓楠,戴睿,萧明炽. 匿名网络应用及仿真平台研究综述[J]. 西安电子科技大学学报, 2021, 48(1): 22-38.