支持隐私保护的智能物联网数据风格转换

doi:10.16180/j.cnki.issn1007-7820.2024.10.001

摘要/Abstract

摘要：

传统智能物联网隐私保护技术主要嵌入在数据生命周期的传输、存储和分析阶段,忽视了在源头上保护数据隐私的重要性。文中提出一种支持隐私保护的智能物联网数据风格转换方法,在CycleGAN风格转换模型的基础上新增混淆身份信息的损失函数,使得真实风格图像和动画风格图像在视觉上能够互相转化。动画风格的数据可用于数字世界(例如元宇宙等)中各类虚拟实体的构建,恶意用户无法根据虚拟实体逆向原始数据,或所逆向的原始数据无法被原深度学习模型正确识别,从而增强对物理世界真实实体的隐私保护。在人脸数据集上的实验结果表明,转换后的数据在不明显降低视觉失真度的条件下可使ArcFace人脸识别模型精度下降30%。

关键词: 风格转换, 对抗样本, 人脸识别, CycleGAN, 数字孪生, 智能物联网, 隐私保护, 元宇宙

Abstract:

In the artificial intelligence of things, traditional privacy protection technologies mainly focus on the transmission, storage, and analysis stages of the data lifecycle, while ignoring the importance of protecting data privacy at the source. This study proposes a privacy-protecting data style transfer method for artificial intelligence of things. Based on cycle-consistent adversarial networks, a new loss function is added to obfuscate identity information, allowing real-style images and animation-style images to visually transform into each other. Animation-style data can be used to construct various virtual entities in the digital world (such as metaverses), and malicious users cannot reverse the original data based on the virtual entities or correctly identify the original data using the original deep learning model, thereby enhancing privacy protection for real entities in the physical world. Experimental results on a face dataset show that the transformed data reduces the accuracy of the ArcFace face recognition model by 30% without significantly reducing visual distortion.

Key words: style transfer, adversarial examples, face recognition, CycleGAN, digital twin, artificial intelligence of things, privacy protection, metaverse

中图分类号:

TP391

程锦科, 李高磊. 支持隐私保护的智能物联网数据风格转换[J]. 电子科技, 2024, 37(10): 1-5.

CHENG Jinke, LI Gaolei. Privacy-Preserving Data Style Transfer Method for Artificial Intelligence of Things[J]. Electronic Science and Technology, 2024, 37(10): 1-5.

图/表 6

图1

图2

图3

图4

表1

图5

参考文献 25

[1]	Mireshghallah F. Privacy in deep learning:A survey[EB/OL].(2020-04-25)[2023-02-25]https://arxiv.org/abs/2004.12254.
[2]	Leino K, Fredrikson M. Stolen memories:Leveraging model memorization for calibrated white-box membership inference[C]. Berkeley: The Twenty-ninth USENIX Security Symposium, 2020:1605-1622.
[3]	Zhang Y, Jia R, Pei H, et al. The secret revealer:Generative model-inversion attacks against deep neural networks[C]. Seattle: IEEE Conference on Computer Vision and Pattern Recognition,2020:253-261.
[4]	Nasr M, Songi S, Thakurta A, et al. Adversary instantiateon:Lower bounds for differentially private machine learning[C]. Oakland: IEEE Symposium on Security and Privacy,2021:866-882.
[5]	Xie L, Lin K, Wang S, et al. Differentially private generative adversarial network[EB/OL].(2018-02-19)[2023-02-25]https://arxiv.org/abs/1802.06739.
[6]	Xu C, Ren J, Zhang D, et al. GANobfuscator:Mitigating information leakage under GAN via differential privacy[J]. IEEE Transactions on Information Forensics and Security, 2019, 14(9):2358-2371.
[7]	Song T, Li R, Mei B, et al. A privacy preserving communication protocol for IoT applications in smart homes[J]. IEEE Internet of Things Journal, 2017, 4(6):1844-1852.
[8]	Loukil F, Ghedira-Guegan C, Boukadi K, et al. Data privacy based on IoT device behavior control using blockchain[J]. ACM Transactions on Internet Technology, 2021, 21(1):1-20.
[9]	Jing Y, Yang Y, Feng Z, et al. Neural style transfer:A review[J]. IEEE Transactions on Visualization and Computer Graphics, 2019, 26(11):3365-3385.
[10]	Liu M Y, Huang X, Yu J, et al. Generative adversarial networks for image and video synthesis:Algorithms and applications[J]. Proceedings of the IEEE, 2021, 109(5):839-862.
[11]	Isola P, Zhu J Y, Zhou T, et al. Image-to-image translationwith conditional adversarial networks[C]. Holunono: IEEE Conference on Computer Vision and Pattern Recognition,2017:1125-1134.
[12]	Zhu J Y, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]. Venice: IEEE International Conference on Computer Vision,2017:2223-2232.
[13]	Choi Y, Choi M, Kim M, et al. StarGAN: Unified generative adversarial networks for multidomain image-to-image translation[C]. Salt Lake City: IEEE Conference on Computer Vision and Pattern Recognition,2018:8789-8797.
[14]	赵晋, 李菲菲. 一种基于GAN的轻量级水墨画风格迁移模型[J]. 电子科技, 2023, 36(2):81-86.
	Zhao Jin, Li Feifei. A GAN-based lightweight style transfer model for ink painting[J]. Electronic Science and Technology, 2023, 36(2):81-86.
[15]	Chen J, Konrad J, Ishwar P. VGAN-based image representation learning for privacy-preserving facial expression recognition[C]. Salt Lake City: IEEE Conference on Computer Vision and Pattern Recognition Workshops,2018:1570-1579.
[16]	Liu Z, Zhao Z. First steps in pixel privacy:Exploring deep learning-based image enhancement against larges-cale image inference[C]. Valbonne: Media Eval Workshop,2018:1-3.
[17]	杨观赐, 林家丞, 李杨, 等. 基于改进Cycle-GAN的机器人视觉隐私保护方法[J]. 华中科技大学学报(自然科学版), 2020, 48(2):73-78.
	Yang Guanci, Lin Jiacheng, Li Yang, et al. Privacy protection method of social robot vision base on improved Cycle-GAN[J]. Journal of Huazhong University of Science and Technology(Natural Science Edition), 2020, 48(2):73-78.
[18]	Schroff F, Kalenichenko D, Philbin J. Facenet:A unified embedding for face recognition and clustering[C]. Boston: IEEE Conference on Computer Vision and Pattern Recognition,2015:815-823.
[19]	Den G J, Guo J, Xue N, et al. Arcface:Additive angular margin loss for deep face recognition[C]. Long Beach: IEEE Conference on Computer Vision and Pattern Recognition,2019:4690-4699.
[20]	Deb D, Zhang J, Jain A K. Advfaces:Adversarial face synthesis[C]. Houston: IEEE International Joint Conferenceon Biometrics,2020:1-10.
[21]	Huang G B, Mattar M, Berg T, et al. Labeled faces in thewild:A database for studying face recognition in unconstrained environments[C]. Marseille:Workshop on Facesin Real-Life Images: Detection,Alignment and Recognition,2008:1-15.
[22]	Liu Z, Luo P, Wang X, et al. Deep learning face attributes in the wild[C]. Santiago: IEEE International Conference on Computer Vision,2015:3730-3738.
[23]	Zhang K, Zhang Z, Li Z, et al. Joint face detection and alignment using multitask cascaded convolutional networks[J]. IEEE Signal Processing Letters, 2016, 23(10):1499-1503.
[24]	Kim J, Kim M, Kang H, et al. U-GET-IT:Unsupervised generative attentional networks with adaptive layer-instance normalization for image-to-image translation[C]. International Conference on Learning Representations,2021:1-18.
[25]	Gwern Branwen. Making anime faces with styleGAN[EB/OL].(2021-01-30)[2022-03-17].

人脸识别模型	FaceNet	ArcFace
原始CycleGAN	96.1	98.8
身份混淆CycleGAN	19.3	60.9