西安电子科技大学学报 ›› 2024, Vol. 51 ›› Issue (5): 149-164.doi: 10.19665/j.issn1001-2400.20240403
收稿日期:
2024-03-05
出版日期:
2024-05-07
发布日期:
2024-05-07
作者简介:
王小鹏(1969—),男,教授,E-mail:wangxp1969@sina.com;基金资助:
Received:
2024-03-05
Online:
2024-05-07
Published:
2024-05-07
摘要:
针对传统跌倒检测算法在特征提取不充分、跌倒判决方法单一以及实时性不强的问题,提出一种改进型YOLOv8结合人体骨骼关键点的跌倒检测算法。首先,算法通过ShuffleNetV2网络替换原有YOLOv8的Darknet-53主干网络,在颈部增加混合注意力机制(Shuffle Attention,SA),使得网络能够更好地提取人体的行为特征,实现人体静态姿势匹配。其次,分析人体动态行为的骨骼关键点位置变化信息,将人体质心下降速度、人体躯干与地面间的夹角变化速度和人体的高宽比三者共同作为跌倒行为的判决依据,提高跌倒判决的准确率。实验结果表明,该算法在COCO Key Points数据集上的检测精度、F1值和mAP50值分别为78.3%、67.9%和70.0%,在UR Fall Detection、Fall Detection Datasets和自建数据集上的检测准确率分别为95.85%、92.8%和96.52%,在区分日常生活行为和跌倒行为方面优于传统算法。
中图分类号:
王小鹏, 石欢. 改进型YOLOv8融合关键点的跌倒检测算法[J]. 西安电子科技大学学报, 2024, 51(5): 149-164.
WANG Xiaopeng, SHI Huan. Fall detection algorithm based on the improved YOLOv8 combined with key points[J]. Journal of Xidian University, 2024, 51(5): 149-164.
表3
不同数据集上的消融实验结果 %"
数据集 | YOLOv8n | ShffleNetV2 | SA | AP | mAP | ||||
---|---|---|---|---|---|---|---|---|---|
跌倒 | 躺 | 蹲下 | 坐 | 走 | |||||
URFD | √ | 96.1 | 89.4 | 98.3 | 99.0 | 95.70 | |||
URFD | √ | √ | 98.0 | 90.4 | 96.7 | 92.7 | 94.40 | ||
URFD | √ | √ | 93.0 | 81.8 | 99.1 | 98.7 | 93.20 | ||
URFD | √ | √ | √ | 95.4 | 90.1 | 99.1 | 98.8 | 95.85 | |
FDD | √ | 99.5 | 98.5 | 68.9 | 95.3 | 99.3 | 92.30 | ||
FDD | √ | √ | 99.1 | 99.1 | 79.3 | 96.4 | 99.0 | 94.58 | |
FDD | √ | √ | 99.4 | 99.4 | 63.5 | 97.2 | 99.0 | 91.70 | |
FDD | √ | √ | √ | 98.9 | 99.7 | 73.2 | 93.1 | 99.1 | 92.80 |
自建数据集 | √ | 96.2 | 98.3 | 88.6 | 97.8 | 98.9 | 95.96 | ||
自建数据集 | √ | √ | 98.4 | 97.9 | 89.9 | 97.2 | 98.2 | 96.32 | |
自建数据集 | √ | √ | 91.3 | 98.4 | 93.4 | 97.6 | 99.2 | 95.98 | |
自建数据集 | √ | √ | √ | 99.0 | 95.8 | 92.7 | 96.3 | 98.8 | 96.52 |
表4
COCO Key points 2017数据集上的检测性能对比结果"
方法 | G. | Param./M | Box | Pose | ||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
P | R | F1-S. | mAP | P | R | F1-S. | mAP | |||||
50 | 50~95 | 50 | 50~95 | |||||||||
v8n. | 9.3 | 3.3 | 0.782 | 0.604 | 0.681 | 0.692 | 0.476 | 0.701 | 0.396 | 0.506 | 0.409 | 0.229 |
v8s. | 30.4 | 11.6 | 0.704 | 0.577 | 0.634 | 0.638 | 0.391 | 0.583 | 0.333 | 0.424 | 0.314 | 0.128 |
v7w. | 101.0 | 80.2 | 0.839 | 0.571 | 0.679 | 0.624 | 0.465 | |||||
v5n. | 11.4 | 15.1 | 0.759 | 0.568 | 0.649 | 0.665 | 0.421 | 0.637 | 0.341 | 0.444 | 0.333 | 0.144 |
v5s. | 25.8 | 9.6 | 0.776 | 0.620 | 0.689 | 0.715 | 0.470 | 0.660 | 0.373 | 0.477 | 0.377 | 0.179 |
LOP | 27.2 | 4.1 | 0.661 | 0.697 | 0.679 | 0.662 | 0.492 | 0.405 | 0.359 | 0.451 | 0.312 | 0.179 |
OPP | 7.3 | 4.1 | 0.723 | 0.594 | 0.652 | 0.621 | 0.413 | 0.534 | 0.425 | 0.473 | 0.354 | 0.221 |
文中算法 | 7.7 | 2.8 | 0.783 | 0.600 | 0.679 | 0.700 | 0.457 | 0.699 | 0.385 | 0.496 | 0.386 | 0.201 |
表6
FDD数据集上的检测准确率对比结果 %"
算法 | 主干网络 | Accuracy | mAccuracy | ||||
---|---|---|---|---|---|---|---|
跌倒 | 躺 | 蹲下 | 坐 | 走 | |||
YOLOv8n | DarkNet-53 | 99.5 | 98.5 | 68.9 | 95.3 | 99.3 | 92.30 |
YOLOv5s | CSP DarkNet-53 | 85.9 | 91.1 | 80.8 | 92.4 | 99.5 | 89.94 |
YOLOv5s+SE[ | CSP DarkNet-53 | 87.6 | 89.0 | 61.7 | 90.6 | 97.7 | 85.32 |
LOP[ | MobileNetV3 | 89.0 | 75.4 | 96.8 | 87.07 | ||
文中算法 | ShuffleNetV2 | 98.9 | 99.7 | 73.2 | 93.1 | 99.1 | 92.80 |
表7
8种算法在识别人体行为时的推理速度的比较结果"
算法 | 帧率/(帧·s-1) | 运行时间/s | 延迟/ms | 吞吐量 |
---|---|---|---|---|
YOLOv8n | 3.7 | 286.754 | 235.28 | 3.73 |
YOLOv8s | 3.1 | 346.706 | 291.31 | 3.09 |
YOLOv8n-pose | 8.1 | 132.008 | 69.62 | 8.11 |
YOLOv8s-pose | 9.0 | 118.659 | 64.78 | 9.04 |
LOP[ | 27.1 | 38.881 | 25.61 | 25.67 |
YOLOv7-w6-pose[ | 11.3 | 94.589 | 59.83 | 14.89 |
Motion history[ | 6.9 | 172.833 | 213.51 | 6.75 |
文中算法 | 16.8 | 63.076 | 33.34 | 16.88 |
[1] | SHAHZAD A, KIM K. FallDroid:An Automated Smart-Phone-Based Fall Detection System Using Multiple Kernel Learning[J]. IEEE Transactions on Industrial Informatics, 2019, 15(1):35-44. |
[2] | CHEN J X, WANG W, FANG B, et al. Digital Twin Empowered Wireless Healthcare Monitoring for Smart Home[J]. IEEE Journal on Selected Areas in Communications, 2023, 41(11):3662-3676. |
[3] | ZHOU X, QIAN L C, YOU P J, et al. Fall Detection Using Convolutional Neural Network with Multi-Sensor Fusion[C]//2018 IEEE International Conference on Multimedia & Expo Workshops(ICMEW). Piscataway:IEEE, 2018:1-5. |
[4] | LUO B. Human Fall Detection for Smart Home Caring Using Yolo Networks[J]. International Journal of Advanced Computer Science and Applications, 2023, 14(4):59-68. |
[5] | SHU Y, HU L. A Vision-Based Human Posture Detection Approach for Smart Home Applications[J]. International Journal of Advanced Computer Science and Applications, 2023, 14(10):209-216. |
[6] | YIN Y, LEI L, LIANG M, et al. Research on Fall Detection Algorithm for the Elderly Living Alone Based on YOLO[C]//2021 IEEE International Conference on Emergency Science and Information Technology(ICESIT). Piscataway:IEEE, 2021:403-408. |
[7] | LOU H, DUAN X, GUO J, et al. DC-YOLOv8:Small-Size Object Detection Algorithm Based on Camera Sensor[J]. Electronics, 2023, 12(10):2323. |
[8] | 张强, 杨欣朋, 赵世祥, 等. 注意力机制的SAR图像车辆目标检测网络[J]. 西安电子科技大学学报, 2023, 50(1):36-47. |
ZHANG Qiang, YANG Xinpeng, ZHAO Shixiang, et al. Vehicle-Target Detection Network for SAR Images Based on the Attention Mechanism[J]. Journal of Xidian University, 2023, 50(1):36-47. | |
[9] | 王娟, 刘子杉, 武明虎, 等. 融合超分辨率重建技术的多尺度目标检测算法[J]. 西安电子科技大学学报, 2023, 50(3):122-131. |
WANG Juan, LIU Zishan, WU Minghu, et al. Multi-Scale Object Detection Algorithm Combined with Super-Resolution Reconstruction Technology[J]. Journal of Xidian University, 2023, 50(3):122-131. | |
[10] | MICHAEL Z, DIETER H, KURT S, et al. Deep Learning-Based 2D Keypoint Detection in Alpine Ski Racing-A Performance Analysis of State-of-the-Art Algorithms Applied to Regular Skiing and Injury Situations[J]. JSAMS Plus, 2023, 2:100034. |
[11] | WANG B H, YU J, WANG K, et al. Fall Detection Based on Dual-Channel Feature Integration[J]. IEEE Access, 2020, 8:103443-103453. |
[12] | 刘国帅, 熊平. 基于三个特征点的人体跌倒检测[J]. 科技视界, 2015, 2015(21):5-6. |
LIU Guoshuai, XIONG Ping. Human Fall Detection Based on Three Feature Points[J]. Technology Perspective, 2015, 2015(21):5-6. | |
[13] | ABOBAKR A, HOSSNY M, NAHAVAND S. A Skeleton-Free Fall Detection System from Depth Images Using Random Decision Forest[J]. IEEE Systems Journal, 2017, 12(3):2994-3005. |
[14] | ZHANG X, XIE Q, SUN W, et al. Dense Spatial-Temporal Graph Convolutional Network Based on Lightweight Open Pose for Detecting Falls[J]. Tech Science Press, 2023, 77(1):47-61. |
[15] | 王鑫, 郑晓岩, 高焕兵, 等. 基于卷积神经网络和多判别特征的跌倒检测算法[J]. 计算机辅助设计与图形学学报, 2023, 35(3):452-462. |
WANG Xin, ZHENG Xiaoyan, GAO Huanbing, et al. A Fall Detection Algorithm Based on Convolutional Neural Network and Multi-Discriminant Feature[J]. Journal of Computer Aided Design and Graphics, 2023, 35(3):452-462. | |
[16] | CAO Z, SIMON T, WEI S E, et al. Realtime Multi-Person 2d Pose Estimation Using Part Affinity Fields[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2017:7291-7299. |
[17] | ROUGIER C, MEUNIER J, ST-AMAUD A, et al. Fall Detection from Human Shape and Motion History Using Video Surveillance[C]// 21st International Conference on Advanced Information Networking and Applications Workshops(AINAW 07).Piscataway:IEEE, 2007:875-880. |
[18] | MA N, ZHANG X, ZHENG H T, et al. Shufflenetv2:PracticalGuidelines for Efficient CNN Architecture Design[C]//Proceedings of the European Conference on Computer Vision(ECCV). Heidelberg:Springer, 2018:116-131. |
[19] | ZHANG Q L, YANG Y B. Sa-Net:Shuffle Attention for Deep Convolutional Neural Networks[C]// ICASSP 2021-2021 IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP).Piscataway:IEEE, 2021:2235-2239. |
[20] | KWOLEK B, KEPSKI M. HumanFall Detection on Embedded Platform Using Depth Maps and Wireless Accelerometer[J]. Computer Methods and Programs in Biomedicine, 2014, 117(3):489-501. |
[21] | ADHIKARI K, BOUCHACHIA H, NAIT-CHARIF H. Activity Recognition for Indoor Fall Detection Using Convolutional Neural Network[C] //2017 Fifteenth IAPR International Conference on Machine Vision Applications(MVA). Piscataway:IEEE, 2017:81-84. |
[22] | YANG J, WANG Y, ZHAO H, et al. MobileNet and Knowledge Distillation-Based Automatic Scenario Recognition Method in Vehicle-to-Vehicle Systems[J]. IEEE Transactions on Vehicular Technology, 2022, 71(10):11006-11016. |
[23] | AKAY M, DU Y, SERSHEN C L, et al. Deep Learning Classification of Systemic Sclerosis Skin Using the MobileNetV2 Model[J]. IEEE Open Journal of Engineering in Medicine and Biology, 2021, 2(3):104-110. |
[24] | WU E Q, TANG Z R, XIONG P, et al. ROpenPose:A Rapider Open Pose Model for Astronaut Operation Attitude Detection[J]. IEEE Transactions on Industrial Electronics, 2021, 69(1):1043-1052. |
[25] | ZHANG X, ZHOU X, LIN M, et al. Shufflenet:An Extremely Efficient Convolutional Neural Network for Mobile Devices[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2018:6848-6856. |
[26] | WOO S, PARK J, LEE J Y, et al. CBAM:Convolutional Block Attention Module[C]//Proceedings of the European Conference on Computer Vision(ECCV). Heidelberg:Springer, 2018:3-19. |
[27] | WANG Q L, WU B G, ZHU P F, et al. ECA-Net:Efficient Channel Attention for Deep Convolutional Neural Networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2020:11534-11542. |
[28] | HU J, SHEN L, SUN G. Squeeze-and-Excitation Networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2018:7132-7141. |
[29] | MAJI D, NAGORI S, MATHEW M, et al. Yolo-Pose:Enhancing Yolo for Multi Person Pose Estimation Using Object Keypoint Similarity Loss[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2022:2637-2646. |
[1] | 管业鹏, 苏光耀, 盛怡. 双向长短期记忆网络的时间序列预测方法[J]. 西安电子科技大学学报, 2024, 51(3): 103-112. |
[2] | 张欣雨, 梁煜, 张为. 融合全局和局部信息的实时烟雾分割算法[J]. 西安电子科技大学学报, 2024, 51(1): 147-156. |
[3] | 钟浩, 边山, 王春桃. 结合自注意力与卷积的真实场景图像篡改定位[J]. 西安电子科技大学学报, 2024, 51(1): 135-146. |
[4] | 王柯俨,成吉聪,黄诗芮,蔡坤伦,王威然,李云松. 融合上下文感知注意力的低光图像去雾网络[J]. 西安电子科技大学学报, 2023, 50(2): 23-32. |
[5] | 秦瑞,张为. 一种无锚框结构的多尺度火灾检测算法[J]. 西安电子科技大学学报, 2022, 49(6): 111-119. |
[6] | 刘侍刚,张同,杨建功,葛宝. 递进式空洞残差深度双目立体匹配网络[J]. 西安电子科技大学学报, 2022, 49(5): 175-180. |
[7] | 王朝宇,郭继昌,王天保,郑司达,张怡. 融合显著性信息的水下图像清晰化算法[J]. 西安电子科技大学学报, 2022, 49(3): 137-146. |
[8] | 孙雅媚,肖嵩,曲家慧,董文倩. 自适应上下文感知相关滤波类目标跟踪算法[J]. 西安电子科技大学学报, 2022, 49(3): 21-27. |
[9] | 孙景荣,谢林昌,杜梦欣,罗丽燕. 一种非线性变换的自适应透射率去雾算法[J]. 西安电子科技大学学报, 2022, 49(1): 208-215. |
[10] | 顾兆军,陈辉,王家亮,高冰. 一种小型四轴飞行器目标跟踪控制算法[J]. 西安电子科技大学学报, 2021, 48(5): 117-127. |
[11] | 董如婵,焦李成,赵进,沈维燕. 一种深度融合机制的遥感图像目标检测技术[J]. 西安电子科技大学学报, 2021, 48(5): 128-138. |
[12] | 王军军,孙岳,李颖. 一种生成对抗网络的遥感图像去云方法[J]. 西安电子科技大学学报, 2021, 48(5): 23-29. |
[13] | 许缓缓,李洪梅,李富余,孙学梅. 一种时空卷积的步态识别方法[J]. 西安电子科技大学学报, 2021, 48(4): 144-150. |
[14] | 崔艳鹏,王元皓,胡建伟. 一种改进YOLOv3的动态小目标检测方法[J]. 西安电子科技大学学报, 2020, 47(3): 1-7. |
[15] | 黄樟钦,母召,岑陈,高寒. 一种多路视频的360度实时拼接技术[J]. 西安电子科技大学学报, 2020, 47(2): 1-8. |
|