融合目标速度变化机制的视频摘要生成模型

doi:10.16180/j.cnki.issn1007-7820.2022.09.005

电子科技 ›› 2022, Vol. 35 ›› Issue (9): 30-36.doi: 10.16180/j.cnki.issn1007-7820.2022.09.005

融合目标速度变化机制的视频摘要生成模型

牛嘉丰,石蕴玉,刘翔,李任斯

上海工程技术大学电子电气工程学院,上海 201620

收稿日期:2021-03-22 出版日期:2022-09-15 发布日期:2022-09-15
作者简介:牛嘉丰(1993-),男,硕士研究生。研究方向:图像处理。|石蕴玉(1982-),女,博士,讲师。研究方向:视频处理分析及质量评价。|刘翔(1972-),男,博士,副教授。研究方向:计算机视觉与人工生命。
基金资助:
国家自然科学基金(81101105);上海市自然科学基金(19ZR1421500);文化部科技创新项目(2015KJCXXM19)

A Video Synopsis Generation Model Incorporating Object Speed Change Mechanism

NIU Jiafeng,SHI Yunyu,LIU Xiang,LI Rensi

College of Electronic and Electrical Engineering,Shanghai University of Engineering Science,Shanghai 201620,China

Received:2021-03-22 Online:2022-09-15 Published:2022-09-15
Supported by:
National Natural Science Foundation of China(81101105);Natural Science Foundation of Shanghai(19ZR1421500);Science and Technology Innovation Project of the Ministry of Culture(2015KJCXXM19)

摘要/Abstract

摘要：

视频摘要是监控视频浏览和存储的有效技术。在压缩比的限制下,现有的视频摘要生成方法容易导致重排目标碰撞和时序错乱。针对该问题,文中提出一种融合目标速度变化机制的视频摘要生成模型。在目标重排的能量函数中,除目标起始位置变量之外还加入了目标速度变量,从而能够同时改变目标的起始位置和速度,避免碰撞和时序错乱问题。随后使用马尔科夫链蒙特卡罗随机采样算法求解能量函数的最优值,得到了目标重排方案的最优解。实验表明,在压缩率相同的情况下,相较于其他方法,该模型生成的摘要视频中的目标碰撞和时序错乱的问题较少。

关键词: 视频摘要, 目标速度, 蒙特卡罗采样, 目标碰撞, 目标重排, 监控视频, 能量函数, 泊松编辑

Abstract:

Video synopsis is an effective technique for surveillance video browsing and storage. Existing video synopsis generation methods are prone to cause rearrangement objects collision and chronological disorder under the limitation of compression ratio. To solve this problem, a video synopsis generation model incorporating object speed change mechanism is proposed in this study. In the energy function of object rearrangement, object speed variable is added in addition to object start position variable, so that object start position and speed can be changed simultaneously to avoid collision and chronological disorder problems. Then the Markov chain Monte Carlo random sampling algorithm is used to solve the optimal value of the energy function, and the optimal solution of the target rearrangement scheme is obtained. Experiments show that when the compression rate is the same, compared with other methods, the summary video generated by the proposed model has fewer problems with target collision and timing disorder.

Key words: video synopsis, object speed, Monte Carlo sampling, object collision, object rearrangement, surveillance video, energy function, Poisson editing

中图分类号:

TP391

牛嘉丰,石蕴玉,刘翔,李任斯. 融合目标速度变化机制的视频摘要生成模型[J]. 电子科技, 2022, 35(9): 30-36.

NIU Jiafeng,SHI Yunyu,LIU Xiang,LI Rensi. A Video Synopsis Generation Model Incorporating Object Speed Change Mechanism[J]. Electronic Science and Technology, 2022, 35(9): 30-36.

图/表 5

图1

表1

图2

表2

图3

参考文献 19

[1]	李鹏程. 基于张量紧凑表示的视频压缩算法[J]. 电子科技, 2017, 30(5):1-4.
	Li Pengcheng. Video compression algorithm based on tensor compact representation[J]. Electronic Science and Technology, 2017, 30(5):1-4.
[2]	Lu M, Wang Y, Pan G. Generating fluent tubes in video synopsis[C]. Vancouver: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal, 2013.
[3]	Zhong R, Hu R, Wang Z, et al. Fast synopsis for moving objects using compressed video[J]. IEEE Signal Processing Letters, 2014, 21(7):834-838. doi: 10.1109/LSP.2014.2317754
[4]	Nie Y, Xiao C, Sun H, et al. Compact video synopsis via global spatiotemporal optimization[J]. IEEE Transactions on Visualization and Computer Graphics, 2013, 19(10):1664-1676. doi: 10.1109/TVCG.2012.176
[5]	He Y, Gao C, Sang N, et al. Graph coloring based surveillance video synopsis[J]. Neurocomputing, 2017, 225(15): 64-79. doi: 10.1016/j.neucom.2016.11.011
[6]	Ruan T, Wei S, Li J, et al. Rearranging online tubes for streaming video synopsis: a dynamic graph coloring approach[J]. IEEE Transactions on Image Processing, 2019, 28(8):3873-3884. doi: 10.1109/TIP.2019.2903322 pmid: 30869618
[7]	Moussa M M, Shoitan R. Object-based video synopsis approach using particle swarm optimization[C]. London: Proceedings of the Signal, Image and Video, 2020.
[8]	Namitha K, Narayanan A. Preserving interactions among moving objects in surveillance video synopsis[J]. Multimedia Tools and Applications, 2020, 79(43):32331-32360. doi: 10.1007/s11042-020-09493-2
[9]	Li X L, Wang Z G, Lu X Q. Surveillance video synopsis via scaling down objects[J]. IEEE Transactions on Image Processing, 2015, 25(2):740-755. doi: 10.1109/TIP.2015.2507942
[10]	Nie Y W, Li Z K, Zhang Z S, et al. Collision-free video synopsis incorporating object speed and size changes[J]. IEEE Transactions on Image Processing, 2020, 29(24):1465-1478. doi: 10.1109/TIP.2019.2942543
[11]	Zivkovic Z, Van Der Heijden F. Efficient adaptive density estimation per image pixel for the task of background subtraction[J]. Pattern Recognition Letters, 2006, 27(7):773-780. doi: 10.1016/j.patrec.2005.11.005
[12]	Wang Q, Zhang L, Bertinetto L, et al. Fast online object tracking and segmentation: A unifying approach[C]. Long Beach: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019.
[13]	曾照, 吴薇, 汪欣. 改进的核相关滤波跟踪算法[J]. 电子科技, 2020, 33(3):50-55.
	Zeng Zhao, Wu Wei, Wang Xin. Improved kernelized correlation filter tracking[J]. Electronic Science and Technology, 2020, 33(3):50-55.
[14]	Gilks W R, Richardson S, Spiegelhalter D. Markov chain monte carlo in practice[J]. Technometrics, 1997, 39(3):338-339.
[15]	Metropolis N, Rosenbluth A W, Rosenbluth M N, et al. Equation of state calculations by fast computing machines[J]. The Journal of Chemical Physics, 1953, 21(6):1087-1092. doi: 10.1063/1.1699114
[16]	Hastings W K. Monte carlo sampling methods using Markov chains and their applications[J]. Biometrika, 1970, 57(1):97-109. doi: 10.1093/biomet/57.1.97
[17]	P'erez P, Gangnet M, Blake A. Poisson image editing[J]. ACM Tranactions on Graphics, 2003, 22(3):313-318.
[18]	Corona K, Osterdahl K, Collins R, et al. MEVA: A large-scale multiview, multimodal video dataset for activity detection[C]. Virtual: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2021.
[19]	周俊, 袁佳明, 万伟江. 基于加权时变泊松模型的电力信用风险判别及预警系统设计[J]. 电子设计工程, 2021, 29(11):21-25.
	Zhou Jun, Yuan Jiaming, Wan Weijiang. Power credit risk discrimination and early warning system design based on weighted time-varying Poisson model[J]. Electronic Design Engineering, 2021, 29(11):21-25.

编号	视频时长 /min	视频帧率 /frame·s^-1	目标个数	分辨率
视频1	3	24	21	1 280×720
视频2	5	25	47	672×376
视频3	5	30	18	1 920 ×1 072

测试视频编号	压缩率			碰撞数			时序错乱数
测试视频编号	本文方法	文献 [4]	文献 [8]	本文方法	文献 [4]	文献 [8]	本文方法	文献 [4]	文献 [8]
视频1	0.21	0.21	0.21	0	0	3	0	5	1
视频2	0.34	0.34	0.34	0	3	4	0	4	3
视频3	0.08	0.08	0.08	1	0	4	1	4	2

融合目标速度变化机制的视频摘要生成模型

A Video Synopsis Generation Model Incorporating Object Speed Change Mechanism

RichHTML

PDF (PC)

赞

可视化

摘要/Abstract

引用本文

使用本文

图/表 5

参考文献 19

相关文章 15

Metrics

本文评价

推荐阅读 0

[1]	赵云飞, 薛存金. 一种融合空洞卷积与池化模型的遥感影像水体提取方法[J]. 电子科技, 2025, 38(3): 40-46.
[2]	陈宇洋, 李峰. 融合CNN与Transformer的视网膜OCT图像积液分割方法[J]. 电子科技, 2025, 38(3): 47-59.
[3]	孙佳倩, 朱金荣, 张小宝, 张云恺, 龚卫娟. 基于集成学习的虹膜分割算法[J]. 电子科技, 2025, 38(3): 88-94.
[4]	郑方亮, 王延年, 廉继红, 阮佩. 基于条件先验Swin Transformer的人脸图像超分辨重建[J]. 电子科技, 2025, 38(2): 35-41.
[5]	潘天艺, 宋燕. 基于信任关系的非线性表征潜在因子模型[J]. 电子科技, 2025, 38(2): 53-61.
[6]	马壮, 甘开宇, 尹钟. 基于脑电信号和周围生理信号的多模态融合情感识别[J]. 电子科技, 2025, 38(2): 62-69.
[7]	赖颖, 巨志勇, 叶雨新. 基于改进YOLOv4的车辆检测算法[J]. 电子科技, 2025, 38(1): 81-87.
[8]	李云鹏, 席志红. 基于RetinaFace与FaceNet的动态人脸识别系统设计[J]. 电子科技, 2024, 37(12): 79-86.
[9]	莫卓锐, 黄强豪, 张琳, 曹雨齐, 葛维挺, 余明晖. 结直肠癌免疫组化图像分级诊断方法[J]. 电子科技, 2024, 37(12): 24-31.
[10]	崔晶楠, 黄春艳, 李艳玲. 改进YOLOv5s算法的钢材表面缺陷检测[J]. 电子科技, 2024, 37(12): 48-55.
[11]	甘开宇, 尹钟. 基于脑电信号和瞬时情感强度标签的情感识别方法[J]. 电子科技, 2024, 37(11): 78-84.
[12]	夏强强, 李菲菲. 基于Co-Teaching的噪声标签深度学习[J]. 电子科技, 2024, 37(11): 1-6.
[13]	程锦科, 李高磊. 支持隐私保护的智能物联网数据风格转换[J]. 电子科技, 2024, 37(10): 1-5.
[14]	鞠萍, 宋岩, 张英杰, 徐一夫, 邵杭. 基于掩码自编码的农作物病虫害分类方法[J]. 电子科技, 2024, 37(10): 23-29.
[15]	冯婷, 应捷, 杨海马, 李芳. 基于多尺度特征与注意力机制的宫颈病变检测[J]. 电子科技, 2024, 37(10): 30-39.