乒乓球机器人核心算法研究综述

doi:10.16180/j.cnki.issn1007-7820.2023.12.003

摘要/Abstract

摘要：

作为一个集机器视觉与运动控制于一体的综合性研究平台,乒乓球机器人研究一直备受关注,其机械机构、视觉算法和运动控制的复杂性以及高实时性是阻碍乒乓球机器人发展的重要因素。为了提高乒乓球机器人的竞技性,需要解决乒乓球旋转测量、运动轨迹精确预测及击球策略选择等难题,这也是近几年乒乓球机器人的研究重点。基于近年来此领域的研究进展,文中对乒乓球机器人的视觉核心算法、回球核心算法等研究成果进行着重介绍,分析不同算法的优缺点并对未来研究趋势进行展望,为乒乓球机器人的研究与设计提供了参考依据。

关键词: 乒乓球机器人, 轨迹预测, 击球策略, 深度学习, 强化学习, 旋转测量, 视觉算法, 碰撞模型

Abstract:

As a comprehensive research platform integrating machine vision and motion control, the research of table tennis robot has always attracted much attention. The complexity and high real-time of its mechanical mechanism, visual algorithm and motion control are important factors hindering the development of table tennis robot. In order to improve the competitive ability of table tennis robot, it is necessary to solve the problems of table tennis rotation measurement, accurate prediction of movement trajectory and selection of hitting strategy, which is also the focus of the research of table tennis robot in recent years. Based on the research progress in this field in recent years, this study focuses on the research results of table tennis robot vision core algorithm and ball return core algorithm, analyzes the advantages and disadvantages of different algorithms and prospects the future research trend, so as to provide a reference for the research and design of table tennis robot.

Key words: table tennis robot, trajectory prediction, hitting strategy, deep learning, reinforcement learning, rotation measurement, visual algorithm, collision model

中图分类号:

TP18

索芳菲,季云峰. 乒乓球机器人核心算法研究综述[J]. 电子科技, 2023, 36(12): 16-24.

SUO Fangfei,JI Yunfeng. Review of the Research on the Core Algorithm of Table Tennis Robot[J]. Electronic Science and Technology, 2023, 36(12): 16-24.

图/表 5

图1

表1

表2

表3

表4

参考文献 44

[1]	Lewis F L, Ge S S. Autonomous mobile robots:Sensing,control,decision making and application[M]. Boca Raton: CRC Press, 2018:1-2.
[2]	章逸丰. 快速飞行物体的状态估计和轨迹预测[D]. 杭州: 浙江大学, 2015:1-7.
	Zhang Yifeng. State estimation and trajectory prediction of fast flying object[D]. Hangzhou: Zhejiang University, 2015:1-7.
[3]	Liu C F. On a modified method of measuring the rotational speed of a table tennis ball[C]. Taipei: Proceedings of SICE Annual Conference, 2010:1567-1572.
[4]	季云峰, 陆爱发, 任杰, 等. 基于乒乓球机器人视觉系统的单色乒乓球旋转三维速度测定[J]. 上海体育学院学报, 2017, 41(3):83-88.
	Ji Yunfeng, Lu Aifa, Ren Jie, et al. 3D velocity measurement of single-color table tennis rotation based on visual system of table tennis robot[J]. Journal of Shanghai University of Sport, 2017, 41(3):83-88.
[5]	季云峰, 施之皓, 任杰, 等. 基于单目视觉伺服系统的高速旋转球体三维速度测定[J]. 中国体育科技, 2017, 53(2):139-145.
	Ji Yunfeng, Shi Zhihao, Ren Jie, et al. 3D velocity meas-urement of high-speed rotating sphere based on the monocular vision servo system[J]. China Sport Scienceand Technology, 2017, 53(2):139-145.
[6]	Nonomura J, Nakashima A, Hayakawa Y. Analysis of effects of rebounds and aerodynamics for trajectory of table tennis ball[C]. Taipei: Proceedings of SICE Annual Conference, 2010:1567-1572.
[7]	Tamaki T, Wang H, Raytchev B, et al. Estimating the spin of a table tennis ball using inverse compositional image alignment[C]. Kyoto: IEEE International Conference on Acoustics,Speech and Signal Processing, 2012:1457-1460.
[8]	张传伟, 王京梅, 林晓明, 等. 基于背景差分的一种运动目标检测方法[J]. 电子科技, 2015, 28(10):69-71.
	Zhang Chuanwei, Wang Jingmei, Lin Xiaoming, et al. A moving objects detection method based on background subtraction[J]. Electronic Science and Technology, 2015, 28(10):69-71.
[9]	官洪运, 苏振涛, 汪晨. 基于特征融合的背景差分改进算法[J]. 电子科技, 2020, 33(12):22-27.
	Guan Hongyun, Su Zhentao, Wang Chen. Improved bac-kground subtraction based on feature fusion[J]. Electronic Science and Technology, 2020, 33(12):22-27.
[10]	Chen X, Huang Q, Zhang W, et al. Ping-pong trajectory perception and prediction by a PC based high speed four-camera vision system[C]. Taipei: The Ninth World Congress on Intelligent Control and Automation, 2011:1087-1092.
[11]	Su H, Fang Z, Xu D, et al. Trajectory prediction of spin-ning ball based on fuzzy filtering and local modeling for robotic ping-pong player[J]. IEEE Transactions on Instrumentation and Measurement, 2013, 62(11):2890-2900. doi: 10.1109/TIM.2013.2263672
[12]	Huang Y, Xu D, Tan M, et al. Trajectory prediction of s-pinning ball for ping-pong player robot[C]. San Franc-isco: IEEE/RSJ International Conference on Intelligent Robots and Systems, 2011:3434-3439.
[13]	Zhao Y, Zhang Y, Xiong R, et al. Optimal state estimation of spinning ping-pong ball using continuous motion model[J]. IEEE Transactions on Instrumentation and Measurement, 2015, 64(8):2208-2216. doi: 10.1109/TIM.2014.2386951
[14]	季云峰. 基于遗传算法的乒乓球轨迹反向推导旋转方法研究[J]. 南京体育学院学报, 2021, 20(2):53-60.
	Ji Yunfeng. Reverse calculation of rotation using trajectory of table tennis based on genetic algorithm[J]. Journal of Nanjing Sports Institute, 2021, 20(2):53-60.
[15]	李伟健. 基于深度学习的乒乓球目标检测与旋转球轨迹预测[D]. 上海: 东华大学, 2021:22-48.
	Li Weijian. Table tennis target detection and rotating ball trajectory prediction based on deep learning[D]. Shanghai: Donghua University, 2021:22-48.
[16]	Zhang Z, Xu D, Tan M. Visual measurement and prediction of ball trajectory for table tennis robot[J]. IEEE Transactions on Instrumentation and Measurement, 2010, 59(12):3195-3205. doi: 10.1109/TIM.2010.2047128
[17]	Zhang Y, Wei W, Yu D, et al. A tracking and predicting scheme for ping pong robot[J]. Journal of Zhejiang University, 2011, 12(2):110-115.
[18]	Mülling K, Kober J, Peters J. A biomimetic approach torobot table tennis[J]. Adaptive Behavior, 2011, 19(5):359-376. doi: 10.1177/1059712311419378
[19]	胡啸川, 李泽森, 杨双, 等. 三目组合视觉系统在乒乓球机器人上的研究与实现[J]. 机器人技术与应用, 2020, 26(5):39-43.
	Hu Xiaochuan, Li Zesen, Yang Shuang, et al. Research and implementation of trinocular combined vision system on table tennis robot[J]. Robot Technique and Application, 2020, 26(5):39-43.
[20]	季云峰, 任杰, 施之皓. 乒乓球机器人视觉系统的实时跟踪[J]. 上海体育学院学报, 2020, 44(6):70-75.
	Ji Yunfeng, Ren Jie, Shi Zhihao. Real-time tracking of table tennis robot’s vision system[J]. Journal of Shanghai University of Sport, 2020, 44(6):70-75.
[21]	张远辉. 基于实时视觉的乒乓球机器人标定和轨迹跟踪技术研究[D]. 杭州: 浙江大学, 2009:81-112.
	Zhang Yuanhui. Study on real-time vision based calib-ration and trajectory tracking technology of ping pong robot[D]. Hangzhou: Zhejiang University, 2009:81-112.
[22]	Zhang Z, Xu D, Yang P. Rebound model of table tennisball for trajectory prediction[C]. Tianjin:IEEE International Conference on Robotics and Biomimetics, 2010:376-380.
[23]	Nakashima A, Kobayashi Y, Ogawa Y, et al. Modeling of rebound phenomenon between ball and racket rubber with spinning effect[C]. Fukuoka:ICCAS-SICE, 2009:2295-2300.
[24]	Chen X, Tian Y, Huang Q, et al. Dynamic model based ball trajectory prediction for a robot ping-pong player[C]. Tianjin:IEEE International Conference on Roboticsand Biomimetics, 2010:603-608.
[25]	任艳青, 徐德, 谭民. 旋转球与乒乓球台/球拍的反弹模型[J]. 控制理论与应用, 2012, 29(11):1433-1439.
	Ren Yanqing, Xu De, Tan Min. Rebound model between spinning table tennis ball and table/racket[J]. Control Theory & Application, 2012, 29(11):1433-1439.
[26]	赵永生. 旋转飞行乒乓球的状态估计和轨迹预测[D]. 杭州: 浙江大学, 2017:29-56.
	Zhao Yongsheng. State estimation and trajectory prediction of spinning-flying ping-pong ball[D]. Hangzhou: Zhejiang University, 2017:29-56.
[27]	曾鉴彬. 七自由度乒乓球机器人系统的研究与设计[D]. 上海: 东华大学, 2020:15-29.
	Zeng Jianbin. Research and design of table tennis robot system with seven degrees of freedom[D]. Shanghai: Donghua University, 2020:15-29.
[28]	Zhao Y, Xiong R, Zhang Y. Model based motion state estimation and trajectory prediction of spinning ball for ping-pong robots using expectation-maximization algorithm[J]. Journal of Intelligent & Robotic Systems, 2017, 87(3):407-423.
[29]	Lin H I, Huang Y C. Ball trajectory tracking and prediction for a ping-pong robot[C]. Hulun Buir: The Ninth International Conference on Information Science and Technology, 2019:222-227.
[30]	Qiao F F. Application of deep learning in automatic detection of technical and tactical indicators of table tennis[J]. Plos One, 2021, 16(3):5259-5266.
[31]	Gao Y P, Tebbe J, Krismer J, et al. Markerless racket pose detection and stroke classification based on stereo vision for table tennis robots[C]. Naples: The Third IEEE International Conference on Robotic Computing, 2019:189-196.
[32]	谈小峰. 人体检测和姿态识别在乒乓球机器人上的应用研究[D]. 上海: 东华大学, 2021:17-72.
	Tan Xiaofeng. Research on application of human body detection and pose recognition in table tennis robot system[D]. Shanghai: Donghua University, 2021:17-72.
[33]	Huang Y L, Schölkopf B, Peters J. Learning optimal striking points for a ping-pong playing robot[C]. Hamburg: IEEE/RSJ International Conference on Intelligent Robots and Systems, 2015:4587-4592.
[34]	Huang Y L, Büchler D, Koc O, et al. Jointly learning trajectory generation and hitting point prediction in robot table tennis[C]. Cancun: IEEE-RAS the Sixteenth International Conference on Humanoid Robots, 2016:650-655.
[35]	Ji Y F, Hu X Y, Chen Y T, et al. Model-based trajectoryprediction and hitting velocity control for a new table tennis robot[C]. Prague: IEEE/RSJ International Conference on Intelligent Robots and Systems, 2021:2728-2734.
[36]	Liu C F, Hayakawa Y, Nakashima A. Racket control for robot playing table tennis ball[C]. Jeju Island: The Twelfth In-ternational Conference on Control,Automation and Systems, 2012:1427-1432.
[37]	Nakashima A, Nonomura J, Liu C F, et al. Hitting back- spin balls by robotic table tennis system based on physical models of ball motion[J]. IFAC Proceedings Volumes, 2012, 45(22):834-841.
[38]	Huang Q L, Wu J, Xiong R. A solution of inverse kinematics for 7-dof manipulaators and its application[C]. Beijing: The Tenth World Congress on IEEE, 2012:3711-3717.
[39]	孙立书, 张翰康. 乒乓球机器人最优击球规划[J]. 江苏科技大学学报(自然科学版), 2019, 33(5):73-82
	Sun Lishu, Zhang Hankang. Table tennis robot optimal stroke planning[J]. Journal of Jiangsu University of Science and Technology(Natural Science Edition), 2019, 33(5):73-82
[40]	Zhu Y F, Zhao Y S, Jin L S, et al. Towards high level skill learning: Learn to return table tennis ball using monte-carlo based policy gradient method[C]. Kandima: IEEE International Conference on Real-time Computing and Robotics, 2018:34-41.
[41]	Mahjourian R, Miikkulainen R, Lazic N, et al. Hierarchical policy design for sample-efficient learning of robot table tennis through self-play[EB/OL].(2022-01-29) [2022-10-16] https://arxiv.org/abs/1811.12927.
[42]	Tebbe J, Krauch L, Gao Y P, et al. Sample-efficient reinforcement learning in robotic table tennis[C]. Xi'an: IEEE International Conference on Robotics and Automation, 2021:4171-4178.
[43]	金礼森. 基于学习的乒乓球机器人回球决策[D]. 杭州: 浙江大学, 2019:15-31.
	Jin Lisen. Learning to return table tennis ball for robots[D]. Hangzhou: Zhejiang University, 2019:15-31.
[44]	Luo Y, Zhang H B, Zhu X Y, et al. Ball motion control in the table tennis robot system using time-series deep reinforcement learning[J]. IEEE Access, 2021(9):99816-99827.

研究	方法	帧率/ frame·s^-1	估计精度
文献[2]	基于球表面标记直接测量	200	误差小于 1.28 rad·s^-1
文献[13]	通过飞行轨迹反向推导	200	在x方向误差小于5.36 rad·s^-1, 在y方向误差小于2.97 rad·s^-1

研究	方法	性能评价
文献[28]	使用K-means算法将轨迹进行分类	对于旋转来球回球成功率达到71.2%
文献[29]	通过反向传播(BPN)进行球的轨迹预测	非旋转球的轨迹预测偏差约为 37 mm,预测成功率达到88%
文献[30]	提出DCNN-LSTM模型	在仿真实验中对旋转球的轨迹预测成功率达到90%

研究	方法	性能评价
文献[36]	将空气动力学模型简化为线性模型	求解速度30 ms,落点误差小
文献[37]	将决策目标从某一点简化为某一块区域	求解速度1 ms,落点误差较大
文献[38]	对击球平面上每种状态进行求解,并将结果以表格形式保存	求解速度μs级,对固定点的落点误差小

作者	学会定点回球的效率	方法	性能评价
文献[40]	超过20 000 回合	基于蒙特卡洛的强化学习算法(仿真环境下)	误差小于0.05 m, 成功率99.22%
文献[41]	7 000条轨迹数据	通过人类演示训练动力学模型、虚拟现实环境	误差小于0.2 m, 成功率86.7%
文献[42]	200个回合以内	改进的DDPG算法	误差小于0.2 m, 成功率98%
文献[43]	超过70万个回合	改进的DDPG算法 (仿真环境下)	误差小于0.05 m, 成功率100%
文献[44]	50 000回合 (回击旋转球)	改进的DDPG算法、 LSTM网络	旋转球回球成功率高于70%