融合模仿学习的双足机器人全向行走步态生成方法

doi:10.16180/j.cnki.issn1007-7820.2025.01.005

Abstract

Abstract:

Due to the complex high-dimensional dynamics and highly dynamic characteristics of bipedal robots, achieving omnidirectional gait is a difficult problem. In order to achieve omnidirectional walking of bipedal robots, this study proposes a gait training method of biped robot based on deep reinforcement learning. Based on expert experience and the periodicity of bipedal walking, periodic symmetric functions that can achieve different gait styles are designed for imitation learning. In order to make the bipedal robot capable of omnidirectional walking, the footstep planner in ROS (Robot Operating System) is used to generate target foothold points for imitation learning. The proposed method is validated on a self-designed bipedal robot. The experimental results show that the proposed method can realize four gait modes of biped robot including forward, side, diagonal and turn, and realize omnidirectional gait of biped robot, and can realize different styles of cycles.

Key words: reinforcement learning, imitation learning, expert experience, periodic gait, omnidirectional movement, steering movement, diagonal movement, bipedal robot

CLC Number:

TP18

FENG Zhen, MOU Haiming, XUE Jie, LI Qingdu. Omnidirectional Gait Generation Method for Biped Robot with Fusion of Imitation Learning[J].Electronic Science and Technology, 2025, 38(1): 29-36.

Figures/Tables 10

Figure 1.

Table 1.

Figure 2.

Figure 3.

Table 2.

Figure 4.

Figure 5.

Table 3.

Figure 6.

Figure 7.

References 20

[1]	黄艳龙, 徐德, 谭民. 机器人运动轨迹的模仿学习综述[J]. 自动化学报, 2022, 48(2):315-334.
	Huang Yanlong, Xu De, Tan Min. On imitation learning of robot movement trajectories:A survey[J]. ACTA Automatica Sinica, 2022, 48(2):315-334.
[2]	Xing B, Liu Y, Wang Z, et al. Dynamic walking locomotion framework based on impulse force design and ZMP principle[C]. Hefei: Chinese Control and Decision Conference, 2022:1995-2000.
[3]	Song H, Peng W Z, Kim J H. Partition-aware stability control for humanoid robot push recovery with whole-body capturability[J]. Journal of Mechanisms and Robotics, 2023, 16(1):11005-11012.
[4]	Kuindersma S, Permenter F, Tedrake R. An efficiently solvable quadratic program for stabilizing dynamic locomotion[C]. Hong Kong: IEEE International Conference on Robotics and Automation, 2014:2589-2594.
[5]	Dai H, Valenzuela A, Tedrake R. Whole-body motion planning with centroidal dynamics and full kinematics[C]. Madrid: IEEE-RAS International Conference on Humanoid Robots, 2014:295-302.
[6]	张佳鹏, 李琳, 朱叶. 基于强化学习的无人驾驶车辆行为决策方法研究进展[J]. 电子科技, 2021, 34(5):66-71.
	Zhang Jiapeng, Li Lin, Zhu Ye. A review of research on decision-making methods of autonomous vehicle based on reinforcement learning[J]. Electronic Science and Technology, 2021, 34(5):66-71.
[7]	董豪, 杨静, 李少波, 等. 基于深度强化学习的机器人运动控制研究进展[J]. 控制与决策, 2022, 37(2):278-292.
	Dong Hao, Yang Jing, Li Shaobo, et al. Research progress of robot motion control based on deep reinforcement learning[J]. Control and Decision, 2022, 37(2):278-292.
[8]	Peng X B, Berseth G, Yin K, et al. Deeploco:Dynamic locomotion skills using hierarchical deep reinforcement learning[J]. ACM Transactions on Graphics, 2017, 36(4):1-13.
[9]	Liu C, Lonsberry A G, Nandor M J, et al. Implementation of deep deterministic policy gradients for controlling dynamic bipedal walking[J]. Biomimetics, 2019, 4(1):28-30.
[10]	Li Z, Cheng X, Peng X B, et al. Reinforcement learning for robust parameterized locomotion control of bipedal robots[C]. Xi'an: IEEE International Conference on Robotics and Automation, 2021:2811-2817.
[11]	Siekmann J, Godse Y, Fern A, et al. Sim-to-real learning of all common bipedal gaits via periodic reward composition[C]. Xi'an: IEEE International Conference on Robotics and Automation, 2021:7309-7315.
[12]	Rodriguez D, Behnke S. DeepWalk:Omnidirectional bipedal gait by deep reinforcement learning[C]. Xi'an: IEEE International Conference on Robotics and Automation, 2021:3033-3039.
[13]	Peng X B, Coumans E, Zhang T, et al. Learning agile robotic locomotion skills by imitating animals[J]. Science and Systems, 2020, 47(10):117-120.
[14]	Ratliff N, Bagnell J A, Srinivasa S S. Imitation learning for locomotion and manipulation[C]. Pittsburgh: IEEE-RAS International Conference on Humanoid Robots, 2007:392-397.
[15]	Ross S, Gordon G J, Bagnell J A. A reduction of imitation learning and structured prediction to no-regret online learning[C]. Fort Lauderdale: Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, 2011:627-635.
[16]	Ho J, Ermon S. Generative adversarial imitation learning[J]. Advances in Neural Information Processing Systems, 2016, 29(9):4565-4573.
[17]	Xie Z, Berseth G, Clary P, et al. Feedback control for cassie with deep reinforcement learning[C]. Madrid: IEEE/RSJ International Conference on Intelligent Robots and Systems, 2018:1241-1246.
[18]	Peng X B, Abbeel P, Levine S, et al. Deepmimic:Example-guided deep reinforcement learning of physics-based character skills[J]. ACM Transactions on Graphics, 2018, 37(4):1-14.
[19]	Wu Q, Zhang C, Liu Y. Custom sine waves are enough for imitation learning of bipedal gaits with different styles[C]. Nanjing: IEEE International Conference on Mechatronics and Automation, 2022:499-505.
[20]	战忠丽, 王强, 陈显亭. 强化学习的模型、算法及应用[J]. 电子科技, 2011, 24(1):47-49.
	Zhan Zhongli, Wang Qiang, Chen Xianting. Rinforcement learning models,algorithms and its application[J]. Electronic Science and Technology, 2011, 24(1):47-49.

参数	符号	数值	单位
小腿长度	l_s	25	cm
大腿长度	l_l	25	cm
小腿质量	m_s	1	kg
大腿质量	m_l	2	kg
髋部质心高度	h_h	69	cm
整体质量	m	10	kg
机器人高度	h	130	cm

参数	数值
批大小	36 728(1 024×32)
最小批	8 192(1 024×8)
学习率	0.000 35
衰减因子	0.95
GAE折扣率	0.99
回合数	4.00

方法	时间/s	前向	侧向	后向
本文方法	15	87	119	30
本文方法	30	77	89	36
文献[20]	15	76	106	33
文献[20]	30	71	71	36

Omnidirectional Gait Generation Method for Biped Robot with Fusion of Imitation Learning

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 10

References 20

Related Articles 7

Metrics

Comments

Recommended 0

[1]	FANG Shuai, CHEN Xu, LI Kangji. A Q-Learning Differential Evolution Algorithm for Combined Heat and Power Dynamic Economic Emission Dispatch [J]. Electronic Science and Technology, 2024, 37(5): 9-17.
[2]	TANG Chao, ZHANG Fan. Motion Planning of Manipulator Based on Improved SoftActor-Critic Algorithm [J]. Electronic Science and Technology, 2024, 37(11): 47-54.
[3]	XIAO Xiong, LIU Hongyan, YI Mengjie. Heterogeneous Converged Network Access Algorithm Based on Deep Reinforcement Learning [J]. Electronic Science and Technology, 2024, 37(11): 7-12.
[4]	SUO Fangfei,JI Yunfeng. Review of the Research on the Core Algorithm of Table Tennis Robot [J]. Electronic Science and Technology, 2023, 36(12): 16-24.
[5]	ZHANG Ye,BAO Liang. Research Progress of Service Composition Based on Machine Learning [J]. Electronic Science and Technology, 2022, 35(11): 58-63.
[6]	ZHANG Jiapeng,LI Lin,ZHU Ye. A Review of Research on Decision-Making Method of Autonomous Vehicle Based on Reinforcement Learning [J]. Electronic Science and Technology, 2021, 34(5): 66-71.
[7]	ZHAN Zhong-Li, WANG Qiang, CHEN Xian-Ting. Reinforcement Learning Model,Algorithms and Its Application [J]. , 2011, 24(1): 47-.