Kubernetes集群中多节点合作博弈负载均衡策略

doi:10.19665/j.issn1001-2400.2021.06.003

西安电子科技大学学报 ›› 2021, Vol. 48 ›› Issue (6): 16-22.doi: 10.19665/j.issn1001-2400.2021.06.003

• 智能嵌入式系统结构与软件关键技术专栏 • 上一篇下一篇

Kubernetes集群中多节点合作博弈负载均衡策略

李华东^1,^2,³(),张学亮^2,³(),王晓磊^2,³(),刘惠¹(),王鹏程¹(),杜军朝¹()

1.西安电子科技大学计算机科学与技术学院,陕西西安 710071
2.中国电子科技集团公司航天信息应用技术重点实验室,河北石家庄 050081
3.中国电子科技集团公司第五十四研究所,河北石家庄 050081

收稿日期:2021-09-02 出版日期:2021-12-20 发布日期:2022-02-24
作者简介:李华东(1997—),男,西安电子科技大学博士研究生,E-mail: r1v3n@foxmail.com|张学亮(1986—),男,硕士,E-mail: 18332186667@163.com|王晓磊(1993—),男,硕士,E-mail: 931104914@qq.com|王鹏程(1997—),男,西安电子科技大学硕士研究生,E-mail: wpc-0806@163.com|杜军朝(1974—),男,教授,博士,E-mail: dujz@xidian.edu.cn
基金资助:
国家自然科学基金重点项目(62032017)

Multi-node cooperative game load balancing strategy in the Kubernetes cluster

LI Huadong^1,^2,³(),ZHANG Xueliang^2,³(),WANG Xiaolei^2,³(),LIU Hui¹(),WANG Pengcheng¹(),DU Junzhao¹()

1. School of Computer Science and Technology,Xidian University,Xi’an 710071,China
2. CETC Key Laboratory of Aerospace Information Applications,Shijiazhuang 050081,China
3. The 54th Research Institute of CETC,Shijiazhuang 050081,China

Received:2021-09-02 Online:2021-12-20 Published:2022-02-24

摘要/Abstract

摘要：

Kubernetes具有成为新一代超融合架构的潜力,但在集群资源负载均衡方面的研究还存在不足:一方面现有调度算法大多为静态调度算法,没有考虑集群资源在实际使用中的动态性;另一方面现有的解决集群资源负载均衡的研究仅针对集群中的CPU和内存资源进行优化,无法给出集群完整的资源画像,算法缺乏全面性。因此,提出了一个基于合作博弈论的多资源负载均衡算法用于Kubernetes集群调度,并提出集群资源负载均衡度这个指标,优化调度从而降低集群资源碎片化程度。首先,使用实时监控的手段获取服务请求的实际资源使用情况,以实现服务的动态调度;其次,考虑集群CPU、内存、网络带宽以及磁盘IO资源之间的负载均衡,建立物理节点间的合作博弈模型,保证了基于合作博弈论的多资源负载均衡算法在面对各类应用不同资源请求时的算法下界,以减少集群资源碎片化现象;最后,将算法在真实Kubernetes集群中实测。实验结果表明,基于合作博弈论的多资源负载均衡算法降低了集群资源碎片化程度,集群中各节点的平均负载均衡度可提升8.40%。

关键词: 云计算, Kubernetes集群, 合作博弈论, 多资源负载均衡

Abstract:

Kubernetes has the potential to become a new generation of hyper-converged architecture,but there are still some problems in the research on cluster resource load balancing:on the one hand,most of the existing scheduling algorithms are static scheduling algorithms,and the dynamics of cluster resources in actual use are not considered; on the other hand,the existing research on solving cluster resource load balancing only optimizes the CPU and memory resources in the cluster,but cannot give a complete resource profile of the cluster,with the algorithm lacking comprehensiveness.In this paper,we introduce the MBCGT,a multi-resource load balancing algorithm based on the cooperative game theory for Kubernetes cluster scheduling,and proposes the index of cluster resource load balance to optimize scheduling and reduce the fragmentation of cluster resources.First,we use real-time monitoring to obtain the actual resource usage of service requests to achieve dynamic service scheduling.Second,we consider the load balance among the four resources of cluster CPU,memory,network bandwidth and disk IO,establish a cooperative game model between physical nodes to ensure the lower bound of the MBCGT algorithm when facing different resource requests of various applications,and solve the cluster Resource fragmentation phenomenon.Finally,the algorithm is tested in a real Kubernetes cluster.The results show that the MBCGT algorithm reduces the fragmentation of cluster resources,and that the average load balance of each node in the cluster can be increased by 8.40%.

Key words: cloud computing, Kubernetes cluster, cooperative game theory, multi-resource load balancing

中图分类号:

TP306

李华东,张学亮,王晓磊,刘惠,王鹏程,杜军朝. Kubernetes集群中多节点合作博弈负载均衡策略[J]. 西安电子科技大学学报, 2021, 48(6): 16-22.

LI Huadong,ZHANG Xueliang,WANG Xiaolei,LIU Hui,WANG Pengcheng,DU Junzhao. Multi-node cooperative game load balancing strategy in the Kubernetes cluster[J]. Journal of Xidian University, 2021, 48(6): 16-22.

图/表 4

表1

表2

表3

图1

参考文献 22

[1]	FELTER W, FERREIRA A, RAJAMONY R, et al. An Updated Performance Comparison of Virtual Machines and Linux Containers[C]// 2015 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS).Piscataway:IEEE, 2015:171-172.
[2]	KIM S, KIM C, KIM J. Reliable Smart Energy IoT-Cloud Service Operation with Container Orchestration[C]// 2017 19th Asia-Pacific Network Operations and Management Symposium (APNOMS).Piscataway:IEEE, 2017:378-381.
[3]	GONG Z. Kubernetes Definitive Guide:Full Contact from Docker to Kubernetes Practice[M]. Beijing:Publishing House of Electronics Industry, 2016:43-49.
[4]	MEDEL V, TOLOSANA-CALASANZ R, BAÑARES J Á, et al. Characterising Resource Management Performance in Kubernetes[J]. Computers & Electrical Engineering, 2018, 68:286-297.
[5]	CHANG C, YANG S, YEH E, et al. A Kubernetes-Based Monitoring Platform for Dynamic Cloud Resource Provisioning[C]// GLOBECOM 2017-2017 IEEE Global Communications Conference.Piscataway:IEEE, 2017:1-6.
[6]	MEDEL V, TOLON C, ARRONATEGUI U, et al. Client-Side Scheduling Based on Application Characterization on Kubernetes[C]// International Conference on the Economics of Grids,Clouds,Systems,and Services.Heidelberg:Springer, 2017:162-176.
[7]	CASQUERO O, ARMENTIA A, SARACHAGA I, et al. Distributed Scheduling in Kubernetes Based on MAS for Fog-in-the-Loop Applications[C]// 2019 24th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA).Piscataway:IEEE, 2019:1213-1217.
[8]	AHMED G E H, GIL-CASTIÑEIRA F, COSTA-MONTENEGRO E. KubCG:A Dynamic Kubernetes Scheduler for Heterogeneous Clusters[J]. Software:Practice and Experience, 2021, 51(2):213-234. doi: 10.1002/spe.v51.2
[9]	ZHANG W, MA X, ZHANG J. Research on Kubernetes' Resource Scheduling Scheme[C]// Proceedings of the 8th International Conference on Communication and Network Security.New York:ACM, 2018:144-148.
[10]	ZHAI G, TIAN L, ZHOU Y, et al. Load Diversity Based Optimal Processing Resource Allocation for Super Base Stations in Centralized Radio Access Networks[J]. Science China Information Sciences, 2014, 57(4):1-12.
[11]	ZHAI G, TIAN L, ZHOU Y, et al. A Computing Resource Adjustment Mechanism for Communication Protocol Processing in Centralized Radio Access Networks[J]. China Communications, 2016, 13(12):79-89.
[12]	LIU Q, E H, SONG M. The Design of Multi-Metric Load Balancer for Kubernetes[C]// 2020 International Conference on Inventive Computation Technologies (ICICT).Piscataway:IEEE, 2020:1114-1117.
[13]	HUANG J, XIAO C, WU W. RLSK:A Job Scheduler for Federated Kubernetes Clusters Based on Reinforcement Learning[C]// 2020 IEEE International Conference on Cloud Engineering (IC2E).Piscataway:IEEE, 2020:116-123.
[14]	左灿, 刘晓洁. 一种改进的Kubernetes动态资源调度方法[J]. 数据通信, 2019, 189(2):50-54.
	ZUO Can, LIU Xiaojie. An Improved Kubernetes Dynamic Resource Scheduling Method[J]. Data Communication, 2019, 189(2):50-54.
[15]	平凡, 陈莉君. 基于Kubernetes的动态负载均衡机制研究与设计[J]. 计算机与数字工程, 2020, 48(1):141-146.
	PING Fan, CHEN Lijun. Research and Design of Dynamic Load Balancing Mechanism Based on Kubernetes[J]. Computer & Digital Engineering, 2020, 48(1):141-146.
[16]	NGUYEN N D, KIM T. Balanced Leader Distribution Algorithm in Kubernetes Clusters[J]. Sensors, 2021, 21(3):869. doi: 10.3390/s21030869
[17]	龚正, 吴治辉, 闫健勇. Kubernetes权威指南[M]. 北京: 电子工业出版社, 2021:346-349.
[18]	CAO K, LI L, CUI Y, et al. Exploring Placement of Heterogeneous Edge Servers for Response Time Minimization in Mobile Edge-Cloud Computing[J]. IEEE Transactions on Industrial Informatics, 2021, 17(1):494-503. doi: 10.1109/TII.9424
[19]	王汝言, 吴和, 崔亚平, 等. 超密集网络中多基站博弈边缘卸载策略[J]. 西安电子科技大学学报, 2021, 48(4):1-10.
	WANG Ruyan, WU He, CUI Yaping, et al. Edge Offloading Strategy for the Multi-Base Station Game in Ultra-Dense Networks[J]. Journal of Xidian University, 2021, 48(4):1-10.
[20]	LIU L, ZHOU Y, GARCIA V, et al. Load Aware Joint CoMP Clustering and Inter-Cell Resource Scheduling in Heterogeneous Ultra Dense Cellular Networks[J]. IEEE Transactions on Vehicular Technology, 2018, 67(3):2741-2755. doi: 10.1109/TVT.25
[21]	赖成喆, 陈瑶, 郭奇立, 等. 车联网中基于博弈论的合作下载激励方案[J]. 西安电子科技大学学报, 2021, 48(3):9-20.
	LAI Chengzhe, CHEN Yao, GUO Qili, et al. Game Theory Based Incentive Scheme for Cooperative Downloading in the Internet of Vehicles[J]. Journal of Xidian University, 2021, 48(3):9-20.
[22]	NASH J. The Bargaining Problem[J]. Econometrica, 1950, 18(2):155-162. doi: 10.2307/1907266

节点名称	操作系统	CPU/核	内存/GB	网络带宽/(MB·s^-1)	磁盘IO/(MB·s^-1)
master	CentOS7	Inter Xeon 2	4	100	100
node1	CentOS7	Inter Xeon 2	4	100	100
node2	CentOS7	Inter Xeon 2	4	100	100
node3	CentOS7	Inter Xeon 2	4	100	100
node4	CentOS7	Inter Xeon 2	4	100	100
grafana	CentOS7	Inter Xeon 2	4	100	100

调度器	第1组实验	第2组实验	第3组实验
Kubernetes默认调度器	0.401	0.351	0.296
动态资源调度器^[12]	0.384	0.347	0.289
MBCGT算法调度器	0.270	0.233	0.221

Kubernetes集群中多节点合作博弈负载均衡策略

Multi-node cooperative game load balancing strategy in the Kubernetes cluster

RichHTML

PDF (PC)

赞

可视化

摘要/Abstract

引用本文

使用本文

图/表 4

参考文献 22

相关文章 13

Metrics

本文评价

推荐阅读 0

Pod	CPU/核	内存/MB	磁盘/MB	网络/MB
Pod 1	0.2	200	10	10
Pod 2	0.2	200	20	5
Pod 3	0.2	200	30	10
Pod 4	0.3	300	10	20
Pod 5	0.3	300	25	10
Pod 6	0.3	300	30	10
Pod 7	0.4	400	15	30
Pod 8	0.4	400	5	25
Pod 9	0.4	400	30	5
Pod 10	0.2	400	1	10
Pod 11	0.3	300	3	7
Pod 12	0.4	200	10	1

[1]	马英红, 吝李婉, 焦毅, 李秦尧. 通信计算联合优化的图分割工作流部署方法[J]. 西安电子科技大学学报, 2024, 51(2): 13-27.
[2]	何欣枫, 杨琴琴. 面向云存储的数据流行度去重方案[J]. 西安电子科技大学学报, 2024, 51(1): 187-200.
[3]	曹来成,吴文涛,冯涛,郭显. 面向数据质量的隐私保护多分类LR方案[J]. 西安电子科技大学学报, 2023, 50(5): 188-198.
[4]	蔡英,张猛,李新,张宇,范艳芳. 面向ASPE的抗合谋攻击图像检索隐私保护方案[J]. 西安电子科技大学学报, 2023, 50(5): 156-165.
[5]	王燕. 改进多目标进化算法的云工作流调度[J]. 西安电子科技大学学报, 2019, 46(1): 130-136.
[6]	韩鹍;张海林;辛丹;吴波;康元基. 利用身份的全同态加密函数库设计方法[J]. 西安电子科技大学学报, 2017, 44(4): 56-61.
[7]	景维鹏;霍帅起;陈广胜;刘亚秋. 混合关键任务可靠调度方法与调度性分析[J]. 西安电子科技大学学报, 2016, 43(6): 158-163.
[8]	黄春水;任艳丽;蔡建兴. 可验证模指数批计算外包方案[J]. 西安电子科技大学学报, 2016, 43(4): 135-140.
[9]	景维鹏;吴智博;刘宏伟;舒燕君. 多DAG工作流在云计算环境下的可靠性调度方法[J]. 西安电子科技大学学报, 2016, 43(2): 83-88.
[10]	柳兴;袁超伟;杨震;李振军. 一种基于李雅普诺夫优化的业务推送策略[J]. J4, 2015, 42(6): 138-144.
[11]	王一川;马建峰;卢笛;张留美;孟宪佳. 云环境下的“云滴冻结”攻击[J]. J4, 2014, 41(3): 116-122.
[12]	卢笛;马建峰;王一川;习宁;张留美;孟宪佳. 云计算下保障公平性的多资源分配算法[J]. J4, 2014, 41(3): 162-168.
[13]	朱旭东;李晖;郭祯. 云计算环境下加密图像检索[J]. J4, 2014, 41(2): 151-158.