Q学习差分进化算法求解热电动态经济排放调度

doi:10.16180/j.cnki.issn1007-7820.2024.05.002

Abstract

Abstract:

The dynamic economic emission scheduling of cogeneration takes into account both fuel cost and pollution gas emission, and the thermoelectricity output in the next period is affected by the thermoelectricity output in the current period, which is an important problem in power system operation in recent years. In this study, a new QLMODE(Q-Learning Multi-Objective Differential Evolution) algorithm is proposed to solve the CHPDEED(Combined Heat and Power Dynamic Economic Emission Dispatch) problem. In QLMODE, the Q-learning technique is used to adjust the scale factor parameters of the algorithm, that is, in the iterative process, the action reward and punishment are determined by using the dominant relationship between the child solution and the parent solution, and the parameter values are adjusted by Q-learning to obtain the most suitable algorithm parameters for the environmental model. The proposed QLMODE is used to solve the CHPDEED with 11 units and 33 units. The simulation results show that compared with four mature multi-objective optimization algorithms, the QLMODE algorithm has the least fuel cost and the least pollution gas emission, the convergence and diversity index of QLMODE algorithm is better than the other four algorithms, and QLMODE has a better Pareto optimal frontier on both sets of problems.

Key words: Q learning, reinforcement learning, multi-objective algorithm, differential evolution, cogeneration combined heat and power, economic emission dispatch, dynamic dispatch, power system

CLC Number:

TP18

FANG Shuai, CHEN Xu, LI Kangji. A Q-Learning Differential Evolution Algorithm for Combined Heat and Power Dynamic Economic Emission Dispatch[J].Electronic Science and Technology, 2024, 37(5): 9-17.

Figures/Tables 13

Figure 1.

Figure 2.

Figure 3.

Figure 4.

Table 1.

Table 2.

Table 3.

Figure 5.

Table 4.

Table 5.

Table 6.

Figure 6.

Table 7.

References 28

[1]	杨德友, 刘世宇. 求解电力系统多目标环境经济调度的帕累托最优MFO算法[J]. 电工电能新技术, 2018, 37(2):30-37. doi: 10.12067/ATEEE1706064
	Yang Deyou, Liu Shiyu. Multi-objective economic-enviro-nmental dispatch based on Pareto optimal moth-flame optimization algorithm[J]. Advanced Technology of Electrical Engineering and Energy, 2018, 37(2):30-37. doi: 10.12067/ATEEE1706064
[2]	王玉梅, 张继钦, 周永鑫. 计及碳交易和价格型需求响应的热电联合低碳经济调度[J]. 电子科技, 2023, 36(10):74-81.
	Wang Yumei, Zhang Jiqin, Zhou Yongxin. Combined heat and power low-carbon economic scheduling consideringcarbon trading and price demand response[J]. Eletronic Science and Techonolgy, 2023, 36(10):74-81.
[3]	Guo T, Henwood M I, Van Ooijen M. An algorithm for combined heat and power economic dispatch[J]. IEEE Transactions on Power Systems, 1996, 11(4):1778-1784.
[4]	Narang N, Sharma E, Dhillon J S. Combined heat and power economic dispatch using integrated civilized swarm optimization and Powell's pattern search method[J]. Applied Soft Computing, 2017, 52(1):190-202.
[5]	Nasir M, Sadollah A, Aydilek Í B, et al. A combination of FA and SRPSO algorithm for combined heat and power economic dispatch[J]. Applied Soft Computing, 2021, 102(1):88-98.
[6]	Khalili M, Nikoukar J, Sedighizadeh M. Combined heat and power economic dispatch using improved shuffled frog leaping algorithm[J]. Journal of Advances in Computer Research, 2021, 12(1):1-11.
[7]	Zou D, Li S, Kong X, et al. Solving the combined heat and power economic dispatch problems by an improved genetic algorithm and a new constraint handling strategy[J]. Applied Energy, 2019, 237(1):646-670.
[8]	Sundaram A. Multiobjective multiverse optimization algorithm to solve combined economic,heat and power emission dispatch problems[J]. Applied Soft Computing, 2020, 91(1):6195-6209.
[9]	Xiong G, Shuai M, Hu X. Combined heat and power economic emission dispatch using improved bare-bone multiobjective particle swarm optimization[J]. Energy, 2022, 244(1):3108-3127.
[10]	Jadoun V K, Prashanth G R, Joshi S S, et al. Optimal fuzzy based economic emission dispatch of combined heat and power units using dynamically controlled whale optimization algorithm[J]. Applied Energy, 2022, 315(1):33-50.
[11]	Niknam T, Azizipanah-Abarghooee R, Roosta A, et al. A new multiobjective reserve constrained combined heatand power dynamic economic emission dispatch[J]. Energy, 2012, 42(1):530-545.
[12]	Elaiw A M, Xia X, Shehata A M. Combined heat and power dynamic economic dispatch with emission limitations using hybrid DE-SQP method[J]. Abstract and Applied Analysis, 2013(7):1-10.
[13]	Basu M. Combined heat and power economic emission dispatch using nondominated sorting genetic algorithm-II[J]. International Journal of Electrical Power and Energy Systems, 2013, 53(1):135-141.
[14]	Alomoush M I. Application of the stochastic fractal search algorithm and compromise programming to combined heat and power economic-emission dispatch[J]. Engineering Optimization, 2020, 52(11):1992-2010.
[15]	Chen X, Li K, Xu B, et al. Biogeography-based learning particle swarm optimization for combined heat and power economic dispatch problem[J]. Knowledge-Based Systems, 2020, 208(15):106463-106469.
[16]	赖俊, 魏竞毅, 陈希亮. 分层强化学习综述[J]. 计算机工程与应用, 2021, 57(3):72-79. doi: 10.3778/j.issn.1002-8331.2010-0038
	Lai Jun, Wei Jingyi, Chen Xiliang. Overview of hierarchical reinforcement learnin[J]. Computer Engineering and Applications, 201, 57(3):72-79.
[17]	郑敦勇, 姚宜斌, 聂文锋, 等. 基于机器学习集成算法的电离层层析算法迭代初值精化[J]. 地球物理学报, 2022, 65(8):2796-2812.
	Zheng Dunyong, Yao Yibin, Nie Wenfeng, et al. Iterative initial value refinement of ionospheric tomography algorithm based on machine learning ensemble algorithm[J]. Chinese Journal of Geophysics, 2022, 65(8):2796-2812.
[18]	Hu Z, Gong W, Li S. Reinforcement learning-based differential evolution for parameters extraction of photovoltaic models[J]. Energy Reports, 2021(7):916-928.
[19]	张秦浩, 敖百强, 张秦雪. Q-learning强化学习制导律[J]. 系统工程与电子技术, 2020, 42(2):414-419. doi: 10.3969/j.issn.1001-506X.2020.02.21
	Zhang Qinhao, Ao Baiqiang, Zhang Qinxue. Reinforcemen tlearning guidance law of Q-learning[J]. Systems Engineering and Electronics, 2020, 42(2):414-419. doi: 10.3969/j.issn.1001-506X.2020.02.21
[20]	Watkins C J C H, Dayan P. Q-learning[J]. Machine Learning, 1992, 8(3):279-292.
[21]	葛传九, 武鹏, 金俊喆, 等. 基于改进差分进化算法的光伏最大功率点跟踪[J]. 电子科技, 2022, 35(9):15-21.
	Ge Chuanjiu, Wu Peng, Jin Junzhe, et al. Photovoltaic maximum power point tracking based on improved differential evolution algorithm[J]. Electronic Science and Technology, 2022, 35(9):15-21.
[22]	冯士刚, 艾芊. 带精英策略的快速非支配排序遗传算法在多目标无功优化中的应用[J]. 电工技术学报, 2007(12):146-151.
	Feng Shigang, Ai Qian. Application of fast and elitist non-dominated sorting generic algorithm in multiobjective reactive power optimization[J]. Transaction of China Electrotechnical Society, 2007(12):146-151.
[23]	Shaabani Y A, Seifi A R, Kouhanjani M J. Stochastic multiobjective optimization of combined heat and power economic/emission dispatch[J]. Energy, 2017, 141(15):1892-1904.
[24]	Kukkonen S, Lampinen J. GDE3:The third evolution step of generalized differential evolution[C]. Edinburgh: IEEE Congress on Evolutionary Computation, 2005:117-128.
[25]	Li H, Zhang Q. Multi-objective optimization problems with complicated Pareto sets,MOEA/D and NSGA-II[J]. IEEE Transactions on Evolutionary Computation, 2008, 13(2):284-302.
[26]	Chen X, Du W, Qian F. Multi-objective differential evolution with ranking-based mutation operator and it's application in chemical process optimization[J]. Chemometrics and Intelligent Laboratory Systems, 2014, 136(15):85-96.
[27]	Zitzler E, Thiele L. Multi-objective evolutionary algorithms:A comparative case study and the strength Pareto approach[J]. IEEE Transactions on Evolutionary Computation, 1999, 3(4):257-271.
[28]	Coello C A C, Cortés N C. Solving multi-objective optimization problems using an artificial immune system[J]. Genetic Programming and Evolvable Machines, 2005, 6(2):163-190.

算法	参数
TVMOPSO	种群大小N=100,权重系数w_min=0.1,w_max=0.9, 加速度系数C₁_f=C₂_i=0.5,C₁_i=C₂_f=2
GDE3	N=100,比例因子F=0.5,交叉因子CR =0.5
NSGA-II-DE	N=100,多项式变异率η =20,F=0.5,CR =0.5
MODE-RMO	N=100,F=0.5,CR =0.5
QLMODE	N=100,CR=0.5,α =0.1,γ =0.5

时间 /h	电力需求 /MW	热量需求 /MWth	时间 /h	电力需求 /MW	热量需求 /MWth
1	1 036	390	13	2 072	470
2	1 110	400	14	1 924	460
3	1 258	410	15	1 776	450
4	1 406	420	16	1 554	450
5	1 480	440	17	1 480	420
6	1 628	450	18	1 628	435
7	1 702	450	19	1 776	445
8	1 776	455	20	1 972	450
9	1 924	460	21	1 924	445
10	2 022	460	22	1 628	435
11	2 106	470	23	1 332	400
12	2 150	480	24	1 184	400

		TV-MOPSO	GDE3	NSGA-II-DE	MODE-RMO	QLMDOE
经济调度	花费/美元	2 571 077.76	2 601 166.27	2 570 790.17	2 586 935.69	2 568 391.84
经济调度	排放/kg	268 580.93	288 674.01	279 342.01	279 073.88	268 279.16
排放调度	花费/美元	2 654 159.60	2 692 465.06	2 692 234.83	2 691 947.73	2 682 440.57
排放调度	排放/kg	250 458.36	255 900.25	248 606.82	254 109.93	246 147.21
经济排放调度	花费/美元	2 598 219.81	2 611 266.56	2 600 049.17	2 606 718.77	2 597 173.09
经济排放调度	排放/kg	256 965.00	271 013.55	260 177.24	265 514.19	256 638.51
	时间/s	884.65	549.20	584.01	888.32	660.29

指标	算法	MIN	MEAN	MAX	STD	Sig.
HV	TVMOPSO	0.024 8	0.025 5	0.026 1	0.000 3	+
	GDE3	0.021 5	0.022 4	0.022 9	0.000 4	+
	NSGA-II-DE	0.025 3	0.026 0	0.026 8	0.000 3	+
	MODE-RMO	0.022 5	0.023 1	0.024 0	0.000 4	+
	QLMODE	0.026 3	0.026 9	0.027 6	0.000 4
IGD	TVMOPSO	5 428.856 1	9 173.081 3	13 339.615 5	1 968.893 5	+
	GDE3	13 765.600 8	18 296.121 3	23 561.906 5	2 109.964 2	+
	NSGA-II-DE	3 735.038 7	5 957.392 7	8 399.362 4	1 277.406 4	+
	MODE-RMO	12 970.203 4	15 633.338 8	18 384.426 1	1 446.620 2	+
	QLMODE	1 469.630 7	3 883.327 9	6 476.832 7	1 327.309 0

时间 /h	电力需求 /MW	热量需求 /MWth	时间 /h	电力需求 /MW	热量需求 /MWth
1	3 108	1 170	13	6 216	1 410
2	3 330	1 200	14	5 772	1 380
3	3 774	1 230	15	5 328	1 350
4	4 218	1 260	16	4 662	1 350
5	4 440	1 320	17	4 440	1 260
6	4 884	1 350	18	4 884	1 305
7	5 106	1 350	19	5 328	1 335
8	5 328	1 365	20	5 916	1 350
9	5 772	1 380	21	5 772	1 335
10	6 066	1 380	22	4 884	1 305
11	6 318	1 410	23	3 996	1 200
12	6 450	1 440	24	3 552	1 200

A Q-Learning Differential Evolution Algorithm for Combined Heat and Power Dynamic Economic Emission Dispatch

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 13

References 28

Related Articles 15

Metrics

Comments

Recommended 0

		TV-MOPSO	GDE3	NSGA-II-DE	MODE-RMO	QLMDOE
经济调度	花费/美元	7 428 285.76	7 608 953.99	7 373 114.36	7 596 409.06	7 336 747.55
经济调度	排放/kg	713 262.09	776 004.69	770 801.48	774 150.25	769 256.11
排放调度	花费/美元	7 641 683.81	7 960 837.33	7 734 528.20	7 917 340.91	7 725 719.38
排放调度	排放/kg	672 885.01	703 802.05	674 066.75	700 204.78	661 161.36
经济排放调度	花费/美元	7 441 042.62	7 633 185.95	7 452 468.02	7 617 008.05	7 428 286.78
经济排放调度	排放/kg	706 826.79	746 138.05	711 124.59	740 809.14	701 931.62
	时间/s	2 702.51	2 647.14	2 656.14	2 597.03	2 750.26

指标	算法	MIN	MEAN	MAX	STD	Sig.
HV	TVMOPSO	0.023 3	0.024 1	0.024 7	0.000 3	+
	GDE3	0.014 9	0.015 5	0.016 0	0.000 3	+
	NSGA-II-DE	0.023 7	0.024 4	0.025 0	0.000 3	+
	MODE-RMO	0.016 0	0.016 6	0.017 2	0.000 4	+
	QLMODE	0.026 6	0.027 0	0.027 4	0.000 2
IGD	TVMOPSO	21 763.739 4	32 689.357 8	40 875.839 6	4 771.838 2	+
	GDE3	148 934.550 5	175 504.554 0	200 852.745 4	11 252.548 3	+
	NSGA-II-DE	20 041.296 2	26 890.651 4	32 933.353 5	3 457.244 3	+
	MODE-RMO	135 719.617 5	151 095.603 0	165 686.411 4	9 395.289 7	+
	QLMODE	2 042.593 9	5 149.226 9	8 596.482 7	1 612.409 9

[1]	NIE Chunfang, HAO Zhenghang, CHEN Zhuo, HE Puxiang. Research on Electromagnetic Transient Acceleration Simulation Technoloy of New-Type Power System [J]. Electronic Science and Technology, 2024, 37(3): 18-25.
[2]	ZHANG Haifang,HE Qinglong,ZHANG Lin. Power Load Forecasting Model Based on Expansion Period [J]. Electronic Science and Technology, 2024, 37(2): 1-5.
[3]	SUO Fangfei,JI Yunfeng. Review of the Research on the Core Algorithm of Table Tennis Robot [J]. Electronic Science and Technology, 2023, 36(12): 16-24.
[4]	GE Chuanjiu,WU Peng,JIN Junzhe,DONG Xiangxiang,LOU Qikai. Photovoltaic Maximum Power Point Tracking Based on Improved Differential Evolution Algorithm [J]. Electronic Science and Technology, 2022, 35(9): 15-21.
[5]	ZHANG Ye,BAO Liang. Research Progress of Service Composition Based on Machine Learning [J]. Electronic Science and Technology, 2022, 35(11): 58-63.
[6]	ZHANG Jiapeng,LI Lin,ZHU Ye. A Review of Research on Decision-Making Method of Autonomous Vehicle Based on Reinforcement Learning [J]. Electronic Science and Technology, 2021, 34(5): 66-71.
[7]	YANG Haizhu,JIANG Zhaoyang,LI Menglong,KANG Le. Parameters Selection for LSSVM Based on Artificial Fish Swarm-Shuffled Frog Jump Algorithms Optimization in Short-Term Load Forecasting [J]. Electronic Science and Technology, 2020, 33(12): 67-74.
[8]	SUN Jing,ZHU Jun,LIU Hua,WANG Canxia. Research on UAV Application of Magnetic Resonance Coupling Wireless Power Transmission System [J]. Electronic Science and Technology, 2019, 32(4): 49-53.
[9]	BING Xiaofeng,TAO Yifei,DONG Yuanyuan,SUN Sihan. Solving the Problem of Replacement Flow Shop Scheduling Based on Improved Cuckoo Algorithm [J]. Electronic Science and Technology, 2019, 32(10): 60-64.
[10]	YANG Jun,WEI Jingxuan. Adaptive Differential Evolution Algorithm Based on Gradient Search Strategy [J]. , 2016, 29(1): 25-.
[11]	MA Lixin,ZHENG Xiaodong,YIN Jingjing. Short-term Load Forecasting Based on Rough Characteristic-component Algorithm [J]. , 2016, 29(1): 40-.
[12]	MA Lixin,LUAN Jian,WANG Jiyin. Three-objective Cauchy Particle Swarm Optimization for Reactive Power Optimization [J]. , 2015, 28(9): 42-.
[13]	BO Pingping,GONG Rujiang. A New Algorithm for OFDM Multi-Band Joint Spectrum Sensing [J]. , 2015, 28(8): 130-.
[14]	LI Hongda,LI Xin,Alexander G Lyublinsky,Sergey V Korotkov,ZHAO Hongyang,CHE Long. RSD Based Generators for the Pulse of Broken Concrete Crushing Experiment and Simulation [J]. , 2015, 28(5): 53-.
[15]	HU Yajun,CHEN Qing. Application of Image Watermark Technology in the Electric Power System [J]. , 2015, 28(4): 149-.