氢电耦合储能系统：模型、应用和深度强化学习算法
DOI:
https://doi.org/10.18686/cncest.v2i1.146关键词:
氢储能；电力系统；深度强化学习；应用场景摘要
随着氢储能技术的成熟，绿色电力和绿色氢气模式的氢电耦合储能将成为理想的能源系统。氢电耦合储能系统（hydrogen-electricity coupling energy storage systems，HECESSs）建设是能源供应和深度脱碳的重要技术途径之一。在HECESS中，氢储能可以维持能源供需平衡，提高能源利用效率。但其在电力系统建立中的场景模型及相应的解决方法仍需深入研究。为加快HECESS建设，首先从制氢、氢气发电、储氢三个方面阐述了氢储能技术的应用现状。其次，基于氢能和电能的互补协同机制，描述了HECESS的结构和运行模式。为了更深入地研究HECESS的工程应用，综述了国内外HECESS在电源侧、电网侧和负荷侧场景的最新进展。对于源-网-荷侧氢储能应用模型来说，求解方法的合理选择将影响模型的最优解和求解效率。传统的优化方法难以解决复杂的多能耦合模型，因此本文探讨了深度强化学习（deep reinforcement learning，DRL）算法的优势及其在HECESS中的应用。最后对HECESS支撑的新型电力系统建设中的技术应用进行了展望，旨在为氢储能在电力系统中的应用研究提供参考。
参考
Fuso Nerini F, Tomei J, To LS, et al. Mapping synergies and trade-offs between energy and the Sustainable Development Goals. Nature Energy. 2017, 3(1): 10-15. doi: 10.1038/s41560-017-0036-5
Iqbal M, Benmouna A, Becherif M, Mekhilef S. Survey on battery technologies and modeling methods for electric vehicles. Batteries. 2023; 9(3):185. https://doi.org/10.3390/batteries9030185
张智刚, 康重庆. 碳中和目标下构建新型电力系统的挑战与展望. 中国电机工程学报. 2022; 42(8): 2806–2819.
Zhang Z, Kang C. Challenges and prospects for constructing the new-type power system towards a carbon neutrality future (Chinese). Proceedings of the CSEE. 2022; 42(8): 2806–2819.
Cai G, Kong L, Xue Y, Sun B. Overview of research on wind power coupled with hydrogen production technology. Automation of Electric Power Systems. 2014; 38(21): 127–135.
Pan G, Gu W, Zhang H, Qiu Y. Electricity and hydrogen energy system towards accomodation of high proportion of renewable energy. Automation of Electric Power Systems. 2020; 44(23): 1–10. doi: 10.7500/AEPS20200202003
Hanley ES, Deane J, Gallachóir BÓ. The role of hydrogen in low carbon energy futures–A review of existing perspectives. Renewable and Sustainable Energy Reviews. 2018, 82: 3027-3045. doi: 10.1016/j.rser.2017.10.034
李佳蓉, 林今, 邢学韬, 等. 主动配电网中基于统一运行模型的电制氢（P2H）模块组合选型与优化规划. 中国电机工程学报. 2021: 1–13. doi: 10.13334/j.0258-8013.pcsee.201307
Li J, Lin J, Xing X, et al. Technology portfolio selection and optimal planning of power-to-hydrogen (p2h) modules in active distribution network (Chinese). Proceedings of the CSEE. 2021; 41(12): 4021–4033. doi: 10.13334/j.0258-8013.pcsee.201307
Abe JO, Popoola API, Ajenifuja E, et al. Hydrogen energy, economy and storage: Review and recommendation. International Journal of Hydrogen Energy. 2019, 44(29): 15072-15086. doi: 10.1016/j.ijhydene.2019.04.068
Razzhivin IA, Suvorov AA, Ufa RA, et al. The energy storage mathematical models for simulation and comprehensive analysis of power system dynamics: A review. Part i. International Journal of Hydrogen Energy. 2023, 48(58): 22141-22160. doi: 10.1016/j.ijhydene.2023.03.070
Moradi R, Groth KM. Hydrogen storage and delivery: Review of the state of the art technologies and risk and reliability analysis. International Journal of Hydrogen Energy. 2019, 44(23): 12254-12269. doi: 10.1016/j.ijhydene.2019.03.041
Pei W, Zhang X, Deng W, et al. Review of operational control strategy for DC microgrids with electric-hydrogen hybrid storage systems. CSEE Journal of Power and Energy Systems. 2022; 8(2): 329–346. doi: 10.17775/CSEEJPES.2021.06960
Egeland-Eriksen T, Hajizadeh A, Sartori S. Hydrogen-based systems for integration of renewable energy in power systems: Achievements and perspectives. International Journal of Hydrogen Energy. 2021, 46(63): 31963-31983. doi: 10.1016/j.ijhydene.2021.06.218
Liu W, Sun L, Li Z, et al. Trends and future challenges in hydrogen production and storage research. Environmental Science and Pollution Research. 2020, 27(25): 31092-31104. doi: 10.1007/s11356-020-09470-0
王士博, 孔令国, 蔡国伟, 等. 电力系统氢储能关键应用技术现状、挑战及展望. 中国电机工程学报. 2023; 43(17): 6660–6681. doi: 10.13334/j.0258-8013.pcsee.230170
Wang S, Kong L, Cai G, et al. Current status, challenges and prospects of key application technologies for hydrogen storage in power system (Chinese). Proceedings of the CSEE. 2023; 43(17): 6660–6681. doi: 10.13334/j.0258-8013.pcsee.230170
Yue M, Lambert H, Pahon E, et al. Hydrogen energy systems: A critical review of technologies, applications, trends and challenges. Renewable and Sustainable Energy Reviews. 2021, 146: 111180. doi: 10.1016/j.rser.2021.111180
李亚楼, 王丹丹, 赵飞, 李芳. 电力多元转换（Power-to-X）：技术路径、应用与挑战. 电网技术. 2023; 1–14. doi: 10.13335/j.1000-3673.pst.2023.1234
Li Y, Wang D, Zhao F, Li F. Path, application and challenge of power-to-x (Chinese). Power System Technology.
; 1–14. doi: 10.13335/j.1000-3673.pst.2023.1234
Gao J, Song J, Wang JX, et al. Form and key technologies of integrated electricity-hydrogen system supporting energy security in China. Automation of Electric Power Systems. 2023; 47(19): 1–15.
Wu X, Li H, Wang X, et al. Cooperative Operation for Wind Turbines and Hydrogen Fueling Stations With On-Site Hydrogen Production. IEEE Transactions on Sustainable Energy. 2020, 11(4): 2775-2789. doi: 10.1109/tste.2020.2975609
Li Q, Zhao S, Pu Y, Chen W, Yu J. Capacity optimization of hybrid energy storage microgrid consid- ering electricity-hydrogen coupling. Transactions of China Electrotechnical Society 2021;36(3):486–95.
Xiong Y, Chen L, Zheng T, et al. Electricity-Heat-Hydrogen Modeling of Hydrogen Storage System Considering Off-Design Characteristics. IEEE Access. 2021, 9: 156768-156777. doi: 10.1109/access.2021.3130175
Liu L, Zhai R, Hu Y. Multi-objective optimization with advanced exergy analysis of a wind-solar‑hydrogen multi-energy supply system. Applied Energy. 2023, 348: 121512. doi: 10.1016/j.apenergy.2023.121512
Li J, Lin J, Song Y, et al. Operation Optimization of Power to Hydrogen and Heat (P2HH) in ADN Coordinated With the District Heating Network. IEEE Transactions on Sustainable Energy. 2019, 10(4): 1672-1683. doi: 10.1109/tste.2018.2868827
Pan G, Gu W, Lu Y, et al. Optimal Planning for Electricity-Hydrogen Integrated Energy System Considering Power to Hydrogen and Heat and Seasonal Storage. IEEE Transactions on Sustainable Energy. 2020, 11(4): 2662-2676. doi: 10.1109/tste.2020.2970078
Lin H, Wu Q, Chen X, et al. Economic and technological feasibility of using power-to-hydrogen technology under higher wind penetration in China. Renewable Energy. 2021, 173: 569-580. doi: 10.1016/j.renene.2021.04.015
Xiong J, Jiao Y, Wang M. A day-ahead optimal scheduling of regional integrated energy system considering power to gas. Modern Electric Power. 2022; 39(5): 554–561. doi: 10.19725/j.cnki.1007-2322.2021.0132
Tao Y, Qiu J, Lai S, et al. Collaborative Planning for Electricity Distribution Network and Transportation System Considering Hydrogen Fuel Cell Vehicles. IEEE Transactions on Transportation Electrification. 2020, 6(3): 1211-1225. doi: 10.1109/tte.2020.2996755
Li J, Lin J, Zhang H, et al. Optimal Investment of Electrolyzers and Seasonal Storages in Hydrogen Supply Chains Incorporated With Renewable Electric Networks. IEEE Transactions on Sustainable Energy. 2020, 11(3): 1773-1784. doi: 10.1109/tste.2019.2940604
El-Taweel NA, Khani H, Farag HEZ. Hydrogen Storage Optimal Scheduling for Fuel Supply and Capacity-Based Demand Response Program Under Dynamic Hydrogen Pricing. IEEE Transactions on Smart Grid. 2019, 10(4): 4531-4542. doi: 10.1109/tsg.2018.2863247
Pan G, Gu W, Qiu H, et al. Bi-level mixed-integer planning for electricity-hydrogen integrated energy system considering levelized cost of hydrogen. Applied Energy. 2020, 270: 115176. doi: 10.1016/j.apenergy.2020.115176
Zheng JH, Guo JC, Li Z, et al. Optimal design for a multi-level energy exploitation unit based on hydrogen storage combining methane reactor and carbon capture, utilization and storage. Journal of Energy Storage. 2023, 62: 106929. doi: 10.1016/j.est.2023.106929
Cheng Y, Zhang N, Lu Z, et al. Planning Multiple Energy Systems Toward Low-Carbon Society: A Decentralized Approach. IEEE Transactions on Smart Grid. 2019, 10(5): 4859-4869. doi: 10.1109/tsg.2018.2870323
Jiang Y, Yang G, Zhu Z. Wind-hydrogen-electricity coupled network planning considering traffic flow capture. Automation of Electric Power Systems. 2021; 45(22): 19–28.
Yang Y, Ma C, Lian C, et al. Optimal power reallocation of large-scale grid-connected photovoltaic power station integrated with hydrogen production. Journal of Cleaner Production. 2021, 298: 126830. doi: 10.1016/j.jclepro.2021.126830
Zhang Y, Hua QS, Sun L, et al. Life Cycle Optimization of Renewable Energy Systems Configuration with Hybrid Battery/Hydrogen Storage: A Comparative Study. Journal of Energy Storage. 2020, 30: 101470. doi: 10.1016/j.est.2020.101470
Garg H. A hybrid GSA-GA algorithm for constrained optimization problems. Information Sciences. 2019, 478: 499-523. doi: 10.1016/j.ins.2018.11.041
Xie Y, Sheng Y, Qiu M, et al. An adaptive decoding biased random key genetic algorithm for cloud workflow scheduling. Engineering Applications of Artificial Intelligence. 2022, 112: 104879. doi: 10.1016/j.engappai.2022.104879
Karimi-Mamaghan M, Mohammadi M, Meyer P, et al. Machine learning at the service of meta-heuristics for solving combinatorial optimization problems: A state-of-the-art. European Journal of Operational Research. 2022, 296(2): 393-422. doi: 10.1016/j.ejor.2021.04.032
Sun H, Li Z, Chen A, et al. Current status and development trend of hydrogen production technology by wind power. Transactions of China Electrotechnical Society. 2019; 34(19): 4071–4083. doi: 10.19595/j.cnki.1000-6753.tces.180241
彭生江, 杨淑霞, 袁铁江. 面向风煤富集区域的风‒氢‒煤耦合系统演化发展系统动力学. 高电压技术. 2023; 49(8): 3478–3489.
Peng S, Yang S, Yuan T. System dynamics of the evolutionary development of coupled wind-hydrogen-coal system for wind-coal enriched areas (Chinese). High Voltage Engineering. 2023; 49(8): 3478‒3489.
Fan H, Wang L, Xing M, et al. Coordinated scheduling of multiple buildings with electric-hydrogen complementary considering frequency stability constraints. Journal of Shanghai Jiaotong University. 2023; 57(12): 1559–1570. doi: 10.16183/j.cnki.jsjtu.2022.380
Xu S, Wang H, Cao Y, et al. Parameter tuning for self-synchronous voltage source doubly-fed wind turbines with stability boundary and multi-objective constraint. Automation of Electric Power Systems. 2023; 47(11): 18–28.
Wang F, Yang H, Li L, et al. Collaborative optimal method for electricity-hydrogen integrated energy system considering spatial-temporal characteristics of hydrogen transportation. Automation of Electric Power Systems. 2023; 47(19): 31–43.
Bai X, Fan Y, Liu Y, Song Y. Wind power storage virtual power plant considering reliability and flexibilitytiered capacity configuration. Power System Protection and Control. 2022; 50(8): 11–24.
Fan H, Yu W, Liu L, Dou Z. Multi-building coordinated dispatch in smart park for carbon emission peak and carbon neutrality considering electricity and hydrogen complementary. Automation of Electric Power Systems. 2022; 46(21): 42–51.
Ernst D, Glavic M, Capitanescu F, et al. Reinforcement Learning Versus Model Predictive Control: A Comparison on a Power System Problem. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics). 2009, 39(2): 517-529. doi: 10.1109/tsmcb.2008.2007630
Sharma P, Dutt Mathur H, Mishra P, et al. A critical and comparative review of energy management strategies for microgrids. Applied Energy. 2022, 327: 120028. doi: 10.1016/j.apenergy.2022.120028
Ceusters G, Rodríguez RC, García AB, et al. Model-predictive control and reinforcement learning in multi-energy system case studies. Applied Energy. 2021, 303: 117634. doi: 10.1016/j.apenergy.2021.117634
Sutton RS, Barto AG. Reinforcement Learning: An Introduction, 2nd ed. MIT Press; 2018.
Du G, Zou Y, Zhang X, et al. Deep reinforcement learning based energy management for a hybrid electric vehicle. Energy. 2020, 201: 117591. doi: 10.1016/j.energy.2020.117591
Ding T, Zeng Z, Bai J, et al. Optimal Electric Vehicle Charging Strategy With Markov Decision Process and Reinforcement Learning Technique. IEEE Transactions on Industry Applications. 2020, 56(5): 5811-5823. doi: 10.1109/tia.2020.2990096
Li Y, Yu C, Shahidehpour M, et al. Deep Reinforcement Learning for Smart Grid Operations: Algorithms, Applications, and Prospects. Proceedings of the IEEE. 2023, 111(9): 1055-1096. doi: 10.1109/jproc.2023.3303358
Zheng JH, Wang WH, Li Z, Wu QH. Multi-layer double deep Q network for active distribution network equivalent modeling with internal identification for EV loads. Applied Soft Computing. 2023; 147: 110834. doi: 10.1016/j.asoc.2023.110834
Yang D, Wang L, Yu K, et al. A reinforcement learning-based energy management strategy for fuel cell hybrid vehicle considering real-time velocity prediction. Energy Conversion and Management. 2022, 274: 116453. doi: 10.1016/j.enconman.2022.116453
Wu JJ, Song DF, Zhang XM, et al. Multi-objective reinforcement learning-based energy management for fuel cell vehicles considering lifecycle costs. International Journal of Hydrogen Energy. 2023, 48(95): 37385-37401. doi: 10.1016/j.ijhydene.2023.06.145
Lecun Y, Bottou L, Bengio Y, et al. Gradient-based learning applied to document recognition. Proceedings of the IEEE. 1998, 86(11): 2278-2324. doi: 10.1109/5.726791
Ravanelli M, Brakel P, Omologo M, et al. Light Gated Recurrent Units for Speech Recognition. IEEE Transactions on Emerging Topics in Computational Intelligence. 2018, 2(2): 92-102. doi: 10.1109/tetci.2017.2762739
Bao S, Tang S, Sun P, et al. LSTM-based energy management algorithm for a vehicle power-split hybrid powertrain. Energy. 2023, 284: 129267. doi: 10.1016/j.energy.2023.129267
Zhang L, Zhang J, Gao T, et al. Improved LSTM based state of health estimation using random segments of the charging curves for lithium-ion batteries. Journal of Energy Storage. 2023, 74: 109370. doi: 10.1016/j.est.2023.109370
Jin R, Zhou Y, Lu C, et al. Deep reinforcement learning-based strategy for charging station participating in demand response. Applied Energy. 2022, 328: 120140. doi: 10.1016/j.apenergy.2022.120140
Lu S, Liu S, Zhu Y, et al. A DRL-Based Decentralized Computation Offloading Method: An Example of an Intelligent Manufacturing Scenario. IEEE Transactions on Industrial Informatics. 2023, 19(9): 9631-9641. doi: 10.1109/tii.2022.3227652
Hwang HS, Lee M, Seok J. Deep reinforcement learning with a critic-value-based branch tree for the inverse design of two-dimensional optical devices. Applied Soft Computing. 2022, 127: 109386. doi: 10.1016/j.asoc.2022.109386
Xiao H, Fu L, Shang C, et al. Ship energy scheduling with DQN-CE algorithm combining bi-directional LSTM and attention mechanism. Applied Energy. 2023, 347: 121378. doi: 10.1016/j.apenergy.2023.121378
Huang R, He H, Zhao X, et al. Longevity-aware energy management for fuel cell hybrid electric bus based on a novel proximal policy optimization deep reinforcement learning framework. Journal of Power Sources. 2023, 561: 232717. doi: 10.1016/j.jpowsour.2023.232717
Zeng S, Huang C, Wang F, et al. A Policy optimization-based Deep Reinforcement Learning method for data-driven output voltage control of grid connected solid oxide fuel cell considering operation constraints. Energy Reports. 2023, 10: 1161-1168. doi: 10.1016/j.egyr.2023.07.036
Martinez Cesena EA, Loukarakis E, Good N, et al. Integrated Electricity– Heat–Gas Systems: Techno–Economic Modeling, Optimization, and Application to Multienergy Districts. Proceedings of the IEEE. 2020, 108(9): 1392-1410. doi: 10.1109/jproc.2020.2989382
Yi Z, Luo Y, Westover T, et al. Deep reinforcement learning based optimization for a tightly coupled nuclear renewable integrated energy system. Applied Energy. 2022, 328: 120113. doi: 10.1016/j.apenergy.2022.120113
Yang T, Zhao L, Li W, et al. Dynamic energy dispatch strategy for integrated energy system based on improved deep reinforcement learning. Energy. 2021, 235: 121377. doi: 10.1016/j.energy.2021.121377
Zhang G, Hu W, Cao D, et al. Data-driven optimal energy management for a wind-solar-diesel-battery-reverse osmosis hybrid energy system using a deep reinforcement learning approach. Energy Conversion and Management. 2021, 227: 113608. doi: 10.1016/j.enconman.2020.113608
Zhang B, Hu W, Cao D, et al. Deep reinforcement learning–based approach for optimizing energy conversion in integrated electrical and heating system with renewable energy. Energy Conversion and Management. 2019, 202: 112199. doi: 10.1016/j.enconman.2019.112199
Xu G, Lin Z, Wu Q, et al. Deep reinforcement learning based model-free optimization for unit commitment against wind power uncertainty. International Journal of Electrical Power & Energy Systems. 2024, 155: 109526. doi: 10.1016/j.ijepes.2023.109526
Alabi TM, Lawrence NP, Lu L, et al. Automated deep reinforcement learning for real-time scheduling strategy of multi-energy system integrated with post-carbon and direct-air carbon captured system. Applied Energy. 2023, 333: 120633. doi: 10.1016/j.apenergy.2022.120633
Monfaredi F, Shayeghi H, Siano P. Multi-agent deep reinforcement learning-based optimal energy management for grid-connected multiple energy carrier microgrids. International Journal of Electrical Power & Energy Systems. 2023, 153: 109292. doi: 10.1016/j.ijepes.2023.109292
Xiong K, Hu W, Cao D, et al. Coordinated energy management strategy for multi-energy hub with thermo-electrochemical effect based power-to-ammonia: A multi-agent deep reinforcement learning enabled approach. Renewable Energy. 2023, 214: 216-232. doi: 10.1016/j.renene.2023.05.067
Guo G, Zhang M, Gong Y, et al. Safe multi-agent deep reinforcement learning for real-time decentralized control of inverter based renewable energy resources considering communication delay. Applied Energy. 2023, 349: 121648. doi: 10.1016/j.apenergy.2023.121648
Li Y, Bu F, Li Y, et al. Optimal scheduling of island integrated energy systems considering multi-uncertainties and hydrothermal simultaneous transmission: A deep reinforcement learning approach. Applied Energy. 2023, 333: 120540. doi: 10.1016/j.apenergy.2022.120540
Zhong S, Wang X, Zhao J, et al. Deep reinforcement learning framework for dynamic pricing demand response of regenerative electric heating. Applied Energy. 2021, 288: 116623. doi: 10.1016/j.apenergy.2021.116623
Li J, Yu T, Zhang X. Coordinated load frequency control of multi-area integrated energy system using multi-agent deep reinforcement learning. Applied Energy. 2022, 306: 117900. doi: 10.1016/j.apenergy.2021.117900
Ye Y, Qiu D, Wu X, et al. Model-Free Real-Time Autonomous Control for a Residential Multi-Energy System Using Deep Reinforcement Learning. IEEE Transactions on Smart Grid. 2020, 11(4): 3068-3082. doi: 10.1109/tsg.2020.2976771
Zhou S, Hu Z, Gu W, et al. Combined heat and power system intelligent economic dispatch: A deep reinforcement learning approach. International Journal of Electrical Power & Energy Systems. 2020, 120: 106016. doi: 10.1016/j.ijepes.2020.106016
Dong Y, Zhang H, Wang C, et al. Soft actor-critic DRL algorithm for interval optimal dispatch of integrated energy systems with uncertainty in demand response and renewable energy. Engineering Applications of Artificial Intelligence. 2024, 127: 107230. doi: 10.1016/j.engappai.2023.107230
Yun L, Wang D, Li L. Explainable multi-agent deep reinforcement learning for real-time demand response towards sustainable manufacturing. Applied Energy. 2023, 347: 121324. doi: 10.1016/j.apenergy.2023.121324
Xie J, Ajagekar A, You F. Multi-Agent attention-based deep reinforcement learning for demand response in grid-responsive buildings. Applied Energy. 2023, 342: 121162. doi: 10.1016/j.apenergy.2023.121162
##submission.downloads##
已出版
文章引用
期
栏目
执照
版权声明
##submission.license.cc.by4.footer##
