[Paper Review] Energy Consumption and Battery Aging Minimization Using a Q-learning Strategy for a Battery/Ultracapacitor Electric Vehicle

Bin Xu, Junzhe Shi+3 more|arXiv (Cornell University)|Oct 27, 2020

Advanced Battery Technologies Research27 references4 citations

TL;DR

This paper proposes a Q-learning-based energy management strategy for battery/ultracapacitor electric vehicles to simultaneously minimize battery degradation and energy consumption. By leveraging reinforcement learning with a severity factor-based degradation model, the method reduces battery aging by 13–20% and extends vehicle range by 1.5–2% compared to a baseline without ultracapacitors.

ABSTRACT

Propulsion system electrification revolution has been undergoing in the automotive industry. The electrified propulsion system improves energy efficiency and reduces the dependence on fossil fuel. However, the batteries of electric vehicles experience degradation process during vehicle operation. Research considering both battery degradation and energy consumption in battery/ supercapacitor electric vehicles is still lacking. This study proposes a Q-learning-based strategy to minimize battery degradation and energy consumption. Besides Q-learning, two heuristic energy management methods are also proposed and optimized using Particle Swarm Optimization algorithm. A vehicle propulsion system model is first presented, where the severity factor battery degradation model is considered and experimentally validated with the help of Genetic Algorithm. In the results analysis, Q-learning is first explained with the optimal policy map after learning. Then, the result from a vehicle without ultracapacitor is used as the baseline, which is compared with the results from the vehicle with ultracapacitor using Q-learning, and two heuristic methods as the energy management strategies. At the learning and validation driving cycles, the results indicate that the Q-learning strategy slows down the battery degradation by 13-20% and increases the vehicle range by 1.5-2% compared with the baseline vehicle without ultracapacitor.

Motivation & Objective

• Address the lack of integrated studies on battery degradation and energy consumption in battery/ultracapacitor electric vehicles.
• Develop an energy management strategy that minimizes both battery aging and energy use during vehicle operation.
• Validate a severity factor-based battery degradation model using experimental data and genetic algorithm optimization.
• Compare the performance of Q-learning against heuristic methods and a baseline system without ultracapacitors.
• Demonstrate the effectiveness of reinforcement learning in improving both vehicle efficiency and battery longevity.

Proposed method

• Formulate a vehicle propulsion system model incorporating a severity factor-based battery degradation model, validated experimentally.
• Implement a Q-learning algorithm to learn optimal energy distribution between battery and ultracapacitor in real time.
• Use Particle Swarm Optimization (PSO) to tune and optimize two heuristic energy management strategies for comparison.
• Define the Q-learning state space based on battery state of charge, power demand, and driving cycle conditions.
• Train the Q-learning agent using simulated driving cycles to learn policies that minimize cumulative cost combining energy use and degradation.
• Validate the learned policy on standard driving cycles to assess performance in terms of energy efficiency and battery health.

Experimental results

Research questions

• RQ1 Can a Q-learning-based energy management strategy effectively reduce both battery degradation and energy consumption in a battery/ultracapacitor electric vehicle?
• RQ2 How does the performance of Q-learning compare to heuristic strategies and a baseline system without ultracapacitors?
• RQ3 To what extent does the inclusion of ultracapacitors, managed via Q-learning, improve vehicle range and battery longevity?
• RQ4 How accurately does the severity factor degradation model reflect real battery aging under dynamic driving conditions?
• RQ5 What is the trade-off between energy efficiency and battery degradation mitigation under different control strategies?

Key findings

• The Q-learning strategy reduced battery degradation by 13–20% compared to the baseline vehicle without ultracapacitors.
• The Q-learning-based system increased vehicle range by 1.5–2% under the same driving cycles, demonstrating improved energy efficiency.
• The severity factor degradation model was experimentally validated and used effectively within the Q-learning framework to reflect real battery aging.
• The Q-learning policy map showed clear convergence to an optimal strategy, with stable and repeatable performance across multiple driving cycles.
• The PSO-optimized heuristic methods performed comparably but did not outperform Q-learning in either degradation reduction or range extension.
• The integration of ultracapacitors with Q-learning-based control significantly improved both energy efficiency and battery longevity.