[Paper Review] Personalized Dynamic Pricing Policy for Electric Vehicles: Reinforcement learning approach

Sangjun Bae, Balaźs Kulcsár+1 more|arXiv (Cornell University)|Jan 1, 2024

Electric Vehicles and Infrastructure4 citations

TL;DR

This paper proposes a personalized dynamic pricing (PeDP) policy for fast-electric vehicle charging stations (fast-EVCSs) using deep Q-learning to maximize revenue. By modeling EV users' selfish behavior via a game-theoretic utility function and simulating competition among fast-EVCSs, the approach demonstrates that incorporating waiting time and privacy-preserved user data significantly boosts revenue, while revealing risks of AI misuse in exploiting shared information.

ABSTRACT

With the increasing number of fast-electric vehicle charging stations (fast-EVCSs) and the popularization of information technology, electricity price competition between fast-EVCSs is highly expected, in which the utilization of public and/or privacy-preserved information will play a crucial role. Self-interest electric vehicle (EV) users, on the other hand, try to select a fast-EVCS for charging in a way to maximize their utilities based on electricity price, estimated waiting time, and their state of charge. While existing studies have largely focused on finding equilibrium prices, this study proposes a personalized dynamic pricing policy (PeDP) for a fast-EVCS to maximize revenue using a reinforcement learning (RL) approach. We first propose a multiple fast-EVCSs competing simulation environment to model the selfish behavior of EV users using a game-based charging station selection model with a monetary utility function. In the environment, we propose a Q-learning-based PeDP to maximize fast-EVCS' revenue. Through numerical simulations based on the environment: (1) we identify the importance of waiting time in the EV charging market by comparing the classic Bertrand competition model with the proposed PeDP for fast-EVCSs (from the system perspective); (2) we evaluate the performance of the proposed PeDP and analyze the effects of the information on the policy (from the service provider perspective); and (3) it can be seen that privacy-preserved information sharing can be misused by artificial intelligence-based PeDP in a certain situation in the EV charging market (from the customer perspective).

Motivation & Objective

• To address the revenue maximization challenge for fast-EVCSs in a competitive, dynamic market with self-interested EV users.
• To model EV user behavior using a monetary utility function that accounts for electricity price, waiting time, and state of charge.
• To design a personalized dynamic pricing policy (PeDP) using deep Q-learning that adapts to real-time user and station conditions.
• To evaluate the impact of information availability—public vs. privacy-preserved data—on pricing performance and revenue.
• To investigate the potential misuse of privacy-preserved data by AI-driven pricing policies from a customer perspective.

Proposed method

• Developed a multi-fast-EVCS simulation environment based on a game-theoretic charging station selection model with a monetary utility function.
• Formulated the EV user's utility as a function of electricity price, waiting time, state of charge (SOC), and energy efficiency.
• Implemented a deep Q-learning agent to learn personalized pricing policies that maximize fast-EVCS revenue over time.
• Used a replay memory and target network to stabilize training, with a loss function minimizing the difference between predicted and target Q-values.
• Incorporated an ε-greedy exploration strategy to balance exploitation and exploration in the learning process.
• Evaluated the PeDP against a classic Bertrand competition model to compare revenue and market dynamics.

Experimental results

Research questions

• RQ1 How does the inclusion of waiting time in the utility function affect market equilibrium and revenue in fast-EVCS competition?
• RQ2 To what extent does personalized dynamic pricing improve revenue compared to public pricing in a competitive EV charging market?
• RQ3 How does the availability of privacy-preserved user information influence the performance of an AI-driven pricing policy?
• RQ4 Can AI-based PeDP policies exploit shared information in ways that disadvantage individual EV users?
• RQ5 Under what conditions does the system converge to a stable pricing and user allocation outcome?

Key findings

• The proposed PeDP significantly outperforms the classic Bertrand competition model in revenue generation, especially when waiting time is factored into user utility.
• Incorporating estimated waiting time into the utility function leads to more efficient market outcomes and higher revenue for fast-EVCSs.
• The PeDP achieves higher revenue when using privacy-preserved information, demonstrating the value of user-level data in dynamic pricing.
• The study reveals that AI-based PeDP can potentially misuse privacy-preserved data to strategically disadvantage individual users, raising ethical concerns.
• The system converges to a Nash-stable partition under the SPAO (Single-Peaked-At-One) condition, ensuring stable user groupings and predictable behavior.
• The deep Q-learning agent successfully learns optimal pricing strategies through interaction with the simulated environment, achieving stable revenue growth over time.