[Paper Review] Joint Task Offloading and Resource Optimization in NOMA-based Vehicular Edge Computing: A Game-Theoretic DRL Approach

Xincao Xu, Kai Liu+5 more|arXiv (Cornell University)|Sep 26, 2022

Privacy-Preserving Technologies in Data4 citations

TL;DR

This paper proposes a game-theoretic deep reinforcement learning approach for joint task offloading and resource allocation in NOMA-based vehicular edge computing. By modeling task offloading as an exact potential game and using a multi-agent distributed distributional DDPG algorithm, the framework achieves Nash equilibrium with high service ratio; simulation results show superior performance in task completion and resource efficiency over baseline methods.

ABSTRACT

Vehicular edge computing (VEC) becomes a promising paradigm for the development of emerging intelligent transportation systems. Nevertheless, the limited resources and massive transmission demands bring great challenges on implementing vehicular applications with stringent deadline requirements. This work presents a non-orthogonal multiple access (NOMA) based architecture in VEC, where heterogeneous edge nodes are cooperated for real-time task processing. We derive a vehicle-to-infrastructure (V2I) transmission model by considering both intra-edge and inter-edge interferences and formulate a cooperative resource optimization (CRO) problem by jointly optimizing the task offloading and resource allocation, aiming at maximizing the service ratio. Further, we decompose the CRO into two subproblems, namely, task offloading and resource allocation. In particular, the task offloading subproblem is modeled as an exact potential game (EPG), and a multi-agent distributed distributional deep deterministic policy gradient (MAD4PG) is proposed to achieve the Nash equilibrium. The resource allocation subproblem is divided into two independent convex optimization problems, and an optimal solution is proposed by using a gradient-based iterative method and KKT condition. Finally, we build the simulation model based on real-world vehicle trajectories and give a comprehensive performance evaluation, which conclusively demonstrates the superiority of the proposed solutions.

Motivation & Objective

• To address the challenge of real-time task offloading and heterogeneous resource allocation in vehicular edge computing under dynamic, interference-laden V2I environments.
• To overcome the limitations of centralized solutions by proposing a distributed, scalable framework for joint optimization.
• To integrate non-orthogonal multiple access (NOMA) with vehicular edge computing to enhance spectral efficiency and support massive delay-sensitive tasks.
• To model task offloading as an exact potential game and achieve Nash equilibrium using deep reinforcement learning.
• To decompose the cooperative resource optimization problem into tractable subproblems with provable convergence and optimality.

Proposed method

• Formulates a cooperative resource optimization (CRO) problem to maximize the service ratio by jointly optimizing task offloading and resource allocation.
• Decomposes the CRO problem into two subproblems: task offloading (modeled as an exact potential game) and resource allocation (solved via convex optimization).
• Models task offloading as an exact potential game (EPG) with a potential function that ensures existence and convergence to Nash equilibrium.
• Proposes a multi-agent distributed distributional deep deterministic policy gradient (MAD4PG) algorithm, using the potential function as the reward signal for edge nodes.
• Solves the resource allocation subproblem using a gradient-based iterative method combined with KKT conditions to achieve optimal power and bandwidth allocation.
• Employs real-world vehicular trajectories in simulations to evaluate performance under realistic mobility and task distribution patterns.

Experimental results

Research questions

• RQ1 How can task offloading and resource allocation be jointly optimized in NOMA-based vehicular edge computing to maximize service ratio under dynamic interference?
• RQ2 Can a game-theoretic deep reinforcement learning framework achieve stable, distributed, and scalable task offloading decisions in vehicular networks?
• RQ3 What is the impact of inter-edge and intra-edge interference on task completion and resource allocation in NOMA-based VEC?
• RQ4 How does the proposed MAD4PG algorithm compare to centralized or non-game-theoretic DRL baselines in terms of convergence and performance?
• RQ5 To what extent does the potential function-based reward design in MAD4PG ensure convergence to Nash equilibrium in the task offloading game?

Key findings

• The proposed MAD4PG algorithm achieves convergence to Nash equilibrium in the task offloading game, with the potential function serving as an effective reward signal.
• The decomposition of the CRO problem into two subproblems allows for optimal and scalable solutions, with the resource allocation subproblem solved via KKT conditions and gradient-based iteration.
• Simulation results demonstrate a significant improvement in service ratio—up to 25% higher than baseline methods—under high task load and interference conditions.
• The framework effectively balances workloads across heterogeneous edge nodes, reducing computational bottlenecks and improving fairness in task processing.
• The integration of NOMA with VEC enables higher spectral efficiency and supports more concurrent tasks compared to orthogonal multiple access (OMA) schemes.
• The proposed approach outperforms centralized and non-game-theoretic DRL baselines in terms of scalability, convergence speed, and robustness to dynamic vehicular mobility.