[Paper Review] Optimization of Image Transmission in a Cooperative Semantic Communication Networks

Wenjing Zhang, Yining Wang+3 more|arXiv (Cornell University)|Jan 1, 2023

Brain Tumor Detection and Classification4 citations

TL;DR

This paper proposes a cooperative semantic communication framework for image transmission using scene graphs to represent image semantics, introducing an image-to-graph semantic similarity (ISS) metric for performance evaluation. It formulates a resource allocation problem to minimize transmission latency under ISS constraints and proposes a value decomposition-based entropy-maximized multi-agent reinforcement learning (VD-ERL) algorithm that enables distributed, cooperative RB allocation, reducing delay by up to 16.1% and improving convergence by 100% compared to baselines.

ABSTRACT

In this paper, a semantic communication framework for image transmission is developed. In the investigated framework, a set of servers cooperatively transmit images to a set of users utilizing semantic communication techniques. To evaluate the performance of studied semantic communication system, a multimodal metric is proposed to measure the correlation between the extracted semantic information and the original image. To meet the ISS requirement of each user, each server must jointly determine the semantic information to be transmitted and the resource blocks (RBs) used for semantic information transmission. We formulate this problem as an optimization problem aiming to minimize each server's transmission latency while reaching the ISS requirement. To solve this problem, a value decomposition based entropy-maximized multi-agent reinforcement learning (RL) is proposed, which enables servers to coordinate for training and execute RB allocation in a distributed manner to approach to a globally optimal performance with less training iterations. Compared to traditional multi-agent RL, the proposed RL improves the valuable action exploration of servers and the probability of finding a globally optimal RB allocation policy based on local observation. Simulation results show that the proposed algorithm can reduce the transmission delay by up to 16.1% compared to traditional multi-agent RL.

Motivation & Objective

• To develop a cooperative semantic communication framework for efficient image transmission using semantic information instead of raw data.
• To design a multimodal metric, image-to-graph semantic similarity (ISS), to evaluate the fidelity of transmitted semantic information relative to the original image.
• To jointly optimize semantic information selection and resource block (RB) allocation across multiple servers under interference constraints.
• To address the challenge of co-channel interference in multi-server, multi-user wireless networks through cooperative, distributed decision-making.
• To improve convergence speed and delay performance over traditional multi-agent RL in dynamic wireless semantic communication environments.

Proposed method

• Models image semantics as scene graphs composed of objects and relationships, enabling human-interpretable semantic representation.
• Introduces the image-to-graph semantic similarity (ISS) metric to quantify the correlation between original images and transmitted semantic information.
• Formulates a joint optimization problem to minimize average transmission latency while satisfying per-user ISS requirements.
• Proposes a value decomposition-based entropy-maximized multi-agent reinforcement learning (VD-ERL) algorithm for distributed, cooperative RB allocation.
• Uses entropy maximization to enhance exploration and value decomposition to enable global policy coordination from local observations.
• Employs a centralized training with decentralized execution (CTDE) paradigm to balance coordination and scalability.

Experimental results

Research questions

• RQ1 How can image semantics be effectively represented in a human-interpretable form suitable for wireless transmission?
• RQ2 What metric can accurately evaluate the semantic fidelity of transmitted image information compared to the original image?
• RQ3 How can resource blocks be jointly allocated across multiple cooperating servers to minimize transmission latency under interference constraints?
• RQ4 Can a multi-agent RL framework with value decomposition and entropy maximization outperform traditional MARL in cooperative semantic communication?
• RQ5 To what extent does the proposed VD-ERL algorithm improve convergence speed and delay reduction in a cooperative semantic image transmission system?

Key findings

• The proposed VD-ERL algorithm reduces average transmission latency by up to 16.1% compared to traditional multi-agent RL baselines.
• The algorithm improves convergence speed by up to 100% compared to conventional multi-agent RL methods.
• The image-to-graph semantic similarity (ISS) metric effectively correlates transmitted semantic information with original image content, with higher ISS achieved by transmitting higher-score semantic triples.
• Semantic triples with higher scores, such as "woman holding racket" (score: 0.1351), are prioritized and more critical for semantic fidelity.
• The VD-ERL algorithm enables cooperative RB allocation, avoiding conflicts such as multi-server allocation to the same user, unlike independent DQN baselines.
• Simulation results confirm that ISS increases monotonically with the number of transmitted semantic triples, validating the metric's consistency and effectiveness.