[论文解读] Recent Advances of End-to-End Video Coding Technologies for AVS Standard Development

Xihua Sheng, Xiongzhuang Liang+9 more|arXiv (Cornell University)|Jan 31, 2026

Video Coding and Compression Technologies被引用 0

一句话总结

该论文综述了 AVS-EEM，一种在严格复杂性限制下的端到端智能视频编码框架，并报告在标准测试条件下，最新版 AVS-EEM v9.2 相较于 AVS3 基准在 BD-rate 上有降低。

ABSTRACT

Video coding standards are essential to enable the interoperability and widespread adoption of efficient video compression technologies. In pursuit of greater video compression efficiency, the AVS video coding working group launched the standardization exploration of end-to-end intelligent video coding, establishing the AVS End-to-End Intelligent Video Coding Exploration Model (AVS-EEM) project. A core design principle of AVS-EEM is its focus on practical deployment, featuring inherently low computational complexity and requiring strict adherence to the common test conditions of conventional video coding. This paper details the development history of AVS-EEM and provides a systematic introduction to its key technical framework, covering model architectures, training strategies, and inference optimizations. These innovations have collectively driven the project's rapid performance evolution, enabling continuous and significant gains under strict complexity constraints. Through over two years of iterative refinement and collaborative effort, the coding performance of AVS-EEM has seen substantial improvement. Experimental results demonstrate that its latest model achieves superior compression efficiency compared to the conventional AVS3 reference software, marking a significant step toward a deployable intelligent video coding standard.

研究动机与目标

• Explain the development history of AVS end-to-end intelligent video coding (AVS-EEM).
• Describe the AVS-EEM framework, architectures, training strategies, and inference optimizations.
• Present compression performance and complexity analysis against the conventional AVS3 anchor.
• Highlight key techniques that enable performance gains under fixed complexity constraints.
• Discuss future directions for AVS-EEM within the AVS standardization effort.

提出的方法

• Present AVS-EEM architecture with a motion branch and a residual branch operating in a learnable end-to-end framework.
• Describe original-domain downsampled motion estimation to reduce computation.
• Explain feature-domain group-wise motion alignment and content/motion conditioned motion compression.
• Detail temporal context mining with reference confidence coefficients for multi-scale residual coding.
• Introduce multi-scale temporal context-assisted feature-domain residual compression and residual checkerboard autoregressive entropy modeling.
• Outline training strategies including progressive, hierarchical quality-based, and multi-frame cascaded training.

实验结果

研究问题

• RQ1 Can end-to-end neural coding under strict complexity constraints achieve competitive compression efficiency for AVS with respect to the AVS3 standard under common test conditions?
• RQ2 What architectural and training strategies enable substantial BD-rate improvements while respecting predefined encoding/decoding complexity limits?
• RQ3 How do motion and residual branches interact through temporal contexts to improve end-to-end coding efficiency?
• RQ4 What are the trade-offs between coding performance and computational complexity across AVS-EEM versions?
• RQ5 How does AVS-EEM compare to other end-to-end and machine-vision–oriented standards in terms of practicality and deployability?

主要发现

• AVS-EEM shows substantial evolution from v0.1 to v9.2, moving from large BD-rate gaps to BD-rate reductions across Y, U, and V components under low-delay YUV420 conditions.
• Latest AVS-EEM v9.2 achieves BD-rate reductions of –4.14% (Y), –9.58% (U), and –24.72% (V) relative to the AVS3 anchor under standard conditions.
• Initial models faced significant gaps (e.g., BD-rate increases of 201.37% for Y in v0.1) but converged to measurable gains within a framework constrained by 300 KMAC/pixel encoding and 200 KMAC/pixel decoding limits.
• The AVS-EEM framework maintains a two-branch (motion and residual) architecture with end-to-end rate-distortion optimization, enabling practical deployment.
• Progress relies on innovations such as content/motion feature conditioning, feature-domain alignment, temporal context mining, hierarchical quality scaling, and checkerboard autoregressive entropy modeling.
• Experimental results demonstrate superior compression efficiency of the latest AVS-EEM over the conventional AVS3 reference software while adhering to the complexity constraints.