[论文解读] Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognition

Yuxin Chen, Ziqi Zhang+4 more|arXiv (Cornell University)|Jul 26, 2021

Human Pose and Action Recognition参考文献 33被引用 49

一句话总结

CTR-GC 动态通过细化共享拓扑来学习通道特定拓扑，从而在骨架动作识别中实现更灵活的特征聚合；结合成 CTR-GCN 时，在 NTU RGB+D、NTU RGB+D 120 和 NW-UCLA 上达到最先进的结果。

ABSTRACT

Graph convolutional networks (GCNs) have been widely used and achieved remarkable results in skeleton-based action recognition. In GCNs, graph topology dominates feature aggregation and therefore is the key to extracting representative features. In this work, we propose a novel Channel-wise Topology Refinement Graph Convolution (CTR-GC) to dynamically learn different topologies and effectively aggregate joint features in different channels for skeleton-based action recognition. The proposed CTR-GC models channel-wise topologies through learning a shared topology as a generic prior for all channels and refining it with channel-specific correlations for each channel. Our refinement method introduces few extra parameters and significantly reduces the difficulty of modeling channel-wise topologies. Furthermore, via reformulating graph convolutions into a unified form, we find that CTR-GC relaxes strict constraints of graph convolutions, leading to stronger representation capability. Combining CTR-GC with temporal modeling modules, we develop a powerful graph convolutional network named CTR-GCN which notably outperforms state-of-the-art methods on the NTU RGB+D, NTU RGB+D 120, and NW-UCLA datasets.

研究动机与目标

• 通过考虑运动特征的通道级变化，激发对骨架基础动作识别中改进拓扑建模的研究。
• 提出 CTR-GC，以对每个通道使用通道特定相关性来对共享拓扑进行细化。
• 证明 CTR-GC 可以集成到 CTR-GCN 骨干网中以实现更优性能。

提出的方法

• 将通道级拓扑建模为在共享邻接矩阵 A 上通过通道特定相关性 Q 进行细化以得到 R。
• 利用相关建模函数 M1 或 M2 从关节特征的降维表示中计算通道级关系。
• 通过 R = A + alpha * Q 将 A 精细化为 R，并在逐通道图上执行通道级聚合以得到 Z。
• 将四种图卷积风格统一为一个共同的公式，以论证 CTR-GC 放宽了某些约束并增强表示能力。
• 构建具有 CTR-GC 块、时序建模模块和残差连接的 CTR-GCN，用于骨架序列动作识别。

Figure 1: Channel-wise topology refinement. Lines of different colors correspond to topologies in different channels and the thickness of lines indicates the correlation strength between joints.

实验结果

研究问题

• RQ1 共享拓扑的通道级细化能否提升图卷积在骨架基础动作识别中的表达能力？
• RQ2 将通道特定相关性（Q）与共享拓扑（A）结合，是否能带来相比静态/可共享拓扑的可测提升？
• RQ3 在 NTU RGB+D、NTU RGB+D 120 和 NW-UCLA 数据集上，CTR-GCN 的表现与最先进方法相比如何？

主要发现

• CTR-GC 在参数和计算量可比的情况下显著超越其他图卷积。
• CTR-GCN 在 NTU RGB+D、NTU RGB+D 120 和 NW-UCLA 数据集上超越了最先进的方法。
• 消融研究表明，移除通道特定相关性（Q）或共享拓扑细化（A）会降低性能，突显通道级拓扑细化的重要性。
• 相关建模函数（M1、M2）以及降维比率 r 的各种配置始终优于基线，表明 CTR-GC 的鲁棒性和有效性。

Figure 2: Framework of the proposed channel-wise topology refinement graph convolution. The channel-wise topology modeling refines the trainable shared topology with inferred channel-specific correlations. The feature transformation aims at transforming input features into high-level representations