[论文解读] Quantitative Survey of the State of the Art in Sign Language Recognition

Oscar Koller|arXiv (Cornell University)|Aug 22, 2020

Hand Gesture Recognition Systems参考文献 278被引用 58

一句话总结

本文对手语识别方法进行了定量综述，突出显示结果中的模态使用情况，表 S6 详细列出 26 种模态组合及按词汇量规模的相对频率。

ABSTRACT

This work presents a meta study covering around 300 published sign language recognition papers with over 400 experimental results. It includes most papers between the start of the field in 1983 and 2020. Additionally, it covers a fine-grained analysis on over 25 studies that have compared their recognition approaches on RWTH-PHOENIX-Weather 2014, the standard benchmark task of the field. Research in the domain of sign language recognition has progressed significantly in the last decade, reaching a point where the task attracts much more attention than ever before. This study compiles the state of the art in a concise way to help advance the field and reveal open questions. Moreover, all of this meta study's source data is made public, easing future work with it and further expansion. The analyzed papers have been manually labeled with a set of categories. The data reveals many insights, such as, among others, shifts in the field from intrusive to non-intrusive capturing, from local to global features and the lack of non-manual parameters included in medium and larger vocabulary recognition systems. Surprisingly, RWTH-PHOENIX-Weather with a vocabulary of 1080 signs represents the only resource for large vocabulary continuous sign language recognition benchmarking world wide.

研究动机与目标

• 评估已发表结果中使用的手语识别模态的分布
• 量化模态选择如何随词汇量（如符号数量）变化
• 识别文献中最常见的手动与非手动参数组合
• 为在手语识别（SLR）实验中选择模态的研究人员提供简明参考

提出的方法

• 收集具有明确词汇范围的已发表手语识别结果
• 按使用的模态组合对每个结果进行分类（Loc, Mov, Shape, Orient, Joints, Fullframe, Depth, Motion）
• 计算在每个词汇范围内每种模态组合的相对频率
• 呈现汇总统计数据（如表 S6），显示模态的流行程度
• 在共享的补充材料中记录任何截断或数据质量问题

实验结果

研究问题

• RQ1 在手语识别研究中，哪些模态组合最常被使用？
• RQ2 随着建模词汇量的增加，模态使用如何变化？
• RQ3 在不同词汇范围内，全帧模态相对于针对性手部/形状/方向模态的相对流行程度如何？
• RQ4 文献中手动参数与非手动参数的使用是否存在显著趋势？

主要发现

• 表 S6 报告了在词汇范围内最常用的 26 种模态组合及其相对频率。
• 结果以相对于同一词汇范围内全部结果的百分比表示。
• 举例说明：建模词汇量超过 1000 个符号的结果中，39% 完全依赖全帧模态，而 7% 依赖手形模态。