[论文解读] Measuring hand use in the home after cervical spinal cord injury using egocentric video

Andrea Bandini, Mehdy Dousty+4 more|arXiv (Cornell University)|Mar 31, 2022

Spinal Cord Injury Research参考文献 43被引用 17

一句话总结

本研究开发并验证了一款基于深度学习的可穿戴自指视频系统，可在家用环境中自动测量四肢瘫痪患者的双手使用情况。研究结果表明，自动提取的指标——互动时间占比（Perc）和每小时互动次数（Num）——与临床对手部功能和独立性的评估显著相关，证实其作为真实世界环境中基于表现的结局指标的有效性。

ABSTRACT

Background: Egocentric video has recently emerged as a potential solution for monitoring hand function in individuals living with tetraplegia in the community, especially for its ability to detect functional use in the home environment. Objective: To develop and validate a wearable vision-based system for measuring hand use in the home among individuals living with tetraplegia. Methods: Several deep learning algorithms for detecting functional hand-object interactions were developed and compared. The most accurate algorithm was used to extract measures of hand function from 65 hours of unscripted video recorded at home by 20 participants with tetraplegia. These measures were: the percentage of interaction time over total recording time (Perc); the average duration of individual interactions (Dur); the number of interactions per hour (Num). To demonstrate the clinical validity of the technology, egocentric measures were correlated with validated clinical assessments of hand function and independence (Graded Redefined Assessment of Strength, Sensibility and Prehension - GRASSP, Upper Extremity Motor Score - UEMS, and Spinal Cord Independent Measure - SCIM). Results: Hand-object interactions were automatically detected with a median F1-score of 0.80 (0.67-0.87). Our results demonstrated that higher UEMS and better prehension were related to greater time spent interacting, whereas higher SCIM and better hand sensation resulted in a higher number of interactions performed during the egocentric video recordings. Conclusions: For the first time, measures of hand function automatically estimated in an unconstrained environment in individuals with tetraplegia have been validated against internationally accepted measures of hand function. Future work will necessitate a formal evaluation of the reliability and responsiveness of the egocentric-based performance measures for hand use.

研究动机与目标

• 开发一种基于视觉的可穿戴系统，用于在个体自然的居家环境中测量四肢瘫痪患者的上肢功能。
• 对比并选择最适合从自指视频中检测功能性手-物交互的深度学习算法。
• 将自动提取的自指视频手部使用指标（Perc、Dur、Num）与公认临床评估（GRASSP、UEMS、SCIM）进行验证。
• 确立基于视觉的指标在真实世界环境中捕捉上肢功能表现的有效性，超越传统临床能力测量的局限。

提出的方法

• 在20名四肢瘫痪参与者居家进行非脚本化日常活动期间，采集其自指视频记录。
• 评估多种深度学习模型在手-物交互检测中的表现，基于F1-score选择性能最佳的模型。
• 提取三项关键手部功能指标：Perc（互动时间占比）、Dur（平均互动时长）和Num（每小时互动次数）。
• 将自指视频指标与临床评估结果进行相关性分析：GRASSP（抓握能力）、UEMS（运动功能）和SCIM（独立性）。
• 采用模块化视频处理流程，以手部定位和交互检测为核心组件。
• 应用计算机视觉技术，通过手-物接触状态推断功能性手部使用情况，并计划进一步扩展至动作识别与抓握类型识别。

实验结果

研究问题

• RQ1 深度学习能否在四肢瘫痪患者居家环境的自指视频记录中准确检测功能性手-物交互？
• RQ2 自动提取的自指视频指标（Perc、Dur、Num）是否与经验证的临床手功能与独立性评估指标显著相关？
• RQ3 哪些临床因素（如运动功能、感觉功能、抓握能力）与特定的自指手部使用指标关联最强？
• RQ4 基于自指视频的指标能否作为真实世界环境中上肢功能的有效、基于表现的结局指标？

主要发现

• 性能最佳的深度学习模型在检测手-物交互方面达到中位F1-score为0.80（IQR：0.67–0.87）。
• UEMS评分越高，Perc值也显著越高（r = 0.55，p = 0.01），表明更好的运动功能与更长的互动时间相关。
• 抓握能力越强（GR-PP），Perc值也越高（r = 0.48，p = 0.03），说明抓握质量提升可增加互动时间。
• SCIM评分更高及手部感觉功能更好者，每小时互动次数（Num）也更多，表明独立性与感觉功能的改善支持更频繁的手部使用。
• 本研究首次在四肢瘫痪患者中，将自指视觉手部使用指标与国际公认的临床评估进行了验证。
• 模块化处理流程为未来集成抓握类型识别与动作检测提供了可能，从而可同时评估手部使用的数量与质量。