[论文解读] Combining Visible Light and Infrared Imaging for Efficient Detection of Respiratory Infections such as COVID-19 on Portable Device

Zheng Jiang, Menghan Hu+5 more|arXiv (Cornell University)|Apr 15, 2020

Non-Invasive Vital Sign Monitoring参考文献 29被引用 36

一句话总结

本文提出一种便携式双RGB-热成像设备和 BiGRU-AT 分类器，通过分析带口罩个体的呼吸模式来筛查呼吸系统感染，在真实世界数据上达到 83.69% 的准确率。

ABSTRACT

Coronavirus Disease 2019 (COVID-19) has become a serious global epidemic in the past few months and caused huge loss to human society worldwide. For such a large-scale epidemic, early detection and isolation of potential virus carriers is essential to curb the spread of the epidemic. Recent studies have shown that one important feature of COVID-19 is the abnormal respiratory status caused by viral infections. During the epidemic, many people tend to wear masks to reduce the risk of getting sick. Therefore, in this paper, we propose a portable non-contact method to screen the health condition of people wearing masks through analysis of the respiratory characteristics. The device mainly consists of a FLIR one thermal camera and an Android phone. This may help identify those potential patients of COVID-19 under practical scenarios such as pre-inspection in schools and hospitals. In this work, we perform the health screening through the combination of the RGB and thermal videos obtained from the dual-mode camera and deep learning architecture.We first accomplish a respiratory data capture technique for people wearing masks by using face recognition. Then, a bidirectional GRU neural network with attention mechanism is applied to the respiratory data to obtain the health screening result. The results of validation experiments show that our model can identify the health status on respiratory with the accuracy of 83.7\% on the real-world dataset. The abnormal respiratory data and part of normal respiratory data are collected from Ruijin Hospital Affiliated to The Shanghai Jiao Tong University Medical School. Other normal respiratory data are obtained from healthy people around our researchers. This work demonstrates that the proposed portable and intelligent health screening device can be used as a pre-scan method for respiratory infections, which may help fight the current COVID-19 epidemic.

研究动机与目标

• 目标实现非接触、便携式筛查呼吸系统感染，如 COVID-19.
• 使用双模成像（RGB 与热成像）从带口罩的脸部提取呼吸数据。
• 提出深度学习分类器，从提取的数据区分正常与异常呼吸。
• 验证在口罩类型、距离和相机角度上的鲁棒性。
• 提供适用于学校、医院和社区环境的预扫描方法。

提出的方法

• 使用 FLIR One 相机和 Android 手机捕获 RGB 和热视频。
• 通过 RGB 人脸检测检测带口罩区域并映射到热图像以选择 ROI。
• 通过最大化温度变化方差来跟踪 ROI 中的遮罩区域以提取呼吸信号。
• 使用带注意力机制的 BiGRU 神经网络（BiGRU-AT）将呼吸分类为正常或异常。
• 将 BiGRU-AT 与 GRU-AT、BiLSTM-AT 以及 LSTM 在准确率、精确度、召回率和 F1 上进行比较。

实验结果

研究问题

• RQ1 可携式双模相机是否能可靠地从佩戴口罩的人身上提取呼吸数据用于健康筛查？
• RQ2 BiGRU-AT 模型是否能够有效地从热导出时间序列数据区分正常与异常呼吸模式？
• RQ3 方法对口罩类型、距离和相机角度的变化有多鲁棒？
• RQ4 在此任务上不同循环结构的比较性能如何？

主要发现

模型	准确率	精确度	召回率	F1%
BiGRU-AT	83.69%	90.23%	79.65%	84.61%
GRU-AT	79.31%	90.62%	74.24%	81.62%
BiLSTM-AT	74.46%	87.50%	69.78%	77.64%
LSTM	71.98%	72.07%	71.98%	71.97%

• BiGRU-AT 模型在测试集对健康 vs 异常呼吸的分类上达到 83.69% 的准确率。
• BiGRU-AT 还达到了精确度 90.23%、召回率 79.65% 和 F1 分数 84.61%。
• 在所测试的模型中，BiGRU-AT 在所有指标上都优于 GRU-AT、BiLSTM-AT 和 LSTM。
• GRU-AT、BiLSTM-AT 与 LSTM 在准确率较低且精确/召回各异，总体上 LSTM 表现最差。
• 该研究展示了一种便携式、非接触的预扫描方法，可在各种真实世界场景中使用，在测试范围内对口罩类型和距离具有鲁棒性。