[Paper Review] Combining Visible Light and Infrared Imaging for Efficient Detection of Respiratory Infections such as COVID-19 on Portable Device

Zheng Jiang, Menghan Hu+5 more|arXiv (Cornell University)|Apr 15, 2020

Non-Invasive Vital Sign MonitoringEngineering29 references36 citations

TL;DR

This paper presents a portable dual RGB-thermal imaging device and a BiGRU-AT classifier to screen respiratory infections by analyzing breathing patterns from masked individuals, achieving 83.69% accuracy on real-world data.

ABSTRACT

Coronavirus Disease 2019 (COVID-19) has become a serious global epidemic in the past few months and caused huge loss to human society worldwide. For such a large-scale epidemic, early detection and isolation of potential virus carriers is essential to curb the spread of the epidemic. Recent studies have shown that one important feature of COVID-19 is the abnormal respiratory status caused by viral infections. During the epidemic, many people tend to wear masks to reduce the risk of getting sick. Therefore, in this paper, we propose a portable non-contact method to screen the health condition of people wearing masks through analysis of the respiratory characteristics. The device mainly consists of a FLIR one thermal camera and an Android phone. This may help identify those potential patients of COVID-19 under practical scenarios such as pre-inspection in schools and hospitals. In this work, we perform the health screening through the combination of the RGB and thermal videos obtained from the dual-mode camera and deep learning architecture.We first accomplish a respiratory data capture technique for people wearing masks by using face recognition. Then, a bidirectional GRU neural network with attention mechanism is applied to the respiratory data to obtain the health screening result. The results of validation experiments show that our model can identify the health status on respiratory with the accuracy of 83.7\% on the real-world dataset. The abnormal respiratory data and part of normal respiratory data are collected from Ruijin Hospital Affiliated to The Shanghai Jiao Tong University Medical School. Other normal respiratory data are obtained from healthy people around our researchers. This work demonstrates that the proposed portable and intelligent health screening device can be used as a pre-scan method for respiratory infections, which may help fight the current COVID-19 epidemic.

Motivation & Objective

• Aim to enable non-contact, portable screening of respiratory infections such as COVID-19.
• Develop respiratory data extraction from masked faces using dual-mode imaging (RGB and thermal).
• Propose a deep learning classifier to distinguish normal vs abnormal respiration from extracted data.
• Validate robustness across mask types, distances, and camera angles.
• Provide a pre-scan method suitable for schools, hospitals, and community settings.

Proposed method

• Capture RGB and thermal videos with a FLIR One camera and Android phone.
• Detect masked regions via RGB face detection and map to thermal images for ROI selection.
• Track an ROI within masked regions by maximizing temperature-change variance to extract breathing signals.
• Classify respiration as normal or abnormal using a BiGRU neural network with an attention mechanism (BiGRU-AT).
• Compare BiGRU-AT to GRU-AT, BiLSTM-AT, and LSTM on accuracy, precision, recall, and F1.

Experimental results

Research questions

• RQ1 Can a portable dual-mode camera reliably extract respiration data from people wearing masks for health screening?
• RQ2 Is a BiGRU-AT model effective at distinguishing normal versus abnormal respiratory patterns from thermal-derived time series data?
• RQ3 How robust is the method to variations in mask type, distance, and camera angle?
• RQ4 What is the comparative performance of different recurrent architectures on this task?

Key findings

• The BiGRU-AT model achieved 83.69% accuracy on the test set for classifying healthy vs abnormal respiration.
• BiGRU-AT also achieved precision 90.23%, recall 79.65%, and F1 score 84.61%.
• Among models tested, BiGRU-AT outperformed GRU-AT, BiLSTM-AT, and LSTM on all metrics.
• GRU-AT, BiLSTM-AT, and LSTM showed lower accuracy and varied precision/recall, with LSTM performing worst overall.
• The study demonstrates a portable, non-contact pre-scan method usable in various real-world settings, with robustness across mask types and distances within tested ranges.