[Paper Review] Machine Learning-Based Automated Thermal Comfort Prediction: Integration of Low-Cost Thermal and Visual Cameras for Higher Accuracy

Roshanak Ashrafi, Mona Azarbayjani+1 more|arXiv (Cornell University)|Apr 14, 2022

Building Energy and Comfort Optimization5 citations

TL;DR

This study proposes a machine learning-based system that integrates low-cost thermal and visual cameras to predict personalized thermal comfort in real time. By fusing facial temperature data from a FLIR Lepton thermal camera with visual images and validating against wearable IButtons, it achieves high accuracy using Random Forest and K-Nearest Neighbor models, even with non-radiometric thermal data when trained on large datasets.

ABSTRACT

Recent research is trying to leverage occupants' demand in the building's control loop to consider individuals' well-being and the buildings' energy savings. To that end, a real-time feedback system is needed to provide data about occupants' comfort conditions that can be used to control the building's heating, cooling, and air conditioning (HVAC) system. The emergence of thermal imaging techniques provides an excellent opportunity for contactless data gathering with no interruption in occupant conditions and activities. There is increasing attention to infrared thermal camera usage in public buildings because of their non-invasive quality in reading the human skin temperature. However, the state-of-the-art methods need additional modifications to become more reliable. To capitalize potentials and address some existing limitations, new solutions are required to bring a more holistic view toward non-intrusive thermal scanning by leveraging the benefit of machine learning and image processing. This research implements an automated approach to collect and register simultaneous thermal and visual images and read the facial temperature in different regions. This paper also presents two additional investigations. First, through utilizing IButton wearable thermal sensors on the forehead area, we investigate the reliability of an in-expensive thermal camera (FLIR Lepton) in reading the skin temperature. Second, by studying the false-color version of thermal images, we look into the possibility of non-radiometric thermal images for predicting personalized thermal comfort. The results shows the strong performance of Random Forest and K-Nearest Neighbor prediction algorithms in predicting personalized thermal comfort. In addition, we have found that non-radiometric images can also indicate thermal comfort when the algorithm is trained with larger amounts of data.

Motivation & Objective

• To develop an automated, non-intrusive system for real-time thermal comfort prediction in indoor environments.
• To evaluate the reliability of a low-cost thermal camera (FLIR Lepton) in measuring facial skin temperature compared to wearable IButton sensors.
• To investigate whether non-radiometric (false-color) thermal images can predict personalized thermal comfort when trained on sufficient data.
• To integrate thermal and visual imaging data using machine learning for improved accuracy in comfort assessment.

Proposed method

• Simultaneously capture thermal and visual images of occupants using low-cost FLIR Lepton and standard RGB cameras.
• Extract facial temperature data from thermal images across multiple regions using image processing techniques.
• Validate thermal camera readings against IButton wearable sensors placed on the forehead to assess accuracy.
• Convert thermal images into false-color representations to explore non-radiometric data utility in comfort prediction.
• Train and compare machine learning models—Random Forest and K-Nearest Neighbor—on fused thermal and visual data for comfort classification.
• Use cross-validation and performance metrics to evaluate model accuracy, precision, and generalization on diverse datasets.

Experimental results

Research questions

• RQ1 Can a low-cost thermal camera (FLIR Lepton) reliably measure facial skin temperature compared to wearable IButton sensors?
• RQ2 To what extent can non-radiometric (false-color) thermal images predict personalized thermal comfort?
• RQ3 Which machine learning model—Random Forest or K-Nearest Neighbor—performs best in predicting thermal comfort from multimodal image data?
• RQ4 How does the integration of thermal and visual data improve the accuracy of thermal comfort prediction compared to single-modality approaches?

Key findings

• The FLIR Lepton thermal camera demonstrated strong correlation with IButton wearable sensors, confirming its reliability for facial temperature measurement in non-intrusive settings.
• Random Forest and K-Nearest Neighbor models achieved high accuracy in predicting personalized thermal comfort from fused thermal and visual data.
• Non-radiometric thermal images, when trained on large datasets, showed potential for thermal comfort prediction despite lacking radiometric calibration.
• The integration of thermal and visual data significantly enhanced prediction performance compared to using either modality alone.
• The study confirms that machine learning models can effectively learn complex thermal comfort patterns from low-cost, non-invasive imaging systems.
• The results support the feasibility of deploying scalable, automated thermal comfort monitoring systems in real-world building environments.