[Paper Review] Transfer Learning for Non-Intrusive Load Monitoring

Michele D'Incecco, Stefano Squartini+1 more|arXiv (Cornell University)|Feb 23, 2019

Smart Grid Energy Management42 references17 citations

TL;DR

This paper proposes transfer learning strategies—appliance transfer learning (ATL) and cross-domain transfer learning (CTL)—to improve non-intrusive load monitoring (NILM) using sequence-to-point (seq2point) deep learning. It demonstrates that pre-trained convolutional neural network (CNN) features from complex appliances like washing machines can be transferred to simpler appliances like kettles, and that seq2point models trained on one data domain (e.g., REFIT) can generalize to others (e.g., UK-DALE, REDD) with or without fine-tuning, significantly reducing training costs and enabling scalable, universal NILM systems.

ABSTRACT

Non-intrusive load monitoring (NILM) is a technique to recover source appliances from only the recorded mains in a household. NILM is unidentifiable and thus a challenge problem because the inferred power value of an appliance given only the mains could not be unique. To mitigate the unidentifiable problem, various methods incorporating domain knowledge into NILM have been proposed and shown effective experimentally. Recently, among these methods, deep neural networks are shown performing best. Arguably, the recently proposed sequence-to-point (seq2point) learning is promising for NILM. However, the results were only carried out on the same data domain. It is not clear if the method could be generalised or transferred to different domains, e.g., the test data were drawn from a different country comparing to the training data. We address this issue in the paper, and two transfer learning schemes are proposed, i.e., appliance transfer learning (ATL) and cross-domain transfer learning (CTL). For ATL, our results show that the latent features learnt by a `complex' appliance, e.g., washing machine, can be transferred to a `simple' appliance, e.g., kettle. For CTL, our conclusion is that the seq2point learning is transferable. Precisely, when the training and test data are in a similar domain, seq2point learning can be directly applied to the test data without fine tuning; when the training and test data are in different domains, seq2point learning needs fine tuning before applying to the test data. Interestingly, we show that only the fully connected layers need fine tuning for transfer learning. Source code can be found at https://github.com/MingjunZhong/transferNILM.

Motivation & Objective

• To address the challenge of generalizing non-intrusive load monitoring (NILM) models across different appliances and data domains.
• To investigate whether sequence-to-point (seq2point) learning, previously tested only within the same domain, can be transferred to unseen appliances or different geographical data domains.
• To develop transfer learning strategies—appliance transfer learning (ATL) and cross-domain transfer learning (CTL)—that reduce dependency on labeled data and computational cost.
• To validate the hypothesis that convolutional neural network (CNN) features are invariant across appliances and data domains, enabling transferability.

Proposed method

• Proposes appliance transfer learning (ATL), where CNN layers pre-trained on a complex appliance (e.g., washing machine) are frozen and fine-tuned only on fully connected layers for a different, simpler appliance (e.g., kettle).
• Introduces cross-domain transfer learning (CTL), where models trained on one dataset (e.g., REFIT) are applied to another (e.g., UK-DALE or REDD), with or without fine-tuning the fully connected layers.
• Uses a sequence-to-point deep learning architecture with a CNN backbone to extract latent features from mains power sequences, followed by dense layers for appliance-specific classification.
• Employs early stopping and transfer learning with feature freezing to reduce overfitting and computational cost during fine-tuning.
• Evaluates transferability by training on REFIT and testing on UK-DALE and REDD datasets, comparing performance with and without fine-tuning.
• Validates transfer learning using quantitative metrics such as mean absolute error (MAE) and energy disaggregation accuracy across multiple appliances and domains.

Experimental results

Research questions

• RQ1 Can convolutional neural network (CNN) features learned from one appliance be effectively transferred to another, unseen appliance, such as from a washing machine to a kettle?
• RQ2 Is seq2point learning transferable across different data domains, such as from UK-DALE or REFIT to REDD?
• RQ3 Does fine-tuning only the fully connected layers improve model performance when transferring to a different domain, and when is it necessary?
• RQ4 Under what conditions—similar or different domains—does fine-tuning provide a performance benefit in cross-domain NILM?
• RQ5 Can transfer learning reduce the need for extensive labeled data and computational resources in NILM while maintaining high disaggregation accuracy?

Key findings

• Appliance transfer learning (ATL) successfully transfers features from a complex appliance (e.g., washing machine) to a simpler one (e.g., kettle), achieving high disaggregation accuracy with minimal retraining.
• Cross-domain transfer learning (CTL) shows that seq2point models can generalize across domains: no fine-tuning is needed when training and test domains are similar, but fine-tuning significantly improves performance when domains differ.
• Fine-tuning only the fully connected layers is sufficient for effective transfer learning, preserving the invariant CNN features across appliances and domains.
• The model trained on REFIT generalizes well to UK-DALE and REDD datasets, with mean absolute error (MAE) values of 26.422 and 26.778, respectively, when using transfer learning.
• Transfer learning reduces computational cost and dependency on labeled data, enabling scalable deployment of a single universal NILM model across diverse appliances and geographical regions.
• The results support the hypothesis that CNN-learned features are invariant across appliances and domains, forming a strong foundation for universal, oracle-like NILM systems.