Showing 1–7 of 7 results for author: Klemenjak, C

Exploring Bayesian Surprise to Prevent Overfitting and to Predict Model Performance in Non-Intrusive Load Monitoring

Authors: Richard Jones, Christoph Klemenjak, Stephen Makonin, Ivan V. Bajic

Abstract: Non-Intrusive Load Monitoring (NILM) is a field of research focused on segregating constituent electrical loads in a system based only on their aggregated signal. Significant computational resources and research time are spent training models, often using as much data as possible, perhaps driven by the preconception that more data equates to more accurate models and better performing algorithms. W… ▽ More Non-Intrusive Load Monitoring (NILM) is a field of research focused on segregating constituent electrical loads in a system based only on their aggregated signal. Significant computational resources and research time are spent training models, often using as much data as possible, perhaps driven by the preconception that more data equates to more accurate models and better performing algorithms. When has enough prior training been done? When has a NILM algorithm encountered new, unseen data? This work applies the notion of Bayesian surprise to answer these questions which are important for both supervised and unsupervised algorithms. We quantify the degree of surprise between the predictive distribution (termed postdictive surprise), as well as the transitional probabilities (termed transitional surprise), before and after a window of observations. We compare the performance of several benchmark NILM algorithms supported by NILMTK, in order to establish a useful threshold on the two combined measures of surprise. We validate the use of transitional surprise by exploring the performance of a popular Hidden Markov Model as a function of surprise threshold. Finally, we explore the use of a surprise threshold as a regularization technique to avoid overfitting in cross-dataset performance. Although the generality of the specific surprise threshold discussed herein may be suspect without further testing, this work provides clear evidence that a point of diminishing returns of model performance with respect to dataset size exists. This has implications for future model development, dataset acquisition, as well as aiding in model flexibility during deployment. △ Less

Submitted 16 September, 2020; originally announced September 2020.

Investigating the Performance Gap between Testing on Real and Denoised Aggregates in Non-Intrusive Load Monitoring

Authors: Christoph Klemenjak, Stephen Makonin, Wilfried Elmenreich

Abstract: Prudent and meaningful performance evaluation of algorithms is essential for the progression of any research field. In the field of Non-Intrusive Load Monitoring (NILM), performance evaluation can be conducted on real-world aggregate signals, provided by smart energy meters or artificial superpositions of individual load signals (i.e., denoised aggregates). It has long been suspected that testing… ▽ More Prudent and meaningful performance evaluation of algorithms is essential for the progression of any research field. In the field of Non-Intrusive Load Monitoring (NILM), performance evaluation can be conducted on real-world aggregate signals, provided by smart energy meters or artificial superpositions of individual load signals (i.e., denoised aggregates). It has long been suspected that testing on these denoised aggregates provides better evaluation results mainly due to the the fact that the signal is less complex. Complexity in real-world aggregate signals increases with the number of unknown/untracked load. Although this is a known performance reporting problem, an investigation in the actual performance gap between real and denoised testing is still pending. In this paper, we examine the performance gap between testing on real-world and denoised aggregates with the aim of bringing clarity into this matter. Starting with an assessment of noise levels in datasets, we find significant differences in test cases. We give broad insights into our evaluation setup comprising three load disaggregation algorithms, two of them relying on neural network architectures. The results presented in this paper, based on studies covering three scenarios with ascending noise levels, show a strong tendency towards load disaggregation algorithms providing significantly better performance on denoised aggregate signals. A closer look into the outcome of our studies reveals that all appliance types could be subject to this phenomenon. We conclude the paper by discussing aspects that could be causing these considerable gaps between real and denoised testing in NILM. △ Less

Submitted 3 October, 2020; v1 submitted 22 August, 2020; originally announced August 2020.

Augmenting an Assisted Living Lab with Non-Intrusive Load Monitoring

Authors: Hafsa Bousbiat, Christoph Klemenjak, Gerhard Leitner, Wilfried Elmenreich

Abstract: The need for reducing our energy consumption footprint and the increasing number of electric devices in today's homes is calling for new solutions that allow users to efficiently manage their energy consumption. Real-time feedback at device level would be of a significant benefit for this application. In addition, the aging population and their wish to be more autonomous have motivated the use of… ▽ More The need for reducing our energy consumption footprint and the increasing number of electric devices in today's homes is calling for new solutions that allow users to efficiently manage their energy consumption. Real-time feedback at device level would be of a significant benefit for this application. In addition, the aging population and their wish to be more autonomous have motivated the use of this same real-time data to indirectly monitor the household's occupants for their safety. By breaking down aggregate power consumption into its components, Non-Intrusive Load Monitoring provides information on individual appliances and their current state of operation. Since no additional metering equipment is required, residents are not confronted with intrusion into their familiar environment. Our work aims to depict an architecture supporting non-intrusive measurement with a smart electricity meter and the handling of these data using an open-source platform that allows to visualize and process real-time data about the total energy consumed. As a case study, we describe a series of measurements from common household devices and show how abnormal behavior can be detected. △ Less

Submitted 13 February, 2020; originally announced February 2020.

Towards Comparability in Non-Intrusive Load Monitoring: On Data and Performance Evaluation

Authors: Christoph Klemenjak, Stephen Makonin, Wilfried Elmenreich

Abstract: Non-Intrusive Load Monitoring (NILM) comprises of a set of techniques that provide insights into the energy consumption of households and industrial facilities. Latest contributions show significant improvements in terms of accuracy and generalisation abilities. Despite all progress made concerning disaggregation techniques, performance evaluation and comparability remains an open research questio… ▽ More Non-Intrusive Load Monitoring (NILM) comprises of a set of techniques that provide insights into the energy consumption of households and industrial facilities. Latest contributions show significant improvements in terms of accuracy and generalisation abilities. Despite all progress made concerning disaggregation techniques, performance evaluation and comparability remains an open research question. The lack of standardisation and consensus on evaluation procedures makes reproducibility and comparability extremely difficult. In this paper, we draw attention to comparability in NILM with a focus on highlighting the considerable differences amongst common energy datasets used to test the performance of algorithms. We divide discussion on comparability into data aspects, performance metrics, and give a close view on evaluation processes. Detailed information on pre-processing as well as data cleaning methods, the importance of unified performance reporting, and the need for complexity measures in load disaggregation are found to be the most urgent issues in NILM-related research. In addition, our evaluation suggests that datasets should be chosen carefully. We conclude by formulating suggestions for future work to enhance comparability. △ Less

Submitted 20 January, 2020; originally announced January 2020.

On Metrics to Assess the Transferability of Machine Learning Models in Non-Intrusive Load Monitoring

Authors: Christoph Klemenjak, Anthony Faustine, Stephen Makonin, Wilfried Elmenreich

Abstract: To assess the performance of load disaggregation algorithms it is common practise to train a candidate algorithm on data from one or multiple households and subsequently apply cross-validation by evaluating the classification and energy estimation performance on unseen portions of the dataset derived from the same households. With an emerging discussion of transferability in Non-Intrusive Load Mon… ▽ More To assess the performance of load disaggregation algorithms it is common practise to train a candidate algorithm on data from one or multiple households and subsequently apply cross-validation by evaluating the classification and energy estimation performance on unseen portions of the dataset derived from the same households. With an emerging discussion of transferability in Non-Intrusive Load Monitoring (NILM), there is a need for domain-specific metrics to assess the performance of NILM algorithms on new test scenarios being unseen buildings. In this paper, we discuss several metrics to assess the generalisation ability of NILM algorithms. These metrics target different aspects of performance evaluation in NILM and are meant to complement the traditional performance evaluation approach. We demonstrate how our metrics can be utilised to evaluate NILM algorithms by means of two case studies. We conduct our studies on several energy consumption datasets and take into consideration five state-of-the-art as well as four baseline NILM solutions. Finally, we formulate research challenges for future work. △ Less

Submitted 12 December, 2019; originally announced December 2019.

Non-Intrusive Load Monitoring: A Review and Outlook

Authors: Christoph Klemenjak, Peter Goldsborough

Abstract: With the roll-out of smart meters the importance of effective non-intrusive load monitoring (NILM) techniques has risen rapidly. NILM estimates the power consumption of individual devices given their aggregate consumption. In this way, the combined consumption must only be monitored at a single, central point in the household, providing various advantages such as reduced cost for metering equipmen… ▽ More With the roll-out of smart meters the importance of effective non-intrusive load monitoring (NILM) techniques has risen rapidly. NILM estimates the power consumption of individual devices given their aggregate consumption. In this way, the combined consumption must only be monitored at a single, central point in the household, providing various advantages such as reduced cost for metering equipment. In this paper we discuss the fundamental building-blocks of NILM, first giving a taxonomy of appliance models and device signatures and then explaining common supervised and unsupervised learning methods. Furthermore, we outline a fundamental algorithm that tackles the task of NILM. Subsequently, this paper reviews recent research that has brought novel insight to the field and more effective techniques. Finally, we formulate future challenges in the domain of NILM and smart meters. △ Less

Submitted 4 October, 2016; originally announced October 2016.

Comments: A shorter version of this paper was presented at the SKILL Students Conference 2016, part of the INFORMATIK2016 congress

YoMo - The Arduino based Smart Metering Board

Authors: Christoph Klemenjak, Dominik Egarter, Wilfried Elmenreich

Abstract: Smart meters are an enabling technology for many smart grid applications. This paper introduces a design for a low-cost smart meter system as well as the fundamentals of smart metering. The smart meter platform, provided as open hardware, is designed with a connector interface compatible to the Arduino platform, thus opening the possibilities for smart meters with flexible hardware and computation… ▽ More Smart meters are an enabling technology for many smart grid applications. This paper introduces a design for a low-cost smart meter system as well as the fundamentals of smart metering. The smart meter platform, provided as open hardware, is designed with a connector interface compatible to the Arduino platform, thus opening the possibilities for smart meters with flexible hardware and computation features, starting from low-cost 8 bit micro controllers up to powerful single board computers that can run Linux. The metering platform features a current transformer which allows a non-intrusive installation of the current measurement unit. The suggested design can switch loads, offers a variable sampling frequency, and provides measurement data such as active power, reactive and apparent power. Results indicate that measurement accuracy and resolution of the proposed metering platform are sufficient for a range of different applications and loads from a few watts up to five kilowatts. △ Less

Submitted 11 September, 2014; originally announced September 2014.