Neural Network Ensembles to Real-time Identification of Plug -level Appliance...

11
Neural Network Ensembles to Real-time Identification of Plug -level Appliance Measurements NILM Workshop 2016, 14 th May Karim Said Barsim, Lukas Mauch, and Bin Yang Institute of Signal Processing and System Theory University of Stuttgart, Stuttgart, Germany {karim.barsim,lukas.mauch,bin.yang}@iss.uni-Stuttgart.de

Transcript of Neural Network Ensembles to Real-time Identification of Plug -level Appliance...

  • Neural Network Ensembles to Real-timeIdentification of Plug -level Appliance

    Measurements

    NILM Workshop 2016, 14th May

    Karim Said Barsim, Lukas Mauch, and Bin Yang

    Institute of Signal Processing and System Theory

    University of Stuttgart, Stuttgart, Germany

    {karim.barsim,lukas.mauch,bin.yang}@iss.uni-Stuttgart.de

  • 14th May. 2016

    L. MauchStuttgartUniversity

    Introduction and motivation Appliance identification

    A sub-task of the NILM problem

    Sometimes a disjoint process

    An upper bound

    Raw signatures (current 𝑖 𝑛 and voltage 𝑣 𝑛 )

    High resolution signals

    Plug-level measurements

    Plug-level appliance identification

    Smart outlets

    Aspects of the identification process

    Training data

    Sampling frequency

    Signature variations

    (2/11)

    Inverse LoadReconstruction

    ApplianceIdentification

    Introduction

    Task & tools

    Signatures

    Model

    Discussion

    –––––––––––

    –––––––––––

    –––––––––––

    Label space

    Signaturevariations

    Samplingfrequency

    Prediction

    Training data

  • 14th May. 2016

    L. MauchStuttgartUniversity

    Task and tools

    Generic appliance identification from high resolution raw measurements

    Neural network ensembles:

    Suitable for raw-data learning

    Unstable w.r.t training data (i.e. suited for models ensembles)

    Plug Load Appliance Identification Dataset (PLAID) [1]

    Plug-level raw measurements

    Generic appliance categories

    (3/11)

    [1] J. Gao, S. Giri, E. C. Kara, and M. Berg´es, “PLAID: A Public Dataset of High-resolution Electrical Appliance Measurements for Load Identification Research: Demo Abstract”. In proceedings of the 1st ACM Conference on Embedded Systems for Energy-Efficient Buildings:198–199, 2014.

    Introduction

    Task & tools

    Signatures

    Model

    Discussion

    –––––––––––

    –––––––––––

    –––––––––––

    Label space

    Signaturevariations

    Samplingfrequency

    Prediction

    Training data

  • 14th May. 2016

    L. MauchStuttgartUniversity

    Appliances’ Signature

    (4/11)

    Raw current 𝑖 𝑛 and voltage 𝑣 𝑛

    Unsupervised feature learning

    𝒙𝜏 = Ƹ𝒊𝜏𝑇 , ෝ𝒗𝜏

    𝑇 𝑇

    Single period (~ 0.017 seconds @ 60Hz)

    Real-time identification

    𝒊𝜏 = 𝑖 𝜏 , 𝑖 𝜏 + 1 , … , 𝑖 𝜏 + 𝑑 + 1𝑇

    Segment-based normalization

    Discarding amplitude information

    Generic labeling

    Ƹ𝑖𝜏 =2 𝒊𝜏− max 𝒊𝜏−min 𝒊𝜏 𝟏𝑑,1

    max 𝒊𝜏−min 𝒊𝜏

    Algorithmic expansion

    Multiple phase shifts from 2 periods

    Translation invariance and robustness to variations

    𝒳 = 𝒙𝜏 𝜏 = 𝜏𝑜 +𝑚 𝜖 , 0 ≤ 𝑚 <𝑑

    𝜖

    Appliance signature (green curves)

    Expansion of training data

    Introduction

    Task & tools

    Signatures

    Model

    Discussion

    –––––––––––

    –––––––––––

    –––––––––––

    Label space

    Signaturevariations

    Samplingfrequency

    Prediction

    Training data

  • 14th May. 2016

    L. MauchStuttgartUniversity

    Prediction model

    (5/11)

    A neural network ensemble

    Unstable models

    Similar to Bootstrap aggregation

    Ex. binary classification networks

    Ex. per class combination 112

    = 55 nets

    Ex. shallow, feedforward, fully connected nets

    𝐷𝜔𝑖,𝜔𝑗 = 𝒙 ∈ 𝒳 𝜔 𝒙 = 𝜔𝑖 or𝜔 𝒙 = 𝜔𝑗

    𝜃𝜔𝑖,𝜔𝑗 𝒙 = Ƹ𝑝𝜔𝑖,𝜔𝑗 , Ƹ𝑝𝜔𝑗,𝜔𝑖

    Confidence-weighted voting

    ෝ𝜔 𝒙 = arg max𝜔𝑖 ∈ Ω

    𝑗≠𝑖

    Ƹ𝑝𝜔𝑗,𝜔𝑖 𝒙

    Fullyconnected

    Weighted voting(pre-assigned)

    Subnetworks

    1000

    30

    30

    30

    2

    2

    2

    11

    GenericLabeling

    ApplianceSignature

    An example of the adopted prediction model

    Introduction

    Task & tools

    Signatures

    Model

    Discussion

    –––––––––––

    –––––––––––

    –––––––––––

    Label space

    Signaturevariations

    Samplingfrequency

    Prediction

    Training data

  • 14th May. 2016

    L. MauchStuttgartUniversity

    Experiments (1) - Prediction

    (6/11)

    PLAID Dataset

    +200 instances

    +1000 measurements

    Category-based ( 11 cat.)

    High frequency ( 30 kHz )

    Residential dataset ( 55 homes )

    Prediction - Training

    Validation-based early stopping

    Building-based validation

    Leave-house-out cross validation [2]

    Complete (54 houses, 30 kHz, 11 cat.)

    Prediction - Evaluation

    Category-based, 𝐹1- score 𝐹1𝑆

    Building-based, Accuracy 𝛼

    Total accuracy: 𝛼 = 0.89%

    [2] J. Gao, E. C. Kara, S. Giri, and M. Berg´es, “A feasibility study of automated plug-load identification from high frequency measurements”. In proceedings of the 3rd IEEE Global Conference on Signal and Information Processing (GlobalSIP):220–224, Dec. 2015.

    Three best/worst predicted categories

    Introduction

    Task & tools

    Signatures

    Model

    Discussion

    –––––––––––

    –––––––––––

    –––––––––––

    Label space

    Signaturevariations

    Samplingfrequency

    Prediction

    Training data

  • 14th May. 2016

    L. MauchStuttgartUniversity

    Experiments (2) - Training data

    (7/11)

    Effect of reducing size of training data 𝒓

    Building-based reduction

    Training on a ratio 𝑟 of labeled data

    Sampling frequency 𝑓𝑠 = 30 kHz

    Test sample: last period of each measurement

    Label space Ω = 11 categories

    Notable degradation w.r.t training data

    Labeled dataset ~𝑟 Unlabeled set

    TestBuildingTraining dataValidation ~30%

    Introduction

    Task & tools

    Signatures

    Model

    Discussion

    –––––––––––

    –––––––––––

    –––––––––––

    Label space

    Signaturevariations

    Samplingfrequency

    Prediction

    Training data

  • 14th May. 2016

    L. MauchStuttgartUniversity

    Experiments (3) - Sampling frequency

    (8/11)

    Complete training data (54 buildings)

    Resampled signals

    Sampling frequency 𝑓𝑠 = 2.5 − 30 kHz

    Test sample: last period of each measurement

    Label space Ω = 11 categories

    Almost stable for a wide range

    Always +80%

    FIR low passFiltering

    RateConversion

    Fractional decimation

    Introduction

    Task & tools

    Signatures

    Model

    Discussion

    –––––––––––

    –––––––––––

    –––––––––––

    Label space

    Signaturevariations

    Samplingfrequency

    Prediction

    Training data

  • 14th May. 2016

    L. MauchStuttgartUniversity

    Experiments (4) - Signature variation

    (9/11)

    Complete training data (54 buildings)

    Sampling frequency 𝑓𝑠 = 30 kHz

    Test sample:

    Phase-shifted sample by 𝜏 seconds

    Label space Ω = 11 categories

    Robust to signature variations

    But only evaluated on 1 second of operation

    A certain phase shift is preferred

    𝜏 = 100 𝑚𝑠

    Introduction

    Task & tools

    Signatures

    Model

    Discussion

    –––––––––––

    –––––––––––

    –––––––––––

    Label space

    Signaturevariations

    Samplingfrequency

    Prediction

    Training data

  • 14th May. 2016

    L. MauchStuttgartUniversity

    Experiments (5) - Label space

    (10/11)

    Complete training data (54 buildings)

    Sampling frequency 𝑓𝑠 = 30 kHz

    Test sample: last period of each measurement

    Label space

    A list of appliances in each building is known

    A per-building label space Ωh

    Useful information but not always available.

    Total accuracy: 𝛼 ≅ 95%

    𝜏 = 100 𝑚𝑠

    Introduction

    Task & tools

    Signatures

    Model

    Discussion

    –––––––––––

    –––––––––––

    –––––––––––

    Label space

    Signaturevariations

    Samplingfrequency

    Prediction

    Training data

  • 14th May. 2016

    L. MauchStuttgartUniversity

    DiscussionIntroductionTask & tools

    Signatures

    Model

    Discussion

    –––––––––––

    –––––––––––

    –––––––––––

    Thank youfor your attention

    (11/11)

    Label space

    Signaturevariations

    Samplingfrequency

    Prediction

    Training data


Clinical Recommender System: Predicting Medical Specialty ...Clinical Recommender System: Predicting Medical Specialty Diagnostic Choices with Neural Network Ensembles Morteza Noshad1,

Clinical Recommender System: Predicting Medical Specialty ...Clinical Recommender System: Predicting Medical Specialty Diagnostic Choices with Neural Network Ensembles Morteza Noshad1,

Deep Ensembles for Inter-Domain Arousal Recognitionproceedings.mlr.press/v86/gjoreski20a/gjoreski20a.pdf · 2020. 8. 18. · The novel method, an inter-domain Deep Neural Network

Deep Ensembles for Inter-Domain Arousal Recognitionproceedings.mlr.press/v86/gjoreski20a/gjoreski20a.pdf · 2020. 8. 18. · The novel method, an inter-domain Deep Neural Network

- Tutorial - Neuromorphic Computing with Spintronics...• Deep neural network • Recurrent neural network –Reservoir computing • Hopfield network • Stochastic neural network

- Tutorial - Neuromorphic Computing with Spintronics...• Deep neural network • Recurrent neural network –Reservoir computing • Hopfield network • Stochastic neural network

Neural Network Ensembles, Cross Validation, and Active ... · PDF fileNeural Network Ensembles, Cross Validation, and Active Learning 233 All these formulas can be averaged over the

Neural Network Ensembles, Cross Validation, and Active ... · PDF fileNeural Network Ensembles, Cross Validation, and Active Learning 233 All these formulas can be averaged over the

Collective Learning by Ensembles of Altruistic Diversifying Neural … · 2020-06-23 · Collective Learning by Ensembles of Altruistic Diversifying Neural Networks Benjamin Brazowski

Collective Learning by Ensembles of Altruistic Diversifying Neural … · 2020-06-23 · Collective Learning by Ensembles of Altruistic Diversifying Neural Networks Benjamin Brazowski

ELEC 677: Recurrent Neural Network Applications & Recurrent Neural Network … · 2016-11-08 · Recurrent Neural Network Applications & Recurrent Neural Network Language Models Lecture

ELEC 677: Recurrent Neural Network Applications & Recurrent Neural Network … · 2016-11-08 · Recurrent Neural Network Applications & Recurrent Neural Network Language Models Lecture

Neural Network Language Models - The University of Edinburgh · Hybrid neural network / HMM acoustic models Neural network features { Tandem, posteriorgrams Deep neural network acoustic

Neural Network Language Models - The University of Edinburgh · Hybrid neural network / HMM acoustic models Neural network features { Tandem, posteriorgrams Deep neural network acoustic

Biological Neural Network & Nonlinear Dynamics Biological Neural Network Similar Neural Network to Real Neural Networks Membrane Potential Potential of.

Biological Neural Network & Nonlinear Dynamics Biological Neural Network Similar Neural Network to Real Neural Networks Membrane Potential Potential of.

Quantification of Uncertainty in Mineral Prospectivity Prediction Using Neural Network Ensembles

Quantification of Uncertainty in Mineral Prospectivity Prediction Using Neural Network Ensembles

Chapter 10 Chapter 10 Neural Network Neural Network.

Chapter 10 Chapter 10 Neural Network Neural Network.

Time Series Prediction_neural Network Ensembles

Time Series Prediction_neural Network Ensembles

Neural Network Ensembles and Combinatorial Optimization with

Neural Network Ensembles and Combinatorial Optimization with

Towards Efficient Learning of Neural Network Ensembles from Arbitrarily Large Datasets Kang Peng, Zoran Obradovic and Slobodan Vucetic Center for Information.

Towards Efficient Learning of Neural Network Ensembles from Arbitrarily Large Datasets Kang Peng, Zoran Obradovic and Slobodan Vucetic Center for Information.

RNA sequencing from neural ensembles activated during fear ...

RNA sequencing from neural ensembles activated during fear ...

NEURAL NETWORK ENSEMBLES UNTUK PERAMALAN NILAI …/Neural...kemampuannya dalam memorisasi dan generalisasi data sebelumnya . Penelitian ini mengkaji ulang Neural Network Ensembles

NEURAL NETWORK ENSEMBLES UNTUK PERAMALAN NILAI …/Neural...kemampuannya dalam memorisasi dan generalisasi data sebelumnya . Penelitian ini mengkaji ulang Neural Network Ensembles

Two-stage procedure based on smoothed ensembles of neural ...

Two-stage procedure based on smoothed ensembles of neural ...

arXiv:2006.04059v1 [cs.LG] 7 Jun 2020building di erentiable tree ensembles on top of a neural network [KFCB15], or using ensemble methods on neural networks [MM19, SS14, HKH19, ZWT02].

arXiv:2006.04059v1 [cs.LG] 7 Jun 2020building di erentiable tree ensembles on top of a neural network [KFCB15], or using ensemble methods on neural networks [MM19, SS14, HKH19, ZWT02].

Improving the Performance of Entropy Ensembles of Neural ... · as Na wve Bayes(NBs), KNN, Decision Tree (DT) Algorithm, Neural Network(NN). etc., were used to classify the patient

Improving the Performance of Entropy Ensembles of Neural ... · as Na wve Bayes(NBs), KNN, Decision Tree (DT) Algorithm, Neural Network(NN). etc., were used to classify the patient

Thresholded ConvNet ensembles: neural networks for …sjrob/Pubs/Ghoshal_et_al...zero mean and unit variance. Fig. 2 A convolutional neural network with 2 convolutional layers. The

Thresholded ConvNet ensembles: neural networks for …sjrob/Pubs/Ghoshal_et_al...zero mean and unit variance. Fig. 2 A convolutional neural network with 2 convolutional layers. The

Comparing ensembles of decision trees and neural networks ... · Comparing ensembles of decision trees and neural networks for one-day-ahead streamflow prediction Onur KARAKURT, Ms.C.

Comparing ensembles of decision trees and neural networks ... · Comparing ensembles of decision trees and neural networks for one-day-ahead streamflow prediction Onur KARAKURT, Ms.C.