reproduce result on REDD

[Hessen525]

Hi, Dr.Zhong @MingjunZhong
Thanks for sharing the code. I follow the instructions and run this project successfully. Keep the same parameter as you submit. But the results I get for REDD are quite different from your reported results. I'm wondering if I missed anything? I RUN 50 epochs.
for train, I run
python3 seq2point_train.py --appliance_name fridge --datadir ./dataset_management/redd/ --save_dir ./trained_model --transfer_model False --crop_dataset None
for test, I run
python3 seq2point_test.py --appliance_name fridge --datadir ./dataset_management/redd/ --trained_model_dir ./trained_model --save_results_dir ./result --transfer False --crop_dataset None --plot_results True

Fridge, I got same SAE value, but the MAE is big different(35.2376 vs 28.104).

Any other application will be test soon.
Any reply/help will be much appreciated!

[MingjunZhong]

@Hessen525 When training, are you using early-stopping as I see it ran 50 iterations? I can't see the reasons why it has so much difference. What about other appliances?

[Hessen525]

[Hessen525]

@MingjunZhong Sorry for disturbing you again, Dr. Zhong.
I try in my mac and windows, for REDD dataset, get same results. the others applications result graph look very well, but the plot graph of washingmachine looks abnormal because the ground truth value of it is keep zero all the time.

[Hessen525]

above graph trained and tested for all data in REDD for specified application, but the following is trained all data and test first 10000 data point. the results look very well.

[Hessen525]

@MingjunZhong Hi, Dr.Zhong.
Thanks for your help again. Your project is very good!
I have found the problem, I guess the parameter set wrong or your dataset is different with that downloaded from http://redd.csail.mit.edu/data/low_freq.tar.bz2.
In redd_parameters.py file, you set as following:
'washingmachine': {
'windowlength': 599,
'on_power_threshold': 20,
'max_on_power': 3999,
'mean': 400,
'std': 700,
's2s_length': 2000,
'houses': [1, 2, 3],
'channels': [19, 7, 13],
'train_build': [2, 3],
'test_build': 1

For washingmachine, you set building 1 as test_dataset where channels_19 as washingmachine data, but I look at that file, it is all zero, so that could be interpreted why ground truth is keep zero in results plot graph.

[Hessen525]

I change Channel_19 TO 20, It works very well, Is that used in your experiment, Dr.Zhong?

[MingjunZhong]

@Hessen525 So now it looks correct for washing machine and as well as other appliances. Thank you for the post, the problems was then the labelling.

[MingjunZhong]

@Hessen525 I think we can close this issue now.

[Hessen525]

Sure, Thanks!

[Engr-KhurramIqbal]

@Hessen525 will you please explain how to set up the virtual environment on windows.Actually I am unable to create environment by using yml file due to difference in Operating system.I am windows os user.Will you please share the yml files for windows os.thanks

[Hessen525]

Hi, @khurramCh
Do not worry about the environment required . All you need is to run the program in Pycharm and install what is missing. In fact, there are not many libraries to install!

[Engr-KhurramIqbal]

hello,@Hessen525
Thanks for your reply.

[hellowangqian]

Hi @Hessen525
Are you able to reproduce similar results to those reported in the paper on REDD dataset? Thanks. Here is what I can get: MingjunZhong/NeuralNetNilm#3 (comment)

[Hessen525]

Hi @hellowangqian
Do not worry about that. If you want to compare with this algorithm, you have two choices:

1. Compare your model with the results in this article, in the same case as their paper.
2. Try to compare results in other data sets.

Both is better.