Machine Learning - Exercise 3

A Neural Network which classifies an image to one of 10 clothing classes.

Explanation

Parameters:

• Hidden layer(s): One layer of 100 neurons.
• Number of epochs: 10.
• Learning rate: 0.001.
• Activation function: LeakyReLU.
• W and B matrices were initialized with random numbers from a uniform distribution function in the range of (-0 .08, 0.08).

Results:

• Validation accuracy: 0.870
• Loss Sum: 0.338
• Validation Loss Sum: 0.352

I've chosen these parameters after experimenting with different parameters and found these to be the best.

Example Output

Note: Output can vary from each training.

Reading resources...    
Reading done!
Pre-trained neural network was not found. Creating a new network.
Training neural network...
Epoch #0, Validation accuracy: 0.828909090909, Loss sum: 0.7352758381373954, Validation loss sum: 0.50803402649789
Epoch #1, Validation accuracy: 0.840454545455, Loss sum: 0.49402603982204113, Validation loss sum: 0.45767131981666404
Epoch #2, Validation accuracy: 0.853181818182, Loss sum: 0.45054050356853026, Validation loss sum: 0.4305516629993061
Epoch #3, Validation accuracy: 0.847363636364, Loss sum: 0.4227981354595133, Validation loss sum: 0.4259559203776533
Epoch #4, Validation accuracy: 0.859636363636, Loss sum: 0.4023585485906246, Validation loss sum: 0.39746483692094525
Epoch #5, Validation accuracy: 0.865818181818, Loss sum: 0.38504833835517277, Validation loss sum: 0.38193218988091654
Epoch #6, Validation accuracy: 0.863363636364, Loss sum: 0.36925837331845923, Validation loss sum: 0.38802740123276896
Epoch #7, Validation accuracy: 0.87, Loss sum: 0.35780475921100713, Validation loss sum: 0.3661885978694088
Epoch #8, Validation accuracy: 0.873727272727, Loss sum: 0.34745452592073944, Validation loss sum: 0.3558625808368603
Epoch #9, Validation accuracy: 0.870272727273, Loss sum: 0.33766276345064306, Validation loss sum: 0.3523274992951357  
Training done!
Saving neural network to file "my_neural_network.pkl" for later use...  
Saving done!
Predicting samples from tests file and writes predictions to output file...
Testing done!