kNN + PCA

[dzisandy]

Dear @mxbi, as you asked in previous issue KMNIST kNN #10

I've calculated scores for K-49 and MNIST.

• Code for K-49 with respect to description of dataset (taken balanced matrics).

from sklearn.neighbors import KNeighborsClassifier
import numpy as np
from sklearn.decomposition import PCA


def load(f):
    return np.load(f)['arr_0']

# Load the data
x_train = load('k49-train-imgs.npz')
x_test = load('k49-test-imgs.npz')
y_train = load('k49-train-labels.npz')
y_test = load('k49-test-labels.npz')

# Flatten images
x_train = x_train.reshape(-1, 784)
x_test = x_test.reshape(-1, 784)

pca = PCA(n_components= 60, random_state= 0 )
x_train = pca.fit_transform(x_train)
x_test = pca.transform(x_test)
clf = KNeighborsClassifier(n_neighbors= 4,weights='distance', n_jobs=-1)
clf.fit(x_train, y_train)
p_test = clf.predict(x_test)

#added by description of K-49
accs = []
for cls in range(49):
  mask = (y_test == cls)
  cls_acc = (p_test == cls)[mask].mean() 
  accs.append(cls_acc)
  
accs = np.mean(accs)
print('Test accuracy:', accs)

The result is Test accuracy: 0.8679612391951115

• Code for MNIST remains similar to the given in previous issue (.npz taken from Kaggle):

from sklearn.neighbors import KNeighborsClassifier
import numpy as np
from sklearn.decomposition import PCA

#Taken from Kaggle 
def load_data(path):
    with np.load(path) as f:
        x_train, y_train = f['x_train'], f['y_train']
        x_test, y_test = f['x_test'], f['y_test']
        return (x_train, y_train), (x_test, y_test)

# Load the data
(x_train, y_train), (x_test, y_test) = load_data('/content/mnist.npz')

x_train = x_train.reshape(-1, 784)
x_test = x_test.reshape(-1, 784)

pca = PCA(n_components= 60, random_state= 0 )
x_train = pca.fit_transform(x_train)
x_test = pca.transform(x_test)

clf = KNeighborsClassifier(n_neighbors= 4,weights='distance', n_jobs=-1)
clf.fit(x_train, y_train)
print('Test accuracy:', clf.score(x_test, y_test))

The result is Test accuracy: 0.9776

Hope it will be a good datapoint and milestone for somebody.

[mxbi]

I've updated the page, thank you!