hw4_97722194.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "nUXoMKBNL2BB"
   },
   "source": [
    "\n",
    "<h3 align=\"center\">به نام خدا </h3>\n",
    "<h2 align=\"center\">تمرین سری چهارم </h2>\n",
    "\n",
    "\n",
    "<p align=\"right\">در این تمرین می خواهیم با کمک دانسته های خود از توزیع نرمال یک متغیره یک دسته بند طراحی کنیم</p>\n",
    "\n",
    "<p align=\"right\">برای پیاده سازی مدل ها از کتابخانه  استفاده کنید scikitlearn</p>\n",
    "\n",
    "<p align=\"right\">https://scikit-learn.org/stable/</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "UzYY1KuUPu2Z"
   },
   "source": [
    "# خواندن داده ها"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "0v0tV8s9P4xZ"
   },
   "source": [
    "<p align=\"right\">ابتدا داده ها را از کتابخانه می خوانیم و سپس داده ها را به دو دسته ی آموزش و آزمایش تقسیم بندی می کنیم</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "3wyhQpoQQgQ0",
    "outputId": "8f4cdba2-a3f2-4f35-cada-696f1d225c17"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "our dataset has 4 features. for more information about data surf the web\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "from sklearn import datasets\n",
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "def print_accuracy(y_prd, y_true, name=''):\n",
    "    acc = accuracy_score(y_prd, y_true, normalize=True) * 100\n",
    "    print(name + 'Accuracy = {0}%'.format(round(acc, 2)))\n",
    "    \n",
    "iris = datasets.load_iris()\n",
    "X = iris.data\n",
    "y = iris.target\n",
    "\n",
    "# store the feature matrix (X) and response vector (y) \n",
    "X = iris.data\n",
    "y = iris.target\n",
    "print(\"our dataset has \" + str(X.shape[1]) + \" features. for more information about data surf the web\")\n",
    "# splitting X and y into training and testing sets\n",
    "#you can change the test size, fit model with more or less data and see results\n",
    "from sklearn.model_selection import train_test_split \n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.4, random_state=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Fq4rKK30R9fz"
   },
   "source": [
    "### q1) print the number of train and test data and number of classes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "RQy7kw4pSR8l"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "the number of train data is :  90\n",
      "the number of test data is :  60\n",
      "there is 3 different classes in the dataset\n"
     ]
    }
   ],
   "source": [
    "print(\"the number of train data is : \", len(y_train) )\n",
    "print(\"the number of test data is : \", len(y_test) )\n",
    "print(\"there is \" + str(len(np.unique(y_train))) + \" different classes in the dataset\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "GDB-hbPVSpHt"
   },
   "source": [
    "# آموزش یک مدل با فرض گوسی بودن توزیع داد ها"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Nqd2cq44Tksm"
   },
   "source": [
    "<p align=\"right\">فرض کنید می دانیم که داده های ما از یک توزیع گوسی پیروی می کنند و همچنین ویژگی ها از هم مستقل هستند لطفا با کمک فرض های مساله و بدون کمک از مدل های آماده یک مدل برای دسته بندی کلاس ها آموزش دهید . دقت کنید که نباید از کدهای آماده استفاده کنید و فقط مجاز هستید برای فرمول های پایه مثل میانگین یا واریانس از توابع آماده استفاده نمایید</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "yAlhGi06Voxt"
   },
   "outputs": [],
   "source": [
    "# training the model on training set \n",
    "\n",
    "# write your code here :\n",
    "train0, train1, train2 = X_train[y_train==0], X_train[y_train==1], X_train[y_train==2]\n",
    "mu = np.vstack((train0.mean(axis=0), train1.mean(axis=0), train2.mean(axis=0)))\n",
    "sigma = np.sqrt(np.vstack((train0.var(axis=0), train1.var(axis=0), train2.var(axis=0))))\n",
    "P_C = len(train0) / len(y_train), len(train1) / len(y_train), len(train2) / len(y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "wkv9HCmSWAk0"
   },
   "source": [
    "<p align=\"right\">اکنون با کمک داده های آزمایشی دقت مدل خود را بسنجید</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "ordqnzrKXcLA"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "training data Accuracy = 95.56%\n",
      "test data Accuracy = 95.0%\n"
     ]
    }
   ],
   "source": [
    "#write your code here :\n",
    "def naive_bayes_classifier(X, mu, sigma, pc):\n",
    "    N, K = X.shape[0], mu.shape[0]\n",
    "    y = np.zeros((N, K))\n",
    "    for c in range(K):\n",
    "        y[:, c] = -np.asarray([sum(np.power(np.true_divide(np.subtract(x, mu[c]), sigma[c]), 2)) for x in X],\n",
    "                              dtype='float32')/2 + np.log(pc[c])\n",
    "    return np.argmax(y, axis=1)\n",
    "\n",
    "print_accuracy(naive_bayes_classifier(X_train, mu, sigma, P_C), y_train, 'training data ')\n",
    "print_accuracy(naive_bayes_classifier(X_test, mu, sigma, P_C), y_test, 'test data ')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "9V49EZ73XsvG"
   },
   "source": [
    "# آموزش مدل بدون دانستن توزیع داده ها"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "uni-0aA_X1ac"
   },
   "source": [
    "<p align=\"right\">داده ها را با کمک دسته بند اس وی ام و همچنین یک شبکه عصبی ساده دسته بندی کنید و دقت را با بخش قبل مقایسه نمایید</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "VftAlrbXZ-Nn"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "kernel: Polynomial with degree=4\n",
      "training data Accuracy = 97.78%\n",
      "test data Accuracy = 100.0%\n",
      "\n",
      "kernel: Polynomial with degree=9\n",
      "training data Accuracy = 100.0%\n",
      "test data Accuracy = 91.67%\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from sklearn import svm\n",
    "#train svm model\n",
    "#write yor code here :\n",
    "\n",
    "for degree in (4, 9):\n",
    "    clf = svm.SVC(kernel='poly', degree=degree)\n",
    "    clf.fit(X_train, y_train)\n",
    "    y = clf.predict(X_train)\n",
    "    print('kernel: Polynomial with degree={0}'.format(degree))\n",
    "    print_accuracy(clf.predict(X_train), y_train, 'training data ')\n",
    "    print_accuracy(clf.predict(X_test), y_test, 'test data ')\n",
    "    print()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "RYJlUDsGaGQT"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "training data Accuracy = 96.67%\n",
      "test data Accuracy = 100.0%\n"
     ]
    }
   ],
   "source": [
    "from sklearn.neural_network import MLPClassifier\n",
    "#use two hidden layers \n",
    "#train multi layer perceptron\n",
    "#write yor code here :\n",
    "clf = MLPClassifier(hidden_layer_sizes=(100, 100), max_iter=1000)\n",
    "clf.fit(X_train, y_train)\n",
    "y = clf.predict(X_train)\n",
    "print_accuracy(clf.predict(X_train), y_train, 'training data ')\n",
    "print_accuracy(clf.predict(X_test), y_test, 'test data ')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "J_bh4KyklqFA"
   },
   "source": [
    "\n",
    "\n",
    "---\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "yYGlkDuckgmz"
   },
   "source": [
    "# خواندن داده های جدید"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "d626_2ecknMc"
   },
   "outputs": [],
   "source": [
    "wine = datasets.load_wine()\n",
    "# store the feature matrix (X) and response vector (y) \n",
    "X = wine.data \n",
    "y = wine.target \n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.4, random_state=1) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "E3ihimJ_lBJq"
   },
   "source": [
    "# آموزش یک مدل با فرض گوسی بودن توزیع داد ها"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "EIbuPy6GlKyw"
   },
   "source": [
    "<p align=\"right\">فرض کنید می دانیم که داده های ما از یک توزیع گوسی پیروی می کنند با کمک کتابخانه های آماده یک مدل برای دسته بندی کلاس ها آموزش دهید می  توانید برای این قسمت از کتابخانه های آماده نیز استفاده کنید و نیازی به پیاده سازی نیست</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "2i8GbQt9lPJO"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GaussianNB(priors=None, var_smoothing=1e-09)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.naive_bayes import GaussianNB\n",
    "# training the model on training set \n",
    "\n",
    "# write your code here :\n",
    "clf = GaussianNB()\n",
    "clf.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "9ugDovFDljq2"
   },
   "source": [
    "<p align=\"right\">اکنون با کمک داده های آزمایشی دقت مدل خود را بسنجید</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "dnkJltLll0qD"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "training data Accuracy = 97.17%\n",
      "test data Accuracy = 98.61%\n"
     ]
    }
   ],
   "source": [
    "#write your code here :\n",
    "print_accuracy(clf.predict(X_train), y_train, 'training data ')\n",
    "print_accuracy(clf.predict(X_test), y_test, 'test data ')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "24nscd6El30c"
   },
   "source": [
    "# آموزش مدل بدون دانستن توزیع داده ها"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "jXoB6ppJl7Vl"
   },
   "source": [
    "<p align=\"right\">داده ها را با کمک دسته بند اس وی ام و همچنین یک شبکه عصبی ساده دسته بندی کنید و دقت را با بخش قبل مقایسه نمایید</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "9cw27u1bmBWE"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "kernel: RBF\n",
      "training data Accuracy = 72.64%\n",
      "test data Accuracy = 69.44%\n"
     ]
    }
   ],
   "source": [
    "#train svm model\n",
    "#write yor code here :\n",
    "\n",
    "from sklearn import svm\n",
    "clf = svm.SVC(kernel='rbf', C=2.0)\n",
    "clf.fit(X_train, y_train)\n",
    "y = clf.predict(X_train)\n",
    "print('kernel: RBF')\n",
    "print_accuracy(clf.predict(X_train), y_train, 'training data ')\n",
    "print_accuracy(clf.predict(X_test), y_test, 'test data ')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "tqS7kFEqmEMG"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "training data Accuracy = 100.0%\n",
      "test data Accuracy = 94.44%\n"
     ]
    }
   ],
   "source": [
    "#train multi layer perceptron\n",
    "#write yor code here\n",
    "from sklearn.neural_network import MLPClassifier\n",
    "clf = MLPClassifier(hidden_layer_sizes=(100, 100), max_iter=1000)\n",
    "clf.fit(X_train, y_train)\n",
    "y = clf.predict(X_train)\n",
    "print_accuracy(clf.predict(X_train), y_train, 'training data ')\n",
    "print_accuracy(clf.predict(X_test), y_test, 'test data ')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Xw54ZdjmlsjN"
   },
   "source": [
    "\n",
    "\n",
    "---\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "X_qYfi-kgFbN"
   },
   "source": [
    "#بخش امتیازی"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "NQ97VUicmoir"
   },
   "source": [
    "<p align=\"right\">آیا راهی وجود دارد که قبل از انتخاب بک دسته بند بتوان توزیع یک داده را حدس زد و دسته بندی را بر اساس این بینش از داده ها انجام داد؟</p>\n",
    "<h1><div style=\"direction:rtl\">پاسخ در گزارش نوشته شده است.</div></h1>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "2Oi6thGwnNiC"
   },
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "hw4-project",
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}