Supervised Learning/ML_KNN-algorithm.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "<br>\n",
    "Steps to Machine Learning<br>\n",
    "1.Data Gathering- Doesn't matter ehich type of ML ,we need data<br>\n",
    "-> Extracted from somewhere<br>\n",
    "-> From internet<br>\n",
    "-> Real life data<br>\n",
    " Some other form of data<br>\n",
    " Types of data-Clean data and mixed data<br>\n",
    " Get data in any form<br>\n",
    " we have to process the data<br>\n",
    " 20% data is clean data ,80% data is raw data in any format like website data or pdf format<br>\n",
    "2.Data Preprocessing- process the data<br>\n",
    "-> cleaning the data<br>\n",
    "-> not in the way model will accept<br>\n",
    "-> delete/append/clean<br>\n",
    "-> model will not accept wrong data<br>\n",
    "-> Prediction will be wrong<br>\n",
    "3. Choose an Algorithm/Build a model<br>\n",
    "4. Train your model<br>\n",
    "-> Learn from example and data<br>\n",
    "5. Evaluation/testing of model<br>\n",
    "-> Checking if model correctly works or not<br>\n",
    "Data will vary ,Model will vary but this steps will remain the same<br>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Seeing Accuracy of model for various datasets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sample dataset -iris"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import load_iris"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Checking without splitting into testing and training data<br>\n",
    "1. Data Gathering<br>\n",
    "iris is a flower<br>\n",
    "three species of iris-setosa,versicolour and virginica<br>\n",
    "one is sepal and one is petal<br>\n",
    "measurements are in cm height and width<br>\n",
    "sepal length width petal length width<br>\n",
    "we are trying to classify flower into this categories<br>\n",
    "We need to build a model which will classify flower based on sepal length width and petal length width as species"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2. Data Preprocessing<br>\n",
    "No preprocessing needed<br>\n",
    "Since it is already cleaned we do not need preprocessing step"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iris Dataset:\n",
      " {'data': array([[5.1, 3.5, 1.4, 0.2],\n",
      "       [4.9, 3. , 1.4, 0.2],\n",
      "       [4.7, 3.2, 1.3, 0.2],\n",
      "       [4.6, 3.1, 1.5, 0.2],\n",
      "       [5. , 3.6, 1.4, 0.2],\n",
      "       [5.4, 3.9, 1.7, 0.4],\n",
      "       [4.6, 3.4, 1.4, 0.3],\n",
      "       [5. , 3.4, 1.5, 0.2],\n",
      "       [4.4, 2.9, 1.4, 0.2],\n",
      "       [4.9, 3.1, 1.5, 0.1],\n",
      "       [5.4, 3.7, 1.5, 0.2],\n",
      "       [4.8, 3.4, 1.6, 0.2],\n",
      "       [4.8, 3. , 1.4, 0.1],\n",
      "       [4.3, 3. , 1.1, 0.1],\n",
      "       [5.8, 4. , 1.2, 0.2],\n",
      "       [5.7, 4.4, 1.5, 0.4],\n",
      "       [5.4, 3.9, 1.3, 0.4],\n",
      "       [5.1, 3.5, 1.4, 0.3],\n",
      "       [5.7, 3.8, 1.7, 0.3],\n",
      "       [5.1, 3.8, 1.5, 0.3],\n",
      "       [5.4, 3.4, 1.7, 0.2],\n",
      "       [5.1, 3.7, 1.5, 0.4],\n",
      "       [4.6, 3.6, 1. , 0.2],\n",
      "       [5.1, 3.3, 1.7, 0.5],\n",
      "       [4.8, 3.4, 1.9, 0.2],\n",
      "       [5. , 3. , 1.6, 0.2],\n",
      "       [5. , 3.4, 1.6, 0.4],\n",
      "       [5.2, 3.5, 1.5, 0.2],\n",
      "       [5.2, 3.4, 1.4, 0.2],\n",
      "       [4.7, 3.2, 1.6, 0.2],\n",
      "       [4.8, 3.1, 1.6, 0.2],\n",
      "       [5.4, 3.4, 1.5, 0.4],\n",
      "       [5.2, 4.1, 1.5, 0.1],\n",
      "       [5.5, 4.2, 1.4, 0.2],\n",
      "       [4.9, 3.1, 1.5, 0.2],\n",
      "       [5. , 3.2, 1.2, 0.2],\n",
      "       [5.5, 3.5, 1.3, 0.2],\n",
      "       [4.9, 3.6, 1.4, 0.1],\n",
      "       [4.4, 3. , 1.3, 0.2],\n",
      "       [5.1, 3.4, 1.5, 0.2],\n",
      "       [5. , 3.5, 1.3, 0.3],\n",
      "       [4.5, 2.3, 1.3, 0.3],\n",
      "       [4.4, 3.2, 1.3, 0.2],\n",
      "       [5. , 3.5, 1.6, 0.6],\n",
      "       [5.1, 3.8, 1.9, 0.4],\n",
      "       [4.8, 3. , 1.4, 0.3],\n",
      "       [5.1, 3.8, 1.6, 0.2],\n",
      "       [4.6, 3.2, 1.4, 0.2],\n",
      "       [5.3, 3.7, 1.5, 0.2],\n",
      "       [5. , 3.3, 1.4, 0.2],\n",
      "       [7. , 3.2, 4.7, 1.4],\n",
      "       [6.4, 3.2, 4.5, 1.5],\n",
      "       [6.9, 3.1, 4.9, 1.5],\n",
      "       [5.5, 2.3, 4. , 1.3],\n",
      "       [6.5, 2.8, 4.6, 1.5],\n",
      "       [5.7, 2.8, 4.5, 1.3],\n",
      "       [6.3, 3.3, 4.7, 1.6],\n",
      "       [4.9, 2.4, 3.3, 1. ],\n",
      "       [6.6, 2.9, 4.6, 1.3],\n",
      "       [5.2, 2.7, 3.9, 1.4],\n",
      "       [5. , 2. , 3.5, 1. ],\n",
      "       [5.9, 3. , 4.2, 1.5],\n",
      "       [6. , 2.2, 4. , 1. ],\n",
      "       [6.1, 2.9, 4.7, 1.4],\n",
      "       [5.6, 2.9, 3.6, 1.3],\n",
      "       [6.7, 3.1, 4.4, 1.4],\n",
      "       [5.6, 3. , 4.5, 1.5],\n",
      "       [5.8, 2.7, 4.1, 1. ],\n",
      "       [6.2, 2.2, 4.5, 1.5],\n",
      "       [5.6, 2.5, 3.9, 1.1],\n",
      "       [5.9, 3.2, 4.8, 1.8],\n",
      "       [6.1, 2.8, 4. , 1.3],\n",
      "       [6.3, 2.5, 4.9, 1.5],\n",
      "       [6.1, 2.8, 4.7, 1.2],\n",
      "       [6.4, 2.9, 4.3, 1.3],\n",
      "       [6.6, 3. , 4.4, 1.4],\n",
      "       [6.8, 2.8, 4.8, 1.4],\n",
      "       [6.7, 3. , 5. , 1.7],\n",
      "       [6. , 2.9, 4.5, 1.5],\n",
      "       [5.7, 2.6, 3.5, 1. ],\n",
      "       [5.5, 2.4, 3.8, 1.1],\n",
      "       [5.5, 2.4, 3.7, 1. ],\n",
      "       [5.8, 2.7, 3.9, 1.2],\n",
      "       [6. , 2.7, 5.1, 1.6],\n",
      "       [5.4, 3. , 4.5, 1.5],\n",
      "       [6. , 3.4, 4.5, 1.6],\n",
      "       [6.7, 3.1, 4.7, 1.5],\n",
      "       [6.3, 2.3, 4.4, 1.3],\n",
      "       [5.6, 3. , 4.1, 1.3],\n",
      "       [5.5, 2.5, 4. , 1.3],\n",
      "       [5.5, 2.6, 4.4, 1.2],\n",
      "       [6.1, 3. , 4.6, 1.4],\n",
      "       [5.8, 2.6, 4. , 1.2],\n",
      "       [5. , 2.3, 3.3, 1. ],\n",
      "       [5.6, 2.7, 4.2, 1.3],\n",
      "       [5.7, 3. , 4.2, 1.2],\n",
      "       [5.7, 2.9, 4.2, 1.3],\n",
      "       [6.2, 2.9, 4.3, 1.3],\n",
      "       [5.1, 2.5, 3. , 1.1],\n",
      "       [5.7, 2.8, 4.1, 1.3],\n",
      "       [6.3, 3.3, 6. , 2.5],\n",
      "       [5.8, 2.7, 5.1, 1.9],\n",
      "       [7.1, 3. , 5.9, 2.1],\n",
      "       [6.3, 2.9, 5.6, 1.8],\n",
      "       [6.5, 3. , 5.8, 2.2],\n",
      "       [7.6, 3. , 6.6, 2.1],\n",
      "       [4.9, 2.5, 4.5, 1.7],\n",
      "       [7.3, 2.9, 6.3, 1.8],\n",
      "       [6.7, 2.5, 5.8, 1.8],\n",
      "       [7.2, 3.6, 6.1, 2.5],\n",
      "       [6.5, 3.2, 5.1, 2. ],\n",
      "       [6.4, 2.7, 5.3, 1.9],\n",
      "       [6.8, 3. , 5.5, 2.1],\n",
      "       [5.7, 2.5, 5. , 2. ],\n",
      "       [5.8, 2.8, 5.1, 2.4],\n",
      "       [6.4, 3.2, 5.3, 2.3],\n",
      "       [6.5, 3. , 5.5, 1.8],\n",
      "       [7.7, 3.8, 6.7, 2.2],\n",
      "       [7.7, 2.6, 6.9, 2.3],\n",
      "       [6. , 2.2, 5. , 1.5],\n",
      "       [6.9, 3.2, 5.7, 2.3],\n",
      "       [5.6, 2.8, 4.9, 2. ],\n",
      "       [7.7, 2.8, 6.7, 2. ],\n",
      "       [6.3, 2.7, 4.9, 1.8],\n",
      "       [6.7, 3.3, 5.7, 2.1],\n",
      "       [7.2, 3.2, 6. , 1.8],\n",
      "       [6.2, 2.8, 4.8, 1.8],\n",
      "       [6.1, 3. , 4.9, 1.8],\n",
      "       [6.4, 2.8, 5.6, 2.1],\n",
      "       [7.2, 3. , 5.8, 1.6],\n",
      "       [7.4, 2.8, 6.1, 1.9],\n",
      "       [7.9, 3.8, 6.4, 2. ],\n",
      "       [6.4, 2.8, 5.6, 2.2],\n",
      "       [6.3, 2.8, 5.1, 1.5],\n",
      "       [6.1, 2.6, 5.6, 1.4],\n",
      "       [7.7, 3. , 6.1, 2.3],\n",
      "       [6.3, 3.4, 5.6, 2.4],\n",
      "       [6.4, 3.1, 5.5, 1.8],\n",
      "       [6. , 3. , 4.8, 1.8],\n",
      "       [6.9, 3.1, 5.4, 2.1],\n",
      "       [6.7, 3.1, 5.6, 2.4],\n",
      "       [6.9, 3.1, 5.1, 2.3],\n",
      "       [5.8, 2.7, 5.1, 1.9],\n",
      "       [6.8, 3.2, 5.9, 2.3],\n",
      "       [6.7, 3.3, 5.7, 2.5],\n",
      "       [6.7, 3. , 5.2, 2.3],\n",
      "       [6.3, 2.5, 5. , 1.9],\n",
      "       [6.5, 3. , 5.2, 2. ],\n",
      "       [6.2, 3.4, 5.4, 2.3],\n",
      "       [5.9, 3. , 5.1, 1.8]]), 'target': array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
      "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
      "       0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
      "       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
      "       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,\n",
      "       2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,\n",
      "       2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2]), 'frame': None, 'target_names': array(['setosa', 'versicolor', 'virginica'], dtype='<U10'), 'DESCR': '.. _iris_dataset:\\n\\nIris plants dataset\\n--------------------\\n\\n**Data Set Characteristics:**\\n\\n    :Number of Instances: 150 (50 in each of three classes)\\n    :Number of Attributes: 4 numeric, predictive attributes and the class\\n    :Attribute Information:\\n        - sepal length in cm\\n        - sepal width in cm\\n        - petal length in cm\\n        - petal width in cm\\n        - class:\\n                - Iris-Setosa\\n                - Iris-Versicolour\\n                - Iris-Virginica\\n                \\n    :Summary Statistics:\\n\\n    ============== ==== ==== ======= ===== ====================\\n                    Min  Max   Mean    SD   Class Correlation\\n    ============== ==== ==== ======= ===== ====================\\n    sepal length:   4.3  7.9   5.84   0.83    0.7826\\n    sepal width:    2.0  4.4   3.05   0.43   -0.4194\\n    petal length:   1.0  6.9   3.76   1.76    0.9490  (high!)\\n    petal width:    0.1  2.5   1.20   0.76    0.9565  (high!)\\n    ============== ==== ==== ======= ===== ====================\\n\\n    :Missing Attribute Values: None\\n    :Class Distribution: 33.3% for each of 3 classes.\\n    :Creator: R.A. Fisher\\n    :Donor: Michael Marshall (MARSHALL%PLU@io.arc.nasa.gov)\\n    :Date: July, 1988\\n\\nThe famous Iris database, first used by Sir R.A. Fisher. The dataset is taken\\nfrom Fisher\\'s paper. Note that it\\'s the same as in R, but not as in the UCI\\nMachine Learning Repository, which has two wrong data points.\\n\\nThis is perhaps the best known database to be found in the\\npattern recognition literature.  Fisher\\'s paper is a classic in the field and\\nis referenced frequently to this day.  (See Duda & Hart, for example.)  The\\ndata set contains 3 classes of 50 instances each, where each class refers to a\\ntype of iris plant.  One class is linearly separable from the other 2; the\\nlatter are NOT linearly separable from each other.\\n\\n.. topic:: References\\n\\n   - Fisher, R.A. \"The use of multiple measurements in taxonomic problems\"\\n     Annual Eugenics, 7, Part II, 179-188 (1936); also in \"Contributions to\\n     Mathematical Statistics\" (John Wiley, NY, 1950).\\n   - Duda, R.O., & Hart, P.E. (1973) Pattern Classification and Scene Analysis.\\n     (Q327.D83) John Wiley & Sons.  ISBN 0-471-22361-1.  See page 218.\\n   - Dasarathy, B.V. (1980) \"Nosing Around the Neighborhood: A New System\\n     Structure and Classification Rule for Recognition in Partially Exposed\\n     Environments\".  IEEE Transactions on Pattern Analysis and Machine\\n     Intelligence, Vol. PAMI-2, No. 1, 67-71.\\n   - Gates, G.W. (1972) \"The Reduced Nearest Neighbor Rule\".  IEEE Transactions\\n     on Information Theory, May 1972, 431-433.\\n   - See also: 1988 MLC Proceedings, 54-64.  Cheeseman et al\"s AUTOCLASS II\\n     conceptual clustering system finds 3 classes in the data.\\n   - Many, many more ...', 'feature_names': ['sepal length (cm)', 'sepal width (cm)', 'petal length (cm)', 'petal width (cm)'], 'filename': 'iris.csv', 'data_module': 'sklearn.datasets.data'}\n",
      "Information about Iris Dataset: .. _iris_dataset:\n",
      "\n",
      "Iris plants dataset\n",
      "--------------------\n",
      "\n",
      "**Data Set Characteristics:**\n",
      "\n",
      "    :Number of Instances: 150 (50 in each of three classes)\n",
      "    :Number of Attributes: 4 numeric, predictive attributes and the class\n",
      "    :Attribute Information:\n",
      "        - sepal length in cm\n",
      "        - sepal width in cm\n",
      "        - petal length in cm\n",
      "        - petal width in cm\n",
      "        - class:\n",
      "                - Iris-Setosa\n",
      "                - Iris-Versicolour\n",
      "                - Iris-Virginica\n",
      "                \n",
      "    :Summary Statistics:\n",
      "\n",
      "    ============== ==== ==== ======= ===== ====================\n",
      "                    Min  Max   Mean    SD   Class Correlation\n",
      "    ============== ==== ==== ======= ===== ====================\n",
      "    sepal length:   4.3  7.9   5.84   0.83    0.7826\n",
      "    sepal width:    2.0  4.4   3.05   0.43   -0.4194\n",
      "    petal length:   1.0  6.9   3.76   1.76    0.9490  (high!)\n",
      "    petal width:    0.1  2.5   1.20   0.76    0.9565  (high!)\n",
      "    ============== ==== ==== ======= ===== ====================\n",
      "\n",
      "    :Missing Attribute Values: None\n",
      "    :Class Distribution: 33.3% for each of 3 classes.\n",
      "    :Creator: R.A. Fisher\n",
      "    :Donor: Michael Marshall (MARSHALL%PLU@io.arc.nasa.gov)\n",
      "    :Date: July, 1988\n",
      "\n",
      "The famous Iris database, first used by Sir R.A. Fisher. The dataset is taken\n",
      "from Fisher's paper. Note that it's the same as in R, but not as in the UCI\n",
      "Machine Learning Repository, which has two wrong data points.\n",
      "\n",
      "This is perhaps the best known database to be found in the\n",
      "pattern recognition literature.  Fisher's paper is a classic in the field and\n",
      "is referenced frequently to this day.  (See Duda & Hart, for example.)  The\n",
      "data set contains 3 classes of 50 instances each, where each class refers to a\n",
      "type of iris plant.  One class is linearly separable from the other 2; the\n",
      "latter are NOT linearly separable from each other.\n",
      "\n",
      ".. topic:: References\n",
      "\n",
      "   - Fisher, R.A. \"The use of multiple measurements in taxonomic problems\"\n",
      "     Annual Eugenics, 7, Part II, 179-188 (1936); also in \"Contributions to\n",
      "     Mathematical Statistics\" (John Wiley, NY, 1950).\n",
      "   - Duda, R.O., & Hart, P.E. (1973) Pattern Classification and Scene Analysis.\n",
      "     (Q327.D83) John Wiley & Sons.  ISBN 0-471-22361-1.  See page 218.\n",
      "   - Dasarathy, B.V. (1980) \"Nosing Around the Neighborhood: A New System\n",
      "     Structure and Classification Rule for Recognition in Partially Exposed\n",
      "     Environments\".  IEEE Transactions on Pattern Analysis and Machine\n",
      "     Intelligence, Vol. PAMI-2, No. 1, 67-71.\n",
      "   - Gates, G.W. (1972) \"The Reduced Nearest Neighbor Rule\".  IEEE Transactions\n",
      "     on Information Theory, May 1972, 431-433.\n",
      "   - See also: 1988 MLC Proceedings, 54-64.  Cheeseman et al\"s AUTOCLASS II\n",
      "     conceptual clustering system finds 3 classes in the data.\n",
      "   - Many, many more ...\n"
     ]
    }
   ],
   "source": [
    "iris_data=load_iris()\n",
    "# Multidimensional array\n",
    "# For information about dataset\n",
    "print(\"Iris Dataset:\\n\",iris_data)\n",
    "print(\"Information about Iris Dataset:\",iris_data['DESCR']) # -To get information about data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " Splitting Dataset into Input and Output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input is:\n",
      " [[5.1 3.5 1.4 0.2]\n",
      " [4.9 3.  1.4 0.2]\n",
      " [4.7 3.2 1.3 0.2]\n",
      " [4.6 3.1 1.5 0.2]\n",
      " [5.  3.6 1.4 0.2]\n",
      " [5.4 3.9 1.7 0.4]\n",
      " [4.6 3.4 1.4 0.3]\n",
      " [5.  3.4 1.5 0.2]\n",
      " [4.4 2.9 1.4 0.2]\n",
      " [4.9 3.1 1.5 0.1]\n",
      " [5.4 3.7 1.5 0.2]\n",
      " [4.8 3.4 1.6 0.2]\n",
      " [4.8 3.  1.4 0.1]\n",
      " [4.3 3.  1.1 0.1]\n",
      " [5.8 4.  1.2 0.2]\n",
      " [5.7 4.4 1.5 0.4]\n",
      " [5.4 3.9 1.3 0.4]\n",
      " [5.1 3.5 1.4 0.3]\n",
      " [5.7 3.8 1.7 0.3]\n",
      " [5.1 3.8 1.5 0.3]\n",
      " [5.4 3.4 1.7 0.2]\n",
      " [5.1 3.7 1.5 0.4]\n",
      " [4.6 3.6 1.  0.2]\n",
      " [5.1 3.3 1.7 0.5]\n",
      " [4.8 3.4 1.9 0.2]\n",
      " [5.  3.  1.6 0.2]\n",
      " [5.  3.4 1.6 0.4]\n",
      " [5.2 3.5 1.5 0.2]\n",
      " [5.2 3.4 1.4 0.2]\n",
      " [4.7 3.2 1.6 0.2]\n",
      " [4.8 3.1 1.6 0.2]\n",
      " [5.4 3.4 1.5 0.4]\n",
      " [5.2 4.1 1.5 0.1]\n",
      " [5.5 4.2 1.4 0.2]\n",
      " [4.9 3.1 1.5 0.2]\n",
      " [5.  3.2 1.2 0.2]\n",
      " [5.5 3.5 1.3 0.2]\n",
      " [4.9 3.6 1.4 0.1]\n",
      " [4.4 3.  1.3 0.2]\n",
      " [5.1 3.4 1.5 0.2]\n",
      " [5.  3.5 1.3 0.3]\n",
      " [4.5 2.3 1.3 0.3]\n",
      " [4.4 3.2 1.3 0.2]\n",
      " [5.  3.5 1.6 0.6]\n",
      " [5.1 3.8 1.9 0.4]\n",
      " [4.8 3.  1.4 0.3]\n",
      " [5.1 3.8 1.6 0.2]\n",
      " [4.6 3.2 1.4 0.2]\n",
      " [5.3 3.7 1.5 0.2]\n",
      " [5.  3.3 1.4 0.2]\n",
      " [7.  3.2 4.7 1.4]\n",
      " [6.4 3.2 4.5 1.5]\n",
      " [6.9 3.1 4.9 1.5]\n",
      " [5.5 2.3 4.  1.3]\n",
      " [6.5 2.8 4.6 1.5]\n",
      " [5.7 2.8 4.5 1.3]\n",
      " [6.3 3.3 4.7 1.6]\n",
      " [4.9 2.4 3.3 1. ]\n",
      " [6.6 2.9 4.6 1.3]\n",
      " [5.2 2.7 3.9 1.4]\n",
      " [5.  2.  3.5 1. ]\n",
      " [5.9 3.  4.2 1.5]\n",
      " [6.  2.2 4.  1. ]\n",
      " [6.1 2.9 4.7 1.4]\n",
      " [5.6 2.9 3.6 1.3]\n",
      " [6.7 3.1 4.4 1.4]\n",
      " [5.6 3.  4.5 1.5]\n",
      " [5.8 2.7 4.1 1. ]\n",
      " [6.2 2.2 4.5 1.5]\n",
      " [5.6 2.5 3.9 1.1]\n",
      " [5.9 3.2 4.8 1.8]\n",
      " [6.1 2.8 4.  1.3]\n",
      " [6.3 2.5 4.9 1.5]\n",
      " [6.1 2.8 4.7 1.2]\n",
      " [6.4 2.9 4.3 1.3]\n",
      " [6.6 3.  4.4 1.4]\n",
      " [6.8 2.8 4.8 1.4]\n",
      " [6.7 3.  5.  1.7]\n",
      " [6.  2.9 4.5 1.5]\n",
      " [5.7 2.6 3.5 1. ]\n",
      " [5.5 2.4 3.8 1.1]\n",
      " [5.5 2.4 3.7 1. ]\n",
      " [5.8 2.7 3.9 1.2]\n",
      " [6.  2.7 5.1 1.6]\n",
      " [5.4 3.  4.5 1.5]\n",
      " [6.  3.4 4.5 1.6]\n",
      " [6.7 3.1 4.7 1.5]\n",
      " [6.3 2.3 4.4 1.3]\n",
      " [5.6 3.  4.1 1.3]\n",
      " [5.5 2.5 4.  1.3]\n",
      " [5.5 2.6 4.4 1.2]\n",
      " [6.1 3.  4.6 1.4]\n",
      " [5.8 2.6 4.  1.2]\n",
      " [5.  2.3 3.3 1. ]\n",
      " [5.6 2.7 4.2 1.3]\n",
      " [5.7 3.  4.2 1.2]\n",
      " [5.7 2.9 4.2 1.3]\n",
      " [6.2 2.9 4.3 1.3]\n",
      " [5.1 2.5 3.  1.1]\n",
      " [5.7 2.8 4.1 1.3]\n",
      " [6.3 3.3 6.  2.5]\n",
      " [5.8 2.7 5.1 1.9]\n",
      " [7.1 3.  5.9 2.1]\n",
      " [6.3 2.9 5.6 1.8]\n",
      " [6.5 3.  5.8 2.2]\n",
      " [7.6 3.  6.6 2.1]\n",
      " [4.9 2.5 4.5 1.7]\n",
      " [7.3 2.9 6.3 1.8]\n",
      " [6.7 2.5 5.8 1.8]\n",
      " [7.2 3.6 6.1 2.5]\n",
      " [6.5 3.2 5.1 2. ]\n",
      " [6.4 2.7 5.3 1.9]\n",
      " [6.8 3.  5.5 2.1]\n",
      " [5.7 2.5 5.  2. ]\n",
      " [5.8 2.8 5.1 2.4]\n",
      " [6.4 3.2 5.3 2.3]\n",
      " [6.5 3.  5.5 1.8]\n",
      " [7.7 3.8 6.7 2.2]\n",
      " [7.7 2.6 6.9 2.3]\n",
      " [6.  2.2 5.  1.5]\n",
      " [6.9 3.2 5.7 2.3]\n",
      " [5.6 2.8 4.9 2. ]\n",
      " [7.7 2.8 6.7 2. ]\n",
      " [6.3 2.7 4.9 1.8]\n",
      " [6.7 3.3 5.7 2.1]\n",
      " [7.2 3.2 6.  1.8]\n",
      " [6.2 2.8 4.8 1.8]\n",
      " [6.1 3.  4.9 1.8]\n",
      " [6.4 2.8 5.6 2.1]\n",
      " [7.2 3.  5.8 1.6]\n",
      " [7.4 2.8 6.1 1.9]\n",
      " [7.9 3.8 6.4 2. ]\n",
      " [6.4 2.8 5.6 2.2]\n",
      " [6.3 2.8 5.1 1.5]\n",
      " [6.1 2.6 5.6 1.4]\n",
      " [7.7 3.  6.1 2.3]\n",
      " [6.3 3.4 5.6 2.4]\n",
      " [6.4 3.1 5.5 1.8]\n",
      " [6.  3.  4.8 1.8]\n",
      " [6.9 3.1 5.4 2.1]\n",
      " [6.7 3.1 5.6 2.4]\n",
      " [6.9 3.1 5.1 2.3]\n",
      " [5.8 2.7 5.1 1.9]\n",
      " [6.8 3.2 5.9 2.3]\n",
      " [6.7 3.3 5.7 2.5]\n",
      " [6.7 3.  5.2 2.3]\n",
      " [6.3 2.5 5.  1.9]\n",
      " [6.5 3.  5.2 2. ]\n",
      " [6.2 3.4 5.4 2.3]\n",
      " [5.9 3.  5.1 1.8]] \n",
      "Output is: [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      " 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
      " 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2\n",
      " 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2\n",
      " 2 2]\n",
      "\n",
      "Shape of Input is: (150, 4) Shape of Output is: (150,)\n"
     ]
    }
   ],
   "source": [
    "X,Y=iris_data['data'],iris_data['target']\n",
    "print(\"Input is:\\n\",X,\"\\nOutput is:\",Y)\n",
    "print(\"\\nShape of Input is:\",X.shape,\"Shape of Output is:\",Y.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3. Importing algorithm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "# Choosing algorithm\n",
    "knn = KNeighborsClassifier()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "4. Training data<br>\n",
    "Takes two parameter input and output<br>\n",
    "It will train the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "KNeighborsClassifier()"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "knn.fit(X,Y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "5. Evaluating the model<br>\n",
    "we will give parameter and it will predict the class<br>\n",
    "Input data in predict"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prediction without target name is: [1]\n",
      "Species is: ['versicolor']\n"
     ]
    }
   ],
   "source": [
    "print(\"Prediction without target name is:\",knn.predict([[3,2,2,3]]))\n",
    "# If we give 1 [] it will give error\n",
    "print(\"Species is:\",iris_data[\"target_names\"][knn.predict([[3,2,2,3]])])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Passing 4 values from X using slicing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "First 4 values of X are: [[5.1 3.5 1.4 0.2]\n",
      " [4.9 3.  1.4 0.2]\n",
      " [4.7 3.2 1.3 0.2]\n",
      " [4.6 3.1 1.5 0.2]]\n",
      "Prediction without target name is: [0 0 0 0]\n",
      "Species is: ['setosa' 'setosa' 'setosa' 'setosa']\n"
     ]
    }
   ],
   "source": [
    "print(\"First 4 values of X are:\",X[:4,:])\n",
    "print(\"Prediction without target name is:\",knn.predict(X[:4,:]))\n",
    "print(\"Species is:\",iris_data[\"target_names\"][knn.predict(X[:4,:])])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Prediction for rows 47 to 51"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prediction without target name is: [0 0 1 1]\n",
      "Species is: ['setosa' 'setosa' 'versicolor' 'versicolor']\n"
     ]
    }
   ],
   "source": [
    "print(\"Prediction without target name is:\",knn.predict(X[48:52,:]))\n",
    "print(\"Species is:\",iris_data[\"target_names\"][knn.predict(X[48:52,:])])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To check the accuracy of model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy of model for iris dataset without splitting into testing and training data is: 96.66666666666667\n"
     ]
    }
   ],
   "source": [
    "print(\"Accuracy of model for iris dataset without splitting into testing and training data is:\",knn.score(X,Y)*100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "breast_cancer data analysis<br>\n",
    "Contains data about tumors and classification is done as malignant and benign"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Data Gathering"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Cancer Dataset\n",
      "Data is:\n",
      " {'data': array([[1.799e+01, 1.038e+01, 1.228e+02, ..., 2.654e-01, 4.601e-01,\n",
      "        1.189e-01],\n",
      "       [2.057e+01, 1.777e+01, 1.329e+02, ..., 1.860e-01, 2.750e-01,\n",
      "        8.902e-02],\n",
      "       [1.969e+01, 2.125e+01, 1.300e+02, ..., 2.430e-01, 3.613e-01,\n",
      "        8.758e-02],\n",
      "       ...,\n",
      "       [1.660e+01, 2.808e+01, 1.083e+02, ..., 1.418e-01, 2.218e-01,\n",
      "        7.820e-02],\n",
      "       [2.060e+01, 2.933e+01, 1.401e+02, ..., 2.650e-01, 4.087e-01,\n",
      "        1.240e-01],\n",
      "       [7.760e+00, 2.454e+01, 4.792e+01, ..., 0.000e+00, 2.871e-01,\n",
      "        7.039e-02]]), 'target': array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1,\n",
      "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0,\n",
      "       0, 0, 1, 0, 1, 1, 1, 1, 1, 0, 0, 1, 0, 0, 1, 1, 1, 1, 0, 1, 0, 0,\n",
      "       1, 1, 1, 1, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 1, 1, 0, 0, 1, 0, 0, 0,\n",
      "       1, 1, 1, 0, 1, 1, 0, 0, 1, 1, 1, 0, 0, 1, 1, 1, 1, 0, 1, 1, 0, 1,\n",
      "       1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 1, 1, 1, 0, 0, 1, 0, 1, 0,\n",
      "       0, 1, 0, 0, 1, 1, 0, 1, 1, 0, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1,\n",
      "       1, 1, 0, 1, 1, 1, 1, 0, 0, 1, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 1,\n",
      "       1, 0, 1, 1, 0, 0, 0, 1, 0, 1, 0, 1, 1, 1, 0, 1, 1, 0, 0, 1, 0, 0,\n",
      "       0, 0, 1, 0, 0, 0, 1, 0, 1, 0, 1, 1, 0, 1, 0, 0, 0, 0, 1, 1, 0, 0,\n",
      "       1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 0, 1, 1, 0, 1, 1, 0, 0, 1, 0, 1, 1,\n",
      "       1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
      "       0, 0, 1, 1, 1, 1, 1, 1, 0, 1, 0, 1, 1, 0, 1, 1, 0, 1, 0, 0, 1, 1,\n",
      "       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 0, 1, 0, 1, 1, 1, 1, 1,\n",
      "       1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 0, 1, 0, 1, 1, 1, 1, 0, 0,\n",
      "       0, 1, 1, 1, 1, 0, 1, 0, 1, 0, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0,\n",
      "       0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 0, 0, 0, 1, 0, 0,\n",
      "       1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 0, 0, 1, 1,\n",
      "       1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 1, 0,\n",
      "       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 0, 0, 1, 0, 1, 1, 1, 1,\n",
      "       1, 0, 1, 1, 0, 1, 0, 1, 1, 0, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0,\n",
      "       1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1,\n",
      "       1, 1, 1, 0, 1, 0, 1, 1, 0, 1, 1, 1, 1, 1, 0, 0, 1, 0, 1, 0, 1, 1,\n",
      "       1, 1, 1, 0, 1, 1, 0, 1, 0, 1, 0, 0, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1,\n",
      "       1, 1, 1, 1, 1, 0, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
      "       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1]), 'frame': None, 'target_names': array(['malignant', 'benign'], dtype='<U9'), 'DESCR': '.. _breast_cancer_dataset:\\n\\nBreast cancer wisconsin (diagnostic) dataset\\n--------------------------------------------\\n\\n**Data Set Characteristics:**\\n\\n    :Number of Instances: 569\\n\\n    :Number of Attributes: 30 numeric, predictive attributes and the class\\n\\n    :Attribute Information:\\n        - radius (mean of distances from center to points on the perimeter)\\n        - texture (standard deviation of gray-scale values)\\n        - perimeter\\n        - area\\n        - smoothness (local variation in radius lengths)\\n        - compactness (perimeter^2 / area - 1.0)\\n        - concavity (severity of concave portions of the contour)\\n        - concave points (number of concave portions of the contour)\\n        - symmetry\\n        - fractal dimension (\"coastline approximation\" - 1)\\n\\n        The mean, standard error, and \"worst\" or largest (mean of the three\\n        worst/largest values) of these features were computed for each image,\\n        resulting in 30 features.  For instance, field 0 is Mean Radius, field\\n        10 is Radius SE, field 20 is Worst Radius.\\n\\n        - class:\\n                - WDBC-Malignant\\n                - WDBC-Benign\\n\\n    :Summary Statistics:\\n\\n    ===================================== ====== ======\\n                                           Min    Max\\n    ===================================== ====== ======\\n    radius (mean):                        6.981  28.11\\n    texture (mean):                       9.71   39.28\\n    perimeter (mean):                     43.79  188.5\\n    area (mean):                          143.5  2501.0\\n    smoothness (mean):                    0.053  0.163\\n    compactness (mean):                   0.019  0.345\\n    concavity (mean):                     0.0    0.427\\n    concave points (mean):                0.0    0.201\\n    symmetry (mean):                      0.106  0.304\\n    fractal dimension (mean):             0.05   0.097\\n    radius (standard error):              0.112  2.873\\n    texture (standard error):             0.36   4.885\\n    perimeter (standard error):           0.757  21.98\\n    area (standard error):                6.802  542.2\\n    smoothness (standard error):          0.002  0.031\\n    compactness (standard error):         0.002  0.135\\n    concavity (standard error):           0.0    0.396\\n    concave points (standard error):      0.0    0.053\\n    symmetry (standard error):            0.008  0.079\\n    fractal dimension (standard error):   0.001  0.03\\n    radius (worst):                       7.93   36.04\\n    texture (worst):                      12.02  49.54\\n    perimeter (worst):                    50.41  251.2\\n    area (worst):                         185.2  4254.0\\n    smoothness (worst):                   0.071  0.223\\n    compactness (worst):                  0.027  1.058\\n    concavity (worst):                    0.0    1.252\\n    concave points (worst):               0.0    0.291\\n    symmetry (worst):                     0.156  0.664\\n    fractal dimension (worst):            0.055  0.208\\n    ===================================== ====== ======\\n\\n    :Missing Attribute Values: None\\n\\n    :Class Distribution: 212 - Malignant, 357 - Benign\\n\\n    :Creator:  Dr. William H. Wolberg, W. Nick Street, Olvi L. Mangasarian\\n\\n    :Donor: Nick Street\\n\\n    :Date: November, 1995\\n\\nThis is a copy of UCI ML Breast Cancer Wisconsin (Diagnostic) datasets.\\nhttps://goo.gl/U2Uwz2\\n\\nFeatures are computed from a digitized image of a fine needle\\naspirate (FNA) of a breast mass.  They describe\\ncharacteristics of the cell nuclei present in the image.\\n\\nSeparating plane described above was obtained using\\nMultisurface Method-Tree (MSM-T) [K. P. Bennett, \"Decision Tree\\nConstruction Via Linear Programming.\" Proceedings of the 4th\\nMidwest Artificial Intelligence and Cognitive Science Society,\\npp. 97-101, 1992], a classification method which uses linear\\nprogramming to construct a decision tree.  Relevant features\\nwere selected using an exhaustive search in the space of 1-4\\nfeatures and 1-3 separating planes.\\n\\nThe actual linear program used to obtain the separating plane\\nin the 3-dimensional space is that described in:\\n[K. P. Bennett and O. L. Mangasarian: \"Robust Linear\\nProgramming Discrimination of Two Linearly Inseparable Sets\",\\nOptimization Methods and Software 1, 1992, 23-34].\\n\\nThis database is also available through the UW CS ftp server:\\n\\nftp ftp.cs.wisc.edu\\ncd math-prog/cpo-dataset/machine-learn/WDBC/\\n\\n.. topic:: References\\n\\n   - W.N. Street, W.H. Wolberg and O.L. Mangasarian. Nuclear feature extraction \\n     for breast tumor diagnosis. IS&T/SPIE 1993 International Symposium on \\n     Electronic Imaging: Science and Technology, volume 1905, pages 861-870,\\n     San Jose, CA, 1993.\\n   - O.L. Mangasarian, W.N. Street and W.H. Wolberg. Breast cancer diagnosis and \\n     prognosis via linear programming. Operations Research, 43(4), pages 570-577, \\n     July-August 1995.\\n   - W.H. Wolberg, W.N. Street, and O.L. Mangasarian. Machine learning techniques\\n     to diagnose breast cancer from fine-needle aspirates. Cancer Letters 77 (1994) \\n     163-171.', 'feature_names': array(['mean radius', 'mean texture', 'mean perimeter', 'mean area',\n",
      "       'mean smoothness', 'mean compactness', 'mean concavity',\n",
      "       'mean concave points', 'mean symmetry', 'mean fractal dimension',\n",
      "       'radius error', 'texture error', 'perimeter error', 'area error',\n",
      "       'smoothness error', 'compactness error', 'concavity error',\n",
      "       'concave points error', 'symmetry error',\n",
      "       'fractal dimension error', 'worst radius', 'worst texture',\n",
      "       'worst perimeter', 'worst area', 'worst smoothness',\n",
      "       'worst compactness', 'worst concavity', 'worst concave points',\n",
      "       'worst symmetry', 'worst fractal dimension'], dtype='<U23'), 'filename': 'breast_cancer.csv', 'data_module': 'sklearn.datasets.data'}\n"
     ]
    }
   ],
   "source": [
    "from sklearn.datasets import load_breast_cancer\n",
    "cancer_data=load_breast_cancer()\n",
    "# Printing data\n",
    "print(\"\\nCancer Dataset\")\n",
    "print(\"Data is:\\n\",cancer_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2. Data Preprocessing<br>\n",
    "No preprocessing needed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Information about Data is:\n",
      " .. _breast_cancer_dataset:\n",
      "\n",
      "Breast cancer wisconsin (diagnostic) dataset\n",
      "--------------------------------------------\n",
      "\n",
      "**Data Set Characteristics:**\n",
      "\n",
      "    :Number of Instances: 569\n",
      "\n",
      "    :Number of Attributes: 30 numeric, predictive attributes and the class\n",
      "\n",
      "    :Attribute Information:\n",
      "        - radius (mean of distances from center to points on the perimeter)\n",
      "        - texture (standard deviation of gray-scale values)\n",
      "        - perimeter\n",
      "        - area\n",
      "        - smoothness (local variation in radius lengths)\n",
      "        - compactness (perimeter^2 / area - 1.0)\n",
      "        - concavity (severity of concave portions of the contour)\n",
      "        - concave points (number of concave portions of the contour)\n",
      "        - symmetry\n",
      "        - fractal dimension (\"coastline approximation\" - 1)\n",
      "\n",
      "        The mean, standard error, and \"worst\" or largest (mean of the three\n",
      "        worst/largest values) of these features were computed for each image,\n",
      "        resulting in 30 features.  For instance, field 0 is Mean Radius, field\n",
      "        10 is Radius SE, field 20 is Worst Radius.\n",
      "\n",
      "        - class:\n",
      "                - WDBC-Malignant\n",
      "                - WDBC-Benign\n",
      "\n",
      "    :Summary Statistics:\n",
      "\n",
      "    ===================================== ====== ======\n",
      "                                           Min    Max\n",
      "    ===================================== ====== ======\n",
      "    radius (mean):                        6.981  28.11\n",
      "    texture (mean):                       9.71   39.28\n",
      "    perimeter (mean):                     43.79  188.5\n",
      "    area (mean):                          143.5  2501.0\n",
      "    smoothness (mean):                    0.053  0.163\n",
      "    compactness (mean):                   0.019  0.345\n",
      "    concavity (mean):                     0.0    0.427\n",
      "    concave points (mean):                0.0    0.201\n",
      "    symmetry (mean):                      0.106  0.304\n",
      "    fractal dimension (mean):             0.05   0.097\n",
      "    radius (standard error):              0.112  2.873\n",
      "    texture (standard error):             0.36   4.885\n",
      "    perimeter (standard error):           0.757  21.98\n",
      "    area (standard error):                6.802  542.2\n",
      "    smoothness (standard error):          0.002  0.031\n",
      "    compactness (standard error):         0.002  0.135\n",
      "    concavity (standard error):           0.0    0.396\n",
      "    concave points (standard error):      0.0    0.053\n",
      "    symmetry (standard error):            0.008  0.079\n",
      "    fractal dimension (standard error):   0.001  0.03\n",
      "    radius (worst):                       7.93   36.04\n",
      "    texture (worst):                      12.02  49.54\n",
      "    perimeter (worst):                    50.41  251.2\n",
      "    area (worst):                         185.2  4254.0\n",
      "    smoothness (worst):                   0.071  0.223\n",
      "    compactness (worst):                  0.027  1.058\n",
      "    concavity (worst):                    0.0    1.252\n",
      "    concave points (worst):               0.0    0.291\n",
      "    symmetry (worst):                     0.156  0.664\n",
      "    fractal dimension (worst):            0.055  0.208\n",
      "    ===================================== ====== ======\n",
      "\n",
      "    :Missing Attribute Values: None\n",
      "\n",
      "    :Class Distribution: 212 - Malignant, 357 - Benign\n",
      "\n",
      "    :Creator:  Dr. William H. Wolberg, W. Nick Street, Olvi L. Mangasarian\n",
      "\n",
      "    :Donor: Nick Street\n",
      "\n",
      "    :Date: November, 1995\n",
      "\n",
      "This is a copy of UCI ML Breast Cancer Wisconsin (Diagnostic) datasets.\n",
      "https://goo.gl/U2Uwz2\n",
      "\n",
      "Features are computed from a digitized image of a fine needle\n",
      "aspirate (FNA) of a breast mass.  They describe\n",
      "characteristics of the cell nuclei present in the image.\n",
      "\n",
      "Separating plane described above was obtained using\n",
      "Multisurface Method-Tree (MSM-T) [K. P. Bennett, \"Decision Tree\n",
      "Construction Via Linear Programming.\" Proceedings of the 4th\n",
      "Midwest Artificial Intelligence and Cognitive Science Society,\n",
      "pp. 97-101, 1992], a classification method which uses linear\n",
      "programming to construct a decision tree.  Relevant features\n",
      "were selected using an exhaustive search in the space of 1-4\n",
      "features and 1-3 separating planes.\n",
      "\n",
      "The actual linear program used to obtain the separating plane\n",
      "in the 3-dimensional space is that described in:\n",
      "[K. P. Bennett and O. L. Mangasarian: \"Robust Linear\n",
      "Programming Discrimination of Two Linearly Inseparable Sets\",\n",
      "Optimization Methods and Software 1, 1992, 23-34].\n",
      "\n",
      "This database is also available through the UW CS ftp server:\n",
      "\n",
      "ftp ftp.cs.wisc.edu\n",
      "cd math-prog/cpo-dataset/machine-learn/WDBC/\n",
      "\n",
      ".. topic:: References\n",
      "\n",
      "   - W.N. Street, W.H. Wolberg and O.L. Mangasarian. Nuclear feature extraction \n",
      "     for breast tumor diagnosis. IS&T/SPIE 1993 International Symposium on \n",
      "     Electronic Imaging: Science and Technology, volume 1905, pages 861-870,\n",
      "     San Jose, CA, 1993.\n",
      "   - O.L. Mangasarian, W.N. Street and W.H. Wolberg. Breast cancer diagnosis and \n",
      "     prognosis via linear programming. Operations Research, 43(4), pages 570-577, \n",
      "     July-August 1995.\n",
      "   - W.H. Wolberg, W.N. Street, and O.L. Mangasarian. Machine learning techniques\n",
      "     to diagnose breast cancer from fine-needle aspirates. Cancer Letters 77 (1994) \n",
      "     163-171.\n"
     ]
    }
   ],
   "source": [
    "print(\"\\nInformation about Data is:\\n\",cancer_data['DESCR'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Seeing dataset in dataframe form"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dataset in Dataframe is:\n",
      "\n",
      "                                   0            1            2           3  \\\n",
      "mean radius                17.990000    20.570000    19.690000   11.420000   \n",
      "mean texture               10.380000    17.770000    21.250000   20.380000   \n",
      "mean perimeter            122.800000   132.900000   130.000000   77.580000   \n",
      "mean area                1001.000000  1326.000000  1203.000000  386.100000   \n",
      "mean smoothness             0.118400     0.084740     0.109600    0.142500   \n",
      "mean compactness            0.277600     0.078640     0.159900    0.283900   \n",
      "mean concavity              0.300100     0.086900     0.197400    0.241400   \n",
      "mean concave points         0.147100     0.070170     0.127900    0.105200   \n",
      "mean symmetry               0.241900     0.181200     0.206900    0.259700   \n",
      "mean fractal dimension      0.078710     0.056670     0.059990    0.097440   \n",
      "radius error                1.095000     0.543500     0.745600    0.495600   \n",
      "texture error               0.905300     0.733900     0.786900    1.156000   \n",
      "perimeter error             8.589000     3.398000     4.585000    3.445000   \n",
      "area error                153.400000    74.080000    94.030000   27.230000   \n",
      "smoothness error            0.006399     0.005225     0.006150    0.009110   \n",
      "compactness error           0.049040     0.013080     0.040060    0.074580   \n",
      "concavity error             0.053730     0.018600     0.038320    0.056610   \n",
      "concave points error        0.015870     0.013400     0.020580    0.018670   \n",
      "symmetry error              0.030030     0.013890     0.022500    0.059630   \n",
      "fractal dimension error     0.006193     0.003532     0.004571    0.009208   \n",
      "worst radius               25.380000    24.990000    23.570000   14.910000   \n",
      "worst texture              17.330000    23.410000    25.530000   26.500000   \n",
      "worst perimeter           184.600000   158.800000   152.500000   98.870000   \n",
      "worst area               2019.000000  1956.000000  1709.000000  567.700000   \n",
      "worst smoothness            0.162200     0.123800     0.144400    0.209800   \n",
      "worst compactness           0.665600     0.186600     0.424500    0.866300   \n",
      "worst concavity             0.711900     0.241600     0.450400    0.686900   \n",
      "worst concave points        0.265400     0.186000     0.243000    0.257500   \n",
      "worst symmetry              0.460100     0.275000     0.361300    0.663800   \n",
      "worst fractal dimension     0.118900     0.089020     0.087580    0.173000   \n",
      "\n",
      "                                   4  \n",
      "mean radius                20.290000  \n",
      "mean texture               14.340000  \n",
      "mean perimeter            135.100000  \n",
      "mean area                1297.000000  \n",
      "mean smoothness             0.100300  \n",
      "mean compactness            0.132800  \n",
      "mean concavity              0.198000  \n",
      "mean concave points         0.104300  \n",
      "mean symmetry               0.180900  \n",
      "mean fractal dimension      0.058830  \n",
      "radius error                0.757200  \n",
      "texture error               0.781300  \n",
      "perimeter error             5.438000  \n",
      "area error                 94.440000  \n",
      "smoothness error            0.011490  \n",
      "compactness error           0.024610  \n",
      "concavity error             0.056880  \n",
      "concave points error        0.018850  \n",
      "symmetry error              0.017560  \n",
      "fractal dimension error     0.005115  \n",
      "worst radius               22.540000  \n",
      "worst texture              16.670000  \n",
      "worst perimeter           152.200000  \n",
      "worst area               1575.000000  \n",
      "worst smoothness            0.137400  \n",
      "worst compactness           0.205000  \n",
      "worst concavity             0.400000  \n",
      "worst concave points        0.162500  \n",
      "worst symmetry              0.236400  \n",
      "worst fractal dimension     0.076780  \n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "data_df = pd.DataFrame(data = cancer_data.data,\n",
    "\t\t\t\t\tcolumns = cancer_data.feature_names)\n",
    "print(\"Dataset in Dataframe is:\\n\")\n",
    "print(data_df.head().T)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " Splitting Dataset into Input and Output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input is:\n",
      " [[1.799e+01 1.038e+01 1.228e+02 ... 2.654e-01 4.601e-01 1.189e-01]\n",
      " [2.057e+01 1.777e+01 1.329e+02 ... 1.860e-01 2.750e-01 8.902e-02]\n",
      " [1.969e+01 2.125e+01 1.300e+02 ... 2.430e-01 3.613e-01 8.758e-02]\n",
      " ...\n",
      " [1.660e+01 2.808e+01 1.083e+02 ... 1.418e-01 2.218e-01 7.820e-02]\n",
      " [2.060e+01 2.933e+01 1.401e+02 ... 2.650e-01 4.087e-01 1.240e-01]\n",
      " [7.760e+00 2.454e+01 4.792e+01 ... 0.000e+00 2.871e-01 7.039e-02]] \n",
      "Output is: [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      " 1 0 0 0 0 0 0 0 0 1 0 1 1 1 1 1 0 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 1 0 0\n",
      " 1 0 1 0 0 1 1 1 0 0 1 0 0 0 1 1 1 0 1 1 0 0 1 1 1 0 0 1 1 1 1 0 1 1 0 1 1\n",
      " 1 1 1 1 1 1 0 0 0 1 0 0 1 1 1 0 0 1 0 1 0 0 1 0 0 1 1 0 1 1 0 1 1 1 1 0 1\n",
      " 1 1 1 1 1 1 1 1 0 1 1 1 1 0 0 1 0 1 1 0 0 1 1 0 0 1 1 1 1 0 1 1 0 0 0 1 0\n",
      " 1 0 1 1 1 0 1 1 0 0 1 0 0 0 0 1 0 0 0 1 0 1 0 1 1 0 1 0 0 0 0 1 1 0 0 1 1\n",
      " 1 0 1 1 1 1 1 0 0 1 1 0 1 1 0 0 1 0 1 1 1 1 0 1 1 1 1 1 0 1 0 0 0 0 0 0 0\n",
      " 0 0 0 0 0 0 0 1 1 1 1 1 1 0 1 0 1 1 0 1 1 0 1 0 0 1 1 1 1 1 1 1 1 1 1 1 1\n",
      " 1 0 1 1 0 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 0 1 0 1 1 1 1 0 0 0 1 1\n",
      " 1 1 0 1 0 1 0 1 1 1 0 1 1 1 1 1 1 1 0 0 0 1 1 1 1 1 1 1 1 1 1 1 0 0 1 0 0\n",
      " 0 1 0 0 1 1 1 1 1 0 1 1 1 1 1 0 1 1 1 0 1 1 0 0 1 1 1 1 1 1 0 1 1 1 1 1 1\n",
      " 1 0 1 1 1 1 1 0 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 0 1 0 0 1 0 1 1 1 1 1 0 1 1\n",
      " 0 1 0 1 1 0 1 0 1 1 1 1 1 1 1 1 0 0 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 0 1\n",
      " 1 1 1 1 1 1 0 1 0 1 1 0 1 1 1 1 1 0 0 1 0 1 0 1 1 1 1 1 0 1 1 0 1 0 1 0 0\n",
      " 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 0 1 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
      " 1 1 1 1 1 1 1 0 0 0 0 0 0 1]\n",
      "Shape of Input is: (569, 30) Shape of Output is: (569,)\n"
     ]
    }
   ],
   "source": [
    "X,Y=cancer_data['data'],cancer_data['target']\n",
    "print(\"Input is:\\n\",X,\"\\nOutput is:\",Y)\n",
    "print(\"Shape of Input is:\",X.shape,\"Shape of Output is:\",Y.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3. Importing algorithm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "# Storing in Variable\n",
    "knn = KNeighborsClassifier()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "4. Training data<br>\n",
    "Takes two parameter input and output<br>\n",
    "It will train the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "KNeighborsClassifier()"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "knn.fit(X,Y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "5. Evaluating<br>\n",
    "Input data in predict"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prediction without target name is: [1]\n",
      "Tumor is: ['benign']\n"
     ]
    }
   ],
   "source": [
    "print(\"Prediction without target name is:\",knn.predict([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1,0,0,1,1,1,1,1,0]]))\n",
    "# If we give 1 [] it will give error\n",
    "print(\"Tumor is:\",cancer_data[\"target_names\"][knn.predict([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1,0,0,1,1,1,1,1,0]])])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Passing 4 values from X using slicing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "First 4 values of X are: [[1.799e+01 1.038e+01 1.228e+02 1.001e+03 1.184e-01 2.776e-01 3.001e-01\n",
      "  1.471e-01 2.419e-01 7.871e-02 1.095e+00 9.053e-01 8.589e+00 1.534e+02\n",
      "  6.399e-03 4.904e-02 5.373e-02 1.587e-02 3.003e-02 6.193e-03 2.538e+01\n",
      "  1.733e+01 1.846e+02 2.019e+03 1.622e-01 6.656e-01 7.119e-01 2.654e-01\n",
      "  4.601e-01 1.189e-01]\n",
      " [2.057e+01 1.777e+01 1.329e+02 1.326e+03 8.474e-02 7.864e-02 8.690e-02\n",
      "  7.017e-02 1.812e-01 5.667e-02 5.435e-01 7.339e-01 3.398e+00 7.408e+01\n",
      "  5.225e-03 1.308e-02 1.860e-02 1.340e-02 1.389e-02 3.532e-03 2.499e+01\n",
      "  2.341e+01 1.588e+02 1.956e+03 1.238e-01 1.866e-01 2.416e-01 1.860e-01\n",
      "  2.750e-01 8.902e-02]\n",
      " [1.969e+01 2.125e+01 1.300e+02 1.203e+03 1.096e-01 1.599e-01 1.974e-01\n",
      "  1.279e-01 2.069e-01 5.999e-02 7.456e-01 7.869e-01 4.585e+00 9.403e+01\n",
      "  6.150e-03 4.006e-02 3.832e-02 2.058e-02 2.250e-02 4.571e-03 2.357e+01\n",
      "  2.553e+01 1.525e+02 1.709e+03 1.444e-01 4.245e-01 4.504e-01 2.430e-01\n",
      "  3.613e-01 8.758e-02]\n",
      " [1.142e+01 2.038e+01 7.758e+01 3.861e+02 1.425e-01 2.839e-01 2.414e-01\n",
      "  1.052e-01 2.597e-01 9.744e-02 4.956e-01 1.156e+00 3.445e+00 2.723e+01\n",
      "  9.110e-03 7.458e-02 5.661e-02 1.867e-02 5.963e-02 9.208e-03 1.491e+01\n",
      "  2.650e+01 9.887e+01 5.677e+02 2.098e-01 8.663e-01 6.869e-01 2.575e-01\n",
      "  6.638e-01 1.730e-01]]\n",
      "Prediction without target name is: [0 0 0 1]\n",
      "Tumor is: is: ['malignant' 'malignant' 'malignant' 'benign']\n"
     ]
    }
   ],
   "source": [
    "print(\"First 4 values of X are:\",X[:4,:])\n",
    "print(\"Prediction without target name is:\",knn.predict(X[:4,:]))\n",
    "print(\"Tumor is: is:\",cancer_data[\"target_names\"][knn.predict(X[:4,:])])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To check the accuracy of model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy of model for breast cancer without splitting into testing and training data is: 94.72759226713534\n"
     ]
    }
   ],
   "source": [
    "print(\"Accuracy of model for breast cancer without splitting into testing and training data is:\",knn.score(X,Y)*100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using wine dataset<br>\n",
    "Contains information about various wine and we need to identify class of wine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Wine Datset:\n",
      "\n",
      "{'data': array([[1.423e+01, 1.710e+00, 2.430e+00, ..., 1.040e+00, 3.920e+00,\n",
      "        1.065e+03],\n",
      "       [1.320e+01, 1.780e+00, 2.140e+00, ..., 1.050e+00, 3.400e+00,\n",
      "        1.050e+03],\n",
      "       [1.316e+01, 2.360e+00, 2.670e+00, ..., 1.030e+00, 3.170e+00,\n",
      "        1.185e+03],\n",
      "       ...,\n",
      "       [1.327e+01, 4.280e+00, 2.260e+00, ..., 5.900e-01, 1.560e+00,\n",
      "        8.350e+02],\n",
      "       [1.317e+01, 2.590e+00, 2.370e+00, ..., 6.000e-01, 1.620e+00,\n",
      "        8.400e+02],\n",
      "       [1.413e+01, 4.100e+00, 2.740e+00, ..., 6.100e-01, 1.600e+00,\n",
      "        5.600e+02]]), 'target': array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
      "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
      "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1,\n",
      "       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
      "       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
      "       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2,\n",
      "       2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,\n",
      "       2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,\n",
      "       2, 2]), 'frame': None, 'target_names': array(['class_0', 'class_1', 'class_2'], dtype='<U7'), 'DESCR': '.. _wine_dataset:\\n\\nWine recognition dataset\\n------------------------\\n\\n**Data Set Characteristics:**\\n\\n    :Number of Instances: 178 (50 in each of three classes)\\n    :Number of Attributes: 13 numeric, predictive attributes and the class\\n    :Attribute Information:\\n \\t\\t- Alcohol\\n \\t\\t- Malic acid\\n \\t\\t- Ash\\n\\t\\t- Alcalinity of ash  \\n \\t\\t- Magnesium\\n\\t\\t- Total phenols\\n \\t\\t- Flavanoids\\n \\t\\t- Nonflavanoid phenols\\n \\t\\t- Proanthocyanins\\n\\t\\t- Color intensity\\n \\t\\t- Hue\\n \\t\\t- OD280/OD315 of diluted wines\\n \\t\\t- Proline\\n\\n    - class:\\n            - class_0\\n            - class_1\\n            - class_2\\n\\t\\t\\n    :Summary Statistics:\\n    \\n    ============================= ==== ===== ======= =====\\n                                   Min   Max   Mean     SD\\n    ============================= ==== ===== ======= =====\\n    Alcohol:                      11.0  14.8    13.0   0.8\\n    Malic Acid:                   0.74  5.80    2.34  1.12\\n    Ash:                          1.36  3.23    2.36  0.27\\n    Alcalinity of Ash:            10.6  30.0    19.5   3.3\\n    Magnesium:                    70.0 162.0    99.7  14.3\\n    Total Phenols:                0.98  3.88    2.29  0.63\\n    Flavanoids:                   0.34  5.08    2.03  1.00\\n    Nonflavanoid Phenols:         0.13  0.66    0.36  0.12\\n    Proanthocyanins:              0.41  3.58    1.59  0.57\\n    Colour Intensity:              1.3  13.0     5.1   2.3\\n    Hue:                          0.48  1.71    0.96  0.23\\n    OD280/OD315 of diluted wines: 1.27  4.00    2.61  0.71\\n    Proline:                       278  1680     746   315\\n    ============================= ==== ===== ======= =====\\n\\n    :Missing Attribute Values: None\\n    :Class Distribution: class_0 (59), class_1 (71), class_2 (48)\\n    :Creator: R.A. Fisher\\n    :Donor: Michael Marshall (MARSHALL%PLU@io.arc.nasa.gov)\\n    :Date: July, 1988\\n\\nThis is a copy of UCI ML Wine recognition datasets.\\nhttps://archive.ics.uci.edu/ml/machine-learning-databases/wine/wine.data\\n\\nThe data is the results of a chemical analysis of wines grown in the same\\nregion in Italy by three different cultivators. There are thirteen different\\nmeasurements taken for different constituents found in the three types of\\nwine.\\n\\nOriginal Owners: \\n\\nForina, M. et al, PARVUS - \\nAn Extendible Package for Data Exploration, Classification and Correlation. \\nInstitute of Pharmaceutical and Food Analysis and Technologies,\\nVia Brigata Salerno, 16147 Genoa, Italy.\\n\\nCitation:\\n\\nLichman, M. (2013). UCI Machine Learning Repository\\n[https://archive.ics.uci.edu/ml]. Irvine, CA: University of California,\\nSchool of Information and Computer Science. \\n\\n.. topic:: References\\n\\n  (1) S. Aeberhard, D. Coomans and O. de Vel, \\n  Comparison of Classifiers in High Dimensional Settings, \\n  Tech. Rep. no. 92-02, (1992), Dept. of Computer Science and Dept. of  \\n  Mathematics and Statistics, James Cook University of North Queensland. \\n  (Also submitted to Technometrics). \\n\\n  The data was used with many others for comparing various \\n  classifiers. The classes are separable, though only RDA \\n  has achieved 100% correct classification. \\n  (RDA : 100%, QDA 99.4%, LDA 98.9%, 1NN 96.1% (z-transformed data)) \\n  (All results using the leave-one-out technique) \\n\\n  (2) S. Aeberhard, D. Coomans and O. de Vel, \\n  \"THE CLASSIFICATION PERFORMANCE OF RDA\" \\n  Tech. Rep. no. 92-01, (1992), Dept. of Computer Science and Dept. of \\n  Mathematics and Statistics, James Cook University of North Queensland. \\n  (Also submitted to Journal of Chemometrics).\\n', 'feature_names': ['alcohol', 'malic_acid', 'ash', 'alcalinity_of_ash', 'magnesium', 'total_phenols', 'flavanoids', 'nonflavanoid_phenols', 'proanthocyanins', 'color_intensity', 'hue', 'od280/od315_of_diluted_wines', 'proline']}\n",
      "Infomation about data is:\n",
      " .. _wine_dataset:\n",
      "\n",
      "Wine recognition dataset\n",
      "------------------------\n",
      "\n",
      "**Data Set Characteristics:**\n",
      "\n",
      "    :Number of Instances: 178 (50 in each of three classes)\n",
      "    :Number of Attributes: 13 numeric, predictive attributes and the class\n",
      "    :Attribute Information:\n",
      " \t\t- Alcohol\n",
      " \t\t- Malic acid\n",
      " \t\t- Ash\n",
      "\t\t- Alcalinity of ash  \n",
      " \t\t- Magnesium\n",
      "\t\t- Total phenols\n",
      " \t\t- Flavanoids\n",
      " \t\t- Nonflavanoid phenols\n",
      " \t\t- Proanthocyanins\n",
      "\t\t- Color intensity\n",
      " \t\t- Hue\n",
      " \t\t- OD280/OD315 of diluted wines\n",
      " \t\t- Proline\n",
      "\n",
      "    - class:\n",
      "            - class_0\n",
      "            - class_1\n",
      "            - class_2\n",
      "\t\t\n",
      "    :Summary Statistics:\n",
      "    \n",
      "    ============================= ==== ===== ======= =====\n",
      "                                   Min   Max   Mean     SD\n",
      "    ============================= ==== ===== ======= =====\n",
      "    Alcohol:                      11.0  14.8    13.0   0.8\n",
      "    Malic Acid:                   0.74  5.80    2.34  1.12\n",
      "    Ash:                          1.36  3.23    2.36  0.27\n",
      "    Alcalinity of Ash:            10.6  30.0    19.5   3.3\n",
      "    Magnesium:                    70.0 162.0    99.7  14.3\n",
      "    Total Phenols:                0.98  3.88    2.29  0.63\n",
      "    Flavanoids:                   0.34  5.08    2.03  1.00\n",
      "    Nonflavanoid Phenols:         0.13  0.66    0.36  0.12\n",
      "    Proanthocyanins:              0.41  3.58    1.59  0.57\n",
      "    Colour Intensity:              1.3  13.0     5.1   2.3\n",
      "    Hue:                          0.48  1.71    0.96  0.23\n",
      "    OD280/OD315 of diluted wines: 1.27  4.00    2.61  0.71\n",
      "    Proline:                       278  1680     746   315\n",
      "    ============================= ==== ===== ======= =====\n",
      "\n",
      "    :Missing Attribute Values: None\n",
      "    :Class Distribution: class_0 (59), class_1 (71), class_2 (48)\n",
      "    :Creator: R.A. Fisher\n",
      "    :Donor: Michael Marshall (MARSHALL%PLU@io.arc.nasa.gov)\n",
      "    :Date: July, 1988\n",
      "\n",
      "This is a copy of UCI ML Wine recognition datasets.\n",
      "https://archive.ics.uci.edu/ml/machine-learning-databases/wine/wine.data\n",
      "\n",
      "The data is the results of a chemical analysis of wines grown in the same\n",
      "region in Italy by three different cultivators. There are thirteen different\n",
      "measurements taken for different constituents found in the three types of\n",
      "wine.\n",
      "\n",
      "Original Owners: \n",
      "\n",
      "Forina, M. et al, PARVUS - \n",
      "An Extendible Package for Data Exploration, Classification and Correlation. \n",
      "Institute of Pharmaceutical and Food Analysis and Technologies,\n",
      "Via Brigata Salerno, 16147 Genoa, Italy.\n",
      "\n",
      "Citation:\n",
      "\n",
      "Lichman, M. (2013). UCI Machine Learning Repository\n",
      "[https://archive.ics.uci.edu/ml]. Irvine, CA: University of California,\n",
      "School of Information and Computer Science. \n",
      "\n",
      ".. topic:: References\n",
      "\n",
      "  (1) S. Aeberhard, D. Coomans and O. de Vel, \n",
      "  Comparison of Classifiers in High Dimensional Settings, \n",
      "  Tech. Rep. no. 92-02, (1992), Dept. of Computer Science and Dept. of  \n",
      "  Mathematics and Statistics, James Cook University of North Queensland. \n",
      "  (Also submitted to Technometrics). \n",
      "\n",
      "  The data was used with many others for comparing various \n",
      "  classifiers. The classes are separable, though only RDA \n",
      "  has achieved 100% correct classification. \n",
      "  (RDA : 100%, QDA 99.4%, LDA 98.9%, 1NN 96.1% (z-transformed data)) \n",
      "  (All results using the leave-one-out technique) \n",
      "\n",
      "  (2) S. Aeberhard, D. Coomans and O. de Vel, \n",
      "  \"THE CLASSIFICATION PERFORMANCE OF RDA\" \n",
      "  Tech. Rep. no. 92-01, (1992), Dept. of Computer Science and Dept. of \n",
      "  Mathematics and Statistics, James Cook University of North Queensland. \n",
      "  (Also submitted to Journal of Chemometrics).\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from sklearn.datasets import load_wine\n",
    "wine_data=load_wine()\n",
    "print(\"\\nWine Datset:\\n\")\n",
    "print(wine_data)\n",
    "print(\"Infomation about data is:\\n\",wine_data['DESCR'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " Splitting Dataset into Input and Output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input is:\n",
      " [[1.423e+01 1.710e+00 2.430e+00 ... 1.040e+00 3.920e+00 1.065e+03]\n",
      " [1.320e+01 1.780e+00 2.140e+00 ... 1.050e+00 3.400e+00 1.050e+03]\n",
      " [1.316e+01 2.360e+00 2.670e+00 ... 1.030e+00 3.170e+00 1.185e+03]\n",
      " ...\n",
      " [1.327e+01 4.280e+00 2.260e+00 ... 5.900e-01 1.560e+00 8.350e+02]\n",
      " [1.317e+01 2.590e+00 2.370e+00 ... 6.000e-01 1.620e+00 8.400e+02]\n",
      " [1.413e+01 4.100e+00 2.740e+00 ... 6.100e-01 1.600e+00 5.600e+02]] \n",
      "Output is: [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      " 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
      " 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
      " 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2\n",
      " 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2]\n",
      "Shape of Input is: (178, 13) Shape of Output is: (178,)\n"
     ]
    }
   ],
   "source": [
    "X,Y=wine_data['data'],wine_data['target']\n",
    "print(\"Input is:\\n\",X,\"\\nOutput is:\",Y)\n",
    "print(\"Shape of Input is:\",X.shape,\"Shape of Output is:\",Y.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3. Importing algorithm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "# Storing in Variable\n",
    "knn = KNeighborsClassifier()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "4. Training data<br>\n",
    "Takes two parameter input and output<br>\n",
    "It will train the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "KNeighborsClassifier()"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "knn.fit(X,Y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "5. Evaluating<br>\n",
    "Input data in predict"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prediction without target name is: [1]\n",
      "Wine is of class: ['class_1']\n"
     ]
    }
   ],
   "source": [
    "print(\"Prediction without target name is:\",knn.predict([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]]))\n",
    "# If we give 1 [] it will give error\n",
    "print(\"Wine is of class:\",wine_data[\"target_names\"][knn.predict([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]])])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Passing 4 values from X using slicing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "First 4 values of X are: [[1.423e+01 1.710e+00 2.430e+00 1.560e+01 1.270e+02 2.800e+00 3.060e+00\n",
      "  2.800e-01 2.290e+00 5.640e+00 1.040e+00 3.920e+00 1.065e+03]\n",
      " [1.320e+01 1.780e+00 2.140e+00 1.120e+01 1.000e+02 2.650e+00 2.760e+00\n",
      "  2.600e-01 1.280e+00 4.380e+00 1.050e+00 3.400e+00 1.050e+03]\n",
      " [1.316e+01 2.360e+00 2.670e+00 1.860e+01 1.010e+02 2.800e+00 3.240e+00\n",
      "  3.000e-01 2.810e+00 5.680e+00 1.030e+00 3.170e+00 1.185e+03]\n",
      " [1.437e+01 1.950e+00 2.500e+00 1.680e+01 1.130e+02 3.850e+00 3.490e+00\n",
      "  2.400e-01 2.180e+00 7.800e+00 8.600e-01 3.450e+00 1.480e+03]]\n",
      "Prediction without target name is: [0 0 0 0]\n",
      "Wine is of class: ['class_0' 'class_0' 'class_0' 'class_0']\n"
     ]
    }
   ],
   "source": [
    "print(\"First 4 values of X are:\",X[:4,:])\n",
    "print(\"Prediction without target name is:\",knn.predict(X[:4,:]))\n",
    "print(\"Wine is of class:\",wine_data[\"target_names\"][knn.predict(X[:4,:])])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To check the accuracy of model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy of model for wine without splitting into testing and training data is: 78.65168539325843\n"
     ]
    }
   ],
   "source": [
    "print(\"Accuracy of model for wine without splitting into testing and training data is:\",knn.score(X,Y)*100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "<br>\n",
    "Checking for accuracy score<br>\n",
    "    First we split input as training and testing data<br>\n",
    "--> 80% is training data and 20% is testing data<br>\n",
    "--> x_train is known data and y_train is unknown data<br>\n",
    "--> Now, in accuracy score we check accuracy between known and unknown data<br>\n",
    "--> For training data it sometimes gives 100% accuracy but this accuracy is less for actual/testing data<br>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Checking Training and Testing accuracy for data by splitting into test and train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import accuracy_score\n",
    "# train_test split is used to split data in two parts\n",
    "from sklearn.model_selection import train_test_split\n",
    "# Importing algorithm\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "# For printing confusion Matrix\n",
    "from sklearn.metrics import confusion_matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For iris Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iris Dataset:\n",
      " {'data': array([[5.1, 3.5, 1.4, 0.2],\n",
      "       [4.9, 3. , 1.4, 0.2],\n",
      "       [4.7, 3.2, 1.3, 0.2],\n",
      "       [4.6, 3.1, 1.5, 0.2],\n",
      "       [5. , 3.6, 1.4, 0.2],\n",
      "       [5.4, 3.9, 1.7, 0.4],\n",
      "       [4.6, 3.4, 1.4, 0.3],\n",
      "       [5. , 3.4, 1.5, 0.2],\n",
      "       [4.4, 2.9, 1.4, 0.2],\n",
      "       [4.9, 3.1, 1.5, 0.1],\n",
      "       [5.4, 3.7, 1.5, 0.2],\n",
      "       [4.8, 3.4, 1.6, 0.2],\n",
      "       [4.8, 3. , 1.4, 0.1],\n",
      "       [4.3, 3. , 1.1, 0.1],\n",
      "       [5.8, 4. , 1.2, 0.2],\n",
      "       [5.7, 4.4, 1.5, 0.4],\n",
      "       [5.4, 3.9, 1.3, 0.4],\n",
      "       [5.1, 3.5, 1.4, 0.3],\n",
      "       [5.7, 3.8, 1.7, 0.3],\n",
      "       [5.1, 3.8, 1.5, 0.3],\n",
      "       [5.4, 3.4, 1.7, 0.2],\n",
      "       [5.1, 3.7, 1.5, 0.4],\n",
      "       [4.6, 3.6, 1. , 0.2],\n",
      "       [5.1, 3.3, 1.7, 0.5],\n",
      "       [4.8, 3.4, 1.9, 0.2],\n",
      "       [5. , 3. , 1.6, 0.2],\n",
      "       [5. , 3.4, 1.6, 0.4],\n",
      "       [5.2, 3.5, 1.5, 0.2],\n",
      "       [5.2, 3.4, 1.4, 0.2],\n",
      "       [4.7, 3.2, 1.6, 0.2],\n",
      "       [4.8, 3.1, 1.6, 0.2],\n",
      "       [5.4, 3.4, 1.5, 0.4],\n",
      "       [5.2, 4.1, 1.5, 0.1],\n",
      "       [5.5, 4.2, 1.4, 0.2],\n",
      "       [4.9, 3.1, 1.5, 0.2],\n",
      "       [5. , 3.2, 1.2, 0.2],\n",
      "       [5.5, 3.5, 1.3, 0.2],\n",
      "       [4.9, 3.6, 1.4, 0.1],\n",
      "       [4.4, 3. , 1.3, 0.2],\n",
      "       [5.1, 3.4, 1.5, 0.2],\n",
      "       [5. , 3.5, 1.3, 0.3],\n",
      "       [4.5, 2.3, 1.3, 0.3],\n",
      "       [4.4, 3.2, 1.3, 0.2],\n",
      "       [5. , 3.5, 1.6, 0.6],\n",
      "       [5.1, 3.8, 1.9, 0.4],\n",
      "       [4.8, 3. , 1.4, 0.3],\n",
      "       [5.1, 3.8, 1.6, 0.2],\n",
      "       [4.6, 3.2, 1.4, 0.2],\n",
      "       [5.3, 3.7, 1.5, 0.2],\n",
      "       [5. , 3.3, 1.4, 0.2],\n",
      "       [7. , 3.2, 4.7, 1.4],\n",
      "       [6.4, 3.2, 4.5, 1.5],\n",
      "       [6.9, 3.1, 4.9, 1.5],\n",
      "       [5.5, 2.3, 4. , 1.3],\n",
      "       [6.5, 2.8, 4.6, 1.5],\n",
      "       [5.7, 2.8, 4.5, 1.3],\n",
      "       [6.3, 3.3, 4.7, 1.6],\n",
      "       [4.9, 2.4, 3.3, 1. ],\n",
      "       [6.6, 2.9, 4.6, 1.3],\n",
      "       [5.2, 2.7, 3.9, 1.4],\n",
      "       [5. , 2. , 3.5, 1. ],\n",
      "       [5.9, 3. , 4.2, 1.5],\n",
      "       [6. , 2.2, 4. , 1. ],\n",
      "       [6.1, 2.9, 4.7, 1.4],\n",
      "       [5.6, 2.9, 3.6, 1.3],\n",
      "       [6.7, 3.1, 4.4, 1.4],\n",
      "       [5.6, 3. , 4.5, 1.5],\n",
      "       [5.8, 2.7, 4.1, 1. ],\n",
      "       [6.2, 2.2, 4.5, 1.5],\n",
      "       [5.6, 2.5, 3.9, 1.1],\n",
      "       [5.9, 3.2, 4.8, 1.8],\n",
      "       [6.1, 2.8, 4. , 1.3],\n",
      "       [6.3, 2.5, 4.9, 1.5],\n",
      "       [6.1, 2.8, 4.7, 1.2],\n",
      "       [6.4, 2.9, 4.3, 1.3],\n",
      "       [6.6, 3. , 4.4, 1.4],\n",
      "       [6.8, 2.8, 4.8, 1.4],\n",
      "       [6.7, 3. , 5. , 1.7],\n",
      "       [6. , 2.9, 4.5, 1.5],\n",
      "       [5.7, 2.6, 3.5, 1. ],\n",
      "       [5.5, 2.4, 3.8, 1.1],\n",
      "       [5.5, 2.4, 3.7, 1. ],\n",
      "       [5.8, 2.7, 3.9, 1.2],\n",
      "       [6. , 2.7, 5.1, 1.6],\n",
      "       [5.4, 3. , 4.5, 1.5],\n",
      "       [6. , 3.4, 4.5, 1.6],\n",
      "       [6.7, 3.1, 4.7, 1.5],\n",
      "       [6.3, 2.3, 4.4, 1.3],\n",
      "       [5.6, 3. , 4.1, 1.3],\n",
      "       [5.5, 2.5, 4. , 1.3],\n",
      "       [5.5, 2.6, 4.4, 1.2],\n",
      "       [6.1, 3. , 4.6, 1.4],\n",
      "       [5.8, 2.6, 4. , 1.2],\n",
      "       [5. , 2.3, 3.3, 1. ],\n",
      "       [5.6, 2.7, 4.2, 1.3],\n",
      "       [5.7, 3. , 4.2, 1.2],\n",
      "       [5.7, 2.9, 4.2, 1.3],\n",
      "       [6.2, 2.9, 4.3, 1.3],\n",
      "       [5.1, 2.5, 3. , 1.1],\n",
      "       [5.7, 2.8, 4.1, 1.3],\n",
      "       [6.3, 3.3, 6. , 2.5],\n",
      "       [5.8, 2.7, 5.1, 1.9],\n",
      "       [7.1, 3. , 5.9, 2.1],\n",
      "       [6.3, 2.9, 5.6, 1.8],\n",
      "       [6.5, 3. , 5.8, 2.2],\n",
      "       [7.6, 3. , 6.6, 2.1],\n",
      "       [4.9, 2.5, 4.5, 1.7],\n",
      "       [7.3, 2.9, 6.3, 1.8],\n",
      "       [6.7, 2.5, 5.8, 1.8],\n",
      "       [7.2, 3.6, 6.1, 2.5],\n",
      "       [6.5, 3.2, 5.1, 2. ],\n",
      "       [6.4, 2.7, 5.3, 1.9],\n",
      "       [6.8, 3. , 5.5, 2.1],\n",
      "       [5.7, 2.5, 5. , 2. ],\n",
      "       [5.8, 2.8, 5.1, 2.4],\n",
      "       [6.4, 3.2, 5.3, 2.3],\n",
      "       [6.5, 3. , 5.5, 1.8],\n",
      "       [7.7, 3.8, 6.7, 2.2],\n",
      "       [7.7, 2.6, 6.9, 2.3],\n",
      "       [6. , 2.2, 5. , 1.5],\n",
      "       [6.9, 3.2, 5.7, 2.3],\n",
      "       [5.6, 2.8, 4.9, 2. ],\n",
      "       [7.7, 2.8, 6.7, 2. ],\n",
      "       [6.3, 2.7, 4.9, 1.8],\n",
      "       [6.7, 3.3, 5.7, 2.1],\n",
      "       [7.2, 3.2, 6. , 1.8],\n",
      "       [6.2, 2.8, 4.8, 1.8],\n",
      "       [6.1, 3. , 4.9, 1.8],\n",
      "       [6.4, 2.8, 5.6, 2.1],\n",
      "       [7.2, 3. , 5.8, 1.6],\n",
      "       [7.4, 2.8, 6.1, 1.9],\n",
      "       [7.9, 3.8, 6.4, 2. ],\n",
      "       [6.4, 2.8, 5.6, 2.2],\n",
      "       [6.3, 2.8, 5.1, 1.5],\n",
      "       [6.1, 2.6, 5.6, 1.4],\n",
      "       [7.7, 3. , 6.1, 2.3],\n",
      "       [6.3, 3.4, 5.6, 2.4],\n",
      "       [6.4, 3.1, 5.5, 1.8],\n",
      "       [6. , 3. , 4.8, 1.8],\n",
      "       [6.9, 3.1, 5.4, 2.1],\n",
      "       [6.7, 3.1, 5.6, 2.4],\n",
      "       [6.9, 3.1, 5.1, 2.3],\n",
      "       [5.8, 2.7, 5.1, 1.9],\n",
      "       [6.8, 3.2, 5.9, 2.3],\n",
      "       [6.7, 3.3, 5.7, 2.5],\n",
      "       [6.7, 3. , 5.2, 2.3],\n",
      "       [6.3, 2.5, 5. , 1.9],\n",
      "       [6.5, 3. , 5.2, 2. ],\n",
      "       [6.2, 3.4, 5.4, 2.3],\n",
      "       [5.9, 3. , 5.1, 1.8]]), 'target': array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
      "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
      "       0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
      "       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
      "       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,\n",
      "       2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,\n",
      "       2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2]), 'frame': None, 'target_names': array(['setosa', 'versicolor', 'virginica'], dtype='<U10'), 'DESCR': '.. _iris_dataset:\\n\\nIris plants dataset\\n--------------------\\n\\n**Data Set Characteristics:**\\n\\n    :Number of Instances: 150 (50 in each of three classes)\\n    :Number of Attributes: 4 numeric, predictive attributes and the class\\n    :Attribute Information:\\n        - sepal length in cm\\n        - sepal width in cm\\n        - petal length in cm\\n        - petal width in cm\\n        - class:\\n                - Iris-Setosa\\n                - Iris-Versicolour\\n                - Iris-Virginica\\n                \\n    :Summary Statistics:\\n\\n    ============== ==== ==== ======= ===== ====================\\n                    Min  Max   Mean    SD   Class Correlation\\n    ============== ==== ==== ======= ===== ====================\\n    sepal length:   4.3  7.9   5.84   0.83    0.7826\\n    sepal width:    2.0  4.4   3.05   0.43   -0.4194\\n    petal length:   1.0  6.9   3.76   1.76    0.9490  (high!)\\n    petal width:    0.1  2.5   1.20   0.76    0.9565  (high!)\\n    ============== ==== ==== ======= ===== ====================\\n\\n    :Missing Attribute Values: None\\n    :Class Distribution: 33.3% for each of 3 classes.\\n    :Creator: R.A. Fisher\\n    :Donor: Michael Marshall (MARSHALL%PLU@io.arc.nasa.gov)\\n    :Date: July, 1988\\n\\nThe famous Iris database, first used by Sir R.A. Fisher. The dataset is taken\\nfrom Fisher\\'s paper. Note that it\\'s the same as in R, but not as in the UCI\\nMachine Learning Repository, which has two wrong data points.\\n\\nThis is perhaps the best known database to be found in the\\npattern recognition literature.  Fisher\\'s paper is a classic in the field and\\nis referenced frequently to this day.  (See Duda & Hart, for example.)  The\\ndata set contains 3 classes of 50 instances each, where each class refers to a\\ntype of iris plant.  One class is linearly separable from the other 2; the\\nlatter are NOT linearly separable from each other.\\n\\n.. topic:: References\\n\\n   - Fisher, R.A. \"The use of multiple measurements in taxonomic problems\"\\n     Annual Eugenics, 7, Part II, 179-188 (1936); also in \"Contributions to\\n     Mathematical Statistics\" (John Wiley, NY, 1950).\\n   - Duda, R.O., & Hart, P.E. (1973) Pattern Classification and Scene Analysis.\\n     (Q327.D83) John Wiley & Sons.  ISBN 0-471-22361-1.  See page 218.\\n   - Dasarathy, B.V. (1980) \"Nosing Around the Neighborhood: A New System\\n     Structure and Classification Rule for Recognition in Partially Exposed\\n     Environments\".  IEEE Transactions on Pattern Analysis and Machine\\n     Intelligence, Vol. PAMI-2, No. 1, 67-71.\\n   - Gates, G.W. (1972) \"The Reduced Nearest Neighbor Rule\".  IEEE Transactions\\n     on Information Theory, May 1972, 431-433.\\n   - See also: 1988 MLC Proceedings, 54-64.  Cheeseman et al\"s AUTOCLASS II\\n     conceptual clustering system finds 3 classes in the data.\\n   - Many, many more ...', 'feature_names': ['sepal length (cm)', 'sepal width (cm)', 'petal length (cm)', 'petal width (cm)'], 'filename': 'iris.csv', 'data_module': 'sklearn.datasets.data'}\n"
     ]
    }
   ],
   "source": [
    "iris_data=load_iris()\n",
    "print(\"Iris Dataset:\\n\",iris_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Separating testing and training data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape of x_train(training data) for iris dataset is: (120, 4) Shape of y_train(testing data) for iris dataset is: (120,)\n"
     ]
    }
   ],
   "source": [
    "X,Y=iris_data['data'],iris_data['target']\n",
    "x_train,x_test,y_train,y_test = train_test_split(X,Y,test_size=0.2)\n",
    "# Printing shape\n",
    "print(\"Shape of x_train(training data) for iris dataset is:\",x_train.shape,\"Shape of y_train(testing data) for iris dataset is:\",y_train.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Choosing algorithm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predicted Data is: [1 1 0 2 1 2 0 0 2 1 0 2 2 1 2 2 0 0 1 0 0 2 1 2 0 2 1 2 0 1]\n"
     ]
    }
   ],
   "source": [
    "knn = KNeighborsClassifier(n_neighbors=13,weights=\"distance\",algorithm=\"ball_tree\")\n",
    "# Training data\n",
    "knn.fit(x_train,y_train)\n",
    "# Testing data\n",
    "predict=knn.predict(x_test)\n",
    "print(\"Predicted Data is:\",predict)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To check the accuracy of model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training Accuracy for iris Dataset: 100.0%\n",
      "Testing Accuracy for iris Dataset:: 93.33333333333333%\n",
      "Confusion Matrix for iris Dataset is:\n",
      " [[10  0  0]\n",
      " [ 0  7  0]\n",
      " [ 0  2 11]]\n"
     ]
    }
   ],
   "source": [
    "print(f\"Training Accuracy for iris Dataset: {knn.score(x_train,y_train)*100}%\")\n",
    "print(f\"Testing Accuracy for iris Dataset:: {accuracy_score(y_test,predict)*100}%\")\n",
    "print(\"Confusion Matrix for iris Dataset is:\\n\",confusion_matrix(y_test,predict))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For breast cancer dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Cancer Dataset:\n",
      " {'data': array([[1.799e+01, 1.038e+01, 1.228e+02, ..., 2.654e-01, 4.601e-01,\n",
      "        1.189e-01],\n",
      "       [2.057e+01, 1.777e+01, 1.329e+02, ..., 1.860e-01, 2.750e-01,\n",
      "        8.902e-02],\n",
      "       [1.969e+01, 2.125e+01, 1.300e+02, ..., 2.430e-01, 3.613e-01,\n",
      "        8.758e-02],\n",
      "       ...,\n",
      "       [1.660e+01, 2.808e+01, 1.083e+02, ..., 1.418e-01, 2.218e-01,\n",
      "        7.820e-02],\n",
      "       [2.060e+01, 2.933e+01, 1.401e+02, ..., 2.650e-01, 4.087e-01,\n",
      "        1.240e-01],\n",
      "       [7.760e+00, 2.454e+01, 4.792e+01, ..., 0.000e+00, 2.871e-01,\n",
      "        7.039e-02]]), 'target': array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1,\n",
      "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0,\n",
      "       0, 0, 1, 0, 1, 1, 1, 1, 1, 0, 0, 1, 0, 0, 1, 1, 1, 1, 0, 1, 0, 0,\n",
      "       1, 1, 1, 1, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 1, 1, 0, 0, 1, 0, 0, 0,\n",
      "       1, 1, 1, 0, 1, 1, 0, 0, 1, 1, 1, 0, 0, 1, 1, 1, 1, 0, 1, 1, 0, 1,\n",
      "       1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 1, 1, 1, 0, 0, 1, 0, 1, 0,\n",
      "       0, 1, 0, 0, 1, 1, 0, 1, 1, 0, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1,\n",
      "       1, 1, 0, 1, 1, 1, 1, 0, 0, 1, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 1,\n",
      "       1, 0, 1, 1, 0, 0, 0, 1, 0, 1, 0, 1, 1, 1, 0, 1, 1, 0, 0, 1, 0, 0,\n",
      "       0, 0, 1, 0, 0, 0, 1, 0, 1, 0, 1, 1, 0, 1, 0, 0, 0, 0, 1, 1, 0, 0,\n",
      "       1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 0, 1, 1, 0, 1, 1, 0, 0, 1, 0, 1, 1,\n",
      "       1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
      "       0, 0, 1, 1, 1, 1, 1, 1, 0, 1, 0, 1, 1, 0, 1, 1, 0, 1, 0, 0, 1, 1,\n",
      "       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 0, 1, 0, 1, 1, 1, 1, 1,\n",
      "       1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 0, 1, 0, 1, 1, 1, 1, 0, 0,\n",
      "       0, 1, 1, 1, 1, 0, 1, 0, 1, 0, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0,\n",
      "       0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 0, 0, 0, 1, 0, 0,\n",
      "       1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 0, 0, 1, 1,\n",
      "       1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 1, 0,\n",
      "       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 0, 0, 1, 0, 1, 1, 1, 1,\n",
      "       1, 0, 1, 1, 0, 1, 0, 1, 1, 0, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0,\n",
      "       1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1,\n",
      "       1, 1, 1, 0, 1, 0, 1, 1, 0, 1, 1, 1, 1, 1, 0, 0, 1, 0, 1, 0, 1, 1,\n",
      "       1, 1, 1, 0, 1, 1, 0, 1, 0, 1, 0, 0, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1,\n",
      "       1, 1, 1, 1, 1, 0, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
      "       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1]), 'frame': None, 'target_names': array(['malignant', 'benign'], dtype='<U9'), 'DESCR': '.. _breast_cancer_dataset:\\n\\nBreast cancer wisconsin (diagnostic) dataset\\n--------------------------------------------\\n\\n**Data Set Characteristics:**\\n\\n    :Number of Instances: 569\\n\\n    :Number of Attributes: 30 numeric, predictive attributes and the class\\n\\n    :Attribute Information:\\n        - radius (mean of distances from center to points on the perimeter)\\n        - texture (standard deviation of gray-scale values)\\n        - perimeter\\n        - area\\n        - smoothness (local variation in radius lengths)\\n        - compactness (perimeter^2 / area - 1.0)\\n        - concavity (severity of concave portions of the contour)\\n        - concave points (number of concave portions of the contour)\\n        - symmetry\\n        - fractal dimension (\"coastline approximation\" - 1)\\n\\n        The mean, standard error, and \"worst\" or largest (mean of the three\\n        worst/largest values) of these features were computed for each image,\\n        resulting in 30 features.  For instance, field 0 is Mean Radius, field\\n        10 is Radius SE, field 20 is Worst Radius.\\n\\n        - class:\\n                - WDBC-Malignant\\n                - WDBC-Benign\\n\\n    :Summary Statistics:\\n\\n    ===================================== ====== ======\\n                                           Min    Max\\n    ===================================== ====== ======\\n    radius (mean):                        6.981  28.11\\n    texture (mean):                       9.71   39.28\\n    perimeter (mean):                     43.79  188.5\\n    area (mean):                          143.5  2501.0\\n    smoothness (mean):                    0.053  0.163\\n    compactness (mean):                   0.019  0.345\\n    concavity (mean):                     0.0    0.427\\n    concave points (mean):                0.0    0.201\\n    symmetry (mean):                      0.106  0.304\\n    fractal dimension (mean):             0.05   0.097\\n    radius (standard error):              0.112  2.873\\n    texture (standard error):             0.36   4.885\\n    perimeter (standard error):           0.757  21.98\\n    area (standard error):                6.802  542.2\\n    smoothness (standard error):          0.002  0.031\\n    compactness (standard error):         0.002  0.135\\n    concavity (standard error):           0.0    0.396\\n    concave points (standard error):      0.0    0.053\\n    symmetry (standard error):            0.008  0.079\\n    fractal dimension (standard error):   0.001  0.03\\n    radius (worst):                       7.93   36.04\\n    texture (worst):                      12.02  49.54\\n    perimeter (worst):                    50.41  251.2\\n    area (worst):                         185.2  4254.0\\n    smoothness (worst):                   0.071  0.223\\n    compactness (worst):                  0.027  1.058\\n    concavity (worst):                    0.0    1.252\\n    concave points (worst):               0.0    0.291\\n    symmetry (worst):                     0.156  0.664\\n    fractal dimension (worst):            0.055  0.208\\n    ===================================== ====== ======\\n\\n    :Missing Attribute Values: None\\n\\n    :Class Distribution: 212 - Malignant, 357 - Benign\\n\\n    :Creator:  Dr. William H. Wolberg, W. Nick Street, Olvi L. Mangasarian\\n\\n    :Donor: Nick Street\\n\\n    :Date: November, 1995\\n\\nThis is a copy of UCI ML Breast Cancer Wisconsin (Diagnostic) datasets.\\nhttps://goo.gl/U2Uwz2\\n\\nFeatures are computed from a digitized image of a fine needle\\naspirate (FNA) of a breast mass.  They describe\\ncharacteristics of the cell nuclei present in the image.\\n\\nSeparating plane described above was obtained using\\nMultisurface Method-Tree (MSM-T) [K. P. Bennett, \"Decision Tree\\nConstruction Via Linear Programming.\" Proceedings of the 4th\\nMidwest Artificial Intelligence and Cognitive Science Society,\\npp. 97-101, 1992], a classification method which uses linear\\nprogramming to construct a decision tree.  Relevant features\\nwere selected using an exhaustive search in the space of 1-4\\nfeatures and 1-3 separating planes.\\n\\nThe actual linear program used to obtain the separating plane\\nin the 3-dimensional space is that described in:\\n[K. P. Bennett and O. L. Mangasarian: \"Robust Linear\\nProgramming Discrimination of Two Linearly Inseparable Sets\",\\nOptimization Methods and Software 1, 1992, 23-34].\\n\\nThis database is also available through the UW CS ftp server:\\n\\nftp ftp.cs.wisc.edu\\ncd math-prog/cpo-dataset/machine-learn/WDBC/\\n\\n.. topic:: References\\n\\n   - W.N. Street, W.H. Wolberg and O.L. Mangasarian. Nuclear feature extraction \\n     for breast tumor diagnosis. IS&T/SPIE 1993 International Symposium on \\n     Electronic Imaging: Science and Technology, volume 1905, pages 861-870,\\n     San Jose, CA, 1993.\\n   - O.L. Mangasarian, W.N. Street and W.H. Wolberg. Breast cancer diagnosis and \\n     prognosis via linear programming. Operations Research, 43(4), pages 570-577, \\n     July-August 1995.\\n   - W.H. Wolberg, W.N. Street, and O.L. Mangasarian. Machine learning techniques\\n     to diagnose breast cancer from fine-needle aspirates. Cancer Letters 77 (1994) \\n     163-171.', 'feature_names': array(['mean radius', 'mean texture', 'mean perimeter', 'mean area',\n",
      "       'mean smoothness', 'mean compactness', 'mean concavity',\n",
      "       'mean concave points', 'mean symmetry', 'mean fractal dimension',\n",
      "       'radius error', 'texture error', 'perimeter error', 'area error',\n",
      "       'smoothness error', 'compactness error', 'concavity error',\n",
      "       'concave points error', 'symmetry error',\n",
      "       'fractal dimension error', 'worst radius', 'worst texture',\n",
      "       'worst perimeter', 'worst area', 'worst smoothness',\n",
      "       'worst compactness', 'worst concavity', 'worst concave points',\n",
      "       'worst symmetry', 'worst fractal dimension'], dtype='<U23'), 'filename': 'breast_cancer.csv', 'data_module': 'sklearn.datasets.data'}\n"
     ]
    }
   ],
   "source": [
    "from sklearn.datasets import load_breast_cancer\n",
    "cancer_data=load_breast_cancer()\n",
    "print(\"\\nCancer Dataset:\\n\",cancer_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Separating testing and training data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape of x_train(training data) for cancer dataset is: (455, 30) Shape of y_train(testing data) for cancer dataset is: (455,)\n"
     ]
    }
   ],
   "source": [
    "X,Y=cancer_data['data'],cancer_data['target']\n",
    "x_train,x_test,y_train,y_test = train_test_split(X,Y,test_size=0.2)\n",
    "print(\"Shape of x_train(training data) for cancer dataset is:\",x_train.shape,\"Shape of y_train(testing data) for cancer dataset is:\",y_train.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Choosing algorithm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predicted Data is: [1 1 1 1 1 0 1 1 0 1 1 1 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 0 1 1 1 1 1 1 1 1 1\n",
      " 1 1 1 1 1 1 0 0 1 1 1 0 0 0 1 0 1 1 1 1 1 0 1 0 1 0 0 1 1 0 0 0 0 1 1 1 1\n",
      " 0 1 0 0 1 1 1 1 1 0 0 0 1 1 1 1 1 1 0 0 1 0 1 0 1 1 1 1 1 0 1 1 0 1 1 1 1\n",
      " 1 1 1]\n"
     ]
    }
   ],
   "source": [
    "knn = KNeighborsClassifier(n_neighbors=13,weights=\"distance\",algorithm=\"ball_tree\")\n",
    "# Training data\n",
    "knn.fit(x_train,y_train)\n",
    "# Testing data\n",
    "predict=knn.predict(x_test)\n",
    "print(\"Predicted Data is:\",predict)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To check the accuracy of model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training Accuracy for Cancer Dataset is: 100.0%\n",
      "Testing Accuracy for Cancer Dataset is: 96.49122807017544%\n",
      "Confusion Matrix is for Cancer Dataset:\n",
      " [[32  3]\n",
      " [ 1 78]]\n"
     ]
    }
   ],
   "source": [
    "print(f\"Training Accuracy for Cancer Dataset is: {knn.score(x_train,y_train)*100}%\")\n",
    "print(f\"Testing Accuracy for Cancer Dataset is: {accuracy_score(y_test,predict)*100}%\")\n",
    "print(\"Confusion Matrix is for Cancer Dataset:\\n\",confusion_matrix(y_test,predict))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For Wine dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Wine Dataset:\n",
      " {'data': array([[1.423e+01, 1.710e+00, 2.430e+00, ..., 1.040e+00, 3.920e+00,\n",
      "        1.065e+03],\n",
      "       [1.320e+01, 1.780e+00, 2.140e+00, ..., 1.050e+00, 3.400e+00,\n",
      "        1.050e+03],\n",
      "       [1.316e+01, 2.360e+00, 2.670e+00, ..., 1.030e+00, 3.170e+00,\n",
      "        1.185e+03],\n",
      "       ...,\n",
      "       [1.327e+01, 4.280e+00, 2.260e+00, ..., 5.900e-01, 1.560e+00,\n",
      "        8.350e+02],\n",
      "       [1.317e+01, 2.590e+00, 2.370e+00, ..., 6.000e-01, 1.620e+00,\n",
      "        8.400e+02],\n",
      "       [1.413e+01, 4.100e+00, 2.740e+00, ..., 6.100e-01, 1.600e+00,\n",
      "        5.600e+02]]), 'target': array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
      "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
      "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1,\n",
      "       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
      "       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
      "       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2,\n",
      "       2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,\n",
      "       2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,\n",
      "       2, 2]), 'frame': None, 'target_names': array(['class_0', 'class_1', 'class_2'], dtype='<U7'), 'DESCR': '.. _wine_dataset:\\n\\nWine recognition dataset\\n------------------------\\n\\n**Data Set Characteristics:**\\n\\n    :Number of Instances: 178 (50 in each of three classes)\\n    :Number of Attributes: 13 numeric, predictive attributes and the class\\n    :Attribute Information:\\n \\t\\t- Alcohol\\n \\t\\t- Malic acid\\n \\t\\t- Ash\\n\\t\\t- Alcalinity of ash  \\n \\t\\t- Magnesium\\n\\t\\t- Total phenols\\n \\t\\t- Flavanoids\\n \\t\\t- Nonflavanoid phenols\\n \\t\\t- Proanthocyanins\\n\\t\\t- Color intensity\\n \\t\\t- Hue\\n \\t\\t- OD280/OD315 of diluted wines\\n \\t\\t- Proline\\n\\n    - class:\\n            - class_0\\n            - class_1\\n            - class_2\\n\\t\\t\\n    :Summary Statistics:\\n    \\n    ============================= ==== ===== ======= =====\\n                                   Min   Max   Mean     SD\\n    ============================= ==== ===== ======= =====\\n    Alcohol:                      11.0  14.8    13.0   0.8\\n    Malic Acid:                   0.74  5.80    2.34  1.12\\n    Ash:                          1.36  3.23    2.36  0.27\\n    Alcalinity of Ash:            10.6  30.0    19.5   3.3\\n    Magnesium:                    70.0 162.0    99.7  14.3\\n    Total Phenols:                0.98  3.88    2.29  0.63\\n    Flavanoids:                   0.34  5.08    2.03  1.00\\n    Nonflavanoid Phenols:         0.13  0.66    0.36  0.12\\n    Proanthocyanins:              0.41  3.58    1.59  0.57\\n    Colour Intensity:              1.3  13.0     5.1   2.3\\n    Hue:                          0.48  1.71    0.96  0.23\\n    OD280/OD315 of diluted wines: 1.27  4.00    2.61  0.71\\n    Proline:                       278  1680     746   315\\n    ============================= ==== ===== ======= =====\\n\\n    :Missing Attribute Values: None\\n    :Class Distribution: class_0 (59), class_1 (71), class_2 (48)\\n    :Creator: R.A. Fisher\\n    :Donor: Michael Marshall (MARSHALL%PLU@io.arc.nasa.gov)\\n    :Date: July, 1988\\n\\nThis is a copy of UCI ML Wine recognition datasets.\\nhttps://archive.ics.uci.edu/ml/machine-learning-databases/wine/wine.data\\n\\nThe data is the results of a chemical analysis of wines grown in the same\\nregion in Italy by three different cultivators. There are thirteen different\\nmeasurements taken for different constituents found in the three types of\\nwine.\\n\\nOriginal Owners: \\n\\nForina, M. et al, PARVUS - \\nAn Extendible Package for Data Exploration, Classification and Correlation. \\nInstitute of Pharmaceutical and Food Analysis and Technologies,\\nVia Brigata Salerno, 16147 Genoa, Italy.\\n\\nCitation:\\n\\nLichman, M. (2013). UCI Machine Learning Repository\\n[https://archive.ics.uci.edu/ml]. Irvine, CA: University of California,\\nSchool of Information and Computer Science. \\n\\n.. topic:: References\\n\\n  (1) S. Aeberhard, D. Coomans and O. de Vel, \\n  Comparison of Classifiers in High Dimensional Settings, \\n  Tech. Rep. no. 92-02, (1992), Dept. of Computer Science and Dept. of  \\n  Mathematics and Statistics, James Cook University of North Queensland. \\n  (Also submitted to Technometrics). \\n\\n  The data was used with many others for comparing various \\n  classifiers. The classes are separable, though only RDA \\n  has achieved 100% correct classification. \\n  (RDA : 100%, QDA 99.4%, LDA 98.9%, 1NN 96.1% (z-transformed data)) \\n  (All results using the leave-one-out technique) \\n\\n  (2) S. Aeberhard, D. Coomans and O. de Vel, \\n  \"THE CLASSIFICATION PERFORMANCE OF RDA\" \\n  Tech. Rep. no. 92-01, (1992), Dept. of Computer Science and Dept. of \\n  Mathematics and Statistics, James Cook University of North Queensland. \\n  (Also submitted to Journal of Chemometrics).\\n', 'feature_names': ['alcohol', 'malic_acid', 'ash', 'alcalinity_of_ash', 'magnesium', 'total_phenols', 'flavanoids', 'nonflavanoid_phenols', 'proanthocyanins', 'color_intensity', 'hue', 'od280/od315_of_diluted_wines', 'proline']}\n"
     ]
    }
   ],
   "source": [
    "from sklearn.datasets import load_wine\n",
    "wine_data=load_wine()\n",
    "print(\"\\nWine Dataset:\\n\",wine_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Separating testing and training data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape of Input is: (178, 13) Shape of Output is: (178,)\n",
      "Shape of x_train(training data) for wine dataset is: (142, 13) Shape of y_train(testing data) for wine dataset is: (142,)\n"
     ]
    }
   ],
   "source": [
    "X,Y=wine_data['data'],wine_data['target']\n",
    "print(\"Shape of Input is:\",X.shape,\"Shape of Output is:\",Y.shape)\n",
    "x_train,x_test,y_train,y_test = train_test_split(X,Y,test_size=0.2)\n",
    "print(\"Shape of x_train(training data) for wine dataset is:\",x_train.shape,\"Shape of y_train(testing data) for wine dataset is:\",y_train.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Choosing algorithm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predicted Data is: [0 2 0 0 0 2 2 1 1 0 2 1 1 0 1 0 0 1 2 2 2 2 0 0 2 0 1 1 2 0 2 1 2 0 1 0]\n"
     ]
    }
   ],
   "source": [
    "knn = KNeighborsClassifier(n_neighbors=13,weights=\"distance\",algorithm=\"ball_tree\")\n",
    "# Training data\n",
    "knn.fit(x_train,y_train)\n",
    "# Testing data\n",
    "predict=knn.predict(x_test)\n",
    "print(\"Predicted Data is:\",predict)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To check the accuracy of model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training Accuracy for Wine Dataset is: 100.0%\n",
      "Testing Accuracy for Wine Dataset is: 75.0%\n"
     ]
    }
   ],
   "source": [
    "print(f\"Training Accuracy for Wine Dataset is: {knn.score(x_train,y_train)*100}%\")\n",
    "print(f\"Testing Accuracy for Wine Dataset is: {accuracy_score(y_test,predict)*100}%\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To get arguments of KNeighbourClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on class KNeighborsClassifier in module sklearn.neighbors._classification:\n",
      "\n",
      "class KNeighborsClassifier(sklearn.neighbors._base.KNeighborsMixin, sklearn.base.ClassifierMixin, sklearn.neighbors._base.NeighborsBase)\n",
      " |  KNeighborsClassifier(n_neighbors=5, *, weights='uniform', algorithm='auto', leaf_size=30, p=2, metric='minkowski', metric_params=None, n_jobs=None)\n",
      " |  \n",
      " |  Classifier implementing the k-nearest neighbors vote.\n",
      " |  \n",
      " |  Read more in the :ref:`User Guide <classification>`.\n",
      " |  \n",
      " |  Parameters\n",
      " |  ----------\n",
      " |  n_neighbors : int, default=5\n",
      " |      Number of neighbors to use by default for :meth:`kneighbors` queries.\n",
      " |  \n",
      " |  weights : {'uniform', 'distance'} or callable, default='uniform'\n",
      " |      Weight function used in prediction.  Possible values:\n",
      " |  \n",
      " |      - 'uniform' : uniform weights.  All points in each neighborhood\n",
      " |        are weighted equally.\n",
      " |      - 'distance' : weight points by the inverse of their distance.\n",
      " |        in this case, closer neighbors of a query point will have a\n",
      " |        greater influence than neighbors which are further away.\n",
      " |      - [callable] : a user-defined function which accepts an\n",
      " |        array of distances, and returns an array of the same shape\n",
      " |        containing the weights.\n",
      " |  \n",
      " |  algorithm : {'auto', 'ball_tree', 'kd_tree', 'brute'}, default='auto'\n",
      " |      Algorithm used to compute the nearest neighbors:\n",
      " |  \n",
      " |      - 'ball_tree' will use :class:`BallTree`\n",
      " |      - 'kd_tree' will use :class:`KDTree`\n",
      " |      - 'brute' will use a brute-force search.\n",
      " |      - 'auto' will attempt to decide the most appropriate algorithm\n",
      " |        based on the values passed to :meth:`fit` method.\n",
      " |  \n",
      " |      Note: fitting on sparse input will override the setting of\n",
      " |      this parameter, using brute force.\n",
      " |  \n",
      " |  leaf_size : int, default=30\n",
      " |      Leaf size passed to BallTree or KDTree.  This can affect the\n",
      " |      speed of the construction and query, as well as the memory\n",
      " |      required to store the tree.  The optimal value depends on the\n",
      " |      nature of the problem.\n",
      " |  \n",
      " |  p : int, default=2\n",
      " |      Power parameter for the Minkowski metric. When p = 1, this is\n",
      " |      equivalent to using manhattan_distance (l1), and euclidean_distance\n",
      " |      (l2) for p = 2. For arbitrary p, minkowski_distance (l_p) is used.\n",
      " |  \n",
      " |  metric : str or callable, default='minkowski'\n",
      " |      The distance metric to use for the tree.  The default metric is\n",
      " |      minkowski, and with p=2 is equivalent to the standard Euclidean\n",
      " |      metric. For a list of available metrics, see the documentation of\n",
      " |      :class:`~sklearn.metrics.DistanceMetric`.\n",
      " |      If metric is \"precomputed\", X is assumed to be a distance matrix and\n",
      " |      must be square during fit. X may be a :term:`sparse graph`,\n",
      " |      in which case only \"nonzero\" elements may be considered neighbors.\n",
      " |  \n",
      " |  metric_params : dict, default=None\n",
      " |      Additional keyword arguments for the metric function.\n",
      " |  \n",
      " |  n_jobs : int, default=None\n",
      " |      The number of parallel jobs to run for neighbors search.\n",
      " |      ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context.\n",
      " |      ``-1`` means using all processors. See :term:`Glossary <n_jobs>`\n",
      " |      for more details.\n",
      " |      Doesn't affect :meth:`fit` method.\n",
      " |  \n",
      " |  Attributes\n",
      " |  ----------\n",
      " |  classes_ : array of shape (n_classes,)\n",
      " |      Class labels known to the classifier\n",
      " |  \n",
      " |  effective_metric_ : str or callble\n",
      " |      The distance metric used. It will be same as the `metric` parameter\n",
      " |      or a synonym of it, e.g. 'euclidean' if the `metric` parameter set to\n",
      " |      'minkowski' and `p` parameter set to 2.\n",
      " |  \n",
      " |  effective_metric_params_ : dict\n",
      " |      Additional keyword arguments for the metric function. For most metrics\n",
      " |      will be same with `metric_params` parameter, but may also contain the\n",
      " |      `p` parameter value if the `effective_metric_` attribute is set to\n",
      " |      'minkowski'.\n",
      " |  \n",
      " |  n_features_in_ : int\n",
      " |      Number of features seen during :term:`fit`.\n",
      " |  \n",
      " |      .. versionadded:: 0.24\n",
      " |  \n",
      " |  feature_names_in_ : ndarray of shape (`n_features_in_`,)\n",
      " |      Names of features seen during :term:`fit`. Defined only when `X`\n",
      " |      has feature names that are all strings.\n",
      " |  \n",
      " |      .. versionadded:: 1.0\n",
      " |  \n",
      " |  n_samples_fit_ : int\n",
      " |      Number of samples in the fitted data.\n",
      " |  \n",
      " |  outputs_2d_ : bool\n",
      " |      False when `y`'s shape is (n_samples, ) or (n_samples, 1) during fit\n",
      " |      otherwise True.\n",
      " |  \n",
      " |  See Also\n",
      " |  --------\n",
      " |  RadiusNeighborsClassifier: Classifier based on neighbors within a fixed radius.\n",
      " |  KNeighborsRegressor: Regression based on k-nearest neighbors.\n",
      " |  RadiusNeighborsRegressor: Regression based on neighbors within a fixed radius.\n",
      " |  NearestNeighbors: Unsupervised learner for implementing neighbor searches.\n",
      " |  \n",
      " |  Notes\n",
      " |  -----\n",
      " |  See :ref:`Nearest Neighbors <neighbors>` in the online documentation\n",
      " |  for a discussion of the choice of ``algorithm`` and ``leaf_size``.\n",
      " |  \n",
      " |  .. warning::\n",
      " |  \n",
      " |     Regarding the Nearest Neighbors algorithms, if it is found that two\n",
      " |     neighbors, neighbor `k+1` and `k`, have identical distances\n",
      " |     but different labels, the results will depend on the ordering of the\n",
      " |     training data.\n",
      " |  \n",
      " |  https://en.wikipedia.org/wiki/K-nearest_neighbor_algorithm\n",
      " |  \n",
      " |  Examples\n",
      " |  --------\n",
      " |  >>> X = [[0], [1], [2], [3]]\n",
      " |  >>> y = [0, 0, 1, 1]\n",
      " |  >>> from sklearn.neighbors import KNeighborsClassifier\n",
      " |  >>> neigh = KNeighborsClassifier(n_neighbors=3)\n",
      " |  >>> neigh.fit(X, y)\n",
      " |  KNeighborsClassifier(...)\n",
      " |  >>> print(neigh.predict([[1.1]]))\n",
      " |  [0]\n",
      " |  >>> print(neigh.predict_proba([[0.9]]))\n",
      " |  [[0.666... 0.333...]]\n",
      " |  \n",
      " |  Method resolution order:\n",
      " |      KNeighborsClassifier\n",
      " |      sklearn.neighbors._base.KNeighborsMixin\n",
      " |      sklearn.base.ClassifierMixin\n",
      " |      sklearn.neighbors._base.NeighborsBase\n",
      " |      sklearn.base.MultiOutputMixin\n",
      " |      sklearn.base.BaseEstimator\n",
      " |      builtins.object\n",
      " |  \n",
      " |  Methods defined here:\n",
      " |  \n",
      " |  __init__(self, n_neighbors=5, *, weights='uniform', algorithm='auto', leaf_size=30, p=2, metric='minkowski', metric_params=None, n_jobs=None)\n",
      " |      Initialize self.  See help(type(self)) for accurate signature.\n",
      " |  \n",
      " |  fit(self, X, y)\n",
      " |      Fit the k-nearest neighbors classifier from the training dataset.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      X : {array-like, sparse matrix} of shape (n_samples, n_features) or                 (n_samples, n_samples) if metric='precomputed'\n",
      " |          Training data.\n",
      " |      \n",
      " |      y : {array-like, sparse matrix} of shape (n_samples,) or                 (n_samples, n_outputs)\n",
      " |          Target values.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      self : KNeighborsClassifier\n",
      " |          The fitted k-nearest neighbors classifier.\n",
      " |  \n",
      " |  predict(self, X)\n",
      " |      Predict the class labels for the provided data.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      X : array-like of shape (n_queries, n_features),                 or (n_queries, n_indexed) if metric == 'precomputed'\n",
      " |          Test samples.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      y : ndarray of shape (n_queries,) or (n_queries, n_outputs)\n",
      " |          Class labels for each data sample.\n",
      " |  \n",
      " |  predict_proba(self, X)\n",
      " |      Return probability estimates for the test data X.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      X : array-like of shape (n_queries, n_features),                 or (n_queries, n_indexed) if metric == 'precomputed'\n",
      " |          Test samples.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      p : ndarray of shape (n_queries, n_classes), or a list of n_outputs                 of such arrays if n_outputs > 1.\n",
      " |          The class probabilities of the input samples. Classes are ordered\n",
      " |          by lexicographic order.\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Data and other attributes defined here:\n",
      " |  \n",
      " |  __abstractmethods__ = frozenset()\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Methods inherited from sklearn.neighbors._base.KNeighborsMixin:\n",
      " |  \n",
      " |  kneighbors(self, X=None, n_neighbors=None, return_distance=True)\n",
      " |      Find the K-neighbors of a point.\n",
      " |      \n",
      " |      Returns indices of and distances to the neighbors of each point.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      X : array-like, shape (n_queries, n_features),             or (n_queries, n_indexed) if metric == 'precomputed',                 default=None\n",
      " |          The query point or points.\n",
      " |          If not provided, neighbors of each indexed point are returned.\n",
      " |          In this case, the query point is not considered its own neighbor.\n",
      " |      \n",
      " |      n_neighbors : int, default=None\n",
      " |          Number of neighbors required for each sample. The default is the\n",
      " |          value passed to the constructor.\n",
      " |      \n",
      " |      return_distance : bool, default=True\n",
      " |          Whether or not to return the distances.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      neigh_dist : ndarray of shape (n_queries, n_neighbors)\n",
      " |          Array representing the lengths to points, only present if\n",
      " |          return_distance=True.\n",
      " |      \n",
      " |      neigh_ind : ndarray of shape (n_queries, n_neighbors)\n",
      " |          Indices of the nearest points in the population matrix.\n",
      " |      \n",
      " |      Examples\n",
      " |      --------\n",
      " |      In the following example, we construct a NearestNeighbors\n",
      " |      class from an array representing our data set and ask who's\n",
      " |      the closest point to [1,1,1]\n",
      " |      \n",
      " |      >>> samples = [[0., 0., 0.], [0., .5, 0.], [1., 1., .5]]\n",
      " |      >>> from sklearn.neighbors import NearestNeighbors\n",
      " |      >>> neigh = NearestNeighbors(n_neighbors=1)\n",
      " |      >>> neigh.fit(samples)\n",
      " |      NearestNeighbors(n_neighbors=1)\n",
      " |      >>> print(neigh.kneighbors([[1., 1., 1.]]))\n",
      " |      (array([[0.5]]), array([[2]]))\n",
      " |      \n",
      " |      As you can see, it returns [[0.5]], and [[2]], which means that the\n",
      " |      element is at distance 0.5 and is the third element of samples\n",
      " |      (indexes start at 0). You can also query for multiple points:\n",
      " |      \n",
      " |      >>> X = [[0., 1., 0.], [1., 0., 1.]]\n",
      " |      >>> neigh.kneighbors(X, return_distance=False)\n",
      " |      array([[1],\n",
      " |             [2]]...)\n",
      " |  \n",
      " |  kneighbors_graph(self, X=None, n_neighbors=None, mode='connectivity')\n",
      " |      Compute the (weighted) graph of k-Neighbors for points in X.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      X : array-like of shape (n_queries, n_features),                 or (n_queries, n_indexed) if metric == 'precomputed',                 default=None\n",
      " |          The query point or points.\n",
      " |          If not provided, neighbors of each indexed point are returned.\n",
      " |          In this case, the query point is not considered its own neighbor.\n",
      " |          For ``metric='precomputed'`` the shape should be\n",
      " |          (n_queries, n_indexed). Otherwise the shape should be\n",
      " |          (n_queries, n_features).\n",
      " |      \n",
      " |      n_neighbors : int, default=None\n",
      " |          Number of neighbors for each sample. The default is the value\n",
      " |          passed to the constructor.\n",
      " |      \n",
      " |      mode : {'connectivity', 'distance'}, default='connectivity'\n",
      " |          Type of returned matrix: 'connectivity' will return the\n",
      " |          connectivity matrix with ones and zeros, in 'distance' the\n",
      " |          edges are distances between points, type of distance\n",
      " |          depends on the selected metric parameter in\n",
      " |          NearestNeighbors class.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      A : sparse-matrix of shape (n_queries, n_samples_fit)\n",
      " |          `n_samples_fit` is the number of samples in the fitted data.\n",
      " |          `A[i, j]` gives the weight of the edge connecting `i` to `j`.\n",
      " |          The matrix is of CSR format.\n",
      " |      \n",
      " |      See Also\n",
      " |      --------\n",
      " |      NearestNeighbors.radius_neighbors_graph : Compute the (weighted) graph\n",
      " |          of Neighbors for points in X.\n",
      " |      \n",
      " |      Examples\n",
      " |      --------\n",
      " |      >>> X = [[0], [3], [1]]\n",
      " |      >>> from sklearn.neighbors import NearestNeighbors\n",
      " |      >>> neigh = NearestNeighbors(n_neighbors=2)\n",
      " |      >>> neigh.fit(X)\n",
      " |      NearestNeighbors(n_neighbors=2)\n",
      " |      >>> A = neigh.kneighbors_graph(X)\n",
      " |      >>> A.toarray()\n",
      " |      array([[1., 0., 1.],\n",
      " |             [0., 1., 1.],\n",
      " |             [1., 0., 1.]])\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Data descriptors inherited from sklearn.neighbors._base.KNeighborsMixin:\n",
      " |  \n",
      " |  __dict__\n",
      " |      dictionary for instance variables (if defined)\n",
      " |  \n",
      " |  __weakref__\n",
      " |      list of weak references to the object (if defined)\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Methods inherited from sklearn.base.ClassifierMixin:\n",
      " |  \n",
      " |  score(self, X, y, sample_weight=None)\n",
      " |      Return the mean accuracy on the given test data and labels.\n",
      " |      \n",
      " |      In multi-label classification, this is the subset accuracy\n",
      " |      which is a harsh metric since you require for each sample that\n",
      " |      each label set be correctly predicted.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      X : array-like of shape (n_samples, n_features)\n",
      " |          Test samples.\n",
      " |      \n",
      " |      y : array-like of shape (n_samples,) or (n_samples, n_outputs)\n",
      " |          True labels for `X`.\n",
      " |      \n",
      " |      sample_weight : array-like of shape (n_samples,), default=None\n",
      " |          Sample weights.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      score : float\n",
      " |          Mean accuracy of ``self.predict(X)`` wrt. `y`.\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Methods inherited from sklearn.base.BaseEstimator:\n",
      " |  \n",
      " |  __getstate__(self)\n",
      " |  \n",
      " |  __repr__(self, N_CHAR_MAX=700)\n",
      " |      Return repr(self).\n",
      " |  \n",
      " |  __setstate__(self, state)\n",
      " |  \n",
      " |  get_params(self, deep=True)\n",
      " |      Get parameters for this estimator.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      deep : bool, default=True\n",
      " |          If True, will return the parameters for this estimator and\n",
      " |          contained subobjects that are estimators.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      params : dict\n",
      " |          Parameter names mapped to their values.\n",
      " |  \n",
      " |  set_params(self, **params)\n",
      " |      Set the parameters of this estimator.\n",
      " |      \n",
      " |      The method works on simple estimators as well as on nested objects\n",
      " |      (such as :class:`~sklearn.pipeline.Pipeline`). The latter have\n",
      " |      parameters of the form ``<component>__<parameter>`` so that it's\n",
      " |      possible to update each component of a nested object.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      **params : dict\n",
      " |          Estimator parameters.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      self : estimator instance\n",
      " |          Estimator instance.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "help(KNeighborsClassifier)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To get confusion matrix(i.e How many times model correctly predicted and how many times it was wrong)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Confusion Matrix for Wine Dataset is:\n",
      " [[12  0  1]\n",
      " [ 0  9  5]\n",
      " [ 2  1  6]]\n"
     ]
    }
   ],
   "source": [
    "print(\"Confusion Matrix for Wine Dataset is:\\n\",confusion_matrix(y_test,predict))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}