SpamDetect_2nd.ipynb

{
 "metadata": {
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.12-final"
  },
  "orig_nbformat": 2,
  "kernelspec": {
   "name": "python3",
   "display_name": "Python 3.6.12 64-bit ('first_venv': conda)",
   "metadata": {
    "interpreter": {
     "hash": "682e14e65e464696f547348b9f455c37a21548f2b45fb5da38ab3f5bf769c5e1"
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2,
 "cells": [
  {
   "source": [
    "# Spam detection with SVMs\n",
    "\n",
    "SVMs are a type of **supervised** algorithms wich has the task to find the best hyperplane that best separet classes of data that can be represente in a  **multidimesional space**. In some cases we can have data that needs different hyperplanes to correctly separate the data from each other, in our case we have to choose an hyperplane that **optimizes the prefixed margin**, that rapresent the distance between the hyperplane and the data.\n",
    "\n",
    "The SVM can be considered as a better implementation of Perceptron. The key difference is that on the Perceptron we have tried to **minimize** classifications errors, in the SVM instead is to maximiza the margin, wich is the distance between the hyperplane and the *closest* training data to the hyperplane(know as **support vector**).\n",
    "\n",
    "## The optimization strategy of SVMs\n",
    "By choosing the **wider margins** correspond to fewer classifications errors, while with **narrower margins** we risk incurring the phenomenon known as **overfitting**(we will discuss it later)\n",
    "\n",
    "So now let's translate the SVM into mathematical terms, similar to what we done in the case of Perceptron, we must define conditions that must be met to assure that SVM identify correctly the best hyperplane that separate the classes of data.\n",
    "\n",
    "$$ y = \\sum w_{i}x_{i} + \\beta \\geq \\mu $$\n",
    "\n",
    "The $\\beta$ costant rapresent the *bias* and the \\mu rapresent the *margin*\n",
    "In practice we add to $\\sum w_{i}x_{i}$ the $\\beta$ bias, this allow us to obtain a value greater than or equal to zero, in the presence of values that fall in the same **class label**(we rember that $y$ can assume value between -1 to +1, to distinguish between the corresponding classes).\n",
    "\n",
    "After that the value calculated is compared with $\\mu$ margin to ensure that the distance between each sample and the separating hyperplane we identified  is greater or equal to our margin.  "
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "source": [
    "## SVM example\n",
    "As for the Perceptron we will also choose a linear classifier for the SVM, to compare it with our previouse one.\n",
    "Now we are tankin basically the same dataset but this time we have stored for each message the harmeless word and the sospicious keywords and after that we compare those numbers.\n"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "df = pd.read_csv(\"./resources/smsSpamSvm.csv\")\n",
    "y = df.iloc[:, 0].values\n",
    "y = np.where(y == 'spam', -1, 1)\n",
    "X = df.iloc[:, [1, 2]].values\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state = 0)\n"
   ]
  },
  {
   "source": [
    "Now let's initializa our SVM(**support vector classifier**), choosing the linear classifier (*kernel = 'linear'*), then we can procede to the model training (*fit()*) and finaly we can estimate the test_data by invoking (*predict()*):"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.svm import SVC\n",
    "\n",
    "svm = SVC(kernel = 'linear', C = 1.0, random_state = 0)\n",
    "svm.fit(X_train, y_train)\n",
    "y_pred = svm.predict(X_test)\n"
   ]
  },
  {
   "source": [
    "Now let's evaluate the accurancy of our predictions *sklearn.metrics*"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Misclassified sample: 7\nAccuracy: 0.84\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "print('Misclassified sample: %d' % (y_test != y_pred).sum())\n",
    "print('Accuracy: %.2f' % accuracy_score(y_test, y_pred))"
   ]
  },
  {
   "source": [
    "## Image spam detection with SVMs\n",
    "\n",
    "The versatility of the SVM algorithm allows us to deal with even more complex real-world classification cases, such as in the case of spam messages rapresented by images, instead of simple text.\n",
    "\n",
    "We know that spammers knows our detection system and therefore they try to adopt all possible solutions to deceive our system. One of the evasion strategies is to use images as a vehicle for spreading spam, instead of simple text.\n",
    "\n",
    "For some time, however, viable image-based spam detection have been available.\n",
    "We can follow 2 possible strategies.\n",
    "\n",
    "- **Content-based filtering: ** The approach consist of trying to identify the suspect keywords that are most commonly used in textual spam emails even within images, to this end, we can implement pattern recognition techniques leveraging optical chatacter recognition (**OCR**) technology are implemented in order to extract text from images(this solution is adopted by SpamAssassin).\n",
    "- **Non content-based filtering** In this case, we try to identify specific feaures of spam images(such as color features and so on), on the grounds that spam images, being computer-generated, show different chatacteristics compared to natural images;for the extraction og the feaures, we make use of advanced recognition techniques based on NNs and **deep learning(DL)** \n",
    "\n",
    "## Brief summary on how to start with SVM ? \n",
    "\n",
    "The approach consist of the following steps\n",
    "\n",
    "- Train the classifier using the linear SVM and the feature set. \n",
    "- Compute the SVM weights for all the features.\n",
    "- Select the first one with the largest weights.\n",
    "- Create a model based on the subset.\n",
    "\n",
    "[Continue with Phishing Detection -->](https://github.com/lorenzodemiri/AI-CyberSecurity/blob/master/SpamDetect_2nd.ipynb)"
   ],
   "cell_type": "markdown",
   "metadata": {}
  }
 ]
}