NLP.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.3"
    },
    "colab": {
      "name": "NLP.ipynb",
      "provenance": [],
      "collapsed_sections": [
        "iLdemP5K9D_J",
        "kTbdP4nwTrBl",
        "zfoSxRxFCXGq",
        "pkRO86XiCzkP",
        "Gyd1nxLUPhPl"
      ],
      "toc_visible": true,
      "include_colab_link": true
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/george-zakharov/ml_notebooks/blob/master/NLP.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4WOT8Qivl2bZ",
        "colab_type": "text"
      },
      "source": [
        "### Homework 8\n",
        "\n",
        "\n",
        "\n",
        "#### Light: \n",
        "\n",
        "Датасет: https://www.kaggle.com/c/stumbleupon\n",
        "\n",
        "Решить NLP задачу и использованием стандартных представлений документов (BOW, Tf-Idf).\n",
        "\n",
        "\n",
        "#### Pro: \n",
        "\n",
        "Датасет: https://www.kaggle.com/c/stumbleupon\n",
        "\n",
        "Решить задачу классификации с использованием текста страницы и других параметров. Протестировать разные способы представления данных. Использовать различные подходы к обработке векторных представлений текстов (PCA, feature Selection). \n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2PfjccI59A1S",
        "colab_type": "text"
      },
      "source": [
        "# Load libs"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LMthwhwTl2be",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 68
        },
        "outputId": "ff9cd4c1-3875-46e0-a233-df7a3e70c065"
      },
      "source": [
        "import pandas as pd\n",
        "\n",
        "import re\n",
        "\n",
        "from sklearn.metrics import accuracy_score, recall_score, precision_score, f1_score, roc_auc_score, roc_curve, auc\n",
        "\n",
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "from sklearn.naive_bayes import MultinomialNB\n",
        "\n",
        "from nltk.stem import WordNetLemmatizer\n",
        "from sklearn.feature_extraction.text import CountVectorizer\n",
        "from sklearn.feature_extraction.text import TfidfVectorizer\n",
        "import nltk\n",
        "nltk.download('wordnet')"
      ],
      "execution_count": 61,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[nltk_data] Downloading package wordnet to /root/nltk_data...\n",
            "[nltk_data]   Package wordnet is already up-to-date!\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 61
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iLdemP5K9D_J",
        "colab_type": "text"
      },
      "source": [
        "# Load Dataset"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6MmAZtWd9Uvg",
        "colab_type": "code",
        "outputId": "e6a20883-7941-4469-ecec-9c6d9c42423e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 122
        }
      },
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive', force_remount = True)"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Go to this URL in a browser: https://accounts.google.com/o/oauth2/auth?client_id=947318989803-6bn6qk8qdgf4n4g3pfee6491hc0brc4i.apps.googleusercontent.com&redirect_uri=urn%3aietf%3awg%3aoauth%3a2.0%3aoob&response_type=code&scope=email%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdocs.test%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive.photos.readonly%20https%3a%2f%2fwww.googleapis.com%2fauth%2fpeopleapi.readonly\n",
            "\n",
            "Enter your authorization code:\n",
            "··········\n",
            "Mounted at /content/drive\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cTiV8eu3l2bk",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Get train dataset\n",
        "df = pd.read_csv(\"/content/drive/My Drive/Colab Notebooks/ML/8/stumbleupon/train.tsv\", sep ='\\t')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kTbdP4nwTrBl",
        "colab_type": "text"
      },
      "source": [
        "# Dataset exploration"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DlzF2HuCT475",
        "colab_type": "code",
        "outputId": "93c463b3-30a0-4e2d-eaba-9e1268092531",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 496
        }
      },
      "source": [
        "df.head()"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>url</th>\n",
              "      <th>urlid</th>\n",
              "      <th>boilerplate</th>\n",
              "      <th>alchemy_category</th>\n",
              "      <th>alchemy_category_score</th>\n",
              "      <th>avglinksize</th>\n",
              "      <th>commonlinkratio_1</th>\n",
              "      <th>commonlinkratio_2</th>\n",
              "      <th>commonlinkratio_3</th>\n",
              "      <th>commonlinkratio_4</th>\n",
              "      <th>compression_ratio</th>\n",
              "      <th>embed_ratio</th>\n",
              "      <th>framebased</th>\n",
              "      <th>frameTagRatio</th>\n",
              "      <th>hasDomainLink</th>\n",
              "      <th>html_ratio</th>\n",
              "      <th>image_ratio</th>\n",
              "      <th>is_news</th>\n",
              "      <th>lengthyLinkDomain</th>\n",
              "      <th>linkwordscore</th>\n",
              "      <th>news_front_page</th>\n",
              "      <th>non_markup_alphanum_characters</th>\n",
              "      <th>numberOfLinks</th>\n",
              "      <th>numwords_in_url</th>\n",
              "      <th>parametrizedLinkRatio</th>\n",
              "      <th>spelling_errors_ratio</th>\n",
              "      <th>label</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>http://www.bloomberg.com/news/2010-12-23/ibm-p...</td>\n",
              "      <td>4042</td>\n",
              "      <td>{\"title\":\"IBM Sees Holographic Calls Air Breat...</td>\n",
              "      <td>business</td>\n",
              "      <td>0.789131</td>\n",
              "      <td>2.055556</td>\n",
              "      <td>0.676471</td>\n",
              "      <td>0.205882</td>\n",
              "      <td>0.047059</td>\n",
              "      <td>0.023529</td>\n",
              "      <td>0.443783</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0</td>\n",
              "      <td>0.090774</td>\n",
              "      <td>0</td>\n",
              "      <td>0.245831</td>\n",
              "      <td>0.003883</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>24</td>\n",
              "      <td>0</td>\n",
              "      <td>5424</td>\n",
              "      <td>170</td>\n",
              "      <td>8</td>\n",
              "      <td>0.152941</td>\n",
              "      <td>0.079130</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>http://www.popsci.com/technology/article/2012-...</td>\n",
              "      <td>8471</td>\n",
              "      <td>{\"title\":\"The Fully Electronic Futuristic Star...</td>\n",
              "      <td>recreation</td>\n",
              "      <td>0.574147</td>\n",
              "      <td>3.677966</td>\n",
              "      <td>0.508021</td>\n",
              "      <td>0.288770</td>\n",
              "      <td>0.213904</td>\n",
              "      <td>0.144385</td>\n",
              "      <td>0.468649</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0</td>\n",
              "      <td>0.098707</td>\n",
              "      <td>0</td>\n",
              "      <td>0.203490</td>\n",
              "      <td>0.088652</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>40</td>\n",
              "      <td>0</td>\n",
              "      <td>4973</td>\n",
              "      <td>187</td>\n",
              "      <td>9</td>\n",
              "      <td>0.181818</td>\n",
              "      <td>0.125448</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>http://www.menshealth.com/health/flu-fighting-...</td>\n",
              "      <td>1164</td>\n",
              "      <td>{\"title\":\"Fruits that Fight the Flu fruits tha...</td>\n",
              "      <td>health</td>\n",
              "      <td>0.996526</td>\n",
              "      <td>2.382883</td>\n",
              "      <td>0.562016</td>\n",
              "      <td>0.321705</td>\n",
              "      <td>0.120155</td>\n",
              "      <td>0.042636</td>\n",
              "      <td>0.525448</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0</td>\n",
              "      <td>0.072448</td>\n",
              "      <td>0</td>\n",
              "      <td>0.226402</td>\n",
              "      <td>0.120536</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>55</td>\n",
              "      <td>0</td>\n",
              "      <td>2240</td>\n",
              "      <td>258</td>\n",
              "      <td>11</td>\n",
              "      <td>0.166667</td>\n",
              "      <td>0.057613</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>http://www.dumblittleman.com/2007/12/10-foolpr...</td>\n",
              "      <td>6684</td>\n",
              "      <td>{\"title\":\"10 Foolproof Tips for Better Sleep \"...</td>\n",
              "      <td>health</td>\n",
              "      <td>0.801248</td>\n",
              "      <td>1.543103</td>\n",
              "      <td>0.400000</td>\n",
              "      <td>0.100000</td>\n",
              "      <td>0.016667</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.480725</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0</td>\n",
              "      <td>0.095861</td>\n",
              "      <td>0</td>\n",
              "      <td>0.265656</td>\n",
              "      <td>0.035343</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>24</td>\n",
              "      <td>0</td>\n",
              "      <td>2737</td>\n",
              "      <td>120</td>\n",
              "      <td>5</td>\n",
              "      <td>0.041667</td>\n",
              "      <td>0.100858</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>http://bleacherreport.com/articles/1205138-the...</td>\n",
              "      <td>9006</td>\n",
              "      <td>{\"title\":\"The 50 Coolest Jerseys You Didn t Kn...</td>\n",
              "      <td>sports</td>\n",
              "      <td>0.719157</td>\n",
              "      <td>2.676471</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>0.222222</td>\n",
              "      <td>0.123457</td>\n",
              "      <td>0.043210</td>\n",
              "      <td>0.446143</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0</td>\n",
              "      <td>0.024908</td>\n",
              "      <td>0</td>\n",
              "      <td>0.228887</td>\n",
              "      <td>0.050473</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>14</td>\n",
              "      <td>0</td>\n",
              "      <td>12032</td>\n",
              "      <td>162</td>\n",
              "      <td>10</td>\n",
              "      <td>0.098765</td>\n",
              "      <td>0.082569</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                                                 url  ...  label\n",
              "0  http://www.bloomberg.com/news/2010-12-23/ibm-p...  ...      0\n",
              "1  http://www.popsci.com/technology/article/2012-...  ...      1\n",
              "2  http://www.menshealth.com/health/flu-fighting-...  ...      1\n",
              "3  http://www.dumblittleman.com/2007/12/10-foolpr...  ...      1\n",
              "4  http://bleacherreport.com/articles/1205138-the...  ...      0\n",
              "\n",
              "[5 rows x 27 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BFm9OsdrTtrK",
        "colab_type": "code",
        "outputId": "3abaa367-089e-4236-9815-5a5c8fccd94a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 561
        }
      },
      "source": [
        "df.info()"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<class 'pandas.core.frame.DataFrame'>\n",
            "RangeIndex: 7395 entries, 0 to 7394\n",
            "Data columns (total 27 columns):\n",
            "url                               7395 non-null object\n",
            "urlid                             7395 non-null int64\n",
            "boilerplate                       7395 non-null object\n",
            "alchemy_category                  7395 non-null object\n",
            "alchemy_category_score            7395 non-null object\n",
            "avglinksize                       7395 non-null float64\n",
            "commonlinkratio_1                 7395 non-null float64\n",
            "commonlinkratio_2                 7395 non-null float64\n",
            "commonlinkratio_3                 7395 non-null float64\n",
            "commonlinkratio_4                 7395 non-null float64\n",
            "compression_ratio                 7395 non-null float64\n",
            "embed_ratio                       7395 non-null float64\n",
            "framebased                        7395 non-null int64\n",
            "frameTagRatio                     7395 non-null float64\n",
            "hasDomainLink                     7395 non-null int64\n",
            "html_ratio                        7395 non-null float64\n",
            "image_ratio                       7395 non-null float64\n",
            "is_news                           7395 non-null object\n",
            "lengthyLinkDomain                 7395 non-null int64\n",
            "linkwordscore                     7395 non-null int64\n",
            "news_front_page                   7395 non-null object\n",
            "non_markup_alphanum_characters    7395 non-null int64\n",
            "numberOfLinks                     7395 non-null int64\n",
            "numwords_in_url                   7395 non-null int64\n",
            "parametrizedLinkRatio             7395 non-null float64\n",
            "spelling_errors_ratio             7395 non-null float64\n",
            "label                             7395 non-null int64\n",
            "dtypes: float64(12), int64(9), object(6)\n",
            "memory usage: 1.5+ MB\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yf1bGE_VTzTi",
        "colab_type": "code",
        "outputId": "3feff9c9-92c9-4bf2-c05d-770ad64231b9",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 317
        }
      },
      "source": [
        "df.describe()"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>urlid</th>\n",
              "      <th>avglinksize</th>\n",
              "      <th>commonlinkratio_1</th>\n",
              "      <th>commonlinkratio_2</th>\n",
              "      <th>commonlinkratio_3</th>\n",
              "      <th>commonlinkratio_4</th>\n",
              "      <th>compression_ratio</th>\n",
              "      <th>embed_ratio</th>\n",
              "      <th>framebased</th>\n",
              "      <th>frameTagRatio</th>\n",
              "      <th>hasDomainLink</th>\n",
              "      <th>html_ratio</th>\n",
              "      <th>image_ratio</th>\n",
              "      <th>lengthyLinkDomain</th>\n",
              "      <th>linkwordscore</th>\n",
              "      <th>non_markup_alphanum_characters</th>\n",
              "      <th>numberOfLinks</th>\n",
              "      <th>numwords_in_url</th>\n",
              "      <th>parametrizedLinkRatio</th>\n",
              "      <th>spelling_errors_ratio</th>\n",
              "      <th>label</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>7395.000000</td>\n",
              "      <td>7395.000000</td>\n",
              "      <td>7395.000000</td>\n",
              "      <td>7395.000000</td>\n",
              "      <td>7395.000000</td>\n",
              "      <td>7395.000000</td>\n",
              "      <td>7395.000000</td>\n",
              "      <td>7395.000000</td>\n",
              "      <td>7395.0</td>\n",
              "      <td>7395.000000</td>\n",
              "      <td>7395.000000</td>\n",
              "      <td>7395.000000</td>\n",
              "      <td>7395.000000</td>\n",
              "      <td>7395.000000</td>\n",
              "      <td>7395.000000</td>\n",
              "      <td>7395.000000</td>\n",
              "      <td>7395.000000</td>\n",
              "      <td>7395.000000</td>\n",
              "      <td>7395.000000</td>\n",
              "      <td>7395.000000</td>\n",
              "      <td>7395.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>mean</th>\n",
              "      <td>5305.704665</td>\n",
              "      <td>2.761823</td>\n",
              "      <td>0.468230</td>\n",
              "      <td>0.214080</td>\n",
              "      <td>0.092062</td>\n",
              "      <td>0.049262</td>\n",
              "      <td>2.255103</td>\n",
              "      <td>-0.103750</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.056423</td>\n",
              "      <td>0.021231</td>\n",
              "      <td>0.233778</td>\n",
              "      <td>0.275709</td>\n",
              "      <td>0.660311</td>\n",
              "      <td>30.077079</td>\n",
              "      <td>5716.598242</td>\n",
              "      <td>178.754564</td>\n",
              "      <td>4.960649</td>\n",
              "      <td>0.172864</td>\n",
              "      <td>0.101221</td>\n",
              "      <td>0.513320</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>std</th>\n",
              "      <td>3048.384114</td>\n",
              "      <td>8.619793</td>\n",
              "      <td>0.203133</td>\n",
              "      <td>0.146743</td>\n",
              "      <td>0.095978</td>\n",
              "      <td>0.072629</td>\n",
              "      <td>5.704313</td>\n",
              "      <td>0.306545</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.041446</td>\n",
              "      <td>0.144162</td>\n",
              "      <td>0.052487</td>\n",
              "      <td>1.919320</td>\n",
              "      <td>0.473636</td>\n",
              "      <td>20.393101</td>\n",
              "      <td>8875.432430</td>\n",
              "      <td>179.466198</td>\n",
              "      <td>3.233111</td>\n",
              "      <td>0.183286</td>\n",
              "      <td>0.079231</td>\n",
              "      <td>0.499856</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>min</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.045564</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25%</th>\n",
              "      <td>2688.500000</td>\n",
              "      <td>1.602062</td>\n",
              "      <td>0.340370</td>\n",
              "      <td>0.105263</td>\n",
              "      <td>0.022222</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.442616</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.028502</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.201061</td>\n",
              "      <td>0.025900</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>14.000000</td>\n",
              "      <td>1579.000000</td>\n",
              "      <td>82.000000</td>\n",
              "      <td>3.000000</td>\n",
              "      <td>0.040984</td>\n",
              "      <td>0.068739</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50%</th>\n",
              "      <td>5304.000000</td>\n",
              "      <td>2.088235</td>\n",
              "      <td>0.481481</td>\n",
              "      <td>0.202454</td>\n",
              "      <td>0.068627</td>\n",
              "      <td>0.022222</td>\n",
              "      <td>0.483680</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.045775</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.230564</td>\n",
              "      <td>0.083051</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>25.000000</td>\n",
              "      <td>3500.000000</td>\n",
              "      <td>139.000000</td>\n",
              "      <td>5.000000</td>\n",
              "      <td>0.113402</td>\n",
              "      <td>0.089312</td>\n",
              "      <td>1.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>75%</th>\n",
              "      <td>7946.500000</td>\n",
              "      <td>2.627451</td>\n",
              "      <td>0.616604</td>\n",
              "      <td>0.300000</td>\n",
              "      <td>0.133333</td>\n",
              "      <td>0.065065</td>\n",
              "      <td>0.578227</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.073459</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.260770</td>\n",
              "      <td>0.236700</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>43.000000</td>\n",
              "      <td>6377.000000</td>\n",
              "      <td>222.000000</td>\n",
              "      <td>7.000000</td>\n",
              "      <td>0.241299</td>\n",
              "      <td>0.112376</td>\n",
              "      <td>1.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>max</th>\n",
              "      <td>10566.000000</td>\n",
              "      <td>363.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.980392</td>\n",
              "      <td>0.980392</td>\n",
              "      <td>21.000000</td>\n",
              "      <td>0.250000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.444444</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.716883</td>\n",
              "      <td>113.333333</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>100.000000</td>\n",
              "      <td>207952.000000</td>\n",
              "      <td>4997.000000</td>\n",
              "      <td>22.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "              urlid  avglinksize  ...  spelling_errors_ratio        label\n",
              "count   7395.000000  7395.000000  ...            7395.000000  7395.000000\n",
              "mean    5305.704665     2.761823  ...               0.101221     0.513320\n",
              "std     3048.384114     8.619793  ...               0.079231     0.499856\n",
              "min        1.000000     0.000000  ...               0.000000     0.000000\n",
              "25%     2688.500000     1.602062  ...               0.068739     0.000000\n",
              "50%     5304.000000     2.088235  ...               0.089312     1.000000\n",
              "75%     7946.500000     2.627451  ...               0.112376     1.000000\n",
              "max    10566.000000   363.000000  ...               1.000000     1.000000\n",
              "\n",
              "[8 rows x 21 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZfelfOwp_Plb",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 54
        },
        "outputId": "d7dd7752-3a92-4c9c-c183-ab4bf9a91f02"
      },
      "source": [
        "df.boilerplate.loc[0]"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'{\"title\":\"IBM Sees Holographic Calls Air Breathing Batteries ibm sees holographic calls, air-breathing batteries\",\"body\":\"A sign stands outside the International Business Machines Corp IBM Almaden Research Center campus in San Jose California Photographer Tony Avelar Bloomberg Buildings stand at the International Business Machines Corp IBM Almaden Research Center campus in the Santa Teresa Hills of San Jose California Photographer Tony Avelar Bloomberg By 2015 your mobile phone will project a 3 D image of anyone who calls and your laptop will be powered by kinetic energy At least that s what International Business Machines Corp sees in its crystal ball The predictions are part of an annual tradition for the Armonk New York based company which surveys its 3 000 researchers to find five ideas expected to take root in the next five years IBM the world s largest provider of computer services looks to Silicon Valley for input gleaning many ideas from its Almaden research center in San Jose California Holographic conversations projected from mobile phones lead this year s list The predictions also include air breathing batteries computer programs that can tell when and where traffic jams will take place environmental information generated by sensors in cars and phones and cities powered by the heat thrown off by computer servers These are all stretch goals and that s good said Paul Saffo managing director of foresight at the investment advisory firm Discern in San Francisco In an era when pessimism is the new black a little dose of technological optimism is not a bad thing For IBM it s not just idle speculation The company is one of the few big corporations investing in long range research projects and it counts on innovation to fuel growth Saffo said Not all of its predictions pan out though IBM was overly optimistic about the spread of speech technology for instance When the ideas do lead to products they can have broad implications for society as well as IBM s bottom line he said Research Spending They have continued to do research when all the other grand research organizations are gone said Saffo who is also a consulting associate professor at Stanford University IBM invested 5 8 billion in research and development last year 6 1 percent of revenue While that s down from about 10 percent in the early 1990s the company spends a bigger share on research than its computing rivals Hewlett Packard Co the top maker of personal computers spent 2 4 percent last year At Almaden scientists work on projects that don t always fit in with IBM s computer business The lab s research includes efforts to develop an electric car battery that runs 500 miles on one charge a filtration system for desalination and a program that shows changes in geographic data IBM rose 9 cents to 146 04 at 11 02 a m in New York Stock Exchange composite trading The stock had gained 11 percent this year before today Citizen Science The list is meant to give a window into the company s innovation engine said Josephine Cheng a vice president at IBM s Almaden lab All this demonstrates a real culture of innovation at IBM and willingness to devote itself to solving some of the world s biggest problems she said Many of the predictions are based on projects that IBM has in the works One of this year s ideas that sensors in cars wallets and personal devices will give scientists better data about the environment is an expansion of the company s citizen science initiative Earlier this year IBM teamed up with the California State Water Resources Control Board and the City of San Jose Environmental Services to help gather information about waterways Researchers from Almaden created an application that lets smartphone users snap photos of streams and creeks and report back on conditions The hope is that these casual observations will help local and state officials who don t have the resources to do the work themselves Traffic Predictors IBM also sees data helping shorten commutes in the next five years Computer programs will use algorithms and real time traffic information to predict which roads will have backups and how to avoid getting stuck Batteries may last 10 times longer in 2015 than today IBM says Rather than using the current lithium ion technology new models could rely on energy dense metals that only need to interact with the air to recharge Some electronic devices might ditch batteries altogether and use something similar to kinetic wristwatches which only need to be shaken to generate a charge The final prediction involves recycling the heat generated by computers and data centers Almost half of the power used by data centers is currently spent keeping the computers cool IBM scientists say it would be better to harness that heat to warm houses and offices In IBM s first list of predictions compiled at the end of 2006 researchers said instantaneous speech translation would become the norm That hasn t happened yet While some programs can quickly translate electronic documents and instant messages and other apps can perform limited speech translation there s nothing widely available that acts like the universal translator in Star Trek Second Life The company also predicted that online immersive environments such as Second Life would become more widespread While immersive video games are as popular as ever Second Life s growth has slowed Internet users are flocking instead to the more 2 D environments of Facebook Inc and Twitter Inc Meanwhile a 2007 prediction that mobile phones will act as a wallet ticket broker concierge bank and shopping assistant is coming true thanks to the explosion of smartphone applications Consumers can pay bills through their banking apps buy movie tickets and get instant feedback on potential purchases all with a few taps on their phones The nice thing about the list is that it provokes thought Saffo said If everything came true they wouldn t be doing their job To contact the reporter on this story Ryan Flinn in San Francisco at rflinn bloomberg net To contact the editor responsible for this story Tom Giles at tgiles5 bloomberg net by 2015, your mobile phone will project a 3-d image of anyone who calls and your laptop will be powered by kinetic energy. at least that\\\\u2019s what international business machines corp. sees in its crystal ball.\",\"url\":\"bloomberg news 2010 12 23 ibm predicts holographic calls air breathing batteries by 2015 html\"}'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gmTZWPWdCI8z",
        "colab_type": "text"
      },
      "source": [
        "# Light"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zfoSxRxFCXGq",
        "colab_type": "text"
      },
      "source": [
        "## User defined functions"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DBrmfBvJCVEc",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def value_of_metrics(y_true, y_pred):\n",
        "    print('Accuracy: ', accuracy_score(y_true, y_pred))\n",
        "    print('Recall: ', recall_score(y_true, y_pred))\n",
        "    print('Precision: ', precision_score(y_true, y_pred))  \n",
        "    print('F1: ', f1_score(y_true, y_pred))\n",
        "    print('Roc_AUC: ', roc_auc_score(y_true, y_pred))\n",
        "    \n",
        "    fpr, tpr, threshold = roc_curve(y_true, y_pred)\n",
        "    roc_auc = auc(fpr, tpr)\n",
        "    plt.title('My DataSet')\n",
        "    plt.plot(fpr, tpr, 'b', label = 'AUC = %0.2f' % roc_auc)\n",
        "    plt.legend(loc = 'lower right')\n",
        "    plt.plot([0, 1], [0, 1],'r--')\n",
        "    plt.xlim([0, 1])\n",
        "    plt.ylim([0, 1])\n",
        "    plt.ylabel('True Positive Rate')\n",
        "    plt.xlabel('False Positive Rate')\n",
        "    plt.show()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8IKfMm21CfNX",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def text_preprocess(text):    \n",
        "    reg = re.compile('[^а-яёА-ЯЁa-zA-Z0-9]')\n",
        "    text = text.lower()\n",
        "    text = reg.sub(' ', text)\n",
        "    lemmatizer = WordNetLemmatizer()\n",
        "    text_ = ''\n",
        "    for word in text.split():\n",
        "        text_ = text_+ ' ' + lemmatizer.lemmatize(word)\n",
        "\n",
        "    return text_"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pkRO86XiCzkP",
        "colab_type": "text"
      },
      "source": [
        "## Extract train data"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "j8fCU6JwC3Rb",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "df_short = df[['boilerplate', 'label']]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LFxAhnTIDUoi",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "outputId": "eb7e765d-98e0-4419-ffa2-ed33504bd4a8"
      },
      "source": [
        "df_short.head()"
      ],
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>boilerplate</th>\n",
              "      <th>label</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>{\"title\":\"IBM Sees Holographic Calls Air Breat...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>{\"title\":\"The Fully Electronic Futuristic Star...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>{\"title\":\"Fruits that Fight the Flu fruits tha...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>{\"title\":\"10 Foolproof Tips for Better Sleep \"...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>{\"title\":\"The 50 Coolest Jerseys You Didn t Kn...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                                         boilerplate  label\n",
              "0  {\"title\":\"IBM Sees Holographic Calls Air Breat...      0\n",
              "1  {\"title\":\"The Fully Electronic Futuristic Star...      1\n",
              "2  {\"title\":\"Fruits that Fight the Flu fruits tha...      1\n",
              "3  {\"title\":\"10 Foolproof Tips for Better Sleep \"...      1\n",
              "4  {\"title\":\"The 50 Coolest Jerseys You Didn t Kn...      0"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 20
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vHqomodbDgDj",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 119
        },
        "outputId": "8b57bae2-76a5-4084-8586-5cd142ff5bb8"
      },
      "source": [
        "df_short['boilerplate'] = df_short['boilerplate'].apply(text_preprocess)"
      ],
      "execution_count": 27,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:1: SettingWithCopyWarning: \n",
            "A value is trying to be set on a copy of a slice from a DataFrame.\n",
            "Try using .loc[row_indexer,col_indexer] = value instead\n",
            "\n",
            "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
            "  \"\"\"Entry point for launching an IPython kernel.\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XtUwwGFCD3qF",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "outputId": "a671134a-b178-45d5-9f0d-3d2f62fbda95"
      },
      "source": [
        "df_short.head()"
      ],
      "execution_count": 28,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>boilerplate</th>\n",
              "      <th>label</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>title ibm see holographic call air breathing ...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>title the fully electronic futuristic startin...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>title fruit that fight the flu fruit that fig...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>title 10 foolproof tip for better sleep body ...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>title the 50 coolest jersey you didn t know e...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                                         boilerplate  label\n",
              "0   title ibm see holographic call air breathing ...      0\n",
              "1   title the fully electronic futuristic startin...      1\n",
              "2   title fruit that fight the flu fruit that fig...      1\n",
              "3   title 10 foolproof tip for better sleep body ...      1\n",
              "4   title the 50 coolest jersey you didn t know e...      0"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 28
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bPhncWhSECHD",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "X = df_short['boilerplate']\n",
        "y = df_short['label']"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a62oqUBPERgn",
        "colab_type": "text"
      },
      "source": [
        "## Test train data"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HSmC-y7NEUQr",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Get vectorizers\n",
        "bow_vec = CountVectorizer(stop_words = 'english', ngram_range = (1, 3), max_features = 60) # BOW\n",
        "tfidf_vec = TfidfVectorizer(stop_words = 'english', ngram_range = (1, 3), max_features = 60) # TF-IDF "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "d8qfo7d0Exnq",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Get word matrixes\n",
        "X_bow = bow_vec.fit_transform(X)\n",
        "X_tfidf = tfidf_vec.fit_transform(X)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "y7d81UFWGmVj",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Prepare dict for further training\n",
        "data_dict = {'BOW': X_bow, 'IF-IDF': X_tfidf}"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FDKei0rgFeZm",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 913
        },
        "outputId": "1c73550f-a32c-40cc-de62-50dfc70cc84a"
      },
      "source": [
        "# Train Bayes classifier with BOW and TF-IDF\n",
        "for method_name, X_vec in data_dict.items():\n",
        "  print(\"Use word vectors on - \" + method_name + \"\\n\")\n",
        "  X_train, X_test, y_train, y_test = train_test_split(X_vec, y, test_size = 0.2, random_state = 42)\n",
        "\n",
        "  clf = MultinomialNB()\n",
        "  clf.fit(X_train,y_train)\n",
        "  clf.score(X_test,y_test)\n",
        "  y_pred = clf.predict(X_test)\n",
        "\n",
        "  value_of_metrics(y_test, y_pred)\n",
        "  print(\"\\n\\n\")"
      ],
      "execution_count": 82,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Use word vectors on - BOW\n",
            "\n",
            "Accuracy:  0.8052738336713996\n",
            "Recall:  0.7209612817089452\n",
            "Precision:  0.8723747980613893\n",
            "F1:  0.7894736842105262\n",
            "Roc_AUC:  0.8063710518133768\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEWCAYAAAB42tAoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nO3dd5iU5fXw8e8BFAQBFdAovUqvK6JR\nwdjAAkZUEEWxoVijxmh+JjHxNYmxxhYVUVGjEiuiwRZFcVcRlt6kCywqrkjvLOf948w4w9bZ3Zl5\nZmbP57r2YmeemXnOPuzOmbudW1QV55xzriTVgg7AOedcavNE4ZxzrlSeKJxzzpXKE4VzzrlSeaJw\nzjlXKk8UzjnnSuWJwjnnXKk8UbiMICLfiMguEWlY6P6ZIqIi0qICrzk29JqbQ1/zROTvIlK/nHGd\nXI7HNxGRN0TkRxHZGDrniBif+6mIXBHruZyLlScKl0lWABeEb4hIF6B2JV/zXlWtCzQCLgX6ADki\nUqeSr1uSF4HVQHOgATAcWJugczkXE08ULpO8CFwcdfsS4IXwDRE5SkTWikj1qPvOEZHZZb2wqu5Q\n1WnAQOwN/NLQ81uLyCcisi7UCnhJRA4KHXsRaAa8IyJbROR3oftfE5HvQy2GySLSKepURwFjVXWr\nqu5R1Zmq+l5UvH1E5AsR2SAis0WkX+j+vwLHA4+FzvVYua6cc6XwROEyyRSgnoh0CCWDocC/wwdD\nb/TrgFOjnjOcqGRSFlXdDHyEvSkDCPB34AigA9AU+HPoscOBVcBZqnqgqt4bes57QFvgUGAG8FKh\nn+FxERkqIs2izy0ijYH/AncDhwC/Bd4QkUaqegfwOXBd6FzXxfozOVcWTxQu04RbFacAC4E1hY4/\nD1wEICKHAKcBL5fzHN9ib9So6lJV/UhVd6pqPvAg0Le0J6vqs6q6WVV3YkmlW9S4x3nYG/4fgRUi\nMktEjgoduwiYqKoTVXWvqn4E5AKnlzN+58rFE4XLNC8Cw4ARFN9S+DdwVmiM4Xzgc1X9rpznaAz8\nBCAih4nIOBFZIyKbQq/fsKQnikh1EblHRJaFHv9N6FBDAFVdr6q3q2on4DBgFjBeRAQbtzgv1O20\nQUQ2AMcBh5czfufKxROFyyiquhIb1D4deLOY42uAL4FzsG6nF8vz+iJyIHAy9qkf4G+AAl1UtR72\nqV+iT1noJYYBg0KvUR9oEX7pYmL9Ebgf69Y6BBvkflFVD4r6qqOq95RwLufiwhOFy0SXA79S1a0l\nHH8B+B3QhWKSSXFEpKaI9ALGA+uB50KH6gJbgI2hMYRbCz11LdAq6nZdYCc2VlIbSzTR5/mHiHQW\nkRoiUhcYBSxV1XVEWkOnhVomtUSkn4g0KeFczsWFJwqXcVR1marmlvKQt7BunLdUdVsZL/c7EdmM\nvbG/AEwHjo1KQn8BegIbsYHmwonn78AfQl1Fvw29xkps7GQBNngdrXYovg3A8lCcA0M/12qsNfJ/\nQD7WwriVyN/xw8C5IrJeRB4p4+dyLmbiGxe5qkhElgFXqer/go7FuVTnLQpX5YjIYKw//5OgY3Eu\nHSQsUYjIsyLyg4jMK+G4iMgjIrJUROaISM9ExeJcmIh8CjwBXKuqewMOx7m0kMgWxVigfynHB2CL\njtoCI7E/XucSSlX7qeqhqvpB0LE4ly4SlihUdTKhueYlGAS8oGYKcJCI+Hxw55xLMTUCPHdjbNZG\nWF7oviKLn0RkJNbqoE6dOr3at2+flACdcy6d7NoF27bB1q2RfxsXrOQgNjCHPT+qaqOKvG6QiSJm\nqjoaGA2QlZWlubmlzXx0zrnM9913kJu779cPP9ix6tWUzp0h6yjh4q1P0LreDzQZ/eeVFT1XkIli\nDVZALawJRevyOOdclZefXzQpfPutHatWDTp0gAEDICsLjmm2hu5PjaL6sCFw4YXYmk1g9J8rfP4g\nE8UE4DoRGQccDWysQM0d55zLKOvXw/TplgymTbN/V62KHD/ySDjxREsKWVnQvTsceCCgCmPGwPDf\nwu7dMPCMuMWUsEQhIq8A/YCGIpIH3AnsB6CqTwITsXo8S4FthOr7O+dcVbFpE8yYsW9LYdmyyPHW\nreGYY+D66y0p9OwJ9eoV80LLlsGVV8KkSZZFnn7anhwnCUsUqnpBGccVuDZR53fOuVSyZQvMmrVv\nUli0KHK8eXNLBldcEUkKhxwS44vPnWvNkNGj7QWkSI3JSkmLwWznnEsn27fD7Nn7JoWFC2FvaIln\n48aWDC66yP7t1QsalXc+0rx51hy5+GI4+2xYvhwaNIj7zwKeKJxzrlJ27rQP9NFJYd48KCiw44ce\nCkcdBeeeG0kKh1dmxdiuXfC3v9nXYYfB+edDrVoJSxLgicI552K2ezfMn79vUpgzx+4He6/OyoIz\nz4wMNjduHMeeoK++gssvtyAuuggeesiSRIJ5onDOuWIUFFh3UXRSmDXLWhAA9etbIrj55khSaN48\n7sMDEWvWwPHHWyvi3XfhjPjNaiqLJwrnXJW3dy8sXrxvUpg501Y3g00/7dULrrsukhRatbI1DAm3\neDG0a2dNk//8B046qYSpT4njicI5V6Wo2rhvdFKYPh02b7bjBxwAPXpEZh9lZdn7dPXqSQ50wwb4\n3e9sbcSnn8IJJ8Cvf53kIIwnCudcxlK1xWqFVzVv2GDHa9aEbt1g+PBIUujQAWoE/c44YQKMGgXf\nfw+33mqj4QEK+nI451xcqFpZi8JJ4ccf7XiNGtC1q00SCieFTp1g//2DjbuIK66AZ56BLl3g7bct\n0IB5onDOpaW1a4smhe+/t2PVq1sSGDgwkhS6dEnKBKGKCW9JLRIZFb/ttpTJYp4onHMp78cfI/WP\nwl95eXZMxLqLTj01khS6dYPatYONOWarV8PVV8PQodYHdvXVQUdUhCcK51xK2bChaFL45pvI8Xbt\nbFw3nBR69AgVxUs3e/fCU09Zy6GgILCB6lh4onDOBWbz5qJF8ZYujRxv1Qp694ZrronUP6pfP7h4\n42bJEhuLmDwZTj7ZajS1bBl0VCXyROGcS4qtW4svihfunm/WzJLBZZdFkkICq1IEa8ECW9L97LMw\nYkQCV+nFhycK51zc7dhRtCjeggWRonhHHGHJYNiwSP2jQw8NNuaEmz3bMuUll8CgQbaY4+CDg44q\nJp4onHOVsmuXFcELJ4Rp0+z2nj12vFEjSwbnnBNJCkccEWzMSbVzJ9x9N9xzj1UDHDLEpl+lSZIA\nTxTOuXLYs8daBtEthdmzLVmAvfdlZdkasfBgc9OmKd+zkjhffmlF/BYutHLgDz6YwnN0S+aJwjlX\nrIICG0MoXP9oxw47Xq+etQ5uvDGSFFq2rMJJobA1a6BvX/jFL2DiRNvUOk15onDOsXevzTaKTgoz\nZtgANECdOja4PGpUJCm0aZOkonjpZuFCW9jRuDG8+qoV8atbN+ioKsUThXNVjCqsWFG0KN6mTXa8\nVi1bmxCefZSVBUceGUBRvHSzfj3ccgs895xNez3+eNt5LgN4onAug6nawt/CpS7Wr7fj++9vq5gv\nvDCSFDp2TIGieOnmrbdssUd+Pvz+94EX8Ys3/3VwLoMUVxQvP9+O1ahh9Y7CW3JmZUHnzilTTih9\nXXaZtSK6d4f//tf66DKMJwrn0tQPPxQtdfHtt3asWjUrihe9JWfXrmk54SY1RRfx69MH2raF3/4W\n9tsv2LgSxBOFc2ngp5+KJoVVq+yYCLRvb2Om0UXx6tQJNuaMtXIlXHWVrRa8+GIYOTLoiBLOE4Vz\nKWbjxqL1j5Yvjxxv2xZ++cvItNQePdJ+Uk162LsXnngCbr/dWhTnnRd0REnjicK5AG3ZYmsTopPC\n4sWR4y1aWDK46qpI/aODDgos3Kpr0SIr4pedbfXMn3rK/nOqCE8UziXJtm1F6x8tXBjp7m7SxJLB\nxRdHSl00bBhszC5k0SKYPx/GjrX/oCq2qtAThXMJsHOnFQeNTgrz59tqZ4DDDrMZlOFtOXv1sgW8\nLoXMnGlF/C691LbKW768yjbnPFE4V0m7d+9bFC83F+bOtfvBWgVZWftuy3nEEVXuQ2n62LED7roL\n7r3XVldfcIFNF6uiSQI8UThXLnv2WHdR4aJ4O3fa8YMOskRwyy2RpNCsmSeFtJGTY0X8Fi2ylsQD\nD/icYjxROFeiggIbWC5cFG/7djtet651GV1/fSQptGrlSSFtrVkDJ55orYgPPrBBawd4onAOsAHl\nZcuK1j/assWO165tM47Cs4+ysmyaqhfFywALFljdksaN4Y03LFmk5SbcieOJwlU5qrZmqnBS2LDB\njteqZdUYRoyIJIX27b0oXsb56Se4+WZ4/nn47DM44QQ466ygo0pJnihcRlO1HoXC9Y/WrbPj++1n\nq5iHDt23KF6GVmJwYW+8Addea78Id9wBvXsHHVFK80ThMsr33xdNCmvX2rHq1a0I3q9/vW9RvJo1\ng43ZJdmIEdaK6NkT3n/fmo+uVJ4oXNrKzy9a/2jNGjtWrZrtHdO//771jw44INiYXUCii/gde6z9\nctxyi9dTj1FCr5KI9AceBqoDY1T1nkLHmwHPAweFHnO7qk5MZEwuPa1fXzQprFwZOX7kkdCvXyQp\ndO/u45EuZMUKK9x30UVwySVVoohfvCUsUYhIdeBx4BQgD5gmIhNUdUHUw/4AvKqqT4hIR2Ai0CJR\nMbn0sGlT0aJ4y5ZFjrdubZWdr7suUv+oXr3g4nUpqqAAHn/cNhKqVs12Z3IVksgWRW9gqaouBxCR\nccAgIDpRKBD+E68PfJvAeFwK2rrVqiRMmxZJCosWRY43b27J4IorIknhkEOCi9eliYULbeHcl1/C\ngAHw5JO28tFVSCITRWNgddTtPODoQo/5M/ChiFwP1AFOLu6FRGQkMBKgmf9np63t24vWP1qwwKo3\ng01jz8qyHoJw/aNGjYKN2aWppUvtE8eLL1pLwldBVkrQIzkXAGNV9QEROQZ4UUQ6q+re6Aep6mhg\nNEBWVpYGEKcrp127rN5RdFKYN89KYAAceqgVxRs8OJIUDj882Jhdmps+3eqpXHaZrYdYscL7JOMk\nkYliDdA06naT0H3RLgf6A6jqlyJSC2gI/JDAuFyc7d5tLYPopDBnjiULgAYNLBmccUZksLlxY/+Q\n5+Jk+3b4y1/g/vuhaVPbea5WLU8ScZTIRDENaCsiLbEEMRQYVugxq4CTgLEi0gGoBeQnMCZXSQUF\n8PXX+yaFWbOs4CZA/fqWCG66KZIUmjf3pOASZPJkG8BassTGJO6/34v4JUDCEoWq7hGR64APsKmv\nz6rqfBG5C8hV1QnALcDTInITNrA9QlW9aylF7N1rf3/RSWHGDNuAB2z6aa9etsA1uiie1z9ySbFm\njW0U3rQp/O9/9r1LCEm39+WsrCzNzc0NOoyMo2r7shSuf7R5sx0/4ADbmzkry8YWsrKgXTtPCi4A\nc+dCly72/bvvWhG/OnWCjSkNiMh0Vc2qyHODHsx2AVCFVauKJoX16+14zZq2inn48EhLoUMHX8Tq\nAvbjj9an+e9/R4r4nXlm0FFVCf6nXwUUVxTvxx/tWI0a0LUrnHdeJCl06gT77x9szM79TBVee81W\nWK5fD3feCUcXnmnvEskTRYZZu7ZoqYvvvrNj1atbEojekrNLFx/7cynukktsPURWFnz8caTbySWN\nJ4o0tm5d0aSwOrTEUcS6i045Zd+ieLVrBxuzczGJLuLXt681e3/zG+//DIhf9TSxYUPR+kcrVkSO\nt2sHxx8fSQo9enhRPJemli+HK6+0JfqXXmrTXl2gPFGkoM2bbW/m6KSwZEnkeKtWNvNo1KhI/aP6\n9YOL17m4KCiARx+1jYSqV4eLLw46IhfiiSJg27bZgrXopPD115GWd7NmlgwuvTSSFBo0CDZm5+Ju\nwQIrvfHVV7aE/8knoUmToKNyIZ4okmjHjqJF8ebPjxTFO+IISwYXXBCpf3ToocHG7FxSrFhhteRf\nftn2pfWl/CnFE0WC7NplRfCik8LcuZGieI0aWTIIb8vZq5clCueqjGnTrDl95ZXWili+HOrWDToq\nVwxPFHGwZ0/RonizZ0eK4h18sCWDW2+NDDY3beofmlwVtW0b/OlP8NBDVghs+HCbo+1JImV5oqiE\nmTPh+uttNtL27XZfvXrWOrjxxkhSaNnSk4JzAHz6qRXxW7YMrroK/vEPX8iTBjxRVMKzz9o6hvDs\no6wsaNPG6x85V6y8PFvY07w5fPKJ1WhyacETRSVkZ8Oxx8KDDwYdiXMpbPZsW+3ZpAm8/Tb06+cr\nP9OMf/atoM2bbQbTL38ZdCTOpaj8fNtEqHt3K+IHcPrpniTSkLcoKmjKFJvW6onCuUJUYdw4uOEG\n2LjRdp875pigo3KVEFOiEJH9gWaqujTB8aSNnBwbi/Dff+cKGT4cXnrJKrw+84xVonRprcyuJxE5\nA5gLfBS63V1E3kp0YKkuJ8eKWPq2vM5hzetwOYETT7SBu5wcTxIZIpYxiruAo4ENAKo6C2iTyKBS\n3Z491vXk3U7OAUuX2jakzz1nty+/3DYYql492Lhc3MSSKHar6oZC96XX/qlxNmcObNniicJVcXv2\nwP33W9N65kzf7SqDxTJGsVBEzgeqiUhL4AZgSmLDSm05OfavJwpXZc2bZ5Uqc3Nh0CD417+8Bk0G\ni6VFcR3QC9gLvAnsBG5MZFCpLicHGje2yq7OVUmrVsHKlTa76a23PElkuFhaFKep6m3AbeE7ROQc\nLGlUSTk5cNxxXpbDVTFffWWL50aOtPUQy5f77lhVRCwtij8Uc98d8Q4kXaxaZZUIvNvJVRlbt8LN\nN9tc8HvvhZ077X5PElVGiS0KETkN6A80FpHoIhX1sG6oKsnHJ1yV8sknVgZ8+XIranbPPVCzZtBR\nuSQrrevpB2AesAOYH3X/ZuD2RAaVyrKzoU4d2+vduYyWlwennWbljz/7DE44IeiIXEBKTBSqOhOY\nKSIvqeqOJMaU0nJyoE8fqOHFT1ymmjkTevSwIn7vvAN9+8IBBwQdlQtQLGMUjUVknIjMEZHF4a+E\nR5aCNm2yXeq828llpLVrYcgQ25g9XMSvf39PEi6mRDEWeA4QYADwKvCfBMaUssKFAI87LuhInIsj\nVfj3v6FjRxg/Hu6+2+rnOxcSS6KoraofAKjqMlX9A5YwqpxwIcA+fYKOxLk4GjbMCvkdeaTtYX3H\nHbDffkFH5VJILD3tO0WkGrBMRK4G1gBVcnPbnBwbxPatfV3a27vXFgKJwKmn2tTXa6/1+kyuWLG0\nKG4C6mClO34JXAlclsigUpEXAnQZY/Fiq/D67LN2+9JLbe8ITxKuBGW2KFT1q9C3m4HhACLSOJFB\npaLZs23dkScKl7b27LHy33feCbVq+SC1i1mpLQoROUpEzhaRhqHbnUTkBeCr0p6XiXyhnUtrc+bY\n4Nptt8GAAbBggY1NOBeDEhOFiPwdeAm4EHhfRP4MTAJmA+2SEl0KycmBpk29EKBLU3l5sHo1vPYa\nvPEGHH540BG5NFJa19MgoJuqbheRQ4DVQBdVXR7ri4tIf+BhoDowRlXvKeYx5wN/xva4mK2qKfcx\nR9USxfHHBx2Jc+XwxRfWkrj66kgRvzp1go7KpaHSup52qOp2AFX9CVhcziRRHXgcm0rbEbhARDoW\nekxb4PfAL1W1E/CbcsafFKtWwZo13u3k0sSWLXDjjbbg54EHIkX8PEm4CiqtRdFKRMKlxAVoGXUb\nVT2njNfuDSwNJxcRGYe1UhZEPeZK4HFVXR96zR/KGX9S+PiESxsffmhlwFetsumuf/ubF/FzlVZa\nohhc6PZj5Xztxlh3VVgetvd2tHYAIpKDdU/9WVXfL/xCIjISGAnQLIBBguxsq6jcpUvST+1c7Fav\nhjPOgNatYfJkLyHg4qa0ooAfJ+n8bYF+QBNgsoh0KbxHt6qOBkYDZGVlJX2/7pwcW4/khQBdSpo+\nHXr1stkWEyfaYFqtWkFH5TJILAvuKmoN0DTqdpPQfdHygAmqultVVwCLscSRMjZu9EKALkV9/z2c\ndx5kZUWK+J1yiicJF3eJTBTTgLYi0lJE9geGAhMKPWY81pogtFajHRDzgHkyTJlis548UbiUoQrP\nP29F/N55x8YhvIifS6CYO1NEpKaq7oz18aq6R0SuAz7Axh+eVdX5InIXkKuqE0LHThWRBUABcKuq\nrivfj5BY4UKARxceXXEuKEOHwquv2qeXMWOgffugI3IZTlRL7/IXkd7AM0B9VW0mIt2AK1T1+mQE\nWFhWVpbm5uYm7XwnnQTr18OMGUk7pXNFRRfxe/552LwZrrnGPsU4FwMRma6qWRV5biy/ZY8AZwLr\nAFR1NnBiRU6Wbnbv9kKALgV8/bVtQ/rMM3b7kkvguus8SbikieU3rZqqrix0X0Eigkk1s2fDtm0+\ny9AFZPduG3/o1s1qMx14YNARuSoqljGK1aHuJw2ttr4em52U8XyhnQvMrFlW/nvWLDj3XHj0UfjF\nL4KOylVRsSSKUVj3UzNgLfC/0H0ZLyfHigA2aRJ0JK7K+f57+3rjDTinrCIIziVWLIlij6oOTXgk\nKSZcCLBv36AjcVVGdrYV8bvmGujfH5Ytg9q1g47KuZjGKKaJyEQRuUREqswmoCtXwrffereTS4LN\nm21w+vjj4Z//jBTx8yThUkSZiUJVWwN3A72AuSIyXkQyvoWRnW3/eqJwCfXBB9C5M/zrX1bxdcYM\nL+LnUk5M8+tU9QtVvQHoCWzCNjTKaDk5ULeuFwJ0CbR6NZx5prUcsrOtNeEzm1wKKjNRiMiBInKh\niLwDTAXygYyvFxAuBOj7zbu4UoWpU+37pk3hvfdg5kwvweFSWiwtinlAH+BeVW2jqreoakbvmb1h\nA8yb591OLs6++w4GD7Z6MOEifief7EX8XMqLZdZTK1Xdm/BIUogXAnRxpQpjx8LNN8OOHfCPf/gv\nl0srJSYKEXlAVW8B3hCRIgWhYtjhLm3l5FiXkxcCdHFx/vnw+us2q2nMGGjXLuiInCuX0loU/wn9\nW96d7dJedrZVTfBxRVdhBQVWwK9aNTjrLPjVr+Cqq7w+k0tLJf7WqmpoxI0Oqvpx9BfQITnhJd/u\n3fDVV17fyVXCwoXWeggX8bv4Yhg1ypOES1ux/OZeVsx9l8c7kFQxaxZs3+5dyK4Cdu+Gu++G7t1h\n0SKoXz/oiJyLi9LGKIZgu9K1FJE3ow7VBTYU/6z054UAXYXMnAkjRlgJjiFD4JFH4NBDg47Kubgo\nbYxiKrYHRRPg8aj7NwMzExlUkHJyoHlzaNw46EhcWlm7Fn78EcaPh0GDgo7GubgqMVGo6gpgBVYt\ntkoIFwI8sUpsy+QqbfJkmDsXrr3WivgtXQoHHBB0VM7FXYljFCLyWejf9SLyU9TXehH5KXkhJs+K\nFbYmyrudXKk2bbIKr337WhdTuIifJwmXoUobzA5/rm4INIr6Ct/OOOHxCZ/x5Eo0cSJ06gRPPWUL\n6LyIn6sCSpseG16N3RSorqoFwDHAVUCdJMSWdDk5UK+evQ84V8Tq1Tb+UL8+fPEFPPAA1MnIPwXn\n9hHL9Njx2DaorYHngLbAywmNKiBeCNAVoWo1XcCK+H34obUifNm+q0JiSRR7VXU3cA7wqKreBGTc\nnKANG2D+fB+fcFG+/RbOPts+PYSL+J14Iuy/f7BxOZdksSSKPSJyHjAceDd0336JCykYX37phQBd\niKrVZOrY0VoQ99/vvxiuSouleuxlwDVYmfHlItISeCWxYSVfdrYXAnQh554Lb75ps5rGjIE2bYKO\nyLlAlZkoVHWeiNwAtBGR9sBSVf1r4kNLrpwc6NHDxyarrOgifmefDaeeClde6fWZnCO2He6OB5YC\nzwDPAotFJKPa4bt326Zj3rtQRYV3qQoX8Rs+3Cu9Ohcllr+Eh4DTVfWXqnoscAbwcGLDSq6ZM70Q\nYJW0axf85S/QsycsWwYHHxx0RM6lpFjGKPZX1QXhG6q6UEQyatqHFwKsgqZPtyJ+8+bBsGHwz39C\no4xcR+pcpcWSKGaIyJPAv0O3LyTDigLm5ECLFnDEEUFH4pJm3TqbE/3OO3DmmUFH41xKiyVRXA3c\nAPwudPtz4NGERZRkqjbj6eSTg47EJdykSVbE74YbbLB6yRKoVSvoqJxLeaUmChHpArQG3lLVe5MT\nUnItX24Vor2+UwbbuBF+9zsYPRrat7eB6po1PUk4F6PSqsf+H1a+40LgIxEpbqe7tOfjExnunXds\n4dyYMfDb39rYhBfxc65cSmtRXAh0VdWtItIImIhNj80oOTlW480LAWag1ath8GBrRYwfD0cdFXRE\nzqWl0qbH7lTVrQCqml/GY9NWuBCgT5nPEKpW2RUiRfxycz1JOFcJpb09thKRN0NfbwGto26/Wcrz\nfiYi/UVkkYgsFZHbS3ncYBFREckq7w9QGevXeyHAjJKXBwMH2n9ouIhfv35exM+5Siqt62lwoduP\nleeFRaQ6ttf2KUAeME1EJkSvyQg9ri5wI/BVeV4/HsIfPH0gO83t3QtPPw233gp79sCDD/p/qnNx\nVNqe2R9X8rV7Y3WhlgOIyDhgELCg0OP+H/AP4NZKnq/ccnKgRg3o3TvZZ3ZxNXiwjUH86leWMFq1\nCjoi5zJKInvmGwOro27nUWgfCxHpCTRV1f+W9kIiMlJEckUkNz8/P24BhgsB1q4dt5d0ybJnj7Uk\nwBLF00/D//7nScK5BAhsCFdEqgEPAreU9VhVHa2qWaqa1ShOZRZ27fJCgGlrzhybgfD003b7oovg\niius+qtzLu5iThQiUt7J52uw/bbDmoTuC6sLdAY+FZFvgD7AhGQNaM+cCTt2eKJIKzt3wp13Qq9e\nsHKl12ZyLkliKTPeW0TmAktCt7uJSCwlPKYBbUWkZaiI4FBgQvigqm5U1Yaq2kJVWwBTgIGqmluR\nH6S8fKFdmpk2zaq83nUXXHABLFwI55wTdFTOVQmxtCgeAc4E1gGo6mzgxLKepKp7gOuAD4CFwKuq\nOl9E7hKRgRUPOT6ys607+/DDg47ExWT9etiyBSZOhBdegAYNgo7IuSojlqKA1VR1pezb/1sQy4ur\n6kRsRXf0fX8q4bH9YnnNeLZs450AABcuSURBVFC1FsVppyXrjK5CPvnEivjdeKMV8Vu82MtvOBeA\nWFoUq0WkN6AiUl1EfgMsTnBcCbVsGfzwg3c7pawNG2wb0pNOgqeesrEJ8CThXEBiSRSjgJuBZsBa\nbNB5VCKDSjQfn0hhb79tRfyefdYqvnoRP+cCV2bXk6r+gA1EZ4ycHDjoIHs/cilk1So47zzo0AEm\nTICspFZ0cc6VoMxEISJPA1r4flUdmZCIksALAaaQ8M5Rxx8PzZrZork+fbw+k3MpJJa3yv8BH4e+\ncoBDgZ2JDCqRfvoJFizwUkApYdUqOOMMOOGESBG/E07wJOFcioml6+k/0bdF5EUgO2ERJVi4EKCP\nTwRo71548km47TZrUTzyiGdu51JYLNNjC2sJHBbvQJIlXAjQtycI0Dnn2KD1KafY9qQtWgQdkXOu\nFLGMUawnMkZRDfgJKHFviVSXk2MLfL0QYJLt2WODQtWqwZAhMGgQjBjh9ZmcSwOljlGIrbLrBjQK\nfR2sqq1U9dVkBBdvu3ZZJQjvdkqy2bPh6KOt9QBWguPSSz1JOJcmSk0UqqrARFUtCH0Vmf2UTmbM\n8EKASbVjB/zhDzbNNS8PfvGLoCNyzlVALLOeZolIj4RHkgTZoSF4TxRJMHWqbfbx17/ChRdaEb+z\nzw46KudcBZQ4RiEiNUKF/Xpg25guA7YCgjU2eiYpxrjJyYHWrf2DbVJs2gTbt8P773tRLefSXGmD\n2VOBnkDglV7jIVwIcMCAoCPJYB9+CPPnw003wcknw6JFXn7DuQxQWqIQAFVdlqRYEmrpUsjP926n\nhFi/Hm6+GcaOhU6d4JprLEF4knAuI5SWKBqJyM0lHVTVBxMQT8J4IcAEefNNuPZay8K//z386U+e\nIJzLMKUliurAgYRaFukuXAiwQ4egI8kgq1bB0KHQubNtKNQjI+Y8OOcKKS1RfKeqdyUtkgTLzrbW\nhBcCrCRVmDwZ+va1In6ffGJrJPbbL+jInHMJUtrbZka0JADWrYOvv/Zup0pbudJmA/TrFynid9xx\nniScy3ClJYqTkhZFgnkhwErauxcee8wGqrOz4dFHrSy4c65KKLHrSVV/SmYgiZSTYx96vRBgBZ19\nNrzzjq2HeOopaN486Iicc0lUkeqxaSdcCPCAA4KOJI3s3g3Vq9ugzgUXwLnnwvDhXp/JuSoo44d2\nd+70QoDlNmMG9O5te0aAJYqLL/Yk4VwVlfGJYvp0Sxa+L04Mtm+3tRC9e8P330PTpkFH5JxLARnf\n9RReaHfsscHGkfKmTIFLLoHFi+Gyy+D+++Hgg4OOyjmXAqpEomjTBg5L2z35kmTrVhuX+Ogjq9Pk\nnHMhGZ0oVG1q7OmnBx1Jinr/fSvid8stcNJJtthk//2Djso5l2IyeoxiyRIvBFisdeusm2nAAHj+\nedv6DzxJOOeKldGJwgsBFqIKr78OHTvCyy/b7nPTpnmCcM6VKqO7nrKz4ZBDoH37oCNJEatWwbBh\n0LWr7R3RrVvQETnn0kDGtyiOPbaKFwJUtcJ9YCuqP/3UZjh5knDOxShj30J//NE2WKvS3U4rVsCp\np9pAdbiI37HHQo2Mbkg65+IsYxNFlS4EWFAADz9s+0R89RU88YQX8XPOVVjGfrQMFwLMygo6kgAM\nGgT//a/NC37ySV9h7ZyrlIxOFL16VaFCgNFF/IYPt/pMw4Z5fSbnXKUltOtJRPqLyCIRWSoitxdz\n/GYRWSAic0TkYxGJS/3qHTts1meVqe+Um2tNpyeesNtDhsCFF3qScM7FRcIShYhUBx4HBgAdgQtE\npGOhh80EslS1K/A6cG88zj19uq0hy/jxie3b4bbbbCvS/HzfJ8I5lxCJbFH0Bpaq6nJV3QWMAwZF\nP0BVJ6nqttDNKUCTeJy4ShQC/PJLm+J6771WxG/BAjjzzKCjcs5loESOUTQGVkfdzgOOLuXxlwPv\nFXdAREYCIwGaNWtW5olzcqBtWzj00JhjTT/bt9sWpf/7n01/dc65BEmJ6bEichGQBdxX3HFVHa2q\nWaqa1ahRo1JfK1wIMCO7nSZOhPtCl+hXv4KFCz1JOOcSLpGJYg0QPS+zSei+fYjIycAdwEBV3VnZ\nky5ebIvtMmog+8cf4aKL4Iwz4KWXIkX89tsv2Licc1VCIhPFNKCtiLQUkf2BocCE6AeISA/gKSxJ\n/BCPk2ZUIUBVGDcOOnSAV1+FO++EqVO9iJ9zLqkSNkahqntE5DrgA6A68KyqzheRu4BcVZ2AdTUd\nCLwmNpVzlaoOrMx5s7OhQQM48shK/gCpYNUqKwferRs88wx06RJ0RM65KiihC+5UdSIwsdB9f4r6\nPu5bqYULAabtEgJV+Phj22WueXOr0XTUUbaYzjnnApASg9nxkp9vYxRp2+20bJkNTp9ySqSIX58+\nniScc4HKqESRtoUACwrgwQeta2n6dHjqKS/i55xLGRlV6yknx8Z5064Q4FlnwXvv2YK5J56AJnFZ\nd+icc3GRcYkiKwtq1Qo6khjs2mX7QlSrBiNGWCG/oUPTeHDFOZepMqbraccOq42XFt1OU6daadt/\n/ctun3++VXv1JOGcS0EZkyhyc9OgEOC2bXDLLXDMMbB+PbRuHXREzjlXpozpekr5QoDZ2bYmYvly\nuOoq+Mc/oH79oKNyzrkyZVSiaNcOyigFFZzwxkKTJkG/fkFH45xzMcuIrqdwIcCUq+/0zjtWBhzg\nxBOtFLgnCedcmsmIRLFoEaxbl0LjE/n5tg3pwIHwyiuRIn41MqYB55yrQjIiUWRn27+BJwpVePll\nK+L3+utw113w1VdexM85l9Yy4iNuTg40bGhjFIFatQouvRR69LAifp06BRyQc85VXka0KAItBLh3\nL3zwgX3fvDl8/rkF5EnCOZch0j5R/PADLFkSULfTkiW201z//jB5st3Xu7cX8XPOZZS0TxThQoBJ\nnfG0Z49tSdq1K8yaZd1MXsTPOZeh0n6MIicHata0ihhJc+aZ1t00aJCV4TjiiCSe3Ln0sXv3bvLy\n8tixY0fQoVQZtWrVokmTJuwXx62S0z5RZGdbIcCaNRN8op07bY/qatXgiivgssvgvPO8PpNzpcjL\ny6Nu3bq0aNEC8b+VhFNV1q1bR15eHi1btozb66Z119P27bZ9Q8LHJ6ZMgZ494fHH7fa551ohP//F\nd65UO3bsoEGDBp4kkkREaNCgQdxbcGmdKHJzrTJGwhLF1q1w0002pWrzZmjbNkEnci5zeZJIrkRc\n77TuekpoIcDPP7cifitWwDXXwN//DvXqJeBEzjmX2tK6RZGTA+3b22K7uNuzx8YkPvvMupw8STiX\ntsaPH4+I8PXXX/9836effsqZZ565z+NGjBjB66+/DthA/O23307btm3p2bMnxxxzDO+9916lY/n7\n3/9OmzZtOPLII/kgvAarkI8//piePXvSvXt3jjvuOJYuXQrA5MmT6dmzJzVq1Pg5zmRI20Sxd69N\njY1rt9P48dZyACviN38+nHBCHE/gnAvCK6+8wnHHHccrr7wS83P++Mc/8t133zFv3jxmzJjB+PHj\n2bx5c6XiWLBgAePGjWP+/Pm8//77XHPNNRQUFBR53KhRo3jppZeYNWsWw4YN4+677wagWbNmjB07\nlmHDhlUqjvJK266nr7+Gn36KU6JYuxauvx5ee80GrW+5xeozeRE/5+LmN7+xZUfx1L07/POfpT9m\ny5YtZGdnM2nSJM466yz+8pe/lPm627Zt4+mnn2bFihXUDE2pPOywwzj//PMrFe/bb7/N0KFDqVmz\nJi1btqRNmzZMnTqVY445Zp/HiQibNm0CYOPGjRwRmoLfokULAKpVS+5n/LR9JwyPT1QqUajCv/9t\nv8FbtsBf/wq33mpdTs65jPD222/Tv39/2rVrR4MGDZg+fTq9ylh4tXTpUpo1a0a9GLqcb7rpJiZN\nmlTk/qFDh3L77bfvc9+aNWvo06fPz7ebNGnCmjVrijx3zJgxnH766RxwwAHUq1ePKVOmlBlHIqV1\nomjUqJITkVatsjURWVm2urp9+7jF55zbV1mf/BPllVde4cYbbwTszfuVV16hV69eJc4OKu+soYce\neqjSMRb3mhMnTuToo4/mvvvu4+abb2bMmDFxP0+s0jpRVKgQYLiI34ABVsQvJ8eqvXp9Jucyzk8/\n/cQnn3zC3LlzEREKCgoQEe677z4aNGjA+vXrizy+YcOGtGnThlWrVrFp06YyWxXlaVE0btyY1atX\n/3w7Ly+Pxo0b7/OY/Px8Zs+ezdFHHw3AkCFD6N+/f7l+7nhLy8HstWth6dIK1HdavNh2mDv9dJvN\nBNaa8CThXEZ6/fXXGT58OCtXruSbb75h9erVtGzZks8//5y2bdvy7bffsnDhQgBWrlzJ7Nmz6d69\nO7Vr1+byyy/nxhtvZFdo47H8/Hxee+21Iud46KGHmDVrVpGvwkkCYODAgYwbN46dO3eyYsUKlixZ\nQu/evfd5zMEHH8zGjRtZvHgxAB999BEdOnSI96UpH1VNq69evXrpm2+qguoXX2hsdu9Wvece1Zo1\nVQ86SPW551T37o3xyc65ilqwYEGg5+/Xr5++9957+9z38MMP69VXX62qqtnZ2Xr00Udrt27dNCsr\nSz/88MOfH7dz50699dZbtXXr1tqpUyft3bu3vv/++5WO6e6779ZWrVppu3btdOLEiT/fP2DAAF2z\nZo2qqr755pvauXNn7dq1q/bt21eXLVumqqpTp07Vxo0ba+3atfWQQw7Rjh07FnuO4q47kKsVfN8V\ne376yMrK0r59c3n8cdi4McYaT6edBh9+COecY2sifvGLhMfpnIOFCxcG/2m4CiruuovIdFXNqsjr\npeUYRU4OHHVUGUlixw6bvVS9OowcaV+DByctRuecyxRpN0axdy/MmFHGtNicHJtgHS7iN3iwJwnn\nnKugtEsU27aVUghwyxa44QbbRGjHDvAmr3OBS7fu7XSXiOuddoliyxb7t0ghwM8+g86d4bHH4Lrr\nYN48OOWUpMfnnIuoVasW69at82SRJBraj6JWrVpxfd20G6PYssUaCg0aFHOwdm2r+hrIBtrOucKa\nNGlCXl4e+fn5QYdSZYR3uIuntEwUP+eBN9+0ok//93/Qty/MnetrIpxLIfvtt19cd1pzwUho15OI\n9BeRRSKyVESKrD4RkZoi8p/Q8a9EpEVZr1lQACd3/t52mRs8GN56C0ILYjxJOOdc/CUsUYhIdeBx\nYADQEbhARDoWetjlwHpVbQM8BPyjrNdtwDrO/VMHePddKwn+xRdW6dU551xCJLJF0RtYqqrLVXUX\nMA4YVOgxg4DnQ9+/DpwkZVTkas5KqnXtDLNnw+23e6VX55xLsESOUTQGVkfdzgOOLukxqrpHRDYC\nDYAfox8kIiOBkaGbO6tlZ8/zSq8ANKTQtarC/FpE+LWI8GsRcWRFn5gWg9mqOhoYDSAiuRVdhp5p\n/FpE+LWI8GsR4dciQkRyK/rcRHY9rQGaRt1uErqv2MeISA2gPrAugTE555wrp0QmimlAWxFpKSL7\nA0OBCYUeMwG4JPT9ucAn6itznHMupSSs6yk05nAd8AFQHXhWVeeLyF1YudsJwDPAiyKyFPgJSyZl\nGZ2omNOQX4sIvxYRfi0i/FpEVPhapF2Zceecc8mVdrWenHPOJZcnCuecc6VK2USRiPIf6SqGa3Gz\niCwQkTki8rGINA8izmQo61pEPW6wiKiIZOzUyFiuhYicH/rdmC8iLyc7xmSJ4W+kmYhMEpGZob+T\n04OIM9FE5FkR+UFE5pVwXETkkdB1miMiPWN64YruoZrIL2zwexnQCtgfmA10LPSYa4AnQ98PBf4T\ndNwBXosTgdqh70dV5WsRelxdYDIwBcgKOu4Afy/aAjOBg0O3Dw067gCvxWhgVOj7jsA3QcedoGtx\nAtATmFfC8dOB9wAB+gBfxfK6qdqiSEj5jzRV5rVQ1Umqui10cwq2ZiUTxfJ7AfD/sLphO5IZXJLF\nci2uBB5X1fUAqvpDkmNMlliuhQL1Qt/XB75NYnxJo6qTsRmkJRkEvKBmCnCQiBxe1uumaqIorvxH\n45Ieo6p7gHD5j0wTy7WIdjn2iSETlXktQk3ppqr632QGFoBYfi/aAe1EJEdEpohI/6RFl1yxXIs/\nAxeJSB4wEbg+OaGlnPK+nwBpUsLDxUZELgKygL5BxxIEEakGPAiMCDiUVFED637qh7UyJ4tIF1Xd\nEGhUwbgAGKuqD4jIMdj6rc6qujfowNJBqrYovPxHRCzXAhE5GbgDGKiqO5MUW7KVdS3qAp2BT0Xk\nG6wPdkKGDmjH8nuRB0xQ1d2qugJYjCWOTBPLtbgceBVAVb8EamEFA6uamN5PCkvVROHlPyLKvBYi\n0gN4CksSmdoPDWVcC1XdqKoNVbWFqrbAxmsGqmqFi6GlsFj+RsZjrQlEpCHWFbU8mUEmSSzXYhVw\nEoCIdMASRVXcn3UCcHFo9lMfYKOqflfWk1Ky60kTV/4j7cR4Le4DDgReC43nr1LVgYEFnSAxXosq\nIcZr8QFwqogsAAqAW1U141rdMV6LW4CnReQmbGB7RCZ+sBSRV7APBw1D4zF3AvsBqOqT2PjM6cBS\nYBtwaUyvm4HXyjnnXBylateTc865FOGJwjnnXKk8UTjnnCuVJwrnnHOl8kThnHOuVJ4oXMoRkQIR\nmRX11aKUx7YoqVJmOc/5aaj66OxQyYsjK/AaV4vIxaHvR4jIEVHHxohIxzjHOU1EusfwnN+ISO3K\nnttVXZ4oXCrarqrdo76+SdJ5L1TVblixyfvK+2RVfVJVXwjdHAEcEXXsClVdEJcoI3H+i9ji/A3g\nicJVmCcKlxZCLYfPRWRG6OvYYh7TSUSmhlohc0Skbej+i6Luf0pEqpdxuslAm9BzTwrtYTA3VOu/\nZuj+eySyB8j9ofv+LCK/FZFzsZpbL4XOeUCoJZAVanX8/OYeank8VsE4vySqoJuIPCEiuWJ7T/wl\ndN8NWMKaJCKTQvedKiJfhq7jayJyYBnncVWcJwqXig6I6nZ6K3TfD8ApqtoTGAI8UszzrgYeVtXu\n2Bt1XqhcwxDgl6H7C4ALyzj/WcBcEakFjAWGqGoXrJLBKBFpAPwa6KSqXYG7o5+sqq8Dudgn/+6q\nuj3q8Buh54YNAcZVMM7+WJmOsDtUNQvoCvQVka6q+ghWUvtEVT0xVMrjD8DJoWuZC9xcxnlcFZeS\nJTxclbc99GYZbT/gsVCffAFWt6iwL4E7RKQJ8KaqLhGRk4BewLRQeZMDsKRTnJdEZDvwDVaG+khg\nhaouDh1/HrgWeAzb6+IZEXkXeDfWH0xV80VkeajOzhKgPZATet3yxLk/VrYl+jqdLyIjsb/rw7EN\neuYUem6f0P05ofPsj10350rkicKli5uAtUA3rCVcZFMiVX1ZRL4CzgAmishV2E5ez6vq72M4x4XR\nBQRF5JDiHhSqLdQbKzJ3LnAd8Kty/CzjgPOBr4G3VFXF3rVjjhOYjo1PPAqcIyItgd8CR6nqehEZ\nixW+K0yAj1T1gnLE66o473py6aI+8F1o/4DhWPG3fYhIK2B5qLvlbawL5mPgXBE5NPSYQyT2PcUX\nAS1EpE3o9nDgs1Cffn1VnYglsG7FPHczVva8OG9hO41dgCUNyhtnqKDdH4E+ItIe271tK7BRRA4D\nBpQQyxTgl+GfSUTqiEhxrTPnfuaJwqWLfwGXiMhsrLtmazGPOR+YJyKzsH0pXgjNNPoD8KGIzAE+\nwrplyqSqO7Dqmq+JyFxgL/Ak9qb7buj1sim+j38s8GR4MLvQ664HFgLNVXVq6L5yxxka+3gAqwo7\nG9sf+2vgZaw7K2w08L6ITFLVfGxG1iuh83yJXU/nSuTVY51zzpXKWxTOOedK5YnCOedcqTxROOec\nK5UnCuecc6XyROGcc65Uniicc86VyhOFc865Uv1/K8BKLm8q7JQAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "\n",
            "\n",
            "\n",
            "Use word vectors on - IF-IDF\n",
            "\n",
            "Accuracy:  0.8059499661933739\n",
            "Recall:  0.8264352469959947\n",
            "Precision:  0.7976804123711341\n",
            "F1:  0.8118032786885246\n",
            "Roc_AUC:  0.8056833769226549\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEWCAYAAAB42tAoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nO3dd3hUZfbA8e8BFUQRBawgNbGACEIE\nrGAHRHBFaYqiuFhXV6z7c4u67tp17SaAYgUVBVGxiyIqUqRIUQg92JDeSzi/P86MmcSUScjMnXI+\nz5PHzMydOyfXcE/e99x7XlFVnHPOuZJUCToA55xzic0ThXPOuVJ5onDOOVcqTxTOOedK5YnCOedc\nqTxROOecK5UnCuecc6XyROFSgogsFpFtIlK3yPPTRERFpFEF9jkstM/1oa9ZInKPiNQqZ1ynl2P7\n+iLyhoj8JiJrQ5/ZP8r3fiYil0f7Wc5FyxOFSyWLgD7hByLSAqixi/u8X1VrAvsDlwLtgS9FZK9d\n3G9JXgSWAQ2BOkA/4JcYfZZzUfFE4VLJi8DFEY8vAV4IPxCRY0XkFxGpGvHceSIyo6wdq+oWVZ0M\ndMNO4JeG3t9URD4VkZWhUcDLIrJv6LUXgQbA2yKyQURuCT3/uoj8HBoxjBeR5hEfdSwwTFU3quoO\nVZ2mqu9FxNteRL4SkTUiMkNEOoae/w9wEvBE6LOeKNeRc64UnihcKpkI7CMiR4aSQW/gpfCLoRP9\nSuDMiPf0IyKZlEVV1wMfYSdlAAHuAQ4BjgQOBe4IbdsPWAqco6p7q+r9ofe8B2QCBwDfAi8X+Rme\nFJHeItIg8rNFpB7wLnA3UBu4CXhDRPZX1duBL4BrQ591bbQ/k3Nl8UThUk14VHEGMBdYXuT154GL\nAESkNnAW8Eo5P+NH7ESNquaq6kequlVVVwAPAx1Ke7OqPquq61V1K5ZUWkbUPS7ATvj/ABaJyHQR\nOTb02kXAWFUdq6o7VfUjYArQpZzxO1cunihcqnkR6Av0p/iRwkvAOaEaQ0/gC1X9qZyfUQ9YBSAi\nB4rICBFZLiLrQvuvW9IbRaSqiNwrIgtC2y8OvVQXQFVXq+ptqtocOBCYDowWEcHqFheEpp3WiMga\n4ETg4HLG71y5eKJwKUVVl2BF7S7Am8W8vhz4GjgPm3Z6sTz7F5G9gdOxv/oB/gso0EJV98H+6pfI\njyyyi75A99A+agGNwrsuJtbfgAexaa3aWJH7RVXdN+JrL1W9t4TPcq5SeKJwqWgAcKqqbizh9ReA\nW4AWFJNMiiMi1USkDTAaWA08F3qpJrABWBuqIdxc5K2/AE0iHtcEtmK1khpYoon8nPtE5CgR2U1E\nagJXAbmqupKC0dBZoZFJdRHpKCL1S/gs5yqFJwqXclR1gapOKWWTUdg0zihV3VTG7m4RkfXYif0F\nYCpwfEQSuhNoDazFCs1FE889wN9DU0U3hfaxBKudzMGK15FqhOJbAywMxdkt9HMtw0Yj/weswEYY\nN1Pw7/hR4HwRWS0ij5XxczkXNfGFi1w6EpEFwBWq+nHQsTiX6HxE4dKOiPTA5vM/DToW55JBzBKF\niDwrIr+KyKwSXhcReUxEckVkpoi0jlUszoWJyGfA08A1qroz4HCcSwqxHFEMAzqV8npn7KajTGAg\n9o/XuZhS1Y6qeoCqfhB0LM4li5glClUdT+ha8xJ0B15QMxHYV0T8enDnnEswuwX42fWwqzbC8kLP\n/eHmJxEZiI062GuvvdocccQRcQnQOecSXX4+bN0KW7bYfyO/37EDGrCEfVnDTHb8pqr7V+QzgkwU\nUVPVHCAHICsrS6dMKe3KR+ecSy3r18P8+ZCba/8Nf+Xmwq+/Ft62Xj1ocZSSkQGZhwlnLniag3f7\nlYOeumNJRT8/yESxHGugFlafP/blcc65tLBhQ+FEEPn9L0UazR9yCGRkwDnnQGamfWVkQNOmsNea\n5XDVVXBKL7jwQuyeTeCpOyocW5CJYgxwrYiMANoBayvQc8c555LGxo0FCaDo6ODnnwtve9BBlgDO\nPrsgEYT/u1dxq6GowpAhcNNNsH27vbGSxCxRiMhwoCNQV0TygH8BuwOo6jPAWKwfTy6wiVB/f+ec\nS2abNsGCBYWTQDgx/Phj4W0PPNBO/p06FYwMMjNtZFCzZjk+dMEC+POfYdw4OOUUGDzYdlJJYpYo\nVLVPGa8rcE2sPt8552Jl8+bCySBydLC8yAT6AQfYKOCMMwpPE2VkwD77VFJA330HU6dCTg5cfjnI\nH3pM7pKkKGY751y8bdliyaDoFNH8+ZCXV3jbunUtAZx22h+niWpFvcJ6Oc2aBd9+CxdfDOeeCwsX\nQp06MfkoTxTOubS1daudX4tOEc2fD8uW2bR/WJ06dvLv2LHwNFFGBuy7bxyD3rYN/vtf+zrwQOjZ\nE6pXj1mSAE8UzrkUt21b4WQQOUJYurRwMqhd2078J51UOBFkZsJ++wX3M/zum29gwACYPRsuugge\necSSRIx5onDOJb1t22DRouLvM1iyBHZGdPXad1878Z9wAvTvX5AIMjMtUSSs5cstgx14ILzzTqVe\n1VQWTxTOuaSwfTssXlz8NNGSJXaHclitWnbib9fO/vCOnCqqXbvSa72xNW8eHHaY3Un36qtWCKm0\nKnh0PFE45xLGjh0FyaDo6GDx4sLJoGZNO/Efeyz07Vt4mqhu3SRLBsVZswZuucXujfjsMzj5ZPjT\nnwIJxROFcy6uduywEUBx00SLFtnrYXvvbSf+Nm2gd+/C00T7758CyaAkY8bY3dU//ww332zZMECe\nKJxzlS4/3wrFxRWQFy2yaaSwvfayE3/LlnD++YWniQ44IIWTQUkuvxyGDoUWLeCttyArK+iIPFE4\n5yomP98uIS1ummjhwsLJoEYNGw20aAHnnVd4muigg9IwGRQVvvRKxBJDw4Zw662wxx7BxhXiicI5\nV6KdO+3msuIKyAsW2NVGYXvuaSf/5s3t/q/IaaKDD/ZkUKJly+DKK21urV8/+z7BeKJwLs3t3GlX\nXhY3TbRggd2UFla9uiWAww+Hrl0LTxMdfDBUieWamalm507IzraRQ35+YIXqaHiicC4N7NxpDemK\na0exYIG1qwirVs36yWVmQpcuhaeJ6tXzZFAp5s+3WsT48XD66dajqXHjoKMqkScK51KEKvz0U/HT\nRLm51sgubI89CpJBp06Fp4nq1/dkEHNz5sDMmfDss3bXX4LPy3micC6JqNoVk8VNE+XmWovrsN13\nt2SQkWF/tEZOE9WvD1WrBvdzpKUZM2D6dLjkEuje3Sr+CdEXpGyeKJxLMKq2ollx9xnk5tpKaGG7\n7QZNmtjJ/9RTC08TNWjgySAhbN0Kd98N995rhZxevazYkyRJAjxROBcIVVixouRpovXrC7bdbTeb\nvs7MhA4dCk8TNWhgr7sE9fXX1sRv7lxrB/7ww3Fp4lfZ/FfMuRhRhd9+K/4+g9xcWLeuYNuqVS0Z\nZGTAiScWniZq2NCTQVJavtwy+0EHwdix0Llz0BFVmP/6ObcLVGHlyuITwfz5sHZtwbZVqkCjRnby\nP/74wtNEjRpZTcGlgLlz4cgj7RKx116zJn7lWtc08XiicC4Kq1YVnwjmz7febWFVqtgIIDPTupZG\nThM1apQwN9q6WFi9Gm68EZ57zi57Pekku/MwBXiicC5k9eqSp4lWrSrYTsSSQUYG9OlTeIGbxo3t\nPgSXZkaNgquvtsLT3/4WeBO/yuaJwqWVNWtKniZaubJgOxE49FBLAD17Fp4matLEk4GLcNllNopo\n1QrefRdatw46okrnicKlnHXrip8imj/fisuRwsmgR4/CBeQmTZLy4hQXL5FN/Nq3t1+am25K2UKT\nJwqXlNavL3ma6NdfC29bv76NBv70p8LTRE2bWiM758plyRK44gpbLenii2HgwKAjijlPFC5hbdjw\nxzuPw9//8kvhbQ85xBJAt26Fp4maNrUW187tsp074emn4bbbbERxwQVBRxQ3nihcoDZuLHyjWeTo\n4OefC2978MF28j/77MLTRE2b2uI3zsXMDz9YE78JE+DMM63ra6NGQUcVN54oXMxt2mQdSot2Lc3N\ntY6mkQ46yEYDnTsXnibKyLBlMZ0LxA8/wOzZMGyYTTcleBO/yuaJwlWKzZsLJ4PI0cHy5YW3PeAA\nSwBnnll4migjI+nvS3KpZNo0a+J36aU2p7lwIey7b9BRBcIThYvali2WDIpb0yAvr/C2++9vJ//T\nTis8TZSRAfvsE0z8zkVlyxa46y64/367u7pPH7sELk2TBHiicEVs3Wp/OBV3n8GyZQVXBQLUrWsn\n/lNOKZwIMjOhVq3gfgbnKuzLL62J3w8/2EjioYf8Omk8UaSlbdsKJ4PIEcLSpYWTQe3aduI/+eQ/\nThMlUZdk58q2fLn91VOvHnzwgc2NOsATRcratg0WLSr+PoMlS+xKv7D99rOT/wkn2GJbkaOD2rUD\n+xGci485c6BZM0sQb7xhycKvnCjEE0UK+egjGynPn2/JID+/4LVatezk37499OtXeHRQp05wMTsX\nmFWrYNAgeP55+PxzGzafc07QUSUkTxQpIj/fplbz8+33/cILC3curVMn7a7oc65kb7wB11xjDb5u\nvx3atg06ooTmiSJFvP++FZtHjrS+Rc65EvTvb6OI1q3tH06rVkFHlPA8UaSI7Gw48EC73Ns5V0Rk\nE7/jj7eFhW680ZcOjFKVWO5cRDqJyA8ikisitxXzegMRGSci00Rkpoh0iWU8qSovz7obX3ZZyjav\ndK7iFi2yK5heeMEeDxwIt97qSaIcYpYoRKQq8CTQGWgG9BGRZkU2+zvwmqoeA/QGnopVPKls6FC7\niunPfw46EucSSH4+PPYYHHUUTJxY+LpvVy6xHFG0BXJVdaGqbgNGAN2LbKNA+D7dWkCRzj+uLPn5\nMGSI/cHUuHHQ0TiXIObOtaVIr78eOnSwPk39+wcdVdKK5dirHrAs4nEe0K7INncAH4rIX4C9gNOL\n25GIDAQGAjRo0KDSA01m771nU0+PPhp0JM4lkNxcu7v6xRftEkC/5G+XxLRGEYU+wDBVrQ90AV4U\nkT/EpKo5qpqlqln7779/3INMZNnZ1nHVL/92aW/qVHj2Wfv+nHOsNnHRRZ4kKkEsE8Vy4NCIx/VD\nz0UaALwGoKpfA9WBujGMKaUsWwZjx3oR26W5zZttMaF27eDf/7amfuDdJytRLBPFZCBTRBqLyB5Y\nsXpMkW2WAqcBiMiRWKJYEcOYUsrQoVafu/zyoCNxLiDjx0PLlnDffVaDmDbNm/jFQMxqFKq6Q0Su\nBT4AqgLPqupsEbkLmKKqY4AbgcEicgNW2O6v6pcmRGPHDksUXsR2aWv5cutjf+ih8PHH9r2LiZhe\nSKyqY4GxRZ77Z8T3c4ATYhlDqgoXsR97LOhInIuz776DFi2sid+oUdbEz9fCjamgi9mugnJyrIjd\ntWvQkTgXJ7/9Zh0tjz7appzA/gF4kog5TxRJKFzEHjDAi9guDajCa69ZK/ARI+Bf/7LCtYsbv4c9\nCQ0Z4kVsl0YuucTuh8jKgk8+sWknF1eeKJJMuIh91lnQqFHQ0TgXI5FN/Dp0sOmmv/7V+zMFxKee\nkszYsXaxxxVXBB2JczGycCGcfjoMG2aPBwyAm27yJBEgTxRJJicHDj4Yzj476Eicq2T5+fC//9nU\n0uTJUMVPT4nC/08kkaVL7bJYL2K7lDNnji3afsMNdrnrnDlWm3AJwcdyScTvxHYpa9EiWLAAXnkF\nevf2/kwJxhNFktixw6526tQJGjYMOhrnKsHkyTB9ui2kcvbZVpuoWTPoqFwxfOopSbz7Lvz4oy3O\n5VxS27TJitPt28M99xQ08fMkkbA8USSJnBw45BC/E9sluc8+s0tdH3rIRhLexC8peKJIAkuWFBSx\n/QpBl7Ty8uCMM+z7Tz+FZ56BWrWCjclFxRNFEhg61P47YECwcThXITNm2H/r14e33oKZM+3KJpc0\nPFEkuPCd2J07exHbJZkVK6BvX2jVCj7/3J7r0gVq1Ag2LldunigSnBexXdJRheHDrYnfyJFw551w\n3HFBR+V2QVQz3qEV6hqoam6M43FFZGdbEdvvxHZJo18/ePll6/A6dCg0bx50RG4XlTmiEJGzge+A\nj0KPW4nIqFgH5mDxYnj/fS9iuySwc2dBI79TToGHH4Yvv/QkkSKimXq6C2gHrAFQ1elARiyDcmbo\nULtB1e/EdgktN9eWIX3uOXs8YIC14qhaNdi4XKWJJlFsV9U1RZ7zda1jbPv2giJ2gwZBR+NcMXbs\ngAcftCZ+06bBHnsEHZGLkWgmNOaKSE+giog0Bq4DJsY2LPfuu/DTT17Edglq1iy49FKYMgW6d4en\nnrJimktJ0YworgXaADuBN4GtwPWxDMpZEbtePbua0LmEs3Sp3Qk6YgSMGuVJIsVFM6I4S1VvBW4N\nPyEi52FJw8XA4sXwwQfwj394EdslkG++sZvnBg60v2AWLoS99w46KhcH0Ywo/l7Mc7dXdiCuwJAh\nVsT2O7FdQti4EQYNsnsh7r8ftm615z1JpI0S/14VkbOATkA9EXk44qV9sGkoFwNexHYJ5dNPrXnf\nwoVw1VVw771QrVrQUbk4K21i41dgFrAFmB3x/HrgtlgGlc7eeQd+/tnXxHYJIC8PzjoLGje2Fhwn\nnxx0RC4gJSYKVZ0GTBORl1V1SxxjSmvZ2dY7rXPnoCNxaWvaNDjmGPtFfPtt6NAB9twz6KhcgKKp\nUdQTkREiMlNE5oW/Yh5ZGlq0CD780O/EdgH55Rfo1Qtaty5o4tepkycJF1WiGAY8BwjQGXgNeDWG\nMaUtL2K7QKjCSy9ZE7/Ro+Huu+H444OOyiWQaBJFDVX9AEBVF6jq37GE4SrR9u3w7LN21eGhhwYd\njUsrfftaI7/DD7c1rG+/HXbfPeioXAKJZoJjq4hUARaIyJXAcsAXt61kb7/tRWwXRzt32vBVBM48\n0y59veYa78/kihXNiOIGYC+sdccJwJ+By2IZVDoKF7E7dQo6Epfy5s2zDq/PPmuPL70UrrvOk4Qr\nUZmJQlW/UdX1qrpUVfupajdgcexDSx/hIvbll3sR28XQjh12w1zLlrYcqRepXZRKTRQicqyInCsi\ndUOPm4vIC8A3cYkuTQweDFWqeBHbxdDMmdC+Pdx6q117PWeO1Saci0KJiUJE7gFeBi4E3heRO4Bx\nwAzgsLhElwbCReyzz7apJ+diIi8Pli2D11+HN96Agw8OOiKXREqb6OgOtFTVzSJSG1gGtFDVhdHu\nXEQ6AY8CVYEhqnpvMdv0BO7A1riYoapp9WfOmDF2+boXsV2l++orG0lceWVBE7+99go6KpeESpt6\n2qKqmwFUdRUwr5xJoirwJHYpbTOgj4g0K7JNJvA34ARVbQ78tZzxJ73sbLsc1ovYrtJs2ADXXw8n\nnggPPVTQxM+ThKug0kYUTUQk3EpcgMYRj1HV88rYd1sgN5xcRGQENkqZE7HNn4EnVXV1aJ+/ljP+\npLZwIXz0Edx5p19w4irJhx9aG/ClS+1y1//+15v4uV1WWqLoUeTxE+Xcdz1suiosD1t7O9JhACLy\nJTY9dYeqvl90RyIyEBgI0CCFWqqGi9iX+cXGrjIsW2bFrqZNYfx4G1E4VwlKawr4SZw+PxPoCNQH\nxotIi6JrdKtqDpADkJWVlRLrdW/bZkXsrl29iO120dSp0KaNzWGOHQsnnQTVqwcdlUsh0dxwV1HL\ngchmFPVDz0XKA8ao6nZVXQTMwxJHyhszBn791YvYbhf8/DNccAFkZRU08TvjDE8SrtLFMlFMBjJF\npLGI7AH0BsYU2WY0NpogdK/GYUDUBfNklpNjCxOddVbQkbikowrPP29N/N5+2+oQ3sTPxVDUiUJE\nylURU9UdwLXAB8Bc4DVVnS0id4lIt9BmHwArRWQOdo/Gzaq6sjyfk4wWLLAi9uWXexHbVUDv3tC/\nvyWK6dPhb3/zJn4upspsGCEibYGhQC2ggYi0BC5X1b+U9V5VHQuMLfLcPyO+V2BQ6CttDB5sCcKL\n2C5qkU38unSxOsTVV9vVEM7FWDS/ZY8BXYGVAKo6AzgllkGlsm3b4LnnrIhdr17Q0bik8P33tgzp\n0KH2+JJL4NprPUm4uInmN62Kqi4p8lx+LIJJB2+9ZUXsgQODjsQlvO3brf7QsqX1Ztp776Ajcmkq\nml6ly0LTTxq62/ov2NVJrgK8iO2iMn26tf+ePh3OPx8efxwOOijoqFyaimZEcRVWQ2gA/AK0Dz3n\nyik3Fz7+GP78Zy9iuzL8/LN9vfGGNfLzJOECFM2IYoeq9o55JGlgyBAvYrtSTJhgTfyuvtqafy1Y\nADVqBB2Vc1GNKCaLyFgRuUREfAnUCgoXsc85Bw45JOhoXEJZv96K0yedBP/7X0ETP08SLkFEs8Jd\nU+BuoA3wnYiMFhEfYZTT6NFexHbF+OADOOooeOop6/j67bfexM8lnKiur1PVr1T1OqA1sA5b0MiV\nQ04ONGxo69g7B1gTv65dbeQwYYKNJvzKJpeAykwUIrK3iFwoIm8Dk4AVgPcLKIfcXPjkE78T22Ht\nNyZNsu8PPRTeew+mTfMWHC6hRTOimIVd6XS/qmao6o2q6mtml4Pfie0A+Okn6NED2rUraOJ3+une\nxM8lvGiuemqiqjtjHkmKChexu3XzInbaUoVhw2DQINiyBe67D044IeionItaiYlCRB5S1RuBN0Tk\nD2tARLHCncOK2CtWeBE7rfXsCSNH2lVNQ4bAYYcFHZFz5VLaiOLV0H/Lu7Kdi5Cd7UXstJSfbw38\nqlSxa6JPPdUWH/H+TC4Jlfhbq6qhihtHquonkV/AkfEJL7nNnw+ffmp3Yvv5IY3MnWujh3ATv4sv\nhquu8l8Cl7Si+c0trgQ7oLIDSUVexE4z27fD3XdDq1bwww9Qq1bQETlXKUqrUfTCVqVrLCJvRrxU\nE1hT/Ltc2NatBUXsgw8OOhoXc9Om2WJCM2dCr17w2GNwwAFBR+VcpSitRjEJW4OiPvBkxPPrgWmx\nDCoVjB4Nv/3ma2KnjV9+sf/ho0dD9+5BR+NcpRJbZC55ZGVl6ZQpU4IOo0ynngqLFllfN5+aTlHj\nx8N338E119jjzZthzz2Djcm5EojIVFXNqsh7SzyFicjnof+uFpFVEV+rRWRVRYNNB/PmwbhxXsRO\nWevWWYfXDh1siincxM+ThEtRpZ3Gwsud1gX2j/gKP3YlGDwYdtvNi9gpaexYaN7crnseNMib+Lm0\nUNrlseG7sQ8FqqpqPnAccAWwVxxiS0pbt9pNuN26+VozKWfZMqs/1KoFX30FDz0Ee/k/BZf6opkY\nGY0tg9oUeA7IBF6JaVRJbNQoL2KnFFWYONG+P/RQ+PBDG0W0axdsXM7FUTSJYqeqbgfOAx5X1RuA\nerENK3llZ0PjxtbrzSW5H3+Ec8+F444raOJ3yimwxx7BxuVcnEWTKHaIyAVAP+Cd0HO7xy6k5DVv\nHnz2mRexk56q9WRq1sxGEA8+6E38XFqLpnvsZcDVWJvxhSLSGBge27CSU06OFbEvvTToSNwuOf98\nePNNu6ppyBDIyAg6IucCFdV9FCKyGxD+15KrqjtiGlUpEvU+ii1boH596NjRGoW6JBPZxO/FF2HT\nJh8aupQSk/soInZ+EpALDAWeBeaJiI/Dixg1Clau9CJ2Upo1y6aWwk38+vXzTq/ORYjmX8IjQBdV\nPUFVjwfOBh6NbVjJJzsbmjSB004LOhIXtW3b4M47oXVru4V+v/2Cjsi5hBRNjWIPVZ0TfqCqc0XE\nL/uI8MMPdlHMPff4H6FJY+pUa+I3axb07Qv/+x/s7/eROlecaBLFtyLyDPBS6PGFeFPAQsJF7P79\ng47ERW3lSlizBt5+G7p2DToa5xJaNIniSuA64JbQ4y+Ax2MWUZLZsgWef94ut/c7sRPcuHHWxO+6\n62zJwfnzoXr1oKNyLuGVmihEpAXQFBilqvfHJ6Tk8uabXsROeGvXwi232NDviCPsf1a1ap4knItS\nad1j/w9r33Eh8JGIeIu7YoSL2KeeGnQkrlhvv203zg0ZAjfdZLUJb+LnXLmUNqK4EDhaVTeKyP7A\nWOzyWBfy/fe2JMG993oROyEtWwY9etgoYvRoOPbYoCNyLimVdnrbqqobAVR1RRnbpiUvYicgVevs\nCgVN/KZM8STh3C4o7eTfRETeDH2NAppGPH6zlPf9TkQ6icgPIpIrIreVsl0PEVERqdBdg0EIF7H/\n9Cc48MCgo3EA5OVZf/cTTiho4texozfxc24XlTb11KPI4yfKs2MRqYqttX0GkAdMFpExkfdkhLar\nCVwPfFOe/QftjTdg1SovYieEnTtttaibb4YdO+Dhh+HEE4OOyrmUUWKiUNVPdnHfbbG+UAsBRGQE\n0B2YU2S7fwP3ATfv4ufFVU4ONG1qXaddwHr0sBrEqadawmjSJOiInEspsaw71AOWRTzOo8g6FiLS\nGjhUVd8tbUciMlBEpojIlBUrVlR+pOU0d64VsQcO9CJ2YHbssJEEWKIYPBg+/tiThHMxENhpTkSq\nAA8DN5a1rarmqGqWqmbtnwBtFnJyYPfdvYgdmJkzbTGhwYPt8UUXweWXW/dX51ylizpRiEh5Lz5f\njq23HVY/9FxYTeAo4DMRWQy0B8YkekE7soh9wAFBR5Nmtm6Ff/0L2rSBJUu8N5NzcRJNm/G2IvId\nMD/0uKWIRNPCYzKQKSKNQ00EewNjwi+q6lpVrauqjVS1ETAR6KaqibfYRISRI2H1apt2cnE0ebJ1\neb3rLujTx+b/zjsv6KicSwvRjCgeA7oCKwFUdQZQZgk3tLjRtcAHwFzgNVWdLSJ3iUi3ioccrJwc\nW/DMi9hxtno1bNgAY8fCCy9AnTpBR+Rc2oimKWAVVV0ihed/86PZuaqOxe7ojnzunyVs2zGafQZp\nzhz44gu4/34vYsfFp59aE7/rr7cmfvPmefsN5wIQzelumYi0BVREqorIX4F5MY4rIQ0ebEXsSy4J\nOpIUt2aNLUN62mnWTGvrVnvek4RzgYgmUVwFDAIaAL9gReerYhlUItq82YrY553nReyYeusta+L3\n7LPW8dWb+DkXuDKnnlT1V++i8RsAABcQSURBVKwQnda8iB0HS5fCBRfAkUfCmDGQldAXwDmXNspM\nFCIyGNCiz6tqWp0yvYgdI6owYQKcdBI0aGA3zbVv7/2ZnEsg0Uw9fQx8Evr6EjgA2BrLoBLN7Nl2\nLhs40O/pqlRLl8LZZ8PJJxc08Tv5ZE8SziWYaKaeXo18LCIvAhNiFlECGjzYzl1+J3Yl2bkTnnkG\nbr3VRhSPPeZN/JxLYNFcHltUYyBtGmtHFrH9RuBKct55VrQ+4wyb02vUKOiInHOliKZGsZqCGkUV\nYBVQ4toSqWbkSLta04vYu2jHDrv5pEoV6NULune3IZrP5TmX8EpNFGJ32bWkoEfTTlX9Q2E7lWVn\nQ2amrX/jKmjGDLjsMrs34sorrQWHcy5plFrMDiWFsaqaH/pKqyQxezZ8+aUXsStsyxb4+9/tMte8\nPDjooKAjcs5VQDRXPU0XkWNiHkkCysnxInaFTZoExxwD//kPXHihNfE799ygo3LOVUCJU08isluo\nsd8x2DKmC4CNgGCDjdZxijEQmzdb77nzzoO6dYOOJgmtW2cH8f334ayzgo7GObcLSqtRTAJaA0nb\n6XVXvP66FbF9Texy+PBDm6+74QY4/XT44Qdvv+FcCigtUQiAqi6IUywJJTsbDjsMOnQIOpIksHo1\nDBoEw4ZB8+Zw9dWWIDxJOJcSSksU+4vIoJJeVNWHYxBPQpg1C776Ch580IvYZXrzTbjmGlixAv72\nN/jnPz1BOJdiSksUVYG9CY0s0km4iO3txMuwdCn07g1HHWULCh2Tltc8OJfySksUP6nqXXGLJEFs\n2mRF7B49vIhdLFUYP97m5Bo0sMWF2rWzhTqccymptMtj024kAVbEXrvWi9jFWrIEOne2uw/DTfxO\nPNGThHMprrREcVrcokgg2dlw+OHWxNSF7NwJTzxhheoJE+Dxx60tuHMuLZQ49aSqq+IZSCL47jv4\n+mt46CEvYhdy7rnw9tt2P0R2NjRsGHREzrk4qkj32JTlRewI27dD1arWxK9PHzj/fOjXzzOoc2ko\nmhYeaWHTJnjxRTsf1qkTdDQB+/ZbaNvW1owASxQXX+xJwrk05Yki5LXXvIjN5s12L0TbtvDzz3Do\noUFH5JxLAD71FJKdDUcckcY12okTbc5t3jxrCf7gg7DffkFH5ZxLAJ4ogJkz7Tz58MNpPLuycaPV\nJT76yPo0OedciCcKrIhdrZpNw6eV99+3Jn433ginnQbff2/VfOeci5D2NYq0LGKvXGnTTJ0724Lg\n27bZ854knHPFSPtE8eqrtnRCWhSxVW0R8GbN4JVXbPW5yZM9QTjnSpX2U0/Z2XDkkdaJIuUtXQp9\n+8LRR9vaES1bBh2Rcy4JpPWIYsYM+OabFF8TW9Ua94HdUf3ZZ1a59yThnItSWieKlC9iL1oEZ55p\nhepwE7/jj4fd0n4g6Zwrh7RNFBs3wksvwQUXQO3aQUdTyfLz4dFHbZ2Ib76Bp59O4xtEnHO7Km3/\ntAwXsQcODDqSGOjeHd59F7p0sTYcfoe1c24XpG2iyMlJsSJ2ZBO/fv2sP1PfvilcfHHOxUtMp55E\npJOI/CAiuSJyWzGvDxKROSIyU0Q+EZG49K8OF7GvuCJFzqNTpkBWlk0xAfTqBRdemCI/nHMuaDFL\nFCJSFXgS6Aw0A/qISLMim00DslT1aGAkcH+s4omUnW1F7H794vFpMbR5M9x6qy1FumKFrxPhnIuJ\nWI4o2gK5qrpQVbcBI4DukRuo6jhV3RR6OBGoH8N4gIIids+eSV7E/vpru8T1/vutid+cOdC1a9BR\nOedSUCxrFPWAZRGP84B2pWw/AHivuBdEZCAwEKBBgwa7FNSIEbB+fQoUsTdvtiVKP/7YLn91zrkY\nSYhitohcBGQBHYp7XVVzgByArKws3ZXPysmxDhYnnLArewnI2LHWxO/mm+HUU2HuXNh996Cjcs6l\nuFhOPS0HIq/LrB96rhAROR24HeimqltjGA/Tp8OkSUl4J/Zvv8FFF8HZZ8PLLxc08fMk4ZyLg1gm\nislApog0FpE9gN7AmMgNROQYIBtLEr/GMBbARhPVqyfRndiqNld25JG2BN+//mWZzpv4OefiKGZT\nT6q6Q0SuBT4AqgLPqupsEbkLmKKqY4AHgL2B18X+xF+qqt1iEc+GDQVF7KRZuG3pUmsH3rIlDB0K\nLVoEHZFzLg3FtEahqmOBsUWe+2fE93FbSi1pitiq8Mkntspcw4bWo+nYY+1mOuecC0Da9HrKyYHm\nza0nXsJasMCuYDrjjIImfu3be5JwzgUqLRLFtGm2Pk/CFrHz823B7hYtYOpUuyPQm/g55xJEQlwe\nG2vhInbC3ol9zjnw3nt2w9zTT0P9mN936JxzUUv5RLFhg11RmnBF7G3bbF2IKlWgf3/LYr17J+iQ\nxzmXzlJ+6ilcxE6oNbEnTYI2beCpp+xxz57W7dWThHMuAaV8osjOtvV7jjsu6EiATZvgxhstmNWr\noWnToCNyzrkypfTU07ffWgfuxx5LgD/WJ0yweyIWLrThzX33Qa1aAQflnHNlS+lEkVBF7PDCQuPG\nQceOQUfjnHNRS9lEsX69FbF79YJ99w0oiLfftsZ9t9wCp5xircB3S9lD7pxLUSlboxgxwq54CqSI\nvWKFLUParRsMH17QxM+ThHMuCaVsoggXsdu3j+OHqsIrr1gTv5Ej4a67bM1Vb+LnnEtiKfkn7tSp\n9vX443EuYi9dCpdeCsccY038mjeP44c751xspOSIIicH9tzTlnCIuZ074YMP7PuGDeGLL+DLLz1J\nOOdSRsolivXrbfYnLkXs+fNtpblOnWD8eHuubVtv4uecSykplyiGD49DEXvHDnjgATj6aFs2b+hQ\nb+LnnEtZKVejyM62Jqzt2sXwQ7p2temm7t2tDcchh8Tww5xLXtu3bycvL48tW7YEHUraqF69OvXr\n12f3SlwqOaUSxdSpdjf2E0/EoIi9dautUV2lClx+OVx2GVxwQQLc8u1c4srLy6NmzZo0atQI8X8r\nMaeqrFy5kry8PBo3blxp+02pqafsbCtiX3hhJe944kRo3RqefNIen3++NfLzX3znSrVlyxbq1Knj\nSSJORIQ6depU+gguZRLFunVWxO7duxKL2Bs3wg032LJ469dDZmYl7di59OFJIr5icbxTZupp+HA7\nr1daEfuLL6yJ36JFcPXVcM89sM8+lbRz55xLHikxolC1aaejj7arUyvFjh1Wk/j8c5ty8iThXNIa\nPXo0IsL333//+3OfffYZXbt2LbRd//79GTlyJGCF+Ntuu43MzExat27Ncccdx3vvvbfLsdxzzz1k\nZGRw+OGH80H4HqwiPvnkE1q3bk2rVq048cQTyc3NBWD8+PG0bt2a3Xbb7fc44yElEsXUqbYu9hVX\n7GLZYPRoGzmANfGbPRtOPrlSYnTOBWf48OGceOKJDB8+POr3/OMf/+Cnn35i1qxZfPvtt4wePZr1\n69fvUhxz5sxhxIgRzJ49m/fff5+rr76a/Pz8P2x31VVX8fLLLzN9+nT69u3L3XffDUCDBg0YNmwY\nffv23aU4yislpp6ys6FGjV0oYv/yC/zlL/D661a0vvFG68/kTfycqzR//avddlSZWrWC//2v9G02\nbNjAhAkTGDduHOeccw533nlnmfvdtGkTgwcPZtGiRVSrVg2AAw88kJ49e+5SvG+99Ra9e/emWrVq\nNG7cmIyMDCZNmsRxRVZWExHWrVsHwNq1azkkdAl+o0aNAKhSJb5/4yf9mXDdOqtP9O5dgXWAVOGl\nl+w3eMMG+M9/4OabbcrJOZcS3nrrLTp16sRhhx1GnTp1mDp1Km3atCn1Pbm5uTRo0IB9ophyvuGG\nGxg3btwfnu/duze33XZboeeWL19O+4hOpfXr12f58uV/eO+QIUPo0qULe+65J/vssw8TJ04sM45Y\nSvpE8corVsQeOLACb1661O6JyMqyu6uPOKLS43POmbL+8o+V4cOHc/311wN28h4+fDht2rQp8eqg\n8l419Mgjj+xyjMXtc+zYsbRr144HHniAQYMGMWTIkEr/nGgldaIIF7FbtixHETvcxK9zZ2vi9+WX\n1u3V+zM5l3JWrVrFp59+ynfffYeIkJ+fj4jwwAMPUKdOHVavXv2H7evWrUtGRgZLly5l3bp1ZY4q\nyjOiqFevHsuWLfv9cV5eHvXq1Su0zYoVK5gxYwbtQu0levXqRadOncr1c1e2pC5mT5lic55RF7Hn\nzbNlSLt0sauZwEYTniScS0kjR46kX79+LFmyhMWLF7Ns2TIaN27MF198QWZmJj/++CNz584FYMmS\nJcyYMYNWrVpRo0YNBgwYwPXXX8+20MJjK1as4PXXX//DZzzyyCNMnz79D19FkwRAt27dGDFiBFu3\nbmXRokXMnz+ftkX+yt1vv/1Yu3Yt8+bNA+Cjjz7iyCOPrOxDUz6qmlRfbdq00bABA1Rr1FBds0ZL\nt3276r33qlarprrvvqrPPae6c2cZb3LO7ao5c+YE+vkdO3bU9957r9Bzjz76qF555ZWqqjphwgRt\n166dtmzZUrOysvTDDz/8fbutW7fqzTffrE2bNtXmzZtr27Zt9f3339/lmO6++25t0qSJHnbYYTp2\n7Njfn+/cubMuX75cVVXffPNNPeqoo/Too4/WDh066IIFC1RVddKkSVqvXj2tUaOG1q5dW5s1a1bs\nZxR33IEpWsHzrtj7k0dWVpZOmTKFdevg4IOhTx8oc+rurLPgww/hvPPsnoiDDopLrM6lu7lz5wb/\n13AaKu64i8hUVc2qyP6Stkbx8suwaVMpRewtW+zqpapVbaOBA6FHj7jG6JxzqSApaxThInarVnDs\nscVs8OWX9mK4iV+PHp4knHOugpIyUUyeDDNm2CChUBF7wwa47jpbRGjLFvAhr3OBS7bp7WQXi+Od\nlIkiOxv22qvIndiffw5HHWWLUVx7LcyaBWecEViMzjlbRGflypWeLOJEQ+tRVK9evVL3m3Q1ivx8\nGDEC+vYtpk9fjRrW9fWEEwKJzTlXWP369cnLy2PFihVBh5I2wivcVaaku+qpYcMsXbp0CpMmwbHL\n3oTvv4f/+z97MT/f74lwzrli7MpVTzGdehKRTiLyg4jkisgf7j4RkWoi8mro9W9EpFFZ+1yxAk4/\n6mey7jvfCtSjRkHohhhPEs45V/lilihEpCrwJNAZaAb0EZFmRTYbAKxW1QzgEeC+svZbY/NK3llw\nJPLOO9YS/KuvrNOrc865mIjliKItkKuqC1V1GzAC6F5km+7A86HvRwKnSRkduRqyhKqtjrLLnm67\nzTu9OudcjMWymF0PWBbxOA9oV9I2qrpDRNYCdYDfIjcSkYFA+Na6rbt/PWGWd3oFoC5FjlUa82NR\nwI9FAT8WBQ6v6BuT4qonVc0BcgBEZEpFCzKpxo9FAT8WBfxYFPBjUUBEplT0vbGceloOHBrxuH7o\nuWK3EZHdgFrAyhjG5JxzrpximSgmA5ki0lhE9gB6A2OKbDMGuCT0/fnAp5ps1+s651yKi9nUU6jm\ncC3wAVAVeFZVZ4vIXVi72zHAUOBFEckFVmHJpCw5sYo5CfmxKODHooAfiwJ+LApU+Fgk3Q13zjnn\n4ispez0555yLH08UzjnnSpWwiSIW7T+SVRTHYpCIzBGRmSLyiYg0DCLOeCjrWERs10NEVERS9tLI\naI6FiPQM/W7MFpFX4h1jvETxb6SBiIwTkWmhfyddgogz1kTkWRH5VURmlfC6iMhjoeM0U0RaR7Xj\niq6hGssvrPi9AGgC7AHMAJoV2eZq4JnQ972BV4OOO8BjcQpQI/T9Vel8LELb1QTGAxOBrKDjDvD3\nIhOYBuwXenxA0HEHeCxygKtC3zcDFgcdd4yOxclAa2BWCa93Ad4DBGgPfBPNfhN1RBGT9h9Jqsxj\noarjVHVT6OFE7J6VVBTN7wXAv7G+YVviGVycRXMs/gw8qaqrAVT11zjHGC/RHAsFwgsT1AJ+jGN8\ncaOq47ErSEvSHXhBzURgXxE5uKz9JmqiKK79R72StlHVHUC4/UeqieZYRBqA/cWQiso8FqGh9KGq\n+m48AwtANL8XhwGHiciXIjJRRDrFLbr4iuZY3AFcJCJ5wFjgL/EJLeGU93wCJEkLDxcdEbkIyAI6\nBB1LEESkCvAw0D/gUBLFbtj0U0dslDleRFqo6ppAowpGH2CYqj4kIsdh928dpao7gw4sGSTqiMLb\nfxSI5lggIqcDtwPdVHVrnGKLt7KORU3gKOAzEVmMzcGOSdGCdjS/F3nAGFXdrqqLgHlY4kg10RyL\nAcBrAKr6NVAdaxiYbqI6nxSVqInC238UKPNYiMgxQDaWJFJ1HhrKOBaqulZV66pqI1VthNVruqlq\nhZuhJbBo/o2MxkYTiEhdbCpqYTyDjJNojsVS4DQAETkSSxTpuD7rGODi0NVP7YG1qvpTWW9KyKkn\njV37j6QT5bF4ANgbeD1Uz1+qqt0CCzpGojwWaSHKY/EBcKaIzAHygZtVNeVG3VEeixuBwSJyA1bY\n7p+Kf1iKyHDsj4O6oXrMv4DdAVT1Gaw+0wXIBTYBl0a13xQ8Vs455ypRok49OeecSxCeKJxzzpXK\nE4VzzrlSeaJwzjlXKk8UzjnnSuWJwiUcEckXkekRX41K2bZRSZ0yy/mZn4W6j84Itbw4vAL7uFJE\nLg59319EDol4bYiINKvkOCeLSKso3vNXEamxq5/t0pcnCpeINqtqq4ivxXH63AtVtSXWbPKB8r5Z\nVZ9R1RdCD/sDh0S8drmqzqmUKAvifIro4vwr4InCVZgnCpcUQiOHL0Tk29DX8cVs01xEJoVGITNF\nJDP0/EURz2eLSNUyPm48kBF672mhNQy+C/X6rxZ6/l4pWAPkwdBzd4jITSJyPtZz6+XQZ+4ZGglk\nhUYdv5/cQyOPJyoY59dENHQTkadFZIrY2hN3hp67DktY40RkXOi5M0Xk69BxfF1E9i7jc1ya80Th\nEtGeEdNOo0LP/QqcoaqtgV7AY8W870rgUVVthZ2o80LtGnoBJ4SezwcuLOPzzwG+E5HqwDCgl6q2\nwDoZXCUidYA/Ac1V9Wjg7sg3q+pIYAr2l38rVd0c8fIbofeG9QJGVDDOTlibjrDbVTULOBroICJH\nq+pjWEvtU1T1lFArj78Dp4eO5RRgUBmf49JcQrbwcGlvc+hkGWl34InQnHw+1reoqK+B20WkPvCm\nqs4XkdOANsDkUHuTPbGkU5yXRWQzsBhrQ304sEhV54Vefx64BngCW+tiqIi8A7wT7Q+mqitEZGGo\nz8584Ajgy9B+yxPnHljblsjj1FNEBmL/rg/GFuiZWeS97UPPfxn6nD2w4+ZciTxRuGRxA/AL0BIb\nCf9hUSJVfUVEvgHOBsaKyBXYSl7Pq+rfoviMCyMbCIpI7eI2CvUWaos1mTsfuBY4tRw/ywigJ/A9\nMEpVVeysHXWcwFSsPvE4cJ6INAZuAo5V1dUiMgxrfFeUAB+pap9yxOvSnE89uWRRC/gptH5AP6z5\nWyEi0gRYGJpueQubgvkEOF9EDghtU1uiX1P8B6CRiGSEHvcDPg/N6ddS1bFYAmtZzHvXY23PizMK\nW2msD5Y0KG+coYZ2/wDai8gR2OptG4G1InIg0LmEWCYCJ4R/JhHZS0SKG5059ztPFC5ZPAVcIiIz\nsOmajcVs0xOYJSLTsXUpXghdafR34EMRmQl8hE3LlElVt2DdNV8Xke+AncAz2En3ndD+JlD8HP8w\n4JlwMbvIflcDc4GGqjop9Fy54wzVPh7CusLOwNbH/h54BZvOCssB3heRcaq6Arsia3joc77Gjqdz\nJfLusc4550rlIwrnnHOl8kThnHOuVJ4onHPOlcoThXPOuVJ5onDOOVcqTxTOOedK5YnCOedcqf4f\nb/1WdA98EiEAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "\n",
            "\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Gyd1nxLUPhPl",
        "colab_type": "text"
      },
      "source": [
        "# Useful tool"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-yRU6OGZl2bs",
        "colab_type": "code",
        "outputId": "68119063-6997-409c-8136-d7a367b52f9e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 68
        }
      },
      "source": [
        "from sklearn.feature_extraction.text import TfidfVectorizer\n",
        "from sklearn.ensemble import RandomForestClassifier\n",
        "from sklearn.pipeline import Pipeline\n",
        " \n",
        " \n",
        "texts = ['текст номер один', 'текст номер два', 'комьютеры в лингвистике', 'компьютеры и обработка текстов']\n",
        "texts_labels = [1, 1, 0, 0]\n",
        " \n",
        "text_clf = Pipeline([\n",
        "                     ('tfidf', TfidfVectorizer()),\n",
        "                     ('clf', RandomForestClassifier())\n",
        "                     ])\n",
        " \n",
        "text_clf.fit(texts, texts_labels)\n",
        " \n",
        "res = text_clf.predict(['текст номер три'])\n",
        "print(res)  # [1]"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[1]\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/ensemble/forest.py:245: FutureWarning: The default value of n_estimators will change from 10 in version 0.20 to 100 in 0.22.\n",
            "  \"10 in version 0.20 to 100 in 0.22.\", FutureWarning)\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "N2fmGRHX2GSf",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "report = classification_report(y_test, lr.predict(X_test), target_names=['Non-churned', 'Churned'])\n",
        "print(report)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XxWNYrzH9tIB",
        "colab_type": "text"
      },
      "source": [
        "# Useful liks"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "65hgV4vB9vvV",
        "colab_type": "text"
      },
      "source": [
        "* http://zabaykin.ru/?p=558\n",
        "\n",
        "\n",
        "```\n",
        "Список улучшений, которые следует попробовать.\n",
        "1. предобработка текста. Попробуйте нормализовать слова, тогда одно смысловое слово в разных склонениях/спряжениях будет интерпретироваться программой одинаково и возможно поднимет качество.\n",
        "2. У TfIdf есть много разных параметров, наиболее существенные это\n",
        "2.1 добавить список стоп слов: параметр stop_words\n",
        "2.2 добавить n-gramm ы слов : параметр ngram_range, например ngram_range=(1,2)\n",
        "2.3 ограничить список фич, взяв только самые важные и отрезав менее важные: параметр max_features\n",
        "3. попробовать другой классификатор, например from sklearn.linear_model import SGDClassifier или SVM , XGB итд. соответсвенно подбирая в каждом из них свои гиперпараметры\n",
        "```\n",
        "\n",
        "* http://nlpx.net/archives/29\n",
        "* http://nlpx.net/archives/57\n",
        "* https://habr.com/ru/company/ods/blog/328372/\n",
        "\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "* https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfVectorizer.html\n",
        "* https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.CountVectorizer.html\n"
      ]
    }
  ]
}