Grammar Correction.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "0e1360b8",
   "metadata": {},
   "source": [
    "# Mini Project 2A"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4989994e",
   "metadata": {},
   "source": [
    "<b>Project Title: </b>Text Utilities <br>\n",
    "<b>Domain: </b>Natural Language Processing"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dbddb9d2",
   "metadata": {},
   "source": [
    "<b>Team Members:</b>\n",
    "1. Mitali Rawat (Roll No. 21)\n",
    "2. Radha Vishwakarma (Roll No. 46)\n",
    "3. Om Achrekar (Roll No. 54)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c1b6192e",
   "metadata": {},
   "source": [
    "<b>Problem Statement</b>: To create a web application that will help user to summerize, paraphrase the given text, and will also be able to correct the grammatical mistakes."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0a80e1b1",
   "metadata": {},
   "source": [
    "## Grammar correction using T5 transformer:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b723d89",
   "metadata": {},
   "source": [
    "#### Done By: Om Achrekar"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "69781108",
   "metadata": {},
   "source": [
    "Hugging Face Hub Link: https://huggingface.co/achrekarom/grammar_correction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "f629dc7a",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\n",
      "To disable this warning, you can either:\n",
      "\t- Avoid using `tokenizers` before the fork if possible\n",
      "\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\n",
      "Requirement already satisfied: happytransformer in /opt/anaconda3/envs/project/lib/python3.9/site-packages (2.4.1)\n",
      "Requirement already satisfied: torch>=1.0 in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from happytransformer) (1.13.0.dev20221005)\n",
      "Requirement already satisfied: sentencepiece in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from happytransformer) (0.1.97)\n",
      "Requirement already satisfied: transformers>=4.4.0 in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from happytransformer) (4.22.2)\n",
      "Requirement already satisfied: tqdm>=4.43 in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from happytransformer) (4.64.0)\n",
      "Requirement already satisfied: datasets>=1.6.0 in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from happytransformer) (2.5.1)\n",
      "Requirement already satisfied: protobuf in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from happytransformer) (3.19.1)\n",
      "Requirement already satisfied: dill<0.3.6 in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from datasets>=1.6.0->happytransformer) (0.3.5.1)\n",
      "Requirement already satisfied: multiprocess in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from datasets>=1.6.0->happytransformer) (0.70.13)\n",
      "Requirement already satisfied: packaging in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from datasets>=1.6.0->happytransformer) (21.3)\n",
      "Requirement already satisfied: xxhash in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from datasets>=1.6.0->happytransformer) (3.0.0)\n",
      "Requirement already satisfied: fsspec[http]>=2021.11.1 in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from datasets>=1.6.0->happytransformer) (2022.2.0)\n",
      "Requirement already satisfied: responses<0.19 in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from datasets>=1.6.0->happytransformer) (0.18.0)\n",
      "Requirement already satisfied: pyarrow>=6.0.0 in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from datasets>=1.6.0->happytransformer) (9.0.0)\n",
      "Requirement already satisfied: pandas in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from datasets>=1.6.0->happytransformer) (1.4.2)\n",
      "Requirement already satisfied: requests>=2.19.0 in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from datasets>=1.6.0->happytransformer) (2.27.1)\n",
      "Requirement already satisfied: aiohttp in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from datasets>=1.6.0->happytransformer) (3.8.1)\n",
      "Requirement already satisfied: huggingface-hub<1.0.0,>=0.1.0 in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from datasets>=1.6.0->happytransformer) (0.10.0)\n",
      "Requirement already satisfied: numpy>=1.17 in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from datasets>=1.6.0->happytransformer) (1.21.5)\n",
      "Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from huggingface-hub<1.0.0,>=0.1.0->datasets>=1.6.0->happytransformer) (4.1.1)\n",
      "Requirement already satisfied: pyyaml>=5.1 in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from huggingface-hub<1.0.0,>=0.1.0->datasets>=1.6.0->happytransformer) (6.0)\n",
      "Requirement already satisfied: filelock in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from huggingface-hub<1.0.0,>=0.1.0->datasets>=1.6.0->happytransformer) (3.6.0)\n",
      "Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from packaging->datasets>=1.6.0->happytransformer) (3.0.4)\n",
      "Requirement already satisfied: charset-normalizer~=2.0.0 in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from requests>=2.19.0->datasets>=1.6.0->happytransformer) (2.0.4)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from requests>=2.19.0->datasets>=1.6.0->happytransformer) (3.3)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from requests>=2.19.0->datasets>=1.6.0->happytransformer) (2021.10.8)\n",
      "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from requests>=2.19.0->datasets>=1.6.0->happytransformer) (1.26.9)\n",
      "Requirement already satisfied: regex!=2019.12.17 in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from transformers>=4.4.0->happytransformer) (2022.3.15)\n",
      "Requirement already satisfied: tokenizers!=0.11.3,<0.13,>=0.11.1 in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from transformers>=4.4.0->happytransformer) (0.12.1)\n",
      "Requirement already satisfied: aiosignal>=1.1.2 in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from aiohttp->datasets>=1.6.0->happytransformer) (1.2.0)\n",
      "Requirement already satisfied: multidict<7.0,>=4.5 in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from aiohttp->datasets>=1.6.0->happytransformer) (5.2.0)\n",
      "Requirement already satisfied: frozenlist>=1.1.1 in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from aiohttp->datasets>=1.6.0->happytransformer) (1.2.0)\n",
      "Requirement already satisfied: attrs>=17.3.0 in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from aiohttp->datasets>=1.6.0->happytransformer) (21.4.0)\n",
      "Requirement already satisfied: yarl<2.0,>=1.0 in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from aiohttp->datasets>=1.6.0->happytransformer) (1.6.3)\n",
      "Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from aiohttp->datasets>=1.6.0->happytransformer) (4.0.1)\n",
      "Requirement already satisfied: python-dateutil>=2.8.1 in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from pandas->datasets>=1.6.0->happytransformer) (2.8.2)\n",
      "Requirement already satisfied: pytz>=2020.1 in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from pandas->datasets>=1.6.0->happytransformer) (2021.3)\n",
      "Requirement already satisfied: six>=1.5 in /opt/anaconda3/envs/project/lib/python3.9/site-packages (from python-dateutil>=2.8.1->pandas->datasets>=1.6.0->happytransformer) (1.16.0)\n",
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "pip install happytransformer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "097e6db7",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "10/08/2022 15:00:28 - INFO - happytransformer.happy_transformer -   Using model: cpu\n"
     ]
    }
   ],
   "source": [
    "from happytransformer import HappyTextToText\n",
    "\n",
    "happy_tt = HappyTextToText(\"T5\", \"t5-base\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "da6f352b",
   "metadata": {},
   "outputs": [],
   "source": [
    "from datasets import load_dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "2cb48b38",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/anaconda3/envs/project/lib/python3.9/site-packages/huggingface_hub/utils/_deprecation.py:97: FutureWarning: Deprecated argument(s) used in 'dataset_info': token. Will not be supported from version '0.12'.\n",
      "  warnings.warn(message, FutureWarning)\n"
     ]
    }
   ],
   "source": [
    "train_dataset = load_dataset(\"jfleg\", split='validation[:]')\n",
    "\n",
    "eval_dataset = load_dataset(\"jfleg\", split='test[:]')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "faa912c4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['So I think we would not be alive if our ancestors did not develop sciences and technologies . ', 'So I think we could not live if older people did not develop science and technologies . ', 'So I think we can not live if old people could not find science and technologies and they did not develop . ', 'So I think we can not live if old people can not find the science and technology that has not been developed . ']\n",
      "So I think we would not be alive if our ancestors did not develop sciences and technologies . \n",
      "['Not for use with a car . ', 'Do not use in the car . ', 'Car not for use . ', 'Can not use the car . ']\n",
      "Not for use with a car . \n"
     ]
    }
   ],
   "source": [
    "for case in train_dataset[\"corrections\"][:2]:\n",
    "  print(case)\n",
    "  print(case[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "24a5961f",
   "metadata": {},
   "outputs": [],
   "source": [
    "import csv\n",
    "\n",
    "def generate_csv(csv_path, dataset):\n",
    "    with open(csv_path, 'w', newline='') as csvfile:\n",
    "        writter = csv.writer(csvfile)\n",
    "        writter.writerow([\"input\", \"target\"])\n",
    "        for case in dataset:\n",
    "     \t    # Adding the task's prefix to input \n",
    "            input_text = \"grammar: \" + case[\"sentence\"]\n",
    "            for correction in case[\"corrections\"]:\n",
    "                # a few of the cases contain blank strings. \n",
    "                if input_text and correction:\n",
    "                    writter.writerow([input_text, correction])\n",
    "                    \n",
    "\n",
    "\n",
    "generate_csv(\"train.csv\", train_dataset)\n",
    "generate_csv(\"eval.csv\", eval_dataset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "e0239063",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "10/08/2022 15:51:01 - INFO - happytransformer.happy_transformer -   Preprocessing evaluating data...\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\n",
      "To disable this warning, you can either:\n",
      "\t- Avoid using `tokenizers` before the fork if possible\n",
      "\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\n",
      "Downloading and preparing dataset csv/default to /Users/omachrekar/.cache/huggingface/datasets/csv/default-ef6e178e80894712/0.0.0/652c3096f041ee27b04d2232d41f10547a8fecda3e284a79a0ec4053c916ef7a...\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "16d84e341dfe4202b9b9914e76e68721",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading data files:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "2ba998f4d23f4255ada8d2b2e4105c7b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Extracting data files:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "0 tables [00:00, ? tables/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dataset csv downloaded and prepared to /Users/omachrekar/.cache/huggingface/datasets/csv/default-ef6e178e80894712/0.0.0/652c3096f041ee27b04d2232d41f10547a8fecda3e284a79a0ec4053c916ef7a. Subsequent calls will reuse this data.\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "4124f17dda7e42dea32f061ac5792572",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1cc3c63b26bd4dc5b8e70cba39be0f7d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/3 [00:00<?, ?ba/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/anaconda3/envs/project/lib/python3.9/site-packages/transformers/tokenization_utils_base.py:3542: UserWarning: `as_target_tokenizer` is deprecated and will be removed in v5 of Transformers. You can tokenize your labels by using the argument `text_target` of the regular `__call__` method (either in the same call as your input texts if you use the same keyword arguments, or in a separate call.\n",
      "  warnings.warn(\n",
      "***** Running Evaluation *****\n",
      "  Num examples = 2988\n",
      "  Batch size = 1\n",
      "You're using a T5TokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='2988' max='2988' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [2988/2988 07:51]\n",
       "    </div>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    " before_result = happy_tt.eval(\"eval.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "c72793cd",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Before loss: 1.2803850173950195\n"
     ]
    }
   ],
   "source": [
    "print(\"Before loss:\", before_result.loss)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "60e6a6bc",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "10/08/2022 16:16:40 - INFO - happytransformer.happy_transformer -   Preprocessing training data...\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading and preparing dataset csv/default to /Users/omachrekar/.cache/huggingface/datasets/csv/default-1bee5995f6642db1/0.0.0/652c3096f041ee27b04d2232d41f10547a8fecda3e284a79a0ec4053c916ef7a...\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "c5f0bc912eb54883bf138864dbad0caa",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading data files:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "482cc3362bdd4728afedf05808ebd0e9",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Extracting data files:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "0 tables [00:00, ? tables/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dataset csv downloaded and prepared to /Users/omachrekar/.cache/huggingface/datasets/csv/default-1bee5995f6642db1/0.0.0/652c3096f041ee27b04d2232d41f10547a8fecda3e284a79a0ec4053c916ef7a. Subsequent calls will reuse this data.\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "8525f0e7e6394c0eb22916ddfdade69d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "088d425a607e4c90af68787633c781f2",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/4 [00:00<?, ?ba/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/anaconda3/envs/project/lib/python3.9/site-packages/transformers/tokenization_utils_base.py:3542: UserWarning: `as_target_tokenizer` is deprecated and will be removed in v5 of Transformers. You can tokenize your labels by using the argument `text_target` of the regular `__call__` method (either in the same call as your input texts if you use the same keyword arguments, or in a separate call.\n",
      "  warnings.warn(\n",
      "10/08/2022 16:16:44 - INFO - happytransformer.happy_transformer -   Training...\n",
      "PyTorch: setting up devices\n",
      "/opt/anaconda3/envs/project/lib/python3.9/site-packages/transformers/optimization.py:306: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning\n",
      "  warnings.warn(\n",
      "***** Running training *****\n",
      "  Num examples = 3016\n",
      "  Num Epochs = 3\n",
      "  Instantaneous batch size per device = 8\n",
      "  Total train batch size (w. parallel, distributed & accumulation) = 8\n",
      "  Gradient Accumulation steps = 1\n",
      "  Total optimization steps = 1131\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='1131' max='1131' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [1131/1131 3:20:57, Epoch 3/3]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       " <tr style=\"text-align: left;\">\n",
       "      <th>Step</th>\n",
       "      <th>Training Loss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>500</td>\n",
       "      <td>0.578400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1000</td>\n",
       "      <td>0.443900</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "Training completed. Do not forget to share your model on huggingface.co/models =)\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from happytransformer import TTTrainArgs\n",
    "\n",
    "args = TTTrainArgs(batch_size=8)\n",
    "happy_tt.train(\"train.csv\", args=args)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "e8ffa4d0",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "10/08/2022 20:20:58 - INFO - happytransformer.happy_transformer -   Preprocessing evaluating data...\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f6d6a2cd454b45c79c69a4af91c61333",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "c84ed67f215e4452822f58fa96669d9d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/3 [00:00<?, ?ba/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "PyTorch: setting up devices\n",
      "***** Running Evaluation *****\n",
      "  Num examples = 2988\n",
      "  Batch size = 1\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='2988' max='2988' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [2988/2988 07:52]\n",
       "    </div>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "After loss:  0.4503001570701599\n"
     ]
    }
   ],
   "source": [
    "before_loss = happy_tt.eval(\"eval.csv\")\n",
    "\n",
    "print(\"After loss: \", before_loss.loss)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "599d9284",
   "metadata": {},
   "outputs": [],
   "source": [
    "from happytransformer import TTSettings\n",
    "\n",
    "beam_settings =  TTSettings(num_beams=5, min_length=1, max_length=20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "b510e52b",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Enter a sentence: I has one book\n",
      "I have one book .\n"
     ]
    }
   ],
   "source": [
    "user_ip = input(\"Enter a sentence: \")\n",
    "pass_this = \"grammar:\" + user_ip\n",
    "result = happy_tt.generate_text(pass_this, args=beam_settings)\n",
    "print(result.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "d85a1aab",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Enter a sentence: He are very bright student\n",
      "He are a very bright student .\n"
     ]
    }
   ],
   "source": [
    "user_ip = input(\"Enter a sentence: \")\n",
    "pass_this = \"grammar:\" + user_ip\n",
    "result = happy_tt.generate_text(pass_this, args=beam_settings)\n",
    "print(result.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "2f845e02",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Enter a sentence: Radha have studied for exams\n",
      "Rahha has studied for exams .\n"
     ]
    }
   ],
   "source": [
    "user_ip = input(\"Enter a sentence: \")\n",
    "pass_this = \"grammar:\" + user_ip\n",
    "result = happy_tt.generate_text(pass_this, args=beam_settings)\n",
    "print(result.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "1d9c846d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Enter a sentence: Do not throow garbage on rod.\n",
      "Do not throw garbage on a rod.\n"
     ]
    }
   ],
   "source": [
    "user_ip = input(\"Enter a sentence: \")\n",
    "pass_this = \"grammar:\" + user_ip\n",
    "result = happy_tt.generate_text(pass_this, args=beam_settings)\n",
    "print(result.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "d38a3a37",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Enter a sentence: He is a nice purson.\n",
      "He is a nice person.\n"
     ]
    }
   ],
   "source": [
    "user_ip = input(\"Enter a sentence: \")\n",
    "pass_this = \"grammar:\" + user_ip\n",
    "result = happy_tt.generate_text(pass_this, args=beam_settings)\n",
    "print(result.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "9a95744f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Enter a sentence: Understandng\n",
      "Understanding and understanding a variety of topics.\n"
     ]
    }
   ],
   "source": [
    "user_ip = input(\"Enter a sentence: \")\n",
    "pass_this = \"grammar:\" + user_ip\n",
    "result = happy_tt.generate_text(pass_this, args=beam_settings)\n",
    "print(result.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "fae1b674",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Enter a sentence: Grammer\n",
      "Grammer:Grammer:Grammer:Grammer:Grammer:\n"
     ]
    }
   ],
   "source": [
    "user_ip = input(\"Enter a sentence: \")\n",
    "pass_this = \"grammar:\" + user_ip\n",
    "result = happy_tt.generate_text(pass_this, args=beam_settings)\n",
    "print(result.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "51599d7d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Enter a sentence: She has do all her work\n",
      "She has done all her work.\n"
     ]
    }
   ],
   "source": [
    "user_ip = input(\"Enter a sentence: \")\n",
    "pass_this = \"grammar:\" + user_ip\n",
    "result = happy_tt.generate_text(pass_this, args=beam_settings)\n",
    "print(result.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "26c13186",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Enter a sentence: Cat has four paws\n",
      "Cat has four paws .\n"
     ]
    }
   ],
   "source": [
    "user_ip = input(\"Enter a sentence: \")\n",
    "pass_this = \"grammar:\" + user_ip\n",
    "result = happy_tt.generate_text(pass_this, args=beam_settings)\n",
    "print(result.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "0041d73d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Enter a sentence: Expectd\n",
      "Expected\n"
     ]
    }
   ],
   "source": [
    "user_ip = input(\"Enter a sentence: \")\n",
    "pass_this = \"grammar:\" + user_ip\n",
    "result = happy_tt.generate_text(pass_this, args=beam_settings)\n",
    "print(result.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "2771e636",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Enter a sentence: His name were John. He used to live next to Anna.\n",
      "His name was John and he used to live next to Anna.\n"
     ]
    }
   ],
   "source": [
    "user_ip = input(\"Enter a sentence: \")\n",
    "pass_this = \"grammar:\" + user_ip\n",
    "result = happy_tt.generate_text(pass_this, args=beam_settings)\n",
    "print(result.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "ee46e3f8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Enter a sentence: Please do not smok here. Its a hospitul.\n",
      "Please do not smoke here, it is a friendly place.\n"
     ]
    }
   ],
   "source": [
    "user_ip = input(\"Enter a sentence: \")\n",
    "pass_this = \"grammar:\" + user_ip\n",
    "result = happy_tt.generate_text(pass_this, args=beam_settings)\n",
    "print(result.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "c76583ba",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Enter a sentence: Kindly mov away from that plac\n",
      "Kindly mov away from that placard\n"
     ]
    }
   ],
   "source": [
    "user_ip = input(\"Enter a sentence: \")\n",
    "pass_this = \"grammar:\" + user_ip\n",
    "result = happy_tt.generate_text(pass_this, args=beam_settings)\n",
    "print(result.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "29b30649",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Enter a sentence: This modl is working fine\n",
      "This modl is working fine .\n"
     ]
    }
   ],
   "source": [
    "user_ip = input(\"Enter a sentence: \")\n",
    "pass_this = \"grammar:\" + user_ip\n",
    "result = happy_tt.generate_text(pass_this, args=beam_settings)\n",
    "print(result.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "782bc81a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Enter a sentence: Confidental\n",
      "Confidential\n"
     ]
    }
   ],
   "source": [
    "user_ip = input(\"Enter a sentence: \")\n",
    "pass_this = \"grammar:\" + user_ip\n",
    "result = happy_tt.generate_text(pass_this, args=beam_settings)\n",
    "print(result.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "178bae04",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Enter a sentence: He is passed\n",
      "He is passed on to the next generation .\n"
     ]
    }
   ],
   "source": [
    "user_ip = input(\"Enter a sentence: \")\n",
    "pass_this = \"grammar:\" + user_ip\n",
    "result = happy_tt.generate_text(pass_this, args=beam_settings)\n",
    "print(result.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "6a5f7b0f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Enter a sentence: we was about to leave but he stopped us.\n",
      "We were about to leave but he stopped us.\n"
     ]
    }
   ],
   "source": [
    "user_ip = input(\"Enter a sentence: \")\n",
    "pass_this = \"grammar:\" + user_ip\n",
    "result = happy_tt.generate_text(pass_this, args=beam_settings)\n",
    "print(result.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "3bdae36e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Enter a sentence: Are it too stubborn looking?\n",
      "Are it too stubborn looking?\n"
     ]
    }
   ],
   "source": [
    "user_ip = input(\"Enter a sentence: \")\n",
    "pass_this = \"grammar:\" + user_ip\n",
    "result = happy_tt.generate_text(pass_this, args=beam_settings)\n",
    "print(result.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "233e65d1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Enter a sentence: She waked up and chose violence.\n",
      "She waked up and chose violence .\n"
     ]
    }
   ],
   "source": [
    "user_ip = input(\"Enter a sentence: \")\n",
    "pass_this = \"grammar:\" + user_ip\n",
    "result = happy_tt.generate_text(pass_this, args=beam_settings)\n",
    "print(result.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "0d5c0cd1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Enter a sentence: He lookd lost today\n",
      "He looked lost today.\n"
     ]
    }
   ],
   "source": [
    "user_ip = input(\"Enter a sentence: \")\n",
    "pass_this = \"grammar:\" + user_ip\n",
    "result = happy_tt.generate_text(pass_this, args=beam_settings)\n",
    "print(result.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "1140aedc",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Enter a sentence: Is it too raini outside?\n",
      "Is it too rainy outside?\n"
     ]
    }
   ],
   "source": [
    "user_ip = input(\"Enter a sentence: \")\n",
    "pass_this = \"grammar:\" + user_ip\n",
    "result = happy_tt.generate_text(pass_this, args=beam_settings)\n",
    "print(result.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "fa016b1f",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Configuration saved in grammar_correction/config.json\n",
      "Model weights saved in grammar_correction/pytorch_model.bin\n",
      "tokenizer config file saved in grammar_correction/tokenizer_config.json\n",
      "Special tokens file saved in grammar_correction/special_tokens_map.json\n",
      "Copy vocab file to grammar_correction/spiece.model\n"
     ]
    }
   ],
   "source": [
    "happy_tt.save(\"grammar_correction/\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}