fixed-time-horizon-prediction/Exp_1_RNN_Boulder.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Experiment 4 summary:\n",
    "1. import bird model ghi values\n",
    "2. import actual data\n",
    "3. merge the two based on the common timestamps\n",
    "4. drop all rows if 'ghi' value is 0\n",
    "5. ** DO NOT drop all rows with dw_solar < 0 **\n",
    "6. Manage missing values: Drop any row if it has -9999.9\n",
    "7. Make Kt column ('dw_solar'/'ghi') \n",
    "8. Group the data into hourly format (by finding mean)\n",
    "9. assign hierarchical index\n",
    "10. get the subdataframes for each day and shift the Kt values\n",
    "11. drop the year, month, day, hour and minute columns\n",
    "12. feed the data into RNN alrogithm\n",
    "    - RNN seq_lenth = 1\n",
    "    - learning rate = 0.001\n",
    "    - single RNN layer\n",
    "    - num of epochs = 5000\n",
    "\n",
    "** in this experiment we are not giving Kt as one of the features. We are using hierarchical index to make sub dataframe for every day and then shift the Kt values one cell up to make it as one hour forecast. **"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### NOTE\n",
    "** the test['Kt'].max() values after grouping into hourly data is 7279.907547 = this is a outlier. The reason might be having a dw_solar reading with QC=2 value. Still haven't checked that. But the second highest value of 'Kt' parameter in the testset is 672 (which is way lower) **"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import datetime\n",
    "import glob\n",
    "import os.path\n",
    "from pandas.compat import StringIO"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### NREL Bird Model implementation: for obtaining clear sky GHI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import itertools\n",
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "import seaborn as sns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "sns.set_color_codes()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pvlib\n",
    "from pvlib import clearsky, atmosphere\n",
    "from pvlib.location import Location"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "bod = Location(40.0150,-105.2705, 'US/Mountain', 1655.064, 'Boulder')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "times2009 = pd.DatetimeIndex(start='2009-01-01', end='2010-01-01', freq='1min',\n",
    "                        tz=bod.tz)   # 12 months of 2009 - For testing\n",
    "times2010and2011 = pd.DatetimeIndex(start='2010-01-01', end='2012-01-01', freq='1min',\n",
    "                        tz=bod.tz)   # 24 months of 2010 and 2011 - For training\n",
    "#times2011 = pd.DatetimeIndex(start='2011-01-01', end='2012-01-01', freq='1min',\n",
    "                       # tz=bod.tz)   # 12 months of 2011  - For training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "cs_2009 = bod.get_clearsky(times2009) \n",
    "cs_2010and2011 = bod.get_clearsky(times2010and2011) # ineichen with climatology table by default\n",
    "#cs_2011 = bod.get_clearsky(times2011) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "cs_2009.drop(['dni','dhi'],axis=1, inplace=True) #updating the same dataframe by dropping two columns\n",
    "cs_2010and2011.drop(['dni','dhi'],axis=1, inplace=True) #updating the same dataframe by dropping two columns\n",
    "#cs_2011.drop(['dni','dhi'],axis=1, inplace=True) #updating the same dataframe by dropping two columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "cs_2009.reset_index(inplace=True)\n",
    "cs_2010and2011.reset_index(inplace=True)\n",
    "#cs_2011.reset_index(inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "cs_2009['index']=cs_2009['index'].apply(lambda x:x.to_datetime())\n",
    "cs_2009['year'] = cs_2009['index'].apply(lambda x:x.year)\n",
    "cs_2009['month'] = cs_2009['index'].apply(lambda x:x.month)\n",
    "cs_2009['day'] = cs_2009['index'].apply(lambda x:x.day)\n",
    "cs_2009['hour'] = cs_2009['index'].apply(lambda x:x.hour)\n",
    "cs_2009['min'] = cs_2009['index'].apply(lambda x:x.minute)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "cs_2010and2011['index']=cs_2010and2011['index'].apply(lambda x:x.to_datetime())\n",
    "cs_2010and2011['year'] = cs_2010and2011['index'].apply(lambda x:x.year)\n",
    "cs_2010and2011['month'] = cs_2010and2011['index'].apply(lambda x:x.month)\n",
    "cs_2010and2011['day'] = cs_2010and2011['index'].apply(lambda x:x.day)\n",
    "cs_2010and2011['hour'] = cs_2010and2011['index'].apply(lambda x:x.hour)\n",
    "cs_2010and2011['min'] = cs_2010and2011['index'].apply(lambda x:x.minute)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(525601, 7)\n",
      "(1051201, 7)\n"
     ]
    }
   ],
   "source": [
    "print(cs_2009.shape)\n",
    "print(cs_2010and2011.shape)\n",
    "#print(cs_2011.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "cs_2009.drop(cs_2009.index[-1], inplace=True)\n",
    "cs_2010and2011.drop(cs_2010and2011.index[-1], inplace=True)\n",
    "#cs_2011.drop(cs_2011.index[-1], inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(525600, 7)\n",
      "(1051200, 7)\n"
     ]
    }
   ],
   "source": [
    "print(cs_2009.shape)\n",
    "print(cs_2010and2011.shape)\n",
    "#print(cs_2011.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>index</th>\n",
       "      <th>ghi</th>\n",
       "      <th>year</th>\n",
       "      <th>month</th>\n",
       "      <th>day</th>\n",
       "      <th>hour</th>\n",
       "      <th>min</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2010-01-01 00:00:00-07:00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2010</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2010-01-01 00:01:00-07:00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2010</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2010-01-01 00:02:00-07:00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2010</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2010-01-01 00:03:00-07:00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2010</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2010-01-01 00:04:00-07:00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2010</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                      index  ghi  year  month  day  hour  min\n",
       "0 2010-01-01 00:00:00-07:00  0.0  2010      1    1     0    0\n",
       "1 2010-01-01 00:01:00-07:00  0.0  2010      1    1     0    1\n",
       "2 2010-01-01 00:02:00-07:00  0.0  2010      1    1     0    2\n",
       "3 2010-01-01 00:03:00-07:00  0.0  2010      1    1     0    3\n",
       "4 2010-01-01 00:04:00-07:00  0.0  2010      1    1     0    4"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cs_2010and2011.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import files from each year in a separate dataframe"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "\n",
    "- year            integer\t year, i.e., 1995\n",
    "- jday            integer\t Julian day (1 through 365 [or 366])\n",
    "- month           integer\t number of the month (1-12)\n",
    "- day             integer\t day of the month(1-31)\n",
    "- hour            integer\t hour of the day (0-23)\n",
    "- min             integer\t minute of the hour (0-59)\n",
    "- dt              real\t decimal time (hour.decimalminutes, e.g., 23.5 = 2330)\n",
    "- zen             real\t solar zenith angle (degrees)\n",
    "- dw_solar        real\t downwelling global solar (Watts m^-2)\n",
    "- uw_solar        real\t upwelling global solar (Watts m^-2)\n",
    "- direct_n        real\t direct-normal solar (Watts m^-2)\n",
    "- diffuse         real\t downwelling diffuse solar (Watts m^-2)\n",
    "- dw_ir           real\t downwelling thermal infrared (Watts m^-2)\n",
    "- dw_casetemp     real\t downwelling IR case temp. (K)\n",
    "- dw_dometemp     real\t downwelling IR dome temp. (K)\n",
    "- uw_ir           real\t upwelling thermal infrared (Watts m^-2)\n",
    "- uw_casetemp     real\t upwelling IR case temp. (K)\n",
    "- uw_dometemp     real\t upwelling IR dome temp. (K)\n",
    "- uvb             real\t global UVB (milliWatts m^-2)\n",
    "- par             real\t photosynthetically active radiation (Watts m^-2)\n",
    "- netsolar        real\t net solar (dw_solar - uw_solar) (Watts m^-2)\n",
    "- netir           real\t net infrared (dw_ir - uw_ir) (Watts m^-2)\n",
    "- totalnet        real\t net radiation (netsolar+netir) (Watts m^-2)\n",
    "- temp            real\t 10-meter air temperature (?C)\n",
    "- rh              real\t relative humidity (%)\n",
    "- windspd         real\t wind speed (ms^-1)\n",
    "- winddir         real\t wind direction (degrees, clockwise from north)\n",
    "- pressure        real\t station pressure (mb)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "cols = ['year', 'jday', 'month', 'day','hour','min','dt','zen','dw_solar','dw_solar_QC','uw_solar',\n",
    "       'uw_solar_QC', 'direct_n','direct_n_QC','diffuse', 'diffuse_QC', 'dw_ir', 'dw_ir_QC', 'dw_casetemp',\n",
    "       'dw_casetemp_QC', 'dw_dometemp','dw_dometemp_QC','uw_ir', 'uw_ir_QC', 'uw_casetemp','uw_casetemp_QC',\n",
    "       'uw_dometemp','uw_dometemp_QC','uvb','uvb_QC','par','par_QC','netsolar','netsolar_QC','netir','netir_QC',\n",
    "       'totalnet','totalnet_QC','temp','temp_QC','rh','rh_QC','windspd','windspd_QC','winddir','winddir_QC',\n",
    "       'pressure','pressure_QC']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1050018, 48)"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "path = r'.\\\\data\\\\Boulder\\\\Exp_1_train'\n",
    "all_files = glob.glob(path + \"/*.dat\")\n",
    "all_files.sort()\n",
    "\n",
    "df_big_train = pd.concat([pd.read_csv(f, skipinitialspace = True, quotechar = '\"',skiprows=(2),delimiter=' ', \n",
    "                 index_col=False,header=None, names=cols) for f in all_files],ignore_index=True)\n",
    "df_big_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(525132, 48)"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "path = r'.\\\\data\\\\Boulder\\\\Exp_1_test'\n",
    "all_files = glob.glob(path + \"/*.dat\")\n",
    "all_files.sort()\n",
    "\n",
    "df_big_test = pd.concat((pd.read_csv(f, skipinitialspace = True, quotechar = '\"',skiprows=(2),delimiter=' ', \n",
    "                 index_col=False,header=None, names=cols) for f in all_files),ignore_index=True)\n",
    "df_big_test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(3702, 48)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_big_test[df_big_test['dw_solar']==-9999.9].shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Merging Clear Sky GHI And the big dataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1050018, 50)"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_train = pd.merge(df_big_train, cs_2010and2011, on=['year','month','day','hour','min'])\n",
    "df_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(525132, 50)"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_test = pd.merge(df_big_test, cs_2009, on=['year','month','day','hour','min'])\n",
    "df_test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_train.drop(['index'],axis=1, inplace=True) #updating the same dataframe by dropping the index columns from clear sky model\n",
    "df_test.drop(['index'], axis=1, inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1050018, 49)"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_train.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Managing missing values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Resetting index\n",
    "df_train.reset_index(drop=True, inplace=True)\n",
    "df_test.reset_index(drop=True, inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Dropping rows with two or more -9999.9 values in columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1034397, 49)"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Step1: Get indices of all rows with 2 or more -999\n",
    "missing_data_indices = np.where((df_train <=-9999.9).apply(sum, axis=1)>=2)[0]\n",
    "# Step2: Drop those indices\n",
    "df_train.drop(missing_data_indices, axis=0, inplace=True)\n",
    "# Checking that the rows are dropped\n",
    "df_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(505639, 49)"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "missing_data_indices_test = np.where((df_test <= -9999.9).apply(sum, axis=1)>=2)[0]\n",
    "df_test.drop(missing_data_indices_test, axis=0, inplace=True)\n",
    "df_test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "# For the rows with only one cell as -9999.9, replacing this cell with the mean of the column"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### First resetting index after dropping rows in the previous part of the code"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 2nd time - Reseting Index\n",
    "df_train.reset_index(drop=True, inplace=True)\n",
    "df_test.reset_index(drop=True, inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "one_miss_train_idx = np.where((df_train <=-9999.9).apply(sum, axis=1)==1)[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10058"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(one_miss_train_idx)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1034397, 49)"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\s269596\\AppData\\Local\\Continuum\\Anaconda3\\lib\\site-packages\\pandas\\core\\indexing.py:194: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n",
      "  self._setitem_with_indexer(indexer, value)\n"
     ]
    }
   ],
   "source": [
    "col_names = df_train.columns\n",
    "from collections import defaultdict\n",
    "stats = defaultdict(int)\n",
    "total_single_missing_values = 0\n",
    "for name in col_names:\n",
    "    col_mean = df_train[~(df_train[name] == -9999.9)][name].mean()\n",
    "    missing_indices = np.where((df_train[name] == -9999.9))\n",
    "    stats[name] = len(missing_indices[0])\n",
    "    df_train[name].loc[missing_indices] = col_mean\n",
    "    total_single_missing_values += sum(df_train[name] == -9999.9)\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_col_min = df_train.apply(min, axis=0)\n",
    "df_col_max = df_train.apply(max, axis =0)\n",
    "#print(df_col_min, df_col_max)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "train = np.where((df_train <=-9999.9).apply(sum, axis=1)==1)[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "# doing the same thing on test dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6657"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "one_miss_test_idx = np.where((df_test <=-9999.9).apply(sum, axis=1)==1)[0]\n",
    "len(one_miss_test_idx)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\s269596\\AppData\\Local\\Continuum\\Anaconda3\\lib\\site-packages\\pandas\\core\\indexing.py:194: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n",
      "  self._setitem_with_indexer(indexer, value)\n"
     ]
    }
   ],
   "source": [
    "col_names_test = df_test.columns\n",
    "from collections import defaultdict\n",
    "stats_test = defaultdict(int)\n",
    "total_single_missing_values_test = 0\n",
    "for name in col_names_test:\n",
    "    col_mean = df_test[~(df_test[name] == -9999.9)][name].mean()\n",
    "    missing_indices = np.where((df_test[name] == -9999.9))\n",
    "    stats_test[name] = len(missing_indices[0])\n",
    "    df_test[name].loc[missing_indices] = col_mean\n",
    "    total_single_missing_values_test += sum(df_test[name] == -9999.9)\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "test = np.where((df_test <=-9999.9).apply(sum, axis=1)==1)[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1034397, 49)"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(505639, 49)"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_test.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exploratory Data Analysis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
    "dw_solar_everyday = df_test.groupby(['jday'])['dw_solar'].mean()\n",
    "ghi_everyday = df_test.groupby(['jday'])['ghi'].mean()\n",
    "j_day = df_test['jday'].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEgCAYAAACjEpTiAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzsnWl4FMXWgN+ZJCRMQljDEgg7lMoq\noCKL4gYCKu5XcUdQUa/gJ4gCV0BBAZGrooIICBcFd0EEBHFFRFRE2UtWycISCBCSyTbL96N7wiTp\nmelJZkgC9T5PHpjururT3TN1us45dY7F7XajUCgUCoVZrOUtgEKhUCgqF0pxKBQKhSIolOJQKBQK\nRVAoxaFQKBSKoFCKQ6FQKBRBoRSHQqFQKIIisrwFUFQchBBdgZeA2mgvFcnACCnltgDt5gNbpZTT\nynDuMvdh0GcX4BMpZVMhxCNADSnl5FD1HyqEEM8Ad+gfWwLpwEn98y1Syj2l6HMe8JqU8i+vbRZg\nBXBCSnmn1/ZqwGZggpRyfqkuwr8sccBMoCMQAfwE/FtKmSeEaAAsABoBDuBBKeVverubgOeBaOB3\nYIiUMlsIkQDMAtoCWcBsKeXboZZb4Rs141AAIISIBr4EnpJStpdStgXeB1YKISLKV7qyI6WcVRGV\nBoCUcrKUsqOUsiPaADnS87k0SkOnN2Apdh43cD9wpRDiDq9dM4Afw6E0dMYDuUB7/a8O8JS+723g\nKynlBcCDwKdCiGghRKK+73opZWs0ZfqC3uYN4DBwPtANuF0IcXWYZFcYoGYcCg82oAYQ57XtfSAT\n7S3RKYR4CHgCcKL9cB+XUv7t3YkQYhDwMFAFqAVMllLOFELcjzYwxAInpZRX+BJECPE9kAGch/am\nekuxz78BU9HeRBsAX0spH9TbDgWeRHtj3+LV53igjpTycSHEdcBoXca6wAIp5X+EEL2AScBetLfZ\nKOBhKeU6/a15BtAd7c14CTBGP2YKcLl+nzYBT0gpM33e6SARQiShDZaN9PO9L6WcIoSIAt4ELgXy\ngd3AA8Bz+nV9KIS4S0r5u6cvKeVh/Vm8p9/nS/S/Ll7nGwoMRnuxPIL2nHcJIdoAr6M9w0S053CH\nft3pwNf6fbtFSll474FvtFNLN+AQQvwJNBFCxAB9gLt12X4TQqQBV6E917VSyv16HzOBH4H/AzoD\nd0spXUCeEGIFcCuwpnR3WBEsasahAEBKeRx4GvhKCLFXCLEQbRBaI6XMF0Jcqe+/QkrZAVgELNHN\nH0ChSWII0E9KeSHwL7QB3kMboJc/peHFcSnlBVLKGQafhwHPSSkvAS4AbhBCdBZCdER7u71MSnkR\n2mBaBF3ep4D7pJRdgK7As0KIOvohlwCv6PK/C7yob38eiEF7y+2IpkAuB55BUySd9fuSBoR6ZvM+\nMEtK2VmXr58Q4magB9BNStlO33cAaCelfAZtwP+Xt9LwIKVcCbwHzAX+C9wupcwGEEL0RlPU3fV7\n8Cbwod70YV2OrmgmtbZoMxvQXjgWSylbF1MaSClXSin36v23AB4DPgHqAznFlGwKmoJMQjOVem9P\nEEJUATYA9wohIoUQ1YGb0BSN4gyhFIeiECnldKAe2qziIDAK2KT/OK8FPpRSpuvHzgcaAk292mcB\n1wH9hRAvoL2Re89gNgfxJr7Wz+f7gBpCiNHAW0BV/TxXAaullIf042YbXKMbuB7oLIQYB0xHM+nE\n6of8I6X8U///H2izJoCrgblSSqeUMl9KebmU8nv9egeg3ac/gRvRlFlIEELEoympl/T+16Pd947A\nX0CEEGKDEOJ54CMp5QaTXT+NpgTfLDbQX6dv/0U/3wtAA/2l4EkgSwgxCm0GkEDR51v8mRW/lq5o\ns4YpUsqv0caf4jmPLGgz2uL7PC8oLrTvZ1XgT+AjYBUGLwmK8KEUhwIAIUR3IcRIKeUpKeWXUsqn\n0WYIbuAaNHOE0Y88yquPRmg/5iZoDtCxxY7PCkKk4sd6f/4R6AfsRJsJpHJ6YPG26zuKdyqEiEUz\nJ3VCUwwjgQKvdjleh7u9tjvwun4hRJIQojbafRnm5aO4GM1sUvy8K4QQf+p/N/i6aAM8/qVLvM5x\nKdrgmwG0Q1MCbuBj3ZwYECllHtrsqLgPJQKY43WuTkAX/aXgMzSlvQ+YhmYK9L7fPp+vEOJeYDna\nvZqub04DYvVn4iERbXZxQP+/9/ZDUkoHUA14UkrZVkrZB+07uNvMdStCg1IcCg/pwFghRA+vbQ2A\n6mgDxFfAHXpEC0KIB4BjFP3BdtH7mQisRnt7JZTOdSFEDeAiYJSU8jM0s0ZLtAFvNdBbV2CgOYKL\n0wqIB8ZKKZcBvdB8JYFkXAPcJ4Sw6oEEn6CZqlYBjwshqgghrMA7aJFpRZBS9vNyeH9h9np1E+JG\ntLd9hBA10WYd1wkhbtTPv05KOQ7NpHWR3tSBl1IPglXA3UKIuvrnYWjmSwuaWWqclPIjtHvWhcD3\nDSHErWgmyyullJ94XVsuml9kiH5cZ6AZ2kvHV8DlQoim+uEPo/mVQJtxjNXbJKI958WluFZFKVGK\nQwGA7uS+EXhR93FsRzMDPCA1vkazh38rhNiG9uZ5ne6g9LAa7W1RAjuAxmiKpGUI5TyBNjD/IYTY\niuZjWAe01E0uTwPfCCF+R/NJFGczWvTYTiHEDjSz1XYTMk5AM4f8hTZjWaErrheA/fq27Whv4E/5\n6KO03AFcJoTYDPyC5sz/UL+OXcBW/Xov4nTk0WfAB7pvyjS6UnsD7TlvQTPD3aqb+J5FUyJb0Jzk\nP2Du2b6CNtYs8Jp1/Vff9xDQR3+W84A7pZTZUsoU4BHgCyHETqA52rMGbZZ5nt5mFVrI+OZgrlNR\nNiwqrbpCoVAogkHNOBQKhUIRFEpxKBQKhSIolOJQKBQKRVAoxaFQKBSKoKiUKUfS00+FzKNfs6aN\n48ftoeoubCg5Q4uSM/RUFlmVnOZISKhm8bXvnJ9xREZWjvx9Ss7QouQMPZVFViVn2TnnFYdCoVAo\ngkMpDoVCoVAEhVIcCoVCoQgKpTgUCoVCERRKcSgUCoUiKJTiUJxb2O1Y9+0Fe8UPx1QoKipKcSjO\nDRwOYseOolbPi6l1aSdo04bYsaPAUaJkh0KhCEClXACoUARL7Pgx2GbPPL1h//7Cz9kTp5STVApF\n5UTNOBRnP3Y70SuXG+6KXrlCma1CiTIFnhMoxaHwzVkyCFgPH8KammK8Ly0F6+FDhvsUQeBwwPDh\nhabAWj0vDokpMC0tlTFjRvL44w8xdOggpk2bjN2eDcCkSeP55ZefQyF9qfnll5+ZNGm832NuuKHP\nGZHlTJ0HlOJQGFHMHxCqQaC8cNWrj6thI+N9iY1w1at/hiU6+4gdPwZee42I5ANYXC4ikg9gmz1T\n215K8vJyeeaZ/2PgwPt4443ZzJw5jzZt2jK+DH0qQoPycShKUNwf4BkEoJL6A2w28vr2L+rj0Mnr\n2w9stnIQ6iwigCkwe/S4Ut3jn3/+iY4dO9GmTdvCbX37Xsfnn39Cqj6D/Pzzj1m8eCFOp5NnnvkP\nCQl1ee65Z8jOziYvL5ehQ5+gU6cufPvtGj788H2sVitdu17Mffc9zNy5b7N162ZycnK48spryMo6\nxaBBD5Gfn8/999/JggUfsHTpp3z99SosFgtXXdWb2267g/379/HSS88TE1OVqlVjqFYtvojcTqeT\nqVMnsW/fXho2bER+fj4nT55g2LBHmT9/EVu3bmbkyOF8+eXXHDt2lMmTX2D69DcM78Fnn33MypVf\nYrVaad++I489NoyDB9OYPPkFHA4HFouFYcNG0KpV68I2mzZt5N133wEgNzeXsWMnEBUVxahRTxIf\nX51LL+3OXXfdF/Tz8EYpDkVRwjQIlDfZ4ycB2jVY01KwJCVh7923cLui9JgxBbqaNQ+637S0VBoa\nzBQbNEjksG5ebNu2A/fccz/r1//EW2+9zuDBj5CRcYxXX32L48ePk5z8D5mZJ5k3723mzFlITEwM\nU6c+z2+//QJAkybNGD58BJmZmTz66GAeeGAIP/30I9269SQlJZlvvvmat96ag8ViYfjwR7nkkq7M\nmTOTwYMf5qKLuvLee/P555/9ReT75Zefyc/PZ/bs+Rw6dIjvv/+G6tVrUL16dQ4fPsSGDeupW7ce\nUu5g584dXHZZL5/3YMWKZQwfPpK2bdvx+eef4HA4ePPNV7n11n/Rs2cvdu2STJ78AnPnLixss2/f\nXp577gXq1Engf/+bx3ffraF3775kZBxj7tz3iIqKCvpZFEcpDkURwjUIlDuRkWRPnEL26HFYDx+i\ndttWZGc7y1uqswKPKTAi+UDJfWUwBSYk1GX79m0ltqekJFNP77NjxwsBTYG8+ebrNG/egptvvp3x\n48fgcDi49dY7SElJ5sSJ44wY8QQABQV5tG6dCkDjxk0AiI+Pp3VrwebNf7Jy5TIef/xJdu/exeHD\nhxg2bCgAp06dIiUlhX379nL++dosqF27jiUUx759ezj//DYA1K9fn7p16wFw2WW9WL9+HVu2/MVd\nd93Hb79tYOvWzTzzzH983oPRo59j8eL3mDVrBm3atANg//79dOjQCYBWrQRHjhwudt8SePXVl6la\n1UZ6+hHatesAaAo3FEoDlI9DUYyz3h9gs2mKrxLOmiosuinQiLKYAnv0uJzff9/A9u1bC7ctW7aE\nGjVqFs5EduzQFMtff22iefMW7NmzG7s9m5dffo0xYybw6qsv06BBQ+rWrcerr77FG2/M5u677y40\nf1mtp0tOXH/9jXz00SLy8vJo0qQpjRs3oWnT5syY8TZvvDGbfv2uo3nzljRu3JStWzcDsHNnScXW\npElTtm3T9h89mk56ejoAPXv2Ys2aVcTGxtK1azfWrv2B/PwCateu4/MefPHFEkaMeJY33pjNrl2S\nLVv+omnTpmzevAmAXbsktWrVLtJmypSJjB49jjFjxlOnTkLhdosldMO9mnEoiqL8AYpSkD1+Eraq\nVXB+tgRrWgquxEbk9e1XJlOgzWZjypT/8vrrr5CZeRKHw0nLlq0Y79Xntm1beOKJR7BYLDz77HPU\nrFmLd9+dzVdfLScyMooHH3yYmjVr8q9/3cXjjz+E0+mkadPGPPVUSQf7hRd2ZurUSdx77yAAWrVq\nTZcuF/Hoow+Sn1/A+ee3ISEhgaeeeoZx455l8eKF1KhRgypVoov007NnLzZv/oshQ+6jfv0G1KhR\nA4C6deuRl5dH584XEx8fT0REBN26dfd7D1q0aMmQIfdSo0ZNEhISuOCCttSv34ApUyayePF7OBwO\nnn226IylT59+PPTQ/VSrVo2aNWtz9Gh6qe6/Pyxud8iK6Z0xQlkBMCGhGunpp0LVXdg4o3I6HMSO\nH1PoDygyCET6f9dQ9zO0VBY5QZf1n8OaObNe/Qr7klFZ7ml5y+mvAqCacShKUswfUJEHAUUFw2MK\nVJjm0KFDTJz4XInt3btfyp13PlAOEgVGKQ6Fb9QgoFCEnfr16/PGG7NLbC/vGYc/lHNcoVAoFEER\nthmHEOJ+4H79YwzQEegFvAY4gNVSyglCCCvwFtAByAMGSyl3h0suhUKhUJSNsCkOKeV8YD6AEOJN\nYB4wC7gF2AssF0J0ApoCMVLKS4UQXYFXgAHhkkuhUCgUZSPsPg4hRBegDfAs8KSUco++fRVwFdAA\n+ApASvmLfrxCoaiE2AvsHLYfop6tPraosgdU7N27h5kzXyc3N5ecnBwuvbQ7gwY9xKZNG1m69FMm\nTHgpBFLD8ePHmTbtRXJycnC73dSv34Dhw0eQkZHBuHGjmT17flD9paamMHPm6xw5coSYmBiio6MZ\nOvQJmjdvwdy5b1O7dm1uvPHWwuMfeuh+Jkx4kYMH00J6XeHiTDjHRwMTgHgg02v7KaC5vv2k13an\nECJSSukzo17NmjYiIyNCJmBCQrWQ9RVOlJyhRckZOhwuB8O/Gs7SnUs5cPIAjas3ZsB5A5jWexqR\n1tINM5mZmUyc+B9mzJhB06ZNcTqdDBs2jG++WU7z5s2Jjo4q9b0p3u7dd2dyxRWXc+eddwIwadIk\n1qxZztVXX01UVERQ58nJyWHs2JG88MILXHihtrJ98+bNvPzyyyxcuJDY2Gji4mKK9BkVFUGtWrHk\n5NiKXFdFffZhVRxCiBrAeVLK74QQ8YD3XagGnABsxbZb/SkNgOPHQ5fmuyJHLnij5AwtSs7QMvan\nUczefHrR6P6T+3ltw2vk5OQzsUfpEmOuXLmc9u07ERtbu/AejBz5H6Kiotiy5S/y8gpITz9VJIFh\n+/YdGTr03xw5cphp0yaTn59HZuZJ7r9/CJdd1ot77rmdli1b4HJZmDDhxcJzxcfXZtmyFVSvnkD7\n9h0YNOhRLBYLR44cpqDAyaFDJ3jxxfE0a9aCatXiSUlJ5rHHhuF0OnnggYHMmbOQKlWqAPDNN6vp\n0KETjRq1LJS7QYNmvPLKm6SnnyI7O4+YmNwiz7WgwElGRjYnTtgLr6u8n70/pRXuGcdlwBoAKWWm\nECJfCNECzcfRB20m0gi4HvhI93FsCbNMCoUihNgL7Kzca5wYc+W+FYy+ZFypzFZHj6aTmNiwyDZb\nsfVExRMYvvDCf/QEhhbuuOMuOnXqwpYtfzF37ttcdlkvcnJyePTRR0lISCrSz0033Up0dDSLFy/k\nP/95hvbtO/LUU6MALdvt88+PpUOHTtx8823Y7dkMGnQ3jzzyOBs2rKdTpy6FSgMgLS2Nhg1P9//M\nM/9HVlYWx44d5bXXNOX6wQeLWLNmdeEx+/fvC/r+lCfhVhwCTUl4eAR4H4hAi6raIIT4DbhGCPEz\nYAEq5ooXhUJhyGH7IVKzjBNjpmWlcNh+iGbVg18PVK9eA/7+e2fR/tJSiyT1K57A0G63k5qaSvv2\nHVmwYC7Lly8FLDi8ask0a9aMrKyiRo0//vida6/tz3XXDSA/P59Fi/7H66+/oic7/JvY2FhycjRL\nh80WS8eOnfj11/WsWPEF998/pJjc9di5c3vh58mTpwOaH8Pp1BJr3nHHwBI+jspEWBWHlPLlYp9/\nAboW2+ZCUygKhaISUs9Wn4ZxjUjOKpkdNzGuEfVspUuM2b17DxYunMdNN91Kw4aNcDgczJjxXy66\n6BKaNm0GUCSBYWRkJCtWLKNVq9bMmTOL66+/kUsv7c7y5V+wcuWXhf1arSWXr3388WLS0lK54Yab\nqFKlCs2aNS+cBQhxPi+//CpDhtzHJZd0o2XLVlx//U28//4CTp48QcuWrYr01aPH5bz33ny2bt1C\n27ZaRtuUlGTS049gsfjM4lGpUCvHFQpFmbBF2ejbvH8RH4eHvs36lTq6KjY2jjFjJjBlykRcLhd2\nu53u3Xty0023smnTRoASCQwbNEjkyiuv4YorruK116axcOG71K1bjxMnTvg918iRo3nllcl8/vnH\nREfHUKNGDUaMeJaCggIAoqNjGDHiWSZOHMfs2fNp06YtqanJ3HTTbSXvh56ccdasGcyadQyn00FE\nRCRPPz2a+vUblOpeVDRUksNK4nxUcoYWJWdocbgcTNk0gc+2LyEtK4XEuEb0bdaP8d0mlTqqKlyE\n4p66XC6GDn2Q6dNnEBsbFyLJilLez14lOVQoFGEl0hrJq9e+ypPtnw3pOo6KSFpaKqNHj2TAgJvD\npjQqOkpxKBSKkGGLspXKEV6ZSExsyPz5i8pbjHJFJTlUKBQKRVAoxaFQKBSKoFCKQ6FQKBRBoRSH\nQqFQKIJCKQ6FQqFQBIVSHAqFQqEICqU4FAqFQhEUSnEoFAqFIiiU4lAoFApFUCjFoVAoFIqgUIpD\noVAoFEGhFIdCoVAogsJUkkMhRBxwBdAKcAG7gTVSytwwyqZQKBSKCohfxSGEsAHjgJuBzcA/gAPo\nBvxXCPEZ8IKUMivcgioUCoWiYhBoxvEeMBt4Vi/xWogQwgpch1ZDfEB4xFMoFApFRSOQ4rhFSmlY\nbU9XJF8IIZb5aiyEeBa4AagCvAX8AMwH3MBW4DEppUsIMQ7ojzabGS6l/DXYC1EoFArFmSGQczxC\nCDFMCPGKEOIy7x1CiPEAvhSLEKIXmkmrO3A5kARMB8ZKKXsCFmCAEKKTvv8S4A7gzVJfjUKhUCjC\nTiDF8TZwIZAGLBBCjPbad0OAtn2ALcDnwDLgS6Az2qwDYCVwNdADWC2ldEspDwCRQoiEoK5CoVAo\nFGeMQKaqLlLKDgBCiP8Ba4QQdinlq2gzBn/UAZqg+UGaAV8AVq8ZyimgOhAPHPNq59me7qvjmjVt\nREZGBDi9eRISqoWsr3Ci5AwtSs7QU1lkVXKWjUCKwyqEiJVSZksp04UQ/YB1Qoh0ND+FP44BO6WU\n+YAUQuSimas8VANOAJn6/4tv98nx4/YApzZPQkI10tNPhay/cKHkDC1KztBTWWRVcpo/vy8Cmapm\nAH8IIa4CkFKmAtcCLwLnB2j7E3CtEMIihEgEYoFvdN8HQF9gLbAO6COEsAohGqPNSo4G6FuhUCgU\n5YRfxSGlnI1matrltW0n0AZ4NkDbL4FNwK9oPo7HgKeACUKI9WiRVp9IKTeiKZD1wKf6cQqFQqGo\noFjc7kAWJxBC9EALnb0z/CIFJj39VGChTVLe00GzKDlDi5Iz9FQWWZWcps/v04/t08chhIgG7gIe\nBlKBOaEXTaFQKBSVDX/O8cfR0o0MllJ+dIbkUSgUCkUFx6ePQ0r5CtAPuFEIsUUIMerMiaVQKBSK\nikog5/hPUsqBwDVozmyFQqFQnOOYSqsupTwkhFhSPO2IlPLH8IilUCgUioqK2Xoci9HShaR6bXYD\nV4ZDKIVCoVBUXEwpDqAjcL6U0hlOYRQKhUJR8TFbOnYD0DKcgigUCoWicmB2xvENsE0IkYZWM8MC\nuKWUzcMmmUKhUCgqJGYVx2g0f8Y/YZRFoVAoFJUAs4rjKLDWV9EmhUKhUJw7mFUcfwO/CCG+BvI9\nG6WUz4dFKoVCoVBUWMwqjgP6HwQu4KRQKBSKsxi/ikMI8SGwApglpTx8ZkRSKBQKRUUm0IzjKbSC\nS28JIWqhFV1aCayXUrrCLZxCoVAoKh6BclWlSCnfkVLegpavag1wA/CDPhtRKBQKxTmGWR8HUkoH\n8L3+hxCiYXhEUigUCkVFxuzK8RLo9ccVCoVCcY5RasWhUCgUinMT04pDCFFF/7elEKK/EEIpHYVC\noTgHMZtW/TngfL0K4I/AdqA3MCxAu03ASf3jPuBt4DW0fFerpZQTdAX0FtAByEMrVbu7FNeiUCgU\nijOAWef4AKAHmqJ4T0r5tBDid38NhBAxAFLKXl7b/gRuAfYCy4UQnYCmQIyU8lIhRFfgFf18CoVC\noaiAmFUcVilljhDiOmCsPkuIDdCmA2ATQqzWzzMeiJZS7gEQQqwCrgIaAF8BSCl/EUJ0CSRMzZo2\nIiMjTIoemISEaiHrK5woOUOLkjP0VBZZlZxlw3RadSHEVsCOZqr6AfgiQBs7MA2YA7RCWzh4wmv/\nKaA5EM9pcxaAUwgRqYf/GnL8uN2k2IFJSKhGevqpkPUXLpScoUXJGXoqi6xKTvPn94VZxTEJeB1I\nlVK6hBD/Bo4HaPM3sFvPqPu3EOIkUMtrfzU0RWLT/+/B6k9pKBQKhaJ88RsZJYRIEkI0Btai1Rhv\nqH/OBFYF6HsQmr8CIUQimoLIFkK0EEJYgD56v+uAfvpxXYEtpb8chUKhUISbQDOOCcAVQCKaicpD\nAbA8QNu5wHwhxE9oSmcQ4ALeByLQoqo2CCF+A64RQvyMlnn3gaCvQqEII/YCO4fth6hnq48tylbe\n4igU5Y5fxSGlHAQghBglpZzivc+zrsNP23xgoMGursWOcwGPmJJWoTiDOFwOxv88hpV7l5OalULD\nuEb0bd6f8d0mEWk1na1HoTjrMLuI70bvD3pU1cbQi6NQVBzG/zyG2Ztnkpx1ABcukrMOMHvzTMb/\nPKa8RVMoypVA9Ti+BXrp//dOo+4gcFSVQlFpsRfYWbnX2Bq7ct8KRl8yTpmtFOcsgUxVVwIIIV6T\nUvpdJa5QnE0cth8iNSvFcF9aVgqH7YdoVr35GZZKoagYmDXUjhJC3ATEoTmwI4BmUsrnwiaZQlGO\n1LPVp2FcI5KzDpTYlxjXiHq2+uUglUJRMTDr41gEPAG8CFyr/3t+uIRSKMobW5SNvs37G+7r26yf\nMlMpzmnMzjjao63+fg2YB4wFVAVAxVnN+G6TAM2nkZaVQmJcI/o261e4XaE4VzGrOI5IKd1CiJ1A\neynl/wKF4yoUlZ1IayQTe0xh9CXj1DoOhcILs4pjqxBiBjATeF9fCW4Jn1gKRcXBFmVTjnCFwguz\nPo6hwEdSyu3AOLSMtkaL+xQKhUJxlhNoHcd3wGq09CBrAaSUX6DWcCgqId6pQ4rm1VQoFMEQyFT1\nEtATeEUI0QItIeHXaIokOdzCKRShwCh1yM1tbmLUheNU6hCFohQEWgC4Gm3G4clNdRGaIvlSCBEt\npTwv/CIqFGXDkzrEQ3LWAV7b8Bo5OflM7DHFT0uFQmGEKR+HEEKgJSIcAdyPVj/89fCJpVCEhkCp\nQ+wFoSsKplCcKwTycbyNVt71MJqJahrwi5TSeQZkUyjKTKlSh9jtWA8fwlWvPthU+K1CUZxABt4b\ngc3Ap8AqKeW+8IukUISOoFKHOBzEjh9D9MrlWFNTcDVsRF7f/mSPnwSRyheiUHjwa6qSUtYDRgE1\ngXeFENuFEG8KIQYIIVRYiqLCE0zqkNjxY7DNnklE8gEsLhcRyQewzZ5J7PizN426vcDOvpN7lclO\nERQBX6OklH8AfwAvCSHigNvRapC3AqLDK55CUXaMUofcfMGNjLpw3OmD7HaiVxr7QqJXriB79Liz\nymylilQpykLAb4gQ4nygu/53KZAFrERzlCsUFR6j1CFNEuuRnn6q8Bjr4UNYU419Ida0FM3n0ezs\nWT1uFGnm+awizRSBCOQcPwYcAb4FvgSeklJmnAnBFIpQ4y91iKtefVwNGxGRXNIX4kpspDnKzxJU\nkSpFWQk042gnpUzzd4AQIkZKmetnf120MrPXoFUOnA+4ga3AY1JKlxBiHNBf3z9cSvmr+UtQKEKA\nzUZe3/7YZs8ssSuvb7+zykxdB3vgAAAgAElEQVSlilQpykqgdRwzhBBDjBzhQohqQojHgMW+Ggsh\nooC3gRx903RgrJSyJ1qSxAFCiE7A5cAlwB3Am8FfhkJRdrLHT8L+0FCcSU1wR0TgTGqC/aGhWlTV\nWYQn0swIVaRKYYZAM47b0BIc/iaEOAGkAAVAM6A2Wn2O2/y0nwbMAp7VP3cGftD/vxLoDUi0FCZu\n4IAQIlIIkSClTC/F9SgUpScykuyJU8gePe6sXsfhiTTz9nF4UEWqFGYIlHLEhTYDeFMI0QEtksoN\n7JZS/uWvrRDifiBdSrlKCOFRHBZdQQCcAqoD8cAxr6ae7T4VR82aNiIjI/ydPigSEipHZLGSM7T4\nlrMaNKl3RmXxRzju55sDXqdq1SoslUtJPplMUvUkBogBTOs9rUxRVZX/2VcsKqqcpr8huqLwqyyK\nMQhwCyGuBjoC/wPqeu2vBpwAMimaqtSz3SfHj4cu5jwhoVqR6JqKipIztCg5YUznF3iy/bNFilQd\nP5YTuKEP1D0NLeUtpz+lFbaAbSnlZZ7/CyG+R8t19bIQopeU8nugL/AdsBuYKoSYBjQCrFLKo+GS\nS6Ewg73AjszYQVp2GomxiYha52OLsnEs5yibDm8kNiqOmMiqhcfnOnLJyDtGrehaRbbHRMbQJL5p\nhTX/qCJVitJwplf6PAW8o2fa3QF8IqV0CiHWAuvRnPWPnWGZFOcg9gI7O45tI9eRR64jl2xHFhfW\n7UT16BqM/WkUC7bOw8nplGxWrFSvUp3j+ceDPleMJYbeza/l4faPF5qBKrpCUSj8YXG73YGPAoQQ\n3YF2wDygq5Tyx3AK5o/09FPmhDZBeU8HzaLkLD32Ajv/ZO4DLDSMa8QLv4zjY7kYu6OoydOChZox\nNcnIPTNLlWKsMfRudlqhGCmTing/fVFZZFVymj6/z/LgpmYcQohhaAkPGwIfA28LIeZKKaeFRkSF\nIrTYC+xsP7qVVzZOZX3qT9idmpKwYMGN8XuHG/cZUxoAua5cvtizhC/2LCnc5lEmwzqNoEWNlqhK\nhYqKiFlT1f1o6yw2SCmPCSEuAn5FC7dVKModz6wiOz+bt/6awcq9XxYxNXnwpTQqCt7KJMYawxXN\nrmDIBY9R21ZHmbYUFQazisMppczX6jkBkAsGv0qF4kxit5OVsovhu15mVfIq8lx55S1RSMl15bJy\nz0pW7lkJQFVrVa5p1qdwNqKUiKK8MKs4ftCjnmKFEDcCDwHfhE8shcIYe4Gdf47vxvnWNN7I/IrP\nm+XiDt2SngpNjiuncDYSbY3m6ia9+b8uo5QSUZxxzCqOkcAQtHUc9wLL0VKJKBThR59ZjPh7KiuT\nV5PjyoMEiq4KCiE1qtTkVMEpnG5H4TYrEVSvEs/xPJNRVT7diqEhz5XH8n3LWL5vGVWtVenboj9T\nL3uV+Oj48J5YocC84rABkVLK24QQDYGHgSpoSQkVirBgz80k7aXhTM/8kiXNcnF4vq1hGpQtWLig\ndhtW3vItLrer5DqOAnBd1YmNpFEtH2K8vv05kZBeM4q6xwuIdMKrXWGZgJwqRU4QFnJcOXy26xM+\n3/UZ/Zr1V7MQRdgxqzgWAVv0/59CW2+xELglHEIpzm0y8zL5v+//zepdS8mt64IQZv+wYsUWZcNe\nYKeurS5dG3TjngsGkZdzgs7uhtRMOh8iYwC4sF5nLqTz6bYpe6mz7xD9XCX7dVutnHhxITXuuQML\ncOlnYI+CvTU0peK2QsZbs5n78TCWJ+WERaG4cRWZhSh/iCJcmFUcTaSUNwBIKTOBsUKIP8MnluJc\nJDMvkxE/PMHS3Z9r0U+BcjcHgZUIBrS8iZcvf5VIaySOmCwic+OwWaqYrjPut2ZHwyQcXS7GldS4\ncL+tANrqGdecSU3I6HADly39FabNKVQoDovB7CQEisTbHxITEcOtrW9nqn7tCkVZMfvTdAsh2nk+\nCCHOQ8uSq1CUmcPZh7h7+e2cN7cpS3Z/VvaQWbf2VzUPrmpwJStuWsPeIam83Xse8dHx2KJstKjV\nAluULbg643rNDiPy+vaD2nX877fZyBk8lKq6QrnoIFyaBh9+BkenwZa34Od34NbNEOE4fR1lJdeZ\ny3s7/oeY25S/jmxS9cUVZcbs68cI4GshhKf6SwJwT3hEUpwrZORk0P+zq9lzcndoOnRDy3SYvxSq\n50P9mwfjfnK67+NLUWfcU5sjeuUKrGkpuBIbkde3X+H2QPtdiQ2LzEo8eM9OPv4M7MtgWx34Jx4W\nt4OvhGb6KgunCjK55pPLqRpRlb7NlTNdUXqCSTlSBS3lSAEgpZTlFjSvUo5UXMzImZmXydM/Dufz\nXZ+GZHZRNR+u2gvvLIP6+su0K64ax/7cAfHGA2NCQjWO/foXtS7thMVV0mnhjogg4+eNvuuM2+3+\na3b42R87dpRhpUF/2KNg4wMDuO+8rezL3BNUW19YieDeNvfxYs+ypVL3cDZ9RysC5S1nKFKONAEe\nB2qhW2CFEEgpB4VEQsU5gUdhfLF7CQ53GQLydGVxnYTRP0Hr49obuzeWHDvWY0dx+VAcUMY64zab\nb6USYH/xWYm7qg1rlv8BwlYA3Zb9xobRf3DMYuf3g7+yWL7Pyj3LcJfSF+TCyfxt81if9jPvXvs+\niXENlRNdYQqzrxkfAWv1v4qds0FR4ShUGLuW4ChDBLctD/r7URbeBBz4ofzqjOfnk/Pgw2Q/ORJr\nZiYuWyy1L70Qa1aW32bWI4exHj5E7WbN6dO8H33q94IJneh5bRqyDtorXSkc6/L4Trot7kwDWwP6\nNu/PxB5TlRNd4Rez344oKeWIsEqiOOvwKIyluz43zBtllmbxzXmn5Vi63vogsfnm3lvMDvyBfBIh\nxeEwjODKuX8wFntgh3VxZWg9fIha+w+x8y04WhXWN4T5HeGrVmCvQtBK5KD9IPO2zuGXg+tZc9ta\npTwUPjH7zfhJCHE9sEpKmR9OgRSVH4fLwcgfhvPBjvdLrTCsWLm6SW9e6flfmk99nehXnsNqUmkU\ntG1nfuAPps64kd/Cn6+j2D5PBJcHTwQXjgKfJjNviitDb1NbnRy4frf2Z4+CLRfUpfftuWQWZJq7\nD15sP7aN7ou68PVtPyrnucIQs4rjVjQfB16JDt1SynMkS5DCLLmOXNq+25KMvFKmJ3fCv/ZW5dW4\nO4l4aFqJwdYM1pOZkJ9fYh2GX/z5LIxmCn36AhC9amXJ9R9Q8vhrehO9epVh99Ffrybv6j7Y3n2n\n5E6LBWejxsazIB+mNlsBtLvkFnY+OImRPwznU/kRua5c8/cC2Je5l9ZzG3NdixvUIkJFCUxHVVUk\nVFRVxSQzL5Mu77XlRJ7fkvGGWB1wyw6Yswzi9TmtfdBgor9eHfBNvDgBI6II7n4GEwVlf2gogOHx\nboytR24sZKz7jaoL5hY1mV1zDbanR5AeU8P3LKhQqRmY2nTFac88yr6da3lw6xj25qYY9xOA2MhY\n7jz/bp7v/pJPE1Zl+I6CkjOI8/s0dppSHEKIBOBuIA7tux8BNJNS3hsqIYNBKY6KhceXsWTXZ7gw\nyMfhhwgH3LY3mrc/yStUGB6cDRKxHj5kGC7rD2eDRDLW/+HXdJTQpJ65+2m3U6vnxaaVl7NBQ4iw\nEpGSXGKfT8UREcHRrbugdp0S5q0Sz92Xacxou8NB7HPPEvPB+1iysnBY4Yn+VuZc6MJRykis+9sM\nYurlrxruq8jfUW+UnKbP71NxmP36fAh0RFMesWimq+B+zYqzDs2X8STnzWvKZ7s+CVpptD4C6XNr\ns+iDghJKAzg9EAZJ3rX9iw6qDgexY0dRq+fF1Lq0E7V6XgzDh4MjcISX9fAhrKnm39KtB1OxGigN\nvzidWDN1X4THZFZc6RlcQ+zYUaevoXg7u524Jx/HNudtrFlZWIAoF8xc5uLYZBi4vzp1o+sGHSM5\nf9s8hnx1H5l5wftOFGcPZhVHopTyPmAZ8BlwGXBh2KRSVHgy8zLp+cHFLNg2N7g1GW4trPaRX2Hb\nLIju1Q9Xw0aGh7oaJmlKIAgK2rYje9KUItuM0orw2mvGaUWKy6A7oM3iLyLW13ZXo8YBFaTp1Cge\nBdO9CzEfLjLsKz4f3p9/ku0be3JPKTLOLd37Oa3nNuH/vnsch0slyD4XMes99BQhkEAHKeUGLye5\nIUKICOAdQKBVC3wA7bczH+09ZyvwmJTSJYQYB/RHS9M+XEr5a5DXoThDlDrE1gm3bSu6BsMNRP/0\nPa7qNYgweEkvtNVb3FRd8C4Wp0EpWGsE4MZVr752/MSpRZ3ipUgrUpz8bj2o6mMQDgV5/fr7lyGI\nawgmmKDGT+t4x1KfGrmHmNUFCoJIaeLCyXs7/scXe5by291/UTOmlvnGikqP2RnHt0KIj4HVwFNC\niFlAToA21wNIKbsDzwHT9b+xUsqeaEpkgBCiE3A5Wk3zO4A3g74KRdgpbpYKRmnER8WTfOQBFm9I\nosOR0wv3LEBEcjJRW7dQ0LYdzqQmuCMicCY1wf7Q0EIHb/bk6eTca5ykIOf+B8hY/wcZ6/8ge/L0\nEpFU/kxN1rQUrIcP+bjg06ahmI8W47LZgrLq+DrWjeaDKXKdT4/Bum8v+FjLYfoajh0letlS0zJa\n09NxdbuM11fB0Zdh4J8QGeQEIjP/JOfNa8bTPwxXs49ziGByVbWQUu7xGug/kFIeDNAmUkrpEELc\nB3RHm1U0klK6hRADgN5osxiblHKy3mYT0FtKme6rX4fD6Y6MVJHAZwqHy0Hntzuz+cjmoNu2rduW\nTQ9v0iJxjh6Fjh0hNbXkgU2bwm+/wcmT0KCBoY2fESNg6VJIToakJBgwAKZN8x92a7dDmzawf7/x\nObdtM37bHz4cXnstiCstRkQEGMyQaNwYNm7UrjMhAZ57TrumAwe0fUbXZLfD+edrxxhdw19/af18\n8onxvfWFp23PnrBZe7aZVWBoP/iwLTiDXP8nagt+HfKrWvtx9lC6qCohxHVSyi+FEIbRU1LK/wU6\nsxBiAXATmkN9vpQyUd9+JTAI2Akck1LO1Lf/CAySUvpMmaqiqs4cDpeDKz7shjy+M6h2EZZI7rng\n3iIJ9Kz79pY+qaCHQMkFDfAVTmt/aCjZE6eUbBBkJJURBW3bEbV1S4ntrrg4cgfeQ/b4ST7NSt5y\nJdSsiv2xJ4hZtNAwJYm/8N9AFJ7H4aDG1ZcRtX1r4b7MKnDJYNgZZHleK1YGnn93ha79oX7zps9f\n6qiqi/R/rzD462Xm5LpTvTWav6Oq165qwAkgU/9/8e2KcsbhcnD1Bz2CVhqtjlmRd+4oMXi4atfB\n7WOwN5VbCnxHHfkhe/wk7A8NLWIKY9gwn6vL/ZmG3ICzbt3TpqbBD2Mf/HAJM9uJFd9gf2gorrhq\nRfvOytKc2mOf9uu3KDRbjRiBbfbMEkrDFVet0Mzlqx+f8jdKOm0KBIiM5MSaH7E/MKTQjBYXGcuW\nWfDIhuDMVy5cvLfjf7R5tyW5juAWHSoqD2FbACiEuAfNLPWSECIe+AvYDUySUn6v+0m+07dNBa4B\nGgHLpJQd/PWtZhzhx56byTMzevFBrPlaGREOGPIHvL7aSua6P0rMHvwtpPP59h9KzK7j8DPjcFut\n4HJpzvj+1512xvtIR1Krx0WGazqc9RtgPXTQ57qOjJ83anJefgn880+JY1yxcRz7ayfWY0d9zuKM\ncNliObZxi7ZmxMe1Ww8fKpJ40WO+WtQObQWXSeKj4vnj3u0VznSlfvOmz1+6tOpCiH348fNJKVv4\naf4Z8K5ueooChgM7gHf02h47gE+klE4hxFpgPdoM6DF/MinCi8PlYMxPo1j553scsgWKf9CIcMDt\n22DWci3U0xUXW3L24CcyyBVXjeynDUJjS2GW8kugVOhex/nKmusZoCMOH8I2bw5YI8h+8WXDvq2H\nD2FNM/Y5+FIacHr2ZT18SPPnGLXPziJuzNNkTZluKs9Vofx5uVpGXl+Kw+s6cgfeg232TC18dwnM\nWAktn4DjsaZORWZBJq3nNubGVjerolFnGYGMkL3QHCTPAXvRQmkdwF1AM38NpZTZwO0Guy43OHY8\nMD6ALIowk5mXybWfXsHuE7vMBWq74batRdOEFJJTdND3Z/6x5NixpibDsara8VXM1wEvFXY77DkC\nkXE+FVKRrLmpyeByGQ70MR8sInvsBIASSs5fvQ+fznMg7+re2gBerz40amTsFAeqrFurHe9DyRlh\n2iRIyczB1es2Yv/Ja3m0k53F8n1TCz5duPhs1ycs3b2EB9o+6DdliaLyYDblyO9Syi7Ftm2UUnYO\nm2R+UKaq0FLatRmPbICZK0tudwMuPV1I4aD/9BhqXdHNuGhSXDXcNWpgTUvF1bARrurVDR3L/pzZ\nRQZtz+f4eO3t2rPdK1lhRGoKTjMKyW4n8ue11Bh4m488U5B76+1U2fCLoZLzZZ7znbcKMtb9jqtV\nawASRv4bFiwwFK3QpJXUmNjxY4hZ9F7AglClMgkazPyc/xnOFRHzgnae+0tZcqZQv3nT5y9zrqqN\nwEgp5bf6577AeCnlJSGTMgiU4ggNDpeDZ9eO5P3tC4Ja/R3hgEd/h+mrIdJklpGyRP94KFEOtnjW\n2sSGuGrUwHriONaUlMK3eldSEnl9rwOXC9uct0vKNmiwtgbEB9Yd26l1eVefA73Rdu+IpRJJCK+5\nhuivVxFhYIZyJjUhY+2GwgE6IdqNq2FDw4iqIsf68ae4AVejJPL6XReaWZvuA3KnHuDffWBWF4Ly\nfQxofhOvXDGj3ExX5/JvPsjzl1lxXAgsABqg+SH2A/dIKbeHSMagUIqj7DhcDq7+6DK2Z2wNfLAX\ntlzY8/rp2t5mcSY1IeO7dcROnXR6EK2fiOXk8YCV77zJ+ddAsmbMAoKv3e2KizM8lzsigpx7B2mp\nSowGVbud2m1bGre1WLAY/IaKD+rWf/YBFlxNmmq1OXyFCBdTYgkJ1bA//GjAsF2/oc5WKxnfrcN1\nfpsS11UaH1Lxc2VEQ4sn4IRJ3wcYh2ufKc7V33wpzl+2JIdSyk1SyvbAeUArKWXn8lIairLjWZsR\nrNIAGLKpqNJwxcXhbJSkhaI2SPTZzpqWgvXYUbInTiFj7QYyft7IiUUfm6p8502VdWs1H4UfZ7sv\nLD4UlMXpxPbuO75zV9ls5N5xl/E+Hy9e1rQUrKkppxMTXtGdGnffTuzzY7Hu3kX202O0EOFGjXGj\nKS83EP31qqLJCzEOJy4STov/nFquhkm4mni5JAMlTAxA8XPVyoP0V6Ctj0X4RjjdDuZvm8eVH3VX\nK84rIaYUhxCiqxBiKVrt8U+EED8IIfaHUzBFeMg8cYgeCzoGtzbDDU0yYNh6mPZ10V25A+8h46ff\nyPh5Ixnf/qStRjagiFNWj9xx1a0bdPZb68E0zdEeZNZaMxRZP1GM7Odf0gf6JNzWCG0txKDBuJKS\nDI93V7VR9Z2ZJRMTzptDrW6dqXVFNwDyrr5GS4zodBamYCmRvFCvUuhRuBlrN2gzDe/ZkR4JZkTx\nyoGmEyb6wuBckS7YNFvze0UEoQd2ZuzgvLlNOZ5bysJfinLBbK6qecAStFibN4EU4PNwCaUIDfYC\nO/tO7sVeYMeRn8voqV0RC1uzN8f8iui2B2HnG7BtJvx3FUS4dJt5XDXsgx/W3no9IZy162gpMwwo\nMnh53nh798J6MC2oa/IooGCz1gK4iy3GK06J3FV2++kcUp7B+6ffyFi/kYyffiN78nTNd2LUV9Yp\nYj750HCfpiC0wdrXMYZKLMDiRzMzE3/5rKKXLYVjRw33+TxXg8TCeP1IlxYskTEVWh4x1Q2ghe2e\nk/muvL9flQyzPo5NUsoLhRATgB/0vy1SygvCLaARysfhH4fLwfifx7By75ekZqWSGJuI/WQ6GZF5\nps8X6YDBf8CMr7QBwdkgkYzlX2M9lYm3rb6EnHqKDH8V6YL1TXjjbdcPth/74IfB5fSZabfQLxFM\nOHBmJrU7nmfs/8BPsp8Ax3gipmpf3CH476evok7jxxC9bAnWgz4WHqJFw+VdP8C8E/3YUWpd2YOI\nYi8ADit0HgKbGwQnessarfjqlu/C6jgv99+8USlig+9XectZ6gWAXuQKIWqhJSTsKqX8Vk+brqiA\njP/pWWZvPR09lJKdYv5J+1ibYT1yGGtBQUkHa3H0N/Ps0eN8VqoL1jcBXg5sr7fnwnUGy7/Emprs\nezAsHlHkthjW9/bMioorJM/sACgRymo9djRoP40ZXImNcMXHw549ftebGGKwGNFMunULEHEwzee1\nGlK7DnnXDyjRd6QLNr4D/+4DczqDw+T3b/eJXbSe25h729xfLo7zM0HxZ+Hv+1VRMWuqegWtCuAy\n4B4hxDbg97BJpSg19gI7X/35fqnbP7zJykefllzQF8zCMcCnWaXUvgm3m5xHHiv6FuwxH637jdx/\nDTRslnvHQM2s5OUTyJ40BYYNMzbpZGYSs2ihYV9G5iN/JrNApjF/x7iqx1Ordy9o3Tpo53UJSqGs\n/fl7iuMxW9FYd/Rbrbg5bbo6NlVL2W41Kb4LF/O3zeOqj3qcfaarALVVKovZyqziyEFLdX4K6IJW\nQvaesEmlKDWHj+0nxWo+vNVDpAP+/Qu8/k2M4f7iDtbS4jf6JzYOd4TxRNbVMMm34rLZyPrvG4b2\n/azpb5SUOzISXnyRE+99QMa364o4m+PGPu0zPNiwfocfp3TuHQNPy2Qs+eljvJzunsy6EckHoDTO\n6+JyB0jaaNgmNVkPITbAbse6YxvWHdtLDnQWC6569XG0bl24yZOy5PhUqJFtXu4dGdvpsaAjmScO\nm29UwSl1fZgKhtl54FQp5XIoTCWyKXwiKcpCQqYDWz5kGY//hogj8Osc7Qfuxo7LFgtWK5Yce1Ef\nRSjwkwcq9657oMDh14zkk0AmMg+6fZnVK6l14EBR+3J+PlXW/ujzFB6nfHGKp+Yo7tfJHj0Oa2oK\nVee8TfSa1UWPGTuB2InjtLBetwucTiL2Gw/YZisWGsrtK/WJnrSxZCMXNe66vaiJz+Eg9rlnifng\n/cLQZndcHM4mTYnapoV2e8xdEQeh4II2RMqdhf6k+Hw4OB2SnoSjceZk35tzALGwFXfbz+OlJ38k\nskoQX+wKiL9nEfSsvhwx6xz/AjgKbMCr8p+ZehzhQDnHjcnIyeCS9ztwMv+kuU7d8OBGmLXCeAV4\nzh0DyZo8PaiBytT9LL6aun4i+T17kjVxquZjKL7SuphzvSz4q82R8+DDfjPNuiMjyRk0xLcsZhbU\nFTsmGAe/d9bcUNUk8VU3pIjIekBCsMEITj3tTPH76bDC0L7wbmdwmrV5ALUd0Wx6NJmYyLIpj3L9\nzTsc1Oh9uamUOuU9NpV5ASBwDO1loitB1uNQhJ/Csq7vNjWvNIBHfoU5X/pOG1Jl3boQSVgMj2/i\nu3Xk3vovsEDMRx9Q64puxI4fQ/b4Sf7XLJSWAPZlV3w8Lj+LGC0Oh3+TkZlaId7HBOl7cMdUperM\nGaVauOcrVNdTN8TZMMmn2Sp65QotjHf5MtOyAj5DrSNdMOvPJHZdvoZWQYTtHovMo+27LcjMywxK\njopE7PgxhkqjoG270M3qzwBmf42pUsqxYZVEUSqKZLQ1gxvi8uC+A7V4seEtOBssx3owzTAiyWNz\nNZWKvBTETp1E1Q8XFX4uHl0S6vMGtC9nZpLf8zKqfrDI8BgPfk1GQaTxCDZQwJqdhW3+3MLPQUXj\n+DHlZU+cQs5d92mLEg1mW9a0FCK3b/OZIt4XFjA2gwF5/a8jrkVbtnxSh4tuOsoWk2G7mQWnEPOa\ncM8F91W+qCs/LwrWk5mQnx+aF6QzgNkZx/VCiEAh6YoziMPlYPTakYi5TcwrDWDg3zHsS7+fcdN2\nk/fiK2Ss/gG3zTjJUFhtruUQXeLXMa9fa9bEqbji/BvgDZ2YpUjj4Tciy2L+5+aZEZhaTOZjVuRq\n0tTvvXFc0AZXYkPTMvnCHRGB/YEhhQtHnddexx/vBLfi3Ol2Mn/bPK7+uGeliro6WxzjEJypaqcQ\nYrEQYp7nL5yCKfwz+oenmLPlbdNp0G2WaAYl3sK0Sbtxv/T66cV4M6ZjtRuHuoQqksqIcvkRmUnL\nER9P7kD/AYNGCrVUaTz8yOMrB5YR1uR/qHVlj5IKy+zKZH2WlHd1H8PdeX37aes1+l9vWiafFAur\nzp48HUuN2oUrzs8LwnS1/dg2eiy6uNKYrsy8uFQWzCqOBcCLwFecXjn+Q7iEUvjG4XLw5Hf/Zv72\nd023sbog15nHt3IJE9/siyNfrwVttxO94kvDNj6r8oWI8voRFa45aNrUZ1oOzzHF64V7KKFQyzB7\nMvQ9+MmBZYQnkslbYdXofbn/2Y/djnXX38SOeqrwuOivv6KgbTucSY0N7032+Enk3H6HT1+IGUqE\nVUdGkvHnDgouaEs1h5XNszTfm9myMHszdyPmNakc6UqCyCdW0QlbzfFwcq5GVe1JSQ3On+GDR0+1\nY/xTPxD3f48T88Ei/3WvPX4GHzW1i28L5n76i3AK9wrahNgIjm3d5d8XkZlJ3JinqbJuLdaDaT4j\nvPymNC9+H31Rhmgrs3iUQGG6i+QDxrVEBg0m5+HHje/NsaPU6tWNiFLOCP0+22NHidyymehlS8he\nt5pWt6RxMohU7WbTlZR3VJXZqMHyHptKXY8jQM1xpJTh8ZoG4FxTHA6Xg5f+GMdbv75l2jRldYDN\nYbyeo2lmBL+duJc683zPWvzmberTF4DoVStL5NpJaFBTu59mnMRB/IhCTVDPPdC16IWNjGLzixdm\nMo1+b2yrV+JOTtYLQPUmesmnRGSULpOsM6kJeVf3NlwnE1DmYms4AubgslpxnHcB1lOZWmVHk8/W\nW2E6rHDhQ7A1iMmnFSsDz7+bqZe/6tNxXiF+8yZ+H+UtZ1kURxN/HUsp//HTNgotq25TIBqYCGxH\nq1vuBrYCj0kpXUKIcZczEDwAACAASURBVEB/tHrmw6WUv/o777mmOJ7+/knmb58b+ECdVkfgo0+g\n8yPgMjBGRjhh+6JatN7jewAqTex+zh0Dqfr2TOwjnwmuXngpCwqVhSLPPQTnD9fsyXtmFPv8WGzz\n5pS6L7fFgruqzadPq/A4g/UisS9OCC6hpKcglUERq6IHeu2vW5davXsVUcAOq5bv6p3O4AziXaJm\ndC3+um+n4ZqPyvCbh/KXs8wVAEuDEOIBoIOUcrgQojbaavM/gelSyu+FELOAVcA/wDTgKiAJ+FRK\neZG/vs8VxZGZl8n/ff84X+xZYrpNm4Pw5zuQHwFtHoX9NUse0zQDts6E2IKS+9xA7i23Y39qFK6a\nNUv8kP3hBixxcWCQsuNMmJ+CISGhGukHj5vPghuIMM2eCr+fdju1Lr2QiIMHS92XyxYbUGkAOBs1\nJu+aPkSvWaXdlwaJWE4cx5pt3NYVG4u7Zi0iDqbhLJZpwOf9hZKr0G02LHa74WwmswpcNBj+DqLG\neXxUPH/cu72E6aoi/+a9KW85y0txxAEWKeUpXXH8hjbzaCSldAshBgC90TLu2qSUk/V2m9DyYqX7\n6vtsVxyFtcC3LcCBeYdf+4NaRlLPgr7hfeC1S0seN2w9vLrKuA9XbBzumjU180K9ej5TcAdLqU02\nYcJsSdagCfHsyfP9tO7bS62uFxqWqQVzKdxdsbE+B39vzKwmL3Jui5UT739IzXbnkV4tofC6/c3C\nIPj686VJ1W7Fyo2tbmbqZa8WKpCEhGr8k3aYw/ZD1LPVxxZVMb6TxSnvsalcFIcHIUQ14AvgHWCa\nlDJR334lMAjYCRyTUs7Ut/8IDJJS7vbVp8PhdEdGnp1Z3R0uBxfO6sjW9G2m2xSvnVHYlxVGXANL\nBSRXh6STMEBqVfx8rRYPGxERICW0aHGGT+wDux0uuAD+MbC2Nm0K27ZVGCUHaPK2aQP79wfftlEj\nuOoqWLjQ54I8QLvufv1g+XLj++KLiAgtfLhxY62Q17Rp2mI2X/e3cWNNjpTgsySX1nQVaYlkcIf7\n+O95T/LM32+xdM8KDpw8QOPqjRlw3gCm9a5kiwnPDGVTHEKIVVJK4yBv/+2S0CoFviWlnCeESJFS\nNtL3DQCuAf4GYqSUU/Xtm4BrpJQ+S5GdrTOOjJwMLn6vPZkF5uPSWx2B3+eUTIPujT0KDsZBgyyw\nGZinQLd/x8b6zAxbVircjCPzCO7WrcsWCXUG8P5++sw3dUEbrKcyiUhOLrHP2SBRK+lb1ebTge+K\njeP4F1/hatES6+FDfvN1mSFQ3i+3xQJud5lmsplV4KKhkfxdM7gQ3DpZxgkWH2o/lIk9zowp1V5g\nNzXbKe+xKRS5qmy6EjCNEKIesBoYJaX0LBbcJITopf+/L7AWWAf0EUJYhRCNAas/pXE24lkFft67\nTYNSGm0OwvZZ/pUGaMqixXHfSsODr4JEbrQByG2xBLWi2ZsKF6feoEGlW4xVZM2H1YqzQSL2QYM5\nsWatzxK2edcP0Er6+kv//v/tnXt8VOWZ+L8zk4RkMuESjIGQcPGSV1u01vtdbAsU0KXd/WldWru/\nunhvi0ULVbCErrCiltXW1aqouFWkiIhFQLGLF26ii7qC4lsVhEACKhFCZkKSycz+cWaGuZxz5py5\nB97v58OHzMyZM895z3vOc573uf34agKnnKpllScpex/weLSeGxhnt0fqfhm19k1xDkXTuxO2PNjN\nDRuxnPMBxlV5V25fga8ru70w/AE/09dO5aJnz+a8Z07nomfPZvraqYWff6KDVcVxDPC5EKJZCLFN\nCLFdCLEtyXfuAPoBdwohXhdCvA5MB2YKITYAJcBiKeUmNAWyAXgeuDmVA+mp+Lp8/HL1Dczb/Ejy\njaM4tRnee+xwD/B0CdTUGpaUCNQNoWX1WjrGXma4xq5HWOH4rplYeAXcemIyVrg45JqNtGx4l5bV\na2m//ufQ2Wmp37ilnuRJFMy+LZ/S8sYG9i9YbKgAwnW/jPbjCARsWRuGPUMG1PLgewPZZ7PPhx5N\nbbvY68tuyY+G9dN49IOHaWzbSYAAjW07efSDh2lYn71E22xhdalKNyzXLBw3mxwJS1VhB/iijxfQ\n3t2e/AshjPwZ6WLmsPRddyPeO2ZQecGZuGwU5Qs6teeStKKVskRsVFXu80isojs/zXpWd3baLu2e\ngJUIMSu5K5EcoBU4dzeCw6Hb6z18MRspk66hwyjW6VHiu2YiAO4n5qWU8xFNXcUQ1ly1MWuOcl+X\nj4uePZvGtsTxMvrtQl6qsnp17AHGAh5CFQ6AYcBv05buKMQf8PPdRReyteUjy99x+eHKD+FPy5Mv\nTVklCAQG1tBx+fiYp069G4azcadhmWwjwuvbBdtT2WrzpwIjWc/qpL4ZnZ7kMeiNC+Bs3Hl4jEwa\nckVbbOH9FG16h75XjDf8ycAxVbi+SgykDHgq2P/ya5TPnUOvFctx7tqpOeO7u+n16it0jB5L1zeH\nU/zhFt571H6P8zClrpKsLhnt9e1hd5v+Q1fY2hnWJ/8+NatYtTiWoC07nYC2rHQpsFZKeUV2xdOn\nJ1scrR2tjFp8CdsOfGb5O329sOOBcIe+5GGXVok4TvsfE/uBQXkRoyfMoMNhaQkrbw5ynSS0vDzN\npRCqW1VVwZc79sbcwDOepW6GmXUDMZaJo64O36gx+hZbEgvFKKM9OjS6fOqt+ttMvB66/JQ+txCH\nz0trCVw7Dp47FVsXS5mzjHHHX87dF89NWrbELpm0OKw619MlE87xU4HvoEVI3QNcgJYRrrBIa0cr\nN7x6DeLxIbaURv+DWrvNTFkZ0UQcp/Hold42Wfv2f3O4pd/Leelov5/yO35N/+EnUHnJeVReci79\nh59A+R2/ttT8KKNy2Cy5Hv4et9wS8z3PbybntKqwadXfaJ/L+k3w4YfGTbeS+JS8s+aY+198Pnr9\nTT/5qNfKFbRfcy37Nm2m5Y0N+Je9zrMbBnLDO/aOtT3QzuJPFlH/+BAmv/bzjFog7mI3Y47TP/4x\nw8ZaUgCF5Fy3anGsl1KeL4S4CWiTUv6XEOJ/pZTfyr6IifQkiyOSzPfRU/iDFk9wEMo6YcJm47au\ndgl4Kgj27WtarC8pRmvft06l/xnDk4by5tri0HtCjYQm//QmvNPuzo0cKZYjMfpewOPRHeuMj6/N\nGlxJryWLvhM9q8y0kCSA0xljDZVPn0LJ/HlMHglPnhaq2WbTVO9d0od3fvK/9AuUpr2U6evy8dn+\nT3j4fx/krab1NHubqPHUMmbYWBrOn6WbQxI/ntPXTuXRDxLnQ7ZCidNOABRCPAp0AA8DzwB/ASZI\nKU/NlJB26CmKo7WjldHPj+Cz/Ya5jIbctBH+c6Xx50GXlgAZLHPjbEsuf9jBnZG1/LiL2+yiTpAh\nFz4Ov5/y6VMom/9ERKZIMuRJsLMPDPa6GHX2NTRcMie7iV+pFkA0+V7AU6F7ziP1oTKE3aq/lq+l\nVLLrTcYjYdOQpRLu7X3zGHjoHGs/k0AQrvqkjD89347n2MG2gzz8AT+/XXc7C7c+Q5tfU/blReWM\nO+5yZl90n+mSWPR4prLUlS6ZWKq6EVgkpfwIzSE+EJiQAdmOSFr37+GGJVciHh+SktIAWHEitPV2\nG4faBoPsX7SUfe9vjTXxawcb91Sw0hPbCnH7Me1kB3TX1iWGfWaR8oZpuJ+YF3PDu22kVn7l835a\n4cfPK7p5dOtjWQ+FTLVhldn3HD4v7T+aQHftYC2fwuUiCPR69RXLPcitkLWeKanMQ7OmV3H0WrkC\nOjvZv+oNvrrmZyw/2eptTgcHLKxvp/+v4fpv7aRkXpLmXHE0rJ/GvM2PRJQGgNfvZdHfF3LPO9av\nBzPn+u6DjWza+07W81CiMR1RIcTFQoiL0XwajtDfB9DyLSpzIF+PotXXws/vPZ2T/lzPkj0vWy6B\nrkdjP/jolb8SqBus+3lgUB3+M86C3r3x3jGD/U8vpGX1OlrWvs3+1etoWfM2Les30bJmo/G6c6Yw\ni/2/agIta9/JvgxhdJoq+Yph6Un6m2c78SvVm6/p9wbV0TZnLh0jR+MAHN3dWqhjY2PyroNgvTNg\ngeW6xCdAGuZ3hBVyURHbbp9EYwb83N1F8PiZUDUZvKuW4tz6UdLx83X5WP7ZMsPPV2x7yfLcq3YP\nYJBHfz44HA6u+Ov4nPo8kqnimSb/GrIqWQ8i4vh+YiiLyj/Fn4EyWjW9h1A9eLj5hVtSctjpeukF\n9P3JlZTPnqk9cWbKurCIUWe9trkPJpfB6o3MAnpP6s0erVaXHllP/Er15pvse2DqLNYdyxSc9JaS\nBrNJ9NyIdsa/to5ArX4xi2iFbHbDTYX9Hjj26mZu+MO5NP7Dt3DcOdlw/Pb69tDk3W24r6a2psNz\nL8k1YOZc7w525zyh0FaRw1DBQpeUcn/2REpOofg4tLLnv2D5Z39Nyboo8sPJX8JmnWqfEYdXfEOf\nATV0XnQRbXfdQ/k9s9Kr7mqxq58dLHXWC2MW6pmqdaKzFu4rNi4xn+3ELyD1kut+P1VzZtK9ZKlu\nXo3droNp9QzJdeMhC3PDMDw37niMnMppEwR3J/xz5zf4t1+vTfCV+bp8XLDgTHZ79ZeYaj11rL1y\nA1V3zdI9zkhTtBD+gJ+G9dNYuX0Fuw824nA46A4m3ncyNacz4Rw/DlgIHI8Wm7ADuFJKmV4P0xTJ\nt+Jo7Wjltjd+yYufvkAwxYIf9V/AO/O0Ln3RFWxrAx5Gn3F1QqRFVa8g7f96LSVr38T5xRdaj4QD\n+1OLrtG7KE26+tm5iRdC61i9/RqVmM9lcbuM5HFYyKvRPf/Z6FKoJ2v0uU/jIcR0boTb3654Ceeu\nxlBCYIBAXZ3unI3ccLe9RFNrI3X74XIJOGBZvXbdDTwAX3igsyS1Yx/mrmPeuGc5vu8JMTdsM6V1\n3ak38h8vG1drcD/ykGEex6a973DFX8cTIPHBweVwsX7CprQTCjOROf4IcI+UcjGAEOJKtDLpI9KS\nrAcRDqd74N25rNi2zHpobRwuP1wbVzLk/ldg1mptSaW6Tx9cX4L3XA4vJPr9MOo7lL3//uH9mJT+\nCK/xGmUH62Yez4utlZX1bG8dX0SYXitX4L1jRso3svAySvQT/u+GfJ/24bByx8s0te2irk8dowaP\noeH8HNbQSpaxbed7FjO3w1hx0sf8Rvim37s3ztZWezf/dC3JJHODLn+spREqY9IxcpTuXC061Mns\nQdczve+P6LjiUmpag5GCn3f/9+HK0SXdcOMYmP9t+5nn232NfHfRhZQ6S7l48Ah+P+KPVJdX03D+\nLALBAAs/XkBbl6YEPMUVXHXSBBpOm0avSecbH6fJstUZ1WcxyFOrG2VV46ml2p3dIp1WLY73pJTf\njnvvgyM9HDesLP7z/T/w8rbl+LpTX4O3WzIkJmP2N5NttQxNNczT9r50sGpx2A31TAmdJ95w1u3w\nISfi3d9tuF2q+880puNpZwnMqsUR3mfM03x36Gn+MtObf1jWdC1J07nhdBKoHoBLp/xN5DjQFGWg\n/zGU3xO1DGRipUdXZGgtgRvHwoJT0Iorpciw3sfx6KgnObGfAGBHq1a9YEjvobiLzcPYgy4XDin5\nsrdxy8Ns53VkwuLoEEKcLqV8F0AIcQaQu9ivHLPXu4dbX/8la3e9mZayCBNelopXGEGHg6C7HKc3\ncSJHnrrB8OnLCDOnq9mTp+72SayXVAlHDenmKWSqrLnOk7q72M2wPsfhLnbj9afYOjYbvplUKCrC\ne8cM2n/8Uwx7eoexaKHEW6Php/lwxBYksUAzYEmazo1QH3Q9nLsb8UydTMn6tTh37yLodscoCTMr\nPVBZiaulBdCu02eWwh9Xwgm/hK/LTcU1ZHvrNkYuvkSzQuoOWyGWjrOmFtfAgeA19p2GreWV21fQ\n1LYrJqEw21gNcL4FeF4IsUkI8S5aOO6k7ImVG3xdPrbs3cJ7e9/lvb3vsnH3Bs55+tuc8lQ9q3a8\nnJ7SCIK7A254Gz406Jlx6J+uwNGu/xvhG7Zz7x6ce4yjfgLucrpr6yxHvJiFeepun63eFAUQ6mla\nTiML38so0RFS8RF1BiSNkDK56YcxW0KB1PNWYkgyNwxzhtzllP1lQeS8GFYyqKiIvWaumag73yo7\n4Ivfw8R3tECWVDkUOMSqHS9zylMncsnCc/m4ZasWhpvmNVDkLOKuC+ew5qqNrJ+wiVX/7zX+9ZTr\n6ezOQn2iOCxHVQkhioF6NGUjpZTZl86AdJeqwtmcz370NN7uNAv5x9MN/zBoNL8aeCWnPLGUfstX\n4tS5mLuGn8L+pSupHHEerl06nduizG6zpSXfz67FO+PfbC2ZGC0l6O7fpqPaVrBBqtFGGaCq3EX3\nSSdnNKM7GyVVjMYzGxFSVioAmC0jhh35GRkfk7mRYBWFMCrFkoDLxb7Va6G0LGLBJDvu8PLVouH2\n/R961HnqGDl4JBNX7cP5t5dxdHQA0O4ppXnURbivmcTA6mP5+utEJV1aVMqx7mPZ2bqDppZtDGwv\n4pmWV/nb7tXs9e5hkKeWMceNMyxlYpW0oqqEEJcBH0kptwkhfgD8K/AucJeUMklPueyQruLISnhe\nEOq/hLWLK6js1U8zlcvKcHr1FVN37WA6Lv0Opc8/h9OXuE2Mj8OoZejwU9i/6g37N9lwSY6nnjTs\njxCoraNjrPmath4phWTmwFcQT6qtY3Pim4mWU288s6W8LPi/zPafKR9HvEwJc0NHqXRecAGlixZa\na3k7dChfvv6WpQi1eFpL4Iax8FyGFIhhUKaVulrR39XZPl1fR8olR4QQtwEzgFIhxKlodapeBPoD\n96YsUR5Jls1pF6cfLtsKzffC1ocdVH1x8LCpbKA0AJy7duL+8/wEpRHwVOh2bmPSpMSWoakoDdDW\nxu+eS/tPr9H9OOfZ3jlOVgRSbh2btTIcNsjIcpAeFsp6WFlCyWjSoN7ciKvK27JmI213z7W+BDte\n6wsSSbizUc6kdyc8s8zFvntgwvvacnRaLTgdBv/sfleHlduWZ60qQrK7wtXAeVJKnxDibuCvUsp5\nQggHYL0LUQGRLJvTKnpRUgG3G3SsBz2M5kawT1/NgRh9wy4qgvvvp+VXt2f0ydw7aw4UFxV0B7y0\nMbJmbIazpv29DJLNwIJIKHNM06S4HIlkWGmQlQkrMy74wei8BDwVONp92vwe/X3cgQCVF50dG9gw\nfSaO1gOULlyQ9L7tP/kb9N6ymWeWgm85/L0fzLoQnv8mBDNQNSJTNB3cmbUGUcnuDkEpZVhlXQo8\nBCClDAohMi5MLqh2D6CmfJBhNqcVxBfwdlyUVKC0DMch6y1gjXDuaTKOYko1D8CIHtoBzxIWIp/0\n8j0iitOEVL+XMbKpvOLnRCp5HFFyJszXLEakGZ6XKdNw7vuKQPUALYDgj3+MRNlG5yu13T2XkrVr\ndH2OMcu302fimXILpQsX4O6C076A55ZA60tw7WWwTEB7CZnruJYiA7xOqh2ZbUgVxtTHIYT4H+B7\naC1jtwFDpJTNoR7kL0kpT8mKVEnIt49jaAt8+DCRJCJI3jc5GrMuft0Da2jZ8G7CRZrv/sNWKRQ5\nk62zx8jZE/M48hhYYISVc5+tagGxOzM4LxZ8Q+WzZ+rK137VBNpCJevDCrVy5CW6SsZXfNgKWfoN\nDteuy7Eiuelt+N2M91N+2Ewnj+Nu4P3QdvNCSuNKYDZaocOkCCHOAeZIKUcIIU4A5qPdO7cAN0sp\nA0KIGcA4wA/cIqV828q+U6Xh/Fk4163h6aItWoMXHUo6Q+UHdIausQ809XZy/L5A5GOzORHwVODw\neQlUD6Bj5Ch6vfbfuBoTJxxAoF8llKRY90ChYSWXgIrDb2YyoztX9ERrMRvVAvSUhMF5seIbMrRa\nps+k/K4ZMZZSoG9fXcURbYX4lsGHx8Decs0SWXU8fN4X64kQKXJaE8zdMpjWLPncTBWHlHKxEGI9\ncIyU8oPQ223ARCnl68l2LoSYguYnCS/8zwWmSylfF0L8CRgvhNgBXAKcA9Sh5YiclcrBWKXoUCcP\nLmrl93tgW19ojxuFMj8MaIOzrjMojHcA+o7+EYHX3tDNYA0TdLlo/+k1eKfNiJjKuN1gEg5b/NEW\nyhum5abh0RGKJefxkGrdz3scuVReaVpYtsuemJHCkpcl35CBQo63lFyNO3E1QteJJ1L0ySe6D45B\noKwLzmzWGq6NLh7KoQ9b2e3dzf0j3Dxzoo+2YguLJ0Y713mvphV+IOGBV6Bz4rjsPUwEg8Gs/auv\nr/+n+vr6E+vr698Kvd5dX1/vCP09vr6+/j/r6+t/WV9f/5uo77xXX19fZbbfri5/MC0+/TQYdDqD\nQTD9N2k0QRoS/00aTTD4wx8m34fTqf1WPF1dweBNNwWDLpf+94YODQa93vSOMRt4vdrxFKJsYXrq\n2OYLK+e0qysYnDRJGzunU/t/0iTtfbu/NXRoZs7LpEn6+5k0KfPf83qDwSFD9L/n8SS9j8T8RtR4\nezu9wc0fvxl8+9Rjgm8PJPj2QIJv1hFcUq/9//ZAgutrCF71jwQH/Iqg406CA24h+OMfEGx2a58v\nqdf+/7KM4KfHOIPeXmmcn0QM78FZXQyVUj4vhBga9ZZDShnWlQeBPkBvYF/UNuH3vzTar15SjC2K\nPFQaPHlEc9+r2v/hyrV1B2C8DL0feCFpwlH3oDpaijygs+7r/JfrqPzTn/QfJhobadnySWptOTNE\nuKZTtXsAbkeJ5ae7fPs4kiU3+kaNwevtpspNQfhikpG18bTxxJ4wpp9/Dg88gK+9M8YytuTjGDVG\n38cROi94rbWerVzygm4Zqe4lS2n51e3GT9pTZ1AV2i5mKWrqDN3rFEK5O42N+tdqW5tl10VEtt7H\nRkqJVFeeRvm5V5jO2fOWaH6TcDHGsG91gA/Oaj68Xf8O2P/ci3jPOEs7/q/TC9apqqow/CzXXrTo\n7JwKYD/QSsyCc+T97GESlRJNUUCrXDt7deJJ0zCfMmYRLjmp1ZQCh0tQL2d32y4GeWq5vLkPD8zb\njCt09rJeOTdVTNbQI8uG8ZFPeUg+LAR0KyTrndMM+yUyEZGW1pJXCqHtZteqHYxk8zbMonjtmxR/\n9KHhd91dcPzX5vuPdAXNwTzOsosmgfeEECNCf48B1gDrgNFCCKcQYjDglFJ+lW1B4jvWBTzG2jV8\n0txxefL6vZ8ddNcNTp7wZJJ01HnOOTh3fJ6Rjnh2aVg/jUc/eJjGtp2RrmIPVWzmtpGJ2xrWLcpg\nRz87mBZwDAZpv+Hmw0/TKXTDO2JIVrY86rxlPNkwnLy36jX2L1pKy6rX7CWa+v2UPfIgOPQf2iw/\ndEUnFiabrybXatDjsSa3mWydnTgPpm9V5rKdb64tjluBx4QQJcBWYLGUslsIsQbYgKbIbs6JJKEJ\n7P6P+7RlIYeD/iPO1y3/YUS49zOQUsx7zNPX7kaCZW7oOETp4kWULl5E0OPh0FU/xvu7f0/tGG3i\n6/Kxcpv+DeVFoVle0coz4QnK74dbbqFyyQt5qRprasUNqou9aG+7zdoT9xGInSf2jFvGaeZxlDdM\nM20xYOnmGVIUCWXXTWQxspQIBBJ62ViWLWTtcuiQ4fkwCt3vGn4KzgOt+ckjIgeKQ0r5OXBu6O+/\no0VQxW/TQL56mJeUUPb4I/RathSHgdIwOnnREyFyofU/xvpvR0VweKZOpuwvC2I+drS1aZPS6YRH\nHrK+3xTZ69vD7jb9CdzYR1uuizaX428c5Q3T4NGHdZOrcnIztpoY5/PB0qW6u0i3iVRPwJYyyHCy\noeUlMj3sLEWa1Lhi1Uoqd+5MLLtuJotR+LPfD05nrEIZ/X0Aer3ysv6NPV55Dqwh6Hbj0PGXBmoH\n0zFyNL3+tipxX52deVtmPULqSqSIzwdTfol7/nzTzYLuctp/NEH/5GWIkrVvGn7Wa8VL4LsvY79l\nRLV7gGFXsboDmo8nmvibcUZj9FP0PVhZQ3fu3QMGeTTOpl0catpOc1WZFhiQzV7k+cKmMshYpnya\nc8TSUiSaM1/PiohWWg7QvVEnlSU+/Nkkn8Y7fabuHE5QniZ9Qjq+Pwbv7Hvxxl8PefbNHZ2KI6zx\nly8Di02NvFNut12+3CrOvXtwNhnXz3I2NUFzsxaNkUXcxW7GHDdON6t+bPEp9BrQStDkZmy6/LFj\ne6SMtenYpVuSwkJiXKB6AAwerEUHRf+0E279oZsX1l/J7rbdGStPXYjYUgYZSjZM2akd1cY22VKk\noUXj76LXq6usyZlK8zKD9r4J71noeWK6/9ZWPL+4gZJ1a3A27T58fUSVVcmFIrHcj6OQSLfkiJ1+\nFBCqUzOwho7Lxye/gaXyJODzUXnBmYZPHt21dbjkx3xp0g0sU0SiqnS6ihUdMjGNTco5BDwVBPv2\njZ3oBuOYtCRFhp60qmbdCQ88EPPeLaPhgfMSt81UK85UyHp4cwafXJPKmmSO7Ht/K/SOqq2k8xAR\n6NOH4i2bE3d93Y1475hhXFJkYA3OvXsslV3PRl+VMFZ6nujKUqKFxJcu+LNuCkDA48Hh82XUr5hy\nWfUjkhQ0vgNwNTeZd3lLJ0rH7aZj3OWGH3eMvSx30RJxXcXWXLWRuy6coz1xm5U/N4k8cbYdxLWr\nMXm3PLOljBXLKZ96a+aioO67L6b098Ghdbxwln6EzMrtK7JWnjrv5LKkfZI5Un5PrLWj12mxeMtm\nuoafoluy3dSiCStHC2QzOsluB86w9RMeC6O8MWdbW067UR51isNuz+14jEJQ020n6m2YhW/i9drT\nOSErx1OBb+L1OY2WCBPdm9sqMX1DXC66B9URMAhXjIxjVCik6YW/ayfuJx/LXLvWuJ4OW//6HLuK\n9JVDU9su9vpS7HGhiME7ZVryOQGmDxHOA620vLgiIZzXtFfKoDo6vq+vtAKeivT7hljFRu8PCAUr\nuMspXfBnWz+TaD5kLgAADxBJREFUrMVvuhx1isOuxo9HN3bdRly8IUVFeGffy74tn9DyxgZa3niL\nfVs+wTv73p7TGyOcXBW6Ge9f8BwOg2N37m7EM3VyjAVR9siDBAYN0t+3S7/RQdoXSOiJu7r/UAZ5\n9OdFjaeWand+EjKPNJz7vjKeE1HXlulDROMOKseNpO8V46kcNeKw5Zmkh7d31hx8P7sWamtjFMW+\n97dGGkLlonmZXqOrruH6hcY7xozFM2uGtZa4UaTV0MvK/rO250LFpsaPRy92PaNJUm43gZO/SeDk\nb/TckNDQzTgwZKihkg66yyn7y4JYC+KJeeA1UAI6LW4hcxdIODBAjzHDxh6Z0VV5wGoHRbPtwkvH\nepanYffByVPx/Orn9Hr1Zdi9m8Cxx9IxcqRmXfTubW25LlOJrTodDPeveiNBbiZNwjtlGiVrjCMu\njch29YmjT3EQO7lwOgkaPM3qkbD+malM1iMRUyWtH9/gatmX8F7XN75JoK5Od/tMjm/D+bO47tQb\nqasYgsvhoq5iCNedeiMN5+d+qfCIJYlVEF0a3c4DXsTyjL4pr17L/vnPQJef/mcO1x5UdjVCMIir\nuRn3E/OsLXVmq8pAtH8pXpmseg1+8Qucu3fhNKnAHTCwjrKdRd5D1kAyTFR4YZW/jfZZc3A/+Zjp\nV4xqHWUkk/UIRi/ss/OCCyhdtNDyPpwH2+j43vd1z1EmxzccGHDHOTMOF3hUlkbGsRoKnFBZobQM\nh8+rm4wbE0Lr91M+e6YWjdW407SinJX8kbSSFu0STkheuRx276KvWXKgu5x96/6H8of/kPNulEdl\nOG40VVUVfNn8tRb2t+xFnM1N+lUwnU5aNrwbG5dtEl4YUTSzMrNmmu+qs1YxlDM67BMMx02PoMtF\ny5sbKXvq8Yx1vOvx41mA2O6qaDUU2OfTrawQTXQIrZ1w+6DLRcv6TcY5Gxa6Blpa4rIY8mxH9piu\niVlICFThuMkIm4mr1xIYWKO7SUKtI2wW1TvaiTbLU4ksGVSbsC6cC0emwiZ2lnVshAKXrF9r+nnE\n8rQZbp9sqTMt/6XdJS4T2QOeCrpr64yjv3IZVs3RulRlRP9j6Lh8vOVSDLaK6ili0FuuCPTprZvc\nFb/2nbd2rYqkZGNZx+zmHQQO/WhCZD7ZDbdPttSZTpFHu2NhJruj3cfXy1fpV1/IQ/kRZXHEYRiV\nobdmaNXR10PxdfnYfmBbeslvPh/OrR/i3PpRbDSKxciSrMfVKzJHJsLSdUiWn9E2Z27E8rQabh/w\nVFibW6le4ymMRdKIsyHDYq2KPLYGUBZHPDbr8mSsAFwBodfMyXbNJr+f8t/eTunCZyKOvZgy8eEl\npjgLIhM1kRR5ork5cz3FozEryjgurqqCybZBHJr18L3vsO/OWbHlTUxI5RpPqS6XzeKTOXXax6Gc\n41UVfLljb/o3qiybi7l0kk5fO1W30KGVmk1hOc2cfDFOvTzRU5zOPUVOgKpyF90nnZyeI9mISN0q\nC8ERetuOHEn7xBsJ1Ayiaki1/TH1+bRCnTgIDBlqySGeklM9SnZX0y66jY4zE077JJg5x49uxeH3\nUzVnJt15ajxkh1zdQHxdPi569mz90uoVQ1hz1UbTENWwIk5WtLFl7Tt5tSZ6yg25p8gJmqy+628y\nL1KZLnYe0Ay2tTWmaVRrTlqwM4nsVf42vizy6B6nWbHEpJFiFjFTHIV1d8wxeW88VICYNXMK12wa\n1sd8QlopE5/ysoWioMn60q2d4IgMBFKksxxkOBZTpmkdCM2Un9sNVdVgoOAy3pnRJkevczxLjrye\nTriZkx5WazYFqgcQqDGoOQUEampUxNmRik7QQ48Nm073HhE/Fq+tA6Dy0vPTd2bnOTDnqFUcGa0v\ndQThdpRweXMf3c8s12wqoDLxVrESQZaRKLOjhSznFeTiXGTsHuF2E6gegOc3kyMVtNtdAbYe2sm2\nFx7GMXOKPcFCNbO8U6blLQqxIB4DhBBO4CHgW0AHMFFK+Wk2fzPfpl6hUt4wjQfmbaZ4JLwotF7j\ndQe0DoDTbdRs8jbMgkCA0oULcLRp5nbQU8GhqyYUVMSZlQiyjESZKTJCLs9FRu4R4ejCZ5/G4fXi\nd8LkkTD/NDhYqm1S0fU4P1od5Hcj7jU/BgN/S8tr63La/Q8KRHEAPwBKpZTnCSHOBX4PjM/qL9oM\nfTsqCJnmrgDc/wrMXg3NHq3XeK8BrbT8ohPcFqdMqEy8d/pMe9EoOaZh/bSYCLLGtp2R1+EIMivb\nKHJDTs9FBu4R5Q3TcM97JPL6tpHwx7gukwdLgsz7+HGcJSWmx5DP8Nt4CmWp6kLgZQAp5VvAmbn4\n0YTGQ0d5wlm8ae7uguO/1v5PefmugMvE+7p8rNymv4Yd7vpnZRtFbsjHubCVEByPz0ev5csOvyyG\nJScZb75i20vGx1BgPtlCsTh6AweiXncLIYqklLpeo3793BQVWS+Fbsr99+OaPRuam3ENHIjb7aaw\nbm+HqaqqyO4PlJ8IgwfD558nfOSoq6P/8BMt3fyzLmeG8Je2mUaQ+Uu1xMVk21RVVmdNRug54wnZ\nlfWzli8ydi5syfnIQ9qN2e494rMvICq6sNkDu/Tdh9rn3qaEY4jI+dkXYOBvcTXtosrfpkVh5YhC\nURytQPSZdBopDYCvv86cdq2qquBLbzf0Pha83eAtzJj5XMXzl48aox97PmoMXgvj01PyDqqqKig6\n5GGQp1Y3Z6XGU0vRIa3FabJtsnm8PWU8IfuyFnUlP19Wfj9lOe3eI4o8VNYMiuQzDWzT/IU7++lv\nPrC8JuYYYuQs8lBp4G/prqmlpchjGLqbKmbKtVCWqtYBYwFCPo7ESneKnJCWad7DsNL1T3UGLBx6\n3LmIiy50d8EPPzbefOxxlxkfQ4HVxSsUi+MFYKQQYj1aZ8if5Vmeoxebtbp6OuHufiu3r6CpbRc1\nnlrGDBsb0/XPyjaK3NDTzkV8dOG9r0J3cRFPfRsOurRFFU9xBVedNCHpMRRSXbyju+QIPWcpQMmZ\nWeLl9HX5knb9s7JNtuUsZHIpazrnIi9jGlfrylcMO1q110N6D9U9BktN0bL4UKdKjigUSXAXu5OW\nUrGyjSI39LhzEYoujLwETu7/TePtk+0rz+V6CsXHoVAoFIoeglIcCoVCobCFUhwKhUKhsIVSHAqF\nQqGwhVIcCoVCobCFUhwKhUKhsIVSHAqFQqGwhVIcCoVCobBFj8wcVygUCkX+UBaHQqFQKGyhFIdC\noVAobKEUh0KhUChsoRSHQqFQKGyhFIdCoVAobKEUh0KhUChsoRSHQqFQKGxxVDZyEkI4gYeAbwEd\nwEQp5af5lSoWIcR7wIHQy+3AI8ADgB9YJaWcmS/ZAIQQ5wBzpJQjhBAnAPOBILAFuFlKGRBCzADG\nocl8i5Ty7TzLeTqwDPgk9PHDUsq/5FNOIUQx8AQwFOgF3AV8RAGOp4Gsuyi8MXUBjwEC6EZrRe2g\nwMbUQM4+FNh46nFUKg7gB0CplPI8IcS5wO+B8XmWKYIQohRASjki6r33gX8CtgHLhRCnSynfzZN8\nU4CrAW/orbnAdCnl60KIPwHjhRA7gEuAc4A64HngrDzLeTowV0r5+6htTs+znD8B9kkprxZC9Afe\nA96nAMfTQNbfUXhjejmAlPICIcQItPnpoPDGVE/OZRTeeCZwtC5VXQi8DCClfAs4M7/iJPAtwC2E\nWCWEWC2EuBjoJaX8TEoZBF4BvptH+T4D/jHq9RnAG6G/VwLfQxvjVVLKoJRyJ1AkhKjKrZi6co4T\nQrwphHhcCFFRAHI+B9wZ9dpP4Y6nkawFNaZSyqXAdaGXQ4C9FOCYmshZUOOpx9GqOHpzeBkIoFsI\nUUjWlw+4DxgN3AA8GXovzEE0kzYvSCmfB7qi3nKEFBocli1+jHMus46cbwO/llJejGa5zSDPckop\n26SUB0M3iMXAdAp3PPVkLbgxDcnqF0I8BfwxJGuhjmm8nAU5nvEcrYqjFaiIeu2UUvrzJYwOfwee\nDj1h/B1t0lRGfV4B7M+LZPoEov4OyxY/xoUg8wtSyk3hv4FvUwByCiHqgNeAP0spF1DA46kja0GO\nKYCU8l+AejQ/QpmOPIUo56pCHc9ojlbFsQ4YCxDycWzOrzgJXIPmd0EIUQO4Aa8Q4nghhAPNElmT\nR/nieS+0RgswBk22dcBoIYRTCDEYTTl/lS8BQ7wihDg79Pd3gU3kWU4hRDWwCpgqpXwi9HZBjqeB\nrIU4plcLIW4PvfShKeL/KbQxNZBzSaGNpx6FtDyTS14ARgoh1qM5zX6WZ3nieRyYL4RYixYFcg3a\npHoGcKE9lWzMo3zx3Ao8JoQoAbYCi6WU3UKINcAGtAeUm/MpYIgbgQeFEJ3AHuA6KWVrnuW8A+gH\n3CmECPsPJgF/KMDx1JN1MnB/gY3pEuBJIcSbQDFwC9o4Ftoc1ZOzkcKbowmosuoKhUKhsMXRulSl\nUCgUihRRikOhUCgUtlCKQ6FQKBS2UIpDoVAoFLZQikOhUCgUtjhaw3EVirQQQgxFS9T8KPRWGbAe\n+I2Ucm++5FIocoGyOBSK1GmSUp4mpTwNOAkt7n5xnmVSKLKOsjgUigwgpQyGSl/vFUKcCvwCGA5U\nAx8A/4yWQOeUUk4DEELMRyu45wKmoJXW3g78REp5KOcHoVBYRFkcCkWGkFJ2ovVR+AHQKaU8DzgB\n6ItW4uZJYIIQwiGEcAPfAV5E62sxSkp5BpriOCkf8isUVlGKQ6HILEG0PhUPCSFuRmu+dSLgkVJu\nAz4HLkbrrbI8ZFksA9YJIe4BnpdSvp8XyRUKiyjFoVBkiFAdJAEch1ZXzIdmZbyJVhMNtA56E0L/\n5gNIKSehKZKvgaeFED/JqeAKhU2U4lAoMkCoHfFM4C3geGCRlPJJtPLXl6L5MUBznn8XGCCl3CiE\nKBJCfAJ8JaX8d+C/0EppKxQFi3KOKxSpUxNq6QuaYngPzQleCywQQvwz0IlWFnsYgJSyXQixgVAp\n/1Ajn98Crwoh2oEvgP+f06NQKGyiquMqFDki1EulAq089nellHvyLJJCkRJqqUqhyB1noTnHH1VK\nQ9GTURaHQqFQKGyhLA6FQqFQ2EIpDoVCoVDYQikOhUKhUNhCKQ6FQqFQ2EIpDoVCoVDY4v8AcbSF\nCt5zOZ0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig = plt.figure()\n",
    "\n",
    "axes1 = fig.add_axes([0.1,0.1,0.8,0.8])\n",
    "#axes2 = fig.add_axes([0.1,0.1,0.8,0.8])\n",
    "\n",
    "axes1.scatter(j_day,dw_solar_everyday,label='Observed dw_solar',color='red')\n",
    "axes1.scatter(j_day, ghi_everyday, label='Clear Sky GHI',color='green')\n",
    "\n",
    "axes1.set_xlabel('Days')\n",
    "axes1.set_ylabel('Solar Irradiance (Watts /m^2)')\n",
    "axes1.set_title('Solar Irradiance - Test Year 2009')\n",
    "axes1.legend(loc='best')\n",
    "\n",
    "fig.savefig('Figure2.png', bbox_inches = 'tight')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\s269596\\AppData\\Local\\Continuum\\Anaconda3\\lib\\site-packages\\matplotlib\\axes\\_axes.py:6462: UserWarning: The 'normed' kwarg is deprecated, and has been replaced by the 'density' kwarg.\n",
      "  warnings.warn(\"The 'normed' kwarg is deprecated, and has been \"\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAawAAAGpCAYAAADRBQIfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3Xl8XGd18PHfvbNptMvavGjzer3v\nju3EieMkZIc0pYHyQkuhhUADCSUhhZSUFtqyNJQWSgOBFwIvO4Q0m5M4mxM7sR3vsp34epcsW9a+\njKTRbPe+f1zNaEa7LI1mRjrfzyexNeszsjRnnuc55zyKaZoIIYQQyU5N9ACEEEKIkZCAJYQQIiVI\nwBJCCJESJGAJIYRICRKwhBBCpAQJWEIIIVKCPdEDuBwNDZ6UysXPy0unpaUr0cOYMPJ6J7ep9Hon\n+rUWFmYpE/ZkKUhmWBPAbrcleggTSl7v5DaVXu9Ueq2pICVnWGL0th+6MOLbXrtyVhxHIoQQl0dm\nWEIIIVKCBCwhhBApQQKWEEKIlCABSwghREqQgCWEECIlSMASQgiREiRgCSGESAkSsIQQQqQECVhC\nCCFSggQsIYQQKUFaM01BhmHS0OqlvsVLU3s3Xl8Qf8BAUcBhVzlyuonivHSK8twU5bkpzksnL9uF\nqgzcl7Nv26eszDQ8Hd0D3lbaPgkhLpcErCnE6wuiV7dysqYVry8UuVxRwGm3YWIS6DRoaO0fbFwO\nGzML0pmZn8HMwgzrz4IM8nPSJvIlCCGmMAlYU0AwZHD0bDOVpxoJhkwcdpUFpTnMyM+gMNeN22VD\n6Zk9mabJmgVF1Ld6qW/por7Fy6XmLi42dnG+voOztZ6Yx3Y6VLLcDnIyXeRmOsnJdDELBcU0I48p\nhBDjQQLWJFfT0MEPnjrGxcZOXA4bqxfkM3dWDg77wNuXiqKQneEkO8PJvFk5MdeFDGv2dbGxkwuN\nndT2/HmhsZOmdl/MbW2qQmGum5mFGZQUZJCT6ZQAJoQYEwlYk9iOyov8ctsJ/EGDBaU5rJpfiMt5\n+ef72FSV6dPSmT4tndULCiOXv3qwho6uAK0dPto6/HT6QjS0dHGp2frvgN5AepqdksJMZuZnML8k\nR4KXEGLUJGBNQoZp8rtXT7Ft73nSXXY++b4ltHf54/Z8atSsjOLepAuvL2jNxho6udjUyYnzrXzj\nlwfIdDuYMzObOTOzrfsMQhI0hBDRJGBNMoFgiB89+y77jtczIz+dz921gsJc96gOcBwvbpedubNy\nmDsrB8MwudTcxZmL7VTXeag83UTl6SaK89xoZbmUFWehqjLrEkIMTgLWJBIIGnzvj0c4eqaZBaW5\nfPb9y8hIcyR6WACoqsLMAiuzMBAs5nx9B6cvtFHb1EVdixe3q575JbksKM0hPUnGLIRILhKwJolg\nyODR/z3K0TPNLJ+bzz13Lhs0sSLRHHY1siTY3ulHr27l1IU2Kk83ceRME7NnZLNk9rRED1MIkWQk\nYE0CIcPgsWfe4dCpRhZX5HHPnUuTNlj1lZ3hZN2iIlbOL+BsbTvvnmvhzMV2zlxsp+qSh5uvKEMr\ny5UkDSGEBKxUtv3QBQzT5M3KWs7WeijOc7NyfgFvHr2U6KGNmlUblsv8khxqGjo5drY5ss81vySH\nP7l6DovK8xI9TCFEAknASmGmabL7WB1naz0U5qZx3ZoS7LbUmFkNRlEUSosyKS3KpKQwk+feOsfh\n0038+68PsrAslzs2zUYrk8AlxFQkAStFmabJ2+/Wc6qmjfxsF9evKUmZZcCRmjcrh/vuWsHZ2nb+\nd8dZjpxp4vivDrKkIo8PXDef0qLMRA9RCDGBJGClINM0+e2rp9CrW8nLcnH92lKcjssvCO4rESnw\nQ5k9I5u/+8AKTl9o48kdZzh2roV/+unbXLNiJndePWfIWi4hxOQhASsFPbnjDNv2nicnw8kNa0tI\nG0P3ilQyd1YOD/z5KipPN/HbV0/y+qGLvP1uHbdvrOA960pTfjlUCDE0CVgp5pk3z/LsW1UU5bm5\nZsVM3K6p90+4fG4+iyvyeP3QRf53xxl+v/00bx27xMdvXcTsGdmJHp4QIk6m3rtdCnt+TxVP7jhL\nQU4aD35oFZVnmhI9pLgabmnSZlO4/aoKDp5o4MT5Nv7lZ/tYVJHHyvkFw862pO2TEKlH1lBSxMv7\nzvP7106Tl+XiCx9axbRsOYcKrHO6NiyZzo3rSslMd/DOuRaeefMcdc1diR6aEGKcScBKAa8fusCv\nXj5JToaTL3xoFYW57kQPKelMz0/nvVdVsLgij46uANvePs/hU40YppnooQkhxoksCSaZvstgpy+0\n8eaRS7gcNjavmsnx6haOV7ckaHTJzW5TWbuwiPLiLN44fJHDp5qoa/Zy9YoZU3KvT4jJRmZYSexs\nbTtvHbmE06HynnUl5Ga6Ej2klFCY5+b2KysoKcrkUnMXz7x5jouNnYkelhBijCRgJalzlzzsrKzF\nble5YW2p7FmNkstpY8uqmaxbWIQ/EOLlfTUcOd2EKUuEQqQsCVhJ6NwlDzsOX8SuqtywtoSCHAlW\nl0NRFBZV5HHzhjLS0+wcPNnIjsO1BENGoocmhLgMErCSTEywWlciCRbjoCDHzW0byynMTePcJQ8v\n7qmmub070cMSQoySBKwksu94fczMSoLV+HG77Nx4RSnzZuXQ1O7jqz/bx6kLbYkelhBiFCRgJYld\nxy7xg6eO9QarPAlW482mqmxcWsy6hUV4uvx861cHePNIbaKHJYQYIQlYSWDb29X86Jl3cDltXC/B\nKq7C+1qf/8BKnHYb//e5d/nNKycJGbKvJUSyk4CVQKZp8vvtp/jNq6fIyXTypQ+vpkiC1YRYMnsa\nD390LTPy09m29zz/+ftKOrsDiR6WEGIIUk2ZICHD4Gcv6OysrKU4z839H1xJQa6bUxdlX2UihAu0\nN6+cyY7KWo6dbeahx3azZfWsfvVu0ndQiOQgM6wE8PqCfO+JI+ysrKV8ehZf+sgaCiTBIiGcDhtb\nVs9i6ZxpeLoCPL+rmpr6jkQPSwgxAAlYE6yuuYt/+fk+Kk83sWT2NB780Co5gDDBVEVh9YJCrl4x\nA8M0efXABQ6dbMQwpMhYiGQiS4ITaL9ez0+2HsfrC3LjulLu2jIXmyqfGZLF7BnZZKc72X7wApWn\nm6hr7mLTihmJHpYQoocErDjbfugCbreT7fvPo1e3YlMVrlo2nen56eyolJTqZJOfk8Z7r6rgraOX\nqK7r4Nk3q5hVkMnKeQWJHpoQU558vI+zhlYvv335BHp1K7mZTm67spy5s3ISPSwxBKfDxuaVM1m/\nuIhAyOC7f6jkVy+dwOcPJXpoQkxpMsOKkw5vgKd2nOXVAzWYwKLyPFYtGP4kXJEcFEVBK8ujKM/N\nvuMNvLy/hoMnG/mLmzSWz81P9PCEmJIkYI2zQNDgjcMXeWrnWTq8AbLTHVy/rowst3yrU1FeVhpf\n+at1PPPWOV7YU81//v4wy+fmc9eWecwqyEj08ISYUuRddJz4/CHeOlrLc7uraG734XLauGvLXJwO\nG7nZbjwd0mw1VTkdNt6/eS5XLCrm1y+foPJ0E0fONHHFomJu3VBOYWFWoocoxJQgAWuMauo7eOvo\nJXZUXqSzO4jDrnLjulJu3VBOdoaz3wnCInWVFmXyhQ+t4vCpJv74xmn2vFPHnnfqWDa3gPWLClmz\noAiX05boYQoxaUnAGqVgyODMxXaOnGni0KlGLjRYJ9lmpTt475UVA3ZKEJOHoiisnF/Ainn5HDnT\nxAt7qjlyupEjpxt53KazuCKPpbOnMb8kl5KiDClbEGIcScAagGmaeH0hGtu8NLR6qW/10tDi5Xx9\nB1V1HZEDAO02hVXzC9i4ZDor5uXjsMun66lCURSWzy1g+dwCgorKs2+c4sCJBipPN1F5ugkAp0Ol\nOC+d4mnpTJ/mpjgvnbwsF5luB5luB1npDvmZGSXDMAkEDQIhw/ozGOrzde9/hmliU1XsNgW7zfrT\nZlOxqQoOu4rTruKw23A6rL/bbSqKoiT6JYohTJmA1drh44+vn6HLF8QwTIKGQShkYhgmIcPEHwzh\n9QXp6g7i9YUwBjhK3aYqlBRmMndWNktn57OwPJc055T5FopBzCjI4E+unsOfXD2HZ986x6XmLupb\nvTS1dVPb1Mn5IVo99b5xqjjtNhyO3jdSu01BURQUxerGoUDv16p1nar0XDbAYw/Wp8Mc4Gd7MAPd\n1Omy4/MFR3bjwcZhQijq9zBkmIRCBkHD7PnaujwSiEIGwaBBKI7dRxSsfw+HXcXpsOGwq6SnOVDA\nCmh2FUUBhd5/E6D338f6B0JVIDfTJY0B4kAZzQ+vEEIIkSgS/oUQQqQECVhCCCFSggQsIYQQKUEC\nlhBCiJQgAUsIIURKkIAlhBAiJUjAEkIIkRIkYAkhhEgJErCEEEKkBAlYQgghUkJKNsJraPCkVD+p\nvLx0Wlq6Ej2MCSOvd3KbSq93ol9rYWHWiLvvptr74EgN9T2QGdYEsE+xjtzyeie3qfR6p9JrTQUS\nsIQQQqQECVhCCCFSggQsIYQQKUEClhBCiJQgAUsIIURKkIAlhBAiJUjAEkIIkRJSsnBYCJHcfvKT\nx9i1ayc2m5177/08ixcvjbn+N7/5Bc8++zS5ubkAPPjgQ5SVVcRtPL/97S95+eVtAGzceBUf//gn\nY67X9eM88si/4XA4mT9/Affd9wCqOvzn+WAwyNe//s/U1tYSCPj56Ef/mk2bNkeu37btBZ544rf8\n8Ic/Hd8XNEVJwBJCjCtdP86hQwd47LGfUVdXx5e//CA//vHPY25z4oTOl7/8zyxcuCju47lwoYZt\n217gscceR1EU7rnnb7jmmi3Mmzc/cptvfetf+dznHmDZshU89tj/8NJLL3DTTbcO+9gvvriV7Oxc\nHn74a7S1tfKxj304ErBOntR57rmnMM1J2ZAiISRgCZEEtm59hh07Xqerq5PW1lY+9rG/4dprr+fg\nwf089tj/YLPZmDlzFg8++A/4fN184xv/QkeHh7a2Vt773ju5884/4zOf+SS5uXl4PB7uv/9B/u3f\nvordbsdms/HlL/8zhYVFfO9736Gy8hAA73nPzXzgAx/iX//1n3A4HFy6VEtTUyMPPfRPaNpC3v/+\n2ykvr6C8fDb33Xd/ZKx33303ra3tka8rKubwwANfjHxdWXmIdes2oCgK06dPJxQK0tLSQl5eXuQ2\nuv4uv/jFT2lqauLKKzfxF3/xMU6e1Nm69dmY5wK46647WLx4CRcv1jB79ly++MWHY2Y/3/jG16ip\nOR/5Ojs7h3/7t3+PfF1cPJ1vf/t72GxW14pgMIjT6Yx5joaGepYtWwHAsmUr2LnzdW666Va+/e1v\ns2vXbgzD5IMf/DDXXXdDzP22bLmBLVuuj3xts1lvqW1trTz66H9z7733881v/suQ//Zi5CRgCZEk\nvN4uvvOd79Pa2sInPvFRNm3azDe/+a88+uiPycubxo9+9Chbtz6Dpi3ihhtuZPPm62hsbOAzn/kk\nd975Z4AVhDZv3sITT/wOTVvIZz/7eQ4fPojH086JEzq1tRd57LHHCYVCfPrTf82aNesAmD59Bg8+\n+A88/fSTPP30H/nCFx6ivr6On/zkF+Tk5MaM84c//CENDZ5BX0dnZ0fMfdLTM+js7IgJWNdffyN/\n+qcfICMjg4ceeoA339zBVVddzX33af0er6Ghjk984r8pKSnl4Ye/yI4d29m8+brI9V/84sNDfl/t\ndju5ubmYpsn3v/9fzJ+vUVZWHnObmTNncfDgflatWsObb+6gu9vLrl1vUlNTw6OP/gSfz8fdd3+M\ndevWk5WVFfXa0gHo6urky1/+ez7xiU8TCoX4xje+xr33fh6XyzXk2MToSMASIkmsXLkaVVWZNi2f\nrKxsGhsbaGpq5OGHrdmLz+fjiis2cOWVm/jd737F66+/Rnp6BsFgMPIY4Tfi22+/g1/+8mfcf/9n\nycjI5O6776Gq6iwrVqxEURTsdjtLlizj3LkzAMyfbwWKoqJijhw5DEBOTm6/YAXDz7AyMjLp6uqM\nfN3V1UlmZu+bvGmafOAD/4fMzEwANm7cxMmTOldddfWA35fi4umUlJQCsGzZcqqrq2KuH26GFf7e\nff3rXyU9PZ377/8ifT300D/yn//5bX71q5+zcOFinE4HZ86c4tixY3zmM9Z+VzAY5NKlWr70JWsG\nuG7dej760b+mru4SDz30Be6888+48cabeeedo5w/f55HHvk6fr+fc+fO8l//9e1+M0cxehKwhEgS\nun4cgObmJjo7OyksLKKoqIhvfOM/yMzMZOfO13G70/n1r/8fS5cu5847/4wDB/axa9fOyGOEl8p2\n7nydFStW8fGPf5KXXnqBX/7yZ2zefB1btz7NBz/4YYLBIEePVnLLLbcDb6Eo/RtkD5Z0MNwMa9my\nFTz66Hf50If+gvr6egzDjCRXAHR2dvKXf/lBfvGL3+N2uzlwYC+33fa+QR+vocEK3Pn5BVRWHubm\nm2P3loabYZmmyZe+dD+rV6/lIx/5qwFv89ZbO3nooX+koKCQ73znW2zYcCWhUIj169dz770PYhgG\njz/+Y2bNmsV///djkfs1Nzfx+c9/hr/7uwdZu/YKABYvXsovfvE7AGprL/KVrzw0omAle13Dk4Al\nRJJobm7ivvs+TUdHB/ff//fYbDbuu+8BvvCF+zBNk/T0DB5++J9RFIVHHvk627Y9T05ODjabDb/f\nH/NYCxcu5qtffRibzYaqqnz2s59H0xZy8OB+7r77YwQCAa677gY0beG4v46FCxexfPlK7r77Y5im\nyec///eAlTHn9XZxxx1/yic/+bfce++ncDgcrF17RWSWNdAeltPp4Dvf+RZ1dXUsWbKMq666ZlTj\neeON7Rw6dAC/38/u3W8B8KlPfYaMjEyeeOJ3PPDAFykpKeOBB+4jLS2NVavWsHHjJkzT5PjxI/zt\n3/4NXm8X11yzhfT0jJjH/vnPf4rH4+Hxx3/M44//GIBvf/u7uFxpoxqjYZi0dvgoKhrV3aYcJRWj\neqqdA1NYmDXkJ9LJRl7v6G3d+gxVVef49Kc/O06jip94/ft6vV5+/vOfcPfd98Rc/r733cTTT784\n7s83EhPxsxwIhmjp8GMYJsu0YjkPa4jzsGSGJYRICqFQiI985KOJHsaE8vqCtHf6mZSRJw4kYAmR\nBG699b2JHkLChZMw+krU7CqeTNPE0xWgyxcc/sYiQgKWEEJMoPB+lT9oJHooKUcClhBCTJDo/Sox\nehKwhBBiAsh+1dhJwBJCiDiS/arxIwFLCCHiRParxpcELCGEiAPZrxp/ErCEEGKcyX5VfEjAEkKI\ncSL7VfElAUsIIcaB7FfFnwQsIYQYI9mvmhgSsIQQYgxkv2riSMASQojLIPtVE08ClhBCjJLsVyWG\nBCwhhBgF2a9KHAlYQggxQrJflVgSsIQQYhiyX5UcJGAJIcQQZL8qeUjAEkKIQfgDIRrbu2W/KklI\nwBJCiAF4fUECrV4JVklETfQAhBAimZimSXunnzZJrkg6MsMSQogesl+V3CRgCSEEUl+VCiRgCSGm\nPKmvSg0SsIQQU5bUV6UWCVhCiClJ9qtSjwQsIcSUI/tVqUkClhBiSpH9qtQlAUsIMSXIflXqk4Al\nhJj0ZL9qcpCAJYSY1GS/avKQgCWEmLRkv2pykYAlhJh0TNPE4w3Q1S37VZOJBCwhxKQi+1WTlwQs\nIcSkEQgatHb4CMl+1aQkAUsIMSl4fUHau/yYEqsmLQlYQoiU5+ny0yn7VZOeBCwhRMoyTJO2Dj++\nQCjRQxETQAKWECIlBUMGrR4fQdmvmjIkYAkhUo7PH6K10yf7VVOMBCwhRErp8Abo8AYSPQyRABKw\nhBApwTRN2jr9dPtlv2qqkoAlhEh6IcOgxeMjGJI1wKlMApYQIqn5AyFaO3xIboWQgCWESFpd3QE8\nXQFpXiuAOAcsTdO+BLwPcAL/A7wOPA6YwFHgHl3XDU3TvgLcBgSBz+m6/nY8xyWESG6madLeFcAr\nhy2KKGq8HljTtGuBK4GrgM1AKfAfwJd1Xb8aUIA7NE1b3XP9euDPge/Ha0xCiORnGCYtHt+UC1Zy\nXtfw4hawgJuAI8CTwDPAs8AarFkWwPPADcAmYJuu66au69WAXdO0wjiOSwiRpALBEI3t3VOq07pp\nmhyvbuG//nA40UNJevFcEiwAyoHbgdnA04Cq63r4Y4QHyAGygaao+4UvbxjsgfPy0rHbbfEYc9wU\nFmYleggTSl7v5BaP19vVHaDV4yM3N7m21qdNy4jbY9fUe/jDq6c4fq551PdNxffBsYrnT0YTcFzX\ndT+ga5rWjbUsGJYFtALtPX/ve/mgWlq6xnmo8VVYmEVDgyfRw5gw8nont3i83vYuf1IetjhtWgbN\nzZ3j/rjtXX5e3nue/XpDJKFEUUb3GKn2PjhSQ30YimfA2gncp2nafwAzgAzgFU3TrtV1fTtwC/Aa\ncAr4lqZpjwAlWLOwxjiOSwiRJKbaYYv+YIidlbW8cehizGueX5LDLRvKEziy1BC3gKXr+rOapl0D\nvI21V3YPcBb4kaZpTuBd4A+6roc0TdsB7Iq6nRBikguGrGLgqXDYomGaHD7ZyLa952nr9EcuL8pz\nc+uGchaU5iZwdKkjrovFuq4/OMDFmwe43T8B/xTPsQghkke3P0hb59Q4bPFsbTtbd1dxoaF3aTEj\nzc4Na0tZu7AImzrKtcApLLl2N4UQk95UOWyxqa2b5/dU8c65lshldpvCVctmsHnlTNKc8vY7WvId\nE0JMiKly2KLXF+TVAzXsPlYXs9y5fG4+N11RSl5WWgJHl9okYAkh4i4QNGjtmNz7VSHDYM87dbyy\n/0JM0XNZcSa3biinrHhqlT7EgwQsIURceX1B2rsm736VaZq8W9XCC3uqaWzrjlyel+XipitKWTYn\nH2W0OetiQBKwhBBxk6z1VePlYmMnW3dXceZie+Qyl8PGllWz2Lh0Og57/JoJmaY55QKhBCwhxLib\n7PVVbZ1+Xtp7noMnegt/VQXWLSrm+jUlZLodcR/DQ4/t5qt/vT6uQTHZSMASQoyrybxf5Q+E2FFZ\nyxuHLxKICsZaaS43byijOC99wsZS1+KlwxsgL8s1Yc+ZaBKwhBDjxusL0t7pn3TnVxmmyaGTjWx7\nu5r2rkDk8unT0rllQxnzS6TwdyJIwBJCjJlpmni8gUm5X3XmYhtbd1dzsbG38DfT7eA9a0tYoxWh\nJrDw15ysmSyDkIAlhBiTybpf1djm5bevnebwyd6DI+w2hU3LZ7J5xUxczqnVKT0ZSMASQly2QDBE\nS4d/Uh0+2NUd5LUDNew6VocRNYNZOa+AG68oJTczefaMDJlhCSHE8CbbflUwZBX+vnqgBq+vtxtH\n+fQsbttQTklRZgJHN4jJ8s0fIQlYQohRMU2T9k4/XZPkCPtw4e/zu6tpau8t/J2W5eL918+nojAj\naeudpli8koAlhBg5wzBpbO2eNMHqQkMHW3dXcba290DKNKeNLatnsXHJdIoKs+JygON4kYAlhBAD\nCO9X5eamfqFqW4ePbXvPc/Bk71mxqgJXLLYKfzPS4l/4Oy5kD0sIIWJ1dQfxdKX+fpUvEGLH4Yvs\nOFxLIBRV+FuWyy0byinKdSdwdKM3xeKVBCwhxOBM06S9KxDTfTwVGYbJwZMNbNt7Hk+fwt9bN5Qz\nryQngaO7fFMsXknAEkIMLGQYtHr8MTORVHT6Qhtbd1dR29QVuSzL7eA960pZvaAwoYW/YyWFw0KI\nKc8fCNHamdr1VQ2tXp7fXc3x6t4Tfx02lU0rZnDNipm4HKlf+JvC/zyXRQKWECJGV3cAT1cgZZeb\nOrsDvLK/hrffqY8prF01v4D3rEuuwt+xCqX47He0JGAJIYDe+iqvPzWPsA+GDHYdu8RrBy7QHfUa\nKqZncevGckoKk7Dwd4yCoVT9WHF5JGAJIVJ6v8o0TY6ebebFPdU0e3yRy6dlu7hlfTmLK/KStvB3\nrIIp+O81FsMGLE3THMD/Ad4HzAcM4BTwFPAbXdcDQ9xdCJHk/IEQrR2+lNwPqanv4LndVVRdii38\nvW51CRuWFGO3pX7N2FBkSTCKpmm3AV8GdgCPA1VAEKgAtgD3apr2NV3Xn47vMIUQ8ZCq+1WtHT62\nvX2eQ6eiC38VNiwp5rrVs0hPlcLfMQrIkmCM+cA1A8yi3gG2aprmBD4bl5EJIeImVferfP4Qrx++\nyM7KizH7N4vK87hlfRkFKVb4O1Yyw4qi6/p/DnO9H/j2uI5ICBFXwZB1hH0qbdgbhsl+vZ6X9tXQ\n4e39/DwzP51bN5YzZ2ZqFv6OVTAV13HHYLglweXAz4FS4H+Bz+m67um57oCu66vjP0QhxHjxBUK0\npdh+1akaq/D3UnNv4W92ulX4u2pBIeokTagYCUm6iPUo8HdAJfA14DVN067Vdb0DmLo/JUKkoM6e\n/apUUd/i5fk9VejVrZHLHHaVa1bM5OrlM3BOgsLfsQpOslOehzNcwErXdf21nr//raZpjwBPa5p2\nU5zHJYQYJ6Zp0tbpj6lNSmYd3gCv7q/h7XfrIjNBBVi1oJD3rCslJ8OZ0PElE5lhxWrXNO1m4EVd\n101d1x/QNO2XwBNAevyHJ4QYi1TarwqGDN46eontB2MLf2fPyOa2jeXMLMhI4OiSky8gASvap4DH\ngCKsvSyAv8RKtLgljuMSQoyRzx+irTP596vChb8v7KmmJarwNz8njVvXl7GwfPIW/o5Vtz+1u+iP\n1nBZgu8CVwNompar63qrrush4HOapv3LRAxQCDF6Hd5ATDZdsjpf7+G5XVVU13VELnO77Fy3ehbr\nF0/+wt+xSpVl3vEykk4Xy4F7gc1YdVkA6LreOOidhBAJYZgmbR1+fIHkfiNr8fh48e1qKk83RS6z\nqeHC3xLcLukaNxISsHpomrYB+DrQAfwIuHuiBiWEGL1gyKDV40vq2pxuf5DXD13kzSO1Mftqiyvy\nuHl9GQU5U6vwd6yS/YPJeBvqY0wmkAucBhp6lgKFEEnI5w/R2ulL2iPTQ4bJvuP1vLy/hs7owt+C\nDG7dUM6cmdkJHF3q6k7xk6DMGW9IAAAgAElEQVRHa9CApev6y8AqTdNuBL6kadocXdeXTtzQxGR0\n9GwTOytraWj1UpjrZtPyGSydnZ/oYaW0ZN+vOnG+la27q6hv8UYuy85wcuO6UlbOL5jShb9jJUuC\nfei6vg3Ypmla+QSMR0xiR8828cTrZyJf17V4I19L0Bq9ZN+vqmvu4vk9VZw43xa5zGlXuWblTDYt\nn4HTLoW/Y5GRZk/qDyrxMKKdTU3TNOCTmqblRV+u6/rH4zIqMSntrKwd9HIJWKOTzPtVni4/r+yv\nYe/x+sgSpQKs0Qq5YV0p2elS+DsesjOctHX6Ez2MCTXSVJwngd9gtWgS4rI0tHoHubx7gkeS2rr9\nQdo6/Um3XxUIGrx1tJbtBy/GzPrmzMzm1g1S+DveTNNaDg6GjCmT/j/SgNWq6/pX4zoSMekV5rqp\na+kftApz0xIwmtSUjPtVpmlSebqJF9+uprWj9xN/QU4at24oRyvLlcLfOHC7rCXVVo9vyhyrMtKA\n9bimaf8KvIJ1gCMAuq6/EZdRiUlp0/IZMXtY0ZeLoSXrflV1nVX4e76+t/A33WXnujUlrF9chE2d\nGp/8EyHcU/FiU6cErD6uBK7q+TPMBK4b9xGJSSu8T2VlCXZTmJsmWYIjkIz7Vc3t3bz49nmOnIkt\n/L1y6XSuXTVLCn8nQE6mC4CLjV0sn5vgwUyQkf5UrdZ1ff7wNxNiaEtn50uAGoVkq6/q9gf542un\neGVvNaGoALp09jRuWl9GfrYs706U3ExrhnWhoWOYW04eIw1YxzRNW67ruiRdCDFBkmm/KmSY7H23\njpf319DV3VusWlKYwa0by6mYLoW/l8uuKqS57KQ5R5fmn5XuxO2ycfJC2/A3niRGGrAWAgc1TasF\n/FhZqqau63PiNjIhpqhkOr/KNM2ewt/qmCzPnAwnN60vY/ncfCn8vQyqqpDmtOF22nHYL2+fT1UV\ntNI8Dp1qpL7VS9EU2McaMmBpmpal67oH+JMJGo8QU1rIMGjxJMf5VZeau9i6q4pTUZ/gnQ6VmzdW\nsGZewWW/0U5VqgIupx230zZupyWvnF/AoVON7H23jts2VozLYyaz4WZYv9c0TcHKDnxe1/UjEzAm\nIaYkfyBEa0fiz6/ydPl5eV8N+/Sowl8F1mpF3LC2hPKSPJqbOxM7yBShKJDmsJHmtON0qOOe3r9G\nK+QX23R2VNZyy/pyVHVyz3aHOw/rZk3TMoDrgb/VNG0Z8C7wPPCyruvtEzBGISa9ru4Anq4AiYxV\ngaDBm0dq2X7oAv6ok2znzcrhlg1lzMiXwt+RUACnw0aa0/ovnjVoGWkOrlw6nTcO17JPr+eKRcVx\ne65kMJJegp3A0z3/oWnaIqzThn8D3BrX0QkxyZmmSXtXAG8Cu24b4cLfPdUxrX4Kc93cuqGMBaVS\n+DsSTrtKmtNKnpjImc4t68vZWXmJP2w/zcp5BeO23JiMRtpLcBpWavvLWPtZq4HPxnNgQoxWqnWC\nDxkGrR4/gZAx/I3jpOqSh+d2naOmoXeJLyPNzvVrS1i3sBjbJF9iGiuHTSXNZc2kElUkXTwtnRvX\nlfLC29U8tfMsd22Zl5BxTISRZgn+GnjJ6oHLXcB3gB8DW+I0LiFGJdU6wfsDIVo7/RgJ2rBqbu/m\nhT3VHD3bHLnMpipctWwG166aSZpTCn8HEw5SLoctaXr4vW9TBftP1PP8nmrmleSwan5hoocUFyP9\nbufpuv4IcAfwuK7r/w/Iit+whBidoTrBJ5uu7gAtHl9CgpXXF+T53VV853eHY4LVsjn5/N0HVnDz\n+jIJVgNw2FQy3Q4KctLIz0kjI82RNMEKIM1p5547l+G0q/zomXc4Wzs50wtG+pOpapq2Bms5cLOm\naStHcV8h4i4VOsGbpkl7px9vAuqrQobB2+/W88q+Grp8sYW/t22soHy6fP7sy25TyM5wooZCSRWc\nBlNWnMXf3L6YR586yrd/c4gvfGjVpPt3HWnQeRD4d+ARXdfPaJq2G/i7+A1LiNFJ9k7wwZBBa8fE\n11eZpolebZ3429jWG7xzM53cdIVV+CsJFb3sNiWSOGG3qWSlO+nu9CV6WCO2dmERfx1cxP999l2+\n+asDfOqOJSyfW5DoYY2bkQasUl3XI41udV3foGnaPcBr8RmWEKOTzJ3gff4QbZ0TX19V29TJ1t1V\nnL7QuzzkctjYvHImVy2bIYW/PfoGqVR35dIZ2G0q//e5d/mvP1Ry17XzuPGK0knRkWS4ThefA7KB\nT2maVt7nfh8Gvh/HsYkRSrXsuHhI1k7wiegH2N7l5+W959mvN0TquhQF1i0s4vo1JWTJib8x/fsm\nQ5Dq64pFxRTmuvnuE5X87rVTvHOumY/ftojcng7vqWq4GdZJYC1WLVx0ePYBfxWnMYlRSLXsuHhK\npk7wiegH6A+G2FlZyxuHLuIP9qbKLyjN4eb15Uyflj5hY0lGtp7+fWlj6N+XbLYfujDk9TeuK+Wt\nI5c4eraZL/1wN+sWFTF7RtZlLwNfu3LWZd1vvAzX6eI54DlN01p0Xf9u9HWapv1pXEcmRmSo7Lhk\nefOeaia6H6Bhmhw+2ciLe8/THlX4W5Tn5tYN5SwozZ2QcSSj3iazNhz2yVtQOxi3y851a2ahV7ey\nX29gZ2Utpy60sWFxMdkZqTfTHm5J8IOAC/i8pmmtfe73EPDHOI5NjEAqZMdNJRPdD/BsbTtbd1Vx\noTG28PeGtaWsXVg0JQt/VYXIntRk7vowUoqisLA8j1mFGbz9bj0XGjp5euc5FpbnsmxuPq4U+h4N\ntySYhXXScBaxRcJB4B9G8gSaphUB+4H39NzvcazTio8C9+i6bmia9hXgtp7rP6fr+tujeA1TWrJn\nx00lXd1BPF3+CekH2NTWzfN7qnjnXEvkMrvNKvzdvHLqFf5Gmsy67Cn1BjyRstKdXLd6FtV1Hew7\nXs8751o4daGN5XPz0UpzsaXAXt5wS4I/Bn6sadr1uq6/MtoH1zTNAfwQCL+j/gfwZV3Xt2ua9gPg\nDk3TqoDNwHqgFHgCWDfa55qqkjk7bqoI11d1TUA/QK8vyKsHath9rC7mxN/lc/O56Yoy8rJSe1N9\nNMJNZt09XSckPX94iqJQPj2LksIMjle3Unm6iX3HG3jnbAtL5kxjQUlOUgeukX4M69Q07SkgE+vn\nxAaU67peMcz9HgF+AHyp5+s1wOs9f38euBHQgW26rptAtaZpdk3TCnVdbxj5y5i6kjU7bqowDJPG\n1u64B6tgyGDPO3W8euBCTKPcsuJMbt1QTlnx5CoQHYqrpxO6y2mbFKnaiWCzqSyZPY25s3I4drYJ\nvbqVve/Wc/RME0tn5zO/NCcpsydHGrB+glU4/FfAd4E/BQ4MdQdN0/4KaNB1/UVN08IBS+kJTAAe\nIAcrbb4p6q7hywcNWHl56dhTbAO1sDB+byhbCrPYckVF3B7/csTz9SYLfyBEc3s3/mCIadPic/SG\n2ZNQ8cfXTlIftfSbn5PGndfOY83CooTMLOL1egfjtNtwp9lxu+wTvi+XrD/LGelO1DE23M0Crp2W\nwfqlMzl0op7KU03sPV7P0bPNrNaKWDInPyajMtHfi5EGLJ+u6z/VNK0CaAH+EhjuMMePA6amaTcA\nK4GfA0VR12cBrUA7sX0Jw5cPqqWla4TDTg6FhVk0NHgSPYwJMxVer9cXpL3T2q+aNi0jLgcaXmjs\nZOuuqpi+cC6HjS2rZrFx6XQcdjUhvwvxer19RRf0gom3I4i3I+5PG2Oif5ZHExA6u/zD32gUls6e\nxrxZ2bxzroXjVS28WXmR/cfrWDJ7GgtKc3HY1Qn5Xgz1PRhpwOruOWJEBzbouv6qpmlDTnF0Xb8m\n/HdN07YDnwL+XdO0a3Vd3451ptZrwCngW5qmPQKUAKqu640jHJcQE8o0TTxdgbguAbZ1+nlp73kO\nnugt/FUVWLeomOvXlJDpdsTtuRMtXCvldtmTcklqsktz2lm9oJAlFdN4p6qF4+da2K83cOxsM4sr\n8tiwuDihCT0jfeb/AH6LtRT4tqZpHwb2Xcbz3Q/8SNM0J9bJxX/QdT2kadoOYBdW9/h7LuNxhYg7\nwzBp7fDFFOWOJ38gxBuHL7KjspZA1HNoZbncvL6M4rzJWfirqgppPckTU7FWKhm5nDZWzS9gcUUe\n755r4d2qFg6caOTBR3dx0xWlXLe6BLdr4gOXYpqDJ+Fqmlam63p1z98VXddNTdMygAXAYV3XE3Ly\nXEODJ5EniY/aVFgiizYZX28gGKKlY+Dzq8a6RGaYJgdPNPDS3vO0d/W2cZo+LZ1bNpQxvyS5Cn/H\nY0kwVdLQE7AkOOINut+/dHzC3gf9gRDvVrVw8nwbXb4gGWl2bryijBvWjH/gGup7MNwz7dI0rQPY\nhnWA46u6rncAB8dzgGLiTVT/wb7PU1KUSU19Bw2tXpx2a2+itdNPIGhgmCaqouCwq5QXZ0VuW1Xn\nGfL6hlYvgWCIto4AvmAIl8NKcw5n05UUZnDblRXDvr5nd51j+8ELdHgDuOw2XE4Vry+EaZoU57m5\ndnXJuAePMxfb2LqriotNvXtRmW4H71lXypoFhRN61Hq8KVif3NOckoaeapwOGyvmFfCJ25fw8v7z\nvLT3PE++cYaX9p7nvVdWcO2qWRPS7mrIGRaApmlzgauBa4CNWNl7LwEv6bq+O+4jHIDMsCyXG3Se\n3XWOF/ZUEwwZ1hEKbgdpLjvv3zyn3/1H+hzRtyspzmadZh1pEF0j1u0L0uLxkZvlQoHeQwwVwITg\nALMXRelpZKkoGIYZyRBTVIVg0MDpsGGzKXR1999TsqlK5Pa5WS4+cuOCQb8/z+46x7NvngOsGU/f\ntkoKkJHu4K5r5/YLWpcz42hs8/LCnup+hb+bls9k84qZuJzJO+sYzesN10qlahq6zLBihXsJen1B\nXtp3nhf2VNPtD1GQk8ad18xhw+LiMX8QGep7MGzAiqZpWi7WqcP3A5qu6wmpUpSA1b/pbdhAQafv\n/X7w1DGCffZh8rJclE/P4lN3LB31c/S9ncOuEggapDlsdAd6m782tHoJBg0URSEYsmZMpmkFJUwu\nq0OEqjCiNkiKYr3Gv7pl4YBB+b+fOBLZNxrs4RSgYnomn3jf0pjLR/MG3tUd5LUDNew6VocR9bu3\ncl4BN15RmhLdtEfyep12NdJoNpVniRKwYvVtfuvp8vPcripePVBDMGSyoCSHj9yoUVKUednPcdlL\ngpqm2YFNwM3ATYAbeBn4R+DVyx6RGLPhmt4ONjPaWVlLMNR/69HjDcT0Hzx6tonHnz9OW8++jYk1\nY8lKd/RrrDvYWGoaOijIdUe+DoYMgoaB0efpR/GZqZ+R9uwzTWjvDPTrZB+ebY4kkcIELjUP3Ltx\nOL2FvzV4fb1BvHx6FrdtKB/TL3iyCKehu102bGOsDxKpISvdyZ9fP58b1pTw61dOcvBkI//0073c\ndEUpf3L1nHFfJhxuD6sFeAv4A3CnruvnxvXZxWXr2/S22xfE4w1wqamTb/5yP60dftJ6NkOjjxxp\naPVit6n9ZljBkBHpPxieMbV1+GPa/4QMk7YOPydr2oYcy2AU6BesJpJhGFxq6uQ/f3cYRVFQVYVQ\nyBjxMtXlzBNM0+Sdcy28sKeapvbeDwTTslzcvKGcJRV5Kb2XI2noAqAg181n37+cw6ca+dXLJ3i+\nZ7n7k+9bzIz88SsyHy5g/RC4HqsIuETTtG3ArkRlB4pe0U1vw3tDAHa7yvmGToJBgzyIBC2wZkKF\nuW46u4O0emKP/bbb1Ej/wfCMaaCMOBPo7HMg4WANeEsKM+gOWD8qXl+QQDCxK7mGCb6e8WCakWBs\njHCKp6owPX/kqeUXGjp4bncV52p7l5TSnDa2rJ7FxiXTU/YN3qYqpKfZp+yRHWJwK+YVoJXl8quX\nTrLzSC3//PhePn3HUlbMKxiXxx/yN0bX9Qd0XV+FVX91FvgMcELTtCc1TfvUuIxAXJbo5raeqACS\n6XZElvw8fQJLQ2s3m5bPwO2yk5vlwm5XQbGC3M3ryyLLZOEZ02Bv46E+06TBGu3edmUF7988hzSH\nSmuHDzAva5aSDBx2ldxMF9euGv4Au7YOH79/7RTff/JoJFipCmxYUsz9f76Sq5fPTLlgpSjW2Up5\nWS6m52eQne6UYCUGlOa08/HbFvGpO5aACf/9xyPseaduXB57RAn0uq7Xapr2K6wTiK/Cas20Hqux\nrUiA6Ka3l5o6sdtVMt0O3C47Hd4AwaDRb6+qMDdtRM1ywzOmnlyIfvruT0Q/5omaNrzdQUKGwePP\nH+faVbPIyXQxI99ajvMHjZgHtTIAredsaO0e8Wwnmt2moKBEAqnDbiMYMmKWMy9XmtNGhtvBzPx0\n1i4sGjKt3ddT+LvzcC2BqO/9wrI8bt5QRlHUfl4qkDR00ddwJxz3tWX1LF49cIEfPn2MM7VtMcuD\nl3N68XBJF+/DClCbgDnAbqxkiw/qun5s1M8mRm2otPLwkfA/eOpozJJcpttBq8fX71P8puUzRpSm\nHj6yRFWVAd/0TeBrP9sbc/+ls/M5d8nDoZON1v6QotDRFeDZN8+R7rbjtNusxxoghmRnOElz2VFV\nMEPDZwuG3zdN05q5qD1Zh3abGjleo6Ft7AdYKgp86Ib5w9ZeGYbJvuP1vLTvPJ4+hb+3bixn3qyc\nMY9loqR6GrpILsXT0rlhTQkvvF3Nzspa3ntVxZhaOw13z89gBajPAftl72pi9U0Xj06eiA4yfc/E\nClee52W58AeMyCwKGNHjgdWFACIlUpGZkNJTuGuY/e+//WDspy/DMDFMkzaPH1VVrHNpbErP5dZt\nnA4bOT2p3E6HjZBq4g+GBs0cdDlshEsxDNOM1HEpihIJVpH6rjFKd9mHDVanL7Tx4v8epaa+tytr\nVk/h7+oUKvy10tCtRrOpMmaRGgrz3KyaX8CBE40cPdPM2oVFw99pEMMFrO/puv7MUDfQNO19uq4/\nfdkjEIMaLnU9bKRnYv3gqaPDPl50kCzISYskc+RmuSJLjVl9mq+Gx9nq8fXUVVlhLjroGKY1u7Kp\nCnabGln6M6PmU1luBybQ4Q3Q7Q/2yygMf9gPP1b4Mew2laJcFzabSkOr97KWFQeSNkTLmfpWLy/s\nruJ4de/BAg6byqYVM7hmxcykbjcUFq6VcjklDV3E16KKPI6ebebMxfYxfZAbLmBV9GQG/gF4A6gB\nAkAFcB3wQeDJy3pmMaSjZ5s4draZQFQ3CiCSuv6Dp44OuDw4lMHSz6Prr6KDZJrLTl7Pc3Z6AyhY\ns7a+b+RVdR3UvX4GRVEwTROzT7BSALuqEjKsYmGn3RZ5PYGetHKnQ7WSMzr9mKaJYVjLfSi9qfCm\nafX0iw5HTrsNp12lobXbenzDHFFd12D7c2E2FWYOkBHY2R3glf01vP1OXUwN2Kr5Bdy4rjQyW0xG\nClbyiAQpMdFsqkppYSanL7bj6fJf9u/JkAFL1/XvaZr2W6wO6r8G5gMh4DTwDNZe1vikf4iI8CzH\nBKtlUdCI1PCoioLdrg65nDeYwdLPw/VX0D+opbns1v6SolCYmzbg/QPBEGlOG1npDto6/P0Chqoq\nmFh9AFWbQmFU8sGHN88BopYqTfCHU89RYpb2oveuwEq2MEyrNgysdk09DzEkm2p9HwMhK2tRVaBP\nJyYcdlvM0kUwZLDr2CVeO3CBbn9v4W/FjCw+dNNCspK0lVLMnpRDlvtE4qS5rN+RsZx2MOzul67r\n9cBXev4TEyA8ywknT0BvTZRqU2LOQ+q7PDiUvntd0ZeHkzHqW7yYEOkvGBZeZhzo/uFq9vCnprbO\n3qAV3c9PURUy3L3BLzxDDC9VhuvJDMPsmQHFRhGzZ0kxeskv+hgOcwT7VtZYTWt2oZgYRk/hcM++\nmokVwK5eMYP5JbmYpsnRs828uKea5qjatWnZLm5ZX87iijzy8zMn5EDDkZLECZGMwv0+nWMoh0jc\nSVxiUA2tXrp9QTq8gUi/PaMnIy43yxXTzj96OW84g+11Qe8MJ6MnSLZ4fDGFx9HLj33vv7OyNjLz\nysl04fWFIstz4WBlmCZGyMBhd/bbY6uq8+DpCtAddSiiGflfrFBUsBppYoWtp9+g3a7GzO58/iCd\n3UHS0+x09aTi21SVq1fMYMuqEs7Xd7B1VxVVdbGFv9etLmHDkuKkqqVSFCshRVLQRTIyDJO6Zi9O\nh0p2xuUfQCoBKwk57TYueKxP7GpPu3LTNHE4bP3OnolezhuJgfa6opMxwo/f4Q3Q4Q1QPj1rRHtl\nT7x+Bm9PkLVmPSZulx0Ta4nPMEwcdqsZavRyJoCny0rmGEn8CccrRRl5s1yHw4bdruLzx3Z0dznt\nrNYKudTUZQXoLBdrFxZRmOvmd6+e4tCp3oOvVUVh/ZJirl89i/S05Dnx1yUzKZECqus8dPmCLCjN\nGdOHKQlYSan/W7E6yDv0YF0mRiO8bxUOOOGapsx0Z0z39sGEa7DCR5bYbArBkElXd9A6/kMFVVHJ\nyXTG3C+89JnldsT02RtI9P6V0lN7NdIZ1rRsK6i7sl1kpjtjglN02rrPH+L1Qxf4fy/qMceLLCrP\n45b1ZTGNfBPJYVNJc9lwp3gndDE1hEIGh041AbC4YtqYHmu4wuG/HOp6Xdd/PqZnFwPyB43eNPJw\n8HA7UBSF4jz3kKnrl6Mw1825S56Y/oLBoIGny8/Rs00jeo6a+g4Kc914fUHaOvzWXpNhEghaCQpZ\nGc5+s0NrOdMkzWXHpqqEjIHrr9LTrPv5AyEMw0RVrcJkRaHfuVUDqW20OoFcu2omW1aV9LveMEz2\n6/W8tK+Gjqh2VjPz07llYzlzZya+8FeazIpUVXm6ifZOPwvLc8nOcA5/hyEMN8PaMsR1JiABKw4K\nc90YLd7IG3y4E3v4s/Sd18we19OBraSL5n6XZ7n7HyUymPAsLfKGb/b+oSqKtT/VJ5U1vJx57pLH\nCmyDxJ5gOIBDJOEkHMyVnkyJocKWiZXlt19voKQwM2ZWdbKmled3V3OpuffE3+x0BzdeUcbK+QUJ\nXWZTFWvZ0u204UyBui4h+qpt6uTomWYy0uysml845scbLq39Y2N+BtFP9FHsmW4H166axe0bKyLX\nR2fjRXdiz81yXVY6+3CWzs4nK91h7SX1OYV4pEkd4ZT5YMjANIi0dLK6Y1izRq8vGDPL2rR8RqSd\nU19qT+q52dNmIxgyyEp3RNq6hB+nodVLIBgiK91Je6d/0BmXaVp7ZfuO1zO/JJe6li6e313NifNR\nhb92lWtWzOTq5TMSFiDCGX5ulyRPiNTW6Q2w43AtKHDNypnjcjbWcEuC/zjU9bquf3XMI5hioo9i\nByL99oBI0IrOxjt2tjmmsW3YaNLZR6K8OGvYGq2h+hCGg6zdptId7K1TCi/f2RwqwZ4i4ejlzJ2V\nteRluWjt9OMPRM2yTOu+edmuIevA0l12Oka4lxUMGjS2dfPUzrPsfbe38FcBVi8o5D3rSse8ZHE5\nwkHK5bCR5pLkCZH6AkGDV3tqFtctKorJzh2L4ZYEo39z7sY6H0uMQd9+e9GXR8+ywtl4X/vZ3gGz\n50aTzj6QvsGnpChzwIAVTuoYSV/DNIdKKGR1s1AUa98l/Oabm+EkPc3BndfMZmdlLU++cYadlbVU\n1XlIc9mZ7rLT7QvS2NbbsT09zY7HG6Clw0em28GqBQWRMZo951k5HTYWF2dy5EzzsPtZJlDX7KW2\nqXf5b87MbG7dUM7MgvE7ZG4kVFXB5bDhcqg4HRKkxORhmCZvHL5Ii8fHgtJcFpYN3Y9zNIZbEvzn\n8N81TfuT6K/F5enoc0ZVWN9DEcMG607hdKj84KmjQ3ZdH0w4+ISzAi80dnL0bDOr5hcQCBoDJnUM\n1dcQ4BfbTtDhDVh7VqrSU8RrdeUILy867Uq/oOfpCmCa1hJfmsse6V+oKEqk0BCsfoH79QbWaIVU\nX/JwqdnLtJ5Mv33H67GpCgN/B2OFg2F+Thq3ri9jYfnEnfhrV5We4zrs4350uBDJYv/xBi40dDIj\nP50rFhWN6+/XaNLaE3tc7CSR6XbQ0dX/rTXDPXBtz0DdJbp9Qby+YKRF0Gj3tXZW1uL1BftlBR48\n2cin7ljS7zGG62v4blULXl8wMkuwqQpBw8Sm0mcpoP8PbpbbgccbiCx3hvsXdvmC+AKhSIZk+Ppz\nte3cde28mB/Gp988iy+qXdJQ3C47168pYf3iognppaeqCm4JUmKKOHG+lXerWsjJdLJ55cxxL7uQ\nOqwJdu2qWTF7WNGXD2Sg7hJtHb7I0fPRRpPRN9BMLxgy+j3GSPoadngDPenoJqoCNpsaqcWK3rN6\n8o3+bZ3SXPYB0/WffONMzFJoeAkw3DoKrAy/7Qcu0NzmG/bTlKrAxqXTuW51Sb/0+vGmKkSO6pDs\nPjGZDHXool7dwi+3nSDT7eBLH1kTlwNLR1OHld+3LkvqsEYvvE+1/eAFOr0BMgbIEuyrb3eJr/1s\nb7/beH1Bjp1t7new4kAKc91caOzf+85uU/vtjYVnY6GQSSBoWB0mwsW7NgWnXaUr6i6maQW+cN3Q\nwx9dG/NYAy1vlhVn9itQjr5tOFh1+4IEQwb/8+QRAsEQzR4/waAxbLBaUjGNm9eXkZ8zuq4go6Eo\n1hliaU47Tocq2X1iSmls9fL9J62OOffcuTRup2uPpg7r1T5fSx3WZbp9Y8WQAWo4ffe1wst79kEO\nVuwrXHcV7NM1Ocvt6Nfqqaqut6A4fPhieLYVLm6OPqojMhMLmbhdSkzh8VDNdwe67InXz2AYVrDy\n+YO0d/rJynDS5QvS0j78rCo/O407r5nDnJnZw9zy8oSz+/KyXDhMQ4KUmJJ8gRDffaKSDm+Av7xJ\nQyvLi9tzSR1WCur7xh9e3hvoYMWBAtbS2fncvL4s0kopuu6qb/CI7oYePh4kGLLenN0uO81DtFTq\n9of4xbYTfOTGBTGzxA7WnikAACAASURBVOfeOkdNgzXDKynMHPC+S2fn4/UF2VlZS4vHZ9VhZThJ\nc1rPOVywKshJ43MfWBGX7DunXcXtskf696WnOej0jC1rU4hU9YfXTlPT0Mm1q2YNurUxXoZdzNc0\n7RbAq+v6dk3T/gBMwzoT6yNyFlZi9N3XGuxgxaFS32/fWEHF9KxhTykeKFFAUawjPi40dAyZSh4M\nGjS3d/PcW+diHrelwx/pPnG+oSMmqAGRM65Ki7L40A1ZAPzPk0cie1qhvkcR92FXIavnGJPx4rCp\nuF3Wkp/07xPCUnm6iVcO1DCzIIM/v25e3J9vuD2svwbuBT7Zc9Ei4G+Bm4EvAZ+L6+jEoKJnLN/8\n5QHON3TQ0uGLmS0N18l9JKcUlxdngWllBIZnVopp4rTZCIaGP4jNMEzOXfJEUvAbW7utjMKeN/1g\n0KDV44sEtWDI+jrYE53CiRUXGzut87BsvScQD8RhV8hKdxIYQY/B4dhtSiR5Qvr3CRHL0+Xnp1vf\nxaYqfPK9iyckwWi4GdbngPfoun6p52u/ruuva5r2NnAYCVgJd/RsEy0dvsh+VDBoRM6yGo9O7puW\nz6CuxRuZvTW0ejENhZxMJ60dPhSsNkx9w0N4DmKY1rJieM+tszsQaXAbPVOpaeik2x+MOfzxZE0r\nf9h+mo6uQOTxg4NkrytARrqD7HSrU0Ve1uUdwS1NZoUYmd+8cpK2Tj93XTuXsuKsCXnOYTtdRAUr\ngJ8A6Lru1TStf7qXmHA7K2tjzrAK70nlZjrHpXXTQMuP4UMk2zp8kZ6BfcVcqijUNnVit6mRYBQy\nzMjhlNbKnUlrz1H3YdsPXMAzQM1az0NSlOsmGDIihzCG+wwCMcfbD0cKeoUYnVM1bew6Vkf59Cxu\nuqJswp53uIDl1DTNqeu6H0DX9e8BaJqWxkBVoGLChbuku132mPoif3D86ryjlw5/8NTRqAzF3h+B\ncLr7QHrikTULjL5dT5d10wRTUThZ08r8klxO1rSy73g9Zy95Bn5ArKM/7vnT5QCR2w92ztVApKBX\niMtjmia/234KgA/fsGBC93SHC1hPAf+ladpndF0PAWiapgCPAE/He3BieIO1bhrtScQjFZ2haGJG\nUt1VVbVmTD3Lg9bPsIKJiWmaGAaRGVVYTyN2VBWy3Q72Ha8H4LldVbR3+hlK+FBGgPklucMGKHqe\nK81pI81lxyUFvUJclneqWjhV08bKeQXMK5nYs+KGC1hfAZ4EzmqathvrPWY9cAy4M85jmzKG6oI+\nnNHUNo2H6CXCxlYvpo1IkkdtUyeoCk6HjfycNGqbOjEMqybL7IlU0TVbNhUcdhsZaXZcTjtNbd08\n8+Y5GtuGTxHv6PJHZmTDcdpVK3lCOqELMWYvvl0NwHuvqpjw5x6uDqsLuEnTtI3Ahp6Lv6fr+s64\nj2yKGEkX9KEM1LpppAHvcgNleInw6NkmfrntRKSjeihkdWnPzrDqwew2laBp9My2eguOFcXaN7LZ\nVKZlp2GaJp3eAB5vYETH3isK+IImL759HmDAoGVXFdJckuEnxHiqa+ni6Jlm5pfkMHtGfAryhzKi\npmq6ru8CdsV5LFPSUF3QRzrLGkl6el/hQBk+zTjcsf3m9WWj6sIRHV7sNut4kfDeVqbbQavH13MY\noxqZZaX1HCUSDIXo7ulgMZKj7hWspUZH1HJe+EBGkH0pIeJt11ErB2+onoLxJL/VCRZOmuh/eXw7\nJ+ysrI2cZhwMGpGkiBf2VHP0bNOIH8PtslOY6yY304XNpoACTW3ddPtDVEzP4varKsjOcGJizb5M\nrEMrAyGTYAia230jClZg1UWpqkJ6Wu/nrBaPH7fLzrQsF0W5brLSnRKshIiTvcfrcTpUVi0oSMjz\ny292gg12Eme8kibCquo8NLZ1EwgZBHsOXYTeju0jEQ624V6GwaB1ojCKldywafkMyooyyc1wYhgm\n/qBp7WcN8FjWMuHAqacOuxrpAh9uzxQ+IHJGvpucDKd0RRcizjq7A9Q2dbGoLC+mhGQijShgaZr2\nfU3T1sV7MFPRYMkR8UqaAGs50NMVsIKUaaWVh0JWXdRAHdsHEw62fY8qcfTsGT3z5jl+99ppugND\nN4ZV6OnybvQ/dE3peZ7r15aQn5NGepoDu03BblNRVYWrV/z/9u48Pq7yPPT475zZtIw2y7K82xib\nl8UYAwbMvkMICWkgTUugWZo0SZulyW1v702bkDRNb9qU3rZpk5LmJg0NSZOGhJIFsM1iwCw2NjZG\ngF/bICzJm2Tt2+zn/vGeGc1II83InpE00vP9fPKJdTTLe46Gec67Pc/ifE9bCHEKjrmVus9aUbzk\ntrnkGyZ3AH+jlFqAydD+w1EbisVJOpVFEydr296jBMt9hMKxjACRSDgEs2Rsz6apuZPegQhHOwdT\nPavkfgyf16a9e4hQOI7fZ+Pz2hMO+000IFhR5uW2y1dw/poF6FXdU3qdhBAjjnWZgHXmTA9YWuv7\ngfuVUsuAO4HnlVKvA/9Pa/3fxWzgXHAyiybyMd4qwI6eYcoDXqoq/fSlpULCMT2aXL279JWNdcEA\nJ3pDxB0Hj2XSGg0OR3EcM/QXiSYIZyk2mQ+vbfHx287m3FVmvLxY10kIkVtnb4jygIelC7JXWJgK\neQ9EKqVOA+7GBKyDwC+A9yulbtdaf3DCJ4uCy7UkfaLl8snNxmU+D4OWRQKzode2rZxlOyBzZWNZ\nwEtVhY+egQjhLMEp/fUsJu5NpbMt8HrtVLASQkyfq9Yt5sdbDrBsQeW07mXMK2AppbYBC4H7gXdo\nrVvc4/8BHC5e8wSMDU5LFwTZpTtSv8+2d2ui5fLJzcb9w1Fs28J2lzokcwTmWlKfvrIxFI7RPzRx\nVgqzQAKqKwP0D5neVzxuunO2BdmSvtu2hUfKeAgxI3T1hYjFEzTOq5jWduTbw7pHa/1k+gGl1GKt\n9RGgsfDNEknZekpNzV0Ey30ZuQMhc+/WRMvlk4/5zsOvgWX2T6W/Xq5FF8kemuM49A5FsgYcMD2q\nYIWPaCyBBSyZX5nK89d8tJendh+h+Uhv1ufG4w51804u47oQorCS6d8a60ojYH1UKfWU1toBUEp9\nGvgSEqyKLltPKRZPMDAcHROw0gNNrhyDa0+r55zT5p1UHsIlDZW8+lYn0Vh83HIfYIb/qtxyHzdf\nvIwzl9e5SXo9nOgdxmNZZBsoTC5ZryybnqWzQohM/cNmFKW60j+t7ch3H1YH8BOl1PlKqReA9wBX\nFK9ZIilbT8nrsbMWT0wPNPkslz+ZJfW7D7Sz4/V2fF57wkKKYELR/JoA775sBRef1UhDbTnBch8/\neHQf//SzvejWnqzlSSzLYl51WUEzzgshTl4obO5My/3Tu98x31WCn1NKfRl4CfgDrfW/F7dZIilb\nT6mq3Ef/8Ng6UemBJp/l8pNZUh+NxekfirLlpTa6+8MMhWM523760ho++77zMiZpf/3C27zQdIwJ\nUwY6jptBo7ibp4UQ+RmOmP/eywLTO+ox4bsrpf6dkfEaCzgBfFwpdSWA1vr3i9s8kS0be1nAy+Xr\nFtHWPjBhoMlnGXiux0RjZvixfzjCtr1H0S09ea30C/g83Hb5yjErirbuPpz3SsFibp4WQuQvWdHc\nN82JpHOFy605fhZFNh0biwHCkTiDoSihaJxXDpxg00utOWtUJVmWGZ586Jm3UqsSk+3N2Pc1jspy\nH3dcvUr2XAkxQ/i8ZigwGju5PZWFkqu8yP3pPyulbOB84E2tdU8xGyZGTNWGWcdxGA7HGQpFiSUc\nmo/28cgLhzh8YjD1GMsyE58T5at1HDjaOYRlmdVF+1t7aKwrp2cwYpazj8MC/D6bP3j32RKshJhB\nkgmlIxOtspoCuYYEVwM/wRRyfBx4BlgAeJRSd2qtnyt+E0WxJRIOu/a389yrx+jqC1FR5iUUidNy\nfCD1GMuCyjIfwXIf0Vic7v7whPNQybpWISfGUIicvTPLgvKAl3dcslyClRAzjN/nBqyTzFpTKLmG\nBL8J3As8Avw+EATWAKuAfwcuL2rrBHBqFYknEo7EGQrHaGruZNOOVhIJh/6hCG0dgxmPW3d6Pe3d\nQ9i2STjr99l4PDYd3cM556OSKwmzBbfk7JZtW1yoGiQ3oBAzVE2l2RPZPRCe1nbkClhLtNY/AVBK\n3Qg8qLWOAfuVUjVFb5045YrEo8UTCTPsF46lekE73jhuFlYMZc4vVQS8fOgWxbIFVfzXUwc40jHI\nQChGLJ7AwsqZa2miYGZbsKyxCoDGunI++Z61kz4XIcTUSK7YPTFOQoKpkitgWQBKKQu4FvhW2s+V\nxW3a3JOtJ1WIisSO4xCOxhkOx4lE46lA4jgObxzqRrf0ZGRTt4DKci81lX5OW1hNVaWfVYtreL25\nO/WYaDwODgR8NtFYYkwPajI7qGQ1oBAz2/waE7DGy6AzVXIFrL1Kqf8FlAFh4DmllB/4U+DFYjdu\nLhmvJxWKxLIWS8unZlUsnmAoHCMUjo0JKEdODPKbFw7RfLQv47jt5vcLR+KUzfNSX1OGZVm89lYn\nlmURdSddHcdko4jFTQ2thOPkXTkYwO/z0FhXuOFNIUTx+Lwe6qsDHD4xiOM4E9a3K6ZcAetTwNcx\niW9/S2udUEr9I3AW8LvFbtxcMronNRyOMTAcJRyJE/B7xuQOHG9TbcJxCIVjDIfjRLNkw+gdjLDl\npRZ27z+R0QtKBqr0D6LXtrAsi6bmTpqP9YNjsmwAqUwbCcfBk7VO8PgCPg9/9N61EqiEKCGrl9ay\n/fXjHOsaYlH99Ayw5VrW3gv80ahjfzTOw8UpSO9qJ0vOgwkgsVgi9XMyaI0eRovG3N5UJJZ1n1Mk\nGueZV47w7N6jGXsp1PJajnUNMhSKu5sDHXxei9qqkdRI2/YeNemgYiNBygFiCQfL/TmRyBxSHK+v\nZVtw62UrJFgJUWLOWGYC1v7WnpkZsMTUSU/BlF5y3u+zU6mYBoejrFxYlRpGSyQchiMxhsMxYnGH\nA2097NzXTnd/mLqqABvOXMDpS2rYvb+DzS+10j808roL51Vwy8bltHUMcKC1h4RbvNHjSSakBb/X\n4r6Hm9hz8AQWlglKFhl7qbwem1gis7Z9tmBlYVYDrl5SzbsuXVm4CyeEmBJnLDXr7Pa19HD1+iXT\n0gYJWDNEegqm9MS2VeU+ygJeygJebMviE7edQzhq9kGlL6A40NbDph2tqed19oX55ba3cXDo6htZ\nihos93HjRcu48IwGnn7lME/sbEvNbzm4wcgDvQNhLCAUTYz0rixSQcsC/F4PNUE/3f3hCXfAW5ab\n0sWCWy9bWZgLJoSYUovnVzK/pow9B0+kpiqmWr4FHD+htf7OZF5YKeUDvg+sBALA14DXgR9gvhub\ngE+582JfBm4FYsDntNY7JvNes0F6CqYTPWZ/UzJYJcvNz6sK0NEznHVP08597al/x+IJ+gYjhCIj\nu9K9Hosr1y3mqvMWE/B7ONDWw7OvHM26us9xzFBkMtFlVbmP7v4wtludOJlPrMwtEzJR1nbLDW5e\nr82yBUEZChSiBG3dY+r0LppfyYk3O/nR4/tZtbiaa6a4p5VvD+szwKQCFnA30Km1/j2lVD2wG9gD\nfFFrvVUpdR/wHqXUIeBq4BJgGfBz4KJJvteskEzBlFwxmHAc4vGRJePr18wfN7tEd384tfF3MJSZ\nSX396vncdPEyaoMjBRF37msnnkhgW2TMeVmWGRZMVvsNhWP0D0fNvJVjAprXawo+dveFc5YY8Xls\n6qoClAW83HrpisleEiHEDHL64mpefbOTt470smpx9ZS/f74Bq1Up9SSwHUitDtBaf3WC5/wMeDDt\n5xhwIfC0+/OjwE2ABja7xSFblFJepVSD1mk14OeQWDzByoXV3HDhEna8Yeaj6qsDqUq94z0nnnA4\n3j2UEXz8bq/m/detznj8wcM9vHm4l3h8JAgl1/k5mHmppQ1BegbCdLuLPWy3q2RjEY879AyEs65C\nTLJt8No286rLWN4YlOXrQswC1ZV+5teUceTEEL3TkPUi34CVvucqrzXMWusBAKVUFSZwfRG4N1m1\nGOgHaoBqoDPtqcnj4wasuroKvN7pLSQ2WQ0NVeP+LpFwePG1ozyxo4XjXUPMry3nsnWL+dT7zx/z\n2Nfe6uT5vUc40TNMfU0ZC+sreemN43Sk1czyeixqgwHKAx7eecUq5s0bWdGjW7p5es8RbNvG43FM\n1oq0AjIe2+KO69awemktf/fDnRnL3BNudLNsE7gmUhHw8T9/bwMXqAV5XqHSNtHfdzaaS+c7U8+1\nssKPbU99uY8NZzfy2AuH2NfSy8em+NrkW8DxL9N/djNdnJbreUqpZcBDwLe11j9WSn0j7ddVQA/Q\n5/579PFxdXcP5dPsGaOhoYqOjv6MY47jEIklCIVj7H2rM2PBxNETg/z8yQMMDIQyelXpCysisThH\nTwzyclpH1O+1mVcdwGObir0L6yvYurOFh546QF1VgMvOXcjLuoNY3KGyzEskGsdjW6kl6WVu8tlr\n3SXzleVe+oeixOJm4QVxh3jCIRJN5MxkESz3sWxe+Zjzno2y/X1ns7l0vlN9rpMJjoND+ZX7KbSG\n6gC1QT/7W7tp2n+cxrqKwr7+BNcg30UXH8ckwU1ffN8MrM7+DFBKNQKbgU9rrZ9wD+9WSl2jtd4K\n3AI8BRwEvqGUuhdYCtha6xP5tKvUOI75sg9FYoSj8dR8VPqCiXQ797VnBKyd+9qJxxN0D4THZE2+\n9JyFXHfhEirLfMDYVYM9AxEe296aypxRFvBSB/QPm4Dk89h84j3nZAzbrWisyqh23HK8P2ctKzB7\nrZY3BnM/UAhRcizL4tzT63n2laP86rm3+di7zp6y9863P/kF4DxMqZHTMYswtud4zp8DdcCXlFJb\nlVJbMcOCf6mUegHwY5Lp7gKeBV7ALLj41GRPYiZz3MwTvQNh2nuG6R4IMxyJZyyeSM4TAYQiMbr6\nQnT0DHGwrZcDbaazGY7GeftYP8e7h7Om+A9WeFPBCjKDoNdjYbuLKJLLz4fdxRTJ3tPShsoxc0zp\nm5N7B8J5BSsAj8eW/IBCzGIrFlZRVxXg+aZjHDzcO2Xvm+8cVrvWulkp9Spwrtb620qpCTNeaK3/\nGPjjLL+6OstjvwJ8Jc+2lIRozCSbDUVi7GnuZuvOlowNvek9p7qqAJ19YUKRGP1pdaMc4LHtLexv\n7WHvwU760zYUp7OAZ185ytKGYOp1u/vDZsWfm14pye/1ZGTSAFP+umcgQlNzZ0bQSl9q39o+Uhtr\nIj6PjVpeJwsshJjFbMvi4rMXsGl7Kw9s0tzz4YtSN8VFfd88HzeolLoW2Au8Wym1ECgvXrNKU8Jx\nGApFOdE7TGdfmKFwDN3aw8NPH6SzzxQ87OwLs2lHa6rnBLDhTLMwYchdjp5ImESykWicY11DPPfq\nsXGDVVI8kWDnvnYOtPXwn4/vp6svRHd/mHAks0Lo8sYgdcEAXq/ZyOv1jiw7z5YZfu1p9VyxbhFO\nju6VbZnM7YsbKvntG87I63oJIUpXY10Fl69dSEv7AE/tPjwl75lvwPoMcBvwGFCPWYr+z8VqVKkJ\nR+P0DoTp6BmmbyiakbV8ovmppDVLa7n54mWpvHxxd6l5whnZI+WxLa67YAkNtYExr5V8t6OdQ2za\n0UrPQIRghZ9YLEF3f5hQeGRf1hXrFhGJxWmoLWdRfSUNteWpDcLpGeCbmju57+Em/ur+l/jBo/tM\n/atxBHw2Xq/NykXV3HH1qjmzMlCIue59166mPODlF8+8lbOqeCHku0rwNeDzSqk6rfUdRW5TSYjF\nEwyHY4QiceIT1Irv7g9n7Sqnz1uBCVorF1Wxv7WX0dn4bNtizdIabtiwjOYjfZwgnGWVnkU0nqCi\nzItlWakkuQPDUXoHIwyFY0Sicb7z8GvYlkVFmTcVqJKSuQMPHe+nfyiayhDfNxgxS9qzCJb7OHtl\nneyzEmIOqqn0c/tVq/jRlv387KmDfLTICzDyXSW4HrPgokIptRF4Bni/1vrlYjZupoknEoQi8VSy\n2XzUVQXozXLnUVc10lOKxRO80HSMNw/3ZQQ/C/DYUBP0pxLiRuMJghU+BoaiqQ2/ZiuGQ8BnZ8xX\nlQe8WEBnXyi1dD0aS6SS5tZXl6WC1nA4Rihscgf29EfcObgYXtsi7mZlN+9i2BYsnl/BVz+6Ma/r\nIISYna49fwnP7j3Cc03HuHr9ElYvLV4x+nyHBL8JvBeTaukI8IfAfUVr1QyScByGw8mVeyF3X1L+\nhQqT81PZjjuOw6tvdfIP//UKj25vIeKu4EsGKr/PpiYYIOD3pgJcXVWAqgo/wQqfSauEGTZsrKtg\n5cKxqVL6h6NjVvfZtkm9FI0nsC2Lxrpy6ty0Tcc6h1JL7h0HomnZMBzMXi+/16Ys4OX9163J+zoI\nIWYn27a460Yzb/3AFp1Raqjg75Xn4yq01m8kf9Bab8EktJ2VHMchFInR3R+mo3uY3sFIKphMRrLc\nRygSZ3A4Qjgap746wM0XL6PM7+Xffvk6//n4gdTwYJnfw8VnLWBhfQWN8yqprykn4FYbTga+DWcu\nIBQxVYQ9toXPY+H1mDL1SxeM3fsUiyfIVhzUAWoqA3zpQxu4Yt0iDh3vp71nmHA0PvbBaRKOWahR\nXeGXIUAhBGCmNC5bu5CW4wM888qRor1Pvsvau5RS5+GOCCml7gK6itaqaRKOxgmFY4Si8bz3HI3n\nqd1tPPvKUeKJBD6Ph/KAh4DPw5kr6nh5fwevHBzJRmVbFpee08i1Fyylosybta5Vcrn6mqW11FT4\nGQrHiMcT+LyeVFb3tvYB7rh6Fdv2HqWjJ0RDbRllPpvWjsGM4ouJhOk1tbb384l7t2Yk2M3F77PN\nQg2/h/sebqKjZ5iGWil1L8Rc99vXnM7L+zv4+dNvcsnZjRkV0gsl31f8Q+B+4BylVA9wAJONveSl\n75cqVE/2QFsPW3cfIRYzKYxiMVMJOOCL8eDWNzOC4Vkr6rjlkuXMrx3ZJbBmae24iW7BzKUtqC3P\nmK8Cs8ovmfE9qam5kwc276enP0xsVGAyQ5uTO+lY3KSTSi44ATjePZyq5SVBS8wUTc2d7s2b3FRN\nhZpggFs2ruChZ97iyZfbuLUIhVrzGhLUWr+ptb4CmAcs11pfpLXWBW/NFInG4vQNRejoGdkvVchh\n160vtxGNjeTbSy5RH46M9NwW11fwsXedxe/drDKC1UQsoLrCz8L6yjHBCqChtmzMsbWn1XP3TWdQ\nXekryDkGy33UBv1Z756y7eMSYjokS/Qc7zb145I3VU3NnbmfLE7a9RcspTzg5fGdbcRz1R46CRP2\nsJRST5HlFlwpBYDW+rqCt6hI4omE6UmFY8SKOCkIcKxrGIvsfRevx+K3rlzF+jXzTcmOPHlsi9qg\nH5/Xk1GdON146ZDWnlZPwjG1rk5lqDPg8/DhW87koWfGvjdk7uMSYjqNd/O0be9R6WUVULKwY7pl\nC4Lsb+3hJ08cZElDZZZnjZVvIchcQ4JfyetVZqhEwiEUMcN9J7No4qTf160QnM1V5y1Ct3Tz1O42\nYrEEXq/N4vrKcetdHWjrYff+DnoGwiyoq8gY1kifq5pouKOpuZOe/vxzAWbjsS1uvWwFa0+rZ9ve\noxlJcZOy9fCEmA4dPWM/n+a43FQVm9lP2sORE4N5B6x8TRiwtNZPK6XqAE8yg7pS6mrg9ZlaYNGs\n8IsTisSJROOTnKE5NQPDUZ7Y1UY4S3JagLqgj30tvWNyBuKQkVk9ueDC57HpH45SWe4DrDFzRdkC\nVFNzJ795/hBtHSb3X12Vn2jMMUOIJxmxAn4PtZV+Vi40af8n28MTYqo11JbLTdU0qa8217irr/A3\nB7mGBM8HHgE+gknLBKZK8I+VUrdorfcWvEUnIb1sRyFW+E1WNGY2/j61+3DGsnDbMpt6/V4v5QEP\n1RU+wjEnlTMwaTAUI+D3snX3YfoGTYn7eCJBPO5g2ybTevqc0XjDGk3Nnfxo8/7URmHHfW3bhnK/\nl6FQ7KQC+MJ5FRnvO9kenhBTTW6qpo/PaxPweVKLsgop15DgvcCdbv0qALTWf6GUegb4v8ANBW/R\nJESi8dSQX5GnpbJKbvx9bHsLPQMjPab5NWWct7qe411D9AxEWFhfybmr5rHlJdOLGj0Zmfz58IlB\nnLQTcYB43KF3IJIRsMYb1ti29yg9AxHiozY2JxIQSzhUB/30Dkwu31d6Wqn09x2vhyfETCA3VdPH\ncRwSjpN1/+epyhWw6tKDVZLWepNS6m8L35z8DIViDISiRd1RnUtrez+/eeEQLcdHym6UB7xcf+ES\nLjm7EU9a6ep58yrp6hpk5752OvvCeGybeHwkaCUfG4sl8LgBwrIsLEwvKRrLvFMZb1ijo2d4zGOT\nIpH4ZFewA1BdMVJjS4ZTRCmRm6rp0T8UJRpLUFPpL/hr5wpYPqWUrbXO6BIopWxMAcZpEY7Gpy1Y\ndfeH2LSjlb1vjiyP9dgWl56zkGsvWDLhZrkNZy5g045WKsq8GXNYlWXmOV6PBWlZ0W3bGtNbguzD\nGk3NnZzoDY3b03QwPdLJ8NgWNcGRhCYynCKEyKXleD8AjfUVBX/tXAHraeDL7v/SfRHYWfDWzGCh\nSIyn9xzhuVePZuQSPGflPN6xcXlqonEiyVWAO/e1c8SyiMXi+LweFs+v4Or1S9i8o4Xmo/2px9uW\nBR7Tc7Mta9xhjeTm4OFw5tzYaPmGeNsyk9bJXIMynCKEyEckFue15m78XptVi8bmNj1VuQLWF4BH\nlFIfAvYAIeACoB1TH2vWiyccdu5r5/GdrQymLZZY0lDJOzeu4LRJ/lFGZ7HweWxqgn68Hhuf1+aB\nzfsZSCtdHyz3cfdNJrHktr1HeeiZt9i29+iY5e0Dw9FxS4BMls/roSzg5a6rV0mQEkLkxXEcXmg6\nTjgaZ/2a+fh9XNOhtwAAGjlJREFUnoK/R65l7f1KqauAa4HzgQTwLa31swVvyQy0v7WHR148RHva\n8tjqSj83XbRs0ht/sykPeKmu8PHa212pFDJ1wQB1QT+9g1EisTiWBb95/m16BiKpUiCHjvXT1NxF\nVYWP2ko/h44PFHRFzsqFQW69bKUEKyFE3pre6uLQsX4W1JWz9rR5RXmPnLkEtdYO8KT7vznhWNcQ\nj754iANtvaljfq/NVesXc8W6Rfi9J3/ncKCth126nd6BCI3zKli6IMiutC1tITcBrwNUVfgJhWMc\nPNxHwnHwe00S3YFhs+BkOByjo3u4YHvNfF6bd1++kncVIQeYEGJ2chyHV9/sZM/BTioCXq46b3HW\norWFUPh0uiWsfyjCE7vaeGlfe2ovlwVcqBq44aJlVFec2jqTA209bNrRitdjYVlmI3BTc1cq23qq\nHW6xRgtTmTi5pyocjY8p/1GoYLW0oYLKMj+793fQ1j4gc1ZCiJziiQQvvdHB/tYeguU+brzIVJwo\nFglYmI2/z716lKf3HMkICKcvqeadG1ewqL4w6UV27+9IBaukWDxB/3CUsoCX4XCMgeEow+EYFpiy\nH4nx0zwVgmVBfXUAj8dDKCrZ14UQ+RkcjvL0niOc6A1RG/Rz/YalVJb5cj/xFMzpgOU4Dnvf7GTT\njsyNvw21ZdyycQVqWW3WrOgno7LMS+9gZMzreT02sXiC4XCMHreQY/IR46V4KqQFteXUBgOpYJVO\nEoUKIUZzHIfmo33seL2dSCzBqsXVbDynEa8n33rAJ2/OBqxDx/p55MVDtLaPbPytCHi5/sKlXHz2\ngoyNv6fCsmBedRkDOFnzmwXLfQwMRxlwhwFhJLtEIsserEK766YzJPu6ECIvQ6EY218/Tmv7AF6P\nxcZzGlmztKZgN/a5zLmA1dUXYtOOFl59a6Rgsse2uGztQq45f+KNv5PltS1qqwKUB7wMkD2/WXnA\nyxXrFrFpewtYpsdVVW661e09w3nnRTyZ3LZlfo9kXxdC5JRIOOiWHvYcOEE0nqCxrpzLzl1I1SnO\n60/WnAlYoUiMp14+zPNNx4inpYNYu2oe77h4OfPy2Pg7GQGfh5qgP2Pp+0T5zdraB8YEjfKAl1Bk\n4qweJsGuRX11GSd6QxnnlsuKxiAgiUKFEONr7x5ixxvtdPWF8XttNp7dyJplU9erSjfrA1Y84fDS\nG8d5fFdbRpb0pQ2V3HrpSla4JTMKKVjuI1ieffJxvPxm2YJGsNyH32szMBTNWnTS67E4fXE156wy\nAS8cS9A7kF/dq5qgn1svW5lqE0iiUCHEiMHhKLv2d/C2m31n1eJqLlQNBR2FmqxZG7Acx3E3/rZk\nFHOrqfRz8yXLWXd6/Slv/B3NtqCmMkDAP/l9WuMFDTAbh98+1k/UndPyeW1WNmbf3PvrF97ml9ua\nM9JHJVmWmadb2lA55rmSKFSI0pNvpd7JCEfjbNrewiMvHiISS7ByYRUfuOEMVi+tKfh7TdasDFhH\nOwd59MUWDh5O2/jrs7lm/RIuP3cRPm/hV7N4PRa1wcAprZQZL2hMJpC869KVrFxYlXWI7w5JtSSE\nGIfjOLy0r52fPXWQzr4w1ZV+7r7pdC47d2HBb+5P1qwKWP1DEbbsbGOXTtv4a8EGtYAbNiwt2gRh\ntvmq6SRDfEKIyTjcMcCPtuxnX0sPXo/FLRuX865LV07r8F82M6s1JykaS7Bt71GefuUwkbS9S6uX\n1PDOS1ekKuYWw0TzVdNJhviEELkMh2P86rm32bKzlXjC4bzT6/ndG9bQWFe878xTUdIBK+E4vHLw\nBJt3tNKbVl9qQV05t1yynDMKuPF3NMsy82Fl/okvYVNzJy89pmk73kdDbbn0dIQQ085xHHa80c5P\nnzxAz0CE+TVlfOCGM1i/Zv50N21CJRuw3j7WxyMvHKKtYzB1rLLMy/UblnLRmY2pyr3F4LHNfFWu\nubCm5k5+/vRb+Lw2CUdSHgkhpt/x7iH+4zHNG4e68Xpsbrt8Je/cuKIo5UAKrSQD1v2P7RtT8ffy\ncxdxzfmLc/Z4TpXfa1MbDOSVjXjb3qPjHpeAJYSYSgnH4cldbTy49U0isQTrTq/nAzesYcEMHf7L\npiQDVnqwOndVPTdfvKzgG3+zqSjzTipje/py+szjkvJICDF1egfC/NuvXueNQ91Ulnn5yDvP4uKz\nFkzL5t9TUZIBC2DZgiC3XrqC5Y2F3/g7moUp3DjZFTPZcgea45LySAgxNd441M13fvkafYMR1q+e\nz4feoagJBqa7WSelJAPWF+6+gGC5b0ruDvKdr8pGUh4JIabTUy+38cCW/diWxe9et5obL1pWcr2q\ndCUZsObXlI8pZFgMp7q/KjlPtVOfoPV4v+yHmgOamjvd/W/DsipUTBvHcXhw65s8ur2F6gofn75j\nHauXTH+milNVkgFrKhRqf9Xa0+q59uKVdHT0F6BVYiZLrgpNklWhYjo4jsNPnjjIlp2tNM6r4PPv\nP48FteXT3ayCKH7FrRJjW1BXFZiRm4HFzDbRqlAhpsov3Y3Ai+dX8oW7L5g1wQqkh5WhEPkAxdwl\nq0LFdHt5fwcPb2umobaMP/md9ZNa1VwK5JvZVeb3MK+6TIKVOGkN49zJyqpQMRU6e0N87zdv4Pfa\nfOb2ddRVleZKwInItzNmvqo2GJgxyWtFaRpv9aesChVT4adPHmA4HOPOG9awdEFwuptTFHN6SPBU\n6lcJMZpkyRfT5UBbDzt1B6cvqebK8xZPd3OKZs4GLJmvEsUgWfLFdNi8oxWA375m9aweKZqTAavc\n76G60l/SG+iEEGLrnsMMh2O8vL+DFY1VrJkBVYGLaU4FLAuoqvBRUSZL1oUQs0NbxyAOsPGcxll/\nEz5nApZtW9RW+ksihb4QQuTrWKcpsbTu9Nk/FD0nApbfa1MT9OOxZb5KCDG7dPWF8XltGotYWX2m\nmPUBq6LMS9UUJcoVQoip5DgO/UNR6qrnxracWRuwTrYkiBBClIpoLEHCcSibI1tzZuW3+amUBBFC\niFKRrFohAatETaaEvRBClLJwxASswBxZTDarAtZkS9gLIWaHuVqHLOE4AHPmBn1WBCyZrxJi7prL\ndcgc9//nRriaBclvbdtiXnVAgpUQc5TUIZs7Svpb3uexqa2S/VVCzGVzug6Zk/shs0nJftOb+lUB\nCVZCzHFShwyYA3uwoEQDVmWZl9pgQDYDCyGkDtkcMiOGBJVSNvBt4DwgDHxMa31wvMdLPkCjFFdG\nZWszUPDzyHVtSvHaieykDtncMSMCFvBbQJnW+lKl1Ebg74H3THObZrRSXBmVrc0PbN6PBZS5i2YK\ncR65rk0pXjsxMalDNjfMlCHBK4DHALTWLwIbprc5M18prozK1raB4Sj9w9G8Hnsq75N+vBSvnRDZ\nuNuw5syy9pnSw6oGetN+jiulvFrrWLYH19VV4PWW1rBgQ0NVQV+veyCSNfVUz2Ck4O91MrK1IVub\n43EHLMYcP5XzyHVtinHtZsI1n0pz6Xxn6rm+99o1vLy/g80vtXLBmY0ztp2FNFMCVh+QfrXt8YIV\nQHf3UPFbVEANDVV0dPQX9DXrgn6Od49dzttYV17w95qs8c43W5s9HnNvGI0lMo6fynnkujaFvnbF\n+PvOZHPpfKf6XCcTdLq7hzjuti0ejc2av8lE12CmDAk+B7wTwJ3DenV6mzPzleLKqGxtC5b7qCof\nWwH6VM4j17UpxWsnRDahSDL57UzpexTXTDnLh4AblVLPY4ZjPzLN7ZnxSnFl1HhtznbsVM4j17Up\nxWsnRDb9gxEAghVjb/pmoxkRsLTWCeCT092OUlOKK6PGa3OhzyPXtSnFayfEaJ19YQDmVc+NTdIz\nZUhQCCHEJHX1m/RT86oC09ySqSEBSwghSlR3X5jygHfOJP+WgCWEECWqqz9EffXc6F2BBCwhhChJ\nQ6EYw+H4nJm/AglYQghRkuba/BVIwBJCiJLUNcdWCIIELCGEKEmpHpbMYQkhhJjJuvqSQ4LSwxJC\nCDGDdaeGBKWHJYQQYgbrGTABqzYoAUsIIcQM1jMQobLMO6cqsEvAEkKIEtQzEJ5TvSuQgCWEECVp\nMBSjNuif7mZMKQlYQghRomqkhyWEEKIUBLMUP53NJGAJIUSJqpgjWdqTJGAJIUSJmitlRZIkYAkh\nRImSgCWEEKIkSMASQghREgL+ufUVPrfOVgghZhGvPbe+wufW2QohxCzi8VjT3YQpJQFLCCFKlEd6\nWEIIIUqBx5YelhBCiBIgQ4JCCCFKgvSwhBBClASPZ259hc+tsxVCiFnEKz0sIYQQM9361fPnXAHH\nuZXXQwghZonPvm/ddDdhykkPSwghREmQgCWEEKIkSMASQghREiRgCSGEKAkSsIQQQpQECVhCCCFK\nggQsIYQQJUEClhBCiJIgAUsIIURJkIAlhBCiJEjAEkIIURIkYAkhhCgJluM4090GIYQQIifpYQkh\nhCgJErCEEEKUBAlYQgghSoIELCGEECVBApYQQoiSIAFLCCFESZCAJYQQoiR4p7sBs5lSyga+DZwH\nhIGPaa0PTm+rTp1Sygd8H1gJBICvAa8DPwAcoAn4lNY6oZT6MnArEAM+p7XeMR1tLgSl1AJgF3Aj\n5nx+wCw9X6XUF4DbAD/mM/w0s/R83c/z/ZjPcxz4A2b537dUSQ+ruH4LKNNaXwr8b+Dvp7k9hXI3\n0Km1vhK4BfgX4P8CX3SPWcB7lFIXAFcDlwC/C3xrmtp7ytwvte8Aw+6hWXu+SqlrgMuAyzHns4xZ\nfL7AOwGv1voy4KvAXzO7z7dkScAqriuAxwC01i8CG6a3OQXzM+BLaT/HgAsxd+EAjwI3YM5/s9ba\n0Vq3AF6lVMOUtrRw7gXuA464P8/m870ZeBV4CPgV8Gtm9/nux7TdBqqBKLP7fEuWBKziqgZ6036O\nK6VKfhhWaz2gte5XSlUBDwJfBCytdTLPVz9Qw9jzTx4vKUqpDwMdWutNaYdn7fkC8zE3V78NfBL4\nEWDP4vMdwAwH7gO+C3yT2f33LVkSsIqrD6hK+9nWWsemqzGFpJRaBjwF/FBr/WMgkfbrKqCHseef\nPF5qfh+4USm1FVgP/AewIO33s+18O4FNWuuI1loDITK/mGfb+X4ec75nYOab78fM3SXNtvMtWRKw\nius5zPg4SqmNmGGWkqeUagQ2A/9La/199/Bud+4DzLzWs5jzv1kpZSullmMC9okpb/Ap0lpfpbW+\nWmt9DbAH+CDw6Gw9X2Ab8A6llKWUWgxUAk/M4vPtZqTn1AX4mMWf51JW8sNTM9xDmDvz5zETtx+Z\n5vYUyp8DdcCXlFLJuaw/Br6plPIDbwAPaq3jSqlngRcwN0efmpbWFsefAN+djeertf61UuoqYAcj\n59HMLD1f4B+A77vn4sd8vncye8+3ZEl5ESGEECVBhgSFEEKUBAlYQgghSoIELCGEECVBApYQQoiS\nIAFLCFGylFIXKaVume52iKkhAatIlFJBpdS3lFIHlVKvKKWeVUpdn/b7rWn7PGYEpVTeS0aVUh9W\nSv0gx2MmdY55vuY17gbeGSO93Uqpt5VSK5VStymlvjrF7ViplHr7FF/jNqXUZ5VS31VKfS7t+KeV\nUo5SaknasRcm+vsqpd6llPof7r8vVkr9bR7v/z2l1NmTaPJfA/cqpTzu8xcppR5TSu1RSr2slLrO\nPX67UurTk3hdMQNJwCoCpZSFycEWAc7WWp8HfBb44UwLUqI4tNa/1FrfM93tmAylVACTpPlfgScx\nyW+TbsZsFr/ZfWw5cCbw/AQvuQGTzgjgbKAxj2acrbV+Pc/2ngl4gK3Au93Dfwf8Smu9HrgT+LFS\nyqO1/gVwu5txX5Qo2ThcHFcDK4DrkvnItNa7lVJfwySN3eo+7uNKqX9w//15rfVWtxf2DUxZg27g\nTq31CaXUB4HPYW4ydmHKHYSUUh2YTY6LMJs7H9Ba/xxAKbUL+Bgm59m/AvXAEPAZtz0rgQeAIPBi\nthNRStVgUhGtBt4ClgLvHfWYjcA/AWXACeATaWVUsp3jEuB7QC2wGPjBRF/uSqmbMJs7Q5h8b8nj\nZwD/BswDBjE3BQng21rrS5RSle41vFJrvV0p9R3gcUx5iF5MgtMlmAzdvwb2aK2XuK99GPgfWuuf\nuqU2Yu41/BawFvNF+bda6/8cp80fBq7RWn/Y7fX8EPNlXwl8UGu9Sym1FlPCwovJpHCL1nr1qNf5\nAPBnmLIXzcDd7t/9zzFZ8+OYQPJno563FvhnzN92AfB1rfV9SqmvABuB5cA/a63/Ne1pdwHPaK2j\nSqknMQl/cTfPngV8GrP5/fvuazyvtY4opa7G9HQqMH/TzwMHMHkIUUr1useCSqm/wNzM/Zt73iHg\nI1rrA0qp84BX3OccA/4bkxn9mPuen8V8/j6stX4as3H3+8Brblv/G7NZ/0n3fA5iPpNBzN/7F+5z\nvpzlTyZKgPSwiuMiYGda8sykZ9zfJQ1orc8HPgQ84N7hfhH4pNZ6A7AFuEApdQ6mRs9l7p1jO/Cn\n7mvMx3xxJnPc3QmglFqDKW2yG5Mb7c+01hcAHwd+4j73XzDBYj0m7Uw29wBaa30O8JfAuem/dL/M\nfgJ82u1J3gekf4lnO8c7gf/UWm90X+9zSqn52d7cffz9wPu01hcyUt4DTLD9ptZ6HeYL8UFM7aLF\nbqC9EhOwrnYffx2QTGC7zP39bcC9WusOoFUptda9c/emPe8dmID2RWCX246rgL9QSq0a57qN1qm1\nvti9Pn/uHrsfuMe9/m+R/Qbya8BN7ns2A2e6cza3YXow52NuJj456nkfA76mtb4IuBbT80gq01qf\nPSpY4b7mMwBa6+NAt1LqNExP6wVM9vIrlMlqfhXm8wnwGUyttwvS3vd191zv01r/A+Zz9Eut9V9j\n/lZ/737Gv4sJfmBSID3q/rsReNT97JQB73VLfXwF83kJuu39hdZ6D1CtlDpba/1zrXW3+xp/CuzW\nWifTLj3jPkeUKAlYxeGQ/cvH7/4u6XsAWuu9mCB0JvBL4CGl1L9g/mPbjPnCWQO8qJTaA7zHfWzS\ndvf/fwNcqkwW9TsxASKICZL/7j73x5g73XrgGuCn7nN/hCmrMNqNmN4BWuudjM2HeAbQrbV+yX3M\nz4DVbsDIeo5a63uBFqXUn2J6Zn5MzyObc4EjWus33J/vBzNHCKx2h3qS5Vu6AIX5Ir0GE6D+Ebja\nnRdp0Vr3ua+z2b2haML00AAeAa53n/dPwJXueTS6738D8En3Oj7jtvmccdo92mPu/zcB85RS84CV\nWutH3OPfz/40fgU8p5T6BvBz98v5ekzAH3KTKX/fPZbuT4Ayt3f4NUwvI2k72a0B2tJ+fgITrG7G\nXK9hTJqidZhgv9l93N3AWmXSdP3JqPfK5jfAvyilvofp+fzYPX49I70jGAleh9KOHwLq3IoBK9w2\nobW+OH0o0Z1/+wQm7yNpz12To21iBpOAVRzbgQ3KFP1Ldylm+C4pPXO7DUTdu9FrMMMZ33CHUDzA\nf2mt17t34xdjhmcASPuPNoL5grsNeD/mi8ADhJLPdZ9/CebL3WHkM+BghpdGizPx5yTb7yz3fbOe\no1Lq7zHDO4cwX6Yn3Odk44z6XfL1xntfL+YL8QbMl+q3MfMn78L0kpJCAKN6wcnnXYsZPooDH2Ck\nV+bBDMklr+NGRgJRLqFR5xNn/HNO0Vr/MXAHpqf4gFLqbsaee/K80/0XZuj2deAvRv1umOwcMm9a\nnsR8Zm9kpDe1GVMXallagHgW85nchRkanPC8tNYPAhdgchV+HrhPKVUNxLXW/WmPi6Q9Le8qB25w\n/wPgKq11a9qvomRWFRAlRgJWEWitn8WMq/9jMmgppS7EDCn9VdpD73J/twFTquCAUmo7UKW1/kfM\nvM0FmDmv9yqlFrgLOv4VM5+VzQ8xd7mdWutD7nDIAfeLDqXUjbjDPpj5nLvdf9+OGXoZ7XHMlzZK\nqXMx8zfpX/IaqFdKXeQ+5v3AIa1113jniPkC/Du3N6Yw80gestsLNLrzG+AOebo9pbeUUre7r78R\nWIjpwWzB9Ari7vnvwSTn/TUTexnTYzxDa70PUz7li2nPexL4Q/f9FrltW57jNbNy2/WmGlmS/QEy\nrytKKa9S6gBwQmv9dcyQ7/luO+5USpUrU1/tI25b092IGW58GDPURnIl3QQOYupCJW3FDItaWutj\n7rEtmJ7L8+5rzsNcs3swPaL3kHmz4h39b6XUT4GLtNbfwczpXuC29/Ec7cvJ7VldC1yutW4b9evT\n3HMUJUoCVvHcDoSBJqXU65ghpru11lvTHhNUSu3GjPV/QGsdxcxv/MBdMPFR4H9rrV/BzB89iQmE\nHuBvsr2p1vo5TO2iB9IO3wV8TCm1F/g68Dtuz+LTwB1KqVcwZVD6R78eJsCudp/7VcwEeOoOXWsd\nBn4HM8TT5L7m7+Q4x69jVkwmH78T82WS7XyimCD1Q6XUy5iJ/aS7gc8qpV7FzMfdrk0Npz6gFVMm\nA/e6DWitD2R7j7T3ctznJIcfn8SscktWnv1LoNxt95OYecE3J3rNHD4I3OOe1yWM6vm4w333AFuU\nUjsxPbq/1Vr/GhNEd2I+Dy2YBRbpvgJscz97VwJvM841TvMrzJd98v17MYtZnkh7zB7MIo4t7mO6\nMMO+r2GuWxVQ4S54eQa4Syn1GUxvaqNS6m+A/4OZ/3sZs8DoDzFBNd/ealbuzdyX3fZtdZe271Gm\nRAruuT18Ku8hppdkaxcTcntmzVrr55SpAfQ0cLrWWoZWTpFS6h7gu1rro25P8S6t9R3T2J4yTMC+\n1L1RmFWUUtswNzXt090WcXJkWbvIZR9mjsGDGf//hASrgmnB9J6imDmqj05nY9zl8n8N/BFmRGDW\nUEq9D1PTSoJVCZMelhBCiJIgc1hCCCFKggQsIYQQJUEClhBCiJIgAUsIIURJkIAlhBCiJEjAEkII\nURL+P+EoIr8OaweZAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x432 with 3 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.jointplot(x=dw_solar_everyday,y=ghi_everyday,kind='reg')\n",
    "#plt.title('observed dw_solar vs clear sky ghi')\n",
    "plt.xlabel('Observed global downwelling solar (Watts/m^2)')\n",
    "plt.ylabel('Clear Sky GHI (Watts/m^2)')\n",
    "plt.savefig('Figure3.png', bbox_inches='tight')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### making the Kt (clear sky index at time t) column by first removing rows with ghi==0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_train = df_train[df_train['ghi']!=0]\n",
    "df_test = df_test[df_test['ghi']!=0]\n",
    "df_train['Kt'] = df_train['dw_solar']/df_train['ghi']\n",
    "df_test['Kt'] = df_test['dw_solar']/df_test['ghi']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_train.reset_index(inplace=True)\n",
    "df_test.reset_index(inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "test Kt max: 18622.003582590547\n",
      "test Kt min: -1997.8414616673917\n",
      "test Kt mean: 3.2082558853348537\n",
      "\n",
      "\n",
      "train Kt max: 275783.5110086593\n",
      "train Kt min: -1561.168664736838\n",
      "train Kt mean: 3.977201231349393\n"
     ]
    }
   ],
   "source": [
    "print(\"test Kt max: \"+str(df_test['Kt'].max()))\n",
    "print(\"test Kt min: \"+str(df_test['Kt'].min()))\n",
    "print(\"test Kt mean: \"+str(df_test['Kt'].mean()))\n",
    "print(\"\\n\")\n",
    "print(\"train Kt max: \"+str(df_train['Kt'].max()))\n",
    "print(\"train Kt min: \"+str(df_train['Kt'].min()))\n",
    "print(\"train Kt mean: \"+str(df_train['Kt'].mean()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0xb8daac8>]"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAD3CAYAAADyvkg2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAGE5JREFUeJzt3XuQnXWd5/F3dzoXgh3oaIuiXFbR\nr1OWsgZXZIQhKyoiO8uWs39QluutLNddZnYYmVFU2OwIUyWuMioXYcHIRXGQQEYRI5kZIIRcCIFE\nrv4CuZA7dJLudCedpNPdZ/84Jz86SXefvhz6cvr9qkrVOb/n+5z+feE5z+e5nNNdUygUkCQJoHa0\nJyBJGjsMBUlSZihIkjJDQZKUGQqSpKxutCcwHE1NbcP66FRDw3Sam9srNZ0xyz6ry0TpEyZOryPd\nZ2NjfU1fyyb0mUJd3aTRnsKIsM/qMlH6hInT61jqc0KHgiTpcIaCJCkzFCRJmaEgScoMBUlSZihI\nkjJDQZKUGQrSGFQoFPjt0g2s29o62lPRBGMoSGPQ1h17ue/RdVx9x8rRnoomGENBGoM6OrtHewqa\noAwFSVJmKEiSMkNBkpQZCpKkzFCQJGWGgiQpMxQkSZmhIEnKDAVJUmYoSJIyQ0GSlBkKkqTMUJAk\nZYaCJCmr629hREwG5gKnAlOBq4HNwP3Ai6Wyn6SU7o6IOcCFQCdwaUppRUScBtwGFIBngUtSSt2D\nqa1gr5KkMsqdKXwW2JlSOge4ALgemAVcm1KaXfp3d0TMAs4FzgQuBm4orX8tcEVp/RrgosHUVqpJ\nSdLA9HumANwDzOvxvBM4A4iIuIji2cKlwNnAwpRSAdgYEXUR0ViqXVRadwHwCSANonb+cBuUJA1c\nv6GQUtoDEBH1FMPhCoqXkW5NKT0ZEd8G5gAtwM4eq7YBxwE1pZ1/z7EZg6jtV0PDdOrqJpUr61dj\nY/2w1h8v7HN8adnfmR/31lO19DkQE6XXsdJnuTMFIuIkikfsN6aU7oqI41NKLaXF84HrgF8DPTuq\npxgU3b2MtQ6itl/Nze3lSvrV2FhPU1PbsF5jPLDP8afntn1kT9XUZzkTpdeR7rO/AOr3nkJEnAAs\nBL6RUppbGn4wIj5Uenwe8CSwBDg/Imoj4mSgNqW0A1gVEbNLtRcAiwdZK0kaQeXOFL4FNABXRsSV\npbGvAT+MiA5gO/CVlFJrRCwGllEMmktKtZcBt0TEFOAFYF5KqWugtRXpUJI0YDWFQqF81RjV1NQ2\nrMl7alpdqqnP9dtauer2lQDMvfyjhy2rpj7LmSi9jsLlo5q+lvnlNUlSZihIkjJDQZKUGQqSpMxQ\nkCRlhoIkKTMUJEmZoSBJygwFSVJmKEiSMkNBkpQZCpKkzFCQJGWGgiQpMxQkSZmhIEnKDAVJUmYo\nSJIyQ0GSlBkKkqTMUJAkZYaCJCkzFCRJmaEgScoMBUlSZihIkjJDQZKUGQqSpKyuv4URMRmYC5wK\nTAWuBp4HbgMKwLPAJSml7oiYA1wIdAKXppRWRMRpw62taLeSpH6VO1P4LLAzpXQOcAFwPXAtcEVp\nrAa4KCJmAecCZwIXAzeU1h9WbWValCQNVL9nCsA9wLwezzuBM4BFpecLgE8ACViYUioAGyOiLiIa\nK1A7v7/JNTRMp65uUvku+9HYWD+s9ccL+xxfWvZ35se99VQtfQ7EROl1rPTZbyiklPYAREQ9xXC4\nAvh+aYcO0AYcB8wAdvZY9dB4zTBr+9Xc3F6upF+NjfU0NbUN6zXGA/scf3pu20f2VE19ljNReh3p\nPvsLoLI3miPiJOBh4M6U0l1Az+v89UAL0Fp6fOT4cGslSSOo31CIiBOAhcA3UkpzS8OrImJ26fEF\nwGJgCXB+RNRGxMlAbUppRwVqJUkjqNw9hW8BDcCVEXFlaeyvgR9HxBTgBWBeSqkrIhYDyygGzSWl\n2suAW4ZaW5EOJUkDVlMoFMpXjVFNTW3DmrzXK6tLNfW5flsrV92+EoC5l3/0sGXV1Gc5E6XXUbin\nUNPXMr+8JknKDAVJUmYoSJIyQ0GSlBkKkqTMUJAkZYaCJCkzFCRJmaEgScoMBUlSZihIkjJDQZKU\nGQqSpMxQkCRlhoIkKTMUJEmZoSBJygwFSVJmKEiSMkNBkpQZCpKkzFCQJGWGgiQpMxQkSZmhIEnK\nDAVJUmYoSJKyuoEURcSZwDUppdkRMQu4H3ixtPgnKaW7I2IOcCHQCVyaUloREacBtwEF4FngkpRS\n92BqK9WoJKm8smcKEfF14FZgWmloFnBtSml26d/dpaA4FzgTuBi4oVR7LXBFSukcoAa4aDC1lWhQ\nkjRwAzlTWAt8Griz9PwMICLiIopnC5cCZwMLU0oFYGNE1EVEY6l2UWm9BcAngDSI2vnDbVCSNHBl\nQyGldG9EnNpjaAVwa0rpyYj4NjAHaAF29qhpA44Dako7/55jMwZR26+GhunU1U0qV9avxsb6Ya0/\nXtjn+NKyvzM/7q2naulzICZKr2OlzwHdUzjC/JRSy6HHwHXAr4GeHdVTDIruXsZaB1Hbr+bm9sHO\n/TCNjfU0NbUN6zXGA/scf3pu20f2VE19ljNReh3pPvsLoKF8+ujBiPhQ6fF5wJPAEuD8iKiNiJOB\n2pTSDmBVRMwu1V4ALB5krSRpBA3lTOF/ANdHRAewHfhKSqk1IhYDyygGzSWl2suAWyJiCvACMC+l\n1DXQ2qE2JUkamppCoVC+aoxqamob1uQ9Na0u1dTn+m2tXHX7SgDmXv7Rw5ZVU5/lTJReR+HyUU1f\ny/zymiQpMxQkSZmhIEnKDAVJUmYoSJIyQ0GSlBkKkqTMUJAkZYaCJCkzFCRJmaEgScoMBUlSZihI\nkjJDQZKUGQqSpMxQkCRlhoIkKTMUJEmZoSBJygwFSVJmKEiSMkNBkpQZCpKkzFCQJGWGgiQpMxQk\nSZmhIEnKDAVJUlY3kKKIOBO4JqU0OyJOA24DCsCzwCUppe6ImANcCHQCl6aUVlSitnKtSpLKKXum\nEBFfB24FppWGrgWuSCmdA9QAF0XELOBc4EzgYuCGStQOvz1J0mAM5PLRWuDTPZ6fASwqPV4AfAw4\nG1iYUiqklDYCdRHRWIFaSdIIKnv5KKV0b0Sc2mOoJqVUKD1uA44DZgA7e9QcGh9ubb8aGqZTVzep\nXFm/Ghvrh7X+eGGf40vL/s78uLeeqqXPgZgovY6VPgd0T+EIPa/z1wMtQGvp8ZHjw63tV3Nz+2Dm\nfZTGxnqamtqG9RrjgX2OPz237SN7qqY+y5kovY50n/0F0FA+fbQqImaXHl8ALAaWAOdHRG1EnAzU\nppR2VKBWkjSChnKmcBlwS0RMAV4A5qWUuiJiMbCMYtBcUonaoTYlSRqamkKhUL5qjGpqahvW5D01\nrS7V1Of6ba1cdftKAOZe/tHDllVTn+VMlF5H4fJRTV/L/PKaJCkzFCRJmaEgScoMBUlSZihIkjJD\nQZKUGQqSpMxQkCRlhoIkKTMUJEmZoSBJygwFSVJmKEiSMkNBkpQZCpKkzFCQJGWGgiQpMxQkSZmh\nIEnKDAVJUmYoSJIyQ0GSlBkKkqTMUJAkZYaCJCkzFCRJmaEgScoMBUlSVjfUFSNiFbC79HQ9cDPw\nI6ATWJhS+vuIqAVuBE4HDgBfTim9FBEfHmjtUOcnSRq8IYVCREwDSCnN7jG2GvgLYB3wQETMAk4F\npqWUzioFwQ+Ai4CbBlErSRohQz1TOB2YHhELS6/xf4CpKaW1ABHxIHAe8Fbg9wAppeUR8cGImDHQ\n2nKTaGiYTl3dpCG2UNTYWD+s9ccL+xxfWvZ35se99VQtfQ7EROl1rPQ51FBoB74P3Aq8C1gAtPRY\n3ga8A5jBa5eYALpKY60DqY2IupRSJ31obm4f4vSLGhvraWpqG9ZrjAf2Of703LaP7Kma+ixnovQ6\n0n32F0BDDYU1wEsppQKwJiJ2AzN7LK+nGBLTS48PqaUYCPUDqe0vECRJlTfUTx99ieI1fyLiRIo7\n9L0R8c6IqAHOBxYDS4BPleo+DDyTUmoFOgZSO+SuJElDMtQzhZ8Ct0XEY0CBYkh0A78AJlH8RNHj\nEfEE8PGIWArUAF8srf/VQdRKkkbIkEIhpdQBfKaXRR8+oq6bYgAcuf7ygdZKkkaOX16TJGWGgiQp\nMxQkSZmhIEnKDAVJUmYoSJIyQ0GSlBkKkqTMUJAkZYaCJCkzFFQRnV3drH5pBx0Hu0Z7KpKGwVBQ\nRTy4YiM/nvc0dz/kX1CVxjNDQRWxYXvxD4Ss2dRSplLSWGYoSJIyQ0GSlBkKkqTMUJAkZYaCxrzn\n1u9i6469h40VCgVWvPAKre0dozQrqToZChrTugsFfnD3aq649fHDxv/w0k5u+vVz/OCfVo/SzKTq\nZChoXNqxex8Am17dM8ozkaqLoSBVkYOdXfzx5Wa6C4XRnorGKUNBqiK3/z7xvV+uYukz20d7KiPi\ngWUb+O3SDaM9japiKKiiugsFurq7R3saE9bTa3cCsGF76yjP5HCvNrfzs9+9wJ59Bwe8zqZX99C+\nv//6exet475H1w1pTmljM4+s2jKkdauZoaCs42AXP1+Yjvqkz2Bs29nOf/+/iyo4K1WDG+c/y+Kn\nt3H9fc8cdmmrue0Aa7fsPqp+5+79zJm7gr/5x9dvW7rmrlXc8WCis8uDmJ4MBWWLVm/loae2cM1d\nTw3rdboLBV5t2cf6bWPraFWjZ0/piH/Nphb+7cnNefyyG5bwD3c+yb4DnYfVN7cdABjWAYqGxlAA\n1m7ZzV/98FHWba38Tqx9/0G6u8fmTb9rfvEUt/72+fx8f0fxjdnWPvBT/L5cftMyrrp95aDX6+8G\n6a7W/b1eFtnRsq/s6za3Hcj9jQWPrN7C6hd3jPY0Kq67UOAX/7KGtLG5z5rn1+86auz1+pXrDz21\nmWXPDu7+SmdXN9++ZTkPLNvwusxprJvQoXBoZ33Pwy+xd38n8x6p7K99PtDRxV/+cDH/cOeTFX3d\n/nQXCoft/Ha17ueJ5197U2zbuZd7HnmJzq5u0qYWlj67nT++3PcbuOzP6y6wflvrgIPvQEcX9y9Z\nz+49B1jw+Mv8bvnLedn9S9bz5WsezkeJR/rbG5fyndtWUjgiOL5+0zJ27t7f9xwLBS67YQmX3bCk\nbC+7947Ml+Hu+H3ix/c+PSI/q9Ka2w7wtesfY9Wapjz2SnM7q1/cwbqtrfzbk5u55q5VozjD1/x8\n4Rpu+e3zR20zR7r6jpX87HcvALBj93627Wzn3kVDu1cx3o2pUIiI2oi4KSKWRcQjEXHa6/Wznlu/\ni4v+7jc81WPDHqxCoZA3tgXLX+a6e58+bONr21fcwYzUZZRCocCXr3mY/3nto+zeU9yx/u2NS/nO\nTx/n3kVr+fpPljJn7goWLN942NHT935ZfAPv63jtaK257QBPrz36SLatveOwa7APLNvAVbevZFUf\nR709LxUA3P3Qi8xfvJ6bf/Mc9zy8lnmPrGVz0x4KhQLzF68HipcUdrTso7Orm3sePjqoe3t7/91P\nlrKjZR+tezvyje6fPvA83/35kxzsLD7fd6D/o9Hr73uGv7nuMZpa9vHI6i3c9S9req078lvUhUKB\n+x5dx4ubh/Zrww92drN9V3ufyzu7unlp8+5Bfcz0oae2MGfuCg50VPYIvFAo8NBTm2nZ08F19z2T\nx75583J+fO/TR23rdy5MLH9uOz2nvmF721G/Yn196VevH7K56bXvnxz6LkpnV3f+b7Blx16++4vX\nLnP2drbYczv9Xz9azA/v+QMHO7v42vWPHfaJpRc3tbBuayuLn9521Gu07u3gX1duYt3W1sPOZlb+\n8VX+8Vd/4Jl1O7n1t88P6mrAkf9PygVWueWVVjPSP7A/EfFp4D+nlL4QER8GvplSuqiv+qamtiFP\n/ju3PcGG7W286bhpHDO1jk2v7uGYqXVc8bkz2Luvk917O9jf0UlTyz7+4wfeRncBtjTtYX9HF2lj\nC+8++Xhu/vVzdBcKfOMzH8hHRv919jt554kzaD/QSfv+Tn76wAv5Z5705jfwJ6c08NSaJs5+/1tp\naz/IY09vY+aMqUyuq6V++hSa2w7wvnfMpLsbJtfV8r53zOSR1Vvp6uqmoX4adZNqWLO5hQMdXZz0\n5jfQ2n6QWe9uZFfrfh5Y9nJf7ZbVUD+1zyP03px3xtvpONjV6xvpSMdMncS+A10c94Yp7N7z+h2J\n19bWlH1z/qc/PZVjp9Xx+8c38qmzTuG9p87k/93/HAc7u9m2s7hj/tz5wR0PJgC++9WzqAFe2ryb\n3Xs7aNlzgIVPbOJjH3w75816O8dMq+O59bu45f7iZbgf/tXZUAMUoGXPAY5/w1Sg+MbuLsCKF16h\ntraGX/7riwBc+5cf4Zq7VvHKrnY++aGTmTK5lmOPmczG7W0sKQX3O06cwbqtrbzn5OP55Jknw6RJ\nbNjSQsMbplIAamtqOHZaHd2FAjfMf3bQ/93e/8438vTanZwwczqvlMLpA+96E9OmTKLx+GP4zZIN\nfa572tuO46VebhRX2kfe9xaWlPmY7RcueA+P/mErzW0H+Oist/V6pP/GGdPY2dr3WeVb3zg9bweD\nccLM6Rw3fTJvnjmdGdOn8LvlL3Pim47lwrNOoW1vB1OmTKJ1bwf/XDrwAfjeV8/i4dVbWLB8I5d/\n/j9QP6WWppZ9zDh2CrU1NRzs7OaPG5u5d9E6PvfJYNrkScycMY1pUyaxq+0Ab505nRNmTh/0XAEa\nG+tr+lo21kLhWmBFSumfSs+3pJTe1lf9UENh06t7mDN3xRBnKUmj772nNnDZxR8Y0rr9hULdkGf0\n+pgB9Dzs6IqIupRSr3cIGxqmU1c3adA/ZNqxU5lSV0tHZzd/cupMamtreG5d8fPdM2dMZVfr4UfM\n/+7EGZx0Qj2P+pnmCeHs00/ksT9sBSidwU0+aps45E/f/1aW9jhbamw4hneddDwHO7t54vlXOOUt\n9bz9hHpqa2qoqeGobejs00/k5e2tbHplT94mNfG8/7Q38eKmlqM+hdWXd510PH9x3rtpbKyv+FzG\n4pnC8pTSr0rPN6eU3t5X/XAuHwE0NtbT1NRWvnCcs8/qMl76LBQK3PFgIk46ng+/9y1Deo3x0utw\njXSf4+lMYQnw58CvSvcUnhnl+UgaopqaGj7/yfeM9jQ0SGMtFOYDH4+IpRRv131xlOcjSRPKmAqF\nlFI38NXRnockTVRj6nsKkqTRZShIkjJDQZKUGQqSpMxQkCRlhoIkKRtT32iWJI0uzxQkSZmhIEnK\nDAVJUmYoSJIyQ0GSlBkKkqTMUJAkZWPqV2ePhIioBW4ETgcOAF9OKb00urMqLyLOBK5JKc2OiNOA\n24AC8CxwSUqpOyLmABcCncClKaUVlagdwR4nA3OBU4GpwNXA88Od/1jrNSImAbcAAXRR/LshNdXW\n5yER8WbgSeDjpbkNa+5juM9VvPbnhNcDNwM/Ks1zYUrp7/va/5T+qNiQayvZx0Q8U/gvwLSU0lnA\n5cAPRnk+ZUXE14FbgWmloWuBK1JK51DcmVwUEbOAc4EzgYuBGypR+3r3doTPAjtLP/8C4Prhzn+M\n9vrnACmljwD/uzSXauzzUNDfDOyrxNzHcJ/TAFJKs0v/vgjcBHwGOBs4szT3vvY/w62tmIkYCmcD\nvwdIKS0HPji60xmQtcCnezw/A1hUerwA+BjFvhamlAoppY1AXUQ0VqB2JN0DXNnjeWcfcxrXvaaU\n/hn4SunpKcArFZj7mOuz5PsUd2JbS8+rtc/TgekRsTAiHoqIPwOmppTWppQKwIPAefSy/4mIGRWo\nrZiJGAozeO0UD6ArIsb0ZbSU0r3AwR5DNaUNAqANOI6j+zo0PtzaEZNS2pNSaouIemAecEUfc6qG\nXjsj4nbgOoq9Vl2fEfEFoCml9GCP4arrs6SdYgCeT/GvR/6sNHZIX/PvKo21DrO2YiZiKLQC9T2e\n16aUOkdrMkPU81ppPdDC0X0dGh9u7YiKiJOAh4E7U0p39TGnqug1pfR54N0U7y8c08t8xnufX6L4\nN9cfAf49cAfw5l7mM977BFgD/Lx0BrOG4s58Zi9zOmr/08vYUGorZiKGwhLgUwClGzbPjO50hmRV\nRMwuPb4AWEyxr/MjojYiTqYYdjsqUDtiIuIEYCHwjZTS3NJw1fUaEf8tIr5ZetpOcYe2str6TCn9\nWUrp3JTSbGA18DlgQbX1WfIlStf8I+JEYDqwNyLeGRE1FM8gDs3/sP1PSqkV6BhmbcWM6csmr5P5\nFI9ellK8IfXFUZ7PUFwG3BIRU4AXgHkppa6IWAwsoxj2l1SidsQ6KvoW0ABcGRGH7i38NfDjKuv1\nPuBnEfEoMBm4tDSHavx/eqRq3XZ/CtwWEY9R/ATUlyiG/S+ASRTvgzweEU/Q+/7nq8OprWQj/ups\nSVI2ES8fSZL6YChIkjJDQZKUGQqSpMxQkCRlhoIkKTMUJEnZ/wf5c9xhMd1wWAAAAABJRU5ErkJg\ngg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(df_train['Kt'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0xc11cda0>]"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD3CAYAAAD10FRmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAHGdJREFUeJzt3XmcVOWd7/FPr2BDg6AdFfclPs4r\nixnNBExcuPE6XpOZYa6Ze++8Zu68Rk0mVy8zkcQxExON5E7GCSZiBHdBGDE4RgzghqAC2oAsQqMs\nzdMsDU03DTS901t1ddX9o6r3qu6q6u461fV836+XL6tPPafq91Dd33rOOc85JyMYDCIiIm7J9LoA\nERFJPoW/iIiDFP4iIg5S+IuIOEjhLyLioGyvC4hVVVVjwtOSJk3Ko7a2eTjLSWnqb3pzqb8u9RVG\npr8FBfkZkZY7MfLPzs7yuoSkUn/Tm0v9damvkNz+OhH+IiLSm8JfRMRBCn8REQcp/EVEHKTwFxFx\nkMJfRMRBCn8REQcp/EU80NDk47X1Bzjd0u51KeIohb+IB15abVm1uYxX1+73uhRxlMJfxAO1jW0A\n1Df5PK5EXKXwFxFxkMJfRMRBCn8REQcp/EVEHKTwFxFxkMJfRMRBCn8REQcp/EVEHKTwFxFxkMJf\nRMRBCn+RNPLO5iM88vJ2AsGg16VIisv2ugARGT7L1h8EoKmlnfy8XI+rkVSmkb+IiIMU/iIiDlL4\ni4g4SOEvIuIghb+IiIMU/iIiDlL4i4g4SOEvIuIghb+IiIMU/iIiDlL4i4g4SOEvIuIghb+IiIMU\n/iIiDlL4i4g4SOEvIuIghb+IiIMU/iIiDorpNo7GmKnAHGvtdGPMNcCbwP7w089Ya181xjwMfBvw\nA7OstVuNMVcAi4EgsBuYaa0NRGo7rL0SEZEBDRr+xpgfA38HNIUXXQPMtdY+1qPNNcBNwFTgQuB1\n4E+AucCD1tr1xphngRnGmCNR2oqISJLEMvI/CNwOLAn/fC1gjDEzCI3+ZwHXA2ustUGgzBiTbYwp\nCLf9MLzeKuBPARuprbW2ath6JSIiAxo0/K21rxtjLumxaCuwwFq73RjzM+BhoA6o7tGmEZgIZIRD\nvueyCVHaDhj+kyblkZ2dNVi5URUU5Ce87mik/qa2nJzQ4bbc3OyEah9snbPOGs/E8WMSqi3VjLbP\ndqiS1d+Y9vn3sdxaW9f5GJgPrAR6VpxP6AshEGFZQ5S2A6qtbU6g1JCCgnyqqhoTXn+0UX9TX3t7\n6E/D5/PHXXss/a2uPo2vxZdwfaliNH62QzES/Y32ZZLIbJ/VxpivhR/fDGwHNgK3GmMyjTEXAZnW\n2lNAkTFmerjtbUDhAG1FRCRJEhn53wM8aYzxAceB71trG4wxhcDHhL5QZobb3ge8YIzJBYqBZdba\njihtRUQkSWIKf2vtYWBa+PEO4OsR2swGZvdZVkJoZs+gbUVEJHl0kpeIiIMU/iIiDlL4i4g4SOEv\nIuIghb+IiIMU/iIiDlL4i4g4SOEvIuIghb+IiIMU/iIiDlL4i4g4SOEvIuIghb+IiIMU/iIiDlL4\ni4g4SOEvIuIghb+IeKah2ceB8nqvy3CSwl9EPPPT5zbzyMvbqW1s87oU5yj8RcQzzW1+ABqbfR5X\n4h6Fv4iIgxT+IiIOUviLiDhI4S8i4iCFv4iIgxT+IiIOUviLiDhI4S8i4iCFv4iIgxT+IiIOUviL\niDhI4S8i4iCFv4iIgxT+IiIOUviLiDhI4S8i4iCFv4iIgxT+IiIOUviLiDhI4S8i4iCFv4iIg7Jj\naWSMmQrMsdZON8ZcASwGgsBuYKa1NmCMeRj4NuAHZllrt8bTdpj7JSIiAxh05G+M+TGwABgbXjQX\neNBaewOQAcwwxlwD3ARMBf4aeCqBtiIikiSx7PY5CNze4+drgQ/Dj1cB/xW4HlhjrQ1aa8uAbGNM\nQZxtRUQkSQbd7WOtfd0Yc0mPRRnW2mD4cSMwEZgAVPdo07k8nrZVA9UxaVIe2dlZg5UbVUFBfsLr\njkbqb2rLyQmNu3JzsxOqfbB1zjprPBPHj0moNi9Mnjwuap9G22c7VMnqb0z7/PsI9HicD9QBDeHH\nfZfH03ZAtbXNCZQaUlCQT1VVY8Lrjzbqb+prbw/9afh8/rhrj6W/1dWn8bX4Eq4v2Wpqmhif039H\nxGj8bIdiJPob7cskkdk+RcaY6eHHtwGFwEbgVmNMpjHmIiDTWnsqzrYiIpIkiYz87wNeMMbkAsXA\nMmtthzGmEPiY0BfKzATaiohIksQU/tbaw8C08OMSQrN1+raZDczusyzmtiIikjw6yUtExEEKfxER\nByn8RUQcpPAXEXGQwl9ExEEKfxERByn8RUQcpPAXEXGQwl9ExEEKfxERByn8RUQcpPAXEXGQwl9E\nxEEKfxERByn8RUQcpPAXEXGQwl9ExEEKfxERByn8RUQcpPAXEXGQwl9ExEEKfxERByn8RUQcpPAX\nSXOBYJA2X4fXZUiKUfiLpLnfvFLEPXM/xNeuLwDppvAXSXP7yuoAON3S7nElkkoU/iIiDlL4i4g4\nSOEvkmRt2vcuKSDb6wJEXPLGhlJWbCglMyPD61LEcRr5iyTRig2lQGj6pYiXFP4iIg5S+IuIOEjh\nL2mjubWdVZuP0Nzq97oUkZSn8Je0sfT9/by2/iC/X3fA61JEUp7CX9LGyboWAKrC/xeR6BT+IiIO\nUviLiDhI4S8i4iCFfwo4UdPMXb9ay8d7jntdiog4IuHLOxhjioD68I+lwHPAE4AfWGOt/YUxJhN4\nGrgaaAO+Z609YIyZ1rftEPow6m3YVQnAi28Xc90XzvW4GhFxQULhb4wZC2Ctnd5j2U7gO8Ah4G1j\nzDXAJcBYa+114cB/DJgBPNu3rbV2xxD6ISIicUh05H81kGeMWRN+jdnAGGvtQQBjzGrgZuA84F0A\na+1mY8xXjTETorRV+EtaOXSsgdMt7Xz58rO8LkWkn0TDvxn4DbAA+DywCqjr8XwjcBkwge5dQwAd\n4WUNEdoOaNKkPLKzsxIsFwoK8hNed6Tl5eV2PR6uOlO5vyOhoCCfnPDvR25uVkr0/65frQXgzcdm\nRG2Tm5udUK2DrXPWWeOZOH5Mv2Vnn3lG3O+VDJMnj4vap1T4LJMpWf1NNPxLgAPW2iBQYoypByb3\neD6f0JdBXvhxp0xCwZ8foe2AamubEyw19I9ZVdXYb3lbewe52ZlkeHx53eZmX9fjSHXGK1p/01Vn\nf9v9oevk+3wdKdX/gWrx+fxx1xrL51tdfRpfi6/fsmB7al76oqamifE5/eefuPq7PNyvGUmis33u\nIrT/HmPMFEIh32SMudwYkwHcChQCG4FvhdtNA3ZZaxsAX4S2SdXc6ueexz5k/uu7kv3WIiKeS3Tk\nvxBYbIzZAAQJfRkEgN8BWYRm8GwxxmwDbjHGbAIygDvD69/dt+0Q+pCQ6oZWAHYeOJXstxYR8VxC\n4W+t9QF/E+GpaX3aBQgFfd/1N/dtKyIiyaOTvEREHKTwFxFxkMI/Ti1tft7cWEpDs2/wxiIiKUrh\nH6cVhaUsLyxl8Tv7vC5FRCRhCv841TSGZgnphiEiMpop/MV57f4Az7+xhwMV9YM3HkD96Tba/YFh\nqkpkZCn8xXlbi0+wee8JHlmyPeHXaPX5+eGTG5m9aOswViYychT+4jx/x9BH600tocsmVFYnfhmS\n0eb5N/bwwHMfe12GJCjh6/mLiNs27z3hdQkyBBr5i4g4SOEvIuIghb+IiIMU/iIiDlL4i4g4SOE/\nygSDQa9L8FTR/ipeW3fA6zJERj2F/yiydkc5352zjprwjWhcNP/1XazaUsbplnavSxEZ1RT+o8jL\na0oAKNqvu4+5vgUkMlQKfxERByn8RUQcpPCXUamiqsnrEoZNbWMbs+ZvYLs96XUp4hCFv4xKj75S\nlDZ3U9vw2TEamnw8tXy316WIQxT+Mmo1NmvGT6fTLe0cr0mPK4pW17fyh48O4WvviHvdhiYfxYdr\nRqCq9OPkVT01UyT9tfs7yMrKJDMjw+tSkmLWvA0EYvi9LjvRyPGaZr72R+ckoarEPLHsU8qrmhib\nm8XfTzkzrnV/vnALDc3t/Pv3p3HO5LyY1+sIhC7rnZXpznjYnZ6GPfafRdz31Eavy5AR9n9+8yH/\nb/G2Adu0++MfWaaqWIIfYPaibTy7cg9tvtTte1Vd6DyWxjh36wWDQRrCW4P1TfGt+4MnCrn3iQ1x\nrTPaOTfy33O41usSJEnKTpyO+lzxkVp+/UoRf3vLlWRn9d86qG1sIyszgwnjckeyRM+ERrpZXpcx\nbE7WNvOT5zYnvH5LW+p+GY4U50b+MnoVlVQN22tt2XscgLc/Phzx+fue2sis+Rv4YHv5sL2njJwd\nJTrxMV4K/yT7zw/28+Qfdnldxqg034N/t9+9V5L090yWjz49xivv7x+R125p87Npd+Ww3CJTRobC\nP8nWbDvKjmEcwYocqKiP+txAM2YWr9rHe58c7bVsa/EJahvbhlzTolX7WPBWMe9tOzp44xHwygf7\n9Xc2CIW/SA8VVdGPEyTqrU2HeeC5j0dsFPzIku1Rn/v5wq0xv87BinqeXbmH2YtiXyea0mOhL6Rj\n1d6cjHfkeGPStrADgWDXbKHRROEvo4ItG94D9dEmxzwUR1gCtPr8PLp0B7tKq3strz/dPXr+w0eH\nOFHbwqn65F+N9WRdS8xtO2fI6PyJ+Nz/zCZmPv6R12XEzbnZPjI6zVlaNCKvmzHE8wC2Fp9kX1kd\n+8rqei3/4ZOjZzpxIBhk4VvFZGUm/m/R0uYnNyfTqXnynYZjN5kX3Puk0sBencE4Ku0pTc3Prfzk\naT7ec5wNuyq7lq3bUd4v1A4da+DZlbv7nSMQCAaZ+fhHPLRg6LuLJHkU/qNQ0f5TNLW6s2k+nGdk\nt7V38NnB6phPigLYtLuSZ1bs7lVHbWMbc3+/k8Wr9sX8OrHMHKqKYzfNcIn0T7FkTQlzX93Za9m/\nvfQJW4tPUvjZsT7rh16g5+UlXDyHfskaS0ub3+syYqbwH6Lt9iQvrbZJv2TELxYNfPZquij89Bjf\nnbMurnXa+sxwaWnzd50t+vIay29f+5SNu45HXb/vXcIWvFXMtn0nqQ7fQa0jEOCFN/ew+9Dwj+R/\n+dInUZ8LBIIEAvF9afVUWtkYVy0Vp3ofrO18Z39H9BpeXmM5dKyh6+cMuncl+TsCzHz8I5aO8umz\n/o4An+w72W8LaN2OCt7dUuZRVfFT+EfQ7g9gy2pjGh0+tXw364sqqGnov9+vtLKBR17eHvNtFzsC\nQY6ejG22yan61lE5wyAeyz86xKI4RtZ14d0UPcMHYObjH3HvvNCp+3v7nOHt7wjw1qYjvZbd/8ym\nqO+xdkc5//Do+n77+IdLY3M7v193IOIB7nvnFXLf07EdSwgGQ19aPT21fORnv6zdURH1C6yhyUdL\nm5/3t5dTWtnAQwu29Hq+3d8R2q3U48s7EAim3LW4Vm0+wtMrdvPKB/2/xDTyH+Veeb+EOUuL+OjT\nY4M3DguGx0U/X7iFxatCf3RP/mEXB8rrWbmhNObXWVdUEXPbB1/YMnijUSQYDLLncA2/fqWI1VvL\neHPT4UHb91SdwL2NG5vb+6030HVvlq0/GPd7DKShycfywt6/H+9uKWPO0iKOHG/kP97dxwfby/nl\nS5/Q1Oqn/nRs16yZs3THsNbZU9H+Kp5d2X356Z0J3FZ00Tv7+m1ZbNt3kq3Fve9pcOe/rubfX97B\nqboWXnynOO7LeL+6dn/Ez/O9bUe576mN/bYSY1EWHqAdrGgYpGVq02yfCD47FJq2d6iigelfOT+u\ndcurmiivauKO2/5oJErr5URt8vcPD5eDx+ppbw9wzuQ8gsEg487I4Z+f2khTa2jkVHxk8KmdP1+4\nlfmzbqAgxvf87GD14I2SbKDrD/1igAvTnahtprWtg4vPzY/4fGV17Jd3bvXFN1rdX977pLKB7kOw\nYVcl//3Gy5iUP2bQ1420O6mmoY2ahjYWvF1MydE6jp44zYzrL+XKC89k6fsl3DbtYs4/e1zU11y9\n9SjjxubwZ1+/pNfyVz4IndlceqyBqy6eNGht6Ujh75GGZh8T8oZ+0bCyE41MOXsc/o4AvvZAvwuR\nBQJBMjMz8HcEyM6Kf0MvGAzS6uugur6VCz43vt/z/o4AmRkZZPaZJtjc2k5mZgZjcrJ6TacMBIPM\nfXVnv90vE8fldgV/PMpPnuaSCyeHix247W9f+zTu1+8p1n3mI31Jg32Ha3ggfBGzF3/yzYReY/3O\n7q3aWKbRxnO+APQ+iPzJvpO0tPk5OsAJdCs3lPbbEli9tfvs4M4JDkdONDLv9c/49nUXs2n3cfaV\n1fLwHX8yYC2d6x6vaWZPaQ3fvKb3gG7jrspIq424nQdOsXZ7Of/0nS+Tk538nTBOhX+0zXlfewe5\nOQNf4bAjEEh4DvOWvSe481u9twR+OG8DCyP84R6vbqLsRCMXnRN5RNfX7D4Hfp+97yYWrNzNFVPy\nqWloZdE73fvM/+9ffpGvXvW5fq/x1qbDXH7+RD49cIobr57ClPBIatWWI7y27iCZGRkEgkEem/kN\nJuWPoaahlTlLd3RderfgzLHMufvrXa/na+/gH39bCMAXL53Mj/7XV+gIBJjzuyIuPW9Cv+CH+C/B\n29Pug6d4fvmuEZ9h8syK2O609dPnE7+6ZCzun1/Y9bgywTNoY919FQgEOXy8kQVv7Y3r9ftOE13R\nZ9dneZ8vgqJBdh019TkI37m7pqahret4zmB+9vxmgsDFff62Fr5dHHmFQQSCwSF9ccxb9hkQ2iK9\n1sS6/Tp8nAl/X3sH98z9MOJz9z+ziX/7h2mMPyOn1/JDlaF9ep2Xi51x/aVdz1Wcauo1wis5WsfZ\nE8+I/N7+ACdqmzlnUvfNJYKE5lLf+JUpvdruK6tj9qJtXHnBRP7+tqs476zom7SRvPhOcb/9pp2e\nXrGbZ350E2NyQ190L4enpn2850RXm427Kvnl96ayrqiCNzYeBrqvFV/f1Mak/DH889O9D4hW1bUy\nb9lnfONL53Kt+Ryb93a/3u7w3PbKU80cqKgf8Do0iXpogEsnbN4bfVbPSEnmmbw/G+HjPis2lPLW\nIMdeBlNaOfR943V9jnUUx3lp9mAw2DU4qKzp/sJcUXgo5vULP6vksikTupZVVjf3++J4f3s5075w\nLvl5OYw/I4czxsQSsf2HLSfrWnh/21Fuv+kyxuaOTEx7Fv7GmEzgaeBqoA34nrX2wEi934790S/y\n1Njczg+eKGT+rBsYN7b7C+DYqSaaWtu7AmzlhlK+dNlZXc/3HAkueKuYz1/Qfdeh/3h3X6/RzwPP\nbebmay7o9b5L1pSwZE3kaW8l5fUJ/WFHC/5O98z9kLMnjuVvb7mStTv6H1xuavUndHbqzgOn2Hng\nFN+56TJe/7D3H9TmPcdHbHbMsvUHB5yV9fwb8Y1YpbehBj/QazAwXPruIhpIMAiz5ndvHfTcGi7p\nc/zivU+OUna8kRN1LeRmZ3LF+RO57ovnsvtQTcxXeO2c7TRhXC4/+7trCQSCFB+p5dipJv5q+uVk\nZmZQUdVdf0tbB6WVDVx63gQOVdSzcOUuDlbU09TqZ9wZOfy3qRcxZpA9E4nI8GoalTHmduAvrLV3\nGGOmAQ9Ya2dEa19V1ZhQof6OAN//9foEq5Serr2ygBO1Lf022UVk6PLGZNMcZarov373a5xf0P+Y\nWywKCvIjXrfDy6me1wPvAlhrNwNfHYk3eXBBek2H9NL2kioFv8gIiRb8EJoSPNy83Oc/Aei5zdVh\njMm21kb8F5g0KY/s7Pg3fe74sy/w6JLoZ02KiIyU3JysAe+pEIvp117AdX984bDPCPIy/BuAnofd\nM6MFP0Btbezzlnu66vwJvPnYDKqq4ju1fTQrKMhP6f4eO9XEmePHkDd28F+/u361Fhh4SmOi/V2y\nxrJuRwXjz8hh3r03xP2+fW3cVckFBeOjzr2PVX2Tjx+G91FHev/O/ra0+fF3BMhPcMrw4lX72Hu4\nhkfv+frgjUdIRyBAmy8Q9XdhuH+Xjxxv7Dp/ItFpsiMpWn/rahO/L0JBQeTfRy/DfyPw58Dvw/v8\ndW9DR0wZ4KScvh7/x28Qx+Vs4tJ5uGuIV3Xu8o0vnTcsrzNxXC6333gZn79g4oDtYptJEt0dt101\npPWHQ1ZmJnljk7f3OZ4L+qU7L8N/OXCLMWYTkAHc6WEtkqImjh/8zNCEhYNgqNf0Hwl9z0iV4aHs\n7+ZZ+FtrA8DdXr2/SOcWRepFv4yUoJMXm45MF3YTh3WO/D0uQ5JGI/9uCn9xVtfIX+nvDoV/F4W/\nuGuYD/hK6tMB327OXNtHpK/Os9sjZf/j/3T9kG5oLpLqFP7irO4xYP+Qnzhu6JfbltSTancF85J2\n+4izhnuev6Q+ZX83hb84K6jZPs7RyL+bwl/cpdk+zlH0d1P4i7MCAxzwlfSkgX83hb84TyN/d2i3\nTzeFvzgroAO+zlH0d1P4i7tS+MJuMjI08u+m8BdnBXVhN+dcel7oBuy3Tb3I40q8p5O8xFnZ4Tsj\n5eZoDOSK/LxcFvzLfyFTW3sKf3HX/5h+OW2+Dv7nN6/wuhRJIgV/iMJfnDV5wlh+8Fdf9roMEU9o\ne1dExEEKfxERByn8RUQcpPAXEXGQwl9ExEEKfxERByn8RUQcpPAXEXFQhi50JCLiHo38RUQcpPAX\nEXGQwl9ExEEKfxERByn8RUQcpPAXEXGQwl9ExEFpfTMXY0wm8DRwNdAGfM9ae8DbquJnjCkC6sM/\nlgLPAU8AfmCNtfYX0fpqjJkWa9ukdqoPY8xUYI61drox5gpgMRAEdgMzrbUBY8zDwLcJ9WWWtXbr\ncLRNZj879envNcCbwP7w089Ya19Nh/4aY3KAF4FLgDHAL4G9Q+3DKOtvOSn4+ab7yP8vgbHW2uuA\nnwCPeVxP3IwxYwGstdPD/90JPAv8DXA9MDUcHtH6Gk9bTxhjfgwsAMaGF80FHrTW3kDo/uozwnXf\nBEwF/hp4ajjajnTfIonQ32uAuT0+41fTqL//G6gO13Ab8GSkutK8vyn5+ab1yJ9Q4L0LYK3dbIz5\nqsf1JOJqIM8Ys4bQ5zUbGGOtPQhgjFkN3AycR5++GmMmxNo2uV3q5yBwO7Ak/PO1wIfhx6uAPwUs\noS2XIFBmjMk2xhQMQ9vlI9qzyCL11xhjZhAaHc4i9LubDv19DVjW42d/lLrS5fON1t+U+3zTfeQ/\nge7dJQAdxpjR9oXXDPwGuBW4G1gUXtapEZhIhL6GlzXE0tbLfxdr7etAe49FGeFfdIhec+fyobZN\nugj93Qrcb629ETgEPEya9Ndae9pa22iMyScUig9GqSud+5uSn2+6h38DkN/j50xrrd+rYhJUArxs\nrQ1aa0sI/RJM7vF8PlBHhL5GWBa1bYr9u/Tcbxmt5s7lQ22bCpZba7d3Pgb+mDTqrzHmQmAdsMRa\nuzRKXenc35T8fNM9/DcC3wIIH/jc5W05CbmL8D55Y8wUIA9oMsZcbozJILRFUEiEvlprGwBfLG2T\n26VBFRljpocf30Z3zbcaYzKNMRcR+sI6NQxtU8FqY8zXwo9vBraTJv01xpwDrAH+xVr7Ynhx2n6+\nUfqbkp/vaNsFEq/lwC3GmE2EDojc6XE9iVgILDbGbCB0VP8uQt/4vwOyCO0L3GKM2Ubkvt4dR9tU\ncR/wgjEmFygGlllrO4wxhcDHhAYtM4ejbdJ6NLB7gCeNMT7gOPB9a21DmvT3p8Ak4CFjzEPhZfcC\n89L0843U3x8Bv021z1eXdBYRcVC67/YREZEIFP4iIg5S+IuIOEjhLyLiIIW/iIiDFP4iIg5S+IuI\nOOj/A9EOoqmJ6EYcAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(df_test['Kt'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_train= df_train[df_train['Kt']< 5000]\n",
    "df_train= df_train[df_train['Kt']> -1000]\n",
    "df_test= df_test[df_test['Kt']< 5000]\n",
    "df_test= df_test[df_test['Kt']> -1000]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>index</th>\n",
       "      <th>year</th>\n",
       "      <th>jday</th>\n",
       "      <th>month</th>\n",
       "      <th>day</th>\n",
       "      <th>hour</th>\n",
       "      <th>min</th>\n",
       "      <th>dt</th>\n",
       "      <th>zen</th>\n",
       "      <th>dw_solar</th>\n",
       "      <th>...</th>\n",
       "      <th>rh</th>\n",
       "      <th>rh_QC</th>\n",
       "      <th>windspd</th>\n",
       "      <th>windspd_QC</th>\n",
       "      <th>winddir</th>\n",
       "      <th>winddir_QC</th>\n",
       "      <th>pressure</th>\n",
       "      <th>pressure_QC</th>\n",
       "      <th>ghi</th>\n",
       "      <th>Kt</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>446</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>26.0</td>\n",
       "      <td>7.433</td>\n",
       "      <td>162.31</td>\n",
       "      <td>-2.2</td>\n",
       "      <td>...</td>\n",
       "      <td>57.4</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.3</td>\n",
       "      <td>0.0</td>\n",
       "      <td>16.5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>830.6</td>\n",
       "      <td>0.0</td>\n",
       "      <td>4.162896</td>\n",
       "      <td>-0.528478</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>447</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>27.0</td>\n",
       "      <td>7.450</td>\n",
       "      <td>162.25</td>\n",
       "      <td>-2.2</td>\n",
       "      <td>...</td>\n",
       "      <td>58.5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.6</td>\n",
       "      <td>0.0</td>\n",
       "      <td>13.7</td>\n",
       "      <td>0.0</td>\n",
       "      <td>830.6</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8.593390</td>\n",
       "      <td>-0.256011</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>448</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>28.0</td>\n",
       "      <td>7.467</td>\n",
       "      <td>162.19</td>\n",
       "      <td>-2.0</td>\n",
       "      <td>...</td>\n",
       "      <td>59.1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.4</td>\n",
       "      <td>0.0</td>\n",
       "      <td>13.1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>830.6</td>\n",
       "      <td>0.0</td>\n",
       "      <td>11.478161</td>\n",
       "      <td>-0.174244</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>449</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>29.0</td>\n",
       "      <td>7.483</td>\n",
       "      <td>162.14</td>\n",
       "      <td>-1.8</td>\n",
       "      <td>...</td>\n",
       "      <td>59.7</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.7</td>\n",
       "      <td>0.0</td>\n",
       "      <td>14.5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>830.6</td>\n",
       "      <td>0.0</td>\n",
       "      <td>13.503592</td>\n",
       "      <td>-0.133298</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>450</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>30.0</td>\n",
       "      <td>7.500</td>\n",
       "      <td>162.07</td>\n",
       "      <td>-1.8</td>\n",
       "      <td>...</td>\n",
       "      <td>60.2</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.8</td>\n",
       "      <td>0.0</td>\n",
       "      <td>20.6</td>\n",
       "      <td>0.0</td>\n",
       "      <td>830.6</td>\n",
       "      <td>0.0</td>\n",
       "      <td>15.052006</td>\n",
       "      <td>-0.119585</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 51 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   index    year  jday  month  day  hour   min     dt     zen  dw_solar  \\\n",
       "0    446  2010.0   1.0    1.0  1.0   7.0  26.0  7.433  162.31      -2.2   \n",
       "1    447  2010.0   1.0    1.0  1.0   7.0  27.0  7.450  162.25      -2.2   \n",
       "2    448  2010.0   1.0    1.0  1.0   7.0  28.0  7.467  162.19      -2.0   \n",
       "3    449  2010.0   1.0    1.0  1.0   7.0  29.0  7.483  162.14      -1.8   \n",
       "4    450  2010.0   1.0    1.0  1.0   7.0  30.0  7.500  162.07      -1.8   \n",
       "\n",
       "     ...       rh  rh_QC  windspd  windspd_QC  winddir  winddir_QC  pressure  \\\n",
       "0    ...     57.4    0.0      2.3         0.0     16.5         0.0     830.6   \n",
       "1    ...     58.5    0.0      2.6         0.0     13.7         0.0     830.6   \n",
       "2    ...     59.1    0.0      2.4         0.0     13.1         0.0     830.6   \n",
       "3    ...     59.7    0.0      2.7         0.0     14.5         0.0     830.6   \n",
       "4    ...     60.2    0.0      2.8         0.0     20.6         0.0     830.6   \n",
       "\n",
       "   pressure_QC        ghi        Kt  \n",
       "0          0.0   4.162896 -0.528478  \n",
       "1          0.0   8.593390 -0.256011  \n",
       "2          0.0  11.478161 -0.174244  \n",
       "3          0.0  13.503592 -0.133298  \n",
       "4          0.0  15.052006 -0.119585  \n",
       "\n",
       "[5 rows x 51 columns]"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Group the data (train dataframe)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [],
   "source": [
    "zen = df_train.groupby(['year','month','day','hour'])['zen'].mean()\n",
    "dw_solar = df_train.groupby(['year','month','day','hour'])['dw_solar'].mean()\n",
    "uw_solar = df_train.groupby(['year','month','day','hour'])['uw_solar'].mean()\n",
    "direct_n = df_train.groupby(['year','month','day','hour'])['direct_n'].mean()\n",
    "diffuse = df_train.groupby(['year','month','day','hour'])['diffuse'].mean()\n",
    "dw_ir = df_train.groupby(['year','month','day','hour'])['dw_ir'].mean()\n",
    "dw_casetemp = df_train.groupby(['year','month','day','hour'])['dw_casetemp'].mean()\n",
    "dw_dometemp = df_train.groupby(['year','month','day','hour'])['dw_dometemp'].mean()\n",
    "uw_ir = df_train.groupby(['year','month','day','hour'])['uw_ir'].mean()\n",
    "uw_casetemp = df_train.groupby(['year','month','day','hour'])['uw_casetemp'].mean()\n",
    "uw_dometemp = df_train.groupby(['year','month','day','hour'])['uw_dometemp'].mean()\n",
    "uvb = df_train.groupby(['year','month','day','hour'])['uvb'].mean()\n",
    "par = df_train.groupby(['year','month','day','hour'])['par'].mean()\n",
    "netsolar = df_train.groupby(['year','month','day','hour'])['netsolar'].mean()\n",
    "netir = df_train.groupby(['year','month','day','hour'])['netir'].mean()\n",
    "totalnet = df_train.groupby(['year','month','day','hour'])['totalnet'].mean()\n",
    "temp = df_train.groupby(['year','month','day','hour'])['temp'].mean()\n",
    "rh = df_train.groupby(['year','month','day','hour'])['rh'].mean()\n",
    "windspd = df_train.groupby(['year','month','day','hour'])['windspd'].mean()\n",
    "winddir = df_train.groupby(['year','month','day','hour'])['winddir'].mean()\n",
    "pressure = df_train.groupby(['year','month','day','hour'])['pressure'].mean()\n",
    "ghi = df_train.groupby(['year','month','day','hour'])['ghi'].mean()\n",
    "Kt = df_train.groupby(['year','month','day','hour'])['Kt'].mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_new_train = pd.concat([zen,dw_solar,uw_solar,direct_n,diffuse,dw_ir,dw_casetemp,dw_dometemp,uw_ir,uw_casetemp,uw_dometemp,\n",
    "                    uvb,par,netsolar,netir,totalnet,temp,rh,windspd,winddir,pressure,ghi,Kt], axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>zen</th>\n",
       "      <th>dw_solar</th>\n",
       "      <th>uw_solar</th>\n",
       "      <th>direct_n</th>\n",
       "      <th>diffuse</th>\n",
       "      <th>dw_ir</th>\n",
       "      <th>dw_casetemp</th>\n",
       "      <th>dw_dometemp</th>\n",
       "      <th>uw_ir</th>\n",
       "      <th>uw_casetemp</th>\n",
       "      <th>...</th>\n",
       "      <th>netsolar</th>\n",
       "      <th>netir</th>\n",
       "      <th>totalnet</th>\n",
       "      <th>temp</th>\n",
       "      <th>rh</th>\n",
       "      <th>windspd</th>\n",
       "      <th>winddir</th>\n",
       "      <th>pressure</th>\n",
       "      <th>ghi</th>\n",
       "      <th>Kt</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>year</th>\n",
       "      <th>month</th>\n",
       "      <th>day</th>\n",
       "      <th>hour</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">2010.0</th>\n",
       "      <th rowspan=\"5\" valign=\"top\">1.0</th>\n",
       "      <th rowspan=\"5\" valign=\"top\">1.0</th>\n",
       "      <th>7.0</th>\n",
       "      <td>161.031471</td>\n",
       "      <td>-2.950000</td>\n",
       "      <td>-0.711765</td>\n",
       "      <td>1.138235</td>\n",
       "      <td>0.538235</td>\n",
       "      <td>224.085294</td>\n",
       "      <td>270.364706</td>\n",
       "      <td>269.979412</td>\n",
       "      <td>268.744118</td>\n",
       "      <td>268.611765</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-44.652941</td>\n",
       "      <td>-44.652941</td>\n",
       "      <td>-3.211765</td>\n",
       "      <td>56.261765</td>\n",
       "      <td>2.835294</td>\n",
       "      <td>107.311765</td>\n",
       "      <td>830.647059</td>\n",
       "      <td>33.080713</td>\n",
       "      <td>-0.116856</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8.0</th>\n",
       "      <td>155.139333</td>\n",
       "      <td>-3.753333</td>\n",
       "      <td>-0.936667</td>\n",
       "      <td>1.285000</td>\n",
       "      <td>0.513333</td>\n",
       "      <td>223.563333</td>\n",
       "      <td>270.761667</td>\n",
       "      <td>270.405000</td>\n",
       "      <td>275.378333</td>\n",
       "      <td>269.453333</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-51.813333</td>\n",
       "      <td>-51.813333</td>\n",
       "      <td>-2.645000</td>\n",
       "      <td>53.351667</td>\n",
       "      <td>3.866667</td>\n",
       "      <td>147.821667</td>\n",
       "      <td>830.728333</td>\n",
       "      <td>143.676328</td>\n",
       "      <td>-0.030061</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9.0</th>\n",
       "      <td>145.065667</td>\n",
       "      <td>-3.293333</td>\n",
       "      <td>-0.251667</td>\n",
       "      <td>1.255000</td>\n",
       "      <td>0.488333</td>\n",
       "      <td>212.618333</td>\n",
       "      <td>270.150000</td>\n",
       "      <td>269.700000</td>\n",
       "      <td>273.045000</td>\n",
       "      <td>268.510000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-60.421667</td>\n",
       "      <td>-60.421667</td>\n",
       "      <td>-3.716667</td>\n",
       "      <td>59.238333</td>\n",
       "      <td>3.315000</td>\n",
       "      <td>63.578333</td>\n",
       "      <td>831.020000</td>\n",
       "      <td>299.581054</td>\n",
       "      <td>-0.011214</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10.0</th>\n",
       "      <td>133.892500</td>\n",
       "      <td>-3.641667</td>\n",
       "      <td>-0.710000</td>\n",
       "      <td>1.795000</td>\n",
       "      <td>0.406667</td>\n",
       "      <td>215.475000</td>\n",
       "      <td>269.760000</td>\n",
       "      <td>269.403333</td>\n",
       "      <td>270.496667</td>\n",
       "      <td>267.760000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-55.031667</td>\n",
       "      <td>-55.031667</td>\n",
       "      <td>-3.923333</td>\n",
       "      <td>60.121667</td>\n",
       "      <td>3.076667</td>\n",
       "      <td>71.128333</td>\n",
       "      <td>831.073333</td>\n",
       "      <td>419.749575</td>\n",
       "      <td>-0.008772</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11.0</th>\n",
       "      <td>122.454000</td>\n",
       "      <td>-3.126667</td>\n",
       "      <td>-0.471667</td>\n",
       "      <td>1.275000</td>\n",
       "      <td>0.456667</td>\n",
       "      <td>216.165000</td>\n",
       "      <td>269.116667</td>\n",
       "      <td>268.663333</td>\n",
       "      <td>269.205000</td>\n",
       "      <td>267.470000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-53.050000</td>\n",
       "      <td>-53.050000</td>\n",
       "      <td>-4.565000</td>\n",
       "      <td>61.175000</td>\n",
       "      <td>3.535000</td>\n",
       "      <td>184.166667</td>\n",
       "      <td>831.015000</td>\n",
       "      <td>486.001859</td>\n",
       "      <td>-0.006431</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 23 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                              zen  dw_solar  uw_solar  direct_n   diffuse  \\\n",
       "year   month day hour                                                       \n",
       "2010.0 1.0   1.0 7.0   161.031471 -2.950000 -0.711765  1.138235  0.538235   \n",
       "                 8.0   155.139333 -3.753333 -0.936667  1.285000  0.513333   \n",
       "                 9.0   145.065667 -3.293333 -0.251667  1.255000  0.488333   \n",
       "                 10.0  133.892500 -3.641667 -0.710000  1.795000  0.406667   \n",
       "                 11.0  122.454000 -3.126667 -0.471667  1.275000  0.456667   \n",
       "\n",
       "                            dw_ir  dw_casetemp  dw_dometemp       uw_ir  \\\n",
       "year   month day hour                                                     \n",
       "2010.0 1.0   1.0 7.0   224.085294   270.364706   269.979412  268.744118   \n",
       "                 8.0   223.563333   270.761667   270.405000  275.378333   \n",
       "                 9.0   212.618333   270.150000   269.700000  273.045000   \n",
       "                 10.0  215.475000   269.760000   269.403333  270.496667   \n",
       "                 11.0  216.165000   269.116667   268.663333  269.205000   \n",
       "\n",
       "                       uw_casetemp    ...     netsolar      netir   totalnet  \\\n",
       "year   month day hour                 ...                                      \n",
       "2010.0 1.0   1.0 7.0    268.611765    ...          0.0 -44.652941 -44.652941   \n",
       "                 8.0    269.453333    ...          0.0 -51.813333 -51.813333   \n",
       "                 9.0    268.510000    ...          0.0 -60.421667 -60.421667   \n",
       "                 10.0   267.760000    ...          0.0 -55.031667 -55.031667   \n",
       "                 11.0   267.470000    ...          0.0 -53.050000 -53.050000   \n",
       "\n",
       "                           temp         rh   windspd     winddir    pressure  \\\n",
       "year   month day hour                                                          \n",
       "2010.0 1.0   1.0 7.0  -3.211765  56.261765  2.835294  107.311765  830.647059   \n",
       "                 8.0  -2.645000  53.351667  3.866667  147.821667  830.728333   \n",
       "                 9.0  -3.716667  59.238333  3.315000   63.578333  831.020000   \n",
       "                 10.0 -3.923333  60.121667  3.076667   71.128333  831.073333   \n",
       "                 11.0 -4.565000  61.175000  3.535000  184.166667  831.015000   \n",
       "\n",
       "                              ghi        Kt  \n",
       "year   month day hour                        \n",
       "2010.0 1.0   1.0 7.0    33.080713 -0.116856  \n",
       "                 8.0   143.676328 -0.030061  \n",
       "                 9.0   299.581054 -0.011214  \n",
       "                 10.0  419.749575 -0.008772  \n",
       "                 11.0  486.001859 -0.006431  \n",
       "\n",
       "[5 rows x 23 columns]"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_new_train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Groupdata - test dataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_zen = df_test.groupby(['month','day','hour'])['zen'].mean()\n",
    "test_dw_solar = df_test.groupby(['month','day','hour'])['dw_solar'].mean()\n",
    "test_uw_solar = df_test.groupby(['month','day','hour'])['uw_solar'].mean()\n",
    "test_direct_n = df_test.groupby(['month','day','hour'])['direct_n'].mean()\n",
    "test_diffuse = df_test.groupby(['month','day','hour'])['diffuse'].mean()\n",
    "test_dw_ir = df_test.groupby(['month','day','hour'])['dw_ir'].mean()\n",
    "test_dw_casetemp = df_test.groupby(['month','day','hour'])['dw_casetemp'].mean()\n",
    "test_dw_dometemp = df_test.groupby(['month','day','hour'])['dw_dometemp'].mean()\n",
    "test_uw_ir = df_test.groupby(['month','day','hour'])['uw_ir'].mean()\n",
    "test_uw_casetemp = df_test.groupby(['month','day','hour'])['uw_casetemp'].mean()\n",
    "test_uw_dometemp = df_test.groupby(['month','day','hour'])['uw_dometemp'].mean()\n",
    "test_uvb = df_test.groupby(['month','day','hour'])['uvb'].mean()\n",
    "test_par = df_test.groupby(['month','day','hour'])['par'].mean()\n",
    "test_netsolar = df_test.groupby(['month','day','hour'])['netsolar'].mean()\n",
    "test_netir = df_test.groupby(['month','day','hour'])['netir'].mean()\n",
    "test_totalnet = df_test.groupby(['month','day','hour'])['totalnet'].mean()\n",
    "test_temp = df_test.groupby(['month','day','hour'])['temp'].mean()\n",
    "test_rh = df_test.groupby(['month','day','hour'])['rh'].mean()\n",
    "test_windspd = df_test.groupby(['month','day','hour'])['windspd'].mean()\n",
    "test_winddir = df_test.groupby(['month','day','hour'])['winddir'].mean()\n",
    "test_pressure = df_test.groupby(['month','day','hour'])['pressure'].mean()\n",
    "test_ghi = df_test.groupby(['month','day','hour'])['ghi'].mean()\n",
    "test_Kt = df_test.groupby(['month','day','hour'])['Kt'].mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_new_test = pd.concat([test_zen,test_dw_solar,test_uw_solar,test_direct_n,test_diffuse,test_dw_ir,\n",
    "                         test_dw_casetemp,test_dw_dometemp,test_uw_ir,test_uw_casetemp,test_uw_dometemp,\n",
    "                    test_uvb,test_par,test_netsolar,test_netir,test_totalnet,test_temp,test_rh,\n",
    "                         test_windspd,test_winddir,test_pressure,test_ghi,test_Kt], axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>zen</th>\n",
       "      <th>dw_solar</th>\n",
       "      <th>uw_solar</th>\n",
       "      <th>direct_n</th>\n",
       "      <th>diffuse</th>\n",
       "      <th>dw_ir</th>\n",
       "      <th>dw_casetemp</th>\n",
       "      <th>dw_dometemp</th>\n",
       "      <th>uw_ir</th>\n",
       "      <th>uw_casetemp</th>\n",
       "      <th>...</th>\n",
       "      <th>netsolar</th>\n",
       "      <th>netir</th>\n",
       "      <th>totalnet</th>\n",
       "      <th>temp</th>\n",
       "      <th>rh</th>\n",
       "      <th>windspd</th>\n",
       "      <th>winddir</th>\n",
       "      <th>pressure</th>\n",
       "      <th>ghi</th>\n",
       "      <th>Kt</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>hour</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>6.0</th>\n",
       "      <td>151.473333</td>\n",
       "      <td>-3.300000</td>\n",
       "      <td>-0.233333</td>\n",
       "      <td>-3.483333</td>\n",
       "      <td>-0.116667</td>\n",
       "      <td>233.200000</td>\n",
       "      <td>280.250000</td>\n",
       "      <td>279.550000</td>\n",
       "      <td>309.550000</td>\n",
       "      <td>279.016667</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-76.316667</td>\n",
       "      <td>-76.316667</td>\n",
       "      <td>6.783333</td>\n",
       "      <td>37.650000</td>\n",
       "      <td>3.866667</td>\n",
       "      <td>253.666667</td>\n",
       "      <td>820.400000</td>\n",
       "      <td>11.691876</td>\n",
       "      <td>-0.326406</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7.0</th>\n",
       "      <td>151.355333</td>\n",
       "      <td>-3.783333</td>\n",
       "      <td>-1.331667</td>\n",
       "      <td>2.156667</td>\n",
       "      <td>-0.118333</td>\n",
       "      <td>232.493333</td>\n",
       "      <td>279.143333</td>\n",
       "      <td>278.730000</td>\n",
       "      <td>301.550000</td>\n",
       "      <td>278.228333</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-69.071667</td>\n",
       "      <td>-69.071667</td>\n",
       "      <td>6.888333</td>\n",
       "      <td>36.541667</td>\n",
       "      <td>2.335000</td>\n",
       "      <td>236.583333</td>\n",
       "      <td>819.853333</td>\n",
       "      <td>85.439689</td>\n",
       "      <td>-0.067258</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8.0</th>\n",
       "      <td>147.215333</td>\n",
       "      <td>-3.510000</td>\n",
       "      <td>-1.576667</td>\n",
       "      <td>1.040000</td>\n",
       "      <td>-0.085000</td>\n",
       "      <td>228.316667</td>\n",
       "      <td>278.118333</td>\n",
       "      <td>277.690000</td>\n",
       "      <td>292.126667</td>\n",
       "      <td>276.498333</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-63.806667</td>\n",
       "      <td>-63.806667</td>\n",
       "      <td>5.211667</td>\n",
       "      <td>41.668333</td>\n",
       "      <td>1.003333</td>\n",
       "      <td>183.796667</td>\n",
       "      <td>819.033333</td>\n",
       "      <td>276.176482</td>\n",
       "      <td>-0.013317</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9.0</th>\n",
       "      <td>139.073167</td>\n",
       "      <td>-4.836667</td>\n",
       "      <td>-1.441667</td>\n",
       "      <td>3.286667</td>\n",
       "      <td>0.153333</td>\n",
       "      <td>228.880000</td>\n",
       "      <td>278.088333</td>\n",
       "      <td>277.798333</td>\n",
       "      <td>301.906667</td>\n",
       "      <td>276.206667</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-73.030000</td>\n",
       "      <td>-73.030000</td>\n",
       "      <td>5.850000</td>\n",
       "      <td>36.901667</td>\n",
       "      <td>2.431667</td>\n",
       "      <td>154.540000</td>\n",
       "      <td>817.888333</td>\n",
       "      <td>459.734986</td>\n",
       "      <td>-0.010574</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10.0</th>\n",
       "      <td>128.876667</td>\n",
       "      <td>-4.575000</td>\n",
       "      <td>-0.933333</td>\n",
       "      <td>0.893333</td>\n",
       "      <td>-0.031667</td>\n",
       "      <td>230.783333</td>\n",
       "      <td>278.736667</td>\n",
       "      <td>278.345000</td>\n",
       "      <td>308.195000</td>\n",
       "      <td>277.050000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-77.415000</td>\n",
       "      <td>-77.415000</td>\n",
       "      <td>5.566667</td>\n",
       "      <td>34.063333</td>\n",
       "      <td>3.446667</td>\n",
       "      <td>180.726667</td>\n",
       "      <td>816.726667</td>\n",
       "      <td>599.596511</td>\n",
       "      <td>-0.007646</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11.0</th>\n",
       "      <td>117.762500</td>\n",
       "      <td>-4.770000</td>\n",
       "      <td>-1.326667</td>\n",
       "      <td>2.743333</td>\n",
       "      <td>0.120000</td>\n",
       "      <td>241.015000</td>\n",
       "      <td>279.501667</td>\n",
       "      <td>279.201667</td>\n",
       "      <td>304.923333</td>\n",
       "      <td>277.420000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-63.898333</td>\n",
       "      <td>-63.898333</td>\n",
       "      <td>6.798333</td>\n",
       "      <td>28.861667</td>\n",
       "      <td>1.726667</td>\n",
       "      <td>144.958333</td>\n",
       "      <td>816.071667</td>\n",
       "      <td>681.707415</td>\n",
       "      <td>-0.006993</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12.0</th>\n",
       "      <td>106.328333</td>\n",
       "      <td>-4.698333</td>\n",
       "      <td>-1.208333</td>\n",
       "      <td>1.531667</td>\n",
       "      <td>0.030000</td>\n",
       "      <td>229.035000</td>\n",
       "      <td>280.325000</td>\n",
       "      <td>279.935000</td>\n",
       "      <td>312.165000</td>\n",
       "      <td>278.568333</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-83.126667</td>\n",
       "      <td>-83.126667</td>\n",
       "      <td>7.463333</td>\n",
       "      <td>28.286667</td>\n",
       "      <td>3.046667</td>\n",
       "      <td>217.635000</td>\n",
       "      <td>815.656667</td>\n",
       "      <td>699.157260</td>\n",
       "      <td>-0.006720</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13.0</th>\n",
       "      <td>94.871333</td>\n",
       "      <td>-3.521667</td>\n",
       "      <td>0.295000</td>\n",
       "      <td>18.040000</td>\n",
       "      <td>1.958333</td>\n",
       "      <td>227.543333</td>\n",
       "      <td>280.648333</td>\n",
       "      <td>280.078333</td>\n",
       "      <td>324.303333</td>\n",
       "      <td>279.133333</td>\n",
       "      <td>...</td>\n",
       "      <td>1.470000</td>\n",
       "      <td>-96.761667</td>\n",
       "      <td>-95.290000</td>\n",
       "      <td>7.078333</td>\n",
       "      <td>32.360000</td>\n",
       "      <td>10.041667</td>\n",
       "      <td>249.865000</td>\n",
       "      <td>815.330000</td>\n",
       "      <td>650.549614</td>\n",
       "      <td>-0.005250</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14.0</th>\n",
       "      <td>83.779333</td>\n",
       "      <td>78.048333</td>\n",
       "      <td>23.100000</td>\n",
       "      <td>542.223333</td>\n",
       "      <td>32.271667</td>\n",
       "      <td>230.468333</td>\n",
       "      <td>281.270000</td>\n",
       "      <td>280.663333</td>\n",
       "      <td>329.951667</td>\n",
       "      <td>278.816667</td>\n",
       "      <td>...</td>\n",
       "      <td>75.555000</td>\n",
       "      <td>-99.493333</td>\n",
       "      <td>-23.943333</td>\n",
       "      <td>6.751667</td>\n",
       "      <td>36.701667</td>\n",
       "      <td>9.171667</td>\n",
       "      <td>242.581667</td>\n",
       "      <td>815.920000</td>\n",
       "      <td>539.852805</td>\n",
       "      <td>0.152844</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15.0</th>\n",
       "      <td>73.632167</td>\n",
       "      <td>268.235000</td>\n",
       "      <td>56.083333</td>\n",
       "      <td>839.303333</td>\n",
       "      <td>49.900000</td>\n",
       "      <td>229.556667</td>\n",
       "      <td>284.176667</td>\n",
       "      <td>283.490000</td>\n",
       "      <td>349.180000</td>\n",
       "      <td>280.603333</td>\n",
       "      <td>...</td>\n",
       "      <td>232.223333</td>\n",
       "      <td>-119.618333</td>\n",
       "      <td>112.601667</td>\n",
       "      <td>8.125000</td>\n",
       "      <td>34.693333</td>\n",
       "      <td>7.085000</td>\n",
       "      <td>223.708333</td>\n",
       "      <td>815.900000</td>\n",
       "      <td>376.947686</td>\n",
       "      <td>0.748434</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16.0</th>\n",
       "      <td>64.618000</td>\n",
       "      <td>455.048333</td>\n",
       "      <td>80.918333</td>\n",
       "      <td>950.301667</td>\n",
       "      <td>60.190000</td>\n",
       "      <td>225.805000</td>\n",
       "      <td>285.555000</td>\n",
       "      <td>284.541667</td>\n",
       "      <td>364.461667</td>\n",
       "      <td>281.606667</td>\n",
       "      <td>...</td>\n",
       "      <td>386.976667</td>\n",
       "      <td>-138.648333</td>\n",
       "      <td>248.321667</td>\n",
       "      <td>8.815000</td>\n",
       "      <td>26.081667</td>\n",
       "      <td>10.761667</td>\n",
       "      <td>257.903333</td>\n",
       "      <td>815.588333</td>\n",
       "      <td>182.049631</td>\n",
       "      <td>2.903754</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17.0</th>\n",
       "      <td>58.554737</td>\n",
       "      <td>579.668421</td>\n",
       "      <td>96.150000</td>\n",
       "      <td>999.321053</td>\n",
       "      <td>64.242105</td>\n",
       "      <td>219.452632</td>\n",
       "      <td>284.897368</td>\n",
       "      <td>283.692105</td>\n",
       "      <td>367.189474</td>\n",
       "      <td>280.934211</td>\n",
       "      <td>...</td>\n",
       "      <td>489.452632</td>\n",
       "      <td>-147.736842</td>\n",
       "      <td>341.721053</td>\n",
       "      <td>8.086842</td>\n",
       "      <td>25.373684</td>\n",
       "      <td>15.331579</td>\n",
       "      <td>255.897368</td>\n",
       "      <td>814.931579</td>\n",
       "      <td>39.993730</td>\n",
       "      <td>24.850513</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>12 rows × 23 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "             zen    dw_solar   uw_solar    direct_n    diffuse       dw_ir  \\\n",
       "hour                                                                         \n",
       "6.0   151.473333   -3.300000  -0.233333   -3.483333  -0.116667  233.200000   \n",
       "7.0   151.355333   -3.783333  -1.331667    2.156667  -0.118333  232.493333   \n",
       "8.0   147.215333   -3.510000  -1.576667    1.040000  -0.085000  228.316667   \n",
       "9.0   139.073167   -4.836667  -1.441667    3.286667   0.153333  228.880000   \n",
       "10.0  128.876667   -4.575000  -0.933333    0.893333  -0.031667  230.783333   \n",
       "11.0  117.762500   -4.770000  -1.326667    2.743333   0.120000  241.015000   \n",
       "12.0  106.328333   -4.698333  -1.208333    1.531667   0.030000  229.035000   \n",
       "13.0   94.871333   -3.521667   0.295000   18.040000   1.958333  227.543333   \n",
       "14.0   83.779333   78.048333  23.100000  542.223333  32.271667  230.468333   \n",
       "15.0   73.632167  268.235000  56.083333  839.303333  49.900000  229.556667   \n",
       "16.0   64.618000  455.048333  80.918333  950.301667  60.190000  225.805000   \n",
       "17.0   58.554737  579.668421  96.150000  999.321053  64.242105  219.452632   \n",
       "\n",
       "      dw_casetemp  dw_dometemp       uw_ir  uw_casetemp    ...      \\\n",
       "hour                                                       ...       \n",
       "6.0    280.250000   279.550000  309.550000   279.016667    ...       \n",
       "7.0    279.143333   278.730000  301.550000   278.228333    ...       \n",
       "8.0    278.118333   277.690000  292.126667   276.498333    ...       \n",
       "9.0    278.088333   277.798333  301.906667   276.206667    ...       \n",
       "10.0   278.736667   278.345000  308.195000   277.050000    ...       \n",
       "11.0   279.501667   279.201667  304.923333   277.420000    ...       \n",
       "12.0   280.325000   279.935000  312.165000   278.568333    ...       \n",
       "13.0   280.648333   280.078333  324.303333   279.133333    ...       \n",
       "14.0   281.270000   280.663333  329.951667   278.816667    ...       \n",
       "15.0   284.176667   283.490000  349.180000   280.603333    ...       \n",
       "16.0   285.555000   284.541667  364.461667   281.606667    ...       \n",
       "17.0   284.897368   283.692105  367.189474   280.934211    ...       \n",
       "\n",
       "        netsolar       netir    totalnet      temp         rh    windspd  \\\n",
       "hour                                                                       \n",
       "6.0     0.000000  -76.316667  -76.316667  6.783333  37.650000   3.866667   \n",
       "7.0     0.000000  -69.071667  -69.071667  6.888333  36.541667   2.335000   \n",
       "8.0     0.000000  -63.806667  -63.806667  5.211667  41.668333   1.003333   \n",
       "9.0     0.000000  -73.030000  -73.030000  5.850000  36.901667   2.431667   \n",
       "10.0    0.000000  -77.415000  -77.415000  5.566667  34.063333   3.446667   \n",
       "11.0    0.000000  -63.898333  -63.898333  6.798333  28.861667   1.726667   \n",
       "12.0    0.000000  -83.126667  -83.126667  7.463333  28.286667   3.046667   \n",
       "13.0    1.470000  -96.761667  -95.290000  7.078333  32.360000  10.041667   \n",
       "14.0   75.555000  -99.493333  -23.943333  6.751667  36.701667   9.171667   \n",
       "15.0  232.223333 -119.618333  112.601667  8.125000  34.693333   7.085000   \n",
       "16.0  386.976667 -138.648333  248.321667  8.815000  26.081667  10.761667   \n",
       "17.0  489.452632 -147.736842  341.721053  8.086842  25.373684  15.331579   \n",
       "\n",
       "         winddir    pressure         ghi         Kt  \n",
       "hour                                                 \n",
       "6.0   253.666667  820.400000   11.691876  -0.326406  \n",
       "7.0   236.583333  819.853333   85.439689  -0.067258  \n",
       "8.0   183.796667  819.033333  276.176482  -0.013317  \n",
       "9.0   154.540000  817.888333  459.734986  -0.010574  \n",
       "10.0  180.726667  816.726667  599.596511  -0.007646  \n",
       "11.0  144.958333  816.071667  681.707415  -0.006993  \n",
       "12.0  217.635000  815.656667  699.157260  -0.006720  \n",
       "13.0  249.865000  815.330000  650.549614  -0.005250  \n",
       "14.0  242.581667  815.920000  539.852805   0.152844  \n",
       "15.0  223.708333  815.900000  376.947686   0.748434  \n",
       "16.0  257.903333  815.588333  182.049631   2.903754  \n",
       "17.0  255.897368  814.931579   39.993730  24.850513  \n",
       "\n",
       "[12 rows x 23 columns]"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_new_test.loc[2].xs(17,level='day')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Shifting Kt values to make 1 hour ahead forecast"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Train dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [],
   "source": [
    "levels_index= []\n",
    "for m in df_new_train.index.levels:\n",
    "    levels_index.append(m)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "for i in levels_index[0]:\n",
    "    for j in levels_index[1]:\n",
    "        df_new_train.loc[i].loc[j]['Kt'] = df_new_train.loc[i].loc[j]['Kt'].shift(-1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_new_train = df_new_train[~(df_new_train['Kt'].isnull())]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Test dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [],
   "source": [
    "levels_index2= []\n",
    "for m in df_new_test.index.levels:\n",
    "    levels_index2.append(m)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i in levels_index2[0]:\n",
    "    for j in levels_index2[1]:\n",
    "        df_new_test.loc[i].loc[j]['Kt'] = df_new_test.loc[i].loc[j]['Kt'].shift(-1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_new_test = df_new_test[~(df_new_test['Kt'].isnull())]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>zen</th>\n",
       "      <th>dw_solar</th>\n",
       "      <th>uw_solar</th>\n",
       "      <th>direct_n</th>\n",
       "      <th>diffuse</th>\n",
       "      <th>dw_ir</th>\n",
       "      <th>dw_casetemp</th>\n",
       "      <th>dw_dometemp</th>\n",
       "      <th>uw_ir</th>\n",
       "      <th>uw_casetemp</th>\n",
       "      <th>...</th>\n",
       "      <th>netsolar</th>\n",
       "      <th>netir</th>\n",
       "      <th>totalnet</th>\n",
       "      <th>temp</th>\n",
       "      <th>rh</th>\n",
       "      <th>windspd</th>\n",
       "      <th>winddir</th>\n",
       "      <th>pressure</th>\n",
       "      <th>ghi</th>\n",
       "      <th>Kt</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>month</th>\n",
       "      <th>day</th>\n",
       "      <th>hour</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3.0</th>\n",
       "      <th>11.0</th>\n",
       "      <th>18.0</th>\n",
       "      <td>44.765833</td>\n",
       "      <td>815.241667</td>\n",
       "      <td>123.763333</td>\n",
       "      <td>1004.053333</td>\n",
       "      <td>79.721667</td>\n",
       "      <td>199.776667</td>\n",
       "      <td>277.12</td>\n",
       "      <td>276.648333</td>\n",
       "      <td>392.78</td>\n",
       "      <td>273.916667</td>\n",
       "      <td>...</td>\n",
       "      <td>668.746667</td>\n",
       "      <td>-193.015</td>\n",
       "      <td>475.741667</td>\n",
       "      <td>-1.26</td>\n",
       "      <td>50.875</td>\n",
       "      <td>2.656667</td>\n",
       "      <td>94.016667</td>\n",
       "      <td>833.296667</td>\n",
       "      <td>75.221941</td>\n",
       "      <td>1026.823214</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1 rows × 23 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                       zen    dw_solar    uw_solar     direct_n    diffuse  \\\n",
       "month day  hour                                                              \n",
       "3.0   11.0 18.0  44.765833  815.241667  123.763333  1004.053333  79.721667   \n",
       "\n",
       "                      dw_ir  dw_casetemp  dw_dometemp   uw_ir  uw_casetemp  \\\n",
       "month day  hour                                                              \n",
       "3.0   11.0 18.0  199.776667       277.12   276.648333  392.78   273.916667   \n",
       "\n",
       "                    ...         netsolar    netir    totalnet  temp      rh  \\\n",
       "month day  hour     ...                                                       \n",
       "3.0   11.0 18.0     ...       668.746667 -193.015  475.741667 -1.26  50.875   \n",
       "\n",
       "                  windspd    winddir    pressure        ghi           Kt  \n",
       "month day  hour                                                           \n",
       "3.0   11.0 18.0  2.656667  94.016667  833.296667  75.221941  1026.823214  \n",
       "\n",
       "[1 rows x 23 columns]"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_new_test[df_new_test['Kt']==df_new_test['Kt'].max()]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Normalize train and test dataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_norm = (df_new_train - df_new_train.mean()) / (df_new_train.max() - df_new_train.min())\n",
    "test_norm =  (df_new_test - df_new_test.mean()) / (df_new_test.max() - df_new_test.min())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_norm.reset_index(inplace=True,drop=True)\n",
    "test_norm.reset_index(inplace=True,drop=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Making train and test sets with train_norm and test_norm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "#### finding the gcf (greatest common factor) of train and test dataset's length and chop off the extra rows to make it divisible with the batchsize"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\s269596\\AppData\\Local\\Continuum\\Anaconda3\\lib\\site-packages\\ipykernel_launcher.py:2: DeprecationWarning: fractions.gcd() is deprecated. Use math.gcd() instead.\n",
      "  \n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from fractions import gcd\n",
    "gcd(train_norm.shape[0],test_norm.shape[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [],
   "source": [
    "import math\n",
    "def roundup(x):\n",
    "    return int(math.ceil(x / 100.0)) * 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_lim = roundup(train_norm.shape[0])\n",
    "test_lim = roundup(test_norm.shape[0])\n",
    "\n",
    "train_random = train_norm.sample(train_lim-train_norm.shape[0])\n",
    "test_random = test_norm.sample(test_lim-test_norm.shape[0])\n",
    "\n",
    "train_norm = train_norm.append(train_random)\n",
    "test_norm = test_norm.append(test_random)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>zen</th>\n",
       "      <th>dw_solar</th>\n",
       "      <th>uw_solar</th>\n",
       "      <th>direct_n</th>\n",
       "      <th>diffuse</th>\n",
       "      <th>dw_ir</th>\n",
       "      <th>dw_casetemp</th>\n",
       "      <th>dw_dometemp</th>\n",
       "      <th>uw_ir</th>\n",
       "      <th>uw_casetemp</th>\n",
       "      <th>...</th>\n",
       "      <th>netsolar</th>\n",
       "      <th>netir</th>\n",
       "      <th>totalnet</th>\n",
       "      <th>temp</th>\n",
       "      <th>rh</th>\n",
       "      <th>windspd</th>\n",
       "      <th>winddir</th>\n",
       "      <th>pressure</th>\n",
       "      <th>ghi</th>\n",
       "      <th>Kt</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>year</th>\n",
       "      <th>month</th>\n",
       "      <th>day</th>\n",
       "      <th>hour</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">2010.0</th>\n",
       "      <th rowspan=\"5\" valign=\"top\">1.0</th>\n",
       "      <th rowspan=\"5\" valign=\"top\">1.0</th>\n",
       "      <th>7.0</th>\n",
       "      <td>161.031471</td>\n",
       "      <td>-2.950000</td>\n",
       "      <td>-0.711765</td>\n",
       "      <td>1.138235</td>\n",
       "      <td>0.538235</td>\n",
       "      <td>224.085294</td>\n",
       "      <td>270.364706</td>\n",
       "      <td>269.979412</td>\n",
       "      <td>268.744118</td>\n",
       "      <td>268.611765</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-44.652941</td>\n",
       "      <td>-44.652941</td>\n",
       "      <td>-3.211765</td>\n",
       "      <td>56.261765</td>\n",
       "      <td>2.835294</td>\n",
       "      <td>107.311765</td>\n",
       "      <td>830.647059</td>\n",
       "      <td>33.080713</td>\n",
       "      <td>-0.030061</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8.0</th>\n",
       "      <td>155.139333</td>\n",
       "      <td>-3.753333</td>\n",
       "      <td>-0.936667</td>\n",
       "      <td>1.285000</td>\n",
       "      <td>0.513333</td>\n",
       "      <td>223.563333</td>\n",
       "      <td>270.761667</td>\n",
       "      <td>270.405000</td>\n",
       "      <td>275.378333</td>\n",
       "      <td>269.453333</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-51.813333</td>\n",
       "      <td>-51.813333</td>\n",
       "      <td>-2.645000</td>\n",
       "      <td>53.351667</td>\n",
       "      <td>3.866667</td>\n",
       "      <td>147.821667</td>\n",
       "      <td>830.728333</td>\n",
       "      <td>143.676328</td>\n",
       "      <td>-0.011214</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9.0</th>\n",
       "      <td>145.065667</td>\n",
       "      <td>-3.293333</td>\n",
       "      <td>-0.251667</td>\n",
       "      <td>1.255000</td>\n",
       "      <td>0.488333</td>\n",
       "      <td>212.618333</td>\n",
       "      <td>270.150000</td>\n",
       "      <td>269.700000</td>\n",
       "      <td>273.045000</td>\n",
       "      <td>268.510000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-60.421667</td>\n",
       "      <td>-60.421667</td>\n",
       "      <td>-3.716667</td>\n",
       "      <td>59.238333</td>\n",
       "      <td>3.315000</td>\n",
       "      <td>63.578333</td>\n",
       "      <td>831.020000</td>\n",
       "      <td>299.581054</td>\n",
       "      <td>-0.008772</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10.0</th>\n",
       "      <td>133.892500</td>\n",
       "      <td>-3.641667</td>\n",
       "      <td>-0.710000</td>\n",
       "      <td>1.795000</td>\n",
       "      <td>0.406667</td>\n",
       "      <td>215.475000</td>\n",
       "      <td>269.760000</td>\n",
       "      <td>269.403333</td>\n",
       "      <td>270.496667</td>\n",
       "      <td>267.760000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-55.031667</td>\n",
       "      <td>-55.031667</td>\n",
       "      <td>-3.923333</td>\n",
       "      <td>60.121667</td>\n",
       "      <td>3.076667</td>\n",
       "      <td>71.128333</td>\n",
       "      <td>831.073333</td>\n",
       "      <td>419.749575</td>\n",
       "      <td>-0.006431</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11.0</th>\n",
       "      <td>122.454000</td>\n",
       "      <td>-3.126667</td>\n",
       "      <td>-0.471667</td>\n",
       "      <td>1.275000</td>\n",
       "      <td>0.456667</td>\n",
       "      <td>216.165000</td>\n",
       "      <td>269.116667</td>\n",
       "      <td>268.663333</td>\n",
       "      <td>269.205000</td>\n",
       "      <td>267.470000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-53.050000</td>\n",
       "      <td>-53.050000</td>\n",
       "      <td>-4.565000</td>\n",
       "      <td>61.175000</td>\n",
       "      <td>3.535000</td>\n",
       "      <td>184.166667</td>\n",
       "      <td>831.015000</td>\n",
       "      <td>486.001859</td>\n",
       "      <td>-0.007064</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 23 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                              zen  dw_solar  uw_solar  direct_n   diffuse  \\\n",
       "year   month day hour                                                       \n",
       "2010.0 1.0   1.0 7.0   161.031471 -2.950000 -0.711765  1.138235  0.538235   \n",
       "                 8.0   155.139333 -3.753333 -0.936667  1.285000  0.513333   \n",
       "                 9.0   145.065667 -3.293333 -0.251667  1.255000  0.488333   \n",
       "                 10.0  133.892500 -3.641667 -0.710000  1.795000  0.406667   \n",
       "                 11.0  122.454000 -3.126667 -0.471667  1.275000  0.456667   \n",
       "\n",
       "                            dw_ir  dw_casetemp  dw_dometemp       uw_ir  \\\n",
       "year   month day hour                                                     \n",
       "2010.0 1.0   1.0 7.0   224.085294   270.364706   269.979412  268.744118   \n",
       "                 8.0   223.563333   270.761667   270.405000  275.378333   \n",
       "                 9.0   212.618333   270.150000   269.700000  273.045000   \n",
       "                 10.0  215.475000   269.760000   269.403333  270.496667   \n",
       "                 11.0  216.165000   269.116667   268.663333  269.205000   \n",
       "\n",
       "                       uw_casetemp    ...     netsolar      netir   totalnet  \\\n",
       "year   month day hour                 ...                                      \n",
       "2010.0 1.0   1.0 7.0    268.611765    ...          0.0 -44.652941 -44.652941   \n",
       "                 8.0    269.453333    ...          0.0 -51.813333 -51.813333   \n",
       "                 9.0    268.510000    ...          0.0 -60.421667 -60.421667   \n",
       "                 10.0   267.760000    ...          0.0 -55.031667 -55.031667   \n",
       "                 11.0   267.470000    ...          0.0 -53.050000 -53.050000   \n",
       "\n",
       "                           temp         rh   windspd     winddir    pressure  \\\n",
       "year   month day hour                                                          \n",
       "2010.0 1.0   1.0 7.0  -3.211765  56.261765  2.835294  107.311765  830.647059   \n",
       "                 8.0  -2.645000  53.351667  3.866667  147.821667  830.728333   \n",
       "                 9.0  -3.716667  59.238333  3.315000   63.578333  831.020000   \n",
       "                 10.0 -3.923333  60.121667  3.076667   71.128333  831.073333   \n",
       "                 11.0 -4.565000  61.175000  3.535000  184.166667  831.015000   \n",
       "\n",
       "                              ghi        Kt  \n",
       "year   month day hour                        \n",
       "2010.0 1.0   1.0 7.0    33.080713 -0.030061  \n",
       "                 8.0   143.676328 -0.011214  \n",
       "                 9.0   299.581054 -0.008772  \n",
       "                 10.0  419.749575 -0.006431  \n",
       "                 11.0  486.001859 -0.007064  \n",
       "\n",
       "[5 rows x 23 columns]"
      ]
     },
     "execution_count": 94,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_new_train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [],
   "source": [
    "X1 = train_norm.drop('Kt',axis=1)\n",
    "y1 = train_norm['Kt']\n",
    "\n",
    "X2 = test_norm.drop('Kt',axis=1)\n",
    "y2 = test_norm['Kt']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X1_train shape is (9400, 22)\n",
      "y1_train shape is (9400,)\n",
      "X2_test shape is (4300, 22)\n",
      "y2_test shape is (4300,)\n"
     ]
    }
   ],
   "source": [
    "print(\"X1_train shape is {}\".format(X1.shape))\n",
    "print(\"y1_train shape is {}\".format(y1.shape))\n",
    "print(\"X2_test shape is {}\".format(X2.shape))\n",
    "print(\"y2_test shape is {}\".format(y2.shape))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train = np.array(X1)\n",
    "y_train  = np.array(y1)\n",
    "X_test = np.array(X2)\n",
    "y_test = np.array(y2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### start of RNN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torch.nn as nn\n",
    "import torchvision.transforms as transforms\n",
    "import torchvision.datasets as dsets\n",
    "from torch.autograd import Variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [],
   "source": [
    "class RNNModel(nn.Module):\n",
    "    def __init__(self, input_dim, hidden_dim, layer_dim, output_dim):\n",
    "        super(RNNModel, self).__init__()\n",
    "        #Hidden Dimension\n",
    "        self.hidden_dim = hidden_dim\n",
    "        \n",
    "        # Number of hidden layers\n",
    "        self.layer_dim = layer_dim\n",
    "        \n",
    "        #Building the RNN\n",
    "        self.rnn = nn.RNN(input_dim, hidden_dim, layer_dim, batch_first=True, nonlinearity='relu')\n",
    "        \n",
    "        # Readout layer\n",
    "        self.fc = nn.Linear(hidden_dim, output_dim)\n",
    "        \n",
    "    def forward(self, x):\n",
    "        # Initializing the hidden state with zeros\n",
    "        # (layer_dim, batch_size, hidden_dim)\n",
    "        h0 = Variable(torch.zeros(self.layer_dim, x.size(0), self.hidden_dim))\n",
    "        \n",
    "        #One time step (the last one perhaps?)\n",
    "        out, hn = self.rnn(x, h0)\n",
    "        \n",
    "        # Indexing hidden state of the last time step\n",
    "        # out.size() --> ??\n",
    "        #out[:,-1,:] --> is it going to be 100,100\n",
    "        out = self.fc(out[:,-1,:])\n",
    "        # out.size() --> 100,1\n",
    "        return out\n",
    "        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Instantiating Model Class\n",
    "input_dim = 22\n",
    "hidden_dim = 15\n",
    "layer_dim = 1\n",
    "output_dim = 1\n",
    "batch_size = 100\n",
    "\n",
    "model = RNNModel(input_dim, hidden_dim, layer_dim, output_dim)\n",
    "\n",
    "# Instantiating Loss Class\n",
    "criterion = nn.MSELoss()\n",
    "\n",
    "# Instantiate Optimizer Class\n",
    "learning_rate = 0.001\n",
    "optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate)\n",
    "\n",
    "# converting numpy array to torch tensor\n",
    "X_train = torch.from_numpy(X_train)\n",
    "y_train = torch.from_numpy(y_train)\n",
    "X_test = torch.from_numpy(X_test)\n",
    "y_test = torch.from_numpy(y_test)\n",
    "\n",
    "# initializing lists to store losses over epochs:\n",
    "train_loss = []\n",
    "test_loss = []\n",
    "train_iter = []\n",
    "test_iter = []"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 2 Iteration: 200. Train_MSE: 0.0022665311116725206. Test_MSE: 0.0004758805569397013\n",
      "Epoch: 4 Iteration: 400. Train_MSE: 0.0013853006530553102. Test_MSE: 0.00048329754716710133\n",
      "Epoch: 6 Iteration: 600. Train_MSE: 0.0012469941284507513. Test_MSE: 0.0004873524110060406\n",
      "Epoch: 8 Iteration: 800. Train_MSE: 0.0015557525912299752. Test_MSE: 0.00048470089958382563\n",
      "Epoch: 10 Iteration: 1000. Train_MSE: 0.0015303556574508548. Test_MSE: 0.00048251811680206604\n",
      "Epoch: 12 Iteration: 1200. Train_MSE: 0.0008829520083963871. Test_MSE: 0.00047851106365711575\n",
      "Epoch: 15 Iteration: 1400. Train_MSE: 0.001470720861107111. Test_MSE: 0.0004732242015260288\n",
      "Epoch: 17 Iteration: 1600. Train_MSE: 0.0015222637448459864. Test_MSE: 0.00046849833108920827\n",
      "Epoch: 19 Iteration: 1800. Train_MSE: 0.0010090837022289634. Test_MSE: 0.0004645333733951604\n",
      "Epoch: 21 Iteration: 2000. Train_MSE: 0.001195061020553112. Test_MSE: 0.0004598670414216901\n",
      "Epoch: 23 Iteration: 2200. Train_MSE: 0.0014515140792354941. Test_MSE: 0.0004547470609750545\n",
      "Epoch: 25 Iteration: 2400. Train_MSE: 0.001378286979161203. Test_MSE: 0.00045181403957944544\n",
      "Epoch: 27 Iteration: 2600. Train_MSE: 0.001071914448402822. Test_MSE: 0.0004480647134193848\n",
      "Epoch: 30 Iteration: 2800. Train_MSE: 0.0012694784672930837. Test_MSE: 0.0004431467026654335\n",
      "Epoch: 32 Iteration: 3000. Train_MSE: 0.0010378042934462428. Test_MSE: 0.0004401986357003079\n",
      "Epoch: 34 Iteration: 3200. Train_MSE: 0.001078316243365407. Test_MSE: 0.00043692600548235754\n",
      "Epoch: 36 Iteration: 3400. Train_MSE: 0.0017385934479534626. Test_MSE: 0.0004329946439931\n",
      "Epoch: 38 Iteration: 3600. Train_MSE: 0.0028041473124176264. Test_MSE: 0.0004297962866106247\n",
      "Epoch: 40 Iteration: 3800. Train_MSE: 0.0010446470696479082. Test_MSE: 0.00042691555687276853\n",
      "Epoch: 43 Iteration: 4000. Train_MSE: 0.001399958855472505. Test_MSE: 0.00042402067414870165\n",
      "Epoch: 45 Iteration: 4200. Train_MSE: 0.0009298547520302236. Test_MSE: 0.00042017976035050073\n",
      "Epoch: 47 Iteration: 4400. Train_MSE: 0.0014316121814772487. Test_MSE: 0.0004179397662039523\n",
      "Epoch: 49 Iteration: 4600. Train_MSE: 0.001526016741991043. Test_MSE: 0.0004151715889419868\n",
      "Epoch: 51 Iteration: 4800. Train_MSE: 0.0011759854387491941. Test_MSE: 0.0004114277213032706\n",
      "Epoch: 53 Iteration: 5000. Train_MSE: 0.0010830375831574202. Test_MSE: 0.00040970563532794407\n",
      "Epoch: 55 Iteration: 5200. Train_MSE: 0.0007747013005428016. Test_MSE: 0.00040704584926827526\n",
      "Epoch: 58 Iteration: 5400. Train_MSE: 0.002027740003541112. Test_MSE: 0.00040385367594202167\n",
      "Epoch: 60 Iteration: 5600. Train_MSE: 0.0010744587052613497. Test_MSE: 0.00040174105325996485\n",
      "Epoch: 62 Iteration: 5800. Train_MSE: 0.0007987567223608494. Test_MSE: 0.00039974046403309037\n",
      "Epoch: 64 Iteration: 6000. Train_MSE: 0.001305467914789915. Test_MSE: 0.00039686057207648825\n",
      "Epoch: 66 Iteration: 6200. Train_MSE: 0.0008515482186339796. Test_MSE: 0.00039407963360777213\n",
      "Epoch: 68 Iteration: 6400. Train_MSE: 0.0012186446692794561. Test_MSE: 0.0003927233131118448\n",
      "Epoch: 70 Iteration: 6600. Train_MSE: 0.0012112840777263045. Test_MSE: 0.00039045134702093536\n",
      "Epoch: 73 Iteration: 6800. Train_MSE: 0.0007395263528451324. Test_MSE: 0.0003874639272907604\n",
      "Epoch: 75 Iteration: 7000. Train_MSE: 0.0011908197775483131. Test_MSE: 0.00038602841925630915\n",
      "Epoch: 77 Iteration: 7200. Train_MSE: 0.0010534238535910845. Test_MSE: 0.0003840820793353528\n",
      "Epoch: 79 Iteration: 7400. Train_MSE: 0.00110770168248564. Test_MSE: 0.0003812509924181228\n",
      "Epoch: 81 Iteration: 7600. Train_MSE: 0.001119974534958601. Test_MSE: 0.0003796642513025868\n",
      "Epoch: 83 Iteration: 7800. Train_MSE: 0.0008070841431617737. Test_MSE: 0.0003777803286535123\n",
      "Epoch: 86 Iteration: 8000. Train_MSE: 0.0007611180189996958. Test_MSE: 0.00037582397922282523\n",
      "Epoch: 88 Iteration: 8200. Train_MSE: 0.002168352948501706. Test_MSE: 0.00037333724192721693\n",
      "Epoch: 90 Iteration: 8400. Train_MSE: 0.0024653656873852015. Test_MSE: 0.0003720922841730548\n",
      "Epoch: 92 Iteration: 8600. Train_MSE: 0.0007470936980098486. Test_MSE: 0.00037010376129100245\n",
      "Epoch: 94 Iteration: 8800. Train_MSE: 0.0010412746341899037. Test_MSE: 0.00036738339894337995\n",
      "Epoch: 96 Iteration: 9000. Train_MSE: 0.0007955101318657398. Test_MSE: 0.00036656901594278476\n",
      "Epoch: 98 Iteration: 9200. Train_MSE: 0.0006292394828051329. Test_MSE: 0.000364633351228354\n",
      "Epoch: 101 Iteration: 9400. Train_MSE: 0.0009425417520105839. Test_MSE: 0.00036209040570231877\n",
      "Epoch: 103 Iteration: 9600. Train_MSE: 0.0009257920901291072. Test_MSE: 0.0003610240764534801\n",
      "Epoch: 105 Iteration: 9800. Train_MSE: 0.0007407761295326054. Test_MSE: 0.0003597040252120651\n",
      "Epoch: 107 Iteration: 10000. Train_MSE: 0.0028876790311187506. Test_MSE: 0.0003574629848581251\n",
      "Epoch: 109 Iteration: 10200. Train_MSE: 0.003112366423010826. Test_MSE: 0.0003555451293064192\n",
      "Epoch: 111 Iteration: 10400. Train_MSE: 0.0009373597567901015. Test_MSE: 0.00035470648264902496\n",
      "Epoch: 113 Iteration: 10600. Train_MSE: 0.0009276830242015421. Test_MSE: 0.0003531167451012711\n",
      "Epoch: 116 Iteration: 10800. Train_MSE: 0.00046941585605964065. Test_MSE: 0.00035068478841381544\n",
      "Epoch: 118 Iteration: 11000. Train_MSE: 0.0008800935465842485. Test_MSE: 0.0003499474378493468\n",
      "Epoch: 120 Iteration: 11200. Train_MSE: 0.0005493790376931429. Test_MSE: 0.0003484173366080371\n",
      "Epoch: 122 Iteration: 11400. Train_MSE: 0.0009024508763104677. Test_MSE: 0.0003459618502525777\n",
      "Epoch: 124 Iteration: 11600. Train_MSE: 0.0008519704570062459. Test_MSE: 0.0003451924019264994\n",
      "Epoch: 126 Iteration: 11800. Train_MSE: 0.0009367833263240755. Test_MSE: 0.0003437942261648335\n",
      "Epoch: 129 Iteration: 12000. Train_MSE: 0.0007207894814200699. Test_MSE: 0.0003421381902817136\n",
      "Epoch: 131 Iteration: 12200. Train_MSE: 0.0021662795916199684. Test_MSE: 0.0003404073639404364\n",
      "Epoch: 133 Iteration: 12400. Train_MSE: 0.009110613726079464. Test_MSE: 0.00033957513323962756\n",
      "Epoch: 135 Iteration: 12600. Train_MSE: 0.0008581144502386451. Test_MSE: 0.0003379685829872528\n",
      "Epoch: 137 Iteration: 12800. Train_MSE: 0.0006765857106074691. Test_MSE: 0.0003356752999500421\n",
      "Epoch: 139 Iteration: 13000. Train_MSE: 0.0006826106691733003. Test_MSE: 0.0003352803875762902\n",
      "Epoch: 141 Iteration: 13200. Train_MSE: 0.0004688453918788582. Test_MSE: 0.0003337156969582701\n",
      "Epoch: 144 Iteration: 13400. Train_MSE: 0.0007793797412887216. Test_MSE: 0.00033154172672680626\n",
      "Epoch: 146 Iteration: 13600. Train_MSE: 0.000747901969589293. Test_MSE: 0.0003309405009369977\n",
      "Epoch: 148 Iteration: 13800. Train_MSE: 0.011114438995718956. Test_MSE: 0.0003299512752687935\n",
      "Epoch: 150 Iteration: 14000. Train_MSE: 0.0006796258967369795. Test_MSE: 0.0003280219268668106\n",
      "Epoch: 152 Iteration: 14200. Train_MSE: 0.004087960813194513. Test_MSE: 0.0003266209838283272\n",
      "Epoch: 154 Iteration: 14400. Train_MSE: 0.011206060647964478. Test_MSE: 0.00032605318957296657\n",
      "Epoch: 156 Iteration: 14600. Train_MSE: 0.0008344344678334892. Test_MSE: 0.0003246156153120705\n",
      "Epoch: 159 Iteration: 14800. Train_MSE: 0.0006169505068100989. Test_MSE: 0.000322588317705078\n",
      "Epoch: 161 Iteration: 15000. Train_MSE: 0.0009406022727489471. Test_MSE: 0.00032216795580968004\n",
      "Epoch: 163 Iteration: 15200. Train_MSE: 0.004636220633983612. Test_MSE: 0.0003210789810777547\n",
      "Epoch: 165 Iteration: 15400. Train_MSE: 0.0010683477157726884. Test_MSE: 0.00031868123522810584\n",
      "Epoch: 167 Iteration: 15600. Train_MSE: 0.0006858233246020973. Test_MSE: 0.0003183311094887572\n",
      "Epoch: 169 Iteration: 15800. Train_MSE: 0.0021149558015167713. Test_MSE: 0.0003171672065300038\n",
      "Epoch: 172 Iteration: 16000. Train_MSE: 0.0006266087875701487. Test_MSE: 0.00031557608795217644\n",
      "Epoch: 174 Iteration: 16200. Train_MSE: 0.0009959450690075755. Test_MSE: 0.00031434813407542816\n",
      "Epoch: 176 Iteration: 16400. Train_MSE: 0.0006469865329563618. Test_MSE: 0.0003137125672988386\n",
      "Epoch: 178 Iteration: 16600. Train_MSE: 0.0005890403408557177. Test_MSE: 0.00031228436365996707\n",
      "Epoch: 180 Iteration: 16800. Train_MSE: 0.0005425842246040702. Test_MSE: 0.0003104357016661367\n",
      "Epoch: 182 Iteration: 17000. Train_MSE: 0.0007684229058213532. Test_MSE: 0.00031023342163313057\n",
      "Epoch: 184 Iteration: 17200. Train_MSE: 0.0011601164005696774. Test_MSE: 0.0003090767706099942\n",
      "Epoch: 187 Iteration: 17400. Train_MSE: 0.002239962574094534. Test_MSE: 0.0003071296216097461\n",
      "Epoch: 189 Iteration: 17600. Train_MSE: 0.0006710703019052744. Test_MSE: 0.00030665122140642907\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 191 Iteration: 17800. Train_MSE: 0.0005244762869551778. Test_MSE: 0.0003058307902814137\n",
      "Epoch: 193 Iteration: 18000. Train_MSE: 0.0006711764144711196. Test_MSE: 0.00030407162032309107\n",
      "Epoch: 195 Iteration: 18200. Train_MSE: 0.002515346510335803. Test_MSE: 0.0003031109595392716\n",
      "Epoch: 197 Iteration: 18400. Train_MSE: 0.000518552609719336. Test_MSE: 0.00030249829273620816\n",
      "Epoch: 199 Iteration: 18600. Train_MSE: 0.0006288933800533414. Test_MSE: 0.00030128028951644653\n",
      "Epoch: 202 Iteration: 18800. Train_MSE: 0.0005897278897464275. Test_MSE: 0.0002995836674212437\n",
      "Epoch: 204 Iteration: 19000. Train_MSE: 0.0007885417435318232. Test_MSE: 0.000299303710222433\n",
      "Epoch: 206 Iteration: 19200. Train_MSE: 0.00042799607035703957. Test_MSE: 0.00029835253081570725\n",
      "Epoch: 208 Iteration: 19400. Train_MSE: 0.0005792034207843244. Test_MSE: 0.0002963060749145437\n",
      "Epoch: 210 Iteration: 19600. Train_MSE: 0.0005205718334764242. Test_MSE: 0.00029612764343043304\n",
      "Epoch: 212 Iteration: 19800. Train_MSE: 0.00041425772360526025. Test_MSE: 0.0002950953535497969\n",
      "Epoch: 215 Iteration: 20000. Train_MSE: 0.0005283962818793952. Test_MSE: 0.0002936158448530355\n",
      "Epoch: 217 Iteration: 20200. Train_MSE: 0.0008478852105326951. Test_MSE: 0.0002926986185356203\n",
      "Epoch: 219 Iteration: 20400. Train_MSE: 0.0005204079207032919. Test_MSE: 0.0002921724742816531\n",
      "Epoch: 221 Iteration: 20600. Train_MSE: 0.000507754331920296. Test_MSE: 0.0002908214438296096\n",
      "Epoch: 223 Iteration: 20800. Train_MSE: 0.0004333551914896816. Test_MSE: 0.0002893236973565976\n",
      "Epoch: 225 Iteration: 21000. Train_MSE: 0.0007374293636530638. Test_MSE: 0.00028923814887286666\n",
      "Epoch: 227 Iteration: 21200. Train_MSE: 0.0004350905364844948. Test_MSE: 0.00028824248945525055\n",
      "Epoch: 230 Iteration: 21400. Train_MSE: 0.0003929268859792501. Test_MSE: 0.00028655834239405784\n",
      "Epoch: 232 Iteration: 21600. Train_MSE: 0.0004934990429319441. Test_MSE: 0.00028618634659915857\n",
      "Epoch: 234 Iteration: 21800. Train_MSE: 0.0005005169659852982. Test_MSE: 0.00028543021621453085\n",
      "Epoch: 236 Iteration: 22000. Train_MSE: 0.0005639956798404455. Test_MSE: 0.0002840037004042612\n",
      "Epoch: 238 Iteration: 22200. Train_MSE: 0.0013403763296082616. Test_MSE: 0.00028329859047787506\n",
      "Epoch: 240 Iteration: 22400. Train_MSE: 0.000472171523142606. Test_MSE: 0.0002826260670290928\n",
      "Epoch: 243 Iteration: 22600. Train_MSE: 0.0006715641939081252. Test_MSE: 0.0002816820807046579\n",
      "Epoch: 245 Iteration: 22800. Train_MSE: 0.00035466867848299444. Test_MSE: 0.000280135640602234\n",
      "Epoch: 247 Iteration: 23000. Train_MSE: 0.0006447979249060154. Test_MSE: 0.00027996673214817253\n",
      "Epoch: 249 Iteration: 23200. Train_MSE: 0.000613485521171242. Test_MSE: 0.0002790317248424447\n",
      "Epoch: 251 Iteration: 23400. Train_MSE: 0.0005584428436122835. Test_MSE: 0.0002773172430461184\n",
      "Epoch: 253 Iteration: 23600. Train_MSE: 0.0005194660043343902. Test_MSE: 0.00027728998877668743\n",
      "Epoch: 255 Iteration: 23800. Train_MSE: 0.00041756630525924265. Test_MSE: 0.0002763636377452358\n",
      "Epoch: 258 Iteration: 24000. Train_MSE: 0.0007557188509963453. Test_MSE: 0.00027495558626301433\n",
      "Epoch: 260 Iteration: 24200. Train_MSE: 0.0005096193635836244. Test_MSE: 0.0002743534783033785\n",
      "Epoch: 262 Iteration: 24400. Train_MSE: 0.0004964670515619218. Test_MSE: 0.00027394191191190275\n",
      "Epoch: 264 Iteration: 24600. Train_MSE: 0.0005202091997489333. Test_MSE: 0.0002725896428428158\n",
      "Epoch: 266 Iteration: 24800. Train_MSE: 0.0003486275381874293. Test_MSE: 0.00027139958130833173\n",
      "Epoch: 268 Iteration: 25000. Train_MSE: 0.0006809068727307022. Test_MSE: 0.00027137767688220225\n",
      "Epoch: 270 Iteration: 25200. Train_MSE: 0.0004949046415276825. Test_MSE: 0.000270458870841674\n",
      "Epoch: 273 Iteration: 25400. Train_MSE: 0.0003180113562848419. Test_MSE: 0.0002690094651755942\n",
      "Epoch: 275 Iteration: 25600. Train_MSE: 0.0006173797883093357. Test_MSE: 0.0002688306408928421\n",
      "Epoch: 277 Iteration: 25800. Train_MSE: 0.0005865946877747774. Test_MSE: 0.0002681659614262668\n",
      "Epoch: 279 Iteration: 26000. Train_MSE: 0.0004644423315767199. Test_MSE: 0.00026668260619962345\n",
      "Epoch: 281 Iteration: 26200. Train_MSE: 0.0005699865287169814. Test_MSE: 0.00026630730827734476\n",
      "Epoch: 283 Iteration: 26400. Train_MSE: 0.00045330473221838474. Test_MSE: 0.0002656614782233953\n",
      "Epoch: 286 Iteration: 26600. Train_MSE: 0.00039743230445310473. Test_MSE: 0.00026474222398293083\n",
      "Epoch: 288 Iteration: 26800. Train_MSE: 0.00148832518607378. Test_MSE: 0.0002635596069572855\n",
      "Epoch: 290 Iteration: 27000. Train_MSE: 0.0021927813068032265. Test_MSE: 0.0002634120882393926\n",
      "Epoch: 292 Iteration: 27200. Train_MSE: 0.00037255213828757405. Test_MSE: 0.00026247114353739835\n",
      "Epoch: 294 Iteration: 27400. Train_MSE: 0.000552718120161444. Test_MSE: 0.0002609911056397733\n",
      "Epoch: 296 Iteration: 27600. Train_MSE: 0.00047196607920341194. Test_MSE: 0.0002610560833735048\n",
      "Epoch: 298 Iteration: 27800. Train_MSE: 0.0002865866699721664. Test_MSE: 0.00026021899537794494\n",
      "Epoch: 301 Iteration: 28000. Train_MSE: 0.00039011333137750626. Test_MSE: 0.00025888874935642514\n",
      "Epoch: 303 Iteration: 28200. Train_MSE: 0.0006418245029635727. Test_MSE: 0.0002585735351005296\n",
      "Epoch: 305 Iteration: 28400. Train_MSE: 0.0004664798907469958. Test_MSE: 0.00025813362033298443\n",
      "Epoch: 307 Iteration: 28600. Train_MSE: 0.0022967278491705656. Test_MSE: 0.00025695983254497184\n",
      "Epoch: 309 Iteration: 28800. Train_MSE: 0.0029526492580771446. Test_MSE: 0.00025594958430446294\n",
      "Epoch: 311 Iteration: 29000. Train_MSE: 0.000594856683164835. Test_MSE: 0.0002559438020037737\n",
      "Epoch: 313 Iteration: 29200. Train_MSE: 0.0004737835843116045. Test_MSE: 0.0002551993829661324\n",
      "Epoch: 316 Iteration: 29400. Train_MSE: 0.00027936685364693403. Test_MSE: 0.00025381547375102854\n",
      "Epoch: 318 Iteration: 29600. Train_MSE: 0.000467973412014544. Test_MSE: 0.00025377151897499646\n",
      "Epoch: 320 Iteration: 29800. Train_MSE: 0.00027570148813538253. Test_MSE: 0.0002531289687614597\n",
      "Epoch: 322 Iteration: 30000. Train_MSE: 0.0004033349687233567. Test_MSE: 0.0002517020664460633\n",
      "Epoch: 324 Iteration: 30200. Train_MSE: 0.0005938870599493384. Test_MSE: 0.00025157545874236476\n",
      "Epoch: 326 Iteration: 30400. Train_MSE: 0.000599283492192626. Test_MSE: 0.0002510246320551827\n",
      "Epoch: 329 Iteration: 30600. Train_MSE: 0.0003669543075375259. Test_MSE: 0.0002501049109737336\n",
      "Epoch: 331 Iteration: 30800. Train_MSE: 0.0017204498872160912. Test_MSE: 0.00024921037175031313\n",
      "Epoch: 333 Iteration: 31000. Train_MSE: 0.008777633309364319. Test_MSE: 0.00024907758754236166\n",
      "Epoch: 335 Iteration: 31200. Train_MSE: 0.00041859920020215213. Test_MSE: 0.0002482087858012782\n",
      "Epoch: 337 Iteration: 31400. Train_MSE: 0.0003464565670583397. Test_MSE: 0.0002468767322681583\n",
      "Epoch: 339 Iteration: 31600. Train_MSE: 0.0004074738535564393. Test_MSE: 0.00024698519058448225\n",
      "Epoch: 341 Iteration: 31800. Train_MSE: 0.0003037653223145753. Test_MSE: 0.0002462353025489699\n",
      "Epoch: 344 Iteration: 32000. Train_MSE: 0.000491094309836626. Test_MSE: 0.00024503389750058847\n",
      "Epoch: 346 Iteration: 32200. Train_MSE: 0.00044609728502109647. Test_MSE: 0.00024483574159280577\n",
      "Epoch: 348 Iteration: 32400. Train_MSE: 0.011148597113788128. Test_MSE: 0.00024453017931493667\n",
      "Epoch: 350 Iteration: 32600. Train_MSE: 0.0004118646611459553. Test_MSE: 0.00024339357429950742\n",
      "Epoch: 352 Iteration: 32800. Train_MSE: 0.003790105227380991. Test_MSE: 0.00024269654691680027\n",
      "Epoch: 354 Iteration: 33000. Train_MSE: 0.011101272888481617. Test_MSE: 0.0002426751647434155\n",
      "Epoch: 356 Iteration: 33200. Train_MSE: 0.0004942538216710091. Test_MSE: 0.00024188712438458316\n",
      "Epoch: 359 Iteration: 33400. Train_MSE: 0.00035506568383425474. Test_MSE: 0.00024065487967190417\n",
      "Epoch: 361 Iteration: 33600. Train_MSE: 0.0005223139887675643. Test_MSE: 0.00024065880362605932\n",
      "Epoch: 363 Iteration: 33800. Train_MSE: 0.004255602136254311. Test_MSE: 0.0002401914803444919\n",
      "Epoch: 365 Iteration: 34000. Train_MSE: 0.0005610916414298117. Test_MSE: 0.00023873722291617417\n",
      "Epoch: 367 Iteration: 34200. Train_MSE: 0.00046297957305796444. Test_MSE: 0.00023876691978306846\n",
      "Epoch: 369 Iteration: 34400. Train_MSE: 0.0020051856990903616. Test_MSE: 0.00023825833393706881\n",
      "Epoch: 372 Iteration: 34600. Train_MSE: 0.00032016128534451127. Test_MSE: 0.0002373262421199679\n",
      "Epoch: 374 Iteration: 34800. Train_MSE: 0.0008423354593105614. Test_MSE: 0.0002366427577522388\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 376 Iteration: 35000. Train_MSE: 0.00037763852742500603. Test_MSE: 0.00023648267742490015\n",
      "Epoch: 378 Iteration: 35200. Train_MSE: 0.0003109891258645803. Test_MSE: 0.00023568193226295266\n",
      "Epoch: 380 Iteration: 35400. Train_MSE: 0.0002954426745418459. Test_MSE: 0.00023458306207476803\n",
      "Epoch: 382 Iteration: 35600. Train_MSE: 0.00047930952860042453. Test_MSE: 0.00023471596004918967\n",
      "Epoch: 384 Iteration: 35800. Train_MSE: 0.000740390329156071. Test_MSE: 0.00023417245272701445\n",
      "Epoch: 387 Iteration: 36000. Train_MSE: 0.0018943487666547298. Test_MSE: 0.0002330308208816669\n",
      "Epoch: 389 Iteration: 36200. Train_MSE: 0.00041098546353168786. Test_MSE: 0.00023281868588157187\n",
      "Epoch: 391 Iteration: 36400. Train_MSE: 0.0002990940702147782. Test_MSE: 0.0002325286898575487\n",
      "Epoch: 393 Iteration: 36600. Train_MSE: 0.0003465164336375892. Test_MSE: 0.00023148363279727592\n",
      "Epoch: 395 Iteration: 36800. Train_MSE: 0.002119992859661579. Test_MSE: 0.00023103697668385483\n",
      "Epoch: 397 Iteration: 37000. Train_MSE: 0.0002874051278922707. Test_MSE: 0.00023086276356345427\n",
      "Epoch: 399 Iteration: 37200. Train_MSE: 0.00037581101059913635. Test_MSE: 0.00023019083562493173\n",
      "Epoch: 402 Iteration: 37400. Train_MSE: 0.00036409313906915486. Test_MSE: 0.00022909933267020865\n",
      "Epoch: 404 Iteration: 37600. Train_MSE: 0.0006255425396375358. Test_MSE: 0.00022912972465924993\n",
      "Epoch: 406 Iteration: 37800. Train_MSE: 0.00024124873743858188. Test_MSE: 0.00022868350749854005\n",
      "Epoch: 408 Iteration: 38000. Train_MSE: 0.00041117510409094393. Test_MSE: 0.00022743654137122698\n",
      "Epoch: 410 Iteration: 38200. Train_MSE: 0.0003076626162510365. Test_MSE: 0.00022750632583583494\n",
      "Epoch: 412 Iteration: 38400. Train_MSE: 0.0002443669072818011. Test_MSE: 0.0002270013857037196\n",
      "Epoch: 415 Iteration: 38600. Train_MSE: 0.00030333735048770905. Test_MSE: 0.00022612868540535517\n",
      "Epoch: 417 Iteration: 38800. Train_MSE: 0.0006929260562174022. Test_MSE: 0.00022557220850256659\n",
      "Epoch: 419 Iteration: 39000. Train_MSE: 0.0003278300864621997. Test_MSE: 0.00022541246633646603\n",
      "Epoch: 421 Iteration: 39200. Train_MSE: 0.00029370636912062764. Test_MSE: 0.00022463741334807143\n",
      "Epoch: 423 Iteration: 39400. Train_MSE: 0.00023400397913064808. Test_MSE: 0.00022374148948161758\n",
      "Epoch: 425 Iteration: 39600. Train_MSE: 0.0005421342793852091. Test_MSE: 0.00022386896413730258\n",
      "Epoch: 427 Iteration: 39800. Train_MSE: 0.00026414633612148464. Test_MSE: 0.00022336685582851758\n",
      "Epoch: 430 Iteration: 40000. Train_MSE: 0.0002445298887323588. Test_MSE: 0.00022231742677198557\n",
      "Epoch: 432 Iteration: 40200. Train_MSE: 0.0003180423809681088. Test_MSE: 0.0002221429400786946\n",
      "Epoch: 434 Iteration: 40400. Train_MSE: 0.0003119783359579742. Test_MSE: 0.00022183588336951738\n",
      "Epoch: 436 Iteration: 40600. Train_MSE: 0.0002957835095003247. Test_MSE: 0.00022096946767759265\n",
      "Epoch: 438 Iteration: 40800. Train_MSE: 0.0009635158348828554. Test_MSE: 0.00022062978078430756\n",
      "Epoch: 440 Iteration: 41000. Train_MSE: 0.0002895637007895857. Test_MSE: 0.00022034515731463413\n",
      "Epoch: 443 Iteration: 41200. Train_MSE: 0.0004386223154142499. Test_MSE: 0.0002198297120967395\n",
      "Epoch: 445 Iteration: 41400. Train_MSE: 0.0002347114495933056. Test_MSE: 0.00021876702372175555\n",
      "Epoch: 447 Iteration: 41600. Train_MSE: 0.0004823537601623684. Test_MSE: 0.00021880407245037736\n",
      "Epoch: 449 Iteration: 41800. Train_MSE: 0.0003465379122644663. Test_MSE: 0.00021833482508956098\n",
      "Epoch: 451 Iteration: 42000. Train_MSE: 0.00043130700942128897. Test_MSE: 0.0002172615829942992\n",
      "Epoch: 453 Iteration: 42200. Train_MSE: 0.0003683821123559028. Test_MSE: 0.0002173977779895614\n",
      "Epoch: 455 Iteration: 42400. Train_MSE: 0.0002707931271288544. Test_MSE: 0.0002169049132548493\n",
      "Epoch: 458 Iteration: 42600. Train_MSE: 0.0004076457116752863. Test_MSE: 0.00021605251524372827\n",
      "Epoch: 460 Iteration: 42800. Train_MSE: 0.0003811022615991533. Test_MSE: 0.00021564978810559627\n",
      "Epoch: 462 Iteration: 43000. Train_MSE: 0.00042771970038302243. Test_MSE: 0.00021552577702424542\n",
      "Epoch: 464 Iteration: 43200. Train_MSE: 0.0002840715169440955. Test_MSE: 0.00021472324391855018\n",
      "Epoch: 466 Iteration: 43400. Train_MSE: 0.00026604783488437533. Test_MSE: 0.00021398666995504564\n",
      "Epoch: 468 Iteration: 43600. Train_MSE: 0.0005668946541845798. Test_MSE: 0.0002141311510421282\n",
      "Epoch: 470 Iteration: 43800. Train_MSE: 0.0002813641040120274. Test_MSE: 0.00021365538674070694\n",
      "Epoch: 473 Iteration: 44000. Train_MSE: 0.00023839750792831182. Test_MSE: 0.00021270918093668615\n",
      "Epoch: 475 Iteration: 44200. Train_MSE: 0.0004436044837348163. Test_MSE: 0.00021265788739280252\n",
      "Epoch: 477 Iteration: 44400. Train_MSE: 0.0004078885540366173. Test_MSE: 0.00021235308089868503\n",
      "Epoch: 479 Iteration: 44600. Train_MSE: 0.000271360797341913. Test_MSE: 0.00021141988728156063\n",
      "Epoch: 481 Iteration: 44800. Train_MSE: 0.00044536739005707204. Test_MSE: 0.00021127203531577538\n",
      "Epoch: 483 Iteration: 45000. Train_MSE: 0.0003391553182154894. Test_MSE: 0.00021096107314178147\n",
      "Epoch: 486 Iteration: 45200. Train_MSE: 0.0002610505325719714. Test_MSE: 0.0002104245476484867\n",
      "Epoch: 488 Iteration: 45400. Train_MSE: 0.0012610646663233638. Test_MSE: 0.0002096140716479579\n",
      "Epoch: 490 Iteration: 45600. Train_MSE: 0.0021178009919822216. Test_MSE: 0.00020961204804862467\n",
      "Epoch: 492 Iteration: 45800. Train_MSE: 0.00023818266345188022. Test_MSE: 0.0002090995592254075\n",
      "Epoch: 494 Iteration: 46000. Train_MSE: 0.00043322984129190445. Test_MSE: 0.00020815359449304505\n",
      "Epoch: 496 Iteration: 46200. Train_MSE: 0.00038550939643755555. Test_MSE: 0.00020831540929235949\n",
      "Epoch: 498 Iteration: 46400. Train_MSE: 0.00017805093375500292. Test_MSE: 0.00020784522132408177\n",
      "Epoch: 501 Iteration: 46600. Train_MSE: 0.00022452206758316606. Test_MSE: 0.000207018837540888\n",
      "Epoch: 503 Iteration: 46800. Train_MSE: 0.0005333327571861446. Test_MSE: 0.00020678863552616702\n",
      "Epoch: 505 Iteration: 47000. Train_MSE: 0.0003857272968161851. Test_MSE: 0.00020658785354271255\n",
      "Epoch: 507 Iteration: 47200. Train_MSE: 0.0020747575908899307. Test_MSE: 0.00020590424758108668\n",
      "Epoch: 509 Iteration: 47400. Train_MSE: 0.002977746771648526. Test_MSE: 0.0002052276654550135\n",
      "Epoch: 511 Iteration: 47600. Train_MSE: 0.0004959543002769351. Test_MSE: 0.0002053566289717309\n",
      "Epoch: 513 Iteration: 47800. Train_MSE: 0.00030881722341291606. Test_MSE: 0.00020496895120022902\n",
      "Epoch: 516 Iteration: 48000. Train_MSE: 0.00022954880842007697. Test_MSE: 0.000204012901773608\n",
      "Epoch: 518 Iteration: 48200. Train_MSE: 0.0003467573842499405. Test_MSE: 0.00020403023991019212\n",
      "Epoch: 520 Iteration: 48400. Train_MSE: 0.0001885357196442783. Test_MSE: 0.00020371355814304584\n",
      "Epoch: 522 Iteration: 48600. Train_MSE: 0.00024328933795914054. Test_MSE: 0.00020280422031695772\n",
      "Epoch: 524 Iteration: 48800. Train_MSE: 0.0005099503323435783. Test_MSE: 0.0002028049302174913\n",
      "Epoch: 526 Iteration: 49000. Train_MSE: 0.0004598403465934098. Test_MSE: 0.00020252678719450372\n",
      "Epoch: 529 Iteration: 49200. Train_MSE: 0.00022731169883627445. Test_MSE: 0.00020196706785907402\n",
      "Epoch: 531 Iteration: 49400. Train_MSE: 0.0015591196715831757. Test_MSE: 0.00020133128311557456\n",
      "Epoch: 533 Iteration: 49600. Train_MSE: 0.008706741034984589. Test_MSE: 0.00020133072949256144\n",
      "Epoch: 535 Iteration: 49800. Train_MSE: 0.00026890207664109766. Test_MSE: 0.00020082113515629044\n",
      "Epoch: 537 Iteration: 50000. Train_MSE: 0.00026463979156687856. Test_MSE: 0.0001999458128883618\n",
      "Epoch: 539 Iteration: 50200. Train_MSE: 0.00032391215790994465. Test_MSE: 0.0002001108067042688\n",
      "Epoch: 541 Iteration: 50400. Train_MSE: 0.00023921536921989173. Test_MSE: 0.00019968163534657383\n",
      "Epoch: 544 Iteration: 50600. Train_MSE: 0.0003796951787080616. Test_MSE: 0.00019891637860777145\n",
      "Epoch: 546 Iteration: 50800. Train_MSE: 0.0003446977643761784. Test_MSE: 0.00019874913140321466\n",
      "Epoch: 548 Iteration: 51000. Train_MSE: 0.01123859267681837. Test_MSE: 0.0001986666043740852\n",
      "Epoch: 550 Iteration: 51200. Train_MSE: 0.00029593706130981445. Test_MSE: 0.0001979295005736949\n",
      "Epoch: 552 Iteration: 51400. Train_MSE: 0.00365405878983438. Test_MSE: 0.00019750344320209137\n",
      "Epoch: 554 Iteration: 51600. Train_MSE: 0.01108180359005928. Test_MSE: 0.0001976082672975587\n",
      "Epoch: 556 Iteration: 51800. Train_MSE: 0.0003416914842091501. Test_MSE: 0.00019711286240424778\n",
      "Epoch: 559 Iteration: 52000. Train_MSE: 0.00025628728326410055. Test_MSE: 0.0001962256375420966\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 561 Iteration: 52200. Train_MSE: 0.0003823031729552895. Test_MSE: 0.0001962718553038181\n",
      "Epoch: 563 Iteration: 52400. Train_MSE: 0.004138375166803598. Test_MSE: 0.0001960623959696018\n",
      "Epoch: 565 Iteration: 52600. Train_MSE: 0.00041552435141056776. Test_MSE: 0.00019509885267889678\n",
      "Epoch: 567 Iteration: 52800. Train_MSE: 0.00038007981493137777. Test_MSE: 0.00019520321537990247\n",
      "Epoch: 569 Iteration: 53000. Train_MSE: 0.001976546598598361. Test_MSE: 0.00019492609049793767\n",
      "Epoch: 572 Iteration: 53200. Train_MSE: 0.00020537880482152104. Test_MSE: 0.00019432510873014215\n",
      "Epoch: 574 Iteration: 53400. Train_MSE: 0.0008028798620216548. Test_MSE: 0.00019382183146829856\n",
      "Epoch: 576 Iteration: 53600. Train_MSE: 0.00028499570908024907. Test_MSE: 0.0001937689176356984\n",
      "Epoch: 578 Iteration: 53800. Train_MSE: 0.00019556704501155764. Test_MSE: 0.00019329242574109133\n",
      "Epoch: 580 Iteration: 54000. Train_MSE: 0.00022340011491905898. Test_MSE: 0.00019255821722507916\n",
      "Epoch: 582 Iteration: 54200. Train_MSE: 0.0003759496321436018. Test_MSE: 0.00019273323044858475\n",
      "Epoch: 584 Iteration: 54400. Train_MSE: 0.0005351909785531461. Test_MSE: 0.00019244279941028517\n",
      "Epoch: 587 Iteration: 54600. Train_MSE: 0.0017635737312957644. Test_MSE: 0.00019167634447312475\n",
      "Epoch: 589 Iteration: 54800. Train_MSE: 0.0003195157623849809. Test_MSE: 0.00019148544992131461\n",
      "Epoch: 591 Iteration: 55000. Train_MSE: 0.00021099265723023564. Test_MSE: 0.0001913911445310072\n",
      "Epoch: 593 Iteration: 55200. Train_MSE: 0.00022813559917267412. Test_MSE: 0.00019069506453182523\n",
      "Epoch: 595 Iteration: 55400. Train_MSE: 0.0019621949177235365. Test_MSE: 0.00019044547345941766\n",
      "Epoch: 597 Iteration: 55600. Train_MSE: 0.00021181894408073276. Test_MSE: 0.0001904041019588518\n",
      "Epoch: 599 Iteration: 55800. Train_MSE: 0.00026133807841688395. Test_MSE: 0.0001899804556113272\n",
      "Epoch: 602 Iteration: 56000. Train_MSE: 0.0003033639513887465. Test_MSE: 0.00018915492747201352\n",
      "Epoch: 604 Iteration: 56200. Train_MSE: 0.0005708368262276053. Test_MSE: 0.0001892222221157974\n",
      "Epoch: 606 Iteration: 56400. Train_MSE: 0.00015961624740157276. Test_MSE: 0.00018900134646192507\n",
      "Epoch: 608 Iteration: 56600. Train_MSE: 0.00035315274726599455. Test_MSE: 0.00018815604740436156\n",
      "Epoch: 610 Iteration: 56800. Train_MSE: 0.00024347934231627733. Test_MSE: 0.00018828630310473355\n",
      "Epoch: 612 Iteration: 57000. Train_MSE: 0.00018544522754382342. Test_MSE: 0.0001879895866514209\n",
      "Epoch: 615 Iteration: 57200. Train_MSE: 0.00020523702551145107. Test_MSE: 0.0001874189889020276\n",
      "Epoch: 617 Iteration: 57400. Train_MSE: 0.0006435625837184489. Test_MSE: 0.00018699026015137995\n",
      "Epoch: 619 Iteration: 57600. Train_MSE: 0.0002532640064600855. Test_MSE: 0.00018693334224108128\n",
      "Epoch: 621 Iteration: 57800. Train_MSE: 0.00019160691590514034. Test_MSE: 0.00018645949915547212\n",
      "Epoch: 623 Iteration: 58000. Train_MSE: 0.00017058741650544107. Test_MSE: 0.00018586304987938863\n",
      "Epoch: 625 Iteration: 58200. Train_MSE: 0.0004765736812260002. Test_MSE: 0.00018603381006622192\n",
      "Epoch: 627 Iteration: 58400. Train_MSE: 0.00018837886455003172. Test_MSE: 0.00018575797328572807\n",
      "Epoch: 630 Iteration: 58600. Train_MSE: 0.00021544260380323976. Test_MSE: 0.0001850190593572711\n",
      "Epoch: 632 Iteration: 58800. Train_MSE: 0.0002595831756480038. Test_MSE: 0.0001848788486559376\n",
      "Epoch: 634 Iteration: 59000. Train_MSE: 0.00023672590032219887. Test_MSE: 0.00018476351847978624\n",
      "Epoch: 636 Iteration: 59200. Train_MSE: 0.00019358153804205358. Test_MSE: 0.0001841871291616347\n",
      "Epoch: 638 Iteration: 59400. Train_MSE: 0.0008407325949519873. Test_MSE: 0.0001840137719724781\n",
      "Epoch: 640 Iteration: 59600. Train_MSE: 0.00022187959984876215. Test_MSE: 0.00018388357947726763\n",
      "Epoch: 643 Iteration: 59800. Train_MSE: 0.00032537613878957927. Test_MSE: 0.00018357635713980774\n",
      "Epoch: 645 Iteration: 60000. Train_MSE: 0.0001946683187270537. Test_MSE: 0.00018275114196822378\n",
      "Epoch: 647 Iteration: 60200. Train_MSE: 0.0004421782505232841. Test_MSE: 0.00018281876153629598\n",
      "Epoch: 649 Iteration: 60400. Train_MSE: 0.00022778272978030145. Test_MSE: 0.00018258380641768282\n",
      "Epoch: 651 Iteration: 60600. Train_MSE: 0.00038765399949625134. Test_MSE: 0.00018185442643274435\n",
      "Epoch: 653 Iteration: 60800. Train_MSE: 0.00031645424314774573. Test_MSE: 0.0001820368561912487\n",
      "Epoch: 655 Iteration: 61000. Train_MSE: 0.00020763816428370774. Test_MSE: 0.0001817297233825921\n",
      "Epoch: 658 Iteration: 61200. Train_MSE: 0.000265024253167212. Test_MSE: 0.00018115669622802315\n",
      "Epoch: 660 Iteration: 61400. Train_MSE: 0.00034288185997866094. Test_MSE: 0.00018082714658218104\n",
      "Epoch: 662 Iteration: 61600. Train_MSE: 0.00040412918315269053. Test_MSE: 0.0001807957332425711\n",
      "Epoch: 664 Iteration: 61800. Train_MSE: 0.00018019223352894187. Test_MSE: 0.00018028307604586066\n",
      "Epoch: 666 Iteration: 62000. Train_MSE: 0.00025078977341763675. Test_MSE: 0.00017977485564626925\n",
      "Epoch: 668 Iteration: 62200. Train_MSE: 0.0005327580729499459. Test_MSE: 0.0001799594623537904\n",
      "Epoch: 670 Iteration: 62400. Train_MSE: 0.00018850882770493627. Test_MSE: 0.00017969631301432355\n",
      "Epoch: 673 Iteration: 62600. Train_MSE: 0.00021789703168906271. Test_MSE: 0.00017900148282362944\n",
      "Epoch: 675 Iteration: 62800. Train_MSE: 0.0003661743539851159. Test_MSE: 0.00017896649663416106\n",
      "Epoch: 677 Iteration: 63000. Train_MSE: 0.00032286776695400476. Test_MSE: 0.00017882768672788335\n",
      "Epoch: 679 Iteration: 63200. Train_MSE: 0.00018936002743430436. Test_MSE: 0.00017819236944427842\n",
      "Epoch: 681 Iteration: 63400. Train_MSE: 0.0004073228337801993. Test_MSE: 0.00017815715484761167\n",
      "Epoch: 683 Iteration: 63600. Train_MSE: 0.0002903631830122322. Test_MSE: 0.00017798461307414718\n",
      "Epoch: 686 Iteration: 63800. Train_MSE: 0.0001902719377540052. Test_MSE: 0.00017764391565162248\n",
      "Epoch: 688 Iteration: 64000. Train_MSE: 0.0011652261018753052. Test_MSE: 0.0001770130165907797\n",
      "Epoch: 690 Iteration: 64200. Train_MSE: 0.0020922452677041292. Test_MSE: 0.00017703729257139276\n",
      "Epoch: 692 Iteration: 64400. Train_MSE: 0.00017019471852108836. Test_MSE: 0.00017675570166881117\n",
      "Epoch: 694 Iteration: 64600. Train_MSE: 0.00038996338844299316. Test_MSE: 0.00017609928456493177\n",
      "Epoch: 696 Iteration: 64800. Train_MSE: 0.000352561502950266. Test_MSE: 0.00017629159538422285\n",
      "Epoch: 698 Iteration: 65000. Train_MSE: 0.00013618296361528337. Test_MSE: 0.000175987366863459\n",
      "Epoch: 701 Iteration: 65200. Train_MSE: 0.0001537469361210242. Test_MSE: 0.00017541766186965753\n",
      "Epoch: 703 Iteration: 65400. Train_MSE: 0.0004788979422301054. Test_MSE: 0.0001752183487791596\n",
      "Epoch: 705 Iteration: 65600. Train_MSE: 0.0003555331495590508. Test_MSE: 0.0001751162379868036\n",
      "Epoch: 707 Iteration: 65800. Train_MSE: 0.0019596365746110678. Test_MSE: 0.00017469665313503755\n",
      "Epoch: 709 Iteration: 66000. Train_MSE: 0.0030036824755370617. Test_MSE: 0.00017421068548973556\n",
      "Epoch: 711 Iteration: 66200. Train_MSE: 0.00045684256474487484. Test_MSE: 0.00017438124425821734\n",
      "Epoch: 713 Iteration: 66400. Train_MSE: 0.00022236416407395154. Test_MSE: 0.00017417977644242907\n",
      "Epoch: 716 Iteration: 66600. Train_MSE: 0.00020919216331094503. Test_MSE: 0.00017344654448453255\n",
      "Epoch: 718 Iteration: 66800. Train_MSE: 0.0002992639201693237. Test_MSE: 0.00017346045859177158\n",
      "Epoch: 720 Iteration: 67000. Train_MSE: 0.00015493972750846297. Test_MSE: 0.0001732983913275499\n",
      "Epoch: 722 Iteration: 67200. Train_MSE: 0.0001746482157614082. Test_MSE: 0.0001726722532822029\n",
      "Epoch: 724 Iteration: 67400. Train_MSE: 0.0004681843565776944. Test_MSE: 0.0001727393104304395\n",
      "Epoch: 726 Iteration: 67600. Train_MSE: 0.00039360366645269096. Test_MSE: 0.00017257122848055465\n",
      "Epoch: 729 Iteration: 67800. Train_MSE: 0.00015977455768734217. Test_MSE: 0.00017219819346914943\n",
      "Epoch: 731 Iteration: 68000. Train_MSE: 0.0014948936877772212. Test_MSE: 0.00017168500300301318\n",
      "Epoch: 733 Iteration: 68200. Train_MSE: 0.008684471249580383. Test_MSE: 0.00017172038189243653\n",
      "Epoch: 735 Iteration: 68400. Train_MSE: 0.00019613801850937307. Test_MSE: 0.00017140570103205366\n",
      "Epoch: 737 Iteration: 68600. Train_MSE: 0.0002392571186646819. Test_MSE: 0.0001707842356134317\n",
      "Epoch: 739 Iteration: 68800. Train_MSE: 0.00028498214669525623. Test_MSE: 0.0001709664801393582\n",
      "Epoch: 741 Iteration: 69000. Train_MSE: 0.00021046608162578195. Test_MSE: 0.00017068723589677888\n",
      "Epoch: 744 Iteration: 69200. Train_MSE: 0.00032606013701297343. Test_MSE: 0.00017014733553494013\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 746 Iteration: 69400. Train_MSE: 0.00030243073706515133. Test_MSE: 0.00016998710927312454\n",
      "Epoch: 748 Iteration: 69600. Train_MSE: 0.0112951984629035. Test_MSE: 0.00017000157060653606\n",
      "Epoch: 750 Iteration: 69800. Train_MSE: 0.00023195697576738894. Test_MSE: 0.00016947778288924762\n",
      "Epoch: 752 Iteration: 70000. Train_MSE: 0.003590972162783146. Test_MSE: 0.00016920448081417298\n",
      "Epoch: 754 Iteration: 70200. Train_MSE: 0.011074689216911793. Test_MSE: 0.00016936093203281886\n",
      "Epoch: 756 Iteration: 70400. Train_MSE: 0.00025547502445988357. Test_MSE: 0.00016900957414184954\n",
      "Epoch: 759 Iteration: 70600. Train_MSE: 0.00021062401356175542. Test_MSE: 0.00016830206398575187\n",
      "Epoch: 761 Iteration: 70800. Train_MSE: 0.00032152843778021634. Test_MSE: 0.00016834779122544026\n",
      "Epoch: 763 Iteration: 71000. Train_MSE: 0.00408405065536499. Test_MSE: 0.0001682657547726942\n",
      "Epoch: 765 Iteration: 71200. Train_MSE: 0.0003626347752287984. Test_MSE: 0.00016756546989824903\n",
      "Epoch: 767 Iteration: 71400. Train_MSE: 0.0003368993930052966. Test_MSE: 0.00016771076757489989\n",
      "Epoch: 769 Iteration: 71600. Train_MSE: 0.0019633660558611155. Test_MSE: 0.0001675290380029386\n",
      "Epoch: 772 Iteration: 71800. Train_MSE: 0.00014803383965045214. Test_MSE: 0.00016709975911957754\n",
      "Epoch: 774 Iteration: 72000. Train_MSE: 0.0007861296180635691. Test_MSE: 0.00016668785727905425\n",
      "Epoch: 776 Iteration: 72200. Train_MSE: 0.0002455778303556144. Test_MSE: 0.00016666152577263503\n",
      "Epoch: 778 Iteration: 72400. Train_MSE: 0.00013606126594822854. Test_MSE: 0.00016636162896567662\n",
      "Epoch: 780 Iteration: 72600. Train_MSE: 0.00019605678971856833. Test_MSE: 0.00016583171217828235\n",
      "Epoch: 782 Iteration: 72800. Train_MSE: 0.0003259860095568001. Test_MSE: 0.00016602108108521035\n",
      "Epoch: 784 Iteration: 73000. Train_MSE: 0.0004249077755957842. Test_MSE: 0.000165853237806499\n",
      "Epoch: 787 Iteration: 73200. Train_MSE: 0.0017031326424330473. Test_MSE: 0.00016528985070283987\n",
      "Epoch: 789 Iteration: 73400. Train_MSE: 0.00027794940979219973. Test_MSE: 0.0001651055689939657\n",
      "Epoch: 791 Iteration: 73600. Train_MSE: 0.00017028868023771793. Test_MSE: 0.00016510584090586949\n",
      "Epoch: 793 Iteration: 73800. Train_MSE: 0.00017148713232018054. Test_MSE: 0.00016459857457759987\n",
      "Epoch: 795 Iteration: 74000. Train_MSE: 0.0018918588757514954. Test_MSE: 0.00016446846859104802\n",
      "Epoch: 797 Iteration: 74200. Train_MSE: 0.00017843670502770692. Test_MSE: 0.00016448584333492622\n",
      "Epoch: 799 Iteration: 74400. Train_MSE: 0.00019696496019605547. Test_MSE: 0.00016418339622456125\n",
      "Epoch: 802 Iteration: 74600. Train_MSE: 0.00027691686409525573. Test_MSE: 0.00016350374667722258\n",
      "Epoch: 804 Iteration: 74800. Train_MSE: 0.000539864762686193. Test_MSE: 0.00016357153396365608\n",
      "Epoch: 806 Iteration: 75000. Train_MSE: 0.00011625000479398295. Test_MSE: 0.0001634717980571778\n",
      "Epoch: 808 Iteration: 75200. Train_MSE: 0.00032494630431756377. Test_MSE: 0.00016285011385013646\n",
      "Epoch: 810 Iteration: 75400. Train_MSE: 0.00021750891755800694. Test_MSE: 0.00016302159039673038\n",
      "Epoch: 812 Iteration: 75600. Train_MSE: 0.0001581071992404759. Test_MSE: 0.00016281214189737346\n",
      "Epoch: 815 Iteration: 75800. Train_MSE: 0.00015576805162709206. Test_MSE: 0.0001623990120711156\n",
      "Epoch: 817 Iteration: 76000. Train_MSE: 0.0006178051116876304. Test_MSE: 0.00016203922700530762\n",
      "Epoch: 819 Iteration: 76200. Train_MSE: 0.00021976818970870227. Test_MSE: 0.0001620115093165122\n",
      "Epoch: 821 Iteration: 76400. Train_MSE: 0.00013637490337714553. Test_MSE: 0.00016170238561220417\n",
      "Epoch: 823 Iteration: 76600. Train_MSE: 0.00014227117935661227. Test_MSE: 0.00016127173673171743\n",
      "Epoch: 825 Iteration: 76800. Train_MSE: 0.00044680587598122656. Test_MSE: 0.00016145304008376897\n",
      "Epoch: 827 Iteration: 77000. Train_MSE: 0.000146925201988779. Test_MSE: 0.0001612852326111234\n",
      "Epoch: 830 Iteration: 77200. Train_MSE: 0.00020954704086761922. Test_MSE: 0.0001607104270963158\n",
      "Epoch: 832 Iteration: 77400. Train_MSE: 0.00023415066243615001. Test_MSE: 0.0001605747606168012\n",
      "Epoch: 834 Iteration: 77600. Train_MSE: 0.0002004296111408621. Test_MSE: 0.00016054760488571173\n",
      "Epoch: 836 Iteration: 77800. Train_MSE: 0.00014329914120025933. Test_MSE: 0.00016012896139105032\n",
      "Epoch: 838 Iteration: 78000. Train_MSE: 0.0007895316230133176. Test_MSE: 0.00016005731266214166\n",
      "Epoch: 840 Iteration: 78200. Train_MSE: 0.00018938635184895247. Test_MSE: 0.00015998775578924577\n",
      "Epoch: 843 Iteration: 78400. Train_MSE: 0.0002564211608842015. Test_MSE: 0.00015977747437480563\n",
      "Epoch: 845 Iteration: 78600. Train_MSE: 0.0001745613117236644. Test_MSE: 0.00015908338604582562\n",
      "Epoch: 847 Iteration: 78800. Train_MSE: 0.0004257599648553878. Test_MSE: 0.0001591412957517885\n",
      "Epoch: 849 Iteration: 79000. Train_MSE: 0.0001647864846745506. Test_MSE: 0.00015903130574044245\n",
      "Epoch: 851 Iteration: 79200. Train_MSE: 0.0003655672771856189. Test_MSE: 0.0001584940077107597\n",
      "Epoch: 853 Iteration: 79400. Train_MSE: 0.0002931819180957973. Test_MSE: 0.00015870140995485606\n",
      "Epoch: 855 Iteration: 79600. Train_MSE: 0.00017606810433790088. Test_MSE: 0.0001584704176580371\n",
      "Epoch: 858 Iteration: 79800. Train_MSE: 0.0001935594918904826. Test_MSE: 0.00015804294170449585\n",
      "Epoch: 860 Iteration: 80000. Train_MSE: 0.00032657745759934187. Test_MSE: 0.00015775469248346588\n",
      "Epoch: 862 Iteration: 80200. Train_MSE: 0.0003916638088412583. Test_MSE: 0.00015774869651307167\n",
      "Epoch: 864 Iteration: 80400. Train_MSE: 0.0001253272930625826. Test_MSE: 0.00015740089841009722\n",
      "Epoch: 866 Iteration: 80600. Train_MSE: 0.00024337219656445086. Test_MSE: 0.00015702732564443436\n",
      "Epoch: 868 Iteration: 80800. Train_MSE: 0.0005166747141629457. Test_MSE: 0.000157226132388136\n",
      "Epoch: 870 Iteration: 81000. Train_MSE: 0.00014096350059844553. Test_MSE: 0.000157072249976202\n",
      "Epoch: 873 Iteration: 81200. Train_MSE: 0.0002055259537883103. Test_MSE: 0.0001565174716183567\n",
      "Epoch: 875 Iteration: 81400. Train_MSE: 0.00032091260072775185. Test_MSE: 0.00015647837352275384\n",
      "Epoch: 877 Iteration: 81600. Train_MSE: 0.0002801505324896425. Test_MSE: 0.0001564193839302782\n",
      "Epoch: 879 Iteration: 81800. Train_MSE: 0.00014810054562985897. Test_MSE: 0.0001559564416413498\n",
      "Epoch: 881 Iteration: 82000. Train_MSE: 0.00038983512786217034. Test_MSE: 0.00015599835492745442\n",
      "Epoch: 883 Iteration: 82200. Train_MSE: 0.00026194556267000735. Test_MSE: 0.0001558808144009559\n",
      "Epoch: 886 Iteration: 82400. Train_MSE: 0.0001502663508290425. Test_MSE: 0.0001556438309016413\n",
      "Epoch: 888 Iteration: 82600. Train_MSE: 0.001114090089686215. Test_MSE: 0.0001551195501213309\n",
      "Epoch: 890 Iteration: 82800. Train_MSE: 0.002077720360830426. Test_MSE: 0.00015514088565944472\n",
      "Epoch: 892 Iteration: 83000. Train_MSE: 0.00013055952149443328. Test_MSE: 0.00015499187474127222\n",
      "Epoch: 894 Iteration: 83200. Train_MSE: 0.0003682039678096771. Test_MSE: 0.00015450737326617835\n",
      "Epoch: 896 Iteration: 83400. Train_MSE: 0.0003343287971802056. Test_MSE: 0.0001547203047885776\n",
      "Epoch: 898 Iteration: 83600. Train_MSE: 0.00011776180326705799. Test_MSE: 0.0001544902140363989\n",
      "Epoch: 901 Iteration: 83800. Train_MSE: 0.0001189679533126764. Test_MSE: 0.00015406417430380493\n",
      "Epoch: 903 Iteration: 84000. Train_MSE: 0.00044391912524588406. Test_MSE: 0.00015388493980590567\n",
      "Epoch: 905 Iteration: 84200. Train_MSE: 0.0003408292250242084. Test_MSE: 0.0001538184089791526\n",
      "Epoch: 907 Iteration: 84400. Train_MSE: 0.00188720494043082. Test_MSE: 0.00015355265776302276\n",
      "Epoch: 909 Iteration: 84600. Train_MSE: 0.003013778943568468. Test_MSE: 0.00015318493002301203\n",
      "Epoch: 911 Iteration: 84800. Train_MSE: 0.0004344870103523135. Test_MSE: 0.00015336751115252936\n",
      "Epoch: 913 Iteration: 85000. Train_MSE: 0.00016992651217151433. Test_MSE: 0.0001532645289069707\n",
      "Epoch: 916 Iteration: 85200. Train_MSE: 0.00019776307453867048. Test_MSE: 0.00015266189403929478\n",
      "Epoch: 918 Iteration: 85400. Train_MSE: 0.0002773302257992327. Test_MSE: 0.00015266653834210557\n",
      "Epoch: 920 Iteration: 85600. Train_MSE: 0.0001386674994137138. Test_MSE: 0.00015258635357833368\n",
      "Epoch: 922 Iteration: 85800. Train_MSE: 0.0001396291481796652. Test_MSE: 0.00015213774675420477\n",
      "Epoch: 924 Iteration: 86000. Train_MSE: 0.00044322735629975796. Test_MSE: 0.0001522581006320255\n",
      "Epoch: 926 Iteration: 86200. Train_MSE: 0.00035727847716771066. Test_MSE: 0.00015213496882013165\n",
      "Epoch: 929 Iteration: 86400. Train_MSE: 0.0001240665587829426. Test_MSE: 0.00015187217889823897\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 931 Iteration: 86600. Train_MSE: 0.0014606601325795054. Test_MSE: 0.00015144055277460596\n",
      "Epoch: 933 Iteration: 86800. Train_MSE: 0.0086682653054595. Test_MSE: 0.00015148401190796215\n",
      "Epoch: 935 Iteration: 87000. Train_MSE: 0.00015442231961060315. Test_MSE: 0.00015128614746655886\n",
      "Epoch: 937 Iteration: 87200. Train_MSE: 0.00022715610975865275. Test_MSE: 0.00015082655576142238\n",
      "Epoch: 939 Iteration: 87400. Train_MSE: 0.0002638073929119855. Test_MSE: 0.00015102246482342853\n",
      "Epoch: 941 Iteration: 87600. Train_MSE: 0.0001949236320797354. Test_MSE: 0.00015082034594439446\n",
      "Epoch: 944 Iteration: 87800. Train_MSE: 0.00029768754029646516. Test_MSE: 0.0001504243501019305\n",
      "Epoch: 946 Iteration: 88000. Train_MSE: 0.00028017471777275205. Test_MSE: 0.00015027627155031974\n",
      "Epoch: 948 Iteration: 88200. Train_MSE: 0.011318212375044823. Test_MSE: 0.00015034025620724604\n",
      "Epoch: 950 Iteration: 88400. Train_MSE: 0.0001922106894198805. Test_MSE: 0.00014995306360894947\n",
      "Epoch: 952 Iteration: 88600. Train_MSE: 0.003555474802851677. Test_MSE: 0.0001497893998337435\n",
      "Epoch: 954 Iteration: 88800. Train_MSE: 0.011056162416934967. Test_MSE: 0.00014996912132309037\n",
      "Epoch: 956 Iteration: 89000. Train_MSE: 0.00020164874149486423. Test_MSE: 0.00014969757606228513\n",
      "Epoch: 959 Iteration: 89200. Train_MSE: 0.000185378739843145. Test_MSE: 0.00014911200448961592\n",
      "Epoch: 961 Iteration: 89400. Train_MSE: 0.0002909506729338318. Test_MSE: 0.0001491515441641506\n",
      "Epoch: 963 Iteration: 89600. Train_MSE: 0.004066746681928635. Test_MSE: 0.00014915075015201594\n",
      "Epoch: 965 Iteration: 89800. Train_MSE: 0.00033785004052333534. Test_MSE: 0.00014862300098899677\n",
      "Epoch: 967 Iteration: 90000. Train_MSE: 0.0003092059923801571. Test_MSE: 0.0001488096501560687\n",
      "Epoch: 969 Iteration: 90200. Train_MSE: 0.0019523869268596172. Test_MSE: 0.00014867270364541193\n",
      "Epoch: 972 Iteration: 90400. Train_MSE: 0.00011487324809422716. Test_MSE: 0.00014835515369996166\n",
      "Epoch: 974 Iteration: 90600. Train_MSE: 0.0007757403654977679. Test_MSE: 0.0001480130935650912\n",
      "Epoch: 976 Iteration: 90800. Train_MSE: 0.00022456380247604102. Test_MSE: 0.0001479939941106889\n",
      "Epoch: 978 Iteration: 91000. Train_MSE: 0.00010250458581140265. Test_MSE: 0.00014780800643001332\n",
      "Epoch: 980 Iteration: 91200. Train_MSE: 0.00018250690482091159. Test_MSE: 0.00014741866796083775\n",
      "Epoch: 982 Iteration: 91400. Train_MSE: 0.0002960415731649846. Test_MSE: 0.00014761861214210688\n",
      "Epoch: 984 Iteration: 91600. Train_MSE: 0.0003603565273806453. Test_MSE: 0.00014751874738527302\n",
      "Epoch: 987 Iteration: 91800. Train_MSE: 0.0016673882491886616. Test_MSE: 0.00014709071757323665\n",
      "Epoch: 989 Iteration: 92000. Train_MSE: 0.00025391721283085644. Test_MSE: 0.00014691514080941817\n",
      "Epoch: 991 Iteration: 92200. Train_MSE: 0.00014966094749979675. Test_MSE: 0.00014696486277082348\n",
      "Epoch: 993 Iteration: 92400. Train_MSE: 0.0001394931023241952. Test_MSE: 0.0001465824098016543\n",
      "Epoch: 995 Iteration: 92600. Train_MSE: 0.001854764181189239. Test_MSE: 0.00014654645001901644\n",
      "Epoch: 997 Iteration: 92800. Train_MSE: 0.00016129827417898923. Test_MSE: 0.00014659238973107258\n",
      "Epoch: 999 Iteration: 93000. Train_MSE: 0.00015641783829778433. Test_MSE: 0.00014636515947178566\n"
     ]
    }
   ],
   "source": [
    "# Training the model\n",
    "seq_dim = 1\n",
    "\n",
    "n_iter =0\n",
    "num_samples = len(X_train)\n",
    "test_samples = len(X_test)\n",
    "batch_size = 100\n",
    "num_epochs = 1000\n",
    "feat_dim = X_train.shape[1]\n",
    "\n",
    "X_train = X_train.type(torch.FloatTensor)\n",
    "y_train = y_train.type(torch.FloatTensor)\n",
    "X_test = X_test.type(torch.FloatTensor)\n",
    "y_test = y_test.type(torch.FloatTensor)\n",
    "\n",
    "for epoch in range(num_epochs):\n",
    "    for i in range(0, int(num_samples/batch_size -1)):\n",
    "        \n",
    "        \n",
    "        features = Variable(X_train[i*batch_size:(i+1)*batch_size, :]).view(-1, seq_dim, feat_dim)\n",
    "        Kt_value = Variable(y_train[i*batch_size:(i+1)*batch_size])\n",
    "        \n",
    "        #print(\"Kt_value={}\".format(Kt_value))\n",
    "        \n",
    "        optimizer.zero_grad()\n",
    "        \n",
    "        outputs = model(features)\n",
    "        #print(\"outputs ={}\".format(outputs))\n",
    "        \n",
    "        loss = criterion(outputs, Kt_value)\n",
    "        \n",
    "        train_loss.append(loss.data[0])\n",
    "        train_iter.append(n_iter)\n",
    "\n",
    "        #print(\"loss = {}\".format(loss))\n",
    "        loss.backward()\n",
    "        \n",
    "        optimizer.step()\n",
    "        \n",
    "        n_iter += 1  \n",
    "            \n",
    "        if n_iter%200 == 0:\n",
    "            for i in range(0,int(test_samples/batch_size -1)):\n",
    "                features = Variable(X_test[i*batch_size:(i+1)*batch_size, :]).view(-1, seq_dim, feat_dim)\n",
    "                Kt_test = Variable(y_test[i*batch_size:(i+1)*batch_size])\n",
    "                \n",
    "                outputs = model(features)\n",
    "                \n",
    "                mse = np.sqrt(np.mean((Kt_test.data.numpy() - outputs.data.numpy().squeeze())**2)/num_samples)\n",
    "                \n",
    "                test_iter.append(n_iter)\n",
    "                test_loss.append(mse)\n",
    "                \n",
    "            print('Epoch: {} Iteration: {}. Train_MSE: {}. Test_MSE: {}'.format(epoch, n_iter, loss.data[0], mse))\n",
    "               "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "93000\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5,1,'train_loss')"
      ]
     },
     "execution_count": 84,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEFCAYAAAAPCDf9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAGdtJREFUeJzt3X9wXOV97/H3SmskG9ZC4HVEuISU\nof3iS2LnJo5jAwbChBJoIO406ZC2Q7GL65vQywXaTp2SQHKvCbSpS8clNCkzLtDGmaZQ3xKnGGid\nUCPjmrT0Ygf4ctU0JA0YFkeWZUtaaX/cP87KPphd6Uhea9f7fF4zHu8+z3POeZ5n7M85elZ7Tqpc\nLiMiImFoa3QHRERk5ij0RUQCotAXEQmIQl9EJCAKfRGRgCj0RUQCotAXEQmIQl9akpk9YWbzptB+\nsZk9fAzHu97Mtkx3e5GZkm50B0SOk8un0tjdvwd84jj1RaRpKPSl5ZjZX1RefsfM/ivwMLAQ+H1g\nrPL3ScB84EF3/7yZXQrc6+7vMbMHgAPAe4GzgOeB69z9YMLj/xfgz4B3A6nKMb5sZmngT4ELK/34\nAbASGKlWnvR4IlOh5R1pOe6+svLyw8CPgT3uvgD4P8BvA7/u7ouBpcBnaywDfQD4KLCAKLw/OYUu\nfB34jru/lyjIf83MrgWWAZcCi9z9A0ThvnCCcpG6U+hLCLYDuHsZuBr4gJndAfwx0ZX4yVW22eru\neXcfA3YDpyU5kJmdTBT0X6kccwB4ALiysp8i8M9m9r+BR9x9xwTlInWn0JcQHITDgfwc8H7gX4Hf\nJVpOSVXZZjj2ulyjTTVtVdq2AbPcfT+wCPgdopD/azP7TK3yhMcTmRKFvrSqIjDrqLKfBeYCn3P3\nbxEtqXQA7fU6qLsPAjuBGwHMrAu4DnjSzD4G/COww92/ADwEfLBWeb36JBKnD3KlVf0N8BRwSqzs\neWAL8JKZ5YmWVV4AzgXydTz2rwJfMbOVRB8YbyJa4mkjWubZY2YHgX5gNdHnDtXKReoupfvpi4iE\nQ1f6IgmZ2XYgU6N6eWVpR6Sp6UpfRCQg+iBXRCQgTb28k8sNTvvHkO7uOfT3D9WzOyckzYPmYJzm\nIZw5yGYzNX/FuGWv9NPpuv0W3glN86A5GKd50BxAC4e+iIi83aTLO2bWBtxH9I3BPHCDu/fF6lcD\na4ACsM7dt8TqbgZ63H3tUfv8c+CnR5eLiMjxlWRNfwXQ6e7LzGwpsB74OICZ9QA3AYuBTuBpM3uS\n6CeI+4EPAY/Ed2Zma4juXvhUvQYhIiLJJFneuQjYCuDuO4kCftwSoLdyY6oBoI/o7oCdRF8lvzO+\nIzNbRnRnw68de9dFRGSqklzpzwUGYu+LZpZ290KVukGgy937gSfM7PrxCjM7A/gC8IvALyfpXHf3\nnGP64CWbrfU9mrBoHjQH4zQPmoMkoX+At34Lsa0S+NXqMsD+Gvv5JDAP+HugB5hjZi+5+wO1Dnws\nv1qVzWbI5fQFSc2D5mCc5iGcOZjoxJYk9HuJ7kH+zcqa/u5Y3S7gTjPrJLpb4QJgT7WduPsGYANE\nzxMFzpso8EVEpP6ShP5m4HIz20F0n/CVZnYr0Ofuj5rZBqKHVLQBt7n7yPHrbjLFUom/fOxFFv1M\nN2ecXu35GCIiYWrqe+9M9xu5O1/Yy58/+gIdJ7XzZ7deUu9unVBC+XF2IpqDiOYhnDkI7hu5w/ki\nAPnRYoN7IiLSXFoy9EVEpDqFvohIQBT6IiIBUeiLiAREoS8iEhCFvohIQBT6IiIBUeiLiASkJUO/\n5lfRREQC15KhLyIi1bV86I+O6VYMIiLjWj70X/hhf6O7ICLSNFoy9F98RUEvIlJNS4b+sy+9cfj1\nv786MEFLEZGwtGTox337mVca3QURkabR8qEvIiJHKPRFRAISROjvP5hvdBdERJpCEKG/4eHnG90F\nEZGmEETo/3DvIM38AHgRkZmSnqyBmbUB9wGLgDxwg7v3xepXA2uAArDO3bfE6m4Getx9beX9p4Cb\ngSLwPPAZdy/Vbzi13b/lBX7z6vNn4lAiIk0ryZX+CqDT3ZcBa4H14xVm1gPcBFwIXAHcZWYdZjbb\nzP4KuDHWdjawDviwu18AdAEfq9tIJrHz+6+T2z88U4cTEWlKSUL/ImArgLvvBBbH6pYAve6ed/cB\noA9YCHQCDwF3xtrmgQvcfajyPg2MHFv3p+b3vvoMQyOFmTykiEhTmXR5B5gLxL/WWjSztLsXqtQN\nAl3u3g88YWbXj1dUlnFeBzCz/wGcAjw50YG7u+eQTrcnGUdiv/Un/8RDd1xB99zOuu63mWWzmUZ3\noeE0BxHNg+YgSegfAOKz1FYJ/Gp1GWB/rR1VPh/4Q+DngF9y9wk/Xe3vH5qoetqu++Lj3PSJhbzv\n3HnHZf/NJJvNkMsNNrobDaU5iGgewpmDiU5sSZZ3eoGrAMxsKbA7VrcLWG5mnWbWBSwA9kywr68R\nLf2siC3zNMSGh59n1d3b9Fs9IhKUJKG/GRgxsx3APcAtZnarmV3j7nuBDcB2YBtwm7tXXac3s/cD\nvwG8F9hmZt81s1+syyiOwW/8wXd4YtePGt0NEZEZkWrmK91cbnBanVt197ZpHe+XLjmHX1j27mlt\n26xC+XF2IpqDiOYhnDnIZjM1nxqbZE0/GI889QMeeeoHANzzWxfSdUpHg3skIlJfCv0abrm39/Dr\nL65awlnzT2lgb0RE6kOhn8AdG3e95f3vfuq/cd67TiWVqvkTlIhIU1LoT8OXv/Hc28ouWngGH13y\nLs44fY5OBiIz4OjPI8uxF2XKjFdHf0fvR/IF8qNFSuXy4fJSOdpXufJ3qbJdqVQ+vJ9SuQyVv8ux\n9kf2E70eb3d4G47ss3z4/dH9O3LM8b6fMnsW557ZdVyyRKFfJ08//xpPP/9azfpzz+xiyYL5LDi7\nm+yps5mVbtPJIVDxwCgWyxRLJcaKZcYKRcYKJUbHSuTHioyMFhkZLTAyWmRopMBQvsDQyBhDIwUO\njVRe56PXh4bHKJaa95cyZOo+eN58Pr3iPXXfr0J/hvT9ZIC+n0zveb3v6J7NWfNP4czsKbzjtNmc\nlulk7sknMacjTcesdtrbU7S3pYI5iZTLZcpEV2LFYpnRQpHRsRLDowVG8kUOjYxxcHiMwaExDgyN\nki+UeH3fIQYOjTJwcJSDw2ONHoLIpJ596Q0+fRz2q9A/AbzeP8zr/cN8z3ON7oqInOCCuJ++iIhE\nFPoiIgFR6IuIBEShLyISEIW+iEhAFPoiIgFR6IuIBEShLyISEIW+iEhAFPoiIgFR6IuIBEShLyIS\nEIW+iEhAFPoiIgGZ9NbKZtYG3AcsAvLADe7eF6tfDawBCsA6d98Sq7sZ6HH3tZX3VwO3V9pudPf7\n6zgWERGZRJL76a8AOt19mZktBdYDHwcwsx7gJmAx0Ak8bWZPEv0EcT/wIeCRSttZwD3AB4FDQK+Z\nfcvd99Z3SCIiUkuS5Z2LgK0A7r6TKODHLQF63T3v7gNAH7CQ6ATwEHBnrO0CoM/d+919FHgaWH7s\nQxARkaSSXOnPBeLP+SuaWdrdC1XqBoEud+8HnjCz6yfYzyDQNdGBu7vnkE63J+iiiEjryWYzdd9n\nktA/AMSP3FYJ/Gp1GWB/wv1M1BaA/v6hBN0TEWlNudzgtLab6GSRJPR7gauBb1bW9HfH6nYBd5pZ\nJ9BBtISzp8Z+XgR+1sxOAw4CFwN/lOD4IiJSJ0lCfzNwuZntAFLASjO7lWh9/lEz2wBsJ/p84DZ3\nH6m2E3cfq2z3eKXtRnf/SV1GISIiiaTK5XKj+1BTLjc4rc6tuntbvbsiIjLjNq69bFrbZbOZVK06\nfTlLRCQgCn0RkYAo9EVEAqLQFxEJiEJfRCQgCn0RkYAo9EVEAqLQFxEJiEJfRCQgCn0RkYAo9EVE\nAqLQFxEJiEJfRCQgCn0RkYAo9EVEAqLQFxEJiEJfRCQgCn0RkYAo9EVEAqLQFxEJiEJfRCQg6cka\nmFkbcB+wCMgDN7h7X6x+NbAGKADr3H2Lmc0DNgGzgVeBle4+ZGa/A3wKKAFfcvfN9R6QiIjUluRK\nfwXQ6e7LgLXA+vEKM+sBbgIuBK4A7jKzDuB2YJO7LweeA9aY2amVtsuAnwf+pJ4DERGRySUJ/YuA\nrQDuvhNYHKtbAvS6e97dB4A+YGF8G+Ax4CPAIeAV4OTKn1I9BiAiIslNurwDzAUGYu+LZpZ290KV\nukGg66jy8TKAHwMvAO3AXZMduLt7Dul0e4Iuioi0nmw2U/d9Jgn9A0D8yG2VwK9WlwH2x8qHY2VX\nAmcAP1Np+7iZ9br7rloH7u8fSjIGEZGWlMsNTmu7iU4WSZZ3eoGrAMxsKbA7VrcLWG5mnWbWBSwA\n9sS3IQr77UA/0Ukg7+4jRCeCU6c0EhEROSZJQn8zMGJmO4B7gFvM7FYzu8bd9wIbiEJ9G3BbJdDX\nAdeaWS/RB7f3uvt24Flgp5k9A7wMPFn/IYmISC2pcrnc6D7UlMsNTqtzq+7eVu+uiIjMuI1rL5vW\ndtlsJlWrTl/OEhEJiEJfRCQgCn0RkYAo9EVEAqLQFxEJiEJfRCQgCn0RkYAo9EVEAqLQFxEJiEJf\nRCQgCn0RkYAo9EVEAqLQFxEJiEJfRCQgCn0RkYAo9EVEAqLQFxEJiEJfRCQgCn0RkYAo9EVEAqLQ\nFxEJSHqyBmbWBtwHLALywA3u3herXw2sAQrAOnffYmbzgE3AbOBVYKW7D5nZlcAdlU3/FbjR3cv1\nHJCIiNSW5Ep/BdDp7suAtcD68Qoz6wFuAi4ErgDuMrMO4HZgk7svB54D1phZBvgy8DF3Xwr8EJhX\nx7GIiMgkkoT+RcBWAHffCSyO1S0Bet097+4DQB+wML4N8BjwEeACYDew3sy2A6+7e64uoxARkUQm\nXd4B5gIDsfdFM0u7e6FK3SDQdVT5eNk84MPA+4CDwHYze8bdX6514O7uOaTT7UnHIiLSUrLZTN33\nmST0DwDxI7dVAr9aXQbYHysfjpXtA551970AZvZPRCeAmqHf3z+UbBQiIi0olxuc1nYTnSySLO/0\nAlcBmNlSoiWacbuA5WbWaWZdwAJgT3wb4EpgO/AvwHvMbJ6ZpYGlwAtTG4qIiByLJKG/GRgxsx3A\nPcAtZnarmV1TuWrfQBTq24Db3H0EWAdca2a9wDLg3sr6/WeBx4F/Bv7W3ffUf0giIlJLqlxu3t+Y\nzOUGp9W5VXdvq3dXRERm3Ma1l01ru2w2k6pVpy9niYgERKEvIhIQhb6ISEAU+iIiAVHoi4gERKEv\nIhIQhb6ISEAU+iIiAVHoi4gERKEvIhIQhb6ISEAU+iIiAVHoi4gERKEvIhIQhb6ISEAU+iIiAVHo\ni4gERKEvIhIQhb6ISEAU+iIiAVHoi4gEJD1ZAzNrA+4DFgF54AZ374vVrwbWAAVgnbtvMbN5wCZg\nNvAqsNLdh2L7+zbwd+7+1TqPR0REJpDkSn8F0Onuy4C1wPrxCjPrAW4CLgSuAO4ysw7gdmCTuy8H\nniM6KYxbB5xWn+6LiMhUJAn9i4CtAO6+E1gcq1sC9Lp73t0HgD5gYXwb4DHgIwBm9gmgVCkTEZEZ\nNunyDjAXGIi9L5pZ2t0LVeoGga6jygeBLjN7D/ArwCeIfhKYVHf3HNLp9iRNRURaTjabqfs+k4T+\nASB+5LZK4FerywD7Y+XDsbLrgDOBbcC7gVEz+6G7b6WG/v6hZKMQEWlBudzgtLab6GSRJPR7gauB\nb5rZUmB3rG4XcKeZdQIdwAJgT2Wbq4AHgCuB7e7+B+MbmdkXgL0TBb6IiNRfktDfDFxuZjuAFLDS\nzG4F+tz9UTPbAGwn+nzgNncfMbN1wIOV3+x5k2hZR0REGixVLpcb3YeacrnBaXVu1d3b6t0VEZEZ\nt3HtZdPaLpvNpGrV6ctZIiIBUeiLiAREoS8iEhCFvohIQBT6IiIBUeiLiAREoS8iEhCFvohIQBT6\nIiIBUeiLiAREoS8iEhCFvohIQBT6IiIBUeiLiAREoS8iEhCFvohIQBT6IiIBUeiLiAREoS8iEhCF\nvohIQBT6IiIBUeiLiAQkPVkDM2sD7gMWAXngBnfvi9WvBtYABWCdu28xs3nAJmA28Cqw0t2HzOwW\n4NrKpn/v7l+s62hERGRCSa70VwCd7r4MWAusH68wsx7gJuBC4ArgLjPrAG4HNrn7cuA5YI2ZnQP8\nKnABsAz4eTNbWM/BiIjIxCa90gcuArYCuPtOM1scq1sC9Lp7HsibWR+wsLLNlyptHqu8vhf4qLsX\nAcxsFjAy0YG7u+eQTrdPYTgiIq0jm83UfZ9JQn8uMBB7XzSztLsXqtQNAl1HlQ8CXe4+BrxpZing\ny8Bz7v7yRAfu7x9KNgoRkRaUyw1Oa7uJThZJlncOAPE9tFUCv1pdBth/VPl4GWbWCXy9UvaZBMcW\nEZE6ShL6vcBVAGa2FNgdq9sFLDezTjPrAhYAe+LbAFcC2ytX+H8H/F93XzO+zCMiIjMnyfLOZuBy\nM9sBpICVZnYr0Ofuj5rZBmA70QnkNncfMbN1wIOV3+x5E/gVog+ELwE6zOzKyr4/6+7P1HlMIiJS\nQ6pcLje6DzXlcoPT6tyqu7fVuysiIjNu49rLprVdNptJ1arTl7NERAKi0BcRCYhCX0QkIAp9EZGA\nKPRFRAKi0BcRCYhCX0QkIC0Z+kvPf0ejuyAi0pRaMvQ/eem5je6CiEhTasnQz8yZ1eguiIg0pZYM\n/XR7G+efc3qjuyEi0nRaMvQBvrB6aaO7ICLSdFo29DtPSnPH9R9sdDdERJpKklsrn7DO7skcvktd\n/2Ceh7/bxzPff73BvRIRaZyWDv247kwHq68+n9VXn/+W8n0DI6z/639j70/1aEYRaX3BhH4tp3d1\n8qXffPv6/3C+wLMvvcE3/uH/kR/TQ75EpDUEH/q1zO5Ic/Gid3Lxone+ra5ULvNG/zA7v7+Xbz/z\nCsVS8z6IRkQkTqE/DW2pFD2nzWHF8nNYsfycqm1K5TIHDo3S958DPPvSGzz70hsz3EsRkbdT6B8n\nbakUp57SweLz5rP4vPl8epL2pXKZkXyB3P4R/mPvAV7+0X5efKWfgUOjM9JfEQmDQr9JtKVSzOmc\nxdk9szi7J8Ol7ztzyvsol8uUymXyoyUOjowxcDDPaAn6ftTPa/sO8dq+IV7bN0ShWDoOIxCRE4FC\nv4WkUinaUynmdLYxpzPN/FNnk81mOP9dpx73Y5fKZYrFEkP5IvsH87y27xA/efMQff85gP94/3E/\nvogkM2nom1kbcB+wCMgDN7h7X6x+NbAGKADr3H2Lmc0DNgGzgVeBle4+VK1tvQckjdGWStGWbqcr\n3U7XySdxdk9mxo5dLpcpV/4uFMsUiiVGx0qMjhUZHi3QObuDV18/wMHhMQaHRhk4NMqBQ6P0D+YP\n/9GH8RKKJFf6K4BOd19mZkuB9cDHAcysB7gJWAx0Ak+b2ZPA7cAmd3/AzNYCa8zsG9Xaunu+7qOS\noKRSKVLRC9rboGNWOyd3HqnPZjP0dHU0qnszrlw+cgIrx8pOPz3Dm28OUipFy4DlMhRLZYqlMqVS\nmbFC8fBJc6wQ/RktRCfP0UKR/FiJ/GiR0bEiI6NFRsaK5EcL0evRIkP5AiP5AsOjRYbzBcYKWkY8\nFr/84eNzt+AkoX8RsBXA3Xea2eJY3RKgtxLceTPrAxZWtvlSpc1jldf/XqPts7UO3N09h3S6fYpD\nOiKbnbmrzWamedAcjDujp6vRXZAGSxL6c4GB2PuimaXdvVClbhDoOqq8Wlm8vKb+/ul/SzabzZDL\nDU57+1ahedAcjNM8hDMHE13kJLnh2gEgvoe2SuBXq8sA+48qr1YWLxcRkRmSJPR7gasAKmv6u2N1\nu4DlZtZpZl3AAmBPfBvgSmD7BG1FRGSGJAn9zcCIme0A7gFuMbNbzewad98LbCAK9W3Abe4+AqwD\nrjWzXmAZcO8EbUVEZIak4p/0N5tcbnDanQtl7W4ymgfNwTjNQzhzkM1mUrXqWvYhKiIi8nYKfRGR\ngCj0RUQC0tRr+iIiUl+60hcRCYhCX0QkIAp9EZGAKPRFRAKi0BcRCYhCX0QkIAp9EZGAtNwzcid7\nvOOJysxmARuBdwMdRDe1ewF4gOgBSXuAG929ZGZ3AL9A9FjKm919l5mdm7TtTI5rusxsPvAvwOVE\nfX+AgObBzD4LXAOcRPTv/SnCm4NZwINE/yeKwGoC/LcwVa14pX/48Y7AWqLHO7aCXwP2uftyottV\n3wv8MfC5SlkK+LiZvR+4BPgQcC3wlcr2U2nb1Cr/2b8GDFeKgpoHM7sUuAC4kKjfZxHYHFRcBaTd\n/QLgfwF3EuY8TEkrhv5bHu9I9EzeVvA3wOdj7wvAB4iu8CB6LOVHiMb/hLuX3f1HQNrMslNs2+z+\nCPgq8GrlfWjzcAXRcy02A98CthDeHAC8TNTPNqIn840R5jxMSSuGftXHOzaqM/Xi7gfdfdDMMsDD\nwOeAlLuP30djssdSTqVt0zKz64Gcuz8eKw5tHuYRXcx8EvjvwNeJnmgX0hwAHCRa2nkJuJ/oeR2h\n/VuYslYM/Yke73hCM7OzgO8Af+num4BSrHqyx1JOpW0zWwVcbmbfBd4HPATMj9WHMA/7gMfdfdTd\nHRjhrcEUwhwA3EI0Dz9H9Bneg0SfcYwLZR6mpBVDf6LHO56wzOwdwBPA77n7xkrxc5X1XTjyWMpe\n4AozazOzdxGd9N6cYtum5e4Xu/sl7n4p8G/AdcBjgc3D08BHzSxlZu8ETgb+MbA5AOjnyFX5T4FZ\nBPh/YqpO+GWPKjYTXQnuIPpwZmWD+1Mvvw90A583s/G1/f8JbDCzk4AXgYfdvWhm24FniE7qN1ba\n/jZwf8K2J5qpjO2Enwd332JmFxM9d3q8v/9BQHNQcQ+wsdLvk4j+j3yP8OZhSnRrZRGRgLTi8o6I\niNSg0BcRCYhCX0QkIAp9EZGAKPRFRAKi0BcRCYhCX0QkIP8fFIYJHkQdKHkAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print(len(train_loss))\n",
    "plt.plot(train_loss,'-')\n",
    "plt.title('train_loss')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "19530\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5,1,'test_loss')"
      ]
     },
     "execution_count": 85,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEFCAYAAAD5bXAgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAH8pJREFUeJzt3XuYHHWd7/F3dfdcwyQMZCBCuK3A\n93DQgIBIJAF2hQ0BFXQ9Lof1eIibGC8rK+CFAy7oY4TdRxFBQATEiBIvLLIiR26rHA3BGMCwy8X9\nhgByCyEzZDKZyWR6prvr/FE1oQnVMz0zPd0zmc/refJk+vf7Vde3qnv6M3XpqiAMQ0RERHaWqnUB\nIiIyMSkgREQkkQJCREQSKSBERCSRAkJERBIpIEREJJECQkREEikgZEoys/vMbOYop73RzI4eZsyB\nZtYzuupEJgYFhExVp4xx2qBShYhMVJlaFyBSbWb2/fjHB8zsvcC3gP2BOuAn7n6ZmWWAbwPHAwPA\ns8Ai4P8A+wC3mtlH3f0PZcyvDvgm8B4gD/wBOM/du83sk8AngH6gD1jq7k+Vaq/MGhApj7YgZMpx\n90Xxj38JfB+42d2PBo4FTjazDwNzgZOAI+K+Z4E57n4xsAH4u3LCIfYlolA5Iv6XAr5uZmmicDrV\n3d8J3ADMK9U+xsUWGTEFhExl04ATga+a2WPAaqItiSOBx4n/2jezrwK3u/tDo5zPQuB6dx9w9wLR\nlslCd88DtwEPmdk1wBbge6XaR72UIqOkgJCpLCQ6lvBudz/S3Y8EjgMuc/ctRH/tf44oKH5qZp8a\n5XzS8bwGpYh2Z+HuHwHeB6wHLgR+PFS7SDUpIGSqyhMdW1gNnA9gZrsDq4Az4mMTvwYecvcvA7cA\n74ynzRF/wJfpHuCTZlZnZing08D9ZjbTzF4EXnP3bxHtinpnqfaxLa7IyCkgZKq6Dfgt8HHgODN7\nnOjg8Y/d/VbgbuBJ4AkzewR4N/CVeNqfAz8ys78uc17LgI3AY8CfiMLlH929I+77tZk9CvwzsKRU\n+1gXWGSkAt0PQkREkug0V5FRMrMric6ESnKeuz9QzXpEKk1bECIikkjHIEREJNGk3MXU3t496s2e\n1tZmOjt7K1lOxai20VFto6PaRmcy19bW1jKiS8RMuS2ITCZd6xJKUm2jo9pGR7WNzlSqbcoFhIiI\nlEcBISIiiRQQIiKSSAEhIiKJFBAiIpJIASEiIokUECIikmhSflFutLofeZiwKU1w+FG1LkVEZMIb\nNiDi69dfR3TzlCyw2N3XF/UvAZYSXSN/mbvfZWYzgRVAE9HtGRe5e288vg14CHi7u/eZ2QzgR8B0\noB44391/X8Fl3OGV668F4NCblo/H04uI7FLK2cV0JtDo7nOJ7mx1xWCHmc0CziW6sfsC4HIzawAu\nAVa4+3xgLVGAYGYLgPuAvYue/3zg1+5+InAOcO0Yl0lERCqgnF1M84juiIW7rzazY4r6jgVWuXsW\nyJrZemBOPM1l8Zi745+vBArAycCjRc9xJdGWyWA9fcMV1NraPKqvlK+L/29raxnxtNWi2kZHtY2O\nahudqVJbOQExHegqepw3s4y75xL6uoEZO7UPtuHu9wOY2Y4J4nv/Dm6N/Aj47HAFjfVCWe3t3WOa\nfry0tbWotlFQbaOj2kZnMtc20vAoZxfTVqD4WVNxOCT1tQBbdmofbCvJzN5OdP/fi9z9t2XUNCb5\n7dvHexYiIpNeOQGxCjgNwMyOAx4v6lsDzDezxvhg82HAE8XTAAuBlaWe3Mz+O9H9gc9297tHvASj\n8PKVX6/GbEREJrVydjHdAZxiZg8BAbDIzM4H1rv7nWZ2NVEApICL4zOTlgE/iM9w6gDOHuL5Lwca\ngaviXU9d7n7G6BdpeH3PPjueTy8isksYNiDcvQB8Yqfm/yrqvxG4cadpXgVOHeI5Dyz6eVzDoJTc\n1q1kpk+vxaxFRCaFKftN6hf/5Wu1LkFEZEKbsgEx8OqrZDdsqHUZIiIT1pQNCIDnL7mI7j8+OvxA\nEZEpaEoHBMAr132bDd+5hjAMa12KiMiEMuUDAqDn0Ud4eski+v78XK1LERGZMBQQRV5Y9hXWLf17\n+jdtqnUpIiI1p4DYWT7Pny/6AusWn0P25ZdrXY2ISM1MqftBjNTzl14MwJ5nfIDWU08jVVdX44pE\nRKpHAVGG135xB6/94g4A9vq7j9Jy3FzSTU01rkpEZHwpIEZo0623sOnWWwCYduQ7mPnB/0H9W95C\nEAQ1rkxEpLIUEGOw7bG1bHts7Y7Hu73jaFpPXUjjgQcRpEd+vwoRkYlEAVFBPWsfpWftG7941/LO\nY2mZ+24aD/oLMi269pOITB4KiHHW/fAauh9e88bGIGC3I49i2pw5NB54EPX7ziZI6YQyEZlYFBC1\nEIaJWxuDt0St22tvmg8/nMb9D6DpUCPdMp10c3P16xSRKU0BMQENbHqVrk2vvuFerjsL6utp2Hc2\nDfvtT/3s2dTvtRf1++xLurmZoKFRB81FZMwUEJNU2N9P33PP0vdc+Tc/SjU2ktljD+ra9iKzx57U\n7bEH9bPeQnq3FtIzZpCeNo1UQwPEB9gVMiJTmwJiCin09dG/YQP9o7jM+bqEtlRjI+nddyezeyup\npqYoYBobyczYnVRjI6lp00g1NkXjdtuNVF09QV0dQX0dQaaOIBO//eIgUiDJrirxYqDFbUEAhQIE\nAWE+H50FWShEw+LHYaFAqr6+ShVHFBAyaoW+PgobNzKwcWNN60gKL9JpCENSjY07fvEG21P19YT5\nAkEmTRCkCOrrCXO5HWNS9fWEYUgQBISFAkE6TZB5/bTlIJ2Jnr9QgDCMxmaKfpUKBUilCNJpNtWl\n6c8O7HhMGBLGv/gEwZs+JIJUinAgRxgWosBMpaAQEuZzr49Np6N6iUM1CAhzA4S5fDSfICAMQ8KB\ngWh8WCBIZygM9BPw+np4PtdPPpeHQgjpFOTzhLlcVF+hQJDJUOjvj54vn48e9/XFzxm+uf4KSnxN\nJ4ha1jZ93gnMOudjVZufAkJ2Tfk8AIXe3jd1FapYxpvnPvkMRsCOwNnRoUvkV9vWB39X1YDQuZUi\nIpJIASEiIokUECIikkgBISIiiRQQIiKSSAEhIiKJFBAiIpJIASEiIomG/aKcmaWA64AjgCyw2N3X\nF/UvAZYCOWCZu99lZjOBFUATsAFY5O698fg24CHg7e7eZ2ZNwI+AvYBu4H+7e3sFl1FEREahnC2I\nM4FGd58LXAhcMdhhZrOAc4HjgQXA5WbWAFwCrHD3+cBaogDBzBYA9wF7Fz3/J4HH47G3AF8a60KJ\niMjYlRMQ84B7ANx9NXBMUd+xwCp3z7p7F7AemFM8DXA3cHL8cyH+eXPS8+80VkREaqicazFNhzfc\nmiBvZhl3zyX0dQMzdmofbMPd7wcws1LPv2PsUFpbm8lkRn7P54l8ATARkXK0tbWMqX8kygmIrUDx\nHFNxOCT1tQBbitq3F7WV8/zDjQWgs3NXuASaiMjItbd3l+xra2sZtn8kytnFtAo4DcDMjgMeL+pb\nA8w3s0YzmwEcBjxRPA2wEFhZzvOXMVZERKqknC2IO4BTzOwhIAAWmdn5wHp3v9PMrib6UE8BF8dn\nJi0DfhCf4dQBnD3E838nHvsg0D/MWBERqZIg8U5HE1x7e/eoil63+JwKVyIiUl2H3rS8ZF8Zu5hG\ndNtGfVFOREQSKSBERCSRAkJERBIpIEREJJECQkREEikgREQkkQJCREQSKSBERCSRAkJERBIpIERE\nJJECQkREEikgREQkkQJCREQSKSBERCSRAkJERBIpIEREJJECQkREEikgREQkkQJCREQSKSBERCSR\nAkJERBIpIEREJJECQkREEikgREQkkQJCREQSKSBERCSRAkJERBJlhhtgZingOuAIIAssdvf1Rf1L\ngKVADljm7neZ2UxgBdAEbAAWuXtvibH7Az8EAmAzcLa791ZyIUVEZOTK2YI4E2h097nAhcAVgx1m\nNgs4FzgeWABcbmYNwCXACnefD6wFlg4x9jzgp+5+AvAk8PeVWjgRERm9cgJiHnAPgLuvBo4p6jsW\nWOXuWXfvAtYDc4qnAe4GTh5i7GNAazx2OjAwpiUSEZGKGHYXE9GHdlfR47yZZdw9l9DXDczYqT2p\nrbj9JeCfzexsoAH48nAFtbY2k8mkyyj9jdaNeAoRkYmlra1lTP0jUU5AbAWK55iKwyGprwXYUtS+\nPaFt57E3AOe4+71mdjpwC3D6UAV1duoQhYhMTe3t3SX72tpahu0fiXJ2Ma0CTgMws+OAx4v61gDz\nzazRzGYAhwFPFE8DLARWDjG2k9e3LDbw+u4mERGpoXK2IO4ATjGzh4jONFpkZucD6939TjO7migA\nUsDF7t5nZsuAH8RnLXUQnZm0rcTYzwDXmFk6fv5PV3wpRURkxIIwDGtdw4i1t3ePquh1i8+pcCUi\nItV16E3LS/aVsYspGMm89EU5ERFJpIAQEZFECggREUmkgBARkUQKCBERSaSAEBGRRAoIERFJpIAQ\nEZFECggREUmkgBARkUQKCBERSaSAEBGRRAoIERFJpIAQEZFECggREUmkgBARkUQKCBERSaSAEBGR\nRAoIERFJpIAQEZFECggREUmkgBARkUQKCBERSaSAEBGRRAoIERFJpIAQEZFECggREUmkgBARkUSZ\n4QaYWQq4DjgCyAKL3X19Uf8SYCmQA5a5+11mNhNYATQBG4BF7t5bYuw04DvAQUA98Bl3X1PJhRQR\nkZErZwviTKDR3ecCFwJXDHaY2SzgXOB4YAFwuZk1AJcAK9x9PrAWWDrE2M8DT8RjlwBWqYUTEZHR\nG3YLApgH3APg7qvN7JiivmOBVe6eBbJmth6YE09zWTzm7vjnZ0qMXQD81MzuBbYCnx6uoNbWZjKZ\ndDnL9wbrRjyFiMjE0tbWMqb+kSgnIKYDXUWP82aWcfdcQl83MGOn9qS24vaZQKu7LzCzjwLfAD46\nVEGdnb1llC0isutpb+8u2dfW1jJs/0iUs4tpK1D8rKk4HJL6WoAtO7UntRW3vwbcGbf9EijeQhER\nkRopJyBWAacBmNlxwONFfWuA+WbWaGYzgMOAJ4qnARYCK4cY+2DR2BOAJ8e0RCIiUhHlBMQdQJ+Z\nPQRcCZxnZueb2fvdfSNwNVEA/Aa42N37gGXAWWa2CpgLXDPE2MuAd5jZ74ELiA5ai4hIjQVhGNa6\nhhFrb+8eVdHrFp9T4UpERKrr0JuWl+wr4xhEMJJ56YtyIiKSSAEhIiKJFBAiIpNIWChUbV4KCBGR\nSSTM56s2LwWEiIgkUkCIiEgiBYSIiCRSQIiISCIFhIiIJFJAiIhIIgWEiMikUr3LIykgREQkkQJC\nRGRSGdH19sZEASEiIokUECIik4qOQYiISI0pIEREJhUdgxARkRpTQIiITCo6BiEiIjWmgBARkUQK\nCBERSaSAEBGZTKp3CEIBISIiyRQQIiKSSAEhIiKJFBAiIpNJWL2DEJnhBphZCrgOOALIAovdfX1R\n/xJgKZADlrn7XWY2E1gBNAEbgEXu3ps0tuh5TgBudff9KrZ0IiIyauVsQZwJNLr7XOBC4IrBDjOb\nBZwLHA8sAC43swbgEmCFu88H1gJLhxiLme0HXADUVWrBRERkbMoJiHnAPQDuvho4pqjvWGCVu2fd\nvQtYD8wpnga4Gzi51FgzawSuBz5VgeUREZEKGXYXEzAd6Cp6nDezjLvnEvq6gRk7tSe1FbdfA3zD\n3V82s7KKbm1tJpNJlzW22LoRTyEiMrHMnLkb6cbGkv1tbS0Vm1c5AbEVKJ5jKg6HpL4WYEtR+/aE\ntuKx/cB84GAzuxTYw8x+4u5nDVVQZ2dvGWWLiOx6Ojp6SDUMJPa1tbXQ3t5dctqRhkc5AbEKeB/w\nMzM7Dni8qG8N8LV4N1EDcBjwRDzNacByYCGwssTYNe6+Y7PBzDYOFw4iIlId5RyDuAPoM7OHgCuB\n88zsfDN7v7tvBK4mCoDfABe7ex+wDDjLzFYBc4FrhhgrIiITUBBW8ZzaSmlv7x5V0esWn1PhSkRE\nquvga64nVeIYRBm7mEZ0Ozp9UU5ERBIpIEREJJECQkREEikgREQmFd2Telzs98WLal2CiMikMaUC\noumQQ2tdgojIpDGlAgJgn/e/t9YliIhMClMvIM48o9YliIiMWjW/ujblAqJhzz3Y97MX1LoMEZEJ\nb8oFBMC0t72dA7781VqXISIyoZVzsb5dUsPs/Tj0puUUBvrp+PntbLn/3lqXJCIyoUzJLYhiqbp6\n9vrb/8lBl3+91qWIiAyvigchpnxADKpra+OQ62+i+fC31boUEZEJQQFRJMhkmH3e5zjgUh2fEBFR\nQCRo2G8/Drnx+7SeelqtSxERqRkFRAlBEND2oQ9z8LXfZbejj6l1OSIiMR2DmDBSDQ3s88l/4K1X\nX0vzYYfXuhwRkaqZsqe5jlS6eRqzL/g8hb7tvPrDW+j+w+9rXZKIyLjSFsQIpRqbeMuSpRxyw820\nnf2RWpcjIlNNFS+1oS2IUQpSKVr/6mRa/+pksi+/xAuXfZUwm611WSIiFaOAqICGfWdzyLXfJczl\n2HzPr3jt335e65JERMZMAVFBQSbDnu99P3u+9/3ke7fRee89bP6/v6x1WSIio6KAGCfp5mnM/MDf\nMPMDf0Ohr48tD/yajttvq3VZIjLZVfFSGwqIKkg1NrLHwtPZY+HphIUCPY+tpeO2nzDQ3l7r0kRE\nSlJAVFmQStFy1NG0HHU0QLQr6p672fyru2pcmYjIGykgaizdPI2ZH/wQMz/4IdraWnjl6RfZ8pt/\nV2CISM0pICaYzO677wgMgHxvLz2PPMzmX93FQId2SYlI9QwbEGaWAq4DjgCywGJ3X1/UvwRYCuSA\nZe5+l5nNBFYATcAGYJG795YYuz9wc1xLAHzc3b2SCzmZpZubmXHCicw44UQAwkKBgVc30rXyd3Te\nd0+NqxORXVk5WxBnAo3uPtfMjgOuAM4AMLNZwLnAMUAj8KCZ3Q9cAqxw9+VmdiGw1Mx+XGLsV4Fr\n3P3fzGwBcDnwwYou5S4kSKWof8s+tH34LNo+fBYAYRiSffEFOm6/jd4nn6hxhSKyqygnIOYB9wC4\n+2ozK7606bHAKnfPAlkzWw/Miae5LB5zd/zzMyXGXgB0FdXTN7ZFmnqCIKBx/wOYfd7ndrSFYci2\n/3iMDddcVcPKRGQyKycgpvP6BzhA3swy7p5L6OsGZuzUntS2o93dOwDMzIBvEG2xDKm1tZlMJl1G\n6cna2lpGPe14q2htp5zAQaecsONhtuM1nr3hJjb/YU3l5iEiVbXnntOom176c6KSnyHlBMRWoHiO\nqTgckvpagC1F7dsT2nYei5n9JdFxjv9VzvGHzs7eMspO1tbWQnt796inH0/jX1s9M5d8iplLPgVA\nmMvR6//FK9/9DoXebeM4XxGplNc6ekhng8S+4T5DRhoe5QTEKuB9wM/iYxCPF/WtAb5mZo1AA3AY\n8EQ8zWnAcmAhsLLU2DgcrgJOdffnR1S9jEmQyTDt8Ldx8NXX7mgrZLNse/w/2XjzjYT9/TWsTkRq\nLQiH+dp20VlMc4jOMlpE9OG/3t3vjM9M+jjRpcMvc/fbzWxv4AdEWwkdwNnuvq3E2P8gCoyN8Szd\n3ZcOVVN7e/eov2s+tbcgRqeQzVK/6UWeW/Eztj+9rtbliExpb73y26RbkrcEytiCSN70KGHYgJiI\nFBDVt3NtYaFA/8aN9PzxETbfdSdhLjfE1CJSKX9x5dVkWqYn9lU6IPRFORmVIJWiYZ99aNgnunrt\noDCXo+/5P9P71JNs+X8PkO/aUsMqRWQsFBBSUUEmQ9NbD6bprQez5/vO2NEeFgrkOjez/el19Dy2\nlp5HHq5hlSJSDgWEVEWQSlG350zq9pzJ9OPe/Ya+sFBgYNMm+p5/jp61a+l96kmdVSUyASggpOaC\nVIr6WbOonzWL6e+a+4a+MAwp9PWRffEF+p55ht6nnqD3T0/VqFKRCUD3pBaJBEFAuqmJ5kON5kON\nPRae9qYxrdMybHz6ebY//TR9zz1L71NPktv8Wg2qFRl/uc2byUxPPkhdaQoImfQyzU007LMvDfvs\nCyee9Kb+MAwJs30MdHSQffFF+p57hu1PP032xReqX6zIGBX6s1WblwJCdnlBEBA0NtEwez8aZu/H\n9LnvThwXhiGFnh5yWzrpe+F5si+9RPbPz9H35+cIBwaqXLVI7SkgRGJBEJBuaSHd0kLDfvuXHBeG\nIYQh+e5ucl1b6H/5ZbIvv0T2hefJvvwS+a6uktOKjFUQjOirDGOigBAZoSAIIAjIzJhBZsYMGvc/\noOTYwS8uhbkc+e295Do7Gdj0Kv0bN9K/4WX6X9lAdsMGyOeruAQi5VFAiFRBkMmQaZlOpmX6kIFS\nLCwUCPuz5Ht6GNi8mVxHB/2bXmWgo53+DRsYaN9EYfv2ca5cpjIFhMgEFaRSBI1NpBqbqJvZBoda\n2dMOhkuhL0tuyxZyW7sYaN9EvquL/lc3kuvqYuCVDeS36fsmk452MYnIWBSHS2b33RPHDHfdnjAM\no4PzhTy5rq0U+raT27yZ3Nat5Da/Rr57KwPt7eS6tjDQ0aGr/1ZJZkby6zku86ranERkUgmCgKC+\nHoD6xqao8YADR/18YaEAQGH7dsLcAPmebeS39VDYto1c52by27aR27KFfE83dHfR99pmcp2dEE8n\nkbq2tqrNSwEhIlURpFIApKdNA4b+S3ikVzYePLMszOcIB3KE/f0UstkogPr6yHd3E2azDHRuprC9\nl/zWbgrbe8l1d5Pv2kK+t5cwW73vF0wWCggRmfQGzywLUvVQVw/NzXHP3hWf144z08Jwx9ZNoa8P\nAij09kII+Z5uCgMD5Ht6CAf6o/+zWfK9vRSyfYR9WXJbuwj7+8lv6yHM5cl1bh72+zb7nv/5ii/P\nUBQQIiKjEAQBpNPA61tF6ebo/2ruBhpPqVoXICIiE5MCQkREEikgREQkkQJCREQSKSBERCSRAkJE\nRBIpIEREJJECQkREEgVhWMU7YIuIyKShLQgREUmkgBARkUQKCBERSaSAEBGRRAoIERFJpIAQEZFE\nCggREUk0ZW4YZGYp4DrgCCALLHb39VWYbx1wM3Ag0AAsA14Cfgk8HQ/7jrv/1MwuBU4HcsBn3X2N\nmR0MLAdC4Ang0+5esZv0mtlaoCt++BzwXeCquIb73P0rpdadmR2389gK1nUOcE78sBE4Ejgb+Drw\nYtx+KbCymrWZ2buAf3H3k0q9NiN5HZPGVqi2I4FvA3mi9fJRd3/VzK4GjgcG7+d5BlAHrACagA3A\nInfvNbMlwNK4tmXufleFajuKMb7/x3G9/QSYFXcdCKx297PM7E5gT2AA2O7uC8ezthKfG0+VO79K\n1TaVtiDOBBrdfS5wIXBFleb7EeA1d58PLASuAY4CvunuJ8X/fhr/0pwIvAs4C7g2nv6bwJfi6QOi\nX+iKMLNGgKI6FgHXE30QzwPeFddVat0lja0Id18+WBfwKHAu0Xr7QlG9v61mbWb2BeAmosCChNdm\nJK/jEGMrUdtVwGfi9fdz4Itx+1HAgqJ12AVcAqyIa1sLLDWzWUTr/HhgAXC5mTVUqLYxvf/Hc725\n+1nxOvsAsAU4Lx56MDAvrndhFWpL+tyo+vttKgXEPOAeAHdfDRxTpfneBvxT0eMccDRwupn9zsy+\nZ2YtcX33uXvo7i8AGTNri8f+Np72buDkCtZ2BNBsZveZ2W/M7ASgwd2fcfcQuBd4Dwnrzsymlxhb\nUWZ2DHC4u99AtC4+ZmYrzewKM8tUubZngA8WPU56bUbyOpYaW4naznL3x+KfM0BfvCV4CHCDma0y\ns4/F/TvWYVFtxwKr3D0bh8h6YE6Fahvr+38819ugrwDfdvdXzGxvYHfgl2b2oJm9t2g5xqu2Up8b\nVX2/TaWAmM7ru1IA8vEHzLhy9x53745/Cf4V+BKwBvi8u58APEu0q2Tn+rqBGUAQf8gVt1VKL/AN\nor8QPwF8P27buYY3rbu4bWvC2Eq7iOiXFeB+4DPACcBucc1Vq83dbyfaxTAo6bUZyetYauyYa3P3\nVwDM7N3APwBXAtOIdjt9BDgV+JSZzdmpjnGvjbG//8ezNsxsL6I/KJbHTfVEW6ZnEoXJlfGYcaut\nxOdG1d9vUykgtgItRY9T7p6rxozNbD/gAeCH7r4CuMPdH4277wDekVBfC9EmbiGhrVLWAT+K/6JY\nR/Tm2SNhfm9ad0PUWzFmtjvw39z9gbjpZnd/Nn7j/4Lk9VaV2mJJr81IXsdxrdPM/pZoV9vp7t5O\nFP5XuXuvu3cDvyHaiiyuoxq1jfX9P96v74eIdrnl48cbgevdPefum4h2w9l415bwuVH199tUCohV\nwGkA8QHMx6sx03jz9D7gi+5+c9x8r5kdG//8HqJ97KuABWaWMrP9iQKsA1hrZifFYxcSHZStlI8R\n77M3s32AZmCbmb3VzAKiLYuVJKw7d98K9CeMraQTgH+P5xsA/2lms+O+4vVWi9og+bUZyetYauyY\nmdlHiLYcTnL3Z+PmQ4EHzSwdHwSdB/yRonVYVNsaYL6ZNZrZDOAwooOdlTDW9/+4rbfYyUS7ZYof\n/wzAzHYD3gb8aTxrK/G5UfX325Q5i4noL5VTzOwhooM2i6o034uAVuCfzGxwn+L5wLfMrJ/or5OP\nu/tWM1sJ/J4ouD8dj70AuNHM6onelP9awdq+Byw3sweJznb4GNFfHrcCaaL9lX8ws4dJXnef2Hls\nBWuD6K+0ZwHcPTSzxcDPzWw70RkdNxLtUqpFbZDw2rh7vtzXcYixY2JmaeBq4AWi9QXwW3e/1Mxu\nBVYT7Va5xd2fNLNlwA8sOmupAzjb3bdZdMbTyri2i929rxL1AZ8Erhnt+3+81luRHe87AHe/28wW\nmNlqot+Pi9y9w8zGs7akz41/BK6u5vtNl/sWEZFEU2kXk4iIjIACQkREEikgREQkkQJCREQSKSBE\nRCSRAkJERBIpIEREJNH/B61975jNj9jdAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print(len(test_loss))\n",
    "plt.plot(test_loss,'r')\n",
    "plt.title('test_loss')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "#### Demornamization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [],
   "source": [
    "# normalized this way:\n",
    "#test_norm =  (df_new_test - df_new_test.mean()) / (df_new_test.max() - df_new_test.min())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [],
   "source": [
    "rmse = np.sqrt(mse)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [],
   "source": [
    "rmse_denorm = (rmse * (df_new_test['Kt'].max() - df_new_test['Kt'].min()))+ df_new_test['Kt'].mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "20.074817306405894"
      ]
     },
     "execution_count": 89,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rmse_denorm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [],
   "source": [
    "#df_new_test['Kt'].sort_values()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    4228.000000\n",
       "mean        7.648923\n",
       "std        39.416136\n",
       "min        -0.267460\n",
       "25%        -0.007315\n",
       "50%         0.016496\n",
       "75%         0.905913\n",
       "max      1026.823214\n",
       "Name: Kt, dtype: float64"
      ]
     },
     "execution_count": 91,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_new_test['Kt'].describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Saving train and test losses to a csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_trainLoss = pd.DataFrame(data={'Train Loss':train_loss}, columns=['Train Loss'])\n",
    "df_trainLoss.to_csv('RNN Paper Results/Exp1_Boulder_1000Epoch_TrainLoss.csv')\n",
    "df_testLoss = pd.DataFrame(data={'Test Loss':test_loss}, columns=['Test Loss'])\n",
    "df_testLoss.to_csv('RNN Paper Results/Exp1_Boulder_1000EpochTestLoss.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}