2. Linear Regression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Linear Regression with Tensorflow\n",
    "\n",
    "This tutorial will show you a simple linear regression example.\n",
    "We will declare some variable X, with corresponding dependent variables Y, and try to model their relationship using linear regression.\n",
    "\n",
    "Here, first declare the packages we will be using: Tensorflow, numpy and matplotlib.pyplot. We will use numpy to create our data and generate some numbers. Matplotlib.pyplot would help us construct matlab-like plots (if you are familiar with Matlab), to plot our resulting fitted line.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below, we will declare some of the fixed parameters we will use in our learning, and plot-drawing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "learning_rate = 0.01\n",
    "training_epochs = 1000\n",
    "display_step = 50"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the following part, we will declare our dataset. We only use one dimensional features here for X, so that we can visualize it later on in a 2-dimensional plot. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "train_X = np.asarray([3.3,4.4,5.5,6.71,6.93,4.168,9.779,6.182,7.59,2.167,\n",
    "                         7.042,10.791,5.313,7.997,5.654,9.27,3.1])\n",
    "train_Y = np.asarray([1.7,2.76,2.09,3.19,1.694,1.573,3.366,2.596,2.53,1.221,\n",
    "                         2.827,3.465,1.65,2.904,2.42,2.94,1.3])\n",
    "n_samples = train_X.shape[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we move on to the part where we will start our learning with Tensorflow. \n",
    "\n",
    "We first declare some Placeholder Tensors to represt our input data. Since they are only one dimensional, we declare their type to be a float number. \n",
    "\n",
    "Next, we will declare the Variable Tensors. This depends on the parameters we will train in our learning method.\n",
    "For our data, we will be training the following model:\n",
    "\n",
    "\\begin{equation}\n",
    "    y = W*x + b\n",
    "\\end{equation}\n",
    "\n",
    "This gives us two parameters which values we need to train, a weighting coefficient*W*, and a bias factor *b*. \n",
    "Since they will be declared as Variable Tensors, we need to give them an initial value to work on. Below, we use a randomizer to do that."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#Declare input data type\n",
    "x=tf.placeholder(tf.float32)\n",
    "y=tf.placeholder(tf.float32)\n",
    "\n",
    "#Declare trainable parameters\n",
    "randomizer=np.random\n",
    "W=tf.Variable(randomizer.randn(), tf.float32, name=\"weight\")\n",
    "b=tf.Variable(randomizer.randn(), tf.float32, name=\"bias\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After declaring our data and parameters, we move on to declaring our model, and a cost function to evaluate it against. \n",
    "\n",
    "There are different cost function that you can use. Here, we give you two examples: the sum of squares, and the mean squared error, that you can choose from.\n",
    "\n",
    "If y' is the predicted value of y given by our learned model, the sum of squares and mean squared error equations are given as follows:\n",
    "\n",
    "\\begin{equation}\n",
    "Sum\\_of\\_squares = \\sum_{n=1}^{N} y_n'² - y_n²\n",
    "\\end{equation}\n",
    "\n",
    "\\begin{equation}\n",
    "MSE = \\frac{Sum\\_of\\_squares}{2 * N}\n",
    "\\end{equation}\n",
    "\n",
    "*N is the number of of data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#Declare model\n",
    "linear_model=tf.add(tf.multiply(W,x),b) #depending on your Tensorflow version, you might need to change tf.mul to tf.multiply\n",
    "\n",
    "#Declare cost function\n",
    "loss=tf.reduce_sum(tf.square(linear_model-y)) #sum of squares\n",
    "loss_=tf.reduce_sum(tf.pow(linear_model-y,2))/(2*n_samples) #mean squared error"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we declare the method that we want to use to optimize our learning parameters. Here, we will use a Gradient Descent optimizer. To declare the optimizer, we need to indicate a learning rate, and associate it with the cost function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "optimizer=tf.train.GradientDescentOptimizer(learning_rate).minimize(loss_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we can move on to fitting our data into a regression model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " Epoch: 0050 cost= 0.310483515 W= 0.519679 b= -1.14151\n",
      "Epoch: 0100 cost= 0.283513367 W= 0.503616 b= -1.02595\n",
      "Epoch: 0150 cost= 0.259656936 W= 0.488508 b= -0.917267\n",
      "Epoch: 0200 cost= 0.238555044 W= 0.474299 b= -0.815046\n",
      "Epoch: 0250 cost= 0.219889790 W= 0.460934 b= -0.718904\n",
      "Epoch: 0300 cost= 0.203379929 W= 0.448365 b= -0.62848\n",
      "Epoch: 0350 cost= 0.188776419 W= 0.436543 b= -0.543435\n",
      "Epoch: 0400 cost= 0.175859481 W= 0.425424 b= -0.463447\n",
      "Epoch: 0450 cost= 0.164434075 W= 0.414967 b= -0.388217\n",
      "Epoch: 0500 cost= 0.154328331 W= 0.405131 b= -0.317462\n",
      "Epoch: 0550 cost= 0.145389676 W= 0.395881 b= -0.250914\n",
      "Epoch: 0600 cost= 0.137483403 W= 0.38718 b= -0.188324\n",
      "Epoch: 0650 cost= 0.130490318 W= 0.378997 b= -0.129456\n",
      "Epoch: 0700 cost= 0.124305055 W= 0.371301 b= -0.0740898\n",
      "Epoch: 0750 cost= 0.118834212 W= 0.364063 b= -0.0220162\n",
      "Epoch: 0800 cost= 0.113995403 W= 0.357255 b= 0.0269602\n",
      "Epoch: 0850 cost= 0.109715641 W= 0.350852 b= 0.0730238\n",
      "Epoch: 0900 cost= 0.105930336 W= 0.344829 b= 0.116348\n",
      "Epoch: 0950 cost= 0.102582395 W= 0.339165 b= 0.157095\n",
      "Epoch: 1000 cost= 0.099621348 W= 0.333838 b= 0.195419\n",
      "Optimization Finished!\n",
      "Training cost= 0.0996213 W= 0.333838 b= 0.195419 \n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgkAAAFkCAYAAACq4KjhAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzt3Xl8VOW9x/HPbzCCgQAiKhrARFCKerUmtLKriIAUcKH2\nNor7Ri1CQbQuVLAlblWQKnrV1gpW00W8CipSFFeIeE3q0hKtCohiRRAMAVwi89w/zmSZZBIyk8mc\nmcn3/XrlJfPMmTm/I8t859mOOecQERERqSvgdwEiIiKSnBQSREREJCKFBBEREYlIIUFEREQiUkgQ\nERGRiBQSREREJCKFBBEREYlIIUFEREQiUkgQERGRiBQSREREJKJmhQQzu9bMgmY2Zw/HnWlmZWb2\nlZm9ZWanNOe8IiIi0vJiDglm9gPgEuCtPRw3AHgUeAD4PvAE8ISZHRHruUVERKTlxRQSzKwD8Cfg\nYuDLPRw+BVjqnJvjnHvPOTcTKAUmxXJuERERSYxYexLmA0uccyuacOwA4Lk6bctC7SIiIpKk9or2\nBWb2U7xhg35NfEk3YFOdtk2h9obOsR8wElgPfB1tjSIiIq1YOyAHWOac+6I5bxRVSDCz7sCdwMnO\nucpmnNcA18jzI4FHmvH+IiIird3ZeHMCYxZtT0I+sD9QYmYWamsDDDWzSUBb51zdD//PgAPrtB1A\n/d6F2tYD/OlPf6Jv375Rlph8pk6dyty5c/0uI250Pckrna4FdD3JLJ2uBdLresrKypgwYQKEPkub\nI9qQ8BzwX3XaHgLKgFsiBASAYuAk4He12k4OtTfka4C+ffuSl5cXZYnJp1OnTmlxHVV0Pckrna4F\ndD3JLJ2uBdLvekKaPVwfVUhwzu0E1tRuM7OdwBfOubLQ4wXARufcdaFD5gEvmdk04GmgAK9H4pJm\n1i4iIiItKB47LtbtPehBrUmJzrlivGBwKfAmcAZwqnNuDSIiIpK0ol7dUJdzblhjj0Nti4BFzT2X\niIiIJI7u3ZAABQUFfpcQV7qe5JVO1wK6nmSWTtcC6Xc98WKR5xr6y8zygJKSkpJ0nEgiIiLSYkpL\nS8nPzwfId86VNue9mj3cICKSjjZs2MCWLVv8LkOknq5du9KzZ8+EnEshQUSkjg0bNtC3b1927drl\ndyki9WRmZlJWVpaQoKCQICJSx5YtW9i1a1fabOgm6aNqo6QtW7YoJIiI+CldNnQTiZVWN4iIiEhE\nCgkiIiISkUKCiIiIRKSQICIiIhEpJIiISMxmzZpFIBDbR8lDDz1EIBBgw4YNca6qxkcffUQgEGDh\nwoUxvT4RNSYzhQQRkVZozZo1TJgwge7du9OuXTuys7OZMGECa9ZEd+89M4s5JJgZZhbTaxOlOTUW\nFRUxb968OFeUWAoJIiKtzOOPP05eXh4vvPACF154Iffeey8XX3wxL774Inl5eTz55JNNfq9f/epX\nMW86de655/LVV18lbPfARHv00UdTPiRonwQRkThwzrXYt+J4vvfatWs599xz6d27Ny+//DJdunSp\nfm7KlCkMHjyYc845h7fffpucnJwG32fXrl1kZmYSCATYe++9Y6rFzGJ+rSSGehJERGJUUVHBzMmT\nGZ6by2k9ejA8N5eZkydTUVGRtO9922238dVXX3H//feHBQSALl268D//8z/s2LGD2267rbq9at5B\nWVkZZ511Fl26dGHIkCFhz9X29ddfM3nyZPbff386duzIaaedxqeffkogEODXv/519XGRxvtzcnIY\nN24cK1eu5LjjjmOfffahV69ePPzww2Hn2LZtG9OnT+foo48mKyuLTp06MXr0aN5+++2Y/9+sWbOG\nYcOGkZmZSY8ePSgsLCQYDNY7bvHixYwZM4bs7GzatWtH7969mT17dtixJ554Ik8//XT1nIhAIMCh\nhx4KQGVlJTfccAP9+vWjc+fOdOjQgaFDh/Liiy/GXHtLUU+CiEgMKioqGD9gANPKypgVDGKAA5bN\nn8/4FStYVFxMVlZW0r33U089RU5ODgMHDoz4/NChQ8nJyeGpp57innvuAajuxTjzzDM5/PDDufnm\nm6m6g3CkMfvzzjuPxx57jHPPPZfjjjuOl156iR/96Ef1jov0WjPj/fff58wzz+Siiy7i/PPP58EH\nH+SCCy6gX79+1dtkr127lsWLF3PmmWeSm5vLpk2buO+++zjhhBNYs2YN3bp1i+r/y6ZNmzjhhBMI\nBoNcd911ZGZmcv/999OuXbt6xz700ENkZWVx5ZVX0qFDB1asWMENN9xARUUFt956KwAzZsygvLyc\njRs3cuedd+Kco0OHDgBs376dBx98kIKCAi699FIqKir4wx/+wKhRo3j99dc5+uijo6q9RTnnku4H\nyANcSUmJExFJtJKSErenf4NuuOIKtzQQcA7q/TwTCLiZkyfHfP6Weu/y8nJnZu70009v9LhTTz3V\nBQIBt2PHDuecc7NmzXJm5s4+++x6x86aNcsFAoHqx6Wlpc7M3JVXXhl23AUXXOACgYC78cYbq9se\neughFwgE3EcffVTdlpOT4wKBgFu5cmV12+bNm127du3cVVddVd327bff1qvlo48+cu3atXOzZ8+u\nblu/fr0zM7dgwYJGr/kXv/iFCwQC7o033qhu27Jli+vcuXO9Gr/++ut6r584caLr0KFDWF1jxoxx\nubm59Y4NBoOusrIyrK28vNx169bNXXzxxY3W2ZQ/m1XHAHmumZ/HGm4QEYnByiVLGBmhKxpgVDDI\nysWLk+69q4Yq9tQLUfX89u3bq9vMjIkTJ+7xHM8++yxmxs9+9rOw9iuuuKK692FPjjjiiLCejq5d\nu9KnTx/Wrl1b3ZaRkVH962AwyNatW8nMzKRPnz6UlpY26Ty1LV26lP79+5Ofn1/dtt9++3H22WfX\nO7Zt27bVv96xYwdffPEFgwcPZteuXbz77rt7PJeZsddeXke+c45t27bx7bff0q9fv5hqb0kKCSIi\nUXLO0b6ykoamEhqQWVnZ5A/FRL131Yf/nuY1NBQmcnNz93iOqjH4usf27t27yXVGWu2w7777sm3b\nturHzjnmzp3L4YcfTtu2benatSsHHHAA77zzDuXl5U0+V+26DzvssHrtffr0qde2Zs0aTj/9dDp3\n7kzHjh3Zf//9OeeccwCafO4FCxZwzDHH0K5dO/bbbz8OOOAAnn766Zhqb0makyAiEiUzY2dGBg4i\nfpg7YGdGRkwrElryvTt27MhBBx20x8l9b7/9NtnZ2dVj6FX22WefqM9ZJZp627RpE7G9djAqLCzk\nhhtu4KKLLmL27Nl06dKFQCDAlClTIk42jLXGumGsvLycoUOH0rlzZ2bPns2hhx5Ku3btKCkp4Zpr\nrmnSuf/0pz9xwQUXcMYZZ3D11VdzwAEH0KZNG2666aaw3pJkoJAgIhKDQWPHsmz+fEZF+FB4NhBg\n8LhxSfneY8aM4fe//z2rVq2KOHnxlVdeYf369fWGC5rqkEMOIRgMsm7dOnr16lXd/u9//zvmmiNZ\ntGgRw4YN44EHHghr//LLL9l///2jfr9DDjkkYo3vvfde2OMXX3yRbdu28eSTTzJo0KDq9g8//LDe\naxsKRosWLaJXr1489thjYe033HBD1HW3NA03iIjEYHphIXP69mVpIEDVd00HLA0EmNu3L1fOnp2U\n733VVVfRrl07LrvsMrZu3Rr23NatW5k4cSLt27dn+vTpMb3/yJEjcc5Vr4yoctddd8V1H4k2bdrU\n+5b/t7/9jY0bN8b0fqNHj+a1117jjTfeqG7bvHkzRUVFEc9bu8fg22+/rXe9AO3bt484fNCmTZt6\n/y9Wr15NcXFxTLW3JPUkiIjEICsri0XFxdwxYwZzFi8ms7KSXRkZDBo3jkWzZ8e8RLGl37t3794s\nWLCACRMm8F//9V9cdNFF5Obmsm7dOh588EG++OIL/vznPzdp/kEkeXl5jB8/njvvvJMtW7bQv39/\nXnrpJd5//30gumGHxowZM4bf/OY3XHjhhQwcOJB33nmHRx55JKz3IhpXX301Dz/8MCNHjmTKlClk\nZmbywAMPcMghh4QNzwwcOJB9992Xc889l8mTJwPe8EGk68rPz+evf/0rV155JT/4wQ/o0KEDY8aM\nYcyYMTz++OOcdtpp/OhHP2Lt2rXcd999HHnkkezYsSO2/yEtpbnLI1riBy2BFBEfNWWZWV3BYLDF\n6mmJ9/7nP//pzj77bJedne3atm3rDj74YDdhwgT3r3/9q96xVcscv/jii4jPtWnTJqztq6++cldc\ncYXr2rWr69ixoxs/frx7//33nZm52267rfq4SEsgc3Nz3bhx4+qd54QTTnDDhg2rfvzNN9+4q666\nymVnZ7v27du7oUOHutWrV7sTTzwx7Lj169e7QCCwxyWQVf9PTjzxRJeZmel69OjhbrrpJvfggw/W\nq7G4uNgNHDjQtW/f3nXv3t1de+21bvny5S4QCLiXXnqp+ridO3e6CRMmuC5durhAIBC2HPKWW25x\nubm5bp999nH5+fnumWeeceeff7479NBDG60x0UsgzcUwQ7almVkeUFJSUkJeXp7f5YhIK1NaWkp+\nfj76Nyh+3nzzTfLy8njkkUcoKCjwu5yU1ZQ/m1XHAPnOuWatqdScBBERiatvvvmmXtudd95JmzZt\nGDp0qA8VSaw0J0FEROLqtttuo6SkhBNOOIG99tqLZ555hmXLlnHZZZeRnZ3td3kSBYUEERGJqwED\nBrB8+XJmz57Njh076NmzJzfeeCPXXXed36VJlBQSREQkroYPH87w4cP9LkPiQHMSREREJCKFBBER\nEYlIIUFEREQiUkgQERGRiKIKCWY20czeMrPy0M8qMxvVyPHnmVnQzHaH/hs0s13NL1tERERaWrSr\nGz4Gfgl8EHp8PvCkmX3fOVfWwGvKgcOpuetp8m3xKCIiIvVEFRKcc0/XaZphZj8D+gMNhQTnnNsc\nS3EiIiLin5jnJJhZwMx+CmQCjd3fsoOZrTezDWb2hJkdEes5RUQkMbp3786ll17qaw0ffvghgUCA\nRx99tNHjnn/+eQKBAKtWrapumzBhAocddlhLl5j2og4JZnaUmVUA3wD3AKc7595t4PD3gAuBccDZ\nofOtMjPtyyki4oMFCxYQCAQi/tTeETEQCITd/vhf//oXN954I5988km995w/fz4PP/xwQupvSN1b\nNZsZgYDm5jdXLDsuvgscA3QGxgMLzWxopKDgnHsNeK3qsZkV4w1LXArMjKliERFpFjPjN7/5DTk5\nOWHtRx11VPWvP/zwQ9q0aVP9+J///Cc33ngjJ598Mt27dw973d13302PHj0455xzWrTuaDz00EMk\n412O42Xnzp0JOU/UIcE59x2wNvSw1Mx+CEwBftaU15rZP4DeTTnX1KlT6dSpU1hbQUGBbjMqItJM\no0aNavQ22BkZGWGPnXP1vq0ns9oBJx1ddf75LH/zTZ566imKiorCnisvL4/beeLRFxMA2jblQDML\nAEcB/2nK8XPnzmXx4sVhPwoIIiItr/achD/84Q+cddZZAAwePJhAIECbNm1YtWoVPXr04L333uO5\n556rHrYYMWJE9ft8+eWXTJ48mZ49e9KuXTsOP/xwbr/99nrn27ZtG+eeey6dO3emS5cuXHTRRWzf\nvj3m+uvOSaia3/C73/2O++67j169erHPPvvQv39//vGPf9R7fVlZGePHj2e//fYjMzOTH/7whzzz\nzDMx1xNvZ69bxx0zZlBQUFDvc3Lu3LlxO09UPQlmVggsxVsKmYU3z+B4YETo+YXAJ86560KPf4U3\n3PAB3vDE1cAhwO/jVL+IiMSgvLycL774Iqxtv/32q/517V6DE088kZ///Ofcc889zJw5s/rDt0+f\nPtx9991cfvnl7Lffflx77bU45zjooIMA2LVrF0OGDOHzzz9n4sSJdO/enVdffZWrr76azz//nNtu\nuw3weinGjh3L6tWrufzyy+nTpw+LFi3iggsuiLn3wswivnbBggXs2rWLyy+/HOcct956K2eccUZ1\niAB45513GDJkCIcccgjXXnstmZmZ/OUvf2HcuHE88cQTjBkzJqaa4mmgc1yzeDHMm9ei54l2uOFA\nYCFwEN7+B28DI5xzK0LPdwe+q3X8vsD9QDdgG1ACDGhkoqOIiLQw5xwnnXRSWJuZsXv37ojHH3ro\noQwePJh77rmHk08+mYEDB1Y/d+qpp3LNNdfQrVu3ej29t912Gxs2bOCtt96qnv9wySWXcOCBBzJv\n3jymTZtGt27dePzxx1m1ahV33nknkydPBmDixIkMHTo0jlft2bhxIx988AEdOnQAoFevXvz4xz/m\nueeeq+4BueKKK+jduzerV6+uHra4/PLL6d+/P9dcc01ShAQDMisrW3wYKNp9Ei7ew/PD6jyeBkyL\noS4RkZSxaxe828Jffb73PcjMjM97mRn33HNPiy8RfOyxxzjhhBPIysoK67UYPnw4t99+O6+88gpn\nnnkmzzzzDG3btg1bchkIBJg0aVLYssZ4OOuss6oDAsCQIUNwzrF2rTfVbsuWLbz88svccsstfPnl\nl9XHOecYOXIks2fPZvPmzey///5xrStaDtiZkdHi80RiWd0gIiK1vPsu5Oe37DlKSqCReYZR+8EP\nftDoxMV4eP/99ykrK4v4gWpmfP755wBs2LCB7Oxs2rVrF3ZMnz594l5Tjx49wh7vu+++gDcnoqpm\ngGuvvZZrrrmmwbr9DgmrzBg8blyLn0chQUSkmb73Pe9DvKXPkWqcc4waNYorr7wy4vNVIaChLvOW\nWMLY0KqHqnMFg0EAfvnLXzJ8+PCIx+bm5sa9rmg9kpvL8tmzW/w8CgkiIs2UmRnfb/nJqLFu7Yae\nO/TQQ9m5cyfDhg2L+HyVnJwcVq5cyddffx3Wm/Dee+/FVmwz9OrVC4C99957j3X76bcPPURWVlaL\nn0fbUYmIyB61b98e51zYOH3t5yK1/+QnP+GVV15hxYoV9Z778ssvq7+1jx49mm+++Yb77ruv+vnd\nu3dz9913J3xvhm7dujF48GDuvffe6uGQ2rZs2ZLQehrSvn37hJxHPQkiIq1MLN34xx57LIFAgJtv\nvpktW7bQtm1bTj75ZLp06UJ+fj5/+MMfuOmmm+jVqxfdunXj+OOP55prrmHJkiWccsopXHDBBRx7\n7LHs2LGDt99+m8cff5yNGzfSsWNHTj/9dPr378/06dP58MMPq5dA7tq1q0WvqSH33nsvQ4cO5aij\njuKSSy4hNzeXTZs2sWrVKjZt2sQbb7wRt3MlO4UEEZFWpinfzuvuM3DwwQdz7733cuutt3LxxRez\ne/duXnnlFQYOHMisWbP45JNPuPXWW9mxYwcnnXQSxx9/PJmZmbz66qsUFhby2GOPsWDBAjp16sTh\nhx/O7Nmzq1cZmBlPP/00U6ZMYeHChbRp04bTTjuNO+64g379+sV8TZHu59DQcbXbjzzySN544w1m\nzZrFH//4R7Zt28YBBxzAscceyw033NCketKFJePe1maWB5SUlJS0+OxbEZG6SktLyc/PR/8GSbJp\nyp/NqmOAfOdcaXPOpzkJIiIiEpFCgoiIiESkkCAiIiIRKSSIiIhIRAoJIiIiEpFCgoiIiESkkCAi\nIiIRKSSIiIhIRNpxUUSkAWVlZX6XIBIm0X8mFRJEROro2rUrmZmZTJgwwe9SROrJzMyka9euCTmX\nQoKItCjnXMLv5NdcPXv2pKysLGnu+JcKVq2CK64Ib1u5Emrd+VnipGvXrvTs2TMh51JIEJG4q6io\n4Pbrr2flkiW0r6xkZ0YGg8aOZXphIVlZWX6X1yQ9e/ZM2D/EqWznTgjdp6na8uUwfLg/9Uh8KSSI\nSFxVVFQwfsAAppWVMSsYxAAHLJs/n/ErVrCouDhlgoI0rm4H0ahRsHSpP7VIy9DqBhGJq9uvv55p\nZWWMCgUEAANGBYNMLSvjjhkz/CxP4uCUU+oHBOcUENKRQoKIxNXKJUsYGQxGfG5UMMjKxYsTXJHE\ny/LlXjh49tmatp07vYAg6UkhQUTixjlH+8pKGpqmaEBmZSVOnyop5auvvHAwYkRN29KlXjjIzPSv\nLml5mpMgInFjZuzMyMBBxKDggJ0ZGSm32qE1q/tbNWwYPP+8P7VI4qknQUTiatDYsSwLRP6n5dlA\ngMHjxiW4IonFuHGR5x0oILQuCgkiElfTCwuZ07cvSwMBqgYVHLA0EGBu375cOXu2n+XJHrzwghcO\nliypaauo0LyD1kohQUTiKisri0XFxayeNIkROTmcmp3NiJwcVk+apOWPSezrr71wMGxYTdtTT3nh\noO4+CNJ6aE6CiMRdVlYWs+bNg3nzUnLHxdam7m/P4MHwyiv+1CLJRT0JItKiFBCS1/jxkecdKCBI\nFfUkiIi0Mq+8AkOHhrdt3w4aCZK61JMgItJKfPON13NQOyA88YTXe6CAIJGoJ0FEpBWoO6zwgx/A\n66/7U4ukDvUkiIiksbPOijzvQAFBmkI9CSIiaWjVKhg0KLztyy+hUyd/6pHUFFVPgplNNLO3zKw8\n9LPKzEbt4TVnmlmZmX0Veu0pzStZREQa8u23Xs9B7YDwt795vQcKCBKtaIcbPgZ+CeSHflYAT5pZ\n30gHm9kA4FHgAeD7wBPAE2Z2RMwVi4hIRGbQtm3N42OO8cLBj3/sX02S2qIKCc65p51zzzrnPgj9\nzAB2AP0beMkUYKlzbo5z7j3n3EygFJjUvLJFRKTKeedFnnfw5pv+1CPpI+aJi2YWMLOfAplAcQOH\nDQCeq9O2LNQuIiLNsHq1Fw4WLqxp27pV91mQ+Il64qKZHYUXCtoBFcDpzrl3Gzi8G7CpTtumULuI\niMSgshL23ju87c9/hv/+b3/qkfQVy+qGd4FjgM7AeGChmQ1tJCjUZUCTcu7UqVPpVGemTUFBAQUF\nBVGUKyKSPuoOK/TtC2vW+FNLLHQvj/gqKiqiqKgorK28vDxu72+umf1SZrYc+MA597MIz30E3OGc\n+12ttlnAqc65Yxt5zzygpKSkhLy8vGbVJyKSDi65BH7/+/C2VBlWqKio4Pbrr2flkiW0r6xkZ0YG\ng8aOZXphoe4K2gJKS0vJz88HyHfOlTbnveKxT0IAaNvAc8XAScDvarWdTMNzGEREpJaSEujXL7xt\nyxbYbz9/6olWRUUF4wcMYFpZGbOCwequ5GXz5zN+xQrdPjzJRbtPQqGZDTazQ8zsKDO7GTge+FPo\n+YVmdlOtl8wDTjGzaWbWJ9SLkA/cHaf6RUTS0nffeUMLtQPCww97vQepEhAAbr/+eqaVlTEqFBDA\nG3MeFQwytayMO2bM8LM82YNoVzccCCzEm5fwHN4H/gjn3IrQ892pNSnROVcMFACXAm8CZ+ANNaTQ\nCJqISPNFM7RrBhkZNY9zc71wMGFCCxTWwlYuWcLIYDDic6OCQVYuXpzgiiQaUQ03OOcu3sPzwyK0\nLQIWRVmXiEjKi3Ys/vLL4d57w9tSZd5BJM452ldW0tA0RQMyKys1mTGJ6d4NIiItIJqx+DffhGPr\nTOXevBm6dk142XFlZuzMyMBBxKDggJ0ZGQoISUx3gRQRaQFNGYvfvdsbWqgdEP74R6/3INUDQpVB\nY8eyLBD5o+bZQIDB48YluCKJRrOXQLYELYEUkVQ3PDeX5evXN/gNOlBnu5jsbPjkk4SUllBVPSpT\nawUmhxcQ5vbtq9UNLSCeSyDVkyAiEmeNjcX/grn1AoJz6RkQALKyslhUXMzqSZMYkZPDqdnZjMjJ\nYfWkSQoIKUBzEkQk6aXaxLZIY/HvcBRH807YcZ99BgcemPDyEi4rK4tZ8+bBvHkp93vZ2qknQUSS\nUkVFBTMnT2Z4bi6n9ejB8NxcZk6eTEVFhd+lNUnVWPxuAhguLCBMtkuZOXlKqwgIdSkgpBaFBBFJ\nOlXj2APmz2f5+vU8uXEjy9evZ8D8+YwfMCAlgsL0wkJOCe5mL3ZXt3VlM88E2lB2xCqunD3bx+pE\nmkYhQUSSTqrv0jdyJHTsGD7WPi67O9/P+aHG4iWlaE6CiCSdlUuWMKuRXfrmLF4M8+YluKo9Ky6G\ngQPD29aurdox8WN1tUvKUU+CiCSVaHbpSxbBoLffQe2AcOWV3qqF3FzvsQKCpCL1JEhS0IxnqZJq\nu/RFKiOJ8otIs6gnQXyT6rPXpeWkwi59Y8fWDwjOKSBIelFPgvhC95iXxkwvLGT8ihW4hnbp83Fl\nwP/9H/zwh+Ft778PvXv7U49IS1JPgvgi1WevS8tKxl36quYd1A4IV1zh9RwoIEi60r0bxBd72td+\nRE4Oy9etS3RZkqT8nrOieQeSSnTvBklpqTh7XfzlV0D48Y/rB4RgUAFBWg+FBEm42rPXI0m22evS\n+vzjH144WLSopu3dd71woD+W0pooJIgvUmH2urQ+VSGg9ijnZZd57X36+FeXiF+0ukF8kcyz16V1\n0rwDkfrUkyC+SMbZ69I65eVp3oFIQ9STIL7RPebFT5Hus/B//wf9+vlTj0gyUkiQpKCAIIniHNSd\nDjNgAKxa5U89IslMIUFEWg3NOxCJjuYkiEjaGzhQ8w5EYqGQICJp6403vHBQXFzTtmqV9jsQaSoN\nN4i0Quk+UTTSvIO8PCgp8acekVSlkCDSSlRUVHD79dezcskS2ldWsjMjg0FjxzK9sDCtlpxq3oFI\n/Gi4QaQVqLo194D581m+fj1PbtzI8vXrGTB/PuMHDKCiosLvEptt2DDNOxCJN4UEkVYgnW/NXXWf\nhRdeqGl7+WXNOxCJB4UEkVZg5ZIljAwGIz43Khhk5eLFCa6o+SLdZ6FvX699yBD/6hJJJ5qTIJLm\nork1d6pMZtS8A5HEUE+CSJpLp1tzjx5dPyDs3h3/gOCUOEQAhQSRViHVb839zjteOFi6tKbt+ecj\nL3WMVUVFBTMnT2Z4bi6n9ejB8NxcZk6enBaTOkViFdVfLzO71sxeN7PtZrbJzP7XzA7fw2vOM7Og\nme0O/TdoZruaV7aIRGN6YSFz+vZlaSBQ3aPggKWhW3NfmcS35jaDo4+ueZyT44WDYcPid47WsPpD\nJBbRZvArjoiJAAAeT0lEQVQhwF3AccBwIAP4u5nts4fXlQPdav0cEuV5RaQZUvHW3Gb1hxacg3Xr\n4n+udF79IdIc1pyxNzPrCnwODHXOvdrAMecBc51zXaJ43zygpKSkhLzaU5dFJC6SeZLi6afDE0+E\nt333HbRp03LnHJ6by/L16yNO7nTAiJwclrdEOhFpAaWlpeTn5wPkO+dKm/NezR3N64z3d2jrHo7r\nYGbrzWyDmT1hZkc087wi0gzJGBDeesvrOagdEJYt83oPWjIgRLP6Q6S1iXkJpHn/ytwJvOqcW9PI\noe8BFwJvA52Aq4BVZnakc25jrOcXkfRRN7McdBB8+mmizl2z+qOhnoRUWf0hEm/N2SfhHuAIYFBj\nBznnXgNeq3psZsVAGXApMLOx106dOpVOnTqFtRUUFFBQUBBjySKSTJJlv4NBY8eybP58RkXYcCoV\nVn9I61VUVERRUVFYW3l5edzeP6Y5CWZ2NzAWGOKc2xDD6/8KVDrnzm7gec1JEElj/fvD6tXhbZWV\nsJdP27tVrW6YWmvyosMLCHP79k3ayZ0ikfg6JyEUEE4FTowxIASAo4D/RPtakXTVWsa7q/Y7qB0Q\nHn3U6z3wKyBAaq7+EEmEqP5amtk9QAEwDthpZgeGnip3zn0dOmYBsNE5d13o8a/whhs+wJvoeDXe\nEsjfx+UKRFJUa7l1c5VkGVpoSFZWFrPmzYN585J69YdIIkWb3Sfi9cK9WKf9AmBh6Nc9gN21ntsX\nuB9vf4RtQAkwwDn3brTFiqSLqu7taWVlzKrVvb1s/nzGr1iRVt9ekz0cRKKAIOKJarjBORdwzrWJ\n8LOw1jHDnHMX1no8zTmX65zbxzl3sHNurHPu7XhehEiqaQ2b95xwQv2A8M03yR8QRKSG7t0g4oN0\nvHVzlXff9cLBSy/VtP3xj1442Htv/+oSkejpVtEiCZaOt26ukopDCyLSMIUEkQRLx817FA5E0pOG\nG0R8kOq3bq5yyin1A8LXXysgiKQLhQQRH6TyrZsB3n/fCwfPPlvTdt99Xjho29a/ukQkvjTcIOKD\nqs177pgxgzmLF5NZWcmujAwGjRvHotmzk3r5o4YWRFoPhQQRn6Ta5j0KByKtj4YbRJJAMgeEM86o\nHxB27VJAEGkNFBJEJKJ167xw8L//W9N2111eONhnH//qEpHE0XCDiNSjoQURAYUEEalF4UBEatNw\ng4hw1ln1A8KOHQoIIq2dQoJIK7ZhgxcOiopq2m6/3QsH7dv7V5eIJAcNN4i0UhpaEJE9UUgQaWUU\nDkSkqTTcINJKzJhRPyBs366AICINU0gQSXOff+6Fg8LCmrabbvLCQRLv/iwiSUDDDSJpTEMLItIc\nCgkiaUjhQETiQcMNImnk17/WvAMRiR/1JIikgS1bYP/9w9vuugsmTfKnHhFJDwoJIilOQwsi0lIU\nEkRSlMKBiLQ0zUkQSTG33FI/IHz5pQKCiMSfehJEUsS2bdClS3jbnDkwdao/9YhI+lNIEEkBGloQ\nET8oJIgkgHMOi/RJvwcKByLiJ81JEGkhFRUVzJw8meG5uZzWowfDc3OZOXkyFRUVe3ztnDn1A8LW\nrQoIIpJY6kkQaQEVFRWMHzCAaWVlzAoGMcABy+bPZ/yKFSwqLiYrwo0Tysuhc+fwtltugV/+MiFl\ni4iEUUgQaQG3X38908rKGBUMVrcZMCoYxJWVcceMGcyaNy/sNRpaEJFko+EGkRawcskSRtYKCLWN\nCgZZuXhx9eP27esHBOcUEETEfwoJInHmnKN9ZSUNTVM0ILOykr/8xWEGu3bVPLd5s8KBiCQPhQSR\nODMzdmZk0NBn/U72YfHGT/jpT2tixP33e+Gga9fE1Cgi0hRRhQQzu9bMXjez7Wa2ycz+18wOb8Lr\nzjSzMjP7yszeMrNTYi9ZJPkNGjuWZYH6f70MRwd2hbU5B5dckqjKRESaLtqehCHAXcBxwHAgA/i7\nme3T0AvMbADwKPAA8H3gCeAJMzsipopFUsD0wkLm9O3L0kAAB2TzCVanb0HzDkQk2UUVEpxzo51z\nDzvnypxz7wDnAz2B/EZeNgVY6pyb45x7zzk3EygFdBNbSVtZWVksKi7moVH3E8DxKdnVz61fX6Fw\nICIpoblzEjrjLf/e2sgxA4Dn6rQtC7WLpKWvvoKOHbP46zMXVbfNn+/1HBxySP39EUREklHM+ySY\nt8fsncCrzrk1jRzaDdhUp21TqF0k7Wi/AxFJF83pSbgHOAL4aQyvrdqATiRt9Oql/Q5EJL3E1JNg\nZncDo4Ehzrn/7OHwz4AD67QdQP3ehXqmTp1Kp06dwtoKCgooKCiIolqRlvXUUzB2bHjb5s1azigi\nLa+oqIiioqKwtvLy8ri9v7kov+aEAsKpwPHOubVNOP7PwD7OuVNrta0E3nLOXd7Aa/KAkpKSEvLy\n8qKqTyRRvv0W2rYNb7vzTpgyxZ96REQASktLyc/PB8h3zpU2572i6kkws3uAAmAcsNPMqnoIyp1z\nX4eOWQBsdM5dF3puHvCSmU0Dng69Ph/QynBJWZp3ICKtQbRzEiYCHYEXgU9r/fyk1jE9qDUp0TlX\njBcMLgXeBM4ATt3DZEeRpHTeeZp3ICKtR1Q9Cc65PYYK59ywCG2LgEXRnEskmbz+Ohx3XHjb1q2w\n777+1CMikgi6d4NIIyorvZ6D2gGhqMjrOVBAEJF0F/M+CSLpru6wwve+B2Vl/tQiIuIH9SSI1HHJ\nJZHnHSggiEhro54EkZCSEujXL7xtyxbYbz9/6hER8Zt6EqTV++47r+egdkB4+GGv90ABQURaM/Uk\nSKtWd1ghNxfW7nGLMBGR1kE9CdIqXX555HkHCggiIjXUkyCtyptvwrHHhrd9/jnsv78/9YiIJDP1\nJEirsHu313NQOyD88Y9e74ECgohIZOpJkLRXd1ghOxs++cSfWkREUol6EiRt/eIXkecdKCCIiDSN\nehIk7bzzDhx9dHjbZ5/BgQdGPl5ERCJTT4KkjWDQ6zmoHRAeeMDrPVBAEBGJnnoSJC3UHVbo2hU2\nb/anFhGRdKGeBElpV18ded6BAoKISPOpJ0FS0po1cOSR4W2ffgoHHeRPPSIi6Ug9CSnAOed3CUmj\nat5B7YBwzz1e74ECgohIfKknIUlVVFRw+/XXs3LJEtpXVrIzI4NBY8cyvbCQrKwsv8vzRd1hhaws\n2L7dn1qkZTjnsLq/0SLiG/UkJKGKigrGDxjAgPnzWb5+PU9u3Mjy9esZMH8+4wcMoKKiwu8SE+q6\n6yLPO1BASA8VFRXMnDyZ4bm5nNajB8Nzc5k5eXKr+3MukowUEpLQ7ddfz7SyMkYFg1R9NhowKhhk\nalkZd8yY4Wd5CfPee144uPnmmrZPPvECgqQHBWKR5KaQkIRWLlnCyGAw4nOjgkFWLl6c4IoSyzkv\nHHzvezVt8+Z57dnZ/tUl8adALJLcFBKSjHOO9pWVNDQqa0BmZWXaTmY0g0CtP5Vt23rhYPJk/2ra\nk3T9vUiE1h6IRZKdQkKSMTN2ZmTQ0MeOA3ZmZKTd5K6ZM+vPOwgG4euv/alnTzSO3nytPRCLpAKF\nhCQ0aOxYlgUi/9Y8GwgweNy4BFfUcj74wAsHv/51TduGDTVDDslI4+jx0VoDsUgqUUhIQtMLC5nT\nty9LA4Hqf0AdsDQQYG7fvlw5e7af5cVFVQg47LCatjvu8Np79PCvrqbQOHr8tKZALJKKFBKSUFZW\nFouKi1k9aRIjcnI4NTubETk5rJ40iUXFxSm/T0LdeQfghYNp0/ypJ1oaR4+f1hCIRVKZNlNKUllZ\nWcyaNw/mzUubDWYKC6Hul+yqHRRTRTTj6Onwe9bSqgLxHTNmMGfxYjIrK9mVkcGgceNYNHt2ygdi\nkVSnkJACUvHDpvaH5Nq10KtX+PPr18MhhyS+ruaqPY4e6XdF4+jRS8dALJIuNNwgcVN3xv9JObmY\nhQeEW27xhhZSMSBU0Th6y1FAEEku6kmQuKia8T+trIxZwSCBCHPW02Ul2/TCQsavWIGrNXnR4QWE\nuX37skjj6CKSJtSTIHFRNeP/m+CYegHhaWvDzMlTfKos/tJ9YqmISBVLxo1KzCwPKCkpKSEvL8/v\ncqQJhvbM55WPS8LaPuRQDmUdDhiRk8Pydev8Ka6FaRxdRJJJaWkp+fn5APnOudLmvJd6EqRZqvY7\nqB0QHmM8DuNQvFCQ7jvnKSCISLqKOiSY2RAzW2xmG80saGaNztIys+NDx9X+2W1mB8RetiSDn/wk\nfL+D0TyNwxjP42HHaca/iEhqiqUnoT3wJvBzaHBH1boccBjQLfRzkHPu8xjOLUngqae83oO//a2m\n7VeTJnNFIHJe1Ix/EZHUFPXqBufcs8CzABbdV8PNzrnt0Z5PkseWLbD//uFtn34KBx0EFRWFjH9B\nM/5FRNJJouYkGPCmmX1qZn83s4EJOq/EQdW8g9oB4S9/8doPOsh7rBn/IiLpJxH7JPwHuAx4A2gL\nXAK8aGY/dM69mYDzSzOcfTY8+mjN4xEjYNmyyMdq5zwRkfTS4iHBOfdv4N+1ml4zs17AVOC8lj6/\nxGbpUhg9OrwtmvssKCCIiKQ+v3ZcfB0YtKeDpk6dSqdOncLaCgoKKCgoaKm6Wr2tW2G//cLbPvkE\nsrP9qUdERBpWVFREUVFRWFt5eXnc3r9ZmymZWRA4zTkX1b1xzezvwHbn3I8beF6bKfmg7pf/Rx6B\ns87ypxYREYlNPDdTironwczaA72puQneoWZ2DLDVOfexmd0MHOycOy90/BRgHfAvoB3enIQTgZOb\nU7jEz3nnwcKFNY+HDYPnn/evHhERSQ6xDDf0A17AW+HmgDtC7QuAC/H2QehR6/i9Q8ccDOwC3gZO\ncs69HGPNEid//zuMHBneFs28AxERSW+x7JPwEo0snXTOXVDn8W+B30ZfmrSUbdugS5fwtg0boEeP\nyMeLiEjrpHs3tDJm4QFh4UJvvwMFBBERqUshoZW4+OLwYYTBg71wcM45/tUkIiLJza8lkJIgzz8P\nw4eHt+3eHX5jJhERkUgUEtJUeTl07hze9tFH0LOnP/WIiEjq0ffJNGQWHhAefNAbWlBAEBGRaCgk\npJHLLgufd3DccV44uOCChl8jIiLSEA03pIEXXvA2QKpN8w5ERKS5FBJS2PbtUOfWFqxdC7m5/tQj\nIiLpRd81fdKce2aAN6xQOyA88IA3tKCAICIi8aKQkEAVFRXMnDyZ4bm5nNajB8Nzc5k5eTIVFRVN\nfo9Jk8LnHeTleeHg4otboGAREWnVNNyQIBUVFYwfMIBpZWXMCgYxvBtfLJs/n/ErVrCouJisrKwG\nX//yy3D88eFt330Hbdq0aNkiItKKqSchQW6//nqmlZUxKhQQwLuN5qhgkKllZdwxY0bE11VUeD0H\ntQPCBx94vQcKCCIi0pIUEhJk5ZIljAwGIz43Khhk5eLF9dr79oWOHWse33uvFw569WqpKkVERGoo\nJCSAc472lZU0dAdmAzIrK6snM86f7/UevPuu9/xRR3nhYOLEhJQrIiICaE5CQpgZOzMycBAxKDhg\nZ0YG//yncfTR4c9p3oGIiPglZXsSmruEMNEGjR3LsgZ2N3rSMnl+/bqwgPDxx5p3ICIi/kqpkBCP\nJYR+mV5YyJy+fVkaCFAVbxxwFC9zuttZfdzjj3vhoHt3X8oUERGpljIhoWoJ4YD581m+fj1PbtzI\n8vXrGTB/PuMHDEj6oJCVlcWi4mJWT5rEiJwcju50DQEc/2IIAOee64WD00/3uVAREZGQlAkJsS4h\nTCZZWVlcfNU8nlu/jnfKb65u/+47WLDAx8JEREQiSJmQEMsSwmRSWQn9+0OPHjVtGzZo3oGIiCSv\nlAgJ0S4hTDY33gh77w2rV3uPi4u9cFA7MIiIiCSblFgC2dQlhGYNxQh/PPccnHxyzeM774QpU/yr\nR0REJBop0ZMAjS8hfDYQYPC4cQmuqGEbN3qbIVUFhNGjvXkHCggiIpJKUiYkNLSEcGkgwNy+fbly\n9mw/ywO8eQcDB4YvX/z8c3j6ac07EBGR1JMyIaHuEsJTs7MZkZPD6kmT9ngHxUSYPdubd1Bc7D1e\ntcqbd7D//r6WJSIiErOUmJNQJSsri1nz5sG8eTjnkmIOwooVcNJJNY/nzIGpU/2rR0REJF5SKiTU\n5ndA+PRTyM6ueTxypIYVREQkvaRsSPDLd9/BiSfCq6/WtG3aBAcc4F9NIiIiLSFl5iQkg5tvhoyM\nmoDwyivevAMFBBERSUfqSWiCl16CE06oefzb38L06b6VIyIikhAKCY347DM46KCax8OHw7PPat6B\niIi0DgoJEXz3nbcR0osv1rR99hkceKBvJYmIiCSc5iTUceut3ryDqoDw8svevAMFBBERaW3UkxDy\n8stw/PE1j2+9Fa6+2r96RERE/BZ1T4KZDTGzxWa20cyCZrbHmyaY2QlmVmJmX5vZv83svNjKjb9N\nm7z7LFQFhBNP9LZXVkAQEZHWLpbhhvbAm8DPgT3em9nMcoCngOeBY4B5wO/N7ORGXtbidu/2JiJ2\n61bT9p//eDso7qX+FRERkeiHG5xzzwLPAljTtj38GbDWOVf13fw9MxsMTAWWR3v+ePjtb8N7Cl58\nMXyoQURERBIzcbE/8FydtmXAgAScO8zKld7QQlVAKCz0JiUqIIiIiNSXiI71bsCmOm2bgI5m1tY5\n901LF/D55+GrE4YM0bCCiIjInvj1MVk1TNHonIapU6fSqVOnsLaCggIKCgqadJLdu2H0aPj732va\nPv00fIMkERGRVFVUVERRUVFYW3l5edzePxEh4TOg7i4DBwDbnXPfNvbCuXPnkpeXF9NJ586FadNq\nHq9Y4a1cEBERSReRvjiXlpaSn58fl/dPxJyEYuCkOm0jQu1xt2qVN++gKiD85jfevAMFBBERkehE\n3ZNgZu2B3tQMGRxqZscAW51zH5vZzcDBzrmqvRD+B5hkZrcCD+IFhh8Do5tdfS2bN4ffjXHQIHjh\nBW/3RBEREYleLD0J/YB/ACV4cwruAEqBG0PPdwN6VB3snFsP/AgYjre/wlTgIudc3RUPMamad1A7\nIHzyiXc7ZwUEERGR2MWyT8JLNBIunHMXNPCa+AyQ1PK738GUKTWPly/3NkgSERGR5kvJRYCffgrZ\n2TWPZ82CmTN9K0dERCQtpWRIeOkl77/9+3s3ZtKwgoiISPyl5K2iCwq8FQvFxQoIIiIiLSUlQ4KI\niIi0PIUEERERiUghQURERCJSSBAREZGIFBJEREQkIoUEERERiUghQURERCJSSBAREZGIFBJEREQk\nIoUEERERiUghQURERCJSSBAREZGIFBJEREQkIoUEERERiUghQURERCJSSBAREZGIFBJEREQkIoUE\nERERiUghQURERCJSSBAREZGIFBJEREQkIoUEERERiUghQURERCJSSBAREZGIFBJEREQkIoUEERER\niUghQURERCJSSBAREZGIFBJEREQkIoWEBCgqKvK7hLjS9SSvdLoW0PUks3S6Fki/64mXmEKCmf3c\nzNaZ2Vdm9pqZ/aCRY88zs6CZ7Q79N2hmu2IvOfWk2x8+XU/ySqdrAV1PMkuna4H0u554iTokmNl/\nA3cAM4FjgbeAZWbWtZGXlQPdav0cEn2pIiIikkix9CRMBe5zzi10zr0LTAR2ARc28hrnnNvsnPs8\n9LM5lmJFREQkcaIKCWaWAeQDz1e1Oecc8BwwoJGXdjCz9Wa2wcyeMLMjYqpWREREEmavKI/vCrQB\nNtVp3wT0aeA17+H1MrwNdAKuAlaZ2ZHOuY0NvKYdQFlZWZTlJafy8nJKS0v9LiNudD3JK52uBXQ9\nySydrgXS63pqfXa2a+57mdcR0MSDzQ4CNgIDnHOra7XfBgx2zg1swnvsBZQBjzrnZjZwzFnAI00u\nTEREROo62zn3aHPeINqehC3AbuDAOu0HUL93ISLn3Hdm9g+gdyOHLQPOBtYDX0dZo4iISGvWDsjB\n+yxtlqhCgnOu0sxKgJOAxQBmZqHHv2vKe5hZADgKeKaR83wBNCv9iIiItGKr4vEm0fYkAMwBFoTC\nwut4qx0ygYcAzGwh8Ilz7rrQ418BrwEfAJ2Bq/GWQP6+ucWLiIhIy4k6JDjn/hraE+HXeMMObwIj\nay1r7A58V+sl+wL34+2PsA0owZvT8G5zChcREZGWFdXERREREWk9dO8GERERiUghQURERCJKmpBg\nZtea2etmtt3MNpnZ/5rZ4X7XFSszm2hmb5lZeehnlZmN8ruueAj9XgXNbI7ftcTCzGbWutlY1c8a\nv+tqDjM72MweNrMtZrYr9Gcvz++6YhG6eVzd35+gmd3ld23RMrOAmf3GzNaGfl8+MLMZftfVHGbW\nwczuDO2iu8vMXjWzfn7X1RRmNsTMFpvZxtCfqXERjvm1mX0aurblZtbYcn3f7OlazOx0M3vWzDaH\nnj86lvMkTUgAhgB3AccBw4EM4O9mto+vVcXuY+CXeNtY5wMrgCfNrK+vVTVT6I6fl+Dd2CuV/RNv\n4m3VTccG+1tO7MysM7AS+AYYCfQFrsSbKJyK+hF+Q7iTAQf81c+iYnQNcBlwOfA9vNVdV5vZJF+r\nap4/4C17PxtvOfty4LnQZnvJrj3eZPuf4/2ZCmNmvwQm4f2e/RDYiXcDw70TWWQTNXotoedfxfsc\ninnyYdJOXAytoPgcGOqce9XveuLBzL4Apjvn/uh3LbEwsw54q1N+BvwK+Idzbpq/VUXPzGYCpzrn\nUvKbdl1mdgveiqHj/a6lJZjZncBo51zK9Sya2RLgM+fcJbXaHgN2OefO9a+y2JhZO6ACGOuce7ZW\n+xvAM865G3wrLkpmFgROc84trtX2KfBb59zc0OOOeBsFnuecS9qQGulaaj13CLAO+L5z7u1o3zuZ\nehLq6oyXfrb6XUhzhbocf4q3n0Sx3/U0w3xgiXNuhd+FxMFhoW66D83sT2bWw++CmmEs8IaZ/TU0\nVFdqZhf7XVQ8hG4qdzbet9dUtAo4ycwOAzCzY4BBNLKZXJLbC+/+Pd/Uaf+KFO6NAzCzXLyeq9o3\nMNwOrKbxGximtVg2U2pxoV0c7wRedc6l7FixmR2FFwqq0vfpqbo/RCjkfB+vKzjVvQacj3fzsYOA\nWcDLZnaUc26nj3XF6lC83p07gEK8IbvfmdnXzrk/+VpZ852Od2O4BX4XEqNbgI7Au2a2G++L2fXO\nuT/7W1ZsnHM7zKwY+JWZvYv3LfssvA/R930trvm64X0xjXQDw26JLyc5JGVIAO4BjsBL3KnsXeAY\nvF6R8cBCMxuaakHBzLrjhbaTnXOVftfTXM652vuZ/9PMXgc+An4CpOJQUAB43Tn3q9Djt8zsSLzg\nkOoh4UJgqXPuM78LidF/432I/hRYgxe055nZp865h32tLHYTgAfxbvb3HVCKt41+WgzfRWA0Y0w/\n1SXdcIOZ3Q2MBk5wzv3H73qawzn3nXNurXOu1Dl3Pd5kvyl+1xWDfGB/oMTMKs2sEjgemGJm34Z6\nflKWc64c+DeN33Qsmf0H786qtZUBPX2oJW7MrCfeJOYH/K6lGW4DbnbO/c059y/n3CPAXOBan+uK\nmXNunXPuRLyJcT2cc/2BvfHGvVPZZ3iBIOYbGKajpAoJoYBwKnCic26D3/W0gADQ1u8iYvAc8F94\n34KOCf28gfct9RiXrLNfmyg0IbMX3odtKloJ9KnT1gevdySVXYj3j3Oqjt+DNw+p7t+PIEn2b28s\nnHNfOec2mdm+eKtqnvC7puZwzq3DCwonVbWFJi4eR5xuluSjmP+NTprhBjO7BygAxgE7zawqzZU7\n51LudtFmVggsxVsKmYU3+ep4YISfdcUiNE4fNjfEzHYCXzjn6n6DTXpm9ltgCd6HaDZwI163aZGf\ndTXDXGClmV2Lt0zwOOBivKWqKSnUO3U+8JBzLuhzOc2xBLjezD4G/oXXJT+VFL7BnZmNwPvG/R5w\nGF5vSRmhm/wlMzNrj9djWNX7eWhoMulW59zHeMOqM8zsA2A98BvgE+BJH8pt1J6uJRTeeuL9G2fA\n90J/rz5zzjW9Z8Q5lxQ/eOl6d4Sfc/2uLcbr+T2wFm/W72fA34FhftcVx+tbAczxu44Yay/C+4v/\nFbABbzw11++6mnlNo4G3gV14H0YX+l1TM6/n5NDf/95+19LM62iPd+fcdXhr7t/HC6V7+V1bM67p\nTLy7+n6FNy9hHpDld11NrP34Bj5rHqx1zCzg09DfpWXJ+mdwT9cCnNfA8zdEc56k3SdBRERE/JXy\n42IiIiLSMhQSREREJCKFBBEREYlIIUFEREQiUkgQERGRiBQSREREJCKFBBEREYlIIUFEREQiUkgQ\nERGRiBQSREREJCKFBBEREYno/wEw+6OEm8vrYQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f7764e69f90>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#Initialize variables\n",
    "init = tf.global_variables_initializer()\n",
    "\n",
    "#Perform learning\n",
    "with tf.Session() as sess:\n",
    "    sess.run(init)\n",
    "    for epoch in range(training_epochs):\n",
    "        for (x_, y_) in zip(train_X, train_Y):\n",
    "            sess.run(optimizer, {x:x_, y:y_})\n",
    "\n",
    "    #Display logs per epoch step\n",
    "        if (epoch+1) % display_step == 0:\n",
    "            c = sess.run(loss_, feed_dict={x: train_X, y:train_Y})\n",
    "            print \"Epoch:\", '%04d' % (epoch+1), \"cost=\", \"{:.9f}\".format(c), \\\n",
    "                \"W=\", sess.run(W), \"b=\", sess.run(b)\n",
    "\n",
    "    print \"Optimization Finished!\"\n",
    "    training_cost = sess.run(loss_, feed_dict={x: train_X, y: train_Y})\n",
    "    print \"Training cost=\", training_cost, \"W=\", sess.run(W), \"b=\", sess.run(b), '\\n'\n",
    "\n",
    "     #Graphic display\n",
    "    plt.plot(train_X, train_Y, 'ro', label='Original data')\n",
    "    plt.plot(train_X, sess.run(W) * train_X + sess.run(b), label='Fitted line')\n",
    "    plt.legend()\n",
    "    plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "tensorflow",
   "language": "python",
   "name": "tensorflow"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}