2.2.1 Supervised Learning - ConvNet HandsOn Part I.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# ConvNet HandsOn with Keras"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Problem Definition\n",
    "\n",
    "*Recognize handwritten digits*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Data\n",
    "\n",
    "The MNIST database ([link](http://yann.lecun.com/exdb/mnist)) has a database of handwritten digits. \n",
    "\n",
    "The training set has $60,000$ samples. \n",
    "The test set has $10,000$ samples.\n",
    "\n",
    "The digits are size-normalized and centered in a fixed-size image. \n",
    "\n",
    "The data page has description on how the data was collected. It also has reports the benchmark of various algorithms on the test dataset. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Load the data\n",
    "\n",
    "The data is available in the repo's `data` folder. Let's load that using the `keras` library. \n",
    "\n",
    "For now, let's load the data and see how it looks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using Theano backend.\n",
      "Using gpu device 0: GeForce GTX 760 (CNMeM is enabled with initial size: 90.0% of memory, cuDNN 4007)\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import keras\n",
    "from keras.datasets import mnist"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "!mkdir -p $HOME/.keras/datasets/euroscipy_2016_dl-keras/data/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# Set the full path to mnist.pkl.gz\n",
    "path_to_dataset = \"euroscipy_2016_dl-keras/data/mnist.pkl.gz\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "code_folding": [],
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading data from https://s3.amazonaws.com/img-datasets/mnist.pkl.gz\n",
      "15024128/15296311 [============================>.] - ETA: 0s"
     ]
    }
   ],
   "source": [
    "# Load the datasets\n",
    "(X_train, y_train), (X_test, y_test) = mnist.load_data(path_to_dataset)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Basic data analysis on the dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# What is the type of X_train?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# What is the type of y_train?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# Find number of observations in training data\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# Find number of observations in test data\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[[0, 0, 0, ..., 0, 0, 0],\n",
       "        [0, 0, 0, ..., 0, 0, 0],\n",
       "        [0, 0, 0, ..., 0, 0, 0],\n",
       "        ..., \n",
       "        [0, 0, 0, ..., 0, 0, 0],\n",
       "        [0, 0, 0, ..., 0, 0, 0],\n",
       "        [0, 0, 0, ..., 0, 0, 0]],\n",
       "\n",
       "       [[0, 0, 0, ..., 0, 0, 0],\n",
       "        [0, 0, 0, ..., 0, 0, 0],\n",
       "        [0, 0, 0, ..., 0, 0, 0],\n",
       "        ..., \n",
       "        [0, 0, 0, ..., 0, 0, 0],\n",
       "        [0, 0, 0, ..., 0, 0, 0],\n",
       "        [0, 0, 0, ..., 0, 0, 0]]], dtype=uint8)"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Display first 2 records of X_train\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([5, 0, 4, 1, 9, 2, 1, 3, 1, 4], dtype=uint8)"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Display the first 10 records of y_train\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=uint8),\n",
       " array([5923, 6742, 5958, 6131, 5842, 5421, 5918, 6265, 5851, 5949]))"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Find the number of observations for each digit in the y_train dataset \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=uint8),\n",
       " array([ 980, 1135, 1032, 1010,  982,  892,  958, 1028,  974, 1009]))"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Find the number of observations for each digit in the y_test dataset \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(60000, 28, 28)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# What is the dimension of X_train?. What does that mean?\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Display Images\n",
    "\n",
    "Let's now display some of the images and see how they look\n",
    "\n",
    "We will be using `matplotlib` library for displaying the image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "from matplotlib import pyplot\n",
    "import matplotlib as mpl\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# Displaying the first training data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP4AAAD7CAYAAABKWyniAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADnRJREFUeJzt3X2MVfWdx/HPV4kPgEEkyxCBBauxazYR0rXGDRpv02xL\n1iZIVUp9iLKEGAUkNj4VY2ZCNCmbKIEoRtEaqNW2S3Sxm2y1xEQ0SmsK7CJQ2bgOD7My4Aa0Y6KB\nne/+cY/jgJffucy5Dwe+71cy8c75zJ3z9ehnzr33dx/M3QUgltPaPQCA1qP4QEAUHwiI4gMBUXwg\nIIoPBNSy4pvZdDP7s5ntNLP7W7XfeplZt5n9h5ltNrM/lmCeZ82s18z+c9C20Wb2mpm9b2avmtmo\nks3XaWZ7zWxT9jW9jfNNMLPXzWybmW01s7uy7aU4hjXmW5htb8kxtFas45vZaZJ2SvqupP+R9K6k\n2e7+56bvvE5m9t+S/s7dD7Z7Fkkysysl9Ula4+6XZtuWSvpfd//n7I/naHd/oETzdUr6i7s/1o6Z\nBjOzcZLGufsWMxsp6U+SZkiaoxIcw8R8P1ILjmGrzviXS/ovd9/l7ocl/UrVf8kyMZXoro+7vyXp\n2D9CMyStzi6vlnRtS4ca5DjzSdXj2Hbuvs/dt2SX+yTtkDRBJTmGx5lvfBY3/Ri26n/08ZL2DPp+\nr776lywLl/R7M3vXzOa1e5jjGOvuvVL1fxxJY9s8Ty0LzGyLmT3Tzrsig5nZZElTJW2U1FG2Yzho\nvj9km5p+DEtzhiuBae7+LUn/KGl+dlO27Mr2fOuVkr7h7lMl7ZNUhpv8IyWtlbQoO7Mee8zaegxr\nzNeSY9iq4vdI+utB30/ItpWGu3+U/fOApJdVvXtSNr1m1iEN3Efc3+Z5juLuB/yrB41WSfp2O+cx\ns2GqluoX7r4u21yaY1hrvlYdw1YV/11JF5nZJDM7Q9JsSa+0aN+5zGx49pdXZjZC0vckvdfeqSRV\n7+sNvr/3iqTbssu3Slp37BVa7Kj5siJ96Ydq/zH8uaTt7r580LYyHcOvzdeqY9iSR/Wl6nKepOWq\n/rF51t1/1pId18HMLlD1LO+Shkn6ZbvnM7MXJFUkjZHUK6lT0r9K+hdJEyXtkjTL3Q+VaL7vqHpf\ntV9St6Tbv7w/3Yb5pknaIGmrqv9dXdJiSX+U9Bu1+Rgm5rtRLTiGLSs+gPLgwT0gIIoPBETxgYAo\nPhBQoeKX/YU3AGob8qP69b7wxsxYNgDaxN1rPu+/yBm/7hfeuPvAV2dn51Hfl+2L+U7d+co8WzPm\nSylS/JPhhTcAauDBPSCgYQWuW/cLb7q6ugYun3vuuQV22XyVSqXdIyQx39CVeTaptfMVeXDvdEnv\nq/rg3keqPgf6x+6+45if86HuA8DQmZn8OA/uDfmM7+7/Z2YLJL2mr154syPnagBKoOkv0uGMD7RH\n6ozPg3tAQBQfCIjiAwFRfCAgig8ERPGBgCg+EBDFBwKi+EBAFB8IiOIDAVF8ICCKDwRE8YGAKD4Q\nEMUHAqL4QEAUHwiI4gMBUXwgIIoPBETxgYAoPhBQkY/QArRnz55kvnz58mS+bNmyZH733Xcn80WL\nFiXziRMnJvOoOOMDAVF8ICCKDwRE8YGAKD4QEMUHAqL4QEBW5LPrzaxb0ieS+iUddvfLa/yMF9kH\n2qunpyeZT5kyJZkfOnSokeN8zejRo5P5gQMHmrr/MjMzubvVyoo+gadfUsXdDxb8PQBaqOhNfWvA\n7wDQYkVL65J+b2bvmtm8RgwEoPmK3tSf5u4fmdlfqfoHYIe7v3XsD3V1dQ1crlQqqlQqBXcLoIhC\nxXf3j7J/HjCzlyVdLilZfADtN+Sb+mY23MxGZpdHSPqepPcaNRiA5ilyxu+Q9LKZefZ7funurzVm\nLADNVGgdv64dsI5fart27UrmeY/H7N69O5mb1VxGHjBq1KhkfuaZZybz/fv3J/OdO3cm80mTJiXz\n008/PZmXWWodn6U4ICCKDwRE8YGAKD4QEMUHAqL4QEAUHwiIdfyT3OHDh5N53jr99OnTk3l3d3cy\nz/tvm7eOf/XVVyfzRx55JJlfeeWVyTxvvqeffjqZz507N5mXGev4AI5C8YGAKD4QEMUHAqL4QEAU\nHwiI4gMBFX3PPbTZvffem8wff/zxFk0yNG+88UYy/+yzz5L5zJkzk/lLL72UzDdv3pzMT1Wc8YGA\nKD4QEMUHAqL4QEAUHwiI4gMBUXwgINbxS27Pnj3J/Pnnn0/mRd8LIW+d/LrrrkvmN998czKfOHFi\nMr/kkkuS+f3335/M165dm8yjvlcEZ3wgIIoPBETxgYAoPhAQxQcCovhAQBQfCCj3ffXN7FlJP5DU\n6+6XZttGS/q1pEmSuiXNcvdPjnN93lc/oaenJ5lPmTIlmR86dKjQ/m+66aZkvmrVqmS+ffv2ZL5p\n06ZkPnv27GQ+fPjwZJ4n7/PtR4wYkcy3bduWzPOeh9BORd9X/zlJ3z9m2wOS1rv7NyW9LumnxUYE\n0Eq5xXf3tyQdPGbzDEmrs8urJV3b4LkANNFQ7+OPdfdeSXL3fZLGNm4kAM3WqOfqJ+/Ed3V1DVyu\nVCqqVCoN2i2AoRhq8XvNrMPde81snKT9qR8eXHwA7VfvTX3Lvr70iqTbssu3SlrXwJkANFlu8c3s\nBUlvS7rYzHab2RxJP5P0D2b2vqTvZt8DOEnkruMX3kHwdfyPP/44mS9ZsiSZP/HEE8m8o6MjmV9w\nwQXJ/NFHH03mV1xxRTIvu7x1fLOay9wD7rzzzmS+YsWKE56pVYqu4wM4xVB8ICCKDwRE8YGAKD4Q\nEMUHAqL4QEC8r35BR44cSeb33HNPMs97X/xRo0Yl81dffTWZX3TRRcn88OHDyTy6Dz/8sN0jNAVn\nfCAgig8ERPGBgCg+EBDFBwKi+EBAFB8IiHX8gnbv3p3M89bp82zcuDGZX3zxxYV+/9lnn13o+jg5\nccYHAqL4QEAUHwiI4gMBUXwgIIoPBETxgYBYxy9o/vz5yTzvMwVmzpyZzIuu00fX39+fzE87LX3u\nO1U/E4IzPhAQxQcCovhAQBQfCIjiAwFRfCAgig8ElLuOb2bPSvqBpF53vzTb1ilpnqT92Y8tdvff\nNW3KNtq8eXMy37BhQzLP+/z1G2644YRnQv3y1unz/vtcdtlljRynNOo54z8n6fs1tj/m7t/Kvk7J\n0gOnqtziu/tbkg7WiNJ/KgGUVpH7+AvMbIuZPWNm6c95AlAqQ32u/kpJS9zdzexhSY9Jmnu8H+7q\n6hq4XKlUVKlUhrhbAI0wpOK7+4FB366S9NvUzw8uPoD2q/emvmnQfXozGzco+6Gk9xo5FIDmqmc5\n7wVJFUljzGy3pE5J3zGzqZL6JXVLur2JMwJosNziu/uNNTY/14RZSunzzz9P5l988UUyP//885P5\nNddcc8IzRXLkyJFkvmLFikK///rrr0/mixcvLvT7y4pn7gEBUXwgIIoPBETxgYAoPhAQxQcCovhA\nQLyvfpOdddZZyXzkyJEtmqSc8tbpn3zyyWR+3333JfPJkycn8wcffDCZn3HGGcn8ZMUZHwiI4gMB\nUXwgIIoPBETxgYAoPhAQxQcCYh2/yW655ZZ2j9BWPT09yXzp0qXJfOXKlcl8zpw5yXzVqlXJPCrO\n+EBAFB8IiOIDAVF8ICCKDwRE8YGAKD4QkLl7c3dg5s3eRzO9/fbbyfyqq65K5nmvB//ggw9OdKRS\nefHFF5P5woULk/nBg7U+iPkrd911VzJftmxZMo/MzOTuNT/VmjM+EBDFBwKi+EBAFB8IiOIDAVF8\nICCKDwSU+3p8M5sgaY2kDkn9kla5+wozGy3p15ImSeqWNMvdP2nirG1hVnMZtO587969yXzJkiXJ\nfO7cucn8nHPOSebbtm1L5k899VQyf/PNN5N5d3d3Mr/wwguT+ezZs5N53jo+hqaeM/4RST9x97+V\n9PeS5pvZ30h6QNJ6d/+mpNcl/bR5YwJopNziu/s+d9+SXe6TtEPSBEkzJK3Ofmy1pGubNSSAxjqh\n+/hmNlnSVEkbJXW4e69U/eMgaWyjhwPQHHW/556ZjZS0VtIid+8zs2OfgH/cJ+R3dXUNXK5UKqpU\nKic2JYCGqqv4ZjZM1dL/wt3XZZt7zazD3XvNbJyk/ce7/uDiA2i/em/q/1zSdndfPmjbK5Juyy7f\nKmndsVcCUE71LOdNk3STpK1mtlnVm/SLJS2V9Bsz+ydJuyTNauagABqH1+PneOedd5J53uvxixo/\nfnwyP++885L51q1bGznO10yfPr1QvmDBgkaOg0F4PT6Ao1B8ICCKDwRE8YGAKD4QEMUHAqL4QECs\n4+f49NNPk/msWennLa1fv77Q/vOOXd77AeQZOzb92qo77rgjmT/00EOF9o/mYR0fwFEoPhAQxQcC\novhAQBQfCIjiAwFRfCAg1vEL6uvrS+Zr1qxJ5nnvG190Hf/hhx9O5vPmzUvmY8aMSeYoL9bxARyF\n4gMBUXwgIIoPBETxgYAoPhAQxQcCYh0fOEWxjg/gKBQfCIjiAwFRfCAgig8ERPGBgHKLb2YTzOx1\nM9tmZlvNbGG2vdPM9prZpuwr/XnIAEojdx3fzMZJGufuW8xspKQ/SZoh6UeS/uLuj+Vcn3V8oA1S\n6/jD8q7s7vsk7csu95nZDknjv/zdDZsSQMuc0H18M5ssaaqkP2SbFpjZFjN7xsxGNXg2AE1Sd/Gz\nm/lrJS1y9z5JKyV9w92nqnqLIHmTH0B51PVcfTMbJunfJP27uy+vkU+S9Ft3v7RG5p2dnQPfVyoV\nVSqVIjMDqEPqPn69xV8j6WN3/8mgbeOy+/8ys7slfdvdb6xxXR7cA9qgUPHNbJqkDZK2SvLsa7Gk\nG1W9v98vqVvS7e7eW+P6FB9og8Jn/II7p/hAG/CyXABHofhAQBQfCIjiAwFRfCAgig8ERPGBgCg+\nEBDFBwKi+EBAFB8IiOIDAVF8ICCKDwRE8YGAKD4QUO7bazeCGe/CDZRJ09+BB0D5cFMfCIjiAwFR\nfCAgig8ERPGBgP4fzOndNhthqDAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x121299e80>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig = pyplot.figure()\n",
    "ax = fig.add_subplot(1,1,1)\n",
    "imgplot = ax.imshow(X_train[1], cmap=mpl.cm.Greys)\n",
    "imgplot.set_interpolation('nearest')\n",
    "ax.xaxis.set_ticks_position('top')\n",
    "ax.yaxis.set_ticks_position('left')\n",
    "pyplot.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# Let's now display the 11th record"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP4AAAD7CAYAAABKWyniAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADbZJREFUeJzt3W+MVfWdx/HPpxKNLYpDXSARVleXrVpjSDfFbNiYW3Gp\n2TRBG2MrJmrdEGOga7Yh8c+TmWz2Ad0HJBpD4r82SKpb1sRFTKq2MZH4BzBtx4UWbBMcqy6M7CLI\nGB4U+e6DueDMOPO7M3PuP+b7fiU3nDnfe+/5cuAz55z7O+dcR4QA5PKlTjcAoP0IPpAQwQcSIvhA\nQgQfSIjgAwm1Lfi2b7C9z/YfbN/XruVOlu0B22/b/q3tXV3Qz5O2B23/94h5PbZftv2O7Zdsz+my\n/nptf2D7N/XHDR3sb6HtV2z/zvZu2/9cn98V63Cc/n5Yn9+Wdeh2jOPb/pKkP0haLul/JL0l6fsR\nsa/lC58k2/sl/W1EfNzpXiTJ9t9LGpL0VERcXZ/3Y0n/FxH/Xv/l2RMR93dRf72SjkXEhk70NJLt\nBZIWRES/7dmSfi1ppaQfqAvWYaG/76kN67BdW/ylkv4YEe9FxJ8l/YeG/5LdxOqiQ5+IeE3S2F9C\nKyVtqk9vknRjW5saYYL+pOH12HERcTAi+uvTQ5L2SlqoLlmHE/R3Ub3c8nXYrv/oF0l6f8TPH+jz\nv2S3CEm/tP2W7dWdbmYC8yJiUBr+jyNpXof7Gc9a2/22n+jkochIti+RtETSDknzu20djuhvZ31W\ny9dh12zhusCyiPiGpH+UtKa+K9vtuu18642SLo2IJZIOSuqGXf7Zkp6VdG99yzp2nXV0HY7TX1vW\nYbuC/6Gkvxzx88L6vK4REQfqfx6S9JyGD0+6zaDt+dLpY8SPOtzPKBFxKD7/0OhxSd/sZD+2Z2k4\nVJsjYmt9dtesw/H6a9c6bFfw35L017Yvtn22pO9Ler5Ny27I9pfrv3ll+yuSVkja09muJA0f6408\n3nte0p316TskbR37gjYb1V89SKd8V51fhz+R9PuIeGjEvG5ah1/or13rsC2f6kvDw3mSHtLwL5sn\nI2J9WxY8Cbb/SsNb+ZA0S9LPOt2f7acl1SR9VdKgpF5J/yXpPyUtkvSepFsi4kgX9fctDR+rnpQ0\nIOnuU8fTHehvmaTtknZr+N81JD0oaZekLerwOiz0t0ptWIdtCz6A7sGHe0BCBB9IiOADCRF8IKFK\nwe/2C28AjG/an+pP9sIb2wwbAB0SEeOe919liz/pC28i4vSjt7d31M/d9qC/mdtfN/fWiv5KqgT/\nTLjwBsA4+HAPSGhWhddO+sKbvr6+09MXXHBBhUW2Xq1W63QLRfQ3fd3cm9Te/qp8uHeWpHc0/OHe\nAQ2fA31rROwd87yY7jIATJ9txQQf7k17ix8Rn9leK+llfX7hzd4GLwPQBVp+kQ5bfKAzSlt8PtwD\nEiL4QEIEH0iI4AMJEXwgIYIPJETwgYQIPpAQwQcSIvhAQgQfSIjgAwkRfCAhgg8kRPCBhAg+kBDB\nBxIi+EBCBB9IiOADCRF8ICGCDyRE8IGEqnyFFrrA4OBgsf7SSy8V6+vXry/Wr7vuumJ96dKlxXoj\nt912W7F+1llnVXp/jI8tPpAQwQcSIvhAQgQfSIjgAwkRfCAhgg8k5CrfXW97QNJRSScl/TkivjCo\nazuqLCO7F154oVhftWpVsX7s2LFmttN0e/fuLdYvv/zyNnUy89hWRHi8WtUTeE5KqkXExxXfB0Ab\nVd3VdxPeA0CbVQ1tSPql7bdsr25GQwBar+qu/rKIOGD7LzT8C2BvRLw29kl9fX2np2u1mmq1WsXF\nAqiiUvAj4kD9z0O2n5O0VFIx+AA6b9q7+ra/bHt2fforklZI2tOsxgC0TpUt/nxJz9mO+vv8LCJe\nbk5bAFqp0jj+pBbAOH4lx48fL9Yvu+yyYv3AgQPNbKfp5s6dW6y/+uqrxfpVV13VzHZmlNI4PkNx\nQEIEH0iI4AMJEXwgIYIPJETwgYQIPpAQ99Xvcueee26x/uijjxbrt956a7H+6aefFuuXXnppsb5/\n//5ivZHDhw8X69u2bSvWGcefHrb4QEIEH0iI4AMJEXwgIYIPJETwgYQIPpAQ1+PPcMuWLSvW33jj\njWJ96dIvfFXCKLt27ZpyT1PRaJy/p6enpcs/k3E9PoBRCD6QEMEHEiL4QEIEH0iI4AMJEXwgIcbx\nZ7gdO3YU6+vWrSvWX3/99Wa2M2WDg4PF+rx589rUyZmHcXwAoxB8ICGCDyRE8IGECD6QEMEHEiL4\nQEINx/FtPynpO5IGI+Lq+rweST+XdLGkAUm3RMTRCV7POH4XGxoaKtavv/76Yn3nzp3NbOcLVq9e\nXaw/9thjLV3+mazqOP5PJX17zLz7Jf0qIr4m6RVJD1RrEUA7NQx+RLwm6eMxs1dK2lSf3iTpxib3\nBaCFpnuMPy8iBiUpIg5K4rxJ4AzSrO/OKx7E9/X1nZ6u1Wqq1WpNWiyA6Zhu8Adtz4+IQdsLJH1U\nevLI4APovMnu6rv+OOV5SXfWp++QtLWJPQFosYbBt/20pDck/Y3tP9n+gaT1kv7B9juSltd/BnCG\naLirHxGrJiiVB3jRFbZv316sNxqHb/V98xtZvnx5R5c/U3HmHpAQwQcSIvhAQgQfSIjgAwkRfCAh\ngg8kxH31u9yhQ4eK9RUrVhTre/bsKdZPnDgx5Z7a6fDhw8V6T09Pmzo583BffQCjEHwgIYIPJETw\ngYQIPpAQwQcSIvhAQs265x5a5N133y3W9+3bV6x3+zh9Iw8//HCx3tvb26ZOZha2+EBCBB9IiOAD\nCRF8ICGCDyRE8IGECD6QEOP4XW7p0qXF+ubNm4v122+/vVg/fvz4lHtqpw8//LDTLcxIbPGBhAg+\nkBDBBxIi+EBCBB9IiOADCRF8IKGG4/i2n5T0HUmDEXF1fV6vpNWSPqo/7cGIeLFlXWJCN998c7G+\nePHiYv2TTz6ptPzPPvusWL/pppuK9SNHjlRaPqZnMlv8n0r69jjzN0TEN+oPQg+cQRoGPyJek/Tx\nOKVxv6EDQPercoy/1na/7Sdsz2laRwBabrrn6m+U9K8REbb/TdIGSf800ZP7+vpOT9dqNdVqtWku\nFkAzTCv4ETHymxwfl7St9PyRwQfQeZPd1bdGHNPbXjCi9l1J5a9kBdBVJjOc97SkmqSv2v6TpF5J\n37K9RNJJSQOS7m5hjwCazK3+7nrb0eploHMa/dtu3LixWF+7dm2xfsUVVxTrb775ZrE+Z07ez51t\nKyLGHX3jzD0gIYIPJETwgYQIPpAQwQcSIvhAQgQfSIj76qOSRtfjNxqnb+Scc84p1m0uEp0OtvhA\nQgQfSIjgAwkRfCAhgg8kRPCBhAg+kBDj+Khkw4YNLX3/devWFevnn39+S5c/U7HFBxIi+EBCBB9I\niOADCRF8ICGCDyRE8IGEuK9+A8ePHy/W77nnnmL9rrvuKtavvfbaKffUTkNDQ8X6okWLivUjR45U\nWv7hw4eL9Z6enkrvP5NxX30AoxB8ICGCDyRE8IGECD6QEMEHEiL4QEINr8e3vVDSU5LmSzop6fGI\neNh2j6SfS7pY0oCkWyLiaAt77Yj77ruvWN+0aVOx3t/fX6xv2bKlWL/wwguL9blz5xbr77//frE+\nMDBQrD/wwAPFetVx+vXr1xfr5513XqX3x/gms8U/IelHEfF1SX8naY3tyyXdL+lXEfE1Sa9IKv8P\nAdA1GgY/Ig5GRH99ekjSXkkLJa2UdGpzt0nSja1qEkBzTekY3/YlkpZI2iFpfkQMSsO/HCTNa3Zz\nAFpj0vfcsz1b0rOS7o2IIdtjT8Cf8IT8vr6+09O1Wk21Wm1qXQJoqkkF3/YsDYd+c0Rsrc8etD0/\nIgZtL5D00USvHxl8AJ032V39n0j6fUQ8NGLe85LurE/fIWnr2BcB6E6TGc5bJuk2Sbtt/1bDu/QP\nSvqxpC2275L0nqRbWtkogObhevwG9u/fX6yvWbOmWH/xxRcrLX/x4sXF+jXXXFOsb9u2rVg/erTa\nqReNvp9+yZIlxfqOHTuK9bPPPnvKPWEY1+MDGIXgAwkRfCAhgg8kRPCBhAg+kBDBBxJiHL+iRteT\nX3nllcX6ypUrm9lO2zW6X8ChQ4fa1AnGYhwfwCgEH0iI4AMJEXwgIYIPJETwgYQIPpAQ4/gtduLE\niWL9mWeeqfT+u3btKtYfeeSRSu/f6Pvn33777WJ90aJFlZaP6WMcH8AoBB9IiOADCRF8ICGCDyRE\n8IGECD6QEOP4wAzFOD6AUQg+kBDBBxIi+EBCBB9IiOADCTUMvu2Ftl+x/Tvbu23/sD6/1/YHtn9T\nf9zQ+nYBNEPDcXzbCyQtiIh+27Ml/VrSSknfk3QsIjY0eD3j+EAHlMbxZzV6cUQclHSwPj1ke6+k\ni069d9O6BNA2UzrGt32JpCWSdtZnrbXdb/sJ23Oa3BuAFpl08Ou7+c9KujcihiRtlHRpRCzR8B5B\ncZcfQPeY1Ln6tmdJekHSLyLioXHqF0vaFhFXj1OL3t7e0z/XajXVarUqPQOYhNIx/mSD/5Sk/42I\nH42Yt6B+/C/b/yLpmxGxapzX8uEe0AGVgm97maTtknZLivrjQUmrNHy8f1LSgKS7I2JwnNcTfKAD\nKm/xKy6c4AMdwGW5AEYh+EBCBB9IiOADCRF8ICGCDyRE8IGECD6QEMEHEiL4QEIEH0iI4AMJEXwg\nIYIPJETwgYQIPpBQw9trN4PNXbiBbtLyO/AA6D7s6gMJEXwgIYIPJETwgYQIPpDQ/wNWC8HvQkSX\neQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x124541dd8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": []
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}