Week6_CIFAR10_First_CNN_HW.ipynb

{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"Week6_CIFAR10_First_CNN_HW.ipynb","version":"0.3.2","views":{},"default_view":{},"provenance":[]},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"}},"cells":[{"metadata":{"id":"zpdQobncwnTs","colab_type":"text"},"cell_type":"markdown","source":["## Building a CNN to classify images in the CIFAR-10 Dataset\n","\n","We will work with the CIFAR-10 Dataset.  This is a well-known dataset for image classification, which consists of 60000 32x32 color images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images.\n","\n","The 10 classes are:\n","\n","<ol start=\"0\">\n","<li> airplane\n","<li>  automobile\n","<li> bird\n","<li>  cat\n","<li> deer\n","<li> dog\n","<li>  frog\n","<li>  horse\n","<li>  ship\n","<li>  truck\n","</ol>\n","\n","For details about CIFAR-10 see:\n","https://www.cs.toronto.edu/~kriz/cifar.html\n","\n","For a compilation of published performance results on CIFAR 10, see:\n","http://rodrigob.github.io/are_we_there_yet/build/classification_datasets_results.html\n","\n","---\n","\n","### Building Convolutional Neural Nets\n","\n","In this exercise we will build and train our first convolutional neural networks.  In the first part, we walk through the different layers and how they are configured.  In the second part, you will build your own model, train it, and compare the performance."]},{"metadata":{"id":"KBXxoLNbwnTu","colab_type":"code","colab":{"autoexec":{"startup":false,"wait_interval":0},"base_uri":"https://localhost:8080/","height":34},"outputId":"ba6a2b70-ce59-4265-c587-592e6efd6165","executionInfo":{"status":"ok","timestamp":1526543596555,"user_tz":-180,"elapsed":2603,"user":{"displayName":"Özer ÇELİK","photoUrl":"//lh5.googleusercontent.com/-zXWHLydZydg/AAAAAAAAAAI/AAAAAAAACUE/Vk-d5hrZI64/s50-c-k-no/photo.jpg","userId":"109393400238666254492"}}},"cell_type":"code","source":["from __future__ import print_function\n","import keras\n","from keras.datasets import cifar10\n","from keras.preprocessing.image import ImageDataGenerator\n","from keras.models import Sequential\n","from keras.layers import Dense, Dropout, Activation, Flatten\n","from keras.layers import Conv2D, MaxPooling2D\n","\n","import matplotlib.pyplot as plt\n","%matplotlib inline"],"execution_count":1,"outputs":[{"output_type":"stream","text":["Using TensorFlow backend.\n"],"name":"stderr"}]},{"metadata":{"id":"pSrhxKDKwnTy","colab_type":"code","colab":{"autoexec":{"startup":false,"wait_interval":0},"base_uri":"https://localhost:8080/","height":102},"outputId":"a2194a18-a3f7-43bf-dfbe-5871e876a7b3","executionInfo":{"status":"ok","timestamp":1526543620495,"user_tz":-180,"elapsed":19099,"user":{"displayName":"Özer ÇELİK","photoUrl":"//lh5.googleusercontent.com/-zXWHLydZydg/AAAAAAAAAAI/AAAAAAAACUE/Vk-d5hrZI64/s50-c-k-no/photo.jpg","userId":"109393400238666254492"}}},"cell_type":"code","source":["# The data, shuffled and split between train and test sets:\n","(x_train, y_train), (x_test, y_test) = cifar10.load_data()\n","print('x_train shape:', x_train.shape)\n","print(x_train.shape[0], 'train samples')\n","print(x_test.shape[0], 'test samples')"],"execution_count":2,"outputs":[{"output_type":"stream","text":["Downloading data from https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz\n","170500096/170498071 [==============================] - 16s 0us/step\n","x_train shape: (50000, 32, 32, 3)\n","50000 train samples\n","10000 test samples\n"],"name":"stdout"}]},{"metadata":{"id":"k_f08PX6wnT2","colab_type":"code","colab":{"autoexec":{"startup":false,"wait_interval":0},"base_uri":"https://localhost:8080/","height":34},"outputId":"75ad34e6-b2f8-4a79-8bb5-bd26f5385542","executionInfo":{"status":"ok","timestamp":1526543625239,"user_tz":-180,"elapsed":722,"user":{"displayName":"Özer ÇELİK","photoUrl":"//lh5.googleusercontent.com/-zXWHLydZydg/AAAAAAAAAAI/AAAAAAAACUE/Vk-d5hrZI64/s50-c-k-no/photo.jpg","userId":"109393400238666254492"}}},"cell_type":"code","source":["## Each image is a 32 x 32 x 3 numpy array\n","x_train[444].shape"],"execution_count":3,"outputs":[{"output_type":"execute_result","data":{"text/plain":["(32, 32, 3)"]},"metadata":{"tags":[]},"execution_count":3}]},{"metadata":{"id":"YPM_2JbdwnT8","colab_type":"code","colab":{"autoexec":{"startup":false,"wait_interval":0},"base_uri":"https://localhost:8080/","height":283},"outputId":"1efe00bf-6bf5-4bc4-c6d5-4f926fa9db90","executionInfo":{"status":"ok","timestamp":1526543648256,"user_tz":-180,"elapsed":863,"user":{"displayName":"Özer ÇELİK","photoUrl":"//lh5.googleusercontent.com/-zXWHLydZydg/AAAAAAAAAAI/AAAAAAAACUE/Vk-d5hrZI64/s50-c-k-no/photo.jpg","userId":"109393400238666254492"}}},"cell_type":"code","source":["## Let's look at one of the images\n","\n","print(y_train[333])\n","plt.imshow(x_train[333]);"],"execution_count":5,"outputs":[{"output_type":"stream","text":["[3]\n"],"name":"stdout"},{"output_type":"display_data","data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAPoAAAD5CAYAAAAOeCiTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJztnXmU5FWV5z+RsWRELpVLZVVWVdZG\nUcVDKAUplB1ZBdw4LGpPKyow49LCOMdm5rh1K2i3c0RbW7RRRgdEx2n1YLfQIiIFKlCygwLig6L2\nqsyM3NfIyIjMmD8ifmVl/t59lSRFJOPvfs6pcyruyxe/F7+IG+8X9/u798ZKpRKKovxlU7PQC1AU\n5ZVHHV1RIoA6uqJEAHV0RYkA6uiKEgHU0RUlAiTmO9EY81XgRKAEfMxa+6j0t9baGRre2rVr2bFj\nBwDDo2PiMfZ173bah4f6xTl1tXXi2OTk5IzHF5z/dn5x1x0ATExMiPNSqZTTvnbN4eKcmpqYODY2\nPjzj8YknnMpDDz8AQD6fF+flJwriWKEw5bSXSvI6mptbZjw+5ZSTefDBLQAsXdIuzqtJyB+bnp59\nTns267YDtLd3zHh84gkn8tDDDwFw2Nr14rwXXrDiWGfnXqe9o2OFOCc2a9s76cRT+N1DDwLQ1Sl/\n5kZHB8WxXL5PHBsfH3Ha0+nMjMcfeN/fcMut/wJAsjYpPt/HPvoZ8c2e145ujHkTsMFaexJwJfD1\nlzK/trZ2Poc95DQ3Ny/0EgBoaGhc6CUA0NjYsNBLAKCh4dWyjlfH+9LWJn/pzpX5XrqfDfw7gLX2\nOaDFGLPoZa9GUZRXhPk6+jKg54DHPRWboiivQmLzuQXWGHMT8HNr7c8qjx8ArrDWPu/6+3w+X3q1\nXK4ryl8w4m/0+Qbj9jFzB18BdEp/HATeAowxWFsOpCxkMO4//dVl/N9//T6wsMG4c86+gHs2/wJY\n2GDc+ee/mbvuuhtY2GDcOWefwz2b7wEWNhh37jnn86t77gIWNhh3zcev5cv/9FngoME4cWy+l+53\nA5cCGGOOA/ZZa92rVhRlwZnXjm6t3WKMedwYswWYBj7q+/s/2odmPDbG7Le1tqwT5xUm3d9Dibi8\na9fE4uJYzPG9FtjiNfKpSCXTTrsvKtu2eLE41p3tDtlaW5YCsHPndnGej9Wr1zjtPT09TjvA4GB4\ntwls6bT7KgYg7blqapl1lRAwNSVfqYw7ruoC25NPPiHOy+XGxbHY7O25QleXfD7Wrl0bstWm6gE4\nYoOs0IyNyTv6iy9OimPppPvn7IqOjpBt/bojABjPyVeePuato1trPzHfuYqiVBe9M05RIoA6uqJE\nAHV0RYkA6uiKEgHU0RUlAsw76v5S6OnbJdqOes0J4ry2pcud9q597htpAMZGhsSxlSuXOmyrAJia\nct9wAjA+7pZx9u7dI87pyco3SkxNTTv+fgCAEc8NRLmcPNbU1Oq0Jzw3t9TVhWXDwNaddd9wApCp\nlWXFpkXulIfREfk2i+C1H8jeveXjp9P14rz2ZUvEsVTKLV0Vi+FzH9DbG5bJAlvSI9uuWb1SHGs6\nxi03AuzZs81p78lmw+vI9gLQutjtEwdDd3RFiQDq6IoSAdTRFSUCqKMrSgRQR1eUCFCVqHt9epVo\n6+0NR1wDcvmc097dJac8phJyMsZAfziqGtjGxuSI9sSEO+o+4okkL1okJ0FMThZDtl27dgAw5EnB\njckZp2zf5k6GWeFNy3Qk+VRsibicDlkU3heA557Z4bQPDMmJHxCOaA/2lz8XR2+UI9qL29xKA0Bf\nr1v1SCXl11WaCn8GAtu2Pc5SC+Wxnb8RxxIp2cVK0+7P6thYOHGlb6ALgJ07/yQ+38WXvEsc0x1d\nUSKAOrqiRAB1dEWJAOroihIB1NEVJQKooytKBKiKvBYrhWWEwLbNU8kzLVRfrfMkasRjnvLVBUe9\nrYqtZlpOaskLdbrqZlXrnPG0ebm217YXXwzbtpbPw7hH5mttkRMkpibcNdmmPbXaGh0dUUYHyzJY\nYUKudbZ3dzhJKSBd564nt8hTX6/gSDTJpMvJNfGEvBf5KpXX1LiTUAb6wwkjAV1dO0O2ffvK79XE\npJws1TMg1/lL18mve+NrznDaTzjs6JDtrLMvAqDfUW9wLuiOrigRQB1dUSKAOrqiRAB1dEWJAOro\nihIB1NEVJQLMS14zxpwB/AR4tmJ62lp7tfT3bYvDslBgmxTqsQGMDrkljd9teUCcU5uSv7vWH3Fk\nyJbdW5aKYil32yWQ667t3hWWYwKefeZZcayhIVwHrWtfuUZawpNd5csaa250y1rJhKxBTeXD536w\nryw/PfPk0+K8lla5VltLa7guH8BkMZyxF9DYGJagAlsq6WkNJbTKAqjPuGvN5Sdk+bJhUZNoO2yD\n3Oxx+y5ZZu3MynUFH3riPqfdbv/9jMdvOP6N3HnvDwE47cQzxefz8XJ09N9Yay99GfMVRakSeumu\nKBHg5ezoRxljbgdagWuttb86RGtSFOUQEyv57iMUMMZ0AKcCPwbWAfcB6621zvsmR0dGSg2O32GK\nohxSxBpE83L02RhjHgHeba113vR79y9/MeMgbz7vAu7+5S+A+QXjHjlEwbgr/+s1fPfrXwb8wbiS\n0Dt99y75nu+XEoy7+Qc/5PL3/jXgD8al4vJrk4Jxbe3t4px0ZmYQ6SMf++/c+M/XA/MPxq1cs9Zp\n9wXj6medjys/9EG+++2bAFjt6FkesGSJO/AHcpmv/gH5Xvf+gc4Zjy+//Cpuvvkb5WO1y695+y75\nXPmCceM5dzOJlpaZJbI++z++xrVf+m+APxh31ukXio4+r9/oxpj3GGOuqfx/GdAOyK09FEVZUOb7\nG/124IfGmAuBFPAR6bId4AgTzsYJbJ175PZKe/e5M3UKnu+n0QE5y6j4/Ash258qtuaWxeK8bG+v\n0757l7z29mXyTuq6iqpNlXfyFk+G2oSnJdPQkLvI5nRcrihZmwpfBXR39wCQ8BTZbG5qE8f6HG2N\nAGrr3C2SAFocWYqpii3b3RkaCxgZlttetS9zF5VMpeQWT83N4XZHga2/Ry4EutXKxUozDXIrp4xQ\nOHJ4qEe0WStfPZx1+oXi2Lwc3Vo7Arx9PnMVRak+Kq8pSgRQR1eUCKCOrigRQB1dUSKAOrqiRICq\nFIdMxMPSSmCblGsykiu6b+Y56fSzxDnPPPW4ODY9Fb5pI17JgGpfHpZWAhK1QpHKevdNKgCrV4X7\nzQU0OgolnnrKKQDs2ilnxI0Myn3qSjG3jDMyIt+Q1NQRlsnS6fJrWn+EfIPIyJAs8yWFbLNkXJaZ\n4o6xwFbf6CvAKb+2VNL90V6+TO5FNzI6GrK1LS5/LjIeWe6Yo94gjh155FHiWHOL+xz394fltYvO\n+yAgy6gHQ3d0RYkA6uiKEgHU0RUlAqijK0oEUEdXlAhQlah7MhmOqga2bLecNrj1+a1O+4CjBl3A\nZMGd+gewdtXqkG1lR9k2OCwnLYwI7ZUmHFH8gKf/KKep1s1KD30v8NTT5Tph00VZhnhxm9z6p7kp\nXO8MoLm5WZwzNjws2go5uT4dnjU2N7oTXool+Vxt3BiOTO+31ch7UV+P3J7oefuc097UJJ+PtY6U\n2JZFZVVmcbNcT2HF8mXiWMKjNvT2uFNY+x1to3p7yglUxYJHpvKgO7qiRAB1dEWJAOroihIB1NEV\nJQKooytKBFBHV5QIUBV5bfuOmTJZ+7K2/bbmFrcsBLB0qbvKp7V/EudkhAQUgIGhsIQW2GrT8ry2\nxe7kg8lxWYIaLxTEsWTjorAtUa4ZV+NpQeRjz153bc7ajJwU0j/QL9oaGhrEeYWcW24E+KPw3qxY\nHZY2A3bOam31uuM27bfV+dZRkCW7UsxdK2/PPrkq6559M2sAbjz2OB7Ycj8AiYQsk7W0ynLv4GD4\nHAcMDbulZddHYHenBaA2KSfX+NAdXVEigDq6okQAdXRFiQDq6IoSAdTRFSUCqKMrSgSYk7xmjNkI\n/Az4qrX2G8aYVcD3gTjQCVxmrc1L8/d1zm5ddOJ+W2uL3Cjv6KPDrZwAxj2ZVfY5d9YSQH823MLn\nySf+AMBhh60V563bsM5pX+Jp4zTSJ8sqkw55KrAtErLQAOoXyRlUPf3u9kSTU7LMVxwNy41DFVt3\nb7huWcCeXTvkdfS4512wXH6fu7vDLY0CW+2QLK81t8htr15z1Oud9pERuWVXtifc/qm1rXyM4RG5\n/VO6Tq4deFir/BlpaHyd0z4xEa7Jd8wxJwFQyMuSoo+D7ujGmHrgBmDzAebrgG9aa08DtgJXzOvo\niqJUhblcuueBtwAHfu2eQbnRIsAdwDmHdlmKohxKDnrpbq0tAkVjzIHm+gMu1bOAXCtZUZQFJ+Zq\n4evCGPM5oLfyGz1rrV1asa8HbrXWnizNHRoeKjUtkn97KopySBB7ZM/3XvdRY0zGWpsDOph5WR9i\n8+a7Zzy++KJ38tN/+wngD8b19boDJ48+9pg4xxeMi80qgfTTn/+Ui996MTC/YNz4iBxw27Y13Is9\noK5+5v3KX/nmt/jbj34Y8Afjnn72GXFsx44dTvsR5ghxTk1s5i+3H/zoNt777ksAmJiQ72efVzDu\nrW8T5xx73HEzHv/nD13Fd779DQBq0/MLxq1Y4b7IfCnBuHe966/48Y//FfAH45Yvl5tCZNJyrkGD\n0JxidjDu9NPewm/vvxPwB+POPucd4th85bV7gEsq/78EuGuez6MoShU46I5ujNkEfAVYCxSMMZcC\n7wFuMcZ8CNgJfM/3HB0d4W+8wJZJhzO5AiZybmno8HXrxTmtHslrz/YdIdvKVWsqxxLVQZLxpNMe\n93xPxjy/iPKO3TKwxVrk4oWLF8uvLSvspIOD8g6WSoVf1+hYuS2R7yddMh1usRWwqNl9RbLhCPnK\nYqWjfVVg6+ntFecNDneJY4XpQad9siC3certDRebzPZuAyBWIxdl7OyWr36Wta8Rx2Zf2QUMD4c/\ni/tt03P7qT2buQTjHqccZZ/NufM6oqIoVUfvjFOUCKCOrigRQB1dUSKAOrqiRAB1dEWJAFUpDlks\nhCWBwDY8Kcs/kwW3bJHOyIX6MhNyccXDjwjLcoFtqjApzisU3esYHR8V54yOhzOQAvoHBkK2HTvL\nxRB9ctLSdvkGkSWt7p5nu3btEuc0NITlnaH+sixVUyPeZMX0tNzf7nWvP95pP/GUs8Q5Kcf72d6x\nEoC9WbmY487tVhybyIf7ygEUPdl801Ph17V129MAxD3FIdO1csHGIza8Vhxbsfwwp331yiNDtuOO\nPRWAwSH5xh0fuqMrSgRQR1eUCKCOrigRQB1dUSKAOrqiRAB1dEWJAFWR15qbwxlqgW1qSs6vHRx0\nS03ty+QsrkGHdBUQSFgzbTsASMRlOak07ZbX8h4JzdcXbGoqnAkV2HxyWGdnuHhhQDzu/s6OxWQp\nLJcLF4cMbFPT8vozjXLO/OHrjXtA6IUGMDYWLvYZ2AqTctZYDDmLbuUKd7bc5KQsr2XSYZlsw7pN\nAMQTctZYj6eQZmla3kvTjuMBTBXCc2KUZeNJOcnSi+7oihIB1NEVJQKooytKBFBHV5QIoI6uKBGg\nKlH3F2ZVRD366Nftt/X0hOt0BUiR5BUrOsQ5S5YuEcd27QonSCSS5WSFbJdcyDbb7U6sSMiBZBrr\n5eql9Y5aYYGtUJCjwj6FIp1JO+3xhPxdPjwcrqs2PV2Ock9MyG2vFi+Tz39dg/t1Z7Py+1wohEPJ\n2a7y3w8NupNTAJYskd/rTZve6LT397lryQEMD4WPFVQprnckAAU0Nsoq0HPP/VEc27M767SvXTNT\nMVjecTzbtpf9peBRDXzojq4oEUAdXVEigDq6okQAdXRFiQDq6IoSAdTRFSUCzEleM8ZsBH4GfLXS\nTfUWYBMQFLC63lr7c2l+k0N+CGyLGuUWRA2NbslooF9OXJny1ARrbnIk11Rsg/1uqQMgmXSfpvGR\ncFJIgKv+WEBNTfj7NZ8vS0y1tXKiRmlaroc3JRyvJia/xclk+PwGtkZP4kpDg/yeSR2Dxsbk+noT\n42Epb7giq7U0ydKVTzrcvdMt56XT7s8UQKEY/uwEtuEh+b2We5hCd49c82665H7PVq46xWErS4ke\nhdXLXHqv1QM3AJtnDX3SWvsf8zusoijVZC6X7nngLRykNbKiKK9e5tJksQgUjQnlGV9ljPk4kAWu\nstbKdYoVRVlQYr72uAdijPkc0Fv5jX420GetfcoY8wlgpbX2Kmnu6OhYyVVDXFGUQ4oYLZjXve7W\n2gN/r98O3Oj7+0cfeXTG4zPPOoP77v01ACXkCiLzCcb19sr3Mu/YNrN6y6f/7lP8w+f/sTy2fas4\nr7vLHVDxBeNqU3JQbXYw7o5f3cvbzy03OPA1Ryh5emNPCfNKQsAHYGxWA4oHH/s9pxx/DABpTw/0\nlnb5Xvc3nhQOJAGsXrlSnDM7GPeBKy7nlv99MyA38QB/MK5RCPL6gnGDQzMvSt932ZXc+v3vAlAT\nkxs44Kni8+L2P4ljS9pWOO1vveCiGY/XrFnDzkp1JF8wbt3hci/2eclrxpjbjDHrKg/PAJ6Zz/Mo\nilId5hJ13wR8BVgLFIwxl1KOwv/IGDMOjAKX+56jVAofJrBNl2Rtoifr3p27hGwygHQmI461Lw+3\nNAps/X1yiKFzn7uOm6u1UoDvF1HKIddle8rynq9lUEKQ+QAyGXfWWDwuz2ltC2d/BbbFrbKshUOW\nC3juuRcE+4vinNzY+IzHH7jicu66+9flBx6ZctLTRmtk3F1cLVMnr/2wNctnPH7fZfD4Y+U97DVH\nrnNNAaA2nRTHkgn5ymg8525H9sgTv5nxeM2a9+23xUvy8/l29LkE4x6nvGvP5raDzVUU5dWB3hmn\nKBFAHV1RIoA6uqJEAHV0RYkA6uiKEgGqUhxyuhS+6SGwJZOyNJGbcEskQyN9TjsANa3yOqbDLze4\nQaWmRr4hIpOuc9rTdW47wOSkLP3EkuFjBbbalLyOUU8GWC7vPl6bQ0ILcK0/sMVTcqYcHsmus8f9\n3uze6ykOmQu/z0/94TkAaqblG6rinntYpoQbXErIcl3X3rCM+rstDwPQuU9uh3XSySeLY8vbZclr\nX7f7Jq3Hn/z1jMfvvOh9+23TNbL8evHF7xbHdEdXlAigjq4oEUAdXVEigDq6okQAdXRFiQDq6IoS\nAaoir7kKAwa2wSFZKsvn3f2/EglZkpuYkPOXJ8bDKWUjo+Wccp8cJvWAa26WiyRO5OV1uI4VvKaR\nYTnHfXBQHmsT+pD5JMBSLPy6Atv4hHw+ilMv7bUBeMubuPLKK7bREVlSjHmksozQiy6dkj87k5Nh\nmS+w7dwtV1KbvP8Bcey44zaIY8Wi+7M/1L9btCXTck8/H7qjK0oEUEdXlAigjq4oEUAdXVEigDq6\nokSAqkTdB4d6RFsuPyzOi8Xc9eRyuTFxTiolR5nz+XCUNmiFVCzK5TXFktier8mkpwpscSr8fDXx\ncjR4ylNDr65BbpPU0tbmtMc9NctGRsdFWzwuR91rU3LdtYSQaZKqlZNkph0x+ZpKfby8J6llqihH\n/0sl9/qTyJ8P1/uSD1Qc4bMIsHunXA+vMCnXIlzW4V7L9HQ4cWUiX7YtbmkUn8+H7uiKEgHU0RUl\nAqijK0oEUEdXlAigjq4oEUAdXVEiwJzkNWPMl4DTKn//ReBR4PtAHOgELrPWugu8AYND4XpbgS0e\nl9Md8nl3fay+3n5xTn29W2YCmHTIa2NjZanOV0ssJkhGeU/iR2+fp66do57ZUCWZJZ6QZahMnTyW\nEOq4jY+HJbQ/E/6eLxbLclYmI3e/zdTJba9yQ+4klLqMLPPhOPfp2rLcWF8vH2t0RD7/Y6PudaRq\nZJms1lEbsKaS0JQbcydYAaSS8vvS3SknIq1ctdZp71h7ZMh22LrjAejpk6U8Hwfd0Y0xZwIbrbUn\nAecDXwOuA75prT0N2ApcMa+jK4pSFeZy6f5b4J2V/w8C9ZR7sd1esd0BnHPIV6YoyiFjLk0Wp4Dg\nVrQrgTuB8w64VM8Cy11zFUV5dRATb++chTHmQuBTwJuBF6y1Syv29cCt1lqxuHVvX7bUtnjpIViu\noigexADEXINx5wGfBs631g4ZY0aNMRlrbQ7oAOTyG8D/+eG3Zjz+2NV/zz/fcB0wv2Bcd/ehCcb9\nwxe+wKc/8xkABnrl4Fl/n7v5QE+lp7mLlxKM+/0fnuGY122sPJJ/TaU8TRVaF7vvgy8W5SDj7GNt\n3ryZs88+G4DGxkXirDpP1Zp+IRg3OiE3HhjPzQx0PbHlPo47+UwABgflHvSjI+7+4gBTQj5EyyL5\nXvHZwbhn//g0Rx/1WgCKBfk8+oJxtRnZxTa98bVOe8fqmU1I/v6TX+a6L14D+INxN3z538SxuQTj\nmoDrgbdZawMPuwe4pPL/S4C7DvY8iqIsHHPZ0d8NtAE/NsYEtvcD3zHGfAjYCXzP9wSvWX+saOvz\n7Hx5oe5afkyu+yVcBADQ1Rm+8Ojq7AJgxLM7ZHvcGUhde+U2PYMD8lVHKhPeEQcGyjuXT15Lp2WJ\nKpCjZtO21POTySHzLWoqXxlMeGvGyZLR2LD7PI7l5EyzZG04Gy4RK++gcY/sOZmTpcPpojvrbWhE\nlskaSuF9b7yy7hqhxRNArEZeY35YPt7YsPvDuvHIE0VbX99h4vP5mEsw7ibgJsfQufM6oqIoVUfv\njFOUCKCOrigRQB1dUSKAOrqiRAB1dEWJAFUpDtm2eJVoGx6WZZdCwS2RjAiZSQAlZOktmQyPBba9\ne/eK81yyHEDO0WoqoFCQdb5SLJzol58o25Yslds8tbS2imNLl7pbMvl6IY1PhNcxUbElPAUg454M\nsBqhXVYJT6ssh0wW2CY9ra1qPOtISK2XSrIUNjoalg0DW0ODfJNQPCG70VRelin/8MzjTvvKNYtn\nPL74okt55NFy26dUWi6W6UN3dEWJAOroihIB1NEVJQKooytKBFBHV5QIoI6uKBGgKvKaS9AIbHUN\nDeK8oqMHFcDwsCxrdWflbLhly9pDtkRFGjnqqKPEeZJ0NTUpSyd4Cnq4BJLj31Au/re0fZk4r1CQ\n+8Nt377Dad/bKWfYdaxcHbINV3qvrV4rZ71NeDLRBofdvfSCIpwuah2y51RFnsyNyZlydXWyBCjt\nYA31ctHLpEMaXLGi/H5Me3rA+fLzpzLyGvsH3PUMHn74QdH22mPXis/nQ3d0RYkA6uiKEgHU0RUl\nAqijK0oEUEdXlAhQlaj7jl1bZzw+/o2v328bGhoU5+Un3V2emlsXO+0AXVl3fTeAffvCySmBzZe0\nsFSou1ZXK9dwk9YOf65DdiBLlpQj+72earSdXe5qtADd3e4Ibsti+Vy1LwuX4w9svgh/d1aufpsQ\nklqWeNYxMhSu9BorlaPc09PyOmpqPFH3mDvhpb5BjrovagxXiG1bUl732Khcny4Rl+v8xWJytD4R\nd39+YoTtga1j2dHi8/nQHV1RIoA6uqJEAHV0RYkA6uiKEgHU0RUlAqijK0oEmGuTxS8Bp1X+/ovA\nO4BNQKAFXW+t/bk0v7Eh3LAvsI2Myk304sLqli2XuzTXxOXvrsH+8LE6OlYAYP9kxXn9/W7JK1Mn\ny2vj47IcMzQUTtR44MEtAIyNyfMaGuQGgRs2bHDa1xy2TpyDQ4Jqaiq/Lz4JzZewk0y6WxeNj8tJ\nLRMFR+26iq0mIbdCKnrWkRGkz7xHNsw56rsFtlTSI+XVyGv0fR5rHa2oACg5JMqKbWnbWvH5fBzU\n0Y0xZwIbrbUnGWMWA08C9wKftNb+x7yOqihKVZnLjv5b4JHK/weBekD+ClMU5VXHXJosTgHBddeV\nwJ2UU6qvMsZ8HMgCV1lr5VvSFEVZUGIlz++cAzHGXAh8CngzcDzQZ619yhjzCWCltfYqae7IyEip\n0XF7oaIohxSx0P1cg3HnAZ8GzrfWDgGbDxi+HbjRN39LJdAUcN755/HLu34JwL6uneK8yYK7gsvk\npFy4P5uV7wefHYy74YYbuPrqq4GFDcZt3foi69cfDsw/GLdq9Rqn/aUE427+X9/i8v/yYcAfjBsc\nkPMTpHvTc55g3NisKjJb7fOsN0cAMDzirlgDkJCCWUAm7R6rd/SmD5hdfWbL/fdz8mmnlY/lcRVf\nMG66JN/rPjDgvghesqRtxuN7772Ps846E4DLLnu/+HyXX/4BeY3iSAVjTBNwPfA2a21/xXabMSb4\nBJ0BPHOw51EUZeGYy47+bqAN+LExJrDdDPzIGDMOjAKX+56gvb1DtOXy8u7Q1+veVeKedkGZjLzr\njSbDu0oiWc48qm/IiPO2b3Pv6P19ctulhEcWStaET3uypvwTak2HLB2uXivvzq1LwvXwQK7hBjAw\nOBSyBdl8kx4ZKiW1OwLyeXfLo1iNvKcUiuFdL7CVPHHforxEEFpzeTplMT4eHgxsi+rlDLWk53z4\n4tYZodac66ousD3/wvOeY8nMJRh3E3CTY+h78zqioihVR++MU5QIoI6uKBFAHV1RIoA6uqJEAHV0\nRYkAVSkOOTYalqcCWwn5RorxnFvWqk3J7YJ6e+TiiiPD4ayxwNbREZYAAxa3Njvtw4P94hz5lh5I\npcI32hy/6TgAkglPocG4LOPs27vXae/pl9c47Vjl4FBZcqv1Fb6ckFtRpWrd6497dK3adFjaDGzD\nnqKMMc9NncmEe/2utksBxUJY5isWynJhccotGwLUTPkKWMqfhGTKfa7GHDcX5Yvl87dj1w7x+Xzo\njq4oEUAdXVEigDq6okQAdXRFiQDq6IoSAdTRFSUCVEVee+Kxu2Y8PuW0N+237er2ZOMIPa1Wd8g5\nxYWCLMdM5HOirVjwFfFzSzV1DU3iHJ++NuGQpwql8vEHB+RsM1+uenHarTUlk7Kc5JqSTJQ/EjGh\ndxnAlEfWGh4dddpLJVmeytQ3iLakQxINyDuKOQaMj7vX0ezoNxfgksIWNZXXURBqIwAUJ2TpsFiU\nx5JJt/u5suEC26gnr9+H7uiKEgHU0RUlAqijK0oEUEdXlAigjq4oEUAdXVEiQFXkta3Ph0spB7a+\nsT3ivIRQ6LG5SS4o6asfn8te9tq2AAAGsUlEQVSF5al0uiydNdbL88bG3JLGkEf6yeXCUl5AsRjW\np3p7y6Wox3MT4ryMp1RxOu1+K31FHgsOeWqqkqU1WZTlpLo6uZDmyIj7ePGE/FGbcmR/1cTKe1Dc\nU0q5OCXLjSOjbplyaFhe+5K2tpAtKPKZiMvnfspTpXI8L68xP+k+x4sWhT+LiWRF4i3Nz2V1R1eU\nCKCOrigRQB1dUSKAOrqiRAB1dEWJAAcN4Rlj6oBbgHYgDXwe+D3wfcr9ZjqBy6y1eek5WlcsE21D\nu+Vuy7Ob7wX09XSKcxrrVotjrnphgU1q6AgwINSG80XWU0I9MIB6R3ufpqZypNXXsC83IUfkc0Iz\nwikh2QWg3tG0MYiOL2pYJM7r7e0Rx6aEJI6mxlZxTm4ifB5TlVZZJc/6pz113GIJd1LOpBDpBshP\nhD/CgS3hqddX62n2mJyQa+/lHElWAMVi+HUFtsEhWenxMZcd/e3AY9baNwHvAv4JuA74prX2NGAr\ncMW8jq4oSlWYS++1Hx3wcBWwh3IH1Q9XbHcA13CQ1smKoiwcc1bfjTFbgJXA24B7DrhUzwJykq+i\nKAtOrFTyVBGYhTHmWOBWYLm1dknFth641Vp7sjQvm91XWrp0xctdq6IofsRqIXMJxm0Cstba3dba\np4wxCWDEGJOx1uaADmCf7zlu/PYXZjz+7N/9C9d+/m8A2L77GXHe2Lg7+LRq+dHiHF8wLpud2W/9\nxhtv5CMf+QgANZ7+3dlst9M+IgTAwB+MSyZnjt1220+55JKLK88p3zLpDcbl3bHQlxKMe+C+X3Hq\nmedWxuYXjCtMutfR2jL3YNzDD/yaE049A4CdO3eK84Y85z8jBMhamt3NOADaWhfPXMfvfssJJ50O\nHCwYJwfcBoYGxDEpGNfYOLPizqNbHuANJ58KQDot34p7/713i2NzCcadDvwtgDGmHWgA7gEuqYxf\nAtzlnqooyquBufxG/xbwXWPM/UAG+CjwGHCrMeZDwE7ge74n2JUNb/iBbTwnJwRI0kpfv3uHBajP\ntItjroSXwNbXJ3/z7tq9y2kvetoMpdOy5OKat6dyDJfEE+BLamlrc7/uvGeNxWK4BVFQK84nNxY8\nzynVmht3JBQFuCSvfOUKpeBJyonJ6holQXprqKsX57iuAgLbpHClAlBTI+/ovpp9eeE58/nw+Q1s\nsZi8Dh9zibrngL92DJ07ryMqilJ19M44RYkA6uiKEgHU0RUlAqijK0oEUEdXlAjwku6MUxTl/090\nR1eUCKCOrigRQB1dUSKAOrqiRAB1dEWJAOroihIBqtKSKcAY81XgRKAEfMxa+2g1j19ZwxnAT4Bn\nK6anrbVXV3kNG4GfAV+11n7DGLOKl1Bs8xVcxy3AJqCv8ifXW2t/XoV1fAk4jfLn8YvAoyzM+Zi9\njndQxfNxKAqxSlRtRzfGvAnYYK09CbgS+Hq1ju3gN9baMyr/qu3k9cANwOYDzFUvtimsA+CTB5yb\najj5mcDGyufifOBrLMz5cK0Dqns+XrFCrNW8dD8b+HcAa+1zQIsxRi5j8pdLHngLM6vynAHcXvn/\nHcA5C7SOheC3wDsr/x8E6lmY8+Fah1x7+xXAWvsja+2XKg8PLMT6ss9FNS/dlwGPH/C4p2KT6wG9\nchxljLkdaAWutdb+qloHttYWgaIx5kBzfbWLbQrrALjKGPPxyjqustbKhfcPzTqmgKBd7ZXAncB5\nC3A+XOuYosrnA16ZQqwLGYwTC9m9wrwAXAtcCLyfcvUcucBb9Vmo8wLl34KfsNaeBTwFfK5aBzbG\nXEjZwa6aNVTV8zFrHQtyPiqFVt8B/ICZr3/e56Kajr6P8g4esIJycKGqWGv3Vi6RStbaF4EuygUu\nF5JRY0zQuPugxTZfKay1m621T1Ue3g68thrHNcacB3wauMBaO8QCnY/Z66j2+TDGbKoEZqkcd38h\n1sqfzPtcVNPR7wYuBTDGHAfss9bOr7/My8AY8x5jzDWV/y+jHOHcW+11zOJVUWzTGHObMWZd5eEZ\ngFyi99Adswm4HnibtTbofVX18+FaxwKcj1esEGtVs9eMMf+T8ouZBj5qrf191Q7+5zU0Aj8EmoEU\n5d/od1bx+JuArwBrgQLlL5n3UJZV0pSLbV5urZUrML5y67gB+AQwDoxW1pGVnuMQreODlC+Jnz/A\n/H7gO1T3fLjWcTPlS/iqnI/Kzv1dyoG4DOWfmI9R7qXwss6FpqkqSgTQO+MUJQKooytKBFBHV5QI\noI6uKBFAHV1RIoA6uqJEAHV0RYkA6uiKEgH+H9EwA4FRsVYLAAAAAElFTkSuQmCC\n","text/plain":["<matplotlib.figure.Figure at 0x7facf03768d0>"]},"metadata":{"tags":[]}}]},{"metadata":{"id":"Upw6c2npwnT_","colab_type":"code","colab":{"autoexec":{"startup":false,"wait_interval":0}}},"cell_type":"code","source":["num_classes = 10\n","\n","y_train = keras.utils.to_categorical(y_train, num_classes)\n","y_test = keras.utils.to_categorical(y_test, num_classes)"],"execution_count":0,"outputs":[]},{"metadata":{"id":"Sf_aRLBbwnUC","colab_type":"code","colab":{"autoexec":{"startup":false,"wait_interval":0},"base_uri":"https://localhost:8080/","height":34},"outputId":"b39765b4-a4cd-499e-e7eb-7ff006f0b4d3","executionInfo":{"status":"ok","timestamp":1526543662487,"user_tz":-180,"elapsed":1195,"user":{"displayName":"Özer ÇELİK","photoUrl":"//lh5.googleusercontent.com/-zXWHLydZydg/AAAAAAAAAAI/AAAAAAAACUE/Vk-d5hrZI64/s50-c-k-no/photo.jpg","userId":"109393400238666254492"}}},"cell_type":"code","source":["# now instead of classes described by an integer between 0-9 we have a vector with a 1 in the (Pythonic) 9th position\n","y_train[444]"],"execution_count":7,"outputs":[{"output_type":"execute_result","data":{"text/plain":["array([0., 0., 0., 0., 0., 0., 0., 0., 0., 1.], dtype=float32)"]},"metadata":{"tags":[]},"execution_count":7}]},{"metadata":{"id":"_8zmWb0MwnUF","colab_type":"code","colab":{"autoexec":{"startup":false,"wait_interval":0}}},"cell_type":"code","source":["# As before, let's make everything float and scale\n","x_train = x_train.astype('float32')\n","x_test = x_test.astype('float32')\n","x_train /= 255\n","x_test /= 255"],"execution_count":0,"outputs":[]},{"metadata":{"id":"cbxPJva-wnUL","colab_type":"text"},"cell_type":"markdown","source":["## Keras Layers for CNNs\n","- Previously we built Neural Networks using primarily the Dense, Activation and Dropout Layers.\n","\n","- Here we will describe how to use some of the CNN-specific layers provided by Keras\n","\n","### Conv2D\n","\n","```python\n","keras.layers.convolutional.Conv2D(filters, kernel_size, strides=(1, 1), padding='valid', data_format=None, dilation_rate=(1, 1), activation=None, use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, bias_constraint=None, **kwargs)\n","```\n","\n","A few parameters explained:\n","- `filters`: the number of filter used per location.  In other words, the depth of the output.\n","- `kernel_size`: an (x,y) tuple giving the height and width of the kernel to be used\n","- `strides`: and (x,y) tuple giving the stride in each dimension.  Default is `(1,1)`\n","- `input_shape`: required only for the first layer\n","\n","Note, the size of the output will be determined by the kernel_size, strides\n","\n","### MaxPooling2D\n","`keras.layers.pooling.MaxPooling2D(pool_size=(2, 2), strides=None, padding='valid', data_format=None)`\n","\n","- `pool_size`: the (x,y) size of the grid to be pooled.\n","- `strides`: Assumed to be the `pool_size` unless otherwise specified\n","\n","### Flatten\n","Turns its input into a one-dimensional vector (per instance).  Usually used when transitioning between convolutional layers and fully connected layers.\n","\n","---\n","\n","## First CNN\n","Below we will build our first CNN.  For demonstration purposes (so that it will train quickly) it is not very deep and has relatively few parameters.  We use strides of 2 in the first two convolutional layers which quickly reduces the dimensions of the output.  After a MaxPooling layer, we flatten, and then have a single fully connected layer before our final classification layer."]},{"metadata":{"id":"ddp7FR6fwnUN","colab_type":"code","colab":{"autoexec":{"startup":false,"wait_interval":0},"base_uri":"https://localhost:8080/","height":544},"outputId":"0974d01c-989a-4583-dd85-7a2e1edb29e0","executionInfo":{"status":"ok","timestamp":1526543697927,"user_tz":-180,"elapsed":740,"user":{"displayName":"Özer ÇELİK","photoUrl":"//lh5.googleusercontent.com/-zXWHLydZydg/AAAAAAAAAAI/AAAAAAAACUE/Vk-d5hrZI64/s50-c-k-no/photo.jpg","userId":"109393400238666254492"}}},"cell_type":"code","source":["# Let's build a CNN using Keras' Sequential capabilities\n","\n","model_1 = Sequential()\n","\n","\n","## 5x5 convolution with 2x2 stride and 32 filters\n","model_1.add(Conv2D(32, (5, 5), strides = (2,2), padding='same',\n","                 input_shape=x_train.shape[1:]))\n","model_1.add(Activation('relu'))\n","\n","## Another 5x5 convolution with 2x2 stride and 32 filters\n","model_1.add(Conv2D(32, (5, 5), strides = (2,2)))\n","model_1.add(Activation('relu'))\n","\n","## 2x2 max pooling reduces to 3 x 3 x 32\n","model_1.add(MaxPooling2D(pool_size=(2, 2)))\n","model_1.add(Dropout(0.25))\n","\n","## Flatten turns 3x3x32 into 288x1\n","model_1.add(Flatten())\n","model_1.add(Dense(512))\n","model_1.add(Activation('relu'))\n","model_1.add(Dropout(0.5))\n","model_1.add(Dense(num_classes))\n","model_1.add(Activation('softmax'))\n","\n","model_1.summary()"],"execution_count":9,"outputs":[{"output_type":"stream","text":["_________________________________________________________________\n","Layer (type)                 Output Shape              Param #   \n","=================================================================\n","conv2d_1 (Conv2D)            (None, 16, 16, 32)        2432      \n","_________________________________________________________________\n","activation_1 (Activation)    (None, 16, 16, 32)        0         \n","_________________________________________________________________\n","conv2d_2 (Conv2D)            (None, 6, 6, 32)          25632     \n","_________________________________________________________________\n","activation_2 (Activation)    (None, 6, 6, 32)          0         \n","_________________________________________________________________\n","max_pooling2d_1 (MaxPooling2 (None, 3, 3, 32)          0         \n","_________________________________________________________________\n","dropout_1 (Dropout)          (None, 3, 3, 32)          0         \n","_________________________________________________________________\n","flatten_1 (Flatten)          (None, 288)               0         \n","_________________________________________________________________\n","dense_1 (Dense)              (None, 512)               147968    \n","_________________________________________________________________\n","activation_3 (Activation)    (None, 512)               0         \n","_________________________________________________________________\n","dropout_2 (Dropout)          (None, 512)               0         \n","_________________________________________________________________\n","dense_2 (Dense)              (None, 10)                5130      \n","_________________________________________________________________\n","activation_4 (Activation)    (None, 10)                0         \n","=================================================================\n","Total params: 181,162\n","Trainable params: 181,162\n","Non-trainable params: 0\n","_________________________________________________________________\n"],"name":"stdout"}]},{"metadata":{"id":"vMxBTQ1NwnUR","colab_type":"text"},"cell_type":"markdown","source":["We still have 181K parameters, even though this is a \"small\" model.\n"]},{"metadata":{"id":"7vqjLoAgwnUT","colab_type":"code","colab":{"autoexec":{"startup":false,"wait_interval":0},"base_uri":"https://localhost:8080/","height":598},"outputId":"caf12816-6092-403d-ab2a-d0a699e9fea8","executionInfo":{"status":"ok","timestamp":1526544308741,"user_tz":-180,"elapsed":597072,"user":{"displayName":"Özer ÇELİK","photoUrl":"//lh5.googleusercontent.com/-zXWHLydZydg/AAAAAAAAAAI/AAAAAAAACUE/Vk-d5hrZI64/s50-c-k-no/photo.jpg","userId":"109393400238666254492"}}},"cell_type":"code","source":["batch_size = 32\n","\n","# initiate RMSprop optimizer\n","opt = keras.optimizers.rmsprop(lr=0.0005, decay=1e-6)\n","\n","# Let's train the model using RMSprop\n","model_1.compile(loss='categorical_crossentropy',\n","              optimizer=opt,\n","              metrics=['accuracy'])\n","\n","model_1.fit(x_train, y_train,\n","              batch_size=batch_size,\n","              epochs=15,\n","              validation_data=(x_test, y_test),\n","              shuffle=True)"],"execution_count":10,"outputs":[{"output_type":"stream","text":["WARNING:tensorflow:Variable *= will be deprecated. Use variable.assign_mul if you want assignment to the variable value or 'x = x * y' if you want a new python Tensor object.\n","Train on 50000 samples, validate on 10000 samples\n","Epoch 1/15\n","28736/50000 [================>.............] - ETA: 15s - loss: 1.8470 - acc: 0.3206"],"name":"stdout"},{"output_type":"stream","text":["50000/50000 [==============================] - 39s 781us/step - loss: 1.7374 - acc: 0.3658 - val_loss: 1.5467 - val_acc: 0.4495\n","Epoch 2/15\n","32992/50000 [==================>...........] - ETA: 12s - loss: 1.4851 - acc: 0.4632"],"name":"stdout"},{"output_type":"stream","text":["50000/50000 [==============================] - 40s 790us/step - loss: 1.4646 - acc: 0.4711 - val_loss: 1.3438 - val_acc: 0.5277\n","Epoch 3/15\n","34944/50000 [===================>..........] - ETA: 11s - loss: 1.3716 - acc: 0.5118"],"name":"stdout"},{"output_type":"stream","text":["50000/50000 [==============================] - 40s 790us/step - loss: 1.3638 - acc: 0.5154 - val_loss: 1.2683 - val_acc: 0.5491\n","Epoch 4/15\n","34976/50000 [===================>..........] - ETA: 11s - loss: 1.3028 - acc: 0.5347"],"name":"stdout"},{"output_type":"stream","text":["50000/50000 [==============================] - 40s 793us/step - loss: 1.2963 - acc: 0.5379 - val_loss: 1.2786 - val_acc: 0.5577\n","Epoch 5/15\n","35360/50000 [====================>.........] - ETA: 10s - loss: 1.2578 - acc: 0.5548"],"name":"stdout"},{"output_type":"stream","text":["50000/50000 [==============================] - 40s 793us/step - loss: 1.2524 - acc: 0.5565 - val_loss: 1.1583 - val_acc: 0.5864\n","Epoch 6/15\n","34976/50000 [===================>..........] - ETA: 11s - loss: 1.2208 - acc: 0.5669"],"name":"stdout"},{"output_type":"stream","text":["50000/50000 [==============================] - 40s 799us/step - loss: 1.2190 - acc: 0.5684 - val_loss: 1.1528 - val_acc: 0.5912\n","Epoch 7/15\n","35008/50000 [====================>.........] - ETA: 11s - loss: 1.1880 - acc: 0.5817"],"name":"stdout"},{"output_type":"stream","text":["50000/50000 [==============================] - 40s 799us/step - loss: 1.1894 - acc: 0.5797 - val_loss: 1.0896 - val_acc: 0.6206\n","Epoch 8/15\n","34528/50000 [===================>..........] - ETA: 11s - loss: 1.1572 - acc: 0.5926"],"name":"stdout"},{"output_type":"stream","text":["50000/50000 [==============================] - 40s 800us/step - loss: 1.1641 - acc: 0.5912 - val_loss: 1.2196 - val_acc: 0.5789\n","Epoch 9/15\n","34880/50000 [===================>..........] - ETA: 11s - loss: 1.1469 - acc: 0.5949"],"name":"stdout"},{"output_type":"stream","text":["50000/50000 [==============================] - 40s 795us/step - loss: 1.1504 - acc: 0.5955 - val_loss: 1.0757 - val_acc: 0.6226\n","Epoch 10/15\n","34848/50000 [===================>..........] - ETA: 11s - loss: 1.1218 - acc: 0.6099"],"name":"stdout"},{"output_type":"stream","text":["50000/50000 [==============================] - 40s 795us/step - loss: 1.1250 - acc: 0.6078 - val_loss: 1.1179 - val_acc: 0.6110\n","Epoch 11/15\n","34880/50000 [===================>..........] - ETA: 11s - loss: 1.1123 - acc: 0.6092"],"name":"stdout"},{"output_type":"stream","text":["50000/50000 [==============================] - 40s 797us/step - loss: 1.1153 - acc: 0.6094 - val_loss: 1.1037 - val_acc: 0.6264\n","Epoch 12/15\n","34720/50000 [===================>..........] - ETA: 11s - loss: 1.1010 - acc: 0.6160"],"name":"stdout"},{"output_type":"stream","text":["50000/50000 [==============================] - 40s 798us/step - loss: 1.1075 - acc: 0.6144 - val_loss: 1.1596 - val_acc: 0.6123\n","Epoch 13/15\n","35104/50000 [====================>.........] - ETA: 11s - loss: 1.0936 - acc: 0.6206"],"name":"stdout"},{"output_type":"stream","text":["50000/50000 [==============================] - 40s 796us/step - loss: 1.0985 - acc: 0.6202 - val_loss: 1.0857 - val_acc: 0.6292\n","Epoch 14/15\n","34912/50000 [===================>..........] - ETA: 11s - loss: 1.0836 - acc: 0.6234"],"name":"stdout"},{"output_type":"stream","text":["50000/50000 [==============================] - 40s 796us/step - loss: 1.0929 - acc: 0.6221 - val_loss: 1.0499 - val_acc: 0.6365\n","Epoch 15/15\n","35072/50000 [====================>.........] - ETA: 11s - loss: 1.0849 - acc: 0.6269"],"name":"stdout"},{"output_type":"stream","text":["50000/50000 [==============================] - 40s 795us/step - loss: 1.0873 - acc: 0.6271 - val_loss: 1.1530 - val_acc: 0.6126\n"],"name":"stdout"},{"output_type":"execute_result","data":{"text/plain":["<keras.callbacks.History at 0x7facee2e3c88>"]},"metadata":{"tags":[]},"execution_count":10}]},{"metadata":{"id":"JN9LP-scwnUX","colab_type":"text"},"cell_type":"markdown","source":["### Exercise\n","Our previous model had the structure:\n","\n","Conv -> Conv -> MaxPool -> (Flatten) -> Dense -> Final Classification\n","\n","(with appropriate activation functions and dropouts)\n","\n","1. Build a more complicated model with the following pattern:\n","- Conv -> Conv -> MaxPool -> Conv -> Conv -> MaxPool -> (Flatten) -> Dense -> Final Classification\n","\n","- Use strides of 1 for all convolutional layers.\n","\n","2. How many parameters does your model have?  How does that compare to the previous model?\n","\n","3. Train it for 5 epochs.  What do you notice about the training time, loss and accuracy numbers (on both the training and validation sets)?\n","\n","5. Try different structures and run times, and see how accurate your model can be.\n"]},{"metadata":{"id":"-1yM7aAlwnUX","colab_type":"code","colab":{"autoexec":{"startup":false,"wait_interval":0}}},"cell_type":"code","source":["# Please provide your solution here\n","# Create model_2 as mentioned in the exercise"],"execution_count":0,"outputs":[]},{"metadata":{"id":"RhdAyKRjwnUa","colab_type":"code","colab":{"autoexec":{"startup":false,"wait_interval":0}}},"cell_type":"code","source":[""],"execution_count":0,"outputs":[]}]}