DLL2_Pure_CNN.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "DLL2-Pure CNN.ipynb",
      "provenance": [],
      "authorship_tag": "ABX9TyO3vfTMO16tyGvzBC+XD3xA",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/NamLam-L/Transfer-Learning-model/blob/main/DLL2_Pure_CNN.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "33OLbtdRFgO_",
        "outputId": "ca7bad42-7331-4dbb-cbc8-251ef43a1ac8"
      },
      "source": [
        "import keras\n",
        "# keras._version_\n",
        "keras.__version__"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'2.7.0'"
            ]
          },
          "metadata": {},
          "execution_count": 31
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RO_nnBPTG1rK"
      },
      "source": [
        "# 5.1 - Introduction to convnets\n",
        "\n",
        "This notebook contains the code sample found in Chapter 5, Section 1 of [Deep Learning with Python](https://www.manning.com/books/deep-learning-with-python?a_aid=keras&a_bid=76564dff). Note that the original text features far more content, in particular further explanations and figures: in this notebook, you will only find source code and related comments.\n",
        "\n",
        "\n",
        "----\n",
        "\n",
        "First, let's take a practical look at a very simple convnet example. We will use our convnet to classify MNIST digits, a task that you've already been \n",
        "through in Chapter 2, using a densely-connected network (our test accuracy then was 97.8%). Even though our convnet will be very basic, its \n",
        "accuracy will still blow out of the water that of the densely-connected model from Chapter 2.\n",
        "\n",
        "The 6 lines of code below show you what a basic convnet looks like. It's a stack of `Conv2D` and `MaxPooling2D` layers. We'll see in a \n",
        "minute what they do concretely.\n",
        "Importantly, a convnet takes as input tensors of shape `(image_height, image_width, image_channels)` (not including the batch dimension). \n",
        "In our case, we will configure our convnet to process inputs of size `(28, 28, 1)`, which is the format of MNIST images. We do this via \n",
        "passing the argument `input_shape=(28, 28, 1)` to our first layer."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cClC4g9aKeDc"
      },
      "source": [
        "from keras import layers\n",
        "from keras import models\n",
        "\n",
        "model = models.Sequential()\n",
        "model.add(layers.Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1))) ## dùng convolution với 32 windows có size (3x3) trên bức ảnh 1 chiều (28x28x1) => output1: (26x26x32) -> độ sâu đã được tăng\n",
        "model.add(layers.MaxPooling2D((2, 2))) ## Pooling để giảm chiều rộng và chiều dài của output1: (26x26x32) => Output2: (13x13x32)\n",
        "model.add(layers.Conv2D(64, (3, 3), activation='relu')) ## dùng convolution với 64 windows có size (3x3)  với input là ouput2 => output3: ...\n",
        "model.add(layers.MaxPooling2D((2, 2))) #Pooling....\n",
        "model.add(layers.Conv2D(64, (3, 3), activation='relu')) ## convolution\n",
        "\n",
        "## Lưu ý: khi ấn \"run\" thì ta chỉ thu về được 1 kiến trúc mạng để học thôi chứ model của ta chưa học bất kỳ gì hết."
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jHZObn5JQms0"
      },
      "source": [
        "Let's display the architecture of our convnet so far:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "82aO2pTuQoxT",
        "outputId": "434d2ab8-760d-41fc-cd4d-009fa9f97122"
      },
      "source": [
        "model.summary()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model: \"sequential_1\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " conv2d_3 (Conv2D)           (None, 26, 26, 32)        320       \n",
            "                                                                 \n",
            " max_pooling2d_2 (MaxPooling  (None, 13, 13, 32)       0         \n",
            " 2D)                                                             \n",
            "                                                                 \n",
            " conv2d_4 (Conv2D)           (None, 11, 11, 64)        18496     \n",
            "                                                                 \n",
            " max_pooling2d_3 (MaxPooling  (None, 5, 5, 64)         0         \n",
            " 2D)                                                             \n",
            "                                                                 \n",
            " conv2d_5 (Conv2D)           (None, 3, 3, 64)          36928     \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 55,744\n",
            "Trainable params: 55,744\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eg__A4joRCVm"
      },
      "source": [
        "You can see above that the output of every `Conv2D` and `MaxPooling2D` layer is a 3D tensor of shape `(height, width, channels)`. The width \n",
        "and height dimensions tend to shrink as we go deeper in the network. The number of channels is controlled by the first argument passed to \n",
        "the `Conv2D` layers (e.g. 32 or 64).\n",
        "\n",
        "The next step would be to feed our last output tensor (of shape `(3, 3, 64)`) into a densely-connected classifier network like those you are \n",
        "already familiar with: a stack of `Dense` layers. These classifiers process vectors, which are 1D, whereas our current output is a 3D tensor. \n",
        "So first, we will have to flatten our 3D outputs to 1D, and then add a few `Dense` layers on top:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2sphH99URkmR"
      },
      "source": [
        "model.add(layers.Flatten()) ## Dãn đều các  3D vector thành  1D vector\n",
        "model.add(layers.Dense(64, activation='relu')) ## Vì ta đã giảm chiều dài và chiều rộng của bức ảnh và đồng thời tăng chiều sâu của nó lên tới 64 nên ta sẽ tạo 1 mạng neuron với 64 neurons\n",
        "model.add(layers.Dense(10, activation='softmax')) ## với mạng neurons ở trên, ta sẽ cho ra 10 output (10 khả năn/xác suất xảy ra)==> trở thành bài toán nhận diện 10 object."
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8XWVSTyRUMZE"
      },
      "source": [
        "We are going to do 10-way classification, so we use a final layer with 10 outputs and a softmax activation. Now here's what our network looks like:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "22xBgGPkUPdd",
        "outputId": "362cc5f6-eb05-496d-b560-f087f015cdb9"
      },
      "source": [
        "model.summary()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model: \"sequential_1\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " conv2d_3 (Conv2D)           (None, 26, 26, 32)        320       \n",
            "                                                                 \n",
            " max_pooling2d_2 (MaxPooling  (None, 13, 13, 32)       0         \n",
            " 2D)                                                             \n",
            "                                                                 \n",
            " conv2d_4 (Conv2D)           (None, 11, 11, 64)        18496     \n",
            "                                                                 \n",
            " max_pooling2d_3 (MaxPooling  (None, 5, 5, 64)         0         \n",
            " 2D)                                                             \n",
            "                                                                 \n",
            " conv2d_5 (Conv2D)           (None, 3, 3, 64)          36928     \n",
            "                                                                 \n",
            " flatten_1 (Flatten)         (None, 576)               0         \n",
            "                                                                 \n",
            " dense_2 (Dense)             (None, 64)                36928     \n",
            "                                                                 \n",
            " dense_3 (Dense)             (None, 10)                650       \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 93,322\n",
            "Trainable params: 93,322\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6scDYVj3VKV9"
      },
      "source": [
        "Lưu ý: từ bảng summary phía trên 5 bước đầu ta gọi là feature extraction, và 3 bước sau gọi là ta áp dụng mạng NN với feature extractions mà ta thu được"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xQsZaB2nUcAU"
      },
      "source": [
        "As you can see, our `(3, 3, 64)` outputs were flattened into vectors of shape `(576,)`, before going through two `Dense` layers.\n",
        "\n",
        "Now, let's train our convnet on the MNIST digits. We will reuse a lot of the code we have already covered in the MNIST example from Chapter \n",
        "2."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QssczR8EVW3l"
      },
      "source": [
        "from keras.datasets import mnist\n",
        "from tensorflow.keras.utils import to_categorical\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "N7M2KEvTWvl2"
      },
      "source": [
        "Segment code trên thực hiện 2 điều chính:\n",
        "<ul>\n",
        "  <li>Import dataset Mnist từ Keras</li>\n",
        "  <li>Import `(to_categorical )` để biến label của mỗi bức ảnh trong tệp dataset Mnist thành 1 vector (10,1) cho khớp với output của mạng NN ta vừa xây dựng.</li> \n",
        "</ul>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qNkgYzLqX60i"
      },
      "source": [
        "(train_images, train_labels), (test_images, test_labels) = mnist.load_data()\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ZzI3btg9YUMt",
        "outputId": "c726e204-1b77-4968-fee3-5012f52f72eb"
      },
      "source": [
        "train_images.shape"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(60000, 28, 28)"
            ]
          },
          "metadata": {},
          "execution_count": 38
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mqEjspIMYeQ7",
        "outputId": "fd089d13-aa30-4ba2-c0c2-c50951039520"
      },
      "source": [
        "train_labels.shape"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([5, 0, 4, ..., 5, 6, 8], dtype=uint8)"
            ]
          },
          "metadata": {},
          "execution_count": 39
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "qFdWvWMbYjjq",
        "outputId": "4339a98f-eac4-44d3-9bbd-ad4c3a8b7088"
      },
      "source": [
        "test_labels.shape"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(10000,)"
            ]
          },
          "metadata": {},
          "execution_count": 40
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0_h9ZEM2YaQ2",
        "outputId": "d263889f-7f10-4de3-b281-9ef9cd029219"
      },
      "source": [
        "test_images.shape"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(10000, 28, 28)"
            ]
          },
          "metadata": {},
          "execution_count": 41
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lmyD9yALZXYA"
      },
      "source": [
        "train_images = train_images.reshape((60000,28,28,1)) ## reshape the input.\n",
        "train_images = train_images.astype('float32')/255 ## vì số với số nguyên mạng neuron học rất bất tiện nên ta sẽ chuyển nó thành số thực và scale giá trị của nó trong khoảng từ 0 tói 1.\n",
        "\n",
        "test_images = test_images.reshape((10000, 28, 28, 1))\n",
        "test_images = test_images.astype('float32') / 255"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "npJ7ub4oaojw"
      },
      "source": [
        "## Chuẩn hóa labels của 2 tệp dataset"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DuaGO87Jaavo"
      },
      "source": [
        "train_labels = to_categorical(train_labels)\n",
        "test_labels = to_categorical(test_labels)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4XjEpzFDatJt",
        "outputId": "78a00d28-355d-4251-b416-ebd365c99e8e"
      },
      "source": [
        "train_labels"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[0., 0., 0., ..., 0., 0., 0.],\n",
              "       [1., 0., 0., ..., 0., 0., 0.],\n",
              "       [0., 0., 0., ..., 0., 0., 0.],\n",
              "       ...,\n",
              "       [0., 0., 0., ..., 0., 0., 0.],\n",
              "       [0., 0., 0., ..., 0., 0., 0.],\n",
              "       [0., 0., 0., ..., 0., 1., 0.]], dtype=float32)"
            ]
          },
          "metadata": {},
          "execution_count": 48
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "pKzzG-BHbuh_",
        "outputId": "10e31ac7-13ee-411c-e60a-fc62bb6efdb3"
      },
      "source": [
        "train_labels[0]"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([0., 0., 0., 0., 0., 1., 0., 0., 0., 0.], dtype=float32)"
            ]
          },
          "metadata": {},
          "execution_count": 49
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pEMHob5acfzZ"
      },
      "source": [
        "Mọi thứ đã được chuẩn bị rất tốt! Giờ ta sẽ bắt đầu giai đoạn train NN model. \\\\\n",
        "Lưu ý: trong model trên, ta có tổng cộng 93,222 trọng số ($w_{i}$ with i from 0 to 93,221). Con số này được lấy ra từ những dòng cuối của lệnh sumary phía trên.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "dw2kiSVndJXF",
        "outputId": "47fa0aa4-cb70-44ca-91b2-9fd5609f1f20"
      },
      "source": [
        "model.compile(optimizer='rmsprop',\n",
        "              loss='categorical_crossentropy',\n",
        "              metrics=['accuracy'])\n",
        "model.fit(train_images, train_labels, epochs=5, batch_size=64)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/5\n",
            "938/938 [==============================] - 53s 55ms/step - loss: 0.3770 - accuracy: 0.9346\n",
            "Epoch 2/5\n",
            "938/938 [==============================] - 52s 55ms/step - loss: 0.0636 - accuracy: 0.9821\n",
            "Epoch 3/5\n",
            "938/938 [==============================] - 51s 55ms/step - loss: 0.0460 - accuracy: 0.9874\n",
            "Epoch 4/5\n",
            "938/938 [==============================] - 52s 55ms/step - loss: 0.0392 - accuracy: 0.9896\n",
            "Epoch 5/5\n",
            "938/938 [==============================] - 52s 55ms/step - loss: 0.0353 - accuracy: 0.9908\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f0d155904d0>"
            ]
          },
          "metadata": {},
          "execution_count": 50
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VaOQiQSYdYZ8"
      },
      "source": [
        "Trong đoạn code trên\n",
        "<ul>\n",
        "  <li>Compile là setup một số tham số mà ta cần model xuất ra khi đang training, chẳng hạn: accuracy, optimize (cách thức tối ưu hóa), loss (hàm mất mát)</li>\n",
        "  <li>epoch = 5, batch size = 64 => học 60,000 dữ liệu trong tệp training 5 lần và học theo từng batch có size là 64 images/batch. Trên thực tế (epoch > 10)</li> \n",
        "</ul>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tIfh0AqPfZjF"
      },
      "source": [
        "Let's evaluate the model on the test data:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tEYrxVadfhlE",
        "outputId": "57404047-cf6b-4651-c224-4826f4d78bb7"
      },
      "source": [
        "test_loss, test_acc = model.evaluate(test_images, test_labels)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "313/313 [==============================] - 3s 11ms/step - loss: 0.0644 - accuracy: 0.9845\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 315
        },
        "id": "EhJjErSSfm1H",
        "outputId": "5cb57c5f-ba5a-4d28-bfd8-f4a7a5c077a6"
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "idx = 20\n",
        "X = train_images[idx]\n",
        "plt.gray()\n",
        "plt.imshow(X.reshape([28,28]))\n",
        "\n",
        "\n",
        "\n",
        "X = X.reshape((1,28,28,1))\n",
        "print(X.shape)\n",
        "img_class = model.predict(X)\n",
        "print(train_labels[idx])\n",
        "print(img_class.argmax())"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(1, 28, 28, 1)\n",
            "[0. 0. 0. 0. 1. 0. 0. 0. 0. 0.]\n",
            "4\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAANr0lEQVR4nO3db6xUdX7H8c9HujwRJFBSvLK0LqvGbGrKNjdYLTE2usTyBPeBm0VtaFy9mKzJqg0tUiMas2raWh+ZNazKotmy2UR2NdBk15JVW2OIV0MFvd31lqALuUIURFcfbJFvH9yDueA9Zy4zZ+YM9/t+JTczc74z53wz4cP5O+fniBCA6e+sphsA0BuEHUiCsANJEHYgCcIOJPEHvVyYbQ79A10WEZ5sekdrdtvX2P617VHb6zqZF4Ducrvn2W3PkPQbSd+QtF/Sq5JWRcRbFZ9hzQ50WTfW7EsljUbE3oj4vaSfSFrZwfwAdFEnYV8o6bcTXu8vpp3E9pDtYdvDHSwLQIe6foAuIjZK2iixGQ80qZM1+wFJiya8/nIxDUAf6iTsr0q60PZXbM+U9G1Jz9XTFoC6tb0ZHxHHbN8m6ReSZkh6MiLerK0zALVq+9RbWwtjnx3ouq5cVAPgzEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBI9HbIZmOiiiy6qrD/22GOV9RtuuKGyPjY2dto9TWes2YEkCDuQBGEHkiDsQBKEHUiCsANJEHYgiWlznn327NmV9VmzZlXWjx49Wln/9NNPT7snVFuxYkVl/Yorrqis33zzzZX1Bx98sLR27Nixys9ORx2F3fY+SR9L+kzSsYgYrKMpAPWrY83+VxHxfg3zAdBF7LMDSXQa9pD0S9uv2R6a7A22h2wP2x7ucFkAOtDpZvyyiDhg+48kPW/7fyLipYlviIiNkjZKku3ocHkA2tTRmj0iDhSPhyT9TNLSOpoCUL+2w277bNuzTzyXtFzSnroaA1AvR7S3ZW17scbX5tL47sC/RcT3W3yma5vx999/f2X9rrvuqqyvXbu2sv7II4+cdk+otmzZssr6Cy+80NH8L7744tLa6OhoR/PuZxHhyaa3vc8eEXsl/VnbHQHoKU69AUkQdiAJwg4kQdiBJAg7kMS0+YlrpzZs2FBZ37t3b2nt2WefrbudFM4999ymW0iFNTuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJMF59kKrW01v2rSptLZ8+fLKzw4P570jV9X3euedd3Z12dddd11preo209MVa3YgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSGLanGfft29fV+d/zjnnlNbuu+++ys/eeOONlfUjR4601dOZ4IILLiitLV3KmCK9xJodSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Joe8jmthbWxSGbZ8yYUVlfv359Zb3VfeM7ceutt1bWH3/88a4tu2nnnXdeaa3VkMyLFy/uaNkM2Xyylmt220/aPmR7z4Rp82w/b/vt4nFunc0CqN9UNuN/JOmaU6atk7QjIi6UtKN4DaCPtQx7RLwk6fApk1dK2lw83yzp2pr7AlCzdq+NXxARY8Xz9yQtKHuj7SFJQ20uB0BNOv4hTERE1YG3iNgoaaPU3QN0AKq1e+rtoO0BSSoeD9XXEoBuaDfsz0laXTxfLYkxi4E+1/I8u+0tkq6UNF/SQUkbJP1c0k8l/bGkdyR9KyJOPYg32bwa24yfM2dOZX3nzp2V9arfZbeye/fuyvrVV19dWf/ggw/aXnbTlixZUlrr9v30Oc9+spb77BGxqqR0VUcdAegpLpcFkiDsQBKEHUiCsANJEHYgiWlzK+lWjh49Wll/+eWXK+udnHq75JJLKuuLFi2qrHfz1NvMmTMr62vWrOlo/lXDJqO3WLMDSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBJpzrO38sorr1TWV69eXVnvxGWXXVZZ37VrV2X98ssvb6smSbNmzaqs33333ZX1Jo2MjFTWp/NQ2O1gzQ4kQdiBJAg7kARhB5Ig7EAShB1IgrADSUybIZu77emnny6tXX/99T3spF5nnVX9//3x48d71En9hobKRx174oknethJb7U9ZDOA6YGwA0kQdiAJwg4kQdiBJAg7kARhB5LgPPsUNTn0cDfZk56S/Vwv/33UbdOmTaW1W265pYed9Fbb59ltP2n7kO09E6bda/uA7V3F34o6mwVQv6lsxv9I0jWTTH8kIpYUf/9eb1sA6tYy7BHxkqTDPegFQBd1coDuNttvFJv5c8veZHvI9rDtM3fHFpgG2g37DyR9VdISSWOSHi57Y0RsjIjBiBhsc1kAatBW2CPiYER8FhHHJf1Q0tJ62wJQt7bCbntgwstvStpT9l4A/aHlfeNtb5F0paT5tvdL2iDpSttLJIWkfZI6G8QbjRkdHa2stzrPvn379sr60aNHS2v33HNP5WdRr5Zhj4hVk0yevr/8B6YpLpcFkiDsQBKEHUiCsANJEHYgCYZsPgMcPlz904R33323tPbww6UXN0qStmzZ0lZPU1X102BOvfUWa3YgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSILz7FO0d+/e0tpTTz1V+dnFixdX1kdGRirrjz76aGV9zx5uJzCZ5cuXl9bmzi29k5ok6ciRI3W30zjW7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBOfZp+ijjz4qrd1000097ARTtXDhwtLazJkze9hJf2DNDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJcJ4dXfXhhx+W1sbGxio/OzAwUHc7n3vggQcq62vWVI9CfuzYsTrb6YmWa3bbi2z/yvZbtt+0/b1i+jzbz9t+u3isvhsAgEZNZTP+mKS/i4ivSfoLSd+1/TVJ6yTtiIgLJe0oXgPoUy3DHhFjEfF68fxjSSOSFkpaKWlz8bbNkq7tVpMAOnda++y2z5f0dUk7JS2IiBM7Xe9JWlDymSFJQ+23CKAOUz4ab3uWpGck3R4RJ/0qJCJCUkz2uYjYGBGDETHYUacAOjKlsNv+ksaD/uOI2FpMPmh7oKgPSDrUnRYB1MHjK+WKN9jW+D754Yi4fcL0f5b0QUQ8ZHudpHkR8fct5lW9MKRy6aWXVta3bt1aWV+wYNI9x1rMmTOnsv7JJ590bdmdighPNn0q++x/KelvJO22vauYtl7SQ5J+avs7kt6R9K06GgXQHS3DHhH/JWnS/ykkXVVvOwC6hctlgSQIO5AEYQeSIOxAEoQdSKLlefZaF8Z5dpyGwcHqiy63bdtWWZ8/f37by77qquoTTS+++GLb8+62svPsrNmBJAg7kARhB5Ig7EAShB1IgrADSRB2IAluJY2+NTw8XFm/4447Kutr164trW3fvr2jZZ+JWLMDSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBL8nh2YZvg9O5AcYQeSIOxAEoQdSIKwA0kQdiAJwg4k0TLsthfZ/pXtt2y/aft7xfR7bR+wvav4W9H9dgG0q+VFNbYHJA1ExOu2Z0t6TdK1Gh+P/XcR8S9TXhgX1QBdV3ZRzVTGZx+TNFY8/9j2iKSF9bYHoNtOa5/d9vmSvi5pZzHpNttv2H7S9tySzwzZHrY9/e7zA5xBpnxtvO1Zkl6U9P2I2Gp7gaT3JYWk+zW+qX9Ti3mwGQ90Wdlm/JTCbvtLkrZJ+kVE/Osk9fMlbYuIP20xH8IOdFnbP4SxbUlPSBqZGPTiwN0J35S0p9MmAXTPVI7GL5P0n5J2SzpeTF4vaZWkJRrfjN8naU1xMK9qXqzZgS7raDO+LoQd6D5+zw4kR9iBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUii5Q0na/a+pHcmvJ5fTOtH/dpbv/Yl0Vu76uztT8oKPf09+xcWbg9HxGBjDVTo1976tS+J3trVq97YjAeSIOxAEk2HfWPDy6/Sr731a18SvbWrJ701us8OoHeaXrMD6BHCDiTRSNhtX2P717ZHba9roocytvfZ3l0MQ93o+HTFGHqHbO+ZMG2e7edtv108TjrGXkO99cUw3hXDjDf63TU9/HnP99ltz5D0G0nfkLRf0quSVkXEWz1tpITtfZIGI6LxCzBsXyHpd5KeOjG0lu1/knQ4Ih4q/qOcGxH/0Ce93avTHMa7S72VDTP+t2rwu6tz+PN2NLFmXyppNCL2RsTvJf1E0soG+uh7EfGSpMOnTF4paXPxfLPG/7H0XElvfSEixiLi9eL5x5JODDPe6HdX0VdPNBH2hZJ+O+H1fvXXeO8h6Ze2X7M91HQzk1gwYZit9yQtaLKZSbQcxruXThlmvG++u3aGP+8UB+i+aFlE/Lmkv5b03WJztS/F+D5YP507/YGkr2p8DMAxSQ832UwxzPgzkm6PiI8m1pr87ibpqyffWxNhPyBp0YTXXy6m9YWIOFA8HpL0M43vdvSTgydG0C0eDzXcz+ci4mBEfBYRxyX9UA1+d8Uw489I+nFEbC0mN/7dTdZXr763JsL+qqQLbX/F9kxJ35b0XAN9fIHts4sDJ7J9tqTl6r+hqJ+TtLp4vlrSsw32cpJ+Gca7bJhxNfzdNT78eUT0/E/SCo0fkf9fSf/YRA8lfS2W9N/F35tN9yZpi8Y36/5P48c2viPpDyXtkPS2pP+QNK+Penta40N7v6HxYA001NsyjW+ivyFpV/G3ounvrqKvnnxvXC4LJMEBOiAJwg4kQdiBJAg7kARhB5Ig7EAShB1I4v8B1DNMfBo+lxwAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    }
  ]
}