VAE_on_MNIST.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "VAE on MNIST.ipynb",
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kypSpHnKmlnR",
        "colab_type": "text"
      },
      "source": [
        "**Variational Auto Encoder trained on MNIST using Tensorflow.**\n",
        "\n",
        "*by Sodikov Makhmood. MIPT DIHT 2019.*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "R1iCXWJFleD6",
        "colab_type": "code",
        "outputId": "dcd499fd-8b20-4457-dc88-d95955e5e485",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 87
        }
      },
      "source": [
        "!pip install imageio"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Requirement already satisfied: imageio in /usr/local/lib/python3.6/dist-packages (2.4.1)\n",
            "Requirement already satisfied: pillow in /usr/local/lib/python3.6/dist-packages (from imageio) (4.3.0)\n",
            "Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from imageio) (1.17.4)\n",
            "Requirement already satisfied: olefile in /usr/local/lib/python3.6/dist-packages (from pillow->imageio) (0.46)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EHzkqxSzli-N",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from __future__ import absolute_import, division, print_function, unicode_literals\n",
        "\n",
        "try:\n",
        "  # %tensorflow_version only exists in Colab.\n",
        "  %tensorflow_version 2.x\n",
        "except Exception:\n",
        "  pass\n",
        "import tensorflow as tf\n",
        "\n",
        "import os\n",
        "import time\n",
        "import numpy as np\n",
        "import glob\n",
        "import matplotlib.pyplot as plt\n",
        "import PIL\n",
        "import imageio\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "from IPython import display"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GPwSe65qmxKf",
        "colab_type": "text"
      },
      "source": [
        "I am going to implement a basic VAE (Variational Autoencoder) model on Tensorflow, trained on MNIST dataset. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8RUhq7J5lnz2",
        "colab_type": "code",
        "outputId": "67fdf693-52a6-437d-822b-86c26cdf06a0",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 52
        }
      },
      "source": [
        "(train_images, _), (test_images, _) = tf.keras.datasets.mnist.load_data()"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz\n",
            "11493376/11490434 [==============================] - 0s 0us/step\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Am9uEOoYm2d6",
        "colab_type": "text"
      },
      "source": [
        "Now we will normalize and reshape our images.\n",
        "Each cell contains one number, which describes darkness of current pixel, so every cell is in [0,255]. Hence, we will normalize every pixel to 255."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oTfKBWP-lrdZ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "train_images = train_images.reshape(train_images.shape[0], 28, 28, 1).astype('float32')\n",
        "test_images = test_images.reshape(test_images.shape[0], 28, 28, 1).astype('float32')\n",
        "\n",
        "# Normalizing the images to the range of [0., 1.]\n",
        "train_images /= 255.\n",
        "test_images /= 255.\n",
        "\n",
        "# Binarization\n",
        "train_images[train_images >= .5] = 1.\n",
        "train_images[train_images < .5] = 0.\n",
        "test_images[test_images >= .5] = 1.\n",
        "test_images[test_images < .5] = 0."
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JxrvSrb05yye",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 300
        },
        "outputId": "f87e9bbe-be2e-47ab-a27a-be851b773aae"
      },
      "source": [
        "def show_num(arr):\n",
        "    img = arr.reshape((28,28))\n",
        "    plt.imshow(img, cmap=\"Greys\")\n",
        "    plt.show()\n",
        "print(0)\n",
        "\n",
        "\n",
        "# testing \n",
        "show_num(tf.keras.datasets.mnist.load_data()[0][0][2])\n",
        "print(tf.keras.datasets.mnist.load_data()[0][1][2])"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "0\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAM/klEQVR4nO3db4hc9b3H8c+n2iAmfRDNEoINNzEG\njVxsWoZYqBYv0aA+MFZBGqGkKE0FhRQqVPRBxSfK5balkUtlew1NL73WQisGCbexsSoFCW5kr4nG\nGqsJzZo/E6LUKBjdfO+DPSlr3DmzmTkzZ3a/7xcMM3O+5+z5cvSTM3N+M/NzRAjA7PeFuhsA0B+E\nHUiCsANJEHYgCcIOJHFuP3e2YMGCWLJkST93CaSyf/9+HTt2zFPVugq77esl/VzSOZL+KyIeKVt/\nyZIlGhkZ6WaXAEo0Go2WtY5fxts+R9J/SrpB0uWS1tm+vNO/B6C3unnPvkrSWxHxdkSclPRbSWur\naQtA1boJ+0WS/j7p+cFi2WfY3mB7xPZIs9nsYncAutHzq/ERMRwRjYhoDA0N9Xp3AFroJuxjkhZP\nev7lYhmAAdRN2F+WtNz2UttzJH1b0tZq2gJQtY6H3iLiU9v3SPqjJobeNkfEa5V1BqBSXY2zR8Q2\nSdsq6gVAD/FxWSAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDs\nQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Lo\nahZXYJDt3bu3Ze3aa68t3XZ0dLS0PjQ01FFPdeoq7Lb3S/pA0rikTyOiUUVTAKpXxZn93yLiWAV/\nB0AP8Z4dSKLbsIek7bZ32d4w1Qq2N9gesT3SbDa73B2ATnUb9qsi4muSbpB0t+1vnrlCRAxHRCMi\nGjPxogYwW3QV9ogYK+6PSnpK0qoqmgJQvY7Dbnuu7S+dfixpjaQ9VTUGoFrdXI1fKOkp26f/zv9E\nxP9W0lUP7Nu3r7T+3nvvldZXreJFy0yzc+fOlrXVq1f3sZPB0HHYI+JtSV+psBcAPcTQG5AEYQeS\nIOxAEoQdSIKwA0mk+Yrrjh07SutvvPFGaZ2ht8ETEaX1suHWN998s+p2Bh5ndiAJwg4kQdiBJAg7\nkARhB5Ig7EAShB1IIs04+6ZNm0rra9as6VMnqMqJEydK6w8//HDL2saNG0u3nY2/qsSZHUiCsANJ\nEHYgCcIOJEHYgSQIO5AEYQeSSDPOPj4+XncLqNhdd93V8bYrVqyosJOZgTM7kARhB5Ig7EAShB1I\ngrADSRB2IAnCDiQxa8bZ33333dL62NhYnzpBvxw/frzjba+77roKO5kZ2p7ZbW+2fdT2nknLLrD9\nrO19xf383rYJoFvTeRn/K0nXn7HsPkk7ImK5pB3FcwADrG3YI+JFSWe+XloraUvxeIukmyvuC0DF\nOr1AtzAiDhWPD0ta2GpF2xtsj9geaTabHe4OQLe6vhofE7PrtZxhLyKGI6IREY3Z+CN+wEzRadiP\n2F4kScX90epaAtALnYZ9q6T1xeP1kp6uph0AvdJ2nN32E5KukbTA9kFJP5b0iKTf2b5T0gFJt/Wy\nyenYvn17af2jjz7qUyeoyocfflha3717d8d/+8ILL+x425mqbdgjYl2L0uqKewHQQ3xcFkiCsANJ\nEHYgCcIOJEHYgSRmzVdc9+zZ036lEitXrqyoE1TlgQceKK23+1rzFVdc0bI2Z86cjnqayTizA0kQ\ndiAJwg4kQdiBJAg7kARhB5Ig7EASs2acvVtXXnll3S3MSB9//HFpfdeuXS1rw8PDpds++eSTHfV0\n2qZNm1rWzjvvvK7+9kzEmR1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkmCcvfD+++/Xtu9238s+depU\naf2FF15oWXvnnXdKtz158mRp/dFHHy2tj4+Pl9bnzp3bsrZmzZrSbduNhX/yySel9RUrVpTWs+HM\nDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJzJpx9vPPP7+0bru0ftNNN5XWL7300rPuabpeeuml0npE\nlNbPPbf1f8Z58+aVbtvue/z33ntvaf3qq68urZf9Hn/ZGLwkLV68uLTebkrnoaGh0no2bc/stjfb\nPmp7z6RlD9oesz1a3G7sbZsAujWdl/G/knT9FMt/FhEri9u2atsCULW2YY+IFyUd70MvAHqomwt0\n99h+tXiZP7/VSrY32B6xPdJsNrvYHYBudBr2X0haJmmlpEOSftJqxYgYjohGRDS4YALUp6OwR8SR\niBiPiFOSfilpVbVtAahaR2G3vWjS029J6m6+ZAA913ac3fYTkq6RtMD2QUk/lnSN7ZWSQtJ+Sd/v\nYY/T8tBDD5XWly1bVlp//vnnK+zm7Cxfvry0fvvtt5fWL7nkkpa1pUuXdtRTP2zbVj6Ic/jw4dL6\nZZddVmU7s17bsEfEuikWP96DXgD0EB+XBZIg7EAShB1IgrADSRB2IIlZ8xXXdtavX99VHdV75pln\nutr+jjvuqKiTHDizA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EASacbZMfvccsstdbcwo3BmB5Ig7EAS\nhB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgST4PjsGVkSU1g8c\nOFBav/jii6tsZ8Zre2a3vdj2n22/bvs12xuL5RfYftb2vuJ+fu/bBdCp6byM/1TSDyPicklfl3S3\n7csl3SdpR0Qsl7SjeA5gQLUNe0QciohXiscfSNor6SJJayVtKVbbIunmXjUJoHtndYHO9hJJX5W0\nU9LCiDhUlA5LWthimw22R2yPNJvNLloF0I1ph932PEm/l/SDiPjH5FpMXEmZ8mpKRAxHRCMiGkND\nQ101C6Bz0wq77S9qIui/iYg/FIuP2F5U1BdJOtqbFgFUYTpX4y3pcUl7I+Knk0pbJZ2e53i9pKer\nbw+Z2S69nTp1qvSGz5rOOPs3JH1H0m7bo8Wy+yU9Iul3tu+UdEDSbb1pEUAV2oY9Iv4iyS3Kq6tt\nB0Cv8HFZIAnCDiRB2IEkCDuQBGEHkuArrpixnnvuudL66tUMFk3GmR1IgrADSRB2IAnCDiRB2IEk\nCDuQBGEHkmCcHQOr3U9J4+xwZgeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJBhnR21uvfXW0vpjjz3W\np05y4MwOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0m0HWe3vVjSryUtlBSShiPi57YflPQ9Sc1i1fsj\nYluvGsXs0+533ZljvVrT+VDNp5J+GBGv2P6SpF22ny1qP4uI/+hdewCqMp352Q9JOlQ8/sD2XkkX\n9boxANU6q/fstpdI+qqkncWie2y/anuz7fktttlge8T2SLPZnGoVAH0w7bDbnifp95J+EBH/kPQL\nScskrdTEmf8nU20XEcMR0YiIxtDQUAUtA+jEtMJu+4uaCPpvIuIPkhQRRyJiPCJOSfqlpFW9axNA\nt9qG3bYlPS5pb0T8dNLyRZNW+5akPdW3B6Aq07ka/w1J35G02/Zosex+Setsr9TEcNx+Sd/vSYcA\nKjGdq/F/keQpSoypAzMIn6ADkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKw\nA0kQdiAJwg4k4Yjo387spqQDkxYtkHSsbw2cnUHtbVD7kuitU1X29i8RMeXvv/U17J/buT0SEY3a\nGigxqL0Nal8SvXWqX73xMh5IgrADSdQd9uGa919mUHsb1L4keutUX3qr9T07gP6p+8wOoE8IO5BE\nLWG3fb3tv9p+y/Z9dfTQiu39tnfbHrU9UnMvm20ftb1n0rILbD9re19xP+UcezX19qDtseLYjdq+\nsabeFtv+s+3Xbb9me2OxvNZjV9JXX45b39+z2z5H0puSrpN0UNLLktZFxOt9baQF2/slNSKi9g9g\n2P6mpBOSfh0R/1os+3dJxyPikeIfyvkR8aMB6e1BSSfqnsa7mK1o0eRpxiXdLOm7qvHYlfR1m/pw\n3Oo4s6+S9FZEvB0RJyX9VtLaGvoYeBHxoqTjZyxeK2lL8XiLJv5n6bsWvQ2EiDgUEa8Ujz+QdHqa\n8VqPXUlffVFH2C+S9PdJzw9qsOZ7D0nbbe+yvaHuZqawMCIOFY8PS1pYZzNTaDuNdz+dMc34wBy7\nTqY/7xYX6D7vqoj4mqQbJN1dvFwdSDHxHmyQxk6nNY13v0wxzfg/1XnsOp3+vFt1hH1M0uJJz79c\nLBsIETFW3B+V9JQGbyrqI6dn0C3uj9bczz8N0jTeU00zrgE4dnVOf15H2F+WtNz2UttzJH1b0tYa\n+vgc23OLCyeyPVfSGg3eVNRbJa0vHq+X9HSNvXzGoEzj3WqacdV87Gqf/jwi+n6TdKMmrsj/TdID\ndfTQoq+LJf1fcXut7t4kPaGJl3WfaOLaxp2SLpS0Q9I+SX+SdMEA9fbfknZLelUTwVpUU29XaeIl\n+quSRovbjXUfu5K++nLc+LgskAQX6IAkCDuQBGEHkiDsQBKEHUiCsANJEHYgif8HGYkDm+DLMm8A\nAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "4\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qQ3YqqSC0e7a",
        "colab_type": "text"
      },
      "source": [
        "Let's divide our dataset to train and test sets. \n",
        "\n",
        "So, we will train on 60k examples vs 10k we use to test."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JVpt9v1KlvmM",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "TRAIN_BUF = 60000\n",
        "BATCH_SIZE = 100\n",
        "\n",
        "TEST_BUF = 10000"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IbtJcyCXm949",
        "colab_type": "text"
      },
      "source": [
        "Let  and  denote the observation $x$ and latent variable $z$ respectively in the following descriptions. We use Keras.Sequential, because the architecture of our NN is pretty simple (on MNIST).\n",
        "\n",
        "Let's shuffle our data and divide to batches, each from 100 tensors.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "s-iOEO_dlx9s",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "train_dataset = tf.data.Dataset.from_tensor_slices(train_images).shuffle(TRAIN_BUF).batch(BATCH_SIZE)\n",
        "test_dataset = tf.data.Dataset.from_tensor_slices(test_images).shuffle(TEST_BUF).batch(BATCH_SIZE)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vH3JFxwF1U3P",
        "colab_type": "text"
      },
      "source": [
        "Our neural net will be divided to two independent layers -- decoder and encoder.\n",
        "\n",
        "Encoder will have the following architecture: \n",
        "\n",
        "* Input layer, that takes one image (matrix) of 28x28 dimension (or 28x28x1 tensor -- classical dim for picture from MNIST)\n",
        "\n",
        "* Two convolutional layers -- каждый из которых имеет фильтры размеров 32 и 64 (перевести)\n",
        "\n",
        "* Flatten layers and fully connected dense layer for summarizing all values from others to give the inference\n",
        "\n",
        "------\n",
        "\n",
        "Decoder layer consist of:\n",
        "\n",
        "* Input layer (latent_dim)\n",
        "\n",
        "* Dense layer of shape 7x7x32 with ReLU activation function\n",
        "\n",
        "* Reshape layer which connected with 2 stacked DEconv layers that will resize to the original image size\n",
        "\n",
        "-------\n",
        "\n",
        "We also have encoder and decoder function. First one splits our inference vector to two others: mean and logvar. \n",
        "\n",
        "Then we will sample new random vector from Norm(mean, logvar) and decode it to get new image. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "n0JNiGP-l3b5",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class VAE(tf.keras.Model):\n",
        "\n",
        "  def __init__(self, latent_dim):\n",
        "    super(VAE, self).__init__()\n",
        "    self.latent_dim = latent_dim\n",
        "\n",
        "    # Decoder Net\n",
        "    self.inference_net = tf.keras.Sequential(\n",
        "      [\n",
        "          tf.keras.layers.InputLayer(input_shape=(28, 28, 1)),\n",
        "          tf.keras.layers.Conv2D(\n",
        "              filters=32, kernel_size=3, strides=(2, 2), activation='relu'),\n",
        "          tf.keras.layers.Conv2D(\n",
        "              filters=64, kernel_size=3, strides=(2, 2), activation='relu'),\n",
        "          tf.keras.layers.Flatten(),\n",
        "          # No activation\n",
        "          tf.keras.layers.Dense(latent_dim + latent_dim),\n",
        "          # Getting 2 vectors, each of latent_dim size \n",
        "      ]\n",
        "    )\n",
        "\n",
        "    # Encoder Net\n",
        "    self.generative_net = tf.keras.Sequential(\n",
        "        [\n",
        "          tf.keras.layers.InputLayer(input_shape=(latent_dim,)),\n",
        "          tf.keras.layers.Dense(units=7*7*32, activation=tf.nn.relu),\n",
        "          tf.keras.layers.Reshape(target_shape=(7, 7, 32)),\n",
        "          tf.keras.layers.Conv2DTranspose(\n",
        "              filters=64,\n",
        "              kernel_size=3,\n",
        "              strides=(2, 2),\n",
        "              padding=\"SAME\",\n",
        "              activation='relu'),\n",
        "          tf.keras.layers.Conv2DTranspose(\n",
        "              filters=32,\n",
        "              kernel_size=3,\n",
        "              strides=(2, 2),\n",
        "              padding=\"SAME\",\n",
        "              activation='relu'),\n",
        "          # No activation\n",
        "          tf.keras.layers.Conv2DTranspose(\n",
        "              filters=1, kernel_size=3, strides=(1, 1), padding=\"SAME\"),\n",
        "        ]\n",
        "    )\n",
        " \n",
        "  @tf.function\n",
        "  def sample(self, eps=None):\n",
        "    # getting samples from norm () and decoding it\n",
        "    if eps is None:\n",
        "      eps = tf.random.normal(shape=(100, self.latent_dim))\n",
        "    return self.decode(eps, apply_sigmoid=True)\n",
        "\n",
        "  def encode(self, x):\n",
        "    # tf.split() splits a tensor into sub tensors.\n",
        "    # so we will split our vector of 2*latent_dim size to mean (expectation) and\n",
        "    # logvar (sigma) of norm(mean, sigm)\n",
        "\n",
        "    mean, logvar = tf.split(self.inference_net(x), num_or_size_splits=2, axis=1)\n",
        "    return mean, logvar\n",
        "\n",
        "  def reparameterize(self, mean, logvar):\n",
        "    # Reparametrization trick to improve \n",
        "    eps = tf.random.normal(shape=mean.shape)\n",
        "    return eps * tf.exp(logvar * .5) + mean\n",
        "\n",
        "  # decoder returns logits with sigmoid cross entropy (if parameter called)\n",
        "  # or without it  \n",
        "  def decode(self, z, apply_sigmoid=False):\n",
        "    logits = self.generative_net(z)\n",
        "    if apply_sigmoid:\n",
        "      probs = tf.sigmoid(logits)\n",
        "      return probs\n",
        "\n",
        "    return logits"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YSH_zem5oNSO",
        "colab_type": "text"
      },
      "source": [
        "Here we set optimizer (you can also try with other optimizers to feel the difference between them).\n",
        "\n",
        "`compute_loss` -- computes Reconstruction Loss between our original and decoded image.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lwlcIWngl6_t",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# you could also try:\n",
        "# optimizer = tf.keras.optimizers.Adam(5e-4)\n",
        "# optimizer = tf.keras.optimizers.Adagrad(learning_rate=0.01) \n",
        "# or:\n",
        "# optimizer = tf.keras.optimizers.SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True(???))\n",
        "\n",
        "optimizer = tf.keras.optimizers.Adam(1e-4)\n",
        "\n",
        "# Log norm pdf (getting sample from norm dist)\n",
        "def log_normal_pdf(sample, mean, logvar, raxis=1):\n",
        "  log2pi = tf.math.log(2. * np.pi)\n",
        "  return tf.reduce_sum(\n",
        "      -.5 * ((sample - mean) ** 2. * tf.exp(-logvar) + logvar + log2pi),\n",
        "      axis=raxis)\n",
        "\n",
        "# Reconstruction loss\n",
        "@tf.function\n",
        "def compute_loss(model, x):\n",
        "# get mean and logvar\n",
        "  mean, logvar = model.encode(x)  # use model in order not to re\n",
        "  z = model.reparameterize(mean, logvar)\n",
        "  x_logit = model.decode(z)\n",
        "\n",
        "  cross_ent = tf.nn.sigmoid_cross_entropy_with_logits(logits=x_logit, labels=x)\n",
        "  logpx_z = -tf.reduce_sum(cross_ent, axis=[1, 2, 3])\n",
        "  logpz = log_normal_pdf(z, 0., 0.)\n",
        "  logqz_x = log_normal_pdf(z, mean, logvar)\n",
        "  return -tf.reduce_mean(logpx_z + logpz - logqz_x)\n",
        "\n",
        "# Countiong grads with given optimizer\n",
        "@tf.function\n",
        "def compute_apply_gradients(model, x, optimizer_):\n",
        "  with tf.GradientTape() as tape:\n",
        "    loss = compute_loss(model, x)\n",
        "  gradients = tape.gradient(loss, model.trainable_variables)\n",
        "  optimizer_.apply_gradients(zip(gradients, model.trainable_variables))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uxs93YOfwB0Y",
        "colab_type": "text"
      },
      "source": [
        "VAEs train by maximizing the evidence lower bound (ELBO) on the marginal log-likelihood:\n",
        "\n",
        "$$\\log p(x) \\ge \\text{ELBO} = \\mathbb{E}_{q(z|x)}\\left[\\log \\frac{p(x, z)}{q(z|x)}\\right].$$\n",
        "\n",
        "In practice, we optimize the single sample Monte Carlo estimate of this expectation:\n",
        "\n",
        "$$\\log p(x| z) + \\log p(z) - \\log q(z|x),$$\n",
        "where $z$ is sampled from $q(z|x)$.\n",
        "\n",
        "We will not compute the KL term, but use the MC approach."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "E-RrbNFxl99V",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "epochs = 100\n",
        "latent_dim = 50\n",
        "num_examples_to_generate = 16\n",
        "\n",
        "# keeping the random vector constant for generation (prediction) so\n",
        "# it will be easier to see the improvement.\n",
        "random_vector_for_generation = tf.random.normal(\n",
        "    shape=[num_examples_to_generate, latent_dim])\n",
        "model = VAE(latent_dim)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8FI0MimfgQvD",
        "colab_type": "text"
      },
      "source": [
        "The following code sample will create a sample from decoder with random sample from distribution.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Qpo-LI7FmAmS",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def generate_and_save_images(model, epoch, test_input):\n",
        "  predictions = model.sample(test_input)\n",
        "  fig = plt.figure(figsize=(4,4))\n",
        "\n",
        "#   for i in range(predictions.shape[0]):\n",
        "#       plt.subplot(4, 12, i+1)\n",
        "#       plt.imshow(predictions[i], cmap='gray')\n",
        "#       plt.axis('off')\n",
        "#       plt.show() \n",
        "\n",
        "  for i in range(predictions.shape[0]):\n",
        "      plt.subplot(4, 4, i+1)\n",
        "      plt.imshow(predictions[i, :, :, 0], cmap='gray')\n",
        "      plt.axis('off')\n",
        "\n",
        "  # tight_layout minimizes the overlap between 2 sub-plots\n",
        "  plt.savefig('image_at_epoch_{:04d}.png'.format(epoch))\n",
        "  plt.show()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Jy9vbLpmfVqJ",
        "colab_type": "text"
      },
      "source": [
        "For each epoch we will count gradients with optimizer and mean for reconst loss, and get the "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "352Dusn1mDYx",
        "colab_type": "code",
        "outputId": "4248e083-3e5b-4b3f-9fa9-e4b9d9cb2033",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 266
        }
      },
      "source": [
        "generate_and_save_images(model, 0, random_vector_for_generation)\n",
        "\n",
        "\n",
        "# running 100 epochs\n",
        "for epoch in range(1, epochs + 1):\n",
        "  # counting time\n",
        "  start_time = time.time()\n",
        "\n",
        "  # compute grads\n",
        "  for train_x in train_dataset:\n",
        "    compute_apply_gradients(model, train_x, optimizer)\n",
        "    # print(train_x, grad)\n",
        "  end_time = time.time()\n",
        "\n",
        "  # сounting weighted mean of our Reconst Losses \n",
        "  loss = tf.keras.metrics.Mean()\n",
        "\n",
        "  for test_x in test_dataset:\n",
        "    loss(compute_loss(model, test_x))\n",
        "  elbo = -loss.result()\n",
        "  # print(elbo)\n",
        "  display.clear_output(wait=False)\n",
        "  print('Epoch: {}, Test set ELBO: {}, '\n",
        "        'time elapse for current epoch {}'.format(epoch,\n",
        "                                                  elbo,\n",
        "                                                  end_time - start_time))\n",
        "    generate_and_save_images(\n",
        "        model, epoch, random_vector_for_generation)"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch: 100, Test set ELBO: -78.14427947998047, time elapse for current epoch 87.87407112121582\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAOwAAADnCAYAAAAdFLrXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nO2daZxU1ZmHn1vdNA3NJmAjmyibgggG\ncImoRDRjNIKjcYmOg0RxMsREx2QURTMRMf5iFNSgRslERqNCguKKcWGJiigqiBqUKJsCIvvW0E1v\nNR8q/3NvVy9Ud9d2m/f50tBVXXXOvfecdz3v60WjUQzDCAeRTA/AMIzEsQVrGCHCFqxhhAhbsIYR\nImzBGkaIyK3rRc/zQu1CjkajXiLvO1jmCQfPXJvqPE3CGkaIsAVrGCHCFqxhhAhbsIYRImzBGkaI\nsAVrGCGizrBOqolEYvuF53lUVlYCUNdhBL2/WbNmVFRUALifdojBaAw5OTkA5OXlUVxcnOHR1I5J\nWMMIERmRsK1btwbgz3/+MwBff/01zz33HACrV68GYOvWrQC0atWKSy65BID27dsD8Pzzz7N8+XIA\n9uzZA0B5eXmaRl+VvLw8AFq2bAnA8OHDATjzzDPdXBYsWADA8uXLKSsry8Ao04ck1RlnnEHPnj0B\nKCkpAWDmzJlV/p8NeF4sP+Evf/kLAH369HH3cMeOHRkbV21kZMHedtttAHzve98DYotz7dq1sQHl\nxoZ0zjnnADB69GgOO+wwABYvXgzAgw8+SFFREeCrxJnA8zwOOeQQAG699VYArrzySiC2kDW20tJS\nAPbt28eMGTMAmDBhAkBWq1+J4nkee/fuBaBFixa1vu/hhx8G4He/+x3jx48HMm/KHH744QBccMEF\nQOxe6RlMF9o04MDXw1RiwwgRaZewubm5XHvttYCvxo4dO5b58+dX+d2wYcMAOP3003nnnXcA+PWv\nfw3Ahg0bMipZg0jFPeKIIwBfRa+oqGDXrl2Ar8p36NCB6667DoCf/vSnALz++usAnHfeeaFTl3v1\n6gXAypUra3xdjkTdUzkNf/7znztJrGchU5J27NixVf6/ePFitm3bVq/P0Ly6desGQN++fQFYuHBh\n0tV/k7CGESLSLmGnTp3qbIR7770XgDlz5jiJqd1KNu1DDz3EwoULAdi9ezeQebsnSKtWrQBo06YN\nAH//+98BeOedd5zNLRvloosu4sILLwR8W+/ss88GYMuWLU5K79y5Mz2DbyCnn346gNOKgjzxxBMA\njBkzxt1Tzf9f//VfAZgxYwZjxowBYNasWQC8+eabKR1zbaxfvx6ANWvWAHD11VfX+XxpLs2aNQPg\nhBNOYPbs2YD/DOizhgwZktAY6vM8ewcYXNJXRllZmfMkSlUMPqC6EJ06dQJg06ZNDVYVU30Uq1mz\nZlxzzTUA/PKXvwRi4wX47W9/y9NPPw3EnE3i0EMPBeDVV18FYNCgQe61LVu2ANCjRw8gcYdUuo7X\nHXPMMYC/KYmioiLn+a8Lvefrr7+moKAAgHXr1gHQs2fPhMycZN/T5s2bA/7Gu2vXrjpj+xI2P/zh\nD4GYA01zkZC57LLLAFi6dGmDhYsdrzOMJkDaVGJJ1ZycHPbv3w/gfkYiEadqaBeWdMlmR0ynTp1c\nOEfjltq+cOFCF+oI7rKSwGeddRYQc6BB7LpI+uozb7nlllRPIWEOO+ww3n33XcB3Jp177rkA/PWv\nf03oM4488kgACgoK3P3u2LEjEDMRFKpLJ/HPYqLIWdWqVSvnpFIsVxpIKkw3k7CGESLSJmE7dOgA\nxMIdsluCTokuXboAfgD7j3/8Y7qGVm8kHe68804nWWWnTpw4EYg5HuraYWUzyckG/o780UcfJX/Q\nDURznTRpEhs3bgR8G+2DDz6o12fJeRhMFJDGkS1hugMh7eKxxx4DYvdRDkRJ2lRm3ZmENYwQkTYJ\nq5DFvn37WLp0aZXXWrRowUMPPQTA0UcfDcC0adPSNbR6U1hYCMDIkSPdbjpnzhwAXnjhBaB2+0W7\n8dy5c4Gq0kbhACVTZAPyPWzZssWFYuorWR988EGAKp5kXZ+rrroKyK784kRQfnjfvn0ZPXo0gNMS\ng/c02aRtwerBXrFihXsg5SK/7bbbnBNGcblszrGV2p6bm8uSJUsAXG6sVLvCwkKnJiuXuE2bNrz2\n2muAH7oR69atc4cc5ADJzc3N2KEGoYdvw4YN7jBDon93//33A/CTn/yk2uva4BR/zXRsPdF8Xr2m\nwynl5eXOrPnwww+rvCfoTNXvotFojb9LFFOJDSNEpE3CfvXVV0As0CyHy6WXXgrAf/3Xf7ld6vrr\nr0/XkOqNxjhixAggliUj54OOYimf9Kc//Sndu3cHcKeN+vXrVy3BYPv27QCcf/75fPbZZ1W+p2vX\nrmzevBnInMYhLahv376MGjUKgHnz5gGxBAjRtm1bIJYwArHEAiXBxFNeXu4SD7LF2STVPxKJOK1G\nDqaakJTs37+/k7bKeQ9KWH2u5llZWemuaXwRhkQwCWsYISJtElanWMrKylw6n0415OTkuF1G6XnZ\niNLYdAZ2w4YNLgSj3ViJAEOHDqVz586A74yIRCJunnKy6OD+tm3byM/PB/yQzxFHHFHFhoTMJZL0\n79+fK664AqhaTgViKZrxjpa67LJLL73UJZVkGo1bUq9Zs2bVyhUF56L3nXfeeUBM25IfQ/dUqYq6\nn4BLCikrK6t2beJt2rpI24LVg5qTk+OqMwQPCssxk2nnQ11ovBq/Fif4C/bjjz8GYt5PLVTFZgcO\nHOhUWy3Adu3aAfDd737XqZX9+vUDYhvDN998A8Bdd90F+KZFuq6T1MOlS5dy8sknA76nO7hINX/l\nhbds2bLKAwvVN6lsQOaHFllwQ6xpYWkznTx5MhBzJOp9ioTonpaXlzuzQd9TUlLiFm8idcyqjTfh\ndxqGkXHSJmG1G+fn57sdJbhDf/755+kaSoORhAievlGoIz4n9bPPPnMHu++++24AbrjhBhdfVtZP\n165dgdiurBi0JHNOTo6TUg3ZjZOBJOyUKVPcUTSV9tEBdoBFixYB/pGyAQMGVPusm2++ucpnZgNS\n73VvS0tLax1fNBp190t533v37uVHP/oR4Id69FxXVFS4zwo+641xtJmENYwQkXYJW1hY6MIcwV1H\nNkE2E38g+8svv6zVeRKNRt3uqtDN9OnTXeVA2eyySVu2bOl2aDnowC88p5zbdCOJvmnTJqZOnQrA\nAw88APhnl/v27evs28svvxyomiOtHFv9XTYhm1U/D3R4XWec9QyMHj3aaR7pwCSsYYSItElYlc+4\n+OKLXThAu9nevXtdSmI2o121T58+AKxatarO4LqQXbtgwYJq3nBJ7f379zubV2GjjRs3uvrL+rts\nQGPXmHbu3Ml//Md/AFRJltC10fXKJttV1McncN1113HUUUcBuHz4Z599NiXjqo2UL1iFQnQo+5hj\njnGGvlS/yy+/PBTJ3/GHrtu1a+diszXFR/Va0ARQmEYPs65PixYtnBq5YsUKIBb6qe/B6nSi6zFh\nwgS+9a1vVXmtvLzcqcfZWJA7nqAKH78JK1wzZcoU95yeeOKJaRtbEFOJDSNEpEzCasc65ZRTAL8i\nfrBEzE033QQkXmIk00h9VdJD9+7d6d27N1D90Hl+fj6DBw8G/HkWFRXxzDPPALjD4LpOmzZtcsfr\n5IjK5vI44B+XGzNmjJO2cjBNnjzZncgJA8GMJ113hdTeeustICZ5pRJnSr03CWsYISIlEtbzPGe/\n3XfffYCfzlZcXMwNN9wAwB/+8Aeg7lMR2YQcFOoN9Oijj7rzrZMmTQJ8e+2KK65woQ5di507d7rU\nRYVwpG2UlJS4tL5sl6ySRurOAL7GMHLkSCBW8jM+aSCb006DyQxyiirnXT6I2bNnO+0nU6SkLrHn\nee5omQ71Sr14/PHH+dnPfgakXq1IVV3iYH1d5ZYGvlOf6X6nh2HGjBmMGzcO8DNrgg9KsmvY1kQy\nak1rzvfccw8QS4T/n//5H8CvdZSTk+M2aXmTNefG3PdU15qORCLOzJEqrAXcu3fverfxaChWl9gw\nmgApUYmj0ahTkdTWQSrgxo0bs1o1SgSFo4YPH+4OLWsXDkpWSRa1lpwyZUqo5x7f9EmnUt5//31e\nfvlloKrGoPlLpdTc9+zZ49T/bIovQ8zpdPHFFwP+XH7zm98A2RGeMglrGCEi7b110kmq7R3wz1F+\n+9vfBnz79u2333aH8VMtVdNlw0p7UE8kZTCtWbPGzbUmB6Iks356nlelZEp9SPU9zc/Pd027ZK+q\nUFw6k1jMhjWMJoBJWA6eecLBM9fGeImVD53JKii1zTPt/WENI5uprKzM6vxtU4kNI0TUqRIbhpFd\nmIQ1jBBhC9YwQoQtWMMIEbZgDSNE2II1jBBhC9YwQoQtWMMIEbZgDSNE1JmaaHmn4cByiavTVOdp\nEtYwQoQtWMMIEbZgDSNE2PE6wwgQiURqrHyp1+LxPM+9Px3FxU3CGkaISIqEjd+JknFkr7CwEIhV\nI0xn/836EIlEXJFwtXBQ57mysjLWrl0L+LWBSkpKQl01sSmiZmS6jxUVFdX6AOu15s2buwqQ55xz\nDgCXXXYZu3fvBuDee+8F4NVXXwVg3759SR+vSVjDCBFJtWFV6T03N9fV7q2vROnatSuAk04lJSWu\n/m2w5m02EIlE6NSpEwB/+ctfAOjZsycQq3wf3wP2448/5ic/+QkQq+UL4WlTEolEXIVIaT+6x6Wl\npa6siuy44LzqegaCr9VmOx7oM+qLPj8Sibhx7t27t9b3ByVujx49AFw/3A4dOjgJO3bsWPc7gJkz\nZ7rPTdb4k7JgNRgV0z7iiCP47LPPgMRLQ+oizpgxIzawf6oqBQUFrihWti3Y8vJyV97ziy++AODQ\nQw8FYuPXuOWsGDhwIPPnzwdg3bp1AK7P0GuvvZaVtYR0Xw455BAeeeQRAI4//njAL/85f/58vv76\na8BfqM2bN3cbrdq2qAXGypUrXY+hZcuWAbEi3fGLXd/teV5S+w3Fb6QHQu8rLi52Y2rbti1Q1fRR\n+derrrrK/a06M6qQfllZWaM2aVOJDSNEJFUllpHdu3dv10NVEuhAyLBXxzeR7N012UjlueiiiwB/\nl925c6fbydu0aQPEus8PGDAA8PvmSrW68MILmTt3LhArQg6wefPmpKtU9SUnJweAcePGccYZZwA4\nJ+Cjjz4KwKpVq9w4jzvuOACGDBnC97//fcCXRmrPsX79eqeJ9OrVC4hJXWkYut+SRNFoNGP9WINE\no1H3fEqb/OCDD3jiiSfc68GfrVq1cr2Bi4uLgdi9feONNwDfpKiP5mgS1jBCRFILiWs3/sMf/uAk\n68033wwc2Lmiplmy8UQ0Gq0xYJ0I2ZwoLvt20KBBANx+++1unmrduGzZsoSaRaUy+V+Nr9577z03\nZjWLWrhwIRCTEHqOFNYaPXq0k56SQJ988gngt50E/5kB328h6RV8jxw7ZWVlGbunnTp14tNPPwV8\nbXLYsGFO44hfS3l5ee56fOc73wFifgz5L15//XUA13M2eK/TUkhcor1Vq1YMHDgQ8B0uB1qwJ5xw\nQo2/lyrR1JCK9+WXXwIwceJEVqxYAfiqUjZ4kLWhtG3blkWLFgG+yq45RKNR54xR79j+/fvz5JNP\nAnV7xIO/kyqsB1cLuEWLFhmNX2tuZ511lhuTVN1169bVOrb9+/fzzTffAPDnP/8ZgDlz5rieRPV1\nfoGpxIYRKlKSS7x48WIn5hPdGS+//PIafy+XeVMjmFkDMedFtoWtwNeQKioqnLSIx/M8py4rTHXS\nSSclbA7Fo/dLgu/bty+j2oYk7KpVq5g0aRIATz/9NJD48x3sjasQoO63SVjDaKKkRML26dOnmp5e\nFzk5OfTr16/G12688cakji2T5ObmurCH+snKzstG6Qp+Qgj4WWhyCmnMeXl5jBw5EvCTBlauXNng\njuXx4ZHy8vKM2rCS7osXL3b2eGM6xzcmVGcS1jBCREokrOd59O/fH/Dzi+vK1bzxxhuruPfBt18U\nZA4LsnfatGnjJNKJJ54IwK233uoSK1544QXAz0EOnqvMJjZv3gzEtKDDDz8c8EM9Src744wzmDp1\nKhDrYA4xbaKh89GzEC9pM002JG+kZMFu2LDBZbeMGjUKgNmzZwOxScs1LkfT7bffXu0z1qxZA5DV\nWU41ISfNaaedxrRp0wDckaxoNOoywBYsWAD4zqdIJJKVarHin9u3b3eOpeHDhwP+MTLP81z4Terj\nhg0b3OZV3wWXDeGsVNKYjchUYsMIESmRsC+88ALXXHMNAOPHjwf87JghQ4a4I2nxGS3gqx06DNwY\n4z4TSDrs37/fnVYRe/fu5dprrwViYRygwY6ZdKH5LFu2zB2ve+uttwDYuHEjEDuRosSXyy67DIgl\nV0i1ra/EzBYVOFU0Zn4mYQ0jRKREwq5evZpXXnkF8E9vjBgxAvBPrgSJRqNu11EYYfr06e61MKHx\n/u1vf3PSVLb6zTff7M5/yjaX3ZqtdpvGtWbNGo499liAYF4vELPDW7ZsCfh5sUuWLAndvUsHnudV\nSUapLylZsPv27atWgUHOlcrKSvcQFBUVATGvoG64DkkHE8TDSGlpKTNnzgTgueeeA2IPuh7ybPOA\nHoiPPvqIs88+G4ChQ4cC/gH2Cy+80CX6P/PMM0Asf1gqsR7Mhs61MR7nbKNt27auaoWKPNTH7DOV\n2DBCREokbDQadXV/FIsUu3fvdmVgpP5OmDDBnfLQka2mgI5gSVuoKWOnoaGPdLN48WK6dOkCwFNP\nPQXgDtwfddRRTtV/7LHHgJjTTfdeEkRx2wNpT8HSMBA7ipjJ8J40hZycnAY7QaVBjhw50h0llTYp\nB2QiKrJJWMMIESmr/C+bVZk9sldvv/12d9JBUjiYOJGNhcgaSrwjKShF5XiQFMnGpIkgq1atclJO\nSTHnnnsuEBu7TlVpPnl5eU6qnHbaaQAuaWTRokU11uytrdJ+ZWVlWjUQjUOSVZldubm57Nq1C/Ad\nb6WlpdWqPWrc7dq1o3PnzgAuV37Pnj0sXboU8J12NVWJrA2TsIYRIlIiYSsrK51Ns2nTJsAv/fLY\nY4+5vOKge1sJE7WduQwzwbKdShZRMa/Vq1cDfuWJbKWkpMR5gK+88krAlwx79+5lypQpgK9JeZ7n\nbFXNX90Runbt6p4LpaBu377dSRw9H/q7YGQh1eTk5DitUNJRGsLxxx/vfC2yz9u0aeMSZFq3bg34\n2tLWrVt58MEHAT/ZZMWKFU7zkJZRH+0qZU6n5cuXA7FaQOBnMBUUFLiFqoc2Go26GlBhD+dobpFI\nxKlUOgAxYMAA57BRPPrMM88Esn/BRqNRHn/8cQDGjBkDVK0l/eGHHwL+IsvLy3MtSlRSRkfvxo4d\n6/KSFbd9+OGHefHFF4Hq1QTToQ7rHl199dVuQxIqDF5YWFhjdp7QeOfNmwfAddddV+2wOvibWkMw\nldgwQkTKnE7aJe+++24ALr30UiCW9RNfw3bbtm3ufdlWdE1SMi8vr1pooUWLFk4dkrYwePBgADp2\n7OicasqzbdOmjfu83//+9wBOMoUBOUukzko9fOSRR6rdt/3791er5B+UNpKaykdes2aNO8qXzmNs\n0hJUc/nXv/61m1ddrUPEli1bXAaYrksqMQlrGCEi5Q2dZcsqt3j8+PEceeSRgH9S5Z133mHWrFlA\n9oU3tLsec8wxTJw4EcCVcC0sLKxiswJVUg/jd+YtW7a4kjcqe5mtOcQ1Ic1IjhSl2H3++ecuHBdM\nuYxvdCzbbsaMGe65eO2114CYdMrEySyNUZ0IZMtCzW1UFZoKaofpJKmFxOtCcdl7773XOVqkIv3q\nV79iyZIlQHIdDMksJN6uXTt35E/xx1atWrnxynmig/rLly93D6XmWVxcnJIFmspC4jWhOGywEmAi\n902Oph49ejjHi9TgRK9LqorDq+D3Bx984DK65AjUIY558+alTaDUNk9TiQ0jRKRNwopIJFKtfWTQ\nCZFMkr0bx6u/wQycTOYCp1vCNjT/WddN2hbU38mYze1XkolJWMNoAqRdwqaTg303rolsmGvQmVNf\nKX2w31OTsIYRIlIe1jGMeLL97G82YxLWMEKELVjDCBF1Op0Mw8guTMIaRoiwBWsYIcIWrGGECFuw\nhhEibMEaRoiwBWsYIcIWrGGECFuwhhEibMEaRoioM/m/qR5RiudgmSccPHNtqvM0CWsYIcIWrGGE\nCFuwhhEibMEaRoiwBWsYIcIWrGGECFuwhhEirAibkVSuv/56INZ+RYXi1fP30UcfBeC2227Luh5K\nYSFtdYlV9f2fnwv41fNS1RAq2UF2dSzIz88HIDc317XVTGeLxHiyIXFCTcBycw8sA55++mkuuuii\nBn2PJU4YhhEaUqYSS4q2b98egO7duwPQuXNnWrduDcCuXbsAePPNN7OukbOQNL3hhhu47rrrAL/T\nWSQScVqCVDzNafr06UyZMgXwO7Q1RA3My8sDoGXLloDfi0btHTNJsMVmUIMS8dqb/t+9e/cq/YnC\nhMatxtzxTb4PRLNmzWpsTRr8Wef31+vbDMPIKCmxYT3Pcz1Ep06dCvg7afv27Z0TorCwEIhJ3X//\n938HYPHixQ35yhppjL2jHVTNp9977z0nWRtKWVkZ+/btA3xp+8033wDQunVr19B4+/btAMyaNYsR\nI0YA0K1bN8C/Pr/4xS+cNM+UDVuX3Xrffffx5JNPAjBt2jTA10wuvPBC1w+4vmTahlUT6//7v/8D\n4Nlnn3X/jtd6KioqnG9DGmdOTo7r3qfrJy2qtLTUPQO1zTNlKrEesKFDhwLw/vvvA7B69WrX/FiN\nkXv27MnChQuBWJNkyLzKp41MqqgcTrW9T8R37Q7SrFmzas2Q9fmlpaXs3r0bgJUrVwKx7uR//etf\nAX8D+fzzz4H6t2lMJmpsHVyoms+xxx4LwD/+8Q/3IP7ud78DcN3LG7pYM43necydOxeA3r17A3Dq\nqae6e/rHP/4R8E2goCMy2I0+fhHr/6YSG0YTI2UStqioCIi1oAdYsGABAG+88QZbtmwBYM6cOe53\nUo8lUaQKZgqp8JJob7/9thuTdsZly5bx3HPPAfDxxx8DMGDAAAAuvvhiJ23kZKioqHBS5sUXXwTg\n97//PQDr169n79697n3ZiFS5a6+9ttpro0aNAuDTTz8FYhqBzAmpfnoGwsr48eOdZBXl5eVMnjwZ\nwJk7idKQZuAmYQ0jRKQsceKwww6r8nPFihVAzDaN/87WrVuzdetWwJcubdq0ARqXkJBMB0WbNm34\nt3/7N8Cf07Rp05y2oHHLrhswYAD33Xcf4NvzL7/8MrfeeiuAs1clyRvT4yhdTqeTTjoJgEWLFumz\nAFi7di2DBg0CfNv6e9/7Hvfccw8ATzzxBACTJk1q6Fc7MuF0OvHEEwF49913q702btw4Hn744WR9\nlcMSJwyjCZAyG3bbtm2A7zGrK0C+Z88el2QwduzYVA2pUezevZvHH38cgK5duwIxO61du3aArwlI\nMzjhhBNcCKCgoACIeRb1t5LIsvXD0EXw1FNPrfJ/3dOlS5cyZswYAH7wgx8AcMoppzgJLNs/GRI2\nnQwfPhyAv/3tb9VeUxRj+vTp6RxS+nKJD8SwYcMA3zU+cOBAABeXagjJVp/0ACpTq3379s4JobDP\nEUccAcDgwYO55JJLAH/Bep7nHnIt1NGjRwPw0ksvNTjrJx0qsed5zqF09NFH63uB2MOrEE5NGU+K\nFysOmw71vzHPrswaCZtg/F0L9fzzzwfglVdeSclmayqxYTQBsuZ4nXbh9evXA9CiRQsgthtLfcx0\n3ql2UjmM9u7d61R/Sd8vvvgCiGXAPPXUUwBceeWVAFxyySVOEivYPnv2bAB69erFl19+mY5pNIj8\n/HwXehOas8I94Kv6kUjEvS4NQ8kfmTzZlAjSEuRAk9mzbds2zjvvPMB3QKXblDEJaxghImsk7Mkn\nnwz46WtyzuTl5bmd+aOPPgKyZ4euqKhw52GFpC/4ecJKGJgwYQIffvghAB06dAB8qTN37lz69OmT\n8jE3FM/znP0Wf4757rvv5pFHHgH8001vvfUWHTt2BODvf/87kD337UAo0WPkyJGA7wj93//9X/cM\nZkrbywqnU7du3ZgxYwbgx2t1Ydq1a+ceFHlpt23b5lSvusaf6UTxmpAzTQtX6lc0GnVmQH3zqNMV\nh1Ws9ZRTTgHgmWeeAfyNCfwDHatWrXIOwy5dugDJyQ9P5z2V+aKNp6ysjB07dgCpz0Yzp5NhNAEy\nKmGPOuooAO6//35uv/12wJesUhXbtWvnjH8dO6usrEzI2M9GCSsHjE4s6YA/wFlnnQXETunUh2wo\nEaNQl0ya/Px8evXqBcC6deuS9j3pvKfxTrW8vDwndRWWKy0ttbCOYRg1k3YJm5ub6xxKsufmzp1b\n6/lOz/MavINlo4TVeV+dYpKWAb5z6rvf/S6QuJ2UaQnreZ6z7RSuWr16tZOwySQT91SSNjc31zkL\ndY55x44dbu7JxCSsYTQB0hbW0S7leZ6zc2THHcDTm/rBpYmCggLnRa0pTU/nYcPGsGHDnGQVxxxz\nTIZGUz/kKxHBcE38s1dZWek0QeWJd+nShbfffrvG96eCtC1YTSbRKnNa4JFIJGsPdB8IOSjOPvts\nAPr27escS3rAFZvcuXOny4wKyyalkNTTTz/tfidnTKZL/CSKnq1gzSUtYt2H4LOoXPE77rgDiOUb\na8GmA1OJDSNEZE2mUzza5QoKCpyqGIZMGe3Gw4YNY/z48YB/ALqgoMCdapHqtXbtWgAWLlzoqklm\ne81ezfGKK64AqJJjrJI6YdOMgkXSNG4lsshR2Lx5cy6++GLAd5iqxE+6MAlrGCEi6yVsjx49WL16\nNeDbR9mC53luF9bB7TvvvBOIlZHRHCQpKysrnQ2/bNkywC8B6nme0yTqKpWaDaiLQ3DsQnNu27Yt\nO3furPHvo9FoVtvpGpuKqinFslmzZqxZswbA/Vy8eHG1XlGpJGsXrC7Cjh07ssaBoTGpqsQFF1zA\n/fffD/iZPqKystIdDJg3b+C/XqMAAA0OSURBVB4QO3L3xhtvALhaULrJ3bp1o1+/foB/gLq+bSDS\nxU033QT4qmIQXYe8vLxaH+BsXqw1IVOsoqLCVcfUffzqq6/SOh9TiQ0jRGS9hA0ehM40Un9Vl/fG\nG2+sJmUUpxs3bpyrsawc6IqKilo7BRQVFTnJqgPTJSUlWSmNgqdzhNRHtVzZvHlzVo69sSh3QDW1\n5TRMFyZhDSNEpLzdpH4mGqII5m1CzCaqaUdPN57nOcmnMjYbNmxwNYrnz58PwH/+538CsHXr1oTm\nLClUUlLi7Npsr6Sok1UPPPAAUDWXOFtDUclCJYF0njnd/hWTsIYRIlImYRX8j3d51yY14s8eBoPW\nNZXOzASSgLNmzQLg9ddfdzusdt6GSpjy8nI357AkHMg2P1iIRqPOYyybPdGz2ckiZcfr4h1FB5pU\nvAqteF5ubq5bFPVdDNl4vK4uGhrPy/TxunQStnvaUOx4nWE0AeqUsIZhZBcmYQ0jRNiCNYwQYQvW\nMEKELVjDCBG2YA0jRNiCNYwQYQvWMEKELVjDCBG2YA0jRNSZ/N9U8zHjOVjmCQfPXJvqPE3CGkaI\nsAVrGCHCFqxhhIisLcLWFNDB+06dOgGxZlcqfWqnpIyGYBLWMEKESdgUok5n99xzDwC9evXi6quv\nBuC9997L1LAajArjqStfsMB2vMYQ/H989RHP87KuWFtwjKnQfoKfX1s1lkS+1ySsYYSIlNV0qtcg\nPK/e5VATIZMxuxNPPJEnnngC8HvBFhQUsHTpUgCGDx8OJGe+qYzDyg4fMGAAgwcPBuD9998H4Ouv\nv9b3V/u7YCG5Qw89FPCbPH/yyScNLsCdzHvqeZ7TFoLPnzSHmuYVLx1zcnJc8TzVIVOHwtzcXNe4\nOz8/H4iVRVULFvkzVMCvtLQ0KG1rnGdSF6wG7Hlena0hdZFOOOEEAG644QYOP/xw92+ABQsWAI17\noNO5YDWnSZMmAXDNNde4ZlBq+Dtq1Ch3M0877TQAFi1a1NivTumC7dChAwB9+vRh69atAKxbt07f\n637q3qtOc8eOHTn22GMBOPXUUwH49re/DcDMmTOZOHFifYbhSOY9zc3NdfdDP0tKSlzzK81Pm1aL\nFi3c9ejatSsQa4L2wx/+EPA3pmBD6PjFX1xczMqVKwH41a9+BcCbb74J4Jqh1TVPU4kNI0Qkxekk\nNaFbt24AdO7c2XX5UpNiveeQQw5h+vTpAJxzzjkAVeoOv/rqqwCuL81VV13F5s2bkzHMpBOJRPiX\nf/kXAJ588knAV3+Li4t55plnAL+e8TnnnOPUp9dffx3wO8Bla5hHWsLixYvd7+pSgdVbSFIKYh0S\nwHdSyXmVaSoqKqqNtybnULBm9vHHHw/gnIennnqqu6fxlJeXu2f3o48+AmLXU9+l56I+3QNMwhpG\niGj0Vud5nuuX+tvf/haAfv36ce+99wLw2muvAdC3b18AbrvtNmfT1ORK1+4jyTNixAhefvllAHbv\n3t3Y4SaFzp07A/DSSy8xaNAgwJcszz//PAB33XWXc6xoLqeddpqTyC1btgT8PjW//OUv0zP4epJo\nFwL5GuRA2b17N6tWrQLglFNOAWDkyJEAvPXWW8keZoMIagqS/pFIpNYQVTQadTb6gAEDAN+ZBP41\n0DN//vnnOw0ziDRK+T3q0+khKbqJvnj58uVAzOEgo1yxSE2wZ8+e7gJowVZUVLi4pB54NVfatGlT\njZNONbqociAUFBRw+umnAzBt2jQg1qhLMVY5EGpSb6QWXX311c7hoGt28803A3DHHXdkTePqxhCM\nzeoa9u7dG/DbsMgJl41UVlbWGDeGmGPqrLPOAqCwsNC9pjlfdNFFADz33HMH/A5oWCMtU4kNI0Qk\nJayjHUhqXufOnZ1U1M8uXboAsayf73znO4DvfNixYwczZ84EcKEDxfhmzZrV4KZL9Q0BeJ7nJKrc\n9/379wdibnxJRV2zWbNmVXHFJ/A93HTTTQDceeedVV575ZVXOPvssxP+rCDZeh5W91dxW7UNbeg8\nIT2huviWp3quR4wYwVNPPQX4qnB5eTmjR48GYMaMGQ39ympYWMcwmgApyXTKzc11u5MkloLK48eP\n59JLLwX8nWvdunVOUikLZPXq1QCMGzfOSd360pDdWDtnx44dATj33HOBmBbw0ksvAX6rwYaEYiS5\nN23aBPjXZ8eOHc6hEQyJJEK2Slglw3z22WcAfOtb3wLg888/b/BnpkPC6hmQz+LKK68EYs9C0MkE\nMX/Gj3/844Z+Va2YhDWMJkBKItgFBQUceeSRgO9NUx7pcccd57xkwUC6vMnxr6W7mbO+v6ioCPA9\neRUVFc4eb0ySgz5XP5VosXHjxqxNnqgPwQQZefw//fRTAL744ouMjStRIpEIAwcOBODBBx8EoEeP\nHu41oWfhxhtvTOv4UrJg+/Tpwx133AFA9+7dAT9PslmzZi53Uou4uLiY1q1bA/Dll18Cflxy165d\nqRhirUgl13il/ubn5ydlQUmlUnaMPnP27Nmh6bxeF3LMPf/88y7zrVevXkD2ZnMFyc3N5YwzzgD8\nzL2aFqrypJXAny5MJTaMEJESCVtUVOSSBRSeWbZsGQBLly51UnTIkCFALANGTim9X46Kuk79pIJ4\nKbBx40YgFtYJnuhoKMcddxzgO5ukGk+bNq3aySTP80IhlYIoDDZs2DCX0ZRuKdRY9MzJHJIDtbi4\n2GXsKT863ZiENYwQkRIJ+9VXX7nzjkox1C5bXl7uHBPKvx0yZEg121E2XsuWLZ0dmU4bT7usTh21\nadPG2TRfffUVELN36yMBc3Jy+NOf/uT+Df6pnZp27LBJV/BTLT3Pc0kGYZpHNBrlxRdfBHx/g2za\niRMnJiRZ9Xzn5OQ46Zys9NqUxGETrY8jD+kDDzzA97//fcB/cHU0bfv27a5KwwcffAAkPvlkxOw0\nl4KCAtq1awfgMrXOPPNM55DQwQcl/JeWlrpNR8n/f/rTn1xsT+pvz549Ad/Z1hCyIQ6ra7NkyRIg\nltnWr18/gAZXl6iJVMVhlefseZ77t+LhWrh79uxJyETTZjxq1Ch+/vOfA/Czn/0M8E3DA2FxWMNo\nAqREJU5UBZJRX1ZW5iSVnE+XXXYZEDshITVSEjadaC5FRUVOXdfh+qFDh7ryIMreCmoX2o0lTYMH\nneVcU7mVmgiL06l58+b86Ec/AvyY6z333ONMh2zF8zx3VPLkk08GYOXKlS4LTc+nTLdE0T3bvHmz\nC13qGR46dGijTmWZhDWMEJHRWh1yIh199NHObtBPGetLlixh8uTJQPIM94ainVOHtK+//nrnXLvi\niisA+MUvfgHEcoYlYQsKCqp91rPPPlvlM2siVdUkk4VywYcPH86IESMAmDp1KgDvvvtuVo45SNu2\nbZ2TTCeI5syZw/r16wFfCxo1ahQQux/KbJLPoa77l5+f7+790UcfDcSSSKSFNASTsIYRIjJal1j6\n/ZIlS1xSgnblhQsXArHCZQrr1JdM1CWWh/DII490ZXJ04uefYwLgpJNOAvyzov8cB+Cn9+Xl5VUL\n4tcktdLlJdb45G+Q/XfLLbc4CaKqC9u3b0+JhE3mPR00aBDz5s0D/IhFMHQYf+IM/NTVu+66C4C7\n777b+Tb0/j59+gCxXGSVQxKDBw9OyFNc2zwzohJLTXj33XcBXw0G38CXitnQxZopdMPXrl3r4rZB\n4kMFyqFu3ry5SzJXwe4vvviCTz75BMi8ORBEzjMdl+vTp497CNOdmdYYdu7c6RajFtuBKjrqdanG\nP/7xj6vNXfevsLCwmkmjzLaGYiqxYYSIlEvYeDWqVatWzJ8/3/07ngkTJgCNSyTIBiorK53KGESV\nBFUBUtKqRYsWbvdVN4ANGzY06rB8stEY2rdvD8B///d/A7FQnEJuaq1ZVlbm1Ef9jP+cTLNu3TpX\n3kVtVVQBFPxxSuXduHGje10Ot4KCAoYOHQr4poxeC4b4VINYuekNxSSsYYSIlEtY2QhyvEyePNml\n4wXR4eaHHnoo1UNKC5WVlW5XldSJRqPccsstAPzjH/8AfGdSNBrNGslzIDRmOVcikYgrByMpU1d9\n32yhsrLS5Q2rJFDPnj2dH0Inzmpqwi1bVs2uwE++GD9+PBCzZZWE8YMf/ABovE8m5V5iqcKzZs0C\n4IILLqj2nj179rh6Rsl0MmWyex34D68O7L/wwgvOu5iJLn2Q3Ht6//33AzBw4EAXz1RLj8rKypQs\n0Ezf00TQYu7WrZszfdTyJNH7brnEhtEESFscVvEuZcSAH6ro1q2byx5KJmHYjZNBpk7rpLpreU2E\n7Z7qGtX3+piENYwmQNokrKoiTp8+3TljFHxO5nnJIGHbjRtKNpyHTRcH+z01CWsYISKjucSp5mDf\njWviYJlrU52nSVjDCBG2YA0jRNSpEhuGkV2YhDWMEGEL1jBChC1YwwgRtmANI0TYgjWMEGEL1jBC\nxP8Dmgm033vNzsQAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 288x288 with 16 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c_ANekWvTJuV",
        "colab_type": "text"
      },
      "source": [
        "Now let's inspect our model. \n",
        "\n",
        "We will try to encode some number from dataset and then try to decode it."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NmuU3TwEWGAL",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 282
        },
        "outputId": "b844faef-1dbc-4ea1-b1d8-b83b47c5d29a"
      },
      "source": [
        "sample = test_images[0]\n",
        "show_num(sample)\n",
        "sample.shape"
      ],
      "execution_count": 27,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAK70lEQVR4nO3dT4ic9R3H8c+nVi/qIWmGZYmhayUU\nQqFRhlBQxGKVmEv0IuYgKQjrQUHBQ8Ue6jGUqvRQhLUG02KVgoo5hNY0CCIUcZQ0fwxtrKyYsGYn\n5GA82ei3h30iY9zZnczzPPM87vf9gmGe+c2z83x5yCfPM7/f88zPESEAa9/3mi4AwGQQdiAJwg4k\nQdiBJAg7kMT3J7mxDRs2xMzMzCQ3CaQyPz+vs2fPern3SoXd9nZJv5d0haQ/RsSeldafmZlRr9cr\ns0kAK+h2u0PfG/s03vYVkv4g6S5JWyTtsr1l3M8DUK8y39m3SfowIj6KiC8kvSxpZzVlAahambBv\nlPTJwOtTRds32J613bPd6/f7JTYHoIzae+MjYi4iuhHR7XQ6dW8OwBBlwn5a0qaB19cVbQBaqEzY\n35W02fb1tq+SdJ+k/dWUBaBqYw+9RcQF2w9L+ruWht72RsTxyioDUKlS4+wRcUDSgYpqAVAjLpcF\nkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARh\nB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJFFqymbb85LO\nS/pS0oWI6FZRFIDqlQp74ecRcbaCzwFQI07jgSTKhj0kvWH7Pduzy61ge9Z2z3av3++X3ByAcZUN\n+y0RcZOkuyQ9ZPvWS1eIiLmI6EZEt9PplNwcgHGVCntEnC6eFyW9JmlbFUUBqN7YYbd9te1rLy5L\nulPSsaoKA1CtMr3xU5Jes33xc/4SEX+rpCoAlRs77BHxkaSfVlgLgBox9AYkQdiBJAg7kARhB5Ig\n7EASVdwIk14x/FibiKj185EDR3YgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIJx9hHVPZbe1m2vVRmv\nXeDIDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJMM4+oibHZRlnr95q+3QtjsNzZAeSIOxAEoQdSIKw\nA0kQdiAJwg4kQdiBJBhn/w5Yi2O+VShz/UHGfbrqkd32XtuLto8NtK23fdD2yeJ5Xb1lAihrlNP4\nFyRtv6TtcUmHImKzpEPFawAttmrYI+ItSecuad4paV+xvE/S3RXXBaBi43bQTUXEQrH8qaSpYSva\nnrXds93r9/tjbg5AWaV742Opp2Nob0dEzEVENyK6nU6n7OYAjGncsJ+xPS1JxfNidSUBqMO4Yd8v\naXexvFvS69WUA6Auowy9vSTpn5J+bPuU7Qck7ZF0h+2Tkn5RvAbQYqteVBMRu4a8dXvFtQCoEZfL\nAkkQdiAJwg4kQdiBJAg7kAS3uKK1yv6EdsbbWFfCkR1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkmCc\nHY1hKurJ4sgOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kwzo7vLO5Xvzwc2YEkCDuQBGEHkiDsQBKE\nHUiCsANJEHYgCcbZUSvuWW+PUeZn32t70faxgbYnbZ+2fbh47Ki3TABljXIa/4Kk7cu0PxMRW4vH\ngWrLAlC1VcMeEW9JOjeBWgDUqEwH3cO2jxSn+euGrWR71nbPdq/f75fYHIAyxg37s5JukLRV0oKk\np4atGBFzEdGNiG6n0xlzcwDKGivsEXEmIr6MiK8kPSdpW7VlAajaWGG3PT3w8h5Jx4atC6AdVh1n\nt/2SpNskbbB9StJvJN1me6ukkDQv6cEaa0RS3K9erVXDHhG7lml+voZaANSIy2WBJAg7kARhB5Ig\n7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCn5JGKWV+KppbWCeLIzuQBGEHkiDs\nQBKEHUiCsANJEHYgCcIOJME4O1bElMtrB0d2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCcfbk6h5H\n55719lj1yG57k+03bX9g+7jtR4r29bYP2j5ZPK+rv1wA4xrlNP6CpMciYoukn0l6yPYWSY9LOhQR\nmyUdKl4DaKlVwx4RCxHxfrF8XtIJSRsl7ZS0r1htn6S76yoSQHmX1UFne0bSjZLekTQVEQvFW59K\nmhryN7O2e7Z7/X6/RKkAyhg57LavkfSKpEcj4rPB92KpF2bZnpiImIuIbkR0O51OqWIBjG+ksNu+\nUktBfzEiXi2az9ieLt6flrRYT4kAqjBKb7wlPS/pREQ8PfDWfkm7i+Xdkl6vvjy0XUSs+EB7jDLO\nfrOk+yUdtX24aHtC0h5Jf7X9gKSPJd1bT4kAqrBq2CPibUnDrry4vdpyANSFy2WBJAg7kARhB5Ig\n7EAShB1Igltc17iyt7AyVr52cGQHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQYZ18DmFYZo+DIDiRB\n2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0mMMj/7Jttv\n2v7A9nHbjxTtT9o+bftw8dhRf7kAxjXKj1dckPRYRLxv+1pJ79k+WLz3TET8rr7yAFRllPnZFyQt\nFMvnbZ+QtLHuwgBU67K+s9uekXSjpHeKpodtH7G91/a6IX8za7tnu9fv90sVC2B8I4fd9jWSXpH0\naER8JulZSTdI2qqlI/9Ty/1dRMxFRDciup1Op4KSAYxjpLDbvlJLQX8xIl6VpIg4ExFfRsRXkp6T\ntK2+MgGUNUpvvCU9L+lERDw90D49sNo9ko5VXx6AqozSG3+zpPslHbV9uGh7QtIu21slhaR5SQ/W\nUiFqxZTMeYzSG/+2pOV+mPxA9eUAqAtX0AFJEHYgCcIOJEHYgSQIO5AEYQeSYMrmNYCxcoyCIzuQ\nBGEHkiDsQBKEHUiCsANJEHYgCcIOJOFJjtHa7kv6eKBpg6SzEyvg8rS1trbWJVHbuKqs7YcRsezv\nv0007N/auN2LiG5jBaygrbW1tS6J2sY1qdo4jQeSIOxAEk2Hfa7h7a+krbW1tS6J2sY1kdoa/c4O\nYHKaPrIDmBDCDiTRSNhtb7f9b9sf2n68iRqGsT1v+2gxDXWv4Vr22l60fWygbb3tg7ZPFs/LzrHX\nUG2tmMZ7hWnGG913TU9/PvHv7LavkPQfSXdIOiXpXUm7IuKDiRYyhO15Sd2IaPwCDNu3Svpc0p8i\n4idF228lnYuIPcV/lOsi4lctqe1JSZ83PY13MVvR9OA045LulvRLNbjvVqjrXk1gvzVxZN8m6cOI\n+CgivpD0sqSdDdTRehHxlqRzlzTvlLSvWN6npX8sEzektlaIiIWIeL9YPi/p4jTjje67FeqaiCbC\nvlHSJwOvT6ld872HpDdsv2d7tuliljEVEQvF8qeSpposZhmrTuM9SZdMM96afTfO9Odl0UH3bbdE\nxE2S7pL0UHG62kqx9B2sTWOnI03jPSnLTDP+tSb33bjTn5fVRNhPS9o08Pq6oq0VIuJ08bwo6TW1\nbyrqMxdn0C2eFxuu52ttmsZ7uWnG1YJ91+T0502E/V1Jm21fb/sqSfdJ2t9AHd9i++qi40S2r5Z0\np9o3FfV+SbuL5d2SXm+wlm9oyzTew6YZV8P7rvHpzyNi4g9JO7TUI/9fSb9uooYhdf1I0r+Kx/Gm\na5P0kpZO6/6npb6NByT9QNIhSScl/UPS+hbV9mdJRyUd0VKwphuq7RYtnaIfkXS4eOxoet+tUNdE\n9huXywJJ0EEHJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0n8HwkFizSaYGWRAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(28, 28, 1)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 27
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ps3dCwFRdgBN",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def reparam_and_decode(model, mean, logvar):\n",
        "    model.reparameterize(mean, logvar)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_Uy4OUgZWc69",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "f5b2b596-f20c-4e6f-8225-3c22f8f0f968"
      },
      "source": [
        "batch = tf.data.Dataset.from_tensor_slices(train_images).shuffle(BATCH_SIZE).batch(BATCH_SIZE)\n",
        "\n",
        "# iterator = sample_dataset.make_one_shot_iterator()\n",
        "# el = iterator.get_next()\n",
        "\n",
        "iterator = tf.compat.v1.data.make_one_shot_iterator(batch)\n",
        "el = iterator.get_next()\n",
        "mean, logvar = model.encode(el)\n",
        "print(mean.shape)\n",
        "z = model.reparameterize(mean, logvar)\n",
        "debatch = model.generative_net(z.numpy())\n",
        "de_im1 = debatch.numpy()[0]\n",
        "de_im2 = debatch.numpy()[1]\n",
        "show_num(de_im1)\n",
        "show_num(de_im2)\n",
        "im1 = el.numpy()[0]\n",
        "im2 = el.numpy()[1]\n",
        "show_num(im1)\n",
        "show_num(im2)\n",
        "z1 = z.numpy()[0]\n",
        "z2 = z.numpy()[1]\n",
        "print(z1,z2)"
      ],
      "execution_count": 195,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "(100, 50)\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAUxElEQVR4nO2dS4xd5ZWF1/YDbJfLj/KjXH5jGxDP\nJlEJWgqKaEUdARPIBIVBREuozQCkRMqgEQzCELU6iTJoRXIaFKeVJoqUIBig7tAoUpRJRAFuYxsa\nY1MGm/L77QJjl3cP6hIVps5al3vqPtT/+qRS3bq7/nP+c+5Zdere9e+9IzNhjPn/z6xuT8AY0xks\ndmMKwWI3phAsdmMKwWI3phDmdHJnCxYsyCVLllTG582bR8fPnTu3MlbXVagzfmJiota2I4LGZ8+e\n3fL+1b7bfd5YXB2XiqvzNmtW6/cytW1FO/f9ySefVMaOHTuGc+fOTbuBWmKPiHsB/AzAbAD/lpnP\nst9fsmQJHnvsscr4li1b6P5Wr15dGasruE8//ZTG2Qtw5swZOlbNTV3UixcvpvHz589Xxj777DM6\nVsUVly5dajnO/vA3E58zh1++8+fPr4zV/UOh9q1uXGz/11xzDR27c+fOytjTTz9dGWv5z09EzAbw\nrwDuA3AzgIcj4uZWt2eMaS913rPfCeD9zNyfmZ8B+A2AB2ZmWsaYmaaO2NcA+GjKzwcbz32BiNga\nESMRMTI+Pl5jd8aYOrT90/jM3JaZw5k5vGDBgnbvzhhTQR2xHwKwbsrPaxvPGWN6kDpifx3A9RFx\nXURcA+C7AF6emWkZY2aalq23zLwcEU8A+C9MWm/PZ+ZuNmbhwoW46667KuObN2+m+2R2h7JSlP2l\nrDf2ecPQ0BAdq+wptn6gGa5cuVIZU8el5nbhwgUa7+vro3E2t4GBATpW2V9q38zCUttWluTSpUtp\nXFlv7FpWrxmLs/3W8tkz8xUAr9TZhjGmM3i5rDGFYLEbUwgWuzGFYLEbUwgWuzGFYLEbUwgdzWef\nNWsW9UZVGirzRlmOL6DTBi9fvkzjbG7Kw1c+utq3iqv9M06fPk3jddYfAEB/f39l7OLFi3QsS1Ft\nBuY5q2tNXS8qrpaGs/UNKuW51RoEvrMbUwgWuzGFYLEbUwgWuzGFYLEbUwgWuzGF0FHrDeC2gbKo\nmMWkUhLPnj1L48pCYlYJS+NsZt8qzVTNjcVPnjxJxyrrTVma1157LY2zCrFr166lYwcHB2lcpZmy\n9FxmCQL6WlT2mLoe2TVz7ty5lrfNtus7uzGFYLEbUwgWuzGFYLEbUwgWuzGFYLEbUwgWuzGF0HGf\nnZXQVeV9maervEnlZSs/uo7Hr7at4idOnKDxsbGxyti+fftq7VudN5WGyjrvrlu3rjIGAJs2baJx\nNX7Dhg2VMZUWXMfDbwaWOqy2zdZGsOPynd2YQrDYjSkEi92YQrDYjSkEi92YQrDYjSkEi92YQuh4\nKWlW3rdOuWeVX6xywlVe9/nz5ytjKuf7yJEjNM58cgDYv38/jb/33nuVMTZvQLe6VqiSyey8q3Ou\nXjN13tn1otZ0qBoF6rhV+W+2NqPOcbF6EbXEHhGjAM4BmABwOTOH62zPGNM+ZuLO/neZeXwGtmOM\naSN+z25MIdQVewL4Q0S8ERFbp/uFiNgaESMRMXLq1KmauzPGtEpdsd+dmV8HcB+AxyPim1f/QmZu\ny8zhzBxWyQXGmPZRS+yZeajx/SiAFwHcOROTMsbMPC2LPSL6IqL/88cAvg1g10xNzBgzs9T5NH4Q\nwIsNn3YOgP/IzP9kAzKTtulVnjCjbl628jZZ/vGhQ4foWOWT79y5k8aVT89qBAwNDdGxy5Yta3nb\ngG7pzOoMHD16lI5V14Pystm1pmr5L1++nMZXrlxJ46xePsD9cLVmhB1XW3z2zNwP4G9aHW+M6Sy2\n3owpBIvdmEKw2I0pBIvdmEKw2I0phI6muE5MTODMmTOVcVWWmNk4Kk1ULdVVNhCz9t599106dseO\nHTSubD9WEhkANm7cWBlT1lqdNtmAtsfYeVeWpTovu3fvpvHDhw9Xxq677jo6VllrytJU8cWLF1fG\nlN2p0m+r8J3dmEKw2I0pBIvdmEKw2I0pBIvdmEKw2I0pBIvdmELoqM9+6dIl6n2y1D2A+67Hj/Oa\nlyoVU6U8jo6OVsbeeustOlYd12233Ubjt956K40vWrSoMqZKJtdJEwW0T8/abKtUTvWaqtfsgw8+\nqIypNtg33ngjjSuvmx23Gt/f30/HsvUmbtlsjLHYjSkFi92YQrDYjSkEi92YQrDYjSkEi92YQuio\nz37lyhXaqlblTjNvUvnFKjf62LFjNM68dOXZ3nTTTTR+++230/j69etpnHm6rLQwAFpfoJnxdXz4\nVatW0bEDAwM0fuHCBRpn+fLKw9+7dy+Nq/biKs7WPyifnW2bteD2nd2YQrDYjSkEi92YQrDYjSkE\ni92YQrDYjSkEi92YQuiozw5wL115un19fS2PVfXNVQ1zVldetfe95ZZbaHz16tU0vnTpUhpnvqtq\nHXzw4EEaZ74tALpuQo1X+eyqfrqCvS779u2jY1UfggMHDtD4vHnzaHzhwoWVMXU9sdoMbC2KvLNH\nxPMRcTQidk15biAiXo2IvY3v/Go0xnSdZv6N/yWAe6967kkAr2Xm9QBea/xsjOlhpNgz808Aru59\n9ACA7Y3H2wE8OMPzMsbMMK1+QDeYmZ+/qTkMYLDqFyNia0SMRMSIet9sjGkftT+Nz8lPxio/HcvM\nbZk5nJnD7EMJY0x7aVXsRyJiCAAa33kLVGNM12lV7C8DeKTx+BEAL83MdIwx7UIamRHxAoB7ACyP\niIMAfgTgWQC/jYhHARwA8FAzO5s1axYWLFhQGVd+MvN0la+pPi9gNcYB7l+uWbOGjlW9ulXetqpB\nzrxsla+u8vzVebt06RKNs/UP6rjUa1onruq+q+NS9Q9YfwSAXzNqbQN7O8zWLkixZ+bDFaFvqbHG\nmN7By2WNKQSL3ZhCsNiNKQSL3ZhCsNiNKYSOp7gyK0aVBmZ2iSpDPT4+TuOsDS4AzJ8/vzKmLEOV\nqqnKMatjU2mmjNOnT9O4ek2UdcdKfKtyy8qaUymyLL1XWWvquE6evDpd5IuwlGiAp6mq42o1Tdx3\ndmMKwWI3phAsdmMKwWI3phAsdmMKwWI3phAsdmMKoaM+++zZs6n3qcoeMy9ceZMqpVHBPF/lB6sy\n18pHVymPrP2v8rLZ+gGA+8GA9vjZ3FSK6uLFi2lcrW9g1wSbF6BbgI+OjtL4qVOnaFz5/Az2mrLj\n8p3dmEKw2I0pBIvdmEKw2I0pBIvdmEKw2I0pBIvdmELouM/OvNM6nrDyRRXKd2VeuPLJ6+5b+dHs\nvA0OVnbmAgBa2hsA5s6d2/K+VXzVqlV07MqVK2m8TilpdVxq/cCKFStoXOW7sxLfql10f39/ZYzp\nwHd2YwrBYjemECx2YwrBYjemECx2YwrBYjemECx2Ywqhoz57RNDcb9aKFuA1zFVOufKD+/r6aJz5\npsqTVfnoyi9etGgRjS9btqwyps6L8ovVsanzynLKlY++fPlyGldrBFjOuMrjV2sf1HErWK8AtWaE\nzY1da/LOHhHPR8TRiNg15blnIuJQROxofN2vtmOM6S7N/Bv/SwD3TvP8TzPzjsbXKzM7LWPMTCPF\nnpl/AsDX/hljep46H9A9ERE7G//mVxYDi4itETESESOqr5gxpn20KvafA9gM4A4AYwB+XPWLmbkt\nM4czc1gVlDTGtI+WxJ6ZRzJzIjOvAPgFgDtndlrGmJmmJbFHxNCUH78DYFfV7xpjegPps0fECwDu\nAbA8Ig4C+BGAeyLiDgAJYBTAY83ukNVQVz2xGcoXZTnAgPZ0mcevaqvXjau3P+zYVX1y5aOrtQ91\n1jeotQ0q51wdG4ura02dF+WFq7UV7NhUn4FWkWLPzIenefq5NszFGNNGvFzWmEKw2I0pBIvdmEKw\n2I0pBIvdmELoeIors4nmzOHTYXaGSllUcZVGysar9ryqrPDAwACNs1bVAJ+7soiUvaXKZKu2yuw1\nU9abarOt7C22PPvs2bN0rDrnddOaaSqqsJHZeWG2ne/sxhSCxW5MIVjsxhSCxW5MIVjsxhSCxW5M\nIVjsxhRCR332zKxV3peleh4+fJiOVeWaWTlmADh48GBljLXfBYDR0VEaV8et0kxXr15dGVOpuwpV\nMln59Oy8Ky9apc+qMmcsLVn57Co+Pj5O4+q8sWNn8wa4D888eN/ZjSkEi92YQrDYjSkEi92YQrDY\njSkEi92YQrDYjSmEnvLZVQte5iEqP/n48eM0rso1s9bGypNlHj1Qv/0v8/kHBwfpWJXHz1ouA/VK\nSau1D6rEtqoT8PHHH7cUayau1lao64mtT1BlrtlrYp/dGGOxG1MKFrsxhWCxG1MIFrsxhWCxG1MI\nFrsxhdDxuvHMd63j6S5dupSOXbduHY2r2u/M81W+qMq137NnD40fOnSIxjdu3FgZW79+PR2rzpuq\n7a7WCLC1E6r2uoqr1+zYsWOVMVVjQPnsdesjsBoGql4+u95q+ewRsS4i/hgReyJid0R8v/H8QES8\nGhF7G9/5VWOM6SrN/Bt/GcAPM/NmAH8L4PGIuBnAkwBey8zrAbzW+NkY06NIsWfmWGa+2Xh8DsA7\nANYAeADA9savbQfwYLsmaYypz1f6gC4iNgL4GoC/ABjMzLFG6DCAaRdhR8TWiBiJiBFVM8wY0z6a\nFntELATwOwA/yMwvZH7kZDe5aTvKZea2zBzOzGGVHGCMaR9NiT0i5mJS6L/OzN83nj4SEUON+BCA\no+2ZojFmJpDWW0zWvH0OwDuZ+ZMpoZcBPALg2cb3l9S2Zs+eTVMqlc3D0mNVqubQ0BCNq/K9zBZU\nqbn79++n8d27d9P4Rx99RONHj1b/nd27dy8dqywi1U66v7+fxlkqp7LWLl68SOPqbSGzPFVLZmVJ\n3nDDDTSurF523lnbZQUrUd2Mz/4NAN8D8HZE7Gg89xQmRf7biHgUwAEAD7U8Q2NM25Fiz8w/A6j6\nc/GtmZ2OMaZdeLmsMYVgsRtTCBa7MYVgsRtTCBa7MYXQ0RTXK1eu1EoVbbVVLaD94g0bNrQ8XpVr\nVh7/ypUraVz59B9++GFljKV5AjqVU6Udq3bSzGefmJigYy9fvkzjdVo+q9db+ehr166lcXVNsLmp\n88Li7PXynd2YQrDYjSkEi92YQrDYjSkEi92YQrDYjSkEi92YQuiozz4xMUHL/6rcauYvMg8e0H7x\n4sWLaZyVTFY++ebNm2l806ZNNM58dIDnuyuPXrWTVj69aqvMctKZBw/oGgVq7QRb36BafKtrUV0v\n6npk60LU+gF2LbKxvrMbUwgWuzGFYLEbUwgWuzGFYLEbUwgWuzGFYLEbUwgdz2dntcJZXXigXvtf\n5V2qmvXM01Vti5WfvGXLFho/f/48jY+NjVXGlEev2kmfOHGiVpy9Lmrtg6rHP2cOv3zZ9lk+OaBf\nU3WtnjlzhsZZy2Z1XpiHz2rO+85uTCFY7MYUgsVuTCFY7MYUgsVuTCFY7MYUgsVuTCE00599HYBf\nARgEkAC2ZebPIuIZAP8I4POE56cy8xW2rblz52LFihWVcdUTu45vqnxR5W0y/1LlXSvU3NXc2HlT\nHr6q1X/27Fkar9MjXa2NUHFVX53l0qua9AqVr662z3x2tW22ZqRuf/bLAH6YmW9GRD+ANyLi1Ubs\np5n5L01swxjTZZrpzz4GYKzx+FxEvANgTbsnZoyZWb7Se/aI2AjgawD+0njqiYjYGRHPR8S0/0tG\nxNaIGImIEVaSyhjTXpoWe0QsBPA7AD/IzLMAfg5gM4A7MHnn//F04zJzW2YOZ+awek9ujGkfTYk9\nIuZiUui/zszfA0BmHsnMicy8AuAXAO5s3zSNMXWRYo/Jj/eeA/BOZv5kyvNTS3d+B8CumZ+eMWam\naObT+G8A+B6AtyNiR+O5pwA8HBF3YNKOGwXwmNpQZlILTNlALOVRWWvz5s2jcWXjsJTHOmObGa/S\nb1mqpxrL7ClAW0jqcxjWuljZduo1VdcLs0vrWrEKZZ8xVOrukSNHKmNs3s18Gv9nANOZd9RTN8b0\nFl5BZ0whWOzGFILFbkwhWOzGFILFbkwhWOzGFEJHS0lfvHgRBw4cqIyrVM+FCxe2vG/lBys/mvmy\nFy5coGNZe95mUF44S7FVfjLzogFgfHycxpUPz8pg1/XRVYluNl6NVah1G+rY2DWhjptdy+z18J3d\nmEKw2I0pBIvdmEKw2I0pBIvdmEKw2I0pBIvdmEII5bPO6M4ijgGYarQvB3C8YxP4avTq3Hp1XoDn\n1iozObcNmTltvfaOiv1LO48Yyczhrk2A0Ktz69V5AZ5bq3Rqbv433phCsNiNKYRui31bl/fP6NW5\n9eq8AM+tVToyt66+ZzfGdI5u39mNMR3CYjemELoi9oi4NyL+NyLej4gnuzGHKiJiNCLejogdETHS\n5bk8HxFHI2LXlOcGIuLViNjb+N6VnloVc3smIg41zt2OiLi/S3NbFxF/jIg9EbE7Ir7feL6r547M\nqyPnrePv2SNiNoD3APw9gIMAXgfwcGbu6ehEKoiIUQDDmdn1BRgR8U0A5wH8KjNvbTz3zwBOZuaz\njT+USzPzn3pkbs8AON/tNt6NbkVDU9uMA3gQwD+gi+eOzOshdOC8dePOfieA9zNzf2Z+BuA3AB7o\nwjx6nsz8E4CTVz39AIDtjcfbMXmxdJyKufUEmTmWmW82Hp8D8Hmb8a6eOzKvjtANsa8B8NGUnw+i\nt/q9J4A/RMQbEbG125OZhsHMHGs8Pgygur9Sd5BtvDvJVW3Ge+bctdL+vC7+gO7L3J2ZXwdwH4DH\nG/+u9iQ5+R6sl7zTptp4d4pp2oz/lW6eu1bbn9elG2I/BGDdlJ/XNp7rCTLzUOP7UQAvovdaUR/5\nvINu4/vRLs/nr/RSG+/p2oyjB85dN9ufd0PsrwO4PiKui4hrAHwXwMtdmMeXiIi+xgcniIg+AN9G\n77WifhnAI43HjwB4qYtz+QK90sa7qs04unzuut7+PDM7/gXgfkx+Ir8PwNPdmEPFvDYB+J/G1+5u\nzw3AC5j8t+4SJj/beBTAMgCvAdgL4L8BDPTQ3P4dwNsAdmJSWENdmtvdmPwXfSeAHY2v+7t97si8\nOnLevFzWmELwB3TGFILFbkwhWOzGFILFbkwhWOzGFILFbkwhWOzGFML/AeR0OmrOzd6AAAAAAElF\nTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAUB0lEQVR4nO3dXYxd1XkG4Pf1gMHzZ3v8Mx45QCjg\nC0CqU42sSkEVVdSIcAO5QfFFRCVU5yJIiZSLIriIL1HVJMpFFckpKE6VEkVKEFygNhRFQrmJGJAL\nBtoaLKwwDB6PbfAf+G++XsymGmD2907OPrP3Udf7SNbMnDXr7HX22Z/PmfOtby1GBMzs/791XQ/A\nzNrhYDcrhIPdrBAOdrNCONjNCnFNmwcbHR2NLVu21LZv2LAh7X/NNfXDXVxcTPs2bc+yFk3vu0sk\n17R93br615OhoaG0r2pX1NiaUFks1Z5dE6rvxYsXa9sWFhZw9uzZFR94o2AneQ+AHwMYAvDPEfF4\n9vtbtmzBY489Vtt+xx13pMfbunVrbdu5c+fSvh999FHarvpnT8CZM2fSvh9//HHarmQBAwBXr17t\nue+1117bqD37DxjI/wPfuHFj2le1q/8MsseuAkqdt0uXLqXtV65cSdsvXLhQ25Y9nwBw5MiR2rb9\n+/fXtvX8Np7kEIB/AvA1ALcD2Evy9l7vz8zWVpO/2fcAeCsijkbEJQC/BHBff4ZlZv3WJNh3Avjj\nsp/frW77FJL7SM6QnFFvlc1s7az5p/ERcSAipiNienR0dK0PZ2Y1mgT7LIAblv38heo2MxtATYL9\nJQC3kbyZ5HoA3wDwbH+GZWb91nPqLSKukHwYwL9jKfX2ZES8nvUZHh7G7t27a9t37dqVHjNL86hU\nyuXLl9P2Dz74IG3PPm/Yvn172vf8+fNpu8oHqzRPlh5TKaT169en7SoNND4+nrZff/31tW2bN29O\n+ypjY2M991XnRaVL1fWmnrMs9aZSuZOTk7Vt2bXQKM8eEc8BeK7JfZhZOzxd1qwQDnazQjjYzQrh\nYDcrhIPdrBAOdrNCtFrPvm7dOoyMjPTcv0k9+3XXXZe2qzx8lm9W5YyqTFTlslVOOKPKQNXY1XlV\nY8+Or855lqMHmj3n6tjqvGV5ckCf16zkWpVjZ/ed5f/9ym5WCAe7WSEc7GaFcLCbFcLBblYIB7tZ\nIVpNvUVEmvJQqZSsbFCVHKoy0w8//DBtP3nyZM/3rZbjUukrlebJzotKManzpsY2PDyctk9MTNS2\n7dz5uVXMPmXTpk2N2rOUp3rcKv2lSliz5Z6BvKT69OnTad9el6H2K7tZIRzsZoVwsJsVwsFuVggH\nu1khHOxmhXCwmxWi1Tz71atX03y1KmnMcroqn6xKDk+cOJG2Z+PO2lZz7KZzALJ2laNXx25aOpxt\n0X3TTTelfe+88860XY0tW2pa5cFVHl49p6osucmWzdm17jy7mTnYzUrhYDcrhIPdrBAOdrNCONjN\nCuFgNytE63n2s2fP1raruu9sG12Vm2yab876q5pvldNVefSFhYW0PZsjMDs7m/ZVcwRUvljVs+/Y\nsaO2relS0mqJ7iyXrZaKVteTGlu27Lnqr/pmW3xnbY2CneQ7AM4CuArgSkRMN7k/M1s7/Xhl/+uI\nyF96zKxz/pvdrBBNgz0A/JbkyyT3rfQLJPeRnCE5c+bMmYaHM7NeNX0bf1dEzJLcDuB5kv8VES8u\n/4WIOADgAADceuut+aceZrZmGr2yR8Rs9XUewNMA9vRjUGbWfz0HO8kRkmOffA/gqwAO92tgZtZf\nTd7GTwJ4usrrXQPgXyPi37IOly9fxvz8fG27yn1m9ckqL6pysiq3uXHjxto2lUdXefgsNwrouu1s\n/sGpU6fSvtm8B0A/J9n650A+d0Ll8NVzpnL82fbgKk++YcOGtL3J9QLk69Kr60XNCanTc7BHxFEA\nf95rfzNrl1NvZoVwsJsVwsFuVggHu1khHOxmhWi1xHVxcRHZlNls2WEgTxOpFJFa+lelebL7z1I8\ngE6lqLGrFFXWvn79+rRvls4EdNpPbW2cpeaOHTuW9lVje/vtt9P2bLvoqamptG+T6wHQ5dpZulaV\nPGfXk5eSNjMHu1kpHOxmhXCwmxXCwW5WCAe7WSEc7GaFaDXPDuSlgSqnm5WCqr4qb6q2Hs7a1bEV\nlVdV5ZZZuyq1zMpjgeblu03uW+WqVXluVt47Pj6e9lXzE9S8DVUCmz02NXchO2/Os5uZg92sFA52\ns0I42M0K4WA3K4SD3awQDnazQrSaZx8aGkprv1Vu89KlS7VtamlgpclyzqqeXWmS4wfyx75p06a0\nr8pVqzy6Glv2nKrnWy0PrrZ8zvLVan6BOi/qemsytiZzG5xnNzMHu1kpHOxmhXCwmxXCwW5WCAe7\nWSEc7GaFaDXPHhFpHlDVL2fro6v64aZb8Gb5ZDVupen2wVl/VcevcrrZOv+AXj89G/vo6GjaV41d\nyeZlqDy6mgOg8ujqOcvy/KpWPuvbKM9O8kmS8yQPL7ttguTzJI9UXzer+zGzbq3mbfzPANzzmdse\nAfBCRNwG4IXqZzMbYDLYI+JFAJ9d3+c+AAer7w8CuL/P4zKzPuv1A7rJiJirvn8fwGTdL5LcR3KG\n5EzTv23NrHeNP42PpU8Eaj8ViIgDETEdEdPqAxkzWzu9BvtxklMAUH2d79+QzGwt9BrszwJ4sPr+\nQQDP9Gc4ZrZWZJ6d5FMA7gawleS7AL4P4HEAvyL5EIBjAB5YzcEWFxdx/vz52vYmNcLDw8Np3yzn\nCug90LPcZ5M1wgE9tibtp0+fTvueOHEibVdjV/XyWb5a1cKrHL6qd8+eM1Wnn12ngN4rINuXHsj3\nClB91XmrI4M9IvbWNH2lpyOaWSc8XdasEA52s0I42M0K4WA3K4SD3awQrW/ZnKVL1HLOY2NjtW0q\nTdM0jZNRZaJNt0VW6a8sVTM3N1fbBgAnT55M21VKUqUFL1y4kLZn1HbTSpZeU8+JKnFV/dW1nKWR\n1bXaK7+ymxXCwW5WCAe7WSEc7GaFcLCbFcLBblYIB7tZIVrfsjnLlTcpl1QlrqoMVbVnOVvVV5VD\nqnJLtWxxlofPtgYGdB5dLees+mdjU2WkExMTjY6d5em3b9+e9lVlpGoOgHrOs3JuNe8imxOS5ej9\nym5WCAe7WSEc7GaFcLCbFcLBblYIB7tZIRzsZoVoNc++bt06jIyM1LarpaSzvirXPT4+nrarbXKz\n+1fb/6rte1X9ssr5ZnMXbrzxxrSvyvGrum7VP6unz5ZTBnQdvzpv2XOu5mWoPLpa/0Bdy9k1o/Ls\n2fLg2dwDv7KbFcLBblYIB7tZIRzsZoVwsJsVwsFuVggHu1khWs2zR0SjuvCsr8p7qq2Lm6zVrXL0\nqm5bPW6Vs52amqpt27ZtW9pXPW41NrW9cLZu/OzsbNpXrUmvxpbNy9i8eXPaN5u7AOg6f7UGQXYt\nqzUIsmNn69XLV3aST5KcJ3l42W37Sc6SPFT9u1fdj5l1azVv438G4J4Vbv9RROyu/j3X32GZWb/J\nYI+IFwGcamEsZraGmnxA9zDJV6u3+bV/AJHcR3KG5IyaQ25ma6fXYP8JgFsA7AYwB+AHdb8YEQci\nYjoiptWHHma2dnoK9og4HhFXI2IRwE8B7OnvsMys33oKdpLLcz1fB3C47nfNbDDIPDvJpwDcDWAr\nyXcBfB/A3SR3AwgA7wD41moOFhFpra6qX85qq1VdtconqzXIs/ylqstW7ap+WeXZs7yrygerfcTV\neVVjb7JHuhpblkcHgNHR0dq2puvlK2oOQDY3Q+Xos3kb2Xr1MtgjYu8KNz+h+pnZYPF0WbNCONjN\nCuFgNyuEg92sEA52s0K0WuIK5OkUlebJUnOqzFS1qxRS1j8r4wR06k09bpWCypaqbppyVOdFpc+y\n0mL1uNRyzk22+FbnRT1udd7Uls1Z+mxhYSHtm5XAZteSX9nNCuFgNyuEg92sEA52s0I42M0K4WA3\nK4SD3awQrS8lneUvmyxb1XR7X7XUdJZPVn1VzlZti6xKXLP+artnlQ9W5Zbz8/Np+/Hjx2vb1HLO\nO3bsSNu3bNmStmfnTV0PTZeKViWuTY595syZ2rZsSXW/spsVwsFuVggHu1khHOxmhXCwmxXCwW5W\nCAe7WSFaz7NnecAmuU1VG622TVa58qwmXW1brOrVh4eH03aVC89yuiofrOr8VR796NGjaXtWe71r\n16607+TkZNqu5ieox5ZRcyPUFuFq2+XseVFbVWdzJ7I6e7+ymxXCwW5WCAe7WSEc7GaFcLCbFcLB\nblYIB7tZIVpfNz7LT65lPlnlLlX/bG14lWdXx1Y5XTW2rNa+6brw7733Xto+NzeXtmfbJt98881p\nX7UuvJI9NrXWv6pHV8+pmluRPWeq1r5X8pWd5A0kf0fyDZKvk/xOdfsEyedJHqm+5isRmFmnVvM2\n/gqA70XE7QD+EsC3Sd4O4BEAL0TEbQBeqH42swElgz0i5iLiler7swDeBLATwH0ADla/dhDA/Ws1\nSDNr7k/6gI7kFwF8CcAfAExGxCd/sL0PYMWJzCT3kZwhOaPWiTOztbPqYCc5CuDXAL4bEZ9a8S6W\nPnVb8ZO3iDgQEdMRMZ19WGNma2tVwU7yWiwF+i8i4jfVzcdJTlXtUwDy8igz65RMvXGpdvQJAG9G\nxA+XNT0L4EEAj1dfn1nFfaVpBZVyyNJ2aslklWJSx85KaNV9Z8spA7q0Vy0lPTY2Vtum0pnqT6sT\nJ06k7arUM0uvbdu2Le2rnlOV3srKmptuZd3k2EC+HLQqj82OnT0fq8mzfxnANwG8RvJQddujWAry\nX5F8CMAxAA+s4r7MrCMy2CPi9wDqXta+0t/hmNla8XRZs0I42M0K4WA3K4SD3awQDnazQrS+lHSW\n91VL/2ZLB6u8qFquWbWPjIzUtqmth9Uy1arcUm1lneXCVZ5d5YtVPnrnzp1p+y233FLbpmZUqjJS\ndV6zMtWmy3urPLqSlbgqam5DHb+ymxXCwW5WCAe7WSEc7GaFcLCbFcLBblYIB7tZIVrPs2fLIquc\ncJZ3VTXhamngiYmJtD2rZ1dbB2/YsCFtX1hYSNtVPXyWp1c5WZVP3rp1a9qutlXevn17bZvKs6tc\nuKrFz86Lek7U/AJ1Panzno1N1bNn8zqy4/qV3awQDnazQjjYzQrhYDcrhIPdrBAOdrNCONjNCtFq\nnn1oaChd4zxrA/L101VNuMqrqjx7lk/O1gAHdN21yrOr2umsXa0RoOYnqDy8ypVn51UdW1H17tka\nB+p6UXMn1PoJas37bN6GyuFncZDdr1/ZzQrhYDcrhIPdrBAOdrNCONjNCuFgNyuEg92sEKvZn/0G\nAD8HMAkgAByIiB+T3A/g7wB8smj5oxHxXHZfQ0NDaV5Wrb+e1eqqPcxVXlS1Zzldtc94k726Ab1u\nfHb/an1yVbet2rP19IE8D6/mAKixq1x4Rl0vSlanD+g5I1kuPVvzAciv1eycrGZSzRUA34uIV0iO\nAXiZ5PNV248i4h9XcR9m1rHV7M8+B2Cu+v4syTcB5NuAmNnA+ZP+Zif5RQBfAvCH6qaHSb5K8kmS\nK74HJ7mP5AzJGfV21czWzqqDneQogF8D+G5EnAHwEwC3ANiNpVf+H6zULyIORMR0REyPj4/3Ychm\n1otVBTvJa7EU6L+IiN8AQEQcj4irEbEI4KcA9qzdMM2sKRnsXCqjeQLAmxHxw2W3Ty37ta8DONz/\n4ZlZv6zm0/gvA/gmgNdIHqpuexTAXpK7sZSOewfAt9QdRUS6PLBaQleVW2ZUikjJjq2WwN64cWPa\nvmnTprRdlXI2WUpa3beiylSzdKpKSaoy1CapOZVqVY9LXU8qLZil3lTaLnvc2eNazafxvwewUpFs\nmlM3s8HiGXRmhXCwmxXCwW5WCAe7WSEc7GaFcLCbFaLVpaQvX76M+fn5Rv3rqKWiVb5ZLf2blXpm\ny/cCumRRUWPPHrvKo6tli1U+WuWTs+Or86bKa9Vzlp13tQS2yrNfvHgxbVdzBLK5FeqcZvNRsjkf\nfmU3K4SD3awQDnazQjjYzQrhYDcrhIPdrBAOdrNCUOVw+3ow8gSAY8tu2gog36+4O4M6tkEdF+Cx\n9aqfY7spIlZc27zVYP/cwcmZiJjubACJQR3boI4L8Nh61dbY/DberBAOdrNCdB3sBzo+fmZQxzao\n4wI8tl61MrZO/2Y3s/Z0/cpuZi1xsJsVopNgJ3kPyf8m+RbJR7oYQx2S75B8jeQhkjMdj+VJkvMk\nDy+7bYLk8ySPVF/zfa7bHdt+krPVuTtE8t6OxnYDyd+RfIPk6yS/U93e6blLxtXKeWv9b3aSQwD+\nB8DfAHgXwEsA9kbEG60OpAbJdwBMR0TnEzBI/hWAcwB+HhF3Vrf9A4BTEfF49R/l5oj4+wEZ234A\n57rexrvarWhq+TbjAO4H8Lfo8Nwl43oALZy3Ll7Z9wB4KyKORsQlAL8EcF8H4xh4EfEigFOfufk+\nAAer7w9i6WJpXc3YBkJEzEXEK9X3ZwF8ss14p+cuGVcrugj2nQD+uOzndzFY+70HgN+SfJnkvq4H\ns4LJiJirvn8fwGSXg1mB3Ma7TZ/ZZnxgzl0v25835Q/oPu+uiPgLAF8D8O3q7epAiqW/wQYpd7qq\nbbzbssI24/+ny3PX6/bnTXUR7LMAblj28xeq2wZCRMxWX+cBPI3B24r6+Cc76FZfe1/Bs88GaRvv\nlbYZxwCcuy63P+8i2F8CcBvJm0muB/ANAM92MI7PITlSfXACkiMAvorB24r6WQAPVt8/COCZDsfy\nKYOyjXfdNuPo+Nx1vv15RLT+D8C9WPpE/m0Aj3Uxhppx/RmA/6z+vd712AA8haW3dZex9NnGQwC2\nAHgBwBEA/wFgYoDG9i8AXgPwKpYCa6qjsd2FpbforwI4VP27t+tzl4yrlfPm6bJmhfAHdGaFcLCb\nFcLBblYIB7tZIRzsZoVwsJsVwsFuVoj/BWCma4xewrvcAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAALSUlEQVR4nO3dT6hc9RnG8eepfzbqImmGEGLotZJN\nKDTKEAqKWKQSs4luxCwkBeG6UFDoomIXugylKl0UIdZgWqwiqJhFaE2DIG7EUdL8MbRauWLCNXdC\nFsaVjb5d3KNc4507k/Nnzsl9vx8Y5sxv5ua8OTdPfjPnnZmfI0IAVr8ftV0AgOkg7EAShB1IgrAD\nSRB2IIkrp7mzdevWxczMzDR3CaQyNzens2fPern7KoXd9nZJf5R0haQ/R8SelR4/MzOjwWBQZZcA\nVtDv90feV/ppvO0rJP1J0l2StkjaZXtL2T8PQLOqvGbfJunjiPgkIr6S9LKknfWUBaBuVcK+UdJn\nS26fKsa+x/as7YHtwXA4rLA7AFU0fjY+IvZGRD8i+r1er+ndARihSthPS9q05Pb1xRiADqoS9vck\nbbZ9g+2rJd0n6UA9ZQGoW+nWW0RcsP2wpH9osfW2LyJO1FYZgFpV6rNHxEFJB2uqBUCDeLsskARh\nB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AE\nYQeSIOxAEoQdSIKwA0lMdclmdI+97Oq+UxERre07I2Z2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiC\nPvsq0GavHJePSmG3PSfpvKSvJV2IiH4dRQGoXx0z+y8j4mwNfw6ABvGaHUiiathD0pu237c9u9wD\nbM/aHtgeDIfDirsDUFbVsN8aETdLukvSQ7Zvu/gBEbE3IvoR0e/1ehV3B6CsSmGPiNPF9YKk1yVt\nq6MoAPUrHXbb19i+7tttSXdKOl5XYQDqVeVs/HpJrxc93isl/S0i/l5LVcms5s+Ur/R3q/r35vPw\nl6Z02CPiE0k/r7EWAA2i9QYkQdiBJAg7kARhB5Ig7EASfMR1CppurWVtQY07rlmPyyjM7EAShB1I\ngrADSRB2IAnCDiRB2IEkCDuQBH32GvBRzWaMOy7jjnuV38tq/J0wswNJEHYgCcIOJEHYgSQIO5AE\nYQeSIOxAEvTZJ8SyyOU02a+u2oev8rOXYx+emR1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkqDPPgWX\nY092Nahy3Kt+Vr6Lv/OxM7vtfbYXbB9fMrbW9iHbHxXXa5otE0BVkzyNf0HS9ovGHpN0OCI2Szpc\n3AbQYWPDHhFvSzp30fBOSfuL7f2S7q65LgA1K3uCbn1EzBfbn0taP+qBtmdtD2wPhsNhyd0BqKry\n2fhYPBMx8mxEROyNiH5E9Hu9XtXdASipbNjP2N4gScX1Qn0lAWhC2bAfkLS72N4t6Y16ygHQlLF9\ndtsvSbpd0jrbpyQ9IWmPpFdsPyDpU0n3NlnkNPAd41jtxoY9InaNuOuOmmsB0CDeLgskQdiBJAg7\nkARhB5Ig7EASfMQVWEbTy0W30a5lZgeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJOizAyU0uVx0U5jZ\ngSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiQx\nNuy299lesH18ydiTtk/bPlJcdjRbJoCqJpnZX5C0fZnxZyJia3E5WG9ZAOo2NuwR8bakc1OoBUCD\nqrxmf9j20eJp/ppRD7I9a3tgezAcDivsDkAVZcP+rKQbJW2VNC/pqVEPjIi9EdGPiH6v1yu5OwBV\nlQp7RJyJiK8j4htJz0naVm9ZAOpWKuy2Nyy5eY+k46MeC6Abxn5vvO2XJN0uaZ3tU5KekHS77a2S\nQtKcpAcbrHEqLsfvAQcuxdiwR8SuZYafb6AWAA3iHXRAEoQdSIKwA0kQdiAJwg4kwZLNNRjXlhvX\n1sPl53JsxTKzA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAS9NmngD48uoCZHUiCsANJEHYgCcIOJEHY\ngSQIO5AEYQeSoM8+oZV64VU/20wffvXp4u+MmR1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkqDPXoOm\nl3umDz99l+P3wo8zdma3vcn2W7Y/tH3C9iPF+Frbh2x/VFyvab5cAGVN8jT+gqTfRMQWSb+Q9JDt\nLZIek3Q4IjZLOlzcBtBRY8MeEfMR8UGxfV7SSUkbJe2UtL942H5JdzdVJIDqLukEne0ZSTdJelfS\n+oiYL+76XNL6ET8za3tgezAcDiuUCqCKicNu+1pJr0p6NCK+WHpfLJ4hWvYsUUTsjYh+RPR7vV6l\nYgGUN1HYbV+lxaC/GBGvFcNnbG8o7t8gaaGZEgHUYZKz8Zb0vKSTEfH0krsOSNpdbO+W9Eb95a0O\nEbHipSrbpS+rWZPHpenfaRMm6bPfIul+ScdsHynGHpe0R9Irth+Q9Kmke5spEUAdxoY9It6RNOq/\nujvqLQdAU3i7LJAEYQeSIOxAEoQdSIKwA0nwEdcOqNqXrdIvX+299rK62iuvgpkdSIKwA0kQdiAJ\nwg4kQdiBJAg7kARhB5Kgz74KNNkT7nIffjX2wpvEzA4kQdiBJAg7kARhB5Ig7EAShB1IgrADSdBn\nx4roZa8ezOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kMQk67Nvsv2W7Q9tn7D9SDH+pO3Tto8Ulx3N\nlwugrEneVHNB0m8i4gPb10l63/ah4r5nIuIPzZUHoC6TrM8+L2m+2D5v+6SkjU0XBqBel/Sa3faM\npJskvVsMPWz7qO19tteM+JlZ2wPbg+FwWKlYAOVNHHbb10p6VdKjEfGFpGcl3ShpqxZn/qeW+7mI\n2BsR/Yjo93q9GkoGUMZEYbd9lRaD/mJEvCZJEXEmIr6OiG8kPSdpW3NlAqhqkrPxlvS8pJMR8fSS\n8Q1LHnaPpOP1lwegLpOcjb9F0v2Sjtk+Uow9LmmX7a2SQtKcpAcbqRBALSY5G/+OpOW+PPxg/eUA\naArvoAOSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiThaS7J\na3so6dMlQ+sknZ1aAZemq7V1tS6J2sqqs7afRMSy3/821bD/YOf2ICL6rRWwgq7W1tW6JGora1q1\n8TQeSIKwA0m0Hfa9Le9/JV2trat1SdRW1lRqa/U1O4DpaXtmBzAlhB1IopWw295u+9+2P7b9WBs1\njGJ7zvaxYhnqQcu17LO9YPv4krG1tg/Z/qi4XnaNvZZq68Qy3issM97qsWt7+fOpv2a3fYWk/0j6\nlaRTkt6TtCsiPpxqISPYnpPUj4jW34Bh+zZJX0r6S0T8rBj7vaRzEbGn+I9yTUT8tiO1PSnpy7aX\n8S5WK9qwdJlxSXdL+rVaPHYr1HWvpnDc2pjZt0n6OCI+iYivJL0saWcLdXReRLwt6dxFwzsl7S+2\n92vxH8vUjaitEyJiPiI+KLbPS/p2mfFWj90KdU1FG2HfKOmzJbdPqVvrvYekN22/b3u27WKWsT4i\n5ovtzyWtb7OYZYxdxnuaLlpmvDPHrszy51Vxgu6Hbo2ImyXdJemh4ulqJ8Xia7Au9U4nWsZ7WpZZ\nZvw7bR67ssufV9VG2E9L2rTk9vXFWCdExOniekHS6+reUtRnvl1Bt7heaLme73RpGe/llhlXB45d\nm8uftxH29yRttn2D7asl3SfpQAt1/IDta4oTJ7J9jaQ71b2lqA9I2l1s75b0Rou1fE9XlvEetcy4\nWj52rS9/HhFTv0jaocUz8v+V9Ls2ahhR108l/au4nGi7NkkvafFp3f+0eG7jAUk/lnRY0keS/ilp\nbYdq+6ukY5KOajFYG1qq7VYtPkU/KulIcdnR9rFboa6pHDfeLgskwQk6IAnCDiRB2IEkCDuQBGEH\nkiDsQBKEHUji/+d31lFfKiLDAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAALH0lEQVR4nO3dT6hc9RnG8eepfzbqImmGS4ih10o2\nodAoQygoYpFKzCa6EbOQFITrQkHBhWIXugylKl0UIdZgWqwiqJhFaE2DIG7EUdL8MbSxcsWEa+6E\nLIwrG327mBO5xjt3JnPOmXO87/cDw5z5zdycl4OPv5nzzpmfI0IAVr+fNF0AgOkg7EAShB1IgrAD\nSRB2IIkrp7mzdevWxezs7DR3CaQyPz+vs2fPernnSoXd9jZJf5R0haQ/R8TulV4/OzurXq9XZpcA\nVtDtdoc+N/HbeNtXSPqTpLskbZa00/bmSf89APUq85l9q6RPIuLTiPha0quSdlRTFoCqlQn7Bkmf\nL3l8qhj7Httztnu2e/1+v8TuAJRR+9n4iNgTEd2I6HY6nbp3B2CIMmE/LWnjksfXF2MAWqhM2D+Q\ntMn2DbavlnSfpP3VlAWgahO33iLigu2HJf1Dg9bb3og4XlllACpVqs8eEQckHaioFgA14uuyQBKE\nHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAS\nhB1IgrADSRB2IAnCDiQx1SWb0T72sqv7ji0iKqoEdWNmB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEk\n6LOvAmV75U3tmx79dJUKu+15SeclfSPpQkR0qygKQPWqmNl/HRFnK/h3ANSIz+xAEmXDHpLetv2h\n7bnlXmB7znbPdq/f75fcHYBJlQ37rRFxs6S7JD1k+7ZLXxAReyKiGxHdTqdTcncAJlUq7BFxurhf\nlPSmpK1VFAWgehOH3fY1tq+7uC3pTknHqioMQLXKnI2fkfRm0We9UtLfIuLvlVSFFEb16OnDV2vi\nsEfEp5J+WWEtAGpE6w1IgrADSRB2IAnCDiRB2IEkuMR1FVipRdXk5a9l0ZqrFjM7kARhB5Ig7EAS\nhB1IgrADSRB2IAnCDiRBnx2tRR+9WszsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEffZVblSv+sd8\nvTsuDzM7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRBnz25Nvfh+d34ao2c2W3vtb1o+9iSsbW2D9o+\nWdyvqbdMAGWN8zb+JUnbLhl7QtKhiNgk6VDxGECLjQx7RLwr6dwlwzsk7Su290m6u+K6AFRs0hN0\nMxGxUGx/IWlm2Attz9nu2e71+/0JdwegrNJn42NwlmTomZKI2BMR3YjodjqdsrsDMKFJw37G9npJ\nKu4XqysJQB0mDft+SbuK7V2S3qqmHAB1Gdlnt/2KpNslrbN9StJTknZLes32A5I+k3RvnUWiPm2+\nnp0+erVGhj0idg556o6KawFQI74uCyRB2IEkCDuQBGEHkiDsQBJc4roKtLl9hvZgZgeSIOxAEoQd\nSIKwA0kQdiAJwg4kQdiBJOizo7X4KelqMbMDSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBL02VeBlfrN\nXOuOi5jZgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJ+uyrXNPXfNPnb4+RM7vtvbYXbR9bMva07dO2\nDxe37fWWCaCscd7GvyRp2zLjz0XEluJ2oNqyAFRtZNgj4l1J56ZQC4AalTlB97DtI8Xb/DXDXmR7\nznbPdq/f75fYHYAyJg3785JulLRF0oKkZ4a9MCL2REQ3IrqdTmfC3QEoa6KwR8SZiPgmIr6V9IKk\nrdWWBaBqE4Xd9volD++RdGzYawG0w8g+u+1XJN0uaZ3tU5KeknS77S2SQtK8pAdrrBFABUaGPSJ2\nLjP8Yg21AKgRX5cFkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJ\nfkoapdT5U9FN/wz2asPMDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJ0GevwKheM/1itAEzO5AEYQeS\nIOxAEoQdSIKwA0kQdiAJwg4kQZ99TGWu2/4x9+HrvF4d0zVyZre90fY7tj+2fdz2I8X4WtsHbZ8s\n7tfUXy6ASY3zNv6CpMciYrOkX0l6yPZmSU9IOhQRmyQdKh4DaKmRYY+IhYj4qNg+L+mEpA2Sdkja\nV7xsn6S76yoSQHmXdYLO9qykmyS9L2kmIhaKp76QNDPkb+Zs92z3+v1+iVIBlDF22G1fK+l1SY9G\nxJdLn4vBGaZlzzJFxJ6I6EZEt9PplCoWwOTGCrvtqzQI+ssR8UYxfMb2+uL59ZIW6ykRQBVGtt48\n6L28KOlERDy75Kn9knZJ2l3cv1VLhS2xUnusbHsqa3urzS3H1WicPvstku6XdNT24WLsSQ1C/prt\nByR9JuneekoEUIWRYY+I9yQNm3ruqLYcAHXh67JAEoQdSIKwA0kQdiAJwg4kwSWuUzCqn7ya++z0\n0tuDmR1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkqDPXoGyveSyf99kn54++o8HMzuQBGEHkiDsQBKE\nHUiCsANJEHYgCcIOJEGffRWg141xMLMDSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBIjw257o+13bH9s\n+7jtR4rxp22ftn24uG2vv1wAkxrnSzUXJD0WER/Zvk7Sh7YPFs89FxF/qK88AFUZZ332BUkLxfZ5\n2yckbai7MADVuqzP7LZnJd0k6f1i6GHbR2zvtb1myN/M2e7Z7vX7/VLFApjc2GG3fa2k1yU9GhFf\nSnpe0o2Stmgw8z+z3N9FxJ6I6EZEt9PpVFAygEmMFXbbV2kQ9Jcj4g1JiogzEfFNRHwr6QVJW+sr\nE0BZ45yNt6QXJZ2IiGeXjK9f8rJ7JB2rvjwAVRnnbPwtku6XdNT24WLsSUk7bW+RFJLmJT1YS4UA\nKjHO2fj3JC33w+QHqi8HQF34Bh2QBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxA\nEoQdSIKwA0kQdiAJT3O5X9t9SZ8tGVon6ezUCrg8ba2trXVJ1DapKmv7WUQs+/tvUw37D3Zu9yKi\n21gBK2hrbW2tS6K2SU2rNt7GA0kQdiCJpsO+p+H9r6SttbW1LonaJjWV2hr9zA5gepqe2QFMCWEH\nkmgk7La32f637U9sP9FEDcPYnrd9tFiGutdwLXttL9o+tmRsre2Dtk8W98uusddQba1YxnuFZcYb\nPXZNL38+9c/stq+Q9B9Jv5F0StIHknZGxMdTLWQI2/OSuhHR+BcwbN8m6StJf4mIXxRjv5d0LiJ2\nF/+jXBMRj7ektqclfdX0Mt7FakXrly4zLuluSb9Vg8duhbru1RSOWxMz+1ZJn0TEpxHxtaRXJe1o\noI7Wi4h3JZ27ZHiHpH3F9j4N/mOZuiG1tUJELETER8X2eUkXlxlv9NitUNdUNBH2DZI+X/L4lNq1\n3ntIetv2h7bnmi5mGTMRsVBsfyFppsliljFyGe9pumSZ8dYcu0mWPy+LE3Q/dGtE3CzpLkkPFW9X\nWykGn8Ha1DsdaxnvaVlmmfHvNHnsJl3+vKwmwn5a0sYlj68vxlohIk4X94uS3lT7lqI+c3EF3eJ+\nseF6vtOmZbyXW2ZcLTh2TS5/3kTYP5C0yfYNtq+WdJ+k/Q3U8QO2rylOnMj2NZLuVPuWot4vaVex\nvUvSWw3W8j1tWcZ72DLjavjYNb78eURM/SZpuwZn5P8r6XdN1DCkrp9L+ldxO950bZJe0eBt3f80\nOLfxgKSfSjok6aSkf0pa26La/irpqKQjGgRrfUO13arBW/Qjkg4Xt+1NH7sV6prKcePrskASnKAD\nkiDsQBKEHUiCsANJEHYgCcIOJEHYgST+D4IWkbBHgZPCAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "[ 1.0729237  -0.27297455 -1.1089635  -1.1617047  -0.5291097  -1.9056933\n",
            " -1.756988   -0.07232621  1.0561675   1.1905509   1.9524217  -0.2083906\n",
            "  0.0668323  -1.8458488  -0.82168645  0.7955886  -0.30425265 -1.258932\n",
            "  0.97465575 -0.24790454 -0.04766461 -0.89146936  0.4877107  -2.4468455\n",
            " -1.2177006   0.5732259  -0.10381003  0.04092159 -0.80939186  0.6323729\n",
            "  0.07389399 -2.0906887  -0.70181096 -0.47821075  1.3350972   1.4248744\n",
            " -0.64441884  0.8107734  -1.3560908   0.7595379  -0.5913695  -1.078864\n",
            "  1.682241    2.3412402   1.8519474  -2.5112815   0.27671024 -0.02218395\n",
            " -1.0123135   1.144046  ] [-1.193376   -0.22866966  0.2264108  -1.2022094   1.3075453  -0.7875304\n",
            "  0.20795664 -0.7149406  -0.901653    0.23782776 -1.1632648  -0.02395703\n",
            "  0.48705694  0.9059143  -1.2254543   0.13320577 -0.20976213 -0.7748824\n",
            " -1.9763774  -0.842216    0.50418544 -1.6818014  -0.3955982  -1.4448719\n",
            "  1.2628107   0.66051316 -1.0359397  -0.47967368  0.8925431   0.09342229\n",
            " -0.37738666  1.4156883   1.3673631  -0.7649078   1.1650331   0.55254877\n",
            "  0.26901397 -1.1918436  -0.75930524  1.4094685   0.0572423  -0.4783893\n",
            "  0.41129532  1.1238277  -0.9286683  -1.1495961   0.21244821  0.74507\n",
            " -0.5795182   0.7901203 ]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0ozj21Bbf8Dq",
        "colab_type": "text"
      },
      "source": [
        "How we can see, decoded image is such blured. \n",
        "\n",
        "Let's see, how similar im1 and im2 are. To do this, we can initialize $\\theta \\in [0;1]$ and vary it in function of $\\gamma = \\theta z_1 + (1 - \\theta) z_2,$ which is parameterized function of the segment between $z_1$ and $z_2$ in the space of this vectors.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6lFm2Bw0f6mm",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 265
        },
        "outputId": "fcecdd09-acfc-445b-98e8-b8551ca9cbea"
      },
      "source": [
        "def make_video(arr, i):\n",
        "    # show_num(arr)\n",
        "    # time.sleep(0.5)\n",
        "    img = arr.reshape((28,28))\n",
        "    plt.imshow(img, cmap=\"Greys\")\n",
        "    plt.savefig('capture_of_evol_{}.png'.format(i))\n",
        "    display.clear_output(wait=False)\n",
        "\n",
        "thetas = np.linspace(0., 1., num=99)\n",
        "new_z = [z2]\n",
        "# new_z = pd.DataFrame(z1)\n",
        "\n",
        "# i = 1\n",
        "# for theta in thetas:\n",
        "#     new_z['{}'.format(i)] = z1*theta + (1-theta)*z2\n",
        "#     i += 1\n",
        "# zs = new_z.to_numpy()\n",
        "for theta in thetas:\n",
        "    new_z.append(z1*theta + (1-theta)*z2)\n",
        "\n",
        "# batch_1 = tf.data.Dataset.from_tensor_slices(train_images).shuffle(BATCH_SIZE).batch(BATCH_SIZE)\n",
        "new_z = np.array(new_z)\n",
        "new_z.shape\n",
        "\n",
        "deb = model.generative_net(new_z)\n",
        "deb_np = deb.numpy()\n",
        "for i in range(0,100):\n",
        "    make_video(deb_np[i], i)\n",
        "    \n",
        "# debatch = model.generative_net(zs)"
      ],
      "execution_count": 196,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAUxElEQVR4nO2dS4xd5ZWF1/YDbJfLj/KjXH5jGxDP\nJlEJWgqKaEUdARPIBIVBREuozQCkRMqgEQzCELU6iTJoRXIaFKeVJoqUIBig7tAoUpRJRAFuYxsa\nY1MGm/L77QJjl3cP6hIVps5al3vqPtT/+qRS3bq7/nP+c+5Zdere9e+9IzNhjPn/z6xuT8AY0xks\ndmMKwWI3phAsdmMKwWI3phDmdHJnCxYsyCVLllTG582bR8fPnTu3MlbXVagzfmJiota2I4LGZ8+e\n3fL+1b7bfd5YXB2XiqvzNmtW6/cytW1FO/f9ySefVMaOHTuGc+fOTbuBWmKPiHsB/AzAbAD/lpnP\nst9fsmQJHnvsscr4li1b6P5Wr15dGasruE8//ZTG2Qtw5swZOlbNTV3UixcvpvHz589Xxj777DM6\nVsUVly5dajnO/vA3E58zh1++8+fPr4zV/UOh9q1uXGz/11xzDR27c+fOytjTTz9dGWv5z09EzAbw\nrwDuA3AzgIcj4uZWt2eMaS913rPfCeD9zNyfmZ8B+A2AB2ZmWsaYmaaO2NcA+GjKzwcbz32BiNga\nESMRMTI+Pl5jd8aYOrT90/jM3JaZw5k5vGDBgnbvzhhTQR2xHwKwbsrPaxvPGWN6kDpifx3A9RFx\nXURcA+C7AF6emWkZY2aalq23zLwcEU8A+C9MWm/PZ+ZuNmbhwoW46667KuObN2+m+2R2h7JSlP2l\nrDf2ecPQ0BAdq+wptn6gGa5cuVIZU8el5nbhwgUa7+vro3E2t4GBATpW2V9q38zCUttWluTSpUtp\nXFlv7FpWrxmLs/3W8tkz8xUAr9TZhjGmM3i5rDGFYLEbUwgWuzGFYLEbUwgWuzGFYLEbUwgdzWef\nNWsW9UZVGirzRlmOL6DTBi9fvkzjbG7Kw1c+utq3iqv9M06fPk3jddYfAEB/f39l7OLFi3QsS1Ft\nBuY5q2tNXS8qrpaGs/UNKuW51RoEvrMbUwgWuzGFYLEbUwgWuzGFYLEbUwgWuzGF0FHrDeC2gbKo\nmMWkUhLPnj1L48pCYlYJS+NsZt8qzVTNjcVPnjxJxyrrTVma1157LY2zCrFr166lYwcHB2lcpZmy\n9FxmCQL6WlT2mLoe2TVz7ty5lrfNtus7uzGFYLEbUwgWuzGFYLEbUwgWuzGFYLEbUwgWuzGF0HGf\nnZXQVeV9maervEnlZSs/uo7Hr7at4idOnKDxsbGxyti+fftq7VudN5WGyjrvrlu3rjIGAJs2baJx\nNX7Dhg2VMZUWXMfDbwaWOqy2zdZGsOPynd2YQrDYjSkEi92YQrDYjSkEi92YQrDYjSkEi92YQuh4\nKWlW3rdOuWeVX6xywlVe9/nz5ytjKuf7yJEjNM58cgDYv38/jb/33nuVMTZvQLe6VqiSyey8q3Ou\nXjN13tn1otZ0qBoF6rhV+W+2NqPOcbF6EbXEHhGjAM4BmABwOTOH62zPGNM+ZuLO/neZeXwGtmOM\naSN+z25MIdQVewL4Q0S8ERFbp/uFiNgaESMRMXLq1KmauzPGtEpdsd+dmV8HcB+AxyPim1f/QmZu\ny8zhzBxWyQXGmPZRS+yZeajx/SiAFwHcOROTMsbMPC2LPSL6IqL/88cAvg1g10xNzBgzs9T5NH4Q\nwIsNn3YOgP/IzP9kAzKTtulVnjCjbl628jZZ/vGhQ4foWOWT79y5k8aVT89qBAwNDdGxy5Yta3nb\ngG7pzOoMHD16lI5V14Pystm1pmr5L1++nMZXrlxJ46xePsD9cLVmhB1XW3z2zNwP4G9aHW+M6Sy2\n3owpBIvdmEKw2I0pBIvdmEKw2I0phI6muE5MTODMmTOVcVWWmNk4Kk1ULdVVNhCz9t599106dseO\nHTSubD9WEhkANm7cWBlT1lqdNtmAtsfYeVeWpTovu3fvpvHDhw9Xxq677jo6VllrytJU8cWLF1fG\nlN2p0m+r8J3dmEKw2I0pBIvdmEKw2I0pBIvdmEKw2I0pBIvdmELoqM9+6dIl6n2y1D2A+67Hj/Oa\nlyoVU6U8jo6OVsbeeustOlYd12233Ubjt956K40vWrSoMqZKJtdJEwW0T8/abKtUTvWaqtfsgw8+\nqIypNtg33ngjjSuvmx23Gt/f30/HsvUmbtlsjLHYjSkFi92YQrDYjSkEi92YQrDYjSkEi92YQuio\nz37lyhXaqlblTjNvUvnFKjf62LFjNM68dOXZ3nTTTTR+++230/j69etpnHm6rLQwAFpfoJnxdXz4\nVatW0bEDAwM0fuHCBRpn+fLKw9+7dy+Nq/biKs7WPyifnW2bteD2nd2YQrDYjSkEi92YQrDYjSkE\ni92YQrDYjSkEi92YQuiozw5wL115un19fS2PVfXNVQ1zVldetfe95ZZbaHz16tU0vnTpUhpnvqtq\nHXzw4EEaZ74tALpuQo1X+eyqfrqCvS779u2jY1UfggMHDtD4vHnzaHzhwoWVMXU9sdoMbC2KvLNH\nxPMRcTQidk15biAiXo2IvY3v/Go0xnSdZv6N/yWAe6967kkAr2Xm9QBea/xsjOlhpNgz808Aru59\n9ACA7Y3H2wE8OMPzMsbMMK1+QDeYmZ+/qTkMYLDqFyNia0SMRMSIet9sjGkftT+Nz8lPxio/HcvM\nbZk5nJnD7EMJY0x7aVXsRyJiCAAa33kLVGNM12lV7C8DeKTx+BEAL83MdIwx7UIamRHxAoB7ACyP\niIMAfgTgWQC/jYhHARwA8FAzO5s1axYWLFhQGVd+MvN0la+pPi9gNcYB7l+uWbOGjlW9ulXetqpB\nzrxsla+u8vzVebt06RKNs/UP6rjUa1onruq+q+NS9Q9YfwSAXzNqbQN7O8zWLkixZ+bDFaFvqbHG\nmN7By2WNKQSL3ZhCsNiNKQSL3ZhCsNiNKYSOp7gyK0aVBmZ2iSpDPT4+TuOsDS4AzJ8/vzKmLEOV\nqqnKMatjU2mmjNOnT9O4ek2UdcdKfKtyy8qaUymyLL1XWWvquE6evDpd5IuwlGiAp6mq42o1Tdx3\ndmMKwWI3phAsdmMKwWI3phAsdmMKwWI3phAsdmMKoaM+++zZs6n3qcoeMy9ceZMqpVHBPF/lB6sy\n18pHVymPrP2v8rLZ+gGA+8GA9vjZ3FSK6uLFi2lcrW9g1wSbF6BbgI+OjtL4qVOnaFz5/Az2mrLj\n8p3dmEKw2I0pBIvdmEKw2I0pBIvdmEKw2I0pBIvdmELouM/OvNM6nrDyRRXKd2VeuPLJ6+5b+dHs\nvA0OVnbmAgBa2hsA5s6d2/K+VXzVqlV07MqVK2m8TilpdVxq/cCKFStoXOW7sxLfql10f39/ZYzp\nwHd2YwrBYjemECx2YwrBYjemECx2YwrBYjemECx2Ywqhoz57RNDcb9aKFuA1zFVOufKD+/r6aJz5\npsqTVfnoyi9etGgRjS9btqwyps6L8ovVsanzynLKlY++fPlyGldrBFjOuMrjV2sf1HErWK8AtWaE\nzY1da/LOHhHPR8TRiNg15blnIuJQROxofN2vtmOM6S7N/Bv/SwD3TvP8TzPzjsbXKzM7LWPMTCPF\nnpl/AsDX/hljep46H9A9ERE7G//mVxYDi4itETESESOqr5gxpn20KvafA9gM4A4AYwB+XPWLmbkt\nM4czc1gVlDTGtI+WxJ6ZRzJzIjOvAPgFgDtndlrGmJmmJbFHxNCUH78DYFfV7xpjegPps0fECwDu\nAbA8Ig4C+BGAeyLiDgAJYBTAY83ukNVQVz2xGcoXZTnAgPZ0mcevaqvXjau3P+zYVX1y5aOrtQ91\n1jeotQ0q51wdG4ura02dF+WFq7UV7NhUn4FWkWLPzIenefq5NszFGNNGvFzWmEKw2I0pBIvdmEKw\n2I0pBIvdmELoeIors4nmzOHTYXaGSllUcZVGysar9ryqrPDAwACNs1bVAJ+7soiUvaXKZKu2yuw1\nU9abarOt7C22PPvs2bN0rDrnddOaaSqqsJHZeWG2ne/sxhSCxW5MIVjsxhSCxW5MIVjsxhSCxW5M\nIVjsxhRCR332zKxV3peleh4+fJiOVeWaWTlmADh48GBljLXfBYDR0VEaV8et0kxXr15dGVOpuwpV\nMln59Oy8Ky9apc+qMmcsLVn57Co+Pj5O4+q8sWNn8wa4D888eN/ZjSkEi92YQrDYjSkEi92YQrDY\njSkEi92YQrDYjSmEnvLZVQte5iEqP/n48eM0rso1s9bGypNlHj1Qv/0v8/kHBwfpWJXHz1ouA/VK\nSau1D6rEtqoT8PHHH7cUayau1lao64mtT1BlrtlrYp/dGGOxG1MKFrsxhWCxG1MIFrsxhWCxG1MI\nFrsxhdDxuvHMd63j6S5dupSOXbduHY2r2u/M81W+qMq137NnD40fOnSIxjdu3FgZW79+PR2rzpuq\n7a7WCLC1E6r2uoqr1+zYsWOVMVVjQPnsdesjsBoGql4+u95q+ewRsS4i/hgReyJid0R8v/H8QES8\nGhF7G9/5VWOM6SrN/Bt/GcAPM/NmAH8L4PGIuBnAkwBey8zrAbzW+NkY06NIsWfmWGa+2Xh8DsA7\nANYAeADA9savbQfwYLsmaYypz1f6gC4iNgL4GoC/ABjMzLFG6DCAaRdhR8TWiBiJiBFVM8wY0z6a\nFntELATwOwA/yMwvZH7kZDe5aTvKZea2zBzOzGGVHGCMaR9NiT0i5mJS6L/OzN83nj4SEUON+BCA\no+2ZojFmJpDWW0zWvH0OwDuZ+ZMpoZcBPALg2cb3l9S2Zs+eTVMqlc3D0mNVqubQ0BCNq/K9zBZU\nqbn79++n8d27d9P4Rx99RONHj1b/nd27dy8dqywi1U66v7+fxlkqp7LWLl68SOPqbSGzPFVLZmVJ\n3nDDDTSurF523lnbZQUrUd2Mz/4NAN8D8HZE7Gg89xQmRf7biHgUwAEAD7U8Q2NM25Fiz8w/A6j6\nc/GtmZ2OMaZdeLmsMYVgsRtTCBa7MYVgsRtTCBa7MYXQ0RTXK1eu1EoVbbVVLaD94g0bNrQ8XpVr\nVh7/ypUraVz59B9++GFljKV5AjqVU6Udq3bSzGefmJigYy9fvkzjdVo+q9db+ehr166lcXVNsLmp\n88Li7PXynd2YQrDYjSkEi92YQrDYjSkEi92YQrDYjSkEi92YQuiozz4xMUHL/6rcauYvMg8e0H7x\n4sWLaZyVTFY++ebNm2l806ZNNM58dIDnuyuPXrWTVj69aqvMctKZBw/oGgVq7QRb36BafKtrUV0v\n6npk60LU+gF2LbKxvrMbUwgWuzGFYLEbUwgWuzGFYLEbUwgWuzGFYLEbUwgdz2dntcJZXXigXvtf\n5V2qmvXM01Vti5WfvGXLFho/f/48jY+NjVXGlEev2kmfOHGiVpy9Lmrtg6rHP2cOv3zZ9lk+OaBf\nU3WtnjlzhsZZy2Z1XpiHz2rO+85uTCFY7MYUgsVuTCFY7MYUgsVuTCFY7MYUgsVuTCE00599HYBf\nARgEkAC2ZebPIuIZAP8I4POE56cy8xW2rblz52LFihWVcdUTu45vqnxR5W0y/1LlXSvU3NXc2HlT\nHr6q1X/27Fkar9MjXa2NUHFVX53l0qua9AqVr662z3x2tW22ZqRuf/bLAH6YmW9GRD+ANyLi1Ubs\np5n5L01swxjTZZrpzz4GYKzx+FxEvANgTbsnZoyZWb7Se/aI2AjgawD+0njqiYjYGRHPR8S0/0tG\nxNaIGImIEVaSyhjTXpoWe0QsBPA7AD/IzLMAfg5gM4A7MHnn//F04zJzW2YOZ+awek9ujGkfTYk9\nIuZiUui/zszfA0BmHsnMicy8AuAXAO5s3zSNMXWRYo/Jj/eeA/BOZv5kyvNTS3d+B8CumZ+eMWam\naObT+G8A+B6AtyNiR+O5pwA8HBF3YNKOGwXwmNpQZlILTNlALOVRWWvz5s2jcWXjsJTHOmObGa/S\nb1mqpxrL7ClAW0jqcxjWuljZduo1VdcLs0vrWrEKZZ8xVOrukSNHKmNs3s18Gv9nANOZd9RTN8b0\nFl5BZ0whWOzGFILFbkwhWOzGFILFbkwhWOzGFEJHS0lfvHgRBw4cqIyrVM+FCxe2vG/lBys/mvmy\nFy5coGNZe95mUF44S7FVfjLzogFgfHycxpUPz8pg1/XRVYluNl6NVah1G+rY2DWhjptdy+z18J3d\nmEKw2I0pBIvdmEKw2I0pBIvdmEKw2I0pBIvdmEII5bPO6M4ijgGYarQvB3C8YxP4avTq3Hp1XoDn\n1iozObcNmTltvfaOiv1LO48Yyczhrk2A0Ktz69V5AZ5bq3Rqbv433phCsNiNKYRui31bl/fP6NW5\n9eq8AM+tVToyt66+ZzfGdI5u39mNMR3CYjemELoi9oi4NyL+NyLej4gnuzGHKiJiNCLejogdETHS\n5bk8HxFHI2LXlOcGIuLViNjb+N6VnloVc3smIg41zt2OiLi/S3NbFxF/jIg9EbE7Ir7feL6r547M\nqyPnrePv2SNiNoD3APw9gIMAXgfwcGbu6ehEKoiIUQDDmdn1BRgR8U0A5wH8KjNvbTz3zwBOZuaz\njT+USzPzn3pkbs8AON/tNt6NbkVDU9uMA3gQwD+gi+eOzOshdOC8dePOfieA9zNzf2Z+BuA3AB7o\nwjx6nsz8E4CTVz39AIDtjcfbMXmxdJyKufUEmTmWmW82Hp8D8Hmb8a6eOzKvjtANsa8B8NGUnw+i\nt/q9J4A/RMQbEbG125OZhsHMHGs8Pgygur9Sd5BtvDvJVW3Ge+bctdL+vC7+gO7L3J2ZXwdwH4DH\nG/+u9iQ5+R6sl7zTptp4d4pp2oz/lW6eu1bbn9elG2I/BGDdlJ/XNp7rCTLzUOP7UQAvovdaUR/5\nvINu4/vRLs/nr/RSG+/p2oyjB85dN9ufd0PsrwO4PiKui4hrAHwXwMtdmMeXiIi+xgcniIg+AN9G\n77WifhnAI43HjwB4qYtz+QK90sa7qs04unzuut7+PDM7/gXgfkx+Ir8PwNPdmEPFvDYB+J/G1+5u\nzw3AC5j8t+4SJj/beBTAMgCvAdgL4L8BDPTQ3P4dwNsAdmJSWENdmtvdmPwXfSeAHY2v+7t97si8\nOnLevFzWmELwB3TGFILFbkwhWOzGFILFbkwhWOzGFILFbkwhWOzGFML/AeR0OmrOzd6AAAAAAElF\nTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KzvyCyI02O-S",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import imageio\n",
        "def make_gif():\n",
        "    images = []\n",
        "    filenames = ['capture_of_evol_{}.png'.format(i) for i in range(0,100,4)]\n",
        "    for filename in filenames:\n",
        "        images.append(imageio.imread(filename))\n",
        "    imageio.mimsave('movie.gif', images)\n",
        "make_gif()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9YdaTpj3wax6",
        "colab_type": "text"
      },
      "source": [
        "# Conclusion.\n",
        "\n",
        "I've trained simple Neural Network based on Bayes approach, or simply called VAE. \n",
        "\n",
        "It shows good results for MNIST on simple CNN net.\n",
        "\n",
        "We realized the model to do many interesting things from creating similar objects to the original, to checking the similarity between two objects.\n",
        "\n",
        "In the future, I plan to supplement this work with some improvements, including the study of various divergences instead of KL-divergence. Also, it is interestings to see what happens if two vectors are connected not by a segment but by some curved line when we try to inspect similarity between two vectors.\n",
        "\n",
        "The training did not last long, but with the reparametrization trick it accelerated several times.\n",
        "\n",
        "\n",
        "\n"
      ]
    }
  ]
}