05_05_GoogLeNet_CIFAR10.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "05_05_GoogLeNet_CIFAR10.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "authorship_tag": "ABX9TyMeeKMVj4nv4Wwi7vEhu6pl",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/OUCTheoryGroup/colab_demo/blob/master/05_05_GoogLeNet_CIFAR10.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wvOCB0G6dKSZ",
        "colab_type": "text"
      },
      "source": [
        "## GoogLeNet 应用于 CIFAR10 分类\n",
        "\n",
        "2014年，GoogLeNet和VGG是当年ImageNet挑战赛的双雄，GoogLeNet获得了第一名、VGG获得了第二名，这两类模型结构的共同特点是层次更深了。GoogLeNet则做了更加大胆的网络结构尝试，虽然深度只有22层，但大小却比AlexNet和VGG小很多，GoogleNet参数为500万个，AlexNet参数个数是GoogleNet的12倍，VGGNet参数又是AlexNet的3倍，因此在内存或计算资源有限时，GoogleNet是比较好的选择；从模型结果来看，GoogLeNet的性能却更加优越。\n",
        "\n",
        "**小知识：**GoogLeNet是谷歌（Google）研究出来的深度网络结构，为什么不叫“GoogleNet”，而叫“GoogLeNet”，据说是为了向“LeNet”致敬，因此取名为“GoogLeNet”\n",
        "\n",
        "有没有一种方法既能保持网络结构的稀疏性，又能利用密集矩阵的高计算性能。大量的文献表明可以将稀疏矩阵聚类为较为密集的子矩阵来提高计算性能，就如人类的大脑是可以看做是神经元的重复堆积，因此，GoogLeNet团队提出了Inception网络结构，就是构造一种“基础神经元”结构，来搭建一个稀疏性、高计算性能的网络结构。\n",
        "\n",
        "**小知识：**Inception 的名字来自于电影《盗梦空间》，电影英文名就是Inception，影片有一句台词：“We need to go deeper.” 即在神经网络中，需要“go deeper”。\n",
        "\n",
        "Inception历经了V1、V2、V3、V4等多个版本的发展，不断趋于完善。今天我们编程，选择的是 InceptionV3，根据Google官方，可以得到 InceptionV3 结构如下：\n",
        "\n",
        "![InceptionV3](https://gaopursuit.oss-cn-beijing.aliyuncs.com/202003/20200229182425.jpg)\n",
        "\n",
        "下面，我们编码实现这个网络。\n",
        "\n",
        "## 1. 定义 dataloader \n",
        "\n",
        "这部分代码和之前 VGG 那个教程是完全一样的。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DHOzzKQRc6iL",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        },
        "outputId": "8f87fd4b-3103-47ce-efae-2c444139a113"
      },
      "source": [
        "import torch\n",
        "import torchvision\n",
        "import torchvision.transforms as transforms\n",
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n",
        "import torch.nn as nn\n",
        "import torch.nn.functional as F\n",
        "import torch.optim as optim\n",
        "\n",
        "# 使用GPU训练，可以在菜单 \"代码执行工具\" -> \"更改运行时类型\" 里进行设置\n",
        "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n",
        "\n",
        "transform_train = transforms.Compose([\n",
        "    transforms.RandomCrop(32, padding=4),\n",
        "    transforms.RandomHorizontalFlip(),\n",
        "    transforms.ToTensor(),\n",
        "    transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010))])\n",
        "\n",
        "transform_test = transforms.Compose([\n",
        "    transforms.ToTensor(),\n",
        "    transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010))])\n",
        "\n",
        "trainset = torchvision.datasets.CIFAR10(root='./data', train=True,  download=True, transform=transform_train)\n",
        "testset  = torchvision.datasets.CIFAR10(root='./data', train=False, download=True, transform=transform_test)\n",
        "\n",
        "trainloader = torch.utils.data.DataLoader(trainset, batch_size=128, shuffle=True, num_workers=2)\n",
        "testloader = torch.utils.data.DataLoader(testset, batch_size=128, shuffle=False, num_workers=2)"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Files already downloaded and verified\n",
            "Files already downloaded and verified\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JbGFLaGgfnIX",
        "colab_type": "text"
      },
      "source": [
        "## 2. 定义 InceptionV3 网络\n",
        "\n",
        "首先定义一个基本卷积类，这个在后面会大量使用。可以看到，这个类把 卷积、BatchNormalization、ReLU 激活，打包在了一起，非常容易理解。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EwJjHSUYf4v8",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class BasicConv2d(nn.Module):\n",
        "\n",
        "    def __init__(self, in_channels, out_channels, **kwargs):\n",
        "        super(BasicConv2d, self).__init__()\n",
        "        self.conv = nn.Conv2d(in_channels, out_channels, bias=False, **kwargs)\n",
        "        self.bn = nn.BatchNorm2d(out_channels, eps=0.001)\n",
        "\n",
        "    def forward(self, x):\n",
        "        x = self.conv(x)\n",
        "        x = self.bn(x)\n",
        "        return F.relu(x, inplace=True)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "B8zQu-fy29FR",
        "colab_type": "text"
      },
      "source": [
        "### 2.1 Inception A 模块\n",
        "\n",
        "从 InceptionV3 总体架构图中可以看出，第一个部分为 Inception A 模块，该模块的示意如下所示：\n",
        "\n",
        "![Inception A 模块示意图](https://gaopursuit.oss-cn-beijing.aliyuncs.com/202003/20200229204301.jpg)\n",
        "\n",
        "该结构将CNN中常用的卷积（1x1，3x3，5x5）堆叠在一起（卷积后的尺寸相同，将通道相加），一方面增加了网络的宽度，另一方面也增加了网络对尺度的适应性。\n",
        "\n",
        "**请大家思考一个问题：1x1的卷积核有什么用呢？**\n",
        "\n",
        "1x1卷积的主要目的是为了减少维度，具体大家可以参考知乎一个文章《[1X1卷积核的作用](https://zhuanlan.zhihu.com/p/37910136)》\n",
        "\n",
        "该模块的代码如下："
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pPqJN8ly5SVI",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class InceptionA(nn.Module):\n",
        "\n",
        "    def __init__(self, in_channels, pool_features):\n",
        "        super(InceptionA, self).__init__()\n",
        "        self.branch1x1 = BasicConv2d(in_channels, 64, kernel_size=1)\n",
        "\n",
        "        self.branch5x5_1 = BasicConv2d(in_channels, 48, kernel_size=1)\n",
        "        self.branch5x5_2 = BasicConv2d(48, 64, kernel_size=5, padding=2)\n",
        "\n",
        "        self.branch3x3dbl_1 = BasicConv2d(in_channels, 64, kernel_size=1)\n",
        "        self.branch3x3dbl_2 = BasicConv2d(64, 96, kernel_size=3, padding=1)\n",
        "        self.branch3x3dbl_3 = BasicConv2d(96, 96, kernel_size=3, padding=1)\n",
        "\n",
        "        self.branch_pool = BasicConv2d(in_channels, pool_features, kernel_size=1)\n",
        "\n",
        "    def forward(self, x):\n",
        "        branch1x1 = self.branch1x1(x)\n",
        "\n",
        "        branch5x5 = self.branch5x5_1(x)\n",
        "        branch5x5 = self.branch5x5_2(branch5x5)\n",
        "\n",
        "        branch3x3dbl = self.branch3x3dbl_1(x)\n",
        "        branch3x3dbl = self.branch3x3dbl_2(branch3x3dbl)\n",
        "        branch3x3dbl = self.branch3x3dbl_3(branch3x3dbl)\n",
        "\n",
        "        branch_pool = F.avg_pool2d(x, kernel_size=3, stride=1, padding=1)\n",
        "        branch_pool = self.branch_pool(branch_pool)\n",
        "\n",
        "        outputs = [branch1x1, branch5x5, branch3x3dbl, branch_pool]\n",
        "        return torch.cat(outputs, 1)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-S0xKMhZ_jKY",
        "colab_type": "text"
      },
      "source": [
        "下面写一段非常非常简的代码来分析一下 Inception A 模块："
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Xpb0I8d__hA6",
        "colab_type": "code",
        "outputId": "7f74611a-d87d-4188-9b14-78089696b5b3",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "# 初始化\n",
        "net = InceptionA(3, 32)\n",
        "# 随机生成一个feature map 大小为  3*32*32 （feature map 都是四维的，第一个维度是样本数量，人们在论文介绍中往往忽略这个样本数量）\n",
        "x = torch.randn(1, 3, 32, 32)\n",
        "y = net(x)\n",
        "print('Feature map size after Inception A: ', y.size())"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Feature map size after Inception A:  torch.Size([1, 256, 32, 32])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0on-wKVnARxA",
        "colab_type": "text"
      },
      "source": [
        "可以看出，现在的 feature map 尺寸是： 256 x 32 x 32 。\n",
        "\n",
        "我们分析下这个256是怎么来的，代码中可以看出来，四个分支得到 feature map 数量分别是：\n",
        " - branch1x1 ==> 64\n",
        " - branch5x5 ==> 64\n",
        " - branch3x3dbl ==> 96\n",
        " - branch_pool ==> pool_features (这个是输入参数)\n",
        " \n",
        " 得到的特征图数量一共是：\n",
        "\n",
        "64 + 64 + 96 + pool_features = 224 + pool_features \n",
        "\n",
        "可以看出， pool_featurs 主要起到一个连接前后两层的目的。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oTkC9Spm7x1Q",
        "colab_type": "text"
      },
      "source": [
        "### 2.2 Inception B 模块\n",
        "\n",
        "从 InceptionV3 总体架构图中可以看出，第二个部分为 Inception B 模块，该模块的示意如下所示：\n",
        "\n",
        "![InceptionB](http://img.huaiwen.me/20190119231001.jpg)\n",
        "\n",
        "该模块的代码如下："
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hBxxeGxPCKn7",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class InceptionB(nn.Module):\n",
        "\n",
        "    def __init__(self, in_channels):\n",
        "        super(InceptionB, self).__init__()\n",
        "        self.branch3x3 = BasicConv2d(in_channels, 384, kernel_size=3, stride=2)\n",
        "\n",
        "        self.branch3x3dbl_1 = BasicConv2d(in_channels, 64, kernel_size=1)\n",
        "        self.branch3x3dbl_2 = BasicConv2d(64, 96, kernel_size=3, padding=1)\n",
        "        self.branch3x3dbl_3 = BasicConv2d(96, 96, kernel_size=3, stride=2)\n",
        "\n",
        "    def forward(self, x):\n",
        "        branch3x3 = self.branch3x3(x)\n",
        "\n",
        "        branch3x3dbl = self.branch3x3dbl_1(x)\n",
        "        branch3x3dbl = self.branch3x3dbl_2(branch3x3dbl)\n",
        "        branch3x3dbl = self.branch3x3dbl_3(branch3x3dbl)\n",
        "\n",
        "        branch_pool = F.max_pool2d(x, kernel_size=3, stride=2)\n",
        "\n",
        "        outputs = [branch3x3, branch3x3dbl, branch_pool]\n",
        "        return torch.cat(outputs, 1)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lMu7cDDVCRjr",
        "colab_type": "text"
      },
      "source": [
        "和上面一样，下面继续写一段非常非常简的代码来分析一下 Inception B 模块："
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wE7vIP0uCTot",
        "colab_type": "code",
        "outputId": "368acb47-9b30-438e-8bdd-3c3683ad18ee",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "net = InceptionB(128)\n",
        "x = torch.randn(1, 128, 32, 32)\n",
        "y = net(x)\n",
        "print('Feature map size after Inception B: ', y.size())"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Feature map size after Inception B:  torch.Size([1, 608, 15, 15])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CqeRod3JDzik",
        "colab_type": "text"
      },
      "source": [
        "需要注意的一点是，Inception B 模块需要一个输入参数是 in_channels，也就是说，这个参数必须和输入数据的通道数一致，不然运行的过程中就会报错。\n",
        "\n",
        "从总体架构图中看，Inception B 这个模块使用频率并不高，只出现过一次。它的作用是什么呢？\n",
        "\n",
        "它的作用是让图像的尺寸变小。从上面的演示可以看出，feature map 尺寸由 32 变成了 15，相当于池化加了一个Inception A 模块。但这样计算量会比较大，为了同时保持特征表示且降低计算量，谷歌就设计了 Inception B 模块，使用两个并行化的模块来降低计算量（卷积、池化并行执行，再进行合并）\n",
        "\n",
        "通过代码可以看出，三个分支得到的 feature map 数量依次是：\n",
        "- branch3x3 ===> 384\n",
        "- branch3x3dbl ==> 96\n",
        "- branch_pool ==> in_channels\n",
        "\n",
        "所以，输出的 feature map 数量是 384 + 96 + in_channels = 480 + in_channels，尺寸为原来的 1/2。\n",
        "\n",
        "### 2.3 Inception C 模块\n",
        "\n",
        "从 InceptionV3 总体架构图中可以看出，第三个部分为 Inception C 模块，该模块的示意如下所示：\n",
        "\n",
        "![InceptionC](http://img.huaiwen.me/20190119231031.jpg)\n",
        "\n",
        "任意nxn的卷积都可以通过1xn卷积后接nx1卷积来替代（可以降低计算量），具体原因可以百度一下。GoogLeNet团队发现在网络的前期使用这种分解效果并不好，在中度大小的特征图（feature map）上使用效果才会更好（特征图大小建议在12到20之间）。\n",
        "\n",
        "从图中可以看出，里面有大量使用 7x1 和 1x7 来取代 7x7 的卷积核。\n",
        "\n",
        "该模块的代码如下："
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wBX0uTNRJiMn",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class InceptionC(nn.Module):\n",
        "\n",
        "    def __init__(self, in_channels, channels_7x7):\n",
        "        super(InceptionC, self).__init__()\n",
        "        self.branch1x1 = BasicConv2d(in_channels, 192, kernel_size=1)\n",
        "\n",
        "        c7 = channels_7x7\n",
        "        self.branch7x7_1 = BasicConv2d(in_channels, c7, kernel_size=1)\n",
        "        self.branch7x7_2 = BasicConv2d(c7, c7, kernel_size=(1, 7), padding=(0, 3))\n",
        "        self.branch7x7_3 = BasicConv2d(c7, 192, kernel_size=(7, 1), padding=(3, 0))\n",
        "\n",
        "        self.branch7x7dbl_1 = BasicConv2d(in_channels, c7, kernel_size=1)\n",
        "        self.branch7x7dbl_2 = BasicConv2d(c7, c7, kernel_size=(7, 1), padding=(3, 0))\n",
        "        self.branch7x7dbl_3 = BasicConv2d(c7, c7, kernel_size=(1, 7), padding=(0, 3))\n",
        "        self.branch7x7dbl_4 = BasicConv2d(c7, c7, kernel_size=(7, 1), padding=(3, 0))\n",
        "        self.branch7x7dbl_5 = BasicConv2d(c7, 192, kernel_size=(1, 7), padding=(0, 3))\n",
        "\n",
        "        self.branch_pool = BasicConv2d(in_channels, 192, kernel_size=1)\n",
        "\n",
        "    def forward(self, x):\n",
        "        branch1x1 = self.branch1x1(x)\n",
        "\n",
        "        branch7x7 = self.branch7x7_1(x)\n",
        "        branch7x7 = self.branch7x7_2(branch7x7)\n",
        "        branch7x7 = self.branch7x7_3(branch7x7)\n",
        "\n",
        "        branch7x7dbl = self.branch7x7dbl_1(x)\n",
        "        branch7x7dbl = self.branch7x7dbl_2(branch7x7dbl)\n",
        "        branch7x7dbl = self.branch7x7dbl_3(branch7x7dbl)\n",
        "        branch7x7dbl = self.branch7x7dbl_4(branch7x7dbl)\n",
        "        branch7x7dbl = self.branch7x7dbl_5(branch7x7dbl)\n",
        "\n",
        "        branch_pool = F.avg_pool2d(x, kernel_size=3, stride=1, padding=1)\n",
        "        branch_pool = self.branch_pool(branch_pool)\n",
        "\n",
        "        outputs = [branch1x1, branch7x7, branch7x7dbl, branch_pool]\n",
        "        return torch.cat(outputs, 1)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bO1QOIbpJmW-",
        "colab_type": "text"
      },
      "source": [
        "和上面一样，下面继续写一段非常非常简的代码来分析一下 Inception C 模块："
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QRV4etZeJpcw",
        "colab_type": "code",
        "outputId": "09bbacf6-52b4-4952-eb98-e19b47e55c03",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "net = InceptionC(128, channels_7x7=128)\n",
        "x = torch.randn(1, 128, 32, 32)\n",
        "y = net(x)\n",
        "print('Feature map size after Inception C: ', y.size())"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Feature map size after Inception C:  torch.Size([1, 768, 32, 32])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VPeedc1JKYj0",
        "colab_type": "text"
      },
      "source": [
        "可以看到，图像的大小不会发生变化，四个分支得到的 feature map 数量都是192，因此，经过这个模块，feature map 的数量永远是 768。\n",
        "\n",
        "### 2.4 Inception D 模块\n",
        "\n",
        "这个模块的示意如下所示：\n",
        "\n",
        "![InceptionD](http://img.huaiwen.me/20190119231104.jpg)\n",
        "\n",
        "下面是具体代码：\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WC2TK5XLLy--",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class InceptionD(nn.Module):\n",
        "\n",
        "    def __init__(self, in_channels):\n",
        "        super(InceptionD, self).__init__()\n",
        "        self.branch3x3_1 = BasicConv2d(in_channels, 192, kernel_size=1)\n",
        "        self.branch3x3_2 = BasicConv2d(192, 320, kernel_size=3, stride=2)\n",
        "\n",
        "        self.branch7x7x3_1 = BasicConv2d(in_channels, 192, kernel_size=1)\n",
        "        self.branch7x7x3_2 = BasicConv2d(192, 192, kernel_size=(1, 7), padding=(0, 3))\n",
        "        self.branch7x7x3_3 = BasicConv2d(192, 192, kernel_size=(7, 1), padding=(3, 0))\n",
        "        self.branch7x7x3_4 = BasicConv2d(192, 192, kernel_size=3, stride=2)\n",
        "\n",
        "    def forward(self, x):\n",
        "        branch3x3 = self.branch3x3_1(x)\n",
        "        branch3x3 = self.branch3x3_2(branch3x3)\n",
        "\n",
        "        branch7x7x3 = self.branch7x7x3_1(x)\n",
        "        branch7x7x3 = self.branch7x7x3_2(branch7x7x3)\n",
        "        branch7x7x3 = self.branch7x7x3_3(branch7x7x3)\n",
        "        branch7x7x3 = self.branch7x7x3_4(branch7x7x3)\n",
        "\n",
        "        branch_pool = F.max_pool2d(x, kernel_size=3, stride=2)\n",
        "        outputs = [branch3x3, branch7x7x3, branch_pool]\n",
        "        return torch.cat(outputs, 1)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NeqAWsRCPI7e",
        "colab_type": "text"
      },
      "source": [
        "我们继续用一段代码来分析这个模块："
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ePMM_fOrPMOu",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "b3c158cc-4e4d-42ae-94db-4574de40b1d1"
      },
      "source": [
        "net = InceptionD(128)\n",
        "x = torch.randn(1, 128, 32, 32)\n",
        "y = net(x)\n",
        "print('Feature map size after Inception D: ', y.size())"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Feature map size after Inception D:  torch.Size([1, 640, 15, 15])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "B-La4ojLMAFK",
        "colab_type": "text"
      },
      "source": [
        "通过代码可以看出，三个分支得到的 feature map 数量依次是：\n",
        "- branch3x3 ===> 320\n",
        "- branch7x7x3 ==> 192\n",
        "- branch_pool ==> in_channels\n",
        "\n",
        "因此，总特征图的数量为：512+in_channels，尺寸为原来的一半\n",
        "\n",
        "### 2.5 Inception E 模块\n",
        "\n",
        "这个模块的示意如下所示：\n",
        "\n",
        "![InceptionE](http://img.huaiwen.me/20190119231132.jpg)\n",
        "\n",
        "下面是具体代码：\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZXOgS2bMMNdG",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class InceptionE(nn.Module):\n",
        "\n",
        "    def __init__(self, in_channels):\n",
        "        super(InceptionE, self).__init__()\n",
        "        self.branch1x1 = BasicConv2d(in_channels, 320, kernel_size=1)\n",
        "\n",
        "        self.branch3x3_1 = BasicConv2d(in_channels, 384, kernel_size=1)\n",
        "        self.branch3x3_2a = BasicConv2d(384, 384, kernel_size=(1, 3), padding=(0, 1))\n",
        "        self.branch3x3_2b = BasicConv2d(384, 384, kernel_size=(3, 1), padding=(1, 0))\n",
        "\n",
        "        self.branch3x3dbl_1 = BasicConv2d(in_channels, 448, kernel_size=1)\n",
        "        self.branch3x3dbl_2 = BasicConv2d(448, 384, kernel_size=3, padding=1)\n",
        "        self.branch3x3dbl_3a = BasicConv2d(384, 384, kernel_size=(1, 3), padding=(0, 1))\n",
        "        self.branch3x3dbl_3b = BasicConv2d(384, 384, kernel_size=(3, 1), padding=(1, 0))\n",
        "\n",
        "        self.branch_pool = BasicConv2d(in_channels, 192, kernel_size=1)\n",
        "\n",
        "    def forward(self, x):\n",
        "        branch1x1 = self.branch1x1(x)\n",
        "\n",
        "        branch3x3 = self.branch3x3_1(x)\n",
        "        branch3x3 = [\n",
        "            self.branch3x3_2a(branch3x3),\n",
        "            self.branch3x3_2b(branch3x3),\n",
        "        ]\n",
        "        branch3x3 = torch.cat(branch3x3, 1)\n",
        "\n",
        "        branch3x3dbl = self.branch3x3dbl_1(x)\n",
        "        branch3x3dbl = self.branch3x3dbl_2(branch3x3dbl)\n",
        "        branch3x3dbl = [\n",
        "            self.branch3x3dbl_3a(branch3x3dbl),\n",
        "            self.branch3x3dbl_3b(branch3x3dbl),\n",
        "        ]\n",
        "        branch3x3dbl = torch.cat(branch3x3dbl, 1)\n",
        "\n",
        "        branch_pool = F.avg_pool2d(x, kernel_size=3, stride=1, padding=1)\n",
        "        branch_pool = self.branch_pool(branch_pool)\n",
        "\n",
        "        outputs = [branch1x1, branch3x3, branch3x3dbl, branch_pool]\n",
        "        return torch.cat(outputs, 1)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_037j1IkQFVx",
        "colab_type": "text"
      },
      "source": [
        "继续用代码分析模块的输出："
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ltK6Zq70QJr9",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "a3756243-00c6-4aad-a71d-bb8893276095"
      },
      "source": [
        "net = InceptionE(128)\n",
        "x = torch.randn(1, 128, 32, 32)\n",
        "y = net(x)\n",
        "print('Feature map size after Inception E: ', y.size())"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Feature map size after Inception E:  torch.Size([1, 2048, 32, 32])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wD8EyZlgMYz_",
        "colab_type": "text"
      },
      "source": [
        "通过代码可以看出，四个分支得到的 feature map 数量依次是：\n",
        "- branch1x1 ===> 320\n",
        "- branch3x3 ==> 768\n",
        "- branch3x3dbl ==> 768\n",
        "- branch_pool ==> 192\n",
        "\n",
        "因此，总的 feature map 数量为 2048，图像尺寸不变\n",
        "\n",
        "本来还有一个 Inception Auxiliary Classifiers，这里不过多介绍了。\n",
        "\n",
        "## 3. 整体架构\n",
        "\n",
        "网络的整体架构如图：\n",
        "\n",
        "![InceptionV3](https://gaopursuit.oss-cn-beijing.aliyuncs.com/202003/20200229182425.jpg)\n",
        "\n",
        "原版的网络实是在太大了，难以放到GPU上，受 colab 的 GPU资源限制，这里我对网重复的模块进行了简化 ~~~ 代码如下："
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "g_jBNnlNMyFT",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class Inception3(nn.Module):\n",
        "\n",
        "    def __init__(self, num_classes=1000):\n",
        "        super(Inception3, self).__init__()\n",
        "        self.Conv2d_1a_3x3 = BasicConv2d(3, 32, kernel_size=3, stride=2)\n",
        "        self.Conv2d_2a_3x3 = BasicConv2d(32, 32, kernel_size=3)\n",
        "        self.Mixed_A = InceptionA(32, pool_features=32)\n",
        "        self.Mixed_B = InceptionB(256)\n",
        "        self.Mixed_C = InceptionC(736, channels_7x7=128)\n",
        "        self.Mixed_D = InceptionD(768)\n",
        "        self.Mixed_E = InceptionE(1280)\n",
        "        self.fc = nn.Linear(2048, num_classes)\n",
        "\n",
        "        for m in self.modules():\n",
        "            if isinstance(m, nn.Conv2d) or isinstance(m, nn.Linear):\n",
        "                import scipy.stats as stats\n",
        "                stddev = m.stddev if hasattr(m, 'stddev') else 0.1\n",
        "                X = stats.truncnorm(-2, 2, scale=stddev)\n",
        "                values = torch.Tensor(X.rvs(m.weight.numel()))\n",
        "                values = values.view(m.weight.size())\n",
        "                m.weight.data.copy_(values)\n",
        "            elif isinstance(m, nn.BatchNorm2d):\n",
        "                nn.init.constant_(m.weight, 1)\n",
        "                nn.init.constant_(m.bias, 0)\n",
        "\n",
        "    def forward(self, x):\n",
        "        x = self.Conv2d_1a_3x3(x)\n",
        "        x = self.Conv2d_2a_3x3(x)\n",
        "        x = self.Mixed_A(x)\n",
        "        x = self.Mixed_B(x)    \n",
        "        x = self.Mixed_C(x)  \n",
        "        x = self.Mixed_D(x)   \n",
        "        x = self.Mixed_E(x)\n",
        "        # 2 x 2 x 2048\n",
        "        x = F.avg_pool2d(x, kernel_size=2)\n",
        "        # 1 x 1 x 2048\n",
        "        x = F.dropout(x, training=self.training)\n",
        "        # 1 x 1 x 2048\n",
        "        x = x.view(x.size(0), -1)\n",
        "        # 2048\n",
        "        x = self.fc(x)\n",
        "        return x"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pD1TU0CwNGV7",
        "colab_type": "text"
      },
      "source": [
        "网络初始化，具体代码如下："
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zxJQ1ToWNUM9",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# 网络放到GPU上\n",
        "net = Inception3( num_classes=10 ).to(device)\n",
        "criterion = nn.CrossEntropyLoss()\n",
        "optimizer = optim.Adam(net.parameters(), lr=0.001)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "96ZITYADYeGz",
        "colab_type": "text"
      },
      "source": [
        "## 4. 网络训练\n",
        "\n",
        "训练的代码和之前是完全一样的："
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "937rB63zYj25",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 714
        },
        "outputId": "a0c83dd2-024d-4537-c4a8-df45e6b30dcb"
      },
      "source": [
        "for epoch in range(10):  # 重复多轮训练\n",
        "    for i, (inputs, labels) in enumerate(trainloader):\n",
        "        inputs = inputs.to(device)\n",
        "        labels = labels.to(device)\n",
        "        # 优化器梯度归零\n",
        "        optimizer.zero_grad()\n",
        "        # 正向传播 +　反向传播 + 优化 \n",
        "        outputs = net(inputs)\n",
        "        loss = criterion(outputs, labels)\n",
        "        loss.backward()\n",
        "        optimizer.step()\n",
        "        # 输出统计信息\n",
        "        if i % 100 == 0:   \n",
        "            print('Epoch: %d Minibatch: %5d loss: %.3f' %(epoch + 1, i + 1, loss.item()))\n",
        "\n",
        "print('Finished Training')"
      ],
      "execution_count": 38,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch: 1 Minibatch:     1 loss: 0.361\n",
            "Epoch: 1 Minibatch:   101 loss: 0.366\n",
            "Epoch: 1 Minibatch:   201 loss: 0.375\n",
            "Epoch: 1 Minibatch:   301 loss: 0.435\n",
            "Epoch: 2 Minibatch:     1 loss: 0.481\n",
            "Epoch: 2 Minibatch:   101 loss: 0.380\n",
            "Epoch: 2 Minibatch:   201 loss: 0.443\n",
            "Epoch: 2 Minibatch:   301 loss: 0.485\n",
            "Epoch: 3 Minibatch:     1 loss: 0.364\n",
            "Epoch: 3 Minibatch:   101 loss: 0.430\n",
            "Epoch: 3 Minibatch:   201 loss: 0.426\n",
            "Epoch: 3 Minibatch:   301 loss: 0.314\n",
            "Epoch: 4 Minibatch:     1 loss: 0.279\n",
            "Epoch: 4 Minibatch:   101 loss: 0.248\n",
            "Epoch: 4 Minibatch:   201 loss: 0.279\n",
            "Epoch: 4 Minibatch:   301 loss: 0.392\n",
            "Epoch: 5 Minibatch:     1 loss: 0.423\n",
            "Epoch: 5 Minibatch:   101 loss: 0.334\n",
            "Epoch: 5 Minibatch:   201 loss: 0.437\n",
            "Epoch: 5 Minibatch:   301 loss: 0.462\n",
            "Epoch: 6 Minibatch:     1 loss: 0.308\n",
            "Epoch: 6 Minibatch:   101 loss: 0.347\n",
            "Epoch: 6 Minibatch:   201 loss: 0.368\n",
            "Epoch: 6 Minibatch:   301 loss: 0.299\n",
            "Epoch: 7 Minibatch:     1 loss: 0.291\n",
            "Epoch: 7 Minibatch:   101 loss: 0.319\n",
            "Epoch: 7 Minibatch:   201 loss: 0.367\n",
            "Epoch: 7 Minibatch:   301 loss: 0.370\n",
            "Epoch: 8 Minibatch:     1 loss: 0.364\n",
            "Epoch: 8 Minibatch:   101 loss: 0.275\n",
            "Epoch: 8 Minibatch:   201 loss: 0.385\n",
            "Epoch: 8 Minibatch:   301 loss: 0.265\n",
            "Epoch: 9 Minibatch:     1 loss: 0.264\n",
            "Epoch: 9 Minibatch:   101 loss: 0.166\n",
            "Epoch: 9 Minibatch:   201 loss: 0.282\n",
            "Epoch: 9 Minibatch:   301 loss: 0.313\n",
            "Epoch: 10 Minibatch:     1 loss: 0.165\n",
            "Epoch: 10 Minibatch:   101 loss: 0.256\n",
            "Epoch: 10 Minibatch:   201 loss: 0.244\n",
            "Epoch: 10 Minibatch:   301 loss: 0.260\n",
            "Finished Training\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-hPc_kfJbVl6",
        "colab_type": "text"
      },
      "source": [
        "## 5. 测试验证准确率\n",
        "\n",
        "测试的代码和之前是完全一样的："
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ENFBUtd3bclT",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "2fc93027-53b0-49b9-84d9-0d804325b9d4"
      },
      "source": [
        "correct = 0\n",
        "total = 0\n",
        "\n",
        "for data in testloader:\n",
        "    images, labels = data\n",
        "    images, labels = images.to(device), labels.to(device)\n",
        "    outputs = net(images)\n",
        "    _, predicted = torch.max(outputs.data, 1)\n",
        "    total += labels.size(0)\n",
        "    correct += (predicted == labels).sum().item()\n",
        "\n",
        "print('Accuracy of the network on the 10000 test images: %.2f %%' % (\n",
        "    100 * correct / total))"
      ],
      "execution_count": 39,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Accuracy of the network on the 10000 test images: 85.40 %\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qY-pvHm4bi1_",
        "colab_type": "text"
      },
      "source": [
        "准确率又提高了一些，目前是因为 EPOCH 跑的较少，进一步多跑到150-200轮，准确率可以非常高。\n",
        "\n",
        "大家可能好奇，Inception明明有更好的版本，这里为什么以 V3 为示例呢？\n",
        "\n",
        "因为 V4 的主要改进是采用了 ResNet 的结构 ~~~ \n",
        "\n",
        "相信你也猜到了，下一节课的内容就是围绕 ResNet 展开。"
      ]
    }
  ]
}