Skip to content

Commit

Permalink
Refine quick start tutorial of pruning. (PaddlePaddle#49)
Browse files Browse the repository at this point in the history
  • Loading branch information
@wanghaoshuang
wanghaoshuang authored Feb 3, 2020
1 parent 648978c commit 1cb8d1b
Show file tree
Hide file tree
Showing 12 changed files with 504 additions and 56 deletions.
6 changes: 5 additions & 1 deletion demo/models/__init__.py
View file
Original file line number Diff line number Diff line change
@@ -1,9 +1,13 @@
from __future__ import absolute_import
from .mobilenet import MobileNet
from .resnet import ResNet34, ResNet50
from .resnet_vd import ResNet50_vd
from .mobilenet_v2 import MobileNetV2
from .pvanet import PVANet

__all__ = [
"model_list", "MobileNet", "ResNet34", "ResNet50", "MobileNetV2", "PVANet",
"ResNet50_vd"
]
model_list = [
'MobileNet', 'ResNet34', 'ResNet50', 'MobileNetV2', 'PVANet', 'ResNet50_vd'
]
0 demo/prune/REAME.md → demo/prune/README.md
View file
File renamed without changes.
352 changes: 352 additions & 0 deletions demo/prune/image_classification_pruning_quick_start.ipynb
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,352 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 图像分类模型通道剪裁-快速开始\n",
"\n",
"该教程以图像分类模型MobileNetV1为例，说明如何快速使用[PaddleSlim的卷积通道剪裁接口]()。\n",
"该示例包含以下步骤：\n",
"\n",
"1. 导入依赖\n",
"2. 构建模型\n",
"3. 剪裁\n",
"4. 训练剪裁后的模型\n",
"\n",
"以下章节依次次介绍每个步骤的内容。\n",
"\n",
"## 1. 导入依赖\n",
"\n",
"PaddleSlim依赖Paddle1.7版本，请确认已正确安装Paddle，然后按以下方式导入Paddle和PaddleSlim:"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {},
"outputs": [],
"source": [
"import paddle\n",
"import paddle.fluid as fluid\n",
"import paddleslim as slim"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 2. 构建网络\n",
"\n",
"该章节构造一个用于对MNIST数据进行分类的分类模型，选用`MobileNetV1`，并将输入大小设置为`[1, 28, 28]`，输出类别数为10。\n",
"为了方便展示示例，我们在`paddleslim.models`下预定义了用于构建分类模型的方法，执行以下代码构建分类模型："
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {},
"outputs": [],
"source": [
"exe, train_program, val_program, inputs, outputs = slim.models.image_classification(\"MobileNet\", [1, 28, 28], 10, use_gpu=False)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
">注意：paddleslim.models下的API并非PaddleSlim常规API，是为了简化示例而封装预定义的一系列方法，比如：模型结构的定义、Program的构建等。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 3. 剪裁卷积层通道\n",
"\n",
"### 3.1 计算剪裁之前的FLOPs"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"FLOPs: 10907072.0\n"
]
}
],
"source": [
"FLOPs = slim.analysis.flops(train_program)\n",
"print(\"FLOPs: {}\".format(FLOPs))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 3.2 剪裁\n",
"\n",
"我们这里对参数名为`conv2_1_sep_weights`和`conv2_2_sep_weights`的卷积层进行剪裁，分别剪掉20%和30%的通道数。\n",
"代码如下所示："
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {},
"outputs": [],
"source": [
"pruner = slim.prune.Pruner()\n",
"pruned_program, _, _ = pruner.prune(\n",
" train_program,\n",
" fluid.global_scope(),\n",
" params=[\"conv2_1_sep_weights\", \"conv2_2_sep_weights\"],\n",
" ratios=[0.33] * 2,\n",
" place=fluid.CPUPlace())"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"以上操作会修改`train_program`中对应卷积层参数的定义，同时对`fluid.global_scope()`中存储的参数数组进行裁剪。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 3.3 计算剪裁之后的FLOPs"
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"FLOPs: 10907072.0\n"
]
}
],
"source": [
"FLOPs = paddleslim.analysis.flops(train_program)\n",
"print(\"FLOPs: {}\".format(FLOPs))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 4. 训练剪裁后的模型\n",
"\n",
"### 4.1 定义输入数据\n",
"\n",
"为了快速执行该示例，我们选取简单的MNIST数据，Paddle框架的`paddle.dataset.mnist`包定义了MNIST数据的下载和读取。\n",
"代码如下："
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {},
"outputs": [],
"source": [
"import paddle.dataset.mnist as reader\n",
"train_reader = paddle.batch(\n",
" reader.train(), batch_size=128, drop_last=True)\n",
"train_feeder = fluid.DataFeeder(inputs, fluid.CPUPlace())"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 4.2 执行训练\n",
"以下代码执行了一个`epoch`的训练："
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[0.1484375] [0.4921875] [2.6727316]\n",
"[0.125] [0.546875] [2.6547904]\n",
"[0.125] [0.5546875] [2.795205]\n",
"[0.1171875] [0.578125] [2.8561475]\n",
"[0.1875] [0.59375] [2.470603]\n",
"[0.1796875] [0.578125] [2.8031898]\n",
"[0.1484375] [0.6015625] [2.7530417]\n",
"[0.1953125] [0.640625] [2.711596]\n",
"[0.125] [0.59375] [2.8637898]\n",
"[0.1796875] [0.53125] [2.9473038]\n",
"[0.25] [0.671875] [2.3943179]\n",
"[0.25] [0.6953125] [2.632146]\n",
"[0.2578125] [0.7265625] [2.723265]\n",
"[0.359375] [0.765625] [2.4263484]\n",
"[0.3828125] [0.8203125] [2.226284]\n",
"[0.421875] [0.8203125] [1.8042578]\n",
"[0.4765625] [0.890625] [1.6841211]\n",
"[0.53125] [0.8671875] [2.1971617]\n",
"[0.5546875] [0.8984375] [1.5361531]\n",
"[0.53125] [0.890625] [1.7211896]\n",
"[0.5078125] [0.8984375] [1.6586945]\n",
"[0.53125] [0.9140625] [1.8980236]\n",
"[0.546875] [0.9453125] [1.5279069]\n",
"[0.5234375] [0.8828125] [1.7356458]\n",
"[0.6015625] [0.9765625] [1.0375824]\n",
"[0.5546875] [0.921875] [1.639497]\n",
"[0.6015625] [0.9375] [1.5469061]\n",
"[0.578125] [0.96875] [1.3573356]\n",
"[0.65625] [0.9453125] [1.3787829]\n",
"[0.640625] [0.9765625] [0.9946856]\n",
"[0.65625] [0.96875] [1.1651027]\n",
"[0.625] [0.984375] [1.0487883]\n",
"[0.7265625] [0.9609375] [1.2526855]\n",
"[0.7265625] [0.9765625] [1.2954011]\n",
"[0.65625] [0.96875] [1.1181556]\n",
"[0.71875] [0.9765625] [0.97891223]\n",
"[0.640625] [0.9609375] [1.2135172]\n",
"[0.7265625] [0.9921875] [0.8950747]\n",
"[0.7578125] [0.96875] [1.0864108]\n",
"[0.734375] [0.9921875] [0.8392239]\n",
"[0.796875] [0.9609375] [0.7012155]\n",
"[0.7734375] [0.9765625] [0.7409136]\n",
"[0.8046875] [0.984375] [0.6108341]\n",
"[0.796875] [0.9765625] [0.63867176]\n",
"[0.7734375] [0.984375] [0.64099216]\n",
"[0.7578125] [0.9453125] [0.83827704]\n",
"[0.8046875] [0.9921875] [0.5311729]\n",
"[0.8984375] [0.9921875] [0.36445504]\n",
"[0.859375] [0.9921875] [0.40577835]\n",
"[0.8125] [0.9765625] [0.64629185]\n",
"[0.84375] [1.] [0.38400555]\n",
"[0.890625] [0.9765625] [0.45866236]\n",
"[0.8828125] [0.9921875] [0.3711415]\n",
"[0.7578125] [0.9921875] [0.6650479]\n",
"[0.7578125] [0.984375] [0.9030752]\n",
"[0.8671875] [0.9921875] [0.3678714]\n",
"[0.7421875] [0.9765625] [0.7424855]\n",
"[0.7890625] [1.] [0.6212543]\n",
"[0.8359375] [1.] [0.58529043]\n",
"[0.8203125] [0.96875] [0.5860813]\n",
"[0.8671875] [0.9921875] [0.415236]\n",
"[0.8125] [1.] [0.60501564]\n",
"[0.796875] [0.9765625] [0.60677457]\n",
"[0.8515625] [1.] [0.5338207]\n",
"[0.8046875] [0.9921875] [0.54180473]\n",
"[0.875] [0.9921875] [0.7293667]\n",
"[0.84375] [0.9765625] [0.5581689]\n",
"[0.8359375] [1.] [0.50712734]\n",
"[0.8671875] [0.9921875] [0.55217856]\n",
"[0.765625] [0.96875] [0.8076792]\n",
"[0.953125] [1.] [0.17031987]\n",
"[0.890625] [0.9921875] [0.42383268]\n",
"[0.828125] [0.9765625] [0.49300486]\n",
"[0.8671875] [0.96875] [0.57985115]\n",
"[0.8515625] [1.] [0.4901033]\n",
"[0.921875] [1.] [0.34583277]\n",
"[0.8984375] [0.984375] [0.41139168]\n",
"[0.9296875] [1.] [0.20420414]\n",
"[0.921875] [0.984375] [0.24322833]\n",
"[0.921875] [0.9921875] [0.30570173]\n",
"[0.875] [0.9921875] [0.3866225]\n",
"[0.9140625] [0.9921875] [0.20813875]\n",
"[0.9140625] [1.] [0.17933217]\n",
"[0.8984375] [0.9921875] [0.32508463]\n",
"[0.9375] [1.] [0.24799153]\n",
"[0.9140625] [1.] [0.26146784]\n",
"[0.90625] [1.] [0.24672262]\n",
"[0.8828125] [1.] [0.34094217]\n",
"[0.90625] [1.] [0.2964819]\n",
"[0.9296875] [1.] [0.18237087]\n",
"[0.84375] [1.] [0.7182543]\n",
"[0.8671875] [0.984375] [0.508474]\n",
"[0.8828125] [0.9921875] [0.367172]\n",
"[0.9453125] [1.] [0.2366665]\n",
"[0.9375] [1.] [0.12494276]\n",
"[0.8984375] [1.] [0.3395289]\n",
"[0.890625] [0.984375] [0.30877113]\n",
"[0.90625] [1.] [0.29763448]\n",
"[0.8828125] [0.984375] [0.4845504]\n",
"[0.8515625] [1.] [0.45548072]\n",
"[0.8828125] [1.] [0.33331633]\n",
"[0.90625] [1.] [0.4024018]\n",
"[0.890625] [0.984375] [0.73405886]\n",
"[0.9609375] [0.9921875] [0.15409982]\n",
"[0.9140625] [0.984375] [0.37103674]\n",
"[0.953125] [1.] [0.17628372]\n",
"[0.890625] [1.] [0.36522508]\n",
"[0.8828125] [1.] [0.407708]\n",
"[0.9375] [0.984375] [0.25090045]\n",
"[0.890625] [0.984375] [0.35742313]\n",
"[0.921875] [0.9921875] [0.2751101]\n",
"[0.890625] [0.984375] [0.43053097]\n",
"[0.875] [0.9921875] [0.34412643]\n",
"[0.90625] [1.] [0.35595697]\n"
]
},
{
"ename": "KeyboardInterrupt",
"evalue": "",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mKeyboardInterrupt\u001b[0m Traceback (most recent call last)",
"\u001b[0;32m<ipython-input-21-92f72657bddc>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mdata\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mtrain_reader\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0macc1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0macc5\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mloss\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mexe\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrun\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpruned_program\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfeed\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mtrain_feeder\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfeed\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdata\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfetch_list\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0moutputs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 3\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0macc1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0macc5\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mloss\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/local/lib/python3.5/dist-packages/paddle/fluid/executor.py\u001b[0m in \u001b[0;36mrun\u001b[0;34m(self, program, feed, fetch_list, feed_var_name, fetch_var_name, scope, return_numpy, use_program_cache)\u001b[0m\n\u001b[1;32m 776\u001b[0m \u001b[0mscope\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mscope\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 777\u001b[0m \u001b[0mreturn_numpy\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mreturn_numpy\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 778\u001b[0;31m use_program_cache=use_program_cache)\n\u001b[0m\u001b[1;32m 779\u001b[0m \u001b[0;32mexcept\u001b[0m \u001b[0mException\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 780\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0misinstance\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0me\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcore\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mEOFException\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/local/lib/python3.5/dist-packages/paddle/fluid/executor.py\u001b[0m in \u001b[0;36m_run_impl\u001b[0;34m(self, program, feed, fetch_list, feed_var_name, fetch_var_name, scope, return_numpy, use_program_cache)\u001b[0m\n\u001b[1;32m 829\u001b[0m \u001b[0mscope\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mscope\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 830\u001b[0m \u001b[0mreturn_numpy\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mreturn_numpy\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 831\u001b[0;31m use_program_cache=use_program_cache)\n\u001b[0m\u001b[1;32m 832\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 833\u001b[0m \u001b[0mprogram\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_compile\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mscope\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mplace\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/local/lib/python3.5/dist-packages/paddle/fluid/executor.py\u001b[0m in \u001b[0;36m_run_program\u001b[0;34m(self, program, feed, fetch_list, feed_var_name, fetch_var_name, scope, return_numpy, use_program_cache)\u001b[0m\n\u001b[1;32m 903\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0muse_program_cache\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 904\u001b[0m self._default_executor.run(program.desc, scope, 0, True, True,\n\u001b[0;32m--> 905\u001b[0;31m fetch_var_name)\n\u001b[0m\u001b[1;32m 906\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 907\u001b[0m self._default_executor.run_prepared_ctx(ctx, scope, False, False,\n",
"\u001b[0;31mKeyboardInterrupt\u001b[0m: "
]
}
],
"source": [
"for data in train_reader():\n",
" acc1, acc5, loss = exe.run(pruned_program, feed=train_feeder.feed(data), fetch_list=outputs)\n",
" print(acc1, acc5, loss)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.5.2"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
Loading

0 comments on commit 1cb8d1b

Please sign in to comment.