Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixing broken links #44

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 32 additions & 26 deletions python/basic/image_io.ipynb
Original file line number Diff line number Diff line change
@@ -1,11 +1,15 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# Image Data IO\n",
"This tutorial explains how to prepare, load and train with image data in MXNet. All IO in MXNet is handled via `mx.io.DataIter` and its subclasses, which is explained [here](./data.ipynb). In this tutorial we focus on how to use pre-built data iterators as while as custom iterators to process image data.\n",
"This tutorial explains how to prepare, load and train with image data in MXNet. All IO in MXNet is handled via `mx.io.DataIter` and its subclasses, which is explained [here](http://mxnet.io/tutorials/python/data.html). In this tutorial we focus on how to use pre-built data iterators as while as custom iterators to process image data.\n",
"\n",
"There are mainly three ways of loading image data in MXNet:\n",
"- [NEW] mx.img.ImageIter: implemented in python, easily customizable, can load from both .rec files and raw image files.\n",
Expand Down Expand Up @@ -67,11 +71,13 @@
]
},
{
"cell_type": "markdown",
"metadata": {},
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"Let's take a look at the data. As you can see, under the [root folder](./data/101_ObjectCategories) every category has a [subfolder](./data/101_ObjectCategories/yin_yang).\n",
"\n",
"Now let's convert them into record io format. First we need to make a list that contains all the image files and their categories:"
]
},
Expand All @@ -98,10 +104,14 @@
]
},
{
"cell_type": "markdown",
"metadata": {},
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"The resulting [list file](./data/caltech_train.lst) is in the format `index\\t(one or more label)\\tpath`. In this case there is only one label for each image but you can modify the list to add in more for multi label training.\n",
"The resulting list file is in the format `index\\t(one or more label)\\tpath`. In this case there is only one label for each image but you can modify the list to add in more for multi label training.\n",
"\n",
"Then we can use this list to create our record io file:"
]
Expand Down Expand Up @@ -129,10 +139,14 @@
]
},
{
"cell_type": "markdown",
"metadata": {},
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"The record io files are now saved at [here](./data)"
"The record io files are now saved at [here](https://github.com/dmlc/mxnet/blob/master/python/mxnet/io.py)"
]
},
{
Expand Down Expand Up @@ -180,25 +194,17 @@
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true
},
"source": [
"## Next Step\n",
"- [Record IO](record_io.ipynb) Read & Write RecordIO files with python interface\n",
"- [Advanced Image IO](advanced_img_io.ipynb) Advanced image IO for detection, segmentation, etc..."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
"source": [
"## Next Step\n",
"- [Record IO](http://mxnet.io/tutorials/python/record_io.html) Read & Write RecordIO files with python interface"
]
}
],
"metadata": {
Expand All @@ -217,7 +223,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.6"
"version": "2.7.12"
}
},
"nbformat": 4,
Expand Down
32 changes: 22 additions & 10 deletions python/basic/mixed.ipynb
Original file line number Diff line number Diff line change
@@ -1,14 +1,18 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# Mixed Programing * \n",
"\n",
"In this tutorial we show how to combine NDArray and Symbol together to train a neural network from scratch. This mixed programming flavor is one of the unique feature that make MXNet different to other frameworks. The `MX` term in MXNet also often means \"mixed\". \n",
"\n",
"Note that [`mx.module`](./module.ipynb) provides all functions will be implemented. So this tutorial is mainly for users who want to build things from scratches. \n",
"Note that [`mx.module`](http://mxnet.io/api/python/module.html) provides all functions will be implemented. So this tutorial is mainly for users who want to build things from scratches. \n",
"\n",
"## Training a Multi-layer Perception. \n",
"\n",
Expand Down Expand Up @@ -134,10 +138,14 @@
]
},
{
"cell_type": "markdown",
"metadata": {},
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"To run forward and backward, we need to bind data to all free variables first. We can create all `NDArray`s and then bind them as we did on the [Symbol tutorial](./symbol.ipynb). There is also function named `simple_bind` that simplifies this procedure. This function first inferences the shapes of all free variables by using the provided data shape, and then allocate and bind data, which can be accessed by the attribute `arg_arrays` of the returned executor. "
"To run forward and backward, we need to bind data to all free variables first. We can create all `NDArray`s and then bind them as we did on the [Symbol tutorial](http://mxnet.io/tutorials/python/symbol.html). There is also function named `simple_bind` that simplifies this procedure. This function first inferences the shapes of all free variables by using the provided data shape, and then allocate and bind data, which can be accessed by the attribute `arg_arrays` of the returned executor. "
]
},
{
Expand Down Expand Up @@ -352,12 +360,16 @@
]
},
{
"cell_type": "markdown",
"metadata": {},
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"## Data Parallelism with Multi-devices\n",
"\n",
"On the [NDArray tutorial](./ndarray.ipynb) we mentioned that the backend system is able to automatically parallel the computations. This feature makes developing parallel programs as easy as writing serial programs in MXNet. \n",
"On the [NDArray tutorial](http://mxnet.io/tutorials/python/ndarray.html) we mentioned that the backend system is able to automatically parallel the computations. This feature makes developing parallel programs as easy as writing serial programs in MXNet. \n",
"\n",
"Here we show how to develope a training program using mutliple devices, such as GPUs and CPUs, with data parallelism. In MXNet, a device means a computation resource with its own memory. It could be **a GPU chip** or **all CPUs chips**:\n",
"\n",
Expand Down Expand Up @@ -837,7 +849,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.6"
"version": "2.7.12"
}
},
"nbformat": 4,
Expand Down
22 changes: 15 additions & 7 deletions python/basic/module.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -125,8 +125,12 @@
]
},
{
"cell_type": "markdown",
"metadata": {},
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"### Create Module\n",
"\n",
Expand All @@ -139,7 +143,7 @@
"- data_names : the list of data variable names \n",
"- label_names : the list of label variable names\n",
"\n",
"One can refer to [data.ipynb](./data.ipynb) for more explanations about the last two arguments. Here we have only one data named `data`, and one label, with the name `softmax_label`, which is automatically named for us following the name `softmax` we specified for the `SoftmaxOutput` operator."
"One can refer to [data.ipynb](https://github.com/dmlc/mxnet-notebooks/blob/master/python/basic/data.ipynb) for more explanations about the last two arguments. Here we have only one data named `data`, and one label, with the name `softmax_label`, which is automatically named for us following the name `softmax` we specified for the `SoftmaxOutput` operator."
]
},
{
Expand Down Expand Up @@ -537,8 +541,12 @@
]
},
{
"cell_type": "markdown",
"metadata": {},
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"## More on Modules\n",
"\n",
Expand All @@ -556,7 +564,7 @@
"\n",
"## Futher Readings\n",
"\n",
"- [module API doc](http://mxnet.io/packages/python/module.html#module-interface-api)"
"- [module API doc](http://mxnet.io/api/python/module.html)"
]
}
],
Expand All @@ -577,7 +585,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.6"
"version": "2.7.12"
}
},
"nbformat": 4,
Expand Down
12 changes: 8 additions & 4 deletions python/basic/ndarray.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -1101,8 +1101,12 @@
]
},
{
"cell_type": "markdown",
"metadata": {},
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"## Current Status\n",
"\n",
Expand All @@ -1115,7 +1119,7 @@
"- Lack of reduce functions such as `max`, `min`...\n",
"\n",
"## Futher Readings\n",
"- [NDArray API](http://mxnet.dmlc.ml/en/latest/packages/python/ndarray.html) Documents for all NDArray methods.\n",
"- [NDArray API](http://mxnet.io/api/python/ndarray.html) Documents for all NDArray methods.\n",
"- [MinPy](https://github.com/dmlc/minpy) on-going project, fully numpy compatible with GPU and auto differentiation supports "
]
}
Expand All @@ -1137,7 +1141,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.6"
"version": "2.7.12"
}
},
"nbformat": 4,
Expand Down
48 changes: 32 additions & 16 deletions python/basic/symbol.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
"source": [
"# Symbol Tutorial\n",
"\n",
"Besides the tensor computation interface [NDArray](./ndarray.ipynb), another main object in MXNet is the `Symbol` provided by `mxnet.symbol`, or `mxnet.sym` for short. A symbol represents a multi-output symbolic expression. They are composited by operators, such as simple matrix operations (e.g. “+”), or a neural network layer (e.g. convolution layer). An operator can take several input variables, produce more than one output variables, and have internal state variables. A variable can be either free, which we can bind with value later, or an output of another symbol. \n",
"Besides the tensor computation interface [NDArray](https://github.com/dmlc/mxnet-notebooks/blob/master/python/basic/ndarray.ipynb), another main object in MXNet is the `Symbol` provided by `mxnet.symbol`, or `mxnet.sym` for short. A symbol represents a multi-output symbolic expression. They are composited by operators, such as simple matrix operations (e.g. “+”), or a neural network layer (e.g. convolution layer). An operator can take several input variables, produce more than one output variables, and have internal state variables. A variable can be either free, which we can bind with value later, or an output of another symbol. \n",
"\n",
"## Symbol Composition \n",
"### Basic Operators\n",
Expand Down Expand Up @@ -1049,10 +1049,14 @@
]
},
{
"cell_type": "markdown",
"metadata": {},
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"Finally we can obtain the whole network by chaining multiple inception modulas. A complete example is available at [mxnet/example/image-classification/symbol_inception-bn.py](https://github.com/dmlc/mxnet/blob/master/example/image-classification/symbol_inception-bn.py)"
"Finally we can obtain the whole network by chaining multiple inception modulas. A complete example is available at [https://github.com/dmlc/mxnet/blob/master/example/image-classification/symbols/inception-bn.py](https://github.com/dmlc/mxnet/blob/master/example/image-classification/symbols/inception-bn.py)"
]
},
{
Expand Down Expand Up @@ -1094,12 +1098,16 @@
]
},
{
"cell_type": "markdown",
"metadata": {},
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"## Relations to NDArray\n",
"\n",
"As can be seen now, both Symbol and NDArray provide multi-dimensional array operations, such as `c=a+b` in MXNet. Sometimes users are confused which way to use. We briefly clarify the difference here, more detailed explanation are available [here](http://mxnet.readthedocs.io/en/latest/system/program_model.html). \n",
"As can be seen now, both Symbol and NDArray provide multi-dimensional array operations, such as `c=a+b` in MXNet. Sometimes users are confused which way to use. We briefly clarify the difference here, more detailed explanation are available [here](http://mxnet.io/architecture/program_model.html). \n",
"\n",
"The `NDArray` provides an imperative programming alike interface, in which the computations are evaluated sentence by sentence. While `Symbol` is closer to declarative programming, in which we first declare the computation, and then evaluate with data. Examples in this category include regular expression and SQL.\n",
"\n",
Expand All @@ -1116,18 +1124,22 @@
"- easy to save, load, and visualization\n",
"- easy for the backend to optimize the computation and memory usage\n",
"\n",
"We will show on the [mixed programming tutorial](./mixed.ipynb) how these two interfaces can be used together to develop a complete training program. This tutorial will focus on the usage of Symbol. "
"We will show on the [mixed programming tutorial](https://github.com/dmlc/mxnet-notebooks/blob/master/python/basic/mixed.ipynb) how these two interfaces can be used together to develop a complete training program. This tutorial will focus on the usage of Symbol. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"## Symbol Manipulation *\n",
"\n",
"One important difference of `Symbol` comparing to `NDArray` is that, we first declare the computation, and then bind with data to run. \n",
"\n",
"In this section we introduce the functions to manipulate a symbol directly. But note that, most of them are wrapped nicely by the [`mx.module`](./module.ipynb). One can skip this section safely. \n",
"In this section we introduce the functions to manipulate a symbol directly. But note that, most of them are wrapped nicely by the [`mx.module`](http://mxnet.io/tutorials/python/module.html). One can skip this section safely. \n",
"\n",
"### Shape Inference\n",
"For each symbol, we can query its inputs (or arguments) and outputs. We can also inference the output shape by given the input shape, which facilitates memory allocation. "
Expand Down Expand Up @@ -1467,13 +1479,17 @@
]
},
{
"cell_type": "markdown",
"metadata": {},
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"## Further Readings\n",
"\n",
"- [Use torch operators in MXNet](http://mxnet.dmlc.ml/en/latest/how_to/torch.html)\n",
"- [Use Caffe operators in MXNet](http://dmlc.ml/mxnet/2016/07/29/use-caffe-operator-in-mxnet.html)"
"- [Use torch operators in MXNet](http://mxnet.io/how_to/torch.html)\n",
"- [Use Caffe operators in MXNet](http://mxnet.io/how_to/caffe.html)"
]
}
],
Expand All @@ -1494,7 +1510,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.6"
"version": "2.7.12"
}
},
"nbformat": 4,
Expand Down