From 422a9a6f17523ae6e35b3cf051d66ad50f2486f8 Mon Sep 17 00:00:00 2001 From: tqchen Date: Fri, 18 Sep 2015 13:17:35 -0700 Subject: [PATCH 1/2] minor fix --- doc/contribute.md | 14 +++----------- doc/index.md | 4 +++- 2 files changed, 6 insertions(+), 12 deletions(-) diff --git a/doc/contribute.md b/doc/contribute.md index f7b4299a4403..39b16c8a47aa 100644 --- a/doc/contribute.md +++ b/doc/contribute.md @@ -16,16 +16,14 @@ Contribute to Documents * The document is created using sphinx and [recommonmark](http://recommonmark.readthedocs.org/en/latest/) * You can build document locally to see the effect. - Contribute to Testcases ----------------------- * All the testcases are in [tests](../tests) * We use python nose for python test cases and gtest for c++ unittests. - Contribute to Examples -------------------------- -* Usecases and examples will be in [examples](../examples) +---------------------- +* Usecases and examples will be in [example](../example) * We are super excited to hear about your story, if you have blogposts, tutorials code solutions using mxnet, please tell us and we will add a link in the example pages. @@ -38,14 +36,8 @@ git remote add upstream https://github.com/dmlc/mxnet git fetch upstream git rebase upstream/master ``` -* If you have multiple small commits that fixes small problems, +* If you have multiple small commits, it might be good to merge them together(use git rebase then squash) into more meaningful groups. * Send the pull request! - Fix the problems reported by automatic checks - If you are contributing a new module, consider add a testcase in [tests](../tests) - - - - - - diff --git a/doc/index.md b/doc/index.md index 65f8d2e1f77d..0693d9381eb6 100644 --- a/doc/index.md +++ b/doc/index.md @@ -5,12 +5,13 @@ This is document for mxnet, an efficient and flexible distributed framework for How to Get Started ------------------ For now, you can take a look at [Python User Guide](python/python_guide.md) and play with the -[examples](../examples) . More to come. +[example](../example) . More to come. Contents -------- * [Python User Guide](python/python_guide.md) * [Build Instruction](build.md) +* [Frequently Asked Questions](faq.md) * [Python API Reference](python/python_api.md) @@ -19,6 +20,7 @@ Developer Guide * [Contributor Guideline](contribute.md) * [Doxygen Version of C++ API](https://mxnet.readthedocs.org/en/latest/doxygen) + Indices and tables ------------------ From 93f21f19adcc55d2a58ef7c525e2cba6f5cdd16a Mon Sep 17 00:00:00 2001 From: tqchen Date: Fri, 18 Sep 2015 18:12:50 -0700 Subject: [PATCH 2/2] update document --- doc/build.md | 13 ++- doc/conf.py | 1 + doc/cpp/c_api.md | 15 --- doc/cpp/cpp_guide.md | 9 -- doc/cpp/symbolic.md | 15 --- doc/developer-guide/engine.md | 29 +++-- doc/developer-guide/index.md | 12 ++ doc/index.md | 15 ++- doc/overview.md | 21 ---- doc/python/index.md | 23 ++++ doc/python/io.md | 11 ++ doc/python/kvstore.md | 7 ++ doc/python/ndarray.md | 12 ++ doc/python/python_api.md | 36 ------ doc/python/symbol.md | 117 ++++++++++++++++++++ doc/python/{python_guide.md => tutorial.md} | 9 +- 16 files changed, 224 insertions(+), 121 deletions(-) delete mode 100644 doc/cpp/c_api.md delete mode 100644 doc/cpp/cpp_guide.md delete mode 100644 doc/cpp/symbolic.md create mode 100644 doc/developer-guide/index.md delete mode 100644 doc/overview.md create mode 100644 doc/python/index.md create mode 100644 doc/python/io.md create mode 100644 doc/python/kvstore.md create mode 100644 doc/python/ndarray.md delete mode 100644 doc/python/python_api.md create mode 100644 doc/python/symbol.md rename doc/python/{python_guide.md => tutorial.md} (98%) diff --git a/doc/build.md b/doc/build.md index 7d56740570d1..921f600d762a 100644 --- a/doc/build.md +++ b/doc/build.md @@ -1,5 +1,5 @@ -Build MXNet -=========== +Build and Installation +====================== - You can clone the mxnet from the [github repo](https://github.com/dmlc/mxnet) - After you clone the repo, update the submodules by ```bash @@ -8,3 +8,12 @@ git submodule update ``` - Copy [make/config.mk](../make/config.mk) to the project root, modify according to your desired setting. - Type ```make``` in the root folder. + + +Install Python Package +---------------------- +After you build the mxnet, you can install python package by +```bash +cd python +python setup.py install +``` diff --git a/doc/conf.py b/doc/conf.py index b2717e18daa8..2551a291a3e2 100644 --- a/doc/conf.py +++ b/doc/conf.py @@ -53,6 +53,7 @@ # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones extensions = [ 'sphinx.ext.autodoc', + 'sphinx.ext.autosummary', 'sphinx.ext.napoleon', 'sphinx.ext.mathjax', 'breathe', diff --git a/doc/cpp/c_api.md b/doc/cpp/c_api.md deleted file mode 100644 index 45172f74bd61..000000000000 --- a/doc/cpp/c_api.md +++ /dev/null @@ -1,15 +0,0 @@ -C API -===== -MXNet provides a C API interface that you can invoke. -This allows MXNet to interfacing with most other languages that supports -interfacing to C. - -If you are interested in porting MXNet to a new language, take a look at -the - - -API Reference -------------- -```eval_rst -.. doxygenfile:: c_api.h -``` \ No newline at end of file diff --git a/doc/cpp/cpp_guide.md b/doc/cpp/cpp_guide.md deleted file mode 100644 index 494aa63d6a99..000000000000 --- a/doc/cpp/cpp_guide.md +++ /dev/null @@ -1,9 +0,0 @@ -C++ API Document -================ -This is the place where you can find general guideline of mxnet's internal -as well as API References to each modules. - -Contents --------- -* [Symbolic Interface](symbolic.md) -* [C API](c_api.md) diff --git a/doc/cpp/symbolic.md b/doc/cpp/symbolic.md deleted file mode 100644 index 19addab2b91f..000000000000 --- a/doc/cpp/symbolic.md +++ /dev/null @@ -1,15 +0,0 @@ -Symbolic Interface -================== - -API Reference -------------- -```eval_rst -.. doxygenclass:: mxnet::StaticGraph - :members: - -.. doxygenclass:: mxnet::Symbol - :members: - -.. doxygenclass:: mxnet::Executor - :members: -``` \ No newline at end of file diff --git a/doc/developer-guide/engine.md b/doc/developer-guide/engine.md index be55cdd30c8c..9bfc7c0c5e56 100644 --- a/doc/developer-guide/engine.md +++ b/doc/developer-guide/engine.md @@ -1,17 +1,16 @@ Execution Engine ================ - MXNet's engine is not only for deep learning or any domain-specific problem. Rather, it is designed to face a general problem: execute a bunch of functions following their dependencies. Execution of any two functions with dependencies should be serialized. Functions with no dependencies *may* be executed in parallel to boost performance. Interface -============== +--------- The core interface of execution engine is: ```c++ virtual void Push(Fn exec_fun, Context exec_ctx, - std::vector const& use_vars, - std::vector const& mutate_vars) = 0; + std::vector const& const_vars, + std::vector const& mutate_vars) = 0; ``` -This API allows users to push a function (`exec_fun`), along with its context information and dependencies to the engine. The `exec_ctx` is the context information in which the `exec_fun` should be executed. `use_vars` denotes the variables that the function would read from while `mutate_vars` are the variables that to be modified. Regardless of the details that would be explained later, the engine guarantees following order: +This API allows users to push a function (`exec_fun`), along with its context information and dependencies to the engine. The `exec_ctx` is the context information in which the `exec_fun` should be executed. `const_vars` denotes the variables that the function would read from while `mutate_vars` are the variables that to be modified. Regardless of the details that would be explained later, the engine guarantees following order: >*The execution of any two functions that any one of them modifies at least one common variable would be serialized in their push order.* @@ -42,9 +41,9 @@ Context -------- User could specify the `Context` of the function to be executed within. This usually includes whether the function should be run on CPU or GPU, and if GPU, which GPU to use. `Context` is different from `RunContext`. `Context` contains device type (gpu/cpu) and device id while `RunContext` contains information that could only be decided during runtime like on which stream the function should be executed. -Variable +VarHandle -------- -`Variable` is used to specify the dependencies of functions. The design of MXNet engine is to decouple it with other modules in MXNet. So `Variable` is like an engine-given token for user to represent the external resources the functions may use or modified. It is designed to be light, so create, delete or copy a variable will incur little overhead. Upon pushing functions, users need to specify the variables that will be used (immutable) in `use_vars` vector and the variables to be modified (mutable) in `mutate_vars` vector. The only rule for the engine to resolve the dependencies among functions pushed is: +`VarHandle` is used to specify the dependencies of functions. The design of MXNet engine is to decouple it with other modules in MXNet. So `VarHandle` is like an engine-given token for user to represent the external resources the functions may use or modified. It is designed to be light, so create, delete or copy a variable will incur little overhead. Upon pushing functions, users need to specify the variables that will be used (immutable) in `const_vars` vector and the variables to be modified (mutable) in `mutate_vars` vector. The only rule for the engine to resolve the dependencies among functions pushed is: >*The execution of any two functions that any one of them modifies at least one common variable would be serialized in their push order.* @@ -65,15 +64,23 @@ If you want to wait for all `Fn` that involves (use/mutate) a certain variable t If you want to wait for all pushed `Fn` to be finished, use `WaitForAll()` API. Save Object Creation Cost ----------------------------- +------------------------- In some cases, you need to push several functions to the engine but for tons of times. If the computation of these functions are light, the overhead of copying lambdas and creating use/mutate variable lists would become relatively high. We provide an API to create an `OprHandle` beforehand: ```c++ virtual OprHandle NewOperator(AsyncFn fn, - std::vector const& use_vars, - std::vector const& mutate_vars) = 0; + std::vector const& const_vars, + std::vector const& mutate_vars) = 0; ``` So you could keep pushing the `OprHandle` without repeatedly creating them: ```c++ virtual void Push(OprHandle op, Context exec_ctx) = 0; ``` -To delete it, simply call `DeleteOperator(OprHandle op)` but please make sure the operator has finished computing. \ No newline at end of file +To delete it, simply call `DeleteOperator(OprHandle op)` but please make sure the operator has finished computing. + + +API Reference +------------- +```eval_rst +.. doxygenclass:: mxnet::Engine + :members: +``` \ No newline at end of file diff --git a/doc/developer-guide/index.md b/doc/developer-guide/index.md new file mode 100644 index 000000000000..4c69c0ff03ce --- /dev/null +++ b/doc/developer-guide/index.md @@ -0,0 +1,12 @@ +MXNet Developer Guide +===================== +This page contains links to all the developer related documents on mxnet. + +Overview of the Design +---------------------- +* [Execution Engine](engine.md) + +List of Resources +----------------- +* [Doxygen Version of C++ API](https://mxnet.readthedocs.org/en/latest/doxygen) +* [Contributor Guide](../contribute.md) diff --git a/doc/index.md b/doc/index.md index 0693d9381eb6..1b26bf2d26a4 100644 --- a/doc/index.md +++ b/doc/index.md @@ -4,19 +4,18 @@ This is document for mxnet, an efficient and flexible distributed framework for How to Get Started ------------------ -For now, you can take a look at [Python User Guide](python/python_guide.md) and play with the -[example](../example) . More to come. +* Check out [Python Getting started Guide](python/tutorial.md) +* The [example](../example) folder contains example usecases of mxnet. -Contents --------- -* [Python User Guide](python/python_guide.md) -* [Build Instruction](build.md) +User Guide +---------- +* [Python Package Document](python/index.md) +* [Build and Installation](build.md) * [Frequently Asked Questions](faq.md) -* [Python API Reference](python/python_api.md) - Developer Guide --------------- +* [Developer Documents](developer-guide/index.md) * [Contributor Guideline](contribute.md) * [Doxygen Version of C++ API](https://mxnet.readthedocs.org/en/latest/doxygen) diff --git a/doc/overview.md b/doc/overview.md deleted file mode 100644 index b04ed5e12b47..000000000000 --- a/doc/overview.md +++ /dev/null @@ -1,21 +0,0 @@ -MXNet Documentation -=================== -This is document of MXNet - -Get Started ------------ - -Tutorials ---------- - -User Guide ----------- -* [Symbolic Construction of Neural Nets](user-guide/symbol.md) -* [Execute the Symbol](user-guide/executor.md) - -Developer Guide ---------------- -* [Symbol Interface](developer-guide/symbol.md) -* [Executor](developer-guide/executor.md) -* [Implementing Operators](user-guide/operator.md) -* [DAG Engine](developer-guide/engine.md) diff --git a/doc/python/index.md b/doc/python/index.md new file mode 100644 index 000000000000..3a1713c81afa --- /dev/null +++ b/doc/python/index.md @@ -0,0 +1,23 @@ +MXNet Python Package +==================== +This page contains links to all the python related documents on python package. +To install the package package, checkout [Build and Installation Instruction](../build.md). +There are three types of documents you can find about mxnet. + +* [Tutorials](#tutorials) are self contained materials that introduces a certain usecases of mxnet. +* [Code Examples](#code-examples) contains links to codes. +* [Python API Documents](#python-api-documents) contains documents about specific module, as well as reference of all API functions. + +Tutorials +--------- +* [Python Overview Tutorial](tutorial.md) + +Code Examples +------------- +* [CIFAR 10 Example](../../example/cifar10) + +Python API Documents +-------------------- +* [NDArray API](ndarray.md) +* [Symbolic API](symbol.md) +* [Data Loading API](io.md) diff --git a/doc/python/io.md b/doc/python/io.md new file mode 100644 index 000000000000..b13d99700666 --- /dev/null +++ b/doc/python/io.md @@ -0,0 +1,11 @@ +Python Data Loading Interface +============================= + + +IO API Reference +---------------- + +```eval_rst +.. automodule:: mxnet.io + :members: +``` diff --git a/doc/python/kvstore.md b/doc/python/kvstore.md new file mode 100644 index 000000000000..148f43a650c4 --- /dev/null +++ b/doc/python/kvstore.md @@ -0,0 +1,7 @@ +KVStore API +=========== + +```eval_rst +.. automodule:: mxnet.kvstore + :members: +``` \ No newline at end of file diff --git a/doc/python/ndarray.md b/doc/python/ndarray.md new file mode 100644 index 000000000000..a661950df0c6 --- /dev/null +++ b/doc/python/ndarray.md @@ -0,0 +1,12 @@ +NDArray API +=========== + + + +NDArray API Reference +--------------------- + +```eval_rst +.. automodule:: mxnet.ndarray + :members: +``` diff --git a/doc/python/python_api.md b/doc/python/python_api.md deleted file mode 100644 index b1aa36418d30..000000000000 --- a/doc/python/python_api.md +++ /dev/null @@ -1,36 +0,0 @@ -# MXNet Python API - -## NDArray API - -```eval_rst -.. automodule:: mxnet.ndarray - :members: -``` - -## Symbol API - -```eval_rst -.. automodule:: mxnet.symbol - :members: -``` - -## Executor API - -```eval_rst -.. automodule:: mxnet.executor - :members: -``` - -## KVStore API - -```eval_rst -.. automodule:: mxnet.kvstore - :members: -``` - -## IO API - -```eval_rst -.. automodule:: mxnet.io - :members: -``` diff --git a/doc/python/symbol.md b/doc/python/symbol.md new file mode 100644 index 000000000000..3b08f72fdd03 --- /dev/null +++ b/doc/python/symbol.md @@ -0,0 +1,117 @@ +MXNet Python Symbolic API +========================= +* [How to Commpose Symbols](#overloaded-operators) introduces operator overloading of symbols +* [Serialization](#serialization) introduces how to save and load symbols. +* [Multiple Outputs](#multiple-outputs) introduces how to configure multiple outputs +* [API Reference](#api-reference) gives reference to all functions. +* [Symbol Object Document](#mxnet.symbol.Symbol) gives API reference to the Symbol Object. + +How to Compose Symbols +---------------------- +The symbolic API provides a way for you to configure the computation graphs. +You can do it in a level of neural network layer operations, as well as fine +grained operations. + +The following code gives an example of two layer neural network configuration. +```python +>>> import mxnet as mx +>>> net = mx.symbol.Variable('data') +>>> net = mx.symbol.FullyConnected(data=net, name='fc1', num_hidden=128) +>>> net = mx.symbol.Activation(data=net, name='relu1', act_type="relu") +>>> net = mx.symbol.FullyConnected(data=net, name='fc2', num_hidden=64) +>>> net = mx.symbol.Softmax(data=net, name='out') +>>> type(net) + +``` + +The basic arithematic operators(plus, minus, div, multiplication) are overloaded for +***elementwise operations*** of symbols. + +The following code gives an example of computation graph that add two inputs together. +```python +>>> import mxnet as mx +>>> a = mx.symbol.Variable('a') +>>> b = mx.symbol.Variable('b') +>>> c = a + b +```` + +Serialization +------------- +There are two ways to save and load the symbols. You can use pickle to serialize the ```Symbol``` objects. +Alternatively, you can use [mxnet.symbol.Symbol.save](#mxnet.symbol.Symbol.save) and [mxnet.symbol.load](#mxnet.symbol.load), functions. +The advantage of using save and load is that it is language agnostic, and also being cloud friendly. +The symbol is saved in json format. You can also directly get a json string using [mxnet.symbol.Symbol.tojson](#mxnet.symbol.Symbol.tojson) + +The following code gives an example of saving a symbol to S3 bucket, load it back and compare two symbols using json string. +```python +>>> import mxnet as mx +>>> a = mx.symbol.Variable('a') +>>> b = mx.symbol.Variable('b') +>>> c = a + b +>>> c.save('s3://my-bucket/symbol-c.json') +>>> c2 = mx.symbol.load('s3://my-bucket/symbol-c.json') +>>> c.tojson() == c2.tojson() +True +``` + +Multiple Ouputs +--------------- +You can use [mxnet.symbol.Group](#mxnet.symbol.Group) function to group the symbols together. + +```python +>>> import mxnet as mx +>>> net = mx.symbol.Variable('data') +>>> fc1 = mx.symbol.FullyConnected(data=net, name='fc1', num_hidden=128) +>>> net = mx.symbol.Activation(data=fc1, name='relu1', act_type="relu") +>>> net = mx.symbol.FullyConnected(data=net, name='fc2', num_hidden=64) +>>> out = mx.symbol.Softmax(data=net, name='softmax') +>>> group = mx.symbol.Group([fc1, out]) +>>> group.list_outputs() +['fc1_output', 'softmax_output'] +``` + +After you get the ```group```, you can go ahead and bind on ```group``` instead, +and the resulting executor will have two outputs, one for fc1_output and one for softmax_output. + +Symbol Creation API Reference +----------------------------- +This section contains the reference to all API functions. +Before you get started, you can check the list of functions in the following table. + +```eval_rst +.. autosummary:: + :nosignatures: + + mxnet.symbol.load + mxnet.symbol.load_json + mxnet.symbol.Group + mxnet.symbol.Variable + mxnet.symbol.Activation + mxnet.symbol.BatchNorm + mxnet.symbol.Concat + mxnet.symbol.Convolution + mxnet.symbol.Dropout + mxnet.symbol.ElementWiseSum + mxnet.symbol.Flatten + mxnet.symbol.FullyConnected + mxnet.symbol.LRN + mxnet.symbol.LeakyReLU + mxnet.symbol.Pooling + mxnet.symbol.Reshape + mxnet.symbol.Softmax +``` + +```eval_rst +.. automodule:: mxnet.symbol + :members: +``` + + +Execution API Reference +----------------------- + +```eval_rst +.. automodule:: mxnet.executor + :members: +``` + diff --git a/doc/python/python_guide.md b/doc/python/tutorial.md similarity index 98% rename from doc/python/python_guide.md rename to doc/python/tutorial.md index f70e73c087df..98d04c1226c2 100644 --- a/doc/python/python_guide.md +++ b/doc/python/tutorial.md @@ -1,19 +1,20 @@ -# MXNet Python Guide +MXNet Python Overview Tutorial +============================== This page gives a general overview of MXNet's python package. MXNet contains a mixed flavor of elements to bake flexible and efficient applications. There are mainly three concepts: -* Numpy style [NDArray](#ndarray-numpy-style-tensor-computations-on-cpus-and-gpus) +* [NDArray](#ndarray-numpy-style-tensor-computations-on-cpus-and-gpus) offers matrix and tensor computations on both CPU and GPU, with automatic parallelization - * [Symbol](#symbol-and-automatic-differentiation) makes defining a neural network extremely easy, and provides automatic differentiation. - * [KVStore](#distributed-key-value-store) easy the data synchronization between multi-GPUs and multi-machines. +You can find more information at [Python Package Overview Page](index.md) + ## NDArray: Numpy style tensor computations on CPUs and GPUs `NDArray` is the basic operation unit in MXNet for matrix and tensor