Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

Commit

Permalink
update document
Browse files Browse the repository at this point in the history
  • Loading branch information
tqchen committed Sep 19, 2015
1 parent 422a9a6 commit 93f21f1
Show file tree
Hide file tree
Showing 16 changed files with 224 additions and 121 deletions.
13 changes: 11 additions & 2 deletions doc/build.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
Build MXNet
===========
Build and Installation
======================
- You can clone the mxnet from the [github repo](https://github.com/dmlc/mxnet)
- After you clone the repo, update the submodules by
```bash
Expand All @@ -8,3 +8,12 @@ git submodule update
```
- Copy [make/config.mk](../make/config.mk) to the project root, modify according to your desired setting.
- Type ```make``` in the root folder.


Install Python Package
----------------------
After you build the mxnet, you can install python package by
```bash
cd python
python setup.py install
```
1 change: 1 addition & 0 deletions doc/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones
extensions = [
'sphinx.ext.autodoc',
'sphinx.ext.autosummary',
'sphinx.ext.napoleon',
'sphinx.ext.mathjax',
'breathe',
Expand Down
15 changes: 0 additions & 15 deletions doc/cpp/c_api.md

This file was deleted.

9 changes: 0 additions & 9 deletions doc/cpp/cpp_guide.md

This file was deleted.

15 changes: 0 additions & 15 deletions doc/cpp/symbolic.md

This file was deleted.

29 changes: 18 additions & 11 deletions doc/developer-guide/engine.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,16 @@
Execution Engine
================

MXNet's engine is not only for deep learning or any domain-specific problem. Rather, it is designed to face a general problem: execute a bunch of functions following their dependencies. Execution of any two functions with dependencies should be serialized. Functions with no dependencies *may* be executed in parallel to boost performance.

Interface
==============
---------
The core interface of execution engine is:
```c++
virtual void Push(Fn exec_fun, Context exec_ctx,
std::vector<Variable> const& use_vars,
std::vector<Variable> const& mutate_vars) = 0;
std::vector<VarHandle> const& const_vars,
std::vector<VarHandle> const& mutate_vars) = 0;
```
This API allows users to push a function (`exec_fun`), along with its context information and dependencies to the engine. The `exec_ctx` is the context information in which the `exec_fun` should be executed. `use_vars` denotes the variables that the function would read from while `mutate_vars` are the variables that to be modified. Regardless of the details that would be explained later, the engine guarantees following order:
This API allows users to push a function (`exec_fun`), along with its context information and dependencies to the engine. The `exec_ctx` is the context information in which the `exec_fun` should be executed. `const_vars` denotes the variables that the function would read from while `mutate_vars` are the variables that to be modified. Regardless of the details that would be explained later, the engine guarantees following order:
>*The execution of any two functions that any one of them modifies at least one common variable would be serialized in their push order.*
Expand Down Expand Up @@ -42,9 +41,9 @@ Context
--------
User could specify the `Context` of the function to be executed within. This usually includes whether the function should be run on CPU or GPU, and if GPU, which GPU to use. `Context` is different from `RunContext`. `Context` contains device type (gpu/cpu) and device id while `RunContext` contains information that could only be decided during runtime like on which stream the function should be executed.

Variable
VarHandle
--------
`Variable` is used to specify the dependencies of functions. The design of MXNet engine is to decouple it with other modules in MXNet. So `Variable` is like an engine-given token for user to represent the external resources the functions may use or modified. It is designed to be light, so create, delete or copy a variable will incur little overhead. Upon pushing functions, users need to specify the variables that will be used (immutable) in `use_vars` vector and the variables to be modified (mutable) in `mutate_vars` vector. The only rule for the engine to resolve the dependencies among functions pushed is:
`VarHandle` is used to specify the dependencies of functions. The design of MXNet engine is to decouple it with other modules in MXNet. So `VarHandle` is like an engine-given token for user to represent the external resources the functions may use or modified. It is designed to be light, so create, delete or copy a variable will incur little overhead. Upon pushing functions, users need to specify the variables that will be used (immutable) in `const_vars` vector and the variables to be modified (mutable) in `mutate_vars` vector. The only rule for the engine to resolve the dependencies among functions pushed is:

>*The execution of any two functions that any one of them modifies at least one common variable would be serialized in their push order.*

Expand All @@ -65,15 +64,23 @@ If you want to wait for all `Fn` that involves (use/mutate) a certain variable t
If you want to wait for all pushed `Fn` to be finished, use `WaitForAll()` API.

Save Object Creation Cost
----------------------------
-------------------------
In some cases, you need to push several functions to the engine but for tons of times. If the computation of these functions are light, the overhead of copying lambdas and creating use/mutate variable lists would become relatively high. We provide an API to create an `OprHandle` beforehand:
```c++
virtual OprHandle NewOperator(AsyncFn fn,
std::vector<Variable> const& use_vars,
std::vector<Variable> const& mutate_vars) = 0;
std::vector<VarHandle> const& const_vars,
std::vector<VarHandle> const& mutate_vars) = 0;
```
So you could keep pushing the `OprHandle` without repeatedly creating them:
```c++
virtual void Push(OprHandle op, Context exec_ctx) = 0;
```
To delete it, simply call `DeleteOperator(OprHandle op)` but please make sure the operator has finished computing.
To delete it, simply call `DeleteOperator(OprHandle op)` but please make sure the operator has finished computing.


API Reference
-------------
```eval_rst
.. doxygenclass:: mxnet::Engine
:members:
```
12 changes: 12 additions & 0 deletions doc/developer-guide/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
MXNet Developer Guide
=====================
This page contains links to all the developer related documents on mxnet.

Overview of the Design
----------------------
* [Execution Engine](engine.md)

List of Resources
-----------------
* [Doxygen Version of C++ API](https://mxnet.readthedocs.org/en/latest/doxygen)
* [Contributor Guide](../contribute.md)
15 changes: 7 additions & 8 deletions doc/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,19 +4,18 @@ This is document for mxnet, an efficient and flexible distributed framework for

How to Get Started
------------------
For now, you can take a look at [Python User Guide](python/python_guide.md) and play with the
[example](../example) . More to come.
* Check out [Python Getting started Guide](python/tutorial.md)
* The [example](../example) folder contains example usecases of mxnet.

Contents
--------
* [Python User Guide](python/python_guide.md)
* [Build Instruction](build.md)
User Guide
----------
* [Python Package Document](python/index.md)
* [Build and Installation](build.md)
* [Frequently Asked Questions](faq.md)
* [Python API Reference](python/python_api.md)


Developer Guide
---------------
* [Developer Documents](developer-guide/index.md)
* [Contributor Guideline](contribute.md)
* [Doxygen Version of C++ API](https://mxnet.readthedocs.org/en/latest/doxygen)

Expand Down
21 changes: 0 additions & 21 deletions doc/overview.md

This file was deleted.

23 changes: 23 additions & 0 deletions doc/python/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
MXNet Python Package
====================
This page contains links to all the python related documents on python package.
To install the package package, checkout [Build and Installation Instruction](../build.md).
There are three types of documents you can find about mxnet.

* [Tutorials](#tutorials) are self contained materials that introduces a certain usecases of mxnet.
* [Code Examples](#code-examples) contains links to codes.
* [Python API Documents](#python-api-documents) contains documents about specific module, as well as reference of all API functions.

Tutorials
---------
* [Python Overview Tutorial](tutorial.md)

Code Examples
-------------
* [CIFAR 10 Example](../../example/cifar10)

Python API Documents
--------------------
* [NDArray API](ndarray.md)
* [Symbolic API](symbol.md)
* [Data Loading API](io.md)
11 changes: 11 additions & 0 deletions doc/python/io.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
Python Data Loading Interface
=============================


IO API Reference
----------------

```eval_rst
.. automodule:: mxnet.io
:members:
```
7 changes: 7 additions & 0 deletions doc/python/kvstore.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
KVStore API
===========

```eval_rst
.. automodule:: mxnet.kvstore
:members:
```
12 changes: 12 additions & 0 deletions doc/python/ndarray.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
NDArray API
===========



NDArray API Reference
---------------------

```eval_rst
.. automodule:: mxnet.ndarray
:members:
```
36 changes: 0 additions & 36 deletions doc/python/python_api.md

This file was deleted.

Loading

0 comments on commit 93f21f1

Please sign in to comment.