Graph partitioner and subgraph op #11251

reminisce · 2018-06-13T00:55:56Z

Description

(Brief description on what this PR is about)

Checklist

Essentials

Please feel free to remove inapplicable items for your PR.

The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant JIRA issue created (except PRs with tiny changes)
Changes are complete (i.e. I finished coding on this PR)
All changes have test coverage:
Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
Code is well-documented:
For user-facing API changes, API doc string has been updated.
For new C++ functions in header files, their functionalities and arguments are documented.
For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes

Feature1, tests, (and when applicable, API doc)
Feature2, tests, (and when applicable, API doc)

Comments

If this change is a backward incompatible change, why must this change be made.
Interesting edge cases to note here

@piiswrong @zheng-da

TaoLv · 2018-06-14T15:05:18Z

example/subgraph_op/imagenet_inference.py

+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+


Could you write a simple README for this file? How to run this test? How can users play with this test?

You can use the following command to try it. Please remember to change ctx passed to the function score into mx.cpu() if you are running on a cpu. It's just a preliminary end-to-end run-through for proof of concept. More changes are coming.

python --model=imagenet1k-resnet-152 --dataset=data --num-inference-batches=10

TaoLv · 2018-06-14T15:07:18Z

example/subgraph_op/imagenet_inference.py

+    sym, arg_params, aux_params = mx.model.load_checkpoint(prefix, epoch)
+    op_names = ['BatchNorm', 'Convolution', 'Pooling', 'Activation']
+    out = SymbolHandle()
+    check_call(_LIB.MXPartitionGraph(sym.handle, mx_uint(len(op_names)), c_str_array(op_names),


How about wrapping this function into Symbol class?

We will hide this from users. The API design is still under discussion.

TaoLv · 2018-06-14T15:29:29Z

include/mxnet/op_attr_types.h

+  kCrossDeviceCopy,
+  /*!
+   * A subgraph execution should happen in the main thread, instead of
+   * in the execution engine.


operators inside a subgraph are still executed in the execution engine, right?

TaoLv · 2018-06-14T15:32:23Z

src/engine/engine_impl.h

@@ -33,8 +33,12 @@
 namespace mxnet {
 namespace engine {

+#if 0


TaoLv · 2018-06-14T15:34:16Z

src/engine/naive_engine.cc


 namespace mxnet {
 namespace engine {

+class NaiveVar final


please add a document for this class.

TaoLv · 2018-06-14T15:35:13Z

src/engine/naive_engine.cc

@@ -71,8 +81,11 @@ class NaiveEngine final : public Engine {

  // new variables
  VarHandle NewVariable() override {
+    return NaiveVar::New();
+#if 0


pengzhao-intel · 2018-06-15T01:05:56Z

Great, this will be very helpful for us 👍
Thanks, @reminisce @zheng-da

TaoLv · 2018-06-19T06:25:03Z

src/c_api/c_api_symbolic.cc

+  if (!op_name_set.empty()) {
+    mxnet::op::SubgraphPropertyPtr property
+        = std::make_shared<mxnet::op::DefaultSubgraphProperty>(op_name_set);
+    g.attrs["subgraph_property"] = std::make_shared<nnvm::any>(std::move(property));


Is it possible for mxnet to enable several acceleration backends at the same time? So then there will be several different subgraph_property for different backends respectively?

Not exactly. Your implementation of SubgraphProperty determines what subgraph op node is used and consequently, what Forward/Backward implementation is adopted. If you want to support multiple backend at the same time, you just need to configure that in the implementation of Forward/Backward. The question is, how do you determine when to use which backend?

zheng-da · 2018-06-20T02:08:30Z

src/operator/subgraph/partition_graph.cc

+
+      // filter out unqualified pre-selected nodes
+      std::vector<nnvm::Node*> filtered_nodes = preselected_nodes;
+      subgraph_selector->Filter(g, &filtered_nodes);


Should we have Filter to return a vector of Nodes that are selected?

Agree, it's easier for users to implement this function. I will make the change.

zheng-da · 2018-06-20T02:11:00Z

src/operator/subgraph/partition_graph.cc

+      }
+      // find out subgraphs from the filtered nodes
+      std::vector<std::vector<SimpleNode*>> subgraphs;
+      PostProcessNodeCandidates(*g, filtered_nodes, simple_nodes, &subgraphs, &subgraph_id);


Should we require the nodes returned from Filter belong to a single subgraph?

I'm not sure about this part. If users only care about the network pattern matching, requiring it to return a single subgraph might be too strict.

I mean finding a single subgraph from each node. The algorithm starts to search for subgraphs from every node. It should be sufficient to find at most one subgraph from every node.

I see. That seems clearer. I will change the logic of getting multiple subgraphs from the filtered nodes into getting a single subgraph starting from the seed node passed to PreSelectSubgraphNodes.

Fix

* add functions for cutting edges. * construct subgraphs. * generalize graph partition. * restructure the code. * create SubgraphOpState. * register subgraph property. * rename. * address comments. * update select API. * rename. * set subgraph property. * fix bugs. * fix bugs.

* remove SubgraphOperator. * change the interface of SubgraphSelector.

* update tests. * fix shape/dtype/storage inference. * fix.

* execute subgraph operators correctly. * add HasSubgraph attr. * support subgraph op in imperative. * rewrite running subgraph in main thread. * execute subgraph op in imperative. * remove need_grad.

eric-haibin-lin · 2018-06-21T00:58:05Z

src/operator/subgraph/common.h

+  exec::DevMaskVector dev_masks(idx_g.num_node_entries(), dev_mask);
+
+  // Put the input and output storages to the storage vector.
+  nnvm::StorageVector stypes(idx_g.num_node_entries(), exec::kBadStorageID);


nnvm::StorageVector is used for plan_memory, not for storage type inference

Thanks. Fixed.

Graph partitioner and subgraph op

Graph partitioner and subgraph op Fix duplicate entry bugs (apache#11767) Make subgraph var node name unique (apache#11876) [DO NOT REVIEW] Fix bug of eliminating cycles (apache#11907) * Fix cycle bug * Fix decycle bug * Fix comment [DO NOT REVIEW] Subgraph API (apache#12104) * Initial commit * Add unit tests * Fix lint * Fix lint * Clean up * Add graph partitiong to Bind * Add property name to graph partitioning c api * Fix unit test gpu context * Address cr * Move subgraph to attrs.subgraphs and fix the example * Fix lint * Add var version unit test * Address cr * Enable unit test that was flaky

* Graph partitioner and subgraph op (#11251) Graph partitioner and subgraph op Fix duplicate entry bugs (#11767) Make subgraph var node name unique (#11876) [DO NOT REVIEW] Fix bug of eliminating cycles (#11907) * Fix cycle bug * Fix decycle bug * Fix comment [DO NOT REVIEW] Subgraph API (#12104) * Initial commit * Add unit tests * Fix lint * Fix lint * Clean up * Add graph partitiong to Bind * Add property name to graph partitioning c api * Fix unit test gpu context * Address cr * Move subgraph to attrs.subgraphs and fix the example * Fix lint * Add var version unit test * Address cr * Enable unit test that was flaky * Clean up * Clean up * Clean up * Change version return type in NDArray * Clean up * Add register or get for subgraph prop registry * Address cr * Remove unnecessary code * Handle var version issue in naive engine * Delete example * Remove registration of resource request for default subgraph op * Add doc string * Improve doc string

* Graph partitioner and subgraph op (apache#11251) Graph partitioner and subgraph op Fix duplicate entry bugs (apache#11767) Make subgraph var node name unique (apache#11876) [DO NOT REVIEW] Fix bug of eliminating cycles (apache#11907) * Fix cycle bug * Fix decycle bug * Fix comment [DO NOT REVIEW] Subgraph API (apache#12104) * Initial commit * Add unit tests * Fix lint * Fix lint * Clean up * Add graph partitiong to Bind * Add property name to graph partitioning c api * Fix unit test gpu context * Address cr * Move subgraph to attrs.subgraphs and fix the example * Fix lint * Add var version unit test * Address cr * Enable unit test that was flaky * Clean up * Clean up * Clean up * Change version return type in NDArray * Clean up * Add register or get for subgraph prop registry * Address cr * Remove unnecessary code * Handle var version issue in naive engine * Delete example * Remove registration of resource request for default subgraph op * Add doc string * Improve doc string

reminisce requested a review from szha as a code owner June 13, 2018 00:55

reminisce changed the title ~~[WIP] Subgraph op~~ [WIP] Graph partitioner and subgraph op Jun 13, 2018

TaoLv reviewed Jun 14, 2018

View reviewed changes

TaoLv reviewed Jun 19, 2018

View reviewed changes

zheng-da reviewed Jun 20, 2018

View reviewed changes

reminisce and others added 23 commits June 20, 2018 16:28

subgraph op

4b2f049

Fix

Add treating whole graph as a subgraph

f343b0c

Add forward executor in subgraph op forward function

d13e489

Add unit test

0ea2763

Add finding subgraphs

89f321a

Add find input/output entries of a subgraph

1e17e87

Add get input/output entries

a337e09

Add debug info

d6bf59c

Handle corner case when a subgraph node is the last node of the graph

7a39d7f

Add var version for ndarray

10b67b6

Fix unit test

051c184

change the interface (#13)

382c792

* remove SubgraphOperator. * change the interface of SubgraphSelector.

Reorder input/output entries to follow original topological sort order

59652ef

Make subgraph op node name unique

22d2c9d

Fix sorting entries

c81da86

Allow for adjacent subgraphs

31cc8fc

Enable partitioning algo to eliminate cycles

1293828

fix shape/dtype/storage inference. (#15)

987f145

* update tests. * fix shape/dtype/storage inference. * fix.

Remove graph executor from subgraph op forward

c95e912

Naive engine end to end run through

7e96475

execute subgraph operators synchronously (#16)

8e3d4ed

* execute subgraph operators correctly. * add HasSubgraph attr. * support subgraph op in imperative. * rewrite running subgraph in main thread. * execute subgraph op in imperative. * remove need_grad.

Fix graph partitioning bug and add var version for naive engine

a642047

reminisce and others added 11 commits June 20, 2018 16:28

Refactor

63dcbcf

Toggle engine var version print out

575ef94

Add virtual destructor to engine var

41a97c8

Fix gpu build and add example

0227c27

Fix FMutateInputs for subgraph op

a9cf101

Add unit test for FMutateInputs attr of subgraph

d71467b

fix DefaultSubgraphOpMutableInputs (#17)

cf7c4fe

Add post processing algorithm for filtering out nodes

50d0e60

Refactor post processing

dcf2750

Change file names

6e29456

Address cr

c6d6c3b

reminisce force-pushed the subgraph_op branch from f1f0ffc to c6d6c3b Compare June 20, 2018 23:29

Fix lint

d24900c

eric-haibin-lin reviewed Jun 21, 2018

View reviewed changes

reminisce added 2 commits June 20, 2018 23:20

Fix storage type vector

5660a85

Temporarily disable test_gluon_trainer.test_trainer_reset_kv

5088603

reminisce changed the title ~~[WIP] Graph partitioner and subgraph op~~ Graph partitioner and subgraph op Jun 21, 2018

reminisce merged commit 9567bd7 into apache:subgraph Jun 21, 2018

reminisce added a commit that referenced this pull request Aug 4, 2018

Graph partitioner and subgraph op (#11251)

20246bf

Graph partitioner and subgraph op

ashokei pushed a commit to NervanaSystems/incubator-mxnet that referenced this pull request Aug 4, 2018

Graph partitioner and subgraph op (apache#11251)

31a9fc8

Graph partitioner and subgraph op

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Graph partitioner and subgraph op #11251

Graph partitioner and subgraph op #11251

reminisce commented Jun 13, 2018

TaoLv Jun 14, 2018

reminisce Jun 14, 2018 •

edited

Loading

TaoLv Jun 14, 2018

reminisce Jun 14, 2018

TaoLv Jun 14, 2018

reminisce Jun 14, 2018

TaoLv Jun 14, 2018

TaoLv Jun 14, 2018

TaoLv Jun 14, 2018

reminisce Jun 14, 2018

pengzhao-intel commented Jun 15, 2018

TaoLv Jun 19, 2018

reminisce Jun 19, 2018

zheng-da Jun 20, 2018

reminisce Jun 20, 2018

zheng-da Jun 20, 2018

reminisce Jun 20, 2018

zheng-da Jun 20, 2018

reminisce Jun 20, 2018

eric-haibin-lin Jun 21, 2018

reminisce Jun 21, 2018

Graph partitioner and subgraph op #11251

Graph partitioner and subgraph op #11251

Conversation

reminisce commented Jun 13, 2018

Description

Checklist

Essentials

Changes

Comments

Choose a reason for hiding this comment

reminisce Jun 14, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pengzhao-intel commented Jun 15, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

reminisce Jun 14, 2018 •

edited

Loading