convenience fluent methods for sym/nd #7584

szha · 2017-08-23T23:20:40Z

No description provided.

piiswrong · 2017-08-23T23:26:12Z

python/mxnet/ndarray/ndarray.py

@@ -673,6 +676,251 @@ def reshape(self, shape):
                                         ctypes.byref(handle)))
        return NDArray(handle=handle, writable=self.writable)

+    def flatten(self):


you can simply do flatten = flatten

If I do this, do I still need to add doc string?

piiswrong · 2017-08-24T06:10:30Z

python/mxnet/ndarray/ndarray.py

@@ -42,6 +42,9 @@
 from . import broadcast_sub, broadcast_div, broadcast_to, broadcast_equal, cast_storage
 from . import broadcast_greater, broadcast_greater_equal, broadcast_lesser, broadcast_lesser_equal
 from . import zeros_like, slice
+from . import flatten, expand_dims, tile, flip


do you have to import these ops directly? How about import the namespace?

piiswrong · 2017-08-24T17:31:53Z

python/mxnet/ndarray/ndarray.py

@@ -673,6 +671,26 @@ def reshape(self, shape):
                                         ctypes.byref(handle)))
        return NDArray(handle=handle, writable=self.writable)

+    flatten = flatten


Does document display properly for these functions?

reminisce · 2017-08-25T21:10:03Z

python/mxnet/ndarray/ndarray.py

+        """Convenience fluent method for ndarray.trunc.
+        See trunc"""
+        return op.trunc(self, **kwargs)
+


Do we need to support repeat as well?

I was limiting the methods to unary ops with 0 or 1 argument before. Let me revisit the op list and see what else can be added.

reminisce · 2017-08-25T21:13:57Z

python/mxnet/ndarray/ndarray.py

@@ -673,6 +686,100 @@ def reshape(self, shape):
                                         ctypes.byref(handle)))
        return NDArray(handle=handle, writable=self.writable)

+    def flatten(self, **kwargs):


Do we need to support positional arguments as well? At least, the following two usages of the op through mx.nd both work:

mx.nd.tile(a, reps=(2, 3)) mx.nd.tile(a, (2, 3))

Will add. Thanks

eric-haibin-lin · 2017-08-28T18:03:56Z

docs/api/python/ndarray.md

@@ -118,7 +118,58 @@ We summarize the interface for each class in the following sections.

    NDArray.T
    NDArray.reshape
+    NDarray.flatten
+    NDarray.expand_dims


NDArray instead of NDarray :P

szha · 2017-08-28T22:34:07Z

Ready for review. Document now works and examples can be found at:
http://mxnet-doc.s3-website-us-east-1.amazonaws.com/api/python/ndarray.html#the-ndarray-class
http://mxnet-doc.s3-website-us-east-1.amazonaws.com/api/python/symbol.html#the-symbol-class

* 1x1 convolution acceleration * GEMM directly without im2col or col2im in 1x1 convolution(stride=1,pad=0). The 1x1 convolution is used very common in modern CNN networks such as Googlenet/Inception/Resnet/Mobilenet etc. * cpplint * fix linalg_impl (#7611) * fix linalg_impl * fix * fix * fix * set build status to success only after job ends (#7628) Earlier code marks status as success initially. So any new PR shows jenkins status as success if we see the check mark on github. On opening the full build status, we see that builds haven't even started or are running. If something fails, variable changes to failure then. So even without this merge, a red mark on github indicates that build has failed correctly. That behavior is unchanged. * Fix build status of a test (#7629) installs bc required by sh2ju.sh and changes the regex match to capital alphabet as it clashes with a warning thrown by opencv driver * entire codebase build with mshadow_use_clas=0 (#7625) * Update README.md (#7630) * unit test for csv iter, doc update for libsvmiter (#7623) * add unit test for csv iter * fix lint * add libsvm to mxnet.io doc * update libsvm doc * gpu access of ndarray (#7496) * gpu access of ndarray * gpu access from C++ api * gpu access fix * Update c_api.cc * Update c_api.cc * refactor cudnn algo reg to no use string (#7561) * refactor cudnn algo reg to no use string * refactor ctx list * fix * refactor save_inputs * Update io.md (#7634) * fix tests (#7633) * [build] explicitly install JDK8 (#7574) * explicitly install openjdk8 * handle earlier version of ubuntu * install software-properties-common * update -y * update commands * Indents correction * Add script to build doc files for all versions (#7636) * Add script to build doc files for all versions * Fix * Use add versipn script of each different version * add fashion mnist and move mnists to s3 (#7635) * add fashion mnist and move mnists to s3 * refactor * add doc for dataset (#7644) * Change apache package URL to https (#7622) * Pip installer for CoreML Converter: mxnet-to-coreml (#7624) * Fixing CoreML converter's README: typos/grammar/etc. * CoreML converter README update: Talk about layers first and then about models. * Providing examples on converting various standard models; calling out issues with InceptionV3. * Fixing CoreML converter's README: typos/grammar/etc. * CoreML converter README update: Talk about layers first and then about models. * Providing examples on converting various standard models; calling out issues with InceptionV3. * Pip installer for converter: mxnet-coreml-converter. Runs only on MacOS and python 2.7. Once inside the directory pip_package, user needs to run: python setup.py bdist_wheel twine upload dist/* Once uploaded it'll look like this: https://testpypi.python.org/pypi/mxnet-coreml-converter Also updated the README for converter to reflect this. Note that we are going with a package per tool for the time being. Please leave feedback if you think it is better to adopt the policy of all the tools in one single package. Unit tests continue to pass. * More informative pypi package documentation. * Updating MacOS in release notes to 10.11 after testing on it. * Changing the name to mxnet-to-coreml and version to 0.1.0. * Added license to setup.py * Updating readme files with the correct pip package name. * Parallelize windows unit tests of python 2 and 3 in jenkins (#7646) * parallelize python windows tests * reordered for clarity * Removed asset loaded insecurely and added the asset to be loaded from the origin securely (#7649) * skip failing test temporarily (#7648) * lower really high threshold to fix test failure (#7650) * Doc updates for install and doc generation (#7647) * fluent (#7584) * add 1x1 convolution to tests * indent * Refactor random linalg contrib namespaces (#7604) * Refactor namespaces contrib, linalg, random, and sparse for op registration Change examples in documentation Change namespace usage in examples Fix pylint Remove unused import Switch name and alias in linalg and random Change stype comparison from string to int for functions used internally Change documentation to use the right namespace Register ops under ndarray/op.py and symbol/op.py Remove unused import Change .cu op names * Add __all__ to ndarray and symbol modules * Revert "Add __all__ to ndarray and symbol modules" This reverts commit 8bc5de7. * Add __all__ to ndarray and symbol modules * fix gluon fasionmnist dataset (#7655) fix gluon fasionmnist dataset * Parallelize Python 2 and 3 unit test cases in Jenkins CI. (#7658) * Parallelize Python 2 and 3 unit test cases. * Parallelize python 2 and 3 unit tests cases in jenkins * Parallelize python 2 and 3 unit tests cases in jenkins * Change namespace and make logging functionality changes (#7627) * Change namespace and make logging functionality changes * Help comment changes * update mklml and mkl mac support (#7587) * 1x1 convolution acceleration * GEMM directly without im2col or col2im in 1x1 convolution(stride=1,pad=0). The 1x1 convolution is used very common in modern CNN networks such as Googlenet/Inception/Resnet/Mobilenet etc. * cpplint * Indents correction * add 1x1 convolution to tests * indent * 1x1 convolution acceleration * GEMM directly without im2col or col2im in 1x1 convolution(stride=1,pad=0). The 1x1 convolution is used very common in modern CNN networks such as Googlenet/Inception/Resnet/Mobilenet etc. * cpplint * Indents correction * add 1x1 convolution to tests * indent * cpplint * indent

* 1x1 convolution acceleration * GEMM directly without im2col or col2im in 1x1 convolution(stride=1,pad=0). The 1x1 convolution is used very common in modern CNN networks such as Googlenet/Inception/Resnet/Mobilenet etc. * cpplint * fix linalg_impl (apache#7611) * fix linalg_impl * fix * fix * fix * set build status to success only after job ends (apache#7628) Earlier code marks status as success initially. So any new PR shows jenkins status as success if we see the check mark on github. On opening the full build status, we see that builds haven't even started or are running. If something fails, variable changes to failure then. So even without this merge, a red mark on github indicates that build has failed correctly. That behavior is unchanged. * Fix build status of a test (apache#7629) installs bc required by sh2ju.sh and changes the regex match to capital alphabet as it clashes with a warning thrown by opencv driver * entire codebase build with mshadow_use_clas=0 (apache#7625) * Update README.md (apache#7630) * unit test for csv iter, doc update for libsvmiter (apache#7623) * add unit test for csv iter * fix lint * add libsvm to mxnet.io doc * update libsvm doc * gpu access of ndarray (apache#7496) * gpu access of ndarray * gpu access from C++ api * gpu access fix * Update c_api.cc * Update c_api.cc * refactor cudnn algo reg to no use string (apache#7561) * refactor cudnn algo reg to no use string * refactor ctx list * fix * refactor save_inputs * Update io.md (apache#7634) * fix tests (apache#7633) * [build] explicitly install JDK8 (apache#7574) * explicitly install openjdk8 * handle earlier version of ubuntu * install software-properties-common * update -y * update commands * Indents correction * Add script to build doc files for all versions (apache#7636) * Add script to build doc files for all versions * Fix * Use add versipn script of each different version * add fashion mnist and move mnists to s3 (apache#7635) * add fashion mnist and move mnists to s3 * refactor * add doc for dataset (apache#7644) * Change apache package URL to https (apache#7622) * Pip installer for CoreML Converter: mxnet-to-coreml (apache#7624) * Fixing CoreML converter's README: typos/grammar/etc. * CoreML converter README update: Talk about layers first and then about models. * Providing examples on converting various standard models; calling out issues with InceptionV3. * Fixing CoreML converter's README: typos/grammar/etc. * CoreML converter README update: Talk about layers first and then about models. * Providing examples on converting various standard models; calling out issues with InceptionV3. * Pip installer for converter: mxnet-coreml-converter. Runs only on MacOS and python 2.7. Once inside the directory pip_package, user needs to run: python setup.py bdist_wheel twine upload dist/* Once uploaded it'll look like this: https://testpypi.python.org/pypi/mxnet-coreml-converter Also updated the README for converter to reflect this. Note that we are going with a package per tool for the time being. Please leave feedback if you think it is better to adopt the policy of all the tools in one single package. Unit tests continue to pass. * More informative pypi package documentation. * Updating MacOS in release notes to 10.11 after testing on it. * Changing the name to mxnet-to-coreml and version to 0.1.0. * Added license to setup.py * Updating readme files with the correct pip package name. * Parallelize windows unit tests of python 2 and 3 in jenkins (apache#7646) * parallelize python windows tests * reordered for clarity * Removed asset loaded insecurely and added the asset to be loaded from the origin securely (apache#7649) * skip failing test temporarily (apache#7648) * lower really high threshold to fix test failure (apache#7650) * Doc updates for install and doc generation (apache#7647) * fluent (apache#7584) * add 1x1 convolution to tests * indent * Refactor random linalg contrib namespaces (apache#7604) * Refactor namespaces contrib, linalg, random, and sparse for op registration Change examples in documentation Change namespace usage in examples Fix pylint Remove unused import Switch name and alias in linalg and random Change stype comparison from string to int for functions used internally Change documentation to use the right namespace Register ops under ndarray/op.py and symbol/op.py Remove unused import Change .cu op names * Add __all__ to ndarray and symbol modules * Revert "Add __all__ to ndarray and symbol modules" This reverts commit 8bc5de7. * Add __all__ to ndarray and symbol modules * fix gluon fasionmnist dataset (apache#7655) fix gluon fasionmnist dataset * Parallelize Python 2 and 3 unit test cases in Jenkins CI. (apache#7658) * Parallelize Python 2 and 3 unit test cases. * Parallelize python 2 and 3 unit tests cases in jenkins * Parallelize python 2 and 3 unit tests cases in jenkins * Change namespace and make logging functionality changes (apache#7627) * Change namespace and make logging functionality changes * Help comment changes * update mklml and mkl mac support (apache#7587) * 1x1 convolution acceleration * GEMM directly without im2col or col2im in 1x1 convolution(stride=1,pad=0). The 1x1 convolution is used very common in modern CNN networks such as Googlenet/Inception/Resnet/Mobilenet etc. * cpplint * Indents correction * add 1x1 convolution to tests * indent * 1x1 convolution acceleration * GEMM directly without im2col or col2im in 1x1 convolution(stride=1,pad=0). The 1x1 convolution is used very common in modern CNN networks such as Googlenet/Inception/Resnet/Mobilenet etc. * cpplint * Indents correction * add 1x1 convolution to tests * indent * cpplint * indent

piiswrong reviewed Aug 23, 2017

View reviewed changes

szha force-pushed the fluent branch from fea6a36 to d993971 Compare August 23, 2017 23:31

piiswrong reviewed Aug 24, 2017

View reviewed changes

szha force-pushed the fluent branch from 792ef4c to 20e34d6 Compare August 24, 2017 06:44

piiswrong reviewed Aug 24, 2017

View reviewed changes

szha mentioned this pull request Aug 24, 2017

Fix import error of broadcast max, min, mod in ndarray.py #7572

Merged

szha force-pushed the fluent branch 5 times, most recently from e17f4c2 to 8fa7984 Compare August 25, 2017 19:57

reminisce reviewed Aug 25, 2017

View reviewed changes

reminisce mentioned this pull request Aug 25, 2017

Refactor random linalg contrib namespaces #7604

Merged

szha force-pushed the fluent branch 4 times, most recently from 673d484 to 3053cd8 Compare August 28, 2017 07:18

eric-haibin-lin suggested changes Aug 28, 2017

View reviewed changes

szha force-pushed the fluent branch 2 times, most recently from 4e874a1 to edfb56f Compare August 28, 2017 18:32

fluent

6a3cd3a

szha force-pushed the fluent branch from edfb56f to 6a3cd3a Compare August 28, 2017 22:33

piiswrong merged commit 5591f42 into apache:master Aug 29, 2017

szha deleted the fluent branch August 29, 2017 03:12

mbaijal pushed a commit to mbaijal/incubator-mxnet that referenced this pull request Sep 6, 2017

fluent (apache#7584)

8701198

cjolivier01 pushed a commit to cjolivier01/mxnet that referenced this pull request Sep 11, 2017

fluent (apache#7584)

6cfb10f

crazy-cat pushed a commit to crazy-cat/incubator-mxnet that referenced this pull request Oct 26, 2017

fluent (apache#7584)

eb68f3e

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

convenience fluent methods for sym/nd #7584

convenience fluent methods for sym/nd #7584

szha commented Aug 23, 2017

piiswrong Aug 23, 2017

szha Aug 23, 2017

piiswrong Aug 24, 2017

piiswrong Aug 24, 2017

reminisce Aug 25, 2017

szha Aug 25, 2017

reminisce Aug 25, 2017

szha Aug 25, 2017

eric-haibin-lin Aug 28, 2017

szha commented Aug 28, 2017

convenience fluent methods for sym/nd #7584

convenience fluent methods for sym/nd #7584

Conversation

szha commented Aug 23, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

szha commented Aug 28, 2017