Skip to content

Commit

Permalink
Squashed commit of the following:
Browse files Browse the repository at this point in the history
commit 0f65ef6
Author: Xi Wang <[email protected]>
Date:   Wed Aug 5 10:48:50 2020 +0800

    nb fix (apache#18858)

commit 7b7cef5
Author: Serge Panev <[email protected]>
Date:   Tue Aug 4 18:23:48 2020 -0700

     MXNet-TRT: Add PrePartition param caching - move init_tensorrt_params logic  (apache#18490)

    * Update to TRT 7 API

    Signed-off-by: Serge Panev <[email protected]>

    * Add PrePartition param caching - move init_tensorrt_params logic

    Signed-off-by: Serge Panev <[email protected]>

    * Handle node with no defined input

    Signed-off-by: Serge Panev <[email protected]>

    * Remove tmp comment

    Signed-off-by: Serge Panev <[email protected]>

commit 59e200a
Author: Haibin Lin <[email protected]>
Date:   Tue Aug 4 17:01:23 2020 -0700

    fix nn.dense doc (apache#18830)

    Co-authored-by: Lin <[email protected]>

commit 2e97226
Author: Leonard Lausen <[email protected]>
Date:   Tue Aug 4 21:11:32 2020 +0000

    Fix edge case when casting gluon Block before export (apache#18853)

    * Fix edge case when casting gluon Block before export

    Fixes apache#18843

    * Fix gpu test

commit b8eccc8
Author: Yang Shi <[email protected]>
Date:   Tue Aug 4 14:08:09 2020 -0700

    fix set default website version rewrite rule for cdn (apache#18856)

commit 7a40219
Author: Serge Panev <[email protected]>
Date:   Tue Aug 4 10:34:21 2020 -0700

    Remove check for subgraph with cycles (apache#18555)

    * Remove check for subgraph with cycles

    Signed-off-by: Serge Panev <[email protected]>

    * Add comments

    Signed-off-by: Serge Panev <[email protected]>

commit 95fa63f
Author: Serge Panev <[email protected]>
Date:   Mon Aug 3 17:15:02 2020 -0700

    Update the onnx-tensorrt submodule - CI to TRT7 (apache#18574)

commit 7f2e314
Author: Haibin Lin <[email protected]>
Date:   Mon Aug 3 16:09:48 2020 -0700

    update setup.py (apache#18850)

    * update setup.py

    * update python version

    Co-authored-by: Lin <[email protected]>

commit f872b43
Author: Leonard Lausen <[email protected]>
Date:   Mon Aug 3 20:11:06 2020 +0000

    Protobuf_USE_STATIC_LIBS must be set on Apple too (apache#18851)

    Fixes apache#18840

commit 4bb8224
Author: Yang Shi <[email protected]>
Date:   Mon Aug 3 12:30:13 2020 -0700

    Fixed python website double scroller and improve UX (apache#18845)

    * make python site header scroll aware and avoid double scroller

    * add compiled assets

    * adjust python site second header height

    * add new line

    * set focus to main content on DOM load

commit 7a5a488
Author: Iblis Lin <[email protected]>
Date:   Tue Aug 4 03:28:08 2020 +0800

    Fix broken link in docs/README.md (apache#18847)

commit 534cdbc
Author: Sheng Zha <[email protected]>
Date:   Mon Aug 3 11:58:33 2020 -0700

    Create greetings.yml (apache#18842)

commit 9fd2cce
Author: kpuatamazon <[email protected]>
Date:   Mon Aug 3 17:40:44 2020 +0100

    Update tests/README.md Docker instructions to match ci/README.md (apache#18848)

    Documentation was missing python3-docker and had an outdated platform.

commit 54b9e9c
Author: Sheng Zha <[email protected]>
Date:   Mon Aug 3 08:59:33 2020 -0700

    remove unnecessary usage of pretrained models, and prefer smaller size (apache#18844)

commit 51340d8
Author: Haibin Lin <[email protected]>
Date:   Sat Aug 1 16:23:03 2020 -0700

    Add compiled_with_cxx11_abi API  (apache#18836)

    * draft

    * add impl

    * add test

    * set default val

    Co-authored-by: Ubuntu <[email protected]>

commit 5a22193
Author: Sheng Zha <[email protected]>
Date:   Fri Jul 31 17:06:17 2020 -0700

    [NumPy] allow mixed array types (apache#18562)

    * allow mixed types in array func protocol

    * fix apache#18746

    * add support for memory share check

commit 08a5ee3
Author: Tao Lv <[email protected]>
Date:   Sat Aug 1 03:38:20 2020 +0800

    fix gelu to use erf based algorithm (apache#18827)

commit ac36089
Author: Leonard Lausen <[email protected]>
Date:   Fri Jul 31 04:54:10 2020 +0000

    Fixup move gluon.metric api docs (apache#18748)

    * Fix metric API page

    * Update index.rst

commit 7a24006
Author: Leonard Lausen <[email protected]>
Date:   Fri Jul 31 02:58:55 2020 +0000

    Enable DIST_KVSTORE by default in staticbuild (apache#18796)

    * Enable DIST_KVSTORE by default in staticbuild

    set(USE_DIST_KVSTORE ON CACHE BOOL "Build with DIST_KVSTORE support")

    * Ensure static linkage of dependencies

    * Fix for OS X

    * Fix shell syntax

    * Alternate approach to force static linkage of libprotobuf

commit aa53291
Author: Yang Shi <[email protected]>
Date:   Thu Jul 30 19:53:27 2020 -0700

    add adaptive left margin for python site document body (apache#18828)

commit 045efb2
Author: Sheng Zha <[email protected]>
Date:   Thu Jul 30 19:19:33 2020 -0700

    [NumPy] DLPack refactor and npx.from_numpy (apache#18656)

    * refactor dlpack and add from_numpy to npx

    * remove reference of DeepNumPy

    * map platform-dependent types to fixed-size types

    * update DMLC_LOG_FATAL_THROW

    * fix flaky

    * fix flaky

    * test no error

commit 608afef
Author: Xi Wang <[email protected]>
Date:   Fri Jul 31 02:30:25 2020 +0800

    Fix dirichlet flaky tests (apache#18817)

    * make parameter smoother

    * minor changes

commit 6bbd531
Author: Leonard Lausen <[email protected]>
Date:   Wed Jul 29 20:31:19 2020 +0000

    Update clang-tidy integration (apache#18815)

    Run clang-tidy via cmake only on the code managed by mxnet (and not 3rdparty dependencies), update to clang-tidy-10 and run clang-tidy-10 -fix to fix all the warnings that are enforced on CI.

    Developers can run clang-tidy by specifying the -DCMAKE_CXX_CLANG_TIDY="clang-tidy-10" to cmake, or using the python ci/build.py -R --platform ubuntu_cpu /work/runtime_functions.sh build_ubuntu_cpu_clang_tidy script.

commit b685fad
Author: Yang Shi <[email protected]>
Date:   Wed Jul 29 12:22:12 2020 -0700

    use regex that is supported by all browsers (apache#18811)

commit 9308aca
Author: Yang Shi <[email protected]>
Date:   Wed Jul 29 12:21:42 2020 -0700

    remove other language bindings section from website api page (apache#18783)

    * remove other language bindings section from api page

    * remove language binding docs redirect

    * add call for contribution banner

    * modify call for contribution wording

    Co-authored-by: Aaron Markham <[email protected]>

    * more wording modification

    Co-authored-by: Aaron Markham <[email protected]>

    * add hyperlink to 1.x version in banner

    * add reference to the C api deprecation github issue

    Co-authored-by: Aaron Markham <[email protected]>

commit 915f6b4
Author: Yang Shi <[email protected]>
Date:   Wed Jul 29 11:28:37 2020 -0700

    Remove deepnumpy reference and move Numpy tutorials to top level (apache#18798)

    * move np tutorials to top level

    * replace deepnumpy reference to np

    * add info in card

    * remove useless entry

    * replace NDArray API card with np.ndarray

    * python site refactor

    * remove duplicated drawer and refactor layout

    * extend document width to 100% for xl devices

commit e9829e7
Author: Joe Evans <[email protected]>
Date:   Tue Jul 28 18:53:29 2020 -0700

    Cherry-pick large tensor support from apache#18752. (apache#18804)

    Co-authored-by: Joe Evans <[email protected]>

commit 126636c
Author: Leonard Lausen <[email protected]>
Date:   Tue Jul 28 22:11:20 2020 +0000

    Fix naming in runtime_functions.sh (apache#18795)

commit f83dbac
Author: Haibin Lin <[email protected]>
Date:   Tue Jul 28 11:48:05 2020 -0700

    remove executor manager from API doc (apache#18802)

    Co-authored-by: Lin <[email protected]>

commit 7908d7e
Author: Yiyan66 <[email protected]>
Date:   Tue Jul 28 15:11:19 2020 +0800

    [numpy] fix flaky mixed precision binary error (apache#18660)

    * temp

    * change test

    * fix bad func call

    * test

    * rectify

    * doc

    * change test

commit a807f6d
Author: Sheng Zha <[email protected]>
Date:   Mon Jul 27 22:06:50 2020 -0700

    [NumPy] loss for np array (apache#17196)

    * loss for np/nd array

    * fix flaky

commit 74430a9
Author: phile <[email protected]>
Date:   Tue Jul 28 06:44:54 2020 +0800

    remove NLL in metric (apache#18794)

commit 9e77e81
Author: Przemyslaw Tredak <[email protected]>
Date:   Mon Jul 27 14:27:52 2020 -0700

    Update CUB and include it only for CUDA < 11 (apache#18799)

commit 98b3f73
Author: Sheng Zha <[email protected]>
Date:   Sat Jul 25 16:19:36 2020 -0700

    add support for np.ndarray in autograd.function (apache#18790)

commit c1db2d5
Author: Leonard Lausen <[email protected]>
Date:   Sat Jul 25 16:58:45 2020 +0000

    Remove caffe plugin (apache#18787)

    * Remove caffe plugin

    * Fix

    * Remove CXX14 feature flag

    * Update test

commit 2fbd182
Author: Leonard Lausen <[email protected]>
Date:   Sat Jul 25 02:48:30 2020 +0000

    Split up CI sanity test functions to enable fine-grained trigger (apache#18786)

    Developers can now trigger fine grained checks:

    python ci/build.py -R --platform ubuntu_cpu /work/runtime_functions.sh sanity_python
    python ci/build.py -R --platform ubuntu_cpu /work/runtime_functions.sh sanity_license
    etc

commit 06b5d22
Author: Serge Panev <[email protected]>
Date:   Fri Jul 24 14:22:42 2020 -0700

    ONNX import: use Conv pad attribute for symmetrical padding (apache#18675)

    Signed-off-by: Serge Panev <[email protected]>

commit e31ad77
Author: Yang Shi <[email protected]>
Date:   Thu Jul 23 11:33:31 2020 -0700

    set website default version to current stable (1.6) version (apache#18738)

    * set website default version - test redirect

    * enable first time redirect on all master website pages

    * update test code

    * remove unnecessary test code

    * fix typo

    * delete test code

commit 02ae456
Author: Dick Carter <[email protected]>
Date:   Thu Jul 23 11:17:10 2020 -0700

    Improve environment variable handling in unittests (apache#18424)

    This PR makes it easy to create unittests that require specific settings of environment variables, while avoiding the pitfalls (discussed in comments section). This PR can be considered a recasting and expansion of the great vision of @larroy in creating the EnvManager class in apache#13140.

    In its base form, the facility is a drop-in replacement for EnvManager, and is called 'environment':

    with environment('MXNET_MY_NEW_FEATURE', '1'):
        <test with feature enabled>
    with environment('MXNET_MY_NEW_FEATURE', '0'):
        <test with feature disabled>

    Like EnvManager, this facility takes care of the save/restore of the previous environment variable state, including when exceptions are raised. In addition though, this PR introduces the features:

        A similarly-named unittest decorator: @with_environment(key, value)
        The ability to pass in multiple env vars as a dict (as is needed for some tests) in both forms, so for example:

    with environment({'MXNET_FEATURE_A': '1',
                      'MXNET_FEATURE_B': '1'}):
        <test with both features enabled>

        Works on Windows! This PR includes a wrapping of the backend's setenv() and getenv() functions, and uses this direct access to the backend environment to keep it in sync with the python environment. This works around the problem that the C Runtime on Windows gets a snapshot of the Python environment at startup that is immutable from Python.
        with environment() has a simple implementation using the @contextmanager decorator
        Tests are included that validate the facility works with all combinations of before_val/set_val, namely unset/unset, unset/set, set/unset, set/set.

    There were 5 unittests previously using EnvManager, and this PR shifts those uses to with environment():, while converting over 20 other ad-hoc uses of os.environ[] within the unittests. This PR also enables those unittests that were bypassed on Windows (due to the inability to set environment variables) to run on all platforms.

    Further Comments

    Environment variables are a two-edged sword- they enable useful operating modes for testing, debugging or niche applications, but like all features they must be tested. The correct approach for testing with a particular env var setting is:

    def set_env_var(key, value):
        if value is None:
            os.environ.pop(key, None)
        else:
            os.environ[key] = value

    old_env_var_value = os.environ.get(env_var_name)
    try:
        set_env_var(env_var_name, test_env_var_value)
        <perform test>
    finally:
        set_env_var(env_var_name, old_env_var_value )

    The above code makes no assumption about whether the before-test and within-test state of the env var is set or unset, and restores the prior environment even if the test raises an exception. This represents a lot of boiler-plate code that could be potentially mishandled. The with environment() context makes it simple to handle all this properly. If an entire unittest wants a forced env var setting, then using the @with_environment() decorator avoids the code indent of the with environment() approach if used otherwise within the test.

commit 18af71e
Author: Leonard Lausen <[email protected]>
Date:   Thu Jul 23 18:09:10 2020 +0000

    CI: Migrate remaining Dockerfiles to docker-compose.yml and remove unused code (apache#18771)

    * Migrate remaining Dockerfiles to docker-compose.yml

    - Delete unused Dockerfiles
    - Delete unused install/*.sh scripts
    - Consolidate ubuntu_gpu_tensorrt and ubuntu_gpu
    - Remove deprecated logic in ci/build.py (no longer needed with
      docker-compose)
    - Remove ci/docker_cache.py (no longer needed with docker-compose)

    * Fix

    * Fix

    * Fix ubuntu_cpu_jekyll

commit 1928117
Author: Przemyslaw Tredak <[email protected]>
Date:   Tue Jul 21 23:35:15 2020 -0700

    Fix crash when accessing already destructed static variables (apache#18768)

commit a330a02
Author: Leonard Lausen <[email protected]>
Date:   Wed Jul 22 06:31:47 2020 +0000

    Fix mx.symbol.numpy._Symbol.__deepcopy__ logic error (apache#18686)

    * Fix mx.symbol.numpy._Symbol.__deepcopy__ logic error

    Performed shallow copy instead of deep copy

    * Test

    * Fix test

commit 9548b0c
Author: Leonard Lausen <[email protected]>
Date:   Tue Jul 21 21:42:01 2020 +0000

    Remove duplicate settings in .codecov.yml (apache#18763)

    New PRs started showing the codecov/project badge again due apparent change in codecov's backend resolving these duplicate options specified in .codecov.yml
  • Loading branch information
zheyuye committed Aug 5, 2020
1 parent 08a3a5d commit 0ce5aa9
Show file tree
Hide file tree
Showing 286 changed files with 2,564 additions and 5,777 deletions.
52 changes: 20 additions & 32 deletions .clang-tidy
Original file line number Diff line number Diff line change
Expand Up @@ -17,45 +17,33 @@

# The checks defined here will be run and will display by default as warnings.
Checks: >
-*, cppcoreguidelines-c-copy-assignment-signature,
cppcoreguidelines-interfaces-global-init, cppcoreguidelines-no-malloc,
cppcoreguidelines-pro-bounds-constant-array-index, cppcoreguidelines-pro-type-const-cast,
cppcoreguidelines-pro-type-cstyle-cast, cppcoreguidelines-pro-type-member-init,
cppcoreguidelines-pro-type-static-cast-downcast, cppcoreguidelines-pro-type-union-access,
cppcoreguidelines-pro-type-vararg, cppcoreguidelines-slicing,
cppcoreguidelines-special-member-functions, clang-analyzer-security.FloatLoopCounter,
clang-analyzer-security.insecureAPI.*, clang-analyzer-core.CallAndMessage,
clang-analyzer-core.DivideZero, clang-analyzer-core.DynamicTypePropagation,
clang-analyzer-core.NonNullParamChecker, clang-analyzer-core.NullDereference,
clang-analyzer-core.StackAddressEscape, clang-analyzer-core.UndefinedBinaryOperatorResult,
clang-analyzer-core.VLASize, clang-analyzer-core.builtin.BuiltinFunctions,
clang-analyzer-core.builtin.NoReturnFunctions, clang-analyzer-core.uninitialized.ArraySubscript,
clang-analyzer-core.uninitialized.Assign, clang-analyzer-core.uninitialized.Branch,
clang-analyzer-core.uninitialized.CapturedBlockVariable,
clang-analyzer-core.uninitialized.UndefReturn, clang-analyzer-cplusplus.NewDelete,
clang-analyzer-cplusplus.NewDeleteLeaks, clang-analyzer-cplusplus.SelfAssignment,
clang-analyzer-deadcode.DeadStores, modernize-avoid-bind, modernize-deprecated-headers,
modernize-loop-convert, modernize-make-shared, modernize-pass-by-value,
-*, cppcoreguidelines-* clang-analyzer-*, modernize-*,
performance-faster-string-find, performance-for-range-copy,
performance-implicit-conversion-in-loop, performance-inefficient-algorithm,
performance-inefficient-string-concatenation, performance-trivially-destructible,
performance-inefficient-vector-operation, performance-move-const-arg,
performance-move-constructor-init, performance-noexcept-move-constructor,
performance-no-automatic-move, performance-unnecessary-copy-initialization,
performance-type-promotion-in-math-fn
# performance checks not enabled due to segmentation fault in clang-tidy v8+:
# performance-unnecessary-value-param

# In order to trigger an error, you must have a rule defined both in checks and in this section.
WarningsAsErrors: >
cppcoreguidelines-no-malloc, modernize-deprecated-headers,
modernize-loop-convert, modernize-make-shared, modernize-pass-by-value, modernize-make-unique,
modernize-raw-string-literal, modernize-redundant-void-arg, modernize-replace-auto-ptr,
modernize-replace-random-shuffle, modernize-return-braced-init-list, modernize-shrink-to-fit,
modernize-unary-static-assert, modernize-use-bool-literals, modernize-use-default-member-init,
modernize-use-emplace, modernize-use-equals-default, modernize-use-equals-delete,
modernize-use-noexcept, modernize-use-nullptr, modernize-use-override,
modernize-use-transparent-functors, modernize-use-using, performance-*
modernize-use-transparent-functors, modernize-use-using,
performance-unnecessary-copy-initialization, performance-move-const-arg
# cppcoreguidelines checks not enabled:
# cppcoreguidelines-pro-bounds-pointer-arithmetic
# cppcoreguidelines-pro-bounds-array-to-pointer-decay
# cppcoreguidelines-pro-type-reinterpret-cast

# modernize checks not enabled:
# modernize checks not enforced:
# modernize-use-auto
# modernize-make-unique (C++14 and newer only)

# In order to trigger an error, you must have a rule defined both in checks and in this section.
WarningsAsErrors: >
cppcoreguidelines-no-malloc, modernize-use-nullptr, performance-unnecessary-copy-initialization,
modernize-use-emplace, performance-move-const-arg
# modernize-avoid-bind

# Todo: define a better regex match that includes most project headers, but excludes third party
# code.
Expand Down
5 changes: 0 additions & 5 deletions .codecov.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,6 @@ coverage:
round: down
range: "70...100"

status:
project: yes
patch: yes
changes: no

parsers:
gcov:
branch_detection:
Expand Down
20 changes: 20 additions & 0 deletions .github/workflows/greetings.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
name: Greetings

on: [pull_request, issues]

jobs:
greeting:
runs-on: ubuntu-latest
steps:
- uses: actions/first-interaction@v1
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
issue-message: |
Welcome to Apache MXNet (incubating)! We are on a mission to democratize AI, and we are glad that you are contributing to it by opening this issue.
Please make sure to include all the relevant context, and one of the @apache/mxnet-committers will be here shortly.
If you are interested in contributing to our project, let us know! Also, be sure to check out our guide on [contributing to MXNet](https://mxnet.apache.org/community/contribute) and our [development guides wiki](https://cwiki.apache.org/confluence/display/MXNET/Developments).
pr-message: |
Welcome to Apache MXNet (incubating)! We are on a mission to democratize AI, and we are glad that you are contributing to it by opening this pull request.
Please make sure that the changes are covered by tests. One of the @apache/mxnet-committers will be here shortly.
If you run into any issue with the CI and tests, we recommend that you first check out our guide on [developer guides wiki](https://cwiki.apache.org/confluence/display/MXNET/Developments).
Let our @apache/mxnet-committers know if you need any help!
2 changes: 1 addition & 1 deletion 3rdparty/nvidia_cub
Submodule nvidia_cub updated 105 files
2 changes: 1 addition & 1 deletion 3rdparty/onnx-tensorrt
Submodule onnx-tensorrt updated 63 files
+1 −0 .gitmodules
+46 −177 CMakeLists.txt
+0 −91 Dockerfile
+0 −101 FancyActivation.cu
+0 −142 FancyActivation.hpp
+187 −70 ImporterContext.hpp
+0 −185 InstanceNormalization.cpp
+0 −133 InstanceNormalization.hpp
+607 −517 ModelImporter.cpp
+60 −46 ModelImporter.hpp
+8 −6 NvOnnxParser.cpp
+46 −21 NvOnnxParser.h
+0 −29 NvOnnxParserRuntime.cpp
+0 −85 NvOnnxParserRuntime.h
+0 −30 NvOnnxParserTypedefs.h
+306 −56 OnnxAttrs.cpp
+37 −21 OnnxAttrs.hpp
+0 −57 PluginFactory.cpp
+0 −59 PluginFactory.hpp
+56 −35 README.md
+0 −120 ResizeNearest.cu
+0 −108 ResizeNearest.hpp
+361 −0 ShapeTensor.cpp
+155 −0 ShapeTensor.hpp
+85 −99 ShapedWeights.cpp
+17 −19 ShapedWeights.hpp
+0 −133 Split.cu
+0 −112 Split.hpp
+175 −78 Status.hpp
+78 −40 TensorOrWeights.hpp
+3,646 −1,808 builtin_op_importers.cpp
+2 −1 builtin_op_importers.hpp
+0 −38 builtin_plugins.cpp
+0 −32 builtin_plugins.hpp
+25 −1 common.hpp
+1 −7 contributing.md
+70 −0 docker/onnx-tensorrt-deb.Dockerfile
+80 −0 docker/onnx-tensorrt-tar.Dockerfile
+6 −1 getSupportedAPITest.cpp
+0 −9 libnvonnxparser_runtime.version
+66 −3 main.cpp
+0 −60 nv_onnx_runtime_bindings.i
+32 −17 onnx2trt.hpp
+43 −30 onnx2trt_common.hpp
+3 −3 onnx2trt_runtime.hpp
+1,684 −54 onnx2trt_utils.cpp
+236 −375 onnx2trt_utils.hpp
+155 −150 onnx_backend_test.py
+27 −49 onnx_tensorrt/backend.py
+30 −0 onnx_tensorrt/config.py
+64 −78 onnx_tensorrt/tensorrt_engine.py
+53 −10 onnx_trt_backend.cpp
+130 −44 onnx_utils.hpp
+162 −138 operators.md
+0 −175 plugin.cpp
+0 −183 plugin.hpp
+0 −27 plugin_common.hpp
+0 −125 serialize.hpp
+30 −14 setup.py
+1 −1 third_party/onnx
+73 −56 toposort.hpp
+149 −198 trt_utils.hpp
+1 −1 utils.hpp
90 changes: 34 additions & 56 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,6 @@ option(USE_JEMALLOC "Build with Jemalloc support" OFF)
option(USE_LIBJPEG_TURBO "Use libjpeg-turbo" OFF)
option(USE_DIST_KVSTORE "Build with DIST_KVSTORE support" OFF)
option(USE_PLUGINS_WARPCTC "Use WARPCTC Plugins" OFF)
option(USE_PLUGIN_CAFFE "Use Caffe Plugin" OFF)
option(USE_CPP_PACKAGE "Build C++ Package" OFF)
option(USE_MXNET_LIB_NAMING "Use MXNet library naming conventions." ON)
option(USE_GPROF "Compile with gprof (profiling) flag" OFF)
Expand All @@ -90,6 +89,7 @@ cmake_dependent_option(ENABLE_TESTCOVERAGE "Enable compilation with test coverag
option(USE_INT64_TENSOR_SIZE "Use int64_t to represent the total number of elements in a tensor" OFF)
option(BUILD_CYTHON_MODULES "Build cython modules." OFF)
option(LOG_FATAL_THROW "Log exceptions but do not abort" ON)
option(USE_CXX11_ABI "Build with GLIBCXX_USE_CXX11_ABI" ON)
cmake_dependent_option(USE_SPLIT_ARCH_DLL "Build a separate DLL for each Cuda arch (Windows only)." ON "MSVC" OFF)
cmake_dependent_option(USE_CCACHE "Attempt using CCache to wrap the compilation" ON "UNIX" OFF)

Expand All @@ -100,6 +100,12 @@ message(STATUS "CMAKE_SYSTEM_PROCESSOR ${CMAKE_SYSTEM_PROCESSOR}")

message(STATUS "CMAKE_SYSTEM_NAME ${CMAKE_SYSTEM_NAME}")

if(USE_CXX11_ABI)
add_definitions(-D_GLIBCXX_USE_CXX11_ABI=1)
else()
add_definitions(-D_GLIBCXX_USE_CXX11_ABI=0)
endif()

if(USE_TVM_OP)
add_definitions(-DMXNET_USE_TVM_OP=1)
endif()
Expand Down Expand Up @@ -142,12 +148,15 @@ else()
execute_process(COMMAND uname -m COMMAND tr -d '\n' OUTPUT_VARIABLE SYSTEM_ARCHITECTURE)
endif()

if(CMAKE_BUILD_TYPE STREQUAL "Distribution" AND UNIX AND NOT APPLE)
set(CMAKE_BUILD_WITH_INSTALL_RPATH ON)
set(CMAKE_INSTALL_RPATH $\{ORIGIN\})
# Enforce DT_PATH instead of DT_RUNPATH
set(CMAKE_SHARED_LINKER_FLAGS "-Wl,--disable-new-dtags")
set(CMAKE_EXE_LINKER_FLAGS "-Wl,--disable-new-dtags")
if(CMAKE_BUILD_TYPE STREQUAL "Distribution")
if(UNIX AND NOT APPLE)
set(CMAKE_BUILD_WITH_INSTALL_RPATH ON)
set(CMAKE_INSTALL_RPATH $\{ORIGIN\})
# Enforce DT_PATH instead of DT_RUNPATH
set(CMAKE_SHARED_LINKER_FLAGS "-Wl,--disable-new-dtags")
set(CMAKE_EXE_LINKER_FLAGS "-Wl,--disable-new-dtags")
endif()
set(Protobuf_USE_STATIC_LIBS ON)
endif()

set(CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/cmake/upstream;${CMAKE_CURRENT_SOURCE_DIR}/cmake/Modules;${CMAKE_MODULE_PATH}")
Expand Down Expand Up @@ -229,6 +238,7 @@ if(USE_TENSORRT)
include_directories(3rdparty/onnx-tensorrt/third_party/onnx/)
add_definitions(-DMXNET_USE_TENSORRT=1)
add_definitions(-DONNX_NAMESPACE=onnx)
add_definitions(-DONNX_ML=1)

find_package(Protobuf REQUIRED)

Expand All @@ -238,14 +248,11 @@ if(USE_TENSORRT)
find_library(ONNX_PROTO_LIBRARY NAMES libonnx_proto.so REQUIRED
PATHS ${ONNX_PATH}
DOC "Path to onnx_proto library.")
find_library(ONNX_TRT_RUNTIME_LIBRARY NAMES libnvonnxparser_runtime.so REQUIRED
PATHS ${ONNX_TRT_PATH}
DOC "Path to onnx_proto library.")
find_library(ONNX_TRT_PARSER_LIBRARY NAMES libnvonnxparser.so REQUIRED
PATHS ${ONNX_TRT_PATH}
DOC "Path to onnx_proto library.")
DOC "Path to onnx_proto parser library.")

list(APPEND mxnet_LINKER_LIBS libnvinfer.so ${ONNX_TRT_PARSER_LIBRARY} ${ONNX_TRT_RUNTIME_LIBRARY}
list(APPEND mxnet_LINKER_LIBS libnvinfer.so ${ONNX_TRT_PARSER_LIBRARY}
${ONNX_PROTO_LIBRARY} ${ONNX_LIBRARY} ${PROTOBUF_LIBRARY})
endif()

Expand Down Expand Up @@ -276,6 +283,7 @@ if(USE_MKLDNN)
include_directories(${PROJECT_BINARY_DIR}/3rdparty/mkldnn/include)
add_definitions(-DMXNET_USE_MKLDNN=1)
list(APPEND mxnet_LINKER_LIBS dnnl)
set_target_properties(dnnl PROPERTIES CXX_CLANG_TIDY "") # don't lint 3rdparty dependency
endif()

# Allow Cuda compiles outside of src tree to find things in 'src' and 'include'
Expand All @@ -299,7 +307,6 @@ foreach(var ${C_CXX_INCLUDE_DIRECTORIES})
endforeach()

include_directories("include")
include_directories("3rdparty/nvidia_cub")
include_directories("3rdparty/tvm/nnvm/include")
include_directories("3rdparty/tvm/include")
include_directories("3rdparty/dmlc-core/include")
Expand Down Expand Up @@ -407,6 +414,7 @@ if(USE_OPENMP)
AND NOT CMAKE_CROSSCOMPILING)
load_omp()
list(APPEND mxnet_LINKER_LIBS omp)
set_target_properties(omp PROPERTIES CXX_CLANG_TIDY "") # don't lint 3rdparty dependency
if(UNIX)
list(APPEND mxnet_LINKER_LIBS pthread)
endif()
Expand Down Expand Up @@ -464,6 +472,8 @@ set(GTEST_MAIN_LIBRARY gtest_main)
set(GTEST_LIBRARY gtest)

add_subdirectory(${GTEST_ROOT})
set_target_properties(gtest PROPERTIES CXX_CLANG_TIDY "") # don't lint 3rdparty dependency
set_target_properties(gtest_main PROPERTIES CXX_CLANG_TIDY "") # don't lint 3rdparty dependency
find_package(GTest REQUIRED)

# cudnn detection
Expand All @@ -480,6 +490,7 @@ endif()

if(EXISTS ${CMAKE_CURRENT_SOURCE_DIR}/3rdparty/dmlc-core/cmake)
add_subdirectory("3rdparty/dmlc-core")
set_target_properties(dmlc PROPERTIES CXX_CLANG_TIDY "") # don't lint 3rdparty dependency
endif()

FILE(GLOB_RECURSE SOURCE "src/*.cc" "src/*.h" "include/*.h")
Expand All @@ -494,7 +505,9 @@ FILE(GLOB_RECURSE NNVMSOURCE
3rdparty/tvm/nnvm/src/core/*.h
3rdparty/tvm/nnvm/src/pass/*.h
3rdparty/tvm/nnvm/include/*.h)
list(APPEND SOURCE ${NNVMSOURCE})
add_library(nnvm OBJECT ${NNVMSOURCE})
set_target_properties(nnvm PROPERTIES CXX_CLANG_TIDY "") # don't lint 3rdparty dependency
list(APPEND SOURCE $<TARGET_OBJECTS:nnvm>)

# add source group
FILE(GLOB_RECURSE GROUP_SOURCE "src/*.cc" "3rdparty/tvm/nnvm/*.cc" "plugin/*.cc")
Expand All @@ -521,39 +534,6 @@ if(USE_OPERATOR_TUNING AND USE_OPENMP)
add_definitions(-DMXNET_USE_OPERATOR_TUNING=1)
endif()

if(USE_PLUGIN_CAFFE)
if(NOT USE_CUDA)
set(CPU_ONLY ON)
add_definitions(-DCPU_ONLY=1)
endif()
if(NOT DEFINED CAFFE_PATH)
if(EXISTS ${CMAKE_CURRENT_SOURCE_DIR}/caffe)
set(CAFFE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/caffe)
else()
set(CAFFE_PATH $ENV{CAFFE_PATH})
endif()
endif()
list(APPEND CMAKE_MODULE_PATH ${CAFFE_PATH}/cmake)
include_directories(${CAFFE_PATH}/include)
include_directories(${CAFFE_PATH}/build/src)
include_directories(${CMAKE_BINARY_DIR}/caffe/include)
link_directories(${CAFFE_PATH}/build/lib)
if(NOT DEFINED CAFFE_PATH)
message(FATAL_ERROR "Please set CAFFE_PATH to point to the caffe source installation")
endif()
FILE(GLOB_RECURSE PLUGINS_SOURCE "plugin/caffe/*.cc" "plugin/caffe/*.h")
FILE(GLOB_RECURSE PLUGINS_CUSRC "plugin/caffe/*.cu")
list(APPEND SOURCE ${PLUGINS_SOURCE})
list(APPEND CUDA ${PLUGINS_CUSRC})
include_directories(${CMAKE_BINARY_DIR}/include)
add_definitions(-DMXNET_USE_CAFFE=1)
list(APPEND mxnet_LINKER_LIBS
protobuf boost_system boost_thread boost_filesystem
gflags glog caffe
${Caffe_LINKER_LIBS}
)
endif()

if (NOT (EXTRA_OPERATORS STREQUAL ""))
mxnet_source_group("Extra" GLOB_RECURSE "${EXTRA_OPERATORS}/*.cc")
mxnet_source_group("Extra\\Cuda" GLOB_RECURSE "${EXTRA_OPERATORS}/*.cu")
Expand Down Expand Up @@ -640,14 +620,10 @@ if(USE_CUDA)
link_directories(${CUDAToolkit_LIBRARY_DIR})
endif()

# unsupported: if caffe is a subdirectory of mxnet, load its CMakeLists.txt as well
if(USE_PLUGIN_CAFFE)
if(EXISTS ${CMAKE_CURRENT_SOURCE_DIR}/caffe)
add_subdirectory(caffe)
endif()
if(CUDAToolkit_VERSION_MAJOR LESS "11")
include_directories("3rdparty/nvidia_cub")
endif()


if(MSVC)
set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} /EHsc")
set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} /EHsc /Gy")
Expand All @@ -674,7 +650,6 @@ if(UNIX)
endif()
target_link_libraries(mxnet PUBLIC mshadow)
target_link_libraries(mxnet PUBLIC ${CMAKE_DL_LIBS})
target_compile_definitions(mxnet PUBLIC DMLC_LOG_FATAL_THROW=$<BOOL:${LOG_FATAL_THROW}>)
if(CMAKE_BUILD_TYPE STREQUAL "RelWithDebInfo")
target_compile_options(mxnet PRIVATE "$<$<COMPILE_LANGUAGE:CXX>:-Werror>")
# Ignore erroneous compiler warnings:
Expand All @@ -701,7 +676,6 @@ elseif(MSVC)
foreach(arch ${arch_code_list})
add_library(mxnet_${arch} SHARED ${SOURCE})
target_link_libraries(mxnet_${arch} PUBLIC mshadow)
target_compile_definitions(mxnet_${arch} PUBLIC DMLC_LOG_FATAL_THROW=$<BOOL:${LOG_FATAL_THROW}>)
target_compile_options(
mxnet_${arch}
PRIVATE
Expand Down Expand Up @@ -737,10 +711,10 @@ elseif(MSVC)
endif(USE_SPLIT_ARCH_DLL)
else()
add_library(mxnet SHARED ${SOURCE})
target_compile_definitions(mxnet PUBLIC DMLC_LOG_FATAL_THROW=$<BOOL:${LOG_FATAL_THROW}>)
target_link_libraries(mxnet PUBLIC mshadow)
endif()
endif()
target_compile_definitions(mxnet PUBLIC DMLC_LOG_FATAL_THROW=$<BOOL:${LOG_FATAL_THROW}>)

# extension libraries (custom operators, custom subgraphs) are built by default
add_library(customop_lib SHARED ${CMAKE_CURRENT_SOURCE_DIR}/example/extensions/lib_custom_op/gemm_lib.cc)
Expand Down Expand Up @@ -782,6 +756,7 @@ if(USE_DIST_KVSTORE)
add_subdirectory("3rdparty/ps-lite")
add_definitions(-DMXNET_USE_DIST_KVSTORE)
list(APPEND mxnet_LINKER_LIBS pslite)
set_target_properties(pslite PROPERTIES CXX_CLANG_TIDY "") # don't lint 3rdparty dependency
endif()

if(USE_MKLDNN)
Expand All @@ -796,6 +771,9 @@ function(BuildTVMOP)
# scope the variables in BuildTVM.cmake to avoid conflict
include(cmake/BuildTVM.cmake)
add_subdirectory("3rdparty/tvm")
set_target_properties(tvm PROPERTIES CXX_CLANG_TIDY "") # don't lint 3rdparty dependency
set_target_properties(tvm_topi PROPERTIES CXX_CLANG_TIDY "") # don't lint 3rdparty dependency
set_target_properties(tvm_runtime PROPERTIES CXX_CLANG_TIDY "") # don't lint 3rdparty dependency
endfunction()

if(USE_TVM_OP)
Expand Down
1 change: 1 addition & 0 deletions CONTRIBUTORS.md
Original file line number Diff line number Diff line change
Expand Up @@ -254,6 +254,7 @@ List of Contributors
* [Connor Goggins](https://github.com/connorgoggins)
* [Wei Chu](https://github.com/waytrue17)
* [Yang Shi](https://github.com/ys2843)
* [Joe Evans](https://github.com/josephevans)

Label Bot
---------
Expand Down
1 change: 0 additions & 1 deletion ci/Jenkinsfile_docker_cache
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,6 @@ core_logic: {
ws('workspace/docker_cache') {
timeout(time: total_timeout, unit: 'MINUTES') {
utils.init_git()
sh "python3 ./ci/docker_cache.py --docker-registry ${env.DOCKER_CACHE_REGISTRY}"
sh "cd ci && python3 ./docker_login.py --secret-name ${env.DOCKERHUB_SECRET_NAME} && docker-compose -f docker/docker-compose.yml pull && docker-compose -f docker/docker-compose.yml build --parallel && COMPOSE_HTTP_TIMEOUT=600 docker-compose -f docker/docker-compose.yml push && docker logout"
}
}
Expand Down
Loading

0 comments on commit 0ce5aa9

Please sign in to comment.