[1.8.x] LICENSE needs update/review #19427

samskalicky · 2020-10-26T16:40:55Z

Description

Per the discussion on Apache general regarding the v1.8.0.rc1 vote, we need to review and update our license file.

We need to review each 3rdparty submodule's LICENSE and update the corresponding entries in the MXNet LICENSE file:

ctc_include
dlpack
dmlc-core
googletest
~~intgemm [DONE][@kpuatamazon link]~~
mkldnn [WIP] @anko-intel @bartekkuncer
~~mshadow [DONE][@szha link]~~
~~nvidia_cub [DONE][@ptrendx link]~~
onnx-tensorrt [Needs Update][@ptrendx link]
openmp
ps-lite
tvm

For each submodule, we need to verify that all entries in that LICENSE file is included in our MXNet license file.

leezu · 2020-10-26T16:58:26Z

An alternative is to stop redistributing the 3rdparty code an provide scripts for users to download the 3rdparty code during / before build configuration.

kpuatamazon · 2020-10-26T18:59:44Z

Regarding intgemm, I think #19406 covers me. MIT (most code) and Boost and explicitly mentioned.

samskalicky · 2020-10-26T19:55:21Z

Regarding intgemm, I think #19406 covers me. MIT (most code) and Boost and explicitly mentioned.

Thanks! Marked integemm as done.

szha · 2020-10-26T23:51:55Z

mshadow has already been handled with the merge. it's not whitelisted in rat-exclude and it's not actually 3rdparty anymore.

szha · 2020-10-27T15:53:30Z

@PatricZhao could someone on your team help verify mkldnn licenses?

samskalicky · 2020-10-27T16:18:53Z

mshadow has already been handled with the merge. it's not whitelisted in rat-exclude and it's not actually 3rdparty anymore.

Thanks @szha does this apply to ctc_include too?

ptrendx · 2020-10-27T16:38:09Z

There was no change in nvidia_cub licensing.

For onnx-tensorrt - there is a change to pybind licensing (3rdparty of onnx, which is 3rdparty of onnx-tensorrt), resolving #15547.

samskalicky · 2020-10-27T18:27:56Z

For onnx-tensorrt - there is a change to pybind licensing (3rdparty of onnx, which is 3rdparty of onnx-tensorrt), resolving #15547.

Does this change include the issues raised in the email thread? Is the pybind license up-to-date with these files:

./3rdparty/onnx-tensorrt/third_party/onnx/third_party/pybind11/tests/test_gil_scoped.cpp
./3rdparty/onnx-tensorrt/third_party/onnx/third_party/pybind11/tests/test_tagbased_polymorphic.cpp
./3rdparty/onnx-tensorrt/third_party/onnx/third_party/pybind11/tests/test_union.cpp

sandeep-krishnamurthy · 2020-10-27T20:31:09Z

@samskalicky - TVM is already covered -https://github.com/apache/incubator-mxnet/blob/v1.8.x/LICENSE#L235 what needs to be done here?

Should we transitively callout TVM's depedencies as well? https://github.com/apache/incubator-tvm/blob/9bd2c7b44208ed992061f8c2688e1137357f1db1/LICENSE#L209

Zha0q1 · 2020-10-27T20:42:00Z

it looks like ctc_include has not been changed since Oct 2018. it uses the exact same apache 2.0 license as ours except for this line at the end of their license file

  Copyright 2015-2016, Baidu USA LLC.

we have this

4. Warp-CTC - For details, see, 3rdparty/ctc_include/LICENSE
         Copyright 2015-2016, Baidu USA LLC.

Update: ctc_include in turn includes morderngpu and we have this in our license:

3. Moderngpu - For details, see, 3rdparty/ctc_include/contrib/moderngpu/LICENSE
         Copyright (c) 2013, NVIDIA CORPORATION.  All rights reserved.

samskalicky · 2020-10-27T20:50:23Z

@samskalicky - TVM is already covered -https://github.com/apache/incubator-mxnet/blob/v1.8.x/LICENSE#L235 what needs to be done here?

Should we transitively callout TVM's depedencies as well? https://github.com/apache/incubator-tvm/blob/9bd2c7b44208ed992061f8c2688e1137357f1db1/LICENSE#L209

We need to review that there have not been any changes to files in TVM and its 3rdparty (and those 3rdparty's 3rdparty....etc), and that if any have changed that their licenses that we update our license.

access2rohit · 2020-10-27T21:09:26Z

@samskalicky ctc_include hasen't changed since past 2years. Moderngpu has 3 clause BSD license and all the remaining files have Apache 2.0 so that can be marked as done.

sandeep-krishnamurthy · 2020-10-27T21:13:10Z

@szha @leezu => Is there a process to do this activity?

access2rohit · 2020-10-27T21:18:14Z

Quick question:

11. 3rdparty/ps-lite - For details, see, 3rdparty/ps-lite/LICENSE
          Copyright 2015 Carnegie Mellon University
          Copyright 2016, ps-lite developers
          Copyright (c) 2015-2016 by Contributors
          Copyright by Contributors
          Copyright (c) 2018-2019 Bytedance Inc.

Some files have Copyright (c) 2015 by Contributors
and some have Copyright (c) 2016 by Contributors
does this line cover both Copyright (c) 2015-2016 by Contributors ? or do we need individual lines. If not then all ps-lite's licenses are included in MXNet's license file

access2rohit · 2020-10-27T21:33:59Z

The following files in dlpack

./3rdparty/dlpack/contrib/mock_c.c
./3rdparty/dlpack/contrib/mock_main.cc

have

// Copyright by contributors

instead of

Copyright 2017 by Contributors

as written in MXNet License file

    5. 3rdparty/dlpack - For details, see, 3rdparty/dlpack/LICENSE
         Copyright 2017 by Contributors

Wee need to add another line

Copyright by contributors

to point 5 in our license file

waytrue17 · 2020-10-27T22:17:02Z

For dmlc-core, we have the following items that are not included in our license:
dmlc-core/include/dmlc/any.h: Copyright (c) 2016 by Contributors
dmlc-core/include/dmlc/build_config_default.h: Copyright (c) 2018 by Contributors
dmlc-core/include/dmlc/endian.h: Copyright (c) 2017 by Contributors
dmlc-core/include/dmlc/strtonum.h: Copyright (c) 2015-2018 by Contributors

leezu · 2020-10-27T23:11:16Z

@sandeep-krishnamurthy I think the main issue here is that our LICENSE file may not currently list all dependencies of dependencies (submodules of the submodules). We can refer to "Dependencies of dependencies" section of https://infra.apache.org/licensing-howto.html

Dependencies of dependencies (including so-called "transitive dependencies") are no different from first-order dependencies for the purposes of assembling LICENSE and NOTICE: LICENSE and NOTICE need only be modified to accommodate them only if their bits are bundled.

mseth10 · 2020-10-28T01:00:34Z

Found the following four LICENSE files for 3rdparty/googletest

In googletest/LICENSE
Copyright 2008, Google Inc.
(No corresponding entry in MXNet license file)
Googletest
In googletest/googletest/LICENSE:
Copyright 2008, Google Inc.
vs
In MXNet license file:
Copyright 2005-2015, Google Inc.
Googlemock
In googletest/googlemock/LICENSE:
Copyright 2008, Google Inc.
vs
In MXNet license file:
Copyright 2006-2015, Google Inc.
googlemock scripts/generator
In googletest/googlemock/scripts/generator/LICENSE:
Copyright [2007] Neal Norwitz
Portions Copyright [2007] Google Inc.
vs
In MXNet license file:
Copyright [2007-2009] Neal Norwitz
Portions Copyright [2007-2009] Google Inc.

MXNet license file gives a range of years that includes the 3rd party copyright year. Is this acceptable? Or should we update it to exactly match the 3rdparty files? @leezu @szha @samskalicky

bartekkuncer · 2020-10-28T13:54:39Z

@szha @PatricZhao

For mkldnn we have:
12. 3rdparty/mkldnn - For details, see, 3rdparty/mkldnn/LICENSE
Copyright 2016-2019 Intel Corporation
Copyright 2018 YANDEX LLC
which is consistent with their LICENSE file, but there is also:
"Copyright 2020 Arm Limited and affiliates." in blas.cmake and .drone.yml files;
"Copyright 2020 Intel Corporation" and "Copyright 2016/7/8/9-2020 Intel Corporation" in many mkldnn files

For its submodules:
2. gtest - For details, see, 3rdparty/mkldnn/tests/gtests/gtest/LICENSE
Copyright 2005-2008, Google Inc.
where there is also "Copyright 2015, Google Inc." listed in mkldnn/THIRD-PARTY-PROGRAMS

9. CMake FindCUDAToolkit.cmake, FindOpenCL.cmake - For details, see,
cmake/upstream/FindCUDAToolkit.cmake
cmake/upstream/select_compute_arch.cmake
3rdparty/mkldnn/cmake/FindOpenCL.cmake
Copyright 2000-2019 Kitware, Inc. and Contributors
where there is also "Copyright 2000-2020 Kitware, Inc. and Contributors"

19. Benchdnn - For details, see 3rdparty/mkldnn/tests/benchdnn/README.md
Copyright 2017-2018 Intel Corporation
where there is "Copyright 2017-2020 Intel Corporation" in benchdnn files

Not listed in our license but listed in mkldnn/THIRD-PARTY-PROGRAMS:
Boost C++ Libraries:
Copyright 2005-2014 Daniel James.
Should it be added or is this one unnecessary?

Zha0q1 · 2020-10-28T19:14:21Z

openmp uses a customized version of the Apache License v2.0

==============================================================================
The LLVM Project is under the Apache License v2.0 with LLVM Exceptions:
==============================================================================

Specifically

---- LLVM Exceptions to the Apache 2.0 License ----

As an exception, if, as a result of your compiling your source code, portions
of this Software are embedded into an Object form of such source code, you
may redistribute such embedded portions in such Object form without complying
with the conditions of Sections 4(a), 4(b) and 4(d) of the License.

In addition, if you combine or link compiled forms of this Software with
software that is licensed under the GPLv2 ("Combined Software") and if a
court of competent jurisdiction determines that the patent provision (Section
3), the indemnity provision (Section 9) or other Section of the License
conflicts with the conditions of the GPLv2, you may retroactively and
prospectively choose to deem waived or otherwise exclude such Section(s) of
the License, but only in their entirety and only with respect to the Combined
Software.

I am not sure if we should mention this in our License.
~
~
~
~
Besides, openmp also has the following Licenses that are NOT included in our license file:
1.

==============================================================================
Legacy LLVM License (https://llvm.org/docs/DeveloperPolicy.html#legacy):
==============================================================================

The software contained in this directory tree is dual licensed under both the
University of Illinois "BSD-Like" license and the MIT license.  As a user of
this code you may choose to use it under either license.  As a contributor,
you agree to allow your code to be used under both.  The full text of the
relevant licenses is included below.

In addition, a license agreement from the copyright/patent holders of the
software contained in this directory tree is included below.

University of Illinois/NCSA
Open Source License

Copyright (c) 1997-2019 Intel Corporation

All rights reserved.

Developed by:
    OpenMP Runtime Team
    Intel Corporation
    http://www.openmprtl.org

Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal with
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
of the Software, and to permit persons to whom the Software is furnished to do
so, subject to the following conditions:

    * Redistributions of source code must retain the above copyright notice,
      this list of conditions and the following disclaimers.

    * Redistributions in binary form must reproduce the above copyright notice,
      this list of conditions and the following disclaimers in the
      documentation and/or other materials provided with the distribution.

    * Neither the names of Intel Corporation OpenMP Runtime Team nor the
      names of its contributors may be used to endorse or promote products
      derived from this Software without specific prior written permission.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL THE
CONTRIBUTORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS WITH THE
SOFTWARE.

Copyright (c) 1997-2019 Intel Corporation

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.


Intel Corporation

Software Grant License Agreement ("Agreement")

Except for the license granted herein to you, Intel Corporation ("Intel") reserves
all right, title, and interest in and to the Software (defined below).

Definition

"Software" means the code and documentation as well as any original work of
authorship, including any modifications or additions to an existing work, that
is intentionally submitted by Intel to llvm.org (http://llvm.org) ("LLVM") for
inclusion in, or documentation of, any of the products owned or managed by LLVM
(the "Work"). For the purposes of this definition, "submitted" means any form of
electronic, verbal, or written communication sent to LLVM or its
representatives, including but not limited to communication on electronic
mailing lists, source code control systems, and issue tracking systems that are
managed by, or on behalf of, LLVM for the purpose of discussing and improving
the Work, but excluding communication that is conspicuously marked otherwise.

1. Grant of Copyright License. Subject to the terms and conditions of this
   Agreement, Intel hereby grants to you and to recipients of the Software
   distributed by LLVM a perpetual, worldwide, non-exclusive, no-charge,
   royalty-free, irrevocable copyright license to reproduce, prepare derivative
   works of, publicly display, publicly perform, sublicense, and distribute the
   Software and such derivative works.

2. Grant of Patent License. Subject to the terms and conditions of this
   Agreement, Intel hereby grants you and to recipients of the Software
   distributed by LLVM a perpetual, worldwide, non-exclusive, no-charge,
   royalty-free, irrevocable (except as stated in this section) patent license
   to make, have made, use, offer to sell, sell, import, and otherwise transfer
   the Work, where such license applies only to those patent claims licensable
   by Intel that are necessarily infringed by Intel's Software alone or by
   combination of the Software with the Work to which such Software was
   submitted. If any entity institutes patent litigation against Intel or any
   other entity (including a cross-claim or counterclaim in a lawsuit) alleging
   that Intel's Software, or the Work to which Intel has contributed constitutes
   direct or contributory patent infringement, then any patent licenses granted
   to that entity under this Agreement for the Software or Work shall terminate
   as of the date such litigation is filed.

Unless required by applicable law or agreed to in writing, the software is
provided on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
either express or implied, including, without limitation, any warranties or
conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE.

Zha0q1 · 2020-10-28T20:37:31Z

prepared a pr #19439

josephevans · 2020-10-29T00:08:24Z

#19440 should have all license updates consolidated for all submodules.

pengzhao-intel · 2020-10-29T02:43:57Z

@PatricZhao could someone on your team help verify mkldnn licenses?

@anko-intel to take care

pengzhao-intel · 2020-10-29T02:47:46Z

@szha @PatricZhao

For mkldnn we have:
12. 3rdparty/mkldnn - For details, see, 3rdparty/mkldnn/LICENSE Copyright 2016-2019 Intel Corporation Copyright 2018 YANDEX LLC
which is consistent with their LICENSE file, but there is also:
"Copyright 2020 Arm Limited and affiliates." in blas.cmake and .drone.yml files;
"Copyright 2020 Intel Corporation" and "Copyright 2016/7/8/9-2020 Intel Corporation" in many mkldnn files

For its submodules:
2. gtest - For details, see, 3rdparty/mkldnn/tests/gtests/gtest/LICENSE Copyright 2005-2008, Google Inc.
where there is also "Copyright 2015, Google Inc." listed in mkldnn/THIRD-PARTY-PROGRAMS

9. CMake FindCUDAToolkit.cmake, FindOpenCL.cmake - For details, see, cmake/upstream/FindCUDAToolkit.cmake cmake/upstream/select_compute_arch.cmake 3rdparty/mkldnn/cmake/FindOpenCL.cmake Copyright 2000-2019 Kitware, Inc. and Contributors
where there is also "Copyright 2000-2020 Kitware, Inc. and Contributors"

19. Benchdnn - For details, see 3rdparty/mkldnn/tests/benchdnn/README.md Copyright 2017-2018 Intel Corporation
where there is "Copyright 2017-2020 Intel Corporation" in benchdnn files

Not listed in our license but listed in mkldnn/THIRD-PARTY-PROGRAMS:
Boost C++ Libraries:
Copyright 2005-2014 Daniel James.
Should it be added or is this one unnecessary?

@ciyongch do you have any sense for this part?

ciyongch · 2020-10-29T06:18:43Z

Regarding 3rdparty/mkldnn, it seems there're some new files/license updates added in v1.6.4 (MX 1.8.x) compared to v1.3 (in MX 1.7.0), so we need to update the new license or statement in the top level LICENSE files to reflect the change as well.

I've the same questions on whether the year range is sufficient or not for the Copyright statement? For example, there're several different year (range) like: 2016-2020, 2017-2020, 2018-2020, 2019-2020, 2016, 2017, 2018, 2019, 2020 used in different mkldnn files, is that acceptable to just keep the year range instead of listing all the individual different years?

josephevans · 2020-10-29T18:22:38Z

Regarding 3rdparty/mkldnn, it seems there're some new files/license updates added in v1.6.4 (MX 1.8.x) compared to v1.3 (in MX 1.7.0), so we need to update the new license or statement in the top level LICENSE files to reflect the change as well.

I've the same questions on whether the year range is sufficient or not for the Copyright statement? For example, there're several different year (range) like: 2016-2020, 2017-2020, 2018-2020, 2019-2020, 2016, 2017, 2018, 2019, 2020 used in different mkldnn files, is that acceptable to just keep the year range instead of listing all the individual different years?

@ciyongch I think it's fine to just update the range of years if the copyright holder is exactly the same.

ciyongch · 2020-10-30T01:26:25Z

Thanks @bartekkuncer to add the missing info for mkldnn.
Hi @samskalicky , mkldnn part should be covered by #19443, can be marked as DONE?

samskalicky · 2020-10-30T16:31:23Z

Thanks everyone, after the #19440 and #19443 we've covered all the submodules. We need to do better about maintaining the LICENSE when updating submodules moving forward. I'll be sure to include this in the summary for the v1.8.0 release when it completes and we can followup on the mailing list.

Thanks again for everyone's help, it was great to see the community come together to address these issues! Cheers!

samskalicky added Bug needs triage labels Oct 26, 2020

szha added Licenses and removed Bug needs triage labels Oct 26, 2020

access2rohit mentioned this issue Oct 27, 2020

updating License for dlpack for v1.8 release #19433

Closed

1 task

waytrue17 mentioned this issue Oct 27, 2020

Updating license for dmlc-core for v1.8 release #19434

Closed

1 task

Zha0q1 mentioned this issue Oct 28, 2020

OpenMP License Update v1.8x #19439

Closed

josephevans mentioned this issue Oct 28, 2020

[v1.8.x] Update LICENSE for all submodules. #19440

Merged

6 tasks

mseth10 mentioned this issue Oct 28, 2020

update googletest license #19442

Closed

bartekkuncer mentioned this issue Oct 29, 2020

[v1.8.x]LICENSE update for mkldnn and its 3rd party #19443

Merged

samskalicky closed this as completed Feb 18, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[1.8.x] LICENSE needs update/review #19427

[1.8.x] LICENSE needs update/review #19427

samskalicky commented Oct 26, 2020 •

edited by pengzhao-intel

Loading

leezu commented Oct 26, 2020

kpuatamazon commented Oct 26, 2020

samskalicky commented Oct 26, 2020

szha commented Oct 26, 2020

szha commented Oct 27, 2020

samskalicky commented Oct 27, 2020

ptrendx commented Oct 27, 2020

samskalicky commented Oct 27, 2020 •

edited

Loading

sandeep-krishnamurthy commented Oct 27, 2020 •

edited

Loading

Zha0q1 commented Oct 27, 2020 •

edited

Loading

samskalicky commented Oct 27, 2020

access2rohit commented Oct 27, 2020

sandeep-krishnamurthy commented Oct 27, 2020

access2rohit commented Oct 27, 2020 •

edited

Loading

access2rohit commented Oct 27, 2020 •

edited

Loading

waytrue17 commented Oct 27, 2020 •

edited

Loading

leezu commented Oct 27, 2020

mseth10 commented Oct 28, 2020 •

edited

Loading

bartekkuncer commented Oct 28, 2020 •

edited

Loading

Zha0q1 commented Oct 28, 2020

Zha0q1 commented Oct 28, 2020

josephevans commented Oct 29, 2020

pengzhao-intel commented Oct 29, 2020

pengzhao-intel commented Oct 29, 2020

ciyongch commented Oct 29, 2020

josephevans commented Oct 29, 2020

ciyongch commented Oct 30, 2020

samskalicky commented Oct 30, 2020

[1.8.x] LICENSE needs update/review #19427

[1.8.x] LICENSE needs update/review #19427

Comments

samskalicky commented Oct 26, 2020 • edited by pengzhao-intel Loading

Description

leezu commented Oct 26, 2020

kpuatamazon commented Oct 26, 2020

samskalicky commented Oct 26, 2020

szha commented Oct 26, 2020

szha commented Oct 27, 2020

samskalicky commented Oct 27, 2020

ptrendx commented Oct 27, 2020

samskalicky commented Oct 27, 2020 • edited Loading

sandeep-krishnamurthy commented Oct 27, 2020 • edited Loading

Zha0q1 commented Oct 27, 2020 • edited Loading

samskalicky commented Oct 27, 2020

access2rohit commented Oct 27, 2020

sandeep-krishnamurthy commented Oct 27, 2020

access2rohit commented Oct 27, 2020 • edited Loading

access2rohit commented Oct 27, 2020 • edited Loading

waytrue17 commented Oct 27, 2020 • edited Loading

leezu commented Oct 27, 2020

mseth10 commented Oct 28, 2020 • edited Loading

bartekkuncer commented Oct 28, 2020 • edited Loading

Zha0q1 commented Oct 28, 2020

Zha0q1 commented Oct 28, 2020

josephevans commented Oct 29, 2020

pengzhao-intel commented Oct 29, 2020

pengzhao-intel commented Oct 29, 2020

ciyongch commented Oct 29, 2020

josephevans commented Oct 29, 2020

ciyongch commented Oct 30, 2020

samskalicky commented Oct 30, 2020

samskalicky commented Oct 26, 2020 •

edited by pengzhao-intel

Loading

samskalicky commented Oct 27, 2020 •

edited

Loading

sandeep-krishnamurthy commented Oct 27, 2020 •

edited

Loading

Zha0q1 commented Oct 27, 2020 •

edited

Loading

access2rohit commented Oct 27, 2020 •

edited

Loading

access2rohit commented Oct 27, 2020 •

edited

Loading

waytrue17 commented Oct 27, 2020 •

edited

Loading

mseth10 commented Oct 28, 2020 •

edited

Loading

bartekkuncer commented Oct 28, 2020 •

edited

Loading