Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove dependencies from libraries under category X licenses. #3718

Merged
merged 1 commit into from
Jan 31, 2019

Conversation

snleee
Copy link
Contributor

@snleee snleee commented Jan 18, 2019

For an ASF project, its distribution should not contain any of work under
category X license. After running "mvn project-info-reports:dependencies",
three libraries are under category X libraries for our distribution package.
This PR resolves this issue.

  1. Removed "pinot-perf" module from distribution package because pinot-perf
    module is pulling "jmh-core" based on GPL 2 license, which is a category X.
  2. Bumped up the version of org.testng from 6.0.1 to 6.11 because the old
    version was pulling in "org.beanshell:bsh:2.0b4", which is also based on
    LGPL license. The newer version no longer pulls beanshell.
  3. Bumped up the swagger version from 1.5.10 to 1.5.15. The old version was
    pulling "org.reflections:reflections:0.9.10", which pulls
    "com.google.code.findbugs:annotations", which was based on LGPL.

For the list of category X licenses, please refer
https://www.apache.org/legal/resolved.html

@snleee snleee force-pushed the dependency-cleanup branch from 37dc74a to cff0bd4 Compare January 18, 2019 10:24
snleee pushed a commit that referenced this pull request Jan 18, 2019
Followed Apache Spark's approach for handling LICENSE, NOTICE.
apache/spark@f825847

- Checked licenses for all libraries that we bundle
- Checked licenses for all js/css files that we bundle
- Removed dependencies of category X libraries (#3718)
@felixcheung
Copy link
Member

is there replacement for "pinot-perf" module?

@snleee
Copy link
Contributor Author

snleee commented Jan 20, 2019

@felixcheung pinot-perf module includes some benchmarking codes that are mainly used for measuring performances (e.g. dictionary, index reader, query engine..etc). Given that this is mainly for testing, I don't think that we need to include this for distribution package.

@felixcheung
Copy link
Member

felixcheung commented Jan 20, 2019

@snleee sure - keep in mind ASF release is primarily a source code release. sounds like pinot-perf (source code) is still there and so long as it doesn't contain source code in a different license.

it will help to have an README in the source to point out pinot-perf module is pulling "jmh-core" based on GPL 2 license, and instruction to build pinot-perf

@snleee snleee force-pushed the dependency-cleanup branch 5 times, most recently from f407cc9 to 7f5d1cf Compare January 31, 2019 08:12
snleee pushed a commit that referenced this pull request Jan 31, 2019
* Update LICENSE and NOTICE files

Followed Apache Spark's approach for handling LICENSE, NOTICE.
apache/spark@f825847

- Checked licenses for all libraries that we bundle
- Checked licenses for all js/css files that we bundle
- Removed dependencies of category X libraries (#3718)

* Add LICENSE-binary, NOTICE-binary to differentiate source code and binary release
@codecov-io
Copy link

codecov-io commented Jan 31, 2019

Codecov Report

Merging #3718 into master will increase coverage by 0.28%.
The diff coverage is n/a.

Impacted file tree graph

@@             Coverage Diff              @@
##             master    #3718      +/-   ##
============================================
+ Coverage     67.01%   67.29%   +0.28%     
  Complexity        4        4              
============================================
  Files          1027     1027              
  Lines         50798    50796       -2     
  Branches       7091     7091              
============================================
+ Hits          34041    34182     +141     
+ Misses        14423    14284     -139     
+ Partials       2334     2330       -4
Impacted Files Coverage Δ Complexity Δ
...elix/core/relocation/RealtimeSegmentRelocator.java 74.48% <0%> (-7.15%) 0% <0%> (ø)
.../apache/pinot/common/config/RealtimeTagConfig.java 93.33% <0%> (-6.67%) 0% <0%> (ø)
...e/operator/dociditerators/BitmapDocIdIterator.java 60.71% <0%> (-3.58%) 0% <0%> (ø)
...e/operator/dociditerators/MVScanDocIdIterator.java 60.6% <0%> (-3.04%) 0% <0%> (ø)
.../core/indexsegment/mutable/MutableSegmentImpl.java 68.59% <0%> (-0.09%) 0% <0%> (ø)
...pinot/core/plan/maker/InstancePlanMakerImplV2.java 95.71% <0%> (-0.07%) 0% <0%> (ø)
...ntroller/helix/core/PinotHelixResourceManager.java 60.16% <0%> (+0.1%) 0% <0%> (ø) ⬇️
.../broker/routing/HelixExternalViewBasedRouting.java 89.06% <0%> (+0.32%) 0% <0%> (ø) ⬇️
...regation/function/customobject/QuantileDigest.java 58.18% <0%> (+0.44%) 0% <0%> (ø) ⬇️
...ot/common/protocols/SegmentCompletionProtocol.java 94.15% <0%> (+0.58%) 0% <0%> (ø) ⬇️
... and 25 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 26f2aad...7a96b5c. Read the comment docs.

@snleee
Copy link
Contributor Author

snleee commented Jan 31, 2019

@felixcheung As you mentioned, pinot-perf will still be included in the source code distribution while it won't be a part of binary distribution. I have added the documentation on how to build & run pinot-perf module from the source code and added a note that this module will pull jmh which is under GPL 2 license.

For an ASF project, its distribution should not contain any of work under
category X license. After running "mvn project-info-reports:dependencies",
three libraries are under category X libraries for our distribution package.
This PR resolves this issue.

1. Removed "pinot-perf" module from distribution package because pinot-perf
   module is pulling "jmh-core" based on GPL 2 license, which is a category X.
2. Fixed pom file to generate *.sh files for benchmarks in "pinot-perf" and
   added on documentation on how to build and run.
3. Bumped up the version of org.testng from 6.0.1 to 6.11 because the old
   version was pulling in "org.beanshell:bsh:2.0b4", which is also based on
   LGPL license. The newer version no longer pulls beanshell.
4. Bumped up the swagger version from 1.5.10 to 1.5.16. The old version was
   pulling "org.reflections:reflections:0.9.10", which pulls
   "com.google.code.findbugs:annotations", which was based on LGPL.

For the list of category X licenses, please refer
https://www.apache.org/legal/resolved.html
@snleee snleee force-pushed the dependency-cleanup branch from 7f5d1cf to 7a96b5c Compare January 31, 2019 21:46
@snleee snleee merged commit 7e28022 into master Jan 31, 2019
@snleee snleee deleted the dependency-cleanup branch January 31, 2019 22:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants