[GOBBLIN-1845] Changes parallelstream to stream in DatasetsFinderFilt…

[vikrambohra]

…eringDecorator to avoid classloader issues in spark

Dear Gobblin maintainers,

Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!

JIRA

• My PR addresses the following Gobblin JIRA issues and references them in the PR title. For example, "[GOBBLIN-1845] My Gobblin PR"
  • https://issues.apache.org/jira/browse/GOBBLIN-1845

Description

• Here are some details about my PR, including screenshots (if applicable):
  DatasetsFinderFilteringDecorator uses parallel stream on datasets to filter them on predicates. When this code runs in spark, system class loader gets used to pickup hive jar instead of the current conext class loader which leads to ClassNotFound issues

Tests

• My PR adds the following unit tests OR does not need testing for this extremely good reason:
  Small change that should already be covered in unit tests
  Also, tested this on the hadoop cluster at linkedin

Commits

• My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
  1. Subject is separated from body by a blank line
  2. Subject is limited to 50 characters
  3. Subject does not end with a period
  4. Subject uses the imperative mood ("add", not "adding")
  5. Body wraps at 72 characters
  6. Body explains "what" and "why", not "how"

[ZihanLi58]

+1

[codecov-commenter]

Codecov Report

Merging #3706 (a5a5924) into master (ee17b15) will increase coverage by 0.01%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##             master    #3706      +/-   ##
============================================
+ Coverage     46.83%   46.84%   +0.01%     
- Complexity    10797    10806       +9     
============================================
  Files          2141     2141              
  Lines         84410    84410              
  Branches       9383     9383              
============================================
+ Hits          39533    39545      +12     
+ Misses        41272    41264       -8     
+ Partials       3605     3601       -4     

Impacted Files	Coverage Δ
...ment/dataset/DatasetsFinderFilteringDecorator.java	89.18% <100.00%> (ø)

... and 9 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[mridulm]

This will work, but is performance a concern ?
If yes, setting the thread's context classloader to spark classloader should fix that.