Snap 293 #337
Merged
Snap 293 #337
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
user can install a jar using install jar utility and it will be available to all executors, store and driver node the jar uploaded via the job server also follows the same norm.
The executor is overridden as it's classloader should also look into the snappy store to load the classes.
Contributor
Author
|
Added another pull request after incorporating the review comments and 2.0 merge. For some open issues with PR open the Jira https://jira.snappydata.io/browse/SNAP-999?filter=-1 |
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes proposed in this pull request
Added support for dynamic Jar loading in snappy. The idea here is to support the existing snappy command for loading the jar file. For that lead node driver and executor class loader are modified in such a way that they look for the jar in the snappy-store if it is not found in regular class path.
Apart form above change there is another problem that once the job is submitted through the job server and if user make any changes to the jar related to submitted application and try to rerun again then executors are not able to pick the new jar. This problem is also resolved by loading the jar to the snappy store.
There are still few open issues with this PR which are logged under https://jira.snappydata.io/browse/SNAP-999
Patch testing
precheckin , dunit and manual testing for lead node path
ReleaseNotes.txt changes
(Does this change require an entry in ReleaseNotes.txt? If yes, has it been added to it?)
Other PRs
Yes.
TIBCOSoftware/snappy-store#88
TIBCOSoftware/snappy-spark#1
TIBCOSoftware/snappy-spark-jobserver#2