[ZEPPELIN-713] Expand ResourceId and put InterpreterResult into ResourcePool #763
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Leemoonsoo
force-pushed
the
ZEPPELIN-713
branch
2 times, most recently
from
54cc78a to
473dd6f
Compare
Member
Author
|
Please review. |
Member
Author
|
Merge if there're no more discussions |
2 tasks
Member
|
LGTM |
fireboy1919
pushed a commit
to fireboy1919/incubator-zeppelin
that referenced
this pull request
Apr 5, 2016
…rcePool ResourcePool is introduced by ZEPPELIN-619. However, it didn't provide any automatic resource release on paragraph / note removal unlike paragraph scope angular object which is quite helpful in practical use. This PR adds noteId and paragraphId to ResourceId, so resource can be identified by noteId and paragraphId in addition to resourcePoolId and name. By using informations in ResourceId, automatically release resource when note / paragraph remove. So now, it's possible to put InterpreterResult into the resource pool safely, which might be useful for ZEPPELIN-374 and ZEPPELIN-533. Improvement * [x] - Add noteId and paragraphId to ResourceId * [x] - Release resource when note / paragraph remove * [x] - Put InterpreterResult into resource pool * [x] - Unit test https://issues.apache.org/jira/browse/ZEPPELIN-713 You can get resourcePool from InterpreterContext and print items inside of spark interpreter. (note. resource pool is not spark interpreter specific feature. But so far spark interpreter provides user api to access it) You can see InterpreterResult is automatically shared to resourcePool and they're removed on note/paragraph remove. ``` %md hello world %spark // print items in resource pool. import collection.JavaConversions._ val pool = z.getInterpreterContext.getResourcePool pool.getAll.foreach(i=>println(i + " " + i.get)) ``` * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Lee moon soo <moon@apache.org> Closes apache#763 from Leemoonsoo/ZEPPELIN-713 and squashes the following commits: 473dd6f [Lee moon soo] Update test b2e4fcc [Lee moon soo] put interpreter result into resource pool 11d138c [Lee moon soo] Remove resource on note / paragraph removal
onkarshedge
pushed a commit
to onkarshedge/incubator-zeppelin
that referenced
this pull request
May 11, 2016
…rcePool ### What is this PR for? ResourcePool is introduced by ZEPPELIN-619. However, it didn't provide any automatic resource release on paragraph / note removal unlike paragraph scope angular object which is quite helpful in practical use. This PR adds noteId and paragraphId to ResourceId, so resource can be identified by noteId and paragraphId in addition to resourcePoolId and name. By using informations in ResourceId, automatically release resource when note / paragraph remove. So now, it's possible to put InterpreterResult into the resource pool safely, which might be useful for ZEPPELIN-374 and ZEPPELIN-533. ### What type of PR is it? Improvement ### Todos * [x] - Add noteId and paragraphId to ResourceId * [x] - Release resource when note / paragraph remove * [x] - Put InterpreterResult into resource pool * [x] - Unit test ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-713 ### How should this be tested? You can get resourcePool from InterpreterContext and print items inside of spark interpreter. (note. resource pool is not spark interpreter specific feature. But so far spark interpreter provides user api to access it) You can see InterpreterResult is automatically shared to resourcePool and they're removed on note/paragraph remove. ``` %md hello world %spark // print items in resource pool. import collection.JavaConversions._ val pool = z.getInterpreterContext.getResourcePool pool.getAll.foreach(i=>println(i + " " + i.get)) ``` ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Lee moon soo <moon@apache.org> Closes apache#763 from Leemoonsoo/ZEPPELIN-713 and squashes the following commits: 473dd6f [Lee moon soo] Update test b2e4fcc [Lee moon soo] put interpreter result into resource pool 11d138c [Lee moon soo] Remove resource on note / paragraph removal
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What is this PR for?
ResourcePool is introduced by ZEPPELIN-619.
However, it didn't provide any automatic resource release on paragraph / note removal unlike paragraph scope angular object which is quite helpful in practical use.
This PR adds noteId and paragraphId to ResourceId, so resource can be identified by noteId and paragraphId in addition to resourcePoolId and name.
By using informations in ResourceId, automatically release resource when note / paragraph remove.
So now, it's possible to put InterpreterResult into the resource pool safely, which might be useful for ZEPPELIN-374 and ZEPPELIN-533.
What type of PR is it?
Improvement
Todos
What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-713
How should this be tested?
You can get resourcePool from InterpreterContext and print items inside of spark interpreter. (note. resource pool is not spark interpreter specific feature. But so far spark interpreter provides user api to access it)
You can see InterpreterResult is automatically shared to resourcePool and they're removed on note/paragraph remove.
Screenshots (if appropriate)
Questions: