ZEPPELIN-335: Apache Pig interpreter #338
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
|
Hi @abajwa-hw, |
Member
|
Yes, tests and update for the docs will be great as part of this PR indeed! BTW Spark interpreter does not require Spark installed on the same machine with Zeppelin. |
Member
There was a problem hiding this comment.
It is not strictly documented yet, but we as many other ASF projects (Hadoop, Zookeeper, Avro, etc) do not encourage use of @author tags.
Would you be willing to remove that information from all for the comments, please?
We definitely want and keep contributors credits, but we use git, jira and mailing list history, so nothing will be lost.
1. Documentation: added pig.md with interpreter documentation and added pig entry to index.md 2. Added test junit test based on passwd file parsing example here https://pig.apache.org/docs/r0.10.0/start.html#run 3. Removed author tag from comment (this was copied from shell interpreter https://github.com/apache/incubator-zeppelin/blob/master/shell/src/main/java/org/apache/zeppelin/shell/ShellInterpreter.java#L42) 4. Implemented cancel functionality 5. Display output stream in case of error
|
Are there plans for pig syntax highlighting similar to what a vim configuration can add? Table/graphing results with this interpreter similar to how the sql interpreter handles results would also be a nice feature |
Member
|
@abajwa-hw any update? it would be a good additional to Zeppelin |
|
@abajwa-hw @felixcheung @bzz is the idea to have a pig interpreter dead? It could be really useful... |
Contributor
|
@abajwa-hw |
Contributor
|
I create #1476 for a new implementation of pig interpreter. |
asfgit
pushed a commit
that referenced
this pull request
Oct 15, 2016
### What is this PR for? Based on #338 , I refactor most of pig interpreter. As I don't think the approach in #338 is the best approach. In #338, we use script `bin/pig` to launch pig script, it is different to control that job (hard to kill and get progress and stats info). In this PR, I use pig api to launch pig script. Besides that I implement another interpreter type `%pig.query` to leverage the display system of zeppelin. For the details you can check `pig.md` ### What type of PR is it? [Feature] ### Todos * Syntax Highlight * new interpreter type `%pig.udf`, so that user can write pig udf in zeppelin directly and don't need to build udf jar manually. ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-335 ### How should this be tested? Unit test is added and also manual test is done ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <[email protected]> Author: Ali Bajwa <[email protected]> Author: AhyoungRyu <[email protected]> Author: Jeff Zhang <[email protected]> Closes #1476 from zjffdu/ZEPPELIN-335 and squashes the following commits: 73a07f0 [Jeff Zhang] minor update a1b742b [Jeff Zhang] minor update on doc e858301 [Jeff Zhang] address comments c85a090 [Jeff Zhang] add license 58b4b2f [Jeff Zhang] minor update of docs 1ae7db2 [Jeff Zhang] Merge pull request #2 from AhyoungRyu/ZEPPELIN-335/docs fe014a7 [AhyoungRyu] Fix docs title in front matter df7a6db [AhyoungRyu] Add pig.md to dropdown menu 5e2e222 [AhyoungRyu] Minor update for pig.md 39f161a [Jeff Zhang] address comments 05a3b9b [Jeff Zhang] add pig.md a09a7f7 [Jeff Zhang] refactor pig Interpreter c28beb5 [Ali Bajwa] Updated based on comments: 1. Documentation: added pig.md with interpreter documentation and added pig entry to index.md 2. Added test junit test based on passwd file parsing example here https://pig.apache.org/docs/r0.10.0/start.html#run 3. Removed author tag from comment (this was copied from shell interpreter https://github.com/apache/incubator-zeppelin/blob/master/shell/src/main/java/org/apache/zeppelin/shell/ShellInterpreter.java#L42) 4. Implemented cancel functionality 5. Display output stream in case of error 2586336 [Ali Bajwa] exposed timeout and pig executable via interpreter and added comments 7abad20 [Ali Bajwa] initial commit of pig interpreter
darionyaphet
pushed a commit
to darionyaphet/zeppelin
that referenced
this pull request
Oct 27, 2016
### What is this PR for? Based on apache#338 , I refactor most of pig interpreter. As I don't think the approach in apache#338 is the best approach. In apache#338, we use script `bin/pig` to launch pig script, it is different to control that job (hard to kill and get progress and stats info). In this PR, I use pig api to launch pig script. Besides that I implement another interpreter type `%pig.query` to leverage the display system of zeppelin. For the details you can check `pig.md` ### What type of PR is it? [Feature] ### Todos * Syntax Highlight * new interpreter type `%pig.udf`, so that user can write pig udf in zeppelin directly and don't need to build udf jar manually. ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-335 ### How should this be tested? Unit test is added and also manual test is done ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <[email protected]> Author: Ali Bajwa <[email protected]> Author: AhyoungRyu <[email protected]> Author: Jeff Zhang <[email protected]> Closes apache#1476 from zjffdu/ZEPPELIN-335 and squashes the following commits: 73a07f0 [Jeff Zhang] minor update a1b742b [Jeff Zhang] minor update on doc e858301 [Jeff Zhang] address comments c85a090 [Jeff Zhang] add license 58b4b2f [Jeff Zhang] minor update of docs 1ae7db2 [Jeff Zhang] Merge pull request apache#2 from AhyoungRyu/ZEPPELIN-335/docs fe014a7 [AhyoungRyu] Fix docs title in front matter df7a6db [AhyoungRyu] Add pig.md to dropdown menu 5e2e222 [AhyoungRyu] Minor update for pig.md 39f161a [Jeff Zhang] address comments 05a3b9b [Jeff Zhang] add pig.md a09a7f7 [Jeff Zhang] refactor pig Interpreter c28beb5 [Ali Bajwa] Updated based on comments: 1. Documentation: added pig.md with interpreter documentation and added pig entry to index.md 2. Added test junit test based on passwd file parsing example here https://pig.apache.org/docs/r0.10.0/start.html#run 3. Removed author tag from comment (this was copied from shell interpreter https://github.com/apache/incubator-zeppelin/blob/master/shell/src/main/java/org/apache/zeppelin/shell/ShellInterpreter.java#L42) 4. Implemented cancel functionality 5. Display output stream in case of error 2586336 [Ali Bajwa] exposed timeout and pig executable via interpreter and added comments 7abad20 [Ali Bajwa] initial commit of pig interpreter
pedrozatta
pushed a commit
to pedrozatta/zeppelin
that referenced
this pull request
Oct 27, 2016
### What is this PR for? Based on apache#338 , I refactor most of pig interpreter. As I don't think the approach in apache#338 is the best approach. In apache#338, we use script `bin/pig` to launch pig script, it is different to control that job (hard to kill and get progress and stats info). In this PR, I use pig api to launch pig script. Besides that I implement another interpreter type `%pig.query` to leverage the display system of zeppelin. For the details you can check `pig.md` ### What type of PR is it? [Feature] ### Todos * Syntax Highlight * new interpreter type `%pig.udf`, so that user can write pig udf in zeppelin directly and don't need to build udf jar manually. ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-335 ### How should this be tested? Unit test is added and also manual test is done ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <[email protected]> Author: Ali Bajwa <[email protected]> Author: AhyoungRyu <[email protected]> Author: Jeff Zhang <[email protected]> Closes apache#1476 from zjffdu/ZEPPELIN-335 and squashes the following commits: 73a07f0 [Jeff Zhang] minor update a1b742b [Jeff Zhang] minor update on doc e858301 [Jeff Zhang] address comments c85a090 [Jeff Zhang] add license 58b4b2f [Jeff Zhang] minor update of docs 1ae7db2 [Jeff Zhang] Merge pull request apache#2 from AhyoungRyu/ZEPPELIN-335/docs fe014a7 [AhyoungRyu] Fix docs title in front matter df7a6db [AhyoungRyu] Add pig.md to dropdown menu 5e2e222 [AhyoungRyu] Minor update for pig.md 39f161a [Jeff Zhang] address comments 05a3b9b [Jeff Zhang] add pig.md a09a7f7 [Jeff Zhang] refactor pig Interpreter c28beb5 [Ali Bajwa] Updated based on comments: 1. Documentation: added pig.md with interpreter documentation and added pig entry to index.md 2. Added test junit test based on passwd file parsing example here https://pig.apache.org/docs/r0.10.0/start.html#run 3. Removed author tag from comment (this was copied from shell interpreter https://github.com/apache/incubator-zeppelin/blob/master/shell/src/main/java/org/apache/zeppelin/shell/ShellInterpreter.java#L42) 4. Implemented cancel functionality 5. Display output stream in case of error 2586336 [Ali Bajwa] exposed timeout and pig executable via interpreter and added comments 7abad20 [Ali Bajwa] initial commit of pig interpreter
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
8 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Added an interpreter for Apache Pig. Based closely on Shell interpreter.
Assumes that pig client is installed on the same node as Zeppelin (similar to how the Spark interpreter assumes that Spark is installed on Zeppelin node)