-
Notifications
You must be signed in to change notification settings - Fork 2.8k
[Zeppelin 546](documentation) Enables interpreter library loading from maven repository #609
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 3 commits
3067025
16b46f4
a9ec7d7
d3cc50f
e180a8e
266dac0
d0675e0
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,117 @@ | ||
| --- | ||
| layout: page | ||
| title: "Dynamic Interpreter Loading" | ||
| description: "" | ||
| group: manual | ||
| --- | ||
| {% include JB/setup %} | ||
|
|
||
| ## Dynamic Interpreter Loading using REST API | ||
|
|
||
| Zeppelin provides pluggable interpreter architecture which results in a wide and variety of the supported backend system. In this section, we will introduce **Dynamic interpreter loading** using **REST API**. Actually this concept is come up with [Zeppelin Helium Proposal](https://cwiki.apache.org/confluence/display/ZEPPELIN/Helium+proposal). | ||
| Before the start, if you are not clear about the concept of **Zeppelin interpreter**, you can ckeck out [Overview of Zeppelin interpreter](../manual/interpreters.html) first. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Before the start, if you are not clear about the concept of |
||
|
|
||
| <br/> | ||
| ## Overview | ||
| In the past, Zeppelin loads interpreter binaries from `/interpreter/[interpreter_name]` directory. And they are configured by `zeppelin.interpreters` property in `conf/zeppelin-site.xml` or `ZEPPELIN_INTERPRETERS` env variables in `conf/zeppelin-env.sh`. They are loaded when Zeppelin server is starting up and stay alive until the server is down. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In the past, Zeppelin loads interpreter binaries And they are configured by They are loaded when Zeppelin server starting up and stay alive until the server is down. |
||
| However, for using 3rd party interpreter much easier, we change this way to **dynamically** loading interpreters from **Maven Repository** using **REST API**. Hopefully, below picture helps you to understand the process easily. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. However, for using 3rd party interpreter much easier, we change this way to dynamically loading Hopefully, below helps you to understand the process easily. |
||
| <center><img src="../assets/themes/zeppelin/img/docs-img/zeppelin_user.png" height="85%" width="85%"></center> | ||
|
|
||
| ## Load & Unload Interpreters Using REST API | ||
|
|
||
| ### 1. Load | ||
| You can **load** interpreters located in Maven repository using REST API, like this: | ||
|
|
||
| ( Maybe, you are unfamiliar with `[interpreter_group_name]` or `[interpreter_name]`. If so, please checkout [Interpreters in Zeppelin](../manual/interpreter.html) again. ) | ||
|
|
||
| ``` | ||
| http://[zeppelin-server]:[zeppelin-port]/api/interpreter/load/[interpreter_group_name]/[interpreter_name] | ||
| ``` | ||
| The Restful method will be <code>**POST**</code>. And the parameters you need are like: | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. And the parameters you need are like: |
||
|
|
||
| 1. **Artifact:** Maven artifact ( groupId:artifactId:version ) | ||
|
|
||
| 2. **Class Name:** Package name + Interpreter class name | ||
|
|
||
| 3. **Repository ( optional ):** Additional maven repository address | ||
|
|
||
| For example, if you want to load `markdown` interpreter to your Zeppelin, above parameters and the URL you need may like: | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. above parameters and the URL you need may like: |
||
|
|
||
| ``` | ||
| http://127.0.0.1:8080/api/interpreter/load/md/markdown | ||
| ``` | ||
|
|
||
| ``` | ||
| { | ||
| "artifact": "org.apache.zeppelin:zeppelin-markdown:0.6.0-incubating-SNAPSHOT", | ||
| "className": "org.apache.zeppelin.markdown.Markdown", | ||
| "repository": { | ||
| "url": "http://dl.bintray.com/spark-packages/maven", | ||
| "snapshot": false | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
|
|
||
| The meaning of each parameters is: | ||
|
|
||
| 1. **Artifact** | ||
| - groupId: org.apache.zeppelin | ||
| - artifactId: zeppelin-markdown | ||
| - version: 0.6.0-incubating-SNAPSHOT | ||
|
|
||
| 2. **Class Name** | ||
| - Package Name: org.apache.zeppelin | ||
| - Interpreter Class Name: markdown.Markdown | ||
|
|
||
| 3. **Repository ( optional )** | ||
| - Url: http://dl.bintray.com/spark-packages/maven | ||
| - Snapshot: false | ||
|
|
||
| > <b>Please note: </b>The interpreters you downloaded need to be **reload**, when your Zeppelin server is down. | ||
|
|
||
| ### 2. Unload | ||
| If you want to **unload** the interpreters using REST API, | ||
|
|
||
| ``` | ||
| http://[zeppelin-server]:[zeppelin-port]/api/interpreter/unload/[interpreter_group_name]/[interpreter_name] | ||
| ``` | ||
| In this time, the Restful method will be <code>**DELETE**</code>. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In this time |
||
|
|
||
| <br/> | ||
| ## What is the next step after Loading ? | ||
|
|
||
| Maybe, you are curious several things about how you can actually use the interpreter in Zeppelin. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not sure this sentence is needed |
||
|
|
||
| ### Q1. Where is the location of interpreters you downloaded ? | ||
|
|
||
| Actually, the answer about this question is in the above picture. Once the REST API is called, the `.jar` files of interpreters you get are saved under `ZEPPELIN_HOME/local-repo` first. Then, they will be copied to `ZEPPELIN_HOME/interpreter` directory. So, please checkout your `ZEPPELIN_HOME/interpreter`. | ||
|
|
||
| ### Q2. Then, how can I use this interpreter ? | ||
|
|
||
| After you downloaded, you can create and configure the interpreter at the Zeppelin **Interpreter tab**. Then, how can you do this ? Please follow these simple steps. It will be really straighforward. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. straightforward
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @prabhjyotsingh Ashamed, I fixed it again : ) |
||
|
|
||
| Oh, you don't need to restart your Zeppelin server. Because it is **Dynamic Interpreter Loading**, you can configure and load it **on runtime** ! | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. perhaps "at runtime"? |
||
|
|
||
| 1. After Zeppelin server up, browse Zeppelin home and click **Interpreter tab**. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. After Zeppelin server up, browse Zeppelin home and click Interpreter tab. |
||
| <center><img src="../assets/themes/zeppelin/img/docs-img/interpreter_setting_1.png" height="85%" width="85%"></center> | ||
|
|
||
| 2. At the **Interpreter** section, click **+Create** button. | ||
| <center><img src="../assets/themes/zeppelin/img/docs-img/interpreter_setting_2.png" height="85%" width="85%"></center> | ||
|
|
||
| 3. Then, you can verify the interpreter list that you loaded. | ||
| <center><img src="../assets/themes/zeppelin/img/docs-img/interpreter_setting_3.png" height="85%" width="85%"></center> | ||
|
|
||
| 4. After choosing interpreter, you can configure and use it. Don't forget to save it. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. After choosing interpreter |
||
|
|
||
| 5. Create a new notebook in the **Notebook** section, then you can bind the interpreters from your interpreter list. Just drag and drop ! | ||
| <center><img src="../assets/themes/zeppelin/img/docs-img/interpreter_binding_1.png" height="85%" width="85%"></center> | ||
| <center><img src="../assets/themes/zeppelin/img/docs-img/interpreter_binding_2.png" height="85%" width="85%"></center> | ||
|
|
||
| 6. At last, you can use your interpreter ! | ||
|
|
||
| If you want to get the specific information about respective interpreters, please checkout each interpreter documentaion. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. documentation |
||
|
|
||
|
|
||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We coul probably remove those empty lines |
||
|
|
||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually this concept is come up with
-> This concept actually comes from