Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 14 additions & 7 deletions docs/development/writingzeppelininterpreter.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ In 'Separate Interpreter(scoped / isolated) for each note' mode which you can se
Creating a new interpreter is quite simple. Just extend [org.apache.zeppelin.interpreter](https://github.com/apache/zeppelin/blob/master/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/Interpreter.java) abstract class and implement some methods.
You can include `org.apache.zeppelin:zeppelin-interpreter:[VERSION]` artifact in your build system. And you should put your jars under your interpreter directory with a specific directory name. Zeppelin server reads interpreter directories recursively and initializes interpreters including your own interpreter.

There are three locations where you can store your interpreter group, name and other information. Zeppelin server tries to find the location below. Next, Zeppelin tries to find `interpreter-setting.json` in your interpreter jar.
There are three locations where you can store your interpreter group, name and other information. Zeppelin server tries to find the location below. Next, Zeppelin tries to find `interpreter-setting.json` in your interpreter jar.

```
{ZEPPELIN_INTERPRETER_DIR}/{YOUR_OWN_INTERPRETER_DIR}/interpreter-setting.json
Expand All @@ -68,12 +68,15 @@ Here is an example of `interpreter-setting.json` on your own interpreter.
"propertyName": null,
"defaultValue": "property2DefaultValue",
"description": "Property 2 description"
}, ...
},...
},
"editor": {
"language": "your-syntax-highlight-language"
}
},
{
...
}
}
]
```

Expand All @@ -96,15 +99,20 @@ some interpreter specific code...
```

## Programming Languages for Interpreter
If the interpreter uses a specific programming language ( like Scala, Python, SQL ), it is generally recommended to add a syntax highlighting supported for that to the notebook paragraph editor.
If the interpreter uses a specific programming language (like Scala, Python, SQL), it is generally recommended to add a syntax highlighting supported for that to the notebook paragraph editor.

To check out the list of languages supported, see the `mode-*.js` files under `zeppelin-web/bower_components/ace-builds/src-noconflict` or from [github.com/ajaxorg/ace-builds](https://github.com/ajaxorg/ace-builds/tree/master/src-noconflict).

If you want to add a new set of syntax highlighting,

1. Add the `mode-*.js` file to <code>[zeppelin-web/bower.json](https://github.com/apache/zeppelin/blob/master/zeppelin-web/bower.json)</code> ( when built, <code>[zeppelin-web/src/index.html](https://github.com/apache/zeppelin/blob/master/zeppelin-web/src/index.html)</code> will be changed automatically. ).
2. Add to the list of `editorMode` in <code>[zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js](https://github.com/apache/zeppelin/blob/master/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js)</code> - it follows the pattern 'ace/mode/x' where x is the name.
3. Add to the code that checks for `%` prefix and calls `session.setMode(editorMode.x)` in `setParagraphMode` located in <code>[zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js](https://github.com/apache/zeppelin/blob/master/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js)</code>.
2. Add `editor` object to `interpreter-setting.json` file. If you want to set your language to java, add:

```
"editor": {
"language": "java"
}
```

## Install your interpreter binary

Expand Down Expand Up @@ -216,4 +224,3 @@ We welcome contribution to a new interpreter. Please follow these few steps:
- Add documentation on how to use your interpreter under `docs/interpreter/`. Follow the Markdown style as this [example](https://github.com/apache/zeppelin/blob/master/docs/interpreter/elasticsearch.md). Make sure you list config settings and provide working examples on using your interpreter in code boxes in Markdown. Link to images as appropriate (images should go to `docs/assets/themes/zeppelin/img/docs-img/`). And add a link to your documentation in the navigation menu (`docs/_includes/themes/zeppelin/_navigation.html`).
- Most importantly, ensure licenses of the transitive closure of all dependencies are list in [license file](https://github.com/apache/zeppelin/blob/master/zeppelin-distribution/src/bin_license/LICENSE).
- Commit your changes and open a [Pull Request](https://github.com/apache/zeppelin/pulls) on the project [Mirror on GitHub](https://github.com/apache/zeppelin); check to make sure Travis CI build is passing.

3 changes: 3 additions & 0 deletions flink/src/main/resources/interpreter-setting.json
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@
"defaultValue": "6123",
"description": "port of running JobManager."
}
},
"editor": {
"language": "scala"
}
}
]
3 changes: 3 additions & 0 deletions jdbc/src/main/resources/interpreter-setting.json
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,9 @@
"defaultValue": "org.postgresql.Driver",
"description": ""
}
},
"editor": {
"language": "sql"
}
}
]
14 changes: 13 additions & 1 deletion livy/src/main/resources/interpreter-setting.json
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,9 @@
"defaultValue": "",
"description": "Kerberos keytab to authenticate livy"
}
},
"editor": {
"language": "scala"
}
},
{
Expand All @@ -100,20 +103,29 @@
"defaultValue": "false",
"description": "Execute multiple SQL concurrently if set true."
}
},
"editor": {
"language": "sql"
}
},
{
"group": "livy",
"name": "pyspark",
"className": "org.apache.zeppelin.livy.LivyPySparkInterpreter",
"properties": {
},
"editor": {
"language": "python"
}
},
{
"group": "livy",
"name": "sparkr",
"className": "org.apache.zeppelin.livy.LivySparkRInterpreter",
"properties": {
},
"editor": {
"language": "r"
}
}
]
]
3 changes: 3 additions & 0 deletions python/src/main/resources/interpreter-setting.json
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@
"defaultValue": "1000",
"description": "Max number of dataframe rows to display."
}
},
"editor": {
"language": "python"
}
},
{
Expand Down
5 changes: 4 additions & 1 deletion shell/src/main/resources/interpreter-setting.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,9 @@
"defaultValue": "60000",
"description": "Shell command time out in millisecs. Default = 60000"
}
},
"editor": {
"language": "sh"
}
}
]
]
12 changes: 12 additions & 0 deletions spark/src/main/resources/interpreter-setting.json
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,9 @@
"defaultValue": "local[*]",
"description": "Spark master uri. ex) spark://masterhost:7077"
}
},
"editor": {
"language": "scala"
}
},
{
Expand Down Expand Up @@ -85,6 +88,9 @@
"defaultValue": "true",
"description": "Import implicits, UDF collection, and sql if set true. true by default."
}
},
"editor": {
"language": "sql"
}
},
{
Expand All @@ -104,6 +110,9 @@
"defaultValue": "spark-packages,http://dl.bintray.com/spark-packages/maven,false;",
"description": "A list of 'id,remote-repository-URL,is-snapshot;' for each remote repository."
}
},
"editor": {
"language": "scala"
}
},
{
Expand All @@ -117,6 +126,9 @@
"defaultValue": "python",
"description": "Python command to run pyspark with"
}
},
"editor": {
"language": "python"
}
}
]
15 changes: 15 additions & 0 deletions spark/src/main/sparkr-resources/interpreter-setting.json
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,9 @@
"defaultValue": "local[*]",
"description": "Spark master uri. ex) spark://masterhost:7077"
}
},
"editor": {
"language": "scala"
}
},
{
Expand Down Expand Up @@ -85,6 +88,9 @@
"defaultValue": "true",
"description": "Import implicits, UDF collection, and sql if set true. true by default."
}
},
"editor": {
"language": "sql"
}
},
{
Expand All @@ -104,6 +110,9 @@
"defaultValue": "spark-packages,http://dl.bintray.com/spark-packages/maven,false;",
"description": "A list of 'id,remote-repository-URL,is-snapshot;' for each remote repository."
}
},
"editor": {
"language": "scala"
}
},
{
Expand All @@ -117,6 +126,9 @@
"defaultValue": "python",
"description": "Python command to run pyspark with"
}
},
"editor": {
"language": "python"
}
},
{
Expand Down Expand Up @@ -148,6 +160,9 @@
"defaultValue": "out.format = 'html', comment = NA, echo = FALSE, results = 'asis', message = F, warning = F",
"description": ""
}
},
"editor": {
"language": "r"
}
}
]
Original file line number Diff line number Diff line change
Expand Up @@ -251,6 +251,7 @@ public static class RegisteredInterpreter {
private String className;
private boolean defaultInterpreter;
private Map<String, InterpreterProperty> properties;
private Map<String, Object> editor;
private String path;

public RegisteredInterpreter(String name, String group, String className,
Expand All @@ -266,6 +267,7 @@ public RegisteredInterpreter(String name, String group, String className,
this.className = className;
this.defaultInterpreter = defaultInterpreter;
this.properties = properties;
this.editor = new HashMap<>();
}

public String getName() {
Expand All @@ -292,6 +294,10 @@ public Map<String, InterpreterProperty> getProperties() {
return properties;
}

public Map<String, Object> getEditor() {
return editor;
}

public void setPath(String path) {
this.path = path;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -226,6 +226,9 @@ public void onMessage(NotebookSocket conn, String msg) {
case LIST_UPDATE_NOTEBOOK_JOBS:
unicastUpdateNotebookJobInfo(conn, messagereceived);
break;
case EDITOR_SETTING:
getEditorSetting(conn, notebook, messagereceived);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I understood we introduce this new message type to current WS api in order just to pass this information to the front-end, right?

I wonder if there is another, simpler way to do it? As we are not changing it from the front, there is much less value in broadcasting it though WS (that's what we use WS mostly for), so it makes me think that there must be a simpler way to just pass this information to the front..or do I miss something here?

What do you think?

break;
default:
break;
}
Expand Down Expand Up @@ -1259,7 +1262,7 @@ public void afterStatusChange(Job job, Status before, Status after) {
}

/**
* This callback is for praragraph that runs on RemoteInterpreterProcess
* This callback is for paragraph that runs on RemoteInterpreterProcess
* @param paragraph
* @param out
* @param output
Expand Down Expand Up @@ -1367,11 +1370,24 @@ public void onRemove(String interpreterGroupId, String name, String noteId, Stri
if (id.equals(interpreterGroupId)) {
broadcast(
note.id(),
new Message(OP.ANGULAR_OBJECT_REMOVE).put("name", name).put(
"noteId", noteId).put("paragraphId", paragraphId));
new Message(OP.ANGULAR_OBJECT_REMOVE)
.put("name", name)
.put("noteId", noteId)
.put("paragraphId", paragraphId));
}
}
}
}

private void getEditorSetting(NotebookSocket conn, Notebook notebook, Message fromMessage)
throws IOException {
String replName = (String) fromMessage.get("magic");
String noteId = getOpenNoteId(conn);
Note note = notebook.getNote(noteId);
Message resp = new Message(OP.EDITOR_SETTING);
resp.put("editor", note.getEditorSetting(replName));
conn.send(serializeMessage(resp));
return;
}
}

4 changes: 2 additions & 2 deletions zeppelin-web/bower.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
"angular-elastic": "~2.4.2",
"angular-elastic-input": "~2.2.0",
"angular-xeditable": "0.1.12",
"highlightjs": "^9.2.0",
"highlightjs": "^9.4.0",
"lodash": "~3.9.3",
"angular-filter": "~0.5.4",
"ngtoast": "~2.0.0",
Expand Down Expand Up @@ -59,7 +59,7 @@
"highlight.pack.js",
"styles/github.css"
],
"version": "8.4.0",
"version": "9.4.0",
"name": "highlightjs"
}
}
Expand Down
Loading