-
Notifications
You must be signed in to change notification settings - Fork 2.8k
[ZEPPELIN-732] Helium Application #836
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from 86 commits
Commits
Show all changes
87 commits
Select commit
Hold shift + click to select a range
7424af2
Remove resource on note / paragraph removal
Leemoonsoo 568ee54
ApplicationLoader
Leemoonsoo b891b98
HeliumRegistry
Leemoonsoo b239f1b
Helium application factory
Leemoonsoo 9f5c493
Application output
Leemoonsoo 4eaeea7
sync -> async api
Leemoonsoo 7aeb64a
Unload app on paragraph / note removal as well as interpreter unbind
Leemoonsoo 134bbe6
Change HeliumRegistry constructor argument type
Leemoonsoo 94b490d
initial rest api impl
Leemoonsoo bd0f467
Style
Leemoonsoo 6223cd4
App output display
Leemoonsoo 16f6887
Angular object update for helium app
Leemoonsoo 412480a
Fix style
Leemoonsoo be3a1fa
Add license header
Leemoonsoo 5503f9c
Improved
Leemoonsoo b47ca74
Fix tests
Leemoonsoo 98f3872
Managed interpreter process and Running interpreter process
Leemoonsoo 024d7fc
Dev mode
Leemoonsoo 71f814d
Better way to find resource dir for InterpreterOutput watcher
Leemoonsoo f2ab95d
Prevent unnecessary output update
Leemoonsoo b4ff52f
Put last value of scala repl into resource pool
Leemoonsoo ec2fdea
Match classname correctly
Leemoonsoo 0e4d81c
Remove unnecessary log
Leemoonsoo fade3c1
Handle output update in angular mode
Leemoonsoo 03be3a1
Pass required resource to run() method
Leemoonsoo c30f53c
null check
Leemoonsoo 864bea1
Merge branch 'master' into ZEPPELIN-732-up
Leemoonsoo 1f1a3b5
Fix test
Leemoonsoo 01a1646
Add example
Leemoonsoo 620f79f
Merge branch 'master' into ZEPPELIN-732-up
Leemoonsoo 0730ece
Add license into package json
Leemoonsoo 3b891de
Provide resource pool for testing
Leemoonsoo 83eba98
UnloadApp on interpreter restart
Leemoonsoo 51fc113
Include examples in CI build
Leemoonsoo ab1de03
Merge branch 'master' into ZEPPELIN-732-up
Leemoonsoo 7451479
Merge branch 'master' into ZEPPELIN-732-up
Leemoonsoo f07ada1
Construct classpath correctly
Leemoonsoo 8186daf
Update Application Status from RemoteInterpreterProcess event
Leemoonsoo b6e4141
helper for printing javascript and inject $z
Leemoonsoo b068065
Replace . to _
Leemoonsoo 5225551
let instead of var
Leemoonsoo b974eb1
Add horizontal bar visualization example
Leemoonsoo f782fb0
clean pom.xml file
Leemoonsoo e89177e
Merge branch 'master' into ZEPPELIN-732-up
Leemoonsoo 99dd0b0
Merge branch 'master' into ZEPPELIN-732-up
Leemoonsoo 93fa766
Merge branch 'master' into ZEPPELIN-732-up
Leemoonsoo e0a2046
Add annotation to api
Leemoonsoo e36d2fb
Merge branch 'master' into ZEPPELIN-732-up
Leemoonsoo 1dc4409
Remove incubating from example pom.xml
Leemoonsoo bfcdf34
Fix style
Leemoonsoo 0671dfd
Fix compile error
Leemoonsoo 7f174e3
Add more comments
Leemoonsoo e60cd8e
Add docs
Leemoonsoo bff2217
Fix SparkParagraphIT
Leemoonsoo 76de11c
Merge branch 'master' into ZEPPELIN-732-up
Leemoonsoo 14efb6c
Merge branch 'master' into ZEPPELIN-732-up
Leemoonsoo 9110897
arrange buttons
Leemoonsoo b6811a6
set SPARK_HOME to find bin/interpreter.sh
Leemoonsoo 3619afd
Exclude test data file from rat check
Leemoonsoo 6c5019e
Merge branch 'master' into ZEPPELIN-732-up
Leemoonsoo 710487b
fix syntax error
Leemoonsoo 4571781
double quote the keys in json
Leemoonsoo 6f1343f
add license header
Leemoonsoo 88dbc71
change style
Leemoonsoo e54d633
Merge branch 'master' into ZEPPELIN-732-up
Leemoonsoo 5bef173
Update NoteTest
Leemoonsoo 6a5a19e
Merge branch 'master' into ZEPPELIN-732-up
Leemoonsoo 644add4
Takecare csv/tsv download button position after merge
Leemoonsoo ac963ff
Remove unnecessary stacktrace and provide more meaningful message
Leemoonsoo 385dab3
Fix xpath in the SparkParagraphIt test
Leemoonsoo 562f083
CLose interpreter after test
Leemoonsoo 1bdea70
Print error message in the test
Leemoonsoo 5b5f6c8
trigger ci
Leemoonsoo eaa68c6
print processes
Leemoonsoo f999f71
Update getParagraphXpath
Leemoonsoo d43ab57
Merge branch 'master' into ZEPPELIN-732-up
Leemoonsoo 0467077
update test
Leemoonsoo d558591
Merge branch 'master' into ZEPPELIN-732-up
Leemoonsoo c714a19
Merge branch 'master' into ZEPPELIN-732-up
Leemoonsoo 4e98cfe
Update DependencyResolver
Leemoonsoo 4030fff
Merge branch 'master' into ZEPPELIN-732-up
Leemoonsoo 3249ddb
Remove incubating
Leemoonsoo 6785441
interpreter order changed
Leemoonsoo e1b0e79
Update unittest
Leemoonsoo 83b529c
Update unittest
Leemoonsoo 0665380
Merge branch 'master' into ZEPPELIN-732-up
Leemoonsoo 3577777
Address Ahyoung's comment
Leemoonsoo File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,185 @@ | ||
| --- | ||
| layout: page | ||
| title: "Writing Zeppelin Application" | ||
| description: "" | ||
| group: development | ||
| --- | ||
| <!-- | ||
| Licensed under the Apache License, Version 2.0 (the "License"); | ||
| you may not use this file except in compliance with the License. | ||
| You may obtain a copy of the License at | ||
|
|
||
| http://www.apache.org/licenses/LICENSE-2.0 | ||
|
|
||
| Unless required by applicable law or agreed to in writing, software | ||
| distributed under the License is distributed on an "AS IS" BASIS, | ||
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| See the License for the specific language governing permissions and | ||
| limitations under the License. | ||
| --> | ||
| {% include JB/setup %} | ||
|
|
||
| ### What is Zeppelin Application (Experimental) | ||
|
|
||
| Zeppelin Application is a package that runs on Interpreter process and display it's output inside of the notebook. While application runs on Interpreter process, it's able to access resources provided by Interpreter through ResourcePool. Output is always rendered by AngularDisplaySystem. Therefore application provides all the possiblity of making interactive graphical application that uses data and processing power of any Interpreter. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
|
|
||
|
|
||
|
|
||
| ### Writing your own Application | ||
|
|
||
| Writing Application means extending `org.apache.zeppelin.helium.Application`. You can use your favorite IDE and language while Java class files are packaged into jar. `Application` class looks like | ||
|
|
||
| ```java | ||
|
|
||
| /** | ||
| * Constructor. Invoked when application is loaded | ||
| */ | ||
| public Application(ApplicationContext context); | ||
|
|
||
| /** | ||
| * Invoked when there're (possible) updates in required resource set. | ||
| * i.e. invoked after application load and after paragraph finishes. | ||
| */ | ||
| public abstract void run(ResourceSet args); | ||
|
|
||
| /** | ||
| * Invoked before application unload. | ||
| * Application is automatically unloaded with paragraph/notebook removal | ||
| */ | ||
| public abstract void unload(); | ||
| ``` | ||
|
|
||
|
|
||
| You can check example applications under [./zeppelin-examples](https://github.com/apache/incubator-zeppelin/tree/master/zeppelin-examples) directory. | ||
|
|
||
|
|
||
| ### Development mode | ||
|
|
||
| In the development mode, you can run your Application in your IDE as a normal java application and see the result inside of Zeppelin notebook. | ||
|
|
||
| org.apache.zeppelin.interpreter.dev.ZeppelinApplicationDevServer can run Zeppelin Application in development mode. | ||
|
|
||
| ```java | ||
|
|
||
| // entry point for development mode | ||
| public static void main(String[] args) throws Exception { | ||
|
|
||
| // add resources for development mode | ||
| LocalResourcePool pool = new LocalResourcePool("dev"); | ||
| pool.put("date", new Date()); | ||
|
|
||
| // run application in devlopment mode with give resource | ||
| // in this case, Clock.class.getName() will be the application class name | ||
| ZeppelinApplicationDevServer devServer = new ZeppelinApplicationDevServer( | ||
| Clock.class.getName(), | ||
| pool.getAll()); | ||
|
|
||
| // start development mode | ||
| devServer.start(); | ||
| devServer.join(); | ||
| } | ||
| ``` | ||
|
|
||
|
|
||
| In the Zeppelin notebook, run `%dev run` will connect to application running in development mode. | ||
|
|
||
|
|
||
|
|
||
|
|
||
| ### Package flie | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
|
|
||
| Package file is a json file that provides information about the application. | ||
| Json file contains following informations | ||
|
|
||
| ``` | ||
| { | ||
| name : "[organization].[name]", | ||
| description : "Description", | ||
| artifact : "groupId:artifactId:version", | ||
| className : "your.package.name.YourApplicationClass", | ||
| resources : [ | ||
| ["resource.name", ":resource.class.name"], | ||
| ["alternative.resource.name", ":alternative.class.name"] | ||
| ], | ||
| icon : "<i class="icon"></i>" | ||
| } | ||
|
|
||
| ``` | ||
|
|
||
| #### name | ||
|
|
||
| Name is a string in '[group].[name]' format. | ||
| [group] and [name] allows only [A-Za-z0-9_]. | ||
| Group is normally organization name who creates this application. | ||
|
|
||
| #### description | ||
|
|
||
| Short description. about application | ||
|
|
||
| #### artifact | ||
|
|
||
| Location of the jar artifact. | ||
| "groupId:artifactId:version" will make load artifact from maven repository. | ||
| If jar is in local filesystem, absolute/relative can be used. | ||
|
|
||
| e.g. | ||
|
|
||
| When artifact exists in Maven repository | ||
|
|
||
| `artifact: "org.apache.zeppelin:zeppelin-examples:0.6.0"` | ||
|
|
||
| When artifact exists in local filesystem | ||
|
|
||
| `artifact: "zeppelin-example/target/zeppelin-example-0.6.0.jar"` | ||
|
|
||
|
|
||
| #### className | ||
|
|
||
| Entry point. Class that extends `org.apache.zeppelin.helium.Application` | ||
|
|
||
|
|
||
| #### resources | ||
|
|
||
| Two dimensional array that defines required resources by name or by className. Helium Application launcher will compare resources in the ResourcePool with informations in this field and suggest application only when all required resources are available in the ResourcePool. | ||
|
|
||
| Resouce name is a string which will be compared with name of objects in the ResourcePool. className is a string with ":" prepended, which will be compared with className of the objects in the ResourcePool. | ||
|
|
||
| Application may require two or more resources. Required resource can be listed inside of json array. For example, if application requires object "name1", "name2" and "className1" type of object to run, resources field can be | ||
|
|
||
| ``` | ||
| resources: [ | ||
| [ "name1", "name2", ":className1", ...] | ||
| ] | ||
| ``` | ||
|
|
||
| If Application can handle alternative combination of required resource, alternative set can be listed as below. | ||
|
|
||
| ``` | ||
| resources: [ | ||
| [ "name", ":className"], | ||
| [ "altName", ":altClassName1"], | ||
| ... | ||
| ] | ||
| ``` | ||
|
|
||
| Easier way of understanding this scheme is | ||
|
|
||
| ``` | ||
| resources: [ | ||
| [ 'resource' AND 'resource' AND ... ] OR | ||
| [ 'resource' AND 'resource' AND ... ] OR | ||
| ... | ||
| ] | ||
| ``` | ||
|
|
||
|
|
||
| #### icon | ||
|
|
||
| Icon to be used on the application button. String in this field will be rendered as a html. | ||
|
|
||
| e.g. | ||
|
|
||
| ``` | ||
| icon: "<i class='fa fa-clock-o'></i>" | ||
| ``` | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Leemoonsoo How about changing this title to "What is Apache Zeppelin Application? (Experimental)" with using
<h1>instead of<h3>?