merge recent changes #1
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
### What is this PR for? This contribution would to introduce Neo4j Cypher intepreter and at the same time provides base APIs that enable other graph databases (or graph framworks such as GraphX or Giraph). ### What type of PR is it? [Feature] ### Todos * [x] - Provide base APIs to manage graph results (under: org.apache.zeppelin.interpreter.graph.*) * [x] - Create the Neo4j intepreter ### What is the Jira issue? [[ZEPPELIN-2761]](https://issues.apache.org/jira/browse/ZEPPELIN-2761) ### How should this be tested? Donwload and execute [Neo4j](https://neo4j.com/download/?ref=home) v3.x, you can also pull a [Docker image](https://neo4j.com/developer/docker/). In order to execute test cases, if you are running Java 7, you need to also provide an environment variable telling the tests where to find Java 8, because Neo4j-the-database needs it to run. ```bash export NEO4J_JAVA=<path/to/java/home> ``` Use this statement to create a dummy dataset ```bash %neo4j UNWIND range(1,100) as id CREATE (p:Person {id:id, name: "Name " + id, age: id % 3}) WITH collect(p) as people UNWIND people as p1 UNWIND range(1,10) as friend WITH p1, people[(p1.id + friend) % size(people)] as p2 CREATE (p1)-[:KNOWS {years: abs(p2.id - p2.id)}]->(p2) ``` Then you can write some simple queries like: ```bash %neo4j MATCH (p:Person)-[r:KNOWS]-(p1:Person) RETURN p, r, p1 LIMIT 10; ``` ```bash %neo4j MATCH (p:Person)-[r:KNOWS]-(p1:Person) RETURN p.id AS ID_A, p.name AS NAME_A, r.years AS YEARS, p1.id AS ID_B, p1.name AS NAME_B LIMIT 20; ``` ### Video  ### Questions: * Does the licenses files need update? **Yes** * Is there breaking changes for older versions? **No** * Does this needs documentation? **Yes** Author: conker84 <santand@gmail.com> Closes #2478 from conker84/master and squashes the following commits: 5fb97ce [conker84] review by felixcheung 18/09 1e3d611 [conker84] Review by felixcheung (09/08) aaf618e [conker84] Review by 1ambda 26/07 b5c2c66 [conker84] Added docs 9eb568d [conker84] Final behavior after discussion 2f88e98 [conker84] Moved business logic to the connection manager 8e4690e [conker84] Create connection manager class 35b4e29 [conker84] First commit
### What is this PR for? Fixed broken javascript source map. The coverage loader should be used only for test env. **that's really annoying because we can't set breakpoint and even can't see the original source code.** ### What type of PR is it? [Bug Fix] ### What is the Jira issue? [ZEPPELIN-2924](https://issues.apache.org/jira/browse/ZEPPELIN-2924) ### How should this be tested? 1. run the webapp as dev mode: `localhost:9000` 2. open the developer console and click any log message 3. check whether source is minified or not. 4. execute `yarn run test` in the terminal to check that coverage feature works well. ### Screenshots (if appropriate) #### Before   #### After   ### Questions: * Does the licenses files need update? - NO * Is there breaking changes for older versions? - NO * Does this needs documentation? - NO Author: 1ambda <1amb4a@gmail.com> Closes #2580 from 1ambda/ZEPPELIN-2924/fix-source-map-is-broken and squashes the following commits: 375e139 [1ambda] fix: Use coverage loader only for test env
…incipal used for FileSystemNotebookRepo ### What is this PR for? Trivial change for configuration name change. ### What type of PR is it? [Refactoring] ### Todos * [ ] - Task ### What is the Jira issue? * No jira created ### How should this be tested? * First time? Setup Travis CI as described on https://zeppelin.apache.org/contribution/contributions.html#continuous-integration * Strongly recommended: add automated unit tests for any new or changed behavior * Outline any manual steps to test the PR here. ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #2620 from zjffdu/minor_refactoring and squashes the following commits: c68ce76 [Jeff Zhang] [MINOR][Code Refactoring] Configuration name change for keytab and principal used for FileSystemNotebookRepo
### What is this PR for? This PR is trying to allow user to add custom http headers when calling livy rest api. User just need to specify `zeppelin.livy.http.headers` in livy interpreter setting ### What type of PR is it? [Feature] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-2953 ### How should this be tested? Outline the steps to test the PR here. ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #2600 from zjffdu/ZEPPELIN-2953 and squashes the following commits: 720d8d0 [Jeff Zhang] ZEPPELIN-2953 Allow custom http header for livy interpreter
### What is this PR for? 2 main changes. * Introduce `interpreter-parent` module, so that all the interpreter can reuse the plugin defined in `interpreter-parent` * Add new plugin for `copying interpreter-setting.json` to interpreter dir ### What type of PR is it? [ Improvement | Refactoring] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-2982 ### How should this be tested? * First time? Setup Travis CI as described on https://zeppelin.apache.org/contribution/contributions.html#continuous-integration * Strongly recommended: add automated unit tests for any new or changed behavior * Outline any manual steps to test the PR here. ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #2613 from zjffdu/ZEPPELIN-2982 and squashes the following commits: ccccd32 [Jeff Zhang] update doc 8f643a7 [Jeff Zhang] ZEPPELIN-2982. Copy interpreter-setting.json to interpreter dir
### What is this PR for? Hardcoded seed/key may cause security issue, so remove potential risk. ### What type of PR is it? [Improvement] ### Todos No. ### What is the Jira issue? No jira created. Author: Yanbo Liang <ybliang8@gmail.com> Closes #2621 from yanboliang/key and squashes the following commits: 2515283 [Yanbo Liang] Remove hardcoded key in zeppelinhub.
### What is this PR for? Follow up of #2577. Main changes on Interpreter * Add throw `InterpreterException` which is checked exception for the abstract methods of `Interpreter`, this would enforce the interpreter implementation to throw `InterpreterException`. * field name refactoring. * `property` -> `properties` * `getProperty()` --> `getProperties()` * Introduce launcher layer for interpreter launching. Currently we only use shell script to launch interpreter, but it could be any other service or component to launch interpreter, such as livy server , other 3rd party tools or even we may create a separate module for interpreter launcher * abstract cass `InterpreterLauncher` * For now, only 2 implementation: `ShellScriptLauncher` & `SparkInterpreterLauncher`. We could add method in class `Interpreter` to allow interpreter to specify its own launcher class, but it could be future work. ### What type of PR is it? [Improvement | Refactoring] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-2685 ### How should this be tested? Unit test is covered. `ShellScriptLauncherTest` & `SparkInterpreterLauncherTest` ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #2592 from zjffdu/ZEPPELIN-2685 and squashes the following commits: 17dc2f1 [Jeff Zhang] address comments e545cc3 [Jeff Zhang] ZEPPELIN-2685. Improvement on Interpreter class
### What is this PR for? Make Zeppelin support Ceph as a notebook storage. Ceph has APIs which are compatible with AWS S3 APIs. However, it supports only AWS Signature V2 and GetObject requests of aws-sdk-java use V4 by default: aws/aws-sdk-java#372 According to aws/aws-sdk-java#372 (comment) , the Zeppelin configuration of `zeppelin.notebook.s3.signerOverride` is added to make the `signerOverride` field of a `ClientConfiguration` instance configurable. ### What type of PR is it? [Improvement] ### Todos ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-2950 ### How should this be tested? Tested manually. ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Keiji Yoshida <kjmrknsn@gmail.com> Closes #2598 from kjmrknsn/feature/ceph-support and squashes the following commits: c66cb63 [Keiji Yoshida] Support Ceph as a notebook storage
### What is this PR for? Trivial fix for matplotlib fail in `IPythonInterpreter`. Besides that, I also make some minor changes on zeppelin code to improve logging. ### What type of PR is it? [Bug Fix] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-2990 ### How should this be tested? UT is added. ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #2622 from zjffdu/ZEPPELIN-2990 and squashes the following commits: 55e6f88 [Jeff Zhang] fix unit test 1e57afe [Jeff Zhang] ZEPPELIN-2990. Matplotlib sometimes fails in IPythonInterpreter
### What is this PR for? interpreter.cmd works incorrect after #2562 ### What type of PR is it? Bug Fix ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: tinkoff-dwh <tinkoff.dwh@gmail.com> Closes #2607 from tinkoff-dwh/master and squashes the following commits: 1932202 [tinkoff-dwh] Merge remote-tracking branch 'upstream/master' c7f5124 [tinkoff-dwh] Merge remote-tracking branch 'upstream/master' 8baa87b [tinkoff-dwh] [HOTFIX] fix interpreter.cmd script
### What is this PR for? The tooltip for "List of shortcut" should read "List of shortcuts" (plural). ### What type of PR is it? Bug Fix ### Todos * N/A ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-2985 ### How should this be tested? * Hover over tooltip. Ensure that tooltip contains changes and behaves like a tooltip. (Shows up then hides based on cursor position) ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Gabe Fernando <gef756@gmail.com> Closes #2617 from gef756/dev-kb-tooltip and squashes the following commits: b0365de [Gabe Fernando] ENH: correct grammar in keyboard tooltip
### What is this PR for? Fixed the bug mentioned in #2554 (comment) ### What type of PR is it? [Bug Fix] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-2998 ### How should this be tested? * Unit test is added ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #2626 from zjffdu/ZEPPELIN-2998 and squashes the following commits: cc11fb6 [Jeff Zhang] ZEPPELIN-2998. Fix bug in restarting interpreter in scoped mode
…session application is killed ### What is this PR for? This PR will display a warning message when the livy session is dead. It won't restart it automatically, because livy session may be dead due to user code (like driver OOM). Restarting it automatically will hide the root cause and confuse users. ### What type of PR is it? [Improvement] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-2988 ### How should this be tested? * Manually tested ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need update? * Is there breaking changes for older versions? * Does this needs documentation? Author: Jeff Zhang <zjffdu@apache.org> Closes #2625 from zjffdu/ZEPPELIN-2988 and squashes the following commits: 0e4d281 [Jeff Zhang] ZEPPELIN-2988. Zeppelin does not start new Livy session if yarn livy session application is killed
### What is this PR for? This PR will add tab as auto complete invoker if paragraph is non-md and user has not pressed the tab as a first character in the line ### What type of PR is it? [Improvement] ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-277 * https://issues.apache.org/jira/browse/ZEPPELIN-2736 ### How should this be tested? - Build: mvn clean package -Denforcer.skip -DskipTests -Drat.skip - Open a paragraph - Press tab with following options: first character, after space ### Questions: * Does the licenses files need an update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Malay Majithia <malay.majithia@gmail.com> Author: Lee moon soo <moon@apache.org> Closes #2542 from malayhm/ZEPPELIN-277 and squashes the following commits: 436f22d [Malay Majithia] Added Tab auto completion flag for python sql and r b37e084 [Malay Majithia] Fixed lint error 18fc814 [Malay Majithia] Merge branch 'master' into ZEPPELIN-277 b09730e [Malay Majithia] Merge branch 'master' into ZEPPELIN-277 63d69e1 [Malay Majithia] Merge branch 'ZEPPELIN-277' of github.com:malayhm/zeppelin into ZEPPELIN-277 a75f0fe [Malay Majithia] Improved the first character check logic 2ec879d [Malay Majithia] Merge pull request #1 from Leemoonsoo/ZEPPELIN-277-completion-key 77afdba [Lee moon soo] fix style 77b47b6 [Malay Majithia] If all the previous line characters are tab, don't show autocomplete on tab 46f612a [Malay Majithia] ZEPPELIN-277 Add Tab as Autocomplete for Notebook non-md paragraphs 865c0a6 [Lee moon soo] Set python and spark interpreter completionKey 05d5860 [Lee moon soo] Update doc 973068b [Lee moon soo] apply tab completion based on editor.completionKey 5f4d81c [Malay Majithia] If all the previous line characters are tab, don't show autocomplete on tab 655ba88 [Malay Majithia] ZEPPELIN-277 Add Tab as Autocomplete for Notebook non-md paragraphs
### What is this PR for? Allow an override of default Spark logic in building the URL path and redirect the Zeppelin user directly to a configured reverse proxy path ### What type of PR is it? [Improvement] ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-2949 ### How should this be tested? * Set "zeppelin.spark.uiWebUrl" to http://localhost/spark-reverse-proxy-path * Run Spark code * Open Spark UI * Expected to be redirected to http://localhost/spark-reverse-proxy-path ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? N * Is there breaking changes for older versions? N * Does this needs documentation? N Author: Nelson Costa <nelson.costa85@gmail.com> Closes #2596 from necosta/zeppelin2949 and squashes the following commits: 2ae8f2d [Nelson Costa] [ZEPPELIN-2949] Last amendments 118339f [Nelson Costa] [ZEPPELIN-2949] Updated documentation 826bad4 [Nelson Costa] [ZEPPELIN-2949] Updated documentation 0a56299 [Nelson Costa] [ZEPPELIN-2949] Allow custom Spark UI URL
### What is this PR for? Fix build interpreters (apark and R) which broken after #2592 and #2596 ### What type of PR is it? [Hot Fix] ### How should this be tested? build interpreters ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: tinkoff-dwh <tinkoff.dwh@gmail.com> Closes #2630 from tinkoff-dwh/fix_r_interpretator and squashes the following commits: 0c9828b [tinkoff-dwh] [HOTFIX] fix build spark and R interpreters
### What is this PR for? This is for the bug hotfix introduced in #2592 . The issue is that new interpreter created can not run properly because the incorrect interpreter dir. Thanks tinkoff-dwh for reporting this issue. ### What type of PR is it? [ Hot Fix] ### Todos * [ ] - Task ### How should this be tested? * First time? Setup Travis CI as described on https://zeppelin.apache.org/contribution/contributions.html#continuous-integration * Strongly recommended: add automated unit tests for any new or changed behavior * Outline any manual steps to test the PR here. ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #2632 from zjffdu/HotFix_Interpreter and squashes the following commits: 2d7ab03 [Jeff Zhang] [HotFix] - Incorrect interpreter dir
### What is this PR for?
Fix converter for forms
### What type of PR is it?
[Bug Fix]
### How should this be tested?
jdbc
1. `select '${checkbox:name=1|2,first|second}'`
### Screenshots (if appropriate)
before
after
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: tinkoff-dwh <tinkoff.dwh@gmail.com>
Closes #2623 from tinkoff-dwh/forms-fix and squashes the following commits:
fdb6cfe [tinkoff-dwh] [FIX] checkstyle
08cf921 [tinkoff-dwh] [FIX] checkstyle
b841550 [tinkoff-dwh] [FIX] refactoring of context converter. add test to check converter of contexts.
17ec2db [tinkoff-dwh] [FIX] fix converter forms to json
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
11 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What is this PR for?
A few sentences describing the overall goals of the pull request's commits.
First time? Check out the contributing guide - https://zeppelin.apache.org/contribution/contributions.html
What type of PR is it?
[Bug Fix | Improvement | Feature | Documentation | Hot Fix | Refactoring]
Todos
What is the Jira issue?
How should this be tested?
Screenshots (if appropriate)
Questions: