Merged
Conversation
…eter is not opened yet ### What is this PR for? The NPE happens because the scheduler of interpreter is null. This is because the interpreter is not opened yet. (https://github.com/apache/zeppelin/blob/v0.9.0-preview1/livy/src/main/java/org/apache/zeppelin/livy/LivySparkSQLInterpreter.java#L232) This PR would check wether interpreter is opened before using its scheduler. ### What type of PR is it? [Bug Fix ] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4829 ### How should this be tested? * CI pass ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #3776 from zjffdu/ZEPPELIN-4829 and squashes the following commits: 1203b7b [Jeff Zhang] [ZEPPELIN-4829]. NPE in RemoteInterpreterServer#getStatus when interpreter is not opened yet (cherry picked from commit 2608005) Signed-off-by: Jeff Zhang <zjffdu@apache.org>
… value ### What is this PR for? Trivial PR to set `zeppelin.spark.sql.stacktrace` to `true` as default value ### What type of PR is it? [Improvement] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4836 ### How should this be tested? * CI pass ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #3777 from zjffdu/ZEPPELIN-4836 and squashes the following commits: 547f138 [Jeff Zhang] [ZEPPELIN-4836]. set zeppelin.spark.sql.stacktrace to true as default value (cherry picked from commit 3034a44) Signed-off-by: Jeff Zhang <zjffdu@apache.org>
…properties between JVM and PVM ### What is this PR for? This PR will first set env `PYSPARK_PIN_THREAD` to be `true` and then suppress the warning message in `pyspark.util`. ### What type of PR is it? [Improvement ] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4802 ### How should this be tested? * CI pass * Manually tested ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? No * Does this needs documentation? Author: Jeff Zhang <zjffdu@apache.org> Closes #3778 from zjffdu/ZEPPELIN-4802 and squashes the following commits: d6a65cb [Jeff Zhang] [ZEPPELIN-4802]. pyspark warnings with spark-3.0.0-preview - Sync of properties between JVM and PVM (cherry picked from commit df86e50) Signed-off-by: Jeff Zhang <zjffdu@apache.org>
### What is this PR for? When updating for the new DataStax Java driver, I forgot to update template for a `HELP` command. This PR fixes this problem, and simplifies maintenance by reading driver version from the resource file generated during compilation, so HELP command will provide information based on the actual driver version used for building interpreter. ### What type of PR is it? Bug Fix ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-4792 ### How should this be tested? * https://travis-ci.org/github/alexott/zeppelin/builds/693123862 () * Tested manually Author: Alex Ott <alexott@gmail.com> Closes #3780 from alexott/ZEPPELIN-4792 and squashes the following commits: f62bc5c [Alex Ott] [ZEPPELIN-4792] Help needs to be updated in Cassandra interpreter (cherry picked from commit def30ee) Signed-off-by: Alex Ott <alexott@apache.org>
### What is this PR for? Trivial PR which just make `zeppelin.spark.concurrentSQL` to be `true` by default, this is what user expect usually. ### What type of PR is it? [Improvement] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4840 ### How should this be tested? CI pass ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Jeff Zhang <zjffdu@apache.org> Closes #3783 from zjffdu/ZEPPELIN-4840 and squashes the following commits: 7ece99b [Jeff Zhang] [ZEPPELIN-4840]. Set zeppelin.spark.concurrentSQL to be true by default (cherry picked from commit a50df08) Signed-off-by: Jeff Zhang <zjffdu@apache.org>
### What is this PR for? Fix a flaky test in SparQL interpreter ### What type of PR is it? Bug Fix ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4860 ### How should this be tested? * https://travis-ci.org/github/alexott/zeppelin/builds/695631580 Author: Alex Ott <alexott@gmail.com> Closes #3789 from alexott/ZEPPELIN-4860 and squashes the following commits: 19fb4ec [Alex Ott] [ZEPPELIN-4860] Non-determenistic results in SparqlJenaEngineTest (cherry picked from commit 50a3c3e) Signed-off-by: Alex Ott <alexott@apache.org>
### What is this PR for? This is a trivial PR which add `getInterpreterGroupId` to `InterpreterClient`, and use `interpreterGroupId` as the identifier of RemoteInterpreterProcess because `interpreterGroupId` is also unique identifier of InterpreterGroup ### What type of PR is it? [Improvement] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4852 ### How should this be tested? * CI pass ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #3785 from zjffdu/ZEPPELIN-4852 and squashes the following commits: 01ec893 [Jeff Zhang] address comment 4fc92cc [Jeff Zhang] address comment a482faa [Jeff Zhang] [ZEPPELIN-4852]. Add name to RemoteInterpreterProcess (cherry picked from commit 2ab8f9e) Signed-off-by: Jeff Zhang <zjffdu@apache.org>
…raph
### What is this PR for?
This PR add paragraph local property `printREPLOutput` for spark interpreter, so that user can switch for enabling Spark REPL output for each paragraph, e.g.
```
%spark(printREPLOutput=false)
System.out.println("hello ")
println("hello world")
```
### What type of PR is it?
[ Feature ]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-4848
### How should this be tested?
* CI pass
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zjffdu@apache.org>
Closes #3787 from zjffdu/ZEPPELIN-4848 and squashes the following commits:
b6997d9 [Jeff Zhang] [ZEPPELIN-4848]. Switch for enabling Spark REPL output for each paragraph
(cherry picked from commit 4d4d532)
Signed-off-by: Jeff Zhang <zjffdu@apache.org>
…spath directly ### What is this PR for? Previous plugin loading strategy is to load them from system classloader first, if fails, then fallback to plugin folder classloader. This would produce misleading logging which make user think that the plugin is failed to load. This PR would load plugin from system classloader only when they are builtin plugins. ### What type of PR is it? [Improvement] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4833 ### How should this be tested? * CI pass ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #3784 from zjffdu/ZEPPELIN-4833 and squashes the following commits: 62cd4db [Jeff Zhang] [ZEPPELIN-4833] misleading logging when fail to load plugin from classpath directly (cherry picked from commit 11780d3) Signed-off-by: Jeff Zhang <zjffdu@apache.org>
… process to zeppelin server ### What is this PR for? This is a improvement and refactoring PR which just use PooledObject for the connection from interpreter process to zeppelin server. Otherwise, once the connection is lost, interpreter process can never connect to zeppelin sever again. In this PR, I create class `PooledRemoteClient` which represent the pooled connection objects and this class will manage the life cycle of the connection between thrift client to thrift server. ### What type of PR is it? [ Improvement | Refactoring] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4859 ### How should this be tested? * CI pass ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #3788 from zjffdu/ZEPPELIN-4859 and squashes the following commits: 1c965ce [Jeff Zhang] [ZEPPELIN-4859]. Use PooledObject for the connection from interpreter process to zeppelin server (cherry picked from commit 7eb125d) Signed-off-by: Jeff Zhang <zjffdu@apache.org>
### What is this PR for? This PR is to recover the running paragraph when recovery is enabled. 1. Get all the running Interpreter processes from RecoveryStorage. 2. Reconnect to the running interpreter process. 3. Resubmit the paragraph to interpreter process. ### What type of PR is it? [ Feature] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4845 ### How should this be tested? * CI pass and manully tested the recovery scenairo. ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #3781 from zjffdu/ZEPPELIN-4845 and squashes the following commits: 2302471 [Jeff Zhang] [ZEPPELIN-4845]. Recover running paragraph when recovery is enabled (cherry picked from commit 446608f) Signed-off-by: Jeff Zhang <zjffdu@apache.org>
### What is this PR for? Although flink 1.11 is not released yet, it is already feature cut, so its api is stable now. This PR is to make flink interpreter of zeppelin to support flink 1.11. Due to api change of flink 1.11, this PR introduce flink-shim which will use different api for different version of flink. ### What type of PR is it? [Feature ] ### Todos * [ ] - Task ### What is the Jira issue? * https://jira.apache.org/jira/browse/ZEPPELIN-4816 ### How should this be tested? * CI pass and tested on flink 1.11 snapshot ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? no * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #3782 from zjffdu/ZEPPELIN-4816 and squashes the following commits: 6ce0652 [Jeff Zhang] [ZEPPELIN-4816]. Support Flink 1.11 (cherry picked from commit c1d6297) Signed-off-by: Jeff Zhang <zjffdu@apache.org>
### What is this PR for? Existing Travis-CI config contained references to previous distributions, etc. This PR fixes this. It also fixes a problem with Livy interpreter, where tests were failing because R wasn't enabled for them ### What type of PR is it? Bug Fix / Improvement ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4864 ### How should this be tested? * https://travis-ci.org/github/alexott/zeppelin/builds/696385230 Author: Alex Ott <alexott@apache.org> Closes #3793 from alexott/ZEPPELIN-4864 and squashes the following commits: 3fccb14 [Alex Ott] [ZEPPELIN-4864] enable R for Livy tests 91a30a3 [Alex Ott] [ZEPPELIN-4864] improve debugability of Livy integration test 8223344 [Alex Ott] [ZEPPELIN-4864] Cleanup Travis-CI config (cherry picked from commit 5cdb51b) Signed-off-by: Alex Ott <alexott@apache.org>
### What is this PR for? Minor PR which allow user to specify job name as paragraph local properties, by default it is the sql statement if user don't specify it. ### What type of PR is it? [Improvement] ### Todos * [ ] - Task ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-4865 ### How should this be tested? * CI pass and manually tested. ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #3792 from zjffdu/ZEPPELIN-4865 and squashes the following commits: 7c6e2f5 [Jeff Zhang] [ZEPPELIN-4865]. Allow specify jobName as paragraph local properties (cherry picked from commit 6bea350) Signed-off-by: Jeff Zhang <zjffdu@apache.org>
### What is this PR for? We can provide properties that are local to the paragraph, that could be used to pass an additional information for interpreter that could affect its behavior. Unfortunately existing parsing functionality relies on the fact that key/value pairs need to be separated by `,` character, and doesn't handle values with special characters (`,`, `=`, ...) inside, like this: ``` %cassandra(locale=ruRU, timeFormat="E, d MMM yy", floatPrecision = 5, outputFormat=cql) ``` This PR changes the parsing logic to perform character-by-character parsing, and handling of the quoted keys & values, escaping of the special characters, etc. ### What type of PR is it? Improvement ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4870 ### How should this be tested? * https://travis-ci.org/github/alexott/zeppelin/builds/697522260 * additional unit tests were added Author: Alex Ott <alexott@gmail.com> Closes #3799 from alexott/ZEPPELIN-4870 and squashes the following commits: 5fb6ee8 [Alex Ott] [ZEPPELIN-4870] Improve parsing of the paragraph properties (cherry picked from commit 1d1b058) Signed-off-by: Alex Ott <alexott@apache.org>
### What is this PR for? Tests didn't cover all of the edge cases, that lead to error when parsing specific message. Also, was found that parsing is also duplicated in the `Paragraph` class - it was replaced with call to `ParagraphTextParser`. The most visible change is that `%interpreter(some text` will know fail as there is no matching `)` character. But it's works the same if we change it to `%interpreter (some text`... ### What type of PR is it? Hot Fix ### What is the Jira issue? * ZEPPELIN-4870 ### How should this be tested? * Added one more test * https://travis-ci.org/github/alexott/zeppelin/builds/697944179 Author: Alex Ott <alexott@gmail.com> Closes #3800 from alexott/ZEPPELIN-4870-hotfix and squashes the following commits: 5942c5d [Alex Ott] [ZEPPELIN-4870][hotfix] fix edge case, and remove duplicate parser (cherry picked from commit d679bb1) Signed-off-by: Alex Ott <alexott@apache.org>
### What is this PR for? This PR is to support yarn interpreter launch mode, that means to launch the interpreter process in yarn container. So that we can mitigate the memory pressure of zeppelin server machine. This PR add new launcher module `yarn` for that. I manually tested shell, python, jdbc and flink interpreted. ### What type of PR is it? [Feature] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4844 ### How should this be tested? * CI pass and manually tested ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #3786 from zjffdu/ZEPPELIN-4844 and squashes the following commits: 899c4e1 [Jeff Zhang] [ZEPPELIN-4844]. Support yarn interpreter launch mode (cherry picked from commit df54df9) Signed-off-by: Jeff Zhang <zjffdu@apache.org>
Trivial PR which display rich duration info instead of just x seconds. See screenshot below. [ Improvement ] * [ ] - Task * https://issues.apache.org/jira/browse/ZEPPELIN-4873 * CI pass  * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #3794 from zjffdu/ZEPPELIN-4873 and squashes the following commits: 459f181 [Jeff Zhang] add java doc 419818e [Jeff Zhang] address comment 9691eae [Jeff Zhang] [ZEPPELIN-4873]. Display rich duration info for insert into flink job (cherry picked from commit 3139ed6) Signed-off-by: Jeff Zhang <zjffdu@apache.org>
…ding ### What is this PR for? Add property `flink.udf.jars.packages` to specify the packages that would be searched for udf, otherwise all the classes will be check, which might be very time consuming especially when the udf jars are large (if your udf depends on other third party libraries) ### What type of PR is it? [Feature ] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4837 ### How should this be tested? * CI pass and manually tested ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #3798 from zjffdu/ZEPPELIN-4837 and squashes the following commits: 78816e5 [Jeff Zhang] [ZEPPELIN-4837]. Add property to only check some packages for udf finding (cherry picked from commit 37953f4) Signed-off-by: Jeff Zhang <zjffdu@apache.org>
…-17788 ### What is this PR for? Flink interpreter doesn't work with flink 1.10.1 in yarn mode due to FLINK-17788, this PR fix it in zeppelin side. Because in scala shell, flink only support yarn session mode, so when it is yarn mode, we can set deployment target to be yarn-session directly. ### What type of PR is it? [Bug Fix ] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4878 ### How should this be tested? * CI pass and manually tested ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #3795 from zjffdu/ZEPPELIN-4878 and squashes the following commits: 498325c [Jeff Zhang] use flink1.10.version & flink1.11.version fb1a761 [Jeff Zhang] [ZEPPELIN-4878]. Unable to run flink 1.10.1 in yarn mode due to FLINK-17788 (cherry picked from commit 951da42) Signed-off-by: Jeff Zhang <zjffdu@apache.org>
### What is this PR for? This PR fix the bug in Splitter that text between 2 sql comment is mistaken as comment. Unit test is added in this PR as well. Besides that, this PR also remove the comment in the middle of sql. e.g. ``` select a -- comment 1 from table_1 ``` will be converted to ``` select a from table_1 ``` ### What type of PR is it? [Bug Fix] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4877 ### How should this be tested? * CI pass ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #3796 from zjffdu/ZEPPELIN-4877 and squashes the following commits: 0b0b728 [Jeff Zhang] address comment 6e9287e [Jeff Zhang] address comment 63afd6a [Jeff Zhang] [ZEPPELIN-4877]. text between 2 sql comment is mistaken as comment (cherry picked from commit badcaa0) Signed-off-by: Jeff Zhang <zjffdu@apache.org>
…he yarn address ### What is this PR for? Trivial PR which add property `flink.webui.yarn.yarnAddress` that allow user to replace the yarn address. This is for some cases that cloud vender would map the internal resource manager address to other url. e.g. some cloud vender will map `http://resource-manager:8088` to `https://xxx-yarn.yy.cn/gateway/kkk/yarn` ### What type of PR is it? [Improvement] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4851 ### How should this be tested? * CI pass ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Jeff Zhang <zjffdu@apache.org> Closes #3797 from zjffdu/ZEPPELIN-4851 and squashes the following commits: c1be762 [Jeff Zhang] [ZEPPELIN-4851]. Add property to flink interpreter to allow replace the yarn address (cherry picked from commit 2b505dc) Signed-off-by: Jeff Zhang <zjffdu@apache.org>
…reter ### What is this PR for? It is a trivial PR which add property `zeppelin.flink.module.enableHive` to control whether load hive udf first when there's same udf name in both hive and flink. ### What type of PR is it? [Improvement] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4885 ### How should this be tested? * CI pass, and manually tested. ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #3802 from zjffdu/ZEPPELIN-4885 and squashes the following commits: 4575458 [Jeff Zhang] [ZEPPELIN-4885]. Add property to specify Module order of flink interpreter (cherry picked from commit 72c127f) Signed-off-by: Jeff Zhang <zjffdu@apache.org>
### What is this PR for? Right now, when operation returns no results, for example, when executing most operations other than `SELECT` (DDL/INSERT/UPDATE/DELETE/...), interpreter shows the extended information - what hosts were used to execute command, etc. This information occupies a significant amount of the screen space and isn't critical for execution. Given PR provides the same information but folds it by default, saving the screen space. ### What type of PR is it? Improvement ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4756 ### How should this be tested? * https://travis-ci.org/github/alexott/zeppelin/builds/698161348 ### Screenshots (if appropriate) Old view:  New view:   Author: Alex Ott <alexott@gmail.com> Closes #3801 from alexott/ZEPPELIN-4756 and squashes the following commits: 3b0618c [Alex Ott] [ZEPPELIN-4756] Cassandra: Show "No Results" message folded (cherry picked from commit 84b6a1b) Signed-off-by: Alex Ott <alexott@apache.org>
### What is this PR for? Fix error logs in catch exception ### What type of PR is it? [Bug Fix ] ### Todos * [ ] - Task ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-4889 ### How should this be tested? ### Screenshots (if appropriate) ### Questions: No questions. Author: dijie <dijie@mogu.com> Closes #3804 from lonelyGhostisdog/ZEPPELIN-4889 and squashes the following commits: 636b698 [dijie] [ZEPPELIN-4889] [Flink] Error logs in catch exception (cherry picked from commit 9294062) Signed-off-by: Jeff Zhang <zjffdu@apache.org>
xiejiajun
added a commit
that referenced
this pull request
Jul 13, 2020
…Interpreter database connection pool ### What is this PR for? - Added support for personalized configuration of the JDBCInterpreter database connection pool. - This feature can be used to solve the runtime error caused by HiveServer2 Session timeout. - This connection pool configuration Compatible with [the official configuration of dbcp2 ](http://commons.apache.org/proper/commons-dbcp/configuration.html) ### What type of PR is it? [Feature] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4941 ### How should this be tested? * manually tested * Supported configuration items - validationQuery - testOnBorrow - testOnCreate - testOnReturn - testWhileIdle - timeBetweenEvictionRunsMillis - maxWaitMillis - maxIdle - minIdle - maxTotal ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? NO * Is there breaking changes for older versions? NO * Does this needs documentation? NO Author: xiejiajun <xiejiajun02@163.com> Author: JakeXie <xiejiajun02@163.com> Author: xie-jia-jun <xiejiajun02@163.com> Closes apache#3844 from xiejiajun/jdbc-pool-config and squashes the following commits: 3bb622b [xiejiajun] use html table display datasource pool configuration in the jdbc.md document 88b1ce0 [xiejiajun] jdbc interpreter datasource connection pool document 1f2eb55 [xiejiajun] Compatible with the official configuration of dbcp2 connection pool 5c7f666 [xiejiajun] JDBCInterprete supports personalized database connection pool configuration to cope with scenarios such as HiveSession timeout 3d57719 [xiejiajun] Merge branch 'branch-0.9' of https://github.com/apache/zeppelin into apache-0.9 0e5f65a [xiejiajun] Merge branch 'branch-0.9' of https://github.com/apache/zeppelin into apache-0.9 076668a [JakeXie] Merge pull request #7 from apache/branch-0.9 00e14dd [JakeXie] Merge pull request #5 from apache/branch-0.9 9bb7341 [xiejiajun] Merge remote-tracking branch 'origin/branch-0.9' into branch-0.9 9bc5605 [xiejiajun] bug fix: when we removing a paragraph , an interpreter process refused to connect due to it abnormal stop , which eventually caused the remove paragraph operation to fail to complete. 97d2714 [xie-jia-jun] Merge pull request #2 from apache/branch-0.9 9b3c744 [xiejiajun] added timeout for getting Thrift client to avoid situations where the interpreter may not be restarted when the interpreter process exits unexpectedly
xiejiajun
added a commit
that referenced
this pull request
Jul 21, 2020
…Interpreter database connection pool ### What is this PR for? - Added support for personalized configuration of the JDBCInterpreter database connection pool. - This feature can be used to solve the runtime error caused by HiveServer2 Session timeout. - This connection pool configuration Compatible with [the official configuration of dbcp2 ](http://commons.apache.org/proper/commons-dbcp/configuration.html) ### What type of PR is it? [Feature] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4941 ### How should this be tested? * manually tested * Supported configuration items - validationQuery - testOnBorrow - testOnCreate - testOnReturn - testWhileIdle - timeBetweenEvictionRunsMillis - maxWaitMillis - maxIdle - minIdle - maxTotal ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? NO * Is there breaking changes for older versions? NO * Does this needs documentation? NO Author: xiejiajun <xiejiajun02@163.com> Author: JakeXie <xiejiajun02@163.com> Author: xie-jia-jun <xiejiajun02@163.com> Closes apache#3844 from xiejiajun/jdbc-pool-config and squashes the following commits: 3bb622b [xiejiajun] use html table display datasource pool configuration in the jdbc.md document 88b1ce0 [xiejiajun] jdbc interpreter datasource connection pool document 1f2eb55 [xiejiajun] Compatible with the official configuration of dbcp2 connection pool 5c7f666 [xiejiajun] JDBCInterprete supports personalized database connection pool configuration to cope with scenarios such as HiveSession timeout 3d57719 [xiejiajun] Merge branch 'branch-0.9' of https://github.com/apache/zeppelin into apache-0.9 0e5f65a [xiejiajun] Merge branch 'branch-0.9' of https://github.com/apache/zeppelin into apache-0.9 076668a [JakeXie] Merge pull request #7 from apache/branch-0.9 00e14dd [JakeXie] Merge pull request #5 from apache/branch-0.9 9bb7341 [xiejiajun] Merge remote-tracking branch 'origin/branch-0.9' into branch-0.9 9bc5605 [xiejiajun] bug fix: when we removing a paragraph , an interpreter process refused to connect due to it abnormal stop , which eventually caused the remove paragraph operation to fail to complete. 97d2714 [xie-jia-jun] Merge pull request #2 from apache/branch-0.9 9b3c744 [xiejiajun] added timeout for getting Thrift client to avoid situations where the interpreter may not be restarted when the interpreter process exits unexpectedly
xiejiajun
added a commit
that referenced
this pull request
Jul 21, 2020
…Spark Interpreter Scala REPL ### What is this PR for? - Support for manually specifying the Java version of Spark Interpreter Scala REPL. - This feature can be used to resolve runtime errors caused by reference to third-party libraries that use high-version Java in Spark Interpreter. ### What type of PR is it? [Feature] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/projects/ZEPPELIN/issues/ZEPPELIN-4950 ### How should this be tested? * manually tested * Specify spark.repl.target through %spark.conf or when configuring Spark Interpreter * Reference and call a Java method in a third-party library developed using Java 8 and using Java 8 features such as interface static methods through the Spark interpreter. If the call is successful, it means the test passed ### Screenshots (if appropriate) - A runtime error will be triggered when we reference a third-party library that uses the new features of Java8 if we using the default value `jvm-1.6` of `settings.target.value` :  ### Questions: * Does the licenses files need update? NO * Is there breaking changes for older versions? NO * Does this needs documentation? Yes Author: xiejiajun <xiejiajun02@163.com> Author: JakeXie <xiejiajun02@163.com> Author: xie-jia-jun <xiejiajun02@163.com> Closes apache#3852 from xiejiajun/spark_repl_target and squashes the following commits: d131f19 [xiejiajun] spark.md docs fix 6872054 [xiejiajun] add spark.repl.target to spark.md docs 6a05b4f [xiejiajun] Support for manually specifying the Java version of Spark Interpreter Scala REPL 6a2cf04 [xiejiajun] Merge branch 'branch-0.9' of https://github.com/apache/zeppelin into branch-0.9 076668a [JakeXie] Merge pull request #7 from apache/branch-0.9 00e14dd [JakeXie] Merge pull request #5 from apache/branch-0.9 9bb7341 [xiejiajun] Merge remote-tracking branch 'origin/branch-0.9' into branch-0.9 9bc5605 [xiejiajun] bug fix: when we removing a paragraph , an interpreter process refused to connect due to it abnormal stop , which eventually caused the remove paragraph operation to fail to complete. 97d2714 [xie-jia-jun] Merge pull request #2 from apache/branch-0.9 9b3c744 [xiejiajun] added timeout for getting Thrift client to avoid situations where the interpreter may not be restarted when the interpreter process exits unexpectedly (cherry picked from commit 054651f) Signed-off-by: Jeff Zhang <zjffdu@apache.org>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is this PR for?
A few sentences describing the overall goals of the pull request's commits.
First time? Check out the contributing guide - https://zeppelin.apache.org/contribution/contributions.html
What type of PR is it?
[Bug Fix | Improvement | Feature | Documentation | Hot Fix | Refactoring]
Todos
What is the Jira issue?
How should this be tested?
Screenshots (if appropriate)
Questions: