Skip to content

[ZEPPELIN-1379] Flink interpreter is missing scala libraries#1370

Closed
lresende wants to merge 1 commit intoapache:masterfrom
lresende:flink-dependencies
Closed

[ZEPPELIN-1379] Flink interpreter is missing scala libraries#1370
lresende wants to merge 1 commit intoapache:masterfrom
lresende:flink-dependencies

Conversation

@lresende
Copy link
Member

What is this PR for?

On Flink interpreter, remove provided scope from scala libraries to enable copying them to interpreter location.

What type of PR is it?

[Bug Fix]

What is the Jira issue?

ZEPPELIN-1379

@lresende
Copy link
Member Author

After the fix, performed a full build and compared scala libraries from flink and ignite

$ cd flink/
$ ls scala*
-rw-r--r-- 1 lresende staff 14M Aug 26 12:39 scala-compiler-2.10.5.jar
-rw-r--r-- 1 lresende staff 6.8M Aug 26 12:39 scala-library-2.10.5.jar
-rw-r--r-- 1 lresende staff 3.1M Aug 26 12:39 scala-reflect-2.10.5.jar
$ cd ..
$ cd ignite/
$ ls scala*
-rw-r--r-- 1 lresende staff 14M Aug 26 12:39 scala-compiler-2.10.5.jar
-rw-r--r-- 1 lresende staff 6.8M Aug 26 12:39 scala-library-2.10.5.jar
-rw-r--r-- 1 lresende staff 3.1M Aug 26 12:39 scala-reflect-2.10.5.jar

@lresende lresende changed the title [FLINK-1379] Flink interpreter is missing scala libraries [ZEPPELIN-1379] Flink interpreter is missing scala libraries Aug 26, 2016
@lresende lresende closed this Aug 27, 2016
@lresende lresende reopened this Aug 27, 2016
@lresende lresende force-pushed the flink-dependencies branch 2 times, most recently from 8084055 to a5b62ce Compare August 27, 2016 03:47
Remove provided scope from scala libraries to enable copying
them to interpreter location.
@lresende lresende force-pushed the flink-dependencies branch from a5b62ce to 8e2ea20 Compare August 28, 2016 00:00
@bzz
Copy link
Member

bzz commented Aug 29, 2016

Looks good to me, thank you @lresende

BTW, from my experience, postin CI failure helps to speed up a review process.

CI fails on Spark 2.0 profile \w ZeppelinSparkClusterTest.zRunTest which looks like a flaky test, not relevant to the changes.

[INFO] Zeppelin: Server ................................... FAILURE [04:11 min]

Failed tests: 
  ZeppelinSparkClusterTest.zRunTest:204 expected:<FINISHED> but was:<ERROR>

00:32:54,193  INFO org.apache.zeppelin.interpreter.InterpreterFactory:665 - Create interpreter instance md for note 2BW956553
00:32:54,193  INFO org.apache.zeppelin.interpreter.InterpreterFactory:696 - Interpreter org.apache.zeppelin.markdown.Markdown 578260863 created
00:32:54,194  INFO org.apache.zeppelin.notebook.Paragraph:295 - run paragraph 20160828-003254_783795023 using md org.apache.zeppelin.interpreter.LazyOpenInterpreter@22778f7f
00:32:54,195  INFO org.apache.zeppelin.interpreter.remote.RemoteInterpreterManagedProcess:116 - Run interpreter process [..//bin/interpreter.sh, -d, ../interpreter/md, -p, 46521, -l, ..//local-repo/2BUN9SMBJ]
00:32:54,195  INFO org.apache.zeppelin.socket.NotebookServer:115 - New connection from null : 0
00:32:54,196  INFO org.apache.zeppelin.socket.NotebookServer:115 - New connection from null : 0
00:32:54,198  INFO org.apache.zeppelin.socket.NotebookServer:529 - New operation from null : 0 : anonymous : GET_NOTE : 2BW956553
00:32:54,209  INFO org.apache.zeppelin.socket.NotebookServer:529 - New operation from null : 0 : anonymous : GET_NOTE : 2BW956553
00:32:54,210 ERROR org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess:234 - Remote interpreter process not started
java.lang.NullPointerException
    at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.updateRemoteAngularObject(RemoteInterpreterProcess.java:229)
    at org.apache.zeppelin.interpreter.remote.RemoteAngularObject.set(RemoteAngularObject.java:48)
    at org.apache.zeppelin.interpreter.remote.RemoteAngularObject.set(RemoteAngularObject.java:40)
    at org.apache.zeppelin.socket.NotebookServer.angularObjectUpdated(NotebookServer.java:841)
    at org.apache.zeppelin.socket.NotebookServer.onMessage(NotebookServer.java:221)
    at org.apache.zeppelin.socket.NotebookServerTest.testMakeSureNoAngularObjectBroadcastToWebsocketWhoFireTheEvent(NotebookServerTest.java:135)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

00:32:54,216 ERROR org.apache.zeppelin.interpreter.remote.RemoteAngularObjectRegistry:111 - Error
java.lang.NullPointerException
    at org.apache.zeppelin.interpreter.remote.RemoteAngularObjectRegistry.removeAndNotifyRemoteProcess(RemoteAngularObjectRegistry.java:105)
    at org.apache.zeppelin.interpreter.remote.RemoteAngularObjectRegistry.removeAllAndNotifyRemoteProcess(RemoteAngularObjectRegistry.java:123)
    at org.apache.zeppelin.notebook.Notebook.removeNote(Notebook.java:326)
    at org.apache.zeppelin.socket.NotebookServerTest.testMakeSureNoAngularObjectBroadcastToWebsocketWhoFireTheEvent(NotebookServerTest.java:147)

Merging to master and branch-0.6 if there is no further discussion.

@lresende
Copy link
Member Author

@bzz Anything else needed here ?

@felixcheung
Copy link
Member

I'll merge it today, thanks!

@asfgit asfgit closed this in df2e77d Sep 1, 2016
asfgit pushed a commit that referenced this pull request Sep 5, 2016
### What is this PR for?
On Flink interpreter, remove provided scope from scala libraries to enable copying them to interpreter location.

### What type of PR is it?
[Bug Fix]

### What is the Jira issue?
[ZEPPELIN-1379](https://issues.apache.org/jira/browse/ZEPPELIN-1379)

Author: Luciano Resende <lresende@apache.org>

Closes #1370 from lresende/flink-dependencies and squashes the following commits:

8e2ea20 [Luciano Resende] [ZEPPELIN-1379] Flink interpreter is missing scala libraries

(cherry picked from commit df2e77d)
Signed-off-by: Mina Lee <minalee@apache.org>
@lresende lresende deleted the flink-dependencies branch December 16, 2016 23:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

Comments