Skip to content

ZEPPELIN-1063: fix PythonInterpreter flaky test #1134

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
bzz wants to merge 3 commits into from
Closed

ZEPPELIN-1063: fix PythonInterpreter flaky test #1134

bzz wants to merge 3 commits into from

Conversation

@bzz
Copy link
Member

@bzz bzz commented Jul 5, 2016

What is this PR for?

fix flaky PythonInterpreter.testClose() test

What type of PR is it?

Hot Fix

What is the Jira issue?

ZEPPELIN-1063

How should this be tested?

CI should pass

Questions:

  • Does the licenses files need update? no
  • Is there breaking changes for older versions? no
  • Does this needs documentation? no

@bzz
Copy link
Member Author

bzz commented Jul 5, 2016

Should fix build on master and #1100

@bzz bzz force-pushed the ZEPPELIN-1063/python/add-retrys branch from b0b3f32 to 3529fe5 Compare July 5, 2016 09:01
@bzz
Copy link
Member Author

bzz commented Jul 5, 2016

Rebased on latest master, waiting CI to finish and merging as a HOTFIX

@bzz
Copy link
Member Author

bzz commented Jul 6, 2016

CI fails with something strange, never seen before

Running org.apache.zeppelin.interpreter.remote.RemoteInterpreterTest
11:02:15,102  INFO org.apache.zeppelin.interpreter.remote.RemoteInterpreterManagedProcess:114 - Run interpreter process [/home/travis/build/apache/zeppelin/zeppelin-interpreter/../bin/interpreter.sh, -d, fake, -p, 36543, -l, fakeRepo]
/home/travis/build/apache/zeppelin/zeppelin-interpreter/../bin/interpreter.sh: line 159: /home/travis/build/apache/zeppelin/run/zeppelin-interpreter-fake-travis-testing-worker-linux-docker-b27ad04a-3371-linux-4.pid: No such file or directory
11:02:15,617  INFO org.apache.zeppelin.interpreter.remote.RemoteInterpreter:223 - Create remote interpreter org.apache.zeppelin.interpreter.remote.mock.MockInterpreterA
11:02:15,876  INFO org.apache.zeppelin.interpreter.remote.RemoteInterpreter:223 - Create remote interpreter org.apache.zeppelin.interpreter.remote.mock.MockInterpreterB
11:02:15,912 ERROR org.apache.zeppelin.interpreter.remote.RemoteInterpreterEventPoller:192 - Can't handle event RemoteInterpreterEvent(type:OUTPUT_UPDATE, data:{"data":"","noteId":"note","paragraphId":"id"})
java.lang.NullPointerException
    at org.apache.zeppelin.interpreter.remote.RemoteInterpreterEventPoller.run(RemoteInterpreterEventPoller.java:174)
11:02:15,946 ERROR org.apache.zeppelin.interpreter.remote.RemoteInterpreterEventPoller:192 - Can't handle event RemoteInterpreterEvent(type:OUTPUT_APPEND, data:{"data":"","noteId":"note","paragraphId":"id"})
java.lang.NullPointerException
    at org.apache.zeppelin.interpreter.remote.RemoteInterpreterEventPoller.run(RemoteInterpreterEventPoller.java:160)
11:02:15,956  INFO org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess:127 - shutdown interpreter process
11:02:18,078  INFO org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess:137 - Exception in RemoteInterpreterProcess while synchronized dereference, can safely ignore exception while client.shutdown() may terminates remote process
11:02:18,080  INFO org.apache.zeppelin.interpreter.remote.RemoteInterpreterManagedProcess:152 - Interpreter process exited 0
11:02:18,580  INFO org.apache.zeppelin.interpreter.InterpreterGroup:140 - Close interpreter group InterpreterGroup_1467716535100_997017074
11:02:18,590  INFO org.apache.zeppelin.interpreter.InterpreterGroup:204 - Destroy interpreter group InterpreterGroup_1467716535100_997017074
11:02:18,601  INFO org.apache.zeppelin.interpreter.remote.RemoteInterpreterManagedProcess:114 - Run interpreter process [/home/travis/build/apache/zeppelin/zeppelin-interpreter/../bin/interpreter.sh, -d, fake, -p, 47761, -l, fakeRepo]
/home/travis/build/apache/zeppelin/zeppelin-interpreter/../bin/interpreter.sh: line 159: /home/travis/build/apache/zeppelin/run/zeppelin-interpreter-fake-travis-testing-worker-linux-docker-b27ad04a-3371-linux-4.pid: No such file or directory
11:02:19,104  INFO org.apache.zeppelin.interpreter.remote.RemoteInterpreter:223 - Create remote interpreter org.apache.zeppelin.interpreter.remote.mock.MockInterpreterA
11:02:19,481  INFO org.apache.zeppelin.scheduler.SchedulerFactory:131 - Job 0 started by scheduler org.apache.zeppelin.interpreter.remote.RemoteInterpreternote1780497356
11:02:19,529 ERROR org.apache.zeppelin.interpreter.remote.RemoteInterpreterEventPoller:192 - Can't handle event RemoteInterpreterEvent(type:OUTPUT_UPDATE, data:{"data":"","noteId":"note","paragraphId":"0"})
java.lang.NullPointerException
    at org.apache.zeppelin.interpreter.remote.RemoteInterpreterEventPoller.run(RemoteInterpreterEventPoller.java:174)
11:02:19,573 ERROR org.apache.zeppelin.interpreter.remote.RemoteInterpreterEventPoller:192 - Can't handle event RemoteInterpreterEvent(type:OUTPUT_APPEND, data:{"data":"","noteId":"note","paragraphId":"0"})
java.lang.NullPointerException
    at org.apache.zeppelin.interpreter.remote.RemoteInterpreterEventPoller.run(RemoteInterpreterEventPoller.java:160)
11:02:19,583  INFO org.apache.zeppelin.scheduler.SchedulerFactory:137 - Job 0 finished by scheduler org.apache.zeppelin.interpreter.remote.RemoteInterpreternote1780497356

re-triggerin, to see if it's reproducable

@bzz bzz closed this Jul 6, 2016
@bzz bzz reopened this Jul 6, 2016
felixcheung
felixcheung reviewed Jul 6, 2016
View reviewed changes
python/src/test/java/org/apache/zeppelin/python/PythonInterpreterTest.java Outdated

assertTrue(serverIsListeningOn(py4jPort));
pythonInterpreter.close();
TimeUnit.SECONDS.sleep(1);
Copy link
Member

@felixcheung felixcheung Jul 6, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this might be too short for slow Travis CI runs?

@felixcheung
Copy link
Member

felixcheung commented Jul 6, 2016

LGTM seems like CI passed.

@bzz
Copy link
Member Author

bzz commented Jul 7, 2016

Reviews addressed in 65698d2, waiting for CI now

@bzz
Copy link
Member Author

bzz commented Jul 7, 2016

CI is green, merging if there is no more discussion

@asfgit asfgit closed this in 399c49b Jul 7, 2016
PhilippGrulich pushed a commit to SWC-SENSE/zeppelin that referenced this pull request Aug 8, 2016
@bzz @PhilippGrulich
ZEPPELIN-1063: fix PythonInterpreter flaky test
c6a5509 
### What is this PR for?
 fix flaky `PythonInterpreter.testClose()` test

### What type of PR is it?
Hot Fix

### What is the Jira issue?
[ZEPPELIN-1063](https://issues.apache.org/jira/browse/ZEPPELIN-1063)

### How should this be tested?
CI should pass

### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no

Author: Alexander Bezzubov <bzz@apache.org>

Closes apache#1134 from bzz/ZEPPELIN-1063/python/add-retrys and squashes the following commits:

65698d2 [Alexander Bezzubov] Reduce 😴 time in python interpreter unit-test
1be7732 [Alexander Bezzubov] Adding timeout for py4j gateway to have time to shutdown
3529fe5 [Alexander Bezzubov] fix PythonInterpreterTest.testClose() test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@bzz @felixcheung