sage-test doesn't handle all of sage-doctest's return values

[wjp]

The sage-doctest script returns some status info in its exit code, like if it was aborted with a KeyboardInterrupt. The sage-ptest script interprets this information, but sage-test mostly ignores it.

One symptom is that Ctrl-C-ing a sage -t run of multiple files doesn't work.

Component: doctest coverage

Author: Willem Jan Palenstijn

Reviewer: Alex Ghitza

Merged: sage-4.3.2.alpha0

Issue created by migration from https://trac.sagemath.org/ticket/7995

[wjp]

comment:2

Patch attached.

In the future we may want to merge the sage-ptest and sage-test scripts entirely. They have a lot of duplicate code and minor unnecessary differences.

[wjp]

Attachment: scripts_7995_sage-test_error_handling.patch.gz

[nthiery]

comment:3

I don't know much the doctest framework. But the code looks good, and this removes two seriously annoying misfeatures of doctests. Thanks so much for scratching this itch for me!

Quick remarks:

• When there is an exception in the doctesting framework; will we get
  a traceback? That would be most handy!

• in "... Check the process exit codes ...": codes -> code?

• The following two comments seem contradictory. Or am I
  misunderstanding something?

# Return value in process exit code:
# 0: all tests passed
# 1: file not found
# 2: KeyboardInterrupt
# 3: doctest process was terminated by a signal
# 4: the doctesting framework raised an exception
# 100: failed doctests
####################################################################

def test_code(filename):
# Return value in the doctest process exit code:
# 0: everything passed
# 1-253: number of failed doctests
# 254: >= 254 doctests failed
# 255: exception raised by doctesting framework

[nthiery]

comment:4

Sorry, I screwed up my citation. Here it is again.

• The following two comments seem contradictory. Or am I
  misunderstanding something?

 # Return value in process exit code: 
 # 0: all tests passed 
 # 1: file not found 
 # 2: KeyboardInterrupt 
 # 3: doctest process was terminated by a signal 
 # 4: the doctesting framework raised an exception 
 # 100: failed doctests 
 #################################################################### 

 def test_code(filename): 
     # Return value in the doctest process exit code: 
     # 0: everything passed 
     # 1-253: number of failed doctests 
     # 254: >= 254 doctests failed 
     # 255: exception raised by doctesting framework 

[nthiery]

comment:5

Replying to @nthiery:

Sorry, I screwed up my citation. Here it is again.

Oops again. This remark is actually about #7993. I was looking at both, and got confused.

[wjp]

comment:6

Thanks for the feedback.

The sage-doctest script generates a new python file that runs the actual doctests, and runs this script in a seperate thread. The second block of process exit codes are for this 'inner' doctest process. I'll clarify the confusing comment (and fix the typo you mentioned).

As for the exception: you can get a traceback if you re-run with the -verbose. Do you think we should show them by default?

[wjp]

comment:7

I attached a new patch to #7993 that changes the exit code comments.

[aghitza]

Author: Willem Jan Palenstijn