whitespace error in doctest causes A Mysterious Error.

[wjp]

When doctesting a file a.sage containing

def foo():
    """
    sage: 1+1
   2
    """
    pass

(note the missing space before the 2), you get:

[wjp@issa sage-4.3.1.rc0]$ ./sage -t a.sage
sage -t  "a.sage"                                           
A mysterious error (perhaps a memory error?) occurred, which may have crashed doctest.
         [2.4 s]
exit code: 768
 
----------------------------------------------------------------------
The following tests failed:


        sage -t  "a.sage"
Total time for all tests: 2.4 seconds

Component: doctest coverage

Author: Willem Jan Palenstijn

Reviewer: Paul Zimmermann

Merged: sage-4.3.4.alpha1

Issue created by migration from https://trac.sagemath.org/ticket/7993

[wjp]

comment:1

I changed the sage-doctest script to make the actual doctesting process catch exceptions and communicate this to sage-doctest via the process exit code.

It now differentiates between a crash and an exception raised by the doctesting code.

This patch depends on the patch at #7995.

[nthiery]

comment:2

As for #7995: thanks much for handling this. I'd love to see this in Sage very shortly.

Please include the new output in the ticket description!

I can try to review this, but I'd rather have a testing framework expert to it.

[qed777]

comment:3

Attachment: scripts_7993_doctest_error_handling.patch.gz

I noticed an existing, unrelated problem. Let foo.py contain

def g():
    """
    sage: 1 + 1
    11
    """
    return

Then sage -tp 1 foo.py ends with

1 items had failures:
   1 of   3 in __main__.example_0
***Test Failed*** 1 failures.
For whitespace errors, see the file /home/.sage//tmp/.doctest_foo.py
         [2.0 s]
 
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/apps/sage/local/bin/sage-ptest", line 361, in <module>
    failed_files[F.split('#')[0].split()[2]] = None
IndexError: list index out of range

[qed777]

comment:4

Anyway, we can make a separate ticket to unify, simplify, and doctest the doctesting framework.

I'm not an expert, but the changes look OK. Can we report at least partial results for interrupted tests?

[qed777]

Author: Willem Jan Palenstijn

[qed777]

comment:6

Minor update: The patch works for me in daily (i.e., not heavy) use. Any other experiences?

[zimmermann6]

comment:7

I wanted to try this patch to see if it would by chance solve #7773, but apparently it needs a rebase
for 4.3.3:

----------------------------------------------------------------------
| Sage Version 4.3.3, Release Date: 2010-02-21                       |
| Type notebook() for the GUI, and license() for information.        |
----------------------------------------------------------------------
Loading Sage library. Current Mercurial branch is: 7993
sage: hg_sage.import_patch("/tmp/scripts_7993_doctest_error_handling.patch")
cd "/usr/local/sage-4.3.3/sage/devel/sage" && hg status
cd "/usr/local/sage-4.3.3/sage/devel/sage" && hg status
cd "/usr/local/sage-4.3.3/sage/devel/sage" && hg import   "/tmp/scripts_7993_doctest_error_handling.patch"
applying /tmp/scripts_7993_doctest_error_handling.patch
unable to find 'sage-doctest' for patching
4 out of 4 hunks FAILED -- saving rejects to file sage-doctest.rej
unable to find 'sage-ptest' for patching
1 out of 1 hunks FAILED -- saving rejects to file sage-ptest.rej
unable to find 'sage-test' for patching
2 out of 2 hunks FAILED -- saving rejects to file sage-test.rej
abort: patch failed to apply

Paul

[qed777]

comment:8

The patch is for the scripts repository, whose root is SAGE_ROOT/local/bin. Try using hg_scripts, instead. I think the patch will still apply cleanly to 4.3.3.