Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Doctest should ignore lines starting with ``` when run on Markdown files #7374

Closed
asmeurer opened this issue Jun 15, 2020 · 6 comments
Closed
Labels
plugin: doctests related to the doctests builtin plugin type: enhancement new feature or API change, should be merged into features branch

Comments

@asmeurer
Copy link

When you use --doctest-glob=*.md to run doctests on Markdown files, you often run into issues doctesting code blocks like

```python
>>> a = 1
>>> a
1
```

You get errors like

___________________________________________________________________________ [doctest] test.md ___________________________________________________________________________
001 ```python
002 >>> a = 1
003 >>> a
Expected:
    1
    ```
Got:
    1

The problem is that it thinks the closing ``` is part of the doctest. This is really a problem with the upstream doctest, but getting a fix like this included in it seems much harder than fixing pytest.

A workaround here is to always add an empty line before the closing ```. But aside from being annoying to do, this affects the rendering of the document with some (all?) Markdown processors, adding an empty line to the end of the rendered codeblock.

When using the base doctest, you can fix this with something like

from doctest import DocTestRunner

orig_run = DocTestRunner.run

def run(self, test, **kwargs):
    for example in test.examples:
        # Remove ```
        example.want = example.want.replace('```\n', '')
        example.exc_msg = example.exc_msg and example.exc_msg.replace('```\n', '')

    return orig_run(self, test, **kwargs)

DocTestRunner.run = run

Presumably a similar fix could be made in pytest when testing Markdown files, so that it works better.

@Zac-HD Zac-HD added plugin: doctests related to the doctests builtin plugin type: enhancement new feature or API change, should be merged into features branch labels Jun 16, 2020
@Zac-HD
Copy link
Member

Zac-HD commented Jun 16, 2020

I think it actually makes a lot of sense to do this in pytest rather than upstream; in particular we can ensure that the logic only applies to markdown files (e.g. .md or .markdown extensions). I'd also want to ensure that we only replace trailing triple-backticks, not any occurence in the expected test, but that's easy enough.

@asottile
Copy link
Member

Is there a bpo bug for the upstream problem? I'd be a little concerned just blindly removing a trailing triple-tick

for example it would break this test:

    >>> my_object = MyObject()
    >>> my_object
    ```my
    repr
    is fancy
    ```

@asmeurer
Copy link
Author

I never opened an upstream bug.

for example it would break this test:

If this is a concern one could modify the doctest matcher to try to determine in a more sophisticated way if the example block is in a code fence.

@asottile
Copy link
Member

I never opened an upstream bug.>

let's start with an upstream bug report before we decide to diverge

@asmeurer
Copy link
Author

I'm sorry, but I'm not interested in trying to push a change into CPython itself. If you want to do it, feel free. Libraries like pytest can move much faster than the standard library, so it makes sense to make changes like this in them first, then backport them into the upstream later if that becomes feasible. It doesn't make sense to block movement on an issue like this on something happening in the extremely slow moving standard library.

@RonnyPfannschmidt
Copy link
Member

pytest has dedicated doctest maintainer - perhaps doctestplus is interested in providing the feature

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
plugin: doctests related to the doctests builtin plugin type: enhancement new feature or API change, should be merged into features branch
Projects
None yet
Development

No branches or pull requests

4 participants