Add an ini option doctest_standalone_namespace #6978

asmeurer · 2020-03-27T19:56:58Z

This makes doctests run in a standalone namespace. The default is False, which
does the pre-existing behavior where doctests reuse the namespace of the
module they are written in.

Fixes scientific-python/pytest-doctestplus#15.

I still need to document this flag. But before I do, is this the best name for this option? I feel like the name I used here isn't great, but I couldn't think of a better one.

Add a test for doctest_namespace https://github.com/pytest-dev/pytest/pull/6978/files#r440397508
Find a better name for the option
Add @asmeurer to authors
Add a description and use case for the option in doctest.rst
Quick reference to it in reference.rst
Add a CHANGELOG entry

This makes doctests run in a standalone namespace. The default is False, which does the pre-existing behavior where doctests reuse the namespace of the module they are written in. Fixes scientific-python/pytest-doctestplus#15.

nicoddemus

Thanks a lot @asmeurer, this looks great already!

We just need a few extra things:

Please add a new section explaining this option, including rationale and a small example, in doc/en/doctest.rst. Also include a link to the reference (see below).
Include reference documentation for the option in doc/en/reference.rst#configuration-options, including a link to the doctest section added above.
A feature CHANGELOG entry. Ideally it would describe the option shortly and include a link to the docs added in 1).

Again, thanks for working on this. 👍

bsipocz · 2020-03-27T21:55:17Z

src/_pytest/doctest.py

@@ -62,6 +62,12 @@ def pytest_addoption(parser):
    parser.addini(
        "doctest_encoding", "encoding used for doctest files", default="utf-8"
    )
+    parser.addini(
+        "doctest_standalone_namespace",


I can't think of any better options for the name either.
How about calling it doctest_context with a non bool type, or even better, doctest_reset_context? That would leave the option up to implement something similar to the plot directive's context. And then, this can remain a bool.

I'm OK with doctest_reset_context, although perhaps doctest_empty_context better conveys what the option actually does?

From a technical point of view, "namespace" is a better term than "context". Two function doctests don't interact with each other, which is what "reset context" seems to imply to me. So if you define a variable in one doctest it won't be available in another doctest.

Rather, they just reuse the module namespace, so if a variable is defined at the module level (not in a doctest), it gets defined automatically in a doctest. Maybe a name like doctest_no_module_namespace (although honestly my biggest issue with the current name is that it is too long).

IMHO a long option name is OK if it correctly describes the behavior, because it will be written in a pytest.ini file only once anyway. 😁

I guess my problem isn't so much the length as the fact that the option name seems convoluted. I don't think many people will look at "doctest_standalone_namespace" and understand what it means unless they already know.

Fair point, agree.

doctest_no_module_namespace seems better in that light then, IMHO.

This flag isn't supported in pytest yet. It will be once pytest-dev/pytest#6978 is merged. However, I would recommend waiting until that PR is merged to merge this since the name of the flag may change. The flag will make it so that the doctests require all names to be imported in order to pass, as opposed to the default behavior which automatically includes all names from the surrounding module in the doctest namespace.

lucafavatellawork · 2020-05-19T15:00:43Z

src/_pytest/doctest.py

            ) -> None:
                if _is_mocked(obj):
                    return
+
+                if self.config.getini("doctest_standalone_namespace"):
+                    globs = {}


Based on https://github.com/pytest-dev/pytest/blob/5.3.1/src/_pytest/doctest.py#L109-L113 that seems to use DoctestModule for *.py and DoctestTextfile for *.txt/*.rst, as this code change is only in DoctestModule, does this mean that this flag would apply only to the doctest examples inside *.py files and not the files that are fully doctest e.g. *.rst? Is that intended?

I don't think non-py files can have variables that are defined outside of the doctest. Or is there something I'm missing?

There is the fixture doctest_namespace.

I am still getting familiar with it, though I have the impression that doctest_namespace carries state across modules / folders and I was wondering whether this patch was going to interact with doctest_namespace.

I'm not sure how this currently interacts with that, nor how it should.

It does interact, but in this case I don't think the new option should interact at all with the doctest_namespace fixture: the intent there is to inject globals into the tests (such as the numpy import), so resetting that would defeat the purpose.

Maybe it makes sense to allow globs to be modified in a fixture in conftest.py (side note: "globs" is an unfortunate name from doctest; it is short for "globals" and has nothing to do with shell globs). Although I do like the simplicity of an ini setting to get the globs = {} behavior.

Actually, taking a second look at the code, doctest_namespace won't be affected by this at all, because the doctest_namespace dict is injected during the setup phase of the test, while here globs is cleared during collection.

@asmeurer could you please add a test which ensures setting doctest_standalone_namespace to True still allow doctests to access the dict returned by doctest_namespace? We also should explicitly mention this in the description of the doctest_standalone_namespace.

nicoddemus

Hi everyone,

Sorry for the lack of response here.

Besides the discussion about the name of the option, we also would need:

Changelog
Add @asmeurer to authors
Add a description and use case for the option in doctest.rst
Quick reference to it in reference.rst

asmeurer · 2020-06-14T06:13:22Z

Sorry for sitting on this. I do intend to fix it. I partly haven't finished it because I don't really like the option name used here, partly because I had some issues building the docs, and partly because I got sidetracked with other things.

nicoddemus · 2020-06-15T19:20:00Z

src/_pytest/doctest.py

@@ -62,6 +62,12 @@ def pytest_addoption(parser):
    parser.addini(
        "doctest_encoding", "encoding used for doctest files", default="utf-8"
    )
+    parser.addini(
+        "doctest_standalone_namespace",


IMHO a long option name is OK if it correctly describes the behavior, because it will be written in a pytest.ini file only once anyway. 😁

nicoddemus · 2020-06-15T19:27:58Z

src/_pytest/doctest.py

            ) -> None:
                if _is_mocked(obj):
                    return
+
+                if self.config.getini("doctest_standalone_namespace"):
+                    globs = {}


Actually, taking a second look at the code, doctest_namespace won't be affected by this at all, because the doctest_namespace dict is injected during the setup phase of the test, while here globs is cleared during collection.

@asmeurer could you please add a test which ensures setting doctest_standalone_namespace to True still allow doctests to access the dict returned by doctest_namespace? We also should explicitly mention this in the description of the doctest_standalone_namespace.

asmeurer · 2020-06-15T19:46:21Z

I added a TODO list to the PR description so I don't forget anything.

Actually another reason I haven't worked on this is that for the project I was going to use it on, I've had to move to a custom doctest runner because I am doctesting Markdown files and I need to be able to tell the doctest runner to ignore lines starting with ```. I do this by monkeypatching DoctestRunner.run. I don't know if there is some way to do the same thing with pytest-doctest.

asmeurer · 2020-06-15T19:57:26Z

Or maybe it should just be smart enough to do that automatically for Markdown files. I can open a new issue.

nicoddemus · 2021-10-05T19:58:57Z

@asmeurer closing this for now to reduce the number of opened PRs, but feel free to reopen in case you want to retake this. 👍

Add an ini option doctest_standalone_namespace

3fad75b

This makes doctests run in a standalone namespace. The default is False, which does the pre-existing behavior where doctests reuse the namespace of the module they are written in. Fixes scientific-python/pytest-doctestplus#15.

asmeurer mentioned this pull request Mar 27, 2020

Check examples to be standalone scientific-python/pytest-doctestplus#15

Open

nicoddemus requested changes Mar 27, 2020

View reviewed changes

bsipocz reviewed Mar 27, 2020

View reviewed changes

asmeurer mentioned this pull request Apr 23, 2020

Enable the doctest_standalone_namespace flag in the tests Quansight-Labs/ndindex#4

Closed

lucafavatellawork reviewed May 19, 2020

View reviewed changes

nicoddemus requested changes Jun 13, 2020

View reviewed changes

nicoddemus requested changes Jun 15, 2020

View reviewed changes

asmeurer mentioned this pull request Jul 17, 2020

Run the doctests with pytest Quansight-Labs/ndindex#58

Open

Base automatically changed from master to main March 9, 2021 20:40

nicoddemus closed this Oct 5, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add an ini option doctest_standalone_namespace #6978

Add an ini option doctest_standalone_namespace #6978

asmeurer commented Mar 27, 2020 •

edited

Loading

nicoddemus left a comment

bsipocz Mar 27, 2020

nicoddemus Jun 13, 2020

asmeurer Jun 14, 2020

nicoddemus Jun 15, 2020

asmeurer Jun 15, 2020

nicoddemus Jun 15, 2020

lucafavatellawork May 19, 2020

asmeurer May 19, 2020

lucafavatellawork May 19, 2020

asmeurer May 19, 2020

nicoddemus Jun 13, 2020

asmeurer Jun 14, 2020

nicoddemus Jun 15, 2020

nicoddemus left a comment •

edited

Loading

asmeurer commented Jun 14, 2020

nicoddemus Jun 15, 2020

nicoddemus Jun 15, 2020

asmeurer commented Jun 15, 2020

asmeurer commented Jun 15, 2020

nicoddemus commented Oct 5, 2021

Add an ini option doctest_standalone_namespace #6978

Add an ini option doctest_standalone_namespace #6978

Conversation

asmeurer commented Mar 27, 2020 • edited Loading

nicoddemus left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nicoddemus left a comment • edited Loading

Choose a reason for hiding this comment

asmeurer commented Jun 14, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

asmeurer commented Jun 15, 2020

asmeurer commented Jun 15, 2020

nicoddemus commented Oct 5, 2021

asmeurer commented Mar 27, 2020 •

edited

Loading

nicoddemus left a comment •

edited

Loading