Fix and enable doctests

[Erotemic]

I noticed that the OCIContainerEngineConfig doctest I wrote was no longer working. It seems like the class now requires oci_platform as a required arugment.

To fix this, I added a classmethod to the enum to auto-detect what the current platform is. Then if we are on a supported platform it runs the test. If not, it gracefully skips the test with pytest.

This effect could also be achieved by using an xdoctest directive: # xdoctest: +REQUIRES(LINUX) (assuming xdoctest was the doctest runner). However, I don't think these doctests are running in CI at all, otherwise this likely would have been caught.

I also noticed two other doctests were failing. One in OptionsReader because there is no setup for the config_file. I didn't see an obvious way to make concise demo data for it, so I just added a skip directive.

The other doctest in util.helpers.format_safe seems to be broken. The doctest was testing for a backslash, but the logic seems to be wanting a hash as the escape character. This was easy to fix. With that xdoctest cibuildwheel has 3 doctests pass and 1 skip. This should also be easy to add to the CI by including xdoctest as a dependency and using --xdoctest in the pytest invocation. It is possible to get these working with stdlib doctest, but xdoctest is a quite a bit more powerful and allows for more concise doctests (biased opinion).

[Erotemic]

Not sure why the pyodide test is failing.

[joerick]

These comments are more documentation than tests, but it's nice to have them checked to ensure that they're not going out of date.

There's a similar map in oci_container_test that could be replaced with OCIPlatform.native().

Could you also add doctest checking to the run_tests.py and the nox tests task? That way we can avoid the doctests falling out-of-date. Happy to use xdoctest for that if you think it's suitable!

[henryiii]

I'm fine with xdoctest FWIW, testing dependencies are low cost.

[Erotemic]

I haven't added xdoctest integration to a project where nox was the driver. I think what I did is idomatic, but let me know if there is a way that makes more sense for cibuildwheel. I've added a similar invocation for doctests to the bin/run_tests.py script.

There are two things that need to happen to get doctests running with xdoctest:

1. pytest needs to be pointed at the source path, or the path to the installed package so it can parse the doctests.
2. the --xdoctest argument needs to be passed so the default doctest plugin is disabled and replaced with xdoctest.

Looking at the noxfile it looked like the unit test and bigger tests were separated, so I added a third call for doctests only, which points at cibuildwheel, assuming that the pytest command is running in the root of the repo. This should work because it seems the tests are run against an editable mode install.

I also added --xdoctest to the pytest section in pyproject. This means that all pytest calls will pick up any doctests in whatever directory they are run on. The other option is pass it to pytest directly in the nox file. Both options have advantages / disadvantages.

Looks like circleci has a "-k" filter that disables all doctests, and if pytest collects no tests, then it returns an exitcode of 5. So, I've added a special case to allow for that to happen. The alternative is that we could add "or cibuildwheel" to PYTEST_ADDOPTS, which will hit all doctests, but it also hits 3 other tests in unit_test that I think are enabled anyway. It's unfortunate that using "-k doctest" or "-k xdoctest" doesn't work. I may look into fixing that in a new xdoctest version.

Other changes:

• I had to add a special case to the doctest in OCIContainer as it hard codes image = _get_pinned_container_images()['x86_64']['manylinux2014']. It would have been nice if _get_pinned_container_images()[oci_platform.name] worked, but the pinned images want "x86_64" as the key and not "linux/amd64". I think it would be a little cleaner if there was a way to lookup corresponding pinned images for a platform enum, but I didn't want to add that complexity without running it by maintainers. So there is an extra skip case for now. I can make a concise helper for this to clean up the doctest if desired.

• I also too @joerick's suggestion to refactor the oci container unit tests with the native classmethod.

• Added "--ignore-glob=*venv*" to pytest to prevent pytest from collecting doctests in the virtual environment folders.

[joerick]

Great, thanks!