ValueError: no option named 'verbose' - during assertion eval, when Terminal plugin is disabled

[yaelmi3]

Execute pytest without the terminal plugin, i.e. pytest -p no:terminal -s, when the test contains an assertion failure, for example:

def test_assert():
    assert 1 == 2, "fail"

An exception is issued:
(https://github.com/pytest-dev/pytest/blob/main/src/_pytest/assertion/util.py#L161)

_pytest/assertion/__init__.py", line 181, in pytest_assertrepr_compare
 return util.assertrepr_compare(config=config, op=op, left=left, right=right)
_pytest/assertion/util.py", line 161, in assertrepr_compare
   verbose = config.getoption("verbose")
_pytest/config/__init__.py", line 1484, in getoption
    raise ValueError(f"no option named {name!r}") from e
ValueError: no option named 'verbose'

Additional reference to config.option.verbose: https://github.com/pytest-dev/pytest/blob/main/src/_pytest/assertion/truncate.py#L29

The verbose parameter is set alongside with the Terminal plugin (https://github.com/pytest-dev/pytest/blob/main/src/_pytest/terminal.py#L144), but apparently is used by the assertion utils as well.
Possible solution, moving verbosity setting from terminal pytest adoption to runner pytest_adoption , since the verbosity setting affects modules outside the terminal output

---

Our specific use case: Disable Terminal Plugin and forward results using dedicated logger. We achieve it by implementing the following:

@pytest.hookimpl(tryfirst=True, hookwrapper=True)
def pytest_runtest_makereport(item, call):
    result = (yield).get_result()
    if call.when == "call" or (call.when == "setup" and call.excinfo is not None):
        test_id = None
        if callspec := getattr(item, "callspec", None):
            test_id = callspec.id
        test_details = dict(test_name=item.name, test_id=test_id, passed=result.passed)
        if call.excinfo is not None:
            test_details.update(dict(type=call.excinfo.type.__name__, reason=str(call.excinfo.value)))
            logger.exception("Test failed", extra=test_details, exc_info=call.excinfo.value)
        else:
            logger.info("Test Passed", extra=test_details)

Then we execute pytest with terminal plugin disabled. The problem occurs upon assertion failures and currently our workaround is setting config.option.verbose during the the pytest_configure on our end.

[cole-floodbase]

I still don't understand this issue, but I run into it from time to time in different environments. I'm using this workaround to manually force the verbose option.

from pathlib import Path
from _pytest.config import PytestPluginManager, Config, default_plugins


def run_pytest_cli(args, plugins):
    # The majority of this function is copied from Pytest internals
    pluginmanager = PytestPluginManager()
    config = Config(
        pluginmanager,
        invocation_params=Config.InvocationParams(
            args=args,
            plugins=plugins,
            dir=Path.cwd(),
        ),
    )

    # Here is the patch I've added to Pytest's source code to get around 
    #   https://github.com/pytest-dev/pytest/issues/9422
    config.option.verbose = True

    pluginmanager.consider_preparse(args, exclude_only=True)
    for default_plugin in default_plugins:
        pluginmanager.import_plugin(default_plugin)

    pluginmanager = config.pluginmanager
    for plugin in plugins:
        pluginmanager.register(plugin)

    config = pluginmanager.hook.pytest_cmdline_parse(
        pluginmanager=pluginmanager, args=args
    )

    exit_code = config.hook.pytest_cmdline_main(config=config)
    return exit_code

# e.g.
if __name__ == "__main__":
    failure_collector_plugin = FailureCollectorPlugin()

    exit_code = run_pytest_cli(
        args=["-p", "no:terminal"],
        plugins=[failure_collector_plugin],
    )

[RonnyPfannschmidt]

Replication of the internals is a last resort i strongly recommend against using

That being said, the use of the option should be changed

However it should also be noted that disabling built in plugins is potentially dangerous

I recommend taking a look at how xdist and sugar manage things

[jpe]

I'm also running into this and have used a hack to get around the verbose problems, but in recent versions the assertion rewriter plugin has been failing if the terminal plugin is disabled.

I wonder if the thing to do is to add a way to disable all output from the terminal reporter. The use case for this is when tests are run in a subprocess launched by an IDE or other tool and test success / failure is not reported through terminal output.

[nicoddemus]

The simple fix would be to add a default to the getoption call, so the error will not happen anymore in case the option is not defined, for example config.getoption("verbose") to config.getoption("verbose", 0). While might seem like a workaround, I think it is reasonable and simple enough to get this in.

Would somebody like to give this a try?

[ka28kumar]

Hello @nicoddemus ,
I'm new to OSS and would love to take this up. Thanks!

[nicoddemus]

Hi @ka28kumar, that's great to hear. No need for permission to get started, feel free to fork the project, and submit a PR (even if incomplete).

[ka28kumar]

Hi @nicoddemus ,
Rather than replacing every call of config.getoption("verbose") to config.getoption("verbose", 0), which also includes some examples in the docs. I'd prefer if we can add a default value of 0 to the verbose option. Before that, I'm not able to reproduce the issue in question. I took the example above, and executed it with both -p no:terminal and without it, and it executes the same.

[RonnyPfannschmidt]

I believe its a good idea to unify all options of the built-in plugins so disabling plugins won't break CLI args and option expectations for the built-in plugins

[nicoddemus]

I'd prefer if we can add a default value of 0 to the verbose option.

What do you mean, add the default to the verbose definition? Note that's already there:

pytest/src/_pytest/terminal.py

Lines 135 to 142 in 9446f02

	group._addoption(
	"-v",
	"--verbose",
	action="count",
	default=0,
	dest="verbose",
	help="Increase verbosity",
	)

The problem is actually that with -p no:terminal the option is not even defined, because the terminal plugin does not get loaded.

I believe its a good idea to unify all options of the built-in plugins so disabling plugins won't break CLI args and option expectations for the built-in plugins

Interesting idea, for some builtin plugins this seems like a good option, not sure the same is true for every builtin plugin (for example stepwise, pastebin, etc).