Would you find it useful if you could reliably fail tests that have unawaited coroutines?

[njsmith]

Right now, this test passes:

async def do_something_broken():
    assert False

@pytest.mark.asyncio
async def test_something_broken():
    do_something_broken()

The reason is that we forgot the await in test_something_broken, so the broken code never actually ran. Oops. Python does issue a RuntimeWarning: coroutine 'do_something_broken' was never awaited, and recent pytest will print this at the end of tests, but this has a few issues:

• if your CI is green then how often do you click through to check for non-fatal warnings?
• since the warning isn't issued until the coroutine is garbage collected, on PyPy this can happen in some random other test, or if the test is near the end of the run it might never happen at all. E.g. with latest pypy3, pytest, and pytest-asyncio, the above test doesn't issue any warnings at all:

============================= test session starts ==============================
platform linux -- Python 3.5.3[pypy-5.8.0-beta], pytest-3.2.2, py-1.4.34, pluggy-0.4.0
rootdir: /tmp, inifile:
plugins: cov-2.5.1, catchlog-1.2.2, asyncio-0.7.0
collected 1 item                                                                

../../tmp/test.py .

=========================== 1 passed in 0.02 seconds ===========================

I'm considering proposing a new feature for Python 3.7, that would make it so pytest-asyncio could do:

# Ask Python to start maintaining a list of unawaited coroutines
sys.set_unawaited_coroutine_tracking(True)
try:
    ... run the test ...
finally:
    # Get the unawaited coroutines
    unawaited_coroutines = sys.get_and_clear_unawaited_coroutines()
    sys.set_unawaited_coroutine_tracking(False)
    if unawaited_coroutines:
        # Issue an error that points to the actual problem
        raise RuntimeError(f"Unawaited coroutines: {unawaited_coroutines}")

(Names etc. to be bikeshedded later; this is "API 2" in python-trio/trio#79 (comment))

This way you could deterministically detect unawaited coroutines, reliably attribute them to the correct test, and cause it to fail with a useful error message.

Is this an API that you'd want to take advantage of if it were available?

[Tinche]

Well, since you explained it so nicely, I'd be interested of course :)

[thedrow]

I just hit this one twice in a row.

[peniqliotuv]

ditto

[yalwan-sage]

Absolutely yes; I can say I've come across tests that were never running for an appreciable amount of time because of this, combined with #77 ; failing the test would have prompted the author to abandon converting to unittest

[mtnpke]

If you're using mypy (can also be integrated into CI), you can catch this statically: https://mypy.readthedocs.io/en/stable/error_code_list2.html#check-that-awaitable-return-value-is-used-unused-awaitable

Ruff might also add this: astral-sh/ruff#9911