Doesn't work with pytest

[shacker]

Under python 3.6, I used ic() regularly in my pytest tests to assist with debugging. Now we have switched our project to python 3.8, and I no longer can. It errors with

ic| Error: Failed to access the underlying source code for analysis. Was ic() invoked in an interpreter (e.g. python -i), a frozen application (e.g. packaged with PyInstaller), or did the underlying source code change during execution?

Is there something I can do to get it working with pytest / python3.8 again?

[alexmojaki]

pytest does various magic to make its informative asserts work, and unfortunately that magic conflicts with how ic works. The recently released version 2.0 is more paranoid about making sure that everything looks the way it expects and that the bytecode matches the source code. I've opened alexmojaki/executing#2 now since the underlying issue is there.

You can pip install 'icecream<2' to avoid the update, although I wouldn't be surprised if there are edge cases where it seems to work but silently picks up the wrong ic() call location.

You can also try https://github.com/alexmojaki/birdseye for an alternative debugging method.

[shacker]

I see. So there's a chance it'll be fixed in the future. I'll try the downgrade for now, or just use other techniques from within tests. Thanks!

[gruns]

@shacker I'm curious: does the older version of icecream < 2.0, eg version v1.3.1, work for your use case?

[alexmojaki]

I've played around with icecream<2 a bit, here's an example of it failing strangely with pytest:

def test_ic():
    x = 1
    assert ic(x)

but changing x to 1 is fine:

def test_ic():
    x = 1
    assert ic(1)

The traceback in the first case is:

icecream.py:522: in __call__
    out = self._format(callFrame, *args)
icecream.py:556: in _format
    out = self._formatArgs(
icecream.py:575: in _formatArgs
    callStr = extractCallStrByOffset(splitSource, relativeCallOffset)

splitSource = 'assert  \\\nic(x)', callOffset = 4

    def extractCallStrByOffset(splitSource, callOffset):
        code = compile(splitSource, '<string>', 'exec')
        lineOffsets = sorted(calculateLineOffsets(code).items())
    
        # For lines with multiple invocations, like 'ic(1); ic(2)', determine which
        # invocation was called.
        for lineno, offset in lineOffsets:
            if callOffset >= offset:
                sourceLineIndex = lineno - 1
            else:
                break
    
        lines = [s.rstrip(' ;') for s in splitSource.splitlines()]
>       line = lines[sourceLineIndex]
E       UnboundLocalError: local variable 'sourceLineIndex' referenced before assignment

icecream.py:429: UnboundLocalError

[shacker]

Ah, ic 2.0 was released recently? So I originally blamed this on the python3.8 upgrade, but what probably actually happened is that I accidentally upgraded to icecream 2 at the same time and that's what caused it to start failing. To answer your question, I've used ic() from within pytest for about a year and it's always worked fine, so yes, the 1.x version did work. I could try going back, though I hate to miss out on other improvements.

[shacker]

Yep, confirmed. I just downgraded to 1.3.1 and it works fine from within pytest.

[alexmojaki]

Do you ever use ic() within larger statements/expressions, or do you generally just use it on its own? For example, if you have assert x, do you sometimes change it to assert ic(x), or do you just write

ic(x)
assert x

[shacker]

@alexmojaki I always use it on its own. For me the basics are all I really need - show the line number, variable's name and value, and to prettyprint dicts. Just a basic timesaver that's become essential workflow for me.

[alexmojaki]

I suspect that's how most people use it. It's probably easier in most cases to just type once than to insert characters in multiple places. I think the idea of ic(x) inside an expression is nicer in theory than in practice.

That's probably why I was able to easily find a simple failure case whereas it's always worked for you. ic probably usually worked outside of an assert and usually failed inside one.

The same principle applies to all the failure cases I listed in #33 - they all happen inside a bigger statement, so no one ever came across them and reported them because they just used ic on its own.

I will try to get executing to work outside of pytest asserts. That should be quite feasible and should roughly restore ic to its previous level of functionality. Making it work within asserts looks too difficult.

[alexmojaki]

OK, pip install -U executing and try again.

[shacker]

Ah! I reinstalled icecream 2.0.0, went to test it in pytest expecting it to fail at first,, and suddenly it worked, even before installing executing. Was it auto-installed as a dependency? Awesome, thanks much!