Bug with prerelease pytest

[larsoner]

• a detailed description of the bug or problem you are having
• output of pip list from the virtual environment you are using
• pytest and operating system versions
• minimal example if possible

In our prerelease CIs we now get an internal error with 8.0.0rc1:

https://github.com/mne-tools/mne-python/actions/runs/7387584833/job/20096612938?pr=12330#step:15:4411

_______________________________ test_dir_warning _______________________________
mne/_fiff/tests/test_meas_info.py:355: in test_dir_warning
    read_info(ctf_fname)
<decorator-gen-34>:12: in read_info
    ???
mne/_fiff/meas_info.py:2045: in read_info
    fname = _check_fname(fname, must_exist=True, overwrite="read")
<decorator-gen-0>:12: in _check_fname
    ???
mne/utils/check.py:255: in _check_fname
    raise OSError(
E   OSError: Need a file for File but found a directory at /home/runner/mne_data/MNE-testing-data/CTF/testdata_ctf.ds

During handling of the above exception, another exception occurred:
    tw._write_source(source_lines, indents)
  File "/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/_pytest/_io/terminalwriter.py", line 193, in _write_source
    new_lines = self._highlight(source).splitlines()
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/_pytest/_io/terminalwriter.py", line 228, in _highlight
    if highlighted[-1] == "\n" and source[-1] != "\n":
                                   ~~~~~~^^^^
IndexError: string index out of range
Error: Process completed with exit code 1.

That's on Linux, but locally I can reproduce on reproduce on macOS arm64 as well:

$ pytest mne/_fiff/tests/test_meas_info.py --pdb
...
INTERNALERROR>   File "/Users/larsoner/Applications/MNE-Python/1.6.0_0/.mne-python/lib/python3.11/site-packages/_pytest/reports.py", line 88, in toterminal
INTERNALERROR>     longrepr_terminal.toterminal(out)
INTERNALERROR>   File "/Users/larsoner/Applications/MNE-Python/1.6.0_0/.mne-python/lib/python3.11/site-packages/_pytest/_code/code.py", line 1140, in toterminal
INTERNALERROR>     element[0].toterminal(tw)
INTERNALERROR>   File "/Users/larsoner/Applications/MNE-Python/1.6.0_0/.mne-python/lib/python3.11/site-packages/_pytest/_code/code.py", line 1170, in toterminal
INTERNALERROR>     entry.toterminal(tw)
INTERNALERROR>   File "/Users/larsoner/Applications/MNE-Python/1.6.0_0/.mne-python/lib/python3.11/site-packages/_pytest/_code/code.py", line 1257, in toterminal
INTERNALERROR>     self._write_entry_lines(tw)
INTERNALERROR>   File "/Users/larsoner/Applications/MNE-Python/1.6.0_0/.mne-python/lib/python3.11/site-packages/_pytest/_code/code.py", line 1247, in _write_entry_lines
INTERNALERROR>     tw._write_source(source_lines, indents)
INTERNALERROR>   File "/Users/larsoner/Applications/MNE-Python/1.6.0_0/.mne-python/lib/python3.11/site-packages/_pytest/_io/terminalwriter.py", line 193, in _write_source
INTERNALERROR>     new_lines = self._highlight(source).splitlines()
INTERNALERROR>                 ^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/larsoner/Applications/MNE-Python/1.6.0_0/.mne-python/lib/python3.11/site-packages/_pytest/_io/terminalwriter.py", line 228, in _highlight
INTERNALERROR>     if highlighted[-1] == "\n" and source[-1] != "\n":
INTERNALERROR>                                    ~~~~~~^^^^
INTERNALERROR> IndexError: string index out of range

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB post_mortem (IO-capturing turned off) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /Users/larsoner/Applications/MNE-Python/1.6.0_0/.mne-python/lib/python3.11/site-packages/_pytest/_io/terminalwriter.py(228)_highlight()
-> if highlighted[-1] == "\n" and source[-1] != "\n":
(Pdb) p source
''
(Pdb) 

Not really minimal (at all!) and it's not trivial to get the MNE tests running locally, so let me know if additional debugging at my end would be helpful. But maybe it's enough to know that source has length zero so source[-1] will fail here?

Env info:

$ pip list | grep pytest
pytest                        8.0.0rc1
pytest-cov                    4.1.0
pytest-harvest                1.10.4
pytest-qt                     4.2.0
pytest-timeout                2.2.0

[bluetech]

Thanks for trying the release candidate!

But maybe it's enough to know that source has length zero so source[-1] will fail here?

Yes, I think the fix is just a matter of adding

if not source:
   return source

to the top of _highlight.

For a regression test, it would nice to reproduce the empty source, I will try see how to do it.

---

Regarding the error E OSError: Need a file for File but found a directory at /home/runner/mne_data/MNE-testing-data/CTF/testdata_ctf.ds, this is something expected from your side, not something in the new pytest in itself, right?

[larsoner]

Regarding the error E OSError: Need a file for File but found a directory at /home/runner/mne_data/MNE-testing-data/CTF/testdata_ctf.ds, this is something expected from your side, not something in the new pytest in itself, right?

Yes that part is just at our end, thanks for the quick response!