Can't import matplotlib on linux: module '_tkinter' has no attribute '__file__'

[konstin]

Basic matplotlib usage such as import matplotlib.pyplot as plt; plt.plot(list(range(10)), list(range(10))) crashes with module '_tkinter' has no attribute '__file__'.

On system python, _tkinter is a shared library (/usr/lib/python3.8/lib-dynload/_tkinter.cpython-38-x86_64-linux-gnu.so), while in these python builds it's a builtin module which does not have __file__. matplotlib expects tkinter to be an .so it can load:
https://github.com/matplotlib/matplotlib/blob/f25c2d025fa8f70d6688e7acf4b85e4eb7590299/src/_tkagg.cpp#L310-L322

To reproduce:

wget https://github.com/indygreg/python-build-standalone/releases/download/20220502/cpython-3.10.4+20220502-x86_64_v3-unknown-linux-gnu-pgo+lto-full.tar.zst
tar xf cpython-3.10.4+20220502-x86_64_v3-unknown-linux-gnu-pgo+lto-full.tar.zst
python/install/bin/python3 -m venv venv
venv/bin/pip install matplotlib
venv/bin/python -c "import matplotlib.pyplot as plt; plt.plot(list(range(10)), list(range(10)))"

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/konsti/monotrail/pbsi/venv/lib/python3.10/site-packages/matplotlib/pyplot.py", line 2769, in plot
    return gca().plot(
  File "/home/konsti/monotrail/pbsi/venv/lib/python3.10/site-packages/matplotlib/pyplot.py", line 2274, in gca
    return gcf().gca(**kwargs)
  File "/home/konsti/monotrail/pbsi/venv/lib/python3.10/site-packages/matplotlib/pyplot.py", line 867, in gcf
    return figure()
  File "/home/konsti/monotrail/pbsi/venv/lib/python3.10/site-packages/matplotlib/pyplot.py", line 808, in figure
    manager = new_figure_manager(
  File "/home/konsti/monotrail/pbsi/venv/lib/python3.10/site-packages/matplotlib/pyplot.py", line 326, in new_figure_manager
    _warn_if_gui_out_of_main_thread()
  File "/home/konsti/monotrail/pbsi/venv/lib/python3.10/site-packages/matplotlib/pyplot.py", line 316, in _warn_if_gui_out_of_main_thread
    if (_get_required_interactive_framework(_get_backend_mod())
  File "/home/konsti/monotrail/pbsi/venv/lib/python3.10/site-packages/matplotlib/pyplot.py", line 217, in _get_backend_mod
    switch_backend(dict.__getitem__(rcParams, "backend"))
  File "/home/konsti/monotrail/pbsi/venv/lib/python3.10/site-packages/matplotlib/pyplot.py", line 268, in switch_backend
    switch_backend(candidate)
  File "/home/konsti/monotrail/pbsi/venv/lib/python3.10/site-packages/matplotlib/pyplot.py", line 288, in switch_backend
    class backend_mod(matplotlib.backend_bases._Backend):
  File "/home/konsti/monotrail/pbsi/venv/lib/python3.10/site-packages/matplotlib/pyplot.py", line 289, in backend_mod
    locals().update(vars(importlib.import_module(backend_name)))
  File "/home/konsti/monotrail/pbsi/python/install/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/konsti/monotrail/pbsi/venv/lib/python3.10/site-packages/matplotlib/backends/backend_tkagg.py", line 1, in <module>
    from . import _backend_tk
  File "/home/konsti/monotrail/pbsi/venv/lib/python3.10/site-packages/matplotlib/backends/_backend_tk.py", line 24, in <module>
    from . import _tkagg
AttributeError: module '_tkinter' has no attribute '__file__'. Did you mean: '__name__'?

Running TCL_LIBRARY="$(pwd)/python/install/lib/tcl8.6" venv/bin/python -c "import matplotlib.pyplot as plt; plt.plot(list(range(10)), list(range(10)))" instead as suggested by the docs makes no difference.

EDIT: I'm mainly using the standalone builds by libloading libpython.so and launching one of the very high level c api commands, so i'd be happy about any workaround that works with libloading or importlib hackery from rust.

[indygreg]

This is similar to indygreg/PyOxidizer#69.

My general response to this is: __file__ is documented as optional by Python: any code requiring __file__ is bugged and should instead be using a mechanism described in https://pyoxidizer.readthedocs.io/en/pyoxidizer-0.20.0/oxidized_importer_resource_files.html.

But this case is more complex than simple resource file handling. It looks like matplotlib is opening the library in order for C code to get a handle on tk function symbols. 🤯

This is highly unorthodox, as it assumes C extensions are standalone libraries (an implementation detail of the Python distribution) and that said libraries export tk symbols (again, an implementation detail).

I'll likely need to converse with a matplotlib developer on how to best support this. I would like to have a compatibility story here, as matplotlib is a popular package and I'd like it to be supported.

[konstin]

fixed in matplotlib 🎉

[engnatha]

Sorry to open an old issue again, but from my following of the comments, it seems like the matplotlib fix was just to gracefully skip over the tkinter backend when it couldn't import it the way it expected. Is there a way to get a standalone build that has the C extensions for tkinter? Installing an interpreter with a tool like pyenv does this automatically.

[TsaiTung-Chen]

I'm building an GUI app using Tkinter and Matplotlib to plot some data. The TkAgg backend is required to achieve this. I tried some dirty work to workaround this. Such as copy and paste a _tkinter SO file to the dynload folder and set the __file__ attribute to that SO file path. However, I finally gave up using this standalone Python distribution due to this kind of issue potentially.

[driftregion]

I have encountered the same issue as @engnatha and @TsaiTung-Chen .

bazel-contrib/rules_python#2540
matplotlib/matplotlib#23074