Custom sysimage builds but still UnsupportedPythonError

[marius311]

I can succesfully use python -m julia.sysimage sys.so to build a custom sysimage, but I still get an UnsupportedPythonError when trying to use it. The only suspicious thing is the Error: build.log for PackageCompiler but I still seem to get the sysimage. I have Python 3.7.3 (miniconda), pyjulia 0.5.6, PyCall v1.91.4, and julia 1.5. Below is the trace of the whole session reproducing this. Any suggestions for how to track down the issue?

~$ python -m julia.sysimage sys.so
 Activating new environment at `/tmp/tmp.pyjulia.sysimage.tfsrekcs/compiler_env/Project.toml`
[ Info: Installing PackageCompiler...
   Updating registry at `~/.julia/registries/General`
   Updating git-repo `https://github.com/JuliaRegistries/General.git`
  Resolving package versions...
Updating `/tmp/tmp.pyjulia.sysimage.tfsrekcs/compiler_env/Project.toml`
  [9b87118b] + PackageCompiler v1.2.1
Updating `/tmp/tmp.pyjulia.sysimage.tfsrekcs/compiler_env/Manifest.toml`
  [9b87118b] + PackageCompiler v1.2.1
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [b77e0a4c] + InteractiveUtils
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [44cfe95a] + Pkg
  [de0858da] + Printf
  [3fa0cd96] + REPL
  [9a3f8284] + Random
  [ea8e919c] + SHA
  [9e88b42a] + Serialization
  [6462fe0b] + Sockets
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode
┌ Error: build.log for PackageCompiler [9b87118b-4619-50d2-8e1e-99f35a4d4d9d] not found
└ @ Main ~/miniconda3/lib/python3.7/site-packages/julia/install-packagecompiler.jl:19
[ Info: Loading PackageCompiler...
[ Info: PackageCompiler is successfully installed at /tmp/tmp.pyjulia.sysimage.tfsrekcs/compiler_env
 Activating environment at `/tmp/tmp.pyjulia.sysimage.tfsrekcs/compiler_env/Project.toml`
[ Info: Loading PackageCompiler...
[ Info: Installing PyCall...
 Activating new environment at `/tmp/tmp.pyjulia.sysimage.tfsrekcs/Project.toml`
   Updating registry at `~/.julia/registries/General`
   Updating git-repo `https://github.com/JuliaRegistries/General.git`
  Resolving package versions...
Updating `/tmp/tmp.pyjulia.sysimage.tfsrekcs/Project.toml`
  [438e738f] + PyCall v1.91.4
Updating `/tmp/tmp.pyjulia.sysimage.tfsrekcs/Manifest.toml`
  [8f4d0f93] + Conda v1.4.1
  [682c06a0] + JSON v0.21.1
  [1914dd2f] + MacroTools v0.5.5
  [69de0a69] + Parsers v1.0.10
  [438e738f] + PyCall v1.91.4
  [81def892] + VersionParsing v1.2.0
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [8ba89e20] + Distributed
  [b77e0a4c] + InteractiveUtils
  [8f399da3] + Libdl
  [37e2e46d] + LinearAlgebra
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [a63ad114] + Mmap
  [de0858da] + Printf
  [9a3f8284] + Random
  [9e88b42a] + Serialization
  [6462fe0b] + Sockets
  [8dfed614] + Test
  [4ec0a83e] + Unicode
[ Info: Compiling a temporary system image without `PyCall`...
[ Info: PackageCompiler: creating system image object file, this might take a while...
[ Info: Compiling system image...
[ Info: PackageCompiler: creating system image object file, this might take a while...
[ Info: System image is created at /home/marius/sys.so

~$ ipython
Python 3.7.3 (default, Mar 27 2019, 22:11:17) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.13.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import julia                                                                                                              

In [2]: julia.Julia(sysimage="sys.so")                                                                                            
---------------------------------------------------------------------------
UnsupportedPythonError                    Traceback (most recent call last)
<ipython-input-2-2289e25f03ec> in <module>
----> 1 julia.Julia(sysimage="sys.so")

~/miniconda3/lib/python3.7/site-packages/julia/core.py in __init__(self, *args, **kwargs)
    678 
    679     def __init__(self, *args, **kwargs):
--> 680         self.__julia = Julia(*args, **kwargs)
    681 
    682     __init__.__doc__ = Julia.__init__.__doc__

~/miniconda3/lib/python3.7/site-packages/julia/core.py in __init__(self, init_julia, jl_init_path, runtime, jl_runtime_path, debug, **julia_options)
    481             logger.debug("compiled_modules = %r", options.compiled_modules)
    482             if not (options.compiled_modules == "no" or is_compatible_python):
--> 483                 raise UnsupportedPythonError(jlinfo)
    484 
    485             self.api.init_julia(options)

UnsupportedPythonError: It seems your Julia and PyJulia setup are not supported.

Julia executable:
    julia
Python interpreter and libpython used by PyCall.jl:
    /home/marius/miniconda3/bin/python3
    /home/marius/miniconda3/lib/libpython3.7m.so.1.0
Python interpreter used to import PyJulia and its libpython.
    /home/marius/miniconda3/bin/python
    /home/marius/miniconda3/lib/libpython3.7m.so.1.0

Your Python interpreter "/home/marius/miniconda3/bin/python"
is statically linked to libpython.  Currently, PyJulia does not fully
support such Python interpreter.

The easiest workaround is to pass `compiled_modules=False` to `Julia`
constructor.  To do so, first *reboot* your Python REPL (if this happened
inside an interactive session) and then evaluate:

    >>> from julia.api import Julia
    >>> jl = Julia(compiled_modules=False)

Another workaround is to run your Python script with `python-jl`
command bundled in PyJulia.  You can simply do:

    $ python-jl PATH/TO/YOUR/SCRIPT.py

See `python-jl --help` for more information.

For more information, see:

    https://pyjulia.readthedocs.io/en/latest/troubleshooting.html

[marius311]

Ok, so loading with Julia(sysimage="sys.so") is what fails, but using instead #310 (comment) it works.

[Jasha10]

I've submitted a patch in #482.
With that patch, Julia(sysimage="sys.so") does not throw UnsupportedPythonError.