Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Limit libraries to be included in the bundle #742

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

simonbyrne
Copy link
Collaborator

If julia_libdir() == "/usr/lib" then it will attempt to bundle the whole directory into the app. Fixes #16.

@jlapeyre, @OliverEvans96, @feanor12, @alexriss: would you please be able to try out this branch (]add PackageCompiler#sb/limit-bundle) and see if it fixes the problem?

If `julia_libdir() == "/usr/lib"` then it will attempt to bundle the whole directory into
the app. Fixes #16.
@alexriss
Copy link

Thank you! This seems to fix the bundle_julia_libraries function. But now I get an error in bundle_cert:

IOError: open("/usr/bin/../share/julia/cert.pem", 0, 0): no such file or directory (ENOENT)

at PackageCompiler.jl:1206

I am somehow not able to find a cert.pem belonging to Julia on my system (Manjaro 22, Julia installed via the package manager).

@simonbyrne
Copy link
Collaborator Author

Thanks @alexriss: I'm not sure how the certificates are used, so don't know what the fix is (can we just drop them?)

FWIW, the recommendation is to use the binaries from https://julialang.org/downloads/, as system packages often don't include patches or other fixes to upstream libraries that Julia requires.

@codecov
Copy link

codecov bot commented Nov 21, 2022

Codecov Report

Merging #742 (59b6528) into master (376e3bf) will decrease coverage by 0.57%.
The diff coverage is 87.50%.

@@            Coverage Diff             @@
##           master     #742      +/-   ##
==========================================
- Coverage   93.46%   92.88%   -0.58%     
==========================================
  Files           2        2              
  Lines         520      520              
==========================================
- Hits          486      483       -3     
- Misses         34       37       +3     
Impacted Files Coverage Δ
src/PackageCompiler.jl 93.29% <87.50%> (-0.42%) ⬇️
src/juliaconfig.jl 88.37% <0.00%> (-2.33%) ⬇️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@feanor12
Copy link

I tried to compile a small demo project with aur/juliaup, aur/julia-bin and the community/julia package.

With the new branch of PackageCompiler aur/juliaup and aur/julia-bin work.

the community/julia package was throwing an error:

ERROR: IOError: open("/usr/bin/../share/julia/cert.pem", 0, 0): no such file or directory (ENOENT)
Stacktrace:
 [1] cp
   @ ./file.jl:376 [inlined]
 [2] bundle_cert
   @ ~/.julia/packages/PackageCompiler/Lz0Nj/src/PackageCompiler.jl:1211 [inlined]
 [3] create_app(package_dir::String, app_dir::String; executables::Nothing, precompile_execution_file::Vector{String}, precompile_statements_file::Vector{String}, incremental::Bool, filter_stdlibs::Bool, force::Bool, c_driver_program::String, cpu_target::String, include_lazy_artifacts::Bool, sysimage_build_args::Cmd, include_transitive_dependencies::Bool)
   @ PackageCompiler ~/.julia/packages/PackageCompiler/Lz0Nj/src/PackageCompiler.jl:712
 [4] create_app(package_dir::String, app_dir::String)
   @ PackageCompiler ~/.julia/packages/PackageCompiler/Lz0Nj/src/PackageCompiler.jl:685
 [5] top-level scope
   @ REPL[3]:1

The community/julia package, however, does not ship cert.pem at all.

@alexriss
Copy link

I have done some more tests.

If I install the official binaries on my Manjaro distro, everything works, even with the previous version without any changes.

With the version from the package manager, everything is more complicated. I have adapted the bundle_cert function as follows:

function bundle_cert(dest_dir)
    cert_path = joinpath(Sys.BINDIR, "..", "share", "julia", "cert.pem")
    if Sys.isunix() && !isfile(cert_path)
        cert_path = "/etc/ssl/cert.pem"
    end
    share_path = joinpath(dest_dir, "share", "julia")
    mkpath(share_path)
    isfile(cert_path) && cp(cert_path, joinpath(share_path, "cert.pem"))
end

I am not quite sure if this is the right cert.pem file. However, I get a different error now for some packages that rely on these certs:

Failed to precompile Blink [ad839575-38b3-5650-b840-f874b8c74a25] to /home/riss/.julia/compiled/v1.8/Blink/jl_z6KvWg.
ERROR: LoadError: InitError: SystemError: opening file "/usr/share/julia/cert.pem": No such file or directory

I have traced the problem down further, it seems to come from the package MozillaCACerts_jll.jl, which seems to be generated by JLLWrappers.jl.

Now somewhere there is a function MozillaCACerts_jll.find_artifact_dir() which does basically this:

global artifact_dir = dirname(Sys.BINDIR)
global cacert = normpath(Sys.BINDIR, Base.DATAROOTDIR, "julia", "cert.pem")

and it ends up with:

julia> Base.DATAROOTDIR
"../share"

julia> Sys.BINDIR
"/usr/bin"

julia> MozillaCACerts_jll.artifact_dir
"/usr"

So this does not seem to be a PackageCompiler problem anymore, since I cannot compile these packages directly under my Julia instance, see also this thread. And yes, this should have been the first thing to try, would have saved quite some time.

tl;dr: Seems like the solution is to not use the community/julia package for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ERROR: open: permission denied (EACCES)
3 participants