Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Put temporary directories and dependencies in a scratch space #48

Merged
merged 3 commits into from
Apr 25, 2021

Conversation

ericphanson
Copy link
Owner

@ericphanson ericphanson commented Apr 21, 2021

Someone sent me error logs once where it looked like somehow they didn't have write permission to the default place temporary directories are. I think a scratch space is probably a better place to put these.

closes #37

@codecov-commenter
Copy link

codecov-commenter commented Apr 21, 2021

Codecov Report

Merging #48 (a8b24cb) into master (3bb5cb0) will not change coverage.
The diff coverage is n/a.

❗ Current head a8b24cb differs from pull request most recent head 623862b. Consider uploading reports for the commit 623862b to get more accurate results
Impacted file tree graph

@@           Coverage Diff           @@
##           master      #48   +/-   ##
=======================================
  Coverage   88.56%   88.56%           
=======================================
  Files           6        6           
  Lines         551      551           
=======================================
  Hits          488      488           
  Misses         63       63           
Impacted Files Coverage Δ
src/MOI_wrapper.jl 82.62% <ø> (ø)
src/SDPAFamily.jl 75.00% <ø> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3bb5cb0...623862b. Read the comment docs.

@ericphanson ericphanson changed the title Put temporary directories in a scratch space Put temporary directories and dependencies in a scratch space Apr 21, 2021
@odow
Copy link
Contributor

odow commented Apr 22, 2021

I came here from slack. Why are these binaries JLL resistant?

@ericphanson
Copy link
Owner Author

Why are these binaries JLL resistant?

That might've been too pessimistic, but I spent awhile in early 2020 trying without much luck. Since then SDPA has been built so it might be easier now. The current blocker is that Yggdrasil's QD hasn't been correctly built on linux (probably my fault; ref JuliaPackaging/Yggdrasil#755). For that reason it hasn't been registered, but I can confirm the bug:

pkg> add https://github.com/JuliaBinaryWrappers/QD_jll.jl
...
julia> using QD_jll
ERROR: InitError: could not load library "/home/ec2-user/.julia/artifacts/6e37ddbeb239f39d492e1eab1ddaf7c43b39ff59/lib/libqdmod.so"
/home/ec2-user/.julia/artifacts/6e37ddbeb239f39d492e1eab1ddaf7c43b39ff59/lib/libqdmod.so: undefined symbol: _ZN7dd_real8_ndigitsE
Stacktrace:
 [1] dlopen(s::String, flags::UInt32; throw_error::Bool)
   @ Base.Libc.Libdl ./libdl.jl:114
 [2] dlopen (repeats 2 times)
   @ ./libdl.jl:114 [inlined]
 [3] __init__()
   @ QD_jll ~/.julia/packages/QD_jll/pav65/src/wrappers/x86_64-linux-gnu-cxx11.jl:79
 [4] _include_from_serialized(path::String, depmods::Vector{Any})
   @ Base ./loading.jl:674
 [5] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String)
   @ Base ./loading.jl:760
 [6] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:998
 [7] require(uuidkey::Base.PkgId)
   @ Base ./loading.jl:914
 [8] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:901
during initialization of module QD_jll

Note that using QD_jll works on MacOS. That build script is based off of https://github.com/ericphanson/QD_Builder.

I also have a builder based on the one for SDPA here: https://github.com/ericphanson/SDPA_GMP_Builder/blob/master/build_tarballs.jl that builds the three binaries, SDPA-GMP, -DD and -QD. I actually build GMP in that builder because at the time I couldn't get it to work with the existing GMP (which is linked into julia). I think that situation may have improved though. We also use patched source code to change the printing so you can actually get full precision solutions out (...since printing is actually how you communicate problems to and solutions from the binary since there isn't a API interface as far as I understand. So these are all just commandline executables, not libraries that actually need to link with julia).

@ericphanson
Copy link
Owner Author

The other issue is WSL; we've got binaries that Jiazheng somehow managed to compile locally on WSL on his laptop and uploaded it to

"https://github.com/ericphanson/SDPA_GMP_Builder/raw/master/deps/sdpa_gmp_wsl"
"https://github.com/JiazhengZhu/sdpa-dd/raw/master/sdpa_dd"
"https://github.com/JiazhengZhu/sdpa-qd/raw/master/sdpa_qd"

(from the build script). It seems like even though standard linux binaries should work on WSL, they don't. Maybe that's been fixed by now though...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Use Scratch spaces instead of temporary directories?
3 participants