Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Freeze when switching to pkg mode, when Pkg has not been loaded already. #56206

Closed
sgaure opened this issue Oct 17, 2024 · 9 comments · Fixed by #56214 or #56215
Closed

Freeze when switching to pkg mode, when Pkg has not been loaded already. #56206

sgaure opened this issue Oct 17, 2024 · 9 comments · Fixed by #56214 or #56215
Labels
regression 1.11 Regression in the 1.11 release REPL Julia's REPL (Read Eval Print Loop)

Comments

@sgaure
Copy link

sgaure commented Oct 17, 2024

This causes a freeze:

$ julia --startup=no -p 1 -t 1
(@v1.11) pkg> status

https://discourse.julialang.org/t/version-1-11-1-argument-issue/121408

Julia Version 1.11.1
Commit 8f5b7ca12ad (2024-10-16 10:53 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 16 × AMD Ryzen 7 2700X Eight-Core Processor
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, znver1)
Threads: 1 default, 0 interactive, 1 GC (on 16 virtual cores)
Environment:
  JULIA_NUM_THREADS = 16
  JULIA_EDITOR = emacs -nw
@IanButterworth
Copy link
Member

IanButterworth commented Oct 17, 2024

Can you try

$ julia --startup=no -p 1 -t 1 -ie "@time using Pkg"

And then see if pkg mode works

@sgaure
Copy link
Author

sgaure commented Oct 17, 2024

Yes. It works if Pkg is imported first. Also with julia ... -L file.jl where file.jl contains import Pkg.

@IanButterworth
Copy link
Member

Is the freeze indefinite or does it just take a while?

@giordano giordano added the regression 1.11 Regression in the 1.11 release label Oct 17, 2024
@giordano
Copy link
Contributor

Is this related to #56054 (and #56207), even if condition to trigger the issue is different?

@sgaure
Copy link
Author

sgaure commented Oct 17, 2024

At least an hour, and can't be interrupted by either ^C or ^Z, and does not use CPU. gdb reveals that thread 1 is in uv__io_poll, whereas the other threads are in pthread_cond_wait (blas threads).

@sgaure
Copy link
Author

sgaure commented Oct 17, 2024

It could be related to the -O3 stuff. I can replicate the -O3 hang, but it too goes away if I preload Pkg with a suitable -L file.jl.

@IanButterworth
Copy link
Member

@vtjnash seems specific to require_stdlib. What can we do to further debug this?

@IanButterworth IanButterworth changed the title Freeze in package mode. Freeze when switching to pkg mode, when Pkg has not been loaded already. Oct 17, 2024
@IanButterworth IanButterworth added the REPL Julia's REPL (Read Eval Print Loop) label Oct 17, 2024
vtjnash added a commit that referenced this issue Oct 17, 2024
CacheFlags could get set, but were never propagated to the target
process, so the result would be unusable. Additionally, the debug and
optimization levels were not synchronized with the sysimg, causing a
regression in pkgimage usability after moving out stdlibs.

Fixes #56207
Fixes #56054
Fixes #56206
@vtjnash
Copy link
Member

vtjnash commented Oct 17, 2024

According to lldb, it happens because load_pkg wants to take the Base.require_lock, but it is forbidden to call into any of the requires logic while holding that lock (that must lead to deadlocks).

Aside: it looks like that REPLExt.__init__ also does wildly unsafe things, like mutate a lot of global arrays without any locks concurrently with expecting them to be simultaneously be getting modified by the REPL itself.

vtjnash added a commit that referenced this issue Oct 17, 2024
CacheFlags could get set, but were never propagated to the target
process, so the result would be unusable. Additionally, the debug and
optimization levels were not synchronized with the sysimg, causing a
regression in pkgimage usability after moving out stdlibs.

Fixes #56207
Fixes #56054
Fixes #56206
vtjnash added a commit that referenced this issue Oct 17, 2024
vtjnash added a commit that referenced this issue Oct 17, 2024
CacheFlags could get set, but were never propagated to the target
process, so the result would be unusable. Additionally, the debug and
optimization levels were not synchronized with the sysimg, causing a
regression in pkgimage usability after moving out stdlibs.

Fixes #56207
Fixes #56054
Fixes #56206
vtjnash added a commit that referenced this issue Oct 17, 2024
Also fix waiting on the package_locks being always a no-op here.

Fixes #56206
@IanButterworth
Copy link
Member

Aside: it looks like that REPLExt.init also does wildly unsafe things, like mutate a lot of global arrays without any locks concurrently with expecting them to be simultaneously be getting modified by the REPL itself.

#54890

vtjnash added a commit that referenced this issue Oct 17, 2024
CacheFlags could get set, but were never propagated to the target
process, so the result would be unusable. Additionally, the debug and
optimization levels were not synchronized with the sysimg, causing a
regression in pkgimage usability after moving out stdlibs. The printing
also failed to meaningfully represent the settings for the same reasons.

Fixes #56207
Fixes #56054
Fixes #56206
vtjnash added a commit that referenced this issue Oct 17, 2024
CacheFlags could get set, but were never propagated to the target
process, so the result would be unusable. Additionally, the debug and
optimization levels were not synchronized with the sysimg, causing a
regression in pkgimage usability after moving out stdlibs. The printing
also failed to meaningfully represent the settings for the same reasons.

Also fixes a concurrency bug in loading.jl tests that was failing.

Fixes #56207
Fixes #56054
Fixes #56206
KristofferC pushed a commit that referenced this issue Oct 18, 2024
vtjnash added a commit that referenced this issue Oct 18, 2024
CacheFlags could get set, but were never propagated to the target
process, so the result would be unusable. Additionally, the debug and
optimization levels were not synchronized with the sysimg, causing a
regression in pkgimage usability after moving out stdlibs.

Fixes #56207
Fixes #56054
Fixes #56206
KristofferC pushed a commit that referenced this issue Oct 21, 2024
CacheFlags could get set, but were never propagated to the target
process, so the result would be unusable. Additionally, the debug and
optimization levels were not synchronized with the sysimg, causing a
regression in pkgimage usability after moving out stdlibs.

Fixes #56207
Fixes #56054
Fixes #56206

(cherry picked from commit 82b1506)
KristofferC pushed a commit that referenced this issue Oct 21, 2024
CacheFlags could get set, but were never propagated to the target
process, so the result would be unusable. Additionally, the debug and
optimization levels were not synchronized with the sysimg, causing a
regression in pkgimage usability after moving out stdlibs.

Fixes #56207
Fixes #56054
Fixes #56206
maleadt pushed a commit that referenced this issue Oct 21, 2024
CacheFlags could get set, but were never propagated to the target
process, so the result would be unusable. Additionally, the debug and
optimization levels were not synchronized with the sysimg, causing a
regression in pkgimage usability after moving out stdlibs.

Fixes #56207
Fixes #56054
Fixes #56206

(cherry picked from commit 82b1506)
KristofferC pushed a commit that referenced this issue Oct 24, 2024
CacheFlags could get set, but were never propagated to the target
process, so the result would be unusable. Additionally, the debug and
optimization levels were not synchronized with the sysimg, causing a
regression in pkgimage usability after moving out stdlibs.

Fixes #56207
Fixes #56054
Fixes #56206

(cherry picked from commit 82b1506)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
regression 1.11 Regression in the 1.11 release REPL Julia's REPL (Read Eval Print Loop)
Projects
None yet
4 participants