Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Precompilation: Limit the default number of tasks to 16 #2552

Merged
merged 2 commits into from
May 6, 2021
Merged

Precompilation: Limit the default number of tasks to 16 #2552

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
8436f1f
limit the default number of precomp tasks to 16
IanButterworth May 5, 2021
b064c81
add JULIA_NUM_PRECOMPILE_TASKS to docs
IanButterworth May 5, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions src/API.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1009,6 +1009,7 @@ function precompile(ctx::Context; internal_call::Bool=false, strict::Bool=false,
# Windows sometimes hits a ReadOnlyMemoryError, so we halve the default number of tasks. Issue #2323
# TODO: Investigate why this happens in windows and restore the full task limit
default_num_tasks = Sys.iswindows() ? div(Sys.CPU_THREADS::Int, 2) + 1 : Sys.CPU_THREADS::Int + 1
default_num_tasks = min(default_num_tasks, 16) # limit for better stability on shared resource systems

num_tasks = parse(Int, get(ENV, "JULIA_NUM_PRECOMPILE_TASKS", string(default_num_tasks)))
parallel_limiter = Base.Semaphore(num_tasks)
Expand Down
3 changes: 2 additions & 1 deletion src/Pkg.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,8 @@ Precompile all the dependencies of the project in parallel.
!!! note
This method is called automatically after any Pkg action that changes the manifest.
Any packages that have previously errored during precompilation won't be retried in auto mode
until they have changed. To disable automatic precompilation set `ENV["JULIA_PKG_PRECOMPILE_AUTO"]=0`
until they have changed. To disable automatic precompilation set `ENV["JULIA_PKG_PRECOMPILE_AUTO"]=0`.
To manually control the number of tasks used set `ENV["JULIA_NUM_PRECOMPILE_TASKS"]`.

!!! compat "Julia 1.3"
This function requires at least Julia 1.3. On earlier versions
Expand Down
3 changes: 2 additions & 1 deletion src/REPLMode/command_declarations.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -336,7 +336,8 @@ not all manifest dependencies may be loaded by the top-level dependencies on the

This method is called automatically after any Pkg action that changes the manifest.
Any packages that have previously errored during precompilation won't be retried in auto mode
until they have changed. To disable automatic precompilation set the environment variable `JULIA_PKG_PRECOMPILE_AUTO=0`
until they have changed. To disable automatic precompilation set the environment variable `JULIA_PKG_PRECOMPILE_AUTO=0`.
To manually control the number of tasks used set the environment variable `JULIA_NUM_PRECOMPILE_TASKS`.
""",
],
PSA[:name => "status",
Expand Down