only store version of packages during load time when generating sysimage

[KristofferC]

I am looking at the number of stat calls during code loading (precompilation). One somewhat significant source of these is the lookup of the version field in the project file of packages (added in #44318) that happens as soon as a package is loaded. Due to the way the precompilation process work (IIUC), the number of these lookup scales quadratically with the depth of the dependency tree of a package.

This PR changes it so that the lookup only happens when a call to pkgversion actually requests the version. An exception is when a sysimage is generated (for example from PackageCompiler) because then we want to also store the version to support the functionality in JuliaLang/Pkg.jl#3002.

The script I am using to measure the number of stat calls is:

❯ rm -rf ~/.julia/compiled/v1.9;  strace -f -o stat_gr~/juliaarray/julia --project -e 'using GR_jll'

❯ cat stat_gr | grep -c statx

executed in an environment with only GR_jll installed. The results are

❯ cat stat_master | grep -c statx
23789

❯ cat stat_master | grep -c statx
21236

This may not seem like a huge reduction but further work on #46690 brings the baseline down heavily where this absolute difference is quite significant.

[timholy]

Due to the way the precompilation process work (IIUC), the number of these lookup scales quadratically with the depth of the dependency tree of a package.

Couldn't this be solved by storing the version in the .ji header?

[KristofferC]

Couldn't this be solved by storing the version in the .ji header?

That probably also works. I am not sure I see a big advantage doing that over this PR though.

[vtjnash]

LGTM. But didn't we already have to parse this TOML file as part of picking the package to load?

[KristofferC]

But didn't we already have to parse this TOML file as part of picking the package to load?

I don't think so. All that is required is the current Project + Manifest to compute what is loaded.