Add support for not triggering invalidation in load_preference()

[JamesWrigley]

In some cases it's desirable to use Preferences.jl as a generic preferences mechanism without wanting it to affect precompilation.

Some examples:

• preference to disable initial cache of fieldtypes(#991) timholy/Revise.jl#992
• change from using a build script to auto installing kernels and providing a jupyter path update function JuliaLang/IJulia.jl#1215

Mostly generated by Claude 🤖

[codecov]

Codecov Report

❌ Patch coverage is 83.33333% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.82%. Comparing base (fb07fd2) to head (1f17881).
⚠️ Report is 3 commits behind head on master.

Files with missing lines	Patch %	Lines
src/Preferences.jl	83.33%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master      #90      +/-   ##
==========================================
+ Coverage   92.65%   93.82%   +1.16%     
==========================================
  Files           2        2              
  Lines         177      178       +1     
==========================================
+ Hits          164      167       +3     
+ Misses         13       11       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[JamesWrigley]

(bump)

[JamesWrigley]

(bump)

[staticfloat]

I think this is useful, but I’d like to name it something a little more clear; perhaps “disable_invalidation”, to communicate to the user that (a) this is disabling something, and (b) it’s not really stopping this from happening at compile time.

[JamesWrigley]

Sure, renamed in 1f17881.

[Keno]

I don't know that I like this. If you're not using the preference to affect the ouput, then why are you reading it at compile time? If the issue is that it's hard to separate the code paths, then I'd rather have an API that returns a fixed value at precompile time every time and only does the dynamic thing at runtime.

[staticfloat]

I can see the use for this for e.g. functions that get called incidentally during compilation, but are not intended to be invalidating. I wanted this kwarg to sound a little dangerous (hence the disable_ naming).

I'm not certain if the "fixed value at precompile time" idea is better; I do like reducing potential sources of nondeterminism, although I am a little worried that such behavior will be endlessly confusing to developers. Ideally though, very few people are using such a feature.

[Keno]

I'm not certain if the "fixed value at precompile time" idea is better; I do like reducing potential sources of nondeterminism, although I am a little worried that such behavior will be endlessly confusing to developers.

The consequence for getting this kwarg wrong may include arbitrary undefined behavior. That will be more than a little confusing.

[JamesWrigley]

What if we made a new function for it named e.g. unsafe_load_preference()?

[Keno]

But why, I don't get it. Either the preference value matters or it doesn't. It it doesn, it's a big problem. If it doesn't, then any arbitrary value should be fine, so might as well make that the interface.

[JamesWrigley]

If we're going to replace the preference with an arbitrary value during precompilation then we might as well just make it throw an exception, because that's what most likely will happen when the user code tries to do something with it (like compare it with a boolean), and that's not really useful.

Like, in IJulia I would use it for getting the Jupyter path and I would call the notebook() function during precompilation to precompile it, but if getting the preference returns an arbitrary value or throws an exception then I can't do that. Even though the Jupyter path doesn't need to invalidate at all.

[Keno]

Even though the Jupyter path doesn't need to invalidate at all.

Doesn't it? What happens if jupyter is not installed, so it early-outs and now you've got a poisoned cache.

[JamesWrigley]

That's an expected case to handle so it would go for a fallback path, but in any case how would it cause a 'poisoned' cache? A preference not being set is normal, an early exit just means some code won't get executed just like it wouldn't if some boolean variable was not set.

[Keno]

The precompile wouldn't run, so you'd be perpetually left with a slow starting IJulia for reasons that nobody can figure out.

[JamesWrigley]

I wouldn't call that a 'poisoned' cache; it's just incomplete, exactly the same situation as if the setting was obtained from an environment variable or scratch file. Unless I'm missing something, using Preferences here does not make things any worse in that scenario?

To give more context, the main reason I'd like to use Preferences is because it allows for persistent storage in a single, familiar format to users. One could also use Scratch.jl for persistence but that's not standardized, and if Preferences is already used for some settings that do require cache invalidation then it makes sense to re-use that for other settings even though they may not require cache invalidation.

[JamesWrigley]

(I acknowledge I'm biased a bit towards user-friendliness, which is why I think this is a reasonable tradeoff)

[Keno]

Yes, there's a million ways to do unsound things during precompile without the compiler catching you (although the compiler will get better at it and we may start doing things like running in a sanitized environment), but the issue here is that Preferences was specifically designed to be precompile-safe. I don't think it's a good idea to add a keyword argument that pokes a giant hole into those safety guarantees. Being precompile-safe is pretty much the whole point of this design - it's not a quirky afterthought.

[JamesWrigley]

I dunno, as I said I feel it's an ok tradeoff 🤷 It's intentionally not accessible through the @load_preference macro and there is a clear warning about it. That being said, I leave the final decision to you and the other maintainers, if you feel it's better to revert then I understand.

[JamesWrigley]

Can I get a final yea/nay before starting to use it?

[Keno]

I think we should revert this PR. @staticfloat ?

[topolarity]

Just to add my $.02:

I think there is a use case to compute paths that do not "meaningfully" affect the result of your pre-compilation (to the extent that they are different locations of "effectively" the same file). This is how the __init__ / OncePerProcess usage in JLL's works today, and notably Overrides.toml provides functionality similar to what @JamesWrigley wants where you can provide an alternate path to an executable / library.

That said, I think the Preferences API makes it too easy to not consider version swaps, etc. that might make your package logic incorrect. It also puts package authors (such as IJulia) in the uncomfortable position of wanting not to invalidate since "most" path overrides won't change program (major / minor) versions, but some will which leaves surprising / broken behavior floating around for users as @Keno explains.

The Artifacts / JLL system is clearer that if you provide an "override" it should be for exactly the same file / version, which makes it relatively safe for packages to query API's, data structures, etc. at compile-time and expect the "same" thing at runtime.

Being precompile-safe is pretty much the whole point of this design - it's not a quirky afterthought.

FWIW unfortunately, there are some notable holes in the pre-compile dependency tracking. Some are just bugs (JuliaLang/julia#59344), but other gaps may have been left on purpose due to wanting to avoid generating "accidental" dependencies in __init__ (see commit message of JuliaLang/julia@c211591).

I am still in favor of being more strict about these though, so I will probably push forward a "fix" soon.

[staticfloat]

I’ve thought about this more and I agree with Keno that we shouldn’t give access to more footguns if we can help it.

I think a happy middle ground would be to change the API here such that disable_invalidation actually means “always return the default value during compilation”. Maybe we change the keyword argument name to “force_compiletime_default_value” or some such.

[JamesWrigley]

Ok, that works for me 👍 I'll make a PR in a bit.