Regression in 3.10 `cabal build`: the `extra-prog-path` ends up first in the PATH, shadowing `build-tools`

[andreasabel]

My installdir is set to ~/.cabal/bin in my ~/.cabal/config file. I have alex-3.4.0.1 installed there.
I also have set extra-prog-path: ~/.cabal/bin set in my config.
I am trying to build this project which requires a newer alex:

cabal-version:  1.12
name:           cabal-bug-build-tool
version:        0
build-type:     Simple

library
  exposed-modules:  Foo.Lex
  default-language: Haskell2010
  build-depends:    base, array
  build-tools:      alex >=3.5.1.0

Foo/Lex.x contains some lexer definition to be processed with alex.

With cabal-3.10.2.1 and current master I am getting this error:

$ cabal build
...(building alex-3.5.1.0)...
Error: cabal-3.10.2.1: The program 'alex' version >=3.5.1.0 is required but
the version found at /Users/abel/.cabal/bin/alex is version 3.4.0.1

Building succeeds with cabal-3.8 and older.

Running with -v3 I see that the extra-prog-path ends up before the path for alex-3.5.1.0 in the PATH component of the environment.

("PATH","/Users/abel/.cabal/bin:/Users/abel/.cabal/store/ghc-9.8.2-b74f/lx-3.5.1.0-60b880d8/bin:/Users/abel/.cabal/bin:...

cabal exec instead calls the correct alex for this project:

$ cabal-master exec -- alex --version
Alex version 3.5.1.0, (c) 2003 Chris Dornan and Simon Marlow

I checked a bit the source of cabal and it seems that cabal exec does not receives the installdir, so it cannot interfere here.

The source files to reproduce it (on my system) are at https://github.com/andreasabel/cabal-bug-build-tool .

However, I could not reproduce the problem on the machines provided by GHA (see the workflow file for my attempt).
There, the PATH is correct:
> ("PATH","/home/runner/.cabal/store/ghc-9.8.2/alex-3.5.1.0-e-alex-075f98fe20ffc7a07799cfa46f51c24d5236956a51dccba7409f8f308610de2a/bin:/home/runner/.cabal/bin

Update: I could now reproduce it, supplying the extra-prog-path. It ends up first in the PATH:
https://github.com/andreasabel/cabal-bug-build-tool/actions/runs/8099486076/job/22135333052#step:10:1936

("PATH","/home/runner/.cabal/bin:/home/runner/.cabal/store/ghc-9.8.2/alex-3.5.1.0-e-alex-075f98fe20ffc7a07799cfa46f51c24d5236956a51dccba7409f8f308610de2a/bin:/home/runner/.cabal/bin:

CI proves that it is a regression in 3.10, as 3.8 succeeds there as well: https://github.com/andreasabel/cabal-bug-build-tool/actions/runs/8099613006

[andreasabel]

I found the original issue now that I tracked it to extra-prog-path:

• Installed executables take precedence over build-tool-depends #8951
• Don't add extra-prog-path to ~/.config/cabal/config (#8951) #8952 (hotfix)

The problem was originally introduced in:

• prepend rather than append extra prog path #8506

There is a PR by @gbaz claiming to fix the problem:

• Fix precedence for PATH for build-tools-depends #8972

But possibly the following PR touching the same lines undid the fix:

• Fix extra-prog-path propagation in the codebase. #9527

ATTN: @jasagredo @fendor

(Classic whack-a-mole game, if you ask me. Only more tests in the suite can save us from evil here...)

[andreasabel]

Maybe a simple list of paths (that can either be pre- or appended) does not satisfy all the different requirements.
It would make sense to maintain the provenance: Where did each entry come from?

• system path
• extra-prog-path
• bindir/ installdir (are they relevant?)
• paths coming from build-tool installations

The entries could then be weighted differently for different purposes. E.g. when trying to run a build-tool, consider the paths coming from build-tool installations first.

What error diagnostics concerns (I spent almost 5 hours in this bug hunt), it could be good if cabal, on error, displays all versions of the build tool it founds on directories contained in the PATH, not just the first one it rejected. E.g. it could say:

alex >= 3.5.1.0 is required, but the version found at is 3.4.0.1
found the following other version, but they are shadowed

• version 3.5.1.0 at .cabal/store....
• ...

If there was be the provenance for each path, a real good diagnostics of the problem could be given.

[gbaz]

In general, the order, from least precedence to most precedence should be

1. system path
2. extra-prog-path
3. paths from build-tool-depends

[jasagredo]

Sorry for not noticing this in my PR. I don't usually use build-tool-depends and it seems none of the tests do.

Indeed I am pretty sure I prepend the extra-prog-path to every other path. I am however surprised that cabal exec finds it, I would have thought it had the same behavior.

I am currently a bit swamped with other stuff so I might not be able to work on this in the short term. The solution you are proposing looks appealing, however I don't know if there will be many different cases where we can take advantage of the provenance. Shouldn't build-tool-depends be always the first ones in the path?

[jasagredo]

Just for clarity

bindir/installdir (are they relevant?)

These paths are not relevant. They might be in the system PATH but that is all, they won't be included in the effective PATH just by their own.