Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

on_system conditionals not respected for :all manifest tab cause issues in keg relocation #18695

Open
4 tasks done
calvinit opened this issue Nov 1, 2024 · 6 comments
Open
4 tasks done
Labels
bug Reproducible Homebrew/brew bug help wanted We want help addressing this

Comments

@calvinit
Copy link

calvinit commented Nov 1, 2024

brew gist-logs <formula> link OR brew config AND brew doctor output

# brew config

HOMEBREW_VERSION: 4.4.3-60-g745596b
ORIGIN: https://github.com/Homebrew/brew
HEAD: 745596b55579f3b9acb198f6aaa4c8aba0ef9844
Last commit: 5 hours ago
Core tap HEAD: bff87d7ce5eb665fdd9b28e1ff8ee510d8cf352b
Core tap last commit: 2 hours ago
Core tap branch: openjdk@8_1.8.0-432
Core tap JSON: 01 Nov 06:23 UTC
Core cask tap origin: https://github.com/Homebrew/homebrew-cask.git
Core cask tap HEAD: 649f802bb9d4ba0a703685b982d661b570173bee
Core cask tap last commit: 6 hours ago
Core cask tap JSON: 01 Nov 06:23 UTC
HOMEBREW_PREFIX: /usr/local
HOMEBREW_BOTTLE_DOMAIN: https://mirrors.tuna.tsinghua.edu.cn/homebrew-bottles
HOMEBREW_CASK_OPTS: []
HOMEBREW_EDITOR: code
HOMEBREW_MAKE_JOBS: 12
HOMEBREW_NO_ENV_HINTS: set
HOMEBREW_SORBET_RUNTIME: set
Homebrew Ruby: 3.3.5 => /usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/3.3.5/bin/ruby
CPU: dodeca-core 64-bit kabylake
Clang: 16.0.0 build 1600
Git: 2.47.0 => /usr/local/bin/git
Curl: 8.7.1 => /usr/bin/curl
macOS: 15.1-x86_64
CLT: 16.1.0.0.1.1729049160
Xcode: 16.1

brew doctor

Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry or file an issue; just ignore this. Thanks!

Warning: Some installed kegs have no formulae!
This means they were either deleted or installed manually.
You should find replacements for the following formulae:
  [email protected]

Warning: Some installed formulae are deprecated or disabled.
You should find replacements for the following formulae:
  [email protected]
  [email protected]
  [email protected]
  [email protected]
  [email protected]
  [email protected]

Verification

What were you trying to do (and why)?

Submit a PR to upgrade openjdk@8 to version 1.8.0-432.

What happened (include all command output)?

brew test --retry --verbose embulk failed because it selected the incorrect version of openjdk@21 (it should actually select openjdk@8 on the Intel architecture):

on_arm do
  depends_on "openjdk@21"
end
on_intel do
  depends_on "openjdk@8"
end
/usr/local/Homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/bin/bundle clean
  ==> Testing embulk
  ==> Downloading https://search.maven.org/remotecontent?filepath=org/jruby/jruby-complete/9.4.8.0/jruby-complete-9.4.8.0.jar
  ==> Downloading from https://repo1.maven.org/maven2/org/jruby/jruby-complete/9.4.8.0/jruby-complete-9.4.8.0.jar
  /usr/bin/env /usr/local/Homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent Homebrew/4.4.3-60-g745596b\ \(Macintosh\;\ Intel\ Mac\ OS\ X\ 14.7\)\ curl/8.7.1 --header Accept-Language:\ en --fail --retry 3 --remote-time --output /Users/brew/Library/Caches/Homebrew/downloads/db8894972d158eba6af122d132a624d09cfd39c16a8db6b30173a70899e18307--jruby-complete-9.4.8.0.jar.incomplete --location https://repo1.maven.org/maven2/org/jruby/jruby-complete/9.4.8.0/jruby-complete-9.4.8.0.jar
    % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                   Dload  Upload   Total   Spent    Left  Speed
  
    0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  100 31.9M  100 31.9M    0     0   100M      0 --:--:-- --:--:-- --:--:--  100M
  ==> Verifying checksum for 'db8894972d158eba6af122d132a624d09cfd39c16a8db6b30173a70899e18307--jruby-complete-9.4.8.0.jar'
  cp -p /Users/brew/Library/Caches/Homebrew/downloads/db8894972d158eba6af122d132a624d09cfd39c16a8db6b30173a70899e18307--jruby-complete-9.4.8.0.jar /private/tmp/jruby-complete-20241101-7906-5s7uk4/jruby-complete-9.4.8.0.jar
  ==> /usr/local/Cellar/embulk/0.11.5/bin/embulk -X jruby=file:///private/tmp/embulk-test-20241101-7906-yw06o/jruby-complete-9.4.8.0.jar gem install embulk --version 0.11.5
  /usr/local/Cellar/embulk/0.11.5/bin/embulk: line 3: /usr/local/opt/openjdk@21/libexec/openjdk.jdk/Contents/Home/bin/java: No such file or directory
  /usr/local/Cellar/embulk/0.11.5/bin/embulk: line 3: exec: /usr/local/opt/openjdk@21/libexec/openjdk.jdk/Contents/Home/bin/java: cannot execute: No such file or directory
  ==> Testing embulk (again)
  cp -p /Users/brew/Library/Caches/Homebrew/downloads/db8894972d158eba6af122d132a624d09cfd39c16a8db6b30173a70899e18307--jruby-complete-9.4.8.0.jar /private/tmp/jruby-complete-20241101-8200-84vjdr/jruby-complete-9.4.8.0.jar
  ==> /usr/local/Cellar/embulk/0.11.5/bin/embulk -X jruby=file:///private/tmp/embulk-test-20241101-8200-lwwcgy/jruby-complete-9.4.8.0.jar gem install embulk --version 0.11.5
  /usr/local/Cellar/embulk/0.11.5/bin/embulk: line 3: /usr/local/opt/openjdk@21/libexec/openjdk.jdk/Contents/Home/bin/java: No such file or directory
  /usr/local/Cellar/embulk/0.11.5/bin/embulk: line 3: exec: /usr/local/opt/openjdk@21/libexec/openjdk.jdk/Contents/Home/bin/java: cannot execute: No such file or directory
  Error: embulk: failed
  An exception occurred within a child process:
    BuildError: Failed executing: /usr/local/Cellar/embulk/0.11.5/bin/embulk -X jruby=file:///private/tmp/embulk-test-20241101-8200-lwwcgy/jruby-complete-9.4.8.0.jar gem install embulk --version 0.11.5
  /usr/local/Homebrew/Library/Homebrew/formula.rb:3082:in `block in system'
  /usr/local/Homebrew/Library/Homebrew/formula.rb:3018:in `open'
  /usr/local/Homebrew/Library/Homebrew/formula.rb:3018:in `system'
  /usr/local/Homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11630/lib/types/private/methods/call_validation.rb:270:in `bind_call'
  /usr/local/Homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11630/lib/types/private/methods/call_validation.rb:270:in `validate_call'
  /usr/local/Homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11630/lib/types/private/methods/_methods.rb:277:in `block in _on_method_added'
  /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/e/embulk.rb:65:in `block in <class:Embulk>'
  /usr/local/Homebrew/Library/Homebrew/formula.rb:2817:in `block (3 levels) in run_test'
  /usr/local/Homebrew/Library/Homebrew/extend/kernel.rb:539:in `with_env'
  /usr/local/Homebrew/Library/Homebrew/formula.rb:2816:in `block (2 levels) in run_test'
  /usr/local/Homebrew/Library/Homebrew/formula.rb:1199:in `with_logging'
  /usr/local/Homebrew/Library/Homebrew/formula.rb:2815:in `block in run_test'
  /usr/local/Homebrew/Library/Homebrew/mktemp.rb:90:in `block in run'
  /usr/local/Homebrew/Library/Homebrew/mktemp.rb:90:in `chdir'
  /usr/local/Homebrew/Library/Homebrew/mktemp.rb:90:in `run'
  /usr/local/Homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11630/lib/types/private/methods/call_validation.rb:270:in `bind_call'
  /usr/local/Homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11630/lib/types/private/methods/call_validation.rb:270:in `validate_call'
  /usr/local/Homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11630/lib/types/private/methods/_methods.rb:277:in `block in _on_method_added'
  /usr/local/Homebrew/Library/Homebrew/formula.rb:3141:in `mktemp'
  /usr/local/Homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11630/lib/types/private/methods/call_validation.rb:270:in `bind_call'
  /usr/local/Homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11630/lib/types/private/methods/call_validation.rb:270:in `validate_call'
  /usr/local/Homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11630/lib/types/private/methods/_methods.rb:277:in `block in _on_method_added'
  /usr/local/Homebrew/Library/Homebrew/formula.rb:2809:in `run_test'
  /usr/local/Homebrew/Library/Homebrew/test.rb:48:in `block in <main>'
  /usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/3.3.5/lib/ruby/3.3.0/timeout.rb:1[86](https://github.com/Homebrew/homebrew-core/actions/runs/11624571402/job/32374214285?pr=194719#step:3:87):in `block in timeout'
  /usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/3.3.5/lib/ruby/3.3.0/timeout.rb:41:in `handle_timeout'
  /usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/3.3.5/lib/ruby/3.3.0/timeout.rb:195:in `timeout'
  /usr/local/Homebrew/Library/Homebrew/test.rb:54:in `<main>'

What did you expect to happen?

Select openjdk@8 correctly (@@HOMEBREW_JAVA@@).

Or

bottle do
  sha256 cellar: :..., arm: "xxxxxxxxxx"
  sha256 cellar: :..., x86_64: "yyyyyyyyyy"
end

Step-by-step reproduction instructions (by running brew commands)

==> brew fetch --retry embulk
==> brew install --only-dependencies embulk
==> brew install embulk
==> brew install --only-dependencies embulk
==> brew linkage --test embulk
==> brew linkage --cached --test --strict embulk
==> brew install --only-dependencies --include-test embulk
==> brew test --retry --verbose embulk

For more detailed logs, see here.

@calvinit calvinit added the bug Reproducible Homebrew/brew bug label Nov 1, 2024
@calvinit
Copy link
Author

calvinit commented Nov 1, 2024

https://formulae.brew.sh/formula/embulk

It’s necessary to distinguish between different architectures here.
QQ_1730446562652

@cho-m
Copy link
Member

cho-m commented Nov 1, 2024

In theory, it should not need multiple bottles as the Java path replacement is done at install time. The bottle should be using a placeholder, e.g. @@HOMEBREW_JAVA@@.

The issue appears to be combining dependencies into its variations isn't handled correctly for API.

When using a local tap, it seems okay, e.g.

brew info embulk --json=v2 --variations | jq '.formulae[].dependencies'
[
  "openjdk@21"
]arch -x86_64 brew info embulk --json=v2 --variations | jq '.formulae[].dependencies'
[
  "openjdk@8"
]
brew info embulk --json=v2 --variations | jq '.formulae[].variations.arm64_sequoia'
nullbrew info embulk --json=v2 --variations | jq '.formulae[].variations.sequoia'
{
  "dependencies": [
    "openjdk@8"
  ]
}arch -x86_64 brew info embulk --json=v2 --variations | jq '.formulae[].variations.arm64_sequoia'
{
  "dependencies": [
    "openjdk@21"
  ]
}arch -x86_64 brew info embulk --json=v2 --variations | jq '.formulae[].variations.sequoia'
null

However, the API data is generated based on single platform:

curl -sL "https://formulae.brew.sh/api/formula/embulk.json" | jq '.dependencies'
[
  "openjdk@21"
]

@carlocab carlocab changed the title Should a Formula that depends on different versions of openjdk for different architectures have multiple bottles? on_system conditionals not respected during API generation Nov 1, 2024
@carlocab carlocab transferred this issue from Homebrew/homebrew-core Nov 1, 2024
@carlocab carlocab added the install from api Relates to API installs label Nov 1, 2024
@carlocab
Copy link
Member

carlocab commented Nov 1, 2024

However, the API data is generated based on single platform:

That's a bug, but I wonder why it affects is in CI when we set HOMEBREW_NO_INSTALL_FROM_API.

@cho-m
Copy link
Member

cho-m commented Nov 1, 2024

As workaround, can just use same OpenJDK on embulk. I split dependency based on upstream status, but probably doesn't matter if we just use newer one.


For issue, could also be a general keg relocation problem. Would need to check on that code. The bottle itself has:

export JAVA_HOME="${JAVA_HOME:-@@HOMEBREW_JAVA@@}"
exec "${JAVA_HOME}/bin/java"  -jar "@@HOMEBREW_CELLAR@@/embulk/0.11.5/libexec/embulk-0.11.5.jar" "$@"

So as long as bottle pour substitutes right path then it should have worked.


EDIT: Another possibility is if it is reading tab from manifest.

@cho-m cho-m removed the install from api Relates to API installs label Nov 1, 2024
@cho-m
Copy link
Member

cho-m commented Nov 1, 2024

It is the manifest rather than JSON API. Sadly no idea how to handle this for accurate tab in all bottle. May need something similar to Perl's preferred option.

Another idea is skipping manifest's tab if OS/arch doesn't match, though this may not work if we remove info based on #17395


Tab from https://github.com/Homebrew/homebrew-core/pkgs/container/core%2Fembulk/290843166

"sh.brew.tab": "{ ... {\"full_name\":\"openjdk@21\",\"version\":\"21.0.5\",\"revision\":0,\"pkg_version\":\"21.0.5\",\"declared_directly\":true}],

@cho-m cho-m changed the title on_system conditionals not respected during API generation on_system conditionals not respected for :all manifest tab cause issues in keg relocation Nov 1, 2024
@carlocab carlocab added the help wanted We want help addressing this label Nov 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Reproducible Homebrew/brew bug help wanted We want help addressing this
Projects
None yet
Development

No branches or pull requests

4 participants
@cho-m @calvinit @carlocab and others