Build for older macOS versions using bump-formula-pr process in a custom tap

[rgoldberg]

In a custom tap, is there a way to make the bump-formula-pr process (using a pr-pull label, etc.) build for older macOS versions (like macOS 12, 11, 10.15, 10.14, …), either by:

• configuring the usage of the GitHub Workflows generated by brew tap-new
• editing the GitHub Workflows generated by brew tap-new
• obtaining different Workflows from elsewhere that handle this
• etc.

Is there a way to easily have the custom tap formula upload bottles to a GitHub Release in the main project repo and either:

• not upload the bottles to the custom tap repo, then use URLs pointing to bottles in the main project repo instead of in the custom tap
• also upload the bottles to the custom tap repo, then use URLs pointing to bottles in the custom tap

I prefer the former, but would settle for the latter if the former wouldn't be simple or advisable.

[ipatch]

for the first part of your question,

In a custom tap, is there a way to make the bump-formula-pr process (using a pr-pull label, etc.) build for older macOS versions (like macOS 12, 11, 10.15, 10.14, …), either by:

if you want to build bottles for older versions of macos ie. 10.14, etc, etc. you'll need to setup a self-hosted runner running that macos version ie. macos 10.14. be advised though if you core tap uses that latest upstream homebrew-core you'll have to build bottles for each and every formula or point the homebrew-core tap to a specific revision which still contains bottles for that macos version. this will need to be done for each macos version you plan on supporting, as out of the box the workflows provided by homebrew are only setup to use github runners that are officially maintained by github.

[rgoldberg]

@ipatch & @carlocab Thanks for the info & for looking at this.

Building bottles for older macOS versions:

The project in question is mas. The current mas version (1.8.7) is written in Swift 5.7 & is configured to be able to run on macOS 10.13+. The prior version (1.8.6) is configured to be able to run on macOS 10.11+.

I'd like our custom tap to build bottles for each newly released version of mas that can be used for macOS 10.13+, but also offer the legacy mas 1.8.6 for macOS 10.11 & 10.12. Is that possible?

According to Apple docs, Xcode 14.2+ (currently 16.1) on macOS 12.5+ can build Swift 5.7 executables that can run on macOS versions back to 10.13.

Assuming Apple's build system works as advertised, is there any need to build our custom tap bottles on any macOS version other than macOS 15? (GitHub macos-15 runners currently default to using Xcode 16, which can build for macOS 10.13)

I plan to build an arm64 executable that would be used for macOS 11+ on Apple Silicon, and an x86_64 executable that would be used for macOS 10.13+ on Intel.

Can you let me know if there is anything wrong with the following, or if there is a better way to handle all this?

• In the test.yml workflow generated by brew tap-new:
  • Modify it to rename the bottles generated by brew test-bot --only-formulae on macos-15:
    • arm64 bottle from *sequoia* to *big_sur*
    • x86_64 bottle from *sequoia* to *high_sierra*
• In the publish.yml workflow generated by brew tap-new:
  • brew pr-pull --debug --tap="$GITHUB_REPOSITORY" "$PULL_REQUEST" will properly upload the bottles from the workflow artifact to the new GitHub Release in the custom tap, despite the macOS versions in the file names being old (i.e. big_sur & high_sierra)
  • Modify it to append links to existing mas 1.8.6 bottles for el_capitan & sierra.
    • Would those bottles be able to be served from a different root_url than the newly built bottles for the newly released version of mas?
    • If not, would it be terrible if old 1.8.6 bottles were copied to the GitHub Release for the new mas version, so they could be served from there?
• After those 2 bottles are published to the GitHub Release in the custom tap repo, they will be used for their respective platform for their respective macOS version & newer

Serving bottles from elsewhere than GitHub Release in custom tap

You can ignore my question about serving the bottles from the main project repo (which I think you took to mean Homebrew/homebrew-core, but by which I meant mas-cli/mas; sorry for any confusion). I now have permission to store the bottles in the usual place: a GitHub Release in our custom tap repo: mas-cli/homebrew-tap (I know that's a nonstandard name, but it predates me, and it might be hard to change it to mas-cli/homebrew-mas).

[ipatch]

my knowledge with swift is limited to say the least. but if your targeting macos 10.13 (high sierra) as your minimum build target, github does not provide a runner image for that macos version. so you'll need to setup a custom github runner ie. a self-hosted runner running that version of macos. then if your formula in question relies on dependencies from homebrew to build the formula in question each one of those dependencies will need to have bottle to build your formula in question or brew test-bot will fail to build your bottle. also you'll need to make you setup your project ie. (cmake or autotools) to use the proper SDK, which shouldn't be a big deal if you go the self-hosted route because macos 10.13 will use the SDK provided by the version of Xcode that runs on macos 10.13.

things begin to get difficult if you want to use a github provided runner and then you use a non standard SDK ie. an older SDK say (10.13) on macos sequoia as your project and all it's deps will require targeting that SDK.

good luck.

[rgoldberg]

@ipatch Thanks for the info.

Do you know anything about the 2 generated GitHub workflows that I mentinoed? If so, is my plan workable?

Also, I'm confused: why would I need to build on High Sierra?

Swift 5.7 requires Xcode 14.2+, which can only run on macOS 12.5+. You cannot build Swift on every target macOS on which it can run. So I cannot build on High Sierra for High Sierra.

Sequoia can run Xcode 14.2+ through current Xcode (i.e. 16.x), all of which can build for 10.13+. A Swift executable built on Sequoia should run on 10.13+, as long as the project is configured to work on 10.13+, which mine is.

I guess I'm mystified about why Homebrew ties the build macOS version to the deploy macOS version. It might be necessary for certain languages, but for many languages (like Swift, Java, etc.), the build & deploy macOS versions shouldn't need to be the same.

[ipatch]

I guess I'm mystified about why Homebrew ties the build macOS version to the deploy macOS version. It might be necessary for certain languages, but for many languages (like Swift, Java, etc.), the build & deploy macOS versions shouldn't need to be the same.

i'd suggest you look the homebrew documentation / api, you'll see that if your project is using cmake or autotools when homebrew installs from a bottle, the bottle will represent what was defined in the formula file for the package that was installed, and from what i have gathered if a homebrew formula in the core tap uses cmake, then it will use the std_cmake_args

https://rubydoc.brew.sh/OS/Mac/Formula.html#std_cmake_args-instance_method
https://rubydoc.brew.sh/

here's another discussion that reiterates what i stated above

https://github.com/orgs/Homebrew/discussions/4831#discussioncomment-7224896ew-core

taking a look at your tap / formula it looks like your formula file doesn't use / rely on any external deps provided by homebrew so that's at least a positive thing.

https://github.com/mas-cli/homebrew-tap/blob/main/Formula/mas.rb

if you have access to a 10.13 box you could always build / install on it and see if thing work as intended.

[rgoldberg]

@ipatch Thanks again for the info.

mas builds using swift build. I haven't seen anything about it using autotools or cmake.

For now, I imagine that we can just build on macOS-15, rename the bottles as appropriate, and use them for macOS 10.13+.

The main questions are if brew pr-pull will properly upload bottles for old macOS versions, and if/how the custom tap can serve old mas versions for old macOS versions.

mas will probably depend on jq & util-linux in the future because it will use them to format terminal output. In that case, users on older macOS versions can either build those formulae via Homebrew, or install the executables some other way.

[Bo98]

Homebrew doesn't really support cross-builds at the moment, mainly because it is not something we support in homebrew-core and nobody has really requested or developed a solution for third-party taps.

With that said we do use cross-builds for the Homebrew Portable Ruby tap, using an undocumented Homebrew environment variable to "fake" the host OS version to be the deployment OS. I'm not really sure if this approach scales well for third-party tap use cases - it was a quick hack solely tested for a Ruby build without much thought put into it. Someone should probably design a better system for these use cases.

An alternative solution here could be to support "unversioned" macOS bottles like arm64_macos. There has been some interest there, more so than cross-builds, but that also isn't supported today. And for the use case of cross-builds specifically: I'm not entirely convinced it makes sense for native binaries when a single cross-built bottle for the oldest deployment target would make more sense, given newer macOS support falling back to that anyway.

Right now the approach taken by some is to compile the binary externally and to just download & install the binary in the formula. I agree this is not great and anything that avoids those scenarios would be an improvement.

[rgoldberg]

@Bo98 Thanks for the info.

Not sure if you saw my reply to @ipatch in the thread above, which offers more details about my plan & my questions about it. Assuming that the executable built by my Swift build on macOS 15 can indeed be deployed back to macOS 10.13, the overall gist of my proposal is to:

• on a macos-15 runner, build 2 bottles, one each for arm64 & x86_64.
• rename the arm64 bottle from *sequoia* to *big_sur*
• rename the x86_64 bottle from *sequoia* to *high_sierra*
• hope that brew pr-pull --debug --tap="$GITHUB_REPOSITORY" "$PULL_REQUEST" in my custom tap's publish.yml publishe the bottles to the GitHub Release in the custom tap repo
• Somehow either:
  • Insert URLs to existing legacy mas 1.8.6 bottles into the bottles block of the newly modified custom tap formula, to provide the newest version of mas that runs on macOS 10.11 & 10.12.
  • copy existing existing legacy mas 1.8.6 bottles from the 1.8.6 release to the GitHub Release for the new version in the custom tap repo, then insert the URLs to the copied 1.8.6 bottles into the bottle block of the newly modified custom tap formula.

Must any files within any of the bottles be updated or removed to be reassigned to a different macOS version? Must any files be added to any of the bottles?

Any help would be appreciated.

[rgoldberg]

@Bo98 How hard do you think it would be to add in the ability to cross build bottles? (by that, I mean, e.g., building on macOS 15, but specifying that the built bottle is for macOS 10.13+)

Could a first-time contributor (outside of minor formula improvements & bumps) like myself get it done in a reasonable amount of time?

Would it be accepted?