Details about brew install bottle pouring

[luigiberrettini]

Disclaimer

I am quite new to the macOS world so I apologize in case I ask things that are well known to anyone else.

Goal

When binary packages are provided for some software, there is the possibility to:

• install them manually (handling requirements/dependencies usually manually)
• install them via a package manager
• put them in any directory and run them (dependencies available on the OS might need some attention)

The MariaDB team provides binaries for Windows and Linux and, once extracted, they do not need to be compiled, they just work.
If they provided binaries for macOS the behaviour would probably be the same.

I was hoping to use the precompiled binaries in Homebrew bottles to be able to run MariaDB from any directory but I am having a hard time in doing so.

Issues

I have been trying to grasp how Homebrew works in detail without success.

I read that when brew --prefix is the standard usr/local bottles are poured i.e. binaries inside them are used.
A compilation occurs if:

• an Homebrew installation (the only one or one out of many) is being used in a non-standard location
• parameters are passed when calling brew install <formula>

Let's take into consideration the [email protected] formula:

• formula page
• GitHub code
• Bintray bottle

I installed the formula on a Catalina machine expecting the installation to pour the bottle i.e. to place the binaries in brew --cellar add a link to them in brew --prefix and maybe set some system configuration.

After downloading and extracting the Catalina bottle I found out that not only text files had @@HOMEBREW_*@@ variables replaced (sounds reasonable) but also binaries differed as can be seen in the log of the Travis CI build job.

Considerations and help request

I understand that one possibility is building binaries on my own and this would probably allow to use them as it happens for Windows and Linux.

Nonetheless I would like to understand what happens when a brew install <formula> is performed i.e.:

• if the install function is called whether the bottle is poured or a compilation occurs
• if variable substitution is performed also, in some way in binaries, or they are changed via a quicker make than the one happening on compilation

Since it is not possible to decide which version of a formula to install (like with Linux package managers) and providing a formula file results in a full compilation as well as using a separate Homebrew installation, it would be nice to be able to manually reproduce what bottle pouring does.

In cases like the MariaDB one, the software already provides for the possibility of multiple installations in different non-standard directories by means of command-line parameters: it would be great to extract a bottle and run the software.

I hope you can give me help, if not achieving the goal, at least understanding what Homebrew does when it pours a bottle and how it changes the extracted binaries: in which way and with which command/function of the formula code.

[jonchang]

The full details of this are in keg_relocate.rb, and OS-specific behavior is in extend/os/{mac,linux}/keg_relocate.rb. We don't really have any documentation beyond the source code as these are implementation details of Homebrew.

Note that the [email protected] bottle is marked as non-relocatable (missing cellar :any). You can determine which files are causing our continuous integration to think that mariadb is non-relocatable by running brew install --build-bottle [email protected] followed by brew bottle [email protected].

[luigiberrettini]

Looking at the code it seems that, when a brew install results in a bottle being poured, the following steps are involved:

1. body of install (entry point)
2. call to install_formula
3. body of install_formula
4. call to FormulaInstaller.install
5. FormulaInstaller.install
6. call to FormulaInstaller.pour
7. body of FormulaInstaller.pour
8. call to Keg.replace_placeholders_with_locations
9. body of Keg.replace_placeholders_with_locations
10. call to OS specific relocate_dynamic_linkage
11. body of macOS relocate_dynamic_linkage

I guess relocate_dynamic_linkage is what allows the OpenSSL libs to be correctly linked at install time.

Even if users cannot expect a tool to expose the internals, it would be amazing to take advantage of this install bottles manually without recompiling all from scratch.

Regarding the [email protected] bottle

Symbolic links

[email protected]/10.3.25/bin/mysql.server -> ../support-files/mysql.server
[email protected]/10.3.25/bin/mysql_install_db -> ../scripts/mysql_install_db
[email protected]/10.3.25/bin/rcmysql -> ../../../../etc/init.d/mysql
[email protected]/10.3.25/bin/wsrep_sst_rsync_wan -> wsrep_sst_rsync
[email protected]/10.3.25/lib/libmariadb.dylib -> libmariadb.3.dylib
[email protected]/10.3.25/lib/libmysqlclient.a -> libmariadbclient.a
[email protected]/10.3.25/lib/libmysqlclient.dylib -> libmariadb.3.dylib
[email protected]/10.3.25/lib/libmysqlclient_r.a -> libmariadbclient.a
[email protected]/10.3.25/lib/libmysqlclient_r.dylib -> libmariadb.3.dylib

Files causing it to be non-relocatable
I tried to execute the suggested commands but the build was terminated as can be seen in the log of the Travis CI build job.