Tracking issue for OS X-specific development cost

[david-german-tri]

This issue is for gathering data on PRs that incur OS X-specific development cost. It is not a forum for discussing the implications. Examples of "cost" include:

Developer effort spent resolving OS X-specific issues.
Delayed merge due to discussing or resolving OS X-specific issues.
OS X-specific flakes in pre-merge continuous integration.
Rollbacks or forward fixes due to failures in OS X post-merge continuous integration.

Where possible, please reference an issue or PR.

[david-german-tri]

#4717 has demonstrated that we need to reproduce in Bazel the platform-specific compiler flag selection logic that we've had in CMake for a while. Specifically, we need to add flags with clang/OS X spellings and gcc/Linux spellings into respective CROSSTOOL stanzas, where if we only supported one platform we could just .bazelrc them.

Five hours so far today.

Update: This was ultimately resolved in #4772. I would guess the additional complexity from supporting Mac was about eight or ten hours of extra headache, in total.

[rpoyner-tri]

Spent maybe 2 hours working out OSX details for #4701.

[david-german-tri]

Grappling with adding IPOPT to Bazel (#4896 with dynamic libraries, #4905 with static libraries). Both approaches work with Linux, and both approaches have their own unique OS X gremlins. Dynamic libraries trigger some kind of deeply confusing loader path problem, which might be a bazel bug and might just be my fault: bazelbuild/bazel#507 (comment)

Static libraries get us past that problem, but then we discover that -lgfortran is only available on Mac via the homebrew gcc package, which in turn requires a wacky -L path. This path is somehow scribed into pkg-config files at ipopt build time - probably by parsing the output of /usr/local/bin/gfortran -print-search-dirs

Gave up after about 8 hours of effort.

[liangfok]

I'm estimating #4956 incurred 1 hour of developer time mostly due to context switching overhead.

To my knowledge, the Linux compilers chose the correct abs() and only the OSX compiler chose the wrong one.

[david-german-tri]

In #4905, I added a new test rule that only works on Linux, leading to build breakage on OS X. I've spent two hours so far this morning trying to figure out how to disable a test on OS X only.

[david-german-tri]

#4970 took about 6 hours, and is still an unsatisfactory kludge around the fact that Homebrew is not a proper distribution. It was superseded by #4983, which took another hour or so.

[david-german-tri]

I spent at least 20 hours developing enough background knowledge to write #5334.

[david-german-tri]

#5362 broke the build and wasted several people's time because we didn't
(and maybe still don't) fully understand the subtleties of a Mach-O specific linker optimization relating to exception unwinds. Perhaps 2 or 3 engineer hours so far.

[david-german-tri]

About 5 hours understanding how launchd interacts with environment variables, leading to #5570.

[david-german-tri]

In #5155, we chose to link Bazel builds statically by default, in part because dynamic linking on OS X is different, and a little complicated, and I hadn't figured out how to make it work. I didn't expect anyone to be disk-constrained in 2017, but it turns out some VM users are running out of disk on debug builds as a result of this decision. Not sure how to measure the cost incurred, but it's a cost.