Proposal to ship the JSON gem with precompiled binaries

[Edouard-chin]

👋 Hello, sorry if I should have created an issue first, wasn't sure where to propose this.

TL;DR I'd like to propose releasing the json gem with precompiled binaries built for different platforms and different ABI version (fat gem).

I'm currently working on a tool to help the Ruby community ship gems with precompiled binaries with the intent to make bundle install much faster for everyone. The main bottleneck when installing gems in a project is the compilation of native extensions.
After testing internally and with private gems, I'd like to get the feedback of a few gem maintainers.

---

What is it

cibuildgem modestly tries to follow the same approach as cibuildwheel from the python community. It works by compiling natively using CI runners (GitHub it the only supported vendor for now) and tries to be as easy to setup as possible. The json gem already relies on Rake Compiler for development purposes, and because cibuildgem piggyback on top of Rake Compiler, there is no extra configuration required.

The CI workflow in this commit was generated with the cibuildgem CLI which reads the gemspec and determine what ruby versions needs to be compiled and tested against. This is the resulting CI when this workflow runs: https://github.com/Shopify/json-1/actions/runs/19836110651

The tool is very new and I did many tests internally to make sure that it create binaries that can be used in other environments. For instance, I used it to precompile almost all gems that a new Rails application depends on and pushed them under a different "namespace" on my RubyGems, I then confirmed that the rails application was bootable using all those gems (I'm on MacOS).

I was hoping I could use the json gem and get the feedback of its maintainers in order to continue the development of cibuildgem and improve it.

Things you may want to know

• On GitHub, the earliest glibc version we can use is 2.35 (using Ubuntu 22). This means that any users that are on a linux distro that comes with a glibc version lower than this will not be able to install the gem with precompiled binaries. They'll have to install the "normal" gem and specify it in their gemfile
  (gem 'json', platform: 'ruby').

• This workflow gets triggered manually through the GitHub action page when you are ready to cut a release. You can optionally decide to let the tool do the release on RubyGems (using a chechbox on the GitHub UI). If you'd prefer doing it from your machine, you can download the gems from the GitHub action artifacts.

• The workflow in this commit compile binaries for MacOS silicon and linux x86_64. It should work on windows but I have not tested extensively to be confident. You should also be able to use macos intel and linus arm64 (GitHub started to offer this recently).

---

If you have any questions, I'm happy to answer! Any feedback would be amazing so feel free to point anything that you'd like to see this tool do or not do 😅 .

[byroot]

• They'll have to install the "normal" gem and specify it in their gemfile
  (gem 'json', platform: 'ruby').

I fear that's a bit of a deal breaker. ubunut-22 isn't that old by linux distros standards, plenty of "stable" distros with glibc versions way older than that.

Also the resulting gem is 7MB? That's massive! Even precompiled nokogiri is smaller than that. I don't understand what's the difference between your approach and the one existing precompiled gems use?

[Edouard-chin]

Thanks for looking Jean !

ubunut-22 isn't that old by linux distros standards

Just to clarify, on CI we compile with glibc 2.35, so a user on ubuntu 22 will be able to use our binaries fine. Any distro that comes with a glibc older than that, like in Ubuntu 20 (glibc 2.31) is not compatible.
FWIW, pretty much all existing gem with precompiled binaries are only compatible with glibc 2.31+ , this is what rake compiler dock compiles with.
If this is a dealbreaker and we want cibuildgem to be compatible with at least glibc 2.31, then I'll have to do a spike to fix this.

Also the resulting gem is 7MB?

I have a branch with a fix. The issue is about stripping debugging symbols LDFLAGS="-s". This is normally added in the extconf.rb but I think it's a sane default that the tool will add it automatically.

I don't understand what's the difference between your approach and the one existing precompiled gems use?

The main difference is cross vs native compilation. Rake Compiler Dock is great but @flavorjones mentions that maintaining it is complex and brittle. For more context, Kou and Mike discussed about this in ruby/rubygems#8645 (reply in thread)

The other difference is the ease of setup for the user. cibuildgem helps maintainers set up the CI compilation/test matrix which is one aspect of shipping gems with precompiled binaries with confidence that is time consuming to setup.