Rails builds fail if BUNDLE_WITHOUT="'development:test'" (note the extra quotes)

[schneems]

Rails builds fail if BUNDLE_WITHOUT="'development:test'" (note the extra quotes), however they don't fail at bundle install time, they fail later.

Internal reference: https://heroku.support/1352094

An example of a problem deploy:

$ heroku config:set BUNDLE_WITHOUT="'development:test'"
$ git push heroku
Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
remote: Updated 1845 paths from 939fafb01
remote: Compressing source files... done.
remote: Building source:
remote:
remote: -----> Building on the Heroku-20 stack
remote: -----> Determining which buildpack to use for this app
remote: -----> Ruby app detected
remote: -----> Installing bundler 2.3.25
remote: -----> Removing BUNDLED WITH version in the Gemfile.lock
remote: -----> Compiling Ruby/Rails
remote: -----> Using Ruby version: ruby-2.7.8
remote: -----> Installing dependencies using bundler 2.3.25
remote:        Running: BUNDLE_WITHOUT=''development:test'' BUNDLE_PATH=vendor/bundle BUNDLE_BIN=vendor/bundle/bin BUNDLE_DEPLOYMENT=1 bundle install -j4
# ...
remote:        Bundle complete! 107 Gemfile dependencies, 598 gems now installed.
remote:        Gems in the groups 'development' and 'test' were not installed.
remote:        Bundled gems are installed into `./vendor/bundle`
remote:        Bundle completed (210.52s)
remote:        Cleaning up the bundler cache.
remote:
remote: ###### WARNING:
remote:
remote:        Installing a default version (20.9.0) of Node.js.
remote:        This version is not pinned and can change over time, causing unexpected failures.
remote:
remote:        Heroku recommends placing the `heroku/nodejs` buildpack in front of
remote:        `heroku/ruby` to install a specific version of node:
remote:
remote:        https://devcenter.heroku.com/articles/ruby-support#node-js-support
remote:
remote: -----> Installing node-v20.9.0-linux-x64
remote: -----> Detecting rake tasks
remote:
remote:  !
remote:  !     Could not detect rake tasks
remote:  !     ensure you can run `$ bundle exec rake -P` against your app
remote:  !     and using the production group of your Gemfile.
remote:  !     /tmp/build_177d58be/vendor/bundle/ruby/2.7.0/gems/bundler-2.3.25/lib/bundler/definition.rb:507:in `materialize': Could not find <redacted>  in locally installed gems (Bundler::GemNotFound)
remote:  !     from /tmp/build_177d58be/vendor/bundle/ruby/2.7.0/gems/bundler-2.3.25/lib/bundler/definition.rb:187:in `specs'
remote:  !     from /tmp/build_177d58be/vendor/bundle/ruby/2.7.0/gems/bundler-2.3.25/lib/bundler/definition.rb:244:in `specs_for'
remote:  !     from /tmp/build_177d58be/vendor/bundle/ruby/2.7.0/gems/bundler-2.3.25/lib/bundler/runtime.rb:18:in `setup'
remote:  !     from /tmp/build_177d58be/vendor/bundle/ruby/2.7.0/gems/bundler-2.3.25/lib/bundler.rb:161:in `setup'
remote:  !     from /tmp/build_177d58be/vendor/bundle/ruby/2.7.0/gems/bundler-2.3.25/lib/bundler/setup.rb:20:in `block in <top (required)>'
remote:  !     from /tmp/build_177d58be/vendor/bundle/ruby/2.7.0/gems/bundler-2.3.25/lib/bundler/ui/shell.rb:136:in `with_level'
remote:  !     from /tmp/build_177d58be/vendor/bundle/ruby/2.7.0/gems/bundler-2.3.25/lib/bundler/ui/shell.rb:88:in `silence'
remote:  !     from /tmp/build_177d58be/vendor/bundle/ruby/2.7.0/gems/bundler-2.3.25/lib/bundler/setup.rb:20:in `<top (required)>'
remote:  !     from /tmp/build_177d58be/vendor/ruby-2.7.8/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in `require'
remote:  !     from /tmp/build_177d58be/vendor/ruby-2.7.8/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in `require'
remote:  !     from /tmp/build_177d58be/config/boot.rb:3:in `<top (required)>'
remote:  !     from /tmp/build_177d58be/bin/rake:2:in `require_relative'
remote:  !     from /tmp/build_177d58be/bin/rake:2:in `<main>'
remote:  !

It seems that if bundle exec rake -P should fail with a given BUNDLE_WITHOUT that the bundle install before it should have failed as well or possibly a warning emitted (stating that we're telling it to skip environments that don't exist, namely 'development (note the leading quote) and test' (note the trailing quote).

This is a fairly obscure failure mode though. As far back as I can remember I don't believe I've seen it. However it's quite tricky to debug as most people's brains will read '' as " in this output:

remote:        Running: BUNDLE_WITHOUT=''development:test'' BUNDLE_PATH=vendor/bundle BUNDLE_BIN=vendor/bundle/bin BUNDLE_DEPLOYMENT=1 bundle install -j4

It's worth noting that this install message doesn't show the issue:

remote:        Gems in the groups 'development' and 'test' were not installed.
remote:        Bundled gems are installed into `./vendor/bundle`

But locally on another project I see that it does:

$ BUNDLE_WITHOUT="'development:test'" bundle _2.3.25_ install
# ...
Gems in the groups ''development' and 'test'' were not installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
1 installed gem you directly depend on is looking for funding.
  Run `bundle fund` for details