Skip to content

Speed up bundler/setup by using the raw Gemfile.lock information without extra processing whenever possible#5695

Merged
deivid-rodriguez merged 2 commits intomasterfrom
speedup-bundler-setup
Jul 9, 2022
Merged

Speed up bundler/setup by using the raw Gemfile.lock information without extra processing whenever possible#5695
deivid-rodriguez merged 2 commits intomasterfrom
speedup-bundler-setup

Conversation

@deivid-rodriguez
Copy link
Copy Markdown
Contributor

What was the end-user or developer problem that led to this PR?

We should make bundler/setup as fast as possible, since it's the cost that all scripts using Bundler to select versions of dependencies pay.

What is your fix for the problem, implemented in this PR?

I noticed that even in the case where the Gemfile has no changes over the Gemfile.lock file, we're still doing some processing over the raw Gemfile.lock recorded resolution that did not seem necessary.

Further investigation revealed that this is only done for two edge cases:

  • When dependencies are deleted from the Gemfile. In this case, we don't need to re-resolve, only to select the proper subset of the existing lockfile resolution.
  • When the lockfile has redundant platform specific gems recorded. This can only happen to lockfiles generated with versions of Bundler prior to 1.16.1, due to this bug.

So my fix is to explicitly detect the above two situations and only do the extra processing there. Otherwise, use the raw lockfile.

This speeds bundling the Gemfile in https://github.com/technicalpickles/big-gemfile by about 5%, but should speed up all invocations of bundler/setup that don't need a re-resolve, the more dependencies, the more speed up.

➜  big-gemfile git:(main) ✗ hyperfine 'BUNDLER_VERSION=2.4.0.dev ruby -rbundler/setup -e1' 'BUNDLER_VERSION=2.3.17 ruby -rbundler/setup -e1'  
Benchmark 1: BUNDLER_VERSION=2.4.0.dev ruby -rbundler/setup -e1
  Time (mean ± σ):     146.3 ms ±   0.9 ms    [User: 121.6 ms, System: 23.3 ms]
  Range (min … max):   144.5 ms … 148.4 ms    19 runs
 
Benchmark 2: BUNDLER_VERSION=2.3.17 ruby -rbundler/setup -e1
  Time (mean ± σ):     153.2 ms ±   0.7 ms    [User: 128.9 ms, System: 22.9 ms]
  Range (min … max):   151.5 ms … 154.3 ms    19 runs
 
Summary
  'BUNDLER_VERSION=2.4.0.dev ruby -rbundler/setup -e1' ran
    1.05 ± 0.01 times faster than 'BUNDLER_VERSION=2.3.17 ruby -rbundler/setup -e1'

Make sure the following tasks are checked

This case is a bit special, because although the Gemfile has been
changed with respect to the lockfile, it doesn't need a re-resolve. I
think it's nice to give a different message about this case.
Doing this has a cost for every `bundler/setup` invocation, however,
it's not necessary in most cases. It's only needed if:

* Resolution includes a pretty convoluted combination of platform
  specific dependencies.
* Lockfile was generated with Bundler < 1.16.2, and was re-bundled
  with a newer version.

So, we can detect the above situation and only do the cleanup in that
case.

On a reasonably big example Gemfile, this provides a `bundler/setup`
speed up of about 5%.
@deivid-rodriguez deivid-rodriguez force-pushed the speedup-bundler-setup branch from d550c2a to 53b71ff Compare July 7, 2022 12:54
@deivid-rodriguez deivid-rodriguez merged commit b7a90d9 into master Jul 9, 2022
@deivid-rodriguez deivid-rodriguez deleted the speedup-bundler-setup branch July 9, 2022 09:41
deivid-rodriguez added a commit that referenced this pull request Jul 13, 2022
Speed up `bundler/setup` by using the raw `Gemfile.lock` information without extra processing whenever possible

(cherry picked from commit b7a90d9)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant