Share omnibus gems with all projects to reduce install time

[mctofu]

This is an alternate to #5520 in attempting to reduce the dev container build time. This approach is similar to https://github.com/dependabot/dependabot-core/pull/5528/files but re-using the omnibus gems instead of common to make sure we have the full set.

As @deivid-rodriguez noticed every ecosystem mainly relies on the "common" gems and we were doing a lot of extra work by installing the same set of gems for each ecosystem. Only the git submodules ecosystem relies on one extra gem. This change makes an optimization to only install the gems once, via the omnibus gem, and then shares those gems with each ecosystem. On my local dev machine this has a pretty significant impact in time to build the dev container from scratch (19m down to 1.3m):

omnibus install only:

=> building image from Dockerfile.development
[+] Building 81.2s (34/34) FINISHED  

all installs:

=> building image from Dockerfile.development
[+] Building 1127.2s (35/35) FINISHED

[deivid-rodriguez]

Due to the path change I need to bundle exec rspec in a dir rather than being able to just run rspec.

Oh, l see, that's inconvenient. Let me see if this can be fixed.

[deivid-rodriguez]

@mctofu I think the following changes should fix the issues you were having

diff --git a/Dockerfile b/Dockerfile
index 4edb14d8c..1a6020098 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -77,7 +77,8 @@ ARG BUNDLER_V1_VERSION=1.17.3
 ARG BUNDLER_V2_VERSION=2.3.14
 ENV BUNDLE_SILENCE_ROOT_WARNING=1
 # Allow gem installs as the dependabot user
-ENV BUNDLE_PATH=".bundle"
+ENV BUNDLE_PATH=".bundle" \
+    BUNDLE_BIN=".bundle/bin"
 
 # Install Ruby, update RubyGems, and install Bundler
 RUN mkdir -p /tmp/ruby-install \
diff --git a/Dockerfile.development b/Dockerfile.development
index 5401eb6c0..d3bcb943c 100644
--- a/Dockerfile.development
+++ b/Dockerfile.development
@@ -42,6 +42,8 @@ RUN cd omnibus \
 # Make omnibus gems available to bundler in root directory
 RUN echo 'eval_gemfile File.join(File.dirname(__FILE__), "omnibus/Gemfile")' > Gemfile
 
+ENV PATH="${CODE_DIR}/omnibus/$BUNDLE_BIN:$PATH"
+
 RUN cd common \
   && bundle config path ${CODE_DIR}/omnibus/.bundle
 

[mctofu]

A few issues/changes I've noticed:

• Due to the path change I need to bundle exec rspec in a dir rather than being able to just run rspec.
• Rspec didn't work on the first try and complained of

Could not find gem 'rubocop (~> 1.33.0)' in locally installed gems.

The source contains the following gems matching 'rubocop':
  * rubocop-1.35.0

A bundle install resulted Installing rubocop 1.33.0 which cleared up the issue.

These were both cleared up by @deivid-rodriguez's suggested changes. I also undid the changes we'd done to the Dockerfile since I don't think they were needed for this to work (removing the path changes in d440c63).

[deivid-rodriguez]

Nice! Yes, the changes in the Dockerfile are not needed for this work 👍.

[deivid-rodriguez]

This looks good to me, assuming CI failures are unrelated!

(I would probably squash commits, but I'm not sure if you usually rewrite git history or prefer not to do that)

[honeyankit]

Wow this is huge! From 19m down to 1.3m. Good work! thanks

[jurre]

🙌

[jakecoffman]

I was able to do a dry-run with this, seems good. ✅