This repository was archived by the owner on Apr 14, 2021. It is now read-only.
Merged
Conversation
segiddins
force-pushed
the
seg-jobs-count
branch
from
ff3c76f to
7229428
Compare
segiddins
changed the title
Collaborator
|
☔ The latest upstream changes (presumably #5995) made this pull request unmergeable. Please resolve the merge conflicts. |
|
@segiddins I am no longer 100% sure if this is doing what I think it does, but the idea when this flag is set is:
That looks to me like what we're doing now? So r+ from me. |
segiddins
changed the title
Contributor
Author
|
@indirect yup! that's exactly what this PR is doing |
segiddins
force-pushed
the
seg-jobs-count
branch
from
a23f51e to
67c56fc
Compare
Collaborator
|
☔ The latest upstream changes (presumably #5964) made this pull request unmergeable. Please resolve the merge conflicts. |
Also, stop subtracting 1 from the configured job count in 2.0
segiddins
force-pushed
the
seg-jobs-count
branch
from
67c56fc to
57b350c
Compare
Contributor
Author
|
Rebased. @bundlerbot r=indirect |
Collaborator
|
📌 Commit 57b350c has been approved by |
Collaborator
bundlerbot
added a commit
that referenced
this pull request
Sep 6, 2017
[2.0] Auto-configure job count Closes #5808. The description of that issue, copied verbatim: This argument comes in two parts, but luckily, the first one is both easier to understand and hopefully to agree with. Background: Bundler 1.4.0 added support for parallel installation via the `--jobs` param. Soon after, [this blog post](http://archlever.blogspot.com/2013/09/lies-damned-lies-and-truths-backed-by.html) (probably greatly amplified by [this Thoughtbot blog post](https://robots.thoughtbot.com/parallel-gem-installing-using-bundler)) popularized the recommendation "set `--jobs` to `nproc - 1`". Not long after, probably also inspired by the popularity of this tip, this "n - 1 jobs" advice got codified into Bundler itself: 66acd02 However, my assertion here is _Bundler should not do that_. The first argument (the easy one) is just that it's not doing what the user asks for. For all the people following the (seemingly popular) tip to set their jobs to `nproc - 1`, they're actually ending up with the probably-worse `- 2`. Even worse than that, if a user does a conservative `--jobs 2`, they're getting _no benefit_ — Bundler is quietly taking their parallelization back down to "no parallelization". Hopefully that's a sufficient argument on its own, but the part II is that this blanket advice is probably out-of-date anyway. Using [this script](https://gist.github.com/tjschuck/ca1d37a8869d1cc01313171b4b318094), I repeatedly installed 29 gems (installing them to a `vendor/` dir and deleting it in between runs). I averaged the time over 10 runs per --jobs value, but the trend holds regardless of how many runs you do. Note that these numbers are for a machine with 2 physical cores and 4 virtual ones (a Mac, reporting 2 and 4 respectively from `sysctl -n hw.physicalcpu` and `sysctl -n hw.ncpu`, the latter corresponding to Linux's `nproc`). ``` ~/Code/tmp/bundler_jobs_bench ☠ ./bundler_jobs_bench.sh Installing 29 gems repeatedly... =============================================== Using Bundler version 1.15.1 (current release version) =============================================== --jobs 1 5.58435780 seconds on average (across 10 runs) --jobs 2 5.35010690 seconds on average (across 10 runs) --jobs 3 3.93493610 seconds on average (across 10 runs) --jobs 4 3.86082760 seconds on average (across 10 runs) --jobs 5 3.24673650 seconds on average (across 10 runs) --jobs 6 3.49340190 seconds on average (across 10 runs) --jobs 7 3.26473430 seconds on average (across 10 runs) --jobs 8 3.34560500 seconds on average (across 10 runs) =============================================== Using development version (no more n - 1 jobs) =============================================== --jobs 1 4.32629540 seconds on average (across 10 runs) --jobs 2 3.48100690 seconds on average (across 10 runs) --jobs 3 3.30937880 seconds on average (across 10 runs) --jobs 4 3.30868200 seconds on average (across 10 runs) --jobs 5 3.54932920 seconds on average (across 10 runs) --jobs 6 3.36123440 seconds on average (across 10 runs) --jobs 7 3.96490630 seconds on average (across 10 runs) --jobs 8 3.39955640 seconds on average (across 10 runs) ``` From the above, you can see: 1. In the first block, no notable change between `--jobs 1` and `--jobs 2` — that's because they're currently the same. 2. In both, a best time corresponding to the value that (effectively) matches nproc, _not_ nproc - 1. 3. Regardless of nproc coming out best in this run, there is close enough performance among the range of `nproc - 1` through to `nproc * 2` that it doesn't seem like anything in particular (like the `- 1` removed in this commit) should be codified — people seeking to particularly optimize their bundler runtimes should do their own tweaking of the value, and it should be respected as given.
Collaborator
|
☀️ Test successful - status-travis |
colby-swandale
pushed a commit
that referenced
this pull request
Sep 22, 2018
[2.0] Auto-configure job count Closes #5808. The description of that issue, copied verbatim: This argument comes in two parts, but luckily, the first one is both easier to understand and hopefully to agree with. Background: Bundler 1.4.0 added support for parallel installation via the `--jobs` param. Soon after, [this blog post](http://archlever.blogspot.com/2013/09/lies-damned-lies-and-truths-backed-by.html) (probably greatly amplified by [this Thoughtbot blog post](https://robots.thoughtbot.com/parallel-gem-installing-using-bundler)) popularized the recommendation "set `--jobs` to `nproc - 1`". Not long after, probably also inspired by the popularity of this tip, this "n - 1 jobs" advice got codified into Bundler itself: 66acd02 However, my assertion here is _Bundler should not do that_. The first argument (the easy one) is just that it's not doing what the user asks for. For all the people following the (seemingly popular) tip to set their jobs to `nproc - 1`, they're actually ending up with the probably-worse `- 2`. Even worse than that, if a user does a conservative `--jobs 2`, they're getting _no benefit_ — Bundler is quietly taking their parallelization back down to "no parallelization". Hopefully that's a sufficient argument on its own, but the part II is that this blanket advice is probably out-of-date anyway. Using [this script](https://gist.github.com/tjschuck/ca1d37a8869d1cc01313171b4b318094), I repeatedly installed 29 gems (installing them to a `vendor/` dir and deleting it in between runs). I averaged the time over 10 runs per --jobs value, but the trend holds regardless of how many runs you do. Note that these numbers are for a machine with 2 physical cores and 4 virtual ones (a Mac, reporting 2 and 4 respectively from `sysctl -n hw.physicalcpu` and `sysctl -n hw.ncpu`, the latter corresponding to Linux's `nproc`). ``` ~/Code/tmp/bundler_jobs_bench ☠ ./bundler_jobs_bench.sh Installing 29 gems repeatedly... =============================================== Using Bundler version 1.15.1 (current release version) =============================================== --jobs 1 5.58435780 seconds on average (across 10 runs) --jobs 2 5.35010690 seconds on average (across 10 runs) --jobs 3 3.93493610 seconds on average (across 10 runs) --jobs 4 3.86082760 seconds on average (across 10 runs) --jobs 5 3.24673650 seconds on average (across 10 runs) --jobs 6 3.49340190 seconds on average (across 10 runs) --jobs 7 3.26473430 seconds on average (across 10 runs) --jobs 8 3.34560500 seconds on average (across 10 runs) =============================================== Using development version (no more n - 1 jobs) =============================================== --jobs 1 4.32629540 seconds on average (across 10 runs) --jobs 2 3.48100690 seconds on average (across 10 runs) --jobs 3 3.30937880 seconds on average (across 10 runs) --jobs 4 3.30868200 seconds on average (across 10 runs) --jobs 5 3.54932920 seconds on average (across 10 runs) --jobs 6 3.36123440 seconds on average (across 10 runs) --jobs 7 3.96490630 seconds on average (across 10 runs) --jobs 8 3.39955640 seconds on average (across 10 runs) ``` From the above, you can see: 1. In the first block, no notable change between `--jobs 1` and `--jobs 2` — that's because they're currently the same. 2. In both, a best time corresponding to the value that (effectively) matches nproc, _not_ nproc - 1. 3. Regardless of nproc coming out best in this run, there is close enough performance among the range of `nproc - 1` through to `nproc * 2` that it doesn't seem like anything in particular (like the `- 1` removed in this commit) should be codified — people seeking to particularly optimize their bundler runtimes should do their own tweaking of the value, and it should be respected as given. (cherry picked from commit 4d5246d)
colby-swandale
pushed a commit
that referenced
this pull request
Sep 22, 2018
[2.0] Auto-configure job count Closes #5808. The description of that issue, copied verbatim: This argument comes in two parts, but luckily, the first one is both easier to understand and hopefully to agree with. Background: Bundler 1.4.0 added support for parallel installation via the `--jobs` param. Soon after, [this blog post](http://archlever.blogspot.com/2013/09/lies-damned-lies-and-truths-backed-by.html) (probably greatly amplified by [this Thoughtbot blog post](https://robots.thoughtbot.com/parallel-gem-installing-using-bundler)) popularized the recommendation "set `--jobs` to `nproc - 1`". Not long after, probably also inspired by the popularity of this tip, this "n - 1 jobs" advice got codified into Bundler itself: 66acd02 However, my assertion here is _Bundler should not do that_. The first argument (the easy one) is just that it's not doing what the user asks for. For all the people following the (seemingly popular) tip to set their jobs to `nproc - 1`, they're actually ending up with the probably-worse `- 2`. Even worse than that, if a user does a conservative `--jobs 2`, they're getting _no benefit_ — Bundler is quietly taking their parallelization back down to "no parallelization". Hopefully that's a sufficient argument on its own, but the part II is that this blanket advice is probably out-of-date anyway. Using [this script](https://gist.github.com/tjschuck/ca1d37a8869d1cc01313171b4b318094), I repeatedly installed 29 gems (installing them to a `vendor/` dir and deleting it in between runs). I averaged the time over 10 runs per --jobs value, but the trend holds regardless of how many runs you do. Note that these numbers are for a machine with 2 physical cores and 4 virtual ones (a Mac, reporting 2 and 4 respectively from `sysctl -n hw.physicalcpu` and `sysctl -n hw.ncpu`, the latter corresponding to Linux's `nproc`). ``` ~/Code/tmp/bundler_jobs_bench ☠ ./bundler_jobs_bench.sh Installing 29 gems repeatedly... =============================================== Using Bundler version 1.15.1 (current release version) =============================================== --jobs 1 5.58435780 seconds on average (across 10 runs) --jobs 2 5.35010690 seconds on average (across 10 runs) --jobs 3 3.93493610 seconds on average (across 10 runs) --jobs 4 3.86082760 seconds on average (across 10 runs) --jobs 5 3.24673650 seconds on average (across 10 runs) --jobs 6 3.49340190 seconds on average (across 10 runs) --jobs 7 3.26473430 seconds on average (across 10 runs) --jobs 8 3.34560500 seconds on average (across 10 runs) =============================================== Using development version (no more n - 1 jobs) =============================================== --jobs 1 4.32629540 seconds on average (across 10 runs) --jobs 2 3.48100690 seconds on average (across 10 runs) --jobs 3 3.30937880 seconds on average (across 10 runs) --jobs 4 3.30868200 seconds on average (across 10 runs) --jobs 5 3.54932920 seconds on average (across 10 runs) --jobs 6 3.36123440 seconds on average (across 10 runs) --jobs 7 3.96490630 seconds on average (across 10 runs) --jobs 8 3.39955640 seconds on average (across 10 runs) ``` From the above, you can see: 1. In the first block, no notable change between `--jobs 1` and `--jobs 2` — that's because they're currently the same. 2. In both, a best time corresponding to the value that (effectively) matches nproc, _not_ nproc - 1. 3. Regardless of nproc coming out best in this run, there is close enough performance among the range of `nproc - 1` through to `nproc * 2` that it doesn't seem like anything in particular (like the `- 1` removed in this commit) should be codified — people seeking to particularly optimize their bundler runtimes should do their own tweaking of the value, and it should be respected as given. (cherry picked from commit 4d5246d)
colby-swandale
pushed a commit
that referenced
this pull request
Sep 22, 2018
[2.0] Auto-configure job count Closes #5808. The description of that issue, copied verbatim: This argument comes in two parts, but luckily, the first one is both easier to understand and hopefully to agree with. Background: Bundler 1.4.0 added support for parallel installation via the `--jobs` param. Soon after, [this blog post](http://archlever.blogspot.com/2013/09/lies-damned-lies-and-truths-backed-by.html) (probably greatly amplified by [this Thoughtbot blog post](https://robots.thoughtbot.com/parallel-gem-installing-using-bundler)) popularized the recommendation "set `--jobs` to `nproc - 1`". Not long after, probably also inspired by the popularity of this tip, this "n - 1 jobs" advice got codified into Bundler itself: 66acd02 However, my assertion here is _Bundler should not do that_. The first argument (the easy one) is just that it's not doing what the user asks for. For all the people following the (seemingly popular) tip to set their jobs to `nproc - 1`, they're actually ending up with the probably-worse `- 2`. Even worse than that, if a user does a conservative `--jobs 2`, they're getting _no benefit_ — Bundler is quietly taking their parallelization back down to "no parallelization". Hopefully that's a sufficient argument on its own, but the part II is that this blanket advice is probably out-of-date anyway. Using [this script](https://gist.github.com/tjschuck/ca1d37a8869d1cc01313171b4b318094), I repeatedly installed 29 gems (installing them to a `vendor/` dir and deleting it in between runs). I averaged the time over 10 runs per --jobs value, but the trend holds regardless of how many runs you do. Note that these numbers are for a machine with 2 physical cores and 4 virtual ones (a Mac, reporting 2 and 4 respectively from `sysctl -n hw.physicalcpu` and `sysctl -n hw.ncpu`, the latter corresponding to Linux's `nproc`). ``` ~/Code/tmp/bundler_jobs_bench ☠ ./bundler_jobs_bench.sh Installing 29 gems repeatedly... =============================================== Using Bundler version 1.15.1 (current release version) =============================================== --jobs 1 5.58435780 seconds on average (across 10 runs) --jobs 2 5.35010690 seconds on average (across 10 runs) --jobs 3 3.93493610 seconds on average (across 10 runs) --jobs 4 3.86082760 seconds on average (across 10 runs) --jobs 5 3.24673650 seconds on average (across 10 runs) --jobs 6 3.49340190 seconds on average (across 10 runs) --jobs 7 3.26473430 seconds on average (across 10 runs) --jobs 8 3.34560500 seconds on average (across 10 runs) =============================================== Using development version (no more n - 1 jobs) =============================================== --jobs 1 4.32629540 seconds on average (across 10 runs) --jobs 2 3.48100690 seconds on average (across 10 runs) --jobs 3 3.30937880 seconds on average (across 10 runs) --jobs 4 3.30868200 seconds on average (across 10 runs) --jobs 5 3.54932920 seconds on average (across 10 runs) --jobs 6 3.36123440 seconds on average (across 10 runs) --jobs 7 3.96490630 seconds on average (across 10 runs) --jobs 8 3.39955640 seconds on average (across 10 runs) ``` From the above, you can see: 1. In the first block, no notable change between `--jobs 1` and `--jobs 2` — that's because they're currently the same. 2. In both, a best time corresponding to the value that (effectively) matches nproc, _not_ nproc - 1. 3. Regardless of nproc coming out best in this run, there is close enough performance among the range of `nproc - 1` through to `nproc * 2` that it doesn't seem like anything in particular (like the `- 1` removed in this commit) should be codified — people seeking to particularly optimize their bundler runtimes should do their own tweaking of the value, and it should be respected as given. (cherry picked from commit 4d5246d)
colby-swandale
pushed a commit
that referenced
this pull request
Oct 5, 2018
[2.0] Auto-configure job count Closes #5808. The description of that issue, copied verbatim: This argument comes in two parts, but luckily, the first one is both easier to understand and hopefully to agree with. Background: Bundler 1.4.0 added support for parallel installation via the `--jobs` param. Soon after, [this blog post](http://archlever.blogspot.com/2013/09/lies-damned-lies-and-truths-backed-by.html) (probably greatly amplified by [this Thoughtbot blog post](https://robots.thoughtbot.com/parallel-gem-installing-using-bundler)) popularized the recommendation "set `--jobs` to `nproc - 1`". Not long after, probably also inspired by the popularity of this tip, this "n - 1 jobs" advice got codified into Bundler itself: 66acd02 However, my assertion here is _Bundler should not do that_. The first argument (the easy one) is just that it's not doing what the user asks for. For all the people following the (seemingly popular) tip to set their jobs to `nproc - 1`, they're actually ending up with the probably-worse `- 2`. Even worse than that, if a user does a conservative `--jobs 2`, they're getting _no benefit_ — Bundler is quietly taking their parallelization back down to "no parallelization". Hopefully that's a sufficient argument on its own, but the part II is that this blanket advice is probably out-of-date anyway. Using [this script](https://gist.github.com/tjschuck/ca1d37a8869d1cc01313171b4b318094), I repeatedly installed 29 gems (installing them to a `vendor/` dir and deleting it in between runs). I averaged the time over 10 runs per --jobs value, but the trend holds regardless of how many runs you do. Note that these numbers are for a machine with 2 physical cores and 4 virtual ones (a Mac, reporting 2 and 4 respectively from `sysctl -n hw.physicalcpu` and `sysctl -n hw.ncpu`, the latter corresponding to Linux's `nproc`). ``` ~/Code/tmp/bundler_jobs_bench ☠ ./bundler_jobs_bench.sh Installing 29 gems repeatedly... =============================================== Using Bundler version 1.15.1 (current release version) =============================================== --jobs 1 5.58435780 seconds on average (across 10 runs) --jobs 2 5.35010690 seconds on average (across 10 runs) --jobs 3 3.93493610 seconds on average (across 10 runs) --jobs 4 3.86082760 seconds on average (across 10 runs) --jobs 5 3.24673650 seconds on average (across 10 runs) --jobs 6 3.49340190 seconds on average (across 10 runs) --jobs 7 3.26473430 seconds on average (across 10 runs) --jobs 8 3.34560500 seconds on average (across 10 runs) =============================================== Using development version (no more n - 1 jobs) =============================================== --jobs 1 4.32629540 seconds on average (across 10 runs) --jobs 2 3.48100690 seconds on average (across 10 runs) --jobs 3 3.30937880 seconds on average (across 10 runs) --jobs 4 3.30868200 seconds on average (across 10 runs) --jobs 5 3.54932920 seconds on average (across 10 runs) --jobs 6 3.36123440 seconds on average (across 10 runs) --jobs 7 3.96490630 seconds on average (across 10 runs) --jobs 8 3.39955640 seconds on average (across 10 runs) ``` From the above, you can see: 1. In the first block, no notable change between `--jobs 1` and `--jobs 2` — that's because they're currently the same. 2. In both, a best time corresponding to the value that (effectively) matches nproc, _not_ nproc - 1. 3. Regardless of nproc coming out best in this run, there is close enough performance among the range of `nproc - 1` through to `nproc * 2` that it doesn't seem like anything in particular (like the `- 1` removed in this commit) should be codified — people seeking to particularly optimize their bundler runtimes should do their own tweaking of the value, and it should be respected as given. (cherry picked from commit 4d5246d)
colby-swandale
pushed a commit
that referenced
this pull request
Oct 25, 2018
* 1-17-stable: (38 commits) Version 1.17.0 with changelog Merge #6754 Version 1.17.0.pre.2 with changelog fix failing bundle remove specs Still document the `--force` option scope specs testing bundler 2 deprecations to bundler 1 only Merge #6718 Merge #6707 Merge #6702 Merge #6316 Auto merge of #6447 - agrim123:agr-update-error-message, r=segiddins Auto merge of #6513 - agrim123:agr-bundler-remove, r=indirect Auto merge of #6318 - jhawthorn:fix_incorrect_test_in_requires_sudo, r=segiddins Auto merge of #6450 - bundler:segiddins/bundle-binstubs-all, r=colby-swandale Auto merge of #6024 - gwerbin:custom-user-bundle-dir, r=colby-swandale Version 1.17.0.pre.1 with changelog Auto merge of #5964 - bundler:colby/deprecate-viz-command, r=segiddins Auto merge of #5986 - bundler:seg-jobs-count, r=indirect Auto merge of #5995 - bundler:seg-gvp-major, r=indirect Auto merge of #5803 - igorbozato:path_relative_to_pwd, r=indirect ...
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #5808.
The description of that issue, copied verbatim:
This argument comes in two parts, but luckily, the first one is both easier to understand and hopefully to agree with.
Background: Bundler 1.4.0 added support for parallel installation via the
--jobsparam. Soon after, this blog post (probably greatly amplified by this Thoughtbot blog post) popularized the recommendation "set--jobstonproc - 1".Not long after, probably also inspired by the popularity of this tip, this "n - 1 jobs" advice got codified into Bundler itself: 66acd02
However, my assertion here is Bundler should not do that.
The first argument (the easy one) is just that it's not doing what the user asks for. For all the people following the (seemingly popular) tip to set their jobs to
nproc - 1, they're actually ending up with the probably-worse- 2. Even worse than that, if a user does a conservative--jobs 2, they're getting no benefit — Bundler is quietly taking their parallelization back down to "no parallelization".Hopefully that's a sufficient argument on its own, but the part II is that this blanket advice is probably out-of-date anyway.
Using this script, I repeatedly installed 29 gems (installing them to a
vendor/dir and deleting it in between runs). I averaged the time over 10 runs per --jobs value, but the trend holds regardless of how many runs you do.Note that these numbers are for a machine with 2 physical cores and 4 virtual ones (a Mac, reporting 2 and 4 respectively from
sysctl -n hw.physicalcpuandsysctl -n hw.ncpu, the latter corresponding to Linux'snproc).From the above, you can see:
In the first block, no notable change between
--jobs 1and--jobs 2— that's because they're currently the same.In both, a best time corresponding to the value that (effectively) matches nproc, not nproc - 1.
Regardless of nproc coming out best in this run, there is close enough performance among the range of
nproc - 1through tonproc * 2that it doesn't seem like anything in particular (like the- 1removed in this commit) should be codified — people seeking to particularly optimize their bundler runtimes should do their own tweaking of the value, and it should be respected as given.