Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run go-install in deps-tools in parallel #31711

Merged
merged 3 commits into from
Jul 28, 2024

Conversation

silverwind
Copy link
Member

@silverwind silverwind commented Jul 26, 2024

go install is far too conservative in regards to parallel HTTP requests, so we can run the commands in parallel to achieve a speedup.

`go install` is far too conservative in regards to parallel HTTP
requests, so we can run the commands in parallel to achieve a speedup of
likely around 300% on CI. In my testing, it does seem to be
concurrency-safe.
@GiteaBot GiteaBot added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label Jul 26, 2024
@pull-request-size pull-request-size bot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Jul 26, 2024
@silverwind silverwind added the skip-changelog This PR is irrelevant for the (next) changelog, for example bug fixes for unreleased features. label Jul 26, 2024
@silverwind
Copy link
Member Author

CI run shows the time is almost the same as before on GitHub actions, but I had other projects that benefitted massively from this change (300% faster).

Maybe setup-go does some magic to speed up go install, not sure. Change is still worth to have imho.

@silverwind
Copy link
Member Author

silverwind commented Jul 27, 2024

Definitely a lot faster for me locally (go clean -modcache before each run):

make deps-tools  11.66s user 13.97s system 38% cpu 1:07.28 total
make deps-tools  8.30s user 9.14s system 127% cpu 13.637 total

I guess the reason it's not faster on CI is because CPU resources are limited on our runners.

@silverwind silverwind changed the title Run deps-tools in parallel Run go-install in deps-tools in parallel Jul 27, 2024
@GiteaBot GiteaBot added lgtm/need 1 This PR needs approval from one additional maintainer to be merged. and removed lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. labels Jul 28, 2024
@GiteaBot GiteaBot added lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. and removed lgtm/need 1 This PR needs approval from one additional maintainer to be merged. labels Jul 28, 2024
@techknowlogick techknowlogick enabled auto-merge (squash) July 28, 2024 16:58
@techknowlogick techknowlogick added the reviewed/wait-merge This pull request is part of the merge queue. It will be merged soon. label Jul 28, 2024
@techknowlogick techknowlogick merged commit 5f526d7 into go-gitea:main Jul 28, 2024
26 checks passed
@GiteaBot GiteaBot added this to the 1.23.0 milestone Jul 28, 2024
@GiteaBot GiteaBot removed the reviewed/wait-merge This pull request is part of the merge queue. It will be merged soon. label Jul 28, 2024
@silverwind silverwind deleted the parallel-tools branch July 28, 2024 18:26
zjjhot added a commit to zjjhot/gitea that referenced this pull request Jul 29, 2024
* giteaofficial/main:
  Make GetRepositoryByName more safer (go-gitea#31712)
  [skip ci] Updated licenses and gitignores
  Run `go-install` in `deps-tools` in parallel (go-gitea#31711)
  Hide the "Details" link of commit status when the user cannot access actions (go-gitea#30156)
  Enable `no-jquery/no-parse-html-literal` and fix violation (go-gitea#31684)
  [skip ci] Updated translations via Crowdin
  OIDC: case-insensitive comparison for auth scheme `Basic` (go-gitea#31706)
  Support `pull_request_target` event for commit status (go-gitea#31703)
  Add types to fetch,toast,bootstrap,svg (go-gitea#31627)
  Run `detectWebAuthnSupport` only if necessary (go-gitea#31691)
  add `username` to OIDC introspection response (go-gitea#31688)
  Add return type to GetRawFileOrLFS and GetRawFile (go-gitea#31680)
  Support delete user email in admin panel (go-gitea#31690)
  Use GetDisplayName() instead of DisplayName() to generate rss feeds (go-gitea#31687)
@go-gitea go-gitea locked as resolved and limited conversation to collaborators Oct 27, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. modifies/internal size/S Denotes a PR that changes 10-29 lines, ignoring generated files. skip-changelog This PR is irrelevant for the (next) changelog, for example bug fixes for unreleased features.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants