Skip to content

Conversation

SamuZad
Copy link
Contributor

@SamuZad SamuZad commented Jul 30, 2025

Resolves #567


Before the change?

  • parallel_requests was explicitly disabled on github.com

After the change?

  • parallel_requests is no longer explicitly disabled on github.com

To paraphrase the rationale behind the change

With ~300 repositories, parallel_requests = false terraform plan -parallelism=30 68.65s user 23.16s system 19% cpu 7:46.27 total

With ~300 repositories, parallel_requests = true terraform plan -parallelism=30 44.51s user 14.47s system 105% cpu 55.640 total

With ~300 repositories, parallel_requests = true, lower parallelism terraform plan -parallelism=10 51.69s user 17.67s system 72% cpu 1:35.17 total

A significant reduction in plan time.

Of course, investigating the reason why it was implemented in such a manner, I've come across #145 and the linked official docs that say the following:

To avoid exceeding secondary rate limits, you should make requests serially instead of concurrently. To achieve this, you can implement a queue system for requests.

That being said, given that it makes such a massive difference, and this recommendation is specifically about the REST API (and a bunch of resources now interact with the GraphQL API instead which have more generous secondary rate limits)

Pull request checklist

  • Tests for the changes have been added (for bug fixes / features) - N/A
  • Docs have been reviewed and added / updated if needed (for bug fixes / features)

Does this introduce a breaking change?

Please see our docs on breaking changes to help!

  • Yes
  • No

@nickfloyd nickfloyd moved this to On Deck in Terraform Provider Oct 20, 2025
@nickfloyd nickfloyd moved this from On Deck to In Progress in Terraform Provider Oct 20, 2025
@nickfloyd nickfloyd moved this from In Progress to In Review in Terraform Provider Oct 20, 2025
Copy link
Contributor

@nickfloyd nickfloyd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll need to keep an eye on this. Pragmatically this change is great but I want to 👀 for any potential side effects.

@nickfloyd nickfloyd changed the title Allowing parallel_requests to be set on github.com fix: Allow parallel_requests to be set on github.com Oct 20, 2025
@nickfloyd nickfloyd added the Type: Bug Something isn't working as documented label Oct 20, 2025
@nickfloyd
Copy link
Contributor

Marking as bug given it's performance profile.

@nickfloyd nickfloyd merged commit bc9d9c0 into integrations:main Oct 20, 2025
3 checks passed
@github-project-automation github-project-automation bot moved this from In Review to Done in Terraform Provider Oct 20, 2025
@github-project-automation github-project-automation bot moved this from 🆕 Triage to ✅ Done in 🧰 Octokit Active Oct 20, 2025
@SamuZad
Copy link
Contributor Author

SamuZad commented Oct 20, 2025

Thank you for merging 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Type: Bug Something isn't working as documented

Projects

Development

Successfully merging this pull request may close these issues.

Slow performance when managing dozens of repositories

4 participants