Disable RateLimitTransport

[lijok]

Either completely disable it or at least give people the option to disable it through config
Imported some 3000 github resources into terraform yesterday, and then spent the better half of today trying to figure out why my terraform runs started taking 15-20 minutes
Commented this line out https://github.com/integrations/terraform-provider-github/blob/master/github/config.go#L34, and runs went down to 20-30 seconds with parallelism of 25
Increasing parallelism above that triggers github api abuse mechanism
The PR that introduced this mechanism #145 acknowledges the slowdown but shrugs the reduced chance of triggering the abuse mechanism a win
Then someone raised a follow-up pr giving users the option to disable this transport, which didn't go anywhere. #235
Please reconsider your stance on this. A 45x slowdown is unacceptable.

Thank you

[jcudit]

give people the option to disable it through config

I like this direction as a first step. Ideally we implement this and keep 👀 out for reported improvements.

[lijok]

Is anything being done about this?
At the very least this: #849 should be addressed to alieviate some of the pressure
we have around 2k~ resources in our state and plans are taking 16min

[jcudit]

Aiming to address this in one of our upcoming releases. Adding provider-level configuration to disable RateLimitTransport functionality seems within reach.

[lijok]

Aiming to address this in one of our upcoming releases. Adding provider-level configuration to disable RateLimitTransport functionality seems within reach.

Thank you, greatly appreciated

[jcudit]

We scoped this out and recommend the following edits:

• Modify github/provider.go and index.md to add a new configuration option controlling the number of milliseconds the provider will sleep between attempts when it detects rate limiting.
• Track this value in the Config struct assembled during providerConfigure() in provider.go.
• Add convenience functions to transport.go to support configuring a NewRateLimitTransport with a desired delay
• Add testing similar to TestRateLimitTransport_abuseLimit_post.

This looks on track for an upcoming release.

[lijok]

We scoped this out and recommend the following edits:

* Modify `github/provider.go` and `index.md` to add a new configuration option controlling the number of milliseconds the provider will sleep between attempts when it detects rate limiting.

* Track this value in the `Config` struct assembled during `providerConfigure()` in `provider.go`.

* Add convenience functions to `transport.go` to support configuring a `NewRateLimitTransport` with a desired delay

* Add testing similar to `TestRateLimitTransport_abuseLimit_post`.

This looks on track for an upcoming release.

Thank you, that sounds great
I'd also like to bring this issue to your attention specifically, as it is closely related to the slowness of the provider:
#849
As far as I understand, githubs official guidelines don't account for this issue?
Would it be reasonable to say that something needs to be done to ensure the provider transport does not sleep between each Query, sleeping inbetween Mutations instead?