You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ABFS has a client-side throttling mechanism which works on the metrics collected from past requests made. I requests are getting failed due to throttling at server, we update our metrics and client side backoff is calculated based on those metrics.
This PR enhances the logic to decide which requests should be considered to compute client side backoff interval as follows:
For each request made by ABFS driver, we will determine if they should contribute to Client-Side Throttling based on the status code and result:
Status code in 2xx range: Successful Operations should contribute.
Status code in 3xx range: Redirection Operations should not contribute.
Status code in 4xx range: User Errors should not contribute.
Status code is 503: Throttling Error should contribute only if they are due to client limits breach as follows:
503, Ingress Over Account Limit: Should Contribute
503, Egress Over Account Limit: Should Contribute
503, TPS Over Account Limit: Should Contribute
503, Other Server Throttling: Should not Contribute.
Status code in 5xx range other than 503: Should not Contribute.
IOException and UnknownHostExceptions: Should not Contribute.
…g Metrics Logic (apache#6276)
ABFS has a client-side throttling mechanism which works on the metrics collected
from past requests
When requests are fail due to server-side throttling it updates its
metrics and recalculates any client side backoff.
The choice of which requests should be used to compute client side
backoff interval is based on the http status code:
- Status code in 2xx range: Successful Operations should contribute.
- Status code in 3xx range: Redirection Operations should not contribute.
- Status code in 4xx range: User Errors should not contribute.
- Status code is 503: Throttling Error should contribute only if they
are due to client limits breach as follows:
* 503, Ingress Over Account Limit: Should Contribute
* 503, Egress Over Account Limit: Should Contribute
* 503, TPS Over Account Limit: Should Contribute
* 503, Other Server Throttling: Should not Contribute.
- Status code in 5xx range other than 503: Should not Contribute.
- IOException and UnknownHostExceptions: Should not Contribute.
Contributed by Anuj Modi
[ERROR] testCloseOfDataBlockOnAppendComplete(org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemAppend) Time elapsed: 14.424 s <<< FAILURE!
[ERROR] testListPathWithValueGreaterThanServerMaximum(org.apache.hadoop.fs.azurebfs.ITestAbfsClient) Time elapsed: 234.055 s <<< FAILURE!
[ERROR] testAbfsStreamOps(org.apache.hadoop.fs.azurebfs.ITestAbfsStreamStatistics) Time elapsed: 6.224 s <<< FAILURE!
[ERROR] testExpect100ContinueFailureInAppend(org.apache.hadoop.fs.azurebfs.services.ITestAbfsOutputStream) Time elapsed: 9.868 s <<< ERROR!
[ERROR] testAppendWithChecksumAtDifferentOffsets(org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemChecksum) Time elapsed: 15.112 s <<< ERROR!
[ERROR] testTwoWritersCreateAppendNoInfiniteLease(org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemLease) Time elapsed: 4.308 s <<< ERROR!
[ERROR] test_120_terasort(org.apache.hadoop.fs.azurebfs.commit.ITestAbfsTerasort) Time elapsed: 4.584 s <<< ERROR!
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of PR
Jira: https://issues.apache.org/jira/browse/HADOOP-19096
Commit in trunk: dbe2d61
ABFS has a client-side throttling mechanism which works on the metrics collected from past requests made. I requests are getting failed due to throttling at server, we update our metrics and client side backoff is calculated based on those metrics.
This PR enhances the logic to decide which requests should be considered to compute client side backoff interval as follows:
For each request made by ABFS driver, we will determine if they should contribute to Client-Side Throttling based on the status code and result:
Status code in 2xx range: Successful Operations should contribute.
Status code in 3xx range: Redirection Operations should not contribute.
Status code in 4xx range: User Errors should not contribute.
Status code is 503: Throttling Error should contribute only if they are due to client limits breach as follows:
503, Ingress Over Account Limit: Should Contribute
503, Egress Over Account Limit: Should Contribute
503, TPS Over Account Limit: Should Contribute
503, Other Server Throttling: Should not Contribute.
Status code in 5xx range other than 503: Should not Contribute.
IOException and UnknownHostExceptions: Should not Contribute.