Skip to content

Retry-reason in x-ms-client-request-id#64

Merged
saxenapranav merged 2 commits intoABFSDriver:ABFS_3.3.2_devfrom
saxenapranav:ABFS_3.3.2_dev_retry_reason
Jun 19, 2023
Merged

Retry-reason in x-ms-client-request-id#64
saxenapranav merged 2 commits intoABFSDriver:ABFS_3.3.2_devfrom
saxenapranav:ABFS_3.3.2_dev_retry_reason

Conversation

@saxenapranav
Copy link
Copy Markdown
Collaborator

@saxenapranav saxenapranav commented Jun 1, 2023 •
edited
Loading

taking code from:
apache#5299: apache@358bf80

apache#5437: apache@759ddeb.

ex of clientRequestId: cid=:f0918642-ef5b-4687-8f31-f6dc90306fbe:db424ea7-6849-4dd4-a4e0-0dad8c621fe0:bff5b9a81814::CF:2_RT:B

:::: AGGREGATED TEST RESULT ::::

HNS-OAuth

[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] TestExponentialRetryPolicy.testOperationOnAccountIdle:212 » AbfsRestOperation ...
[INFO]
[ERROR] Tests run: 138, Failures: 0, Errors: 1, Skipped: 1
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemLease.testAcquireRetry:365 » TestTimedOut test timed o...
[ERROR] ITestAzureBlobFileSystemOauth.testBlobDataContributor:80 » AccessDenied Operat...
[ERROR] ITestAzureBlobFileSystemOauth.testBlobDataReader:137 » AccessDenied Operation ...
[INFO]
[ERROR] Tests run: 746, Failures: 0, Errors: 3, Skipped: 226
[INFO] Results:
[INFO]
[WARNING] Tests run: 273, Failures: 0, Errors: 0, Skipped: 53

HNS-SharedKey

[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] TestExponentialRetryPolicy.testOperationOnAccountIdle:212 » AbfsRestOperation ...
[INFO]
[ERROR] Tests run: 138, Failures: 0, Errors: 1, Skipped: 2
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemLease.testAcquireRetry:363 » TestTimedOut test timed o...
[INFO]
[ERROR] Tests run: 746, Failures: 0, Errors: 1, Skipped: 182
[INFO] Results:
[INFO]
[WARNING] Tests run: 273, Failures: 0, Errors: 0, Skipped: 41

NonHNS-SharedKey

[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] TestExponentialRetryPolicy.testCreateMultipleAccountThrottling:165 [iff singleton is not enabled, for different accounts instances should return different value]
Expecting:
org.apache.hadoop.fs.azurebfs.services.AbfsClientThrottlingIntercept@5c04970e
not to be equal to:
org.apache.hadoop.fs.azurebfs.services.AbfsClientThrottlingIntercept@5c04970e

[ERROR] Errors:
[ERROR] TestExponentialRetryPolicy.testOperationOnAccountIdle:212 » AccessDenied Opera...
[INFO]
[ERROR] Tests run: 138, Failures: 1, Errors: 1, Skipped: 2
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] ITestAzureBlobFileSystemCheckAccess.testCheckAccessForAccountWithoutNS:191 Expecting org.apache.hadoop.security.AccessControlException with text "This request is not authorized to perform this operation using this permission.", 403 but got : "void"
[ERROR] ITestAzureBlobFileSystemRandomRead.testValidateSeekBounds:275->Assert.assertTrue:42->Assert.fail:89 There should not be any network I/O (elapsedTimeMs=141).
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemLease.testAcquireRetry:358 » TestTimedOut test timed o...
[INFO]
[ERROR] Tests run: 746, Failures: 2, Errors: 1, Skipped: 274
[INFO] Results:
[INFO]
[WARNING] Tests run: 273, Failures: 0, Errors: 0, Skipped: 45

NonHNS-OAuth

[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] TestExponentialRetryPolicy.testCreateMultipleAccountThrottling:165 [iff singleton is not enabled, for different accounts instances should return different value]
Expecting:
org.apache.hadoop.fs.azurebfs.services.AbfsClientThrottlingIntercept@7e2f14be
not to be equal to:
org.apache.hadoop.fs.azurebfs.services.AbfsClientThrottlingIntercept@7e2f14be

[ERROR] Errors:
[ERROR] TestExponentialRetryPolicy.testOperationOnAccountIdle:212 » AccessDenied Opera...
[INFO]
[ERROR] Tests run: 138, Failures: 1, Errors: 1, Skipped: 2
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] ITestAzureBlobFileSystemCheckAccess.testCheckAccessForAccountWithoutNS:191 Expecting org.apache.hadoop.security.AccessControlException with text "This request is not authorized to perform this operation using this permission.", 403 but got : "void"
[ERROR] ITestAzureBlobFileSystemRandomRead.testValidateSeekBounds:275->Assert.assertTrue:42->Assert.fail:89 There should not be any network I/O (elapsedTimeMs=137).
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemFlush.testTracingHeaderForAppendBlob:319 » IO AppendBl...
[ERROR] ITestAzureBlobFileSystemLease.testAcquireRetry:358 » TestTimedOut test timed o...
[ERROR] ITestAzureBlobFileSystemOauth.testBlobDataContributor:80 » AbfsRestOperation O...
[ERROR] ITestAzureBlobFileSystemOauth.testBlobDataReader:137 » AccessDenied Operation ...
[INFO]
[ERROR] Tests run: 746, Failures: 2, Errors: 4, Skipped: 280
[INFO] Results:
[INFO]
[WARNING] Tests run: 273, Failures: 0, Errors: 0, Skipped: 57

AppendBlob-HNS-OAuth

[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] TestExponentialRetryPolicy.testOperationOnAccountIdle:212 » AbfsRestOperation ...
[INFO]
[ERROR] Tests run: 138, Failures: 0, Errors: 1, Skipped: 1
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] ITestAzureBlobFileSystemRandomRead.testSkipBounds:224->Assert.assertTrue:42->Assert.fail:89 There should not be any network I/O (elapsedTimeMs=120).
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemLease.testAcquireRetry:365 » TestTimedOut test timed o...
[ERROR] ITestAzureBlobFileSystemOauth.testBlobDataContributor:80 » AccessDenied Operat...
[ERROR] ITestAzureBlobFileSystemOauth.testBlobDataReader:137 » AccessDenied Operation ...
[INFO]
[ERROR] Tests run: 746, Failures: 1, Errors: 3, Skipped: 226
[INFO] Results:
[INFO]
[WARNING] Tests run: 273, Failures: 0, Errors: 0, Skipped: 53

Time taken: 48 mins 12 secs.

@saxenapranav
HADOOP-18606. ABFS: Add reason in x-ms-client-request-id on a retried…
4ad9424 
… API call. (apache#5299)

Contributed by Pranav Saxena

(cherry picked from commit 358bf80)
@saxenapranav
HADOOP-18647. x-ms-client-request-id to identify the retry of an API. (…
89a7096 
…apache#5437)

The x-ms-client-request-id now includes a field to indicate a call is a retry of a previous
operation

Contributed by Pranav Saxena

(cherry picked from commit 759ddeb)
@saxenapranav saxenapranav marked this pull request as ready for review June 1, 2023 09:23
@saxenapranav
Copy link
Copy Markdown
Collaborator Author

saxenapranav commented Jun 1, 2023

:::: AGGREGATED TEST RESULT ::::

HNS-OAuth

[INFO] Results:
[INFO]
[WARNING] Tests run: 138, Failures: 0, Errors: 0, Skipped: 4
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemLease.testAcquireRetry:360 » TestTimedOut test timed o...
[INFO]
[ERROR] Tests run: 746, Failures: 0, Errors: 1, Skipped: 157
[INFO] Results:
[INFO]
[WARNING] Tests run: 273, Failures: 0, Errors: 0, Skipped: 41

HNS-SharedKey

[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] TestAbfsClientThrottlingAnalyzer.testManySuccessAndErrorsAndWaiting:181->fuzzyValidate:64 The actual value 9 is not within the expected range: [5.60, 8.40].
[INFO]
[ERROR] Tests run: 138, Failures: 1, Errors: 0, Skipped: 4
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemLease.testAcquireRetry:363 » TestTimedOut test timed o...
[INFO]
[ERROR] Tests run: 746, Failures: 0, Errors: 1, Skipped: 157
[INFO] Results:
[INFO]
[WARNING] Tests run: 273, Failures: 0, Errors: 0, Skipped: 41

NonHNS-SharedKey

[INFO] Results:
[INFO]
[WARNING] Tests run: 138, Failures: 0, Errors: 0, Skipped: 4
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] ITestAzureBlobFileSystemRandomRead.testSkipBounds:224->Assert.assertTrue:42->Assert.fail:89 There should not be any network I/O (elapsedTimeMs=90).
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemLease.testAcquireRetry:358 » TestTimedOut test timed o...
[INFO]
[ERROR] Tests run: 746, Failures: 1, Errors: 2, Skipped: 275
[INFO] Results:
[INFO]
[WARNING] Tests run: 273, Failures: 0, Errors: 0, Skipped: 45

NonHNS-OAuth

[INFO] Results:
[INFO]
[WARNING] Tests run: 138, Failures: 0, Errors: 0, Skipped: 4
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] ITestAzureBlobFileSystemRandomRead.testSkipBounds:224->Assert.assertTrue:42->Assert.fail:89 There should not be any network I/O (elapsedTimeMs=281).
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemLease.testAcquireRetry:358 » TestTimedOut test timed o...
[INFO]
[ERROR] Tests run: 746, Failures: 1, Errors: 1, Skipped: 275
[INFO] Results:
[INFO]
[WARNING] Tests run: 273, Failures: 0, Errors: 0, Skipped: 45

AppendBlob-HNS-OAuth

[INFO] Results:
[INFO]
[WARNING] Tests run: 138, Failures: 0, Errors: 0, Skipped: 4
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] ITestAzureBlobFileSystemRandomRead.testValidateSeekBounds:275->Assert.assertTrue:42->Assert.fail:89 There should not be any network I/O (elapsedTimeMs=105).
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemLease.testAcquireRetry:363 » TestTimedOut test timed o...
[INFO]
[ERROR] Tests run: 746, Failures: 1, Errors: 1, Skipped: 157
[INFO] Results:
[INFO]
[WARNING] Tests run: 273, Failures: 0, Errors: 0, Skipped: 41

Time taken: 47 mins 16 secs.

anujmodi2021
anujmodi2021 approved these changes Jun 9, 2023
View reviewed changes
Copy link
Copy Markdown
Collaborator

@anujmodi2021 anujmodi2021 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given both the commits are Cherry-picked.
This LGTM

@saxenapranav saxenapranav merged commit 4a47e8b into ABFSDriver:ABFS_3.3.2_dev Jun 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@anmolanmol1234 anmolanmol1234 anmolanmol1234 approved these changes

@anujmodi2021 anujmodi2021 anujmodi2021 approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@saxenapranav @anmolanmol1234 @anujmodi2021