Skip to content

Conversation

@adoroszlai
Copy link
Contributor

What changes were proposed in this pull request?

Some of the slowest unit tests use OmTestManagers to launch OM. It takes ~10 seconds to initialize (half of that is Ratis leader election). The test cases are pretty quick after that.

[INFO] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 148.9 s -- in org.apache.hadoop.ozone.om.service.TestOpenKeyCleanupService
[INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 133.0 s -- in org.apache.hadoop.ozone.om.TestBucketManagerImpl
[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 108.3 s -- in org.apache.hadoop.ozone.om.TestKeyManagerUnit
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 83.42 s -- in org.apache.hadoop.ozone.om.TestOmSnapshotManager

This change speeds up four of these tests, mostly by converting per-method setup to per-class, to reduce the overhead.

Also, wait for OM leader election before proceeding with the test, to avoid an exception related to failover retry while trying to create the RPC client.

Use test method name as volume name to avoid conflicts, and unexpected number of buckets/keys.

Some of the assertions need to be tweaked to account for objects left over and operations performed by other test cases.

https://issues.apache.org/jira/browse/HDDS-10215

How was this patch tested?

[INFO] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.262 s - in org.apache.hadoop.ozone.om.service.TestOpenKeyCleanupService
[INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.457 s - in org.apache.hadoop.ozone.om.TestBucketManagerImpl
[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.444 s - in org.apache.hadoop.ozone.om.TestKeyManagerUnit
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.75 s - in org.apache.hadoop.ozone.om.TestOmSnapshotManager

https://github.com/adoroszlai/ozone/actions/runs/7670459056/job/20907076486

@adoroszlai adoroszlai self-assigned this Jan 26, 2024
@adoroszlai adoroszlai requested a review from hemantk-12 January 29, 2024 21:34
Copy link
Contributor

@sodonnel sodonnel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. This should speed up every test run by a few minutes which is very nice.

@adoroszlai adoroszlai merged commit 8ab0d07 into apache:master Jan 30, 2024
@adoroszlai adoroszlai deleted the HDDS-10215 branch January 30, 2024 17:24
@adoroszlai
Copy link
Contributor Author

Thanks @sodonnel for the review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants