HDDS-9328. Speed up testListStatusIteratorOnPageSize

[adoroszlai]

What changes were proposed in this pull request?

Speed up testListStatusIteratorOnPageSize by:

• Reduce the number of directory entries created for testing iterator paging (set smaller page size in config).
• Keep entries from smaller batch to larger one, only create additional entries. Delete only at the end.

Also reduce code duplication, extract method to common test class.

https://issues.apache.org/jira/browse/HDDS-9328

How was this patch tested?

Before:

Tests run: 176, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 337.616 s - in org.apache.hadoop.fs.ozone.TestOzoneFileSystem
Tests run: 102, Failures: 0, Errors: 0, Skipped: 10, Time elapsed: 135.095 s - in org.apache.hadoop.fs.ozone.TestOzoneFileSystemWithFSO
Tests run: 250, Failures: 0, Errors: 0, Skipped: 9, Time elapsed: 547.686 s - in org.apache.hadoop.fs.ozone.TestRootedOzoneFileSystem
Tests run: 108, Failures: 0, Errors: 0, Skipped: 16, Time elapsed: 203.204 s - in org.apache.hadoop.fs.ozone.TestRootedOzoneFileSystemWithFSO

https://github.com/apache/ozone/actions/runs/6245661295/job/16955525758

After:

Tests run: 176, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 166.418 s - in org.apache.hadoop.fs.ozone.TestOzoneFileSystem
Tests run: 102, Failures: 0, Errors: 0, Skipped: 10, Time elapsed: 69.49 s - in org.apache.hadoop.fs.ozone.TestOzoneFileSystemWithFSO
Tests run: 250, Failures: 0, Errors: 0, Skipped: 9, Time elapsed: 250.157 s - in org.apache.hadoop.fs.ozone.TestRootedOzoneFileSystem
Tests run: 108, Failures: 0, Errors: 0, Skipped: 16, Time elapsed: 119.134 s - in org.apache.hadoop.fs.ozone.TestRootedOzoneFileSystemWithFSO

https://github.com/adoroszlai/hadoop-ozone/actions/runs/6248472464/job/16964041346

[sadanand48]

Thanks @adoroszlai for the patch. Moving the tests to a common place sounds like a good idea. We can maybe do it for other tests too in another jira to reduce duplication and have a common place to debug.

[adoroszlai]

It looks like there is some flakiness (visibility problem?) in FS listing with small page size.

[adoroszlai]

@sadanand48 patch is updated, please take another look

[sadanand48]

Thanks for the update @adoroszlai , LGTM

[adoroszlai]

Thanks @sadanand48 for the review (and the fix for HDDS-9347).