Core: Add RESTCatalog tests for FileIOTracker

[gaborkaszab]

FileIOTracker related functionality in RESTSessionCatalog code isn't covered with tests. The PR adds test coverage, and also some adjustments to make the FileIOTracker available in tests.

[nastra]

@gaborkaszab what's the reason we want to test the track with the rest catalog? We do already have TestFileIOTracker where we test the basic functionality of the tracker

[gaborkaszab]

Hey @nastra ,
Thanks for taking a look! In my opinion the unit tests for TestFileIOTracker are required but not entirely enough. There is untested code in RESTSessionCatalog where we have no detection of a regression if someone messes up the usage of the tracker on the catalog side. I was working on this areas of the code for the client-side table cache for the freshness-aware table loading and I had a false impression that everything is fine with my ongoing changes because the tests passed, however, I did it wrong with the tracker and was surprised there are no test coverage that detects this. Also, with my mentioned freshness loading change I'd like to add further tests relevant for IO tracking together with the table cache and this PR is the first step for that.
Hope this makes sense!

[nastra]

Hey @nastra , Thanks for taking a look! In my opinion the unit tests for TestFileIOTracker are required but not entirely enough. There is untested code in RESTSessionCatalog where we have no detection of a regression if someone messes up the usage of the tracker on the catalog side. I was working on this areas of the code for the client-side table cache for the freshness-aware table loading and I had a false impression that everything is fine with my ongoing changes because the tests passed, however, I did it wrong with the tracker and was surprised there are no test coverage that detects this. Also, with my mentioned freshness loading change I'd like to add further tests relevant for IO tracking together with the table cache and this PR is the first step for that. Hope this makes sense!

I'm currently undecided on this, so let's see what others think

[gaborkaszab]

FYI, this change in freshness-aware loading made me think testing this area has value. But in general, the usage of FileIOTracker inRESTSessionCatalog is untested and some test coverage still makes sense to me.

[gaborkaszab]

Last change is rebase with main to resolve conflicts

[github-actions]

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions.

[gaborkaszab]

Just an FYI, that the freshness-aware loading implementation was changed a bit and it no longer requires FileIOTracker testability.
However, I still think that this part of the code is untested and could use some test coverage. We have test coverage for the FileIOTracker itself, I get it, however, we don't have for the RESTSessionCatalog leveraging the tracker.

Would you mind sharing your view, @amogh-jahagirdar, @danielcweeks ? There is a merge conflict that I'd resolve only if we have consensus that this useful.