Remove synthetic page from adaptive row selection #5
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This reverts commit 8dd46f7.
This reverts commit df78cbf.
Co-authored-by: Andrew Lamb <[email protected]>
Co-authored-by: Andrew Lamb <[email protected]>
|
The fact the tests pass would imply that we don't have good enough test coverage for the situation where the adapative pushdown logic expands the selections to cover pages that aren't technically required... |
Author
|
Yes, I am thinking it is time for a fuzz test with a file with a bunch of small pages and applies filters to one column and selects another |
hhhizzz
force-pushed
the
rowselectionempty
branch
2 times, most recently
from
8742cd1 to
5e81ee4
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Targets this PR:
(merging this PR will update apache#8733)
Per comments from myself and @tustvold https://github.com/apache/arrow-rs/pull/8733/files#r2479110401 the notion of a synthetic page in the reader is somewhat confusing.
Thus I propose removing that notion. I did so and all tests seem to still pass, we'll see if I can get a clean CI run here