Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Minor: refine row selection example more #5850

Merged
merged 1 commit into from
Jun 11, 2024

Conversation

alamb
Copy link
Contributor

@alamb alamb commented Jun 6, 2024

Which issue does this PR close?

Follow on to #5824

Rationale for this change

In discussing how row selections interacted with the row group selection with @advancedxy on apache/datafusion#10738 (see https://github.com/apache/datafusion/pull/10738/files#r1628074742) it became clear that the example could be improved to better explain what is going on

What changes are included in this PR?

Refine example added in #5824 to better show how how skip / select works at the end of a row group.

Are there any user-facing changes?

Moar docs, no functional change

@alamb alamb marked this pull request as ready for review June 6, 2024 11:43
@github-actions github-actions bot added the parquet Changes to the parquet crate label Jun 6, 2024
/// Select 50-100 (scan rows 50-100 in row group 2)
/// Skip 50 (skip the first 50 rows in row group 2)
/// Select 50 (scan rows 50-100 in row group 2)
/// Skip 900 (skip the remaining rows in row group 2)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not showing this "skip remaining rows in row group" I think was part of what was confusing

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't remember if this is actually required, the readers actually trim trailing skips, but good to document regardless

@alamb alamb merged commit b8c2741 into apache:master Jun 11, 2024
16 checks passed
@alamb
Copy link
Contributor Author

alamb commented Jun 11, 2024

Thanks @tustvold

@alamb alamb deleted the alamb/row_selection_example branch June 11, 2024 18:28
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
parquet Changes to the parquet crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants