Support explicit row counts for zero-column tables - #22765
Conversation
madsbk
added
improvement
github-actions
Bot
added
libcudf
b6631bf to
e275230
Compare
e275230 to
cf7a76a
Compare
cf7a76a to
96896b1
Compare
e3a5193 to
5e9d898
Compare
Matt711
left a comment
Matt711
left a comment
There was a problem hiding this comment.
Can you please also remove the workarounds for zero-width tables we use in cudf_polars? https://github.com/rapidsai/cudf/blob/96896b17420f158d0ce2a024ee0cc24ab712dd6a/python/cudf_polars/cudf_polars/containers/dataframe.py#L109
5e9d898 to
a3b3e87
Compare
We need some follow-up work to remove the workaround completely: #22935 |
97c7fdb to
3f79450
Compare
…bk/cudf into cudf-zero-column-with-multiple-rows
|
Seems likely the Java failures here are directly related to these changes |
| } | ||
|
|
||
| auto keys = input_table->select(key_indices); | ||
| auto keys = key_indices.empty() ? cudf::table_view{} : input_table->select(key_indices); |
There was a problem hiding this comment.
I don't think this change is necessary since table_view.select() automatically returns empty table (no column).
There was a problem hiding this comment.
I don't think this change is necessary since
table_view.select()automatically returns empty table (no column).
It is necessary because this PR changes that behavior. With zero-column table support, table_view::select({}) now preserves the source row count, returning an (N, 0) view rather than a (0, 0) table. This is the breaking change described in the PR.
|
/merge |
Follow-up to #22765. Propagate the row count of a zero-column table through the cudf-polars `DataFrame` container and `Projection` node, so a frame with no columns and `N` rows preserves its shape as `(N, 0)` instead of collapsing to `(0, 0)`. Authors: - Mads R. B. Kristensen (https://github.com/madsbk) Approvers: - Matthew Roeschke (https://github.com/mroeschke) - Lawrence Mitchell (https://github.com/wence-) URL: #23234
7 participants
Closes #21428
Part of #21644
Description
Implements support for zero-column tables with
Nrows. Such tables are representable in other ecosystems:StructArray/RecordBatchlengths are independent of fields.(N, 0)DataFrames.Until now, libcudf derived a table's row count from its columns, so a zero-column table always reported
0rows. This PR makes the row count explicit.Motivation
The cudf-polars streaming executor moves tables between partitions and workers using
cudf::packandcudf::unpack. Any intermediate crossing those boundaries must survive a round-trip unchanged.Previously, zero-column tables did not. A
(N, 0)table became(0, 0)after apack/unpackround-trip, causing incorrect results in queries such asselect()orlen()over column-free projections.What this PR changes
tableandtable_viewvia new two-argument constructors. When columns are present, the supplied row count is validated against column sizes.from_arrow_host/from_arrow_device) to preserve the length of zero-field Arrow arrays.pylibcudf.Table.Breaking change
This PR is largely non-breaking. No existing signatures change: the new
table/table_viewtwo-argument constructors are additional overloads, andpylibcudf.Table.num_rowsis a new optional parameter.The one behavioral change is in
table::selectandtable_view::select. An empty selection now preserves the source table's row count. That is,select({})on anN-row table now returns an(N, 0)table/view rather than(0, 0).Non-empty selections are unchanged, as their row count already matched the source table. The only affected callers are those that pass an empty column selection and rely on the result reporting
0rows.The high-level
cudfPython library requires no changes, as it tracks the row count independently.Follow-up work
IO readers are intentionally out of scope. Reading files with rows but no projected columns still needs to preserve row counts. That work (Parquet, ORC, CSV, AVRO, JSON, and experimental readers) will be handled in a follow-up PR as part of #21644.
Tracked in #22935