Skip to content

Make pylibcudf.Table.columns() return a tuple - #23040

Merged
rapids-bot[bot] merged 2 commits into
NVIDIA:mainfrom
madsbk:table-columns-tuple
Jun 30, 2026
Merged

Make pylibcudf.Table.columns() return a tuple#23040
rapids-bot[bot] merged 2 commits into
NVIDIA:mainfrom
madsbk:table-columns-tuple

Conversation

@madsbk

@madsbk madsbk commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Description

pylibcudf.Table.columns() now returns a tuple[Column, ...] instead of a list[Column], preventing callers from mutating the table's internal column collection through a borrowed reference.

This aligns with the convention already used in cuDF, where Frame._columns and Frame._column_names are exposed as tuples.

Additionally, this simplifies handling zero-column tables: #22765.

This PR also adds pylibcudf.Table.release(), which transfers ownership of the columns to the caller as a fresh mutable list[Column] and empties the table (mirroring libcudf's table::release()).

Breaking change

This is a breaking change to the public pylibcudf.Table.columns() return type. Pass-through into pylibcudf functions still works (those functions now accept any Sequence), but callers that mutate the returned value must either copy it (list(table.columns())) or use Table.release().

Also, pylibcudf.io.types.TableWithMetadata.columns delegates to Table.columns(), so it now returns a tuple[Column, ...] as well.

@madsbk madsbk self-assigned this Jun 30, 2026
@madsbk madsbk added improvement Improvement / enhancement to an existing function breaking Breaking change labels Jun 30, 2026
@github-actions github-actions Bot added Python Affects Python cuDF API. cudf-polars Issues specific to cudf-polars pylibcudf Issues specific to the pylibcudf package labels Jun 30, 2026
@GPUtester GPUtester moved this to In Progress in cuDF Python Jun 30, 2026
@madsbk
madsbk force-pushed the table-columns-tuple branch 2 times, most recently from 78ad816 to 842a2ed Compare June 30, 2026 12:22
@madsbk
madsbk force-pushed the table-columns-tuple branch from 842a2ed to d732b63 Compare June 30, 2026 12:22
@madsbk
madsbk marked this pull request as ready for review June 30, 2026 13:37
@madsbk
madsbk requested review from a team as code owners June 30, 2026 13:37
@madsbk
madsbk requested review from Matt711 and vyasr June 30, 2026 13:37
@NVIDIA NVIDIA deleted a comment from copy-pr-bot Bot Jun 30, 2026
Comment thread python/pylibcudf/pylibcudf/table.pxd Outdated

@wence- wence- left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good

@coderabbitai

coderabbitai Bot commented Jun 30, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

pylibcudf.Table now stores columns as a tuple, returns a tuple from columns(), and adds release() to transfer ownership as a list while clearing internal state. Several pylibcudf APIs widen sequence inputs from list to Sequence/Iterable, and downstream cudf/cudf_polars call sites update to release() and tuple return types.

Changes

pylibcudf API and downstream adoption

Layer / File(s) Summary
Table: tuple storage, columns() → tuple, new release()
python/pylibcudf/pylibcudf/table.pxd, python/pylibcudf/pylibcudf/table.pyx, python/pylibcudf/pylibcudf/table.pyi
Table._columns is stored as a tuple; columns() returns tuple[Column, ...]; release() returns a list copy and clears _columns. Declarations and stubs are updated.
Column: children accept Iterable
python/pylibcudf/pylibcudf/column.pyx, python/pylibcudf/pylibcudf/column.pyi
Column.__init__ and Column.from_rmm_buffer accept Iterable[Column] for children, materializing to list internally before validation.
concatenate, null_mask, transform: listSequence inputs
python/pylibcudf/pylibcudf/concatenate.pxd, python/pylibcudf/pylibcudf/concatenate.pyx, python/pylibcudf/pylibcudf/concatenate.pyi, python/pylibcudf/pylibcudf/null_mask.pxd, python/pylibcudf/pylibcudf/null_mask.pyx, python/pylibcudf/pylibcudf/null_mask.pyi, python/pylibcudf/pylibcudf/transform.pxd, python/pylibcudf/pylibcudf/transform.pyx, python/pylibcudf/pylibcudf/transform.pyi
Primary sequence parameters for concatenate, bitmask_and, bitmask_or, and transform widen from list to Sequence across declarations, stubs, and docstrings.
TableWithMetadata.columns stub: listtuple
python/pylibcudf/pylibcudf/io/types.pyi, python/pylibcudf/pylibcudf/io/types.pyx
TableWithMetadata.columns returns tuple[Column, ...] in the stub and docstring.
cudf internal helpers: listtuple return annotations
python/cudf/cudf/core/_internals/copying.py, python/cudf/cudf/core/_internals/sorting.py, python/cudf/cudf/core/frame.py, python/cudf/cudf/core/indexed_frame.py, python/cudf_polars/cudf_polars/dsl/ir.py
Return annotations for gather, scatter, columns_split, sort_by_key, _drop_duplicates_columns, _drop_nulls_columns, split_with_dtypes, and Join._reorder_maps change from list to tuple; scatter returns () on empty input.
Callers adopt tbl.release() for parquet and string split
python/cudf/cudf/io/parquet.py, python/cudf_polars/cudf_polars/dsl/ir.py, python/cudf_polars/cudf_polars/dsl/expressions/string.py, python/cudf_polars/cudf_polars/dsl/expressions/rolling.py
The cudf parquet low-memory path and cudf_polars parquet chunked scan switch from .tbl.columns() to .tbl.release(). StringFunction split handling uses release(), and GroupedWindow._build_groupby_requests uses gathered_cols.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

  • rapidsai/cudf#22905 — Both PRs touch python/cudf/cudf/core/_internals/copying.py’s scatter function, with changes to its type/annotation behavior.

Suggested labels

Python, improvement, breaking, cudf-polars, pylibcudf

Suggested reviewers

  • vyasr
  • mroeschke
  • wence-
  • Matt711
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 20.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately summarizes the main change: changing pylibcudf.Table.columns() to return a tuple.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description check ✅ Passed The description matches the changeset, explaining the tuple return type, added release method, and related sequence-accepting updates.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@Matt711 Matt711 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also checked for uses throughout RAPIDS and didn't see any we needed to change outside cudf https://github.com/search?q=org%3Arapidsai+%22.columns%28%29%22&type=code&p=1

Comment thread python/pylibcudf/pylibcudf/table.pyx
@vyasr

vyasr commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

/merge

@rapids-bot
rapids-bot Bot merged commit accbaa6 into NVIDIA:main Jun 30, 2026
136 checks passed
@github-project-automation github-project-automation Bot moved this from In Progress to Done in cuDF Python Jun 30, 2026
@madsbk
madsbk deleted the table-columns-tuple branch June 30, 2026 19:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking Breaking change cudf-polars Issues specific to cudf-polars improvement Improvement / enhancement to an existing function pylibcudf Issues specific to the pylibcudf package Python Affects Python cuDF API.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants