Skip to content

Forward-merge main into pandas3 - #21819

Merged
AyodeAwe merged 7 commits into
pandas3from
main
Mar 17, 2026
Merged

Forward-merge main into pandas3#21819
AyodeAwe merged 7 commits into
pandas3from
main

Conversation

@AyodeAwe

Copy link
Copy Markdown
Contributor

Forward-merge triggered by automated cron job to keep pandas3 up-to-date with main.

If this PR has conflicts, it will remain open for manual resolution.

See forward-merger docs for more info.

vuule and others added 7 commits March 16, 2026 19:53
AI-assisted, human-verified, fixes for the Doxygen parameter documentation - limited to `/src` directory.

We often find these kinds of errors in new PRs. With AI code review we will probably stop merging new errors of this type. This PR aims to fix old error so we hopefully stay 100% accurate from now on.

Authors:
  - Vukasin Milovanovic (https://github.com/vuule)

Approvers:
  - Nghia Truong (https://github.com/ttnghia)
  - Bradley Dice (https://github.com/bdice)
  - Tianyu Liu (https://github.com/kingcrimsontianyu)

URL: #21764
AI-assisted, human-verified, fixes for the Doxygen parameter documentation - limited to `/include` directory.

We often find these kinds of errors in new PRs. With AI code review we will probably stop merging new errors of this type. This PR aims to fix old error so we hopefully stay 100% accurate from now on.

Authors:
  - Vukasin Milovanovic (https://github.com/vuule)

Approvers:
  - Bradley Dice (https://github.com/bdice)
  - Shruti Shivakumar (https://github.com/shrshi)

URL: #21762
Follow up to #21787

Discovered on my cudf_polars unit tests with rapidsmpf branch #21485

Also folds in a change in that PR where a stream wasn't passed to a `plc.Scalar.to_py` call

Authors:
  - Matthew Roeschke (https://github.com/mroeschke)

Approvers:
  - Tom Augspurger (https://github.com/TomAugspurger)

URL: #21798
CSV reader has an option to deduce the compression type from the file extension (default compression option, `AUTO`). However, many tests pass the default compression type when reading for a host buffer. The reader cannot determine the compression type, assumes uncompressed, and emits a warning.

We don't want the noise of these warnings in the unit tests, so this PR specifies the compression type when not reading from a file.

Authors:
  - Vukasin Milovanovic (https://github.com/vuule)

Approvers:
  - Paul Mattione (https://github.com/pmattione-nvidia)
  - David Wendt (https://github.com/davidwendt)

URL: #21794
RAPIDS-CCCL nightly CI failed, probably due to missing headers. This should fix it.

https://github.com/NVIDIA/cccl/actions/runs/23127431740

```
FAILED: CMakeFiles/cudf.dir/src/io/parquet/reader_impl_chunking_utils.cu.o
/usr/bin/sccache /home/coder/.conda/envs/rapids/bin/nvcc -forward-unknown-to-host-compiler -ccbin=/home/coder/.conda/envs/rapids/bin/x86_64-conda-linux-gnu-c++ -DBS_THREAD_POOL_ENABLE_PAUSE=1 -DCCCL_AVOID_SORT_UNROLL=1 -DCCCL_DISABLE_PDL -DCUB_DISABLE_NAMESPACE_MAGIC -DCUB_IGNORE_NAMESPACE_MAGIC_ERROR -DCUDF_KVIKIO_REMOTE_IO -DCUDF_LOG_ACTIVE_LEVEL=RAPIDS_LOGGER_LOG_LEVEL_INFO -DJITIFY_PRINT_LOG=0 -DKVIKIO_CUFILE_FOUND -DKVIKIO_CUFILE_VERSION_API_FOUND -DKVIKIO_LIBCURL_FOUND -DTHRUST_DEVICE_SYSTEM=THRUST_DEVICE_SYSTEM_CUDA -DTHRUST_DISABLE_ABI_NAMESPACE -DTHRUST_FORCE_32_BIT_OFFSET_TYPE=1 -DTHRUST_HOST_SYSTEM=THRUST_HOST_SYSTEM_CPP -DTHRUST_IGNORE_ABI_NAMESPACE_ERROR -DZSTD_STATIC_LINKING_ONLY=0N -Dcudf_EXPORTS -I/home/coder/cudf/cpp/build/conda/cuda-13.0/release/_deps/dlpack-src/include -I/home/coder/cudf/cpp/build/conda/cuda-13.0/release/_deps/jitify-src -I/home/coder/cudf/cpp/include -I/home/coder/cudf/cpp/build/conda/cuda-13.0/release/include -I/home/coder/cudf/cpp/src -I/home/coder/cudf/cpp/build/conda/cuda-13.0/release/_deps/nanoarrow-src/src -I/include -I/home/coder/cudf/cpp/build/conda/cuda-13.0/release/_deps/zstd-src/lib -I/home/coder/cudf/cpp/build/conda/cuda-13.0/release/_deps/cccl-src/lib/cmake/thrust/../../../thrust -I/home/coder/cudf/cpp/build/conda/cuda-13.0/release/_deps/cccl-src/lib/cmake/libcudacxx/../../../libcudacxx/include -I/home/coder/cudf/cpp/build/conda/cuda-13.0/release/_deps/cccl-src/lib/cmake/cub/../../../cub -I/home/coder/cudf/cpp/build/conda/cuda-13.0/release/_deps/nvtx3-src/c/include -I/home/coder/cudf/cpp/build/conda/cuda-13.0/release/_deps/cuco-src/include -I/home/coder/cudf/cpp/build/conda/cuda-13.0/release/_deps/nanoarrow-build/src -I/home/coder/cudf/cpp/build/conda/cuda-13.0/release/_deps/zstd-src/build/cmake/../../lib -isystem /home/coder/rmm/cpp/include -isystem /home/coder/rmm/cpp/build/conda/cuda-13.0/release/include -isystem /home/coder/.conda/envs/rapids/targets/x86_64-linux/include -isystem /home/coder/.conda/envs/rapids/targets/x86_64-linux/include/cccl -isystem /home/coder/kvikio/cpp/build/conda/cuda-13.0/release/_deps/bs_thread_pool-src/include -isystem /home/coder/kvikio/cpp/include -t=1 -O3 -DNDEBUG -std=c++20 "--generate-code=arch=compute_75,code=[sm_75]" -Xcompiler=-fPIC -Xcompiler=-fvisibility=hidden --expt-extended-lambda --expt-relaxed-constexpr -Werror=all-warnings -Xcompiler=-Wall,-Werror,-Wno-error=deprecated-declarations -diag-suppress=1407 -Xfatbin=-compress-all --compress-mode=size -MD -MT CMakeFiles/cudf.dir/src/io/parquet/reader_impl_chunking_utils.cu.o -MF CMakeFiles/cudf.dir/src/io/parquet/reader_impl_chunking_utils.cu.o.d -x cu -c /home/coder/cudf/cpp/src/io/parquet/reader_impl_chunking_utils.cu -o CMakeFiles/cudf.dir/src/io/parquet/reader_impl_chunking_utils.cu.o
/home/coder/cudf/cpp/src/io/parquet/reader_impl_chunking_utils.cu(805): error: namespace "cuda::std" has no member "ceil"
                                 static_cast<size_t>(cuda::std::ceil(cost * adjustment_ratio));
                                                                ^

1 error detected in the compilation of "/home/coder/cudf/cpp/src/io/parquet/reader_impl_chunking_utils.cu".
```

Authors:
  - Bradley Dice (https://github.com/bdice)

Approvers:
  - Muhammad Haseeb (https://github.com/mhaseeb123)
  - David Wendt (https://github.com/davidwendt)

URL: #21784
…tion (#21746)

Introduces a Ray-based execution frontend for **cudf-polars** backed by the RapidsMPF streaming engine, complementing the existing SPMD mode.

Authors:
  - Mads R. B. Kristensen (https://github.com/madsbk)

Approvers:
  - Tom Augspurger (https://github.com/TomAugspurger)
  - Lawrence Mitchell (https://github.com/wence-)
  - Richard (Rick) Zamora (https://github.com/rjzamora)

URL: #21746
The sortedness check in `assert_tpch_result_equal` re-sorted results without passing `nulls_last`, so polars defaulted to `nulls_last=False`. This caused validation failures for queries whose results correctly sorted NULLs last . The re-sort moved NULLs to the front and broke the comparison even though the data was correct.

The fix is to sort NULLs last by default in `assert_tpch_result_equal`, and override for queries that require `NULLS FIRST`.

- Contributes to #21750  by fixing 8 / 14 queries that fail due to sorting issues.

Authors:
  - Matthew Murray (https://github.com/Matt711)

Approvers:
  - Bradley Dice (https://github.com/bdice)
  - Tom Augspurger (https://github.com/TomAugspurger)

URL: #21814
@AyodeAwe
AyodeAwe requested review from a team as code owners March 17, 2026 18:00
@AyodeAwe
AyodeAwe requested review from PointKernel, brandon-b-miller, galipremsagar and mhaseeb123 and removed request for a team March 17, 2026 18:00
@AyodeAwe
AyodeAwe merged commit f022d9e into pandas3 Mar 17, 2026
54 checks passed
@AyodeAwe

Copy link
Copy Markdown
Contributor Author

SUCCESS - forward-merge complete (automated).

@github-actions github-actions Bot added libcudf Affects libcudf (C++/CUDA) code. Python Affects Python cuDF API. cudf-polars Issues specific to cudf-polars labels Mar 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cudf-polars Issues specific to cudf-polars libcudf Affects libcudf (C++/CUDA) code. Python Affects Python cuDF API.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants