Add multifile hybrid scan pass construction API

[mhaseeb123]

Description

Contributes to #22583

This PR adds an API to construct row group passes for multifile hybrid scan reader.

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

📝 Walkthrough

Walkthrough

Adds a multifile public API to compute per-pass, per-source row-group partitions constrained by a memory hint; refactors internal pass construction to accept per-source inputs and return a source map; adds a metadata accessor, updates single-file wrapper/docs, and extends tests.

Changes

Multifile row group pass partitioning API

Layer / File(s)	Summary
Metadata accessor and docs cpp/src/io/parquet/reader_impl_helpers.hpp, cpp/include/cudf/io/experimental/hybrid_scan.hpp	aggregate_reader_metadata::get_num_sources() added; construct_row_group_passes Doxygen updated to throw std::invalid_argument when empty and pass_read_limit text reworded to say a memory limit with 0 meaning no limit (single pass).
Internal API signature and header update cpp/src/io/parquet/experimental/hybrid_scan_impl.hpp	Add <utility> and redeclare hybrid_scan_reader_impl::construct_row_group_passes to accept host_span<std::vector<size_type> const>, total_row_groups, and return pair<vector<vector<size_type>>, vector<size_type>> with updated docs.
Implementation refactor and source tracking cpp/src/io/parquet/experimental/hybrid_scan_impl.cpp	Refactor implementation to validate total_row_groups and source count, build flattened row_groups_info tracking start_row and source_index, early-return when pass_read_limit==0, compute pass offsets, and emit an optional row_group_source_map.
Single-file reader wrapper cpp/src/io/parquet/experimental/hybrid_scan.cpp	Compute total_row_groups, validate non-empty input via CUDF_EXPECTS, wrap single-source indices into per-source vectors, call internal impl with total_row_groups and pass_read_limit, and return the flattened passes.
Public multifile API and implementation cpp/include/cudf/io/experimental/hybrid_scan_multifile.hpp, cpp/src/io/parquet/experimental/hybrid_scan_multifile.cpp	Add public hybrid_scan_multifile::construct_row_group_passes declaration and implementation: compute total_row_groups (std::accumulate), validate non-empty input, call impl to obtain passes and source_map, return early for pass_read_limit==0, else reshape flat passes into per-pass, per-source nested vectors (handles single-source special case).
Tests for pass construction cpp/tests/io/experimental/hybrid_scan_multifile_filters_test.cpp	Extend EmptySource test and add RowGroupPasses and RowGroupPassesSingleSourceParity tests covering invalid/empty input errors, pass partitioning for limits 0, 1, and large values, flattened ordering verification, and parity between multifile and single-file behaviors.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related issues

• [STORY] Support reading multiple files with hybrid scan #22583: Implements multisource support, signature changes, and multifile pass partitioning described in this PR.

Suggested reviewers

• ttnghia
• bdice
• qbacpey
• PointKernel

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 15.38% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely describes the main change: adding a multifile hybrid scan pass construction API, which aligns with the primary objective of the PR.
Description check	✅ Passed	The description is directly related to the changeset, explaining that the PR adds an API to construct row group passes for multifile hybrid scan reader, which matches the implemented changes.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@cpp/src/io/parquet/experimental/hybrid_scan_multifile.cpp`:
- Around line 74-77: Summary: Early return when pass_read_limit == 0 skips
per-source count validation, allowing invalid input in unlimited mode. Fix: move
or replicate the per-source validation logic into the branch handling
pass_read_limit == 0 so that before returning the vector built from
row_group_indices you run the same source-count checks the bounded path
performs; reference the variables/functionality in the file such as
pass_read_limit, row_group_indices and the per-source validation routine (or
inline validation code used in the bounded path) and ensure the branch either
calls that validation helper or performs identical checks before producing the
return value.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 600cc6b3-0b91-42fc-9e84-2ddd86719091

📥 Commits

Reviewing files that changed from the base of the PR and between 1060966 and 7b8bb43.

📒 Files selected for processing (8)

• cpp/include/cudf/io/experimental/hybrid_scan.hpp
• cpp/include/cudf/io/experimental/hybrid_scan_multifile.hpp
• cpp/src/io/parquet/experimental/hybrid_scan.cpp
• cpp/src/io/parquet/experimental/hybrid_scan_impl.cpp
• cpp/src/io/parquet/experimental/hybrid_scan_impl.hpp
• cpp/src/io/parquet/experimental/hybrid_scan_multifile.cpp
• cpp/src/io/parquet/reader_impl_helpers.hpp
• cpp/tests/io/experimental/hybrid_scan_multifile_filters_test.cpp

[coderabbitai]

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Validate source-count even when pass_read_limit == 0.

Line 74 returns early before source-count validation, so invalid per-source input can be accepted in unlimited mode while the bounded path rejects it.

Suggested fix

 std::vector<std::vector<std::vector<size_type>>> hybrid_scan_multifile::construct_row_group_passes(
   cudf::host_span<std::vector<size_type> const> row_group_indices,
   std::size_t pass_read_limit) const
 {
+  auto const num_sources = _impl->parquet_metadatas().size();
+  CUDF_EXPECTS(
+    row_group_indices.size() == num_sources,
+    "Encountered a mismatch in the number of row group indices vectors and the number of input "
+    "datasources",
+    std::invalid_argument);
+
   auto const total_row_groups =
     std::accumulate(row_group_indices.begin(),
                     row_group_indices.end(),
                     std::size_t{0},
                     [](auto sum, auto const& rgs) { return sum + rgs.size(); });
   CUDF_EXPECTS(total_row_groups > 0, "Empty input row group indices encountered");

   if (pass_read_limit == 0) {
     return {
       std::vector<std::vector<size_type>>{row_group_indices.begin(), row_group_indices.end()}};
   }

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@cpp/src/io/parquet/experimental/hybrid_scan_multifile.cpp` around lines 74 -
77, Summary: Early return when pass_read_limit == 0 skips per-source count
validation, allowing invalid input in unlimited mode. Fix: move or replicate the
per-source validation logic into the branch handling pass_read_limit == 0 so
that before returning the vector built from row_group_indices you run the same
source-count checks the bounded path performs; reference the
variables/functionality in the file such as pass_read_limit, row_group_indices
and the per-source validation routine (or inline validation code used in the
bounded path) and ensure the branch either calls that validation helper or
performs identical checks before producing the return value.

[coderabbitai]

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

cpp/include/cudf/io/experimental/hybrid_scan_multifile.hpp (1)

179-179: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

HIGH: Incorrect exception type in documentation

Issue: Documentation claims cudf::logic_error is thrown, but implementation throws std::invalid_argument
Why: Public API contract mismatch will cause user exception handling to fail

Evidence from implementation (cpp/src/io/parquet/experimental/hybrid_scan_multifile.cpp:98-100):

CUDF_EXPECTS(
  total_row_groups > 0, "Empty input row group indices encountered", std::invalid_argument);

Tests confirm std::invalid_argument is expected (cpp/tests/io/experimental/hybrid_scan_multifile_filters_test.cpp:274-279).

Suggested fix:

-   * `@throws` cudf::logic_error if no row group indices in the input
+   * `@throws` std::invalid_argument if no row group indices in the input

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@cpp/include/cudf/io/experimental/hybrid_scan_multifile.hpp` at line 179, The
docstring in hybrid_scan_multifile.hpp incorrectly states that cudf::logic_error
is thrown; the implementation (CUDF_EXPECTS check on total_row_groups producing
std::invalid_argument) and tests expect std::invalid_argument—update the `@throws`
tag to std::invalid_argument to match CUDF_EXPECTS (the total_row_groups check)
and existing tests, ensuring the public API documentation aligns with the actual
exception type thrown.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@cpp/include/cudf/io/experimental/hybrid_scan_multifile.hpp`:
- Line 179: The docstring in hybrid_scan_multifile.hpp incorrectly states that
cudf::logic_error is thrown; the implementation (CUDF_EXPECTS check on
total_row_groups producing std::invalid_argument) and tests expect
std::invalid_argument—update the `@throws` tag to std::invalid_argument to match
CUDF_EXPECTS (the total_row_groups check) and existing tests, ensuring the
public API documentation aligns with the actual exception type thrown.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 7b7a0508-e92d-4543-b73a-4e77ab0d87a3

📥 Commits

Reviewing files that changed from the base of the PR and between 4b45eef and 6aa7efe.

📒 Files selected for processing (5)

• cpp/include/cudf/io/experimental/hybrid_scan_multifile.hpp
• cpp/src/io/parquet/experimental/hybrid_scan_impl.cpp
• cpp/src/io/parquet/experimental/hybrid_scan_impl.hpp
• cpp/src/io/parquet/experimental/hybrid_scan_multifile.cpp
• cpp/tests/io/experimental/hybrid_scan_multifile_filters_test.cpp

🚧 Files skipped from review as they are similar to previous changes (4)

• cpp/src/io/parquet/experimental/hybrid_scan_multifile.cpp
• cpp/src/io/parquet/experimental/hybrid_scan_impl.cpp
• cpp/src/io/parquet/experimental/hybrid_scan_impl.hpp
• cpp/tests/io/experimental/hybrid_scan_multifile_filters_test.cpp

[mhaseeb123]

/merge

[mhaseeb123]

/merge