Reorganize cuDF documentation to highlight all projects

[mroeschke]

Description

The cuDF documentation, https://docs.rapids.ai/api/cudf/stable/, is primarily cudf Python focused. Similar to what was done in #20357 with the README, I think it would be good to reorient our documentation to highlight all projects on the landing page and migrate all our cuDF Python specific pages into the source/cudf/ directory.

This PR primarily moves documentation and only renames/modifies content for "unification"

• The landing page now mirrors the "About" section in our README, noting all the projects that live in the cudf repo
• The top level "developer guide" now links to each project's developer guide where content was moved
• A few directories renamed for unification e.g. libcudf_docs -> libcudf, user_guide -> cudf (where all the content was cudf Python focused)

I'd recommend reviewing docs/cudf/source/index.rst as it underwent the most change

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: bef83728-a86f-4831-af91-cd9e0be9ab2f

📥 Commits

Reviewing files that changed from the base of the PR and between 730b62d and 3335cfd.

📒 Files selected for processing (1)

• docs/cudf/source/cudf_polars/index.md

✅ Files skipped from review due to trivial changes (1)

• docs/cudf/source/cudf_polars/index.md

---

📝 Walkthrough

Summary by CodeRabbit

• Documentation

  • Reorganized cuDF docs navigation, added developer guide pages, and introduced new cudf/libcudf/pylibcudf sections.
  • Updated numerous internal documentation links and headings for consistency (including README and quick-start references).
  • Added a comprehensive cudf-polars developer guide.

• Chores

  • Adjusted Sphinx build/configuration and updated docstring/help links across the project to reflect the new documentation layout.

Walkthrough

This PR reorganizes the cuDF documentation structure, migrating from user_guide/ and libcudf_docs/ paths to project-aligned cudf/ and libcudf/ locations, adds developer guides, and updates internal links and Python docstring URLs.

Changes

Documentation Structure and Navigation Reorganization

Layer / File(s)	Summary
Top-level documentation landing pages and root navigation docs/cudf/source/index.rst, docs/cudf/source/developer_guide/index.md	Main landing page now describes cuDF as a GPU-accelerated library and adds a list-table of RAPIDS sub-projects; developer guide hub simplified to link to subproject developer docs.
libcudf documentation migration docs/cudf/source/libcudf_docs/index.rst, docs/cudf/source/libcudf/index.rst, docs/cudf/source/libcudf/api_docs/index.rst	Removes deprecated libcudf_docs index and establishes libcudf/ as canonical location with a toctree including API docs and supporting pages.
cuDF core documentation reorganization docs/cudf/source/cudf/index.md, docs/cudf/source/cudf/developer_guide/, docs/cudf/source/conf.py, docs/cudf/README.md	Adds new cuDF developer guide, updates cuDF index header and toctree, and adjusts Sphinx config to reference new API docs paths and navbar alignment.
Sub-project developer documentation docs/cudf/source/cudf_polars/developer_docs.md, docs/cudf/source/cudf_pandas/, docs/cudf/source/pylibcudf/index.rst	Adds comprehensive cudf-polars developer guide and integrates developer_guide into cudf.pandas toctree; simplifies pylibcudf heading.
Documentation content cross-references docs/cudf/source/cudf/data-types.md, docs/cudf/source/cudf/io/io.md, docs/cudf/source/cudf/pandas-comparison.md, docs/dask_cudf/source/best_practices.rst, docs/dask_cudf/source/index.rst	Updates internal cross-references to use new cudf/ and api_docs/ paths; adjusts Dask cuDF tutorial and PandasCompat links.

Python Source Code Docstring URL Updates

Layer / File(s)	Summary
Apply method docstrings python/cudf/cudf/core/dataframe.py, python/cudf/cudf/core/series.py, python/cudf/cudf/core/groupby/groupby.py	Updates DataFrame.apply, Series.apply, and GroupBy.apply docstrings to reference cudf/guide-to-udfs.html instead of user_guide/guide-to-udfs.html.
I/O utilities and UDF configuration python/cudf/cudf/utils/ioutils.py, python/cudf/cudf/core/udf/groupby_typing.py	Updates Parquet/JSON reader docstring option links to api_docs/options and replaces the _UDF_DOC_URL constant and header year to 2026.

Configuration and Root-Level Documentation

Layer / File(s)	Summary
Root documentation files .agents/skills/review-cudf/SKILL.md, README.md	Update SKILL and README to reference libcudf/ endpoint and adjust pylibcudf description formatting.
Contributing guide CONTRIBUTING.md	Fixes local docs build link and updates Python Developer Guide reference to the new cuDF developer guide index.
Project README resources python/dask_cudf/README.md	Update dask-cudf README to reference cudf/10min tutorial and add LocalCUDACluster/spilling guidance.
Build and linting configuration pyproject.toml	Move ruff E711 per-file ignore entry to the notebook's new docs path.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Possibly related PRs

• rapidsai/cudf#22252: Related edits to docs/cudf/source/cudf_polars/index.md and Polars executor docs.

Suggested reviewers

• wence-
• bdice
• vyasr

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately describes the main objective: reorganizing cuDF documentation to highlight all projects in the repository rather than just cuDF Python.
Description check	✅ Passed	The description clearly explains the reorganization effort, including moving documentation into source/cudf/, renaming directories, and updating links throughout the codebase.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 4

🧹 Nitpick comments (1)

docs/cudf/source/cudf/io/io.md (1)

180-180: ⚡ Quick win

Consider using stable docs instead of nightly.

The link points to the nightly version of the options documentation (/api/cudf/nightly/cudf/api_docs/options/), but the context doesn't explain why nightly is used instead of stable. For a general usage guide, linking to stable documentation would typically be more appropriate unless there's a specific reason to reference nightly.

Consider changing to:

-To read a parquet or json file in low memory mode, there are [cuDF options](https://docs.rapids.ai/api/cudf/nightly/cudf/api_docs/options/#api-options) that must be set globally prior to calling the reader. To set those options, call:
+To read a parquet or json file in low memory mode, there are [cuDF options](https://docs.rapids.ai/api/cudf/stable/cudf/api_docs/options/#api-options) that must be set globally prior to calling the reader. To set those options, call:

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/cudf/source/cudf/io/io.md` at line 180, Replace the nightly docs link
used in the "[cuDF options]" anchor with the stable docs URL (change
"/api/cudf/nightly/cudf/api_docs/options/" to the stable path, e.g.
"/api/cudf/stable/cudf/api_docs/options/"), and if the original choice of
nightly was intentional, add a brief parenthetical note after the link
explaining why nightly is referenced (e.g., "nightly features required") so
readers understand the rationale; update the link target in the sentence that
begins "To read a parquet or json file in low memory mode..." to point at the
stable options docs unless there is a documented reason to keep nightly.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@CONTRIBUTING.md`:
- Line 325: Update the broken documentation link referenced by the "[Python
Developer Guide]" text: replace the incorrect URL that contains the extra
"/cudf/" segment
("https://docs.rapids.ai/api/cudf/stable/cudf/developer_guide/") with the
correct path "https://docs.rapids.ai/api/cudf/stable/developer_guide/" so the
CONTRIBUTING.md link points to the proper developer guide.

In `@docs/cudf/source/cudf_polars/developer_docs.md`:
- Around line 672-679: The doc currently describes `roots`, `partition_info`,
and `nodes` IDs as integers but later says they are serialized as strings;
update the descriptions for `roots`, `partition_info`, and the keys in `nodes`
to call them "string-encoded integer IDs" (e.g., "roots: the string-encoded
integer ID for the root nodes") and change the final note to explicitly state
that all node IDs are string-encoded integer IDs to ensure consistency with
`children` references and the serialization format.
- Around line 119-123: The example calls q.collect(engine=pl.GPUEngine(device=1,
memory_resource=mr)) but never defines mr; update the snippet to create and
assign a GPU memory resource object to mr before passing it to pl.GPUEngine
(e.g., instantiate the appropriate Polars/ CUDA memory resource), so that mr is
a valid memory resource when used by GPUEngine and q.collect.

In `@docs/dask_cudf/source/best_practices.rst`:
- Line 75: The link in best_practices.rst points to a non-existent anchor
'#spilling-to-host-memory' on the cuDF developer guide
(https://docs.rapids.ai/api/cudf/stable/cudf/developer_guide/library_design/);
fix by either adding a corresponding section/anchor titled "Spilling to host
memory" in the cuDF developer_guide/library_design page (so the fragment
`#spilling-to-host-memory` resolves) or update the link in best_practices.rst to a
valid existing target (e.g., remove the fragment or replace it with an existing
anchor on the library_design page) and verify the updated URL resolves.

---

Nitpick comments:
In `@docs/cudf/source/cudf/io/io.md`:
- Line 180: Replace the nightly docs link used in the "[cuDF options]" anchor
with the stable docs URL (change "/api/cudf/nightly/cudf/api_docs/options/" to
the stable path, e.g. "/api/cudf/stable/cudf/api_docs/options/"), and if the
original choice of nightly was intentional, add a brief parenthetical note after
the link explaining why nightly is referenced (e.g., "nightly features
required") so readers understand the rationale; update the link target in the
sentence that begins "To read a parquet or json file in low memory mode..." to
point at the stable options docs unless there is a documented reason to keep
nightly.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: f08a4526-a3f0-4555-aded-8bd3d7f54934

📥 Commits

Reviewing files that changed from the base of the PR and between 64a3109 and f15047e.

⛔ Files ignored due to path filters (1)

• docs/cudf/source/cudf/developer_guide/frame_class_diagram.png is excluded by !**/*.png

📒 Files selected for processing (177)

• .agents/skills/review-cudf/SKILL.md
• CONTRIBUTING.md
• README.md
• docs/cudf/README.md
• docs/cudf/source/conf.py
• docs/cudf/source/cudf/10min.ipynb
• docs/cudf/source/cudf/PandasCompat.md
• docs/cudf/source/cudf/api_docs/character_normalizer.rst
• docs/cudf/source/cudf/api_docs/dataframe.rst
• docs/cudf/source/cudf/api_docs/extension_dtypes.rst
• docs/cudf/source/cudf/api_docs/general_functions.rst
• docs/cudf/source/cudf/api_docs/general_utilities.rst
• docs/cudf/source/cudf/api_docs/groupby.rst
• docs/cudf/source/cudf/api_docs/index.rst
• docs/cudf/source/cudf/api_docs/index_objects.rst
• docs/cudf/source/cudf/api_docs/io.rst
• docs/cudf/source/cudf/api_docs/options.rst
• docs/cudf/source/cudf/api_docs/performance_tracking.rst
• docs/cudf/source/cudf/api_docs/series.rst
• docs/cudf/source/cudf/api_docs/tokenize_vocabulary.rst
• docs/cudf/source/cudf/api_docs/window.rst
• docs/cudf/source/cudf/api_docs/wordpiece_tokenizer.rst
• docs/cudf/source/cudf/copy-on-write.md
• docs/cudf/source/cudf/cupy-interop.ipynb
• docs/cudf/source/cudf/data-types.md
• docs/cudf/source/cudf/developer_guide/benchmarking.md
• docs/cudf/source/cudf/developer_guide/contributing_guide.md
• docs/cudf/source/cudf/developer_guide/documentation.md
• docs/cudf/source/cudf/developer_guide/index.md
• docs/cudf/source/cudf/developer_guide/library_design.md
• docs/cudf/source/cudf/developer_guide/options.md
• docs/cudf/source/cudf/developer_guide/testing.md
• docs/cudf/source/cudf/developer_guide/udf_memory_management.md
• docs/cudf/source/cudf/groupby.md
• docs/cudf/source/cudf/guide-to-udfs.ipynb
• docs/cudf/source/cudf/index.md
• docs/cudf/source/cudf/io/index.md
• docs/cudf/source/cudf/io/io.md
• docs/cudf/source/cudf/io/read-json.md
• docs/cudf/source/cudf/memory-profiling.md
• docs/cudf/source/cudf/missing-data.ipynb
• docs/cudf/source/cudf/options.md
• docs/cudf/source/cudf/pandas-2.0-breaking-changes.md
• docs/cudf/source/cudf/pandas-comparison.md
• docs/cudf/source/cudf/performance-comparisons/index.md
• docs/cudf/source/cudf/performance-comparisons/performance-comparisons.ipynb
• docs/cudf/source/cudf_pandas/developer_guide.md
• docs/cudf/source/cudf_pandas/index.rst
• docs/cudf/source/cudf_polars/developer_docs.md
• docs/cudf/source/cudf_polars/index.rst
• docs/cudf/source/developer_guide/index.md
• docs/cudf/source/index.rst
• docs/cudf/source/libcudf/api_docs/aggregation_factories.rst
• docs/cudf/source/libcudf/api_docs/aggregation_groupby.rst
• docs/cudf/source/libcudf/api_docs/aggregation_reduction.rst
• docs/cudf/source/libcudf/api_docs/aggregation_rolling.rst
• docs/cudf/source/libcudf/api_docs/column_aggregation.rst
• docs/cudf/source/libcudf/api_docs/column_apis.rst
• docs/cudf/source/libcudf/api_docs/column_classes.rst
• docs/cudf/source/libcudf/api_docs/column_copy.rst
• docs/cudf/source/libcudf/api_docs/column_factories.rst
• docs/cudf/source/libcudf/api_docs/column_hash.rst
• docs/cudf/source/libcudf/api_docs/column_interop.rst
• docs/cudf/source/libcudf/api_docs/column_join.rst
• docs/cudf/source/libcudf/api_docs/column_merge.rst
• docs/cudf/source/libcudf/api_docs/column_nullmask.rst
• docs/cudf/source/libcudf/api_docs/column_quantiles.rst
• docs/cudf/source/libcudf/api_docs/column_reduction.rst
• docs/cudf/source/libcudf/api_docs/column_reorder.rst
• docs/cudf/source/libcudf/api_docs/column_reshape.rst
• docs/cudf/source/libcudf/api_docs/column_search.rst
• docs/cudf/source/libcudf/api_docs/column_sort.rst
• docs/cudf/source/libcudf/api_docs/column_transformation.rst
• docs/cudf/source/libcudf/api_docs/copy_concatenate.rst
• docs/cudf/source/libcudf/api_docs/copy_gather.rst
• docs/cudf/source/libcudf/api_docs/copy_scatter.rst
• docs/cudf/source/libcudf/api_docs/copy_shift.rst
• docs/cudf/source/libcudf/api_docs/copy_slice.rst
• docs/cudf/source/libcudf/api_docs/copy_split.rst
• docs/cudf/source/libcudf/api_docs/cudf_classes.rst
• docs/cudf/source/libcudf/api_docs/cudf_namespace.rst
• docs/cudf/source/libcudf/api_docs/datetime_apis.rst
• docs/cudf/source/libcudf/api_docs/datetime_compute.rst
• docs/cudf/source/libcudf/api_docs/datetime_extract.rst
• docs/cudf/source/libcudf/api_docs/default_stream.rst
• docs/cudf/source/libcudf/api_docs/dictionary_apis.rst
• docs/cudf/source/libcudf/api_docs/dictionary_classes.rst
• docs/cudf/source/libcudf/api_docs/dictionary_encode.rst
• docs/cudf/source/libcudf/api_docs/dictionary_search.rst
• docs/cudf/source/libcudf/api_docs/dictionary_update.rst
• docs/cudf/source/libcudf/api_docs/expressions.rst
• docs/cudf/source/libcudf/api_docs/fixed_point_classes.rst
• docs/cudf/source/libcudf/api_docs/index.rst
• docs/cudf/source/libcudf/api_docs/interop_arrow.rst
• docs/cudf/source/libcudf/api_docs/interop_dlpack.rst
• docs/cudf/source/libcudf/api_docs/io_apis.rst
• docs/cudf/source/libcudf/api_docs/io_datasinks.rst
• docs/cudf/source/libcudf/api_docs/io_datasources.rst
• docs/cudf/source/libcudf/api_docs/io_readers.rst
• docs/cudf/source/libcudf/api_docs/io_types.rst
• docs/cudf/source/libcudf/api_docs/io_utils.rst
• docs/cudf/source/libcudf/api_docs/io_writers.rst
• docs/cudf/source/libcudf/api_docs/json_apis.rst
• docs/cudf/source/libcudf/api_docs/json_object.rst
• docs/cudf/source/libcudf/api_docs/label_bins.rst
• docs/cudf/source/libcudf/api_docs/labeling_apis.rst
• docs/cudf/source/libcudf/api_docs/lists_apis.rst
• docs/cudf/source/libcudf/api_docs/lists_classes.rst
• docs/cudf/source/libcudf/api_docs/lists_combine.rst
• docs/cudf/source/libcudf/api_docs/lists_contains.rst
• docs/cudf/source/libcudf/api_docs/lists_elements.rst
• docs/cudf/source/libcudf/api_docs/lists_extract.rst
• docs/cudf/source/libcudf/api_docs/lists_filling.rst
• docs/cudf/source/libcudf/api_docs/lists_filtering.rst
• docs/cudf/source/libcudf/api_docs/lists_gather.rst
• docs/cudf/source/libcudf/api_docs/lists_modify.rst
• docs/cudf/source/libcudf/api_docs/lists_sort.rst
• docs/cudf/source/libcudf/api_docs/memory_resource.rst
• docs/cudf/source/libcudf/api_docs/nvtext_apis.rst
• docs/cudf/source/libcudf/api_docs/nvtext_edit_distance.rst
• docs/cudf/source/libcudf/api_docs/nvtext_jaccard.rst
• docs/cudf/source/libcudf/api_docs/nvtext_minhash.rst
• docs/cudf/source/libcudf/api_docs/nvtext_ngrams.rst
• docs/cudf/source/libcudf/api_docs/nvtext_normalize.rst
• docs/cudf/source/libcudf/api_docs/nvtext_replace.rst
• docs/cudf/source/libcudf/api_docs/nvtext_stemmer.rst
• docs/cudf/source/libcudf/api_docs/nvtext_tokenize.rst
• docs/cudf/source/libcudf/api_docs/reorder_compact.rst
• docs/cudf/source/libcudf/api_docs/reorder_partition.rst
• docs/cudf/source/libcudf/api_docs/reshape_transpose.rst
• docs/cudf/source/libcudf/api_docs/scalar_classes.rst
• docs/cudf/source/libcudf/api_docs/scalar_factories.rst
• docs/cudf/source/libcudf/api_docs/set_operations.rst
• docs/cudf/source/libcudf/api_docs/strings_apis.rst
• docs/cudf/source/libcudf/api_docs/strings_case.rst
• docs/cudf/source/libcudf/api_docs/strings_classes.rst
• docs/cudf/source/libcudf/api_docs/strings_combine.rst
• docs/cudf/source/libcudf/api_docs/strings_contains.rst
• docs/cudf/source/libcudf/api_docs/strings_convert.rst
• docs/cudf/source/libcudf/api_docs/strings_copy.rst
• docs/cudf/source/libcudf/api_docs/strings_extract.rst
• docs/cudf/source/libcudf/api_docs/strings_find.rst
• docs/cudf/source/libcudf/api_docs/strings_modify.rst
• docs/cudf/source/libcudf/api_docs/strings_regex.rst
• docs/cudf/source/libcudf/api_docs/strings_replace.rst
• docs/cudf/source/libcudf/api_docs/strings_slice.rst
• docs/cudf/source/libcudf/api_docs/strings_split.rst
• docs/cudf/source/libcudf/api_docs/strings_types.rst
• docs/cudf/source/libcudf/api_docs/structs_classes.rst
• docs/cudf/source/libcudf/api_docs/table_classes.rst
• docs/cudf/source/libcudf/api_docs/tdigest.rst
• docs/cudf/source/libcudf/api_docs/timestamp_classes.rst
• docs/cudf/source/libcudf/api_docs/transformation_binaryops.rst
• docs/cudf/source/libcudf/api_docs/transformation_fill.rst
• docs/cudf/source/libcudf/api_docs/transformation_replace.rst
• docs/cudf/source/libcudf/api_docs/transformation_transform.rst
• docs/cudf/source/libcudf/api_docs/transformation_unaryops.rst
• docs/cudf/source/libcudf/api_docs/utility_apis.rst
• docs/cudf/source/libcudf/api_docs/utility_bitmask.rst
• docs/cudf/source/libcudf/api_docs/utility_dispatcher.rst
• docs/cudf/source/libcudf/api_docs/utility_error.rst
• docs/cudf/source/libcudf/api_docs/utility_span.rst
• docs/cudf/source/libcudf/api_docs/utility_types.rst
• docs/cudf/source/libcudf/index.rst
• docs/cudf/source/libcudf/md_regex.rst
• docs/cudf/source/libcudf/unicode_limitations.rst
• docs/cudf/source/libcudf_docs/index.rst
• docs/cudf/source/pylibcudf/index.rst
• docs/dask_cudf/source/best_practices.rst
• docs/dask_cudf/source/index.rst
• pyproject.toml
• python/cudf/cudf/core/dataframe.py
• python/cudf/cudf/core/groupby/groupby.py
• python/cudf/cudf/core/series.py
• python/cudf/cudf/core/udf/groupby_typing.py
• python/cudf/cudf/utils/ioutils.py
• python/dask_cudf/README.md

💤 Files with no reviewable changes (2)

• docs/cudf/source/libcudf/api_docs/index.rst
• docs/cudf/source/libcudf_docs/index.rst

[madsbk]

LGTM, thanks @mroeschke

[vyasr]

These changes look fine to me, thanks Matt!

[vyasr]

I clicked through the generated preview and things generally look good. My one gripe is that the top-level landing page doesn't look so great anymore. It's just the wide table pasted below, which seems too wide, and there is also no corresponding sidebar. If we could either add back some performance plot like before or introduce some navigational bar to this page I think it would improve the appearance.

[mroeschke]

/merge