Re-derive cudf.pandas intermediate proxies when their parents change

[galipremsagar]

Split out of #22903 per review.

Problem

An _IntermediateProxy (e.g. a groupby, rolling, or accessor object) caches its wrapped object at creation time. When a parent proxy (the originating frame/series, or a proxy passed as an argument such as a grouping key) is mutated or replaced after the intermediate is created, the intermediate keeps returning the stale snapshot instead of reflecting the live parent.

For example, a column added to a frame after a groupby was created is not visible through that groupby, which diverges from pandas' live-reference semantics:

df = xpd.DataFrame({"a": ["x", "y", "x", "y"], "b": [1, 2, 3, 4]})
gb = df.groupby(df["a"].values)   # falls back to slow path -> proxy created in slow state
df["c"] = df["b"] * 10
gb["c"].sum()                     # previously failed to see column "c"

Fix

Snapshot the identity of each parent proxy's wrapped object when the intermediate is created, and on _fsproxy_slow_to_fast / _fsproxy_fast_to_slow re-derive the intermediate from the originating call when a parent has changed.

Parent collection (_collect_parent_proxies) is intentionally iterative (an explicit stack) rather than a recursive closure: a recursive closure would hold a cell referencing itself, forming a reference cycle that delays reference-counted teardown and breaks code relying on prompt teardown (e.g. the pandas Series.str-accessor circular-reference test).

Tests

Adds test_groupby_reflects_parent_frame_mutation, and removes the now-passing test_iloc_setitem_axis_argument[True/False] xfails from the pandas-tests plugin.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

📝 Walkthrough

Walkthrough

The proxy layer now tracks parent proxies for intermediate results, re-derives cached wrapped objects when parents change, and restores that tracking after unpickling. New regression tests cover groupby behavior after parent mutation, and the pandas-testing-plugin drops two failure-reason entries.

Changes

Proxy parent staleness tracking

Layer / File(s)	Summary
Parent collection and snapshotting python/cudf/cudf/pandas/fast_slow_proxy.py	_collect_parent_proxies walks method-chain args and kwargs, and _fsproxy_wrap records parent proxies plus wrapped-object id snapshots.
Stale value refresh paths python/cudf/cudf/pandas/fast_slow_proxy.py	The slow-to-fast and fast-to-slow conversion paths re-check parent state before returning cached wrapped objects, and __setstate__ rebuilds parent tracking after unpickling.
Groupby mutation regression python/cudf/cudf_pandas_tests/test_cudf_pandas.py	The new tests check groupby behavior after parent DataFrame mutation and assert the proxy state refreshes cached wrapped objects.

Pandas testing plugin mapping cleanup

Layer / File(s)	Summary
Removed failure-reason entries python/cudf/cudf/pandas/scripts/pandas-testing-plugin.py	The mapping removes failure-reason entries for one groupby test and one iloc test case.

Estimated code review effort: 4 (Complex) | ~45 minutes

Possibly related PRs

• rapidsai/cudf#22903: Both PRs modify python/cudf/cudf/pandas/fast_slow_proxy.py to track parent proxies’ wrapped-object identity and re-derive cached conversions when parents change.

Suggested labels: improvement

Suggested reviewers: vyasr, brandon-b-miller, bdice

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main behavior change: re-deriving intermediate cudf.pandas proxies when their parents change.
Description check	✅ Passed	The description matches the code changes and tests, describing the stale-proxy fix and related xfail cleanup.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@python/cudf/cudf/pandas/fast_slow_proxy.py`:
- Around line 424-431: The fast-path in fast_slow_proxy._fsproxy_slow_to_fast()
refreshes the derived object when _fsproxy_parents_changed() is true, but it
only updates the parent snapshot and leaves _fsproxy_wrapped stale. Update the
cached wrapped value to the newly derived result before returning, so subsequent
calls on the same proxy continue to use the refreshed object instead of falling
back to an outdated cache.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 95affa84-66ff-46a8-9c60-e44dd83a96a4

📥 Commits

Reviewing files that changed from the base of the PR and between 69dc079 and 5378f5c.

📒 Files selected for processing (3)

• python/cudf/cudf/pandas/fast_slow_proxy.py
• python/cudf/cudf/pandas/scripts/pandas-testing-plugin.py
• python/cudf/cudf_pandas_tests/test_cudf_pandas.py

💤 Files with no reviewable changes (1)

• python/cudf/cudf/pandas/scripts/pandas-testing-plugin.py

[galipremsagar]

/okay to test 13eca13

[galipremsagar]

/okay to test 78a899e

[vyasr]

Couple of small questions but the implementation seems fine to me.

[galipremsagar]

/merge