fix(cudf.pandas): fix Excel/openpyxl test failures and add xlsxwriter dependency - #22722
Merged
Merged
Conversation
vyasr
added
bug
📝 WalkthroughSummary by CodeRabbit
WalkthroughThis PR adds xlsxwriter to test dependency bundles and generated conda envs, implements ExcelWriter proxy construction and engine-specific proxy subclasses (OpenpyxlWriter, XlsxWriter), updates pandas-testing expected-failure reasons for openpyxl cases, and extends SKILL.md with dependency/environment-gap diagnosis and proxy-fix constraints. ChangesExcelWriter proxy support with xlsxwriter dependency
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.agents/skills/debug-cudf-pandas/SKILL.md:
- Around line 295-296: Update the xfail dict entry so its explanation string
states the root cause (e.g., "openpyxl limitation" or "pandas test bug: assumes
xlsxwriter present") instead of the raw error message, and fix the accidental
setext heading parsing by inserting a blank line (or rewording/escaping)
immediately before the `---` horizontal rule so the `---` is treated as a rule
rather than a setext underline; locate the xfail dict and the `---` separator in
SKILL.md to apply these changes.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 020e9111-9a17-4540-84fc-145d38441cdf
📒 Files selected for processing (9)
.agents/skills/debug-cudf-pandas/SKILL.mdconda/environments/all_cuda-129_arch-aarch64.yamlconda/environments/all_cuda-129_arch-x86_64.yamlconda/environments/all_cuda-132_arch-aarch64.yamlconda/environments/all_cuda-132_arch-x86_64.yamldependencies.yamlpython/cudf/cudf/pandas/_wrappers/pandas.pypython/cudf/cudf/pandas/scripts/pandas-testing-plugin.pypython/cudf/pyproject.toml
Comment on lines
+295
to
296
| The explanation string in the xfail dict should describe the *root cause* (e.g. "openpyxl limitation", "pandas test bug: assumes xlsxwriter present"), not just the error message. | ||
| --- |
There was a problem hiding this comment.
Fix accidental setext heading parsing before horizontal rule.
Line 296 is being interpreted as a setext underline for Line 295, which triggers markdownlint MD003. Add a blank line before --- (or escape/reword) so it’s treated as a horizontal rule.
Suggested fix
The explanation string in the xfail dict should describe the *root cause* (e.g. "openpyxl limitation", "pandas test bug: assumes xlsxwriter present"), not just the error message.
+
---🧰 Tools
🪛 markdownlint-cli2 (0.22.1)
[warning] 295-295: Heading style
Expected: atx; Actual: setext
(MD003, heading-style)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.agents/skills/debug-cudf-pandas/SKILL.md around lines 295 - 296, Update the
xfail dict entry so its explanation string states the root cause (e.g.,
"openpyxl limitation" or "pandas test bug: assumes xlsxwriter present") instead
of the raw error message, and fix the accidental setext heading parsing by
inserting a blank line (or rewording/escaping) immediately before the `---`
horizontal rule so the `---` is treated as a rule rather than a setext
underline; locate the xfail dict and the `---` separator in SKILL.md to apply
these changes.
jameslamb
approved these changes
Jun 1, 2026
galipremsagar
approved these changes
Jun 1, 2026
…r dep The _ExcelWriter__exit__ override was suppressing an IndexError that vanilla pandas also raises on empty workbooks. This made cudf.pandas diverge from pandas behavior. The root cause was a missing xlsxwriter dependency — pandas CI has it installed, causing ExcelWriter to dispatch to xlsxwriter (which handles empty workbooks) rather than openpyxl. Add xlsxwriter to test_python_cudf_pandas dependencies to match pandas CI, and remove the unnecessary proxy override, gc, and zipfile imports. Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Add lessons learned from debugging openpyxl test failures: - New failure category: dependency/environment gaps - New diagnostic step: verify vanilla pandas behavior (Step 3d) - New resolution path: fix dependency gaps (Step 4c) - New unacceptable fix: diverging from pandas to pass a test - Guidance on rapids-dependency-file-generator workflow - xfail strings should describe root cause, not error type Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
xlsxwriter was incorrectly placed in test_python_cudf_pandas, which is for cudf.pandas's own unit tests (run_tests.sh). The pandas test suite under cudf.pandas gets xlsxwriter via pip's pandas[excel] extra, but conda environments need it listed explicitly. Create a new test_cudf_pandas_pandas_tests group with output_types: [conda] for dependencies that pandas CI has installed (via pip extras) but conda environments require explicitly. Add it to the 'all' includes. Update SKILL.md to reference the correct dependency group.
Same root cause as test_engine_kwargs_append_data_only: cudf.pandas fast path raises NotImplementedError, falls back to openpyxl which raises IndexError on workbook with no visible sheets.
vyasr
force-pushed
the
fix/pandas-excel-openpyxl
branch
from
June 1, 2026 16:16
edd9f17 to
ae4b616
Compare
There was a problem hiding this comment.
♻️ Duplicate comments (1)
.agents/skills/debug-cudf-pandas/SKILL.md (1)
297-298:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winFix markdown linting issue: add blank line before horizontal rule.
Line 297 is being interpreted as a setext heading because of the
---on line 298. Add a blank line before the---so it's properly treated as a horizontal rule, not a setext underline.📝 Suggested fix
The explanation string in the xfail dict should describe the *root cause* (e.g. "openpyxl limitation", "pandas test bug: assumes xlsxwriter present"), not just the error message. + ---🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.agents/skills/debug-cudf-pandas/SKILL.md around lines 297 - 298, Add a blank line before the horizontal rule `---` in SKILL.md around the xfail dict section so the `---` is treated as a horizontal rule (not a setext underline); while there, update the xfail dict's `explanation` string to describe the root cause (e.g., "openpyxl limitation" or "pandas test bug: assumes xlsxwriter present") rather than just the raw error message to make the intent clear.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Duplicate comments:
In @.agents/skills/debug-cudf-pandas/SKILL.md:
- Around line 297-298: Add a blank line before the horizontal rule `---` in
SKILL.md around the xfail dict section so the `---` is treated as a horizontal
rule (not a setext underline); while there, update the xfail dict's
`explanation` string to describe the root cause (e.g., "openpyxl limitation" or
"pandas test bug: assumes xlsxwriter present") rather than just the raw error
message to make the intent clear.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 47b7e4ee-a769-4465-b19a-2e8a4c8c67d9
📒 Files selected for processing (8)
.agents/skills/debug-cudf-pandas/SKILL.mdconda/environments/all_cuda-129_arch-aarch64.yamlconda/environments/all_cuda-129_arch-x86_64.yamlconda/environments/all_cuda-132_arch-aarch64.yamlconda/environments/all_cuda-132_arch-x86_64.yamldependencies.yamlpython/cudf/cudf/pandas/_wrappers/pandas.pypython/cudf/cudf/pandas/scripts/pandas-testing-plugin.py
✅ Files skipped from review due to trivial changes (1)
- conda/environments/all_cuda-129_arch-aarch64.yaml
🚧 Files skipped from review as they are similar to previous changes (5)
- conda/environments/all_cuda-132_arch-aarch64.yaml
- conda/environments/all_cuda-129_arch-x86_64.yaml
- conda/environments/all_cuda-132_arch-x86_64.yaml
- python/cudf/cudf/pandas/scripts/pandas-testing-plugin.py
- python/cudf/cudf/pandas/_wrappers/pandas.py
mroeschke
reviewed
Jun 1, 2026
| - output_types: [conda] | ||
| packages: | ||
| - numexpr | ||
| - xlsxwriter |
Contributor
There was a problem hiding this comment.
Just FYI, the pandas unit test job installs these pandas options dependencies specified the test_python_pandas_cudf section, so at some point it would be nice to consolidate test_python_pandas_cudf with test_cudf_pandas_pandas_tests 😅
Contributor
Author
There was a problem hiding this comment.
I specifically created a new list here because I didn't want to force that job to install extra dependencies that are only needed for the pandas test suite. What I'd really love is if we can pull this information out of the pandas's pyproject.toml or something, but that feels like overkill to engineer 😅
Contributor
|
/merge |
This was referenced Jun 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fix cudf.pandas Excel/openpyxl integration test failures by correcting engine dispatch, removing an over-engineered
__exit__override that diverged from vanilla pandas behavior, and addingxlsxwriteras a test dependency (matching pandas CI).Changes:
ExcelWriterengine dispatch so openpyxl is correctly selected for.xlsxfilesengine_kwargspassthrough for openpyxldata_onlyoption__exit__override and relatedgc/zipfileworkarounds that masked real issuesxlsxwriterto cudf.pandas test dependencies (viadependencies.yaml)test_engine_kwargs_append_data_only— an inherent openpyxl limitation (freshly-written files have no formula cache)Checklist