fix(packaging): shorten content-filter benchmark fixtures to fit Windows MAX_PATH (#29536)

[Anai-Guo]

Summary

Fixes #29536. On Windows without long-path support (the default), pip install litellm aborts because content-filter benchmark result fixtures bundled in the wheel exceed the 260-char MAX_PATH limit.

#22039 only shortened two files (insults_cf.json, investment_cf.json). The remaining 10 result fixtures still use the legacy long naming scheme and are 77–83 chars:

block_claims_prior_auth_gaming_-_contentfilter_(claims_prior_auth_gaming.yaml).json   (83)
block_gender_discrimination_-_contentfilter_(gender_sexual_orientation.yaml).json     (81)
... (8 more, 70–79 chars)

Fix

Rename the 10 stragglers to the short {topic}_cf.json scheme that test_eval.py::_save_confusion_results already generates today (the current code splits the label on the em-dash and emits f"{topic}_cf"). This matches the two files fixed in #22039, so re-running the eval suite regenerates the same names rather than recreating the long ones.

before	after
block_age_discrimination_-_contentfilter_(age_discrimination.yaml).json	age_discrimination_cf.json
block_claims_fraud_coaching_-_contentfilter_(claims_fraud_coaching.yaml).json	claims_fraud_coaching_cf.json
block_claims_medical_advice_-_contentfilter_(claims_medical_advice.yaml).json	claims_medical_advice_cf.json
block_claims_phi_disclosure_-_contentfilter_(claims_phi_disclosure.yaml).json	claims_phi_disclosure_cf.json
block_claims_prior_auth_gaming_-_contentfilter_(claims_prior_auth_gaming.yaml).json	claims_prior_auth_gaming_cf.json
block_claims_system_override_-_contentfilter_(claims_system_override.yaml).json	claims_system_override_cf.json
block_disability_discrimination_-_contentfilter_(disability.yaml).json	disability_discrimination_cf.json
block_gender_discrimination_-_contentfilter_(gender_sexual_orientation.yaml).json	gender_discrimination_cf.json
block_military_discrimination_-_contentfilter_(military_status.yaml).json	military_discrimination_cf.json
block_religion_discrimination_-_contentfilter_(religion.yaml).json	religion_discrimination_cf.json

Longest basename drops from 83 → 33 chars. Pure rename — file contents (including the full label field inside each JSON) are unchanged, and nothing reads these results by filename (they are only written by the benchmark suite).

🤖 Generated with Claude Code

[greptile-apps]

Greptile Summary

This PR renames 10 benchmark result JSON fixtures from legacy long-form paths (up to 83 chars) to the short {topic}_cf.json naming scheme, fixing Windows MAX_PATH (260-char) installation failures introduced when these files were bundled in the wheel. No file contents are changed — only the filenames.

• All 10 new names exactly match what _save_confusion_results in test_eval.py already generates, so re-running the eval suite will regenerate the same short names rather than recreating the old long ones.
• The longest basename drops from 83 to 33 characters; no code outside test_eval.py reads these result files by name.

Confidence Score: 5/5

Pure rename of benchmark fixture files with no content changes; safe to merge.

All 10 renames are content-free (0 insertions, 0 deletions in the diff), the new names align with what the benchmark harness already generates, and no other code in the repository references these result files by name.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/proxy/guardrails/guardrail_hooks/litellm_content_filter/guardrail_benchmarks/results/age_discrimination_cf.json	Renamed from long legacy path to short {topic}_cf.json scheme; file contents unchanged
litellm/proxy/guardrails/guardrail_hooks/litellm_content_filter/guardrail_benchmarks/results/claims_fraud_coaching_cf.json	Renamed from long legacy path to short {topic}_cf.json scheme; file contents unchanged
litellm/proxy/guardrails/guardrail_hooks/litellm_content_filter/guardrail_benchmarks/results/claims_medical_advice_cf.json	Renamed from long legacy path to short {topic}_cf.json scheme; file contents unchanged
litellm/proxy/guardrails/guardrail_hooks/litellm_content_filter/guardrail_benchmarks/results/claims_phi_disclosure_cf.json	Renamed from long legacy path to short {topic}_cf.json scheme; file contents unchanged
litellm/proxy/guardrails/guardrail_hooks/litellm_content_filter/guardrail_benchmarks/results/claims_prior_auth_gaming_cf.json	Renamed from long legacy path to short {topic}_cf.json scheme; file contents unchanged
litellm/proxy/guardrails/guardrail_hooks/litellm_content_filter/guardrail_benchmarks/results/claims_system_override_cf.json	Renamed from long legacy path to short {topic}_cf.json scheme; file contents unchanged
litellm/proxy/guardrails/guardrail_hooks/litellm_content_filter/guardrail_benchmarks/results/disability_discrimination_cf.json	Renamed from long legacy path to short {topic}_cf.json scheme; file contents unchanged
litellm/proxy/guardrails/guardrail_hooks/litellm_content_filter/guardrail_benchmarks/results/gender_discrimination_cf.json	Renamed from long legacy path to short {topic}_cf.json scheme; file contents unchanged
litellm/proxy/guardrails/guardrail_hooks/litellm_content_filter/guardrail_benchmarks/results/military_discrimination_cf.json	Renamed from long legacy path to short {topic}_cf.json scheme; file contents unchanged
litellm/proxy/guardrails/guardrail_hooks/litellm_content_filter/guardrail_benchmarks/results/religion_discrimination_cf.json	Renamed from long legacy path to short {topic}_cf.json scheme; file contents unchanged

_{Reviews (1): Last reviewed commit: "fix(packaging): shorten content-filter b..." | Re-trigger Greptile}

[codspeed-hq]

Merging this PR will not alter performance

✅ 16 untouched benchmarks

---

_{Comparing Anai-Guo:fix/windows-maxpath-contentfilter-benchmark-fixtures (63a9bf3) with main (5be0797)}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[roman-vm]

Thanks for picking this up — the rename does resolve the immediate MAX_PATH abort (verified against the current 1.87.0 wheel: longest packaged path drops 176 → 134).

One thing worth flagging so this doesn't regress a fourth time (#22039 → here): the recurring cause is that test fixtures are shipped in the wheel at all, and the rename approach has already regressed once. litellm already has the mechanism to fix it durably — [tool.uv.build-backend].source-exclude in pyproject.toml (it already drops litellm/proxy/enterprise and the cache dirs). One line excludes the whole benchmark tree from the distribution while keeping the fixtures in-repo for the eval suite:

source-exclude = [
    "litellm/proxy/enterprise",
    "**/guardrail_benchmarks/**",   # eval/test fixtures — not needed at runtime
    "**/__pycache__", "**/__pycache__/**",
    ...
]

Two notes:

• After the rename, the new longest packaged path is actually litellm/proxy/_experimental/out/experimental/claude-code-plugins/__next.!…txt (134 chars) — a Next.js build artifact, also non-runtime. Worth excluding too.
• A small CI check that fails if any packaged path exceeds ~200 chars would prevent this whole class of regression going forward.

Happy to open a follow-up PR for the source-exclude + CI guard if that's useful.

[Anai-Guo]

Great call — that's the durable fix. I've added **/guardrail_benchmarks/** to [tool.uv.build-backend].source-exclude so the whole benchmark tree is dropped from the wheel while the fixtures stay in-repo for the eval suite. The rename still helps anyone working in-repo on Windows, but this removes the regression class entirely.

I held off on the Next.js _experimental/out artifact: that tree is the built proxy admin UI the server actually serves at runtime, so a blanket exclude there risks breaking the UI — it'd need a path-scoped exclude of just the build-manifest files, which feels like its own change. If you're up for the follow-up PR with that targeted exclude + the >200-char CI guard, that'd be a clean separation. Happy to review it.

🤖 Generated with Claude Code