-
-
Notifications
You must be signed in to change notification settings - Fork 11.6k
ci: guard against MAX_PATH-busting packaged wheel paths #29587
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| """Guard against shipping wheel paths long enough to break the Windows 260-char | ||
| MAX_PATH limit at install time. | ||
|
|
||
| litellm has repeatedly shipped content-filter benchmark *test fixtures* with very | ||
| long names under | ||
| ``litellm/proxy/guardrails/guardrail_hooks/litellm_content_filter/guardrail_benchmarks/``. | ||
| On a default Windows machine (long-path support off, the OS default) ``pip install | ||
| litellm`` then aborts mid-unpack with ``OSError: [Errno 2] No such file or directory``, | ||
| leaving a half-installed package (no ``litellm.types``) -> ``ModuleNotFoundError``. | ||
| See issues/PRs #21941, #22039, #29536, #29553 -- it has regressed three times. | ||
|
|
||
| The path that actually lands on disk is ``<site-packages prefix> + <path inside the | ||
| wheel>``. A realistic worst-case Windows prefix (long profile name + roaming AppData | ||
| venv), e.g.:: | ||
|
|
||
| C:\\Users\\Administrator\\AppData\\Roaming\\<app>\\<sub>\\venv\\Lib\\site-packages\\ (~95 chars) | ||
|
|
||
| so we budget ``260 - 100 = 160`` chars for any single path inside the wheel. This is | ||
| measured on the BUILT wheel, so it honours ``[tool.uv.build-backend].source-exclude`` | ||
| (i.e. it passes once excluded fixtures are no longer packaged). | ||
| """ | ||
| import glob | ||
| import os | ||
| import sys | ||
| import zipfile | ||
|
|
||
| # Windows MAX_PATH (260) minus ~100 chars for a realistic install prefix. | ||
| MAX_RELATIVE = 160 | ||
|
|
||
|
|
||
| def main(dist_dir: str) -> int: | ||
| wheels = glob.glob(os.path.join(dist_dir, "*.whl")) | ||
| if not wheels: | ||
| print(f"::error::no .whl found in {dist_dir!r}") | ||
| return 1 | ||
|
|
||
| rc = 0 | ||
| for whl in wheels: | ||
| names = zipfile.ZipFile(whl).namelist() | ||
| longest = max((len(n) for n in names), default=0) | ||
| offenders = sorted( | ||
| (n for n in names if len(n) > MAX_RELATIVE), key=len, reverse=True | ||
| ) | ||
| print(f"{os.path.basename(whl)}: {len(names)} entries, longest path = {longest}") | ||
| if offenders: | ||
| rc = 1 | ||
| print( | ||
| f"::error::{len(offenders)} packaged path(s) exceed {MAX_RELATIVE} chars " | ||
| f"and risk the Windows MAX_PATH limit at install time:" | ||
| ) | ||
| for n in offenders[:15]: | ||
| print(f" {len(n):4} {n}") | ||
| return rc | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| sys.exit(main(sys.argv[1] if len(sys.argv) > 1 else "dist")) | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,28 @@ | ||
| name: Wheel path length | ||
|
|
||
| # Fails if the built wheel contains any path long enough to break the Windows | ||
| # 260-char MAX_PATH limit at install time. See | ||
| # .github/scripts/check_wheel_path_length.py for the rationale (issues #21941, | ||
| # #22039, #29536, #29553). | ||
|
|
||
| on: | ||
| pull_request: | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
The Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time! |
||
| push: | ||
| branches: | ||
| - main | ||
| - litellm_oss_branch | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| check-wheel-path-length: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - name: Install uv | ||
| uses: astral-sh/setup-uv@v5 | ||
| - name: Build wheel | ||
| run: uv build --wheel --out-dir dist | ||
| - name: Guard against MAX_PATH-busting packaged paths | ||
| run: python .github/scripts/check_wheel_path_length.py dist | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ZipFileis opened without a context manager, so the file handle is never explicitly closed. On Windows, this can hold a file lock for longer than necessary (until GC), which can matter if the script is extended to write or move wheel files later. Usewithto guarantee cleanup.