fix(proxy): unwrap nested managed batch output file IDs on retrieve - #34083
Conversation
chore(ci): promote internal staging to main
chore(ci): promote internal staging to main
chore(ci): promote internal staging to main
chore(ci): promote internal staging to main
chore(ci): promote internal staging to main
chore(ci): promote internal staging to main
Repeated managed batch retrieve could nest a unified output/error file ID inside another unified ID, so model_mappings pointed at a managed ID instead of the raw provider file-* / gs:// value and content download broke. Add _unwrap_managed_output_file_id to peel nested managed output IDs, keep the outermost response ID, and always persist the raw provider ID. Cover single-layer and nested cases with unit tests. Fixes #33988
Greptile SummaryThis PR makes repeated managed batch retrieval preserve raw provider file IDs. The main changes are:
Confidence Score: 4/5The production ID handling looks correct, but the process-wide test stub needs to be scoped before merging.
tests/test_litellm/enterprise/proxy/test_managed_files_hook.py
|
| Filename | Overview |
|---|---|
| enterprise/litellm_enterprise/proxy/hooks/managed_files.py | Adds cycle-aware nested managed-ID unwrapping and persists the resulting raw provider ID. |
| tests/test_litellm/enterprise/proxy/test_managed_files_hook.py | Adds meaningful retrieval tests but introduces a process-wide proxy-server stub that can leak into other tests. |
Reviews (1): Last reviewed commit: "fix(proxy): unwrap nested managed batch ..." | Re-trigger Greptile
| if "litellm.proxy.proxy_server" not in sys.modules: | ||
| _proxy_server_stub = types.ModuleType("litellm.proxy.proxy_server") | ||
| _proxy_server_stub.llm_router = None | ||
| sys.modules["litellm.proxy.proxy_server"] = _proxy_server_stub |
There was a problem hiding this comment.
Stub Leaks Across Test Modules
This collection-time mutation permanently replaces litellm.proxy.proxy_server in the worker's import cache. Under xdist, later tests in the same worker can receive this stub, which only defines llm_router, and fail when managed-file paths import other real attributes such as scheduler, prisma_client, or user_api_key_cache.
Rule Used: What: Flag any modifications to existing tests and... (source)
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Summary
output_file_id/error_file_idinside another unified ID.model_mappingsthen pointed at a managed ID instead of the raw provider file/gs URI, so content download failed._unwrap_managed_output_file_idto peel nested managed output IDs, keep the outermost response ID, and always persist the raw provider ID.Fixes #33988
Test plan