feat(bedrock): support file delete and list for S3-backed managed files - #39836
feat(bedrock): support file delete and list for S3-backed managed files#39836mateo-berri wants to merge 6 commits into
Conversation
Greptile SummaryAdds S3-backed delete and paginated listing support for Bedrock managed files, including purpose-specific input and output bucket selection, trusted deployment credential forwarding, SigV4 request signing, and regression coverage. The latest commit only deduplicates next-page header preparation without changing behavior Confidence Score: 5/5The PR appears safe to merge, with no outstanding correctness, security, or repository-rule issues The previously reported output-bucket and pagination concerns are fixed, all earlier threads are resolved, and the latest helper extraction preserves the existing next-page header behavior
|
| Filename | Overview |
|---|---|
| litellm/llms/bedrock/files/transformation.py | Implements bucket-scoped, SigV4-signed S3 deletion and purpose-aware paginated listing |
| litellm/llms/custom_httpx/llm_http_handler.py | Adds bounded sync and async provider-list pagination while preserving timeout and refreshed-header handling |
| enterprise/litellm_enterprise/proxy/hooks/managed_files.py | Passes an immutable deployment credential snapshot through managed unified-ID deletion |
| litellm/proxy/openai_files_endpoints/files_endpoints.py | Supplies trusted internal deployment credentials to model-routed file listing |
| tests/test_litellm/llms/bedrock/files/test_bedrock_files_transformation.py | Covers request signing, bucket and prefix validation, output listings, pagination, limits, and sync and async behavior |
Reviews (6): Last reviewed commit: "refactor(files): build the next listing ..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
|
bugbot run |
![veria-ai[bot]](https://avatars.githubusercontent.com/in/1723776?s=60&v=4){kind=small}
PR overviewAll previously flagged issues have been addressed. No open security concerns remain on this pull request. Security reviewNo open security issues remain on this pull request. Fixed/addressed: 1 · PR risk: 0/10 |
…every page Following S3 continuation tokens let GET /v1/files walk an entire managed prefix however large it grew, and the follow-up page fetches dropped the caller's timeout. The handler now stops once MAX_FILE_LIST_LIMIT files are collected (10,000, the most OpenAI returns per list call), slicing the last page to fit, and hands the request timeout to the first and every later page fetch. MAX_FILE_LIST_LIMIT moves to litellm.constants so the proxy's limit validation and the handler share one number
… the managed files hook The enterprise hook reached the constant through the common_utils re-export, which no longer defines it, so point it at the constant's new home
|
bugbot run |
… helper Staging sits exactly at the LIT002 ceiling, so the duplicated validate_environment call for the next page is shared to keep the merged tree under it
|
bugbot run |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 7bddb65. Configure here.
1 participant
TLDR
Problem this solves:
DELETE /v1/files/{id}on a Bedrock managed file returned HTTP 500GET /v1/files?target_model_names=<bedrock model>returned HTTP 500 tooHow it solves it:
purposepicks the bucket and prefix:batch_outputlists the configured output bucket underlitellm-batch-outputs/,batchthe input bucket underlitellm-bedrock-fileslimitceiling) with the request timeout on every pageUser Flow
Before: a developer cleaning up Bedrock batch inputs through the gateway gets HTTP 500 on delete and on listing, and the objects stay in their S3 bucket
purpose=batch,target_model_names=bedrock-claude-haiku-batch, and a JSONL file; they get 200 with a long scrambled id and"status": "uploaded"BedrockFilesConfig does not support file listingBedrockFilesConfig does not support file deletionAfter: the same developer lists the batch files in their bucket and deletes one, and the object is gone from S3
purpose=batch,target_model_names=bedrock-claude-haiku-batch, and a JSONL file; they get 200 with a long scrambled id and"status": "uploaded"s3://bucket/keyid, size, and upload time"purpose": "batch_output"id,bytes,filename,purpose,status)Relevant issues
Supersedes #39731, which added the delete transform only and could not be merged without a signed CLA. Credit to its author for the SigV4 DeleteObject approach
Linear ticket
Resolves LIT-6975
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*,make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more@greptileaito re-request a review after pushing changes)Screenshots / Proof of Fix
Each leg ran two proxy processes booted from that commit, each with
--num_workers 2, sharing one Postgres: instance A takes the uploads and the post-delete checks, instance B the retrieve, list, and delete calls, so every read after a write crosses instances. Real AWS: a Bedrock deployment (bedrock-claude-haiku-batch) withs3_bucket_name: litellm-lit6975-qaands3_output_bucket_name: litellm-lit6975-qa/batch-outin us-west-2, plus an OpenAI deployment on the same routes.aws s3api list-objects-v2at the end of each leg shows what is left in the bucket, and oneaws s3 cpstands in for the result object a finished Bedrock batch job writes under the output bucketBefore (b3c867c, merge base)
Every Bedrock list and delete call returns HTTP 500 and all three objects stay in S3; the OpenAI deployment works either way
Before log (both instances)
Bedrock deployment: two proxy processes sharing one Postgres, each --num_workers 2: A=http://127.0.0.1:40433 (uploads), B=http://127.0.0.1:57590 (retrieve, list, delete)
curl -s -w '\nHTTP %{http_code}' http://127.0.0.1:40433/v1/files -H 'Authorization: Bearer $MK' -F purpose=batch -F file=@lit6975_batch_input.jsonl -F target_model_names=bedrock-claude-haiku-batchcurl -s -w '\nHTTP %{http_code}' http://127.0.0.1:40433/v1/files -H 'Authorization: Bearer $MK' -F purpose=user_data -F file=@lit6975_note.txt -F target_model_names=bedrock-claude-haiku-batchaws s3 cp lit6975_batch_output.jsonl.out s3://litellm-lit6975-qa/batch-out/litellm-batch-outputs/lit6975-job/input.jsonl.out (stands in for the result a finished Bedrock batch job writes under s3_output_bucket_name)curl -s -w '\nHTTP %{http_code}' http://127.0.0.1:57590/v1/files/$FILE_ID -H 'Authorization: Bearer $MK'curl -s -w '\nHTTP %{http_code}' 'http://127.0.0.1:57590/v1/files?purpose=batch&target_model_names=bedrock-claude-haiku-batch' -H 'Authorization: Bearer $MK'aws s3 sync lit6975_page_objs/ s3://litellm-lit6975-qa/litellm-bedrock-files/lit6975-page/ --only-show-errors (stages 1001 managed inputs so the S3 listing spans two 1000-key pages)curl -s -w '\nHTTP %{http_code}' 'http://127.0.0.1:57590/v1/files?purpose=batch&target_model_names=bedrock-claude-haiku-batch' -H 'Authorization: Bearer $MK' | python3 lit6975_page_summary.py (counts the listing instead of dumping 1003 entries)aws s3 rm --recursive s3://litellm-lit6975-qa/litellm-bedrock-files/lit6975-page/ --only-show-errors (removes the 1001 staged inputs again)curl -s -w '\nHTTP %{http_code}' 'http://127.0.0.1:57590/v1/files?target_model_names=bedrock-claude-haiku-batch' -H 'Authorization: Bearer $MK'curl -s -w '\nHTTP %{http_code}' 'http://127.0.0.1:57590/v1/files?purpose=batch_output&target_model_names=bedrock-claude-haiku-batch' -H 'Authorization: Bearer $MK'curl -s -w '\nHTTP %{http_code}' 'http://127.0.0.1:57590/v1/files' -H 'Authorization: Bearer $MK'curl -s -w '\nHTTP %{http_code}' -X DELETE http://127.0.0.1:57590/v1/files/$FILE_ID -H 'Authorization: Bearer $MK'curl -s -w '\nHTTP %{http_code}' -X DELETE http://127.0.0.1:57590/v1/files/$FILE_ID2 -H 'Authorization: Bearer $MK'curl -s -w '\nHTTP %{http_code}' -X DELETE 'http://127.0.0.1:57590/v1/files/$OUT_ID?model=bedrock-claude-haiku-batch' -H 'Authorization: Bearer $MK'curl -s -w '\nHTTP %{http_code}' http://127.0.0.1:40433/v1/files/$FILE_ID -H 'Authorization: Bearer $MK' (after delete, via the other instance)curl -s -w '\nHTTP %{http_code}' 'http://127.0.0.1:40433/v1/files?target_model_names=bedrock-claude-haiku-batch' -H 'Authorization: Bearer $MK' (after delete, via the other instance)curl -s -w '\nHTTP %{http_code}' 'http://127.0.0.1:40433/v1/files?purpose=batch_output&target_model_names=bedrock-claude-haiku-batch' -H 'Authorization: Bearer $MK' (after delete, via the other instance)aws s3api list-objects-v2 --bucket litellm-lit6975-qa --query 'Contents[].Key' --output text (what is left in S3)OpenAI-backed deployment on the same routes (unchanged provider path, same two instances)
curl -s -w '\nHTTP %{http_code}' http://127.0.0.1:40433/v1/files -H 'Authorization: Bearer $MK' -F purpose=batch -F file=@lit6975_openai_batch_input.jsonl -F target_model_names=openai-gpt-minicurl -s -w '\nHTTP %{http_code}' 'http://127.0.0.1:57590/v1/files?purpose=batch&target_model_names=openai-gpt-mini' -H 'Authorization: Bearer $MK' | python3 lit6975_list_summary.py (summarises the OpenAI listing instead of dumping it)curl -s -w '\nHTTP %{http_code}' -X DELETE http://127.0.0.1:57590/v1/files/$FILE_ID3 -H 'Authorization: Bearer $MK'curl -s -w '\nHTTP %{http_code}' http://127.0.0.1:40433/v1/files/$FILE_ID3 -H 'Authorization: Bearer $MK' (after delete, via the other instance)After (7bddb65, PR tip)
Listing returns the batch inputs with their
s3://ids and follows S3 continuation tokens past 1000 keys,purpose=batch_outputreturns the result object from the output bucket, every delete returns 200, the retrieve afterwards is 404 through the other instance, and the bucket is emptyAfter log (both instances)
Bedrock deployment: two proxy processes sharing one Postgres, each --num_workers 2: A=http://127.0.0.1:32309 (uploads), B=http://127.0.0.1:48732 (retrieve, list, delete)
curl -s -w '\nHTTP %{http_code}' http://127.0.0.1:32309/v1/files -H 'Authorization: Bearer $MK' -F purpose=batch -F file=@lit6975_batch_input.jsonl -F target_model_names=bedrock-claude-haiku-batchcurl -s -w '\nHTTP %{http_code}' http://127.0.0.1:32309/v1/files -H 'Authorization: Bearer $MK' -F purpose=user_data -F file=@lit6975_note.txt -F target_model_names=bedrock-claude-haiku-batchaws s3 cp lit6975_batch_output.jsonl.out s3://litellm-lit6975-qa/batch-out/litellm-batch-outputs/lit6975-job/input.jsonl.out (stands in for the result a finished Bedrock batch job writes under s3_output_bucket_name)curl -s -w '\nHTTP %{http_code}' http://127.0.0.1:48732/v1/files/$FILE_ID -H 'Authorization: Bearer $MK'curl -s -w '\nHTTP %{http_code}' 'http://127.0.0.1:48732/v1/files?purpose=batch&target_model_names=bedrock-claude-haiku-batch' -H 'Authorization: Bearer $MK'aws s3 sync lit6975_page_objs/ s3://litellm-lit6975-qa/litellm-bedrock-files/lit6975-page/ --only-show-errors (stages 1001 managed inputs so the S3 listing spans two 1000-key pages)curl -s -w '\nHTTP %{http_code}' 'http://127.0.0.1:48732/v1/files?purpose=batch&target_model_names=bedrock-claude-haiku-batch' -H 'Authorization: Bearer $MK' | python3 lit6975_page_summary.py (counts the listing instead of dumping 1003 entries)aws s3 rm --recursive s3://litellm-lit6975-qa/litellm-bedrock-files/lit6975-page/ --only-show-errors (removes the 1001 staged inputs again)curl -s -w '\nHTTP %{http_code}' 'http://127.0.0.1:48732/v1/files?target_model_names=bedrock-claude-haiku-batch' -H 'Authorization: Bearer $MK'curl -s -w '\nHTTP %{http_code}' 'http://127.0.0.1:48732/v1/files?purpose=batch_output&target_model_names=bedrock-claude-haiku-batch' -H 'Authorization: Bearer $MK'curl -s -w '\nHTTP %{http_code}' 'http://127.0.0.1:48732/v1/files' -H 'Authorization: Bearer $MK'curl -s -w '\nHTTP %{http_code}' -X DELETE http://127.0.0.1:48732/v1/files/$FILE_ID -H 'Authorization: Bearer $MK'curl -s -w '\nHTTP %{http_code}' -X DELETE http://127.0.0.1:48732/v1/files/$FILE_ID2 -H 'Authorization: Bearer $MK'curl -s -w '\nHTTP %{http_code}' -X DELETE 'http://127.0.0.1:48732/v1/files/$OUT_ID?model=bedrock-claude-haiku-batch' -H 'Authorization: Bearer $MK'curl -s -w '\nHTTP %{http_code}' http://127.0.0.1:32309/v1/files/$FILE_ID -H 'Authorization: Bearer $MK' (after delete, via the other instance)curl -s -w '\nHTTP %{http_code}' 'http://127.0.0.1:32309/v1/files?target_model_names=bedrock-claude-haiku-batch' -H 'Authorization: Bearer $MK' (after delete, via the other instance)curl -s -w '\nHTTP %{http_code}' 'http://127.0.0.1:32309/v1/files?purpose=batch_output&target_model_names=bedrock-claude-haiku-batch' -H 'Authorization: Bearer $MK' (after delete, via the other instance)aws s3api list-objects-v2 --bucket litellm-lit6975-qa --query 'Contents[].Key' --output text (what is left in S3)OpenAI-backed deployment on the same routes (unchanged provider path, same two instances)
curl -s -w '\nHTTP %{http_code}' http://127.0.0.1:32309/v1/files -H 'Authorization: Bearer $MK' -F purpose=batch -F file=@lit6975_openai_batch_input.jsonl -F target_model_names=openai-gpt-minicurl -s -w '\nHTTP %{http_code}' 'http://127.0.0.1:48732/v1/files?purpose=batch&target_model_names=openai-gpt-mini' -H 'Authorization: Bearer $MK' | python3 lit6975_list_summary.py (summarises the OpenAI listing instead of dumping it)curl -s -w '\nHTTP %{http_code}' -X DELETE http://127.0.0.1:48732/v1/files/$FILE_ID3 -H 'Authorization: Bearer $MK'curl -s -w '\nHTTP %{http_code}' http://127.0.0.1:32309/v1/files/$FILE_ID3 -H 'Authorization: Bearer $MK' (after delete, via the other instance)Observations from the run:
deleted: trues3://id delete answersdeleted: trueType
🆕 New Feature
🐛 Bug Fix
Caveats (if any)
Low
limitandafterreach no provider listing path yet (OpenAI and Azure included), so threading them through is separate workpurposecomes from the key prefix, so auser_dataupload lists asbatch; reading it per object would cost one HeadObject per keypurposecovers the input bucket only; passpurpose=batch_outputto list a separates3_output_bucket_namedeleted: true, the pre-existing managed-files behavior for every provider (the OpenAI leg at the merge base shows the same shape)s3://ids still work on delete whenrequire_managed_filesis off, scoped to the deployment's buckets and managed prefixes, the same trust boundary the content route already hasproxy_store_model_in_db_testsred staging-wide; test(store_model_in_db): assert the 400 contract in the unknown-model spend log test #39842 fixes itFinal Attestation
Note
Medium Risk
Changes touch S3 delete/list against customer buckets with SigV4 and bucket validation; misconfiguration or credential handoff bugs could delete or expose wrong objects, though scope is limited to managed prefixes and deployment snapshots.
Overview
Adds SigV4-signed S3 delete and list for Bedrock managed batch files, replacing the previous
NotImplementedErrorpaths onDELETEandGET /v1/fileswhen routed to Bedrock.Bedrock
BedrockFilesConfignow builds signed DeleteObject and ListObjectsV2 requests over LiteLLM-managed prefixes (litellm-bedrock-files,litellm-batch-outputs/), validatess3://ids against the deployment bucket, and maps listing XML to OpenAI file objects withpurposefiltering (batchvsbatch_output, including optional output bucket). Shared signing is generalized via_sign_s3_empty_body_requestandS3_SIGNED_REQUEST_HEADERS_PARAM.Listing pagination is extended in the files HTTP handler: providers can implement
transform_list_files_next_request; sync/async list loops follow S3 continuation tokens, apply timeouts on each page, and cap results atMAX_FILE_LIST_LIMIT(centralized inlitellm.constants).Proxy and managed-files wiring passes the deployment credential snapshot (
_litellm_internal_model_credentials) into router delete/list calls so bucket resolution does not depend on env vars alone;litellm.file_deletealso merges trusted credentials for custom provider configs.Reviewed by Cursor Bugbot for commit 7bddb65. Bugbot is set up for automated code reviews on this repo. Configure here.