test(e2e): cover Azure code_interpreter container files by native id with a service-account key

[devin-ai-integration]

TLDR

Problem this solves:

• The customer's Azure container shape had no e2e coverage at all
• fix(router): use forwarded model_id for native Azure container IDs #27921 and fix(containers): record ownership for service-account keys + fix Prisma Json serialization #28990 both shipped without any e2e test turning red
• Their shape: implicit code_interpreter container, native cntr_<hex> id, service-account key, two Azure deployments

How it solves it:

• One live cell under tests/e2e/llm_translation/test_containers_e2e.py that walks that exact shape
• Two Azure deployments registered through /model/new, the decoy first, so routing by container id is what makes it pass
• Every LLM-side call uses a team service-account key of a non-admin user, never the master key
• Red at the merge base of fix(router): use forwarded model_id for native Azure container IDs #27921 and of fix(containers): record ownership for service-account keys + fix Prisma Json serialization #28990, green on main

User Flow

Before: a developer whose app runs on a team service-account key cannot read back the file it just wrote into the container code_interpreter created

1. They send POST https://litellm-domain/v1/responses with their service-account key, their Azure model, and tools: [{"type": "code_interpreter", "container": {"type": "auto"}}], and get a code_interpreter_call output naming a container
2. They keep Azure's own cntr_<hex> id from that output, the way their store already does, and send POST https://litellm-domain/v1/containers/cntr_/files?custom_llm_provider=azure with a small file
3. They get 403 {"detail": "Forbidden"} (on a proxy with no Azure credentials in its env, 500 api_base is required for Azure AI Studio ... Passed api_base=None instead)
4. GET https://litellm-domain/v1/containers/cntr_/files/{file_id}/content?custom_llm_provider=azure never happens, since there is no file id to ask for

After: the same key writes the file and reads it back by the native container id

1. They send the same POST https://litellm-domain/v1/responses and get the same code_interpreter_call output naming a container
2. They send the same POST https://litellm-domain/v1/containers/cntr_/files?custom_llm_provider=azure with the small file
3. They get 200 with a file id shaped like cfile_<hex>
4. GET https://litellm-domain/v1/containers/cntr_/files/{file_id}/content?custom_llm_provider=azure returns 200 with the file's bytes

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Shared setup for every run below: a local proxy booted with unset AZURE_API_BASE AZURE_API_KEY in front of it, Postgres and store_model_in_db: true, and a config whose only deployment is an unrelated placeholder-model. The two Azure deployments come in through POST /model/new (real azure/gpt-5.4-nano, api_version: v1), the decoy with a bogus key registered first. Before was captured on a proxy built at each fix's parent commit, After on this PR's tip, both with the suite file from this PR. Keys are redacted

Before (183092d, parent of #27921, and 9cac0471ae6d, parent of #28990)

Service-account key writes and reads a container file by its native id

1. Ran the suite against the proxy at 9cac0471ae6d: cd tests/e2e && python -m pytest llm_translation/test_containers_e2e.py -p no:cacheprovider -rA --timeout 600
2. Observed:

   E               openai.PermissionDeniedError: Error code: 403 - {'detail': 'Forbidden'}
   FAILED llm_translation/test_containers_e2e.py::TestAzureContainerFiles::test_service_account_key_reads_container_file_by_native_id
   ============================== 1 failed in 39.45s ==============================
   

   with the proxy log showing POST /containers/cntr_.../files?custom_llm_provider=azure HTTP/1.1" 403 Forbidden
3. Ran the same suite against the proxy at 183092d and observed the same 403 (1 failed in 36.38s): the ownership bug fires before the routing bug gets a chance to

Master key writes a container file by its native id with no Azure credentials in the proxy env

1. Against the proxy at 183092d, with the master key so the ownership 403 stays out of the way: POST /model/new (200, probe-azure-27921), then POST /v1/responses with {"model": "probe-azure-27921", "input": "Use python to print 1+1", "tools": [{"type": "code_interpreter", "container": {"type": "auto"}}]}
2. Decoded the container_id in the code_interpreter_call output to its native id: cntr_6ab5e7186f748190841fae3f037cf6630783f99814852b83
3. curl -F file=@probe.txt "$P/v1/containers/$NATIVE/files?custom_llm_provider=azure" with the master key
4. Observed HTTP 500:

   {"error":{"message":"litellm.APIConnectionError: api_base is required for Azure AI Studio. Please set the api_base parameter. Passed `api_base=None` ...
   

After (9caa4b3)

Service-account key writes and reads a container file by its native id

1. Walked the suite's exact routes with curl, master key for setup only:

   ### 1. two Azure deployments through the API, the decoy with a bogus key first (master key)
   model/new -> e2e-containers-walk-decoy id 2aa26d8c-20cd-4847-8b5b-4a16a8ffc42e
   model/new -> e2e-containers-walk id e54bc4ae-59a4-4941-9dc4-dcc021c07690
   
   ### 2. a team, a non-admin user on it, and a service-account key for the team (master key)
   team/new -> team_id 83a0350a-452a-4527-bfc6-6b8738fe3332
   user/new -> user_id 2155eec3-5b6a-41f3-9cc1-5675f1bf8b67 (internal_user)
   team/member_add -> 1 membership
   key/service-account/generate -> key sk-...FPzw (every call below uses it, never the master key)
   
   ### 3. POST /v1/responses with the code_interpreter tool (service-account key)
   status completed, output types ['code_interpreter_call', 'message'], text '42'
   container_id in the output: cntr_bGl0ZWxsbTp... (a litellm envelope), native id inside it: cntr_6ab5f6a5cd948190b18a37c1ad56305f0fbfd7015766957f
   
   ### 4. POST /v1/containers/{native_id}/files (service-account key)
   files -> file id cfile_6ab5f6a99e4881909fcb028ecf5d1490
   
   ### 5. GET /v1/containers/{native_id}/files/{file_id}/content (service-account key)
   hello from e2e-containers-walk
   HTTP 200
   
   ### 6. cleanup
   DELETE /v1/containers/{native_id} -> HTTP 200
   key/delete -> HTTP 200
   team/member_delete -> HTTP 200
   user/delete -> HTTP 200
   team/delete -> HTTP 200
   model/delete e2e-containers-walk-decoy -> HTTP 200
   model/delete e2e-containers-walk -> HTTP 200
   

2. Ran the suite against the same proxy (env | grep -c '^AZURE_API_BASE=' printed 0 first):

   PASSED llm_translation/test_containers_e2e.py::TestAzureContainerFiles::test_service_account_key_reads_container_file_by_native_id
   1 passed in 39.06s
   

Master key writes a container file by its native id with no Azure credentials in the proxy env

1. Same probe as Before against the tip proxy: decoy and real deployment through /model/new, POST /v1/responses with the master key, native id decoded
2. curl -F file=@master.txt "$P/v1/containers/$NATIVE/files?custom_llm_provider=azure" with the master key
3. Observed:

   responses -> status completed, native id cntr_6ab5f632eaf481909be33efce0af68330c76fc94e77df25a
   --- upload with the master key to the native id ---
   {"id":"cfile_6ab5f6363884819094c32da1c287c8a4","object":"container.file","container_id":"cntr_6ab5f632eaf481909be33efce0af68330c76fc94e77df25a","bytes":33,"created_at":1790309942,"path":"/mnt/data/f6be9ee48893ae9cb783f83d95626636-master.txt","source":"user"}
   HTTP 200
   
   DELETE container -> HTTP 200
   

Missing credentials fail the suite instead of skipping it

1. Ran the same test with AZURE_API_BASE and AZURE_API_KEY blank in the pytest env
2. Observed:

   >           pytest.fail("set AZURE_API_BASE and AZURE_API_KEY in the pytest env; the deployments are registered with them")
   E           Failed: set AZURE_API_BASE and AZURE_API_KEY in the pytest env; the deployments are registered with them
   FAILED llm_translation/test_containers_e2e.py::TestAzureContainerFiles::test_service_account_key_reads_container_file_by_native_id
   1 failed in 0.13s
   

CI runs at 9caa4b3

1. The litellm-repo e2e workflow (e2e-changed, run https://github.com/BerriAI/litellm/actions/runs/36093833771) selected this file and ran it three times against the stage-mirror stack:

   test_containers_e2e.py: 1 collected, 0 skipped
   pass 1 of 3 passed
   pass 2 of 3 passed
   pass 3 of 3 passed
   

2. The Buildkite runner: https://buildkite.com/berriai-1/litellm-e2e-pr/builds/1004 ran this suite at this tip on BerriAI/project-releaser#279's branch (the runner pod carrying AZURE_API_BASE and AZURE_API_KEY):

   [gw0] [100%] PASSED llm_translation/test_containers_e2e.py::TestAzureContainerFiles::test_service_account_key_reads_container_file_by_native_id
   1 passed in 43.85s
   

   The PR gate's own trigger, https://buildkite.com/berriai-1/litellm-e2e-pr/builds/1003 on the releaser's main before documentation about replicate is not correct #279 merged, failed the same test with Failed: set AZURE_API_BASE and AZURE_API_KEY in the pytest env; the deployments are registered with them (1 failed, 13 passed), which was the red buildkite/e2e-tests check on this PR until documentation about replicate is not correct #279 merged; documentation about replicate is not correct #279 merged at 131e4eaa, and rebuilding the gate (https://buildkite.com/berriai-1/e2e-tests/builds/18638) triggered https://buildkite.com/berriai-1/litellm-e2e-pr/builds/1005 on the releaser's main, which passed with the suite actually running, so buildkite/e2e-tests is green on this PR now:

   [gw1] [100%] PASSED llm_translation/test_containers_e2e.py::TestAzureContainerFiles::test_service_account_key_reads_container_file_by_native_id
   14 passed, 1 warning in 43.44s
   

Type

✅ Test

Caveats (if any)

Low

• In CI the shared proxy config (tests/e2e/gateway/stage_mirror_ci_config.yml) feeds Azure credentials into the proxy env through files_settings, so a fix(router): use forwarded model_id for native Azure container IDs #27921-style regression (routing lost, global-credential fallback) stays green there, while the fix(containers): record ownership for service-account keys + fix Prisma Json serialization #28990 half (ownership lost, the customer's 403) turns red anywhere. Both halves turned red locally with the proxy booted without Azure env credentials, which is the runbook below and how the Before above was captured. Fixing it in CI is net negative here: the container endpoints return no header naming the deployment that served a call, so the suite cannot assert routing from the response; the other routes are a second Azure resource (new secrets in both CI runners) or a per-suite proxy redeploy without those env credentials, an e2e infra change other Azure suites constrain, tracked as a follow-up
• The streamed /v1/responses variant is not covered here: the container ownership row is written after [DONE] and the OpenAI SDK closes the stream at [DONE], so every follow-up container call 403s (LIT-8612). That cell lands with its fix
• GET /v1/containers?custom_llm_provider=azure with no model to route by and no Azure env credentials answers 500 with the same api_base=None message; not covered here, since the customer's flow names a container every time
• CI reds not from this PR (test-only change under tests/e2e/): the mcp-integration job flaked on its first run and passed on rerun (job https://github.com/BerriAI/litellm/actions/runs/36093833771/job/107944542296); CircleCI local_testing_part1 fails test_get_model_info_bedrock_cross_region_capability_parity exactly as main's own pipeline does (main job https://app.circleci.com/pipelines/github/BerriAI/litellm/90381/workflows/717108e9-cccb-41cc-a28b-782677eb61b9/jobs/2215854), and the rest of the CircleCI reds at pipeline 90383 are also not from this PR, each given its one from-failed rerun (https://app.circleci.com/pipelines/github/BerriAI/litellm/90383/workflows/084b9924-4786-4875-9ab4-4ee3dbf6b5ab): local_testing_part1 also fails test_get_model_info_bedrock_region on the rerun, a CircleCI flaky-insight test with 5 flakes; local_testing_part2 fails test_openai_stream_options_call_text_completion before and after the rerun, a live OpenAI text-completion stream with 12 flakes in the flaky insight and green on main's own pipeline; unit shares 19 failures with main's own unit job (https://app.circleci.com/pipelines/github/BerriAI/litellm/90381/workflows/717108e9-cccb-41cc-a28b-782677eb61b9/jobs/2215836) plus the two prompt-caching event-loop tests in test_prompt_caching_deployment_check.py (flaky insight 3 and 1 flakes), both of which passed on the rerun, where the only failure not on main's list is test_stream_completion_counts_tokens_off_the_event_loop in test_a2a_streaming_iterator.py, another entry in the same flaky insight (1 flake); litellm_router_unit_testing failed one counter test on the first run and passed on the rerun; build_and_test fails the two test_rust_python_harness ocr imports exactly as main's own job (https://app.circleci.com/pipelines/github/BerriAI/litellm/90381/workflows/717108e9-cccb-41cc-a28b-782677eb61b9/jobs/2215842) plus test_key_model_list[/v1/models-key-gpt-3.5-turbo] and [/v1/models-team-gpt-3.5-turbo], whose assert 2 == 1 expectation main rewrote in fix(proxy): authorize key model aliases the same way as team aliases #43049 (1d51a8d, after this branch's merge base), unchanged on the rerun; using_litellm_on_windows times out in its Install Dependencies step on both runs with zero test failures, as main's own windows job did (https://app.circleci.com/pipelines/github/BerriAI/litellm/90381/workflows/717108e9-cccb-41cc-a28b-782677eb61b9/jobs/2215860). None of these checks is required on main, and a test-only PR under tests/e2e/ cannot introduce them

QA runbook

• tests/e2e/llm_translation/test_containers_e2e.py::TestAzureContainerFiles::test_service_account_key_reads_container_file_by_native_id - a team service-account key creates an Azure container implicitly through code_interpreter, writes a file into it by Azure's native id, and reads the bytes back, on a proxy that routes by container id rather than by env credentials
  • Boot the proxy with AZURE_API_BASE and AZURE_API_KEY unset in its env, Postgres, and store_model_in_db: true; export both values only in the shell that runs the test
  • POST /model/new twice with the master key: first azure/gpt-5.4-nano with api_key: decoy-key, then the same model with the real key (api_version: v1), both carrying api_base
  • POST /team/new with models naming the real deployment; POST /user/new with user_role: internal_user; POST /team/member_add; POST /key/service-account/generate with that team and model; use only that key from here on
  • POST /v1/responses with tools: [{"type": "code_interpreter", "container": {"type": "auto"}}] and expect a code_interpreter_call output whose container_id decodes to a cntr_<hex> id
  • POST /v1/containers/{cntr_hex}/files?custom_llm_provider=azure with a small text file and expect 200 with a cfile_ id
  • GET /v1/containers/{cntr_hex}/files/{cfile_id}/content?custom_llm_provider=azure and expect 200 with exactly the bytes uploaded
  • Expect 403 on the upload with the proxy at 9cac0471ae6d, and 500 api_base is required for Azure AI Studio on it with the proxy at 183092d and the master key
  • Sanity check: this test makes sense to add and is not hand-wavey (e.g., assert actual expected spend instead of just spend > 0) or potentially flaky

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

• 9caa4b3 passes /live-pr-risk

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[greptile-apps]

Confidence Score: 5/5

[Low risk] Adds end-to-end test coverage for Azure code interpreter.

The PR appears safe to merge; no new actionable issue was found

Findings

1.  Proxy credentials mask routing failures ▶

Summary

Adds a live Azure Responses E2E test that creates an implicit code-interpreter container with a team service-account key and reads an uploaded file by its native container ID

• Registers the coverage cell and capability, and updates the coverage matrix
• Changes missing Azure credentials from a collection-time skip to a test failure

_{Reviews (3) · Last reviewed commit: "fix(e2e): fail the containers suite when..."}

[greptile-apps]

Proxy credentials mask routing failures The changed-E2E runner exports these Azure credentials to the proxy too. If deployment-ID recovery breaks, container calls can still succeed with global credentials, so this test misses the regression it claims to catch. Run it with credentials available to pytest but absent from the proxy.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[mateo-berri]

Seven e2e suites register Azure deployments through os.environ/AZURE_API_BASE on the proxy, so CI keeps that env; the no-env walk in the proof section covers routing

[greptile-apps]

That context makes sense. Keeping Azure credentials in CI is necessary for the existing suites, and the no-env walk in the proof section validates the deployment-ID routing path. I’ll treat this as an intentional CI limitation rather than request a test change; the remaining risk is only that a future routing regression could be masked in the automated run.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 9caa4b3. Configure here.}

[mateo-berri]

LGTM