fix(batches): encode public model group on background-created output file ids

[devin-ai-integration]

TLDR

Problem this solves:

• Batch results are unreadable when the poller creates the output file id
• That id encodes the provider model, not the model group
• Key model-access checks then reject every content read

How it solves it:

• Resolve the model group from the batch's managed input file
• Fall back to the deployment's model_name, never the provider model

Relevant issues

Linear ticket

Resolves LIT-4964

Pre-Submission checklist

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Run end to end against a live proxy on localhost:4000 backed by a real Postgres, with a virtual key scoped to the model group only. The upstream is a local OpenAI-compatible batch server (/v1/files, /v1/batches, /v1/batches/{id}, /v1/files/{id}/content) because this sandbox has no batch-capable provider credentials; everything inside litellm, the managed-files hook, the background CheckBatchCost job, and the auth_utils -> can_key_call_model check, is the real code path

Proxy config; the point is that the public model group and the provider model are different strings, and the poll interval is short so the background job runs during the test:

model_list:
  - model_name: batch-model-group
    litellm_params:
      model: openai/gpt-4.1-nano
      api_key: sk-mock-provider-key
      api_base: http://127.0.0.1:8791/v1

general_settings:
  master_key: sk-1234
  proxy_batch_polling_interval: 10

P=http://127.0.0.1:4000
b64d() { python3 -c "import base64,sys; s=sys.argv[1]; print(base64.urlsafe_b64decode(s+'='*(-len(s)%4)).decode())" "$1"; }

# 1. virtual key scoped ONLY to the model group
KEY=$(curl -s -X POST $P/key/generate -H "Authorization: Bearer sk-1234" -H 'content-type: application/json' \
  -d '{"models":["batch-model-group"],"user_id":"lit4964-user"}' | jq -r .key)

# 2. upload the batch input as a managed file
cat > /tmp/batch_input.jsonl <<'JSONL'
{"custom_id": "request-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "batch-model-group", "messages": [{"role": "user", "content": "hello from LIT-4964 qa"}], "max_tokens": 10}}
JSONL
FILE_ID=$(curl -s -X POST $P/v1/files -H "Authorization: Bearer $KEY" \
  -F purpose=batch -F target_model_names=batch-model-group \
  -F 'file=@/tmp/batch_input.jsonl;type=application/jsonl' | jq -r .id)

# 3. create the batch
BATCH_ID=$(curl -s -X POST $P/v1/batches -H "Authorization: Bearer $KEY" -H 'content-type: application/json' \
  -d '{"input_file_id":"'"$FILE_ID"'","endpoint":"/v1/chat/completions","completion_window":"24h"}' | jq -r .id)

# 4. never call GET /v1/batches/{id}; wait for the background CheckBatchCost job and read the
#    output id back from the DB-backed list, so the poller is the only writer of that id
until OUT=$(curl -s -G $P/v1/batches -H "Authorization: Bearer $KEY" \
  | jq -r --arg b "$BATCH_ID" '.data[] | select(.id==$b) | .output_file_id // empty') && [ -n "$OUT" ]; do sleep 5; done

# 5. what the poller encoded, and 6. what the customer gets
b64d "$OUT"
curl -s -o /tmp/out.txt -w 'HTTP %{http_code}\n' $P/v1/files/$OUT/content -H "Authorization: Bearer $KEY"; cat /tmp/out.txt

Before, with only check_batch_cost.py checked out at the parent commit fa56283806:

=== [BEFORE] 4. wait for the BACKGROUND CheckBatchCost job (client never polls GET /v1/batches/{id}) ===
background job stamped output_file_id after ~25s

=== [BEFORE] 5. output file id minted by the background job ===
decoded: litellm_proxy:application/json;unified_id,71c70c71-c258-4d46-b4e7-c1997a4daad8;target_model_names,gpt-4.1-nano;llm_output_file_id,file-mockoutput-batch_mock_bf2fd40a02b6;llm_output_file_model_id,806c87c3c715ae94391a4ad22c8bd190afe50d80d089a9f27ffbe751e82707fd
ASSERT FAIL: id does NOT encode 'batch-model-group' (see target_model_names above)

=== [BEFORE] 6. customer reads results: GET /v1/files/{output_file_id}/content with the scoped key ===
HTTP 403
{"error":{"message":"key not allowed to access model. This key can only access models=['batch-model-group']. Tried to access gpt-4.1-nano","type":"key_model_access_denied","param":"model","code":"403"}}
ASSERT FAIL: batch output NOT readable (HTTP 403)

After, with this branch and nothing else changed:

=== [AFTER] 4. wait for the BACKGROUND CheckBatchCost job (client never polls GET /v1/batches/{id}) ===
background job stamped output_file_id after ~15s

=== [AFTER] 5. output file id minted by the background job ===
decoded: litellm_proxy:application/json;unified_id,3858151b-41dd-4e3b-830a-edeef3dcab39;target_model_names,batch-model-group;llm_output_file_id,file-mockoutput-batch_mock_9f8dd7f41002;llm_output_file_model_id,806c87c3c715ae94391a4ad22c8bd190afe50d80d089a9f27ffbe751e82707fd
ASSERT PASS: id encodes the public model group 'batch-model-group'

=== [AFTER] 6. customer reads results: GET /v1/files/{output_file_id}/content with the scoped key ===
HTTP 200
{"id": "batch_req_mock", "custom_id": "request-1", "response": {"status_code": 200, "request_id": "req_mock", "body": {"id": "chatcmpl-mock", "object": "chat.completion", "created": 1710000000, "model": "gpt-4.1-nano", "choices": [{"index": 0, "message": {"role": "assistant", "content": "mock batch answer"}, "finish_reason": "stop"}], "usage": {"prompt_tokens": 11, "completion_tokens": 7, "total_tokens": 18}}}, "error": null}
ASSERT PASS: batch output readable (HTTP 200)

The id really was minted by the poller and not by a client retrieve: the proxy log carries CheckBatchCost: converted output_file_id 'file-mockoutput-batch_mock_9f8dd7f41002' -> managed ID, and grep -c 'GET /v1/batches/bGl' litellm_after.log returns 0

Regressions checked on the same live proxy: reading the managed input file (GET /v1/files/{input_file_id}/content) returns 200, and the client-poll path (GET /v1/batches/{id} right after create, then reading its output file) still returns 200

One unrelated finding from the regression pass, same on the parent commit and not touched here: the id minted on the retrieve path decodes to target_model_names, with nothing after it, because litellm/proxy/batches_endpoints/endpoints.py sets the unified_file_id hidden param on create but not on retrieve, so the hook's fallback that recovers the group from the input file never fires. That read only succeeds because an empty target_model_names yields no model candidates and skips the key model-access check altogether

Type

🐛 Bug Fix

Changes

CheckBatchCost._track_completed_batch_cost converts the provider's raw output_file_id / error_file_id into managed unified ids when it beats the client to a completed batch. It passed model_name from get_llm_provider(deployment.litellm_params.model), so the id carried target_model_names,gpt-5.5. The HTTP retrieve path (managed_files.async_post_call_success_hook) instead resolves that name from the batch's managed input file, which holds the model group the caller uploaded with.

That difference is not cosmetic: auth_utils._extract_models_from_managed_resource_id turns target_model_names back into the model a key must be allowed to call, so an id minted by the poller was checked against a provider model no key can call and every GET /v1/files/{output_file_id}/content failed; whichever writer won the race decided whether the customer could read their results.

_get_managed_file_model_name now derives the name the same way the HTTP path does, from the job's managed input file, falling back to deployment_info.model_name (still the model group) when the batch has no managed input file. get_llm_provider's model name stays where it belongs, in the cost calculation.

test_raw_output_file_id_converted_to_managed_id asserted the old provider-model behavior, so its expectation moves to the deployment's model group.

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

---

Note

Medium Risk
Touches batch auth metadata and managed file ID encoding on a background path customers rely on for reading results; scope is narrow with strong regression tests but incorrect fallback could still break key-scoped file access.

Overview
When the CheckBatchCost background poller finishes a batch before the client polls, it mints managed unified IDs for raw output_file_id / error_file_id. Those IDs previously used the provider model from get_llm_provider as target_model_names, so API keys scoped to a public model group failed GET /v1/files/{id}/content with model-access denials.

This change adds _get_managed_file_model_name, which derives the name from the batch job’s managed input file’s target_model_names (same idea as the HTTP managed-files path), and falls back to deployment_info.model_name when there is no managed input file. get_unified_output_file_id now receives that value instead of the provider model.

Tests are updated for the conversion hook expectation and new LIT-4964 coverage asserts encoded model groups and can_key_call_model behavior.

^{Reviewed by Cursor Bugbot for commit 6062ed7. Bugbot is set up for automated code reviews on this repo. Configure here.}

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Greptile Summary

Fixes background-created batch output and error file IDs to preserve the public model group used by access checks.

• Resolves target model names from the batch’s managed input file.
• Falls back to the deployment model group when the input file has no managed metadata.
• Adds regression coverage for ID encoding, model-scoped access, and fallback behavior.

Confidence Score: 5/5

The PR appears safe to merge with no actionable regressions identified.

The managed input-file parser safely falls back for raw or malformed IDs, supports multiple comma-separated model groups, and valid deployments provide the public model name required by the fallback path.

Important Files Changed

Filename	Overview
enterprise/litellm_enterprise/proxy/common_utils/check_batch_cost.py	Background batch tracking now encodes managed output IDs with the public model group while retaining the provider model for cost calculation.
tests/proxy_unit_tests/test_check_batch_cost.py	Updates the previous expectation and adds end-to-end regression coverage for model extraction, authorization, and unmanaged-input fallback.

_{Reviews (1): Last reviewed commit: "fix(batches): encode public model group ..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[devin-ai-integration]

QA'd end to end against a real proxy on localhost:4000 (local Postgres, mock OpenAI-compatible batch provider as the upstream so no provider keys are needed). The deployment maps public group batch-model-group to provider model openai/gpt-4.1-nano, and the virtual key is scoped to models=['batch-model-group'] only. The client never calls GET /v1/batches/{id}, so the background CheckBatchCost job is the only writer of the output file id (confirmed by CheckBatchCost: converted output_file_id ... -> managed ID in the proxy log and zero client batch polls); the id is read back from the DB-backed GET /v1/batches list.

Before (only check_batch_cost.py checked out at parent commit fa56283806):

decoded output_file_id -> ...;target_model_names,gpt-4.1-nano
GET /v1/files/{output_file_id}/content -> 403
  key not allowed to access model. This key can only access models=['batch-model-group'].
  Tried to access gpt-4.1-nano

After (PR code, nothing else changed):

decoded output_file_id -> ...;target_model_names,batch-model-group
GET /v1/files/{output_file_id}/content -> 200, batch output JSONL returned

Regressions checked: plain managed input-file content retrieval -> 200; client-poll path (GET /v1/batches/{id} before the background job) still yields a working output file id -> 200.

Unit suites: LITELLM_LOCAL_MODEL_COST_MAP=True pytest tests/proxy_unit_tests/test_check_batch_cost.py tests/test_litellm/proxy/openai_files_endpoint tests/test_litellm/proxy/batches_endpoints tests/test_litellm/enterprise/proxy -> 200 passed, 2 skipped, 2 xfailed.

Unrelated pre-existing issue noticed while checking (same on the parent commit, not touched by this PR): on the retrieve path the minted output id decodes to target_model_names, (empty), because litellm/proxy/batches_endpoints/endpoints.py sets the unified_file_id hidden param only on create, not on retrieve. Reading that file returns 200 only because an empty target_model_names skips the key-model-access check entirely.

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 6062ed7. Configure here.}

[mateo-berri]

LGTM. Thanks!