test(responses): expect the 404 OpenAI now returns for an unknown model

[mateo-berri]

TLDR

Problem this solves:

• e2e_openai_endpoints is red on every CircleCI pipeline since 2026-09-02
• OpenAI now answers an unknown model with 404, not 400
• test_bad_request_error still expects a 400 BadRequestError for that call

How it solves it:

• Rename it to test_model_not_found_error and expect openai.NotFoundError
• No proxy change: it already mirrors OpenAI's status for that model

User Flow

Before: an engineer with a PR open watches e2e_openai_endpoints go red on a test whose request the proxy actually answers correctly

1. They send POST https://litellm-domain/v1/responses with {"model": "non-existent-model", "input": "This should fail"} to a proxy whose openai/* wildcard route forwards unknown models to OpenAI
2. They get HTTP 404 with code: model_not_found and the text "The model non-existent-model does not exist or you do not have access to it", the same status and text OpenAI itself returns for that model since 2026-09-02
3. The CircleCI job e2e_openai_endpoints runs test_bad_request_error, which still expects a 400 BadRequestError for that call, sees the 404 NotFoundError and fails, so the job is red on their PR and on every other PR

After: the same request gets the same 404 and the job is green again

1. They send POST https://litellm-domain/v1/responses with {"model": "non-existent-model", "input": "This should fail"} to the same proxy
2. They get the same HTTP 404 with code: model_not_found
3. The CircleCI job e2e_openai_endpoints runs test_model_not_found_error, which expects that 404 NotFoundError, and the job passes on their PR

Relevant issues

Linear ticket

Resolves LIT-6776

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Setup shared by both legs: the CI job's own config (litellm/proxy/example_config_yaml/oai_misc_config.yaml, "*" -> openai/*, master key sk-1234, num_retries: 5), a fresh Postgres DB, and the proxy booted with 2 uvicorn workers on a random port. Port 4000 was taken by another proxy on the box and the test file hardcodes http://0.0.0.0:4000, so each leg runs a copy of the test file with only that host:port swapped for 127.0.0.1:20023; the CircleCI runs linked below are the job's own unmodified command

python litellm/proxy/proxy_cli.py --config litellm/proxy/example_config_yaml/oai_misc_config.yaml --port 20023 --num_workers 2

Before (ff1f21a)

1. Control, straight to OpenAI with the request the test sends

curl -s -w '\nHTTP %{http_code}\n' https://api.openai.com/v1/responses -H "Authorization: Bearer $OPENAI_API_KEY" -H 'Content-Type: application/json' -d '{"model":"non-existent-model","input":"This should fail"}'

{"error": {"message": "The model `non-existent-model` does not exist or you do not have access to it.", "type": "invalid_request_error", "param": null, "code": "model_not_found"}}
HTTP 404

2. The test as it is at the merge base, through the proxy

git show ff1f21aea9:tests/openai_endpoints_tests/test_e2e_openai_responses_api.py | sed 's#0.0.0.0:4000#127.0.0.1:20023#g' > legs/test_before_ff1f21aea9.py
python -m pytest legs/test_before_ff1f21aea9.py::test_bad_request_error -s -vv

test_before_ff1f21aea9.py::test_bad_request_error FAILED
E   openai.NotFoundError: Error code: 404 - {'error': {'message': 'litellm.NotFoundError: OpenAIException - {"error": {"message": "The model `non-existent-model` does not exist or you do not have access to it.", "type": "invalid_request_error", "param": null, "code": "model_not_found"}}. Received Model Group=non-existent-model\nAvailable Model Group Fallbacks=None', 'type': None, 'param': None, 'code': '404'}}
1 failed in 0.88s

3. The same failure on CircleCI, the job's own command, on two pipelines from minutes ago: job 2148446 and job 2148355, both red on test_bad_request_error with that same 404. The last green run, job 2143493 on 2026-09-01, shows in its proxy log that OpenAI answered the same request with 400 Bad Request back then

After (d33fe95)

1. The renamed test, same proxy

sed 's#0.0.0.0:4000#127.0.0.1:20023#g' tests/openai_endpoints_tests/test_e2e_openai_responses_api.py > legs/test_after_d33fe95d19.py
python -m pytest legs/test_after_d33fe95d19.py::test_model_not_found_error -s -vv

test_after_d33fe95d19.py::test_model_not_found_error PASSED
1 passed in 7.29s

2. The whole file, real OpenAI and Anthropic calls

python -m pytest legs/test_after_d33fe95d19.py -vv

test_basic_response PASSED
test_streaming_response PASSED
test_model_not_found_error PASSED
test_bad_request_bad_param_error PASSED
test_anthropic_with_responses_api PASSED
test_cancel_response PASSED
test_cancel_streaming_response PASSED
test_cancel_invalid_response_id PASSED
8 passed in 16.63s

3. CircleCI e2e_openai_endpoints at this tip: green, job 2148657, the job's own unmodified command

Type

✅ Test

Caveats (if any)

Low

• The test now tracks OpenAI's live status for an unknown model, so it flips again if OpenAI does
• The proxy mirrors the upstream status on wildcard routes, so this contract is OpenAI's, not LiteLLM's

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR
• d33fe95 passes /live-pr-risk

[devin-ai-integration]

PR #39457 labeled risk:low (1 test file, +3/-4, no sensitive areas). No enterprise label, so Linear routing skipped. #39457

[greptile-apps]

Greptile Summary

This PR updates the Responses API end-to-end test to match OpenAI’s current 404 response for an unknown model.

• Renames the error-path test to describe the model-not-found behavior.
• Expects openai.NotFoundError instead of openai.BadRequestError.
• Leaves the separate malformed-parameter 400 assertion intact.

Confidence Score: 5/5

The PR appears safe to merge because the isolated test update matches the demonstrated upstream response and introduces no production behavior change.

The changed assertion now expects the 404 exception observed through the configured OpenAI wildcard route, while the suite continues to verify 400 handling for malformed parameters separately.

Important Files Changed

Filename	Overview
tests/openai_endpoints_tests/test_e2e_openai_responses_api.py	The test-only exception update matches the demonstrated live wildcard-route behavior without weakening the separate bad-parameter coverage.

_{Reviews (1): Last reviewed commit: "test(responses): expect the 404 OpenAI n..." | Re-trigger Greptile}

[mateo-berri]

bugbot run

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit d33fe95. Configure here.}