Skip to content

[Fix] WatsonX Tests Failing on CI Due to Missing Env Vars - #24814

Merged
yuneng-berri merged 1 commit into
mainfrom
litellm_/agitated-montalcini
Mar 30, 2026
Merged

[Fix] WatsonX Tests Failing on CI Due to Missing Env Vars#24814
yuneng-berri merged 1 commit into
mainfrom
litellm_/agitated-montalcini

Conversation

@yuneng-berri

Copy link
Copy Markdown
Contributor

Summary

Failure Path (Before Fix)

WatsonX tests in tests/llm_translation/test_watsonx.py fail on CI with assert mock_post.call_count == 0 == 1. The tests are fully mocked but the WatsonX provider requires WATSONX_URL and WATSONX_PROJECT_ID env vars to pass validation before reaching the mocked HTTP client. Without them, a validation error is raised and silently swallowed by the fixture's except block.

Fix

Added an autouse fixture that sets dummy WATSONX_URL and WATSONX_PROJECT_ID via monkeypatch, and clears WATSONX_ZENAPIKEY/WATSONX_TOKEN to prevent env leakage from .env files loaded by dotenv.load_dotenv(). No real API keys or CI/CD env var changes needed.

Testing

All 11 tests in tests/llm_translation/test_watsonx.py pass locally.

Type

🐛 Bug Fix
✅ Test

Tests are fully mocked but require WATSONX_URL and WATSONX_PROJECT_ID
to pass provider validation before reaching the mocked HTTP client.
Add an autouse fixture with dummy values and clear WATSONX_ZENAPIKEY/
WATSONX_TOKEN to prevent env leakage from dotenv.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@vercel

vercel Bot commented Mar 30, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
litellm Ready Ready Preview, Comment Mar 30, 2026 8:28pm

Request Review

@CLAassistant

CLAassistant commented Mar 30, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@codspeed-hq

codspeed-hq Bot commented Mar 30, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 16 untouched benchmarks


Comparing litellm_/agitated-montalcini (123052e) with main (1757e1d)

Open in CodSpeed

@greptile-apps

greptile-apps Bot commented Mar 30, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes WatsonX tests in tests/llm_translation/test_watsonx.py that were failing on CI because the provider's validation requires WATSONX_URL and WATSONX_PROJECT_ID env vars to be present before it reaches the mocked HTTP client. Without them, a validation error was silently swallowed inside the fixture's except block, leaving mock_post.call_count == 0 and causing the == 1 assertions to fail.

The fix adds a single autouse=True pytest fixture (watsonx_env_vars) that:

  • Injects dummy WATSONX_URL and WATSONX_PROJECT_ID via monkeypatch (automatically torn down after each test)
  • Clears WATSONX_ZENAPIKEY and WATSONX_TOKEN to prevent real credentials from leaking in via .env files loaded by dotenv

The approach is clean: individual tests that need to override these values (e.g., test_watsonx_token_in_env_var re-sets the token vars; test_watsonx_chat_completions_endpoint_space_id deletes WATSONX_PROJECT_ID) do so via their own monkeypatch calls which correctly override the autouse fixture's values within the same test scope. No real network calls are introduced and no test assertions are weakened.

Confidence Score: 5/5

  • Safe to merge — adds a minimal, correctly-scoped autouse fixture with no assertion changes, no real network calls, and proper monkeypatch teardown.
  • The change is a single, well-targeted autouse fixture. All existing test assertions remain intact. Individual tests that override the fixture's env vars do so correctly via their own monkeypatch calls. No P0 or P1 issues found.
  • No files require special attention.

Important Files Changed

Filename Overview
tests/llm_translation/test_watsonx.py Adds an autouse pytest fixture that injects dummy WATSONX_URL/WATSONX_PROJECT_ID env vars and clears WATSONX_ZENAPIKEY/WATSONX_TOKEN to allow fully-mocked tests to reach the mocked HTTP client on CI without real credentials.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[pytest test starts] --> B[autouse fixture: watsonx_env_vars]
    B --> B1[monkeypatch.setenv WATSONX_URL]
    B1 --> B2[monkeypatch.setenv WATSONX_PROJECT_ID]
    B2 --> B3[monkeypatch.delenv WATSONX_ZENAPIKEY]
    B3 --> B4[monkeypatch.delenv WATSONX_TOKEN]
    B4 --> C{Test type}
    C -- token env var test --> D[test sets WATSONX_ZENAPIKEY or WATSONX_TOKEN]
    C -- space_id test --> E[test deletes WATSONX_PROJECT_ID, sets WATSONX_SPACE_ID]
    C -- other tests --> F[proceed with WATSONX_URL + WATSONX_PROJECT_ID set]
    D --> G[WatsonX provider passes validation]
    E --> G
    F --> G
    G --> H[mocked HTTP client is reached]
    H --> I[mock_post.call_count == 1 assertion passes]
Loading

Reviews (1): Last reviewed commit: "[Fix] WatsonX tests failing on CI due to..." | Re-trigger Greptile

@yuneng-berri
yuneng-berri merged commit f2deefe into main Mar 30, 2026
85 of 118 checks passed
@yuneng-berri
yuneng-berri deleted the litellm_/agitated-montalcini branch March 30, 2026 21:35
fzowl pushed a commit to fzowl/litellm that referenced this pull request Jun 24, 2026
…lcini

[Fix] WatsonX Tests Failing on CI Due to Missing Env Vars
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants