fix(llm): auto-append /v1 to LLM_BASE_URL for openai_compatible (#1934) - #2310
ilblackdragon wants to merge 3 commits into
Conversation
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
ilblackdragon
left a comment
There was a problem hiding this comment.
Patch looks small and coherent, and the intent matches the documented local-server use case.
Residual concern: this now rewrites every OpenAI-compatible base URL by appending /v1 unless it already ends in /v1. That is correct for the local servers named in the PR, but it bakes in an assumption about all custom proxies. If you keep this approach, I would at least add a caller-level test around the provider factory so we assert the constructed client base URL for both a bare local endpoint and an already-versioned custom endpoint. Right now the coverage is helper-only.
… handling Addresses PR #2310 review feedback: prior coverage was helper-only. Now asserts OpenAICompatibleProvider constructs the correct client base URL for both bare and pre-versioned LLM_BASE_URL values. Extracts build_openai_compat_client() from create_openai_compat_from_registry() so tests can drive the exact construction path (headers, api key, base URL normalization, rig-core client build, completions_api switch) and inspect the resulting client's base_url(). Adds three caller-level tests covering: bare local endpoint (http://localhost:8080 -> /v1), already-versioned custom proxy (https://custom.proxy/v1 unchanged, no double suffix), and trailing-slash versioned URL. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
Addressed in 25baf2b: added caller-level factory test asserting the constructed OpenAICompatibleProvider base URL for both a bare endpoint ( |
Local model servers (MLX, vLLM, llama.cpp) expect requests at /v1/chat/completions, but rig-core's openai client appends /chat/completions directly to the base URL. Without this fix, LLM_BASE_URL=http://localhost:8080 produces 404 errors. Add normalize_openai_base_url() that appends /v1 when not already present, handling trailing slashes and existing /v1 suffixes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… handling Addresses PR #2310 review feedback: prior coverage was helper-only. Now asserts OpenAICompatibleProvider constructs the correct client base URL for both bare and pre-versioned LLM_BASE_URL values. Extracts build_openai_compat_client() from create_openai_compat_from_registry() so tests can drive the exact construction path (headers, api key, base URL normalization, rig-core client build, completions_api switch) and inspect the resulting client's base_url(). Adds three caller-level tests covering: bare local endpoint (http://localhost:8080 -> /v1), already-versioned custom proxy (https://custom.proxy/v1 unchanged, no double suffix), and trailing-slash versioned URL. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
25baf2b to
275eb7e
Compare
serrrfirat
left a comment
There was a problem hiding this comment.
Review of PR #2310 -- 2 findings (both Medium).
| fn normalize_openai_base_url(url: &str) -> String { | ||
| let trimmed = url.trim_end_matches('/'); | ||
| if trimmed.ends_with("/v1") { | ||
| trimmed.to_string() |
There was a problem hiding this comment.
Medium Severity -- False positive path-segment matching
ends_with("/v1") is a substring check on the full string. A URL ending in /apiv1 or /myv1 would incorrectly match and would NOT get /v1 appended, silently sending requests to the wrong endpoint.
Suggested fix: Check the last path segment explicitly:
trimmed.rsplit_once('/').map(|(_, last)| last) == Some("v1")| @@ -707,6 +718,21 @@ | |||
There was a problem hiding this comment.
Medium Severity -- Debug log shows pre-normalization URL
This tracing::debug! log emits config.base_url (the original, pre-normalization URL), but actual requests go to the normalized URL with /v1 appended (line 22 above). During incident response, this mismatch between logged URL and actual request URL is a debugging trap.
Suggested fix: Log the normalized URL, or log both original and normalized values.
Addresses PR #2310 review: ends_with("/v1") replaced with proper last-path-segment check to avoid false positives like /apiv1. Debug log now shows the normalized base URL. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
Pushed
Quality gate: |
ilblackdragon
left a comment
There was a problem hiding this comment.
All prior review feedback has been addressed in the current diff:
rsplit_once('/').map(|(_, last)| last) == Some("v1")correctly handles false positives like/apiv1and/myv1-- two dedicated tests confirm this.- Debug logging now shows both
original_base_urlandnormalized_base_url-- no more debugging traps. - The refactoring to extract
build_openai_compat_client()enables clean caller-level testing. Three factory tests verify that normalization actually flows through to the constructed rig-core client. - 9 helper-level normalization tests + 3 caller-level factory tests provide thorough coverage.
One minor observation: normalize_openai_base_url with "http://localhost:8080///" (multiple trailing slashes) normalizes to "http://localhost:8080/v1" via trim_end_matches('/'). This is correct behavior but slightly surprising -- trim_end_matches strips all trailing slashes, so the intermediate empty path segments are removed. The test covers it, so this is fine.
LGTM -- ready to merge.
|
Closing — already fixed on Shipped via:
The staging version of `normalize_openai_base_url` only appends `/v1` when the URL has no path (bare `scheme://host[:port]`), and is case-insensitive on the `/v1` match. This PR appends `/v1` whenever the last path segment isn't `v1`, which would wrongly mutate custom-path providers:
The `build_openai_compat_client` extraction + caller-level tests are a nice pattern, but the staging helper already has unit coverage and the policy there is the one we want. |
Summary
Fixes #1934. Local model servers (MLX, vLLM, llama.cpp) returned 404 when
LLM_BASE_URLlacked a/v1suffix because rig-core's openai client appends/chat/completionsdirectly to the base URL.normalize_openai_base_url()insrc/llm/mod.rsthat auto-appends/v1when not already presentcreate_openai_compat_from_registry()/v1, path prefixes, etc.Test plan
cargo fmt,cargo clippy --all-featureszero warningsLLM_BACKEND=openai_compatible LLM_BASE_URL=http://localhost:8080 ironclawsucceeds against a local server🤖 Generated with Claude Code