feat: support more AI providers for BYOK (JEF-52) - #155
Conversation
Extends JEF-6's bring-your-own-key support beyond OpenRouter/Google AI to OpenAI, Anthropic, Mistral, Groq, xAI, DeepSeek, and a custom OpenAI-compatible endpoint (own base URL + key + model) for anything else. - OpenAICompatibleLLMProvider generalizes the old OpenRouterLLMProvider (same request/response shape covers OpenAI, Mistral, Groq, xAI, DeepSeek, OpenRouter, and custom); AnthropicLLMProvider is new and bespoke (Messages API, x-api-key/anthropic-version headers, separate system field). - PROVIDER_REGISTRY replaces the old if/else in UserLLMProviderFactory with a lookup table — adding a provider is a registry entry now. - New llmModel/llmBaseUrl columns: model is an optional override for named providers (falls back to a per-provider default), and required for the custom provider along with a validated http(s) base URL. - Fixed a bug found during manual verification: ClearLlmApiKeyUseCase only nulled llmProvider/llmApiKey, leaving a stale model/baseUrl behind after clearing. - Frontend: Account settings' provider dropdown now lists all supported providers, with Base URL/Model fields that appear only for the custom provider. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N2PBmsuzPhrmNnfZf6C3BM
WalkthroughAdds multi-provider LLM BYOK support with persisted model and base URL fields, custom-provider validation, Anthropic and OpenAI-compatible providers, registry-based resolution, expanded GraphQL status/configuration, and account UI updates. ChangesMulti-provider LLM configuration
Estimated code review effort: 4 (Complex) | ~45 minutes Possibly related issues
Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Preview deployments for this PR: |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
apps/web/src/routes/_authenticated/account.tsx (1)
653-674: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winStale
baseUrlvalue can silently block form submission after switching away fromcustom.React Hook Form keeps field values registered by default (no
shouldUnregister), so if a user fills inbaseUrlwhilecustomis selected, then switches the provider to a named one, the oldbaseUrlvalue survives even though its input is now unmounted. On submit,llmApiKeySchema'ssuperRefineraises abaseUrlissue ("Only valid for a custom provider"), but the non-custom branch (Lines 1344-1358) never renderserrors.baseUrl, so the form just fails to submit with no visible error — a confusing dead end for the user.Clear
baseUrl(and optionally reset validation state) whenever the provider changes away fromcustom.🐛 Proposed fix
const llmApiKeyProvider = llmApiKeyForm.watch('provider'); const isCustomLlmProvider = llmApiKeyProvider === CUSTOM_LLM_PROVIDER; + useEffect(() => { + if (!isCustomLlmProvider) { + llmApiKeyForm.setValue('baseUrl', '', { shouldValidate: false }); + } + }, [isCustomLlmProvider]); const onSaveLlmApiKey = async (data: LlmApiKeyForm) => {Also applies to: 1313-1359
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/web/src/routes/_authenticated/account.tsx` around lines 653 - 674, Update the provider-change handling associated with llmApiKeyForm and llmApiKeyProvider so switching away from CUSTOM_LLM_PROVIDER clears the registered baseUrl value and its validation state. Preserve baseUrl when the provider remains custom, and ensure subsequent submission no longer includes the stale field or produces the hidden schema error.
🧹 Nitpick comments (2)
apps/api/src/infrastructure/llm/providerRegistry.ts (1)
20-20: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winTighten the registry key type for exhaustiveness.
Record<string, ...>means a newLLM_PROVIDERvalue can be added without the compiler flagging a missing registry entry; keying on the constant's value union restores that check.♻️ Proposed change
-export const PROVIDER_REGISTRY: Record<string, LLMProviderRegistryEntry> = { +type LlmProviderId = (typeof LLM_PROVIDER)[keyof typeof LLM_PROVIDER]; + +export const PROVIDER_REGISTRY: Record<LlmProviderId, LLMProviderRegistryEntry> = {🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/api/src/infrastructure/llm/providerRegistry.ts` at line 20, Update PROVIDER_REGISTRY to use a key type derived from the LLM_PROVIDER constant’s value union instead of string, so TypeScript enforces an entry for every provider and rejects unknown keys. Preserve the existing LLMProviderRegistryEntry value type.apps/api/src/__tests__/infrastructure/llm/UserLLMProviderFactory.test.ts (1)
113-130: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win
toBeInstanceOfcannot distinguish the OpenAI-compatible providers.OpenAI, OpenRouter and custom all resolve to
OpenAICompatibleLLMProvider, so a mis-wired base URL or model in the registry would still pass. Consider asserting the resolved endpoint/model (e.g. stubfetch, callcomplete, and check the URL andmodelin the body), and add a case forCUSTOMwithoutllmBaseUrl/llmModelto cover the throw path inproviderRegistry.ts.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/api/src/__tests__/infrastructure/llm/UserLLMProviderFactory.test.ts` around lines 113 - 130, Strengthen the UserLLMProviderFactory custom-provider tests beyond toBeInstanceOf(OpenAICompatibleLLMProvider): stub fetch, invoke complete, and assert the stored llmBaseUrl and llmModel are used in the request URL and body. Add a CUSTOM case with missing llmBaseUrl or llmModel that asserts the factory throws through the providerRegistry.ts validation path.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/api/src/constants.ts`:
- Around line 231-234: Update LLM.GROQ_DEFAULT_MODEL from
llama-3.3-70b-versatile to Groq’s currently recommended replacement, preferably
openai/gpt-oss-120b, so PROVIDER_REGISTRY.GROQ uses a supported fallback when
llmModel is null.
In `@apps/api/src/infrastructure/llm/AnthropicLLMProvider.ts`:
- Around line 43-47: Update the response handling in the AnthropicLLMProvider
method to collect every content block whose type is “text” and concatenate their
text values in order, rather than returning only the first match. Preserve the
empty-string fallback when no text blocks are present.
- Around line 23-36: Update the outbound fetch in AnthropicLLMProvider to
include an AbortSignal timeout so stalled Anthropic requests terminate instead
of waiting indefinitely; use the provider’s existing configuration conventions
for the timeout when available, otherwise define a clear default. Preserve the
current request payload and headers.
In `@apps/api/src/infrastructure/llm/OpenAICompatibleLLMProvider.ts`:
- Line 31: Update the error handling in OpenAICompatibleLLMProvider to avoid
embedding the upstream body in the thrown Error message. Log a truncated version
of body separately using the provider’s existing logger, while keeping the
thrown message limited to safe context such as the response status.
In `@apps/api/src/use-cases/user/SaveLlmApiKeyUseCase.ts`:
- Around line 16-23: Harden isValidUrl before custom baseUrl is persisted:
continue requiring http/https, reject loopback, link-local, private/reserved IPs
and blocked hostnames, and resolve hostnames to validate every resulting address
against those ranges to prevent DNS rebinding. Apply the same validation to the
baseUrl handling around the referenced persistence path, and reject invalid
targets before saving them.
---
Outside diff comments:
In `@apps/web/src/routes/_authenticated/account.tsx`:
- Around line 653-674: Update the provider-change handling associated with
llmApiKeyForm and llmApiKeyProvider so switching away from CUSTOM_LLM_PROVIDER
clears the registered baseUrl value and its validation state. Preserve baseUrl
when the provider remains custom, and ensure subsequent submission no longer
includes the stale field or produces the hidden schema error.
---
Nitpick comments:
In `@apps/api/src/__tests__/infrastructure/llm/UserLLMProviderFactory.test.ts`:
- Around line 113-130: Strengthen the UserLLMProviderFactory custom-provider
tests beyond toBeInstanceOf(OpenAICompatibleLLMProvider): stub fetch, invoke
complete, and assert the stored llmBaseUrl and llmModel are used in the request
URL and body. Add a CUSTOM case with missing llmBaseUrl or llmModel that asserts
the factory throws through the providerRegistry.ts validation path.
In `@apps/api/src/infrastructure/llm/providerRegistry.ts`:
- Line 20: Update PROVIDER_REGISTRY to use a key type derived from the
LLM_PROVIDER constant’s value union instead of string, so TypeScript enforces an
entry for every provider and rejects unknown keys. Preserve the existing
LLMProviderRegistryEntry value type.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 43c6c771-f24c-460f-b3ed-dafe7c05c874
📒 Files selected for processing (32)
apps/api/drizzle/0002_silly_umar.sqlapps/api/drizzle/meta/0002_snapshot.jsonapps/api/drizzle/meta/_journal.jsonapps/api/src/__tests__/application/user/ClearLlmApiKeyUseCase.test.tsapps/api/src/__tests__/application/user/GetLlmKeyStatusUseCase.test.tsapps/api/src/__tests__/application/user/SaveLlmApiKeyUseCase.test.tsapps/api/src/__tests__/helpers/createTestDb.tsapps/api/src/__tests__/helpers/mocks.tsapps/api/src/__tests__/infrastructure/llm/AnthropicLLMProvider.test.tsapps/api/src/__tests__/infrastructure/llm/OpenAICompatibleLLMProvider.test.tsapps/api/src/__tests__/infrastructure/llm/OpenRouterLLMProvider.test.tsapps/api/src/__tests__/infrastructure/llm/UserLLMProviderFactory.test.tsapps/api/src/__tests__/infrastructure/llm/providerRegistry.test.tsapps/api/src/__tests__/interface-adapters/resolvers/UserResolver.test.tsapps/api/src/constants.tsapps/api/src/domain/user/User.tsapps/api/src/http/schema/mutations/userMutations.tsapps/api/src/http/schema/types/LlmKeyStatusType.tsapps/api/src/infrastructure/db/repositories/DrizzleUserRepository.tsapps/api/src/infrastructure/db/schema.tsapps/api/src/infrastructure/llm/AnthropicLLMProvider.tsapps/api/src/infrastructure/llm/OpenAICompatibleLLMProvider.tsapps/api/src/infrastructure/llm/UserLLMProviderFactory.tsapps/api/src/infrastructure/llm/providerRegistry.tsapps/api/src/interface-adapters/resolvers/UserResolver.tsapps/api/src/use-cases/ports/IUserRepository.tsapps/api/src/use-cases/user/ClearLlmApiKeyUseCase.tsapps/api/src/use-cases/user/GetLlmKeyStatusUseCase.tsapps/api/src/use-cases/user/IGetLlmKeyStatusUseCase.tsapps/api/src/use-cases/user/ISaveLlmApiKeyUseCase.tsapps/api/src/use-cases/user/SaveLlmApiKeyUseCase.tsapps/web/src/routes/_authenticated/account.tsx
💤 Files with no reviewable changes (1)
- apps/api/src/tests/infrastructure/llm/OpenRouterLLMProvider.test.ts
| GROQ_API_URL: 'https://api.groq.com/openai/v1/chat/completions', | ||
| GROQ_DEFAULT_MODEL: 'llama-3.3-70b-versatile', | ||
| XAI_API_URL: 'https://api.x.ai/v1/chat/completions', | ||
| XAI_DEFAULT_MODEL: 'grok-2-latest', |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== Locate constants and LLM default usages =="
git ls-files 'apps/api/src/**/*.ts' | rg '(^|/)constants\.ts$|LLM|llm|api-key' || true
echo
echo "== constants.ts relevant section =="
if [ -f apps/api/src/constants.ts ]; then
sed -n '180,250p' apps/api/src/constants.ts | nl -ba -v180
fi
echo
echo "== Search default model constants and factory/use-case references =="
rg -n "LLM_|LLM_PROVIDER|GROQ_DEFAULT_MODEL|XAI_DEFAULT_MODEL|gpt|defaultModel|llmModel|model:" apps/api/src -g '*.ts' | head -n 200
echo
echo "== Inspect provider factory/use-case candidates =="
fd -i '.*llm.*|.*api.*|.*key.*' apps/api/src -t f | sed -n '1,120p'Repository: mankatcheung/job-finder
Length of output: 1142
🌐 Web query:
Groq API deprecations llama-3.3-70b-versatile shutdown date
💡 Result:
The Groq API model llama-3.3-70b-versatile has a scheduled shutdown date of August 16, 2026 [1][2]. According to official Groq documentation, the deprecation of this model, along with llama-3.1-8b-instant, was announced to users via email on June 17, 2026 [1]. Groq recommends that users migrate to alternative models, specifically openai/gpt-oss-120b or qwen/qwen3.6-27b, to replace llama-3.3-70b-versatile [1]. This deprecation applies to free and developer-tier usage; enterprise customers with a committed-spend contract are not affected [1].
Citations:
🌐 Web query:
xAI API current model catalog grok-2-latest Grok 4 Grok 4.5
💡 Result:
As of July 30, 2026, the current flagship model in the xAI API catalog is Grok 4.5, which was released on July 16, 2026 [1][2]. The xAI API catalog features several models, with Grok 4.5 serving as the primary model for coding, agentic tasks, and knowledge work [3][1][2]. Other models currently available or referenced in the official documentation include Grok 4.3 and various versions of the Grok 4.20 reasoning and multi-agent models [4][5]. Regarding your specific queries: - Grok-2-latest: This identifier is not the current flagship alias. The xAI documentation utilizes aliases such as -latest to provide access to the most recent stable versions of models [4][5]. - Grok 4: This model was released on July 9, 2025 [6][7]. It established foundational capabilities in native tool use and real-time search [6]. - Grok 4.5: This is the current latest flagship model released by xAI [1][2]. It supports a 500,000 token context window, includes reasoning effort parameters (low, medium, high), and is integrated into platforms including the xAI API, Grok web/apps, and GitHub Copilot [1][2][8][9]. For the most up-to-date and specific model identifiers and pricing, please refer to the official xAI models documentation [4][5].
Citations:
- 1: https://x.ai/news/grok-4-5
- 2: https://docs.x.ai/developers/grok-4-5
- 3: https://x.ai/api
- 4: https://docs.x.ai/developers/models
- 5: https://docs.x.ai/developers/models?ps_partner_key=aXZvanVuaW9yOTk2&ps_xid=XrRiHkBqNYlsG3
- 6: https://x.ai/news/grok-4
- 7: https://web.archive.org/web/20260509094258/https:/x.ai/news/grok-4
- 8: https://x.ai/news/grok-4-5-everywhere
- 9: https://github.blog/changelog/2026-07-28-grok-4-5-is-now-available-in-github-copilot/
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== constants.ts relevant section =="
sed -n '180,260p' apps/api/src/constants.ts
echo
echo "== LLM provider and factory implementations =="
sed -n '1,240p' apps/api/src/use-cases/ports/ILLMProvider.ts
sed -n '1,240p' apps/api/src/use-cases/ports/ILLMProviderFactory.ts
sed -n '1,260p' apps/api/src/infrastructure/llm/UserLLMProviderFactory.ts
sed -n '1,260p' apps/api/src/infrastructure/llm/providerRegistry.ts
echo
echo "== AI service/usage call sites =="
rg -n "GROQ_DEFAULT_MODEL|XAI_DEFAULT_MODEL|LLM_DEFAULT|GROQ_API_URL|XAI_API_URL|model: user\.llmModel|create.*Provider|send|generate|chat" apps/api/src -g '*.ts' | head -n 250
echo
echo "== LLM entity/user schema/model fields =="
rg -n "llmProvider|llmModel|Model|api.*Key|Groq|xAI" apps/api/src -g '*.ts' | head -n 250Repository: mankatcheung/job-finder
Length of output: 50379
Update the Groq default before merge.
PROVIDER_REGISTRY.GROQ falls back to LLM.GROQ_DEFAULT_MODEL when a user’s llmModel is null, and llama-3.3-70b-versatile is scheduled to be shut down by Groq. Use Groq’s currently recommended alternative, such as openai/gpt-oss-120b, or add an explicit provider default with periodic review coverage.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@apps/api/src/constants.ts` around lines 231 - 234, Update
LLM.GROQ_DEFAULT_MODEL from llama-3.3-70b-versatile to Groq’s currently
recommended replacement, preferably openai/gpt-oss-120b, so
PROVIDER_REGISTRY.GROQ uses a supported fallback when llmModel is null.
| const response = await fetch(LLM.ANTHROPIC_API_URL, { | ||
| method: 'POST', | ||
| headers: { | ||
| 'Content-Type': 'application/json', | ||
| 'x-api-key': this.apiKey, | ||
| 'anthropic-version': LLM.ANTHROPIC_VERSION, | ||
| }, | ||
| body: JSON.stringify({ | ||
| model: this.model, | ||
| max_tokens: maxTokens, | ||
| ...(system ? { system } : {}), | ||
| messages: conversation, | ||
| }), | ||
| }); |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
No timeout on the outbound Anthropic call.
fetch here has no AbortSignal, so a stalled upstream keeps the request thread and connection held indefinitely. Pass a timeout signal (and consider making it configurable).
🛡️ Proposed fix
const response = await fetch(LLM.ANTHROPIC_API_URL, {
method: 'POST',
+ signal: AbortSignal.timeout(LLM.REQUEST_TIMEOUT_MS),
headers: {📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| const response = await fetch(LLM.ANTHROPIC_API_URL, { | |
| method: 'POST', | |
| headers: { | |
| 'Content-Type': 'application/json', | |
| 'x-api-key': this.apiKey, | |
| 'anthropic-version': LLM.ANTHROPIC_VERSION, | |
| }, | |
| body: JSON.stringify({ | |
| model: this.model, | |
| max_tokens: maxTokens, | |
| ...(system ? { system } : {}), | |
| messages: conversation, | |
| }), | |
| }); | |
| const response = await fetch(LLM.ANTHROPIC_API_URL, { | |
| method: 'POST', | |
| signal: AbortSignal.timeout(LLM.REQUEST_TIMEOUT_MS), | |
| headers: { | |
| 'Content-Type': 'application/json', | |
| 'x-api-key': this.apiKey, | |
| 'anthropic-version': LLM.ANTHROPIC_VERSION, | |
| }, | |
| body: JSON.stringify({ | |
| model: this.model, | |
| max_tokens: maxTokens, | |
| ...(system ? { system } : {}), | |
| messages: conversation, | |
| }), | |
| }); |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@apps/api/src/infrastructure/llm/AnthropicLLMProvider.ts` around lines 23 -
36, Update the outbound fetch in AnthropicLLMProvider to include an AbortSignal
timeout so stalled Anthropic requests terminate instead of waiting indefinitely;
use the provider’s existing configuration conventions for the timeout when
available, otherwise define a clear default. Preserve the current request
payload and headers.
| const json = (await response.json()) as { | ||
| content?: Array<{ type: string; text?: string }>; | ||
| }; | ||
|
|
||
| return json.content?.find((block) => block.type === 'text')?.text ?? ''; |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Only the first text block is returned.
Anthropic may return several text blocks in content; taking just the first silently truncates the completion.
♻️ Proposed fix
- return json.content?.find((block) => block.type === 'text')?.text ?? '';
+ return (
+ json.content
+ ?.filter((block) => block.type === 'text')
+ .map((block) => block.text ?? '')
+ .join('') ?? ''
+ );📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| const json = (await response.json()) as { | |
| content?: Array<{ type: string; text?: string }>; | |
| }; | |
| return json.content?.find((block) => block.type === 'text')?.text ?? ''; | |
| const json = (await response.json()) as { | |
| content?: Array<{ type: string; text?: string }>; | |
| }; | |
| return ( | |
| json.content | |
| ?.filter((block) => block.type === 'text') | |
| .map((block) => block.text ?? '') | |
| .join('') ?? '' | |
| ); |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@apps/api/src/infrastructure/llm/AnthropicLLMProvider.ts` around lines 43 -
47, Update the response handling in the AnthropicLLMProvider method to collect
every content block whose type is “text” and concatenate their text values in
order, rather than returning only the first match. Preserve the empty-string
fallback when no text blocks are present.
| if (!response.ok) { | ||
| const body = await response.text(); | ||
| throw new Error(`OpenRouter error ${response.status}: ${body}`); | ||
| throw new Error(`LLM provider error ${response.status}: ${body}`); |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win
Upstream response body is embedded verbatim in the thrown error.
Provider error bodies can contain request echoes or account details; if this message surfaces in GraphQL errors or logs it becomes a leak vector. Truncate and log the body separately rather than putting it in the message.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@apps/api/src/infrastructure/llm/OpenAICompatibleLLMProvider.ts` at line 31,
Update the error handling in OpenAICompatibleLLMProvider to avoid embedding the
upstream body in the thrown Error message. Log a truncated version of body
separately using the provider’s existing logger, while keeping the thrown
message limited to safe context such as the response status.
| function isValidUrl(value: string): boolean { | ||
| try { | ||
| const url = new URL(value); | ||
| return url.protocol === 'http:' || url.protocol === 'https:'; | ||
| } catch { | ||
| return false; | ||
| } | ||
| } |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Locate and inspect the target file plus nearby provider references without executing repo code.
printf 'Files matching SaveLlmApiKeyUseCase:\n'
fd -a 'SaveLlmApiKeyUseCase\.ts$' . || true
printf '\nOutline:\n'
target="$(fd 'SaveLlmApiKeyUseCase\.ts$' . | head -n 1 || true)"
if [ -n "${target:-}" ]; then
wc -l "$target"
ast-grep outline "$target" || true
printf '\nTarget file with line numbers:\n'
cat -n "$target"
fi
printf '\nSearch for UserLLMProviderFactory / OpenAICompatibleLLMProvider / useLlmBaseUrl references:\n'
rg -n "UserLLMProviderFactory|OpenAICompatibleLLMProvider|llmBaseUrl|baseUrl|isValidUrl|SaveLlmApiKeyUseCase" -S . --glob '!node_modules' --glob '!dist' --glob '!build' | head -n 200Repository: mankatcheung/job-finder
Length of output: 18904
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf 'SaveLlmApiKeyUseCase.ts lines 1-85:\n'
cat -n apps/api/src/use-cases/user/SaveLlmApiKeyUseCase.ts | sed -n '1,90p'
printf '\nOpenAICompatibleLLMProvider.ts:\n'
cat -n apps/api/src/infrastructure/llm/OpenAICompatibleLLMProvider.ts
printf '\nUserLLMProviderFactory.ts:\n'
cat -n apps/api/src/infrastructure/llm/UserLLMProviderFactory.ts
printf '\nproviderRegistry.ts lines 60-85:\n'
cat -n apps/api/src/infrastructure/llm/providerRegistry.ts | sed -n '60,85p'
printf '\nRead-only URL validation probe for current isValidUrl behaviour:\n'
node - <<'JS'
function isValidUrl(value) {
try {
const url = new URL(value);
return url.protocol === 'http:' || url.protocol === 'https:';
} catch {
return false;
}
}
const inputs = [
'https://my-llm.example.com/v1/chat/completions',
'http://127.0.0.1:8080/v1/chat/completions',
'http://localhost:8080/v1/chat/completions',
'http://169.254.169.254/latest/meta-data/token',
'http://10.internal.example/v1/chat/completions',
'http://192.168.1.10/v1/chat/completions',
'http://[::1]/v1/chat/completions',
'ftp://example.com/foo',
'not-a-url',
];
for (const input of inputs) {
console.log(`${input} -> ${isValidUrl(input)}`);
}
JS
printf '\nCheck for outbound fetch configuration / egress proxy / global fetch config:\n'
rg -n "globalThis\.fetch|AbortSignal\.timeout|NO_PROXY|proxy|hostFilter|request\.agent|fetch\(" apps/api/src --glob '*.ts' | head -n 200 || trueRepository: mankatcheung/job-finder
Length of output: 9438
Block internal and DNS-rebinding targets before persisting custom baseUrl.
isValidUrl() only accepts http:/https: URLs, but custom provider baseUrl is persisted and later passed to OpenAICompatibleLLMProvider.complete(), which calls fetch(this.baseUrl). This allows authenticated users to set values such as http://localhost, http://127.0.0.1, http://169.254.169.254/..., or private RFC1918 targets, causing the API server to issue outbound requests on the user’s behalf. Reject loopback, link-local, private IP ranges, blocked hostnames, and resolve hostnames before persisting so DNS rebinding is handled.
Also applies to: 46-50
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@apps/api/src/use-cases/user/SaveLlmApiKeyUseCase.ts` around lines 16 - 23,
Harden isValidUrl before custom baseUrl is persisted: continue requiring
http/https, reject loopback, link-local, private/reserved IPs and blocked
hostnames, and resolve hostnames to validate every resulting address against
those ranges to prevent DNS rebinding. Apply the same validation to the baseUrl
handling around the referenced persistence path, and reject invalid targets
before saving them.
1 participant
Summary
Extends JEF-6's bring-your-own-key support beyond OpenRouter/Google AI to the rest of the mainstream field, plus a custom-endpoint escape hatch (Linear JEF-52):
Architecture:
OpenAICompatibleLLMProvidergeneralizes the oldOpenRouterLLMProvider({apiKey, baseUrl, model}) — covers every provider that implements OpenAI's/chat/completionsshape: OpenAI, Mistral, Groq, xAI, DeepSeek, OpenRouter, andcustom.AnthropicLLMProvideris new and bespoke (Messages API:/v1/messages,x-api-key/anthropic-versionheaders, system prompt as a separate top-level field,content[0].textresponse shape).PROVIDER_REGISTRY(keyed by provider id) replaces the old if/else inUserLLMProviderFactory— adding a provider is now a registry entry, not new branching logic.Data model: two new nullable
Usercolumns —llmModel(optional override for named providers, falls back to a per-provider default; required forcustom) andllmBaseUrl(only used, and required, forcustom).Validation (
SaveLlmApiKeyUseCase):customrequires both a well-formed http(s) base URL and a model; named providers reject abaseUrlif one is given (prevents stale/confusing state).Bug found and fixed during manual verification:
ClearLlmApiKeyUseCaseonly nulledllmProvider/llmApiKey, leaving a stalemodel/baseUrlbehind after clearing a custom-provider key. Now clears all four fields.Frontend: Account settings' provider
<select>lists all 9 providers; selectingcustomreveals required Base URL/Model fields, and named providers get an optional "override the default" Model field.Test plan
pnpm --filter @job-finder/api typecheck && pnpm --filter @job-finder/api test(793 tests passing, including new coverage forOpenAICompatibleLLMProvider,AnthropicLLMProvider,PROVIDER_REGISTRY, and updated factory/use-case tests)pnpm --filter @job-finder/web typecheckpnpm --filter @job-finder/api build && pnpm --filter @job-finder/web buildpnpm lint(both packages)llmKeyStatusreflects provider/model/baseUrl correctly) plus the custom provider's validation paths (missing base URL, malformed base URL, missing model, base URL rejected on a named provider) and the clear flow (this is where the stale-field bug was caught and fixed).🤖 Generated with Claude Code
https://claude.ai/code/session_01N2PBmsuzPhrmNnfZf6C3BM
Summary by CodeRabbit