feat(inference): add interactive managed-vLLM model picker

[laitingsheng]

Summary

Express managed-vLLM install now offers an interactive picker for the curated VLLM_MODELS registry instead of silently selecting the per-platform default. NEMOCLAW_VLLM_MODEL and non-interactive runs keep their existing behaviour.

Related Issue

Resolves #4705

Changes

• Tag each registry entry with platforms: VllmPlatform[] (spark, station, linux) and add modelsForPlatform() so the picker only offers models that fit the host.
• Add promptVllmModel (new src/lib/inference/vllm-prompt.ts, also re-exported from model-prompts) that lists the per-profile subset, marks the default as recommended, prints HF ids inline, and re-runs assertGatedModelAccess on the selection.
• VllmProfile gains a platform key and the three profiles are tagged accordingly.
• installVllm() routes through env override → non-interactive default → interactive picker, keeping the gated-token preflight before any docker side effect.
• Track model origin so the summary line only labels env-driven choices as (NEMOCLAW_VLLM_MODEL override).
• Tests cover the platform filter, Enter-as-default, numeric picks, invalid indices, gated-without-token retry, gated-with-HF-token acceptance, and back navigation.

Type of Change

• Code change (feature, bug fix, or refactor)
• Code change with doc updates
• Doc only (prose changes, no code sample modifications)
• Doc only (includes code sample changes)

Verification

• npx prek run --all-files passes
• npm test passes
• Tests added or updated for new or changed behavior
• No secrets, API keys, or credentials committed
• Docs updated for user-facing behavior changes
• npm run docs builds without warnings (doc changes only)
• Doc pages follow the style guide (doc changes only)
• New doc pages include SPDX header and frontmatter (new pages only)

---

Signed-off-by: Tinson Lai tinsonl@nvidia.com

Summary by CodeRabbit

• New Features

  • Interactive vLLM model picker for installers with installer-aware resolution.

• Improvements

  • Platform-scoped model lists so available models vary by deployment platform.
  • Environment override annotated in install summaries.
  • Default model shown first and marked as recommended; registry order preserved.
  • Robust numeric input validation, gated-model HF-token prompts, and "back" navigation.

• Tests

  • New tests covering picker behavior, platform filtering, gating, resolution and installer flows.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 4e21a510-7de6-47eb-b81f-784c8a4a671f

📥 Commits

Reviewing files that changed from the base of the PR and between e4e1527 and 4293c76.

📒 Files selected for processing (2)

• src/lib/inference/model-prompts.test.ts
• src/lib/inference/vllm-prompt.ts

🚧 Files skipped from review as they are similar to previous changes (2)

• src/lib/inference/model-prompts.test.ts
• src/lib/inference/vllm-prompt.ts

---

📝 Walkthrough

Walkthrough

Adds platform-aware vLLM registry filtering, an interactive picker with gated-model validation and default-first ordering (re-exported), and refactors installVllm to resolve models via the new resolver with profile platform discriminators and tests covering resolution flows.

Changes

vLLM Platform-Aware Interactive Model Picker

Layer / File(s)	Summary
Model registry platform constraints and filtering src/lib/inference/vllm-models.ts, src/lib/inference/vllm-models.test.ts	Adds VllmPlatform union and platforms: readonly VllmPlatform[] to VllmModelDef; updates registry entries with platform allowlists; exports modelsForPlatform(platform) which filters VLLM_MODELS by platform while preserving registry order; tests verify subsets and ordering.
vLLM interactive model selection prompt src/lib/inference/vllm-prompt.ts, src/lib/inference/model-prompts.ts, src/lib/inference/model-prompts.test.ts	Adds promptVllmModel() and VllmModelPromptOptions with injectable prompt/io handlers, default-first ordering, navigation (back/exit), numeric parsing/validation, and gated-model access checks using provided env; re-exports prompt via model-prompts.ts; tests cover Enter/default, ordering, invalid input, gated-model reprompt/acceptance, and back.
Profile-aware onboarding and installVllm refactoring src/lib/inference/vllm.ts, src/lib/inference/vllm.test.ts	Profiles gain platform discriminator; installVllm() refactored to call resolveVllmInstallModel(profile, { nonInteractive, promptFn }), returns early on unresolved selection, tracks modelSource (env/default/picker), validates gated access per resolution path, and annotates env overrides in the summary; tests cover non-interactive default, env override annotation, interactive prompt invocation, and gated-model failure without HF token.

Sequence Diagram(s)

sequenceDiagram
  participant installVllm
  participant Env
  participant modelsForPlatform
  participant promptVllmModel
  installVllm->>Env: check NEMOCLAW_VLLM_MODEL
  alt env override present
    installVllm->>Env: selectVllmModelFromEnv + assertGatedModelAccess
    installVllm-->>installVllm: modelSource = env
  else no env override and non-interactive
    installVllm-->>installVllm: model = profile.defaultModel
    installVllm-->>installVllm: modelSource = default
  else interactive
    installVllm->>modelsForPlatform: get platform-curated list
    installVllm->>promptVllmModel: show picker
    promptVllmModel-->>installVllm: selected model or BACK_TO_SELECTION
    installVllm-->>installVllm: modelSource = picker or default
  end

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Poem

I’m a rabbit in the model tree, 🐇
Platforms sorted neat and free,
Press Enter, the default takes flight,
Gated checks keep pulls polite,
Pick a model—install goes right.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 37.50% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and specifically describes the main change: adding an interactive model picker for managed vLLM, which is the primary objective of this pull request.
Linked Issues check	✅ Passed	The PR successfully implements all core coding objectives from #4705: interactive platform-aware model picker, NEMOCLAW_VLLM_MODEL override preservation, gated-model preflight validation, platform-specific model filtering, profile defaults maintained, and comprehensive test coverage.
Out of Scope Changes check	✅ Passed	All changes are directly aligned with #4705 objectives; no unrelated code modifications or scope creep detected across the six modified files and one new file.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/4705-managed-vllm-model-picker

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[github-actions]

E2E Advisor Recommendation

Required E2E: onboard-inference-smoke-e2e, inference-routing-e2e, test-spark-install.sh
Optional E2E: gpu-e2e, openclaw-inference-switch-e2e

Auto-dispatched E2E: inference-routing-e2e via nightly-e2e.yaml at 4293c7680d8c48c74218cd6a544e6ac479299999 — nightly run

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/main
Head: HEAD
Confidence: medium

Required E2E

• onboard-inference-smoke-e2e (low): Required because this PR changes onboarding-time inference install/model selection code. This regression job verifies onboard does not report success until the configured inference route has served a real chat completion and surfaces actionable diagnostics on runtime-broken inference.
• inference-routing-e2e (medium): Required because the changed code is in the inference stack and can affect provider/model routing behavior visible through OpenShell gateway inference. This job covers inference routing, credential isolation, and error classification.
• test-spark-install.sh (high): Required as the closest existing DGX Spark install smoke for the changed vLLM profile/default model path. The PR changes Spark-specific vLLM model defaults, platform filtering, and non-interactive model resolution that are only meaningfully exercised on a Spark-class Linux install path.

Optional E2E

• gpu-e2e (high): Useful adjacent coverage for local GPU inference onboarding and sandbox inference, but it uses the Ollama provider rather than vLLM, so it is not sufficient as merge-blocking coverage for the changed vLLM picker/registry behavior.
• openclaw-inference-switch-e2e (medium): Optional confidence for real assistant traffic after inference configuration changes. It validates switching an OpenClaw sandbox inference route and live requests, but does not directly exercise vLLM install model selection.

New E2E recommendations

• vllm-installer (high): No existing E2E appears to directly exercise the new vLLM interactive model picker, platform-filtered model menu, env override precedence, gated Hugging Face token failure, and successful vLLM container startup/readiness.
  • Suggested test: Add a vLLM install model-selection E2E with mocked/lightweight vLLM readiness where possible, covering non-interactive default, NEMOCLAW_VLLM_MODEL override, gated model without HF_TOKEN, and interactive picker back/selection behavior.
• dgx-station-vllm (medium): The PR adds Station-specific platform filtering and permits Station-only DeepSeek V4 Flash in the picker, but existing visible E2E coverage does not validate Station model availability or GB300 GPU selection in the vLLM path.
  • Suggested test: Add a DGX Station vLLM install smoke that asserts the Station picker/default model set and verifies the generated docker run flags on a Station-class runner.

[github-actions]

E2E Scenario Advisor Recommendation

Required scenario E2E: None
Optional scenario E2E: None

Workflow run

Full scenario advisor summary

E2E Scenario Advisor

Base: origin/main
Head: HEAD
Confidence: medium

Required scenario E2E

• None. Changed files are outside test/e2e-scenario and the scenario workflows/catalogs. The production changes target the vLLM model picker/install path, but the dispatchable scenario ROUTES table has no vLLM/DGX Spark/Station scenario; the available GPU scenario covers local Ollama, not vLLM. No current scenario E2E job directly exercises this changed surface.

Optional scenario E2E

• None.

Relevant changed files

• None.

[github-actions]

PR Review Advisor

Findings: 1 needs attention, 5 worth checking, 0 nice ideas
Since last review: 0 prior items resolved, 3 still apply, 1 new item found

Review findings

🛠️ Needs attention

• Generated skills still publish stale managed-vLLM defaults and slugs (skills/nemoclaw-user-configure-inference/SKILL.md:221): Issue Add interactive managed-vLLM model picker and curated Spark options #4705 explicitly requires docs and generated user skills to agree on managed-vLLM defaults and recognized override slugs. The docs page now appears updated, but generated skill/reference files still disagree with the registry and changed behavior.
  • Recommendation: Refresh the generated skill references so they consistently list DGX Spark as qwen3.6-35b-a3b-nvfp4 / nvidia/Qwen3.6-35B-A3B-NVFP4, DGX Station as qwen3.6-27b / Qwen/Qwen3.6-27B-FP8, generic Linux as nemotron-3-nano-4b / nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8, and include all recognized override slugs including deepseek-v4-flash.
  • Evidence: Acceptance clause: “Docs and generated user skills agree on managed-vLLM defaults and recognized override slugs.” docs/inference/inference-options.mdx now has the current defaults and slugs, but skills/nemoclaw-user-configure-inference/SKILL.md still lists DGX Spark as Qwen/Qwen3.6-27B-FP8; skills/nemoclaw-user-configure-inference/references/use-local-inference-details.md still says qwen3.6-27b is default on both Spark and Station and omits qwen3.6-35b-a3b-nvfp4 and deepseek-v4-flash; skills/nemoclaw-user-reference/references/commands.md still omits deepseek-v4-flash from the NEMOCLAW_VLLM_MODEL recognized slugs.

🔎 Worth checking

• Source-of-truth review needed: resolveVllmInstallModel back navigation: The advisor marked localized patch analysis as needs_followup.
  • Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
  • Evidence: resolveVllmInstallModel() returns null for isBackToSelection(pick), and installVllm() maps a null result to `{ ok: false }`.
• Source-of-truth review needed: resolveVllmInstallModel catch-all resolution failure: The advisor marked localized patch analysis as needs_followup.
  • Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
  • Evidence: resolveVllmInstallModel() catches all errors, logs `vLLM install failed: ...`, and returns null.
• Model-resolution errors and picker back navigation collapse to the same null result (src/lib/inference/vllm-prompt.ts:158): The back-navigation path and all resolution exceptions both return null, and installVllm maps null to { ok: false }. Back navigation is a valid user action, but the catch-all error path also covers unexpected programmer/configuration errors with the same untyped result, which makes caller behavior and future regression tests less precise.
  • Recommendation: Consider returning a typed resolution result such as { kind: 'back' } versus { kind: 'error', message } or narrowing the catch to expected env/gated-model failures. At minimum, add install-level tests for picker back and unknown NEMOCLAW_VLLM_MODEL returning before Docker prerequisites/adapters.
  • Evidence: resolveVllmInstallModel() has `if (isBackToSelection(pick)) return null;` and a catch block that logs `vLLM install failed: ...` then `return null;`; installVllm() immediately maps a falsy resolution to `{ ok: false }` before Docker prerequisites.
• Runtime-sensitive managed-vLLM path still relies mostly on mocked unit boundaries (src/lib/inference/vllm.ts:509): The PR adds strong unit coverage for picker parsing, registry filtering, env overrides, default behavior, and gated-token checks. However, the changed behavior sits at a Docker-backed onboarding/install boundary, and current install tests intentionally stop before image pull, model download, or container launch.
  • Recommendation: Add or identify targeted runtime/integration validation for the changed managed-vLLM path, especially onboarding-to-picker invocation, picker back navigation with no Docker side effects, and propagation of a picker-selected non-default model into both `hf download` and the final `vllm serve` command. Keep this separate from external E2E job status reporting.
  • Evidence: vllm.test.ts covers installVllm() with mocked runner/Docker adapters and makes dockerPrereqsOk fail via mocked runCapture, so dockerPullWithProgressWatchdog, downloadModel(), and startContainer() are not exercised for selected-model propagation.
• Managed-vLLM installer trust boundary depends on curated model and image discipline (src/lib/inference/vllm-models.ts:313): The changed picker increases user-facing access to the managed-vLLM registry, whose serve command includes `--trust-remote-code` and runs inside tag-pinned NGC images rather than digest-pinned images. The PR still uses an allowlisted registry and does not introduce arbitrary model IDs, so this is not a confirmed vulnerability, but it remains a security-sensitive installer trust boundary.
  • Recommendation: Keep the interactive picker limited to validated registry entries, document/track the validation criteria for any new model entry, and consider digest pinning or equivalent provenance controls for managed installer images where feasible.
  • Evidence: buildVllmServeCommand() includes shared `--trust-remote-code`; VLLM_IMAGES uses tags such as `nvcr.io/nvidia/vllm:26.05.post1-py3`; issue Add interactive managed-vLLM model picker and curated Spark options #4705 also says not to add unvalidated model IDs and requires validated flags, context length, memory/disk requirements, and tool-call behavior.

🌱 Nice ideas

• None.

Consider writing more tests for

• **Runtime validation** — Runtime/integration validation: interactive Spark managed-vLLM onboarding shows the model picker before install confirmation.. Unit coverage is broad for picker and registry behavior, but the changed path is a Docker-backed managed-vLLM onboarding/install boundary. Current install tests use mocked runner/Docker adapters and intentionally stop before image pull, model download, and container launch.
• **Runtime validation** — Runtime/integration validation: picker back navigation returns ok:false before Docker prerequisites, image pull, model download, or container start.. Unit coverage is broad for picker and registry behavior, but the changed path is a Docker-backed managed-vLLM onboarding/install boundary. Current install tests use mocked runner/Docker adapters and intentionally stop before image pull, model download, and container launch.
• **Runtime validation** — Runtime/integration or deeper mocked validation: a picker-selected non-default model is used for `hf download` and the final `vllm serve` command without the NEMOCLAW_VLLM_MODEL override label.. Unit coverage is broad for picker and registry behavior, but the changed path is a Docker-backed managed-vLLM onboarding/install boundary. Current install tests use mocked runner/Docker adapters and intentionally stop before image pull, model download, and container launch.
• **Runtime validation** — Unit invariant: every detectVllmProfile(...).defaultModel is included in modelsForPlatform(profile.platform).. Unit coverage is broad for picker and registry behavior, but the changed path is a Docker-backed managed-vLLM onboarding/install boundary. Current install tests use mocked runner/Docker adapters and intentionally stop before image pull, model download, and container launch.
• **Runtime validation** — Install-level invariant: unknown NEMOCLAW_VLLM_MODEL returns ok:false before Docker prerequisites or Docker adapter calls.. Unit coverage is broad for picker and registry behavior, but the changed path is a Docker-backed managed-vLLM onboarding/install boundary. Current install tests use mocked runner/Docker adapters and intentionally stop before image pull, model download, and container launch.
• **Runtime-sensitive managed-vLLM path still relies mostly on mocked unit boundaries** — Add or identify targeted runtime/integration validation for the changed managed-vLLM path, especially onboarding-to-picker invocation, picker back navigation with no Docker side effects, and propagation of a picker-selected non-default model into both `hf download` and the final `vllm serve` command. Keep this separate from external E2E job status reporting.
• **Acceptance clause:** Docs and generated user skills agree on managed-vLLM defaults and recognized override slugs. — add test evidence or identify existing coverage. docs/inference/inference-options.mdx now reflects the current defaults and includes deepseek-v4-flash, but generated skill/reference files still contain stale Spark defaults or omit current slugs: skills/nemoclaw-user-configure-inference/SKILL.md, skills/nemoclaw-user-configure-inference/references/use-local-inference-details.md, and skills/nemoclaw-user-reference/references/commands.md.
• **resolveVllmInstallModel back navigation** — Prompt-level test covers returning BACK_TO_SELECTION when the user types back; an install-level test should assert `{ ok: false }` with no Docker prerequisites, image pull, model download, or container start.. resolveVllmInstallModel() returns null for isBackToSelection(pick), and installVllm() maps a null result to `{ ok: false }`.

Since last review details

Current findings:

• Source-of-truth review needed: resolveVllmInstallModel back navigation: The advisor marked localized patch analysis as needs_followup.
  • Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
  • Evidence: resolveVllmInstallModel() returns null for isBackToSelection(pick), and installVllm() maps a null result to `{ ok: false }`.
• Source-of-truth review needed: resolveVllmInstallModel catch-all resolution failure: The advisor marked localized patch analysis as needs_followup.
  • Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
  • Evidence: resolveVllmInstallModel() catches all errors, logs `vLLM install failed: ...`, and returns null.
• Generated skills still publish stale managed-vLLM defaults and slugs (skills/nemoclaw-user-configure-inference/SKILL.md:221): Issue Add interactive managed-vLLM model picker and curated Spark options #4705 explicitly requires docs and generated user skills to agree on managed-vLLM defaults and recognized override slugs. The docs page now appears updated, but generated skill/reference files still disagree with the registry and changed behavior.
  • Recommendation: Refresh the generated skill references so they consistently list DGX Spark as qwen3.6-35b-a3b-nvfp4 / nvidia/Qwen3.6-35B-A3B-NVFP4, DGX Station as qwen3.6-27b / Qwen/Qwen3.6-27B-FP8, generic Linux as nemotron-3-nano-4b / nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8, and include all recognized override slugs including deepseek-v4-flash.
  • Evidence: Acceptance clause: “Docs and generated user skills agree on managed-vLLM defaults and recognized override slugs.” docs/inference/inference-options.mdx now has the current defaults and slugs, but skills/nemoclaw-user-configure-inference/SKILL.md still lists DGX Spark as Qwen/Qwen3.6-27B-FP8; skills/nemoclaw-user-configure-inference/references/use-local-inference-details.md still says qwen3.6-27b is default on both Spark and Station and omits qwen3.6-35b-a3b-nvfp4 and deepseek-v4-flash; skills/nemoclaw-user-reference/references/commands.md still omits deepseek-v4-flash from the NEMOCLAW_VLLM_MODEL recognized slugs.
• Model-resolution errors and picker back navigation collapse to the same null result (src/lib/inference/vllm-prompt.ts:158): The back-navigation path and all resolution exceptions both return null, and installVllm maps null to { ok: false }. Back navigation is a valid user action, but the catch-all error path also covers unexpected programmer/configuration errors with the same untyped result, which makes caller behavior and future regression tests less precise.
  • Recommendation: Consider returning a typed resolution result such as { kind: 'back' } versus { kind: 'error', message } or narrowing the catch to expected env/gated-model failures. At minimum, add install-level tests for picker back and unknown NEMOCLAW_VLLM_MODEL returning before Docker prerequisites/adapters.
  • Evidence: resolveVllmInstallModel() has `if (isBackToSelection(pick)) return null;` and a catch block that logs `vLLM install failed: ...` then `return null;`; installVllm() immediately maps a falsy resolution to `{ ok: false }` before Docker prerequisites.
• Runtime-sensitive managed-vLLM path still relies mostly on mocked unit boundaries (src/lib/inference/vllm.ts:509): The PR adds strong unit coverage for picker parsing, registry filtering, env overrides, default behavior, and gated-token checks. However, the changed behavior sits at a Docker-backed onboarding/install boundary, and current install tests intentionally stop before image pull, model download, or container launch.
  • Recommendation: Add or identify targeted runtime/integration validation for the changed managed-vLLM path, especially onboarding-to-picker invocation, picker back navigation with no Docker side effects, and propagation of a picker-selected non-default model into both `hf download` and the final `vllm serve` command. Keep this separate from external E2E job status reporting.
  • Evidence: vllm.test.ts covers installVllm() with mocked runner/Docker adapters and makes dockerPrereqsOk fail via mocked runCapture, so dockerPullWithProgressWatchdog, downloadModel(), and startContainer() are not exercised for selected-model propagation.
• Managed-vLLM installer trust boundary depends on curated model and image discipline (src/lib/inference/vllm-models.ts:313): The changed picker increases user-facing access to the managed-vLLM registry, whose serve command includes `--trust-remote-code` and runs inside tag-pinned NGC images rather than digest-pinned images. The PR still uses an allowlisted registry and does not introduce arbitrary model IDs, so this is not a confirmed vulnerability, but it remains a security-sensitive installer trust boundary.
  • Recommendation: Keep the interactive picker limited to validated registry entries, document/track the validation criteria for any new model entry, and consider digest pinning or equivalent provenance controls for managed installer images where feasible.
  • Evidence: buildVllmServeCommand() includes shared `--trust-remote-code`; VLLM_IMAGES uses tags such as `nvcr.io/nvidia/vllm:26.05.post1-py3`; issue Add interactive managed-vLLM model picker and curated Spark options #4705 also says not to add unvalidated model IDs and requires validated flags, context length, memory/disk requirements, and tool-call behavior.

Workflow run details

This is an automated advisory review. A human maintainer must make the final merge decision.

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/inference/vllm-prompt.ts`:
- Around line 69-73: The current logic in the selection loop (using trimmed,
index, parseInt and ordered) accepts partially numeric input like "2abc" because
parseInt tolerates trailing text; update the validation to reject any
non-pure-numeric input before parsing: if trimmed is empty treat as 0 as before,
otherwise ensure trimmed matches a strict integer regex (e.g., /^\d+$/) or that
String(Number(trimmed)) === trimmed, and only then parse to compute index; if
validation fails call errorLine and continue. Apply this change around the
trimmed/index logic used to validate choices so partially numeric inputs are
rejected.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 41a42993-fb08-4fc7-be8e-134646095131

📥 Commits

Reviewing files that changed from the base of the PR and between 784f573 and 3e8fc64.

📒 Files selected for processing (6)

• src/lib/inference/model-prompts.test.ts
• src/lib/inference/model-prompts.ts
• src/lib/inference/vllm-models.test.ts
• src/lib/inference/vllm-models.ts
• src/lib/inference/vllm-prompt.ts
• src/lib/inference/vllm.ts

[github-actions]

Selective E2E Results — ✅ All requested jobs passed

Run: 27200903801
Target ref: 3e8fc6477cd55603ed9d79521aa09fcaa9b1e7c2
Workflow ref: main
Requested jobs: inference-routing-e2e
Summary: 1 passed, 0 failed, 0 skipped

Job	Result
inference-routing-e2e	✅ success

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/inference/vllm.test.ts`:
- Line 188: The test uses untyped map callbacks over Jest mock calls and an
unsafe access to questions[0]; update each map callback (the calls mapping that
builds summary and the other two similar maps) to explicitly type the parameter
as unknown[] (e.g., (c: unknown[]) => String(c[0])) to remove implicit any, and
protect the questions[0] access in the assertion around the questions array (in
the test referencing questions) by first asserting questions.length > 0 or using
a safe expression like (questions.length ? questions[0] : '') or questions.at(0)
?? '' so the build no longer fails.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: b79c47c9-33e2-4393-a29e-9fb8d5318bde

📥 Commits

Reviewing files that changed from the base of the PR and between 3e8fc64 and 46fca40.

📒 Files selected for processing (4)

• src/lib/inference/model-prompts.test.ts
• src/lib/inference/vllm-models.ts
• src/lib/inference/vllm-prompt.ts
• src/lib/inference/vllm.test.ts

🚧 Files skipped from review as they are similar to previous changes (3)

• src/lib/inference/vllm-models.ts
• src/lib/inference/model-prompts.test.ts
• src/lib/inference/vllm-prompt.ts

[github-actions]

Selective E2E Results — ✅ All requested jobs passed

Run: 27203986387
Target ref: 46fca407cc6c10230ada6f37c3b55f3ebc25b3ac
Workflow ref: main
Requested jobs: cloud-onboard-e2e,inference-routing-e2e
Summary: 2 passed, 0 failed, 0 skipped

Job	Result
cloud-onboard-e2e	✅ success
inference-routing-e2e	✅ success

[github-actions]

Selective E2E Results — ✅ All requested jobs passed

Run: 27205547622
Target ref: c1ae1b24d79f5351f120cd1a3a874174dbcfac69
Workflow ref: main
Requested jobs: inference-routing-e2e
Summary: 1 passed, 0 failed, 0 skipped

Job	Result
inference-routing-e2e	✅ success

[github-actions]

Selective E2E Results — ✅ All requested jobs passed

Run: 27249069423
Target ref: e4e152754f96bcfd020d25ef7f68a070186f04d4
Workflow ref: main
Requested jobs: inference-routing-e2e
Summary: 0 passed, 0 failed, 0 skipped

Job	Result
inference-routing-e2e	⚠️ cancelled

[github-actions]

Selective E2E Results — ✅ All requested jobs passed

Run: 27249232580
Target ref: 4293c7680d8c48c74218cd6a544e6ac479299999
Workflow ref: main
Requested jobs: inference-routing-e2e
Summary: 1 passed, 0 failed, 0 skipped

Job	Result
inference-routing-e2e	✅ success