docs: add inference model task-fit guide - #5527
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds a "Model Task-Fit Guide" section to ChangesModel Task-Fit Guide
Estimated code review effort🎯 2 (Simple) | ⏱️ ~12 minutes Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
2e4efbc to
fa25f95
Compare
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds and validates a new “Model Task-Fit Guide” section in the inference options docs to keep curated onboarding model guidance up to date.
Changes:
- Added a “Model Task-Fit Guide” comparison table to
docs/inference/inference-options.mdx. - Added a Vitest that ensures the section exists, has no TBD/TODO markers, and includes rows for curated onboarding models plus select additional model IDs.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| test/inference-options-docs.test.ts | Adds a docs integrity test that locates the new section and asserts required table content/model rows. |
| docs/inference/inference-options.mdx | Introduces the new model task-fit comparison table and supporting guidance text. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| const repoRoot = path.dirname(import.meta.dirname); | ||
| const inferenceOptionsPath = path.join(repoRoot, "docs", "inference", "inference-options.mdx"); | ||
| const inferenceConfigPath = path.join(repoRoot, "src", "lib", "inference", "config.ts"); |
| function readCuratedNvidiaModelIds(): string[] { | ||
| const config = fs.readFileSync(inferenceConfigPath, "utf8"); | ||
| const match = config.match(/export const CLOUD_MODEL_OPTIONS = \[(?<body>[\s\S]*?)\];/); | ||
| expect(match?.groups?.body).toBeTruthy(); | ||
| return [...match!.groups!.body.matchAll(/id:\s*"([^"]+)"/g)].map((entry) => entry[1]); | ||
| } |
| return markdown.slice(start, end); | ||
| } | ||
|
|
||
| function readCuratedNvidiaModelIds(): string[] { |
fa25f95 to
704865e
Compare
There was a problem hiding this comment.
🧹 Nitpick comments (1)
docs/inference/inference-options.mdx (1)
100-100: 💤 Low valueConsider replacing "Very large context" with a stronger descriptor.
The phrase "Very large context" appears on lines 100 and 103. Per the coding guidelines, avoid weak intensifiers like "very" in documentation. Since this table lacks specific numeric context-window values (by design, per PR objectives), consider alternatives such as:
Largest contextExtensive contextMaximum contextThese provide the same meaningful distinction from the "Large context" tier without relying on the weak intensifier "very."
Also applies to: 103-103
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/inference/inference-options.mdx` at line 100, The table contains the weak intensifier "very" in the context window descriptions listed as "Very large context" which appears multiple times in the inference options table. Replace all instances of "Very large context" with a stronger descriptor such as "Largest context", "Extensive context", or "Maximum context" to align with documentation guidelines that avoid weak intensifiers while maintaining the same meaningful distinction from the "Large context" tier.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@docs/inference/inference-options.mdx`:
- Line 100: The table contains the weak intensifier "very" in the context window
descriptions listed as "Very large context" which appears multiple times in the
inference options table. Replace all instances of "Very large context" with a
stronger descriptor such as "Largest context", "Extensive context", or "Maximum
context" to align with documentation guidelines that avoid weak intensifiers
while maintaining the same meaningful distinction from the "Large context" tier.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: a8065e5f-755f-456b-add4-7748b09eb023
📒 Files selected for processing (2)
docs/inference/inference-options.mdxtest/inference-options-docs.test.ts
|
✨ Thanks for adding the per-model task-fit guide with qualitative comparisons for task type, latency, tool-use fit, context-window fit, and relative cost. This proposes a way to extend the Inference Options page with a curated model selection table and a regression test to verify its coverage. Related open issues: |
3da1589 to
384d6fe
Compare
384d6fe to
d5309ba
Compare
|
Thanks for the updates. Two required checks still need small metadata fixes:
After those two metadata changes, push/apply the requested test update and the checks should rerun. |
<!-- markdownlint-disable MD041 --> ## Summary Refreshes the public documentation for NemoClaw v0.0.71 after scanning commits since v0.0.70. Adds release notes and fills the remaining doc gaps for Windows bootstrap diagnostics, OpenClaw agent auto-relock warnings, auto-pair cadence tuning, and plugin-install recovery hints. ## Changes - `docs/about/release-notes.mdx`: adds the v0.0.71 release-note section, grouped by gateway recovery, OpenShell auth, policy provenance, day-two maintenance, messaging/inference, and Windows setup. - `docs/get-started/windows-preparation.mdx`: documents sanitized WSL install output and reboot gating in the Windows bootstrap. - `docs/reference/commands.mdx`: documents the host `agent` wrapper's shields auto-relock warning and OpenClaw auto-pair watcher tuning variables. - `docs/reference/troubleshooting.mdx`: adds plugin-install network failure recovery guidance and updates Windows WSL troubleshooting for sanitized install logs and reboot-required handling. Source summary: - #6065 -> `docs/about/release-notes.mdx`: Notes explicit model override preservation and gateway-log guard-chain recovery diagnostics. - #5874 -> `docs/about/release-notes.mdx`: Summarizes host-mediated `recover` and `gateway restart`, linking to lifecycle, command, troubleshooting, and trusted-boundary docs already added by the source PR. - #5596 -> `docs/about/release-notes.mdx`: Summarizes OpenShell 0.0.71 gateway auth, loopback binding, and compatibility-container docs already added by the source PR. - #5797 and #5798 -> `docs/about/release-notes.mdx`: Summarizes `policy-list` provenance, Restricted tier suppression, and Balanced tier weather behavior already reflected in policy docs. - #5784 -> `docs/about/release-notes.mdx`: Summarizes `--destroy-user-data` and the safe `--yes` uninstall behavior already documented in lifecycle and command docs. - #6034 -> `docs/about/release-notes.mdx`: Summarizes custom Dockerfile warm-build cache behavior already documented in the command reference. - #5951 -> `docs/reference/commands.mdx`: Documents the stderr-only host `agent` wrapper warning after recent shields auto-relock. - #5387 -> `docs/reference/commands.mdx`: Documents OpenClaw auto-pair watcher cadence and fast-reentry tuning variables. - #5835 -> `docs/reference/troubleshooting.mdx`: Adds recovery guidance for OpenClaw plugin-install network failures. - #5995 and #5956 -> `docs/about/release-notes.mdx`: Summarizes Microsoft Teams final-message delivery and runtime mention hints already covered by messaging docs. - #5716 -> `docs/about/release-notes.mdx`: Summarizes non-interactive Ollama loopback safety already covered by local inference docs. - #5505, #5527, and #5528 -> `docs/about/release-notes.mdx`: Summarizes compatible local endpoint, model task-fit, and model capability audit docs. - #6009 -> `docs/get-started/windows-preparation.mdx`, `docs/reference/troubleshooting.mdx`: Documents sanitized Windows bootstrap WSL output and reboot-required gating. - #6055 -> no additional source doc page change needed beyond the already-merged quickstart update; release notes did not duplicate routine quickstart cleanup. No matching v0.0.71 GitHub announcement discussion was found in the latest 20 discussions, so this refresh is based on the commit scan and existing source PR docs. ## Type of Change - [ ] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [x] Doc only (includes code sample changes) ## Quality Gates <!-- Check all that apply. For any "covered by existing tests", "not applicable", or waiver entry, add a brief justification on the same line or in the Changes section. --> - [ ] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [x] Tests not applicable — justification: docs-only refresh with no runtime behavior changes. - [x] Docs updated for user-facing behavior changes - [ ] Docs not applicable — justification: - [ ] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [ ] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Verification <!-- Check each item you ran and confirmed. Leave unchecked items you skipped. Doc-only changes do not require npm test unless you ran it. --> - [x] PR description includes the DCO sign-off declaration and every commit appears as `Verified` in GitHub - [x] Git hooks passed during commit and push, or `npx prek run --from-ref main --to-ref HEAD` passes - [ ] Targeted tests pass for changed behavior - [ ] Full `npm test` passes (broad runtime changes only) - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) — ran `npm run docs`; Fern reported 0 errors and 2 existing warnings. - [x] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) --- <!-- DCO sign-off is required in this PR description, and every commit must appear as Verified in GitHub. Run: git config user.name && git config user.email --> Signed-off-by: Julie Yaunches <jyaunches@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Added a new release-notes entry covering gateway recovery, authentication, network policy/provenance output, uninstall safety, Windows bootstrap diagnostics, messaging defaults, and inference setup guidance. * Clarified Windows preparation steps around reboot behavior and redacting troubleshooting transcripts. * Expanded command reference details for OpenClaw wrapper behavior and new auto-pair tuning options. * Improved troubleshooting guidance for plugin installation issues, WSL repair/reboot cases, and install timing problems. <!-- end of auto-generated comment: release notes by coderabbit.ai --> Signed-off-by: Julie Yaunches <jyaunches@nvidia.com> Co-authored-by: Prekshi Vyas <34834085+prekshivyas@users.noreply.github.com>
## Summary - Adds a per-model task-fit guide to the Inference Options page so users can choose models by task type, latency, tool-use fit, context-window fit, and relative cost. - Keeps exact context-window limits and pricing delegated to the provider catalog because those values change outside NemoClaw releases. - Adds a focused docs regression test that ensures the guide exists and covers curated onboarding model IDs. ## Validation - `python3 scripts/generate-platform-docs.py --check` - `NODE_PATH=/Users/holim/code/NemoClaw/node_modules /Users/holim/code/NemoClaw/node_modules/.bin/vitest run test/docs-copyable-command-blocks.test.ts test/inference-options-docs.test.ts` Fixes NVIDIA#4755 Signed-off-by: Ho Lim <subhoya@gmail.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Added a “Model Task-Fit Guide” section to the inference options documentation, including a starter table mapping curated model IDs to recommended task types with qualitative comparisons for latency, tool-use quality, context-window fit, and relative cost. * **Tests** * Added automated checks to verify the guide’s table structure, ensure all expected curated model IDs are present, and confirm placeholder or undesired placeholder language is not included. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Miyoung Choi <miyoungc@nvidia.com>
<!-- markdownlint-disable MD041 --> ## Summary Refreshes the public documentation for NemoClaw v0.0.71 after scanning commits since v0.0.70. Adds release notes and fills the remaining doc gaps for Windows bootstrap diagnostics, OpenClaw agent auto-relock warnings, auto-pair cadence tuning, and plugin-install recovery hints. ## Changes - `docs/about/release-notes.mdx`: adds the v0.0.71 release-note section, grouped by gateway recovery, OpenShell auth, policy provenance, day-two maintenance, messaging/inference, and Windows setup. - `docs/get-started/windows-preparation.mdx`: documents sanitized WSL install output and reboot gating in the Windows bootstrap. - `docs/reference/commands.mdx`: documents the host `agent` wrapper's shields auto-relock warning and OpenClaw auto-pair watcher tuning variables. - `docs/reference/troubleshooting.mdx`: adds plugin-install network failure recovery guidance and updates Windows WSL troubleshooting for sanitized install logs and reboot-required handling. Source summary: - NVIDIA#6065 -> `docs/about/release-notes.mdx`: Notes explicit model override preservation and gateway-log guard-chain recovery diagnostics. - NVIDIA#5874 -> `docs/about/release-notes.mdx`: Summarizes host-mediated `recover` and `gateway restart`, linking to lifecycle, command, troubleshooting, and trusted-boundary docs already added by the source PR. - NVIDIA#5596 -> `docs/about/release-notes.mdx`: Summarizes OpenShell 0.0.71 gateway auth, loopback binding, and compatibility-container docs already added by the source PR. - NVIDIA#5797 and NVIDIA#5798 -> `docs/about/release-notes.mdx`: Summarizes `policy-list` provenance, Restricted tier suppression, and Balanced tier weather behavior already reflected in policy docs. - NVIDIA#5784 -> `docs/about/release-notes.mdx`: Summarizes `--destroy-user-data` and the safe `--yes` uninstall behavior already documented in lifecycle and command docs. - NVIDIA#6034 -> `docs/about/release-notes.mdx`: Summarizes custom Dockerfile warm-build cache behavior already documented in the command reference. - NVIDIA#5951 -> `docs/reference/commands.mdx`: Documents the stderr-only host `agent` wrapper warning after recent shields auto-relock. - NVIDIA#5387 -> `docs/reference/commands.mdx`: Documents OpenClaw auto-pair watcher cadence and fast-reentry tuning variables. - NVIDIA#5835 -> `docs/reference/troubleshooting.mdx`: Adds recovery guidance for OpenClaw plugin-install network failures. - NVIDIA#5995 and NVIDIA#5956 -> `docs/about/release-notes.mdx`: Summarizes Microsoft Teams final-message delivery and runtime mention hints already covered by messaging docs. - NVIDIA#5716 -> `docs/about/release-notes.mdx`: Summarizes non-interactive Ollama loopback safety already covered by local inference docs. - NVIDIA#5505, NVIDIA#5527, and NVIDIA#5528 -> `docs/about/release-notes.mdx`: Summarizes compatible local endpoint, model task-fit, and model capability audit docs. - NVIDIA#6009 -> `docs/get-started/windows-preparation.mdx`, `docs/reference/troubleshooting.mdx`: Documents sanitized Windows bootstrap WSL output and reboot-required gating. - NVIDIA#6055 -> no additional source doc page change needed beyond the already-merged quickstart update; release notes did not duplicate routine quickstart cleanup. No matching v0.0.71 GitHub announcement discussion was found in the latest 20 discussions, so this refresh is based on the commit scan and existing source PR docs. ## Type of Change - [ ] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [x] Doc only (includes code sample changes) ## Quality Gates <!-- Check all that apply. For any "covered by existing tests", "not applicable", or waiver entry, add a brief justification on the same line or in the Changes section. --> - [ ] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [x] Tests not applicable — justification: docs-only refresh with no runtime behavior changes. - [x] Docs updated for user-facing behavior changes - [ ] Docs not applicable — justification: - [ ] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [ ] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Verification <!-- Check each item you ran and confirmed. Leave unchecked items you skipped. Doc-only changes do not require npm test unless you ran it. --> - [x] PR description includes the DCO sign-off declaration and every commit appears as `Verified` in GitHub - [x] Git hooks passed during commit and push, or `npx prek run --from-ref main --to-ref HEAD` passes - [ ] Targeted tests pass for changed behavior - [ ] Full `npm test` passes (broad runtime changes only) - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) — ran `npm run docs`; Fern reported 0 errors and 2 existing warnings. - [x] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) --- <!-- DCO sign-off is required in this PR description, and every commit must appear as Verified in GitHub. Run: git config user.name && git config user.email --> Signed-off-by: Julie Yaunches <jyaunches@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Added a new release-notes entry covering gateway recovery, authentication, network policy/provenance output, uninstall safety, Windows bootstrap diagnostics, messaging defaults, and inference setup guidance. * Clarified Windows preparation steps around reboot behavior and redacting troubleshooting transcripts. * Expanded command reference details for OpenClaw wrapper behavior and new auto-pair tuning options. * Improved troubleshooting guidance for plugin installation issues, WSL repair/reboot cases, and install timing problems. <!-- end of auto-generated comment: release notes by coderabbit.ai --> Signed-off-by: Julie Yaunches <jyaunches@nvidia.com> Co-authored-by: Prekshi Vyas <34834085+prekshivyas@users.noreply.github.com>
Summary
Validation
python3 scripts/generate-platform-docs.py --checkNODE_PATH=/Users/holim/code/NemoClaw/node_modules /Users/holim/code/NemoClaw/node_modules/.bin/vitest run test/docs-copyable-command-blocks.test.ts test/inference-options-docs.test.tsFixes #4755
Signed-off-by: Ho Lim subhoya@gmail.com
Summary by CodeRabbit
Documentation
Tests