docs(inference): add model task-fit guidance

[harjothkhara]

Summary

The Inference Options page compares providers and hosts but never maps a model to a task type, so a reader trying to pick the right model for their use case (JTBD-4) has no basis for the decision. This adds a repo-grounded "Choosing a Model for Your Task" section so readers can match a workload to an inference option.

Related Issue

Fixes #4755

Changes

• Add a "Choosing a Model for Your Task" section to docs/inference/inference-options.mdx, placed between the provider tables and the Nemotron routing section, with a use-case table covering tool-heavy/agentic work, cost-sensitive/mixed workloads, long-context prompts, multimodal (image input), and local/offline use.
• Ground every row in existing material: Tool-Calling Reliability, the Model Router tolerance mechanism, NEMOCLAW_CONTEXT_WINDOW, NEMOCLAW_INFERENCE_INPUTS, and the local-inference guide. A short note states that relative latency and price are intentionally not ranked, since those are not repo-owned facts and vary by provider/region/revision.
• Add test/repro-4755-model-task-fit.test.ts (the doc-validation test requested in the issue): asserts the section, each use-case row, the cited NEMOCLAW_* knobs, and the internal links are present and that the section precedes the Nemotron section.
• Verification run manually (prek hooks were not installed in this environment): vitest run test/repro-4755-model-task-fit.test.ts (14/14), npm run docs (0 errors; the only 2 warnings are pre-existing global ones — Fern redirects-auth and theme accent-contrast — not introduced by this change), biome check (clean), and npm run typecheck:cli (0 errors).

Type of Change

• Code change (feature, bug fix, or refactor)
• Code change with doc updates
• Doc only (prose changes, no code sample modifications)
• Doc only (includes code sample changes)

Verification

• Git hooks passed during commit and push, or npx prek run --from-ref main --to-ref HEAD passes
• Targeted tests pass for changed behavior
• Full npm test passes (broad runtime changes only)
• Tests added or updated for new or changed behavior
• No secrets, API keys, or credentials committed
• Docs updated for user-facing behavior changes
• npm run docs builds without warnings (doc changes only)
• Doc pages follow the style guide (doc changes only)
• New doc pages include SPDX header and frontmatter (new pages only)

---

Signed-off-by: harjoth harjoth.khara@gmail.com

Summary by CodeRabbit

• Documentation

  • Added a new “Choosing a Model for Your Task” section with a workload-to-recommendation guide, including guidance for tool-heavy/agentic work, cost-sensitive use cases, long-context prompts, multimodal image input, and local/offline/privacy-sensitive scenarios.
  • Clarified that latency and price aren’t ranked, and recommended automated quality-vs-cost balancing.

• Tests

  • Added a regression test to confirm the section’s placement, expected table/content, anchor links, and that it does not claim latency/price ranking.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 0ec0ee37-4a53-4240-aa57-446442428a5c

📥 Commits

Reviewing files that changed from the base of the PR and between c327437 and ae16584.

📒 Files selected for processing (2)

• docs/inference/inference-options.mdx
• test/repro-4755-model-task-fit.test.ts

✅ Files skipped from review due to trivial changes (1)

• docs/inference/inference-options.mdx

🚧 Files skipped from review as they are similar to previous changes (1)

• test/repro-4755-model-task-fit.test.ts

---

📝 Walkthrough

Walkthrough

A new "Choosing a Model for Your Task" section is added to docs/inference/inference-options.mdx, containing a workload-to-model recommendation table and a note about Model Router and non-ranking of latency/price. A companion Vitest regression test in test/repro-4755-model-task-fit.test.ts validates the section's placement, content, anchors, and non-ranking assertion.

Changes

Task-fit guidance section and regression test

Layer / File(s)	Summary
New task-fit section in inference-options.mdx docs/inference/inference-options.mdx	Inserts a "Choosing a Model for Your Task" section (lines 77–94) with a five-row workload-to-model table covering tool-heavy/agentic, cost-sensitive, long-context, multimodal, and local/offline scenarios, plus a note that the page does not rank by latency or price.
Regression test for task-fit section test/repro-4755-model-task-fit.test.ts	Adds a 70-line Vitest test that reads the MDX file and asserts: section precedes the Nemotron routing section, table header is present, use-case strings and NEMOCLAW_* knob names appear, expected internal anchors are cited, and a regex confirms no relative latency/price ranking claim exists.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Poem

🐇 Hop hop, the docs now show the way,
A table of tasks to guide your day!
Tool-heavy agents, long contexts too,
Model Router picks the best for you.
No rankings by price or latency speed—
Just clear task-fit for every need! ✨

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'docs(inference): add model task-fit guidance' clearly summarizes the main change: adding model task-fit guidance documentation to the inference options page.
Linked Issues check	✅ Passed	The PR fully addresses issue #4755 by adding a 'Choosing a Model for Your Task' section with use-case mapping and a regression test to validate the documentation remains current.
Out of Scope Changes check	✅ Passed	All changes are directly scoped to the requirements: documentation additions to inference-options.mdx and a validation test file with no extraneous modifications.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

🧹 Nitpick comments (1)

docs/inference/inference-options.mdx (1)

85-85: ⚡ Quick win

Remove non-essential bold emphasis in routine guidance.

Line [85], Line [88], and Line [92] use bold for ordinary guidance terms rather than UI labels or warnings.
Please switch these to plain text to match the docs style rule.
LLM pattern detected.

As per coding guidelines, bold in docs should be reserved for UI labels, parameter names, and genuine warnings.

Also applies to: 88-88, 92-92

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/inference/inference-options.mdx` at line 85, The documentation file
contains unnecessary bold emphasis on ordinary guidance terms across three
locations. Remove the bold formatting (** delimiters) from routine guidance
words on lines 85, 88, and 92 that are not UI labels, parameter names, or
warnings. Keep bold formatting only for actual UI element names and feature
labels like "Model Router" if they function as proper nouns or UI references,
but remove bold from descriptive terms that describe functionality or workflow
guidance. Review each of the three affected line ranges to identify which bolded
text should be converted to plain text based on whether it is a genuine UI label
or a routine descriptive term.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@docs/inference/inference-options.mdx`:
- Line 85: The documentation file contains unnecessary bold emphasis on ordinary
guidance terms across three locations. Remove the bold formatting (**
delimiters) from routine guidance words on lines 85, 88, and 92 that are not UI
labels, parameter names, or warnings. Keep bold formatting only for actual UI
element names and feature labels like "Model Router" if they function as proper
nouns or UI references, but remove bold from descriptive terms that describe
functionality or workflow guidance. Review each of the three affected line
ranges to identify which bolded text should be converted to plain text based on
whether it is a genuine UI label or a routine descriptive term.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 3fa69db3-9442-440b-9fc2-6e73d354d056

📥 Commits

Reviewing files that changed from the base of the PR and between f4f3c58 and 1a9961a.

📒 Files selected for processing (2)

• docs/inference/inference-options.mdx
• test/repro-4755-model-task-fit.test.ts

[wscurran]

✨ Thanks for adding the model task-fit guidance section to the inference options page with a use-case table covering tool-heavy agentic work, cost-sensitive workloads, long-context prompts, multimodal input, and local offline use. This proposes a way to ground model selection in existing repo material like Tool-Calling Reliability, the Model Router tolerance mechanism, and the local inference guide.

---

Related open issues:

• #4755 [All Platforms][Docs] inference-options page lacks per-model task-fit comparison

[harjothkhara]

Closing this as superseded. Issue #4755 was resolved and closed (completed 2026-06-30) by #5527, which added a ## Model Task-Fit Guide to docs/inference/inference-options.mdx — a full per-model table (best-for task type, relative latency, tool-use quality, context-window fit, relative cost) covering the curated onboarding models. That already delivers the per-model task-fit comparison this PR set out to add, so this section would now duplicate existing docs. Thanks — good to see the guidance is in place. 🙏