docs: clarify local compatible endpoints

[glenn-agent]

Summary

• clarify that raw GGUF files should be served through an OpenAI-compatible server such as llama.cpp
• document that the Ollama local option expects Ollama model tags, not direct .gguf filesystem paths
• add example llama.cpp and non-interactive compatible-endpoint setup, including legacy host-gateway URL guidance

Fixes #2412

Testing

• NODE_PATH=/workspace/openclaw/projects/nemoclaw/upstream/node_modules PATH=/workspace/openclaw/projects/nemoclaw/upstream/node_modules/.bin:$PATH npm run docs
• git -c gpg.ssh.allowedSignersFile=/workspace/openclaw/projects/nemoclaw/upstream/.git/info/allowed_signers verify-commit HEAD

Signed-off-by: Glenn-Agent glenn_agent@163.com

Summary by CodeRabbit

Documentation

• Expanded configuration guidance for OpenAI-compatible inference endpoints with additional setup examples
• Added comprehensive instructions for serving local model files using compatible backends, including startup command examples
• Clarified model ID selection from server endpoints and improved endpoint verification guidance
• Enhanced documentation for legacy containerized gateway configurations

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: fa05736a-97f1-40b5-9bf3-24e02d304e13

📥 Commits

Reviewing files that changed from the base of the PR and between 6c0fb04 and 3a1d65e.

📒 Files selected for processing (2)

• docs/inference/inference-options.mdx
• docs/inference/use-local-inference.mdx

---

📝 Walkthrough

Walkthrough

Two documentation files are updated to address GGUF model local inference. use-local-inference.mdx gains a new subsection describing how to serve a GGUF file via llama-server and onboard it as an "Other OpenAI-compatible endpoint." inference-options.mdx expands the interactive base URL prompt with legacy gateway, GGUF server, and Ollama model-tag clarifications.

Changes

GGUF Local Inference Documentation

Layer / File(s)

Summary

GGUF llama-server onboarding guide and base URL prompt expansion docs/inference/use-local-inference.mdx, docs/inference/inference-options.mdx

use-local-inference.mdx adds a new subsection with an example llama-server startup command, a non-interactive NemoClaw onboarding command template, instructions to use /v1/models output as the model ID, a legacy host.openshell.internal gateway URL note, and a step to verify the Inference status row. inference-options.mdx expands the "base URL" interactive prompt text with the same legacy gateway URL, GGUF server guidance referencing /v1/models, and a clarification that the Ollama local path picker selects model tags rather than raw .gguf file paths.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Poem

🐇 Hippity-hop, the docs have grown,
A GGUF model no longer alone!
llama-server starts with a whirr,
/v1/models tells you which to prefer.
The gateway host is noted with care —
Now fuzzy ears know exactly what's there! 🌟

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'docs: clarify local compatible endpoints' directly reflects the main change—clarifying documentation for local inference endpoints in the changeset.
Linked Issues check	✅ Passed	Changes address all key requirements from #2412: clarifying GGUF file serving via OpenAI-compatible endpoints, documenting Ollama's expectation of model tags, and providing practical llama.cpp setup examples with host-gateway guidance.
Out of Scope Changes check	✅ Passed	All changes are documentation updates directly addressing local inference endpoint clarification as defined in the PR objectives and linked issue requirements.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[wscurran]

✨ Thanks for clarifying that raw GGUF files require an OpenAI-compatible server like llama.cpp and that Ollama expects model tags rather than direct filesystem paths. This proposes a way to document compatible-endpoint setup with example llama.cpp and non-interactive configuration including legacy host-gateway URL guidance.

---

Related open issues:

• #2412 local-inference enhancement

[prekshivyas]

Clean additions. The GGUF/llama-server section fills a real gap — previously there was no guidance for users who downloaded a model file directly. Docker-driver topology URL note and legacy gateway path caveat are both accurate. Non-interactive env var example looks correct. No open findings.