Skip to content

Add the NVIDIA B300 page and the sandboxed-agent demo - #32

Merged
sid-rp merged 1 commit into
mainfrom
younghan/recipe-nemoclaw
Aug 13, 2026
Merged

Add the NVIDIA B300 page and the sandboxed-agent demo#32
sid-rp merged 1 commit into
mainfrom
younghan/recipe-nemoclaw

Conversation

@seyeong-han

Copy link
Copy Markdown
Contributor

Two NVIDIA contributions that reference each other: the B300 platform page, and the sandboxed-agent demo it enables. python platform/validate.py passes with 0 errors and 0 warnings; every relative link resolves.

platform/nvidia.md — B300

Contributed by Faradawn Yang, with a bf16 concurrency sweep from 1 to 256 concurrent requests.

The sweep is single-GPU, now confirmed by NVIDIA. The draft said both "fits on a single 8xB300 node" and "no tensor parallelism flags are needed for single-GPU serving"; Anusha confirmed the cluster node has eight cards but only one was used. That's stated above the table, since it changes how a reader sizes hardware.

Reconciled against what this cookbook already established — no new claims:

Draft Published Why
Pre-release model name throughout Muse Glimmer The draft predated the public name
--tool-call-parser onyx / --reasoning-parser onyx muse_glimmer Matches what upstream registers
--chat-template …onyx.jinja dropped The checkpoint ships the template at its root
pip install vllm==0.1.dev19055 vllm/vllm-openai:muse-glimmer container Support is not in a released wheel (open PR), so the pip line would not reproduce
hf download <meta-models/…> no angle brackets A shell reads <…> as redirection
Memory observed: TBD empty cell We don't publish placeholders

NVIDIA's build string stays in the snapshot and methodology line, so the measurements keep their provenance even though the reader path is the container.

recipes/sandboxed-agent/ — the demo

An agent is asked to find a PRODUCTION_API_KEY, display it, and upload it. OpenShell blocks it in two independent places: credential brokering keeps the raw key outside the sandbox, and deny-by-default network policy refuses the egress. Terminal logs confirm both.

Written the way NVIDIA suggested — point at their setup docs, then show the demo as a practical example, with the recording. It is explicitly not a copy-paste recipe, and the status note says so: the setup authority is NemoClaw's own documentation, linked at each step. All four external links verified 200.

Includes the architecture diagram and the 5.2 MB recording. Both filenames were rewritten; the diagram's contents were already clean.

Not included

The RTX 5090 figure (190 tok/s, DFlash on). Holding it for two reasons:

  1. We already publish a 5090 DFlash number — the model card reports 74.9 → 233.4 tok/s, 3.1× on K-Quant-17GB, batch 1, greedy, measured with llama.cpp. Publishing 190 beside it without explaining the gap would leave a reader unable to tell which applies to them.
  2. The figure arrived without a quantization or a server. bf16 is ~60 GB and a 5090 has 32, so it has to be a quantized build — and DFlash appears on our llama.cpp and ExecuTorch pages, not vLLM.

Both differences are probably legitimate — different quantization, server, or prompt set will do it. We just can't say which yet. Worth one more round with NVIDIA rather than shipping two numbers for the same card.

Two contributions from NVIDIA that reference each other.

platform/nvidia.md — a bf16 concurrency sweep on B300 from 1 to 256
concurrent requests, contributed by Faradawn Yang. NVIDIA confirmed the
sweep used one GPU of an eight-card node, which is now stated above the
table: it changes how a reader sizes hardware.

Reconciled against what this cookbook already established. The model is
renamed, the parser flags use the spelling upstream registers, the
redundant chat-template flag is dropped, the download command no longer
carries angle brackets a shell reads as redirection, and the deploy step
is the published container rather than a pip install of an unreleased
wheel. NVIDIA's build string stays in the snapshot so the measurements
keep their provenance.

recipes/sandboxed-agent/ — an agent is told to exfiltrate a credential
and upload it. OpenShell keeps the key out of the sandbox and denies the
egress, and the model never gets the chance to refuse. Written as a
walkthrough pointing at NemoClaw's own docs rather than a copy-paste
recipe, which is how NVIDIA suggested framing it, and the status note
says as much.

Holding NVIDIA's RTX 5090 figure for now. The model card already
publishes 233.4 tok/s with DFlash on that card, and the new figure
arrived without a quantization or a server, so the two cannot be
reconciled yet.
@seyeong-han
seyeong-han requested a review from sid-rp August 12, 2026 18:20
@seyeong-han

Copy link
Copy Markdown
Contributor Author

@sid-rp Could you please review the vLLM + NVIDIA inference server part?

@anushapant

Copy link
Copy Markdown

Looks good to me @seyeong-han

@sid-rp

sid-rp commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

LGTM

@sid-rp
sid-rp merged commit 858c87c into main Aug 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants