Skip to content

[Doc] cookbook: Refresh SemiAnalysis AgentX launches to InferenceX main c5e8b9b3 and revert #47560 #47929

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
nvpohanh merged 2 commits into from
Nov 3, 2026
Merged

[Doc] cookbook: Refresh SemiAnalysis AgentX launches to InferenceX main c5e8b9b3 and revert #47560 #47929

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b721c54
[Doc] cookbook: Revert hand-added Qwen3.8-Flash-Next B300 NVFP4 Agent…
nvpohanh Nov 2, 2026
3a3825d
[Doc] cookbook: Refresh SemiAnalysis AgentX launches to InferenceX ma…
nvpohanh Nov 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 1 addition & 6 deletions docs/cookbook/autoregressive/Qwen/Qwen3.8-Flash-Next.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,13 +79,8 @@ Agentic workloads, such as coding agents and other multi-turn, tool-using assist

import { AgentX } from "/src/snippets/_agentx.jsx";
import { agentx } from "/src/snippets/agentx/Qwen/qwen3.8-flash-next.jsx";
import { b300Cells } from "/src/snippets/agentx/Qwen/qwen3.8-flash-next-b300.jsx";

The B300 HiCache configurations reserve 298 GB of host memory for the combined target/draft KV and Mamba pool, in addition to pinned PLE embeddings and process overhead. Provision sufficient free host RAM and container shared memory before starting them. GPU KV configurations also offload PLE embeddings to host memory.

For the B300 NVIDIA configurations at concurrency 12 and above, `--mamba-track-interval 262144` reduces intermediate state checkpoints. The concurrency-80 configuration also uses `--strip-thinking-cache` and `--mamba-max-states-per-path 2`. These settings target replay traffic that does not resend model output; reassess prefix reuse for conversational clients that do.

<AgentX data={{ ...agentx, cells: [...agentx.cells, ...b300Cells] }} />
<AgentX data={agentx} />

### MI355X deployment measured with SemiAnalysis InferenceX

Expand Down
Loading
Loading