Super 3.5 vllm model tuning (Qwen3.5-122B-A10B, Inkling-Small) by fsiino-nvidia · Pull Request #2599 · NVIDIA-NeMo/Gym

fsiino-nvidia · 2026-08-18T18:30:28Z

Summary

This PR adds model-specific serving configurations for Qwen3.5-122B-A10B and Inkling Small, configurable decode routing and evaluation concurrency, resumable rollouts, fail-fast multi-node serving, and benchmark-level failure containment.

Shared infrastructure and benchmark reliability

Configurable decode routing: allow tuned runs to override the default cache_aware decode-routing policy while preserving that default for callers that do not set an override.
```
VLLM_DECODE_POLICY=power_of_two \
NUM_DECODE_NODES=4 \
bash benchmarks/nemotron_3.5_super/sbatch_external_vllm.sh ...
```
Opt-in evaluation concurrency: pass the number of parallel Gym samples only when explicitly configured. This enables throughput tuning without changing existing model runs by default.
```
NUM_SAMPLES_IN_PARALLEL=64 \
bash benchmarks/nemotron_3.5_super/sbatch_external_vllm.sh ...
```
Resumable evaluations: use a stable output path and reuse completed rollouts after a Slurm requeue or manual resubmission with the same compatible experiment configuration. Note: reuses compatible completed rollouts when a command is restarted with the same experiment name. It does not itself request a Slurm requeue.
```
RESUME_EVAL_ON_REQUEUE=1 \
EXPERIMENT_NAME="<stable-experiment-name>" \
bash benchmarks/nemotron_3.5_super/sbatch_external_vllm.sh ...
```
Fail-fast multi-node serving: add Slurm's --kill-on-bad-exit=1 so one failed vLLM worker terminates the server step and propagates through the launcher's existing lifecycle monitoring instead of leaving a partial deployment running. The Gym readiness timeout remains configurable.
```
bash benchmarks/nemotron_3.5_super/sbatch_external_vllm.sh \
  --config <config.yaml> \
  ++model_endpoint_readiness_timeout_seconds=1200
```
Tau2 setup and rollout containment: serialize Tau2 data initialization; validate model-emitted tool arguments; retry malformed JSON up to five generations; and, if a rollout still fails, emit a structured zero-reward failure record instead of an HTTP 500 that aborts the full suite.

Container prerequisite: install the locked profiling dependency required by gym eval prepare and verify it while building the evaluation image.

INPUT_CONTAINER="<base.sqsh>" \
OUTPUT_CONTAINER="<with-gym.sqsh>" \
MOUNTS="<mounts>" \
GYM_CONFIG="<config.yaml>" \
sbatch benchmarks/nemotron_3.5_super/build_eval_container.sh

OpenCode export paths: strip the sandbox pwd newline before building the export path, preventing valid /testbed/export.json artifacts from being requested with a trailing newline.

Qwen3.5-122B-A10B

Model-specific changes

Configure Qwen's tool-call and reasoning parsers, multimodal encoder parallelism, and TP4/EP4 execution.
Split NIXL KV-cache transfer into prefill-producer and decode-consumer settings.
Use one DP1-per-node TP4 prefill server and four DP1-per-node TP4/EP4 decode servers (P1/D4), matching main's launcher topology.
Use fixed-scale FP8 KV cache, CUDA graphs, synchronous scheduling, and a 512-sequence engine limit.
Enable two-token multi-token prediction (MTP, speculative decoding from the model's prediction heads) on both serving roles when QWEN_ENABLE_MTP=1; matching settings preserve compatible transferred KV-cache layouts.
Implementation: benchmarks/nemotron_3.5_super/vllm_configs/qwen3.5-122b-a10b.sh.

How to run

MODEL=/lustre/fsw/portfolios/llmservice/users/igitman/hf_models/Qwen3.5-122B-A10B \
VLLM_CONFIG=benchmarks/nemotron_3.5_super/vllm_configs/qwen3.5-122b-a10b.sh \
QWEN_ENABLE_MTP=1 \
RESUME_EVAL_ON_REQUEUE=1 \
EXPERIMENT_NAME=super3.5-e2e/qwen3.5-122b-a10b-p1d4-dp1-c64-p2-ep4x4-seqs512-cg256-fp8kv-scale1-sync-nopc-mtp2 \
NUM_SAMPLES_IN_PARALLEL=64 \
NUM_PREFILL_NODES=1 \
NUM_DECODE_NODES=4 \
VLLM_DECODE_POLICY=power_of_two \
EXPORT_TO_CSV=1 \
SBATCH_TIME=<time-limit> \
SBATCH_ACCOUNT=<slurm-account> \
SBATCH_PARTITION=<slurm-partition> \
SBATCH_QOS=normal \
SBATCH_GRES=gpu:4 \
CONTAINER=$(pwd)/results/vllm/vllm-openai:v0.25.1___tomer_with_gym.sqsh \
MOUNTS=/lustre:/lustre,$(pwd):/opt/Gym \
bash benchmarks/nemotron_3.5_super/sbatch_external_vllm.sh \
  --config benchmarks/nemotron_3.5_super/eval_container_config.yaml \
  ++model_endpoint_readiness_timeout_seconds=1200

Validation

The full 11,213-rollout suite completed on HSG GB200 with P1/D4, concurrency 64, TP4/EP4, MTP2, and zero failed rollouts.

Serving metric	Average	Minimum	Maximum
Output tokens/s/request	149.6	127.2	177.4
Waiting requests	0	0	0
GPU KV-cache usage	10.7%	3.0%	14.5%
Aggregate output throughput	8,701 tok/s	6,400 tok/s	9,492 tok/s

Benchmark	Score
Tau3-Banking	14.2%
Tau3-Average	60.1%
SciCode — whole problem	8.1%
SciCode — subtask	38.2%
AA-LCR	65.9%
AA-Omniscience (OmniIndex)	25.4%
GPQA Diamond	85.9%

Inkling Small

Model-specific changes

Add Inkling's V2 runner, tokenizer mode, FlashAttention CuTe DSL cache, tool-call parser, reasoning parser, and FlashInfer-autotune setting from the upstream recipe while intentionally serving the local BF16 checkpoint.
Use TP4/EP4, prefix caching, chunked prefill, synchronous scheduling, and full-decode CUDA graphs.
Split NIXL transfer roles and use two DP1-per-node prefill servers plus four DP1-per-node decode servers (P2/D4), matching main's launcher topology. The second prefill node removes the sustained queue observed with P1/D4; a larger token budget alone did not remove it.
Give each prefill engine a 16,384-token scheduling budget and each decode replica an 8,192-token budget.
Make MTP draft width configurable and apply it symmetrically to prefill and decode. MTP2 was selected because its approximately 63% acceptance avoided the throughput collapse observed when drafting all eight tokens.
Implementation: benchmarks/nemotron_3.5_super/vllm_configs/inkling_small.sh.

Container

Inkling required a separate recipe-compatible image layered with Gym and vllm-router. The validated image is results/vllm/vllm-openai:nightly-inkling-small-20260815_with_gym.sqsh, built from vLLM 0.27.2rc1.dev77+gac7509e2b. The recipe's compatibility floor is vLLM 0.26; this records the exact newer build tested.

How to run

MODEL=/lustre/fsw/portfolios/llmservice/users/lvega/models/Inkling-Small \
VLLM_CONFIG=benchmarks/nemotron_3.5_super/vllm_configs/inkling_small.sh \
INKLING_ENABLE_MTP=1 \
INKLING_MTP_NUM_SPECULATIVE_TOKENS=2 \
RESUME_EVAL_ON_REQUEUE=1 \
EXPERIMENT_NAME=super3.5-e2e/inkling-small-bf16-p2d4-dp1-c64-p2-ep4x4-seqs256-cg256-sync-pc-mtp2-pbt16384 \
NUM_SAMPLES_IN_PARALLEL=64 \
NUM_PREFILL_NODES=2 \
NUM_DECODE_NODES=4 \
VLLM_DECODE_POLICY=power_of_two \
EXPORT_TO_CSV=1 \
SBATCH_ACCOUNT=<slurm-account> \
SBATCH_PARTITION=<slurm-partition> \
SBATCH_QOS=normal \
SBATCH_TIME=<time-limit> \
SBATCH_GRES=gpu:4 \
CONTAINER=$(pwd)/results/vllm/vllm-openai:nightly-inkling-small-20260815_with_gym.sqsh \
MOUNTS=/lustre:/lustre,$(pwd):/opt/Gym \
bash benchmarks/nemotron_3.5_super/sbatch_external_vllm.sh \
  --config benchmarks/nemotron_3.5_super/eval_container_config.yaml \
  ++model_endpoint_readiness_timeout_seconds=1200

Note: The full suite also requires the externally staged, gitignored benchmarks/scicode/data/test_data.h5 fixture.

Validation

The validation run completed all 11,213 rows on HSG GB200 in 2:30:13. It produced 11,212 normal rows and one structured failure caused by an external Omniscience judge HTTP 500, not by Inkling or vLLM.

Serving metric	Result
Output tokens/s/request	107.06 mean; 96.77 p5; 89.74 minimum; 826/827 loaded windows at least 90
Derived local decode queue	0 in all 3,432 samples
Literal decode waiting	NIXL-transfer deferrals only; per-replica maxima 2, 3, 4, and 6
Prefill waiting	Nonzero in 14/1,718 samples; brief maxima 8 and 5; no sustained buildup
GPU KV-cache usage	3.74-3.97% mean; 8.04-9.45% p95; 14.2-18.9% maximum

Benchmark	Score
Tau3-Banking	11.3%
Tau3-Average	59.5%
SciCode — whole problem	16.9%
SciCode — subtask	44.5%
AA-LCR	66.9%
AA-Omniscience (OmniIndex)	33.8%
GPQA Diamond	87.9%

Note on CRITPT batching: Main commit 54cee4f1 (merge commit ef1dd91b) enabled CRITPT in the Super suite. CRITPT's public scorer waits for 70 distinct problems before scoring a batch. The suite interleaves 5x70-item batches. At concurrency 64 (our tuned setting for optimal output tokens/s/request) every Gym slot can become occupied by a rollout waiting for an incomplete batch and block eval progress.

To clear CRITPT, you can cancel and resubmit with RESUME_EVAL_ON_REQUEUE=1 and NUM_SAMPLES_IN_PARALLEL=350. This will lower output tokens/s/request but provide enough slots for all 350 CRITPT rollouts to complete their batches. Once all 350 CRITPT rows have been scored and written to resumable.jsonl, either cancel and resume at concurrency 64 or allow the run to finish at concurrency 350.

Signed-off-by: Frankie Siino <fsiino@nvidia.com>

Disable prefix caching, reduce the maximum sequence count from 512 to 256, match CUDA-graph capture to 256, and enforce eager execution to isolate the decode stalls observed during full-suite evaluation. Signed-off-by: Frankie Siino <fsiino@nvidia.com>

Pass num_samples_in_parallel only when explicitly configured. Validate the value and filter stale exclusions against Slurm's current node inventory before submission. Signed-off-by: Frankie Siino <fsiino@nvidia.com>

Remove FP8 KV caching and eager execution while retaining the 256 sequence and capture limits. Bound each request to four images and one video. Signed-off-by: Frankie Siino <fsiino@nvidia.com>

Store attention KV blocks in FP8 to reduce cache memory traffic and preserve decode throughput. Disable runtime scale calculation because Qwen3.5 hybrid-cache warmup produced invalid inferred scales. Signed-off-by: Frankie Siino <fsiino@nvidia.com>

Allow each decode node to run a local TP/EP replica instead of forming a cross- node DP/EP group. Register every replica with the PD router and add an opt-in decode policy for balanced variable-length eval traffic. Preserve the existing coupled and consistent-hash defaults. Signed-off-by: Frankie Siino <fsiino@nvidia.com>

Parse combined Slurm logs from coupled or independent decode deployments and report synchronized aggregate throughput, output tokens per second per request, waiting requests, and GPU KV-cache usage. Signed-off-by: Frankie Siino <fsiino@nvidia.com>

Sandbox pwd output included a trailing newline, causing the agent to request /testbed\n/export.json and receive FILE_NOT_FOUND. Strip the output before building the path and test that /testbed/export.json is downloaded. Signed-off-by: Frankie Siino <fsiino@nvidia.com>

…odel-tuning Signed-off-by: Frankie Siino <fsiino@nvidia.com> # Conflicts: # benchmarks/nemotron_3.5_super/build-super-vl-rl-v0251-thin.sh # benchmarks/nemotron_3.5_super/build_eval_container.sh # benchmarks/nemotron_3.5_super/sbatch_external_vllm.sh

Use a stable output path and enable Gym's rollout cache when RESUME_EVAL_ON_REQUEUE=1. This prevents requeued jobs from losing completed work by starting under a new timestamped directory, while keeping the existing behavior unchanged by default. Signed-off-by: Frankie Siino <fsiino@nvidia.com>

Move topology and batching arguments into the prefill and decode configs, raise max sequences to 512, and optionally enable MTP on both roles. Signed-off-by: Frankie Siino <fsiino@nvidia.com>

copy-pr-bot · 2026-08-18T18:30:32Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Signed-off-by: Frankie Siino <fsiino@nvidia.com>

…odel-tuning Signed-off-by: Frankie Siino <fsiino@nvidia.com>

Signed-off-by: Frankie Siino <fsiino@nvidia.com>

…odel-tuning Signed-off-by: Frankie Siino <fsiino@nvidia.com> # Conflicts: # benchmarks/nemotron_3.5_super/sbatch_external_vllm.sh

Default to zero retries after a malformed tool response and expose malformed_tool_call_max_retries for explicit overrides. Signed-off-by: Frankie Siino <fsiino@nvidia.com>

Signed-off-by: Frankie Siino <fsiino@nvidia.com>

Signed-off-by: Brian Yu <bxyu@nvidia.com>

…/super35-vllm-model-tuning Signed-off-by: Brian Yu <bxyu@nvidia.com>

bxyu-nvidia · 2026-08-26T16:52:47Z

/ok to test 4eb599a

Merges `main` at b793621 ("Super 3.5 vllm model tuning (Qwen3.5-122B-A10B, Inkling-Small)", #2599) rather than the current `main` tip. WHY THIS TARGET b793621 is the commit immediately before 65129dd ("chore(deps): pin openai to 2.44.0", #2456). That PR added class NeMoGymChatCompletionCreateParamsNonStreaming(BaseModel): model_config = ConfigDict(extra="forbid") which makes the proxy reject every agentic chat-completions request that carries `chat_template_kwargs` — the field agentic GDPVal runs use to drive thinking budgets on self-hosted models. Each such request comes back as 422 Unprocessable Entity {"detail":[{"type":"extra_forbidden","loc":["body","chat_template_kwargs"], "msg":"Extra inputs are not permitted","input":{"thinking":true}}]} so the rollout fails before it reaches the model. Merging `main` past 65129dd would take that regression into the GDPVal branch and break the production runs this branch exists to serve. b793621 was chosen because it is the last commit that is regression-free while still giving the branch everything else it needs: * `requires-python = ">=3.13.14"`, so the py313 runtime container stays correct; * `extra="forbid"` appears exactly once in nemo_gym/openai_utils.py, on `NeMoGymResponseCreateParamsNonStreaming` (the pre-existing 2025 Responses-class validation) and NOT on the ChatCompletion class; * the rollout-observability work (bc521f7) is an ancestor. CONFLICTS AND RESOLUTIONS Eight files conflicted; all GDPVal files auto-merged. Every conflict was resolved to keep both sides' behaviour rather than pick a winner. * nemo_gym/cli/env.py, cli/setup_command.py and their tests: `get_venv_path` and `resolve_server_venv_path` are the same function under two names (byte-identical bodies). Canonicalised on devel's name and docstring, kept main's `Path(...)` normalisation of `root_venv_path`, renamed main's call sites and the `monkeypatch.setattr` in test_cli.py, and dropped the now-unused `ROOT_DIR` import. * nemo_gym/openai_utils.py: took main's content-part types — its `video_url` part accepts `Union[str, Dict[str, Any]]` where devel's was dict-only, and it adds `NeMoGymChatCompletionContentPartFileParam` — while keeping devel's comment recording the 422 that motivated adding a video part at all. * responses_api_models/local_vllm_model/setup.py: both sides pin vllm==0.24.0 at this merge target, so the resolution is main's file (which additionally pins flashinfer-python==0.6.12) plus devel's note that 0.24.0 is the first release with MiniMax-M3 support. * nemo_gym/rollout_reverification.py: plain union of both import lists from nemo_gym.rollout_collection; dropped devel's `NG_TERMINAL_KEY as NG_TERMINAL_KEY` re-export idiom, nothing needs it. * nemo_gym/rollout_collection.py: kept both sides' features — main's exporters (`upload_rollouts`/`export_rollouts`/`get_exporters`, replacing the W&B-specific path) and token-capture retirement, devel's dispatch budget, drain margin and `DispatchLatencyTracker`, `kill_shaped` no-persist rows, `_validate_dispatch_concurrency` and `ordered_tasks`. Three resolutions here are semantic rather than textual: - main's `from time import time` SHADOWS the `time` module that devel's `time.monotonic()` calls need, so the module import was kept and main's two bare `time()` calls rewritten as `time.time()`; taking main's side verbatim compiles and then fails at runtime; - the persistence branch takes main's flat `if no_persist / elif failure_class is not None / else` chain with its token-capture retirement, dropping devel's `result_strs.append` (it fed the removed W&B table) and the stale "not the W&B table either" wording in the kill_shaped comment; - devel's `ordered_tasks = [asyncio.ensure_future(...)]` scheduling was kept, because main still passes bare coroutines to `as_completed`, which makes `dispatch_longest_first` a no-op, and the shared code after the conflict references `ordered_tasks`. * tests/unit_tests/test_rollout_collection.py: import union, and devel's version of the fresh-run cleanup test, which is a superset of main's and additionally asserts the aggregate-metrics file is cleared. The fresh-run cleanup in `run_from_config` now unlinks the failures sidecar and the aggregate-metrics file so a re-run cannot inherit retry attempts or published metrics from an older run at the same output path. VERIFICATION `ruff check` and `ruff format --check` with the CI-pinned ruff 0.9.9 are clean across nemo_gym, tests, responses_api_models and resources_servers. resources_servers/gdpval/tests: 386 passed, 11 skipped — the `reference_missing` path (#2796) and `strict_comparison_trials` (#2807) are both intact. tests/unit_tests against an `origin/main` baseline built in a scratch worktree on the same venv: 79 failures on the baseline, 101 here. The 44 new failures are all in test_openai_utils.py (43) plus test_responses_api_model_streaming.py::test_prunes_nested_extra_fields, and are an environment artefact, not a merge defect: b793621 predates #2456 and still declares `openai<=2.7.2`, while the venv has openai 2.44.0 installed. Those tests enumerate the *installed* SDK's item tags (`shell_call`, `apply_patch_call`, `compaction`, `tool_search_call`, ...) and require a Gym union member for each; the pre-2.44 schemas in this tree have none. Re-running the same suite with an openai inside the declared pin (2.7.2 shadowed onto PYTHONPATH, venv untouched) gives 1 failed / 2976 passed — the single failure, test_opensandbox_cleanup.py::test_script_help_runs_by_ direct_path, is also present in the origin/main baseline. So the new-failure set against baseline is EMPTY once the SDK matches the pin, and no failure touches rollout collection, reverification, the CLI or GDPVal. Signed-off-by: Alex Gronskiy <agronskiy@nvidia.com>

fsiino-nvidia added 16 commits August 12, 2026 05:45

Fix eval container profiling dependency

693e911

Signed-off-by: Frankie Siino <fsiino@nvidia.com>

Harden external vllm jobs

e367f34

Signed-off-by: Frankie Siino <fsiino@nvidia.com>

Init Qwen config

7c5ecb9

Signed-off-by: Frankie Siino <fsiino@nvidia.com>

Fix concurrent Tau2 data setup

dbd666d

Signed-off-by: Frankie Siino <fsiino@nvidia.com>

Fail fast during Super 3.5 eval

900d181

Signed-off-by: Frankie Siino <fsiino@nvidia.com>

Make Super 3.5 vllm launcher portable

9df2268

Signed-off-by: Frankie Siino <fsiino@nvidia.com>

Add opt-in eval concurrency and safe node exclusions

c11be37

Pass num_samples_in_parallel only when explicitly configured. Validate the value and filter stale exclusions against Slurm's current node inventory before submission. Signed-off-by: Frankie Siino <fsiino@nvidia.com>

Enable Qwen3.5 CUDA graphs with BF16 KV cache

a728408

Remove FP8 KV caching and eager execution while retaining the 256 sequence and capture limits. Bound each request to four images and one video. Signed-off-by: Frankie Siino <fsiino@nvidia.com>

Add a vLLM decode metrics summarizer

c4cea5b

Parse combined Slurm logs from coupled or independent decode deployments and report synchronized aggregate throughput, output tokens per second per request, waiting requests, and GPU KV-cache usage. Signed-off-by: Frankie Siino <fsiino@nvidia.com>

Configure role-specific Qwen serving and optional MTP

42812b4

Move topology and batching arguments into the prefill and decode configs, raise max sequences to 512, and optionally enable MTP on both roles. Signed-off-by: Frankie Siino <fsiino@nvidia.com>

fsiino-nvidia added 6 commits August 19, 2026 17:41

Init inkling small config

7b8433f

Signed-off-by: Frankie Siino <fsiino@nvidia.com>

Handle malformed Tau2 tool calls without aborting evals

495dc92

Signed-off-by: Frankie Siino <fsiino@nvidia.com>

Tune Inkling Small prefill and MTP settings

c1f3def

Signed-off-by: Frankie Siino <fsiino@nvidia.com>

Merge remote-tracking branch 'origin/main' into fsiino/super35-vllm-m…

ef1dd91

…odel-tuning Signed-off-by: Frankie Siino <fsiino@nvidia.com>

Fix eval container dependencies

c685f1f

Signed-off-by: Frankie Siino <fsiino@nvidia.com>

Remove summarize vllm metrics script

18b1a74

Signed-off-by: Frankie Siino <fsiino@nvidia.com>

fsiino-nvidia marked this pull request as ready for review August 20, 2026 16:45

fsiino-nvidia changed the title ~~Super 3.5 vllm model tuning~~ Super 3.5 vllm model tuning (Qwen3.5-122B-A10B, Inkling-Small) Aug 20, 2026

fsiino-nvidia added 2 commits August 20, 2026 18:29

Fix Tau2 sanity test data initialization

832d40b

Signed-off-by: Frankie Siino <fsiino@nvidia.com>

Merge remote-tracking branch 'origin/main' into fsiino/super35-vllm-m…

8a767fe

…odel-tuning Signed-off-by: Frankie Siino <fsiino@nvidia.com> # Conflicts: # benchmarks/nemotron_3.5_super/sbatch_external_vllm.sh

fsiino-nvidia requested review from bxyu-nvidia and jkyi-nvidia August 20, 2026 19:44

junkeun-nexusflow suggested changes Aug 20, 2026

View reviewed changes

Comment thread responses_api_agents/tau2/app.py

Comment thread responses_api_agents/tau2/app.py Outdated

fsiino-nvidia added 3 commits August 20, 2026 22:40

fix(tau2): make malformed tool-call retries opt-in

5852530

Default to zero retries after a malformed tool response and expose malformed_tool_call_max_retries for explicit overrides. Signed-off-by: Frankie Siino <fsiino@nvidia.com>

Ignore dummy secret

0124c76

Signed-off-by: Frankie Siino <fsiino@nvidia.com>

lint

7c0dafe

Signed-off-by: Frankie Siino <fsiino@nvidia.com>

fsiino-nvidia mentioned this pull request Aug 21, 2026

Ultra 3 vllm tuning #2645

Open

Lint again

2ca1aa8

Signed-off-by: Frankie Siino <fsiino@nvidia.com>

github-actions Bot added the sla:review-overdue Review response is over the one-business-day SLA label Aug 21, 2026

github-actions Bot mentioned this pull request Aug 21, 2026

[Tracker] Pull request handoff SLA #2067

Open

jkyi-nvidia previously approved these changes Aug 24, 2026

View reviewed changes

bxyu-nvidia added 2 commits August 26, 2026 09:51

revert

7a3d659

Signed-off-by: Brian Yu <bxyu@nvidia.com>

Merge branch 'main' of https://github.com/NVIDIA-NeMo/Gym into fsiino…

4eb599a

…/super35-vllm-model-tuning Signed-off-by: Brian Yu <bxyu@nvidia.com>

bxyu-nvidia dismissed jkyi-nvidia’s stale review via 4eb599a August 26, 2026 16:52

bxyu-nvidia approved these changes Aug 26, 2026

View reviewed changes

copy-pr-bot Bot temporarily deployed to public August 26, 2026 17:06 Inactive

copy-pr-bot Bot temporarily deployed to public August 26, 2026 17:07 Inactive

copy-pr-bot Bot temporarily deployed to public August 26, 2026 17:08 Inactive

jkyi-nvidia enabled auto-merge (squash) August 26, 2026 17:11

bxyu-nvidia merged commit b793621 into main Aug 26, 2026
28 checks passed

bxyu-nvidia deleted the fsiino/super35-vllm-model-tuning branch August 26, 2026 17:12

agronskiy mentioned this pull request Sep 1, 2026

Merge main into gdpval-devel #2923

Open

Navigation Menu

Uh oh!

Super 3.5 vllm model tuning (Qwen3.5-122B-A10B, Inkling-Small) - #2599

Super 3.5 vllm model tuning (Qwen3.5-122B-A10B, Inkling-Small)#2599
bxyu-nvidia merged 30 commits into
mainfrom
fsiino/super35-vllm-model-tuning

fsiino-nvidia commented Aug 18, 2026 •

edited

Loading

Uh oh!

copy-pr-bot Bot commented Aug 18, 2026

Uh oh!

Uh oh!

Uh oh!

bxyu-nvidia commented Aug 26, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

fsiino-nvidia commented Aug 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Shared infrastructure and benchmark reliability

Qwen3.5-122B-A10B

Model-specific changes

How to run

Validation

Inkling Small

Model-specific changes

Container

How to run

Validation

Uh oh!

copy-pr-bot Bot commented Aug 18, 2026

Uh oh!

Uh oh!

Uh oh!

bxyu-nvidia commented Aug 26, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

fsiino-nvidia commented Aug 18, 2026 •

edited

Loading