Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
308 commits
Select commit Hold shift + click to select a range
76c4374
fix(router): restore per-worker load gauges on the embedded partition
ishandhanani Sep 10, 2026
e5fe3f0
fix(router): restore router queue metrics and the prefill overlap obs…
ishandhanani Sep 10, 2026
ca0a0c6
fix(router): route without affinity when the session table is full
ishandhanani Sep 10, 2026
9493e06
fix(router): report remote indexer outages as not-ready
ishandhanani Sep 10, 2026
17c73cc
fix(epp): surface selection client errors as 400
ishandhanani Sep 10, 2026
20c307d
chore(router): gate affinity test hooks and drop dead re-exports
ishandhanani Sep 10, 2026
f81f949
fix(router): reject cache_salt on tiered indexer queries and skip no-…
ishandhanani Sep 10, 2026
7741152
fix(router): reject unsupported data-parallel shapes and correct stan…
ishandhanani Sep 10, 2026
960500e
fix(router): free core bookings the caller never consumes
ishandhanani Sep 10, 2026
3003cb8
fix(router): hold the rebound session lease on concurrent affinity fa…
ishandhanani Sep 10, 2026
92ea625
fix(router): re-register KV listeners a cancelled worker update removed
ishandhanani Sep 10, 2026
66411bc
fix(test): pin CPU platform for vLLM LoRA argument tests (#14607)
xianlubird Sep 10, 2026
f97a17a
docs: refresh community events
dynamo-ops Sep 10, 2026
47f05b2
build: refresh bundled runtime-image dependencies over the base copie…
dagil-nvidia Sep 10, 2026
7dda647
chore(deps): bump dynamo-renderer to 5.1.2 (#14595)
yankay Sep 10, 2026
a3ab180
refactor(operator): remove legacy Kustomize deployment path (#14252)
carlory Sep 10, 2026
64623e7
fix(planner): clamp KalmanPredictor raw-mode forecasts at zero (#14497)
aylonsk Sep 10, 2026
537e6e5
fix(planner): make load-predictor pre-sweep scoring honest about fail…
aylonsk Sep 10, 2026
061385a
fix(ci): test standalone slot_tracker in rust-tests job (#14421)
flpanbin Sep 10, 2026
fde9480
fix(sglang): enable memory saver before snapshot launch (#14555)
galletas1712 Sep 10, 2026
757b944
feat(metrics): export metrics over OTLP from a typed engine contract …
nnshah1 Sep 10, 2026
b4a81a5
fix(router): purge index blocks for a rank whose listener is already …
ishandhanani Sep 10, 2026
1fa623b
fix(recipes): enable Kimi K3 metadata self-hosting (#14617)
milesial Sep 10, 2026
629bc45
fix(runtime): share the process runtime with model fetching (#14534)
diffractiongoo Sep 10, 2026
ea39f80
test(sglang): guard against sending a session id SGLang never created…
glamr-agent Sep 10, 2026
e2e4e61
feat(router): guard core bookings with the lifecycle lease and claim …
ishandhanani Sep 10, 2026
8e9a96f
fix(runtime): reject oversized TCP request frames before send (#14110)
ZenAlexa Sep 10, 2026
fe816fc
feat(router): share the session affinity commit and configure its mode
ishandhanani Sep 10, 2026
4581db2
fix(epp): return 431 for oversized metadata headers and carry typed M…
flpanbin Sep 10, 2026
9740a38
refactor(router): route every selection through one core operation
ishandhanani Sep 10, 2026
e0b0c43
test(trtllm): cover speculative decoding in CI (#14339)
nv-tusharma Sep 10, 2026
9542b02
build(grove): bump dependency to v0.1.0-alpha.13 (#14553)
julienmancuso Sep 10, 2026
79ad15b
fix(vllm): start the omni stage router on a host with no accelerator …
glamr-agent Sep 10, 2026
012b0a3
test(sglang): cover disaggregated n=2 HTTP error (#14106)
yankay Sep 10, 2026
fa46ffb
docs(observability): stop promising that a missing S3 bucket fails st…
glamr-agent Sep 10, 2026
5d66517
feat(discovery): log each model alias registration for every worker r…
glamr-agent Sep 10, 2026
d6b1c8f
perf(router): trim the core hint and readiness hot paths and carry th…
ishandhanani Sep 10, 2026
93df3d3
fix(kv-router): restore server-selected buffer-first recovery (#14627)
PeaBrane Sep 10, 2026
384d7fd
chore: merge Triton/protobuf compatibility update
dagil-nvidia Sep 10, 2026
712d21c
build(deps): upgrade AIPerf to 0.12.0 (#14492)
dagil-nvidia Sep 10, 2026
dc79a5a
fix(discovery): restore first-wins MDC admission (#14632)
PeaBrane Sep 10, 2026
82a57fe
docs(epp): stop pinning vLLM on-ramp version (#14481)
JulienDarve Sep 10, 2026
6616406
feat(router): wire the frontend to the shared selection core without …
ishandhanani Sep 10, 2026
b5bd1a1
fix(operator): validate durable metadata on terminating DGD updates (…
ayaangazali Sep 10, 2026
6862e76
fix: shorten vLLM worker k8s service names for Grove 45-char limit (#…
Jont828 Sep 10, 2026
0873e6c
feat(skills): find-serving-recipe - recipe hunting with provenance ga…
BenHamm Sep 10, 2026
f2a30ed
fix(container): stop the vLLM sidecar failing to start on a missing v…
glamr-agent Sep 10, 2026
65e2061
docs(recipes): add upstream vLLM image disclaimer to Qwen3.8-Flash-Ne…
ynpandey-nv Sep 10, 2026
ce5d378
fix(frontend): honor explicit SGLang skip_special_tokens (#14602)
bzsuni Sep 10, 2026
50485c0
fix(llm): suppress truncated structured tool calls (#13792)
keivenchang Sep 10, 2026
c670214
refactor(router): route the frontend through the shared selection core
ishandhanani Sep 10, 2026
1172f06
refactor(router): commit session affinity through the shared table
ishandhanani Sep 11, 2026
fabac4b
refactor(router): split the selection core into one module per concern
ishandhanani Sep 11, 2026
77c6ff1
fix(vllm-omni): validate client-supplied image size and base64 audio …
nnshah1 Sep 11, 2026
9c4b334
chore(router): delete the selection paths the core cutover left unused
ishandhanani Sep 11, 2026
89785a3
refactor(router): start replica ingress after the lease manager is in…
ishandhanani Sep 11, 2026
38bb25d
fix(trtllm): stop a configured --context-length being silently discar…
glamr-agent Sep 11, 2026
ac4599e
feat: port the networking kustomization to v1beta1 (#14142)
yunzhoul-nv Sep 11, 2026
25aea16
fix(router): hold the routed selection future once on the cleanup path
ishandhanani Sep 11, 2026
58f1e01
feat(gms): enable gms xpu phase 2 (#12484)
zxue2 Sep 11, 2026
6b78d18
fix(kv-router): prevent lookup repair from restoring removed entries …
jthomson04 Sep 11, 2026
db46c14
feat(llm): add FP16/BF16 variants to tensor protocol (#14150)
Vinya567 Sep 11, 2026
2d975ac
feat(vllm): improve FPM self-benchmark fidelity with real-KV seeding …
liyuanzhe1991 Sep 11, 2026
a1f84a3
docs: refresh community events
dynamo-ops Sep 11, 2026
0852047
fix(anthropic): honor disable_parallel_tool_use (#14605)
xianlubird Sep 11, 2026
2ab0c9b
docs: refresh community events
dynamo-ops Sep 11, 2026
fccaf24
fix(operator): restrict multinode component types (#14533)
sttts Sep 11, 2026
dd7c211
docs(fern): settle Enterprise Support naming and pair the landing CTA…
dagil-nvidia Sep 11, 2026
7e2716e
fix(ci): name test artifacts from the test_type input, not GITHUB_ENV…
nv-tusharma Sep 11, 2026
a44f267
fix(request-trace): await final gzip flush on sink shutdown (#13298)
glamr-agent Sep 11, 2026
744f581
feat(skills): report-skillpack-issue - defect telemetry from running …
BenHamm Sep 11, 2026
c255b9a
perf(vllm): isolate sidecar request shutdown waiters (#14726)
jthomson04 Sep 11, 2026
f91ec9e
fix(operator): stop a profiling job hanging forever when the sidecar …
glamr-agent Sep 11, 2026
64419a6
ci: remove H100 KVBM post-merge job (#14626)
nv-tusharma Sep 11, 2026
3f2e2f4
feat(router): retrieve optional session_id field from vLLM engine KV …
karen-sy Sep 11, 2026
f3e600f
fix(vllm): stop the encode worker's embedding cache growing without l…
glamr-agent Sep 11, 2026
f459cda
fix(frontend): shut down runtime on server startup failure (#13415)
MatejKosec Sep 11, 2026
b3d4176
fix(profiler): stop THOROUGH profiling crashing on its first candidat…
glamr-agent Sep 11, 2026
da8901c
docs(benchmarks): correct multi-turn trace semantics (#14184)
cpakkamisaac-sae Sep 11, 2026
d8dfcd1
fix(router): forward AIPerf trace hints (#14165)
cpakkamisaac-sae Sep 11, 2026
71480f9
fix(planner): stop a metrics outage leaving fewer replicas than min_e…
glamr-agent Sep 11, 2026
f1683cc
fix(lora): skip base-weight downloads during card registration (#11164)
AmeenP Sep 11, 2026
361eb87
feat(operator): add explicit component role replicas (#14477)
sttts Sep 11, 2026
55fae8f
fix(ci): pass framework through shared-test pytest-local calls (#14688)
nv-rinig Sep 11, 2026
f338910
fix(backend-common): track worker readiness on the runtime health rou…
glamr-agent Sep 11, 2026
5ef49b4
fix(helm): use valid KAI staleness duration (#14722)
julienmancuso Sep 11, 2026
6c66d4b
fix(kv-router): make standalone indexer cleanup safe (#14652)
PeaBrane Sep 11, 2026
7cc262f
fix(llm): withhold hidden stop-sequence prefixes split across tokens …
Efeselimsurekli Sep 11, 2026
901b95e
fix(llm): bound speculative prefill warmup lifetime (#13438)
glamr-agent Sep 11, 2026
e22fab5
fix(router): apply the review round on the selection core
ishandhanani Sep 11, 2026
0000c56
fix(vllm-sidecar): preserve buffered empty text deltas (#14699)
alec-flowers Sep 11, 2026
a86fcea
fix(vllm): project generate request controls (#14398)
biswapanda Sep 11, 2026
07952cb
feat(operator): add gated role PodTemplate API (#14544)
sttts Sep 11, 2026
d7d1b59
fix(sglang): stop co-located ranks crashing on a shared NIXL exporter…
glamr-agent Sep 11, 2026
317eb30
chore(router): clear the small self-review findings
ishandhanani Sep 11, 2026
bed4d0b
fix(responses): reject unsupported tools before dispatch (#14440)
git-jxj Sep 11, 2026
062a495
fix(benchmarks): isolate router benchmark imports (#14745)
jthomson04 Sep 11, 2026
d677960
fix(planner): weight frontend histogram averages by observations (#14…
bzsuni Sep 11, 2026
0027a8e
feat(triton): Deliver dynamo.triton Functionality (#13774)
whoisj Sep 11, 2026
ea037f3
test(router): pin the booking lifecycle invariants
ishandhanani Sep 11, 2026
c95307e
fix(router): gate the LoRA narrowing helper behind standalone-selection
ishandhanani Sep 11, 2026
1f91e53
refactor(router): hand bookings to hosts as one BookingHandle
ishandhanani Sep 11, 2026
bbd8412
refactor(router): wire the request lease manager directly as the part…
ishandhanani Sep 11, 2026
45383b4
Merge branch 'main' into idhanani/dyn-14310-selection-core-full
ishandhanani Sep 12, 2026
4b72f79
feat(recipes): add DeepSeek-V4.1-Flash SGLang GB200 agg and disagg re…
sungsooha Sep 12, 2026
4a3943c
docs: refresh community events
dynamo-ops Sep 12, 2026
5c2eb04
chore(router): cleanup round one, drop test-only shims and duplicate …
ishandhanani Sep 12, 2026
63f7ac4
chore(router): cleanup round two, one result shape and fewer wrappers
ishandhanani Sep 12, 2026
99f8574
chore(router): cleanup round three, verbatim moves and shorter tests
ishandhanani Sep 12, 2026
37cfba9
test(router): restore two pins the cutover dropped
ishandhanani Sep 12, 2026
7d637ba
ci: refresh the kvbm bindings lockfile and fix a moved docs link
ishandhanani Sep 13, 2026
29a05b4
chore(router): finish the deferred mechanical cleanups
ishandhanani Sep 13, 2026
7b2f6c3
fix(router): track hint capability per partition and clear stale rank…
ishandhanani Sep 13, 2026
9a18ffb
fix(router): reconcile a catalog snapshot as one batch, retry a faile…
ishandhanani Sep 13, 2026
3cecd2a
fix(router): read hint capability from the published partition snapshot
ishandhanani Sep 13, 2026
29b2b9c
fix(router): construct the worker-selection policy once per partition
ishandhanani Sep 13, 2026
4cc6f49
fix(router): publish a completed partition move before the batch cont…
ishandhanani Sep 13, 2026
9e1e92a
fix(router): keep a lease when a session commit lands behind a compet…
ishandhanani Sep 13, 2026
2dfc529
fix(router): derive the embedded partition key in one place
ishandhanani Sep 13, 2026
d4be244
fix(router): uncount a joined lease released before its initializer c…
ishandhanani Sep 13, 2026
02165d7
feat(router): instrument the active-sequence ingress funnel
ishandhanani Sep 13, 2026
27c03ee
fix(router): label ingress metrics by worker type and pin the queued …
ishandhanani Sep 13, 2026
13a8523
fix(operator): decouple snapshot compatibility hash (#14724)
julienmancuso Sep 13, 2026
ed64b7d
docs: refresh community events
dynamo-ops Sep 13, 2026
461fff6
fix(epp): address EPP-specific bugs and config layering issues in sel…
tmonty12 Sep 13, 2026
41d1e68
fix(router): address PR review comments on session affinity
tmonty12 Sep 13, 2026
3b660db
refactor(router): unify session affinity config into KvRouterConfig
tmonty12 Sep 13, 2026
6b6d5c2
docs(router): restore canonical doc comment on session affinity fields
tmonty12 Sep 13, 2026
f2becb5
fix(ci): pass framework to Triton test workflows (#14760)
bzsuni Sep 14, 2026
11b85b9
docs(rl): repoint the refit example links at the split ModelExpress d…
ayaangazali Sep 14, 2026
e7bbe8f
docs: refresh community events
dynamo-ops Sep 14, 2026
3fe293a
fix(ci): recover deploy tests from vcluster tunnel drops (#14246)
xianlubird Sep 14, 2026
04f803f
feat(frontend): support mutual TLS for HTTP server (#14173)
xianlubird Sep 14, 2026
df09fa6
fix(ci): install networkx in test images (#14771)
bzsuni Sep 14, 2026
702f532
doc: Follow up DOC fixes to go epp deprecation fixes [DYNO-102] (#14263)
atchernych Sep 14, 2026
4dd64df
refactor(frontend): couple Anthropic errors with metric classificatio…
nihalmaddala Sep 14, 2026
e57f3bc
docs(fern): remove the Enterprise Support hero button (#14805)
dagil-nvidia Sep 14, 2026
f682011
fix(container): stop every vLLM sidecar inference request failing in …
glamr-agent Sep 14, 2026
250213a
perf(sglang): isolate sidecar request shutdown waiters (#14727)
jthomson04 Sep 14, 2026
2d8632f
chore: bump trtllm to v1.3.0rc26 (#14535)
dynamo-ops Sep 14, 2026
5e4a179
docs(epp): document the EPP RBAC boundary between runtime and standal…
tmonty12 Sep 14, 2026
ea7ab6e
fix(epp): address EPP-specific bugs and config layering issues in sel…
ishandhanani Sep 14, 2026
b2d456f
feat(sglang): forward session_id and parent_session_id to sglang engi…
nvjullin Sep 14, 2026
4e88998
feat(ext-proc): standalone EPP honors cache namespace/salt consistent…
flpanbin Sep 14, 2026
ebd1e63
fix(recipes): tolerate vLLM JIT stalls on DeepSeek-V4-Pro-0813 GB200 …
shugandhi Sep 14, 2026
cf6d288
feat(frontend): add opt-in jemalloc preload (#14678)
jthomson04 Sep 14, 2026
038d6f2
fix(frontend): preserve SGLang worker routing hints (#14696)
bzsuni Sep 14, 2026
5d9e2a6
feat(experimental): add unary endpoint utilities (#12920)
furionw Sep 14, 2026
5e7f31b
build(deps): bump google.golang.org/grpc from 1.83.1 to 1.83.2 in /de…
dependabot[bot] Sep 14, 2026
c0b88ba
fix(request-trace): stop losing accepted trace records when the front…
glamr-agent Sep 14, 2026
0ab7d48
fix(runtime): return 400, not 500, when a backend refuses a request i…
glamr-agent Sep 14, 2026
28e2a55
ci(datadog): put the venv's pip on PATH so ddtrace installs (#14233)
nv-nmailhot Sep 14, 2026
7a3d29a
perf(llm): share prompt tokens across request clones (#14701)
zhaohuizh Sep 14, 2026
8741e42
fix(simulation): upgrade AISimulate to 0.12.0.dev1 (#14541)
tedzhouhk Sep 14, 2026
ae60a70
test(gpu-memory-service): stop the flock race test failing when the r…
glamr-agent Sep 14, 2026
9f342d5
docs(review): add frontend and runtime codeowner prompts (#14633)
MatejKosec Sep 14, 2026
cf944ae
build(vllm): consume upstream vllm-proto crate (#14738)
JulienDarve Sep 14, 2026
77554da
fix(sidecar): allow 30 minutes for gRPC startup (#14825)
alec-flowers Sep 15, 2026
9062a66
fix(llm): align stop-sequence limits and return 400 (#14462)
flpanbin Sep 15, 2026
b113cea
fix(planner): publish scaling decisions atomically (#14809)
PeaBrane Sep 15, 2026
7c7aed1
fix(http): drain SSE streams after graceful generation stops (#14645)
jacksonrnewhouse Sep 15, 2026
90c8842
feat(http): gate streaming responses on the first backend event (#14662)
jorgeantonio21 Sep 15, 2026
39da83d
Merge origin/main into idhanani/dyn-14310-selection-core-full
ishandhanani Sep 15, 2026
d481e51
fix(metrics): restore documented histogram bucket env var names (#14514)
jh-nv Sep 15, 2026
b6a5b70
fix(container): use root cache for user overrides with a group (#14820)
bojiang-li Sep 15, 2026
bc17acf
refactor(errors): unify semantic error classification (#14395)
biswapanda Sep 15, 2026
c52945c
docs: refresh community events
dynamo-ops Sep 15, 2026
055e989
feat(llm): carry KServe requested outputs on tensor requests (#14618)
yinggeh Sep 15, 2026
5292fc9
test(operator): align DGDR goldens with current AIC backend slot (#14…
xianlubird Sep 15, 2026
95cadce
fix(vllm): support RL discovery with released vLLM 0.28 for dynamo re…
biswapanda Sep 15, 2026
fd09b4b
revert(router): back out EPP affinity follow-up
ishandhanani Sep 15, 2026
71335cf
fix(router): validate session affinity DP ranks during failover
ishandhanani Sep 15, 2026
54c2076
perf(router): keep LoRA candidate filtering linear
ishandhanani Sep 15, 2026
800070d
perf(router): move the committed booking request ID
ishandhanani Sep 15, 2026
928321e
docs(router): clarify default lookup retention
ishandhanani Sep 15, 2026
0a477b1
fix(router): preserve affinity ownership across concurrent failover
ishandhanani Sep 15, 2026
0049ca3
perf(router): omit unused metadata from frontend leases
ishandhanani Sep 15, 2026
a6fa367
feat(recipes): add A.X-K2 NVFP4 B200 serving and benchmark recipes (#…
vanshilshah97 Sep 15, 2026
c9a7299
test(benchmarks): cover trace command regression (#14861)
jthomson04 Sep 15, 2026
dbf505b
fix(operator): preserve alpha service metadata precedence (#14855)
julienmancuso Sep 15, 2026
db6774e
build(deps): bump AISimulate to 0.12.0.dev2 (#14831)
saturley-hall Sep 15, 2026
165fca7
fix(vllm): stop the embedding worker pool crashing at startup (#14449)
glamr-agent Sep 15, 2026
0949758
feat(rl): add Slime external rollout example (#12856)
Aphoh Sep 15, 2026
c71f3d3
refactor(router): defer standalone HTTP indexer integration
ishandhanani Sep 15, 2026
477d31d
feat(triton): drop FP16/BF16 input guard now that the tensor protocol…
Vinya567 Sep 15, 2026
f1cb3ae
refactor(http): consolidate to a single aiohttp backend; remove httpx…
nnshah1 Sep 15, 2026
5c76ed4
fix(frontend): label aliased model metrics with the primary name (#14…
glamr-agent Sep 15, 2026
4c89e3d
fix(gms): stop V1 snapshot save and load crashing at startup (#14797)
glamr-agent Sep 15, 2026
13ca6fd
test(router): isolate request lifecycle counters (#14875)
jthomson04 Sep 15, 2026
90557bb
test(bindings): update selection service lifecycle constructor call
ishandhanani Sep 15, 2026
be9623d
fix(container): make TRT-LLM runtime Triton JIT independent of image …
nv-yna Sep 15, 2026
b43c373
fix(RL): stop RL worker discovery returning an empty list on Kubernet…
glamr-agent Sep 15, 2026
7163b48
feat(mm-routing): add Nemotron 3 Nano Omni image routing (#14564)
krishung5 Sep 15, 2026
9fcc771
fix(planner): allow scale-down while workers are starting (#14671)
tedzhouhk Sep 15, 2026
395fb8e
feat: KV DC Relay file based source mode (#14807)
Kaonael Sep 16, 2026
3b40f4d
fix(router): preserve frontend accounting before discovery
ishandhanani Sep 16, 2026
9d1859a
fix(router): fence pending reservation claims by identity
ishandhanani Sep 16, 2026
6bef1c4
fix(router): yield during repeated affinity invalidation
ishandhanani Sep 16, 2026
b94be66
feat(sglang): expose cross-encoder reranking through /v1/rerank (#14032)
xianlubird Sep 16, 2026
8309ffb
fix(profiler): explain inaccessible model paths during trust checks (…
tedzhouhk Sep 16, 2026
8c5a737
fix(sglang): sync discovery from native pause state (#13951)
Aphoh Sep 16, 2026
0bd2059
feat(recipes): add Solar Open2 250B NVFP4 aggregated and disaggregate…
snarravula-dl Sep 16, 2026
43b6344
refactor(agents): session_id reader from AgentContext + forward to vL…
karen-sy Sep 16, 2026
a5c242e
fix(discovery): allow served aliases for the same model source (#14857)
jthomson04 Sep 16, 2026
5e21f9c
fix(router): reject unknown explicit worker targets (#14858)
jthomson04 Sep 16, 2026
30e1580
fix(xpu): stabilize XPU test workers (#14539)
VincyZhang Sep 16, 2026
6aeee7f
feat(mm-routing): add Nemotron 3 Nano Omni video routing (#14653)
krishung5 Sep 16, 2026
7a77a5d
fix(sglang): validate diffusion input_reference and bound media fetch…
nnshah1 Sep 16, 2026
883db9b
chore(deps): upgrade fastokens to 0.3.2 (#14798)
jthomson04 Sep 16, 2026
c76d320
fix(vllm): ship codec-free OpenCV for image inputs (#14361)
glamr-agent Sep 16, 2026
e1878f6
Merge main into selection-core branch
ishandhanani Sep 16, 2026
059cae3
docs: refresh community events
dynamo-ops Sep 16, 2026
ef235cf
ci: refresh the compliance baseline in auto-upgrade pipeline (#14206)
nv-anants Sep 16, 2026
d866a9c
feat(triton): honor KServe classification on tensor outputs (#14783)
yinggeh Sep 16, 2026
ccda2d2
docs(rl): stop the verl guide sending readers to a vLLM version it ca…
glamr-agent Sep 16, 2026
e292c76
feat(mocker): publish native KV events from the vLLM gRPC server (#14…
jthomson04 Sep 16, 2026
8f95101
fix(kv-router): release unowned radix branches after eviction (#14878)
jthomson04 Sep 16, 2026
16d6048
fix: show correct backend versions in the install selectors (#13599)
nv-anants Sep 16, 2026
cfada2f
build(vllm): prepare v0.29.0 bump (#14543)
JulienDarve Sep 16, 2026
8950fe1
feat(triton): Update Triton Base Image to 26.08 (#14854)
whoisj Sep 16, 2026
fb7da85
fix(operator): normalize equivalent worker hash inputs (#14721)
bzsuni Sep 16, 2026
adeaae1
test(sglang): exercise NIXL in embedding cache E/PD test (#14795)
pskiran1 Sep 16, 2026
f937ad0
fix(sglang): stop the elastic-EP scale-up worker crash-looping at sta…
glamr-agent Sep 16, 2026
40a853b
fix(responses): honor tool_choice when parsing tool calls from text (…
xianlubird Sep 16, 2026
0d80f5c
Merge branch 'main' into idhanani/dyn-14310-selection-core-full
ishandhanani Sep 16, 2026
5743b3c
ci: accept trusted full-CI request comments (#14868)
MatejKosec Sep 16, 2026
c970209
docs: clarify EPP mode boundary and single-replica Dynamo mode fixes …
atchernych Sep 16, 2026
fd997d6
ci(docs): move the generated-tables determinism gate out of link chec…
dagil-nvidia Sep 16, 2026
61c51c7
ci(docs): generate the Kubernetes API reference at publish time (#14122)
dagil-nvidia Sep 16, 2026
f92f6f6
fix(operator): discover pull secrets for init containers (#14922)
bojiang-li Sep 16, 2026
5296909
fix(sglang): stop an unusable mooncake backend crashing workers after…
glamr-agent Sep 16, 2026
88c7770
fix(sglang): emit prefill handoff before completion in sidecar (#14260)
connorcarpenter15 Sep 16, 2026
0dbd9a8
test(trtllm): enable fault tolerance coverage (#14609)
tanmayv25 Sep 16, 2026
9922482
fix(frontend): evict async tokenizer executors when the tokenizer is …
panpan0000 Sep 16, 2026
18a75aa
fix(llm): report KServe datatypes by their wire names, not protobuf v…
Vinya567 Sep 16, 2026
8827c61
fix(router): address selection core review feedback
ishandhanani Sep 16, 2026
633776f
docs(mm-routing): document video KV routing (#14958)
krishung5 Sep 16, 2026
a1c9d4c
fix(sidecar): honor worker namespace suffix (#14955)
biswapanda Sep 16, 2026
0cb120e
fix(bindings): drain bridge tasks before interpreter finalization (#1…
glamr-agent Sep 16, 2026
69f4476
fix(discovery): stop a Qwen3-VL worker from serving video with anothe…
glamr-agent Sep 16, 2026
4df3f3a
fix(gms): honor configured timeout during initial weights admission (…
glamr-agent Sep 16, 2026
3335e4b
refactor(router): remove unused indexer lookup wrappers
ishandhanani Sep 16, 2026
3165ff2
Merge main into idhanani/dyn-14310-selection-core-full
ishandhanani Sep 16, 2026
232f881
feat(kv-router): add construction-time indexer delegates (#14945)
ryanolson Sep 17, 2026
e5ef9d7
fix(sglang): support min_tokens on tokenizer-free decode workers (#14…
glamr-agent Sep 17, 2026
adfe2aa
feat(router): add SessionPrefixIndexer for session-block lineage (#13…
karen-sy Sep 17, 2026
3dd603b
fix(vllm): settle kvwarm stages through a per-step round on every att…
Arsene12358 Sep 17, 2026
5eaea24
Merge main into idhanani/dyn-14310-selection-core-full
ishandhanani Sep 17, 2026
c66159b
Merge latest main into idhanani/dyn-14310-selection-core-full
ishandhanani Sep 17, 2026
1b82fe2
perf(router): borrow membership snapshots during reconciliation
ishandhanani Sep 17, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
10 changes: 7 additions & 3 deletions .agents/skills/author-baseline-dgd/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,9 +64,13 @@ prefer aggregated unless the user's SLOs demand otherwise).
workloads; raise TP above `min_tp` only when headroom demands it, recording the replica cost).
2. **Choose topology conservatively**: an aggregated single-node layout unless the user's hardware or SLOs force
otherwise. The baseline's job is to run and measure, not to win; the optimization loop owns improvement.
3. **Scaffold from the nearest recipe**: copy its structure (components, probes, service wiring, image versions
for the chosen backend) and replace model, parallelism, resources, and any hardware-bound fields, naming every
replacement. Never carry a hardware-bound topology, transport, or checkpoint choice across without evidence it
3. **Scaffold from the nearest recipe**: copy its structure (components, probes, service wiring) and replace
model, parallelism, resources, and any hardware-bound fields, naming every replacement. Image versions are NOT
copied blindly: when a recipe dossier snapshot exists (`<EXP_ROOT>/analysis/recipe-dossier/`), read the
scaffold candidate's recorded verdict and failed conditions, and never carry forward an image the dossier
marked mutable, unresolved, stale, or quarantined (Tier 4); choose an image that resolves to an immutable
release tag or digest for the chosen backend and record the resolution in the evidence table. A draft whose
image fails that gate is not ready for confirmation. Never carry a hardware-bound topology, transport, or checkpoint choice across without evidence it
fits the target. A manifest expresses REQUIREMENTS (GPU type, count, memory), never observed cluster state:
do not pin node names or encode which nodes happen to be free, and preserve the recipe's scheduling
MECHANISMS (tolerations, product-label node selectors) while retargeting their VALUES to the contract's
Expand Down
9 changes: 8 additions & 1 deletion .agents/skills/consult-perf-knowledge/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,14 @@ when current evidence shows that family dominates the objective or a broad scan
Follow `agent-docs/guides/knob-tuning/tuning-hierarchy.md`:

1. Classify the exact model and compute memory fit, minimum parallelism, and headroom.
2. Complete the exploration-versus-exploitation calibration and full broad-lever scan above.
2. Complete the exploration-versus-exploitation calibration and full broad-lever scan above. As part of the broad
scan, invoke `find-serving-recipe` once per engagement, reusing the latest snapshot listed in
`EXP_ROOT/analysis/recipe-dossier/index.md` on later iterations when one exists, and record the snapshot's
path and SHA256 in `knowledge-consult.md`, so that every
recipe-shaped candidate under consideration carries a provenance verdict: a `deployable` or `hypothesis` grade
candidate enters the lever shortlist with its dossier entry attached, and a `ceiling-only` candidate may inform
expected-performance headroom but is never shortlisted for deployment. A recipe-sourced candidate goes through
this same selection and the adversarial review like any other hypothesis; it never replaces the baseline.
3. Consider topology first as a hypothesis category per the tuning hierarchy; an inherited layout is a candidate,
not a settled decision.
4. When topology is viable, consider Tier 2 families in default priority order, then finish screening the remaining
Expand Down
248 changes: 248 additions & 0 deletions .agents/skills/find-serving-recipe/SKILL.md

Large diffs are not rendered by default.

188 changes: 188 additions & 0 deletions .agents/skills/report-skillpack-issue/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,188 @@
---
name: report-skillpack-issue
description: Reports a defect in the optimization skillpack itself (the skills under .agents/skills/, the documents under agent-docs/, the role contracts under agents/, and the repository AGENTS.md files) as a GitHub issue on ai-dynamo/dynamo, using the repository's agent-reported issue conventions. Use when a pack rule contradicts another pack rule, a cross-reference points at a file or section that does not exist, an instruction cannot be executed as written, a factual claim about a tool or flag is wrong, or a rule repeatedly fights the observed environment. Do not use for bugs in Dynamo itself, for engagement-specific problems, or to request new features.
license: Apache-2.0
user-invocable: true
metadata:
author: NVIDIA
tags:
- dynamo
- skillpack
- telemetry
- github
---

# Skill: Report a Skillpack Defect

<!--
SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
SPDX-License-Identifier: Apache-2.0
-->

Issues filed against the skillpack are the maintainers' primary field telemetry. A defect you hit and route around
silently gets hit by every other user's agent too. Filing takes two minutes; the report is more valuable than the
workaround.

This skill follows the repository's agent-reported issue conventions
(`.github/ISSUE_TEMPLATE/agent-reported.yml`): the `[AGENT]: ` title prefix, the `agent-reported` label, a
declared agent identity, and at most one issue per session.

## Step 1: Confirm the defect is in the pack, not the environment

Re-read the suspect rule or skill file in full before reporting. Classify the defect:

- `contradiction` — two pack statements cannot both be followed.
- `dead-reference` — a cross-reference targets a file, section, or field that does not exist.
- `unexecutable` — an instruction cannot be carried out as written (missing input, impossible ordering, undefined
term with no declared source).
- `factual-error` — a claim about a tool, flag, default, or format is wrong; verify against the tool's own
source or help output first, and quote that verification in the report.
- `environment-mismatch` — the rule assumes something (paths, permissions, cluster shape) that the target
environment class does not satisfy; report only when the mismatch looks general, not site-specific.
- `missing-coverage` — the pack gives no guidance for a component or situation it plainly should cover.

If the problem disappears on a careful re-read, or is specific to one site's configuration, do not file.

## Step 2: Record your agent identity and the exact pack version

The report must state the driver model and the skills commit it was running (for example
`claude-opus-5, skills @ abc1234`):

```bash
git rev-parse --short HEAD
```

Run that from the repository root. In every command in this skill, replace placeholders BEFORE running
and never leave angle brackets in a shell line: `<...>` parses as redirection and can truncate files.

If the pack was vendored without git history, record the release or image tag it came from. A report that cannot
say which version it observed cannot be triaged.

## Step 3: Sanitize

The issue lands in a public repository. Nothing you emit, in the title, the body, a comment, a search term,
or a version string, may contain: Quote pack text from the public repository freely (not from a locally modified or vendored copy, which may
carry private details); describe your environment only in generic terms (GPU class, backend, harness). When in doubt, leave it out — the rule file,
version, and defect class are usually enough to reproduce.

## Step 4: Check for an existing report

Search titles AND bodies in one bounded, body-inclusive query. A labeled query adds nothing (`--label`
ANDs into the search, so it is a strict subset of the unlabeled one), the default limit of 30 hides real
matches for common keys such as `README.md`, and `--json` returns bodies inline instead of costing one
`gh issue view` per hit. Use the bare filename as the search term (no backticks):

```bash
gh issue list --repo ai-dynamo/dynamo --state all --limit 100 --json number,title,body \
--search 'benchmark-isolation.md in:title,body'
```

Review every hit whose title starts with `[AGENT]: ` or whose body mentions the same file and
defect class, and read each candidate's comments too, because confirmations of an existing defect
are filed as comments (below) and do not appear in title or body search:

```bash
ISSUE_NUMBER=123
gh issue view "$ISSUE_NUMBER" --repo ai-dynamo/dynamo --comments
```

If a matching issue exists, do not file a duplicate: draft the same body (step 5), route it through
step 6 with the target recorded as "comment on #123", and on approval run the command with BOTH
variables assigned in the same shell invocation (nothing persists between harness shell calls):

```bash
ISSUE_NUMBER=123
BODY_FILE=$EXP_ROOT/analysis/skillpack-issue-drafts/001-perf-analyzer-body.md
gh issue comment "$ISSUE_NUMBER" --repo ai-dynamo/dynamo --body-file "$BODY_FILE"
```

File at most one new issue per session; if the session surfaced several defects, put the most
impactful one in the issue and list the rest briefly in its body.

## Step 5: Draft the issue

Title: `[AGENT]: <file path relative to repo root>: <one-line defect>`.

Write the body with a file-writing tool, never by echoing it through a shell, to a run-scoped path that
names the invoking role, because several roles can hit defects in the same engagement window: inside an
engagement `<EXP_ROOT>/analysis/skillpack-issue-drafts/NNN-<role>-body.md` (NNN increasing), otherwise
`./skillpack-issue-drafts/NNN-<role>-body.md` under the current working directory. Never use a shared
fixed path such as `/tmp`, where another role's write can replace an approved body before it is
submitted. Delete the draft file once its issue or comment is filed, so a later invocation cannot inherit
a stale body. Follow the
agent-reported template's structure:

```markdown
### Agent identity

<driver model>, skills @ <commit or tag>

### What the instructions said vs what you verified

Defect class: <contradiction | dead-reference | unexecutable | factual-error | environment-mismatch | missing-coverage>
Location: <file path and the section heading or quoted sentence>

<exact quote(s) of what the pack says; for a contradiction, quote both sides>

<the generic, sanitized observation that contradicts it, how it was verified, and what the agent could not do,
did wrong, or had to route around>

### Suggested correction

<the wording you would have needed; omit the section if unsure>
```

## Step 6: Who files, and how

Filing a public issue or comment is an external side effect that needs the operator's approval of the
exact drafted text. Two cases:

- **Dispatched sub-role** (interviewer, deployer, analyzer, generator, challenger): you do not talk to
the operator, so you never run `gh issue create` or `gh issue comment`. Record the draft in
`<EXP_ROOT>/analysis/skillpack-defects.md` (dated section, defect class, location, draft title,
draft body path, proposed target: new issue or comment on #N, status `unfiled`) and return that
path to your parent. The top-level agent batches every unfiled draft, presents them to the operator,
and on approval files at most ONE new issue per session (secondary defects go in its body) plus any
approved comments, then updates each draft's status to `filed` with the URL. This is what makes
"one issue per session" hold across roles.
- **Standalone invocation by an operator**: the operator is present; show the complete title and body,
file only on approval.

Whoever files: assign the title from a quoted heredoc in the same shell invocation as the command
(harness shells do not persist variables between calls), and pass the approved body file unchanged.
Quoting `"$ISSUE_TITLE"` at the call site protects nothing on its own: backticks and `$()` in a drafted
title are interpreted when the variable is *assigned*, and the pack's own style backticks every flag.
Do not edit the body between approval and filing; file the same path the operator saw.

```bash
BODY_FILE=$EXP_ROOT/analysis/skillpack-issue-drafts/001-perf-analyzer-body.md
ISSUE_TITLE=$(cat <<'EOF'
[AGENT]: agent-docs/rules/benchmarking/benchmark-isolation.md: one-line defect
EOF
)
gh issue create --repo ai-dynamo/dynamo --title "$ISSUE_TITLE" --body-file "$BODY_FILE" --label agent-reported
```

Labels behave two ways depending on the client path: the REST API silently drops labels the
reporter may not set ("Only users with push access can set labels for new issues. Labels are silently
dropped otherwise."), while `gh` may instead fail before creating the issue when it cannot resolve a
label. Handle both: if the command fails on the label, retry without `--label`; if it succeeds,
verify:

```bash
ISSUE_NUMBER=123
gh issue view "$ISSUE_NUMBER" --repo ai-dynamo/dynamo --json labels --jq '[.labels[].name]'
```

If `agent-reported` is absent, tell the operator so a maintainer can add it; maintainers also triage
by the `[AGENT]: ` title prefix, and the step 4 search covers unlabeled reports.

## Fallback: no GitHub access or no approval

Write the complete draft to an append-only artifact the invoking role owns and tell the operator
where it is. Inside an optimization engagement that is `<EXP_ROOT>/analysis/skillpack-defects.md`
(one dated section per draft, shaped like an `asks.jsonl` entry: defect class, location, draft
title, draft body, status `unfiled`); never write under `final/`, whose three artifacts belong to
`hypothesis-generator` and are written only at stop-request time. Outside an engagement, write
`skillpack-issue-draft-<n>.md` at the root of the current working directory. A
drafted-but-unfiled report is still telemetry; a silent workaround is not.
6 changes: 4 additions & 2 deletions .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -25,13 +25,15 @@
**/*.libtorch_model_store/*
**/.git
**/.github
# Planner test images exercise the version-stamping script without copying the
# rest of the GitHub metadata into every Docker build context.
# Test images exercise these scripts without copying the rest of the GitHub
# metadata into every Docker build context.
!.github
.github/*
!.github/scripts
.github/scripts/*
!.github/scripts/apply_dev_version.py
!.github/scripts/retry_kubectl.sh
!.github/scripts/collect_vcluster_diagnostics.sh
**/*backup*/
.dockerignore
**/target/*
Expand Down
Loading
Loading