Skip to content

fix(profiler): explain inaccessible model paths during trust checks - #14860

Merged
tedzhouhk merged 7 commits into
mainfrom
hzhou/dyn-4443
Sep 16, 2026
Merged

tedzhouhk merged 7 commits into
mainfrom
hzhou/dyn-4443

Conversation

@tedzhouhk

@tedzhouhk tedzhouhk commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor

Summary

When a local model path cannot be inspected (for example, a protected symlink on a model-cache PVC), DGD materialization currently leaks an unhandled filesystem error. Report an actionable error with the original cause, including directory/symlink ownership checks and the option to use the actual snapshot path in modelCache.pvcModelPath.

Preserve config-inspection errors in resolve_model_path() before it can fall back to a Hub model ID. Missing configs and non-directory paths retain the existing fallback.

Use one explicit stat() so filesystem errors remain distinguishable from missing/non-directory references. Preserve explicit worker trust flags and the existing remote-model trust policy. Correct the auto-map warning/docstring so inconclusive detection does not claim that --trust-remote-code was injected.

Validation

  • Profiler protocol, DGD materialization, and model-path resolution suites: 131 passed, including resolution through materialization and persistence of inaccessible-PVC errors in profiler_status.yaml. The new inaccessible-config regressions fail before the resolver fix and pass afterward.
  • Actual run_profile with a real chmod(000) parent directory: preserves the PermissionError cause and complete actionable status, without mocked filesystem calls.
  • pre-commit run --all-files and git diff --check: passed.
  • QA cluster reproduction with stock 1.5.0-rc.7: protected alias fails; the direct snapshot path completes profiling/spec generation.
  • Repeated both RAPID DGDR cases with the two updated functions overlaid onto rc.7 through a ConfigMap: alias reports the new actionable error; snapshot still completes with profiler exit 0 and DGDR Ready / SpecGenerated. This was a source-overlay check, not a rebuilt image or worker inference test.

The inaccessible alias intentionally remains an error; fix its ownership or use the snapshot path. No operator change is required.

Summary by CodeRabbit

  • Bug Fixes
    • Improved validation for local model paths when determining whether implicit remote-code trust is allowed.
    • Invalid, missing, or non-directory model paths are now handled consistently, with clearer runtime errors and diagnostic logging.
    • Explicit --trust-remote-code settings continue to work without being overridden or modified.
    • Added coverage for symlinked paths and other inaccessible or invalid model-path scenarios.

Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
@tedzhouhk
tedzhouhk marked this pull request as ready for review September 15, 2026 17:05
@tedzhouhk
tedzhouhk requested a review from a team as a code owner September 15, 2026 17:05
@coderabbitai

coderabbitai Bot commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 187cab32-1c0c-4aa4-a616-c16c8b93b9ae

📥 Commits

Reviewing files that changed from the base of the PR and between 5292fc9 and 1d93100.

📒 Files selected for processing (2)
  • components/src/dynamo/profiler/tests/unit/test_profiler_protocol.py
  • components/src/dynamo/profiler/utils/model_info.py

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.


Walkthrough

The change updates model path inspection for implicit remote-code trust. It distinguishes valid directories from inaccessible paths and adds tests for symlinks, invalid paths, wrapped errors, configuration preservation, and diagnostic logging.

Changes

Model path trust validation

Layer / File(s) Summary
Model path inspection and trust-policy messaging
components/src/dynamo/profiler/utils/model_info.py
model_ref_allows_implicit_trust_remote_code uses stat() and S_ISDIR. Missing and non-directory paths return False; other OSError failures raise diagnostic RuntimeError exceptions. Auto-map documentation and logging describe inconclusive trust-policy results.
Trust-policy behavior tests
components/src/dynamo/profiler/tests/unit/test_profiler_protocol.py
Tests cover eligible directories and symlinks, invalid paths, inaccessible paths, explicit trust configuration, preserved input configuration, original error causes, and diagnostic logging.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~25 minutes

Merge Risk: ⚪ Minimal · up to 1d931

No actionable merge-blocking risk remains in the reviewed change.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 57.14% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 7 functions across 2 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check ⚠️ Warning The description explains the change and validation results, but it does not follow the required template. It omits the required Related Issues section and the Where should the reviewer start? section. Add the required Related Issues section and select either the linked-issue option with a real issue number or confirm that no related issue exists. Add a Where should the reviewer start? section with the key files for review. Align the exis…
✅ Passed checks (3 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly and concisely describes the main change: improving diagnostics for inaccessible model paths during profiler trust checks.
Full details: Description check

Resolution

Add the required Related Issues section and select either the linked-issue option with a real issue number or confirm that no related issue exists. Add a Where should the reviewer start? section with the key files for review. Align the existing Summary and Validation content with the template's Overview and Details sections.

  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch

Comment @coderabbitai help to get the list of available commands.

Comment thread components/src/dynamo/profiler/tests/unit/test_profiler_protocol.py Outdated
Comment thread components/src/dynamo/profiler/tests/unit/test_profiler_protocol.py Outdated
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
Comment thread components/src/dynamo/profiler/tests/unit/test_profiler_protocol.py Outdated
Comment thread components/src/dynamo/profiler/tests/unit/test_profiler_protocol.py Outdated
Comment thread components/src/dynamo/profiler/utils/model_info.py
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
@tedzhouhk
tedzhouhk enabled auto-merge (squash) September 15, 2026 17:56
@glamr-agent

Copy link
Copy Markdown
Contributor
Automated evidence record — validation complete

factory: This is the machine-generated evidence record for the automated review of this
request. It is advisory: it is not an approval, and it does not replace human review.

Reviewed commit: 5e1dbbc80e3a616c5b149ccd84184815a76520ea — the head of this
request when the checks below ran. The change was reproduced on a local branch at
that head and examined there; nothing was pushed to this request's branch.

Validation status: complete

Evidence summary: [3/3 validated]

AI review assessment: sound. This assessment is produced by an automated reviewer
and is advisory only.

Validation result: complete — pass. Every check listed below ran to completion and
exited 0.

Evidence audit: complete [3/3 validated] — the command report below comes from
recorded runs.

Two things the checks below do not cover, stated so the record is not read as more
than it is: the Kubernetes deploy-and-infer retest with the corrected PVC alias needs a
cluster and was not run here, and this request's own full pipeline was neither triggered
nor waited on by this run.

Commands and results [3/3 validated]

Generated from the commands recorded during this run.

Check 1

Builds the changed Dynamo source and confirms that Python can import its compiled extension.

Result: Passed (exit 0)

Command:

Not shown because the exact command contained private run data.

Check 2

Checks the changed files with the repository's fast lint and formatting commands.

Result: Passed (exit 0)

Command:

Not shown because the exact command contained private run data.

Check 3

Runs the relevant Python unit tests without requiring a GPU.

Result: Passed (exit 0)

Command:

Not shown because the exact command contained private run data.

Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
Comment thread components/src/dynamo/profiler/tests/unit/test_resolve_model_path.py Outdated
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
@tedzhouhk
tedzhouhk merged commit 8309ffb into main Sep 16, 2026
113 of 114 checks passed
@tedzhouhk
tedzhouhk deleted the hzhou/dyn-4443 branch September 16, 2026 01:43
nv-nmailhot pushed a commit that referenced this pull request Sep 16, 2026
…cherry-pick of #14860) (#14943)

Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
aung-san-i added a commit to aung-san-i/dynamo that referenced this pull request Sep 28, 2026
* feat: KV DC Relay file based source mode (ai-dynamo#14807)

Add live-reloaded file sources for KV DC Relay namespace selection and expose readiness and source revisions through /engine/state.

Preserve applied membership on invalid updates, coalesce discovery refreshes, and isolate native integration tests in forked processes.

Signed-off-by: Nikita Sukharev <kaonael@gmail.com>

* feat(sglang): expose cross-encoder reranking through /v1/rerank (ai-dynamo#14032)

Signed-off-by: xianlubird <xianlubird@gmail.com>

* fix(profiler): explain inaccessible model paths during trust checks (ai-dynamo#14860)

Signed-off-by: hongkuanz <hongkuanz@nvidia.com>

* fix(sglang): sync discovery from native pause state (ai-dynamo#13951)

Signed-off-by: William Arnold <7565007+Aphoh@users.noreply.github.com>
Co-authored-by: Zero Rains <57100978+zeroRains@users.noreply.github.com>

* feat(recipes): add Solar Open2 250B NVFP4 aggregated and disaggregated recipes for B200 (ai-dynamo#14376)

Signed-off-by: Sandhya Rani Narravula <snarravula@nvidia.com>

* refactor(agents): session_id reader from AgentContext + forward to vLLM (ai-dynamo#14428)

Signed-off-by: Karen Chung <karenc@nvidia.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* fix(discovery): allow served aliases for the same model source (ai-dynamo#14857)

Signed-off-by: jthomson04 <jwillthomson19@gmail.com>

* fix(router): reject unknown explicit worker targets (ai-dynamo#14858)

Signed-off-by: jthomson04 <jwillthomson19@gmail.com>

* fix(xpu): stabilize XPU test workers (ai-dynamo#14539)

Signed-off-by: Wenxin Zhang <wenxin.zhang@intel.com>
Signed-off-by: VincyZhang <wenxin.zhang@intel.com>

* feat(mm-routing): add Nemotron 3 Nano Omni video routing (ai-dynamo#14653)

Signed-off-by: krishung5 <krish@nvidia.com>

* fix(sglang): validate diffusion input_reference and bound media fetches (ai-dynamo#14435)

The sglang image-diffusion and video-generation handlers passed the
client-supplied input_reference through to the generator's image_path after only
a non-empty check. Validate it first, and for remote references materialize it
locally before the generator sees it, so the generator is always handed a
trusted local path. This brings the sglang diffusion path in line with the
vLLM/omni and trtllm backends, which already validate the same field.

Behavior change: local I2I/I2V references now require DYN_MM_LOCAL_PATH to be
set to the allowed directory; previously any path was accepted.

common/http:

- validate_media_reference() returns a plain filesystem path for local
  references; local_media_reference() is an async context manager that fetches a
  remote one through fetch_bytes(policy=...), which revalidates every redirect
  hop, into a temp file removed on exit. data: is rejected -- a URI is not a path.
- fetch_bytes() gained max_bytes, streaming through collect_capped at an explicit
  read granularity so the cap is an allocation bound and not only a rejection: a
  128 MiB-decoded gzip body against the 64 MiB cap peaks at 68,032,217 bytes
  rather than the whole decompressed body. Content-Length is caller-controlled
  and absent when chunked, and aiohttp's read(n) returns at most n bytes, so
  neither a header check nor a single capped read suffices. Defaults to None,
  leaving existing callers unchanged.
- DYN_MM_MAX_FILE_SIZE_MB makes that cap operator-tunable, in megabytes, as the
  SGLang arg it replaces was. Read per call; empty, unparseable or non-positive
  falls back to 64 with a warning, so a malformed value neither takes the worker
  down nor reads as unlimited.
- Messages built from caller input are bounded via describe_media_source, moved
  from multimodal/media_source.py (it pulls in torch) into url_validator.py and
  re-exported from its old home; a no-op below 120 characters.
- HttpStatusError bounds its .message attribute, not only the rendered string:
  errors.rs::extract_http_like_error reads .status and .message off this class by
  name and forwards .message on a 4xx without calling str(). Backend exception
  text is bounded head-and-tail, since aiohttp renders the host before the errno.
- validate_local_path uses exc.strerror rather than the raw OSError, whose text
  repeats the filename, and now catches the ValueError that Path.resolve() raises
  on an embedded NUL so callers keep their 4xx-vs-5xx decision.

Rebased onto ai-dynamo#14563 (single aiohttp backend); the httpx-side half of the
max_bytes plumbing went with that backend.

Signed-off-by: nnshah1 <neelays@nvidia.com>
Signed-off-by: Dmitry Tokarev <dtokarev@nvidia.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(deps): upgrade fastokens to 0.3.2 (ai-dynamo#14798)

Signed-off-by: jthomson04 <jwillthomson19@gmail.com>

* fix(vllm): ship codec-free OpenCV for image inputs (ai-dynamo#14361)

Signed-off-by: svc-glamr@nvidia.com <svc-glamr@nvidia.com>
Signed-off-by: GLAMR <svc-glamr@nvidia.com>
Co-authored-by: Anant Sharma <anants@nvidia.com>
Co-authored-by: yunzhoul-nv <232973175+yunzhoul-nv@users.noreply.github.com>

* docs: refresh community events

Automated refresh from the public Dynamo Google Calendar.

Generated by .github/workflows/community-events-refresh.yml.

Signed-off-by: dynamo-ops <170655669+dynamo-ops@users.noreply.github.com>

* ci: refresh the compliance baseline in auto-upgrade pipeline (ai-dynamo#14206)

Signed-off-by: Anant Sharma <anants@nvidia.com>

* feat(triton): honor KServe classification on tensor outputs (ai-dynamo#14783)

Signed-off-by: Yingge He <yinggeh@nvidia.com>

* docs(rl): stop the verl guide sending readers to a vLLM version it cannot run on (ai-dynamo#14571)

Signed-off-by: svc-glamr@nvidia.com <svc-glamr@nvidia.com>

* feat(mocker): publish native KV events from the vLLM gRPC server (ai-dynamo#14737)

Signed-off-by: jthomson04 <jwillthomson19@gmail.com>

* fix(kv-router): release unowned radix branches after eviction (ai-dynamo#14878)

Signed-off-by: jthomson04 <jwillthomson19@gmail.com>

* fix: show correct backend versions in the install selectors (ai-dynamo#13599)

Signed-off-by: Anant Sharma <anants@nvidia.com>

* build(vllm): prepare v0.29.0 bump (ai-dynamo#14543)

Signed-off-by: Julien Darve <jdarve@NVIDIA.com>

* ci(xpu): validation PR for the re-applied XPU workflows and Dockerfile

Throwaway PR to prove the CI merged in #22 actually runs end to end on XPU
hardware. Adds only a comment to container/templates/vllm_runtime.Dockerfile,
which matches the `vllm` path filter (container/templates/vllm_*) and so makes
changed-files set vllm=true, which is what gates build-xpu and the
heterog-test-px-dn / heterog-test-pn-dx jobs.

What this exercises:
  - .github/workflows/pr-xpu.yaml            (push to pull-request/[0-9]+, needs the xpu label)
  - .github/workflows/pr-xpu-heterogeneous.yaml (push; its guard deliberately skips the label gate)
  - .github/workflows/epd-test-template.yml  (workflow_call, from the heterog jobs)
  - .github/scripts/test-filters.js          (the brace fix from #22)
  - container/templates/vllm_runtime.Dockerfile rendered and built for device=xpu

Not exercised: .github/workflows/xpu-heterogeneous-dispatch.yaml is
workflow_dispatch only and has to be run by hand from the Actions tab.

The marker comment must be removed before this branch is ever merged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(triton): Update Triton Base Image to 26.08 (ai-dynamo#14854)

Signed-off-by: J Wyman <jwyman@nvidia.com>
Co-authored-by: Rini Gupta <rinig@nvidia.com>

* fix(operator): normalize equivalent worker hash inputs (ai-dynamo#14721)

Signed-off-by: bzsuni <bingzhe.sun@daocloud.io>

* test(sglang): exercise NIXL in embedding cache E/PD test (ai-dynamo#14795)

Signed-off-by: Sai Kiran Polisetty <spolisetty@nvidia.com>

* fix(sglang): stop the elastic-EP scale-up worker crash-looping at startup (ai-dynamo#14568)

Signed-off-by: svc-glamr@nvidia.com <svc-glamr@nvidia.com>
Co-authored-by: yunzhoul-nv <232973175+yunzhoul-nv@users.noreply.github.com>

* fix(responses): honor tool_choice when parsing tool calls from text (ai-dynamo#14843)

Signed-off-by: xianlubird <xianlubird@gmail.com>

* ci: accept trusted full-CI request comments (ai-dynamo#14868)

Signed-off-by: Matej Kosec <mkosec@nvidia.com>

* docs: clarify EPP mode boundary and single-replica Dynamo mode fixes [DYN-4310] (ai-dynamo#14756)

Signed-off-by: Anna Tchernych <atchernych@nvidia.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

* ci(docs): move the generated-tables determinism gate out of link checking (ai-dynamo#14135)

Signed-off-by: Dan Gil <dagil@nvidia.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

* ci(docs): generate the Kubernetes API reference at publish time (ai-dynamo#14122)

Signed-off-by: Dan Gil <dagil@nvidia.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

* fix(operator): discover pull secrets for init containers (ai-dynamo#14922)

Signed-off-by: bojiang-li <327132355+bojiang-li@users.noreply.github.com>

* fix(sglang): stop an unusable mooncake backend crashing workers after model load (ai-dynamo#14461)

Signed-off-by: svc-glamr@nvidia.com <svc-glamr@nvidia.com>
Signed-off-by: glamr-agent <glamr-agent@users.noreply.github.com>
Signed-off-by: GLAMR <svc-glamr@nvidia.com>

* fix(sglang): emit prefill handoff before completion in sidecar (ai-dynamo#14260)

Signed-off-by: jain-ria <riajain@NVIDIA.com>
Co-authored-by: jain-ria <riajain@NVIDIA.com>
Co-authored-by: Connor Carpenter <connorc@nvidia.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>

* test(trtllm): enable fault tolerance coverage (ai-dynamo#14609)

Signed-off-by: tanmayv25 <tanmay2592@gmail.com>

* fix(frontend): evict async tokenizer executors when the tokenizer is retired (ai-dynamo#13368)

Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>

* fix(llm): report KServe datatypes by their wire names, not protobuf variants (ai-dynamo#14957)

`ModelMetadata` reported each Triton-registered tensor's `datatype` using
`inference::DataType::as_str_name()`, which returns the `model_config.proto`
variant name (`TYPE_FP32`, `TYPE_STRING`, ...) instead of the KServe v2 wire
names (`FP32`, `BYTES`, ...). Every datatype was wrong, so spec-conforming
clients cannot parse any tensor the RPC describes. Adds `oip_name()` next to
`tensor::DataType::to_kserve` covering all fifteen proto variants (incl. FP16
and BF16) and mapping `TYPE_STRING → BYTES`.

Original PR by @ayaangazali: ai-dynamo#14770. Reissued under a signed commit to
unblock the copy-pr-bot signature gate; diff is byte-identical.

Closes ai-dynamo#14520.

Signed-off-by: ayaangazali <ayaangazali@users.noreply.github.com>
Signed-off-by: ayaangazali <ayaangazali.work@gmail.com>
Signed-off-by: Vinya Kestur <vinyak@nvidia.com>
Co-authored-by: ayaangazali <ayaangazali.work@gmail.com>

* docs(mm-routing): document video KV routing (ai-dynamo#14958)

Signed-off-by: krishung5 <krish@nvidia.com>

* fix(sidecar): honor worker namespace suffix (ai-dynamo#14955)

Signed-off-by: Biswa Panda <biswa.panda@gmail.com>

* fix(bindings): drain bridge tasks before interpreter finalization (ai-dynamo#14813)

Signed-off-by: svc-glamr@nvidia.com <svc-glamr@nvidia.com>
Signed-off-by: GLAMR <svc-glamr@nvidia.com>
Co-authored-by: Tushar Sharma <tusharma@nvidia.com>

* fix(discovery): stop a Qwen3-VL worker from serving video with another worker's contract (ai-dynamo#14624)

Signed-off-by: svc-glamr@nvidia.com <svc-glamr@nvidia.com>
Signed-off-by: GLAMR <svc-glamr@nvidia.com>

* fix(gms): honor configured timeout during initial weights admission (ai-dynamo#14877)

Signed-off-by: svc-glamr@nvidia.com <svc-glamr@nvidia.com>
Signed-off-by: Schwinn Saereesitthipitak <schwinns@nvidia.com>
Signed-off-by: GLAMR <svc-glamr@nvidia.com>
Co-authored-by: Schwinn Saereesitthipitak <schwinns@nvidia.com>

* feat(kv-router): add construction-time indexer delegates (ai-dynamo#14945)

* fix(sglang): support min_tokens on tokenizer-free decode workers (ai-dynamo#14276)

Signed-off-by: svc-glamr@nvidia.com <svc-glamr@nvidia.com>
Signed-off-by: GLAMR <svc-glamr@nvidia.com>
Signed-off-by: jain-ria <riajain@NVIDIA.com>
Co-authored-by: jain-ria <riajain@NVIDIA.com>
Co-authored-by: MatejKosec <mkosec@nvidia.com>

* feat(router): add SessionPrefixIndexer for session-block lineage (ai-dynamo#13807)

Signed-off-by: svc-glamr@nvidia.com <svc-glamr@nvidia.com>
Signed-off-by: Karen Chung <karenc@nvidia.com>
Signed-off-by: Matej Kosec <mkosec@nvidia.com>
Co-authored-by: svc-glamr@nvidia.com <svc-glamr@nvidia.com>
Co-authored-by: Matej Kosec <mkosec@nvidia.com>

* fix(vllm): settle kvwarm stages through a per-step round on every attention-DP rank (ai-dynamo#14728)

Signed-off-by: Yiming Liu <yimingl@nvidia.com>

* feat(vllm): benchmark hybrid caches with random KDA state (ai-dynamo#14900)

Signed-off-by: hongkuanz <hongkuanz@nvidia.com>

* fix(runtime): fix QUIC reassembly and reduce response stalls (ai-dynamo#14876)

Signed-off-by: jthomson04 <jwillthomson19@gmail.com>

* feat(router): unify frontend and standalone selection core (ai-dynamo#14570)

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: Thomas Montfort <tjmontfort12@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Thomas Montfort <tjmontfort12@gmail.com>

* fix(planner): keep control APIs responsive during Prometheus collection (ai-dynamo#14377)

Signed-off-by: xianlubird <xianlubird@gmail.com>
Co-authored-by: Hongkuan Zhou <tedzhouhk@gmail.com>

* fix(router): record SGLang prefill completion after stream ends (ai-dynamo#14968)

Signed-off-by: jain-ria <riajain@NVIDIA.com>

* fix(frontend): send inline media once on the TCP request plane (ai-dynamo#14801)

Signed-off-by: Sumit Mishra <sah299610@gmail.com>
Co-authored-by: Indrajit Bhosale <iamindrajitb@gmail.com>

* docs: refresh community events

Automated refresh from the public Dynamo Google Calendar.

Generated by .github/workflows/community-events-refresh.yml.

Signed-off-by: dynamo-ops <170655669+dynamo-ops@users.noreply.github.com>

* fix(vllm): initialize synchronizer in KV warmup capacity test (ai-dynamo#14984)

Signed-off-by: Alec Flowers <aflowers@nvidia.com>

* fix(recipes): make the Solar Open2 250B benchmark and docs link usable (ai-dynamo#14956)

Signed-off-by: Sandhya Rani Narravula <snarravula@nvidia.com>

* feat(recipes): add K-EXAONE 2.0 750B-A37B NVFP4 vLLM recipes for B200 (ai-dynamo#14822)

Signed-off-by: Cheng Wang <chengwa@nvidia.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat: KVCR Resiliency Deployment Example (ai-dynamo#14695)

Add two-node DynamoGraphDeployment examples for process-local KVCR and
the KVCR memory service. Run one vLLM worker per GPU node, use stable
Grove ordinals for cache-owner slots, and request GPU-local RDMA
resources for engines and Guard services. Provide a deployment helper
for rendering and selecting either variant.

Run the KV state agent alongside vLLM for process-local host memory. In
memory-service mode, keep KVCR and the state agent in a separate
container so its Guard and shared-memory pool survive engine restarts.
Document that restarting the services sidecar invalidates the MVP
recovery contract and requires deployment-level replacement.

Add manifest coverage and an opt-in two-host lifecycle test. Kill the
source EngineCore, hold it offline, and verify that the promoted Guard
serves its preserved cache to the surviving target. Correlate response
equality and KVCR transfer metrics with transmit and receive counters
from the selected active HCA to prove RDMA transport.

Pin compatible KVCR and vLLM revisions and document the runtime,
discovery, compatibility-digest, and recovery prerequisites.

Signed-off-by: Adit Ranadive <aranadive@nvidia.com>

* feat(omni): add Nemotron Audex speech synthesis to /v1/audio/speech (ai-dynamo#12788)

Signed-off-by: Thanaji Rao Thakkalapelli <thanaji.rao.thakkalapelli@intel.com>

* ci: allow glamr-agent to request CI on its own unsigned PRs (ai-dynamo#14964)

Signed-off-by: Matej Kosec <mkosec@nvidia.com>

* fix(vllm): isolate multimodal worker ports (ai-dynamo#14751)

Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>

* fix(runtime): reject invalid DYN_REQUEST_PLANE values (ai-dynamo#12612)

Signed-off-by: svc-glamr@nvidia.com <svc-glamr@nvidia.com>
Signed-off-by: Matej Kosec <mkosec@nvidia.com>
Signed-off-by: Coding Agent <svc-glamr@nvidia.com>
Signed-off-by: GLAMR <svc-glamr@nvidia.com>
Co-authored-by: MatejKosec <mkosec@nvidia.com>

* fix(responses): preserve text instead of inferring tool calls (ai-dynamo#14846)

Signed-off-by: xianlubird <xianlubird@gmail.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>

* chore: temporarily increase frontend build time limit 45 --> 90 min (ai-dynamo#15019)

Signed-off-by: Dmitry Tokarev <dtokarev@nvidia.com>

* test(operator): cover scoped CA injection ownership (ai-dynamo#14961)

Signed-off-by: Julien Mancuso <jmancuso@nvidia.com>

* feat(frontend): map semantic errors to HTTP responses (ai-dynamo#14396)

Signed-off-by: Biswa Panda <biswa.panda@gmail.com>

* docs: correct fault-tolerance architecture details (ai-dynamo#14880)

Signed-off-by: Elizabeth Thomas <email2eliza@gmail.com>

* build(deps): bump nats-server to v2.14.7 (ai-dynamo#14919)

Signed-off-by: Dan Gil <dagil@nvidia.com>

* build(deps): bump AISimulate to 0.12.0 (ai-dynamo#15012)

Signed-off-by: Harrison King Saturley-Hall <hsaturleyhal@nvidia.com>

* remove oneAPI env for XPU detection

* feat(backends): expose native LoRA capacity in model registration (ai-dynamo#14754)

Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
Signed-off-by: bzsuni <bingzhe.sun@daocloud.io>
Co-authored-by: bzsuni <86399306+bzsuni@users.noreply.github.com>

* fix(planner): handle pending decisions in virtual connector wait (ai-dynamo#14841)

Signed-off-by: bzsuni <bingzhe.sun@daocloud.io>
Co-authored-by: Hongkuan Zhou <tedzhouhk@gmail.com>

* feat(vllm): add sidecar LoRA lifecycle (ai-dynamo#13068)

Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
Signed-off-by: bzsuni <bingzhe.sun@daocloud.io>
Co-authored-by: Julien Darve <jdarve@NVIDIA.com>
Co-authored-by: bzsuni <86399306+bzsuni@users.noreply.github.com>

* fix(vllm/omni): pass response_format into video EngineInputs (ai-dynamo#14667) (ai-dynamo#14844)

* chore: bump version to 1.6.0 post 1.5.0 branch cut (ai-dynamo#15009)

Signed-off-by: pvijayakrish <pvijayakrish@nvidia.com>
Signed-off-by: Pavithra Vijayakrishnan <160681768+pvijayakrish@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(ci): Use `pytest --ignore` to Skip Tests Based on Framework (ai-dynamo#14815)

Signed-off-by: J Wyman <jwyman@nvidia.com>

* feat(sidecar): add e2e CI testing for sidecar launch scripts (ai-dynamo#14508)

Signed-off-by: tanmayv25 <tanmay2592@gmail.com>
Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
Co-authored-by: Julien Darve <jdarve@NVIDIA.com>

* chore(xpu): upgrade vllm and omni to 0.29.0

Signed-off-by: wenxin.zhang <wenxin.zhang@intel.com>

* docs(operator): document the DGDR workload-creation trust boundary (ai-dynamo#14429)

Signed-off-by: nnshah1 <neelays@nvidia.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

* fix(xpu): use released vllm-omni prerelease

Signed-off-by: Wenxin Zhang <wenxin.zhang@intel.com>

* test(efa): add the EFA disaggregated deploy test for sglang (ai-dynamo#13893)

Signed-off-by: Jie Hao <jihao@nvidia.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(runtime): support IPv6-only IP resolution (ai-dynamo#13126)

Signed-off-by: jthomson04 <jwillthomson19@gmail.com>

* docs(fault-tolerance): clarify migration after shutdown grace expires (ai-dynamo#14872)

Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>

* feat(vllm-omni): preserve generated video audio (ai-dynamo#13707)

Signed-off-by: Guan Luo <gluo@nvidia.com>
Co-authored-by: Guan Luo <gluo@nvidia.com>

* feat(vllm-omni): pass model-specific video parameters (ai-dynamo#13708)

Signed-off-by: Guan Luo <gluo@nvidia.com>
Co-authored-by: Guan Luo <gluo@nvidia.com>

* feat(vllm-omni): qualify MiniMax-H3 T2VA on B200 (ai-dynamo#13589)

Signed-off-by: Guan Luo <gluo@nvidia.com>
Signed-off-by: GuanLuo <41310872+GuanLuo@users.noreply.github.com>
Co-authored-by: Guan Luo <gluo@nvidia.com>
Co-authored-by: GuanLuo <41310872+GuanLuo@users.noreply.github.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>

* fix(vllm): remove obsolete Omni compatibility guard

Signed-off-by: Wenxin Zhang <wenxin.zhang@intel.com>

* fix(vllm): retain Omni compatibility guard

Signed-off-by: Wenxin Zhang <wenxin.zhang@intel.com>

* .github/workflows/pr-xpu-heterogeneous.yaml; pin GPU_TAG to latest

* .github/workflows/; add post-merge and nightly XPU heterogeneous CI

Extract the XPU heterogeneous P/D pipeline out of pr-xpu-heterogeneous.yaml
into xpu-heterogeneous-run.yml, a workflow_call reusable workflow, and call it
from three thin trigger workflows so all three merge phases run the identical
pipeline instead of drifting copies.

  xpu-heterogeneous-run.yml           new, reusable. guard, changed-files,
                                      build-xpu, build-nvidia, resolve-images
                                      and both heterog tests, unchanged, plus
                                      7 inputs.
  pr-xpu-heterogeneous.yaml           reduced to the pre-merge trigger, the
                                      slash-command gate and the reaction.
  post-merge-xpu-heterogeneous.yaml   new. push to main.
  nightly-xpu-heterogeneous.yaml      new file, but the cron is MOVED, not
                                      added: it is the 0 23 * * * schedule
                                      that was already in
                                      pr-xpu-heterogeneous.yaml.

No behaviour change per phase. force_all_tests replaces the old
  github.event_name == 'schedule' || github.event_name == 'issue_comment'
expression with the same truth table: pre-merge passes
github.event_name == 'issue_comment', nightly passes true. Post-merge also
passes true, because a push to main has no PR base for
.github/actions/changed-files to diff against, and post-merge exists to catch
what per-PR gating missed.

xpu-status-check stays a TOP-LEVEL job in each caller rather than moving into
the reusable workflow. A job contributed by a reusable workflow reports to the
Checks API as "run / xpu-status-check", so hosting it there would rename the
context and leave any branch protection rule requiring xpu-status-check waiting
forever on a check that no longer reports.

The concurrency mapping stays byte-identical across all four workflows that
touch this hardware, now including xpu-heterogeneous-dispatch.yaml. Three files
do NOT get three slots: the cluster, the dynamo-system namespace and the
onexpu-/onenvidia-rdma-kueue ResourceClaimTemplates are one global resource.
The reusable workflow deliberately carries no concurrency block of its own,
which would deadlock against the slot the caller's run already holds.

Parameterised gpu_tag, model, tensor_parallel and runner as inputs so the
callers can diverge; all default to the previously hardcoded values. Added
workflow_dispatch to the nightly, without which a schedule-only workflow cannot
be exercised before it reaches the default branch.

Verified: all files parse; the four concurrency mappings are byte-identical; the
reusable workflow declares no concurrency; every input each caller passes exists
and every required input is supplied; nesting is depth 3 of the 4 GitHub allows.
actionlint was not available to run, and will report queue:max as an unknown key
in all four files, a known false positive.

---------

Signed-off-by: Nikita Sukharev <kaonael@gmail.com>
Signed-off-by: xianlubird <xianlubird@gmail.com>
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
Signed-off-by: William Arnold <7565007+Aphoh@users.noreply.github.com>
Signed-off-by: Sandhya Rani Narravula <snarravula@nvidia.com>
Signed-off-by: Karen Chung <karenc@nvidia.com>
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
Signed-off-by: Wenxin Zhang <wenxin.zhang@intel.com>
Signed-off-by: VincyZhang <wenxin.zhang@intel.com>
Signed-off-by: krishung5 <krish@nvidia.com>
Signed-off-by: nnshah1 <neelays@nvidia.com>
Signed-off-by: Dmitry Tokarev <dtokarev@nvidia.com>
Signed-off-by: svc-glamr@nvidia.com <svc-glamr@nvidia.com>
Signed-off-by: GLAMR <svc-glamr@nvidia.com>
Signed-off-by: dynamo-ops <170655669+dynamo-ops@users.noreply.github.com>
Signed-off-by: Anant Sharma <anants@nvidia.com>
Signed-off-by: Yingge He <yinggeh@nvidia.com>
Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
Signed-off-by: J Wyman <jwyman@nvidia.com>
Signed-off-by: bzsuni <bingzhe.sun@daocloud.io>
Signed-off-by: Sai Kiran Polisetty <spolisetty@nvidia.com>
Signed-off-by: Matej Kosec <mkosec@nvidia.com>
Signed-off-by: Anna Tchernych <atchernych@nvidia.com>
Signed-off-by: Dan Gil <dagil@nvidia.com>
Signed-off-by: bojiang-li <327132355+bojiang-li@users.noreply.github.com>
Signed-off-by: glamr-agent <glamr-agent@users.noreply.github.com>
Signed-off-by: jain-ria <riajain@NVIDIA.com>
Signed-off-by: tanmayv25 <tanmay2592@gmail.com>
Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
Signed-off-by: ayaangazali <ayaangazali@users.noreply.github.com>
Signed-off-by: ayaangazali <ayaangazali.work@gmail.com>
Signed-off-by: Vinya Kestur <vinyak@nvidia.com>
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
Signed-off-by: Schwinn Saereesitthipitak <schwinns@nvidia.com>
Signed-off-by: Yiming Liu <yimingl@nvidia.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: Thomas Montfort <tjmontfort12@gmail.com>
Signed-off-by: Sumit Mishra <sah299610@gmail.com>
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
Signed-off-by: Cheng Wang <chengwa@nvidia.com>
Signed-off-by: Adit Ranadive <aranadive@nvidia.com>
Signed-off-by: Thanaji Rao Thakkalapelli <thanaji.rao.thakkalapelli@intel.com>
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Signed-off-by: Coding Agent <svc-glamr@nvidia.com>
Signed-off-by: Julien Mancuso <jmancuso@nvidia.com>
Signed-off-by: Elizabeth Thomas <email2eliza@gmail.com>
Signed-off-by: Harrison King Saturley-Hall <hsaturleyhal@nvidia.com>
Signed-off-by: pvijayakrish <pvijayakrish@nvidia.com>
Signed-off-by: Pavithra Vijayakrishnan <160681768+pvijayakrish@users.noreply.github.com>
Signed-off-by: wenxin.zhang <wenxin.zhang@intel.com>
Signed-off-by: Jie Hao <jihao@nvidia.com>
Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
Signed-off-by: Guan Luo <gluo@nvidia.com>
Signed-off-by: GuanLuo <41310872+GuanLuo@users.noreply.github.com>
Co-authored-by: Nikita Sukharev <kaonael@gmail.com>
Co-authored-by: Xianlu Bird <xianlubird@gmail.com>
Co-authored-by: Hongkuan Zhou <tedzhouhk@gmail.com>
Co-authored-by: William Arnold <7565007+Aphoh@users.noreply.github.com>
Co-authored-by: Zero Rains <57100978+zeroRains@users.noreply.github.com>
Co-authored-by: snarravula-dl <snarravula@nvidia.com>
Co-authored-by: Karen Chung <karenc@nvidia.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: jthomson04 <jwillthomson19@gmail.com>
Co-authored-by: VincyZhang <wenxin.zhang@intel.com>
Co-authored-by: Kris Hung <krish@nvidia.com>
Co-authored-by: Neelay Shah <neelays@nvidia.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: GLAMR <svc-glamr@nvidia.com>
Co-authored-by: Anant Sharma <anants@nvidia.com>
Co-authored-by: yunzhoul-nv <232973175+yunzhoul-nv@users.noreply.github.com>
Co-authored-by: dynamo-ops <170655669+dynamo-ops@users.noreply.github.com>
Co-authored-by: Yingge He <157551214+yinggeh@users.noreply.github.com>
Co-authored-by: JulienDarve <86800349+JulienDarve@users.noreply.github.com>
Co-authored-by: J Wyman <jwyman@nvidia.com>
Co-authored-by: Rini Gupta <rinig@nvidia.com>
Co-authored-by: bzsuni <86399306+bzsuni@users.noreply.github.com>
Co-authored-by: Sai Kiran Polisetty <spolisetty@nvidia.com>
Co-authored-by: MatejKosec <mkosec@nvidia.com>
Co-authored-by: atchernych <atchernych@nvidia.com>
Co-authored-by: Dan Gil <dagil@nvidia.com>
Co-authored-by: Bojiang Li <327132355+bojiang-li@users.noreply.github.com>
Co-authored-by: Connor Carpenter <connorcarpenter15@gmail.com>
Co-authored-by: jain-ria <riajain@NVIDIA.com>
Co-authored-by: Connor Carpenter <connorc@nvidia.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
Co-authored-by: Tanmay Verma <tanmayv@nvidia.com>
Co-authored-by: Peter Pan <peter.pan@daocloud.io>
Co-authored-by: Vinya Kestur Tumakuru Arun Kumar <vinyak@nvidia.com>
Co-authored-by: ayaangazali <ayaangazali.work@gmail.com>
Co-authored-by: Biswa Panda <biswa.panda@gmail.com>
Co-authored-by: Tushar Sharma <tusharma@nvidia.com>
Co-authored-by: Schwinn Saereesitthipitak <schwinns@nvidia.com>
Co-authored-by: Ryan Olson <ryanolson@users.noreply.github.com>
Co-authored-by: Yimingl_Nvidia <yimingl@nvidia.com>
Co-authored-by: Thomas Montfort <tjmontfort12@gmail.com>
Co-authored-by: Sumit884-byte <sah299610@gmail.com>
Co-authored-by: Indrajit Bhosale <iamindrajitb@gmail.com>
Co-authored-by: Alec <35311602+alec-flowers@users.noreply.github.com>
Co-authored-by: chw001 <chengwa@nvidia.com>
Co-authored-by: Adit Ranadive <aranadive@nvidia.com>
Co-authored-by: Thanaji Rao Thakkalapelli <thanaji.rao.thakkalapelli@intel.com>
Co-authored-by: Keiven C <213854356+keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
Co-authored-by: Dmitry Tokarev <dtokarev@nvidia.com>
Co-authored-by: Julien Mancuso <161955438+julienmancuso@users.noreply.github.com>
Co-authored-by: Elizabeth Thomas <email2eliza@gmail.com>
Co-authored-by: Harrison Saturley-Hall <hsaturleyhal@nvidia.com>
Co-authored-by: Julien Darve <jdarve@NVIDIA.com>
Co-authored-by: Jasim Kareem <mj9034812@gmail.com>
Co-authored-by: Pavithra Vijayakrishnan <160681768+pvijayakrish@users.noreply.github.com>
Co-authored-by: Jie Hao <jihao@nvidia.com>
Co-authored-by: Jacky <18255193+kthui@users.noreply.github.com>
Co-authored-by: Qi Wang <qiwa@nvidia.com>
Co-authored-by: Guan Luo <gluo@nvidia.com>
Co-authored-by: GuanLuo <41310872+GuanLuo@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants