[None][feat] Cosmos3 action generation by ishovkun · Pull Request #17325 · NVIDIA/TensorRT-LLM

ishovkun · 2026-08-05T23:11:23Z

Dev Engineer Review

Added Cosmos3 action generation for policy, forward-dynamics, and inverse-dynamics modes.
Added action configuration, domain presets, embodiment-specific dimensions, action projections, mRoPE alignment, scheduling, latent preparation, and tensor outputs.
Added validation for modes, domains, trajectories, resolutions, media formats, and incompatible audio or output settings.
Added NVDEC video probing and fit-and-pad resizing.
Updated serving routes to force tensor-compatible formats for action responses.
Updated APIs, README documentation, prompts, and output serialization.
Added Literal[...] validation and removed the inactive range constraint from action_resolution.
Fixed scheduler restoration, empty trajectory rejection, inverse-dynamics error text, filename documentation, aspect-ratio labels, and action latent preparation.
The inverse-dynamics accuracy result is within the reference limit: MSE 0.0091 versus maximum 0.05.
Review should confirm API consistency with CODING_GUIDELINES.md, no regressions in video or audio modes, and valid NVDEC behavior on supported devices.

QA Engineer Review

Test changes

Added or updated:

test_cosmos3_action.py: resolution, domain, prompt, aspect-ratio, mRoPE, latent preparation, masking, and validation tests.
test_cosmos3_pipeline.py: policy, forward-dynamics, inverse-dynamics, validation, incompatibility, and scheduler tests.
test_cosmos3_transformer.py: action-module, structure, dimension, forward-pass, mask, cache, and domain tests.
test_cosmos3_transformer_parallel.py: single-GPU and Ulysses action-output parity.
test_cosmos3_distilled.py: action mode resolution and conditioning callbacks.
test_tensor_payload.py: action batching, serialization, and slicing.
test_trtllm_serve_endpoints.py: tensor-only format resolution.
test_visual_gen_params.py: literal validation and video-reference validation.
test_output.py: action output defaults, propagation, and tensor-only saving.
tests/integration/test_lists/test-db/l0_b200.yml: added the Cosmos3 action visual-generation test.

The Cosmos3 action test has CI coverage through l0_b200.yml. Coverage for the remaining new test suites is not identified. CBTS coverage data is unavailable.

Verdict: needs follow-up. Run the requested NVIDIA-triggered CI job and confirm coverage for the remaining tests.

Supersedes #15890 by
@NVShreyas, whose commits are preserved in this branch's history.

Description

Adds action generation to the Cosmos3 OmniMoT pipeline: alongside video and
audio, the model predicts a robot control trajectory — a [T, D] array of
pose / grasp / fingertip numbers, not pixels. Three modes share one denoise
pass and differ only in which stream starts clean:

mode	video	action	answers
`policy`	first frame given	predicted	"here's the scene — what should I do?"
`forward_dynamics`	predicted	given	"if I do this, what happens?"
`inverse_dynamics`	whole clip given	predicted	"here's what happened — what did it do?"

Action tokens join the generator sequence after the video tokens, are projected
by per-embodiment weights (DomainAwareLinear, selected by domain_id, since
ten numbers from a WidowX are not ten numbers from a Franka), and denoise on
their own scheduler instance against the same flow-matching velocity as video.

Carried over from #15890 with the following changes, each verified against
cosmos-framework and the diffusers Cosmos3 port:

mRoPE alignment. Action positions were 4x too large:
compute_mrope_position_ids_action accepted base_temporal_compression_factor
and dropped it, so both rates used the action factor (1) instead of the vision
factor (4) for the base rate. Action tokens advanced a whole latent frame per
step instead of a quarter, desynchronising them from the video they condition.
Canonical action widths. raw_action_dim is fixed per embodiment but
lived in the sampling presets, which several embodiments share by alias:
robomind-franka-dual resolved to 10 instead of 20, galbot to 29 instead of
30. Widths moved to action.EMBODIMENT_TO_RAW_ACTION_DIM, keyed by the
unaliased domain name.
Trained caption. Action checkpoints are trained on a structured JSON
caption (cinematography/actions/duration/fps/resolution/
aspect_ratio), not the flat duration/resolution templates. Added with the
four trained viewpoint sentences and a view_point extra param.
NVDEC reference decoding. The action path decoded its reference with
torchvision.io.read_video, removed in torchvision 0.24+, so
inverse_dynamics did not run at all on a current install. It now takes the
same encoded-bytes contract as V2V and decodes worker-side on NVDEC.
decode_video_reference_window gains a resize selector: V2V keeps
cover-scale + center-crop, action gets contain-scale + pad, because a gripper
works at the frame edge and cropping it away removes what the policy acts on.
Serve output. An action request returned 200 with an mp4 and discarded the
trajectory. format now resolves against what the request produces: auto
selects a tensor payload, an explicit mp4/avi is rejected with 400. The
rule is declared by the pipeline (ExtraParamSchema.requires_tensor_output),
so the serving layer needs no per-model knowledge.
Test fixes. The five new TestCosmos3Action forward tests and the
multi-GPU helper omitted raw_timestep, which forward() rejects — the L0
failure in pipelines 48022 and 48728.

Review follow-ups folded in (all with tests): action requests no longer inherit
the video sampling recipe through infer() (35 steps / guidance 6 / 24 fps
instead of the action recipe and the embodiment's frame rate); the reference
image loader accepts the https URLs the bundled prompt files actually use; both
per-rank reads of the reference converge so a rank-local failure cannot hang a
multi-GPU job; _apply_flow_shift rebuilds the action scheduler alongside video
and audio; a domain_id that contradicts domain_name is rejected instead of
silently picking one; and DomainAwareLinear's range check no longer forces two
device-to-host syncs on every denoise step.

Also reconciles the feature with main after V2V landed: the video extra
param adopts main's bytes contract (retiring this branch's path_or_list and
the serve-side path rejection it needed), flow_shift routes through the
sampling policy, and post_step_fn keeps the two-argument signature review
asked for, with main's one-argument callers updated.

Test Coverage

Unit (CPU, no checkpoint) — tests/unittest/_torch/visual_gen/test_cosmos3_action.py:
mRoPE positions against a transcription of the reference formula, including the
invariant that the last action token lands exactly on the last vision latent
frame for every paired config; canonical widths for all 15 embodiments; the
trained JSON caption incl. aspect-label round-trip over all 20 bucket entries;
reference sizing via header probe.

Integration — test_cosmos3_transformer.py (action forward, domain-id range,
noisy mask, multiframe), test_cosmos3_pipeline.py::TestCosmos3Action (all
three modes end to end, incl. inverse_dynamics on an encoded fixture),
test_trtllm_serve_endpoints.py::TestTensorOnlyFormat* (format resolution and
rejection), multi_gpu/test_cosmos3_transformer_parallel.py -k action
(Ulysses-2 parity vs single GPU).

Accuracy — inverse_dynamics on cosmos-framework's own
action_inverse_dynamics_robot case recovers the episode's recorded WidowX
trajectory at mse 0.0091 against the reference's golden_mse_max = 0.05
(corr +0.9865), including the discrete grasp transition at the correct step.

Local runs on B300, at this commit: 283 passed across the Cosmos3
pipeline / action / transformer / distilled suites against a real Cosmos3-Nano
checkpoint, 348 across output / tensor payload / serve endpoints / params, and
the Ulysses-2 action parity case. An earlier full sweep of
tests/unittest/{visual_gen,_torch/visual_gen} was 1444 passed; the failures
remaining on that host need assets it lacks (Wan VAE checkpoints,
trtllm-serve on PATH) and are untouched by this branch.

PR Checklist

Open questions for reviewers

VisualGenOutput.action. The output schema gains exactly one field,
the trajectory tensor itself. An earlier revision also returned
raw_action_dim, action_mode and domain_id (zhenhuaw-me flagged these on
[None][feat] Cosmos3 Action Support #15890); they are gone. Nothing read them back, and each was recoverable
without the worker sending it — raw_action_dim is action.shape[-1],
action_mode is the caller's own string, and domain_id is a weight-table
index that is strictly lossier than the domain_name the caller already
holds (id 8 is droid_lerobot or robomind-franka). The remaining
question is only whether a trajectory tensor belongs on the shared output
class at all, or whether that eventually wants a generic extra_outputs
channel — a larger API change, better as its own PR.
Short reference clips. cosmos-framework pads by repeating the last frame;
this branch raises. Deliberate: freezing a frame yields plausible-looking but
meaningless actions with no signal to the caller.
flow_shift for action. Left at the checkpoint default. The framework
sets 10.0 for all three action modes, but the value is inert on Nano/Edge —
use_karras_sigmas takes precedence in UniPCMultistepScheduler, verified.

Co-authored-by: Shreyas Misra shreyasm@nvidia.com

🤖 Generated with Claude Code

Signed-off-by: Shreyas Misra <shreyasm@nvidia.com>

This reverts commit c115ea6. Signed-off-by: Shreyas Misra <shreyasm@nvidia.com>

Signed-off-by: Shreyas Misra <shreyasm@nvidia.com>

…tion Adopted-PR review findings, each verified against cosmos-framework and the diffusers Cosmos3 port. mRoPE: action token positions were 4x too large. compute_mrope_position_ids_action accepted base_temporal_compression_factor and dropped it, so the scaled positions used the action tcf (1) for both rates instead of the vision tcf (4) for the base rate. Action tokens therefore advanced one latent frame per step instead of a quarter, desynchronising them from the video they condition. Threaded the factor through compute_mrope_position_ids_vision (defaulting to temporal_compression_factor, so vision and audio are bit-identical). Action widths: raw_action_dim is fixed per embodiment, but it lived in the sampling presets, which several embodiments share via COSMOS3_DOMAIN_PRESET_ALIASES. robomind-franka-dual resolved to 10 instead of 20 and galbot to 29 instead of 30. Moved the canonical widths to action.EMBODIMENT_TO_RAW_ACTION_DIM, keyed by the unaliased domain name; presets now carry sampling settings only. libero stays absent, as in both references, because its width is dataset-dependent. Caption: action checkpoints are trained on a structured JSON caption (cinematography/actions/duration/fps/resolution/aspect_ratio) rather than the flat duration/resolution templates. Added build_action_json_prompt with the four trained viewpoint sentences and a view_point extra param. Aspect ratio snaps to the canonical bucket label; reducing H/W gave "15,26" where the trained label is "16,9". Tests: the five new TestCosmos3Action forward tests and the multi-GPU _forward_with_action helper omitted raw_timestep, which forward() rejects - this is the L0 failure in pipelines 48022 and 48728. Also tightened the domain-id range test, whose match= was satisfied by the raw_timestep error. Signed-off-by: Igor Shovkun <ishovkun@nvidia.com>

Adopts the V2V work that landed upstream. Conflict resolutions worth naming: post_step_fn: main added a one-argument hook for V2V while this branch had generalised it to (latents, extra_streams) -> (latents, extra_streams) for the action stream. Kept the two-argument form - a single fixed signature was the review request - and updated the two one-argument callers, wan's _pin_i2v_first_frame and the cosmos3 distilled conditioning anchor. video: dropped this branch's path_or_list parameter for main's bytes contract, so action's inverse_dynamics reference travels as encoded bytes and is decoded worker-side on NVDEC like V2V's. That also retires the path_or_list entry in the params type map and the serve-side rejection of extra_params['video'], both of which existed only to work around passing server-local paths. flow_shift: _set_flow_shift is gone upstream in favour of _apply_flow_shift via the sampling policy. Action uses the checkpoint default; its 5.0 was inert anyway, since Cosmos3-Nano and Edge set use_karras_sigmas, which takes precedence over flow_shift in UniPCMultistepScheduler.set_timesteps. num_frames: no longer carried in COSMOS3_ACTION_PARAMS. Both references fix it at action_chunk_size + 1 - diffusers rejects a caller-supplied num_frames for action runs outright - so it is derived rather than resolved from the request. Two defects introduced by the merge itself, fixed here: a stale UniPCMultistepScheduler reference left by main's scheduler refactor, and infer() resolving height/width from the video table before forward() runs, which pinned action requests to 720p because resolve_action_size honours explicit values and so never consulted the resolution bucket. Known gap: action's mp4 path still calls torchvision.io.read_video, removed in torchvision 0.24+, so inverse_dynamics does not yet run. The swap to decode_video_reference_window needs a fit+pad resize mode; policy and forward_dynamics take an image and are unaffected. Signed-off-by: Igor Shovkun <ishovkun@nvidia.com>

Action's inverse_dynamics decoded its reference with torchvision.io.read_video, an API removed in torchvision 0.24+, so the mode did not run at all on a current install. It now takes the same encoded-bytes contract as V2V and decodes worker-side on NVDEC. The two modes need opposite framing, so decode_video_reference_window gains a resize selector. V2V continues to cover-scale and center-crop (default, unchanged): losing a border strip costs a scene continuation nothing, and no invented pixels enter the frame. Action gets "fit": contain-scale and pad, because a gripper and its target sit at the frame edge, and cropping them away removes exactly what the policy is supposed to act on. resize_fit_pad_uint8 mirrors the action reference's reflection_pad_to_target - contain-scale by min(target/source, 1.0) so a small clip keeps its own pixels rather than being enlarged, round-not-ceil resize, pad bottom/right by reflection, switching to edge replication once a pad run reaches the resized extent and reflection has no pixels left to mirror. It reuses this module's Lanczos-3 taps rather than the reference's bicubic: the geometry is what preserves content, and a second filter would buy sub-pixel differences at the cost of a second code path. The action decode is wrapped in the same try/except plus synchronize_media_prepare_status convergence as V2V and I2V, so a per-rank NVDEC failure surfaces on every rank instead of hanging the collectives. Signed-off-by: Igor Shovkun <ishovkun@nvidia.com>

Follows the NVDEC swap: action.py carried a private mp4/frame-directory reader (decode_action_video_file, normalize_action_video_path, normalize_action_video_input) and the pipeline a _preprocess_action_video built on it. All of that is superseded by decode_video_reference_window. action_reference_image returned a decoded PIL frame that callers used only for its dimensions - the canvas is the resolution bucket closest to the source's aspect - so it becomes action_reference_size, and video bytes report their size from the container header via probe_video_dimensions rather than by decoding. resolve_action_size takes those dimensions instead of an image. policy and forward_dynamics accept either source: an image goes through PIL, video bytes take frame 0 off NVDEC, and both land on the padded canvas, so the two entry points produce the same conditioning for the same picture. num_frames is now derived as action_chunk_size + 1 wherever it is needed and is gone from the presets and from resolve_domain_action_config's signature. It had been resolvable from a preset, so overriding action_chunk_size left a frame count that disagreed with it - the inverse_dynamics smoke test hit exactly that, asking for a 9-frame chunk and being handed the preset's 17. Tests: the mp4-decoding cases are replaced by TestActionReferenceSize, which covers image measurement, image-over-video precedence, and the header probe for bytes. The inverse_dynamics pipeline tests now feed the checked-in 9-frame V2V fixture as bytes; 178 pass across the three cosmos3 files. Signed-off-by: Igor Shovkun <ishovkun@nvidia.com>

Five failures surfaced by running the whole visual-gen suite after the merge; all five are this branch's, and each is a place where the action work and main had independently changed the same behaviour. post_step_fn takes (latents, extra_streams) and returns both, so the distilled conditioning anchor's hook does too. Two tests in test_cosmos3_distilled called it with one argument. T2I + audio: main force-disables audio for image requests so they never reach the audio-weight presence check, and asserts that. The action PR had added a raise a few lines earlier, which pre-empted it. Main owns T2I, so the raise goes; action's own T2I rejection stays. The serve-side rejection of extra_params['video'] tested a workaround that no longer exists. It guarded against a client passing a server-local path, which main now prevents with a stronger mechanism: `video` is declared bytes, so a path fails preflight type validation - verified directly, a str is rejected with "expected type 'bytes'". The obsolete endpoint test is dropped and test_visual_gen_params' path_or_list case becomes the bytes case, keeping the property under test rather than the removed implementation of it. Suite status: 1444 passed. The remaining failures need assets this host lacks - seven Wan VAE checkpoint comparisons and twelve serve e2e errors from `trtllm-serve` not being on PATH - and are untouched by this branch. Signed-off-by: Igor Shovkun <ishovkun@nvidia.com>

…drop An action request over /v1/videos returned 200 with an mp4 and threw the trajectory away. The action tensor reaches the coordinator intact - to_handle walks every field, so it rides back like video does - but the encoder branch of the route only ever reads output.video, so the thing the request was made for was discarded at the last step. format now resolves against what the request will actually produce. 'auto' selects a tensor payload, which carries every populated modality plus its scalar metadata; an explicit safetensors/pt passes through; an explicit mp4/avi is rejected with 400, because the caller has stated two incompatible things and neither guess is right - encoding it drops the trajectory, ignoring the format disregards what they asked for. The message names the parameter that forced the choice and the formats that work. The rule is declared, not hard-coded: ExtraParamSchema gains requires_tensor_output, Cosmos3 sets it on action_mode, and the route reads the declaration without knowing what an action is. Specs already travel to the coordinator in the READY handshake, so this needs no new plumbing, and any pipeline with a non-encodable modality gets the same behaviour by declaring it. The async route resolves the format before queueing the job and passes it to the background task, so a rejected request never becomes one. Closes the "silently dropping action" review thread on visual_gen/output.py. Signed-off-by: Igor Shovkun <ishovkun@nvidia.com>

… output --video_path for inverse_dynamics took a frame directory before the NVDEC swap; it is an MP4/AVI clip now, decoded worker-side like V2V. Also states why action runs are tensor payloads and what trtllm-serve does with format. Signed-off-by: Igor Shovkun <ishovkun@nvidia.com>

… sync Eight items from review, each with a test: - Wan 2.2 5B I2V pinned its first frame through a one-argument post-step callback while the shared loop now passes two. Restore the two-argument signature. - Action requests inherited the video sampling recipe through infer(): unset steps, guidance and frame rate were materialized as 35 / 6.0 / 24 fps, so the action branch's 30 / 1.0 and the embodiment's frame rate (bridge 5, av 10) could never win, and CFG doubled the transformer work. Pass them through unset for action, as height and width already were. frame_rate stays materialized in the pipeline defaults because the serve layer derives num_frames from seconds x frame_rate. - The example assigned the reference clip to both params.image and the video extra param, which the pipeline rejects, so the documented inverse_dynamics invocation could not run. - pil_to_rgb opened every string as a local path, while both bundled action prompts carry https frame URLs. Route through the repo's URL-aware loader, and resolve the reference once per request instead of once per read. - Both per-rank reads of the action reference (the canvas probe and the policy / forward_dynamics decode) now converge like inverse_dynamics and V2V already did; a rank-local failure otherwise leaves healthy ranks in the transformer's collectives. - _apply_flow_shift rebuilds the action scheduler alongside video and audio. Unreachable on Nano and Edge, whose UniPC config takes the karras branch before flow_shift is read, but the distilled checkpoints do not share that. - A domain_id that contradicts domain_name is rejected instead of silently applying one robot's timing to another robot's weights. - DomainAwareLinear's range check used a device predicate as a Python condition, forcing two blocking device-to-host syncs on every denoise step. It moves to a once-per-request check, plus a free host-side check on the scalar before the reference is decoded. Signed-off-by: Igor Shovkun <ishovkun@nvidia.com>

… prompt Two findings from a second review pass. infer() nulled frame_rate for every action request, not only the one the executor had materialized, so a caller could not override the embodiment preset at all. The explicit/default distinction is not recoverable here -- both VisualGen.default_params and parse_visual_gen_params construct VisualGenParams(**defaults), so pydantic marks every field as set before the pipeline sees the request -- but the value is: drop frame_rate only while it still equals what default_generation_params supplied. An explicit value that happens to equal the video default remains indistinguishable. is_v2v was true whenever video bytes were present, which is how an action reference arrives too. inverse_dynamics therefore always forced the system prompt, and a video-backed policy request tokenized differently from the same frame passed as an image. cosmos-framework attaches a system prompt for image editing and transfer only, never for action, so the checkpoint default is the right answer. Re-scored inverse_dynamics against the framework golden under the corrected prompt: mse 0.009148, corr +0.9865, against golden_mse_max 0.05. Signed-off-by: Igor Shovkun <ishovkun@nvidia.com>

_encode_video_tensor was defined twice in Cosmos3OmniMoTPipeline, byte for byte. The action-section copy shadowed the one the I2V and V2V paths read, so a future fix to the first would have silently done nothing. It is not action code either -- it VAE-encodes any preprocessed pixel video. Keep the original. _prepare_action_latents defaulted action_dim to 64 if the transformer lacked it, but the transformer applies that same default when it builds the action heads and this path only runs when action_gen is true, so the fallback covered a state that cannot occur. defaults.py's module docstring described COSMOS3_DOMAIN_PRESETS, the preset merge and raw_action_dim's absence from the presets. The first and third were already stated at their definitions; the second describes resolve_domain_action_config, whose own docstring was one line. Moved there. Signed-off-by: Igor Shovkun <ishovkun@nvidia.com>

…d for it VisualGenOutput and PipelineOutput carried raw_action_dim, action_mode and domain_id alongside the action tensor. Nothing in the codebase read them back: they flowed pipeline -> output -> safetensors header and stopped. Each is recoverable without the worker sending it: - raw_action_dim is action.shape[-1]. One line produces both, since the tensor is sliced to that width on the way out. - action_mode is the caller's own string, normalized. - domain_id is an index into DomainAwareLinear's weight table, resolved from the domain_name the caller sent. It is also the lossier of the pair -- id 8 is droid_lerobot or robomind-franka, id 15 is any of three agibot variants -- so a consumer holding it cannot recover what it asked for, while the request's domain_name says exactly. Dropping them leaves the shared output schema one field wider than main, for the tensor itself, and takes tensor_payload back to media tensors plus rates with no model-specific keys (a test now pins that). The offline example writes its own sidecar from argparse instead, which also lets it record domain_name -- something the output never carried. Also comment why Wan's post_step_fn takes side-stream latents it does not use: the shared denoise loop passes them for Cosmos3, and the Wan diff otherwise shows an unexplained parameter. Signed-off-by: Igor Shovkun <ishovkun@nvidia.com>

The action modes are listed in the mode bullets; the heading does not need to enumerate them. Signed-off-by: Igor Shovkun <ishovkun@nvidia.com>

tensorrt-cicd · 2026-08-23T14:41:23Z

PR_Github #68585 [ run ] completed with state SUCCESS. Commit: 1f90daa
/LLM/main/L0_MergeRequest_PR pipeline #55999 completed with status: 'SUCCESS'

CI Report

Link to invocation

Two conflicts, both in the serving layer against the deliberate response_format break on main (NVIDIA#17490: video API drops b64_json for "file"/"path" plus Server-Timing headers and a "postprocessing" job status): - openai_video_routes.py: the tensor-payload sync branch adopts the new conventions (timing headers, path envelope, FileResponse with headers); the dead b64 helper goes with the rest of the b64 paths. The background task keeps resolving through request_format (the route's tensor-only resolution) rather than raw request.format, and gains upstream's postprocessing status transition. - test_trtllm_serve_endpoints.py: keeps the thread-settling TestClient (pytest-threadleak guard) wired into upstream's reworked _create_server. Signed-off-by: Igor Shovkun <igshov@gmail.com>

ishovkun · 2026-08-26T21:33:49Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-08-26T21:42:33Z

PR_Github #69536 [ run ] triggered by Bot. Commit: fdda076 Link to invocation

tensorrt-cicd · 2026-08-27T01:56:51Z

PR_Github #69536 [ run ] completed with state FAILURE. Commit: fdda076
/LLM/main/L0_MergeRequest_PR pipeline #56861 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

ishovkun · 2026-08-27T02:04:54Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-08-27T02:10:19Z

PR_Github #69592 [ run ] triggered by Bot. Commit: fdda076 Link to invocation

…CI storage) The Wan2.2-T2V-A14B-Diffusers-NVFP4 transformer shard 00002-of-00002 on CI's shared model storage fails safetensors header deserialization ("header too large"), so the example dies loading weights before any pipeline code runs. Deterministic across every PR whose selection includes visual-gen examples (L0_Test-x86_64-Single-GPU 7181, 7182, 7186); unrelated to this PR. Waived until the checkpoint is re-synced. Signed-off-by: Igor Shovkun <igshov@gmail.com>

ishovkun · 2026-08-27T05:24:54Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-08-27T05:31:02Z

PR_Github #69642 [ run ] triggered by Bot. Commit: 4b2b6d8 Link to invocation

tensorrt-cicd · 2026-08-27T05:32:55Z

PR_Github #69592 [ run ] completed with state ABORTED. Commit: fdda076

Link to invocation

tensorrt-cicd · 2026-08-27T12:04:57Z

PR_Github #69642 [ run ] completed with state FAILURE. Commit: 4b2b6d8
/LLM/main/L0_MergeRequest_PR pipeline #56949 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

ishovkun · 2026-08-27T12:14:58Z

/bot run --disable-fail-fast

Main waived test_wan_t2v_example under nvbugs/6676844 for the same corrupt-checkpoint failure this branch had waived with a free-text reason; main's NVBug-linked line wins and waives.txt returns to main's content. Signed-off-by: Igor Shovkun <igshov@gmail.com>

ishovkun · 2026-08-27T12:16:47Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-08-27T12:22:13Z

PR_Github #69716 [ run ] triggered by Bot. Commit: 1fcf462 Link to invocation

tensorrt-cicd · 2026-08-27T19:27:24Z

PR_Github #69716 [ run ] completed with state FAILURE. Commit: 1fcf462
/LLM/main/L0_MergeRequest_PR pipeline #57015 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

ishovkun · 2026-08-27T23:18:35Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-08-27T23:24:07Z

PR_Github #69821 [ run ] triggered by Bot. Commit: 1fcf462 Link to invocation

tensorrt-cicd · 2026-08-28T02:54:49Z

PR_Github #69821 [ run ] completed with state SUCCESS. Commit: 1fcf462
/LLM/main/L0_MergeRequest_PR pipeline #57115 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

ishovkun · 2026-08-28T03:08:01Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-08-28T03:14:22Z

PR_Github #69866 [ run ] triggered by Bot. Commit: 1fcf462 Link to invocation

tensorrt-cicd · 2026-08-28T04:41:25Z

PR_Github #69866 [ run ] completed with state SUCCESS. Commit: 1fcf462
/LLM/main/L0_MergeRequest_PR pipeline #57156 completed with status: 'SUCCESS'

CI Report

Link to invocation

NVShreyas and others added 21 commits July 17, 2026 13:39

cosmos3 action init

d2a42c3

Signed-off-by: Shreyas Misra <shreyasm@nvidia.com>

updates

c369c86

Signed-off-by: Shreyas Misra <shreyasm@nvidia.com>

add action fps optional input

8816958

Signed-off-by: Shreyas Misra <shreyasm@nvidia.com>

add action to media API, example script fixes

d7b3b4b

Signed-off-by: Shreyas Misra <shreyasm@nvidia.com>

fix visualgen output test

e39caa8

Signed-off-by: Shreyas Misra <shreyasm@nvidia.com>

unwaive test

944e5a8

Signed-off-by: Shreyas Misra <shreyasm@nvidia.com>

fix test

8b8aa44

Signed-off-by: Shreyas Misra <shreyasm@nvidia.com>

Revert "unwaive test"

f3b25da

This reverts commit c115ea6. Signed-off-by: Shreyas Misra <shreyasm@nvidia.com>

address Cosmos3 action review comments

6bf9ba9

Signed-off-by: Shreyas Misra <shreyasm@nvidia.com>

[None][doc] Cosmos3 README: keep the original title

a98e031

The action modes are listed in the mode bullets; the heading does not need to enumerate them. Signed-off-by: Igor Shovkun <ishovkun@nvidia.com>

ishovkun requested review from a team as code owners August 5, 2026 23:11

ishovkun requested review from YihuiLu512, lori-ren, luyiyun1021 and zhenhuaw-me August 5, 2026 23:11

BowenFu approved these changes Aug 24, 2026

View reviewed changes

mikeiovine approved these changes Aug 25, 2026

View reviewed changes

yuanjingx87 approved these changes Aug 26, 2026

View reviewed changes

BowenFu approved these changes Aug 27, 2026

View reviewed changes

karljang mentioned this pull request Aug 27, 2026

[None][fix] fix Cosmos3 multi-control attention semantics #18277

Open

tburt-nv approved these changes Aug 27, 2026

View reviewed changes

chang-l merged commit 2938ada into NVIDIA:main Aug 28, 2026
7 checks passed

Uh oh!

Conversation

ishovkun commented Aug 5, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Dev Engineer Review

QA Engineer Review

Test changes

Description

Test Coverage

PR Checklist

Open questions for reviewers

Uh oh!

tensorrt-cicd commented Aug 23, 2026

Uh oh!

ishovkun commented Aug 26, 2026

Uh oh!

tensorrt-cicd commented Aug 26, 2026

Uh oh!

tensorrt-cicd commented Aug 27, 2026

Uh oh!

ishovkun commented Aug 27, 2026

Uh oh!

tensorrt-cicd commented Aug 27, 2026

Uh oh!

ishovkun commented Aug 27, 2026

Uh oh!

tensorrt-cicd commented Aug 27, 2026

Uh oh!

tensorrt-cicd commented Aug 27, 2026

Uh oh!

tensorrt-cicd commented Aug 27, 2026

Uh oh!

ishovkun commented Aug 27, 2026

Uh oh!

ishovkun commented Aug 27, 2026

Uh oh!

tensorrt-cicd commented Aug 27, 2026

Uh oh!

tensorrt-cicd commented Aug 27, 2026

Uh oh!

ishovkun commented Aug 27, 2026

Uh oh!

tensorrt-cicd commented Aug 27, 2026

Uh oh!

tensorrt-cicd commented Aug 28, 2026

Uh oh!

ishovkun commented Aug 28, 2026

Uh oh!

tensorrt-cicd commented Aug 28, 2026

Uh oh!

tensorrt-cicd commented Aug 28, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

ishovkun commented Aug 5, 2026 •

edited by coderabbitai Bot

Loading