Skip to content

[MM] MultimodalInputs.merge: adopt the M-RoPE delta and clear its decode cache - #41739

Open
rodamani wants to merge 4 commits into
sgl-project:mainfrom
modal-projects:rohan/up/mm-mrope-delta-merge
Open

rodamani wants to merge 4 commits into
sgl-project:mainfrom
modal-projects:rohan/up/mm-mrope-delta-merge

Conversation

@rodamani

@rodamani rodamani commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Motivation

MultimodalInputs.merge (session continuation) drops the new turn's M-RoPE position delta when the earlier turn had none, and never clears mrope_position_delta_repeated_cache. Decode positions after a multimodal session continuation then come from a stale or missing delta.

Modifications

  • merge() adopts the other input's delta when self has none, and always clears mrope_position_delta_repeated_cache.
  • test/registered/unit/managers/test_mm_inputs_merge_mrope_delta.py.

Accuracy Tests

CPU: 1 passed. With the schedule_batch.py change reverted: 1 failed.

Speed Tests and Profiling

No hot-path change beyond the fix itself; not separately benchmarked.

Checklist

Review and Merge Process

  1. Ping Merge Oncalls to start the process. See the PR Merge Process.
  2. Get approvals from CODEOWNERS and other reviewers.
  3. Trigger CI tests with comments or contact authorized users to do so.
    • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
  4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

CI States

Latest PR Test (Base): ✅ Run #36767650274
Latest PR Test (Extra): ❌ Run #36767649664
Latest PR Test (AMD ROCm 10): ❌ Run #36767650253

…and adopts the other's delta (sgl-project#87)

When an earlier session turn had no M-RoPE delta, merge() dropped the new
turn's delta, and it never cleared mrope_position_delta_repeated_cache, so
decode positions after a multimodal session continuation came from a stale
or missing delta.

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@github-actions github-actions Bot added the npu label Sep 29, 2026
@rodamani
rodamani marked this pull request as ready for review September 29, 2026 21:13
@rodamani

Copy link
Copy Markdown
Contributor Author

/rerun-test -c

@github-actions

github-actions Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

Results for /rerun-test -c:

⛔ test/registered/unit/managers/test_mm_inputs_merge_mrope_delta.py: File not found: test/registered/unit/managers/test_mm_inputs_merge_mrope_delta.py

@rodamani

Copy link
Copy Markdown
Contributor Author

/tag-and-rerun-ci

@github-actions github-actions Bot added the run-ci CI: run the baseline test suite on this PR label Sep 30, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

npu run-ci CI: run the baseline test suite on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants