Skip to content

Oct16 updated try1#27098

Closed
tomeras91 wants to merge 18 commits intovllm-project:mainfrom
tomeras91:oct16-updated-try1
Closed

Oct16 updated try1#27098
tomeras91 wants to merge 18 commits intovllm-project:mainfrom
tomeras91:oct16-updated-try1

Conversation

@tomeras91
Copy link
Copy Markdown
Member

@tomeras91 tomeras91 commented Oct 17, 2025 •
edited by github-actions bot
Loading

Purpose

Test Plan

Test Result

Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

BloodAxe and others added 13 commits October 8, 2025 11:32
@BloodAxe
Allow passing "mm_processor_kwargs": dict(max_num_tiles=2),
a3bd4e2 
Signed-off-by: Eugene Khvedchenia <ekhvedchenia@nvidia.com>
@BloodAxe
Ensure video modality always uses 1 tile (performance optimization)
1656894 
Signed-off-by: Eugene Khvedchenia <ekhvedchenia@nvidia.com>
@BloodAxe
Cherry pick video loading support
e8562b8 
Signed-off-by: Eugene Khvedchenia <ekhvedchenia@nvidia.com>
@BloodAxe
Cherry-pick image normalization
5809b49 
Signed-off-by: Eugene Khvedchenia <ekhvedchenia@nvidia.com>
@BloodAxe
Add video timestamps support
44f364b 
Fix image normalization precision bug
Fix possible inconsistent tokenization of frame prefixes
Change logic of video sampling in OpenCVDynamicBackend
Signed-off-by: Eugene Khvedchenia <ekhvedchenia@nvidia.com>
@nvnbagrov
bugfix: respect fps in case it implies less frames
73ac36b
@nvnbagrov
Merge branch 'feature/reproduce-nano-results-video-input-with-norm-fi…
915fcee 
…x-with-timestamps' into bugfix/fps
@nvnbagrov
ruff
4646a17
@nvnbagrov
Merge pull request #3 from tomeras91/bugfix/fps
9bc40f4 
Bugfix/fps
@tomeras91
Merge branch 'main' into oct16-updated-try1
56522fd 
Signed-off-by: Tomer Asida <57313761+tomeras91@users.noreply.github.com>
@tomeras91
fix: a more robust way to ensure only a single video item is processe…
8da7256 
…d at a time

Instead of relying on input shapes and that the first dim is the batch
dim, just process a single item. Changes in
group_mm_kwargs_by_modality() which now has merge_by_field_config=True
for almost all relevant models mean that the batch dimension doesn't
exist and frames from different videos are gathered in the same dim.
That's why we need a more robust way to deal with processing just a
single video

Signed-off-by: Tomer Asida <57313761+tomeras91@users.noreply.github.com>
@BloodAxe
Fix wrong dtype (int32) of dummy video input to correct one (np.uint8)
9e45ae4 
Signed-off-by: Eugene Khvedchenia <ekhvedchenia@nvidia.com>
@BloodAxe
Remove hard-coded 16 frames used for memory profiling
ae377da 
Signed-off-by: Eugene Khvedchenia <ekhvedchenia@nvidia.com>
@mergify mergify bot added multi-modality Related to multi-modality (#4194) v1 labels Oct 17, 2025
BloodAxe and others added 5 commits October 17, 2025 15:14
@BloodAxe
Revert file
f6ffcb4 
Signed-off-by: Eugene Khvedchenia <ekhvedchenia@nvidia.com>
@nvnbagrov @BloodAxe
created a new class for video sampling, revert original glm behavior (#4
31dc3e0 
)

* created a new class for video sampling, revert original glm behavior

* ruff

* ruff
@BloodAxe
Rollback OpenCVVideoBackend logic
4e70952 
Signed-off-by: Eugene Khvedchenia <ekhvedchenia@nvidia.com>
@BloodAxe
Cosmetics
e66ba3b 
Signed-off-by: Eugene Khvedchenia <ekhvedchenia@nvidia.com>
@BloodAxe
Merge branch 'main' into oct16-updated-try1
23d6d9e
@BloodAxe BloodAxe deleted the oct16-updated-try1 branch October 22, 2025 13:37
@tomeras91
Copy link
Copy Markdown
Member Author

tomeras91 commented Oct 23, 2025 •
edited
Loading

closed in favor of #27107

@tomeras91 tomeras91 closed this Oct 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@njhill njhill Awaiting requested review from njhill njhill will be requested when the pull request is marked ready for review njhill is a code owner

@DarkLight1337 DarkLight1337 Awaiting requested review from DarkLight1337 DarkLight1337 will be requested when the pull request is marked ready for review DarkLight1337 is a code owner

@ywang96 ywang96 Awaiting requested review from ywang96 ywang96 will be requested when the pull request is marked ready for review ywang96 is a code owner

@NickLucche NickLucche Awaiting requested review from NickLucche NickLucche will be requested when the pull request is marked ready for review NickLucche is a code owner

@tjtanaa tjtanaa Awaiting requested review from tjtanaa tjtanaa will be requested when the pull request is marked ready for review tjtanaa is a code owner

Assignees

No one assigned

Labels

multi-modality Related to multi-modality (#4194) v1

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@tomeras91 @BloodAxe @nvnbagrov