Skip to content

Align VideoLlavaProcessor with Transformers v4.51.3#2101

Closed
yafshar wants to merge 4 commits into
huggingface:mainfrom
yafshar:sync/videollava-with-transformers
Closed

Align VideoLlavaProcessor with Transformers v4.51.3#2101
yafshar wants to merge 4 commits into
huggingface:mainfrom
yafshar:sync/videollava-with-transformers

Conversation

@yafshar
Copy link
Copy Markdown
Contributor

@yafshar yafshar commented Jul 3, 2025

The GaudiVideoLlavaProcessor has been removed from optimum-habana as its functionality is now fully aligned with the v4.51.3 Transformers implementation. No custom logic is required, and maintaining a separate class is redundant.

Update GaudiVideoLlavaForConditionalGeneration implementation with the v4.51.3 transformers changes while retaining token_idx argument for compatibility with Gaudi optimizations.

>>> python -m pytest tests/test_image_to_text_example.py -k llava
================= 7 passed, 7 deselected in 704.09s (0:11:44) ==================

>>> python -m pytest tests/test_video_llava.py
======================== 1 passed in 109.60s (0:01:49) =========================

The masked_scatter is not a bottleneck on G3 anymore.

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests?

The GaudiVideoLlavaProcessor has been removed from optimum-habana
as its functionality is now fully aligned with the upstream
Transformers implementation. No custom logic is required, and
maintaining a separate class is redundant.
@yafshar yafshar requested a review from regisss as a code owner July 3, 2025 11:37
@yafshar yafshar marked this pull request as draft July 3, 2025 11:45
This update aligns the GaudiVideoLlavaForConditionalGeneration
implementation with the v4.51.3 transformers changes while retaining
`token_idx` argument for compatibility with Gaudi optimizations.
@yafshar yafshar closed this Jul 3, 2025
@yafshar yafshar reopened this Jul 9, 2025
@yafshar
Copy link
Copy Markdown
Contributor Author

yafshar commented Jul 9, 2025

@regisss can you please label this PR for synapse 1.22

@astachowiczhabana astachowiczhabana marked this pull request as ready for review July 10, 2025 12:18
@HuggingFaceDocBuilderDev
Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@astachowiczhabana astachowiczhabana changed the base branch from main to v1.19-release July 10, 2025 12:25
@astachowiczhabana astachowiczhabana changed the base branch from v1.19-release to main July 10, 2025 12:28
@yafshar
Copy link
Copy Markdown
Contributor Author

yafshar commented Jul 10, 2025

@astachowiczhabana I will close this PR and open #2129 with the new base of v1.19-release

@yafshar yafshar closed this Jul 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants