Skip to content

[qwen] Add VideoChatFlashQwenModel text-decoder builder - #371

Merged
xadupre merged 2 commits into
mainfrom
copilot/import-changes-fast-unit-test
May 25, 2026
Merged

[qwen] Add VideoChatFlashQwenModel text-decoder builder#371
xadupre merged 2 commits into
mainfrom
copilot/import-changes-fast-unit-test

Conversation

Copilot AI commented May 25, 2026

Copy link
Copy Markdown
Contributor

Imports the Python text-decoder builder additions from onnxruntime-genai PR #2147 (file hash 11d14636…cfcb50src/python/py/models/builder.py) so mbext can export the text component of VideoChatFlashQwenForCausalLM (OpenGVLab/VideoChat-Flash, Qwen2.5-7B backbone).

Changes

  • modelbuilder/builders/qwen.py — new VideoChatFlashQwenModel(QwenModel):

    • Sets self.model_type = "VideoChat_Flash_QwenForCausalLM" so the base class derives the genai_config type videochat_flash_qwen.
    • Overrides load_weights to call Qwen2ForCausalLM.from_pretrained directly, bypassing the upstream VCF AutoConfig which pulls in av/cv2/decord/imageio.
  • modelbuilder/builder.py — dispatch case for VideoChatFlashQwenForCausalLM that forces extra_options["exclude_embeds"] = True (decoder consumes inputs_embeds from the embedding-merger that fuses InternVideo2 visual tokens).

  • tests/fast/test_random_videochat_flash_qwen.py — fast offline test (fp32/fp16/int4 CPU + fp16 CUDA) following the test_random_gemma3_conditional.py pattern:

    • Builds a tiny random Qwen2 model, rewrites architectures in the saved config.json to VideoChatFlashQwenForCausalLM (since Qwen2ForCausalLM.save_pretrained would otherwise overwrite it) to exercise the new dispatch.
    • Asserts the exported ONNX exposes inputs_embeds (not input_ids) and runs the prefill/decode discrepancy check with embed_fn=model.model.embed_tokens.

The VLM vision encoder and embedding-merger remain out of scope, matching upstream Phase 1.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/repos/microsoft/onnxruntime-genai/pulls/2147
    • Triggering command: /usr/bin/curl curl -sL -H Accept: application/vnd.github.v3.diff REDACTED -o /tmp/pr2147.diff (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Agent-Logs-Url: https://github.com/xadupre/mbext/sessions/60fa1191-8bd3-435b-8e09-b2dbb6074563

Co-authored-by: xadupre <22452781+xadupre@users.noreply.github.com>
Copilot AI changed the title [WIP] Import changes and add fast unit test Add VideoChatFlashQwenModel text-decoder builder May 25, 2026
Copilot AI requested a review from xadupre May 25, 2026 11:46
@github-actions

Copy link
Copy Markdown

📂 Impacted Files

3 file(s) changed in this pull request:

🔵 modelbuilder/builder.py (+6 / -0)
🔵 modelbuilder/builders/qwen.py (+32 / -0)
🟢 tests/fast/test_random_videochat_flash_qwen.py (+150 / -0)

@github-actions github-actions Bot changed the title Add VideoChatFlashQwenModel text-decoder builder [qwen] Add VideoChatFlashQwenModel text-decoder builder May 25, 2026
@codecov

codecov Bot commented May 25, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 78.86%. Comparing base (d8cd68a) to head (e7e226a).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #371      +/-   ##
==========================================
+ Coverage   78.82%   78.86%   +0.03%     
==========================================
  Files          27       27              
  Lines        8058     8071      +13     
==========================================
+ Hits         6352     6365      +13     
  Misses       1706     1706              
Flag Coverage Δ
fast-tests 78.86% <100.00%> (+0.03%) ⬆️
fast-tests-ort-nightly 78.81% <100.00%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@xadupre
xadupre marked this pull request as ready for review May 25, 2026 11:59
@xadupre
xadupre merged commit 1858525 into main May 25, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants